text_shortening_model_v10

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.7343
• Rouge1: 0.5944
• Rouge2: 0.3803
• Rougel: 0.5562
• Rougelsum: 0.5556
• Bert precision: 0.8982
• Bert recall: 0.9028
• Average word count: 11.1571
• Max word count: 16
• Min word count: 7
• Average token count: 16.4
• % shortened texts with length > 12: 22.1429

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
2.1519	1.0	16	1.6818	0.5512	0.3185	0.4947	0.4946	0.8804	0.8891	11.8643	18	5	17.0071	45.7143
1.824	2.0	32	1.5909	0.5691	0.3406	0.5167	0.5163	0.8858	0.8957	11.8714	18	3	16.8429	47.1429
1.6784	3.0	48	1.5260	0.582	0.3562	0.5308	0.5295	0.8871	0.8997	12.0214	18	4	17.0929	47.1429
1.5584	4.0	64	1.4877	0.5749	0.3431	0.5258	0.5249	0.8878	0.8996	11.8	17	6	16.9786	42.1429
1.4831	5.0	80	1.4597	0.5849	0.3527	0.5337	0.5329	0.8871	0.901	12.0071	17	6	17.2857	44.2857
1.4536	6.0	96	1.4384	0.587	0.3562	0.5375	0.536	0.8889	0.9015	11.8929	16	6	17.2071	41.4286
1.3631	7.0	112	1.4238	0.5928	0.3616	0.5438	0.5434	0.89	0.9024	11.7643	16	7	17.1	40.0
1.285	8.0	128	1.4128	0.5876	0.3566	0.5364	0.5355	0.8898	0.9008	11.55	16	6	16.9286	36.4286
1.2515	9.0	144	1.4009	0.5953	0.3631	0.5444	0.5436	0.8913	0.9015	11.6286	16	6	16.9857	36.4286
1.2159	10.0	160	1.3975	0.5898	0.3583	0.54	0.5398	0.8915	0.9017	11.5714	16	6	16.9643	33.5714
1.1865	11.0	176	1.3955	0.5977	0.3641	0.5465	0.5457	0.8933	0.9034	11.4857	16	6	16.7643	32.8571
1.1476	12.0	192	1.3925	0.5903	0.3584	0.5436	0.5429	0.8906	0.9026	11.6571	17	7	17.0	34.2857
1.1196	13.0	208	1.3878	0.5972	0.3667	0.5482	0.5471	0.8935	0.9036	11.4857	16	6	16.7643	33.5714
1.0767	14.0	224	1.3949	0.5978	0.3687	0.5481	0.5474	0.8946	0.9041	11.45	16	6	16.7643	32.1429
1.0343	15.0	240	1.3930	0.6013	0.3776	0.5547	0.5544	0.8961	0.9046	11.4357	16	6	16.6714	30.0
1.0302	16.0	256	1.3959	0.597	0.3702	0.5453	0.5446	0.8921	0.9057	11.75	17	6	17.2071	34.2857
1.0251	17.0	272	1.3876	0.5999	0.3752	0.5524	0.5519	0.893	0.9061	11.7429	16	6	17.1643	34.2857
0.9773	18.0	288	1.3929	0.6002	0.3724	0.5463	0.5462	0.8946	0.9041	11.4929	16	6	16.7929	30.7143
0.9437	19.0	304	1.3921	0.6038	0.3824	0.5555	0.5553	0.897	0.9053	11.2857	16	6	16.5571	27.8571
0.9267	20.0	320	1.4016	0.6046	0.3774	0.5542	0.554	0.8923	0.9048	11.8429	16	7	17.1929	35.7143
0.9178	21.0	336	1.4047	0.6054	0.3775	0.5553	0.5545	0.8957	0.9056	11.5429	16	4	16.8643	29.2857
0.8941	22.0	352	1.4112	0.6039	0.3775	0.5556	0.555	0.8937	0.9052	11.6143	17	4	17.0	29.2857
0.8715	23.0	368	1.4156	0.602	0.3811	0.5552	0.5548	0.8947	0.9057	11.4929	16	6	16.8714	27.8571
0.847	24.0	384	1.4283	0.6013	0.3771	0.5516	0.5509	0.8935	0.905	11.5071	16	6	16.9571	30.0
0.8319	25.0	400	1.4349	0.5991	0.3747	0.5525	0.5519	0.8951	0.9049	11.3	16	6	16.7857	27.1429
0.8081	26.0	416	1.4421	0.6014	0.3797	0.5529	0.5528	0.8946	0.9056	11.5	16	6	16.9071	30.0
0.8098	27.0	432	1.4406	0.6038	0.3811	0.5546	0.5543	0.8965	0.9053	11.3429	16	6	16.6786	28.5714
0.7738	28.0	448	1.4544	0.5993	0.3774	0.5544	0.5535	0.8951	0.9042	11.3	17	5	16.7429	25.7143
0.7651	29.0	464	1.4711	0.6024	0.383	0.5573	0.5568	0.8979	0.9053	11.15	17	6	16.4357	22.1429
0.7495	30.0	480	1.4666	0.6066	0.3842	0.5609	0.5597	0.8979	0.9065	11.1643	16	5	16.6429	21.4286
0.7216	31.0	496	1.4779	0.6009	0.3801	0.5555	0.5549	0.8968	0.9048	11.2643	16	6	16.7143	25.7143
0.7074	32.0	512	1.4918	0.6	0.3792	0.5547	0.5537	0.8959	0.9041	11.3	16	7	16.7643	26.4286
0.7241	33.0	528	1.4914	0.6029	0.3859	0.5606	0.5596	0.8984	0.9044	11.1071	16	7	16.4857	20.7143
0.7001	34.0	544	1.4851	0.6035	0.3831	0.5591	0.5584	0.8971	0.9044	11.25	16	7	16.65	26.4286
0.6863	35.0	560	1.4942	0.6001	0.3818	0.556	0.5551	0.8961	0.902	11.2429	16	7	16.6214	25.7143
0.6724	36.0	576	1.4976	0.5958	0.3721	0.5496	0.5493	0.8959	0.9018	11.2143	16	7	16.5429	25.0
0.6652	37.0	592	1.5006	0.5976	0.3714	0.5522	0.5513	0.8964	0.9018	11.1071	16	7	16.4714	22.8571
0.6326	38.0	608	1.5064	0.6013	0.3802	0.5573	0.5563	0.8975	0.9017	11.0643	16	7	16.3071	22.1429
0.6408	39.0	624	1.5209	0.5969	0.3797	0.5557	0.555	0.8947	0.9025	11.35	16	7	16.6	27.8571
0.6209	40.0	640	1.5194	0.599	0.3773	0.5547	0.5538	0.8961	0.9036	11.35	16	7	16.6429	27.1429
0.6094	41.0	656	1.5285	0.6076	0.3917	0.5677	0.5667	0.9002	0.9064	11.1071	16	7	16.35	21.4286
0.6007	42.0	672	1.5403	0.607	0.3844	0.5657	0.5646	0.8991	0.9052	11.2214	15	6	16.3714	22.1429
0.5916	43.0	688	1.5546	0.5991	0.3768	0.5608	0.5602	0.8964	0.9044	11.3857	15	7	16.7357	25.7143
0.5816	44.0	704	1.5533	0.5955	0.3687	0.556	0.5556	0.8959	0.9017	11.1857	15	7	16.5571	22.1429
0.5714	45.0	720	1.5604	0.6025	0.3785	0.5592	0.5589	0.8978	0.9037	11.2357	16	7	16.4786	22.8571
0.563	46.0	736	1.5673	0.6034	0.3795	0.5604	0.5598	0.8969	0.9027	11.3	16	7	16.5571	25.0
0.546	47.0	752	1.5723	0.6005	0.381	0.5595	0.5592	0.8979	0.9035	11.2714	17	7	16.4429	25.0
0.5386	48.0	768	1.5735	0.5942	0.3768	0.5541	0.554	0.898	0.9028	11.1929	16	7	16.3214	25.0
0.5489	49.0	784	1.5781	0.5923	0.372	0.5527	0.5531	0.8966	0.9017	11.1857	16	7	16.3643	24.2857
0.5267	50.0	800	1.5837	0.5928	0.3729	0.5519	0.5513	0.8966	0.9019	11.2786	17	7	16.3929	25.0
0.5274	51.0	816	1.5907	0.5974	0.3751	0.5586	0.558	0.8988	0.9029	11.1857	17	6	16.25	26.4286
0.5206	52.0	832	1.5964	0.5913	0.3673	0.5515	0.5515	0.8966	0.9014	11.2429	16	7	16.3857	25.0
0.4979	53.0	848	1.6073	0.59	0.3719	0.5546	0.555	0.8965	0.9012	11.1357	17	7	16.3143	22.8571
0.5007	54.0	864	1.6126	0.5923	0.3733	0.5561	0.5559	0.8961	0.9012	11.2643	17	7	16.4286	24.2857
0.5035	55.0	880	1.6188	0.5972	0.3749	0.5567	0.5567	0.8985	0.9024	11.0786	16	7	16.2143	20.7143
0.504	56.0	896	1.6320	0.5996	0.3776	0.5593	0.5597	0.8985	0.9038	11.1357	16	7	16.3143	21.4286
0.4908	57.0	912	1.6333	0.5941	0.3757	0.5552	0.5554	0.897	0.9034	11.1929	16	7	16.3857	23.5714
0.4748	58.0	928	1.6339	0.5968	0.3704	0.5541	0.554	0.8977	0.9025	11.1571	16	7	16.3786	23.5714
0.4751	59.0	944	1.6352	0.6006	0.3791	0.5601	0.5599	0.8988	0.9032	11.1214	17	7	16.3071	20.7143
0.474	60.0	960	1.6349	0.6006	0.3865	0.5618	0.5609	0.8987	0.9045	11.1786	17	7	16.3786	22.8571
0.4673	61.0	976	1.6357	0.5924	0.3756	0.5561	0.5559	0.8982	0.9024	11.0786	16	6	16.2	20.7143
0.4708	62.0	992	1.6488	0.5964	0.3833	0.5607	0.5602	0.8985	0.9043	11.2214	16	7	16.4143	25.0
0.4721	63.0	1008	1.6555	0.5949	0.3781	0.5569	0.5563	0.8966	0.9036	11.2643	16	7	16.5071	25.0
0.46	64.0	1024	1.6644	0.5955	0.3749	0.5536	0.5536	0.8982	0.903	11.1714	16	7	16.3286	20.7143
0.4469	65.0	1040	1.6648	0.5962	0.3791	0.558	0.5581	0.8968	0.9041	11.3714	16	7	16.5429	27.1429
0.4395	66.0	1056	1.6675	0.5912	0.3767	0.5526	0.5523	0.8976	0.903	11.2357	16	7	16.4071	22.8571
0.4363	67.0	1072	1.6699	0.5913	0.3747	0.5538	0.5533	0.8987	0.902	11.0	16	7	16.0786	19.2857
0.4313	68.0	1088	1.6751	0.5854	0.3666	0.5464	0.5452	0.897	0.9	10.9929	16	7	16.1071	20.0
0.4237	69.0	1104	1.6777	0.5956	0.3787	0.554	0.5535	0.8975	0.9024	11.2	16	7	16.3571	22.8571
0.4265	70.0	1120	1.6787	0.5907	0.3713	0.552	0.5516	0.8964	0.9011	11.1429	15	7	16.3286	20.7143
0.4219	71.0	1136	1.6846	0.5868	0.3699	0.5487	0.5481	0.895	0.9004	11.2357	15	7	16.4214	22.1429
0.4237	72.0	1152	1.6891	0.5895	0.3702	0.5506	0.5503	0.895	0.9013	11.2643	15	7	16.5071	24.2857
0.4146	73.0	1168	1.6914	0.5963	0.3772	0.5552	0.5548	0.8967	0.9019	11.2429	15	7	16.4357	22.1429
0.4103	74.0	1184	1.6948	0.5878	0.372	0.5496	0.549	0.8962	0.9006	11.2	16	7	16.3571	21.4286
0.4099	75.0	1200	1.6970	0.5932	0.3755	0.5566	0.5563	0.897	0.9021	11.2357	16	7	16.3286	22.8571
0.403	76.0	1216	1.6966	0.5922	0.3768	0.5545	0.5542	0.8987	0.9026	11.0857	16	7	16.2214	20.7143
0.3999	77.0	1232	1.6991	0.5946	0.3778	0.5549	0.5552	0.8986	0.9026	11.1	16	6	16.2286	21.4286
0.4176	78.0	1248	1.7002	0.5963	0.3783	0.5568	0.5571	0.8984	0.9032	11.1643	16	6	16.3286	23.5714
0.4007	79.0	1264	1.7038	0.5921	0.3729	0.553	0.5529	0.8976	0.9015	11.1071	16	6	16.2286	22.1429
0.3918	80.0	1280	1.7114	0.595	0.3745	0.5551	0.5544	0.8982	0.9021	11.1714	16	6	16.3071	22.1429
0.3936	81.0	1296	1.7153	0.5914	0.3724	0.5527	0.5524	0.8979	0.9014	11.0929	16	6	16.2286	21.4286
0.3997	82.0	1312	1.7154	0.5924	0.3755	0.5528	0.5527	0.8972	0.9021	11.2286	16	6	16.3571	23.5714
0.396	83.0	1328	1.7187	0.5943	0.3765	0.5552	0.5549	0.897	0.9029	11.3	16	6	16.4571	24.2857
0.4049	84.0	1344	1.7198	0.5958	0.3757	0.5555	0.5548	0.8972	0.9031	11.2929	16	6	16.4929	25.0
0.3983	85.0	1360	1.7201	0.5948	0.3776	0.558	0.5574	0.8974	0.9025	11.25	16	6	16.4286	24.2857
0.3936	86.0	1376	1.7211	0.5945	0.3764	0.5579	0.5572	0.8973	0.9028	11.2286	16	6	16.4143	24.2857
0.3847	87.0	1392	1.7211	0.5965	0.3808	0.5582	0.5583	0.8977	0.9032	11.1929	16	6	16.3857	24.2857
0.3979	88.0	1408	1.7227	0.5928	0.374	0.5552	0.555	0.8973	0.902	11.15	16	6	16.3286	22.8571
0.3851	89.0	1424	1.7262	0.5908	0.3731	0.5538	0.5532	0.8978	0.9016	11.1143	16	6	16.25	20.7143
0.3762	90.0	1440	1.7262	0.591	0.3726	0.5542	0.5536	0.8975	0.9017	11.1214	16	6	16.3	20.0
0.3752	91.0	1456	1.7250	0.5924	0.3756	0.5555	0.555	0.8969	0.902	11.2	16	6	16.3571	23.5714
0.3825	92.0	1472	1.7273	0.5905	0.3728	0.5542	0.5541	0.8968	0.9015	11.2357	16	6	16.3714	23.5714
0.3731	93.0	1488	1.7295	0.5916	0.373	0.5541	0.5536	0.8972	0.902	11.2143	16	7	16.4286	22.1429
0.3707	94.0	1504	1.7313	0.5928	0.3746	0.5548	0.5545	0.8975	0.9023	11.1786	16	7	16.3929	22.1429
0.3708	95.0	1520	1.7323	0.5919	0.3737	0.5536	0.5536	0.8972	0.9019	11.2071	16	7	16.4286	22.1429
0.372	96.0	1536	1.7334	0.5914	0.3737	0.5537	0.5536	0.8971	0.9019	11.2	16	7	16.4	22.1429
0.3754	97.0	1552	1.7339	0.5905	0.3733	0.5527	0.5521	0.8975	0.9018	11.1714	16	7	16.3857	21.4286
0.3829	98.0	1568	1.7342	0.5923	0.3779	0.5548	0.5545	0.8979	0.9025	11.1571	16	7	16.3786	22.1429
0.3723	99.0	1584	1.7343	0.5936	0.3795	0.5556	0.5549	0.8978	0.9026	11.1643	16	7	16.4	22.1429
0.3846	100.0	1600	1.7343	0.5944	0.3803	0.5562	0.5556	0.8982	0.9028	11.1571	16	7	16.4	22.1429

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3