text_shortening_model_v51

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4778
• Rouge1: 0.5085
• Rouge2: 0.2885
• Rougel: 0.455
• Rougelsum: 0.4548
• Bert precision: 0.8747
• Bert recall: 0.8765
• Average word count: 8.5688
• Max word count: 16
• Min word count: 3
• Average token count: 13.0873
• % shortened texts with length > 12: 11.1111

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
2.2018	1.0	83	1.7246	0.4863	0.2654	0.437	0.4366	0.8646	0.8704	9.1111	18	3	13.5952	15.3439
1.7958	2.0	166	1.5840	0.4838	0.2672	0.4335	0.4329	0.8659	0.8697	8.8228	17	3	13.3386	13.2275
1.6387	3.0	249	1.5093	0.4927	0.2733	0.4416	0.4421	0.868	0.8728	8.9339	17	4	13.4656	15.0794
1.5337	4.0	332	1.4567	0.4967	0.2722	0.4399	0.4402	0.8681	0.8745	9.0847	17	4	13.6614	17.1958
1.4477	5.0	415	1.4176	0.5015	0.2728	0.4432	0.443	0.8698	0.8738	8.836	17	4	13.3069	13.4921
1.3548	6.0	498	1.3917	0.5068	0.28	0.4494	0.4498	0.8698	0.8771	9.1429	17	4	13.7222	15.6085
1.3128	7.0	581	1.3777	0.5054	0.2856	0.4452	0.4445	0.8698	0.8772	9.1323	17	5	13.7037	14.8148
1.2417	8.0	664	1.3575	0.5118	0.2904	0.4587	0.4591	0.8737	0.8779	8.8228	17	3	13.3466	12.963
1.1854	9.0	747	1.3508	0.5121	0.2894	0.4549	0.4552	0.8723	0.8768	8.828	18	4	13.418	13.4921
1.1384	10.0	830	1.3482	0.5203	0.2918	0.458	0.4579	0.8742	0.8809	9.0767	17	4	13.7196	14.2857
1.0986	11.0	913	1.3373	0.5083	0.2874	0.453	0.4521	0.8726	0.8765	8.8333	16	3	13.3254	12.963
1.0575	12.0	996	1.3345	0.5106	0.2872	0.4557	0.4552	0.8716	0.8773	9.045	17	3	13.5979	15.6085
1.0196	13.0	1079	1.3331	0.5127	0.2876	0.4578	0.4571	0.8734	0.8781	8.8836	16	3	13.4762	13.4921
0.9666	14.0	1162	1.3465	0.5052	0.2807	0.4467	0.4461	0.8717	0.8764	8.9048	16	3	13.3228	12.1693
0.9521	15.0	1245	1.3419	0.5053	0.2828	0.4464	0.4458	0.8727	0.8748	8.6508	16	3	13.0952	12.4339
0.917	16.0	1328	1.3438	0.512	0.2936	0.4563	0.456	0.8743	0.8784	8.7778	16	3	13.3783	13.7566
0.8852	17.0	1411	1.3436	0.5034	0.2753	0.4434	0.4422	0.871	0.8743	8.6984	16	3	13.1958	12.963
0.8778	18.0	1494	1.3529	0.5036	0.2765	0.4409	0.4404	0.8711	0.8748	8.7143	16	3	13.2646	11.9048
0.8513	19.0	1577	1.3494	0.5127	0.2886	0.4512	0.4508	0.8733	0.8783	8.7937	16	3	13.4233	11.3757
0.8265	20.0	1660	1.3512	0.5133	0.2854	0.4552	0.455	0.8724	0.8784	8.9497	16	4	13.5582	12.4339
0.8058	21.0	1743	1.3561	0.5087	0.2825	0.452	0.4516	0.8722	0.8765	8.8016	16	4	13.3122	11.9048
0.7751	22.0	1826	1.3602	0.5022	0.2802	0.4459	0.4455	0.8726	0.8741	8.5556	16	3	12.9656	9.7884
0.7625	23.0	1909	1.3737	0.5077	0.2827	0.4518	0.4515	0.874	0.8747	8.4444	16	3	12.9259	8.4656
0.7568	24.0	1992	1.3807	0.5078	0.284	0.4527	0.4523	0.8737	0.8758	8.5423	16	3	13.0741	8.4656
0.7309	25.0	2075	1.3857	0.5105	0.2863	0.4522	0.4528	0.8743	0.8761	8.5661	16	3	13.1296	10.8466
0.7115	26.0	2158	1.3948	0.5107	0.2887	0.4556	0.4554	0.8743	0.8777	8.6561	16	3	13.2381	10.582
0.6933	27.0	2241	1.4063	0.5155	0.2905	0.4582	0.4583	0.8748	0.8778	8.7116	16	3	13.3201	10.8466
0.6751	28.0	2324	1.4170	0.5109	0.2894	0.453	0.4529	0.8748	0.8764	8.5635	16	3	13.1561	10.582
0.6655	29.0	2407	1.4178	0.5105	0.2821	0.4513	0.4506	0.8744	0.8767	8.6349	16	3	13.1429	11.9048
0.6577	30.0	2490	1.4196	0.5112	0.2895	0.4506	0.451	0.8735	0.8779	8.8439	16	3	13.4339	12.963
0.6584	31.0	2573	1.4155	0.51	0.285	0.4523	0.4526	0.8745	0.876	8.5926	16	3	13.0608	10.582
0.6464	32.0	2656	1.4257	0.5095	0.2882	0.455	0.4553	0.8753	0.876	8.5423	16	3	13.0794	9.2593
0.6319	33.0	2739	1.4304	0.5112	0.2886	0.4559	0.456	0.8752	0.8768	8.5661	16	3	13.1085	9.7884
0.6236	34.0	2822	1.4383	0.5117	0.2922	0.4581	0.4589	0.8753	0.8771	8.5741	16	3	13.1534	11.1111
0.6045	35.0	2905	1.4499	0.5081	0.2891	0.4561	0.4561	0.8742	0.8761	8.5529	16	3	13.1138	10.582
0.6041	36.0	2988	1.4628	0.5111	0.2875	0.455	0.4552	0.8747	0.8766	8.6111	16	3	13.1429	10.582
0.5983	37.0	3071	1.4532	0.512	0.2886	0.4559	0.4562	0.8751	0.8781	8.672	16	3	13.2196	10.582
0.5869	38.0	3154	1.4504	0.5108	0.2857	0.4555	0.4551	0.8747	0.8783	8.7407	16	3	13.3148	11.3757
0.5875	39.0	3237	1.4620	0.5061	0.2859	0.4535	0.4537	0.8741	0.8759	8.5899	16	3	13.1058	10.8466
0.5722	40.0	3320	1.4642	0.5088	0.2858	0.4549	0.4546	0.874	0.8772	8.672	16	3	13.2116	10.8466
0.5818	41.0	3403	1.4630	0.5109	0.288	0.4561	0.456	0.8748	0.8771	8.5979	16	3	13.127	10.3175
0.5694	42.0	3486	1.4699	0.5113	0.2887	0.4567	0.4566	0.8754	0.8764	8.5317	16	3	13.0238	10.3175
0.5575	43.0	3569	1.4752	0.5111	0.2914	0.4574	0.4575	0.8751	0.8762	8.5106	16	3	12.963	10.3175
0.5566	44.0	3652	1.4740	0.5097	0.2878	0.4553	0.4554	0.8746	0.8769	8.619	16	3	13.1296	10.8466
0.5654	45.0	3735	1.4732	0.5048	0.2849	0.4519	0.4518	0.8738	0.8754	8.5529	16	3	13.0529	10.582
0.5569	46.0	3818	1.4763	0.5089	0.2889	0.4564	0.4562	0.8749	0.8765	8.5344	16	3	13.0397	10.0529
0.5527	47.0	3901	1.4786	0.5109	0.2895	0.4562	0.4566	0.8751	0.8771	8.6217	16	3	13.1508	11.3757
0.5461	48.0	3984	1.4769	0.5069	0.2869	0.454	0.4538	0.8742	0.8761	8.6085	16	3	13.1138	11.9048
0.5476	49.0	4067	1.4775	0.5084	0.2871	0.4537	0.4534	0.8747	0.8764	8.545	16	3	13.0529	10.8466
0.5525	50.0	4150	1.4778	0.5085	0.2885	0.455	0.4548	0.8747	0.8765	8.5688	16	3	13.0873	11.1111

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3