t5-small-finetuned-it-to-en

This model is a fine-tuned version of t5-small on the ccmatrix dataset. It achieves the following results on the evaluation set:

• Loss: 2.4066
• Bleu: 13.5927
• Gen Len: 28.6473

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	282	2.9032	7.2452	35.1327
3.2663	2.0	564	2.8156	8.3336	31.2673
3.2663	3.0	846	2.7615	9.1403	30.248
3.0467	4.0	1128	2.7211	9.2676	30.2893
3.0467	5.0	1410	2.6872	9.5597	30.4067
2.9575	6.0	1692	2.6600	9.907	30.3153
2.9575	7.0	1974	2.6361	10.2292	29.902
2.8814	8.0	2256	2.6132	10.4384	30.1187
2.8284	9.0	2538	2.5930	10.572	30.0447
2.8284	10.0	2820	2.5774	10.9557	29.5547
2.7827	11.0	3102	2.5604	11.1435	29.5847
2.7827	12.0	3384	2.5484	11.4067	29.4807
2.7496	13.0	3666	2.5342	11.569	29.5827
2.7496	14.0	3948	2.5208	11.7581	30.07
2.7094	15.0	4230	2.5105	11.9629	29.6993
2.6764	16.0	4512	2.5007	12.2675	29.1
2.6764	17.0	4794	2.4916	12.2227	29.4
2.6516	18.0	5076	2.4817	12.3529	29.222
2.6516	19.0	5358	2.4747	12.6053	29.036
2.6271	20.0	5640	2.4672	12.6659	29.0993
2.6271	21.0	5922	2.4602	12.8286	29.2087
2.602	22.0	6204	2.4546	12.8915	29.0233
2.602	23.0	6486	2.4486	12.7892	29.2173
2.5922	24.0	6768	2.4438	12.8928	29.042
2.5781	25.0	7050	2.4386	13.1954	28.8607
2.5781	26.0	7332	2.4341	13.0077	28.8367
2.5578	27.0	7614	2.4306	13.1084	28.7487
2.5578	28.0	7896	2.4258	13.0929	28.9067
2.5477	29.0	8178	2.4236	13.2008	28.966
2.5477	30.0	8460	2.4203	13.3476	28.7133
2.5331	31.0	8742	2.4170	13.3539	28.8787
2.5312	32.0	9024	2.4148	13.3781	28.742
2.5312	33.0	9306	2.4130	13.3425	28.8393
2.5234	34.0	9588	2.4113	13.4549	28.732
2.5234	35.0	9870	2.4099	13.5228	28.8313
2.5131	36.0	10152	2.4084	13.547	28.6733
2.5131	37.0	10434	2.4076	13.6099	28.5193
2.5101	38.0	10716	2.4071	13.5853	28.64
2.5101	39.0	10998	2.4067	13.572	28.6687
2.5136	40.0	11280	2.4066	13.5927	28.6473

Framework versions

• Transformers 4.22.1
• Pytorch 1.12.1
• Datasets 2.5.1
• Tokenizers 0.11.0