din0s/t5-small-ro-finetuned-en-to-it

t5-small_ro-finetuned-en-to-it

This model is a fine-tuned version of din0s/t5-small-finetuned-en-to-ro on the ccmatrix dataset. It achieves the following results on the evaluation set:

• Loss: 2.3301
• Bleu: 7.11
• Gen Len: 59.538

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 96
• eval_batch_size: 96
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	94	3.1873	1.8829	94.336
No log	2.0	188	3.0132	2.5572	83.6413
No log	3.0	282	2.9226	3.0999	76.166
No log	4.0	376	2.8559	3.3859	73.906
No log	5.0	470	2.8040	3.8793	70.5667
3.4136	6.0	564	2.7585	3.8974	69.404
3.4136	7.0	658	2.7191	4.1236	68.6387
3.4136	8.0	752	2.6836	4.3281	67.682
3.4136	9.0	846	2.6509	4.343	67.0113
3.4136	10.0	940	2.6231	4.561	66.094
3.0128	11.0	1034	2.5984	4.9314	64.3033
3.0128	12.0	1128	2.5755	5.0149	64.502
3.0128	13.0	1222	2.5529	5.0758	65.194
3.0128	14.0	1316	2.5332	5.2774	64.0773
3.0128	15.0	1410	2.5128	5.3165	64.5213
2.859	16.0	1504	2.4976	5.4714	62.884
2.859	17.0	1598	2.4803	5.532	63.9593
2.859	18.0	1692	2.4662	5.6314	63.7273
2.859	19.0	1786	2.4529	5.8855	62.8953
2.859	20.0	1880	2.4401	6.1225	61.496
2.859	21.0	1974	2.4285	6.2697	60.034
2.7544	22.0	2068	2.4171	6.4153	60.6027
2.7544	23.0	2162	2.4067	6.3731	61.08
2.7544	24.0	2256	2.3974	6.5355	61.3707
2.7544	25.0	2350	2.3882	6.5259	60.966
2.7544	26.0	2444	2.3808	6.7178	60.1707
2.6867	27.0	2538	2.3733	6.8196	60.398
2.6867	28.0	2632	2.3672	6.811	60.2753
2.6867	29.0	2726	2.3609	6.8322	60.0973
2.6867	30.0	2820	2.3551	6.8524	60.16
2.6867	31.0	2914	2.3502	6.8363	60.2867
2.6395	32.0	3008	2.3455	6.9495	59.8547
2.6395	33.0	3102	2.3422	6.9939	59.4813
2.6395	34.0	3196	2.3387	6.9953	59.812
2.6395	35.0	3290	2.3362	7.0318	59.8813
2.6395	36.0	3384	2.3339	7.1114	59.232
2.6395	37.0	3478	2.3322	7.1488	59.292
2.6138	38.0	3572	2.3310	7.1334	59.6093
2.6138	39.0	3666	2.3303	7.0984	59.6887
2.6138	40.0	3760	2.3301	7.11	59.538

Framework versions

• Transformers 4.22.1
• Pytorch 1.12.1
• Datasets 2.5.1
• Tokenizers 0.11.0