ukrt5-formality-transfer-ukrainian-v3

This model is a fine-tuned version of uaritm/ukrt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3119
• Bleu: 31.7578
• Mean Pred Words: 8.914

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.05
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Mean Pred Words
7.6742	0.1822	2000	2.6238	9.2852	11.759
5.2228	0.3644	4000	1.9450	26.1399	9.0575
4.5354	0.5466	6000	1.7585	28.3055	9.0335
4.1649	0.7288	8000	1.6620	29.4042	8.944
3.9500	0.9110	10000	1.5934	29.9195	8.935
3.7671	1.0931	12000	1.5413	30.1013	8.907
3.6561	1.2753	14000	1.5023	30.5751	8.942
3.5798	1.4575	16000	1.4695	30.7396	8.9365
3.4816	1.6397	18000	1.4521	30.9153	8.933
3.4415	1.8219	20000	1.4239	31.0094	8.9305
3.3383	2.0040	22000	1.4107	31.3153	8.9195
3.2935	2.1862	24000	1.3933	31.3959	8.9175
3.2463	2.3684	26000	1.3768	31.3865	8.907
3.2235	2.5506	28000	1.3618	31.5856	8.9415
3.1978	2.7328	30000	1.3462	31.5948	8.9025
3.1572	2.9150	32000	1.3413	31.6598	8.915
3.1086	3.0971	34000	1.3328	31.6810	8.9105
3.0953	3.2793	36000	1.3248	32.0267	8.92
3.0892	3.4615	38000	1.3165	31.9114	8.909
3.0602	3.6437	40000	1.3119	31.7578	8.914

Framework versions

• Transformers 5.0.0
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2