ukrt5-formality-transfer-ukrainian-v4

This model is a fine-tuned version of uaritm/ukrt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3509
• Bleu: 29.8446
• Mean Pred Words: 8.8865

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.05
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Mean Pred Words
6.9646	0.2611	2000	2.3834	12.7318	11.2575
5.0468	0.5221	4000	1.8917	24.5780	9.2275
4.5162	0.7832	6000	1.7397	26.1762	9.0015
4.1834	1.0443	8000	1.6485	27.3852	9.011
3.9961	1.3053	10000	1.5766	27.8863	8.981
3.8509	1.5664	12000	1.5287	28.2907	8.9795
3.7421	1.8274	14000	1.4980	28.6894	8.926
3.6209	2.0885	16000	1.4653	28.8614	8.9455
3.5443	2.3496	18000	1.4389	28.9359	8.93
3.5049	2.6106	20000	1.4210	29.2368	8.8765
3.4444	2.8717	22000	1.4088	29.2687	8.9095
3.3854	3.1328	24000	1.3931	29.5429	8.897
3.3632	3.3938	26000	1.3801	29.4986	8.9135
3.3391	3.6549	28000	1.3725	29.6629	8.881
3.2867	3.9159	30000	1.3643	29.7246	8.9105
3.2754	4.1770	32000	1.3577	29.8422	8.897
3.2765	4.4381	34000	1.3546	29.8180	8.9015
3.2598	4.6991	36000	1.3519	29.8762	8.8915
3.2576	4.9602	38000	1.3509	29.8446	8.8865

Framework versions

• Transformers 5.0.0
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2