ukrt5-formality-transfer-ukrainian-v2

This model is a fine-tuned version of uaritm/ukrt5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4157
• Bleu: 27.4996
• Mean Pred Words: 8.7825

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 0.05
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Mean Pred Words
5.7873	0.3220	2000	2.3162	15.1974	10.4785
4.4623	0.6440	4000	1.8967	23.6807	8.908
4.0487	0.9660	6000	1.7493	24.9980	8.7445
3.7900	1.2880	8000	1.6613	25.6807	8.826
3.6372	1.6100	10000	1.6000	26.0377	8.8365
3.5140	1.9321	12000	1.5544	26.4619	8.763
3.3908	2.2541	14000	1.5228	26.8278	8.859
3.3445	2.5761	16000	1.4974	26.5964	8.7925
3.3143	2.8981	18000	1.4715	27.1185	8.7945
3.2164	3.2201	20000	1.4577	26.9899	8.7645
3.2085	3.5421	22000	1.4430	27.3377	8.779
3.1858	3.8641	24000	1.4332	27.3143	8.788
3.1476	4.1861	26000	1.4237	27.5502	8.7675
3.1263	4.5081	28000	1.4195	27.5021	8.7745
3.1078	4.8301	30000	1.4157	27.4996	8.7825

Framework versions

• Transformers 5.0.0
• Pytorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2