6801d24267114dec75e8918333a4bcdd

This model is a fine-tuned version of google/long-t5-tglobal-xl on the Helsinki-NLP/opus_books [de-ru] dataset. It achieves the following results on the evaluation set:

• Loss: 0.9725
• Data Size: 1.0
• Epoch Runtime: 244.6763
• Bleu: 10.7738

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Bleu
No log	0	0	2.7339	0	18.3415	0.1742
No log	1	434	2.3745	0.0078	21.8807	0.5254
No log	2	868	2.2218	0.0156	24.9837	0.5852
No log	3	1302	2.0547	0.0312	32.3317	0.7811
No log	4	1736	1.8924	0.0625	40.9173	0.9680
0.09	5	2170	1.7585	0.125	57.1947	2.2052
1.8871	6	2604	1.6081	0.25	82.5556	3.2407
1.6463	7	3038	1.4369	0.5	135.9702	4.2056
1.4239	8.0	3472	1.2427	1.0	244.7927	5.8541
1.2399	9.0	3906	1.1342	1.0	240.8457	6.8686
1.133	10.0	4340	1.0635	1.0	243.3272	7.8421
1.0354	11.0	4774	1.0145	1.0	241.9183	8.4413
0.9343	12.0	5208	0.9887	1.0	242.9128	8.9251
0.8684	13.0	5642	0.9627	1.0	243.2171	9.4522
0.7948	14.0	6076	0.9558	1.0	243.0954	9.8031
0.7418	15.0	6510	0.9395	1.0	242.1694	9.9913
0.6833	16.0	6944	0.9458	1.0	243.8600	10.3211
0.6286	17.0	7378	0.9499	1.0	244.5621	10.5272
0.5797	18.0	7812	0.9623	1.0	243.3430	10.6203
0.5235	19.0	8246	0.9725	1.0	244.6763	10.7738

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.2.0
• Tokenizers 0.22.1