f115407087a1e022b8ca7f1c9cf00ce4

This model is a fine-tuned version of google/long-t5-tglobal-xl on the Helsinki-NLP/opus_books [de-en] dataset. It achieves the following results on the evaluation set:

• Loss: 1.6752
• Data Size: 1.0
• Epoch Runtime: 728.5588
• Bleu: 11.0172

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Bleu
No log	0	0	3.7799	0	52.6414	1.9010
No log	1	1286	2.2956	0.0078	58.4889	5.1387
0.0549	2	2572	2.1885	0.0156	68.3799	5.6244
0.0692	3	3858	2.1139	0.0312	81.7533	6.2257
0.102	4	5144	2.0311	0.0625	103.2459	6.9937
2.1964	5	6430	1.9438	0.125	141.1829	7.5338
2.061	6	7716	1.8360	0.25	219.6152	7.9932
1.9017	7	9002	1.7254	0.5	376.1191	9.0838
1.7254	8.0	10288	1.6044	1.0	699.9676	10.2669
1.5405	9.0	11574	1.5638	1.0	696.5851	10.6310
1.3652	10.0	12860	1.5488	1.0	693.7357	10.8673
1.2162	11.0	14146	1.5654	1.0	699.0673	11.0604
1.1305	12.0	15432	1.5931	1.0	726.2251	11.1675
0.9859	13.0	16718	1.6241	1.0	729.3197	11.2405
0.8998	14.0	18004	1.6752	1.0	728.5588	11.0172

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.2.0
• Tokenizers 0.22.1