finetuned_byT5base_multi_joint

This model is a fine-tuned version of google/byt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8197
• Chrf++: 63.26
• Gen Len: 98.3686

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 13

Training results

Training Loss	Epoch	Step	Validation Loss	Chrf++	Gen Len
3.0857	1.9967	1052	0.6548	52.9032	105.0823
1.2148	3.9929	2104	0.6039	62.7426	100.1646
0.7625	5.9891	3156	0.6561	62.9483	105.9513
0.5638	7.9853	4208	0.7659	63.7187	98.958
0.5107	9.9815	5260	0.7403	64.3382	102.4148
0.4202	11.9777	6312	0.8155	61.7826	117.5038
0.4735	13.0	6851	0.8197	63.26	98.3686

Framework versions

• Transformers 5.5.0
• Pytorch 2.11.0+cu130
• Datasets 4.3.0
• Tokenizers 0.22.2