vit5-correction

This model is a fine-tuned version of VietAI/vit5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1700
• Model Preparation Time: 0.0058
• Sacrebleu: 38.8388

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 48
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Sacrebleu
0.319	1.6476	5000	0.1981	0.0058	38.1039
0.1877	3.2949	10000	0.1761	0.0058	38.5248
0.1452	4.9425	15000	0.1700	0.0058	38.8388
0.1157	6.5899	20000	0.1703	0.0058	38.8842
0.0977	8.2373	25000	0.1752	0.0058	38.9902
0.0827	9.8848	30000	0.1765	0.0058	39.0245
0.0713	11.5322	35000	0.1865	0.0058	39.0606
0.0636	13.1796	40000	0.1902	0.0058	39.1080
0.0574	14.8272	45000	0.1944	0.0058	39.1167
0.0533	16.4745	50000	0.1978	0.0058	39.1640
0.0513	18.1219	55000	0.1980	0.0058	39.1460
0.0501	19.7695	60000	0.1986	0.0058	39.1743

Framework versions

• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.21.1