akkadian-v2

This model is a fine-tuned version of google/byt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5212
• Bleu: 25.8487
• Chrf: 45.3423
• Geo Mean: 34.2350

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• gradient_accumulation_steps: 8
• total_train_batch_size: 96
• total_eval_batch_size: 12
• optimizer: Use OptimizerNames.ADAFACTOR and the args are: No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Chrf	Geo Mean
1.704	5.0	150	0.7985	11.2906	33.0021	19.3032
0.828	10.0	300	0.6436	20.2554	40.6504	28.6948
0.7496	15.0	450	0.5792	21.9499	42.6915	30.6117
0.6686	20.0	600	0.5435	23.7677	44.0246	32.3476
0.646	25.0	750	0.5269	25.3723	44.9495	33.7708
0.6244	30.0	900	0.5212	25.8487	45.3423	34.2350

Framework versions

• Transformers 4.57.1
• Pytorch 2.8.0+cu126
• Datasets 4.4.2
• Tokenizers 0.22.1