git-base-naruto

This model is a fine-tuned version of microsoft/git-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0529
• Wer Score: 1.4091

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Score
7.3586	3.7037	50	4.5383	8.8030
2.3507	7.4074	100	0.4544	0.4697
0.1281	11.1111	150	0.0543	0.5152
0.0161	14.8148	200	0.0491	0.4545
0.0115	18.5185	250	0.0501	0.4394
0.0099	22.2222	300	0.0528	0.4697
0.0085	25.9259	350	0.0536	0.4697
0.0075	29.6296	400	0.0532	0.4848
0.0068	33.3333	450	0.0520	0.4697
0.0061	37.0370	500	0.0528	0.6818
0.0054	40.7407	550	0.0530	0.8030
0.0044	44.4444	600	0.0535	1.2121
0.0038	48.1481	650	0.0529	1.4091

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1