README.md

metadata

license: mit
base_model: microsoft/git-base
tags:
  - generated_from_trainer
model-index:
  - name: git-base-naruto
    results: []

git-base-naruto

This model is a fine-tuned version of microsoft/git-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0495
• Wer Score: 4.7488

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Score
7.2487	1.8182	50	4.3718	22.0605
2.0953	3.6364	100	0.2876	4.0186
0.0846	5.4545	150	0.0417	0.4419
0.023	7.2727	200	0.0380	0.4233
0.018	9.0909	250	0.0369	0.4186
0.0144	10.9091	300	0.0393	3.0093
0.0116	12.7273	350	0.0407	6.9628
0.0087	14.5455	400	0.0406	3.5209
0.0062	16.3636	450	0.0423	14.7023
0.0034	18.1818	500	0.0429	9.0372
0.0024	20.0	550	0.0471	8.3442
0.0013	21.8182	600	0.0469	13.5907
0.0009	23.6364	650	0.0464	14.6186
0.0005	25.4545	700	0.0468	11.1674
0.0004	27.2727	750	0.0476	7.9907
0.0003	29.0909	800	0.0480	7.3070
0.0003	30.9091	850	0.0480	7.2140
0.0003	32.7273	900	0.0484	6.9628
0.0003	34.5455	950	0.0487	6.8512
0.0003	36.3636	1000	0.0489	6.0698
0.0003	38.1818	1050	0.0491	5.4837
0.0003	40.0	1100	0.0492	4.9256
0.0002	41.8182	1150	0.0493	4.7860
0.0002	43.6364	1200	0.0493	4.8140
0.0002	45.4545	1250	0.0494	4.8
0.0002	47.2727	1300	0.0495	4.7581
0.0002	49.0909	1350	0.0495	4.7488

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1