whisper-tiny-v2-ta_tf32_adamBetas_2

This model is a fine-tuned version of openai/whisper-tiny on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3491
• Wer: 43.1477

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.5e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-06
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.6508	0.11	300	0.5656	54.3341
0.4612	0.21	600	0.4615	50.4600
0.408	0.32	900	0.4190	47.4576
0.3692	0.42	1200	0.4001	52.0581
0.3486	0.53	1500	0.3858	51.2349
0.3245	0.63	1800	0.3791	45.6174
0.3274	0.74	2100	0.3632	42.1792
0.2985	0.84	2400	0.3566	43.7288
0.2849	0.95	2700	0.3548	43.3898
0.2717	1.05	3000	0.3491	43.1477

Framework versions

• Transformers 4.28.0.dev0
• Pytorch 1.12.1
• Datasets 2.11.0
• Tokenizers 0.13.3