voice-clone-large-finetune

This model is a fine-tuned version of openai/whisper-large-v3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4491
• Wer: 16.9582

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1608	0.8460	250	0.5171	25.8227
0.0607	1.6920	500	0.4735	28.3427
0.0255	2.5381	750	0.4274	25.4966
0.0138	3.3841	1000	0.4327	18.9742
0.0013	4.2301	1250	0.4508	20.8123
0.0129	5.0761	1500	0.4107	21.2274
0.0005	5.9222	1750	0.4218	21.5535
0.0018	6.7682	2000	0.4256	17.5215
0.0021	7.6142	2250	0.4224	18.1441
0.0015	8.4602	2500	0.4298	18.0255
0.0008	9.3063	2750	0.4376	18.1441
0.0005	10.1523	3000	0.4418	17.6697
0.0014	10.9983	3250	0.4442	17.5808
0.0002	11.8443	3500	0.4422	17.1064
0.0009	12.6904	3750	0.4408	17.1657
0.0002	13.5364	4000	0.4438	16.9878
0.0009	14.3824	4250	0.4452	16.7803
0.0007	15.2284	4500	0.4457	16.8989
0.0	16.0745	4750	0.4485	16.8693
0.0	16.9205	5000	0.4491	16.9582

Framework versions

• Transformers 4.45.2
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3