asr-cherokee

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5317
• Wer: 0.5301
• Cer: 0.0658

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
6.4965	1.0753	100	3.0036	1.0	1.0
2.8794	2.1505	200	2.8098	1.0	1.0
2.742	3.2258	300	2.2956	1.0	0.8959
1.4778	4.3011	400	0.6221	1.0	0.2640
0.7063	5.3763	500	0.4566	0.9167	0.1796
0.5382	6.4516	600	0.4052	0.8429	0.1337
0.446	7.5269	700	0.3548	0.6557	0.0876
0.3996	8.6022	800	0.3311	0.6052	0.0835
0.3443	9.6774	900	0.3242	0.5943	0.0785
0.298	10.7527	1000	0.3383	0.6079	0.0831
0.3016	11.8280	1100	0.3065	0.5738	0.0745
0.2707	12.9032	1200	0.3193	0.5697	0.0768
0.2406	13.9785	1300	0.3305	0.5861	0.0847
0.2325	15.0538	1400	0.3313	0.5847	0.0803
0.2033	16.1290	1500	0.3353	0.5492	0.0715
0.19	17.2043	1600	0.3326	0.5505	0.0732
0.1637	18.2796	1700	0.3355	0.5587	0.0734
0.1528	19.3548	1800	0.3814	0.5738	0.0751
0.1441	20.4301	1900	0.3737	0.5683	0.0742
0.1314	21.5054	2000	0.3856	0.5505	0.0729
0.134	22.5806	2100	0.4051	0.5724	0.0772
0.1137	23.6559	2200	0.4194	0.5423	0.0708
0.1179	24.7312	2300	0.4052	0.5396	0.0694
0.1109	25.8065	2400	0.4156	0.5464	0.0717
0.1013	26.8817	2500	0.4085	0.5328	0.0680
0.0979	27.9570	2600	0.4508	0.5451	0.0695
0.0887	29.0323	2700	0.4390	0.5505	0.0703
0.0839	30.1075	2800	0.4677	0.5642	0.0739
0.0795	31.1828	2900	0.4676	0.5464	0.0701
0.0803	32.2581	3000	0.4713	0.5410	0.0684
0.0713	33.3333	3100	0.5049	0.5287	0.0675
0.0736	34.4086	3200	0.4983	0.5396	0.0689
0.0669	35.4839	3300	0.5125	0.5314	0.0692
0.0631	36.5591	3400	0.5243	0.5383	0.0696
0.065	37.6344	3500	0.5170	0.5492	0.0700
0.0566	38.7097	3600	0.5260	0.5369	0.0692
0.057	39.7849	3700	0.5125	0.5301	0.0674
0.0522	40.8602	3800	0.5222	0.5287	0.0666
0.051	41.9355	3900	0.5204	0.5383	0.0678
0.0505	43.0108	4000	0.5086	0.5287	0.0664
0.054	44.0860	4100	0.5119	0.5410	0.0686
0.0458	45.1613	4200	0.5193	0.5383	0.0679
0.047	46.2366	4300	0.5217	0.5328	0.0675
0.0444	47.3118	4400	0.5290	0.5246	0.0658
0.0458	48.3871	4500	0.5314	0.5355	0.0663
0.0423	49.4624	4600	0.5317	0.5301	0.0658

Framework versions

• Transformers 4.44.2
• Pytorch 2.3.1
• Datasets 2.21.0
• Tokenizers 0.19.1