tun_msa_wav2vec2

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5373
• Wer: 0.5598
• Cer: 0.1779

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
4.1696	2.2727	700	1.2685	0.8366	0.3108
1.4028	4.5455	1400	0.7872	0.7136	0.2375
1.0872	6.8182	2100	0.6787	0.6778	0.2198
0.9573	9.0909	2800	0.6569	0.6605	0.2121
0.939	11.3636	3500	0.6115	0.6409	0.2040
0.8378	13.6364	4200	0.6097	0.6288	0.2000
0.8026	15.9091	4900	0.5843	0.6125	0.1949
0.7712	18.1818	5600	0.5830	0.6080	0.1942
0.7561	20.4545	6300	0.5637	0.5951	0.1889
0.7255	22.7273	7000	0.5630	0.5888	0.1869
0.7051	25.0	7700	0.5573	0.5852	0.1848
0.6873	27.2727	8400	0.5577	0.5799	0.1841
0.6722	29.5455	9100	0.5460	0.5748	0.1827
0.6673	31.8182	9800	0.5495	0.5725	0.1827
0.6359	34.0909	10500	0.5509	0.5720	0.1831
0.6513	36.3636	11200	0.5455	0.5676	0.1804
0.6402	38.6364	11900	0.5373	0.5649	0.1797
0.6302	40.9091	12600	0.5399	0.5643	0.1797
0.617	43.1818	13300	0.5368	0.5628	0.1793
0.6353	45.4545	14000	0.5369	0.5596	0.1786
0.6277	47.7273	14700	0.5361	0.5597	0.1783
0.5956	50.0	15400	0.5373	0.5598	0.1779

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1