Speech5

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9659
• Wer: 1

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.01
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• training_steps: 5000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.9422	0.19	100	2.9742	1
3.006	0.38	200	2.9864	1
2.9665	0.56	300	2.9967	1
2.9411	0.75	400	2.9926	1
2.9937	0.94	500	3.0086	1
2.9476	1.13	600	3.0102	1
3.3385	1.31	700	2.9926	1
2.9979	1.5	800	2.9922	1
2.9534	1.69	900	3.0587	1
2.9544	1.88	1000	2.9880	1
2.9607	2.06	1100	2.9828	1
2.9926	2.25	1200	3.0249	1
2.9731	2.44	1300	2.9755	1
2.9438	2.63	1400	2.9707	1
2.9688	2.81	1500	3.0368	1
2.9335	3.0	1600	2.9695	1
2.9479	3.19	1700	3.0050	1
3.0182	3.38	1800	3.2477	1
2.9549	3.56	1900	2.9778	1
2.9212	3.75	2000	2.9697	1
2.9718	3.94	2100	2.9677	1
2.9288	4.13	2200	3.0418	1
2.9838	4.32	2300	2.9823	1
2.9812	4.5	2400	2.9902	1
2.9766	4.69	2500	2.9719	1
2.9479	4.88	2600	2.9805	1
2.9462	5.07	2700	3.1010	1
2.9998	5.25	2800	3.0566	1
2.9451	5.44	2900	3.1167	1
3.0465	5.63	3000	3.1573	1
2.9672	5.82	3100	2.9748	1
2.9555	6.0	3200	2.9929	1
2.9354	6.19	3300	2.9743	1
2.9281	6.38	3400	2.9746	1
3.0183	6.57	3500	2.9809	1
2.9602	6.75	3600	2.9662	1
2.9344	6.94	3700	3.0192	1
2.9748	7.13	3800	2.9653	1
3.0283	7.32	3900	3.0159	1
2.9367	7.5	4000	2.9780	1
2.9534	7.69	4100	2.9890	1
2.9481	7.88	4200	2.9701	1
2.89	8.07	4300	2.9625	1
2.9153	8.26	4400	2.9650	1
2.9648	8.44	4500	2.9726	1
2.9245	8.63	4600	3.0114	1
2.9608	8.82	4700	2.9645	1
2.9074	9.01	4800	2.9613	1
2.9059	9.19	4900	2.9639	1
2.9538	9.38	5000	2.9659	1

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3