abbenedek/wav2vec2-base-wer

wav2vec2-tokenizer

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0005
• Wer: 0.2412

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
2.8291	4.0	100	1.7138	0.9862
1.2768	8.0	200	0.7349	0.7488
0.53	12.0	300	0.2418	0.705
0.2342	16.0	400	0.1818	0.7362
0.1375	20.0	500	0.1053	0.73
0.1286	24.0	600	0.0886	0.7063
0.0978	28.0	700	0.0634	0.74
0.0952	32.0	800	0.0642	0.6963
0.088	36.0	900	0.0674	0.7025
0.0802	40.0	1000	0.0140	0.2587
0.0624	44.0	1100	0.0185	0.1862
0.029	48.0	1200	0.0234	0.2725
0.0176	52.0	1300	0.0072	0.2275
0.016	56.0	1400	0.0036	0.265
0.0047	60.0	1500	0.0019	0.235
0.0066	64.0	1600	0.0014	0.2075
0.0041	68.0	1700	0.0009	0.2712
0.0019	72.0	1800	0.0008	0.2863
0.002	76.0	1900	0.0007	0.2888
0.0031	80.0	2000	0.0006	0.2863
0.0032	84.0	2100	0.0006	0.2762
0.0026	88.0	2200	0.0005	0.2325
0.0019	92.0	2300	0.0005	0.2362
0.0046	96.0	2400	0.0005	0.2412
0.0018	100.0	2500	0.0005	0.2412

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.2+cu121
• Datasets 2.14.5
• Tokenizers 0.15.2