NLP_Project

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5308
• Wer: 0.3428

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
3.5939	1.0	500	2.1356	1.0014
0.9126	2.01	1000	0.5469	0.5354
0.4491	3.01	1500	0.4636	0.4503
0.3008	4.02	2000	0.4269	0.4330
0.2229	5.02	2500	0.4164	0.4073
0.188	6.02	3000	0.4717	0.4107
0.1739	7.03	3500	0.4306	0.4031
0.159	8.03	4000	0.4394	0.3993
0.1342	9.04	4500	0.4462	0.3904
0.1093	10.04	5000	0.4387	0.3759
0.1005	11.04	5500	0.5033	0.3847
0.0857	12.05	6000	0.4805	0.3876
0.0779	13.05	6500	0.5269	0.3810
0.072	14.06	7000	0.5109	0.3710
0.0641	15.06	7500	0.4865	0.3638
0.0584	16.06	8000	0.5041	0.3646
0.0552	17.07	8500	0.4987	0.3537
0.0535	18.07	9000	0.4947	0.3586
0.0475	19.08	9500	0.5237	0.3647
0.042	20.08	10000	0.5338	0.3561
0.0416	21.08	10500	0.5068	0.3483
0.0358	22.09	11000	0.5126	0.3532
0.0334	23.09	11500	0.5213	0.3536
0.0331	24.1	12000	0.5378	0.3496
0.03	25.1	12500	0.5167	0.3470
0.0254	26.1	13000	0.5245	0.3418
0.0233	27.11	13500	0.5393	0.3456
0.0232	28.11	14000	0.5279	0.3425
0.022	29.12	14500	0.5308	0.3428

Framework versions

• Transformers 4.17.0
• Pytorch 1.11.0+cu113
• Datasets 1.18.3
• Tokenizers 0.12.1