hubert-base-libri-demo-feature_extractor_frozen_v2

This model is a fine-tuned version of facebook/hubert-base-ls960 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1202
• Wer: 0.1115

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00015
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.401	1.12	500	3.5086	1.0
2.8748	2.24	1000	3.3953	1.0
2.2716	3.36	1500	0.7177	0.6110
0.5536	4.48	2000	0.2387	0.2692
0.2897	5.61	2500	0.1593	0.1946
0.2077	6.73	3000	0.1401	0.1558
0.1778	7.85	3500	0.1225	0.1423
0.1639	8.97	4000	0.1156	0.1342
0.1478	10.09	4500	0.1186	0.1290
0.1146	11.21	5000	0.1131	0.1244
0.1172	12.33	5500	0.1189	0.1235
0.0925	13.45	6000	0.1175	0.1214
0.092	14.57	6500	0.1224	0.1194
0.0865	15.7	7000	0.1160	0.1196
0.0786	16.82	7500	0.1151	0.1152
0.0743	17.94	8000	0.1124	0.1153
0.0739	19.06	8500	0.1214	0.1146
0.0774	20.18	9000	0.1219	0.1143
0.0667	21.3	9500	0.1188	0.1129
0.0661	22.42	10000	0.1176	0.1123
0.0606	23.54	10500	0.1201	0.1118
0.0584	24.66	11000	0.1202	0.1115

Framework versions

• Transformers 4.30.0.dev0
• Pytorch 2.0.1
• Datasets 2.12.1.dev0
• Tokenizers 0.13.3