hubert-base-libri-pruning-TEST13

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: -0.0766
• Wer: 0.1385

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00015
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1633	1.12	500	0.1342	0.1128
0.1467	2.24	1000	0.1452	0.1173
0.137	3.36	1500	0.1459	0.1174
0.1382	4.48	2000	0.1362	0.1172
0.1123	5.61	2500	0.1036	0.1172
0.0701	6.73	3000	0.0685	0.1135
0.0496	7.85	3500	0.0547	0.1172
0.0329	8.97	4000	0.0333	0.1172
0.0105	10.09	4500	0.0148	0.1175
-0.0203	11.21	5000	-0.0041	0.1171
-0.0334	12.33	5500	-0.0208	0.1170
-0.0549	13.45	6000	-0.0392	0.1170
-0.0688	14.57	6500	-0.0535	0.1170
-0.0751	15.7	7000	-0.0670	0.1170
-0.09	16.82	7500	-0.0816	0.1169
-0.1026	17.94	8000	-0.0919	0.1177
-0.1161	19.06	8500	-0.1012	0.1176
-0.1192	20.18	9000	-0.1104	0.1176
-0.1303	21.3	9500	-0.0413	0.1386
-0.1426	22.42	10000	-0.0510	0.1389
-0.141	23.54	10500	-0.0576	0.1385
-0.1489	24.66	11000	-0.0637	0.1386
-0.1492	25.78	11500	-0.0681	0.1386
-0.1619	26.91	12000	-0.0728	0.1383
-0.1567	28.03	12500	-0.0755	0.1384
-0.1627	29.15	13000	-0.0766	0.1385

Framework versions

• Transformers 4.30.0.dev0
• Pytorch 2.0.1
• Datasets 2.12.1.dev0
• Tokenizers 0.13.3