hubert-base-libri-pruning-TEST15

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: -0.1647
• Wer: 0.1120

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00015
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1439	1.12	500	0.1094	0.1124
0.1385	2.24	1000	0.1164	0.1121
0.1382	3.36	1500	0.1255	0.1124
0.1471	4.48	2000	0.1223	0.1117
0.1273	5.61	2500	0.0958	0.1121
0.0876	6.73	3000	0.0712	0.1120
0.067	7.85	3500	0.0461	0.1121
0.0502	8.97	4000	0.0251	0.1119
0.0279	10.09	4500	0.0051	0.1123
-0.003	11.21	5000	-0.0139	0.1123
-0.016	12.33	5500	-0.0303	0.1117
-0.0375	13.45	6000	-0.0479	0.1118
-0.0515	14.57	6500	-0.0630	0.1124
-0.0578	15.7	7000	-0.0768	0.1123
-0.0727	16.82	7500	-0.0911	0.1123
-0.0854	17.94	8000	-0.1032	0.1123
-0.0987	19.06	8500	-0.1132	0.1123
-0.1018	20.18	9000	-0.1225	0.1122
-0.1129	21.3	9500	-0.1321	0.1123
-0.1252	22.42	10000	-0.1399	0.1121
-0.1237	23.54	10500	-0.1468	0.1120
-0.1316	24.66	11000	-0.1523	0.1122
-0.1317	25.78	11500	-0.1571	0.1120
-0.1445	26.91	12000	-0.1610	0.1123
-0.1393	28.03	12500	-0.1635	0.1120
-0.1453	29.15	13000	-0.1647	0.1120

Framework versions

• Transformers 4.30.0.dev0
• Pytorch 2.0.1
• Datasets 2.12.1.dev0
• Tokenizers 0.13.3