hubert-base-libri-pruning-v2-testing4

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: -7567.2393
• Wer: 0.4152

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00015
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
-14.3764	1.12	500	-47.0110	0.5199
-106.5551	2.24	1000	-189.6375	0.4788
-290.9837	3.36	1500	-427.6701	0.4637
-566.8718	4.48	2000	-760.7963	0.4523
-932.8283	5.61	2500	-1188.8673	0.4457
-1393.9495	6.73	3000	-1711.7064	0.4362
-1926.4679	7.85	3500	-2267.1279	0.4336
-2446.4408	8.97	4000	-2795.6897	0.4307
-2944.8128	10.09	4500	-3294.3000	0.4279
-3416.905	11.21	5000	-3766.8943	0.4254
-3865.4817	12.33	5500	-4211.6885	0.4219
-4285.8215	13.45	6000	-4628.7329	0.4223
-4660.695	14.57	6500	-5017.9912	0.4184
-5032.002	15.7	7000	-5379.4785	0.4213
-5351.629	16.82	7500	-5713.4419	0.4186
-5659.1765	17.94	8000	-6019.7510	0.4195
-5957.58	19.06	8500	-6296.5054	0.4189
-6215.3305	20.18	9000	-6547.0381	0.4165
-6441.2955	21.3	9500	-6770.7163	0.4172
-6651.3045	22.42	10000	-6966.9087	0.4160
-6811.32	23.54	10500	-7135.6860	0.4155
-6964.2775	24.66	11000	-7277.1060	0.4171
-7115.7955	25.78	11500	-7390.9673	0.4156
-7187.0235	26.91	12000	-7477.7656	0.4151
-7259.9035	28.03	12500	-7535.0112	0.4148
-7302.289	29.15	13000	-7567.2393	0.4152

Framework versions

• Transformers 4.30.0.dev0
• Pytorch 2.0.1
• Datasets 2.12.1.dev0
• Tokenizers 0.13.3