hubert-base-libri-pruning-v1

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: -7634.6143
• Wer: 0.4250

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00015
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 3000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
-14.3776	1.12	500	-47.0374	0.5235
-106.5986	2.24	1000	-189.7623	0.4825
-291.2003	3.36	1500	-428.0651	0.4726
-567.486	4.48	2000	-761.7246	0.4628
-934.1564	5.61	2500	-1190.6781	0.4580
-1396.4066	6.73	3000	-1714.8335	0.4487
-1930.5184	7.85	3500	-2271.9685	0.4422
-2452.421	8.97	4000	-2802.5127	0.4418
-2953.0952	10.09	4500	-3304.3562	0.4386
-3427.7243	11.21	5000	-3779.5505	0.4357
-3879.0445	12.33	5500	-4227.1289	0.4339
-4302.3395	13.45	6000	-4647.1260	0.4311
-4680.295	14.57	6500	-5039.4692	0.4283
-5054.8855	15.7	7000	-5404.1592	0.4306
-5377.8435	16.82	7500	-5741.4082	0.4286
-5688.8665	17.94	8000	-6051.0688	0.4290
-5990.955	19.06	8500	-6333.1387	0.4284
-6252.404	20.18	9000	-6587.1460	0.4257
-6481.961	21.3	9500	-6814.2788	0.4268
-6695.5835	22.42	10000	-7013.8809	0.4256
-6859.0875	23.54	10500	-7185.9956	0.4255
-7015.5155	24.66	11000	-7330.6577	0.4271
-7170.5215	25.78	11500	-7447.6372	0.4256
-7244.894	26.91	12000	-7537.4111	0.4243
-7320.932	28.03	12500	-7599.7690	0.4248
-7366.4105	29.15	13000	-7634.6143	0.4250

Framework versions

• Transformers 4.30.0.dev0
• Pytorch 2.0.1
• Datasets 2.12.1.dev0
• Tokenizers 0.13.3