libri-alpha-1-Temp-1-mse

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 53.0700
• Wer: 0.1137

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
406.6609	0.75	100	92.6621	0.1514
332.8654	1.49	200	75.1632	0.1429
300.7975	2.24	300	69.9229	0.1413
288.1078	2.99	400	66.7404	0.1361
262.5671	3.73	500	60.1363	0.1314
246.594	4.48	600	59.8725	0.1289
232.3433	5.22	700	59.7349	0.1274
232.5868	5.97	800	58.1594	0.1266
225.2652	6.72	900	57.0715	0.1248
222.3506	7.46	1000	56.8740	0.1253
210.8775	8.21	1100	55.6375	0.1235
210.2934	8.96	1200	54.7285	0.1223
212.0601	9.7	1300	54.6263	0.1224
203.2785	10.45	1400	54.7651	0.1241
200.5735	11.19	1500	54.7737	0.1238
199.7971	11.94	1600	54.7460	0.1214
198.5452	12.69	1700	54.5335	0.1222
192.6993	13.43	1800	54.3382	0.1209
195.9604	14.18	1900	54.0138	0.1188
190.6209	14.93	2000	55.2720	0.1219
197.205	15.67	2100	54.4430	0.1193
182.2428	16.42	2200	53.7938	0.1195
183.4877	17.16	2300	53.0349	0.1151
178.3634	17.91	2400	53.0706	0.1157
184.3548	18.66	2500	53.0254	0.1158
184.6175	19.4	2600	53.2929	0.1159
190.0462	20.15	2700	52.8959	0.1157
179.8124	20.9	2800	53.0652	0.1147
178.4741	21.64	2900	53.5907	0.1155
175.1646	22.39	3000	53.5027	0.1155
172.7706	23.13	3100	53.3478	0.1146
182.5294	23.88	3200	53.2863	0.1152
183.6617	24.63	3300	53.3587	0.1146
180.0207	25.37	3400	53.2285	0.1140
180.7319	26.12	3500	53.1544	0.1140
171.5148	26.87	3600	53.0734	0.1144
177.4159	27.61	3700	53.1536	0.1138
168.6823	28.36	3800	53.1117	0.1138
176.7611	29.1	3900	53.0962	0.1135
176.1258	29.85	4000	53.0700	0.1137

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1
• Datasets 2.7.1
• Tokenizers 0.11.0