libri-alpha-0.75-Temp-1-mse

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 40.9574
• Wer: 0.1140

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
314.2983	0.75	100	73.1996	0.1509
254.3135	1.49	200	61.7104	0.1436
238.1834	2.24	300	53.4787	0.1426
212.833	2.99	400	51.2417	0.1390
202.8803	3.73	500	48.4136	0.1323
191.6817	4.48	600	45.9843	0.1288
178.3099	5.22	700	44.9262	0.1264
176.7728	5.97	800	44.0379	0.1287
176.0927	6.72	900	43.9972	0.1251
175.7854	7.46	1000	43.3336	0.1253
170.2058	8.21	1100	44.0316	0.1285
157.8171	8.96	1200	42.9391	0.1246
158.1243	9.7	1300	41.9089	0.1216
160.1174	10.45	1400	41.7764	0.1212
157.8857	11.19	1500	41.7168	0.1199
155.323	11.94	1600	41.5339	0.1205
149.4838	12.69	1700	41.1427	0.1189
148.4661	13.43	1800	41.1901	0.1181
147.2941	14.18	1900	41.1430	0.1184
151.4415	14.93	2000	41.1998	0.1188
142.9946	15.67	2100	41.3427	0.1180
143.8573	16.42	2200	40.9182	0.1167
144.2671	17.16	2300	41.0134	0.1158
145.2445	17.91	2400	40.8738	0.1170
148.3202	18.66	2500	40.6994	0.1166
138.238	19.4	2600	40.9133	0.1173
139.9513	20.15	2700	40.8462	0.1165
142.0799	20.9	2800	40.9641	0.1155
136.6577	21.64	2900	40.9828	0.1162
143.027	22.39	3000	41.1197	0.1152
138.5346	23.13	3100	41.0499	0.1153
132.8113	23.88	3200	41.1406	0.1155
136.3161	24.63	3300	41.0155	0.1147
138.4135	25.37	3400	40.8724	0.1147
132.4531	26.12	3500	40.8390	0.1143
131.0835	26.87	3600	41.0005	0.1143
132.015	27.61	3700	40.9447	0.1142
132.5831	28.36	3800	40.9810	0.1142
134.858	29.1	3900	40.9652	0.1140
135.4016	29.85	4000	40.9574	0.1140

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1
• Datasets 2.7.1
• Tokenizers 0.11.0