libri-alpha-0.25-Temp-1-mse

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 16.7446
• Wer: 0.1212

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
122.8866	0.75	100	35.8349	0.1501
96.5187	1.49	200	31.6506	0.1453
88.8912	2.24	300	23.5288	0.1459
82.3478	2.99	400	20.1852	0.1386
74.9987	3.73	500	19.2132	0.1327
71.3929	4.48	600	18.6715	0.1320
67.5429	5.22	700	18.1638	0.1275
65.8387	5.97	800	17.9401	0.1274
62.2889	6.72	900	17.5666	0.1254
62.4649	7.46	1000	17.5202	0.1251
61.3213	8.21	1100	17.1763	0.1250
59.1153	8.96	1200	17.2310	0.1229
61.095	9.7	1300	17.0079	0.1228
61.0961	10.45	1400	16.8989	0.1214
59.0814	11.19	1500	16.9785	0.1195
58.3763	11.94	1600	17.0034	0.1198
57.9529	12.69	1700	16.8352	0.1203
56.6213	13.43	1800	16.8661	0.1206
56.4495	14.18	1900	16.8180	0.1216
54.3606	14.93	2000	16.7446	0.1212

Framework versions

• Transformers 4.25.1
• Pytorch 1.12.1
• Datasets 2.7.1
• Tokenizers 0.11.0