libri-finetune

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 349.4102
• Wer: 0.8141

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
13179.94	2.99	400	5612.7529	1.0
6948.38	5.97	800	1633.0823	0.9563
2144.6125	8.96	1200	578.3821	0.8487
1293.3905	11.94	1600	448.8980	0.8405
955.5785	14.93	2000	403.0979	0.8327
843.732	17.91	2400	374.1770	0.8220
739.1473	20.9	2800	360.7842	0.8179
651.852	23.88	3200	353.6803	0.8159
658.5995	26.87	3600	350.6870	0.8099
608.4441	29.85	4000	349.4102	0.8141

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1
• Datasets 2.6.1
• Tokenizers 0.13.1