no_distil_librispeech_100_clean_6_attention

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2333.5361
• Wer: 1.1878

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
8374.3469	0.75	100	2762.3215	1.0
4303.0578	1.49	200	2503.2461	1.0
4307.1169	2.24	300	2498.8477	1.0
4236.7513	2.99	400	2489.2173	1.0
4242.8606	3.73	500	2405.0710	1.0801
4132.4353	4.48	600	2333.5361	1.1878

Framework versions

• Transformers 4.24.0
• Pytorch 1.12.1
• Datasets 2.7.0
• Tokenizers 0.11.0