rohitp1/libri-alpha-0.75-Temp-1-att-take-3

libri-alpha-0.25-Temp-1-att

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 41.4611
• Wer: 0.1002

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.2
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
115.5029	0.45	400	41.6672	0.1012
127.311	0.9	800	40.5683	0.1009
118.505	1.35	1200	41.3756	0.1000
116.651	1.79	1600	41.1326	0.0994
121.336	2.24	2000	40.8370	0.0996
121.9217	2.69	2400	41.6449	0.0996
123.789	3.14	2800	39.4157	0.1003
120.0042	3.59	3200	42.2503	0.0997
125.504	4.04	3600	41.4611	0.1002

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1
• Datasets 2.6.1
• Tokenizers 0.13.1