korean-aihub-learning-2

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9945
• Wer: 0.9533

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.99	35	46.3840	1.0
No log	1.99	70	26.0949	1.0
37.1581	2.99	105	19.0168	1.0
37.1581	3.99	140	13.3294	1.0
37.1581	4.99	175	7.9410	1.0
12.5054	5.99	210	5.0323	1.0
12.5054	6.99	245	4.6242	1.0
12.5054	7.99	280	4.6206	1.0
4.8394	8.99	315	4.5820	1.0
4.8394	9.99	350	4.5629	1.0
4.8394	10.99	385	4.5385	1.0
4.6489	11.99	420	4.5627	1.0
4.6489	12.99	455	4.5276	1.0
4.6489	13.99	490	4.5292	1.0
4.5654	14.99	525	4.5179	1.0
4.5654	15.99	560	4.4928	1.0
4.5654	16.99	595	4.4791	1.0
4.521	17.99	630	4.4649	1.0
4.521	18.99	665	4.4588	1.0
4.3529	19.99	700	4.3632	1.0
4.3529	20.99	735	4.2990	1.0
4.3529	21.99	770	4.2326	0.9988
4.1301	22.99	805	4.0843	1.0
4.1301	23.99	840	3.9784	0.9975
4.1301	24.99	875	3.7876	1.0
3.7047	25.99	910	3.6109	0.9988
3.7047	26.99	945	3.4049	0.9828
3.7047	27.99	980	3.1913	0.9606
3.006	28.99	1015	3.0567	0.9508
3.006	29.99	1050	2.9945	0.9533

Framework versions

• Transformers 4.20.1
• Pytorch 1.12.0+cu113
• Datasets 2.3.2
• Tokenizers 0.12.1