korean-aihub-learning-math-8batch

This model is a fine-tuned version of kresnik/wav2vec2-large-xlsr-korean on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1867
• Wer: 0.5315

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	1.0	20	33.1529	1.0
No log	2.0	40	28.0161	1.0
No log	3.0	60	8.7324	1.0
No log	4.0	80	4.9786	1.0
21.6269	5.0	100	4.5335	1.0
21.6269	6.0	120	4.4517	1.0
21.6269	7.0	140	4.4068	1.0
21.6269	8.0	160	4.3210	1.0
21.6269	9.0	180	4.0041	0.9932
4.1788	10.0	200	3.0921	0.9712
4.1788	11.0	220	2.1650	0.8603
4.1788	12.0	240	1.6135	0.7192
4.1788	13.0	260	1.3842	0.6466
4.1788	14.0	280	1.2872	0.5918
1.205	15.0	300	1.2234	0.5808
1.205	16.0	320	1.2694	0.6
1.205	17.0	340	1.2287	0.5575
1.205	18.0	360	1.1776	0.5877
1.205	19.0	380	1.2418	0.5671
0.2825	20.0	400	1.2469	0.5616
0.2825	21.0	420	1.2203	0.5425
0.2825	22.0	440	1.2270	0.5863
0.2825	23.0	460	1.1930	0.5548
0.2825	24.0	480	1.1242	0.5521
0.1831	25.0	500	1.2245	0.5575
0.1831	26.0	520	1.2276	0.5342
0.1831	27.0	540	1.1641	0.5205
0.1831	28.0	560	1.1727	0.5329
0.1831	29.0	580	1.1885	0.5534
0.14	30.0	600	1.1867	0.5315

Framework versions

• Transformers 4.21.0
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1