korean-aihub-learning-math-1-test

This model is a fine-tuned version of kresnik/wav2vec2-large-xlsr-korean on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2537
• Wer: 0.4765

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	1.0	35	29.8031	1.0
No log	2.0	70	5.7158	1.0
19.8789	3.0	105	4.5005	1.0
19.8789	4.0	140	4.3677	0.9984
19.8789	5.0	175	3.8013	0.9882
3.9785	6.0	210	2.4132	0.8730
3.9785	7.0	245	1.5867	0.7045
3.9785	8.0	280	1.3179	0.6082
1.2266	9.0	315	1.2431	0.6066
1.2266	10.0	350	1.1791	0.5384
1.2266	11.0	385	1.0994	0.5298
0.3916	12.0	420	1.1552	0.5196
0.3916	13.0	455	1.1495	0.5486
0.3916	14.0	490	1.1340	0.5290
0.2488	15.0	525	1.2208	0.5525
0.2488	16.0	560	1.1682	0.5024
0.2488	17.0	595	1.1479	0.5008
0.1907	18.0	630	1.1735	0.4882
0.1907	19.0	665	1.2302	0.4914
0.1461	20.0	700	1.2497	0.4890
0.1461	21.0	735	1.2434	0.4914
0.1461	22.0	770	1.2031	0.5031
0.1147	23.0	805	1.2451	0.4976
0.1147	24.0	840	1.2746	0.4937
0.1147	25.0	875	1.2405	0.4828
0.0892	26.0	910	1.2228	0.4929
0.0892	27.0	945	1.2642	0.4898
0.0892	28.0	980	1.2586	0.4843
0.0709	29.0	1015	1.2518	0.4788
0.0709	30.0	1050	1.2537	0.4765

Framework versions

• Transformers 4.20.1
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1