korean-aihub-learning-math-16batch

This model is a fine-tuned version of kresnik/wav2vec2-large-xlsr-korean on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1497
• Wer: 0.5260

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	1.0	20	32.0718	1.0
No log	2.0	40	24.7403	1.0808
No log	3.0	60	5.8389	1.0
No log	4.0	80	4.8543	1.0
19.6583	5.0	100	4.4453	1.0
19.6583	6.0	120	4.3923	1.0
19.6583	7.0	140	4.2902	1.0
19.6583	8.0	160	3.9026	0.9959
19.6583	9.0	180	3.0616	0.9740
3.7358	10.0	200	2.2049	0.8534
3.7358	11.0	220	1.6666	0.7288
3.7358	12.0	240	1.4123	0.6603
3.7358	13.0	260	1.3113	0.6164
3.7358	14.0	280	1.2269	0.6356
0.8398	15.0	300	1.2349	0.5945
0.8398	16.0	320	1.1970	0.5658
0.8398	17.0	340	1.2144	0.5562
0.8398	18.0	360	1.2551	0.5658
0.8398	19.0	380	1.1971	0.5493
0.2649	20.0	400	1.1967	0.5247
0.2649	21.0	420	1.2796	0.5849
0.2649	22.0	440	1.2156	0.5521
0.2649	23.0	460	1.2118	0.5425
0.2649	24.0	480	1.1637	0.5384
0.1801	25.0	500	1.1846	0.5562
0.1801	26.0	520	1.1927	0.5534
0.1801	27.0	540	1.2015	0.5384
0.1801	28.0	560	1.2077	0.5397
0.1801	29.0	580	1.1554	0.5260
0.1364	30.0	600	1.1497	0.5260

Framework versions

• Transformers 4.22.0.dev0
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1