--- license: apache-2.0 tags: - generated_from_trainer model-index: - name: korean-aihub-learning-3 results: [] --- # korean-aihub-learning-3 This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the None dataset. It achieves the following results on the evaluation set: - Loss: 2.2854 - Wer: 0.7921 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0003 - train_batch_size: 4 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 2 - total_train_batch_size: 8 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 500 - num_epochs: 100 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Wer | |:-------------:|:-----:|:----:|:---------------:|:------:| | No log | 0.99 | 35 | 45.5713 | 1.0 | | No log | 1.99 | 70 | 24.4376 | 1.0 | | 35.4145 | 2.99 | 105 | 18.3030 | 1.0 | | 35.4145 | 3.99 | 140 | 12.6702 | 1.0 | | 35.4145 | 4.99 | 175 | 7.4939 | 1.0 | | 11.687 | 5.99 | 210 | 4.9592 | 1.0 | | 11.687 | 6.99 | 245 | 4.6777 | 1.0 | | 11.687 | 7.99 | 280 | 4.6597 | 1.0 | | 4.8003 | 8.99 | 315 | 4.6777 | 1.0 | | 4.8003 | 9.99 | 350 | 4.7003 | 1.0 | | 4.8003 | 10.99 | 385 | 4.6129 | 1.0 | | 4.6383 | 11.99 | 420 | 4.6209 | 1.0 | | 4.6383 | 12.99 | 455 | 4.6035 | 1.0 | | 4.6383 | 13.99 | 490 | 4.6166 | 1.0 | | 4.577 | 14.99 | 525 | 4.6026 | 1.0 | | 4.577 | 15.99 | 560 | 4.5337 | 1.0 | | 4.577 | 16.99 | 595 | 4.5284 | 1.0 | | 4.5124 | 17.99 | 630 | 4.5710 | 1.0 | | 4.5124 | 18.99 | 665 | 4.5223 | 1.0 | | 4.3818 | 19.99 | 700 | 4.4472 | 1.0 | | 4.3818 | 20.99 | 735 | 4.4272 | 0.9977 | | 4.3818 | 21.99 | 770 | 4.4160 | 0.9977 | | 4.2796 | 22.99 | 805 | 4.3741 | 0.9988 | | 4.2796 | 23.99 | 840 | 4.3087 | 1.0 | | 4.2796 | 24.99 | 875 | 4.2336 | 1.0 | | 4.0489 | 25.99 | 910 | 4.1352 | 0.9988 | | 4.0489 | 26.99 | 945 | 4.0669 | 1.0 | | 4.0489 | 27.99 | 980 | 3.8551 | 0.9988 | | 3.6122 | 28.99 | 1015 | 3.6699 | 0.9919 | | 3.6122 | 29.99 | 1050 | 3.4580 | 0.9781 | | 3.6122 | 30.99 | 1085 | 3.1899 | 0.9434 | | 2.8886 | 31.99 | 1120 | 3.0746 | 0.9550 | | 2.8886 | 32.99 | 1155 | 2.8143 | 0.9353 | | 2.8886 | 33.99 | 1190 | 2.7004 | 0.9122 | | 2.0277 | 34.99 | 1225 | 2.5284 | 0.9076 | | 2.0277 | 35.99 | 1260 | 2.4677 | 0.8972 | | 2.0277 | 36.99 | 1295 | 2.3426 | 0.8568 | | 1.2486 | 37.99 | 1330 | 2.2456 | 0.8822 | | 1.2486 | 38.99 | 1365 | 2.3250 | 0.9238 | | 0.7572 | 39.99 | 1400 | 2.2832 | 0.8557 | | 0.7572 | 40.99 | 1435 | 2.2671 | 0.8406 | | 0.7572 | 41.99 | 1470 | 2.3070 | 0.8857 | | 0.4768 | 42.99 | 1505 | 2.2138 | 0.8476 | | 0.4768 | 43.99 | 1540 | 2.2034 | 0.8799 | | 0.4768 | 44.99 | 1575 | 2.2215 | 0.8487 | | 0.3362 | 45.99 | 1610 | 2.3416 | 0.8834 | | 0.3362 | 46.99 | 1645 | 2.3452 | 0.8383 | | 0.3362 | 47.99 | 1680 | 2.2449 | 0.8360 | | 0.257 | 48.99 | 1715 | 2.2249 | 0.8199 | | 0.257 | 49.99 | 1750 | 2.3455 | 0.8106 | | 0.257 | 50.99 | 1785 | 2.2537 | 0.8233 | | 0.2116 | 51.99 | 1820 | 2.2501 | 0.8025 | | 0.2116 | 52.99 | 1855 | 2.3180 | 0.8649 | | 0.2116 | 53.99 | 1890 | 2.1855 | 0.8106 | | 0.1787 | 54.99 | 1925 | 2.2140 | 0.8014 | | 0.1787 | 55.99 | 1960 | 2.3140 | 0.8453 | | 0.1787 | 56.99 | 1995 | 2.2140 | 0.8025 | | 0.1498 | 57.99 | 2030 | 2.3381 | 0.8314 | | 0.1498 | 58.99 | 2065 | 2.2591 | 0.8256 | | 0.1372 | 59.99 | 2100 | 2.2538 | 0.7979 | | 0.1372 | 60.99 | 2135 | 2.2052 | 0.7933 | | 0.1372 | 61.99 | 2170 | 2.2370 | 0.8233 | | 0.129 | 62.99 | 2205 | 2.2331 | 0.7898 | | 0.129 | 63.99 | 2240 | 2.3022 | 0.8002 | | 0.129 | 64.99 | 2275 | 2.3514 | 0.7956 | | 0.1075 | 65.99 | 2310 | 2.3303 | 0.8279 | | 0.1075 | 66.99 | 2345 | 2.2747 | 0.8025 | | 0.1075 | 67.99 | 2380 | 2.2899 | 0.8152 | | 0.0979 | 68.99 | 2415 | 2.3299 | 0.8164 | | 0.0979 | 69.99 | 2450 | 2.1819 | 0.7945 | | 0.0979 | 70.99 | 2485 | 2.2141 | 0.8222 | | 0.0973 | 71.99 | 2520 | 2.3683 | 0.8395 | | 0.0973 | 72.99 | 2555 | 2.2235 | 0.8199 | | 0.0973 | 73.99 | 2590 | 2.2474 | 0.8048 | | 0.0814 | 74.99 | 2625 | 2.3116 | 0.7968 | | 0.0814 | 75.99 | 2660 | 2.2494 | 0.7945 | | 0.0814 | 76.99 | 2695 | 2.2441 | 0.7968 | | 0.0745 | 77.99 | 2730 | 2.2489 | 0.7864 | | 0.0745 | 78.99 | 2765 | 2.2568 | 0.7921 | | 0.0741 | 79.99 | 2800 | 2.2598 | 0.7875 | | 0.0741 | 80.99 | 2835 | 2.3131 | 0.8002 | | 0.0741 | 81.99 | 2870 | 2.2719 | 0.7898 | | 0.0662 | 82.99 | 2905 | 2.2901 | 0.7875 | | 0.0662 | 83.99 | 2940 | 2.3092 | 0.7979 | | 0.0662 | 84.99 | 2975 | 2.3361 | 0.8048 | | 0.0556 | 85.99 | 3010 | 2.3308 | 0.8152 | | 0.0556 | 86.99 | 3045 | 2.3106 | 0.8164 | | 0.0556 | 87.99 | 3080 | 2.3363 | 0.8002 | | 0.0504 | 88.99 | 3115 | 2.3588 | 0.7910 | | 0.0504 | 89.99 | 3150 | 2.3528 | 0.7956 | | 0.0504 | 90.99 | 3185 | 2.3201 | 0.7794 | | 0.0496 | 91.99 | 3220 | 2.3386 | 0.7991 | | 0.0496 | 92.99 | 3255 | 2.3423 | 0.7956 | | 0.0496 | 93.99 | 3290 | 2.3312 | 0.7956 | | 0.0468 | 94.99 | 3325 | 2.3362 | 0.7968 | | 0.0468 | 95.99 | 3360 | 2.2962 | 0.7887 | | 0.0468 | 96.99 | 3395 | 2.2864 | 0.7841 | | 0.0475 | 97.99 | 3430 | 2.2870 | 0.7898 | | 0.0475 | 98.99 | 3465 | 2.2866 | 0.7898 | | 0.0411 | 99.99 | 3500 | 2.2854 | 0.7921 | ### Framework versions - Transformers 4.20.1 - Pytorch 1.12.0+cu113 - Datasets 2.3.2 - Tokenizers 0.12.1