--- license: apache-2.0 tags: - generated_from_trainer model-index: - name: wav2vec2-large-xls-r-300m-marathi results: [] --- # wav2vec2-large-xls-r-300m-marathi This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.5656 - Wer: 0.2156 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0003 - train_batch_size: 4 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 2 - total_train_batch_size: 8 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 500 - num_epochs: 20 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Wer | |:-------------:|:-----:|:------:|:---------------:|:------:| | 5.7504 | 0.04 | 400 | 2.5012 | 1.0481 | | 1.2669 | 0.08 | 800 | 0.7129 | 0.5178 | | 0.8259 | 0.12 | 1200 | 0.6041 | 0.4784 | | 0.6682 | 0.16 | 1600 | 0.4670 | 0.3708 | | 0.6116 | 0.2 | 2000 | 0.4416 | 0.3566 | | 0.5631 | 0.24 | 2400 | 0.3988 | 0.3241 | | 0.509 | 0.28 | 2800 | 0.3597 | 0.3061 | | 0.4889 | 0.32 | 3200 | 0.3639 | 0.3094 | | 0.4637 | 0.36 | 3600 | 0.3724 | 0.3105 | | 0.4447 | 0.4 | 4000 | 0.3508 | 0.2845 | | 0.4258 | 0.44 | 4400 | 0.3143 | 0.2665 | | 0.4094 | 0.48 | 4800 | 0.3533 | 0.2774 | | 0.3924 | 0.52 | 5200 | 0.3435 | 0.2735 | | 0.3682 | 0.56 | 5600 | 0.3208 | 0.2592 | | 0.383 | 0.6 | 6000 | 0.3226 | 0.2640 | | 0.3707 | 0.64 | 6400 | 0.3128 | 0.2477 | | 0.3459 | 0.68 | 6800 | 0.3261 | 0.2512 | | 0.3373 | 0.73 | 7200 | 0.3390 | 0.2681 | | 0.3311 | 0.77 | 7600 | 0.3037 | 0.2474 | | 0.3273 | 0.81 | 8000 | 0.3042 | 0.2518 | | 0.3131 | 0.85 | 8400 | 0.2971 | 0.2434 | | 0.2997 | 0.89 | 8800 | 0.3161 | 0.2382 | | 0.2955 | 0.93 | 9200 | 0.3142 | 0.2409 | | 0.2962 | 0.97 | 9600 | 0.3063 | 0.2446 | | 0.2747 | 1.01 | 10000 | 0.3432 | 0.2447 | | 0.2468 | 1.05 | 10400 | 0.3165 | 0.2385 | | 0.2455 | 1.09 | 10800 | 0.3021 | 0.2394 | | 0.2421 | 1.13 | 11200 | 0.3205 | 0.2350 | | 0.2429 | 1.17 | 11600 | 0.3104 | 0.2350 | | 0.2406 | 1.21 | 12000 | 0.3060 | 0.2350 | | 0.2508 | 1.25 | 12400 | 0.2988 | 0.2340 | | 0.2263 | 1.29 | 12800 | 0.3221 | 0.2307 | | 0.2211 | 1.33 | 13200 | 0.3098 | 0.2332 | | 0.2198 | 1.37 | 13600 | 0.2946 | 0.2216 | | 0.2197 | 1.41 | 14000 | 0.2966 | 0.2265 | | 0.2188 | 1.45 | 14400 | 0.2932 | 0.2272 | | 0.2165 | 1.49 | 14800 | 0.3272 | 0.2356 | | 0.2147 | 1.53 | 15200 | 0.3107 | 0.2257 | | 0.2063 | 1.57 | 15600 | 0.3122 | 0.2280 | | 0.2034 | 1.61 | 16000 | 0.3374 | 0.2384 | | 0.1965 | 1.65 | 16400 | 0.3152 | 0.2294 | | 0.1854 | 1.69 | 16800 | 0.3117 | 0.2284 | | 0.1869 | 1.73 | 17200 | 0.3207 | 0.2242 | | 0.2088 | 1.77 | 17600 | 0.3166 | 0.2256 | | 0.189 | 1.81 | 18000 | 0.3317 | 0.2231 | | 0.1899 | 1.85 | 18400 | 0.3062 | 0.2232 | | 0.1891 | 1.89 | 18800 | 0.3241 | 0.2284 | | 0.1834 | 1.93 | 19200 | 0.3156 | 0.2292 | | 0.1788 | 1.97 | 19600 | 0.3276 | 0.2237 | | 0.1738 | 2.01 | 20000 | 0.3364 | 0.2146 | | 0.1412 | 2.05 | 20400 | 0.3254 | 0.2190 | | 0.1539 | 2.09 | 20800 | 0.3430 | 0.2246 | | 0.1413 | 2.14 | 21200 | 0.3386 | 0.2262 | | 0.1443 | 2.18 | 21600 | 0.3512 | 0.2266 | | 0.142 | 2.22 | 22000 | 0.3271 | 0.2148 | | 0.1546 | 2.26 | 22400 | 0.3290 | 0.2185 | | 0.1357 | 2.3 | 22800 | 0.3599 | 0.2200 | | 0.142 | 2.34 | 23200 | 0.3348 | 0.2182 | | 0.1433 | 2.38 | 23600 | 0.3427 | 0.2196 | | 0.1332 | 2.42 | 24000 | 0.3294 | 0.2193 | | 0.1436 | 2.46 | 24400 | 0.3370 | 0.2228 | | 0.134 | 2.5 | 24800 | 0.3480 | 0.2280 | | 0.132 | 2.54 | 25200 | 0.3411 | 0.2230 | | 0.145 | 2.58 | 25600 | 0.3305 | 0.2205 | | 0.1411 | 2.62 | 26000 | 0.3450 | 0.2200 | | 0.1269 | 2.66 | 26400 | 0.3412 | 0.2199 | | 0.1349 | 2.7 | 26800 | 0.3445 | 0.2203 | | 0.1313 | 2.74 | 27200 | 0.3406 | 0.2239 | | 0.1311 | 2.78 | 27600 | 0.3501 | 0.2257 | | 0.135 | 2.82 | 28000 | 0.3329 | 0.2189 | | 0.1295 | 2.86 | 28400 | 0.3407 | 0.2183 | | 0.1236 | 2.9 | 28800 | 0.3503 | 0.2166 | | 0.1248 | 2.94 | 29200 | 0.3522 | 0.2237 | | 0.122 | 2.98 | 29600 | 0.3336 | 0.2235 | | 0.1158 | 3.02 | 30000 | 0.3429 | 0.2174 | | 0.0999 | 3.06 | 30400 | 0.3650 | 0.2212 | | 0.097 | 3.1 | 30800 | 0.3760 | 0.2231 | | 0.0969 | 3.14 | 31200 | 0.3697 | 0.2231 | | 0.0969 | 3.18 | 31600 | 0.3720 | 0.2223 | | 0.0964 | 3.22 | 32000 | 0.3638 | 0.2192 | | 0.0954 | 3.26 | 32400 | 0.3735 | 0.2211 | | 0.0982 | 3.3 | 32800 | 0.3579 | 0.2252 | | 0.1007 | 3.34 | 33200 | 0.3505 | 0.2200 | | 0.0933 | 3.38 | 33600 | 0.3614 | 0.2318 | | 0.0948 | 3.42 | 34000 | 0.3550 | 0.2203 | | 0.0989 | 3.46 | 34400 | 0.3729 | 0.2245 | | 0.1005 | 3.5 | 34800 | 0.3563 | 0.2267 | | 0.0886 | 3.55 | 35200 | 0.3671 | 0.2217 | | 0.0858 | 3.59 | 35600 | 0.3764 | 0.2169 | | 0.0908 | 3.63 | 36000 | 0.3724 | 0.2226 | | 0.0908 | 3.67 | 36400 | 0.3784 | 0.2288 | | 0.0852 | 3.71 | 36800 | 0.3607 | 0.2193 | | 0.0919 | 3.75 | 37200 | 0.3630 | 0.2192 | | 0.0863 | 3.79 | 37600 | 0.3750 | 0.2258 | | 0.0873 | 3.83 | 38000 | 0.3808 | 0.2170 | | 0.0875 | 3.87 | 38400 | 0.3692 | 0.2225 | | 0.0871 | 3.91 | 38800 | 0.3954 | 0.2249 | | 0.0866 | 3.95 | 39200 | 0.3828 | 0.2204 | | 0.0883 | 3.99 | 39600 | 0.3830 | 0.2249 | | 0.0763 | 4.03 | 40000 | 0.3855 | 0.2177 | | 0.0613 | 4.07 | 40400 | 0.4117 | 0.2264 | | 0.0627 | 4.11 | 40800 | 0.3775 | 0.2215 | | 0.0709 | 4.15 | 41200 | 0.3965 | 0.2227 | | 0.0724 | 4.19 | 41600 | 0.3789 | 0.2141 | | 0.0707 | 4.23 | 42000 | 0.3834 | 0.2193 | | 0.0702 | 4.27 | 42400 | 0.3802 | 0.2216 | | 0.0646 | 4.31 | 42800 | 0.3858 | 0.2141 | | 0.0717 | 4.35 | 43200 | 0.3930 | 0.2184 | | 0.0651 | 4.39 | 43600 | 0.3901 | 0.2170 | | 0.0676 | 4.43 | 44000 | 0.4074 | 0.2158 | | 0.0677 | 4.47 | 44400 | 0.3999 | 0.2164 | | 0.0637 | 4.51 | 44800 | 0.3937 | 0.2241 | | 0.0655 | 4.55 | 45200 | 0.4010 | 0.2248 | | 0.0697 | 4.59 | 45600 | 0.4060 | 0.2236 | | 0.0644 | 4.63 | 46000 | 0.4144 | 0.2179 | | 0.0713 | 4.67 | 46400 | 0.3829 | 0.2172 | | 0.066 | 4.71 | 46800 | 0.4152 | 0.2193 | | 0.0667 | 4.75 | 47200 | 0.4015 | 0.2175 | | 0.0621 | 4.79 | 47600 | 0.4058 | 0.2191 | | 0.0664 | 4.83 | 48000 | 0.3898 | 0.2194 | | 0.067 | 4.87 | 48400 | 0.3966 | 0.2180 | | 0.0657 | 4.91 | 48800 | 0.4060 | 0.2156 | | 0.0612 | 4.96 | 49200 | 0.3980 | 0.2218 | | 0.0639 | 5.0 | 49600 | 0.3828 | 0.2189 | | 0.0491 | 5.04 | 50000 | 0.3902 | 0.2128 | | 0.0437 | 5.08 | 50400 | 0.3901 | 0.2159 | | 0.0449 | 5.12 | 50800 | 0.3821 | 0.2176 | | 0.0491 | 5.16 | 51200 | 0.3931 | 0.2194 | | 0.0527 | 5.2 | 51600 | 0.4034 | 0.2186 | | 0.0468 | 5.24 | 52000 | 0.4107 | 0.2153 | | 0.047 | 5.28 | 52400 | 0.4234 | 0.2215 | | 0.0516 | 5.32 | 52800 | 0.3908 | 0.2150 | | 0.0495 | 5.36 | 53200 | 0.3987 | 0.2124 | | 0.049 | 5.4 | 53600 | 0.4003 | 0.2140 | | 0.0466 | 5.44 | 54000 | 0.4234 | 0.2176 | | 0.0458 | 5.48 | 54400 | 0.4027 | 0.2195 | | 0.0471 | 5.52 | 54800 | 0.4244 | 0.2235 | | 0.0491 | 5.56 | 55200 | 0.4246 | 0.2223 | | 0.0465 | 5.6 | 55600 | 0.4190 | 0.2233 | | 0.0481 | 5.64 | 56000 | 0.4031 | 0.2181 | | 0.0445 | 5.68 | 56400 | 0.4024 | 0.2171 | | 0.0476 | 5.72 | 56800 | 0.4303 | 0.2230 | | 0.0509 | 5.76 | 57200 | 0.4229 | 0.2232 | | 0.0499 | 5.8 | 57600 | 0.4248 | 0.2237 | | 0.0391 | 5.84 | 58000 | 0.4337 | 0.2232 | | 0.043 | 5.88 | 58400 | 0.4243 | 0.2191 | | 0.0421 | 5.92 | 58800 | 0.4376 | 0.2265 | | 0.042 | 5.96 | 59200 | 0.4246 | 0.2234 | | 0.0394 | 6.0 | 59600 | 0.4444 | 0.2205 | | 0.0354 | 6.04 | 60000 | 0.4144 | 0.2134 | | 0.0319 | 6.08 | 60400 | 0.4246 | 0.2150 | | 0.0325 | 6.12 | 60800 | 0.4540 | 0.2240 | | 0.0324 | 6.16 | 61200 | 0.4582 | 0.2201 | | 0.0333 | 6.2 | 61600 | 0.4268 | 0.2199 | | 0.0358 | 6.24 | 62000 | 0.4560 | 0.2158 | | 0.0327 | 6.28 | 62400 | 0.4448 | 0.2175 | | 0.0329 | 6.32 | 62800 | 0.4369 | 0.2181 | | 0.0304 | 6.37 | 63200 | 0.4362 | 0.2194 | | 0.0366 | 6.41 | 63600 | 0.4401 | 0.2174 | | 0.0357 | 6.45 | 64000 | 0.4440 | 0.2189 | | 0.0313 | 6.49 | 64400 | 0.4424 | 0.2128 | | 0.0354 | 6.53 | 64800 | 0.4318 | 0.2182 | | 0.0295 | 6.57 | 65200 | 0.4497 | 0.2255 | | 0.0303 | 6.61 | 65600 | 0.4448 | 0.2117 | | 0.03 | 6.65 | 66000 | 0.4285 | 0.2104 | | 0.0325 | 6.69 | 66400 | 0.4422 | 0.2187 | | 0.0302 | 6.73 | 66800 | 0.4419 | 0.2150 | | 0.0335 | 6.77 | 67200 | 0.4264 | 0.2201 | | 0.0306 | 6.81 | 67600 | 0.4614 | 0.2215 | | 0.0321 | 6.85 | 68000 | 0.4526 | 0.2195 | | 0.0349 | 6.89 | 68400 | 0.4607 | 0.2202 | | 0.0277 | 6.93 | 68800 | 0.4634 | 0.2175 | | 0.0346 | 6.97 | 69200 | 0.4348 | 0.2188 | | 0.0332 | 7.01 | 69600 | 0.4605 | 0.2177 | | 0.0271 | 7.05 | 70000 | 0.4501 | 0.2177 | | 0.0244 | 7.09 | 70400 | 0.4722 | 0.2169 | | 0.0258 | 7.13 | 70800 | 0.4501 | 0.2193 | | 0.0224 | 7.17 | 71200 | 0.4730 | 0.2175 | | 0.0254 | 7.21 | 71600 | 0.4693 | 0.2223 | | 0.0257 | 7.25 | 72000 | 0.4684 | 0.2231 | | 0.0206 | 7.29 | 72400 | 0.4526 | 0.2197 | | 0.0276 | 7.33 | 72800 | 0.4569 | 0.2158 | | 0.0212 | 7.37 | 73200 | 0.4873 | 0.2201 | | 0.0233 | 7.41 | 73600 | 0.4592 | 0.2190 | | 0.0231 | 7.45 | 74000 | 0.4647 | 0.2209 | | 0.0202 | 7.49 | 74400 | 0.4766 | 0.2165 | | 0.0218 | 7.53 | 74800 | 0.4735 | 0.2155 | | 0.0198 | 7.57 | 75200 | 0.4477 | 0.2153 | | 0.0222 | 7.61 | 75600 | 0.4498 | 0.2091 | | 0.0228 | 7.65 | 76000 | 0.4563 | 0.2151 | | 0.0214 | 7.69 | 76400 | 0.4636 | 0.2162 | | 0.021 | 7.73 | 76800 | 0.4467 | 0.2146 | | 0.0224 | 7.78 | 77200 | 0.4580 | 0.2171 | | 0.0172 | 7.82 | 77600 | 0.4678 | 0.2141 | | 0.0226 | 7.86 | 78000 | 0.4567 | 0.2133 | | 0.0197 | 7.9 | 78400 | 0.4792 | 0.2162 | | 0.0186 | 7.94 | 78800 | 0.4555 | 0.2174 | | 0.0191 | 7.98 | 79200 | 0.4593 | 0.2164 | | 0.0175 | 8.02 | 79600 | 0.4725 | 0.2181 | | 0.0136 | 8.06 | 80000 | 0.4716 | 0.2148 | | 0.0155 | 8.1 | 80400 | 0.4723 | 0.2143 | | 0.016 | 8.14 | 80800 | 0.4632 | 0.2091 | | 0.016 | 8.18 | 81200 | 0.4853 | 0.2153 | | 0.0162 | 8.22 | 81600 | 0.4717 | 0.2176 | | 0.0162 | 8.26 | 82000 | 0.4638 | 0.2179 | | 0.013 | 8.3 | 82400 | 0.4661 | 0.2153 | | 0.0151 | 8.34 | 82800 | 0.4745 | 0.2144 | | 0.0147 | 8.38 | 83200 | 0.4694 | 0.2143 | | 0.0124 | 8.42 | 83600 | 0.4776 | 0.2156 | | 0.0129 | 8.46 | 84000 | 0.4754 | 0.2156 | | 0.0147 | 8.5 | 84400 | 0.4770 | 0.2193 | | 0.0143 | 8.54 | 84800 | 0.4585 | 0.2167 | | 0.0135 | 8.58 | 85200 | 0.4832 | 0.2215 | | 0.013 | 8.62 | 85600 | 0.4736 | 0.2202 | | 0.0124 | 8.66 | 86000 | 0.4861 | 0.2203 | | 0.0123 | 8.7 | 86400 | 0.4794 | 0.2189 | | 0.0162 | 8.74 | 86800 | 0.4827 | 0.2186 | | 0.0132 | 8.78 | 87200 | 0.4688 | 0.2163 | | 0.0119 | 8.82 | 87600 | 0.4682 | 0.2157 | | 0.012 | 8.86 | 88000 | 0.4611 | 0.2147 | | 0.0113 | 8.9 | 88400 | 0.4841 | 0.2152 | | 0.0138 | 8.94 | 88800 | 0.4841 | 0.2160 | | 0.0128 | 8.98 | 89200 | 0.4901 | 0.2170 | | 0.0099 | 9.02 | 89600 | 0.4744 | 0.2141 | | 0.0087 | 9.06 | 90000 | 0.4894 | 0.2149 | | 0.0083 | 9.1 | 90400 | 0.4797 | 0.2131 | | 0.0085 | 9.14 | 90800 | 0.4842 | 0.2138 | | 0.0104 | 9.19 | 91200 | 0.4754 | 0.2135 | | 0.0091 | 9.23 | 91600 | 0.4813 | 0.2140 | | 0.0108 | 9.27 | 92000 | 0.4829 | 0.2139 | | 0.0121 | 9.31 | 92400 | 0.4905 | 0.2139 | | 0.008 | 9.35 | 92800 | 0.4940 | 0.2147 | | 0.0114 | 9.39 | 93200 | 0.4925 | 0.2148 | | 0.008 | 9.43 | 93600 | 0.5015 | 0.2149 | | 0.0089 | 9.47 | 94000 | 0.5049 | 0.2145 | | 0.0088 | 9.51 | 94400 | 0.5087 | 0.2156 | | 0.0076 | 9.55 | 94800 | 0.5092 | 0.2165 | | 0.0092 | 9.59 | 95200 | 0.5097 | 0.2151 | | 0.0076 | 9.63 | 95600 | 0.5042 | 0.2141 | | 0.0084 | 9.67 | 96000 | 0.5060 | 0.2142 | | 0.0099 | 9.71 | 96400 | 0.5066 | 0.2127 | | 0.0086 | 9.75 | 96800 | 0.5057 | 0.2123 | | 0.0069 | 9.79 | 97200 | 0.5075 | 0.2128 | | 0.0071 | 9.83 | 97600 | 0.5045 | 0.2122 | | 0.0073 | 9.87 | 98000 | 0.5064 | 0.2119 | | 0.0073 | 9.91 | 98400 | 0.5057 | 0.2120 | | 0.0091 | 9.95 | 98800 | 0.5063 | 0.2116 | | 0.0077 | 9.99 | 99200 | 0.5054 | 0.2115 | | 0.0226 | 10.03 | 99600 | 0.4884 | 0.2309 | | 0.0353 | 10.07 | 100000 | 0.4751 | 0.2313 | | 0.0314 | 10.11 | 100400 | 0.5053 | 0.2347 | | 0.0308 | 10.15 | 100800 | 0.4879 | 0.2306 | | 0.0373 | 10.19 | 101200 | 0.4946 | 0.2335 | | 0.0324 | 10.23 | 101600 | 0.5066 | 0.2319 | | 0.0306 | 10.27 | 102000 | 0.4822 | 0.2267 | | 0.0354 | 10.31 | 102400 | 0.4749 | 0.2219 | | 0.0294 | 10.35 | 102800 | 0.5108 | 0.2327 | | 0.0321 | 10.39 | 103200 | 0.4560 | 0.2253 | | 0.0334 | 10.43 | 103600 | 0.4857 | 0.2286 | | 0.0347 | 10.47 | 104000 | 0.4572 | 0.2310 | | 0.0289 | 10.51 | 104400 | 0.4465 | 0.2241 | | 0.032 | 10.55 | 104800 | 0.4580 | 0.2286 | | 0.0299 | 10.6 | 105200 | 0.4726 | 0.2257 | | 0.0324 | 10.64 | 105600 | 0.4799 | 0.2303 | | 0.0349 | 10.68 | 106000 | 0.4662 | 0.2281 | | 0.0306 | 10.72 | 106400 | 0.5001 | 0.2345 | | 0.0303 | 10.76 | 106800 | 0.4906 | 0.2352 | | 0.033 | 10.8 | 107200 | 0.4656 | 0.2268 | | 0.0341 | 10.84 | 107600 | 0.4706 | 0.2264 | | 0.0316 | 10.88 | 108000 | 0.4634 | 0.2213 | | 0.0302 | 10.92 | 108400 | 0.4661 | 0.2232 | | 0.0319 | 10.96 | 108800 | 0.4725 | 0.2302 | | 0.033 | 11.0 | 109200 | 0.4672 | 0.2314 | | 0.0273 | 11.04 | 109600 | 0.4667 | 0.2285 | | 0.0275 | 11.08 | 110000 | 0.4699 | 0.2260 | | 0.0265 | 11.12 | 110400 | 0.4715 | 0.2242 | | 0.0313 | 11.16 | 110800 | 0.4758 | 0.2339 | | 0.0275 | 11.2 | 111200 | 0.4691 | 0.2304 | | 0.0285 | 11.24 | 111600 | 0.4910 | 0.2298 | | 0.0251 | 11.28 | 112000 | 0.4911 | 0.2241 | | 0.0281 | 11.32 | 112400 | 0.4614 | 0.2282 | | 0.0324 | 11.36 | 112800 | 0.4898 | 0.2324 | | 0.0279 | 11.4 | 113200 | 0.4967 | 0.2275 | | 0.027 | 11.44 | 113600 | 0.5056 | 0.2293 | | 0.0277 | 11.48 | 114000 | 0.4847 | 0.2254 | | 0.031 | 11.52 | 114400 | 0.4646 | 0.2236 | | 0.0326 | 11.56 | 114800 | 0.4747 | 0.2280 | | 0.0284 | 11.6 | 115200 | 0.4782 | 0.2241 | | 0.026 | 11.64 | 115600 | 0.4842 | 0.2209 | | 0.0248 | 11.68 | 116000 | 0.5005 | 0.2268 | | 0.0255 | 11.72 | 116400 | 0.4809 | 0.2256 | | 0.0261 | 11.76 | 116800 | 0.4723 | 0.2293 | | 0.0248 | 11.8 | 117200 | 0.4681 | 0.2328 | | 0.0249 | 11.84 | 117600 | 0.5021 | 0.2301 | | 0.0274 | 11.88 | 118000 | 0.4819 | 0.2345 | | 0.0263 | 11.92 | 118400 | 0.4903 | 0.2328 | | 0.0273 | 11.96 | 118800 | 0.4871 | 0.2307 | | 0.029 | 12.01 | 119200 | 0.4827 | 0.2288 | | 0.0213 | 12.05 | 119600 | 0.4805 | 0.2227 | | 0.0205 | 12.09 | 120000 | 0.4879 | 0.2295 | | 0.0206 | 12.13 | 120400 | 0.5054 | 0.2280 | | 0.0218 | 12.17 | 120800 | 0.4952 | 0.2319 | | 0.0232 | 12.21 | 121200 | 0.5364 | 0.2409 | | 0.0229 | 12.25 | 121600 | 0.4965 | 0.2308 | | 0.0284 | 12.29 | 122000 | 0.4749 | 0.2235 | | 0.0209 | 12.33 | 122400 | 0.5203 | 0.2244 | | 0.0282 | 12.37 | 122800 | 0.4933 | 0.2292 | | 0.021 | 12.41 | 123200 | 0.4915 | 0.2315 | | 0.0257 | 12.45 | 123600 | 0.5016 | 0.2254 | | 0.0262 | 12.49 | 124000 | 0.5045 | 0.2262 | | 0.0235 | 12.53 | 124400 | 0.4914 | 0.2186 | | 0.0218 | 12.57 | 124800 | 0.5212 | 0.2321 | | 0.0233 | 12.61 | 125200 | 0.4886 | 0.2271 | | 0.0217 | 12.65 | 125600 | 0.4841 | 0.2245 | | 0.0221 | 12.69 | 126000 | 0.5069 | 0.2236 | | 0.0208 | 12.73 | 126400 | 0.5107 | 0.2249 | | 0.0218 | 12.77 | 126800 | 0.4972 | 0.2269 | | 0.022 | 12.81 | 127200 | 0.5061 | 0.2330 | | 0.0205 | 12.85 | 127600 | 0.5061 | 0.2243 | | 0.0232 | 12.89 | 128000 | 0.4948 | 0.2268 | | 0.0246 | 12.93 | 128400 | 0.4933 | 0.2275 | | 0.02 | 12.97 | 128800 | 0.4996 | 0.2311 | | 0.0218 | 13.01 | 129200 | 0.5027 | 0.2273 | | 0.0189 | 13.05 | 129600 | 0.5151 | 0.2266 | | 0.0179 | 13.09 | 130000 | 0.5245 | 0.2329 | | 0.02 | 13.13 | 130400 | 0.4942 | 0.2332 | | 0.0219 | 13.17 | 130800 | 0.5184 | 0.2265 | | 0.0204 | 13.21 | 131200 | 0.4930 | 0.2222 | | 0.0182 | 13.25 | 131600 | 0.5008 | 0.2239 | | 0.0176 | 13.29 | 132000 | 0.4822 | 0.2189 | | 0.0194 | 13.33 | 132400 | 0.4885 | 0.2200 | | 0.0196 | 13.37 | 132800 | 0.4844 | 0.2245 | | 0.0162 | 13.42 | 133200 | 0.4992 | 0.2212 | | 0.0182 | 13.46 | 133600 | 0.4969 | 0.2227 | | 0.0211 | 13.5 | 134000 | 0.4803 | 0.2179 | | 0.0192 | 13.54 | 134400 | 0.4986 | 0.2220 | | 0.0176 | 13.58 | 134800 | 0.4841 | 0.2173 | | 0.0164 | 13.62 | 135200 | 0.5040 | 0.2255 | | 0.0161 | 13.66 | 135600 | 0.5113 | 0.2208 | | 0.021 | 13.7 | 136000 | 0.4884 | 0.2264 | | 0.0174 | 13.74 | 136400 | 0.5125 | 0.2246 | | 0.0181 | 13.78 | 136800 | 0.5072 | 0.2276 | | 0.0167 | 13.82 | 137200 | 0.5052 | 0.2245 | | 0.0181 | 13.86 | 137600 | 0.4920 | 0.2252 | | 0.0168 | 13.9 | 138000 | 0.5125 | 0.2254 | | 0.0142 | 13.94 | 138400 | 0.5220 | 0.2272 | | 0.0144 | 13.98 | 138800 | 0.5281 | 0.2281 | | 0.0165 | 14.02 | 139200 | 0.5131 | 0.2252 | | 0.0193 | 14.06 | 139600 | 0.4828 | 0.2211 | | 0.013 | 14.1 | 140000 | 0.5067 | 0.2199 | | 0.0133 | 14.14 | 140400 | 0.5213 | 0.2222 | | 0.0168 | 14.18 | 140800 | 0.4974 | 0.2212 | | 0.0159 | 14.22 | 141200 | 0.5002 | 0.2199 | | 0.0145 | 14.26 | 141600 | 0.4941 | 0.2255 | | 0.0133 | 14.3 | 142000 | 0.5112 | 0.2299 | | 0.0147 | 14.34 | 142400 | 0.4975 | 0.2265 | | 0.0149 | 14.38 | 142800 | 0.5090 | 0.2246 | | 0.0148 | 14.42 | 143200 | 0.5004 | 0.2248 | | 0.016 | 14.46 | 143600 | 0.4839 | 0.2213 | | 0.0171 | 14.5 | 144000 | 0.4889 | 0.2267 | | 0.0123 | 14.54 | 144400 | 0.5289 | 0.2252 | | 0.0136 | 14.58 | 144800 | 0.4981 | 0.2144 | | 0.0169 | 14.62 | 145200 | 0.5038 | 0.2191 | | 0.0137 | 14.66 | 145600 | 0.4941 | 0.2180 | | 0.0149 | 14.7 | 146000 | 0.5302 | 0.2241 | | 0.0144 | 14.74 | 146400 | 0.5034 | 0.2199 | | 0.0117 | 14.78 | 146800 | 0.4957 | 0.2201 | | 0.015 | 14.83 | 147200 | 0.4869 | 0.2194 | | 0.016 | 14.87 | 147600 | 0.5238 | 0.2245 | | 0.0145 | 14.91 | 148000 | 0.4982 | 0.2190 | | 0.0141 | 14.95 | 148400 | 0.5113 | 0.2215 | | 0.0115 | 14.99 | 148800 | 0.5172 | 0.2192 | | 0.0134 | 15.03 | 149200 | 0.5040 | 0.2203 | | 0.0106 | 15.07 | 149600 | 0.5259 | 0.2146 | | 0.0116 | 15.11 | 150000 | 0.4711 | 0.2118 | | 0.014 | 15.15 | 150400 | 0.4938 | 0.2179 | | 0.0104 | 15.19 | 150800 | 0.5199 | 0.2155 | | 0.012 | 15.23 | 151200 | 0.5314 | 0.2290 | | 0.0146 | 15.27 | 151600 | 0.5313 | 0.2219 | | 0.0105 | 15.31 | 152000 | 0.5135 | 0.2200 | | 0.0176 | 15.35 | 152400 | 0.5106 | 0.2277 | | 0.0149 | 15.39 | 152800 | 0.5098 | 0.2174 | | 0.013 | 15.43 | 153200 | 0.5256 | 0.2187 | | 0.0136 | 15.47 | 153600 | 0.5182 | 0.2207 | | 0.0136 | 15.51 | 154000 | 0.5150 | 0.2238 | | 0.0108 | 15.55 | 154400 | 0.5363 | 0.2241 | | 0.0108 | 15.59 | 154800 | 0.5387 | 0.2252 | | 0.0082 | 15.63 | 155200 | 0.5448 | 0.2248 | | 0.0106 | 15.67 | 155600 | 0.5418 | 0.2262 | | 0.0117 | 15.71 | 156000 | 0.5220 | 0.2262 | | 0.0108 | 15.75 | 156400 | 0.5192 | 0.2263 | | 0.0146 | 15.79 | 156800 | 0.5346 | 0.2250 | | 0.0118 | 15.83 | 157200 | 0.5443 | 0.2256 | | 0.01 | 15.87 | 157600 | 0.5449 | 0.2223 | | 0.0111 | 15.91 | 158000 | 0.5086 | 0.2206 | | 0.0117 | 15.95 | 158400 | 0.5152 | 0.2219 | | 0.0081 | 15.99 | 158800 | 0.5383 | 0.2195 | | 0.0083 | 16.03 | 159200 | 0.5276 | 0.2197 | | 0.0105 | 16.07 | 159600 | 0.5387 | 0.2188 | | 0.0071 | 16.11 | 160000 | 0.5450 | 0.2232 | | 0.0076 | 16.15 | 160400 | 0.5597 | 0.2228 | | 0.0137 | 16.19 | 160800 | 0.5336 | 0.2261 | | 0.0101 | 16.24 | 161200 | 0.5302 | 0.2240 | | 0.012 | 16.28 | 161600 | 0.5084 | 0.2231 | | 0.0103 | 16.32 | 162000 | 0.5295 | 0.2226 | | 0.0095 | 16.36 | 162400 | 0.5294 | 0.2213 | | 0.0091 | 16.4 | 162800 | 0.5171 | 0.2232 | | 0.0074 | 16.44 | 163200 | 0.5284 | 0.2228 | | 0.009 | 16.48 | 163600 | 0.5443 | 0.2221 | | 0.0088 | 16.52 | 164000 | 0.5384 | 0.2227 | | 0.0086 | 16.56 | 164400 | 0.5337 | 0.2201 | | 0.0093 | 16.6 | 164800 | 0.5533 | 0.2212 | | 0.0082 | 16.64 | 165200 | 0.5351 | 0.2196 | | 0.0092 | 16.68 | 165600 | 0.5266 | 0.2197 | | 0.0077 | 16.72 | 166000 | 0.5452 | 0.2242 | | 0.0073 | 16.76 | 166400 | 0.5516 | 0.2204 | | 0.0116 | 16.8 | 166800 | 0.5154 | 0.2199 | | 0.0102 | 16.84 | 167200 | 0.5303 | 0.2179 | | 0.0093 | 16.88 | 167600 | 0.5087 | 0.2156 | | 0.0066 | 16.92 | 168000 | 0.5439 | 0.2155 | | 0.0098 | 16.96 | 168400 | 0.5251 | 0.2202 | | 0.0084 | 17.0 | 168800 | 0.5373 | 0.2215 | | 0.0051 | 17.04 | 169200 | 0.5583 | 0.2177 | | 0.0062 | 17.08 | 169600 | 0.5332 | 0.2179 | | 0.01 | 17.12 | 170000 | 0.5069 | 0.2156 | | 0.0092 | 17.16 | 170400 | 0.5374 | 0.2197 | | 0.0053 | 17.2 | 170800 | 0.5399 | 0.2167 | | 0.0055 | 17.24 | 171200 | 0.5527 | 0.2201 | | 0.0069 | 17.28 | 171600 | 0.5675 | 0.2203 | | 0.012 | 17.32 | 172000 | 0.5578 | 0.2197 | | 0.0076 | 17.36 | 172400 | 0.5632 | 0.2192 | | 0.0109 | 17.4 | 172800 | 0.5455 | 0.2226 | | 0.0069 | 17.44 | 173200 | 0.5708 | 0.2195 | | 0.006 | 17.48 | 173600 | 0.5729 | 0.2169 | | 0.0109 | 17.52 | 174000 | 0.5501 | 0.2192 | | 0.0066 | 17.56 | 174400 | 0.5383 | 0.2179 | | 0.0066 | 17.6 | 174800 | 0.5477 | 0.2196 | | 0.0052 | 17.65 | 175200 | 0.5419 | 0.2164 | | 0.0071 | 17.69 | 175600 | 0.5397 | 0.2206 | | 0.0067 | 17.73 | 176000 | 0.5451 | 0.2213 | | 0.0079 | 17.77 | 176400 | 0.5569 | 0.2204 | | 0.0048 | 17.81 | 176800 | 0.5741 | 0.2193 | | 0.0067 | 17.85 | 177200 | 0.5624 | 0.2197 | | 0.0071 | 17.89 | 177600 | 0.5543 | 0.2192 | | 0.0052 | 17.93 | 178000 | 0.5546 | 0.2204 | | 0.0056 | 17.97 | 178400 | 0.5610 | 0.2204 | | 0.0063 | 18.01 | 178800 | 0.5549 | 0.2173 | | 0.0048 | 18.05 | 179200 | 0.5730 | 0.2180 | | 0.008 | 18.09 | 179600 | 0.5603 | 0.2172 | | 0.0037 | 18.13 | 180000 | 0.5627 | 0.2163 | | 0.0042 | 18.17 | 180400 | 0.5759 | 0.2182 | | 0.0051 | 18.21 | 180800 | 0.5676 | 0.2192 | | 0.004 | 18.25 | 181200 | 0.5797 | 0.2184 | | 0.0051 | 18.29 | 181600 | 0.5814 | 0.2206 | | 0.0088 | 18.33 | 182000 | 0.5677 | 0.2171 | | 0.0037 | 18.37 | 182400 | 0.5713 | 0.2165 | | 0.0058 | 18.41 | 182800 | 0.5910 | 0.2185 | | 0.005 | 18.45 | 183200 | 0.5775 | 0.2183 | | 0.0067 | 18.49 | 183600 | 0.5604 | 0.2158 | | 0.0053 | 18.53 | 184000 | 0.5669 | 0.2171 | | 0.0042 | 18.57 | 184400 | 0.5655 | 0.2174 | | 0.0062 | 18.61 | 184800 | 0.5632 | 0.2187 | | 0.005 | 18.65 | 185200 | 0.5596 | 0.2178 | | 0.0044 | 18.69 | 185600 | 0.5597 | 0.2189 | | 0.0033 | 18.73 | 186000 | 0.5607 | 0.2177 | | 0.0046 | 18.77 | 186400 | 0.5523 | 0.2156 | | 0.0077 | 18.81 | 186800 | 0.5691 | 0.2193 | | 0.0044 | 18.85 | 187200 | 0.5637 | 0.2190 | | 0.004 | 18.89 | 187600 | 0.5712 | 0.2183 | | 0.0047 | 18.93 | 188000 | 0.5643 | 0.2208 | | 0.0053 | 18.97 | 188400 | 0.5678 | 0.2203 | | 0.0037 | 19.02 | 188800 | 0.5730 | 0.2190 | | 0.0034 | 19.06 | 189200 | 0.5660 | 0.2175 | | 0.0039 | 19.1 | 189600 | 0.5544 | 0.2148 | | 0.0037 | 19.14 | 190000 | 0.5622 | 0.2145 | | 0.0029 | 19.18 | 190400 | 0.5654 | 0.2147 | | 0.0045 | 19.22 | 190800 | 0.5624 | 0.2145 | | 0.0026 | 19.26 | 191200 | 0.5651 | 0.2143 | | 0.0039 | 19.3 | 191600 | 0.5685 | 0.2145 | | 0.0042 | 19.34 | 192000 | 0.5711 | 0.2155 | | 0.0033 | 19.38 | 192400 | 0.5624 | 0.2145 | | 0.0059 | 19.42 | 192800 | 0.5654 | 0.2144 | | 0.0038 | 19.46 | 193200 | 0.5693 | 0.2152 | | 0.0044 | 19.5 | 193600 | 0.5674 | 0.2153 | | 0.0047 | 19.54 | 194000 | 0.5650 | 0.2156 | | 0.0032 | 19.58 | 194400 | 0.5695 | 0.2157 | | 0.0033 | 19.62 | 194800 | 0.5740 | 0.2149 | | 0.0039 | 19.66 | 195200 | 0.5688 | 0.2152 | | 0.0048 | 19.7 | 195600 | 0.5685 | 0.2156 | | 0.0055 | 19.74 | 196000 | 0.5652 | 0.2157 | | 0.0051 | 19.78 | 196400 | 0.5620 | 0.2154 | | 0.0051 | 19.82 | 196800 | 0.5602 | 0.2157 | | 0.0038 | 19.86 | 197200 | 0.5629 | 0.2158 | | 0.0029 | 19.9 | 197600 | 0.5651 | 0.2156 | | 0.0031 | 19.94 | 198000 | 0.5656 | 0.2157 | | 0.0031 | 19.98 | 198400 | 0.5656 | 0.2156 | ### Framework versions - Transformers 4.16.0.dev0 - Pytorch 1.10.1+cu102 - Datasets 1.17.1.dev0 - Tokenizers 0.11.0