|
--- |
|
license: apache-2.0 |
|
language: |
|
- es |
|
tags: |
|
- generated_from_trainer |
|
- es |
|
- robust-speech-event |
|
datasets: |
|
- common_voice |
|
model-index: |
|
- name: wav2vec2-xls-r-300m-36-tokens-with-lm-es |
|
results: |
|
- task: |
|
name: Speech Recognition |
|
type: automatic-speech-recognition |
|
dataset: |
|
name: common_voice es |
|
type: common_voice |
|
args: es |
|
metrics: |
|
- name: Test WER |
|
type: wer |
|
value: 0.08677014042867702 |
|
- name: Test CER |
|
type: cer |
|
value: 0.02810974186831335 |
|
- task: |
|
name: Automatic Speech Recognition |
|
type: automatic-speech-recognition |
|
dataset: |
|
name: Robust Speech Event - Dev Data |
|
type: speech-recognition-community-v2/dev_data |
|
args: es |
|
metrics: |
|
- name: Test WER |
|
type: wer |
|
value: 31.68 |
|
- task: |
|
name: Automatic Speech Recognition |
|
type: automatic-speech-recognition |
|
dataset: |
|
name: Robust Speech Event - Test Data |
|
type: speech-recognition-community-v2/eval_data |
|
args: es |
|
metrics: |
|
- name: Test WER |
|
type: wer |
|
value: 34.45 |
|
--- |
|
# Wav2Vec2-xls-r-300m-36-tokens-with-lm-es |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
This model is a fine-tuned version of [facebook/wav2vec2-xls-r-300m](https://huggingface.co/facebook/wav2vec2-xls-r-300m) on the common_voice dataset. |
|
It achieves the following results on the evaluation set: |
|
- Wer: 0.0868 |
|
- Cer: 0.0281 |
|
|
|
This model consists of a Wav2Vec2 model with an additional KenLM 5-gram language model for CTC decoding. |
|
The model is trained removing all characters that are not lower-case unaccented letters between `a-z` or the Spanish accented vowels `á`, `é`, `í`, `ó`, `ú` or the dieresis on the vowel `u` - `ü`. |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0003 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 2 |
|
- total_train_batch_size: 32 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 500 |
|
- num_epochs: 30 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Wer | |
|
|:-------------:|:-----:|:------:|:---------------:|:------:| |
|
| 3.6512 | 0.07 | 400 | 0.5734 | 0.4325 | |
|
| 0.4404 | 0.14 | 800 | 0.3329 | 0.3021 | |
|
| 0.3465 | 0.22 | 1200 | 0.3067 | 0.2871 | |
|
| 0.3214 | 0.29 | 1600 | 0.2808 | 0.2694 | |
|
| 0.319 | 0.36 | 2000 | 0.2755 | 0.2677 | |
|
| 0.3015 | 0.43 | 2400 | 0.2667 | 0.2437 | |
|
| 0.3102 | 0.51 | 2800 | 0.2679 | 0.2475 | |
|
| 0.2955 | 0.58 | 3200 | 0.2591 | 0.2421 | |
|
| 0.292 | 0.65 | 3600 | 0.2547 | 0.2404 | |
|
| 0.2961 | 0.72 | 4000 | 0.2824 | 0.2716 | |
|
| 0.2906 | 0.8 | 4400 | 0.2531 | 0.2321 | |
|
| 0.2886 | 0.87 | 4800 | 0.2668 | 0.2573 | |
|
| 0.2934 | 0.94 | 5200 | 0.2608 | 0.2454 | |
|
| 0.2844 | 1.01 | 5600 | 0.2414 | 0.2233 | |
|
| 0.2649 | 1.09 | 6000 | 0.2412 | 0.2198 | |
|
| 0.2587 | 1.16 | 6400 | 0.2432 | 0.2211 | |
|
| 0.2631 | 1.23 | 6800 | 0.2414 | 0.2225 | |
|
| 0.2584 | 1.3 | 7200 | 0.2489 | 0.2290 | |
|
| 0.2588 | 1.37 | 7600 | 0.2341 | 0.2156 | |
|
| 0.2581 | 1.45 | 8000 | 0.2323 | 0.2155 | |
|
| 0.2603 | 1.52 | 8400 | 0.2423 | 0.2231 | |
|
| 0.2527 | 1.59 | 8800 | 0.2381 | 0.2192 | |
|
| 0.2588 | 1.66 | 9200 | 0.2323 | 0.2176 | |
|
| 0.2543 | 1.74 | 9600 | 0.2391 | 0.2151 | |
|
| 0.2528 | 1.81 | 10000 | 0.2295 | 0.2091 | |
|
| 0.2535 | 1.88 | 10400 | 0.2317 | 0.2099 | |
|
| 0.2501 | 1.95 | 10800 | 0.2225 | 0.2105 | |
|
| 0.2441 | 2.03 | 11200 | 0.2356 | 0.2180 | |
|
| 0.2275 | 2.1 | 11600 | 0.2341 | 0.2115 | |
|
| 0.2281 | 2.17 | 12000 | 0.2269 | 0.2117 | |
|
| 0.227 | 2.24 | 12400 | 0.2367 | 0.2125 | |
|
| 0.2471 | 2.32 | 12800 | 0.2307 | 0.2090 | |
|
| 0.229 | 2.39 | 13200 | 0.2231 | 0.2005 | |
|
| 0.2325 | 2.46 | 13600 | 0.2243 | 0.2100 | |
|
| 0.2314 | 2.53 | 14000 | 0.2252 | 0.2098 | |
|
| 0.2309 | 2.6 | 14400 | 0.2269 | 0.2089 | |
|
| 0.2267 | 2.68 | 14800 | 0.2155 | 0.1976 | |
|
| 0.225 | 2.75 | 15200 | 0.2263 | 0.2067 | |
|
| 0.2309 | 2.82 | 15600 | 0.2196 | 0.2041 | |
|
| 0.225 | 2.89 | 16000 | 0.2212 | 0.2052 | |
|
| 0.228 | 2.97 | 16400 | 0.2192 | 0.2028 | |
|
| 0.2136 | 3.04 | 16800 | 0.2169 | 0.2042 | |
|
| 0.2038 | 3.11 | 17200 | 0.2173 | 0.1998 | |
|
| 0.2035 | 3.18 | 17600 | 0.2185 | 0.2002 | |
|
| 0.207 | 3.26 | 18000 | 0.2358 | 0.2120 | |
|
| 0.2102 | 3.33 | 18400 | 0.2213 | 0.2019 | |
|
| 0.211 | 3.4 | 18800 | 0.2176 | 0.1980 | |
|
| 0.2099 | 3.47 | 19200 | 0.2186 | 0.1960 | |
|
| 0.2093 | 3.55 | 19600 | 0.2208 | 0.2016 | |
|
| 0.2046 | 3.62 | 20000 | 0.2138 | 0.1960 | |
|
| 0.2095 | 3.69 | 20400 | 0.2222 | 0.2023 | |
|
| 0.2106 | 3.76 | 20800 | 0.2159 | 0.1964 | |
|
| 0.2066 | 3.83 | 21200 | 0.2083 | 0.1931 | |
|
| 0.2119 | 3.91 | 21600 | 0.2130 | 0.1957 | |
|
| 0.2167 | 3.98 | 22000 | 0.2210 | 0.1987 | |
|
| 0.1973 | 4.05 | 22400 | 0.2112 | 0.1930 | |
|
| 0.1917 | 4.12 | 22800 | 0.2107 | 0.1891 | |
|
| 0.1903 | 4.2 | 23200 | 0.2132 | 0.1911 | |
|
| 0.1903 | 4.27 | 23600 | 0.2077 | 0.1883 | |
|
| 0.1914 | 4.34 | 24000 | 0.2054 | 0.1901 | |
|
| 0.1943 | 4.41 | 24400 | 0.2059 | 0.1885 | |
|
| 0.1943 | 4.49 | 24800 | 0.2095 | 0.1899 | |
|
| 0.1936 | 4.56 | 25200 | 0.2078 | 0.1879 | |
|
| 0.1963 | 4.63 | 25600 | 0.2018 | 0.1884 | |
|
| 0.1934 | 4.7 | 26000 | 0.2034 | 0.1872 | |
|
| 0.2011 | 4.78 | 26400 | 0.2051 | 0.1896 | |
|
| 0.1901 | 4.85 | 26800 | 0.2059 | 0.1858 | |
|
| 0.1934 | 4.92 | 27200 | 0.2028 | 0.1832 | |
|
| 0.191 | 4.99 | 27600 | 0.2046 | 0.1870 | |
|
| 0.1775 | 5.07 | 28000 | 0.2081 | 0.1891 | |
|
| 0.175 | 5.14 | 28400 | 0.2084 | 0.1904 | |
|
| 0.19 | 5.21 | 28800 | 0.2086 | 0.1920 | |
|
| 0.1798 | 5.28 | 29200 | 0.2079 | 0.1935 | |
|
| 0.1765 | 5.35 | 29600 | 0.2145 | 0.1930 | |
|
| 0.181 | 5.43 | 30000 | 0.2062 | 0.1918 | |
|
| 0.1808 | 5.5 | 30400 | 0.2083 | 0.1875 | |
|
| 0.1769 | 5.57 | 30800 | 0.2117 | 0.1895 | |
|
| 0.1788 | 5.64 | 31200 | 0.2055 | 0.1857 | |
|
| 0.181 | 5.72 | 31600 | 0.2057 | 0.1870 | |
|
| 0.1781 | 5.79 | 32000 | 0.2053 | 0.1872 | |
|
| 0.1852 | 5.86 | 32400 | 0.2077 | 0.1904 | |
|
| 0.1832 | 5.93 | 32800 | 0.1979 | 0.1821 | |
|
| 0.1758 | 6.01 | 33200 | 0.1957 | 0.1754 | |
|
| 0.1611 | 6.08 | 33600 | 0.2028 | 0.1773 | |
|
| 0.1606 | 6.15 | 34000 | 0.2018 | 0.1780 | |
|
| 0.1702 | 6.22 | 34400 | 0.1977 | 0.1759 | |
|
| 0.1649 | 6.3 | 34800 | 0.2073 | 0.1845 | |
|
| 0.1641 | 6.37 | 35200 | 0.1947 | 0.1774 | |
|
| 0.1703 | 6.44 | 35600 | 0.2009 | 0.1811 | |
|
| 0.1716 | 6.51 | 36000 | 0.2091 | 0.1817 | |
|
| 0.1732 | 6.58 | 36400 | 0.1942 | 0.1743 | |
|
| 0.1642 | 6.66 | 36800 | 0.1930 | 0.1749 | |
|
| 0.1685 | 6.73 | 37200 | 0.1962 | 0.1716 | |
|
| 0.1647 | 6.8 | 37600 | 0.1977 | 0.1822 | |
|
| 0.1647 | 6.87 | 38000 | 0.1917 | 0.1748 | |
|
| 0.1667 | 6.95 | 38400 | 0.1948 | 0.1774 | |
|
| 0.1647 | 7.02 | 38800 | 0.2018 | 0.1783 | |
|
| 0.15 | 7.09 | 39200 | 0.2010 | 0.1796 | |
|
| 0.1663 | 7.16 | 39600 | 0.1969 | 0.1731 | |
|
| 0.1536 | 7.24 | 40000 | 0.1935 | 0.1726 | |
|
| 0.1544 | 7.31 | 40400 | 0.2030 | 0.1799 | |
|
| 0.1536 | 7.38 | 40800 | 0.1973 | 0.1772 | |
|
| 0.1559 | 7.45 | 41200 | 0.1973 | 0.1763 | |
|
| 0.1547 | 7.53 | 41600 | 0.2052 | 0.1782 | |
|
| 0.1584 | 7.6 | 42000 | 0.1965 | 0.1737 | |
|
| 0.1542 | 7.67 | 42400 | 0.1878 | 0.1725 | |
|
| 0.1525 | 7.74 | 42800 | 0.1946 | 0.1750 | |
|
| 0.1547 | 7.81 | 43200 | 0.1934 | 0.1691 | |
|
| 0.1534 | 7.89 | 43600 | 0.1919 | 0.1711 | |
|
| 0.1574 | 7.96 | 44000 | 0.1935 | 0.1745 | |
|
| 0.1471 | 8.03 | 44400 | 0.1915 | 0.1689 | |
|
| 0.1433 | 8.1 | 44800 | 0.1956 | 0.1719 | |
|
| 0.1433 | 8.18 | 45200 | 0.1980 | 0.1720 | |
|
| 0.1424 | 8.25 | 45600 | 0.1906 | 0.1681 | |
|
| 0.1428 | 8.32 | 46000 | 0.1892 | 0.1649 | |
|
| 0.1424 | 8.39 | 46400 | 0.1916 | 0.1698 | |
|
| 0.1466 | 8.47 | 46800 | 0.1970 | 0.1739 | |
|
| 0.1496 | 8.54 | 47200 | 0.1902 | 0.1662 | |
|
| 0.1408 | 8.61 | 47600 | 0.1858 | 0.1649 | |
|
| 0.1445 | 8.68 | 48000 | 0.1893 | 0.1648 | |
|
| 0.1459 | 8.76 | 48400 | 0.1875 | 0.1686 | |
|
| 0.1433 | 8.83 | 48800 | 0.1920 | 0.1673 | |
|
| 0.1448 | 8.9 | 49200 | 0.1833 | 0.1631 | |
|
| 0.1461 | 8.97 | 49600 | 0.1904 | 0.1693 | |
|
| 0.1451 | 9.04 | 50000 | 0.1969 | 0.1661 | |
|
| 0.1336 | 9.12 | 50400 | 0.1950 | 0.1674 | |
|
| 0.1362 | 9.19 | 50800 | 0.1971 | 0.1685 | |
|
| 0.1316 | 9.26 | 51200 | 0.1928 | 0.1648 | |
|
| 0.132 | 9.33 | 51600 | 0.1908 | 0.1615 | |
|
| 0.1301 | 9.41 | 52000 | 0.1842 | 0.1569 | |
|
| 0.1322 | 9.48 | 52400 | 0.1892 | 0.1616 | |
|
| 0.1391 | 9.55 | 52800 | 0.1956 | 0.1656 | |
|
| 0.132 | 9.62 | 53200 | 0.1876 | 0.1598 | |
|
| 0.1349 | 9.7 | 53600 | 0.1870 | 0.1624 | |
|
| 0.1325 | 9.77 | 54000 | 0.1834 | 0.1586 | |
|
| 0.1389 | 9.84 | 54400 | 0.1892 | 0.1647 | |
|
| 0.1364 | 9.91 | 54800 | 0.1840 | 0.1597 | |
|
| 0.1339 | 9.99 | 55200 | 0.1858 | 0.1626 | |
|
| 0.1269 | 10.06 | 55600 | 0.1875 | 0.1619 | |
|
| 0.1229 | 10.13 | 56000 | 0.1909 | 0.1619 | |
|
| 0.1258 | 10.2 | 56400 | 0.1933 | 0.1631 | |
|
| 0.1256 | 10.27 | 56800 | 0.1930 | 0.1640 | |
|
| 0.1207 | 10.35 | 57200 | 0.1823 | 0.1585 | |
|
| 0.1248 | 10.42 | 57600 | 0.1889 | 0.1596 | |
|
| 0.1264 | 10.49 | 58000 | 0.1845 | 0.1584 | |
|
| 0.1251 | 10.56 | 58400 | 0.1869 | 0.1588 | |
|
| 0.1251 | 10.64 | 58800 | 0.1885 | 0.1613 | |
|
| 0.1276 | 10.71 | 59200 | 0.1855 | 0.1575 | |
|
| 0.1303 | 10.78 | 59600 | 0.1836 | 0.1597 | |
|
| 0.1246 | 10.85 | 60000 | 0.1810 | 0.1573 | |
|
| 0.1283 | 10.93 | 60400 | 0.1830 | 0.1581 | |
|
| 0.1273 | 11.0 | 60800 | 0.1837 | 0.1619 | |
|
| 0.1202 | 11.07 | 61200 | 0.1865 | 0.1588 | |
|
| 0.119 | 11.14 | 61600 | 0.1889 | 0.1580 | |
|
| 0.1179 | 11.22 | 62000 | 0.1884 | 0.1592 | |
|
| 0.1187 | 11.29 | 62400 | 0.1824 | 0.1565 | |
|
| 0.1198 | 11.36 | 62800 | 0.1848 | 0.1552 | |
|
| 0.1154 | 11.43 | 63200 | 0.1866 | 0.1565 | |
|
| 0.1211 | 11.51 | 63600 | 0.1862 | 0.1563 | |
|
| 0.1177 | 11.58 | 64000 | 0.1816 | 0.1527 | |
|
| 0.1156 | 11.65 | 64400 | 0.1834 | 0.1540 | |
|
| 0.1144 | 11.72 | 64800 | 0.1837 | 0.1524 | |
|
| 0.119 | 11.79 | 65200 | 0.1859 | 0.1538 | |
|
| 0.1183 | 11.87 | 65600 | 0.1869 | 0.1558 | |
|
| 0.122 | 11.94 | 66000 | 0.1853 | 0.1535 | |
|
| 0.1197 | 12.01 | 66400 | 0.1871 | 0.1586 | |
|
| 0.1096 | 12.08 | 66800 | 0.1838 | 0.1540 | |
|
| 0.1074 | 12.16 | 67200 | 0.1915 | 0.1592 | |
|
| 0.1084 | 12.23 | 67600 | 0.1845 | 0.1545 | |
|
| 0.1097 | 12.3 | 68000 | 0.1904 | 0.1552 | |
|
| 0.112 | 12.37 | 68400 | 0.1846 | 0.1578 | |
|
| 0.1109 | 12.45 | 68800 | 0.1862 | 0.1549 | |
|
| 0.1114 | 12.52 | 69200 | 0.1889 | 0.1552 | |
|
| 0.1119 | 12.59 | 69600 | 0.1828 | 0.1530 | |
|
| 0.1124 | 12.66 | 70000 | 0.1822 | 0.1540 | |
|
| 0.1127 | 12.74 | 70400 | 0.1865 | 0.1589 | |
|
| 0.1128 | 12.81 | 70800 | 0.1786 | 0.1498 | |
|
| 0.1069 | 12.88 | 71200 | 0.1813 | 0.1522 | |
|
| 0.1069 | 12.95 | 71600 | 0.1895 | 0.1558 | |
|
| 0.1083 | 13.02 | 72000 | 0.1925 | 0.1557 | |
|
| 0.1009 | 13.1 | 72400 | 0.1883 | 0.1522 | |
|
| 0.1007 | 13.17 | 72800 | 0.1829 | 0.1480 | |
|
| 0.1014 | 13.24 | 73200 | 0.1861 | 0.1510 | |
|
| 0.0974 | 13.31 | 73600 | 0.1836 | 0.1486 | |
|
| 0.1006 | 13.39 | 74000 | 0.1821 | 0.1462 | |
|
| 0.0973 | 13.46 | 74400 | 0.1857 | 0.1484 | |
|
| 0.1011 | 13.53 | 74800 | 0.1822 | 0.1471 | |
|
| 0.1031 | 13.6 | 75200 | 0.1823 | 0.1489 | |
|
| 0.1034 | 13.68 | 75600 | 0.1809 | 0.1452 | |
|
| 0.0998 | 13.75 | 76000 | 0.1817 | 0.1490 | |
|
| 0.1071 | 13.82 | 76400 | 0.1808 | 0.1501 | |
|
| 0.1083 | 13.89 | 76800 | 0.1796 | 0.1475 | |
|
| 0.1053 | 13.97 | 77200 | 0.1785 | 0.1470 | |
|
| 0.0978 | 14.04 | 77600 | 0.1886 | 0.1495 | |
|
| 0.094 | 14.11 | 78000 | 0.1854 | 0.1489 | |
|
| 0.0915 | 14.18 | 78400 | 0.1854 | 0.1498 | |
|
| 0.0947 | 14.25 | 78800 | 0.1888 | 0.1500 | |
|
| 0.0939 | 14.33 | 79200 | 0.1885 | 0.1494 | |
|
| 0.0973 | 14.4 | 79600 | 0.1877 | 0.1466 | |
|
| 0.0946 | 14.47 | 80000 | 0.1904 | 0.1494 | |
|
| 0.0931 | 14.54 | 80400 | 0.1815 | 0.1473 | |
|
| 0.0958 | 14.62 | 80800 | 0.1905 | 0.1508 | |
|
| 0.0982 | 14.69 | 81200 | 0.1881 | 0.1511 | |
|
| 0.0963 | 14.76 | 81600 | 0.1823 | 0.1449 | |
|
| 0.0943 | 14.83 | 82000 | 0.1782 | 0.1458 | |
|
| 0.0981 | 14.91 | 82400 | 0.1795 | 0.1465 | |
|
| 0.0995 | 14.98 | 82800 | 0.1811 | 0.1484 | |
|
| 0.0909 | 15.05 | 83200 | 0.1822 | 0.1450 | |
|
| 0.0872 | 15.12 | 83600 | 0.1890 | 0.1466 | |
|
| 0.0878 | 15.2 | 84000 | 0.1859 | 0.1468 | |
|
| 0.0884 | 15.27 | 84400 | 0.1825 | 0.1429 | |
|
| 0.0871 | 15.34 | 84800 | 0.1816 | 0.1438 | |
|
| 0.0883 | 15.41 | 85200 | 0.1817 | 0.1433 | |
|
| 0.0844 | 15.48 | 85600 | 0.1821 | 0.1412 | |
|
| 0.0843 | 15.56 | 86000 | 0.1863 | 0.1411 | |
|
| 0.0805 | 15.63 | 86400 | 0.1863 | 0.1441 | |
|
| 0.085 | 15.7 | 86800 | 0.1808 | 0.1440 | |
|
| 0.0848 | 15.77 | 87200 | 0.1808 | 0.1421 | |
|
| 0.0844 | 15.85 | 87600 | 0.1841 | 0.1406 | |
|
| 0.082 | 15.92 | 88000 | 0.1850 | 0.1442 | |
|
| 0.0854 | 15.99 | 88400 | 0.1773 | 0.1426 | |
|
| 0.0835 | 16.06 | 88800 | 0.1888 | 0.1436 | |
|
| 0.0789 | 16.14 | 89200 | 0.1922 | 0.1434 | |
|
| 0.081 | 16.21 | 89600 | 0.1864 | 0.1448 | |
|
| 0.0799 | 16.28 | 90000 | 0.1902 | 0.1428 | |
|
| 0.0848 | 16.35 | 90400 | 0.1873 | 0.1422 | |
|
| 0.084 | 16.43 | 90800 | 0.1835 | 0.1421 | |
|
| 0.083 | 16.5 | 91200 | 0.1878 | 0.1390 | |
|
| 0.0794 | 16.57 | 91600 | 0.1877 | 0.1398 | |
|
| 0.0807 | 16.64 | 92000 | 0.1800 | 0.1385 | |
|
| 0.0829 | 16.71 | 92400 | 0.1910 | 0.1434 | |
|
| 0.0839 | 16.79 | 92800 | 0.1843 | 0.1381 | |
|
| 0.0815 | 16.86 | 93200 | 0.1812 | 0.1365 | |
|
| 0.0831 | 16.93 | 93600 | 0.1889 | 0.1383 | |
|
| 0.0803 | 17.0 | 94000 | 0.1902 | 0.1403 | |
|
| 0.0724 | 17.08 | 94400 | 0.1934 | 0.1380 | |
|
| 0.0734 | 17.15 | 94800 | 0.1865 | 0.1394 | |
|
| 0.0739 | 17.22 | 95200 | 0.1876 | 0.1395 | |
|
| 0.0758 | 17.29 | 95600 | 0.1938 | 0.1411 | |
|
| 0.0733 | 17.37 | 96000 | 0.1933 | 0.1410 | |
|
| 0.077 | 17.44 | 96400 | 0.1848 | 0.1385 | |
|
| 0.0754 | 17.51 | 96800 | 0.1876 | 0.1407 | |
|
| 0.0746 | 17.58 | 97200 | 0.1863 | 0.1371 | |
|
| 0.0732 | 17.66 | 97600 | 0.1927 | 0.1401 | |
|
| 0.0746 | 17.73 | 98000 | 0.1874 | 0.1390 | |
|
| 0.0755 | 17.8 | 98400 | 0.1853 | 0.1381 | |
|
| 0.0724 | 17.87 | 98800 | 0.1849 | 0.1365 | |
|
| 0.0716 | 17.94 | 99200 | 0.1848 | 0.1380 | |
|
| 0.074 | 18.02 | 99600 | 0.1891 | 0.1362 | |
|
| 0.0687 | 18.09 | 100000 | 0.1974 | 0.1357 | |
|
| 0.0651 | 18.16 | 100400 | 0.1942 | 0.1353 | |
|
| 0.0672 | 18.23 | 100800 | 0.1823 | 0.1363 | |
|
| 0.0671 | 18.31 | 101200 | 0.1959 | 0.1357 | |
|
| 0.0684 | 18.38 | 101600 | 0.1959 | 0.1374 | |
|
| 0.0688 | 18.45 | 102000 | 0.1904 | 0.1353 | |
|
| 0.0696 | 18.52 | 102400 | 0.1926 | 0.1364 | |
|
| 0.0661 | 18.6 | 102800 | 0.1905 | 0.1351 | |
|
| 0.0684 | 18.67 | 103200 | 0.1955 | 0.1343 | |
|
| 0.0712 | 18.74 | 103600 | 0.1873 | 0.1353 | |
|
| 0.0701 | 18.81 | 104000 | 0.1822 | 0.1354 | |
|
| 0.0688 | 18.89 | 104400 | 0.1905 | 0.1373 | |
|
| 0.0695 | 18.96 | 104800 | 0.1879 | 0.1335 | |
|
| 0.0661 | 19.03 | 105200 | 0.2005 | 0.1351 | |
|
| 0.0644 | 19.1 | 105600 | 0.1972 | 0.1351 | |
|
| 0.0627 | 19.18 | 106000 | 0.1956 | 0.1340 | |
|
| 0.0633 | 19.25 | 106400 | 0.1962 | 0.1340 | |
|
| 0.0629 | 19.32 | 106800 | 0.1937 | 0.1342 | |
|
| 0.0636 | 19.39 | 107200 | 0.1905 | 0.1355 | |
|
| 0.0631 | 19.46 | 107600 | 0.1917 | 0.1326 | |
|
| 0.0624 | 19.54 | 108000 | 0.1977 | 0.1355 | |
|
| 0.0621 | 19.61 | 108400 | 0.1941 | 0.1345 | |
|
| 0.0635 | 19.68 | 108800 | 0.1949 | 0.1336 | |
|
| 0.063 | 19.75 | 109200 | 0.1919 | 0.1317 | |
|
| 0.0636 | 19.83 | 109600 | 0.1928 | 0.1317 | |
|
| 0.0612 | 19.9 | 110000 | 0.1923 | 0.1314 | |
|
| 0.0636 | 19.97 | 110400 | 0.1923 | 0.1343 | |
|
| 0.0581 | 20.04 | 110800 | 0.2036 | 0.1332 | |
|
| 0.0573 | 20.12 | 111200 | 0.2007 | 0.1315 | |
|
| 0.0566 | 20.19 | 111600 | 0.1974 | 0.1319 | |
|
| 0.0589 | 20.26 | 112000 | 0.1958 | 0.1322 | |
|
| 0.0577 | 20.33 | 112400 | 0.1946 | 0.1307 | |
|
| 0.0587 | 20.41 | 112800 | 0.1957 | 0.1295 | |
|
| 0.0588 | 20.48 | 113200 | 0.2013 | 0.1306 | |
|
| 0.0594 | 20.55 | 113600 | 0.2010 | 0.1312 | |
|
| 0.0602 | 20.62 | 114000 | 0.1993 | 0.1314 | |
|
| 0.0583 | 20.69 | 114400 | 0.1931 | 0.1297 | |
|
| 0.059 | 20.77 | 114800 | 0.1974 | 0.1305 | |
|
| 0.0566 | 20.84 | 115200 | 0.1979 | 0.1294 | |
|
| 0.0588 | 20.91 | 115600 | 0.1944 | 0.1292 | |
|
| 0.0569 | 20.98 | 116000 | 0.1974 | 0.1309 | |
|
| 0.0554 | 21.06 | 116400 | 0.2080 | 0.1307 | |
|
| 0.0542 | 21.13 | 116800 | 0.2056 | 0.1301 | |
|
| 0.0532 | 21.2 | 117200 | 0.2027 | 0.1309 | |
|
| 0.0535 | 21.27 | 117600 | 0.1970 | 0.1287 | |
|
| 0.0533 | 21.35 | 118000 | 0.2124 | 0.1310 | |
|
| 0.0546 | 21.42 | 118400 | 0.2043 | 0.1300 | |
|
| 0.0544 | 21.49 | 118800 | 0.2056 | 0.1281 | |
|
| 0.0562 | 21.56 | 119200 | 0.1986 | 0.1273 | |
|
| 0.0549 | 21.64 | 119600 | 0.2075 | 0.1283 | |
|
| 0.0522 | 21.71 | 120000 | 0.2058 | 0.1278 | |
|
| 0.052 | 21.78 | 120400 | 0.2057 | 0.1280 | |
|
| 0.0563 | 21.85 | 120800 | 0.1966 | 0.1295 | |
|
| 0.0546 | 21.92 | 121200 | 0.2002 | 0.1285 | |
|
| 0.0539 | 22.0 | 121600 | 0.1996 | 0.1279 | |
|
| 0.0504 | 22.07 | 122000 | 0.2077 | 0.1273 | |
|
| 0.0602 | 22.14 | 122400 | 0.2055 | 0.1278 | |
|
| 0.0503 | 22.21 | 122800 | 0.2037 | 0.1283 | |
|
| 0.0496 | 22.29 | 123200 | 0.2109 | 0.1279 | |
|
| 0.0523 | 22.36 | 123600 | 0.2068 | 0.1276 | |
|
| 0.0508 | 22.43 | 124000 | 0.2051 | 0.1257 | |
|
| 0.0505 | 22.5 | 124400 | 0.2056 | 0.1269 | |
|
| 0.05 | 22.58 | 124800 | 0.1995 | 0.1268 | |
|
| 0.0496 | 22.65 | 125200 | 0.2022 | 0.1290 | |
|
| 0.0484 | 22.72 | 125600 | 0.2095 | 0.1291 | |
|
| 0.0518 | 22.79 | 126000 | 0.2132 | 0.1271 | |
|
| 0.0499 | 22.87 | 126400 | 0.2124 | 0.1263 | |
|
| 0.0485 | 22.94 | 126800 | 0.2092 | 0.1252 | |
|
| 0.0476 | 23.01 | 127200 | 0.2138 | 0.1256 | |
|
| 0.0467 | 23.08 | 127600 | 0.2119 | 0.1256 | |
|
| 0.048 | 23.15 | 128000 | 0.2138 | 0.1269 | |
|
| 0.0461 | 23.23 | 128400 | 0.2036 | 0.1244 | |
|
| 0.0467 | 23.3 | 128800 | 0.2163 | 0.1255 | |
|
| 0.0475 | 23.37 | 129200 | 0.2180 | 0.1258 | |
|
| 0.0468 | 23.44 | 129600 | 0.2129 | 0.1245 | |
|
| 0.0456 | 23.52 | 130000 | 0.2122 | 0.1250 | |
|
| 0.0458 | 23.59 | 130400 | 0.2157 | 0.1257 | |
|
| 0.0453 | 23.66 | 130800 | 0.2088 | 0.1242 | |
|
| 0.045 | 23.73 | 131200 | 0.2144 | 0.1247 | |
|
| 0.0469 | 23.81 | 131600 | 0.2113 | 0.1246 | |
|
| 0.0453 | 23.88 | 132000 | 0.2151 | 0.1234 | |
|
| 0.0471 | 23.95 | 132400 | 0.2130 | 0.1229 | |
|
| 0.0443 | 24.02 | 132800 | 0.2150 | 0.1225 | |
|
| 0.0446 | 24.1 | 133200 | 0.2166 | 0.1235 | |
|
| 0.0435 | 24.17 | 133600 | 0.2143 | 0.1222 | |
|
| 0.0407 | 24.24 | 134000 | 0.2175 | 0.1218 | |
|
| 0.0421 | 24.31 | 134400 | 0.2147 | 0.1227 | |
|
| 0.0435 | 24.38 | 134800 | 0.2193 | 0.1233 | |
|
| 0.0414 | 24.46 | 135200 | 0.2172 | 0.1225 | |
|
| 0.0419 | 24.53 | 135600 | 0.2156 | 0.1225 | |
|
| 0.0419 | 24.6 | 136000 | 0.2143 | 0.1235 | |
|
| 0.0423 | 24.67 | 136400 | 0.2179 | 0.1226 | |
|
| 0.0423 | 24.75 | 136800 | 0.2144 | 0.1221 | |
|
| 0.0424 | 24.82 | 137200 | 0.2135 | 0.1210 | |
|
| 0.0419 | 24.89 | 137600 | 0.2166 | 0.1218 | |
|
| 0.0408 | 24.96 | 138000 | 0.2151 | 0.1211 | |
|
| 0.0433 | 25.04 | 138400 | 0.2174 | 0.1214 | |
|
| 0.0395 | 25.11 | 138800 | 0.2242 | 0.1210 | |
|
| 0.0403 | 25.18 | 139200 | 0.2219 | 0.1215 | |
|
| 0.0413 | 25.25 | 139600 | 0.2225 | 0.1207 | |
|
| 0.0389 | 25.33 | 140000 | 0.2187 | 0.1202 | |
|
| 0.0395 | 25.4 | 140400 | 0.2244 | 0.1204 | |
|
| 0.0398 | 25.47 | 140800 | 0.2263 | 0.1199 | |
|
| 0.0386 | 25.54 | 141200 | 0.2165 | 0.1187 | |
|
| 0.0396 | 25.61 | 141600 | 0.2171 | 0.1187 | |
|
| 0.0406 | 25.69 | 142000 | 0.2199 | 0.1190 | |
|
| 0.0404 | 25.76 | 142400 | 0.2224 | 0.1190 | |
|
| 0.0391 | 25.83 | 142800 | 0.2230 | 0.1185 | |
|
| 0.04 | 25.9 | 143200 | 0.2208 | 0.1200 | |
|
| 0.0396 | 25.98 | 143600 | 0.2179 | 0.1191 | |
|
| 0.0353 | 26.05 | 144000 | 0.2285 | 0.1178 | |
|
| 0.0368 | 26.12 | 144400 | 0.2273 | 0.1186 | |
|
| 0.0393 | 26.19 | 144800 | 0.2247 | 0.1196 | |
|
| 0.0368 | 26.27 | 145200 | 0.2314 | 0.1181 | |
|
| 0.0373 | 26.34 | 145600 | 0.2215 | 0.1188 | |
|
| 0.038 | 26.41 | 146000 | 0.2262 | 0.1180 | |
|
| 0.0363 | 26.48 | 146400 | 0.2250 | 0.1172 | |
|
| 0.0365 | 26.56 | 146800 | 0.2299 | 0.1174 | |
|
| 0.0382 | 26.63 | 147200 | 0.2292 | 0.1165 | |
|
| 0.0365 | 26.7 | 147600 | 0.2282 | 0.1165 | |
|
| 0.0371 | 26.77 | 148000 | 0.2276 | 0.1172 | |
|
| 0.0365 | 26.85 | 148400 | 0.2280 | 0.1173 | |
|
| 0.0376 | 26.92 | 148800 | 0.2248 | 0.1164 | |
|
| 0.0365 | 26.99 | 149200 | 0.2230 | 0.1158 | |
|
| 0.0343 | 27.06 | 149600 | 0.2300 | 0.1157 | |
|
| 0.0354 | 27.13 | 150000 | 0.2298 | 0.1166 | |
|
| 0.0333 | 27.21 | 150400 | 0.2307 | 0.1158 | |
|
| 0.0353 | 27.28 | 150800 | 0.2300 | 0.1157 | |
|
| 0.036 | 27.35 | 151200 | 0.2335 | 0.1160 | |
|
| 0.0343 | 27.42 | 151600 | 0.2324 | 0.1155 | |
|
| 0.0361 | 27.5 | 152000 | 0.2300 | 0.1150 | |
|
| 0.0352 | 27.57 | 152400 | 0.2279 | 0.1146 | |
|
| 0.0353 | 27.64 | 152800 | 0.2307 | 0.1149 | |
|
| 0.0342 | 27.71 | 153200 | 0.2315 | 0.1152 | |
|
| 0.0345 | 27.79 | 153600 | 0.2290 | 0.1146 | |
|
| 0.034 | 27.86 | 154000 | 0.2319 | 0.1141 | |
|
| 0.0347 | 27.93 | 154400 | 0.2312 | 0.1144 | |
|
| 0.0338 | 28.0 | 154800 | 0.2328 | 0.1146 | |
|
| 0.0347 | 28.08 | 155200 | 0.2352 | 0.1151 | |
|
| 0.033 | 28.15 | 155600 | 0.2337 | 0.1142 | |
|
| 0.0336 | 28.22 | 156000 | 0.2345 | 0.1141 | |
|
| 0.0337 | 28.29 | 156400 | 0.2315 | 0.1143 | |
|
| 0.0314 | 28.36 | 156800 | 0.2353 | 0.1140 | |
|
| 0.0333 | 28.44 | 157200 | 0.2338 | 0.1146 | |
|
| 0.0317 | 28.51 | 157600 | 0.2345 | 0.1139 | |
|
| 0.0326 | 28.58 | 158000 | 0.2336 | 0.1143 | |
|
| 0.033 | 28.65 | 158400 | 0.2352 | 0.1137 | |
|
| 0.0325 | 28.73 | 158800 | 0.2312 | 0.1130 | |
|
| 0.0321 | 28.8 | 159200 | 0.2338 | 0.1133 | |
|
| 0.0334 | 28.87 | 159600 | 0.2335 | 0.1130 | |
|
| 0.0317 | 28.94 | 160000 | 0.2340 | 0.1126 | |
|
| 0.0321 | 29.02 | 160400 | 0.2349 | 0.1126 | |
|
| 0.032 | 29.09 | 160800 | 0.2369 | 0.1127 | |
|
| 0.0312 | 29.16 | 161200 | 0.2363 | 0.1124 | |
|
| 0.0303 | 29.23 | 161600 | 0.2363 | 0.1123 | |
|
| 0.0322 | 29.31 | 162000 | 0.2354 | 0.1124 | |
|
| 0.03 | 29.38 | 162400 | 0.2360 | 0.1122 | |
|
| 0.0299 | 29.45 | 162800 | 0.2378 | 0.1124 | |
|
| 0.0313 | 29.52 | 163200 | 0.2377 | 0.1120 | |
|
| 0.0299 | 29.59 | 163600 | 0.2367 | 0.1124 | |
|
| 0.0313 | 29.67 | 164000 | 0.2380 | 0.1120 | |
|
| 0.031 | 29.74 | 164400 | 0.2369 | 0.1120 | |
|
| 0.0327 | 29.81 | 164800 | 0.2358 | 0.1117 | |
|
| 0.0316 | 29.88 | 165200 | 0.2358 | 0.1118 | |
|
| 0.0307 | 29.96 | 165600 | 0.2362 | 0.1118 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.16.0.dev0 |
|
- Pytorch 1.10.1+cu102 |
|
- Datasets 1.17.1.dev0 |
|
- Tokenizers 0.11.0 |
|
|
