--- base_model: allenai/scibert_scivocab_uncased tags: - generated_from_trainer model-index: - name: mlm_scibert_uncased results: [] --- # mlm_scibert_uncased This model is a fine-tuned version of [allenai/scibert_scivocab_uncased](https://huggingface.co/allenai/scibert_scivocab_uncased) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.2966 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 16 - eval_batch_size: 8 - seed: 1234 - gradient_accumulation_steps: 16 - total_train_batch_size: 256 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 500 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:--------:|:-----:|:---------------:| | 1.3835 | 0.9963 | 152 | 1.2575 | | 1.3098 | 1.9992 | 305 | 1.2320 | | 1.2883 | 2.9955 | 457 | 1.2204 | | 1.2612 | 3.9984 | 610 | 1.2113 | | 1.2501 | 4.9947 | 762 | 1.2043 | | 1.2292 | 5.9975 | 915 | 1.1910 | | 1.2269 | 6.9939 | 1067 | 1.1859 | | 1.2096 | 7.9967 | 1220 | 1.1841 | | 1.1992 | 8.9996 | 1373 | 1.1772 | | 1.1973 | 9.9959 | 1525 | 1.1806 | | 1.1801 | 10.9988 | 1678 | 1.1720 | | 1.1822 | 11.9951 | 1830 | 1.1699 | | 1.1693 | 12.9980 | 1983 | 1.1674 | | 1.1677 | 13.9943 | 2135 | 1.1641 | | 1.1529 | 14.9971 | 2288 | 1.1597 | | 1.1448 | 16.0 | 2441 | 1.1613 | | 1.1499 | 16.9963 | 2593 | 1.1579 | | 1.1363 | 17.9992 | 2746 | 1.1580 | | 1.1369 | 18.9955 | 2898 | 1.1640 | | 1.1247 | 19.9984 | 3051 | 1.1560 | | 1.1283 | 20.9947 | 3203 | 1.1505 | | 1.1173 | 21.9975 | 3356 | 1.1497 | | 1.1171 | 22.9939 | 3508 | 1.1522 | | 1.1064 | 23.9967 | 3661 | 1.1505 | | 1.1034 | 24.9996 | 3814 | 1.1471 | | 1.1051 | 25.9959 | 3966 | 1.1447 | | 1.0949 | 26.9988 | 4119 | 1.1414 | | 1.0975 | 27.9951 | 4271 | 1.1470 | | 1.0852 | 28.9980 | 4424 | 1.1497 | | 1.0888 | 29.9943 | 4576 | 1.1472 | | 1.0779 | 30.9971 | 4729 | 1.1418 | | 1.0741 | 32.0 | 4882 | 1.1430 | | 1.0761 | 32.9963 | 5034 | 1.1448 | | 1.0674 | 33.9992 | 5187 | 1.1447 | | 1.0712 | 34.9955 | 5339 | 1.1451 | | 1.0604 | 35.9984 | 5492 | 1.1440 | | 1.0612 | 36.9947 | 5644 | 1.1423 | | 1.0549 | 37.9975 | 5797 | 1.1460 | | 1.0553 | 38.9939 | 5949 | 1.1456 | | 1.0469 | 39.9967 | 6102 | 1.1436 | | 1.0411 | 40.9996 | 6255 | 1.1401 | | 1.0474 | 41.9959 | 6407 | 1.1395 | | 1.0373 | 42.9988 | 6560 | 1.1423 | | 1.0399 | 43.9951 | 6712 | 1.1442 | | 1.0317 | 44.9980 | 6865 | 1.1443 | | 1.0355 | 45.9943 | 7017 | 1.1427 | | 1.0259 | 46.9971 | 7170 | 1.1424 | | 1.0228 | 48.0 | 7323 | 1.1396 | | 1.0285 | 48.9963 | 7475 | 1.1434 | | 1.0179 | 49.9992 | 7628 | 1.1407 | | 1.0209 | 50.9955 | 7780 | 1.1427 | | 1.0132 | 51.9984 | 7933 | 1.1418 | | 1.0159 | 52.9947 | 8085 | 1.1344 | | 1.0058 | 53.9975 | 8238 | 1.1401 | | 1.0113 | 54.9939 | 8390 | 1.1429 | | 1.0021 | 55.9967 | 8543 | 1.1424 | | 0.9995 | 56.9996 | 8696 | 1.1426 | | 1.0048 | 57.9959 | 8848 | 1.1389 | | 0.9951 | 58.9988 | 9001 | 1.1387 | | 1.0011 | 59.9951 | 9153 | 1.1410 | | 0.9901 | 60.9980 | 9306 | 1.1399 | | 0.9925 | 61.9943 | 9458 | 1.1416 | | 0.9835 | 62.9971 | 9611 | 1.1416 | | 0.9846 | 64.0 | 9764 | 1.1458 | | 0.9878 | 64.9963 | 9916 | 1.1452 | | 0.9792 | 65.9992 | 10069 | 1.1459 | | 0.9813 | 66.9955 | 10221 | 1.1415 | | 0.9747 | 67.9984 | 10374 | 1.1476 | | 0.9764 | 68.9947 | 10526 | 1.1474 | | 0.971 | 69.9975 | 10679 | 1.1509 | | 0.9728 | 70.9939 | 10831 | 1.1441 | | 0.9672 | 71.9967 | 10984 | 1.1466 | | 0.9627 | 72.9996 | 11137 | 1.1425 | | 0.9678 | 73.9959 | 11289 | 1.1445 | | 0.9609 | 74.9988 | 11442 | 1.1435 | | 0.9636 | 75.9951 | 11594 | 1.1408 | | 0.9553 | 76.9980 | 11747 | 1.1468 | | 0.9608 | 77.9943 | 11899 | 1.1460 | | 0.9506 | 78.9971 | 12052 | 1.1475 | | 0.9505 | 80.0 | 12205 | 1.1460 | | 0.9535 | 80.9963 | 12357 | 1.1467 | | 0.9471 | 81.9992 | 12510 | 1.1495 | | 0.9509 | 82.9955 | 12662 | 1.1484 | | 0.9412 | 83.9984 | 12815 | 1.1486 | | 0.9456 | 84.9947 | 12967 | 1.1468 | | 0.9385 | 85.9975 | 13120 | 1.1497 | | 0.945 | 86.9939 | 13272 | 1.1501 | | 0.9351 | 87.9967 | 13425 | 1.1483 | | 0.9324 | 88.9996 | 13578 | 1.1497 | | 0.9376 | 89.9959 | 13730 | 1.1501 | | 0.9295 | 90.9988 | 13883 | 1.1469 | | 0.9345 | 91.9951 | 14035 | 1.1554 | | 0.9267 | 92.9980 | 14188 | 1.1485 | | 0.931 | 93.9943 | 14340 | 1.1508 | | 0.9225 | 94.9971 | 14493 | 1.1536 | | 0.9208 | 96.0 | 14646 | 1.1495 | | 0.9254 | 96.9963 | 14798 | 1.1522 | | 0.9177 | 97.9992 | 14951 | 1.1550 | | 0.9199 | 98.9955 | 15103 | 1.1575 | | 0.9144 | 99.9984 | 15256 | 1.1563 | | 0.9174 | 100.9947 | 15408 | 1.1518 | | 0.911 | 101.9975 | 15561 | 1.1560 | | 0.9135 | 102.9939 | 15713 | 1.1543 | | 0.9044 | 103.9967 | 15866 | 1.1549 | | 0.905 | 104.9996 | 16019 | 1.1568 | | 0.9106 | 105.9959 | 16171 | 1.1567 | | 0.902 | 106.9988 | 16324 | 1.1555 | | 0.9068 | 107.9951 | 16476 | 1.1580 | | 0.8973 | 108.9980 | 16629 | 1.1562 | | 0.9038 | 109.9943 | 16781 | 1.1612 | | 0.8957 | 110.9971 | 16934 | 1.1514 | | 0.8949 | 112.0 | 17087 | 1.1571 | | 0.8989 | 112.9963 | 17239 | 1.1634 | | 0.8927 | 113.9992 | 17392 | 1.1621 | | 0.8954 | 114.9955 | 17544 | 1.1572 | | 0.8876 | 115.9984 | 17697 | 1.1604 | | 0.8917 | 116.9947 | 17849 | 1.1660 | | 0.8841 | 117.9975 | 18002 | 1.1564 | | 0.8893 | 118.9939 | 18154 | 1.1624 | | 0.8808 | 119.9967 | 18307 | 1.1668 | | 0.8825 | 120.9996 | 18460 | 1.1608 | | 0.8848 | 121.9959 | 18612 | 1.1600 | | 0.878 | 122.9988 | 18765 | 1.1650 | | 0.8818 | 123.9951 | 18917 | 1.1671 | | 0.8748 | 124.9980 | 19070 | 1.1668 | | 0.8787 | 125.9943 | 19222 | 1.1605 | | 0.8727 | 126.9971 | 19375 | 1.1649 | | 0.8701 | 128.0 | 19528 | 1.1675 | | 0.875 | 128.9963 | 19680 | 1.1639 | | 0.8669 | 129.9992 | 19833 | 1.1698 | | 0.8714 | 130.9955 | 19985 | 1.1726 | | 0.8657 | 131.9984 | 20138 | 1.1680 | | 0.8682 | 132.9947 | 20290 | 1.1695 | | 0.8623 | 133.9975 | 20443 | 1.1774 | | 0.8659 | 134.9939 | 20595 | 1.1718 | | 0.8606 | 135.9967 | 20748 | 1.1691 | | 0.8587 | 136.9996 | 20901 | 1.1668 | | 0.8635 | 137.9959 | 21053 | 1.1742 | | 0.8567 | 138.9988 | 21206 | 1.1707 | | 0.8607 | 139.9951 | 21358 | 1.1756 | | 0.8519 | 140.9980 | 21511 | 1.1742 | | 0.8558 | 141.9943 | 21663 | 1.1733 | | 0.8518 | 142.9971 | 21816 | 1.1761 | | 0.85 | 144.0 | 21969 | 1.1734 | | 0.8536 | 144.9963 | 22121 | 1.1788 | | 0.8469 | 145.9992 | 22274 | 1.1782 | | 0.85 | 146.9955 | 22426 | 1.1773 | | 0.8416 | 147.9984 | 22579 | 1.1731 | | 0.8496 | 148.9947 | 22731 | 1.1767 | | 0.842 | 149.9975 | 22884 | 1.1743 | | 0.8452 | 150.9939 | 23036 | 1.1778 | | 0.8379 | 151.9967 | 23189 | 1.1843 | | 0.8379 | 152.9996 | 23342 | 1.1804 | | 0.8425 | 153.9959 | 23494 | 1.1803 | | 0.8332 | 154.9988 | 23647 | 1.1818 | | 0.8394 | 155.9951 | 23799 | 1.1805 | | 0.8307 | 156.9980 | 23952 | 1.1841 | | 0.836 | 157.9943 | 24104 | 1.1835 | | 0.8305 | 158.9971 | 24257 | 1.1823 | | 0.8298 | 160.0 | 24410 | 1.1768 | | 0.8329 | 160.9963 | 24562 | 1.1836 | | 0.8271 | 161.9992 | 24715 | 1.1841 | | 0.8316 | 162.9955 | 24867 | 1.1848 | | 0.825 | 163.9984 | 25020 | 1.1807 | | 0.8287 | 164.9947 | 25172 | 1.1866 | | 0.821 | 165.9975 | 25325 | 1.1866 | | 0.8249 | 166.9939 | 25477 | 1.1887 | | 0.8188 | 167.9967 | 25630 | 1.1882 | | 0.8192 | 168.9996 | 25783 | 1.1891 | | 0.8215 | 169.9959 | 25935 | 1.1921 | | 0.8162 | 170.9988 | 26088 | 1.1891 | | 0.8213 | 171.9951 | 26240 | 1.1929 | | 0.8145 | 172.9980 | 26393 | 1.1881 | | 0.8177 | 173.9943 | 26545 | 1.1878 | | 0.8123 | 174.9971 | 26698 | 1.1919 | | 0.8097 | 176.0 | 26851 | 1.1922 | | 0.8156 | 176.9963 | 27003 | 1.1957 | | 0.8077 | 177.9992 | 27156 | 1.1945 | | 0.812 | 178.9955 | 27308 | 1.1942 | | 0.8069 | 179.9984 | 27461 | 1.1913 | | 0.8108 | 180.9947 | 27613 | 1.1962 | | 0.8041 | 181.9975 | 27766 | 1.1992 | | 0.8072 | 182.9939 | 27918 | 1.1976 | | 0.8021 | 183.9967 | 28071 | 1.1981 | | 0.8018 | 184.9996 | 28224 | 1.1958 | | 0.8041 | 185.9959 | 28376 | 1.2022 | | 0.7978 | 186.9988 | 28529 | 1.1981 | | 0.8019 | 187.9951 | 28681 | 1.1957 | | 0.7966 | 188.9980 | 28834 | 1.1995 | | 0.7989 | 189.9943 | 28986 | 1.1947 | | 0.7928 | 190.9971 | 29139 | 1.1966 | | 0.7915 | 192.0 | 29292 | 1.2022 | | 0.7975 | 192.9963 | 29444 | 1.2062 | | 0.7918 | 193.9992 | 29597 | 1.2031 | | 0.7952 | 194.9955 | 29749 | 1.2034 | | 0.7894 | 195.9984 | 29902 | 1.2060 | | 0.791 | 196.9947 | 30054 | 1.2040 | | 0.7868 | 197.9975 | 30207 | 1.2054 | | 0.7899 | 198.9939 | 30359 | 1.2046 | | 0.7859 | 199.9967 | 30512 | 1.2023 | | 0.7851 | 200.9996 | 30665 | 1.2075 | | 0.7885 | 201.9959 | 30817 | 1.2074 | | 0.7822 | 202.9988 | 30970 | 1.2052 | | 0.7868 | 203.9951 | 31122 | 1.2048 | | 0.7809 | 204.9980 | 31275 | 1.2070 | | 0.7847 | 205.9943 | 31427 | 1.2096 | | 0.7778 | 206.9971 | 31580 | 1.2082 | | 0.7782 | 208.0 | 31733 | 1.2147 | | 0.7813 | 208.9963 | 31885 | 1.2137 | | 0.775 | 209.9992 | 32038 | 1.2115 | | 0.7785 | 210.9955 | 32190 | 1.2203 | | 0.7733 | 211.9984 | 32343 | 1.2108 | | 0.7771 | 212.9947 | 32495 | 1.2173 | | 0.7711 | 213.9975 | 32648 | 1.2123 | | 0.7765 | 214.9939 | 32800 | 1.2156 | | 0.77 | 215.9967 | 32953 | 1.2182 | | 0.7673 | 216.9996 | 33106 | 1.2223 | | 0.774 | 217.9959 | 33258 | 1.2144 | | 0.7666 | 218.9988 | 33411 | 1.2144 | | 0.7721 | 219.9951 | 33563 | 1.2165 | | 0.7646 | 220.9980 | 33716 | 1.2195 | | 0.769 | 221.9943 | 33868 | 1.2157 | | 0.7625 | 222.9971 | 34021 | 1.2166 | | 0.7619 | 224.0 | 34174 | 1.2171 | | 0.7662 | 224.9963 | 34326 | 1.2183 | | 0.7585 | 225.9992 | 34479 | 1.2243 | | 0.764 | 226.9955 | 34631 | 1.2159 | | 0.76 | 227.9984 | 34784 | 1.2215 | | 0.7619 | 228.9947 | 34936 | 1.2161 | | 0.758 | 229.9975 | 35089 | 1.2174 | | 0.7613 | 230.9939 | 35241 | 1.2236 | | 0.7547 | 231.9967 | 35394 | 1.2234 | | 0.7562 | 232.9996 | 35547 | 1.2258 | | 0.7572 | 233.9959 | 35699 | 1.2218 | | 0.7514 | 234.9988 | 35852 | 1.2235 | | 0.7559 | 235.9951 | 36004 | 1.2264 | | 0.7515 | 236.9980 | 36157 | 1.2243 | | 0.7555 | 237.9943 | 36309 | 1.2245 | | 0.7497 | 238.9971 | 36462 | 1.2238 | | 0.7467 | 240.0 | 36615 | 1.2260 | | 0.7524 | 240.9963 | 36767 | 1.2251 | | 0.7448 | 241.9992 | 36920 | 1.2267 | | 0.7498 | 242.9955 | 37072 | 1.2293 | | 0.7433 | 243.9984 | 37225 | 1.2358 | | 0.7468 | 244.9947 | 37377 | 1.2337 | | 0.7431 | 245.9975 | 37530 | 1.2285 | | 0.7474 | 246.9939 | 37682 | 1.2304 | | 0.7413 | 247.9967 | 37835 | 1.2341 | | 0.7385 | 248.9996 | 37988 | 1.2318 | | 0.7453 | 249.9959 | 38140 | 1.2336 | | 0.7377 | 250.9988 | 38293 | 1.2301 | | 0.7415 | 251.9951 | 38445 | 1.2303 | | 0.7388 | 252.9980 | 38598 | 1.2327 | | 0.7397 | 253.9943 | 38750 | 1.2364 | | 0.7347 | 254.9971 | 38903 | 1.2324 | | 0.7334 | 256.0 | 39056 | 1.2358 | | 0.7407 | 256.9963 | 39208 | 1.2335 | | 0.7322 | 257.9992 | 39361 | 1.2353 | | 0.7354 | 258.9955 | 39513 | 1.2348 | | 0.7287 | 259.9984 | 39666 | 1.2342 | | 0.7351 | 260.9947 | 39818 | 1.2341 | | 0.7294 | 261.9975 | 39971 | 1.2317 | | 0.7321 | 262.9939 | 40123 | 1.2390 | | 0.7278 | 263.9967 | 40276 | 1.2386 | | 0.7264 | 264.9996 | 40429 | 1.2357 | | 0.7303 | 265.9959 | 40581 | 1.2428 | | 0.7254 | 266.9988 | 40734 | 1.2405 | | 0.7273 | 267.9951 | 40886 | 1.2439 | | 0.7248 | 268.9980 | 41039 | 1.2351 | | 0.7293 | 269.9943 | 41191 | 1.2394 | | 0.7217 | 270.9971 | 41344 | 1.2433 | | 0.7212 | 272.0 | 41497 | 1.2461 | | 0.7256 | 272.9963 | 41649 | 1.2419 | | 0.7189 | 273.9992 | 41802 | 1.2393 | | 0.7247 | 274.9955 | 41954 | 1.2442 | | 0.7186 | 275.9984 | 42107 | 1.2400 | | 0.7242 | 276.9947 | 42259 | 1.2433 | | 0.7165 | 277.9975 | 42412 | 1.2464 | | 0.7208 | 278.9939 | 42564 | 1.2397 | | 0.7142 | 279.9967 | 42717 | 1.2488 | | 0.7161 | 280.9996 | 42870 | 1.2467 | | 0.7182 | 281.9959 | 43022 | 1.2499 | | 0.7145 | 282.9988 | 43175 | 1.2444 | | 0.7182 | 283.9951 | 43327 | 1.2507 | | 0.7117 | 284.9980 | 43480 | 1.2477 | | 0.715 | 285.9943 | 43632 | 1.2499 | | 0.7122 | 286.9971 | 43785 | 1.2483 | | 0.7101 | 288.0 | 43938 | 1.2442 | | 0.7138 | 288.9963 | 44090 | 1.2497 | | 0.7078 | 289.9992 | 44243 | 1.2477 | | 0.7111 | 290.9955 | 44395 | 1.2485 | | 0.7053 | 291.9984 | 44548 | 1.2483 | | 0.7105 | 292.9947 | 44700 | 1.2529 | | 0.7056 | 293.9975 | 44853 | 1.2566 | | 0.7088 | 294.9939 | 45005 | 1.2476 | | 0.7054 | 295.9967 | 45158 | 1.2536 | | 0.704 | 296.9996 | 45311 | 1.2519 | | 0.7082 | 297.9959 | 45463 | 1.2581 | | 0.7009 | 298.9988 | 45616 | 1.2609 | | 0.7052 | 299.9951 | 45768 | 1.2549 | | 0.6984 | 300.9980 | 45921 | 1.2517 | | 0.7056 | 301.9943 | 46073 | 1.2585 | | 0.7002 | 302.9971 | 46226 | 1.2567 | | 0.6981 | 304.0 | 46379 | 1.2573 | | 0.7016 | 304.9963 | 46531 | 1.2585 | | 0.6971 | 305.9992 | 46684 | 1.2632 | | 0.7008 | 306.9955 | 46836 | 1.2587 | | 0.6975 | 307.9984 | 46989 | 1.2580 | | 0.6984 | 308.9947 | 47141 | 1.2535 | | 0.6946 | 309.9975 | 47294 | 1.2576 | | 0.6982 | 310.9939 | 47446 | 1.2610 | | 0.6922 | 311.9967 | 47599 | 1.2632 | | 0.694 | 312.9996 | 47752 | 1.2518 | | 0.6967 | 313.9959 | 47904 | 1.2588 | | 0.6895 | 314.9988 | 48057 | 1.2643 | | 0.6954 | 315.9951 | 48209 | 1.2630 | | 0.6899 | 316.9980 | 48362 | 1.2620 | | 0.6932 | 317.9943 | 48514 | 1.2606 | | 0.6878 | 318.9971 | 48667 | 1.2632 | | 0.6895 | 320.0 | 48820 | 1.2623 | | 0.6916 | 320.9963 | 48972 | 1.2665 | | 0.6873 | 321.9992 | 49125 | 1.2636 | | 0.6914 | 322.9955 | 49277 | 1.2631 | | 0.6852 | 323.9984 | 49430 | 1.2631 | | 0.6891 | 324.9947 | 49582 | 1.2628 | | 0.6843 | 325.9975 | 49735 | 1.2654 | | 0.6875 | 326.9939 | 49887 | 1.2656 | | 0.6818 | 327.9967 | 50040 | 1.2660 | | 0.683 | 328.9996 | 50193 | 1.2654 | | 0.6866 | 329.9959 | 50345 | 1.2701 | | 0.6803 | 330.9988 | 50498 | 1.2647 | | 0.6843 | 331.9951 | 50650 | 1.2735 | | 0.68 | 332.9980 | 50803 | 1.2663 | | 0.6836 | 333.9943 | 50955 | 1.2659 | | 0.6792 | 334.9971 | 51108 | 1.2723 | | 0.6775 | 336.0 | 51261 | 1.2719 | | 0.681 | 336.9963 | 51413 | 1.2684 | | 0.6772 | 337.9992 | 51566 | 1.2722 | | 0.6806 | 338.9955 | 51718 | 1.2745 | | 0.6749 | 339.9984 | 51871 | 1.2762 | | 0.6778 | 340.9947 | 52023 | 1.2767 | | 0.6752 | 341.9975 | 52176 | 1.2727 | | 0.6783 | 342.9939 | 52328 | 1.2757 | | 0.6725 | 343.9967 | 52481 | 1.2732 | | 0.6744 | 344.9996 | 52634 | 1.2728 | | 0.6756 | 345.9959 | 52786 | 1.2736 | | 0.6709 | 346.9988 | 52939 | 1.2731 | | 0.6763 | 347.9951 | 53091 | 1.2749 | | 0.6708 | 348.9980 | 53244 | 1.2774 | | 0.673 | 349.9943 | 53396 | 1.2710 | | 0.6685 | 350.9971 | 53549 | 1.2692 | | 0.6677 | 352.0 | 53702 | 1.2675 | | 0.6711 | 352.9963 | 53854 | 1.2767 | | 0.6683 | 353.9992 | 54007 | 1.2760 | | 0.6732 | 354.9955 | 54159 | 1.2743 | | 0.6676 | 355.9984 | 54312 | 1.2797 | | 0.6713 | 356.9947 | 54464 | 1.2764 | | 0.6651 | 357.9975 | 54617 | 1.2807 | | 0.6689 | 358.9939 | 54769 | 1.2758 | | 0.6632 | 359.9967 | 54922 | 1.2839 | | 0.6632 | 360.9996 | 55075 | 1.2807 | | 0.6659 | 361.9959 | 55227 | 1.2760 | | 0.6622 | 362.9988 | 55380 | 1.2812 | | 0.6669 | 363.9951 | 55532 | 1.2761 | | 0.6616 | 364.9980 | 55685 | 1.2868 | | 0.6656 | 365.9943 | 55837 | 1.2766 | | 0.6606 | 366.9971 | 55990 | 1.2851 | | 0.659 | 368.0 | 56143 | 1.2815 | | 0.665 | 368.9963 | 56295 | 1.2810 | | 0.6585 | 369.9992 | 56448 | 1.2818 | | 0.6636 | 370.9955 | 56600 | 1.2826 | | 0.658 | 371.9984 | 56753 | 1.2799 | | 0.6633 | 372.9947 | 56905 | 1.2915 | | 0.657 | 373.9975 | 57058 | 1.2803 | | 0.6623 | 374.9939 | 57210 | 1.2872 | | 0.6561 | 375.9967 | 57363 | 1.2847 | | 0.656 | 376.9996 | 57516 | 1.2834 | | 0.6595 | 377.9959 | 57668 | 1.2858 | | 0.6546 | 378.9988 | 57821 | 1.2834 | | 0.6572 | 379.9951 | 57973 | 1.2869 | | 0.653 | 380.9980 | 58126 | 1.2772 | | 0.6566 | 381.9943 | 58278 | 1.2936 | | 0.6533 | 382.9971 | 58431 | 1.2910 | | 0.6543 | 384.0 | 58584 | 1.2846 | | 0.6555 | 384.9963 | 58736 | 1.2881 | | 0.6508 | 385.9992 | 58889 | 1.2898 | | 0.6547 | 386.9955 | 59041 | 1.2879 | | 0.6496 | 387.9984 | 59194 | 1.2865 | | 0.6531 | 388.9947 | 59346 | 1.2861 | | 0.6481 | 389.9975 | 59499 | 1.2832 | | 0.6539 | 390.9939 | 59651 | 1.2895 | | 0.6476 | 391.9967 | 59804 | 1.2838 | | 0.6489 | 392.9996 | 59957 | 1.2923 | | 0.6519 | 393.9959 | 60109 | 1.2871 | | 0.647 | 394.9988 | 60262 | 1.2846 | | 0.6491 | 395.9951 | 60414 | 1.2914 | | 0.6459 | 396.9980 | 60567 | 1.2886 | | 0.6496 | 397.9943 | 60719 | 1.2891 | | 0.6452 | 398.9971 | 60872 | 1.2861 | | 0.6439 | 400.0 | 61025 | 1.2917 | | 0.6484 | 400.9963 | 61177 | 1.2934 | | 0.6446 | 401.9992 | 61330 | 1.2872 | | 0.6493 | 402.9955 | 61482 | 1.2900 | | 0.6423 | 403.9984 | 61635 | 1.2940 | | 0.6469 | 404.9947 | 61787 | 1.2867 | | 0.6412 | 405.9975 | 61940 | 1.2958 | | 0.6468 | 406.9939 | 62092 | 1.2906 | | 0.6428 | 407.9967 | 62245 | 1.2904 | | 0.6409 | 408.9996 | 62398 | 1.2924 | | 0.6464 | 409.9959 | 62550 | 1.2953 | | 0.6404 | 410.9988 | 62703 | 1.2918 | | 0.6452 | 411.9951 | 62855 | 1.2894 | | 0.6406 | 412.9980 | 63008 | 1.2975 | | 0.6442 | 413.9943 | 63160 | 1.2928 | | 0.638 | 414.9971 | 63313 | 1.2948 | | 0.6379 | 416.0 | 63466 | 1.2936 | | 0.6416 | 416.9963 | 63618 | 1.2892 | | 0.639 | 417.9992 | 63771 | 1.2959 | | 0.6414 | 418.9955 | 63923 | 1.2940 | | 0.6363 | 419.9984 | 64076 | 1.2949 | | 0.6409 | 420.9947 | 64228 | 1.2943 | | 0.6346 | 421.9975 | 64381 | 1.2974 | | 0.6393 | 422.9939 | 64533 | 1.3000 | | 0.6331 | 423.9967 | 64686 | 1.2944 | | 0.636 | 424.9996 | 64839 | 1.2915 | | 0.6383 | 425.9959 | 64991 | 1.2986 | | 0.6338 | 426.9988 | 65144 | 1.2981 | | 0.6378 | 427.9951 | 65296 | 1.2980 | | 0.634 | 428.9980 | 65449 | 1.2958 | | 0.6374 | 429.9943 | 65601 | 1.2959 | | 0.6312 | 430.9971 | 65754 | 1.2918 | | 0.6317 | 432.0 | 65907 | 1.2972 | | 0.6352 | 432.9963 | 66059 | 1.2970 | | 0.6319 | 433.9992 | 66212 | 1.2969 | | 0.6334 | 434.9955 | 66364 | 1.2997 | | 0.6296 | 435.9984 | 66517 | 1.2967 | | 0.6352 | 436.9947 | 66669 | 1.2979 | | 0.6302 | 437.9975 | 66822 | 1.2999 | | 0.6323 | 438.9939 | 66974 | 1.2989 | | 0.6287 | 439.9967 | 67127 | 1.2933 | | 0.6295 | 440.9996 | 67280 | 1.2979 | | 0.6335 | 441.9959 | 67432 | 1.2979 | | 0.6273 | 442.9988 | 67585 | 1.2917 | | 0.6308 | 443.9951 | 67737 | 1.3001 | | 0.6278 | 444.9980 | 67890 | 1.2948 | | 0.6303 | 445.9943 | 68042 | 1.3005 | | 0.6278 | 446.9971 | 68195 | 1.2962 | | 0.6274 | 448.0 | 68348 | 1.2969 | | 0.6287 | 448.9963 | 68500 | 1.2953 | | 0.6276 | 449.9992 | 68653 | 1.2983 | | 0.629 | 450.9955 | 68805 | 1.3040 | | 0.6249 | 451.9984 | 68958 | 1.2992 | | 0.6307 | 452.9947 | 69110 | 1.2992 | | 0.626 | 453.9975 | 69263 | 1.2975 | | 0.6283 | 454.9939 | 69415 | 1.2983 | | 0.6262 | 455.9967 | 69568 | 1.3002 | | 0.6217 | 456.9996 | 69721 | 1.3029 | | 0.6284 | 457.9959 | 69873 | 1.3001 | | 0.6238 | 458.9988 | 70026 | 1.3011 | | 0.6258 | 459.9951 | 70178 | 1.2993 | | 0.6217 | 460.9980 | 70331 | 1.2971 | | 0.6265 | 461.9943 | 70483 | 1.2996 | | 0.622 | 462.9971 | 70636 | 1.2977 | | 0.6228 | 464.0 | 70789 | 1.2981 | | 0.6274 | 464.9963 | 70941 | 1.3028 | | 0.6218 | 465.9992 | 71094 | 1.2995 | | 0.6245 | 466.9955 | 71246 | 1.2990 | | 0.621 | 467.9984 | 71399 | 1.3032 | | 0.6254 | 468.9947 | 71551 | 1.2992 | | 0.6217 | 469.9975 | 71704 | 1.2964 | | 0.6236 | 470.9939 | 71856 | 1.3012 | | 0.6216 | 471.9967 | 72009 | 1.3004 | | 0.6191 | 472.9996 | 72162 | 1.3032 | | 0.6234 | 473.9959 | 72314 | 1.3043 | | 0.6202 | 474.9988 | 72467 | 1.3015 | | 0.6248 | 475.9951 | 72619 | 1.3018 | | 0.6194 | 476.9980 | 72772 | 1.3030 | | 0.6217 | 477.9943 | 72924 | 1.3040 | | 0.6193 | 478.9971 | 73077 | 1.3058 | | 0.6198 | 480.0 | 73230 | 1.2999 | | 0.6219 | 480.9963 | 73382 | 1.3016 | | 0.6165 | 481.9992 | 73535 | 1.3048 | | 0.6223 | 482.9955 | 73687 | 1.3044 | | 0.6165 | 483.9984 | 73840 | 1.3040 | | 0.6223 | 484.9947 | 73992 | 1.3059 | | 0.6179 | 485.9975 | 74145 | 1.2996 | | 0.621 | 486.9939 | 74297 | 1.3052 | | 0.6173 | 487.9967 | 74450 | 1.3019 | | 0.6179 | 488.9996 | 74603 | 1.3009 | | 0.6195 | 489.9959 | 74755 | 1.3023 | | 0.6177 | 490.9988 | 74908 | 1.2976 | | 0.6214 | 491.9951 | 75060 | 1.3044 | | 0.6168 | 492.9980 | 75213 | 1.3022 | | 0.6189 | 493.9943 | 75365 | 1.3029 | | 0.6182 | 494.9971 | 75518 | 1.3043 | | 0.6168 | 496.0 | 75671 | 1.3027 | | 0.6222 | 496.9963 | 75823 | 1.3022 | | 0.6155 | 497.9992 | 75976 | 1.3005 | | 0.6144 | 498.1565 | 76000 | 1.2966 | ### Framework versions - Transformers 4.41.2 - Pytorch 2.2.1 - Datasets 2.19.2 - Tokenizers 0.19.1