--- license: mit base_model: cardiffnlp/twitter-roberta-base-2019-90m tags: - generated_from_trainer model-index: - name: 2020-Q2-full_tweets_combined90 results: [] --- # 2020-Q2-full_tweets_combined90 This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.9422 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 4.1e-07 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 - lr_scheduler_type: linear - training_steps: 2400000 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:-------:|:---------------:| | No log | 0.02 | 8000 | 2.2220 | | 2.4154 | 0.03 | 16000 | 2.1427 | | 2.4154 | 0.05 | 24000 | 2.1028 | | 2.2273 | 0.07 | 32000 | 2.0824 | | 2.2273 | 0.08 | 40000 | 2.0645 | | 2.1774 | 0.1 | 48000 | 2.0478 | | 2.1774 | 0.12 | 56000 | 2.0327 | | 2.1569 | 0.13 | 64000 | 2.0248 | | 2.1569 | 0.15 | 72000 | 2.0209 | | 2.1439 | 0.17 | 80000 | 2.0049 | | 2.1439 | 0.19 | 88000 | 2.0113 | | 2.1271 | 0.2 | 96000 | 2.0038 | | 2.1271 | 0.22 | 104000 | 2.0065 | | 2.1211 | 0.24 | 112000 | 1.9987 | | 2.1211 | 0.25 | 120000 | 1.9929 | | 2.1194 | 0.27 | 128000 | 1.9922 | | 2.1194 | 0.29 | 136000 | 1.9917 | | 2.1118 | 0.3 | 144000 | 1.9885 | | 2.1118 | 0.32 | 152000 | 1.9870 | | 2.1047 | 0.34 | 160000 | 1.9843 | | 2.1047 | 0.35 | 168000 | 1.9827 | | 2.1015 | 0.37 | 176000 | 1.9826 | | 2.1015 | 0.39 | 184000 | 1.9774 | | 2.1042 | 0.4 | 192000 | 1.9771 | | 2.1042 | 0.42 | 200000 | 1.9770 | | 2.0919 | 0.44 | 208000 | 1.9752 | | 2.0919 | 0.45 | 216000 | 1.9775 | | 2.0953 | 0.47 | 224000 | 1.9684 | | 2.0953 | 0.49 | 232000 | 1.9748 | | 2.0848 | 0.51 | 240000 | 1.9714 | | 2.0848 | 0.52 | 248000 | 1.9781 | | 2.0882 | 0.54 | 256000 | 1.9709 | | 2.0882 | 0.56 | 264000 | 1.9660 | | 2.0922 | 0.57 | 272000 | 1.9651 | | 2.0922 | 0.59 | 280000 | 1.9678 | | 2.0938 | 0.61 | 288000 | 1.9667 | | 2.0938 | 0.62 | 296000 | 1.9630 | | 2.095 | 0.64 | 304000 | 1.9642 | | 2.095 | 0.66 | 312000 | 1.9624 | | 2.0908 | 0.67 | 320000 | 1.9603 | | 2.0908 | 0.69 | 328000 | 1.9649 | | 2.0927 | 0.71 | 336000 | 1.9641 | | 2.0927 | 0.72 | 344000 | 1.9603 | | 2.0931 | 0.74 | 352000 | 1.9590 | | 2.0931 | 0.76 | 360000 | 1.9644 | | 2.087 | 0.77 | 368000 | 1.9635 | | 2.087 | 0.79 | 376000 | 1.9614 | | 2.0792 | 0.81 | 384000 | 1.9591 | | 2.0792 | 0.83 | 392000 | 1.9575 | | 2.0899 | 0.84 | 400000 | 1.9592 | | 2.0899 | 0.86 | 408000 | 1.9619 | | 2.0812 | 0.88 | 416000 | 1.9582 | | 2.0812 | 0.89 | 424000 | 1.9580 | | 2.0948 | 0.91 | 432000 | 1.9587 | | 2.0948 | 0.93 | 440000 | 1.9593 | | 2.0895 | 0.94 | 448000 | 1.9608 | | 2.0895 | 0.96 | 456000 | 1.9566 | | 2.0756 | 0.98 | 464000 | 1.9525 | | 2.0756 | 0.99 | 472000 | 1.9541 | | 2.0842 | 1.01 | 480000 | 1.9601 | | 2.0842 | 1.03 | 488000 | 1.9564 | | 2.0935 | 1.04 | 496000 | 1.9522 | | 2.0935 | 1.06 | 504000 | 1.9532 | | 2.0836 | 1.08 | 512000 | 1.9537 | | 2.0836 | 1.09 | 520000 | 1.9553 | | 2.0876 | 1.11 | 528000 | 1.9469 | | 2.0876 | 1.13 | 536000 | 1.9497 | | 2.0778 | 1.15 | 544000 | 1.9542 | | 2.0778 | 1.16 | 552000 | 1.9516 | | 2.0829 | 1.18 | 560000 | 1.9506 | | 2.0829 | 1.2 | 568000 | 1.9505 | | 2.0864 | 1.21 | 576000 | 1.9531 | | 2.0864 | 1.23 | 584000 | 1.9455 | | 2.0893 | 1.25 | 592000 | 1.9471 | | 2.0893 | 1.26 | 600000 | 1.9539 | | 2.0808 | 1.28 | 608000 | 1.9455 | | 2.0808 | 1.3 | 616000 | 1.9497 | | 2.0838 | 1.31 | 624000 | 1.9466 | | 2.0838 | 1.33 | 632000 | 1.9498 | | 2.0812 | 1.35 | 640000 | 1.9510 | | 2.0812 | 1.36 | 648000 | 1.9526 | | 2.0793 | 1.38 | 656000 | 1.9471 | | 2.0793 | 1.4 | 664000 | 1.9469 | | 2.0789 | 1.41 | 672000 | 1.9455 | | 2.0789 | 1.43 | 680000 | 1.9469 | | 2.0883 | 1.45 | 688000 | 1.9439 | | 2.0883 | 1.47 | 696000 | 1.9439 | | 2.09 | 1.48 | 704000 | 1.9416 | | 2.09 | 1.5 | 712000 | 1.9492 | | 2.0845 | 1.52 | 720000 | 1.9430 | | 2.0845 | 1.53 | 728000 | 1.9484 | | 2.0742 | 1.55 | 736000 | 1.9456 | | 2.0742 | 1.57 | 744000 | 1.9380 | | 2.0839 | 1.58 | 752000 | 1.9418 | | 2.0839 | 1.6 | 760000 | 1.9434 | | 2.0806 | 1.62 | 768000 | 1.9450 | | 2.0806 | 1.63 | 776000 | 1.9426 | | 2.0805 | 1.65 | 784000 | 1.9441 | | 2.0805 | 1.67 | 792000 | 1.9459 | | 2.0833 | 1.68 | 800000 | 1.9435 | | 2.0833 | 1.7 | 808000 | 1.9455 | | 2.0763 | 1.72 | 816000 | 1.9421 | | 2.0763 | 1.73 | 824000 | 1.9438 | | 2.0758 | 1.75 | 832000 | 1.9371 | | 2.0758 | 1.77 | 840000 | 1.9432 | | 2.0888 | 1.79 | 848000 | 1.9414 | | 2.0888 | 1.8 | 856000 | 1.9444 | | 2.0786 | 1.82 | 864000 | 1.9408 | | 2.0786 | 1.84 | 872000 | 1.9397 | | 2.079 | 1.85 | 880000 | 1.9406 | | 2.079 | 1.87 | 888000 | 1.9442 | | 2.0817 | 1.89 | 896000 | 1.9404 | | 2.0817 | 1.9 | 904000 | 1.9450 | | 2.0792 | 1.92 | 912000 | 1.9380 | | 2.0792 | 1.94 | 920000 | 1.9385 | | 2.0741 | 1.95 | 928000 | 1.9449 | | 2.0741 | 1.97 | 936000 | 1.9414 | | 2.0832 | 1.99 | 944000 | 1.9402 | | 2.0832 | 2.0 | 952000 | 1.9410 | | 2.0695 | 2.02 | 960000 | 1.9371 | | 2.0695 | 2.04 | 968000 | 1.9342 | | 2.0813 | 2.05 | 976000 | 1.9376 | | 2.0813 | 2.07 | 984000 | 1.9397 | | 2.0804 | 2.09 | 992000 | 1.9394 | | 2.0804 | 2.11 | 1000000 | 1.9370 | | 2.0789 | 2.12 | 1008000 | 1.9350 | | 2.0789 | 2.14 | 1016000 | 1.9327 | | 2.0754 | 2.16 | 1024000 | 1.9421 | | 2.0754 | 2.17 | 1032000 | 1.9371 | | 2.0774 | 2.19 | 1040000 | 1.9411 | | 2.0774 | 2.21 | 1048000 | 1.9337 | | 2.0766 | 2.22 | 1056000 | 1.9387 | | 2.0766 | 2.24 | 1064000 | 1.9334 | | 2.079 | 2.26 | 1072000 | 1.9386 | | 2.079 | 2.27 | 1080000 | 1.9335 | | 2.068 | 2.29 | 1088000 | 1.9363 | | 2.068 | 2.31 | 1096000 | 1.9420 | | 2.0786 | 2.32 | 1104000 | 1.9331 | | 2.0786 | 2.34 | 1112000 | 1.9327 | | 2.0734 | 2.36 | 1120000 | 1.9391 | | 2.0734 | 2.37 | 1128000 | 1.9363 | | 2.0787 | 2.39 | 1136000 | 1.9321 | | 2.0787 | 2.41 | 1144000 | 1.9333 | | 2.0731 | 2.43 | 1152000 | 1.9369 | | 2.0731 | 2.44 | 1160000 | 1.9357 | | 2.0816 | 2.46 | 1168000 | 1.9353 | | 2.0816 | 2.48 | 1176000 | 1.9319 | | 2.0758 | 2.49 | 1184000 | 1.9366 | | 2.0758 | 2.51 | 1192000 | 1.9301 | | 2.0725 | 2.53 | 1200000 | 1.9329 | | 2.0725 | 2.54 | 1208000 | 1.9370 | | 2.085 | 2.56 | 1216000 | 1.9251 | | 2.085 | 2.58 | 1224000 | 1.9369 | | 2.0809 | 2.59 | 1232000 | 1.9377 | | 2.0809 | 2.61 | 1240000 | 1.9398 | | 2.0742 | 2.63 | 1248000 | 1.9368 | | 2.0742 | 2.64 | 1256000 | 1.9389 | | 2.0743 | 2.66 | 1264000 | 1.9287 | | 2.0743 | 2.68 | 1272000 | 1.9337 | | 2.0822 | 2.69 | 1280000 | 1.9323 | | 2.0822 | 2.71 | 1288000 | 1.9348 | | 2.0845 | 2.73 | 1296000 | 1.9328 | | 2.0845 | 2.75 | 1304000 | 1.9324 | | 2.0706 | 2.76 | 1312000 | 1.9304 | | 2.0706 | 2.78 | 1320000 | 1.9322 | | 2.0813 | 2.8 | 1328000 | 1.9320 | | 2.0813 | 2.81 | 1336000 | 1.9379 | | 2.0768 | 2.83 | 1344000 | 1.9283 | | 2.0768 | 2.85 | 1352000 | 1.9352 | | 2.0776 | 2.86 | 1360000 | 1.9266 | | 2.0776 | 2.88 | 1368000 | 1.9339 | | 2.0776 | 2.9 | 1376000 | 1.9371 | | 2.0776 | 2.91 | 1384000 | 1.9353 | | 2.072 | 2.93 | 1392000 | 1.9290 | | 2.072 | 2.95 | 1400000 | 1.9337 | | 2.077 | 2.96 | 1408000 | 1.9318 | | 2.077 | 2.98 | 1416000 | 1.9326 | | 2.0777 | 3.0 | 1424000 | 1.9338 | | 2.0777 | 3.01 | 1432000 | 1.9307 | | 2.0846 | 3.03 | 1440000 | 1.9305 | | 2.0846 | 3.05 | 1448000 | 1.9312 | | 2.0744 | 3.07 | 1456000 | 1.9332 | | 2.0744 | 3.08 | 1464000 | 1.9313 | | 2.0767 | 3.1 | 1472000 | 1.9311 | | 2.0767 | 3.12 | 1480000 | 1.9322 | | 2.082 | 3.13 | 1488000 | 1.9362 | | 2.082 | 3.15 | 1496000 | 1.9329 | | 2.0774 | 3.17 | 1504000 | 1.9335 | | 2.0774 | 3.18 | 1512000 | 1.9342 | | 2.0793 | 3.2 | 1520000 | 1.9326 | | 2.0793 | 3.22 | 1528000 | 1.9313 | | 2.0834 | 3.23 | 1536000 | 1.9302 | | 2.0834 | 3.25 | 1544000 | 1.9299 | | 2.0698 | 3.27 | 1552000 | 1.9288 | | 2.0698 | 3.28 | 1560000 | 1.9311 | | 2.0721 | 3.3 | 1568000 | 1.9262 | | 2.0721 | 3.32 | 1576000 | 1.9320 | | 2.0742 | 3.33 | 1584000 | 1.9278 | | 2.0742 | 3.35 | 1592000 | 1.9333 | | 2.0774 | 3.37 | 1600000 | 1.9252 | | 2.0774 | 3.39 | 1608000 | 1.9301 | | 2.0766 | 3.4 | 1616000 | 1.9344 | | 2.0766 | 3.42 | 1624000 | 1.9320 | | 2.0702 | 3.44 | 1632000 | 1.9307 | | 2.0702 | 3.45 | 1640000 | 1.9304 | | 2.0772 | 3.47 | 1648000 | 1.9280 | | 2.0772 | 3.49 | 1656000 | 1.9324 | | 2.0757 | 3.5 | 1664000 | 1.9343 | | 2.0757 | 3.52 | 1672000 | 1.9312 | | 2.0747 | 3.54 | 1680000 | 1.9304 | | 2.0747 | 3.55 | 1688000 | 1.9360 | | 2.068 | 3.57 | 1696000 | 1.9297 | | 2.068 | 3.59 | 1704000 | 1.9337 | | 2.0825 | 3.6 | 1712000 | 1.9293 | | 2.0825 | 3.62 | 1720000 | 1.9295 | | 2.0811 | 3.64 | 1728000 | 1.9315 | | 2.0811 | 3.65 | 1736000 | 1.9279 | | 2.0844 | 3.67 | 1744000 | 1.9289 | | 2.0844 | 3.69 | 1752000 | 1.9279 | | 2.0827 | 3.71 | 1760000 | 1.9283 | | 2.0827 | 3.72 | 1768000 | 1.9295 | | 2.0684 | 3.74 | 1776000 | 1.9281 | | 2.0684 | 3.76 | 1784000 | 1.9330 | | 2.0724 | 3.77 | 1792000 | 1.9294 | | 2.0724 | 3.79 | 1800000 | 1.9276 | | 2.074 | 3.81 | 1808000 | 1.9227 | | 2.074 | 3.82 | 1816000 | 1.9320 | | 2.0801 | 3.84 | 1824000 | 1.9275 | | 2.0801 | 3.86 | 1832000 | 1.9302 | | 2.0783 | 3.87 | 1840000 | 1.9333 | | 2.0783 | 3.89 | 1848000 | 1.9296 | | 2.0787 | 3.91 | 1856000 | 1.9302 | | 2.0787 | 3.92 | 1864000 | 1.9347 | | 2.0733 | 3.94 | 1872000 | 1.9298 | | 2.0733 | 3.96 | 1880000 | 1.9302 | | 2.0742 | 3.97 | 1888000 | 1.9279 | | 2.0742 | 3.99 | 1896000 | 1.9258 | | 2.0769 | 4.01 | 1904000 | 1.9255 | | 2.0769 | 4.03 | 1912000 | 1.9282 | | 2.0736 | 4.04 | 1920000 | 1.9298 | | 2.0736 | 4.06 | 1928000 | 1.9325 | | 2.0713 | 4.08 | 1936000 | 1.9296 | | 2.0713 | 4.09 | 1944000 | 1.9293 | | 2.0825 | 4.11 | 1952000 | 1.9345 | | 2.0825 | 4.13 | 1960000 | 1.9346 | | 2.0828 | 4.14 | 1968000 | 1.9311 | | 2.0828 | 4.16 | 1976000 | 1.9307 | | 2.0821 | 4.18 | 1984000 | 1.9336 | | 2.0821 | 4.19 | 1992000 | 1.9265 | | 2.0768 | 4.21 | 2000000 | 1.9284 | | 2.0768 | 4.23 | 2008000 | 1.9290 | | 2.0695 | 4.24 | 2016000 | 1.9306 | | 2.0695 | 4.26 | 2024000 | 1.9299 | | 2.0698 | 4.28 | 2032000 | 1.9230 | | 2.0698 | 4.29 | 2040000 | 1.9272 | | 2.0776 | 4.31 | 2048000 | 1.9306 | | 2.0776 | 4.33 | 2056000 | 1.9243 | | 2.0797 | 4.35 | 2064000 | 1.9266 | | 2.0797 | 4.36 | 2072000 | 1.9249 | | 2.0808 | 4.38 | 2080000 | 1.9279 | | 2.0808 | 4.4 | 2088000 | 1.9262 | | 2.0776 | 4.41 | 2096000 | 1.9350 | | 2.0776 | 4.43 | 2104000 | 1.9297 | | 2.0805 | 4.45 | 2112000 | 1.9337 | | 2.0805 | 4.46 | 2120000 | 1.9302 | | 2.0791 | 4.48 | 2128000 | 1.9337 | | 2.0791 | 4.5 | 2136000 | 1.9298 | | 2.0771 | 4.51 | 2144000 | 1.9268 | | 2.0771 | 4.53 | 2152000 | 1.9370 | | 2.0807 | 4.55 | 2160000 | 1.9307 | | 2.0807 | 4.56 | 2168000 | 1.9292 | | 2.0856 | 4.58 | 2176000 | 1.9300 | | 2.0856 | 4.6 | 2184000 | 1.9329 | | 2.0744 | 4.61 | 2192000 | 1.9319 | | 2.0744 | 4.63 | 2200000 | 1.9352 | | 2.0839 | 4.65 | 2208000 | 1.9368 | | 2.0839 | 4.67 | 2216000 | 1.9343 | | 2.0706 | 4.68 | 2224000 | 1.9290 | | 2.0706 | 4.7 | 2232000 | 1.9347 | | 2.0745 | 4.72 | 2240000 | 1.9294 | | 2.0745 | 4.73 | 2248000 | 1.9255 | | 2.0767 | 4.75 | 2256000 | 1.9271 | | 2.0767 | 4.77 | 2264000 | 1.9296 | | 2.0753 | 4.78 | 2272000 | 1.9268 | | 2.0753 | 4.8 | 2280000 | 1.9292 | | 2.0716 | 4.82 | 2288000 | 1.9310 | | 2.0716 | 4.83 | 2296000 | 1.9267 | | 2.0778 | 4.85 | 2304000 | 1.9301 | | 2.0778 | 4.87 | 2312000 | 1.9280 | | 2.0724 | 4.88 | 2320000 | 1.9283 | | 2.0724 | 4.9 | 2328000 | 1.9289 | | 2.0811 | 4.92 | 2336000 | 1.9315 | | 2.0811 | 4.93 | 2344000 | 1.9268 | | 2.0816 | 4.95 | 2352000 | 1.9304 | | 2.0816 | 4.97 | 2360000 | 1.9302 | | 2.0775 | 4.99 | 2368000 | 1.9292 | | 2.0775 | 5.0 | 2376000 | 1.9274 | | 2.0807 | 5.02 | 2384000 | 1.9317 | | 2.0807 | 5.04 | 2392000 | 1.9298 | | 2.0668 | 5.05 | 2400000 | 1.9349 | ### Framework versions - Transformers 4.35.0.dev0 - Pytorch 2.0.1+cu117 - Datasets 2.14.5 - Tokenizers 0.14.0