|
--- |
|
license: mit |
|
base_model: cardiffnlp/twitter-roberta-base-2019-90m |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: 2020-Q3-50p-filtered |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# 2020-Q3-50p-filtered |
|
|
|
This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 2.6316 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 4.1e-07 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- training_steps: 2400000 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-----:|:-------:|:---------------:| |
|
| No log | 0.03 | 8000 | 2.9380 | |
|
| 3.1308 | 0.07 | 16000 | 2.8593 | |
|
| 3.1308 | 0.1 | 24000 | 2.8063 | |
|
| 2.9519 | 0.14 | 32000 | 2.7832 | |
|
| 2.9519 | 0.17 | 40000 | 2.7521 | |
|
| 2.889 | 0.2 | 48000 | 2.7322 | |
|
| 2.889 | 0.24 | 56000 | 2.7259 | |
|
| 2.8592 | 0.27 | 64000 | 2.7218 | |
|
| 2.8592 | 0.3 | 72000 | 2.7106 | |
|
| 2.8345 | 0.34 | 80000 | 2.7071 | |
|
| 2.8345 | 0.37 | 88000 | 2.6903 | |
|
| 2.8303 | 0.41 | 96000 | 2.7001 | |
|
| 2.8303 | 0.44 | 104000 | 2.6929 | |
|
| 2.824 | 0.47 | 112000 | 2.6907 | |
|
| 2.824 | 0.51 | 120000 | 2.6852 | |
|
| 2.8223 | 0.54 | 128000 | 2.6804 | |
|
| 2.8223 | 0.57 | 136000 | 2.6727 | |
|
| 2.8141 | 0.61 | 144000 | 2.6784 | |
|
| 2.8141 | 0.64 | 152000 | 2.6775 | |
|
| 2.8124 | 0.68 | 160000 | 2.6723 | |
|
| 2.8124 | 0.71 | 168000 | 2.6683 | |
|
| 2.8042 | 0.74 | 176000 | 2.6712 | |
|
| 2.8042 | 0.78 | 184000 | 2.6661 | |
|
| 2.8051 | 0.81 | 192000 | 2.6783 | |
|
| 2.8051 | 0.85 | 200000 | 2.6683 | |
|
| 2.798 | 0.88 | 208000 | 2.6656 | |
|
| 2.798 | 0.91 | 216000 | 2.6659 | |
|
| 2.8043 | 0.95 | 224000 | 2.6700 | |
|
| 2.8043 | 0.98 | 232000 | 2.6680 | |
|
| 2.8055 | 1.01 | 240000 | 2.6597 | |
|
| 2.8055 | 1.05 | 248000 | 2.6597 | |
|
| 2.8048 | 1.08 | 256000 | 2.6569 | |
|
| 2.8048 | 1.12 | 264000 | 2.6502 | |
|
| 2.806 | 1.15 | 272000 | 2.6593 | |
|
| 2.806 | 1.18 | 280000 | 2.6597 | |
|
| 2.8012 | 1.22 | 288000 | 2.6604 | |
|
| 2.8012 | 1.25 | 296000 | 2.6545 | |
|
| 2.8029 | 1.28 | 304000 | 2.6571 | |
|
| 2.8029 | 1.32 | 312000 | 2.6534 | |
|
| 2.7991 | 1.35 | 320000 | 2.6650 | |
|
| 2.7991 | 1.39 | 328000 | 2.6680 | |
|
| 2.7949 | 1.42 | 336000 | 2.6544 | |
|
| 2.7949 | 1.45 | 344000 | 2.6460 | |
|
| 2.7972 | 1.49 | 352000 | 2.6553 | |
|
| 2.7972 | 1.52 | 360000 | 2.6428 | |
|
| 2.7924 | 1.56 | 368000 | 2.6536 | |
|
| 2.7924 | 1.59 | 376000 | 2.6550 | |
|
| 2.805 | 1.62 | 384000 | 2.6524 | |
|
| 2.805 | 1.66 | 392000 | 2.6524 | |
|
| 2.7972 | 1.69 | 400000 | 2.6579 | |
|
| 2.7972 | 1.72 | 408000 | 2.6500 | |
|
| 2.8003 | 1.76 | 416000 | 2.6526 | |
|
| 2.8003 | 1.79 | 424000 | 2.6444 | |
|
| 2.8005 | 1.83 | 432000 | 2.6463 | |
|
| 2.8005 | 1.86 | 440000 | 2.6549 | |
|
| 2.7957 | 1.89 | 448000 | 2.6530 | |
|
| 2.7957 | 1.93 | 456000 | 2.6504 | |
|
| 2.7949 | 1.96 | 464000 | 2.6480 | |
|
| 2.7949 | 1.99 | 472000 | 2.6497 | |
|
| 2.7978 | 2.03 | 480000 | 2.6490 | |
|
| 2.7978 | 2.06 | 488000 | 2.6505 | |
|
| 2.8041 | 2.1 | 496000 | 2.6388 | |
|
| 2.8041 | 2.13 | 504000 | 2.6460 | |
|
| 2.7935 | 2.16 | 512000 | 2.6519 | |
|
| 2.7935 | 2.2 | 520000 | 2.6494 | |
|
| 2.7982 | 2.23 | 528000 | 2.6550 | |
|
| 2.7982 | 2.27 | 536000 | 2.6460 | |
|
| 2.7949 | 2.3 | 544000 | 2.6497 | |
|
| 2.7949 | 2.33 | 552000 | 2.6478 | |
|
| 2.7953 | 2.37 | 560000 | 2.6487 | |
|
| 2.7953 | 2.4 | 568000 | 2.6400 | |
|
| 2.7942 | 2.43 | 576000 | 2.6440 | |
|
| 2.7942 | 2.47 | 584000 | nan | |
|
| 2.803 | 2.5 | 592000 | 2.6455 | |
|
| 2.803 | 2.54 | 600000 | 2.6401 | |
|
| 2.7961 | 2.57 | 608000 | 2.6511 | |
|
| 2.7961 | 2.6 | 616000 | 2.6401 | |
|
| 2.7975 | 2.64 | 624000 | 2.6437 | |
|
| 2.7975 | 2.67 | 632000 | 2.6432 | |
|
| 2.7946 | 2.7 | 640000 | 2.6461 | |
|
| 2.7946 | 2.74 | 648000 | 2.6491 | |
|
| 2.7963 | 2.77 | 656000 | 2.6442 | |
|
| 2.7963 | 2.81 | 664000 | 2.6416 | |
|
| 2.7924 | 2.84 | 672000 | 2.6403 | |
|
| 2.7924 | 2.87 | 680000 | 2.6466 | |
|
| 2.8004 | 2.91 | 688000 | 2.6436 | |
|
| 2.8004 | 2.94 | 696000 | 2.6447 | |
|
| 2.8039 | 2.98 | 704000 | 2.6412 | |
|
| 2.8039 | 3.01 | 712000 | 2.6400 | |
|
| 2.7958 | 3.04 | 720000 | 2.6419 | |
|
| 2.7958 | 3.08 | 728000 | 2.6413 | |
|
| 2.7967 | 3.11 | 736000 | nan | |
|
| 2.7967 | 3.14 | 744000 | 2.6399 | |
|
| 2.7934 | 3.18 | 752000 | 2.6405 | |
|
| 2.7934 | 3.21 | 760000 | 2.6387 | |
|
| 2.7988 | 3.25 | 768000 | 2.6463 | |
|
| 2.7988 | 3.28 | 776000 | 2.6308 | |
|
| 2.793 | 3.31 | 784000 | 2.6343 | |
|
| 2.793 | 3.35 | 792000 | 2.6358 | |
|
| 2.797 | 3.38 | 800000 | 2.6397 | |
|
| 2.797 | 3.41 | 808000 | 2.6341 | |
|
| 2.7832 | 3.45 | 816000 | 2.6394 | |
|
| 2.7832 | 3.48 | 824000 | 2.6341 | |
|
| 2.792 | 3.52 | 832000 | 2.6424 | |
|
| 2.792 | 3.55 | 840000 | 2.6380 | |
|
| 2.7945 | 3.58 | 848000 | 2.6373 | |
|
| 2.7945 | 3.62 | 856000 | 2.6366 | |
|
| 2.7876 | 3.65 | 864000 | 2.6409 | |
|
| 2.7876 | 3.69 | 872000 | 2.6382 | |
|
| 2.7975 | 3.72 | 880000 | 2.6259 | |
|
| 2.7975 | 3.75 | 888000 | 2.6443 | |
|
| 2.7965 | 3.79 | 896000 | 2.6248 | |
|
| 2.7965 | 3.82 | 904000 | 2.6395 | |
|
| 2.7991 | 3.85 | 912000 | 2.6325 | |
|
| 2.7991 | 3.89 | 920000 | 2.6354 | |
|
| 2.7947 | 3.92 | 928000 | 2.6342 | |
|
| 2.7947 | 3.96 | 936000 | 2.6290 | |
|
| 2.7977 | 3.99 | 944000 | 2.6315 | |
|
| 2.7977 | 4.02 | 952000 | 2.6347 | |
|
| 2.8 | 4.06 | 960000 | 2.6318 | |
|
| 2.8 | 4.09 | 968000 | 2.6328 | |
|
| 2.7945 | 4.12 | 976000 | 2.6315 | |
|
| 2.7945 | 4.16 | 984000 | 2.6297 | |
|
| 2.7946 | 4.19 | 992000 | 2.6378 | |
|
| 2.7946 | 4.23 | 1000000 | 2.6328 | |
|
| 2.7962 | 4.26 | 1008000 | 2.6296 | |
|
| 2.7962 | 4.29 | 1016000 | 2.6347 | |
|
| 2.7932 | 4.33 | 1024000 | 2.6355 | |
|
| 2.7932 | 4.36 | 1032000 | 2.6364 | |
|
| 2.7992 | 4.4 | 1040000 | 2.6327 | |
|
| 2.7992 | 4.43 | 1048000 | 2.6273 | |
|
| 2.7922 | 4.46 | 1056000 | 2.6301 | |
|
| 2.7922 | 4.5 | 1064000 | 2.6350 | |
|
| 2.7939 | 4.53 | 1072000 | 2.6358 | |
|
| 2.7939 | 4.56 | 1080000 | nan | |
|
| 2.789 | 4.6 | 1088000 | 2.6288 | |
|
| 2.789 | 4.63 | 1096000 | 2.6267 | |
|
| 2.7965 | 4.67 | 1104000 | 2.6229 | |
|
| 2.7965 | 4.7 | 1112000 | 2.6331 | |
|
| 2.7963 | 4.73 | 1120000 | 2.6368 | |
|
| 2.7963 | 4.77 | 1128000 | 2.6436 | |
|
| 2.7993 | 4.8 | 1136000 | 2.6363 | |
|
| 2.7993 | 4.83 | 1144000 | 2.6288 | |
|
| 2.7952 | 4.87 | 1152000 | 2.6294 | |
|
| 2.7952 | 4.9 | 1160000 | 2.6337 | |
|
| 2.7972 | 4.94 | 1168000 | 2.6235 | |
|
| 2.7972 | 4.97 | 1176000 | 2.6405 | |
|
| 2.7988 | 5.0 | 1184000 | 2.6266 | |
|
| 2.7988 | 5.04 | 1192000 | 2.6328 | |
|
| 2.7901 | 5.07 | 1200000 | 2.6335 | |
|
| 2.7901 | 5.11 | 1208000 | 2.6405 | |
|
| 2.7975 | 5.14 | 1216000 | 2.6246 | |
|
| 2.7975 | 5.17 | 1224000 | 2.6315 | |
|
| 2.7974 | 5.21 | 1232000 | 2.6390 | |
|
| 2.7974 | 5.24 | 1240000 | 2.6318 | |
|
| 2.7909 | 5.27 | 1248000 | 2.6237 | |
|
| 2.7909 | 5.31 | 1256000 | 2.6343 | |
|
| 2.7899 | 5.34 | 1264000 | 2.6288 | |
|
| 2.7899 | 5.38 | 1272000 | 2.6297 | |
|
| 2.7937 | 5.41 | 1280000 | 2.6343 | |
|
| 2.7937 | 5.44 | 1288000 | 2.6306 | |
|
| 2.7916 | 5.48 | 1296000 | 2.6268 | |
|
| 2.7916 | 5.51 | 1304000 | 2.6317 | |
|
| 2.7874 | 5.54 | 1312000 | 2.6380 | |
|
| 2.7874 | 5.58 | 1320000 | 2.6281 | |
|
| 2.7967 | 5.61 | 1328000 | 2.6334 | |
|
| 2.7967 | 5.65 | 1336000 | 2.6273 | |
|
| 2.791 | 5.68 | 1344000 | 2.6339 | |
|
| 2.791 | 5.71 | 1352000 | 2.6276 | |
|
| 2.791 | 5.75 | 1360000 | 2.6247 | |
|
| 2.791 | 5.78 | 1368000 | 2.6303 | |
|
| 2.7909 | 5.82 | 1376000 | 2.6355 | |
|
| 2.7909 | 5.85 | 1384000 | 2.6352 | |
|
| 2.7833 | 5.88 | 1392000 | 2.6321 | |
|
| 2.7833 | 5.92 | 1400000 | 2.6336 | |
|
| 2.7944 | 5.95 | 1408000 | 2.6312 | |
|
| 2.7944 | 5.98 | 1416000 | 2.6223 | |
|
| 2.8001 | 6.02 | 1424000 | 2.6369 | |
|
| 2.8001 | 6.05 | 1432000 | 2.6299 | |
|
| 2.7954 | 6.09 | 1440000 | 2.6373 | |
|
| 2.7954 | 6.12 | 1448000 | 2.6223 | |
|
| 2.7914 | 6.15 | 1456000 | 2.6225 | |
|
| 2.7914 | 6.19 | 1464000 | 2.6277 | |
|
| 2.7896 | 6.22 | 1472000 | 2.6334 | |
|
| 2.7896 | 6.26 | 1480000 | 2.6260 | |
|
| 2.7925 | 6.29 | 1488000 | 2.6312 | |
|
| 2.7925 | 6.32 | 1496000 | 2.6336 | |
|
| 2.7976 | 6.36 | 1504000 | 2.6270 | |
|
| 2.7976 | 6.39 | 1512000 | 2.6286 | |
|
| 2.8025 | 6.42 | 1520000 | 2.6320 | |
|
| 2.8025 | 6.46 | 1528000 | 2.6252 | |
|
| 2.7953 | 6.49 | 1536000 | 2.6319 | |
|
| 2.7953 | 6.53 | 1544000 | 2.6223 | |
|
| 2.7994 | 6.56 | 1552000 | 2.6358 | |
|
| 2.7994 | 6.59 | 1560000 | 2.6296 | |
|
| 2.7966 | 6.63 | 1568000 | 2.6360 | |
|
| 2.7966 | 6.66 | 1576000 | 2.6327 | |
|
| 2.7883 | 6.69 | 1584000 | 2.6365 | |
|
| 2.7883 | 6.73 | 1592000 | 2.6258 | |
|
| 2.7963 | 6.76 | 1600000 | 2.6401 | |
|
| 2.7963 | 6.8 | 1608000 | 2.6318 | |
|
| 2.7923 | 6.83 | 1616000 | 2.6330 | |
|
| 2.7923 | 6.86 | 1624000 | 2.6372 | |
|
| 2.789 | 6.9 | 1632000 | 2.6363 | |
|
| 2.789 | 6.93 | 1640000 | 2.6346 | |
|
| 2.7883 | 6.97 | 1648000 | 2.6292 | |
|
| 2.7883 | 7.0 | 1656000 | 2.6284 | |
|
| 2.7965 | 7.03 | 1664000 | 2.6408 | |
|
| 2.7965 | 7.07 | 1672000 | 2.6296 | |
|
| 2.7963 | 7.1 | 1680000 | 2.6331 | |
|
| 2.7963 | 7.13 | 1688000 | 2.6339 | |
|
| 2.7911 | 7.17 | 1696000 | 2.6206 | |
|
| 2.7911 | 7.2 | 1704000 | 2.6268 | |
|
| 2.794 | 7.24 | 1712000 | 2.6278 | |
|
| 2.794 | 7.27 | 1720000 | 2.6242 | |
|
| 2.7893 | 7.3 | 1728000 | 2.6329 | |
|
| 2.7893 | 7.34 | 1736000 | 2.6342 | |
|
| 2.7935 | 7.37 | 1744000 | 2.6329 | |
|
| 2.7935 | 7.4 | 1752000 | 2.6294 | |
|
| 2.7936 | 7.44 | 1760000 | 2.6301 | |
|
| 2.7936 | 7.47 | 1768000 | 2.6295 | |
|
| 2.7922 | 7.51 | 1776000 | 2.6261 | |
|
| 2.7922 | 7.54 | 1784000 | 2.6370 | |
|
| 2.7911 | 7.57 | 1792000 | 2.6364 | |
|
| 2.7911 | 7.61 | 1800000 | 2.6232 | |
|
| 2.795 | 7.64 | 1808000 | 2.6201 | |
|
| 2.795 | 7.68 | 1816000 | 2.6329 | |
|
| 2.7898 | 7.71 | 1824000 | 2.6249 | |
|
| 2.7898 | 7.74 | 1832000 | 2.6249 | |
|
| 2.7931 | 7.78 | 1840000 | 2.6361 | |
|
| 2.7931 | 7.81 | 1848000 | nan | |
|
| 2.7919 | 7.84 | 1856000 | 2.6270 | |
|
| 2.7919 | 7.88 | 1864000 | 2.6362 | |
|
| 2.7833 | 7.91 | 1872000 | 2.6278 | |
|
| 2.7833 | 7.95 | 1880000 | 2.6232 | |
|
| 2.8067 | 7.98 | 1888000 | 2.6260 | |
|
| 2.8067 | 8.01 | 1896000 | 2.6262 | |
|
| 2.7953 | 8.05 | 1904000 | 2.6271 | |
|
| 2.7953 | 8.08 | 1912000 | 2.6270 | |
|
| 2.7953 | 8.11 | 1920000 | 2.6305 | |
|
| 2.7953 | 8.15 | 1928000 | 2.6254 | |
|
| 2.7881 | 8.18 | 1936000 | 2.6297 | |
|
| 2.7881 | 8.22 | 1944000 | 2.6271 | |
|
| 2.7928 | 8.25 | 1952000 | 2.6254 | |
|
| 2.7928 | 8.28 | 1960000 | 2.6286 | |
|
| 2.8003 | 8.32 | 1968000 | 2.6330 | |
|
| 2.8003 | 8.35 | 1976000 | 2.6286 | |
|
| 2.7935 | 8.39 | 1984000 | 2.6408 | |
|
| 2.7935 | 8.42 | 1992000 | 2.6275 | |
|
| 2.7925 | 8.45 | 2000000 | 2.6259 | |
|
| 2.7925 | 8.49 | 2008000 | 2.6302 | |
|
| 2.7924 | 8.52 | 2016000 | 2.6320 | |
|
| 2.7924 | 8.55 | 2024000 | 2.6295 | |
|
| 2.799 | 8.59 | 2032000 | 2.6259 | |
|
| 2.799 | 8.62 | 2040000 | 2.6246 | |
|
| 2.7983 | 8.66 | 2048000 | 2.6295 | |
|
| 2.7983 | 8.69 | 2056000 | 2.6194 | |
|
| 2.7901 | 8.72 | 2064000 | 2.6258 | |
|
| 2.7901 | 8.76 | 2072000 | 2.6334 | |
|
| 2.7956 | 8.79 | 2080000 | 2.6361 | |
|
| 2.7956 | 8.82 | 2088000 | 2.6177 | |
|
| 2.8008 | 8.86 | 2096000 | 2.6322 | |
|
| 2.8008 | 8.89 | 2104000 | 2.6281 | |
|
| 2.791 | 8.93 | 2112000 | 2.6249 | |
|
| 2.791 | 8.96 | 2120000 | 2.6284 | |
|
| 2.7933 | 8.99 | 2128000 | 2.6270 | |
|
| 2.7933 | 9.03 | 2136000 | 2.6241 | |
|
| 2.7825 | 9.06 | 2144000 | 2.6254 | |
|
| 2.7825 | 9.1 | 2152000 | 2.6283 | |
|
| 2.7854 | 9.13 | 2160000 | 2.6343 | |
|
| 2.7854 | 9.16 | 2168000 | 2.6208 | |
|
| 2.7949 | 9.2 | 2176000 | 2.6293 | |
|
| 2.7949 | 9.23 | 2184000 | 2.6266 | |
|
| 2.7938 | 9.26 | 2192000 | 2.6270 | |
|
| 2.7938 | 9.3 | 2200000 | 2.6238 | |
|
| 2.7905 | 9.33 | 2208000 | 2.6282 | |
|
| 2.7905 | 9.37 | 2216000 | 2.6246 | |
|
| 2.8004 | 9.4 | 2224000 | 2.6274 | |
|
| 2.8004 | 9.43 | 2232000 | 2.6252 | |
|
| 2.7921 | 9.47 | 2240000 | 2.6343 | |
|
| 2.7921 | 9.5 | 2248000 | 2.6328 | |
|
| 2.7964 | 9.53 | 2256000 | 2.6206 | |
|
| 2.7964 | 9.57 | 2264000 | 2.6235 | |
|
| 2.7954 | 9.6 | 2272000 | 2.6288 | |
|
| 2.7954 | 9.64 | 2280000 | 2.6204 | |
|
| 2.7902 | 9.67 | 2288000 | 2.6232 | |
|
| 2.7902 | 9.7 | 2296000 | 2.6239 | |
|
| 2.8046 | 9.74 | 2304000 | 2.6241 | |
|
| 2.8046 | 9.77 | 2312000 | 2.6259 | |
|
| 2.793 | 9.81 | 2320000 | 2.6275 | |
|
| 2.793 | 9.84 | 2328000 | 2.6264 | |
|
| 2.7893 | 9.87 | 2336000 | 2.6332 | |
|
| 2.7893 | 9.91 | 2344000 | 2.6214 | |
|
| 2.7898 | 9.94 | 2352000 | 2.6318 | |
|
| 2.7898 | 9.97 | 2360000 | 2.6239 | |
|
| 2.7906 | 10.01 | 2368000 | 2.6215 | |
|
| 2.7906 | 10.04 | 2376000 | 2.6336 | |
|
| 2.7942 | 10.08 | 2384000 | 2.6218 | |
|
| 2.7942 | 10.11 | 2392000 | 2.6299 | |
|
| 2.7997 | 10.14 | 2400000 | 2.6303 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.35.0.dev0 |
|
- Pytorch 2.0.1+cu117 |
|
- Datasets 2.14.5 |
|
- Tokenizers 0.14.0 |
|
|
