Hugging Face's logo

DouglasPontes
/
2020-Q2-50p-filtered

Fill-Mask
Transformers
PyTorch
roberta
Generated from Trainer
Inference Endpoints
Model card Files Community
2020-Q2-50p-filtered / README.md
DouglasPontes's picture
DouglasPontes
End of training
06e4e76 verified
preview code
|
raw
history blame contribute delete
17.6 kB
metadata
license: mit
base_model: cardiffnlp/twitter-roberta-base-2019-90m
tags:
  - generated_from_trainer
model-index:
  - name: 2020-Q2-50p-filtered
    results: []

2020-Q2-50p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 2.5608

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1400
  • training_steps: 2400000

Training results

Training Loss Epoch Step Validation Loss
No log 0.03 8000 2.6645
2.8656 0.07 16000 2.6465
2.8656 0.1 24000 2.6186
2.7946 0.13 32000 2.6235
2.7946 0.17 40000 2.6151
2.7911 0.2 48000 2.6128
2.7911 0.24 56000 2.6010
2.7898 0.27 64000 2.6144
2.7898 0.3 72000 2.5976
2.7791 0.34 80000 2.6006
2.7791 0.37 88000 2.5889
2.7776 0.4 96000 2.5888
2.7776 0.44 104000 2.5842
2.7702 0.47 112000 2.5760
2.7702 0.51 120000 2.5720
2.7661 0.54 128000 2.5710
2.7661 0.57 136000 2.5673
2.7609 0.61 144000 2.5693
2.7609 0.64 152000 2.5623
2.7557 0.67 160000 2.5559
2.7557 0.71 168000 2.5650
2.7584 0.74 176000 2.5584
2.7584 0.77 184000 2.5591
2.7619 0.81 192000 2.5597
2.7619 0.84 200000 2.5650
2.7678 0.88 208000 2.5728
2.7678 0.91 216000 2.5712
2.7735 0.94 224000 2.5729
2.7735 0.98 232000 2.5755
2.777 1.01 240000 2.5715
2.777 1.04 248000 2.5747
2.7692 1.08 256000 2.5782
2.7692 1.11 264000 2.5841
2.7826 1.15 272000 2.5731
2.7826 1.18 280000 2.5836
2.7845 1.21 288000 2.5841
2.7845 1.25 296000 2.5811
2.7909 1.28 304000 2.5928
2.7909 1.31 312000 2.5977
2.7993 1.35 320000 2.6025
2.7993 1.38 328000 2.6072
2.8107 1.41 336000 2.6110
2.8107 1.45 344000 2.6020
2.8102 1.48 352000 2.6065
2.8102 1.52 360000 2.6207
2.8247 1.55 368000 2.6192
2.8247 1.58 376000 2.6224
2.8271 1.62 384000 2.6205
2.8271 1.65 392000 2.6292
2.8415 1.68 400000 2.6348
2.8415 1.72 408000 2.6518
2.842 1.75 416000 2.6465
2.842 1.79 424000 2.6434
2.8431 1.82 432000 2.6414
2.8431 1.85 440000 2.6532
2.8599 1.89 448000 2.6645
2.8599 1.92 456000 2.6651
2.8567 1.95 464000 2.6694
2.8567 1.99 472000 2.6610
2.8682 2.02 480000 2.6877
2.8682 2.05 488000 2.6724
2.8693 2.09 496000 2.6839
2.8693 2.12 504000 2.6923
2.8881 2.16 512000 2.6964
2.8881 2.19 520000 2.6982
2.8874 2.22 528000 2.6961
2.8874 2.26 536000 2.6884
2.8899 2.29 544000 2.7055
2.8899 2.32 552000 2.6988
2.8966 2.36 560000 2.7103
2.8966 2.39 568000 2.7100
2.9 2.43 576000 2.7169
2.9 2.46 584000 2.7180
2.9237 2.49 592000 2.7270
2.9237 2.53 600000 2.7265
2.9236 2.56 608000 2.7323
2.9236 2.59 616000 2.7350
2.9276 2.63 624000 2.7333
2.9276 2.66 632000 2.7345
2.9252 2.69 640000 2.7497
2.9252 2.73 648000 2.7428
2.9364 2.76 656000 2.7392
2.9364 2.8 664000 2.7505
2.9366 2.83 672000 2.7393
2.9366 2.86 680000 2.7372
2.9437 2.9 688000 2.7451
2.9437 2.93 696000 2.7488
2.9483 2.96 704000 2.7586
2.9483 3.0 712000 2.7613
2.9588 3.03 720000 2.7619
2.9588 3.07 728000 2.7680
2.9422 3.1 736000 2.7546
2.9422 3.13 744000 2.7629
2.965 3.17 752000 2.7595
2.965 3.2 760000 2.7763
2.959 3.23 768000 2.7739
2.959 3.27 776000 2.7839
2.9604 3.3 784000 2.7681
2.9604 3.33 792000 2.7816
2.9638 3.37 800000 2.7812
2.9638 3.4 808000 2.7846
2.9704 3.44 816000 2.7766
2.9704 3.47 824000 2.7869
2.9684 3.5 832000 2.7741
2.9684 3.54 840000 2.7735
2.9723 3.57 848000 2.7701
2.9723 3.6 856000 2.7780
2.9734 3.64 864000 2.7833
2.9734 3.67 872000 2.7910
2.9806 3.71 880000 2.7941
2.9806 3.74 888000 2.7997
2.9808 3.77 896000 2.8027
2.9808 3.81 904000 2.7972
3.0008 3.84 912000 2.8026
3.0008 3.87 920000 2.7975
2.9934 3.91 928000 2.7971
2.9934 3.94 936000 2.8030
2.9927 3.97 944000 2.8082
2.9927 4.01 952000 2.8208
3.0013 4.04 960000 2.8129
3.0013 4.08 968000 2.8236
2.9996 4.11 976000 2.8226
2.9996 4.14 984000 2.8273
3.0125 4.18 992000 2.8161
3.0125 4.21 1000000 2.8249
3.0086 4.24 1008000 2.8320
3.0086 4.28 1016000 2.8313
3.0077 4.31 1024000 2.8321
3.0077 4.35 1032000 2.8332
3.0186 4.38 1040000 2.8288
3.0186 4.41 1048000 2.8392
3.0311 4.45 1056000 2.8243
3.0311 4.48 1064000 2.8524
3.0199 4.51 1072000 2.8347
3.0199 4.55 1080000 2.8438
3.0198 4.58 1088000 2.8415
3.0198 4.61 1096000 2.8460
3.0279 4.65 1104000 2.8551
3.0279 4.68 1112000 2.8528
3.0319 4.72 1120000 2.8601
3.0319 4.75 1128000 2.8544
3.0371 4.78 1136000 2.8553
3.0371 4.82 1144000 2.8597
3.038 4.85 1152000 2.8653
3.038 4.88 1160000 2.8560
3.0318 4.92 1168000 2.8602
3.0318 4.95 1176000 2.8484
3.0449 4.99 1184000 2.8612
3.0449 5.02 1192000 2.8598
3.0384 5.05 1200000 2.8581
3.0384 5.09 1208000 2.8481
3.0243 5.12 1216000 2.8458
3.0243 5.15 1224000 2.8494
3.0345 5.19 1232000 2.8544
3.0345 5.22 1240000 2.8488
3.0251 5.25 1248000 2.8453
3.0251 5.29 1256000 2.8464
3.0234 5.32 1264000 2.8486
3.0234 5.36 1272000 2.8436
3.0205 5.39 1280000 2.8476
3.0205 5.42 1288000 2.8327
3.0228 5.46 1296000 2.8452
3.0228 5.49 1304000 2.8372
3.0063 5.52 1312000 2.8306
3.0063 5.56 1320000 2.8411
3.0068 5.59 1328000 2.8273
3.0068 5.63 1336000 2.8343
3.0109 5.66 1344000 2.8328
3.0109 5.69 1352000 2.8431
3.0068 5.73 1360000 2.8332
3.0068 5.76 1368000 2.8275
3.002 5.79 1376000 2.8314
3.002 5.83 1384000 2.8324
3.0037 5.86 1392000 2.8394
3.0037 5.89 1400000 2.8338
3.0086 5.93 1408000 2.8448
3.0086 5.96 1416000 2.8326
2.9977 6.0 1424000 2.8311
2.9977 6.03 1432000 2.8410
2.9984 6.06 1440000 2.8359
2.9984 6.1 1448000 2.8393
3.0095 6.13 1456000 2.8388
3.0095 6.16 1464000 2.8448
3.0051 6.2 1472000 2.8472
3.0051 6.23 1480000 2.8421
3.0142 6.27 1488000 2.8424
3.0142 6.3 1496000 2.8477
3.0149 6.33 1504000 2.8428
3.0149 6.37 1512000 2.8529
3.0147 6.4 1520000 2.8541
3.0147 6.43 1528000 2.8519
3.0205 6.47 1536000 2.8527
3.0205 6.5 1544000 2.8471
3.029 6.53 1552000 2.8583
3.029 6.57 1560000 2.8497
3.024 6.6 1568000 2.8653
3.024 6.64 1576000 2.8553
3.0371 6.67 1584000 2.8653
3.0371 6.7 1592000 2.8604
3.0319 6.74 1600000 2.8624
3.0319 6.77 1608000 2.8657
3.0369 6.8 1616000 2.8616
3.0369 6.84 1624000 2.8667
3.0357 6.87 1632000 2.8660
3.0357 6.91 1640000 2.8682
3.0342 6.94 1648000 2.8676
3.0342 6.97 1656000 2.8815
3.0375 7.01 1664000 2.8667
3.0375 7.04 1672000 2.8735
3.0419 7.07 1680000 2.8788
3.0419 7.11 1688000 2.8767
3.0403 7.14 1696000 2.8812
3.0403 7.17 1704000 2.8795
3.0482 7.21 1712000 2.8805
3.0482 7.24 1720000 2.8794
3.0533 7.28 1728000 2.8788
3.0533 7.31 1736000 2.8844
3.0453 7.34 1744000 2.8709
3.0453 7.38 1752000 2.8835
3.0562 7.41 1760000 2.8891
3.0562 7.44 1768000 2.8903
3.0617 7.48 1776000 2.8849
3.0617 7.51 1784000 2.8766
3.0539 7.55 1792000 2.8872
3.0539 7.58 1800000 2.8981
3.0561 7.61 1808000 2.8862
3.0561 7.65 1816000 2.8940
3.0529 7.68 1824000 2.8874
3.0529 7.71 1832000 2.8839
3.0484 7.75 1840000 2.8838
3.0484 7.78 1848000 2.8856
3.0562 7.81 1856000 2.8984
3.0562 7.85 1864000 2.8844
3.0578 7.88 1872000 2.8874
3.0578 7.92 1880000 2.8887
3.0553 7.95 1888000 2.8798
3.0553 7.98 1896000 2.8789
3.0623 8.02 1904000 2.8968
3.0623 8.05 1912000 2.8834
3.0652 8.08 1920000 2.8902
3.0652 8.12 1928000 2.8822
3.0487 8.15 1936000 2.8844
3.0487 8.19 1944000 2.8909
3.0546 8.22 1952000 2.8915
3.0546 8.25 1960000 2.8870
3.0524 8.29 1968000 2.8828
3.0524 8.32 1976000 2.8781
3.0491 8.35 1984000 2.8948
3.0491 8.39 1992000 2.8904
3.0534 8.42 2000000 2.8839
3.0534 8.45 2008000 2.8918
3.0547 8.49 2016000 2.8739
3.0547 8.52 2024000 2.8684
3.0544 8.56 2032000 2.8740
3.0544 8.59 2040000 2.8784
3.0448 8.62 2048000 2.8758
3.0448 8.66 2056000 2.8801
3.0499 8.69 2064000 2.8793
3.0499 8.72 2072000 2.8707
3.0368 8.76 2080000 2.8722
3.0368 8.79 2088000 2.8752
3.0548 8.83 2096000 2.8880
3.0548 8.86 2104000 2.8781
3.0457 8.89 2112000 2.8825
3.0457 8.93 2120000 2.8827
3.0377 8.96 2128000 2.8810
3.0377 8.99 2136000 2.8727
3.0341 9.03 2144000 2.8750
3.0341 9.06 2152000 2.8638
3.0275 9.09 2160000 2.8690
3.0275 9.13 2168000 2.8660
3.0413 9.16 2176000 2.8578
3.0413 9.2 2184000 2.8692
3.0272 9.23 2192000 2.8702
3.0272 9.26 2200000 2.8707
3.034 9.3 2208000 2.8666
3.034 9.33 2216000 2.8734
3.0346 9.36 2224000 2.8685
3.0346 9.4 2232000 2.8675
3.0234 9.43 2240000 2.8662
3.0234 9.47 2248000 2.8670
3.0256 9.5 2256000 2.8764
3.0256 9.53 2264000 2.8664
3.0232 9.57 2272000 2.8625
3.0232 9.6 2280000 2.8647
3.0309 9.63 2288000 2.8561
3.0309 9.67 2296000 2.8657
3.0254 9.7 2304000 2.8667
3.0254 9.73 2312000 2.8618
3.0198 9.77 2320000 2.8650
3.0198 9.8 2328000 2.8630
3.0109 9.84 2336000 2.8533
3.0109 9.87 2344000 2.8656
3.0316 9.9 2352000 2.8607
3.0316 9.94 2360000 2.8572
3.0225 9.97 2368000 2.8617
3.0225 10.0 2376000 2.8604
3.0132 10.04 2384000 2.8577
3.0132 10.07 2392000 2.8535
3.0202 10.11 2400000 2.8566

Framework versions

  • Transformers 4.35.0.dev0
  • Pytorch 2.0.1+cu117
  • Datasets 2.14.5
  • Tokenizers 0.14.0