2020-Q2-50p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.5608

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1400
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.03	8000	2.6645
2.8656	0.07	16000	2.6465
2.8656	0.1	24000	2.6186
2.7946	0.13	32000	2.6235
2.7946	0.17	40000	2.6151
2.7911	0.2	48000	2.6128
2.7911	0.24	56000	2.6010
2.7898	0.27	64000	2.6144
2.7898	0.3	72000	2.5976
2.7791	0.34	80000	2.6006
2.7791	0.37	88000	2.5889
2.7776	0.4	96000	2.5888
2.7776	0.44	104000	2.5842
2.7702	0.47	112000	2.5760
2.7702	0.51	120000	2.5720
2.7661	0.54	128000	2.5710
2.7661	0.57	136000	2.5673
2.7609	0.61	144000	2.5693
2.7609	0.64	152000	2.5623
2.7557	0.67	160000	2.5559
2.7557	0.71	168000	2.5650
2.7584	0.74	176000	2.5584
2.7584	0.77	184000	2.5591
2.7619	0.81	192000	2.5597
2.7619	0.84	200000	2.5650
2.7678	0.88	208000	2.5728
2.7678	0.91	216000	2.5712
2.7735	0.94	224000	2.5729
2.7735	0.98	232000	2.5755
2.777	1.01	240000	2.5715
2.777	1.04	248000	2.5747
2.7692	1.08	256000	2.5782
2.7692	1.11	264000	2.5841
2.7826	1.15	272000	2.5731
2.7826	1.18	280000	2.5836
2.7845	1.21	288000	2.5841
2.7845	1.25	296000	2.5811
2.7909	1.28	304000	2.5928
2.7909	1.31	312000	2.5977
2.7993	1.35	320000	2.6025
2.7993	1.38	328000	2.6072
2.8107	1.41	336000	2.6110
2.8107	1.45	344000	2.6020
2.8102	1.48	352000	2.6065
2.8102	1.52	360000	2.6207
2.8247	1.55	368000	2.6192
2.8247	1.58	376000	2.6224
2.8271	1.62	384000	2.6205
2.8271	1.65	392000	2.6292
2.8415	1.68	400000	2.6348
2.8415	1.72	408000	2.6518
2.842	1.75	416000	2.6465
2.842	1.79	424000	2.6434
2.8431	1.82	432000	2.6414
2.8431	1.85	440000	2.6532
2.8599	1.89	448000	2.6645
2.8599	1.92	456000	2.6651
2.8567	1.95	464000	2.6694
2.8567	1.99	472000	2.6610
2.8682	2.02	480000	2.6877
2.8682	2.05	488000	2.6724
2.8693	2.09	496000	2.6839
2.8693	2.12	504000	2.6923
2.8881	2.16	512000	2.6964
2.8881	2.19	520000	2.6982
2.8874	2.22	528000	2.6961
2.8874	2.26	536000	2.6884
2.8899	2.29	544000	2.7055
2.8899	2.32	552000	2.6988
2.8966	2.36	560000	2.7103
2.8966	2.39	568000	2.7100
2.9	2.43	576000	2.7169
2.9	2.46	584000	2.7180
2.9237	2.49	592000	2.7270
2.9237	2.53	600000	2.7265
2.9236	2.56	608000	2.7323
2.9236	2.59	616000	2.7350
2.9276	2.63	624000	2.7333
2.9276	2.66	632000	2.7345
2.9252	2.69	640000	2.7497
2.9252	2.73	648000	2.7428
2.9364	2.76	656000	2.7392
2.9364	2.8	664000	2.7505
2.9366	2.83	672000	2.7393
2.9366	2.86	680000	2.7372
2.9437	2.9	688000	2.7451
2.9437	2.93	696000	2.7488
2.9483	2.96	704000	2.7586
2.9483	3.0	712000	2.7613
2.9588	3.03	720000	2.7619
2.9588	3.07	728000	2.7680
2.9422	3.1	736000	2.7546
2.9422	3.13	744000	2.7629
2.965	3.17	752000	2.7595
2.965	3.2	760000	2.7763
2.959	3.23	768000	2.7739
2.959	3.27	776000	2.7839
2.9604	3.3	784000	2.7681
2.9604	3.33	792000	2.7816
2.9638	3.37	800000	2.7812
2.9638	3.4	808000	2.7846
2.9704	3.44	816000	2.7766
2.9704	3.47	824000	2.7869
2.9684	3.5	832000	2.7741
2.9684	3.54	840000	2.7735
2.9723	3.57	848000	2.7701
2.9723	3.6	856000	2.7780
2.9734	3.64	864000	2.7833
2.9734	3.67	872000	2.7910
2.9806	3.71	880000	2.7941
2.9806	3.74	888000	2.7997
2.9808	3.77	896000	2.8027
2.9808	3.81	904000	2.7972
3.0008	3.84	912000	2.8026
3.0008	3.87	920000	2.7975
2.9934	3.91	928000	2.7971
2.9934	3.94	936000	2.8030
2.9927	3.97	944000	2.8082
2.9927	4.01	952000	2.8208
3.0013	4.04	960000	2.8129
3.0013	4.08	968000	2.8236
2.9996	4.11	976000	2.8226
2.9996	4.14	984000	2.8273
3.0125	4.18	992000	2.8161
3.0125	4.21	1000000	2.8249
3.0086	4.24	1008000	2.8320
3.0086	4.28	1016000	2.8313
3.0077	4.31	1024000	2.8321
3.0077	4.35	1032000	2.8332
3.0186	4.38	1040000	2.8288
3.0186	4.41	1048000	2.8392
3.0311	4.45	1056000	2.8243
3.0311	4.48	1064000	2.8524
3.0199	4.51	1072000	2.8347
3.0199	4.55	1080000	2.8438
3.0198	4.58	1088000	2.8415
3.0198	4.61	1096000	2.8460
3.0279	4.65	1104000	2.8551
3.0279	4.68	1112000	2.8528
3.0319	4.72	1120000	2.8601
3.0319	4.75	1128000	2.8544
3.0371	4.78	1136000	2.8553
3.0371	4.82	1144000	2.8597
3.038	4.85	1152000	2.8653
3.038	4.88	1160000	2.8560
3.0318	4.92	1168000	2.8602
3.0318	4.95	1176000	2.8484
3.0449	4.99	1184000	2.8612
3.0449	5.02	1192000	2.8598
3.0384	5.05	1200000	2.8581
3.0384	5.09	1208000	2.8481
3.0243	5.12	1216000	2.8458
3.0243	5.15	1224000	2.8494
3.0345	5.19	1232000	2.8544
3.0345	5.22	1240000	2.8488
3.0251	5.25	1248000	2.8453
3.0251	5.29	1256000	2.8464
3.0234	5.32	1264000	2.8486
3.0234	5.36	1272000	2.8436
3.0205	5.39	1280000	2.8476
3.0205	5.42	1288000	2.8327
3.0228	5.46	1296000	2.8452
3.0228	5.49	1304000	2.8372
3.0063	5.52	1312000	2.8306
3.0063	5.56	1320000	2.8411
3.0068	5.59	1328000	2.8273
3.0068	5.63	1336000	2.8343
3.0109	5.66	1344000	2.8328
3.0109	5.69	1352000	2.8431
3.0068	5.73	1360000	2.8332
3.0068	5.76	1368000	2.8275
3.002	5.79	1376000	2.8314
3.002	5.83	1384000	2.8324
3.0037	5.86	1392000	2.8394
3.0037	5.89	1400000	2.8338
3.0086	5.93	1408000	2.8448
3.0086	5.96	1416000	2.8326
2.9977	6.0	1424000	2.8311
2.9977	6.03	1432000	2.8410
2.9984	6.06	1440000	2.8359
2.9984	6.1	1448000	2.8393
3.0095	6.13	1456000	2.8388
3.0095	6.16	1464000	2.8448
3.0051	6.2	1472000	2.8472
3.0051	6.23	1480000	2.8421
3.0142	6.27	1488000	2.8424
3.0142	6.3	1496000	2.8477
3.0149	6.33	1504000	2.8428
3.0149	6.37	1512000	2.8529
3.0147	6.4	1520000	2.8541
3.0147	6.43	1528000	2.8519
3.0205	6.47	1536000	2.8527
3.0205	6.5	1544000	2.8471
3.029	6.53	1552000	2.8583
3.029	6.57	1560000	2.8497
3.024	6.6	1568000	2.8653
3.024	6.64	1576000	2.8553
3.0371	6.67	1584000	2.8653
3.0371	6.7	1592000	2.8604
3.0319	6.74	1600000	2.8624
3.0319	6.77	1608000	2.8657
3.0369	6.8	1616000	2.8616
3.0369	6.84	1624000	2.8667
3.0357	6.87	1632000	2.8660
3.0357	6.91	1640000	2.8682
3.0342	6.94	1648000	2.8676
3.0342	6.97	1656000	2.8815
3.0375	7.01	1664000	2.8667
3.0375	7.04	1672000	2.8735
3.0419	7.07	1680000	2.8788
3.0419	7.11	1688000	2.8767
3.0403	7.14	1696000	2.8812
3.0403	7.17	1704000	2.8795
3.0482	7.21	1712000	2.8805
3.0482	7.24	1720000	2.8794
3.0533	7.28	1728000	2.8788
3.0533	7.31	1736000	2.8844
3.0453	7.34	1744000	2.8709
3.0453	7.38	1752000	2.8835
3.0562	7.41	1760000	2.8891
3.0562	7.44	1768000	2.8903
3.0617	7.48	1776000	2.8849
3.0617	7.51	1784000	2.8766
3.0539	7.55	1792000	2.8872
3.0539	7.58	1800000	2.8981
3.0561	7.61	1808000	2.8862
3.0561	7.65	1816000	2.8940
3.0529	7.68	1824000	2.8874
3.0529	7.71	1832000	2.8839
3.0484	7.75	1840000	2.8838
3.0484	7.78	1848000	2.8856
3.0562	7.81	1856000	2.8984
3.0562	7.85	1864000	2.8844
3.0578	7.88	1872000	2.8874
3.0578	7.92	1880000	2.8887
3.0553	7.95	1888000	2.8798
3.0553	7.98	1896000	2.8789
3.0623	8.02	1904000	2.8968
3.0623	8.05	1912000	2.8834
3.0652	8.08	1920000	2.8902
3.0652	8.12	1928000	2.8822
3.0487	8.15	1936000	2.8844
3.0487	8.19	1944000	2.8909
3.0546	8.22	1952000	2.8915
3.0546	8.25	1960000	2.8870
3.0524	8.29	1968000	2.8828
3.0524	8.32	1976000	2.8781
3.0491	8.35	1984000	2.8948
3.0491	8.39	1992000	2.8904
3.0534	8.42	2000000	2.8839
3.0534	8.45	2008000	2.8918
3.0547	8.49	2016000	2.8739
3.0547	8.52	2024000	2.8684
3.0544	8.56	2032000	2.8740
3.0544	8.59	2040000	2.8784
3.0448	8.62	2048000	2.8758
3.0448	8.66	2056000	2.8801
3.0499	8.69	2064000	2.8793
3.0499	8.72	2072000	2.8707
3.0368	8.76	2080000	2.8722
3.0368	8.79	2088000	2.8752
3.0548	8.83	2096000	2.8880
3.0548	8.86	2104000	2.8781
3.0457	8.89	2112000	2.8825
3.0457	8.93	2120000	2.8827
3.0377	8.96	2128000	2.8810
3.0377	8.99	2136000	2.8727
3.0341	9.03	2144000	2.8750
3.0341	9.06	2152000	2.8638
3.0275	9.09	2160000	2.8690
3.0275	9.13	2168000	2.8660
3.0413	9.16	2176000	2.8578
3.0413	9.2	2184000	2.8692
3.0272	9.23	2192000	2.8702
3.0272	9.26	2200000	2.8707
3.034	9.3	2208000	2.8666
3.034	9.33	2216000	2.8734
3.0346	9.36	2224000	2.8685
3.0346	9.4	2232000	2.8675
3.0234	9.43	2240000	2.8662
3.0234	9.47	2248000	2.8670
3.0256	9.5	2256000	2.8764
3.0256	9.53	2264000	2.8664
3.0232	9.57	2272000	2.8625
3.0232	9.6	2280000	2.8647
3.0309	9.63	2288000	2.8561
3.0309	9.67	2296000	2.8657
3.0254	9.7	2304000	2.8667
3.0254	9.73	2312000	2.8618
3.0198	9.77	2320000	2.8650
3.0198	9.8	2328000	2.8630
3.0109	9.84	2336000	2.8533
3.0109	9.87	2344000	2.8656
3.0316	9.9	2352000	2.8607
3.0316	9.94	2360000	2.8572
3.0225	9.97	2368000	2.8617
3.0225	10.0	2376000	2.8604
3.0132	10.04	2384000	2.8577
3.0132	10.07	2392000	2.8535
3.0202	10.11	2400000	2.8566

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0