2020-Q1-90p-filtered_2

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9012

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.16	8000	2.2376
2.4394	0.33	16000	2.1557
2.4394	0.49	24000	2.0965
2.2403	0.66	32000	2.0637
2.2403	0.82	40000	2.0620
2.1859	0.98	48000	2.0427
2.1859	1.15	56000	2.0440
2.1472	1.31	64000	2.0177
2.1472	1.48	72000	1.9980
2.1334	1.64	80000	2.0021
2.1334	1.8	88000	1.9963
2.1271	1.97	96000	1.9918
2.1271	2.13	104000	1.9889
2.1065	2.29	112000	1.9689
2.1065	2.46	120000	1.9919
2.105	2.62	128000	1.9706
2.105	2.79	136000	1.9725
2.1033	2.95	144000	2.0009
2.1033	3.11	152000	1.9661
2.0934	3.28	160000	1.9641
2.0934	3.44	168000	1.9733
2.0899	3.61	176000	1.9747
2.0899	3.77	184000	1.9442
2.0903	3.93	192000	1.9586
2.0903	4.1	200000	1.9586
2.0842	4.26	208000	1.9402
2.0842	4.43	216000	1.9483
2.0761	4.59	224000	1.9532
2.0761	4.75	232000	1.9456
2.0799	4.92	240000	1.9322
2.0799	5.08	248000	1.9460
2.0704	5.24	256000	1.9478
2.0704	5.41	264000	1.9435
2.0727	5.57	272000	1.9356
2.0727	5.74	280000	1.9543
2.073	5.9	288000	1.9542
2.073	6.06	296000	1.9503
2.0647	6.23	304000	1.9437
2.0647	6.39	312000	1.9450
2.0668	6.56	320000	1.9221
2.0668	6.72	328000	1.9277
2.0695	6.88	336000	1.9357
2.0695	7.05	344000	1.9244
2.0652	7.21	352000	1.9387
2.0652	7.38	360000	1.9355
2.0607	7.54	368000	1.9390
2.0607	7.7	376000	1.9359
2.0707	7.87	384000	1.9393
2.0707	8.03	392000	1.9217
2.0621	8.2	400000	1.9284
2.0621	8.36	408000	1.9383
2.0643	8.52	416000	1.9224
2.0643	8.69	424000	1.9284
2.0498	8.85	432000	1.9399
2.0498	9.01	440000	1.9427
2.0689	9.18	448000	1.9351
2.0689	9.34	456000	1.9313
2.0511	9.51	464000	1.9392
2.0511	9.67	472000	1.9278
2.0664	9.83	480000	1.9385
2.0664	10.0	488000	1.9472
2.0565	10.16	496000	1.9377
2.0565	10.33	504000	1.9481
2.0566	10.49	512000	1.9454
2.0566	10.65	520000	1.9244
2.0523	10.82	528000	1.9358
2.0523	10.98	536000	1.9176
2.0554	11.15	544000	1.9284
2.0554	11.31	552000	1.9287
2.0485	11.47	560000	1.9237
2.0485	11.64	568000	1.9209
2.0485	11.8	576000	1.9262
2.0485	11.97	584000	1.9208
2.0542	12.13	592000	1.9320
2.0542	12.29	600000	1.9077
2.0527	12.46	608000	1.9249
2.0527	12.62	616000	1.9192
2.0606	12.78	624000	1.9152
2.0606	12.95	632000	1.9194
2.0542	13.11	640000	1.9198
2.0542	13.28	648000	1.9135
2.0593	13.44	656000	1.9192
2.0593	13.6	664000	1.9257
2.0467	13.77	672000	1.9135
2.0467	13.93	680000	1.8995
2.0535	14.1	688000	1.9306
2.0535	14.26	696000	1.9278
2.0559	14.42	704000	1.9137
2.0559	14.59	712000	1.9165
2.0544	14.75	720000	1.9219
2.0544	14.92	728000	1.9200
2.0493	15.08	736000	1.9242
2.0493	15.24	744000	1.9264
2.0538	15.41	752000	1.9333
2.0538	15.57	760000	1.9126
2.0457	15.73	768000	1.9023
2.0457	15.9	776000	1.9154
2.0436	16.06	784000	1.8947
2.0436	16.23	792000	1.9198
2.0527	16.39	800000	1.9125
2.0527	16.55	808000	1.9129
2.0484	16.72	816000	1.9302
2.0484	16.88	824000	1.9152
2.0535	17.05	832000	1.9223
2.0535	17.21	840000	1.9195
2.0516	17.37	848000	1.9072
2.0516	17.54	856000	1.9210
2.0546	17.7	864000	1.9014
2.0546	17.87	872000	1.9130
2.0439	18.03	880000	1.9064
2.0439	18.19	888000	1.9166
2.0391	18.36	896000	1.9310
2.0391	18.52	904000	1.9064
2.0568	18.69	912000	1.9107
2.0568	18.85	920000	1.9317
2.047	19.01	928000	1.9235
2.047	19.18	936000	1.9173
2.0431	19.34	944000	1.8946
2.0431	19.5	952000	1.9247
2.0444	19.67	960000	1.9106
2.0444	19.83	968000	1.9023
2.0465	20.0	976000	1.9196
2.0465	20.16	984000	1.9112
2.0454	20.32	992000	1.9046
2.0454	20.49	1000000	1.9032
2.04	20.65	1008000	1.9130
2.04	20.82	1016000	1.9223
2.0406	20.98	1024000	1.9261
2.0406	21.14	1032000	1.9014
2.0401	21.31	1040000	1.9052
2.0401	21.47	1048000	1.9005
2.044	21.64	1056000	1.9044
2.044	21.8	1064000	1.9170
2.0401	21.96	1072000	1.9103
2.0401	22.13	1080000	1.8971
2.0458	22.29	1088000	1.9257
2.0458	22.45	1096000	1.9029
2.0414	22.62	1104000	1.9150
2.0414	22.78	1112000	1.9124
2.0419	22.95	1120000	1.9030
2.0419	23.11	1128000	1.9145
2.0415	23.27	1136000	1.9132
2.0415	23.44	1144000	1.9054
2.0394	23.6	1152000	1.9155
2.0394	23.77	1160000	1.9147
2.0414	23.93	1168000	1.9130
2.0414	24.09	1176000	1.9002
2.036	24.26	1184000	1.8991
2.036	24.42	1192000	1.9203
2.0393	24.59	1200000	1.9327
2.0393	24.75	1208000	1.9099
2.0375	24.91	1216000	1.9097
2.0375	25.08	1224000	1.9006
2.0384	25.24	1232000	1.9063
2.0384	25.41	1240000	1.9057
2.0392	25.57	1248000	1.9037
2.0392	25.73	1256000	1.9013
2.0503	25.9	1264000	1.9037
2.0503	26.06	1272000	1.9042
2.0418	26.22	1280000	1.8966
2.0418	26.39	1288000	1.9187
2.0416	26.55	1296000	1.9098
2.0416	26.72	1304000	1.9153
2.0396	26.88	1312000	1.9164
2.0396	27.04	1320000	1.8867
2.0397	27.21	1328000	1.8969
2.0397	27.37	1336000	1.9155
2.0442	27.54	1344000	1.9004
2.0442	27.7	1352000	1.9027
2.0332	27.86	1360000	1.9095
2.0332	28.03	1368000	1.9134
2.0398	28.19	1376000	1.9083
2.0398	28.36	1384000	1.9041
2.0387	28.52	1392000	1.8980
2.0387	28.68	1400000	1.9209
2.0378	28.85	1408000	1.8962
2.0378	29.01	1416000	1.8981
2.0359	29.17	1424000	1.9078
2.0359	29.34	1432000	1.8962
2.0357	29.5	1440000	1.8843
2.0357	29.67	1448000	1.9157
2.0367	29.83	1456000	1.9278
2.0367	29.99	1464000	1.9009
2.0442	30.16	1472000	1.8969
2.0442	30.32	1480000	1.9086
2.0401	30.49	1488000	1.9059
2.0401	30.65	1496000	1.8997
2.0293	30.81	1504000	1.9014
2.0293	30.98	1512000	1.8971
2.035	31.14	1520000	1.9114
2.035	31.31	1528000	1.9108
2.0389	31.47	1536000	1.8971
2.0389	31.63	1544000	1.9082
2.0346	31.8	1552000	1.9209
2.0346	31.96	1560000	1.9018
2.0428	32.13	1568000	1.8988
2.0428	32.29	1576000	1.9089
2.0286	32.45	1584000	1.8983
2.0286	32.62	1592000	1.9030
2.037	32.78	1600000	1.9042
2.037	32.94	1608000	1.9022
2.0348	33.11	1616000	1.8988
2.0348	33.27	1624000	1.9159
2.042	33.44	1632000	1.8934
2.042	33.6	1640000	1.8908
2.0426	33.76	1648000	1.8878
2.0426	33.93	1656000	1.8882
2.0293	34.09	1664000	1.9031
2.0293	34.26	1672000	1.9006
2.0401	34.42	1680000	1.9066
2.0401	34.58	1688000	1.8970
2.0315	34.75	1696000	1.8953
2.0315	34.91	1704000	1.9030
2.0393	35.08	1712000	1.9089
2.0393	35.24	1720000	1.9037
2.0422	35.4	1728000	1.8991
2.0422	35.57	1736000	1.8826
2.0307	35.73	1744000	1.9027
2.0307	35.9	1752000	1.9173
2.0414	36.06	1760000	1.9065
2.0414	36.22	1768000	1.8985
2.0393	36.39	1776000	1.8849
2.0393	36.55	1784000	1.8945
2.0302	36.71	1792000	1.8962
2.0302	36.88	1800000	1.9015
2.0344	37.04	1808000	1.8924
2.0344	37.21	1816000	1.8804
2.0303	37.37	1824000	1.8947
2.0303	37.53	1832000	1.8917
2.0318	37.7	1840000	1.8992
2.0318	37.86	1848000	1.9018
2.039	38.03	1856000	1.8903
2.039	38.19	1864000	1.9089
2.0319	38.35	1872000	1.9093
2.0319	38.52	1880000	1.8951
2.0359	38.68	1888000	1.8970
2.0359	38.85	1896000	1.8995
2.0353	39.01	1904000	1.8888
2.0353	39.17	1912000	1.9149
2.0343	39.34	1920000	1.8881
2.0343	39.5	1928000	1.8935
2.0395	39.66	1936000	1.8938
2.0395	39.83	1944000	1.8929
2.0316	39.99	1952000	1.9186
2.0316	40.16	1960000	1.9189
2.0302	40.32	1968000	1.9125
2.0302	40.48	1976000	1.9078
2.0355	40.65	1984000	1.8975
2.0355	40.81	1992000	1.8929
2.0332	40.98	2000000	1.8899
2.0332	41.14	2008000	1.9043
2.0327	41.3	2016000	1.9086
2.0327	41.47	2024000	1.8944
2.0414	41.63	2032000	1.9029
2.0414	41.8	2040000	1.8990
2.0327	41.96	2048000	1.9175
2.0327	42.12	2056000	1.8888
2.0428	42.29	2064000	1.8971
2.0428	42.45	2072000	1.9071
2.0337	42.62	2080000	1.8968
2.0337	42.78	2088000	1.9060
2.0394	42.94	2096000	1.9028
2.0394	43.11	2104000	1.8917
2.0314	43.27	2112000	1.8868
2.0314	43.43	2120000	1.9087
2.0368	43.6	2128000	1.8951
2.0368	43.76	2136000	1.8938
2.0298	43.93	2144000	1.8874
2.0298	44.09	2152000	1.9065
2.0353	44.25	2160000	1.9097
2.0353	44.42	2168000	1.8985
2.0324	44.58	2176000	1.9160
2.0324	44.75	2184000	1.9060
2.0316	44.91	2192000	1.8912
2.0316	45.07	2200000	1.9014
2.0322	45.24	2208000	1.9031
2.0322	45.4	2216000	1.9086
2.035	45.57	2224000	1.9147
2.035	45.73	2232000	1.9004
2.0431	45.89	2240000	1.9027
2.0431	46.06	2248000	1.8916
2.0347	46.22	2256000	1.9049
2.0347	46.38	2264000	1.8973
2.0353	46.55	2272000	1.8932
2.0353	46.71	2280000	1.9121
2.0309	46.88	2288000	1.8973
2.0309	47.04	2296000	1.8986
2.0359	47.2	2304000	1.9202
2.0359	47.37	2312000	1.8778
2.037	47.53	2320000	1.9137
2.037	47.7	2328000	1.9095
2.0306	47.86	2336000	1.9040
2.0306	48.02	2344000	1.8932
2.0342	48.19	2352000	1.9152
2.0342	48.35	2360000	1.9059
2.0457	48.52	2368000	1.8845
2.0457	48.68	2376000	1.9040
2.0349	48.84	2384000	1.9000
2.0349	49.01	2392000	1.9059
2.0322	49.17	2400000	1.8926

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0