2020-Q2-full_tweets

This model is a fine-tuned version of DouglasPontes/2020-Q1-full_tweets on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.0061

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.01	8000	2.1043
2.2608	0.02	16000	2.0934
2.2608	0.03	24000	2.0862
2.2409	0.03	32000	2.0805
2.2409	0.04	40000	2.0793
2.2278	0.05	48000	2.0718
2.2278	0.06	56000	2.0753
2.2059	0.07	64000	2.0668
2.2059	0.08	72000	2.0657
2.1997	0.09	80000	2.0620
2.1997	0.1	88000	2.0553
2.1988	0.1	96000	2.0569
2.1988	0.11	104000	2.0525
2.1861	0.12	112000	2.0556
2.1861	0.13	120000	2.0493
2.1823	0.14	128000	2.0509
2.1823	0.15	136000	2.0461
2.1851	0.16	144000	2.0476
2.1851	0.17	152000	2.0450
2.1862	0.17	160000	2.0469
2.1862	0.18	168000	2.0442
2.1741	0.19	176000	2.0456
2.1741	0.2	184000	2.0442
2.181	0.21	192000	2.0402
2.181	0.22	200000	2.0423
2.1692	0.23	208000	2.0413
2.1692	0.24	216000	2.0448
2.1678	0.24	224000	2.0418
2.1678	0.25	232000	2.0417
2.1756	0.26	240000	2.0342
2.1756	0.27	248000	2.0377
2.1752	0.28	256000	2.0381
2.1752	0.29	264000	2.0354
2.1673	0.3	272000	2.0381
2.1673	0.31	280000	2.0375
2.1585	0.31	288000	2.0336
2.1585	0.32	296000	2.0344
2.1703	0.33	304000	2.0348
2.1703	0.34	312000	2.0330
2.1667	0.35	320000	2.0352
2.1667	0.36	328000	2.0359
2.1649	0.37	336000	2.0317
2.1649	0.38	344000	2.0314
2.1564	0.38	352000	2.0306
2.1564	0.39	360000	2.0299
2.161	0.4	368000	2.0317
2.161	0.41	376000	2.0325
2.1551	0.42	384000	2.0274
2.1551	0.43	392000	2.0282
2.1602	0.44	400000	2.0301
2.1602	0.45	408000	2.0303
2.1581	0.45	416000	2.0260
2.1581	0.46	424000	2.0248
2.1494	0.47	432000	2.0265
2.1494	0.48	440000	2.0247
2.1508	0.49	448000	2.0231
2.1508	0.5	456000	2.0276
2.153	0.51	464000	2.0276
2.153	0.51	472000	2.0242
2.1489	0.52	480000	2.0259
2.1489	0.53	488000	2.0257
2.1468	0.54	496000	2.0275
2.1468	0.55	504000	2.0303
2.1446	0.56	512000	2.0248
2.1446	0.57	520000	2.0286
2.1409	0.58	528000	2.0211
2.1409	0.58	536000	2.0204
2.1536	0.59	544000	2.0199
2.1536	0.6	552000	2.0281
2.1416	0.61	560000	2.0237
2.1416	0.62	568000	2.0231
2.1502	0.63	576000	2.0205
2.1502	0.64	584000	2.0217
2.1424	0.65	592000	2.0242
2.1424	0.65	600000	2.0238
2.1469	0.66	608000	2.0192
2.1469	0.67	616000	2.0249
2.145	0.68	624000	2.0196
2.145	0.69	632000	2.0224
2.1503	0.7	640000	2.0216
2.1503	0.71	648000	2.0228
2.1355	0.72	656000	2.0197
2.1355	0.72	664000	2.0240
2.1392	0.73	672000	2.0232
2.1392	0.74	680000	2.0209
2.1378	0.75	688000	2.0219
2.1378	0.76	696000	2.0192
2.1446	0.77	704000	2.0195
2.1446	0.78	712000	2.0197
2.1351	0.79	720000	2.0184
2.1351	0.79	728000	2.0162
2.1437	0.8	736000	2.0151
2.1437	0.81	744000	2.0202
2.1249	0.82	752000	2.0169
2.1249	0.83	760000	2.0189
2.1355	0.84	768000	2.0221
2.1355	0.85	776000	2.0194
2.1387	0.86	784000	2.0189
2.1387	0.86	792000	2.0165
2.1334	0.87	800000	2.0169
2.1334	0.88	808000	2.0189
2.137	0.89	816000	2.0162
2.137	0.9	824000	2.0168
2.1331	0.91	832000	2.0193
2.1331	0.92	840000	2.0166
2.1293	0.93	848000	2.0137
2.1293	0.93	856000	2.0183
2.1358	0.94	864000	2.0184
2.1358	0.95	872000	2.0171
2.1296	0.96	880000	2.0179
2.1296	0.97	888000	2.0152
2.1319	0.98	896000	2.0174
2.1319	0.99	904000	2.0206
2.1344	1.0	912000	2.0179
2.1344	1.0	920000	2.0154
2.1352	1.01	928000	2.0185
2.1352	1.02	936000	2.0170
2.1336	1.03	944000	2.0164
2.1336	1.04	952000	2.0137
2.1315	1.05	960000	2.0176
2.1315	1.06	968000	2.0155
2.1255	1.06	976000	2.0145
2.1255	1.07	984000	2.0233
2.1249	1.08	992000	2.0148
2.1249	1.09	1000000	2.0162
2.123	1.1	1008000	2.0174
2.123	1.11	1016000	2.0150
2.1263	1.12	1024000	2.0161
2.1263	1.13	1032000	2.0129
2.1232	1.13	1040000	2.0167
2.1232	1.14	1048000	2.0125
2.1168	1.15	1056000	2.0113
2.1168	1.16	1064000	2.0136
2.1307	1.17	1072000	2.0143
2.1307	1.18	1080000	2.0166
2.1336	1.19	1088000	2.0103
2.1336	1.2	1096000	2.0130
2.1227	1.2	1104000	2.0125
2.1227	1.21	1112000	2.0183
2.1223	1.22	1120000	2.0148
2.1223	1.23	1128000	2.0147
2.1289	1.24	1136000	2.0109
2.1289	1.25	1144000	2.0164
2.1278	1.26	1152000	2.0163
2.1278	1.27	1160000	2.0121
2.1261	1.27	1168000	2.0113
2.1261	1.28	1176000	2.0137
2.126	1.29	1184000	2.0152
2.126	1.3	1192000	2.0104
2.1235	1.31	1200000	2.0132
2.1235	1.32	1208000	2.0114
2.1229	1.33	1216000	2.0105
2.1229	1.34	1224000	2.0131
2.1213	1.34	1232000	2.0141
2.1213	1.35	1240000	2.0109
2.1185	1.36	1248000	2.0129
2.1185	1.37	1256000	2.0110
2.131	1.38	1264000	2.0123
2.131	1.39	1272000	2.0105
2.1141	1.4	1280000	2.0104
2.1141	1.41	1288000	2.0150
2.1219	1.41	1296000	2.0161
2.1219	1.42	1304000	2.0093
2.1203	1.43	1312000	2.0104
2.1203	1.44	1320000	2.0144
2.1264	1.45	1328000	2.0085
2.1264	1.46	1336000	2.0119
2.1194	1.47	1344000	2.0118
2.1194	1.48	1352000	2.0110
2.117	1.48	1360000	2.0147
2.117	1.49	1368000	2.0135
2.1311	1.5	1376000	2.0077
2.1311	1.51	1384000	2.0066
2.1215	1.52	1392000	2.0089
2.1215	1.53	1400000	2.0118
2.1185	1.54	1408000	2.0105
2.1185	1.54	1416000	2.0123
2.1284	1.55	1424000	2.0134
2.1284	1.56	1432000	2.0093
2.1174	1.57	1440000	2.0102
2.1174	1.58	1448000	2.0076
2.1108	1.59	1456000	2.0074
2.1108	1.6	1464000	2.0071
2.1252	1.61	1472000	2.0092
2.1252	1.61	1480000	2.0080
2.121	1.62	1488000	2.0053
2.121	1.63	1496000	2.0072
2.1178	1.64	1504000	2.0059
2.1178	1.65	1512000	2.0084
2.1154	1.66	1520000	2.0106
2.1154	1.67	1528000	2.0117
2.1214	1.68	1536000	2.0070
2.1214	1.68	1544000	2.0079
2.1175	1.69	1552000	2.0102
2.1175	1.7	1560000	2.0097
2.1206	1.71	1568000	2.0092
2.1206	1.72	1576000	2.0055
2.1302	1.73	1584000	2.0085
2.1302	1.74	1592000	2.0110
2.1177	1.75	1600000	2.0065
2.1177	1.75	1608000	2.0132
2.1101	1.76	1616000	2.0086
2.1101	1.77	1624000	2.0077
2.1194	1.78	1632000	2.0081
2.1194	1.79	1640000	2.0088
2.1167	1.8	1648000	2.0022
2.1167	1.81	1656000	2.0077
2.1083	1.82	1664000	2.0066
2.1083	1.82	1672000	2.0137
2.1232	1.83	1680000	2.0067
2.1232	1.84	1688000	2.0039
2.1212	1.85	1696000	2.0090
2.1212	1.86	1704000	2.0079
2.1246	1.87	1712000	2.0083
2.1246	1.88	1720000	2.0039
2.1129	1.89	1728000	2.0069
2.1129	1.89	1736000	2.0079
2.1209	1.9	1744000	2.0058
2.1209	1.91	1752000	2.0072
2.1209	1.92	1760000	2.0068
2.1209	1.93	1768000	2.0079
2.1184	1.94	1776000	2.0036
2.1184	1.95	1784000	2.0065
2.1065	1.96	1792000	2.0077
2.1065	1.96	1800000	2.0062
2.109	1.97	1808000	2.0090
2.109	1.98	1816000	2.0124
2.1081	1.99	1824000	2.0066
2.1081	2.0	1832000	2.0081
2.1151	2.01	1840000	2.0085
2.1151	2.02	1848000	2.0054
2.1178	2.03	1856000	2.0058
2.1178	2.03	1864000	2.0048
2.1035	2.04	1872000	2.0040
2.1035	2.05	1880000	2.0059
2.1197	2.06	1888000	2.0071
2.1197	2.07	1896000	2.0057
2.1143	2.08	1904000	2.0059
2.1143	2.09	1912000	2.0043
2.1082	2.09	1920000	2.0068
2.1082	2.1	1928000	2.0057
2.1202	2.11	1936000	2.0072
2.1202	2.12	1944000	2.0057
2.1138	2.13	1952000	2.0051
2.1138	2.14	1960000	2.0085
2.1082	2.15	1968000	2.0076
2.1082	2.16	1976000	2.0077
2.1084	2.16	1984000	2.0020
2.1084	2.17	1992000	2.0050
2.1151	2.18	2000000	2.0066
2.1151	2.19	2008000	2.0031
2.1141	2.2	2016000	2.0128
2.1141	2.21	2024000	2.0022
2.1129	2.22	2032000	2.0065
2.1129	2.23	2040000	2.0054
2.1164	2.23	2048000	2.0039
2.1164	2.24	2056000	2.0031
2.1121	2.25	2064000	2.0101
2.1121	2.26	2072000	2.0099
2.1071	2.27	2080000	2.0042
2.1071	2.28	2088000	2.0030
2.1094	2.29	2096000	2.0048
2.1094	2.3	2104000	2.0046
2.1017	2.3	2112000	2.0039
2.1017	2.31	2120000	2.0011
2.1124	2.32	2128000	2.0071
2.1124	2.33	2136000	2.0061
2.1064	2.34	2144000	2.0040
2.1064	2.35	2152000	2.0075
2.115	2.36	2160000	2.0026
2.115	2.37	2168000	2.0068
2.114	2.37	2176000	2.0066
2.114	2.38	2184000	2.0080
2.1171	2.39	2192000	2.0032
2.1171	2.4	2200000	2.0036
2.1119	2.41	2208000	2.0048
2.1119	2.42	2216000	2.0059
2.1097	2.43	2224000	2.0058
2.1097	2.44	2232000	2.0049
2.1091	2.44	2240000	2.0058
2.1091	2.45	2248000	2.0032
2.1107	2.46	2256000	2.0077
2.1107	2.47	2264000	2.0032
2.1126	2.48	2272000	2.0055
2.1126	2.49	2280000	2.0026
2.1173	2.5	2288000	2.0062
2.1173	2.51	2296000	2.0039
2.114	2.51	2304000	2.0064
2.114	2.52	2312000	2.0113
2.1131	2.53	2320000	2.0065
2.1131	2.54	2328000	2.0098
2.1045	2.55	2336000	2.0061
2.1045	2.56	2344000	2.0066
2.1144	2.57	2352000	2.0060
2.1144	2.57	2360000	2.0059
2.1086	2.58	2368000	2.0039
2.1086	2.59	2376000	2.0076
2.1058	2.6	2384000	2.0036
2.1058	2.61	2392000	2.0077
2.1112	2.62	2400000	2.0000

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0