2020-Q4-50p-filtered-random

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2570

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.03	8000	2.5888
2.8176	0.07	16000	2.4814
2.8176	0.1	24000	2.4264
2.5609	0.13	32000	2.3993
2.5609	0.17	40000	2.3761
2.4969	0.2	48000	2.3624
2.4969	0.24	56000	2.3481
2.48	0.27	64000	2.3399
2.48	0.3	72000	2.3289
2.451	0.34	80000	2.3221
2.451	0.37	88000	2.3183
2.4367	0.4	96000	2.3221
2.4367	0.44	104000	2.3142
2.4388	0.47	112000	2.3028
2.4388	0.51	120000	2.3066
2.4215	0.54	128000	2.3013
2.4215	0.57	136000	2.3039
2.4178	0.61	144000	2.2907
2.4178	0.64	152000	2.2996
2.4103	0.67	160000	2.2943
2.4103	0.71	168000	2.2900
2.4122	0.74	176000	2.2902
2.4122	0.77	184000	2.2961
2.4173	0.81	192000	2.2906
2.4173	0.84	200000	2.2925
2.4067	0.88	208000	2.2911
2.4067	0.91	216000	2.2844
2.4059	0.94	224000	2.2855
2.4059	0.98	232000	2.2811
2.4089	1.01	240000	2.2788
2.4089	1.04	248000	2.2796
2.4034	1.08	256000	2.2827
2.4034	1.11	264000	2.2803
2.408	1.15	272000	2.2746
2.408	1.18	280000	2.2851
2.3985	1.21	288000	2.2781
2.3985	1.25	296000	2.2795
2.4009	1.28	304000	2.2777
2.4009	1.31	312000	2.2770
2.4017	1.35	320000	2.2763
2.4017	1.38	328000	2.2734
2.4056	1.41	336000	2.2758
2.4056	1.45	344000	2.2763
2.4017	1.48	352000	2.2700
2.4017	1.52	360000	2.2736
2.3993	1.55	368000	2.2763
2.3993	1.58	376000	2.2792
2.3994	1.62	384000	2.2666
2.3994	1.65	392000	2.2699
2.3969	1.68	400000	2.2753
2.3969	1.72	408000	2.2707
2.4094	1.75	416000	2.2731
2.4094	1.79	424000	2.2709
2.4102	1.82	432000	2.2623
2.4102	1.85	440000	2.2751
2.4042	1.89	448000	2.2728
2.4042	1.92	456000	2.2714
2.3991	1.95	464000	2.2634
2.3991	1.99	472000	2.2695
2.3976	2.02	480000	2.2731
2.3976	2.05	488000	2.2736
2.4019	2.09	496000	2.2803
2.4019	2.12	504000	2.2699
2.4044	2.16	512000	2.2731
2.4044	2.19	520000	2.2709
2.3989	2.22	528000	2.2716
2.3989	2.26	536000	2.2668
2.4068	2.29	544000	2.2728
2.4068	2.32	552000	2.2709
2.4047	2.36	560000	2.2683
2.4047	2.39	568000	2.2731
2.3976	2.43	576000	2.2676
2.3976	2.46	584000	2.2736
2.3994	2.49	592000	2.2624
2.3994	2.53	600000	2.2773
2.3997	2.56	608000	2.2719
2.3997	2.59	616000	2.2701
2.4042	2.63	624000	2.2695
2.4042	2.66	632000	2.2666
2.3994	2.69	640000	2.2719
2.3994	2.73	648000	2.2686
2.3953	2.76	656000	2.2623
2.3953	2.8	664000	2.2662
2.402	2.83	672000	2.2707
2.402	2.86	680000	2.2662
2.3929	2.9	688000	2.2726
2.3929	2.93	696000	2.2682
2.3977	2.96	704000	2.2634
2.3977	3.0	712000	2.2685
2.4022	3.03	720000	2.2693
2.4022	3.07	728000	2.2666
2.4046	3.1	736000	2.2690
2.4046	3.13	744000	2.2641
2.3977	3.17	752000	2.2658
2.3977	3.2	760000	2.2645
2.4015	3.23	768000	2.2619
2.4015	3.27	776000	2.2671
2.393	3.3	784000	2.2694
2.393	3.33	792000	2.2662
2.3907	3.37	800000	2.2691
2.3907	3.4	808000	2.2679
2.3987	3.44	816000	2.2688
2.3987	3.47	824000	2.2655
2.4116	3.5	832000	2.2668
2.4116	3.54	840000	2.2675
2.3913	3.57	848000	2.2689
2.3913	3.6	856000	2.2642
2.3974	3.64	864000	2.2667
2.3974	3.67	872000	2.2717
2.4046	3.71	880000	2.2661
2.4046	3.74	888000	2.2705
2.4006	3.77	896000	2.2637
2.4006	3.81	904000	2.2635
2.3987	3.84	912000	2.2642
2.3987	3.87	920000	2.2691
2.4068	3.91	928000	2.2689
2.4068	3.94	936000	2.2730
2.4092	3.97	944000	2.2644
2.4092	4.01	952000	2.2706
2.4035	4.04	960000	2.2671
2.4035	4.08	968000	2.2562
2.4005	4.11	976000	2.2622
2.4005	4.14	984000	2.2642
2.406	4.18	992000	2.2625
2.406	4.21	1000000	2.2662
2.3972	4.24	1008000	2.2658
2.3972	4.28	1016000	2.2668
2.3937	4.31	1024000	2.2593
2.3937	4.35	1032000	2.2712
2.3982	4.38	1040000	2.2695
2.3982	4.41	1048000	2.2684
2.4034	4.45	1056000	2.2643
2.4034	4.48	1064000	2.2665
2.3996	4.51	1072000	2.2692
2.3996	4.55	1080000	2.2628
2.4054	4.58	1088000	2.2673
2.4054	4.61	1096000	2.2577
2.4039	4.65	1104000	2.2671
2.4039	4.68	1112000	2.2586
2.4033	4.72	1120000	2.2730
2.4033	4.75	1128000	2.2655
2.4036	4.78	1136000	2.2694
2.4036	4.82	1144000	2.2630
2.4036	4.85	1152000	2.2618
2.4036	4.88	1160000	2.2665
2.4005	4.92	1168000	2.2609
2.4005	4.95	1176000	2.2617
2.4065	4.99	1184000	2.2646
2.4065	5.02	1192000	2.2606
2.4044	5.05	1200000	2.2656
2.4044	5.09	1208000	2.2630
2.3997	5.12	1216000	2.2737
2.3997	5.15	1224000	2.2762
2.407	5.19	1232000	2.2669
2.407	5.22	1240000	2.2695
2.4013	5.25	1248000	2.2704
2.4013	5.29	1256000	2.2612
2.4118	5.32	1264000	2.2654
2.4118	5.36	1272000	2.2683
2.3953	5.39	1280000	2.2628
2.3953	5.42	1288000	2.2605
2.3973	5.46	1296000	2.2667
2.3973	5.49	1304000	2.2640
2.4027	5.52	1312000	2.2619
2.4027	5.56	1320000	2.2687
2.3967	5.59	1328000	2.2598
2.3967	5.63	1336000	2.2621
2.4028	5.66	1344000	2.2602
2.4028	5.69	1352000	2.2713
2.4053	5.73	1360000	2.2623
2.4053	5.76	1368000	2.2697
2.3987	5.79	1376000	2.2638
2.3987	5.83	1384000	2.2601
2.3987	5.86	1392000	2.2642
2.3987	5.89	1400000	2.2656
2.401	5.93	1408000	2.2712
2.401	5.96	1416000	2.2639
2.4011	6.0	1424000	2.2646
2.4011	6.03	1432000	2.2669
2.4022	6.06	1440000	2.2619
2.4022	6.1	1448000	2.2580
2.3998	6.13	1456000	2.2612
2.3998	6.16	1464000	2.2652
2.3999	6.2	1472000	2.2610
2.3999	6.23	1480000	2.2567
2.3984	6.27	1488000	2.2590
2.3984	6.3	1496000	2.2565
2.4017	6.33	1504000	2.2658
2.4017	6.37	1512000	2.2626
2.4055	6.4	1520000	2.2656
2.4055	6.43	1528000	2.2622
2.3959	6.47	1536000	2.2691
2.3959	6.5	1544000	2.2604
2.4016	6.53	1552000	2.2599
2.4016	6.57	1560000	2.2655
2.3986	6.6	1568000	2.2684
2.3986	6.64	1576000	2.2716
2.4051	6.67	1584000	2.2605
2.4051	6.7	1592000	2.2569
2.4057	6.74	1600000	2.2687
2.4057	6.77	1608000	2.2571
2.3956	6.8	1616000	2.2664
2.3956	6.84	1624000	2.2612
2.4048	6.87	1632000	2.2643
2.4048	6.91	1640000	2.2633
2.4042	6.94	1648000	2.2634
2.4042	6.97	1656000	2.2637
2.4008	7.01	1664000	2.2619
2.4008	7.04	1672000	2.2579
2.397	7.07	1680000	2.2628
2.397	7.11	1688000	2.2593
2.4044	7.14	1696000	2.2593
2.4044	7.17	1704000	2.2613
2.3979	7.21	1712000	2.2685
2.3979	7.24	1720000	2.2683
2.4017	7.28	1728000	2.2611
2.4017	7.31	1736000	2.2672
2.4017	7.34	1744000	2.2577
2.4017	7.38	1752000	2.2609
2.4018	7.41	1760000	2.2567
2.4018	7.44	1768000	2.2661
2.3905	7.48	1776000	2.2671
2.3905	7.51	1784000	2.2663
2.4063	7.55	1792000	2.2619
2.4063	7.58	1800000	2.2587
2.4015	7.61	1808000	2.2584
2.4015	7.65	1816000	2.2580
2.3984	7.68	1824000	2.2586
2.3984	7.71	1832000	2.2620
2.3962	7.75	1840000	2.2584
2.3962	7.78	1848000	2.2607
2.3998	7.81	1856000	2.2638
2.3998	7.85	1864000	2.2629
2.4005	7.88	1872000	2.2716
2.4005	7.92	1880000	2.2623
2.4006	7.95	1888000	2.2555
2.4006	7.98	1896000	2.2653
2.3946	8.02	1904000	2.2629
2.3946	8.05	1912000	2.2654
2.3983	8.08	1920000	2.2623
2.3983	8.12	1928000	2.2544
2.4038	8.15	1936000	2.2605
2.4038	8.19	1944000	2.2622
2.399	8.22	1952000	2.2600
2.399	8.25	1960000	2.2629
2.3983	8.29	1968000	2.2621
2.3983	8.32	1976000	2.2609
2.4059	8.35	1984000	2.2705
2.4059	8.39	1992000	2.2572
2.4058	8.42	2000000	2.2602
2.4058	8.45	2008000	2.2626
2.3954	8.49	2016000	2.2668
2.3954	8.52	2024000	2.2599
2.3932	8.56	2032000	2.2643
2.3932	8.59	2040000	2.2559
2.4001	8.62	2048000	2.2614
2.4001	8.66	2056000	2.2577
2.3912	8.69	2064000	2.2665
2.3912	8.72	2072000	2.2576
2.4015	8.76	2080000	2.2672
2.4015	8.79	2088000	2.2598
2.4015	8.83	2096000	2.2599
2.4015	8.86	2104000	2.2641
2.399	8.89	2112000	2.2612
2.399	8.93	2120000	2.2607
2.3963	8.96	2128000	2.2633
2.3963	8.99	2136000	2.2567
2.3957	9.03	2144000	2.2630
2.3957	9.06	2152000	2.2597
2.3943	9.09	2160000	2.2624
2.3943	9.13	2168000	2.2599
2.4025	9.16	2176000	2.2578
2.4025	9.2	2184000	2.2640
2.3944	9.23	2192000	2.2562
2.3944	9.26	2200000	2.2660
2.3964	9.3	2208000	2.2556
2.3964	9.33	2216000	2.2697
2.4026	9.36	2224000	2.2652
2.4026	9.4	2232000	2.2571
2.398	9.43	2240000	2.2555
2.398	9.47	2248000	2.2607
2.4038	9.5	2256000	2.2558
2.4038	9.53	2264000	2.2660
2.4027	9.57	2272000	2.2587
2.4027	9.6	2280000	2.2605
2.4025	9.63	2288000	2.2578
2.4025	9.67	2296000	2.2609
2.3969	9.7	2304000	2.2597
2.3969	9.73	2312000	2.2619
2.3886	9.77	2320000	2.2645
2.3886	9.8	2328000	2.2717
2.3942	9.84	2336000	2.2627
2.3942	9.87	2344000	2.2582
2.396	9.9	2352000	2.2634
2.396	9.94	2360000	2.2582
2.3998	9.97	2368000	2.2643
2.3998	10.0	2376000	2.2690
2.4014	10.04	2384000	2.2655
2.4014	10.07	2392000	2.2660
2.4004	10.11	2400000	2.2650

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0