2020-Q3-25p-filtered-random

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.02	8000	2.5582
2.8015	0.04	16000	2.4569
2.8015	0.07	24000	2.4054
2.5403	0.09	32000	2.3788
2.5403	0.11	40000	2.3619
2.475	0.13	48000	2.3437
2.475	0.16	56000	2.3306
2.4451	0.18	64000	2.3220
2.4451	0.2	72000	2.3136
2.4333	0.22	80000	2.3125
2.4333	0.25	88000	2.3113
2.4234	0.27	96000	2.3007
2.4234	0.29	104000	2.3005
2.4151	0.31	112000	2.2946
2.4151	0.34	120000	2.2902
2.4156	0.36	128000	2.2845
2.4156	0.38	136000	2.2922
2.3994	0.4	144000	2.2819
2.3994	0.43	152000	2.2835
2.4088	0.45	160000	2.2824
2.4088	0.47	168000	2.2797
2.3996	0.49	176000	2.2816
2.3996	0.52	184000	2.2791
2.396	0.54	192000	2.2770
2.396	0.56	200000	2.2788
2.396	0.58	208000	2.2701
2.396	0.61	216000	2.2703
2.403	0.63	224000	2.2720
2.403	0.65	232000	2.2788
2.3889	0.67	240000	2.2739
2.3889	0.7	248000	2.2721
2.3976	0.72	256000	2.2786
2.3976	0.74	264000	2.2715
2.3939	0.76	272000	2.2716
2.3939	0.79	280000	2.2699
2.393	0.81	288000	2.2702
2.393	0.83	296000	2.2722
2.3884	0.85	304000	2.2711
2.3884	0.88	312000	2.2697
2.3939	0.9	320000	2.2653
2.3939	0.92	328000	2.2678
2.3981	0.94	336000	2.2675
2.3981	0.97	344000	2.2681
2.3936	0.99	352000	2.2644
2.3936	1.01	360000	2.2698
2.3916	1.03	368000	2.2729
2.3916	1.06	376000	2.2722
2.3975	1.08	384000	2.2694
2.3975	1.1	392000	2.2626
2.3946	1.12	400000	2.2714
2.3946	1.15	408000	2.2756
2.3974	1.17	416000	2.2653
2.3974	1.19	424000	2.2649
2.3873	1.21	432000	2.2722
2.3873	1.24	440000	2.2651
2.3922	1.26	448000	2.2638
2.3922	1.28	456000	2.2621
2.3983	1.3	464000	2.2671
2.3983	1.32	472000	2.2651
2.3883	1.35	480000	2.2631
2.3883	1.37	488000	2.2729
2.3909	1.39	496000	2.2618
2.3909	1.41	504000	2.2631
2.3885	1.44	512000	2.2639
2.3885	1.46	520000	2.2590
2.3977	1.48	528000	2.2652
2.3977	1.5	536000	2.2632
2.3968	1.53	544000	2.2666
2.3968	1.55	552000	2.2697
2.3941	1.57	560000	2.2703
2.3941	1.59	568000	2.2632
2.3916	1.62	576000	2.2613
2.3916	1.64	584000	2.2663
2.3878	1.66	592000	2.2593
2.3878	1.68	600000	2.2636
2.3955	1.71	608000	2.2624
2.3955	1.73	616000	2.2627
2.3921	1.75	624000	2.2676
2.3921	1.77	632000	2.2675
2.3971	1.8	640000	2.2690
2.3971	1.82	648000	2.2617
2.3979	1.84	656000	2.2619
2.3979	1.86	664000	2.2666
2.3917	1.89	672000	2.2586
2.3917	1.91	680000	2.2634
2.4004	1.93	688000	2.2631
2.4004	1.95	696000	2.2656
2.3881	1.98	704000	2.2650
2.3881	2.0	712000	2.2618
2.3988	2.02	720000	2.2623
2.3988	2.04	728000	2.2654
2.3919	2.07	736000	2.2622
2.3919	2.09	744000	2.2658
2.3872	2.11	752000	2.2639
2.3872	2.13	760000	2.2578
2.3921	2.16	768000	2.2647
2.3921	2.18	776000	2.2635
2.3956	2.2	784000	2.2609
2.3956	2.22	792000	2.2617
2.4026	2.25	800000	2.2605
2.4026	2.27	808000	2.2619
2.3931	2.29	816000	2.2663
2.3931	2.31	824000	2.2649
2.3958	2.34	832000	2.2655
2.3958	2.36	840000	2.2611
2.3968	2.38	848000	2.2693
2.3968	2.4	856000	2.2639
2.3963	2.43	864000	2.2589
2.3963	2.45	872000	2.2650
2.3921	2.47	880000	2.2654
2.3921	2.49	888000	2.2626
2.3912	2.52	896000	2.2655
2.3912	2.54	904000	2.2635
2.3978	2.56	912000	2.2634
2.3978	2.58	920000	2.2605
2.4009	2.6	928000	2.2601
2.4009	2.63	936000	2.2603
2.3917	2.65	944000	2.2678
2.3917	2.67	952000	2.2693
2.3955	2.69	960000	2.2640
2.3955	2.72	968000	2.2613
2.3962	2.74	976000	2.2723
2.3962	2.76	984000	2.2613
2.396	2.78	992000	2.2600
2.396	2.81	1000000	2.2651
2.3961	2.83	1008000	2.2630
2.3961	2.85	1016000	2.2596
2.399	2.87	1024000	2.2606
2.399	2.9	1032000	2.2570
2.3981	2.92	1040000	2.2623
2.3981	2.94	1048000	2.2630
2.4028	2.96	1056000	2.2661
2.4028	2.99	1064000	2.2604
2.403	3.01	1072000	2.2642
2.403	3.03	1080000	2.2600
2.3975	3.05	1088000	2.2654
2.3975	3.08	1096000	2.2660
2.3974	3.1	1104000	2.2703
2.3974	3.12	1112000	2.2650
2.4014	3.14	1120000	2.2652
2.4014	3.17	1128000	2.2660
2.3964	3.19	1136000	2.2625
2.3964	3.21	1144000	2.2614
2.3942	3.23	1152000	2.2656
2.3942	3.26	1160000	2.2653
2.3969	3.28	1168000	2.2617
2.3969	3.3	1176000	2.2617
2.3953	3.32	1184000	2.2610
2.3953	3.35	1192000	2.2649
2.402	3.37	1200000	2.2695
2.402	3.39	1208000	2.2630
2.3974	3.41	1216000	2.2667
2.3974	3.44	1224000	2.2631
2.3993	3.46	1232000	2.2646
2.3993	3.48	1240000	2.2682
2.3999	3.5	1248000	2.2665
2.3999	3.53	1256000	2.2631
2.3952	3.55	1264000	2.2640
2.3952	3.57	1272000	2.2618
2.3914	3.59	1280000	2.2626
2.3914	3.62	1288000	2.2658
2.4113	3.64	1296000	2.2582
2.4113	3.66	1304000	2.2590
2.4021	3.68	1312000	2.2641
2.4021	3.71	1320000	2.2554
2.402	3.73	1328000	2.2629
2.402	3.75	1336000	2.2635
2.3989	3.77	1344000	2.2699
2.3989	3.8	1352000	2.2639
2.3998	3.82	1360000	2.2627
2.3998	3.84	1368000	2.2654
2.3968	3.86	1376000	2.2674
2.3968	3.88	1384000	2.2633
2.3993	3.91	1392000	2.2672
2.3993	3.93	1400000	2.2599
2.3991	3.95	1408000	2.2602
2.3991	3.97	1416000	2.2573
2.3971	4.0	1424000	2.2686
2.3971	4.02	1432000	2.2629
2.4047	4.04	1440000	2.2650
2.4047	4.06	1448000	2.2637
2.3952	4.09	1456000	2.2654
2.3952	4.11	1464000	2.2669
2.3994	4.13	1472000	2.2636
2.3994	4.15	1480000	2.2661
2.4003	4.18	1488000	2.2649
2.4003	4.2	1496000	2.2640
2.3959	4.22	1504000	2.2634
2.3959	4.24	1512000	2.2706
2.4023	4.27	1520000	2.2580
2.4023	4.29	1528000	2.2693
2.3974	4.31	1536000	2.2666
2.3974	4.33	1544000	2.2633
2.3944	4.36	1552000	2.2657
2.3944	4.38	1560000	2.2611
2.3974	4.4	1568000	2.2558
2.3974	4.42	1576000	2.2614
2.4024	4.45	1584000	2.2690
2.4024	4.47	1592000	2.2642
2.4024	4.49	1600000	2.2616
2.4024	4.51	1608000	2.2639
2.3981	4.54	1616000	2.2636
2.3981	4.56	1624000	2.2696
2.4041	4.58	1632000	2.2675
2.4041	4.6	1640000	2.2653
2.3972	4.63	1648000	2.2658
2.3972	4.65	1656000	2.2591
2.3997	4.67	1664000	2.2671
2.3997	4.69	1672000	2.2607
2.3918	4.72	1680000	2.2585
2.3918	4.74	1688000	2.2621
2.4069	4.76	1696000	2.2623
2.4069	4.78	1704000	2.2633
2.4039	4.81	1712000	2.2622
2.4039	4.83	1720000	2.2627
2.4077	4.85	1728000	2.2686
2.4077	4.87	1736000	2.2594
2.398	4.9	1744000	2.2659
2.398	4.92	1752000	2.2684
2.4007	4.94	1760000	2.2617
2.4007	4.96	1768000	2.2646
2.4059	4.99	1776000	2.2610
2.4059	5.01	1784000	2.2591
2.3996	5.03	1792000	2.2641
2.3996	5.05	1800000	2.2607
2.4015	5.08	1808000	2.2580
2.4015	5.1	1816000	2.2605
2.4007	5.12	1824000	2.2649
2.4007	5.14	1832000	2.2641
2.4019	5.16	1840000	2.2626
2.4019	5.19	1848000	2.2580
2.4017	5.21	1856000	2.2643
2.4017	5.23	1864000	2.2598
2.3997	5.25	1872000	2.2604
2.3997	5.28	1880000	2.2674
2.3973	5.3	1888000	2.2661
2.3973	5.32	1896000	2.2667
2.4004	5.34	1904000	2.2663
2.4004	5.37	1912000	2.2639
2.4034	5.39	1920000	2.2657
2.4034	5.41	1928000	2.2637
2.3907	5.43	1936000	2.2622
2.3907	5.46	1944000	2.2630
2.3935	5.48	1952000	2.2547
2.3935	5.5	1960000	2.2676
2.3954	5.52	1968000	2.2630
2.3954	5.55	1976000	2.2677
2.3995	5.57	1984000	2.2678
2.3995	5.59	1992000	2.2642
2.398	5.61	2000000	2.2613
2.398	5.64	2008000	2.2627
2.3971	5.66	2016000	2.2584
2.3971	5.68	2024000	2.2700
2.3988	5.7	2032000	2.2715
2.3988	5.73	2040000	2.2640
2.3933	5.75	2048000	2.2628
2.3933	5.77	2056000	2.2619
2.4007	5.79	2064000	2.2672
2.4007	5.82	2072000	2.2653
2.3978	5.84	2080000	2.2631
2.3978	5.86	2088000	2.2632
2.4002	5.88	2096000	2.2599
2.4002	5.91	2104000	2.2642
2.4041	5.93	2112000	2.2616
2.4041	5.95	2120000	2.2602
2.4008	5.97	2128000	2.2553
2.4008	6.0	2136000	2.2599
2.4003	6.02	2144000	2.2645
2.4003	6.04	2152000	2.2596
2.3998	6.06	2160000	2.2614
2.3998	6.09	2168000	2.2666
2.4007	6.11	2176000	2.2570
2.4007	6.13	2184000	2.2628
2.3891	6.15	2192000	2.2558
2.3891	6.18	2200000	2.2666
2.4011	6.2	2208000	2.2614
2.4011	6.22	2216000	2.2646
2.3957	6.24	2224000	2.2645
2.3957	6.27	2232000	2.2653
2.3973	6.29	2240000	2.2630
2.3973	6.31	2248000	2.2630
2.3964	6.33	2256000	2.2621
2.3964	6.36	2264000	2.2608
2.3988	6.38	2272000	2.2651
2.3988	6.4	2280000	2.2636
2.4004	6.42	2288000	2.2602
2.4004	6.44	2296000	2.2613
2.4006	6.47	2304000	2.2661
2.4006	6.49	2312000	2.2635
2.401	6.51	2320000	2.2601
2.401	6.53	2328000	2.2653
2.4048	6.56	2336000	2.2623
2.4048	6.58	2344000	2.2608
2.404	6.6	2352000	2.2592
2.404	6.62	2360000	2.2612
2.3997	6.65	2368000	2.2584
2.3997	6.67	2376000	2.2646
2.4044	6.69	2384000	2.2646
2.4044	6.71	2392000	2.2654
2.4003	6.74	2400000	2.2660

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0