2020-Q4-50p-filtered-random-prog_from_Q3

This model is a fine-tuned version of DouglasPontes/2020-Q3-50p-filtered-random on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2730

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.03	8000	2.3816
2.5089	0.07	16000	2.3647
2.5089	0.1	24000	2.3552
2.4989	0.13	32000	2.3528
2.4989	0.17	40000	2.3486
2.4836	0.2	48000	2.3463
2.4836	0.24	56000	2.3411
2.4904	0.27	64000	2.3394
2.4904	0.3	72000	2.3350
2.4733	0.34	80000	2.3309
2.4733	0.37	88000	2.3289
2.4675	0.4	96000	2.3381
2.4675	0.44	104000	2.3317
2.4762	0.47	112000	2.3218
2.4762	0.51	120000	2.3267
2.4616	0.54	128000	2.3241
2.4616	0.57	136000	2.3281
2.4601	0.61	144000	2.3152
2.4601	0.64	152000	2.3243
2.4563	0.67	160000	2.3202
2.4563	0.71	168000	2.3193
2.459	0.74	176000	2.3182
2.459	0.77	184000	2.3247
2.4639	0.81	192000	2.3201
2.4639	0.84	200000	2.3243
2.4561	0.88	208000	2.3218
2.4561	0.91	216000	2.3137
2.4556	0.94	224000	2.3180
2.4556	0.98	232000	2.3147
2.4573	1.01	240000	2.3100
2.4573	1.04	248000	2.3118
2.4516	1.08	256000	2.3158
2.4516	1.11	264000	2.3133
2.4561	1.15	272000	2.3065
2.4561	1.18	280000	2.3183
2.4476	1.21	288000	2.3106
2.4476	1.25	296000	2.3131
2.4503	1.28	304000	2.3104
2.4503	1.31	312000	2.3101
2.4495	1.35	320000	2.3086
2.4495	1.38	328000	2.3057
2.4534	1.41	336000	2.3086
2.4534	1.45	344000	2.3093
2.4486	1.48	352000	2.3018
2.4486	1.52	360000	2.3060
2.4457	1.55	368000	2.3083
2.4457	1.58	376000	2.3110
2.4443	1.62	384000	2.2975
2.4443	1.65	392000	2.3009
2.4405	1.68	400000	2.3067
2.4405	1.72	408000	2.3027
2.4531	1.75	416000	2.3050
2.4531	1.79	424000	2.3026
2.4539	1.82	432000	2.2929
2.4539	1.85	440000	2.3051
2.4499	1.89	448000	2.3035
2.4499	1.92	456000	2.3011
2.4401	1.95	464000	2.2920
2.4401	1.99	472000	2.2999
2.4401	2.02	480000	2.3034
2.4401	2.05	488000	2.3021
2.4433	2.09	496000	2.3102
2.4433	2.12	504000	2.2985
2.4445	2.16	512000	2.3018
2.4445	2.19	520000	2.2996
2.4379	2.22	528000	2.3006
2.4379	2.26	536000	2.2970
2.4454	2.29	544000	2.3014
2.4454	2.32	552000	2.2992
2.4457	2.36	560000	2.2962
2.4457	2.39	568000	2.3009
2.4354	2.43	576000	2.2960
2.4354	2.46	584000	2.3008
2.4361	2.49	592000	2.2898
2.4361	2.53	600000	2.3060
2.4377	2.56	608000	2.2990
2.4377	2.59	616000	2.2989
2.4416	2.63	624000	2.2969
2.4416	2.66	632000	2.2933
2.434	2.69	640000	2.2998
2.434	2.73	648000	2.2948
2.433	2.76	656000	2.2892
2.433	2.8	664000	2.2929
2.44	2.83	672000	2.2973
2.44	2.86	680000	2.2926
2.4291	2.9	688000	2.2990
2.4291	2.93	696000	2.2937
2.4336	2.96	704000	2.2894
2.4336	3.0	712000	2.2958
2.439	3.03	720000	2.2956
2.439	3.07	728000	2.2928
2.4405	3.1	736000	2.2956
2.4405	3.13	744000	2.2905
2.4332	3.17	752000	2.2921
2.4332	3.2	760000	2.2907
2.4353	3.23	768000	2.2879
2.4353	3.27	776000	2.2929
2.4273	3.3	784000	2.2953
2.4273	3.33	792000	2.2917
2.4233	3.37	800000	2.2947
2.4233	3.4	808000	2.2943
2.4324	3.44	816000	2.2940
2.4324	3.47	824000	2.2911
2.4461	3.5	832000	2.2920
2.4461	3.54	840000	2.2911
2.4267	3.57	848000	2.2940
2.4267	3.6	856000	2.2890
2.4313	3.64	864000	2.2913
2.4313	3.67	872000	2.2967
2.4388	3.71	880000	2.2907
2.4388	3.74	888000	2.2952
2.4326	3.77	896000	2.2873
2.4326	3.81	904000	2.2871
2.4312	3.84	912000	2.2880
2.4312	3.87	920000	2.2941
2.4398	3.91	928000	2.2925
2.4398	3.94	936000	2.2965
2.441	3.97	944000	2.2890
2.441	4.01	952000	2.2946
2.4345	4.04	960000	2.2910
2.4345	4.08	968000	2.2792
2.4332	4.11	976000	2.2856
2.4332	4.14	984000	2.2879
2.4375	4.18	992000	2.2861
2.4375	4.21	1000000	2.2892
2.4282	4.24	1008000	2.2894
2.4282	4.28	1016000	2.2902
2.4231	4.31	1024000	2.2830
2.4231	4.35	1032000	2.2948
2.4299	4.38	1040000	2.2915
2.4299	4.41	1048000	2.2922
2.4353	4.45	1056000	2.2876
2.4353	4.48	1064000	2.2893
2.4308	4.51	1072000	2.2920
2.4308	4.55	1080000	2.2860
2.4358	4.58	1088000	2.2907
2.4358	4.61	1096000	2.2808
2.4341	4.65	1104000	2.2902
2.4341	4.68	1112000	2.2815
2.4315	4.72	1120000	2.2961
2.4315	4.75	1128000	2.2885
2.434	4.78	1136000	2.2917
2.434	4.82	1144000	2.2851
2.4324	4.85	1152000	2.2837
2.4324	4.88	1160000	2.2883
2.4297	4.92	1168000	2.2824
2.4297	4.95	1176000	2.2832
2.436	4.99	1184000	2.2865
2.436	5.02	1192000	2.2816
2.4329	5.05	1200000	2.2862
2.4329	5.09	1208000	2.2847
2.4276	5.12	1216000	2.2951
2.4276	5.15	1224000	2.2980
2.4362	5.19	1232000	2.2889
2.4362	5.22	1240000	2.2914
2.4309	5.25	1248000	2.2915
2.4309	5.29	1256000	2.2822
2.4414	5.32	1264000	2.2871
2.4414	5.36	1272000	2.2890
2.4241	5.39	1280000	2.2844
2.4241	5.42	1288000	2.2812
2.4251	5.46	1296000	2.2874
2.4251	5.49	1304000	2.2846
2.4318	5.52	1312000	2.2831
2.4318	5.56	1320000	2.2895
2.4247	5.59	1328000	2.2796
2.4247	5.63	1336000	2.2834
2.4305	5.66	1344000	2.2811
2.4305	5.69	1352000	2.2922
2.4336	5.73	1360000	2.2830
2.4336	5.76	1368000	2.2904
2.428	5.79	1376000	2.2843
2.428	5.83	1384000	2.2804
2.4254	5.86	1392000	2.2852
2.4254	5.89	1400000	2.2858
2.4287	5.93	1408000	2.2922
2.4287	5.96	1416000	2.2847
2.4291	6.0	1424000	2.2856
2.4291	6.03	1432000	2.2876
2.4289	6.06	1440000	2.2822
2.4289	6.1	1448000	2.2787
2.4272	6.13	1456000	2.2811
2.4272	6.16	1464000	2.2853
2.4267	6.2	1472000	2.2818
2.4267	6.23	1480000	2.2765
2.4237	6.27	1488000	2.2791
2.4237	6.3	1496000	2.2768
2.4277	6.33	1504000	2.2866
2.4277	6.37	1512000	2.2821
2.4316	6.4	1520000	2.2856
2.4316	6.43	1528000	2.2820
2.4222	6.47	1536000	2.2891
2.4222	6.5	1544000	2.2803
2.426	6.53	1552000	2.2797
2.426	6.57	1560000	2.2844
2.422	6.6	1568000	2.2872
2.422	6.64	1576000	2.2904
2.4323	6.67	1584000	2.2797
2.4323	6.7	1592000	2.2757
2.4315	6.74	1600000	2.2874
2.4315	6.77	1608000	2.2763
2.421	6.8	1616000	2.2857
2.421	6.84	1624000	2.2804
2.4299	6.87	1632000	2.2825
2.4299	6.91	1640000	2.2819
2.4289	6.94	1648000	2.2824
2.4289	6.97	1656000	2.2821
2.4257	7.01	1664000	2.2802
2.4257	7.04	1672000	2.2760
2.4227	7.07	1680000	2.2810
2.4227	7.11	1688000	2.2777
2.4287	7.14	1696000	2.2772
2.4287	7.17	1704000	2.2786
2.4227	7.21	1712000	2.2859
2.4227	7.24	1720000	2.2862
2.4262	7.28	1728000	2.2789
2.4262	7.31	1736000	2.2848
2.4263	7.34	1744000	2.2754
2.4263	7.38	1752000	2.2778
2.4246	7.41	1760000	2.2735
2.4246	7.44	1768000	2.2827
2.4147	7.48	1776000	2.2850
2.4147	7.51	1784000	2.2821
2.4288	7.55	1792000	2.2803
2.4288	7.58	1800000	2.2760
2.4231	7.61	1808000	2.2749
2.4231	7.65	1816000	2.2749
2.4243	7.68	1824000	2.2743
2.4243	7.71	1832000	2.2792
2.4215	7.75	1840000	2.2752
2.4215	7.78	1848000	2.2770
2.4213	7.81	1856000	2.2802
2.4213	7.85	1864000	2.2796
2.4236	7.88	1872000	2.2883
2.4236	7.92	1880000	2.2792
2.4237	7.95	1888000	2.2726
2.4237	7.98	1896000	2.2816
2.4183	8.02	1904000	2.2790
2.4183	8.05	1912000	2.2815
2.4215	8.08	1920000	2.2774
2.4215	8.12	1928000	2.2700
2.4258	8.15	1936000	2.2763
2.4258	8.19	1944000	2.2786
2.4209	8.22	1952000	2.2763
2.4209	8.25	1960000	2.2789
2.4217	8.29	1968000	2.2784
2.4217	8.32	1976000	2.2773
2.4279	8.35	1984000	2.2861
2.4279	8.39	1992000	2.2728
2.4268	8.42	2000000	2.2762
2.4268	8.45	2008000	2.2789
2.4177	8.49	2016000	2.2822
2.4177	8.52	2024000	2.2759
2.4166	8.56	2032000	2.2792
2.4166	8.59	2040000	2.2721
2.4223	8.62	2048000	2.2768
2.4223	8.66	2056000	2.2726
2.4139	8.69	2064000	2.2825
2.4139	8.72	2072000	2.2739
2.4236	8.76	2080000	2.2834
2.4236	8.79	2088000	2.2750
2.4235	8.83	2096000	2.2752
2.4235	8.86	2104000	2.2803
2.4193	8.89	2112000	2.2763
2.4193	8.93	2120000	2.2755
2.4179	8.96	2128000	2.2794
2.4179	8.99	2136000	2.2711
2.4181	9.03	2144000	2.2792
2.4181	9.06	2152000	2.2752
2.4173	9.09	2160000	2.2775
2.4173	9.13	2168000	2.2752
2.4242	9.16	2176000	2.2729
2.4242	9.2	2184000	2.2793
2.4166	9.23	2192000	2.2719
2.4166	9.26	2200000	2.2820
2.4181	9.3	2208000	2.2716
2.4181	9.33	2216000	2.2855
2.4245	9.36	2224000	2.2805
2.4245	9.4	2232000	2.2721
2.4204	9.43	2240000	2.2707
2.4204	9.47	2248000	2.2767
2.4255	9.5	2256000	2.2710
2.4255	9.53	2264000	2.2814
2.4254	9.57	2272000	2.2746
2.4254	9.6	2280000	2.2766
2.4232	9.63	2288000	2.2725
2.4232	9.67	2296000	2.2765
2.4189	9.7	2304000	2.2756
2.4189	9.73	2312000	2.2768
2.4105	9.77	2320000	2.2804
2.4105	9.8	2328000	2.2873
2.415	9.84	2336000	2.2783
2.415	9.87	2344000	2.2737
2.4174	9.9	2352000	2.2786
2.4174	9.94	2360000	2.2730
2.4199	9.97	2368000	2.2794
2.4199	10.0	2376000	2.2848
2.4224	10.04	2384000	2.2811
2.4224	10.07	2392000	2.2818
2.4226	10.11	2400000	2.2798

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0