scenario-NON-KD-SCR-D2_data-AmazonScience_massive_all_1_1_gamma-jason

This model is a fine-tuned version of xlm-roberta-base on the massive dataset. It achieves the following results on the evaluation set:

• Loss: 1.9069
• Accuracy: 0.8067
• F1: 0.7754

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 333
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.4977	0.27	5000	1.4424	0.6121	0.4883
1.0666	0.53	10000	1.0799	0.7100	0.6376
0.8956	0.8	15000	0.9316	0.7553	0.6882
0.6187	1.07	20000	0.9192	0.7669	0.7139
0.6149	1.34	25000	0.8627	0.7810	0.7321
0.5816	1.6	30000	0.8322	0.7860	0.7348
0.5658	1.87	35000	0.7785	0.7965	0.7507
0.3827	2.14	40000	0.8172	0.8005	0.7622
0.3917	2.41	45000	0.8092	0.8041	0.7669
0.3967	2.67	50000	0.7971	0.8059	0.7698
0.412	2.94	55000	0.8060	0.8060	0.7676
0.2572	3.21	60000	0.9054	0.8044	0.7731
0.2717	3.47	65000	0.8967	0.8041	0.7704
0.2788	3.74	70000	0.8620	0.8081	0.7698
0.2396	4.01	75000	0.9314	0.8108	0.7740
0.1779	4.28	80000	0.9958	0.8078	0.7708
0.1993	4.54	85000	1.0189	0.8017	0.7677
0.2035	4.81	90000	0.9530	0.8098	0.7813
0.1212	5.08	95000	1.1017	0.8065	0.7820
0.1408	5.34	100000	1.0842	0.8046	0.7709
0.148	5.61	105000	1.0908	0.8044	0.7735
0.1627	5.88	110000	1.0639	0.8059	0.7752
0.1025	6.15	115000	1.2316	0.8069	0.7792
0.1115	6.41	120000	1.2271	0.8058	0.7751
0.1184	6.68	125000	1.1966	0.8072	0.7791
0.1254	6.95	130000	1.2027	0.8040	0.7712
0.0945	7.22	135000	1.3307	0.8014	0.7731
0.1011	7.48	140000	1.2414	0.8099	0.7808
0.0979	7.75	145000	1.2843	0.8047	0.7750
0.0805	8.02	150000	1.3527	0.8065	0.7790
0.0796	8.28	155000	1.3635	0.8072	0.7779
0.0847	8.55	160000	1.3899	0.8046	0.7757
0.0888	8.82	165000	1.3633	0.8050	0.7754
0.0584	9.09	170000	1.4016	0.8053	0.7788
0.0666	9.35	175000	1.4972	0.8008	0.7724
0.0708	9.62	180000	1.4603	0.8055	0.7718
0.0816	9.89	185000	1.4165	0.8063	0.7777
0.0557	10.15	190000	1.4906	0.8019	0.7708
0.0619	10.42	195000	1.5149	0.8021	0.7674
0.0674	10.69	200000	1.5291	0.8076	0.7809
0.0657	10.96	205000	1.4904	0.8079	0.7805
0.0421	11.22	210000	1.5688	0.8052	0.7712
0.0478	11.49	215000	1.6017	0.8025	0.7685
0.0589	11.76	220000	1.5721	0.8029	0.7699
0.0448	12.03	225000	1.6069	0.8046	0.7750
0.0506	12.29	230000	1.5687	0.8068	0.7749
0.0464	12.56	235000	1.6193	0.8027	0.7695
0.0503	12.83	240000	1.6064	0.8033	0.7737
0.0345	13.09	245000	1.6466	0.8045	0.7700
0.0423	13.36	250000	1.6533	0.8004	0.7652
0.0398	13.63	255000	1.7020	0.8022	0.7721
0.0521	13.9	260000	1.6553	0.8029	0.7720
0.0377	14.16	265000	1.7035	0.8000	0.7636
0.0405	14.43	270000	1.7346	0.8005	0.7696
0.0409	14.7	275000	1.6876	0.8032	0.7729
0.0376	14.96	280000	1.6858	0.8042	0.7740
0.029	15.23	285000	1.7151	0.8045	0.7756
0.0355	15.5	290000	1.6751	0.8020	0.7747
0.0378	15.77	295000	1.7474	0.8032	0.7700
0.0255	16.03	300000	1.7624	0.8037	0.7725
0.0269	16.3	305000	1.7465	0.8034	0.7742
0.0289	16.57	310000	1.7289	0.8035	0.7772
0.0296	16.84	315000	1.7882	0.8010	0.7695
0.0265	17.1	320000	1.7604	0.8038	0.7733
0.0246	17.37	325000	1.8082	0.8008	0.7682
0.0254	17.64	330000	1.7370	0.8036	0.7718
0.0279	17.9	335000	1.7997	0.8026	0.7734
0.023	18.17	340000	1.7900	0.7988	0.7656
0.0261	18.44	345000	1.7862	0.8032	0.7683
0.0262	18.71	350000	1.8031	0.7999	0.7693
0.0257	18.97	355000	1.8355	0.7991	0.7679
0.0175	19.24	360000	1.8461	0.8015	0.7724
0.0239	19.51	365000	1.8358	0.7995	0.7669
0.0264	19.77	370000	1.8008	0.8021	0.7712
0.0136	20.04	375000	1.8389	0.7994	0.7651
0.0189	20.31	380000	1.8344	0.8006	0.7672
0.019	20.58	385000	1.8654	0.8026	0.7714
0.0228	20.84	390000	1.8569	0.8015	0.7701
0.0134	21.11	395000	1.8587	0.8026	0.7708
0.0166	21.38	400000	1.8480	0.8050	0.7763
0.0191	21.65	405000	1.8835	0.8025	0.7675
0.0188	21.91	410000	1.8918	0.8031	0.7736
0.0162	22.18	415000	1.8598	0.8015	0.7725
0.017	22.45	420000	1.8991	0.8001	0.7644
0.0147	22.71	425000	1.8656	0.7992	0.7690
0.0188	22.98	430000	1.8882	0.8018	0.7687
0.0116	23.25	435000	1.8724	0.8017	0.7677
0.0147	23.52	440000	1.8837	0.8032	0.7721
0.0164	23.78	445000	1.8735	0.8054	0.7724
0.0103	24.05	450000	1.8956	0.8047	0.7731
0.0096	24.32	455000	1.9296	0.8029	0.7677
0.0127	24.58	460000	1.8898	0.8019	0.7725
0.0104	24.85	465000	1.9208	0.8017	0.7679
0.0075	25.12	470000	1.9089	0.8034	0.7706
0.0108	25.39	475000	1.9093	0.8023	0.7664
0.0116	25.65	480000	1.9088	0.8044	0.7722
0.0155	25.92	485000	1.8528	0.8063	0.7754
0.0082	26.19	490000	1.9082	0.8072	0.7739
0.0117	26.46	495000	1.9002	0.8047	0.7727
0.0072	26.72	500000	1.8967	0.8058	0.7735
0.0082	26.99	505000	1.9000	0.8046	0.7747
0.0088	27.26	510000	1.8946	0.8053	0.7744
0.0074	27.52	515000	1.8956	0.8057	0.7717
0.0099	27.79	520000	1.8367	0.8057	0.7725
0.0072	28.06	525000	1.8863	0.8083	0.7768
0.0094	28.33	530000	1.8948	0.8070	0.7754
0.0084	28.59	535000	1.8845	0.8072	0.7756
0.0069	28.86	540000	1.8865	0.8075	0.7751
0.0055	29.13	545000	1.8999	0.8062	0.7735
0.0051	29.39	550000	1.9113	0.8066	0.7751
0.006	29.66	555000	1.9058	0.8066	0.7751
0.0085	29.93	560000	1.9069	0.8067	0.7754

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3