scenario-NON-KD-SCR-D2_data-AmazonScience_massive_all_1_1333

This model is a fine-tuned version of xlm-roberta-base on the massive dataset. It achieves the following results on the evaluation set:

• Loss: 1.9549
• Accuracy: 0.8043
• F1: 0.7729

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 333
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.5003	0.27	5000	1.4803	0.6001	0.4750
1.0696	0.53	10000	1.0629	0.7133	0.6331
0.8993	0.8	15000	0.9337	0.7549	0.6941
0.6223	1.07	20000	0.9072	0.7691	0.7180
0.6122	1.34	25000	0.8510	0.7820	0.7315
0.5894	1.6	30000	0.8077	0.7893	0.7424
0.5692	1.87	35000	0.7765	0.7970	0.7542
0.3814	2.14	40000	0.8363	0.7967	0.7574
0.3888	2.41	45000	0.8421	0.7970	0.7587
0.3886	2.67	50000	0.8302	0.8035	0.7672
0.407	2.94	55000	0.8024	0.8052	0.7692
0.2541	3.21	60000	0.8741	0.8075	0.7743
0.2687	3.47	65000	0.8732	0.8077	0.7736
0.2857	3.74	70000	0.8756	0.8060	0.7696
0.2347	4.01	75000	0.9543	0.8079	0.7698
0.1788	4.28	80000	1.0152	0.8046	0.7722
0.1896	4.54	85000	0.9846	0.8074	0.7737
0.207	4.81	90000	0.9621	0.8066	0.7709
0.1175	5.08	95000	1.1112	0.8069	0.7770
0.1389	5.34	100000	1.0817	0.8085	0.7697
0.145	5.61	105000	1.0714	0.8048	0.7755
0.1591	5.88	110000	1.0711	0.8051	0.7788
0.1008	6.15	115000	1.2102	0.8086	0.7794
0.1145	6.41	120000	1.2193	0.8066	0.7680
0.1268	6.68	125000	1.1596	0.8066	0.7787
0.1238	6.95	130000	1.1909	0.8084	0.7752
0.0894	7.22	135000	1.3205	0.8054	0.7792
0.0983	7.48	140000	1.3175	0.8085	0.7742
0.1056	7.75	145000	1.3009	0.8034	0.7725
0.0796	8.02	150000	1.3048	0.8076	0.7797
0.079	8.28	155000	1.3814	0.8031	0.7787
0.088	8.55	160000	1.3723	0.8077	0.7798
0.0936	8.82	165000	1.3706	0.8052	0.7770
0.0558	9.09	170000	1.4600	0.8057	0.7741
0.0655	9.35	175000	1.4752	0.8023	0.7737
0.0752	9.62	180000	1.4717	0.8032	0.7726
0.0799	9.89	185000	1.4064	0.8071	0.7783
0.0536	10.15	190000	1.5280	0.8029	0.7703
0.0596	10.42	195000	1.5051	0.8045	0.7772
0.0605	10.69	200000	1.5007	0.8040	0.7771
0.0723	10.96	205000	1.5009	0.8053	0.7776
0.0424	11.22	210000	1.6065	0.7998	0.7671
0.0587	11.49	215000	1.5795	0.8014	0.7664
0.06	11.76	220000	1.6210	0.7959	0.7637
0.0444	12.03	225000	1.6069	0.8042	0.7778
0.0427	12.29	230000	1.5564	0.8031	0.7745
0.0462	12.56	235000	1.6148	0.8007	0.7716
0.0507	12.83	240000	1.6110	0.8022	0.7715
0.0398	13.09	245000	1.6613	0.8036	0.7728
0.0411	13.36	250000	1.6634	0.8047	0.7782
0.0462	13.63	255000	1.6522	0.8046	0.7756
0.0489	13.9	260000	1.6642	0.8015	0.7738
0.0395	14.16	265000	1.6743	0.8019	0.7683
0.0407	14.43	270000	1.6974	0.8036	0.7713
0.0435	14.7	275000	1.6568	0.8038	0.7725
0.0383	14.96	280000	1.6867	0.8044	0.7736
0.0331	15.23	285000	1.7553	0.8015	0.7720
0.0355	15.5	290000	1.7463	0.7982	0.7667
0.0309	15.77	295000	1.7347	0.8017	0.7735
0.0205	16.03	300000	1.7513	0.8028	0.7714
0.0322	16.3	305000	1.7507	0.8013	0.7726
0.031	16.57	310000	1.7373	0.8043	0.7724
0.0352	16.84	315000	1.7256	0.8022	0.7706
0.0246	17.1	320000	1.7548	0.8036	0.7712
0.0276	17.37	325000	1.8002	0.7984	0.7686
0.0259	17.64	330000	1.7736	0.8011	0.7701
0.0218	17.9	335000	1.8022	0.7996	0.7718
0.0228	18.17	340000	1.8162	0.8021	0.7671
0.0202	18.44	345000	1.8392	0.8011	0.7678
0.0251	18.71	350000	1.7928	0.7997	0.7673
0.0263	18.97	355000	1.8359	0.8000	0.7671
0.0202	19.24	360000	1.8644	0.8021	0.7739
0.0215	19.51	365000	1.8412	0.7988	0.7685
0.0277	19.77	370000	1.7734	0.8009	0.7685
0.018	20.04	375000	1.8197	0.7997	0.7716
0.0188	20.31	380000	1.8411	0.8017	0.7703
0.0233	20.58	385000	1.8631	0.7993	0.7658
0.0216	20.84	390000	1.8590	0.8010	0.7676
0.0159	21.11	395000	1.8778	0.7977	0.7639
0.0191	21.38	400000	1.8380	0.8028	0.7732
0.0173	21.65	405000	1.8699	0.8027	0.7754
0.0216	21.91	410000	1.9152	0.7983	0.7651
0.0139	22.18	415000	1.8876	0.7987	0.7680
0.0172	22.45	420000	1.8977	0.7996	0.7656
0.0135	22.71	425000	1.8767	0.7996	0.7687
0.016	22.98	430000	1.8973	0.8015	0.7689
0.0135	23.25	435000	1.9112	0.8007	0.7668
0.0162	23.52	440000	1.9312	0.7994	0.7695
0.0178	23.78	445000	1.8928	0.8012	0.7679
0.0089	24.05	450000	1.9162	0.8012	0.7643
0.0123	24.32	455000	1.9334	0.8006	0.7671
0.0137	24.58	460000	1.8863	0.8018	0.7683
0.01	24.85	465000	1.9100	0.8012	0.7667
0.008	25.12	470000	1.9313	0.8011	0.7687
0.0113	25.39	475000	1.9219	0.8025	0.7718
0.0129	25.65	480000	1.9310	0.8015	0.7705
0.0165	25.92	485000	1.9018	0.8033	0.7736
0.0085	26.19	490000	1.9537	0.8035	0.7705
0.0108	26.46	495000	1.9116	0.8019	0.7701
0.0086	26.72	500000	1.9266	0.8034	0.7720
0.0094	26.99	505000	1.9232	0.8036	0.7721
0.0083	27.26	510000	1.9425	0.8027	0.7728
0.0078	27.52	515000	1.9477	0.8044	0.7727
0.0095	27.79	520000	1.9127	0.8031	0.7687
0.0067	28.06	525000	1.9433	0.8030	0.7692
0.009	28.33	530000	1.9325	0.8042	0.7719
0.0074	28.59	535000	1.9332	0.8048	0.7740
0.0071	28.86	540000	1.9393	0.8049	0.7734
0.006	29.13	545000	1.9489	0.8039	0.7719
0.0061	29.39	550000	1.9571	0.8041	0.7716
0.0065	29.66	555000	1.9528	0.8046	0.7728
0.0084	29.93	560000	1.9549	0.8043	0.7729

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3