scenario-NON-KD-SCR-D2_data-AmazonScience_massive_all_1_1_c

This model is a fine-tuned version of xlm-roberta-base on the massive dataset. It achieves the following results on the evaluation set:

• Loss: 1.9121
• Accuracy: 0.8047
• F1: 0.7716

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 12314
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.5487	0.27	5000	1.5563	0.5831	0.4524
1.0791	0.53	10000	1.1113	0.7054	0.6276
0.8995	0.8	15000	0.9537	0.7459	0.6746
0.6338	1.07	20000	0.9092	0.7688	0.7152
0.6358	1.34	25000	0.8358	0.7808	0.7288
0.5907	1.6	30000	0.8245	0.7874	0.7415
0.5691	1.87	35000	0.7885	0.7961	0.7539
0.3711	2.14	40000	0.8669	0.7932	0.7534
0.397	2.41	45000	0.8034	0.8056	0.7655
0.3954	2.67	50000	0.8340	0.7997	0.7603
0.3883	2.94	55000	0.8063	0.8036	0.7682
0.241	3.21	60000	0.9218	0.8009	0.7639
0.2533	3.47	65000	0.9109	0.8060	0.7747
0.2734	3.74	70000	0.8859	0.8056	0.7726
0.2514	4.01	75000	0.9639	0.8073	0.7680
0.1712	4.28	80000	1.0070	0.8062	0.7752
0.1847	4.54	85000	1.0244	0.8051	0.7782
0.202	4.81	90000	0.9732	0.8076	0.7756
0.1237	5.08	95000	1.1069	0.8057	0.7788
0.1397	5.34	100000	1.1061	0.8050	0.7764
0.1467	5.61	105000	1.1381	0.8058	0.7786
0.1618	5.88	110000	1.0749	0.8063	0.7796
0.1026	6.15	115000	1.1894	0.8068	0.7797
0.1131	6.41	120000	1.2012	0.8075	0.7824
0.1216	6.68	125000	1.1979	0.8080	0.7783
0.1307	6.95	130000	1.2131	0.8072	0.7790
0.0872	7.22	135000	1.3463	0.8073	0.7851
0.0996	7.48	140000	1.2800	0.8066	0.7779
0.0995	7.75	145000	1.3234	0.8048	0.7770
0.0776	8.02	150000	1.3706	0.8048	0.7781
0.0725	8.28	155000	1.4063	0.8036	0.7752
0.087	8.55	160000	1.4033	0.8042	0.7800
0.0846	8.82	165000	1.4005	0.8079	0.7794
0.0578	9.09	170000	1.4568	0.8057	0.7747
0.0706	9.35	175000	1.4622	0.8033	0.7775
0.0707	9.62	180000	1.4854	0.8046	0.7758
0.075	9.89	185000	1.4571	0.8042	0.7780
0.0547	10.15	190000	1.5205	0.8037	0.7756
0.0555	10.42	195000	1.5116	0.8063	0.7815
0.0587	10.69	200000	1.5209	0.8048	0.7740
0.0738	10.96	205000	1.4777	0.8040	0.7785
0.0513	11.22	210000	1.5682	0.8029	0.7762
0.0553	11.49	215000	1.5381	0.8033	0.7741
0.0587	11.76	220000	1.5377	0.8067	0.7789
0.0397	12.03	225000	1.5889	0.7996	0.7683
0.0467	12.29	230000	1.5984	0.8062	0.7761
0.0449	12.56	235000	1.6123	0.8040	0.7716
0.0507	12.83	240000	1.6441	0.8016	0.7701
0.0375	13.09	245000	1.6379	0.8014	0.7695
0.032	13.36	250000	1.6873	0.8036	0.7735
0.0403	13.63	255000	1.6864	0.8036	0.7738
0.0436	13.9	260000	1.6326	0.8007	0.7704
0.0307	14.16	265000	1.6662	0.8044	0.7736
0.0382	14.43	270000	1.6827	0.8029	0.7719
0.0433	14.7	275000	1.7244	0.8025	0.7757
0.0393	14.96	280000	1.6844	0.8010	0.7671
0.0359	15.23	285000	1.6920	0.8012	0.7708
0.034	15.5	290000	1.7410	0.8029	0.7715
0.0376	15.77	295000	1.7083	0.7983	0.7693
0.026	16.03	300000	1.7172	0.8041	0.7733
0.034	16.3	305000	1.7877	0.7995	0.7699
0.0306	16.57	310000	1.7450	0.8039	0.7741
0.0382	16.84	315000	1.7176	0.8015	0.7703
0.0208	17.1	320000	1.7829	0.8004	0.7714
0.0278	17.37	325000	1.7812	0.8032	0.7747
0.0259	17.64	330000	1.7818	0.7991	0.7655
0.0281	17.9	335000	1.7525	0.7992	0.7663
0.0177	18.17	340000	1.7905	0.7984	0.7677
0.0251	18.44	345000	1.8388	0.8004	0.7730
0.0232	18.71	350000	1.8065	0.7986	0.7671
0.0251	18.97	355000	1.8058	0.8021	0.7720
0.0226	19.24	360000	1.8176	0.8016	0.7742
0.0254	19.51	365000	1.8424	0.7988	0.7650
0.0205	19.77	370000	1.8720	0.7988	0.7705
0.0136	20.04	375000	1.8504	0.7990	0.7679
0.0173	20.31	380000	1.8601	0.8029	0.7729
0.0151	20.58	385000	1.8526	0.8006	0.7691
0.0231	20.84	390000	1.8459	0.8011	0.7697
0.0175	21.11	395000	1.8371	0.7996	0.7654
0.0217	21.38	400000	1.8381	0.7997	0.7681
0.0192	21.65	405000	1.8748	0.7982	0.7657
0.0206	21.91	410000	1.8194	0.8010	0.7695
0.0114	22.18	415000	1.8965	0.8013	0.7699
0.0166	22.45	420000	1.8760	0.7999	0.7714
0.0174	22.71	425000	1.8612	0.8047	0.7698
0.014	22.98	430000	1.8984	0.8027	0.7714
0.0113	23.25	435000	1.8922	0.7999	0.7680
0.0144	23.52	440000	1.9121	0.7996	0.7682
0.0126	23.78	445000	1.8775	0.8011	0.7680
0.0107	24.05	450000	1.9192	0.8012	0.7706
0.0118	24.32	455000	1.9090	0.8021	0.7673
0.0131	24.58	460000	1.9099	0.8010	0.7665
0.0202	24.85	465000	1.9177	0.7989	0.7653
0.0101	25.12	470000	1.8774	0.8038	0.7716
0.0116	25.39	475000	1.8925	0.8012	0.7694
0.0141	25.65	480000	1.8787	0.8033	0.7718
0.0109	25.92	485000	1.9213	0.8016	0.7673
0.0101	26.19	490000	1.9285	0.8025	0.7702
0.011	26.46	495000	1.9084	0.8028	0.7675
0.0094	26.72	500000	1.8996	0.8033	0.7708
0.0111	26.99	505000	1.8915	0.8049	0.7720
0.0074	27.26	510000	1.9096	0.8038	0.7687
0.0104	27.52	515000	1.8931	0.8039	0.7724
0.01	27.79	520000	1.8974	0.8032	0.7692
0.0078	28.06	525000	1.9191	0.8043	0.7731
0.0078	28.33	530000	1.9150	0.8044	0.7727
0.0084	28.59	535000	1.9034	0.8038	0.7709
0.0067	28.86	540000	1.8946	0.8049	0.7717
0.0054	29.13	545000	1.9101	0.8046	0.7720
0.0067	29.39	550000	1.9201	0.8052	0.7733
0.0075	29.66	555000	1.9159	0.8048	0.7718
0.0075	29.93	560000	1.9121	0.8047	0.7716

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3