scenario-NON-KD-PO-COPY-D2_data-AmazonScience_massive_all_1_1_betta-jason

This model is a fine-tuned version of haryoaw/scenario-TCR-data-AmazonScience-massive-all_1.1-model-xlm-roberta-base on the massive dataset. It achieves the following results on the evaluation set:

• Loss: 1.4801
• Accuracy: 0.8370
• F1: 0.8111

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 222
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.3665	0.27	5000	1.2360	0.6634	0.5275
0.9916	0.53	10000	0.9671	0.7381	0.6405
0.8313	0.8	15000	0.8562	0.7710	0.6917
0.6696	1.07	20000	0.8081	0.7872	0.7324
0.6193	1.34	25000	0.7671	0.7975	0.7450
0.5688	1.6	30000	0.7323	0.8079	0.7597
0.5479	1.87	35000	0.7190	0.8117	0.7700
0.4385	2.14	40000	0.7314	0.8163	0.7753
0.4299	2.41	45000	0.7425	0.8143	0.7733
0.413	2.67	50000	0.7358	0.8196	0.7822
0.4052	2.94	55000	0.7085	0.8236	0.7889
0.3252	3.21	60000	0.7465	0.8233	0.7858
0.3302	3.47	65000	0.7548	0.8228	0.7849
0.3219	3.74	70000	0.7672	0.8223	0.7858
0.2917	4.01	75000	0.7561	0.8281	0.7928
0.2428	4.28	80000	0.7956	0.8266	0.7919
0.2623	4.54	85000	0.7942	0.8265	0.7989
0.2682	4.81	90000	0.7655	0.8281	0.7936
0.192	5.08	95000	0.8214	0.8263	0.7922
0.2032	5.34	100000	0.8261	0.8293	0.8013
0.2061	5.61	105000	0.8237	0.8298	0.8008
0.224	5.88	110000	0.8159	0.8271	0.8002
0.1545	6.15	115000	0.8678	0.8286	0.8009
0.1849	6.41	120000	0.8773	0.8285	0.7992
0.1838	6.68	125000	0.8788	0.8293	0.8061
0.1902	6.95	130000	0.8582	0.8288	0.8027
0.1363	7.22	135000	0.9172	0.8313	0.8029
0.1482	7.48	140000	0.9271	0.8280	0.8042
0.148	7.75	145000	0.9208	0.8310	0.8079
0.1268	8.02	150000	0.9440	0.8293	0.8010
0.1201	8.28	155000	0.9771	0.8286	0.8024
0.1297	8.55	160000	0.9942	0.8312	0.8047
0.1202	8.82	165000	0.9894	0.8317	0.8067
0.0999	9.09	170000	1.0048	0.8319	0.8054
0.1024	9.35	175000	1.0417	0.8294	0.8052
0.1072	9.62	180000	1.0356	0.8311	0.8072
0.1103	9.89	185000	1.0453	0.8311	0.8068
0.0794	10.15	190000	1.0755	0.8296	0.8040
0.0898	10.42	195000	1.1006	0.8285	0.8041
0.1014	10.69	200000	1.0702	0.8324	0.8070
0.0973	10.96	205000	1.0824	0.8321	0.8055
0.0748	11.22	210000	1.1196	0.8306	0.8043
0.0754	11.49	215000	1.1264	0.8311	0.8019
0.08	11.76	220000	1.1238	0.8322	0.8072
0.0589	12.03	225000	1.1340	0.8319	0.8046
0.0689	12.29	230000	1.1760	0.8299	0.8039
0.073	12.56	235000	1.1680	0.8316	0.8066
0.0771	12.83	240000	1.1842	0.8324	0.8069
0.0499	13.09	245000	1.1968	0.8312	0.8042
0.0647	13.36	250000	1.2064	0.8305	0.8047
0.057	13.63	255000	1.2182	0.8343	0.8077
0.0652	13.9	260000	1.2107	0.8304	0.8064
0.0517	14.16	265000	1.2469	0.8326	0.8055
0.0641	14.43	270000	1.2504	0.8325	0.8062
0.0548	14.7	275000	1.2677	0.8328	0.8036
0.059	14.96	280000	1.2540	0.8332	0.8067
0.0459	15.23	285000	1.2857	0.8329	0.8068
0.0428	15.5	290000	1.2958	0.8308	0.8044
0.0496	15.77	295000	1.3194	0.8303	0.8045
0.0404	16.03	300000	1.2971	0.8329	0.8060
0.0401	16.3	305000	1.3161	0.8328	0.8071
0.0496	16.57	310000	1.3313	0.8321	0.8021
0.0396	16.84	315000	1.3337	0.8324	0.8043
0.0358	17.1	320000	1.3431	0.8323	0.8050
0.038	17.37	325000	1.3318	0.8343	0.8045
0.0396	17.64	330000	1.3302	0.8336	0.8078
0.0368	17.9	335000	1.3578	0.8322	0.8062
0.0314	18.17	340000	1.3658	0.8331	0.8071
0.0293	18.44	345000	1.3867	0.8317	0.8048
0.0388	18.71	350000	1.3902	0.8319	0.8087
0.0377	18.97	355000	1.3737	0.8333	0.8070
0.0325	19.24	360000	1.3954	0.8323	0.8062
0.0309	19.51	365000	1.3987	0.8327	0.8044
0.0286	19.77	370000	1.4183	0.8318	0.8054
0.0238	20.04	375000	1.4039	0.8334	0.8083
0.027	20.31	380000	1.4045	0.8336	0.8081
0.0253	20.58	385000	1.4079	0.8342	0.8085
0.0302	20.84	390000	1.4127	0.8336	0.8071
0.0242	21.11	395000	1.4179	0.8337	0.8074
0.028	21.38	400000	1.4142	0.8342	0.8073
0.0236	21.65	405000	1.4268	0.8348	0.8075
0.0272	21.91	410000	1.4186	0.8341	0.8067
0.0184	22.18	415000	1.4355	0.8345	0.8080
0.0236	22.45	420000	1.4347	0.8353	0.8105
0.0247	22.71	425000	1.4383	0.8346	0.8093
0.0229	22.98	430000	1.4541	0.8352	0.8090
0.0217	23.25	435000	1.4548	0.8346	0.8059
0.0224	23.52	440000	1.4683	0.8330	0.8054
0.0221	23.78	445000	1.4612	0.8347	0.8080
0.0179	24.05	450000	1.4599	0.8342	0.8074
0.0179	24.32	455000	1.4621	0.8342	0.8059
0.0195	24.58	460000	1.4714	0.8342	0.8052
0.0202	24.85	465000	1.4715	0.8346	0.8069
0.0199	25.12	470000	1.4595	0.8350	0.8080
0.0151	25.39	475000	1.4613	0.8366	0.8108
0.0159	25.65	480000	1.4724	0.8349	0.8084
0.0232	25.92	485000	1.4785	0.8362	0.8092
0.0152	26.19	490000	1.4812	0.8365	0.8111
0.0174	26.46	495000	1.4819	0.8353	0.8078
0.018	26.72	500000	1.4764	0.8354	0.8095
0.0153	26.99	505000	1.4682	0.8358	0.8095
0.0153	27.26	510000	1.4840	0.8355	0.8093
0.0119	27.52	515000	1.4922	0.8350	0.8091
0.0128	27.79	520000	1.4847	0.8359	0.8098
0.0146	28.06	525000	1.4831	0.8362	0.8095
0.0175	28.33	530000	1.4859	0.8367	0.8105
0.0136	28.59	535000	1.4812	0.8365	0.8101
0.0122	28.86	540000	1.4789	0.8365	0.8103
0.0116	29.13	545000	1.4800	0.8372	0.8110
0.0108	29.39	550000	1.4807	0.8369	0.8109
0.013	29.66	555000	1.4793	0.8370	0.8110
0.0114	29.93	560000	1.4801	0.8370	0.8111

Framework versions

• Transformers 4.33.3
• Pytorch 2.1.1+cu121
• Datasets 2.14.5
• Tokenizers 0.13.3