GUE_virus_covid-seqsight_65536_512_94M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_94M on the mahdibaghbanzadeh/GUE_virus_covid dataset. It achieves the following results on the evaluation set:

• Loss: 0.9368
• F1 Score: 0.6476
• Accuracy: 0.6438

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
2.1813	0.35	200	2.1726	0.1093	0.1488
2.127	0.7	400	1.9895	0.2430	0.2703
1.9165	1.05	600	1.7929	0.3068	0.3167
1.7616	1.4	800	1.6426	0.3641	0.3685
1.6482	1.75	1000	1.5434	0.4014	0.4087
1.5701	2.09	1200	1.4785	0.4366	0.4374
1.4971	2.44	1400	1.3871	0.4788	0.4761
1.4481	2.79	1600	1.3357	0.4813	0.4947
1.3898	3.14	1800	1.2972	0.5103	0.5140
1.3527	3.49	2000	1.2700	0.5254	0.5179
1.3399	3.84	2200	1.2412	0.5270	0.5295
1.2984	4.19	2400	1.2127	0.5502	0.5428
1.27	4.54	2600	1.1879	0.5506	0.5535
1.2677	4.89	2800	1.1645	0.5575	0.5592
1.2329	5.24	3000	1.1460	0.5790	0.5786
1.2198	5.58	3200	1.1306	0.5773	0.5740
1.1959	5.93	3400	1.1074	0.5875	0.5896
1.1788	6.28	3600	1.1118	0.5952	0.5780
1.1566	6.63	3800	1.0782	0.6034	0.5984
1.1476	6.98	4000	1.0731	0.6090	0.6045
1.1238	7.33	4200	1.0448	0.6078	0.6054
1.1075	7.68	4400	1.0331	0.6190	0.6118
1.104	8.03	4600	1.0223	0.6222	0.6185
1.0826	8.38	4800	1.0180	0.6161	0.6139
1.0836	8.73	5000	1.0036	0.6230	0.6178
1.069	9.08	5200	1.0049	0.6247	0.6174
1.0635	9.42	5400	0.9956	0.6218	0.6211
1.051	9.77	5600	0.9843	0.6297	0.6276
1.0535	10.12	5800	0.9776	0.6309	0.6263
1.0471	10.47	6000	0.9832	0.6281	0.6210
1.0343	10.82	6200	0.9793	0.6395	0.6322
1.0117	11.17	6400	0.9803	0.6360	0.6308
1.0206	11.52	6600	0.9661	0.6339	0.6286
1.033	11.87	6800	0.9714	0.6357	0.6315
1.0207	12.22	7000	0.9679	0.6394	0.6355
1.0031	12.57	7200	0.9640	0.6365	0.6338
1.0094	12.91	7400	0.9617	0.6390	0.6363
1.0074	13.26	7600	0.9603	0.6417	0.6363
1.0034	13.61	7800	0.9554	0.6414	0.6329
0.9972	13.96	8000	0.9469	0.6477	0.6447
0.9904	14.31	8200	0.9471	0.6453	0.6406
0.9918	14.66	8400	0.9428	0.6446	0.6422
0.9922	15.01	8600	0.9432	0.6454	0.6418
0.9911	15.36	8800	0.9426	0.6495	0.6425
0.9812	15.71	9000	0.9380	0.6477	0.6440
0.9877	16.06	9200	0.9370	0.6494	0.6444
0.9799	16.4	9400	0.9354	0.6488	0.6450
0.978	16.75	9600	0.9368	0.6498	0.6442
0.9868	17.1	9800	0.9354	0.6489	0.6456
0.98	17.45	10000	0.9357	0.6476	0.6437

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2