GUE_virus_covid-seqsight_65536_512_94M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_94M on the mahdibaghbanzadeh/GUE_virus_covid dataset. It achieves the following results on the evaluation set:

• Loss: 1.1209
• F1 Score: 0.5718
• Accuracy: 0.5761

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
2.1819	0.35	200	2.1755	0.1029	0.1467
2.1677	0.7	400	2.1525	0.1504	0.1719
2.1037	1.05	600	1.9905	0.2502	0.2532
1.9386	1.4	800	1.8186	0.3074	0.3119
1.8048	1.75	1000	1.7022	0.3385	0.3515
1.734	2.09	1200	1.6465	0.3576	0.3661
1.6632	2.44	1400	1.5601	0.4099	0.4053
1.6174	2.79	1600	1.5219	0.4037	0.4100
1.5773	3.14	1800	1.4875	0.4190	0.4308
1.5454	3.49	2000	1.4510	0.4448	0.4375
1.5272	3.84	2200	1.4365	0.4488	0.4473
1.492	4.19	2400	1.4061	0.4640	0.4566
1.4653	4.54	2600	1.3640	0.4748	0.4801
1.4521	4.89	2800	1.3456	0.4885	0.4905
1.4251	5.24	3000	1.3373	0.4907	0.4887
1.4218	5.58	3200	1.3264	0.4925	0.4948
1.3943	5.93	3400	1.3019	0.5027	0.5056
1.3865	6.28	3600	1.2961	0.5126	0.5048
1.3631	6.63	3800	1.2763	0.5137	0.5183
1.3598	6.98	4000	1.2698	0.5284	0.5300
1.3456	7.33	4200	1.2596	0.5220	0.5274
1.3319	7.68	4400	1.2486	0.5356	0.5326
1.3299	8.03	4600	1.2338	0.5419	0.5405
1.3123	8.38	4800	1.2289	0.5317	0.5358
1.3115	8.73	5000	1.2201	0.5315	0.5357
1.2927	9.08	5200	1.2126	0.5536	0.5514
1.2886	9.42	5400	1.2008	0.5548	0.5561
1.2679	9.77	5600	1.1877	0.5533	0.5552
1.2716	10.12	5800	1.1765	0.5546	0.5584
1.2708	10.47	6000	1.1744	0.5581	0.5589
1.2533	10.82	6200	1.1765	0.5658	0.5624
1.2398	11.17	6400	1.1763	0.5618	0.5586
1.2432	11.52	6600	1.1606	0.5637	0.5627
1.2456	11.87	6800	1.1618	0.5618	0.5623
1.2334	12.22	7000	1.1554	0.5649	0.5674
1.2211	12.57	7200	1.1455	0.5668	0.5699
1.2312	12.91	7400	1.1470	0.5707	0.5721
1.2275	13.26	7600	1.1453	0.5711	0.5720
1.2151	13.61	7800	1.1451	0.5727	0.5696
1.2197	13.96	8000	1.1406	0.5724	0.5728
1.21	14.31	8200	1.1402	0.5698	0.5709
1.2119	14.66	8400	1.1310	0.5724	0.5736
1.2067	15.01	8600	1.1371	0.5746	0.5752
1.2151	15.36	8800	1.1322	0.5758	0.5783
1.1961	15.71	9000	1.1317	0.5731	0.5750
1.207	16.06	9200	1.1302	0.5749	0.5760
1.2039	16.4	9400	1.1299	0.5742	0.5760
1.1971	16.75	9600	1.1310	0.5751	0.5767
1.2029	17.1	9800	1.1299	0.5745	0.5767
1.1994	17.45	10000	1.1293	0.5733	0.5756

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2