GUE_virus_covid-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_virus_covid dataset. It achieves the following results on the evaluation set:

• Loss: 0.8911
• F1 Score: 0.6614
• Accuracy: 0.6627

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
2.1502	5.56	200	1.9532	0.2539	0.2794
1.7911	11.11	400	1.5566	0.4198	0.4216
1.577	16.67	600	1.3911	0.4866	0.4820
1.4645	22.22	800	1.3114	0.5226	0.5159
1.3956	27.78	1000	1.2558	0.5337	0.5296
1.3406	33.33	1200	1.2095	0.5505	0.5470
1.3009	38.89	1400	1.1875	0.5572	0.5532
1.27	44.44	1600	1.1603	0.5650	0.5633
1.2405	50.0	1800	1.1324	0.5758	0.5770
1.2126	55.56	2000	1.1088	0.5902	0.5898
1.1864	61.11	2200	1.0813	0.5981	0.5990
1.1546	66.67	2400	1.0571	0.6077	0.6058
1.1303	72.22	2600	1.0436	0.6140	0.6124
1.1097	77.78	2800	1.0236	0.6233	0.6218
1.0861	83.33	3000	1.0071	0.6278	0.6251
1.0663	88.89	3200	0.9930	0.6346	0.6321
1.0468	94.44	3400	0.9837	0.6383	0.6353
1.0312	100.0	3600	0.9733	0.6379	0.6363
1.014	105.56	3800	0.9593	0.6442	0.6408
1.0005	111.11	4000	0.9502	0.6450	0.6444
0.9896	116.67	4200	0.9446	0.6461	0.6442
0.9794	122.22	4400	0.9395	0.6483	0.6471
0.9704	127.78	4600	0.9294	0.6521	0.6506
0.9614	133.33	4800	0.9301	0.6520	0.6514
0.9544	138.89	5000	0.9255	0.6534	0.6520
0.9478	144.44	5200	0.9251	0.6539	0.6537
0.9407	150.0	5400	0.9191	0.6544	0.6532
0.9353	155.56	5600	0.9162	0.6570	0.6559
0.9304	161.11	5800	0.9141	0.6588	0.6575
0.9254	166.67	6000	0.9104	0.6605	0.6597
0.9214	172.22	6200	0.9093	0.6612	0.6600
0.9178	177.78	6400	0.9099	0.6612	0.6606
0.9108	183.33	6600	0.9074	0.6610	0.6604
0.9092	188.89	6800	0.9057	0.6650	0.6644
0.9037	194.44	7000	0.9055	0.6633	0.6628
0.9021	200.0	7200	0.9023	0.6660	0.6653
0.8966	205.56	7400	0.8984	0.6672	0.6666
0.8946	211.11	7600	0.8970	0.6630	0.6634
0.8907	216.67	7800	0.8968	0.6666	0.6665
0.8878	222.22	8000	0.8948	0.6670	0.6666
0.8846	227.78	8200	0.8934	0.6652	0.6650
0.882	233.33	8400	0.8934	0.6676	0.6677
0.8814	238.89	8600	0.8919	0.6666	0.6665
0.8799	244.44	8800	0.8908	0.6679	0.6677
0.8765	250.0	9000	0.8911	0.6670	0.6672
0.8765	255.56	9200	0.8907	0.6664	0.6667
0.875	261.11	9400	0.8906	0.6675	0.6675
0.8743	266.67	9600	0.8909	0.6680	0.6679
0.8736	272.22	9800	0.8907	0.6671	0.6671
0.8731	277.78	10000	0.8906	0.6676	0.6676

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2