GUE_virus_covid-seqsight_65536_512_94M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_94M on the mahdibaghbanzadeh/GUE_virus_covid dataset. It achieves the following results on the evaluation set:

• Loss: 1.5140
• F1 Score: 0.4269
• Accuracy: 0.4271

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
2.1826	0.35	200	2.1788	0.0988	0.1432
2.1742	0.7	400	2.1698	0.1151	0.1482
2.1629	1.05	600	2.1590	0.1381	0.1610
2.1492	1.4	800	2.1383	0.1493	0.1802
2.119	1.75	1000	2.0626	0.2013	0.2234
2.0594	2.09	1200	2.0274	0.2056	0.2305
2.0141	2.44	1400	1.9740	0.2361	0.2531
1.9764	2.79	1600	1.9216	0.2621	0.2719
1.9353	3.14	1800	1.8812	0.2712	0.2905
1.8921	3.49	2000	1.8299	0.2973	0.3091
1.8607	3.84	2200	1.8001	0.3058	0.3133
1.8253	4.19	2400	1.7755	0.3260	0.3285
1.7988	4.54	2600	1.7376	0.3448	0.3494
1.7907	4.89	2800	1.7195	0.3453	0.3497
1.7635	5.24	3000	1.6938	0.3506	0.3585
1.7487	5.58	3200	1.6894	0.3511	0.3547
1.7341	5.93	3400	1.6865	0.3426	0.3540
1.7229	6.28	3600	1.6569	0.3603	0.3645
1.6987	6.63	3800	1.6428	0.3582	0.3622
1.6999	6.98	4000	1.6326	0.3793	0.3779
1.6869	7.33	4200	1.6264	0.3687	0.3677
1.6839	7.68	4400	1.6202	0.3783	0.3728
1.6699	8.03	4600	1.6138	0.3734	0.3777
1.669	8.38	4800	1.6159	0.3784	0.3792
1.6692	8.73	5000	1.6031	0.3801	0.3866
1.6569	9.08	5200	1.5977	0.3859	0.3835
1.6524	9.42	5400	1.5764	0.3998	0.4019
1.643	9.77	5600	1.5814	0.3957	0.3956
1.6399	10.12	5800	1.5682	0.4044	0.3984
1.6386	10.47	6000	1.5535	0.4127	0.4122
1.6245	10.82	6200	1.5559	0.4152	0.4157
1.6184	11.17	6400	1.5645	0.4011	0.3996
1.6184	11.52	6600	1.5460	0.4148	0.4106
1.6249	11.87	6800	1.5414	0.4130	0.4123
1.617	12.22	7000	1.5475	0.4053	0.4066
1.6072	12.57	7200	1.5445	0.4078	0.4072
1.6128	12.91	7400	1.5402	0.4125	0.4123
1.6082	13.26	7600	1.5340	0.4115	0.4151
1.6017	13.61	7800	1.5252	0.4230	0.4240
1.601	13.96	8000	1.5265	0.4217	0.4234
1.5928	14.31	8200	1.5236	0.4182	0.4175
1.5985	14.66	8400	1.5153	0.4286	0.4273
1.5978	15.01	8600	1.5189	0.4193	0.4212
1.5966	15.36	8800	1.5214	0.4261	0.4248
1.5812	15.71	9000	1.5145	0.4238	0.4261
1.5875	16.06	9200	1.5150	0.4259	0.4252
1.5885	16.4	9400	1.5133	0.4212	0.4224
1.583	16.75	9600	1.5118	0.4242	0.4261
1.5913	17.1	9800	1.5117	0.4256	0.4272
1.595	17.45	10000	1.5124	0.4254	0.4268

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2