GUE_EMP_H4ac-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_EMP_H4ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.8201
• F1 Score: 0.5710
• Accuracy: 0.5718

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6805	14.29	200	0.6853	0.5689	0.5710
0.6395	28.57	400	0.6995	0.5691	0.5686
0.6159	42.86	600	0.7143	0.5664	0.5660
0.5949	57.14	800	0.7225	0.5648	0.5654
0.5785	71.43	1000	0.7283	0.5662	0.5663
0.5681	85.71	1200	0.7326	0.5562	0.5589
0.5597	100.0	1400	0.7360	0.5689	0.5686
0.5547	114.29	1600	0.7403	0.5672	0.5680
0.5494	128.57	1800	0.7393	0.5718	0.5713
0.5446	142.86	2000	0.7412	0.5749	0.5748
0.5397	157.14	2200	0.7314	0.5750	0.5786
0.5375	171.43	2400	0.7367	0.5735	0.5736
0.5325	185.71	2600	0.7544	0.5751	0.5789
0.53	200.0	2800	0.7400	0.5754	0.5771
0.5263	214.29	3000	0.7604	0.5752	0.5754
0.523	228.57	3200	0.7603	0.5775	0.5783
0.5183	242.86	3400	0.7549	0.5794	0.5792
0.5149	257.14	3600	0.7430	0.5734	0.5730
0.5101	271.43	3800	0.7624	0.5749	0.5754
0.5068	285.71	4000	0.7612	0.5754	0.5754
0.5025	300.0	4200	0.7625	0.5775	0.5774
0.4987	314.29	4400	0.7628	0.5760	0.5757
0.4935	328.57	4600	0.7906	0.5749	0.5795
0.4896	342.86	4800	0.7928	0.5793	0.5812
0.4854	357.14	5000	0.7995	0.5792	0.5806
0.4819	371.43	5200	0.7655	0.5741	0.5736
0.4764	385.71	5400	0.8003	0.5749	0.5745
0.473	400.0	5600	0.7854	0.5795	0.5815
0.4686	414.29	5800	0.8072	0.5783	0.5780
0.4643	428.57	6000	0.8164	0.5771	0.5801
0.4638	442.86	6200	0.7924	0.5767	0.5812
0.4582	457.14	6400	0.8014	0.5768	0.5771
0.4539	471.43	6600	0.8059	0.5831	0.5848
0.4509	485.71	6800	0.8146	0.5777	0.5780
0.4479	500.0	7000	0.8200	0.5816	0.5830
0.4431	514.29	7200	0.8061	0.5808	0.5809
0.442	528.57	7400	0.8272	0.5796	0.5801
0.4394	542.86	7600	0.8340	0.5743	0.5745
0.4382	557.14	7800	0.8198	0.5811	0.5812
0.4352	571.43	8000	0.8341	0.5752	0.5748
0.434	585.71	8200	0.8357	0.5783	0.5789
0.4307	600.0	8400	0.8420	0.5789	0.5792
0.4301	614.29	8600	0.8443	0.5775	0.5774
0.4286	628.57	8800	0.8396	0.5797	0.5801
0.427	642.86	9000	0.8509	0.5781	0.5786
0.4256	657.14	9200	0.8464	0.5785	0.5792
0.4259	671.43	9400	0.8405	0.5776	0.5783
0.4237	685.71	9600	0.8473	0.5774	0.5777
0.4231	700.0	9800	0.8457	0.5758	0.5762
0.4243	714.29	10000	0.8451	0.5767	0.5771

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2