GUE_EMP_H3K14ac-seqsight_4096_512_46M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_46M on the mahdibaghbanzadeh/GUE_EMP_H3K14ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.4760
• F1 Score: 0.7749
• Accuracy: 0.7746

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5506	0.97	200	0.5042	0.7665	0.7649
0.5061	1.93	400	0.4880	0.7721	0.7707
0.4929	2.9	600	0.5016	0.7645	0.7631
0.4883	3.86	800	0.4736	0.7791	0.7785
0.4837	4.83	1000	0.5029	0.7614	0.7604
0.4758	5.8	1200	0.4965	0.7623	0.7610
0.4725	6.76	1400	0.4706	0.7836	0.7821
0.4665	7.73	1600	0.4736	0.7857	0.7843
0.4634	8.7	1800	0.4804	0.7809	0.7794
0.4562	9.66	2000	0.4784	0.7785	0.7770
0.4592	10.63	2200	0.4830	0.7806	0.7791
0.4498	11.59	2400	0.4708	0.7844	0.7831
0.4515	12.56	2600	0.4800	0.7815	0.7800
0.445	13.53	2800	0.4796	0.7728	0.7713
0.4446	14.49	3000	0.4770	0.7803	0.7788
0.4338	15.46	3200	0.4799	0.7835	0.7825
0.4396	16.43	3400	0.4798	0.7797	0.7782
0.4335	17.39	3600	0.4743	0.7841	0.7828
0.429	18.36	3800	0.4714	0.7858	0.7858
0.4269	19.32	4000	0.4705	0.7920	0.7912
0.4222	20.29	4200	0.4872	0.7809	0.7800
0.426	21.26	4400	0.4792	0.7833	0.7818
0.4192	22.22	4600	0.4964	0.7758	0.7743
0.418	23.19	4800	0.4780	0.7823	0.7812
0.4172	24.15	5000	0.4955	0.7748	0.7734
0.4118	25.12	5200	0.5083	0.7752	0.7737
0.4093	26.09	5400	0.4897	0.7761	0.7746
0.4119	27.05	5600	0.5046	0.7707	0.7691
0.4055	28.02	5800	0.4882	0.7847	0.7834
0.405	28.99	6000	0.4886	0.7788	0.7773
0.4024	29.95	6200	0.4903	0.7714	0.7700
0.4001	30.92	6400	0.4825	0.7804	0.7803
0.3992	31.88	6600	0.4916	0.7755	0.7746
0.3932	32.85	6800	0.5003	0.7751	0.7737
0.3965	33.82	7000	0.5031	0.7695	0.7679
0.3912	34.78	7200	0.5025	0.7734	0.7719
0.3922	35.75	7400	0.4921	0.7713	0.7700
0.3893	36.71	7600	0.4995	0.7765	0.7752
0.386	37.68	7800	0.5018	0.7730	0.7716
0.3874	38.65	8000	0.5012	0.7749	0.7737
0.3909	39.61	8200	0.4984	0.7721	0.7710
0.382	40.58	8400	0.5084	0.7713	0.7697
0.3837	41.55	8600	0.5034	0.7743	0.7731
0.3819	42.51	8800	0.5033	0.7757	0.7746
0.3829	43.48	9000	0.5079	0.7757	0.7743
0.381	44.44	9200	0.5102	0.7727	0.7713
0.3843	45.41	9400	0.5049	0.7747	0.7734
0.376	46.38	9600	0.5101	0.7730	0.7716
0.3797	47.34	9800	0.5075	0.7729	0.7716
0.3789	48.31	10000	0.5064	0.7740	0.7728

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2