GUE_EMP_H3K14ac-seqsight_32768_512_43M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_EMP_H3K14ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.4984
• F1 Score: 0.7700
• Accuracy: 0.7691

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5881	0.97	200	0.5331	0.7519	0.7504
0.5288	1.93	400	0.5084	0.7643	0.7628
0.5108	2.9	600	0.5162	0.7548	0.7534
0.5075	3.86	800	0.4914	0.7690	0.7682
0.5005	4.83	1000	0.5060	0.7655	0.7640
0.4943	5.8	1200	0.4978	0.7701	0.7685
0.4904	6.76	1400	0.4867	0.7751	0.7737
0.4863	7.73	1600	0.4914	0.7740	0.7725
0.4831	8.7	1800	0.4916	0.7698	0.7682
0.4792	9.66	2000	0.4948	0.7734	0.7719
0.4808	10.63	2200	0.4976	0.7713	0.7697
0.4736	11.59	2400	0.4820	0.7721	0.7707
0.4753	12.56	2600	0.4928	0.7758	0.7743
0.4685	13.53	2800	0.4896	0.7722	0.7707
0.469	14.49	3000	0.4958	0.7746	0.7731
0.4594	15.46	3200	0.4800	0.7779	0.7767
0.4653	16.43	3400	0.4969	0.7736	0.7722
0.4602	17.39	3600	0.4808	0.7778	0.7764
0.4567	18.36	3800	0.4809	0.7765	0.7761
0.4558	19.32	4000	0.4864	0.7802	0.7788
0.4537	20.29	4200	0.4880	0.7760	0.7746
0.4516	21.26	4400	0.4905	0.7761	0.7746
0.4498	22.22	4600	0.5092	0.7702	0.7688
0.4484	23.19	4800	0.4872	0.7731	0.7719
0.4479	24.15	5000	0.4912	0.7679	0.7664
0.4463	25.12	5200	0.5022	0.7737	0.7722
0.4407	26.09	5400	0.4960	0.7710	0.7694
0.4414	27.05	5600	0.5094	0.7707	0.7691
0.4399	28.02	5800	0.4877	0.7719	0.7707
0.44	28.99	6000	0.4894	0.7752	0.7737
0.4353	29.95	6200	0.4999	0.7692	0.7676
0.4355	30.92	6400	0.4850	0.7729	0.7725
0.4349	31.88	6600	0.4909	0.7722	0.7710
0.432	32.85	6800	0.5072	0.7674	0.7658
0.4368	33.82	7000	0.5021	0.7707	0.7691
0.4289	34.78	7200	0.5049	0.7716	0.7700
0.4296	35.75	7400	0.4976	0.7747	0.7734
0.4261	36.71	7600	0.5024	0.7698	0.7682
0.425	37.68	7800	0.5051	0.7701	0.7685
0.4272	38.65	8000	0.4953	0.7735	0.7722
0.432	39.61	8200	0.4941	0.7711	0.7697
0.4189	40.58	8400	0.5041	0.7701	0.7685
0.421	41.55	8600	0.5030	0.7710	0.7694
0.4204	42.51	8800	0.4993	0.7706	0.7691
0.421	43.48	9000	0.5108	0.7710	0.7694
0.4199	44.44	9200	0.5078	0.7677	0.7661
0.4216	45.41	9400	0.5051	0.7692	0.7676
0.4155	46.38	9600	0.5062	0.7683	0.7667
0.4253	47.34	9800	0.5025	0.7701	0.7685
0.4169	48.31	10000	0.5015	0.7724	0.7710

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2