GUE_EMP_H3K9ac-seqsight_4096_512_46M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_46M on the mahdibaghbanzadeh/GUE_EMP_H3K9ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.4860
• F1 Score: 0.7868
• Accuracy: 0.7863

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5557	1.15	200	0.5435	0.7278	0.7272
0.5083	2.3	400	0.5559	0.7251	0.7280
0.4847	3.45	600	0.5117	0.7588	0.7585
0.4722	4.6	800	0.4942	0.7637	0.7632
0.4661	5.75	1000	0.4936	0.7687	0.7683
0.4575	6.9	1200	0.4923	0.7702	0.7697
0.4504	8.05	1400	0.5031	0.7624	0.7621
0.442	9.2	1600	0.4930	0.7698	0.7697
0.4356	10.34	1800	0.4876	0.7700	0.7697
0.434	11.49	2000	0.4839	0.7726	0.7722
0.4251	12.64	2200	0.4829	0.7725	0.7726
0.4233	13.79	2400	0.4823	0.7755	0.7751
0.4205	14.94	2600	0.4722	0.7765	0.7765
0.4118	16.09	2800	0.4861	0.7733	0.7729
0.4088	17.24	3000	0.4833	0.7799	0.7794
0.4075	18.39	3200	0.4762	0.7748	0.7744
0.4032	19.54	3400	0.4768	0.7716	0.7711
0.3952	20.69	3600	0.4839	0.7788	0.7791
0.3926	21.84	3800	0.4781	0.7741	0.7737
0.391	22.99	4000	0.5085	0.7598	0.7603
0.3901	24.14	4200	0.4865	0.7719	0.7715
0.3786	25.29	4400	0.5031	0.7738	0.7733
0.3817	26.44	4600	0.4994	0.7695	0.7690
0.381	27.59	4800	0.4967	0.7763	0.7758
0.374	28.74	5000	0.4907	0.7727	0.7722
0.3769	29.89	5200	0.5001	0.7741	0.7737
0.3672	31.03	5400	0.5043	0.7671	0.7668
0.3688	32.18	5600	0.5008	0.7745	0.7740
0.3603	33.33	5800	0.5100	0.7799	0.7794
0.3643	34.48	6000	0.4972	0.7741	0.7737
0.3533	35.63	6200	0.5166	0.7758	0.7755
0.3604	36.78	6400	0.5027	0.7749	0.7744
0.3553	37.93	6600	0.5220	0.7687	0.7683
0.35	39.08	6800	0.5126	0.7741	0.7737
0.3499	40.23	7000	0.5196	0.7677	0.7672
0.3457	41.38	7200	0.5229	0.7684	0.7679
0.3458	42.53	7400	0.5237	0.7684	0.7679
0.3435	43.68	7600	0.5272	0.7708	0.7704
0.3402	44.83	7800	0.5261	0.7709	0.7704
0.3401	45.98	8000	0.5282	0.7696	0.7693
0.3397	47.13	8200	0.5327	0.7655	0.7650
0.3374	48.28	8400	0.5306	0.7691	0.7686
0.3336	49.43	8600	0.5371	0.7659	0.7654
0.335	50.57	8800	0.5357	0.7687	0.7683
0.3384	51.72	9000	0.5340	0.7695	0.7690
0.3308	52.87	9200	0.5367	0.7666	0.7661
0.3318	54.02	9400	0.5352	0.7677	0.7672
0.3341	55.17	9600	0.5344	0.7659	0.7654
0.3304	56.32	9800	0.5349	0.7673	0.7668
0.3319	57.47	10000	0.5345	0.7673	0.7668

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2