GUE_EMP_H3K9ac-seqsight_32768_512_43M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_EMP_H3K9ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.4635
• F1 Score: 0.7915
• Accuracy: 0.7909

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5867	1.15	200	0.5738	0.7174	0.7172
0.5175	2.3	400	0.5902	0.6854	0.6909
0.4952	3.45	600	0.5512	0.7323	0.7330
0.4899	4.6	800	0.5397	0.7364	0.7370
0.4814	5.75	1000	0.5230	0.7506	0.7503
0.4769	6.9	1200	0.5291	0.7465	0.7463
0.4718	8.05	1400	0.5302	0.7483	0.7481
0.4688	9.2	1600	0.5332	0.7482	0.7488
0.4642	10.34	1800	0.5266	0.7500	0.7496
0.4591	11.49	2000	0.5179	0.7547	0.7542
0.4529	12.64	2200	0.5190	0.7553	0.7549
0.4541	13.79	2400	0.5267	0.7575	0.7575
0.4482	14.94	2600	0.5170	0.7601	0.7596
0.4441	16.09	2800	0.5429	0.7522	0.7531
0.441	17.24	3000	0.5347	0.7582	0.7578
0.4424	18.39	3200	0.5122	0.7648	0.7643
0.4418	19.54	3400	0.5085	0.7645	0.7643
0.4304	20.69	3600	0.4982	0.7665	0.7661
0.4322	21.84	3800	0.5246	0.7578	0.7582
0.4253	22.99	4000	0.5274	0.7545	0.7549
0.4304	24.14	4200	0.4977	0.7694	0.7690
0.4166	25.29	4400	0.5094	0.7738	0.7733
0.4239	26.44	4600	0.5087	0.7705	0.7701
0.4218	27.59	4800	0.5072	0.7675	0.7672
0.4143	28.74	5000	0.5074	0.7714	0.7711
0.4182	29.89	5200	0.5124	0.7705	0.7701
0.4117	31.03	5400	0.5165	0.7694	0.7693
0.4108	32.18	5600	0.5017	0.7777	0.7773
0.4025	33.33	5800	0.5173	0.7698	0.7693
0.4101	34.48	6000	0.5022	0.7781	0.7776
0.4003	35.63	6200	0.5014	0.7777	0.7773
0.4053	36.78	6400	0.5066	0.7756	0.7751
0.4024	37.93	6600	0.5323	0.7710	0.7708
0.398	39.08	6800	0.5153	0.7737	0.7733
0.3991	40.23	7000	0.5225	0.7634	0.7632
0.3957	41.38	7200	0.5148	0.7716	0.7711
0.3949	42.53	7400	0.5232	0.7682	0.7679
0.3934	43.68	7600	0.5160	0.7698	0.7693
0.3899	44.83	7800	0.5210	0.7700	0.7697
0.3933	45.98	8000	0.5074	0.7737	0.7733
0.3914	47.13	8200	0.5191	0.7682	0.7679
0.3847	48.28	8400	0.5182	0.7727	0.7722
0.3832	49.43	8600	0.5328	0.7643	0.7639
0.3883	50.57	8800	0.5249	0.7679	0.7675
0.384	51.72	9000	0.5237	0.7712	0.7708
0.3826	52.87	9200	0.5268	0.7668	0.7665
0.3849	54.02	9400	0.5224	0.7730	0.7726
0.3828	55.17	9600	0.5249	0.7694	0.7690
0.3827	56.32	9800	0.5188	0.7730	0.7726
0.3813	57.47	10000	0.5204	0.7705	0.7701

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2