GUE_EMP_H3K9ac-seqsight_16384_512_56M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3K9ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.4826
• F1 Score: 0.7893
• Accuracy: 0.7888

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5716	1.15	200	0.5526	0.7232	0.7229
0.5198	2.3	400	0.5708	0.6990	0.7039
0.4906	3.45	600	0.5257	0.7421	0.7424
0.4834	4.6	800	0.5103	0.7440	0.7442
0.4792	5.75	1000	0.5061	0.7574	0.7571
0.4697	6.9	1200	0.5028	0.7583	0.7578
0.4663	8.05	1400	0.5187	0.7451	0.7452
0.4617	9.2	1600	0.5189	0.7366	0.7384
0.4539	10.34	1800	0.5051	0.7600	0.7596
0.4513	11.49	2000	0.5022	0.7568	0.7567
0.4441	12.64	2200	0.5134	0.7474	0.7485
0.4441	13.79	2400	0.5256	0.7420	0.7442
0.4386	14.94	2600	0.4957	0.7596	0.7596
0.4343	16.09	2800	0.5198	0.7446	0.7463
0.4309	17.24	3000	0.5055	0.7608	0.7607
0.4261	18.39	3200	0.5004	0.7610	0.7607
0.427	19.54	3400	0.4949	0.7589	0.7589
0.4197	20.69	3600	0.4976	0.7673	0.7668
0.4211	21.84	3800	0.5279	0.7488	0.7503
0.4137	22.99	4000	0.5355	0.7462	0.7478
0.4159	24.14	4200	0.4833	0.7741	0.7737
0.4065	25.29	4400	0.5006	0.7661	0.7657
0.4073	26.44	4600	0.5198	0.7591	0.7593
0.4071	27.59	4800	0.5177	0.7584	0.7589
0.3981	28.74	5000	0.5070	0.7573	0.7575
0.4038	29.89	5200	0.5085	0.7685	0.7683
0.3935	31.03	5400	0.5313	0.7532	0.7542
0.3959	32.18	5600	0.5124	0.7676	0.7675
0.387	33.33	5800	0.5151	0.7710	0.7708
0.3946	34.48	6000	0.5046	0.7737	0.7733
0.3824	35.63	6200	0.5079	0.7748	0.7744
0.3887	36.78	6400	0.5168	0.7655	0.7654
0.3817	37.93	6600	0.5358	0.7587	0.7593
0.3819	39.08	6800	0.5097	0.7685	0.7683
0.3795	40.23	7000	0.5268	0.7590	0.7593
0.377	41.38	7200	0.5260	0.7626	0.7625
0.3792	42.53	7400	0.5261	0.7598	0.7600
0.376	43.68	7600	0.5163	0.7693	0.7690
0.3694	44.83	7800	0.5214	0.7647	0.7647
0.3722	45.98	8000	0.5140	0.7697	0.7693
0.3719	47.13	8200	0.5319	0.7581	0.7582
0.3696	48.28	8400	0.5281	0.7608	0.7607
0.3648	49.43	8600	0.5329	0.7561	0.7560
0.3661	50.57	8800	0.5336	0.7633	0.7632
0.3686	51.72	9000	0.5273	0.7692	0.7690
0.3636	52.87	9200	0.5321	0.7598	0.7596
0.3651	54.02	9400	0.5381	0.7581	0.7582
0.366	55.17	9600	0.5369	0.7596	0.7596
0.3648	56.32	9800	0.5287	0.7678	0.7675
0.3621	57.47	10000	0.5303	0.7641	0.7639

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2