GUE_EMP_H3K4me2-seqsight_16384_512_56M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3K4me2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5947
• F1 Score: 0.6824
• Accuracy: 0.6891

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.636	1.04	200	0.6087	0.6525	0.6683
0.6027	2.08	400	0.6371	0.6472	0.6448
0.5907	3.12	600	0.6005	0.6783	0.6839
0.5864	4.17	800	0.5999	0.6793	0.6807
0.5755	5.21	1000	0.6054	0.6726	0.6768
0.5691	6.25	1200	0.5921	0.6819	0.6856
0.5599	7.29	1400	0.5950	0.6850	0.6914
0.5552	8.33	1600	0.6075	0.6805	0.6817
0.5444	9.38	1800	0.6082	0.6826	0.6813
0.5373	10.42	2000	0.6086	0.6776	0.6820
0.5321	11.46	2200	0.6017	0.6780	0.6856
0.5172	12.5	2400	0.6228	0.6803	0.6852
0.508	13.54	2600	0.6226	0.6767	0.6771
0.504	14.58	2800	0.6280	0.6654	0.6686
0.4956	15.62	3000	0.6334	0.6742	0.6742
0.4791	16.67	3200	0.6394	0.6778	0.6794
0.4715	17.71	3400	0.6500	0.6612	0.6592
0.4647	18.75	3600	0.6655	0.6715	0.6699
0.4541	19.79	3800	0.6991	0.6647	0.6628
0.4432	20.83	4000	0.6577	0.6676	0.6667
0.4384	21.88	4200	0.7055	0.6646	0.6631
0.4261	22.92	4400	0.7187	0.6459	0.6432
0.4209	23.96	4600	0.6900	0.6736	0.6735
0.4069	25.0	4800	0.7107	0.6618	0.6601
0.3971	26.04	5000	0.7382	0.6633	0.6618
0.3943	27.08	5200	0.7328	0.6578	0.6556
0.3864	28.12	5400	0.7531	0.6634	0.6611
0.3762	29.17	5600	0.7479	0.6719	0.6722
0.3672	30.21	5800	0.7751	0.6542	0.6520
0.36	31.25	6000	0.7859	0.6605	0.6588
0.3614	32.29	6200	0.7895	0.6550	0.6527
0.3536	33.33	6400	0.7837	0.6685	0.6676
0.3483	34.38	6600	0.7955	0.6657	0.6667
0.3391	35.42	6800	0.8129	0.6653	0.6641
0.3407	36.46	7000	0.7978	0.6617	0.6595
0.3335	37.5	7200	0.8079	0.6648	0.6644
0.3227	38.54	7400	0.8304	0.6615	0.6615
0.3291	39.58	7600	0.8175	0.6639	0.6647
0.321	40.62	7800	0.8559	0.6608	0.6585
0.3141	41.67	8000	0.8459	0.6617	0.6605
0.3091	42.71	8200	0.8625	0.6639	0.6637
0.3056	43.75	8400	0.8581	0.6616	0.6598
0.3027	44.79	8600	0.8863	0.6657	0.6641
0.2968	45.83	8800	0.8766	0.6633	0.6611
0.2979	46.88	9000	0.8802	0.6583	0.6559
0.2954	47.92	9200	0.8858	0.6617	0.6598
0.2919	48.96	9400	0.8817	0.6678	0.6667
0.2894	50.0	9600	0.8913	0.6630	0.6611
0.2894	51.04	9800	0.8927	0.6666	0.6650
0.2844	52.08	10000	0.8937	0.6663	0.6647

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2