GUE_EMP_H3K4me2-seqsight_4096_512_27M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_EMP_H3K4me2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.6008
• F1 Score: 0.6949
• Accuracy: 0.6966

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6458	1.04	200	0.6226	0.6116	0.6530
0.6129	2.08	400	0.6141	0.6644	0.6660
0.603	3.12	600	0.6090	0.6474	0.6716
0.6003	4.17	800	0.6077	0.6654	0.6696
0.5943	5.21	1000	0.6039	0.6616	0.6748
0.5915	6.25	1200	0.5994	0.6663	0.6745
0.5844	7.29	1400	0.6025	0.6727	0.6742
0.5824	8.33	1600	0.6039	0.6751	0.6764
0.5792	9.38	1800	0.6118	0.6720	0.6699
0.5753	10.42	2000	0.5926	0.6798	0.6875
0.575	11.46	2200	0.5912	0.6809	0.6869
0.5641	12.5	2400	0.5905	0.6838	0.6875
0.5619	13.54	2600	0.5914	0.6821	0.6852
0.5613	14.58	2800	0.5963	0.6792	0.6852
0.5629	15.62	3000	0.5991	0.6801	0.6823
0.5555	16.67	3200	0.5909	0.6881	0.6898
0.5535	17.71	3400	0.5917	0.6846	0.6875
0.5504	18.75	3600	0.5947	0.6876	0.6953
0.5497	19.79	3800	0.5970	0.6926	0.6947
0.5426	20.83	4000	0.5979	0.6873	0.6885
0.5442	21.88	4200	0.6118	0.6855	0.6839
0.5419	22.92	4400	0.6027	0.6879	0.6898
0.5412	23.96	4600	0.6037	0.6882	0.6875
0.5382	25.0	4800	0.6052	0.6881	0.6882
0.5318	26.04	5000	0.6095	0.6861	0.6859
0.5315	27.08	5200	0.6105	0.6846	0.6836
0.5292	28.12	5400	0.6067	0.6856	0.6862
0.527	29.17	5600	0.6062	0.6890	0.6895
0.5219	30.21	5800	0.6187	0.6903	0.6898
0.5243	31.25	6000	0.6131	0.6895	0.6891
0.5184	32.29	6200	0.6067	0.6894	0.6924
0.5204	33.33	6400	0.6196	0.6910	0.6895
0.5193	34.38	6600	0.6086	0.6923	0.6950
0.5179	35.42	6800	0.6108	0.6929	0.6940
0.5166	36.46	7000	0.6078	0.6878	0.6898
0.5112	37.5	7200	0.6146	0.6900	0.6901
0.5084	38.54	7400	0.6157	0.6921	0.6934
0.5122	39.58	7600	0.6130	0.6883	0.6911
0.5104	40.62	7800	0.6234	0.6849	0.6843
0.5106	41.67	8000	0.6163	0.6912	0.6921
0.5039	42.71	8200	0.6181	0.6899	0.6911
0.5064	43.75	8400	0.6206	0.6891	0.6888
0.5071	44.79	8600	0.6204	0.6871	0.6865
0.5018	45.83	8800	0.6169	0.6884	0.6898
0.5034	46.88	9000	0.6281	0.6864	0.6852
0.501	47.92	9200	0.6240	0.6899	0.6895
0.504	48.96	9400	0.6217	0.6918	0.6921
0.4986	50.0	9600	0.6262	0.6884	0.6878
0.5038	51.04	9800	0.6240	0.6892	0.6888
0.497	52.08	10000	0.6238	0.6884	0.6882

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2