GUE_EMP_H3K4me2-seqsight_32768_512_43M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_EMP_H3K4me2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5958
• F1 Score: 0.6827
• Accuracy: 0.6859

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6544	1.04	200	0.6235	0.6306	0.6556
0.6187	2.08	400	0.6353	0.6397	0.6370
0.6082	3.12	600	0.6119	0.6639	0.6670
0.6041	4.17	800	0.6275	0.6549	0.6527
0.5998	5.21	1000	0.6067	0.6745	0.6807
0.5941	6.25	1200	0.6047	0.6746	0.6777
0.5862	7.29	1400	0.6132	0.6688	0.6676
0.5851	8.33	1600	0.6192	0.6728	0.6712
0.583	9.38	1800	0.6262	0.6607	0.6582
0.5799	10.42	2000	0.5997	0.6783	0.6843
0.58	11.46	2200	0.6031	0.6759	0.6774
0.5704	12.5	2400	0.6035	0.6793	0.6820
0.569	13.54	2600	0.6077	0.6813	0.6813
0.5687	14.58	2800	0.6074	0.6732	0.6777
0.5694	15.62	3000	0.6038	0.6775	0.6787
0.5639	16.67	3200	0.6062	0.6764	0.6761
0.56	17.71	3400	0.6144	0.6696	0.6686
0.5615	18.75	3600	0.6066	0.6847	0.6865
0.5586	19.79	3800	0.6191	0.6777	0.6764
0.5537	20.83	4000	0.6056	0.6795	0.6797
0.5519	21.88	4200	0.6202	0.6727	0.6709
0.5497	22.92	4400	0.6200	0.6798	0.6787
0.5489	23.96	4600	0.6198	0.6710	0.6693
0.5436	25.0	4800	0.6249	0.6795	0.6787
0.5427	26.04	5000	0.6220	0.6797	0.6790
0.5429	27.08	5200	0.6125	0.6775	0.6768
0.5397	28.12	5400	0.6088	0.6769	0.6774
0.5375	29.17	5600	0.6170	0.6782	0.6790
0.5335	30.21	5800	0.6257	0.6752	0.6748
0.5343	31.25	6000	0.6239	0.6785	0.6777
0.5323	32.29	6200	0.6155	0.6747	0.6755
0.5325	33.33	6400	0.6229	0.6756	0.6755
0.5274	34.38	6600	0.6185	0.6718	0.6745
0.5289	35.42	6800	0.6177	0.6784	0.6790
0.5255	36.46	7000	0.6233	0.6782	0.6781
0.5242	37.5	7200	0.6262	0.6801	0.6794
0.5206	38.54	7400	0.6232	0.6783	0.6790
0.5248	39.58	7600	0.6167	0.6799	0.6823
0.5231	40.62	7800	0.6301	0.6737	0.6725
0.5205	41.67	8000	0.6185	0.6763	0.6771
0.515	42.71	8200	0.6307	0.6749	0.6748
0.5195	43.75	8400	0.6224	0.6778	0.6777
0.5169	44.79	8600	0.6281	0.6767	0.6761
0.5146	45.83	8800	0.6279	0.6794	0.6804
0.5139	46.88	9000	0.6355	0.6762	0.6748
0.5144	47.92	9200	0.6329	0.6781	0.6774
0.5148	48.96	9400	0.6308	0.6771	0.6774
0.5131	50.0	9600	0.6336	0.6774	0.6768
0.5143	51.04	9800	0.6331	0.6783	0.6777
0.5076	52.08	10000	0.6350	0.6765	0.6758

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2