GUE_EMP_H3K4me2-seqsight_32768_512_43M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_EMP_H3K4me2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5889
• F1 Score: 0.6823
• Accuracy: 0.6859

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6634	1.04	200	0.6368	0.5949	0.6370
0.6269	2.08	400	0.6301	0.6479	0.6478
0.6197	3.12	600	0.6218	0.6430	0.6637
0.6175	4.17	800	0.6171	0.6532	0.6634
0.6135	5.21	1000	0.6189	0.6562	0.6572
0.6077	6.25	1200	0.6137	0.6643	0.6699
0.6004	7.29	1400	0.6209	0.6650	0.6641
0.6018	8.33	1600	0.6177	0.6605	0.6618
0.5998	9.38	1800	0.6248	0.6571	0.6546
0.5971	10.42	2000	0.6112	0.6675	0.6689
0.5978	11.46	2200	0.6064	0.6649	0.6725
0.5902	12.5	2400	0.6080	0.6656	0.6709
0.5888	13.54	2600	0.6064	0.6657	0.6742
0.591	14.58	2800	0.6076	0.6601	0.6712
0.5931	15.62	3000	0.6061	0.6685	0.6748
0.5876	16.67	3200	0.6108	0.6668	0.6686
0.5866	17.71	3400	0.6083	0.6722	0.6764
0.587	18.75	3600	0.6062	0.6657	0.6722
0.5859	19.79	3800	0.6069	0.6705	0.6751
0.5817	20.83	4000	0.6080	0.6707	0.6729
0.5844	21.88	4200	0.6106	0.6720	0.6738
0.5821	22.92	4400	0.6090	0.6717	0.6748
0.5835	23.96	4600	0.6083	0.6711	0.6729
0.5788	25.0	4800	0.6077	0.6734	0.6777
0.5792	26.04	5000	0.6075	0.6742	0.6777
0.5789	27.08	5200	0.6058	0.6730	0.6771
0.5787	28.12	5400	0.6047	0.6737	0.6777
0.577	29.17	5600	0.6072	0.6742	0.6764
0.5749	30.21	5800	0.6089	0.6764	0.6797
0.5777	31.25	6000	0.6071	0.6751	0.6787
0.5757	32.29	6200	0.6042	0.6748	0.6810
0.5751	33.33	6400	0.6049	0.6777	0.6823
0.5745	34.38	6600	0.6049	0.6736	0.6804
0.5729	35.42	6800	0.6059	0.6732	0.6787
0.5747	36.46	7000	0.6046	0.6749	0.6804
0.5719	37.5	7200	0.6063	0.6790	0.6830
0.5712	38.54	7400	0.6065	0.6757	0.6817
0.576	39.58	7600	0.6048	0.6730	0.6790
0.5734	40.62	7800	0.6080	0.6770	0.6790
0.572	41.67	8000	0.6053	0.6790	0.6826
0.5691	42.71	8200	0.6060	0.6743	0.6830
0.5714	43.75	8400	0.6064	0.6729	0.6777
0.5698	44.79	8600	0.6076	0.6774	0.6807
0.5691	45.83	8800	0.6062	0.6757	0.6810
0.5708	46.88	9000	0.6077	0.6771	0.6800
0.5687	47.92	9200	0.6071	0.6779	0.6813
0.57	48.96	9400	0.6062	0.6772	0.6826
0.5693	50.0	9600	0.6070	0.6768	0.6810
0.5705	51.04	9800	0.6063	0.6778	0.6823
0.5675	52.08	10000	0.6066	0.6770	0.6813

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2