GUE_EMP_H3K4me2-seqsight_16384_512_56M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3K4me2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5991
• F1 Score: 0.6846
• Accuracy: 0.6888

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6391	1.04	200	0.6195	0.6206	0.6562
0.6074	2.08	400	0.6297	0.6565	0.6540
0.5975	3.12	600	0.5994	0.6742	0.6820
0.5937	4.17	800	0.6089	0.6670	0.6673
0.5889	5.21	1000	0.6060	0.6689	0.6781
0.5855	6.25	1200	0.5963	0.6736	0.6820
0.5777	7.29	1400	0.5950	0.6811	0.6849
0.5782	8.33	1600	0.6053	0.6769	0.6777
0.5733	9.38	1800	0.6000	0.6743	0.6745
0.5691	10.42	2000	0.5975	0.6730	0.6843
0.5691	11.46	2200	0.5956	0.6726	0.6859
0.5583	12.5	2400	0.6013	0.6764	0.6846
0.5588	13.54	2600	0.6029	0.6834	0.6852
0.5587	14.58	2800	0.6056	0.6715	0.6856
0.5565	15.62	3000	0.5997	0.6864	0.6908
0.5473	16.67	3200	0.6002	0.6847	0.6911
0.5471	17.71	3400	0.6040	0.6811	0.6820
0.5459	18.75	3600	0.6023	0.6906	0.6947
0.544	19.79	3800	0.6026	0.6819	0.6830
0.5368	20.83	4000	0.5968	0.6882	0.6937
0.5393	21.88	4200	0.6100	0.6830	0.6836
0.5357	22.92	4400	0.6072	0.6861	0.6882
0.5333	23.96	4600	0.6071	0.6864	0.6859
0.5287	25.0	4800	0.6097	0.6803	0.6810
0.5251	26.04	5000	0.6108	0.6854	0.6878
0.5247	27.08	5200	0.6076	0.6877	0.6885
0.5213	28.12	5400	0.6099	0.6815	0.6826
0.5185	29.17	5600	0.6080	0.6862	0.6895
0.5158	30.21	5800	0.6185	0.6785	0.6810
0.5142	31.25	6000	0.6114	0.6845	0.6872
0.5144	32.29	6200	0.6159	0.6741	0.6755
0.5152	33.33	6400	0.6179	0.6809	0.6804
0.5086	34.38	6600	0.6245	0.6812	0.6875
0.5058	35.42	6800	0.6241	0.6732	0.6732
0.5064	36.46	7000	0.6216	0.6814	0.6813
0.5041	37.5	7200	0.6184	0.6840	0.6856
0.4978	38.54	7400	0.6274	0.6766	0.6794
0.5034	39.58	7600	0.6228	0.6807	0.6865
0.5008	40.62	7800	0.6338	0.6775	0.6764
0.4983	41.67	8000	0.6237	0.6793	0.6790
0.4917	42.71	8200	0.6340	0.6759	0.6771
0.4972	43.75	8400	0.6290	0.6775	0.6768
0.4955	44.79	8600	0.6269	0.6775	0.6777
0.4902	45.83	8800	0.6266	0.6778	0.6794
0.4928	46.88	9000	0.6318	0.6731	0.6722
0.4917	47.92	9200	0.6288	0.6773	0.6771
0.4911	48.96	9400	0.6282	0.6727	0.6738
0.4903	50.0	9600	0.6311	0.6742	0.6742
0.4888	51.04	9800	0.6316	0.6721	0.6722
0.4845	52.08	10000	0.6314	0.6734	0.6735

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2