GUE_EMP_H3K4me3-seqsight_4096_512_46M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_46M on the mahdibaghbanzadeh/GUE_EMP_H3K4me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5650
• F1 Score: 0.7048
• Accuracy: 0.7049

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6432	0.87	200	0.6165	0.6638	0.6641
0.6108	1.74	400	0.6013	0.6774	0.6772
0.6	2.61	600	0.5975	0.6740	0.675
0.5904	3.48	800	0.5925	0.6789	0.6796
0.5862	4.35	1000	0.5976	0.6748	0.6772
0.5837	5.22	1200	0.5871	0.6850	0.6864
0.5789	6.09	1400	0.5926	0.6843	0.6861
0.5751	6.96	1600	0.5854	0.6834	0.6832
0.5716	7.83	1800	0.5896	0.6761	0.6774
0.569	8.7	2000	0.5889	0.6859	0.6856
0.567	9.57	2200	0.5760	0.6869	0.6870
0.5665	10.43	2400	0.5823	0.6916	0.6913
0.5622	11.3	2600	0.5757	0.6900	0.6897
0.5658	12.17	2800	0.5766	0.6880	0.6880
0.5611	13.04	3000	0.5799	0.6917	0.6916
0.5585	13.91	3200	0.5750	0.6940	0.6937
0.5556	14.78	3400	0.5772	0.6939	0.6943
0.5572	15.65	3600	0.5763	0.6949	0.6946
0.5507	16.52	3800	0.5802	0.6937	0.6935
0.5539	17.39	4000	0.5754	0.6975	0.6973
0.5526	18.26	4200	0.5799	0.6991	0.6989
0.5506	19.13	4400	0.5792	0.6945	0.6943
0.5481	20.0	4600	0.5740	0.7030	0.7033
0.5481	20.87	4800	0.5770	0.7003	0.7003
0.5488	21.74	5000	0.5765	0.6978	0.6976
0.5472	22.61	5200	0.5760	0.7022	0.7019
0.5451	23.48	5400	0.5786	0.6971	0.6986
0.5438	24.35	5600	0.5770	0.6996	0.6997
0.5451	25.22	5800	0.5758	0.7026	0.7033
0.5398	26.09	6000	0.5825	0.6993	0.6997
0.5445	26.96	6200	0.5784	0.7024	0.7033
0.539	27.83	6400	0.5798	0.6992	0.7
0.5415	28.7	6600	0.5787	0.7003	0.7
0.5385	29.57	6800	0.5747	0.7048	0.7046
0.5353	30.43	7000	0.5783	0.7036	0.7041
0.5421	31.3	7200	0.5766	0.7032	0.7033
0.5388	32.17	7400	0.5753	0.7044	0.7043
0.5366	33.04	7600	0.5734	0.7035	0.7033
0.5372	33.91	7800	0.5777	0.7014	0.7016
0.5361	34.78	8000	0.5769	0.7032	0.7030
0.5349	35.65	8200	0.5768	0.7032	0.7030
0.5339	36.52	8400	0.5764	0.7048	0.7046
0.5352	37.39	8600	0.5759	0.7034	0.7033
0.5284	38.26	8800	0.5802	0.7026	0.7030
0.5395	39.13	9000	0.5747	0.7060	0.7063
0.5328	40.0	9200	0.5767	0.7039	0.7038
0.5306	40.87	9400	0.5771	0.7043	0.7041
0.5328	41.74	9600	0.5774	0.7044	0.7043
0.5359	42.61	9800	0.5761	0.7039	0.7038
0.5272	43.48	10000	0.5771	0.7048	0.7046

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2