GUE_EMP_H3K4me3-seqsight_16384_512_56M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3K4me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5969
• F1 Score: 0.6975
• Accuracy: 0.6976

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6432	0.87	200	0.6124	0.6610	0.6609
0.6088	1.74	400	0.6067	0.6690	0.6707
0.5947	2.61	600	0.5922	0.6828	0.6829
0.5882	3.48	800	0.5893	0.6794	0.6793
0.5822	4.35	1000	0.5875	0.6769	0.6766
0.5758	5.22	1200	0.5858	0.6832	0.6848
0.5728	6.09	1400	0.6043	0.6695	0.6742
0.5666	6.96	1600	0.5931	0.6826	0.6840
0.5612	7.83	1800	0.5899	0.6813	0.6810
0.5593	8.7	2000	0.5884	0.6871	0.6875
0.5557	9.57	2200	0.5817	0.6863	0.6864
0.5536	10.43	2400	0.5959	0.6865	0.6891
0.5501	11.3	2600	0.5791	0.6954	0.6970
0.5528	12.17	2800	0.5763	0.6920	0.6924
0.5447	13.04	3000	0.5880	0.6907	0.6929
0.5401	13.91	3200	0.5858	0.6926	0.6946
0.5375	14.78	3400	0.5954	0.6903	0.6937
0.5371	15.65	3600	0.5845	0.6852	0.6883
0.5352	16.52	3800	0.5785	0.6947	0.6948
0.5285	17.39	4000	0.6022	0.6984	0.7003
0.5315	18.26	4200	0.5866	0.6940	0.6959
0.5242	19.13	4400	0.5850	0.6995	0.6995
0.5238	20.0	4600	0.5912	0.6982	0.7008
0.5193	20.87	4800	0.5875	0.6972	0.6976
0.5196	21.74	5000	0.5850	0.6949	0.6951
0.5183	22.61	5200	0.5878	0.6933	0.6948
0.5173	23.48	5400	0.5961	0.6909	0.6943
0.5097	24.35	5600	0.5933	0.6947	0.6965
0.5118	25.22	5800	0.5924	0.6993	0.7
0.5061	26.09	6000	0.6060	0.6951	0.6970
0.5106	26.96	6200	0.5891	0.6928	0.6957
0.5045	27.83	6400	0.6064	0.6856	0.6889
0.5042	28.7	6600	0.5888	0.6982	0.6981
0.5017	29.57	6800	0.5842	0.6985	0.6989
0.4993	30.43	7000	0.5908	0.6971	0.6984
0.5033	31.3	7200	0.5922	0.7005	0.7011
0.5005	32.17	7400	0.5878	0.6983	0.6986
0.4961	33.04	7600	0.5890	0.7012	0.7014
0.4948	33.91	7800	0.5893	0.6981	0.6989
0.4955	34.78	8000	0.5919	0.7009	0.7014
0.4931	35.65	8200	0.5915	0.7000	0.7
0.4898	36.52	8400	0.5890	0.6999	0.7
0.4875	37.39	8600	0.5926	0.6985	0.6984
0.4874	38.26	8800	0.5965	0.7008	0.7014
0.4915	39.13	9000	0.5920	0.7020	0.7022
0.486	40.0	9200	0.5944	0.6986	0.6984
0.4873	40.87	9400	0.5935	0.7029	0.7030
0.4862	41.74	9600	0.5929	0.7023	0.7024
0.4929	42.61	9800	0.5914	0.7015	0.7016
0.4828	43.48	10000	0.5937	0.7034	0.7035

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2