GUE_EMP_H3K4me2-seqsight_4096_512_15M-L8

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_15M on the mahdibaghbanzadeh/GUE_EMP_H3K4me2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.6501
• F1 Score: 0.6111
• Accuracy: 0.6149

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 20000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6676	2.08	400	0.6491	0.6233	0.6289
0.6437	4.17	800	0.6440	0.6196	0.6325
0.6351	6.25	1200	0.6476	0.5929	0.6325
0.6288	8.33	1600	0.6507	0.6118	0.6282
0.6229	10.42	2000	0.6528	0.5995	0.6351
0.6185	12.5	2400	0.6508	0.6093	0.6312
0.6147	14.58	2800	0.6626	0.5964	0.6266
0.6142	16.67	3200	0.6684	0.6093	0.6132
0.6096	18.75	3600	0.6662	0.5982	0.6233
0.6069	20.83	4000	0.6684	0.6040	0.6155
0.6066	22.92	4400	0.6680	0.6080	0.6142
0.6047	25.0	4800	0.6657	0.5989	0.6220
0.6005	27.08	5200	0.6764	0.6039	0.6106
0.6003	29.17	5600	0.6646	0.6077	0.6230
0.599	31.25	6000	0.6842	0.6020	0.6051
0.5972	33.33	6400	0.6896	0.6032	0.6106
0.5946	35.42	6800	0.6996	0.6022	0.6090
0.5931	37.5	7200	0.6772	0.6002	0.6116
0.5898	39.58	7600	0.6896	0.5849	0.6168
0.5896	41.67	8000	0.6790	0.6032	0.6109
0.5873	43.75	8400	0.6714	0.5965	0.6116
0.5865	45.83	8800	0.6750	0.5981	0.6119
0.5838	47.92	9200	0.6875	0.5960	0.6168
0.5814	50.0	9600	0.6819	0.5990	0.6096
0.5792	52.08	10000	0.7014	0.6018	0.6100
0.5791	54.17	10400	0.6982	0.5939	0.5976
0.5757	56.25	10800	0.6949	0.5923	0.6044
0.5742	58.33	11200	0.7037	0.5960	0.6077
0.5701	60.42	11600	0.7101	0.5943	0.6002
0.5715	62.5	12000	0.6863	0.5946	0.6031
0.571	64.58	12400	0.7031	0.5968	0.6035
0.5712	66.67	12800	0.6976	0.5978	0.6038
0.5651	68.75	13200	0.6920	0.5993	0.6051
0.5653	70.83	13600	0.7145	0.5993	0.6028
0.5633	72.92	14000	0.6949	0.5968	0.5982
0.5614	75.0	14400	0.7069	0.5987	0.6096
0.5618	77.08	14800	0.7128	0.6004	0.6087
0.5594	79.17	15200	0.7238	0.5977	0.5995
0.5588	81.25	15600	0.7187	0.5962	0.6048
0.5571	83.33	16000	0.7269	0.5958	0.5999
0.556	85.42	16400	0.7159	0.5975	0.6051
0.559	87.5	16800	0.7159	0.5985	0.6051
0.555	89.58	17200	0.7089	0.5921	0.5973
0.5536	91.67	17600	0.7231	0.5947	0.6054
0.5552	93.75	18000	0.7201	0.5940	0.5966
0.5508	95.83	18400	0.7150	0.5957	0.6048
0.5528	97.92	18800	0.7199	0.5975	0.6025
0.5532	100.0	19200	0.7209	0.5985	0.6048
0.5517	102.08	19600	0.7159	0.5988	0.6044
0.5503	104.17	20000	0.7178	0.5993	0.6051

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2