GUE_EMP_H3K9ac-seqsight_4096_512_15M-L8

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_15M on the mahdibaghbanzadeh/GUE_EMP_H3K9ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.6285
• F1 Score: 0.6482
• Accuracy: 0.6477

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 20000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6674	2.3	400	0.6659	0.5593	0.5880
0.6272	4.6	800	0.6373	0.6213	0.6214
0.6134	6.9	1200	0.6421	0.6189	0.6204
0.6059	9.2	1600	0.6502	0.6071	0.6142
0.6019	11.49	2000	0.6527	0.6253	0.6322
0.5938	13.79	2400	0.6438	0.6325	0.6322
0.5885	16.09	2800	0.6452	0.6216	0.6211
0.587	18.39	3200	0.6448	0.6118	0.6139
0.5854	20.69	3600	0.6519	0.6201	0.6258
0.5802	22.99	4000	0.6462	0.6201	0.6193
0.5766	25.29	4400	0.6542	0.6161	0.6153
0.576	27.59	4800	0.6671	0.6068	0.6071
0.5752	29.89	5200	0.6601	0.6246	0.6254
0.5704	32.18	5600	0.6689	0.6245	0.6258
0.5705	34.48	6000	0.6492	0.6112	0.6114
0.5625	36.78	6400	0.6586	0.6244	0.6250
0.5601	39.08	6800	0.6769	0.6185	0.6182
0.5571	41.38	7200	0.6726	0.6207	0.6200
0.5555	43.68	7600	0.6772	0.6129	0.6121
0.5512	45.98	8000	0.6787	0.6161	0.6157
0.5476	48.28	8400	0.6736	0.6160	0.6157
0.5436	50.57	8800	0.6811	0.6150	0.6142
0.5401	52.87	9200	0.6851	0.6208	0.6204
0.5386	55.17	9600	0.6851	0.6201	0.6193
0.5368	57.47	10000	0.6857	0.6167	0.6160
0.5345	59.77	10400	0.6902	0.6120	0.6132
0.533	62.07	10800	0.6817	0.6185	0.6178
0.5258	64.37	11200	0.7068	0.6115	0.6110
0.5261	66.67	11600	0.7208	0.6219	0.6211
0.5252	68.97	12000	0.6775	0.6154	0.6146
0.5222	71.26	12400	0.7020	0.6128	0.6128
0.5207	73.56	12800	0.6988	0.6118	0.6110
0.52	75.86	13200	0.6988	0.6197	0.6189
0.5177	78.16	13600	0.7216	0.6183	0.6175
0.5161	80.46	14000	0.7127	0.6127	0.6121
0.514	82.76	14400	0.6943	0.6157	0.6150
0.515	85.06	14800	0.7077	0.6147	0.6139
0.511	87.36	15200	0.7019	0.6171	0.6164
0.5113	89.66	15600	0.7156	0.6172	0.6164
0.5109	91.95	16000	0.7194	0.6172	0.6164
0.5085	94.25	16400	0.7196	0.6168	0.6160
0.5079	96.55	16800	0.7131	0.6176	0.6168
0.5063	98.85	17200	0.7138	0.6164	0.6157
0.5048	101.15	17600	0.7200	0.6172	0.6164
0.5064	103.45	18000	0.7138	0.6196	0.6189
0.5022	105.75	18400	0.7192	0.6193	0.6186
0.5059	108.05	18800	0.7194	0.6179	0.6171
0.5045	110.34	19200	0.7149	0.6190	0.6182
0.5027	112.64	19600	0.7179	0.6186	0.6178
0.5037	114.94	20000	0.7169	0.6179	0.6171

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2