GUE_EMP_H3K9ac-seqsight_4096_512_15M-L32

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_15M on the mahdibaghbanzadeh/GUE_EMP_H3K9ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.6684
• F1 Score: 0.6475
• Accuracy: 0.6466

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 20000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6563	2.3	400	0.6624	0.5624	0.5883
0.6214	4.6	800	0.6420	0.6168	0.6196
0.6062	6.9	1200	0.6454	0.6192	0.6225
0.5987	9.2	1600	0.6448	0.6184	0.6200
0.594	11.49	2000	0.6543	0.6258	0.6297
0.5826	13.79	2400	0.6504	0.6206	0.6214
0.5744	16.09	2800	0.6659	0.6213	0.6211
0.5674	18.39	3200	0.6616	0.6226	0.6225
0.5611	20.69	3600	0.6819	0.6042	0.6178
0.5498	22.99	4000	0.6634	0.6213	0.6211
0.5419	25.29	4400	0.6863	0.6192	0.6189
0.5359	27.59	4800	0.6987	0.6278	0.6272
0.5296	29.89	5200	0.6849	0.6264	0.6258
0.5232	32.18	5600	0.7045	0.6140	0.6135
0.5187	34.48	6000	0.6941	0.6200	0.6193
0.5102	36.78	6400	0.7068	0.6221	0.6222
0.504	39.08	6800	0.7208	0.6182	0.6175
0.5005	41.38	7200	0.7050	0.6189	0.6182
0.4978	43.68	7600	0.7107	0.6273	0.6265
0.4922	45.98	8000	0.7217	0.6172	0.6164
0.4832	48.28	8400	0.7313	0.6180	0.6175
0.482	50.57	8800	0.7445	0.6201	0.6193
0.4796	52.87	9200	0.7469	0.6169	0.6175
0.4756	55.17	9600	0.7503	0.6176	0.6168
0.4716	57.47	10000	0.7346	0.6099	0.6092
0.4699	59.77	10400	0.7611	0.6165	0.6157
0.4679	62.07	10800	0.7419	0.6098	0.6092
0.4598	64.37	11200	0.7466	0.6133	0.6132
0.457	66.67	11600	0.7797	0.6164	0.6157
0.456	68.97	12000	0.7518	0.6233	0.6229
0.4514	71.26	12400	0.7476	0.6159	0.6153
0.4478	73.56	12800	0.7651	0.6116	0.6121
0.4472	75.86	13200	0.7737	0.6147	0.6139
0.4462	78.16	13600	0.7619	0.6207	0.6200
0.4428	80.46	14000	0.7756	0.6158	0.6153
0.439	82.76	14400	0.7422	0.6130	0.6125
0.439	85.06	14800	0.7920	0.6125	0.6117
0.4335	87.36	15200	0.7711	0.6104	0.6099
0.4352	89.66	15600	0.7922	0.6197	0.6189
0.4347	91.95	16000	0.7891	0.6157	0.6150
0.4279	94.25	16400	0.8052	0.6137	0.6132
0.4271	96.55	16800	0.7765	0.6147	0.6139
0.4255	98.85	17200	0.8000	0.6173	0.6168
0.424	101.15	17600	0.8041	0.6154	0.6146
0.4212	103.45	18000	0.8116	0.6164	0.6157
0.4207	105.75	18400	0.7954	0.6110	0.6103
0.4195	108.05	18800	0.8095	0.6189	0.6182
0.4202	110.34	19200	0.8015	0.6136	0.6128
0.4198	112.64	19600	0.8013	0.6125	0.6117
0.4176	114.94	20000	0.7996	0.6140	0.6132

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2