GUE_EMP_H3K79me3-seqsight_16384_512_34M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_34M on the mahdibaghbanzadeh/GUE_EMP_H3K79me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4351
• F1 Score: 0.8097
• Accuracy: 0.8096

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5289	1.1	200	0.4655	0.7962	0.7961
0.4736	2.21	400	0.4711	0.7904	0.7926
0.4658	3.31	600	0.4509	0.8034	0.8044
0.4533	4.42	800	0.4448	0.8066	0.8076
0.4528	5.52	1000	0.4530	0.7972	0.7992
0.4433	6.63	1200	0.4542	0.8020	0.8041
0.4454	7.73	1400	0.4403	0.8083	0.8096
0.439	8.84	1600	0.4545	0.8003	0.8027
0.4355	9.94	1800	0.4351	0.8177	0.8176
0.4344	11.05	2000	0.4376	0.8087	0.8100
0.4338	12.15	2200	0.4301	0.8167	0.8169
0.4272	13.26	2400	0.4300	0.8168	0.8173
0.4274	14.36	2600	0.4305	0.8138	0.8141
0.4282	15.47	2800	0.4323	0.8138	0.8148
0.4228	16.57	3000	0.4306	0.8184	0.8183
0.4252	17.68	3200	0.4266	0.8161	0.8162
0.4178	18.78	3400	0.4305	0.8170	0.8176
0.4178	19.89	3600	0.4257	0.8165	0.8169
0.418	20.99	3800	0.4340	0.8162	0.8169
0.4157	22.1	4000	0.4258	0.8159	0.8166
0.4149	23.2	4200	0.4268	0.8176	0.8180
0.4158	24.31	4400	0.4398	0.8099	0.8117
0.4075	25.41	4600	0.4275	0.8176	0.8180
0.4134	26.52	4800	0.4275	0.8130	0.8135
0.4144	27.62	5000	0.4281	0.8160	0.8169
0.406	28.73	5200	0.4276	0.8117	0.8124
0.4059	29.83	5400	0.4247	0.8151	0.8155
0.4056	30.94	5600	0.4279	0.8113	0.8117
0.4063	32.04	5800	0.4252	0.8153	0.8155
0.4043	33.15	6000	0.4262	0.8153	0.8159
0.4057	34.25	6200	0.4269	0.8130	0.8135
0.4015	35.36	6400	0.4291	0.8160	0.8166
0.4067	36.46	6600	0.4256	0.8128	0.8131
0.4048	37.57	6800	0.4282	0.8121	0.8128
0.4019	38.67	7000	0.4293	0.8166	0.8173
0.4011	39.78	7200	0.4268	0.8141	0.8145
0.4029	40.88	7400	0.4259	0.8148	0.8152
0.402	41.99	7600	0.4275	0.8130	0.8135
0.3994	43.09	7800	0.4304	0.8147	0.8152
0.3999	44.2	8000	0.4306	0.8117	0.8124
0.3982	45.3	8200	0.4267	0.8145	0.8148
0.3986	46.41	8400	0.4283	0.8130	0.8135
0.4009	47.51	8600	0.4303	0.8151	0.8159
0.3988	48.62	8800	0.4304	0.8117	0.8124
0.3943	49.72	9000	0.4295	0.8115	0.8121
0.4009	50.83	9200	0.4309	0.8133	0.8141
0.3995	51.93	9400	0.4287	0.8111	0.8117
0.3994	53.04	9600	0.4285	0.8118	0.8124
0.396	54.14	9800	0.4284	0.8122	0.8128
0.3996	55.25	10000	0.4297	0.8121	0.8128

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2