GUE_EMP_H3K79me3-seqsight_16384_512_56M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3K79me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4254
• F1 Score: 0.8273
• Accuracy: 0.8277

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.4821	1.1	200	0.4463	0.8110	0.8110
0.4456	2.21	400	0.4363	0.8128	0.8135
0.4377	3.31	600	0.4392	0.8029	0.8048
0.4231	4.42	800	0.4418	0.8026	0.8041
0.4221	5.52	1000	0.4398	0.8081	0.8100
0.4099	6.63	1200	0.4558	0.8065	0.8089
0.4116	7.73	1400	0.4356	0.8135	0.8152
0.4011	8.84	1600	0.4595	0.8074	0.8103
0.3996	9.94	1800	0.4245	0.8146	0.8152
0.3953	11.05	2000	0.4438	0.8073	0.8079
0.3926	12.15	2200	0.4207	0.8227	0.8232
0.3855	13.26	2400	0.4189	0.8243	0.8249
0.3876	14.36	2600	0.4192	0.8281	0.8284
0.3807	15.47	2800	0.4265	0.8216	0.8225
0.3775	16.57	3000	0.4232	0.8248	0.8249
0.3745	17.68	3200	0.4212	0.8239	0.8245
0.3687	18.78	3400	0.4597	0.8051	0.8083
0.3681	19.89	3600	0.4259	0.8195	0.8207
0.364	20.99	3800	0.4339	0.8158	0.8173
0.3606	22.1	4000	0.4220	0.8201	0.8204
0.3589	23.2	4200	0.4268	0.8186	0.8193
0.3531	24.31	4400	0.4384	0.8144	0.8162
0.3495	25.41	4600	0.4317	0.8262	0.8263
0.3546	26.52	4800	0.4296	0.8186	0.8193
0.3484	27.62	5000	0.4367	0.8198	0.8214
0.3459	28.73	5200	0.4349	0.8184	0.8197
0.3405	29.83	5400	0.4344	0.8154	0.8162
0.3405	30.94	5600	0.4304	0.8230	0.8239
0.3381	32.04	5800	0.4300	0.8195	0.8197
0.3366	33.15	6000	0.4373	0.8240	0.8252
0.335	34.25	6200	0.4381	0.8191	0.8193
0.3281	35.36	6400	0.4550	0.8225	0.8235
0.3323	36.46	6600	0.4338	0.8224	0.8232
0.3295	37.57	6800	0.4406	0.8192	0.8204
0.3261	38.67	7000	0.4415	0.8204	0.8214
0.3243	39.78	7200	0.4425	0.8224	0.8235
0.3262	40.88	7400	0.4315	0.8198	0.8200
0.3232	41.99	7600	0.4392	0.8171	0.8183
0.3241	43.09	7800	0.4418	0.8228	0.8235
0.3202	44.2	8000	0.4426	0.8187	0.8197
0.3201	45.3	8200	0.4383	0.8210	0.8214
0.3166	46.41	8400	0.4383	0.8208	0.8214
0.3186	47.51	8600	0.4454	0.8218	0.8228
0.3102	48.62	8800	0.4445	0.8212	0.8221
0.3143	49.72	9000	0.4470	0.8209	0.8218
0.3164	50.83	9200	0.4476	0.8190	0.8204
0.3113	51.93	9400	0.4463	0.8208	0.8218
0.3099	53.04	9600	0.4432	0.8211	0.8218
0.3081	54.14	9800	0.4443	0.8208	0.8214
0.3096	55.25	10000	0.4462	0.8220	0.8228

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2