GUE_EMP_H3K9ac-seqsight_32768_512_43M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_EMP_H3K9ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.4802
• F1 Score: 0.7833
• Accuracy: 0.7827

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6197	1.15	200	0.5705	0.7183	0.7179
0.5503	2.3	400	0.5731	0.7118	0.7125
0.5252	3.45	600	0.5792	0.7139	0.7157
0.5201	4.6	800	0.5674	0.7232	0.7240
0.5124	5.75	1000	0.5417	0.7324	0.7319
0.5082	6.9	1200	0.5598	0.7310	0.7308
0.5026	8.05	1400	0.5465	0.7388	0.7384
0.5014	9.2	1600	0.5725	0.7203	0.7226
0.4945	10.34	1800	0.5384	0.7429	0.7424
0.4922	11.49	2000	0.5424	0.7436	0.7434
0.4867	12.64	2200	0.5651	0.7278	0.7294
0.4894	13.79	2400	0.5483	0.7323	0.7334
0.4871	14.94	2600	0.5391	0.7400	0.7402
0.4809	16.09	2800	0.5321	0.7439	0.7438
0.4791	17.24	3000	0.5445	0.7382	0.7384
0.4785	18.39	3200	0.5470	0.7407	0.7416
0.4804	19.54	3400	0.5253	0.7463	0.7463
0.4729	20.69	3600	0.5203	0.7514	0.7510
0.4743	21.84	3800	0.5228	0.7468	0.7470
0.4701	22.99	4000	0.5275	0.7437	0.7442
0.4734	24.14	4200	0.5078	0.7547	0.7542
0.4626	25.29	4400	0.5260	0.7533	0.7531
0.4698	26.44	4600	0.5283	0.7494	0.7496
0.4677	27.59	4800	0.5292	0.7437	0.7445
0.4641	28.74	5000	0.5166	0.7538	0.7539
0.47	29.89	5200	0.5211	0.7492	0.7492
0.4622	31.03	5400	0.5256	0.7467	0.7474
0.4644	32.18	5600	0.5069	0.7594	0.7589
0.4554	33.33	5800	0.5209	0.7527	0.7528
0.4678	34.48	6000	0.5253	0.7440	0.7449
0.4559	35.63	6200	0.5153	0.7511	0.7510
0.4638	36.78	6400	0.5167	0.7497	0.7499
0.4579	37.93	6600	0.5228	0.7478	0.7481
0.4589	39.08	6800	0.5101	0.7548	0.7546
0.4589	40.23	7000	0.5161	0.7516	0.7517
0.4573	41.38	7200	0.5168	0.7512	0.7513
0.457	42.53	7400	0.5161	0.7534	0.7535
0.4565	43.68	7600	0.5145	0.7564	0.7564
0.4535	44.83	7800	0.5226	0.7500	0.7506
0.4568	45.98	8000	0.5133	0.7541	0.7542
0.4581	47.13	8200	0.5187	0.7503	0.7506
0.4531	48.28	8400	0.5167	0.7520	0.7521
0.4507	49.43	8600	0.5164	0.7519	0.7521
0.4548	50.57	8800	0.5161	0.7528	0.7528
0.4545	51.72	9000	0.5210	0.7469	0.7474
0.4486	52.87	9200	0.5196	0.7488	0.7492
0.4547	54.02	9400	0.5173	0.7503	0.7506
0.4513	55.17	9600	0.5190	0.7485	0.7488
0.4511	56.32	9800	0.5142	0.7527	0.7528
0.4546	57.47	10000	0.5164	0.7504	0.7506

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2