GUE_EMP_H3K4me3-seqsight_4096_512_15M-L8

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_15M on the mahdibaghbanzadeh/GUE_EMP_H3K4me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.6830
• F1 Score: 0.5741
• Accuracy: 0.5739

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 20000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.693	1.74	400	0.6866	0.5442	0.5571
0.6769	3.48	800	0.6826	0.5403	0.5677
0.67	5.22	1200	0.6875	0.5634	0.5630
0.664	6.96	1600	0.6841	0.5596	0.5620
0.6587	8.7	2000	0.6848	0.5460	0.5617
0.6544	10.43	2400	0.6903	0.5609	0.5630
0.654	12.17	2800	0.6898	0.5587	0.5647
0.6516	13.91	3200	0.6895	0.5600	0.5620
0.6464	15.65	3600	0.6904	0.5609	0.5625
0.6442	17.39	4000	0.6976	0.5554	0.5606
0.6439	19.13	4400	0.6953	0.5539	0.5636
0.6417	20.87	4800	0.6915	0.5578	0.5601
0.6382	22.61	5200	0.6916	0.5595	0.5617
0.638	24.35	5600	0.7019	0.5507	0.5611
0.6388	26.09	6000	0.7075	0.5515	0.5514
0.6366	27.83	6400	0.7027	0.5512	0.5508
0.6346	29.57	6800	0.7084	0.5489	0.5524
0.6329	31.3	7200	0.7128	0.5596	0.5606
0.6324	33.04	7600	0.7076	0.5564	0.5582
0.6272	34.78	8000	0.6934	0.5508	0.5560
0.6283	36.52	8400	0.7111	0.5500	0.55
0.6283	38.26	8800	0.7116	0.5529	0.5524
0.6253	40.0	9200	0.7220	0.5561	0.5557
0.6256	41.74	9600	0.7084	0.5583	0.5587
0.6211	43.48	10000	0.7032	0.5564	0.5576
0.6196	45.22	10400	0.7046	0.5561	0.5611
0.6194	46.96	10800	0.7151	0.5554	0.5554
0.6167	48.7	11200	0.7196	0.5407	0.5437
0.618	50.43	11600	0.7181	0.5615	0.5617
0.6157	52.17	12000	0.7230	0.5628	0.5636
0.6138	53.91	12400	0.7148	0.5582	0.5582
0.6133	55.65	12800	0.7145	0.5554	0.5552
0.6118	57.39	13200	0.7284	0.5556	0.5552
0.6099	59.13	13600	0.7165	0.5601	0.5601
0.611	60.87	14000	0.7159	0.5593	0.5595
0.6088	62.61	14400	0.7160	0.5581	0.5590
0.6061	64.35	14800	0.7206	0.5572	0.5568
0.6065	66.09	15200	0.7177	0.5616	0.5620
0.6058	67.83	15600	0.7263	0.5579	0.5614
0.6059	69.57	16000	0.7219	0.5586	0.5584
0.6046	71.3	16400	0.7240	0.5575	0.5571
0.6046	73.04	16800	0.7272	0.5582	0.5579
0.6003	74.78	17200	0.7207	0.5555	0.5552
0.6027	76.52	17600	0.7220	0.5621	0.5633
0.6015	78.26	18000	0.7220	0.5621	0.5620
0.599	80.0	18400	0.7293	0.5553	0.5549
0.5987	81.74	18800	0.7297	0.5556	0.5552
0.6001	83.48	19200	0.7244	0.5576	0.5576
0.6011	85.22	19600	0.7253	0.5569	0.5565
0.5982	86.96	20000	0.7262	0.5566	0.5563

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2