GUE_EMP_H3K36me3-seqsight_65536_512_94M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_94M on the mahdibaghbanzadeh/GUE_EMP_H3K36me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4394
• F1 Score: 0.8107
• Accuracy: 0.8119

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5346	0.92	200	0.5123	0.7534	0.7563
0.4904	1.83	400	0.4997	0.7653	0.7681
0.4745	2.75	600	0.4834	0.7773	0.7790
0.4767	3.67	800	0.4741	0.7883	0.7896
0.4599	4.59	1000	0.4719	0.7900	0.7916
0.4554	5.5	1200	0.4672	0.7881	0.7901
0.4565	6.42	1400	0.4645	0.7970	0.7979
0.4494	7.34	1600	0.4739	0.7931	0.7953
0.4452	8.26	1800	0.4655	0.7948	0.7964
0.4468	9.17	2000	0.4598	0.7966	0.7985
0.4443	10.09	2200	0.4803	0.7826	0.7864
0.4412	11.01	2400	0.4631	0.7928	0.7953
0.44	11.93	2600	0.4517	0.8031	0.8042
0.4367	12.84	2800	0.4557	0.8011	0.8030
0.4348	13.76	3000	0.4677	0.7944	0.7970
0.432	14.68	3200	0.4516	0.8045	0.8056
0.4325	15.6	3400	0.4519	0.8030	0.8048
0.4331	16.51	3600	0.4589	0.8027	0.8042
0.4323	17.43	3800	0.4457	0.8084	0.8093
0.4281	18.35	4000	0.4562	0.8034	0.8048
0.428	19.27	4200	0.4584	0.8023	0.8039
0.4286	20.18	4400	0.4656	0.8007	0.8028
0.4237	21.1	4600	0.4550	0.8056	0.8068
0.426	22.02	4800	0.4555	0.8050	0.8062
0.425	22.94	5000	0.4514	0.8041	0.8053
0.4241	23.85	5200	0.4488	0.8070	0.8082
0.4187	24.77	5400	0.4639	0.8024	0.8045
0.4231	25.69	5600	0.4555	0.8038	0.8056
0.4196	26.61	5800	0.4512	0.8051	0.8062
0.4201	27.52	6000	0.4649	0.8036	0.8053
0.4184	28.44	6200	0.4562	0.8049	0.8062
0.4173	29.36	6400	0.4535	0.8032	0.8048
0.4208	30.28	6600	0.4532	0.8047	0.8065
0.417	31.19	6800	0.4523	0.8063	0.8073
0.4172	32.11	7000	0.4580	0.8034	0.8053
0.4185	33.03	7200	0.4561	0.8035	0.8050
0.4159	33.94	7400	0.4559	0.8061	0.8073
0.4153	34.86	7600	0.4532	0.8045	0.8059
0.4151	35.78	7800	0.4537	0.8058	0.8071
0.4139	36.7	8000	0.4520	0.8048	0.8062
0.4148	37.61	8200	0.4564	0.8039	0.8059
0.4168	38.53	8400	0.4523	0.8046	0.8059
0.4137	39.45	8600	0.4546	0.8035	0.8050
0.4118	40.37	8800	0.4612	0.8046	0.8065
0.4166	41.28	9000	0.4511	0.8044	0.8059
0.4095	42.2	9200	0.4535	0.8059	0.8073
0.4089	43.12	9400	0.4556	0.8053	0.8068
0.4144	44.04	9600	0.4528	0.8050	0.8065
0.4128	44.95	9800	0.4548	0.8049	0.8065
0.4104	45.87	10000	0.4543	0.8044	0.8059

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2