hyenadna-medium-160k-seqlen-hf_ft_BioS2_1kbpHG19_DHSs_H3K27AC

This model is a fine-tuned version of LongSafari/hyenadna-medium-160k-seqlen-hf on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4893
• F1 Score: 0.8260
• Precision: 0.7676
• Recall: 0.8941
• Accuracy: 0.8037
• Auc: 0.8850
• Prc: 0.8777

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Precision	Recall	Accuracy	Auc	Prc
0.554	0.1678	500	0.5523	0.6784	0.7894	0.5948	0.7060	0.8144	0.7936
0.5267	0.3356	1000	0.4857	0.7992	0.7245	0.8912	0.7666	0.8409	0.8211
0.4995	0.5034	1500	0.5247	0.7956	0.6810	0.9565	0.7438	0.8443	0.8191
0.4755	0.6711	2000	0.4548	0.8058	0.7793	0.8342	0.7904	0.8619	0.8468
0.4578	0.8389	2500	0.4798	0.7471	0.8350	0.6759	0.7614	0.8664	0.8614
0.4543	1.0067	3000	0.4430	0.8082	0.7822	0.8359	0.7931	0.8743	0.8660
0.44	1.1745	3500	0.4306	0.8168	0.7751	0.8632	0.7982	0.8786	0.8726
0.4235	1.3423	4000	0.4809	0.8190	0.7707	0.8738	0.7987	0.8783	0.8689
0.4178	1.5101	4500	0.4363	0.8117	0.8076	0.8159	0.8027	0.8823	0.8760
0.4178	1.6779	5000	0.4392	0.8203	0.7734	0.8732	0.8005	0.8824	0.8767
0.4178	1.8456	5500	0.4148	0.8200	0.8077	0.8326	0.8094	0.8899	0.8853
0.4134	2.0134	6000	0.4275	0.8148	0.8169	0.8127	0.8074	0.8882	0.8821
0.3808	2.1812	6500	0.4376	0.8149	0.7910	0.8404	0.8010	0.8847	0.8811
0.3854	2.3490	7000	0.4326	0.8241	0.7560	0.9057	0.7985	0.8885	0.8845
0.374	2.5168	7500	0.4445	0.8198	0.8016	0.8388	0.8077	0.8870	0.8828
0.3844	2.6846	8000	0.4454	0.8251	0.7468	0.9218	0.7963	0.8894	0.8838
0.3817	2.8523	8500	0.4368	0.8115	0.8159	0.8072	0.8045	0.8880	0.8833
0.3838	3.0201	9000	0.4934	0.7985	0.8372	0.7631	0.7992	0.8887	0.8811
0.3444	3.1879	9500	0.4439	0.8257	0.7778	0.8799	0.8064	0.8856	0.8803
0.3336	3.3557	10000	0.4541	0.8128	0.8179	0.8079	0.8060	0.8888	0.8856
0.3524	3.5235	10500	0.4765	0.8266	0.7449	0.9285	0.7970	0.8879	0.8811
0.3532	3.6913	11000	0.4351	0.8259	0.7786	0.8793	0.8067	0.8848	0.8791
0.3342	3.8591	11500	0.4457	0.8227	0.8040	0.8423	0.8107	0.8878	0.8826
0.342	4.0268	12000	0.4544	0.8153	0.7777	0.8568	0.7977	0.8805	0.8792
0.2962	4.1946	12500	0.5764	0.7800	0.8422	0.7264	0.7864	0.8819	0.8804
0.3026	4.3624	13000	0.4893	0.8260	0.7676	0.8941	0.8037	0.8850	0.8777

Framework versions

• Transformers 4.42.3
• Pytorch 2.3.0+cu121
• Datasets 2.18.0
• Tokenizers 0.19.0