nucleotide-transformer-2.5b-1000g_ft_Hepg2_1kbpHG19_DHSs_H3K27AC

This model is a fine-tuned version of InstaDeepAI/nucleotide-transformer-2.5b-1000g on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2831
• F1 Score: 0.8768
• Precision: 0.8600
• Recall: 0.8943
• Accuracy: 0.8661

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Precision	Recall	Accuracy
0.45	0.1864	500	0.4118	0.8434	0.8428	0.8439	0.8329
0.3862	0.3729	1000	0.3639	0.8621	0.8498	0.8747	0.8508
0.3975	0.5593	1500	0.3904	0.8690	0.8097	0.9377	0.8493
0.3681	0.7457	2000	0.3590	0.8738	0.8395	0.9111	0.8598
0.3567	0.9321	2500	0.4015	0.8733	0.8063	0.9524	0.8527
0.2808	1.1186	3000	0.6744	0.8251	0.9155	0.7509	0.8303
0.2164	1.3050	3500	0.5770	0.8747	0.8233	0.9328	0.8575
0.2104	1.4914	4000	0.5941	0.8817	0.8416	0.9258	0.8676
0.2115	1.6779	4500	0.5546	0.8755	0.8542	0.8978	0.8639
0.2075	1.8643	5000	0.5431	0.8701	0.8754	0.8649	0.8624
0.176	2.0507	5500	0.9443	0.8696	0.8666	0.8726	0.8605
0.0508	2.2371	6000	0.8301	0.8777	0.8743	0.8810	0.8691
0.0488	2.4236	6500	0.8394	0.8783	0.8584	0.8992	0.8672
0.0567	2.6100	7000	1.1510	0.8785	0.8122	0.9566	0.8590
0.0588	2.7964	7500	0.8502	0.8568	0.8912	0.8251	0.8530
0.0541	2.9828	8000	1.4982	0.8013	0.9155	0.7124	0.8116
0.0119	3.1693	8500	1.2710	0.8699	0.8838	0.8565	0.8635
0.019	3.3557	9000	1.1534	0.8753	0.8663	0.8845	0.8657
0.0267	3.5421	9500	1.3990	0.8719	0.7909	0.9713	0.8478
0.0318	3.7286	10000	1.3099	0.8697	0.8444	0.8964	0.8568
0.0562	3.9150	10500	0.8772	0.8662	0.8739	0.8586	0.8586
0.0231	4.1014	11000	1.1277	0.8851	0.8398	0.9356	0.8706
0.0281	4.2878	11500	1.0566	0.8828	0.8442	0.9251	0.8691
0.0149	4.4743	12000	1.2550	0.8705	0.8485	0.8936	0.8583
0.0278	4.6607	12500	1.1960	0.8279	0.8984	0.7677	0.8299
0.0265	4.8471	13000	1.2293	0.8588	0.8826	0.8362	0.8534
0.0319	5.0336	13500	1.1519	0.8838	0.8663	0.9020	0.8736
0.0226	5.2200	14000	1.1934	0.8803	0.8609	0.9006	0.8695
0.0127	5.4064	14500	1.3537	0.8854	0.8404	0.9356	0.8709
0.0299	5.5928	15000	1.1620	0.8514	0.8926	0.8139	0.8486
0.0259	5.7793	15500	1.2213	0.8827	0.8360	0.9349	0.8676
0.0236	5.9657	16000	1.1988	0.8777	0.8552	0.9013	0.8661
0.013	6.1521	16500	1.4039	0.8788	0.8511	0.9083	0.8665
0.0174	6.3386	17000	1.1774	0.8787	0.8565	0.9020	0.8672
0.0197	6.5250	17500	1.3383	0.8732	0.8536	0.8936	0.8616
0.0185	6.7114	18000	1.1582	0.8679	0.8694	0.8663	0.8594
0.0136	6.8978	18500	1.3086	0.8532	0.8923	0.8174	0.8501
0.0072	7.0843	19000	1.2864	0.8733	0.8643	0.8824	0.8635
0.0031	7.2707	19500	1.4482	0.8729	0.8616	0.8845	0.8627
0.0202	7.4571	20000	1.4021	0.8721	0.8573	0.8873	0.8612
0.021	7.6435	20500	1.2587	0.8807	0.8493	0.9146	0.8680
0.0097	7.8300	21000	1.2831	0.8768	0.8600	0.8943	0.8661

Framework versions

• Transformers 4.42.3
• Pytorch 2.3.0+cu121
• Datasets 2.18.0
• Tokenizers 0.19.0