gena-lm-bert-base-t2t_ft_BioS2_1kbpHG19_DHSs_H3K27AC

This model is a fine-tuned version of AIRI-Institute/gena-lm-bert-base-t2t on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5515
• F1 Score: 0.8104
• Precision: 0.8830
• Recall: 0.7488
• Accuracy: 0.8168
• Auc: 0.9083
• Prc: 0.8965

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Precision	Recall	Accuracy	Auc	Prc
0.6889	0.0839	500	0.6417	0.7291	0.7593	0.7013	0.7277	0.7941	0.7841
0.6164	0.1679	1000	0.5381	0.7867	0.7507	0.8262	0.7658	0.8316	0.8141
0.5271	0.2518	1500	0.5026	0.8036	0.7353	0.8860	0.7737	0.8393	0.8118
0.5221	0.3357	2000	0.4936	0.8063	0.7676	0.8490	0.7868	0.8466	0.8365
0.4903	0.4197	2500	0.4795	0.8185	0.7613	0.8850	0.7948	0.8621	0.8476
0.4543	0.5036	3000	0.4643	0.8251	0.7756	0.8815	0.8047	0.8644	0.8371
0.4603	0.5875	3500	0.4686	0.8273	0.7670	0.8978	0.8041	0.8766	0.8565
0.4561	0.6715	4000	0.4631	0.8265	0.7954	0.8603	0.8113	0.8828	0.8755
0.4477	0.7554	4500	0.4512	0.8306	0.7798	0.8885	0.8106	0.8800	0.8707
0.4514	0.8393	5000	0.4408	0.8235	0.8139	0.8333	0.8133	0.8820	0.8795
0.4559	0.9233	5500	0.4425	0.8316	0.8087	0.8558	0.8188	0.8900	0.8760
0.4467	1.0072	6000	0.4247	0.8223	0.8348	0.8102	0.8170	0.8940	0.8817
0.4283	1.0912	6500	0.5164	0.8296	0.8175	0.8420	0.8192	0.8773	0.8659
0.4361	1.1751	7000	0.4595	0.8387	0.8072	0.8728	0.8245	0.8951	0.8886
0.443	1.2590	7500	0.4806	0.8396	0.7957	0.8885	0.8225	0.8809	0.8557
0.42	1.3430	8000	0.4259	0.8297	0.8202	0.8394	0.8198	0.8988	0.8939
0.4233	1.4269	8500	0.4146	0.8225	0.8404	0.8053	0.8183	0.9047	0.8980
0.4175	1.5108	9000	0.4482	0.8420	0.7790	0.9162	0.8203	0.8960	0.8710
0.4078	1.5948	9500	0.4259	0.8426	0.8192	0.8673	0.8306	0.9049	0.8949
0.4071	1.6787	10000	0.4400	0.8432	0.8106	0.8786	0.8292	0.8929	0.8537
0.4279	1.7626	10500	0.4670	0.8368	0.8443	0.8294	0.8309	0.8996	0.8766
0.4278	1.8466	11000	0.4540	0.8468	0.7851	0.9190	0.8262	0.8796	0.8380
0.419	1.9305	11500	0.4370	0.8502	0.8095	0.8953	0.8351	0.8920	0.8528
0.4077	2.0144	12000	0.4640	0.8499	0.7902	0.9194	0.8303	0.8821	0.8342
0.3942	2.0984	12500	0.4793	0.8488	0.8334	0.8648	0.8390	0.9079	0.8876
0.4104	2.1823	13000	0.4884	0.8460	0.8504	0.8416	0.8398	0.8969	0.8530
0.3959	2.2662	13500	0.5016	0.8484	0.8428	0.8542	0.8405	0.9054	0.8851
0.4071	2.3502	14000	0.5515	0.8104	0.8830	0.7488	0.8168	0.9083	0.8965

Framework versions

• Transformers 4.42.3
• Pytorch 2.3.0+cu121
• Datasets 2.18.0
• Tokenizers 0.19.0