GUE_EMP_H3K4me3-seqsight_16384_512_56M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3K4me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.7298
• F1 Score: 0.7016
• Accuracy: 0.7014

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6389	0.87	200	0.6063	0.6685	0.6682
0.6006	1.74	400	0.6004	0.6810	0.6823
0.5855	2.61	600	0.5854	0.6826	0.6834
0.5775	3.48	800	0.5817	0.6907	0.6905
0.5686	4.35	1000	0.5836	0.6871	0.6870
0.5597	5.22	1200	0.5855	0.6867	0.6878
0.555	6.09	1400	0.5977	0.6832	0.6856
0.5465	6.96	1600	0.5786	0.7001	0.7
0.5366	7.83	1800	0.5892	0.6940	0.6937
0.5307	8.7	2000	0.5852	0.6975	0.6973
0.5226	9.57	2200	0.5890	0.6929	0.6940
0.5192	10.43	2400	0.6053	0.6946	0.6962
0.5129	11.3	2600	0.5802	0.6979	0.6984
0.5075	12.17	2800	0.6029	0.6850	0.6856
0.4983	13.04	3000	0.5983	0.6980	0.6989
0.4894	13.91	3200	0.5995	0.6991	0.6992
0.4812	14.78	3400	0.6421	0.6874	0.6889
0.4747	15.65	3600	0.6179	0.6899	0.6929
0.4691	16.52	3800	0.6068	0.6935	0.6943
0.4593	17.39	4000	0.6400	0.6920	0.6924
0.458	18.26	4200	0.6236	0.6997	0.7014
0.4482	19.13	4400	0.6311	0.6921	0.6921
0.4433	20.0	4600	0.6343	0.6947	0.6951
0.4326	20.87	4800	0.6531	0.6964	0.6965
0.4294	21.74	5000	0.6335	0.6938	0.6937
0.425	22.61	5200	0.6397	0.6950	0.6954
0.4206	23.48	5400	0.6499	0.6965	0.6970
0.4128	24.35	5600	0.6704	0.7029	0.7038
0.4089	25.22	5800	0.6735	0.6975	0.6973
0.4042	26.09	6000	0.6734	0.7021	0.7027
0.4003	26.96	6200	0.6617	0.6964	0.6976
0.3907	27.83	6400	0.6731	0.6968	0.6976
0.3843	28.7	6600	0.6912	0.6900	0.6899
0.3804	29.57	6800	0.6820	0.6957	0.6954
0.3831	30.43	7000	0.6843	0.6929	0.6927
0.3766	31.3	7200	0.6948	0.7019	0.7019
0.3749	32.17	7400	0.6839	0.6965	0.6965
0.3661	33.04	7600	0.6864	0.6994	0.6997
0.3648	33.91	7800	0.6997	0.6982	0.6984
0.3635	34.78	8000	0.7016	0.6964	0.6962
0.3593	35.65	8200	0.7018	0.6965	0.6962
0.3513	36.52	8400	0.7165	0.6962	0.6959
0.3509	37.39	8600	0.7196	0.7045	0.7043
0.3461	38.26	8800	0.7234	0.7018	0.7016
0.349	39.13	9000	0.7181	0.6974	0.6973
0.3445	40.0	9200	0.7203	0.6981	0.6978
0.3464	40.87	9400	0.7161	0.6948	0.6946
0.3407	41.74	9600	0.7187	0.6967	0.6965
0.343	42.61	9800	0.7229	0.6978	0.6976
0.3365	43.48	10000	0.7276	0.6959	0.6957

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2