GUE_EMP_H3K4me3-seqsight_16384_512_56M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3K4me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5803
• F1 Score: 0.6958
• Accuracy: 0.6962

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6495	0.87	200	0.6251	0.6533	0.6546
0.6245	1.74	400	0.6136	0.6606	0.6603
0.6133	2.61	600	0.6028	0.6698	0.6696
0.6036	3.48	800	0.5988	0.6740	0.6739
0.5992	4.35	1000	0.5990	0.6716	0.6717
0.5932	5.22	1200	0.5979	0.6684	0.6704
0.5904	6.09	1400	0.6170	0.6531	0.6598
0.5855	6.96	1600	0.5982	0.6715	0.6728
0.5823	7.83	1800	0.5914	0.6750	0.6747
0.5822	8.7	2000	0.5944	0.6728	0.6731
0.5776	9.57	2200	0.5857	0.6815	0.6813
0.5782	10.43	2400	0.5919	0.6794	0.6807
0.5738	11.3	2600	0.5848	0.6793	0.6807
0.5775	12.17	2800	0.5838	0.6824	0.6826
0.574	13.04	3000	0.5863	0.6777	0.6780
0.5706	13.91	3200	0.5819	0.6848	0.6851
0.5682	14.78	3400	0.5903	0.6730	0.6753
0.5686	15.65	3600	0.5853	0.6833	0.6842
0.5688	16.52	3800	0.5854	0.6798	0.6802
0.565	17.39	4000	0.5885	0.6834	0.6842
0.5676	18.26	4200	0.5839	0.6875	0.6880
0.5633	19.13	4400	0.5891	0.6838	0.6837
0.5633	20.0	4600	0.5894	0.6824	0.6837
0.5635	20.87	4800	0.5853	0.6881	0.6886
0.5612	21.74	5000	0.5876	0.6830	0.6840
0.5616	22.61	5200	0.5826	0.6879	0.6883
0.5609	23.48	5400	0.5954	0.6762	0.6802
0.5588	24.35	5600	0.5846	0.6876	0.6883
0.5608	25.22	5800	0.5918	0.6831	0.6861
0.555	26.09	6000	0.5926	0.6805	0.6829
0.5598	26.96	6200	0.5937	0.6812	0.6845
0.5559	27.83	6400	0.5982	0.6811	0.6853
0.5572	28.7	6600	0.5832	0.6869	0.6875
0.5538	29.57	6800	0.5808	0.6892	0.6899
0.5524	30.43	7000	0.5905	0.6841	0.6867
0.5589	31.3	7200	0.5872	0.6862	0.6883
0.5546	32.17	7400	0.5859	0.6849	0.6867
0.554	33.04	7600	0.5824	0.6875	0.6883
0.553	33.91	7800	0.5832	0.6861	0.6872
0.5554	34.78	8000	0.5845	0.6885	0.6897
0.5508	35.65	8200	0.5826	0.6879	0.6889
0.552	36.52	8400	0.5838	0.6890	0.6902
0.5521	37.39	8600	0.5829	0.6895	0.6902
0.5482	38.26	8800	0.5892	0.6860	0.6880
0.5518	39.13	9000	0.5868	0.6884	0.6902
0.5496	40.0	9200	0.5825	0.6890	0.6897
0.5477	40.87	9400	0.5829	0.6902	0.6908
0.5498	41.74	9600	0.5841	0.6865	0.6875
0.5556	42.61	9800	0.5824	0.6879	0.6889
0.5468	43.48	10000	0.5833	0.6873	0.6883

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2