GUE_EMP_H3K4me2-seqsight_4096_512_46M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_46M on the mahdibaghbanzadeh/GUE_EMP_H3K4me2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5833
• F1 Score: 0.6924
• Accuracy: 0.6966

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6394	1.04	200	0.6318	0.5864	0.6422
0.6149	2.08	400	0.6151	0.6606	0.6631
0.6069	3.12	600	0.6077	0.6453	0.6693
0.6023	4.17	800	0.6002	0.6715	0.6748
0.5966	5.21	1000	0.6032	0.6610	0.6725
0.5931	6.25	1200	0.5932	0.6712	0.6833
0.5879	7.29	1400	0.5943	0.6770	0.6781
0.5854	8.33	1600	0.5951	0.6791	0.6813
0.5836	9.38	1800	0.5983	0.6824	0.6810
0.5814	10.42	2000	0.5874	0.6765	0.6846
0.582	11.46	2200	0.5864	0.6747	0.6882
0.572	12.5	2400	0.5876	0.6792	0.6862
0.5725	13.54	2600	0.5870	0.6763	0.6859
0.5749	14.58	2800	0.5877	0.6759	0.6872
0.5739	15.62	3000	0.5879	0.6828	0.6878
0.5675	16.67	3200	0.5866	0.6881	0.6914
0.5688	17.71	3400	0.5848	0.6846	0.6905
0.5664	18.75	3600	0.5884	0.6729	0.6852
0.5685	19.79	3800	0.5850	0.6848	0.6901
0.5622	20.83	4000	0.5844	0.6847	0.6882
0.5624	21.88	4200	0.5881	0.6838	0.6849
0.5596	22.92	4400	0.5862	0.6853	0.6891
0.5617	23.96	4600	0.5843	0.6885	0.6898
0.5599	25.0	4800	0.5830	0.6886	0.6940
0.5584	26.04	5000	0.5874	0.6839	0.6859
0.5565	27.08	5200	0.5853	0.6832	0.6843
0.5557	28.12	5400	0.5837	0.6852	0.6898
0.554	29.17	5600	0.5870	0.6819	0.6852
0.5543	30.21	5800	0.5883	0.6856	0.6885
0.5509	31.25	6000	0.5886	0.6846	0.6869
0.5536	32.29	6200	0.5829	0.6862	0.6918
0.5529	33.33	6400	0.5874	0.6859	0.6882
0.5495	34.38	6600	0.5847	0.6912	0.6957
0.5501	35.42	6800	0.5840	0.6881	0.6927
0.5507	36.46	7000	0.5848	0.6868	0.6924
0.5482	37.5	7200	0.5848	0.6889	0.6924
0.5465	38.54	7400	0.5842	0.6874	0.6918
0.5518	39.58	7600	0.5826	0.6908	0.6960
0.5495	40.62	7800	0.5877	0.6859	0.6862
0.5457	41.67	8000	0.5833	0.6893	0.6921
0.544	42.71	8200	0.5836	0.6927	0.6976
0.548	43.75	8400	0.5827	0.6925	0.6960
0.5431	44.79	8600	0.5845	0.6912	0.6931
0.542	45.83	8800	0.5829	0.6928	0.6979
0.5444	46.88	9000	0.5848	0.6899	0.6918
0.5439	47.92	9200	0.5845	0.6898	0.6921
0.5454	48.96	9400	0.5835	0.6933	0.6970
0.5424	50.0	9600	0.5845	0.6906	0.6931
0.5423	51.04	9800	0.5845	0.6899	0.6927
0.5406	52.08	10000	0.5845	0.6905	0.6934

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2