GUE_EMP_H3K4me2-seqsight_4096_512_27M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_EMP_H3K4me2 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5898
• F1 Score: 0.6921
• Accuracy: 0.6934

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6502	1.04	200	0.6228	0.6132	0.6497
0.6176	2.08	400	0.6153	0.6635	0.6654
0.6091	3.12	600	0.6141	0.6396	0.6657
0.6077	4.17	800	0.6082	0.6542	0.6650
0.604	5.21	1000	0.6069	0.6645	0.6699
0.6024	6.25	1200	0.6063	0.6576	0.6657
0.5951	7.29	1400	0.6088	0.6620	0.6631
0.5958	8.33	1600	0.6107	0.6644	0.6676
0.5946	9.38	1800	0.6171	0.6619	0.6595
0.5922	10.42	2000	0.6048	0.6718	0.6742
0.5938	11.46	2200	0.6004	0.6701	0.6777
0.5872	12.5	2400	0.6023	0.6703	0.6751
0.5858	13.54	2600	0.6018	0.6688	0.6745
0.5867	14.58	2800	0.6038	0.6652	0.6729
0.5896	15.62	3000	0.6037	0.6729	0.6777
0.5836	16.67	3200	0.6056	0.6686	0.6693
0.5829	17.71	3400	0.6005	0.6724	0.6771
0.5826	18.75	3600	0.6013	0.6693	0.6751
0.5857	19.79	3800	0.5976	0.6772	0.6813
0.5773	20.83	4000	0.6037	0.6721	0.6729
0.5821	21.88	4200	0.6042	0.6738	0.6735
0.5801	22.92	4400	0.6021	0.6698	0.6719
0.5807	23.96	4600	0.6018	0.6684	0.6689
0.578	25.0	4800	0.5986	0.6762	0.6790
0.5754	26.04	5000	0.6005	0.6795	0.6810
0.5755	27.08	5200	0.6008	0.6698	0.6709
0.5752	28.12	5400	0.6007	0.6711	0.6719
0.5734	29.17	5600	0.6028	0.6764	0.6768
0.5715	30.21	5800	0.6040	0.6744	0.6745
0.5732	31.25	6000	0.6013	0.6740	0.6751
0.5715	32.29	6200	0.5981	0.6736	0.6771
0.5722	33.33	6400	0.6014	0.6723	0.6732
0.5721	34.38	6600	0.5959	0.6747	0.6787
0.5679	35.42	6800	0.5997	0.6746	0.6774
0.5705	36.46	7000	0.5979	0.6760	0.6790
0.5672	37.5	7200	0.5994	0.6788	0.6800
0.5659	38.54	7400	0.5986	0.6751	0.6777
0.5707	39.58	7600	0.5981	0.6697	0.6738
0.5708	40.62	7800	0.6034	0.6691	0.6686
0.5671	41.67	8000	0.5993	0.6756	0.6768
0.5645	42.71	8200	0.5973	0.6774	0.6820
0.5685	43.75	8400	0.5986	0.6717	0.6742
0.5659	44.79	8600	0.6003	0.6733	0.6742
0.5643	45.83	8800	0.5976	0.6754	0.6784
0.5668	46.88	9000	0.6026	0.6722	0.6722
0.5644	47.92	9200	0.6013	0.6753	0.6761
0.5645	48.96	9400	0.5995	0.6738	0.6758
0.5636	50.0	9600	0.6008	0.6735	0.6748
0.5666	51.04	9800	0.5999	0.6726	0.6742
0.562	52.08	10000	0.5998	0.6736	0.6751

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2