GUE_EMP_H3K4me1-seqsight_4096_512_15M-L8

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_15M on the mahdibaghbanzadeh/GUE_EMP_H3K4me1 dataset. It achieves the following results on the evaluation set:

• Loss: 0.6797
• F1 Score: 0.6072
• Accuracy: 0.6064

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 20000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6789	2.02	400	0.6693	0.5903	0.5906
0.6546	4.04	800	0.6674	0.6020	0.6051
0.6463	6.06	1200	0.6657	0.6010	0.6102
0.6398	8.08	1600	0.6719	0.6009	0.6032
0.6358	10.1	2000	0.6728	0.6018	0.6105
0.6316	12.12	2400	0.6699	0.6045	0.6054
0.6294	14.14	2800	0.6681	0.6068	0.6089
0.6261	16.16	3200	0.6706	0.5947	0.6102
0.6222	18.18	3600	0.6811	0.6039	0.6039
0.6209	20.2	4000	0.6798	0.6076	0.6105
0.6198	22.22	4400	0.6865	0.6034	0.6064
0.616	24.24	4800	0.6780	0.5998	0.6117
0.6165	26.26	5200	0.6881	0.6103	0.6098
0.6134	28.28	5600	0.6805	0.5890	0.6035
0.6136	30.3	6000	0.6824	0.6053	0.6089
0.6119	32.32	6400	0.6781	0.5980	0.6026
0.6073	34.34	6800	0.6747	0.6044	0.6089
0.6071	36.36	7200	0.6935	0.5984	0.6026
0.6027	38.38	7600	0.6971	0.5970	0.6007
0.6048	40.4	8000	0.6824	0.5934	0.6029
0.6026	42.42	8400	0.6899	0.5994	0.6054
0.6003	44.44	8800	0.6972	0.6030	0.6051
0.6007	46.46	9200	0.6875	0.6022	0.6067
0.597	48.48	9600	0.6887	0.5973	0.6023
0.5946	50.51	10000	0.7047	0.6024	0.6054
0.5926	52.53	10400	0.6904	0.6014	0.6032
0.5934	54.55	10800	0.6835	0.5995	0.6045
0.5899	56.57	11200	0.6920	0.5937	0.5988
0.5872	58.59	11600	0.7048	0.6042	0.6057
0.5877	60.61	12000	0.6956	0.6037	0.6045
0.584	62.63	12400	0.7031	0.5912	0.6007
0.5817	64.65	12800	0.6979	0.5933	0.5985
0.5818	66.67	13200	0.7000	0.5943	0.5972
0.579	68.69	13600	0.6988	0.5985	0.5991
0.5819	70.71	14000	0.7029	0.5947	0.5972
0.5781	72.73	14400	0.6969	0.5964	0.6010
0.576	74.75	14800	0.7067	0.5998	0.6020
0.5759	76.77	15200	0.7012	0.5978	0.6004
0.5755	78.79	15600	0.7093	0.6008	0.6023
0.5718	80.81	16000	0.7134	0.5966	0.5991
0.5719	82.83	16400	0.6988	0.6018	0.6042
0.5718	84.85	16800	0.7069	0.5987	0.6010
0.5727	86.87	17200	0.7072	0.6004	0.6010
0.5707	88.89	17600	0.7065	0.6004	0.6020
0.5691	90.91	18000	0.7075	0.6003	0.6020
0.5696	92.93	18400	0.7111	0.5963	0.6001
0.5666	94.95	18800	0.7099	0.5942	0.5972
0.5688	96.97	19200	0.7139	0.5962	0.5994
0.5673	98.99	19600	0.7105	0.5986	0.6007
0.5663	101.01	20000	0.7108	0.6003	0.6023

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2