GUE_EMP_H3K4me1-seqsight_4096_512_15M-L1

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_15M on the mahdibaghbanzadeh/GUE_EMP_H3K4me1 dataset. It achieves the following results on the evaluation set:

• Loss: 0.6525
• F1 Score: 0.6153
• Accuracy: 0.6162

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 20000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6911	2.02	400	0.6810	0.5324	0.5698
0.664	4.04	800	0.6697	0.5907	0.5941
0.6564	6.06	1200	0.6679	0.5891	0.6054
0.6513	8.08	1600	0.6675	0.5979	0.6016
0.6483	10.1	2000	0.6659	0.5937	0.6026
0.6439	12.12	2400	0.6665	0.6047	0.6051
0.6418	14.14	2800	0.6675	0.6002	0.6023
0.639	16.16	3200	0.6678	0.5879	0.6048
0.6356	18.18	3600	0.6763	0.5981	0.5979
0.634	20.2	4000	0.6747	0.6009	0.6032
0.6319	22.22	4400	0.6765	0.6013	0.6039
0.6291	24.24	4800	0.6741	0.5864	0.5966
0.6284	26.26	5200	0.6820	0.5951	0.5947
0.624	28.28	5600	0.6788	0.5903	0.5997
0.6253	30.3	6000	0.6751	0.6041	0.6064
0.6233	32.32	6400	0.6745	0.5938	0.6013
0.6192	34.34	6800	0.6749	0.5966	0.6026
0.6198	36.36	7200	0.6813	0.6027	0.6051
0.6182	38.38	7600	0.6912	0.5991	0.6023
0.619	40.4	8000	0.6786	0.5945	0.6020
0.6172	42.42	8400	0.6873	0.5978	0.6013
0.6158	44.44	8800	0.6865	0.5998	0.6016
0.6154	46.46	9200	0.6869	0.5974	0.5975
0.6123	48.48	9600	0.6920	0.5955	0.5997
0.6135	50.51	10000	0.6920	0.5976	0.6007
0.6132	52.53	10400	0.6835	0.6037	0.6064
0.6118	54.55	10800	0.6777	0.5982	0.5988
0.6117	56.57	11200	0.6889	0.5986	0.6032
0.6092	58.59	11600	0.6845	0.6001	0.6039
0.6112	60.61	12000	0.6907	0.5959	0.6007
0.6088	62.63	12400	0.6949	0.5897	0.5963
0.6077	64.65	12800	0.6912	0.5945	0.6004
0.609	66.67	13200	0.6906	0.5965	0.6004
0.6072	68.69	13600	0.6846	0.5958	0.5960
0.6079	70.71	14000	0.6823	0.5985	0.6026
0.6073	72.73	14400	0.6848	0.5969	0.6020
0.607	74.75	14800	0.6954	0.6009	0.6048
0.6046	76.77	15200	0.6913	0.5972	0.6023
0.6056	78.79	15600	0.6888	0.5998	0.6029
0.6034	80.81	16000	0.6917	0.5984	0.6016
0.6037	82.83	16400	0.6920	0.5983	0.6013
0.6027	84.85	16800	0.6930	0.6002	0.6035
0.6055	86.87	17200	0.6940	0.5944	0.5950
0.6031	88.89	17600	0.6854	0.5985	0.6013
0.6043	90.91	18000	0.6975	0.5994	0.6020
0.6029	92.93	18400	0.6945	0.5952	0.5994
0.6032	94.95	18800	0.6887	0.5981	0.6016
0.6032	96.97	19200	0.6912	0.5954	0.5997
0.6025	98.99	19600	0.6930	0.5989	0.6020
0.6021	101.01	20000	0.6920	0.5989	0.6020

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2