GUE_EMP_H4ac-seqsight_16384_512_56M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H4ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.5400
• F1 Score: 0.7389
• Accuracy: 0.7387

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6113	0.93	200	0.5680	0.7095	0.7100
0.572	1.87	400	0.5594	0.7164	0.7164
0.5552	2.8	600	0.5536	0.7272	0.7273
0.5502	3.74	800	0.5479	0.7287	0.7284
0.5447	4.67	1000	0.5498	0.7288	0.7287
0.5353	5.61	1200	0.5618	0.7185	0.7205
0.5363	6.54	1400	0.5655	0.7144	0.7170
0.5237	7.48	1600	0.5516	0.7353	0.7355
0.533	8.41	1800	0.5478	0.7296	0.7299
0.5298	9.35	2000	0.5565	0.7226	0.7238
0.5184	10.28	2200	0.5374	0.7390	0.7387
0.5243	11.21	2400	0.5541	0.7308	0.7317
0.5154	12.15	2600	0.5691	0.7251	0.7270
0.5176	13.08	2800	0.5562	0.7323	0.7331
0.519	14.02	3000	0.5338	0.7395	0.7393
0.5141	14.95	3200	0.5441	0.7395	0.7396
0.511	15.89	3400	0.5451	0.7396	0.7399
0.5109	16.82	3600	0.5474	0.7370	0.7375
0.5124	17.76	3800	0.5658	0.7261	0.7282
0.51	18.69	4000	0.5441	0.7386	0.7387
0.5065	19.63	4200	0.5371	0.7436	0.7437
0.5079	20.56	4400	0.5356	0.7442	0.7443
0.5038	21.5	4600	0.5512	0.7350	0.7361
0.5053	22.43	4800	0.5326	0.7442	0.7440
0.5014	23.36	5000	0.5475	0.7416	0.7422
0.5036	24.3	5200	0.5289	0.7474	0.7472
0.503	25.23	5400	0.5268	0.7440	0.7437
0.503	26.17	5600	0.5320	0.7409	0.7408
0.5008	27.1	5800	0.5317	0.7413	0.7411
0.4931	28.04	6000	0.5367	0.7431	0.7428
0.501	28.97	6200	0.5425	0.7423	0.7425
0.4986	29.91	6400	0.5394	0.7416	0.7416
0.4991	30.84	6600	0.5435	0.7396	0.7402
0.4947	31.78	6800	0.5304	0.7430	0.7428
0.4952	32.71	7000	0.5355	0.7411	0.7411
0.492	33.64	7200	0.5465	0.7395	0.7402
0.4942	34.58	7400	0.5327	0.7427	0.7425
0.4941	35.51	7600	0.5377	0.7401	0.7402
0.4893	36.45	7800	0.5352	0.7436	0.7434
0.4958	37.38	8000	0.5437	0.7408	0.7413
0.4902	38.32	8200	0.5360	0.7425	0.7425
0.4922	39.25	8400	0.5329	0.7429	0.7428
0.4945	40.19	8600	0.5353	0.7409	0.7408
0.4909	41.12	8800	0.5414	0.7419	0.7422
0.4882	42.06	9000	0.5362	0.7408	0.7408
0.4898	42.99	9200	0.5449	0.7430	0.7434
0.4889	43.93	9400	0.5376	0.7427	0.7428
0.4879	44.86	9600	0.5355	0.7416	0.7416
0.4867	45.79	9800	0.5374	0.7424	0.7425
0.4924	46.73	10000	0.5380	0.7433	0.7434

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2