GUE_EMP_H4ac-seqsight_16384_512_56M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H4ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.5424
• F1 Score: 0.7384
• Accuracy: 0.7381

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.598	0.93	200	0.5625	0.7171	0.7179
0.5513	1.87	400	0.5641	0.7209	0.7229
0.536	2.8	600	0.5415	0.7383	0.7381
0.5305	3.74	800	0.5354	0.7355	0.7352
0.526	4.67	1000	0.5325	0.7405	0.7402
0.514	5.61	1200	0.5421	0.7363	0.7370
0.5144	6.54	1400	0.5380	0.7371	0.7375
0.4999	7.48	1600	0.5358	0.7453	0.7452
0.5078	8.41	1800	0.5257	0.7483	0.7481
0.5022	9.35	2000	0.5268	0.7487	0.7484
0.4926	10.28	2200	0.5264	0.7454	0.7452
0.4939	11.21	2400	0.5519	0.7339	0.7355
0.4868	12.15	2600	0.5432	0.7401	0.7408
0.4841	13.08	2800	0.5397	0.7461	0.7460
0.4847	14.02	3000	0.5271	0.7430	0.7431
0.4782	14.95	3200	0.5273	0.7484	0.7481
0.4763	15.89	3400	0.5244	0.7534	0.7531
0.4726	16.82	3600	0.5343	0.7436	0.7437
0.474	17.76	3800	0.5673	0.7270	0.7296
0.4703	18.69	4000	0.5288	0.7443	0.7440
0.4653	19.63	4200	0.5236	0.7454	0.7452
0.4639	20.56	4400	0.5356	0.7444	0.7443
0.4622	21.5	4600	0.5348	0.7427	0.7431
0.4596	22.43	4800	0.5321	0.7449	0.7446
0.4561	23.36	5000	0.5373	0.7439	0.7437
0.458	24.3	5200	0.5286	0.7464	0.7463
0.454	25.23	5400	0.5276	0.7507	0.7504
0.4527	26.17	5600	0.5275	0.7454	0.7452
0.4511	27.1	5800	0.5334	0.7457	0.7455
0.4405	28.04	6000	0.5433	0.7466	0.7463
0.4505	28.97	6200	0.5300	0.7490	0.7487
0.4461	29.91	6400	0.5396	0.7477	0.7475
0.4465	30.84	6600	0.5380	0.7435	0.7437
0.4421	31.78	6800	0.5272	0.7466	0.7463
0.4398	32.71	7000	0.5429	0.7438	0.7437
0.4378	33.64	7200	0.5481	0.7425	0.7428
0.4374	34.58	7400	0.5395	0.7477	0.7475
0.433	35.51	7600	0.5425	0.7427	0.7425
0.4309	36.45	7800	0.5489	0.7467	0.7466
0.4355	37.38	8000	0.5436	0.7482	0.7481
0.4284	38.32	8200	0.5459	0.7502	0.7501
0.4317	39.25	8400	0.5448	0.7428	0.7425
0.4327	40.19	8600	0.5481	0.7469	0.7466
0.4287	41.12	8800	0.5515	0.7480	0.7481
0.4256	42.06	9000	0.5487	0.7515	0.7513
0.427	42.99	9200	0.5510	0.7469	0.7469
0.425	43.93	9400	0.5452	0.7495	0.7493
0.4242	44.86	9600	0.5466	0.7498	0.7496
0.4253	45.79	9800	0.5469	0.7500	0.7499
0.4268	46.73	10000	0.5457	0.7500	0.7499

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2