GUE_EMP_H4ac-seqsight_16384_512_56M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H4ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.5443
• F1 Score: 0.7415
• Accuracy: 0.7413

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5865	0.93	200	0.5616	0.7236	0.7246
0.5409	1.87	400	0.5523	0.7232	0.7246
0.5243	2.8	600	0.5323	0.7402	0.7399
0.5133	3.74	800	0.5351	0.7370	0.7372
0.5091	4.67	1000	0.5197	0.7474	0.7472
0.4952	5.61	1200	0.5306	0.7454	0.7455
0.4908	6.54	1400	0.5291	0.7436	0.7437
0.4748	7.48	1600	0.5288	0.7397	0.7396
0.4777	8.41	1800	0.5187	0.7454	0.7452
0.4683	9.35	2000	0.5285	0.7318	0.7328
0.4579	10.28	2200	0.5254	0.7501	0.7499
0.4525	11.21	2400	0.5367	0.7453	0.7452
0.4419	12.15	2600	0.5284	0.7412	0.7416
0.4354	13.08	2800	0.5425	0.7490	0.7487
0.4326	14.02	3000	0.5501	0.7409	0.7413
0.425	14.95	3200	0.5560	0.7504	0.7501
0.4155	15.89	3400	0.5385	0.7507	0.7504
0.4054	16.82	3600	0.5621	0.7375	0.7372
0.4034	17.76	3800	0.6042	0.7287	0.7314
0.3951	18.69	4000	0.5603	0.7334	0.7334
0.3892	19.63	4200	0.5567	0.7455	0.7452
0.38	20.56	4400	0.5779	0.7408	0.7405
0.376	21.5	4600	0.5861	0.7414	0.7413
0.3681	22.43	4800	0.5816	0.7367	0.7364
0.3586	23.36	5000	0.6062	0.7376	0.7378
0.3575	24.3	5200	0.5973	0.7431	0.7428
0.3537	25.23	5400	0.5922	0.7384	0.7381
0.3443	26.17	5600	0.5948	0.7375	0.7372
0.341	27.1	5800	0.6103	0.7323	0.7323
0.3265	28.04	6000	0.6109	0.7393	0.7390
0.3317	28.97	6200	0.6055	0.7329	0.7326
0.3274	29.91	6400	0.6146	0.7270	0.7267
0.3222	30.84	6600	0.6171	0.7323	0.7320
0.3159	31.78	6800	0.5983	0.7299	0.7296
0.3057	32.71	7000	0.6538	0.7258	0.7255
0.3081	33.64	7200	0.6444	0.7245	0.7243
0.3031	34.58	7400	0.6478	0.7320	0.7317
0.299	35.51	7600	0.6399	0.7263	0.7261
0.2883	36.45	7800	0.6671	0.7349	0.7346
0.2941	37.38	8000	0.6549	0.7273	0.7270
0.2869	38.32	8200	0.6615	0.7320	0.7317
0.2848	39.25	8400	0.6594	0.7293	0.7290
0.2852	40.19	8600	0.6697	0.7323	0.7320
0.2811	41.12	8800	0.6715	0.7291	0.7287
0.2754	42.06	9000	0.6837	0.7296	0.7293
0.278	42.99	9200	0.6753	0.7314	0.7311
0.2715	43.93	9400	0.6735	0.7257	0.7255
0.2657	44.86	9600	0.6834	0.7284	0.7282
0.2685	45.79	9800	0.6874	0.7296	0.7293
0.2717	46.73	10000	0.6834	0.7284	0.7282

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2