GUE_prom_prom_core_notata-seqsight_32768_512_43M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_prom_prom_core_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.3786
• F1 Score: 0.8327
• Accuracy: 0.8327

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5136	0.6	200	0.3951	0.8217	0.8217
0.4153	1.2	400	0.3880	0.8265	0.8268
0.4002	1.81	600	0.3924	0.8262	0.8268
0.3984	2.41	800	0.3814	0.8318	0.8321
0.3895	3.01	1000	0.3794	0.8325	0.8331
0.3846	3.61	1200	0.3729	0.8345	0.8347
0.3866	4.22	1400	0.3690	0.8381	0.8381
0.3879	4.82	1600	0.3693	0.8370	0.8372
0.3746	5.42	1800	0.3728	0.8346	0.8346
0.382	6.02	2000	0.3697	0.8375	0.8378
0.378	6.63	2200	0.3666	0.8365	0.8366
0.3741	7.23	2400	0.3731	0.8346	0.8351
0.3749	7.83	2600	0.3636	0.8391	0.8391
0.3707	8.43	2800	0.3775	0.8349	0.8357
0.3751	9.04	3000	0.3640	0.8409	0.8410
0.3674	9.64	3200	0.3633	0.8393	0.8393
0.3683	10.24	3400	0.3623	0.8411	0.8412
0.3655	10.84	3600	0.3600	0.8419	0.8419
0.3654	11.45	3800	0.3603	0.8396	0.8396
0.3636	12.05	4000	0.3616	0.8423	0.8423
0.3606	12.65	4200	0.3641	0.8406	0.8406
0.3643	13.25	4400	0.3632	0.8388	0.8389
0.3628	13.86	4600	0.3650	0.8390	0.8391
0.3605	14.46	4800	0.3636	0.8388	0.8389
0.3612	15.06	5000	0.3580	0.8400	0.8400
0.3563	15.66	5200	0.3614	0.8388	0.8389
0.3597	16.27	5400	0.3646	0.8402	0.8402
0.3565	16.87	5600	0.3689	0.8380	0.8385
0.3534	17.47	5800	0.3653	0.8390	0.8393
0.3618	18.07	6000	0.3601	0.8410	0.8412
0.3549	18.67	6200	0.3577	0.8422	0.8423
0.3548	19.28	6400	0.3606	0.8434	0.8434
0.3523	19.88	6600	0.3596	0.8404	0.8406
0.3461	20.48	6800	0.3600	0.8412	0.8413
0.359	21.08	7000	0.3598	0.8411	0.8413
0.3558	21.69	7200	0.3595	0.8437	0.8438
0.3468	22.29	7400	0.3587	0.8410	0.8412
0.3469	22.89	7600	0.3605	0.8402	0.8404
0.3479	23.49	7800	0.3592	0.8407	0.8408
0.3521	24.1	8000	0.3627	0.8383	0.8385
0.3509	24.7	8200	0.3631	0.8395	0.8398
0.3451	25.3	8400	0.3639	0.8402	0.8404
0.3518	25.9	8600	0.3595	0.8410	0.8412
0.3502	26.51	8800	0.3592	0.8413	0.8413
0.3503	27.11	9000	0.3583	0.8420	0.8421
0.3528	27.71	9200	0.3609	0.8402	0.8404
0.3399	28.31	9400	0.3624	0.8392	0.8395
0.349	28.92	9600	0.3598	0.8412	0.8413
0.3499	29.52	9800	0.3596	0.8403	0.8404
0.3414	30.12	10000	0.3604	0.8406	0.8408

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2