GUE_prom_prom_core_notata-seqsight_4096_512_27M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_core_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.3711
• F1 Score: 0.8381
• Accuracy: 0.8381

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.4946	0.6	200	0.3979	0.8210	0.8210
0.4113	1.2	400	0.3835	0.8268	0.8268
0.3993	1.81	600	0.3786	0.8324	0.8325
0.3946	2.41	800	0.3719	0.8341	0.8342
0.3859	3.01	1000	0.3699	0.8347	0.8347
0.3787	3.61	1200	0.3684	0.8363	0.8364
0.3826	4.22	1400	0.3691	0.8334	0.8334
0.38	4.82	1600	0.3659	0.8376	0.8378
0.3683	5.42	1800	0.3761	0.8320	0.8321
0.3727	6.02	2000	0.3677	0.8348	0.8349
0.37	6.63	2200	0.3631	0.8394	0.8395
0.3673	7.23	2400	0.3682	0.8388	0.8391
0.3668	7.83	2600	0.3654	0.8370	0.8370
0.3611	8.43	2800	0.3695	0.8393	0.8396
0.366	9.04	3000	0.3630	0.8379	0.8379
0.3581	9.64	3200	0.3654	0.8410	0.8410
0.3567	10.24	3400	0.3664	0.8414	0.8413
0.3565	10.84	3600	0.3609	0.8408	0.8408
0.3568	11.45	3800	0.3625	0.8398	0.8398
0.3566	12.05	4000	0.3623	0.8431	0.8430
0.3516	12.65	4200	0.3641	0.8423	0.8423
0.3555	13.25	4400	0.3625	0.8413	0.8413
0.356	13.86	4600	0.3627	0.8419	0.8419
0.3493	14.46	4800	0.3636	0.8410	0.8410
0.3501	15.06	5000	0.3611	0.8406	0.8406
0.3442	15.66	5200	0.3626	0.8410	0.8410
0.3424	16.27	5400	0.3660	0.8421	0.8421
0.347	16.87	5600	0.3637	0.8410	0.8412
0.3425	17.47	5800	0.3662	0.8407	0.8408
0.3485	18.07	6000	0.3633	0.8407	0.8408
0.3434	18.67	6200	0.3618	0.8451	0.8451
0.3447	19.28	6400	0.3648	0.8412	0.8412
0.3414	19.88	6600	0.3630	0.8423	0.8423
0.3355	20.48	6800	0.3638	0.8428	0.8428
0.3486	21.08	7000	0.3632	0.8414	0.8415
0.3436	21.69	7200	0.3641	0.8417	0.8417
0.3344	22.29	7400	0.3638	0.8409	0.8410
0.3402	22.89	7600	0.3635	0.8436	0.8436
0.3402	23.49	7800	0.3638	0.8413	0.8413
0.3409	24.1	8000	0.3655	0.8426	0.8427
0.3419	24.7	8200	0.3634	0.8430	0.8430
0.3345	25.3	8400	0.3666	0.8426	0.8427
0.3385	25.9	8600	0.3644	0.8421	0.8421
0.3397	26.51	8800	0.3656	0.8408	0.8408
0.3379	27.11	9000	0.3643	0.8427	0.8427
0.3405	27.71	9200	0.3648	0.8413	0.8413
0.3298	28.31	9400	0.3653	0.8422	0.8423
0.3339	28.92	9600	0.3653	0.8415	0.8415
0.3384	29.52	9800	0.3649	0.8419	0.8419
0.3296	30.12	10000	0.3652	0.8419	0.8419

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2