GUE_splice_reconstructed-seqsight_65536_512_94M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_94M on the mahdibaghbanzadeh/GUE_splice_reconstructed dataset. It achieves the following results on the evaluation set:

• Loss: 0.3132
• F1 Score: 0.8831
• Accuracy: 0.8825

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.9459	0.7	200	0.8973	0.4921	0.5655
0.8312	1.4	400	0.6411	0.7412	0.7394
0.5143	2.1	600	0.4459	0.8295	0.8286
0.4393	2.8	800	0.4513	0.8249	0.8238
0.4195	3.5	1000	0.4200	0.8381	0.8371
0.4068	4.2	1200	0.4111	0.8424	0.8415
0.3921	4.9	1400	0.3841	0.8534	0.8527
0.3919	5.59	1600	0.3926	0.8494	0.8488
0.3724	6.29	1800	0.3971	0.8509	0.8501
0.3673	6.99	2000	0.3792	0.8563	0.8555
0.3633	7.69	2200	0.3575	0.8648	0.8643
0.3592	8.39	2400	0.3812	0.8585	0.8577
0.3544	9.09	2600	0.3951	0.8563	0.8553
0.3517	9.79	2800	0.3631	0.8640	0.8632
0.3475	10.49	3000	0.3686	0.8650	0.8643
0.3414	11.19	3200	0.3826	0.8572	0.8562
0.3441	11.89	3400	0.3601	0.8682	0.8676
0.3337	12.59	3600	0.3547	0.8698	0.8691
0.3362	13.29	3800	0.3568	0.8689	0.8683
0.3368	13.99	4000	0.3699	0.8635	0.8628
0.3314	14.69	4200	0.3620	0.8626	0.8619
0.3279	15.38	4400	0.3475	0.8735	0.8729
0.3287	16.08	4600	0.3822	0.8581	0.8573
0.3247	16.78	4800	0.3627	0.8644	0.8637
0.3162	17.48	5000	0.3650	0.8606	0.8597
0.3238	18.18	5200	0.3447	0.8733	0.8726
0.3239	18.88	5400	0.3576	0.8681	0.8674
0.3199	19.58	5600	0.3376	0.8765	0.8759
0.3258	20.28	5800	0.3438	0.8763	0.8757
0.3133	20.98	6000	0.3425	0.8776	0.8770
0.3199	21.68	6200	0.3637	0.8666	0.8658
0.3161	22.38	6400	0.3523	0.8705	0.8698
0.3074	23.08	6600	0.3526	0.8710	0.8702
0.3161	23.78	6800	0.3446	0.8753	0.8746
0.3114	24.48	7000	0.3505	0.8716	0.8709
0.3138	25.17	7200	0.3436	0.8755	0.8748
0.3087	25.87	7400	0.3452	0.8751	0.8744
0.3112	26.57	7600	0.3448	0.8755	0.8748
0.3073	27.27	7800	0.3395	0.8763	0.8757
0.3092	27.97	8000	0.3439	0.8749	0.8742
0.3103	28.67	8200	0.3509	0.8731	0.8724
0.3025	29.37	8400	0.3427	0.8751	0.8744
0.3045	30.07	8600	0.3391	0.8770	0.8764
0.3088	30.77	8800	0.3475	0.8733	0.8726
0.3066	31.47	9000	0.3439	0.8751	0.8744
0.3082	32.17	9200	0.3472	0.8734	0.8726
0.2984	32.87	9400	0.3430	0.8755	0.8748
0.3062	33.57	9600	0.3475	0.8740	0.8733
0.3039	34.27	9800	0.3441	0.8748	0.8742
0.2995	34.97	10000	0.3455	0.8746	0.8740

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2