GUE_splice_reconstructed-seqsight_65536_512_94M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_94M on the mahdibaghbanzadeh/GUE_splice_reconstructed dataset. It achieves the following results on the evaluation set:

• Loss: 0.2732
• F1 Score: 0.9070
• Accuracy: 0.9066

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.9185	0.7	200	0.8145	0.5984	0.6232
0.523	1.4	400	0.4500	0.8186	0.8167
0.4063	2.1	600	0.3903	0.8496	0.8488
0.3641	2.8	800	0.4174	0.8400	0.8391
0.3519	3.5	1000	0.3574	0.8657	0.8650
0.3403	4.2	1200	0.3609	0.8672	0.8665
0.324	4.9	1400	0.3443	0.8713	0.8707
0.3249	5.59	1600	0.3387	0.8766	0.8762
0.3059	6.29	1800	0.3562	0.8640	0.8632
0.2976	6.99	2000	0.3166	0.8835	0.8829
0.2909	7.69	2200	0.3065	0.8875	0.8871
0.2867	8.39	2400	0.3253	0.8787	0.8781
0.2811	9.09	2600	0.3220	0.8809	0.8801
0.2741	9.79	2800	0.3175	0.8829	0.8823
0.2694	10.49	3000	0.3180	0.8841	0.8836
0.2584	11.19	3200	0.3230	0.8844	0.8838
0.261	11.89	3400	0.3123	0.8844	0.8838
0.2491	12.59	3600	0.3081	0.8892	0.8886
0.2513	13.29	3800	0.3020	0.8915	0.8911
0.2459	13.99	4000	0.3084	0.8885	0.8880
0.245	14.69	4200	0.3190	0.8831	0.8825
0.2402	15.38	4400	0.2886	0.8964	0.8961
0.2369	16.08	4600	0.3465	0.8758	0.8751
0.2346	16.78	4800	0.2994	0.8937	0.8932
0.224	17.48	5000	0.3257	0.8819	0.8812
0.2328	18.18	5200	0.3000	0.8961	0.8957
0.2279	18.88	5400	0.3010	0.8959	0.8954
0.2193	19.58	5600	0.2972	0.8954	0.8950
0.2262	20.28	5800	0.3019	0.8939	0.8935
0.2183	20.98	6000	0.2910	0.8982	0.8979
0.2195	21.68	6200	0.3058	0.8911	0.8906
0.2159	22.38	6400	0.3010	0.8929	0.8924
0.2057	23.08	6600	0.3020	0.8950	0.8946
0.216	23.78	6800	0.2929	0.8967	0.8963
0.2082	24.48	7000	0.3037	0.8918	0.8913
0.212	25.17	7200	0.2970	0.8944	0.8939
0.2064	25.87	7400	0.2921	0.8983	0.8979
0.2096	26.57	7600	0.3028	0.8948	0.8943
0.2014	27.27	7800	0.2981	0.8974	0.8970
0.2048	27.97	8000	0.2914	0.8993	0.8989
0.2055	28.67	8200	0.3029	0.8926	0.8922
0.1953	29.37	8400	0.2979	0.8996	0.8992
0.1996	30.07	8600	0.2938	0.8989	0.8985
0.1982	30.77	8800	0.2960	0.8965	0.8961
0.2026	31.47	9000	0.3025	0.8924	0.8919
0.1971	32.17	9200	0.2988	0.8953	0.8948
0.1943	32.87	9400	0.2978	0.8974	0.8970
0.1985	33.57	9600	0.3001	0.8948	0.8943
0.1914	34.27	9800	0.2994	0.8953	0.8948
0.1912	34.97	10000	0.2992	0.8950	0.8946

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2