GUE_splice_reconstructed-seqsight_65536_512_94M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_94M on the mahdibaghbanzadeh/GUE_splice_reconstructed dataset. It achieves the following results on the evaluation set:

• Loss: 0.2648
• F1 Score: 0.9154
• Accuracy: 0.9152

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.8295	0.7	200	0.4909	0.7992	0.7979
0.4099	1.4	400	0.4244	0.8331	0.8317
0.3649	2.1	600	0.3627	0.8619	0.8610
0.3222	2.8	800	0.3664	0.8659	0.8652
0.3058	3.5	1000	0.3049	0.8886	0.8882
0.2879	4.2	1200	0.3270	0.8757	0.8751
0.2716	4.9	1400	0.2982	0.8914	0.8911
0.264	5.59	1600	0.3278	0.8832	0.8827
0.2524	6.29	1800	0.3012	0.8889	0.8884
0.239	6.99	2000	0.2736	0.9027	0.9025
0.2309	7.69	2200	0.2903	0.8974	0.8970
0.2252	8.39	2400	0.2860	0.9024	0.9020
0.2195	9.09	2600	0.2955	0.8940	0.8935
0.2099	9.79	2800	0.3007	0.8927	0.8922
0.205	10.49	3000	0.2708	0.9068	0.9066
0.1967	11.19	3200	0.2972	0.8988	0.8983
0.1972	11.89	3400	0.2852	0.8983	0.8979
0.1772	12.59	3600	0.2925	0.8963	0.8959
0.1839	13.29	3800	0.2729	0.9053	0.9051
0.1779	13.99	4000	0.2727	0.9087	0.9084
0.1728	14.69	4200	0.3094	0.8951	0.8946
0.1689	15.38	4400	0.2939	0.9019	0.9016
0.1605	16.08	4600	0.3058	0.9004	0.9000
0.1599	16.78	4800	0.2817	0.9088	0.9086
0.1526	17.48	5000	0.3020	0.9059	0.9055
0.1542	18.18	5200	0.3059	0.9011	0.9007
0.1512	18.88	5400	0.2805	0.9119	0.9117
0.1389	19.58	5600	0.3007	0.9097	0.9095
0.1479	20.28	5800	0.2910	0.9086	0.9084
0.1376	20.98	6000	0.2912	0.9079	0.9077
0.1359	21.68	6200	0.2953	0.9062	0.9060
0.1334	22.38	6400	0.3101	0.9048	0.9044
0.1286	23.08	6600	0.2861	0.9101	0.9099
0.1306	23.78	6800	0.3097	0.9061	0.9057
0.1278	24.48	7000	0.3100	0.9048	0.9044
0.1243	25.17	7200	0.3115	0.9050	0.9046
0.1228	25.87	7400	0.2946	0.9108	0.9106
0.1207	26.57	7600	0.3040	0.9099	0.9097
0.1148	27.27	7800	0.3103	0.9088	0.9086
0.1161	27.97	8000	0.3058	0.9105	0.9103
0.1157	28.67	8200	0.3100	0.9071	0.9068
0.1103	29.37	8400	0.3046	0.9106	0.9103
0.1089	30.07	8600	0.3152	0.9091	0.9088
0.1088	30.77	8800	0.3110	0.9099	0.9097
0.1136	31.47	9000	0.3157	0.9104	0.9101
0.106	32.17	9200	0.3121	0.9077	0.9075
0.1085	32.87	9400	0.3154	0.9097	0.9095
0.1092	33.57	9600	0.3091	0.9099	0.9097
0.1008	34.27	9800	0.3126	0.9102	0.9099
0.1037	34.97	10000	0.3150	0.9091	0.9088

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2