GUE_prom_prom_300_notata-seqsight_4096_512_27M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_300_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.1394
• F1 Score: 0.9567
• Accuracy: 0.9567

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.2325	0.6	200	0.1278	0.9501	0.9501
0.1385	1.2	400	0.1188	0.9563	0.9563
0.1292	1.81	600	0.1125	0.9563	0.9563
0.1177	2.41	800	0.1102	0.9595	0.9595
0.1159	3.01	1000	0.1079	0.9617	0.9617
0.1077	3.61	1200	0.1107	0.9578	0.9578
0.1097	4.22	1400	0.1042	0.9625	0.9625
0.1045	4.82	1600	0.1030	0.9608	0.9608
0.0988	5.42	1800	0.1036	0.9634	0.9634
0.0963	6.02	2000	0.0993	0.9638	0.9638
0.0936	6.63	2200	0.1034	0.9623	0.9623
0.0917	7.23	2400	0.1039	0.9631	0.9631
0.087	7.83	2600	0.1046	0.9633	0.9633
0.0879	8.43	2800	0.1094	0.9604	0.9604
0.0883	9.04	3000	0.1065	0.9619	0.9619
0.0834	9.64	3200	0.1074	0.9621	0.9621
0.0794	10.24	3400	0.0981	0.9636	0.9636
0.0851	10.84	3600	0.0976	0.9651	0.9651
0.0746	11.45	3800	0.0968	0.9636	0.9636
0.0736	12.05	4000	0.1052	0.9655	0.9655
0.0716	12.65	4200	0.0987	0.9663	0.9663
0.0699	13.25	4400	0.1020	0.9655	0.9655
0.0705	13.86	4600	0.0979	0.9649	0.9650
0.0666	14.46	4800	0.1057	0.9642	0.9642
0.068	15.06	5000	0.0984	0.9657	0.9657
0.0635	15.66	5200	0.1025	0.9651	0.9651
0.0632	16.27	5400	0.1039	0.9648	0.9648
0.0607	16.87	5600	0.1035	0.9644	0.9644
0.0575	17.47	5800	0.1075	0.9648	0.9648
0.0618	18.07	6000	0.1061	0.9661	0.9661
0.0558	18.67	6200	0.1059	0.9665	0.9665
0.055	19.28	6400	0.1113	0.9650	0.9650
0.056	19.88	6600	0.1104	0.9661	0.9661
0.0549	20.48	6800	0.1051	0.9657	0.9657
0.0507	21.08	7000	0.1087	0.9661	0.9661
0.0512	21.69	7200	0.1129	0.9650	0.9650
0.05	22.29	7400	0.1122	0.9657	0.9657
0.0515	22.89	7600	0.1071	0.9670	0.9670
0.0449	23.49	7800	0.1137	0.9668	0.9668
0.049	24.1	8000	0.1120	0.9650	0.9650
0.0455	24.7	8200	0.1252	0.9646	0.9646
0.0463	25.3	8400	0.1175	0.9651	0.9651
0.0442	25.9	8600	0.1164	0.9655	0.9655
0.0452	26.51	8800	0.1179	0.9651	0.9651
0.0435	27.11	9000	0.1177	0.9657	0.9657
0.0434	27.71	9200	0.1195	0.9651	0.9651
0.041	28.31	9400	0.1194	0.9659	0.9659
0.0431	28.92	9600	0.1191	0.9651	0.9651
0.041	29.52	9800	0.1185	0.9649	0.9650
0.0408	30.12	10000	0.1189	0.9651	0.9651

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2