GUE_prom_prom_300_notata-seqsight_4096_512_27M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_300_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.1230
• F1 Score: 0.9565
• Accuracy: 0.9565

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.2902	0.6	200	0.1596	0.9346	0.9346
0.1676	1.2	400	0.1395	0.9432	0.9433
0.1539	1.81	600	0.1288	0.9489	0.9489
0.1408	2.41	800	0.1238	0.9489	0.9489
0.1368	3.01	1000	0.1215	0.9510	0.9510
0.1325	3.61	1200	0.1167	0.9544	0.9544
0.1329	4.22	1400	0.1129	0.9584	0.9584
0.1261	4.82	1600	0.1119	0.9585	0.9585
0.1234	5.42	1800	0.1110	0.9587	0.9587
0.1211	6.02	2000	0.1089	0.9589	0.9589
0.1172	6.63	2200	0.1088	0.9587	0.9587
0.117	7.23	2400	0.1079	0.9593	0.9593
0.1138	7.83	2600	0.1066	0.9600	0.9601
0.1147	8.43	2800	0.1074	0.9606	0.9606
0.1151	9.04	3000	0.1079	0.9565	0.9565
0.1108	9.64	3200	0.1120	0.9582	0.9582
0.1089	10.24	3400	0.1068	0.9578	0.9578
0.1128	10.84	3600	0.1039	0.9610	0.9610
0.1058	11.45	3800	0.1045	0.9608	0.9608
0.1075	12.05	4000	0.1041	0.9612	0.9612
0.107	12.65	4200	0.1022	0.9617	0.9617
0.1077	13.25	4400	0.1020	0.9614	0.9614
0.1061	13.86	4600	0.1016	0.9629	0.9629
0.1071	14.46	4800	0.1030	0.9616	0.9616
0.1029	15.06	5000	0.1016	0.9621	0.9621
0.1031	15.66	5200	0.1011	0.9623	0.9623
0.1077	16.27	5400	0.1015	0.9616	0.9616
0.1018	16.87	5600	0.1004	0.9623	0.9623
0.1	17.47	5800	0.1014	0.9627	0.9627
0.106	18.07	6000	0.0995	0.9627	0.9627
0.1018	18.67	6200	0.0998	0.9619	0.9619
0.1016	19.28	6400	0.1001	0.9623	0.9623
0.1007	19.88	6600	0.1018	0.9625	0.9625
0.1052	20.48	6800	0.0991	0.9619	0.9619
0.0988	21.08	7000	0.0995	0.9627	0.9627
0.0985	21.69	7200	0.1001	0.9631	0.9631
0.0995	22.29	7400	0.1008	0.9629	0.9629
0.1036	22.89	7600	0.0991	0.9633	0.9633
0.0974	23.49	7800	0.0994	0.9638	0.9638
0.1001	24.1	8000	0.0992	0.9627	0.9627
0.0993	24.7	8200	0.0999	0.9634	0.9634
0.0998	25.3	8400	0.0996	0.9627	0.9627
0.1001	25.9	8600	0.0991	0.9633	0.9633
0.1	26.51	8800	0.0993	0.9636	0.9636
0.0965	27.11	9000	0.0993	0.9634	0.9634
0.0992	27.71	9200	0.0992	0.9629	0.9629
0.0967	28.31	9400	0.0991	0.9625	0.9625
0.1002	28.92	9600	0.0988	0.9625	0.9625
0.0959	29.52	9800	0.0990	0.9625	0.9625
0.0996	30.12	10000	0.0990	0.9627	0.9627

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2