GUE_prom_prom_300_notata-seqsight_4096_512_27M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_300_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.1198
• F1 Score: 0.9557
• Accuracy: 0.9557

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.2559	0.6	200	0.1344	0.9472	0.9472
0.1465	1.2	400	0.1278	0.9508	0.9508
0.1356	1.81	600	0.1165	0.9561	0.9561
0.1234	2.41	800	0.1167	0.9550	0.9550
0.1221	3.01	1000	0.1154	0.9549	0.9550
0.1158	3.61	1200	0.1097	0.9576	0.9576
0.1168	4.22	1400	0.1045	0.9597	0.9597
0.1117	4.82	1600	0.1048	0.9612	0.9612
0.1089	5.42	1800	0.1065	0.9599	0.9599
0.1059	6.02	2000	0.1032	0.9616	0.9616
0.1035	6.63	2200	0.1037	0.9608	0.9608
0.1029	7.23	2400	0.1047	0.9623	0.9623
0.0983	7.83	2600	0.1056	0.9595	0.9595
0.1008	8.43	2800	0.1061	0.9606	0.9606
0.1002	9.04	3000	0.1063	0.9623	0.9623
0.0958	9.64	3200	0.1155	0.9561	0.9561
0.0943	10.24	3400	0.1021	0.9623	0.9623
0.0979	10.84	3600	0.1029	0.9629	0.9629
0.0911	11.45	3800	0.1023	0.9629	0.9629
0.0916	12.05	4000	0.1040	0.9625	0.9625
0.0905	12.65	4200	0.1002	0.9642	0.9642
0.0896	13.25	4400	0.1041	0.9610	0.9610
0.0902	13.86	4600	0.1017	0.9619	0.9619
0.089	14.46	4800	0.1029	0.9633	0.9633
0.086	15.06	5000	0.1006	0.9636	0.9636
0.0855	15.66	5200	0.1036	0.9642	0.9642
0.0894	16.27	5400	0.1004	0.9632	0.9633
0.0835	16.87	5600	0.1004	0.9623	0.9623
0.0805	17.47	5800	0.1021	0.9610	0.9610
0.0879	18.07	6000	0.0991	0.9627	0.9627
0.0823	18.67	6200	0.1008	0.9653	0.9653
0.0825	19.28	6400	0.1046	0.9608	0.9608
0.0815	19.88	6600	0.1034	0.9648	0.9648
0.0841	20.48	6800	0.0986	0.9633	0.9633
0.0792	21.08	7000	0.0995	0.9649	0.9650
0.0793	21.69	7200	0.1021	0.9625	0.9625
0.0787	22.29	7400	0.1027	0.9610	0.9610
0.0822	22.89	7600	0.0986	0.9640	0.9640
0.0755	23.49	7800	0.1014	0.9629	0.9629
0.0801	24.1	8000	0.0987	0.9634	0.9634
0.0766	24.7	8200	0.1041	0.9646	0.9646
0.0769	25.3	8400	0.1015	0.9655	0.9655
0.0766	25.9	8600	0.1013	0.9636	0.9636
0.0775	26.51	8800	0.1007	0.9631	0.9631
0.0748	27.11	9000	0.1009	0.9636	0.9636
0.0767	27.71	9200	0.1009	0.9640	0.9640
0.0732	28.31	9400	0.1006	0.9648	0.9648
0.0772	28.92	9600	0.1006	0.9636	0.9636
0.0732	29.52	9800	0.1004	0.9640	0.9640
0.075	30.12	10000	0.1004	0.9644	0.9644

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2