GUE_prom_prom_core_tata-seqsight_32768_512_43M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 0.4468
• F1 Score: 0.8203
• Accuracy: 0.8206

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6029	5.13	200	0.5832	0.6980	0.7015
0.5406	10.26	400	0.5696	0.7163	0.7194
0.5176	15.38	600	0.5599	0.7281	0.7308
0.4955	20.51	800	0.5382	0.7455	0.7455
0.4756	25.64	1000	0.5299	0.7423	0.7423
0.465	30.77	1200	0.5255	0.7438	0.7439
0.4532	35.9	1400	0.5213	0.7534	0.7537
0.4388	41.03	1600	0.5134	0.7548	0.7553
0.4319	46.15	1800	0.5187	0.7551	0.7553
0.4203	51.28	2000	0.5093	0.7683	0.7684
0.4066	56.41	2200	0.5230	0.7714	0.7716
0.4086	61.54	2400	0.4994	0.7716	0.7716
0.4016	66.67	2600	0.5033	0.7667	0.7667
0.391	71.79	2800	0.5018	0.7732	0.7732
0.3842	76.92	3000	0.5181	0.7677	0.7684
0.3755	82.05	3200	0.4979	0.7732	0.7732
0.3695	87.18	3400	0.5117	0.7694	0.7700
0.3637	92.31	3600	0.4982	0.7749	0.7749
0.3508	97.44	3800	0.5016	0.7748	0.7749
0.3503	102.56	4000	0.4929	0.7830	0.7830
0.3429	107.69	4200	0.4888	0.7862	0.7863
0.3379	112.82	4400	0.4902	0.7797	0.7798
0.3324	117.95	4600	0.4944	0.7812	0.7814
0.3301	123.08	4800	0.4942	0.7794	0.7798
0.3202	128.21	5000	0.4894	0.7862	0.7863
0.3263	133.33	5200	0.4753	0.7928	0.7928
0.3215	138.46	5400	0.4740	0.7895	0.7896
0.3123	143.59	5600	0.4865	0.7845	0.7847
0.3151	148.72	5800	0.4858	0.7895	0.7896
0.309	153.85	6000	0.4865	0.7845	0.7847
0.3092	158.97	6200	0.4841	0.7863	0.7863
0.3031	164.1	6400	0.4883	0.7862	0.7863
0.3065	169.23	6600	0.4861	0.7895	0.7896
0.3016	174.36	6800	0.4825	0.7912	0.7912
0.299	179.49	7000	0.4909	0.7974	0.7977
0.2988	184.62	7200	0.4942	0.7975	0.7977
0.296	189.74	7400	0.4839	0.7976	0.7977
0.2923	194.87	7600	0.4837	0.7879	0.7879
0.2932	200.0	7800	0.4832	0.7911	0.7912
0.2949	205.13	8000	0.4968	0.7909	0.7912
0.2924	210.26	8200	0.4875	0.7960	0.7961
0.2963	215.38	8400	0.4904	0.7959	0.7961
0.2914	220.51	8600	0.5002	0.7925	0.7928
0.2892	225.64	8800	0.4993	0.7942	0.7945
0.2917	230.77	9000	0.4928	0.7975	0.7977
0.2858	235.9	9200	0.4917	0.7959	0.7961
0.2924	241.03	9400	0.4853	0.7960	0.7961
0.2868	246.15	9600	0.4926	0.7992	0.7993
0.2873	251.28	9800	0.4913	0.7976	0.7977
0.2875	256.41	10000	0.4899	0.7976	0.7977

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2