GUE_prom_prom_core_tata-seqsight_4096_512_27M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 0.4048
• F1 Score: 0.8284
• Accuracy: 0.8287

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5669	5.13	200	0.5362	0.7234	0.7243
0.4686	10.26	400	0.5079	0.7520	0.7553
0.4079	15.38	600	0.4564	0.7851	0.7863
0.3718	20.51	800	0.4229	0.8108	0.8108
0.3403	25.64	1000	0.4323	0.8105	0.8108
0.3226	30.77	1200	0.4169	0.8189	0.8189
0.299	35.9	1400	0.4195	0.8319	0.8320
0.286	41.03	1600	0.4204	0.8364	0.8369
0.2714	46.15	1800	0.4206	0.8320	0.8320
0.2548	51.28	2000	0.4415	0.8170	0.8173
0.2454	56.41	2200	0.4503	0.8219	0.8222
0.2378	61.54	2400	0.4227	0.8320	0.8320
0.2271	66.67	2600	0.4641	0.8267	0.8271
0.2226	71.79	2800	0.4556	0.8335	0.8336
0.2052	76.92	3000	0.5019	0.8199	0.8206
0.1932	82.05	3200	0.4784	0.8302	0.8303
0.184	87.18	3400	0.5076	0.8299	0.8303
0.1753	92.31	3600	0.5294	0.8249	0.8254
0.1677	97.44	3800	0.5041	0.8302	0.8303
0.1612	102.56	4000	0.5040	0.8270	0.8271
0.1543	107.69	4200	0.5714	0.8214	0.8222
0.1509	112.82	4400	0.5209	0.8302	0.8303
0.1397	117.95	4600	0.5513	0.8219	0.8222
0.1372	123.08	4800	0.5749	0.8232	0.8238
0.1294	128.21	5000	0.5562	0.8235	0.8238
0.1263	133.33	5200	0.5656	0.8302	0.8303
0.1208	138.46	5400	0.5864	0.8286	0.8287
0.114	143.59	5600	0.6225	0.8134	0.8140
0.1147	148.72	5800	0.6308	0.8216	0.8222
0.1099	153.85	6000	0.6045	0.8253	0.8254
0.107	158.97	6200	0.6583	0.8200	0.8206
0.1038	164.1	6400	0.6717	0.8198	0.8206
0.1012	169.23	6600	0.6425	0.8202	0.8206
0.1005	174.36	6800	0.6677	0.8217	0.8222
0.0968	179.49	7000	0.6629	0.8154	0.8157
0.093	184.62	7200	0.6758	0.8219	0.8222
0.0951	189.74	7400	0.6438	0.8252	0.8254
0.089	194.87	7600	0.6909	0.8186	0.8189
0.0879	200.0	7800	0.6710	0.8172	0.8173
0.0873	205.13	8000	0.6793	0.8251	0.8254
0.0913	210.26	8200	0.6639	0.8205	0.8206
0.0847	215.38	8400	0.6647	0.8205	0.8206
0.0833	220.51	8600	0.7092	0.8118	0.8124
0.0832	225.64	8800	0.6935	0.8137	0.8140
0.0826	230.77	9000	0.6918	0.8154	0.8157
0.0869	235.9	9200	0.6959	0.8136	0.8140
0.0809	241.03	9400	0.6956	0.8203	0.8206
0.0816	246.15	9600	0.7071	0.8136	0.8140
0.0804	251.28	9800	0.6933	0.8203	0.8206
0.0769	256.41	10000	0.6983	0.8187	0.8189

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2