GUE_prom_prom_core_tata-seqsight_4096_512_46M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_46M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 0.4677
• F1 Score: 0.8303
• Accuracy: 0.8303

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.564	5.13	200	0.5603	0.7059	0.7064
0.5228	10.26	400	0.5456	0.7313	0.7325
0.4936	15.38	600	0.5083	0.7529	0.7537
0.459	20.51	800	0.4685	0.7673	0.7684
0.4227	25.64	1000	0.4269	0.8025	0.8026
0.3929	30.77	1200	0.4184	0.8203	0.8206
0.3703	35.9	1400	0.4158	0.8204	0.8206
0.3566	41.03	1600	0.3927	0.8400	0.8401
0.3452	46.15	1800	0.3935	0.8385	0.8385
0.33	51.28	2000	0.3986	0.8368	0.8369
0.3209	56.41	2200	0.3908	0.8433	0.8434
0.3114	61.54	2400	0.3818	0.8449	0.8450
0.3025	66.67	2600	0.3809	0.8531	0.8532
0.2974	71.79	2800	0.3810	0.8515	0.8515
0.278	76.92	3000	0.3911	0.8548	0.8548
0.2771	82.05	3200	0.3951	0.8385	0.8385
0.2645	87.18	3400	0.4001	0.8434	0.8434
0.2592	92.31	3600	0.4055	0.8562	0.8564
0.2448	97.44	3800	0.4128	0.8513	0.8515
0.2415	102.56	4000	0.4101	0.8531	0.8532
0.2343	107.69	4200	0.4071	0.8449	0.8450
0.2232	112.82	4400	0.4219	0.8463	0.8467
0.2209	117.95	4600	0.4118	0.8514	0.8515
0.2116	123.08	4800	0.4258	0.8532	0.8532
0.2072	128.21	5000	0.4340	0.8578	0.8581
0.2006	133.33	5200	0.4217	0.8547	0.8548
0.1946	138.46	5400	0.4435	0.8430	0.8434
0.185	143.59	5600	0.4495	0.8482	0.8483
0.183	148.72	5800	0.4562	0.8399	0.8401
0.1738	153.85	6000	0.4683	0.8495	0.8499
0.1735	158.97	6200	0.4558	0.8546	0.8548
0.17	164.1	6400	0.4687	0.8564	0.8564
0.1651	169.23	6600	0.4706	0.8531	0.8532
0.1628	174.36	6800	0.4622	0.8515	0.8515
0.1592	179.49	7000	0.4657	0.8579	0.8581
0.1568	184.62	7200	0.4697	0.8564	0.8564
0.1531	189.74	7400	0.4754	0.8515	0.8515
0.1519	194.87	7600	0.4839	0.8481	0.8483
0.1456	200.0	7800	0.4810	0.8513	0.8515
0.1439	205.13	8000	0.4818	0.8433	0.8434
0.1409	210.26	8200	0.4847	0.8433	0.8434
0.1398	215.38	8400	0.4923	0.8481	0.8483
0.1384	220.51	8600	0.4877	0.8482	0.8483
0.1407	225.64	8800	0.4909	0.8400	0.8401
0.1375	230.77	9000	0.4941	0.8481	0.8483
0.1377	235.9	9200	0.4932	0.8450	0.8450
0.1371	241.03	9400	0.4942	0.8449	0.8450
0.1392	246.15	9600	0.4937	0.8417	0.8418
0.1329	251.28	9800	0.4935	0.8465	0.8467
0.1306	256.41	10000	0.4939	0.8481	0.8483

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2