GUE_prom_prom_core_tata-seqsight_4096_512_27M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 0.7712
• F1 Score: 0.8254
• Accuracy: 0.8254

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5463	5.13	200	0.5140	0.7355	0.7357
0.4209	10.26	400	0.4571	0.7812	0.7830
0.3466	15.38	600	0.3850	0.8271	0.8271
0.3056	20.51	800	0.3829	0.8448	0.8450
0.2597	25.64	1000	0.4199	0.8269	0.8271
0.2279	30.77	1200	0.4251	0.8336	0.8336
0.1888	35.9	1400	0.4467	0.8418	0.8418
0.1625	41.03	1600	0.4917	0.8237	0.8238
0.1397	46.15	1800	0.5283	0.8251	0.8254
0.1145	51.28	2000	0.5479	0.8351	0.8352
0.1062	56.41	2200	0.5837	0.8384	0.8385
0.093	61.54	2400	0.6136	0.8434	0.8434
0.0849	66.67	2600	0.6030	0.8515	0.8515
0.0737	71.79	2800	0.6642	0.8433	0.8434
0.0679	76.92	3000	0.7257	0.8310	0.8320
0.0638	82.05	3200	0.7174	0.8464	0.8467
0.0594	87.18	3400	0.6558	0.8416	0.8418
0.0567	92.31	3600	0.6852	0.8332	0.8336
0.0505	97.44	3800	0.6678	0.8498	0.8499
0.0453	102.56	4000	0.7559	0.8315	0.8320
0.0467	107.69	4200	0.7465	0.8410	0.8418
0.0454	112.82	4400	0.7221	0.8515	0.8515
0.0393	117.95	4600	0.7106	0.8515	0.8515
0.0382	123.08	4800	0.8130	0.8247	0.8254
0.0353	128.21	5000	0.7361	0.8499	0.8499
0.0366	133.33	5200	0.7672	0.8432	0.8434
0.033	138.46	5400	0.7653	0.8499	0.8499
0.0304	143.59	5600	0.8166	0.8482	0.8483
0.0326	148.72	5800	0.8561	0.8345	0.8352
0.0309	153.85	6000	0.8551	0.8366	0.8369
0.0294	158.97	6200	0.8265	0.8398	0.8401
0.0249	164.1	6400	0.8584	0.8362	0.8369
0.0261	169.23	6600	0.7970	0.8482	0.8483
0.0258	174.36	6800	0.7971	0.8417	0.8418
0.0245	179.49	7000	0.8322	0.8332	0.8336
0.024	184.62	7200	0.8219	0.8465	0.8467
0.0252	189.74	7400	0.8064	0.8384	0.8385
0.0238	194.87	7600	0.8080	0.8513	0.8515
0.0227	200.0	7800	0.8130	0.8466	0.8467
0.0237	205.13	8000	0.8048	0.8417	0.8418
0.0229	210.26	8200	0.7948	0.8417	0.8418
0.0218	215.38	8400	0.7989	0.8499	0.8499
0.0173	220.51	8600	0.8605	0.8432	0.8434
0.0197	225.64	8800	0.8345	0.8449	0.8450
0.018	230.77	9000	0.8549	0.8483	0.8483
0.0197	235.9	9200	0.8607	0.8449	0.8450
0.0192	241.03	9400	0.8476	0.8416	0.8418
0.0175	246.15	9600	0.8688	0.8350	0.8352
0.0181	251.28	9800	0.8570	0.8465	0.8467
0.0177	256.41	10000	0.8566	0.8432	0.8434

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2