GUE_prom_prom_core_tata-seqsight_16384_512_22M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_22M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 1.2856
• F1 Score: 0.7125
• Accuracy: 0.7129

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5676	66.67	200	0.6626	0.6813	0.6819
0.3539	133.33	400	0.7576	0.6819	0.6819
0.2535	200.0	600	0.8903	0.6920	0.6933
0.2066	266.67	800	0.9645	0.6785	0.6786
0.1824	333.33	1000	0.9986	0.6899	0.6900
0.1673	400.0	1200	1.0135	0.6980	0.6982
0.1528	466.67	1400	1.0369	0.6975	0.6982
0.1427	533.33	1600	1.0808	0.6998	0.6998
0.1303	600.0	1800	1.0985	0.6946	0.6949
0.1212	666.67	2000	1.1009	0.7055	0.7064
0.1113	733.33	2200	1.1829	0.7015	0.7015
0.1057	800.0	2400	1.1634	0.7139	0.7145
0.0966	866.67	2600	1.1133	0.7076	0.7080
0.0919	933.33	2800	1.1767	0.7144	0.7145
0.0861	1000.0	3000	1.1778	0.7128	0.7129
0.0809	1066.67	3200	1.2290	0.7142	0.7145
0.0749	1133.33	3400	1.2717	0.7112	0.7113
0.0693	1200.0	3600	1.1900	0.7338	0.7341
0.0659	1266.67	3800	1.2033	0.7418	0.7423
0.061	1333.33	4000	1.2243	0.7323	0.7325
0.0579	1400.0	4200	1.2337	0.7194	0.7194
0.0537	1466.67	4400	1.2379	0.7292	0.7292
0.0506	1533.33	4600	1.3006	0.7322	0.7325
0.0485	1600.0	4800	1.3530	0.7259	0.7259
0.0454	1666.67	5000	1.3203	0.7274	0.7276
0.0433	1733.33	5200	1.2862	0.7307	0.7308
0.0415	1800.0	5400	1.3767	0.7341	0.7341
0.0388	1866.67	5600	1.3645	0.7292	0.7292
0.0382	1933.33	5800	1.3704	0.7357	0.7357
0.0354	2000.0	6000	1.4379	0.7357	0.7357
0.0352	2066.67	6200	1.3992	0.7322	0.7325
0.0337	2133.33	6400	1.3997	0.7341	0.7341
0.0322	2200.0	6600	1.3643	0.7275	0.7276
0.0319	2266.67	6800	1.4137	0.7341	0.7341
0.03	2333.33	7000	1.4727	0.7275	0.7276
0.0294	2400.0	7200	1.4124	0.7308	0.7308
0.029	2466.67	7400	1.3733	0.7259	0.7259
0.028	2533.33	7600	1.4484	0.7276	0.7276
0.0276	2600.0	7800	1.3802	0.7406	0.7406
0.0265	2666.67	8000	1.4590	0.7259	0.7259
0.0262	2733.33	8200	1.5033	0.7308	0.7308
0.0256	2800.0	8400	1.4550	0.7276	0.7276
0.0242	2866.67	8600	1.4723	0.7324	0.7325
0.0248	2933.33	8800	1.4258	0.7276	0.7276
0.025	3000.0	9000	1.4105	0.7341	0.7341
0.0238	3066.67	9200	1.4746	0.7308	0.7308
0.0239	3133.33	9400	1.4528	0.7325	0.7325
0.0235	3200.0	9600	1.4520	0.7357	0.7357
0.0231	3266.67	9800	1.4640	0.7325	0.7325
0.0221	3333.33	10000	1.4663	0.7308	0.7308

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2