GUE_prom_prom_core_tata-seqsight_32768_512_43M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 0.9752
• F1 Score: 0.8271
• Accuracy: 0.8271

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5578	5.13	200	0.5322	0.7502	0.7504
0.464	10.26	400	0.5083	0.7701	0.7716
0.3882	15.38	600	0.4438	0.8074	0.8075
0.3241	20.51	800	0.4506	0.8234	0.8238
0.2722	25.64	1000	0.4721	0.8303	0.8303
0.2338	30.77	1200	0.4767	0.8320	0.8320
0.1976	35.9	1400	0.5198	0.8336	0.8336
0.1754	41.03	1600	0.4998	0.8303	0.8303
0.1428	46.15	1800	0.6118	0.8269	0.8271
0.1281	51.28	2000	0.5731	0.8302	0.8303
0.1127	56.41	2200	0.6563	0.8319	0.8320
0.0994	61.54	2400	0.6877	0.8222	0.8222
0.0901	66.67	2600	0.7150	0.8352	0.8352
0.0817	71.79	2800	0.7223	0.8254	0.8254
0.0725	76.92	3000	0.7396	0.8334	0.8336
0.0663	82.05	3200	0.7565	0.8335	0.8336
0.0601	87.18	3400	0.7511	0.8418	0.8418
0.0589	92.31	3600	0.7803	0.8383	0.8385
0.0521	97.44	3800	0.8330	0.8385	0.8385
0.0525	102.56	4000	0.8002	0.8434	0.8434
0.0466	107.69	4200	0.7893	0.8385	0.8385
0.0414	112.82	4400	0.8864	0.8369	0.8369
0.0385	117.95	4600	0.8732	0.8335	0.8336
0.0402	123.08	4800	0.8392	0.8401	0.8401
0.0382	128.21	5000	0.8185	0.8285	0.8287
0.0384	133.33	5200	0.8188	0.8401	0.8401
0.0334	138.46	5400	0.8668	0.8433	0.8434
0.0297	143.59	5600	0.8826	0.8319	0.8320
0.033	148.72	5800	0.8982	0.8336	0.8336
0.0285	153.85	6000	0.9081	0.8352	0.8352
0.0299	158.97	6200	0.8908	0.8384	0.8385
0.0296	164.1	6400	0.8685	0.8368	0.8369
0.0288	169.23	6600	0.8841	0.8401	0.8401
0.0265	174.36	6800	0.8954	0.8336	0.8336
0.0277	179.49	7000	0.8666	0.8417	0.8418
0.0243	184.62	7200	0.8899	0.8401	0.8401
0.023	189.74	7400	0.8804	0.8418	0.8418
0.0233	194.87	7600	0.9357	0.8401	0.8401
0.0244	200.0	7800	0.8806	0.8401	0.8401
0.0212	205.13	8000	0.9329	0.8385	0.8385
0.022	210.26	8200	0.9356	0.8434	0.8434
0.0212	215.38	8400	0.9286	0.8400	0.8401
0.0205	220.51	8600	0.9201	0.8434	0.8434
0.0215	225.64	8800	0.9130	0.8434	0.8434
0.021	230.77	9000	0.9020	0.8434	0.8434
0.0205	235.9	9200	0.9081	0.8385	0.8385
0.0194	241.03	9400	0.9260	0.8320	0.8320
0.0182	246.15	9600	0.9300	0.8352	0.8352
0.0172	251.28	9800	0.9393	0.8352	0.8352
0.0167	256.41	10000	0.9422	0.8352	0.8352

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2