GUE_prom_prom_core_tata-seqsight_4096_512_46M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_46M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 0.6392
• F1 Score: 0.8303
• Accuracy: 0.8303

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5536	5.13	200	0.5507	0.7178	0.7178
0.4771	10.26	400	0.4528	0.7846	0.7847
0.3954	15.38	600	0.4052	0.8091	0.8091
0.3501	20.51	800	0.4084	0.8120	0.8124
0.3223	25.64	1000	0.4058	0.8278	0.8287
0.2912	30.77	1200	0.4098	0.8314	0.8320
0.2756	35.9	1400	0.3914	0.8384	0.8385
0.2552	41.03	1600	0.3971	0.8350	0.8352
0.2373	46.15	1800	0.4074	0.8365	0.8369
0.2217	51.28	2000	0.4023	0.8352	0.8352
0.2042	56.41	2200	0.4607	0.8334	0.8336
0.1924	61.54	2400	0.4388	0.8286	0.8287
0.1848	66.67	2600	0.4548	0.8349	0.8352
0.1709	71.79	2800	0.4728	0.8366	0.8369
0.1558	76.92	3000	0.4994	0.8352	0.8352
0.1493	82.05	3200	0.5037	0.8352	0.8352
0.1371	87.18	3400	0.5434	0.8401	0.8401
0.1331	92.31	3600	0.5410	0.8221	0.8222
0.1206	97.44	3800	0.5585	0.8432	0.8434
0.1183	102.56	4000	0.5698	0.8416	0.8418
0.1081	107.69	4200	0.5582	0.8417	0.8418
0.105	112.82	4400	0.6159	0.8401	0.8401
0.0991	117.95	4600	0.6073	0.8368	0.8369
0.094	123.08	4800	0.6109	0.8254	0.8254
0.0881	128.21	5000	0.6315	0.8352	0.8352
0.0883	133.33	5200	0.6070	0.8401	0.8401
0.0805	138.46	5400	0.6284	0.8433	0.8434
0.076	143.59	5600	0.6523	0.8319	0.8320
0.0798	148.72	5800	0.6554	0.8401	0.8401
0.0728	153.85	6000	0.6709	0.8466	0.8467
0.0701	158.97	6200	0.6738	0.8449	0.8450
0.0679	164.1	6400	0.6782	0.8417	0.8418
0.0687	169.23	6600	0.6762	0.8434	0.8434
0.0611	174.36	6800	0.6971	0.8368	0.8369
0.0628	179.49	7000	0.7038	0.8352	0.8352
0.0577	184.62	7200	0.6977	0.8368	0.8369
0.0569	189.74	7400	0.6989	0.8450	0.8450
0.0579	194.87	7600	0.6972	0.8450	0.8450
0.0572	200.0	7800	0.7021	0.8416	0.8418
0.0567	205.13	8000	0.7044	0.8320	0.8320
0.0549	210.26	8200	0.7075	0.8433	0.8434
0.0493	215.38	8400	0.7109	0.8369	0.8369
0.0514	220.51	8600	0.7240	0.8336	0.8336
0.0511	225.64	8800	0.7316	0.8401	0.8401
0.05	230.77	9000	0.7390	0.8418	0.8418
0.0501	235.9	9200	0.7306	0.8385	0.8385
0.0506	241.03	9400	0.7358	0.8401	0.8401
0.0482	246.15	9600	0.7364	0.8418	0.8418
0.0464	251.28	9800	0.7357	0.8401	0.8401
0.0482	256.41	10000	0.7352	0.8434	0.8434

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2