GUE_prom_prom_core_tata-seqsight_32768_512_43M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 0.6247
• F1 Score: 0.8222
• Accuracy: 0.8222

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5763	5.13	200	0.5555	0.7217	0.7227
0.498	10.26	400	0.5365	0.7505	0.7520
0.4604	15.38	600	0.5318	0.7472	0.7488
0.4267	20.51	800	0.4895	0.7798	0.7798
0.3931	25.64	1000	0.4848	0.7749	0.7749
0.362	30.77	1200	0.4607	0.8057	0.8059
0.338	35.9	1400	0.4576	0.8026	0.8026
0.315	41.03	1600	0.4507	0.8006	0.8010
0.2968	46.15	1800	0.4532	0.8140	0.8140
0.2813	51.28	2000	0.4684	0.8087	0.8091
0.2655	56.41	2200	0.4970	0.8123	0.8124
0.2577	61.54	2400	0.4923	0.8007	0.8010
0.2449	66.67	2600	0.4722	0.8204	0.8206
0.2349	71.79	2800	0.4885	0.8173	0.8173
0.2217	76.92	3000	0.5013	0.8172	0.8173
0.2111	82.05	3200	0.5198	0.8205	0.8206
0.2005	87.18	3400	0.5395	0.8170	0.8173
0.1939	92.31	3600	0.5382	0.8123	0.8124
0.1867	97.44	3800	0.5531	0.8254	0.8254
0.1777	102.56	4000	0.5748	0.8187	0.8189
0.171	107.69	4200	0.5901	0.8138	0.8140
0.1625	112.82	4400	0.5725	0.8222	0.8222
0.1571	117.95	4600	0.5986	0.8157	0.8157
0.1574	123.08	4800	0.6007	0.8138	0.8140
0.1467	128.21	5000	0.6231	0.8169	0.8173
0.1462	133.33	5200	0.5896	0.8204	0.8206
0.1371	138.46	5400	0.6265	0.8222	0.8222
0.1308	143.59	5600	0.6411	0.8253	0.8254
0.1304	148.72	5800	0.6175	0.8254	0.8254
0.1274	153.85	6000	0.6336	0.8205	0.8206
0.1276	158.97	6200	0.6744	0.8155	0.8157
0.1225	164.1	6400	0.6494	0.8220	0.8222
0.1239	169.23	6600	0.6373	0.8124	0.8124
0.1165	174.36	6800	0.6363	0.8238	0.8238
0.1151	179.49	7000	0.6376	0.8302	0.8303
0.1117	184.62	7200	0.6631	0.8173	0.8173
0.1078	189.74	7400	0.6730	0.8270	0.8271
0.1058	194.87	7600	0.6678	0.8271	0.8271
0.1015	200.0	7800	0.6791	0.8254	0.8254
0.104	205.13	8000	0.6991	0.8186	0.8189
0.1034	210.26	8200	0.6741	0.8189	0.8189
0.1026	215.38	8400	0.6680	0.8287	0.8287
0.1	220.51	8600	0.6933	0.8171	0.8173
0.0987	225.64	8800	0.6859	0.8254	0.8254
0.0976	230.77	9000	0.6847	0.8254	0.8254
0.0966	235.9	9200	0.6927	0.8237	0.8238
0.0968	241.03	9400	0.6888	0.8238	0.8238
0.0931	246.15	9600	0.6931	0.8253	0.8254
0.0906	251.28	9800	0.6998	0.8254	0.8254
0.0916	256.41	10000	0.6957	0.8254	0.8254

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2