GUE_prom_prom_core_tata-seqsight_4096_512_27M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_core_tata dataset. It achieves the following results on the evaluation set:

• Loss: 0.4158
• F1 Score: 0.8367
• Accuracy: 0.8369

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5875	5.13	200	0.5748	0.7050	0.7080
0.53	10.26	400	0.5527	0.7108	0.7145
0.4871	15.38	600	0.5520	0.7308	0.7374
0.4537	20.51	800	0.4943	0.7524	0.7537
0.4267	25.64	1000	0.4760	0.7633	0.7635
0.4119	30.77	1200	0.4547	0.7793	0.7798
0.3932	35.9	1400	0.4493	0.7926	0.7928
0.3768	41.03	1600	0.4332	0.7943	0.7945
0.3671	46.15	1800	0.4303	0.8074	0.8075
0.3551	51.28	2000	0.4469	0.8020	0.8026
0.3487	56.41	2200	0.4419	0.8120	0.8124
0.3411	61.54	2400	0.4270	0.8189	0.8189
0.3347	66.67	2600	0.4421	0.8118	0.8124
0.3342	71.79	2800	0.4239	0.8254	0.8254
0.324	76.92	3000	0.4416	0.8002	0.8010
0.3203	82.05	3200	0.4325	0.8136	0.8140
0.3129	87.18	3400	0.4325	0.8269	0.8271
0.3088	92.31	3600	0.4201	0.8287	0.8287
0.3016	97.44	3800	0.4261	0.8286	0.8287
0.3019	102.56	4000	0.4237	0.8271	0.8271
0.3016	107.69	4200	0.4335	0.8118	0.8124
0.2978	112.82	4400	0.4227	0.8156	0.8157
0.2942	117.95	4600	0.4463	0.8119	0.8124
0.2842	123.08	4800	0.4366	0.8170	0.8173
0.2877	128.21	5000	0.4306	0.8124	0.8124
0.2805	133.33	5200	0.4267	0.8205	0.8206
0.2838	138.46	5400	0.4198	0.8271	0.8271
0.2801	143.59	5600	0.4294	0.8172	0.8173
0.2791	148.72	5800	0.4394	0.8187	0.8189
0.2711	153.85	6000	0.4366	0.8287	0.8287
0.2731	158.97	6200	0.4305	0.8238	0.8238
0.2681	164.1	6400	0.4437	0.8233	0.8238
0.273	169.23	6600	0.4265	0.8287	0.8287
0.2681	174.36	6800	0.4337	0.8352	0.8352
0.2678	179.49	7000	0.4389	0.8236	0.8238
0.2602	184.62	7200	0.4337	0.8335	0.8336
0.2641	189.74	7400	0.4402	0.8235	0.8238
0.263	194.87	7600	0.4368	0.8253	0.8254
0.2605	200.0	7800	0.4275	0.8222	0.8222
0.2605	205.13	8000	0.4403	0.8203	0.8206
0.2627	210.26	8200	0.4301	0.8237	0.8238
0.2551	215.38	8400	0.4336	0.8204	0.8206
0.2574	220.51	8600	0.4409	0.8170	0.8173
0.2538	225.64	8800	0.4423	0.8220	0.8222
0.2579	230.77	9000	0.4403	0.8220	0.8222
0.2585	235.9	9200	0.4422	0.8186	0.8189
0.257	241.03	9400	0.4378	0.8236	0.8238
0.256	246.15	9600	0.4427	0.8202	0.8206
0.2517	251.28	9800	0.4393	0.8219	0.8222
0.249	256.41	10000	0.4379	0.8236	0.8238

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2