GUE_prom_prom_core_notata-seqsight_4096_512_27M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_core_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.3849
• F1 Score: 0.8389
• Accuracy: 0.8389

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.4752	0.6	200	0.3927	0.8251	0.8253
0.3995	1.2	400	0.3736	0.8333	0.8332
0.3885	1.81	600	0.3747	0.8361	0.8363
0.3838	2.41	800	0.3656	0.8376	0.8376
0.3771	3.01	1000	0.3656	0.8401	0.8402
0.3686	3.61	1200	0.3635	0.8391	0.8393
0.3736	4.22	1400	0.3639	0.8391	0.8391
0.3697	4.82	1600	0.3631	0.8403	0.8404
0.3546	5.42	1800	0.3739	0.8358	0.8359
0.3609	6.02	2000	0.3671	0.8366	0.8368
0.354	6.63	2200	0.3606	0.8421	0.8421
0.3516	7.23	2400	0.3677	0.8441	0.8444
0.3522	7.83	2600	0.3630	0.8398	0.8398
0.3443	8.43	2800	0.3652	0.8434	0.8436
0.3496	9.04	3000	0.3636	0.8411	0.8412
0.3398	9.64	3200	0.3654	0.8408	0.8408
0.3375	10.24	3400	0.3706	0.8428	0.8428
0.3367	10.84	3600	0.3582	0.8438	0.8438
0.334	11.45	3800	0.3623	0.8432	0.8432
0.336	12.05	4000	0.3645	0.8432	0.8432
0.3294	12.65	4200	0.3638	0.8441	0.8442
0.3333	13.25	4400	0.3661	0.8449	0.8449
0.3318	13.86	4600	0.3664	0.8444	0.8444
0.3246	14.46	4800	0.3698	0.8442	0.8442
0.3244	15.06	5000	0.3620	0.8461	0.8461
0.316	15.66	5200	0.3694	0.8449	0.8449
0.3185	16.27	5400	0.3725	0.8453	0.8453
0.3206	16.87	5600	0.3702	0.8444	0.8447
0.3126	17.47	5800	0.3728	0.8432	0.8432
0.3201	18.07	6000	0.3708	0.8416	0.8417
0.3123	18.67	6200	0.3676	0.8472	0.8472
0.3133	19.28	6400	0.3782	0.8417	0.8417
0.3101	19.88	6600	0.3693	0.8466	0.8466
0.3041	20.48	6800	0.3739	0.8453	0.8453
0.3139	21.08	7000	0.3737	0.8423	0.8425
0.3097	21.69	7200	0.3740	0.8427	0.8427
0.302	22.29	7400	0.3712	0.8466	0.8466
0.3033	22.89	7600	0.3771	0.8419	0.8419
0.3045	23.49	7800	0.3736	0.8452	0.8453
0.3038	24.1	8000	0.3799	0.8416	0.8417
0.3031	24.7	8200	0.3794	0.8425	0.8427
0.2975	25.3	8400	0.3820	0.8435	0.8436
0.3013	25.9	8600	0.3777	0.8447	0.8447
0.3009	26.51	8800	0.3792	0.8413	0.8413
0.2994	27.11	9000	0.3782	0.8474	0.8474
0.3003	27.71	9200	0.3807	0.8447	0.8447
0.2913	28.31	9400	0.3808	0.8452	0.8453
0.2949	28.92	9600	0.3821	0.8439	0.8440
0.2986	29.52	9800	0.3807	0.8441	0.8442
0.2918	30.12	10000	0.3810	0.8441	0.8442

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2