GUE_prom_prom_core_notata-seqsight_4096_512_27M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_prom_prom_core_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.3761
• F1 Score: 0.8319
• Accuracy: 0.8319

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.513	0.6	200	0.4105	0.8110	0.8110
0.4285	1.2	400	0.3944	0.8248	0.8248
0.4171	1.81	600	0.3870	0.8242	0.8242
0.4149	2.41	800	0.3811	0.8285	0.8285
0.4023	3.01	1000	0.3781	0.8316	0.8315
0.3974	3.61	1200	0.3764	0.8311	0.8312
0.3996	4.22	1400	0.3751	0.8332	0.8332
0.3985	4.82	1600	0.3719	0.8357	0.8357
0.3867	5.42	1800	0.3783	0.8292	0.8293
0.3902	6.02	2000	0.3708	0.8380	0.8381
0.3882	6.63	2200	0.3686	0.8355	0.8355
0.3873	7.23	2400	0.3708	0.8358	0.8361
0.3839	7.83	2600	0.3672	0.8351	0.8351
0.3785	8.43	2800	0.3707	0.8363	0.8366
0.3835	9.04	3000	0.3676	0.8379	0.8379
0.3774	9.64	3200	0.3665	0.8340	0.8340
0.3786	10.24	3400	0.3659	0.8381	0.8381
0.3766	10.84	3600	0.3652	0.8359	0.8359
0.3782	11.45	3800	0.3643	0.8377	0.8378
0.3758	12.05	4000	0.3644	0.8344	0.8344
0.3733	12.65	4200	0.3650	0.8378	0.8378
0.3766	13.25	4400	0.3643	0.8366	0.8366
0.3782	13.86	4600	0.3645	0.8372	0.8372
0.3719	14.46	4800	0.3645	0.8366	0.8366
0.3725	15.06	5000	0.3664	0.8349	0.8349
0.3686	15.66	5200	0.3636	0.8389	0.8389
0.3675	16.27	5400	0.3659	0.8391	0.8391
0.3702	16.87	5600	0.3658	0.8398	0.8400
0.3663	17.47	5800	0.3657	0.8382	0.8383
0.3736	18.07	6000	0.3640	0.8404	0.8406
0.3679	18.67	6200	0.3627	0.8394	0.8395
0.3682	19.28	6400	0.3647	0.8389	0.8389
0.3685	19.88	6600	0.3632	0.8394	0.8395
0.3622	20.48	6800	0.3645	0.8393	0.8395
0.3736	21.08	7000	0.3627	0.8412	0.8413
0.3691	21.69	7200	0.3637	0.8378	0.8378
0.3628	22.29	7400	0.3633	0.8379	0.8379
0.366	22.89	7600	0.3635	0.8404	0.8404
0.3676	23.49	7800	0.3635	0.8383	0.8383
0.3687	24.1	8000	0.3634	0.8397	0.8398
0.3699	24.7	8200	0.3628	0.8388	0.8389
0.3622	25.3	8400	0.3642	0.8407	0.8408
0.3661	25.9	8600	0.3630	0.8392	0.8393
0.3672	26.51	8800	0.3641	0.8387	0.8387
0.3653	27.11	9000	0.3631	0.8383	0.8383
0.3693	27.71	9200	0.3630	0.8379	0.8379
0.3568	28.31	9400	0.3638	0.8396	0.8396
0.3645	28.92	9600	0.3635	0.8377	0.8378
0.367	29.52	9800	0.3633	0.8375	0.8376
0.3569	30.12	10000	0.3635	0.8377	0.8378

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2