GUE_prom_prom_core_notata-seqsight_32768_512_43M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_prom_prom_core_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.3860
• F1 Score: 0.8313
• Accuracy: 0.8314

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.4902	0.6	200	0.3884	0.8259	0.8259
0.4053	1.2	400	0.3797	0.8339	0.8342
0.3903	1.81	600	0.3945	0.8235	0.8244
0.3882	2.41	800	0.3731	0.8377	0.8379
0.3811	3.01	1000	0.3734	0.8361	0.8366
0.3737	3.61	1200	0.3654	0.8376	0.8378
0.3779	4.22	1400	0.3625	0.8389	0.8389
0.3767	4.82	1600	0.3628	0.8380	0.8381
0.3617	5.42	1800	0.3680	0.8387	0.8387
0.37	6.02	2000	0.3670	0.8377	0.8379
0.3637	6.63	2200	0.3608	0.8407	0.8408
0.3596	7.23	2400	0.3738	0.8340	0.8346
0.3578	7.83	2600	0.3667	0.8380	0.8379
0.3545	8.43	2800	0.3747	0.8374	0.8379
0.3584	9.04	3000	0.3673	0.8394	0.8395
0.3481	9.64	3200	0.3652	0.8387	0.8387
0.3498	10.24	3400	0.3640	0.8411	0.8412
0.3455	10.84	3600	0.3607	0.8394	0.8395
0.3435	11.45	3800	0.3607	0.8385	0.8385
0.3419	12.05	4000	0.3671	0.8397	0.8396
0.335	12.65	4200	0.3724	0.8379	0.8379
0.3397	13.25	4400	0.3717	0.8371	0.8372
0.3396	13.86	4600	0.3731	0.8393	0.8395
0.3337	14.46	4800	0.3753	0.8361	0.8364
0.3357	15.06	5000	0.3635	0.8403	0.8404
0.3269	15.66	5200	0.3699	0.8403	0.8404
0.3319	16.27	5400	0.3785	0.8403	0.8404
0.3289	16.87	5600	0.3847	0.8364	0.8370
0.3236	17.47	5800	0.3771	0.8395	0.8396
0.3314	18.07	6000	0.3719	0.8401	0.8404
0.3246	18.67	6200	0.3693	0.8448	0.8449
0.3216	19.28	6400	0.3742	0.8404	0.8404
0.3206	19.88	6600	0.3721	0.8375	0.8378
0.3143	20.48	6800	0.3731	0.8386	0.8387
0.3233	21.08	7000	0.3797	0.8370	0.8374
0.3197	21.69	7200	0.3799	0.8373	0.8374
0.3108	22.29	7400	0.3766	0.8383	0.8385
0.3106	22.89	7600	0.3814	0.8365	0.8368
0.3089	23.49	7800	0.3778	0.8389	0.8391
0.3158	24.1	8000	0.3849	0.8356	0.8359
0.3121	24.7	8200	0.3848	0.8352	0.8357
0.306	25.3	8400	0.3883	0.8365	0.8368
0.3119	25.9	8600	0.3806	0.8370	0.8372
0.3095	26.51	8800	0.3817	0.8365	0.8366
0.311	27.11	9000	0.3797	0.8392	0.8393
0.3079	27.71	9200	0.3860	0.8368	0.8370
0.2988	28.31	9400	0.3883	0.8370	0.8374
0.3086	28.92	9600	0.3826	0.8380	0.8381
0.3066	29.52	9800	0.3831	0.8372	0.8374
0.3023	30.12	10000	0.3839	0.8376	0.8378

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2