GUE_prom_prom_core_notata-seqsight_32768_512_43M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_prom_prom_core_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.3840
• F1 Score: 0.8338
• Accuracy: 0.8338

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5472	0.6	200	0.4181	0.8117	0.8119
0.4381	1.2	400	0.4003	0.8190	0.8191
0.4205	1.81	600	0.3911	0.8243	0.8244
0.4179	2.41	800	0.3876	0.8264	0.8266
0.4072	3.01	1000	0.3833	0.8287	0.8289
0.4051	3.61	1200	0.3853	0.8272	0.8276
0.4021	4.22	1400	0.3797	0.8318	0.8319
0.4066	4.82	1600	0.3777	0.8310	0.8312
0.3943	5.42	1800	0.3787	0.8297	0.8297
0.3998	6.02	2000	0.3801	0.8315	0.8319
0.3971	6.63	2200	0.3780	0.8335	0.8336
0.392	7.23	2400	0.3841	0.8294	0.8300
0.3939	7.83	2600	0.3736	0.8331	0.8332
0.3904	8.43	2800	0.3861	0.8293	0.8300
0.3951	9.04	3000	0.3779	0.8299	0.8302
0.387	9.64	3200	0.3752	0.8328	0.8329
0.3886	10.24	3400	0.3737	0.8326	0.8327
0.3848	10.84	3600	0.3716	0.8332	0.8332
0.3857	11.45	3800	0.3736	0.8307	0.8308
0.3849	12.05	4000	0.3704	0.8332	0.8332
0.3814	12.65	4200	0.3767	0.8328	0.8331
0.3859	13.25	4400	0.3726	0.8339	0.8340
0.3851	13.86	4600	0.3712	0.8315	0.8315
0.383	14.46	4800	0.3728	0.8327	0.8329
0.3822	15.06	5000	0.3713	0.8318	0.8319
0.3802	15.66	5200	0.3708	0.8330	0.8331
0.3821	16.27	5400	0.3712	0.8321	0.8321
0.3788	16.87	5600	0.3812	0.8313	0.8319
0.375	17.47	5800	0.3789	0.8334	0.8338
0.385	18.07	6000	0.3745	0.8341	0.8346
0.3775	18.67	6200	0.3698	0.8334	0.8336
0.379	19.28	6400	0.3706	0.8330	0.8331
0.3764	19.88	6600	0.3706	0.8324	0.8327
0.3714	20.48	6800	0.3743	0.8340	0.8344
0.3842	21.08	7000	0.3683	0.8345	0.8347
0.3801	21.69	7200	0.3683	0.8347	0.8347
0.3727	22.29	7400	0.3686	0.8348	0.8349
0.3725	22.89	7600	0.3691	0.8333	0.8334
0.3754	23.49	7800	0.3689	0.8342	0.8344
0.3772	24.1	8000	0.3725	0.8335	0.8338
0.3773	24.7	8200	0.3736	0.8335	0.8340
0.371	25.3	8400	0.3721	0.8337	0.8340
0.379	25.9	8600	0.3688	0.8335	0.8336
0.3786	26.51	8800	0.3682	0.8347	0.8347
0.3773	27.11	9000	0.3680	0.8329	0.8331
0.3799	27.71	9200	0.3692	0.8329	0.8331
0.3689	28.31	9400	0.3715	0.8326	0.8329
0.3744	28.92	9600	0.3692	0.8334	0.8336
0.3783	29.52	9800	0.3690	0.8334	0.8336
0.3679	30.12	10000	0.3695	0.8334	0.8336

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2