GUE_prom_prom_core_all-seqsight_32768_512_43M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_32768_512_43M on the mahdibaghbanzadeh/GUE_prom_prom_core_all dataset. It achieves the following results on the evaluation set:

• Loss: 0.4199
• F1 Score: 0.8070
• Accuracy: 0.8071

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5555	0.54	200	0.4758	0.7774	0.7779
0.4767	1.08	400	0.4572	0.7886	0.7887
0.4563	1.62	600	0.4501	0.7949	0.7949
0.4509	2.16	800	0.4547	0.7884	0.7885
0.4489	2.7	1000	0.4525	0.7882	0.7887
0.445	3.24	1200	0.4484	0.7905	0.7910
0.4429	3.78	1400	0.4511	0.7871	0.7878
0.4348	4.32	1600	0.4540	0.7863	0.7872
0.4345	4.86	1800	0.4499	0.7895	0.7902
0.4338	5.41	2000	0.4474	0.7908	0.7914
0.4304	5.95	2200	0.4445	0.7945	0.7946
0.4344	6.49	2400	0.4385	0.7952	0.7953
0.4264	7.03	2600	0.4390	0.7949	0.7949
0.4301	7.57	2800	0.4420	0.7960	0.7963
0.4222	8.11	3000	0.4452	0.7921	0.7927
0.4248	8.65	3200	0.4342	0.8013	0.8014
0.4263	9.19	3400	0.4370	0.7990	0.7992
0.4228	9.73	3600	0.4425	0.7960	0.7966
0.4249	10.27	3800	0.4392	0.7987	0.7990
0.4195	10.81	4000	0.4414	0.7981	0.7981
0.4209	11.35	4200	0.4423	0.7993	0.7998
0.4208	11.89	4400	0.4417	0.7967	0.7975
0.418	12.43	4600	0.4351	0.8032	0.8032
0.4167	12.97	4800	0.4373	0.7991	0.7995
0.4183	13.51	5000	0.4469	0.7908	0.7919
0.4157	14.05	5200	0.4344	0.8017	0.8019
0.416	14.59	5400	0.4360	0.8029	0.8029
0.4178	15.14	5600	0.4340	0.8032	0.8032
0.4171	15.68	5800	0.4405	0.7979	0.7983
0.4105	16.22	6000	0.4423	0.7991	0.7995
0.4182	16.76	6200	0.4335	0.7993	0.7997
0.4151	17.3	6400	0.4370	0.7992	0.7997
0.4169	17.84	6600	0.4377	0.7986	0.7990
0.4132	18.38	6800	0.4418	0.7956	0.7963
0.4124	18.92	7000	0.4354	0.7996	0.8
0.4086	19.46	7200	0.4377	0.8000	0.8003
0.4164	20.0	7400	0.4349	0.8032	0.8034
0.4164	20.54	7600	0.4379	0.7982	0.7986
0.4095	21.08	7800	0.4377	0.7996	0.8
0.4119	21.62	8000	0.4336	0.8024	0.8025
0.4127	22.16	8200	0.4347	0.8016	0.8019
0.4159	22.7	8400	0.4366	0.7975	0.7980
0.41	23.24	8600	0.4344	0.8003	0.8005
0.4089	23.78	8800	0.4366	0.7993	0.7997
0.4088	24.32	9000	0.4348	0.8035	0.8037
0.4105	24.86	9200	0.4354	0.8009	0.8012
0.4193	25.41	9400	0.4341	0.8007	0.8010
0.4059	25.95	9600	0.4347	0.8016	0.8019
0.4151	26.49	9800	0.4356	0.7996	0.8
0.4067	27.03	10000	0.4354	0.8003	0.8007

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2