GUE_prom_prom_core_notata-seqsight_16384_512_22M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_22M on the mahdibaghbanzadeh/GUE_prom_prom_core_notata dataset. It achieves the following results on the evaluation set:

• Loss: 0.5802
• F1 Score: 0.7160
• Accuracy: 0.7160

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6477	9.52	200	0.5985	0.6851	0.6863
0.5922	19.05	400	0.5761	0.7034	0.7036
0.5682	28.57	600	0.5697	0.7135	0.7138
0.5501	38.1	800	0.5640	0.7177	0.7177
0.5344	47.62	1000	0.5587	0.7222	0.7224
0.5229	57.14	1200	0.5560	0.7313	0.7313
0.5141	66.67	1400	0.5495	0.7339	0.7339
0.5065	76.19	1600	0.5478	0.7364	0.7364
0.499	85.71	1800	0.5484	0.7344	0.7345
0.4927	95.24	2000	0.5577	0.7367	0.7368
0.4874	104.76	2200	0.5572	0.7345	0.7345
0.4795	114.29	2400	0.5518	0.7351	0.7351
0.4759	123.81	2600	0.5569	0.7362	0.7362
0.4712	133.33	2800	0.5571	0.7339	0.7339
0.4664	142.86	3000	0.5575	0.7281	0.7287
0.4608	152.38	3200	0.5622	0.7355	0.7354
0.457	161.9	3400	0.5571	0.7335	0.7336
0.4518	171.43	3600	0.5716	0.7281	0.7287
0.4479	180.95	3800	0.5673	0.7228	0.7239
0.4435	190.48	4000	0.5713	0.7215	0.7221
0.4398	200.0	4200	0.5829	0.7345	0.7345
0.435	209.52	4400	0.5769	0.7265	0.7270
0.4326	219.05	4600	0.5762	0.7282	0.7285
0.4286	228.57	4800	0.5749	0.7311	0.7311
0.4247	238.1	5000	0.5846	0.7303	0.7307
0.4231	247.62	5200	0.5876	0.7311	0.7313
0.4192	257.14	5400	0.5797	0.7317	0.7321
0.4168	266.67	5600	0.5908	0.7295	0.7296
0.4138	276.19	5800	0.6108	0.7205	0.7217
0.4109	285.71	6000	0.5874	0.7271	0.7273
0.4087	295.24	6200	0.6094	0.7274	0.7279
0.4056	304.76	6400	0.6137	0.7237	0.7251
0.4029	314.29	6600	0.5969	0.7229	0.7234
0.4006	323.81	6800	0.6054	0.7284	0.7288
0.3983	333.33	7000	0.6050	0.7279	0.7283
0.3954	342.86	7200	0.6094	0.7223	0.7230
0.3946	352.38	7400	0.6067	0.7260	0.7262
0.3935	361.9	7600	0.6080	0.7259	0.7262
0.3907	371.43	7800	0.6118	0.7259	0.7262
0.3907	380.95	8000	0.6142	0.7264	0.7268
0.3881	390.48	8200	0.6193	0.7239	0.7243
0.3867	400.0	8400	0.6040	0.7234	0.7236
0.3856	409.52	8600	0.6176	0.7213	0.7221
0.3813	419.05	8800	0.6185	0.7230	0.7236
0.3836	428.57	9000	0.6124	0.7203	0.7207
0.3816	438.1	9200	0.6200	0.7243	0.7249
0.3811	447.62	9400	0.6194	0.7227	0.7232
0.3805	457.14	9600	0.6214	0.7218	0.7224
0.3799	466.67	9800	0.6197	0.7208	0.7213
0.3801	476.19	10000	0.6192	0.7211	0.7217

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2