GUE_splice_reconstructed-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_splice_reconstructed dataset. It achieves the following results on the evaluation set:

• Loss: 0.9268
• F1 Score: 0.6104
• Accuracy: 0.6192

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.9721	11.11	200	0.9267	0.4980	0.5971
0.8957	22.22	400	0.8953	0.5500	0.6118
0.8593	33.33	600	0.8799	0.5755	0.6127
0.8267	44.44	800	0.8708	0.5888	0.6072
0.7956	55.56	1000	0.8686	0.5976	0.6125
0.7723	66.67	1200	0.8550	0.5989	0.6230
0.7545	77.78	1400	0.8607	0.6054	0.6247
0.7387	88.89	1600	0.8554	0.6105	0.6228
0.7276	100.0	1800	0.8702	0.6127	0.6184
0.7202	111.11	2000	0.8810	0.6117	0.6289
0.7124	122.22	2200	0.8644	0.6137	0.6252
0.7053	133.33	2400	0.8663	0.6108	0.6219
0.6987	144.44	2600	0.8684	0.6117	0.6241
0.6928	155.56	2800	0.8729	0.6143	0.6249
0.6881	166.67	3000	0.8677	0.6173	0.6278
0.6827	177.78	3200	0.8776	0.6100	0.6162
0.6763	188.89	3400	0.8687	0.6158	0.6315
0.6711	200.0	3600	0.8735	0.6105	0.6317
0.6663	211.11	3800	0.8725	0.6116	0.6326
0.6597	222.22	4000	0.8821	0.6144	0.6265
0.6552	233.33	4200	0.8672	0.6105	0.6245
0.6488	244.44	4400	0.8847	0.6098	0.6282
0.6438	255.56	4600	0.8961	0.6104	0.6225
0.6393	266.67	4800	0.8717	0.6112	0.6263
0.6323	277.78	5000	0.8906	0.6062	0.6285
0.6264	288.89	5200	0.8846	0.6165	0.6278
0.6219	300.0	5400	0.9003	0.6087	0.6293
0.6154	311.11	5600	0.8922	0.6179	0.6337
0.6114	322.22	5800	0.9030	0.6138	0.6285
0.6065	333.33	6000	0.8958	0.6115	0.6219
0.5997	344.44	6200	0.9092	0.6109	0.6265
0.5949	355.56	6400	0.9194	0.6131	0.6263
0.5914	366.67	6600	0.9015	0.6142	0.6258
0.587	377.78	6800	0.9139	0.6155	0.6300
0.5821	388.89	7000	0.9148	0.6151	0.6313
0.5768	400.0	7200	0.8992	0.6140	0.6282
0.5746	411.11	7400	0.9159	0.6131	0.6260
0.5715	422.22	7600	0.9260	0.6165	0.6291
0.5677	433.33	7800	0.9193	0.6164	0.6293
0.5632	444.44	8000	0.9310	0.6127	0.6238
0.5606	455.56	8200	0.9283	0.6201	0.6291
0.5606	466.67	8400	0.9315	0.6165	0.6304
0.5562	477.78	8600	0.9282	0.6156	0.6258
0.5534	488.89	8800	0.9374	0.6155	0.6247
0.5526	500.0	9000	0.9272	0.6155	0.6258
0.552	511.11	9200	0.9341	0.6163	0.6256
0.5487	522.22	9400	0.9343	0.6157	0.6274
0.5478	533.33	9600	0.9328	0.6143	0.6258
0.5469	544.44	9800	0.9341	0.6163	0.6276
0.5472	555.56	10000	0.9338	0.6155	0.6269

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2