GUE_splice_reconstructed-seqsight_8192_512_30M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_8192_512_30M on the mahdibaghbanzadeh/GUE_splice_reconstructed dataset. It achieves the following results on the evaluation set:

• Loss: 0.9664
• F1 Score: 0.7014
• Accuracy: 0.7030

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.9343	11.11	200	0.8346	0.5809	0.6241
0.7818	22.22	400	0.7676	0.6464	0.6611
0.7135	33.33	600	0.7563	0.6612	0.6620
0.6651	44.44	800	0.7568	0.6691	0.6710
0.627	55.56	1000	0.7487	0.6738	0.6767
0.5948	66.67	1200	0.7433	0.6800	0.6850
0.5688	77.78	1400	0.7494	0.6815	0.6870
0.5437	88.89	1600	0.7587	0.6811	0.6815
0.5212	100.0	1800	0.7816	0.6787	0.6789
0.5014	111.11	2000	0.7846	0.6859	0.6857
0.4824	122.22	2200	0.7872	0.6852	0.6859
0.4664	133.33	2400	0.7993	0.6910	0.6951
0.4508	144.44	2600	0.8120	0.6901	0.6949
0.4369	155.56	2800	0.8342	0.6886	0.6898
0.4241	166.67	3000	0.8359	0.6855	0.6872
0.41	177.78	3200	0.8491	0.6895	0.6903
0.3981	188.89	3400	0.8528	0.6881	0.6874
0.3863	200.0	3600	0.8736	0.6874	0.6914
0.3748	211.11	3800	0.8668	0.6905	0.6940
0.3662	222.22	4000	0.8681	0.6891	0.6896
0.355	233.33	4200	0.8869	0.6923	0.6942
0.3465	244.44	4400	0.8918	0.6886	0.6911
0.3395	255.56	4600	0.9159	0.6880	0.6887
0.3315	266.67	4800	0.9279	0.6934	0.6953
0.3231	277.78	5000	0.9232	0.6917	0.6927
0.3162	288.89	5200	0.9350	0.6913	0.6925
0.3108	300.0	5400	0.9520	0.6959	0.6979
0.3042	311.11	5600	0.9396	0.6899	0.6918
0.3	322.22	5800	0.9521	0.6927	0.6982
0.2898	333.33	6000	0.9616	0.6928	0.6951
0.2887	344.44	6200	0.9716	0.6937	0.6960
0.2819	355.56	6400	0.9720	0.6886	0.6876
0.2774	366.67	6600	0.9822	0.6907	0.6918
0.2727	377.78	6800	0.9960	0.6888	0.6907
0.2694	388.89	7000	0.9855	0.6936	0.6957
0.2635	400.0	7200	0.9976	0.6907	0.6918
0.261	411.11	7400	1.0112	0.6915	0.6938
0.2582	422.22	7600	1.0083	0.6880	0.6885
0.2557	433.33	7800	1.0171	0.6923	0.6936
0.2534	444.44	8000	1.0165	0.6962	0.6977
0.2498	455.56	8200	1.0167	0.6922	0.6929
0.2468	466.67	8400	1.0320	0.6914	0.6927
0.2447	477.78	8600	1.0256	0.6906	0.6916
0.2424	488.89	8800	1.0227	0.6895	0.6903
0.2401	500.0	9000	1.0330	0.6923	0.6938
0.2397	511.11	9200	1.0332	0.6920	0.6933
0.2387	522.22	9400	1.0357	0.6937	0.6951
0.2369	533.33	9600	1.0390	0.6926	0.6940
0.236	544.44	9800	1.0405	0.6924	0.6933
0.2364	555.56	10000	1.0352	0.6910	0.6920

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2