GUE_mouse_3-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_mouse_3 dataset. It achieves the following results on the evaluation set:

• Loss: 2.2061
• F1 Score: 0.6943
• Accuracy: 0.6946

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.3842	200.0	200	1.2987	0.6440	0.6444
0.0869	400.0	400	1.5612	0.6778	0.6778
0.043	600.0	600	1.7566	0.6736	0.6736
0.0282	800.0	800	1.8686	0.6904	0.6904
0.0205	1000.0	1000	2.0206	0.6860	0.6862
0.0168	1200.0	1200	1.9966	0.6945	0.6946
0.0141	1400.0	1400	2.0739	0.7105	0.7113
0.0119	1600.0	1600	2.1799	0.6987	0.6987
0.0102	1800.0	1800	2.2821	0.6862	0.6862
0.0094	2000.0	2000	2.2383	0.6778	0.6778
0.0085	2200.0	2200	2.2737	0.6818	0.6820
0.0078	2400.0	2400	2.3127	0.6987	0.6987
0.007	2600.0	2600	2.3584	0.6903	0.6904
0.0065	2800.0	2800	2.2929	0.6945	0.6946
0.0066	3000.0	3000	2.3720	0.7105	0.7113
0.0062	3200.0	3200	2.3486	0.7111	0.7113
0.0059	3400.0	3400	2.3332	0.6902	0.6904
0.0061	3600.0	3600	2.5363	0.6860	0.6862
0.0051	3800.0	3800	2.4039	0.6820	0.6820
0.0049	4000.0	4000	2.4326	0.6820	0.6820
0.0048	4200.0	4200	2.6798	0.6980	0.6987
0.0048	4400.0	4400	2.6345	0.6694	0.6695
0.0046	4600.0	4600	2.4664	0.6980	0.6987
0.0042	4800.0	4800	2.4284	0.6986	0.6987
0.0043	5000.0	5000	2.4493	0.6858	0.6862
0.0038	5200.0	5200	2.6022	0.6943	0.6946
0.0041	5400.0	5400	2.5375	0.6940	0.6946
0.0039	5600.0	5600	2.4498	0.6736	0.6736
0.0036	5800.0	5800	2.5372	0.6860	0.6862
0.0032	6000.0	6000	2.9134	0.6735	0.6736
0.0034	6200.0	6200	2.6953	0.6695	0.6695
0.0031	6400.0	6400	2.7115	0.6736	0.6736
0.0032	6600.0	6600	2.7506	0.7070	0.7071
0.0031	6800.0	6800	2.8463	0.6903	0.6904
0.0032	7000.0	7000	2.6918	0.6904	0.6904
0.0028	7200.0	7200	2.7421	0.7028	0.7029
0.0029	7400.0	7400	2.5392	0.6819	0.6820
0.0028	7600.0	7600	2.7772	0.6819	0.6820
0.0026	7800.0	7800	2.9030	0.6901	0.6904
0.0025	8000.0	8000	2.8849	0.6903	0.6904
0.0026	8200.0	8200	2.9484	0.6903	0.6904
0.0025	8400.0	8400	2.8952	0.6862	0.6862
0.0023	8600.0	8600	2.8349	0.6778	0.6778
0.0025	8800.0	8800	2.9036	0.6862	0.6862
0.0024	9000.0	9000	2.9186	0.6986	0.6987
0.0022	9200.0	9200	2.9094	0.6778	0.6778
0.0023	9400.0	9400	2.9813	0.6861	0.6862
0.0022	9600.0	9600	2.9257	0.6904	0.6904
0.0022	9800.0	9800	2.9049	0.6945	0.6946
0.0021	10000.0	10000	2.9241	0.6945	0.6946

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2