GUE_mouse_1-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_mouse_1 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4685
• F1 Score: 0.7955
• Accuracy: 0.7967

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5946	7.41	200	0.5181	0.7242	0.7305
0.5101	14.81	400	0.4879	0.7543	0.7563
0.4823	22.22	600	0.4674	0.7718	0.7739
0.4587	29.63	800	0.4527	0.7799	0.7803
0.4441	37.04	1000	0.4451	0.7834	0.7844
0.4322	44.44	1200	0.4384	0.7897	0.7910
0.4236	51.85	1400	0.4384	0.7863	0.7887
0.4145	59.26	1600	0.4270	0.7957	0.7967
0.4093	66.67	1800	0.4280	0.7965	0.7973
0.4045	74.07	2000	0.4224	0.7995	0.8004
0.4009	81.48	2200	0.4181	0.8005	0.8013
0.3944	88.89	2400	0.4210	0.8015	0.8022
0.3899	96.3	2600	0.4220	0.8012	0.8015
0.3871	103.7	2800	0.4174	0.8024	0.8037
0.3805	111.11	3000	0.4174	0.8031	0.8036
0.378	118.52	3200	0.4160	0.8051	0.8065
0.3719	125.93	3400	0.4182	0.8055	0.8059
0.3695	133.33	3600	0.4261	0.8060	0.8068
0.3638	140.74	3800	0.4232	0.8031	0.8040
0.362	148.15	4000	0.4271	0.8062	0.8074
0.3568	155.56	4200	0.4268	0.8038	0.8050
0.3529	162.96	4400	0.4247	0.8063	0.8071
0.3499	170.37	4600	0.4262	0.8044	0.8058
0.3461	177.78	4800	0.4247	0.8064	0.8077
0.3431	185.19	5000	0.4315	0.8053	0.8064
0.3406	192.59	5200	0.4328	0.8048	0.8064
0.337	200.0	5400	0.4297	0.8052	0.8062
0.3335	207.41	5600	0.4345	0.8050	0.8061
0.3313	214.81	5800	0.4340	0.8036	0.8050
0.3277	222.22	6000	0.4359	0.8052	0.8062
0.3277	229.63	6200	0.4252	0.8040	0.8050
0.3244	237.04	6400	0.4326	0.8062	0.8070
0.3226	244.44	6600	0.4417	0.8054	0.8064
0.3193	251.85	6800	0.4428	0.8053	0.8062
0.3182	259.26	7000	0.4430	0.8062	0.8073
0.3162	266.67	7200	0.4372	0.8072	0.8082
0.3143	274.07	7400	0.4376	0.8049	0.8062
0.312	281.48	7600	0.4419	0.8050	0.8061
0.3118	288.89	7800	0.4416	0.8048	0.8058
0.3104	296.3	8000	0.4388	0.8055	0.8065
0.3078	303.7	8200	0.4407	0.8056	0.8065
0.307	311.11	8400	0.4355	0.8062	0.8070
0.3049	318.52	8600	0.4499	0.8067	0.8079
0.3044	325.93	8800	0.4435	0.8064	0.8076
0.3042	333.33	9000	0.4443	0.8077	0.8086
0.3027	340.74	9200	0.4471	0.8078	0.8089
0.3022	348.15	9400	0.4483	0.8054	0.8067
0.3024	355.56	9600	0.4446	0.8067	0.8077
0.3018	362.96	9800	0.4455	0.8065	0.8076
0.3005	370.37	10000	0.4465	0.8069	0.8080

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2