GUE_EMP_H3-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_EMP_H3 dataset. It achieves the following results on the evaluation set:

• Loss: 1.1778
• F1 Score: 0.7101
• Accuracy: 0.7101

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6193	33.33	200	0.6140	0.6798	0.6800
0.4917	66.67	400	0.6637	0.6767	0.6767
0.4223	100.0	600	0.7294	0.6661	0.6667
0.375	133.33	800	0.7597	0.6725	0.6727
0.3486	166.67	1000	0.7553	0.6767	0.6767
0.3312	200.0	1200	0.7957	0.6780	0.6780
0.3149	233.33	1400	0.8104	0.6864	0.6874
0.2992	266.67	1600	0.8862	0.6800	0.6800
0.2846	300.0	1800	0.9101	0.6755	0.6760
0.2711	333.33	2000	0.8681	0.6819	0.6820
0.2579	366.67	2200	0.9457	0.6837	0.6840
0.2471	400.0	2400	0.9000	0.6865	0.6867
0.2381	433.33	2600	0.9523	0.6853	0.6854
0.2272	466.67	2800	0.9455	0.6907	0.6907
0.2173	500.0	3000	0.9313	0.6953	0.6954
0.2071	533.33	3200	0.9904	0.6947	0.6947
0.1974	566.67	3400	0.9905	0.6967	0.6967
0.1903	600.0	3600	1.0286	0.6894	0.6894
0.1806	633.33	3800	1.0613	0.6906	0.6907
0.1728	666.67	4000	1.0811	0.6947	0.6947
0.1676	700.0	4200	1.0990	0.7007	0.7007
0.1603	733.33	4400	1.1473	0.6961	0.6961
0.1541	766.67	4600	1.1673	0.7003	0.7007
0.1502	800.0	4800	1.1601	0.6910	0.6914
0.1453	833.33	5000	1.1174	0.6947	0.6947
0.1395	866.67	5200	1.1713	0.7001	0.7001
0.1361	900.0	5400	1.2269	0.6967	0.6967
0.131	933.33	5600	1.1908	0.6947	0.6947
0.1287	966.67	5800	1.1921	0.6968	0.6967
0.125	1000.0	6000	1.1799	0.6947	0.6947
0.1204	1033.33	6200	1.1874	0.6954	0.6954
0.1183	1066.67	6400	1.2756	0.6987	0.6987
0.1159	1100.0	6600	1.2427	0.6994	0.6994
0.1146	1133.33	6800	1.2666	0.6994	0.6994
0.1118	1166.67	7000	1.2582	0.7007	0.7007
0.1096	1200.0	7200	1.2400	0.7041	0.7041
0.1073	1233.33	7400	1.2841	0.7081	0.7081
0.1068	1266.67	7600	1.2657	0.7028	0.7027
0.1049	1300.0	7800	1.2802	0.7007	0.7007
0.1021	1333.33	8000	1.2890	0.6967	0.6967
0.1007	1366.67	8200	1.2750	0.7041	0.7041
0.0994	1400.0	8400	1.2710	0.7047	0.7047
0.0988	1433.33	8600	1.2899	0.7081	0.7081
0.0971	1466.67	8800	1.2848	0.7014	0.7014
0.0974	1500.0	9000	1.2695	0.7001	0.7001
0.0953	1533.33	9200	1.3040	0.7021	0.7021
0.0963	1566.67	9400	1.3123	0.7054	0.7054
0.0946	1600.0	9600	1.2959	0.7008	0.7007
0.0947	1633.33	9800	1.3086	0.7061	0.7061
0.0948	1666.67	10000	1.3010	0.7034	0.7034

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2