GUE_mouse_0-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_mouse_0 dataset. It achieves the following results on the evaluation set:

• Loss: 1.6470
• F1 Score: 0.5617
• Accuracy: 0.5617

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6443	50.0	200	0.7420	0.5861	0.5864
0.4985	100.0	400	0.8800	0.5566	0.5568
0.391	150.0	600	0.9882	0.5716	0.5728
0.3329	200.0	800	1.0680	0.5608	0.5630
0.3003	250.0	1000	1.0885	0.5851	0.5852
0.2844	300.0	1200	1.1914	0.5911	0.5914
0.2693	350.0	1400	1.1644	0.5863	0.5889
0.2598	400.0	1600	1.1619	0.5876	0.5889
0.2487	450.0	1800	1.2034	0.5877	0.5877
0.2383	500.0	2000	1.2792	0.6049	0.6049
0.2317	550.0	2200	1.2357	0.6024	0.6025
0.2208	600.0	2400	1.3531	0.5919	0.5951
0.2116	650.0	2600	1.3232	0.5924	0.5938
0.2025	700.0	2800	1.3744	0.6062	0.6062
0.1981	750.0	3000	1.3268	0.5911	0.5914
0.1893	800.0	3200	1.3673	0.5923	0.5926
0.1832	850.0	3400	1.3710	0.5985	0.5988
0.1769	900.0	3600	1.3232	0.5940	0.5951
0.1679	950.0	3800	1.4335	0.6012	0.6025
0.1613	1000.0	4000	1.4186	0.5959	0.5963
0.156	1050.0	4200	1.4299	0.5984	0.5988
0.1517	1100.0	4400	1.4396	0.5938	0.5951
0.1471	1150.0	4600	1.4829	0.6043	0.6049
0.1395	1200.0	4800	1.5019	0.6094	0.6099
0.1361	1250.0	5000	1.3642	0.6110	0.6111
0.1329	1300.0	5200	1.4592	0.5941	0.5951
0.1288	1350.0	5400	1.5022	0.6094	0.6099
0.1249	1400.0	5600	1.4542	0.6024	0.6025
0.1176	1450.0	5800	1.5842	0.6012	0.6012
0.1148	1500.0	6000	1.5441	0.6048	0.6049
0.1137	1550.0	6200	1.5358	0.6099	0.6099
0.1109	1600.0	6400	1.5550	0.6071	0.6074
0.1053	1650.0	6600	1.5509	0.6087	0.6086
0.1027	1700.0	6800	1.5171	0.6046	0.6049
0.1	1750.0	7000	1.5449	0.6012	0.6012
0.0976	1800.0	7200	1.5314	0.6038	0.6037
0.0948	1850.0	7400	1.5012	0.6207	0.6210
0.0936	1900.0	7600	1.6573	0.6063	0.6074
0.0907	1950.0	7800	1.5893	0.6010	0.6025
0.091	2000.0	8000	1.4911	0.6108	0.6111
0.0894	2050.0	8200	1.6058	0.6073	0.6074
0.0872	2100.0	8400	1.6656	0.6055	0.6062
0.0866	2150.0	8600	1.6268	0.6104	0.6111
0.0833	2200.0	8800	1.6478	0.6001	0.6
0.084	2250.0	9000	1.5717	0.6040	0.6049
0.0839	2300.0	9200	1.6142	0.6046	0.6049
0.0807	2350.0	9400	1.6460	0.6049	0.6049
0.0809	2400.0	9600	1.6330	0.6037	0.6037
0.0796	2450.0	9800	1.6165	0.6098	0.6099
0.08	2500.0	10000	1.6272	0.6086	0.6086

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2