GUE_mouse_2-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_mouse_2 dataset. It achieves the following results on the evaluation set:

• Loss: 1.4923
• F1 Score: 0.8199
• Accuracy: 0.8201

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.4054	100.0	200	0.7253	0.7405	0.7409
0.1346	200.0	400	1.0998	0.7193	0.7195
0.0689	300.0	600	1.2385	0.7530	0.7530
0.0433	400.0	800	1.4264	0.7408	0.7409
0.0303	500.0	1000	1.5952	0.7439	0.7439
0.0237	600.0	1200	1.5753	0.7341	0.7348
0.0196	700.0	1400	1.6173	0.7497	0.75
0.0167	800.0	1600	1.7331	0.7529	0.7530
0.014	900.0	1800	1.8038	0.7499	0.75
0.013	1000.0	2000	1.8529	0.7469	0.7470
0.0114	1100.0	2200	1.7976	0.7561	0.7561
0.0106	1200.0	2400	1.8884	0.7529	0.7530
0.0101	1300.0	2600	1.9043	0.7528	0.7530
0.0097	1400.0	2800	1.9710	0.7528	0.7530
0.0083	1500.0	3000	1.8886	0.7591	0.7591
0.0081	1600.0	3200	1.8498	0.7530	0.7530
0.0076	1700.0	3400	1.9551	0.7591	0.7591
0.0077	1800.0	3600	1.9208	0.7561	0.7561
0.0066	1900.0	3800	1.8954	0.7589	0.7591
0.0069	2000.0	4000	1.8680	0.7407	0.7409
0.0064	2100.0	4200	2.0258	0.7683	0.7683
0.0059	2200.0	4400	1.9716	0.7587	0.7591
0.0068	2300.0	4600	2.0603	0.7713	0.7713
0.0059	2400.0	4800	2.0135	0.7651	0.7652
0.0061	2500.0	5000	1.9758	0.7621	0.7622
0.005	2600.0	5200	2.1556	0.7652	0.7652
0.0053	2700.0	5400	2.0520	0.7498	0.75
0.0054	2800.0	5600	2.2497	0.7560	0.7561
0.0047	2900.0	5800	2.0620	0.7559	0.7561
0.005	3000.0	6000	1.9706	0.7618	0.7622
0.0045	3100.0	6200	2.1524	0.7587	0.7591
0.0042	3200.0	6400	2.2165	0.7561	0.7561
0.0049	3300.0	6600	1.9786	0.7589	0.7591
0.0039	3400.0	6800	2.2495	0.7713	0.7713
0.004	3500.0	7000	2.3557	0.7591	0.7591
0.0039	3600.0	7200	2.1475	0.7621	0.7622
0.0038	3700.0	7400	2.1291	0.7591	0.7591
0.0038	3800.0	7600	2.2240	0.7591	0.7591
0.0036	3900.0	7800	2.2950	0.7683	0.7683
0.0039	4000.0	8000	2.1987	0.7591	0.7591
0.0035	4100.0	8200	2.2783	0.7621	0.7622
0.0036	4200.0	8400	2.2651	0.7591	0.7591
0.0032	4300.0	8600	2.2795	0.7591	0.7591
0.003	4400.0	8800	2.3454	0.7591	0.7591
0.0032	4500.0	9000	2.3081	0.7591	0.7591
0.003	4600.0	9200	2.2963	0.7652	0.7652
0.0027	4700.0	9400	2.3278	0.7622	0.7622
0.0028	4800.0	9600	2.3769	0.7622	0.7622
0.0026	4900.0	9800	2.3410	0.7622	0.7622
0.003	5000.0	10000	2.3142	0.7622	0.7622

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2