GUE_EMP_H4-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_EMP_H4 dataset. It achieves the following results on the evaluation set:

• Loss: 0.9009
• F1 Score: 0.7288
• Accuracy: 0.7296

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6004	33.33	200	0.5918	0.6971	0.7036
0.4689	66.67	400	0.6275	0.6999	0.7009
0.4049	100.0	600	0.6454	0.7097	0.7091
0.3609	133.33	800	0.6548	0.7217	0.7214
0.3367	166.67	1000	0.6638	0.7320	0.7317
0.3193	200.0	1200	0.6910	0.7322	0.7358
0.3064	233.33	1400	0.6906	0.7302	0.7296
0.2926	266.67	1600	0.7215	0.7220	0.7242
0.2807	300.0	1800	0.7537	0.7241	0.7248
0.2684	333.33	2000	0.7420	0.7304	0.7303
0.2578	366.67	2200	0.7572	0.7275	0.7269
0.2461	400.0	2400	0.8048	0.7315	0.7317
0.2353	433.33	2600	0.7902	0.7282	0.7296
0.2247	466.67	2800	0.8239	0.7309	0.7317
0.2143	500.0	3000	0.8040	0.7279	0.7283
0.2072	533.33	3200	0.8647	0.7362	0.7372
0.1999	566.67	3400	0.8706	0.7318	0.7324
0.1913	600.0	3600	0.8544	0.7223	0.7228
0.1846	633.33	3800	0.8859	0.7290	0.7296
0.1771	666.67	4000	0.9072	0.7208	0.7207
0.1692	700.0	4200	0.9304	0.7252	0.7262
0.1636	733.33	4400	0.9465	0.7258	0.7269
0.1575	766.67	4600	0.9440	0.7262	0.7262
0.1533	800.0	4800	0.9363	0.7213	0.7242
0.1467	833.33	5000	0.9269	0.7182	0.7187
0.1434	866.67	5200	0.9126	0.7156	0.7166
0.1378	900.0	5400	0.9863	0.7282	0.7290
0.1365	933.33	5600	0.9797	0.7267	0.7283
0.1324	966.67	5800	0.9849	0.7278	0.7283
0.1283	1000.0	6000	1.0046	0.7264	0.7276
0.1246	1033.33	6200	0.9894	0.7241	0.7242
0.1211	1066.67	6400	1.0089	0.7245	0.7262
0.1198	1100.0	6600	1.0040	0.7225	0.7228
0.1169	1133.33	6800	1.0021	0.7249	0.7255
0.1145	1166.67	7000	1.0293	0.7323	0.7337
0.1122	1200.0	7200	1.0010	0.7323	0.7324
0.1112	1233.33	7400	1.0087	0.7275	0.7276
0.1088	1266.67	7600	0.9907	0.7291	0.7296
0.1076	1300.0	7800	1.0307	0.7276	0.7283
0.106	1333.33	8000	1.0398	0.7318	0.7317
0.1035	1366.67	8200	1.0240	0.7238	0.7248
0.1021	1400.0	8400	1.0345	0.7302	0.7303
0.1026	1433.33	8600	1.0392	0.7300	0.7303
0.1012	1466.67	8800	1.0445	0.7314	0.7324
0.099	1500.0	9000	1.0577	0.7346	0.7351
0.0988	1533.33	9200	1.0422	0.7314	0.7317
0.0978	1566.67	9400	1.0469	0.7285	0.7290
0.0984	1600.0	9600	1.0278	0.7313	0.7317
0.0971	1633.33	9800	1.0458	0.7286	0.7290
0.0974	1666.67	10000	1.0454	0.7278	0.7283

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2