GUE_EMP_H3-seqsight_16384_512_56M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2646
• F1 Score: 0.8951
• Accuracy: 0.8951

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.3885	2.13	200	0.3562	0.8477	0.8477
0.2813	4.26	400	0.3305	0.8675	0.8677
0.2493	6.38	600	0.3649	0.8522	0.8524
0.2349	8.51	800	0.3031	0.8838	0.8838
0.2193	10.64	1000	0.3812	0.8577	0.8584
0.2032	12.77	1200	0.3416	0.8764	0.8764
0.1925	14.89	1400	0.3750	0.8708	0.8711
0.1779	17.02	1600	0.3903	0.8597	0.8597
0.1674	19.15	1800	0.3564	0.8724	0.8724
0.1489	21.28	2000	0.4619	0.8612	0.8617
0.1423	23.4	2200	0.4485	0.8735	0.8737
0.1215	25.53	2400	0.4759	0.8784	0.8784
0.1185	27.66	2600	0.5499	0.8436	0.8444
0.0993	29.79	2800	0.5338	0.8520	0.8524
0.0962	31.91	3000	0.5457	0.8514	0.8517
0.0823	34.04	3200	0.5406	0.8577	0.8577
0.0787	36.17	3400	0.6370	0.8559	0.8564
0.0708	38.3	3600	0.6247	0.8574	0.8577
0.0674	40.43	3800	0.6834	0.8478	0.8484
0.057	42.55	4000	0.8145	0.8462	0.8470
0.0536	44.68	4200	0.7901	0.8400	0.8410
0.0505	46.81	4400	0.7505	0.8659	0.8664
0.0463	48.94	4600	0.7752	0.8490	0.8497
0.0449	51.06	4800	0.7215	0.8601	0.8604
0.0384	53.19	5000	0.8821	0.8376	0.8383
0.0351	55.32	5200	0.9139	0.8465	0.8470
0.0349	57.45	5400	0.9360	0.8387	0.8397
0.0361	59.57	5600	0.8710	0.8575	0.8577
0.0308	61.7	5800	0.8229	0.8597	0.8597
0.0294	63.83	6000	0.9199	0.8517	0.8524
0.0293	65.96	6200	0.8718	0.8588	0.8591
0.0271	68.09	6400	0.8787	0.8617	0.8617
0.0238	70.21	6600	0.9513	0.8581	0.8584
0.0241	72.34	6800	0.9352	0.8629	0.8631
0.0225	74.47	7000	0.9943	0.8548	0.8550
0.0231	76.6	7200	0.9241	0.8602	0.8604
0.0204	78.72	7400	1.0017	0.8622	0.8624
0.0206	80.85	7600	1.0763	0.8498	0.8504
0.0182	82.98	7800	1.0418	0.8575	0.8577
0.0166	85.11	8000	1.0393	0.8567	0.8570
0.0172	87.23	8200	1.0861	0.8492	0.8497
0.0167	89.36	8400	1.1617	0.8470	0.8477
0.015	91.49	8600	1.0801	0.8621	0.8624
0.0151	93.62	8800	1.1022	0.8541	0.8544
0.014	95.74	9000	1.1847	0.8438	0.8444
0.0125	97.87	9200	1.1438	0.8534	0.8537
0.0131	100.0	9400	1.1487	0.8554	0.8557
0.0121	102.13	9600	1.1538	0.8533	0.8537
0.0124	104.26	9800	1.1753	0.8513	0.8517
0.0121	106.38	10000	1.1525	0.8501	0.8504

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2