GUE_EMP_H3-seqsight_16384_512_56M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_56M on the mahdibaghbanzadeh/GUE_EMP_H3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2977
• F1 Score: 0.8871
• Accuracy: 0.8871

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.4231	2.13	200	0.4139	0.8159	0.8183
0.344	4.26	400	0.3559	0.8449	0.8450
0.321	6.38	600	0.3584	0.8530	0.8530
0.306	8.51	800	0.3437	0.8548	0.8550
0.292	10.64	1000	0.3478	0.8510	0.8510
0.2772	12.77	1200	0.3449	0.8597	0.8597
0.2726	14.89	1400	0.3547	0.8533	0.8537
0.2607	17.02	1600	0.3273	0.8704	0.8704
0.2592	19.15	1800	0.3434	0.8536	0.8537
0.2537	21.28	2000	0.3457	0.8615	0.8617
0.2524	23.4	2200	0.3281	0.8683	0.8684
0.241	25.53	2400	0.3780	0.8463	0.8464
0.2465	27.66	2600	0.3381	0.8608	0.8611
0.2397	29.79	2800	0.3359	0.8682	0.8684
0.2367	31.91	3000	0.3365	0.8696	0.8697
0.2323	34.04	3200	0.3274	0.8743	0.8744
0.2315	36.17	3400	0.3487	0.8635	0.8637
0.228	38.3	3600	0.3534	0.8635	0.8637
0.2271	40.43	3800	0.3564	0.8640	0.8644
0.2244	42.55	4000	0.3537	0.8608	0.8611
0.221	44.68	4200	0.3461	0.8676	0.8677
0.2205	46.81	4400	0.3504	0.8615	0.8617
0.2163	48.94	4600	0.3609	0.8586	0.8591
0.217	51.06	4800	0.3217	0.8784	0.8784
0.2146	53.19	5000	0.3550	0.8640	0.8644
0.2155	55.32	5200	0.3291	0.8730	0.8731
0.2103	57.45	5400	0.3674	0.8662	0.8664
0.2057	59.57	5600	0.3479	0.8744	0.8744
0.2108	61.7	5800	0.3268	0.8744	0.8744
0.2054	63.83	6000	0.3677	0.8674	0.8677
0.2057	65.96	6200	0.3632	0.8668	0.8671
0.2051	68.09	6400	0.3511	0.8722	0.8724
0.2032	70.21	6600	0.3648	0.8688	0.8691
0.2031	72.34	6800	0.3417	0.8730	0.8731
0.1995	74.47	7000	0.3788	0.8626	0.8631
0.195	76.6	7200	0.3478	0.8743	0.8744
0.2002	78.72	7400	0.3553	0.8723	0.8724
0.1986	80.85	7600	0.3591	0.8710	0.8711
0.1954	82.98	7800	0.3469	0.8757	0.8758
0.1976	85.11	8000	0.3576	0.8716	0.8717
0.1959	87.23	8200	0.3583	0.8723	0.8724
0.1972	89.36	8400	0.3552	0.8763	0.8764
0.1954	91.49	8600	0.3648	0.8702	0.8704
0.1937	93.62	8800	0.3511	0.8730	0.8731
0.1933	95.74	9000	0.3704	0.8662	0.8664
0.1914	97.87	9200	0.3564	0.8729	0.8731
0.195	100.0	9400	0.3591	0.8723	0.8724
0.1923	102.13	9600	0.3608	0.8723	0.8724
0.1919	104.26	9800	0.3586	0.8730	0.8731
0.1924	106.38	10000	0.3575	0.8736	0.8737

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2