GUE_EMP_H4-seqsight_4096_512_46M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_46M on the mahdibaghbanzadeh/GUE_EMP_H4 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2558
• F1 Score: 0.9111
• Accuracy: 0.9110

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.3011	2.17	200	0.2643	0.8981	0.8980
0.2454	4.35	400	0.2584	0.8979	0.8980
0.2237	6.52	600	0.2645	0.9016	0.9014
0.2107	8.7	800	0.2743	0.8915	0.8912
0.1936	10.87	1000	0.2737	0.8958	0.8960
0.181	13.04	1200	0.2963	0.8827	0.8823
0.1593	15.22	1400	0.3184	0.8908	0.8905
0.1453	17.39	1600	0.3405	0.8839	0.8836
0.1285	19.57	1800	0.3479	0.8939	0.8939
0.1111	21.74	2000	0.4011	0.8771	0.8768
0.1005	23.91	2200	0.4055	0.8819	0.8816
0.0903	26.09	2400	0.4202	0.8913	0.8912
0.0782	28.26	2600	0.4638	0.8853	0.8850
0.0666	30.43	2800	0.4875	0.8773	0.8768
0.063	32.61	3000	0.5041	0.8791	0.8789
0.0549	34.78	3200	0.4648	0.8886	0.8884
0.0479	36.96	3400	0.5217	0.8907	0.8905
0.0426	39.13	3600	0.6087	0.8800	0.8802
0.0398	41.3	3800	0.5759	0.8764	0.8761
0.0347	43.48	4000	0.6083	0.8818	0.8816
0.0293	45.65	4200	0.6258	0.8877	0.8877
0.0259	47.83	4400	0.7382	0.8804	0.8802
0.0279	50.0	4600	0.6818	0.8866	0.8864
0.0255	52.17	4800	0.6983	0.8873	0.8871
0.0221	54.35	5000	0.7424	0.8886	0.8884
0.0243	56.52	5200	0.6928	0.8826	0.8823
0.0181	58.7	5400	0.7622	0.8814	0.8816
0.0172	60.87	5600	0.7647	0.8856	0.8857
0.0187	63.04	5800	0.7383	0.8818	0.8816
0.0152	65.22	6000	0.7824	0.8879	0.8877
0.0144	67.39	6200	0.8176	0.8908	0.8905
0.0144	69.57	6400	0.7774	0.8872	0.8871
0.0133	71.74	6600	0.8605	0.8885	0.8884
0.0127	73.91	6800	0.8442	0.8865	0.8864
0.0128	76.09	7000	0.8120	0.8866	0.8864
0.0108	78.26	7200	0.8403	0.8839	0.8836
0.0109	80.43	7400	0.8822	0.8873	0.8871
0.0086	82.61	7600	0.8667	0.8878	0.8877
0.0099	84.78	7800	0.8767	0.8858	0.8857
0.0086	86.96	8000	0.9134	0.8872	0.8871
0.01	89.13	8200	0.9166	0.8891	0.8891
0.0078	91.3	8400	0.9330	0.8934	0.8932
0.0073	93.48	8600	0.9231	0.8926	0.8925
0.0078	95.65	8800	0.9328	0.8900	0.8898
0.0085	97.83	9000	0.9496	0.8881	0.8877
0.0076	100.0	9200	0.9058	0.8906	0.8905
0.0062	102.17	9400	0.9272	0.8893	0.8891
0.0072	104.35	9600	0.9439	0.8846	0.8843
0.0073	106.52	9800	0.9272	0.8866	0.8864
0.007	108.7	10000	0.9262	0.8873	0.8871

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2