GUE_EMP_H4-seqsight_4096_512_27M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_EMP_H4 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2553
• F1 Score: 0.9075
• Accuracy: 0.9076

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.3464	2.17	200	0.2854	0.8926	0.8925
0.2702	4.35	400	0.2757	0.8975	0.8973
0.2566	6.52	600	0.2777	0.8946	0.8946
0.2531	8.7	800	0.2828	0.8982	0.8980
0.2364	10.87	1000	0.2706	0.8973	0.8973
0.2326	13.04	1200	0.2738	0.9000	0.9001
0.2221	15.22	1400	0.2717	0.9077	0.9076
0.2124	17.39	1600	0.2961	0.8929	0.8925
0.2089	19.57	1800	0.2771	0.8959	0.8960
0.2005	21.74	2000	0.2934	0.8962	0.8960
0.1924	23.91	2200	0.2988	0.8959	0.8960
0.1896	26.09	2400	0.2897	0.8912	0.8912
0.183	28.26	2600	0.3030	0.8896	0.8898
0.1757	30.43	2800	0.3046	0.8877	0.8877
0.1693	32.61	3000	0.3091	0.8933	0.8932
0.1624	34.78	3200	0.3127	0.8852	0.8850
0.1625	36.96	3400	0.3129	0.8920	0.8919
0.1544	39.13	3600	0.3324	0.8822	0.8823
0.1483	41.3	3800	0.3317	0.8889	0.8891
0.1473	43.48	4000	0.3315	0.8836	0.8836
0.1454	45.65	4200	0.3341	0.8840	0.8843
0.1392	47.83	4400	0.3500	0.8776	0.8775
0.1348	50.0	4600	0.3604	0.8771	0.8775
0.1301	52.17	4800	0.3675	0.8794	0.8795
0.1285	54.35	5000	0.3700	0.8751	0.8754
0.1233	56.52	5200	0.3709	0.8859	0.8857
0.1243	58.7	5400	0.3766	0.8738	0.8741
0.1199	60.87	5600	0.3872	0.8817	0.8816
0.1162	63.04	5800	0.3914	0.8795	0.8795
0.1153	65.22	6000	0.3962	0.8736	0.8741
0.1063	67.39	6200	0.3987	0.8748	0.8747
0.1061	69.57	6400	0.4121	0.8685	0.8686
0.1072	71.74	6600	0.4133	0.8754	0.8754
0.1044	73.91	6800	0.4176	0.8774	0.8775
0.1001	76.09	7000	0.4241	0.8772	0.8775
0.1025	78.26	7200	0.4178	0.8717	0.8720
0.0978	80.43	7400	0.4276	0.8725	0.8727
0.0962	82.61	7600	0.4393	0.8707	0.8713
0.0963	84.78	7800	0.4390	0.8787	0.8789
0.0934	86.96	8000	0.4465	0.8703	0.8706
0.0917	89.13	8200	0.4537	0.8696	0.8700
0.0902	91.3	8400	0.4595	0.8702	0.8706
0.0857	93.48	8600	0.4673	0.8737	0.8741
0.0884	95.65	8800	0.4660	0.8701	0.8706
0.0851	97.83	9000	0.4629	0.8689	0.8693
0.0847	100.0	9200	0.4669	0.8691	0.8693
0.0842	102.17	9400	0.4653	0.8704	0.8706
0.083	104.35	9600	0.4690	0.8697	0.8700
0.0825	106.52	9800	0.4758	0.8730	0.8734
0.0844	108.7	10000	0.4728	0.8703	0.8706

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2