GUE_EMP_H3-seqsight_4096_512_27M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_4096_512_27M on the mahdibaghbanzadeh/GUE_EMP_H3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2909
• F1 Score: 0.8864
• Accuracy: 0.8864

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.453	2.13	200	0.3927	0.8214	0.8230
0.3536	4.26	400	0.3496	0.8550	0.8550
0.3302	6.38	600	0.3367	0.8637	0.8637
0.313	8.51	800	0.3278	0.8663	0.8664
0.3007	10.64	1000	0.3103	0.8704	0.8704
0.2861	12.77	1200	0.3074	0.8684	0.8684
0.2816	14.89	1400	0.2988	0.8771	0.8771
0.2709	17.02	1600	0.3005	0.8724	0.8724
0.2681	19.15	1800	0.3125	0.8717	0.8717
0.2618	21.28	2000	0.3041	0.8791	0.8791
0.264	23.4	2200	0.2929	0.8737	0.8737
0.2535	25.53	2400	0.3042	0.8764	0.8764
0.2518	27.66	2600	0.2958	0.8791	0.8791
0.2522	29.79	2800	0.2968	0.8818	0.8818
0.2477	31.91	3000	0.3036	0.8777	0.8778
0.2443	34.04	3200	0.2954	0.8804	0.8804
0.2436	36.17	3400	0.3083	0.8790	0.8791
0.2384	38.3	3600	0.2989	0.8764	0.8764
0.2392	40.43	3800	0.2959	0.8784	0.8784
0.2368	42.55	4000	0.3013	0.8751	0.8751
0.2335	44.68	4200	0.2980	0.8804	0.8804
0.2334	46.81	4400	0.3032	0.8798	0.8798
0.233	48.94	4600	0.3021	0.8791	0.8791
0.2269	51.06	4800	0.2990	0.8791	0.8791
0.2268	53.19	5000	0.3092	0.8784	0.8784
0.229	55.32	5200	0.2956	0.8778	0.8778
0.2244	57.45	5400	0.3177	0.8751	0.8751
0.222	59.57	5600	0.3026	0.8784	0.8784
0.2233	61.7	5800	0.3011	0.8777	0.8778
0.2192	63.83	6000	0.3196	0.8757	0.8758
0.2198	65.96	6200	0.3030	0.8791	0.8791
0.2187	68.09	6400	0.3085	0.8798	0.8798
0.2165	70.21	6600	0.3110	0.8804	0.8804
0.2191	72.34	6800	0.3040	0.8811	0.8811
0.2142	74.47	7000	0.3198	0.8717	0.8717
0.2109	76.6	7200	0.3124	0.8804	0.8804
0.218	78.72	7400	0.3112	0.8798	0.8798
0.2138	80.85	7600	0.3121	0.8811	0.8811
0.2111	82.98	7800	0.3130	0.8804	0.8804
0.2122	85.11	8000	0.3129	0.8804	0.8804
0.212	87.23	8200	0.3127	0.8811	0.8811
0.2116	89.36	8400	0.3131	0.8811	0.8811
0.2102	91.49	8600	0.3216	0.8764	0.8764
0.2085	93.62	8800	0.3163	0.8811	0.8811
0.211	95.74	9000	0.3180	0.8778	0.8778
0.2105	97.87	9200	0.3133	0.8818	0.8818
0.2059	100.0	9400	0.3156	0.8791	0.8791
0.208	102.13	9600	0.3154	0.8791	0.8791
0.2068	104.26	9800	0.3155	0.8784	0.8784
0.2075	106.38	10000	0.3158	0.8784	0.8784

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2