GUE_prom_prom_core_all-seqsight_16384_512_22M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_16384_512_22M on the mahdibaghbanzadeh/GUE_prom_prom_core_all dataset. It achieves the following results on the evaluation set:

• Loss: 0.5969
• F1 Score: 0.7051
• Accuracy: 0.7051

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6534	8.33	200	0.6127	0.6644	0.6647
0.6042	16.67	400	0.5937	0.6868	0.6878
0.585	25.0	600	0.5858	0.6930	0.6949
0.5691	33.33	800	0.5818	0.6998	0.7
0.5555	41.67	1000	0.5794	0.7025	0.7025
0.5451	50.0	1200	0.5739	0.7067	0.7068
0.5354	58.33	1400	0.5730	0.7064	0.7064
0.5287	66.67	1600	0.5770	0.7060	0.7061
0.5227	75.0	1800	0.5775	0.7028	0.7029
0.5168	83.33	2000	0.5750	0.7070	0.7071
0.5136	91.67	2200	0.5747	0.7031	0.7032
0.5066	100.0	2400	0.5749	0.7101	0.7101
0.5038	108.33	2600	0.5885	0.7066	0.7071
0.4998	116.67	2800	0.5957	0.7067	0.7068
0.4949	125.0	3000	0.5748	0.7087	0.7090
0.4919	133.33	3200	0.5937	0.7058	0.7064
0.4884	141.67	3400	0.5876	0.7029	0.7035
0.4857	150.0	3600	0.5799	0.7129	0.7132
0.4824	158.33	3800	0.5979	0.7080	0.7084
0.4806	166.67	4000	0.5895	0.7077	0.7088
0.4758	175.0	4200	0.5952	0.7046	0.7057
0.474	183.33	4400	0.5880	0.7131	0.7132
0.4708	191.67	4600	0.5841	0.7135	0.7139
0.4686	200.0	4800	0.5902	0.7125	0.7127
0.4649	208.33	5000	0.5926	0.7142	0.7144
0.464	216.67	5200	0.5935	0.7092	0.7098
0.4619	225.0	5400	0.6059	0.7022	0.7037
0.4583	233.33	5600	0.5904	0.7124	0.7125
0.4565	241.67	5800	0.6008	0.7126	0.7128
0.455	250.0	6000	0.5984	0.7116	0.7120
0.4519	258.33	6200	0.5892	0.7096	0.7100
0.4508	266.67	6400	0.5943	0.7098	0.7101
0.4493	275.0	6600	0.5935	0.7076	0.7078
0.4467	283.33	6800	0.6051	0.7071	0.7074
0.4457	291.67	7000	0.6103	0.7025	0.7035
0.4452	300.0	7200	0.5967	0.7079	0.7083
0.4421	308.33	7400	0.6110	0.7059	0.7071
0.4417	316.67	7600	0.6163	0.7014	0.7032
0.4399	325.0	7800	0.6253	0.7013	0.7025
0.4377	333.33	8000	0.6139	0.7053	0.7063
0.4368	341.67	8200	0.6145	0.7070	0.7073
0.4375	350.0	8400	0.6128	0.7045	0.7051
0.4356	358.33	8600	0.6098	0.7071	0.7074
0.4344	366.67	8800	0.6091	0.7024	0.7032
0.4331	375.0	9000	0.6130	0.7030	0.7037
0.4331	383.33	9200	0.6141	0.7057	0.7064
0.4321	391.67	9400	0.6160	0.7039	0.7047
0.4306	400.0	9600	0.6180	0.7042	0.7049
0.4304	408.33	9800	0.6200	0.7031	0.7041
0.4311	416.67	10000	0.6166	0.7045	0.7051

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2