GUE_mouse_4-seqsight_65536_512_47M-L32_all

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_mouse_4 dataset. It achieves the following results on the evaluation set:

• Loss: 0.9630
• F1 Score: 0.5363
• Accuracy: 0.5364

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 2048
• eval_batch_size: 2048
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6709	25.0	200	0.7104	0.5514	0.5512
0.6081	50.0	400	0.7520	0.5332	0.5358
0.5559	75.0	600	0.7935	0.5472	0.5486
0.5218	100.0	800	0.8150	0.5376	0.5374
0.5034	125.0	1000	0.8141	0.5418	0.5417
0.4907	150.0	1200	0.8255	0.5510	0.5512
0.4829	175.0	1400	0.8471	0.5455	0.5475
0.4753	200.0	1600	0.8409	0.5481	0.5481
0.4669	225.0	1800	0.8286	0.5502	0.5502
0.4605	250.0	2000	0.8440	0.5438	0.5438
0.4521	275.0	2200	0.8756	0.5428	0.5433
0.4428	300.0	2400	0.8696	0.5456	0.5454
0.4333	325.0	2600	0.8581	0.5438	0.5465
0.4227	350.0	2800	0.8902	0.5438	0.5449
0.4109	375.0	3000	0.8924	0.5433	0.5433
0.3994	400.0	3200	0.9358	0.5466	0.5481
0.388	425.0	3400	0.9573	0.5509	0.5512
0.3742	450.0	3600	0.9472	0.5588	0.5587
0.3611	475.0	3800	0.9667	0.5487	0.5528
0.3506	500.0	4000	1.0085	0.5577	0.5576
0.34	525.0	4200	1.0322	0.5577	0.5576
0.3274	550.0	4400	0.9904	0.5552	0.5555
0.3158	575.0	4600	1.0181	0.5507	0.5507
0.3046	600.0	4800	1.0350	0.5549	0.5550
0.2928	625.0	5000	1.0384	0.5510	0.5512
0.2847	650.0	5200	1.0845	0.5535	0.5534
0.2754	675.0	5400	1.1063	0.5535	0.5534
0.2688	700.0	5600	1.1460	0.5549	0.5550
0.2587	725.0	5800	1.1243	0.5557	0.5555
0.2506	750.0	6000	1.1989	0.5525	0.5523
0.2434	775.0	6200	1.1586	0.5482	0.5481
0.2382	800.0	6400	1.1869	0.5509	0.5507
0.2307	825.0	6600	1.2121	0.5426	0.5428
0.2275	850.0	6800	1.1873	0.5353	0.5353
0.2211	875.0	7000	1.1901	0.5450	0.5449
0.216	900.0	7200	1.2012	0.5503	0.5502
0.2109	925.0	7400	1.2088	0.5476	0.5475
0.2068	950.0	7600	1.2467	0.5402	0.5406
0.2043	975.0	7800	1.2640	0.5402	0.5401
0.2001	1000.0	8000	1.2730	0.5432	0.5433
0.1968	1025.0	8200	1.2403	0.5461	0.5459
0.1934	1050.0	8400	1.2645	0.5323	0.5321
0.1924	1075.0	8600	1.2582	0.5434	0.5433
0.1908	1100.0	8800	1.2580	0.5403	0.5401
0.1876	1125.0	9000	1.2909	0.5423	0.5422
0.1872	1150.0	9200	1.2931	0.5445	0.5443
0.1847	1175.0	9400	1.2972	0.5450	0.5449
0.1825	1200.0	9600	1.2851	0.5428	0.5428
0.1819	1225.0	9800	1.3081	0.5439	0.5438
0.1819	1250.0	10000	1.2927	0.5418	0.5417

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2