fewabu/1e-07_AmpGPT2

1e07_output_dir_clean_df_10-100_noX_100_50_epoch_cluster

This model is a fine-tuned version of nferruz/ProtGPT2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.1547
• Accuracy: 0.2115

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-07
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	148	6.5084	0.1890
No log	2.0	296	6.4833	0.1906
No log	3.0	444	6.4602	0.1922
6.5344	4.0	592	6.4383	0.1934
6.5344	5.0	740	6.4181	0.1948
6.5344	6.0	888	6.3991	0.1961
6.4574	7.0	1036	6.3814	0.1972
6.4574	8.0	1184	6.3648	0.1982
6.4574	9.0	1332	6.3493	0.1993
6.4574	10.0	1480	6.3349	0.2004
6.4006	11.0	1628	6.3212	0.2013
6.4006	12.0	1776	6.3085	0.2022
6.4006	13.0	1924	6.2966	0.2031
6.3522	14.0	2072	6.2854	0.2038
6.3522	15.0	2220	6.2750	0.2044
6.3522	16.0	2368	6.2652	0.2051
6.3149	17.0	2516	6.2562	0.2057
6.3149	18.0	2664	6.2478	0.2064
6.3149	19.0	2812	6.2399	0.2069
6.3149	20.0	2960	6.2326	0.2073
6.285	21.0	3108	6.2258	0.2078
6.285	22.0	3256	6.2196	0.2081
6.285	23.0	3404	6.2137	0.2084
6.2609	24.0	3552	6.2082	0.2088
6.2609	25.0	3700	6.2030	0.2091
6.2609	26.0	3848	6.1983	0.2093
6.2609	27.0	3996	6.1938	0.2096
6.242	28.0	4144	6.1897	0.2099
6.242	29.0	4292	6.1859	0.2100
6.242	30.0	4440	6.1824	0.2102
6.2272	31.0	4588	6.1791	0.2104
6.2272	32.0	4736	6.1761	0.2105
6.2272	33.0	4884	6.1733	0.2107
6.216	34.0	5032	6.1707	0.2108
6.216	35.0	5180	6.1684	0.2110
6.216	36.0	5328	6.1662	0.2110
6.216	37.0	5476	6.1643	0.2112
6.2101	38.0	5624	6.1626	0.2112
6.2101	39.0	5772	6.1610	0.2112
6.2101	40.0	5920	6.1597	0.2113
6.2026	41.0	6068	6.1585	0.2113
6.2026	42.0	6216	6.1575	0.2114
6.2026	43.0	6364	6.1567	0.2114
6.1985	44.0	6512	6.1560	0.2114
6.1985	45.0	6660	6.1555	0.2115
6.1985	46.0	6808	6.1551	0.2115
6.1985	47.0	6956	6.1549	0.2115
6.1947	48.0	7104	6.1547	0.2115
6.1947	49.0	7252	6.1547	0.2115
6.1947	50.0	7400	6.1547	0.2115

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.2.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0