sbuedenb/beetle-gpn

third-try-big-data

This model is a a GPN trained on the sbuedenb/big_beetle_dataset dataset. It achieves the following results on the evaluation set:

• Loss: 1.1655

Model description

The default GPN configuration was used.

Intended uses & limitations

This model is meant for DNA analysis of the Cucujiformia infraorder of insects.

Training and evaluation data

The dataset was created from 12 NCBI reference genomes from Cucujiformia.

Training procedure

240000 steps with linear LR 30000 steps with cosine LR

Training hyperparameters

The following hyperparameters were used for first 240000 steps training:

• learning_rate: 0.001
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 1024
• total_eval_batch_size: 1024
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 240000

The following hyperparameters were used for last 30000 steps training:

• learning_rate: 0.001
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 1024
• total_eval_batch_size: 1024
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• training_steps: 270000

Training results

Training Loss	Epoch	Step	Validation Loss
1.2452	0.0083	1000	1.2548
1.2349	0.0167	2000	1.2469
1.2276	0.025	3000	1.2390
1.2233	0.0333	4000	1.2361
1.2206	0.0417	5000	1.2353
1.2178	0.05	6000	1.2331
1.2166	0.0583	7000	1.2318
1.2141	0.0667	8000	1.2283
1.2124	0.075	9000	1.2293
1.2105	0.0833	10000	1.2264
1.2079	0.0917	11000	1.2242
1.2055	0.1	12000	1.2235
1.2026	0.1083	13000	1.2220
1.2005	0.1167	14000	1.2214
1.1986	0.125	15000	1.2195
1.1955	0.1333	16000	1.2167
1.1931	0.1417	17000	1.2170
1.1914	0.15	18000	1.2165
1.1898	0.1583	19000	1.2154
1.1872	0.1667	20000	1.2142
1.1863	0.175	21000	1.2131
1.1849	0.1833	22000	1.2127
1.1831	0.1917	23000	1.2122
1.1812	0.2	24000	1.2117
1.18	0.2083	25000	1.2098
1.1785	0.2167	26000	1.2090
1.1767	0.225	27000	1.2076
1.1761	0.2333	28000	1.2073
1.1754	0.2417	29000	1.2073
1.1732	0.25	30000	1.2080
1.1731	0.2583	31000	1.2055
1.1722	0.2667	32000	1.2052
1.1713	0.275	33000	1.2061
1.1702	0.2833	34000	1.2038
1.169	0.2917	35000	1.2044
1.1679	0.3	36000	1.2047
1.1683	0.3083	37000	1.2019
1.1665	0.3167	38000	1.2036
1.1659	0.325	39000	1.2019
1.165	0.3333	40000	1.2018
1.1636	0.3417	41000	1.2006
1.1634	0.35	42000	1.2009
1.1621	0.3583	43000	1.2003
1.161	0.3667	44000	1.1987
1.1601	0.375	45000	1.1991
1.1595	0.3833	46000	1.1991
1.1597	0.3917	47000	1.1985
1.1582	0.4	48000	1.1967
1.1584	0.4083	49000	1.1983
1.1575	0.4167	50000	1.1968
1.1569	0.425	51000	1.1963
1.1562	0.4333	52000	1.1953
1.1553	0.4417	53000	1.1956
1.1551	0.45	54000	1.1945
1.1538	0.4583	55000	1.1954
1.154	0.4667	56000	1.1943
1.1526	0.475	57000	1.1938
1.1531	0.4833	58000	1.1945
1.1526	0.4917	59000	1.1947
1.1515	0.5	60000	1.1935
1.1504	0.5083	61000	1.1925
1.1512	0.5167	62000	1.1942
1.15	0.525	63000	1.1931
1.1498	0.5333	64000	1.1925
1.1479	0.5417	65000	1.1921
1.1486	0.55	66000	1.1918
1.1482	0.5583	67000	1.1922
1.1475	0.5667	68000	1.1921
1.1473	0.575	69000	1.1907
1.1476	0.5833	70000	1.1904
1.1459	0.5917	71000	1.1910
1.1467	0.6	72000	1.1896
1.1453	0.6083	73000	1.1889
1.1451	0.6167	74000	1.1896
1.1448	0.625	75000	1.1897
1.1437	0.6333	76000	1.1900
1.1433	0.6417	77000	1.1890
1.1435	0.65	78000	1.1895
1.1416	0.6583	79000	1.1884
1.1424	0.6667	80000	1.1891
1.1417	0.675	81000	1.1886
1.142	0.6833	82000	1.1893
1.1406	0.6917	83000	1.1866
1.1411	0.7	84000	1.1889
1.141	0.7083	85000	1.1866
1.14	0.7167	86000	1.1866
1.1397	0.725	87000	1.1875
1.1389	0.7333	88000	1.1870
1.1388	0.7417	89000	1.1863
1.1377	0.75	90000	1.1866
1.1391	0.7583	91000	1.1855
1.1379	0.7667	92000	1.1869
1.1381	0.775	93000	1.1872
1.1383	0.7833	94000	1.1859
1.1366	0.7917	95000	1.1861
1.1365	0.8	96000	1.1854
1.136	0.8083	97000	1.1849
1.1364	0.8167	98000	1.1859
1.1358	0.825	99000	1.1852
1.1344	0.8333	100000	1.1847
1.1364	0.8417	101000	1.1851
1.1349	0.85	102000	1.1853
1.1341	0.8583	103000	1.1851
1.1344	0.8667	104000	1.1847
1.1351	0.875	105000	1.1847
1.1335	0.8833	106000	1.1842
1.1339	0.8917	107000	1.1852
1.1331	0.9	108000	1.1839
1.1332	0.9083	109000	1.1845
1.1328	0.9167	110000	1.1840
1.1316	0.925	111000	1.1838
1.1315	0.9333	112000	1.1835
1.1315	0.9417	113000	1.1833
1.1308	0.95	114000	1.1845
1.1312	0.9583	115000	1.1830
1.1303	0.0056	116000	1.1829
1.132	0.0111	117000	1.1826
1.1305	0.0167	118000	1.1841
1.1307	0.0222	119000	1.1825
1.1311	0.0278	120000	1.1826
1.1297	0.0333	121000	1.1815
1.1293	0.0389	122000	1.1818
1.1286	0.0444	123000	1.1818
1.1279	0.05	124000	1.1831
1.1281	0.0556	125000	1.1822
1.1273	0.0611	126000	1.1817
1.1291	0.0667	127000	1.1817
1.1271	0.0722	128000	1.1813
1.1273	0.0778	129000	1.1802
1.1282	0.0833	130000	1.1815
1.1274	0.0889	131000	1.1808
1.1276	0.0944	132000	1.1824
1.1265	0.1	133000	1.1809
1.1268	0.1056	134000	1.1819
1.1258	0.1111	135000	1.1818
1.126	0.1167	136000	1.1807
1.1251	0.1222	137000	1.1806
1.1256	0.1278	138000	1.1805
1.1259	0.1333	139000	1.1799
1.1253	0.1389	140000	1.1805
1.1242	0.1444	141000	1.1802
1.1242	0.15	142000	1.1800
1.1242	0.1556	143000	1.1794
1.1242	0.1611	144000	1.1798
1.1231	0.1667	145000	1.1798
1.1249	0.1722	146000	1.1794
1.1237	0.1778	147000	1.1794
1.1235	0.1833	148000	1.1795
1.1237	0.1889	149000	1.1794
1.1237	0.1944	150000	1.1783
1.1226	0.2	151000	1.1800
1.1242	0.2056	152000	1.1789
1.1224	0.2111	153000	1.1804
1.1233	0.2167	154000	1.1792
1.1224	0.2222	155000	1.1790
1.122	0.2278	156000	1.1782
1.1222	0.2333	157000	1.1793
1.1217	0.2389	158000	1.1791
1.121	0.2444	159000	1.1776
1.1206	0.25	160000	1.1776
1.1204	0.2556	161000	1.1780
1.1217	0.2611	162000	1.1778
1.1205	0.2667	163000	1.1776
1.121	0.2722	164000	1.1783
1.1211	0.2778	165000	1.1783
1.1203	0.2833	166000	1.1776
1.1199	0.2889	167000	1.1786
1.1204	0.2944	168000	1.1770
1.1188	0.3	169000	1.1763
1.1193	0.3056	170000	1.1769
1.1198	0.3111	171000	1.1779
1.1182	0.3167	172000	1.1777
1.119	0.3222	173000	1.1772
1.1198	0.3278	174000	1.1776
1.118	0.3333	175000	1.1775
1.1189	0.0042	176000	1.1772
1.1191	0.0083	177000	1.1770
1.1178	0.0125	178000	1.1777
1.1192	0.0167	179000	1.1766
1.1183	0.0208	180000	1.1770
1.1174	0.025	181000	1.1775
1.1177	0.0292	182000	1.1766
1.1177	0.0333	183000	1.1762
1.1163	0.0375	184000	1.1757
1.1169	0.0417	185000	1.1757
1.1162	0.0458	186000	1.1762
1.1185	0.05	187000	1.1770
1.1169	0.0542	188000	1.1761
1.1164	0.0583	189000	1.1757
1.1168	0.0625	190000	1.1767
1.1166	0.0667	191000	1.1760
1.1163	0.0708	192000	1.1759
1.1157	0.075	193000	1.1744
1.1153	0.0792	194000	1.1762
1.1157	0.0833	195000	1.1761
1.1156	0.0875	196000	1.1748
1.1157	0.0917	197000	1.1752
1.1155	0.0958	198000	1.1750
1.1156	0.1	199000	1.1752
1.1151	0.1042	200000	1.1751
1.1144	0.1083	201000	1.1756
1.1145	0.1125	202000	1.1753
1.1143	0.1167	203000	1.1750
1.1146	0.1208	204000	1.1756
1.113	0.125	205000	1.1746
1.1149	0.1292	206000	1.1749
1.114	0.1333	207000	1.1742
1.1146	0.1375	208000	1.1739
1.1146	0.1417	209000	1.1762
1.1147	0.1458	210000	1.1738
1.113	0.15	211000	1.1742
1.115	0.1542	212000	1.1744
1.1133	0.1583	213000	1.1750
1.1136	0.1625	214000	1.1751
1.1133	0.1667	215000	1.1745
1.1131	0.1708	216000	1.1743
1.1132	0.175	217000	1.1738
1.1129	0.1792	218000	1.1747
1.1121	0.1833	219000	1.1749
1.1123	0.1875	220000	1.1736
1.1118	0.1917	221000	1.1734
1.1133	0.1958	222000	1.1746
1.1118	0.2	223000	1.1731
1.1126	0.2042	224000	1.1734
1.1125	0.2083	225000	1.1736
1.1122	0.2125	226000	1.1739
1.1123	0.2167	227000	1.1730
1.1116	0.2208	228000	1.1733
1.1109	0.225	229000	1.1730
1.1111	0.2292	230000	1.1745
1.1118	0.2333	231000	1.1738
1.1111	0.2375	232000	1.1736
1.1114	0.2417	233000	1.1726
1.1121	0.2458	234000	1.1727
1.1106	0.25	235000	1.1742
1.1106	0.2542	236000	1.1732
1.1103	0.2583	237000	1.1733
1.1104	0.2625	238000	1.1733
1.111	0.2667	239000	1.1730
1.1101	0.2708	240000	1.1732
1.1051	0.0037	241000	1.1693
1.106	0.0074	242000	1.1688
1.1025	0.0111	243000	1.1687
1.103	0.0148	244000	1.1673
1.1031	0.0185	245000	1.1673
1.1014	0.0222	246000	1.1671
1.1009	0.0259	247000	1.1673
1.0995	0.0296	248000	1.1668
1.1005	0.0333	249000	1.1668
1.1002	0.0370	250000	1.1668
1.0994	0.0407	251000	1.1664
1.1001	0.0444	252000	1.1660
1.1004	0.0481	253000	1.1670
1.0993	0.0519	254000	1.1660
1.0997	0.0556	255000	1.1674
1.099	0.0593	256000	1.1662
1.1005	0.0630	257000	1.1665
1.0992	0.0667	258000	1.1658
1.0987	0.0704	259000	1.1665
1.0986	0.0741	260000	1.1660
1.098	0.0778	261000	1.1669
1.0989	0.0815	262000	1.1660
1.0992	0.0852	263000	1.1655
1.0998	0.0889	264000	1.1663
1.0975	0.0926	265000	1.1659
1.0986	0.0963	266000	1.1664
1.0987	0.1	267000	1.1665
1.0987	0.1037	268000	1.1667
1.0986	0.1074	269000	1.1656
1.0965	0.1111	270000	1.1659

Framework versions

• Transformers 4.51.3
• Pytorch 2.7.0+cu126
• Datasets 3.6.0.dev0
• Tokenizers 0.21.1