Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 5000 examples [00:00, 44572.65 examples/s]
Generating train split: 12000 examples [00:00, 54195.28 examples/s]
Generating train split: 22000 examples [00:00, 67508.97 examples/s]
Generating train split: 31000 examples [00:00, 72629.53 examples/s]
Generating train split: 41000 examples [00:00, 77636.11 examples/s]
Generating train split: 50000 examples [00:00, 77979.67 examples/s]
Generating train split: 59000 examples [00:00, 79177.37 examples/s]
Generating train split: 68000 examples [00:00, 80538.99 examples/s]
Generating train split: 78000 examples [00:01, 82615.57 examples/s]
Generating train split: 87000 examples [00:01, 81599.59 examples/s]
Generating train split: 97000 examples [00:01, 82593.17 examples/s]
Generating train split: 107000 examples [00:01, 83015.82 examples/s]
Generating train split: 117000 examples [00:01, 83831.98 examples/s]
Generating train split: 127000 examples [00:01, 84779.68 examples/s]
Generating train split: 137000 examples [00:01, 85352.50 examples/s]
Generating train split: 147000 examples [00:01, 85119.13 examples/s]
Generating train split: 157000 examples [00:01, 85485.00 examples/s]
Generating train split: 166000 examples [00:02, 84088.87 examples/s]
Generating train split: 176000 examples [00:02, 84969.68 examples/s]
Generating train split: 186000 examples [00:02, 85537.71 examples/s]
Generating train split: 196000 examples [00:02, 85606.58 examples/s]
Generating train split: 206000 examples [00:02, 85290.14 examples/s]
Generating train split: 216000 examples [00:02, 86448.46 examples/s]
Generating train split: 226000 examples [00:02, 86478.09 examples/s]
Generating train split: 236000 examples [00:02, 87534.40 examples/s]
Generating train split: 246000 examples [00:02, 87786.52 examples/s]
Generating train split: 256000 examples [00:03, 88077.21 examples/s]
Generating train split: 266000 examples [00:03, 87827.01 examples/s]
Generating train split: 276000 examples [00:03, 87483.30 examples/s]
Generating train split: 286000 examples [00:03, 88079.86 examples/s]
Generating train split: 296000 examples [00:03, 87518.55 examples/s]
Generating train split: 306000 examples [00:03, 87396.77 examples/s]
Generating train split: 316000 examples [00:03, 87720.91 examples/s]
Generating train split: 326000 examples [00:03, 87335.85 examples/s]
Generating train split: 336000 examples [00:04, 86268.21 examples/s]
Generating train split: 346000 examples [00:04, 85782.40 examples/s]
Generating train split: 356000 examples [00:04, 86303.47 examples/s]
Generating train split: 366000 examples [00:04, 86465.35 examples/s]
Generating train split: 376000 examples [00:04, 86993.31 examples/s]
Generating train split: 386000 examples [00:04, 87173.88 examples/s]
Generating train split: 395000 examples [00:04, 86251.37 examples/s]
Generating train split: 405000 examples [00:04, 85768.37 examples/s]
Generating train split: 415000 examples [00:04, 85826.72 examples/s]
Generating train split: 424000 examples [00:05, 84907.07 examples/s]
Generating train split: 434000 examples [00:05, 85463.56 examples/s]
Generating train split: 444000 examples [00:05, 85237.03 examples/s]
Generating train split: 453000 examples [00:05, 83677.40 examples/s]
Generating train split: 463000 examples [00:05, 85739.56 examples/s]
Generating train split: 473000 examples [00:05, 87080.91 examples/s]
Generating train split: 483000 examples [00:05, 87952.32 examples/s]
Generating train split: 493000 examples [00:05, 87323.73 examples/s]
Generating train split: 502000 examples [00:05, 86521.97 examples/s]
Generating train split: 512000 examples [00:06, 85768.10 examples/s]
Generating train split: 522000 examples [00:06, 86452.02 examples/s]
Generating train split: 532000 examples [00:06, 85774.66 examples/s]
Generating train split: 542000 examples [00:06, 85602.01 examples/s]
Generating train split: 551000 examples [00:06, 84063.43 examples/s]
Generating train split: 561000 examples [00:06, 84681.42 examples/s]
Generating train split: 571000 examples [00:06, 86243.97 examples/s]
Generating train split: 580000 examples [00:06, 85380.43 examples/s]
Generating train split: 590000 examples [00:06, 85419.35 examples/s]
Generating train split: 600000 examples [00:07, 85187.39 examples/s]
Generating train split: 610000 examples [00:07, 85628.48 examples/s]
Generating train split: 620000 examples [00:07, 84383.54 examples/s]
Generating train split: 630000 examples [00:07, 85976.11 examples/s]
Generating train split: 640000 examples [00:07, 86427.96 examples/s]
Generating train split: 650000 examples [00:07, 86781.65 examples/s]
Generating train split: 659000 examples [00:07, 85167.62 examples/s]
Generating train split: 669000 examples [00:07, 85381.75 examples/s]
Generating train split: 679000 examples [00:08, 84192.42 examples/s]
Generating train split: 692000 examples [00:08, 82228.60 examples/s]
Generating train split: 702000 examples [00:08, 83823.11 examples/s]
Generating train split: 712000 examples [00:08, 83865.01 examples/s]
Generating train split: 722000 examples [00:08, 84569.33 examples/s]
Generating train split: 732000 examples [00:08, 84326.81 examples/s]
Generating train split: 742000 examples [00:08, 85500.13 examples/s]
Generating train split: 751000 examples [00:08, 85068.02 examples/s]
Generating train split: 760000 examples [00:08, 84033.09 examples/s]
Generating train split: 770000 examples [00:09, 85594.70 examples/s]
Generating train split: 780000 examples [00:09, 85603.06 examples/s]
Generating train split: 790000 examples [00:09, 87034.27 examples/s]
Generating train split: 800000 examples [00:09, 86862.60 examples/s]
Generating train split: 810000 examples [00:09, 87001.93 examples/s]
Generating train split: 820000 examples [00:09, 86474.57 examples/s]
Generating train split: 830000 examples [00:09, 87176.33 examples/s]
Generating train split: 840000 examples [00:09, 87095.25 examples/s]
Generating train split: 850000 examples [00:10, 86921.37 examples/s]
Generating train split: 860000 examples [00:10, 87126.06 examples/s]
Generating train split: 870000 examples [00:10, 86371.07 examples/s]
Generating train split: 880000 examples [00:10, 85422.99 examples/s]
Generating train split: 890000 examples [00:10, 86206.93 examples/s]
Generating train split: 900000 examples [00:10, 86779.75 examples/s]
Generating train split: 910000 examples [00:10, 85026.62 examples/s]
Generating train split: 919000 examples [00:10, 83353.26 examples/s]
Generating train split: 929000 examples [00:10, 84224.84 examples/s]
Generating train split: 938000 examples [00:11, 83207.09 examples/s]
Generating train split: 948000 examples [00:11, 83910.61 examples/s]
Generating train split: 957000 examples [00:11, 83243.06 examples/s]
Generating train split: 966000 examples [00:11, 83572.59 examples/s]
Generating train split: 976000 examples [00:11, 83087.39 examples/s]
Generating train split: 986000 examples [00:11, 84170.94 examples/s]
Generating train split: 996000 examples [00:11, 85265.90 examples/s]
Generating train split: 1006000 examples [00:11, 85801.57 examples/s]
Generating train split: 1015000 examples [00:11, 83710.26 examples/s]
Generating train split: 1025000 examples [00:12, 84595.62 examples/s]
Generating train split: 1035000 examples [00:12, 85143.56 examples/s]
Generating train split: 1045000 examples [00:12, 85297.17 examples/s]
Generating train split: 1055000 examples [00:12, 81195.35 examples/s]
Generating train split: 1065000 examples [00:12, 82424.47 examples/s]
Generating train split: 1075000 examples [00:12, 84795.84 examples/s]
Generating train split: 1085000 examples [00:12, 83831.64 examples/s]
Generating train split: 1095000 examples [00:12, 84412.54 examples/s]
Generating train split: 1105000 examples [00:13, 84748.39 examples/s]
Generating train split: 1115000 examples [00:13, 85113.14 examples/s]
Generating train split: 1125000 examples [00:13, 86185.27 examples/s]
Generating train split: 1135000 examples [00:13, 86039.93 examples/s]
Generating train split: 1145000 examples [00:13, 86385.33 examples/s]
Generating train split: 1155000 examples [00:13, 85573.92 examples/s]
Generating train split: 1165000 examples [00:13, 85976.58 examples/s]
Generating train split: 1175000 examples [00:13, 86828.71 examples/s]
Generating train split: 1185000 examples [00:13, 86599.84 examples/s]
Generating train split: 1195000 examples [00:14, 86734.08 examples/s]
Generating train split: 1204000 examples [00:14, 85231.90 examples/s]
Generating train split: 1214000 examples [00:14, 86569.05 examples/s]
Generating train split: 1224000 examples [00:14, 85371.92 examples/s]
Generating train split: 1234000 examples [00:14, 86467.82 examples/s]
Generating train split: 1244000 examples [00:14, 86658.77 examples/s]
Generating train split: 1254000 examples [00:14, 86159.01 examples/s]
Generating train split: 1264000 examples [00:14, 86840.36 examples/s]
Generating train split: 1274000 examples [00:14, 86993.39 examples/s]
Generating train split: 1284000 examples [00:15, 87076.52 examples/s]
Generating train split: 1294000 examples [00:15, 86498.42 examples/s]
Generating train split: 1303000 examples [00:15, 84741.48 examples/s]
Generating train split: 1313000 examples [00:15, 85346.44 examples/s]
Generating train split: 1322000 examples [00:15, 83071.77 examples/s]
Generating train split: 1332000 examples [00:15, 83930.58 examples/s]
Generating train split: 1341000 examples [00:15, 83561.79 examples/s]
Generating train split: 1351000 examples [00:15, 84996.70 examples/s]
Generating train split: 1361000 examples [00:16, 85930.24 examples/s]
Generating train split: 1371000 examples [00:16, 86096.76 examples/s]
Generating train split: 1380000 examples [00:16, 84852.41 examples/s]
Generating train split: 1389000 examples [00:16, 84047.23 examples/s]
Generating train split: 1399000 examples [00:16, 85205.79 examples/s]
Generating train split: 1408000 examples [00:16, 84498.64 examples/s]
Generating train split: 1418000 examples [00:16, 83932.59 examples/s]
Generating train split: 1427000 examples [00:16, 81871.55 examples/s]
Generating train split: 1437000 examples [00:16, 82546.97 examples/s]
Generating train split: 1447000 examples [00:17, 84250.01 examples/s]
Generating train split: 1457000 examples [00:17, 85382.51 examples/s]
Generating train split: 1467000 examples [00:17, 85543.98 examples/s]
Generating train split: 1477000 examples [00:17, 85370.28 examples/s]
Generating train split: 1487000 examples [00:17, 84414.87 examples/s]
Generating train split: 1497000 examples [00:17, 83606.03 examples/s]
Generating train split: 1507000 examples [00:17, 84278.86 examples/s]
Generating train split: 1517000 examples [00:17, 85417.13 examples/s]
Generating train split: 1527000 examples [00:17, 85126.27 examples/s]
Generating train split: 1537000 examples [00:18, 84379.75 examples/s]
Generating train split: 1547000 examples [00:18, 85167.33 examples/s]
Generating train split: 1557000 examples [00:18, 85467.08 examples/s]
Generating train split: 1567000 examples [00:18, 85949.81 examples/s]
Generating train split: 1577000 examples [00:18, 86682.34 examples/s]
Generating train split: 1587000 examples [00:18, 86801.66 examples/s]
Generating train split: 1597000 examples [00:18, 87588.23 examples/s]
Generating train split: 1606000 examples [00:18, 85745.83 examples/s]
Generating train split: 1616000 examples [00:19, 85619.21 examples/s]
Generating train split: 1626000 examples [00:19, 85977.79 examples/s]
Generating train split: 1636000 examples [00:19, 86286.98 examples/s]
Generating train split: 1646000 examples [00:19, 84246.00 examples/s]
Generating train split: 1656000 examples [00:19, 85178.17 examples/s]
Generating train split: 1666000 examples [00:19, 85607.44 examples/s]
Generating train split: 1676000 examples [00:19, 86495.07 examples/s]
Generating train split: 1686000 examples [00:19, 86204.19 examples/s]
Generating train split: 1695000 examples [00:19, 83788.64 examples/s]
Generating train split: 1704000 examples [00:20, 81523.79 examples/s]
Generating train split: 1713000 examples [00:20, 81297.08 examples/s]
Generating train split: 1723000 examples [00:20, 82966.70 examples/s]
Generating train split: 1733000 examples [00:20, 84573.66 examples/s]
Generating train split: 1743000 examples [00:20, 84958.03 examples/s]
Generating train split: 1752000 examples [00:20, 83264.35 examples/s]
Generating train split: 1762000 examples [00:20, 84485.49 examples/s]
Generating train split: 1772000 examples [00:20, 84983.35 examples/s]
Generating train split: 1781000 examples [00:20, 84385.72 examples/s]
Generating train split: 1790000 examples [00:21, 82980.87 examples/s]
Generating train split: 1802000 examples [00:21, 79163.73 examples/s]
Generating train split: 1811000 examples [00:21, 79954.08 examples/s]
Generating train split: 1821000 examples [00:21, 82058.27 examples/s]
Generating train split: 1830000 examples [00:21, 82273.01 examples/s]
Generating train split: 1840000 examples [00:21, 82295.37 examples/s]
Generating train split: 1850000 examples [00:21, 83519.01 examples/s]
Generating train split: 1860000 examples [00:21, 84805.92 examples/s]
Generating train split: 1870000 examples [00:22, 86241.41 examples/s]
Generating train split: 1880000 examples [00:22, 86657.26 examples/s]
Generating train split: 1890000 examples [00:22, 87002.45 examples/s]
Generating train split: 1900000 examples [00:22, 87286.37 examples/s]
Generating train split: 1910000 examples [00:22, 86725.64 examples/s]
Generating train split: 1920000 examples [00:22, 86498.75 examples/s]
Generating train split: 1929000 examples [00:22, 84832.86 examples/s]
Generating train split: 1938000 examples [00:22, 83030.79 examples/s]
Generating train split: 1948000 examples [00:22, 84234.37 examples/s]
Generating train split: 1958000 examples [00:23, 85729.51 examples/s]
Generating train split: 1968000 examples [00:23, 87127.56 examples/s]
Generating train split: 1978000 examples [00:23, 87116.19 examples/s]
Generating train split: 1988000 examples [00:23, 86331.77 examples/s]
Generating train split: 1997000 examples [00:23, 85465.35 examples/s]
Generating train split: 2007000 examples [00:23, 85575.89 examples/s]
Generating train split: 2017000 examples [00:23, 85162.66 examples/s]
Generating train split: 2027000 examples [00:23, 85314.68 examples/s]
Generating train split: 2037000 examples [00:23, 86654.08 examples/s]
Generating train split: 2046000 examples [00:24, 83157.90 examples/s]
Generating train split: 2055000 examples [00:24, 82434.35 examples/s]
Generating train split: 2065000 examples [00:24, 83803.40 examples/s]
Generating train split: 2074000 examples [00:24, 83040.87 examples/s]
Generating train split: 2084000 examples [00:24, 84108.12 examples/s]
Generating train split: 2094000 examples [00:24, 85434.44 examples/s]
Generating train split: 2103000 examples [00:24, 83255.41 examples/s]
Generating train split: 2113000 examples [00:24, 83096.58 examples/s]
Generating train split: 2123000 examples [00:25, 83863.81 examples/s]
Generating train split: 2133000 examples [00:25, 84164.04 examples/s]
Generating train split: 2143000 examples [00:25, 83652.80 examples/s]
Generating train split: 2153000 examples [00:25, 85322.31 examples/s]
Generating train split: 2163000 examples [00:25, 86352.06 examples/s]
Generating train split: 2173000 examples [00:25, 86989.64 examples/s]
Generating train split: 2183000 examples [00:25, 86270.05 examples/s]
Generating train split: 2192000 examples [00:25, 85468.50 examples/s]
Generating train split: 2202000 examples [00:25, 86009.06 examples/s]
Generating train split: 2212000 examples [00:26, 86529.87 examples/s]
Generating train split: 2222000 examples [00:26, 87088.38 examples/s]
Generating train split: 2232000 examples [00:26, 87230.40 examples/s]
Generating train split: 2241000 examples [00:26, 85819.02 examples/s]
Generating train split: 2251000 examples [00:26, 86536.42 examples/s]
Generating train split: 2261000 examples [00:26, 87030.70 examples/s]
Generating train split: 2271000 examples [00:26, 87656.68 examples/s]
Generating train split: 2280000 examples [00:26, 85438.40 examples/s]
Generating train split: 2289000 examples [00:26, 84266.14 examples/s]
Generating train split: 2299000 examples [00:27, 84877.86 examples/s]
Generating train split: 2308000 examples [00:27, 84140.23 examples/s]
Generating train split: 2318000 examples [00:27, 84360.55 examples/s]
Generating train split: 2327000 examples [00:27, 82922.41 examples/s]
Generating train split: 2337000 examples [00:27, 83900.95 examples/s]
Generating train split: 2347000 examples [00:27, 84090.43 examples/s]
Generating train split: 2357000 examples [00:27, 84309.20 examples/s]
Generating train split: 2367000 examples [00:27, 85122.00 examples/s]
Generating train split: 2377000 examples [00:27, 85562.36 examples/s]
Generating train split: 2387000 examples [00:28, 86407.40 examples/s]
Generating train split: 2397000 examples [00:28, 87161.97 examples/s]
Generating train split: 2406000 examples [00:28, 85384.80 examples/s]
Generating train split: 2416000 examples [00:28, 87048.88 examples/s]
Generating train split: 2426000 examples [00:28, 86160.43 examples/s]
Generating train split: 2435000 examples [00:28, 84241.15 examples/s]
Generating train split: 2445000 examples [00:28, 85114.08 examples/s]
Generating train split: 2455000 examples [00:28, 85890.50 examples/s]
Generating train split: 2465000 examples [00:29, 85727.37 examples/s]
Generating train split: 2475000 examples [00:29, 85104.98 examples/s]
Generating train split: 2485000 examples [00:29, 86208.58 examples/s]
Generating train split: 2495000 examples [00:29, 85065.36 examples/s]
Generating train split: 2504000 examples [00:29, 83005.92 examples/s]
Generating train split: 2514000 examples [00:29, 84011.89 examples/s]
Generating train split: 2524000 examples [00:29, 84838.98 examples/s]
Generating train split: 2534000 examples [00:29, 85895.42 examples/s]
Generating train split: 2544000 examples [00:29, 86117.86 examples/s]
Generating train split: 2554000 examples [00:30, 86702.70 examples/s]
Generating train split: 2564000 examples [00:30, 86681.65 examples/s]
Generating train split: 2573000 examples [00:30, 84818.90 examples/s]
Generating train split: 2583000 examples [00:30, 85757.50 examples/s]
Generating train split: 2593000 examples [00:30, 86307.90 examples/s]
Generating train split: 2603000 examples [00:30, 86480.51 examples/s]
Generating train split: 2612000 examples [00:30, 84614.68 examples/s]
Generating train split: 2621000 examples [00:30, 83748.86 examples/s]
Generating train split: 2631000 examples [00:30, 85147.30 examples/s]
Generating train split: 2641000 examples [00:31, 86189.33 examples/s]
Generating train split: 2651000 examples [00:31, 85994.95 examples/s]
Generating train split: 2661000 examples [00:31, 85263.25 examples/s]
Generating train split: 2671000 examples [00:31, 86553.31 examples/s]
Generating train split: 2680000 examples [00:31, 85941.37 examples/s]
Generating train split: 2689000 examples [00:31, 85337.24 examples/s]
Generating train split: 2699000 examples [00:31, 85710.21 examples/s]
Generating train split: 2709000 examples [00:31, 85463.83 examples/s]
Generating train split: 2718000 examples [00:31, 83560.95 examples/s]
Generating train split: 2728000 examples [00:32, 83678.08 examples/s]
Generating train split: 2737000 examples [00:32, 82913.19 examples/s]
Generating train split: 2747000 examples [00:32, 84182.39 examples/s]
Generating train split: 2757000 examples [00:32, 85404.39 examples/s]
Generating train split: 2767000 examples [00:32, 85299.97 examples/s]
Generating train split: 2777000 examples [00:32, 85239.91 examples/s]
Generating train split: 2787000 examples [00:32, 86632.37 examples/s]
Generating train split: 2797000 examples [00:32, 86744.12 examples/s]
Generating train split: 2807000 examples [00:33, 86650.80 examples/s]
Generating train split: 2817000 examples [00:33, 88043.98 examples/s]
Generating train split: 2827000 examples [00:33, 88720.06 examples/s]
Generating train split: 2837000 examples [00:33, 87771.42 examples/s]
Generating train split: 2847000 examples [00:33, 86457.74 examples/s]
Generating train split: 2857000 examples [00:33, 87083.19 examples/s]
Generating train split: 2867000 examples [00:33, 87874.70 examples/s]
Generating train split: 2877000 examples [00:33, 87668.99 examples/s]
Generating train split: 2886000 examples [00:33, 84386.38 examples/s]
Generating train split: 2895000 examples [00:34, 81970.58 examples/s]
Generating train split: 2904000 examples [00:34, 81421.48 examples/s]
Generating train split: 2914000 examples [00:34, 81800.11 examples/s]
Generating train split: 2923000 examples [00:34, 82011.82 examples/s]
Generating train split: 2932000 examples [00:34, 82227.84 examples/s]
Generating train split: 2942000 examples [00:34, 83218.29 examples/s]
Generating train split: 2952000 examples [00:34, 83115.66 examples/s]
Generating train split: 2962000 examples [00:34, 84290.87 examples/s]
Generating train split: 2972000 examples [00:34, 84661.59 examples/s]
Generating train split: 2981000 examples [00:35, 81227.65 examples/s]
Generating train split: 2993000 examples [00:35, 79047.45 examples/s]
Generating train split: 3002000 examples [00:35, 79688.35 examples/s]
Generating train split: 3011000 examples [00:35, 79776.55 examples/s]
Generating train split: 3021000 examples [00:35, 80930.07 examples/s]
Generating train split: 3030000 examples [00:35, 80281.84 examples/s]
Generating train split: 3040000 examples [00:35, 82525.34 examples/s]
Generating train split: 3050000 examples [00:35, 84391.66 examples/s]
Generating train split: 3059000 examples [00:36, 83409.26 examples/s]
Generating train split: 3068000 examples [00:36, 83659.22 examples/s]
Generating train split: 3078000 examples [00:36, 84166.60 examples/s]
Generating train split: 3088000 examples [00:36, 85513.42 examples/s]
Generating train split: 3098000 examples [00:36, 83816.73 examples/s]
Generating train split: 3108000 examples [00:36, 85782.41 examples/s]
Generating train split: 3118000 examples [00:36, 85999.17 examples/s]
Generating train split: 3128000 examples [00:36, 85923.91 examples/s]
Generating train split: 3138000 examples [00:36, 86406.53 examples/s]
Generating train split: 3151000 examples [00:37, 81947.24 examples/s]
Generating train split: 3161000 examples [00:37, 83302.28 examples/s]
Generating train split: 3170000 examples [00:37, 83325.31 examples/s]
Generating train split: 3180000 examples [00:37, 83673.87 examples/s]
Generating train split: 3189000 examples [00:37, 83274.59 examples/s]
Generating train split: 3199000 examples [00:37, 82983.10 examples/s]
Generating train split: 3208000 examples [00:37, 82320.13 examples/s]
Generating train split: 3217000 examples [00:37, 80439.34 examples/s]
Generating train split: 3227000 examples [00:38, 82688.13 examples/s]
Generating train split: 3237000 examples [00:38, 82671.70 examples/s]
Generating train split: 3247000 examples [00:38, 84222.44 examples/s]
Generating train split: 3257000 examples [00:38, 85107.69 examples/s]
Generating train split: 3266000 examples [00:38, 83977.81 examples/s]
Generating train split: 3276000 examples [00:38, 84712.60 examples/s]
Generating train split: 3286000 examples [00:38, 85930.73 examples/s]
Generating train split: 3295000 examples [00:38, 84487.70 examples/s]
Generating train split: 3305000 examples [00:38, 85347.66 examples/s]
Generating train split: 3315000 examples [00:39, 85489.81 examples/s]
Generating train split: 3325000 examples [00:39, 85427.25 examples/s]
Generating train split: 3335000 examples [00:39, 84148.04 examples/s]
Generating train split: 3345000 examples [00:39, 84932.83 examples/s]
Generating train split: 3354000 examples [00:39, 83971.40 examples/s]
Generating train split: 3364000 examples [00:39, 85109.25 examples/s]
Generating train split: 3374000 examples [00:39, 85493.40 examples/s]
Generating train split: 3384000 examples [00:39, 84126.12 examples/s]
Generating train split: 3393000 examples [00:40, 82408.22 examples/s]
Generating train split: 3406000 examples [00:40, 81852.36 examples/s]
Generating train split: 3416000 examples [00:40, 83461.63 examples/s]
Generating train split: 3429000 examples [00:40, 81898.44 examples/s]
Generating train split: 3438000 examples [00:40, 82566.83 examples/s]
Generating train split: 3448000 examples [00:40, 84321.79 examples/s]
Generating train split: 3458000 examples [00:40, 84103.36 examples/s]
Generating train split: 3468000 examples [00:40, 84921.74 examples/s]
Generating train split: 3478000 examples [00:41, 84890.96 examples/s]
Generating train split: 3488000 examples [00:41, 85235.91 examples/s]
Generating train split: 3498000 examples [00:41, 85554.57 examples/s]
Generating train split: 3508000 examples [00:41, 85830.02 examples/s]
Generating train split: 3518000 examples [00:41, 85934.44 examples/s]
Generating train split: 3528000 examples [00:41, 85656.43 examples/s]
Generating train split: 3538000 examples [00:41, 85890.45 examples/s]
Generating train split: 3547000 examples [00:41, 84735.32 examples/s]
Generating train split: 3556000 examples [00:41, 83381.79 examples/s]
Generating train split: 3566000 examples [00:42, 83913.35 examples/s]
Generating train split: 3575000 examples [00:42, 83551.84 examples/s]
Generating train split: 3585000 examples [00:42, 85403.40 examples/s]
Generating train split: 3595000 examples [00:42, 85851.76 examples/s]
Generating train split: 3608000 examples [00:42, 83085.97 examples/s]
Generating train split: 3618000 examples [00:42, 83827.37 examples/s]
Generating train split: 3628000 examples [00:42, 84343.04 examples/s]
Generating train split: 3637000 examples [00:42, 83689.03 examples/s]
Generating train split: 3647000 examples [00:43, 84596.17 examples/s]
Generating train split: 3657000 examples [00:43, 85359.19 examples/s]
Generating train split: 3667000 examples [00:43, 86784.57 examples/s]
Generating train split: 3677000 examples [00:43, 87067.48 examples/s]
Generating train split: 3687000 examples [00:43, 86792.71 examples/s]
Generating train split: 3697000 examples [00:43, 87344.48 examples/s]
Generating train split: 3707000 examples [00:43, 86769.42 examples/s]
Generating train split: 3717000 examples [00:43, 87283.94 examples/s]
Generating train split: 3727000 examples [00:43, 87074.25 examples/s]
Generating train split: 3736000 examples [00:44, 84719.66 examples/s]
Generating train split: 3745000 examples [00:44, 84193.20 examples/s]
Generating train split: 3754000 examples [00:44, 83782.08 examples/s]
Generating train split: 3764000 examples [00:44, 84359.10 examples/s]
Generating train split: 3774000 examples [00:44, 85567.16 examples/s]
Generating train split: 3784000 examples [00:44, 85591.97 examples/s]
Generating train split: 3794000 examples [00:44, 84275.55 examples/s]
Generating train split: 3804000 examples [00:44, 84672.96 examples/s]
Generating train split: 3814000 examples [00:44, 81540.18 examples/s]
Generating train split: 3824000 examples [00:45, 82479.97 examples/s]
Generating train split: 3834000 examples [00:45, 82557.70 examples/s]
Generating train split: 3844000 examples [00:45, 82378.61 examples/s]
Generating train split: 3854000 examples [00:45, 82230.40 examples/s]
Generating train split: 3864000 examples [00:45, 83018.79 examples/s]
Generating train split: 3874000 examples [00:45, 84110.90 examples/s]
Generating train split: 3883000 examples [00:45, 83772.61 examples/s]
Generating train split: 3893000 examples [00:45, 85338.69 examples/s]
Generating train split: 3902000 examples [00:46, 84721.76 examples/s]
Generating train split: 3911000 examples [00:46, 84104.05 examples/s]
Generating train split: 3921000 examples [00:46, 85106.98 examples/s]
Generating train split: 3931000 examples [00:46, 84736.08 examples/s]
Generating train split: 3941000 examples [00:46, 85587.24 examples/s]
Generating train split: 3951000 examples [00:46, 86275.87 examples/s]
Generating train split: 3961000 examples [00:46, 86891.04 examples/s]
Generating train split: 3971000 examples [00:46, 86377.42 examples/s]
Generating train split: 3981000 examples [00:46, 86069.18 examples/s]
Generating train split: 3991000 examples [00:47, 86391.65 examples/s]
Generating train split: 4000000 examples [00:47, 84129.89 examples/s]
Generating train split: 4009000 examples [00:47, 80222.22 examples/s]
Generating train split: 4018000 examples [00:47, 79965.19 examples/s]
Generating train split: 4027000 examples [00:47, 79112.16 examples/s]
Generating train split: 4037000 examples [00:47, 82097.08 examples/s]
Generating train split: 4047000 examples [00:47, 82506.56 examples/s]
Generating train split: 4060000 examples [00:47, 81439.75 examples/s]
Generating train split: 4069000 examples [00:48, 80436.36 examples/s]
Generating train split: 4078000 examples [00:48, 80862.88 examples/s]
Generating train split: 4088000 examples [00:48, 82398.61 examples/s]
Generating train split: 4097000 examples [00:48, 81883.33 examples/s]
Generating train split: 4107000 examples [00:48, 82627.15 examples/s]
Generating train split: 4116000 examples [00:48, 81517.76 examples/s]
Generating train split: 4126000 examples [00:48, 82445.96 examples/s]
Generating train split: 4136000 examples [00:48, 82118.60 examples/s]
Generating train split: 4146000 examples [00:48, 83941.45 examples/s]
Generating train split: 4156000 examples [00:49, 84547.44 examples/s]
Generating train split: 4166000 examples [00:49, 84400.85 examples/s]
Generating train split: 4176000 examples [00:49, 84172.10 examples/s]
Generating train split: 4186000 examples [00:49, 84931.71 examples/s]
Generating train split: 4196000 examples [00:49, 81978.80 examples/s]
Generating train split: 4206000 examples [00:49, 83801.76 examples/s]
Generating train split: 4216000 examples [00:49, 84676.01 examples/s]
Generating train split: 4226000 examples [00:49, 84928.64 examples/s]
Generating train split: 4235000 examples [00:50, 84065.64 examples/s]
Generating train split: 4244000 examples [00:50, 83564.61 examples/s]
Generating train split: 4254000 examples [00:50, 84870.65 examples/s]
Generating train split: 4264000 examples [00:50, 86076.72 examples/s]
Generating train split: 4273000 examples [00:50, 85692.46 examples/s]
Generating train split: 4283000 examples [00:50, 85814.18 examples/s]
Generating train split: 4292000 examples [00:50, 82118.61 examples/s]
Generating train split: 4302000 examples [00:50, 82579.27 examples/s]
Generating train split: 4312000 examples [00:50, 83150.28 examples/s]
Generating train split: 4322000 examples [00:51, 83462.84 examples/s]
Generating train split: 4331000 examples [00:51, 82870.28 examples/s]
Generating train split: 4341000 examples [00:51, 80922.30 examples/s]
Generating train split: 4351000 examples [00:51, 81903.79 examples/s]
Generating train split: 4360000 examples [00:51, 82394.51 examples/s]
Generating train split: 4370000 examples [00:51, 83823.09 examples/s]
Generating train split: 4380000 examples [00:51, 84703.29 examples/s]
Generating train split: 4390000 examples [00:51, 85328.62 examples/s]
Generating train split: 4400000 examples [00:51, 86017.02 examples/s]
Generating train split: 4410000 examples [00:52, 86532.90 examples/s]
Generating train split: 4420000 examples [00:52, 85785.28 examples/s]
Generating train split: 4430000 examples [00:52, 85980.49 examples/s]
Generating train split: 4440000 examples [00:52, 85610.07 examples/s]
Generating train split: 4449000 examples [00:52, 83282.41 examples/s]
Generating train split: 4459000 examples [00:52, 84082.12 examples/s]
Generating train split: 4469000 examples [00:52, 85324.04 examples/s]
Generating train split: 4479000 examples [00:52, 85914.88 examples/s]
Generating train split: 4489000 examples [00:53, 85803.30 examples/s]
Generating train split: 4499000 examples [00:53, 85469.68 examples/s]
Generating train split: 4509000 examples [00:53, 86304.35 examples/s]
Generating train split: 4519000 examples [00:53, 86391.98 examples/s]
Generating train split: 4529000 examples [00:53, 86526.42 examples/s]
Generating train split: 4539000 examples [00:53, 85911.08 examples/s]
Generating train split: 4549000 examples [00:53, 85992.40 examples/s]
Generating train split: 4559000 examples [00:53, 86838.91 examples/s]
Generating train split: 4569000 examples [00:53, 86402.05 examples/s]
Generating train split: 4579000 examples [00:54, 84858.84 examples/s]
Generating train split: 4592000 examples [00:54, 83912.26 examples/s]
Generating train split: 4602000 examples [00:54, 84291.96 examples/s]
Generating train split: 4612000 examples [00:54, 84086.82 examples/s]
Generating train split: 4622000 examples [00:54, 83665.21 examples/s]
Generating train split: 4632000 examples [00:54, 84719.00 examples/s]
Generating train split: 4642000 examples [00:54, 84997.57 examples/s]
Generating train split: 4652000 examples [00:54, 84943.13 examples/s]
Generating train split: 4662000 examples [00:55, 85632.90 examples/s]
Generating train split: 4672000 examples [00:55, 85676.10 examples/s]
Generating train split: 4682000 examples [00:55, 86204.91 examples/s]
Generating train split: 4691000 examples [00:55, 84901.61 examples/s]
Generating train split: 4700000 examples [00:55, 84447.00 examples/s]
Generating train split: 4710000 examples [00:55, 84703.51 examples/s]
Generating train split: 4719000 examples [00:55, 83801.24 examples/s]
Generating train split: 4728000 examples [00:55, 82857.45 examples/s]
Generating train split: 4738000 examples [00:55, 82706.94 examples/s]
Generating train split: 4748000 examples [00:56, 83695.67 examples/s]
Generating train split: 4758000 examples [00:56, 84588.73 examples/s]
Generating train split: 4768000 examples [00:56, 84206.44 examples/s]
Generating train split: 4777000 examples [00:56, 83566.29 examples/s]
Generating train split: 4787000 examples [00:56, 83711.52 examples/s]
Generating train split: 4796000 examples [00:56, 83497.71 examples/s]
Generating train split: 4806000 examples [00:56, 84084.08 examples/s]
Generating train split: 4816000 examples [00:56, 83172.92 examples/s]
Generating train split: 4826000 examples [00:57, 83939.58 examples/s]
Generating train split: 4836000 examples [00:57, 84761.05 examples/s]
Generating train split: 4846000 examples [00:57, 84826.84 examples/s]
Generating train split: 4855000 examples [00:57, 83674.78 examples/s]
Generating train split: 4864000 examples [00:57, 83364.68 examples/s]
Generating train split: 4873000 examples [00:57, 82572.41 examples/s]
Generating train split: 4882000 examples [00:57, 82019.75 examples/s]
Generating train split: 4891000 examples [00:57, 82161.85 examples/s]
Generating train split: 4901000 examples [00:57, 83747.54 examples/s]
Generating train split: 4911000 examples [00:58, 84472.76 examples/s]
Generating train split: 4920000 examples [00:58, 84432.30 examples/s]
Generating train split: 4929000 examples [00:58, 83502.29 examples/s]
Generating train split: 4939000 examples [00:58, 84992.93 examples/s]
Generating train split: 4949000 examples [00:58, 85554.75 examples/s]
Generating train split: 4958000 examples [00:58, 84290.15 examples/s]
Generating train split: 4968000 examples [00:58, 84601.80 examples/s]
Generating train split: 4978000 examples [00:58, 85351.54 examples/s]
Generating train split: 4988000 examples [00:58, 85936.79 examples/s]
Generating train split: 4998000 examples [00:59, 86379.31 examples/s]
Generating train split: 5008000 examples [00:59, 87139.46 examples/s]
Generating train split: 5018000 examples [00:59, 87078.52 examples/s]
Generating train split: 5027000 examples [00:59, 85754.05 examples/s]
Generating train split: 5037000 examples [00:59, 85255.52 examples/s]
Generating train split: 5047000 examples [00:59, 85373.64 examples/s]
Generating train split: 5057000 examples [00:59, 85498.75 examples/s]
Generating train split: 5067000 examples [00:59, 86417.01 examples/s]
Generating train split: 5077000 examples [00:59, 86569.18 examples/s]
Generating train split: 5087000 examples [01:00, 86105.86 examples/s]
Generating train split: 5097000 examples [01:00, 86549.66 examples/s]
Generating train split: 5107000 examples [01:00, 86708.77 examples/s]
Generating train split: 5117000 examples [01:00, 86416.66 examples/s]
Generating train split: 5126000 examples [01:00, 84827.00 examples/s]
Generating train split: 5135000 examples [01:00, 84100.47 examples/s]
Generating train split: 5145000 examples [01:00, 85576.40 examples/s]
Generating train split: 5155000 examples [01:00, 86608.94 examples/s]
Generating train split: 5165000 examples [01:00, 87399.99 examples/s]
Generating train split: 5175000 examples [01:01, 87005.48 examples/s]
Generating train split: 5185000 examples [01:01, 86554.36 examples/s]
Generating train split: 5195000 examples [01:01, 85886.51 examples/s]
Generating train split: 5205000 examples [01:01, 86418.77 examples/s]
Generating train split: 5214000 examples [01:01, 85159.27 examples/s]
Generating train split: 5227000 examples [01:01, 82195.67 examples/s]
Generating train split: 5237000 examples [01:01, 81465.97 examples/s]
Generating train split: 5247000 examples [01:01, 83475.11 examples/s]
Generating train split: 5257000 examples [01:02, 84941.64 examples/s]
Generating train split: 5267000 examples [01:02, 86049.24 examples/s]
Generating train split: 5276000 examples [01:02, 84949.82 examples/s]
Generating train split: 5285000 examples [01:02, 84678.34 examples/s]
Generating train split: 5295000 examples [01:02, 85072.13 examples/s]
Generating train split: 5305000 examples [01:02, 85001.39 examples/s]
Generating train split: 5315000 examples [01:02, 85927.46 examples/s]
Generating train split: 5325000 examples [01:02, 87040.59 examples/s]
Generating train split: 5335000 examples [01:02, 86376.93 examples/s]
Generating train split: 5345000 examples [01:03, 86005.52 examples/s]
Generating train split: 5355000 examples [01:03, 85717.88 examples/s]
Generating train split: 5364000 examples [01:03, 84171.99 examples/s]
Generating train split: 5374000 examples [01:03, 84271.75 examples/s]
Generating train split: 5384000 examples [01:03, 84131.83 examples/s]
Generating train split: 5394000 examples [01:03, 84295.39 examples/s]
Generating train split: 5403000 examples [01:03, 84223.19 examples/s]
Generating train split: 5413000 examples [01:03, 84010.35 examples/s]
Generating train split: 5423000 examples [01:04, 84584.77 examples/s]
Generating train split: 5432000 examples [01:04, 83512.80 examples/s]
Generating train split: 5442000 examples [01:04, 84661.36 examples/s]
Generating train split: 5452000 examples [01:04, 85719.13 examples/s]
Generating train split: 5462000 examples [01:04, 86660.20 examples/s]
Generating train split: 5472000 examples [01:04, 87948.60 examples/s]
Generating train split: 5482000 examples [01:04, 88807.64 examples/s]
Generating train split: 5492000 examples [01:04, 88772.44 examples/s]
Generating train split: 5502000 examples [01:04, 87988.47 examples/s]
Generating train split: 5512000 examples [01:05, 88083.88 examples/s]
Generating train split: 5522000 examples [01:05, 88015.42 examples/s]
Generating train split: 5532000 examples [01:05, 86499.63 examples/s]
Generating train split: 5542000 examples [01:05, 85925.89 examples/s]
Generating train split: 5551000 examples [01:05, 84320.82 examples/s]
Generating train split: 5560000 examples [01:05, 83238.12 examples/s]
Generating train split: 5569000 examples [01:05, 82148.22 examples/s]
Generating train split: 5579000 examples [01:05, 83330.21 examples/s]
Generating train split: 5589000 examples [01:05, 84431.95 examples/s]
Generating train split: 5598000 examples [01:06, 83774.34 examples/s]
Generating train split: 5608000 examples [01:06, 85013.09 examples/s]
Generating train split: 5618000 examples [01:06, 86087.77 examples/s]
Generating train split: 5628000 examples [01:06, 86935.72 examples/s]
Generating train split: 5638000 examples [01:06, 86813.11 examples/s]
Generating train split: 5648000 examples [01:06, 87365.79 examples/s]
Generating train split: 5658000 examples [01:06, 87890.32 examples/s]
Generating train split: 5668000 examples [01:06, 87273.73 examples/s]
Generating train split: 5678000 examples [01:06, 87929.76 examples/s]
Generating train split: 5688000 examples [01:07, 88276.62 examples/s]
Generating train split: 5698000 examples [01:07, 87936.75 examples/s]
Generating train split: 5707000 examples [01:07, 87376.23 examples/s]
Generating train split: 5717000 examples [01:07, 87175.80 examples/s]
Generating train split: 5727000 examples [01:07, 86407.49 examples/s]
Generating train split: 5737000 examples [01:07, 86657.85 examples/s]
Generating train split: 5747000 examples [01:07, 85200.14 examples/s]
Generating train split: 5757000 examples [01:07, 85615.65 examples/s]
Generating train split: 5767000 examples [01:08, 86676.59 examples/s]
Generating train split: 5779000 examples [01:08, 80627.42 examples/s]
Generating train split: 5789000 examples [01:08, 82301.93 examples/s]
Generating train split: 5799000 examples [01:08, 83424.07 examples/s]
Generating train split: 5809000 examples [01:08, 84468.19 examples/s]
Generating train split: 5819000 examples [01:08, 85428.27 examples/s]
Generating train split: 5829000 examples [01:08, 85908.13 examples/s]
Generating train split: 5838000 examples [01:08, 84554.36 examples/s]
Generating train split: 5848000 examples [01:08, 84579.43 examples/s]
Generating train split: 5858000 examples [01:09, 85538.05 examples/s]
Generating train split: 5867000 examples [01:09, 84876.62 examples/s]
Generating train split: 5877000 examples [01:09, 85291.99 examples/s]
Generating train split: 5887000 examples [01:09, 85475.58 examples/s]
Generating train split: 5896000 examples [01:09, 84057.50 examples/s]
Generating train split: 5905000 examples [01:09, 82516.32 examples/s]
Generating train split: 5915000 examples [01:09, 82566.68 examples/s]
Generating train split: 5924000 examples [01:09, 81831.05 examples/s]
Generating train split: 5934000 examples [01:10, 83091.08 examples/s]
Generating train split: 5944000 examples [01:10, 84069.81 examples/s]
Generating train split: 5954000 examples [01:10, 83723.13 examples/s]
Generating train split: 5964000 examples [01:10, 84531.60 examples/s]
Generating train split: 5973000 examples [01:10, 83145.72 examples/s]
Generating train split: 5983000 examples [01:10, 83693.07 examples/s]
Generating train split: 5992000 examples [01:10, 82014.20 examples/s]
Generating train split: 6002000 examples [01:10, 82120.59 examples/s]
Generating train split: 6012000 examples [01:10, 82263.29 examples/s]
Generating train split: 6022000 examples [01:11, 84265.80 examples/s]
Generating train split: 6032000 examples [01:11, 85101.03 examples/s]
Generating train split: 6041000 examples [01:11, 84256.94 examples/s]
Generating train split: 6051000 examples [01:11, 85443.03 examples/s]
Generating train split: 6061000 examples [01:11, 85043.23 examples/s]
Generating train split: 6071000 examples [01:11, 85513.91 examples/s]
Generating train split: 6081000 examples [01:11, 85033.60 examples/s]
Generating train split: 6091000 examples [01:11, 85435.60 examples/s]
Generating train split: 6101000 examples [01:11, 85440.88 examples/s]
Generating train split: 6111000 examples [01:12, 85171.68 examples/s]
Generating train split: 6121000 examples [01:12, 85528.80 examples/s]
Generating train split: 6131000 examples [01:12, 86003.03 examples/s]
Generating train split: 6141000 examples [01:12, 86584.19 examples/s]
Generating train split: 6151000 examples [01:12, 86552.33 examples/s]
Generating train split: 6161000 examples [01:12, 86365.13 examples/s]
Generating train split: 6171000 examples [01:12, 86395.18 examples/s]
Generating train split: 6180000 examples [01:12, 84406.75 examples/s]
Generating train split: 6190000 examples [01:13, 85935.85 examples/s]
Generating train split: 6200000 examples [01:13, 86249.62 examples/s]
Generating train split: 6210000 examples [01:13, 86155.00 examples/s]
Generating train split: 6220000 examples [01:13, 86787.54 examples/s]
Generating train split: 6230000 examples [01:13, 86855.18 examples/s]
Generating train split: 6240000 examples [01:13, 86612.81 examples/s]
Generating train split: 6250000 examples [01:13, 84951.22 examples/s]
Generating train split: 6260000 examples [01:13, 85470.42 examples/s]
Generating train split: 6270000 examples [01:13, 86042.92 examples/s]
Generating train split: 6279000 examples [01:14, 80006.15 examples/s]
Generating train split: 6288000 examples [01:14, 79864.95 examples/s]
Generating train split: 6298000 examples [01:14, 80606.38 examples/s]
Generating train split: 6307000 examples [01:14, 80965.68 examples/s]
Generating train split: 6317000 examples [01:14, 82297.59 examples/s]
Generating train split: 6326000 examples [01:14, 82249.82 examples/s]
Generating train split: 6336000 examples [01:14, 83310.77 examples/s]
Generating train split: 6345000 examples [01:14, 83303.32 examples/s]
Generating train split: 6355000 examples [01:15, 84160.64 examples/s]
Generating train split: 6364000 examples [01:15, 81682.44 examples/s]
Generating train split: 6373000 examples [01:15, 82477.56 examples/s]
Generating train split: 6382000 examples [01:15, 81848.96 examples/s]
Generating train split: 6392000 examples [01:15, 82310.96 examples/s]
Generating train split: 6401000 examples [01:15, 82163.94 examples/s]
Generating train split: 6411000 examples [01:15, 83719.89 examples/s]
Generating train split: 6421000 examples [01:15, 84700.10 examples/s]
Generating train split: 6434000 examples [01:15, 83155.47 examples/s]
Generating train split: 6444000 examples [01:16, 83578.16 examples/s]
Generating train split: 6454000 examples [01:16, 83471.56 examples/s]
Generating train split: 6464000 examples [01:16, 84775.92 examples/s]
Generating train split: 6474000 examples [01:16, 85949.91 examples/s]
Generating train split: 6484000 examples [01:16, 85115.97 examples/s]
Generating train split: 6494000 examples [01:16, 85458.98 examples/s]
Generating train split: 6503000 examples [01:16, 84422.36 examples/s]
Generating train split: 6513000 examples [01:16, 84980.80 examples/s]
Generating train split: 6523000 examples [01:17, 85286.76 examples/s]
Generating train split: 6532000 examples [01:17, 83965.98 examples/s]
Generating train split: 6542000 examples [01:17, 85359.35 examples/s]
Generating train split: 6551000 examples [01:17, 84121.88 examples/s]
Generating train split: 6561000 examples [01:17, 84163.27 examples/s]
Generating train split: 6571000 examples [01:17, 85330.97 examples/s]
Generating train split: 6581000 examples [01:17, 85816.01 examples/s]
Generating train split: 6591000 examples [01:17, 86263.67 examples/s]
Generating train split: 6601000 examples [01:17, 86230.20 examples/s]
Generating train split: 6610000 examples [01:18, 85236.68 examples/s]
Generating train split: 6620000 examples [01:18, 85490.63 examples/s]
Generating train split: 6630000 examples [01:18, 86064.18 examples/s]
Generating train split: 6640000 examples [01:18, 85600.27 examples/s]
Generating train split: 6650000 examples [01:18, 86856.78 examples/s]
Generating train split: 6659000 examples [01:18, 85240.47 examples/s]
Generating train split: 6669000 examples [01:18, 85981.98 examples/s]
Generating train split: 6678000 examples [01:18, 83978.67 examples/s]
Generating train split: 6688000 examples [01:18, 84947.42 examples/s]
Generating train split: 6698000 examples [01:19, 85667.16 examples/s]
Generating train split: 6708000 examples [01:19, 85909.89 examples/s]
Generating train split: 6718000 examples [01:19, 87004.17 examples/s]
Generating train split: 6728000 examples [01:19, 87265.78 examples/s]
Generating train split: 6738000 examples [01:19, 86203.54 examples/s]
Generating train split: 6748000 examples [01:19, 87443.02 examples/s]
Generating train split: 6758000 examples [01:19, 87327.64 examples/s]
Generating train split: 6768000 examples [01:19, 87820.97 examples/s]
Generating train split: 6777000 examples [01:19, 85538.33 examples/s]
Generating train split: 6787000 examples [01:20, 86298.14 examples/s]
Generating train split: 6797000 examples [01:20, 85735.95 examples/s]
Generating train split: 6807000 examples [01:20, 85067.65 examples/s]
Generating train split: 6817000 examples [01:20, 85407.21 examples/s]
Generating train split: 6827000 examples [01:20, 86067.10 examples/s]
Generating train split: 6837000 examples [01:20, 86225.73 examples/s]
Generating train split: 6846000 examples [01:20, 83681.32 examples/s]
Generating train split: 6856000 examples [01:20, 84660.78 examples/s]
Generating train split: 6865000 examples [01:21, 83277.41 examples/s]
Generating train split: 6874000 examples [01:21, 82559.60 examples/s]
Generating train split: 6883000 examples [01:21, 82289.91 examples/s]
Generating train split: 6892000 examples [01:21, 81691.52 examples/s]
Generating train split: 6905000 examples [01:21, 80741.44 examples/s]
Generating train split: 6915000 examples [01:21, 81948.76 examples/s]
Generating train split: 6924000 examples [01:21, 81671.54 examples/s]
Generating train split: 6934000 examples [01:21, 82326.19 examples/s]
Generating train split: 6944000 examples [01:21, 82986.44 examples/s]
Generating train split: 6954000 examples [01:22, 83584.51 examples/s]
Generating train split: 6967000 examples [01:22, 81942.21 examples/s]
Generating train split: 6977000 examples [01:22, 82914.07 examples/s]
Generating train split: 6987000 examples [01:22, 83583.78 examples/s]
Generating train split: 6997000 examples [01:22, 83562.60 examples/s]
Generating train split: 7007000 examples [01:22, 84266.58 examples/s]
Generating train split: 7017000 examples [01:22, 85172.00 examples/s]
Generating train split: 7027000 examples [01:22, 85297.92 examples/s]
Generating train split: 7037000 examples [01:23, 84078.15 examples/s]
Generating train split: 7047000 examples [01:23, 84321.64 examples/s]
Generating train split: 7057000 examples [01:23, 84918.22 examples/s]
Generating train split: 7067000 examples [01:23, 86179.60 examples/s]
Generating train split: 7077000 examples [01:23, 86397.96 examples/s]
Generating train split: 7087000 examples [01:23, 87577.35 examples/s]
Generating train split: 7097000 examples [01:23, 87820.72 examples/s]
Generating train split: 7107000 examples [01:23, 87546.93 examples/s]
Generating train split: 7117000 examples [01:23, 88317.21 examples/s]
Generating train split: 7127000 examples [01:24, 87784.71 examples/s]
Generating train split: 7137000 examples [01:24, 88073.06 examples/s]
Generating train split: 7147000 examples [01:24, 87916.74 examples/s]
Generating train split: 7157000 examples [01:24, 86842.35 examples/s]
Generating train split: 7167000 examples [01:24, 87164.42 examples/s]
Generating train split: 7177000 examples [01:24, 86868.92 examples/s]
Generating train split: 7190000 examples [01:24, 84683.17 examples/s]
Generating train split: 7199000 examples [01:24, 83630.68 examples/s]
Generating train split: 7209000 examples [01:25, 84490.40 examples/s]
Generating train split: 7218000 examples [01:25, 83321.58 examples/s]
Generating train split: 7228000 examples [01:25, 84579.77 examples/s]
Generating train split: 7238000 examples [01:25, 85228.94 examples/s]
Generating train split: 7248000 examples [01:25, 84874.60 examples/s]
Generating train split: 7257000 examples [01:25, 83193.30 examples/s]
Generating train split: 7267000 examples [01:25, 84202.01 examples/s]
Generating train split: 7277000 examples [01:25, 85159.18 examples/s]
Generating train split: 7287000 examples [01:25, 84941.94 examples/s]
Generating train split: 7297000 examples [01:26, 86196.72 examples/s]
Generating train split: 7307000 examples [01:26, 85806.65 examples/s]
Generating train split: 7316000 examples [01:26, 84723.12 examples/s]
Generating train split: 7329000 examples [01:26, 81049.73 examples/s]
Generating train split: 7338000 examples [01:26, 80644.90 examples/s]
Generating train split: 7351000 examples [01:26, 80333.92 examples/s]
Generating train split: 7360000 examples [01:26, 79719.44 examples/s]
Generating train split: 7370000 examples [01:27, 81188.42 examples/s]
Generating train split: 7379000 examples [01:27, 81249.62 examples/s]
Generating train split: 7389000 examples [01:27, 83899.44 examples/s]
Generating train split: 7399000 examples [01:27, 84490.37 examples/s]
Generating train split: 7408000 examples [01:27, 83447.13 examples/s]
Generating train split: 7418000 examples [01:27, 84374.59 examples/s]
Generating train split: 7428000 examples [01:27, 84743.80 examples/s]
Generating train split: 7438000 examples [01:27, 84158.34 examples/s]
Generating train split: 7447000 examples [01:27, 83707.05 examples/s]
Generating train split: 7457000 examples [01:28, 84145.82 examples/s]
Generating train split: 7467000 examples [01:28, 84882.61 examples/s]
Generating train split: 7477000 examples [01:28, 85112.61 examples/s]
Generating train split: 7487000 examples [01:28, 85782.93 examples/s]
Generating train split: 7497000 examples [01:28, 85803.85 examples/s]
Generating train split: 7507000 examples [01:28, 85458.24 examples/s]
Generating train split: 7517000 examples [01:28, 84420.05 examples/s]
Generating train split: 7526000 examples [01:28, 84205.48 examples/s]
Generating train split: 7535000 examples [01:28, 83631.17 examples/s]
Generating train split: 7544000 examples [01:29, 82892.71 examples/s]
Generating train split: 7554000 examples [01:29, 85016.37 examples/s]
Generating train split: 7564000 examples [01:29, 84434.95 examples/s]
Generating train split: 7574000 examples [01:29, 84856.26 examples/s]
Generating train split: 7584000 examples [01:29, 84695.63 examples/s]
Generating train split: 7593000 examples [01:29, 81435.05 examples/s]
Generating train split: 7603000 examples [01:29, 83152.57 examples/s]
Generating train split: 7613000 examples [01:29, 84623.83 examples/s]
Generating train split: 7623000 examples [01:30, 84490.98 examples/s]
Generating train split: 7633000 examples [01:30, 84453.27 examples/s]
Generating train split: 7643000 examples [01:30, 84304.04 examples/s]
Generating train split: 7652000 examples [01:30, 81958.26 examples/s]
Generating train split: 7662000 examples [01:30, 82673.00 examples/s]
Generating train split: 7672000 examples [01:30, 83037.90 examples/s]
Generating train split: 7682000 examples [01:30, 82986.28 examples/s]
Generating train split: 7692000 examples [01:30, 83151.59 examples/s]
Generating train split: 7702000 examples [01:30, 83296.28 examples/s]
Generating train split: 7712000 examples [01:31, 83676.14 examples/s]
Generating train split: 7721000 examples [01:31, 82504.28 examples/s]
Generating train split: 7731000 examples [01:31, 83375.72 examples/s]
Generating train split: 7741000 examples [01:31, 83785.27 examples/s]
Generating train split: 7751000 examples [01:31, 84672.33 examples/s]
Generating train split: 7761000 examples [01:31, 85108.38 examples/s]
Generating train split: 7771000 examples [01:31, 85555.16 examples/s]
Generating train split: 7781000 examples [01:31, 85548.83 examples/s]
Generating train split: 7790000 examples [01:31, 84473.14 examples/s]
Generating train split: 7799000 examples [01:32, 83810.46 examples/s]
Generating train split: 7809000 examples [01:32, 83665.26 examples/s]
Generating train split: 7818000 examples [01:32, 83465.84 examples/s]
Generating train split: 7828000 examples [01:32, 82681.78 examples/s]
Generating train split: 7838000 examples [01:32, 83018.21 examples/s]
Generating train split: 7848000 examples [01:32, 83786.64 examples/s]
Generating train split: 7857000 examples [01:32, 83011.70 examples/s]
Generating train split: 7867000 examples [01:32, 83059.23 examples/s]
Generating train split: 7877000 examples [01:33, 83816.25 examples/s]
Generating train split: 7887000 examples [01:33, 84577.38 examples/s]
Generating train split: 7897000 examples [01:33, 84653.10 examples/s]
Generating train split: 7907000 examples [01:33, 84536.65 examples/s]
Generating train split: 7917000 examples [01:33, 84204.73 examples/s]
Generating train split: 7927000 examples [01:33, 84554.51 examples/s]
Generating train split: 7936000 examples [01:33, 83338.34 examples/s]
Generating train split: 7946000 examples [01:33, 84008.91 examples/s]
Generating train split: 7956000 examples [01:33, 84890.56 examples/s]
Generating train split: 7966000 examples [01:34, 84137.40 examples/s]
Generating train split: 7976000 examples [01:34, 85572.47 examples/s]
Generating train split: 7986000 examples [01:34, 85787.48 examples/s]
Generating train split: 7996000 examples [01:34, 85594.30 examples/s]
Generating train split: 8006000 examples [01:34, 84532.47 examples/s]
Generating train split: 8015000 examples [01:34, 83352.31 examples/s]
Generating train split: 8025000 examples [01:34, 84395.47 examples/s]
Generating train split: 8034000 examples [01:34, 83741.59 examples/s]
Generating train split: 8044000 examples [01:35, 84625.20 examples/s]
Generating train split: 8054000 examples [01:35, 84812.40 examples/s]
Generating train split: 8064000 examples [01:35, 85297.69 examples/s]
Generating train split: 8073000 examples [01:35, 84921.79 examples/s]
Generating train split: 8083000 examples [01:35, 86074.08 examples/s]
Generating train split: 8093000 examples [01:35, 85815.61 examples/s]
Generating train split: 8103000 examples [01:35, 85654.82 examples/s]
Generating train split: 8113000 examples [01:35, 85340.97 examples/s]
Generating train split: 8123000 examples [01:35, 85784.34 examples/s]
Generating train split: 8133000 examples [01:36, 86165.67 examples/s]
Generating train split: 8143000 examples [01:36, 86468.53 examples/s]
Generating train split: 8153000 examples [01:36, 86476.20 examples/s]
Generating train split: 8162000 examples [01:36, 84770.30 examples/s]
Generating train split: 8171000 examples [01:36, 84240.22 examples/s]
Generating train split: 8181000 examples [01:36, 84904.83 examples/s]
Generating train split: 8191000 examples [01:36, 85209.31 examples/s]
Generating train split: 8201000 examples [01:36, 85553.04 examples/s]
Generating train split: 8211000 examples [01:36, 86513.14 examples/s]
Generating train split: 8221000 examples [01:37, 86065.38 examples/s]
Generating train split: 8231000 examples [01:37, 86285.90 examples/s]
Generating train split: 8241000 examples [01:37, 86113.97 examples/s]
Generating train split: 8251000 examples [01:37, 85975.20 examples/s]
Generating train split: 8261000 examples [01:37, 86940.39 examples/s]
Generating train split: 8271000 examples [01:37, 87355.89 examples/s]
Generating train split: 8281000 examples [01:37, 87057.32 examples/s]
Generating train split: 8291000 examples [01:37, 87063.04 examples/s]
Generating train split: 8301000 examples [01:37, 86562.71 examples/s]
Generating train split: 8311000 examples [01:38, 86908.52 examples/s]
Generating train split: 8321000 examples [01:38, 87139.24 examples/s]
Generating train split: 8331000 examples [01:38, 87342.25 examples/s]
Generating train split: 8341000 examples [01:38, 87164.92 examples/s]
Generating train split: 8350000 examples [01:38, 86000.09 examples/s]
Generating train split: 8360000 examples [01:38, 85246.71 examples/s]
Generating train split: 8372000 examples [01:38, 80056.46 examples/s]
Generating train split: 8381000 examples [01:38, 80414.94 examples/s]
Generating train split: 8391000 examples [01:39, 82309.12 examples/s]
Generating train split: 8401000 examples [01:39, 83773.41 examples/s]
Generating train split: 8411000 examples [01:39, 84637.77 examples/s]
Generating train split: 8421000 examples [01:39, 85055.62 examples/s]
Generating train split: 8431000 examples [01:39, 85198.06 examples/s]
Generating train split: 8441000 examples [01:39, 85567.81 examples/s]
Generating train split: 8451000 examples [01:39, 85997.50 examples/s]
Generating train split: 8460000 examples [01:39, 84658.57 examples/s]
Generating train split: 8470000 examples [01:39, 84938.28 examples/s]
Generating train split: 8480000 examples [01:40, 84605.94 examples/s]
Generating train split: 8490000 examples [01:40, 85050.60 examples/s]
Generating train split: 8500000 examples [01:40, 85047.84 examples/s]
Generating train split: 8510000 examples [01:40, 84452.51 examples/s]
Generating train split: 8520000 examples [01:40, 84797.08 examples/s]
Generating train split: 8529000 examples [01:40, 83543.31 examples/s]
Generating train split: 8539000 examples [01:40, 84731.46 examples/s]
Generating train split: 8549000 examples [01:40, 85748.87 examples/s]
Generating train split: 8559000 examples [01:41, 86540.91 examples/s]
Generating train split: 8569000 examples [01:41, 87056.88 examples/s]
Generating train split: 8579000 examples [01:41, 87332.29 examples/s]
Generating train split: 8589000 examples [01:41, 87432.76 examples/s]
Generating train split: 8599000 examples [01:41, 86432.48 examples/s]
Generating train split: 8609000 examples [01:41, 87280.91 examples/s]
Generating train split: 8619000 examples [01:41, 86644.26 examples/s]
Generating train split: 8629000 examples [01:41, 87185.68 examples/s]
Generating train split: 8638000 examples [01:41, 84520.50 examples/s]
Generating train split: 8648000 examples [01:42, 85409.37 examples/s]
Generating train split: 8658000 examples [01:42, 85595.68 examples/s]
Generating train split: 8668000 examples [01:42, 85097.46 examples/s]
Generating train split: 8678000 examples [01:42, 85361.23 examples/s]
Generating train split: 8688000 examples [01:42, 85012.00 examples/s]
Generating train split: 8698000 examples [01:42, 83743.67 examples/s]
Generating train split: 8707000 examples [01:42, 83265.29 examples/s]
Generating train split: 8717000 examples [01:42, 84436.58 examples/s]
Generating train split: 8727000 examples [01:43, 84240.13 examples/s]
Generating train split: 8737000 examples [01:43, 83978.17 examples/s]
Generating train split: 8747000 examples [01:43, 84533.20 examples/s]
Generating train split: 8757000 examples [01:43, 84468.41 examples/s]
Generating train split: 8767000 examples [01:43, 84469.18 examples/s]
Generating train split: 8780000 examples [01:43, 82383.82 examples/s]
Generating train split: 8790000 examples [01:43, 83692.69 examples/s]
Generating train split: 8799000 examples [01:43, 82973.72 examples/s]
Generating train split: 8808000 examples [01:43, 82842.59 examples/s]
Generating train split: 8817000 examples [01:44, 81056.48 examples/s]
Generating train split: 8827000 examples [01:44, 80572.35 examples/s]
Generating train split: 8837000 examples [01:44, 82023.67 examples/s]
Generating train split: 8846000 examples [01:44, 81330.66 examples/s]
Generating train split: 8859000 examples [01:44, 80598.39 examples/s]
Generating train split: 8869000 examples [01:44, 81741.78 examples/s]
Generating train split: 8882000 examples [01:44, 81323.22 examples/s]
Generating train split: 8892000 examples [01:45, 81173.60 examples/s]
Generating train split: 8902000 examples [01:45, 82348.13 examples/s]
Generating train split: 8912000 examples [01:45, 82942.78 examples/s]
Generating train split: 8922000 examples [01:45, 84651.95 examples/s]
Generating train split: 8932000 examples [01:45, 84402.74 examples/s]
Generating train split: 8942000 examples [01:45, 85472.30 examples/s]
Generating train split: 8952000 examples [01:45, 84811.25 examples/s]
Generating train split: 8962000 examples [01:45, 85152.77 examples/s]
Generating train split: 8972000 examples [01:45, 84981.19 examples/s]
Generating train split: 8982000 examples [01:46, 85855.02 examples/s]
Generating train split: 8992000 examples [01:46, 86570.94 examples/s]
Generating train split: 9002000 examples [01:46, 86307.01 examples/s]
Generating train split: 9011000 examples [01:46, 84966.41 examples/s]
Generating train split: 9020000 examples [01:46, 84269.04 examples/s]
Generating train split: 9030000 examples [01:46, 84505.58 examples/s]
Generating train split: 9039000 examples [01:46, 82872.11 examples/s]
Generating train split: 9049000 examples [01:46, 83438.47 examples/s]
Generating train split: 9059000 examples [01:46, 83495.16 examples/s]
Generating train split: 9069000 examples [01:47, 83959.90 examples/s]
Generating train split: 9079000 examples [01:47, 84563.25 examples/s]
Generating train split: 9089000 examples [01:47, 84724.83 examples/s]
Generating train split: 9099000 examples [01:47, 84586.37 examples/s]
Generating train split: 9109000 examples [01:47, 85318.59 examples/s]
Generating train split: 9119000 examples [01:47, 84701.39 examples/s]
Generating train split: 9129000 examples [01:47, 85648.90 examples/s]
Generating train split: 9139000 examples [01:47, 85457.37 examples/s]
Generating train split: 9148000 examples [01:48, 85345.16 examples/s]
Generating train split: 9158000 examples [01:48, 85606.48 examples/s]
Generating train split: 9168000 examples [01:48, 85682.22 examples/s]
Generating train split: 9178000 examples [01:48, 83754.03 examples/s]
Generating train split: 9187000 examples [01:48, 82795.04 examples/s]
Generating train split: 9196000 examples [01:48, 81563.77 examples/s]
Generating train split: 9206000 examples [01:48, 83083.20 examples/s]
Generating train split: 9219000 examples [01:48, 81983.82 examples/s]
Generating train split: 9229000 examples [01:49, 82273.39 examples/s]
Generating train split: 9239000 examples [01:49, 83823.60 examples/s]
Generating train split: 9249000 examples [01:49, 83793.09 examples/s]
Generating train split: 9259000 examples [01:49, 84437.31 examples/s]
Generating train split: 9268000 examples [01:49, 83536.71 examples/s]
Generating train split: 9278000 examples [01:49, 84991.96 examples/s]
Generating train split: 9288000 examples [01:49, 85222.54 examples/s]
Generating train split: 9298000 examples [01:49, 84037.34 examples/s]
Generating train split: 9307000 examples [01:49, 83504.69 examples/s]
Generating train split: 9317000 examples [01:50, 84334.16 examples/s]
Generating train split: 9327000 examples [01:50, 84521.68 examples/s]
Generating train split: 9336000 examples [01:50, 82992.88 examples/s]
Generating train split: 9346000 examples [01:50, 81956.01 examples/s]
Generating train split: 9356000 examples [01:50, 81890.48 examples/s]
Generating train split: 9366000 examples [01:50, 81877.42 examples/s]
Generating train split: 9376000 examples [01:50, 81221.27 examples/s]
Generating train split: 9386000 examples [01:50, 82266.66 examples/s]
Generating train split: 9396000 examples [01:51, 82483.00 examples/s]
Generating train split: 9405000 examples [01:51, 80248.18 examples/s]
Generating train split: 9414000 examples [01:51, 80129.52 examples/s]
Generating train split: 9424000 examples [01:51, 81783.57 examples/s]
Generating train split: 9433000 examples [01:51, 81845.64 examples/s]
Generating train split: 9446000 examples [01:51, 81337.15 examples/s]
Generating train split: 9455000 examples [01:51, 81107.20 examples/s]
Generating train split: 9465000 examples [01:51, 81440.80 examples/s]
Generating train split: 9478000 examples [01:52, 79515.68 examples/s]
Generating train split: 9488000 examples [01:52, 81142.27 examples/s]
Generating train split: 9498000 examples [01:52, 82047.68 examples/s]
Generating train split: 9508000 examples [01:52, 83192.83 examples/s]
Generating train split: 9520000 examples [01:52, 80518.70 examples/s]
Generating train split: 9530000 examples [01:52, 81566.09 examples/s]
Generating train split: 9539000 examples [01:52, 81395.35 examples/s]
Generating train split: 9548000 examples [01:52, 81683.25 examples/s]
Generating train split: 9557000 examples [01:53, 80513.27 examples/s]
Generating train split: 9567000 examples [01:53, 82547.38 examples/s]
Generating train split: 9577000 examples [01:53, 83813.51 examples/s]
Generating train split: 9586000 examples [01:53, 82605.47 examples/s]
Generating train split: 9596000 examples [01:53, 82792.42 examples/s]
Generating train split: 9606000 examples [01:53, 84031.03 examples/s]
Generating train split: 9616000 examples [01:53, 84420.20 examples/s]
Generating train split: 9625000 examples [01:53, 83685.63 examples/s]
Generating train split: 9635000 examples [01:53, 83479.63 examples/s]
Generating train split: 9644000 examples [01:54, 81758.13 examples/s]
Generating train split: 9654000 examples [01:54, 83245.66 examples/s]
Generating train split: 9664000 examples [01:54, 83993.04 examples/s]
Generating train split: 9674000 examples [01:54, 83748.91 examples/s]
Generating train split: 9684000 examples [01:54, 83928.95 examples/s]
Generating train split: 9693000 examples [01:54, 82882.77 examples/s]
Generating train split: 9703000 examples [01:54, 83153.82 examples/s]
Generating train split: 9713000 examples [01:54, 83829.87 examples/s]
Generating train split: 9723000 examples [01:54, 84482.98 examples/s]
Generating train split: 9732000 examples [01:55, 82770.32 examples/s]
Generating train split: 9741000 examples [01:55, 80788.27 examples/s]
Generating train split: 9751000 examples [01:55, 82538.93 examples/s]
Generating train split: 9761000 examples [01:55, 83999.20 examples/s]
Generating train split: 9771000 examples [01:55, 84242.68 examples/s]
Generating train split: 9780000 examples [01:55, 83793.71 examples/s]
Generating train split: 9790000 examples [01:55, 84742.45 examples/s]
Generating train split: 9800000 examples [01:55, 84391.86 examples/s]
Generating train split: 9809000 examples [01:56, 83600.97 examples/s]
Generating train split: 9822000 examples [01:56, 80700.07 examples/s]
Generating train split: 9832000 examples [01:56, 82039.34 examples/s]
Generating train split: 9842000 examples [01:56, 83360.35 examples/s]
Generating train split: 9855000 examples [01:56, 81037.96 examples/s]
Generating train split: 9865000 examples [01:56, 82711.45 examples/s]
Generating train split: 9875000 examples [01:56, 83231.48 examples/s]
Generating train split: 9884000 examples [01:56, 82866.50 examples/s]
Generating train split: 9894000 examples [01:57, 83636.78 examples/s]
Generating train split: 9904000 examples [01:57, 83425.65 examples/s]
Generating train split: 9914000 examples [01:57, 82506.78 examples/s]
Generating train split: 9924000 examples [01:57, 83886.21 examples/s]
Generating train split: 9934000 examples [01:57, 84524.65 examples/s]
Generating train split: 9943000 examples [01:57, 83542.22 examples/s]
Generating train split: 9952000 examples [01:57, 83044.56 examples/s]
Generating train split: 9962000 examples [01:57, 83355.29 examples/s]
Generating train split: 9972000 examples [01:57, 81536.05 examples/s]
Generating train split: 9982000 examples [01:58, 82965.15 examples/s]
Generating train split: 9991000 examples [01:58, 82346.32 examples/s]
Generating train split: 10001000 examples [01:58, 83551.74 examples/s]
Generating train split: 10011000 examples [01:58, 85099.89 examples/s]
Generating train split: 10021000 examples [01:58, 84327.86 examples/s]
Generating train split: 10030000 examples [01:58, 82900.48 examples/s]
Generating train split: 10039000 examples [01:58, 82423.70 examples/s]
Generating train split: 10049000 examples [01:58, 83157.58 examples/s]
Generating train split: 10058000 examples [01:59, 82525.48 examples/s]
Generating train split: 10068000 examples [01:59, 83164.43 examples/s]
Generating train split: 10078000 examples [01:59, 83628.71 examples/s]
Generating train split: 10088000 examples [01:59, 84878.61 examples/s]
Generating train split: 10098000 examples [01:59, 85684.74 examples/s]
Generating train split: 10107000 examples [01:59, 85460.86 examples/s]
Generating train split: 10116000 examples [01:59, 83057.35 examples/s]
Generating train split: 10126000 examples [01:59, 82869.56 examples/s]
Generating train split: 10135000 examples [01:59, 82290.44 examples/s]
Generating train split: 10144000 examples [02:00, 81814.80 examples/s]
Generating train split: 10154000 examples [02:00, 82970.87 examples/s]
Generating train split: 10164000 examples [02:00, 83432.22 examples/s]
Generating train split: 10174000 examples [02:00, 84332.17 examples/s]
Generating train split: 10184000 examples [02:00, 84826.04 examples/s]
Generating train split: 10193000 examples [02:00, 83711.19 examples/s]
Generating train split: 10203000 examples [02:00, 84372.82 examples/s]
Generating train split: 10213000 examples [02:00, 85333.48 examples/s]
Generating train split: 10223000 examples [02:00, 85575.03 examples/s]
Generating train split: 10233000 examples [02:01, 85589.89 examples/s]
Generating train split: 10243000 examples [02:01, 85997.17 examples/s]
Generating train split: 10252000 examples [02:01, 84626.41 examples/s]
Generating train split: 10262000 examples [02:01, 84410.62 examples/s]
Generating train split: 10272000 examples [02:01, 85282.53 examples/s]
Generating train split: 10282000 examples [02:01, 85504.87 examples/s]
Generating train split: 10292000 examples [02:01, 84601.46 examples/s]
Generating train split: 10302000 examples [02:01, 84976.41 examples/s]
Generating train split: 10312000 examples [02:02, 85266.66 examples/s]
Generating train split: 10321000 examples [02:02, 83367.64 examples/s]
Generating train split: 10331000 examples [02:02, 82937.09 examples/s]
Generating train split: 10340000 examples [02:02, 82260.95 examples/s]
Generating train split: 10350000 examples [02:02, 81922.94 examples/s]
Generating train split: 10360000 examples [02:02, 82706.13 examples/s]
Generating train split: 10373000 examples [02:02, 81496.83 examples/s]
Generating train split: 10382000 examples [02:02, 80829.21 examples/s]
Generating train split: 10392000 examples [02:03, 81348.19 examples/s]
Generating train split: 10402000 examples [02:03, 82966.80 examples/s]
Generating train split: 10412000 examples [02:03, 82861.68 examples/s]
Generating train split: 10421000 examples [02:03, 82908.80 examples/s]
Generating train split: 10431000 examples [02:03, 83923.39 examples/s]
Generating train split: 10440000 examples [02:03, 83531.37 examples/s]
Generating train split: 10453000 examples [02:03, 80100.48 examples/s]
Generating train split: 10463000 examples [02:03, 81042.20 examples/s]
Generating train split: 10472000 examples [02:03, 81477.90 examples/s]
Generating train split: 10482000 examples [02:04, 82294.66 examples/s]
Generating train split: 10495000 examples [02:04, 81704.61 examples/s]
Generating train split: 10505000 examples [02:04, 83242.79 examples/s]
Generating train split: 10515000 examples [02:04, 84012.11 examples/s]
Generating train split: 10524000 examples [02:04, 82374.86 examples/s]
Generating train split: 10534000 examples [02:04, 83016.61 examples/s]
Generating train split: 10543000 examples [02:04, 83089.57 examples/s]
Generating train split: 10553000 examples [02:04, 83172.66 examples/s]
Generating train split: 10563000 examples [02:05, 83827.40 examples/s]
Generating train split: 10573000 examples [02:05, 84862.31 examples/s]
Generating train split: 10583000 examples [02:05, 84126.42 examples/s]
Generating train split: 10592000 examples [02:05, 83614.57 examples/s]
Generating train split: 10602000 examples [02:05, 84787.01 examples/s]
Generating train split: 10612000 examples [02:05, 85624.03 examples/s]
Generating train split: 10621000 examples [02:05, 85070.63 examples/s]
Generating train split: 10631000 examples [02:05, 85828.42 examples/s]
Generating train split: 10641000 examples [02:05, 85991.29 examples/s]
Generating train split: 10651000 examples [02:06, 85668.23 examples/s]
Generating train split: 10661000 examples [02:06, 85979.76 examples/s]
Generating train split: 10671000 examples [02:06, 85461.29 examples/s]
Generating train split: 10681000 examples [02:06, 85985.22 examples/s]
Generating train split: 10690000 examples [02:06, 84616.84 examples/s]
Generating train split: 10700000 examples [02:06, 84443.72 examples/s]
Generating train split: 10710000 examples [02:06, 85284.68 examples/s]
Generating train split: 10720000 examples [02:06, 85780.11 examples/s]
Generating train split: 10730000 examples [02:07, 85181.33 examples/s]
Generating train split: 10740000 examples [02:07, 85563.31 examples/s]
Generating train split: 10750000 examples [02:07, 85460.12 examples/s]
Generating train split: 10760000 examples [02:07, 85192.80 examples/s]
Generating train split: 10770000 examples [02:07, 83686.48 examples/s]
Generating train split: 10779000 examples [02:07, 83512.69 examples/s]
Generating train split: 10789000 examples [02:07, 83773.95 examples/s]
Generating train split: 10799000 examples [02:07, 83840.91 examples/s]
Generating train split: 10809000 examples [02:07, 83329.52 examples/s]
Generating train split: 10819000 examples [02:08, 84314.53 examples/s]
Generating train split: 10829000 examples [02:08, 85031.52 examples/s]
Generating train split: 10838000 examples [02:08, 84048.86 examples/s]
Generating train split: 10848000 examples [02:08, 84446.47 examples/s]
Generating train split: 10857000 examples [02:08, 81284.84 examples/s]
Generating train split: 10867000 examples [02:08, 81742.07 examples/s]
Generating train split: 10877000 examples [02:08, 82639.34 examples/s]
Generating train split: 10887000 examples [02:08, 82588.40 examples/s]
Generating train split: 10896000 examples [02:09, 82108.38 examples/s]
Generating train split: 10906000 examples [02:09, 82004.95 examples/s]
Generating train split: 10916000 examples [02:09, 81763.05 examples/s]
Generating train split: 10926000 examples [02:09, 82539.92 examples/s]
Generating train split: 10935000 examples [02:09, 81549.22 examples/s]
Generating train split: 10945000 examples [02:09, 83273.28 examples/s]
Generating train split: 10955000 examples [02:09, 84291.33 examples/s]
Generating train split: 10964000 examples [02:09, 84164.76 examples/s]
Generating train split: 10973000 examples [02:09, 82048.25 examples/s]
Generating train split: 10983000 examples [02:10, 80972.75 examples/s]
Generating train split: 10993000 examples [02:10, 82019.65 examples/s]
Generating train split: 11003000 examples [02:10, 83357.16 examples/s]
Generating train split: 11016000 examples [02:10, 81905.87 examples/s]
Generating train split: 11026000 examples [02:10, 82749.11 examples/s]
Generating train split: 11035000 examples [02:10, 81311.80 examples/s]
Generating train split: 11045000 examples [02:10, 82241.62 examples/s]
Generating train split: 11055000 examples [02:10, 82699.27 examples/s]
Generating train split: 11065000 examples [02:11, 83329.34 examples/s]
Generating train split: 11075000 examples [02:11, 84757.21 examples/s]
Generating train split: 11085000 examples [02:11, 85290.55 examples/s]
Generating train split: 11095000 examples [02:11, 85967.50 examples/s]
Generating train split: 11105000 examples [02:11, 85789.15 examples/s]
Generating train split: 11115000 examples [02:11, 85302.12 examples/s]
Generating train split: 11124000 examples [02:11, 82927.94 examples/s]
Generating train split: 11136000 examples [02:11, 78679.76 examples/s]
Generating train split: 11146000 examples [02:12, 81157.62 examples/s]
Generating train split: 11156000 examples [02:12, 82169.49 examples/s]
Generating train split: 11165000 examples [02:12, 82092.02 examples/s]
Generating train split: 11175000 examples [02:12, 83213.02 examples/s]
Generating train split: 11185000 examples [02:12, 84121.57 examples/s]
Generating train split: 11195000 examples [02:12, 83790.19 examples/s]
Generating train split: 11204000 examples [02:12, 83420.82 examples/s]
Generating train split: 11214000 examples [02:12, 83366.59 examples/s]
Generating train split: 11223000 examples [02:12, 81847.91 examples/s]
Generating train split: 11233000 examples [02:13, 83389.75 examples/s]
Generating train split: 11243000 examples [02:13, 83511.63 examples/s]
Generating train split: 11253000 examples [02:13, 84318.65 examples/s]
Generating train split: 11263000 examples [02:13, 84509.72 examples/s]
Generating train split: 11273000 examples [02:13, 85165.52 examples/s]
Generating train split: 11286000 examples [02:13, 83386.71 examples/s]
Generating train split: 11295000 examples [02:13, 81384.92 examples/s]
Generating train split: 11305000 examples [02:13, 82354.71 examples/s]
Generating train split: 11315000 examples [02:14, 82438.56 examples/s]
Generating train split: 11325000 examples [02:14, 82689.75 examples/s]
Generating train split: 11334000 examples [02:14, 82990.41 examples/s]
Generating train split: 11344000 examples [02:14, 83770.95 examples/s]
Generating train split: 11353000 examples [02:14, 82020.49 examples/s]
Generating train split: 11363000 examples [02:14, 82322.70 examples/s]
Generating train split: 11376000 examples [02:14, 80975.53 examples/s]
Generating train split: 11386000 examples [02:14, 81824.99 examples/s]
Generating train split: 11396000 examples [02:15, 83589.45 examples/s]
Generating train split: 11406000 examples [02:15, 84815.08 examples/s]
Generating train split: 11416000 examples [02:15, 84961.45 examples/s]
Generating train split: 11426000 examples [02:15, 85484.90 examples/s]
Generating train split: 11436000 examples [02:15, 85873.85 examples/s]
Generating train split: 11446000 examples [02:15, 86976.20 examples/s]
Generating train split: 11456000 examples [02:15, 85930.86 examples/s]
Generating train split: 11466000 examples [02:15, 86136.07 examples/s]
Generating train split: 11475000 examples [02:15, 85006.19 examples/s]
Generating train split: 11488000 examples [02:16, 80845.26 examples/s]
Generating train split: 11498000 examples [02:16, 82249.17 examples/s]
Generating train split: 11508000 examples [02:16, 83249.84 examples/s]
Generating train split: 11518000 examples [02:16, 84690.33 examples/s]
Generating train split: 11528000 examples [02:16, 84952.45 examples/s]
Generating train split: 11538000 examples [02:16, 85087.28 examples/s]
Generating train split: 11548000 examples [02:16, 84735.32 examples/s]
Generating train split: 11557000 examples [02:16, 83126.97 examples/s]
Generating train split: 11567000 examples [02:17, 83235.45 examples/s]
Generating train split: 11577000 examples [02:17, 84908.13 examples/s]
Generating train split: 11587000 examples [02:17, 85469.41 examples/s]
Generating train split: 11597000 examples [02:17, 85606.82 examples/s]
Generating train split: 11607000 examples [02:17, 85155.25 examples/s]
Generating train split: 11617000 examples [02:17, 85549.20 examples/s]
Generating train split: 11626000 examples [02:17, 83881.14 examples/s]
Generating train split: 11635000 examples [02:17, 82716.44 examples/s]
Generating train split: 11645000 examples [02:17, 83848.65 examples/s]
Generating train split: 11655000 examples [02:18, 83639.43 examples/s]
Generating train split: 11665000 examples [02:18, 84110.80 examples/s]
Generating train split: 11675000 examples [02:18, 84518.29 examples/s]
Generating train split: 11684000 examples [02:18, 83774.02 examples/s]
Generating train split: 11694000 examples [02:18, 85327.93 examples/s]
Generating train split: 11704000 examples [02:18, 84867.42 examples/s]
Generating train split: 11714000 examples [02:18, 85219.39 examples/s]
Generating train split: 11723000 examples [02:18, 82994.07 examples/s]
Generating train split: 11733000 examples [02:19, 83306.40 examples/s]
Generating train split: 11742000 examples [02:19, 82770.39 examples/s]
Generating train split: 11754000 examples [02:19, 80731.79 examples/s]
Generating train split: 11764000 examples [02:19, 82393.61 examples/s]
Generating train split: 11774000 examples [02:19, 83313.41 examples/s]
Generating train split: 11784000 examples [02:19, 83578.93 examples/s]
Generating train split: 11793000 examples [02:19, 82742.65 examples/s]
Generating train split: 11803000 examples [02:19, 81475.28 examples/s]
Generating train split: 11813000 examples [02:20, 82853.61 examples/s]
Generating train split: 11823000 examples [02:20, 83266.74 examples/s]
Generating train split: 11832000 examples [02:20, 82414.34 examples/s]
Generating train split: 11842000 examples [02:20, 83892.03 examples/s]
Generating train split: 11852000 examples [02:20, 84053.97 examples/s]
Generating train split: 11861000 examples [02:20, 83378.28 examples/s]
Generating train split: 11870000 examples [02:20, 83392.38 examples/s]
Generating train split: 11880000 examples [02:20, 84191.11 examples/s]
Generating train split: 11890000 examples [02:20, 84528.80 examples/s]
Generating train split: 11900000 examples [02:21, 84595.54 examples/s]
Generating train split: 11910000 examples [02:21, 85074.47 examples/s]
Generating train split: 11920000 examples [02:21, 85108.08 examples/s]
Generating train split: 11930000 examples [02:21, 85280.50 examples/s]
Generating train split: 11940000 examples [02:21, 85384.68 examples/s]
Generating train split: 11950000 examples [02:21, 86062.80 examples/s]
Generating train split: 11960000 examples [02:21, 85828.92 examples/s]
Generating train split: 11970000 examples [02:21, 85423.38 examples/s]
Generating train split: 11980000 examples [02:21, 86448.92 examples/s]
Generating train split: 11990000 examples [02:22, 86324.88 examples/s]
Generating train split: 12000000 examples [02:22, 86046.66 examples/s]
Generating train split: 12010000 examples [02:22, 84570.13 examples/s]
Generating train split: 12020000 examples [02:22, 84934.27 examples/s]
Generating train split: 12030000 examples [02:22, 85295.63 examples/s]
Generating train split: 12040000 examples [02:22, 85291.95 examples/s]
Generating train split: 12053000 examples [02:22, 82947.18 examples/s]
Generating train split: 12063000 examples [02:22, 82989.87 examples/s]
Generating train split: 12073000 examples [02:23, 83800.95 examples/s]
Generating train split: 12083000 examples [02:23, 83726.85 examples/s]
Generating train split: 12093000 examples [02:23, 83251.17 examples/s]
Generating train split: 12103000 examples [02:23, 84340.44 examples/s]
Generating train split: 12113000 examples [02:23, 85095.76 examples/s]
Generating train split: 12123000 examples [02:23, 85764.57 examples/s]
Generating train split: 12133000 examples [02:23, 85923.91 examples/s]
Generating train split: 12143000 examples [02:23, 85630.98 examples/s]
Generating train split: 12153000 examples [02:24, 84528.68 examples/s]
Generating train split: 12162000 examples [02:24, 83203.99 examples/s]
Generating train split: 12172000 examples [02:24, 83516.43 examples/s]
Generating train split: 12182000 examples [02:24, 84295.00 examples/s]
Generating train split: 12192000 examples [02:24, 84022.01 examples/s]
Generating train split: 12202000 examples [02:24, 84750.61 examples/s]
Generating train split: 12212000 examples [02:24, 84472.50 examples/s]
Generating train split: 12221000 examples [02:24, 83442.83 examples/s]
Generating train split: 12231000 examples [02:24, 83025.99 examples/s]
Generating train split: 12241000 examples [02:25, 84562.08 examples/s]
Generating train split: 12251000 examples [02:25, 84445.11 examples/s]
Generating train split: 12261000 examples [02:25, 84604.28 examples/s]
Generating train split: 12271000 examples [02:25, 85402.85 examples/s]
Generating train split: 12281000 examples [02:25, 84351.87 examples/s]
Generating train split: 12291000 examples [02:25, 84767.99 examples/s]
Generating train split: 12301000 examples [02:25, 85252.89 examples/s]
Generating train split: 12311000 examples [02:25, 84611.43 examples/s]
Generating train split: 12321000 examples [02:26, 85079.47 examples/s]
Generating train split: 12330000 examples [02:26, 84447.90 examples/s]
Generating train split: 12340000 examples [02:26, 84584.78 examples/s]
Generating train split: 12350000 examples [02:26, 84750.05 examples/s]
Generating train split: 12360000 examples [02:26, 85062.29 examples/s]
Generating train split: 12370000 examples [02:26, 85182.85 examples/s]
Generating train split: 12380000 examples [02:26, 85885.22 examples/s]
Generating train split: 12390000 examples [02:26, 85834.98 examples/s]
Generating train split: 12399000 examples [02:26, 83175.92 examples/s]
Generating train split: 12408000 examples [02:27, 81548.19 examples/s]
Generating train split: 12417000 examples [02:27, 82013.81 examples/s]
Generating train split: 12427000 examples [02:27, 82686.58 examples/s]
Generating train split: 12437000 examples [02:27, 83180.63 examples/s]
Generating train split: 12446000 examples [02:27, 82789.17 examples/s]
Generating train split: 12455000 examples [02:27, 82650.43 examples/s]
Generating train split: 12465000 examples [02:27, 83694.38 examples/s]
Generating train split: 12475000 examples [02:27, 83901.14 examples/s]
Generating train split: 12484000 examples [02:27, 82640.88 examples/s]
Generating train split: 12493000 examples [02:28, 82090.65 examples/s]
Generating train split: 12503000 examples [02:28, 83128.87 examples/s]
Generating train split: 12516000 examples [02:28, 80433.84 examples/s]
Generating train split: 12525000 examples [02:28, 80653.67 examples/s]
Generating train split: 12534000 examples [02:28, 81123.29 examples/s]
Generating train split: 12544000 examples [02:28, 82168.35 examples/s]
Generating train split: 12554000 examples [02:28, 82330.85 examples/s]
Generating train split: 12563000 examples [02:28, 82126.99 examples/s]
Generating train split: 12572000 examples [02:29, 81389.96 examples/s]
Generating train split: 12581000 examples [02:29, 81317.10 examples/s]
Generating train split: 12591000 examples [02:29, 82471.69 examples/s]
Generating train split: 12601000 examples [02:29, 83702.08 examples/s]
Generating train split: 12611000 examples [02:29, 82821.94 examples/s]
Generating train split: 12621000 examples [02:29, 83985.59 examples/s]
Generating train split: 12630000 examples [02:29, 83443.94 examples/s]
Generating train split: 12640000 examples [02:29, 84858.16 examples/s]
Generating train split: 12650000 examples [02:29, 84843.72 examples/s]
Generating train split: 12660000 examples [02:30, 84848.08 examples/s]
Generating train split: 12670000 examples [02:30, 84496.94 examples/s]
Generating train split: 12680000 examples [02:30, 84553.15 examples/s]
Generating train split: 12690000 examples [02:30, 83339.05 examples/s]
Generating train split: 12700000 examples [02:30, 84399.59 examples/s]
Generating train split: 12710000 examples [02:30, 84796.23 examples/s]
Generating train split: 12720000 examples [02:30, 84224.23 examples/s]
Generating train split: 12729000 examples [02:30, 83706.05 examples/s]
Generating train split: 12739000 examples [02:31, 84455.56 examples/s]
Generating train split: 12748000 examples [02:31, 83128.10 examples/s]
Generating train split: 12757000 examples [02:31, 82476.81 examples/s]
Generating train split: 12766000 examples [02:31, 82235.51 examples/s]
Generating train split: 12776000 examples [02:31, 82826.31 examples/s]
Generating train split: 12786000 examples [02:31, 83562.89 examples/s]
Generating train split: 12795000 examples [02:31, 81764.97 examples/s]
Generating train split: 12805000 examples [02:31, 82966.09 examples/s]
Generating train split: 12815000 examples [02:31, 82988.02 examples/s]
Generating train split: 12824000 examples [02:32, 80955.24 examples/s]
Generating train split: 12834000 examples [02:32, 82421.36 examples/s]
Generating train split: 12843000 examples [02:32, 82563.72 examples/s]
Generating train split: 12852000 examples [02:32, 81012.96 examples/s]
Generating train split: 12862000 examples [02:32, 82645.68 examples/s]
Generating train split: 12872000 examples [02:32, 83354.84 examples/s]
Generating train split: 12882000 examples [02:32, 83052.63 examples/s]
Generating train split: 12892000 examples [02:32, 83421.46 examples/s]
Generating train split: 12902000 examples [02:33, 83344.18 examples/s]
Generating train split: 12912000 examples [02:33, 85037.08 examples/s]
Generating train split: 12922000 examples [02:33, 85377.63 examples/s]
Generating train split: 12932000 examples [02:33, 85686.78 examples/s]
Generating train split: 12942000 examples [02:33, 86053.94 examples/s]
Generating train split: 12952000 examples [02:33, 85991.79 examples/s]
Generating train split: 12962000 examples [02:33, 85573.39 examples/s]
Generating train split: 12972000 examples [02:33, 85318.53 examples/s]
Generating train split: 12981000 examples [02:33, 83154.55 examples/s]
Generating train split: 12991000 examples [02:34, 83606.24 examples/s]
Generating train split: 13000000 examples [02:34, 82503.76 examples/s]
Generating train split: 13010000 examples [02:34, 83374.41 examples/s]
Generating train split: 13020000 examples [02:34, 84083.75 examples/s]
Generating train split: 13029000 examples [02:34, 82949.26 examples/s]
Generating train split: 13042000 examples [02:34, 82298.38 examples/s]
Generating train split: 13051000 examples [02:34, 82302.23 examples/s]
Generating train split: 13061000 examples [02:34, 83076.11 examples/s]
Generating train split: 13070000 examples [02:35, 82403.46 examples/s]
Generating train split: 13080000 examples [02:35, 82634.23 examples/s]
Generating train split: 13090000 examples [02:35, 83466.78 examples/s]
Generating train split: 13100000 examples [02:35, 83580.55 examples/s]
Generating train split: 13109000 examples [02:35, 83143.50 examples/s]
Generating train split: 13119000 examples [02:35, 83651.90 examples/s]
Generating train split: 13129000 examples [02:35, 83795.91 examples/s]
Generating train split: 13139000 examples [02:35, 84020.81 examples/s]
Generating train split: 13149000 examples [02:35, 84494.87 examples/s]
Generating train split: 13159000 examples [02:36, 83987.80 examples/s]
Generating train split: 13168000 examples [02:36, 83332.46 examples/s]
Generating train split: 13177000 examples [02:36, 82945.10 examples/s]
Generating train split: 13186000 examples [02:36, 83400.17 examples/s]
Generating train split: 13196000 examples [02:36, 83810.08 examples/s]
Generating train split: 13206000 examples [02:36, 84941.41 examples/s]
Generating train split: 13215000 examples [02:36, 84440.35 examples/s]
Generating train split: 13225000 examples [02:36, 85409.26 examples/s]
Generating train split: 13235000 examples [02:36, 85655.82 examples/s]
Generating train split: 13245000 examples [02:37, 85809.75 examples/s]
Generating train split: 13255000 examples [02:37, 85612.98 examples/s]
Generating train split: 13265000 examples [02:37, 84917.42 examples/s]
Generating train split: 13275000 examples [02:37, 83567.08 examples/s]
Generating train split: 13284000 examples [02:37, 82294.37 examples/s]
Generating train split: 13294000 examples [02:37, 83074.75 examples/s]
Generating train split: 13304000 examples [02:37, 84023.16 examples/s]
Generating train split: 13314000 examples [02:37, 84746.68 examples/s]
Generating train split: 13324000 examples [02:38, 84652.80 examples/s]
Generating train split: 13333000 examples [02:38, 82591.14 examples/s]
Generating train split: 13343000 examples [02:38, 82636.33 examples/s]
Generating train split: 13352000 examples [02:38, 81792.02 examples/s]
Generating train split: 13362000 examples [02:38, 81782.67 examples/s]
Generating train split: 13372000 examples [02:38, 81950.32 examples/s]
Generating train split: 13381000 examples [02:38, 81159.52 examples/s]
Generating train split: 13391000 examples [02:38, 81858.93 examples/s]
Generating train split: 13400000 examples [02:38, 82246.02 examples/s]
Generating train split: 13410000 examples [02:39, 83881.81 examples/s]
Generating train split: 13420000 examples [02:39, 84663.97 examples/s]
Generating train split: 13433000 examples [02:39, 81848.53 examples/s]
Generating train split: 13443000 examples [02:39, 82920.77 examples/s]
Generating train split: 13452000 examples [02:39, 81316.60 examples/s]
Generating train split: 13462000 examples [02:39, 81772.52 examples/s]
Generating train split: 13472000 examples [02:39, 82994.01 examples/s]
Generating train split: 13482000 examples [02:39, 82617.51 examples/s]
Generating train split: 13492000 examples [02:40, 82999.91 examples/s]
Generating train split: 13501000 examples [02:40, 83016.28 examples/s]
Generating train split: 13511000 examples [02:40, 83528.23 examples/s]
Generating train split: 13521000 examples [02:40, 83448.17 examples/s]
Generating train split: 13531000 examples [02:40, 84678.61 examples/s]
Generating train split: 13540000 examples [02:40, 84020.41 examples/s]
Generating train split: 13553000 examples [02:40, 80603.92 examples/s]
Generating train split: 13563000 examples [02:40, 82457.97 examples/s]
Generating train split: 13573000 examples [02:41, 83877.03 examples/s]
Generating train split: 13582000 examples [02:41, 82851.47 examples/s]
Generating train split: 13595000 examples [02:41, 80409.59 examples/s]
Generating train split: 13605000 examples [02:41, 80872.99 examples/s]
Generating train split: 13614000 examples [02:41, 80716.33 examples/s]
Generating train split: 13624000 examples [02:41, 82056.50 examples/s]
Generating train split: 13633000 examples [02:41, 81568.49 examples/s]
Generating train split: 13643000 examples [02:41, 82565.40 examples/s]
Generating train split: 13653000 examples [02:42, 83063.66 examples/s]
Generating train split: 13663000 examples [02:42, 84287.45 examples/s]
Generating train split: 13673000 examples [02:42, 84011.52 examples/s]
Generating train split: 13683000 examples [02:42, 84736.69 examples/s]
Generating train split: 13692000 examples [02:42, 83914.36 examples/s]
Generating train split: 13702000 examples [02:42, 82527.34 examples/s]
Generating train split: 13712000 examples [02:42, 83273.94 examples/s]
Generating train split: 13722000 examples [02:42, 84059.73 examples/s]
Generating train split: 13731000 examples [02:42, 83239.89 examples/s]
Generating train split: 13740000 examples [02:43, 83006.13 examples/s]
Generating train split: 13750000 examples [02:43, 84603.68 examples/s]
Generating train split: 13760000 examples [02:43, 84373.42 examples/s]
Generating train split: 13770000 examples [02:43, 84796.44 examples/s]
Generating train split: 13780000 examples [02:43, 85876.29 examples/s]
Generating train split: 13789000 examples [02:43, 85067.74 examples/s]
Generating train split: 13798000 examples [02:43, 82700.63 examples/s]
Generating train split: 13808000 examples [02:43, 83480.52 examples/s]
Generating train split: 13818000 examples [02:43, 83381.23 examples/s]
Generating train split: 13828000 examples [02:44, 83835.52 examples/s]
Generating train split: 13838000 examples [02:44, 83889.67 examples/s]
Generating train split: 13848000 examples [02:44, 85206.32 examples/s]
Generating train split: 13858000 examples [02:44, 84824.84 examples/s]
Generating train split: 13868000 examples [02:44, 85222.04 examples/s]
Generating train split: 13877000 examples [02:44, 81765.23 examples/s]
Generating train split: 13887000 examples [02:44, 82686.87 examples/s]
Generating train split: 13897000 examples [02:44, 83932.18 examples/s]
Generating train split: 13907000 examples [02:45, 84352.09 examples/s]
Generating train split: 13917000 examples [02:45, 84329.97 examples/s]
Generating train split: 13927000 examples [02:45, 85242.32 examples/s]
Generating train split: 13937000 examples [02:45, 85803.07 examples/s]
Generating train split: 13947000 examples [02:45, 85964.90 examples/s]
Generating train split: 13957000 examples [02:45, 86215.27 examples/s]
Generating train split: 13966000 examples [02:45, 84176.87 examples/s]
Generating train split: 13976000 examples [02:45, 83880.11 examples/s]
Generating train split: 13985000 examples [02:45, 83352.86 examples/s]
Generating train split: 13994000 examples [02:46, 82089.80 examples/s]
Generating train split: 14004000 examples [02:46, 83378.45 examples/s]
Generating train split: 14013000 examples [02:46, 82081.20 examples/s]
Generating train split: 14023000 examples [02:46, 82322.12 examples/s]
Generating train split: 14032000 examples [02:46, 81392.24 examples/s]
Generating train split: 14042000 examples [02:46, 83412.59 examples/s]
Generating train split: 14052000 examples [02:46, 84933.88 examples/s]
Generating train split: 14061000 examples [02:46, 83566.63 examples/s]
Generating train split: 14071000 examples [02:46, 84243.84 examples/s]
Generating train split: 14081000 examples [02:47, 82075.94 examples/s]
Generating train split: 14091000 examples [02:47, 82956.93 examples/s]
Generating train split: 14101000 examples [02:47, 82653.56 examples/s]
Generating train split: 14110000 examples [02:47, 81702.90 examples/s]
Generating train split: 14119000 examples [02:47, 80378.19 examples/s]
Generating train split: 14129000 examples [02:47, 80875.19 examples/s]
Generating train split: 14139000 examples [02:47, 81906.25 examples/s]
Generating train split: 14148000 examples [02:47, 80839.08 examples/s]
Generating train split: 14161000 examples [02:48, 81032.43 examples/s]
Generating train split: 14171000 examples [02:48, 82732.05 examples/s]
Generating train split: 14181000 examples [02:48, 83871.08 examples/s]
Generating train split: 14190000 examples [02:48, 83168.96 examples/s]
Generating train split: 14200000 examples [02:48, 84722.25 examples/s]
Generating train split: 14210000 examples [02:48, 83814.47 examples/s]
Generating train split: 14220000 examples [02:48, 85052.61 examples/s]
Generating train split: 14230000 examples [02:48, 84380.65 examples/s]
Generating train split: 14240000 examples [02:49, 84372.25 examples/s]
Generating train split: 14250000 examples [02:49, 85385.05 examples/s]
Generating train split: 14260000 examples [02:49, 84941.44 examples/s]
Generating train split: 14270000 examples [02:49, 85086.73 examples/s]
Generating train split: 14280000 examples [02:49, 86057.73 examples/s]
Generating train split: 14290000 examples [02:49, 86497.73 examples/s]
Generating train split: 14300000 examples [02:49, 86647.29 examples/s]
Generating train split: 14310000 examples [02:49, 86165.51 examples/s]
Generating train split: 14320000 examples [02:49, 86772.71 examples/s]
Generating train split: 14330000 examples [02:50, 86255.25 examples/s]
Generating train split: 14339000 examples [02:50, 83287.17 examples/s]
Generating train split: 14348000 examples [02:50, 83134.10 examples/s]
Generating train split: 14357000 examples [02:50, 81276.95 examples/s]
Generating train split: 14366000 examples [02:50, 80854.21 examples/s]
Generating train split: 14375000 examples [02:50, 81549.08 examples/s]
Generating train split: 14384000 examples [02:50, 81453.40 examples/s]
Generating train split: 14394000 examples [02:50, 82881.95 examples/s]
Generating train split: 14404000 examples [02:50, 84664.53 examples/s]
Generating train split: 14414000 examples [02:51, 84461.77 examples/s]
Generating train split: 14424000 examples [02:51, 84893.41 examples/s]
Generating train split: 14434000 examples [02:51, 85654.74 examples/s]
Generating train split: 14444000 examples [02:51, 86618.39 examples/s]
Generating train split: 14454000 examples [02:51, 86817.17 examples/s]
Generating train split: 14464000 examples [02:51, 86922.43 examples/s]
Generating train split: 14474000 examples [02:51, 87488.73 examples/s]
Generating train split: 14484000 examples [02:51, 87660.02 examples/s]
Generating train split: 14494000 examples [02:52, 87678.91 examples/s]
Generating train split: 14504000 examples [02:52, 88359.65 examples/s]
Generating train split: 14514000 examples [02:52, 88572.24 examples/s]
Generating train split: 14524000 examples [02:52, 88412.35 examples/s]
Generating train split: 14534000 examples [02:52, 89178.76 examples/s]
Generating train split: 14544000 examples [02:52, 89102.66 examples/s]
Generating train split: 14554000 examples [02:52, 88990.18 examples/s]
Generating train split: 14564000 examples [02:52, 88899.30 examples/s]
Generating train split: 14574000 examples [02:52, 88921.41 examples/s]
Generating train split: 14584000 examples [02:53, 88830.47 examples/s]
Generating train split: 14593000 examples [02:53, 86810.49 examples/s]
Generating train split: 14602000 examples [02:53, 85561.70 examples/s]
Generating train split: 14612000 examples [02:53, 86716.29 examples/s]
Generating train split: 14622000 examples [02:53, 87437.63 examples/s]
Generating train split: 14632000 examples [02:53, 87463.74 examples/s]
Generating train split: 14642000 examples [02:53, 88043.15 examples/s]
Generating train split: 14652000 examples [02:53, 87252.61 examples/s]
Generating train split: 14662000 examples [02:53, 86936.20 examples/s]
Generating train split: 14672000 examples [02:54, 87558.33 examples/s]
Generating train split: 14682000 examples [02:54, 86614.84 examples/s]
Generating train split: 14692000 examples [02:54, 87182.84 examples/s]
Generating train split: 14702000 examples [02:54, 87778.74 examples/s]
Generating train split: 14712000 examples [02:54, 88020.81 examples/s]
Generating train split: 14722000 examples [02:54, 87972.08 examples/s]
Generating train split: 14732000 examples [02:54, 87250.59 examples/s]
Generating train split: 14742000 examples [02:54, 87328.73 examples/s]
Generating train split: 14752000 examples [02:54, 87451.02 examples/s]
Generating train split: 14762000 examples [02:55, 87439.94 examples/s]
Generating train split: 14772000 examples [02:55, 88271.75 examples/s]
Generating train split: 14782000 examples [02:55, 87955.47 examples/s]
Generating train split: 14792000 examples [02:55, 87556.97 examples/s]
Generating train split: 14802000 examples [02:55, 87115.21 examples/s]
Generating train split: 14812000 examples [02:55, 88537.49 examples/s]
Generating train split: 14822000 examples [02:55, 89097.83 examples/s]
Generating train split: 14831000 examples [02:55, 87345.58 examples/s]
Generating train split: 14841000 examples [02:55, 87585.72 examples/s]
Generating train split: 14851000 examples [02:56, 88450.09 examples/s]
Generating train split: 14860000 examples [02:56, 86663.10 examples/s]
Generating train split: 14870000 examples [02:56, 86948.12 examples/s]
Generating train split: 14873731 examples [02:56, 84345.73 examples/s]

Shard 0:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 0:   0%|          | 33042/100000000 [00:00<05:02, 330295.10tokens/s]
Shard 0:   0%|          | 385536/100000000 [00:00<00:45, 2208092.01tokens/s]
Shard 0:   1%|          | 618555/100000000 [00:00<00:44, 2237654.49tokens/s]
Shard 0:   1%|          | 919810/100000000 [00:00<00:38, 2541299.79tokens/s]
Shard 0:   1%|▏         | 1275915/100000000 [00:00<00:34, 2898516.35tokens/s]
Shard 0:   2%|▏         | 1612000/100000000 [00:00<00:32, 3026788.80tokens/s]
Shard 0:   2%|▏         | 2097274/100000000 [00:00<00:27, 3614909.14tokens/s]
Shard 0:   3%|▎         | 2639044/100000000 [00:00<00:23, 4172232.15tokens/s]
Shard 0:   3%|▎         | 3092503/100000000 [00:00<00:22, 4264137.64tokens/s]
Shard 0:   4%|▎         | 3519409/100000000 [00:01<00:25, 3727949.65tokens/s]
Shard 0:   4%|▍         | 3904117/100000000 [00:01<00:28, 3410159.19tokens/s]
Shard 0:   4%|▍         | 4257301/100000000 [00:01<00:30, 3173917.01tokens/s]
Shard 0:   5%|▍         | 4594564/100000000 [00:01<00:29, 3197731.71tokens/s]
Shard 0:   5%|▍         | 4972429/100000000 [00:01<00:28, 3336340.79tokens/s]
Shard 0:   5%|▌         | 5313013/100000000 [00:01<00:28, 3323198.56tokens/s]
Shard 0:   6%|▌         | 5788985/100000000 [00:01<00:25, 3695797.00tokens/s]
Shard 0:   6%|▋         | 6291737/100000000 [00:01<00:23, 4048339.08tokens/s]
Shard 0:   7%|▋         | 6701878/100000000 [00:01<00:23, 3920477.16tokens/s]
Shard 0:   7%|▋         | 7098026/100000000 [00:02<00:26, 3452243.54tokens/s]
Shard 0:   8%|▊         | 7596541/100000000 [00:02<00:23, 3850216.60tokens/s]
Shard 0:   8%|▊         | 8027746/100000000 [00:02<00:23, 3966253.86tokens/s]
Shard 0:   8%|▊         | 8435314/100000000 [00:02<00:26, 3427272.33tokens/s]
Shard 0:   9%|▉         | 8805318/100000000 [00:02<00:26, 3457672.11tokens/s]
Shard 0:   9%|▉         | 9195999/100000000 [00:02<00:25, 3570672.01tokens/s]
Shard 0:  10%|▉         | 9988388/100000000 [00:02<00:18, 4760535.66tokens/s]
Shard 0:  10%|█         | 10483132/100000000 [00:02<00:21, 4190816.34tokens/s]
Shard 0:  11%|█         | 10956364/100000000 [00:03<00:20, 4325233.84tokens/s]
Shard 0:  11%|█▏        | 11408816/100000000 [00:03<00:21, 4039257.95tokens/s]
Shard 0:  12%|█▏        | 12473174/100000000 [00:03<00:15, 5757317.55tokens/s]
Shard 0:  13%|█▎        | 13083103/100000000 [00:03<00:16, 5343743.12tokens/s]
Shard 0:  14%|█▎        | 13645860/100000000 [00:03<00:16, 5356589.42tokens/s]
Shard 0:  14%|█▍        | 14201806/100000000 [00:03<00:18, 4583925.95tokens/s]
Shard 0:  15%|█▍        | 14693424/100000000 [00:03<00:18, 4507965.48tokens/s]
Shard 0:  15%|█▌        | 15164819/100000000 [00:03<00:19, 4303216.75tokens/s]
Shard 0:  16%|█▌        | 15609281/100000000 [00:04<00:19, 4290484.73tokens/s]
Shard 0:  16%|█▋        | 16275653/100000000 [00:04<00:17, 4906777.82tokens/s]
Shard 0:  17%|█▋        | 16781548/100000000 [00:04<00:18, 4441377.44tokens/s]
Shard 0:  17%|█▋        | 17390271/100000000 [00:04<00:17, 4852999.91tokens/s]
Shard 0:  18%|█▊        | 18117014/100000000 [00:04<00:14, 5485235.28tokens/s]
Shard 0:  19%|█▊        | 18686015/100000000 [00:04<00:14, 5459246.46tokens/s]
Shard 0:  21%|██        | 20718365/100000000 [00:04<00:08, 9603326.72tokens/s]
Shard 0:  22%|██▏       | 21711987/100000000 [00:04<00:08, 8802449.79tokens/s]
Shard 0:  23%|██▎       | 22628323/100000000 [00:04<00:10, 7393994.32tokens/s]
Shard 0:  23%|██▎       | 23425975/100000000 [00:05<00:12, 6249188.53tokens/s]
Shard 0:  24%|██▍       | 24114346/100000000 [00:05<00:13, 5489745.14tokens/s]
Shard 0:  25%|██▍       | 24765074/100000000 [00:05<00:13, 5705093.86tokens/s]
Shard 0:  26%|██▌       | 25516556/100000000 [00:05<00:12, 6113055.96tokens/s]
Shard 0:  26%|██▋       | 26261689/100000000 [00:05<00:11, 6448066.78tokens/s]
Shard 0:  27%|██▋       | 26942576/100000000 [00:05<00:11, 6179408.59tokens/s]
Shard 0:  28%|██▊       | 27634991/100000000 [00:05<00:11, 6371982.59tokens/s]
Shard 0:  28%|██▊       | 28293341/100000000 [00:05<00:12, 5788823.45tokens/s]
Shard 0:  29%|██▉       | 28894296/100000000 [00:06<00:12, 5594949.12tokens/s]
Shard 0:  30%|██▉       | 29861728/100000000 [00:06<00:10, 6653089.57tokens/s]
Shard 0:  31%|███       | 30553485/100000000 [00:06<00:10, 6442769.25tokens/s]
Shard 0:  31%|███       | 31216342/100000000 [00:06<00:11, 6037412.66tokens/s]
Shard 0:  32%|███▏      | 32044254/100000000 [00:06<00:10, 6619205.33tokens/s]
Shard 0:  33%|███▎      | 32782192/100000000 [00:06<00:09, 6821722.91tokens/s]
Shard 0:  34%|███▎      | 33627601/100000000 [00:06<00:09, 7271813.51tokens/s]
Shard 0:  34%|███▍      | 34367949/100000000 [00:06<00:09, 6949851.85tokens/s]
Shard 0:  35%|███▌      | 35075109/100000000 [00:06<00:10, 6483660.27tokens/s]
Shard 0:  36%|███▌      | 35793741/100000000 [00:07<00:09, 6661373.92tokens/s]
Shard 0:  36%|███▋      | 36497636/100000000 [00:07<00:09, 6761595.21tokens/s]
Shard 0:  37%|███▋      | 37181910/100000000 [00:07<00:09, 6571873.30tokens/s]
Shard 0:  38%|███▊      | 37845828/100000000 [00:07<00:09, 6410489.56tokens/s]
Shard 0:  39%|███▊      | 38537624/100000000 [00:07<00:09, 6549988.36tokens/s]
Shard 0:  39%|███▉      | 39233527/100000000 [00:07<00:09, 6656615.41tokens/s]
Shard 0:  40%|███▉      | 39902409/100000000 [00:07<00:09, 6498669.68tokens/s]
Shard 0:  41%|████      | 40556710/100000000 [00:07<00:09, 6445765.92tokens/s]
Shard 0:  42%|████▏     | 41620547/100000000 [00:07<00:07, 7619279.99tokens/s]
Shard 0:  42%|████▏     | 42387264/100000000 [00:08<00:08, 6874257.72tokens/s]
Shard 0:  43%|████▎     | 43090814/100000000 [00:08<00:08, 6709970.81tokens/s]
Shard 0:  44%|████▍     | 43772512/100000000 [00:08<00:09, 5982956.18tokens/s]
Shard 0:  44%|████▍     | 44390457/100000000 [00:08<00:10, 5086051.96tokens/s]
Shard 0:  45%|████▍     | 44959374/100000000 [00:08<00:10, 5206578.92tokens/s]
Shard 0:  46%|████▌     | 45763251/100000000 [00:08<00:09, 5881174.38tokens/s]
Shard 0:  46%|████▋     | 46442972/100000000 [00:08<00:08, 6111234.58tokens/s]
Shard 0:  48%|████▊     | 47920268/100000000 [00:08<00:06, 8471435.72tokens/s]
Shard 0:  49%|████▉     | 48806766/100000000 [00:09<00:06, 7798289.50tokens/s]
Shard 0:  50%|████▉     | 49623476/100000000 [00:09<00:07, 7154861.25tokens/s]
Shard 0:  51%|█████     | 50539753/100000000 [00:09<00:06, 7648945.75tokens/s]
Shard 0:  51%|█████▏    | 51335325/100000000 [00:09<00:07, 6326594.19tokens/s]
Shard 0:  52%|█████▏    | 52022626/100000000 [00:09<00:08, 5608593.35tokens/s]
Shard 0:  53%|█████▎    | 52747945/100000000 [00:09<00:07, 5983872.62tokens/s]
Shard 0:  53%|█████▎    | 53391982/100000000 [00:09<00:08, 5599319.93tokens/s]
Shard 0:  54%|█████▍    | 53992957/100000000 [00:09<00:08, 5699434.14tokens/s]
Shard 0:  55%|█████▍    | 54697365/100000000 [00:10<00:07, 5960923.41tokens/s]
Shard 0:  55%|█████▌    | 55316987/100000000 [00:10<00:07, 5777612.64tokens/s]
Shard 0:  56%|█████▌    | 55908778/100000000 [00:10<00:07, 5795440.84tokens/s]
Shard 0:  57%|█████▋    | 56726228/100000000 [00:10<00:06, 6449045.96tokens/s]
Shard 0:  57%|█████▋    | 57471879/100000000 [00:10<00:06, 6733823.50tokens/s]
Shard 0:  58%|█████▊    | 58167593/100000000 [00:10<00:06, 6197476.31tokens/s]
Shard 0:  59%|█████▉    | 58821577/100000000 [00:10<00:06, 6236138.17tokens/s]
Shard 0:  59%|█████▉    | 59457827/100000000 [00:10<00:07, 5370133.11tokens/s]
Shard 0:  60%|██████    | 60373608/100000000 [00:10<00:06, 6328030.84tokens/s]
Shard 0:  61%|██████▏   | 61299717/100000000 [00:11<00:05, 7100100.31tokens/s]
Shard 0:  63%|██████▎   | 62782623/100000000 [00:11<00:04, 9219544.67tokens/s]
Shard 0:  64%|██████▎   | 63748701/100000000 [00:11<00:03, 9152925.45tokens/s]
Shard 0:  65%|██████▍   | 64693865/100000000 [00:11<00:04, 8613067.83tokens/s]
Shard 0:  66%|██████▌   | 65581894/100000000 [00:11<00:04, 8592676.44tokens/s]
Shard 0:  66%|██████▋   | 66459267/100000000 [00:11<00:04, 6808175.58tokens/s]
Shard 0:  67%|██████▋   | 67207496/100000000 [00:11<00:05, 6023433.17tokens/s]
Shard 0:  68%|██████▊   | 67867939/100000000 [00:11<00:05, 5978036.64tokens/s]
Shard 0:  69%|██████▊   | 68504575/100000000 [00:12<00:05, 5860814.82tokens/s]
Shard 0:  69%|██████▉   | 69116599/100000000 [00:12<00:05, 5850200.21tokens/s]
Shard 0:  70%|██████▉   | 69719478/100000000 [00:12<00:05, 5499210.36tokens/s]
Shard 0:  71%|███████   | 70565973/100000000 [00:12<00:04, 6245812.84tokens/s]
Shard 0:  71%|███████   | 71212446/100000000 [00:12<00:04, 6040314.20tokens/s]
Shard 0:  72%|███████▏  | 71831720/100000000 [00:12<00:04, 6048098.94tokens/s]
Shard 0:  72%|███████▏  | 72447085/100000000 [00:12<00:05, 5429238.79tokens/s]
Shard 0:  73%|███████▎  | 73189823/100000000 [00:12<00:04, 5946712.53tokens/s]
Shard 0:  74%|███████▍  | 73869871/100000000 [00:13<00:04, 6159415.04tokens/s]
Shard 0:  75%|███████▍  | 74502150/100000000 [00:13<00:04, 5584109.44tokens/s]
Shard 0:  75%|███████▌  | 75080068/100000000 [00:13<00:04, 5105692.92tokens/s]
Shard 0:  76%|███████▌  | 75609359/100000000 [00:13<00:05, 4802012.12tokens/s]
Shard 0:  76%|███████▌  | 76103037/100000000 [00:13<00:05, 4652747.84tokens/s]
Shard 0:  77%|███████▋  | 76576715/100000000 [00:13<00:05, 4331424.12tokens/s]
Shard 0:  77%|███████▋  | 77354709/100000000 [00:13<00:04, 5200995.56tokens/s]
Shard 0:  78%|███████▊  | 78033179/100000000 [00:13<00:03, 5566733.27tokens/s]
Shard 0:  79%|███████▉  | 78775324/100000000 [00:13<00:03, 6059924.91tokens/s]
Shard 0:  79%|███████▉  | 79398525/100000000 [00:14<00:03, 6028983.21tokens/s]
Shard 0:  80%|████████  | 80013166/100000000 [00:14<00:03, 5652761.97tokens/s]
Shard 0:  81%|████████  | 80590738/100000000 [00:14<00:03, 5370189.92tokens/s]
Shard 0:  81%|████████  | 81194927/100000000 [00:14<00:03, 5448787.47tokens/s]
Shard 0:  82%|████████▏ | 81770497/100000000 [00:14<00:03, 5520476.19tokens/s]
Shard 0:  82%|████████▏ | 82328246/100000000 [00:14<00:03, 5228003.17tokens/s]
Shard 0:  84%|████████▎ | 83553664/100000000 [00:14<00:02, 7144828.33tokens/s]
Shard 0:  84%|████████▍ | 84288505/100000000 [00:14<00:02, 6562904.61tokens/s]
Shard 0:  85%|████████▍ | 84968292/100000000 [00:15<00:02, 5441653.38tokens/s]
Shard 0:  86%|████████▌ | 85555720/100000000 [00:15<00:02, 5466855.56tokens/s]
Shard 0:  86%|████████▌ | 86177332/100000000 [00:15<00:02, 5595625.44tokens/s]
Shard 0:  87%|████████▋ | 86765056/100000000 [00:15<00:02, 5651123.59tokens/s]
Shard 0:  87%|████████▋ | 87463039/100000000 [00:15<00:02, 5988444.45tokens/s]
Shard 0:  88%|████████▊ | 88085097/100000000 [00:15<00:02, 5784092.35tokens/s]
Shard 0:  89%|████████▊ | 88676716/100000000 [00:15<00:02, 4801713.00tokens/s]
Shard 0:  89%|████████▉ | 89191427/100000000 [00:15<00:02, 4113240.43tokens/s]
Shard 0:  90%|████████▉ | 89638461/100000000 [00:16<00:02, 3870567.28tokens/s]
Shard 0:  90%|█████████ | 90255435/100000000 [00:16<00:02, 4379665.80tokens/s]
Shard 0:  91%|█████████ | 90727262/100000000 [00:16<00:02, 4310318.07tokens/s]
Shard 0:  91%|█████████ | 91218904/100000000 [00:16<00:01, 4429512.09tokens/s]
Shard 0:  92%|█████████▏| 91712106/100000000 [00:16<00:01, 4562612.21tokens/s]
Shard 0:  92%|█████████▏| 92181764/100000000 [00:16<00:01, 4269019.36tokens/s]
Shard 0:  93%|█████████▎| 92621841/100000000 [00:16<00:01, 3954677.67tokens/s]
Shard 0:  93%|█████████▎| 93124322/100000000 [00:16<00:01, 4209505.64tokens/s]
Shard 0:  94%|█████████▎| 93556397/100000000 [00:17<00:01, 3554899.84tokens/s]
Shard 0:  94%|█████████▍| 93991309/100000000 [00:17<00:01, 3747745.47tokens/s]
Shard 0:  94%|█████████▍| 94386911/100000000 [00:17<00:01, 3631114.61tokens/s]
Shard 0:  95%|█████████▍| 94821948/100000000 [00:17<00:01, 3808227.68tokens/s]
Shard 0:  95%|█████████▌| 95231050/100000000 [00:17<00:01, 3854331.98tokens/s]
Shard 0:  96%|█████████▌| 95709380/100000000 [00:17<00:01, 4066226.14tokens/s]
Shard 0:  96%|█████████▌| 96123562/100000000 [00:17<00:01, 3804475.12tokens/s]
Shard 0:  97%|█████████▋| 96511725/100000000 [00:17<00:01, 3451938.36tokens/s]
Shard 0:  97%|█████████▋| 97073719/100000000 [00:17<00:00, 3977504.33tokens/s]
Shard 0:  98%|█████████▊| 97651624/100000000 [00:18<00:00, 4449665.08tokens/s]
Shard 0:  98%|█████████▊| 98112209/100000000 [00:18<00:00, 3854778.24tokens/s]
Shard 0:  99%|█████████▊| 98520535/100000000 [00:18<00:00, 3512817.86tokens/s]
Shard 0:  99%|█████████▉| 98891533/100000000 [00:18<00:00, 3451558.19tokens/s]
Shard 0:  99%|█████████▉| 99254451/100000000 [00:18<00:00, 3485955.58tokens/s]
Shard 0: 100%|█████████▉| 99625270/100000000 [00:18<00:00, 3151391.96tokens/s]
Shard 0: 100%|█████████▉| 99951844/100000000 [00:18<00:00, 3040462.97tokens/s]
Shard 0: 100%|██████████| 100000000/100000000 [00:27<00:00, 3696967.23tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_val_000000.bin (200,001,024 bytes) in the gpt-2 format

Shard 1:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 1:   0%|          | 234032/100000000 [00:00<00:42, 2337001.41tokens/s]
Shard 1:   0%|          | 467957/100000000 [00:00<00:54, 1841338.73tokens/s]
Shard 1:   1%|          | 659161/100000000 [00:00<00:54, 1813591.12tokens/s]
Shard 1:   1%|          | 891515/100000000 [00:00<00:50, 1978128.20tokens/s]
Shard 1:   1%|          | 1121045/100000000 [00:00<00:47, 2084354.90tokens/s]
Shard 1:   1%|▏         | 1383617/100000000 [00:00<00:43, 2254144.83tokens/s]
Shard 1:   2%|▏         | 1611926/100000000 [00:00<00:48, 2028319.38tokens/s]
Shard 1:   2%|▏         | 2082289/100000000 [00:00<00:35, 2781479.74tokens/s]
Shard 1:   2%|▏         | 2371431/100000000 [00:01<00:38, 2524346.37tokens/s]
Shard 1:   3%|▎         | 2635211/100000000 [00:01<00:38, 2538195.04tokens/s]
Shard 1:   3%|▎         | 2956996/100000000 [00:01<00:35, 2726814.82tokens/s]
Shard 1:   3%|▎         | 3237388/100000000 [00:01<00:36, 2670451.37tokens/s]
Shard 1:   4%|▎         | 3515229/100000000 [00:01<00:35, 2683516.87tokens/s]
Shard 1:   4%|▍         | 3799263/100000000 [00:01<00:35, 2715830.93tokens/s]
Shard 1:   4%|▍         | 4089573/100000000 [00:01<00:34, 2766306.91tokens/s]
Shard 1:   4%|▍         | 4368313/100000000 [00:01<00:38, 2495770.25tokens/s]
Shard 1:   5%|▍         | 4623950/100000000 [00:01<00:40, 2355609.59tokens/s]
Shard 1:   5%|▍         | 4922882/100000000 [00:02<00:37, 2511814.81tokens/s]
Shard 1:   5%|▌         | 5203665/100000000 [00:02<00:36, 2592429.56tokens/s]
Shard 1:   6%|▌         | 5509981/100000000 [00:02<00:34, 2712904.47tokens/s]
Shard 1:   6%|▌         | 5795525/100000000 [00:02<00:34, 2735467.97tokens/s]
Shard 1:   6%|▌         | 6108039/100000000 [00:02<00:32, 2847502.45tokens/s]
Shard 1:   6%|▋         | 6395123/100000000 [00:02<00:33, 2757765.62tokens/s]
Shard 1:   7%|▋         | 6758891/100000000 [00:02<00:31, 2997608.10tokens/s]
Shard 1:   7%|▋         | 7182109/100000000 [00:02<00:27, 3352057.99tokens/s]
Shard 1:   8%|▊         | 7621840/100000000 [00:02<00:25, 3654430.76tokens/s]
Shard 1:   8%|▊         | 8029762/100000000 [00:02<00:24, 3762430.71tokens/s]
Shard 1:   8%|▊         | 8463312/100000000 [00:03<00:23, 3930433.55tokens/s]
Shard 1:   9%|▉         | 8858087/100000000 [00:03<00:24, 3650765.29tokens/s]
Shard 1:   9%|▉         | 9245520/100000000 [00:03<00:24, 3710778.63tokens/s]
Shard 1:  10%|▉         | 9621745/100000000 [00:03<00:26, 3417811.23tokens/s]
Shard 1:  10%|▉         | 9970106/100000000 [00:03<00:26, 3381698.15tokens/s]
Shard 1:  10%|█         | 10313402/100000000 [00:03<00:31, 2846551.29tokens/s]
Shard 1:  11%|█         | 10614039/100000000 [00:03<00:35, 2489538.86tokens/s]
Shard 1:  11%|█         | 10951804/100000000 [00:03<00:33, 2695140.31tokens/s]
Shard 1:  11%|█▏        | 11283881/100000000 [00:04<00:31, 2819223.55tokens/s]
Shard 1:  12%|█▏        | 11580861/100000000 [00:04<00:31, 2774652.76tokens/s]
Shard 1:  12%|█▏        | 11968745/100000000 [00:04<00:28, 3068386.05tokens/s]
Shard 1:  13%|█▎        | 12515015/100000000 [00:04<00:23, 3725313.58tokens/s]
Shard 1:  13%|█▎        | 12900506/100000000 [00:04<00:23, 3673339.18tokens/s]
Shard 1:  13%|█▎        | 13277074/100000000 [00:04<00:23, 3683145.22tokens/s]
Shard 1:  14%|█▎        | 13653859/100000000 [00:04<00:25, 3326572.00tokens/s]
Shard 1:  14%|█▍        | 13996978/100000000 [00:04<00:27, 3145411.82tokens/s]
Shard 1:  14%|█▍        | 14320210/100000000 [00:04<00:28, 2989225.40tokens/s]
Shard 1:  15%|█▍        | 14637663/100000000 [00:05<00:28, 3035443.80tokens/s]
Shard 1:  15%|█▍        | 14967783/100000000 [00:05<00:27, 3106838.08tokens/s]
Shard 1:  15%|█▌        | 15388167/100000000 [00:05<00:24, 3403404.87tokens/s]
Shard 1:  16%|█▌        | 15733255/100000000 [00:05<00:25, 3296040.43tokens/s]
Shard 1:  16%|█▌        | 16067273/100000000 [00:05<00:27, 3084343.85tokens/s]
Shard 1:  16%|█▋        | 16473535/100000000 [00:05<00:24, 3342086.19tokens/s]
Shard 1:  17%|█▋        | 17047810/100000000 [00:05<00:20, 4012775.15tokens/s]
Shard 1:  17%|█▋        | 17463552/100000000 [00:05<00:20, 4032259.24tokens/s]
Shard 1:  18%|█▊        | 17873263/100000000 [00:05<00:21, 3758780.40tokens/s]
Shard 1:  18%|█▊        | 18256604/100000000 [00:06<00:21, 3748820.44tokens/s]
Shard 1:  19%|█▊        | 18636670/100000000 [00:06<00:21, 3744659.76tokens/s]
Shard 1:  19%|█▉        | 19014924/100000000 [00:06<00:23, 3453655.12tokens/s]
Shard 1:  19%|█▉        | 19452223/100000000 [00:06<00:21, 3704310.35tokens/s]
Shard 1:  20%|█▉        | 19845188/100000000 [00:06<00:21, 3760110.92tokens/s]
Shard 1:  21%|██        | 20670488/100000000 [00:06<00:15, 5029270.22tokens/s]
Shard 1:  21%|██        | 21184998/100000000 [00:06<00:16, 4904552.81tokens/s]
Shard 1:  22%|██▏       | 21682280/100000000 [00:06<00:17, 4521325.58tokens/s]
Shard 1:  22%|██▏       | 22146955/100000000 [00:06<00:18, 4154524.75tokens/s]
Shard 1:  23%|██▎       | 22576349/100000000 [00:07<00:20, 3791471.97tokens/s]
Shard 1:  23%|██▎       | 23037945/100000000 [00:07<00:19, 3997339.48tokens/s]
Shard 1:  23%|██▎       | 23449663/100000000 [00:07<00:20, 3807067.72tokens/s]
Shard 1:  24%|██▍       | 23864021/100000000 [00:07<00:19, 3893851.49tokens/s]
Shard 1:  24%|██▍       | 24261337/100000000 [00:07<00:21, 3566795.41tokens/s]
Shard 1:  25%|██▍       | 24658928/100000000 [00:07<00:20, 3661986.10tokens/s]
Shard 1:  25%|██▌       | 25033751/100000000 [00:07<00:22, 3402677.69tokens/s]
Shard 1:  25%|██▌       | 25381453/100000000 [00:07<00:22, 3254168.79tokens/s]
Shard 1:  26%|██▌       | 25712147/100000000 [00:08<00:24, 3072710.32tokens/s]
Shard 1:  26%|██▌       | 26024136/100000000 [00:08<00:24, 3081890.44tokens/s]
Shard 1:  26%|██▋       | 26335388/100000000 [00:08<00:25, 2926750.00tokens/s]
Shard 1:  27%|██▋       | 26656858/100000000 [00:08<00:24, 2988263.75tokens/s]
Shard 1:  27%|██▋       | 26965823/100000000 [00:08<00:25, 2912245.82tokens/s]
Shard 1:  27%|██▋       | 27269484/100000000 [00:08<00:25, 2826741.44tokens/s]
Shard 1:  28%|██▊       | 27554060/100000000 [00:08<00:26, 2733265.51tokens/s]
Shard 1:  28%|██▊       | 27829997/100000000 [00:08<00:26, 2734506.83tokens/s]
Shard 1:  29%|██▊       | 28626386/100000000 [00:08<00:16, 4205521.69tokens/s]
Shard 1:  29%|██▉       | 29417260/100000000 [00:08<00:13, 5174260.89tokens/s]
Shard 1:  30%|██▉       | 29944328/100000000 [00:09<00:16, 4286633.14tokens/s]
Shard 1:  30%|███       | 30403800/100000000 [00:09<00:18, 3764978.77tokens/s]
Shard 1:  31%|███       | 30821705/100000000 [00:09<00:18, 3695913.75tokens/s]
Shard 1:  31%|███       | 31210864/100000000 [00:09<00:18, 3701635.41tokens/s]
Shard 1:  32%|███▏      | 31595320/100000000 [00:09<00:21, 3256266.36tokens/s]
Shard 1:  32%|███▏      | 31937681/100000000 [00:09<00:21, 3238260.69tokens/s]
Shard 1:  32%|███▏      | 32272694/100000000 [00:09<00:20, 3243331.52tokens/s]
Shard 1:  33%|███▎      | 32744147/100000000 [00:10<00:18, 3620226.15tokens/s]
Shard 1:  33%|███▎      | 33117078/100000000 [00:10<00:18, 3635099.25tokens/s]
Shard 1:  34%|███▎      | 33519137/100000000 [00:10<00:17, 3741291.21tokens/s]
Shard 1:  34%|███▍      | 33927248/100000000 [00:10<00:17, 3826118.29tokens/s]
Shard 1:  34%|███▍      | 34341293/100000000 [00:10<00:16, 3910528.84tokens/s]
Shard 1:  35%|███▍      | 34796322/100000000 [00:10<00:15, 4081254.75tokens/s]
Shard 1:  35%|███▌      | 35221010/100000000 [00:10<00:15, 4129745.61tokens/s]
Shard 1:  36%|███▌      | 36060417/100000000 [00:10<00:11, 5382302.19tokens/s]
Shard 1:  37%|███▋      | 36601581/100000000 [00:10<00:12, 5068188.75tokens/s]
Shard 1:  37%|███▋      | 37382152/100000000 [00:10<00:10, 5833075.23tokens/s]
Shard 1:  38%|███▊      | 37973073/100000000 [00:11<00:11, 5532898.31tokens/s]
Shard 1:  39%|███▉      | 38897617/100000000 [00:11<00:09, 6573960.56tokens/s]
Shard 1:  40%|███▉      | 39624914/100000000 [00:11<00:08, 6755465.12tokens/s]
Shard 1:  40%|████      | 40310160/100000000 [00:11<00:08, 6776798.75tokens/s]
Shard 1:  41%|████      | 41021615/100000000 [00:11<00:09, 6069430.80tokens/s]
Shard 1:  42%|████▏     | 41693245/100000000 [00:11<00:09, 6223955.08tokens/s]
Shard 1:  42%|████▏     | 42331475/100000000 [00:11<00:09, 6125953.16tokens/s]
Shard 1:  43%|████▎     | 43023233/100000000 [00:11<00:09, 6329320.89tokens/s]
Shard 1:  44%|████▎     | 43709526/100000000 [00:11<00:08, 6449200.57tokens/s]
Shard 1:  44%|████▍     | 44364130/100000000 [00:12<00:09, 6151750.01tokens/s]
Shard 1:  45%|████▍     | 44986726/100000000 [00:12<00:09, 5865666.99tokens/s]
Shard 1:  46%|████▌     | 45579691/100000000 [00:12<00:09, 5610430.44tokens/s]
Shard 1:  46%|████▌     | 46145881/100000000 [00:12<00:10, 5250459.14tokens/s]
Shard 1:  47%|████▋     | 46728310/100000000 [00:12<00:09, 5391742.79tokens/s]
Shard 1:  47%|████▋     | 47375729/100000000 [00:12<00:09, 5677566.83tokens/s]
Shard 1:  48%|████▊     | 47950811/100000000 [00:12<00:09, 5293533.78tokens/s]
Shard 1:  49%|████▊     | 48631592/100000000 [00:12<00:09, 5704768.08tokens/s]
Shard 1:  49%|████▉     | 49212986/100000000 [00:12<00:08, 5718256.80tokens/s]
Shard 1:  50%|████▉     | 49792120/100000000 [00:13<00:09, 5524301.93tokens/s]
Shard 1:  50%|█████     | 50352731/100000000 [00:13<00:09, 5395790.69tokens/s]
Shard 1:  51%|█████     | 50918726/100000000 [00:13<00:08, 5468981.09tokens/s]
Shard 1:  51%|█████▏    | 51469269/100000000 [00:13<00:09, 5098818.29tokens/s]
Shard 1:  52%|█████▏    | 52011452/100000000 [00:13<00:09, 5185271.51tokens/s]
Shard 1:  53%|█████▎    | 52547676/100000000 [00:13<00:09, 5219691.56tokens/s]
Shard 1:  53%|█████▎    | 53158828/100000000 [00:13<00:08, 5473813.00tokens/s]
Shard 1:  54%|█████▎    | 53710065/100000000 [00:13<00:08, 5405339.25tokens/s]
Shard 1:  55%|█████▍    | 54573109/100000000 [00:13<00:07, 6324062.34tokens/s]
Shard 1:  55%|█████▌    | 55482221/100000000 [00:13<00:06, 7111657.64tokens/s]
Shard 1:  56%|█████▋    | 56251670/100000000 [00:14<00:06, 7249702.05tokens/s]
Shard 1:  57%|█████▋    | 56980257/100000000 [00:14<00:06, 6734464.48tokens/s]
Shard 1:  58%|█████▊    | 57663979/100000000 [00:14<00:06, 6679149.09tokens/s]
Shard 1:  58%|█████▊    | 58340097/100000000 [00:14<00:06, 6687953.05tokens/s]
Shard 1:  59%|█████▉    | 59013534/100000000 [00:14<00:06, 6235544.98tokens/s]
Shard 1:  60%|█████▉    | 59645776/100000000 [00:14<00:06, 5775080.21tokens/s]
Shard 1:  60%|██████    | 60233935/100000000 [00:14<00:07, 5443396.36tokens/s]
Shard 1:  61%|██████    | 60787047/100000000 [00:14<00:07, 4988554.73tokens/s]
Shard 1:  61%|██████▏   | 61296262/100000000 [00:15<00:07, 5015039.15tokens/s]
Shard 1:  62%|██████▏   | 61881561/100000000 [00:15<00:07, 5239185.60tokens/s]
Shard 1:  63%|██████▎   | 62523980/100000000 [00:15<00:06, 5525121.70tokens/s]
Shard 1:  63%|██████▎   | 63311621/100000000 [00:15<00:05, 6155538.78tokens/s]
Shard 1:  64%|██████▍   | 63935986/100000000 [00:15<00:06, 5707545.26tokens/s]
Shard 1:  65%|██████▍   | 64518229/100000000 [00:15<00:06, 5611755.38tokens/s]
Shard 1:  65%|██████▌   | 65141982/100000000 [00:15<00:06, 5782700.64tokens/s]
Shard 1:  66%|██████▌   | 65727226/100000000 [00:15<00:06, 5594283.14tokens/s]
Shard 1:  66%|██████▋   | 66294778/100000000 [00:15<00:06, 5513230.10tokens/s]
Shard 1:  67%|██████▋   | 66906985/100000000 [00:15<00:05, 5648323.23tokens/s]
Shard 1:  67%|██████▋   | 67479816/100000000 [00:16<00:05, 5571211.21tokens/s]
Shard 1:  68%|██████▊   | 68077534/100000000 [00:16<00:05, 5680015.48tokens/s]
Shard 1:  69%|██████▉   | 68884065/100000000 [00:16<00:04, 6369433.80tokens/s]
Shard 1:  70%|██████▉   | 69524686/100000000 [00:16<00:05, 5632291.04tokens/s]
Shard 1:  70%|███████   | 70105972/100000000 [00:16<00:05, 5100275.00tokens/s]
Shard 1:  71%|███████   | 70668422/100000000 [00:16<00:05, 5182741.75tokens/s]
Shard 1:  71%|███████   | 71241013/100000000 [00:16<00:05, 5325620.07tokens/s]
Shard 1:  72%|███████▏  | 72108765/100000000 [00:16<00:04, 6249761.15tokens/s]
Shard 1:  73%|███████▎  | 72750456/100000000 [00:17<00:04, 5714368.08tokens/s]
Shard 1:  73%|███████▎  | 73341423/100000000 [00:17<00:06, 4373567.66tokens/s]
Shard 1:  74%|███████▍  | 73836268/100000000 [00:17<00:06, 4299788.64tokens/s]
Shard 1:  74%|███████▍  | 74341452/100000000 [00:17<00:05, 4470846.27tokens/s]
Shard 1:  75%|███████▌  | 75191098/100000000 [00:17<00:04, 5397422.43tokens/s]
Shard 1:  76%|███████▌  | 75766812/100000000 [00:17<00:04, 5190928.19tokens/s]
Shard 1:  76%|███████▋  | 76468723/100000000 [00:17<00:04, 5662893.95tokens/s]
Shard 1:  77%|███████▋  | 77246130/100000000 [00:17<00:03, 6229003.12tokens/s]
Shard 1:  78%|███████▊  | 77893056/100000000 [00:18<00:03, 6060915.69tokens/s]
Shard 1:  79%|███████▊  | 78642977/100000000 [00:18<00:03, 6418938.55tokens/s]
Shard 1:  80%|████████  | 80215097/100000000 [00:18<00:02, 9036152.43tokens/s]
Shard 1:  81%|████████  | 81144209/100000000 [00:18<00:02, 7913761.87tokens/s]
Shard 1:  82%|████████▏ | 81977008/100000000 [00:18<00:02, 7593246.56tokens/s]
Shard 1:  83%|████████▎ | 82765814/100000000 [00:18<00:02, 5854166.42tokens/s]
Shard 1:  84%|████████▎ | 83549457/100000000 [00:18<00:02, 6291591.06tokens/s]
Shard 1:  84%|████████▍ | 84246300/100000000 [00:18<00:02, 6210369.62tokens/s]
Shard 1:  85%|████████▍ | 84914314/100000000 [00:19<00:02, 5643714.10tokens/s]
Shard 1:  86%|████████▌ | 85518006/100000000 [00:19<00:02, 5689787.55tokens/s]
Shard 1:  86%|████████▌ | 86114493/100000000 [00:19<00:02, 5477246.74tokens/s]
Shard 1:  87%|████████▋ | 86680775/100000000 [00:19<00:02, 4485023.33tokens/s]
Shard 1:  87%|████████▋ | 87189970/100000000 [00:19<00:02, 4595157.65tokens/s]
Shard 1:  88%|████████▊ | 87679202/100000000 [00:19<00:02, 4587160.63tokens/s]
Shard 1:  88%|████████▊ | 88158447/100000000 [00:19<00:02, 4153556.59tokens/s]
Shard 1:  89%|████████▊ | 88592744/100000000 [00:20<00:03, 3605675.97tokens/s]
Shard 1:  89%|████████▉ | 89176599/100000000 [00:20<00:02, 4117061.10tokens/s]
Shard 1:  90%|████████▉ | 89749066/100000000 [00:20<00:02, 4511268.09tokens/s]
Shard 1:  90%|█████████ | 90228708/100000000 [00:20<00:02, 4399721.62tokens/s]
Shard 1:  91%|█████████ | 90689537/100000000 [00:20<00:02, 4134698.21tokens/s]
Shard 1:  91%|█████████ | 91118300/100000000 [00:20<00:02, 3660999.72tokens/s]
Shard 1:  92%|█████████▏| 91502145/100000000 [00:20<00:02, 3591646.25tokens/s]
Shard 1:  92%|█████████▏| 91889793/100000000 [00:20<00:02, 3135339.17tokens/s]
Shard 1:  92%|█████████▏| 92311739/100000000 [00:20<00:02, 3384273.42tokens/s]
Shard 1:  93%|█████████▎| 92667729/100000000 [00:21<00:02, 3316467.15tokens/s]
Shard 1:  93%|█████████▎| 93124282/100000000 [00:21<00:01, 3630537.09tokens/s]
Shard 1:  94%|█████████▎| 93501146/100000000 [00:21<00:01, 3553513.71tokens/s]
Shard 1:  94%|█████████▍| 93879641/100000000 [00:21<00:01, 3598338.91tokens/s]
Shard 1:  94%|█████████▍| 94246145/100000000 [00:21<00:01, 3240439.29tokens/s]
Shard 1:  95%|█████████▍| 94580608/100000000 [00:21<00:01, 3050492.99tokens/s]
Shard 1:  95%|█████████▍| 94893561/100000000 [00:21<00:01, 3045120.36tokens/s]
Shard 1:  95%|█████████▌| 95302697/100000000 [00:21<00:01, 3304762.03tokens/s]
Shard 1:  96%|█████████▌| 95639726/100000000 [00:22<00:01, 3105505.69tokens/s]
Shard 1:  96%|█████████▌| 95957821/100000000 [00:22<00:01, 2924215.24tokens/s]
Shard 1:  96%|█████████▋| 96262217/100000000 [00:22<00:01, 2871005.52tokens/s]
Shard 1:  97%|█████████▋| 96604979/100000000 [00:22<00:01, 3002776.34tokens/s]
Shard 1:  97%|█████████▋| 97029285/100000000 [00:22<00:00, 3317975.48tokens/s]
Shard 1:  97%|█████████▋| 97432168/100000000 [00:22<00:00, 3470061.01tokens/s]
Shard 1:  98%|█████████▊| 97790285/100000000 [00:22<00:00, 3495515.60tokens/s]
Shard 1:  98%|█████████▊| 98142603/100000000 [00:22<00:00, 3431717.44tokens/s]
Shard 1:  99%|█████████▊| 98521674/100000000 [00:22<00:00, 3534482.13tokens/s]
Shard 1:  99%|█████████▉| 98882835/100000000 [00:23<00:00, 3194305.31tokens/s]
Shard 1:  99%|█████████▉| 99209482/100000000 [00:23<00:00, 2931184.03tokens/s]
Shard 1: 100%|█████████▉| 99511163/100000000 [00:23<00:00, 2793485.49tokens/s]
Shard 1: 100%|█████████▉| 99813187/100000000 [00:23<00:00, 2843934.47tokens/s]
Shard 1: 100%|█████████▉| 99999079/100000000 [00:31<00:00, 3191058.15tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000001.bin (200,001,024 bytes) in the gpt-2 format

Shard 2:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 2:   0%|          | 250602/100000000 [00:00<00:40, 2451976.37tokens/s]
Shard 2:   1%|          | 610855/100000000 [00:00<00:32, 3087147.63tokens/s]
Shard 2:   1%|          | 922073/100000000 [00:00<00:34, 2865761.72tokens/s]
Shard 2:   1%|▏         | 1450661/100000000 [00:00<00:26, 3730988.31tokens/s]
Shard 2:   2%|▏         | 1871371/100000000 [00:00<00:25, 3888322.58tokens/s]
Shard 2:   2%|▏         | 2264635/100000000 [00:00<00:27, 3506682.71tokens/s]
Shard 2:   3%|▎         | 2623216/100000000 [00:00<00:28, 3390943.20tokens/s]
Shard 2:   3%|▎         | 2967522/100000000 [00:00<00:28, 3382653.89tokens/s]
Shard 2:   4%|▍         | 3998211/100000000 [00:00<00:17, 5399446.04tokens/s]
Shard 2:   5%|▍         | 4555110/100000000 [00:01<00:18, 5203095.54tokens/s]
Shard 2:   5%|▌         | 5088863/100000000 [00:01<00:20, 4586124.77tokens/s]
Shard 2:   6%|▌         | 5571832/100000000 [00:01<00:23, 3948090.50tokens/s]
Shard 2:   6%|▌         | 5992743/100000000 [00:01<00:24, 3881306.26tokens/s]
Shard 2:   6%|▋         | 6399666/100000000 [00:01<00:23, 3928115.02tokens/s]
Shard 2:   7%|▋         | 6805558/100000000 [00:01<00:24, 3811851.65tokens/s]
Shard 2:   7%|▋         | 7195454/100000000 [00:01<00:24, 3821798.19tokens/s]
Shard 2:   8%|▊         | 7771131/100000000 [00:01<00:21, 4338317.79tokens/s]
Shard 2:   8%|▊         | 8410771/100000000 [00:02<00:18, 4915619.07tokens/s]
Shard 2:   9%|▉         | 8913301/100000000 [00:02<00:18, 4868416.12tokens/s]
Shard 2:   9%|▉         | 9408749/100000000 [00:02<00:20, 4528682.49tokens/s]
Shard 2:  10%|▉         | 9871718/100000000 [00:02<00:22, 3974024.72tokens/s]
Shard 2:  10%|█         | 10288560/100000000 [00:02<00:23, 3820992.60tokens/s]
Shard 2:  11%|█         | 10681963/100000000 [00:02<00:25, 3533277.19tokens/s]
Shard 2:  11%|█         | 11073910/100000000 [00:02<00:27, 3272387.43tokens/s]
Shard 2:  11%|█▏        | 11409939/100000000 [00:02<00:28, 3149050.11tokens/s]
Shard 2:  12%|█▏        | 11730476/100000000 [00:03<00:28, 3060045.65tokens/s]
Shard 2:  12%|█▏        | 12228860/100000000 [00:03<00:24, 3550478.07tokens/s]
Shard 2:  13%|█▎        | 12800505/100000000 [00:03<00:21, 4129550.87tokens/s]
Shard 2:  13%|█▎        | 13278324/100000000 [00:03<00:20, 4304432.77tokens/s]
Shard 2:  14%|█▍        | 14399337/100000000 [00:03<00:13, 6243213.11tokens/s]
Shard 2:  15%|█▌        | 15048568/100000000 [00:03<00:14, 5954143.66tokens/s]
Shard 2:  16%|█▌        | 15659473/100000000 [00:03<00:15, 5611339.11tokens/s]
Shard 2:  16%|█▌        | 16233988/100000000 [00:03<00:15, 5566853.50tokens/s]
Shard 2:  17%|█▋        | 16806322/100000000 [00:03<00:17, 4890168.44tokens/s]
Shard 2:  17%|█▋        | 17315171/100000000 [00:04<00:18, 4473096.36tokens/s]
Shard 2:  18%|█▊        | 17785716/100000000 [00:04<00:18, 4426542.85tokens/s]
Shard 2:  18%|█▊        | 18239593/100000000 [00:04<00:18, 4415168.67tokens/s]
Shard 2:  19%|█▉        | 18755021/100000000 [00:04<00:17, 4607783.05tokens/s]
Shard 2:  19%|█▉        | 19223738/100000000 [00:04<00:18, 4389919.53tokens/s]
Shard 2:  20%|█▉        | 19808940/100000000 [00:04<00:16, 4742622.89tokens/s]
Shard 2:  20%|██        | 20290676/100000000 [00:04<00:16, 4719391.90tokens/s]
Shard 2:  21%|██        | 20788896/100000000 [00:04<00:16, 4793502.87tokens/s]
Shard 2:  21%|██▏       | 21272123/100000000 [00:04<00:18, 4202909.24tokens/s]
Shard 2:  22%|██▏       | 21750781/100000000 [00:05<00:17, 4356975.50tokens/s]
Shard 2:  22%|██▏       | 22199504/100000000 [00:05<00:18, 4224579.92tokens/s]
Shard 2:  23%|██▎       | 22781841/100000000 [00:05<00:16, 4656813.43tokens/s]
Shard 2:  23%|██▎       | 23261412/100000000 [00:05<00:17, 4505716.66tokens/s]
Shard 2:  24%|██▍       | 23761397/100000000 [00:05<00:16, 4610377.67tokens/s]
Shard 2:  24%|██▍       | 24229152/100000000 [00:05<00:19, 3795205.44tokens/s]
Shard 2:  25%|██▍       | 24635682/100000000 [00:05<00:21, 3426172.97tokens/s]
Shard 2:  25%|██▌       | 25027894/100000000 [00:05<00:21, 3523129.60tokens/s]
Shard 2:  25%|██▌       | 25399428/100000000 [00:06<00:21, 3433919.72tokens/s]
Shard 2:  26%|██▌       | 25803612/100000000 [00:06<00:20, 3581352.16tokens/s]
Shard 2:  26%|██▌       | 26214206/100000000 [00:06<00:19, 3719665.17tokens/s]
Shard 2:  27%|██▋       | 26625980/100000000 [00:06<00:19, 3788927.44tokens/s]
Shard 2:  27%|██▋       | 27025892/100000000 [00:06<00:19, 3836415.34tokens/s]
Shard 2:  27%|██▋       | 27445096/100000000 [00:06<00:18, 3929351.80tokens/s]
Shard 2:  28%|██▊       | 27850052/100000000 [00:06<00:18, 3951805.81tokens/s]
Shard 2:  28%|██▊       | 28363745/100000000 [00:06<00:16, 4295875.56tokens/s]
Shard 2:  29%|██▉       | 28796223/100000000 [00:06<00:17, 4136317.78tokens/s]
Shard 2:  29%|██▉       | 29213191/100000000 [00:07<00:17, 4125580.58tokens/s]
Shard 2:  30%|██▉       | 29628087/100000000 [00:07<00:18, 3783575.32tokens/s]
Shard 2:  30%|███       | 30044138/100000000 [00:07<00:18, 3882824.96tokens/s]
Shard 2:  31%|███       | 30857396/100000000 [00:07<00:13, 5036713.66tokens/s]
Shard 2:  31%|███▏      | 31371158/100000000 [00:07<00:18, 3767130.76tokens/s]
Shard 2:  32%|███▏      | 31800545/100000000 [00:07<00:18, 3718207.29tokens/s]
Shard 2:  32%|███▏      | 32208662/100000000 [00:07<00:18, 3752680.84tokens/s]
Shard 2:  33%|███▎      | 32609609/100000000 [00:07<00:18, 3687194.24tokens/s]
Shard 2:  33%|███▎      | 32996439/100000000 [00:08<00:20, 3321308.28tokens/s]
Shard 2:  33%|███▎      | 33345851/100000000 [00:08<00:20, 3216583.11tokens/s]
Shard 2:  34%|███▍      | 33790223/100000000 [00:08<00:18, 3503180.56tokens/s]
Shard 2:  34%|███▍      | 34270847/100000000 [00:08<00:17, 3842684.28tokens/s]
Shard 2:  35%|███▍      | 34671292/100000000 [00:08<00:19, 3367660.34tokens/s]
Shard 2:  36%|███▌      | 35529406/100000000 [00:08<00:13, 4684812.43tokens/s]
Shard 2:  36%|███▌      | 36035059/100000000 [00:08<00:13, 4771418.47tokens/s]
Shard 2:  37%|███▋      | 36539884/100000000 [00:08<00:14, 4329669.43tokens/s]
Shard 2:  37%|███▋      | 37180404/100000000 [00:08<00:12, 4841648.18tokens/s]
Shard 2:  38%|███▊      | 37689846/100000000 [00:09<00:14, 4431934.30tokens/s]
Shard 2:  38%|███▊      | 38156861/100000000 [00:09<00:15, 3990517.57tokens/s]
Shard 2:  39%|███▊      | 38696008/100000000 [00:09<00:14, 4308981.16tokens/s]
Shard 2:  39%|███▉      | 39147885/100000000 [00:09<00:15, 3991103.98tokens/s]
Shard 2:  40%|███▉      | 39567014/100000000 [00:09<00:15, 3956307.12tokens/s]
Shard 2:  41%|████      | 40942281/100000000 [00:09<00:09, 6515649.05tokens/s]
Shard 2:  42%|████▏     | 41639950/100000000 [00:09<00:12, 4499024.95tokens/s]
Shard 2:  42%|████▏     | 42204664/100000000 [00:10<00:13, 4230588.41tokens/s]
Shard 2:  43%|████▎     | 42708732/100000000 [00:10<00:14, 3981528.11tokens/s]
Shard 2:  43%|████▎     | 43161737/100000000 [00:10<00:15, 3740710.83tokens/s]
Shard 2:  44%|████▎     | 43572518/100000000 [00:10<00:15, 3665917.15tokens/s]
Shard 2:  44%|████▍     | 43963147/100000000 [00:10<00:17, 3200193.39tokens/s]
Shard 2:  44%|████▍     | 44437382/100000000 [00:10<00:15, 3531127.09tokens/s]
Shard 2:  45%|████▍     | 44818162/100000000 [00:10<00:15, 3514761.75tokens/s]
Shard 2:  45%|████▌     | 45258331/100000000 [00:11<00:14, 3736495.24tokens/s]
Shard 2:  46%|████▌     | 45793180/100000000 [00:11<00:13, 4156382.15tokens/s]
Shard 2:  46%|████▌     | 46226960/100000000 [00:11<00:14, 3728464.67tokens/s]
Shard 2:  47%|████▋     | 46619293/100000000 [00:11<00:14, 3759399.48tokens/s]
Shard 2:  47%|████▋     | 47104125/100000000 [00:11<00:13, 4040932.50tokens/s]
Shard 2:  48%|████▊     | 47536009/100000000 [00:11<00:12, 4080910.84tokens/s]
Shard 2:  48%|████▊     | 48238581/100000000 [00:11<00:10, 4902054.52tokens/s]
Shard 2:  49%|████▊     | 48740801/100000000 [00:11<00:11, 4335252.04tokens/s]
Shard 2:  49%|████▉     | 49194600/100000000 [00:11<00:11, 4258032.06tokens/s]
Shard 2:  50%|████▉     | 49802574/100000000 [00:12<00:10, 4738222.48tokens/s]
Shard 2:  51%|█████     | 50789912/100000000 [00:12<00:08, 6129745.60tokens/s]
Shard 2:  51%|█████▏    | 51424381/100000000 [00:12<00:08, 5440601.04tokens/s]
Shard 2:  52%|█████▏    | 51996225/100000000 [00:12<00:09, 4877415.95tokens/s]
Shard 2:  53%|█████▎    | 52745577/100000000 [00:12<00:08, 5525888.27tokens/s]
Shard 2:  53%|█████▎    | 53330512/100000000 [00:12<00:09, 4809192.48tokens/s]
Shard 2:  54%|█████▍    | 53847058/100000000 [00:12<00:10, 4346862.05tokens/s]
Shard 2:  54%|█████▍    | 54310593/100000000 [00:13<00:11, 4012993.35tokens/s]
Shard 2:  55%|█████▍    | 54733561/100000000 [00:13<00:11, 3891220.81tokens/s]
Shard 2:  55%|█████▌    | 55397535/100000000 [00:13<00:09, 4553397.37tokens/s]
Shard 2:  56%|█████▌    | 55879738/100000000 [00:13<00:10, 4353396.87tokens/s]
Shard 2:  56%|█████▋    | 56408410/100000000 [00:13<00:09, 4573403.54tokens/s]
Shard 2:  57%|█████▋    | 56881499/100000000 [00:13<00:09, 4558187.55tokens/s]
Shard 2:  57%|█████▋    | 57348296/100000000 [00:13<00:10, 4189011.11tokens/s]
Shard 2:  58%|█████▊    | 57781046/100000000 [00:13<00:12, 3517918.71tokens/s]
Shard 2:  58%|█████▊    | 58156028/100000000 [00:14<00:14, 2927830.64tokens/s]
Shard 2:  58%|█████▊    | 58477030/100000000 [00:14<00:14, 2832468.13tokens/s]
Shard 2:  59%|█████▉    | 58779247/100000000 [00:14<00:15, 2645545.63tokens/s]
Shard 2:  59%|█████▉    | 59181215/100000000 [00:14<00:13, 2942673.06tokens/s]
Shard 2:  60%|█████▉    | 59597810/100000000 [00:14<00:12, 3218467.56tokens/s]
Shard 2:  60%|█████▉    | 59960971/100000000 [00:14<00:12, 3317272.04tokens/s]
Shard 2:  60%|██████    | 60306284/100000000 [00:14<00:13, 2931786.51tokens/s]
Shard 2:  61%|██████    | 60782385/100000000 [00:14<00:11, 3386520.65tokens/s]
Shard 2:  61%|██████    | 61140987/100000000 [00:14<00:11, 3347085.05tokens/s]
Shard 2:  61%|██████▏   | 61490317/100000000 [00:15<00:11, 3340325.81tokens/s]
Shard 2:  62%|██████▏   | 61912603/100000000 [00:15<00:10, 3559376.58tokens/s]
Shard 2:  62%|██████▏   | 62277023/100000000 [00:15<00:10, 3506643.82tokens/s]
Shard 2:  63%|██████▎   | 62633404/100000000 [00:15<00:10, 3431005.40tokens/s]
Shard 2:  63%|██████▎   | 63026318/100000000 [00:15<00:10, 3559632.21tokens/s]
Shard 2:  63%|██████▎   | 63395072/100000000 [00:15<00:10, 3588891.37tokens/s]
Shard 2:  64%|██████▍   | 63818901/100000000 [00:15<00:09, 3768839.14tokens/s]
Shard 2:  64%|██████▍   | 64226131/100000000 [00:15<00:09, 3838687.37tokens/s]
Shard 2:  65%|██████▍   | 64612216/100000000 [00:15<00:09, 3579309.41tokens/s]
Shard 2:  65%|██████▌   | 65169971/100000000 [00:16<00:08, 4139457.78tokens/s]
Shard 2:  66%|██████▌   | 65591383/100000000 [00:16<00:08, 4058792.05tokens/s]
Shard 2:  66%|██████▌   | 66220673/100000000 [00:16<00:07, 4685302.41tokens/s]
Shard 2:  67%|██████▋   | 66725325/100000000 [00:16<00:07, 4514876.95tokens/s]
Shard 2:  67%|██████▋   | 67182992/100000000 [00:16<00:08, 4039545.98tokens/s]
Shard 2:  68%|██████▊   | 67600028/100000000 [00:16<00:07, 4066847.45tokens/s]
Shard 2:  68%|██████▊   | 68275120/100000000 [00:16<00:06, 4786350.07tokens/s]
Shard 2:  69%|██████▉   | 68823629/100000000 [00:16<00:06, 4974090.97tokens/s]
Shard 2:  69%|██████▉   | 69339923/100000000 [00:16<00:06, 4995656.11tokens/s]
Shard 2:  70%|██████▉   | 69847064/100000000 [00:17<00:06, 4488681.44tokens/s]
Shard 2:  70%|███████   | 70310638/100000000 [00:17<00:06, 4497221.05tokens/s]
Shard 2:  71%|███████   | 70770252/100000000 [00:17<00:07, 4132675.71tokens/s]
Shard 2:  71%|███████   | 71194668/100000000 [00:17<00:06, 4161695.61tokens/s]
Shard 2:  72%|███████▏  | 71659752/100000000 [00:17<00:06, 4295895.92tokens/s]
Shard 2:  72%|███████▏  | 72096458/100000000 [00:17<00:06, 4297170.27tokens/s]
Shard 2:  73%|███████▎  | 72556520/100000000 [00:17<00:06, 4329364.88tokens/s]
Shard 2:  73%|███████▎  | 72993064/100000000 [00:17<00:06, 3906517.21tokens/s]
Shard 2:  73%|███████▎  | 73396762/100000000 [00:17<00:06, 3845165.12tokens/s]
Shard 2:  74%|███████▍  | 73853659/100000000 [00:18<00:06, 4041837.47tokens/s]
Shard 2:  74%|███████▍  | 74264209/100000000 [00:18<00:06, 3876013.03tokens/s]
Shard 2:  75%|███████▍  | 74657289/100000000 [00:18<00:07, 3167895.07tokens/s]
Shard 2:  75%|███████▍  | 74999209/100000000 [00:18<00:07, 3175863.20tokens/s]
Shard 2:  75%|███████▌  | 75348004/100000000 [00:18<00:07, 3223481.36tokens/s]
Shard 2:  76%|███████▌  | 75682506/100000000 [00:18<00:07, 3114636.27tokens/s]
Shard 2:  76%|███████▌  | 76002269/100000000 [00:18<00:07, 3070421.83tokens/s]
Shard 2:  76%|███████▋  | 76389334/100000000 [00:18<00:07, 3283042.26tokens/s]
Shard 2:  77%|███████▋  | 76802069/100000000 [00:18<00:06, 3513262.64tokens/s]
Shard 2:  77%|███████▋  | 77159802/100000000 [00:19<00:06, 3390735.95tokens/s]
Shard 2:  78%|███████▊  | 77503505/100000000 [00:19<00:06, 3220877.19tokens/s]
Shard 2:  78%|███████▊  | 77878815/100000000 [00:19<00:06, 3365229.54tokens/s]
Shard 2:  78%|███████▊  | 78263539/100000000 [00:19<00:06, 3483511.96tokens/s]
Shard 2:  79%|███████▉  | 78830840/100000000 [00:19<00:05, 4104980.32tokens/s]
Shard 2:  79%|███████▉  | 79247761/100000000 [00:19<00:06, 3431558.48tokens/s]
Shard 2:  80%|███████▉  | 79612758/100000000 [00:19<00:06, 3012878.00tokens/s]
Shard 2:  80%|███████▉  | 79936146/100000000 [00:20<00:07, 2764826.91tokens/s]
Shard 2:  80%|████████  | 80238316/100000000 [00:20<00:07, 2781480.86tokens/s]
Shard 2:  81%|████████  | 80589439/100000000 [00:20<00:06, 2937291.74tokens/s]
Shard 2:  81%|████████  | 80894685/100000000 [00:20<00:06, 2947965.19tokens/s]
Shard 2:  81%|████████  | 81197723/100000000 [00:20<00:06, 2779888.93tokens/s]
Shard 2:  81%|████████▏ | 81482575/100000000 [00:20<00:06, 2689717.52tokens/s]
Shard 2:  82%|████████▏ | 81929239/100000000 [00:20<00:05, 3162303.56tokens/s]
Shard 2:  82%|████████▏ | 82360000/100000000 [00:20<00:05, 3464956.90tokens/s]
Shard 2:  83%|████████▎ | 83199481/100000000 [00:20<00:03, 4739759.76tokens/s]
Shard 2:  84%|████████▎ | 83689604/100000000 [00:21<00:04, 4012282.80tokens/s]
Shard 2:  84%|████████▍ | 84113430/100000000 [00:21<00:04, 3441421.73tokens/s]
Shard 2:  84%|████████▍ | 84488639/100000000 [00:21<00:04, 3250970.08tokens/s]
Shard 2:  85%|████████▍ | 84831808/100000000 [00:21<00:05, 2959031.39tokens/s]
Shard 2:  85%|████████▌ | 85141519/100000000 [00:21<00:05, 2944807.72tokens/s]
Shard 2:  86%|████████▌ | 85592053/100000000 [00:21<00:04, 3320382.75tokens/s]
Shard 2:  86%|████████▌ | 85970598/100000000 [00:21<00:04, 3409525.75tokens/s]
Shard 2:  86%|████████▋ | 86360202/100000000 [00:21<00:03, 3524612.69tokens/s]
Shard 2:  87%|████████▋ | 86744021/100000000 [00:22<00:03, 3547351.21tokens/s]
Shard 2:  87%|████████▋ | 87105198/100000000 [00:22<00:03, 3254549.05tokens/s]
Shard 2:  87%|████████▋ | 87439489/100000000 [00:22<00:04, 3018687.46tokens/s]
Shard 2:  88%|████████▊ | 87748978/100000000 [00:22<00:04, 2675390.94tokens/s]
Shard 2:  88%|████████▊ | 88027716/100000000 [00:22<00:04, 2642974.58tokens/s]
Shard 2:  88%|████████▊ | 88428668/100000000 [00:22<00:03, 2983286.24tokens/s]
Shard 2:  89%|████████▉ | 89003595/100000000 [00:22<00:02, 3718586.75tokens/s]
Shard 2:  89%|████████▉ | 89392090/100000000 [00:22<00:03, 3464001.54tokens/s]
Shard 2:  90%|████████▉ | 89752360/100000000 [00:22<00:03, 3303801.61tokens/s]
Shard 2:  90%|█████████ | 90307236/100000000 [00:23<00:02, 3868266.08tokens/s]
Shard 2:  91%|█████████ | 90710640/100000000 [00:23<00:02, 3664019.49tokens/s]
Shard 2:  91%|█████████ | 91185858/100000000 [00:23<00:02, 3932470.50tokens/s]
Shard 2:  92%|█████████▏| 91620290/100000000 [00:23<00:02, 4035931.09tokens/s]
Shard 2:  92%|█████████▏| 92310780/100000000 [00:23<00:01, 4832195.65tokens/s]
Shard 2:  93%|█████████▎| 92929763/100000000 [00:23<00:01, 5217865.42tokens/s]
Shard 2:  93%|█████████▎| 93460865/100000000 [00:23<00:01, 4854285.12tokens/s]
Shard 2:  94%|█████████▍| 93957274/100000000 [00:23<00:01, 4601776.64tokens/s]
Shard 2:  94%|█████████▍| 94427081/100000000 [00:24<00:01, 4343388.21tokens/s]
Shard 2:  95%|█████████▍| 94869186/100000000 [00:24<00:01, 4242424.29tokens/s]
Shard 2:  95%|█████████▌| 95299476/100000000 [00:24<00:01, 3980881.89tokens/s]
Shard 2:  96%|█████████▌| 95702741/100000000 [00:24<00:01, 3190986.54tokens/s]
Shard 2:  96%|█████████▌| 96047339/100000000 [00:24<00:01, 2793397.24tokens/s]
Shard 2:  96%|█████████▋| 96432953/100000000 [00:24<00:01, 3017130.93tokens/s]
Shard 2:  97%|█████████▋| 96809281/100000000 [00:24<00:00, 3195910.77tokens/s]
Shard 2:  97%|█████████▋| 97149478/100000000 [00:24<00:00, 3143005.85tokens/s]
Shard 2:  98%|█████████▊| 97501594/100000000 [00:25<00:00, 3237350.91tokens/s]
Shard 2:  98%|█████████▊| 97846571/100000000 [00:25<00:00, 3257949.87tokens/s]
Shard 2:  98%|█████████▊| 98182160/100000000 [00:25<00:00, 3285064.66tokens/s]
Shard 2:  99%|█████████▊| 98518045/100000000 [00:25<00:00, 2891896.78tokens/s]
Shard 2:  99%|█████████▉| 98819083/100000000 [00:25<00:00, 2480561.95tokens/s]
Shard 2:  99%|█████████▉| 99188838/100000000 [00:25<00:00, 2772683.40tokens/s]
Shard 2: 100%|█████████▉| 99525316/100000000 [00:25<00:00, 2902735.15tokens/s]
Shard 2: 100%|█████████▉| 99830583/100000000 [00:25<00:00, 2428452.78tokens/s]
Shard 2: 100%|█████████▉| 99999774/100000000 [00:34<00:00, 2927491.00tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000002.bin (200,001,024 bytes) in the gpt-2 format

Shard 3:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 3:   0%|          | 243382/100000000 [00:00<00:41, 2428772.87tokens/s]
Shard 3:   1%|          | 597622/100000000 [00:00<00:32, 3064476.70tokens/s]
Shard 3:   1%|          | 904241/100000000 [00:00<00:39, 2506732.35tokens/s]
Shard 3:   1%|          | 1164685/100000000 [00:00<00:44, 2224201.56tokens/s]
Shard 3:   2%|▏         | 1854130/100000000 [00:00<00:27, 3624979.72tokens/s]
Shard 3:   2%|▏         | 2248470/100000000 [00:00<00:29, 3365184.51tokens/s]
Shard 3:   3%|▎         | 2607308/100000000 [00:00<00:32, 3040363.91tokens/s]
Shard 3:   3%|▎         | 2929582/100000000 [00:01<00:35, 2698076.86tokens/s]
Shard 3:   3%|▎         | 3315867/100000000 [00:01<00:32, 2980019.80tokens/s]
Shard 3:   4%|▎         | 3742892/100000000 [00:01<00:29, 3307908.31tokens/s]
Shard 3:   4%|▍         | 4151047/100000000 [00:01<00:27, 3502119.22tokens/s]
Shard 3:   5%|▍         | 4516540/100000000 [00:01<00:27, 3488777.22tokens/s]
Shard 3:   5%|▍         | 4881009/100000000 [00:01<00:26, 3532384.29tokens/s]
Shard 3:   5%|▌         | 5360716/100000000 [00:01<00:24, 3871884.72tokens/s]
Shard 3:   6%|▌         | 5763079/100000000 [00:01<00:24, 3911206.07tokens/s]
Shard 3:   6%|▌         | 6189709/100000000 [00:01<00:23, 3971370.93tokens/s]
Shard 3:   7%|▋         | 6649090/100000000 [00:01<00:22, 4140500.38tokens/s]
Shard 3:   7%|▋         | 7066602/100000000 [00:02<00:23, 3946216.91tokens/s]
Shard 3:   7%|▋         | 7464864/100000000 [00:02<00:28, 3281817.94tokens/s]
Shard 3:   8%|▊         | 7813271/100000000 [00:02<00:28, 3240034.68tokens/s]
Shard 3:   8%|▊         | 8308647/100000000 [00:02<00:24, 3673747.35tokens/s]
Shard 3:   9%|▉         | 8799386/100000000 [00:02<00:22, 3980123.93tokens/s]
Shard 3:   9%|▉         | 9211641/100000000 [00:02<00:23, 3828292.60tokens/s]
Shard 3:  10%|▉         | 9635948/100000000 [00:02<00:22, 3929405.38tokens/s]
Shard 3:  10%|█         | 10073468/100000000 [00:02<00:22, 4051103.71tokens/s]
Shard 3:  10%|█         | 10485075/100000000 [00:02<00:23, 3821799.78tokens/s]
Shard 3:  11%|█         | 10892873/100000000 [00:03<00:22, 3887410.05tokens/s]
Shard 3:  11%|█▏        | 11286967/100000000 [00:03<00:25, 3456953.07tokens/s]
Shard 3:  12%|█▏        | 11644527/100000000 [00:03<00:26, 3311426.57tokens/s]
Shard 3:  12%|█▏        | 12090005/100000000 [00:03<00:24, 3605206.05tokens/s]
Shard 3:  13%|█▎        | 12592696/100000000 [00:03<00:21, 3989037.34tokens/s]
Shard 3:  13%|█▎        | 13048212/100000000 [00:03<00:20, 4146590.04tokens/s]
Shard 3:  13%|█▎        | 13472447/100000000 [00:03<00:22, 3931975.18tokens/s]
Shard 3:  14%|█▍        | 13918874/100000000 [00:03<00:21, 4066876.16tokens/s]
Shard 3:  14%|█▍        | 14454848/100000000 [00:03<00:19, 4430098.90tokens/s]
Shard 3:  15%|█▍        | 14904778/100000000 [00:04<00:21, 3994263.21tokens/s]
Shard 3:  15%|█▌        | 15321646/100000000 [00:04<00:21, 3871641.38tokens/s]
Shard 3:  16%|█▌        | 15737970/100000000 [00:04<00:21, 3948245.96tokens/s]
Shard 3:  16%|█▌        | 16139933/100000000 [00:04<00:22, 3677066.42tokens/s]
Shard 3:  17%|█▋        | 16568653/100000000 [00:04<00:21, 3837010.38tokens/s]
Shard 3:  17%|█▋        | 17040099/100000000 [00:04<00:20, 4065332.69tokens/s]
Shard 3:  17%|█▋        | 17453465/100000000 [00:04<00:23, 3560020.52tokens/s]
Shard 3:  18%|█▊        | 17824153/100000000 [00:04<00:23, 3458337.19tokens/s]
Shard 3:  18%|█▊        | 18287088/100000000 [00:05<00:21, 3766147.75tokens/s]
Shard 3:  19%|█▊        | 18712103/100000000 [00:05<00:20, 3889400.82tokens/s]
Shard 3:  19%|█▉        | 19110226/100000000 [00:05<00:21, 3730054.92tokens/s]
Shard 3:  19%|█▉        | 19491263/100000000 [00:05<00:22, 3503148.26tokens/s]
Shard 3:  20%|█▉        | 19925420/100000000 [00:05<00:21, 3716149.52tokens/s]
Shard 3:  20%|██        | 20361679/100000000 [00:05<00:20, 3889206.69tokens/s]
Shard 3:  21%|██        | 20756810/100000000 [00:05<00:21, 3642383.82tokens/s]
Shard 3:  21%|██        | 21128305/100000000 [00:05<00:21, 3656934.82tokens/s]
Shard 3:  21%|██▏       | 21498862/100000000 [00:05<00:21, 3631952.45tokens/s]
Shard 3:  22%|██▏       | 21898483/100000000 [00:06<00:20, 3732342.18tokens/s]
Shard 3:  22%|██▏       | 22323695/100000000 [00:06<00:20, 3850387.95tokens/s]
Shard 3:  23%|██▎       | 22711011/100000000 [00:06<00:20, 3822641.68tokens/s]
Shard 3:  23%|██▎       | 23171118/100000000 [00:06<00:19, 4024344.52tokens/s]
Shard 3:  24%|██▎       | 23596314/100000000 [00:06<00:18, 4084457.05tokens/s]
Shard 3:  24%|██▍       | 24007144/100000000 [00:06<00:19, 3836733.81tokens/s]
Shard 3:  25%|██▍       | 24771156/100000000 [00:06<00:15, 4905881.24tokens/s]
Shard 3:  25%|██▌       | 25408495/100000000 [00:06<00:14, 5308202.22tokens/s]
Shard 3:  26%|██▌       | 25947046/100000000 [00:06<00:14, 4993077.56tokens/s]
Shard 3:  26%|██▋       | 26454923/100000000 [00:07<00:17, 4257096.03tokens/s]
Shard 3:  27%|██▋       | 26905246/100000000 [00:07<00:18, 3919320.89tokens/s]
Shard 3:  27%|██▋       | 27335748/100000000 [00:07<00:19, 3664785.67tokens/s]
Shard 3:  28%|██▊       | 27716135/100000000 [00:07<00:20, 3555609.95tokens/s]
Shard 3:  28%|██▊       | 28329287/100000000 [00:07<00:17, 4189602.67tokens/s]
Shard 3:  29%|██▉       | 28767191/100000000 [00:07<00:16, 4227481.12tokens/s]
Shard 3:  29%|██▉       | 29203076/100000000 [00:07<00:16, 4249748.99tokens/s]
Shard 3:  30%|██▉       | 29638008/100000000 [00:07<00:17, 3955183.59tokens/s]
Shard 3:  30%|███       | 30043135/100000000 [00:07<00:18, 3746388.08tokens/s]
Shard 3:  30%|███       | 30425576/100000000 [00:08<00:19, 3565255.72tokens/s]
Shard 3:  31%|███       | 30823439/100000000 [00:08<00:18, 3657085.27tokens/s]
Shard 3:  31%|███▏      | 31269293/100000000 [00:08<00:17, 3872932.28tokens/s]
Shard 3:  32%|███▏      | 31685436/100000000 [00:08<00:17, 3938762.18tokens/s]
Shard 3:  32%|███▏      | 32088478/100000000 [00:08<00:17, 3964747.06tokens/s]
Shard 3:  33%|███▎      | 32656784/100000000 [00:08<00:15, 4456244.32tokens/s]
Shard 3:  33%|███▎      | 33106283/100000000 [00:08<00:16, 4058389.30tokens/s]
Shard 3:  34%|███▎      | 33521268/100000000 [00:08<00:18, 3682371.93tokens/s]
Shard 3:  34%|███▍      | 33901131/100000000 [00:09<00:19, 3404053.45tokens/s]
Shard 3:  34%|███▍      | 34270793/100000000 [00:09<00:18, 3466214.82tokens/s]
Shard 3:  35%|███▍      | 34793182/100000000 [00:09<00:16, 3927360.92tokens/s]
Shard 3:  35%|███▌      | 35197167/100000000 [00:09<00:17, 3601534.58tokens/s]
Shard 3:  36%|███▌      | 35641668/100000000 [00:09<00:16, 3818677.36tokens/s]
Shard 3:  36%|███▌      | 36061378/100000000 [00:09<00:16, 3920385.75tokens/s]
Shard 3:  37%|███▋      | 36765836/100000000 [00:09<00:13, 4788995.84tokens/s]
Shard 3:  37%|███▋      | 37256559/100000000 [00:09<00:14, 4423438.28tokens/s]
Shard 3:  38%|███▊      | 37712094/100000000 [00:09<00:15, 4103591.78tokens/s]
Shard 3:  38%|███▊      | 38324451/100000000 [00:10<00:13, 4621217.93tokens/s]
Shard 3:  39%|███▉      | 38801935/100000000 [00:10<00:13, 4608157.81tokens/s]
Shard 3:  39%|███▉      | 39273444/100000000 [00:10<00:13, 4361597.20tokens/s]
Shard 3:  40%|████      | 40000373/100000000 [00:10<00:11, 5151866.00tokens/s]
Shard 3:  41%|████      | 40530004/100000000 [00:10<00:13, 4530388.36tokens/s]
Shard 3:  41%|████      | 41100231/100000000 [00:10<00:12, 4761689.68tokens/s]
Shard 3:  42%|████▏     | 41596825/100000000 [00:10<00:13, 4471360.93tokens/s]
Shard 3:  42%|████▏     | 42124370/100000000 [00:10<00:12, 4676772.27tokens/s]
Shard 3:  43%|████▎     | 42605508/100000000 [00:10<00:13, 4253587.28tokens/s]
Shard 3:  43%|████▎     | 43046185/100000000 [00:11<00:14, 4023757.32tokens/s]
Shard 3:  43%|████▎     | 43459218/100000000 [00:11<00:15, 3738522.29tokens/s]
Shard 3:  44%|████▍     | 43846962/100000000 [00:11<00:14, 3773753.73tokens/s]
Shard 3:  44%|████▍     | 44256671/100000000 [00:11<00:14, 3850418.14tokens/s]
Shard 3:  45%|████▍     | 44647366/100000000 [00:11<00:14, 3760699.22tokens/s]
Shard 3:  45%|████▌     | 45028449/100000000 [00:11<00:15, 3642997.14tokens/s]
Shard 3:  46%|████▌     | 45539549/100000000 [00:11<00:13, 4000235.04tokens/s]
Shard 3:  46%|████▌     | 45949138/100000000 [00:11<00:13, 4019330.60tokens/s]
Shard 3:  46%|████▋     | 46436480/100000000 [00:11<00:12, 4243858.81tokens/s]
Shard 3:  47%|████▋     | 46921381/100000000 [00:12<00:12, 4404008.45tokens/s]
Shard 3:  47%|████▋     | 47454696/100000000 [00:12<00:11, 4653508.89tokens/s]
Shard 3:  48%|████▊     | 47924021/100000000 [00:12<00:12, 4180755.91tokens/s]
Shard 3:  48%|████▊     | 48498582/100000000 [00:12<00:11, 4600268.52tokens/s]
Shard 3:  49%|████▉     | 48969994/100000000 [00:12<00:11, 4358232.88tokens/s]
Shard 3:  49%|████▉     | 49415986/100000000 [00:12<00:11, 4306272.85tokens/s]
Shard 3:  50%|████▉     | 49854737/100000000 [00:12<00:12, 3960173.75tokens/s]
Shard 3:  50%|█████     | 50259710/100000000 [00:12<00:13, 3706034.65tokens/s]
Shard 3:  51%|█████     | 50692822/100000000 [00:13<00:12, 3855357.34tokens/s]
Shard 3:  51%|█████     | 51176104/100000000 [00:13<00:11, 4094150.11tokens/s]
Shard 3:  52%|█████▏    | 51592770/100000000 [00:13<00:11, 4054560.52tokens/s]
Shard 3:  52%|█████▏    | 52003268/100000000 [00:13<00:12, 3804428.08tokens/s]
Shard 3:  52%|█████▏    | 52398944/100000000 [00:13<00:12, 3687390.01tokens/s]
Shard 3:  53%|█████▎    | 52907726/100000000 [00:13<00:11, 4051688.60tokens/s]
Shard 3:  53%|█████▎    | 53325441/100000000 [00:13<00:11, 4083010.77tokens/s]
Shard 3:  54%|█████▍    | 53813808/100000000 [00:13<00:10, 4303301.66tokens/s]
Shard 3:  54%|█████▍    | 54339528/100000000 [00:13<00:09, 4577851.24tokens/s]
Shard 3:  55%|█████▍    | 54885107/100000000 [00:13<00:09, 4833695.70tokens/s]
Shard 3:  55%|█████▌    | 55372261/100000000 [00:14<00:09, 4522040.44tokens/s]
Shard 3:  56%|█████▌    | 55830766/100000000 [00:14<00:10, 4161939.60tokens/s]
Shard 3:  56%|█████▋    | 56255992/100000000 [00:14<00:10, 4024001.64tokens/s]
Shard 3:  57%|█████▋    | 56664600/100000000 [00:14<00:12, 3499719.44tokens/s]
Shard 3:  57%|█████▋    | 57045151/100000000 [00:14<00:12, 3574803.68tokens/s]
Shard 3:  58%|█████▊    | 57515535/100000000 [00:14<00:10, 3865248.85tokens/s]
Shard 3:  58%|█████▊    | 58038408/100000000 [00:14<00:10, 4160107.00tokens/s]
Shard 3:  58%|█████▊    | 58467285/100000000 [00:14<00:10, 3946469.12tokens/s]
Shard 3:  59%|█████▉    | 58870075/100000000 [00:15<00:12, 3306899.19tokens/s]
Shard 3:  59%|█████▉    | 59227517/100000000 [00:15<00:12, 3309857.54tokens/s]
Shard 3:  60%|█████▉    | 59573349/100000000 [00:15<00:13, 3014341.89tokens/s]
Shard 3:  60%|█████▉    | 59887685/100000000 [00:15<00:13, 3007015.96tokens/s]
Shard 3:  60%|██████    | 60197572/100000000 [00:15<00:14, 2824192.79tokens/s]
Shard 3:  61%|██████    | 60500536/100000000 [00:15<00:13, 2870587.63tokens/s]
Shard 3:  61%|██████    | 60793543/100000000 [00:15<00:13, 2814062.24tokens/s]
Shard 3:  61%|██████    | 61128010/100000000 [00:15<00:13, 2911397.18tokens/s]
Shard 3:  62%|██████▏   | 61539196/100000000 [00:15<00:11, 3221943.03tokens/s]
Shard 3:  62%|██████▏   | 61865851/100000000 [00:16<00:12, 3091740.85tokens/s]
Shard 3:  62%|██████▏   | 62178796/100000000 [00:16<00:12, 2945407.12tokens/s]
Shard 3:  62%|██████▏   | 62493193/100000000 [00:16<00:12, 2966955.41tokens/s]
Shard 3:  63%|██████▎   | 62817768/100000000 [00:16<00:12, 3041693.75tokens/s]
Shard 3:  63%|██████▎   | 63124245/100000000 [00:16<00:12, 2889460.46tokens/s]
Shard 3:  63%|██████▎   | 63416071/100000000 [00:16<00:12, 2872422.62tokens/s]
Shard 3:  64%|██████▎   | 63705072/100000000 [00:16<00:13, 2723253.30tokens/s]
Shard 3:  64%|██████▍   | 63983787/100000000 [00:16<00:13, 2708759.99tokens/s]
Shard 3:  64%|██████▍   | 64256529/100000000 [00:16<00:13, 2684465.52tokens/s]
Shard 3:  65%|██████▍   | 64526102/100000000 [00:17<00:13, 2653875.11tokens/s]
Shard 3:  65%|██████▍   | 64798018/100000000 [00:17<00:14, 2387612.72tokens/s]
Shard 3:  65%|██████▌   | 65198857/100000000 [00:17<00:12, 2806318.70tokens/s]
Shard 3:  66%|██████▌   | 65583206/100000000 [00:17<00:11, 3073904.59tokens/s]
Shard 3:  66%|██████▌   | 65936304/100000000 [00:17<00:10, 3200428.69tokens/s]
Shard 3:  66%|██████▋   | 66399094/100000000 [00:17<00:09, 3588564.89tokens/s]
Shard 3:  67%|██████▋   | 66763695/100000000 [00:17<00:09, 3470948.41tokens/s]
Shard 3:  67%|██████▋   | 67223938/100000000 [00:17<00:08, 3778781.35tokens/s]
Shard 3:  68%|██████▊   | 68142786/100000000 [00:17<00:05, 5335312.87tokens/s]
Shard 3:  69%|██████▉   | 68787582/100000000 [00:18<00:05, 5641435.07tokens/s]
Shard 3:  69%|██████▉   | 69359098/100000000 [00:18<00:05, 5216987.50tokens/s]
Shard 3:  70%|██████▉   | 69892097/100000000 [00:18<00:06, 4611739.75tokens/s]
Shard 3:  70%|███████   | 70373713/100000000 [00:18<00:07, 4157087.74tokens/s]
Shard 3:  71%|███████   | 70844385/100000000 [00:18<00:07, 4029128.38tokens/s]
Shard 3:  71%|███████▏  | 71260449/100000000 [00:18<00:07, 3595065.87tokens/s]
Shard 3:  72%|███████▏  | 71634108/100000000 [00:18<00:07, 3582204.44tokens/s]
Shard 3:  72%|███████▏  | 72001534/100000000 [00:19<00:08, 3219538.02tokens/s]
Shard 3:  72%|███████▏  | 72336594/100000000 [00:19<00:09, 2906893.67tokens/s]
Shard 3:  73%|███████▎  | 72813999/100000000 [00:19<00:08, 3324657.42tokens/s]
Shard 3:  73%|███████▎  | 73259650/100000000 [00:19<00:07, 3604693.44tokens/s]
Shard 3:  74%|███████▎  | 73637572/100000000 [00:19<00:07, 3444099.51tokens/s]
Shard 3:  74%|███████▍  | 74195642/100000000 [00:19<00:06, 3979242.16tokens/s]
Shard 3:  75%|███████▍  | 74610868/100000000 [00:19<00:07, 3295544.87tokens/s]
Shard 3:  75%|███████▍  | 74969699/100000000 [00:19<00:08, 3007315.39tokens/s]
Shard 3:  75%|███████▌  | 75292149/100000000 [00:20<00:08, 2764172.55tokens/s]
Shard 3:  76%|███████▌  | 75615902/100000000 [00:20<00:09, 2687341.42tokens/s]
Shard 3:  76%|███████▌  | 75903692/100000000 [00:20<00:08, 2724794.47tokens/s]
Shard 3:  76%|███████▌  | 76184642/100000000 [00:20<00:09, 2445423.39tokens/s]
Shard 3:  76%|███████▋  | 76439140/100000000 [00:20<00:09, 2391126.40tokens/s]
Shard 3:  77%|███████▋  | 76770668/100000000 [00:20<00:08, 2596492.48tokens/s]
Shard 3:  77%|███████▋  | 77037627/100000000 [00:20<00:09, 2492634.01tokens/s]
Shard 3:  77%|███████▋  | 77304374/100000000 [00:20<00:08, 2530273.74tokens/s]
Shard 3:  78%|███████▊  | 77617261/100000000 [00:20<00:08, 2686317.10tokens/s]
Shard 3:  78%|███████▊  | 77890507/100000000 [00:21<00:09, 2426790.82tokens/s]
Shard 3:  78%|███████▊  | 78143270/100000000 [00:21<00:09, 2362592.18tokens/s]
Shard 3:  79%|███████▊  | 78535973/100000000 [00:21<00:07, 2779432.63tokens/s]
Shard 3:  79%|███████▉  | 78823411/100000000 [00:21<00:07, 2692338.05tokens/s]
Shard 3:  79%|███████▉  | 79268805/100000000 [00:21<00:06, 3154335.85tokens/s]
Shard 3:  80%|███████▉  | 79626890/100000000 [00:21<00:07, 2855611.82tokens/s]
Shard 3:  80%|███████▉  | 79923728/100000000 [00:21<00:07, 2757771.33tokens/s]
Shard 3:  80%|████████  | 80207624/100000000 [00:21<00:07, 2658022.31tokens/s]
Shard 3:  81%|████████  | 80659457/100000000 [00:22<00:06, 3132673.66tokens/s]
Shard 3:  81%|████████  | 80983359/100000000 [00:22<00:06, 3105068.15tokens/s]
Shard 3:  81%|████████▏ | 81301094/100000000 [00:22<00:05, 3123012.66tokens/s]
Shard 3:  82%|████████▏ | 81618138/100000000 [00:23<00:20, 901292.47tokens/s] 
Shard 3:  82%|████████▏ | 81998289/100000000 [00:23<00:14, 1200234.60tokens/s]
Shard 3:  82%|████████▏ | 82354081/100000000 [00:23<00:11, 1503595.95tokens/s]
Shard 3:  83%|████████▎ | 82747642/100000000 [00:23<00:09, 1882161.12tokens/s]
Shard 3:  83%|████████▎ | 83110856/100000000 [00:23<00:07, 2191208.77tokens/s]
Shard 3:  83%|████████▎ | 83446587/100000000 [00:23<00:06, 2370164.12tokens/s]
Shard 3:  84%|████████▍ | 83771244/100000000 [00:23<00:06, 2491107.43tokens/s]
Shard 3:  84%|████████▍ | 84319747/100000000 [00:23<00:04, 3197113.05tokens/s]
Shard 3:  85%|████████▍ | 84813919/100000000 [00:24<00:04, 3612504.38tokens/s]
Shard 3:  85%|████████▌ | 85231421/100000000 [00:24<00:03, 3749611.98tokens/s]
Shard 3:  86%|████████▌ | 85644357/100000000 [00:24<00:04, 3014800.77tokens/s]
Shard 3:  86%|████████▌ | 86061809/100000000 [00:24<00:04, 3280780.16tokens/s]
Shard 3:  87%|████████▋ | 86623384/100000000 [00:24<00:03, 3859535.63tokens/s]
Shard 3:  87%|████████▋ | 87067849/100000000 [00:24<00:03, 3991312.80tokens/s]
Shard 3:  87%|████████▋ | 87497017/100000000 [00:24<00:03, 3908283.39tokens/s]
Shard 3:  88%|████████▊ | 87937946/100000000 [00:24<00:03, 4010128.80tokens/s]
Shard 3:  88%|████████▊ | 88354535/100000000 [00:24<00:02, 3999653.26tokens/s]
Shard 3:  89%|████████▉ | 88765493/100000000 [00:25<00:03, 3619445.34tokens/s]
Shard 3:  89%|████████▉ | 89141308/100000000 [00:25<00:03, 3042396.03tokens/s]
Shard 3:  89%|████████▉ | 89468193/100000000 [00:25<00:03, 2922961.28tokens/s]
Shard 3:  90%|████████▉ | 89775599/100000000 [00:25<00:03, 2855191.57tokens/s]
Shard 3:  90%|█████████ | 90071899/100000000 [00:25<00:03, 2574166.47tokens/s]
Shard 3:  90%|█████████ | 90339069/100000000 [00:25<00:03, 2560162.08tokens/s]
Shard 3:  91%|█████████ | 90604178/100000000 [00:25<00:03, 2529020.71tokens/s]
Shard 3:  91%|█████████ | 90971365/100000000 [00:26<00:03, 2826718.98tokens/s]
Shard 3:  91%|█████████▏| 91290279/100000000 [00:26<00:02, 2917028.04tokens/s]
Shard 3:  92%|█████████▏| 91588429/100000000 [00:26<00:02, 2826154.66tokens/s]
Shard 3:  92%|█████████▏| 91900508/100000000 [00:26<00:02, 2890205.78tokens/s]
Shard 3:  92%|█████████▏| 92209375/100000000 [00:26<00:02, 2909625.19tokens/s]
Shard 3:  93%|█████████▎| 92503385/100000000 [00:26<00:02, 2854769.64tokens/s]
Shard 3:  93%|█████████▎| 92956342/100000000 [00:26<00:02, 3332024.85tokens/s]
Shard 3:  94%|█████████▎| 93612841/100000000 [00:26<00:01, 4251616.59tokens/s]
Shard 3:  94%|█████████▍| 94044215/100000000 [00:26<00:01, 3557636.33tokens/s]
Shard 3:  94%|█████████▍| 94422411/100000000 [00:27<00:01, 3235423.72tokens/s]
Shard 3:  95%|█████████▍| 94790337/100000000 [00:27<00:01, 3331174.13tokens/s]
Shard 3:  95%|█████████▌| 95219856/100000000 [00:27<00:01, 3554657.21tokens/s]
Shard 3:  96%|█████████▌| 95698462/100000000 [00:27<00:01, 3878897.67tokens/s]
Shard 3:  96%|█████████▌| 96107301/100000000 [00:27<00:00, 3925504.41tokens/s]
Shard 3:  97%|█████████▋| 96582286/100000000 [00:27<00:00, 4153313.10tokens/s]
Shard 3:  97%|█████████▋| 97005899/100000000 [00:27<00:00, 4173629.25tokens/s]
Shard 3:  97%|█████████▋| 97428877/100000000 [00:27<00:00, 3709298.47tokens/s]
Shard 3:  98%|█████████▊| 97812769/100000000 [00:27<00:00, 3423889.26tokens/s]
Shard 3:  98%|█████████▊| 98167224/100000000 [00:28<00:00, 3417950.69tokens/s]
Shard 3:  99%|█████████▊| 98517695/100000000 [00:28<00:00, 3216596.59tokens/s]
Shard 3:  99%|█████████▉| 98846225/100000000 [00:28<00:00, 2915227.21tokens/s]
Shard 3:  99%|█████████▉| 99370967/100000000 [00:28<00:00, 3458196.60tokens/s]
Shard 3: 100%|█████████▉| 99730071/100000000 [00:28<00:00, 3171297.21tokens/s]
Shard 3: 100%|█████████▉| 99999941/100000000 [00:36<00:00, 2719397.14tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000003.bin (200,001,024 bytes) in the gpt-2 format

Shard 4:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 4:   0%|          | 267179/100000000 [00:00<00:39, 2533401.64tokens/s]
Shard 4:   1%|          | 542218/100000000 [00:00<00:37, 2644964.75tokens/s]
Shard 4:   1%|          | 807696/100000000 [00:00<00:41, 2416509.03tokens/s]
Shard 4:   1%|          | 1149099/100000000 [00:00<00:35, 2785102.24tokens/s]
Shard 4:   2%|▏         | 1658285/100000000 [00:00<00:27, 3581508.35tokens/s]
Shard 4:   2%|▏         | 2022450/100000000 [00:00<00:29, 3285735.95tokens/s]
Shard 4:   2%|▏         | 2426009/100000000 [00:00<00:27, 3509766.42tokens/s]
Shard 4:   3%|▎         | 2784844/100000000 [00:00<00:28, 3411945.04tokens/s]
Shard 4:   3%|▎         | 3132335/100000000 [00:00<00:29, 3330482.67tokens/s]
Shard 4:   3%|▎         | 3475867/100000000 [00:01<00:28, 3360550.45tokens/s]
Shard 4:   4%|▍         | 3880986/100000000 [00:01<00:26, 3560792.39tokens/s]
Shard 4:   4%|▍         | 4240339/100000000 [00:01<00:27, 3436012.06tokens/s]
Shard 4:   5%|▍         | 4587167/100000000 [00:01<00:28, 3335531.04tokens/s]
Shard 4:   5%|▌         | 5085937/100000000 [00:01<00:25, 3776027.10tokens/s]
Shard 4:   5%|▌         | 5467250/100000000 [00:01<00:29, 3159081.07tokens/s]
Shard 4:   6%|▌         | 5801627/100000000 [00:01<00:30, 3105540.62tokens/s]
Shard 4:   6%|▌         | 6181653/100000000 [00:01<00:28, 3283294.61tokens/s]
Shard 4:   7%|▋         | 6521254/100000000 [00:01<00:28, 3241529.15tokens/s]
Shard 4:   7%|▋         | 6866716/100000000 [00:02<00:28, 3299078.54tokens/s]
Shard 4:   7%|▋         | 7238797/100000000 [00:02<00:27, 3417970.25tokens/s]
Shard 4:   8%|▊         | 7636009/100000000 [00:02<00:25, 3573932.24tokens/s]
Shard 4:   8%|▊         | 8074743/100000000 [00:02<00:24, 3800562.93tokens/s]
Shard 4:   8%|▊         | 8462427/100000000 [00:02<00:24, 3812883.19tokens/s]
Shard 4:   9%|▉         | 8889449/100000000 [00:02<00:23, 3927072.83tokens/s]
Shard 4:   9%|▉         | 9284227/100000000 [00:02<00:24, 3700963.86tokens/s]
Shard 4:  10%|▉         | 9727402/100000000 [00:02<00:23, 3878874.20tokens/s]
Shard 4:  10%|█         | 10219604/100000000 [00:02<00:21, 4165841.91tokens/s]
Shard 4:  11%|█         | 10986597/100000000 [00:03<00:17, 5167042.08tokens/s]
Shard 4:  12%|█▏        | 11508309/100000000 [00:03<00:18, 4797457.55tokens/s]
Shard 4:  12%|█▏        | 11997157/100000000 [00:03<00:19, 4431071.87tokens/s]
Shard 4:  12%|█▏        | 12456230/100000000 [00:03<00:20, 4310543.93tokens/s]
Shard 4:  13%|█▎        | 13161348/100000000 [00:03<00:17, 5046741.02tokens/s]
Shard 4:  14%|█▎        | 13684458/100000000 [00:03<00:16, 5086319.38tokens/s]
Shard 4:  14%|█▍        | 14202989/100000000 [00:03<00:19, 4295292.76tokens/s]
Shard 4:  15%|█▍        | 14699867/100000000 [00:03<00:19, 4430639.35tokens/s]
Shard 4:  15%|█▌        | 15163284/100000000 [00:03<00:19, 4414554.50tokens/s]
Shard 4:  16%|█▌        | 15623732/100000000 [00:04<00:21, 3845247.14tokens/s]
Shard 4:  16%|█▌        | 16036210/100000000 [00:04<00:21, 3897036.38tokens/s]
Shard 4:  16%|█▋        | 16442398/100000000 [00:04<00:22, 3772580.94tokens/s]
Shard 4:  17%|█▋        | 16840991/100000000 [00:04<00:21, 3790769.60tokens/s]
Shard 4:  17%|█▋        | 17344973/100000000 [00:04<00:20, 4115170.26tokens/s]
Shard 4:  18%|█▊        | 17765058/100000000 [00:04<00:21, 3845178.17tokens/s]
Shard 4:  18%|█▊        | 18158237/100000000 [00:04<00:21, 3736820.88tokens/s]
Shard 4:  19%|█▊        | 18537660/100000000 [00:04<00:22, 3573600.06tokens/s]
Shard 4:  19%|█▉        | 18915018/100000000 [00:05<00:22, 3614604.75tokens/s]
Shard 4:  19%|█▉        | 19287047/100000000 [00:05<00:22, 3643697.82tokens/s]
Shard 4:  20%|█▉        | 19799670/100000000 [00:05<00:19, 4064289.38tokens/s]
Shard 4:  20%|██        | 20210660/100000000 [00:05<00:21, 3756799.80tokens/s]
Shard 4:  21%|██        | 20593818/100000000 [00:05<00:21, 3625015.89tokens/s]
Shard 4:  21%|██        | 21134357/100000000 [00:05<00:19, 4087903.40tokens/s]
Shard 4:  22%|██▏       | 21550939/100000000 [00:05<00:19, 3987260.52tokens/s]
Shard 4:  22%|██▏       | 21954880/100000000 [00:05<00:20, 3826982.01tokens/s]
Shard 4:  22%|██▏       | 22342232/100000000 [00:05<00:23, 3299858.91tokens/s]
Shard 4:  23%|██▎       | 22686207/100000000 [00:06<00:24, 3219506.67tokens/s]
Shard 4:  23%|██▎       | 23054457/100000000 [00:06<00:23, 3316632.06tokens/s]
Shard 4:  23%|██▎       | 23466081/100000000 [00:06<00:21, 3522390.21tokens/s]
Shard 4:  24%|██▍       | 23831129/100000000 [00:06<00:21, 3552974.12tokens/s]
Shard 4:  24%|██▍       | 24197687/100000000 [00:06<00:21, 3566775.28tokens/s]
Shard 4:  25%|██▍       | 24581806/100000000 [00:06<00:20, 3621560.78tokens/s]
Shard 4:  25%|██▍       | 24949335/100000000 [00:06<00:20, 3577954.78tokens/s]
Shard 4:  25%|██▌       | 25352075/100000000 [00:06<00:20, 3702866.57tokens/s]
Shard 4:  26%|██▌       | 25768185/100000000 [00:06<00:19, 3815569.28tokens/s]
Shard 4:  26%|██▌       | 26162042/100000000 [00:06<00:19, 3805203.16tokens/s]
Shard 4:  27%|██▋       | 26543647/100000000 [00:07<00:22, 3266670.49tokens/s]
Shard 4:  27%|██▋       | 26883821/100000000 [00:07<00:23, 3062439.90tokens/s]
Shard 4:  27%|██▋       | 27212696/100000000 [00:07<00:23, 3110407.86tokens/s]
Shard 4:  28%|██▊       | 27545251/100000000 [00:07<00:23, 3107669.09tokens/s]
Shard 4:  28%|██▊       | 28026358/100000000 [00:07<00:20, 3573642.24tokens/s]
Shard 4:  28%|██▊       | 28392099/100000000 [00:07<00:24, 2950156.93tokens/s]
Shard 4:  29%|██▊       | 28710302/100000000 [00:07<00:24, 2964691.39tokens/s]
Shard 4:  29%|██▉       | 29022900/100000000 [00:07<00:24, 2891996.60tokens/s]
Shard 4:  29%|██▉       | 29324302/100000000 [00:08<00:25, 2803085.55tokens/s]
Shard 4:  30%|██▉       | 29853175/100000000 [00:08<00:20, 3453310.00tokens/s]
Shard 4:  30%|███       | 30242747/100000000 [00:08<00:19, 3565210.44tokens/s]
Shard 4:  31%|███       | 30753677/100000000 [00:08<00:17, 3985449.54tokens/s]
Shard 4:  31%|███       | 31162417/100000000 [00:08<00:19, 3524124.50tokens/s]
Shard 4:  32%|███▏      | 31547589/100000000 [00:08<00:18, 3604963.14tokens/s]
Shard 4:  32%|███▏      | 31920716/100000000 [00:08<00:20, 3396109.04tokens/s]
Shard 4:  32%|███▏      | 32270801/100000000 [00:08<00:21, 3146337.95tokens/s]
Shard 4:  33%|███▎      | 32595705/100000000 [00:09<00:27, 2411813.47tokens/s]
Shard 4:  33%|███▎      | 32875432/100000000 [00:09<00:28, 2340502.62tokens/s]
Shard 4:  33%|███▎      | 33129496/100000000 [00:09<00:28, 2349805.72tokens/s]
Shard 4:  33%|███▎      | 33404908/100000000 [00:09<00:27, 2433381.52tokens/s]
Shard 4:  34%|███▎      | 33740728/100000000 [00:09<00:24, 2669797.82tokens/s]
Shard 4:  34%|███▍      | 34019956/100000000 [00:09<00:24, 2668919.95tokens/s]
Shard 4:  34%|███▍      | 34316834/100000000 [00:09<00:23, 2743130.73tokens/s]
Shard 4:  35%|███▍      | 34599288/100000000 [00:09<00:24, 2665754.94tokens/s]
Shard 4:  35%|███▍      | 34875548/100000000 [00:10<00:26, 2498071.83tokens/s]
Shard 4:  35%|███▌      | 35130388/100000000 [00:10<00:26, 2434151.16tokens/s]
Shard 4:  35%|███▌      | 35384565/100000000 [00:10<00:27, 2328011.99tokens/s]
Shard 4:  36%|███▌      | 35620182/100000000 [00:10<00:31, 2029662.78tokens/s]
Shard 4:  36%|███▌      | 35842713/100000000 [00:10<00:31, 2067142.09tokens/s]
Shard 4:  36%|███▌      | 36055931/100000000 [00:10<00:34, 1832973.85tokens/s]
Shard 4:  36%|███▋      | 36288234/100000000 [00:10<00:32, 1950637.80tokens/s]
Shard 4:  36%|███▋      | 36491200/100000000 [00:10<00:32, 1957389.41tokens/s]
Shard 4:  37%|███▋      | 36709047/100000000 [00:10<00:31, 2000898.65tokens/s]
Shard 4:  37%|███▋      | 36913491/100000000 [00:11<00:35, 1760970.93tokens/s]
Shard 4:  37%|███▋      | 37145604/100000000 [00:11<00:33, 1903028.21tokens/s]
Shard 4:  37%|███▋      | 37470510/100000000 [00:11<00:27, 2263717.25tokens/s]
Shard 4:  38%|███▊      | 37864007/100000000 [00:11<00:22, 2701980.35tokens/s]
Shard 4:  38%|███▊      | 38176430/100000000 [00:11<00:22, 2805647.02tokens/s]
Shard 4:  38%|███▊      | 38463910/100000000 [00:11<00:22, 2700381.45tokens/s]
Shard 4:  39%|███▊      | 38740078/100000000 [00:11<00:22, 2708759.66tokens/s]
Shard 4:  39%|███▉      | 39015625/100000000 [00:11<00:22, 2662136.39tokens/s]
Shard 4:  39%|███▉      | 39287384/100000000 [00:11<00:23, 2587460.95tokens/s]
Shard 4:  40%|███▉      | 39631874/100000000 [00:12<00:21, 2822160.13tokens/s]
Shard 4:  40%|███▉      | 39921968/100000000 [00:12<00:24, 2469136.16tokens/s]
Shard 4:  40%|████      | 40179578/100000000 [00:12<00:25, 2317594.58tokens/s]
Shard 4:  40%|████      | 40418808/100000000 [00:12<00:26, 2288247.50tokens/s]
Shard 4:  41%|████      | 40653707/100000000 [00:12<00:26, 2276392.45tokens/s]
Shard 4:  41%|████      | 40885837/100000000 [00:12<00:26, 2211575.51tokens/s]
Shard 4:  41%|████      | 41109356/100000000 [00:12<00:27, 2142009.56tokens/s]
Shard 4:  41%|████▏     | 41343870/100000000 [00:12<00:26, 2174505.80tokens/s]
Shard 4:  42%|████▏     | 41563791/100000000 [00:13<00:28, 2070495.61tokens/s]
Shard 4:  42%|████▏     | 41905680/100000000 [00:13<00:23, 2428644.44tokens/s]
Shard 4:  42%|████▏     | 42271146/100000000 [00:13<00:20, 2757133.01tokens/s]
Shard 4:  43%|████▎     | 42553588/100000000 [00:13<00:21, 2727215.13tokens/s]
Shard 4:  43%|████▎     | 43185228/100000000 [00:13<00:15, 3709336.34tokens/s]
Shard 4:  44%|████▎     | 43560540/100000000 [00:13<00:15, 3655019.53tokens/s]
Shard 4:  44%|████▍     | 43929061/100000000 [00:13<00:18, 3050767.14tokens/s]
Shard 4:  44%|████▍     | 44252280/100000000 [00:13<00:18, 2954510.34tokens/s]
Shard 4:  45%|████▍     | 44559963/100000000 [00:13<00:19, 2831728.93tokens/s]
Shard 4:  45%|████▍     | 44858359/100000000 [00:14<00:19, 2866287.89tokens/s]
Shard 4:  45%|████▌     | 45151490/100000000 [00:14<00:19, 2753387.18tokens/s]
Shard 4:  45%|████▌     | 45449307/100000000 [00:14<00:19, 2795252.87tokens/s]
Shard 4:  46%|████▌     | 45733174/100000000 [00:14<00:20, 2600361.09tokens/s]
Shard 4:  46%|████▌     | 46100242/100000000 [00:14<00:18, 2881991.64tokens/s]
Shard 4:  46%|████▋     | 46396493/100000000 [00:14<00:20, 2583708.29tokens/s]
Shard 4:  47%|████▋     | 46752481/100000000 [00:14<00:18, 2836076.65tokens/s]
Shard 4:  47%|████▋     | 47142090/100000000 [00:14<00:16, 3117337.76tokens/s]
Shard 4:  48%|████▊     | 47576383/100000000 [00:14<00:15, 3451195.69tokens/s]
Shard 4:  48%|████▊     | 48006748/100000000 [00:15<00:14, 3677705.92tokens/s]
Shard 4:  49%|████▊     | 48548356/100000000 [00:15<00:12, 4177062.05tokens/s]
Shard 4:  49%|████▉     | 49071022/100000000 [00:15<00:11, 4466365.98tokens/s]
Shard 4:  50%|████▉     | 49596484/100000000 [00:15<00:10, 4693643.39tokens/s]
Shard 4:  50%|█████     | 50070462/100000000 [00:15<00:11, 4173275.66tokens/s]
Shard 4:  51%|█████     | 50502434/100000000 [00:15<00:12, 3934101.57tokens/s]
Shard 4:  51%|█████     | 50966869/100000000 [00:15<00:11, 4120766.71tokens/s]
Shard 4:  51%|█████▏    | 51389171/100000000 [00:15<00:11, 4089979.74tokens/s]
Shard 4:  52%|█████▏    | 51805209/100000000 [00:16<00:14, 3261442.01tokens/s]
Shard 4:  52%|█████▏    | 52164890/100000000 [00:16<00:14, 3204098.49tokens/s]
Shard 4:  53%|█████▎    | 52505515/100000000 [00:16<00:15, 3081810.25tokens/s]
Shard 4:  53%|█████▎    | 52951512/100000000 [00:16<00:13, 3423262.88tokens/s]
Shard 4:  53%|█████▎    | 53428143/100000000 [00:16<00:12, 3767180.46tokens/s]
Shard 4:  54%|█████▍    | 54021096/100000000 [00:16<00:10, 4352596.60tokens/s]
Shard 4:  54%|█████▍    | 54472895/100000000 [00:16<00:10, 4210075.72tokens/s]
Shard 4:  56%|█████▌    | 56225113/100000000 [00:16<00:05, 7888383.43tokens/s]
Shard 4:  57%|█████▋    | 57051426/100000000 [00:16<00:06, 6332168.47tokens/s]
Shard 4:  58%|█████▊    | 57758943/100000000 [00:17<00:08, 4948837.00tokens/s]
Shard 4:  58%|█████▊    | 58346943/100000000 [00:17<00:10, 4058553.83tokens/s]
Shard 4:  59%|█████▉    | 58841240/100000000 [00:17<00:09, 4217935.88tokens/s]
Shard 4:  60%|██████    | 60048389/100000000 [00:17<00:06, 5793685.47tokens/s]
Shard 4:  61%|██████    | 60725261/100000000 [00:17<00:07, 5141081.91tokens/s]
Shard 4:  61%|██████▏   | 61315804/100000000 [00:17<00:07, 4965364.58tokens/s]
Shard 4:  62%|██████▏   | 61864405/100000000 [00:18<00:08, 4254242.57tokens/s]
Shard 4:  62%|██████▏   | 62336771/100000000 [00:18<00:08, 4251736.07tokens/s]
Shard 4:  63%|██████▎   | 62966926/100000000 [00:18<00:07, 4711143.93tokens/s]
Shard 4:  63%|██████▎   | 63475561/100000000 [00:18<00:08, 4313288.14tokens/s]
Shard 4:  64%|██████▍   | 63950424/100000000 [00:18<00:08, 4416988.12tokens/s]
Shard 4:  64%|██████▍   | 64415179/100000000 [00:18<00:08, 4403375.32tokens/s]
Shard 4:  65%|██████▍   | 64871103/100000000 [00:18<00:08, 4295304.97tokens/s]
Shard 4:  65%|██████▌   | 65315991/100000000 [00:18<00:08, 4032733.93tokens/s]
Shard 4:  66%|██████▌   | 65778387/100000000 [00:19<00:08, 4172954.13tokens/s]
Shard 4:  66%|██████▌   | 66208193/100000000 [00:19<00:10, 3077852.04tokens/s]
Shard 4:  67%|██████▋   | 66565217/100000000 [00:19<00:10, 3102287.17tokens/s]
Shard 4:  67%|██████▋   | 66907999/100000000 [00:19<00:10, 3048830.89tokens/s]
Shard 4:  67%|██████▋   | 67238892/100000000 [00:19<00:10, 3061671.03tokens/s]
Shard 4:  68%|██████▊   | 67599815/100000000 [00:19<00:10, 3198249.85tokens/s]
Shard 4:  68%|██████▊   | 67991785/100000000 [00:19<00:09, 3381966.70tokens/s]
Shard 4:  68%|██████▊   | 68342841/100000000 [00:19<00:10, 2956942.46tokens/s]
Shard 4:  69%|██████▊   | 68676047/100000000 [00:20<00:10, 3048892.05tokens/s]
Shard 4:  69%|██████▉   | 68994307/100000000 [00:20<00:10, 2961937.50tokens/s]
Shard 4:  69%|██████▉   | 69301855/100000000 [00:20<00:11, 2712128.46tokens/s]
Shard 4:  70%|██████▉   | 69620990/100000000 [00:20<00:10, 2834383.88tokens/s]
Shard 4:  70%|██████▉   | 69912915/100000000 [00:20<00:10, 2766185.31tokens/s]
Shard 4:  70%|███████   | 70269653/100000000 [00:20<00:09, 2980661.27tokens/s]
Shard 4:  71%|███████   | 70644839/100000000 [00:20<00:09, 3181484.86tokens/s]
Shard 4:  71%|███████   | 70969537/100000000 [00:20<00:10, 2723777.29tokens/s]
Shard 4:  71%|███████▏  | 71322232/100000000 [00:20<00:09, 2923821.19tokens/s]
Shard 4:  72%|███████▏  | 71804659/100000000 [00:21<00:08, 3426429.31tokens/s]
Shard 4:  72%|███████▏  | 72162551/100000000 [00:21<00:08, 3129093.51tokens/s]
Shard 4:  72%|███████▏  | 72490612/100000000 [00:21<00:09, 2929640.19tokens/s]
Shard 4:  73%|███████▎  | 72795259/100000000 [00:21<00:09, 2873949.75tokens/s]
Shard 4:  73%|███████▎  | 73161403/100000000 [00:21<00:08, 3073804.14tokens/s]
Shard 4:  73%|███████▎  | 73477298/100000000 [00:21<00:09, 2747751.46tokens/s]
Shard 4:  74%|███████▍  | 73868645/100000000 [00:21<00:08, 3040458.93tokens/s]
Shard 4:  74%|███████▍  | 74189884/100000000 [00:21<00:08, 2991448.44tokens/s]
Shard 4:  74%|███████▍  | 74498478/100000000 [00:22<00:08, 2944532.35tokens/s]
Shard 4:  75%|███████▍  | 74798683/100000000 [00:22<00:09, 2707094.73tokens/s]
Shard 4:  75%|███████▌  | 75100114/100000000 [00:22<00:08, 2774469.76tokens/s]
Shard 4:  75%|███████▌  | 75384074/100000000 [00:22<00:09, 2719492.73tokens/s]
Shard 4:  76%|███████▌  | 75686233/100000000 [00:22<00:08, 2779356.71tokens/s]
Shard 4:  76%|███████▌  | 75969558/100000000 [00:22<00:08, 2793840.77tokens/s]
Shard 4:  76%|███████▋  | 76251615/100000000 [00:22<00:09, 2581248.53tokens/s]
Shard 4:  77%|███████▋  | 76555752/100000000 [00:22<00:08, 2706131.88tokens/s]
Shard 4:  77%|███████▋  | 76830866/100000000 [00:22<00:08, 2649429.06tokens/s]
Shard 4:  77%|███████▋  | 77106620/100000000 [00:23<00:08, 2677691.96tokens/s]
Shard 4:  77%|███████▋  | 77376668/100000000 [00:23<00:08, 2572316.05tokens/s]
Shard 4:  78%|███████▊  | 77636653/100000000 [00:23<00:09, 2440481.81tokens/s]
Shard 4:  78%|███████▊  | 77884062/100000000 [00:23<00:09, 2443422.72tokens/s]
Shard 4:  78%|███████▊  | 78184565/100000000 [00:23<00:08, 2600648.69tokens/s]
Shard 4:  79%|███████▊  | 78564162/100000000 [00:23<00:07, 2932033.48tokens/s]
Shard 4:  79%|███████▉  | 78981301/100000000 [00:23<00:06, 3263869.72tokens/s]
Shard 4:  79%|███████▉  | 79310394/100000000 [00:23<00:06, 3142152.45tokens/s]
Shard 4:  80%|███████▉  | 79682803/100000000 [00:23<00:06, 3307501.11tokens/s]
Shard 4:  80%|████████  | 80019494/100000000 [00:23<00:06, 3316662.91tokens/s]
Shard 4:  80%|████████  | 80353341/100000000 [00:24<00:06, 3154532.94tokens/s]
Shard 4:  81%|████████  | 80671549/100000000 [00:24<00:06, 3049304.25tokens/s]
Shard 4:  81%|████████  | 80979301/100000000 [00:24<00:07, 2527971.36tokens/s]
Shard 4:  81%|████████  | 81247779/100000000 [00:24<00:07, 2456813.11tokens/s]
Shard 4:  82%|████████▏ | 81555905/100000000 [00:24<00:07, 2608778.29tokens/s]
Shard 4:  82%|████████▏ | 81904245/100000000 [00:24<00:06, 2841303.02tokens/s]
Shard 4:  82%|████████▏ | 82198267/100000000 [00:24<00:06, 2661037.32tokens/s]
Shard 4:  82%|████████▏ | 82475626/100000000 [00:24<00:06, 2565196.91tokens/s]
Shard 4:  83%|████████▎ | 82741174/100000000 [00:25<00:07, 2359117.48tokens/s]
Shard 4:  83%|████████▎ | 82983475/100000000 [00:25<00:07, 2349063.95tokens/s]
Shard 4:  83%|████████▎ | 83309428/100000000 [00:25<00:06, 2565185.44tokens/s]
Shard 4:  84%|████████▎ | 83570874/100000000 [00:25<00:06, 2460216.95tokens/s]
Shard 4:  84%|████████▍ | 83896435/100000000 [00:25<00:06, 2673651.48tokens/s]
Shard 4:  84%|████████▍ | 84168580/100000000 [00:25<00:06, 2529706.79tokens/s]
Shard 4:  84%|████████▍ | 84428828/100000000 [00:25<00:06, 2472749.46tokens/s]
Shard 4:  85%|████████▍ | 84724977/100000000 [00:25<00:05, 2606699.58tokens/s]
Shard 4:  85%|████████▍ | 84989552/100000000 [00:25<00:06, 2456825.49tokens/s]
Shard 4:  85%|████████▌ | 85238787/100000000 [00:26<00:06, 2179452.27tokens/s]
Shard 4:  85%|████████▌ | 85476187/100000000 [00:26<00:06, 2220360.89tokens/s]
Shard 4:  86%|████████▌ | 85792734/100000000 [00:26<00:05, 2463470.58tokens/s]
Shard 4:  86%|████████▌ | 86045640/100000000 [00:26<00:06, 2323361.14tokens/s]
Shard 4:  86%|████████▋ | 86283620/100000000 [00:26<00:06, 2227519.33tokens/s]
Shard 4:  87%|████████▋ | 86510422/100000000 [00:26<00:06, 2224008.11tokens/s]
Shard 4:  87%|████████▋ | 86741217/100000000 [00:26<00:05, 2239052.73tokens/s]
Shard 4:  87%|████████▋ | 86987533/100000000 [00:26<00:05, 2294441.51tokens/s]
Shard 4:  87%|████████▋ | 87218853/100000000 [00:26<00:05, 2224124.05tokens/s]
Shard 4:  87%|████████▋ | 87490444/100000000 [00:27<00:05, 2361651.64tokens/s]
Shard 4:  88%|████████▊ | 87849360/100000000 [00:27<00:04, 2707448.13tokens/s]
Shard 4:  88%|████████▊ | 88240333/100000000 [00:27<00:03, 3043668.30tokens/s]
Shard 4:  89%|████████▊ | 88571764/100000000 [00:27<00:03, 3119694.79tokens/s]
Shard 4:  89%|████████▉ | 88982767/100000000 [00:27<00:03, 3402242.19tokens/s]
Shard 4:  89%|████████▉ | 89337868/100000000 [00:27<00:03, 3417177.90tokens/s]
Shard 4:  90%|████████▉ | 89704789/100000000 [00:27<00:02, 3490035.35tokens/s]
Shard 4:  90%|█████████ | 90054911/100000000 [00:27<00:02, 3450492.99tokens/s]
Shard 4:  90%|█████████ | 90400853/100000000 [00:27<00:03, 3098624.55tokens/s]
Shard 4:  91%|█████████ | 90718455/100000000 [00:28<00:03, 2793405.74tokens/s]
Shard 4:  91%|█████████ | 91077087/100000000 [00:28<00:02, 2997179.41tokens/s]
Shard 4:  92%|█████████▏| 91579245/100000000 [00:28<00:02, 3538222.61tokens/s]
Shard 4:  92%|█████████▏| 92128716/100000000 [00:28<00:01, 4070523.08tokens/s]
Shard 4:  93%|█████████▎| 92593869/100000000 [00:28<00:01, 4230725.26tokens/s]
Shard 4:  93%|█████████▎| 93026511/100000000 [00:28<00:01, 4257447.97tokens/s]
Shard 4:  93%|█████████▎| 93459180/100000000 [00:28<00:01, 4225173.47tokens/s]
Shard 4:  94%|█████████▍| 93886550/100000000 [00:28<00:01, 4102035.87tokens/s]
Shard 4:  94%|█████████▍| 94403341/100000000 [00:28<00:01, 4372368.35tokens/s]
Shard 4:  95%|█████████▍| 94844051/100000000 [00:29<00:01, 4333879.12tokens/s]
Shard 4:  95%|█████████▌| 95346204/100000000 [00:29<00:01, 4476040.11tokens/s]
Shard 4:  96%|█████████▌| 95926567/100000000 [00:29<00:00, 4754692.34tokens/s]
Shard 4:  96%|█████████▋| 96403762/100000000 [00:29<00:00, 4209159.55tokens/s]
Shard 4:  97%|█████████▋| 96837528/100000000 [00:29<00:00, 3875827.39tokens/s]
Shard 4:  97%|█████████▋| 97236444/100000000 [00:29<00:00, 3783054.97tokens/s]
Shard 4:  98%|█████████▊| 97821652/100000000 [00:29<00:00, 4239108.20tokens/s]
Shard 4:  98%|█████████▊| 98314975/100000000 [00:29<00:00, 4401684.06tokens/s]
Shard 4:  99%|█████████▉| 98806083/100000000 [00:29<00:00, 4537277.52tokens/s]
Shard 4:  99%|█████████▉| 99271414/100000000 [00:30<00:00, 4439195.17tokens/s]
Shard 4: 100%|█████████▉| 99775940/100000000 [00:30<00:00, 4589496.31tokens/s]
Shard 4: 100%|█████████▉| 99998805/100000000 [00:38<00:00, 2627825.96tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000004.bin (200,001,024 bytes) in the gpt-2 format

Shard 5:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 5:   0%|          | 363850/100000000 [00:00<00:27, 3613039.99tokens/s]
Shard 5:   1%|          | 725216/100000000 [00:00<00:36, 2738214.86tokens/s]
Shard 5:   1%|          | 1012300/100000000 [00:00<00:40, 2456232.37tokens/s]
Shard 5:   1%|▏         | 1416875/100000000 [00:00<00:33, 2961129.23tokens/s]
Shard 5:   2%|▏         | 1796455/100000000 [00:00<00:30, 3217143.24tokens/s]
Shard 5:   2%|▏         | 2251327/100000000 [00:00<00:26, 3629588.06tokens/s]
Shard 5:   3%|▎         | 2628165/100000000 [00:00<00:28, 3443641.19tokens/s]
Shard 5:   3%|▎         | 3066395/100000000 [00:00<00:26, 3714976.83tokens/s]
Shard 5:   4%|▎         | 3555576/100000000 [00:01<00:23, 4049324.85tokens/s]
Shard 5:   4%|▍         | 3969274/100000000 [00:01<00:24, 3957586.68tokens/s]
Shard 5:   4%|▍         | 4389640/100000000 [00:01<00:23, 3992866.94tokens/s]
Shard 5:   5%|▍         | 4793280/100000000 [00:01<00:25, 3735534.06tokens/s]
Shard 5:   5%|▌         | 5420505/100000000 [00:01<00:21, 4438121.59tokens/s]
Shard 5:   6%|▌         | 5917798/100000000 [00:01<00:20, 4584872.94tokens/s]
Shard 5:   7%|▋         | 6517115/100000000 [00:01<00:18, 4963598.92tokens/s]
Shard 5:   7%|▋         | 7049381/100000000 [00:01<00:18, 5062626.14tokens/s]
Shard 5:   8%|▊         | 7561417/100000000 [00:01<00:21, 4325317.59tokens/s]
Shard 5:   8%|▊         | 8081711/100000000 [00:02<00:20, 4555994.61tokens/s]
Shard 5:   9%|▊         | 8556030/100000000 [00:02<00:20, 4468242.99tokens/s]
Shard 5:   9%|▉         | 9015940/100000000 [00:02<00:20, 4461199.44tokens/s]
Shard 5:   9%|▉         | 9472226/100000000 [00:02<00:21, 4137536.06tokens/s]
Shard 5:  10%|█         | 10006863/100000000 [00:02<00:20, 4450742.64tokens/s]
Shard 5:  10%|█         | 10462494/100000000 [00:02<00:22, 3937346.16tokens/s]
Shard 5:  11%|█         | 10878469/100000000 [00:02<00:22, 3989145.07tokens/s]
Shard 5:  11%|█▏        | 11289654/100000000 [00:02<00:23, 3763931.38tokens/s]
Shard 5:  12%|█▏        | 11760017/100000000 [00:02<00:22, 4004750.06tokens/s]
Shard 5:  12%|█▏        | 12170327/100000000 [00:03<00:25, 3467801.45tokens/s]
Shard 5:  13%|█▎        | 12538500/100000000 [00:03<00:25, 3391627.68tokens/s]
Shard 5:  13%|█▎        | 12889516/100000000 [00:03<00:26, 3308670.97tokens/s]
Shard 5:  13%|█▎        | 13370180/100000000 [00:03<00:23, 3701359.82tokens/s]
Shard 5:  14%|█▍        | 13751479/100000000 [00:03<00:25, 3398093.91tokens/s]
Shard 5:  14%|█▍        | 14110877/100000000 [00:03<00:25, 3351011.08tokens/s]
Shard 5:  15%|█▍        | 14514162/100000000 [00:03<00:24, 3530262.28tokens/s]
Shard 5:  15%|█▍        | 14875344/100000000 [00:03<00:24, 3430105.79tokens/s]
Shard 5:  15%|█▌        | 15311351/100000000 [00:03<00:22, 3682573.83tokens/s]
Shard 5:  16%|█▌        | 15688750/100000000 [00:04<00:22, 3708329.81tokens/s]
Shard 5:  16%|█▌        | 16121319/100000000 [00:04<00:21, 3866156.91tokens/s]
Shard 5:  17%|█▋        | 16557279/100000000 [00:04<00:20, 4007726.39tokens/s]
Shard 5:  17%|█▋        | 17018624/100000000 [00:04<00:20, 4137943.47tokens/s]
Shard 5:  17%|█▋        | 17434717/100000000 [00:04<00:21, 3926891.89tokens/s]
Shard 5:  18%|█▊        | 17982888/100000000 [00:04<00:18, 4366281.54tokens/s]
Shard 5:  18%|█▊        | 18424457/100000000 [00:04<00:20, 3982785.64tokens/s]
Shard 5:  19%|█▉        | 18832177/100000000 [00:04<00:20, 3906665.29tokens/s]
Shard 5:  19%|█▉        | 19229453/100000000 [00:04<00:21, 3674917.87tokens/s]
Shard 5:  20%|█▉        | 19604056/100000000 [00:05<00:24, 3335121.44tokens/s]
Shard 5:  20%|██        | 20064432/100000000 [00:05<00:21, 3661575.91tokens/s]
Shard 5:  20%|██        | 20441862/100000000 [00:05<00:23, 3421371.07tokens/s]
Shard 5:  21%|██        | 20962971/100000000 [00:05<00:20, 3886052.98tokens/s]
Shard 5:  22%|██▏       | 21581046/100000000 [00:05<00:17, 4497641.92tokens/s]
Shard 5:  22%|██▏       | 22048427/100000000 [00:05<00:17, 4525579.34tokens/s]
Shard 5:  23%|██▎       | 22581152/100000000 [00:05<00:16, 4746999.83tokens/s]
Shard 5:  23%|██▎       | 23360694/100000000 [00:05<00:13, 5618301.56tokens/s]
Shard 5:  24%|██▍       | 23931667/100000000 [00:06<00:17, 4358654.26tokens/s]
Shard 5:  25%|██▍       | 24623025/100000000 [00:06<00:15, 4973746.11tokens/s]
Shard 5:  25%|██▌       | 25169220/100000000 [00:06<00:14, 5092641.49tokens/s]
Shard 5:  26%|██▌       | 25720837/100000000 [00:06<00:14, 5172072.79tokens/s]
Shard 5:  26%|██▋       | 26263904/100000000 [00:06<00:16, 4420678.38tokens/s]
Shard 5:  27%|██▋       | 26743401/100000000 [00:06<00:16, 4327106.03tokens/s]
Shard 5:  27%|██▋       | 27200979/100000000 [00:06<00:17, 4183542.84tokens/s]
Shard 5:  28%|██▊       | 27637553/100000000 [00:06<00:18, 3938601.12tokens/s]
Shard 5:  28%|██▊       | 28045312/100000000 [00:07<00:18, 3893373.94tokens/s]
Shard 5:  28%|██▊       | 28450052/100000000 [00:07<00:18, 3932508.53tokens/s]
Shard 5:  29%|██▉       | 28849960/100000000 [00:07<00:19, 3617846.62tokens/s]
Shard 5:  29%|██▉       | 29228082/100000000 [00:07<00:19, 3559969.40tokens/s]
Shard 5:  30%|██▉       | 29589167/100000000 [00:07<00:19, 3535852.50tokens/s]
Shard 5:  30%|██▉       | 29989464/100000000 [00:07<00:19, 3648915.17tokens/s]
Shard 5:  31%|███       | 30509059/100000000 [00:07<00:17, 4077932.09tokens/s]
Shard 5:  31%|███       | 31080014/100000000 [00:07<00:15, 4530629.84tokens/s]
Shard 5:  32%|███▏      | 31538743/100000000 [00:07<00:15, 4360146.90tokens/s]
Shard 5:  32%|███▏      | 31980085/100000000 [00:07<00:15, 4371643.61tokens/s]
Shard 5:  32%|███▏      | 32426167/100000000 [00:08<00:16, 4110762.59tokens/s]
Shard 5:  33%|███▎      | 32974845/100000000 [00:08<00:14, 4482596.22tokens/s]
Shard 5:  33%|███▎      | 33431819/100000000 [00:08<00:16, 3944904.94tokens/s]
Shard 5:  34%|███▍      | 33878965/100000000 [00:08<00:16, 4082715.41tokens/s]
Shard 5:  34%|███▍      | 34446183/100000000 [00:08<00:14, 4489291.01tokens/s]
Shard 5:  35%|███▌      | 35013824/100000000 [00:08<00:13, 4808830.42tokens/s]
Shard 5:  36%|███▌      | 35506290/100000000 [00:08<00:13, 4738994.50tokens/s]
Shard 5:  36%|███▌      | 35988148/100000000 [00:08<00:14, 4454876.33tokens/s]
Shard 5:  36%|███▋      | 36442587/100000000 [00:09<00:15, 4063406.05tokens/s]
Shard 5:  37%|███▋      | 36889819/100000000 [00:09<00:15, 4148562.38tokens/s]
Shard 5:  37%|███▋      | 37478648/100000000 [00:09<00:13, 4614020.45tokens/s]
Shard 5:  38%|███▊      | 37950648/100000000 [00:09<00:14, 4412131.30tokens/s]
Shard 5:  38%|███▊      | 38456031/100000000 [00:09<00:13, 4555088.97tokens/s]
Shard 5:  39%|███▉      | 38920909/100000000 [00:09<00:13, 4365829.89tokens/s]
Shard 5:  39%|███▉      | 39363810/100000000 [00:09<00:14, 4151453.57tokens/s]
Shard 5:  40%|███▉      | 39993525/100000000 [00:09<00:12, 4731165.94tokens/s]
Shard 5:  40%|████      | 40476492/100000000 [00:09<00:14, 4246281.29tokens/s]
Shard 5:  41%|████      | 40915473/100000000 [00:10<00:14, 4206187.70tokens/s]
Shard 5:  41%|████▏     | 41387517/100000000 [00:10<00:13, 4342478.48tokens/s]
Shard 5:  42%|████▏     | 41831541/100000000 [00:10<00:14, 3950119.15tokens/s]
Shard 5:  42%|████▏     | 42350767/100000000 [00:10<00:13, 4254319.34tokens/s]
Shard 5:  43%|████▎     | 42789064/100000000 [00:10<00:13, 4105143.40tokens/s]
Shard 5:  43%|████▎     | 43208776/100000000 [00:10<00:13, 4069658.89tokens/s]
Shard 5:  44%|████▍     | 43816635/100000000 [00:10<00:12, 4611857.84tokens/s]
Shard 5:  44%|████▍     | 44363383/100000000 [00:10<00:11, 4849120.72tokens/s]
Shard 5:  45%|████▍     | 44855644/100000000 [00:10<00:11, 4660713.64tokens/s]
Shard 5:  45%|████▌     | 45329611/100000000 [00:11<00:12, 4498200.15tokens/s]
Shard 5:  46%|████▌     | 45823068/100000000 [00:11<00:11, 4619735.12tokens/s]
Shard 5:  46%|████▋     | 46289344/100000000 [00:11<00:12, 4349656.48tokens/s]
Shard 5:  47%|████▋     | 46730154/100000000 [00:11<00:13, 3881365.28tokens/s]
Shard 5:  47%|████▋     | 47133418/100000000 [00:11<00:13, 3910832.34tokens/s]
Shard 5:  48%|████▊     | 47534331/100000000 [00:11<00:14, 3582973.24tokens/s]
Shard 5:  48%|████▊     | 47946285/100000000 [00:11<00:13, 3722247.09tokens/s]
Shard 5:  48%|████▊     | 48327190/100000000 [00:11<00:14, 3568784.43tokens/s]
Shard 5:  49%|████▊     | 48690525/100000000 [00:11<00:16, 3202241.15tokens/s]
Shard 5:  49%|████▉     | 49157094/100000000 [00:12<00:14, 3570535.77tokens/s]
Shard 5:  50%|████▉     | 49527144/100000000 [00:12<00:14, 3369321.03tokens/s]
Shard 5:  50%|████▉     | 49874534/100000000 [00:12<00:14, 3369444.51tokens/s]
Shard 5:  50%|█████     | 50278425/100000000 [00:12<00:14, 3549374.82tokens/s]
Shard 5:  51%|█████     | 50883848/100000000 [00:12<00:11, 4223418.14tokens/s]
Shard 5:  51%|█████▏    | 51314996/100000000 [00:12<00:12, 4024524.26tokens/s]
Shard 5:  52%|█████▏    | 51896347/100000000 [00:12<00:10, 4509244.31tokens/s]
Shard 5:  53%|█████▎    | 52561055/100000000 [00:12<00:09, 5102127.69tokens/s]
Shard 5:  53%|█████▎    | 53090767/100000000 [00:12<00:09, 4796216.23tokens/s]
Shard 5:  54%|█████▎    | 53580301/100000000 [00:13<00:10, 4342442.52tokens/s]
Shard 5:  54%|█████▍    | 54028191/100000000 [00:13<00:10, 4194025.33tokens/s]
Shard 5:  54%|█████▍    | 54457415/100000000 [00:13<00:11, 4003156.02tokens/s]
Shard 5:  55%|█████▍    | 54864557/100000000 [00:13<00:11, 3876544.32tokens/s]
Shard 5:  55%|█████▌    | 55371500/100000000 [00:13<00:10, 4176813.01tokens/s]
Shard 5:  56%|█████▌    | 55796509/100000000 [00:13<00:10, 4172337.43tokens/s]
Shard 5:  56%|█████▌    | 56233675/100000000 [00:13<00:10, 4223247.10tokens/s]
Shard 5:  57%|█████▋    | 56661807/100000000 [00:13<00:10, 4230467.14tokens/s]
Shard 5:  57%|█████▋    | 57087714/100000000 [00:14<00:11, 3585720.54tokens/s]
Shard 5:  57%|█████▋    | 57464465/100000000 [00:14<00:13, 3183936.24tokens/s]
Shard 5:  58%|█████▊    | 57801996/100000000 [00:14<00:13, 3172765.25tokens/s]
Shard 5:  58%|█████▊    | 58159590/100000000 [00:14<00:12, 3263109.23tokens/s]
Shard 5:  58%|█████▊    | 58496388/100000000 [00:14<00:13, 3177288.48tokens/s]
Shard 5:  59%|█████▉    | 58821172/100000000 [00:14<00:13, 2987950.82tokens/s]
Shard 5:  59%|█████▉    | 59147854/100000000 [00:14<00:13, 3061261.44tokens/s]
Shard 5:  60%|█████▉    | 59601689/100000000 [00:14<00:11, 3462421.76tokens/s]
Shard 5:  60%|█████▉    | 59955325/100000000 [00:14<00:13, 2992759.68tokens/s]
Shard 5:  60%|██████    | 60379155/100000000 [00:15<00:12, 3299007.08tokens/s]
Shard 5:  61%|██████    | 61100293/100000000 [00:15<00:08, 4333303.75tokens/s]
Shard 5:  62%|██████▏   | 61674257/100000000 [00:15<00:08, 4671807.77tokens/s]
Shard 5:  62%|██████▏   | 62159323/100000000 [00:15<00:08, 4395014.64tokens/s]
Shard 5:  63%|██████▎   | 62619787/100000000 [00:15<00:08, 4316490.16tokens/s]
Shard 5:  63%|██████▎   | 63061834/100000000 [00:15<00:09, 3749884.12tokens/s]
Shard 5:  63%|██████▎   | 63457002/100000000 [00:15<00:11, 3147305.79tokens/s]
Shard 5:  64%|██████▍   | 63799097/100000000 [00:15<00:11, 3195124.61tokens/s]
Shard 5:  64%|██████▍   | 64138485/100000000 [00:16<00:11, 3053616.20tokens/s]
Shard 5:  64%|██████▍   | 64457043/100000000 [00:16<00:11, 3083451.80tokens/s]
Shard 5:  65%|██████▍   | 64775475/100000000 [00:16<00:11, 2943521.51tokens/s]
Shard 5:  65%|██████▌   | 65080573/100000000 [00:16<00:11, 2961834.27tokens/s]
Shard 5:  65%|██████▌   | 65403862/100000000 [00:16<00:11, 3026002.70tokens/s]
Shard 5:  66%|██████▌   | 65719711/100000000 [00:16<00:11, 3042612.52tokens/s]
Shard 5:  66%|██████▌   | 66032788/100000000 [00:16<00:11, 2832762.68tokens/s]
Shard 5:  66%|██████▋   | 66396562/100000000 [00:16<00:11, 3008295.34tokens/s]
Shard 5:  67%|██████▋   | 66820886/100000000 [00:16<00:09, 3335677.34tokens/s]
Shard 5:  67%|██████▋   | 67159753/100000000 [00:17<00:10, 3059807.51tokens/s]
Shard 5:  67%|██████▋   | 67476057/100000000 [00:17<00:11, 2896856.52tokens/s]
Shard 5:  68%|██████▊   | 67900073/100000000 [00:17<00:09, 3240061.39tokens/s]
Shard 5:  68%|██████▊   | 68232807/100000000 [00:17<00:10, 3111640.42tokens/s]
Shard 5:  69%|██████▊   | 68552306/100000000 [00:17<00:10, 3130289.56tokens/s]
Shard 5:  69%|██████▉   | 69203350/100000000 [00:17<00:07, 4062470.59tokens/s]
Shard 5:  70%|██████▉   | 69619453/100000000 [00:17<00:07, 4042468.14tokens/s]
Shard 5:  70%|███████   | 70032105/100000000 [00:17<00:07, 4044233.04tokens/s]
Shard 5:  70%|███████   | 70480463/100000000 [00:17<00:07, 4156608.45tokens/s]
Shard 5:  71%|███████   | 70900477/100000000 [00:18<00:07, 3832050.64tokens/s]
Shard 5:  71%|███████▏  | 71291202/100000000 [00:18<00:08, 3307131.63tokens/s]
Shard 5:  72%|███████▏  | 71688394/100000000 [00:18<00:08, 3470576.71tokens/s]
Shard 5:  72%|███████▏  | 72167425/100000000 [00:18<00:07, 3814199.10tokens/s]
Shard 5:  73%|███████▎  | 72563804/100000000 [00:18<00:07, 3774016.28tokens/s]
Shard 5:  73%|███████▎  | 72950967/100000000 [00:18<00:07, 3581011.87tokens/s]
Shard 5:  73%|███████▎  | 73317330/100000000 [00:18<00:07, 3355374.82tokens/s]
Shard 5:  74%|███████▎  | 73662384/100000000 [00:18<00:08, 3279368.28tokens/s]
Shard 5:  74%|███████▍  | 73995285/100000000 [00:19<00:08, 3077675.65tokens/s]
Shard 5:  74%|███████▍  | 74388769/100000000 [00:19<00:07, 3284908.27tokens/s]
Shard 5:  75%|███████▍  | 74724171/100000000 [00:19<00:08, 2941761.81tokens/s]
Shard 5:  75%|███████▌  | 75027548/100000000 [00:19<00:09, 2728442.84tokens/s]
Shard 5:  75%|███████▌  | 75308348/100000000 [00:19<00:09, 2714434.78tokens/s]
Shard 5:  76%|███████▌  | 75584958/100000000 [00:19<00:09, 2652099.68tokens/s]
Shard 5:  76%|███████▌  | 75893095/100000000 [00:19<00:08, 2758974.38tokens/s]
Shard 5:  76%|███████▌  | 76172584/100000000 [00:19<00:08, 2687187.14tokens/s]
Shard 5:  77%|███████▋  | 76587098/100000000 [00:19<00:07, 3064031.70tokens/s]
Shard 5:  77%|███████▋  | 76898941/100000000 [00:20<00:08, 2838234.00tokens/s]
Shard 5:  77%|███████▋  | 77256031/100000000 [00:20<00:07, 3030833.91tokens/s]
Shard 5:  78%|███████▊  | 77564788/100000000 [00:20<00:07, 2881160.26tokens/s]
Shard 5:  78%|███████▊  | 77955056/100000000 [00:20<00:06, 3158457.67tokens/s]
Shard 5:  78%|███████▊  | 78277713/100000000 [00:20<00:07, 3066738.70tokens/s]
Shard 5:  79%|███████▊  | 78589487/100000000 [00:20<00:07, 2832243.91tokens/s]
Shard 5:  79%|███████▉  | 78896427/100000000 [00:20<00:07, 2893629.49tokens/s]
Shard 5:  79%|███████▉  | 79197294/100000000 [00:20<00:07, 2702323.94tokens/s]
Shard 5:  80%|███████▉  | 79825637/100000000 [00:20<00:05, 3633067.78tokens/s]
Shard 5:  80%|████████  | 80431664/100000000 [00:21<00:04, 4299016.44tokens/s]
Shard 5:  81%|████████  | 80876992/100000000 [00:21<00:04, 4312746.22tokens/s]
Shard 5:  81%|████████▏ | 81320212/100000000 [00:21<00:04, 4082573.50tokens/s]
Shard 5:  82%|████████▏ | 81742310/100000000 [00:21<00:04, 3794938.94tokens/s]
Shard 5:  82%|████████▏ | 82131779/100000000 [00:21<00:04, 3770003.83tokens/s]
Shard 5:  83%|████████▎ | 82535910/100000000 [00:21<00:04, 3831875.60tokens/s]
Shard 5:  83%|████████▎ | 83034089/100000000 [00:21<00:04, 4146809.08tokens/s]
Shard 5:  84%|████████▎ | 83595733/100000000 [00:21<00:03, 4556283.98tokens/s]
Shard 5:  84%|████████▍ | 84057701/100000000 [00:21<00:03, 4376370.01tokens/s]
Shard 5:  85%|████████▍ | 84500666/100000000 [00:22<00:03, 4378096.47tokens/s]
Shard 5:  85%|████████▍ | 84942155/100000000 [00:22<00:03, 4174530.07tokens/s]
Shard 5:  85%|████████▌ | 85363689/100000000 [00:22<00:03, 4122222.94tokens/s]
Shard 5:  86%|████████▌ | 85798562/100000000 [00:22<00:04, 3397691.59tokens/s]
Shard 5:  86%|████████▌ | 86161213/100000000 [00:22<00:04, 2991821.04tokens/s]
Shard 5:  87%|████████▋ | 86750026/100000000 [00:22<00:03, 3667246.46tokens/s]
Shard 5:  87%|████████▋ | 87151345/100000000 [00:22<00:03, 3360281.25tokens/s]
Shard 5:  88%|████████▊ | 87539575/100000000 [00:22<00:03, 3461351.77tokens/s]
Shard 5:  88%|████████▊ | 87939145/100000000 [00:23<00:03, 3589473.28tokens/s]
Shard 5:  88%|████████▊ | 88314758/100000000 [00:23<00:03, 3382696.10tokens/s]
Shard 5:  89%|████████▊ | 88667276/100000000 [00:23<00:03, 3239444.59tokens/s]
Shard 5:  89%|████████▉ | 89001000/100000000 [00:23<00:03, 3144905.63tokens/s]
Shard 5:  89%|████████▉ | 89321659/100000000 [00:23<00:03, 2862922.83tokens/s]
Shard 5:  90%|████████▉ | 89685813/100000000 [00:23<00:03, 3055644.49tokens/s]
Shard 5:  90%|████████▉ | 89999972/100000000 [00:23<00:03, 2935028.17tokens/s]
Shard 5:  90%|█████████ | 90301742/100000000 [00:23<00:03, 2685486.22tokens/s]
Shard 5:  91%|█████████ | 90586880/100000000 [00:24<00:03, 2721802.27tokens/s]
Shard 5:  91%|█████████ | 90895869/100000000 [00:24<00:03, 2819407.87tokens/s]
Shard 5:  91%|█████████ | 91183272/100000000 [00:24<00:03, 2752715.17tokens/s]
Shard 5:  91%|█████████▏| 91471648/100000000 [00:24<00:03, 2757927.81tokens/s]
Shard 5:  92%|█████████▏| 92008116/100000000 [00:24<00:02, 3488037.58tokens/s]
Shard 5:  92%|█████████▏| 92491243/100000000 [00:24<00:01, 3855810.13tokens/s]
Shard 5:  93%|█████████▎| 92882506/100000000 [00:24<00:01, 3685016.00tokens/s]
Shard 5:  93%|█████████▎| 93256611/100000000 [00:24<00:01, 3474538.39tokens/s]
Shard 5:  94%|█████████▎| 93736527/100000000 [00:24<00:01, 3829514.87tokens/s]
Shard 5:  94%|█████████▍| 94126943/100000000 [00:25<00:01, 3699501.37tokens/s]
Shard 5:  95%|█████████▍| 94613666/100000000 [00:25<00:01, 4023446.27tokens/s]
Shard 5:  95%|█████████▌| 95108206/100000000 [00:25<00:01, 4281466.31tokens/s]
Shard 5:  96%|█████████▌| 95542090/100000000 [00:25<00:01, 4159208.62tokens/s]
Shard 5:  96%|█████████▌| 96094973/100000000 [00:25<00:00, 4547109.48tokens/s]
Shard 5:  97%|█████████▋| 96581037/100000000 [00:25<00:00, 4635280.01tokens/s]
Shard 5:  97%|█████████▋| 97056935/100000000 [00:25<00:00, 4433822.23tokens/s]
Shard 5:  98%|█████████▊| 97506462/100000000 [00:25<00:00, 4158151.53tokens/s]
Shard 5:  98%|█████████▊| 97928184/100000000 [00:25<00:00, 3604909.20tokens/s]
Shard 5:  98%|█████████▊| 98306251/100000000 [00:26<00:00, 3294030.19tokens/s]
Shard 5: 100%|█████████▉| 99626040/100000000 [00:26<00:00, 5716161.02tokens/s]
Shard 5: 100%|█████████▉| 99996948/100000000 [00:34<00:00, 2938260.37tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000005.bin (200,001,024 bytes) in the gpt-2 format

Shard 6:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 6:   0%|          | 182764/100000000 [00:00<00:55, 1782765.51tokens/s]
Shard 6:   0%|          | 362606/100000000 [00:00<01:01, 1617726.44tokens/s]
Shard 6:   1%|          | 584644/100000000 [00:00<00:53, 1868144.64tokens/s]
Shard 6:   1%|          | 836738/100000000 [00:00<00:47, 2102047.35tokens/s]
Shard 6:   1%|          | 1052760/100000000 [00:00<00:47, 2084679.35tokens/s]
Shard 6:   1%|▏         | 1366301/100000000 [00:00<00:40, 2413052.98tokens/s]
Shard 6:   2%|▏         | 1609033/100000000 [00:00<00:42, 2318286.00tokens/s]
Shard 6:   2%|▏         | 1928487/100000000 [00:00<00:38, 2574487.30tokens/s]
Shard 6:   2%|▏         | 2188780/100000000 [00:00<00:40, 2434704.45tokens/s]
Shard 6:   2%|▏         | 2435089/100000000 [00:01<00:45, 2165365.53tokens/s]
Shard 6:   3%|▎         | 2737700/100000000 [00:01<00:41, 2367118.36tokens/s]
Shard 6:   3%|▎         | 3011204/100000000 [00:01<00:39, 2464285.78tokens/s]
Shard 6:   3%|▎         | 3277825/100000000 [00:01<00:38, 2520888.08tokens/s]
Shard 6:   4%|▎         | 3706811/100000000 [00:01<00:31, 3015292.97tokens/s]
Shard 6:   4%|▍         | 4013674/100000000 [00:01<00:35, 2717135.80tokens/s]
Shard 6:   4%|▍         | 4331675/100000000 [00:01<00:33, 2832935.28tokens/s]
Shard 6:   5%|▍         | 4632422/100000000 [00:01<00:33, 2855598.48tokens/s]
Shard 6:   5%|▌         | 5086680/100000000 [00:01<00:28, 3323988.55tokens/s]
Shard 6:   5%|▌         | 5425605/100000000 [00:02<00:28, 3321155.07tokens/s]
Shard 6:   6%|▌         | 5795453/100000000 [00:02<00:27, 3418303.76tokens/s]
Shard 6:   6%|▋         | 6320957/100000000 [00:02<00:23, 3940021.23tokens/s]
Shard 6:   7%|▋         | 6720233/100000000 [00:02<00:26, 3515912.68tokens/s]
Shard 6:   7%|▋         | 7082433/100000000 [00:02<00:26, 3507084.30tokens/s]
Shard 6:   7%|▋         | 7484394/100000000 [00:02<00:25, 3647198.59tokens/s]
Shard 6:   8%|▊         | 7855901/100000000 [00:02<00:28, 3265880.34tokens/s]
Shard 6:   8%|▊         | 8202186/100000000 [00:02<00:29, 3068040.61tokens/s]
Shard 6:   9%|▊         | 8648956/100000000 [00:02<00:27, 3369372.10tokens/s]
Shard 6:   9%|▉         | 9147991/100000000 [00:03<00:23, 3790205.67tokens/s]
Shard 6:  10%|▉         | 9649353/100000000 [00:03<00:22, 4083917.20tokens/s]
Shard 6:  10%|█         | 10067690/100000000 [00:03<00:25, 3475821.85tokens/s]
Shard 6:  11%|█         | 10509646/100000000 [00:03<00:24, 3689411.68tokens/s]
Shard 6:  11%|█         | 10898311/100000000 [00:03<00:24, 3595841.34tokens/s]
Shard 6:  11%|█▏        | 11270753/100000000 [00:03<00:24, 3610162.41tokens/s]
Shard 6:  12%|█▏        | 11642085/100000000 [00:03<00:25, 3429676.34tokens/s]
Shard 6:  12%|█▏        | 11992538/100000000 [00:03<00:26, 3342509.50tokens/s]
Shard 6:  12%|█▏        | 12427557/100000000 [00:04<00:24, 3614452.11tokens/s]
Shard 6:  13%|█▎        | 12804730/100000000 [00:04<00:23, 3643901.45tokens/s]
Shard 6:  13%|█▎        | 13174819/100000000 [00:04<00:28, 3046975.39tokens/s]
Shard 6:  13%|█▎        | 13498109/100000000 [00:04<00:29, 2889657.92tokens/s]
Shard 6:  14%|█▍        | 13870189/100000000 [00:04<00:27, 3094959.46tokens/s]
Shard 6:  14%|█▍        | 14399361/100000000 [00:04<00:23, 3665209.39tokens/s]
Shard 6:  15%|█▍        | 14969616/100000000 [00:04<00:20, 4197813.35tokens/s]
Shard 6:  15%|█▌        | 15406003/100000000 [00:04<00:22, 3717678.01tokens/s]
Shard 6:  16%|█▌        | 15799001/100000000 [00:05<00:24, 3507893.96tokens/s]
Shard 6:  16%|█▋        | 16300762/100000000 [00:05<00:21, 3894711.73tokens/s]
Shard 6:  17%|█▋        | 16707812/100000000 [00:05<00:22, 3629773.46tokens/s]
Shard 6:  17%|█▋        | 17084797/100000000 [00:05<00:25, 3279103.68tokens/s]
Shard 6:  17%|█▋        | 17426851/100000000 [00:05<00:28, 2944281.09tokens/s]
Shard 6:  18%|█▊        | 17747509/100000000 [00:05<00:27, 2990417.64tokens/s]
Shard 6:  18%|█▊        | 18217682/100000000 [00:05<00:23, 3420422.44tokens/s]
Shard 6:  19%|█▊        | 18622232/100000000 [00:05<00:22, 3572470.54tokens/s]
Shard 6:  19%|█▉        | 18999351/100000000 [00:05<00:23, 3501633.25tokens/s]
Shard 6:  20%|█▉        | 19531701/100000000 [00:06<00:20, 3982055.53tokens/s]
Shard 6:  20%|█▉        | 19939682/100000000 [00:06<00:22, 3525477.63tokens/s]
Shard 6:  20%|██        | 20310152/100000000 [00:06<00:24, 3232502.47tokens/s]
Shard 6:  21%|██        | 20699085/100000000 [00:06<00:23, 3392566.67tokens/s]
Shard 6:  21%|██▏       | 21438748/100000000 [00:06<00:17, 4439731.36tokens/s]
Shard 6:  22%|██▏       | 21904622/100000000 [00:06<00:18, 4262404.89tokens/s]
Shard 6:  22%|██▏       | 22349328/100000000 [00:06<00:23, 3349814.35tokens/s]
Shard 6:  23%|██▎       | 22723611/100000000 [00:07<00:26, 2939168.91tokens/s]
Shard 6:  23%|██▎       | 23050193/100000000 [00:07<00:26, 2898827.41tokens/s]
Shard 6:  23%|██▎       | 23362069/100000000 [00:07<00:26, 2861897.74tokens/s]
Shard 6:  24%|██▎       | 23663168/100000000 [00:07<00:28, 2714479.39tokens/s]
Shard 6:  24%|██▍       | 23944771/100000000 [00:07<00:27, 2730313.71tokens/s]
Shard 6:  24%|██▍       | 24228046/100000000 [00:07<00:31, 2410840.09tokens/s]
Shard 6:  25%|██▍       | 24538715/100000000 [00:07<00:29, 2580840.87tokens/s]
Shard 6:  25%|██▍       | 24945096/100000000 [00:07<00:25, 2961865.88tokens/s]
Shard 6:  25%|██▌       | 25256472/100000000 [00:08<00:28, 2588248.86tokens/s]
Shard 6:  26%|██▌       | 25532998/100000000 [00:08<00:31, 2366571.85tokens/s]
Shard 6:  26%|██▌       | 25784268/100000000 [00:08<00:33, 2213465.47tokens/s]
Shard 6:  26%|██▌       | 26016328/100000000 [00:08<00:33, 2204296.31tokens/s]
Shard 6:  26%|██▋       | 26278084/100000000 [00:08<00:31, 2306631.00tokens/s]
Shard 6:  27%|██▋       | 26515721/100000000 [00:08<00:32, 2281948.22tokens/s]
Shard 6:  27%|██▋       | 26749140/100000000 [00:08<00:34, 2152556.50tokens/s]
Shard 6:  27%|██▋       | 27007532/100000000 [00:08<00:32, 2255236.48tokens/s]
Shard 6:  27%|██▋       | 27237068/100000000 [00:08<00:32, 2228731.05tokens/s]
Shard 6:  27%|██▋       | 27462915/100000000 [00:09<00:34, 2109278.11tokens/s]
Shard 6:  28%|██▊       | 27706747/100000000 [00:09<00:32, 2192916.29tokens/s]
Shard 6:  28%|██▊       | 27928641/100000000 [00:09<00:34, 2098576.26tokens/s]
Shard 6:  28%|██▊       | 28141545/100000000 [00:09<00:36, 1992198.36tokens/s]
Shard 6:  28%|██▊       | 28382902/100000000 [00:09<00:34, 2092441.75tokens/s]
Shard 6:  29%|██▊       | 28594938/100000000 [00:09<00:34, 2088754.48tokens/s]
Shard 6:  29%|██▉       | 28806419/100000000 [00:09<00:34, 2067954.84tokens/s]
Shard 6:  29%|██▉       | 29014377/100000000 [00:09<00:36, 1957737.31tokens/s]
Shard 6:  29%|██▉       | 29291989/100000000 [00:09<00:32, 2182829.30tokens/s]
Shard 6:  30%|██▉       | 29542562/100000000 [00:10<00:30, 2274295.93tokens/s]
Shard 6:  30%|██▉       | 29829508/100000000 [00:10<00:29, 2401714.56tokens/s]
Shard 6:  30%|███       | 30120596/100000000 [00:10<00:27, 2544863.66tokens/s]
Shard 6:  31%|███       | 30528648/100000000 [00:10<00:23, 2942353.79tokens/s]
Shard 6:  31%|███       | 30823879/100000000 [00:10<00:24, 2872642.31tokens/s]
Shard 6:  31%|███       | 31112355/100000000 [00:10<00:24, 2811853.07tokens/s]
Shard 6:  31%|███▏      | 31395699/100000000 [00:10<00:24, 2814808.71tokens/s]
Shard 6:  32%|███▏      | 31710099/100000000 [00:10<00:23, 2901374.08tokens/s]
Shard 6:  32%|███▏      | 32000749/100000000 [00:10<00:26, 2548359.59tokens/s]
Shard 6:  32%|███▏      | 32411637/100000000 [00:11<00:23, 2934773.43tokens/s]
Shard 6:  33%|███▎      | 32729580/100000000 [00:11<00:22, 2982809.14tokens/s]
Shard 6:  33%|███▎      | 33089871/100000000 [00:11<00:21, 3147747.42tokens/s]
Shard 6:  34%|███▎      | 33517962/100000000 [00:11<00:19, 3461637.75tokens/s]
Shard 6:  34%|███▍      | 33869626/100000000 [00:11<00:19, 3422877.84tokens/s]
Shard 6:  34%|███▍      | 34215737/100000000 [00:11<00:20, 3181591.15tokens/s]
Shard 6:  35%|███▍      | 34685577/100000000 [00:11<00:18, 3595907.14tokens/s]
Shard 6:  35%|███▌      | 35156362/100000000 [00:11<00:16, 3887939.27tokens/s]
Shard 6:  36%|███▌      | 35553305/100000000 [00:11<00:18, 3469830.88tokens/s]
Shard 6:  36%|███▌      | 35912755/100000000 [00:12<00:20, 3052005.91tokens/s]
Shard 6:  36%|███▋      | 36301809/100000000 [00:12<00:19, 3260542.43tokens/s]
Shard 6:  37%|███▋      | 36695307/100000000 [00:12<00:18, 3420936.91tokens/s]
Shard 6:  37%|███▋      | 37076666/100000000 [00:12<00:17, 3527627.41tokens/s]
Shard 6:  38%|███▊      | 37502858/100000000 [00:12<00:16, 3731780.46tokens/s]
Shard 6:  38%|███▊      | 37919910/100000000 [00:12<00:16, 3856815.31tokens/s]
Shard 6:  38%|███▊      | 38312736/100000000 [00:12<00:19, 3222886.19tokens/s]
Shard 6:  39%|███▊      | 38656651/100000000 [00:12<00:18, 3243578.06tokens/s]
Shard 6:  39%|███▉      | 39029772/100000000 [00:12<00:18, 3363441.70tokens/s]
Shard 6:  39%|███▉      | 39434446/100000000 [00:13<00:17, 3548092.70tokens/s]
Shard 6:  40%|███▉      | 39799665/100000000 [00:13<00:19, 3070600.35tokens/s]
Shard 6:  40%|████      | 40195164/100000000 [00:13<00:18, 3285092.14tokens/s]
Shard 6:  41%|████      | 40539799/100000000 [00:13<00:19, 3109963.29tokens/s]
Shard 6:  41%|████      | 40863829/100000000 [00:13<00:20, 2881276.29tokens/s]
Shard 6:  41%|████      | 41220976/100000000 [00:13<00:19, 3050331.14tokens/s]
Shard 6:  42%|████▏     | 41554004/100000000 [00:13<00:18, 3124524.82tokens/s]
Shard 6:  42%|████▏     | 41874642/100000000 [00:13<00:19, 3028823.94tokens/s]
Shard 6:  42%|████▏     | 42185241/100000000 [00:13<00:19, 2892852.75tokens/s]
Shard 6:  43%|████▎     | 42560510/100000000 [00:14<00:18, 3126437.12tokens/s]
Shard 6:  43%|████▎     | 42878715/100000000 [00:14<00:18, 3119583.42tokens/s]
Shard 6:  43%|████▎     | 43229809/100000000 [00:14<00:17, 3225471.50tokens/s]
Shard 6:  44%|████▎     | 43556153/100000000 [00:14<00:19, 2930679.11tokens/s]
Shard 6:  44%|████▍     | 43892351/100000000 [00:14<00:18, 2994667.82tokens/s]
Shard 6:  44%|████▍     | 44231590/100000000 [00:14<00:17, 3104315.31tokens/s]
Shard 6:  45%|████▍     | 44668018/100000000 [00:14<00:16, 3458040.08tokens/s]
Shard 6:  45%|████▌     | 45040759/100000000 [00:14<00:15, 3523413.61tokens/s]
Shard 6:  46%|████▌     | 45538537/100000000 [00:14<00:13, 3938117.71tokens/s]
Shard 6:  46%|████▌     | 45936892/100000000 [00:15<00:13, 3888710.47tokens/s]
Shard 6:  46%|████▋     | 46330456/100000000 [00:15<00:13, 3885197.55tokens/s]
Shard 6:  47%|████▋     | 47292796/100000000 [00:15<00:09, 5556526.50tokens/s]
Shard 6:  48%|████▊     | 47852909/100000000 [00:15<00:10, 4893065.89tokens/s]
Shard 6:  48%|████▊     | 48359397/100000000 [00:15<00:12, 4187457.72tokens/s]
Shard 6:  49%|████▉     | 48805413/100000000 [00:15<00:14, 3478308.58tokens/s]
Shard 6:  49%|████▉     | 49196632/100000000 [00:15<00:15, 3382301.81tokens/s]
Shard 6:  50%|████▉     | 49559789/100000000 [00:16<00:15, 3165437.06tokens/s]
Shard 6:  50%|█████     | 50145464/100000000 [00:16<00:13, 3794222.19tokens/s]
Shard 6:  51%|█████     | 50554010/100000000 [00:16<00:13, 3798908.64tokens/s]
Shard 6:  51%|█████     | 50955265/100000000 [00:16<00:13, 3729405.07tokens/s]
Shard 6:  51%|█████▏    | 51342043/100000000 [00:16<00:13, 3589907.31tokens/s]
Shard 6:  52%|█████▏    | 51710766/100000000 [00:16<00:13, 3534476.56tokens/s]
Shard 6:  52%|█████▏    | 52070672/100000000 [00:16<00:14, 3365347.95tokens/s]
Shard 6:  52%|█████▏    | 52426002/100000000 [00:16<00:13, 3411583.15tokens/s]
Shard 6:  53%|█████▎    | 52783802/100000000 [00:16<00:13, 3453997.33tokens/s]
Shard 6:  53%|█████▎    | 53219538/100000000 [00:16<00:12, 3698023.56tokens/s]
Shard 6:  54%|█████▍    | 53881218/100000000 [00:17<00:10, 4534646.86tokens/s]
Shard 6:  54%|█████▍    | 54340922/100000000 [00:17<00:13, 3472954.34tokens/s]
Shard 6:  55%|█████▍    | 54729449/100000000 [00:17<00:13, 3431477.30tokens/s]
Shard 6:  55%|█████▌    | 55208220/100000000 [00:17<00:12, 3729578.74tokens/s]
Shard 6:  56%|█████▌    | 55635611/100000000 [00:17<00:11, 3863766.13tokens/s]
Shard 6:  56%|█████▌    | 56041749/100000000 [00:17<00:11, 3813169.51tokens/s]
Shard 6:  57%|█████▋    | 56603522/100000000 [00:17<00:10, 4274876.20tokens/s]
Shard 6:  57%|█████▋    | 57044066/100000000 [00:17<00:10, 3993563.89tokens/s]
Shard 6:  57%|█████▋    | 57455326/100000000 [00:18<00:11, 3756486.94tokens/s]
Shard 6:  58%|█████▊    | 57842718/100000000 [00:18<00:11, 3704709.35tokens/s]
Shard 6:  58%|█████▊    | 58219727/100000000 [00:18<00:11, 3695341.49tokens/s]
Shard 6:  59%|█████▊    | 58729037/100000000 [00:18<00:10, 4082041.20tokens/s]
Shard 6:  59%|█████▉    | 59227912/100000000 [00:18<00:09, 4329374.50tokens/s]
Shard 6:  61%|██████    | 60656521/100000000 [00:18<00:05, 7190917.45tokens/s]
Shard 6:  61%|██████▏   | 61391982/100000000 [00:18<00:08, 4556538.33tokens/s]
Shard 6:  62%|██████▏   | 61979857/100000000 [00:19<00:08, 4564176.93tokens/s]
Shard 6:  63%|██████▎   | 62528870/100000000 [00:19<00:08, 4533376.31tokens/s]
Shard 6:  63%|██████▎   | 63046442/100000000 [00:19<00:08, 4328665.90tokens/s]
Shard 6:  64%|██████▎   | 63523733/100000000 [00:19<00:10, 3561217.86tokens/s]
Shard 6:  64%|██████▍   | 63927729/100000000 [00:19<00:10, 3348329.69tokens/s]
Shard 6:  64%|██████▍   | 64295323/100000000 [00:19<00:11, 3072969.16tokens/s]
Shard 6:  65%|██████▍   | 64642103/100000000 [00:19<00:11, 3148566.07tokens/s]
Shard 6:  65%|██████▍   | 64975544/100000000 [00:19<00:11, 3113206.51tokens/s]
Shard 6:  65%|██████▌   | 65341945/100000000 [00:20<00:10, 3244972.36tokens/s]
Shard 6:  66%|██████▌   | 65724875/100000000 [00:20<00:10, 3397413.07tokens/s]
Shard 6:  66%|██████▌   | 66085243/100000000 [00:20<00:09, 3449711.71tokens/s]
Shard 6:  67%|██████▋   | 66682671/100000000 [00:20<00:08, 4147810.65tokens/s]
Shard 6:  67%|██████▋   | 67109913/100000000 [00:20<00:08, 4078535.36tokens/s]
Shard 6:  68%|██████▊   | 67555498/100000000 [00:20<00:07, 4170705.77tokens/s]
Shard 6:  68%|██████▊   | 67978438/100000000 [00:20<00:08, 3895911.77tokens/s]
Shard 6:  68%|██████▊   | 68379594/100000000 [00:20<00:09, 3234281.85tokens/s]
Shard 6:  69%|██████▊   | 68725078/100000000 [00:21<00:09, 3158355.71tokens/s]
Shard 6:  69%|██████▉   | 69118169/100000000 [00:21<00:09, 3309613.61tokens/s]
Shard 6:  69%|██████▉   | 69462815/100000000 [00:21<00:09, 3163190.35tokens/s]
Shard 6:  70%|██████▉   | 69808940/100000000 [00:21<00:09, 3233332.05tokens/s]
Shard 6:  70%|███████   | 70139725/100000000 [00:21<00:09, 3112651.42tokens/s]
Shard 6:  70%|███████   | 70458031/100000000 [00:21<00:09, 2977357.70tokens/s]
Shard 6:  71%|███████   | 70759810/100000000 [00:21<00:09, 2985755.94tokens/s]
Shard 6:  71%|███████   | 71061206/100000000 [00:21<00:10, 2879545.58tokens/s]
Shard 6:  71%|███████▏  | 71359697/100000000 [00:21<00:09, 2883018.24tokens/s]
Shard 6:  72%|███████▏  | 71651057/100000000 [00:22<00:10, 2801297.09tokens/s]
Shard 6:  72%|███████▏  | 71977973/100000000 [00:22<00:09, 2925947.38tokens/s]
Shard 6:  72%|███████▏  | 72321838/100000000 [00:22<00:09, 3072015.69tokens/s]
Shard 6:  73%|███████▎  | 72725040/100000000 [00:22<00:08, 3341573.14tokens/s]
Shard 6:  73%|███████▎  | 73106944/100000000 [00:22<00:07, 3479929.69tokens/s]
Shard 6:  73%|███████▎  | 73456807/100000000 [00:22<00:08, 3311832.57tokens/s]
Shard 6:  74%|███████▍  | 73791402/100000000 [00:22<00:08, 3157591.07tokens/s]
Shard 6:  74%|███████▍  | 74110345/100000000 [00:22<00:09, 2830383.25tokens/s]
Shard 6:  74%|███████▍  | 74407620/100000000 [00:22<00:08, 2859788.12tokens/s]
Shard 6:  75%|███████▍  | 74699279/100000000 [00:23<00:09, 2811008.67tokens/s]
Shard 6:  75%|███████▌  | 75088777/100000000 [00:23<00:08, 3108946.44tokens/s]
Shard 6:  75%|███████▌  | 75409478/100000000 [00:23<00:08, 3034879.90tokens/s]
Shard 6:  76%|███████▌  | 75776110/100000000 [00:23<00:07, 3210044.97tokens/s]
Shard 6:  76%|███████▌  | 76241671/100000000 [00:23<00:06, 3622883.84tokens/s]
Shard 6:  77%|███████▋  | 76611462/100000000 [00:23<00:06, 3604509.27tokens/s]
Shard 6:  77%|███████▋  | 76975149/100000000 [00:23<00:06, 3305714.69tokens/s]
Shard 6:  77%|███████▋  | 77312412/100000000 [00:23<00:07, 3185829.65tokens/s]
Shard 6:  78%|███████▊  | 77635858/100000000 [00:23<00:07, 2917195.11tokens/s]
Shard 6:  78%|███████▊  | 77933994/100000000 [00:24<00:07, 2781743.04tokens/s]
Shard 6:  78%|███████▊  | 78345884/100000000 [00:24<00:06, 3122544.08tokens/s]
Shard 6:  79%|███████▊  | 78666728/100000000 [00:24<00:07, 2999359.53tokens/s]
Shard 6:  79%|███████▉  | 79047575/100000000 [00:24<00:06, 3210754.49tokens/s]
Shard 6:  79%|███████▉  | 79376482/100000000 [00:24<00:06, 3148452.55tokens/s]
Shard 6:  80%|███████▉  | 79726914/100000000 [00:24<00:06, 3241473.16tokens/s]
Shard 6:  80%|████████  | 80056714/100000000 [00:24<00:06, 3250584.00tokens/s]
Shard 6:  80%|████████  | 80455966/100000000 [00:24<00:05, 3452945.77tokens/s]
Shard 6:  81%|████████  | 80803899/100000000 [00:24<00:05, 3290677.93tokens/s]
Shard 6:  81%|████████  | 81136096/100000000 [00:24<00:05, 3151441.85tokens/s]
Shard 6:  81%|████████▏ | 81497720/100000000 [00:25<00:05, 3277850.70tokens/s]
Shard 6:  82%|████████▏ | 81875117/100000000 [00:25<00:05, 3407174.87tokens/s]
Shard 6:  82%|████████▏ | 82232988/100000000 [00:25<00:05, 3446693.23tokens/s]
Shard 6:  83%|████████▎ | 82580083/100000000 [00:25<00:05, 3209106.66tokens/s]
Shard 6:  83%|████████▎ | 82922691/100000000 [00:25<00:05, 3239638.94tokens/s]
Shard 6:  83%|████████▎ | 83373725/100000000 [00:25<00:04, 3598565.79tokens/s]
Shard 6:  84%|████████▍ | 83939231/100000000 [00:25<00:03, 4144509.56tokens/s]
Shard 6:  84%|████████▍ | 84418026/100000000 [00:25<00:03, 4327135.06tokens/s]
Shard 6:  85%|████████▍ | 84854279/100000000 [00:25<00:03, 4218340.95tokens/s]
Shard 6:  85%|████████▌ | 85364062/100000000 [00:26<00:03, 4444343.98tokens/s]
Shard 6:  86%|████████▌ | 85861092/100000000 [00:26<00:03, 4586985.04tokens/s]
Shard 6:  86%|████████▋ | 86395338/100000000 [00:26<00:02, 4802562.31tokens/s]
Shard 6:  88%|████████▊ | 87697331/100000000 [00:26<00:01, 7072704.50tokens/s]
Shard 6:  88%|████████▊ | 88398999/100000000 [00:26<00:01, 6688118.06tokens/s]
Shard 6:  89%|████████▉ | 89067992/100000000 [00:26<00:01, 5472668.09tokens/s]
Shard 6:  90%|████████▉ | 89722125/100000000 [00:26<00:01, 5704216.89tokens/s]
Shard 6:  90%|█████████ | 90320879/100000000 [00:26<00:01, 4850897.45tokens/s]
Shard 6:  91%|█████████ | 90843054/100000000 [00:27<00:02, 4487374.26tokens/s]
Shard 6:  91%|█████████▏| 91319737/100000000 [00:27<00:02, 4172093.00tokens/s]
Shard 6:  92%|█████████▏| 91755886/100000000 [00:27<00:02, 3850203.55tokens/s]
Shard 6:  92%|█████████▏| 92156452/100000000 [00:27<00:02, 3885785.04tokens/s]
Shard 6:  93%|█████████▎| 92582171/100000000 [00:27<00:01, 3973546.10tokens/s]
Shard 6:  93%|█████████▎| 93002930/100000000 [00:27<00:01, 3658610.80tokens/s]
Shard 6:  93%|█████████▎| 93478649/100000000 [00:27<00:01, 3936517.29tokens/s]
Shard 6:  94%|█████████▍| 93883801/100000000 [00:27<00:01, 3634135.57tokens/s]
Shard 6:  94%|█████████▍| 94257830/100000000 [00:28<00:01, 3584220.11tokens/s]
Shard 6:  95%|█████████▍| 94702049/100000000 [00:28<00:01, 3790902.85tokens/s]
Shard 6:  95%|█████████▌| 95170308/100000000 [00:28<00:01, 3978735.93tokens/s]
Shard 6:  96%|█████████▌| 95582894/100000000 [00:28<00:01, 4014800.55tokens/s]
Shard 6:  96%|█████████▌| 95989185/100000000 [00:28<00:00, 4022178.09tokens/s]
Shard 6:  96%|█████████▋| 96475055/100000000 [00:28<00:00, 4250272.80tokens/s]
Shard 6:  97%|█████████▋| 96935156/100000000 [00:28<00:00, 4351044.32tokens/s]
Shard 6:  97%|█████████▋| 97372784/100000000 [00:28<00:00, 4210248.38tokens/s]
Shard 6:  98%|█████████▊| 97799374/100000000 [00:28<00:00, 3627646.41tokens/s]
Shard 6:  98%|█████████▊| 98229015/100000000 [00:29<00:00, 3802200.43tokens/s]
Shard 6:  99%|█████████▊| 98622431/100000000 [00:29<00:00, 3520735.85tokens/s]
Shard 6:  99%|█████████▉| 98987680/100000000 [00:29<00:00, 3541849.23tokens/s]
Shard 6:  99%|█████████▉| 99376194/100000000 [00:29<00:00, 3614541.05tokens/s]
Shard 6: 100%|█████████▉| 99808540/100000000 [00:29<00:00, 3799613.09tokens/s]
Shard 6: 100%|█████████▉| 99999549/100000000 [00:37<00:00, 2653805.26tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000006.bin (200,001,024 bytes) in the gpt-2 format

Shard 7:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 7:   0%|          | 328989/100000000 [00:00<00:30, 3283684.80tokens/s]
Shard 7:   1%|          | 657688/100000000 [00:00<00:35, 2793184.08tokens/s]
Shard 7:   1%|          | 951230/100000000 [00:00<00:34, 2836398.60tokens/s]
Shard 7:   1%|          | 1238472/100000000 [00:00<00:34, 2836416.21tokens/s]
Shard 7:   2%|▏         | 1586172/100000000 [00:00<00:32, 3047721.03tokens/s]
Shard 7:   2%|▏         | 2017772/100000000 [00:00<00:28, 3447252.04tokens/s]
Shard 7:   2%|▏         | 2364522/100000000 [00:00<00:30, 3193410.10tokens/s]
Shard 7:   3%|▎         | 2952251/100000000 [00:00<00:24, 3941511.33tokens/s]
Shard 7:   3%|▎         | 3352920/100000000 [00:00<00:26, 3712371.17tokens/s]
Shard 7:   4%|▎         | 3731075/100000000 [00:01<00:26, 3669641.50tokens/s]
Shard 7:   4%|▍         | 4102157/100000000 [00:01<00:27, 3471322.48tokens/s]
Shard 7:   5%|▍         | 4561476/100000000 [00:01<00:25, 3757672.90tokens/s]
Shard 7:   5%|▍         | 4943423/100000000 [00:01<00:27, 3406622.21tokens/s]
Shard 7:   5%|▌         | 5292808/100000000 [00:01<00:30, 3061053.93tokens/s]
Shard 7:   6%|▌         | 5700383/100000000 [00:01<00:28, 3314601.31tokens/s]
Shard 7:   6%|▋         | 6257512/100000000 [00:01<00:23, 3910082.55tokens/s]
Shard 7:   7%|▋         | 6742615/100000000 [00:01<00:22, 4152520.10tokens/s]
Shard 7:   7%|▋         | 7171416/100000000 [00:02<00:23, 3992192.72tokens/s]
Shard 7:   8%|▊         | 7656930/100000000 [00:02<00:21, 4218557.51tokens/s]
Shard 7:   8%|▊         | 8093694/100000000 [00:02<00:23, 3873524.92tokens/s]
Shard 7:   8%|▊         | 8492179/100000000 [00:02<00:26, 3447026.01tokens/s]
Shard 7:   9%|▉         | 8852057/100000000 [00:02<00:27, 3331362.14tokens/s]
Shard 7:   9%|▉         | 9195099/100000000 [00:02<00:27, 3275480.75tokens/s]
Shard 7:  10%|▉         | 9593251/100000000 [00:02<00:26, 3434712.68tokens/s]
Shard 7:  10%|▉         | 9945147/100000000 [00:02<00:26, 3446822.90tokens/s]
Shard 7:  10%|█         | 10321466/100000000 [00:02<00:25, 3521385.58tokens/s]
Shard 7:  11%|█         | 10833651/100000000 [00:03<00:22, 3900169.73tokens/s]
Shard 7:  11%|█         | 11227270/100000000 [00:03<00:24, 3681400.77tokens/s]
Shard 7:  12%|█▏        | 11600804/100000000 [00:03<00:25, 3453638.07tokens/s]
Shard 7:  12%|█▏        | 12030887/100000000 [00:03<00:23, 3671385.56tokens/s]
Shard 7:  12%|█▏        | 12403637/100000000 [00:03<00:26, 3359684.48tokens/s]
Shard 7:  13%|█▎        | 12747148/100000000 [00:03<00:30, 2855099.96tokens/s]
Shard 7:  13%|█▎        | 13048763/100000000 [00:03<00:32, 2702976.57tokens/s]
Shard 7:  13%|█▎        | 13331772/100000000 [00:03<00:32, 2666903.90tokens/s]
Shard 7:  14%|█▎        | 13615440/100000000 [00:04<00:31, 2707208.69tokens/s]
Shard 7:  14%|█▍        | 13937815/100000000 [00:04<00:30, 2842250.21tokens/s]
Shard 7:  14%|█▍        | 14227718/100000000 [00:04<00:36, 2352001.91tokens/s]
Shard 7:  14%|█▍        | 14479653/100000000 [00:04<00:35, 2378622.84tokens/s]
Shard 7:  15%|█▍        | 14729777/100000000 [00:04<00:35, 2378103.47tokens/s]
Shard 7:  15%|█▍        | 14976488/100000000 [00:04<00:36, 2323765.46tokens/s]
Shard 7:  16%|█▌        | 15649004/100000000 [00:04<00:24, 3505148.36tokens/s]
Shard 7:  16%|█▋        | 16365583/100000000 [00:04<00:18, 4464821.87tokens/s]
Shard 7:  17%|█▋        | 16828704/100000000 [00:05<00:21, 3816486.06tokens/s]
Shard 7:  17%|█▋        | 17300341/100000000 [00:05<00:20, 4044121.88tokens/s]
Shard 7:  18%|█▊        | 17727708/100000000 [00:05<00:23, 3483981.45tokens/s]
Shard 7:  18%|█▊        | 18202181/100000000 [00:05<00:21, 3780187.84tokens/s]
Shard 7:  19%|█▊        | 18606247/100000000 [00:05<00:21, 3840241.64tokens/s]
Shard 7:  19%|█▉        | 19009502/100000000 [00:05<00:21, 3805715.48tokens/s]
Shard 7:  19%|█▉        | 19405369/100000000 [00:05<00:22, 3594810.67tokens/s]
Shard 7:  20%|█▉        | 19776206/100000000 [00:05<00:22, 3508247.06tokens/s]
Shard 7:  20%|██        | 20146638/100000000 [00:05<00:26, 2968114.01tokens/s]
Shard 7:  21%|██        | 20516055/100000000 [00:06<00:25, 3135218.96tokens/s]
Shard 7:  21%|██        | 20846176/100000000 [00:06<00:25, 3119172.43tokens/s]
Shard 7:  21%|██        | 21170350/100000000 [00:06<00:27, 2823547.31tokens/s]
Shard 7:  21%|██▏       | 21464072/100000000 [00:06<00:32, 2451516.03tokens/s]
Shard 7:  22%|██▏       | 21723838/100000000 [00:06<00:32, 2426894.14tokens/s]
Shard 7:  22%|██▏       | 22094118/100000000 [00:06<00:28, 2712226.64tokens/s]
Shard 7:  23%|██▎       | 22510206/100000000 [00:06<00:25, 3087273.66tokens/s]
Shard 7:  23%|██▎       | 22833021/100000000 [00:06<00:27, 2803023.81tokens/s]
Shard 7:  23%|██▎       | 23246820/100000000 [00:07<00:24, 3143754.16tokens/s]
Shard 7:  24%|██▎       | 23576277/100000000 [00:07<00:24, 3144207.18tokens/s]
Shard 7:  24%|██▍       | 23910013/100000000 [00:07<00:23, 3191795.28tokens/s]
Shard 7:  24%|██▍       | 24401825/100000000 [00:07<00:20, 3673376.06tokens/s]
Shard 7:  25%|██▍       | 24781313/100000000 [00:07<00:20, 3691480.30tokens/s]
Shard 7:  26%|██▌       | 25705368/100000000 [00:07<00:14, 5240488.69tokens/s]
Shard 7:  26%|██▌       | 26235212/100000000 [00:07<00:15, 4853565.60tokens/s]
Shard 7:  27%|██▋       | 26729693/100000000 [00:07<00:15, 4820702.93tokens/s]
Shard 7:  27%|██▋       | 27218495/100000000 [00:07<00:15, 4717702.43tokens/s]
Shard 7:  28%|██▊       | 27694584/100000000 [00:08<00:17, 4074182.56tokens/s]
Shard 7:  28%|██▊       | 28118666/100000000 [00:08<00:19, 3774120.85tokens/s]
Shard 7:  29%|██▊       | 28558408/100000000 [00:08<00:18, 3931108.52tokens/s]
Shard 7:  29%|██▉       | 28965491/100000000 [00:08<00:19, 3659669.19tokens/s]
Shard 7:  29%|██▉       | 29396250/100000000 [00:08<00:18, 3816555.96tokens/s]
Shard 7:  30%|██▉       | 29787825/100000000 [00:08<00:18, 3809095.43tokens/s]
Shard 7:  30%|███       | 30175494/100000000 [00:08<00:20, 3423280.68tokens/s]
Shard 7:  31%|███       | 30528614/100000000 [00:08<00:20, 3423257.45tokens/s]
Shard 7:  31%|███       | 31039863/100000000 [00:09<00:17, 3864386.54tokens/s]
Shard 7:  31%|███▏      | 31435916/100000000 [00:09<00:19, 3490361.84tokens/s]
Shard 7:  32%|███▏      | 32071981/100000000 [00:09<00:16, 4242362.50tokens/s]
Shard 7:  33%|███▎      | 32518870/100000000 [00:09<00:16, 4151355.80tokens/s]
Shard 7:  33%|███▎      | 32948052/100000000 [00:09<00:17, 3855847.98tokens/s]
Shard 7:  33%|███▎      | 33346011/100000000 [00:09<00:20, 3312117.72tokens/s]
Shard 7:  34%|███▎      | 33696442/100000000 [00:09<00:19, 3334166.82tokens/s]
Shard 7:  34%|███▍      | 34043965/100000000 [00:09<00:21, 3135394.61tokens/s]
Shard 7:  34%|███▍      | 34374294/100000000 [00:09<00:20, 3149686.09tokens/s]
Shard 7:  35%|███▍      | 34786586/100000000 [00:10<00:19, 3401538.42tokens/s]
Shard 7:  35%|███▌      | 35137483/100000000 [00:10<00:19, 3363117.85tokens/s]
Shard 7:  35%|███▌      | 35480123/100000000 [00:10<00:21, 3005927.11tokens/s]
Shard 7:  36%|███▌      | 35820694/100000000 [00:10<00:20, 3109759.93tokens/s]
Shard 7:  36%|███▌      | 36140249/100000000 [00:10<00:20, 3041947.44tokens/s]
Shard 7:  37%|███▋      | 36568972/100000000 [00:10<00:19, 3318533.35tokens/s]
Shard 7:  37%|███▋      | 37053567/100000000 [00:10<00:16, 3733214.48tokens/s]
Shard 7:  37%|███▋      | 37434467/100000000 [00:10<00:16, 3701581.07tokens/s]
Shard 7:  38%|███▊      | 37810043/100000000 [00:10<00:17, 3530615.75tokens/s]
Shard 7:  38%|███▊      | 38168747/100000000 [00:11<00:20, 3011937.51tokens/s]
Shard 7:  39%|███▊      | 38552105/100000000 [00:11<00:19, 3206217.32tokens/s]
Shard 7:  39%|███▉      | 38886757/100000000 [00:11<00:19, 3209351.77tokens/s]
Shard 7:  39%|███▉      | 39348467/100000000 [00:11<00:16, 3571029.70tokens/s]
Shard 7:  40%|███▉      | 39738623/100000000 [00:11<00:16, 3644829.56tokens/s]
Shard 7:  40%|████      | 40189142/100000000 [00:11<00:15, 3876205.94tokens/s]
Shard 7:  41%|████      | 40585784/100000000 [00:11<00:20, 2897465.91tokens/s]
Shard 7:  41%|████      | 40952517/100000000 [00:11<00:19, 3077257.48tokens/s]
Shard 7:  41%|████▏     | 41307348/100000000 [00:12<00:18, 3183050.26tokens/s]
Shard 7:  42%|████▏     | 41657689/100000000 [00:12<00:17, 3266282.12tokens/s]
Shard 7:  42%|████▏     | 42094157/100000000 [00:12<00:16, 3555585.53tokens/s]
Shard 7:  42%|████▏     | 42466013/100000000 [00:12<00:17, 3277742.01tokens/s]
Shard 7:  43%|████▎     | 42870269/100000000 [00:12<00:16, 3469092.88tokens/s]
Shard 7:  43%|████▎     | 43230275/100000000 [00:12<00:16, 3434939.44tokens/s]
Shard 7:  44%|████▎     | 43582757/100000000 [00:12<00:16, 3401489.94tokens/s]
Shard 7:  44%|████▍     | 43981849/100000000 [00:12<00:15, 3561288.01tokens/s]
Shard 7:  44%|████▍     | 44479587/100000000 [00:12<00:13, 3966122.67tokens/s]
Shard 7:  45%|████▌     | 45005548/100000000 [00:13<00:12, 4291950.06tokens/s]
Shard 7:  45%|████▌     | 45439093/100000000 [00:13<00:13, 4170895.68tokens/s]
Shard 7:  46%|████▌     | 45859662/100000000 [00:13<00:19, 2714098.13tokens/s]
Shard 7:  46%|████▌     | 46197808/100000000 [00:13<00:19, 2771038.13tokens/s]
Shard 7:  47%|████▋     | 46791061/100000000 [00:13<00:15, 3484361.25tokens/s]
Shard 7:  47%|████▋     | 47197220/100000000 [00:13<00:14, 3597285.28tokens/s]
Shard 7:  48%|████▊     | 47715137/100000000 [00:13<00:13, 3993400.42tokens/s]
Shard 7:  48%|████▊     | 48150838/100000000 [00:14<00:14, 3640246.29tokens/s]
Shard 7:  49%|████▊     | 48545122/100000000 [00:14<00:14, 3553160.40tokens/s]
Shard 7:  49%|████▉     | 49002086/100000000 [00:14<00:13, 3814103.90tokens/s]
Shard 7:  49%|████▉     | 49402384/100000000 [00:14<00:13, 3756135.99tokens/s]
Shard 7:  50%|████▉     | 49791059/100000000 [00:14<00:14, 3395814.05tokens/s]
Shard 7:  50%|█████     | 50144824/100000000 [00:14<00:15, 3219493.30tokens/s]
Shard 7:  50%|█████     | 50477009/100000000 [00:14<00:16, 3031769.85tokens/s]
Shard 7:  51%|█████     | 50829933/100000000 [00:14<00:15, 3144608.63tokens/s]
Shard 7:  51%|█████     | 51151647/100000000 [00:14<00:18, 2700509.83tokens/s]
Shard 7:  52%|█████▏    | 51503541/100000000 [00:15<00:16, 2899320.71tokens/s]
Shard 7:  52%|█████▏    | 51823395/100000000 [00:15<00:16, 2927378.49tokens/s]
Shard 7:  52%|█████▏    | 52126197/100000000 [00:15<00:18, 2646895.66tokens/s]
Shard 7:  53%|█████▎    | 52579388/100000000 [00:15<00:15, 3121481.30tokens/s]
Shard 7:  53%|█████▎    | 52908536/100000000 [00:15<00:15, 3036558.66tokens/s]
Shard 7:  53%|█████▎    | 53223483/100000000 [00:15<00:17, 2710125.61tokens/s]
Shard 7:  54%|█████▎    | 53543326/100000000 [00:15<00:16, 2830844.75tokens/s]
Shard 7:  54%|█████▍    | 53905790/100000000 [00:15<00:15, 3031541.62tokens/s]
Shard 7:  54%|█████▍    | 54220208/100000000 [00:16<00:15, 2940537.65tokens/s]
Shard 7:  55%|█████▍    | 54521468/100000000 [00:16<00:17, 2672463.41tokens/s]
Shard 7:  55%|█████▍    | 54850884/100000000 [00:16<00:15, 2825056.23tokens/s]
Shard 7:  55%|█████▌    | 55142240/100000000 [00:16<00:17, 2536568.03tokens/s]
Shard 7:  55%|█████▌    | 55435479/100000000 [00:16<00:16, 2632992.61tokens/s]
Shard 7:  56%|█████▌    | 55711525/100000000 [00:16<00:16, 2663559.88tokens/s]
Shard 7:  56%|█████▌    | 56090521/100000000 [00:16<00:14, 2968452.97tokens/s]
Shard 7:  56%|█████▋    | 56394761/100000000 [00:16<00:17, 2475769.71tokens/s]
Shard 7:  57%|█████▋    | 56677262/100000000 [00:16<00:17, 2543818.21tokens/s]
Shard 7:  57%|█████▋    | 56945764/100000000 [00:17<00:18, 2294717.15tokens/s]
Shard 7:  57%|█████▋    | 57408586/100000000 [00:17<00:14, 2866804.11tokens/s]
Shard 7:  58%|█████▊    | 57714877/100000000 [00:17<00:14, 2910265.37tokens/s]
Shard 7:  58%|█████▊    | 58028240/100000000 [00:17<00:15, 2779043.26tokens/s]
Shard 7:  58%|█████▊    | 58317744/100000000 [00:17<00:16, 2459992.85tokens/s]
Shard 7:  59%|█████▊    | 58576849/100000000 [00:17<00:17, 2311628.24tokens/s]
Shard 7:  59%|█████▉    | 58843428/100000000 [00:17<00:17, 2390083.40tokens/s]
Shard 7:  59%|█████▉    | 59091154/100000000 [00:17<00:17, 2351849.65tokens/s]
Shard 7:  59%|█████▉    | 59449899/100000000 [00:18<00:15, 2677084.74tokens/s]
Shard 7:  60%|█████▉    | 59734243/100000000 [00:18<00:15, 2608033.64tokens/s]
Shard 7:  60%|██████    | 60055223/100000000 [00:18<00:14, 2759799.64tokens/s]
Shard 7:  60%|██████    | 60336297/100000000 [00:18<00:15, 2565752.95tokens/s]
Shard 7:  61%|██████    | 60598424/100000000 [00:18<00:15, 2547768.83tokens/s]
Shard 7:  61%|██████    | 60857590/100000000 [00:18<00:15, 2449858.18tokens/s]
Shard 7:  61%|██████    | 61194323/100000000 [00:18<00:14, 2701185.82tokens/s]
Shard 7:  62%|██████▏   | 61656538/100000000 [00:18<00:11, 3241414.39tokens/s]
Shard 7:  62%|██████▏   | 61987402/100000000 [00:19<00:14, 2654649.50tokens/s]
Shard 7:  62%|██████▏   | 62359614/100000000 [00:19<00:12, 2908359.83tokens/s]
Shard 7:  63%|██████▎   | 62807596/100000000 [00:19<00:11, 3316218.61tokens/s]
Shard 7:  63%|██████▎   | 63158796/100000000 [00:19<00:11, 3222193.37tokens/s]
Shard 7:  63%|██████▎   | 63495825/100000000 [00:19<00:11, 3165279.30tokens/s]
Shard 7:  64%|██████▍   | 63822692/100000000 [00:19<00:11, 3149658.09tokens/s]
Shard 7:  64%|██████▍   | 64144522/100000000 [00:19<00:12, 2780619.22tokens/s]
Shard 7:  64%|██████▍   | 64434228/100000000 [00:19<00:13, 2655300.95tokens/s]
Shard 7:  65%|██████▍   | 64711098/100000000 [00:19<00:15, 2349867.80tokens/s]
Shard 7:  65%|██████▍   | 64956416/100000000 [00:20<00:15, 2282846.42tokens/s]
Shard 7:  65%|██████▌   | 65191289/100000000 [00:20<00:16, 2071905.35tokens/s]
Shard 7:  65%|██████▌   | 65408658/100000000 [00:20<00:16, 2051355.65tokens/s]
Shard 7:  66%|██████▌   | 65622565/100000000 [00:20<00:18, 1880121.61tokens/s]
Shard 7:  66%|██████▌   | 65892922/100000000 [00:20<00:16, 2080649.49tokens/s]
Shard 7:  66%|██████▌   | 66108320/100000000 [00:20<00:16, 2047222.56tokens/s]
Shard 7:  67%|██████▋   | 66569298/100000000 [00:20<00:12, 2723845.73tokens/s]
Shard 7:  67%|██████▋   | 66908034/100000000 [00:20<00:11, 2902241.23tokens/s]
Shard 7:  67%|██████▋   | 67207840/100000000 [00:21<00:12, 2652180.52tokens/s]
Shard 7:  67%|██████▋   | 67498495/100000000 [00:21<00:11, 2714569.78tokens/s]
Shard 7:  68%|██████▊   | 67784866/100000000 [00:21<00:11, 2755568.93tokens/s]
Shard 7:  68%|██████▊   | 68066330/100000000 [00:21<00:13, 2339531.63tokens/s]
Shard 7:  68%|██████▊   | 68314422/100000000 [00:21<00:14, 2143047.66tokens/s]
Shard 7:  69%|██████▊   | 68540232/100000000 [00:21<00:14, 2105184.50tokens/s]
Shard 7:  69%|██████▉   | 68967258/100000000 [00:21<00:11, 2656663.37tokens/s]
Shard 7:  69%|██████▉   | 69318732/100000000 [00:21<00:10, 2877983.64tokens/s]
Shard 7:  70%|██████▉   | 69620303/100000000 [00:22<00:12, 2453028.89tokens/s]
Shard 7:  70%|██████▉   | 69885155/100000000 [00:22<00:12, 2365004.20tokens/s]
Shard 7:  70%|███████   | 70144917/100000000 [00:22<00:13, 2279874.01tokens/s]
Shard 7:  70%|███████   | 70382077/100000000 [00:22<00:14, 2106878.89tokens/s]
Shard 7:  71%|███████   | 70604962/100000000 [00:22<00:14, 2018090.71tokens/s]
Shard 7:  71%|███████   | 70812121/100000000 [00:22<00:14, 2018852.74tokens/s]
Shard 7:  71%|███████   | 71074180/100000000 [00:22<00:13, 2143039.73tokens/s]
Shard 7:  71%|███████▏  | 71292438/100000000 [00:22<00:14, 2003923.41tokens/s]
Shard 7:  72%|███████▏  | 71635721/100000000 [00:22<00:11, 2363700.53tokens/s]
Shard 7:  72%|███████▏  | 71936592/100000000 [00:23<00:11, 2534675.43tokens/s]
Shard 7:  72%|███████▏  | 72303049/100000000 [00:23<00:09, 2838592.91tokens/s]
Shard 7:  73%|███████▎  | 72773825/100000000 [00:23<00:08, 3359996.35tokens/s]
Shard 7:  73%|███████▎  | 73208561/100000000 [00:23<00:07, 3635581.95tokens/s]
Shard 7:  74%|███████▎  | 73597583/100000000 [00:23<00:07, 3702218.27tokens/s]
Shard 7:  74%|███████▍  | 73972172/100000000 [00:23<00:07, 3640469.35tokens/s]
Shard 7:  74%|███████▍  | 74442203/100000000 [00:23<00:06, 3941725.34tokens/s]
Shard 7:  75%|███████▌  | 75054678/100000000 [00:23<00:05, 4566891.83tokens/s]
Shard 7:  76%|███████▌  | 75514320/100000000 [00:23<00:05, 4242144.06tokens/s]
Shard 7:  76%|███████▌  | 75945812/100000000 [00:24<00:05, 4161480.37tokens/s]
Shard 7:  76%|███████▋  | 76497325/100000000 [00:24<00:05, 4530243.04tokens/s]
Shard 7:  77%|███████▋  | 77181491/100000000 [00:24<00:04, 5160942.12tokens/s]
Shard 7:  78%|███████▊  | 77703956/100000000 [00:24<00:04, 4646297.98tokens/s]
Shard 7:  78%|███████▊  | 78209138/100000000 [00:24<00:04, 4578787.61tokens/s]
Shard 7:  79%|███████▊  | 78676054/100000000 [00:24<00:04, 4320670.03tokens/s]
Shard 7:  79%|███████▉  | 79120392/100000000 [00:24<00:05, 4174668.14tokens/s]
Shard 7:  80%|███████▉  | 79546589/100000000 [00:24<00:04, 4191787.23tokens/s]
Shard 7:  80%|███████▉  | 79969954/100000000 [00:24<00:04, 4194194.58tokens/s]
Shard 7:  80%|████████  | 80392050/100000000 [00:25<00:05, 3750720.99tokens/s]
Shard 7:  81%|████████  | 80776957/100000000 [00:25<00:05, 3656593.53tokens/s]
Shard 7:  81%|████████  | 81153043/100000000 [00:25<00:05, 3669647.63tokens/s]
Shard 7:  82%|████████▏ | 81575370/100000000 [00:25<00:04, 3820364.19tokens/s]
Shard 7:  82%|████████▏ | 81961918/100000000 [00:25<00:05, 3560892.29tokens/s]
Shard 7:  82%|████████▏ | 82483845/100000000 [00:25<00:04, 3999103.13tokens/s]
Shard 7:  83%|████████▎ | 82892865/100000000 [00:25<00:04, 3996877.33tokens/s]
Shard 7:  83%|████████▎ | 83298334/100000000 [00:25<00:04, 3735671.15tokens/s]
Shard 7:  84%|████████▎ | 83678597/100000000 [00:25<00:04, 3654961.38tokens/s]
Shard 7:  84%|████████▍ | 84048538/100000000 [00:26<00:04, 3625520.18tokens/s]
Shard 7:  85%|████████▍ | 84547313/100000000 [00:26<00:03, 3995077.72tokens/s]
Shard 7:  85%|████████▌ | 85006371/100000000 [00:26<00:03, 4094675.57tokens/s]
Shard 7:  85%|████████▌ | 85418998/100000000 [00:26<00:03, 4087631.31tokens/s]
Shard 7:  86%|████████▌ | 85830371/100000000 [00:26<00:03, 3884633.29tokens/s]
Shard 7:  86%|████████▌ | 86221944/100000000 [00:26<00:03, 3817082.90tokens/s]
Shard 7:  87%|████████▋ | 86698893/100000000 [00:26<00:03, 4084993.01tokens/s]
Shard 7:  87%|████████▋ | 87128495/100000000 [00:26<00:03, 4122244.33tokens/s]
Shard 7:  88%|████████▊ | 87542958/100000000 [00:26<00:03, 3994862.05tokens/s]
Shard 7:  88%|████████▊ | 87945043/100000000 [00:27<00:03, 3742142.42tokens/s]
Shard 7:  88%|████████▊ | 88341831/100000000 [00:27<00:03, 3791609.29tokens/s]
Shard 7:  89%|████████▉ | 88816583/100000000 [00:27<00:02, 4061370.31tokens/s]
Shard 7:  89%|████████▉ | 89226765/100000000 [00:27<00:02, 4039719.48tokens/s]
Shard 7:  90%|████████▉ | 89710784/100000000 [00:27<00:02, 4226989.67tokens/s]
Shard 7:  90%|█████████ | 90179293/100000000 [00:27<00:02, 4357682.68tokens/s]
Shard 7:  91%|█████████ | 90693934/100000000 [00:27<00:02, 4578998.20tokens/s]
Shard 7:  91%|█████████ | 91153918/100000000 [00:27<00:01, 4498645.03tokens/s]
Shard 7:  92%|█████████▏| 91676034/100000000 [00:27<00:01, 4672599.37tokens/s]
Shard 7:  92%|█████████▏| 92144718/100000000 [00:27<00:01, 4152301.97tokens/s]
Shard 7:  93%|█████████▎| 92572367/100000000 [00:28<00:01, 4185647.19tokens/s]
Shard 7:  93%|█████████▎| 93097817/100000000 [00:28<00:01, 4480004.08tokens/s]
Shard 7:  94%|█████████▎| 93653625/100000000 [00:28<00:01, 4768574.70tokens/s]
Shard 7:  94%|█████████▍| 94288421/100000000 [00:28<00:01, 5221379.89tokens/s]
Shard 7:  95%|█████████▍| 94817705/100000000 [00:28<00:01, 4863128.25tokens/s]
Shard 7:  95%|█████████▌| 95315984/100000000 [00:28<00:00, 4702980.93tokens/s]
Shard 7:  96%|█████████▌| 95793162/100000000 [00:28<00:00, 4446558.15tokens/s]
Shard 7:  96%|█████████▌| 96246441/100000000 [00:28<00:00, 4454273.32tokens/s]
Shard 7:  97%|█████████▋| 96788833/100000000 [00:28<00:00, 4709705.21tokens/s]
Shard 7:  97%|█████████▋| 97264712/100000000 [00:29<00:00, 4340282.77tokens/s]
Shard 7:  98%|█████████▊| 97816493/100000000 [00:29<00:00, 4654389.12tokens/s]
Shard 7:  99%|█████████▉| 99250848/100000000 [00:29<00:00, 7346072.98tokens/s]
Shard 7: 100%|█████████▉| 99999832/100000000 [00:37<00:00, 2654144.56tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000007.bin (200,001,024 bytes) in the gpt-2 format

Shard 8:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 8:   0%|          | 232204/100000000 [00:00<00:43, 2313190.51tokens/s]
Shard 8:   0%|          | 493151/100000000 [00:00<00:40, 2483204.63tokens/s]
Shard 8:   1%|          | 741522/100000000 [00:00<00:40, 2422637.29tokens/s]
Shard 8:   1%|          | 983994/100000000 [00:00<00:49, 2010192.79tokens/s]
Shard 8:   1%|▏         | 1398743/100000000 [00:00<00:37, 2663306.70tokens/s]
Shard 8:   2%|▏         | 1910992/100000000 [00:00<00:28, 3422767.16tokens/s]
Shard 8:   2%|▏         | 2336138/100000000 [00:00<00:26, 3654179.46tokens/s]
Shard 8:   3%|▎         | 2714614/100000000 [00:00<00:27, 3596501.08tokens/s]
Shard 8:   3%|▎         | 3250144/100000000 [00:00<00:23, 4089825.72tokens/s]
Shard 8:   4%|▎         | 3667665/100000000 [00:01<00:24, 3901621.74tokens/s]
Shard 8:   4%|▍         | 4064713/100000000 [00:01<00:26, 3629743.64tokens/s]
Shard 8:   5%|▍         | 4699982/100000000 [00:01<00:21, 4361193.58tokens/s]
Shard 8:   5%|▌         | 5148892/100000000 [00:01<00:21, 4391005.34tokens/s]
Shard 8:   6%|▌         | 5598349/100000000 [00:01<00:21, 4293564.70tokens/s]
Shard 8:   6%|▌         | 6034032/100000000 [00:01<00:23, 3937804.20tokens/s]
Shard 8:   6%|▋         | 6436903/100000000 [00:01<00:25, 3656725.02tokens/s]
Shard 8:   7%|▋         | 6811777/100000000 [00:01<00:26, 3555866.67tokens/s]
Shard 8:   7%|▋         | 7172889/100000000 [00:02<00:26, 3512669.08tokens/s]
Shard 8:   8%|▊         | 7576138/100000000 [00:02<00:25, 3639915.82tokens/s]
Shard 8:   8%|▊         | 7995662/100000000 [00:02<00:24, 3769093.09tokens/s]
Shard 8:   8%|▊         | 8375926/100000000 [00:02<00:25, 3528180.25tokens/s]
Shard 8:   9%|▊         | 8734076/100000000 [00:02<00:26, 3392522.49tokens/s]
Shard 8:   9%|▉         | 9076838/100000000 [00:02<00:30, 2960680.32tokens/s]
Shard 8:   9%|▉         | 9467228/100000000 [00:02<00:28, 3194844.30tokens/s]
Shard 8:  10%|▉         | 9798230/100000000 [00:02<00:29, 3062857.45tokens/s]
Shard 8:  10%|█         | 10113713/100000000 [00:02<00:29, 3035746.29tokens/s]
Shard 8:  11%|█         | 10726534/100000000 [00:03<00:23, 3843788.42tokens/s]
Shard 8:  11%|█         | 11245470/100000000 [00:03<00:21, 4201873.12tokens/s]
Shard 8:  12%|█▏        | 11712059/100000000 [00:03<00:20, 4331494.51tokens/s]
Shard 8:  12%|█▏        | 12153514/100000000 [00:03<00:23, 3763572.59tokens/s]
Shard 8:  13%|█▎        | 12547933/100000000 [00:03<00:24, 3534978.75tokens/s]
Shard 8:  13%|█▎        | 12915369/100000000 [00:03<00:25, 3460952.34tokens/s]
Shard 8:  13%|█▎        | 13271437/100000000 [00:03<00:25, 3411373.39tokens/s]
Shard 8:  14%|█▎        | 13676025/100000000 [00:03<00:24, 3575844.18tokens/s]
Shard 8:  14%|█▍        | 14185107/100000000 [00:03<00:21, 3960336.00tokens/s]
Shard 8:  15%|█▍        | 14587711/100000000 [00:04<00:22, 3729893.80tokens/s]
Shard 8:  15%|█▍        | 14970200/100000000 [00:04<00:24, 3465905.64tokens/s]
Shard 8:  15%|█▌        | 15323721/100000000 [00:04<00:25, 3367895.62tokens/s]
Shard 8:  16%|█▌        | 15665727/100000000 [00:04<00:27, 3103666.93tokens/s]
Shard 8:  16%|█▌        | 16062823/100000000 [00:04<00:25, 3323080.33tokens/s]
Shard 8:  16%|█▋        | 16405461/100000000 [00:04<00:26, 3173274.17tokens/s]
Shard 8:  17%|█▋        | 16761137/100000000 [00:04<00:25, 3271485.07tokens/s]
Shard 8:  17%|█▋        | 17100081/100000000 [00:04<00:25, 3302426.17tokens/s]
Shard 8:  17%|█▋        | 17434243/100000000 [00:04<00:26, 3167602.82tokens/s]
Shard 8:  18%|█▊        | 17754804/100000000 [00:05<00:29, 2797310.12tokens/s]
Shard 8:  18%|█▊        | 18246111/100000000 [00:05<00:24, 3334955.22tokens/s]
Shard 8:  19%|█▊        | 18594543/100000000 [00:05<00:25, 3137296.26tokens/s]
Shard 8:  19%|█▉        | 18960505/100000000 [00:05<00:24, 3254140.85tokens/s]
Shard 8:  19%|█▉        | 19295442/100000000 [00:05<00:25, 3128730.84tokens/s]
Shard 8:  20%|█▉        | 19614983/100000000 [00:05<00:26, 3073318.35tokens/s]
Shard 8:  20%|██        | 20095409/100000000 [00:05<00:22, 3536547.53tokens/s]
Shard 8:  20%|██        | 20481533/100000000 [00:05<00:22, 3606293.65tokens/s]
Shard 8:  21%|██        | 20974748/100000000 [00:06<00:19, 3960312.88tokens/s]
Shard 8:  21%|██▏       | 21447538/100000000 [00:06<00:18, 4181322.00tokens/s]
Shard 8:  22%|██▏       | 21871021/100000000 [00:06<00:18, 4135821.85tokens/s]
Shard 8:  22%|██▏       | 22337359/100000000 [00:06<00:18, 4275311.98tokens/s]
Shard 8:  23%|██▎       | 22833838/100000000 [00:06<00:17, 4455059.01tokens/s]
Shard 8:  23%|██▎       | 23281781/100000000 [00:06<00:20, 3793417.00tokens/s]
Shard 8:  24%|██▎       | 23715577/100000000 [00:06<00:19, 3911327.76tokens/s]
Shard 8:  24%|██▍       | 24154988/100000000 [00:06<00:18, 4013305.45tokens/s]
Shard 8:  25%|██▍       | 24567459/100000000 [00:06<00:20, 3719495.73tokens/s]
Shard 8:  25%|██▍       | 24956406/100000000 [00:07<00:20, 3648256.27tokens/s]
Shard 8:  25%|██▌       | 25328943/100000000 [00:07<00:21, 3537032.64tokens/s]
Shard 8:  26%|██▌       | 25788177/100000000 [00:07<00:19, 3819907.73tokens/s]
Shard 8:  26%|██▌       | 26234605/100000000 [00:07<00:18, 3988411.80tokens/s]
Shard 8:  27%|██▋       | 26639863/100000000 [00:07<00:20, 3603891.44tokens/s]
Shard 8:  27%|██▋       | 27010286/100000000 [00:07<00:20, 3518841.44tokens/s]
Shard 8:  27%|██▋       | 27369511/100000000 [00:07<00:23, 3070388.06tokens/s]
Shard 8:  28%|██▊       | 27797852/100000000 [00:07<00:21, 3368780.10tokens/s]
Shard 8:  28%|██▊       | 28149774/100000000 [00:07<00:21, 3379252.91tokens/s]
Shard 8:  29%|██▊       | 28563577/100000000 [00:08<00:20, 3563334.40tokens/s]
Shard 8:  29%|██▉       | 28929029/100000000 [00:08<00:21, 3285642.71tokens/s]
Shard 8:  29%|██▉       | 29267231/100000000 [00:08<00:21, 3220870.74tokens/s]
Shard 8:  30%|██▉       | 29783458/100000000 [00:08<00:18, 3741871.61tokens/s]
Shard 8:  30%|███       | 30267355/100000000 [00:08<00:17, 4023525.28tokens/s]
Shard 8:  31%|███       | 30679057/100000000 [00:08<00:19, 3631012.13tokens/s]
Shard 8:  31%|███       | 31055390/100000000 [00:08<00:20, 3330118.53tokens/s]
Shard 8:  31%|███▏      | 31401218/100000000 [00:08<00:20, 3361087.75tokens/s]
Shard 8:  32%|███▏      | 32000613/100000000 [00:08<00:16, 4065999.97tokens/s]
Shard 8:  33%|███▎      | 32623949/100000000 [00:09<00:14, 4660498.26tokens/s]
Shard 8:  33%|███▎      | 33105055/100000000 [00:09<00:14, 4466505.28tokens/s]
Shard 8:  34%|███▎      | 33564144/100000000 [00:09<00:19, 3372284.68tokens/s]
Shard 8:  34%|███▍      | 33947464/100000000 [00:09<00:22, 2998555.42tokens/s]
Shard 8:  34%|███▍      | 34286112/100000000 [00:09<00:23, 2850308.18tokens/s]
Shard 8:  35%|███▍      | 34596058/100000000 [00:09<00:27, 2411199.12tokens/s]
Shard 8:  35%|███▍      | 34862671/100000000 [00:10<00:29, 2230913.33tokens/s]
Shard 8:  35%|███▌      | 35217892/100000000 [00:10<00:25, 2508823.66tokens/s]
Shard 8:  35%|███▌      | 35491589/100000000 [00:10<00:26, 2425343.26tokens/s]
Shard 8:  36%|███▌      | 35749021/100000000 [00:10<00:28, 2257820.19tokens/s]
Shard 8:  36%|███▌      | 36060462/100000000 [00:10<00:25, 2460581.82tokens/s]
Shard 8:  36%|███▋      | 36319105/100000000 [00:10<00:27, 2295860.21tokens/s]
Shard 8:  37%|███▋      | 36558040/100000000 [00:10<00:28, 2192884.85tokens/s]
Shard 8:  37%|███▋      | 36797946/100000000 [00:10<00:28, 2233649.94tokens/s]
Shard 8:  37%|███▋      | 37269376/100000000 [00:10<00:21, 2880508.01tokens/s]
Shard 8:  38%|███▊      | 37576518/100000000 [00:11<00:21, 2910674.58tokens/s]
Shard 8:  38%|███▊      | 37875928/100000000 [00:11<00:23, 2684914.70tokens/s]
Shard 8:  38%|███▊      | 38153052/100000000 [00:11<00:25, 2442466.38tokens/s]
Shard 8:  38%|███▊      | 38406478/100000000 [00:11<00:25, 2427956.44tokens/s]
Shard 8:  39%|███▊      | 38662751/100000000 [00:11<00:24, 2456018.20tokens/s]
Shard 8:  39%|███▉      | 38964281/100000000 [00:11<00:23, 2582532.52tokens/s]
Shard 8:  39%|███▉      | 39229765/100000000 [00:11<00:25, 2346803.54tokens/s]
Shard 8:  39%|███▉      | 39471057/100000000 [00:11<00:27, 2213571.74tokens/s]
Shard 8:  40%|███▉      | 39706125/100000000 [00:12<00:26, 2237898.29tokens/s]
Shard 8:  40%|███▉      | 39947057/100000000 [00:12<00:26, 2278580.02tokens/s]
Shard 8:  40%|████      | 40178768/100000000 [00:12<00:26, 2276754.73tokens/s]
Shard 8:  40%|████      | 40408730/100000000 [00:12<00:27, 2186093.66tokens/s]
Shard 8:  41%|████      | 40680556/100000000 [00:12<00:25, 2313939.14tokens/s]
Shard 8:  41%|████      | 40914154/100000000 [00:12<00:26, 2215474.16tokens/s]
Shard 8:  41%|████      | 41138419/100000000 [00:12<00:28, 2101201.06tokens/s]
Shard 8:  41%|████▏     | 41350612/100000000 [00:12<00:28, 2094133.94tokens/s]
Shard 8:  42%|████▏     | 41561996/100000000 [00:12<00:28, 2065968.17tokens/s]
Shard 8:  42%|████▏     | 41782466/100000000 [00:13<00:27, 2091318.95tokens/s]
Shard 8:  42%|████▏     | 42053859/100000000 [00:13<00:25, 2263059.31tokens/s]
Shard 8:  42%|████▏     | 42283231/100000000 [00:13<00:27, 2120966.50tokens/s]
Shard 8:  43%|████▎     | 42572521/100000000 [00:13<00:24, 2318887.69tokens/s]
Shard 8:  43%|████▎     | 42808687/100000000 [00:13<00:26, 2146313.32tokens/s]
Shard 8:  43%|████▎     | 43097878/100000000 [00:13<00:24, 2341858.48tokens/s]
Shard 8:  43%|████▎     | 43336800/100000000 [00:13<00:24, 2326869.22tokens/s]
Shard 8:  44%|████▎     | 43574551/100000000 [00:13<00:25, 2184465.61tokens/s]
Shard 8:  44%|████▍     | 44090312/100000000 [00:13<00:18, 2997402.19tokens/s]
Shard 8:  44%|████▍     | 44400881/100000000 [00:14<00:18, 3009562.72tokens/s]
Shard 8:  45%|████▍     | 44709506/100000000 [00:14<00:20, 2697380.51tokens/s]
Shard 8:  45%|████▍     | 44991160/100000000 [00:14<00:20, 2649996.97tokens/s]
Shard 8:  45%|████▌     | 45286047/100000000 [00:14<00:22, 2465016.90tokens/s]
Shard 8:  46%|████▌     | 45539902/100000000 [00:14<00:25, 2146566.61tokens/s]
Shard 8:  46%|████▌     | 45764917/100000000 [00:14<00:25, 2106794.57tokens/s]
Shard 8:  46%|████▌     | 45982547/100000000 [00:14<00:27, 1975798.24tokens/s]
Shard 8:  46%|████▌     | 46224374/100000000 [00:14<00:25, 2080801.29tokens/s]
Shard 8:  46%|████▋     | 46468550/100000000 [00:15<00:24, 2166234.78tokens/s]
Shard 8:  47%|████▋     | 46828157/100000000 [00:15<00:20, 2554456.68tokens/s]
Shard 8:  47%|████▋     | 47091604/100000000 [00:15<00:20, 2571881.88tokens/s]
Shard 8:  47%|████▋     | 47354082/100000000 [00:15<00:21, 2403843.38tokens/s]
Shard 8:  48%|████▊     | 47600345/100000000 [00:15<00:22, 2321926.51tokens/s]
Shard 8:  48%|████▊     | 47837186/100000000 [00:15<00:23, 2204773.79tokens/s]
Shard 8:  48%|████▊     | 48061338/100000000 [00:15<00:26, 1963270.06tokens/s]
Shard 8:  48%|████▊     | 48292155/100000000 [00:15<00:25, 2050947.87tokens/s]
Shard 8:  49%|████▊     | 48557898/100000000 [00:15<00:23, 2211952.30tokens/s]
Shard 8:  49%|████▉     | 48810107/100000000 [00:16<00:22, 2288921.10tokens/s]
Shard 8:  49%|████▉     | 49095637/100000000 [00:16<00:20, 2446783.73tokens/s]
Shard 8:  49%|████▉     | 49443039/100000000 [00:16<00:18, 2728100.99tokens/s]
Shard 8:  50%|████▉     | 49776092/100000000 [00:16<00:17, 2890921.76tokens/s]
Shard 8:  50%|█████     | 50097384/100000000 [00:16<00:16, 2973471.03tokens/s]
Shard 8:  50%|█████     | 50397974/100000000 [00:16<00:16, 2962819.99tokens/s]
Shard 8:  51%|█████     | 50795642/100000000 [00:16<00:15, 3253835.03tokens/s]
Shard 8:  51%|█████     | 51125542/100000000 [00:16<00:15, 3117584.42tokens/s]
Shard 8:  51%|█████▏    | 51440435/100000000 [00:16<00:15, 3096278.73tokens/s]
Shard 8:  52%|█████▏    | 51795836/100000000 [00:16<00:15, 3202372.91tokens/s]
Shard 8:  52%|█████▏    | 52117506/100000000 [00:17<00:16, 2865730.73tokens/s]
Shard 8:  52%|█████▏    | 52451887/100000000 [00:17<00:15, 2987606.32tokens/s]
Shard 8:  53%|█████▎    | 52756790/100000000 [00:17<00:16, 2846148.98tokens/s]
Shard 8:  53%|█████▎    | 53060410/100000000 [00:17<00:19, 2442162.55tokens/s]
Shard 8:  53%|█████▎    | 53317966/100000000 [00:17<00:21, 2198001.66tokens/s]
Shard 8:  54%|█████▎    | 53549172/100000000 [00:17<00:22, 2073438.37tokens/s]
Shard 8:  54%|█████▍    | 53999990/100000000 [00:17<00:17, 2623892.41tokens/s]
Shard 8:  54%|█████▍    | 54278068/100000000 [00:17<00:17, 2595806.48tokens/s]
Shard 8:  55%|█████▍    | 54558780/100000000 [00:18<00:19, 2336595.46tokens/s]
Shard 8:  55%|█████▍    | 54923062/100000000 [00:18<00:17, 2605323.49tokens/s]
Shard 8:  55%|█████▌    | 55195855/100000000 [00:18<00:17, 2622146.31tokens/s]
Shard 8:  55%|█████▌    | 55476527/100000000 [00:18<00:16, 2663448.96tokens/s]
Shard 8:  56%|█████▌    | 55749795/100000000 [00:18<00:16, 2643299.27tokens/s]
Shard 8:  56%|█████▌    | 56018696/100000000 [00:18<00:17, 2529453.69tokens/s]
Shard 8:  56%|█████▋    | 56378385/100000000 [00:18<00:15, 2803376.21tokens/s]
Shard 8:  57%|█████▋    | 56737815/100000000 [00:18<00:14, 3002138.88tokens/s]
Shard 8:  57%|█████▋    | 57091653/100000000 [00:18<00:13, 3110948.64tokens/s]
Shard 8:  57%|█████▋    | 57406029/100000000 [00:19<00:14, 3006327.95tokens/s]
Shard 8:  58%|█████▊    | 57722897/100000000 [00:19<00:13, 3046705.08tokens/s]
Shard 8:  58%|█████▊    | 58029664/100000000 [00:19<00:15, 2758527.05tokens/s]
Shard 8:  58%|█████▊    | 58316482/100000000 [00:19<00:15, 2775246.90tokens/s]
Shard 8:  59%|█████▉    | 59043449/100000000 [00:19<00:10, 4020308.27tokens/s]
Shard 8:  60%|█████▉    | 59527613/100000000 [00:19<00:09, 4240694.39tokens/s]
Shard 8:  60%|█████▉    | 59961721/100000000 [00:19<00:10, 3991143.27tokens/s]
Shard 8:  60%|██████    | 60379153/100000000 [00:19<00:09, 3990414.01tokens/s]
Shard 8:  61%|██████    | 60838326/100000000 [00:19<00:09, 4150279.81tokens/s]
Shard 8:  62%|██████▏   | 61957243/100000000 [00:20<00:06, 6162071.64tokens/s]
Shard 8:  63%|██████▎   | 62590498/100000000 [00:20<00:06, 5406651.03tokens/s]
Shard 8:  63%|██████▎   | 63156912/100000000 [00:20<00:08, 4596651.46tokens/s]
Shard 8:  64%|██████▎   | 63652175/100000000 [00:20<00:09, 3857159.73tokens/s]
Shard 8:  64%|██████▍   | 64079641/100000000 [00:20<00:10, 3579079.50tokens/s]
Shard 8:  64%|██████▍   | 64466391/100000000 [00:20<00:11, 3052592.44tokens/s]
Shard 8:  65%|██████▍   | 64857098/100000000 [00:21<00:10, 3209117.83tokens/s]
Shard 8:  65%|██████▌   | 65214853/100000000 [00:21<00:10, 3289971.37tokens/s]
Shard 8:  66%|██████▌   | 65565135/100000000 [00:21<00:10, 3209813.96tokens/s]
Shard 8:  66%|██████▌   | 65902038/100000000 [00:21<00:10, 3178305.92tokens/s]
Shard 8:  66%|██████▌   | 66241876/100000000 [00:21<00:10, 3229489.85tokens/s]
Shard 8:  67%|██████▋   | 66571989/100000000 [00:21<00:10, 3085716.93tokens/s]
Shard 8:  67%|██████▋   | 66901168/100000000 [00:21<00:10, 3110618.32tokens/s]
Shard 8:  67%|██████▋   | 67261690/100000000 [00:21<00:10, 3239711.38tokens/s]
Shard 8:  68%|██████▊   | 67722003/100000000 [00:21<00:08, 3616764.13tokens/s]
Shard 8:  68%|██████▊   | 68092544/100000000 [00:21<00:09, 3543809.12tokens/s]
Shard 8:  68%|██████▊   | 68451254/100000000 [00:22<00:09, 3273340.04tokens/s]
Shard 8:  69%|██████▉   | 68931790/100000000 [00:22<00:08, 3665157.10tokens/s]
Shard 8:  69%|██████▉   | 69307823/100000000 [00:22<00:08, 3674723.91tokens/s]
Shard 8:  70%|██████▉   | 69774547/100000000 [00:22<00:07, 3910377.55tokens/s]
Shard 8:  70%|███████   | 70171013/100000000 [00:22<00:08, 3319498.09tokens/s]
Shard 8:  71%|███████   | 70555866/100000000 [00:22<00:08, 3438308.76tokens/s]
Shard 8:  71%|███████   | 70915287/100000000 [00:22<00:08, 3417090.77tokens/s]
Shard 8:  72%|███████▏  | 71538658/100000000 [00:22<00:06, 4169526.12tokens/s]
Shard 8:  72%|███████▏  | 71969338/100000000 [00:23<00:08, 3123742.60tokens/s]
Shard 8:  72%|███████▏  | 72328270/100000000 [00:23<00:09, 2940731.22tokens/s]
Shard 8:  73%|███████▎  | 72674834/100000000 [00:23<00:08, 3045535.81tokens/s]
Shard 8:  73%|███████▎  | 73006507/100000000 [00:23<00:09, 2931217.60tokens/s]
Shard 8:  73%|███████▎  | 73317471/100000000 [00:23<00:09, 2749243.15tokens/s]
Shard 8:  74%|███████▎  | 73605244/100000000 [00:23<00:10, 2552554.55tokens/s]
Shard 8:  74%|███████▍  | 73870019/100000000 [00:23<00:10, 2384332.76tokens/s]
Shard 8:  74%|███████▍  | 74203014/100000000 [00:23<00:09, 2591494.09tokens/s]
Shard 8:  75%|███████▍  | 74533835/100000000 [00:24<00:09, 2745652.42tokens/s]
Shard 8:  75%|███████▍  | 74816495/100000000 [00:24<00:09, 2721729.12tokens/s]
Shard 8:  75%|███████▌  | 75232522/100000000 [00:24<00:07, 3109515.76tokens/s]
Shard 8:  76%|███████▌  | 75716459/100000000 [00:24<00:06, 3537505.38tokens/s]
Shard 8:  76%|███████▌  | 76077926/100000000 [00:24<00:06, 3537710.30tokens/s]
Shard 8:  76%|███████▋  | 76436803/100000000 [00:24<00:07, 3363936.26tokens/s]
Shard 8:  77%|███████▋  | 77176031/100000000 [00:24<00:05, 4472081.00tokens/s]
Shard 8:  78%|███████▊  | 77634779/100000000 [00:24<00:05, 4171142.56tokens/s]
Shard 8:  78%|███████▊  | 78063403/100000000 [00:24<00:05, 4111769.09tokens/s]
Shard 8:  78%|███████▊  | 78481952/100000000 [00:25<00:05, 3623350.00tokens/s]
Shard 8:  79%|███████▉  | 78953190/100000000 [00:25<00:05, 3859316.46tokens/s]
Shard 8:  79%|███████▉  | 79351984/100000000 [00:25<00:05, 3776602.50tokens/s]
Shard 8:  80%|███████▉  | 79784420/100000000 [00:25<00:05, 3923076.29tokens/s]
Shard 8:  80%|████████  | 80184658/100000000 [00:25<00:05, 3562998.47tokens/s]
Shard 8:  81%|████████  | 80645297/100000000 [00:25<00:05, 3825866.42tokens/s]
Shard 8:  81%|████████  | 81039792/100000000 [00:25<00:05, 3742658.05tokens/s]
Shard 8:  81%|████████▏ | 81421703/100000000 [00:25<00:04, 3755341.43tokens/s]
Shard 8:  82%|████████▏ | 81805098/100000000 [00:26<00:05, 3294948.18tokens/s]
Shard 8:  82%|████████▏ | 82147469/100000000 [00:26<00:05, 3247324.87tokens/s]
Shard 8:  83%|████████▎ | 82617754/100000000 [00:26<00:04, 3626224.03tokens/s]
Shard 8:  83%|████████▎ | 83072201/100000000 [00:26<00:04, 3873778.58tokens/s]
Shard 8:  84%|████████▍ | 84117066/100000000 [00:26<00:02, 5722112.41tokens/s]
Shard 8:  85%|████████▍ | 84710199/100000000 [00:26<00:03, 4819932.43tokens/s]
Shard 8:  85%|████████▌ | 85228954/100000000 [00:26<00:03, 4032726.91tokens/s]
Shard 8:  86%|████████▌ | 85676600/100000000 [00:26<00:03, 3895759.90tokens/s]
Shard 8:  86%|████████▌ | 86095347/100000000 [00:27<00:03, 3889438.34tokens/s]
Shard 8:  87%|████████▋ | 86504879/100000000 [00:27<00:03, 3519542.52tokens/s]
Shard 8:  87%|████████▋ | 86874726/100000000 [00:27<00:03, 3560877.20tokens/s]
Shard 8:  87%|████████▋ | 87250453/100000000 [00:27<00:03, 3233017.16tokens/s]
Shard 8:  88%|████████▊ | 87586611/100000000 [00:27<00:03, 3173140.18tokens/s]
Shard 8:  88%|████████▊ | 87967600/100000000 [00:27<00:03, 3313821.89tokens/s]
Shard 8:  88%|████████▊ | 88357296/100000000 [00:27<00:03, 3463456.31tokens/s]
Shard 8:  89%|████████▉ | 88786196/100000000 [00:27<00:03, 3690297.68tokens/s]
Shard 8:  89%|████████▉ | 89363433/100000000 [00:27<00:02, 4271155.36tokens/s]
Shard 8:  90%|████████▉ | 89828250/100000000 [00:28<00:02, 4376054.64tokens/s]
Shard 8:  90%|█████████ | 90273797/100000000 [00:28<00:02, 4170958.85tokens/s]
Shard 8:  91%|█████████ | 90777573/100000000 [00:28<00:02, 4415562.68tokens/s]
Shard 8:  91%|█████████ | 91226708/100000000 [00:28<00:02, 4349417.28tokens/s]
Shard 8:  92%|█████████▏| 91666530/100000000 [00:28<00:02, 4160371.61tokens/s]
Shard 8:  92%|█████████▏| 92087012/100000000 [00:28<00:01, 3987074.22tokens/s]
Shard 8:  93%|█████████▎| 92520548/100000000 [00:28<00:01, 4028326.63tokens/s]
Shard 8:  93%|█████████▎| 92938991/100000000 [00:28<00:01, 4049691.33tokens/s]
Shard 8:  93%|█████████▎| 93346238/100000000 [00:28<00:01, 4031951.45tokens/s]
Shard 8:  94%|█████████▍| 93966180/100000000 [00:29<00:01, 4649849.65tokens/s]
Shard 8:  94%|█████████▍| 94435003/100000000 [00:29<00:01, 4235813.15tokens/s]
Shard 8:  95%|█████████▍| 94867400/100000000 [00:29<00:01, 3763842.38tokens/s]
Shard 8:  95%|█████████▌| 95264126/100000000 [00:29<00:01, 3712444.48tokens/s]
Shard 8:  96%|█████████▌| 95720646/100000000 [00:29<00:01, 3929782.33tokens/s]
Shard 8:  96%|█████████▌| 96178950/100000000 [00:29<00:00, 4035747.44tokens/s]
Shard 8:  97%|█████████▋| 96589833/100000000 [00:29<00:00, 3918533.61tokens/s]
Shard 8:  97%|█████████▋| 97170625/100000000 [00:29<00:00, 4403848.06tokens/s]
Shard 8:  98%|█████████▊| 97617676/100000000 [00:29<00:00, 4019904.43tokens/s]
Shard 8:  98%|█████████▊| 98154020/100000000 [00:30<00:00, 4328476.54tokens/s]
Shard 8:  99%|█████████▊| 98610378/100000000 [00:30<00:00, 4343668.58tokens/s]
Shard 8:  99%|█████████▉| 99231241/100000000 [00:30<00:00, 4842931.51tokens/s]
Shard 8: 100%|█████████▉| 99723999/100000000 [00:30<00:00, 4195936.28tokens/s]
Shard 8: 100%|█████████▉| 99999442/100000000 [00:38<00:00, 2615278.37tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000008.bin (200,001,024 bytes) in the gpt-2 format

Shard 9:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 9:   0%|          | 196908/100000000 [00:00<00:50, 1960495.39tokens/s]
Shard 9:   0%|          | 465530/100000000 [00:00<00:41, 2381471.00tokens/s]
Shard 9:   1%|          | 722159/100000000 [00:00<00:40, 2453242.30tokens/s]
Shard 9:   1%|          | 1028718/100000000 [00:00<00:36, 2684314.67tokens/s]
Shard 9:   1%|▏         | 1474611/100000000 [00:00<00:29, 3316928.78tokens/s]
Shard 9:   2%|▏         | 1823860/100000000 [00:00<00:29, 3353827.87tokens/s]
Shard 9:   2%|▏         | 2369007/100000000 [00:00<00:24, 3996518.73tokens/s]
Shard 9:   3%|▎         | 2770931/100000000 [00:00<00:25, 3875812.65tokens/s]
Shard 9:   3%|▎         | 3289502/100000000 [00:00<00:22, 4260786.09tokens/s]
Shard 9:   4%|▎         | 3719867/100000000 [00:01<00:23, 4126296.58tokens/s]
Shard 9:   4%|▍         | 4194281/100000000 [00:01<00:22, 4304494.99tokens/s]
Shard 9:   5%|▍         | 4662684/100000000 [00:01<00:21, 4414638.89tokens/s]
Shard 9:   6%|▌         | 5537799/100000000 [00:01<00:16, 5663473.88tokens/s]
Shard 9:   6%|▋         | 6292212/100000000 [00:01<00:15, 6219189.19tokens/s]
Shard 9:   7%|▋         | 6933673/100000000 [00:01<00:14, 6276811.15tokens/s]
Shard 9:   8%|▊         | 7566898/100000000 [00:01<00:15, 5834287.22tokens/s]
Shard 9:   8%|▊         | 8157741/100000000 [00:01<00:16, 5600794.57tokens/s]
Shard 9:   9%|▉         | 8844696/100000000 [00:01<00:15, 5934742.31tokens/s]
Shard 9:   9%|▉         | 9490108/100000000 [00:01<00:14, 6039565.36tokens/s]
Shard 9:  10%|█         | 10295609/100000000 [00:02<00:13, 6571902.84tokens/s]
Shard 9:  11%|█         | 10957947/100000000 [00:02<00:14, 6002021.84tokens/s]
Shard 9:  12%|█▏        | 11570290/100000000 [00:02<00:15, 5884954.36tokens/s]
Shard 9:  12%|█▏        | 12167154/100000000 [00:02<00:15, 5650711.19tokens/s]
Shard 9:  13%|█▎        | 12741832/100000000 [00:02<00:16, 5323315.80tokens/s]
Shard 9:  13%|█▎        | 13283962/100000000 [00:02<00:17, 4967675.17tokens/s]
Shard 9:  14%|█▍        | 13805181/100000000 [00:02<00:17, 5029005.43tokens/s]
Shard 9:  14%|█▍        | 14318623/100000000 [00:02<00:17, 4975186.82tokens/s]
Shard 9:  15%|█▍        | 14821130/100000000 [00:03<00:18, 4727807.71tokens/s]
Shard 9:  15%|█▌        | 15298598/100000000 [00:03<00:18, 4569124.54tokens/s]
Shard 9:  16%|█▌        | 15758373/100000000 [00:03<00:20, 4171196.37tokens/s]
Shard 9:  16%|█▌        | 16181942/100000000 [00:03<00:21, 3863197.97tokens/s]
Shard 9:  17%|█▋        | 16731162/100000000 [00:03<00:19, 4279135.09tokens/s]
Shard 9:  17%|█▋        | 17237491/100000000 [00:03<00:18, 4467004.21tokens/s]
Shard 9:  18%|█▊        | 17694779/100000000 [00:03<00:20, 3963122.15tokens/s]
Shard 9:  18%|█▊        | 18107266/100000000 [00:03<00:21, 3768501.38tokens/s]
Shard 9:  19%|█▊        | 18572566/100000000 [00:03<00:20, 3983551.64tokens/s]
Shard 9:  19%|█▉        | 18993750/100000000 [00:04<00:20, 4000082.20tokens/s]
Shard 9:  19%|█▉        | 19401976/100000000 [00:04<00:20, 3963951.75tokens/s]
Shard 9:  20%|█▉        | 19880780/100000000 [00:04<00:19, 4189100.07tokens/s]
Shard 9:  20%|██        | 20305763/100000000 [00:04<00:21, 3747673.62tokens/s]
Shard 9:  21%|██        | 20692570/100000000 [00:04<00:21, 3725015.21tokens/s]
Shard 9:  21%|██        | 21073107/100000000 [00:04<00:21, 3618345.91tokens/s]
Shard 9:  21%|██▏       | 21462290/100000000 [00:04<00:21, 3692423.86tokens/s]
Shard 9:  22%|██▏       | 21836775/100000000 [00:04<00:22, 3461189.86tokens/s]
Shard 9:  22%|██▏       | 22319766/100000000 [00:04<00:20, 3833263.72tokens/s]
Shard 9:  23%|██▎       | 22866403/100000000 [00:05<00:18, 4244891.61tokens/s]
Shard 9:  23%|██▎       | 23389474/100000000 [00:05<00:17, 4497344.14tokens/s]
Shard 9:  24%|██▍       | 23846438/100000000 [00:05<00:17, 4477669.03tokens/s]
Shard 9:  24%|██▍       | 24323111/100000000 [00:05<00:16, 4561100.92tokens/s]
Shard 9:  25%|██▍       | 24782616/100000000 [00:05<00:18, 4050095.40tokens/s]
Shard 9:  25%|██▌       | 25200640/100000000 [00:05<00:18, 4040562.70tokens/s]
Shard 9:  26%|██▌       | 25613359/100000000 [00:05<00:18, 4034620.41tokens/s]
Shard 9:  26%|██▌       | 26023590/100000000 [00:05<00:18, 4051239.57tokens/s]
Shard 9:  26%|██▋       | 26433086/100000000 [00:05<00:19, 3770941.19tokens/s]
Shard 9:  27%|██▋       | 26816366/100000000 [00:06<00:19, 3693595.04tokens/s]
Shard 9:  27%|██▋       | 27191701/100000000 [00:06<00:21, 3335794.51tokens/s]
Shard 9:  28%|██▊       | 27533768/100000000 [00:06<00:22, 3241835.37tokens/s]
Shard 9:  28%|██▊       | 27863205/100000000 [00:06<00:24, 2947322.05tokens/s]
Shard 9:  28%|██▊       | 28173096/100000000 [00:06<00:24, 2983319.06tokens/s]
Shard 9:  29%|██▊       | 28568011/100000000 [00:06<00:22, 3237136.62tokens/s]
Shard 9:  29%|██▉       | 28899030/100000000 [00:06<00:23, 3019556.26tokens/s]
Shard 9:  29%|██▉       | 29386780/100000000 [00:06<00:20, 3504232.52tokens/s]
Shard 9:  30%|██▉       | 29749341/100000000 [00:07<00:21, 3326915.17tokens/s]
Shard 9:  30%|███       | 30103029/100000000 [00:07<00:25, 2778161.30tokens/s]
Shard 9:  30%|███       | 30470945/100000000 [00:07<00:23, 2935325.27tokens/s]
Shard 9:  31%|███       | 31104586/100000000 [00:07<00:18, 3787813.58tokens/s]
Shard 9:  32%|███▏      | 31620842/100000000 [00:07<00:16, 4135793.64tokens/s]
Shard 9:  32%|███▏      | 32059374/100000000 [00:07<00:19, 3464822.66tokens/s]
Shard 9:  32%|███▏      | 32439409/100000000 [00:07<00:20, 3353726.27tokens/s]
Shard 9:  33%|███▎      | 32797548/100000000 [00:07<00:21, 3145228.31tokens/s]
Shard 9:  33%|███▎      | 33128651/100000000 [00:08<00:22, 2994537.07tokens/s]
Shard 9:  34%|███▎      | 33520892/100000000 [00:08<00:20, 3223010.58tokens/s]
Shard 9:  34%|███▍      | 33855985/100000000 [00:08<00:20, 3164775.49tokens/s]
Shard 9:  34%|███▍      | 34408617/100000000 [00:08<00:17, 3779218.18tokens/s]
Shard 9:  35%|███▍      | 34799884/100000000 [00:08<00:17, 3665908.36tokens/s]
Shard 9:  35%|███▌      | 35184047/100000000 [00:08<00:18, 3459242.53tokens/s]
Shard 9:  36%|███▌      | 35564691/100000000 [00:08<00:18, 3508971.45tokens/s]
Shard 9:  36%|███▌      | 36035265/100000000 [00:08<00:16, 3795850.33tokens/s]
Shard 9:  36%|███▋      | 36494612/100000000 [00:08<00:15, 4013448.45tokens/s]
Shard 9:  37%|███▋      | 36903892/100000000 [00:09<00:17, 3660344.35tokens/s]
Shard 9:  37%|███▋      | 37289488/100000000 [00:09<00:16, 3707856.22tokens/s]
Shard 9:  38%|███▊      | 37667353/100000000 [00:09<00:16, 3709993.00tokens/s]
Shard 9:  38%|███▊      | 38045547/100000000 [00:09<00:16, 3682118.26tokens/s]
Shard 9:  38%|███▊      | 38472375/100000000 [00:09<00:16, 3843336.45tokens/s]
Shard 9:  39%|███▉      | 38859866/100000000 [00:09<00:17, 3588090.20tokens/s]
Shard 9:  39%|███▉      | 39349927/100000000 [00:09<00:15, 3932633.67tokens/s]
Shard 9:  40%|███▉      | 39749569/100000000 [00:09<00:15, 3938844.02tokens/s]
Shard 9:  40%|████      | 40149551/100000000 [00:09<00:15, 3942790.67tokens/s]
Shard 9:  41%|████      | 40610080/100000000 [00:10<00:14, 4111215.79tokens/s]
Shard 9:  41%|████▏     | 41307712/100000000 [00:10<00:11, 4909428.58tokens/s]
Shard 9:  42%|████▏     | 41848757/100000000 [00:10<00:11, 5037533.90tokens/s]
Shard 9:  42%|████▏     | 42357047/100000000 [00:10<00:12, 4730866.54tokens/s]
Shard 9:  43%|████▎     | 42835870/100000000 [00:10<00:12, 4514879.94tokens/s]
Shard 9:  43%|████▎     | 43291929/100000000 [00:10<00:13, 4114197.54tokens/s]
Shard 9:  44%|████▎     | 43745202/100000000 [00:10<00:14, 3843188.09tokens/s]
Shard 9:  44%|████▍     | 44137933/100000000 [00:10<00:15, 3565361.44tokens/s]
Shard 9:  45%|████▍     | 44504022/100000000 [00:10<00:15, 3555152.47tokens/s]
Shard 9:  45%|████▍     | 44871857/100000000 [00:11<00:15, 3582878.05tokens/s]
Shard 9:  45%|████▌     | 45273924/100000000 [00:11<00:14, 3701593.22tokens/s]
Shard 9:  46%|████▌     | 45649022/100000000 [00:11<00:14, 3649883.02tokens/s]
Shard 9:  46%|████▌     | 46017072/100000000 [00:11<00:14, 3616783.81tokens/s]
Shard 9:  46%|████▋     | 46381576/100000000 [00:11<00:14, 3576084.05tokens/s]
Shard 9:  47%|████▋     | 46773526/100000000 [00:11<00:14, 3664037.38tokens/s]
Shard 9:  47%|████▋     | 47162590/100000000 [00:11<00:14, 3713446.54tokens/s]
Shard 9:  48%|████▊     | 47535341/100000000 [00:11<00:14, 3514142.97tokens/s]
Shard 9:  48%|████▊     | 47889266/100000000 [00:11<00:15, 3297921.14tokens/s]
Shard 9:  48%|████▊     | 48408175/100000000 [00:12<00:13, 3807689.42tokens/s]
Shard 9:  49%|████▉     | 48795913/100000000 [00:12<00:13, 3689011.48tokens/s]
Shard 9:  49%|████▉     | 49296887/100000000 [00:12<00:12, 4050480.78tokens/s]
Shard 9:  50%|████▉     | 49714057/100000000 [00:12<00:13, 3837787.16tokens/s]
Shard 9:  50%|█████     | 50105346/100000000 [00:12<00:12, 3858170.42tokens/s]
Shard 9:  50%|█████     | 50495849/100000000 [00:12<00:13, 3747060.18tokens/s]
Shard 9:  51%|█████     | 50939754/100000000 [00:12<00:12, 3941704.67tokens/s]
Shard 9:  51%|█████▏    | 51338523/100000000 [00:12<00:13, 3722619.04tokens/s]
Shard 9:  52%|█████▏    | 51715523/100000000 [00:12<00:12, 3721880.60tokens/s]
Shard 9:  52%|█████▏    | 52177008/100000000 [00:13<00:12, 3966906.26tokens/s]
Shard 9:  53%|█████▎    | 52614304/100000000 [00:13<00:11, 4082078.14tokens/s]
Shard 9:  53%|█████▎    | 53026557/100000000 [00:13<00:11, 3934094.41tokens/s]
Shard 9:  53%|█████▎    | 53457178/100000000 [00:13<00:11, 3998234.26tokens/s]
Shard 9:  54%|█████▍    | 53862262/100000000 [00:13<00:11, 3873133.42tokens/s]
Shard 9:  54%|█████▍    | 54253865/100000000 [00:13<00:11, 3871368.98tokens/s]
Shard 9:  55%|█████▍    | 54647294/100000000 [00:13<00:13, 3316225.64tokens/s]
Shard 9:  55%|█████▌    | 55003078/100000000 [00:13<00:13, 3378272.39tokens/s]
Shard 9:  55%|█████▌    | 55383007/100000000 [00:13<00:12, 3488652.10tokens/s]
Shard 9:  56%|█████▌    | 55774234/100000000 [00:13<00:12, 3579843.15tokens/s]
Shard 9:  56%|█████▌    | 56175130/100000000 [00:14<00:11, 3687924.20tokens/s]
Shard 9:  57%|█████▋    | 56550332/100000000 [00:14<00:12, 3406006.50tokens/s]
Shard 9:  57%|█████▋    | 56899024/100000000 [00:14<00:12, 3420736.04tokens/s]
Shard 9:  57%|█████▋    | 57410431/100000000 [00:14<00:10, 3888945.32tokens/s]
Shard 9:  58%|█████▊    | 57954214/100000000 [00:14<00:09, 4296318.87tokens/s]
Shard 9:  58%|█████▊    | 58391765/100000000 [00:14<00:09, 4199301.18tokens/s]
Shard 9:  59%|█████▉    | 58816936/100000000 [00:14<00:11, 3512951.31tokens/s]
Shard 9:  59%|█████▉    | 59286639/100000000 [00:14<00:10, 3795260.70tokens/s]
Shard 9:  60%|█████▉    | 59685374/100000000 [00:15<00:10, 3843002.79tokens/s]
Shard 9:  60%|██████    | 60084212/100000000 [00:15<00:10, 3639682.31tokens/s]
Shard 9:  60%|██████    | 60459775/100000000 [00:15<00:11, 3378267.52tokens/s]
Shard 9:  61%|██████    | 60833644/100000000 [00:15<00:11, 3469445.07tokens/s]
Shard 9:  61%|██████▏   | 61276356/100000000 [00:15<00:10, 3719753.08tokens/s]
Shard 9:  62%|██████▏   | 61657305/100000000 [00:15<00:11, 3465336.80tokens/s]
Shard 9:  62%|██████▏   | 62132427/100000000 [00:15<00:09, 3811687.35tokens/s]
Shard 9:  63%|██████▎   | 62523570/100000000 [00:15<00:10, 3534026.66tokens/s]
Shard 9:  63%|██████▎   | 62886873/100000000 [00:15<00:11, 3347083.55tokens/s]
Shard 9:  63%|██████▎   | 63229209/100000000 [00:16<00:11, 3228135.82tokens/s]
Shard 9:  64%|██████▎   | 63557705/100000000 [00:16<00:11, 3232258.00tokens/s]
Shard 9:  64%|██████▍   | 64181615/100000000 [00:16<00:08, 4057196.32tokens/s]
Shard 9:  65%|██████▍   | 64721424/100000000 [00:16<00:07, 4429205.02tokens/s]
Shard 9:  65%|██████▌   | 65261614/100000000 [00:16<00:07, 4697963.82tokens/s]
Shard 9:  66%|██████▌   | 65739967/100000000 [00:16<00:08, 4077914.48tokens/s]
Shard 9:  66%|██████▌   | 66187443/100000000 [00:16<00:08, 4174295.29tokens/s]
Shard 9:  67%|██████▋   | 66622747/100000000 [00:16<00:08, 3806503.24tokens/s]
Shard 9:  67%|██████▋   | 67019171/100000000 [00:16<00:08, 3827286.05tokens/s]
Shard 9:  67%|██████▋   | 67491727/100000000 [00:17<00:07, 4063535.91tokens/s]
Shard 9:  68%|██████▊   | 67908510/100000000 [00:17<00:08, 3817307.58tokens/s]
Shard 9:  68%|██████▊   | 68304912/100000000 [00:17<00:09, 3420953.77tokens/s]
Shard 9:  69%|██████▊   | 68660728/100000000 [00:17<00:09, 3295471.75tokens/s]
Shard 9:  69%|██████▉   | 69005061/100000000 [00:17<00:10, 3016655.32tokens/s]
Shard 9:  69%|██████▉   | 69342291/100000000 [00:17<00:09, 3102990.73tokens/s]
Shard 9:  70%|██████▉   | 69661203/100000000 [00:17<00:10, 2889827.82tokens/s]
Shard 9:  70%|██████▉   | 69956656/100000000 [00:17<00:10, 2797214.60tokens/s]
Shard 9:  70%|███████   | 70325163/100000000 [00:18<00:09, 3026736.30tokens/s]
Shard 9:  71%|███████   | 70727615/100000000 [00:18<00:08, 3293094.34tokens/s]
Shard 9:  71%|███████   | 71160365/100000000 [00:18<00:08, 3568813.81tokens/s]
Shard 9:  72%|███████▏  | 71523945/100000000 [00:18<00:08, 3424661.41tokens/s]
Shard 9:  72%|███████▏  | 71875645/100000000 [00:18<00:08, 3440714.06tokens/s]
Shard 9:  72%|███████▏  | 72244945/100000000 [00:18<00:08, 3461768.65tokens/s]
Shard 9:  73%|███████▎  | 72635408/100000000 [00:18<00:07, 3572094.89tokens/s]
Shard 9:  73%|███████▎  | 72996807/100000000 [00:18<00:07, 3442803.57tokens/s]
Shard 9:  73%|███████▎  | 73428843/100000000 [00:18<00:07, 3673396.61tokens/s]
Shard 9:  74%|███████▍  | 73833217/100000000 [00:18<00:06, 3776059.70tokens/s]
Shard 9:  74%|███████▍  | 74213132/100000000 [00:19<00:07, 3629783.86tokens/s]
Shard 9:  75%|███████▍  | 74587986/100000000 [00:19<00:07, 3477317.79tokens/s]
Shard 9:  75%|███████▌  | 75112244/100000000 [00:19<00:06, 3961057.73tokens/s]
Shard 9:  76%|███████▌  | 75567191/100000000 [00:19<00:05, 4115861.72tokens/s]
Shard 9:  76%|███████▌  | 75983987/100000000 [00:19<00:05, 4048484.31tokens/s]
Shard 9:  76%|███████▋  | 76476296/100000000 [00:19<00:05, 4234249.00tokens/s]
Shard 9:  77%|███████▋  | 76902059/100000000 [00:19<00:05, 4058217.79tokens/s]
Shard 9:  77%|███████▋  | 77310685/100000000 [00:19<00:05, 3950429.62tokens/s]
Shard 9:  78%|███████▊  | 77708712/100000000 [00:20<00:06, 3636976.85tokens/s]
Shard 9:  78%|███████▊  | 78078003/100000000 [00:20<00:06, 3439671.87tokens/s]
Shard 9:  79%|███████▊  | 78678651/100000000 [00:20<00:05, 4116452.29tokens/s]
Shard 9:  79%|███████▉  | 79170722/100000000 [00:20<00:05, 3927981.45tokens/s]
Shard 9:  80%|███████▉  | 79575670/100000000 [00:20<00:05, 3959880.12tokens/s]
Shard 9:  80%|████████  | 80013299/100000000 [00:20<00:04, 4031478.42tokens/s]
Shard 9:  81%|████████  | 80593668/100000000 [00:20<00:04, 4516424.90tokens/s]
Shard 9:  81%|████████  | 81053574/100000000 [00:20<00:04, 4269711.64tokens/s]
Shard 9:  81%|████████▏ | 81487898/100000000 [00:20<00:04, 4287245.52tokens/s]
Shard 9:  82%|████████▏ | 81922005/100000000 [00:21<00:04, 4033302.54tokens/s]
Shard 9:  83%|████████▎ | 82530492/100000000 [00:21<00:03, 4549550.11tokens/s]
Shard 9:  83%|████████▎ | 82994290/100000000 [00:21<00:04, 3695598.74tokens/s]
Shard 9:  83%|████████▎ | 83431867/100000000 [00:21<00:04, 3859878.93tokens/s]
Shard 9:  84%|████████▍ | 83842384/100000000 [00:21<00:04, 3794318.71tokens/s]
Shard 9:  84%|████████▍ | 84241034/100000000 [00:21<00:04, 3746919.83tokens/s]
Shard 9:  85%|████████▍ | 84627054/100000000 [00:21<00:04, 3344742.69tokens/s]
Shard 9:  86%|████████▌ | 85678844/100000000 [00:21<00:02, 5147425.55tokens/s]
Shard 9:  86%|████████▌ | 86235085/100000000 [00:22<00:02, 4816374.41tokens/s]
Shard 9:  87%|████████▋ | 86749068/100000000 [00:22<00:03, 3751173.63tokens/s]
Shard 9:  87%|████████▋ | 87179159/100000000 [00:22<00:03, 3824292.38tokens/s]
Shard 9:  88%|████████▊ | 87608687/100000000 [00:22<00:03, 3503400.71tokens/s]
Shard 9:  88%|████████▊ | 87997703/100000000 [00:22<00:03, 3341273.30tokens/s]
Shard 9:  88%|████████▊ | 88352618/100000000 [00:22<00:03, 3231971.98tokens/s]
Shard 9:  89%|████████▉ | 88812724/100000000 [00:22<00:03, 3563111.60tokens/s]
Shard 9:  89%|████████▉ | 89186426/100000000 [00:22<00:03, 3466946.21tokens/s]
Shard 9:  90%|████████▉ | 89545742/100000000 [00:23<00:03, 3375948.37tokens/s]
Shard 9:  90%|████████▉ | 89998020/100000000 [00:23<00:02, 3669165.09tokens/s]
Shard 9:  90%|█████████ | 90375927/100000000 [00:23<00:02, 3674851.92tokens/s]
Shard 9:  91%|█████████ | 90786251/100000000 [00:23<00:02, 3793398.22tokens/s]
Shard 9:  91%|█████████ | 91173770/100000000 [00:23<00:02, 3328720.13tokens/s]
Shard 9:  92%|█████████▏| 91660941/100000000 [00:23<00:02, 3731585.52tokens/s]
Shard 9:  92%|█████████▏| 92049140/100000000 [00:23<00:02, 3215271.82tokens/s]
Shard 9:  92%|█████████▏| 92392054/100000000 [00:23<00:02, 3157897.16tokens/s]
Shard 9:  93%|█████████▎| 92780068/100000000 [00:24<00:02, 3322120.21tokens/s]
Shard 9:  94%|█████████▎| 93519250/100000000 [00:24<00:01, 4364344.56tokens/s]
Shard 9:  94%|█████████▍| 94031838/100000000 [00:24<00:01, 4555653.05tokens/s]
Shard 9:  95%|█████████▍| 94502433/100000000 [00:24<00:01, 3922974.67tokens/s]
Shard 9:  95%|█████████▍| 94922607/100000000 [00:24<00:01, 3926255.88tokens/s]
Shard 9:  95%|█████████▌| 95411802/100000000 [00:24<00:01, 4167230.00tokens/s]
Shard 9:  96%|█████████▌| 95953609/100000000 [00:24<00:00, 4496809.82tokens/s]
Shard 9:  96%|█████████▋| 96417125/100000000 [00:24<00:00, 4232923.83tokens/s]
Shard 9:  97%|█████████▋| 96852311/100000000 [00:24<00:00, 3778002.70tokens/s]
Shard 9:  98%|█████████▊| 97834335/100000000 [00:25<00:00, 5316808.44tokens/s]
Shard 9:  98%|█████████▊| 98402993/100000000 [00:25<00:00, 5119409.72tokens/s]
Shard 9:  99%|█████████▉| 98944480/100000000 [00:25<00:00, 4493467.55tokens/s]
Shard 9:  99%|█████████▉| 99424073/100000000 [00:25<00:00, 4186470.32tokens/s]
Shard 9: 100%|█████████▉| 99865886/100000000 [00:25<00:00, 3981355.62tokens/s]
Shard 9: 100%|█████████▉| 99997998/100000000 [00:33<00:00, 2954017.75tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000009.bin (200,001,024 bytes) in the gpt-2 format

Shard 10:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 10:   0%|          | 366517/100000000 [00:00<00:27, 3637654.53tokens/s]
Shard 10:   1%|          | 731211/100000000 [00:00<00:38, 2552845.99tokens/s]
Shard 10:   1%|          | 1016822/100000000 [00:00<00:37, 2647180.88tokens/s]
Shard 10:   1%|▏         | 1294053/100000000 [00:00<00:37, 2602593.13tokens/s]
Shard 10:   2%|▏         | 1573898/100000000 [00:00<00:38, 2527180.34tokens/s]
Shard 10:   2%|▏         | 1831152/100000000 [00:00<00:41, 2387968.04tokens/s]
Shard 10:   2%|▏         | 2177177/100000000 [00:00<00:36, 2648898.11tokens/s]
Shard 10:   2%|▏         | 2458195/100000000 [00:00<00:36, 2680790.63tokens/s]
Shard 10:   3%|▎         | 2769411/100000000 [00:01<00:34, 2801171.42tokens/s]
Shard 10:   3%|▎         | 3063926/100000000 [00:01<00:34, 2788002.03tokens/s]
Shard 10:   4%|▎         | 3503177/100000000 [00:01<00:29, 3253655.85tokens/s]
Shard 10:   4%|▍         | 3847136/100000000 [00:01<00:29, 3289721.46tokens/s]
Shard 10:   4%|▍         | 4179767/100000000 [00:01<00:29, 3208766.07tokens/s]
Shard 10:   5%|▍         | 4523326/100000000 [00:01<00:29, 3267062.22tokens/s]
Shard 10:   5%|▍         | 4853914/100000000 [00:01<00:29, 3255718.10tokens/s]
Shard 10:   5%|▌         | 5180936/100000000 [00:01<00:30, 3127915.90tokens/s]
Shard 10:   6%|▌         | 5640072/100000000 [00:01<00:26, 3535243.67tokens/s]
Shard 10:   6%|▌         | 6008607/100000000 [00:01<00:26, 3516402.48tokens/s]
Shard 10:   6%|▋         | 6363114/100000000 [00:02<00:26, 3479334.31tokens/s]
Shard 10:   7%|▋         | 6712917/100000000 [00:02<00:28, 3312676.43tokens/s]
Shard 10:   7%|▋         | 7046660/100000000 [00:02<00:30, 3095837.17tokens/s]
Shard 10:   7%|▋         | 7380321/100000000 [00:02<00:29, 3129599.24tokens/s]
Shard 10:   8%|▊         | 7696201/100000000 [00:02<00:29, 3095274.50tokens/s]
Shard 10:   8%|▊         | 8033693/100000000 [00:02<00:29, 3141960.37tokens/s]
Shard 10:   8%|▊         | 8350222/100000000 [00:02<00:31, 2929337.73tokens/s]
Shard 10:   9%|▊         | 8646427/100000000 [00:02<00:32, 2808080.49tokens/s]
Shard 10:   9%|▉         | 8932678/100000000 [00:02<00:32, 2804310.45tokens/s]
Shard 10:   9%|▉         | 9218103/100000000 [00:03<00:35, 2587142.76tokens/s]
Shard 10:  10%|▉         | 9526745/100000000 [00:03<00:33, 2672491.87tokens/s]
Shard 10:  10%|▉         | 9797376/100000000 [00:03<00:34, 2604222.37tokens/s]
Shard 10:  10%|█         | 10103142/100000000 [00:03<00:33, 2715541.75tokens/s]
Shard 10:  10%|█         | 10379424/100000000 [00:03<00:32, 2726362.41tokens/s]
Shard 10:  11%|█         | 10676123/100000000 [00:03<00:32, 2790856.62tokens/s]
Shard 10:  11%|█         | 11047521/100000000 [00:03<00:29, 3055986.55tokens/s]
Shard 10:  11%|█▏        | 11355053/100000000 [00:03<00:31, 2784416.93tokens/s]
Shard 10:  12%|█▏        | 11690720/100000000 [00:03<00:30, 2933978.42tokens/s]
Shard 10:  12%|█▏        | 11989586/100000000 [00:04<00:33, 2617868.83tokens/s]
Shard 10:  12%|█▏        | 12262376/100000000 [00:04<00:33, 2615613.76tokens/s]
Shard 10:  13%|█▎        | 12550894/100000000 [00:04<00:32, 2680896.77tokens/s]
Shard 10:  13%|█▎        | 12824252/100000000 [00:04<00:33, 2627737.41tokens/s]
Shard 10:  13%|█▎        | 13159252/100000000 [00:04<00:30, 2823230.77tokens/s]
Shard 10:  14%|█▎        | 13542060/100000000 [00:04<00:28, 3077023.06tokens/s]
Shard 10:  14%|█▍        | 13859216/100000000 [00:04<00:29, 2941685.25tokens/s]
Shard 10:  14%|█▍        | 14345473/100000000 [00:04<00:24, 3453956.20tokens/s]
Shard 10:  15%|█▍        | 14731993/100000000 [00:04<00:23, 3559250.24tokens/s]
Shard 10:  15%|█▌        | 15122285/100000000 [00:05<00:23, 3654288.31tokens/s]
Shard 10:  16%|█▌        | 15678719/100000000 [00:05<00:20, 4199172.52tokens/s]
Shard 10:  16%|█▌        | 16103409/100000000 [00:05<00:19, 4212797.85tokens/s]
Shard 10:  17%|█▋        | 16527393/100000000 [00:05<00:21, 3959550.44tokens/s]
Shard 10:  17%|█▋        | 16929545/100000000 [00:05<00:22, 3766458.21tokens/s]
Shard 10:  17%|█▋        | 17311916/100000000 [00:05<00:23, 3446273.14tokens/s]
Shard 10:  18%|█▊        | 17663912/100000000 [00:05<00:24, 3303398.83tokens/s]
Shard 10:  18%|█▊        | 18000967/100000000 [00:05<00:25, 3228791.53tokens/s]
Shard 10:  18%|█▊        | 18331591/100000000 [00:05<00:26, 3127911.35tokens/s]
Shard 10:  19%|█▊        | 18646924/100000000 [00:06<00:26, 3118722.08tokens/s]
Shard 10:  19%|█▉        | 19011931/100000000 [00:06<00:25, 3210969.41tokens/s]
Shard 10:  19%|█▉        | 19403212/100000000 [00:06<00:24, 3348223.48tokens/s]
Shard 10:  20%|█▉        | 19754438/100000000 [00:06<00:23, 3387355.92tokens/s]
Shard 10:  20%|██        | 20103088/100000000 [00:06<00:25, 3119576.66tokens/s]
Shard 10:  20%|██        | 20422584/100000000 [00:06<00:29, 2676601.33tokens/s]
Shard 10:  21%|██        | 20748924/100000000 [00:06<00:28, 2822699.69tokens/s]
Shard 10:  21%|██        | 21044404/100000000 [00:06<00:30, 2619873.68tokens/s]
Shard 10:  21%|██▏       | 21343029/100000000 [00:07<00:29, 2709876.52tokens/s]
Shard 10:  22%|██▏       | 21664759/100000000 [00:07<00:27, 2811089.35tokens/s]
Shard 10:  22%|██▏       | 21952801/100000000 [00:07<00:27, 2827751.86tokens/s]
Shard 10:  22%|██▏       | 22296964/100000000 [00:07<00:25, 2991971.70tokens/s]
Shard 10:  23%|██▎       | 22655284/100000000 [00:07<00:24, 3154412.46tokens/s]
Shard 10:  23%|██▎       | 23060182/100000000 [00:07<00:22, 3407875.80tokens/s]
Shard 10:  23%|██▎       | 23493004/100000000 [00:07<00:20, 3665413.76tokens/s]
Shard 10:  24%|██▍       | 23863572/100000000 [00:07<00:21, 3514029.63tokens/s]
Shard 10:  24%|██▍       | 24229097/100000000 [00:07<00:21, 3552119.50tokens/s]
Shard 10:  25%|██▍       | 24600729/100000000 [00:07<00:20, 3593356.47tokens/s]
Shard 10:  25%|██▍       | 24984591/100000000 [00:08<00:20, 3654829.19tokens/s]
Shard 10:  26%|██▌       | 25555880/100000000 [00:08<00:17, 4259313.85tokens/s]
Shard 10:  26%|██▌       | 25984243/100000000 [00:08<00:18, 4095075.54tokens/s]
Shard 10:  26%|██▋       | 26396689/100000000 [00:08<00:19, 3706957.47tokens/s]
Shard 10:  27%|██▋       | 26775529/100000000 [00:08<00:19, 3703787.49tokens/s]
Shard 10:  27%|██▋       | 27152199/100000000 [00:08<00:21, 3429785.88tokens/s]
Shard 10:  28%|██▊       | 27502926/100000000 [00:08<00:21, 3321958.06tokens/s]
Shard 10:  28%|██▊       | 27840070/100000000 [00:08<00:22, 3138716.59tokens/s]
Shard 10:  28%|██▊       | 28166768/100000000 [00:08<00:22, 3148448.26tokens/s]
Shard 10:  29%|██▊       | 28641187/100000000 [00:09<00:20, 3535610.99tokens/s]
Shard 10:  29%|██▉       | 28999421/100000000 [00:09<00:23, 3084082.73tokens/s]
Shard 10:  29%|██▉       | 29323824/100000000 [00:09<00:22, 3120222.40tokens/s]
Shard 10:  30%|██▉       | 29772550/100000000 [00:09<00:20, 3483208.13tokens/s]
Shard 10:  30%|███       | 30138813/100000000 [00:09<00:20, 3413986.76tokens/s]
Shard 10:  30%|███       | 30490005/100000000 [00:09<00:21, 3212957.20tokens/s]
Shard 10:  31%|███       | 31116853/100000000 [00:09<00:17, 4026122.16tokens/s]
Shard 10:  32%|███▏      | 32453389/100000000 [00:09<00:10, 6604992.72tokens/s]
Shard 10:  33%|███▎      | 33143912/100000000 [00:10<00:10, 6159293.76tokens/s]
Shard 10:  34%|███▍      | 33785777/100000000 [00:10<00:13, 4913373.40tokens/s]
Shard 10:  34%|███▍      | 34332811/100000000 [00:10<00:14, 4388181.71tokens/s]
Shard 10:  35%|███▍      | 34815266/100000000 [00:10<00:15, 4302559.35tokens/s]
Shard 10:  36%|███▌      | 35527991/100000000 [00:10<00:13, 4944154.07tokens/s]
Shard 10:  36%|███▌      | 36058902/100000000 [00:10<00:14, 4437849.45tokens/s]
Shard 10:  37%|███▋      | 36534308/100000000 [00:10<00:15, 4192125.25tokens/s]
Shard 10:  37%|███▋      | 36975325/100000000 [00:11<00:16, 3898375.41tokens/s]
Shard 10:  37%|███▋      | 37384286/100000000 [00:11<00:16, 3912545.50tokens/s]
Shard 10:  38%|███▊      | 38073340/100000000 [00:11<00:13, 4669811.64tokens/s]
Shard 10:  39%|███▊      | 38561669/100000000 [00:11<00:14, 4235978.00tokens/s]
Shard 10:  39%|███▉      | 39008340/100000000 [00:11<00:16, 3713983.04tokens/s]
Shard 10:  39%|███▉      | 39421307/100000000 [00:11<00:15, 3790763.61tokens/s]
Shard 10:  40%|███▉      | 39819184/100000000 [00:11<00:16, 3650285.63tokens/s]
Shard 10:  40%|████      | 40260165/100000000 [00:11<00:15, 3829770.99tokens/s]
Shard 10:  41%|████      | 40781584/100000000 [00:11<00:14, 4198310.60tokens/s]
Shard 10:  41%|████      | 41214195/100000000 [00:12<00:14, 4190756.15tokens/s]
Shard 10:  42%|████▏     | 41642260/100000000 [00:12<00:15, 3876088.30tokens/s]
Shard 10:  42%|████▏     | 42045645/100000000 [00:12<00:14, 3914451.16tokens/s]
Shard 10:  42%|████▏     | 42444353/100000000 [00:12<00:15, 3776358.23tokens/s]
Shard 10:  43%|████▎     | 43074082/100000000 [00:12<00:12, 4457873.95tokens/s]
Shard 10:  44%|████▎     | 43529869/100000000 [00:12<00:13, 4262524.70tokens/s]
Shard 10:  44%|████▍     | 43964158/100000000 [00:12<00:13, 4171079.59tokens/s]
Shard 10:  44%|████▍     | 44449269/100000000 [00:12<00:12, 4355094.38tokens/s]
Shard 10:  45%|████▍     | 44890395/100000000 [00:12<00:13, 4182184.02tokens/s]
Shard 10:  45%|████▌     | 45313311/100000000 [00:13<00:13, 4135666.08tokens/s]
Shard 10:  46%|████▌     | 45753051/100000000 [00:13<00:12, 4205754.12tokens/s]
Shard 10:  47%|████▋     | 46501521/100000000 [00:13<00:10, 5111366.07tokens/s]
Shard 10:  47%|████▋     | 47016710/100000000 [00:13<00:12, 4123580.32tokens/s]
Shard 10:  47%|████▋     | 47462197/100000000 [00:13<00:15, 3498871.34tokens/s]
Shard 10:  48%|████▊     | 47873368/100000000 [00:13<00:14, 3626886.52tokens/s]
Shard 10:  48%|████▊     | 48301723/100000000 [00:13<00:13, 3776402.06tokens/s]
Shard 10:  49%|████▉     | 48775194/100000000 [00:13<00:12, 4009801.29tokens/s]
Shard 10:  49%|████▉     | 49196135/100000000 [00:14<00:13, 3730826.64tokens/s]
Shard 10:  50%|████▉     | 49609478/100000000 [00:14<00:13, 3728979.77tokens/s]
Shard 10:  50%|████▉     | 49994845/100000000 [00:14<00:14, 3553295.12tokens/s]
Shard 10:  50%|█████     | 50358639/100000000 [00:14<00:14, 3476359.80tokens/s]
Shard 10:  51%|█████     | 50712964/100000000 [00:14<00:18, 2707346.64tokens/s]
Shard 10:  51%|█████     | 51011699/100000000 [00:14<00:18, 2657542.65tokens/s]
Shard 10:  51%|█████▏    | 51316382/100000000 [00:14<00:17, 2749652.89tokens/s]
Shard 10:  52%|█████▏    | 51623520/100000000 [00:14<00:17, 2829101.21tokens/s]
Shard 10:  52%|█████▏    | 52001768/100000000 [00:15<00:15, 3057223.51tokens/s]
Shard 10:  52%|█████▏    | 52334897/100000000 [00:15<00:15, 3073626.65tokens/s]
Shard 10:  53%|█████▎    | 52696300/100000000 [00:15<00:14, 3203497.60tokens/s]
Shard 10:  53%|█████▎    | 53229380/100000000 [00:15<00:12, 3793769.16tokens/s]
Shard 10:  54%|█████▎    | 53616477/100000000 [00:15<00:12, 3772841.27tokens/s]
Shard 10:  54%|█████▍    | 54000070/100000000 [00:15<00:13, 3417576.38tokens/s]
Shard 10:  54%|█████▍    | 54351280/100000000 [00:15<00:13, 3426031.20tokens/s]
Shard 10:  55%|█████▍    | 54716409/100000000 [00:15<00:12, 3485627.42tokens/s]
Shard 10:  55%|█████▌    | 55155677/100000000 [00:15<00:11, 3743189.78tokens/s]
Shard 10:  56%|█████▌    | 55535716/100000000 [00:15<00:12, 3598376.44tokens/s]
Shard 10:  56%|█████▌    | 55957023/100000000 [00:16<00:11, 3765710.68tokens/s]
Shard 10:  56%|█████▋    | 56499428/100000000 [00:16<00:10, 4236569.26tokens/s]
Shard 10:  57%|█████▋    | 56928885/100000000 [00:16<00:10, 4000360.14tokens/s]
Shard 10:  57%|█████▋    | 57407889/100000000 [00:16<00:10, 4219356.60tokens/s]
Shard 10:  58%|█████▊    | 57907316/100000000 [00:16<00:09, 4427578.88tokens/s]
Shard 10:  58%|█████▊    | 58407124/100000000 [00:16<00:09, 4592652.33tokens/s]
Shard 10:  59%|█████▉    | 58946471/100000000 [00:16<00:08, 4795185.13tokens/s]
Shard 10:  59%|█████▉    | 59431249/100000000 [00:16<00:08, 4545913.86tokens/s]
Shard 10:  60%|█████▉    | 59932423/100000000 [00:16<00:08, 4660156.01tokens/s]
Shard 10:  60%|██████    | 60402121/100000000 [00:17<00:10, 3847047.19tokens/s]
Shard 10:  61%|██████    | 60812131/100000000 [00:17<00:10, 3668965.07tokens/s]
Shard 10:  61%|██████    | 61202518/100000000 [00:17<00:11, 3512068.63tokens/s]
Shard 10:  62%|██████▏   | 61596692/100000000 [00:17<00:10, 3618543.12tokens/s]
Shard 10:  62%|██████▏   | 61969319/100000000 [00:17<00:10, 3510699.49tokens/s]
Shard 10:  62%|██████▏   | 62327492/100000000 [00:17<00:11, 3179641.54tokens/s]
Shard 10:  63%|██████▎   | 62663690/100000000 [00:17<00:11, 3217814.21tokens/s]
Shard 10:  63%|██████▎   | 63002215/100000000 [00:17<00:11, 3250349.74tokens/s]
Shard 10:  63%|██████▎   | 63333070/100000000 [00:18<00:11, 3079957.07tokens/s]
Shard 10:  64%|██████▎   | 63695745/100000000 [00:18<00:11, 3210471.18tokens/s]
Shard 10:  64%|██████▍   | 64022173/100000000 [00:18<00:11, 3187383.24tokens/s]
Shard 10:  65%|██████▍   | 64998139/100000000 [00:18<00:06, 5020680.44tokens/s]
Shard 10:  66%|██████▌   | 65512804/100000000 [00:18<00:07, 4555693.61tokens/s]
Shard 10:  66%|██████▌   | 65988894/100000000 [00:18<00:08, 4237260.11tokens/s]
Shard 10:  66%|██████▋   | 66426823/100000000 [00:18<00:08, 4076485.54tokens/s]
Shard 10:  67%|██████▋   | 66889004/100000000 [00:18<00:07, 4215740.06tokens/s]
Shard 10:  67%|██████▋   | 67319380/100000000 [00:18<00:07, 4140852.05tokens/s]
Shard 10:  68%|██████▊   | 67740571/100000000 [00:19<00:07, 4080783.57tokens/s]
Shard 10:  68%|██████▊   | 68152587/100000000 [00:19<00:08, 3754777.18tokens/s]
Shard 10:  69%|██████▊   | 68534400/100000000 [00:19<00:08, 3535356.08tokens/s]
Shard 10:  69%|██████▉   | 68910245/100000000 [00:19<00:08, 3593214.73tokens/s]
Shard 10:  69%|██████▉   | 69301421/100000000 [00:19<00:08, 3649246.08tokens/s]
Shard 10:  70%|██████▉   | 69734759/100000000 [00:19<00:07, 3839494.14tokens/s]
Shard 10:  70%|███████   | 70123354/100000000 [00:19<00:08, 3642493.36tokens/s]
Shard 10:  70%|███████   | 70492746/100000000 [00:19<00:08, 3280230.10tokens/s]
Shard 10:  71%|███████   | 71041655/100000000 [00:19<00:07, 3851306.45tokens/s]
Shard 10:  71%|███████▏  | 71440653/100000000 [00:20<00:07, 3591053.36tokens/s]
Shard 10:  72%|███████▏  | 71812655/100000000 [00:20<00:07, 3613176.92tokens/s]
Shard 10:  72%|███████▏  | 72183201/100000000 [00:20<00:08, 3227424.21tokens/s]
Shard 10:  73%|███████▎  | 72523634/100000000 [00:20<00:08, 3255569.31tokens/s]
Shard 10:  73%|███████▎  | 72858058/100000000 [00:20<00:08, 3157460.55tokens/s]
Shard 10:  73%|███████▎  | 73180188/100000000 [00:20<00:08, 3080205.83tokens/s]
Shard 10:  73%|███████▎  | 73492338/100000000 [00:20<00:08, 2946150.45tokens/s]
Shard 10:  74%|███████▍  | 73841572/100000000 [00:20<00:08, 3079051.27tokens/s]
Shard 10:  74%|███████▍  | 74354703/100000000 [00:21<00:07, 3637424.67tokens/s]
Shard 10:  75%|███████▍  | 74785604/100000000 [00:21<00:06, 3796818.99tokens/s]
Shard 10:  75%|███████▌  | 75266470/100000000 [00:21<00:06, 4071573.57tokens/s]
Shard 10:  76%|███████▌  | 75684727/100000000 [00:21<00:06, 3843909.86tokens/s]
Shard 10:  76%|███████▌  | 76079120/100000000 [00:21<00:06, 3668377.67tokens/s]
Shard 10:  76%|███████▋  | 76450997/100000000 [00:21<00:07, 3362384.82tokens/s]
Shard 10:  77%|███████▋  | 76796227/100000000 [00:21<00:07, 3131169.17tokens/s]
Shard 10:  77%|███████▋  | 77153278/100000000 [00:21<00:07, 3243665.68tokens/s]
Shard 10:  78%|███████▊  | 77601176/100000000 [00:21<00:06, 3574858.16tokens/s]
Shard 10:  78%|███████▊  | 77966962/100000000 [00:22<00:06, 3235276.05tokens/s]
Shard 10:  78%|███████▊  | 78302042/100000000 [00:22<00:07, 3021352.47tokens/s]
Shard 10:  79%|███████▊  | 78678708/100000000 [00:22<00:06, 3189542.27tokens/s]
Shard 10:  79%|███████▉  | 79049451/100000000 [00:22<00:06, 3327557.13tokens/s]
Shard 10:  79%|███████▉  | 79389811/100000000 [00:22<00:06, 3260405.34tokens/s]
Shard 10:  80%|███████▉  | 79807272/100000000 [00:22<00:05, 3499859.46tokens/s]
Shard 10:  80%|████████  | 80253592/100000000 [00:22<00:05, 3761117.12tokens/s]
Shard 10:  81%|████████  | 80636242/100000000 [00:22<00:05, 3674822.05tokens/s]
Shard 10:  81%|████████  | 81007418/100000000 [00:22<00:05, 3475207.56tokens/s]
Shard 10:  81%|████████▏ | 81359662/100000000 [00:23<00:05, 3485351.49tokens/s]
Shard 10:  82%|████████▏ | 81711449/100000000 [00:23<00:05, 3282536.68tokens/s]
Shard 10:  82%|████████▏ | 82058207/100000000 [00:23<00:06, 2986623.56tokens/s]
Shard 10:  82%|████████▏ | 82364005/100000000 [00:23<00:06, 2873695.90tokens/s]
Shard 10:  83%|████████▎ | 82732721/100000000 [00:23<00:05, 3087436.26tokens/s]
Shard 10:  83%|████████▎ | 83048436/100000000 [00:23<00:05, 3072922.94tokens/s]
Shard 10:  83%|████████▎ | 83440992/100000000 [00:23<00:05, 3277053.10tokens/s]
Shard 10:  84%|████████▍ | 83844509/100000000 [00:23<00:04, 3476786.58tokens/s]
Shard 10:  84%|████████▍ | 84196379/100000000 [00:23<00:04, 3445856.26tokens/s]
Shard 10:  85%|████████▍ | 84669029/100000000 [00:24<00:04, 3805682.84tokens/s]
Shard 10:  85%|████████▌ | 85084958/100000000 [00:24<00:03, 3906401.69tokens/s]
Shard 10:  85%|████████▌ | 85479463/100000000 [00:24<00:03, 3722330.06tokens/s]
Shard 10:  86%|████████▌ | 86127256/100000000 [00:24<00:03, 4468824.00tokens/s]
Shard 10:  87%|████████▋ | 86579973/100000000 [00:24<00:03, 4173518.16tokens/s]
Shard 10:  87%|████████▋ | 87412740/100000000 [00:24<00:02, 5298958.39tokens/s]
Shard 10:  88%|████████▊ | 87956089/100000000 [00:24<00:02, 4713652.75tokens/s]
Shard 10:  88%|████████▊ | 88447047/100000000 [00:24<00:02, 3982669.53tokens/s]
Shard 10:  89%|████████▉ | 88876174/100000000 [00:25<00:02, 3742002.32tokens/s]
Shard 10:  89%|████████▉ | 89271136/100000000 [00:25<00:02, 3682934.82tokens/s]
Shard 10:  90%|████████▉ | 89696131/100000000 [00:25<00:02, 3759125.04tokens/s]
Shard 10:  90%|█████████ | 90142917/100000000 [00:25<00:02, 3939606.46tokens/s]
Shard 10:  91%|█████████ | 90547587/100000000 [00:25<00:02, 3662817.67tokens/s]
Shard 10:  91%|█████████ | 91027481/100000000 [00:25<00:02, 3930826.54tokens/s]
Shard 10:  91%|█████████▏| 91430534/100000000 [00:25<00:02, 3939205.40tokens/s]
Shard 10:  92%|█████████▏| 91831636/100000000 [00:25<00:02, 3640815.61tokens/s]
Shard 10:  92%|█████████▏| 92211460/100000000 [00:25<00:02, 3679859.30tokens/s]
Shard 10:  93%|█████████▎| 92586141/100000000 [00:26<00:02, 3500181.81tokens/s]
Shard 10:  93%|█████████▎| 93162218/100000000 [00:26<00:01, 4110619.34tokens/s]
Shard 10:  94%|█████████▎| 93584081/100000000 [00:26<00:01, 3968369.70tokens/s]
Shard 10:  94%|█████████▍| 93988424/100000000 [00:26<00:01, 3837591.12tokens/s]
Shard 10:  94%|█████████▍| 94408575/100000000 [00:26<00:01, 3930340.66tokens/s]
Shard 10:  95%|█████████▌| 95116785/100000000 [00:26<00:01, 4816823.69tokens/s]
Shard 10:  96%|█████████▌| 95607291/100000000 [00:26<00:01, 4258858.96tokens/s]
Shard 10:  96%|█████████▋| 96266603/100000000 [00:26<00:00, 4845872.03tokens/s]
Shard 10:  97%|█████████▋| 96769971/100000000 [00:26<00:00, 4555161.17tokens/s]
Shard 10:  97%|█████████▋| 97412216/100000000 [00:27<00:00, 5024929.56tokens/s]
Shard 10:  98%|█████████▊| 97930392/100000000 [00:27<00:00, 4705860.07tokens/s]
Shard 10:  99%|█████████▊| 98551451/100000000 [00:27<00:00, 5081531.26tokens/s]
Shard 10:  99%|█████████▉| 99073200/100000000 [00:27<00:00, 5056986.40tokens/s]
Shard 10: 100%|█████████▉| 99715314/100000000 [00:27<00:00, 5422158.07tokens/s]
Shard 10: 100%|█████████▉| 99999983/100000000 [00:36<00:00, 2763706.64tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000010.bin (200,001,024 bytes) in the gpt-2 format

Shard 11:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 11:   1%|          | 746622/100000000 [00:00<00:13, 7312149.50tokens/s]
Shard 11:   1%|▏         | 1478302/100000000 [00:00<00:26, 3667119.44tokens/s]
Shard 11:   2%|▏         | 1938591/100000000 [00:00<00:30, 3249218.46tokens/s]
Shard 11:   2%|▏         | 2310744/100000000 [00:00<00:29, 3268627.58tokens/s]
Shard 11:   3%|▎         | 2667159/100000000 [00:00<00:30, 3196271.26tokens/s]
Shard 11:   3%|▎         | 3055171/100000000 [00:00<00:28, 3375989.42tokens/s]
Shard 11:   3%|▎         | 3414832/100000000 [00:00<00:28, 3392188.42tokens/s]
Shard 11:   4%|▍         | 3765470/100000000 [00:01<00:28, 3391743.67tokens/s]
Shard 11:   4%|▍         | 4113241/100000000 [00:01<00:28, 3312697.24tokens/s]
Shard 11:   5%|▍         | 4505233/100000000 [00:01<00:27, 3479220.61tokens/s]
Shard 11:   5%|▍         | 4861762/100000000 [00:01<00:27, 3501767.74tokens/s]
Shard 11:   5%|▌         | 5352665/100000000 [00:01<00:24, 3899066.24tokens/s]
Shard 11:   6%|▌         | 5748192/100000000 [00:01<00:25, 3680461.72tokens/s]
Shard 11:   6%|▌         | 6121294/100000000 [00:01<00:28, 3349073.75tokens/s]
Shard 11:   6%|▋         | 6465005/100000000 [00:01<00:28, 3306013.93tokens/s]
Shard 11:   7%|▋         | 6963850/100000000 [00:01<00:24, 3759240.69tokens/s]
Shard 11:   7%|▋         | 7408155/100000000 [00:02<00:23, 3947158.52tokens/s]
Shard 11:   8%|▊         | 7885235/100000000 [00:02<00:22, 4169868.43tokens/s]
Shard 11:   8%|▊         | 8355991/100000000 [00:02<00:21, 4308824.87tokens/s]
Shard 11:   9%|▉         | 8792456/100000000 [00:02<00:22, 4140090.39tokens/s]
Shard 11:   9%|▉         | 9210809/100000000 [00:02<00:23, 3923631.37tokens/s]
Shard 11:  10%|▉         | 9608542/100000000 [00:02<00:26, 3384842.40tokens/s]
Shard 11:  10%|█         | 10079326/100000000 [00:02<00:24, 3713258.51tokens/s]
Shard 11:  11%|█         | 10502421/100000000 [00:02<00:23, 3836321.11tokens/s]
Shard 11:  11%|█         | 10898913/100000000 [00:02<00:23, 3834532.89tokens/s]
Shard 11:  11%|█▏        | 11312405/100000000 [00:03<00:22, 3907992.18tokens/s]
Shard 11:  12%|█▏        | 11713564/100000000 [00:03<00:24, 3658959.53tokens/s]
Shard 11:  12%|█▏        | 12088409/100000000 [00:03<00:25, 3504438.38tokens/s]
Shard 11:  12%|█▏        | 12444565/100000000 [00:03<00:27, 3208167.43tokens/s]
Shard 11:  13%|█▎        | 12772548/100000000 [00:03<00:31, 2802524.41tokens/s]
Shard 11:  13%|█▎        | 13084022/100000000 [00:03<00:30, 2852542.66tokens/s]
Shard 11:  13%|█▎        | 13473691/100000000 [00:03<00:28, 3090065.70tokens/s]
Shard 11:  14%|█▍        | 13792585/100000000 [00:03<00:28, 3018774.97tokens/s]
Shard 11:  15%|█▍        | 14597724/100000000 [00:04<00:19, 4370799.77tokens/s]
Shard 11:  15%|█▌        | 15054190/100000000 [00:04<00:21, 3955584.02tokens/s]
Shard 11:  15%|█▌        | 15470551/100000000 [00:04<00:24, 3508217.87tokens/s]
Shard 11:  16%|█▌        | 15869311/100000000 [00:04<00:23, 3620525.58tokens/s]
Shard 11:  16%|█▌        | 16248358/100000000 [00:04<00:24, 3374747.94tokens/s]
Shard 11:  17%|█▋        | 16599180/100000000 [00:04<00:25, 3321742.81tokens/s]
Shard 11:  17%|█▋        | 16941125/100000000 [00:04<00:25, 3221323.57tokens/s]
Shard 11:  17%|█▋        | 17361798/100000000 [00:04<00:23, 3462098.02tokens/s]
Shard 11:  18%|█▊        | 17794552/100000000 [00:04<00:22, 3698496.60tokens/s]
Shard 11:  18%|█▊        | 18195706/100000000 [00:05<00:21, 3765479.17tokens/s]
Shard 11:  19%|█▊        | 18577809/100000000 [00:05<00:22, 3553071.97tokens/s]
Shard 11:  19%|█▉        | 18945844/100000000 [00:05<00:23, 3505804.51tokens/s]
Shard 11:  19%|█▉        | 19302085/100000000 [00:05<00:23, 3418097.35tokens/s]
Shard 11:  20%|█▉        | 19646682/100000000 [00:05<00:23, 3353308.84tokens/s]
Shard 11:  21%|██        | 20506237/100000000 [00:05<00:16, 4822362.46tokens/s]
Shard 11:  21%|██        | 20999896/100000000 [00:05<00:19, 4103345.54tokens/s]
Shard 11:  21%|██▏       | 21436190/100000000 [00:05<00:19, 4055463.79tokens/s]
Shard 11:  22%|██▏       | 21860584/100000000 [00:06<00:20, 3794671.94tokens/s]
Shard 11:  22%|██▏       | 22254849/100000000 [00:06<00:23, 3377976.93tokens/s]
Shard 11:  23%|██▎       | 22649113/100000000 [00:06<00:22, 3484643.72tokens/s]
Shard 11:  23%|██▎       | 23010726/100000000 [00:06<00:24, 3155435.10tokens/s]
Shard 11:  23%|██▎       | 23439907/100000000 [00:06<00:22, 3390606.55tokens/s]
Shard 11:  24%|██▍       | 23852037/100000000 [00:06<00:21, 3574052.49tokens/s]
Shard 11:  24%|██▍       | 24317502/100000000 [00:06<00:19, 3854216.65tokens/s]
Shard 11:  25%|██▍       | 24736869/100000000 [00:06<00:19, 3948327.43tokens/s]
Shard 11:  25%|██▌       | 25140185/100000000 [00:06<00:19, 3835510.66tokens/s]
Shard 11:  26%|██▌       | 25530419/100000000 [00:07<00:19, 3807147.50tokens/s]
Shard 11:  26%|██▌       | 25974129/100000000 [00:07<00:18, 3980748.51tokens/s]
Shard 11:  26%|██▋       | 26382287/100000000 [00:07<00:29, 2472609.33tokens/s]
Shard 11:  27%|██▋       | 26851787/100000000 [00:07<00:25, 2916161.85tokens/s]
Shard 11:  27%|██▋       | 27217625/100000000 [00:07<00:25, 2859105.50tokens/s]
Shard 11:  28%|██▊       | 27736860/100000000 [00:07<00:21, 3391482.78tokens/s]
Shard 11:  28%|██▊       | 28131329/100000000 [00:07<00:21, 3390398.16tokens/s]
Shard 11:  29%|██▊       | 28506330/100000000 [00:08<00:21, 3327097.96tokens/s]
Shard 11:  29%|██▉       | 28956996/100000000 [00:08<00:19, 3607465.62tokens/s]
Shard 11:  29%|██▉       | 29340274/100000000 [00:08<00:19, 3643591.44tokens/s]
Shard 11:  30%|██▉       | 29720728/100000000 [00:08<00:24, 2918387.27tokens/s]
Shard 11:  30%|███       | 30107477/100000000 [00:08<00:22, 3132708.19tokens/s]
Shard 11:  30%|███       | 30449144/100000000 [00:08<00:22, 3101046.09tokens/s]
Shard 11:  31%|███       | 30978142/100000000 [00:08<00:18, 3662921.97tokens/s]
Shard 11:  31%|███▏      | 31367950/100000000 [00:08<00:19, 3451530.26tokens/s]
Shard 11:  32%|███▏      | 31730373/100000000 [00:09<00:20, 3347029.71tokens/s]
Shard 11:  32%|███▏      | 32079641/100000000 [00:09<00:22, 3066122.40tokens/s]
Shard 11:  32%|███▏      | 32397636/100000000 [00:09<00:23, 2925862.18tokens/s]
Shard 11:  33%|███▎      | 32698111/100000000 [00:09<00:24, 2767607.78tokens/s]
Shard 11:  33%|███▎      | 32980912/100000000 [00:09<00:24, 2761718.22tokens/s]
Shard 11:  33%|███▎      | 33265787/100000000 [00:09<00:27, 2440104.15tokens/s]
Shard 11:  34%|███▎      | 33519059/100000000 [00:09<00:28, 2360129.02tokens/s]
Shard 11:  34%|███▍      | 33759968/100000000 [00:09<00:29, 2251560.86tokens/s]
Shard 11:  34%|███▍      | 34159736/100000000 [00:10<00:24, 2672520.84tokens/s]
Shard 11:  34%|███▍      | 34449535/100000000 [00:10<00:24, 2727903.72tokens/s]
Shard 11:  35%|███▍      | 34875510/100000000 [00:10<00:20, 3147300.85tokens/s]
Shard 11:  35%|███▌      | 35302603/100000000 [00:10<00:18, 3457231.56tokens/s]
Shard 11:  36%|███▌      | 36054062/100000000 [00:10<00:13, 4602534.22tokens/s]
Shard 11:  37%|███▋      | 36524469/100000000 [00:10<00:13, 4562902.09tokens/s]
Shard 11:  37%|███▋      | 36988541/100000000 [00:10<00:15, 4184647.69tokens/s]
Shard 11:  38%|███▊      | 37509118/100000000 [00:10<00:14, 4451516.57tokens/s]
Shard 11:  38%|███▊      | 37963941/100000000 [00:10<00:15, 3939046.26tokens/s]
Shard 11:  38%|███▊      | 38400096/100000000 [00:10<00:15, 4045587.33tokens/s]
Shard 11:  39%|███▉      | 38817632/100000000 [00:11<00:16, 3670784.00tokens/s]
Shard 11:  39%|███▉      | 39198736/100000000 [00:11<00:18, 3361710.67tokens/s]
Shard 11:  40%|███▉      | 39547456/100000000 [00:11<00:18, 3319462.41tokens/s]
Shard 11:  40%|███▉      | 39923591/100000000 [00:11<00:17, 3403300.50tokens/s]
Shard 11:  40%|████      | 40271001/100000000 [00:11<00:18, 3212668.10tokens/s]
Shard 11:  41%|████      | 40622447/100000000 [00:11<00:18, 3288384.29tokens/s]
Shard 11:  41%|████      | 40958205/100000000 [00:11<00:18, 3236989.17tokens/s]
Shard 11:  41%|████▏     | 41407676/100000000 [00:11<00:16, 3583344.16tokens/s]
Shard 11:  42%|████▏     | 41949007/100000000 [00:12<00:14, 4098565.56tokens/s]
Shard 11:  42%|████▏     | 42365465/100000000 [00:12<00:15, 3821156.11tokens/s]
Shard 11:  43%|████▎     | 42837684/100000000 [00:12<00:14, 4060815.92tokens/s]
Shard 11:  43%|████▎     | 43251416/100000000 [00:12<00:16, 3538754.58tokens/s]
Shard 11:  44%|████▎     | 43657438/100000000 [00:12<00:15, 3662107.28tokens/s]
Shard 11:  44%|████▍     | 44036633/100000000 [00:12<00:17, 3163558.95tokens/s]
Shard 11:  44%|████▍     | 44373263/100000000 [00:12<00:17, 3102864.56tokens/s]
Shard 11:  45%|████▍     | 44725993/100000000 [00:12<00:17, 3205178.84tokens/s]
Shard 11:  45%|████▌     | 45057861/100000000 [00:13<00:18, 3002342.82tokens/s]
Shard 11:  45%|████▌     | 45367117/100000000 [00:13<00:18, 2923234.02tokens/s]
Shard 11:  46%|████▌     | 45665295/100000000 [00:13<00:19, 2785895.40tokens/s]
Shard 11:  46%|████▌     | 45948348/100000000 [00:13<00:20, 2650249.90tokens/s]
Shard 11:  46%|████▌     | 46217752/100000000 [00:13<00:21, 2535652.98tokens/s]
Shard 11:  46%|████▋     | 46479770/100000000 [00:13<00:20, 2555966.72tokens/s]
Shard 11:  47%|████▋     | 46808970/100000000 [00:13<00:19, 2721235.59tokens/s]
Shard 11:  47%|████▋     | 47083281/100000000 [00:13<00:22, 2395038.19tokens/s]
Shard 11:  47%|████▋     | 47388024/100000000 [00:13<00:20, 2563864.40tokens/s]
Shard 11:  48%|████▊     | 47686620/100000000 [00:14<00:19, 2669656.07tokens/s]
Shard 11:  48%|████▊     | 48014706/100000000 [00:14<00:18, 2829538.40tokens/s]
Shard 11:  48%|████▊     | 48304146/100000000 [00:14<00:18, 2816067.46tokens/s]
Shard 11:  49%|████▊     | 48590233/100000000 [00:14<00:19, 2611471.65tokens/s]
Shard 11:  49%|████▉     | 48857306/100000000 [00:14<00:19, 2581273.80tokens/s]
Shard 11:  49%|████▉     | 49119014/100000000 [00:14<00:19, 2547120.89tokens/s]
Shard 11:  49%|████▉     | 49470058/100000000 [00:14<00:17, 2815471.64tokens/s]
Shard 11:  50%|████▉     | 49842797/100000000 [00:14<00:16, 2989613.03tokens/s]
Shard 11:  50%|█████     | 50253716/100000000 [00:14<00:15, 3289247.82tokens/s]
Shard 11:  51%|█████     | 50585776/100000000 [00:15<00:15, 3187442.21tokens/s]
Shard 11:  51%|█████     | 50990203/100000000 [00:15<00:14, 3424191.29tokens/s]
Shard 11:  51%|█████▏    | 51335480/100000000 [00:15<00:15, 3196134.65tokens/s]
Shard 11:  52%|█████▏    | 51661591/100000000 [00:15<00:15, 3173455.86tokens/s]
Shard 11:  52%|█████▏    | 51982323/100000000 [00:15<00:16, 2962897.42tokens/s]
Shard 11:  52%|█████▏    | 52283450/100000000 [00:15<00:16, 2975709.12tokens/s]
Shard 11:  53%|█████▎    | 52586100/100000000 [00:15<00:18, 2598200.58tokens/s]
Shard 11:  53%|█████▎    | 52884582/100000000 [00:15<00:17, 2692126.84tokens/s]
Shard 11:  53%|█████▎    | 53162267/100000000 [00:15<00:17, 2679899.94tokens/s]
Shard 11:  53%|█████▎    | 53446178/100000000 [00:16<00:17, 2698167.22tokens/s]
Shard 11:  54%|█████▎    | 53720064/100000000 [00:16<00:17, 2632559.51tokens/s]
Shard 11:  54%|█████▍    | 53986380/100000000 [00:16<00:17, 2615205.56tokens/s]
Shard 11:  54%|█████▍    | 54284169/100000000 [00:16<00:17, 2679905.63tokens/s]
Shard 11:  55%|█████▍    | 54643540/100000000 [00:16<00:15, 2932406.70tokens/s]
Shard 11:  55%|█████▍    | 54986236/100000000 [00:16<00:14, 3064744.33tokens/s]
Shard 11:  55%|█████▌    | 55295087/100000000 [00:16<00:15, 2961694.76tokens/s]
Shard 11:  56%|█████▌    | 55616216/100000000 [00:16<00:14, 3018544.49tokens/s]
Shard 11:  56%|█████▌    | 55919842/100000000 [00:16<00:15, 2872323.16tokens/s]
Shard 11:  56%|█████▌    | 56220849/100000000 [00:16<00:15, 2903535.75tokens/s]
Shard 11:  57%|█████▋    | 56718815/100000000 [00:17<00:12, 3487765.29tokens/s]
Shard 11:  57%|█████▋    | 57278076/100000000 [00:17<00:10, 4070416.27tokens/s]
Shard 11:  58%|█████▊    | 57688710/100000000 [00:17<00:11, 3638184.59tokens/s]
Shard 11:  58%|█████▊    | 58063794/100000000 [00:17<00:11, 3583504.13tokens/s]
Shard 11:  58%|█████▊    | 58429557/100000000 [00:17<00:11, 3589015.42tokens/s]
Shard 11:  59%|█████▉    | 58870545/100000000 [00:17<00:10, 3812607.44tokens/s]
Shard 11:  59%|█████▉    | 59279963/100000000 [00:17<00:10, 3874929.24tokens/s]
Shard 11:  60%|█████▉    | 59671349/100000000 [00:17<00:11, 3558464.24tokens/s]
Shard 11:  60%|██████    | 60035297/100000000 [00:18<00:13, 2973172.92tokens/s]
Shard 11:  60%|██████    | 60352050/100000000 [00:18<00:13, 3005654.39tokens/s]
Shard 11:  61%|██████    | 60748106/100000000 [00:18<00:12, 3251100.09tokens/s]
Shard 11:  61%|██████    | 61087562/100000000 [00:18<00:12, 3098082.37tokens/s]
Shard 11:  61%|██████▏   | 61437518/100000000 [00:18<00:12, 3192398.96tokens/s]
Shard 11:  62%|██████▏   | 61765704/100000000 [00:18<00:12, 2992825.50tokens/s]
Shard 11:  62%|██████▏   | 62153476/100000000 [00:18<00:11, 3214743.47tokens/s]
Shard 11:  63%|██████▎   | 62510797/100000000 [00:18<00:11, 3312465.48tokens/s]
Shard 11:  63%|██████▎   | 62848226/100000000 [00:18<00:12, 3076577.76tokens/s]
Shard 11:  63%|██████▎   | 63193795/100000000 [00:19<00:11, 3170064.00tokens/s]
Shard 11:  64%|██████▎   | 63516684/100000000 [00:19<00:12, 2925360.66tokens/s]
Shard 11:  64%|██████▍   | 63816227/100000000 [00:19<00:13, 2720162.87tokens/s]
Shard 11:  64%|██████▍   | 64149324/100000000 [00:19<00:12, 2850069.28tokens/s]
Shard 11:  65%|██████▍   | 64514772/100000000 [00:19<00:11, 3050214.45tokens/s]
Shard 11:  65%|██████▍   | 64825951/100000000 [00:19<00:11, 3028675.22tokens/s]
Shard 11:  65%|██████▌   | 65203966/100000000 [00:19<00:10, 3223345.84tokens/s]
Shard 11:  66%|██████▌   | 65943727/100000000 [00:19<00:07, 4413383.78tokens/s]
Shard 11:  66%|██████▋   | 66393888/100000000 [00:19<00:08, 4009395.37tokens/s]
Shard 11:  67%|██████▋   | 66817705/100000000 [00:20<00:09, 3676272.44tokens/s]
Shard 11:  67%|██████▋   | 67238541/100000000 [00:20<00:08, 3785066.38tokens/s]
Shard 11:  68%|██████▊   | 67628096/100000000 [00:20<00:09, 3439908.91tokens/s]
Shard 11:  68%|██████▊   | 67984464/100000000 [00:20<00:10, 2951490.76tokens/s]
Shard 11:  68%|██████▊   | 68297223/100000000 [00:20<00:10, 2953020.93tokens/s]
Shard 11:  69%|██████▊   | 68604852/100000000 [00:20<00:11, 2700599.77tokens/s]
Shard 11:  69%|██████▉   | 68957106/100000000 [00:20<00:10, 2902134.35tokens/s]
Shard 11:  69%|██████▉   | 69336690/100000000 [00:20<00:09, 3132239.29tokens/s]
Shard 11:  70%|██████▉   | 69765030/100000000 [00:21<00:08, 3440303.98tokens/s]
Shard 11:  70%|███████   | 70163827/100000000 [00:21<00:08, 3592700.06tokens/s]
Shard 11:  71%|███████   | 70546597/100000000 [00:21<00:09, 3214291.22tokens/s]
Shard 11:  71%|███████   | 70970701/100000000 [00:21<00:08, 3455549.19tokens/s]
Shard 11:  71%|███████▏  | 71359761/100000000 [00:21<00:08, 3554493.29tokens/s]
Shard 11:  72%|███████▏  | 71790405/100000000 [00:21<00:07, 3749057.68tokens/s]
Shard 11:  72%|███████▏  | 72174497/100000000 [00:21<00:07, 3722807.59tokens/s]
Shard 11:  73%|███████▎  | 72572969/100000000 [00:21<00:07, 3796940.87tokens/s]
Shard 11:  73%|███████▎  | 72957962/100000000 [00:21<00:07, 3536667.74tokens/s]
Shard 11:  73%|███████▎  | 73318354/100000000 [00:22<00:07, 3535907.33tokens/s]
Shard 11:  74%|███████▎  | 73676299/100000000 [00:22<00:08, 3162308.96tokens/s]
Shard 11:  74%|███████▍  | 74002043/100000000 [00:22<00:08, 2976549.70tokens/s]
Shard 11:  74%|███████▍  | 74312479/100000000 [00:22<00:08, 2924679.74tokens/s]
Shard 11:  75%|███████▍  | 74610145/100000000 [00:22<00:09, 2807113.24tokens/s]
Shard 11:  75%|███████▍  | 74894218/100000000 [00:22<00:09, 2596322.88tokens/s]
Shard 11:  75%|███████▌  | 75174393/100000000 [00:22<00:09, 2645047.26tokens/s]
Shard 11:  76%|███████▌  | 75527979/100000000 [00:22<00:08, 2884682.66tokens/s]
Shard 11:  76%|███████▌  | 75942097/100000000 [00:22<00:07, 3234411.52tokens/s]
Shard 11:  77%|███████▋  | 76665063/100000000 [00:23<00:05, 4364733.00tokens/s]
Shard 11:  77%|███████▋  | 77111901/100000000 [00:23<00:05, 3973869.90tokens/s]
Shard 11:  78%|███████▊  | 77648550/100000000 [00:23<00:05, 4346086.44tokens/s]
Shard 11:  78%|███████▊  | 78097812/100000000 [00:23<00:05, 4295058.17tokens/s]
Shard 11:  79%|███████▊  | 78538359/100000000 [00:23<00:05, 3635095.08tokens/s]
Shard 11:  79%|███████▉  | 78961215/100000000 [00:23<00:05, 3783317.46tokens/s]
Shard 11:  80%|███████▉  | 79565150/100000000 [00:23<00:04, 4378286.58tokens/s]
Shard 11:  80%|████████  | 80026700/100000000 [00:23<00:04, 4350356.62tokens/s]
Shard 11:  80%|████████  | 80476834/100000000 [00:24<00:05, 3719682.03tokens/s]
Shard 11:  81%|████████  | 80874255/100000000 [00:24<00:05, 3473122.73tokens/s]
Shard 11:  81%|████████▏ | 81284281/100000000 [00:24<00:05, 3618179.71tokens/s]
Shard 11:  82%|████████▏ | 81662067/100000000 [00:24<00:05, 3570524.13tokens/s]
Shard 11:  82%|████████▏ | 82043880/100000000 [00:24<00:04, 3626887.59tokens/s]
Shard 11:  82%|████████▏ | 82414834/100000000 [00:24<00:05, 3227171.09tokens/s]
Shard 11:  83%|████████▎ | 82787068/100000000 [00:24<00:05, 3344140.07tokens/s]
Shard 11:  83%|████████▎ | 83151645/100000000 [00:24<00:04, 3418087.44tokens/s]
Shard 11:  84%|████████▎ | 83503418/100000000 [00:24<00:04, 3328441.63tokens/s]
Shard 11:  84%|████████▍ | 83847158/100000000 [00:25<00:05, 3218261.85tokens/s]
Shard 11:  84%|████████▍ | 84223540/100000000 [00:25<00:04, 3328171.77tokens/s]
Shard 11:  85%|████████▍ | 84560457/100000000 [00:25<00:04, 3329725.79tokens/s]
Shard 11:  85%|████████▍ | 84896069/100000000 [00:25<00:04, 3233112.92tokens/s]
Shard 11:  85%|████████▌ | 85221439/100000000 [00:25<00:04, 3175707.30tokens/s]
Shard 11:  86%|████████▌ | 85544216/100000000 [00:25<00:04, 3091913.70tokens/s]
Shard 11:  86%|████████▌ | 85857573/100000000 [00:25<00:04, 3034888.83tokens/s]
Shard 11:  86%|████████▌ | 86224053/100000000 [00:25<00:04, 3209732.46tokens/s]
Shard 11:  87%|████████▋ | 86547617/100000000 [00:25<00:04, 3089186.52tokens/s]
Shard 11:  87%|████████▋ | 86901753/100000000 [00:26<00:04, 3205544.91tokens/s]
Shard 11:  87%|████████▋ | 87224482/100000000 [00:26<00:04, 3092684.88tokens/s]
Shard 11:  88%|████████▊ | 87535653/100000000 [00:26<00:04, 3063060.57tokens/s]
Shard 11:  88%|████████▊ | 87887384/100000000 [00:26<00:03, 3189166.89tokens/s]
Shard 11:  88%|████████▊ | 88208977/100000000 [00:26<00:03, 3043028.46tokens/s]
Shard 11:  89%|████████▊ | 88515399/100000000 [00:26<00:03, 2915982.41tokens/s]
Shard 11:  89%|████████▉ | 88816724/100000000 [00:26<00:03, 2869777.08tokens/s]
Shard 11:  89%|████████▉ | 89175330/100000000 [00:26<00:03, 3068111.36tokens/s]
Shard 11:  90%|████████▉ | 89575134/100000000 [00:26<00:03, 3332923.43tokens/s]
Shard 11:  90%|████████▉ | 89938700/100000000 [00:27<00:02, 3419078.98tokens/s]
Shard 11:  90%|█████████ | 90301199/100000000 [00:27<00:02, 3440744.05tokens/s]
Shard 11:  91%|█████████ | 90648217/100000000 [00:27<00:02, 3182189.13tokens/s]
Shard 11:  91%|█████████ | 91185954/100000000 [00:27<00:02, 3791984.08tokens/s]
Shard 11:  92%|█████████▏| 91632642/100000000 [00:27<00:02, 3956038.18tokens/s]
Shard 11:  92%|█████████▏| 92125896/100000000 [00:27<00:01, 4223984.27tokens/s]
Shard 11:  93%|█████████▎| 92556484/100000000 [00:27<00:01, 4058724.15tokens/s]
Shard 11:  93%|█████████▎| 92977575/100000000 [00:27<00:01, 3734068.78tokens/s]
Shard 11:  93%|█████████▎| 93365404/100000000 [00:27<00:01, 3765522.46tokens/s]
Shard 11:  94%|█████████▍| 93792167/100000000 [00:27<00:01, 3892334.40tokens/s]
Shard 11:  94%|█████████▍| 94186735/100000000 [00:28<00:01, 3874874.76tokens/s]
Shard 11:  95%|█████████▌| 95036086/100000000 [00:28<00:00, 5196348.14tokens/s]
Shard 11:  96%|█████████▌| 95565187/100000000 [00:28<00:00, 4735115.18tokens/s]
Shard 11:  96%|█████████▌| 96052901/100000000 [00:28<00:00, 4151692.81tokens/s]
Shard 11:  96%|█████████▋| 96488630/100000000 [00:28<00:00, 3649359.09tokens/s]
Shard 11:  97%|█████████▋| 96876357/100000000 [00:28<00:00, 3510564.57tokens/s]
Shard 11:  97%|█████████▋| 97241416/100000000 [00:28<00:00, 3266798.16tokens/s]
Shard 11:  98%|█████████▊| 97578294/100000000 [00:29<00:00, 3287979.02tokens/s]
Shard 11:  98%|█████████▊| 97915331/100000000 [00:29<00:00, 2855883.90tokens/s]
Shard 11:  98%|█████████▊| 98213618/100000000 [00:29<00:00, 2652131.64tokens/s]
Shard 11:  99%|█████████▊| 98503846/100000000 [00:29<00:00, 2707567.76tokens/s]
Shard 11:  99%|█████████▉| 98792533/100000000 [00:29<00:00, 2670651.80tokens/s]
Shard 11:  99%|█████████▉| 99090728/100000000 [00:29<00:00, 2747765.20tokens/s]
Shard 11:  99%|█████████▉| 99433114/100000000 [00:29<00:00, 2894447.88tokens/s]
Shard 11: 100%|█████████▉| 99727304/100000000 [00:29<00:00, 2706958.53tokens/s]
Shard 11: 100%|█████████▉| 99999397/100000000 [00:37<00:00, 2649434.26tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000011.bin (200,001,024 bytes) in the gpt-2 format

Shard 12:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 12:   0%|          | 231257/100000000 [00:00<00:43, 2300712.92tokens/s]
Shard 12:   1%|          | 558044/100000000 [00:00<00:34, 2861869.26tokens/s]
Shard 12:   1%|          | 846009/100000000 [00:00<00:36, 2748757.05tokens/s]
Shard 12:   1%|          | 1124333/100000000 [00:00<00:38, 2600784.44tokens/s]
Shard 12:   1%|▏         | 1482095/100000000 [00:00<00:33, 2932674.46tokens/s]
Shard 12:   2%|▏         | 2103497/100000000 [00:00<00:24, 3991482.69tokens/s]
Shard 12:   3%|▎         | 2507782/100000000 [00:00<00:28, 3432117.12tokens/s]
Shard 12:   3%|▎         | 3001485/100000000 [00:00<00:25, 3828272.76tokens/s]
Shard 12:   3%|▎         | 3409576/100000000 [00:00<00:24, 3899807.94tokens/s]
Shard 12:   4%|▍         | 3900883/100000000 [00:01<00:22, 4190364.94tokens/s]
Shard 12:   4%|▍         | 4354670/100000000 [00:01<00:22, 4270731.59tokens/s]
Shard 12:   5%|▍         | 4788886/100000000 [00:01<00:24, 3936309.73tokens/s]
Shard 12:   6%|▌         | 5732631/100000000 [00:01<00:17, 5451341.05tokens/s]
Shard 12:   6%|▋         | 6297435/100000000 [00:01<00:17, 5341027.84tokens/s]
Shard 12:   7%|▋         | 6845370/100000000 [00:01<00:19, 4695702.75tokens/s]
Shard 12:   7%|▋         | 7339439/100000000 [00:01<00:21, 4347530.86tokens/s]
Shard 12:   8%|▊         | 7792668/100000000 [00:01<00:24, 3759762.42tokens/s]
Shard 12:   8%|▊         | 8191135/100000000 [00:02<00:28, 3259157.08tokens/s]
Shard 12:   9%|▊         | 8612918/100000000 [00:02<00:26, 3470398.54tokens/s]
Shard 12:   9%|▉         | 9005383/100000000 [00:02<00:25, 3579595.38tokens/s]
Shard 12:   9%|▉         | 9382463/100000000 [00:02<00:26, 3375983.71tokens/s]
Shard 12:  10%|▉         | 9733952/100000000 [00:02<00:27, 3315574.77tokens/s]
Shard 12:  10%|█         | 10075254/100000000 [00:02<00:29, 3095092.52tokens/s]
Shard 12:  10%|█         | 10468540/100000000 [00:02<00:27, 3305384.07tokens/s]
Shard 12:  11%|█         | 10912266/100000000 [00:02<00:24, 3595730.46tokens/s]
Shard 12:  11%|█▏        | 11298172/100000000 [00:03<00:24, 3651683.51tokens/s]
Shard 12:  12%|█▏        | 11671461/100000000 [00:03<00:26, 3384574.49tokens/s]
Shard 12:  12%|█▏        | 12017837/100000000 [00:03<00:26, 3364889.64tokens/s]
Shard 12:  12%|█▏        | 12362394/100000000 [00:03<00:26, 3348047.26tokens/s]
Shard 12:  13%|█▎        | 12701407/100000000 [00:03<00:27, 3159553.57tokens/s]
Shard 12:  13%|█▎        | 13021394/100000000 [00:03<00:27, 3138476.06tokens/s]
Shard 12:  13%|█▎        | 13338621/100000000 [00:03<00:28, 2994986.71tokens/s]
Shard 12:  14%|█▎        | 13640935/100000000 [00:03<00:29, 2970079.15tokens/s]
Shard 12:  14%|█▍        | 13946378/100000000 [00:03<00:28, 2969222.70tokens/s]
Shard 12:  14%|█▍        | 14337345/100000000 [00:04<00:26, 3213418.82tokens/s]
Shard 12:  15%|█▍        | 14748249/100000000 [00:04<00:24, 3457050.02tokens/s]
Shard 12:  15%|█▌        | 15096678/100000000 [00:04<00:27, 3062001.49tokens/s]
Shard 12:  15%|█▌        | 15412868/100000000 [00:04<00:31, 2677265.79tokens/s]
Shard 12:  16%|█▌        | 15700318/100000000 [00:04<00:33, 2500270.61tokens/s]
Shard 12:  16%|█▌        | 16014976/100000000 [00:04<00:31, 2653152.78tokens/s]
Shard 12:  16%|█▋        | 16431065/100000000 [00:04<00:27, 2998572.03tokens/s]
Shard 12:  17%|█▋        | 16769033/100000000 [00:04<00:27, 3051620.85tokens/s]
Shard 12:  17%|█▋        | 17082390/100000000 [00:04<00:27, 3037561.87tokens/s]
Shard 12:  17%|█▋        | 17391646/100000000 [00:05<00:27, 3020630.95tokens/s]
Shard 12:  18%|█▊        | 17698950/100000000 [00:05<00:28, 2932558.67tokens/s]
Shard 12:  18%|█▊        | 18262774/100000000 [00:05<00:22, 3663680.40tokens/s]
Shard 12:  19%|█▊        | 18635183/100000000 [00:05<00:25, 3248464.52tokens/s]
Shard 12:  19%|█▉        | 18972124/100000000 [00:05<00:29, 2782460.36tokens/s]
Shard 12:  19%|█▉        | 19363237/100000000 [00:05<00:26, 3050686.60tokens/s]
Shard 12:  20%|█▉        | 19773817/100000000 [00:05<00:24, 3321689.53tokens/s]
Shard 12:  20%|██        | 20125230/100000000 [00:05<00:25, 3147154.06tokens/s]
Shard 12:  21%|██        | 20576519/100000000 [00:06<00:22, 3502238.15tokens/s]
Shard 12:  21%|██        | 21035950/100000000 [00:06<00:20, 3785564.23tokens/s]
Shard 12:  21%|██▏       | 21426908/100000000 [00:06<00:21, 3634114.62tokens/s]
Shard 12:  22%|██▏       | 21799874/100000000 [00:06<00:22, 3413916.41tokens/s]
Shard 12:  22%|██▏       | 22150430/100000000 [00:06<00:24, 3235549.50tokens/s]
Shard 12:  22%|██▏       | 22487011/100000000 [00:06<00:23, 3260608.63tokens/s]
Shard 12:  23%|██▎       | 22817953/100000000 [00:06<00:24, 3117725.73tokens/s]
Shard 12:  23%|██▎       | 23134469/100000000 [00:06<00:26, 2856774.95tokens/s]
Shard 12:  23%|██▎       | 23453792/100000000 [00:06<00:26, 2936419.39tokens/s]
Shard 12:  24%|██▍       | 23848977/100000000 [00:07<00:23, 3194316.07tokens/s]
Shard 12:  24%|██▍       | 24219029/100000000 [00:07<00:23, 3277472.21tokens/s]
Shard 12:  25%|██▍       | 24552801/100000000 [00:07<00:22, 3287276.08tokens/s]
Shard 12:  25%|██▍       | 24886076/100000000 [00:07<00:24, 3012085.79tokens/s]
Shard 12:  25%|██▌       | 25281423/100000000 [00:07<00:23, 3241883.59tokens/s]
Shard 12:  26%|██▌       | 25613785/100000000 [00:07<00:24, 3008524.87tokens/s]
Shard 12:  26%|██▌       | 25928847/100000000 [00:07<00:24, 3024798.61tokens/s]
Shard 12:  26%|██▋       | 26269104/100000000 [00:07<00:23, 3128772.03tokens/s]
Shard 12:  27%|██▋       | 26675503/100000000 [00:07<00:21, 3389757.60tokens/s]
Shard 12:  27%|██▋       | 27019820/100000000 [00:08<00:23, 3125665.10tokens/s]
Shard 12:  27%|██▋       | 27402419/100000000 [00:08<00:21, 3309803.18tokens/s]
Shard 12:  28%|██▊       | 27845331/100000000 [00:08<00:19, 3608300.81tokens/s]
Shard 12:  28%|██▊       | 28334988/100000000 [00:08<00:18, 3896509.19tokens/s]
Shard 12:  29%|██▉       | 28967704/100000000 [00:08<00:15, 4582722.00tokens/s]
Shard 12:  29%|██▉       | 29432258/100000000 [00:08<00:18, 3827424.73tokens/s]
Shard 12:  30%|███       | 30082187/100000000 [00:08<00:15, 4500504.09tokens/s]
Shard 12:  31%|███       | 30604696/100000000 [00:08<00:14, 4693634.02tokens/s]
Shard 12:  31%|███       | 31106214/100000000 [00:08<00:14, 4769022.78tokens/s]
Shard 12:  32%|███▏      | 31601878/100000000 [00:09<00:17, 3850417.75tokens/s]
Shard 12:  32%|███▏      | 32026333/100000000 [00:09<00:18, 3652240.71tokens/s]
Shard 12:  32%|███▏      | 32419007/100000000 [00:09<00:18, 3638950.37tokens/s]
Shard 12:  33%|███▎      | 32846412/100000000 [00:09<00:17, 3780767.50tokens/s]
Shard 12:  33%|███▎      | 33240022/100000000 [00:09<00:18, 3596211.47tokens/s]
Shard 12:  34%|███▎      | 33720002/100000000 [00:09<00:16, 3906308.45tokens/s]
Shard 12:  34%|███▍      | 34202314/100000000 [00:09<00:15, 4156010.61tokens/s]
Shard 12:  35%|███▍      | 34629230/100000000 [00:09<00:17, 3768441.27tokens/s]
Shard 12:  35%|███▌      | 35020853/100000000 [00:10<00:17, 3781037.06tokens/s]
Shard 12:  36%|███▌      | 35957109/100000000 [00:10<00:12, 5270086.73tokens/s]
Shard 12:  37%|███▋      | 36503049/100000000 [00:10<00:14, 4428413.71tokens/s]
Shard 12:  37%|███▋      | 36979953/100000000 [00:10<00:14, 4206161.99tokens/s]
Shard 12:  37%|███▋      | 37424582/100000000 [00:10<00:15, 4034247.23tokens/s]
Shard 12:  38%|███▊      | 38011884/100000000 [00:10<00:13, 4476170.27tokens/s]
Shard 12:  38%|███▊      | 38478635/100000000 [00:10<00:13, 4458150.48tokens/s]
Shard 12:  39%|███▉      | 38937800/100000000 [00:10<00:14, 4159399.61tokens/s]
Shard 12:  39%|███▉      | 39365256/100000000 [00:11<00:16, 3743946.30tokens/s]
Shard 12:  40%|███▉      | 39753049/100000000 [00:11<00:16, 3627503.71tokens/s]
Shard 12:  40%|████      | 40124394/100000000 [00:11<00:18, 3257970.66tokens/s]
Shard 12:  40%|████      | 40460921/100000000 [00:11<00:18, 3281109.28tokens/s]
Shard 12:  41%|████▏     | 41288417/100000000 [00:11<00:12, 4544348.25tokens/s]
Shard 12:  42%|████▏     | 41768158/100000000 [00:11<00:14, 4012128.23tokens/s]
Shard 12:  42%|████▏     | 42195503/100000000 [00:11<00:15, 3785497.98tokens/s]
Shard 12:  43%|████▎     | 42592724/100000000 [00:11<00:15, 3735064.02tokens/s]
Shard 12:  43%|████▎     | 42978188/100000000 [00:12<00:18, 3063691.80tokens/s]
Shard 12:  43%|████▎     | 43320030/100000000 [00:12<00:18, 3066242.08tokens/s]
Shard 12:  44%|████▎     | 43643908/100000000 [00:12<00:18, 3102518.65tokens/s]
Shard 12:  44%|████▍     | 43971057/100000000 [00:12<00:17, 3145473.05tokens/s]
Shard 12:  44%|████▍     | 44312965/100000000 [00:12<00:19, 2802008.19tokens/s]
Shard 12:  45%|████▍     | 44607483/100000000 [00:12<00:19, 2773771.61tokens/s]
Shard 12:  45%|████▍     | 44894515/100000000 [00:12<00:19, 2774479.72tokens/s]
Shard 12:  45%|████▌     | 45178006/100000000 [00:12<00:20, 2720620.04tokens/s]
Shard 12:  45%|████▌     | 45454121/100000000 [00:13<00:20, 2681045.34tokens/s]
Shard 12:  46%|████▌     | 45725923/100000000 [00:13<00:20, 2666472.41tokens/s]
Shard 12:  46%|████▌     | 46019787/100000000 [00:13<00:19, 2739351.39tokens/s]
Shard 12:  46%|████▋     | 46334103/100000000 [00:13<00:18, 2847098.18tokens/s]
Shard 12:  47%|████▋     | 46625061/100000000 [00:13<00:18, 2847936.83tokens/s]
Shard 12:  47%|████▋     | 46984158/100000000 [00:13<00:17, 3046766.92tokens/s]
Shard 12:  47%|████▋     | 47290682/100000000 [00:13<00:18, 2774245.49tokens/s]
Shard 12:  48%|████▊     | 47573238/100000000 [00:13<00:18, 2785904.53tokens/s]
Shard 12:  48%|████▊     | 47858739/100000000 [00:13<00:18, 2777051.66tokens/s]
Shard 12:  48%|████▊     | 48218603/100000000 [00:13<00:17, 2994095.17tokens/s]
Shard 12:  49%|████▊     | 48626625/100000000 [00:14<00:15, 3306034.28tokens/s]
Shard 12:  49%|████▉     | 48960818/100000000 [00:14<00:16, 3028018.77tokens/s]
Shard 12:  49%|████▉     | 49368892/100000000 [00:14<00:15, 3309191.79tokens/s]
Shard 12:  50%|████▉     | 49829213/100000000 [00:14<00:13, 3662585.49tokens/s]
Shard 12:  50%|█████     | 50310213/100000000 [00:14<00:12, 3978504.40tokens/s]
Shard 12:  51%|█████     | 50772322/100000000 [00:14<00:11, 4156683.15tokens/s]
Shard 12:  51%|█████▏    | 51373593/100000000 [00:14<00:10, 4680724.95tokens/s]
Shard 12:  52%|█████▏    | 51846292/100000000 [00:14<00:11, 4272542.39tokens/s]
Shard 12:  53%|█████▎    | 52618761/100000000 [00:14<00:09, 5213747.54tokens/s]
Shard 12:  53%|█████▎    | 53154356/100000000 [00:15<00:10, 4592362.52tokens/s]
Shard 12:  54%|█████▎    | 53636141/100000000 [00:15<00:10, 4475120.00tokens/s]
Shard 12:  54%|█████▍    | 54098564/100000000 [00:15<00:10, 4318093.95tokens/s]
Shard 12:  55%|█████▍    | 54540819/100000000 [00:15<00:11, 4038517.45tokens/s]
Shard 12:  55%|█████▍    | 54957389/100000000 [00:15<00:11, 4049736.97tokens/s]
Shard 12:  55%|█████▌    | 55369243/100000000 [00:15<00:11, 3796874.90tokens/s]
Shard 12:  56%|█████▌    | 55785898/100000000 [00:15<00:11, 3894233.00tokens/s]
Shard 12:  56%|█████▌    | 56204714/100000000 [00:15<00:11, 3955273.48tokens/s]
Shard 12:  57%|█████▋    | 56658846/100000000 [00:15<00:10, 4081845.38tokens/s]
Shard 12:  57%|█████▋    | 57071266/100000000 [00:16<00:11, 3865651.63tokens/s]
Shard 12:  57%|█████▋    | 57461934/100000000 [00:16<00:11, 3634924.90tokens/s]
Shard 12:  58%|█████▊    | 57903787/100000000 [00:16<00:10, 3839225.54tokens/s]
Shard 12:  58%|█████▊    | 58293323/100000000 [00:16<00:12, 3262349.55tokens/s]
Shard 12:  59%|█████▉    | 58814242/100000000 [00:16<00:10, 3744921.42tokens/s]
Shard 12:  59%|█████▉    | 59229306/100000000 [00:16<00:10, 3850982.62tokens/s]
Shard 12:  60%|█████▉    | 59630380/100000000 [00:16<00:13, 2960698.25tokens/s]
Shard 12:  60%|█████▉    | 59967434/100000000 [00:17<00:14, 2767160.96tokens/s]
Shard 12:  60%|██████    | 60286731/100000000 [00:17<00:15, 2569193.36tokens/s]
Shard 12:  61%|██████    | 60564485/100000000 [00:17<00:15, 2584680.27tokens/s]
Shard 12:  61%|██████    | 60838172/100000000 [00:17<00:15, 2454007.72tokens/s]
Shard 12:  61%|██████    | 61093516/100000000 [00:17<00:15, 2451088.32tokens/s]
Shard 12:  61%|██████▏   | 61348397/100000000 [00:17<00:16, 2316133.61tokens/s]
Shard 12:  62%|██████▏   | 61586269/100000000 [00:17<00:16, 2313272.10tokens/s]
Shard 12:  62%|██████▏   | 61821698/100000000 [00:17<00:17, 2131962.28tokens/s]
Shard 12:  62%|██████▏   | 62063041/100000000 [00:18<00:17, 2178573.21tokens/s]
Shard 12:  62%|██████▏   | 62416723/100000000 [00:18<00:14, 2508804.09tokens/s]
Shard 12:  63%|██████▎   | 62842912/100000000 [00:18<00:12, 2988661.87tokens/s]
Shard 12:  63%|██████▎   | 63194493/100000000 [00:18<00:11, 3129091.46tokens/s]
Shard 12:  64%|██████▎   | 63534889/100000000 [00:18<00:11, 3183648.71tokens/s]
Shard 12:  64%|██████▍   | 63869167/100000000 [00:18<00:11, 3039195.59tokens/s]
Shard 12:  64%|██████▍   | 64178110/100000000 [00:18<00:12, 2819976.24tokens/s]
Shard 12:  64%|██████▍   | 64490678/100000000 [00:18<00:12, 2896494.24tokens/s]
Shard 12:  65%|██████▍   | 64803800/100000000 [00:18<00:13, 2679949.04tokens/s]
Shard 12:  65%|██████▌   | 65276695/100000000 [00:19<00:10, 3219714.12tokens/s]
Shard 12:  66%|██████▌   | 65758854/100000000 [00:19<00:09, 3607359.87tokens/s]
Shard 12:  66%|██████▌   | 66130108/100000000 [00:19<00:09, 3533185.03tokens/s]
Shard 12:  66%|██████▋   | 66497706/100000000 [00:19<00:09, 3514316.59tokens/s]
Shard 12:  67%|██████▋   | 66928560/100000000 [00:19<00:08, 3725659.15tokens/s]
Shard 12:  67%|██████▋   | 67438436/100000000 [00:19<00:07, 4112184.55tokens/s]
Shard 12:  68%|██████▊   | 67877396/100000000 [00:19<00:07, 4185082.30tokens/s]
Shard 12:  68%|██████▊   | 68370900/100000000 [00:19<00:07, 4404333.25tokens/s]
Shard 12:  69%|██████▉   | 68820522/100000000 [00:19<00:07, 4280491.53tokens/s]
Shard 12:  69%|██████▉   | 69273941/100000000 [00:19<00:07, 4345124.01tokens/s]
Shard 12:  70%|██████▉   | 69710904/100000000 [00:20<00:07, 3954535.56tokens/s]
Shard 12:  70%|███████   | 70114318/100000000 [00:20<00:08, 3607687.43tokens/s]
Shard 12:  71%|███████   | 70903944/100000000 [00:20<00:06, 4724762.84tokens/s]
Shard 12:  71%|███████▏  | 71400021/100000000 [00:20<00:06, 4549512.76tokens/s]
Shard 12:  72%|███████▏  | 71924378/100000000 [00:20<00:05, 4706412.90tokens/s]
Shard 12:  72%|███████▏  | 72414804/100000000 [00:20<00:06, 4569889.81tokens/s]
Shard 12:  73%|███████▎  | 72880932/100000000 [00:20<00:06, 4193246.38tokens/s]
Shard 12:  73%|███████▎  | 73312212/100000000 [00:20<00:06, 4178094.59tokens/s]
Shard 12:  74%|███████▎  | 73740957/100000000 [00:21<00:06, 3795302.37tokens/s]
Shard 12:  74%|███████▍  | 74278203/100000000 [00:21<00:06, 4180939.72tokens/s]
Shard 12:  75%|███████▍  | 74903915/100000000 [00:21<00:05, 4714557.80tokens/s]
Shard 12:  75%|███████▌  | 75389567/100000000 [00:21<00:05, 4242432.94tokens/s]
Shard 12:  76%|███████▌  | 75831398/100000000 [00:21<00:05, 4169778.85tokens/s]
Shard 12:  76%|███████▋  | 76396431/100000000 [00:21<00:05, 4560413.95tokens/s]
Shard 12:  77%|███████▋  | 77049466/100000000 [00:21<00:04, 5080669.41tokens/s]
Shard 12:  78%|███████▊  | 77630684/100000000 [00:21<00:04, 5285642.61tokens/s]
Shard 12:  78%|███████▊  | 78170650/100000000 [00:22<00:05, 4000948.35tokens/s]
Shard 12:  79%|███████▊  | 78699228/100000000 [00:22<00:04, 4294743.20tokens/s]
Shard 12:  79%|███████▉  | 79223557/100000000 [00:22<00:04, 4494358.54tokens/s]
Shard 12:  80%|███████▉  | 79708432/100000000 [00:22<00:04, 4217095.60tokens/s]
Shard 12:  80%|████████  | 80156502/100000000 [00:22<00:04, 3984673.53tokens/s]
Shard 12:  81%|████████  | 80573991/100000000 [00:22<00:04, 3913377.46tokens/s]
Shard 12:  81%|████████  | 80978303/100000000 [00:22<00:05, 3665552.26tokens/s]
Shard 12:  81%|████████▏ | 81355823/100000000 [00:22<00:05, 3620129.99tokens/s]
Shard 12:  82%|████████▏ | 81724516/100000000 [00:22<00:05, 3294296.81tokens/s]
Shard 12:  82%|████████▏ | 82067231/100000000 [00:23<00:05, 3325237.91tokens/s]
Shard 12:  83%|████████▎ | 82526043/100000000 [00:23<00:04, 3660970.89tokens/s]
Shard 12:  83%|████████▎ | 82900533/100000000 [00:23<00:05, 2935377.78tokens/s]
Shard 12:  83%|████████▎ | 83221081/100000000 [00:23<00:06, 2724482.63tokens/s]
Shard 12:  84%|████████▎ | 83513466/100000000 [00:23<00:05, 2751675.27tokens/s]
Shard 12:  84%|████████▍ | 83803172/100000000 [00:23<00:06, 2624361.28tokens/s]
Shard 12:  84%|████████▍ | 84398049/100000000 [00:23<00:04, 3456038.11tokens/s]
Shard 12:  85%|████████▍ | 84766377/100000000 [00:23<00:04, 3391446.60tokens/s]
Shard 12:  85%|████████▌ | 85146504/100000000 [00:24<00:04, 3500952.68tokens/s]
Shard 12:  86%|████████▌ | 85510398/100000000 [00:24<00:04, 3535164.51tokens/s]
Shard 12:  86%|████████▌ | 85895608/100000000 [00:24<00:03, 3624735.08tokens/s]
Shard 12:  86%|████████▋ | 86265101/100000000 [00:24<00:03, 3638787.70tokens/s]
Shard 12:  87%|████████▋ | 86633871/100000000 [00:24<00:03, 3647804.83tokens/s]
Shard 12:  87%|████████▋ | 87076742/100000000 [00:24<00:03, 3864265.38tokens/s]
Shard 12:  88%|████████▊ | 87532669/100000000 [00:24<00:03, 4060043.94tokens/s]
Shard 12:  88%|████████▊ | 88057913/100000000 [00:24<00:02, 4412461.53tokens/s]
Shard 12:  89%|████████▊ | 88501204/100000000 [00:24<00:03, 3663634.13tokens/s]
Shard 12:  89%|████████▉ | 88890497/100000000 [00:25<00:03, 3212436.81tokens/s]
Shard 12:  89%|████████▉ | 89235617/100000000 [00:25<00:03, 3159237.72tokens/s]
Shard 12:  90%|████████▉ | 89567926/100000000 [00:25<00:03, 2933045.74tokens/s]
Shard 12:  90%|████████▉ | 89874905/100000000 [00:25<00:03, 2802480.86tokens/s]
Shard 12:  90%|█████████ | 90198879/100000000 [00:25<00:03, 2885542.07tokens/s]
Shard 12:  91%|█████████ | 90618204/100000000 [00:25<00:02, 3219191.66tokens/s]
Shard 12:  91%|█████████ | 90949967/100000000 [00:25<00:03, 2987278.87tokens/s]
Shard 12:  91%|█████████▏| 91257863/100000000 [00:25<00:03, 2860452.71tokens/s]
Shard 12:  92%|█████████▏| 91550420/100000000 [00:26<00:03, 2753050.42tokens/s]
Shard 12:  92%|█████████▏| 91829831/100000000 [00:26<00:03, 2664244.44tokens/s]
Shard 12:  92%|█████████▏| 92098925/100000000 [00:26<00:02, 2659172.13tokens/s]
Shard 12:  92%|█████████▏| 92494988/100000000 [00:26<00:02, 3012987.75tokens/s]
Shard 12:  93%|█████████▎| 92964015/100000000 [00:26<00:02, 3475647.33tokens/s]
Shard 12:  93%|█████████▎| 93431602/100000000 [00:26<00:01, 3802076.64tokens/s]
Shard 12:  94%|█████████▍| 93817090/100000000 [00:26<00:01, 3575708.93tokens/s]
Shard 12:  94%|█████████▍| 94192234/100000000 [00:26<00:01, 3600402.64tokens/s]
Shard 12:  95%|█████████▍| 94560590/100000000 [00:26<00:01, 3615598.05tokens/s]
Shard 12:  95%|█████████▌| 95167557/100000000 [00:26<00:01, 4313968.11tokens/s]
Shard 12:  96%|█████████▌| 95604376/100000000 [00:27<00:01, 4241729.98tokens/s]
Shard 12:  96%|█████████▌| 96062774/100000000 [00:27<00:00, 4335519.38tokens/s]
Shard 12:  97%|█████████▋| 96591952/100000000 [00:27<00:00, 4569765.99tokens/s]
Shard 12:  97%|█████████▋| 97052103/100000000 [00:27<00:00, 4431534.91tokens/s]
Shard 12:  97%|█████████▋| 97497356/100000000 [00:27<00:00, 3744367.17tokens/s]
Shard 12:  98%|█████████▊| 97890591/100000000 [00:27<00:00, 3630879.34tokens/s]
Shard 12:  98%|█████████▊| 98266963/100000000 [00:27<00:00, 3472238.52tokens/s]
Shard 12:  99%|█████████▊| 98623874/100000000 [00:27<00:00, 3073443.25tokens/s]
Shard 12:  99%|█████████▉| 98943464/100000000 [00:28<00:00, 2721656.77tokens/s]
Shard 12:  99%|█████████▉| 99290085/100000000 [00:28<00:00, 2885802.88tokens/s]
Shard 12: 100%|█████████▉| 99594018/100000000 [00:28<00:00, 2919036.54tokens/s]
Shard 12: 100%|█████████▉| 99966738/100000000 [00:28<00:00, 3127925.66tokens/s]
Shard 12: 100%|█████████▉| 99999254/100000000 [00:36<00:00, 2754404.52tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000012.bin (200,001,024 bytes) in the gpt-2 format

Shard 13:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 13:   0%|          | 191080/100000000 [00:00<00:52, 1895096.33tokens/s]
Shard 13:   0%|          | 489233/100000000 [00:00<00:39, 2524248.38tokens/s]
Shard 13:   1%|          | 742025/100000000 [00:00<00:41, 2403258.69tokens/s]
Shard 13:   1%|          | 984565/100000000 [00:00<01:01, 1602755.93tokens/s]
Shard 13:   1%|          | 1172344/100000000 [00:00<01:00, 1638450.83tokens/s]
Shard 13:   1%|▏         | 1421267/100000000 [00:00<00:52, 1868891.61tokens/s]
Shard 13:   2%|▏         | 1626164/100000000 [00:00<01:03, 1557539.98tokens/s]
Shard 13:   2%|▏         | 1904497/100000000 [00:01<00:53, 1836708.31tokens/s]
Shard 13:   2%|▏         | 2140586/100000000 [00:01<00:49, 1964095.27tokens/s]
Shard 13:   2%|▏         | 2353589/100000000 [00:01<00:49, 1957825.60tokens/s]
Shard 13:   3%|▎         | 2561077/100000000 [00:01<00:55, 1761160.13tokens/s]
Shard 13:   3%|▎         | 2791812/100000000 [00:01<00:51, 1873072.03tokens/s]
Shard 13:   3%|▎         | 2989146/100000000 [00:01<00:51, 1890948.15tokens/s]
Shard 13:   3%|▎         | 3185382/100000000 [00:01<00:54, 1776478.57tokens/s]
Shard 13:   3%|▎         | 3425521/100000000 [00:01<00:49, 1938086.08tokens/s]
Shard 13:   4%|▎         | 3681871/100000000 [00:01<00:45, 2101691.38tokens/s]
Shard 13:   4%|▍         | 4082741/100000000 [00:02<00:36, 2637456.89tokens/s]
Shard 13:   4%|▍         | 4354542/100000000 [00:02<00:37, 2557368.11tokens/s]
Shard 13:   5%|▍         | 4681375/100000000 [00:02<00:34, 2741130.14tokens/s]
Shard 13:   5%|▌         | 5031200/100000000 [00:02<00:32, 2949246.39tokens/s]
Shard 13:   5%|▌         | 5379217/100000000 [00:02<00:30, 3102688.19tokens/s]
Shard 13:   6%|▌         | 5778314/100000000 [00:02<00:28, 3362550.95tokens/s]
Shard 13:   7%|▋         | 6840509/100000000 [00:02<00:16, 5498896.33tokens/s]
Shard 13:   7%|▋         | 7396433/100000000 [00:02<00:19, 4761421.60tokens/s]
Shard 13:   8%|▊         | 7905043/100000000 [00:02<00:21, 4292093.46tokens/s]
Shard 13:   8%|▊         | 8369947/100000000 [00:03<00:20, 4379815.20tokens/s]
Shard 13:   9%|▉         | 8825120/100000000 [00:03<00:22, 3983983.50tokens/s]
Shard 13:   9%|▉         | 9239939/100000000 [00:03<00:23, 3945296.18tokens/s]
Shard 13:  10%|▉         | 9784418/100000000 [00:03<00:20, 4330785.39tokens/s]
Shard 13:  10%|█         | 10245074/100000000 [00:03<00:20, 4404696.26tokens/s]
Shard 13:  11%|█         | 10894881/100000000 [00:03<00:17, 4980418.48tokens/s]
Shard 13:  11%|█▏        | 11403632/100000000 [00:03<00:17, 4981294.89tokens/s]
Shard 13:  12%|█▏        | 11912272/100000000 [00:03<00:17, 4965859.31tokens/s]
Shard 13:  12%|█▏        | 12433942/100000000 [00:03<00:17, 4916054.77tokens/s]
Shard 13:  13%|█▎        | 12929465/100000000 [00:04<00:19, 4400555.39tokens/s]
Shard 13:  13%|█▎        | 13391918/100000000 [00:04<00:21, 4077638.47tokens/s]
Shard 13:  14%|█▍        | 13811030/100000000 [00:04<00:22, 3888883.92tokens/s]
Shard 13:  14%|█▍        | 14232866/100000000 [00:04<00:22, 3876132.72tokens/s]
Shard 13:  15%|█▍        | 14626815/100000000 [00:04<00:22, 3760073.33tokens/s]
Shard 13:  15%|█▌        | 15295756/100000000 [00:04<00:18, 4545663.21tokens/s]
Shard 13:  16%|█▌        | 15762100/100000000 [00:04<00:20, 4184822.59tokens/s]
Shard 13:  16%|█▌        | 16193413/100000000 [00:04<00:20, 4021526.10tokens/s]
Shard 13:  17%|█▋        | 16606763/100000000 [00:05<00:20, 3979021.44tokens/s]
Shard 13:  17%|█▋        | 17036291/100000000 [00:05<00:20, 4061146.87tokens/s]
Shard 13:  18%|█▊        | 17563248/100000000 [00:05<00:18, 4398291.89tokens/s]
Shard 13:  18%|█▊        | 18167380/100000000 [00:05<00:16, 4854687.37tokens/s]
Shard 13:  19%|█▊        | 18683259/100000000 [00:05<00:16, 4904151.94tokens/s]
Shard 13:  19%|█▉        | 19326007/100000000 [00:05<00:15, 5346147.86tokens/s]
Shard 13:  20%|█▉        | 19994809/100000000 [00:05<00:13, 5739155.49tokens/s]
Shard 13:  21%|██        | 20589581/100000000 [00:05<00:13, 5682854.00tokens/s]
Shard 13:  21%|██        | 21160885/100000000 [00:05<00:18, 4175451.36tokens/s]
Shard 13:  22%|██▏       | 21637474/100000000 [00:06<00:19, 4029852.21tokens/s]
Shard 13:  22%|██▏       | 22081115/100000000 [00:06<00:20, 3738946.70tokens/s]
Shard 13:  23%|██▎       | 22514253/100000000 [00:06<00:19, 3876853.93tokens/s]
Shard 13:  23%|██▎       | 22941271/100000000 [00:06<00:19, 3974897.27tokens/s]
Shard 13:  23%|██▎       | 23363578/100000000 [00:06<00:21, 3640910.81tokens/s]
Shard 13:  24%|██▍       | 23800717/100000000 [00:06<00:19, 3818854.02tokens/s]
Shard 13:  24%|██▍       | 24217349/100000000 [00:06<00:19, 3896466.43tokens/s]
Shard 13:  25%|██▍       | 24777288/100000000 [00:06<00:17, 4360265.65tokens/s]
Shard 13:  26%|██▌       | 25632406/100000000 [00:06<00:13, 5465528.11tokens/s]
Shard 13:  26%|██▌       | 26191090/100000000 [00:07<00:15, 4768549.21tokens/s]
Shard 13:  27%|██▋       | 26691006/100000000 [00:07<00:15, 4679436.46tokens/s]
Shard 13:  27%|██▋       | 27320560/100000000 [00:07<00:14, 5103218.69tokens/s]
Shard 13:  28%|██▊       | 27847418/100000000 [00:07<00:14, 4980374.71tokens/s]
Shard 13:  28%|██▊       | 28406180/100000000 [00:07<00:14, 5006988.50tokens/s]
Shard 13:  29%|██▉       | 28918010/100000000 [00:07<00:14, 4902546.33tokens/s]
Shard 13:  29%|██▉       | 29415112/100000000 [00:07<00:16, 4397018.85tokens/s]
Shard 13:  30%|██▉       | 29943503/100000000 [00:07<00:15, 4617612.48tokens/s]
Shard 13:  30%|███       | 30421426/100000000 [00:08<00:16, 4187968.08tokens/s]
Shard 13:  31%|███       | 31095523/100000000 [00:08<00:14, 4839371.23tokens/s]
Shard 13:  32%|███▏      | 31599068/100000000 [00:08<00:14, 4619983.50tokens/s]
Shard 13:  32%|███▏      | 32160806/100000000 [00:08<00:13, 4874750.47tokens/s]
Shard 13:  33%|███▎      | 32662036/100000000 [00:08<00:14, 4758282.55tokens/s]
Shard 13:  33%|███▎      | 33216548/100000000 [00:08<00:13, 4963223.84tokens/s]
Shard 13:  34%|███▍      | 34265650/100000000 [00:08<00:10, 6515034.02tokens/s]
Shard 13:  35%|███▍      | 34932525/100000000 [00:08<00:10, 6090862.74tokens/s]
Shard 13:  36%|███▌      | 35557207/100000000 [00:08<00:12, 5307514.78tokens/s]
Shard 13:  36%|███▌      | 36115368/100000000 [00:09<00:11, 5363943.36tokens/s]
Shard 13:  37%|███▋      | 36671015/100000000 [00:09<00:12, 5223559.20tokens/s]
Shard 13:  37%|███▋      | 37206736/100000000 [00:09<00:14, 4464375.28tokens/s]
Shard 13:  38%|███▊      | 37678288/100000000 [00:09<00:14, 4422655.06tokens/s]
Shard 13:  38%|███▊      | 38144121/100000000 [00:09<00:13, 4446159.71tokens/s]
Shard 13:  39%|███▊      | 38602042/100000000 [00:09<00:15, 3977901.87tokens/s]
Shard 13:  39%|███▉      | 39159429/100000000 [00:09<00:13, 4347211.42tokens/s]
Shard 13:  40%|███▉      | 39705555/100000000 [00:09<00:13, 4608389.29tokens/s]
Shard 13:  40%|████      | 40181509/100000000 [00:10<00:13, 4282961.16tokens/s]
Shard 13:  41%|████      | 40623081/100000000 [00:10<00:14, 4117576.68tokens/s]
Shard 13:  41%|████      | 41044186/100000000 [00:10<00:16, 3670099.62tokens/s]
Shard 13:  41%|████▏     | 41466752/100000000 [00:10<00:15, 3807158.39tokens/s]
Shard 13:  42%|████▏     | 41910770/100000000 [00:10<00:14, 3972065.00tokens/s]
Shard 13:  42%|████▏     | 42434154/100000000 [00:10<00:13, 4313249.58tokens/s]
Shard 13:  43%|████▎     | 42895189/100000000 [00:10<00:12, 4394593.04tokens/s]
Shard 13:  43%|████▎     | 43343110/100000000 [00:10<00:12, 4390228.08tokens/s]
Shard 13:  44%|████▍     | 43789289/100000000 [00:10<00:13, 4185497.96tokens/s]
Shard 13:  44%|████▍     | 44229129/100000000 [00:11<00:15, 3623715.83tokens/s]
Shard 13:  45%|████▍     | 44663659/100000000 [00:11<00:14, 3793606.44tokens/s]
Shard 13:  45%|████▌     | 45071139/100000000 [00:11<00:14, 3855855.66tokens/s]
Shard 13:  45%|████▌     | 45467468/100000000 [00:11<00:14, 3686397.80tokens/s]
Shard 13:  46%|████▌     | 45848190/100000000 [00:11<00:15, 3458673.56tokens/s]
Shard 13:  46%|████▋     | 46320981/100000000 [00:11<00:14, 3789064.36tokens/s]
Shard 13:  47%|████▋     | 46841243/100000000 [00:11<00:12, 4176958.86tokens/s]
Shard 13:  47%|████▋     | 47269347/100000000 [00:11<00:13, 3915553.67tokens/s]
Shard 13:  48%|████▊     | 47673659/100000000 [00:11<00:13, 3943838.54tokens/s]
Shard 13:  48%|████▊     | 48408129/100000000 [00:12<00:10, 4894835.95tokens/s]
Shard 13:  49%|████▉     | 48910171/100000000 [00:12<00:11, 4581767.60tokens/s]
Shard 13:  49%|████▉     | 49380083/100000000 [00:12<00:13, 3891086.50tokens/s]
Shard 13:  50%|████▉     | 49793244/100000000 [00:12<00:13, 3612139.07tokens/s]
Shard 13:  50%|█████     | 50212379/100000000 [00:12<00:13, 3753283.59tokens/s]
Shard 13:  51%|█████     | 50610495/100000000 [00:12<00:13, 3790945.64tokens/s]
Shard 13:  51%|█████▏    | 51252154/100000000 [00:12<00:10, 4440235.63tokens/s]
Shard 13:  52%|█████▏    | 51720115/100000000 [00:12<00:10, 4497132.15tokens/s]
Shard 13:  52%|█████▏    | 52179832/100000000 [00:13<00:10, 4443582.62tokens/s]
Shard 13:  53%|█████▎    | 52844399/100000000 [00:13<00:09, 5048923.19tokens/s]
Shard 13:  53%|█████▎    | 53357348/100000000 [00:13<00:10, 4650881.01tokens/s]
Shard 13:  54%|█████▍    | 53840138/100000000 [00:13<00:11, 4121160.53tokens/s]
Shard 13:  54%|█████▍    | 54268962/100000000 [00:13<00:11, 4082081.85tokens/s]
Shard 13:  55%|█████▍    | 54688840/100000000 [00:13<00:11, 3926361.33tokens/s]
Shard 13:  55%|█████▌    | 55089847/100000000 [00:13<00:12, 3539790.18tokens/s]
Shard 13:  56%|█████▌    | 56116532/100000000 [00:13<00:08, 5240030.76tokens/s]
Shard 13:  58%|█████▊    | 57652546/100000000 [00:13<00:05, 7915133.43tokens/s]
Shard 13:  59%|█████▊    | 58508329/100000000 [00:14<00:07, 5426217.62tokens/s]
Shard 13:  59%|█████▉    | 59199563/100000000 [00:14<00:08, 5099509.91tokens/s]
Shard 13:  60%|█████▉    | 59814293/100000000 [00:14<00:08, 4618840.28tokens/s]
Shard 13:  60%|██████    | 60349569/100000000 [00:14<00:09, 4369641.39tokens/s]
Shard 13:  61%|██████    | 60876634/100000000 [00:14<00:08, 4560089.13tokens/s]
Shard 13:  61%|██████▏   | 61375512/100000000 [00:14<00:09, 4244066.44tokens/s]
Shard 13:  62%|██████▏   | 61830812/100000000 [00:15<00:09, 3938826.15tokens/s]
Shard 13:  62%|██████▏   | 62247381/100000000 [00:15<00:10, 3591759.73tokens/s]
Shard 13:  63%|██████▎   | 62835492/100000000 [00:15<00:09, 4113096.14tokens/s]
Shard 13:  63%|██████▎   | 63274294/100000000 [00:15<00:10, 3502104.42tokens/s]
Shard 13:  64%|██████▎   | 63654599/100000000 [00:15<00:10, 3500161.90tokens/s]
Shard 13:  64%|██████▍   | 64026215/100000000 [00:15<00:10, 3368106.60tokens/s]
Shard 13:  64%|██████▍   | 64376812/100000000 [00:15<00:11, 3057430.08tokens/s]
Shard 13:  65%|██████▍   | 64694588/100000000 [00:16<00:11, 3049246.86tokens/s]
Shard 13:  65%|██████▌   | 65059342/100000000 [00:16<00:11, 3165990.15tokens/s]
Shard 13:  65%|██████▌   | 65383478/100000000 [00:16<00:11, 2971393.36tokens/s]
Shard 13:  66%|██████▌   | 65810160/100000000 [00:16<00:10, 3296188.70tokens/s]
Shard 13:  66%|██████▌   | 66149034/100000000 [00:16<00:11, 2983289.20tokens/s]
Shard 13:  67%|██████▋   | 66627176/100000000 [00:16<00:09, 3403175.02tokens/s]
Shard 13:  67%|██████▋   | 66980141/100000000 [00:16<00:09, 3325166.18tokens/s]
Shard 13:  67%|██████▋   | 67322030/100000000 [00:16<00:11, 2902084.42tokens/s]
Shard 13:  68%|██████▊   | 67800680/100000000 [00:16<00:09, 3367978.47tokens/s]
Shard 13:  68%|██████▊   | 68157406/100000000 [00:17<00:10, 3138294.43tokens/s]
Shard 13:  68%|██████▊   | 68486579/100000000 [00:17<00:10, 2971752.84tokens/s]
Shard 13:  69%|██████▉   | 68806864/100000000 [00:17<00:10, 3020140.79tokens/s]
Shard 13:  69%|██████▉   | 69221618/100000000 [00:17<00:09, 3318328.09tokens/s]
Shard 13:  70%|██████▉   | 69564212/100000000 [00:17<00:09, 3342257.63tokens/s]
Shard 13:  70%|██████▉   | 69905503/100000000 [00:17<00:10, 2994211.18tokens/s]
Shard 13:  70%|███████   | 70342160/100000000 [00:17<00:08, 3349342.91tokens/s]
Shard 13:  71%|███████   | 70689659/100000000 [00:17<00:08, 3316898.92tokens/s]
Shard 13:  71%|███████   | 71033464/100000000 [00:17<00:08, 3340357.42tokens/s]
Shard 13:  71%|███████▏  | 71373750/100000000 [00:18<00:09, 2982472.51tokens/s]
Shard 13:  72%|███████▏  | 71682684/100000000 [00:18<00:09, 2932836.02tokens/s]
Shard 13:  72%|███████▏  | 71997419/100000000 [00:18<00:09, 2989636.11tokens/s]
Shard 13:  72%|███████▏  | 72302486/100000000 [00:18<00:09, 2809281.07tokens/s]
Shard 13:  73%|███████▎  | 72743336/100000000 [00:18<00:08, 3240621.72tokens/s]
Shard 13:  73%|███████▎  | 73076064/100000000 [00:18<00:08, 3234124.74tokens/s]
Shard 13:  74%|███████▎  | 73514017/100000000 [00:18<00:07, 3557891.04tokens/s]
Shard 13:  74%|███████▍  | 73876355/100000000 [00:18<00:08, 3218709.41tokens/s]
Shard 13:  74%|███████▍  | 74208302/100000000 [00:19<00:08, 3201572.27tokens/s]
Shard 13:  75%|███████▍  | 74623548/100000000 [00:19<00:07, 3452957.31tokens/s]
Shard 13:  75%|███████▍  | 74976236/100000000 [00:19<00:08, 3060191.05tokens/s]
Shard 13:  75%|███████▌  | 75294517/100000000 [00:19<00:08, 3003911.76tokens/s]
Shard 13:  76%|███████▌  | 75603970/100000000 [00:19<00:08, 2951127.89tokens/s]
Shard 13:  76%|███████▌  | 75993069/100000000 [00:19<00:07, 3193442.23tokens/s]
Shard 13:  76%|███████▋  | 76318618/100000000 [00:19<00:08, 2919637.61tokens/s]
Shard 13:  77%|███████▋  | 76618932/100000000 [00:19<00:09, 2544524.67tokens/s]
Shard 13:  77%|███████▋  | 76926467/100000000 [00:19<00:08, 2666885.26tokens/s]
Shard 13:  77%|███████▋  | 77204088/100000000 [00:20<00:08, 2559110.89tokens/s]
Shard 13:  78%|███████▊  | 77539063/100000000 [00:20<00:08, 2756750.19tokens/s]
Shard 13:  78%|███████▊  | 78022133/100000000 [00:20<00:06, 3316603.49tokens/s]
Shard 13:  78%|███████▊  | 78389746/100000000 [00:20<00:06, 3412751.91tokens/s]
Shard 13:  79%|███████▊  | 78739841/100000000 [00:20<00:07, 2863131.28tokens/s]
Shard 13:  79%|███████▉  | 79096477/100000000 [00:20<00:06, 3028616.13tokens/s]
Shard 13:  79%|███████▉  | 79420732/100000000 [00:20<00:06, 2965492.41tokens/s]
Shard 13:  80%|███████▉  | 79729351/100000000 [00:20<00:07, 2790777.46tokens/s]
Shard 13:  80%|████████  | 80041548/100000000 [00:21<00:06, 2872416.69tokens/s]
Shard 13:  80%|████████  | 80495932/100000000 [00:21<00:05, 3316268.05tokens/s]
Shard 13:  81%|████████  | 80837559/100000000 [00:21<00:05, 3215449.02tokens/s]
Shard 13:  81%|████████  | 81166034/100000000 [00:21<00:05, 3182598.03tokens/s]
Shard 13:  81%|████████▏ | 81490093/100000000 [00:21<00:06, 2893266.47tokens/s]
Shard 13:  82%|████████▏ | 81787003/100000000 [00:21<00:06, 2825382.29tokens/s]
Shard 13:  82%|████████▏ | 82074431/100000000 [00:21<00:06, 2673339.42tokens/s]
Shard 13:  83%|████████▎ | 82630877/100000000 [00:21<00:05, 3440053.88tokens/s]
Shard 13:  83%|████████▎ | 82987749/100000000 [00:21<00:05, 3153911.24tokens/s]
Shard 13:  83%|████████▎ | 83326051/100000000 [00:22<00:05, 3178554.34tokens/s]
Shard 13:  84%|████████▍ | 83948023/100000000 [00:22<00:04, 4001621.15tokens/s]
Shard 13:  84%|████████▍ | 84388275/100000000 [00:22<00:03, 4102189.86tokens/s]
Shard 13:  85%|████████▌ | 85013465/100000000 [00:22<00:03, 4708281.87tokens/s]
Shard 13:  86%|████████▌ | 85500810/100000000 [00:22<00:03, 4728401.61tokens/s]
Shard 13:  86%|████████▌ | 85982038/100000000 [00:22<00:03, 4565169.06tokens/s]
Shard 13:  86%|████████▋ | 86445751/100000000 [00:22<00:03, 4406231.36tokens/s]
Shard 13:  87%|████████▋ | 86892044/100000000 [00:22<00:02, 4396621.11tokens/s]
Shard 13:  87%|████████▋ | 87393841/100000000 [00:22<00:02, 4563653.49tokens/s]
Shard 13:  88%|████████▊ | 87854936/100000000 [00:23<00:03, 3986081.54tokens/s]
Shard 13:  88%|████████▊ | 88276982/100000000 [00:23<00:03, 3752740.60tokens/s]
Shard 13:  89%|████████▊ | 88666649/100000000 [00:23<00:03, 3717054.31tokens/s]
Shard 13:  89%|████████▉ | 89182149/100000000 [00:23<00:02, 4100437.18tokens/s]
Shard 13:  90%|████████▉ | 89625420/100000000 [00:23<00:02, 4181766.59tokens/s]
Shard 13:  90%|█████████ | 90109581/100000000 [00:23<00:02, 4367791.06tokens/s]
Shard 13:  91%|█████████ | 90555236/100000000 [00:23<00:02, 4392978.14tokens/s]
Shard 13:  91%|█████████ | 91131784/100000000 [00:23<00:01, 4789648.47tokens/s]
Shard 13:  92%|█████████▏| 91615847/100000000 [00:23<00:01, 4555418.42tokens/s]
Shard 13:  92%|█████████▏| 92078690/100000000 [00:24<00:01, 4342981.46tokens/s]
Shard 13:  93%|█████████▎| 92546844/100000000 [00:24<00:01, 4419451.22tokens/s]
Shard 13:  93%|█████████▎| 92992993/100000000 [00:24<00:01, 4315305.80tokens/s]
Shard 13:  93%|█████████▎| 93431430/100000000 [00:24<00:01, 4330149.41tokens/s]
Shard 13:  94%|█████████▍| 93867087/100000000 [00:24<00:01, 4051959.23tokens/s]
Shard 13:  94%|█████████▍| 94277270/100000000 [00:24<00:01, 3292974.81tokens/s]
Shard 13:  95%|█████████▍| 94632233/100000000 [00:24<00:01, 3240567.84tokens/s]
Shard 13:  95%|█████████▌| 95197680/100000000 [00:24<00:01, 3845243.35tokens/s]
Shard 13:  96%|█████████▌| 95605421/100000000 [00:24<00:01, 3696875.17tokens/s]
Shard 13:  96%|█████████▌| 95991472/100000000 [00:25<00:01, 3738689.19tokens/s]
Shard 13:  97%|█████████▋| 96622379/100000000 [00:25<00:00, 4433086.07tokens/s]
Shard 13:  97%|█████████▋| 97083224/100000000 [00:25<00:00, 4370846.71tokens/s]
Shard 13:  98%|█████████▊| 97531395/100000000 [00:25<00:00, 3719780.02tokens/s]
Shard 13:  98%|█████████▊| 97926333/100000000 [00:25<00:00, 3623015.62tokens/s]
Shard 13:  99%|█████████▊| 98577811/100000000 [00:25<00:00, 4354836.33tokens/s]
Shard 13:  99%|█████████▉| 99035553/100000000 [00:25<00:00, 3732700.29tokens/s]
Shard 13:  99%|█████████▉| 99437756/100000000 [00:25<00:00, 3668117.38tokens/s]
Shard 13: 100%|█████████▉| 99827344/100000000 [00:26<00:00, 3148313.19tokens/s]
Shard 13: 100%|█████████▉| 99999922/100000000 [00:34<00:00, 2886976.48tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000013.bin (200,001,024 bytes) in the gpt-2 format

Shard 14:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 14:   0%|          | 210243/100000000 [00:00<00:47, 2096528.75tokens/s]
Shard 14:   1%|          | 541336/100000000 [00:00<00:35, 2805227.44tokens/s]
Shard 14:   1%|          | 825864/100000000 [00:00<00:43, 2262030.14tokens/s]
Shard 14:   1%|          | 1063167/100000000 [00:00<00:44, 2248217.90tokens/s]
Shard 14:   2%|▏         | 1688314/100000000 [00:00<00:27, 3543521.79tokens/s]
Shard 14:   2%|▏         | 2061382/100000000 [00:00<00:29, 3266062.67tokens/s]
Shard 14:   2%|▏         | 2476715/100000000 [00:00<00:27, 3502413.50tokens/s]
Shard 14:   3%|▎         | 2840015/100000000 [00:00<00:28, 3398736.06tokens/s]
Shard 14:   3%|▎         | 3238264/100000000 [00:01<00:27, 3549320.56tokens/s]
Shard 14:   4%|▎         | 3600573/100000000 [00:01<00:27, 3488746.40tokens/s]
Shard 14:   4%|▍         | 3954835/100000000 [00:01<00:28, 3392672.59tokens/s]
Shard 14:   4%|▍         | 4339786/100000000 [00:01<00:27, 3456676.51tokens/s]
Shard 14:   5%|▍         | 4689122/100000000 [00:01<00:27, 3443338.77tokens/s]
Shard 14:   5%|▌         | 5087560/100000000 [00:01<00:26, 3598095.52tokens/s]
Shard 14:   5%|▌         | 5449944/100000000 [00:01<00:26, 3551206.69tokens/s]
Shard 14:   6%|▌         | 5806592/100000000 [00:01<00:26, 3548700.89tokens/s]
Shard 14:   6%|▌         | 6162689/100000000 [00:01<00:28, 3272026.67tokens/s]
Shard 14:   6%|▋         | 6499727/100000000 [00:01<00:29, 3173476.60tokens/s]
Shard 14:   7%|▋         | 6820416/100000000 [00:02<00:30, 3036820.39tokens/s]
Shard 14:   7%|▋         | 7127069/100000000 [00:02<00:30, 3010377.93tokens/s]
Shard 14:   7%|▋         | 7465061/100000000 [00:02<00:29, 3109118.47tokens/s]
Shard 14:   8%|▊         | 7849276/100000000 [00:02<00:27, 3302564.23tokens/s]
Shard 14:   8%|▊         | 8206520/100000000 [00:02<00:27, 3369647.02tokens/s]
Shard 14:   9%|▊         | 8635800/100000000 [00:02<00:25, 3621024.03tokens/s]
Shard 14:   9%|▉         | 9072579/100000000 [00:02<00:23, 3834969.24tokens/s]
Shard 14:   9%|▉         | 9478923/100000000 [00:02<00:23, 3892569.04tokens/s]
Shard 14:  10%|▉         | 9870141/100000000 [00:02<00:23, 3833809.67tokens/s]
Shard 14:  10%|█         | 10292344/100000000 [00:03<00:22, 3932100.82tokens/s]
Shard 14:  11%|█         | 10687334/100000000 [00:03<00:24, 3653963.03tokens/s]
Shard 14:  11%|█         | 11170819/100000000 [00:03<00:22, 3983251.66tokens/s]
Shard 14:  12%|█▏        | 11581955/100000000 [00:03<00:22, 3994479.26tokens/s]
Shard 14:  12%|█▏        | 12435506/100000000 [00:03<00:16, 5267048.00tokens/s]
Shard 14:  13%|█▎        | 12995504/100000000 [00:03<00:16, 5352758.09tokens/s]
Shard 14:  14%|█▎        | 13534611/100000000 [00:03<00:19, 4337715.13tokens/s]
Shard 14:  14%|█▍        | 14002500/100000000 [00:03<00:21, 3962526.33tokens/s]
Shard 14:  14%|█▍        | 14426306/100000000 [00:04<00:22, 3747585.11tokens/s]
Shard 14:  15%|█▍        | 14820926/100000000 [00:04<00:23, 3672301.52tokens/s]
Shard 14:  15%|█▌        | 15343924/100000000 [00:04<00:21, 4010711.44tokens/s]
Shard 14:  16%|█▌        | 15763058/100000000 [00:04<00:24, 3416293.36tokens/s]
Shard 14:  16%|█▌        | 16126508/100000000 [00:04<00:29, 2809918.96tokens/s]
Shard 14:  16%|█▋        | 16498345/100000000 [00:04<00:27, 3000060.02tokens/s]
Shard 14:  17%|█▋        | 16825721/100000000 [00:04<00:28, 2892967.73tokens/s]
Shard 14:  17%|█▋        | 17133337/100000000 [00:04<00:30, 2702523.27tokens/s]
Shard 14:  17%|█▋        | 17419262/100000000 [00:05<00:30, 2716813.27tokens/s]
Shard 14:  18%|█▊        | 17830239/100000000 [00:05<00:26, 3068879.00tokens/s]
Shard 14:  18%|█▊        | 18150118/100000000 [00:05<00:27, 3020820.62tokens/s]
Shard 14:  18%|█▊        | 18460831/100000000 [00:05<00:27, 3001077.76tokens/s]
Shard 14:  19%|█▉        | 18788619/100000000 [00:05<00:26, 3077195.06tokens/s]
Shard 14:  19%|█▉        | 19362166/100000000 [00:05<00:21, 3816882.83tokens/s]
Shard 14:  20%|█▉        | 19774188/100000000 [00:05<00:20, 3893202.79tokens/s]
Shard 14:  20%|██        | 20169094/100000000 [00:05<00:22, 3593629.37tokens/s]
Shard 14:  21%|██        | 20536876/100000000 [00:05<00:23, 3414045.55tokens/s]
Shard 14:  21%|██        | 20885256/100000000 [00:06<00:25, 3088740.15tokens/s]
Shard 14:  21%|██        | 21202864/100000000 [00:06<00:26, 3002204.25tokens/s]
Shard 14:  22%|██▏       | 21714462/100000000 [00:06<00:22, 3523613.52tokens/s]
Shard 14:  22%|██▏       | 22076708/100000000 [00:06<00:23, 3351616.98tokens/s]
Shard 14:  23%|██▎       | 22533307/100000000 [00:06<00:21, 3672847.42tokens/s]
Shard 14:  23%|██▎       | 23022567/100000000 [00:06<00:19, 4002698.16tokens/s]
Shard 14:  23%|██▎       | 23433714/100000000 [00:06<00:20, 3768678.93tokens/s]
Shard 14:  24%|██▍       | 23952424/100000000 [00:06<00:18, 4143104.62tokens/s]
Shard 14:  24%|██▍       | 24494318/100000000 [00:06<00:16, 4488022.71tokens/s]
Shard 14:  25%|██▍       | 24969347/100000000 [00:07<00:17, 4343099.34tokens/s]
Shard 14:  25%|██▌       | 25460924/100000000 [00:07<00:16, 4485797.42tokens/s]
Shard 14:  26%|██▌       | 25930044/100000000 [00:07<00:18, 4109301.14tokens/s]
Shard 14:  26%|██▋       | 26351458/100000000 [00:07<00:19, 3844077.43tokens/s]
Shard 14:  27%|██▋       | 27061874/100000000 [00:07<00:15, 4698338.31tokens/s]
Shard 14:  28%|██▊       | 27549700/100000000 [00:07<00:18, 3940307.29tokens/s]
Shard 14:  28%|██▊       | 27978836/100000000 [00:07<00:19, 3665614.20tokens/s]
Shard 14:  28%|██▊       | 28368105/100000000 [00:07<00:19, 3689179.34tokens/s]
Shard 14:  29%|██▉       | 28754093/100000000 [00:08<00:19, 3626974.00tokens/s]
Shard 14:  29%|██▉       | 29128105/100000000 [00:08<00:19, 3608922.73tokens/s]
Shard 14:  30%|██▉       | 29564824/100000000 [00:08<00:18, 3812691.02tokens/s]
Shard 14:  30%|██▉       | 29954804/100000000 [00:08<00:21, 3203298.65tokens/s]
Shard 14:  32%|███▏      | 31917316/100000000 [00:08<00:09, 7276870.40tokens/s]
Shard 14:  33%|███▎      | 32740704/100000000 [00:08<00:12, 5389405.34tokens/s]
Shard 14:  33%|███▎      | 33430490/100000000 [00:08<00:13, 4784559.22tokens/s]
Shard 14:  34%|███▍      | 34013958/100000000 [00:09<00:13, 4725142.59tokens/s]
Shard 14:  35%|███▍      | 34556422/100000000 [00:09<00:15, 4361942.58tokens/s]
Shard 14:  35%|███▌      | 35183252/100000000 [00:09<00:13, 4762673.04tokens/s]
Shard 14:  36%|███▌      | 35709048/100000000 [00:09<00:13, 4612645.50tokens/s]
Shard 14:  36%|███▌      | 36204120/100000000 [00:09<00:15, 4041765.73tokens/s]
Shard 14:  37%|███▋      | 36648113/100000000 [00:09<00:18, 3421868.36tokens/s]
Shard 14:  37%|███▋      | 37071706/100000000 [00:09<00:17, 3585618.93tokens/s]
Shard 14:  37%|███▋      | 37459674/100000000 [00:10<00:19, 3281121.12tokens/s]
Shard 14:  38%|███▊      | 37893792/100000000 [00:10<00:17, 3507668.97tokens/s]
Shard 14:  38%|███▊      | 38266382/100000000 [00:10<00:18, 3278135.02tokens/s]
Shard 14:  39%|███▊      | 38619865/100000000 [00:10<00:18, 3334557.98tokens/s]
Shard 14:  39%|███▉      | 38966487/100000000 [00:10<00:18, 3240820.34tokens/s]
Shard 14:  39%|███▉      | 39298860/100000000 [00:10<00:20, 3017625.84tokens/s]
Shard 14:  40%|███▉      | 39607572/100000000 [00:10<00:20, 2891746.54tokens/s]
Shard 14:  40%|███▉      | 39902211/100000000 [00:10<00:23, 2585076.60tokens/s]
Shard 14:  40%|████      | 40202425/100000000 [00:11<00:22, 2678254.83tokens/s]
Shard 14:  40%|████      | 40477101/100000000 [00:11<00:25, 2346236.68tokens/s]
Shard 14:  41%|████      | 40721804/100000000 [00:11<00:28, 2080155.32tokens/s]
Shard 14:  41%|████      | 40940205/100000000 [00:11<00:28, 2097194.52tokens/s]
Shard 14:  41%|████      | 41177836/100000000 [00:11<00:27, 2159042.99tokens/s]
Shard 14:  41%|████▏     | 41464879/100000000 [00:11<00:24, 2345137.14tokens/s]
Shard 14:  42%|████▏     | 41735279/100000000 [00:11<00:23, 2441818.55tokens/s]
Shard 14:  42%|████▏     | 42130387/100000000 [00:11<00:20, 2854895.91tokens/s]
Shard 14:  42%|████▏     | 42466781/100000000 [00:11<00:19, 3000093.99tokens/s]
Shard 14:  43%|████▎     | 42848038/100000000 [00:12<00:17, 3225970.30tokens/s]
Shard 14:  43%|████▎     | 43225076/100000000 [00:12<00:16, 3370063.94tokens/s]
Shard 14:  44%|████▎     | 43634359/100000000 [00:12<00:15, 3575564.67tokens/s]
Shard 14:  44%|████▍     | 43995066/100000000 [00:12<00:15, 3564447.85tokens/s]
Shard 14:  44%|████▍     | 44353462/100000000 [00:12<00:15, 3491845.49tokens/s]
Shard 14:  45%|████▍     | 44793233/100000000 [00:12<00:14, 3751805.25tokens/s]
Shard 14:  45%|████▌     | 45208488/100000000 [00:12<00:14, 3828711.84tokens/s]
Shard 14:  46%|████▌     | 45592675/100000000 [00:12<00:15, 3491390.40tokens/s]
Shard 14:  46%|████▌     | 46184148/100000000 [00:12<00:12, 4161403.32tokens/s]
Shard 14:  47%|████▋     | 46613817/100000000 [00:13<00:14, 3639367.74tokens/s]
Shard 14:  47%|████▋     | 47158647/100000000 [00:13<00:12, 4102324.81tokens/s]
Shard 14:  48%|████▊     | 47588443/100000000 [00:13<00:13, 3829318.41tokens/s]
Shard 14:  48%|████▊     | 47987486/100000000 [00:13<00:14, 3472466.51tokens/s]
Shard 14:  48%|████▊     | 48350684/100000000 [00:13<00:15, 3416419.98tokens/s]
Shard 14:  49%|████▊     | 48707246/100000000 [00:13<00:15, 3297726.35tokens/s]
Shard 14:  49%|████▉     | 49056951/100000000 [00:13<00:15, 3349720.78tokens/s]
Shard 14:  49%|████▉     | 49397473/100000000 [00:13<00:15, 3309979.44tokens/s]
Shard 14:  50%|████▉     | 49766625/100000000 [00:13<00:14, 3400312.10tokens/s]
Shard 14:  50%|█████     | 50256868/100000000 [00:14<00:13, 3821751.91tokens/s]
Shard 14:  51%|█████     | 50692632/100000000 [00:14<00:12, 3960826.23tokens/s]
Shard 14:  51%|█████     | 51092900/100000000 [00:14<00:13, 3565457.63tokens/s]
Shard 14:  51%|█████▏    | 51490624/100000000 [00:14<00:13, 3666022.92tokens/s]
Shard 14:  52%|█████▏    | 51865254/100000000 [00:14<00:13, 3572068.48tokens/s]
Shard 14:  52%|█████▏    | 52227912/100000000 [00:14<00:14, 3378296.31tokens/s]
Shard 14:  53%|█████▎    | 52757043/100000000 [00:14<00:12, 3888715.32tokens/s]
Shard 14:  53%|█████▎    | 53154286/100000000 [00:14<00:14, 3205945.49tokens/s]
Shard 14:  54%|█████▎    | 53544157/100000000 [00:15<00:13, 3362663.81tokens/s]
Shard 14:  54%|█████▍    | 54025887/100000000 [00:15<00:12, 3684397.86tokens/s]
Shard 14:  54%|█████▍    | 54412077/100000000 [00:15<00:14, 3234174.05tokens/s]
Shard 14:  55%|█████▍    | 54755811/100000000 [00:15<00:14, 3220359.74tokens/s]
Shard 14:  55%|█████▌    | 55092772/100000000 [00:15<00:14, 3197750.80tokens/s]
Shard 14:  56%|█████▌    | 55531710/100000000 [00:15<00:12, 3505764.37tokens/s]
Shard 14:  56%|█████▌    | 55892989/100000000 [00:15<00:12, 3456149.92tokens/s]
Shard 14:  56%|█████▌    | 56246731/100000000 [00:15<00:13, 3291748.90tokens/s]
Shard 14:  57%|█████▋    | 56581741/100000000 [00:15<00:13, 3174253.58tokens/s]
Shard 14:  57%|█████▋    | 56989174/100000000 [00:16<00:12, 3410214.40tokens/s]
Shard 14:  57%|█████▋    | 57482784/100000000 [00:16<00:11, 3836009.79tokens/s]
Shard 14:  58%|█████▊    | 57902210/100000000 [00:16<00:10, 3922134.57tokens/s]
Shard 14:  58%|█████▊    | 58475992/100000000 [00:16<00:09, 4445958.11tokens/s]
Shard 14:  59%|█████▉    | 58926490/100000000 [00:16<00:10, 4034467.12tokens/s]
Shard 14:  59%|█████▉    | 59342917/100000000 [00:16<00:11, 3568602.38tokens/s]
Shard 14:  60%|█████▉    | 59789213/100000000 [00:16<00:10, 3783952.25tokens/s]
Shard 14:  60%|██████    | 60182477/100000000 [00:16<00:10, 3726604.59tokens/s]
Shard 14:  61%|██████    | 60669074/100000000 [00:16<00:09, 4030293.93tokens/s]
Shard 14:  61%|██████    | 61092088/100000000 [00:17<00:09, 4063664.38tokens/s]
Shard 14:  62%|██████▏   | 61506363/100000000 [00:17<00:10, 3550734.77tokens/s]
Shard 14:  62%|██████▏   | 61882173/100000000 [00:17<00:10, 3550713.95tokens/s]
Shard 14:  62%|██████▏   | 62249811/100000000 [00:17<00:10, 3526808.61tokens/s]
Shard 14:  63%|██████▎   | 62658186/100000000 [00:17<00:10, 3677344.84tokens/s]
Shard 14:  63%|██████▎   | 63081514/100000000 [00:17<00:09, 3791304.94tokens/s]
Shard 14:  63%|██████▎   | 63490663/100000000 [00:17<00:09, 3864525.78tokens/s]
Shard 14:  64%|██████▍   | 63881098/100000000 [00:17<00:09, 3820815.28tokens/s]
Shard 14:  64%|██████▍   | 64345535/100000000 [00:17<00:08, 4047251.26tokens/s]
Shard 14:  65%|██████▍   | 64753100/100000000 [00:18<00:10, 3436399.32tokens/s]
Shard 14:  65%|██████▌   | 65196226/100000000 [00:18<00:09, 3692311.95tokens/s]
Shard 14:  66%|██████▌   | 65583248/100000000 [00:18<00:09, 3674816.58tokens/s]
Shard 14:  66%|██████▌   | 65962858/100000000 [00:18<00:09, 3666031.83tokens/s]
Shard 14:  66%|██████▋   | 66388472/100000000 [00:18<00:08, 3823180.35tokens/s]
Shard 14:  67%|██████▋   | 66778667/100000000 [00:18<00:08, 3761969.94tokens/s]
Shard 14:  67%|██████▋   | 67159730/100000000 [00:18<00:08, 3739357.19tokens/s]
Shard 14:  68%|██████▊   | 67536918/100000000 [00:18<00:09, 3324352.83tokens/s]
Shard 14:  68%|██████▊   | 67910191/100000000 [00:18<00:09, 3433348.57tokens/s]
Shard 14:  68%|██████▊   | 68358487/100000000 [00:19<00:08, 3705916.96tokens/s]
Shard 14:  69%|██████▉   | 68848197/100000000 [00:19<00:07, 4039994.66tokens/s]
Shard 14:  69%|██████▉   | 69324312/100000000 [00:19<00:07, 4245890.18tokens/s]
Shard 14:  70%|██████▉   | 69758236/100000000 [00:19<00:07, 4104159.33tokens/s]
Shard 14:  70%|███████   | 70241000/100000000 [00:19<00:06, 4310331.13tokens/s]
Shard 14:  71%|███████   | 70676735/100000000 [00:19<00:06, 4312101.27tokens/s]
Shard 14:  71%|███████   | 71111416/100000000 [00:19<00:06, 4222346.68tokens/s]
Shard 14:  72%|███████▏  | 71536322/100000000 [00:19<00:07, 3959403.46tokens/s]
Shard 14:  72%|███████▏  | 71940811/100000000 [00:19<00:07, 3683217.53tokens/s]
Shard 14:  72%|███████▏  | 72315271/100000000 [00:20<00:07, 3478803.06tokens/s]
Shard 14:  73%|███████▎  | 72671670/100000000 [00:20<00:07, 3493256.10tokens/s]
Shard 14:  73%|███████▎  | 73057225/100000000 [00:20<00:07, 3590309.58tokens/s]
Shard 14:  73%|███████▎  | 73473949/100000000 [00:20<00:07, 3749259.58tokens/s]
Shard 14:  74%|███████▍  | 73922388/100000000 [00:20<00:06, 3946896.17tokens/s]
Shard 14:  74%|███████▍  | 74321761/100000000 [00:20<00:06, 3933148.03tokens/s]
Shard 14:  75%|███████▍  | 74733449/100000000 [00:20<00:06, 3980155.96tokens/s]
Shard 14:  75%|███████▌  | 75134453/100000000 [00:20<00:07, 3458728.00tokens/s]
Shard 14:  76%|███████▌  | 75534980/100000000 [00:20<00:06, 3600289.77tokens/s]
Shard 14:  76%|███████▌  | 75966143/100000000 [00:21<00:06, 3796356.46tokens/s]
Shard 14:  76%|███████▋  | 76355233/100000000 [00:21<00:06, 3747998.14tokens/s]
Shard 14:  77%|███████▋  | 76739621/100000000 [00:21<00:06, 3767966.98tokens/s]
Shard 14:  77%|███████▋  | 77121508/100000000 [00:21<00:06, 3636616.93tokens/s]
Shard 14:  78%|███████▊  | 77752122/100000000 [00:21<00:05, 4380539.11tokens/s]
Shard 14:  78%|███████▊  | 78213969/100000000 [00:21<00:04, 4444903.65tokens/s]
Shard 14:  79%|███████▊  | 78663466/100000000 [00:21<00:05, 4105429.48tokens/s]
Shard 14:  79%|███████▉  | 79082357/100000000 [00:21<00:05, 3702725.15tokens/s]
Shard 14:  80%|███████▉  | 79521095/100000000 [00:21<00:05, 3866676.56tokens/s]
Shard 14:  80%|███████▉  | 79918166/100000000 [00:22<00:05, 3546841.20tokens/s]
Shard 14:  80%|████████  | 80362225/100000000 [00:22<00:05, 3756628.70tokens/s]
Shard 14:  81%|████████  | 80751456/100000000 [00:22<00:05, 3720080.60tokens/s]
Shard 14:  81%|████████  | 81130646/100000000 [00:22<00:05, 3707650.86tokens/s]
Shard 14:  82%|████████▏ | 81722126/100000000 [00:22<00:04, 4302448.37tokens/s]
Shard 14:  82%|████████▏ | 82160888/100000000 [00:22<00:04, 4308952.23tokens/s]
Shard 14:  83%|████████▎ | 83261196/100000000 [00:22<00:02, 6238085.48tokens/s]
Shard 14:  84%|████████▍ | 83896626/100000000 [00:22<00:02, 5680520.65tokens/s]
Shard 14:  84%|████████▍ | 84481960/100000000 [00:22<00:03, 4958401.12tokens/s]
Shard 14:  85%|████████▌ | 85004867/100000000 [00:23<00:03, 4499130.83tokens/s]
Shard 14:  85%|████████▌ | 85493116/100000000 [00:23<00:03, 4584368.40tokens/s]
Shard 14:  86%|████████▌ | 86237328/100000000 [00:23<00:02, 5304534.32tokens/s]
Shard 14:  87%|████████▋ | 86791015/100000000 [00:23<00:02, 4786955.12tokens/s]
Shard 14:  87%|████████▋ | 87293214/100000000 [00:23<00:02, 4591401.90tokens/s]
Shard 14:  88%|████████▊ | 87768323/100000000 [00:23<00:02, 4331126.38tokens/s]
Shard 14:  88%|████████▊ | 88213365/100000000 [00:23<00:02, 4243711.40tokens/s]
Shard 14:  89%|████████▊ | 88651303/100000000 [00:23<00:03, 3731028.22tokens/s]
Shard 14:  89%|████████▉ | 89100081/100000000 [00:24<00:02, 3914616.30tokens/s]
Shard 14:  90%|████████▉ | 89506394/100000000 [00:24<00:02, 3579791.15tokens/s]
Shard 14:  90%|████████▉ | 89910300/100000000 [00:24<00:02, 3694675.09tokens/s]
Shard 14:  90%|█████████ | 90291205/100000000 [00:24<00:02, 3498060.50tokens/s]
Shard 14:  91%|█████████ | 90833405/100000000 [00:24<00:02, 3992306.30tokens/s]
Shard 14:  91%|█████████▏| 91254869/100000000 [00:24<00:02, 3783550.62tokens/s]
Shard 14:  92%|█████████▏| 91643350/100000000 [00:24<00:02, 3787057.93tokens/s]
Shard 14:  92%|█████████▏| 92326452/100000000 [00:24<00:01, 4611683.15tokens/s]
Shard 14:  93%|█████████▎| 92805808/100000000 [00:25<00:01, 3957570.85tokens/s]
Shard 14:  93%|█████████▎| 93282793/100000000 [00:25<00:01, 4158195.46tokens/s]
Shard 14:  94%|█████████▎| 93718909/100000000 [00:25<00:01, 3890838.63tokens/s]
Shard 14:  94%|█████████▍| 94125234/100000000 [00:25<00:01, 3391791.11tokens/s]
Shard 14:  95%|█████████▍| 94541654/100000000 [00:25<00:01, 3561438.52tokens/s]
Shard 14:  95%|█████████▌| 95079196/100000000 [00:25<00:01, 4021903.61tokens/s]
Shard 14:  96%|█████████▌| 95562606/100000000 [00:25<00:01, 4235283.06tokens/s]
Shard 14:  96%|█████████▌| 96019873/100000000 [00:25<00:00, 4255660.50tokens/s]
Shard 14:  96%|█████████▋| 96457721/100000000 [00:25<00:00, 4272756.87tokens/s]
Shard 14:  97%|█████████▋| 96893621/100000000 [00:26<00:00, 3952489.31tokens/s]
Shard 14:  97%|█████████▋| 97299183/100000000 [00:26<00:00, 3580341.20tokens/s]
Shard 14:  98%|█████████▊| 97669500/100000000 [00:26<00:00, 3278472.81tokens/s]
Shard 14:  98%|█████████▊| 98063954/100000000 [00:26<00:00, 3418936.45tokens/s]
Shard 14:  99%|█████████▊| 98695948/100000000 [00:26<00:00, 4173863.37tokens/s]
Shard 14:  99%|█████████▉| 99130587/100000000 [00:26<00:00, 4057064.82tokens/s]
Shard 14: 100%|█████████▉| 99548410/100000000 [00:26<00:00, 3347995.04tokens/s]
Shard 14: 100%|█████████▉| 99910660/100000000 [00:27<00:00, 2961454.23tokens/s]
Shard 14: 100%|█████████▉| 99999883/100000000 [00:36<00:00, 2762704.09tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000014.bin (200,001,024 bytes) in the gpt-2 format

Shard 15:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 15:   1%|          | 510760/100000000 [00:00<00:20, 4858158.48tokens/s]
Shard 15:   1%|          | 996700/100000000 [00:00<00:34, 2906769.96tokens/s]
Shard 15:   1%|▏         | 1388616/100000000 [00:00<00:30, 3235980.67tokens/s]
Shard 15:   2%|▏         | 1747934/100000000 [00:00<00:31, 3107759.21tokens/s]
Shard 15:   2%|▏         | 2079463/100000000 [00:00<00:34, 2872065.10tokens/s]
Shard 15:   2%|▏         | 2381429/100000000 [00:00<00:33, 2886883.58tokens/s]
Shard 15:   3%|▎         | 2733345/100000000 [00:00<00:31, 3060899.93tokens/s]
Shard 15:   3%|▎         | 3051540/100000000 [00:00<00:31, 3073205.84tokens/s]
Shard 15:   3%|▎         | 3434566/100000000 [00:01<00:29, 3292346.75tokens/s]
Shard 15:   4%|▍         | 3947063/100000000 [00:01<00:25, 3814312.95tokens/s]
Shard 15:   4%|▍         | 4359565/100000000 [00:01<00:24, 3870989.23tokens/s]
Shard 15:   5%|▍         | 4752089/100000000 [00:01<00:26, 3557421.09tokens/s]
Shard 15:   5%|▌         | 5181286/100000000 [00:01<00:25, 3748391.94tokens/s]
Shard 15:   6%|▌         | 5563433/100000000 [00:01<00:25, 3717973.38tokens/s]
Shard 15:   6%|▌         | 5940312/100000000 [00:01<00:26, 3535450.05tokens/s]
Shard 15:   6%|▋         | 6298398/100000000 [00:01<00:28, 3333022.31tokens/s]
Shard 15:   7%|▋         | 6638716/100000000 [00:01<00:28, 3298497.71tokens/s]
Shard 15:   7%|▋         | 7049539/100000000 [00:02<00:26, 3506628.69tokens/s]
Shard 15:   7%|▋         | 7414069/100000000 [00:02<00:26, 3491410.93tokens/s]
Shard 15:   8%|▊         | 7765766/100000000 [00:02<00:27, 3305405.57tokens/s]
Shard 15:   8%|▊         | 8099592/100000000 [00:02<00:30, 3009796.24tokens/s]
Shard 15:   8%|▊         | 8434197/100000000 [00:02<00:29, 3098672.58tokens/s]
Shard 15:   9%|▉         | 8864721/100000000 [00:02<00:26, 3426903.62tokens/s]
Shard 15:   9%|▉         | 9214805/100000000 [00:02<00:26, 3432092.91tokens/s]
Shard 15:  10%|▉         | 9669376/100000000 [00:02<00:24, 3740451.28tokens/s]
Shard 15:  10%|█         | 10116700/100000000 [00:02<00:22, 3929455.30tokens/s]
Shard 15:  11%|█         | 10514065/100000000 [00:03<00:24, 3704765.47tokens/s]
Shard 15:  11%|█         | 10890187/100000000 [00:03<00:25, 3442315.99tokens/s]
Shard 15:  11%|█         | 11240746/100000000 [00:03<00:28, 3083649.36tokens/s]
Shard 15:  12%|█▏        | 11559765/100000000 [00:03<00:30, 2917578.91tokens/s]
Shard 15:  12%|█▏        | 11858291/100000000 [00:03<00:32, 2671812.57tokens/s]
Shard 15:  12%|█▏        | 12132207/100000000 [00:03<00:35, 2481917.28tokens/s]
Shard 15:  13%|█▎        | 12566936/100000000 [00:03<00:29, 2941408.48tokens/s]
Shard 15:  13%|█▎        | 13087875/100000000 [00:03<00:24, 3533127.12tokens/s]
Shard 15:  14%|█▍        | 13774053/100000000 [00:04<00:19, 4426135.36tokens/s]
Shard 15:  14%|█▍        | 14365041/100000000 [00:04<00:17, 4821799.03tokens/s]
Shard 15:  15%|█▍        | 14865165/100000000 [00:04<00:30, 2829101.73tokens/s]
Shard 15:  15%|█▌        | 15383353/100000000 [00:04<00:25, 3274512.83tokens/s]
Shard 15:  16%|█▌        | 15812812/100000000 [00:04<00:24, 3469190.60tokens/s]
Shard 15:  16%|█▋        | 16287926/100000000 [00:04<00:22, 3767662.87tokens/s]
Shard 15:  17%|█▋        | 16738889/100000000 [00:04<00:21, 3858617.36tokens/s]
Shard 15:  17%|█▋        | 17189957/100000000 [00:05<00:20, 4024394.56tokens/s]
Shard 15:  18%|█▊        | 17688918/100000000 [00:05<00:19, 4264006.07tokens/s]
Shard 15:  18%|█▊        | 18142000/100000000 [00:05<00:19, 4152852.78tokens/s]
Shard 15:  19%|█▉        | 18801019/100000000 [00:05<00:16, 4806394.23tokens/s]
Shard 15:  19%|█▉        | 19300306/100000000 [00:05<00:16, 4811675.39tokens/s]
Shard 15:  20%|█▉        | 19821274/100000000 [00:05<00:16, 4909483.57tokens/s]
Shard 15:  20%|██        | 20322035/100000000 [00:05<00:20, 3929824.08tokens/s]
Shard 15:  21%|██        | 20752038/100000000 [00:05<00:21, 3661728.08tokens/s]
Shard 15:  21%|██        | 21166810/100000000 [00:05<00:20, 3774909.66tokens/s]
Shard 15:  22%|██▏       | 21568239/100000000 [00:06<00:22, 3531521.43tokens/s]
Shard 15:  22%|██▏       | 21937762/100000000 [00:06<00:22, 3435321.44tokens/s]
Shard 15:  22%|██▏       | 22320757/100000000 [00:06<00:22, 3526040.46tokens/s]
Shard 15:  23%|██▎       | 22718246/100000000 [00:06<00:21, 3636940.60tokens/s]
Shard 15:  23%|██▎       | 23321050/100000000 [00:06<00:17, 4298731.97tokens/s]
Shard 15:  24%|██▍       | 23761990/100000000 [00:06<00:17, 4287991.63tokens/s]
Shard 15:  24%|██▍       | 24229990/100000000 [00:06<00:17, 4398099.35tokens/s]
Shard 15:  25%|██▍       | 24769016/100000000 [00:06<00:16, 4682878.43tokens/s]
Shard 15:  25%|██▌       | 25242379/100000000 [00:06<00:17, 4372950.46tokens/s]
Shard 15:  26%|██▌       | 25688507/100000000 [00:07<00:17, 4179806.28tokens/s]
Shard 15:  26%|██▋       | 26253352/100000000 [00:07<00:16, 4580218.50tokens/s]
Shard 15:  27%|██▋       | 26720582/100000000 [00:07<00:16, 4380173.56tokens/s]
Shard 15:  27%|██▋       | 27165548/100000000 [00:07<00:16, 4336911.65tokens/s]
Shard 15:  28%|██▊       | 27606404/100000000 [00:07<00:16, 4330081.01tokens/s]
Shard 15:  28%|██▊       | 28042688/100000000 [00:07<00:20, 3459640.17tokens/s]
Shard 15:  28%|██▊       | 28419318/100000000 [00:07<00:23, 3092499.12tokens/s]
Shard 15:  29%|██▉       | 28754870/100000000 [00:07<00:25, 2778715.70tokens/s]
Shard 15:  29%|██▉       | 29054018/100000000 [00:08<00:26, 2727040.67tokens/s]
Shard 15:  29%|██▉       | 29339751/100000000 [00:08<00:27, 2596476.98tokens/s]
Shard 15:  30%|██▉       | 29613956/100000000 [00:08<00:26, 2628219.77tokens/s]
Shard 15:  30%|██▉       | 29945012/100000000 [00:08<00:25, 2794716.92tokens/s]
Shard 15:  30%|███       | 30233451/100000000 [00:08<00:27, 2553942.22tokens/s]
Shard 15:  31%|███       | 30578605/100000000 [00:08<00:24, 2778605.21tokens/s]
Shard 15:  31%|███       | 30919995/100000000 [00:08<00:23, 2947587.77tokens/s]
Shard 15:  31%|███       | 31223129/100000000 [00:08<00:23, 2965887.77tokens/s]
Shard 15:  32%|███▏      | 31617970/100000000 [00:08<00:21, 3240929.04tokens/s]
Shard 15:  32%|███▏      | 31947853/100000000 [00:09<00:22, 3060754.63tokens/s]
Shard 15:  32%|███▏      | 32293991/100000000 [00:09<00:21, 3095754.87tokens/s]
Shard 15:  33%|███▎      | 32662648/100000000 [00:09<00:20, 3253763.63tokens/s]
Shard 15:  33%|███▎      | 33004744/100000000 [00:09<00:22, 3022730.02tokens/s]
Shard 15:  33%|███▎      | 33312989/100000000 [00:09<00:22, 2976418.89tokens/s]
Shard 15:  34%|███▍      | 33751543/100000000 [00:09<00:19, 3345694.82tokens/s]
Shard 15:  34%|███▍      | 34094410/100000000 [00:09<00:21, 3110741.45tokens/s]
Shard 15:  34%|███▍      | 34412269/100000000 [00:09<00:21, 3022964.20tokens/s]
Shard 15:  35%|███▍      | 34718897/100000000 [00:10<00:22, 2898431.55tokens/s]
Shard 15:  35%|███▌      | 35011941/100000000 [00:10<00:27, 2396468.47tokens/s]
Shard 15:  35%|███▌      | 35271189/100000000 [00:10<00:30, 2150769.78tokens/s]
Shard 15:  36%|███▌      | 35500149/100000000 [00:10<00:29, 2167323.12tokens/s]
Shard 15:  36%|███▌      | 36092885/100000000 [00:10<00:20, 3077708.60tokens/s]
Shard 15:  37%|███▋      | 36504670/100000000 [00:10<00:18, 3347138.82tokens/s]
Shard 15:  37%|███▋      | 36861016/100000000 [00:10<00:18, 3401431.34tokens/s]
Shard 15:  37%|███▋      | 37216058/100000000 [00:10<00:19, 3302047.54tokens/s]
Shard 15:  38%|███▊      | 37561059/100000000 [00:10<00:18, 3320636.21tokens/s]
Shard 15:  38%|███▊      | 38290089/100000000 [00:11<00:13, 4408921.57tokens/s]
Shard 15:  39%|███▉      | 38797619/100000000 [00:11<00:13, 4582081.57tokens/s]
Shard 15:  39%|███▉      | 39263920/100000000 [00:11<00:13, 4569436.42tokens/s]
Shard 15:  40%|███▉      | 39726823/100000000 [00:11<00:15, 3886452.16tokens/s]
Shard 15:  40%|████      | 40137991/100000000 [00:11<00:16, 3675780.46tokens/s]
Shard 15:  41%|████      | 40522507/100000000 [00:11<00:16, 3526933.83tokens/s]
Shard 15:  41%|████      | 40887853/100000000 [00:11<00:16, 3514998.55tokens/s]
Shard 15:  41%|████▏     | 41282009/100000000 [00:11<00:16, 3600105.34tokens/s]
Shard 15:  42%|████▏     | 41682059/100000000 [00:11<00:15, 3708872.43tokens/s]
Shard 15:  42%|████▏     | 42138814/100000000 [00:12<00:14, 3936015.06tokens/s]
Shard 15:  43%|████▎     | 42564644/100000000 [00:12<00:14, 4009187.04tokens/s]
Shard 15:  43%|████▎     | 43205181/100000000 [00:12<00:12, 4675066.71tokens/s]
Shard 15:  44%|████▎     | 43676792/100000000 [00:12<00:15, 3750730.57tokens/s]
Shard 15:  44%|████▍     | 44128116/100000000 [00:12<00:14, 3940903.12tokens/s]
Shard 15:  45%|████▍     | 44549353/100000000 [00:12<00:16, 3462672.51tokens/s]
Shard 15:  45%|████▍     | 44931307/100000000 [00:12<00:15, 3538721.50tokens/s]
Shard 15:  45%|████▌     | 45305451/100000000 [00:12<00:15, 3529213.04tokens/s]
Shard 15:  46%|████▌     | 45675658/100000000 [00:13<00:15, 3448638.91tokens/s]
Shard 15:  46%|████▌     | 46080974/100000000 [00:13<00:15, 3580785.84tokens/s]
Shard 15:  46%|████▋     | 46452896/100000000 [00:13<00:17, 3075429.27tokens/s]
Shard 15:  47%|████▋     | 46777852/100000000 [00:13<00:17, 3105761.87tokens/s]
Shard 15:  47%|████▋     | 47158272/100000000 [00:13<00:16, 3281643.67tokens/s]
Shard 15:  48%|████▊     | 47513515/100000000 [00:13<00:15, 3334546.99tokens/s]
Shard 15:  48%|████▊     | 47855462/100000000 [00:13<00:15, 3267229.24tokens/s]
Shard 15:  48%|████▊     | 48270350/100000000 [00:13<00:14, 3511379.08tokens/s]
Shard 15:  49%|████▊     | 48674024/100000000 [00:13<00:14, 3652280.81tokens/s]
Shard 15:  49%|████▉     | 49045577/100000000 [00:14<00:14, 3426255.07tokens/s]
Shard 15:  49%|████▉     | 49411320/100000000 [00:14<00:14, 3473609.75tokens/s]
Shard 15:  50%|████▉     | 49764142/100000000 [00:14<00:15, 3250205.33tokens/s]
Shard 15:  50%|█████     | 50424318/100000000 [00:14<00:11, 4166804.40tokens/s]
Shard 15:  51%|█████     | 50855684/100000000 [00:14<00:14, 3435322.75tokens/s]
Shard 15:  51%|█████     | 51228051/100000000 [00:14<00:15, 3080102.44tokens/s]
Shard 15:  52%|█████▏    | 51679337/100000000 [00:14<00:14, 3404195.17tokens/s]
Shard 15:  52%|█████▏    | 52068537/100000000 [00:14<00:13, 3514190.70tokens/s]
Shard 15:  52%|█████▏    | 52439561/100000000 [00:15<00:14, 3369111.48tokens/s]
Shard 15:  53%|█████▎    | 52790316/100000000 [00:15<00:14, 3332202.80tokens/s]
Shard 15:  53%|█████▎    | 53135098/100000000 [00:15<00:14, 3287852.61tokens/s]
Shard 15:  53%|█████▎    | 53486985/100000000 [00:15<00:13, 3350842.55tokens/s]
Shard 15:  54%|█████▍    | 53829629/100000000 [00:15<00:14, 3242654.82tokens/s]
Shard 15:  54%|█████▍    | 54266832/100000000 [00:15<00:12, 3552640.94tokens/s]
Shard 15:  55%|█████▍    | 54681266/100000000 [00:15<00:12, 3717193.83tokens/s]
Shard 15:  55%|█████▌    | 55060686/100000000 [00:15<00:13, 3337098.46tokens/s]
Shard 15:  55%|█████▌    | 55404549/100000000 [00:15<00:13, 3237471.73tokens/s]
Shard 15:  56%|█████▌    | 55735610/100000000 [00:16<00:13, 3244788.22tokens/s]
Shard 15:  56%|█████▌    | 56102318/100000000 [00:16<00:13, 3351309.69tokens/s]
Shard 15:  56%|█████▋    | 56444345/100000000 [00:16<00:13, 3270739.02tokens/s]
Shard 15:  57%|█████▋    | 56774857/100000000 [00:16<00:13, 3134527.33tokens/s]
Shard 15:  57%|█████▋    | 57091776/100000000 [00:16<00:14, 3025570.47tokens/s]
Shard 15:  58%|█████▊    | 57504375/100000000 [00:16<00:12, 3280846.70tokens/s]
Shard 15:  58%|█████▊    | 58450339/100000000 [00:16<00:08, 5007505.78tokens/s]
Shard 15:  59%|█████▉    | 59042860/100000000 [00:16<00:07, 5261552.74tokens/s]
Shard 15:  60%|█████▉    | 59579852/100000000 [00:16<00:08, 4866549.86tokens/s]
Shard 15:  60%|██████    | 60080014/100000000 [00:17<00:08, 4691032.80tokens/s]
Shard 15:  61%|██████    | 60575856/100000000 [00:17<00:08, 4750106.05tokens/s]
Shard 15:  61%|██████    | 61058183/100000000 [00:17<00:08, 4751366.87tokens/s]
Shard 15:  62%|██████▏   | 61538679/100000000 [00:17<00:08, 4541847.98tokens/s]
Shard 15:  62%|██████▏   | 62001330/100000000 [00:17<00:08, 4257035.55tokens/s]
Shard 15:  62%|██████▏   | 62433464/100000000 [00:17<00:09, 3832597.52tokens/s]
Shard 15:  63%|██████▎   | 62922623/100000000 [00:17<00:09, 4095278.72tokens/s]
Shard 15:  63%|██████▎   | 63343616/100000000 [00:17<00:08, 4094499.47tokens/s]
Shard 15:  64%|██████▍   | 63768817/100000000 [00:17<00:09, 3995170.08tokens/s]
Shard 15:  64%|██████▍   | 64175496/100000000 [00:18<00:09, 3942417.12tokens/s]
Shard 15:  65%|██████▍   | 64573511/100000000 [00:18<00:10, 3228955.30tokens/s]
Shard 15:  65%|██████▍   | 64918780/100000000 [00:18<00:10, 3224150.67tokens/s]
Shard 15:  65%|██████▌   | 65256902/100000000 [00:18<00:11, 3152708.60tokens/s]
Shard 15:  66%|██████▌   | 65586885/100000000 [00:18<00:11, 2918456.57tokens/s]
Shard 15:  66%|██████▌   | 65889743/100000000 [00:18<00:11, 2886460.43tokens/s]
Shard 15:  66%|██████▋   | 66264972/100000000 [00:18<00:10, 3103008.78tokens/s]
Shard 15:  67%|██████▋   | 66583172/100000000 [00:18<00:10, 3124157.87tokens/s]
Shard 15:  67%|██████▋   | 66938750/100000000 [00:19<00:10, 3241581.20tokens/s]
Shard 15:  67%|██████▋   | 67305286/100000000 [00:19<00:09, 3355022.90tokens/s]
Shard 15:  68%|██████▊   | 67790328/100000000 [00:19<00:08, 3767763.12tokens/s]
Shard 15:  68%|██████▊   | 68171532/100000000 [00:19<00:08, 3594466.90tokens/s]
Shard 15:  69%|██████▊   | 68535165/100000000 [00:19<00:08, 3555702.92tokens/s]
Shard 15:  69%|██████▉   | 68919224/100000000 [00:19<00:08, 3630193.24tokens/s]
Shard 15:  69%|██████▉   | 69285381/100000000 [00:19<00:08, 3429951.58tokens/s]
Shard 15:  70%|██████▉   | 69821902/100000000 [00:19<00:07, 3930344.94tokens/s]
Shard 15:  70%|███████   | 70258711/100000000 [00:19<00:07, 4042356.12tokens/s]
Shard 15:  71%|███████   | 70666945/100000000 [00:19<00:07, 4028076.61tokens/s]
Shard 15:  71%|███████   | 71073768/100000000 [00:20<00:07, 4034495.66tokens/s]
Shard 15:  72%|███████▏  | 71520877/100000000 [00:20<00:06, 4150222.43tokens/s]
Shard 15:  72%|███████▏  | 71972756/100000000 [00:20<00:06, 4257755.61tokens/s]
Shard 15:  72%|███████▏  | 72400189/100000000 [00:20<00:07, 3750752.55tokens/s]
Shard 15:  73%|███████▎  | 72916647/100000000 [00:20<00:06, 4124935.91tokens/s]
Shard 15:  73%|███████▎  | 73343329/100000000 [00:20<00:06, 4164459.87tokens/s]
Shard 15:  74%|███████▍  | 73768480/100000000 [00:20<00:06, 4164923.15tokens/s]
Shard 15:  74%|███████▍  | 74192117/100000000 [00:20<00:06, 3713363.81tokens/s]
Shard 15:  75%|███████▍  | 74576230/100000000 [00:20<00:07, 3534754.56tokens/s]
Shard 15:  75%|███████▍  | 74979215/100000000 [00:21<00:06, 3649916.55tokens/s]
Shard 15:  75%|███████▌  | 75394526/100000000 [00:21<00:06, 3776177.26tokens/s]
Shard 15:  76%|███████▌  | 75778821/100000000 [00:21<00:06, 3624297.51tokens/s]
Shard 15:  76%|███████▌  | 76189240/100000000 [00:21<00:06, 3681587.29tokens/s]
Shard 15:  77%|███████▋  | 76769599/100000000 [00:21<00:05, 4271037.36tokens/s]
Shard 15:  77%|███████▋  | 77204221/100000000 [00:21<00:05, 3940836.95tokens/s]
Shard 15:  78%|███████▊  | 77607669/100000000 [00:21<00:05, 3774842.38tokens/s]
Shard 15:  78%|███████▊  | 78208272/100000000 [00:21<00:04, 4364537.35tokens/s]
Shard 15:  79%|███████▊  | 78655515/100000000 [00:21<00:04, 4276423.36tokens/s]
Shard 15:  79%|███████▉  | 79090663/100000000 [00:22<00:05, 4103648.33tokens/s]
Shard 15:  80%|███████▉  | 79507912/100000000 [00:22<00:05, 3824631.53tokens/s]
Shard 15:  80%|███████▉  | 79897108/100000000 [00:22<00:06, 3346938.81tokens/s]
Shard 15:  80%|████████  | 80244708/100000000 [00:22<00:06, 3256747.17tokens/s]
Shard 15:  81%|████████  | 80579162/100000000 [00:22<00:06, 3123019.68tokens/s]
Shard 15:  81%|████████  | 80898082/100000000 [00:22<00:06, 3134228.01tokens/s]
Shard 15:  81%|████████▏ | 81355537/100000000 [00:22<00:05, 3503029.51tokens/s]
Shard 15:  82%|████████▏ | 81816810/100000000 [00:22<00:04, 3808926.09tokens/s]
Shard 15:  82%|████████▏ | 82236568/100000000 [00:23<00:04, 3914625.15tokens/s]
Shard 15:  83%|████████▎ | 82634187/100000000 [00:23<00:04, 3733986.83tokens/s]
Shard 15:  83%|████████▎ | 83030252/100000000 [00:23<00:04, 3784950.43tokens/s]
Shard 15:  83%|████████▎ | 83413560/100000000 [00:23<00:04, 3393455.22tokens/s]
Shard 15:  84%|████████▍ | 83916411/100000000 [00:23<00:04, 3824413.73tokens/s]
Shard 15:  84%|████████▍ | 84310612/100000000 [00:23<00:04, 3798831.47tokens/s]
Shard 15:  85%|████████▍ | 84699017/100000000 [00:23<00:04, 3719987.37tokens/s]
Shard 15:  85%|████████▌ | 85081566/100000000 [00:23<00:04, 3506566.42tokens/s]
Shard 15:  85%|████████▌ | 85438265/100000000 [00:23<00:04, 3411575.35tokens/s]
Shard 15:  86%|████████▌ | 85850587/100000000 [00:24<00:03, 3602135.30tokens/s]
Shard 15:  86%|████████▌ | 86216326/100000000 [00:24<00:04, 3341538.94tokens/s]
Shard 15:  87%|████████▋ | 86556444/100000000 [00:24<00:04, 3211449.22tokens/s]
Shard 15:  87%|████████▋ | 86882473/100000000 [00:24<00:04, 2895144.41tokens/s]
Shard 15:  87%|████████▋ | 87179117/100000000 [00:24<00:04, 2832537.88tokens/s]
Shard 15:  87%|████████▋ | 87496043/100000000 [00:24<00:04, 2919041.80tokens/s]
Shard 15:  88%|████████▊ | 87797071/100000000 [00:24<00:04, 2942087.11tokens/s]
Shard 15:  88%|████████▊ | 88094758/100000000 [00:24<00:04, 2590260.49tokens/s]
Shard 15:  88%|████████▊ | 88481257/100000000 [00:24<00:03, 2919498.16tokens/s]
Shard 15:  89%|████████▉ | 88824039/100000000 [00:25<00:03, 3056498.05tokens/s]
Shard 15:  89%|████████▉ | 89250743/100000000 [00:25<00:03, 3385084.14tokens/s]
Shard 15:  90%|████████▉ | 89618398/100000000 [00:25<00:02, 3463757.16tokens/s]
Shard 15:  90%|████████▉ | 89973533/100000000 [00:25<00:03, 3221273.29tokens/s]
Shard 15:  90%|█████████ | 90443574/100000000 [00:25<00:02, 3605474.98tokens/s]
Shard 15:  91%|█████████ | 90815682/100000000 [00:25<00:02, 3343132.20tokens/s]
Shard 15:  91%|█████████▏| 91261019/100000000 [00:25<00:02, 3633045.13tokens/s]
Shard 15:  92%|█████████▏| 91633530/100000000 [00:25<00:02, 3402225.13tokens/s]
Shard 15:  92%|█████████▏| 91982122/100000000 [00:25<00:02, 3324824.05tokens/s]
Shard 15:  92%|█████████▏| 92343877/100000000 [00:26<00:02, 3335655.67tokens/s]
Shard 15:  93%|█████████▎| 92682550/100000000 [00:26<00:02, 3314511.43tokens/s]
Shard 15:  93%|█████████▎| 93016891/100000000 [00:26<00:02, 3305083.02tokens/s]
Shard 15:  93%|█████████▎| 93427007/100000000 [00:26<00:01, 3518909.74tokens/s]
Shard 15:  94%|█████████▍| 93781434/100000000 [00:26<00:01, 3469063.80tokens/s]
Shard 15:  94%|█████████▍| 94132839/100000000 [00:26<00:01, 3430411.66tokens/s]
Shard 15:  94%|█████████▍| 94477971/100000000 [00:26<00:01, 3353804.08tokens/s]
Shard 15:  95%|█████████▍| 94990055/100000000 [00:26<00:01, 3857789.18tokens/s]
Shard 15:  95%|█████████▌| 95378677/100000000 [00:26<00:01, 3635840.12tokens/s]
Shard 15:  96%|█████████▌| 95747330/100000000 [00:27<00:01, 3257636.62tokens/s]
Shard 15:  96%|█████████▌| 96082425/100000000 [00:27<00:01, 2861340.83tokens/s]
Shard 15:  96%|█████████▋| 96381332/100000000 [00:27<00:01, 2869034.72tokens/s]
Shard 15:  97%|█████████▋| 96682529/100000000 [00:27<00:01, 2763596.78tokens/s]
Shard 15:  97%|█████████▋| 96965851/100000000 [00:27<00:01, 2777433.20tokens/s]
Shard 15:  97%|█████████▋| 97278090/100000000 [00:27<00:00, 2865445.66tokens/s]
Shard 15:  98%|█████████▊| 97572455/100000000 [00:27<00:00, 2832793.70tokens/s]
Shard 15:  98%|█████████▊| 97863042/100000000 [00:27<00:00, 2833442.22tokens/s]
Shard 15:  98%|█████████▊| 98151701/100000000 [00:27<00:00, 2747783.64tokens/s]
Shard 15:  98%|█████████▊| 98428184/100000000 [00:28<00:00, 2732733.16tokens/s]
Shard 15:  99%|█████████▊| 98702817/100000000 [00:28<00:00, 2688057.84tokens/s]
Shard 15:  99%|█████████▉| 99064231/100000000 [00:28<00:00, 2952027.86tokens/s]
Shard 15:  99%|█████████▉| 99361614/100000000 [00:28<00:00, 2915124.64tokens/s]
Shard 15: 100%|█████████▉| 99656114/100000000 [00:28<00:00, 2907161.47tokens/s]
Shard 15: 100%|█████████▉| 99977243/100000000 [00:28<00:00, 2993441.14tokens/s]
Shard 15: 100%|█████████▉| 99996207/100000000 [00:37<00:00, 2675123.32tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000015.bin (200,001,024 bytes) in the gpt-2 format

Shard 16:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 16:   0%|          | 207644/100000000 [00:00<00:50, 1979323.33tokens/s]
Shard 16:   0%|          | 406337/100000000 [00:00<00:53, 1856218.53tokens/s]
Shard 16:   1%|          | 593122/100000000 [00:00<00:57, 1719862.56tokens/s]
Shard 16:   1%|          | 824979/100000000 [00:00<00:51, 1936591.08tokens/s]
Shard 16:   1%|          | 1021667/100000000 [00:00<00:53, 1841865.01tokens/s]
Shard 16:   1%|          | 1210375/100000000 [00:00<01:02, 1580299.52tokens/s]
Shard 16:   1%|▏         | 1458746/100000000 [00:00<00:54, 1815284.77tokens/s]
Shard 16:   2%|▏         | 1674992/100000000 [00:00<00:51, 1900706.43tokens/s]
Shard 16:   2%|▏         | 1871394/100000000 [00:01<00:52, 1858297.04tokens/s]
Shard 16:   2%|▏         | 2084239/100000000 [00:01<00:54, 1805243.10tokens/s]
Shard 16:   2%|▏         | 2359867/100000000 [00:01<00:47, 2058280.74tokens/s]
Shard 16:   3%|▎         | 2586227/100000000 [00:01<00:46, 2108878.17tokens/s]
Shard 16:   3%|▎         | 2829830/100000000 [00:01<00:44, 2197328.04tokens/s]
Shard 16:   3%|▎         | 3141246/100000000 [00:01<00:39, 2462551.68tokens/s]
Shard 16:   3%|▎         | 3390945/100000000 [00:01<00:40, 2376506.61tokens/s]
Shard 16:   4%|▎         | 3632880/100000000 [00:01<00:40, 2377795.74tokens/s]
Shard 16:   4%|▍         | 4051838/100000000 [00:01<00:33, 2889762.92tokens/s]
Shard 16:   4%|▍         | 4344227/100000000 [00:02<00:37, 2524376.04tokens/s]
Shard 16:   5%|▍         | 4608027/100000000 [00:02<00:39, 2429987.90tokens/s]
Shard 16:   5%|▍         | 4884124/100000000 [00:02<00:37, 2516597.43tokens/s]
Shard 16:   5%|▌         | 5171388/100000000 [00:02<00:36, 2609660.64tokens/s]
Shard 16:   5%|▌         | 5496226/100000000 [00:02<00:34, 2757171.18tokens/s]
Shard 16:   6%|▌         | 5776081/100000000 [00:02<00:34, 2701074.43tokens/s]
Shard 16:   6%|▌         | 6145820/100000000 [00:02<00:31, 2977982.98tokens/s]
Shard 16:   6%|▋         | 6448193/100000000 [00:02<00:31, 2949489.91tokens/s]
Shard 16:   7%|▋         | 6765189/100000000 [00:02<00:31, 2994015.17tokens/s]
Shard 16:   7%|▋         | 7068310/100000000 [00:02<00:32, 2850346.35tokens/s]
Shard 16:   7%|▋         | 7367303/100000000 [00:03<00:32, 2861097.61tokens/s]
Shard 16:   8%|▊         | 8439120/100000000 [00:03<00:17, 5096366.48tokens/s]
Shard 16:   9%|▉         | 8961060/100000000 [00:03<00:19, 4688372.57tokens/s]
Shard 16:   9%|▉         | 9444371/100000000 [00:03<00:23, 3829179.54tokens/s]
Shard 16:  10%|▉         | 9936355/100000000 [00:03<00:22, 4085277.04tokens/s]
Shard 16:  10%|█         | 10452866/100000000 [00:03<00:20, 4342217.25tokens/s]
Shard 16:  11%|█         | 10912424/100000000 [00:03<00:26, 3343596.63tokens/s]
Shard 16:  11%|█▏        | 11296705/100000000 [00:04<00:31, 2811486.80tokens/s]
Shard 16:  12%|█▏        | 11622565/100000000 [00:04<00:33, 2672899.07tokens/s]
Shard 16:  12%|█▏        | 11921203/100000000 [00:04<00:36, 2417555.02tokens/s]
Shard 16:  12%|█▏        | 12219439/100000000 [00:04<00:36, 2409625.20tokens/s]
Shard 16:  12%|█▏        | 12475250/100000000 [00:04<00:37, 2338668.71tokens/s]
Shard 16:  13%|█▎        | 12719719/100000000 [00:04<00:37, 2310443.88tokens/s]
Shard 16:  13%|█▎        | 13000405/100000000 [00:04<00:35, 2431753.38tokens/s]
Shard 16:  13%|█▎        | 13274726/100000000 [00:04<00:34, 2507529.12tokens/s]
Shard 16:  14%|█▎        | 13531917/100000000 [00:05<00:38, 2232730.46tokens/s]
Shard 16:  14%|█▍        | 13833779/100000000 [00:05<00:35, 2429836.42tokens/s]
Shard 16:  14%|█▍        | 14133099/100000000 [00:05<00:33, 2577463.69tokens/s]
Shard 16:  14%|█▍        | 14399221/100000000 [00:05<00:39, 2171224.15tokens/s]
Shard 16:  15%|█▍        | 14632327/100000000 [00:05<00:39, 2165822.93tokens/s]
Shard 16:  15%|█▍        | 14860028/100000000 [00:05<00:40, 2086911.97tokens/s]
Shard 16:  15%|█▌        | 15076254/100000000 [00:05<00:43, 1962620.10tokens/s]
Shard 16:  15%|█▌        | 15318418/100000000 [00:05<00:40, 2073958.12tokens/s]
Shard 16:  16%|█▌        | 15611580/100000000 [00:06<00:36, 2302621.55tokens/s]
Shard 16:  16%|█▌        | 15935120/100000000 [00:06<00:32, 2552931.57tokens/s]
Shard 16:  16%|█▌        | 16197733/100000000 [00:06<00:35, 2372671.69tokens/s]
Shard 16:  16%|█▋        | 16442591/100000000 [00:06<00:36, 2311574.94tokens/s]
Shard 16:  17%|█▋        | 16678566/100000000 [00:06<00:38, 2183882.13tokens/s]
Shard 16:  17%|█▋        | 16927430/100000000 [00:06<00:36, 2259358.85tokens/s]
Shard 16:  17%|█▋        | 17218581/100000000 [00:06<00:33, 2438577.75tokens/s]
Shard 16:  17%|█▋        | 17466675/100000000 [00:06<00:35, 2309242.59tokens/s]
Shard 16:  18%|█▊        | 17780814/100000000 [00:06<00:32, 2532845.77tokens/s]
Shard 16:  18%|█▊        | 18044379/100000000 [00:07<00:34, 2403699.57tokens/s]
Shard 16:  18%|█▊        | 18289332/100000000 [00:07<00:36, 2212220.42tokens/s]
Shard 16:  19%|█▊        | 18529157/100000000 [00:07<00:36, 2246262.62tokens/s]
Shard 16:  19%|█▉        | 18760841/100000000 [00:07<00:37, 2163350.42tokens/s]
Shard 16:  19%|█▉        | 18988206/100000000 [00:07<00:37, 2170071.03tokens/s]
Shard 16:  19%|█▉        | 19403421/100000000 [00:07<00:29, 2704564.56tokens/s]
Shard 16:  20%|█▉        | 19958237/100000000 [00:07<00:22, 3505352.23tokens/s]
Shard 16:  20%|██        | 20317424/100000000 [00:07<00:23, 3440871.46tokens/s]
Shard 16:  21%|██        | 20667570/100000000 [00:08<00:28, 2830018.95tokens/s]
Shard 16:  21%|██        | 20974369/100000000 [00:08<00:30, 2632960.58tokens/s]
Shard 16:  21%|██▏       | 21254525/100000000 [00:08<00:30, 2576321.63tokens/s]
Shard 16:  22%|██▏       | 21573923/100000000 [00:08<00:28, 2732212.84tokens/s]
Shard 16:  22%|██▏       | 21900309/100000000 [00:08<00:27, 2841044.01tokens/s]
Shard 16:  22%|██▏       | 22408392/100000000 [00:08<00:22, 3436003.37tokens/s]
Shard 16:  23%|██▎       | 22763647/100000000 [00:08<00:23, 3281166.43tokens/s]
Shard 16:  23%|██▎       | 23177627/100000000 [00:08<00:21, 3513096.09tokens/s]
Shard 16:  24%|██▎       | 23554735/100000000 [00:08<00:21, 3530517.83tokens/s]
Shard 16:  24%|██▍       | 23990506/100000000 [00:09<00:20, 3764192.30tokens/s]
Shard 16:  24%|██▍       | 24372662/100000000 [00:09<00:21, 3485246.31tokens/s]
Shard 16:  25%|██▍       | 24728787/100000000 [00:09<00:21, 3434442.51tokens/s]
Shard 16:  25%|██▌       | 25083575/100000000 [00:09<00:21, 3462895.63tokens/s]
Shard 16:  25%|██▌       | 25482579/100000000 [00:09<00:20, 3572303.68tokens/s]
Shard 16:  26%|██▌       | 25991592/100000000 [00:09<00:18, 3994858.37tokens/s]
Shard 16:  26%|██▋       | 26394991/100000000 [00:09<00:20, 3529174.43tokens/s]
Shard 16:  27%|██▋       | 26797241/100000000 [00:09<00:20, 3656476.95tokens/s]
Shard 16:  27%|██▋       | 27173768/100000000 [00:09<00:22, 3254757.60tokens/s]
Shard 16:  28%|██▊       | 27560911/100000000 [00:10<00:21, 3407630.39tokens/s]
Shard 16:  28%|██▊       | 27923292/100000000 [00:10<00:22, 3150057.97tokens/s]
Shard 16:  28%|██▊       | 28249390/100000000 [00:10<00:23, 3103891.55tokens/s]
Shard 16:  29%|██▊       | 28567455/100000000 [00:10<00:27, 2619288.89tokens/s]
Shard 16:  29%|██▉       | 28903379/100000000 [00:10<00:25, 2785721.73tokens/s]
Shard 16:  29%|██▉       | 29289327/100000000 [00:10<00:23, 3042002.87tokens/s]
Shard 16:  30%|██▉       | 29608318/100000000 [00:10<00:24, 2916097.75tokens/s]
Shard 16:  30%|██▉       | 29910542/100000000 [00:10<00:26, 2604539.69tokens/s]
Shard 16:  30%|███       | 30219638/100000000 [00:11<00:25, 2712430.88tokens/s]
Shard 16:  31%|███       | 30501676/100000000 [00:11<00:25, 2683274.51tokens/s]
Shard 16:  31%|███       | 30841136/100000000 [00:11<00:24, 2868265.03tokens/s]
Shard 16:  31%|███       | 31137677/100000000 [00:11<00:23, 2875464.80tokens/s]
Shard 16:  31%|███▏      | 31430473/100000000 [00:11<00:26, 2573147.43tokens/s]
Shard 16:  32%|███▏      | 31764622/100000000 [00:11<00:24, 2750452.44tokens/s]
Shard 16:  32%|███▏      | 32047582/100000000 [00:11<00:24, 2728373.84tokens/s]
Shard 16:  32%|███▏      | 32335751/100000000 [00:11<00:24, 2764810.65tokens/s]
Shard 16:  33%|███▎      | 32702828/100000000 [00:11<00:22, 2991242.30tokens/s]
Shard 16:  33%|███▎      | 33027004/100000000 [00:12<00:22, 3039073.25tokens/s]
Shard 16:  33%|███▎      | 33334247/100000000 [00:12<00:22, 2975421.05tokens/s]
Shard 16:  34%|███▎      | 33638341/100000000 [00:12<00:24, 2727756.63tokens/s]
Shard 16:  34%|███▍      | 33921879/100000000 [00:12<00:26, 2511145.14tokens/s]
Shard 16:  34%|███▍      | 34183032/100000000 [00:12<00:26, 2529492.82tokens/s]
Shard 16:  34%|███▍      | 34443632/100000000 [00:12<00:27, 2422279.78tokens/s]
Shard 16:  35%|███▍      | 34689323/100000000 [00:12<00:26, 2420598.70tokens/s]
Shard 16:  35%|███▍      | 34965516/100000000 [00:12<00:25, 2506577.11tokens/s]
Shard 16:  35%|███▌      | 35348134/100000000 [00:12<00:22, 2870896.22tokens/s]
Shard 16:  36%|███▌      | 35650659/100000000 [00:13<00:22, 2912693.44tokens/s]
Shard 16:  36%|███▌      | 36096711/100000000 [00:13<00:19, 3352255.18tokens/s]
Shard 16:  36%|███▋      | 36435621/100000000 [00:13<00:20, 3117997.02tokens/s]
Shard 16:  37%|███▋      | 36753101/100000000 [00:13<00:21, 2980462.96tokens/s]
Shard 16:  37%|███▋      | 37123358/100000000 [00:13<00:19, 3177563.36tokens/s]
Shard 16:  37%|███▋      | 37478492/100000000 [00:13<00:19, 3258039.89tokens/s]
Shard 16:  38%|███▊      | 37811624/100000000 [00:13<00:20, 3101287.25tokens/s]
Shard 16:  38%|███▊      | 38125832/100000000 [00:13<00:23, 2645424.26tokens/s]
Shard 16:  38%|███▊      | 38403845/100000000 [00:13<00:25, 2458266.64tokens/s]
Shard 16:  39%|███▊      | 38659932/100000000 [00:14<00:25, 2440896.32tokens/s]
Shard 16:  39%|███▉      | 38910798/100000000 [00:14<00:24, 2452026.08tokens/s]
Shard 16:  39%|███▉      | 39278844/100000000 [00:14<00:21, 2766155.00tokens/s]
Shard 16:  40%|███▉      | 39707866/100000000 [00:14<00:18, 3178104.88tokens/s]
Shard 16:  40%|████      | 40035043/100000000 [00:14<00:19, 3068611.33tokens/s]
Shard 16:  40%|████      | 40348046/100000000 [00:14<00:19, 2982901.97tokens/s]
Shard 16:  41%|████      | 40713027/100000000 [00:14<00:18, 3168168.68tokens/s]
Shard 16:  41%|████      | 41034741/100000000 [00:14<00:18, 3168477.97tokens/s]
Shard 16:  41%|████▏     | 41356401/100000000 [00:14<00:21, 2719834.93tokens/s]
Shard 16:  42%|████▏     | 41644985/100000000 [00:15<00:21, 2745141.66tokens/s]
Shard 16:  42%|████▏     | 41928839/100000000 [00:15<00:22, 2542557.37tokens/s]
Shard 16:  42%|████▏     | 42231293/100000000 [00:15<00:21, 2668411.59tokens/s]
Shard 16:  43%|████▎     | 42506421/100000000 [00:15<00:25, 2253888.88tokens/s]
Shard 16:  43%|████▎     | 43013013/100000000 [00:15<00:19, 2930933.52tokens/s]
Shard 16:  43%|████▎     | 43330755/100000000 [00:15<00:20, 2741923.62tokens/s]
Shard 16:  44%|████▎     | 43623624/100000000 [00:15<00:20, 2735501.11tokens/s]
Shard 16:  44%|████▍     | 43910482/100000000 [00:15<00:22, 2537182.51tokens/s]
Shard 16:  44%|████▍     | 44177549/100000000 [00:16<00:23, 2411559.03tokens/s]
Shard 16:  44%|████▍     | 44426710/100000000 [00:16<00:23, 2396259.39tokens/s]
Shard 16:  45%|████▍     | 44840735/100000000 [00:16<00:19, 2852736.64tokens/s]
Shard 16:  45%|████▌     | 45165967/100000000 [00:16<00:18, 2957096.41tokens/s]
Shard 16:  46%|████▌     | 45563201/100000000 [00:16<00:16, 3215427.47tokens/s]
Shard 16:  46%|████▌     | 45891828/100000000 [00:16<00:17, 3163396.00tokens/s]
Shard 16:  46%|████▌     | 46213242/100000000 [00:16<00:18, 2916889.10tokens/s]
Shard 16:  47%|████▋     | 46523920/100000000 [00:16<00:18, 2961719.94tokens/s]
Shard 16:  47%|████▋     | 46887800/100000000 [00:16<00:16, 3139970.23tokens/s]
Shard 16:  47%|████▋     | 47207028/100000000 [00:17<00:18, 2818081.18tokens/s]
Shard 16:  48%|████▊     | 47543215/100000000 [00:17<00:17, 2951293.70tokens/s]
Shard 16:  48%|████▊     | 47846607/100000000 [00:17<00:19, 2630489.78tokens/s]
Shard 16:  48%|████▊     | 48125257/100000000 [00:17<00:21, 2445274.41tokens/s]
Shard 16:  48%|████▊     | 48439206/100000000 [00:17<00:19, 2617994.91tokens/s]
Shard 16:  49%|████▉     | 48754741/100000000 [00:17<00:18, 2737419.02tokens/s]
Shard 16:  49%|████▉     | 49090994/100000000 [00:17<00:17, 2907107.30tokens/s]
Shard 16:  49%|████▉     | 49449042/100000000 [00:17<00:16, 3069577.02tokens/s]
Shard 16:  50%|████▉     | 49762793/100000000 [00:18<00:17, 2950611.97tokens/s]
Shard 16:  50%|█████     | 50063787/100000000 [00:18<00:16, 2966515.30tokens/s]
Shard 16:  50%|█████     | 50371246/100000000 [00:18<00:16, 2984009.74tokens/s]
Shard 16:  51%|█████     | 50695154/100000000 [00:18<00:16, 3049951.46tokens/s]
Shard 16:  51%|█████     | 51095846/100000000 [00:18<00:14, 3314717.31tokens/s]
Shard 16:  51%|█████▏    | 51452747/100000000 [00:18<00:14, 3358667.74tokens/s]
Shard 16:  52%|█████▏    | 51791854/100000000 [00:18<00:14, 3273756.57tokens/s]
Shard 16:  52%|█████▏    | 52149975/100000000 [00:18<00:14, 3349528.10tokens/s]
Shard 16:  53%|█████▎    | 52535615/100000000 [00:18<00:13, 3496754.38tokens/s]
Shard 16:  53%|█████▎    | 52887025/100000000 [00:18<00:13, 3457073.93tokens/s]
Shard 16:  53%|█████▎    | 53233939/100000000 [00:19<00:16, 2913122.79tokens/s]
Shard 16:  54%|█████▎    | 53610789/100000000 [00:19<00:14, 3129110.29tokens/s]
Shard 16:  54%|█████▍    | 54144628/100000000 [00:19<00:12, 3700707.74tokens/s]
Shard 16:  55%|█████▍    | 54529668/100000000 [00:19<00:13, 3256818.18tokens/s]
Shard 16:  55%|█████▌    | 55095580/100000000 [00:19<00:11, 3833412.19tokens/s]
Shard 16:  56%|█████▌    | 55500402/100000000 [00:19<00:11, 3827437.22tokens/s]
Shard 16:  56%|█████▌    | 55897995/100000000 [00:19<00:12, 3485443.20tokens/s]
Shard 16:  56%|█████▋    | 56261416/100000000 [00:19<00:13, 3307909.83tokens/s]
Shard 16:  57%|█████▋    | 56603053/100000000 [00:20<00:15, 2884291.24tokens/s]
Shard 16:  57%|█████▋    | 56906587/100000000 [00:20<00:15, 2826067.70tokens/s]
Shard 16:  57%|█████▋    | 57233189/100000000 [00:20<00:14, 2906091.45tokens/s]
Shard 16:  58%|█████▊    | 57532345/100000000 [00:20<00:15, 2714836.32tokens/s]
Shard 16:  58%|█████▊    | 57918945/100000000 [00:20<00:14, 3004392.31tokens/s]
Shard 16:  58%|█████▊    | 58275365/100000000 [00:20<00:13, 3148145.21tokens/s]
Shard 16:  59%|█████▊    | 58643576/100000000 [00:20<00:12, 3278273.95tokens/s]
Shard 16:  59%|█████▉    | 58978511/100000000 [00:20<00:13, 3053377.74tokens/s]
Shard 16:  59%|█████▉    | 59492121/100000000 [00:20<00:11, 3608182.17tokens/s]
Shard 16:  60%|█████▉    | 59953313/100000000 [00:21<00:10, 3888087.47tokens/s]
Shard 16:  60%|██████    | 60351564/100000000 [00:21<00:11, 3590044.16tokens/s]
Shard 16:  61%|██████    | 60722218/100000000 [00:21<00:11, 3417439.00tokens/s]
Shard 16:  61%|██████    | 61072112/100000000 [00:21<00:11, 3394818.36tokens/s]
Shard 16:  61%|██████▏   | 61417643/100000000 [00:21<00:11, 3388199.01tokens/s]
Shard 16:  62%|██████▏   | 61761550/100000000 [00:21<00:13, 2921232.91tokens/s]
Shard 16:  62%|██████▏   | 62125891/100000000 [00:21<00:12, 3104026.40tokens/s]
Shard 16:  62%|██████▏   | 62448396/100000000 [00:21<00:12, 3093472.20tokens/s]
Shard 16:  63%|██████▎   | 62769145/100000000 [00:22<00:12, 3033345.32tokens/s]
Shard 16:  63%|██████▎   | 63185919/100000000 [00:22<00:11, 3337883.15tokens/s]
Shard 16:  64%|██████▎   | 63693856/100000000 [00:22<00:09, 3803983.32tokens/s]
Shard 16:  64%|██████▍   | 64082713/100000000 [00:22<00:10, 3380784.57tokens/s]
Shard 16:  65%|██████▍   | 64538419/100000000 [00:22<00:09, 3692231.39tokens/s]
Shard 16:  65%|██████▌   | 65047734/100000000 [00:22<00:08, 4074285.96tokens/s]
Shard 16:  65%|██████▌   | 65471257/100000000 [00:22<00:08, 4119349.00tokens/s]
Shard 16:  66%|██████▌   | 65892876/100000000 [00:22<00:09, 3600510.80tokens/s]
Shard 16:  66%|██████▋   | 66439475/100000000 [00:22<00:08, 4055404.83tokens/s]
Shard 16:  67%|██████▋   | 66862893/100000000 [00:23<00:08, 3963872.55tokens/s]
Shard 16:  67%|██████▋   | 67271774/100000000 [00:23<00:09, 3304375.60tokens/s]
Shard 16:  68%|██████▊   | 67627017/100000000 [00:23<00:10, 3137748.54tokens/s]
Shard 16:  68%|██████▊   | 67961736/100000000 [00:23<00:11, 2816658.21tokens/s]
Shard 16:  68%|██████▊   | 68289538/100000000 [00:23<00:10, 2924011.73tokens/s]
Shard 16:  69%|██████▊   | 68662932/100000000 [00:23<00:10, 3128078.81tokens/s]
Shard 16:  69%|██████▉   | 68989401/100000000 [00:23<00:10, 3021739.35tokens/s]
Shard 16:  69%|██████▉   | 69422073/100000000 [00:23<00:09, 3362357.58tokens/s]
Shard 16:  70%|██████▉   | 69881961/100000000 [00:24<00:08, 3700178.86tokens/s]
Shard 16:  70%|███████   | 70262632/100000000 [00:24<00:08, 3647234.39tokens/s]
Shard 16:  71%|███████   | 70647713/100000000 [00:24<00:07, 3691615.42tokens/s]
Shard 16:  71%|███████   | 71089025/100000000 [00:24<00:07, 3885861.04tokens/s]
Shard 16:  71%|███████▏  | 71487634/100000000 [00:24<00:07, 3886897.90tokens/s]
Shard 16:  72%|███████▏  | 72050792/100000000 [00:24<00:06, 4386209.60tokens/s]
Shard 16:  73%|███████▎  | 72570523/100000000 [00:24<00:05, 4624129.89tokens/s]
Shard 16:  73%|███████▎  | 73052581/100000000 [00:24<00:05, 4644829.97tokens/s]
Shard 16:  74%|███████▍  | 73817430/100000000 [00:24<00:04, 5517961.38tokens/s]
Shard 16:  74%|███████▍  | 74372911/100000000 [00:24<00:05, 4692595.34tokens/s]
Shard 16:  75%|███████▍  | 74865709/100000000 [00:25<00:05, 4607350.25tokens/s]
Shard 16:  75%|███████▌  | 75342170/100000000 [00:25<00:05, 4183500.85tokens/s]
Shard 16:  76%|███████▌  | 75782097/100000000 [00:25<00:05, 4095729.01tokens/s]
Shard 16:  76%|███████▌  | 76202493/100000000 [00:25<00:05, 4000428.58tokens/s]
Shard 16:  77%|███████▋  | 76609967/100000000 [00:25<00:06, 3831507.37tokens/s]
Shard 16:  77%|███████▋  | 77147608/100000000 [00:25<00:05, 4239205.15tokens/s]
Shard 16:  78%|███████▊  | 77666424/100000000 [00:25<00:04, 4477212.48tokens/s]
Shard 16:  78%|███████▊  | 78123090/100000000 [00:25<00:04, 4418993.25tokens/s]
Shard 16:  79%|███████▉  | 78853809/100000000 [00:25<00:04, 5232580.08tokens/s]
Shard 16:  79%|███████▉  | 79385799/100000000 [00:26<00:04, 4827596.51tokens/s]
Shard 16:  80%|███████▉  | 79880032/100000000 [00:26<00:04, 4747841.16tokens/s]
Shard 16:  80%|████████  | 80381687/100000000 [00:26<00:04, 4283928.42tokens/s]
Shard 16:  81%|████████▏ | 81309164/100000000 [00:26<00:03, 5573290.43tokens/s]
Shard 16:  82%|████████▏ | 81897369/100000000 [00:26<00:04, 4345370.03tokens/s]
Shard 16:  83%|████████▎ | 82632979/100000000 [00:26<00:03, 5018776.83tokens/s]
Shard 16:  83%|████████▎ | 83196448/100000000 [00:26<00:03, 4508941.27tokens/s]
Shard 16:  84%|████████▎ | 83696995/100000000 [00:27<00:03, 4498290.10tokens/s]
Shard 16:  84%|████████▍ | 84181058/100000000 [00:27<00:03, 4224793.80tokens/s]
Shard 16:  85%|████████▍ | 84628252/100000000 [00:27<00:03, 4045060.35tokens/s]
Shard 16:  85%|████████▌ | 85050483/100000000 [00:27<00:03, 3875655.60tokens/s]
Shard 16:  85%|████████▌ | 85448679/100000000 [00:27<00:04, 3414036.88tokens/s]
Shard 16:  86%|████████▌ | 85810319/100000000 [00:27<00:04, 3304846.16tokens/s]
Shard 16:  86%|████████▌ | 86149363/100000000 [00:27<00:04, 3274289.51tokens/s]
Shard 16:  87%|████████▋ | 86526652/100000000 [00:27<00:03, 3397042.47tokens/s]
Shard 16:  87%|████████▋ | 86872584/100000000 [00:28<00:03, 3391811.02tokens/s]
Shard 16:  87%|████████▋ | 87215906/100000000 [00:28<00:04, 3120631.59tokens/s]
Shard 16:  88%|████████▊ | 87534327/100000000 [00:28<00:04, 2760785.88tokens/s]
Shard 16:  88%|████████▊ | 87819840/100000000 [00:28<00:04, 2592469.97tokens/s]
Shard 16:  88%|████████▊ | 88131956/100000000 [00:28<00:04, 2719616.09tokens/s]
Shard 16:  88%|████████▊ | 88481623/100000000 [00:28<00:03, 2904843.93tokens/s]
Shard 16:  89%|████████▉ | 88779684/100000000 [00:28<00:04, 2636911.52tokens/s]
Shard 16:  89%|████████▉ | 89098976/100000000 [00:28<00:03, 2778131.13tokens/s]
Shard 16:  89%|████████▉ | 89385108/100000000 [00:29<00:04, 2628675.33tokens/s]
Shard 16:  90%|████████▉ | 89765745/100000000 [00:29<00:03, 2941607.54tokens/s]
Shard 16:  90%|█████████ | 90286241/100000000 [00:29<00:02, 3555110.90tokens/s]
Shard 16:  91%|█████████ | 90678059/100000000 [00:29<00:02, 3653188.98tokens/s]
Shard 16:  91%|█████████ | 91051873/100000000 [00:29<00:02, 3074419.64tokens/s]
Shard 16:  91%|█████████▏| 91380457/100000000 [00:29<00:02, 3031496.02tokens/s]
Shard 16:  92%|█████████▏| 91722920/100000000 [00:29<00:02, 3131068.89tokens/s]
Shard 16:  92%|█████████▏| 92047780/100000000 [00:29<00:02, 3154485.08tokens/s]
Shard 16:  92%|█████████▏| 92489392/100000000 [00:29<00:02, 3490271.28tokens/s]
Shard 16:  93%|█████████▎| 92847233/100000000 [00:30<00:02, 3442609.59tokens/s]
Shard 16:  93%|█████████▎| 93274458/100000000 [00:30<00:01, 3675020.85tokens/s]
Shard 16:  94%|█████████▎| 93648111/100000000 [00:30<00:01, 3363611.89tokens/s]
Shard 16:  94%|█████████▍| 94076761/100000000 [00:30<00:01, 3612403.05tokens/s]
Shard 16:  95%|█████████▍| 94571750/100000000 [00:30<00:01, 3978420.82tokens/s]
Shard 16:  95%|█████████▍| 94977909/100000000 [00:30<00:01, 3782251.36tokens/s]
Shard 16:  95%|█████████▌| 95476781/100000000 [00:30<00:01, 4116857.14tokens/s]
Shard 16:  96%|█████████▌| 96018406/100000000 [00:30<00:00, 4486290.50tokens/s]
Shard 16:  96%|█████████▋| 96475533/100000000 [00:30<00:00, 4324826.69tokens/s]
Shard 16:  97%|█████████▋| 96914133/100000000 [00:30<00:00, 4091033.88tokens/s]
Shard 16:  97%|█████████▋| 97329805/100000000 [00:31<00:00, 3455139.46tokens/s]
Shard 16:  98%|█████████▊| 97696377/100000000 [00:31<00:00, 3390776.59tokens/s]
Shard 16:  98%|█████████▊| 98163926/100000000 [00:31<00:00, 3718123.08tokens/s]
Shard 16:  99%|█████████▉| 98766872/100000000 [00:31<00:00, 4329131.16tokens/s]
Shard 16:  99%|█████████▉| 99218648/100000000 [00:31<00:00, 4311206.79tokens/s]
Shard 16: 100%|█████████▉| 99895881/100000000 [00:31<00:00, 4999556.29tokens/s]
Shard 16: 100%|█████████▉| 99998744/100000000 [00:39<00:00, 2509470.47tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000016.bin (200,001,024 bytes) in the gpt-2 format

Shard 17:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 17:   0%|          | 354817/100000000 [00:00<00:28, 3475835.76tokens/s]
Shard 17:   1%|          | 704912/100000000 [00:00<00:29, 3388263.67tokens/s]
Shard 17:   1%|          | 1048525/100000000 [00:00<00:31, 3146531.60tokens/s]
Shard 17:   1%|▏         | 1371439/100000000 [00:00<00:31, 3166108.42tokens/s]
Shard 17:   2%|▏         | 1963971/100000000 [00:00<00:23, 4104898.06tokens/s]
Shard 17:   2%|▏         | 2379334/100000000 [00:00<00:27, 3600661.76tokens/s]
Shard 17:   3%|▎         | 2886205/100000000 [00:00<00:24, 4014635.68tokens/s]
Shard 17:   3%|▎         | 3334170/100000000 [00:00<00:23, 4126241.10tokens/s]
Shard 17:   4%|▍         | 3822051/100000000 [00:00<00:22, 4334628.88tokens/s]
Shard 17:   4%|▍         | 4263943/100000000 [00:01<00:27, 3508096.02tokens/s]
Shard 17:   5%|▍         | 4644162/100000000 [00:01<00:29, 3252643.22tokens/s]
Shard 17:   5%|▍         | 4990994/100000000 [00:01<00:32, 2958944.90tokens/s]
Shard 17:   5%|▌         | 5305409/100000000 [00:01<00:34, 2772505.89tokens/s]
Shard 17:   6%|▌         | 5612268/100000000 [00:01<00:33, 2835038.85tokens/s]
Shard 17:   6%|▌         | 5905381/100000000 [00:01<00:33, 2790364.49tokens/s]
Shard 17:   6%|▌         | 6194065/100000000 [00:01<00:36, 2571969.65tokens/s]
Shard 17:   6%|▋         | 6477957/100000000 [00:02<00:35, 2612172.81tokens/s]
Shard 17:   7%|▋         | 6744043/100000000 [00:02<00:37, 2479702.71tokens/s]
Shard 17:   7%|▋         | 7006114/100000000 [00:02<00:37, 2499952.89tokens/s]
Shard 17:   7%|▋         | 7340285/100000000 [00:02<00:34, 2723144.45tokens/s]
Shard 17:   8%|▊         | 7617263/100000000 [00:02<00:35, 2582677.40tokens/s]
Shard 17:   8%|▊         | 7879390/100000000 [00:02<00:36, 2531019.80tokens/s]
Shard 17:   8%|▊         | 8159241/100000000 [00:02<00:35, 2596866.21tokens/s]
Shard 17:   9%|▊         | 8686245/100000000 [00:02<00:27, 3345203.88tokens/s]
Shard 17:   9%|▉         | 9026776/100000000 [00:02<00:27, 3317637.40tokens/s]
Shard 17:   9%|▉         | 9363343/100000000 [00:02<00:27, 3325280.15tokens/s]
Shard 17:  10%|▉         | 9740728/100000000 [00:03<00:26, 3449262.14tokens/s]
Shard 17:  10%|█         | 10088398/100000000 [00:03<00:26, 3367401.43tokens/s]
Shard 17:  10%|█         | 10427151/100000000 [00:03<00:28, 3135222.26tokens/s]
Shard 17:  11%|█         | 10845545/100000000 [00:03<00:26, 3403138.00tokens/s]
Shard 17:  11%|█▏        | 11251662/100000000 [00:03<00:24, 3575254.05tokens/s]
Shard 17:  12%|█▏        | 11613390/100000000 [00:03<00:26, 3388461.97tokens/s]
Shard 17:  12%|█▏        | 12081599/100000000 [00:03<00:23, 3749137.69tokens/s]
Shard 17:  12%|█▏        | 12462486/100000000 [00:03<00:25, 3470155.82tokens/s]
Shard 17:  13%|█▎        | 12891809/100000000 [00:03<00:23, 3678832.35tokens/s]
Shard 17:  13%|█▎        | 13315754/100000000 [00:04<00:22, 3822955.17tokens/s]
Shard 17:  14%|█▎        | 13710950/100000000 [00:04<00:23, 3720363.39tokens/s]
Shard 17:  14%|█▍        | 14088205/100000000 [00:04<00:23, 3629907.04tokens/s]
Shard 17:  15%|█▍        | 14513304/100000000 [00:04<00:22, 3791131.60tokens/s]
Shard 17:  15%|█▌        | 15097190/100000000 [00:04<00:19, 4370260.45tokens/s]
Shard 17:  16%|█▌        | 15539093/100000000 [00:04<00:20, 4169870.76tokens/s]
Shard 17:  16%|█▌        | 16027470/100000000 [00:04<00:21, 3926824.57tokens/s]
Shard 17:  17%|█▋        | 16534538/100000000 [00:04<00:19, 4205862.60tokens/s]
Shard 17:  17%|█▋        | 16964076/100000000 [00:05<00:21, 3862684.98tokens/s]
Shard 17:  17%|█▋        | 17372253/100000000 [00:05<00:21, 3881085.31tokens/s]
Shard 17:  18%|█▊        | 17767764/100000000 [00:05<00:22, 3724889.99tokens/s]
Shard 17:  18%|█▊        | 18145858/100000000 [00:05<00:22, 3570474.92tokens/s]
Shard 17:  19%|█▊        | 18543945/100000000 [00:05<00:22, 3675936.39tokens/s]
Shard 17:  19%|█▉        | 19202469/100000000 [00:05<00:18, 4463003.70tokens/s]
Shard 17:  20%|█▉        | 19656863/100000000 [00:05<00:19, 4136846.81tokens/s]
Shard 17:  20%|██        | 20080013/100000000 [00:05<00:20, 3936311.55tokens/s]
Shard 17:  21%|██        | 20541161/100000000 [00:05<00:19, 4112090.85tokens/s]
Shard 17:  21%|██        | 20959617/100000000 [00:06<00:20, 3869211.66tokens/s]
Shard 17:  21%|██▏       | 21353416/100000000 [00:06<00:20, 3783199.42tokens/s]
Shard 17:  22%|██▏       | 21843175/100000000 [00:06<00:19, 4044520.75tokens/s]
Shard 17:  22%|██▏       | 22252609/100000000 [00:06<00:22, 3418556.83tokens/s]
Shard 17:  23%|██▎       | 22613240/100000000 [00:06<00:23, 3233051.00tokens/s]
Shard 17:  23%|██▎       | 22999157/100000000 [00:06<00:22, 3385918.19tokens/s]
Shard 17:  23%|██▎       | 23423518/100000000 [00:06<00:21, 3565474.44tokens/s]
Shard 17:  24%|██▍       | 24049173/100000000 [00:06<00:17, 4266092.31tokens/s]
Shard 17:  25%|██▍       | 24549232/100000000 [00:06<00:16, 4439599.23tokens/s]
Shard 17:  25%|██▌       | 25003855/100000000 [00:07<00:18, 4018517.12tokens/s]
Shard 17:  25%|██▌       | 25419586/100000000 [00:07<00:18, 3996042.18tokens/s]
Shard 17:  26%|██▌       | 25980774/100000000 [00:07<00:16, 4432573.09tokens/s]
Shard 17:  27%|██▋       | 26697257/100000000 [00:07<00:14, 5143847.51tokens/s]
Shard 17:  27%|██▋       | 27222767/100000000 [00:07<00:15, 4595373.05tokens/s]
Shard 17:  28%|██▊       | 27722170/100000000 [00:07<00:15, 4697468.55tokens/s]
Shard 17:  28%|██▊       | 28301502/100000000 [00:07<00:14, 4990581.07tokens/s]
Shard 17:  29%|██▉       | 29104273/100000000 [00:07<00:12, 5834414.91tokens/s]
Shard 17:  30%|██▉       | 29703658/100000000 [00:07<00:12, 5569635.53tokens/s]
Shard 17:  30%|███       | 30272557/100000000 [00:08<00:12, 5454672.46tokens/s]
Shard 17:  31%|███       | 30826446/100000000 [00:08<00:13, 5130965.33tokens/s]
Shard 17:  31%|███▏      | 31403623/100000000 [00:08<00:12, 5300972.62tokens/s]
Shard 17:  32%|███▏      | 31991989/100000000 [00:08<00:12, 5449070.31tokens/s]
Shard 17:  33%|███▎      | 32543335/100000000 [00:08<00:12, 5281954.78tokens/s]
Shard 17:  33%|███▎      | 33320843/100000000 [00:08<00:11, 5983187.08tokens/s]
Shard 17:  34%|███▍      | 33927728/100000000 [00:08<00:10, 6006909.24tokens/s]
Shard 17:  35%|███▍      | 34534710/100000000 [00:08<00:11, 5753872.02tokens/s]
Shard 17:  35%|███▌      | 35115924/100000000 [00:08<00:11, 5458543.58tokens/s]
Shard 17:  36%|███▌      | 35668732/100000000 [00:09<00:12, 5217920.33tokens/s]
Shard 17:  36%|███▌      | 36200736/100000000 [00:09<00:12, 4933401.91tokens/s]
Shard 17:  37%|███▋      | 36699560/100000000 [00:09<00:15, 4071746.98tokens/s]
Shard 17:  37%|███▋      | 37131868/100000000 [00:09<00:15, 3998194.30tokens/s]
Shard 17:  38%|███▊      | 37548385/100000000 [00:09<00:16, 3835268.60tokens/s]
Shard 17:  38%|███▊      | 37943364/100000000 [00:09<00:16, 3805637.94tokens/s]
Shard 17:  38%|███▊      | 38373725/100000000 [00:09<00:15, 3924909.44tokens/s]
Shard 17:  39%|███▉      | 38773285/100000000 [00:09<00:17, 3596762.26tokens/s]
Shard 17:  39%|███▉      | 39141458/100000000 [00:10<00:17, 3459118.80tokens/s]
Shard 17:  40%|███▉      | 39502339/100000000 [00:10<00:17, 3494712.06tokens/s]
Shard 17:  40%|███▉      | 39856784/100000000 [00:10<00:17, 3376637.00tokens/s]
Shard 17:  40%|████      | 40257294/100000000 [00:10<00:16, 3544409.76tokens/s]
Shard 17:  41%|████      | 40827641/100000000 [00:10<00:14, 4135602.80tokens/s]
Shard 17:  41%|████▏     | 41420423/100000000 [00:10<00:12, 4645761.65tokens/s]
Shard 17:  42%|████▏     | 42023373/100000000 [00:10<00:11, 5039623.82tokens/s]
Shard 17:  43%|████▎     | 42534106/100000000 [00:10<00:12, 4672327.36tokens/s]
Shard 17:  43%|████▎     | 43036424/100000000 [00:10<00:12, 4705921.90tokens/s]
Shard 17:  44%|████▎     | 43686812/100000000 [00:11<00:10, 5212349.68tokens/s]
Shard 17:  44%|████▍     | 44221286/100000000 [00:11<00:10, 5095241.19tokens/s]
Shard 17:  45%|████▌     | 45072759/100000000 [00:11<00:09, 6057193.99tokens/s]
Shard 17:  46%|████▌     | 45689436/100000000 [00:11<00:09, 5816387.02tokens/s]
Shard 17:  46%|████▋     | 46279483/100000000 [00:11<00:09, 5688842.64tokens/s]
Shard 17:  47%|████▋     | 46885254/100000000 [00:11<00:09, 5784164.93tokens/s]
Shard 17:  47%|████▋     | 47468465/100000000 [00:11<00:10, 5155537.61tokens/s]
Shard 17:  48%|████▊     | 47998705/100000000 [00:11<00:11, 4391011.25tokens/s]
Shard 17:  49%|████▊     | 48540546/100000000 [00:11<00:11, 4596984.45tokens/s]
Shard 17:  49%|████▉     | 49022684/100000000 [00:12<00:11, 4462729.47tokens/s]
Shard 17:  50%|████▉     | 49554126/100000000 [00:12<00:10, 4634536.38tokens/s]
Shard 17:  50%|█████     | 50192122/100000000 [00:12<00:09, 5103080.70tokens/s]
Shard 17:  51%|█████     | 51000641/100000000 [00:12<00:08, 5928250.10tokens/s]
Shard 17:  52%|█████▏    | 51706899/100000000 [00:12<00:07, 6203951.28tokens/s]
Shard 17:  52%|█████▏    | 52399681/100000000 [00:12<00:07, 6373053.26tokens/s]
Shard 17:  53%|█████▎    | 53046812/100000000 [00:12<00:07, 6279872.40tokens/s]
Shard 17:  54%|█████▍    | 53758524/100000000 [00:12<00:07, 6434449.31tokens/s]
Shard 17:  54%|█████▍    | 54432123/100000000 [00:12<00:07, 5962014.23tokens/s]
Shard 17:  55%|█████▌    | 55038929/100000000 [00:13<00:07, 5962757.92tokens/s]
Shard 17:  56%|█████▌    | 55803099/100000000 [00:13<00:06, 6415799.62tokens/s]
Shard 17:  58%|█████▊    | 57505114/100000000 [00:13<00:04, 9413999.29tokens/s]
Shard 17:  58%|█████▊    | 58465510/100000000 [00:13<00:06, 6651558.99tokens/s]
Shard 17:  59%|█████▉    | 59255974/100000000 [00:13<00:06, 6250925.66tokens/s]
Shard 17:  60%|█████▉    | 59967812/100000000 [00:13<00:06, 5901404.65tokens/s]
Shard 17:  61%|██████    | 60617998/100000000 [00:13<00:06, 5904880.30tokens/s]
Shard 17:  61%|██████▏   | 61265390/100000000 [00:14<00:06, 5655825.16tokens/s]
Shard 17:  62%|██████▏   | 62126949/100000000 [00:14<00:05, 6339648.39tokens/s]
Shard 17:  63%|██████▎   | 62795397/100000000 [00:14<00:06, 5367345.66tokens/s]
Shard 17:  63%|██████▎   | 63394175/100000000 [00:14<00:07, 4976750.85tokens/s]
Shard 17:  64%|██████▍   | 63924388/100000000 [00:14<00:08, 4392385.67tokens/s]
Shard 17:  64%|██████▍   | 64430822/100000000 [00:14<00:08, 4136220.12tokens/s]
Shard 17:  65%|██████▍   | 64882433/100000000 [00:14<00:08, 4211525.53tokens/s]
Shard 17:  65%|██████▌   | 65320801/100000000 [00:14<00:08, 3922102.18tokens/s]
Shard 17:  66%|██████▌   | 65754638/100000000 [00:15<00:08, 4021347.63tokens/s]
Shard 17:  66%|██████▌   | 66167482/100000000 [00:15<00:08, 3815929.82tokens/s]
Shard 17:  67%|██████▋   | 66793064/100000000 [00:15<00:07, 4414424.96tokens/s]
Shard 17:  67%|██████▋   | 67249182/100000000 [00:15<00:07, 4135974.56tokens/s]
Shard 17:  68%|██████▊   | 67686914/100000000 [00:15<00:08, 3679446.70tokens/s]
Shard 17:  68%|██████▊   | 68150571/100000000 [00:15<00:08, 3887558.93tokens/s]
Shard 17:  69%|██████▊   | 68605793/100000000 [00:15<00:07, 4056424.06tokens/s]
Shard 17:  69%|██████▉   | 69025198/100000000 [00:15<00:08, 3830960.69tokens/s]
Shard 17:  70%|██████▉   | 69611426/100000000 [00:16<00:06, 4369783.85tokens/s]
Shard 17:  70%|███████   | 70100897/100000000 [00:16<00:06, 4512625.44tokens/s]
Shard 17:  71%|███████   | 70565847/100000000 [00:16<00:06, 4410085.70tokens/s]
Shard 17:  71%|███████   | 71151725/100000000 [00:16<00:05, 4816686.81tokens/s]
Shard 17:  72%|███████▏  | 71642159/100000000 [00:16<00:06, 4497652.27tokens/s]
Shard 17:  72%|███████▏  | 72101779/100000000 [00:16<00:06, 4172798.25tokens/s]
Shard 17:  73%|███████▎  | 72616262/100000000 [00:16<00:06, 4430449.98tokens/s]
Shard 17:  73%|███████▎  | 73069613/100000000 [00:16<00:06, 4207381.23tokens/s]
Shard 17:  74%|███████▍  | 74063096/100000000 [00:16<00:04, 5723676.36tokens/s]
Shard 17:  75%|███████▍  | 74655551/100000000 [00:17<00:04, 5623866.14tokens/s]
Shard 17:  75%|███████▌  | 75232374/100000000 [00:17<00:04, 5369726.34tokens/s]
Shard 17:  76%|███████▌  | 75780340/100000000 [00:17<00:04, 5250443.63tokens/s]
Shard 17:  76%|███████▋  | 76312819/100000000 [00:17<00:05, 4561456.96tokens/s]
Shard 17:  77%|███████▋  | 76787932/100000000 [00:17<00:05, 4301090.06tokens/s]
Shard 17:  77%|███████▋  | 77233390/100000000 [00:17<00:05, 4127442.48tokens/s]
Shard 17:  78%|███████▊  | 77656385/100000000 [00:17<00:05, 4119926.91tokens/s]
Shard 17:  78%|███████▊  | 78076112/100000000 [00:17<00:05, 4009560.70tokens/s]
Shard 17:  78%|███████▊  | 78481046/100000000 [00:18<00:05, 3657704.83tokens/s]
Shard 17:  79%|███████▉  | 78853369/100000000 [00:18<00:05, 3579224.84tokens/s]
Shard 17:  80%|████████  | 80082294/100000000 [00:18<00:03, 5839023.76tokens/s]
Shard 17:  81%|████████  | 80694013/100000000 [00:18<00:03, 5274867.49tokens/s]
Shard 17:  81%|████████  | 81249349/100000000 [00:18<00:03, 4818839.12tokens/s]
Shard 17:  82%|████████▏ | 81755038/100000000 [00:18<00:04, 4279176.34tokens/s]
Shard 17:  82%|████████▏ | 82206257/100000000 [00:18<00:04, 3921684.68tokens/s]
Shard 17:  83%|████████▎ | 82702110/100000000 [00:18<00:04, 4154330.83tokens/s]
Shard 17:  83%|████████▎ | 83137137/100000000 [00:19<00:04, 4080037.10tokens/s]
Shard 17:  84%|████████▎ | 83557344/100000000 [00:19<00:04, 3989191.31tokens/s]
Shard 17:  84%|████████▍ | 83964046/100000000 [00:19<00:04, 3912319.50tokens/s]
Shard 17:  84%|████████▍ | 84360406/100000000 [00:19<00:04, 3810764.90tokens/s]
Shard 17:  85%|████████▍ | 84746044/100000000 [00:19<00:04, 3405151.44tokens/s]
Shard 17:  85%|████████▌ | 85274775/100000000 [00:19<00:03, 3862733.48tokens/s]
Shard 17:  86%|████████▌ | 85674633/100000000 [00:19<00:03, 3827573.82tokens/s]
Shard 17:  86%|████████▌ | 86151434/100000000 [00:19<00:03, 4083093.35tokens/s]
Shard 17:  87%|████████▋ | 86649565/100000000 [00:19<00:03, 4302262.71tokens/s]
Shard 17:  87%|████████▋ | 87128144/100000000 [00:20<00:02, 4416222.50tokens/s]
Shard 17:  88%|████████▊ | 87587460/100000000 [00:20<00:02, 4422271.86tokens/s]
Shard 17:  88%|████████▊ | 88121779/100000000 [00:20<00:02, 4679144.99tokens/s]
Shard 17:  89%|████████▊ | 88593389/100000000 [00:20<00:02, 4206619.67tokens/s]
Shard 17:  89%|████████▉ | 89024926/100000000 [00:20<00:02, 4153148.93tokens/s]
Shard 17:  89%|████████▉ | 89449864/100000000 [00:20<00:02, 3639581.27tokens/s]
Shard 17:  90%|████████▉ | 89828680/100000000 [00:20<00:02, 3477975.74tokens/s]
Shard 17:  90%|█████████ | 90203300/100000000 [00:20<00:02, 3546779.06tokens/s]
Shard 17:  91%|█████████ | 90690839/100000000 [00:20<00:02, 3900768.56tokens/s]
Shard 17:  91%|█████████ | 91150093/100000000 [00:21<00:02, 4086109.23tokens/s]
Shard 17:  92%|█████████▏| 91638396/100000000 [00:21<00:01, 4255214.60tokens/s]
Shard 17:  92%|█████████▏| 92318741/100000000 [00:21<00:01, 4966576.69tokens/s]
Shard 17:  93%|█████████▎| 92827670/100000000 [00:21<00:01, 4560899.62tokens/s]
Shard 17:  93%|█████████▎| 93296371/100000000 [00:21<00:01, 4480180.01tokens/s]
Shard 17:  94%|█████████▍| 93752178/100000000 [00:21<00:01, 4311679.38tokens/s]
Shard 17:  94%|█████████▍| 94189954/100000000 [00:21<00:01, 4032592.07tokens/s]
Shard 17:  95%|█████████▍| 94600756/100000000 [00:21<00:01, 3766036.19tokens/s]
Shard 17:  95%|█████████▌| 95043604/100000000 [00:21<00:01, 3939068.36tokens/s]
Shard 17:  96%|█████████▌| 95504798/100000000 [00:22<00:01, 4103630.49tokens/s]
Shard 17:  96%|█████████▌| 96247779/100000000 [00:22<00:00, 5020205.50tokens/s]
Shard 17:  97%|█████████▋| 96760987/100000000 [00:22<00:00, 4892987.70tokens/s]
Shard 17:  97%|█████████▋| 97268742/100000000 [00:22<00:00, 4931935.63tokens/s]
Shard 17:  98%|█████████▊| 97767975/100000000 [00:22<00:00, 4294883.80tokens/s]
Shard 17:  98%|█████████▊| 98215087/100000000 [00:22<00:00, 4048371.43tokens/s]
Shard 17:  99%|█████████▊| 98633243/100000000 [00:22<00:00, 3359422.50tokens/s]
Shard 17:  99%|█████████▉| 98994720/100000000 [00:22<00:00, 3151819.53tokens/s]
Shard 17:  99%|█████████▉| 99327570/100000000 [00:23<00:00, 3035898.96tokens/s]
Shard 17: 100%|█████████▉| 99642041/100000000 [00:23<00:00, 2777079.26tokens/s]
Shard 17: 100%|█████████▉| 99928861/100000000 [00:23<00:00, 2516517.67tokens/s]
Shard 17: 100%|█████████▉| 99999969/100000000 [00:32<00:00, 3103866.83tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000017.bin (200,001,024 bytes) in the gpt-2 format

Shard 18:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 18:   0%|          | 391495/100000000 [00:00<00:25, 3912752.15tokens/s]
Shard 18:   1%|          | 783367/100000000 [00:00<00:32, 3037695.73tokens/s]
Shard 18:   1%|          | 1102582/100000000 [00:00<00:32, 3031594.72tokens/s]
Shard 18:   1%|▏         | 1413083/100000000 [00:00<00:32, 3046192.75tokens/s]
Shard 18:   2%|▏         | 1883141/100000000 [00:00<00:27, 3594064.08tokens/s]
Shard 18:   2%|▏         | 2249264/100000000 [00:00<00:28, 3374539.40tokens/s]
Shard 18:   3%|▎         | 2640510/100000000 [00:00<00:27, 3528356.89tokens/s]
Shard 18:   3%|▎         | 3243227/100000000 [00:00<00:22, 4248420.85tokens/s]
Shard 18:   4%|▍         | 3798353/100000000 [00:00<00:20, 4617855.64tokens/s]
Shard 18:   4%|▍         | 4266249/100000000 [00:01<00:23, 4118629.25tokens/s]
Shard 18:   5%|▍         | 4691361/100000000 [00:01<00:24, 3851216.32tokens/s]
Shard 18:   5%|▌         | 5089356/100000000 [00:01<00:25, 3744760.45tokens/s]
Shard 18:   6%|▌         | 5536630/100000000 [00:01<00:24, 3926334.83tokens/s]
Shard 18:   6%|▌         | 5988833/100000000 [00:01<00:23, 4082472.99tokens/s]
Shard 18:   6%|▋         | 6403425/100000000 [00:01<00:24, 3842865.72tokens/s]
Shard 18:   7%|▋         | 6794103/100000000 [00:01<00:27, 3422151.66tokens/s]
Shard 18:   7%|▋         | 7167987/100000000 [00:01<00:26, 3497277.18tokens/s]
Shard 18:   8%|▊         | 7725246/100000000 [00:02<00:22, 4052474.91tokens/s]
Shard 18:   8%|▊         | 8145454/100000000 [00:02<00:22, 4020748.41tokens/s]
Shard 18:   9%|▊         | 8556832/100000000 [00:02<00:22, 4025365.40tokens/s]
Shard 18:   9%|▉         | 8989834/100000000 [00:02<00:22, 4095956.23tokens/s]
Shard 18:   9%|▉         | 9410296/100000000 [00:02<00:22, 4042480.63tokens/s]
Shard 18:  10%|▉         | 9962467/100000000 [00:02<00:20, 4458516.02tokens/s]
Shard 18:  10%|█         | 10412834/100000000 [00:02<00:22, 4030762.14tokens/s]
Shard 18:  11%|█         | 10842848/100000000 [00:02<00:21, 4098398.16tokens/s]
Shard 18:  11%|█▏        | 11262805/100000000 [00:02<00:22, 3947728.07tokens/s]
Shard 18:  12%|█▏        | 11663519/100000000 [00:03<00:23, 3692001.60tokens/s]
Shard 18:  13%|█▎        | 12875834/100000000 [00:03<00:14, 5955400.93tokens/s]
Shard 18:  14%|█▎        | 13503186/100000000 [00:03<00:18, 4705872.88tokens/s]
Shard 18:  14%|█▍        | 14043726/100000000 [00:03<00:18, 4657879.44tokens/s]
Shard 18:  15%|█▍        | 14556637/100000000 [00:03<00:19, 4428005.77tokens/s]
Shard 18:  15%|█▌        | 15029295/100000000 [00:03<00:19, 4263741.35tokens/s]
Shard 18:  15%|█▌        | 15475417/100000000 [00:03<00:22, 3699532.66tokens/s]
Shard 18:  16%|█▌        | 16013591/100000000 [00:03<00:20, 4074261.91tokens/s]
Shard 18:  17%|█▋        | 16505226/100000000 [00:04<00:19, 4276759.81tokens/s]
Shard 18:  17%|█▋        | 16957602/100000000 [00:04<00:20, 4119345.92tokens/s]
Shard 18:  17%|█▋        | 17385058/100000000 [00:04<00:20, 4119484.24tokens/s]
Shard 18:  18%|█▊        | 17807800/100000000 [00:04<00:22, 3718357.22tokens/s]
Shard 18:  18%|█▊        | 18241272/100000000 [00:04<00:21, 3866943.02tokens/s]
Shard 18:  19%|█▊        | 18736744/100000000 [00:04<00:19, 4147793.68tokens/s]
Shard 18:  19%|█▉        | 19216611/100000000 [00:04<00:18, 4322724.77tokens/s]
Shard 18:  20%|█▉        | 19667870/100000000 [00:04<00:18, 4360629.13tokens/s]
Shard 18:  20%|██        | 20112117/100000000 [00:04<00:18, 4336323.14tokens/s]
Shard 18:  21%|██        | 20550622/100000000 [00:05<00:21, 3755141.22tokens/s]
Shard 18:  21%|██        | 20942427/100000000 [00:05<00:23, 3373761.09tokens/s]
Shard 18:  21%|██▏       | 21296717/100000000 [00:05<00:23, 3374634.63tokens/s]
Shard 18:  22%|██▏       | 21646181/100000000 [00:05<00:24, 3228059.78tokens/s]
Shard 18:  22%|██▏       | 21977176/100000000 [00:05<00:26, 2984594.18tokens/s]
Shard 18:  22%|██▏       | 22283449/100000000 [00:05<00:26, 2897763.20tokens/s]
Shard 18:  23%|██▎       | 22787054/100000000 [00:05<00:22, 3453652.28tokens/s]
Shard 18:  23%|██▎       | 23166176/100000000 [00:05<00:21, 3540099.94tokens/s]
Shard 18:  24%|██▎       | 23529276/100000000 [00:06<00:23, 3294676.63tokens/s]
Shard 18:  24%|██▍       | 23868514/100000000 [00:06<00:25, 2964732.02tokens/s]
Shard 18:  24%|██▍       | 24236793/100000000 [00:06<00:24, 3126351.36tokens/s]
Shard 18:  25%|██▍       | 24617130/100000000 [00:06<00:22, 3283354.88tokens/s]
Shard 18:  25%|██▌       | 25142380/100000000 [00:06<00:19, 3815691.42tokens/s]
Shard 18:  26%|██▌       | 25536677/100000000 [00:06<00:20, 3704861.64tokens/s]
Shard 18:  26%|██▌       | 25915323/100000000 [00:06<00:20, 3582907.73tokens/s]
Shard 18:  26%|██▋       | 26374449/100000000 [00:06<00:19, 3833893.49tokens/s]
Shard 18:  27%|██▋       | 26852550/100000000 [00:06<00:17, 4083482.05tokens/s]
Shard 18:  27%|██▋       | 27273991/100000000 [00:07<00:17, 4100804.29tokens/s]
Shard 18:  28%|██▊       | 27690331/100000000 [00:07<00:18, 3959767.49tokens/s]
Shard 18:  28%|██▊       | 28090157/100000000 [00:07<00:18, 3817053.37tokens/s]
Shard 18:  28%|██▊       | 28474895/100000000 [00:07<00:20, 3481254.89tokens/s]
Shard 18:  29%|██▉       | 28868811/100000000 [00:07<00:19, 3574286.53tokens/s]
Shard 18:  29%|██▉       | 29269503/100000000 [00:07<00:19, 3671750.80tokens/s]
Shard 18:  30%|██▉       | 29642450/100000000 [00:07<00:20, 3366397.18tokens/s]
Shard 18:  30%|██▉       | 29986532/100000000 [00:07<00:22, 3068449.61tokens/s]
Shard 18:  30%|███       | 30306627/100000000 [00:08<00:22, 3054172.53tokens/s]
Shard 18:  31%|███       | 30635259/100000000 [00:08<00:22, 3113552.79tokens/s]
Shard 18:  31%|███       | 31026237/100000000 [00:08<00:20, 3314320.90tokens/s]
Shard 18:  31%|███▏      | 31411135/100000000 [00:08<00:20, 3410862.37tokens/s]
Shard 18:  32%|███▏      | 31756082/100000000 [00:08<00:22, 3069979.15tokens/s]
Shard 18:  32%|███▏      | 32307273/100000000 [00:08<00:18, 3704639.21tokens/s]
Shard 18:  33%|███▎      | 32691745/100000000 [00:08<00:18, 3575682.05tokens/s]
Shard 18:  33%|███▎      | 33073439/100000000 [00:08<00:18, 3605651.36tokens/s]
Shard 18:  34%|███▎      | 33557340/100000000 [00:08<00:16, 3945946.75tokens/s]
Shard 18:  34%|███▍      | 33961703/100000000 [00:08<00:16, 3906741.48tokens/s]
Shard 18:  34%|███▍      | 34363679/100000000 [00:09<00:19, 3309711.64tokens/s]
Shard 18:  35%|███▍      | 34713725/100000000 [00:09<00:23, 2767716.35tokens/s]
Shard 18:  35%|███▌      | 35205409/100000000 [00:09<00:19, 3253139.19tokens/s]
Shard 18:  36%|███▌      | 35713153/100000000 [00:09<00:17, 3702107.87tokens/s]
Shard 18:  36%|███▌      | 36141937/100000000 [00:09<00:16, 3847036.79tokens/s]
Shard 18:  37%|███▋      | 36550713/100000000 [00:09<00:16, 3903990.75tokens/s]
Shard 18:  37%|███▋      | 37004146/100000000 [00:09<00:15, 4047743.69tokens/s]
Shard 18:  37%|███▋      | 37421881/100000000 [00:09<00:16, 3742161.67tokens/s]
Shard 18:  38%|███▊      | 37817926/100000000 [00:10<00:16, 3784246.58tokens/s]
Shard 18:  38%|███▊      | 38205808/100000000 [00:10<00:17, 3562880.55tokens/s]
Shard 18:  39%|███▊      | 38570509/100000000 [00:10<00:17, 3495796.04tokens/s]
Shard 18:  39%|███▉      | 39161703/100000000 [00:10<00:14, 4137500.82tokens/s]
Shard 18:  40%|███▉      | 39585091/100000000 [00:10<00:14, 4034287.11tokens/s]
Shard 18:  40%|████      | 40021137/100000000 [00:10<00:14, 4111969.00tokens/s]
Shard 18:  40%|████      | 40442176/100000000 [00:10<00:15, 3737045.31tokens/s]
Shard 18:  41%|████      | 40914516/100000000 [00:10<00:14, 4000075.04tokens/s]
Shard 18:  41%|████▏     | 41324300/100000000 [00:10<00:15, 3708401.54tokens/s]
Shard 18:  42%|████▏     | 41734822/100000000 [00:11<00:15, 3812581.86tokens/s]
Shard 18:  42%|████▏     | 42275028/100000000 [00:11<00:13, 4251492.12tokens/s]
Shard 18:  43%|████▎     | 42740110/100000000 [00:11<00:14, 3947143.05tokens/s]
Shard 18:  43%|████▎     | 43146767/100000000 [00:11<00:15, 3755497.97tokens/s]
Shard 18:  44%|████▎     | 43536406/100000000 [00:11<00:15, 3660472.95tokens/s]
Shard 18:  44%|████▍     | 44017618/100000000 [00:11<00:14, 3947759.02tokens/s]
Shard 18:  44%|████▍     | 44420165/100000000 [00:11<00:14, 3849121.88tokens/s]
Shard 18:  45%|████▍     | 44810007/100000000 [00:11<00:14, 3723091.77tokens/s]
Shard 18:  45%|████▌     | 45198068/100000000 [00:11<00:14, 3766437.68tokens/s]
Shard 18:  46%|████▌     | 45634669/100000000 [00:12<00:13, 3913045.66tokens/s]
Shard 18:  46%|████▌     | 46029592/100000000 [00:12<00:14, 3722960.29tokens/s]
Shard 18:  46%|████▋     | 46405272/100000000 [00:12<00:15, 3484092.78tokens/s]
Shard 18:  47%|████▋     | 46866451/100000000 [00:12<00:14, 3785474.87tokens/s]
Shard 18:  48%|████▊     | 47512741/100000000 [00:12<00:11, 4533144.45tokens/s]
Shard 18:  48%|████▊     | 48020661/100000000 [00:12<00:11, 4688229.10tokens/s]
Shard 18:  48%|████▊     | 48497174/100000000 [00:12<00:11, 4406121.58tokens/s]
Shard 18:  49%|████▉     | 48947100/100000000 [00:12<00:15, 3358407.09tokens/s]
Shard 18:  49%|████▉     | 49324703/100000000 [00:13<00:14, 3454856.70tokens/s]
Shard 18:  50%|████▉     | 49899435/100000000 [00:13<00:12, 4016594.19tokens/s]
Shard 18:  51%|█████     | 50759807/100000000 [00:13<00:09, 5217622.23tokens/s]
Shard 18:  51%|█████▏    | 51323014/100000000 [00:13<00:09, 4923426.94tokens/s]
Shard 18:  52%|█████▏    | 51846643/100000000 [00:13<00:12, 3896264.28tokens/s]
Shard 18:  52%|█████▏    | 52289365/100000000 [00:13<00:14, 3362062.22tokens/s]
Shard 18:  53%|█████▎    | 52671575/100000000 [00:13<00:14, 3314291.50tokens/s]
Shard 18:  53%|█████▎    | 53034495/100000000 [00:14<00:14, 3295635.81tokens/s]
Shard 18:  53%|█████▎    | 53449649/100000000 [00:14<00:13, 3495111.78tokens/s]
Shard 18:  54%|█████▍    | 53818877/100000000 [00:14<00:15, 2973422.82tokens/s]
Shard 18:  54%|█████▍    | 54140918/100000000 [00:14<00:17, 2696228.50tokens/s]
Shard 18:  54%|█████▍    | 54452347/100000000 [00:14<00:16, 2785932.14tokens/s]
Shard 18:  55%|█████▍    | 54751174/100000000 [00:14<00:15, 2830819.22tokens/s]
Shard 18:  55%|█████▌    | 55172766/100000000 [00:14<00:14, 3151579.81tokens/s]
Shard 18:  56%|█████▌    | 55621626/100000000 [00:14<00:12, 3503975.85tokens/s]
Shard 18:  56%|█████▌    | 55984543/100000000 [00:14<00:13, 3284356.45tokens/s]
Shard 18:  57%|█████▋    | 56527848/100000000 [00:15<00:11, 3854793.89tokens/s]
Shard 18:  57%|█████▋    | 56929262/100000000 [00:15<00:12, 3581524.59tokens/s]
Shard 18:  57%|█████▋    | 57300149/100000000 [00:15<00:12, 3340838.92tokens/s]
Shard 18:  58%|█████▊    | 57915776/100000000 [00:15<00:10, 4067130.38tokens/s]
Shard 18:  58%|█████▊    | 58342235/100000000 [00:15<00:10, 4049696.21tokens/s]
Shard 18:  59%|█████▉    | 58762228/100000000 [00:15<00:10, 4034942.80tokens/s]
Shard 18:  59%|█████▉    | 59174430/100000000 [00:15<00:12, 3225324.21tokens/s]
Shard 18:  60%|█████▉    | 59527632/100000000 [00:15<00:13, 3110313.57tokens/s]
Shard 18:  60%|█████▉    | 59870803/100000000 [00:16<00:13, 2949662.50tokens/s]
Shard 18:  60%|██████    | 60181106/100000000 [00:16<00:13, 2859452.95tokens/s]
Shard 18:  61%|██████    | 60587700/100000000 [00:16<00:12, 3110314.84tokens/s]
Shard 18:  61%|██████    | 60934553/100000000 [00:16<00:12, 3200697.13tokens/s]
Shard 18:  61%|██████▏   | 61263582/100000000 [00:16<00:12, 3208820.28tokens/s]
Shard 18:  62%|██████▏   | 61590953/100000000 [00:16<00:12, 3058458.59tokens/s]
Shard 18:  62%|██████▏   | 61979345/100000000 [00:16<00:11, 3257463.47tokens/s]
Shard 18:  62%|██████▏   | 62331072/100000000 [00:16<00:11, 3320406.96tokens/s]
Shard 18:  63%|██████▎   | 62908989/100000000 [00:16<00:09, 4012669.27tokens/s]
Shard 18:  63%|██████▎   | 63425252/100000000 [00:17<00:08, 4343983.46tokens/s]
Shard 18:  64%|██████▍   | 63865308/100000000 [00:17<00:08, 4293518.58tokens/s]
Shard 18:  64%|██████▍   | 64299530/100000000 [00:17<00:08, 4264146.24tokens/s]
Shard 18:  65%|██████▍   | 64747074/100000000 [00:17<00:08, 4316114.71tokens/s]
Shard 18:  65%|██████▌   | 65182827/100000000 [00:17<00:08, 4043976.15tokens/s]
Shard 18:  66%|██████▌   | 65592659/100000000 [00:17<00:09, 3684717.01tokens/s]
Shard 18:  66%|██████▌   | 65970500/100000000 [00:17<00:09, 3576656.03tokens/s]
Shard 18:  66%|██████▋   | 66333751/100000000 [00:17<00:09, 3559978.27tokens/s]
Shard 18:  67%|██████▋   | 66693797/100000000 [00:17<00:10, 3306981.60tokens/s]
Shard 18:  67%|██████▋   | 67054591/100000000 [00:18<00:09, 3356841.70tokens/s]
Shard 18:  67%|██████▋   | 67396183/100000000 [00:18<00:11, 2957303.12tokens/s]
Shard 18:  68%|██████▊   | 67706526/100000000 [00:18<00:12, 2492326.39tokens/s]
Shard 18:  68%|██████▊   | 67972820/100000000 [00:18<00:15, 2035068.01tokens/s]
Shard 18:  68%|██████▊   | 68198592/100000000 [00:18<00:15, 2026928.29tokens/s]
Shard 18:  68%|██████▊   | 68420484/100000000 [00:18<00:15, 2022451.85tokens/s]
Shard 18:  69%|██████▊   | 68678913/100000000 [00:18<00:14, 2156806.40tokens/s]
Shard 18:  69%|██████▉   | 69006423/100000000 [00:19<00:12, 2403628.91tokens/s]
Shard 18:  69%|██████▉   | 69371530/100000000 [00:19<00:11, 2732666.55tokens/s]
Shard 18:  70%|██████▉   | 69723989/100000000 [00:19<00:10, 2882305.83tokens/s]
Shard 18:  70%|███████   | 70020942/100000000 [00:19<00:10, 2877346.91tokens/s]
Shard 18:  70%|███████   | 70314695/100000000 [00:19<00:10, 2724534.88tokens/s]
Shard 18:  71%|███████   | 70770415/100000000 [00:19<00:09, 3224919.80tokens/s]
Shard 18:  71%|███████   | 71116569/100000000 [00:19<00:08, 3290023.65tokens/s]
Shard 18:  71%|███████▏  | 71451852/100000000 [00:19<00:08, 3210323.46tokens/s]
Shard 18:  72%|███████▏  | 72095278/100000000 [00:19<00:06, 4123728.22tokens/s]
Shard 18:  73%|███████▎  | 72516212/100000000 [00:20<00:06, 4035624.84tokens/s]
Shard 18:  73%|███████▎  | 72925600/100000000 [00:20<00:07, 3544522.98tokens/s]
Shard 18:  73%|███████▎  | 73294761/100000000 [00:20<00:08, 3135558.42tokens/s]
Shard 18:  74%|███████▎  | 73624580/100000000 [00:20<00:08, 3051875.77tokens/s]
Shard 18:  74%|███████▍  | 73979642/100000000 [00:20<00:08, 3169204.40tokens/s]
Shard 18:  74%|███████▍  | 74379754/100000000 [00:20<00:07, 3376774.29tokens/s]
Shard 18:  75%|███████▍  | 74819734/100000000 [00:20<00:06, 3651735.60tokens/s]
Shard 18:  75%|███████▌  | 75255289/100000000 [00:20<00:06, 3818626.57tokens/s]
Shard 18:  76%|███████▌  | 75644326/100000000 [00:21<00:07, 3073854.31tokens/s]
Shard 18:  76%|███████▌  | 75979409/100000000 [00:21<00:08, 2861097.09tokens/s]
Shard 18:  76%|███████▋  | 76288581/100000000 [00:21<00:08, 2691779.94tokens/s]
Shard 18:  77%|███████▋  | 76571922/100000000 [00:21<00:08, 2716224.07tokens/s]
Shard 18:  77%|███████▋  | 76945257/100000000 [00:21<00:07, 2973705.53tokens/s]
Shard 18:  77%|███████▋  | 77263277/100000000 [00:21<00:07, 2940844.91tokens/s]
Shard 18:  78%|███████▊  | 77667920/100000000 [00:21<00:06, 3214504.25tokens/s]
Shard 18:  78%|███████▊  | 78071392/100000000 [00:21<00:06, 3421357.58tokens/s]
Shard 18:  78%|███████▊  | 78420423/100000000 [00:21<00:06, 3224993.29tokens/s]
Shard 18:  79%|███████▉  | 78754590/100000000 [00:22<00:06, 3228310.97tokens/s]
Shard 18:  79%|███████▉  | 79082083/100000000 [00:22<00:06, 3206633.11tokens/s]
Shard 18:  79%|███████▉  | 79419313/100000000 [00:22<00:06, 3252614.63tokens/s]
Shard 18:  80%|███████▉  | 79747790/100000000 [00:22<00:06, 3255835.12tokens/s]
Shard 18:  80%|████████  | 80091384/100000000 [00:22<00:06, 3304258.06tokens/s]
Shard 18:  80%|████████  | 80423207/100000000 [00:22<00:05, 3298222.18tokens/s]
Shard 18:  81%|████████  | 80755240/100000000 [00:22<00:05, 3299761.12tokens/s]
Shard 18:  81%|████████  | 81085863/100000000 [00:22<00:05, 3212872.46tokens/s]
Shard 18:  81%|████████▏ | 81408708/100000000 [00:22<00:05, 3119399.41tokens/s]
Shard 18:  82%|████████▏ | 81721981/100000000 [00:23<00:07, 2418333.51tokens/s]
Shard 18:  82%|████████▏ | 81989683/100000000 [00:23<00:07, 2439720.45tokens/s]
Shard 18:  82%|████████▏ | 82251198/100000000 [00:23<00:07, 2468271.54tokens/s]
Shard 18:  83%|████████▎ | 82510947/100000000 [00:23<00:07, 2289461.57tokens/s]
Shard 18:  83%|████████▎ | 82762021/100000000 [00:23<00:07, 2343929.61tokens/s]
Shard 18:  83%|████████▎ | 83074783/100000000 [00:23<00:06, 2514557.32tokens/s]
Shard 18:  83%|████████▎ | 83333158/100000000 [00:23<00:07, 2263968.73tokens/s]
Shard 18:  84%|████████▎ | 83568320/100000000 [00:23<00:07, 2268613.62tokens/s]
Shard 18:  84%|████████▍ | 83815533/100000000 [00:23<00:06, 2323211.70tokens/s]
Shard 18:  84%|████████▍ | 84190146/100000000 [00:24<00:05, 2707842.18tokens/s]
Shard 18:  85%|████████▍ | 84561383/100000000 [00:24<00:05, 2987064.00tokens/s]
Shard 18:  85%|████████▍ | 84867529/100000000 [00:24<00:05, 2743015.64tokens/s]
Shard 18:  85%|████████▌ | 85217590/100000000 [00:24<00:05, 2944438.57tokens/s]
Shard 18:  86%|████████▌ | 85519312/100000000 [00:24<00:05, 2892983.82tokens/s]
Shard 18:  86%|████████▌ | 85858429/100000000 [00:24<00:04, 3008221.04tokens/s]
Shard 18:  86%|████████▋ | 86412087/100000000 [00:24<00:03, 3722907.77tokens/s]
Shard 18:  87%|████████▋ | 86838915/100000000 [00:24<00:03, 3870662.02tokens/s]
Shard 18:  87%|████████▋ | 87231578/100000000 [00:24<00:03, 3578657.13tokens/s]
Shard 18:  88%|████████▊ | 87603920/100000000 [00:25<00:03, 3590684.39tokens/s]
Shard 18:  88%|████████▊ | 87968363/100000000 [00:25<00:03, 3283294.83tokens/s]
Shard 18:  88%|████████▊ | 88304797/100000000 [00:25<00:03, 3099265.36tokens/s]
Shard 18:  89%|████████▊ | 88656107/100000000 [00:25<00:03, 3207960.72tokens/s]
Shard 18:  89%|████████▉ | 89098960/100000000 [00:25<00:03, 3534224.05tokens/s]
Shard 18:  90%|████████▉ | 89568016/100000000 [00:25<00:02, 3854821.11tokens/s]
Shard 18:  90%|████████▉ | 89960935/100000000 [00:25<00:02, 3690957.00tokens/s]
Shard 18:  90%|█████████ | 90336639/100000000 [00:25<00:02, 3468342.37tokens/s]
Shard 18:  91%|█████████ | 90689690/100000000 [00:25<00:02, 3267011.95tokens/s]
Shard 18:  91%|█████████ | 91022926/100000000 [00:26<00:02, 3272526.79tokens/s]
Shard 18:  91%|█████████▏| 91354635/100000000 [00:26<00:02, 3283187.75tokens/s]
Shard 18:  92%|█████████▏| 91820886/100000000 [00:26<00:02, 3671180.70tokens/s]
Shard 18:  92%|█████████▏| 92193345/100000000 [00:26<00:02, 3626783.98tokens/s]
Shard 18:  93%|█████████▎| 92559105/100000000 [00:26<00:02, 3542218.90tokens/s]
Shard 18:  94%|█████████▎| 93532076/100000000 [00:26<00:01, 5299773.67tokens/s]
Shard 18:  94%|█████████▍| 94071894/100000000 [00:26<00:01, 5072717.59tokens/s]
Shard 18:  95%|█████████▍| 94588000/100000000 [00:26<00:01, 4698277.14tokens/s]
Shard 18:  95%|█████████▌| 95074819/100000000 [00:26<00:01, 4137471.90tokens/s]
Shard 18:  96%|█████████▌| 95506702/100000000 [00:27<00:01, 3698636.96tokens/s]
Shard 18:  96%|█████████▌| 96029214/100000000 [00:27<00:00, 4065947.61tokens/s]
Shard 18:  96%|█████████▋| 96456492/100000000 [00:27<00:00, 3595830.56tokens/s]
Shard 18:  97%|█████████▋| 96837534/100000000 [00:27<00:00, 3598001.63tokens/s]
Shard 18:  97%|█████████▋| 97212957/100000000 [00:27<00:00, 3362090.74tokens/s]
Shard 18:  98%|█████████▊| 97563679/100000000 [00:27<00:00, 3331699.83tokens/s]
Shard 18:  98%|█████████▊| 98189768/100000000 [00:27<00:00, 4078710.19tokens/s]
Shard 18:  99%|█████████▊| 98613770/100000000 [00:28<00:00, 3662414.97tokens/s]
Shard 18:  99%|█████████▉| 99136559/100000000 [00:28<00:00, 4057206.00tokens/s]
Shard 18: 100%|█████████▉| 99576682/100000000 [00:28<00:00, 3668817.11tokens/s]
Shard 18: 100%|█████████▉| 99962636/100000000 [00:28<00:00, 2909961.46tokens/s]
Shard 18: 100%|█████████▉| 99999232/100000000 [00:36<00:00, 2753402.55tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000018.bin (200,001,024 bytes) in the gpt-2 format

Shard 19:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 19:   0%|          | 252325/100000000 [00:00<00:40, 2444191.89tokens/s]
Shard 19:   0%|          | 497474/100000000 [00:00<00:43, 2287015.50tokens/s]
Shard 19:   1%|          | 727586/100000000 [00:00<00:44, 2207673.32tokens/s]
Shard 19:   1%|          | 948654/100000000 [00:00<00:47, 2093998.91tokens/s]
Shard 19:   1%|          | 1195366/100000000 [00:00<00:44, 2205719.53tokens/s]
Shard 19:   1%|▏         | 1440540/100000000 [00:00<00:43, 2262272.04tokens/s]
Shard 19:   2%|▏         | 1753931/100000000 [00:00<00:38, 2534964.11tokens/s]
Shard 19:   2%|▏         | 2009624/100000000 [00:00<00:38, 2514477.72tokens/s]
Shard 19:   2%|▏         | 2493559/100000000 [00:00<00:30, 3221896.02tokens/s]
Shard 19:   3%|▎         | 2829111/100000000 [00:01<00:29, 3250610.81tokens/s]
Shard 19:   3%|▎         | 3156754/100000000 [00:01<00:30, 3200000.05tokens/s]
Shard 19:   4%|▎         | 3561244/100000000 [00:01<00:28, 3436283.43tokens/s]
Shard 19:   4%|▍         | 3926572/100000000 [00:01<00:27, 3496805.13tokens/s]
Shard 19:   4%|▍         | 4326536/100000000 [00:01<00:26, 3597770.82tokens/s]
Shard 19:   5%|▍         | 4686986/100000000 [00:01<00:27, 3435856.78tokens/s]
Shard 19:   5%|▌         | 5243062/100000000 [00:01<00:23, 4040520.37tokens/s]
Shard 19:   6%|▌         | 5651180/100000000 [00:01<00:24, 3811623.74tokens/s]
Shard 19:   6%|▌         | 6037415/100000000 [00:01<00:27, 3384825.99tokens/s]
Shard 19:   6%|▋         | 6460581/100000000 [00:02<00:25, 3601669.95tokens/s]
Shard 19:   7%|▋         | 6831439/100000000 [00:02<00:26, 3524501.71tokens/s]
Shard 19:   7%|▋         | 7191431/100000000 [00:02<00:30, 3057255.14tokens/s]
Shard 19:   8%|▊         | 7542152/100000000 [00:02<00:29, 3132429.71tokens/s]
Shard 19:   8%|▊         | 7866877/100000000 [00:02<00:29, 3162258.93tokens/s]
Shard 19:   8%|▊         | 8226018/100000000 [00:02<00:28, 3269999.59tokens/s]
Shard 19:   9%|▊         | 8563561/100000000 [00:02<00:30, 3022147.51tokens/s]
Shard 19:   9%|▉         | 8929970/100000000 [00:02<00:28, 3177375.83tokens/s]
Shard 19:   9%|▉         | 9282835/100000000 [00:02<00:27, 3262965.79tokens/s]
Shard 19:  10%|▉         | 9636104/100000000 [00:03<00:28, 3162031.20tokens/s]
Shard 19:  10%|▉         | 9978548/100000000 [00:03<00:27, 3229224.45tokens/s]
Shard 19:  10%|█         | 10305235/100000000 [00:03<00:29, 3046002.28tokens/s]
Shard 19:  11%|█         | 10614615/100000000 [00:03<00:31, 2796911.16tokens/s]
Shard 19:  11%|█         | 11020351/100000000 [00:03<00:28, 3119950.02tokens/s]
Shard 19:  11%|█▏        | 11471881/100000000 [00:03<00:25, 3498419.22tokens/s]
Shard 19:  12%|█▏        | 11869562/100000000 [00:03<00:24, 3632687.91tokens/s]
Shard 19:  12%|█▏        | 12349884/100000000 [00:03<00:22, 3966350.27tokens/s]
Shard 19:  13%|█▎        | 12842167/100000000 [00:03<00:20, 4217346.06tokens/s]
Shard 19:  13%|█▎        | 13270250/100000000 [00:04<00:21, 3944974.68tokens/s]
Shard 19:  14%|█▎        | 13671597/100000000 [00:04<00:25, 3436137.98tokens/s]
Shard 19:  14%|█▍        | 14030539/100000000 [00:04<00:25, 3330952.13tokens/s]
Shard 19:  15%|█▍        | 14613224/100000000 [00:04<00:21, 3975746.61tokens/s]
Shard 19:  15%|█▌        | 15028265/100000000 [00:04<00:22, 3751033.67tokens/s]
Shard 19:  15%|█▌        | 15417253/100000000 [00:04<00:22, 3750735.04tokens/s]
Shard 19:  16%|█▌        | 15801748/100000000 [00:04<00:24, 3394043.98tokens/s]
Shard 19:  16%|█▌        | 16152661/100000000 [00:04<00:26, 3219539.20tokens/s]
Shard 19:  16%|█▋        | 16483166/100000000 [00:05<00:28, 2961512.35tokens/s]
Shard 19:  17%|█▋        | 16877084/100000000 [00:05<00:26, 3171291.75tokens/s]
Shard 19:  17%|█▋        | 17441670/100000000 [00:05<00:21, 3820344.42tokens/s]
Shard 19:  18%|█▊        | 17838748/100000000 [00:05<00:23, 3511606.64tokens/s]
Shard 19:  19%|█▊        | 18520281/100000000 [00:05<00:18, 4373608.65tokens/s]
Shard 19:  19%|█▉        | 18980534/100000000 [00:05<00:20, 3927173.12tokens/s]
Shard 19:  19%|█▉        | 19395888/100000000 [00:05<00:22, 3509486.76tokens/s]
Shard 19:  20%|█▉        | 19813972/100000000 [00:05<00:21, 3672970.43tokens/s]
Shard 19:  20%|██        | 20200330/100000000 [00:06<00:22, 3503194.44tokens/s]
Shard 19:  21%|██        | 20564379/100000000 [00:06<00:23, 3415999.30tokens/s]
Shard 19:  21%|██        | 20915462/100000000 [00:06<00:23, 3392027.20tokens/s]
Shard 19:  21%|██▏       | 21260918/100000000 [00:06<00:26, 3020549.65tokens/s]
Shard 19:  22%|██▏       | 21609817/100000000 [00:06<00:24, 3137522.25tokens/s]
Shard 19:  22%|██▏       | 21935171/100000000 [00:06<00:25, 3082345.30tokens/s]
Shard 19:  22%|██▏       | 22387112/100000000 [00:06<00:22, 3464605.00tokens/s]
Shard 19:  23%|██▎       | 22741804/100000000 [00:06<00:25, 3037652.11tokens/s]
Shard 19:  23%|██▎       | 23059655/100000000 [00:06<00:25, 2962216.51tokens/s]
Shard 19:  23%|██▎       | 23366270/100000000 [00:07<00:26, 2846078.63tokens/s]
Shard 19:  24%|██▎       | 23657873/100000000 [00:07<00:27, 2743230.28tokens/s]
Shard 19:  24%|██▍       | 23936840/100000000 [00:07<00:28, 2681873.24tokens/s]
Shard 19:  24%|██▍       | 24207715/100000000 [00:07<00:32, 2351657.60tokens/s]
Shard 19:  24%|██▍       | 24459841/100000000 [00:07<00:31, 2394182.80tokens/s]
Shard 19:  25%|██▍       | 24706006/100000000 [00:07<00:32, 2321416.37tokens/s]
Shard 19:  25%|██▌       | 25028456/100000000 [00:07<00:29, 2561128.26tokens/s]
Shard 19:  25%|██▌       | 25290459/100000000 [00:07<00:30, 2475988.04tokens/s]
Shard 19:  26%|██▌       | 25542808/100000000 [00:08<00:31, 2339328.23tokens/s]
Shard 19:  26%|██▌       | 25792333/100000000 [00:08<00:31, 2377367.24tokens/s]
Shard 19:  26%|██▌       | 26084955/100000000 [00:08<00:29, 2528809.33tokens/s]
Shard 19:  26%|██▋       | 26341195/100000000 [00:08<00:32, 2249732.63tokens/s]
Shard 19:  27%|██▋       | 26573634/100000000 [00:08<00:33, 2174372.20tokens/s]
Shard 19:  27%|██▋       | 26911681/100000000 [00:08<00:29, 2494293.90tokens/s]
Shard 19:  27%|██▋       | 27169152/100000000 [00:08<00:29, 2467558.07tokens/s]
Shard 19:  27%|██▋       | 27421784/100000000 [00:08<00:31, 2270255.62tokens/s]
Shard 19:  28%|██▊       | 27655105/100000000 [00:08<00:31, 2280195.31tokens/s]
Shard 19:  28%|██▊       | 27887481/100000000 [00:09<00:53, 1356479.64tokens/s]
Shard 19:  28%|██▊       | 28279654/100000000 [00:09<00:39, 1824178.36tokens/s]
Shard 19:  29%|██▊       | 28519950/100000000 [00:09<00:37, 1925839.41tokens/s]
Shard 19:  29%|██▉       | 28796645/100000000 [00:09<00:33, 2112961.41tokens/s]
Shard 19:  29%|██▉       | 29044835/100000000 [00:09<00:36, 1964208.33tokens/s]
Shard 19:  29%|██▉       | 29279945/100000000 [00:09<00:34, 2046109.97tokens/s]
Shard 19:  30%|██▉       | 29505533/100000000 [00:09<00:35, 2008199.63tokens/s]
Shard 19:  30%|██▉       | 29785304/100000000 [00:10<00:31, 2211221.48tokens/s]
Shard 19:  30%|███       | 30110327/100000000 [00:10<00:28, 2488824.44tokens/s]
Shard 19:  31%|███       | 30572673/100000000 [00:10<00:22, 3077022.32tokens/s]
Shard 19:  31%|███       | 31079551/100000000 [00:10<00:18, 3627526.25tokens/s]
Shard 19:  31%|███▏      | 31454487/100000000 [00:10<00:19, 3565548.55tokens/s]
Shard 19:  32%|███▏      | 31908306/100000000 [00:10<00:17, 3801196.77tokens/s]
Shard 19:  32%|███▏      | 32295259/100000000 [00:10<00:18, 3760020.90tokens/s]
Shard 19:  33%|███▎      | 32676080/100000000 [00:10<00:20, 3290768.74tokens/s]
Shard 19:  33%|███▎      | 33136420/100000000 [00:10<00:18, 3636604.58tokens/s]
Shard 19:  34%|███▎      | 33513976/100000000 [00:11<00:18, 3511813.97tokens/s]
Shard 19:  34%|███▍      | 34039542/100000000 [00:11<00:16, 3938625.54tokens/s]
Shard 19:  34%|███▍      | 34466688/100000000 [00:11<00:16, 4019847.81tokens/s]
Shard 19:  35%|███▍      | 34876312/100000000 [00:11<00:17, 3822560.41tokens/s]
Shard 19:  35%|███▌      | 35292790/100000000 [00:11<00:16, 3910633.73tokens/s]
Shard 19:  36%|███▌      | 35730361/100000000 [00:11<00:16, 4002416.10tokens/s]
Shard 19:  36%|███▌      | 36156671/100000000 [00:11<00:16, 3780708.89tokens/s]
Shard 19:  37%|███▋      | 36540262/100000000 [00:11<00:18, 3493915.53tokens/s]
Shard 19:  37%|███▋      | 36902284/100000000 [00:11<00:17, 3520933.14tokens/s]
Shard 19:  37%|███▋      | 37332534/100000000 [00:12<00:16, 3732037.76tokens/s]
Shard 19:  38%|███▊      | 37721620/100000000 [00:12<00:16, 3776323.03tokens/s]
Shard 19:  38%|███▊      | 38177079/100000000 [00:12<00:15, 3974650.99tokens/s]
Shard 19:  39%|███▊      | 38580237/100000000 [00:12<00:17, 3581587.22tokens/s]
Shard 19:  39%|███▉      | 38968511/100000000 [00:12<00:17, 3572566.14tokens/s]
Shard 19:  39%|███▉      | 39333489/100000000 [00:12<00:17, 3521534.79tokens/s]
Shard 19:  40%|███▉      | 39689930/100000000 [00:12<00:19, 3172667.82tokens/s]
Shard 19:  40%|████      | 40015307/100000000 [00:12<00:20, 2916096.57tokens/s]
Shard 19:  40%|████      | 40335320/100000000 [00:12<00:20, 2944386.23tokens/s]
Shard 19:  41%|████      | 40696768/100000000 [00:13<00:19, 3119905.25tokens/s]
Shard 19:  41%|████      | 41015953/100000000 [00:13<00:19, 3043971.42tokens/s]
Shard 19:  41%|████▏     | 41365717/100000000 [00:13<00:18, 3146125.61tokens/s]
Shard 19:  42%|████▏     | 41716145/100000000 [00:13<00:18, 3223679.01tokens/s]
Shard 19:  42%|████▏     | 42255076/100000000 [00:13<00:15, 3840079.05tokens/s]
Shard 19:  43%|████▎     | 42693045/100000000 [00:13<00:14, 3987470.84tokens/s]
Shard 19:  43%|████▎     | 43096168/100000000 [00:13<00:14, 3815354.17tokens/s]
Shard 19:  43%|████▎     | 43485264/100000000 [00:13<00:15, 3746235.03tokens/s]
Shard 19:  44%|████▍     | 43896311/100000000 [00:13<00:14, 3840197.65tokens/s]
Shard 19:  44%|████▍     | 44307164/100000000 [00:14<00:14, 3902570.62tokens/s]
Shard 19:  45%|████▍     | 44700322/100000000 [00:14<00:14, 3815080.09tokens/s]
Shard 19:  45%|████▌     | 45121791/100000000 [00:14<00:14, 3896462.23tokens/s]
Shard 19:  46%|████▌     | 45513223/100000000 [00:14<00:14, 3833084.76tokens/s]
Shard 19:  46%|████▌     | 45898272/100000000 [00:14<00:14, 3805996.85tokens/s]
Shard 19:  46%|████▋     | 46300525/100000000 [00:14<00:13, 3854673.36tokens/s]
Shard 19:  47%|████▋     | 46686796/100000000 [00:14<00:14, 3670592.29tokens/s]
Shard 19:  47%|████▋     | 47055985/100000000 [00:14<00:14, 3583640.88tokens/s]
Shard 19:  47%|████▋     | 47416428/100000000 [00:14<00:15, 3401905.62tokens/s]
Shard 19:  48%|████▊     | 47772158/100000000 [00:15<00:15, 3439523.55tokens/s]
Shard 19:  48%|████▊     | 48119280/100000000 [00:15<00:16, 3203195.11tokens/s]
Shard 19:  48%|████▊     | 48443873/100000000 [00:15<00:16, 3186367.81tokens/s]
Shard 19:  49%|████▉     | 48765060/100000000 [00:15<00:16, 3164789.92tokens/s]
Shard 19:  49%|████▉     | 49083416/100000000 [00:15<00:18, 2725798.90tokens/s]
Shard 19:  49%|████▉     | 49372073/100000000 [00:15<00:18, 2760233.24tokens/s]
Shard 19:  50%|████▉     | 49693270/100000000 [00:15<00:17, 2879464.68tokens/s]
Shard 19:  50%|█████     | 50009626/100000000 [00:15<00:17, 2938366.94tokens/s]
Shard 19:  50%|█████     | 50332155/100000000 [00:15<00:16, 3015176.11tokens/s]
Shard 19:  51%|█████     | 50667533/100000000 [00:16<00:15, 3110984.38tokens/s]
Shard 19:  51%|█████     | 51037985/100000000 [00:16<00:14, 3276110.50tokens/s]
Shard 19:  51%|█████▏    | 51368832/100000000 [00:16<00:14, 3246039.70tokens/s]
Shard 19:  52%|█████▏    | 51703993/100000000 [00:16<00:14, 3273683.55tokens/s]
Shard 19:  52%|█████▏    | 52033102/100000000 [00:16<00:15, 3018552.93tokens/s]
Shard 19:  52%|█████▏    | 52377939/100000000 [00:16<00:15, 3129265.04tokens/s]
Shard 19:  53%|█████▎    | 52718274/100000000 [00:16<00:14, 3202892.73tokens/s]
Shard 19:  53%|█████▎    | 53098287/100000000 [00:16<00:13, 3372155.67tokens/s]
Shard 19:  53%|█████▎    | 53438657/100000000 [00:16<00:13, 3352237.48tokens/s]
Shard 19:  54%|█████▍    | 53776193/100000000 [00:16<00:14, 3235573.26tokens/s]
Shard 19:  54%|█████▍    | 54104674/100000000 [00:17<00:14, 3249411.26tokens/s]
Shard 19:  55%|█████▍    | 54559724/100000000 [00:17<00:12, 3621025.82tokens/s]
Shard 19:  55%|█████▍    | 54954479/100000000 [00:17<00:12, 3647697.72tokens/s]
Shard 19:  55%|█████▌    | 55321068/100000000 [00:17<00:12, 3516745.57tokens/s]
Shard 19:  56%|█████▌    | 55770262/100000000 [00:17<00:11, 3785793.54tokens/s]
Shard 19:  56%|█████▌    | 56151693/100000000 [00:17<00:13, 3210311.52tokens/s]
Shard 19:  57%|█████▋    | 56554820/100000000 [00:17<00:12, 3413740.44tokens/s]
Shard 19:  57%|█████▋    | 56910929/100000000 [00:17<00:14, 3068815.69tokens/s]
Shard 19:  57%|█████▋    | 57299776/100000000 [00:17<00:13, 3258217.99tokens/s]
Shard 19:  58%|█████▊    | 57677585/100000000 [00:18<00:12, 3395766.19tokens/s]
Shard 19:  58%|█████▊    | 58037933/100000000 [00:18<00:12, 3426715.71tokens/s]
Shard 19:  59%|█████▊    | 58522336/100000000 [00:18<00:10, 3793165.66tokens/s]
Shard 19:  59%|█████▉    | 58909032/100000000 [00:18<00:10, 3754240.06tokens/s]
Shard 19:  59%|█████▉    | 59289900/100000000 [00:18<00:11, 3533423.93tokens/s]
Shard 19:  60%|█████▉    | 59756348/100000000 [00:18<00:10, 3846242.09tokens/s]
Shard 19:  60%|██████    | 60208519/100000000 [00:18<00:09, 4036630.45tokens/s]
Shard 19:  61%|██████    | 60622461/100000000 [00:18<00:09, 3995157.09tokens/s]
Shard 19:  61%|██████    | 61026087/100000000 [00:18<00:10, 3589058.07tokens/s]
Shard 19:  61%|██████▏   | 61394842/100000000 [00:19<00:11, 3394497.76tokens/s]
Shard 19:  62%|██████▏   | 61748886/100000000 [00:19<00:11, 3412495.48tokens/s]
Shard 19:  62%|██████▏   | 62110842/100000000 [00:19<00:11, 3159405.64tokens/s]
Shard 19:  62%|██████▏   | 62433847/100000000 [00:19<00:12, 3091977.90tokens/s]
Shard 19:  63%|██████▎   | 62747765/100000000 [00:19<00:12, 3007126.65tokens/s]
Shard 19:  63%|██████▎   | 63068827/100000000 [00:19<00:12, 3061089.15tokens/s]
Shard 19:  63%|██████▎   | 63483621/100000000 [00:19<00:10, 3363569.51tokens/s]
Shard 19:  64%|██████▍   | 63825065/100000000 [00:19<00:10, 3306496.64tokens/s]
Shard 19:  64%|██████▍   | 64384656/100000000 [00:19<00:09, 3949404.67tokens/s]
Shard 19:  65%|██████▍   | 64784740/100000000 [00:20<00:09, 3897393.40tokens/s]
Shard 19:  65%|██████▌   | 65276873/100000000 [00:20<00:08, 4108605.10tokens/s]
Shard 19:  66%|██████▌   | 65714764/100000000 [00:20<00:08, 4172120.02tokens/s]
Shard 19:  66%|██████▌   | 66133916/100000000 [00:20<00:08, 3967216.69tokens/s]
Shard 19:  67%|██████▋   | 66534266/100000000 [00:20<00:08, 3741659.61tokens/s]
Shard 19:  67%|██████▋   | 66979932/100000000 [00:20<00:08, 3916037.91tokens/s]
Shard 19:  67%|██████▋   | 67375512/100000000 [00:20<00:09, 3489694.81tokens/s]
Shard 19:  68%|██████▊   | 67835487/100000000 [00:20<00:08, 3779738.47tokens/s]
Shard 19:  68%|██████▊   | 68224625/100000000 [00:20<00:08, 3698996.77tokens/s]
Shard 19:  69%|██████▊   | 68703015/100000000 [00:21<00:07, 3988343.80tokens/s]
Shard 19:  69%|██████▉   | 69180434/100000000 [00:21<00:07, 4191808.56tokens/s]
Shard 19:  70%|██████▉   | 69612170/100000000 [00:21<00:07, 4187858.23tokens/s]
Shard 19:  70%|███████   | 70035597/100000000 [00:21<00:07, 3896130.33tokens/s]
Shard 19:  70%|███████   | 70481430/100000000 [00:21<00:07, 4038354.76tokens/s]
Shard 19:  71%|███████   | 70898115/100000000 [00:21<00:07, 3842016.33tokens/s]
Shard 19:  71%|███████▏  | 71287756/100000000 [00:21<00:07, 3825838.38tokens/s]
Shard 19:  72%|███████▏  | 71674310/100000000 [00:21<00:07, 3774839.09tokens/s]
Shard 19:  72%|███████▏  | 72054507/100000000 [00:21<00:08, 3400787.18tokens/s]
Shard 19:  72%|███████▏  | 72403157/100000000 [00:22<00:08, 3300699.59tokens/s]
Shard 19:  73%|███████▎  | 72738440/100000000 [00:22<00:08, 3306882.46tokens/s]
Shard 19:  73%|███████▎  | 73100709/100000000 [00:22<00:07, 3380322.27tokens/s]
Shard 19:  73%|███████▎  | 73442510/100000000 [00:22<00:08, 3229218.90tokens/s]
Shard 19:  74%|███████▍  | 73821941/100000000 [00:22<00:07, 3380672.08tokens/s]
Shard 19:  74%|███████▍  | 74270551/100000000 [00:22<00:07, 3659921.55tokens/s]
Shard 19:  75%|███████▌  | 75003069/100000000 [00:22<00:05, 4659618.07tokens/s]
Shard 19:  76%|███████▌  | 75520684/100000000 [00:22<00:05, 4797813.46tokens/s]
Shard 19:  76%|███████▌  | 76010011/100000000 [00:22<00:05, 4449637.59tokens/s]
Shard 19:  76%|███████▋  | 76462988/100000000 [00:23<00:06, 3474163.37tokens/s]
Shard 19:  77%|███████▋  | 76847266/100000000 [00:23<00:07, 3288328.99tokens/s]
Shard 19:  77%|███████▋  | 77201765/100000000 [00:23<00:07, 3174839.38tokens/s]
Shard 19:  78%|███████▊  | 77570561/100000000 [00:23<00:06, 3265595.22tokens/s]
Shard 19:  78%|███████▊  | 77911272/100000000 [00:23<00:06, 3203117.85tokens/s]
Shard 19:  78%|███████▊  | 78317108/100000000 [00:23<00:06, 3417158.00tokens/s]
Shard 19:  79%|███████▊  | 78668310/100000000 [00:23<00:06, 3379031.28tokens/s]
Shard 19:  79%|███████▉  | 79040380/100000000 [00:23<00:06, 3452653.08tokens/s]
Shard 19:  79%|███████▉  | 79411042/100000000 [00:24<00:05, 3520897.59tokens/s]
Shard 19:  80%|███████▉  | 79767631/100000000 [00:24<00:05, 3376518.90tokens/s]
Shard 19:  80%|████████  | 80168296/100000000 [00:24<00:05, 3537541.91tokens/s]
Shard 19:  81%|████████  | 80525797/100000000 [00:24<00:06, 2983902.73tokens/s]
Shard 19:  81%|████████  | 80919497/100000000 [00:24<00:05, 3216315.91tokens/s]
Shard 19:  81%|████████▏ | 81256188/100000000 [00:24<00:05, 3231039.49tokens/s]
Shard 19:  82%|████████▏ | 81667740/100000000 [00:24<00:05, 3462736.65tokens/s]
Shard 19:  82%|████████▏ | 82061855/100000000 [00:24<00:04, 3592365.45tokens/s]
Shard 19:  82%|████████▏ | 82428551/100000000 [00:24<00:05, 3128441.08tokens/s]
Shard 19:  83%|████████▎ | 82757694/100000000 [00:25<00:06, 2748581.64tokens/s]
Shard 19:  83%|████████▎ | 83055316/100000000 [00:25<00:06, 2796521.53tokens/s]
Shard 19:  83%|████████▎ | 83354189/100000000 [00:25<00:05, 2844837.12tokens/s]
Shard 19:  84%|████████▎ | 83649789/100000000 [00:25<00:05, 2846784.22tokens/s]
Shard 19:  84%|████████▍ | 83987100/100000000 [00:25<00:05, 2980101.26tokens/s]
Shard 19:  84%|████████▍ | 84342175/100000000 [00:25<00:05, 3130329.65tokens/s]
Shard 19:  85%|████████▍ | 84720449/100000000 [00:25<00:04, 3302183.54tokens/s]
Shard 19:  85%|████████▌ | 85054943/100000000 [00:25<00:04, 3174775.70tokens/s]
Shard 19:  85%|████████▌ | 85458647/100000000 [00:25<00:04, 3378118.06tokens/s]
Shard 19:  86%|████████▌ | 85829972/100000000 [00:26<00:04, 3422773.25tokens/s]
Shard 19:  86%|████████▌ | 86177704/100000000 [00:26<00:04, 3434671.59tokens/s]
Shard 19:  87%|████████▋ | 86523255/100000000 [00:26<00:04, 3266026.98tokens/s]
Shard 19:  87%|████████▋ | 86922997/100000000 [00:26<00:03, 3472660.88tokens/s]
Shard 19:  87%|████████▋ | 87273346/100000000 [00:26<00:03, 3338070.34tokens/s]
Shard 19:  88%|████████▊ | 87610137/100000000 [00:26<00:03, 3288279.37tokens/s]
Shard 19:  88%|████████▊ | 87942721/100000000 [00:26<00:03, 3160113.70tokens/s]
Shard 19:  88%|████████▊ | 88261101/100000000 [00:26<00:03, 3147989.08tokens/s]
Shard 19:  89%|████████▊ | 88628395/100000000 [00:26<00:03, 3287088.16tokens/s]
Shard 19:  89%|████████▉ | 89000342/100000000 [00:27<00:03, 3394435.20tokens/s]
Shard 19:  89%|████████▉ | 89341184/100000000 [00:27<00:03, 3172449.15tokens/s]
Shard 19:  90%|████████▉ | 89671624/100000000 [00:27<00:03, 3194177.01tokens/s]
Shard 19:  90%|████████▉ | 89993700/100000000 [00:27<00:03, 3000382.90tokens/s]
Shard 19:  90%|█████████ | 90327664/100000000 [00:27<00:03, 3072089.84tokens/s]
Shard 19:  91%|█████████ | 90639150/100000000 [00:27<00:03, 2973666.69tokens/s]
Shard 19:  91%|█████████ | 91062372/100000000 [00:27<00:02, 3322479.43tokens/s]
Shard 19:  91%|█████████▏| 91405233/100000000 [00:27<00:02, 3157493.16tokens/s]
Shard 19:  92%|█████████▏| 91755794/100000000 [00:27<00:02, 3238515.50tokens/s]
Shard 19:  92%|█████████▏| 92208445/100000000 [00:28<00:02, 3576677.88tokens/s]
Shard 19:  93%|█████████▎| 92607988/100000000 [00:28<00:02, 3693182.98tokens/s]
Shard 19:  93%|█████████▎| 93060994/100000000 [00:28<00:01, 3917384.86tokens/s]
Shard 19:  93%|█████████▎| 93455639/100000000 [00:28<00:01, 3659523.62tokens/s]
Shard 19:  94%|█████████▍| 93826618/100000000 [00:28<00:01, 3191647.88tokens/s]
Shard 19:  94%|█████████▍| 94158858/100000000 [00:28<00:01, 3029354.61tokens/s]
Shard 19:  94%|█████████▍| 94473787/100000000 [00:28<00:01, 3027949.50tokens/s]
Shard 19:  95%|█████████▍| 94794073/100000000 [00:28<00:01, 2888625.48tokens/s]
Shard 19:  95%|█████████▌| 95159190/100000000 [00:28<00:01, 3082285.67tokens/s]
Shard 19:  95%|█████████▌| 95488077/100000000 [00:29<00:01, 3123216.08tokens/s]
Shard 19:  96%|█████████▌| 95870304/100000000 [00:29<00:01, 3316513.58tokens/s]
Shard 19:  96%|█████████▌| 96224818/100000000 [00:29<00:01, 3371957.86tokens/s]
Shard 19:  97%|█████████▋| 96566512/100000000 [00:29<00:01, 3287775.89tokens/s]
Shard 19:  97%|█████████▋| 96950177/100000000 [00:29<00:00, 3444716.77tokens/s]
Shard 19:  97%|█████████▋| 97298173/100000000 [00:29<00:00, 3215879.40tokens/s]
Shard 19:  98%|█████████▊| 97818705/100000000 [00:29<00:00, 3767824.60tokens/s]
Shard 19:  98%|█████████▊| 98202911/100000000 [00:29<00:00, 3609870.96tokens/s]
Shard 19:  99%|█████████▊| 98603901/100000000 [00:29<00:00, 3707152.87tokens/s]
Shard 19:  99%|█████████▉| 98980620/100000000 [00:30<00:00, 3680938.53tokens/s]
Shard 19: 100%|█████████▉| 99516784/100000000 [00:30<00:00, 4130707.08tokens/s]
Shard 19: 100%|█████████▉| 99933529/100000000 [00:30<00:00, 3549483.53tokens/s]
Shard 19: 100%|█████████▉| 99999935/100000000 [00:37<00:00, 2642292.86tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000019.bin (200,001,024 bytes) in the gpt-2 format

Shard 20:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 20:   0%|          | 197308/100000000 [00:00<00:51, 1944477.76tokens/s]
Shard 20:   0%|          | 407596/100000000 [00:00<00:48, 2033911.66tokens/s]
Shard 20:   1%|          | 650080/100000000 [00:00<00:45, 2204818.70tokens/s]
Shard 20:   1%|          | 983217/100000000 [00:00<00:37, 2637255.54tokens/s]
Shard 20:   1%|▏         | 1327920/100000000 [00:00<00:33, 2926500.36tokens/s]
Shard 20:   2%|▏         | 1669404/100000000 [00:00<00:32, 3069092.12tokens/s]
Shard 20:   2%|▏         | 1982515/100000000 [00:00<00:31, 3088362.76tokens/s]
Shard 20:   2%|▏         | 2291954/100000000 [00:00<00:32, 2965060.34tokens/s]
Shard 20:   3%|▎         | 2715122/100000000 [00:00<00:29, 3346506.85tokens/s]
Shard 20:   3%|▎         | 3110349/100000000 [00:01<00:27, 3528340.13tokens/s]
Shard 20:   3%|▎         | 3465458/100000000 [00:01<00:28, 3367848.77tokens/s]
Shard 20:   4%|▍         | 3905138/100000000 [00:01<00:26, 3666086.01tokens/s]
Shard 20:   4%|▍         | 4276260/100000000 [00:01<00:26, 3647958.08tokens/s]
Shard 20:   5%|▍         | 4657517/100000000 [00:01<00:25, 3683811.46tokens/s]
Shard 20:   5%|▌         | 5130180/100000000 [00:01<00:23, 3978417.17tokens/s]
Shard 20:   6%|▌         | 5538389/100000000 [00:01<00:23, 4000504.77tokens/s]
Shard 20:   6%|▌         | 6057526/100000000 [00:01<00:21, 4346962.97tokens/s]
Shard 20:   7%|▋         | 6508016/100000000 [00:01<00:21, 4387079.00tokens/s]
Shard 20:   7%|▋         | 6994598/100000000 [00:01<00:20, 4497901.80tokens/s]
Shard 20:   7%|▋         | 7445017/100000000 [00:02<00:21, 4391625.99tokens/s]
Shard 20:   8%|▊         | 7885660/100000000 [00:02<00:21, 4254156.22tokens/s]
Shard 20:   8%|▊         | 8312529/100000000 [00:02<00:21, 4191376.30tokens/s]
Shard 20:   9%|▊         | 8732854/100000000 [00:02<00:23, 3945960.87tokens/s]
Shard 20:   9%|▉         | 9188921/100000000 [00:02<00:22, 4116030.92tokens/s]
Shard 20:  10%|▉         | 9603785/100000000 [00:02<00:24, 3750378.67tokens/s]
Shard 20:  10%|█         | 10293902/100000000 [00:02<00:19, 4585075.29tokens/s]
Shard 20:  11%|█         | 10766100/100000000 [00:02<00:22, 4024008.59tokens/s]
Shard 20:  11%|█         | 11188849/100000000 [00:03<00:24, 3553760.12tokens/s]
Shard 20:  12%|█▏        | 11571301/100000000 [00:03<00:24, 3613249.11tokens/s]
Shard 20:  12%|█▏        | 12017146/100000000 [00:03<00:23, 3814277.51tokens/s]
Shard 20:  12%|█▏        | 12414798/100000000 [00:03<00:23, 3664740.12tokens/s]
Shard 20:  13%|█▎        | 12803735/100000000 [00:03<00:23, 3724216.32tokens/s]
Shard 20:  13%|█▎        | 13314013/100000000 [00:03<00:21, 4104159.39tokens/s]
Shard 20:  14%|█▎        | 13733445/100000000 [00:03<00:25, 3387436.73tokens/s]
Shard 20:  14%|█▍        | 14098203/100000000 [00:03<00:26, 3290205.31tokens/s]
Shard 20:  14%|█▍        | 14473341/100000000 [00:03<00:25, 3398028.99tokens/s]
Shard 20:  15%|█▍        | 14886233/100000000 [00:04<00:23, 3590069.24tokens/s]
Shard 20:  15%|█▌        | 15265084/100000000 [00:04<00:23, 3626693.88tokens/s]
Shard 20:  16%|█▌        | 15691936/100000000 [00:04<00:22, 3793748.40tokens/s]
Shard 20:  16%|█▌        | 16182795/100000000 [00:04<00:20, 4066958.62tokens/s]
Shard 20:  17%|█▋        | 16623851/100000000 [00:04<00:20, 4138844.62tokens/s]
Shard 20:  17%|█▋        | 17050065/100000000 [00:04<00:20, 4038984.31tokens/s]
Shard 20:  18%|█▊        | 17618889/100000000 [00:04<00:18, 4506274.47tokens/s]
Shard 20:  18%|█▊        | 18142839/100000000 [00:04<00:17, 4710318.90tokens/s]
Shard 20:  19%|█▊        | 18617769/100000000 [00:04<00:18, 4484414.67tokens/s]
Shard 20:  19%|█▉        | 19072626/100000000 [00:05<00:19, 4137305.68tokens/s]
Shard 20:  19%|█▉        | 19493849/100000000 [00:05<00:19, 4037663.34tokens/s]
Shard 20:  20%|█▉        | 19947265/100000000 [00:05<00:19, 4164411.99tokens/s]
Shard 20:  20%|██        | 20439675/100000000 [00:05<00:18, 4299941.53tokens/s]
Shard 20:  21%|██        | 20920693/100000000 [00:05<00:17, 4428201.73tokens/s]
Shard 20:  21%|██▏       | 21366831/100000000 [00:05<00:19, 3981424.41tokens/s]
Shard 20:  22%|██▏       | 21774860/100000000 [00:05<00:21, 3680099.14tokens/s]
Shard 20:  22%|██▏       | 22153258/100000000 [00:05<00:21, 3646759.99tokens/s]
Shard 20:  23%|██▎       | 22694148/100000000 [00:05<00:18, 4116944.58tokens/s]
Shard 20:  23%|██▎       | 23146281/100000000 [00:06<00:18, 4223874.19tokens/s]
Shard 20:  24%|██▎       | 23580263/100000000 [00:06<00:17, 4247590.23tokens/s]
Shard 20:  24%|██▍       | 24034293/100000000 [00:06<00:17, 4315023.54tokens/s]
Shard 20:  24%|██▍       | 24470026/100000000 [00:06<00:20, 3770809.55tokens/s]
Shard 20:  25%|██▍       | 24861652/100000000 [00:06<00:20, 3686302.40tokens/s]
Shard 20:  25%|██▌       | 25241040/100000000 [00:06<00:20, 3639264.34tokens/s]
Shard 20:  26%|██▌       | 25611664/100000000 [00:06<00:20, 3550906.36tokens/s]
Shard 20:  26%|██▌       | 25973884/100000000 [00:06<00:22, 3356666.26tokens/s]
Shard 20:  26%|██▋       | 26334214/100000000 [00:06<00:21, 3405564.47tokens/s]
Shard 20:  27%|██▋       | 26680428/100000000 [00:07<00:22, 3290186.56tokens/s]
Shard 20:  27%|██▋       | 27083395/100000000 [00:07<00:20, 3484100.68tokens/s]
Shard 20:  27%|██▋       | 27435333/100000000 [00:07<00:21, 3413315.44tokens/s]
Shard 20:  28%|██▊       | 28116694/100000000 [00:07<00:16, 4338078.05tokens/s]
Shard 20:  29%|██▊       | 28556290/100000000 [00:07<00:21, 3375098.07tokens/s]
Shard 20:  29%|██▉       | 28929190/100000000 [00:07<00:22, 3180987.06tokens/s]
Shard 20:  29%|██▉       | 29382619/100000000 [00:07<00:20, 3486714.59tokens/s]
Shard 20:  30%|██▉       | 29774582/100000000 [00:07<00:19, 3586839.32tokens/s]
Shard 20:  30%|███       | 30152199/100000000 [00:08<00:20, 3386834.65tokens/s]
Shard 20:  31%|███       | 30558375/100000000 [00:08<00:19, 3553758.71tokens/s]
Shard 20:  31%|███       | 30926374/100000000 [00:08<00:19, 3535967.47tokens/s]
Shard 20:  32%|███▏      | 31892323/100000000 [00:08<00:13, 5214391.86tokens/s]
Shard 20:  32%|███▏      | 32433730/100000000 [00:08<00:13, 5008985.52tokens/s]
Shard 20:  33%|███▎      | 32950059/100000000 [00:08<00:14, 4632039.43tokens/s]
Shard 20:  34%|███▎      | 33557010/100000000 [00:08<00:13, 5002282.48tokens/s]
Shard 20:  34%|███▍      | 34071907/100000000 [00:08<00:14, 4462277.72tokens/s]
Shard 20:  35%|███▍      | 34538052/100000000 [00:08<00:14, 4414436.96tokens/s]
Shard 20:  35%|███▍      | 34992919/100000000 [00:09<00:17, 3784307.93tokens/s]
Shard 20:  35%|███▌      | 35395092/100000000 [00:09<00:19, 3392257.60tokens/s]
Shard 20:  36%|███▌      | 35875723/100000000 [00:09<00:17, 3718422.01tokens/s]
Shard 20:  36%|███▋      | 36269870/100000000 [00:09<00:17, 3729133.66tokens/s]
Shard 20:  37%|███▋      | 36681569/100000000 [00:09<00:16, 3823617.20tokens/s]
Shard 20:  37%|███▋      | 37076338/100000000 [00:09<00:17, 3642450.92tokens/s]
Shard 20:  37%|███▋      | 37451237/100000000 [00:09<00:17, 3492537.16tokens/s]
Shard 20:  38%|███▊      | 37868033/100000000 [00:09<00:17, 3637098.16tokens/s]
Shard 20:  38%|███▊      | 38240415/100000000 [00:10<00:16, 3656675.42tokens/s]
Shard 20:  39%|███▊      | 38630966/100000000 [00:10<00:16, 3722620.82tokens/s]
Shard 20:  39%|███▉      | 39006972/100000000 [00:10<00:19, 3203454.83tokens/s]
Shard 20:  39%|███▉      | 39363255/100000000 [00:10<00:18, 3287063.76tokens/s]
Shard 20:  40%|███▉      | 39706527/100000000 [00:10<00:18, 3316767.37tokens/s]
Shard 20:  40%|████      | 40066354/100000000 [00:10<00:17, 3392499.91tokens/s]
Shard 20:  40%|████      | 40418189/100000000 [00:10<00:18, 3268027.62tokens/s]
Shard 20:  41%|████      | 40750479/100000000 [00:10<00:19, 3043295.06tokens/s]
Shard 20:  41%|████      | 41097552/100000000 [00:10<00:18, 3151259.95tokens/s]
Shard 20:  41%|████▏     | 41418091/100000000 [00:11<00:18, 3092088.53tokens/s]
Shard 20:  42%|████▏     | 41747292/100000000 [00:11<00:18, 3135157.43tokens/s]
Shard 20:  42%|████▏     | 42155033/100000000 [00:11<00:16, 3403006.75tokens/s]
Shard 20:  43%|████▎     | 42511150/100000000 [00:11<00:16, 3448292.58tokens/s]
Shard 20:  43%|████▎     | 42859040/100000000 [00:11<00:18, 3121984.12tokens/s]
Shard 20:  43%|████▎     | 43181928/100000000 [00:11<00:18, 3148821.22tokens/s]
Shard 20:  44%|████▎     | 43570112/100000000 [00:11<00:17, 3298745.82tokens/s]
Shard 20:  44%|████▍     | 43904147/100000000 [00:11<00:18, 3050889.24tokens/s]
Shard 20:  44%|████▍     | 44274563/100000000 [00:11<00:17, 3214383.01tokens/s]
Shard 20:  45%|████▍     | 44743872/100000000 [00:12<00:15, 3619040.32tokens/s]
Shard 20:  45%|████▌     | 45367440/100000000 [00:12<00:12, 4346868.15tokens/s]
Shard 20:  46%|████▌     | 45810451/100000000 [00:12<00:12, 4357141.99tokens/s]
Shard 20:  46%|████▋     | 46298000/100000000 [00:12<00:11, 4500797.25tokens/s]
Shard 20:  47%|████▋     | 46753812/100000000 [00:12<00:12, 4245702.76tokens/s]
Shard 20:  47%|████▋     | 47184316/100000000 [00:12<00:12, 4088357.57tokens/s]
Shard 20:  48%|████▊     | 47597985/100000000 [00:12<00:13, 3840147.68tokens/s]
Shard 20:  48%|████▊     | 47987293/100000000 [00:12<00:15, 3349913.62tokens/s]
Shard 20:  48%|████▊     | 48478136/100000000 [00:12<00:13, 3725725.42tokens/s]
Shard 20:  49%|████▉     | 48866053/100000000 [00:13<00:13, 3720733.97tokens/s]
Shard 20:  49%|████▉     | 49249495/100000000 [00:13<00:13, 3678007.94tokens/s]
Shard 20:  50%|████▉     | 49630736/100000000 [00:13<00:13, 3668405.35tokens/s]
Shard 20:  50%|█████     | 50099096/100000000 [00:13<00:12, 3950937.79tokens/s]
Shard 20:  51%|█████     | 50504640/100000000 [00:13<00:13, 3754743.22tokens/s]
Shard 20:  51%|█████     | 50885656/100000000 [00:13<00:13, 3709235.81tokens/s]
Shard 20:  51%|█████▏    | 51261713/100000000 [00:13<00:13, 3666649.78tokens/s]
Shard 20:  52%|█████▏    | 51640213/100000000 [00:13<00:13, 3538513.75tokens/s]
Shard 20:  52%|█████▏    | 52014309/100000000 [00:13<00:13, 3554115.84tokens/s]
Shard 20:  52%|█████▏    | 52408383/100000000 [00:14<00:13, 3624906.10tokens/s]
Shard 20:  53%|█████▎    | 52950828/100000000 [00:14<00:11, 4124664.93tokens/s]
Shard 20:  53%|█████▎    | 53411097/100000000 [00:14<00:10, 4255007.82tokens/s]
Shard 20:  54%|█████▍    | 53839778/100000000 [00:14<00:11, 4003649.42tokens/s]
Shard 20:  54%|█████▍    | 54283510/100000000 [00:14<00:11, 4080352.61tokens/s]
Shard 20:  55%|█████▍    | 54695206/100000000 [00:14<00:12, 3680950.08tokens/s]
Shard 20:  55%|█████▌    | 55125270/100000000 [00:14<00:13, 3350040.93tokens/s]
Shard 20:  56%|█████▌    | 55564463/100000000 [00:14<00:12, 3586859.46tokens/s]
Shard 20:  56%|█████▌    | 55935891/100000000 [00:14<00:13, 3294319.61tokens/s]
Shard 20:  56%|█████▋    | 56276826/100000000 [00:15<00:13, 3287911.85tokens/s]
Shard 20:  57%|█████▋    | 56613960/100000000 [00:15<00:14, 3050700.52tokens/s]
Shard 20:  57%|█████▋    | 56929596/100000000 [00:15<00:14, 3050126.22tokens/s]
Shard 20:  57%|█████▋    | 57254309/100000000 [00:15<00:13, 3081498.90tokens/s]
Shard 20:  58%|█████▊    | 57566327/100000000 [00:15<00:14, 2946318.81tokens/s]
Shard 20:  58%|█████▊    | 58083433/100000000 [00:15<00:11, 3541197.98tokens/s]
Shard 20:  58%|█████▊    | 58445520/100000000 [00:15<00:12, 3397346.29tokens/s]
Shard 20:  59%|█████▉    | 58793805/100000000 [00:15<00:12, 3170262.66tokens/s]
Shard 20:  59%|█████▉    | 59117303/100000000 [00:16<00:14, 2904702.58tokens/s]
Shard 20:  59%|█████▉    | 59436502/100000000 [00:16<00:13, 2966011.55tokens/s]
Shard 20:  60%|█████▉    | 59740304/100000000 [00:16<00:14, 2704226.23tokens/s]
Shard 20:  60%|██████    | 60075301/100000000 [00:16<00:13, 2867314.48tokens/s]
Shard 20:  61%|██████    | 60578329/100000000 [00:16<00:11, 3424819.85tokens/s]
Shard 20:  61%|██████    | 60998832/100000000 [00:16<00:10, 3625053.26tokens/s]
Shard 20:  61%|██████▏   | 61370691/100000000 [00:16<00:11, 3219221.31tokens/s]
Shard 20:  62%|██████▏   | 62014075/100000000 [00:16<00:09, 4059546.29tokens/s]
Shard 20:  62%|██████▏   | 62442448/100000000 [00:16<00:09, 4118372.45tokens/s]
Shard 20:  63%|██████▎   | 62874192/100000000 [00:17<00:11, 3293547.91tokens/s]
Shard 20:  63%|██████▎   | 63239688/100000000 [00:17<00:13, 2754743.30tokens/s]
Shard 20:  64%|██████▎   | 63593021/100000000 [00:17<00:12, 2903158.60tokens/s]
Shard 20:  64%|██████▍   | 63919678/100000000 [00:17<00:12, 2980692.94tokens/s]
Shard 20:  64%|██████▍   | 64241283/100000000 [00:17<00:12, 2974273.08tokens/s]
Shard 20:  65%|██████▍   | 64559507/100000000 [00:17<00:11, 2956873.49tokens/s]
Shard 20:  65%|██████▍   | 64870792/100000000 [00:17<00:11, 2988620.32tokens/s]
Shard 20:  65%|██████▌   | 65178843/100000000 [00:17<00:11, 2955707.34tokens/s]
Shard 20:  66%|██████▌   | 65579461/100000000 [00:18<00:10, 3245713.38tokens/s]
Shard 20:  66%|██████▌   | 65967832/100000000 [00:18<00:09, 3425356.65tokens/s]
Shard 20:  66%|██████▋   | 66316761/100000000 [00:18<00:10, 3331089.25tokens/s]
Shard 20:  67%|██████▋   | 66653956/100000000 [00:18<00:10, 3274901.00tokens/s]
Shard 20:  67%|██████▋   | 66984488/100000000 [00:18<00:10, 3045921.95tokens/s]
Shard 20:  67%|██████▋   | 67410282/100000000 [00:18<00:09, 3370938.01tokens/s]
Shard 20:  68%|██████▊   | 67904978/100000000 [00:18<00:08, 3808502.05tokens/s]
Shard 20:  68%|██████▊   | 68457506/100000000 [00:18<00:07, 4277088.89tokens/s]
Shard 20:  69%|██████▉   | 68892090/100000000 [00:18<00:07, 4129720.96tokens/s]
Shard 20:  69%|██████▉   | 69369947/100000000 [00:18<00:07, 4308985.39tokens/s]
Shard 20:  70%|██████▉   | 69947658/100000000 [00:19<00:06, 4725815.30tokens/s]
Shard 20:  70%|███████   | 70458096/100000000 [00:19<00:06, 4825720.41tokens/s]
Shard 20:  71%|███████   | 70944697/100000000 [00:19<00:06, 4411705.00tokens/s]
Shard 20:  71%|███████▏  | 71394878/100000000 [00:19<00:06, 4253991.47tokens/s]
Shard 20:  72%|███████▏  | 71827534/100000000 [00:19<00:07, 3960508.58tokens/s]
Shard 20:  72%|███████▏  | 72238229/100000000 [00:19<00:07, 3530076.92tokens/s]
Shard 20:  73%|███████▎  | 72602739/100000000 [00:19<00:07, 3432574.86tokens/s]
Shard 20:  73%|███████▎  | 72973449/100000000 [00:19<00:07, 3497631.71tokens/s]
Shard 20:  73%|███████▎  | 73330067/100000000 [00:20<00:07, 3397623.25tokens/s]
Shard 20:  74%|███████▎  | 73675994/100000000 [00:20<00:07, 3291895.05tokens/s]
Shard 20:  74%|███████▍  | 74023903/100000000 [00:20<00:07, 3332290.20tokens/s]
Shard 20:  74%|███████▍  | 74359483/100000000 [00:20<00:07, 3319251.37tokens/s]
Shard 20:  75%|███████▍  | 74925667/100000000 [00:20<00:06, 3963335.11tokens/s]
Shard 20:  75%|███████▌  | 75326422/100000000 [00:20<00:06, 3817370.06tokens/s]
Shard 20:  76%|███████▌  | 75734327/100000000 [00:20<00:06, 3885082.18tokens/s]
Shard 20:  76%|███████▌  | 76125754/100000000 [00:20<00:06, 3856253.76tokens/s]
Shard 20:  77%|███████▋  | 76582121/100000000 [00:20<00:05, 4040963.33tokens/s]
Shard 20:  77%|███████▋  | 77145708/100000000 [00:20<00:05, 4494442.66tokens/s]
Shard 20:  78%|███████▊  | 77626673/100000000 [00:21<00:04, 4583000.66tokens/s]
Shard 20:  78%|███████▊  | 78121803/100000000 [00:21<00:04, 4646067.55tokens/s]
Shard 20:  79%|███████▊  | 78587813/100000000 [00:21<00:04, 4483879.89tokens/s]
Shard 20:  79%|███████▉  | 79047071/100000000 [00:21<00:04, 4261360.41tokens/s]
Shard 20:  80%|███████▉  | 79509308/100000000 [00:21<00:04, 4351912.81tokens/s]
Shard 20:  80%|███████▉  | 79947909/100000000 [00:21<00:05, 3762152.97tokens/s]
Shard 20:  80%|████████  | 80343044/100000000 [00:21<00:05, 3808076.82tokens/s]
Shard 20:  81%|████████  | 80735955/100000000 [00:21<00:05, 3628330.66tokens/s]
Shard 20:  81%|████████  | 81125910/100000000 [00:22<00:05, 3687362.51tokens/s]
Shard 20:  82%|████████▏ | 81524282/100000000 [00:22<00:04, 3769105.59tokens/s]
Shard 20:  82%|████████▏ | 82033931/100000000 [00:22<00:04, 4144230.40tokens/s]
Shard 20:  82%|████████▏ | 82454622/100000000 [00:22<00:04, 4100961.02tokens/s]
Shard 20:  83%|████████▎ | 82868955/100000000 [00:22<00:04, 3692017.57tokens/s]
Shard 20:  83%|████████▎ | 83248620/100000000 [00:22<00:05, 3347697.29tokens/s]
Shard 20:  84%|████████▎ | 83671135/100000000 [00:22<00:04, 3566069.22tokens/s]
Shard 20:  84%|████████▍ | 84227195/100000000 [00:22<00:03, 4089773.10tokens/s]
Shard 20:  85%|████████▍ | 84651711/100000000 [00:22<00:04, 3646898.30tokens/s]
Shard 20:  85%|████████▌ | 85033828/100000000 [00:23<00:04, 3533681.82tokens/s]
Shard 20:  85%|████████▌ | 85399143/100000000 [00:23<00:04, 3396366.32tokens/s]
Shard 20:  86%|████████▌ | 85747700/100000000 [00:23<00:04, 3316914.40tokens/s]
Shard 20:  86%|████████▌ | 86161098/100000000 [00:23<00:03, 3532795.66tokens/s]
Shard 20:  87%|████████▋ | 86658694/100000000 [00:23<00:03, 3916807.76tokens/s]
Shard 20:  87%|████████▋ | 87086954/100000000 [00:23<00:03, 4012428.67tokens/s]
Shard 20:  88%|████████▊ | 87563621/100000000 [00:23<00:02, 4217088.88tokens/s]
Shard 20:  88%|████████▊ | 87991461/100000000 [00:23<00:02, 4032121.95tokens/s]
Shard 20:  88%|████████▊ | 88417257/100000000 [00:23<00:02, 4025544.05tokens/s]
Shard 20:  89%|████████▉ | 88878698/100000000 [00:24<00:02, 4182039.01tokens/s]
Shard 20:  89%|████████▉ | 89303678/100000000 [00:24<00:02, 4200141.30tokens/s]
Shard 20:  90%|████████▉ | 89777298/100000000 [00:24<00:02, 4300227.30tokens/s]
Shard 20:  90%|█████████ | 90324502/100000000 [00:24<00:02, 4641616.55tokens/s]
Shard 20:  91%|█████████ | 90791125/100000000 [00:24<00:02, 4312087.30tokens/s]
Shard 20:  91%|█████████ | 91228392/100000000 [00:24<00:02, 4029722.95tokens/s]
Shard 20:  92%|█████████▏| 91638259/100000000 [00:24<00:02, 3943566.42tokens/s]
Shard 20:  92%|█████████▏| 92037651/100000000 [00:24<00:02, 3729314.50tokens/s]
Shard 20:  93%|█████████▎| 92563583/100000000 [00:24<00:01, 4105689.25tokens/s]
Shard 20:  93%|█████████▎| 93153231/100000000 [00:25<00:01, 4579273.75tokens/s]
Shard 20:  94%|█████████▎| 93619353/100000000 [00:25<00:01, 4244738.68tokens/s]
Shard 20:  94%|█████████▍| 94053310/100000000 [00:25<00:01, 4158781.17tokens/s]
Shard 20:  94%|█████████▍| 94475459/100000000 [00:25<00:01, 4027764.20tokens/s]
Shard 20:  95%|█████████▍| 94883009/100000000 [00:25<00:01, 3816031.69tokens/s]
Shard 20:  95%|█████████▌| 95269303/100000000 [00:25<00:01, 3730140.04tokens/s]
Shard 20:  96%|█████████▌| 95675477/100000000 [00:25<00:01, 3813667.00tokens/s]
Shard 20:  96%|█████████▌| 96170599/100000000 [00:25<00:00, 4122107.85tokens/s]
Shard 20:  97%|█████████▋| 96631475/100000000 [00:25<00:00, 4258101.93tokens/s]
Shard 20:  97%|█████████▋| 97061308/100000000 [00:26<00:00, 4145520.54tokens/s]
Shard 20:  97%|█████████▋| 97479449/100000000 [00:26<00:00, 4026491.43tokens/s]
Shard 20:  98%|█████████▊| 97890730/100000000 [00:26<00:00, 3901795.28tokens/s]
Shard 20:  98%|█████████▊| 98283589/100000000 [00:26<00:00, 3620485.67tokens/s]
Shard 20:  99%|█████████▊| 98651411/100000000 [00:26<00:00, 3399072.83tokens/s]
Shard 20:  99%|█████████▉| 98995732/100000000 [00:26<00:00, 3386156.14tokens/s]
Shard 20:  99%|█████████▉| 99356574/100000000 [00:26<00:00, 3439925.93tokens/s]
Shard 20: 100%|█████████▉| 99704712/100000000 [00:26<00:00, 3360965.76tokens/s]
Shard 20: 100%|█████████▉| 99999036/100000000 [00:35<00:00, 2784220.47tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000020.bin (200,001,024 bytes) in the gpt-2 format

Shard 21:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 21:   0%|          | 193182/100000000 [00:00<00:51, 1931715.91tokens/s]
Shard 21:   0%|          | 482253/100000000 [00:00<00:40, 2436636.20tokens/s]
Shard 21:   1%|          | 740962/100000000 [00:00<00:40, 2445796.62tokens/s]
Shard 21:   1%|          | 985110/100000000 [00:00<00:42, 2328757.62tokens/s]
Shard 21:   1%|▏         | 1321084/100000000 [00:00<00:36, 2675385.77tokens/s]
Shard 21:   2%|▏         | 1757455/100000000 [00:00<00:30, 3196001.46tokens/s]
Shard 21:   2%|▏         | 2079564/100000000 [00:00<00:32, 3031907.95tokens/s]
Shard 21:   2%|▏         | 2430551/100000000 [00:00<00:31, 3133287.50tokens/s]
Shard 21:   3%|▎         | 2774781/100000000 [00:00<00:30, 3221990.75tokens/s]
Shard 21:   3%|▎         | 3104770/100000000 [00:01<00:29, 3241955.10tokens/s]
Shard 21:   3%|▎         | 3476211/100000000 [00:01<00:28, 3374489.04tokens/s]
Shard 21:   4%|▍         | 3815496/100000000 [00:01<00:32, 2984043.75tokens/s]
Shard 21:   4%|▍         | 4172943/100000000 [00:01<00:30, 3128208.44tokens/s]
Shard 21:   4%|▍         | 4493285/100000000 [00:01<00:31, 3003753.79tokens/s]
Shard 21:   5%|▍         | 4799428/100000000 [00:01<00:31, 2988963.70tokens/s]
Shard 21:   5%|▌         | 5102758/100000000 [00:01<00:33, 2859514.52tokens/s]
Shard 21:   5%|▌         | 5391991/100000000 [00:01<00:35, 2646211.08tokens/s]
Shard 21:   6%|▌         | 5699911/100000000 [00:01<00:34, 2756955.50tokens/s]
Shard 21:   6%|▌         | 5979962/100000000 [00:02<00:37, 2509521.28tokens/s]
Shard 21:   6%|▋         | 6327854/100000000 [00:02<00:34, 2737879.12tokens/s]
Shard 21:   7%|▋         | 6743125/100000000 [00:02<00:30, 3089766.01tokens/s]
Shard 21:   7%|▋         | 7065778/100000000 [00:02<00:29, 3112589.12tokens/s]
Shard 21:   7%|▋         | 7382930/100000000 [00:02<00:29, 3129136.14tokens/s]
Shard 21:   8%|▊         | 7700314/100000000 [00:02<00:30, 3057219.58tokens/s]
Shard 21:   8%|▊         | 8010060/100000000 [00:02<00:30, 2991776.11tokens/s]
Shard 21:   8%|▊         | 8320764/100000000 [00:02<00:30, 3018500.99tokens/s]
Shard 21:   9%|▊         | 8672629/100000000 [00:02<00:28, 3162830.64tokens/s]
Shard 21:   9%|▉         | 9013745/100000000 [00:03<00:28, 3229533.44tokens/s]
Shard 21:   9%|▉         | 9338188/100000000 [00:03<00:29, 3054658.07tokens/s]
Shard 21:  10%|▉         | 9688500/100000000 [00:03<00:28, 3160206.14tokens/s]
Shard 21:  10%|█         | 10016179/100000000 [00:03<00:28, 3185160.89tokens/s]
Shard 21:  10%|█         | 10336581/100000000 [00:03<00:32, 2741515.15tokens/s]
Shard 21:  11%|█         | 10624075/100000000 [00:03<00:34, 2572008.36tokens/s]
Shard 21:  11%|█         | 10890886/100000000 [00:03<00:37, 2388136.48tokens/s]
Shard 21:  11%|█         | 11138758/100000000 [00:03<00:41, 2122282.45tokens/s]
Shard 21:  11%|█▏        | 11360732/100000000 [00:04<00:42, 2100967.29tokens/s]
Shard 21:  12%|█▏        | 11588117/100000000 [00:04<00:41, 2144238.05tokens/s]
Shard 21:  12%|█▏        | 11893832/100000000 [00:04<00:36, 2383092.57tokens/s]
Shard 21:  12%|█▏        | 12168717/100000000 [00:04<00:35, 2475897.47tokens/s]
Shard 21:  12%|█▏        | 12424947/100000000 [00:04<00:36, 2415961.64tokens/s]
Shard 21:  13%|█▎        | 12670382/100000000 [00:04<00:36, 2421355.38tokens/s]
Shard 21:  13%|█▎        | 12969937/100000000 [00:04<00:33, 2578991.18tokens/s]
Shard 21:  13%|█▎        | 13310251/100000000 [00:04<00:31, 2763512.62tokens/s]
Shard 21:  14%|█▎        | 13588400/100000000 [00:04<00:32, 2635363.28tokens/s]
Shard 21:  14%|█▍        | 13911459/100000000 [00:04<00:30, 2778126.04tokens/s]
Shard 21:  14%|█▍        | 14255221/100000000 [00:05<00:28, 2963746.74tokens/s]
Shard 21:  15%|█▍        | 14697297/100000000 [00:05<00:25, 3362151.42tokens/s]
Shard 21:  15%|█▌        | 15036970/100000000 [00:05<00:27, 3065363.84tokens/s]
Shard 21:  15%|█▌        | 15350342/100000000 [00:05<00:32, 2579869.03tokens/s]
Shard 21:  16%|█▌        | 15625058/100000000 [00:05<00:33, 2487561.49tokens/s]
Shard 21:  16%|█▌        | 15885210/100000000 [00:05<00:35, 2378716.64tokens/s]
Shard 21:  16%|█▌        | 16131956/100000000 [00:05<00:35, 2357734.00tokens/s]
Shard 21:  16%|█▋        | 16373077/100000000 [00:05<00:36, 2264010.53tokens/s]
Shard 21:  17%|█▋        | 16717204/100000000 [00:06<00:32, 2569700.23tokens/s]
Shard 21:  17%|█▋        | 17005307/100000000 [00:06<00:31, 2652054.29tokens/s]
Shard 21:  17%|█▋        | 17440750/100000000 [00:06<00:26, 3079455.80tokens/s]
Shard 21:  18%|█▊        | 17753713/100000000 [00:06<00:29, 2802508.90tokens/s]
Shard 21:  18%|█▊        | 18044848/100000000 [00:06<00:29, 2757363.11tokens/s]
Shard 21:  18%|█▊        | 18325818/100000000 [00:06<00:30, 2716421.28tokens/s]
Shard 21:  19%|█▊        | 18655581/100000000 [00:06<00:28, 2871706.02tokens/s]
Shard 21:  19%|█▉        | 19076171/100000000 [00:06<00:25, 3235529.34tokens/s]
Shard 21:  19%|█▉        | 19404291/100000000 [00:06<00:26, 3033433.64tokens/s]
Shard 21:  20%|█▉        | 19787836/100000000 [00:07<00:24, 3246516.93tokens/s]
Shard 21:  20%|██        | 20306618/100000000 [00:07<00:21, 3791880.00tokens/s]
Shard 21:  21%|██        | 20736616/100000000 [00:07<00:20, 3931995.22tokens/s]
Shard 21:  21%|██        | 21135280/100000000 [00:07<00:20, 3859166.99tokens/s]
Shard 21:  22%|██▏       | 21525637/100000000 [00:07<00:21, 3680800.39tokens/s]
Shard 21:  22%|██▏       | 22015170/100000000 [00:07<00:19, 4013324.97tokens/s]
Shard 21:  22%|██▏       | 22433266/100000000 [00:07<00:19, 4047789.77tokens/s]
Shard 21:  23%|██▎       | 22841977/100000000 [00:07<00:22, 3437649.14tokens/s]
Shard 21:  23%|██▎       | 23205679/100000000 [00:07<00:25, 3067690.65tokens/s]
Shard 21:  24%|██▎       | 23538551/100000000 [00:08<00:24, 3118146.85tokens/s]
Shard 21:  24%|██▍       | 23883732/100000000 [00:08<00:23, 3197422.06tokens/s]
Shard 21:  24%|██▍       | 24275965/100000000 [00:08<00:22, 3323251.37tokens/s]
Shard 21:  25%|██▍       | 24747276/100000000 [00:08<00:20, 3696753.05tokens/s]
Shard 21:  25%|██▌       | 25126676/100000000 [00:08<00:21, 3413681.14tokens/s]
Shard 21:  25%|██▌       | 25495831/100000000 [00:08<00:21, 3480423.13tokens/s]
Shard 21:  26%|██▌       | 25852467/100000000 [00:08<00:23, 3214482.40tokens/s]
Shard 21:  26%|██▌       | 26182925/100000000 [00:08<00:24, 3055580.32tokens/s]
Shard 21:  26%|██▋       | 26494746/100000000 [00:09<00:25, 2935064.90tokens/s]
Shard 21:  27%|██▋       | 26792600/100000000 [00:09<00:25, 2911685.28tokens/s]
Shard 21:  27%|██▋       | 27286249/100000000 [00:09<00:21, 3447062.71tokens/s]
Shard 21:  28%|██▊       | 27809035/100000000 [00:09<00:18, 3920957.54tokens/s]
Shard 21:  28%|██▊       | 28210662/100000000 [00:09<00:18, 3933086.58tokens/s]
Shard 21:  29%|██▊       | 28609233/100000000 [00:09<00:19, 3587614.62tokens/s]
Shard 21:  29%|██▉       | 28976881/100000000 [00:09<00:21, 3282677.92tokens/s]
Shard 21:  29%|██▉       | 29469605/100000000 [00:09<00:19, 3707506.06tokens/s]
Shard 21:  30%|██▉       | 29853799/100000000 [00:09<00:19, 3686636.43tokens/s]
Shard 21:  30%|███       | 30245248/100000000 [00:10<00:18, 3722128.31tokens/s]
Shard 21:  31%|███       | 30649918/100000000 [00:10<00:18, 3811407.95tokens/s]
Shard 21:  31%|███       | 31039099/100000000 [00:10<00:22, 3090927.04tokens/s]
Shard 21:  31%|███▏      | 31373013/100000000 [00:10<00:24, 2746758.51tokens/s]
Shard 21:  32%|███▏      | 31670172/100000000 [00:10<00:25, 2671262.58tokens/s]
Shard 21:  32%|███▏      | 31952361/100000000 [00:10<00:25, 2692616.38tokens/s]
Shard 21:  32%|███▏      | 32473280/100000000 [00:10<00:20, 3338653.69tokens/s]
Shard 21:  33%|███▎      | 33066583/100000000 [00:10<00:16, 4029991.75tokens/s]
Shard 21:  33%|███▎      | 33490243/100000000 [00:10<00:17, 3893967.24tokens/s]
Shard 21:  34%|███▍      | 33907006/100000000 [00:11<00:18, 3653632.94tokens/s]
Shard 21:  34%|███▍      | 34285384/100000000 [00:11<00:18, 3630891.39tokens/s]
Shard 21:  35%|███▍      | 34701898/100000000 [00:11<00:17, 3737380.37tokens/s]
Shard 21:  35%|███▌      | 35277774/100000000 [00:11<00:15, 4295591.93tokens/s]
Shard 21:  36%|███▌      | 35717591/100000000 [00:11<00:15, 4233795.95tokens/s]
Shard 21:  36%|███▌      | 36150748/100000000 [00:11<00:17, 3648951.73tokens/s]
Shard 21:  37%|███▋      | 36534104/100000000 [00:11<00:17, 3597930.36tokens/s]
Shard 21:  37%|███▋      | 36942221/100000000 [00:11<00:16, 3711974.36tokens/s]
Shard 21:  37%|███▋      | 37323351/100000000 [00:12<00:16, 3738050.18tokens/s]
Shard 21:  38%|███▊      | 37712570/100000000 [00:12<00:16, 3677053.77tokens/s]
Shard 21:  38%|███▊      | 38086189/100000000 [00:12<00:17, 3556252.68tokens/s]
Shard 21:  38%|███▊      | 38445931/100000000 [00:12<00:18, 3383344.74tokens/s]
Shard 21:  39%|███▉      | 38977297/100000000 [00:12<00:15, 3874131.61tokens/s]
Shard 21:  39%|███▉      | 39370850/100000000 [00:12<00:16, 3654593.68tokens/s]
Shard 21:  40%|███▉      | 39806130/100000000 [00:12<00:15, 3843770.62tokens/s]
Shard 21:  40%|████      | 40196535/100000000 [00:12<00:16, 3659738.85tokens/s]
Shard 21:  41%|████      | 40572203/100000000 [00:12<00:16, 3664815.01tokens/s]
Shard 21:  41%|████      | 41007578/100000000 [00:12<00:15, 3848378.54tokens/s]
Shard 21:  41%|████▏     | 41396835/100000000 [00:13<00:16, 3567458.23tokens/s]
Shard 21:  42%|████▏     | 41794455/100000000 [00:13<00:15, 3670506.52tokens/s]
Shard 21:  42%|████▏     | 42316463/100000000 [00:13<00:14, 4094898.99tokens/s]
Shard 21:  43%|████▎     | 42824835/100000000 [00:13<00:13, 4366447.38tokens/s]
Shard 21:  43%|████▎     | 43267805/100000000 [00:13<00:13, 4269883.85tokens/s]
Shard 21:  44%|████▍     | 43984802/100000000 [00:13<00:10, 5095984.50tokens/s]
Shard 21:  45%|████▍     | 44500806/100000000 [00:13<00:11, 4719209.86tokens/s]
Shard 21:  45%|████▍     | 44982255/100000000 [00:13<00:13, 4060122.69tokens/s]
Shard 21:  45%|████▌     | 45409956/100000000 [00:14<00:15, 3436986.89tokens/s]
Shard 21:  46%|████▌     | 45782675/100000000 [00:14<00:17, 3137124.44tokens/s]
Shard 21:  46%|████▌     | 46116738/100000000 [00:14<00:18, 2947673.03tokens/s]
Shard 21:  46%|████▋     | 46447528/100000000 [00:14<00:17, 3027377.48tokens/s]
Shard 21:  47%|████▋     | 46763041/100000000 [00:14<00:18, 2934014.98tokens/s]
Shard 21:  47%|████▋     | 47064193/100000000 [00:14<00:18, 2933356.27tokens/s]
Shard 21:  47%|████▋     | 47362736/100000000 [00:14<00:18, 2918682.54tokens/s]
Shard 21:  48%|████▊     | 47776919/100000000 [00:14<00:16, 3242649.14tokens/s]
Shard 21:  48%|████▊     | 48106576/100000000 [00:15<00:16, 3098440.92tokens/s]
Shard 21:  48%|████▊     | 48420936/100000000 [00:15<00:19, 2684141.45tokens/s]
Shard 21:  49%|████▉     | 48772039/100000000 [00:15<00:17, 2888965.60tokens/s]
Shard 21:  49%|████▉     | 49072642/100000000 [00:15<00:18, 2733528.91tokens/s]
Shard 21:  50%|████▉     | 49590937/100000000 [00:15<00:14, 3375250.10tokens/s]
Shard 21:  50%|████▉     | 49944797/100000000 [00:15<00:14, 3363712.40tokens/s]
Shard 21:  50%|█████     | 50291971/100000000 [00:15<00:16, 2994308.73tokens/s]
Shard 21:  51%|█████     | 50607682/100000000 [00:15<00:16, 3018044.74tokens/s]
Shard 21:  51%|█████     | 50919741/100000000 [00:15<00:16, 2958475.53tokens/s]
Shard 21:  51%|█████     | 51224664/100000000 [00:16<00:17, 2738534.97tokens/s]
Shard 21:  52%|█████▏    | 51641718/100000000 [00:16<00:15, 3113227.79tokens/s]
Shard 21:  52%|█████▏    | 52017518/100000000 [00:16<00:14, 3286743.57tokens/s]
Shard 21:  52%|█████▏    | 52355186/100000000 [00:16<00:14, 3302463.54tokens/s]
Shard 21:  53%|█████▎    | 52721002/100000000 [00:16<00:14, 3361021.65tokens/s]
Shard 21:  53%|█████▎    | 53071194/100000000 [00:16<00:15, 3115216.67tokens/s]
Shard 21:  53%|█████▎    | 53395935/100000000 [00:16<00:14, 3147909.74tokens/s]
Shard 21:  54%|█████▎    | 53716117/100000000 [00:16<00:15, 3026693.33tokens/s]
Shard 21:  54%|█████▍    | 54060842/100000000 [00:16<00:14, 3136366.81tokens/s]
Shard 21:  55%|█████▍    | 54559876/100000000 [00:17<00:12, 3612848.59tokens/s]
Shard 21:  55%|█████▍    | 54925113/100000000 [00:17<00:12, 3483069.70tokens/s]
Shard 21:  55%|█████▌    | 55367706/100000000 [00:17<00:11, 3748138.28tokens/s]
Shard 21:  56%|█████▌    | 55747729/100000000 [00:17<00:12, 3593194.48tokens/s]
Shard 21:  56%|█████▌    | 56111529/100000000 [00:17<00:13, 3219207.37tokens/s]
Shard 21:  57%|█████▋    | 56601747/100000000 [00:17<00:11, 3647704.32tokens/s]
Shard 21:  57%|█████▋    | 56978418/100000000 [00:17<00:12, 3442298.76tokens/s]
Shard 21:  57%|█████▋    | 57332084/100000000 [00:17<00:12, 3384371.93tokens/s]
Shard 21:  58%|█████▊    | 57676954/100000000 [00:17<00:12, 3353392.90tokens/s]
Shard 21:  58%|█████▊    | 58017058/100000000 [00:18<00:12, 3250561.62tokens/s]
Shard 21:  58%|█████▊    | 58345682/100000000 [00:18<00:13, 3157949.84tokens/s]
Shard 21:  59%|█████▊    | 58664070/100000000 [00:18<00:13, 3070446.94tokens/s]
Shard 21:  59%|█████▉    | 59154559/100000000 [00:18<00:11, 3577038.71tokens/s]
Shard 21:  60%|█████▉    | 59661041/100000000 [00:18<00:10, 3992767.06tokens/s]
Shard 21:  60%|██████    | 60162409/100000000 [00:18<00:09, 4280442.13tokens/s]
Shard 21:  61%|██████    | 60629692/100000000 [00:18<00:08, 4377286.91tokens/s]
Shard 21:  61%|██████    | 61073359/100000000 [00:18<00:09, 3923567.18tokens/s]
Shard 21:  61%|██████▏   | 61477898/100000000 [00:18<00:10, 3729826.84tokens/s]
Shard 21:  62%|██████▏   | 61859679/100000000 [00:19<00:11, 3182866.88tokens/s]
Shard 21:  62%|██████▏   | 62195612/100000000 [00:19<00:12, 3055641.03tokens/s]
Shard 21:  63%|██████▎   | 62515385/100000000 [00:19<00:14, 2673387.72tokens/s]
Shard 21:  63%|██████▎   | 62964844/100000000 [00:19<00:12, 3030124.11tokens/s]
Shard 21:  63%|██████▎   | 63284050/100000000 [00:19<00:12, 3052135.69tokens/s]
Shard 21:  64%|██████▎   | 63602069/100000000 [00:19<00:12, 2902002.90tokens/s]
Shard 21:  64%|██████▍   | 63906590/100000000 [00:19<00:12, 2938655.18tokens/s]
Shard 21:  64%|██████▍   | 64207359/100000000 [00:19<00:12, 2926189.63tokens/s]
Shard 21:  65%|██████▍   | 64643910/100000000 [00:20<00:10, 3315454.02tokens/s]
Shard 21:  65%|██████▌   | 65040764/100000000 [00:20<00:10, 3493760.99tokens/s]
Shard 21:  65%|██████▌   | 65415123/100000000 [00:20<00:09, 3554203.91tokens/s]
Shard 21:  66%|██████▌   | 65775424/100000000 [00:20<00:10, 3253467.98tokens/s]
Shard 21:  66%|██████▌   | 66151891/100000000 [00:20<00:10, 3382757.17tokens/s]
Shard 21:  67%|██████▋   | 66591558/100000000 [00:20<00:09, 3663558.75tokens/s]
Shard 21:  67%|██████▋   | 66964224/100000000 [00:20<00:09, 3571713.63tokens/s]
Shard 21:  67%|██████▋   | 67326246/100000000 [00:20<00:10, 3184028.48tokens/s]
Shard 21:  68%|██████▊   | 68467523/100000000 [00:20<00:05, 5302310.68tokens/s]
Shard 21:  69%|██████▉   | 69025728/100000000 [00:21<00:06, 5053778.53tokens/s]
Shard 21:  70%|██████▉   | 69552015/100000000 [00:21<00:06, 4904783.70tokens/s]
Shard 21:  70%|███████   | 70072383/100000000 [00:21<00:06, 4922563.26tokens/s]
Shard 21:  71%|███████   | 70597161/100000000 [00:21<00:05, 5008231.95tokens/s]
Shard 21:  71%|███████   | 71106144/100000000 [00:21<00:05, 4932844.33tokens/s]
Shard 21:  72%|███████▏  | 71605598/100000000 [00:21<00:06, 4579393.72tokens/s]
Shard 21:  72%|███████▏  | 72071295/100000000 [00:21<00:06, 4502191.94tokens/s]
Shard 21:  73%|███████▎  | 72546294/100000000 [00:21<00:06, 4254751.30tokens/s]
Shard 21:  73%|███████▎  | 72977436/100000000 [00:22<00:06, 4233988.07tokens/s]
Shard 21:  73%|███████▎  | 73404497/100000000 [00:22<00:06, 4055440.73tokens/s]
Shard 21:  74%|███████▍  | 73822881/100000000 [00:22<00:07, 3574234.32tokens/s]
Shard 21:  74%|███████▍  | 74201013/100000000 [00:22<00:07, 3616966.66tokens/s]
Shard 21:  75%|███████▍  | 74698660/100000000 [00:22<00:06, 3977324.61tokens/s]
Shard 21:  75%|███████▌  | 75118953/100000000 [00:22<00:06, 3890248.52tokens/s]
Shard 21:  76%|███████▌  | 75578859/100000000 [00:22<00:05, 4072448.25tokens/s]
Shard 21:  76%|███████▌  | 75993787/100000000 [00:22<00:06, 3915485.28tokens/s]
Shard 21:  76%|███████▋  | 76390649/100000000 [00:22<00:06, 3636979.69tokens/s]
Shard 21:  77%|███████▋  | 76788316/100000000 [00:23<00:06, 3724011.13tokens/s]
Shard 21:  77%|███████▋  | 77282190/100000000 [00:23<00:05, 4044802.99tokens/s]
Shard 21:  78%|███████▊  | 77902796/100000000 [00:23<00:04, 4648872.15tokens/s]
Shard 21:  78%|███████▊  | 78375839/100000000 [00:23<00:04, 4572574.30tokens/s]
Shard 21:  79%|███████▉  | 78876530/100000000 [00:23<00:04, 4682399.63tokens/s]
Shard 21:  79%|███████▉  | 79349346/100000000 [00:23<00:04, 4553750.12tokens/s]
Shard 21:  80%|███████▉  | 79809602/100000000 [00:23<00:04, 4224826.19tokens/s]
Shard 21:  80%|████████  | 80239154/100000000 [00:23<00:04, 4240994.98tokens/s]
Shard 21:  81%|████████  | 80767074/100000000 [00:23<00:04, 4531960.38tokens/s]
Shard 21:  81%|████████  | 81225913/100000000 [00:23<00:04, 4508614.37tokens/s]
Shard 21:  82%|████████▏ | 81681261/100000000 [00:24<00:04, 4452993.95tokens/s]
Shard 21:  82%|████████▏ | 82130863/100000000 [00:24<00:04, 3739423.35tokens/s]
Shard 21:  83%|████████▎ | 82737667/100000000 [00:24<00:03, 4327287.74tokens/s]
Shard 21:  83%|████████▎ | 83194172/100000000 [00:24<00:04, 4014071.12tokens/s]
Shard 21:  84%|████████▎ | 83617956/100000000 [00:24<00:04, 3437424.38tokens/s]
Shard 21:  84%|████████▍ | 83987189/100000000 [00:24<00:05, 3160524.91tokens/s]
Shard 21:  84%|████████▍ | 84360719/100000000 [00:24<00:04, 3295006.76tokens/s]
Shard 21:  85%|████████▍ | 84920386/100000000 [00:25<00:03, 3871469.00tokens/s]
Shard 21:  85%|████████▌ | 85330318/100000000 [00:25<00:03, 3901233.35tokens/s]
Shard 21:  86%|████████▌ | 85914995/100000000 [00:25<00:03, 4429110.36tokens/s]
Shard 21:  86%|████████▋ | 86375211/100000000 [00:25<00:03, 4013072.97tokens/s]
Shard 21:  87%|████████▋ | 86796217/100000000 [00:25<00:03, 3812576.89tokens/s]
Shard 21:  87%|████████▋ | 87193795/100000000 [00:25<00:03, 3855206.21tokens/s]
Shard 21:  88%|████████▊ | 87628618/100000000 [00:25<00:03, 3978896.60tokens/s]
Shard 21:  88%|████████▊ | 88054654/100000000 [00:25<00:03, 3962664.05tokens/s]
Shard 21:  88%|████████▊ | 88458654/100000000 [00:25<00:02, 3870581.37tokens/s]
Shard 21:  89%|████████▉ | 88855082/100000000 [00:26<00:02, 3887947.45tokens/s]
Shard 21:  89%|████████▉ | 89247134/100000000 [00:26<00:02, 3594445.48tokens/s]
Shard 21:  90%|████████▉ | 89614961/100000000 [00:26<00:03, 3217838.99tokens/s]
Shard 21:  90%|████████▉ | 89959124/100000000 [00:26<00:03, 3254329.47tokens/s]
Shard 21:  90%|█████████ | 90339413/100000000 [00:26<00:02, 3393844.50tokens/s]
Shard 21:  91%|█████████ | 90766919/100000000 [00:26<00:02, 3581631.05tokens/s]
Shard 21:  91%|█████████ | 91186238/100000000 [00:26<00:02, 3740632.49tokens/s]
Shard 21:  92%|█████████▏| 91567211/100000000 [00:26<00:02, 3629981.28tokens/s]
Shard 21:  92%|█████████▏| 91935165/100000000 [00:26<00:02, 3586943.54tokens/s]
Shard 21:  92%|█████████▏| 92351897/100000000 [00:27<00:02, 3745961.64tokens/s]
Shard 21:  93%|█████████▎| 92810397/100000000 [00:27<00:01, 3974481.32tokens/s]
Shard 21:  94%|█████████▎| 93615500/100000000 [00:27<00:01, 5129713.27tokens/s]
Shard 21:  94%|█████████▍| 94142481/100000000 [00:27<00:01, 5164308.02tokens/s]
Shard 21:  95%|█████████▍| 94661767/100000000 [00:27<00:01, 4642334.89tokens/s]
Shard 21:  95%|█████████▌| 95138765/100000000 [00:27<00:01, 4523217.06tokens/s]
Shard 21:  96%|█████████▌| 95599109/100000000 [00:27<00:01, 4153961.73tokens/s]
Shard 21:  96%|█████████▌| 96024219/100000000 [00:27<00:01, 3755722.55tokens/s]
Shard 21:  96%|█████████▋| 96418173/100000000 [00:27<00:00, 3782048.89tokens/s]
Shard 21:  97%|█████████▋| 96805138/100000000 [00:28<00:00, 3626839.14tokens/s]
Shard 21:  97%|█████████▋| 97174023/100000000 [00:28<00:00, 3007294.21tokens/s]
Shard 21:  97%|█████████▋| 97494642/100000000 [00:28<00:00, 2929636.01tokens/s]
Shard 21:  98%|█████████▊| 97800060/100000000 [00:28<00:00, 2681226.45tokens/s]
Shard 21:  98%|█████████▊| 98093692/100000000 [00:28<00:00, 2739483.17tokens/s]
Shard 21:  98%|█████████▊| 98449166/100000000 [00:28<00:00, 2947025.17tokens/s]
Shard 21:  99%|█████████▉| 98754290/100000000 [00:28<00:00, 2951908.83tokens/s]
Shard 21:  99%|█████████▉| 99182385/100000000 [00:28<00:00, 3310288.98tokens/s]
Shard 21: 100%|█████████▉| 99521787/100000000 [00:29<00:00, 3080877.06tokens/s]
Shard 21: 100%|█████████▉| 99858765/100000000 [00:29<00:00, 3137966.94tokens/s]
Shard 21: 100%|█████████▉| 99999818/100000000 [00:37<00:00, 2668762.33tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000021.bin (200,001,024 bytes) in the gpt-2 format

Shard 22:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 22:   0%|          | 345914/100000000 [00:00<00:29, 3430865.63tokens/s]
Shard 22:   1%|          | 690048/100000000 [00:00<00:36, 2693103.44tokens/s]
Shard 22:   1%|          | 969918/100000000 [00:00<00:36, 2717793.62tokens/s]
Shard 22:   1%|▏         | 1279313/100000000 [00:00<00:34, 2842144.80tokens/s]
Shard 22:   2%|▏         | 1581391/100000000 [00:00<00:33, 2900210.00tokens/s]
Shard 22:   2%|▏         | 1957541/100000000 [00:00<00:30, 3180917.29tokens/s]
Shard 22:   2%|▏         | 2419664/100000000 [00:00<00:26, 3627340.02tokens/s]
Shard 22:   3%|▎         | 2791524/100000000 [00:00<00:26, 3644314.97tokens/s]
Shard 22:   3%|▎         | 3165277/100000000 [00:01<00:31, 3110942.69tokens/s]
Shard 22:   4%|▎         | 3516757/100000000 [00:01<00:29, 3220122.87tokens/s]
Shard 22:   4%|▍         | 3850521/100000000 [00:01<00:30, 3133302.94tokens/s]
Shard 22:   4%|▍         | 4213152/100000000 [00:01<00:29, 3226908.72tokens/s]
Shard 22:   5%|▍         | 4542353/100000000 [00:01<00:29, 3185750.41tokens/s]
Shard 22:   5%|▍         | 4871954/100000000 [00:01<00:29, 3195824.85tokens/s]
Shard 22:   5%|▌         | 5231051/100000000 [00:01<00:28, 3291745.06tokens/s]
Shard 22:   6%|▌         | 5698367/100000000 [00:01<00:25, 3679046.95tokens/s]
Shard 22:   6%|▌         | 6069609/100000000 [00:01<00:28, 3322128.05tokens/s]
Shard 22:   7%|▋         | 6600337/100000000 [00:01<00:24, 3855170.60tokens/s]
Shard 22:   7%|▋         | 6996759/100000000 [00:02<00:27, 3350979.53tokens/s]
Shard 22:   7%|▋         | 7350143/100000000 [00:02<00:29, 3166605.26tokens/s]
Shard 22:   8%|▊         | 7737539/100000000 [00:02<00:27, 3325534.81tokens/s]
Shard 22:   8%|▊         | 8163207/100000000 [00:02<00:25, 3571447.75tokens/s]
Shard 22:   9%|▊         | 8707151/100000000 [00:02<00:22, 4069349.27tokens/s]
Shard 22:   9%|▉         | 9147617/100000000 [00:02<00:21, 4152375.37tokens/s]
Shard 22:  10%|▉         | 9578597/100000000 [00:02<00:24, 3701927.75tokens/s]
Shard 22:  10%|▉         | 9963719/100000000 [00:02<00:24, 3675810.75tokens/s]
Shard 22:  10%|█         | 10425519/100000000 [00:03<00:22, 3908116.78tokens/s]
Shard 22:  11%|█         | 10826883/100000000 [00:03<00:24, 3659635.57tokens/s]
Shard 22:  11%|█         | 11202042/100000000 [00:03<00:25, 3465161.80tokens/s]
Shard 22:  12%|█▏        | 11557133/100000000 [00:03<00:26, 3334194.97tokens/s]
Shard 22:  12%|█▏        | 11949632/100000000 [00:03<00:25, 3488995.65tokens/s]
Shard 22:  12%|█▏        | 12305720/100000000 [00:03<00:25, 3478952.74tokens/s]
Shard 22:  13%|█▎        | 12666357/100000000 [00:03<00:26, 3251093.03tokens/s]
Shard 22:  13%|█▎        | 13070162/100000000 [00:03<00:25, 3460025.85tokens/s]
Shard 22:  13%|█▎        | 13471486/100000000 [00:03<00:24, 3597308.25tokens/s]
Shard 22:  14%|█▍        | 13996139/100000000 [00:04<00:21, 4053992.86tokens/s]
Shard 22:  15%|█▍        | 14628795/100000000 [00:04<00:18, 4694485.53tokens/s]
Shard 22:  15%|█▌        | 15112223/100000000 [00:04<00:17, 4735054.05tokens/s]
Shard 22:  16%|█▌        | 15592260/100000000 [00:04<00:18, 4497179.55tokens/s]
Shard 22:  16%|█▌        | 16175774/100000000 [00:04<00:17, 4873206.57tokens/s]
Shard 22:  17%|█▋        | 16669664/100000000 [00:04<00:17, 4680986.29tokens/s]
Shard 22:  17%|█▋        | 17143302/100000000 [00:04<00:18, 4366262.79tokens/s]
Shard 22:  18%|█▊        | 17587400/100000000 [00:04<00:19, 4157360.63tokens/s]
Shard 22:  18%|█▊        | 18012114/100000000 [00:04<00:19, 4178112.48tokens/s]
Shard 22:  18%|█▊        | 18434103/100000000 [00:05<00:19, 4186434.08tokens/s]
Shard 22:  19%|█▉        | 18863158/100000000 [00:05<00:20, 3924437.53tokens/s]
Shard 22:  19%|█▉        | 19265709/100000000 [00:05<00:20, 3948901.02tokens/s]
Shard 22:  20%|█▉        | 19666012/100000000 [00:05<00:20, 3908898.50tokens/s]
Shard 22:  20%|██        | 20128518/100000000 [00:05<00:19, 4088560.40tokens/s]
Shard 22:  21%|██        | 20540365/100000000 [00:05<00:21, 3704768.92tokens/s]
Shard 22:  21%|██        | 20950090/100000000 [00:05<00:20, 3771977.10tokens/s]
Shard 22:  21%|██▏       | 21333619/100000000 [00:05<00:21, 3583627.69tokens/s]
Shard 22:  22%|██▏       | 21697539/100000000 [00:05<00:21, 3574381.50tokens/s]
Shard 22:  22%|██▏       | 22059185/100000000 [00:06<00:23, 3321013.53tokens/s]
Shard 22:  22%|██▏       | 22401295/100000000 [00:06<00:25, 3012564.08tokens/s]
Shard 22:  23%|██▎       | 22794457/100000000 [00:06<00:24, 3206220.28tokens/s]
Shard 22:  23%|██▎       | 23123566/100000000 [00:06<00:24, 3144663.91tokens/s]
Shard 22:  23%|██▎       | 23476448/100000000 [00:06<00:23, 3242552.05tokens/s]
Shard 22:  24%|██▍       | 23806060/100000000 [00:06<00:25, 2934130.00tokens/s]
Shard 22:  24%|██▍       | 24108247/100000000 [00:06<00:26, 2877299.65tokens/s]
Shard 22:  25%|██▍       | 24504799/100000000 [00:06<00:24, 3137444.97tokens/s]
Shard 22:  25%|██▍       | 24824870/100000000 [00:06<00:24, 3104799.85tokens/s]
Shard 22:  25%|██▌       | 25305473/100000000 [00:07<00:20, 3574759.63tokens/s]
Shard 22:  26%|██▌       | 25669282/100000000 [00:07<00:21, 3456607.06tokens/s]
Shard 22:  26%|██▌       | 26020582/100000000 [00:07<00:24, 2988207.30tokens/s]
Shard 22:  26%|██▋       | 26333061/100000000 [00:07<00:29, 2508554.05tokens/s]
Shard 22:  27%|██▋       | 26603823/100000000 [00:07<00:28, 2552465.29tokens/s]
Shard 22:  27%|██▋       | 26874159/100000000 [00:07<00:31, 2355674.59tokens/s]
Shard 22:  27%|██▋       | 27121574/100000000 [00:07<00:33, 2145293.91tokens/s]
Shard 22:  27%|██▋       | 27354399/100000000 [00:08<00:37, 1958811.48tokens/s]
Shard 22:  28%|██▊       | 27607075/100000000 [00:08<00:34, 2082602.50tokens/s]
Shard 22:  28%|██▊       | 27840314/100000000 [00:08<00:33, 2145041.18tokens/s]
Shard 22:  28%|██▊       | 28181505/100000000 [00:08<00:29, 2464724.21tokens/s]
Shard 22:  28%|██▊       | 28437046/100000000 [00:08<00:29, 2428790.24tokens/s]
Shard 22:  29%|██▉       | 28750512/100000000 [00:08<00:27, 2606350.59tokens/s]
Shard 22:  29%|██▉       | 29017479/100000000 [00:08<00:28, 2453249.66tokens/s]
Shard 22:  29%|██▉       | 29320659/100000000 [00:08<00:27, 2582288.25tokens/s]
Shard 22:  30%|██▉       | 29606113/100000000 [00:08<00:26, 2634597.82tokens/s]
Shard 22:  30%|██▉       | 29873198/100000000 [00:09<00:30, 2262295.22tokens/s]
Shard 22:  30%|███       | 30110779/100000000 [00:09<00:34, 2051332.55tokens/s]
Shard 22:  30%|███       | 30326396/100000000 [00:09<00:36, 1925070.41tokens/s]
Shard 22:  31%|███       | 30537612/100000000 [00:09<00:38, 1815773.17tokens/s]
Shard 22:  31%|███       | 30724789/100000000 [00:09<00:38, 1813590.17tokens/s]
Shard 22:  31%|███       | 30981429/100000000 [00:09<00:34, 2000775.99tokens/s]
Shard 22:  31%|███       | 31230866/100000000 [00:09<00:32, 2131704.16tokens/s]
Shard 22:  32%|███▏      | 31539512/100000000 [00:09<00:28, 2387975.64tokens/s]
Shard 22:  32%|███▏      | 31946650/100000000 [00:09<00:23, 2863394.38tokens/s]
Shard 22:  32%|███▏      | 32253383/100000000 [00:10<00:23, 2920376.12tokens/s]
Shard 22:  33%|███▎      | 32551748/100000000 [00:10<00:29, 2311876.18tokens/s]
Shard 22:  33%|███▎      | 32806308/100000000 [00:10<00:29, 2299420.21tokens/s]
Shard 22:  33%|███▎      | 33053152/100000000 [00:10<00:33, 1984996.45tokens/s]
Shard 22:  33%|███▎      | 33268763/100000000 [00:10<00:33, 2011650.35tokens/s]
Shard 22:  34%|███▎      | 33535278/100000000 [00:10<00:30, 2171562.84tokens/s]
Shard 22:  34%|███▍      | 33790287/100000000 [00:10<00:29, 2267799.17tokens/s]
Shard 22:  34%|███▍      | 34032324/100000000 [00:10<00:28, 2287120.40tokens/s]
Shard 22:  34%|███▍      | 34268705/100000000 [00:11<00:28, 2292043.31tokens/s]
Shard 22:  35%|███▍      | 34537642/100000000 [00:11<00:27, 2401071.58tokens/s]
Shard 22:  35%|███▍      | 34781773/100000000 [00:11<00:27, 2342039.71tokens/s]
Shard 22:  35%|███▌      | 35152411/100000000 [00:11<00:24, 2687504.82tokens/s]
Shard 22:  35%|███▌      | 35424276/100000000 [00:11<00:24, 2590106.10tokens/s]
Shard 22:  36%|███▌      | 35816459/100000000 [00:11<00:21, 2953145.61tokens/s]
Shard 22:  36%|███▌      | 36115402/100000000 [00:11<00:22, 2814913.51tokens/s]
Shard 22:  37%|███▋      | 36541328/100000000 [00:11<00:19, 3216857.10tokens/s]
Shard 22:  37%|███▋      | 36868480/100000000 [00:11<00:20, 3128543.40tokens/s]
Shard 22:  37%|███▋      | 37207946/100000000 [00:12<00:19, 3182671.84tokens/s]
Shard 22:  38%|███▊      | 37653680/100000000 [00:12<00:17, 3544389.29tokens/s]
Shard 22:  38%|███▊      | 38047439/100000000 [00:12<00:16, 3655725.98tokens/s]
Shard 22:  39%|███▉      | 39003424/100000000 [00:12<00:11, 5373832.82tokens/s]
Shard 22:  40%|███▉      | 39547162/100000000 [00:12<00:11, 5256309.53tokens/s]
Shard 22:  40%|████      | 40076977/100000000 [00:12<00:11, 5139883.45tokens/s]
Shard 22:  41%|████      | 40596340/100000000 [00:12<00:14, 4202208.73tokens/s]
Shard 22:  41%|████      | 41054438/100000000 [00:12<00:15, 3898915.64tokens/s]
Shard 22:  41%|████▏     | 41475553/100000000 [00:13<00:16, 3560285.82tokens/s]
Shard 22:  42%|████▏     | 41850608/100000000 [00:13<00:17, 3303307.54tokens/s]
Shard 22:  42%|████▏     | 42195169/100000000 [00:13<00:17, 3272505.19tokens/s]
Shard 22:  43%|████▎     | 42531726/100000000 [00:13<00:19, 2923126.77tokens/s]
Shard 22:  43%|████▎     | 42838758/100000000 [00:13<00:19, 2947833.54tokens/s]
Shard 22:  43%|████▎     | 43148445/100000000 [00:13<00:19, 2985870.50tokens/s]
Shard 22:  43%|████▎     | 43452893/100000000 [00:13<00:19, 2831152.20tokens/s]
Shard 22:  44%|████▎     | 43741793/100000000 [00:13<00:20, 2785794.05tokens/s]
Shard 22:  44%|████▍     | 44040095/100000000 [00:13<00:19, 2823149.68tokens/s]
Shard 22:  44%|████▍     | 44395880/100000000 [00:14<00:18, 3023989.04tokens/s]
Shard 22:  45%|████▍     | 44723853/100000000 [00:14<00:17, 3090710.62tokens/s]
Shard 22:  45%|████▌     | 45036149/100000000 [00:14<00:19, 2802129.94tokens/s]
Shard 22:  45%|████▌     | 45465674/100000000 [00:14<00:17, 3204221.68tokens/s]
Shard 22:  46%|████▌     | 45827543/100000000 [00:14<00:16, 3318970.40tokens/s]
Shard 22:  46%|████▌     | 46166841/100000000 [00:14<00:16, 3313257.20tokens/s]
Shard 22:  47%|████▋     | 46554838/100000000 [00:14<00:15, 3463836.73tokens/s]
Shard 22:  47%|████▋     | 46910675/100000000 [00:14<00:15, 3484581.74tokens/s]
Shard 22:  47%|████▋     | 47261986/100000000 [00:14<00:15, 3373683.53tokens/s]
Shard 22:  48%|████▊     | 47605855/100000000 [00:15<00:15, 3296167.04tokens/s]
Shard 22:  48%|████▊     | 47952476/100000000 [00:15<00:15, 3301574.50tokens/s]
Shard 22:  48%|████▊     | 48310733/100000000 [00:15<00:15, 3371646.14tokens/s]
Shard 22:  49%|████▊     | 48649472/100000000 [00:15<00:17, 2985181.82tokens/s]
Shard 22:  49%|████▉     | 48993985/100000000 [00:15<00:16, 3100813.74tokens/s]
Shard 22:  49%|████▉     | 49311714/100000000 [00:15<00:16, 3086318.16tokens/s]
Shard 22:  50%|████▉     | 49721432/100000000 [00:15<00:15, 3347601.58tokens/s]
Shard 22:  50%|█████     | 50061368/100000000 [00:15<00:15, 3192431.32tokens/s]
Shard 22:  50%|█████     | 50385249/100000000 [00:15<00:17, 2863674.26tokens/s]
Shard 22:  51%|█████     | 50701225/100000000 [00:16<00:16, 2930664.23tokens/s]
Shard 22:  51%|█████     | 51001395/100000000 [00:16<00:17, 2816549.27tokens/s]
Shard 22:  51%|█████▏    | 51287938/100000000 [00:16<00:17, 2755601.91tokens/s]
Shard 22:  52%|█████▏    | 51625162/100000000 [00:16<00:16, 2901870.09tokens/s]
Shard 22:  52%|█████▏    | 52015439/100000000 [00:16<00:15, 3178994.28tokens/s]
Shard 22:  52%|█████▏    | 52337980/100000000 [00:16<00:16, 2962511.83tokens/s]
Shard 22:  53%|█████▎    | 52693800/100000000 [00:16<00:15, 3107035.60tokens/s]
Shard 22:  53%|█████▎    | 53056061/100000000 [00:16<00:14, 3217772.13tokens/s]
Shard 22:  53%|█████▎    | 53381811/100000000 [00:16<00:14, 3119061.84tokens/s]
Shard 22:  54%|█████▎    | 53710294/100000000 [00:17<00:14, 3151775.14tokens/s]
Shard 22:  54%|█████▍    | 54044421/100000000 [00:17<00:14, 3197402.66tokens/s]
Shard 22:  54%|█████▍    | 54378235/100000000 [00:17<00:14, 3219641.28tokens/s]
Shard 22:  55%|█████▍    | 54701553/100000000 [00:17<00:15, 2989164.88tokens/s]
Shard 22:  55%|█████▌    | 55151215/100000000 [00:17<00:13, 3396682.64tokens/s]
Shard 22:  56%|█████▌    | 55503684/100000000 [00:17<00:13, 3407529.73tokens/s]
Shard 22:  56%|█████▌    | 55848301/100000000 [00:17<00:14, 3124399.38tokens/s]
Shard 22:  56%|█████▋    | 56264420/100000000 [00:17<00:12, 3401020.37tokens/s]
Shard 22:  57%|█████▋    | 56638418/100000000 [00:17<00:12, 3470105.79tokens/s]
Shard 22:  57%|█████▋    | 57027503/100000000 [00:17<00:12, 3572656.76tokens/s]
Shard 22:  57%|█████▋    | 57389328/100000000 [00:18<00:12, 3420564.67tokens/s]
Shard 22:  58%|█████▊    | 57737237/100000000 [00:18<00:12, 3426052.70tokens/s]
Shard 22:  58%|█████▊    | 58194802/100000000 [00:18<00:11, 3712047.41tokens/s]
Shard 22:  59%|█████▊    | 58571867/100000000 [00:18<00:11, 3510086.63tokens/s]
Shard 22:  59%|█████▉    | 58993166/100000000 [00:18<00:11, 3690679.51tokens/s]
Shard 22:  59%|█████▉    | 59366150/100000000 [00:18<00:11, 3561121.43tokens/s]
Shard 22:  60%|█████▉    | 59730673/100000000 [00:18<00:11, 3504407.47tokens/s]
Shard 22:  60%|██████    | 60084123/100000000 [00:18<00:11, 3401046.28tokens/s]
Shard 22:  60%|██████    | 60426059/100000000 [00:18<00:12, 3150681.27tokens/s]
Shard 22:  61%|██████    | 60745012/100000000 [00:19<00:12, 3020423.47tokens/s]
Shard 22:  61%|██████    | 61051644/100000000 [00:19<00:13, 2910752.40tokens/s]
Shard 22:  61%|██████▏   | 61421869/100000000 [00:19<00:12, 3121060.04tokens/s]
Shard 22:  62%|██████▏   | 61737694/100000000 [00:19<00:12, 3040206.43tokens/s]
Shard 22:  62%|██████▏   | 62045269/100000000 [00:19<00:13, 2777512.45tokens/s]
Shard 22:  62%|██████▏   | 62370912/100000000 [00:19<00:13, 2886510.63tokens/s]
Shard 22:  63%|██████▎   | 62722898/100000000 [00:19<00:12, 3056291.60tokens/s]
Shard 22:  63%|██████▎   | 63037141/100000000 [00:19<00:12, 3069322.96tokens/s]
Shard 22:  63%|██████▎   | 63358735/100000000 [00:19<00:12, 2862374.77tokens/s]
Shard 22:  64%|██████▎   | 63650233/100000000 [00:20<00:13, 2682749.96tokens/s]
Shard 22:  64%|██████▍   | 63927760/100000000 [00:20<00:14, 2563536.85tokens/s]
Shard 22:  64%|██████▍   | 64208715/100000000 [00:20<00:13, 2622730.60tokens/s]
Shard 22:  65%|██████▍   | 64585844/100000000 [00:20<00:12, 2937709.85tokens/s]
Shard 22:  65%|██████▍   | 64939740/100000000 [00:20<00:11, 3092548.15tokens/s]
Shard 22:  66%|██████▌   | 65593375/100000000 [00:20<00:08, 4045783.73tokens/s]
Shard 22:  66%|██████▌   | 66124857/100000000 [00:20<00:07, 4394880.07tokens/s]
Shard 22:  67%|██████▋   | 66845499/100000000 [00:20<00:06, 5153478.82tokens/s]
Shard 22:  67%|██████▋   | 67439663/100000000 [00:20<00:06, 5375366.20tokens/s]
Shard 22:  68%|██████▊   | 68157393/100000000 [00:21<00:05, 5866785.01tokens/s]
Shard 22:  69%|██████▊   | 68748584/100000000 [00:21<00:05, 5761925.45tokens/s]
Shard 22:  69%|██████▉   | 69327836/100000000 [00:21<00:05, 5665598.22tokens/s]
Shard 22:  70%|██████▉   | 69896307/100000000 [00:21<00:05, 5277969.97tokens/s]
Shard 22:  70%|███████   | 70431624/100000000 [00:21<00:06, 4874024.38tokens/s]
Shard 22:  71%|███████   | 70929521/100000000 [00:21<00:06, 4640758.93tokens/s]
Shard 22:  71%|███████▏  | 71401383/100000000 [00:21<00:06, 4501742.31tokens/s]
Shard 22:  72%|███████▏  | 71855769/100000000 [00:21<00:06, 4440137.57tokens/s]
Shard 22:  72%|███████▏  | 72302730/100000000 [00:21<00:06, 4167248.55tokens/s]
Shard 22:  73%|███████▎  | 72723225/100000000 [00:22<00:06, 4011150.97tokens/s]
Shard 22:  73%|███████▎  | 73344713/100000000 [00:22<00:05, 4589000.08tokens/s]
Shard 22:  74%|███████▍  | 74008845/100000000 [00:22<00:05, 5139652.37tokens/s]
Shard 22:  75%|███████▍  | 74532382/100000000 [00:22<00:06, 3956409.57tokens/s]
Shard 22:  75%|███████▍  | 74996745/100000000 [00:22<00:06, 4115750.06tokens/s]
Shard 22:  76%|███████▌  | 75621270/100000000 [00:22<00:05, 4644615.63tokens/s]
Shard 22:  76%|███████▌  | 76239599/100000000 [00:22<00:04, 5040518.85tokens/s]
Shard 22:  77%|███████▋  | 77083016/100000000 [00:22<00:03, 5931848.09tokens/s]
Shard 22:  78%|███████▊  | 77704089/100000000 [00:23<00:04, 5432817.17tokens/s]
Shard 22:  78%|███████▊  | 78280840/100000000 [00:23<00:04, 5382643.10tokens/s]
Shard 22:  79%|███████▉  | 78838002/100000000 [00:23<00:04, 4903058.17tokens/s]
Shard 22:  79%|███████▉  | 79346885/100000000 [00:23<00:04, 4757080.20tokens/s]
Shard 22:  80%|███████▉  | 79835133/100000000 [00:23<00:04, 4427724.72tokens/s]
Shard 22:  80%|████████  | 80441351/100000000 [00:23<00:04, 4807799.94tokens/s]
Shard 22:  81%|████████  | 80935539/100000000 [00:23<00:03, 4841019.85tokens/s]
Shard 22:  81%|████████▏ | 81433173/100000000 [00:23<00:03, 4870355.47tokens/s]
Shard 22:  82%|████████▏ | 81955572/100000000 [00:23<00:03, 4915256.10tokens/s]
Shard 22:  82%|████████▏ | 82452543/100000000 [00:24<00:03, 4917175.76tokens/s]
Shard 22:  83%|████████▎ | 82948630/100000000 [00:24<00:03, 4347188.22tokens/s]
Shard 22:  83%|████████▎ | 83396893/100000000 [00:24<00:04, 3956072.77tokens/s]
Shard 22:  84%|████████▍ | 83806858/100000000 [00:24<00:04, 3973859.38tokens/s]
Shard 22:  84%|████████▍ | 84355684/100000000 [00:24<00:03, 4368103.87tokens/s]
Shard 22:  85%|████████▍ | 84809484/100000000 [00:24<00:03, 4399912.07tokens/s]
Shard 22:  85%|████████▌ | 85258046/100000000 [00:24<00:03, 4016903.88tokens/s]
Shard 22:  86%|████████▌ | 85671249/100000000 [00:24<00:03, 3857163.68tokens/s]
Shard 22:  86%|████████▌ | 86065633/100000000 [00:25<00:03, 3744859.87tokens/s]
Shard 22:  87%|████████▋ | 86560973/100000000 [00:25<00:03, 4056787.60tokens/s]
Shard 22:  87%|████████▋ | 86973653/100000000 [00:25<00:03, 4000299.31tokens/s]
Shard 22:  87%|████████▋ | 87452025/100000000 [00:25<00:02, 4208033.74tokens/s]
Shard 22:  88%|████████▊ | 88037099/100000000 [00:25<00:02, 4635118.65tokens/s]
Shard 22:  89%|████████▊ | 88620337/100000000 [00:25<00:02, 4961022.17tokens/s]
Shard 22:  89%|████████▉ | 89120773/100000000 [00:25<00:02, 4675559.61tokens/s]
Shard 22:  90%|████████▉ | 89594734/100000000 [00:25<00:02, 4461947.19tokens/s]
Shard 22:  90%|█████████ | 90047828/100000000 [00:25<00:02, 4316887.97tokens/s]
Shard 22:  90%|█████████ | 90483315/100000000 [00:25<00:02, 4276657.45tokens/s]
Shard 22:  91%|█████████ | 90917168/100000000 [00:26<00:02, 4240231.41tokens/s]
Shard 22:  91%|█████████▏| 91380687/100000000 [00:26<00:01, 4351986.51tokens/s]
Shard 22:  92%|█████████▏| 91831838/100000000 [00:26<00:01, 4352840.29tokens/s]
Shard 22:  92%|█████████▏| 92268976/100000000 [00:26<00:01, 4297400.02tokens/s]
Shard 22:  93%|█████████▎| 92778008/100000000 [00:26<00:01, 4504002.27tokens/s]
Shard 22:  93%|█████████▎| 93232110/100000000 [00:26<00:01, 3698750.96tokens/s]
Shard 22:  94%|█████████▎| 93638619/100000000 [00:26<00:01, 3787455.44tokens/s]
Shard 22:  94%|█████████▍| 94076795/100000000 [00:26<00:01, 3929309.61tokens/s]
Shard 22:  95%|█████████▍| 94523829/100000000 [00:26<00:01, 4058653.69tokens/s]
Shard 22:  95%|█████████▌| 95111422/100000000 [00:27<00:01, 4549233.66tokens/s]
Shard 22:  96%|█████████▌| 95618696/100000000 [00:27<00:00, 4689774.84tokens/s]
Shard 22:  96%|█████████▌| 96096920/100000000 [00:27<00:00, 4152029.90tokens/s]
Shard 22:  97%|█████████▋| 96528598/100000000 [00:27<00:00, 4108025.64tokens/s]
Shard 22:  97%|█████████▋| 96983166/100000000 [00:27<00:00, 4198133.75tokens/s]
Shard 22:  97%|█████████▋| 97411522/100000000 [00:27<00:00, 4061985.37tokens/s]
Shard 22:  98%|█████████▊| 97903624/100000000 [00:27<00:00, 4275088.40tokens/s]
Shard 22:  98%|█████████▊| 98338183/100000000 [00:27<00:00, 4283764.61tokens/s]
Shard 22:  99%|█████████▉| 98771204/100000000 [00:27<00:00, 3973280.49tokens/s]
Shard 22:  99%|█████████▉| 99181574/100000000 [00:28<00:00, 3489764.27tokens/s]
Shard 22: 100%|█████████▉| 99600469/100000000 [00:28<00:00, 3667566.88tokens/s]
Shard 22: 100%|█████████▉| 99980142/100000000 [00:28<00:00, 3240610.35tokens/s]
Shard 22: 100%|█████████▉| 99999753/100000000 [00:36<00:00, 2723213.92tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000022.bin (200,001,024 bytes) in the gpt-2 format

Shard 23:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 23:   0%|          | 278586/100000000 [00:00<00:35, 2772990.40tokens/s]
Shard 23:   1%|          | 683578/100000000 [00:00<00:28, 3487415.11tokens/s]
Shard 23:   1%|          | 1032721/100000000 [00:00<00:29, 3375974.56tokens/s]
Shard 23:   1%|▏         | 1447444/100000000 [00:00<00:27, 3599671.34tokens/s]
Shard 23:   2%|▏         | 1807448/100000000 [00:00<00:37, 2651989.66tokens/s]
Shard 23:   2%|▏         | 2132580/100000000 [00:00<00:39, 2490149.40tokens/s]
Shard 23:   3%|▎         | 2714822/100000000 [00:00<00:29, 3272487.10tokens/s]
Shard 23:   3%|▎         | 3267726/100000000 [00:00<00:25, 3808017.85tokens/s]
Shard 23:   4%|▎         | 3695795/100000000 [00:01<00:24, 3928340.90tokens/s]
Shard 23:   4%|▍         | 4217564/100000000 [00:01<00:22, 4284134.15tokens/s]
Shard 23:   5%|▍         | 4665514/100000000 [00:01<00:23, 4024189.41tokens/s]
Shard 23:   5%|▌         | 5083827/100000000 [00:01<00:26, 3595531.20tokens/s]
Shard 23:   6%|▌         | 5755572/100000000 [00:01<00:21, 4363137.35tokens/s]
Shard 23:   6%|▌         | 6217673/100000000 [00:01<00:23, 4037993.01tokens/s]
Shard 23:   7%|▋         | 6782232/100000000 [00:01<00:20, 4445789.16tokens/s]
Shard 23:   7%|▋         | 7361349/100000000 [00:01<00:19, 4804310.76tokens/s]
Shard 23:   8%|▊         | 8029329/100000000 [00:01<00:17, 5301429.89tokens/s]
Shard 23:   9%|▊         | 8693246/100000000 [00:02<00:16, 5664588.82tokens/s]
Shard 23:  10%|█         | 10044687/100000000 [00:02<00:11, 7917736.31tokens/s]
Shard 23:  11%|█         | 10886051/100000000 [00:02<00:14, 6018749.22tokens/s]
Shard 23:  12%|█▏        | 11570925/100000000 [00:02<00:16, 5205113.64tokens/s]
Shard 23:  12%|█▏        | 12163173/100000000 [00:02<00:19, 4495448.15tokens/s]
Shard 23:  13%|█▎        | 12674245/100000000 [00:02<00:22, 3915493.30tokens/s]
Shard 23:  13%|█▎        | 13113287/100000000 [00:03<00:22, 3823374.11tokens/s]
Shard 23:  14%|█▎        | 13526447/100000000 [00:03<00:24, 3535940.77tokens/s]
Shard 23:  14%|█▍        | 13900178/100000000 [00:03<00:24, 3455490.77tokens/s]
Shard 23:  14%|█▍        | 14404558/100000000 [00:03<00:22, 3817848.47tokens/s]
Shard 23:  15%|█▍        | 14805386/100000000 [00:03<00:24, 3515947.46tokens/s]
Shard 23:  15%|█▌        | 15171928/100000000 [00:03<00:27, 3055307.32tokens/s]
Shard 23:  15%|█▌        | 15496585/100000000 [00:03<00:28, 3010189.45tokens/s]
Shard 23:  16%|█▌        | 15809749/100000000 [00:04<00:30, 2799187.25tokens/s]
Shard 23:  16%|█▌        | 16097773/100000000 [00:04<00:30, 2768451.24tokens/s]
Shard 23:  16%|█▋        | 16383167/100000000 [00:04<00:31, 2616496.44tokens/s]
Shard 23:  17%|█▋        | 16661124/100000000 [00:04<00:32, 2564019.15tokens/s]
Shard 23:  17%|█▋        | 16978744/100000000 [00:04<00:30, 2718003.92tokens/s]
Shard 23:  17%|█▋        | 17276294/100000000 [00:04<00:29, 2786724.94tokens/s]
Shard 23:  18%|█▊        | 17593166/100000000 [00:04<00:28, 2890508.87tokens/s]
Shard 23:  18%|█▊        | 17886230/100000000 [00:04<00:30, 2704155.53tokens/s]
Shard 23:  18%|█▊        | 18187502/100000000 [00:04<00:29, 2782387.97tokens/s]
Shard 23:  18%|█▊        | 18475446/100000000 [00:05<00:31, 2610074.62tokens/s]
Shard 23:  19%|█▊        | 18741610/100000000 [00:05<00:33, 2447634.10tokens/s]
Shard 23:  19%|█▉        | 19028627/100000000 [00:05<00:31, 2555268.25tokens/s]
Shard 23:  19%|█▉        | 19406112/100000000 [00:05<00:27, 2883376.69tokens/s]
Shard 23:  20%|█▉        | 19854463/100000000 [00:05<00:24, 3323157.81tokens/s]
Shard 23:  20%|██        | 20198591/100000000 [00:05<00:23, 3343915.07tokens/s]
Shard 23:  21%|██        | 20537632/100000000 [00:05<00:23, 3329255.04tokens/s]
Shard 23:  21%|██        | 21057163/100000000 [00:05<00:20, 3851200.27tokens/s]
Shard 23:  21%|██▏       | 21446467/100000000 [00:05<00:26, 2935932.88tokens/s]
Shard 23:  22%|██▏       | 21774560/100000000 [00:06<00:26, 2958002.08tokens/s]
Shard 23:  22%|██▏       | 22244255/100000000 [00:06<00:22, 3395477.48tokens/s]
Shard 23:  23%|██▎       | 22610196/100000000 [00:06<00:22, 3413589.67tokens/s]
Shard 23:  23%|██▎       | 22970658/100000000 [00:06<00:24, 3131743.47tokens/s]
Shard 23:  23%|██▎       | 23300560/100000000 [00:06<00:27, 2833281.04tokens/s]
Shard 23:  24%|██▎       | 23598624/100000000 [00:06<00:28, 2707267.62tokens/s]
Shard 23:  24%|██▍       | 23904096/100000000 [00:06<00:27, 2779910.11tokens/s]
Shard 23:  24%|██▍       | 24219934/100000000 [00:06<00:26, 2866723.92tokens/s]
Shard 23:  25%|██▍       | 24546347/100000000 [00:06<00:25, 2973857.24tokens/s]
Shard 23:  25%|██▍       | 24849881/100000000 [00:07<00:27, 2710225.25tokens/s]
Shard 23:  25%|██▌       | 25217879/100000000 [00:07<00:25, 2967369.74tokens/s]
Shard 23:  26%|██▌       | 25542366/100000000 [00:07<00:24, 3043513.19tokens/s]
Shard 23:  26%|██▌       | 25897643/100000000 [00:07<00:23, 3186921.98tokens/s]
Shard 23:  26%|██▌       | 26222536/100000000 [00:07<00:23, 3106521.29tokens/s]
Shard 23:  27%|██▋       | 26563766/100000000 [00:07<00:23, 3189898.17tokens/s]
Shard 23:  27%|██▋       | 26926985/100000000 [00:07<00:22, 3311804.96tokens/s]
Shard 23:  27%|██▋       | 27459953/100000000 [00:07<00:18, 3888256.14tokens/s]
Shard 23:  28%|██▊       | 27865604/100000000 [00:07<00:18, 3922194.60tokens/s]
Shard 23:  28%|██▊       | 28260599/100000000 [00:08<00:19, 3762346.99tokens/s]
Shard 23:  29%|██▊       | 28733045/100000000 [00:08<00:17, 4029400.93tokens/s]
Shard 23:  29%|██▉       | 29253107/100000000 [00:08<00:16, 4364336.97tokens/s]
Shard 23:  30%|██▉       | 29757399/100000000 [00:08<00:15, 4557654.00tokens/s]
Shard 23:  30%|███       | 30215913/100000000 [00:08<00:16, 4152741.76tokens/s]
Shard 23:  31%|███       | 30639630/100000000 [00:08<00:18, 3674214.54tokens/s]
Shard 23:  31%|███       | 31174784/100000000 [00:08<00:16, 4085157.18tokens/s]
Shard 23:  32%|███▏      | 31707292/100000000 [00:08<00:15, 4414433.00tokens/s]
Shard 23:  32%|███▏      | 32164742/100000000 [00:08<00:16, 4083464.24tokens/s]
Shard 23:  33%|███▎      | 32587515/100000000 [00:09<00:17, 3751225.40tokens/s]
Shard 23:  33%|███▎      | 32977029/100000000 [00:09<00:17, 3788187.63tokens/s]
Shard 23:  33%|███▎      | 33366053/100000000 [00:09<00:18, 3655422.59tokens/s]
Shard 23:  34%|███▎      | 33738570/100000000 [00:09<00:18, 3535473.29tokens/s]
Shard 23:  34%|███▍      | 34097925/100000000 [00:09<00:20, 3159375.17tokens/s]
Shard 23:  34%|███▍      | 34422911/100000000 [00:09<00:20, 3124980.05tokens/s]
Shard 23:  35%|███▍      | 34771475/100000000 [00:09<00:20, 3211494.38tokens/s]
Shard 23:  35%|███▌      | 35097832/100000000 [00:09<00:21, 3001528.08tokens/s]
Shard 23:  36%|███▌      | 35605889/100000000 [00:10<00:18, 3555265.10tokens/s]
Shard 23:  36%|███▌      | 35987778/100000000 [00:10<00:17, 3617521.63tokens/s]
Shard 23:  36%|███▋      | 36357477/100000000 [00:10<00:17, 3550405.98tokens/s]
Shard 23:  37%|███▋      | 36745803/100000000 [00:10<00:18, 3355363.73tokens/s]
Shard 23:  37%|███▋      | 37087607/100000000 [00:10<00:20, 3139230.69tokens/s]
Shard 23:  37%|███▋      | 37407538/100000000 [00:10<00:21, 2940958.30tokens/s]
Shard 23:  38%|███▊      | 37713104/100000000 [00:10<00:21, 2899451.66tokens/s]
Shard 23:  38%|███▊      | 38083320/100000000 [00:10<00:19, 3109781.93tokens/s]
Shard 23:  38%|███▊      | 38401388/100000000 [00:10<00:21, 2924473.49tokens/s]
Shard 23:  39%|███▉      | 38767964/100000000 [00:11<00:19, 3122608.13tokens/s]
Shard 23:  39%|███▉      | 39085916/100000000 [00:11<00:19, 3056684.75tokens/s]
Shard 23:  39%|███▉      | 39454362/100000000 [00:11<00:18, 3228026.61tokens/s]
Shard 23:  40%|███▉      | 39785669/100000000 [00:11<00:18, 3249236.08tokens/s]
Shard 23:  40%|████      | 40114047/100000000 [00:11<00:20, 2927839.24tokens/s]
Shard 23:  40%|████      | 40431052/100000000 [00:11<00:19, 2989844.65tokens/s]
Shard 23:  41%|████      | 40756579/100000000 [00:11<00:19, 3057118.35tokens/s]
Shard 23:  41%|████      | 41069872/100000000 [00:11<00:20, 2938875.43tokens/s]
Shard 23:  42%|████▏     | 41586857/100000000 [00:11<00:16, 3558445.01tokens/s]
Shard 23:  42%|████▏     | 41950922/100000000 [00:12<00:18, 3136583.07tokens/s]
Shard 23:  42%|████▏     | 42278326/100000000 [00:12<00:19, 3027721.13tokens/s]
Shard 23:  43%|████▎     | 42590956/100000000 [00:12<00:19, 2942610.23tokens/s]
Shard 23:  43%|████▎     | 42897597/100000000 [00:12<00:19, 2929615.27tokens/s]
Shard 23:  43%|████▎     | 43198723/100000000 [00:12<00:20, 2798974.73tokens/s]
Shard 23:  43%|████▎     | 43482130/100000000 [00:12<00:20, 2701855.68tokens/s]
Shard 23:  44%|████▍     | 43784646/100000000 [00:12<00:20, 2770116.84tokens/s]
Shard 23:  44%|████▍     | 44063872/100000000 [00:12<00:20, 2707944.66tokens/s]
Shard 23:  44%|████▍     | 44336192/100000000 [00:12<00:22, 2478104.58tokens/s]
Shard 23:  45%|████▍     | 44713566/100000000 [00:13<00:19, 2819580.11tokens/s]
Shard 23:  45%|████▌     | 45012873/100000000 [00:13<00:19, 2866419.51tokens/s]
Shard 23:  45%|████▌     | 45414385/100000000 [00:13<00:17, 3158353.45tokens/s]
Shard 23:  46%|████▌     | 45836321/100000000 [00:13<00:15, 3453489.18tokens/s]
Shard 23:  46%|████▌     | 46187323/100000000 [00:13<00:16, 3209235.93tokens/s]
Shard 23:  47%|████▋     | 46515181/100000000 [00:13<00:16, 3191292.48tokens/s]
Shard 23:  47%|████▋     | 46858860/100000000 [00:13<00:16, 3248018.59tokens/s]
Shard 23:  47%|████▋     | 47226126/100000000 [00:13<00:15, 3364340.84tokens/s]
Shard 23:  48%|████▊     | 47566027/100000000 [00:13<00:16, 3199647.53tokens/s]
Shard 23:  48%|████▊     | 47890631/100000000 [00:14<00:18, 2866568.96tokens/s]
Shard 23:  48%|████▊     | 48280923/100000000 [00:14<00:16, 3136669.57tokens/s]
Shard 23:  49%|████▊     | 48721737/100000000 [00:14<00:14, 3476520.81tokens/s]
Shard 23:  49%|████▉     | 49117293/100000000 [00:14<00:14, 3598564.95tokens/s]
Shard 23:  50%|████▉     | 49569870/100000000 [00:14<00:13, 3852357.68tokens/s]
Shard 23:  50%|████▉     | 49961464/100000000 [00:14<00:14, 3519703.54tokens/s]
Shard 23:  50%|█████     | 50359687/100000000 [00:14<00:13, 3636694.73tokens/s]
Shard 23:  51%|█████     | 50731145/100000000 [00:14<00:15, 3114636.82tokens/s]
Shard 23:  51%|█████     | 51059937/100000000 [00:15<00:15, 3063831.44tokens/s]
Shard 23:  51%|█████▏    | 51377860/100000000 [00:15<00:16, 3006068.04tokens/s]
Shard 23:  52%|█████▏    | 51759758/100000000 [00:15<00:15, 3214453.88tokens/s]
Shard 23:  52%|█████▏    | 52115674/100000000 [00:15<00:14, 3297094.00tokens/s]
Shard 23:  53%|█████▎    | 52505535/100000000 [00:15<00:13, 3444268.22tokens/s]
Shard 23:  53%|█████▎    | 52909065/100000000 [00:15<00:13, 3604171.72tokens/s]
Shard 23:  53%|█████▎    | 53281225/100000000 [00:15<00:14, 3177736.82tokens/s]
Shard 23:  54%|█████▎    | 53611111/100000000 [00:15<00:14, 3145678.23tokens/s]
Shard 23:  54%|█████▍    | 54141885/100000000 [00:15<00:12, 3715147.73tokens/s]
Shard 23:  55%|█████▍    | 54525194/100000000 [00:16<00:14, 3181580.38tokens/s]
Shard 23:  55%|█████▍    | 54867590/100000000 [00:16<00:14, 3174717.73tokens/s]
Shard 23:  55%|█████▌    | 55200532/100000000 [00:16<00:15, 2846631.37tokens/s]
Shard 23:  56%|█████▌    | 55517152/100000000 [00:16<00:15, 2919169.67tokens/s]
Shard 23:  56%|█████▌    | 55841137/100000000 [00:16<00:14, 2993805.88tokens/s]
Shard 23:  56%|█████▋    | 56416549/100000000 [00:16<00:11, 3732304.39tokens/s]
Shard 23:  57%|█████▋    | 56948984/100000000 [00:16<00:10, 4170478.61tokens/s]
Shard 23:  57%|█████▋    | 57403827/100000000 [00:16<00:10, 4236057.26tokens/s]
Shard 23:  58%|█████▊    | 57932362/100000000 [00:16<00:09, 4536752.58tokens/s]
Shard 23:  58%|█████▊    | 58455584/100000000 [00:17<00:08, 4736234.91tokens/s]
Shard 23:  59%|█████▉    | 59024756/100000000 [00:17<00:08, 5007686.34tokens/s]
Shard 23:  60%|█████▉    | 59531539/100000000 [00:17<00:08, 4787860.12tokens/s]
Shard 23:  60%|██████    | 60015621/100000000 [00:17<00:08, 4594985.77tokens/s]
Shard 23:  61%|██████    | 60625413/100000000 [00:17<00:07, 5008900.25tokens/s]
Shard 23:  61%|██████    | 61139202/100000000 [00:17<00:07, 5039411.34tokens/s]
Shard 23:  62%|██████▏   | 61649229/100000000 [00:17<00:07, 5023321.51tokens/s]
Shard 23:  62%|██████▏   | 62155055/100000000 [00:17<00:07, 4967574.53tokens/s]
Shard 23:  63%|██████▎   | 62654198/100000000 [00:17<00:08, 4324631.30tokens/s]
Shard 23:  63%|██████▎   | 63103402/100000000 [00:18<00:08, 4258940.19tokens/s]
Shard 23:  64%|██████▎   | 63541025/100000000 [00:18<00:09, 3975608.92tokens/s]
Shard 23:  64%|██████▍   | 63948329/100000000 [00:18<00:09, 3845400.05tokens/s]
Shard 23:  64%|██████▍   | 64340077/100000000 [00:18<00:09, 3660187.15tokens/s]
Shard 23:  65%|██████▍   | 64864977/100000000 [00:18<00:08, 4078286.42tokens/s]
Shard 23:  65%|██████▌   | 65371099/100000000 [00:18<00:07, 4329842.93tokens/s]
Shard 23:  66%|██████▌   | 65828987/100000000 [00:18<00:07, 4397603.01tokens/s]
Shard 23:  66%|██████▋   | 66275038/100000000 [00:18<00:09, 3606320.74tokens/s]
Shard 23:  67%|██████▋   | 66662245/100000000 [00:18<00:09, 3668505.91tokens/s]
Shard 23:  67%|██████▋   | 67050411/100000000 [00:19<00:09, 3498573.59tokens/s]
Shard 23:  67%|██████▋   | 67418875/100000000 [00:19<00:09, 3520452.13tokens/s]
Shard 23:  68%|██████▊   | 67926281/100000000 [00:19<00:08, 3910180.78tokens/s]
Shard 23:  68%|██████▊   | 68328314/100000000 [00:19<00:08, 3917207.22tokens/s]
Shard 23:  69%|██████▊   | 68727984/100000000 [00:19<00:09, 3181193.13tokens/s]
Shard 23:  69%|██████▉   | 69073592/100000000 [00:19<00:10, 3020843.36tokens/s]
Shard 23:  69%|██████▉   | 69394187/100000000 [00:19<00:10, 2891696.77tokens/s]
Shard 23:  70%|██████▉   | 69722603/100000000 [00:19<00:10, 2980846.05tokens/s]
Shard 23:  70%|███████   | 70074631/100000000 [00:20<00:09, 3118495.70tokens/s]
Shard 23:  71%|███████   | 70546888/100000000 [00:20<00:08, 3546241.09tokens/s]
Shard 23:  71%|███████   | 71048029/100000000 [00:20<00:07, 3955442.76tokens/s]
Shard 23:  71%|███████▏  | 71456647/100000000 [00:20<00:08, 3507352.48tokens/s]
Shard 23:  72%|███████▏  | 71823404/100000000 [00:20<00:08, 3345115.11tokens/s]
Shard 23:  72%|███████▏  | 72169633/100000000 [00:20<00:08, 3356203.56tokens/s]
Shard 23:  73%|███████▎  | 72546228/100000000 [00:20<00:07, 3443480.56tokens/s]
Shard 23:  73%|███████▎  | 72897855/100000000 [00:20<00:08, 3316937.11tokens/s]
Shard 23:  73%|███████▎  | 73303358/100000000 [00:20<00:07, 3509530.16tokens/s]
Shard 23:  74%|███████▎  | 73665675/100000000 [00:21<00:07, 3535383.70tokens/s]
Shard 23:  74%|███████▍  | 74073555/100000000 [00:21<00:07, 3678933.39tokens/s]
Shard 23:  74%|███████▍  | 74444726/100000000 [00:21<00:06, 3688359.76tokens/s]
Shard 23:  75%|███████▍  | 74815929/100000000 [00:21<00:07, 3501393.94tokens/s]
Shard 23:  75%|███████▌  | 75345392/100000000 [00:21<00:06, 4003834.21tokens/s]
Shard 23:  76%|███████▌  | 75785829/100000000 [00:21<00:05, 4109205.29tokens/s]
Shard 23:  76%|███████▌  | 76200795/100000000 [00:21<00:05, 3971727.80tokens/s]
Shard 23:  77%|███████▋  | 76601763/100000000 [00:21<00:06, 3758994.40tokens/s]
Shard 23:  77%|███████▋  | 77044102/100000000 [00:21<00:05, 3939506.44tokens/s]
Shard 23:  77%|███████▋  | 77473759/100000000 [00:21<00:05, 4037922.17tokens/s]
Shard 23:  78%|███████▊  | 77881285/100000000 [00:22<00:05, 3858808.89tokens/s]
Shard 23:  78%|███████▊  | 78275984/100000000 [00:22<00:05, 3840064.41tokens/s]
Shard 23:  79%|███████▊  | 78663062/100000000 [00:22<00:10, 1968633.31tokens/s]
Shard 23:  79%|███████▉  | 79022791/100000000 [00:22<00:09, 2249153.22tokens/s]
Shard 23:  79%|███████▉  | 79338737/100000000 [00:22<00:08, 2352438.92tokens/s]
Shard 23:  80%|███████▉  | 79643362/100000000 [00:22<00:08, 2422178.91tokens/s]
Shard 23:  80%|████████  | 80006829/100000000 [00:23<00:07, 2697389.93tokens/s]
Shard 23:  80%|████████  | 80420281/100000000 [00:23<00:06, 3050315.72tokens/s]
Shard 23:  81%|████████  | 80892874/100000000 [00:23<00:05, 3486045.35tokens/s]
Shard 23:  81%|████████▏ | 81311607/100000000 [00:23<00:05, 3674927.43tokens/s]
Shard 23:  82%|████████▏ | 81777263/100000000 [00:23<00:04, 3873924.65tokens/s]
Shard 23:  82%|████████▏ | 82182744/100000000 [00:23<00:04, 3802388.64tokens/s]
Shard 23:  83%|████████▎ | 82589485/100000000 [00:23<00:04, 3854629.22tokens/s]
Shard 23:  83%|████████▎ | 83032603/100000000 [00:23<00:04, 4014263.62tokens/s]
Shard 23:  83%|████████▎ | 83441520/100000000 [00:23<00:04, 3776816.69tokens/s]
Shard 23:  84%|████████▍ | 83828002/100000000 [00:24<00:04, 3781689.12tokens/s]
Shard 23:  84%|████████▍ | 84419667/100000000 [00:24<00:03, 4385846.40tokens/s]
Shard 23:  85%|████████▍ | 84866064/100000000 [00:24<00:04, 3782120.36tokens/s]
Shard 23:  85%|████████▌ | 85420849/100000000 [00:24<00:03, 4233137.68tokens/s]
Shard 23:  86%|████████▌ | 85936381/100000000 [00:24<00:03, 4478333.59tokens/s]
Shard 23:  86%|████████▋ | 86400939/100000000 [00:24<00:03, 4099009.94tokens/s]
Shard 23:  87%|████████▋ | 86833725/100000000 [00:24<00:03, 3755835.07tokens/s]
Shard 23:  87%|████████▋ | 87225579/100000000 [00:24<00:03, 3753640.18tokens/s]
Shard 23:  88%|████████▊ | 87620151/100000000 [00:24<00:03, 3687382.41tokens/s]
Shard 23:  88%|████████▊ | 87997781/100000000 [00:25<00:03, 3683758.21tokens/s]
Shard 23:  88%|████████▊ | 88390458/100000000 [00:25<00:03, 3749975.83tokens/s]
Shard 23:  89%|████████▉ | 88801746/100000000 [00:25<00:02, 3837839.62tokens/s]
Shard 23:  89%|████████▉ | 89247909/100000000 [00:25<00:02, 4008681.92tokens/s]
Shard 23:  90%|████████▉ | 89655819/100000000 [00:25<00:02, 3702395.94tokens/s]
Shard 23:  90%|█████████ | 90039320/100000000 [00:25<00:02, 3491305.50tokens/s]
Shard 23:  90%|█████████ | 90394532/100000000 [00:25<00:02, 3365179.20tokens/s]
Shard 23:  91%|█████████ | 90735552/100000000 [00:25<00:02, 3105928.46tokens/s]
Shard 23:  91%|█████████ | 91067798/100000000 [00:26<00:03, 2974849.24tokens/s]
Shard 23:  91%|█████████▏| 91369280/100000000 [00:26<00:03, 2802895.92tokens/s]
Shard 23:  92%|█████████▏| 91758671/100000000 [00:26<00:02, 3084564.04tokens/s]
Shard 23:  93%|█████████▎| 92584490/100000000 [00:26<00:01, 4452526.07tokens/s]
Shard 23:  93%|█████████▎| 93047434/100000000 [00:26<00:01, 4099350.59tokens/s]
Shard 23:  93%|█████████▎| 93474176/100000000 [00:26<00:01, 3418732.08tokens/s]
Shard 23:  94%|█████████▍| 93843973/100000000 [00:26<00:01, 3157760.80tokens/s]
Shard 23:  94%|█████████▍| 94213490/100000000 [00:26<00:01, 3282011.06tokens/s]
Shard 23:  95%|█████████▍| 94559729/100000000 [00:27<00:01, 3038240.78tokens/s]
Shard 23:  95%|█████████▍| 94877199/100000000 [00:27<00:01, 2922584.73tokens/s]
Shard 23:  95%|█████████▌| 95178561/100000000 [00:27<00:01, 2882778.29tokens/s]
Shard 23:  95%|█████████▌| 95472685/100000000 [00:27<00:01, 2868041.88tokens/s]
Shard 23:  96%|█████████▌| 95819839/100000000 [00:27<00:01, 3028691.57tokens/s]
Shard 23:  96%|█████████▌| 96127551/100000000 [00:27<00:01, 2896859.34tokens/s]
Shard 23:  96%|█████████▋| 96451969/100000000 [00:27<00:01, 2910369.02tokens/s]
Shard 23:  97%|█████████▋| 96748000/100000000 [00:27<00:01, 2837592.44tokens/s]
Shard 23:  97%|█████████▋| 97046981/100000000 [00:27<00:01, 2879608.45tokens/s]
Shard 23:  97%|█████████▋| 97336720/100000000 [00:28<00:01, 2567274.28tokens/s]
Shard 23:  98%|█████████▊| 97692566/100000000 [00:28<00:00, 2826204.42tokens/s]
Shard 23:  98%|█████████▊| 98016071/100000000 [00:28<00:00, 2936563.75tokens/s]
Shard 23:  98%|█████████▊| 98321966/100000000 [00:28<00:00, 2940565.57tokens/s]
Shard 23:  99%|█████████▉| 99221643/100000000 [00:28<00:00, 4618503.74tokens/s]
Shard 23: 100%|█████████▉| 99692084/100000000 [00:28<00:00, 3841504.17tokens/s]
Shard 23: 100%|█████████▉| 99999918/100000000 [00:37<00:00, 2665882.15tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000023.bin (200,001,024 bytes) in the gpt-2 format

Shard 24:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 24:   0%|          | 295841/100000000 [00:00<00:33, 2950414.77tokens/s]
Shard 24:   1%|          | 600563/100000000 [00:00<00:34, 2868594.78tokens/s]
Shard 24:   1%|          | 1006087/100000000 [00:00<00:29, 3393399.34tokens/s]
Shard 24:   1%|▏         | 1368711/100000000 [00:00<00:28, 3483455.90tokens/s]
Shard 24:   2%|▏         | 1721468/100000000 [00:00<00:28, 3483665.39tokens/s]
Shard 24:   2%|▏         | 2070909/100000000 [00:00<00:33, 2953431.53tokens/s]
Shard 24:   2%|▏         | 2409842/100000000 [00:00<00:31, 3071993.54tokens/s]
Shard 24:   3%|▎         | 2727781/100000000 [00:00<00:32, 3031806.10tokens/s]
Shard 24:   3%|▎         | 3052379/100000000 [00:00<00:31, 3039881.07tokens/s]
Shard 24:   3%|▎         | 3450112/100000000 [00:01<00:29, 3302974.58tokens/s]
Shard 24:   4%|▍         | 3858874/100000000 [00:01<00:27, 3524698.37tokens/s]
Shard 24:   4%|▍         | 4218512/100000000 [00:01<00:29, 3194754.23tokens/s]
Shard 24:   5%|▍         | 4629954/100000000 [00:01<00:27, 3436322.79tokens/s]
Shard 24:   5%|▍         | 4982909/100000000 [00:01<00:28, 3344381.37tokens/s]
Shard 24:   5%|▌         | 5323637/100000000 [00:01<00:31, 2991994.97tokens/s]
Shard 24:   6%|▌         | 5705676/100000000 [00:01<00:29, 3204005.66tokens/s]
Shard 24:   6%|▌         | 6036090/100000000 [00:01<00:30, 3118253.61tokens/s]
Shard 24:   6%|▋         | 6467858/100000000 [00:01<00:27, 3442819.69tokens/s]
Shard 24:   7%|▋         | 6820939/100000000 [00:02<00:28, 3289718.95tokens/s]
Shard 24:   7%|▋         | 7156688/100000000 [00:02<00:29, 3162329.00tokens/s]
Shard 24:   7%|▋         | 7479199/100000000 [00:02<00:29, 3164383.70tokens/s]
Shard 24:   8%|▊         | 7821194/100000000 [00:02<00:28, 3235866.37tokens/s]
Shard 24:   8%|▊         | 8147828/100000000 [00:02<00:31, 2933102.22tokens/s]
Shard 24:   9%|▊         | 8585643/100000000 [00:02<00:27, 3322196.64tokens/s]
Shard 24:   9%|▉         | 9035595/100000000 [00:02<00:24, 3639243.36tokens/s]
Shard 24:   9%|▉         | 9437175/100000000 [00:02<00:24, 3715722.67tokens/s]
Shard 24:  10%|▉         | 9815538/100000000 [00:03<00:27, 3289249.01tokens/s]
Shard 24:  10%|█         | 10157138/100000000 [00:03<00:30, 2925064.76tokens/s]
Shard 24:  11%|█         | 10521049/100000000 [00:03<00:28, 3103117.67tokens/s]
Shard 24:  11%|█         | 10855160/100000000 [00:03<00:31, 2864890.02tokens/s]
Shard 24:  11%|█         | 11202257/100000000 [00:03<00:29, 3014436.60tokens/s]
Shard 24:  12%|█▏        | 11524701/100000000 [00:03<00:29, 3045919.97tokens/s]
Shard 24:  12%|█▏        | 11839822/100000000 [00:03<00:28, 3074697.65tokens/s]
Shard 24:  12%|█▏        | 12158123/100000000 [00:03<00:28, 3078066.87tokens/s]
Shard 24:  12%|█▏        | 12472304/100000000 [00:03<00:30, 2870873.84tokens/s]
Shard 24:  13%|█▎        | 12764964/100000000 [00:04<00:31, 2795830.23tokens/s]
Shard 24:  13%|█▎        | 13076918/100000000 [00:04<00:30, 2854938.96tokens/s]
Shard 24:  14%|█▎        | 13528210/100000000 [00:04<00:26, 3317772.54tokens/s]
Shard 24:  14%|█▍        | 13866683/100000000 [00:04<00:29, 2920051.54tokens/s]
Shard 24:  14%|█▍        | 14201396/100000000 [00:04<00:28, 3018797.44tokens/s]
Shard 24:  15%|█▍        | 14697053/100000000 [00:04<00:24, 3538601.25tokens/s]
Shard 24:  15%|█▌        | 15109209/100000000 [00:04<00:22, 3700014.49tokens/s]
Shard 24:  15%|█▌        | 15488475/100000000 [00:04<00:22, 3710152.53tokens/s]
Shard 24:  16%|█▌        | 15912520/100000000 [00:04<00:21, 3837454.26tokens/s]
Shard 24:  16%|█▋        | 16350306/100000000 [00:05<00:20, 3993775.22tokens/s]
Shard 24:  17%|█▋        | 16753634/100000000 [00:05<00:24, 3451511.99tokens/s]
Shard 24:  17%|█▋        | 17114221/100000000 [00:05<00:24, 3441034.97tokens/s]
Shard 24:  18%|█▊        | 17668125/100000000 [00:05<00:21, 3912754.84tokens/s]
Shard 24:  19%|█▊        | 18641717/100000000 [00:05<00:14, 5504719.08tokens/s]
Shard 24:  19%|█▉        | 19213657/100000000 [00:05<00:15, 5136515.08tokens/s]
Shard 24:  20%|█▉        | 19745682/100000000 [00:05<00:16, 4914682.29tokens/s]
Shard 24:  20%|██        | 20251496/100000000 [00:05<00:16, 4760855.49tokens/s]
Shard 24:  21%|██        | 20737303/100000000 [00:06<00:20, 3844373.66tokens/s]
Shard 24:  21%|██        | 21153755/100000000 [00:06<00:22, 3445174.96tokens/s]
Shard 24:  22%|██▏       | 21523820/100000000 [00:06<00:23, 3383966.12tokens/s]
Shard 24:  22%|██▏       | 22010431/100000000 [00:06<00:20, 3725209.31tokens/s]
Shard 24:  22%|██▏       | 22455685/100000000 [00:06<00:20, 3871583.69tokens/s]
Shard 24:  23%|██▎       | 22883078/100000000 [00:06<00:21, 3568761.49tokens/s]
Shard 24:  23%|██▎       | 23255475/100000000 [00:06<00:21, 3495869.37tokens/s]
Shard 24:  24%|██▎       | 23618232/100000000 [00:06<00:21, 3474635.05tokens/s]
Shard 24:  24%|██▍       | 23974087/100000000 [00:07<00:23, 3276935.36tokens/s]
Shard 24:  24%|██▍       | 24447225/100000000 [00:07<00:20, 3658466.08tokens/s]
Shard 24:  25%|██▍       | 24822184/100000000 [00:07<00:22, 3308752.97tokens/s]
Shard 24:  25%|██▌       | 25164688/100000000 [00:07<00:26, 2830019.66tokens/s]
Shard 24:  26%|██▌       | 25502463/100000000 [00:07<00:25, 2946476.56tokens/s]
Shard 24:  26%|██▌       | 25811849/100000000 [00:07<00:27, 2740494.83tokens/s]
Shard 24:  26%|██▌       | 26104227/100000000 [00:07<00:26, 2769411.50tokens/s]
Shard 24:  26%|██▋       | 26389586/100000000 [00:07<00:29, 2505109.06tokens/s]
Shard 24:  27%|██▋       | 26648970/100000000 [00:08<00:29, 2482776.79tokens/s]
Shard 24:  27%|██▋       | 26948982/100000000 [00:08<00:28, 2603170.09tokens/s]
Shard 24:  27%|██▋       | 27215525/100000000 [00:08<00:29, 2459664.78tokens/s]
Shard 24:  27%|██▋       | 27467380/100000000 [00:08<00:31, 2298526.18tokens/s]
Shard 24:  28%|██▊       | 27701677/100000000 [00:08<00:35, 2056625.54tokens/s]
Shard 24:  28%|██▊       | 27913571/100000000 [00:08<00:35, 2034749.29tokens/s]
Shard 24:  28%|██▊       | 28140389/100000000 [00:08<00:34, 2088687.45tokens/s]
Shard 24:  28%|██▊       | 28391654/100000000 [00:08<00:32, 2202876.15tokens/s]
Shard 24:  29%|██▊       | 28707836/100000000 [00:08<00:28, 2462963.59tokens/s]
Shard 24:  29%|██▉       | 28989852/100000000 [00:09<00:27, 2550780.02tokens/s]
Shard 24:  29%|██▉       | 29416759/100000000 [00:09<00:23, 3023168.57tokens/s]
Shard 24:  30%|██▉       | 29723324/100000000 [00:09<00:25, 2745017.77tokens/s]
Shard 24:  30%|███       | 30005313/100000000 [00:09<00:26, 2681380.70tokens/s]
Shard 24:  30%|███       | 30278619/100000000 [00:09<00:25, 2692388.04tokens/s]
Shard 24:  31%|███       | 30553565/100000000 [00:09<00:26, 2602954.97tokens/s]
Shard 24:  31%|███       | 30816927/100000000 [00:09<00:26, 2597754.52tokens/s]
Shard 24:  31%|███       | 31116139/100000000 [00:09<00:25, 2701051.55tokens/s]
Shard 24:  31%|███▏      | 31440944/100000000 [00:09<00:23, 2857636.99tokens/s]
Shard 24:  32%|███▏      | 31729175/100000000 [00:10<00:25, 2628277.06tokens/s]
Shard 24:  32%|███▏      | 31998196/100000000 [00:10<00:26, 2582291.52tokens/s]
Shard 24:  32%|███▏      | 32259577/100000000 [00:10<00:26, 2510636.67tokens/s]
Shard 24:  33%|███▎      | 32513676/100000000 [00:10<00:27, 2480777.52tokens/s]
Shard 24:  33%|███▎      | 32838000/100000000 [00:10<00:24, 2694659.85tokens/s]
Shard 24:  33%|███▎      | 33161969/100000000 [00:10<00:23, 2843927.50tokens/s]
Shard 24:  34%|███▎      | 33538345/100000000 [00:10<00:21, 3105557.04tokens/s]
Shard 24:  34%|███▍      | 33851900/100000000 [00:10<00:21, 3063075.32tokens/s]
Shard 24:  34%|███▍      | 34173288/100000000 [00:10<00:21, 3091511.30tokens/s]
Shard 24:  35%|███▍      | 34696636/100000000 [00:10<00:17, 3716889.70tokens/s]
Shard 24:  35%|███▌      | 35083152/100000000 [00:11<00:17, 3734650.08tokens/s]
Shard 24:  35%|███▌      | 35459910/100000000 [00:11<00:19, 3354078.80tokens/s]
Shard 24:  36%|███▌      | 35805146/100000000 [00:11<00:19, 3245969.65tokens/s]
Shard 24:  36%|███▌      | 36138900/100000000 [00:11<00:19, 3253258.13tokens/s]
Shard 24:  37%|███▋      | 36635756/100000000 [00:11<00:17, 3727043.00tokens/s]
Shard 24:  37%|███▋      | 37015256/100000000 [00:11<00:18, 3389589.41tokens/s]
Shard 24:  37%|███▋      | 37377169/100000000 [00:11<00:19, 3263632.85tokens/s]
Shard 24:  38%|███▊      | 37775752/100000000 [00:11<00:18, 3449186.29tokens/s]
Shard 24:  38%|███▊      | 38128019/100000000 [00:12<00:19, 3172700.98tokens/s]
Shard 24:  38%|███▊      | 38454486/100000000 [00:12<00:19, 3140442.19tokens/s]
Shard 24:  39%|███▉      | 38822539/100000000 [00:12<00:18, 3271892.96tokens/s]
Shard 24:  39%|███▉      | 39378862/100000000 [00:12<00:15, 3887293.05tokens/s]
Shard 24:  40%|███▉      | 39776602/100000000 [00:12<00:16, 3572279.24tokens/s]
Shard 24:  40%|████      | 40144964/100000000 [00:12<00:21, 2778228.19tokens/s]
Shard 24:  40%|████      | 40454887/100000000 [00:12<00:21, 2778791.80tokens/s]
Shard 24:  41%|████      | 40755240/100000000 [00:12<00:20, 2823403.65tokens/s]
Shard 24:  41%|████      | 41054652/100000000 [00:12<00:20, 2855221.08tokens/s]
Shard 24:  41%|████▏     | 41353031/100000000 [00:13<00:22, 2653288.60tokens/s]
Shard 24:  42%|████▏     | 41628584/100000000 [00:13<00:22, 2593024.27tokens/s]
Shard 24:  42%|████▏     | 41898687/100000000 [00:13<00:22, 2564584.34tokens/s]
Shard 24:  42%|████▏     | 42168368/100000000 [00:13<00:23, 2410751.01tokens/s]
Shard 24:  42%|████▏     | 42436735/100000000 [00:13<00:23, 2481233.03tokens/s]
Shard 24:  43%|████▎     | 42689258/100000000 [00:13<00:23, 2467854.85tokens/s]
Shard 24:  43%|████▎     | 42952431/100000000 [00:13<00:22, 2509265.55tokens/s]
Shard 24:  43%|████▎     | 43220321/100000000 [00:13<00:22, 2555333.13tokens/s]
Shard 24:  43%|████▎     | 43479295/100000000 [00:14<00:25, 2223229.52tokens/s]
Shard 24:  44%|████▍     | 43885448/100000000 [00:14<00:20, 2701589.57tokens/s]
Shard 24:  44%|████▍     | 44394309/100000000 [00:14<00:16, 3339782.85tokens/s]
Shard 24:  45%|████▍     | 44742651/100000000 [00:14<00:17, 3246617.83tokens/s]
Shard 24:  45%|████▌     | 45077934/100000000 [00:14<00:17, 3194345.87tokens/s]
Shard 24:  46%|████▌     | 45508881/100000000 [00:14<00:15, 3482292.68tokens/s]
Shard 24:  46%|████▌     | 45932831/100000000 [00:14<00:14, 3663665.64tokens/s]
Shard 24:  46%|████▋     | 46305388/100000000 [00:14<00:16, 3353188.93tokens/s]
Shard 24:  47%|████▋     | 46737035/100000000 [00:14<00:14, 3574623.75tokens/s]
Shard 24:  47%|████▋     | 47134258/100000000 [00:14<00:14, 3676808.32tokens/s]
Shard 24:  48%|████▊     | 47508016/100000000 [00:15<00:14, 3533550.64tokens/s]
Shard 24:  48%|████▊     | 47866226/100000000 [00:15<00:15, 3387023.76tokens/s]
Shard 24:  48%|████▊     | 48230498/100000000 [00:15<00:15, 3447635.59tokens/s]
Shard 24:  49%|████▊     | 48579478/100000000 [00:15<00:14, 3437430.91tokens/s]
Shard 24:  49%|████▉     | 48925816/100000000 [00:15<00:15, 3204325.83tokens/s]
Shard 24:  50%|████▉     | 49877237/100000000 [00:15<00:10, 4890397.76tokens/s]
Shard 24:  50%|█████     | 50380127/100000000 [00:15<00:11, 4244914.98tokens/s]
Shard 24:  51%|█████     | 50827524/100000000 [00:15<00:11, 4281617.47tokens/s]
Shard 24:  51%|█████▏    | 51272967/100000000 [00:16<00:12, 3812582.58tokens/s]
Shard 24:  52%|█████▏    | 51707233/100000000 [00:16<00:13, 3645019.83tokens/s]
Shard 24:  52%|█████▏    | 52085474/100000000 [00:16<00:13, 3517082.06tokens/s]
Shard 24:  53%|█████▎    | 52573385/100000000 [00:16<00:12, 3842962.21tokens/s]
Shard 24:  53%|█████▎    | 52969600/100000000 [00:16<00:13, 3484819.21tokens/s]
Shard 24:  53%|█████▎    | 53330370/100000000 [00:16<00:14, 3302343.65tokens/s]
Shard 24:  54%|█████▍    | 53775127/100000000 [00:16<00:12, 3589200.94tokens/s]
Shard 24:  54%|█████▍    | 54235416/100000000 [00:16<00:11, 3852363.28tokens/s]
Shard 24:  55%|█████▍    | 54631766/100000000 [00:16<00:11, 3787201.16tokens/s]
Shard 24:  55%|█████▌    | 55060422/100000000 [00:17<00:11, 3888959.18tokens/s]
Shard 24:  56%|█████▌    | 55515837/100000000 [00:17<00:10, 4066908.80tokens/s]
Shard 24:  56%|█████▌    | 55928028/100000000 [00:17<00:11, 3772554.11tokens/s]
Shard 24:  56%|█████▋    | 56313486/100000000 [00:17<00:12, 3433841.27tokens/s]
Shard 24:  57%|█████▋    | 56666424/100000000 [00:17<00:12, 3391719.53tokens/s]
Shard 24:  57%|█████▋    | 57012411/100000000 [00:17<00:12, 3381365.09tokens/s]
Shard 24:  58%|█████▊    | 57531609/100000000 [00:17<00:11, 3859669.51tokens/s]
Shard 24:  58%|█████▊    | 57939361/100000000 [00:17<00:10, 3907668.49tokens/s]
Shard 24:  58%|█████▊    | 58335166/100000000 [00:17<00:10, 3869631.98tokens/s]
Shard 24:  59%|█████▊    | 58725593/100000000 [00:18<00:12, 3345281.43tokens/s]
Shard 24:  60%|█████▉    | 59984874/100000000 [00:18<00:07, 5673132.65tokens/s]
Shard 24:  61%|██████    | 60586220/100000000 [00:18<00:07, 5174119.01tokens/s]
Shard 24:  61%|██████    | 61133892/100000000 [00:18<00:07, 4895107.86tokens/s]
Shard 24:  62%|██████▏   | 61662617/100000000 [00:18<00:07, 4980764.05tokens/s]
Shard 24:  62%|██████▏   | 62177030/100000000 [00:18<00:07, 5005998.83tokens/s]
Shard 24:  63%|██████▎   | 62689799/100000000 [00:18<00:07, 4734608.85tokens/s]
Shard 24:  63%|██████▎   | 63173298/100000000 [00:18<00:08, 4545182.90tokens/s]
Shard 24:  64%|██████▎   | 63635366/100000000 [00:19<00:08, 4426483.28tokens/s]
Shard 24:  64%|██████▍   | 64083024/100000000 [00:19<00:09, 3969607.14tokens/s]
Shard 24:  64%|██████▍   | 64491137/100000000 [00:19<00:09, 3866806.26tokens/s]
Shard 24:  65%|██████▍   | 64957264/100000000 [00:19<00:08, 4064952.85tokens/s]
Shard 24:  65%|██████▌   | 65403360/100000000 [00:19<00:08, 4145545.21tokens/s]
Shard 24:  66%|██████▌   | 65824344/100000000 [00:19<00:08, 3897531.28tokens/s]
Shard 24:  66%|██████▌   | 66219913/100000000 [00:19<00:09, 3692353.29tokens/s]
Shard 24:  67%|██████▋   | 66671061/100000000 [00:19<00:08, 3902414.47tokens/s]
Shard 24:  67%|██████▋   | 67113400/100000000 [00:19<00:08, 4042688.16tokens/s]
Shard 24:  68%|██████▊   | 67664752/100000000 [00:20<00:07, 4444757.06tokens/s]
Shard 24:  68%|██████▊   | 68121233/100000000 [00:20<00:07, 4465782.16tokens/s]
Shard 24:  69%|██████▊   | 68572335/100000000 [00:20<00:07, 4389761.53tokens/s]
Shard 24:  69%|██████▉   | 69037299/100000000 [00:20<00:06, 4440072.36tokens/s]
Shard 24:  70%|██████▉   | 69517226/100000000 [00:20<00:06, 4544386.93tokens/s]
Shard 24:  70%|███████   | 70006144/100000000 [00:20<00:06, 4632997.41tokens/s]
Shard 24:  71%|███████   | 70549075/100000000 [00:20<00:06, 4833681.07tokens/s]
Shard 24:  71%|███████   | 71169223/100000000 [00:20<00:05, 5228142.82tokens/s]
Shard 24:  72%|███████▏  | 71693916/100000000 [00:20<00:06, 4713604.66tokens/s]
Shard 24:  72%|███████▏  | 72271018/100000000 [00:21<00:05, 4987160.49tokens/s]
Shard 24:  73%|███████▎  | 72779332/100000000 [00:21<00:05, 4832026.41tokens/s]
Shard 24:  73%|███████▎  | 73269563/100000000 [00:21<00:06, 4405107.75tokens/s]
Shard 24:  74%|███████▎  | 73721176/100000000 [00:21<00:06, 4354097.72tokens/s]
Shard 24:  74%|███████▍  | 74165903/100000000 [00:21<00:06, 3978018.03tokens/s]
Shard 24:  75%|███████▍  | 74700042/100000000 [00:21<00:05, 4260269.32tokens/s]
Shard 24:  75%|███████▌  | 75136348/100000000 [00:21<00:06, 3893372.04tokens/s]
Shard 24:  76%|███████▌  | 75542924/100000000 [00:21<00:06, 3925948.17tokens/s]
Shard 24:  76%|███████▌  | 75944034/100000000 [00:21<00:06, 3813896.88tokens/s]
Shard 24:  76%|███████▋  | 76330640/100000000 [00:22<00:06, 3734660.71tokens/s]
Shard 24:  77%|███████▋  | 76707614/100000000 [00:22<00:06, 3589421.56tokens/s]
Shard 24:  77%|███████▋  | 77091856/100000000 [00:22<00:06, 3648386.03tokens/s]
Shard 24:  78%|███████▊  | 77617722/100000000 [00:22<00:05, 4090516.33tokens/s]
Shard 24:  78%|███████▊  | 78135735/100000000 [00:22<00:04, 4392676.20tokens/s]
Shard 24:  79%|███████▊  | 78700683/100000000 [00:22<00:04, 4755926.98tokens/s]
Shard 24:  79%|███████▉  | 79194333/100000000 [00:22<00:05, 3698924.08tokens/s]
Shard 24:  80%|███████▉  | 79603711/100000000 [00:22<00:06, 3333453.04tokens/s]
Shard 24:  80%|███████▉  | 79984392/100000000 [00:23<00:05, 3433552.66tokens/s]
Shard 24:  80%|████████  | 80353753/100000000 [00:23<00:05, 3315550.64tokens/s]
Shard 24:  81%|████████  | 80704357/100000000 [00:23<00:05, 3304645.76tokens/s]
Shard 24:  81%|████████  | 81231708/100000000 [00:23<00:04, 3816794.73tokens/s]
Shard 24:  82%|████████▏ | 81635324/100000000 [00:23<00:04, 3741302.83tokens/s]
Shard 24:  82%|████████▏ | 82132403/100000000 [00:23<00:04, 4065839.56tokens/s]
Shard 24:  83%|████████▎ | 82630731/100000000 [00:23<00:04, 4312605.52tokens/s]
Shard 24:  84%|████████▎ | 83623435/100000000 [00:23<00:02, 5909238.11tokens/s]
Shard 24:  84%|████████▍ | 84286804/100000000 [00:24<00:03, 4866583.54tokens/s]
Shard 24:  85%|████████▍ | 84816354/100000000 [00:24<00:03, 4365712.82tokens/s]
Shard 24:  85%|████████▌ | 85331495/100000000 [00:24<00:03, 4534191.20tokens/s]
Shard 24:  86%|████████▌ | 85815540/100000000 [00:24<00:03, 4525454.31tokens/s]
Shard 24:  86%|████████▋ | 86386435/100000000 [00:24<00:02, 4820960.67tokens/s]
Shard 24:  87%|████████▋ | 86892559/100000000 [00:24<00:02, 4858899.58tokens/s]
Shard 24:  87%|████████▋ | 87394131/100000000 [00:24<00:02, 4642244.95tokens/s]
Shard 24:  88%|████████▊ | 87869037/100000000 [00:24<00:02, 4593352.22tokens/s]
Shard 24:  88%|████████▊ | 88335636/100000000 [00:24<00:02, 4583557.70tokens/s]
Shard 24:  89%|████████▉ | 88800680/100000000 [00:25<00:02, 4110838.46tokens/s]
Shard 24:  89%|████████▉ | 89223018/100000000 [00:25<00:02, 4121529.80tokens/s]
Shard 24:  90%|████████▉ | 89643206/100000000 [00:25<00:02, 4053300.89tokens/s]
Shard 24:  90%|█████████ | 90171881/100000000 [00:25<00:02, 4379777.46tokens/s]
Shard 24:  91%|█████████ | 90838644/100000000 [00:25<00:01, 5009600.57tokens/s]
Shard 24:  92%|█████████▏| 91641954/100000000 [00:25<00:01, 5833363.82tokens/s]
Shard 24:  92%|█████████▏| 92233361/100000000 [00:25<00:01, 4923203.70tokens/s]
Shard 24:  93%|█████████▎| 92755195/100000000 [00:25<00:01, 4731650.01tokens/s]
Shard 24:  93%|█████████▎| 93248208/100000000 [00:25<00:01, 4403956.51tokens/s]
Shard 24:  94%|█████████▍| 93908970/100000000 [00:26<00:01, 4919050.27tokens/s]
Shard 24:  94%|█████████▍| 94419656/100000000 [00:26<00:01, 4929084.19tokens/s]
Shard 24:  95%|█████████▍| 94927619/100000000 [00:26<00:01, 4761680.83tokens/s]
Shard 24:  95%|█████████▌| 95413250/100000000 [00:26<00:00, 4696008.77tokens/s]
Shard 24:  96%|█████████▌| 96012773/100000000 [00:26<00:00, 5043186.04tokens/s]
Shard 24:  97%|█████████▋| 96525723/100000000 [00:26<00:00, 4593653.39tokens/s]
Shard 24:  97%|█████████▋| 96996911/100000000 [00:26<00:00, 4341550.73tokens/s]
Shard 24:  97%|█████████▋| 97440250/100000000 [00:26<00:00, 4193360.79tokens/s]
Shard 24:  98%|█████████▊| 97899662/100000000 [00:26<00:00, 4249953.43tokens/s]
Shard 24:  98%|█████████▊| 98329408/100000000 [00:27<00:00, 4104074.42tokens/s]
Shard 24:  99%|█████████▉| 98775767/100000000 [00:27<00:00, 4183174.84tokens/s]
Shard 24:  99%|█████████▉| 99197309/100000000 [00:27<00:00, 3813688.07tokens/s]
Shard 24: 100%|█████████▉| 99586211/100000000 [00:27<00:00, 3714595.38tokens/s]
Shard 24: 100%|█████████▉| 99962443/100000000 [00:27<00:00, 3501938.60tokens/s]
Shard 24: 100%|█████████▉| 99999971/100000000 [00:35<00:00, 2793285.76tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000024.bin (200,001,024 bytes) in the gpt-2 format

Shard 25:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 25:   0%|          | 357870/100000000 [00:00<00:28, 3516823.81tokens/s]
Shard 25:   1%|          | 872364/100000000 [00:00<00:22, 4440018.91tokens/s]
Shard 25:   1%|▏         | 1352256/100000000 [00:00<00:21, 4592369.90tokens/s]
Shard 25:   2%|▏         | 1811949/100000000 [00:00<00:31, 3123981.45tokens/s]
Shard 25:   2%|▏         | 2172665/100000000 [00:00<00:35, 2742139.71tokens/s]
Shard 25:   2%|▏         | 2481601/100000000 [00:00<00:40, 2422405.00tokens/s]
Shard 25:   3%|▎         | 2755893/100000000 [00:00<00:40, 2396498.40tokens/s]
Shard 25:   3%|▎         | 3029473/100000000 [00:01<00:39, 2475696.00tokens/s]
Shard 25:   3%|▎         | 3294556/100000000 [00:01<00:44, 2160094.70tokens/s]
Shard 25:   4%|▎         | 3524430/100000000 [00:01<00:45, 2098284.86tokens/s]
Shard 25:   4%|▎         | 3743229/100000000 [00:01<00:47, 2008713.59tokens/s]
Shard 25:   4%|▍         | 3979225/100000000 [00:01<00:45, 2089829.02tokens/s]
Shard 25:   4%|▍         | 4201421/100000000 [00:01<00:50, 1902901.61tokens/s]
Shard 25:   4%|▍         | 4474696/100000000 [00:01<00:45, 2109001.43tokens/s]
Shard 25:   5%|▍         | 4703236/100000000 [00:01<00:44, 2150464.83tokens/s]
Shard 25:   5%|▍         | 4979317/100000000 [00:02<00:45, 2088984.67tokens/s]
Shard 25:   5%|▌         | 5198373/100000000 [00:02<00:45, 2094324.34tokens/s]
Shard 25:   5%|▌         | 5412050/100000000 [00:02<00:48, 1931604.88tokens/s]
Shard 25:   6%|▌         | 5609302/100000000 [00:02<00:49, 1903143.97tokens/s]
Shard 25:   6%|▌         | 5805487/100000000 [00:02<00:50, 1866048.08tokens/s]
Shard 25:   6%|▌         | 6057897/100000000 [00:02<00:46, 2041760.49tokens/s]
Shard 25:   6%|▋         | 6266133/100000000 [00:02<00:53, 1745249.36tokens/s]
Shard 25:   7%|▋         | 6562693/100000000 [00:02<00:45, 2033898.50tokens/s]
Shard 25:   7%|▋         | 6801880/100000000 [00:02<00:44, 2098627.02tokens/s]
Shard 25:   7%|▋         | 7066248/100000000 [00:03<00:41, 2228870.55tokens/s]
Shard 25:   7%|▋         | 7313310/100000000 [00:03<00:40, 2295486.08tokens/s]
Shard 25:   8%|▊         | 7555576/100000000 [00:03<00:39, 2331527.58tokens/s]
Shard 25:   8%|▊         | 7792777/100000000 [00:03<00:40, 2288187.52tokens/s]
Shard 25:   8%|▊         | 8024489/100000000 [00:03<00:40, 2261085.64tokens/s]
Shard 25:   8%|▊         | 8400639/100000000 [00:03<00:34, 2676617.64tokens/s]
Shard 25:   9%|▊         | 8681020/100000000 [00:03<00:35, 2582684.59tokens/s]
Shard 25:   9%|▉         | 9011977/100000000 [00:03<00:32, 2760925.32tokens/s]
Shard 25:   9%|▉         | 9290618/100000000 [00:03<00:33, 2724010.26tokens/s]
Shard 25:  10%|▉         | 9579606/100000000 [00:04<00:32, 2746136.15tokens/s]
Shard 25:  10%|▉         | 9895009/100000000 [00:04<00:31, 2854953.31tokens/s]
Shard 25:  10%|█         | 10186060/100000000 [00:04<00:34, 2633723.12tokens/s]
Shard 25:  10%|█         | 10453682/100000000 [00:04<00:38, 2350428.77tokens/s]
Shard 25:  11%|█         | 10697003/100000000 [00:04<00:38, 2294997.62tokens/s]
Shard 25:  11%|█         | 10931462/100000000 [00:04<00:43, 2037732.39tokens/s]
Shard 25:  11%|█         | 11143200/100000000 [00:04<00:44, 1996760.13tokens/s]
Shard 25:  11%|█▏        | 11347869/100000000 [00:04<00:45, 1939813.83tokens/s]
Shard 25:  12%|█▏        | 11561847/100000000 [00:05<00:44, 1978396.11tokens/s]
Shard 25:  12%|█▏        | 11813985/100000000 [00:05<00:41, 2121830.77tokens/s]
Shard 25:  12%|█▏        | 12031829/100000000 [00:05<00:42, 2059067.46tokens/s]
Shard 25:  12%|█▏        | 12240807/100000000 [00:05<00:44, 1986775.59tokens/s]
Shard 25:  12%|█▏        | 12457743/100000000 [00:05<00:43, 2030653.86tokens/s]
Shard 25:  13%|█▎        | 12771957/100000000 [00:05<00:37, 2341269.13tokens/s]
Shard 25:  13%|█▎        | 13009432/100000000 [00:05<00:37, 2303133.80tokens/s]
Shard 25:  13%|█▎        | 13282936/100000000 [00:05<00:35, 2417981.19tokens/s]
Shard 25:  14%|█▎        | 13537469/100000000 [00:05<00:35, 2453515.66tokens/s]
Shard 25:  14%|█▍        | 13784344/100000000 [00:05<00:36, 2394202.80tokens/s]
Shard 25:  14%|█▍        | 14025551/100000000 [00:06<00:37, 2273677.69tokens/s]
Shard 25:  14%|█▍        | 14318965/100000000 [00:06<00:34, 2448334.99tokens/s]
Shard 25:  15%|█▍        | 14634634/100000000 [00:06<00:32, 2611120.59tokens/s]
Shard 25:  15%|█▍        | 14897661/100000000 [00:06<00:34, 2485241.00tokens/s]
Shard 25:  15%|█▌        | 15245268/100000000 [00:06<00:31, 2727402.55tokens/s]
Shard 25:  16%|█▌        | 15520465/100000000 [00:06<00:38, 2179093.56tokens/s]
Shard 25:  16%|█▌        | 15881812/100000000 [00:06<00:33, 2528661.45tokens/s]
Shard 25:  16%|█▋        | 16297008/100000000 [00:06<00:28, 2945230.69tokens/s]
Shard 25:  17%|█▋        | 16682133/100000000 [00:07<00:26, 3188494.32tokens/s]
Shard 25:  17%|█▋        | 17036034/100000000 [00:07<00:25, 3285515.27tokens/s]
Shard 25:  17%|█▋        | 17416554/100000000 [00:07<00:24, 3432938.61tokens/s]
Shard 25:  18%|█▊        | 17779624/100000000 [00:07<00:23, 3489018.61tokens/s]
Shard 25:  18%|█▊        | 18253716/100000000 [00:07<00:21, 3847351.82tokens/s]
Shard 25:  19%|█▊        | 18645109/100000000 [00:07<00:21, 3708568.75tokens/s]
Shard 25:  19%|█▉        | 19021670/100000000 [00:07<00:24, 3321681.81tokens/s]
Shard 25:  19%|█▉        | 19381950/100000000 [00:07<00:24, 3350793.46tokens/s]
Shard 25:  20%|█▉        | 19750319/100000000 [00:07<00:23, 3433448.43tokens/s]
Shard 25:  20%|██        | 20100921/100000000 [00:07<00:24, 3299548.81tokens/s]
Shard 25:  20%|██        | 20440072/100000000 [00:08<00:24, 3295078.63tokens/s]
Shard 25:  21%|██        | 20855691/100000000 [00:08<00:22, 3528962.08tokens/s]
Shard 25:  21%|██▏       | 21364219/100000000 [00:08<00:19, 3974759.82tokens/s]
Shard 25:  22%|██▏       | 21766633/100000000 [00:08<00:20, 3769146.35tokens/s]
Shard 25:  22%|██▏       | 22148778/100000000 [00:08<00:24, 3117249.20tokens/s]
Shard 25:  22%|██▏       | 22481401/100000000 [00:08<00:24, 3158066.26tokens/s]
Shard 25:  23%|██▎       | 22812823/100000000 [00:08<00:26, 2959240.63tokens/s]
Shard 25:  23%|██▎       | 23154484/100000000 [00:08<00:24, 3076579.96tokens/s]
Shard 25:  24%|██▎       | 23526716/100000000 [00:09<00:23, 3242326.26tokens/s]
Shard 25:  24%|██▍       | 23986801/100000000 [00:09<00:21, 3618094.53tokens/s]
Shard 25:  24%|██▍       | 24360215/100000000 [00:09<00:21, 3571828.33tokens/s]
Shard 25:  25%|██▍       | 24761098/100000000 [00:09<00:20, 3694984.50tokens/s]
Shard 25:  25%|██▌       | 25136068/100000000 [00:09<00:21, 3458969.23tokens/s]
Shard 25:  26%|██▌       | 25508394/100000000 [00:09<00:21, 3512189.68tokens/s]
Shard 25:  26%|██▌       | 25870512/100000000 [00:09<00:20, 3539092.55tokens/s]
Shard 25:  26%|██▋       | 26403392/100000000 [00:09<00:18, 4047292.77tokens/s]
Shard 25:  27%|██▋       | 26814543/100000000 [00:09<00:18, 4007041.37tokens/s]
Shard 25:  27%|██▋       | 27218935/100000000 [00:09<00:18, 3995967.53tokens/s]
Shard 25:  28%|██▊       | 27621579/100000000 [00:10<00:18, 3980239.26tokens/s]
Shard 25:  28%|██▊       | 28021940/100000000 [00:10<00:18, 3872172.02tokens/s]
Shard 25:  28%|██▊       | 28411161/100000000 [00:10<00:20, 3568695.67tokens/s]
Shard 25:  29%|██▉       | 28774364/100000000 [00:10<00:19, 3567782.02tokens/s]
Shard 25:  29%|██▉       | 29137693/100000000 [00:10<00:21, 3344602.43tokens/s]
Shard 25:  29%|██▉       | 29476974/100000000 [00:10<00:22, 3131905.86tokens/s]
Shard 25:  30%|██▉       | 29794748/100000000 [00:10<00:23, 3009595.74tokens/s]
Shard 25:  30%|███       | 30099240/100000000 [00:10<00:23, 2964448.80tokens/s]
Shard 25:  30%|███       | 30478357/100000000 [00:10<00:22, 3149417.20tokens/s]
Shard 25:  31%|███       | 30796681/100000000 [00:11<00:22, 3025682.90tokens/s]
Shard 25:  31%|███       | 31111536/100000000 [00:11<00:23, 2915252.36tokens/s]
Shard 25:  31%|███▏      | 31422786/100000000 [00:11<00:23, 2955133.55tokens/s]
Shard 25:  32%|███▏      | 31720149/100000000 [00:11<00:23, 2892399.96tokens/s]
Shard 25:  32%|███▏      | 32010650/100000000 [00:11<00:24, 2738316.28tokens/s]
Shard 25:  32%|███▏      | 32289341/100000000 [00:11<00:25, 2611851.46tokens/s]
Shard 25:  33%|███▎      | 32713953/100000000 [00:11<00:22, 3042246.99tokens/s]
Shard 25:  33%|███▎      | 33024174/100000000 [00:11<00:25, 2676959.42tokens/s]
Shard 25:  33%|███▎      | 33303780/100000000 [00:12<00:25, 2595938.49tokens/s]
Shard 25:  34%|███▎      | 33570665/100000000 [00:12<00:27, 2384160.27tokens/s]
Shard 25:  34%|███▍      | 33829317/100000000 [00:12<00:27, 2429875.89tokens/s]
Shard 25:  34%|███▍      | 34077915/100000000 [00:12<00:27, 2400570.19tokens/s]
Shard 25:  34%|███▍      | 34322868/100000000 [00:12<00:28, 2318948.10tokens/s]
Shard 25:  35%|███▍      | 34557378/100000000 [00:12<00:28, 2274162.60tokens/s]
Shard 25:  35%|███▍      | 34786456/100000000 [00:12<00:29, 2204625.20tokens/s]
Shard 25:  35%|███▌      | 35008156/100000000 [00:12<00:30, 2166293.40tokens/s]
Shard 25:  35%|███▌      | 35261531/100000000 [00:12<00:28, 2268488.49tokens/s]
Shard 25:  36%|███▌      | 35578683/100000000 [00:13<00:25, 2518758.23tokens/s]
Shard 25:  36%|███▌      | 35838985/100000000 [00:13<00:25, 2533688.46tokens/s]
Shard 25:  36%|███▌      | 36101733/100000000 [00:13<00:24, 2559166.68tokens/s]
Shard 25:  36%|███▋      | 36401755/100000000 [00:13<00:23, 2684604.05tokens/s]
Shard 25:  37%|███▋      | 36750641/100000000 [00:13<00:21, 2920191.83tokens/s]
Shard 25:  37%|███▋      | 37069041/100000000 [00:13<00:21, 2978763.75tokens/s]
Shard 25:  37%|███▋      | 37390610/100000000 [00:13<00:20, 3041969.43tokens/s]
Shard 25:  38%|███▊      | 37696909/100000000 [00:13<00:21, 2927428.56tokens/s]
Shard 25:  38%|███▊      | 38012187/100000000 [00:13<00:20, 2975921.88tokens/s]
Shard 25:  38%|███▊      | 38310929/100000000 [00:13<00:21, 2807535.80tokens/s]
Shard 25:  39%|███▊      | 38594260/100000000 [00:14<00:22, 2716733.89tokens/s]
Shard 25:  39%|███▉      | 38905862/100000000 [00:14<00:21, 2799733.62tokens/s]
Shard 25:  39%|███▉      | 39241075/100000000 [00:14<00:20, 2935572.89tokens/s]
Shard 25:  40%|███▉      | 39591770/100000000 [00:14<00:19, 3088047.00tokens/s]
Shard 25:  40%|███▉      | 39902905/100000000 [00:14<00:20, 2884047.98tokens/s]
Shard 25:  40%|████      | 40199676/100000000 [00:14<00:23, 2549894.94tokens/s]
Shard 25:  40%|████      | 40463590/100000000 [00:14<00:25, 2308666.22tokens/s]
Shard 25:  41%|████      | 40786603/100000000 [00:14<00:23, 2534466.81tokens/s]
Shard 25:  41%|████      | 41057347/100000000 [00:15<00:22, 2563985.76tokens/s]
Shard 25:  41%|████▏     | 41321560/100000000 [00:15<00:22, 2575397.68tokens/s]
Shard 25:  42%|████▏     | 41591585/100000000 [00:15<00:22, 2594825.55tokens/s]
Shard 25:  42%|████▏     | 41931164/100000000 [00:15<00:20, 2804650.14tokens/s]
Shard 25:  42%|████▏     | 42217357/100000000 [00:15<00:21, 2686649.80tokens/s]
Shard 25:  43%|████▎     | 42568544/100000000 [00:15<00:19, 2910607.58tokens/s]
Shard 25:  43%|████▎     | 42864171/100000000 [00:15<00:21, 2696340.51tokens/s]
Shard 25:  43%|████▎     | 43171689/100000000 [00:15<00:21, 2659138.05tokens/s]
Shard 25:  44%|████▍     | 43866249/100000000 [00:15<00:14, 3787214.64tokens/s]
Shard 25:  44%|████▍     | 44259945/100000000 [00:16<00:15, 3563158.85tokens/s]
Shard 25:  45%|████▍     | 44626703/100000000 [00:16<00:17, 3150023.11tokens/s]
Shard 25:  45%|████▍     | 44960041/100000000 [00:16<00:17, 3163269.50tokens/s]
Shard 25:  45%|████▌     | 45288428/100000000 [00:16<00:18, 2918285.94tokens/s]
Shard 25:  46%|████▌     | 45601218/100000000 [00:16<00:18, 2969215.52tokens/s]
Shard 25:  46%|████▌     | 45905422/100000000 [00:16<00:18, 2899752.74tokens/s]
Shard 25:  46%|████▌     | 46209817/100000000 [00:16<00:18, 2937960.24tokens/s]
Shard 25:  47%|████▋     | 46604111/100000000 [00:16<00:16, 3200480.37tokens/s]
Shard 25:  47%|████▋     | 47081819/100000000 [00:16<00:14, 3641895.93tokens/s]
Shard 25:  47%|████▋     | 47454712/100000000 [00:17<00:15, 3398084.22tokens/s]
Shard 25:  48%|████▊     | 47801480/100000000 [00:17<00:16, 3255348.78tokens/s]
Shard 25:  48%|████▊     | 48132370/100000000 [00:17<00:17, 2945281.56tokens/s]
Shard 25:  48%|████▊     | 48439341/100000000 [00:17<00:17, 2976205.56tokens/s]
Shard 25:  49%|████▉     | 48781035/100000000 [00:17<00:16, 3089122.18tokens/s]
Shard 25:  49%|████▉     | 49240420/100000000 [00:17<00:14, 3505376.59tokens/s]
Shard 25:  50%|████▉     | 49597826/100000000 [00:17<00:15, 3218793.70tokens/s]
Shard 25:  50%|████▉     | 49958715/100000000 [00:17<00:15, 3315280.67tokens/s]
Shard 25:  50%|█████     | 50321570/100000000 [00:17<00:14, 3389121.85tokens/s]
Shard 25:  51%|█████     | 50764406/100000000 [00:18<00:13, 3678717.14tokens/s]
Shard 25:  51%|█████     | 51193179/100000000 [00:18<00:12, 3839678.08tokens/s]
Shard 25:  52%|█████▏    | 51632884/100000000 [00:18<00:12, 4001439.55tokens/s]
Shard 25:  52%|█████▏    | 52085887/100000000 [00:18<00:11, 4152848.70tokens/s]
Shard 25:  53%|█████▎    | 52503946/100000000 [00:18<00:12, 3937160.08tokens/s]
Shard 25:  53%|█████▎    | 53065529/100000000 [00:18<00:10, 4407377.69tokens/s]
Shard 25:  54%|█████▎    | 53530426/100000000 [00:18<00:10, 4442375.01tokens/s]
Shard 25:  54%|█████▍    | 54044925/100000000 [00:18<00:09, 4646411.51tokens/s]
Shard 25:  55%|█████▍    | 54513726/100000000 [00:18<00:09, 4619664.59tokens/s]
Shard 25:  55%|█████▍    | 54977915/100000000 [00:19<00:10, 4183766.96tokens/s]
Shard 25:  55%|█████▌    | 55406128/100000000 [00:19<00:10, 4147666.50tokens/s]
Shard 25:  56%|█████▌    | 55830091/100000000 [00:19<00:10, 4136381.15tokens/s]
Shard 25:  56%|█████▌    | 56248712/100000000 [00:19<00:11, 3916565.23tokens/s]
Shard 25:  57%|█████▋    | 57051016/100000000 [00:19<00:08, 5029830.53tokens/s]
Shard 25:  58%|█████▊    | 57565694/100000000 [00:19<00:08, 4806769.79tokens/s]
Shard 25:  58%|█████▊    | 58057010/100000000 [00:19<00:09, 4277268.24tokens/s]
Shard 25:  59%|█████▊    | 58715795/100000000 [00:19<00:08, 4865475.16tokens/s]
Shard 25:  59%|█████▉    | 59222545/100000000 [00:19<00:09, 4275157.84tokens/s]
Shard 25:  60%|█████▉    | 59674478/100000000 [00:20<00:10, 4006965.53tokens/s]
Shard 25:  60%|██████    | 60249921/100000000 [00:20<00:08, 4429893.90tokens/s]
Shard 25:  61%|██████    | 60714346/100000000 [00:20<00:08, 4422578.37tokens/s]
Shard 25:  61%|██████▏   | 61274996/100000000 [00:20<00:08, 4698022.95tokens/s]
Shard 25:  62%|██████▏   | 61758153/100000000 [00:20<00:08, 4725735.70tokens/s]
Shard 25:  62%|██████▏   | 62247397/100000000 [00:20<00:07, 4737256.22tokens/s]
Shard 25:  63%|██████▎   | 62808416/100000000 [00:20<00:07, 4976243.40tokens/s]
Shard 25:  63%|██████▎   | 63311751/100000000 [00:20<00:07, 4743329.96tokens/s]
Shard 25:  64%|██████▍   | 63791903/100000000 [00:20<00:07, 4653261.49tokens/s]
Shard 25:  64%|██████▍   | 64261321/100000000 [00:21<00:09, 3813906.65tokens/s]
Shard 25:  65%|██████▍   | 64670068/100000000 [00:21<00:10, 3453930.70tokens/s]
Shard 25:  65%|██████▌   | 65119666/100000000 [00:21<00:09, 3679275.44tokens/s]
Shard 25:  66%|██████▌   | 65507827/100000000 [00:21<00:09, 3537125.30tokens/s]
Shard 25:  66%|██████▌   | 65875594/100000000 [00:21<00:10, 3158304.81tokens/s]
Shard 25:  66%|██████▋   | 66278675/100000000 [00:21<00:10, 3363649.74tokens/s]
Shard 25:  67%|██████▋   | 66633057/100000000 [00:21<00:11, 2921250.37tokens/s]
Shard 25:  67%|██████▋   | 67010911/100000000 [00:22<00:10, 3113407.38tokens/s]
Shard 25:  67%|██████▋   | 67339632/100000000 [00:22<00:11, 2895444.04tokens/s]
Shard 25:  68%|██████▊   | 67642439/100000000 [00:22<00:11, 2819326.80tokens/s]
Shard 25:  68%|██████▊   | 67935749/100000000 [00:22<00:11, 2833580.24tokens/s]
Shard 25:  68%|██████▊   | 68302165/100000000 [00:22<00:10, 3023685.91tokens/s]
Shard 25:  69%|██████▊   | 68618863/100000000 [00:22<00:10, 3027638.02tokens/s]
Shard 25:  69%|██████▉   | 68926539/100000000 [00:22<00:11, 2780030.66tokens/s]
Shard 25:  69%|██████▉   | 69221217/100000000 [00:22<00:11, 2587052.65tokens/s]
Shard 25:  69%|██████▉   | 69486048/100000000 [00:22<00:12, 2400098.85tokens/s]
Shard 25:  70%|██████▉   | 69774239/100000000 [00:23<00:12, 2515284.72tokens/s]
Shard 25:  70%|███████   | 70094797/100000000 [00:23<00:11, 2698817.88tokens/s]
Shard 25:  71%|███████   | 70624937/100000000 [00:23<00:08, 3413324.53tokens/s]
Shard 25:  71%|███████▏  | 71295312/100000000 [00:23<00:06, 4336367.96tokens/s]
Shard 25:  72%|███████▏  | 71742043/100000000 [00:23<00:06, 4248084.19tokens/s]
Shard 25:  72%|███████▏  | 72178076/100000000 [00:23<00:06, 4202673.57tokens/s]
Shard 25:  73%|███████▎  | 72624472/100000000 [00:23<00:06, 4258816.98tokens/s]
Shard 25:  73%|███████▎  | 73056348/100000000 [00:23<00:06, 4122957.72tokens/s]
Shard 25:  74%|███████▎  | 73510850/100000000 [00:23<00:06, 4231776.36tokens/s]
Shard 25:  74%|███████▍  | 73937663/100000000 [00:24<00:06, 4057631.42tokens/s]
Shard 25:  74%|███████▍  | 74348088/100000000 [00:24<00:06, 3966491.97tokens/s]
Shard 25:  75%|███████▍  | 74828650/100000000 [00:24<00:05, 4199314.95tokens/s]
Shard 25:  75%|███████▌  | 75371142/100000000 [00:24<00:05, 4522649.23tokens/s]
Shard 25:  76%|███████▌  | 75826729/100000000 [00:24<00:05, 4177598.93tokens/s]
Shard 25:  76%|███████▋  | 76251366/100000000 [00:24<00:05, 4051794.48tokens/s]
Shard 25:  77%|███████▋  | 76673108/100000000 [00:24<00:05, 3931458.33tokens/s]
Shard 25:  77%|███████▋  | 77070453/100000000 [00:24<00:06, 3445476.84tokens/s]
Shard 25:  78%|███████▊  | 77594366/100000000 [00:24<00:05, 3901318.26tokens/s]
Shard 25:  78%|███████▊  | 78256080/100000000 [00:25<00:04, 4626338.83tokens/s]
Shard 25:  79%|███████▉  | 78895930/100000000 [00:25<00:04, 5096545.94tokens/s]
Shard 25:  80%|███████▉  | 79597538/100000000 [00:25<00:03, 5604488.83tokens/s]
Shard 25:  80%|████████  | 80214082/100000000 [00:25<00:03, 5764630.71tokens/s]
Shard 25:  81%|████████  | 80801701/100000000 [00:25<00:03, 5404840.73tokens/s]
Shard 25:  81%|████████▏ | 81354478/100000000 [00:25<00:03, 4785142.45tokens/s]
Shard 25:  82%|████████▏ | 82009475/100000000 [00:25<00:03, 5228435.89tokens/s]
Shard 25:  83%|████████▎ | 82552742/100000000 [00:25<00:03, 5131485.53tokens/s]
Shard 25:  83%|████████▎ | 83080947/100000000 [00:25<00:03, 4897316.42tokens/s]
Shard 25:  84%|████████▍ | 83872833/100000000 [00:26<00:02, 5708570.80tokens/s]
Shard 25:  84%|████████▍ | 84460556/100000000 [00:26<00:03, 5132877.32tokens/s]
Shard 25:  85%|████████▍ | 84994435/100000000 [00:26<00:03, 4714985.31tokens/s]
Shard 25:  85%|████████▌ | 85489799/100000000 [00:26<00:03, 4703931.38tokens/s]
Shard 25:  86%|████████▌ | 85973154/100000000 [00:26<00:03, 3909913.69tokens/s]
Shard 25:  86%|████████▋ | 86392013/100000000 [00:26<00:03, 3747172.38tokens/s]
Shard 25:  87%|████████▋ | 86784843/100000000 [00:26<00:03, 3464823.33tokens/s]
Shard 25:  87%|████████▋ | 87267342/100000000 [00:26<00:03, 3785285.29tokens/s]
Shard 25:  88%|████████▊ | 87663397/100000000 [00:27<00:03, 3589007.06tokens/s]
Shard 25:  88%|████████▊ | 88037802/100000000 [00:27<00:03, 3522065.59tokens/s]
Shard 25:  88%|████████▊ | 88485321/100000000 [00:27<00:03, 3767232.85tokens/s]
Shard 25:  89%|████████▉ | 89175312/100000000 [00:27<00:02, 4616034.97tokens/s]
Shard 25:  90%|████████▉ | 89810342/100000000 [00:27<00:02, 5043209.08tokens/s]
Shard 25:  90%|█████████ | 90327388/100000000 [00:27<00:02, 4487447.35tokens/s]
Shard 25:  91%|█████████ | 90797248/100000000 [00:27<00:02, 3871968.36tokens/s]
Shard 25:  91%|█████████ | 91210377/100000000 [00:27<00:02, 3818888.81tokens/s]
Shard 25:  92%|█████████▏| 92408683/100000000 [00:28<00:01, 5866056.68tokens/s]
Shard 25:  93%|█████████▎| 93318746/100000000 [00:28<00:00, 6693251.40tokens/s]
Shard 25:  94%|█████████▍| 94033734/100000000 [00:28<00:01, 5785631.99tokens/s]
Shard 25:  95%|█████████▍| 94664070/100000000 [00:28<00:01, 4844615.70tokens/s]
Shard 25:  95%|█████████▌| 95205262/100000000 [00:28<00:01, 4563756.17tokens/s]
Shard 25:  96%|█████████▌| 95700819/100000000 [00:28<00:00, 4419836.20tokens/s]
Shard 25:  96%|█████████▌| 96176577/100000000 [00:28<00:00, 3867532.27tokens/s]
Shard 25:  97%|█████████▋| 96608862/100000000 [00:29<00:00, 3966701.34tokens/s]
Shard 25:  97%|█████████▋| 97226761/100000000 [00:29<00:00, 4481759.04tokens/s]
Shard 25:  98%|█████████▊| 97700857/100000000 [00:29<00:00, 4351307.00tokens/s]
Shard 25:  98%|█████████▊| 98168856/100000000 [00:29<00:00, 4177974.46tokens/s]
Shard 25:  99%|█████████▊| 98599864/100000000 [00:29<00:00, 4080366.97tokens/s]
Shard 25:  99%|█████████▉| 99146441/100000000 [00:29<00:00, 4403231.58tokens/s]
Shard 25: 100%|█████████▉| 99599385/100000000 [00:29<00:00, 4019245.07tokens/s]
Shard 25: 100%|█████████▉| 99998369/100000000 [00:37<00:00, 2635065.78tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000025.bin (200,001,024 bytes) in the gpt-2 format

Shard 26:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 26:   0%|          | 181551/100000000 [00:00<00:55, 1800759.78tokens/s]
Shard 26:   0%|          | 458609/100000000 [00:00<00:42, 2357820.84tokens/s]
Shard 26:   1%|          | 844946/100000000 [00:00<00:32, 3037403.18tokens/s]
Shard 26:   1%|          | 1148895/100000000 [00:00<00:32, 2996625.34tokens/s]
Shard 26:   1%|▏         | 1470787/100000000 [00:00<00:32, 3035748.30tokens/s]
Shard 26:   2%|▏         | 1901284/100000000 [00:00<00:28, 3435909.95tokens/s]
Shard 26:   2%|▏         | 2245793/100000000 [00:00<00:31, 3062790.78tokens/s]
Shard 26:   3%|▎         | 2621028/100000000 [00:00<00:29, 3254792.18tokens/s]
Shard 26:   3%|▎         | 3015893/100000000 [00:00<00:28, 3455570.73tokens/s]
Shard 26:   4%|▎         | 3572186/100000000 [00:01<00:23, 4070948.06tokens/s]
Shard 26:   4%|▍         | 4217672/100000000 [00:01<00:20, 4732284.46tokens/s]
Shard 26:   5%|▍         | 4782510/100000000 [00:01<00:19, 4975833.00tokens/s]
Shard 26:   5%|▌         | 5284496/100000000 [00:01<00:19, 4827408.31tokens/s]
Shard 26:   6%|▌         | 5770836/100000000 [00:01<00:25, 3714250.82tokens/s]
Shard 26:   6%|▋         | 6267798/100000000 [00:01<00:23, 3991248.92tokens/s]
Shard 26:   7%|▋         | 6701192/100000000 [00:01<00:23, 4021189.23tokens/s]
Shard 26:   7%|▋         | 7127664/100000000 [00:01<00:22, 4075843.39tokens/s]
Shard 26:   8%|▊         | 7553032/100000000 [00:01<00:22, 4020615.04tokens/s]
Shard 26:   8%|▊         | 7967306/100000000 [00:02<00:23, 3906595.69tokens/s]
Shard 26:   8%|▊         | 8369349/100000000 [00:02<00:25, 3562451.32tokens/s]
Shard 26:   9%|▉         | 8788082/100000000 [00:02<00:24, 3689386.85tokens/s]
Shard 26:   9%|▉         | 9245179/100000000 [00:02<00:23, 3917811.86tokens/s]
Shard 26:  10%|▉         | 9646323/100000000 [00:02<00:25, 3574434.15tokens/s]
Shard 26:  10%|█         | 10014414/100000000 [00:02<00:26, 3449962.77tokens/s]
Shard 26:  10%|█         | 10367005/100000000 [00:02<00:26, 3359159.01tokens/s]
Shard 26:  11%|█         | 10708299/100000000 [00:02<00:28, 3138982.23tokens/s]
Shard 26:  11%|█         | 11165951/100000000 [00:03<00:25, 3510135.19tokens/s]
Shard 26:  12%|█▏        | 11584349/100000000 [00:03<00:24, 3678332.97tokens/s]
Shard 26:  12%|█▏        | 11959920/100000000 [00:03<00:25, 3387438.89tokens/s]
Shard 26:  12%|█▏        | 12335763/100000000 [00:03<00:25, 3484765.68tokens/s]
Shard 26:  13%|█▎        | 12700274/100000000 [00:03<00:24, 3519618.42tokens/s]
Shard 26:  13%|█▎        | 13158849/100000000 [00:03<00:22, 3795151.70tokens/s]
Shard 26:  14%|█▎        | 13596441/100000000 [00:03<00:21, 3955039.94tokens/s]
Shard 26:  14%|█▍        | 13998040/100000000 [00:03<00:21, 3962178.66tokens/s]
Shard 26:  14%|█▍        | 14399301/100000000 [00:03<00:22, 3808810.93tokens/s]
Shard 26:  15%|█▍        | 14785640/100000000 [00:04<00:25, 3386054.92tokens/s]
Shard 26:  15%|█▌        | 15135694/100000000 [00:04<00:25, 3344846.03tokens/s]
Shard 26:  16%|█▌        | 15517349/100000000 [00:04<00:24, 3457797.30tokens/s]
Shard 26:  16%|█▌        | 15988248/100000000 [00:04<00:22, 3715488.29tokens/s]
Shard 26:  16%|█▋        | 16482144/100000000 [00:04<00:20, 4045191.52tokens/s]
Shard 26:  17%|█▋        | 16927883/100000000 [00:04<00:20, 4145864.38tokens/s]
Shard 26:  17%|█▋        | 17346729/100000000 [00:04<00:20, 4039923.70tokens/s]
Shard 26:  18%|█▊        | 17755727/100000000 [00:04<00:21, 3890913.89tokens/s]
Shard 26:  18%|█▊        | 18193853/100000000 [00:04<00:20, 3998312.80tokens/s]
Shard 26:  19%|█▊        | 18596230/100000000 [00:05<00:22, 3639434.49tokens/s]
Shard 26:  19%|█▉        | 18967212/100000000 [00:05<00:22, 3585971.66tokens/s]
Shard 26:  19%|█▉        | 19398177/100000000 [00:05<00:21, 3772672.83tokens/s]
Shard 26:  20%|█▉        | 19780298/100000000 [00:05<00:21, 3716645.97tokens/s]
Shard 26:  20%|██        | 20155293/100000000 [00:05<00:24, 3250131.22tokens/s]
Shard 26:  20%|██        | 20493584/100000000 [00:05<00:24, 3240722.17tokens/s]
Shard 26:  21%|██        | 20826056/100000000 [00:05<00:24, 3175818.82tokens/s]
Shard 26:  21%|██        | 21149037/100000000 [00:05<00:25, 3145452.63tokens/s]
Shard 26:  22%|██▏       | 21808130/100000000 [00:05<00:19, 4070063.58tokens/s]
Shard 26:  22%|██▏       | 22232354/100000000 [00:06<00:18, 4116466.45tokens/s]
Shard 26:  23%|██▎       | 22650154/100000000 [00:06<00:18, 4110355.05tokens/s]
Shard 26:  23%|██▎       | 23072621/100000000 [00:06<00:18, 4143372.92tokens/s]
Shard 26:  23%|██▎       | 23490832/100000000 [00:06<00:19, 3983516.86tokens/s]
Shard 26:  24%|██▍       | 23892458/100000000 [00:06<00:20, 3770611.01tokens/s]
Shard 26:  24%|██▍       | 24289308/100000000 [00:06<00:19, 3804317.52tokens/s]
Shard 26:  26%|██▌       | 25653243/100000000 [00:06<00:11, 6591215.63tokens/s]
Shard 26:  26%|██▋       | 26332093/100000000 [00:06<00:13, 5313877.25tokens/s]
Shard 26:  27%|██▋       | 26915916/100000000 [00:07<00:16, 4520776.06tokens/s]
Shard 26:  27%|██▋       | 27420279/100000000 [00:07<00:16, 4355124.05tokens/s]
Shard 26:  28%|██▊       | 27981092/100000000 [00:07<00:15, 4646943.50tokens/s]
Shard 26:  29%|██▊       | 28579112/100000000 [00:07<00:14, 4977972.60tokens/s]
Shard 26:  29%|██▉       | 29107681/100000000 [00:07<00:14, 4998597.34tokens/s]
Shard 26:  30%|██▉       | 29629241/100000000 [00:07<00:15, 4500812.71tokens/s]
Shard 26:  30%|███       | 30101947/100000000 [00:07<00:18, 3785836.93tokens/s]
Shard 26:  31%|███       | 30511693/100000000 [00:07<00:18, 3698256.30tokens/s]
Shard 26:  31%|███       | 30902275/100000000 [00:08<00:20, 3450375.01tokens/s]
Shard 26:  31%|███▏      | 31262443/100000000 [00:08<00:20, 3375935.89tokens/s]
Shard 26:  32%|███▏      | 31609902/100000000 [00:08<00:21, 3134859.14tokens/s]
Shard 26:  32%|███▏      | 31940447/100000000 [00:08<00:21, 3168613.37tokens/s]
Shard 26:  32%|███▏      | 32326016/100000000 [00:08<00:20, 3327248.75tokens/s]
Shard 26:  33%|███▎      | 32844723/100000000 [00:08<00:17, 3829670.91tokens/s]
Shard 26:  33%|███▎      | 33237944/100000000 [00:08<00:17, 3770732.81tokens/s]
Shard 26:  34%|███▎      | 33622629/100000000 [00:08<00:19, 3441308.20tokens/s]
Shard 26:  34%|███▍      | 33977676/100000000 [00:08<00:19, 3400803.40tokens/s]
Shard 26:  34%|███▍      | 34336763/100000000 [00:09<00:19, 3444010.17tokens/s]
Shard 26:  35%|███▍      | 34685945/100000000 [00:09<00:19, 3359243.69tokens/s]
Shard 26:  35%|███▌      | 35025605/100000000 [00:09<00:20, 3136400.25tokens/s]
Shard 26:  35%|███▌      | 35343774/100000000 [00:09<00:21, 3004697.63tokens/s]
Shard 26:  36%|███▌      | 35648143/100000000 [00:09<00:22, 2875771.65tokens/s]
Shard 26:  36%|███▌      | 36084240/100000000 [00:09<00:19, 3269013.62tokens/s]
Shard 26:  36%|███▋      | 36499945/100000000 [00:09<00:18, 3499383.12tokens/s]
Shard 26:  37%|███▋      | 36938015/100000000 [00:09<00:16, 3713950.37tokens/s]
Shard 26:  37%|███▋      | 37420769/100000000 [00:09<00:15, 4008791.08tokens/s]
Shard 26:  38%|███▊      | 37830241/100000000 [00:10<00:15, 4027340.98tokens/s]
Shard 26:  38%|███▊      | 38392424/100000000 [00:10<00:13, 4481594.44tokens/s]
Shard 26:  39%|███▉      | 38844071/100000000 [00:10<00:14, 4307264.96tokens/s]
Shard 26:  39%|███▉      | 39314356/100000000 [00:10<00:13, 4412066.47tokens/s]
Shard 26:  40%|███▉      | 39758646/100000000 [00:10<00:14, 4119576.45tokens/s]
Shard 26:  40%|████      | 40176176/100000000 [00:10<00:17, 3506351.09tokens/s]
Shard 26:  41%|████      | 40544969/100000000 [00:10<00:18, 3285965.63tokens/s]
Shard 26:  41%|████      | 40886683/100000000 [00:10<00:17, 3292164.16tokens/s]
Shard 26:  41%|████      | 41225266/100000000 [00:10<00:18, 3139902.37tokens/s]
Shard 26:  42%|████▏     | 41624311/100000000 [00:11<00:17, 3350261.87tokens/s]
Shard 26:  42%|████▏     | 41967320/100000000 [00:11<00:17, 3342641.58tokens/s]
Shard 26:  42%|████▏     | 42338150/100000000 [00:11<00:16, 3438599.51tokens/s]
Shard 26:  43%|████▎     | 42713125/100000000 [00:11<00:16, 3525387.96tokens/s]
Shard 26:  43%|████▎     | 43221954/100000000 [00:11<00:14, 3970479.16tokens/s]
Shard 26:  44%|████▎     | 43623381/100000000 [00:11<00:14, 3758518.77tokens/s]
Shard 26:  44%|████▍     | 44004250/100000000 [00:11<00:17, 3290485.20tokens/s]
Shard 26:  44%|████▍     | 44359354/100000000 [00:11<00:16, 3353912.46tokens/s]
Shard 26:  45%|████▍     | 44704953/100000000 [00:11<00:17, 3216285.43tokens/s]
Shard 26:  45%|████▌     | 45034126/100000000 [00:12<00:18, 3041989.29tokens/s]
Shard 26:  45%|████▌     | 45363653/100000000 [00:12<00:17, 3109023.26tokens/s]
Shard 26:  46%|████▌     | 45746401/100000000 [00:12<00:16, 3260585.09tokens/s]
Shard 26:  46%|████▋     | 46296965/100000000 [00:12<00:13, 3886487.08tokens/s]
Shard 26:  47%|████▋     | 46753988/100000000 [00:12<00:13, 4076138.14tokens/s]
Shard 26:  47%|████▋     | 47191401/100000000 [00:12<00:13, 3991218.16tokens/s]
Shard 26:  48%|████▊     | 47596578/100000000 [00:12<00:13, 3904689.80tokens/s]
Shard 26:  48%|████▊     | 47990811/100000000 [00:12<00:13, 3759753.05tokens/s]
Shard 26:  48%|████▊     | 48370722/100000000 [00:12<00:14, 3621111.72tokens/s]
Shard 26:  49%|████▊     | 48735704/100000000 [00:13<00:14, 3596404.95tokens/s]
Shard 26:  49%|████▉     | 49164356/100000000 [00:13<00:13, 3783857.25tokens/s]
Shard 26:  50%|████▉     | 49546927/100000000 [00:13<00:13, 3606027.02tokens/s]
Shard 26:  50%|████▉     | 49960143/100000000 [00:13<00:13, 3744765.92tokens/s]
Shard 26:  50%|█████     | 50411348/100000000 [00:13<00:12, 3954881.83tokens/s]
Shard 26:  51%|█████     | 50811101/100000000 [00:13<00:12, 3862662.07tokens/s]
Shard 26:  51%|█████▏    | 51253199/100000000 [00:13<00:12, 4022769.31tokens/s]
Shard 26:  52%|█████▏    | 51658407/100000000 [00:13<00:13, 3593533.69tokens/s]
Shard 26:  52%|█████▏    | 52027557/100000000 [00:13<00:14, 3277704.93tokens/s]
Shard 26:  52%|█████▏    | 52400443/100000000 [00:14<00:14, 3377759.11tokens/s]
Shard 26:  53%|█████▎    | 52747033/100000000 [00:14<00:13, 3391000.69tokens/s]
Shard 26:  53%|█████▎    | 53107666/100000000 [00:14<00:13, 3434960.55tokens/s]
Shard 26:  53%|█████▎    | 53496479/100000000 [00:14<00:13, 3516467.70tokens/s]
Shard 26:  54%|█████▍    | 53851750/100000000 [00:14<00:13, 3522767.51tokens/s]
Shard 26:  54%|█████▍    | 54207449/100000000 [00:14<00:13, 3366921.54tokens/s]
Shard 26:  55%|█████▍    | 54547323/100000000 [00:14<00:13, 3287085.14tokens/s]
Shard 26:  55%|█████▌    | 55022954/100000000 [00:14<00:12, 3699555.97tokens/s]
Shard 26:  55%|█████▌    | 55398087/100000000 [00:14<00:12, 3587670.57tokens/s]
Shard 26:  56%|█████▌    | 55761566/100000000 [00:15<00:13, 3396673.39tokens/s]
Shard 26:  56%|█████▌    | 56133213/100000000 [00:15<00:12, 3457048.98tokens/s]
Shard 26:  56%|█████▋    | 56482092/100000000 [00:15<00:13, 3157908.92tokens/s]
Shard 26:  57%|█████▋    | 56858113/100000000 [00:15<00:13, 3318288.32tokens/s]
Shard 26:  57%|█████▋    | 57329590/100000000 [00:15<00:11, 3698537.08tokens/s]
Shard 26:  58%|█████▊    | 57729711/100000000 [00:15<00:11, 3776774.47tokens/s]
Shard 26:  58%|█████▊    | 58113437/100000000 [00:15<00:11, 3626549.38tokens/s]
Shard 26:  58%|█████▊    | 58481618/100000000 [00:15<00:12, 3396960.07tokens/s]
Shard 26:  59%|█████▉    | 58826997/100000000 [00:15<00:12, 3305716.35tokens/s]
Shard 26:  59%|█████▉    | 59193948/100000000 [00:16<00:12, 3397931.03tokens/s]
Shard 26:  60%|█████▉    | 59537254/100000000 [00:16<00:12, 3279844.63tokens/s]
Shard 26:  60%|█████▉    | 59868178/100000000 [00:16<00:13, 3072778.20tokens/s]
Shard 26:  60%|██████    | 60182855/100000000 [00:16<00:14, 2821264.40tokens/s]
Shard 26:  61%|██████    | 60575186/100000000 [00:16<00:12, 3107913.66tokens/s]
Shard 26:  61%|██████    | 60893879/100000000 [00:16<00:12, 3054834.02tokens/s]
Shard 26:  61%|██████    | 61221499/100000000 [00:16<00:12, 3098261.42tokens/s]
Shard 26:  62%|██████▏   | 61639301/100000000 [00:16<00:11, 3384758.53tokens/s]
Shard 26:  62%|██████▏   | 61982361/100000000 [00:16<00:11, 3382247.21tokens/s]
Shard 26:  62%|██████▏   | 62452289/100000000 [00:17<00:09, 3760618.80tokens/s]
Shard 26:  63%|██████▎   | 62853257/100000000 [00:17<00:09, 3756810.40tokens/s]
Shard 26:  63%|██████▎   | 63274658/100000000 [00:17<00:09, 3861448.79tokens/s]
Shard 26:  64%|██████▍   | 63781473/100000000 [00:17<00:08, 4198458.11tokens/s]
Shard 26:  64%|██████▍   | 64294456/100000000 [00:17<00:07, 4470941.54tokens/s]
Shard 26:  65%|██████▍   | 64945637/100000000 [00:17<00:06, 5015395.20tokens/s]
Shard 26:  66%|██████▌   | 65505457/100000000 [00:17<00:06, 5183990.04tokens/s]
Shard 26:  66%|██████▌   | 66045706/100000000 [00:17<00:07, 4799462.93tokens/s]
Shard 26:  67%|██████▋   | 66641138/100000000 [00:17<00:06, 5117553.70tokens/s]
Shard 26:  67%|██████▋   | 67163046/100000000 [00:18<00:07, 4654962.14tokens/s]
Shard 26:  68%|██████▊   | 67935813/100000000 [00:18<00:05, 5442967.18tokens/s]
Shard 26:  68%|██████▊   | 68496800/100000000 [00:18<00:06, 5141119.73tokens/s]
Shard 26:  70%|██████▉   | 69860048/100000000 [00:18<00:04, 7360328.04tokens/s]
Shard 26:  71%|███████   | 70624428/100000000 [00:18<00:04, 6279803.84tokens/s]
Shard 26:  71%|███████▏  | 71296614/100000000 [00:18<00:05, 5547428.42tokens/s]
Shard 26:  72%|███████▏  | 71892806/100000000 [00:18<00:05, 4790236.31tokens/s]
Shard 26:  72%|███████▏  | 72414900/100000000 [00:18<00:06, 4581897.97tokens/s]
Shard 26:  73%|███████▎  | 73065793/100000000 [00:19<00:05, 5016866.56tokens/s]
Shard 26:  74%|███████▎  | 73602320/100000000 [00:19<00:05, 4833800.03tokens/s]
Shard 26:  74%|███████▍  | 74107772/100000000 [00:19<00:06, 4109813.33tokens/s]
Shard 26:  75%|███████▍  | 74547342/100000000 [00:19<00:07, 3476682.36tokens/s]
Shard 26:  75%|███████▍  | 74926097/100000000 [00:19<00:07, 3390502.32tokens/s]
Shard 26:  75%|███████▌  | 75490708/100000000 [00:19<00:06, 3902648.04tokens/s]
Shard 26:  76%|███████▌  | 75910994/100000000 [00:19<00:06, 3609715.19tokens/s]
Shard 26:  76%|███████▋  | 76294392/100000000 [00:20<00:06, 3442504.53tokens/s]
Shard 26:  77%|███████▋  | 76815470/100000000 [00:20<00:05, 3874201.27tokens/s]
Shard 26:  77%|███████▋  | 77279754/100000000 [00:20<00:05, 4067914.68tokens/s]
Shard 26:  78%|███████▊  | 77704786/100000000 [00:20<00:05, 3986191.90tokens/s]
Shard 26:  78%|███████▊  | 78129019/100000000 [00:20<00:05, 4038080.92tokens/s]
Shard 26:  79%|███████▉  | 78814672/100000000 [00:20<00:04, 4810039.33tokens/s]
Shard 26:  79%|███████▉  | 79462840/100000000 [00:20<00:03, 5276279.52tokens/s]
Shard 26:  80%|████████  | 80001965/100000000 [00:20<00:03, 5156700.70tokens/s]
Shard 26:  81%|████████  | 80526017/100000000 [00:20<00:03, 5179897.07tokens/s]
Shard 26:  81%|████████  | 81053534/100000000 [00:20<00:03, 5144046.12tokens/s]
Shard 26:  82%|████████▏ | 81571850/100000000 [00:21<00:03, 5144684.40tokens/s]
Shard 26:  82%|████████▏ | 82092209/100000000 [00:21<00:03, 5112747.20tokens/s]
Shard 26:  83%|████████▎ | 82678530/100000000 [00:21<00:03, 5331504.15tokens/s]
Shard 26:  83%|████████▎ | 83215547/100000000 [00:21<00:03, 4634808.90tokens/s]
Shard 26:  84%|████████▍ | 83961283/100000000 [00:21<00:03, 5320018.65tokens/s]
Shard 26:  85%|████████▍ | 84511186/100000000 [00:21<00:03, 4772492.41tokens/s]
Shard 26:  85%|████████▌ | 85009435/100000000 [00:21<00:03, 4478860.29tokens/s]
Shard 26:  85%|████████▌ | 85488272/100000000 [00:21<00:03, 4387116.80tokens/s]
Shard 26:  86%|████████▌ | 85937877/100000000 [00:22<00:03, 4213702.44tokens/s]
Shard 26:  87%|████████▋ | 86590961/100000000 [00:22<00:02, 4806807.21tokens/s]
Shard 26:  87%|████████▋ | 87089840/100000000 [00:22<00:02, 4675912.92tokens/s]
Shard 26:  88%|████████▊ | 87566648/100000000 [00:22<00:02, 4484014.83tokens/s]
Shard 26:  88%|████████▊ | 88023848/100000000 [00:22<00:02, 4251652.00tokens/s]
Shard 26:  88%|████████▊ | 88455019/100000000 [00:22<00:02, 4175060.82tokens/s]
Shard 26:  89%|████████▉ | 88877244/100000000 [00:22<00:02, 3827447.65tokens/s]
Shard 26:  89%|████████▉ | 89267554/100000000 [00:22<00:02, 3783266.08tokens/s]
Shard 26:  90%|████████▉ | 89805520/100000000 [00:22<00:02, 4207122.87tokens/s]
Shard 26:  90%|█████████ | 90251047/100000000 [00:23<00:02, 4216155.42tokens/s]
Shard 26:  91%|█████████ | 90701121/100000000 [00:23<00:02, 4251919.80tokens/s]
Shard 26:  91%|█████████ | 91131717/100000000 [00:23<00:02, 3930022.81tokens/s]
Shard 26:  92%|█████████▏| 91532225/100000000 [00:23<00:02, 3689429.01tokens/s]
Shard 26:  92%|█████████▏| 92355938/100000000 [00:23<00:01, 4898839.45tokens/s]
Shard 26:  93%|█████████▎| 93165505/100000000 [00:23<00:01, 5779272.22tokens/s]
Shard 26:  94%|█████████▍| 93766435/100000000 [00:23<00:01, 5059276.06tokens/s]
Shard 26:  94%|█████████▍| 94302604/100000000 [00:23<00:01, 4682682.64tokens/s]
Shard 26:  95%|█████████▍| 94794950/100000000 [00:24<00:01, 3665661.83tokens/s]
Shard 26:  95%|█████████▌| 95207705/100000000 [00:24<00:01, 3735908.17tokens/s]
Shard 26:  96%|█████████▌| 95668885/100000000 [00:24<00:01, 3576678.33tokens/s]
Shard 26:  96%|█████████▌| 96051189/100000000 [00:24<00:01, 3459641.80tokens/s]
Shard 26:  96%|█████████▋| 96442378/100000000 [00:24<00:00, 3568526.28tokens/s]
Shard 26:  97%|█████████▋| 96814355/100000000 [00:24<00:00, 3370425.31tokens/s]
Shard 26:  97%|█████████▋| 97220460/100000000 [00:24<00:00, 3541680.50tokens/s]
Shard 26:  98%|█████████▊| 97594743/100000000 [00:24<00:00, 3441650.62tokens/s]
Shard 26:  98%|█████████▊| 98069325/100000000 [00:25<00:00, 3769015.70tokens/s]
Shard 26:  99%|█████████▊| 98610055/100000000 [00:25<00:00, 4215914.16tokens/s]
Shard 26:  99%|█████████▉| 99058178/100000000 [00:25<00:00, 4260316.31tokens/s]
Shard 26: 100%|█████████▉| 99503691/100000000 [00:25<00:00, 4299292.16tokens/s]
Shard 26: 100%|█████████▉| 99939202/100000000 [00:25<00:00, 4311981.07tokens/s]
Shard 26: 100%|█████████▉| 99999622/100000000 [00:34<00:00, 2881886.79tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000026.bin (200,001,024 bytes) in the gpt-2 format

Shard 27:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 27:   0%|          | 286606/100000000 [00:00<00:34, 2857105.12tokens/s]
Shard 27:   1%|          | 572471/100000000 [00:00<00:39, 2528456.80tokens/s]
Shard 27:   1%|          | 828829/100000000 [00:00<00:39, 2482280.83tokens/s]
Shard 27:   1%|          | 1078623/100000000 [00:00<00:46, 2124771.66tokens/s]
Shard 27:   1%|▏         | 1329743/100000000 [00:00<00:44, 2217595.67tokens/s]
Shard 27:   2%|▏         | 1635746/100000000 [00:00<00:39, 2468016.60tokens/s]
Shard 27:   2%|▏         | 1928269/100000000 [00:00<00:37, 2606697.53tokens/s]
Shard 27:   2%|▏         | 2230363/100000000 [00:00<00:35, 2725540.86tokens/s]
Shard 27:   3%|▎         | 2586147/100000000 [00:00<00:32, 2964893.83tokens/s]
Shard 27:   3%|▎         | 2886152/100000000 [00:01<00:34, 2786499.55tokens/s]
Shard 27:   3%|▎         | 3248486/100000000 [00:01<00:32, 3019760.11tokens/s]
Shard 27:   4%|▎         | 3609765/100000000 [00:01<00:30, 3181931.33tokens/s]
Shard 27:   4%|▍         | 3931894/100000000 [00:01<00:32, 2999805.91tokens/s]
Shard 27:   4%|▍         | 4283330/100000000 [00:01<00:30, 3141162.74tokens/s]
Shard 27:   5%|▍         | 4601907/100000000 [00:01<00:31, 3016034.45tokens/s]
Shard 27:   5%|▌         | 5015448/100000000 [00:01<00:28, 3330229.59tokens/s]
Shard 27:   5%|▌         | 5489636/100000000 [00:01<00:25, 3706930.89tokens/s]
Shard 27:   6%|▌         | 5865900/100000000 [00:01<00:27, 3442423.06tokens/s]
Shard 27:   6%|▌         | 6216647/100000000 [00:02<00:30, 3054883.13tokens/s]
Shard 27:   7%|▋         | 6584789/100000000 [00:02<00:29, 3214767.52tokens/s]
Shard 27:   7%|▋         | 7110776/100000000 [00:02<00:24, 3759735.59tokens/s]
Shard 27:   8%|▊         | 7526025/100000000 [00:02<00:23, 3860695.22tokens/s]
Shard 27:   8%|▊         | 7924441/100000000 [00:02<00:29, 3070800.83tokens/s]
Shard 27:   8%|▊         | 8263917/100000000 [00:02<00:32, 2810032.48tokens/s]
Shard 27:   9%|▉         | 8761833/100000000 [00:02<00:27, 3321259.42tokens/s]
Shard 27:   9%|▉         | 9322477/100000000 [00:02<00:23, 3879872.55tokens/s]
Shard 27:  10%|▉         | 9742066/100000000 [00:03<00:24, 3651775.84tokens/s]
Shard 27:  10%|█         | 10131469/100000000 [00:03<00:25, 3463272.63tokens/s]
Shard 27:  11%|█         | 10591092/100000000 [00:03<00:23, 3744861.87tokens/s]
Shard 27:  11%|█         | 11037674/100000000 [00:03<00:22, 3925112.24tokens/s]
Shard 27:  11%|█▏        | 11443927/100000000 [00:03<00:23, 3756352.93tokens/s]
Shard 27:  12%|█▏        | 11830120/100000000 [00:03<00:25, 3415067.55tokens/s]
Shard 27:  12%|█▏        | 12183128/100000000 [00:03<00:27, 3199289.35tokens/s]
Shard 27:  13%|█▎        | 12702192/100000000 [00:03<00:23, 3698484.03tokens/s]
Shard 27:  13%|█▎        | 13120486/100000000 [00:04<00:22, 3824549.66tokens/s]
Shard 27:  14%|█▎        | 13515989/100000000 [00:04<00:23, 3713962.05tokens/s]
Shard 27:  14%|█▍        | 13900923/100000000 [00:04<00:22, 3751055.28tokens/s]
Shard 27:  14%|█▍        | 14282140/100000000 [00:04<00:23, 3663907.10tokens/s]
Shard 27:  15%|█▍        | 14663064/100000000 [00:04<00:23, 3697652.94tokens/s]
Shard 27:  15%|█▌        | 15036499/100000000 [00:04<00:26, 3224444.10tokens/s]
Shard 27:  15%|█▌        | 15371170/100000000 [00:04<00:27, 3133136.35tokens/s]
Shard 27:  16%|█▌        | 15741743/100000000 [00:04<00:25, 3257290.98tokens/s]
Shard 27:  16%|█▌        | 16075478/100000000 [00:04<00:28, 2984306.08tokens/s]
Shard 27:  16%|█▋        | 16382785/100000000 [00:05<00:29, 2822810.45tokens/s]
Shard 27:  17%|█▋        | 16721721/100000000 [00:05<00:28, 2961046.18tokens/s]
Shard 27:  17%|█▋        | 17024297/100000000 [00:05<00:28, 2936039.79tokens/s]
Shard 27:  17%|█▋        | 17322096/100000000 [00:05<00:30, 2690592.98tokens/s]
Shard 27:  18%|█▊        | 17700260/100000000 [00:05<00:27, 2956666.69tokens/s]
Shard 27:  18%|█▊        | 18011240/100000000 [00:05<00:27, 2966884.30tokens/s]
Shard 27:  18%|█▊        | 18314074/100000000 [00:05<00:27, 2947608.53tokens/s]
Shard 27:  19%|█▊        | 18658372/100000000 [00:05<00:26, 3068616.73tokens/s]
Shard 27:  19%|█▉        | 18970843/100000000 [00:05<00:26, 3083920.91tokens/s]
Shard 27:  19%|█▉        | 19281615/100000000 [00:06<00:26, 2990717.82tokens/s]
Shard 27:  20%|█▉        | 19722166/100000000 [00:06<00:23, 3384854.90tokens/s]
Shard 27:  20%|██        | 20064684/100000000 [00:06<00:25, 3150213.82tokens/s]
Shard 27:  20%|██        | 20477024/100000000 [00:06<00:23, 3407758.33tokens/s]
Shard 27:  21%|██        | 20823279/100000000 [00:06<00:25, 3164713.25tokens/s]
Shard 27:  21%|██        | 21146705/100000000 [00:06<00:26, 3014177.92tokens/s]
Shard 27:  21%|██▏       | 21473149/100000000 [00:06<00:25, 3055587.96tokens/s]
Shard 27:  22%|██▏       | 21782619/100000000 [00:06<00:27, 2889493.50tokens/s]
Shard 27:  22%|██▏       | 22076103/100000000 [00:07<00:32, 2396592.26tokens/s]
Shard 27:  22%|██▏       | 22330929/100000000 [00:07<00:32, 2423752.58tokens/s]
Shard 27:  23%|██▎       | 22808715/100000000 [00:07<00:25, 3022319.43tokens/s]
Shard 27:  23%|██▎       | 23150860/100000000 [00:07<00:24, 3129039.02tokens/s]
Shard 27:  23%|██▎       | 23477500/100000000 [00:07<00:24, 3117782.15tokens/s]
Shard 27:  24%|██▍       | 23804431/100000000 [00:07<00:26, 2907400.64tokens/s]
Shard 27:  24%|██▍       | 24216970/100000000 [00:07<00:23, 3230585.49tokens/s]
Shard 27:  25%|██▍       | 24550625/100000000 [00:07<00:25, 2903397.10tokens/s]
Shard 27:  25%|██▌       | 25014751/100000000 [00:07<00:22, 3337132.86tokens/s]
Shard 27:  25%|██▌       | 25362824/100000000 [00:08<00:23, 3122906.07tokens/s]
Shard 27:  26%|██▌       | 25693892/100000000 [00:08<00:23, 3158650.73tokens/s]
Shard 27:  26%|██▌       | 26026520/100000000 [00:08<00:23, 3187920.36tokens/s]
Shard 27:  26%|██▋       | 26352097/100000000 [00:08<00:22, 3204235.39tokens/s]
Shard 27:  27%|██▋       | 26845854/100000000 [00:08<00:19, 3697684.62tokens/s]
Shard 27:  27%|██▋       | 27221446/100000000 [00:08<00:19, 3707235.93tokens/s]
Shard 27:  28%|██▊       | 27596227/100000000 [00:08<00:20, 3547332.93tokens/s]
Shard 27:  28%|██▊       | 28066007/100000000 [00:08<00:18, 3849866.44tokens/s]
Shard 27:  28%|██▊       | 28455070/100000000 [00:08<00:19, 3704484.19tokens/s]
Shard 27:  29%|██▉       | 29305948/100000000 [00:08<00:13, 5060382.30tokens/s]
Shard 27:  30%|██▉       | 29822839/100000000 [00:09<00:16, 4145438.63tokens/s]
Shard 27:  30%|███       | 30271473/100000000 [00:09<00:16, 4135961.93tokens/s]
Shard 27:  31%|███       | 30740873/100000000 [00:09<00:16, 4263906.91tokens/s]
Shard 27:  31%|███       | 31187288/100000000 [00:09<00:16, 4150902.79tokens/s]
Shard 27:  32%|███▏      | 31615389/100000000 [00:09<00:19, 3511707.89tokens/s]
Shard 27:  32%|███▏      | 32022477/100000000 [00:09<00:18, 3646340.87tokens/s]
Shard 27:  32%|███▏      | 32406793/100000000 [00:09<00:19, 3505122.75tokens/s]
Shard 27:  33%|███▎      | 32770776/100000000 [00:09<00:19, 3500007.73tokens/s]
Shard 27:  33%|███▎      | 33132434/100000000 [00:10<00:21, 3100160.67tokens/s]
Shard 27:  33%|███▎      | 33455981/100000000 [00:10<00:23, 2881474.11tokens/s]
Shard 27:  34%|███▍      | 33755150/100000000 [00:10<00:24, 2669132.48tokens/s]
Shard 27:  34%|███▍      | 34071393/100000000 [00:10<00:23, 2771497.33tokens/s]
Shard 27:  34%|███▍      | 34358112/100000000 [00:10<00:23, 2781634.96tokens/s]
Shard 27:  35%|███▍      | 34680826/100000000 [00:10<00:22, 2892636.93tokens/s]
Shard 27:  35%|███▌      | 35033491/100000000 [00:10<00:21, 3065777.34tokens/s]
Shard 27:  35%|███▌      | 35356433/100000000 [00:10<00:24, 2655291.70tokens/s]
Shard 27:  36%|███▌      | 35634873/100000000 [00:11<00:24, 2598439.94tokens/s]
Shard 27:  36%|███▌      | 35959228/100000000 [00:11<00:23, 2762853.44tokens/s]
Shard 27:  36%|███▌      | 36248674/100000000 [00:11<00:22, 2785258.37tokens/s]
Shard 27:  37%|███▋      | 36538710/100000000 [00:11<00:23, 2670998.43tokens/s]
Shard 27:  37%|███▋      | 36810961/100000000 [00:11<00:24, 2580912.15tokens/s]
Shard 27:  37%|███▋      | 37072567/100000000 [00:11<00:24, 2523795.22tokens/s]
Shard 27:  37%|███▋      | 37373267/100000000 [00:11<00:23, 2656877.99tokens/s]
Shard 27:  38%|███▊      | 37753888/100000000 [00:11<00:20, 2979999.58tokens/s]
Shard 27:  38%|███▊      | 38082798/100000000 [00:11<00:20, 3045391.05tokens/s]
Shard 27:  39%|███▊      | 38542203/100000000 [00:12<00:17, 3476676.07tokens/s]
Shard 27:  39%|███▉      | 38893413/100000000 [00:12<00:18, 3337297.60tokens/s]
Shard 27:  39%|███▉      | 39232573/100000000 [00:12<00:18, 3237674.17tokens/s]
Shard 27:  40%|███▉      | 39558898/100000000 [00:12<00:20, 2901642.75tokens/s]
Shard 27:  40%|███▉      | 39877521/100000000 [00:12<00:20, 2943611.06tokens/s]
Shard 27:  40%|████      | 40211840/100000000 [00:12<00:19, 3051678.25tokens/s]
Shard 27:  41%|████      | 40522207/100000000 [00:12<00:19, 3038978.33tokens/s]
Shard 27:  41%|████      | 40829563/100000000 [00:12<00:20, 2881802.84tokens/s]
Shard 27:  41%|████      | 41134523/100000000 [00:12<00:20, 2926138.19tokens/s]
Shard 27:  41%|████▏     | 41452311/100000000 [00:13<00:19, 2983300.99tokens/s]
Shard 27:  42%|████▏     | 41753084/100000000 [00:13<00:21, 2772427.80tokens/s]
Shard 27:  42%|████▏     | 42034802/100000000 [00:13<00:22, 2609371.33tokens/s]
Shard 27:  42%|████▏     | 42375521/100000000 [00:13<00:20, 2819536.24tokens/s]
Shard 27:  43%|████▎     | 42776892/100000000 [00:13<00:18, 3126117.78tokens/s]
Shard 27:  43%|████▎     | 43126975/100000000 [00:13<00:17, 3231875.10tokens/s]
Shard 27:  43%|████▎     | 43463330/100000000 [00:13<00:17, 3263283.53tokens/s]
Shard 27:  44%|████▍     | 43810193/100000000 [00:13<00:16, 3309646.98tokens/s]
Shard 27:  44%|████▍     | 44145045/100000000 [00:13<00:18, 3023780.43tokens/s]
Shard 27:  44%|████▍     | 44453572/100000000 [00:14<00:18, 3021445.64tokens/s]
Shard 27:  45%|████▍     | 44809906/100000000 [00:14<00:17, 3168195.04tokens/s]
Shard 27:  45%|████▌     | 45209342/100000000 [00:14<00:16, 3404689.18tokens/s]
Shard 27:  46%|████▌     | 45553704/100000000 [00:14<00:15, 3403117.42tokens/s]
Shard 27:  46%|████▌     | 45971477/100000000 [00:14<00:14, 3624007.60tokens/s]
Shard 27:  46%|████▋     | 46387277/100000000 [00:14<00:14, 3767232.44tokens/s]
Shard 27:  47%|████▋     | 46768245/100000000 [00:14<00:14, 3770629.56tokens/s]
Shard 27:  47%|████▋     | 47146674/100000000 [00:14<00:14, 3546902.14tokens/s]
Shard 27:  48%|████▊     | 47584245/100000000 [00:14<00:13, 3766034.84tokens/s]
Shard 27:  48%|████▊     | 47964833/100000000 [00:14<00:13, 3740673.59tokens/s]
Shard 27:  48%|████▊     | 48399376/100000000 [00:15<00:13, 3882707.70tokens/s]
Shard 27:  49%|████▉     | 48789810/100000000 [00:15<00:13, 3881638.88tokens/s]
Shard 27:  49%|████▉     | 49183622/100000000 [00:15<00:14, 3483006.76tokens/s]
Shard 27:  50%|████▉     | 49540332/100000000 [00:15<00:15, 3355976.36tokens/s]
Shard 27:  50%|████▉     | 49955380/100000000 [00:15<00:14, 3558393.21tokens/s]
Shard 27:  50%|█████     | 50318955/100000000 [00:15<00:14, 3480264.51tokens/s]
Shard 27:  51%|█████     | 50671745/100000000 [00:15<00:14, 3490148.73tokens/s]
Shard 27:  51%|█████     | 51024792/100000000 [00:15<00:14, 3491351.35tokens/s]
Shard 27:  51%|█████▏    | 51409218/100000000 [00:15<00:13, 3560407.00tokens/s]
Shard 27:  52%|█████▏    | 51772465/100000000 [00:16<00:13, 3539639.22tokens/s]
Shard 27:  52%|█████▏    | 52129806/100000000 [00:16<00:13, 3549013.70tokens/s]
Shard 27:  52%|█████▏    | 52485629/100000000 [00:16<00:13, 3447340.17tokens/s]
Shard 27:  53%|█████▎    | 52913058/100000000 [00:16<00:12, 3666466.66tokens/s]
Shard 27:  53%|█████▎    | 53339624/100000000 [00:16<00:12, 3815193.79tokens/s]
Shard 27:  54%|█████▍    | 53921547/100000000 [00:16<00:10, 4367613.60tokens/s]
Shard 27:  55%|█████▌    | 55023787/100000000 [00:16<00:07, 6303693.42tokens/s]
Shard 27:  56%|█████▌    | 55659497/100000000 [00:16<00:07, 6286140.54tokens/s]
Shard 27:  56%|█████▋    | 56291215/100000000 [00:16<00:08, 5067012.75tokens/s]
Shard 27:  57%|█████▋    | 57066850/100000000 [00:17<00:07, 5714706.74tokens/s]
Shard 27:  58%|█████▊    | 57678016/100000000 [00:17<00:08, 4719024.08tokens/s]
Shard 27:  58%|█████▊    | 58203147/100000000 [00:17<00:09, 4182463.85tokens/s]
Shard 27:  59%|█████▊    | 58664803/100000000 [00:17<00:10, 3985250.61tokens/s]
Shard 27:  59%|█████▉    | 59096821/100000000 [00:17<00:10, 4041771.33tokens/s]
Shard 27:  60%|█████▉    | 59524564/100000000 [00:17<00:09, 4084584.35tokens/s]
Shard 27:  60%|██████    | 60103811/100000000 [00:17<00:08, 4527055.73tokens/s]
Shard 27:  61%|██████    | 60574907/100000000 [00:17<00:09, 4018170.31tokens/s]
Shard 27:  61%|██████    | 61001866/100000000 [00:18<00:09, 4052878.26tokens/s]
Shard 27:  61%|██████▏   | 61429033/100000000 [00:18<00:09, 4097283.21tokens/s]
Shard 27:  62%|██████▏   | 61850503/100000000 [00:18<00:10, 3641533.44tokens/s]
Shard 27:  62%|██████▏   | 62230475/100000000 [00:18<00:10, 3681624.50tokens/s]
Shard 27:  63%|██████▎   | 62706129/100000000 [00:18<00:09, 3969485.03tokens/s]
Shard 27:  63%|██████▎   | 63133262/100000000 [00:18<00:09, 3910456.98tokens/s]
Shard 27:  64%|██████▎   | 63532791/100000000 [00:18<00:09, 3658039.74tokens/s]
Shard 27:  64%|██████▍   | 63906682/100000000 [00:18<00:09, 3642351.52tokens/s]
Shard 27:  64%|██████▍   | 64468373/100000000 [00:18<00:08, 4170455.88tokens/s]
Shard 27:  65%|██████▌   | 65034880/100000000 [00:19<00:07, 4559332.40tokens/s]
Shard 27:  66%|██████▌   | 65564081/100000000 [00:19<00:07, 4768301.72tokens/s]
Shard 27:  66%|██████▌   | 66083860/100000000 [00:19<00:06, 4868843.01tokens/s]
Shard 27:  67%|██████▋   | 66575402/100000000 [00:19<00:07, 4237004.85tokens/s]
Shard 27:  67%|██████▋   | 67111274/100000000 [00:19<00:07, 4517136.43tokens/s]
Shard 27:  68%|██████▊   | 67579475/100000000 [00:19<00:07, 4072138.12tokens/s]
Shard 27:  68%|██████▊   | 68004297/100000000 [00:19<00:08, 3875893.70tokens/s]
Shard 27:  68%|██████▊   | 68404462/100000000 [00:19<00:08, 3576400.91tokens/s]
Shard 27:  69%|██████▉   | 68785286/100000000 [00:20<00:08, 3629478.63tokens/s]
Shard 27:  69%|██████▉   | 69372258/100000000 [00:20<00:07, 4178927.25tokens/s]
Shard 27:  70%|██████▉   | 69860688/100000000 [00:20<00:06, 4357516.89tokens/s]
Shard 27:  70%|███████   | 70436022/100000000 [00:20<00:06, 4746128.83tokens/s]
Shard 27:  71%|███████   | 70920923/100000000 [00:20<00:06, 4542391.07tokens/s]
Shard 27:  71%|███████▏  | 71498434/100000000 [00:20<00:05, 4866032.95tokens/s]
Shard 27:  72%|███████▏  | 72284146/100000000 [00:20<00:04, 5706996.10tokens/s]
Shard 27:  73%|███████▎  | 72866147/100000000 [00:20<00:05, 5177731.46tokens/s]
Shard 27:  73%|███████▎  | 73399457/100000000 [00:20<00:05, 4638259.02tokens/s]
Shard 27:  74%|███████▍  | 74050568/100000000 [00:21<00:05, 5116845.68tokens/s]
Shard 27:  75%|███████▍  | 74583592/100000000 [00:21<00:05, 4901665.94tokens/s]
Shard 27:  75%|███████▌  | 75089264/100000000 [00:21<00:05, 4912479.04tokens/s]
Shard 27:  76%|███████▌  | 75592539/100000000 [00:21<00:04, 4903655.25tokens/s]
Shard 27:  76%|███████▌  | 76233274/100000000 [00:21<00:04, 5323751.61tokens/s]
Shard 27:  77%|███████▋  | 76774383/100000000 [00:21<00:04, 4741424.42tokens/s]
Shard 27:  77%|███████▋  | 77274597/100000000 [00:21<00:04, 4810581.55tokens/s]
Shard 27:  78%|███████▊  | 77774454/100000000 [00:21<00:05, 4182318.39tokens/s]
Shard 27:  78%|███████▊  | 78214887/100000000 [00:22<00:05, 3952795.03tokens/s]
Shard 27:  79%|███████▊  | 78625414/100000000 [00:22<00:06, 3471155.06tokens/s]
Shard 27:  79%|███████▉  | 78990441/100000000 [00:22<00:06, 3129676.95tokens/s]
Shard 27:  79%|███████▉  | 79442262/100000000 [00:22<00:05, 3448002.99tokens/s]
Shard 27:  80%|███████▉  | 79806724/100000000 [00:22<00:06, 3364565.57tokens/s]
Shard 27:  80%|████████  | 80156008/100000000 [00:22<00:06, 3294626.60tokens/s]
Shard 27:  81%|████████  | 80503459/100000000 [00:22<00:05, 3324042.89tokens/s]
Shard 27:  81%|████████  | 80897569/100000000 [00:22<00:05, 3421086.59tokens/s]
Shard 27:  81%|████████▏ | 81467862/100000000 [00:22<00:04, 4037285.76tokens/s]
Shard 27:  82%|████████▏ | 81880888/100000000 [00:23<00:04, 3672122.87tokens/s]
Shard 27:  82%|████████▏ | 82260298/100000000 [00:23<00:04, 3626185.42tokens/s]
Shard 27:  83%|████████▎ | 82656167/100000000 [00:23<00:04, 3713148.43tokens/s]
Shard 27:  83%|████████▎ | 83034793/100000000 [00:23<00:04, 3660292.08tokens/s]
Shard 27:  84%|████████▎ | 83707125/100000000 [00:23<00:03, 4481277.94tokens/s]
Shard 27:  84%|████████▍ | 84163170/100000000 [00:23<00:04, 3839199.25tokens/s]
Shard 27:  85%|████████▍ | 84566676/100000000 [00:23<00:04, 3762408.76tokens/s]
Shard 27:  85%|████████▍ | 84956122/100000000 [00:23<00:04, 3654377.26tokens/s]
Shard 27:  85%|████████▌ | 85330398/100000000 [00:24<00:04, 3577453.27tokens/s]
Shard 27:  86%|████████▌ | 85786014/100000000 [00:24<00:03, 3840686.85tokens/s]
Shard 27:  86%|████████▌ | 86227874/100000000 [00:24<00:03, 3995002.36tokens/s]
Shard 27:  87%|████████▋ | 86634212/100000000 [00:24<00:03, 3794393.62tokens/s]
Shard 27:  87%|████████▋ | 87019356/100000000 [00:24<00:03, 3365056.98tokens/s]
Shard 27:  87%|████████▋ | 87384427/100000000 [00:24<00:03, 3424225.44tokens/s]
Shard 27:  88%|████████▊ | 87856386/100000000 [00:24<00:03, 3752999.73tokens/s]
Shard 27:  88%|████████▊ | 88241155/100000000 [00:24<00:03, 3701999.78tokens/s]
Shard 27:  89%|████████▊ | 88618641/100000000 [00:24<00:03, 3531560.91tokens/s]
Shard 27:  89%|████████▉ | 88994267/100000000 [00:25<00:03, 3592571.93tokens/s]
Shard 27:  89%|████████▉ | 89368713/100000000 [00:25<00:02, 3615935.32tokens/s]
Shard 27:  90%|████████▉ | 89864247/100000000 [00:25<00:02, 3980749.53tokens/s]
Shard 27:  90%|█████████ | 90302632/100000000 [00:25<00:02, 4086322.78tokens/s]
Shard 27:  91%|█████████ | 90739893/100000000 [00:25<00:02, 4160255.15tokens/s]
Shard 27:  91%|█████████ | 91160726/100000000 [00:25<00:02, 3666932.10tokens/s]
Shard 27:  92%|█████████▏| 91543540/100000000 [00:25<00:02, 3704015.60tokens/s]
Shard 27:  92%|█████████▏| 91968652/100000000 [00:25<00:02, 3840965.73tokens/s]
Shard 27:  92%|█████████▏| 92413652/100000000 [00:25<00:01, 4003380.65tokens/s]
Shard 27:  93%|█████████▎| 92820750/100000000 [00:26<00:02, 3559991.75tokens/s]
Shard 27:  93%|█████████▎| 93189384/100000000 [00:26<00:02, 3183198.89tokens/s]
Shard 27:  94%|█████████▎| 93522041/100000000 [00:26<00:02, 3186610.18tokens/s]
Shard 27:  94%|█████████▍| 93853446/100000000 [00:26<00:02, 2928902.12tokens/s]
Shard 27:  94%|█████████▍| 94155802/100000000 [00:26<00:02, 2827557.90tokens/s]
Shard 27:  95%|█████████▍| 94656944/100000000 [00:26<00:01, 3385917.31tokens/s]
Shard 27:  95%|█████████▌| 95008502/100000000 [00:26<00:01, 3166619.60tokens/s]
Shard 27:  95%|█████████▌| 95339113/100000000 [00:26<00:01, 3027499.07tokens/s]
Shard 27:  96%|█████████▌| 95673404/100000000 [00:26<00:01, 3101348.30tokens/s]
Shard 27:  96%|█████████▌| 96091324/100000000 [00:27<00:01, 3394029.71tokens/s]
Shard 27:  96%|█████████▋| 96440560/100000000 [00:27<00:01, 3247137.94tokens/s]
Shard 27:  97%|█████████▋| 96771944/100000000 [00:27<00:01, 3189231.43tokens/s]
Shard 27:  97%|█████████▋| 97325376/100000000 [00:27<00:00, 3836652.48tokens/s]
Shard 27:  98%|█████████▊| 97717477/100000000 [00:27<00:00, 3441923.68tokens/s]
Shard 27:  98%|█████████▊| 98214596/100000000 [00:27<00:00, 3844951.98tokens/s]
Shard 27:  99%|█████████▊| 98677500/100000000 [00:27<00:00, 4051831.26tokens/s]
Shard 27:  99%|█████████▉| 99095215/100000000 [00:27<00:00, 3472614.83tokens/s]
Shard 27:  99%|█████████▉| 99464796/100000000 [00:28<00:00, 3165626.78tokens/s]
Shard 27: 100%|█████████▉| 99829238/100000000 [00:28<00:00, 3268972.28tokens/s]
Shard 27: 100%|█████████▉| 99999484/100000000 [00:36<00:00, 2732585.76tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000027.bin (200,001,024 bytes) in the gpt-2 format

Shard 28:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 28:   0%|          | 262769/100000000 [00:00<00:38, 2564090.02tokens/s]
Shard 28:   1%|          | 570737/100000000 [00:00<00:34, 2861945.51tokens/s]
Shard 28:   1%|          | 857540/100000000 [00:00<00:39, 2517759.60tokens/s]
Shard 28:   1%|          | 1123565/100000000 [00:00<00:39, 2534109.28tokens/s]
Shard 28:   1%|▏         | 1441465/100000000 [00:00<00:35, 2744331.50tokens/s]
Shard 28:   2%|▏         | 1769308/100000000 [00:00<00:33, 2889386.98tokens/s]
Shard 28:   2%|▏         | 2061143/100000000 [00:00<00:37, 2613322.91tokens/s]
Shard 28:   2%|▏         | 2472990/100000000 [00:00<00:32, 3037050.43tokens/s]
Shard 28:   3%|▎         | 2785263/100000000 [00:00<00:33, 2894062.01tokens/s]
Shard 28:   3%|▎         | 3281298/100000000 [00:01<00:27, 3468757.40tokens/s]
Shard 28:   4%|▎         | 3729909/100000000 [00:01<00:25, 3757521.97tokens/s]
Shard 28:   4%|▍         | 4114027/100000000 [00:01<00:28, 3317280.79tokens/s]
Shard 28:   4%|▍         | 4460090/100000000 [00:01<00:29, 3239141.69tokens/s]
Shard 28:   5%|▍         | 4795318/100000000 [00:01<00:30, 3148514.84tokens/s]
Shard 28:   5%|▌         | 5150376/100000000 [00:01<00:29, 3254943.42tokens/s]
Shard 28:   6%|▌         | 5525734/100000000 [00:01<00:27, 3374528.37tokens/s]
Shard 28:   6%|▌         | 6032090/100000000 [00:01<00:24, 3804958.02tokens/s]
Shard 28:   6%|▋         | 6455914/100000000 [00:01<00:23, 3909956.63tokens/s]
Shard 28:   7%|▋         | 6906408/100000000 [00:02<00:23, 4040146.20tokens/s]
Shard 28:   7%|▋         | 7315024/100000000 [00:02<00:25, 3693974.47tokens/s]
Shard 28:   8%|▊         | 7693084/100000000 [00:02<00:27, 3330328.31tokens/s]
Shard 28:   8%|▊         | 8090743/100000000 [00:02<00:26, 3495858.20tokens/s]
Shard 28:   8%|▊         | 8450103/100000000 [00:02<00:27, 3323791.99tokens/s]
Shard 28:   9%|▉         | 8899174/100000000 [00:02<00:25, 3577548.02tokens/s]
Shard 28:   9%|▉         | 9276442/100000000 [00:02<00:27, 3336254.43tokens/s]
Shard 28:  10%|▉         | 9617352/100000000 [00:02<00:27, 3305650.92tokens/s]
Shard 28:  10%|▉         | 9953486/100000000 [00:03<00:29, 3039548.24tokens/s]
Shard 28:  10%|█         | 10340140/100000000 [00:03<00:27, 3249512.02tokens/s]
Shard 28:  11%|█         | 10720326/100000000 [00:03<00:26, 3376228.60tokens/s]
Shard 28:  11%|█         | 11064593/100000000 [00:03<00:28, 3087874.11tokens/s]
Shard 28:  11%|█▏        | 11381577/100000000 [00:03<00:29, 2977825.32tokens/s]
Shard 28:  12%|█▏        | 11873749/100000000 [00:03<00:25, 3487279.49tokens/s]
Shard 28:  12%|█▏        | 12430880/100000000 [00:03<00:21, 4059094.76tokens/s]
Shard 28:  13%|█▎        | 13028990/100000000 [00:03<00:19, 4566893.76tokens/s]
Shard 28:  13%|█▎        | 13496726/100000000 [00:03<00:20, 4313480.61tokens/s]
Shard 28:  14%|█▍        | 13940929/100000000 [00:04<00:20, 4250542.19tokens/s]
Shard 28:  14%|█▍        | 14417069/100000000 [00:04<00:19, 4386082.82tokens/s]
Shard 28:  15%|█▍        | 14861476/100000000 [00:04<00:19, 4360195.90tokens/s]
Shard 28:  15%|█▌        | 15302443/100000000 [00:04<00:20, 4221120.62tokens/s]
Shard 28:  16%|█▌        | 15728048/100000000 [00:04<00:23, 3623181.43tokens/s]
Shard 28:  16%|█▋        | 16276001/100000000 [00:04<00:20, 4095491.75tokens/s]
Shard 28:  17%|█▋        | 16726291/100000000 [00:04<00:19, 4204912.43tokens/s]
Shard 28:  17%|█▋        | 17163896/100000000 [00:04<00:20, 3948582.33tokens/s]
Shard 28:  18%|█▊        | 17603835/100000000 [00:04<00:20, 4047772.77tokens/s]
Shard 28:  18%|█▊        | 18018461/100000000 [00:05<00:20, 3946208.72tokens/s]
Shard 28:  18%|█▊        | 18422585/100000000 [00:05<00:22, 3596888.43tokens/s]
Shard 28:  19%|█▉        | 18878676/100000000 [00:05<00:21, 3794222.08tokens/s]
Shard 28:  19%|█▉        | 19389634/100000000 [00:05<00:19, 4131714.68tokens/s]
Shard 28:  20%|█▉        | 19936146/100000000 [00:05<00:17, 4493568.33tokens/s]
Shard 28:  20%|██        | 20478963/100000000 [00:05<00:16, 4725178.07tokens/s]
Shard 28:  21%|██        | 20959142/100000000 [00:05<00:18, 4299958.03tokens/s]
Shard 28:  22%|██▏       | 21543427/100000000 [00:05<00:16, 4715073.76tokens/s]
Shard 28:  22%|██▏       | 22027755/100000000 [00:06<00:20, 3897263.59tokens/s]
Shard 28:  22%|██▏       | 22449419/100000000 [00:06<00:21, 3655350.54tokens/s]
Shard 28:  23%|██▎       | 22836914/100000000 [00:06<00:22, 3356067.23tokens/s]
Shard 28:  23%|██▎       | 23192926/100000000 [00:06<00:23, 3218969.11tokens/s]
Shard 28:  24%|██▎       | 23544940/100000000 [00:06<00:25, 3050609.76tokens/s]
Shard 28:  24%|██▍       | 23930061/100000000 [00:06<00:23, 3240325.91tokens/s]
Shard 28:  24%|██▍       | 24299409/100000000 [00:06<00:22, 3353864.42tokens/s]
Shard 28:  25%|██▍       | 24643489/100000000 [00:06<00:22, 3357597.05tokens/s]
Shard 28:  25%|██▍       | 24988977/100000000 [00:07<00:25, 2922750.53tokens/s]
Shard 28:  25%|██▌       | 25364751/100000000 [00:07<00:23, 3124294.33tokens/s]
Shard 28:  26%|██▌       | 25815455/100000000 [00:07<00:21, 3477280.87tokens/s]
Shard 28:  26%|██▌       | 26176083/100000000 [00:07<00:22, 3288207.38tokens/s]
Shard 28:  27%|██▋       | 26549213/100000000 [00:07<00:21, 3352148.36tokens/s]
Shard 28:  27%|██▋       | 26892580/100000000 [00:07<00:22, 3228253.40tokens/s]
Shard 28:  27%|██▋       | 27324799/100000000 [00:07<00:20, 3526263.80tokens/s]
Shard 28:  28%|██▊       | 27684443/100000000 [00:07<00:24, 2978676.24tokens/s]
Shard 28:  28%|██▊       | 28008733/100000000 [00:07<00:23, 3034606.85tokens/s]
Shard 28:  28%|██▊       | 28398549/100000000 [00:08<00:21, 3256396.37tokens/s]
Shard 28:  29%|██▊       | 28736768/100000000 [00:08<00:22, 3152321.03tokens/s]
Shard 28:  29%|██▉       | 29152540/100000000 [00:08<00:20, 3414652.77tokens/s]
Shard 28:  30%|██▉       | 29644112/100000000 [00:08<00:18, 3825705.32tokens/s]
Shard 28:  30%|███       | 30039263/100000000 [00:08<00:20, 3392938.81tokens/s]
Shard 28:  30%|███       | 30393514/100000000 [00:08<00:22, 3140886.27tokens/s]
Shard 28:  31%|███       | 30725338/100000000 [00:08<00:21, 3185729.05tokens/s]
Shard 28:  31%|███       | 31160682/100000000 [00:08<00:19, 3464773.02tokens/s]
Shard 28:  32%|███▏      | 31534591/100000000 [00:08<00:20, 3296948.09tokens/s]
Shard 28:  32%|███▏      | 31872884/100000000 [00:09<00:21, 3126099.94tokens/s]
Shard 28:  32%|███▏      | 32260429/100000000 [00:09<00:20, 3301738.50tokens/s]
Shard 28:  33%|███▎      | 32688676/100000000 [00:09<00:18, 3565106.09tokens/s]
Shard 28:  33%|███▎      | 33052264/100000000 [00:09<00:20, 3344945.28tokens/s]
Shard 28:  33%|███▎      | 33468190/100000000 [00:09<00:18, 3563474.43tokens/s]
Shard 28:  34%|███▍      | 33885121/100000000 [00:09<00:17, 3721117.83tokens/s]
Shard 28:  34%|███▍      | 34303485/100000000 [00:09<00:17, 3818865.28tokens/s]
Shard 28:  35%|███▌      | 35023004/100000000 [00:09<00:13, 4766371.92tokens/s]
Shard 28:  36%|███▌      | 35505743/100000000 [00:09<00:14, 4558699.70tokens/s]
Shard 28:  36%|███▌      | 35967868/100000000 [00:10<00:15, 4108709.53tokens/s]
Shard 28:  36%|███▋      | 36389967/100000000 [00:10<00:17, 3692716.80tokens/s]
Shard 28:  37%|███▋      | 36795453/100000000 [00:10<00:16, 3771887.67tokens/s]
Shard 28:  37%|███▋      | 37207278/100000000 [00:10<00:16, 3860360.15tokens/s]
Shard 28:  38%|███▊      | 37602825/100000000 [00:10<00:16, 3682779.18tokens/s]
Shard 28:  38%|███▊      | 37980386/100000000 [00:10<00:17, 3518216.60tokens/s]
Shard 28:  38%|███▊      | 38337638/100000000 [00:10<00:18, 3355746.89tokens/s]
Shard 28:  39%|███▊      | 38713567/100000000 [00:10<00:17, 3443528.16tokens/s]
Shard 28:  39%|███▉      | 39061677/100000000 [00:11<00:18, 3346605.92tokens/s]
Shard 28:  39%|███▉      | 39399339/100000000 [00:11<00:18, 3353608.90tokens/s]
Shard 28:  40%|███▉      | 39738984/100000000 [00:11<00:18, 3182623.04tokens/s]
Shard 28:  40%|████      | 40106891/100000000 [00:11<00:18, 3316464.24tokens/s]
Shard 28:  41%|████      | 40602929/100000000 [00:11<00:15, 3781278.45tokens/s]
Shard 28:  41%|████      | 40985861/100000000 [00:11<00:16, 3634586.19tokens/s]
Shard 28:  42%|████▏     | 41521240/100000000 [00:11<00:14, 4095274.42tokens/s]
Shard 28:  42%|████▏     | 41940291/100000000 [00:11<00:14, 3948513.44tokens/s]
Shard 28:  42%|████▏     | 42344238/100000000 [00:11<00:14, 3972879.28tokens/s]
Shard 28:  43%|████▎     | 42814016/100000000 [00:11<00:13, 4180632.90tokens/s]
Shard 28:  43%|████▎     | 43330931/100000000 [00:12<00:12, 4459829.25tokens/s]
Shard 28:  44%|████▍     | 43795878/100000000 [00:12<00:12, 4515172.70tokens/s]
Shard 28:  44%|████▍     | 44255697/100000000 [00:12<00:12, 4509737.15tokens/s]
Shard 28:  45%|████▍     | 44708542/100000000 [00:12<00:12, 4421139.62tokens/s]
Shard 28:  45%|████▌     | 45302404/100000000 [00:12<00:11, 4857709.63tokens/s]
Shard 28:  46%|████▌     | 45790412/100000000 [00:12<00:12, 4255508.34tokens/s]
Shard 28:  46%|████▌     | 46230380/100000000 [00:12<00:14, 3832558.35tokens/s]
Shard 28:  47%|████▋     | 46629908/100000000 [00:12<00:13, 3849827.71tokens/s]
Shard 28:  47%|████▋     | 47109506/100000000 [00:12<00:12, 4082053.21tokens/s]
Shard 28:  48%|████▊     | 47567899/100000000 [00:13<00:12, 4217086.45tokens/s]
Shard 28:  48%|████▊     | 47998946/100000000 [00:13<00:12, 4196222.91tokens/s]
Shard 28:  48%|████▊     | 48424814/100000000 [00:13<00:14, 3601722.76tokens/s]
Shard 28:  49%|████▉     | 48802915/100000000 [00:13<00:15, 3226551.17tokens/s]
Shard 28:  49%|████▉     | 49142886/100000000 [00:13<00:18, 2718065.44tokens/s]
Shard 28:  49%|████▉     | 49436737/100000000 [00:13<00:19, 2604546.93tokens/s]
Shard 28:  50%|████▉     | 49713008/100000000 [00:13<00:20, 2470040.73tokens/s]
Shard 28:  50%|█████     | 50001346/100000000 [00:14<00:19, 2556159.39tokens/s]
Shard 28:  50%|█████     | 50265518/100000000 [00:14<00:20, 2398666.64tokens/s]
Shard 28:  51%|█████     | 50512116/100000000 [00:14<00:21, 2310080.92tokens/s]
Shard 28:  51%|█████     | 50779271/100000000 [00:14<00:20, 2396062.20tokens/s]
Shard 28:  51%|█████     | 51102367/100000000 [00:14<00:18, 2612282.67tokens/s]
Shard 28:  51%|█████▏    | 51369065/100000000 [00:14<00:19, 2501431.03tokens/s]
Shard 28:  52%|█████▏    | 51640870/100000000 [00:14<00:18, 2557085.71tokens/s]
Shard 28:  52%|█████▏    | 51900137/100000000 [00:14<00:20, 2339137.52tokens/s]
Shard 28:  52%|█████▏    | 52139999/100000000 [00:14<00:21, 2201343.93tokens/s]
Shard 28:  52%|█████▏    | 52435090/100000000 [00:15<00:19, 2392226.66tokens/s]
Shard 28:  53%|█████▎    | 52761043/100000000 [00:15<00:18, 2596462.72tokens/s]
Shard 28:  53%|█████▎    | 53117344/100000000 [00:15<00:16, 2841729.98tokens/s]
Shard 28:  53%|█████▎    | 53455361/100000000 [00:15<00:15, 2973927.87tokens/s]
Shard 28:  54%|█████▍    | 53777594/100000000 [00:15<00:15, 3035821.83tokens/s]
Shard 28:  54%|█████▍    | 54085804/100000000 [00:15<00:15, 3003481.75tokens/s]
Shard 28:  55%|█████▍    | 54621552/100000000 [00:15<00:12, 3679103.02tokens/s]
Shard 28:  55%|█████▌    | 55125160/100000000 [00:15<00:11, 4062470.69tokens/s]
Shard 28:  56%|█████▌    | 55536744/100000000 [00:15<00:11, 3905265.09tokens/s]
Shard 28:  56%|█████▌    | 55931890/100000000 [00:16<00:13, 3341150.11tokens/s]
Shard 28:  56%|█████▋    | 56281713/100000000 [00:16<00:13, 3347267.48tokens/s]
Shard 28:  57%|█████▋    | 56627545/100000000 [00:16<00:14, 2947745.94tokens/s]
Shard 28:  57%|█████▋    | 56979258/100000000 [00:16<00:13, 3081807.95tokens/s]
Shard 28:  57%|█████▋    | 57406747/100000000 [00:16<00:12, 3370723.24tokens/s]
Shard 28:  58%|█████▊    | 57869321/100000000 [00:16<00:11, 3693081.70tokens/s]
Shard 28:  58%|█████▊    | 58250377/100000000 [00:16<00:12, 3251776.17tokens/s]
Shard 28:  59%|█████▊    | 58594321/100000000 [00:16<00:13, 3171084.70tokens/s]
Shard 28:  59%|█████▉    | 58929032/100000000 [00:17<00:12, 3167624.89tokens/s]
Shard 28:  59%|█████▉    | 59294350/100000000 [00:17<00:12, 3293500.44tokens/s]
Shard 28:  60%|█████▉    | 59718027/100000000 [00:17<00:11, 3525872.64tokens/s]
Shard 28:  60%|██████    | 60082296/100000000 [00:17<00:11, 3537434.16tokens/s]
Shard 28:  60%|██████    | 60441305/100000000 [00:17<00:11, 3383037.51tokens/s]
Shard 28:  61%|██████    | 60786527/100000000 [00:17<00:11, 3376808.98tokens/s]
Shard 28:  61%|██████    | 61127097/100000000 [00:17<00:11, 3340717.86tokens/s]
Shard 28:  61%|██████▏   | 61463665/100000000 [00:17<00:11, 3216041.18tokens/s]
Shard 28:  62%|██████▏   | 61787347/100000000 [00:17<00:12, 3182826.14tokens/s]
Shard 28:  62%|██████▏   | 62107285/100000000 [00:17<00:12, 2929770.80tokens/s]
Shard 28:  62%|██████▏   | 62404723/100000000 [00:18<00:13, 2856616.56tokens/s]
Shard 28:  63%|██████▎   | 62959404/100000000 [00:18<00:10, 3586576.55tokens/s]
Shard 28:  63%|██████▎   | 63327602/100000000 [00:18<00:10, 3507300.36tokens/s]
Shard 28:  64%|██████▎   | 63684329/100000000 [00:18<00:11, 3253900.89tokens/s]
Shard 28:  64%|██████▍   | 64065534/100000000 [00:18<00:10, 3388012.06tokens/s]
Shard 28:  64%|██████▍   | 64410429/100000000 [00:18<00:11, 3152721.97tokens/s]
Shard 28:  65%|██████▍   | 64732196/100000000 [00:18<00:11, 3158367.52tokens/s]
Shard 28:  65%|██████▌   | 65122343/100000000 [00:18<00:10, 3340149.13tokens/s]
Shard 28:  66%|██████▌   | 65503788/100000000 [00:18<00:09, 3466854.84tokens/s]
Shard 28:  66%|██████▌   | 65854693/100000000 [00:19<00:10, 3117209.28tokens/s]
Shard 28:  66%|██████▌   | 66175218/100000000 [00:19<00:11, 2985133.09tokens/s]
Shard 28:  66%|██████▋   | 66480170/100000000 [00:19<00:11, 2976526.29tokens/s]
Shard 28:  67%|██████▋   | 66782127/100000000 [00:19<00:11, 2929016.17tokens/s]
Shard 28:  67%|██████▋   | 67078035/100000000 [00:19<00:11, 2769035.22tokens/s]
Shard 28:  67%|██████▋   | 67360154/100000000 [00:19<00:12, 2545463.08tokens/s]
Shard 28:  68%|██████▊   | 67625241/100000000 [00:19<00:13, 2386155.75tokens/s]
Shard 28:  68%|██████▊   | 67868997/100000000 [00:19<00:13, 2340606.52tokens/s]
Shard 28:  68%|██████▊   | 68135277/100000000 [00:20<00:13, 2417898.91tokens/s]
Shard 28:  68%|██████▊   | 68379659/100000000 [00:20<00:13, 2413787.38tokens/s]
Shard 28:  69%|██████▉   | 68876170/100000000 [00:20<00:09, 3127038.08tokens/s]
Shard 28:  69%|██████▉   | 69195476/100000000 [00:20<00:09, 3142471.02tokens/s]
Shard 28:  70%|██████▉   | 69517274/100000000 [00:20<00:11, 2737889.12tokens/s]
Shard 28:  70%|██████▉   | 69807765/100000000 [00:20<00:10, 2781732.11tokens/s]
Shard 28:  70%|███████   | 70095161/100000000 [00:20<00:10, 2756067.43tokens/s]
Shard 28:  70%|███████   | 70377347/100000000 [00:20<00:10, 2729088.97tokens/s]
Shard 28:  71%|███████   | 70752258/100000000 [00:20<00:09, 2983590.35tokens/s]
Shard 28:  71%|███████   | 71054774/100000000 [00:21<00:09, 2959953.48tokens/s]
Shard 28:  71%|███████▏  | 71353679/100000000 [00:21<00:10, 2752832.63tokens/s]
Shard 28:  72%|███████▏  | 71635035/100000000 [00:21<00:10, 2619106.37tokens/s]
Shard 28:  72%|███████▏  | 71936763/100000000 [00:21<00:10, 2726845.49tokens/s]
Shard 28:  72%|███████▏  | 72213448/100000000 [00:21<00:10, 2692273.04tokens/s]
Shard 28:  73%|███████▎  | 72656472/100000000 [00:21<00:08, 3178598.99tokens/s]
Shard 28:  73%|███████▎  | 72988017/100000000 [00:21<00:08, 3210078.83tokens/s]
Shard 28:  73%|███████▎  | 73312997/100000000 [00:21<00:08, 3143418.48tokens/s]
Shard 28:  74%|███████▎  | 73683171/100000000 [00:21<00:07, 3300703.48tokens/s]
Shard 28:  74%|███████▍  | 74019620/100000000 [00:21<00:07, 3300938.08tokens/s]
Shard 28:  74%|███████▍  | 74358471/100000000 [00:22<00:08, 3038674.41tokens/s]
Shard 28:  75%|███████▍  | 74956488/100000000 [00:22<00:06, 3839059.10tokens/s]
Shard 28:  75%|███████▌  | 75352311/100000000 [00:22<00:06, 3831243.06tokens/s]
Shard 28:  76%|███████▌  | 75776191/100000000 [00:22<00:06, 3939005.84tokens/s]
Shard 28:  76%|███████▌  | 76184329/100000000 [00:22<00:06, 3788548.58tokens/s]
Shard 28:  77%|███████▋  | 76570187/100000000 [00:22<00:06, 3453984.37tokens/s]
Shard 28:  77%|███████▋  | 76995804/100000000 [00:22<00:06, 3668240.52tokens/s]
Shard 28:  77%|███████▋  | 77371027/100000000 [00:22<00:06, 3511683.07tokens/s]
Shard 28:  78%|███████▊  | 77729965/100000000 [00:22<00:06, 3503777.78tokens/s]
Shard 28:  78%|███████▊  | 78339115/100000000 [00:23<00:05, 4225875.15tokens/s]
Shard 28:  79%|███████▉  | 78850315/100000000 [00:23<00:04, 4466587.23tokens/s]
Shard 28:  79%|███████▉  | 79308494/100000000 [00:23<00:04, 4493684.76tokens/s]
Shard 28:  80%|███████▉  | 79763157/100000000 [00:23<00:04, 4161593.48tokens/s]
Shard 28:  80%|████████  | 80188366/100000000 [00:23<00:05, 3820109.88tokens/s]
Shard 28:  81%|████████  | 80581232/100000000 [00:23<00:05, 3706625.75tokens/s]
Shard 28:  81%|████████  | 80959746/100000000 [00:23<00:05, 3353261.62tokens/s]
Shard 28:  81%|████████▏ | 81334603/100000000 [00:23<00:05, 3453113.66tokens/s]
Shard 28:  82%|████████▏ | 81687880/100000000 [00:24<00:05, 3330910.27tokens/s]
Shard 28:  82%|████████▏ | 82029036/100000000 [00:24<00:06, 2938372.13tokens/s]
Shard 28:  82%|████████▏ | 82333266/100000000 [00:24<00:06, 2845269.88tokens/s]
Shard 28:  83%|████████▎ | 82634873/100000000 [00:24<00:06, 2889030.97tokens/s]
Shard 28:  83%|████████▎ | 83038597/100000000 [00:24<00:05, 3183048.00tokens/s]
Shard 28:  83%|████████▎ | 83363862/100000000 [00:24<00:05, 3096481.83tokens/s]
Shard 28:  84%|████████▍ | 83804625/100000000 [00:24<00:04, 3433990.61tokens/s]
Shard 28:  84%|████████▍ | 84153749/100000000 [00:24<00:04, 3177264.72tokens/s]
Shard 28:  85%|████████▍ | 84588823/100000000 [00:24<00:04, 3465910.80tokens/s]
Shard 28:  85%|████████▍ | 84948311/100000000 [00:25<00:04, 3477034.40tokens/s]
Shard 28:  85%|████████▌ | 85404455/100000000 [00:25<00:03, 3766658.56tokens/s]
Shard 28:  86%|████████▌ | 85786413/100000000 [00:25<00:04, 3440946.02tokens/s]
Shard 28:  86%|████████▌ | 86166443/100000000 [00:25<00:04, 3354126.61tokens/s]
Shard 28:  87%|████████▋ | 86636262/100000000 [00:25<00:03, 3709808.00tokens/s]
Shard 28:  87%|████████▋ | 87055055/100000000 [00:25<00:03, 3828757.82tokens/s]
Shard 28:  87%|████████▋ | 87444872/100000000 [00:25<00:03, 3472788.00tokens/s]
Shard 28:  88%|████████▊ | 87802396/100000000 [00:25<00:03, 3224759.07tokens/s]
Shard 28:  88%|████████▊ | 88237577/100000000 [00:25<00:03, 3517147.05tokens/s]
Shard 28:  89%|████████▊ | 88604667/100000000 [00:26<00:03, 3320157.27tokens/s]
Shard 28:  89%|████████▉ | 88946425/100000000 [00:26<00:03, 3172254.63tokens/s]
Shard 28:  89%|████████▉ | 89382044/100000000 [00:26<00:03, 3463763.71tokens/s]
Shard 28:  90%|████████▉ | 89738191/100000000 [00:26<00:03, 3371507.23tokens/s]
Shard 28:  90%|█████████ | 90081677/100000000 [00:26<00:03, 2987340.61tokens/s]
Shard 28:  90%|█████████ | 90439673/100000000 [00:26<00:03, 3136287.66tokens/s]
Shard 28:  91%|█████████ | 90763019/100000000 [00:26<00:03, 3048249.41tokens/s]
Shard 28:  91%|█████████ | 91106259/100000000 [00:26<00:02, 3084233.11tokens/s]
Shard 28:  91%|█████████▏| 91426551/100000000 [00:27<00:02, 3106303.84tokens/s]
Shard 28:  92%|█████████▏| 91740754/100000000 [00:27<00:02, 2913799.42tokens/s]
Shard 28:  92%|█████████▏| 92123124/100000000 [00:27<00:02, 3157643.66tokens/s]
Shard 28:  93%|█████████▎| 92556595/100000000 [00:27<00:02, 3483998.48tokens/s]
Shard 28:  93%|█████████▎| 93021101/100000000 [00:27<00:01, 3811330.01tokens/s]
Shard 28:  93%|█████████▎| 93446925/100000000 [00:27<00:01, 3917033.50tokens/s]
Shard 28:  94%|█████████▍| 93843375/100000000 [00:27<00:01, 3876285.23tokens/s]
Shard 28:  94%|█████████▍| 94234271/100000000 [00:27<00:01, 3464174.12tokens/s]
Shard 28:  95%|█████████▍| 94591159/100000000 [00:27<00:01, 3276873.38tokens/s]
Shard 28:  95%|█████████▍| 94926991/100000000 [00:28<00:01, 3171178.01tokens/s]
Shard 28:  95%|█████████▌| 95312347/100000000 [00:28<00:01, 3352941.03tokens/s]
Shard 28:  96%|█████████▌| 95865103/100000000 [00:28<00:01, 3938558.89tokens/s]
Shard 28:  96%|█████████▋| 96267457/100000000 [00:28<00:00, 3961580.50tokens/s]
Shard 28:  97%|█████████▋| 96676593/100000000 [00:28<00:00, 3736867.51tokens/s]
Shard 28:  97%|█████████▋| 97057065/100000000 [00:28<00:00, 3444866.60tokens/s]
Shard 28:  97%|█████████▋| 97409345/100000000 [00:28<00:00, 3285114.42tokens/s]
Shard 28:  98%|█████████▊| 97850029/100000000 [00:28<00:00, 3582319.33tokens/s]
Shard 28:  98%|█████████▊| 98216336/100000000 [00:28<00:00, 3394594.44tokens/s]
Shard 28:  99%|█████████▊| 98563351/100000000 [00:29<00:00, 3291545.28tokens/s]
Shard 28:  99%|█████████▉| 98898174/100000000 [00:29<00:00, 3288780.75tokens/s]
Shard 28:  99%|█████████▉| 99346911/100000000 [00:29<00:00, 3611352.32tokens/s]
Shard 28: 100%|█████████▉| 99713056/100000000 [00:29<00:00, 3605315.27tokens/s]
Shard 28: 100%|█████████▉| 99999295/100000000 [00:37<00:00, 2657065.59tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000028.bin (200,001,024 bytes) in the gpt-2 format

Shard 29:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 29:   0%|          | 290748/100000000 [00:00<00:34, 2897218.21tokens/s]
Shard 29:   1%|          | 585730/100000000 [00:00<00:45, 2193670.20tokens/s]
Shard 29:   1%|          | 816312/100000000 [00:00<00:48, 2058276.08tokens/s]
Shard 29:   1%|          | 1041719/100000000 [00:00<00:47, 2088301.09tokens/s]
Shard 29:   1%|▏         | 1254705/100000000 [00:00<00:49, 2003104.39tokens/s]
Shard 29:   2%|▏         | 1511807/100000000 [00:00<00:45, 2160413.00tokens/s]
Shard 29:   2%|▏         | 1798513/100000000 [00:00<00:41, 2366547.97tokens/s]
Shard 29:   2%|▏         | 2055431/100000000 [00:00<00:40, 2421459.84tokens/s]
Shard 29:   2%|▏         | 2351430/100000000 [00:01<00:38, 2539070.26tokens/s]
Shard 29:   3%|▎         | 2876343/100000000 [00:01<00:29, 3333606.75tokens/s]
Shard 29:   3%|▎         | 3253714/100000000 [00:01<00:28, 3416316.75tokens/s]
Shard 29:   4%|▎         | 3697138/100000000 [00:01<00:26, 3699749.65tokens/s]
Shard 29:   4%|▍         | 4312661/100000000 [00:01<00:21, 4421114.23tokens/s]
Shard 29:   5%|▍         | 4759301/100000000 [00:01<00:21, 4373440.71tokens/s]
Shard 29:   5%|▌         | 5199640/100000000 [00:01<00:24, 3904705.84tokens/s]
Shard 29:   6%|▌         | 5602549/100000000 [00:01<00:27, 3432687.41tokens/s]
Shard 29:   6%|▌         | 5962312/100000000 [00:01<00:28, 3342629.66tokens/s]
Shard 29:   6%|▋         | 6313818/100000000 [00:02<00:27, 3351417.68tokens/s]
Shard 29:   7%|▋         | 6731430/100000000 [00:02<00:26, 3567110.17tokens/s]
Shard 29:   7%|▋         | 7095932/100000000 [00:02<00:53, 1730246.65tokens/s]
Shard 29:   7%|▋         | 7373385/100000000 [00:02<00:49, 1881165.46tokens/s]
Shard 29:   8%|▊         | 7768468/100000000 [00:02<00:40, 2263275.28tokens/s]
Shard 29:   8%|▊         | 8378340/100000000 [00:02<00:29, 3060086.98tokens/s]
Shard 29:   9%|▉         | 9017050/100000000 [00:03<00:23, 3814150.19tokens/s]
Shard 29:   9%|▉         | 9486537/100000000 [00:03<00:22, 3987042.52tokens/s]
Shard 29:  10%|▉         | 9954927/100000000 [00:03<00:22, 3922086.92tokens/s]
Shard 29:  10%|█         | 10443648/100000000 [00:03<00:21, 4148003.01tokens/s]
Shard 29:  11%|█▏        | 11449289/100000000 [00:03<00:15, 5708434.68tokens/s]
Shard 29:  12%|█▏        | 12061373/100000000 [00:03<00:16, 5356427.41tokens/s]
Shard 29:  13%|█▎        | 12630066/100000000 [00:03<00:20, 4310130.60tokens/s]
Shard 29:  13%|█▎        | 13113472/100000000 [00:03<00:20, 4140568.06tokens/s]
Shard 29:  14%|█▎        | 13563205/100000000 [00:04<00:21, 4078819.48tokens/s]
Shard 29:  14%|█▍        | 13995287/100000000 [00:04<00:24, 3577489.90tokens/s]
Shard 29:  14%|█▍        | 14379081/100000000 [00:04<00:23, 3569606.63tokens/s]
Shard 29:  15%|█▍        | 14844838/100000000 [00:04<00:22, 3820024.32tokens/s]
Shard 29:  15%|█▌        | 15252501/100000000 [00:04<00:21, 3881867.15tokens/s]
Shard 29:  16%|█▌        | 15667273/100000000 [00:04<00:21, 3938581.61tokens/s]
Shard 29:  16%|█▌        | 16070818/100000000 [00:04<00:22, 3770517.70tokens/s]
Shard 29:  17%|█▋        | 16581522/100000000 [00:04<00:20, 4108520.52tokens/s]
Shard 29:  17%|█▋        | 17005279/100000000 [00:04<00:21, 3945039.63tokens/s]
Shard 29:  17%|█▋        | 17406664/100000000 [00:05<00:23, 3449281.91tokens/s]
Shard 29:  18%|█▊        | 17781120/100000000 [00:05<00:23, 3510862.92tokens/s]
Shard 29:  18%|█▊        | 18176651/100000000 [00:05<00:22, 3626172.30tokens/s]
Shard 29:  19%|█▊        | 18548353/100000000 [00:05<00:22, 3590397.48tokens/s]
Shard 29:  19%|█▉        | 18913830/100000000 [00:05<00:22, 3573592.37tokens/s]
Shard 29:  19%|█▉        | 19276192/100000000 [00:05<00:23, 3398066.96tokens/s]
Shard 29:  20%|█▉        | 19620507/100000000 [00:05<00:24, 3332494.46tokens/s]
Shard 29:  20%|█▉        | 19963010/100000000 [00:05<00:23, 3348634.62tokens/s]
Shard 29:  20%|██        | 20300157/100000000 [00:05<00:25, 3184061.84tokens/s]
Shard 29:  21%|██        | 20748690/100000000 [00:06<00:22, 3541051.35tokens/s]
Shard 29:  21%|██        | 21107644/100000000 [00:06<00:25, 3149797.66tokens/s]
Shard 29:  21%|██▏       | 21437657/100000000 [00:06<00:25, 3037793.17tokens/s]
Shard 29:  22%|██▏       | 21822376/100000000 [00:06<00:24, 3250425.97tokens/s]
Shard 29:  22%|██▏       | 22155858/100000000 [00:06<00:23, 3268734.35tokens/s]
Shard 29:  22%|██▏       | 22489300/100000000 [00:06<00:27, 2861487.86tokens/s]
Shard 29:  23%|██▎       | 22787539/100000000 [00:06<00:27, 2796779.87tokens/s]
Shard 29:  23%|██▎       | 23179691/100000000 [00:06<00:24, 3080056.36tokens/s]
Shard 29:  23%|██▎       | 23497198/100000000 [00:07<00:26, 2922810.49tokens/s]
Shard 29:  24%|██▍       | 23914714/100000000 [00:07<00:23, 3256230.74tokens/s]
Shard 29:  24%|██▍       | 24311564/100000000 [00:07<00:22, 3437408.96tokens/s]
Shard 29:  25%|██▍       | 24663743/100000000 [00:07<00:23, 3245271.55tokens/s]
Shard 29:  25%|██▍       | 24995473/100000000 [00:07<00:23, 3166926.49tokens/s]
Shard 29:  25%|██▌       | 25317405/100000000 [00:07<00:23, 3179027.55tokens/s]
Shard 29:  26%|██▌       | 25741536/100000000 [00:07<00:21, 3478419.07tokens/s]
Shard 29:  26%|██▌       | 26095281/100000000 [00:07<00:21, 3363895.02tokens/s]
Shard 29:  27%|██▋       | 26570012/100000000 [00:07<00:19, 3747352.49tokens/s]
Shard 29:  27%|██▋       | 27067684/100000000 [00:07<00:18, 3991869.91tokens/s]
Shard 29:  28%|██▊       | 27545391/100000000 [00:08<00:17, 4199692.68tokens/s]
Shard 29:  28%|██▊       | 27968724/100000000 [00:08<00:17, 4136876.73tokens/s]
Shard 29:  28%|██▊       | 28457051/100000000 [00:08<00:17, 4171882.49tokens/s]
Shard 29:  29%|██▉       | 28876266/100000000 [00:08<00:19, 3620279.26tokens/s]
Shard 29:  29%|██▉       | 29250925/100000000 [00:08<00:19, 3569379.13tokens/s]
Shard 29:  30%|██▉       | 29616262/100000000 [00:08<00:19, 3542634.10tokens/s]
Shard 29:  30%|███       | 30101002/100000000 [00:08<00:17, 3888052.85tokens/s]
Shard 29:  30%|███       | 30497436/100000000 [00:08<00:18, 3837866.42tokens/s]
Shard 29:  31%|███       | 30888437/100000000 [00:09<00:18, 3682071.86tokens/s]
Shard 29:  31%|███▏      | 31261495/100000000 [00:09<00:19, 3563673.54tokens/s]
Shard 29:  32%|███▏      | 31621103/100000000 [00:09<00:21, 3128087.91tokens/s]
Shard 29:  32%|███▏      | 32134061/100000000 [00:09<00:18, 3610771.36tokens/s]
Shard 29:  33%|███▎      | 32683490/100000000 [00:09<00:16, 4098566.64tokens/s]
Shard 29:  33%|███▎      | 33108527/100000000 [00:09<00:16, 4030305.06tokens/s]
Shard 29:  34%|███▎      | 33586576/100000000 [00:09<00:15, 4234806.89tokens/s]
Shard 29:  34%|███▍      | 34025600/100000000 [00:09<00:15, 4269052.40tokens/s]
Shard 29:  34%|███▍      | 34461920/100000000 [00:09<00:16, 4052946.37tokens/s]
Shard 29:  35%|███▍      | 34989950/100000000 [00:10<00:14, 4376115.30tokens/s]
Shard 29:  36%|███▌      | 35533480/100000000 [00:10<00:13, 4674838.48tokens/s]
Shard 29:  36%|███▌      | 36018949/100000000 [00:10<00:13, 4711619.58tokens/s]
Shard 29:  36%|███▋      | 36494657/100000000 [00:10<00:14, 4528218.69tokens/s]
Shard 29:  37%|███▋      | 37047338/100000000 [00:10<00:13, 4721104.53tokens/s]
Shard 29:  38%|███▊      | 37522920/100000000 [00:10<00:13, 4595559.76tokens/s]
Shard 29:  38%|███▊      | 37985060/100000000 [00:10<00:14, 4297808.82tokens/s]
Shard 29:  38%|███▊      | 38420419/100000000 [00:10<00:15, 4079858.15tokens/s]
Shard 29:  39%|███▉      | 38948717/100000000 [00:10<00:13, 4394642.03tokens/s]
Shard 29:  39%|███▉      | 39394266/100000000 [00:11<00:14, 4222718.07tokens/s]
Shard 29:  40%|███▉      | 39821463/100000000 [00:11<00:16, 3621345.29tokens/s]
Shard 29:  40%|████      | 40199946/100000000 [00:11<00:16, 3576475.74tokens/s]
Shard 29:  41%|████      | 40657278/100000000 [00:11<00:15, 3825089.34tokens/s]
Shard 29:  41%|████      | 41143970/100000000 [00:11<00:14, 4084997.10tokens/s]
Shard 29:  42%|████▏     | 41587605/100000000 [00:11<00:13, 4182487.83tokens/s]
Shard 29:  42%|████▏     | 42199404/100000000 [00:11<00:12, 4679516.44tokens/s]
Shard 29:  43%|████▎     | 42675978/100000000 [00:11<00:12, 4515935.43tokens/s]
Shard 29:  43%|████▎     | 43133487/100000000 [00:11<00:14, 3887317.17tokens/s]
Shard 29:  44%|████▎     | 43541112/100000000 [00:12<00:15, 3597555.19tokens/s]
Shard 29:  44%|████▍     | 43915164/100000000 [00:12<00:16, 3329013.98tokens/s]
Shard 29:  44%|████▍     | 44259365/100000000 [00:12<00:17, 3206238.01tokens/s]
Shard 29:  45%|████▍     | 44590353/100000000 [00:12<00:19, 2803758.80tokens/s]
Shard 29:  45%|████▍     | 44883111/100000000 [00:12<00:20, 2745903.75tokens/s]
Shard 29:  45%|████▌     | 45164600/100000000 [00:12<00:20, 2621771.71tokens/s]
Shard 29:  45%|████▌     | 45432133/100000000 [00:12<00:22, 2421589.87tokens/s]
Shard 29:  46%|████▌     | 45678348/100000000 [00:12<00:22, 2412193.07tokens/s]
Shard 29:  46%|████▌     | 45935142/100000000 [00:13<00:22, 2386254.97tokens/s]
Shard 29:  46%|████▌     | 46195047/100000000 [00:13<00:22, 2430816.10tokens/s]
Shard 29:  46%|████▋     | 46439708/100000000 [00:13<00:22, 2376209.93tokens/s]
Shard 29:  47%|████▋     | 46678501/100000000 [00:13<00:23, 2237740.29tokens/s]
Shard 29:  47%|████▋     | 46904689/100000000 [00:13<00:24, 2182973.54tokens/s]
Shard 29:  47%|████▋     | 47199441/100000000 [00:13<00:22, 2391952.90tokens/s]
Shard 29:  48%|████▊     | 47515922/100000000 [00:13<00:20, 2605995.77tokens/s]
Shard 29:  48%|████▊     | 47849646/100000000 [00:13<00:18, 2803142.37tokens/s]
Shard 29:  48%|████▊     | 48257726/100000000 [00:13<00:16, 3158612.41tokens/s]
Shard 29:  49%|████▊     | 48593404/100000000 [00:14<00:15, 3214311.79tokens/s]
Shard 29:  49%|████▉     | 48917912/100000000 [00:14<00:15, 3215323.31tokens/s]
Shard 29:  49%|████▉     | 49301164/100000000 [00:14<00:14, 3393507.85tokens/s]
Shard 29:  50%|████▉     | 49642049/100000000 [00:14<00:16, 3139332.92tokens/s]
Shard 29:  50%|█████     | 50035739/100000000 [00:14<00:15, 3314924.72tokens/s]
Shard 29:  50%|█████     | 50371644/100000000 [00:14<00:17, 2896738.35tokens/s]
Shard 29:  51%|█████     | 50672872/100000000 [00:14<00:16, 2905431.20tokens/s]
Shard 29:  51%|█████     | 51077722/100000000 [00:14<00:15, 3201863.90tokens/s]
Shard 29:  51%|█████▏    | 51452436/100000000 [00:14<00:14, 3349908.93tokens/s]
Shard 29:  52%|█████▏    | 51924191/100000000 [00:15<00:12, 3731329.19tokens/s]
Shard 29:  52%|█████▏    | 52305246/100000000 [00:15<00:13, 3476606.84tokens/s]
Shard 29:  53%|█████▎    | 52661146/100000000 [00:15<00:16, 2800943.04tokens/s]
Shard 29:  53%|█████▎    | 53093724/100000000 [00:15<00:14, 3161009.57tokens/s]
Shard 29:  53%|█████▎    | 53436214/100000000 [00:15<00:14, 3126113.32tokens/s]
Shard 29:  54%|█████▍    | 53881061/100000000 [00:15<00:13, 3470268.15tokens/s]
Shard 29:  54%|█████▍    | 54292715/100000000 [00:15<00:12, 3625887.48tokens/s]
Shard 29:  55%|█████▍    | 54750099/100000000 [00:15<00:11, 3874984.42tokens/s]
Shard 29:  55%|█████▌    | 55149314/100000000 [00:15<00:12, 3598170.68tokens/s]
Shard 29:  56%|█████▌    | 55520514/100000000 [00:16<00:12, 3557362.13tokens/s]
Shard 29:  56%|█████▌    | 56149000/100000000 [00:16<00:10, 4301428.13tokens/s]
Shard 29:  57%|█████▋    | 56591857/100000000 [00:16<00:10, 4110257.35tokens/s]
Shard 29:  57%|█████▋    | 57078470/100000000 [00:16<00:09, 4309097.07tokens/s]
Shard 29:  58%|█████▊    | 57517904/100000000 [00:16<00:10, 4176663.94tokens/s]
Shard 29:  58%|█████▊    | 58055097/100000000 [00:16<00:09, 4502672.80tokens/s]
Shard 29:  59%|█████▊    | 58583390/100000000 [00:16<00:08, 4711486.08tokens/s]
Shard 29:  59%|█████▉    | 59060174/100000000 [00:16<00:09, 4504022.29tokens/s]
Shard 29:  60%|█████▉    | 59516066/100000000 [00:16<00:09, 4273789.04tokens/s]
Shard 29:  60%|█████▉    | 59980581/100000000 [00:17<00:09, 4374461.67tokens/s]
Shard 29:  60%|██████    | 60423090/100000000 [00:17<00:09, 4110612.38tokens/s]
Shard 29:  61%|██████    | 60839888/100000000 [00:17<00:11, 3426571.97tokens/s]
Shard 29:  61%|██████▏   | 61257835/100000000 [00:17<00:10, 3603933.69tokens/s]
Shard 29:  62%|██████▏   | 61871072/100000000 [00:17<00:09, 4234318.47tokens/s]
Shard 29:  62%|██████▏   | 62315900/100000000 [00:17<00:10, 3699645.10tokens/s]
Shard 29:  63%|██████▎   | 62725692/100000000 [00:17<00:09, 3780972.03tokens/s]
Shard 29:  63%|██████▎   | 63159481/100000000 [00:17<00:09, 3915456.73tokens/s]
Shard 29:  64%|██████▎   | 63607172/100000000 [00:18<00:08, 4058407.29tokens/s]
Shard 29:  64%|██████▍   | 64031631/100000000 [00:18<00:09, 3884253.20tokens/s]
Shard 29:  64%|██████▍   | 64429713/100000000 [00:18<00:10, 3544089.95tokens/s]
Shard 29:  65%|██████▍   | 64795218/100000000 [00:18<00:10, 3499219.19tokens/s]
Shard 29:  65%|██████▌   | 65155697/100000000 [00:18<00:09, 3527467.01tokens/s]
Shard 29:  66%|██████▌   | 65514233/100000000 [00:18<00:10, 3265549.88tokens/s]
Shard 29:  66%|██████▌   | 65847303/100000000 [00:18<00:11, 2979011.49tokens/s]
Shard 29:  66%|██████▌   | 66152768/100000000 [00:18<00:11, 2926832.88tokens/s]
Shard 29:  66%|██████▋   | 66478337/100000000 [00:18<00:11, 3011766.95tokens/s]
Shard 29:  67%|██████▋   | 66784068/100000000 [00:19<00:11, 2861098.13tokens/s]
Shard 29:  67%|██████▋   | 67088901/100000000 [00:19<00:11, 2907555.97tokens/s]
Shard 29:  68%|██████▊   | 67550535/100000000 [00:19<00:09, 3360612.92tokens/s]
Shard 29:  68%|██████▊   | 68015605/100000000 [00:19<00:08, 3722549.21tokens/s]
Shard 29:  68%|██████▊   | 68394265/100000000 [00:19<00:08, 3646186.55tokens/s]
Shard 29:  69%|██████▉   | 68763268/100000000 [00:19<00:09, 3351975.33tokens/s]
Shard 29:  69%|██████▉   | 69106383/100000000 [00:19<00:09, 3236478.81tokens/s]
Shard 29:  69%|██████▉   | 69494980/100000000 [00:19<00:08, 3406951.28tokens/s]
Shard 29:  70%|██████▉   | 69842399/100000000 [00:19<00:09, 3160509.60tokens/s]
Shard 29:  70%|███████   | 70164807/100000000 [00:20<00:10, 2938890.97tokens/s]
Shard 29:  70%|███████   | 70473375/100000000 [00:20<00:09, 2976793.57tokens/s]
Shard 29:  71%|███████   | 70832128/100000000 [00:20<00:09, 3142049.33tokens/s]
Shard 29:  71%|███████▏  | 71252104/100000000 [00:20<00:08, 3438226.15tokens/s]
Shard 29:  72%|███████▏  | 71602442/100000000 [00:20<00:08, 3178309.45tokens/s]
Shard 29:  72%|███████▏  | 71927523/100000000 [00:20<00:08, 3172548.22tokens/s]
Shard 29:  72%|███████▏  | 72303347/100000000 [00:20<00:08, 3333973.25tokens/s]
Shard 29:  73%|███████▎  | 72642613/100000000 [00:20<00:09, 3032457.75tokens/s]
Shard 29:  73%|███████▎  | 73040517/100000000 [00:21<00:08, 3262822.12tokens/s]
Shard 29:  73%|███████▎  | 73376865/100000000 [00:21<00:08, 2995383.17tokens/s]
Shard 29:  74%|███████▎  | 73720892/100000000 [00:21<00:08, 3112029.35tokens/s]
Shard 29:  74%|███████▍  | 74039910/100000000 [00:21<00:08, 3074317.90tokens/s]
Shard 29:  74%|███████▍  | 74399541/100000000 [00:21<00:07, 3208210.20tokens/s]
Shard 29:  75%|███████▌  | 75056692/100000000 [00:21<00:06, 4156372.38tokens/s]
Shard 29:  75%|███████▌  | 75481247/100000000 [00:21<00:06, 3835815.58tokens/s]
Shard 29:  76%|███████▌  | 75875863/100000000 [00:21<00:06, 3801147.77tokens/s]
Shard 29:  76%|███████▋  | 76263097/100000000 [00:21<00:06, 3671494.13tokens/s]
Shard 29:  77%|███████▋  | 76639060/100000000 [00:22<00:06, 3555708.32tokens/s]
Shard 29:  77%|███████▋  | 76998503/100000000 [00:22<00:07, 3063562.21tokens/s]
Shard 29:  77%|███████▋  | 77318449/100000000 [00:22<00:07, 2996545.62tokens/s]
Shard 29:  78%|███████▊  | 77665292/100000000 [00:22<00:07, 3118013.83tokens/s]
Shard 29:  78%|███████▊  | 77989204/100000000 [00:22<00:06, 3145705.32tokens/s]
Shard 29:  78%|███████▊  | 78309688/100000000 [00:22<00:06, 3139410.73tokens/s]
Shard 29:  79%|███████▊  | 78627792/100000000 [00:22<00:07, 2998015.89tokens/s]
Shard 29:  79%|███████▉  | 78931240/100000000 [00:22<00:07, 2787860.39tokens/s]
Shard 29:  79%|███████▉  | 79297294/100000000 [00:22<00:06, 3018444.41tokens/s]
Shard 29:  80%|███████▉  | 79788806/100000000 [00:23<00:05, 3543179.23tokens/s]
Shard 29:  80%|████████  | 80189358/100000000 [00:23<00:05, 3665353.80tokens/s]
Shard 29:  81%|████████  | 80562600/100000000 [00:23<00:05, 3652160.92tokens/s]
Shard 29:  81%|████████  | 81018588/100000000 [00:23<00:04, 3905715.44tokens/s]
Shard 29:  81%|████████▏ | 81415086/100000000 [00:23<00:04, 3766231.72tokens/s]
Shard 29:  82%|████████▏ | 81899441/100000000 [00:23<00:04, 4072729.20tokens/s]
Shard 29:  82%|████████▏ | 82351294/100000000 [00:23<00:04, 4192139.43tokens/s]
Shard 29:  83%|████████▎ | 82774407/100000000 [00:23<00:04, 4072531.08tokens/s]
Shard 29:  83%|████████▎ | 83185206/100000000 [00:23<00:04, 3976207.47tokens/s]
Shard 29:  84%|████████▎ | 83585014/100000000 [00:24<00:04, 3631945.17tokens/s]
Shard 29:  84%|████████▍ | 83955021/100000000 [00:24<00:04, 3560242.97tokens/s]
Shard 29:  84%|████████▍ | 84315487/100000000 [00:24<00:04, 3287541.11tokens/s]
Shard 29:  85%|████████▍ | 84650500/100000000 [00:24<00:04, 3112374.14tokens/s]
Shard 29:  85%|████████▌ | 85356727/100000000 [00:24<00:03, 4123843.05tokens/s]
Shard 29:  86%|████████▌ | 85784910/100000000 [00:24<00:03, 3738791.12tokens/s]
Shard 29:  86%|████████▌ | 86174866/100000000 [00:24<00:03, 3484892.63tokens/s]
Shard 29:  87%|████████▋ | 86536456/100000000 [00:24<00:04, 3149133.54tokens/s]
Shard 29:  87%|████████▋ | 86906492/100000000 [00:25<00:03, 3275332.45tokens/s]
Shard 29:  87%|████████▋ | 87328144/100000000 [00:25<00:03, 3514127.15tokens/s]
Shard 29:  88%|████████▊ | 87817153/100000000 [00:25<00:03, 3883287.76tokens/s]
Shard 29:  88%|████████▊ | 88218464/100000000 [00:25<00:03, 3555497.97tokens/s]
Shard 29:  89%|████████▊ | 88586880/100000000 [00:25<00:03, 3229247.92tokens/s]
Shard 29:  89%|████████▉ | 89002967/100000000 [00:25<00:03, 3443573.74tokens/s]
Shard 29:  89%|████████▉ | 89359847/100000000 [00:25<00:03, 3358795.65tokens/s]
Shard 29:  90%|████████▉ | 89823274/100000000 [00:25<00:02, 3698622.96tokens/s]
Shard 29:  90%|█████████ | 90294933/100000000 [00:25<00:02, 3979678.71tokens/s]
Shard 29:  91%|█████████▏| 91377071/100000000 [00:25<00:01, 5900249.26tokens/s]
Shard 29:  92%|█████████▏| 91985469/100000000 [00:26<00:01, 4993626.91tokens/s]
Shard 29:  93%|█████████▎| 92520464/100000000 [00:26<00:01, 5037976.98tokens/s]
Shard 29:  93%|█████████▎| 93051138/100000000 [00:26<00:01, 5015548.09tokens/s]
Shard 29:  94%|█████████▎| 93571618/100000000 [00:26<00:01, 4507812.74tokens/s]
Shard 29:  94%|█████████▍| 94043713/100000000 [00:26<00:01, 4149479.40tokens/s]
Shard 29:  95%|█████████▍| 94649559/100000000 [00:26<00:01, 4626831.48tokens/s]
Shard 29:  95%|█████████▌| 95176506/100000000 [00:26<00:01, 4777115.26tokens/s]
Shard 29:  96%|█████████▌| 95671144/100000000 [00:26<00:00, 4526803.28tokens/s]
Shard 29:  96%|█████████▋| 96310014/100000000 [00:27<00:00, 4977340.20tokens/s]
Shard 29:  97%|█████████▋| 96907337/100000000 [00:27<00:00, 5245120.92tokens/s]
Shard 29:  97%|█████████▋| 97443840/100000000 [00:27<00:00, 5114436.60tokens/s]
Shard 29:  98%|█████████▊| 97963689/100000000 [00:27<00:00, 4874892.08tokens/s]
Shard 29:  99%|█████████▊| 98534939/100000000 [00:27<00:00, 5098247.93tokens/s]
Shard 29:  99%|█████████▉| 99052051/100000000 [00:27<00:00, 4724086.42tokens/s]
Shard 29: 100%|█████████▉| 99534265/100000000 [00:27<00:00, 4709996.25tokens/s]
Shard 29: 100%|█████████▉| 99999016/100000000 [00:36<00:00, 2759309.54tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000029.bin (200,001,024 bytes) in the gpt-2 format

Shard 30:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 30:   0%|          | 288617/100000000 [00:00<00:34, 2882331.29tokens/s]
Shard 30:   1%|          | 577080/100000000 [00:00<00:38, 2589596.15tokens/s]
Shard 30:   1%|          | 849362/100000000 [00:00<00:37, 2637500.22tokens/s]
Shard 30:   1%|          | 1114650/100000000 [00:00<00:39, 2522191.37tokens/s]
Shard 30:   1%|▏         | 1485187/100000000 [00:00<00:33, 2923828.92tokens/s]
Shard 30:   2%|▏         | 1867670/100000000 [00:00<00:30, 3211450.86tokens/s]
Shard 30:   2%|▏         | 2191851/100000000 [00:00<00:31, 3088749.57tokens/s]
Shard 30:   3%|▎         | 2570110/100000000 [00:00<00:29, 3292858.73tokens/s]
Shard 30:   3%|▎         | 2922232/100000000 [00:00<00:28, 3351999.09tokens/s]
Shard 30:   3%|▎         | 3259603/100000000 [00:01<00:30, 3163071.70tokens/s]
Shard 30:   4%|▎         | 3615776/100000000 [00:01<00:29, 3270574.80tokens/s]
Shard 30:   4%|▍         | 4011137/100000000 [00:01<00:27, 3461644.87tokens/s]
Shard 30:   4%|▍         | 4412643/100000000 [00:01<00:26, 3615445.34tokens/s]
Shard 30:   5%|▍         | 4776601/100000000 [00:01<00:27, 3435708.66tokens/s]
Shard 30:   5%|▌         | 5135209/100000000 [00:01<00:27, 3472901.49tokens/s]
Shard 30:   5%|▌         | 5487818/100000000 [00:01<00:31, 3041252.45tokens/s]
Shard 30:   6%|▌         | 5937352/100000000 [00:01<00:27, 3426290.47tokens/s]
Shard 30:   6%|▋         | 6292682/100000000 [00:01<00:28, 3269412.65tokens/s]
Shard 30:   7%|▋         | 6696742/100000000 [00:02<00:26, 3455754.38tokens/s]
Shard 30:   8%|▊         | 7716143/100000000 [00:02<00:17, 5320494.00tokens/s]
Shard 30:   8%|▊         | 8274230/100000000 [00:02<00:24, 3709176.63tokens/s]
Shard 30:   9%|▊         | 8726422/100000000 [00:02<00:24, 3684061.30tokens/s]
Shard 30:   9%|▉         | 9151137/100000000 [00:02<00:27, 3280541.89tokens/s]
Shard 30:  10%|▉         | 9525154/100000000 [00:02<00:29, 3119169.25tokens/s]
Shard 30:  10%|▉         | 9867499/100000000 [00:02<00:28, 3132030.87tokens/s]
Shard 30:  10%|█         | 10206402/100000000 [00:03<00:30, 2961121.38tokens/s]
Shard 30:  11%|█         | 10560978/100000000 [00:03<00:28, 3085349.98tokens/s]
Shard 30:  11%|█         | 10882521/100000000 [00:03<00:28, 3096034.99tokens/s]
Shard 30:  11%|█▏        | 11284605/100000000 [00:03<00:26, 3337282.28tokens/s]
Shard 30:  12%|█▏        | 11629643/100000000 [00:03<00:27, 3193663.16tokens/s]
Shard 30:  12%|█▏        | 11956749/100000000 [00:03<00:30, 2928388.97tokens/s]
Shard 30:  12%|█▏        | 12262815/100000000 [00:03<00:29, 2950760.27tokens/s]
Shard 30:  13%|█▎        | 12902347/100000000 [00:03<00:22, 3857043.14tokens/s]
Shard 30:  13%|█▎        | 13300655/100000000 [00:03<00:23, 3696368.68tokens/s]
Shard 30:  14%|█▍        | 13750822/100000000 [00:04<00:22, 3900822.88tokens/s]
Shard 30:  14%|█▍        | 14149168/100000000 [00:04<00:22, 3777118.34tokens/s]
Shard 30:  15%|█▍        | 14655728/100000000 [00:04<00:20, 4135388.38tokens/s]
Shard 30:  15%|█▌        | 15099072/100000000 [00:04<00:21, 3897582.31tokens/s]
Shard 30:  16%|█▌        | 15922407/100000000 [00:04<00:16, 5067321.07tokens/s]
Shard 30:  16%|█▋        | 16447941/100000000 [00:04<00:17, 4780590.74tokens/s]
Shard 30:  17%|█▋        | 16940039/100000000 [00:04<00:19, 4231704.14tokens/s]
Shard 30:  17%|█▋        | 17382156/100000000 [00:04<00:20, 3951602.53tokens/s]
Shard 30:  19%|█▊        | 18732535/100000000 [00:05<00:12, 6350817.42tokens/s]
Shard 30:  19%|█▉        | 19435331/100000000 [00:05<00:14, 5488020.80tokens/s]
Shard 30:  20%|██        | 20043129/100000000 [00:05<00:15, 5020526.89tokens/s]
Shard 30:  21%|██        | 20590027/100000000 [00:05<00:18, 4278013.81tokens/s]
Shard 30:  21%|██        | 21061272/100000000 [00:05<00:21, 3717674.04tokens/s]
Shard 30:  21%|██▏       | 21473103/100000000 [00:05<00:21, 3628458.63tokens/s]
Shard 30:  22%|██▏       | 21859701/100000000 [00:06<00:26, 2987928.33tokens/s]
Shard 30:  22%|██▏       | 22204948/100000000 [00:06<00:25, 3074639.76tokens/s]
Shard 30:  23%|██▎       | 22552707/100000000 [00:06<00:24, 3162965.45tokens/s]
Shard 30:  23%|██▎       | 22962901/100000000 [00:06<00:22, 3387397.63tokens/s]
Shard 30:  23%|██▎       | 23322356/100000000 [00:06<00:24, 3068411.99tokens/s]
Shard 30:  24%|██▎       | 23703547/100000000 [00:06<00:23, 3249794.85tokens/s]
Shard 30:  24%|██▍       | 24044016/100000000 [00:06<00:25, 3001661.02tokens/s]
Shard 30:  24%|██▍       | 24357285/100000000 [00:06<00:25, 2996229.04tokens/s]
Shard 30:  25%|██▍       | 24666301/100000000 [00:06<00:24, 3018326.30tokens/s]
Shard 30:  25%|██▍       | 24987427/100000000 [00:07<00:24, 3056814.31tokens/s]
Shard 30:  25%|██▌       | 25299001/100000000 [00:07<00:25, 2956839.49tokens/s]
Shard 30:  26%|██▌       | 25609656/100000000 [00:07<00:25, 2971983.05tokens/s]
Shard 30:  26%|██▌       | 25911223/100000000 [00:07<00:28, 2566469.11tokens/s]
Shard 30:  26%|██▌       | 26193681/100000000 [00:07<00:30, 2417479.01tokens/s]
Shard 30:  27%|██▋       | 26547741/100000000 [00:07<00:27, 2696821.00tokens/s]
Shard 30:  27%|██▋       | 26894861/100000000 [00:07<00:25, 2899866.29tokens/s]
Shard 30:  27%|██▋       | 27196204/100000000 [00:07<00:26, 2758593.56tokens/s]
Shard 30:  28%|██▊       | 27531293/100000000 [00:07<00:24, 2910046.04tokens/s]
Shard 30:  28%|██▊       | 27829847/100000000 [00:08<00:24, 2923819.46tokens/s]
Shard 30:  28%|██▊       | 28276550/100000000 [00:08<00:21, 3349429.19tokens/s]
Shard 30:  29%|██▊       | 28732516/100000000 [00:08<00:19, 3697185.80tokens/s]
Shard 30:  29%|██▉       | 29109866/100000000 [00:08<00:20, 3446088.60tokens/s]
Shard 30:  29%|██▉       | 29468595/100000000 [00:08<00:21, 3269856.65tokens/s]
Shard 30:  30%|██▉       | 29830926/100000000 [00:08<00:20, 3362851.93tokens/s]
Shard 30:  30%|███       | 30172838/100000000 [00:08<00:20, 3355058.39tokens/s]
Shard 30:  31%|███       | 30520293/100000000 [00:08<00:20, 3376612.03tokens/s]
Shard 30:  31%|███       | 30860685/100000000 [00:08<00:21, 3256993.71tokens/s]
Shard 30:  31%|███       | 31189370/100000000 [00:09<00:22, 3031893.85tokens/s]
Shard 30:  31%|███▏      | 31496702/100000000 [00:09<00:23, 2870741.61tokens/s]
Shard 30:  32%|███▏      | 31787682/100000000 [00:09<00:24, 2746023.03tokens/s]
Shard 30:  32%|███▏      | 32090584/100000000 [00:09<00:24, 2806110.00tokens/s]
Shard 30:  32%|███▏      | 32374355/100000000 [00:09<00:28, 2361395.26tokens/s]
Shard 30:  33%|███▎      | 32638104/100000000 [00:09<00:27, 2416979.87tokens/s]
Shard 30:  33%|███▎      | 33005001/100000000 [00:09<00:24, 2743097.59tokens/s]
Shard 30:  33%|███▎      | 33291850/100000000 [00:09<00:24, 2706440.37tokens/s]
Shard 30:  34%|███▎      | 33570681/100000000 [00:10<00:28, 2368283.08tokens/s]
Shard 30:  34%|███▍      | 33819636/100000000 [00:10<00:29, 2231252.92tokens/s]
Shard 30:  34%|███▍      | 34051955/100000000 [00:10<00:30, 2137138.32tokens/s]
Shard 30:  34%|███▍      | 34271692/100000000 [00:10<00:31, 2056028.13tokens/s]
Shard 30:  34%|███▍      | 34495349/100000000 [00:10<00:31, 2087463.69tokens/s]
Shard 30:  35%|███▍      | 34707275/100000000 [00:10<00:31, 2068293.39tokens/s]
Shard 30:  35%|███▍      | 34977454/100000000 [00:10<00:29, 2227419.26tokens/s]
Shard 30:  35%|███▌      | 35278700/100000000 [00:10<00:26, 2435489.19tokens/s]
Shard 30:  36%|███▌      | 35554879/100000000 [00:10<00:25, 2513219.50tokens/s]
Shard 30:  36%|███▌      | 35809934/100000000 [00:11<00:30, 2131764.68tokens/s]
Shard 30:  36%|███▌      | 36038079/100000000 [00:11<00:29, 2169021.28tokens/s]
Shard 30:  36%|███▋      | 36286377/100000000 [00:11<00:28, 2213349.68tokens/s]
Shard 30:  37%|███▋      | 36548952/100000000 [00:11<00:30, 2086359.20tokens/s]
Shard 30:  37%|███▋      | 36765903/100000000 [00:11<00:30, 2106727.13tokens/s]
Shard 30:  37%|███▋      | 37029389/100000000 [00:11<00:28, 2240635.10tokens/s]
Shard 30:  37%|███▋      | 37260288/100000000 [00:11<00:27, 2254619.06tokens/s]
Shard 30:  37%|███▋      | 37497675/100000000 [00:11<00:27, 2280973.60tokens/s]
Shard 30:  38%|███▊      | 37904086/100000000 [00:11<00:22, 2790225.81tokens/s]
Shard 30:  38%|███▊      | 38206392/100000000 [00:12<00:21, 2850659.58tokens/s]
Shard 30:  38%|███▊      | 38495239/100000000 [00:12<00:23, 2672423.26tokens/s]
Shard 30:  39%|███▉      | 38860020/100000000 [00:12<00:20, 2941464.30tokens/s]
Shard 30:  39%|███▉      | 39158649/100000000 [00:12<00:20, 2931526.90tokens/s]
Shard 30:  39%|███▉      | 39455810/100000000 [00:12<00:22, 2745912.67tokens/s]
Shard 30:  40%|███▉      | 39735096/100000000 [00:12<00:22, 2639722.47tokens/s]
Shard 30:  40%|████      | 40002480/100000000 [00:12<00:23, 2536704.24tokens/s]
Shard 30:  40%|████      | 40259918/100000000 [00:12<00:27, 2178383.43tokens/s]
Shard 30:  40%|████      | 40487096/100000000 [00:13<00:28, 2122930.12tokens/s]
Shard 30:  41%|████      | 40778080/100000000 [00:13<00:25, 2303136.68tokens/s]
Shard 30:  41%|████      | 41074044/100000000 [00:13<00:23, 2468118.34tokens/s]
Shard 30:  41%|████▏     | 41327584/100000000 [00:13<00:24, 2428930.42tokens/s]
Shard 30:  42%|████▏     | 41656800/100000000 [00:13<00:21, 2666326.83tokens/s]
Shard 30:  42%|████▏     | 42077462/100000000 [00:13<00:18, 3098200.52tokens/s]
Shard 30:  43%|████▎     | 42659717/100000000 [00:13<00:14, 3874431.04tokens/s]
Shard 30:  43%|████▎     | 43057389/100000000 [00:13<00:16, 3517708.39tokens/s]
Shard 30:  43%|████▎     | 43420105/100000000 [00:13<00:15, 3537831.44tokens/s]
Shard 30:  44%|████▍     | 43781795/100000000 [00:14<00:17, 3305052.17tokens/s]
Shard 30:  44%|████▍     | 44120406/100000000 [00:14<00:18, 3059943.51tokens/s]
Shard 30:  44%|████▍     | 44435248/100000000 [00:14<00:19, 2914315.28tokens/s]
Shard 30:  45%|████▍     | 44732767/100000000 [00:14<00:20, 2702137.65tokens/s]
Shard 30:  45%|████▌     | 45008432/100000000 [00:14<00:21, 2587263.14tokens/s]
Shard 30:  45%|████▌     | 45445018/100000000 [00:14<00:17, 3035848.56tokens/s]
Shard 30:  46%|████▌     | 45757513/100000000 [00:14<00:20, 2708180.41tokens/s]
Shard 30:  46%|████▌     | 46053924/100000000 [00:14<00:19, 2727158.99tokens/s]
Shard 30:  46%|████▋     | 46400107/100000000 [00:14<00:18, 2890180.56tokens/s]
Shard 30:  47%|████▋     | 46698683/100000000 [00:15<00:20, 2636677.35tokens/s]
Shard 30:  47%|████▋     | 47000491/100000000 [00:15<00:19, 2680542.66tokens/s]
Shard 30:  47%|████▋     | 47275129/100000000 [00:15<00:20, 2633313.30tokens/s]
Shard 30:  48%|████▊     | 47599612/100000000 [00:15<00:18, 2763802.05tokens/s]
Shard 30:  48%|████▊     | 47928853/100000000 [00:15<00:17, 2907714.53tokens/s]
Shard 30:  48%|████▊     | 48249337/100000000 [00:15<00:17, 2991193.33tokens/s]
Shard 30:  49%|████▊     | 48583114/100000000 [00:15<00:16, 3085713.53tokens/s]
Shard 30:  49%|████▉     | 48894644/100000000 [00:15<00:16, 3024350.99tokens/s]
Shard 30:  49%|████▉     | 49200188/100000000 [00:15<00:17, 2863303.87tokens/s]
Shard 30:  50%|████▉     | 49583098/100000000 [00:16<00:16, 3133958.42tokens/s]
Shard 30:  50%|████▉     | 49900374/100000000 [00:16<00:16, 3049682.59tokens/s]
Shard 30:  50%|█████     | 50209461/100000000 [00:16<00:16, 2996499.94tokens/s]
Shard 30:  51%|█████     | 50538093/100000000 [00:16<00:16, 3072458.97tokens/s]
Shard 30:  51%|█████     | 50848438/100000000 [00:16<00:16, 3054099.82tokens/s]
Shard 30:  51%|█████     | 51186153/100000000 [00:16<00:15, 3136676.63tokens/s]
Shard 30:  52%|█████▏    | 52296971/100000000 [00:16<00:08, 5333232.55tokens/s]
Shard 30:  53%|█████▎    | 52825249/100000000 [00:16<00:09, 4994300.09tokens/s]
Shard 30:  53%|█████▎    | 53324095/100000000 [00:16<00:10, 4321450.10tokens/s]
Shard 30:  54%|█████▍    | 54041392/100000000 [00:17<00:09, 5047879.08tokens/s]
Shard 30:  55%|█████▍    | 54567367/100000000 [00:17<00:10, 4198489.21tokens/s]
Shard 30:  55%|█████▌    | 55023391/100000000 [00:17<00:11, 3959344.06tokens/s]
Shard 30:  55%|█████▌    | 55444433/100000000 [00:17<00:12, 3569515.27tokens/s]
Shard 30:  56%|█████▌    | 55821825/100000000 [00:17<00:12, 3496149.52tokens/s]
Shard 30:  56%|█████▌    | 56184712/100000000 [00:17<00:12, 3409115.12tokens/s]
Shard 30:  57%|█████▋    | 57053644/100000000 [00:17<00:09, 4710636.89tokens/s]
Shard 30:  58%|█████▊    | 57553122/100000000 [00:18<00:10, 3904939.81tokens/s]
Shard 30:  58%|█████▊    | 57983503/100000000 [00:18<00:10, 3857632.54tokens/s]
Shard 30:  58%|█████▊    | 58410188/100000000 [00:18<00:11, 3709292.12tokens/s]
Shard 30:  59%|█████▉    | 58872450/100000000 [00:18<00:10, 3932335.60tokens/s]
Shard 30:  59%|█████▉    | 59294332/100000000 [00:18<00:10, 3993924.50tokens/s]
Shard 30:  60%|█████▉    | 59759330/100000000 [00:18<00:09, 4162094.80tokens/s]
Shard 30:  60%|██████    | 60195023/100000000 [00:18<00:10, 3783649.06tokens/s]
Shard 30:  61%|██████    | 60587467/100000000 [00:18<00:10, 3677269.54tokens/s]
Shard 30:  61%|██████    | 60965608/100000000 [00:18<00:10, 3692101.21tokens/s]
Shard 30:  61%|██████▏   | 61341692/100000000 [00:19<00:10, 3558665.22tokens/s]
Shard 30:  62%|██████▏   | 61702328/100000000 [00:19<00:10, 3539809.01tokens/s]
Shard 30:  62%|██████▏   | 62097194/100000000 [00:19<00:10, 3653267.75tokens/s]
Shard 30:  63%|██████▎   | 62529666/100000000 [00:19<00:09, 3834659.96tokens/s]
Shard 30:  63%|██████▎   | 62916678/100000000 [00:19<00:09, 3733857.23tokens/s]
Shard 30:  63%|██████▎   | 63293025/100000000 [00:19<00:10, 3505774.52tokens/s]
Shard 30:  64%|██████▎   | 63647434/100000000 [00:19<00:10, 3394305.04tokens/s]
Shard 30:  64%|██████▍   | 63993738/100000000 [00:19<00:12, 2958895.62tokens/s]
Shard 30:  64%|██████▍   | 64464954/100000000 [00:19<00:10, 3403617.85tokens/s]
Shard 30:  65%|██████▌   | 65082809/100000000 [00:20<00:08, 4139662.72tokens/s]
Shard 30:  66%|██████▌   | 65516362/100000000 [00:20<00:08, 3979847.87tokens/s]
Shard 30:  66%|██████▌   | 66060128/100000000 [00:20<00:07, 4375556.09tokens/s]
Shard 30:  67%|██████▋   | 66511713/100000000 [00:20<00:09, 3575467.78tokens/s]
Shard 30:  67%|██████▋   | 66903021/100000000 [00:20<00:09, 3565719.63tokens/s]
Shard 30:  67%|██████▋   | 67282503/100000000 [00:20<00:10, 3121712.11tokens/s]
Shard 30:  68%|██████▊   | 67617614/100000000 [00:20<00:10, 3153078.98tokens/s]
Shard 30:  68%|██████▊   | 67950497/100000000 [00:21<00:11, 2865393.41tokens/s]
Shard 30:  68%|██████▊   | 68252288/100000000 [00:21<00:11, 2741318.48tokens/s]
Shard 30:  69%|██████▊   | 68561951/100000000 [00:21<00:11, 2826264.02tokens/s]
Shard 30:  69%|██████▉   | 68856520/100000000 [00:21<00:11, 2725918.00tokens/s]
Shard 30:  69%|██████▉   | 69282690/100000000 [00:21<00:09, 3123941.68tokens/s]
Shard 30:  70%|██████▉   | 69607411/100000000 [00:21<00:10, 2917591.72tokens/s]
Shard 30:  70%|██████▉   | 69907637/100000000 [00:21<00:10, 2835612.21tokens/s]
Shard 30:  70%|███████   | 70206484/100000000 [00:21<00:10, 2876084.34tokens/s]
Shard 30:  71%|███████   | 70685306/100000000 [00:21<00:08, 3404118.37tokens/s]
Shard 30:  71%|███████   | 71033402/100000000 [00:22<00:09, 3199505.95tokens/s]
Shard 30:  71%|███████▏  | 71369584/100000000 [00:22<00:08, 3232944.38tokens/s]
Shard 30:  72%|███████▏  | 71698466/100000000 [00:22<00:10, 2711487.02tokens/s]
Shard 30:  72%|███████▏  | 71987134/100000000 [00:22<00:10, 2623219.84tokens/s]
Shard 30:  72%|███████▏  | 72324664/100000000 [00:22<00:09, 2808620.95tokens/s]
Shard 30:  73%|███████▎  | 72688916/100000000 [00:22<00:09, 2964238.79tokens/s]
Shard 30:  73%|███████▎  | 72994400/100000000 [00:22<00:09, 2968946.31tokens/s]
Shard 30:  73%|███████▎  | 73315173/100000000 [00:22<00:08, 3027572.73tokens/s]
Shard 30:  74%|███████▎  | 73736063/100000000 [00:22<00:07, 3354378.82tokens/s]
Shard 30:  74%|███████▍  | 74076631/100000000 [00:23<00:07, 3305445.98tokens/s]
Shard 30:  75%|███████▍  | 74561871/100000000 [00:23<00:06, 3748052.14tokens/s]
Shard 30:  75%|███████▌  | 75012225/100000000 [00:23<00:06, 3925345.52tokens/s]
Shard 30:  75%|███████▌  | 75408076/100000000 [00:23<00:06, 3786998.35tokens/s]
Shard 30:  76%|███████▌  | 75789872/100000000 [00:23<00:06, 3626234.73tokens/s]
Shard 30:  76%|███████▌  | 76155948/100000000 [00:23<00:06, 3559042.61tokens/s]
Shard 30:  77%|███████▋  | 76514342/100000000 [00:23<00:06, 3433020.38tokens/s]
Shard 30:  77%|███████▋  | 76863755/100000000 [00:23<00:06, 3433456.17tokens/s]
Shard 30:  77%|███████▋  | 77208376/100000000 [00:23<00:07, 3152868.83tokens/s]
Shard 30:  78%|███████▊  | 77669120/100000000 [00:24<00:06, 3538880.39tokens/s]
Shard 30:  78%|███████▊  | 78030086/100000000 [00:24<00:06, 3221803.38tokens/s]
Shard 30:  78%|███████▊  | 78391301/100000000 [00:24<00:06, 3325274.80tokens/s]
Shard 30:  79%|███████▉  | 78769442/100000000 [00:24<00:06, 3447447.37tokens/s]
Shard 30:  79%|███████▉  | 79120887/100000000 [00:24<00:06, 3206877.91tokens/s]
Shard 30:  79%|███████▉  | 79449124/100000000 [00:24<00:06, 3068838.54tokens/s]
Shard 30:  80%|███████▉  | 79761573/100000000 [00:24<00:07, 2822588.66tokens/s]
Shard 30:  80%|████████  | 80051052/100000000 [00:24<00:07, 2706512.19tokens/s]
Shard 30:  80%|████████  | 80325866/100000000 [00:24<00:07, 2629141.23tokens/s]
Shard 30:  81%|████████  | 80608634/100000000 [00:25<00:07, 2665152.83tokens/s]
Shard 30:  81%|████████  | 80917757/100000000 [00:25<00:06, 2777915.36tokens/s]
Shard 30:  81%|████████▏ | 81288592/100000000 [00:25<00:06, 3027323.80tokens/s]
Shard 30:  82%|████████▏ | 81599017/100000000 [00:25<00:06, 3043752.20tokens/s]
Shard 30:  82%|████████▏ | 81970444/100000000 [00:25<00:05, 3237244.52tokens/s]
Shard 30:  83%|████████▎ | 82533923/100000000 [00:25<00:04, 3937752.37tokens/s]
Shard 30:  83%|████████▎ | 82934630/100000000 [00:25<00:05, 3204934.99tokens/s]
Shard 30:  83%|████████▎ | 83282297/100000000 [00:25<00:05, 3174797.73tokens/s]
Shard 30:  84%|████████▎ | 83617135/100000000 [00:26<00:05, 2857811.30tokens/s]
Shard 30:  84%|████████▍ | 83931175/100000000 [00:26<00:05, 2922701.82tokens/s]
Shard 30:  84%|████████▍ | 84334217/100000000 [00:26<00:04, 3209637.40tokens/s]
Shard 30:  85%|████████▍ | 84668295/100000000 [00:26<00:04, 3156587.16tokens/s]
Shard 30:  85%|████████▍ | 84992862/100000000 [00:26<00:04, 3152039.55tokens/s]
Shard 30:  85%|████████▌ | 85315482/100000000 [00:26<00:04, 2955977.02tokens/s]
Shard 30:  86%|████████▌ | 85769435/100000000 [00:26<00:04, 3370141.20tokens/s]
Shard 30:  86%|████████▌ | 86116260/100000000 [00:26<00:04, 3215318.82tokens/s]
Shard 30:  86%|████████▋ | 86444317/100000000 [00:26<00:04, 3182237.19tokens/s]
Shard 30:  87%|████████▋ | 86811902/100000000 [00:26<00:04, 3296980.40tokens/s]
Shard 30:  87%|████████▋ | 87146067/100000000 [00:27<00:04, 3030749.82tokens/s]
Shard 30:  87%|████████▋ | 87464724/100000000 [00:27<00:04, 2750632.02tokens/s]
Shard 30:  88%|████████▊ | 87778577/100000000 [00:27<00:04, 2839931.34tokens/s]
Shard 30:  88%|████████▊ | 88069699/100000000 [00:27<00:04, 2732450.00tokens/s]
Shard 30:  88%|████████▊ | 88353210/100000000 [00:27<00:04, 2758672.20tokens/s]
Shard 30:  89%|████████▊ | 88632909/100000000 [00:27<00:04, 2664519.19tokens/s]
Shard 30:  89%|████████▉ | 88973773/100000000 [00:27<00:03, 2865479.34tokens/s]
Shard 30:  89%|████████▉ | 89292406/100000000 [00:27<00:03, 2949041.24tokens/s]
Shard 30:  90%|████████▉ | 89591915/100000000 [00:28<00:03, 2946776.65tokens/s]
Shard 30:  90%|████████▉ | 89897763/100000000 [00:28<00:03, 2977818.06tokens/s]
Shard 30:  90%|█████████ | 90286799/100000000 [00:28<00:02, 3244620.07tokens/s]
Shard 30:  91%|█████████ | 90613779/100000000 [00:28<00:03, 3052612.39tokens/s]
Shard 30:  91%|█████████ | 91080581/100000000 [00:28<00:02, 3504063.50tokens/s]
Shard 30:  91%|█████████▏| 91460657/100000000 [00:28<00:02, 3586236.89tokens/s]
Shard 30:  92%|█████████▏| 91823857/100000000 [00:28<00:02, 3401154.82tokens/s]
Shard 30:  92%|█████████▏| 92170041/100000000 [00:28<00:02, 3328127.57tokens/s]
Shard 30:  93%|█████████▎| 92506571/100000000 [00:28<00:02, 2976928.53tokens/s]
Shard 30:  93%|█████████▎| 92812292/100000000 [00:29<00:02, 2703346.49tokens/s]
Shard 30:  93%|█████████▎| 93107877/100000000 [00:29<00:02, 2748245.38tokens/s]
Shard 30:  93%|█████████▎| 93407475/100000000 [00:29<00:02, 2801341.68tokens/s]
Shard 30:  94%|█████████▍| 93766366/100000000 [00:29<00:02, 3016737.36tokens/s]
Shard 30:  94%|█████████▍| 94080376/100000000 [00:29<00:01, 3040433.23tokens/s]
Shard 30:  94%|█████████▍| 94389540/100000000 [00:29<00:01, 2905688.37tokens/s]
Shard 30:  95%|█████████▍| 94686511/100000000 [00:29<00:01, 2844301.60tokens/s]
Shard 30:  95%|█████████▌| 95033211/100000000 [00:29<00:01, 3009086.17tokens/s]
Shard 30:  95%|█████████▌| 95388838/100000000 [00:29<00:01, 3157955.75tokens/s]
Shard 30:  96%|█████████▌| 95707848/100000000 [00:30<00:01, 2664259.54tokens/s]
Shard 30:  96%|█████████▌| 95995912/100000000 [00:30<00:01, 2694294.64tokens/s]
Shard 30:  96%|█████████▋| 96290638/100000000 [00:30<00:01, 2744749.10tokens/s]
Shard 30:  97%|█████████▋| 96640960/100000000 [00:30<00:01, 2952683.49tokens/s]
Shard 30:  97%|█████████▋| 97013966/100000000 [00:30<00:00, 3158763.66tokens/s]
Shard 30:  97%|█████████▋| 97404172/100000000 [00:30<00:00, 3359269.66tokens/s]
Shard 30:  98%|█████████▊| 97745478/100000000 [00:30<00:00, 3301928.47tokens/s]
Shard 30:  98%|█████████▊| 98130377/100000000 [00:30<00:00, 3394016.23tokens/s]
Shard 30:  99%|█████████▊| 98528674/100000000 [00:30<00:00, 3551584.12tokens/s]
Shard 30:  99%|█████████▉| 98886399/100000000 [00:30<00:00, 3529390.36tokens/s]
Shard 30:  99%|█████████▉| 99241369/100000000 [00:31<00:00, 3392906.33tokens/s]
Shard 30: 100%|█████████▉| 99583051/100000000 [00:31<00:00, 3286630.20tokens/s]
Shard 30: 100%|█████████▉| 99913793/100000000 [00:31<00:00, 2371917.29tokens/s]
Shard 30: 100%|█████████▉| 99996032/100000000 [00:40<00:00, 2494255.99tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000030.bin (200,001,024 bytes) in the gpt-2 format

Shard 31:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 31:   0%|          | 449807/100000000 [00:00<00:22, 4439269.29tokens/s]
Shard 31:   1%|          | 894218/100000000 [00:00<00:30, 3225794.44tokens/s]
Shard 31:   1%|▏         | 1397869/100000000 [00:00<00:25, 3899466.25tokens/s]
Shard 31:   2%|▏         | 1812408/100000000 [00:00<00:29, 3371025.70tokens/s]
Shard 31:   2%|▏         | 2172775/100000000 [00:00<00:31, 3147754.08tokens/s]
Shard 31:   3%|▎         | 2503088/100000000 [00:00<00:33, 2873476.34tokens/s]
Shard 31:   3%|▎         | 2800182/100000000 [00:00<00:35, 2727137.42tokens/s]
Shard 31:   3%|▎         | 3172872/100000000 [00:01<00:33, 2921531.65tokens/s]
Shard 31:   3%|▎         | 3476261/100000000 [00:01<00:38, 2497617.16tokens/s]
Shard 31:   4%|▍         | 3844546/100000000 [00:01<00:34, 2777731.48tokens/s]
Shard 31:   4%|▍         | 4194191/100000000 [00:01<00:32, 2957867.81tokens/s]
Shard 31:   5%|▍         | 4503941/100000000 [00:01<00:35, 2685882.66tokens/s]
Shard 31:   5%|▍         | 4787225/100000000 [00:01<00:40, 2329836.36tokens/s]
Shard 31:   5%|▌         | 5035133/100000000 [00:01<00:43, 2207767.25tokens/s]
Shard 31:   5%|▌         | 5418253/100000000 [00:01<00:36, 2587166.67tokens/s]
Shard 31:   6%|▌         | 5706883/100000000 [00:02<00:35, 2649724.62tokens/s]
Shard 31:   6%|▌         | 6045426/100000000 [00:02<00:33, 2840402.10tokens/s]
Shard 31:   6%|▋         | 6370069/100000000 [00:02<00:31, 2946519.94tokens/s]
Shard 31:   7%|▋         | 6673169/100000000 [00:02<00:31, 2926194.29tokens/s]
Shard 31:   7%|▋         | 7059275/100000000 [00:02<00:29, 3177364.57tokens/s]
Shard 31:   7%|▋         | 7421774/100000000 [00:02<00:28, 3303281.68tokens/s]
Shard 31:   8%|▊         | 7756684/100000000 [00:02<00:28, 3259200.85tokens/s]
Shard 31:   8%|▊         | 8087275/100000000 [00:02<00:28, 3226122.55tokens/s]
Shard 31:   8%|▊         | 8416614/100000000 [00:02<00:28, 3240088.45tokens/s]
Shard 31:   9%|▉         | 8754983/100000000 [00:02<00:27, 3281675.29tokens/s]
Shard 31:   9%|▉         | 9221861/100000000 [00:03<00:24, 3690137.14tokens/s]
Shard 31:  10%|▉         | 9592642/100000000 [00:03<00:25, 3532777.38tokens/s]
Shard 31:  10%|█         | 10370482/100000000 [00:03<00:18, 4739755.71tokens/s]
Shard 31:  11%|█         | 10851690/100000000 [00:03<00:19, 4653461.56tokens/s]
Shard 31:  11%|█▏        | 11322320/100000000 [00:03<00:20, 4378830.45tokens/s]
Shard 31:  12%|█▏        | 11812948/100000000 [00:03<00:21, 4154862.89tokens/s]
Shard 31:  12%|█▏        | 12236415/100000000 [00:03<00:25, 3482063.96tokens/s]
Shard 31:  13%|█▎        | 12693470/100000000 [00:03<00:23, 3727647.13tokens/s]
Shard 31:  13%|█▎        | 13086192/100000000 [00:04<00:26, 3336161.44tokens/s]
Shard 31:  13%|█▎        | 13495489/100000000 [00:04<00:24, 3510280.43tokens/s]
Shard 31:  14%|█▍        | 13863302/100000000 [00:04<00:24, 3524906.46tokens/s]
Shard 31:  14%|█▍        | 14228115/100000000 [00:04<00:24, 3466610.28tokens/s]
Shard 31:  15%|█▍        | 14618010/100000000 [00:04<00:24, 3553719.39tokens/s]
Shard 31:  15%|█▍        | 14981362/100000000 [00:04<00:27, 3077788.60tokens/s]
Shard 31:  15%|█▌        | 15306618/100000000 [00:04<00:27, 3103916.25tokens/s]
Shard 31:  16%|█▌        | 15627943/100000000 [00:04<00:28, 2969771.61tokens/s]
Shard 31:  16%|█▌        | 15933064/100000000 [00:04<00:28, 2969361.20tokens/s]
Shard 31:  16%|█▋        | 16271797/100000000 [00:05<00:27, 3083256.82tokens/s]
Shard 31:  17%|█▋        | 16605691/100000000 [00:05<00:26, 3141043.97tokens/s]
Shard 31:  17%|█▋        | 16923809/100000000 [00:05<00:27, 3046670.93tokens/s]
Shard 31:  17%|█▋        | 17354315/100000000 [00:05<00:24, 3401599.33tokens/s]
Shard 31:  18%|█▊        | 17749007/100000000 [00:05<00:23, 3545881.70tokens/s]
Shard 31:  18%|█▊        | 18115812/100000000 [00:05<00:22, 3573651.37tokens/s]
Shard 31:  19%|█▊        | 18521199/100000000 [00:05<00:21, 3705879.55tokens/s]
Shard 31:  19%|█▉        | 18894136/100000000 [00:05<00:22, 3661697.69tokens/s]
Shard 31:  19%|█▉        | 19261867/100000000 [00:05<00:24, 3348063.30tokens/s]
Shard 31:  20%|█▉        | 19603457/100000000 [00:06<00:26, 3013679.28tokens/s]
Shard 31:  20%|█▉        | 19959673/100000000 [00:06<00:25, 3137210.27tokens/s]
Shard 31:  20%|██        | 20281586/100000000 [00:06<00:27, 2860297.82tokens/s]
Shard 31:  21%|██        | 20577515/100000000 [00:06<00:27, 2872171.62tokens/s]
Shard 31:  21%|██        | 20871291/100000000 [00:06<00:29, 2693263.28tokens/s]
Shard 31:  21%|██        | 21146362/100000000 [00:06<00:29, 2689520.04tokens/s]
Shard 31:  21%|██▏       | 21472680/100000000 [00:06<00:27, 2808292.49tokens/s]
Shard 31:  22%|██▏       | 21761201/100000000 [00:06<00:27, 2796695.63tokens/s]
Shard 31:  22%|██▏       | 22079658/100000000 [00:06<00:26, 2905848.21tokens/s]
Shard 31:  22%|██▏       | 22467161/100000000 [00:07<00:24, 3183079.56tokens/s]
Shard 31:  23%|██▎       | 22864049/100000000 [00:07<00:22, 3409278.74tokens/s]
Shard 31:  23%|██▎       | 23211935/100000000 [00:07<00:28, 2728283.83tokens/s]
Shard 31:  24%|██▎       | 23557797/100000000 [00:07<00:26, 2901588.83tokens/s]
Shard 31:  24%|██▍       | 24034926/100000000 [00:07<00:22, 3372921.13tokens/s]
Shard 31:  24%|██▍       | 24453221/100000000 [00:07<00:21, 3584204.42tokens/s]
Shard 31:  25%|██▍       | 24830408/100000000 [00:07<00:22, 3398954.47tokens/s]
Shard 31:  25%|██▌       | 25188378/100000000 [00:07<00:21, 3422470.69tokens/s]
Shard 31:  26%|██▌       | 25544046/100000000 [00:07<00:22, 3309701.39tokens/s]
Shard 31:  26%|██▌       | 26061164/100000000 [00:08<00:19, 3816168.44tokens/s]
Shard 31:  26%|██▋       | 26451886/100000000 [00:08<00:21, 3472245.08tokens/s]
Shard 31:  27%|██▋       | 26816373/100000000 [00:08<00:22, 3294910.64tokens/s]
Shard 31:  27%|██▋       | 27166737/100000000 [00:08<00:21, 3338312.90tokens/s]
Shard 31:  28%|██▊       | 27509212/100000000 [00:08<00:23, 3085748.10tokens/s]
Shard 31:  28%|██▊       | 27829859/100000000 [00:08<00:23, 3058280.24tokens/s]
Shard 31:  28%|██▊       | 28140456/100000000 [00:08<00:23, 3012651.10tokens/s]
Shard 31:  28%|██▊       | 28444906/100000000 [00:08<00:24, 2908013.29tokens/s]
Shard 31:  29%|██▉       | 28752099/100000000 [00:09<00:24, 2952276.51tokens/s]
Shard 31:  29%|██▉       | 29063697/100000000 [00:09<00:23, 2997000.33tokens/s]
Shard 31:  30%|██▉       | 29511669/100000000 [00:09<00:20, 3412526.50tokens/s]
Shard 31:  30%|██▉       | 29976095/100000000 [00:09<00:18, 3767667.05tokens/s]
Shard 31:  30%|███       | 30480304/100000000 [00:09<00:16, 4139745.94tokens/s]
Shard 31:  31%|███       | 30936982/100000000 [00:09<00:16, 4245073.22tokens/s]
Shard 31:  31%|███▏      | 31379767/100000000 [00:09<00:15, 4298479.62tokens/s]
Shard 31:  32%|███▏      | 31934088/100000000 [00:09<00:14, 4637584.35tokens/s]
Shard 31:  32%|███▏      | 32401445/100000000 [00:09<00:14, 4541151.10tokens/s]
Shard 31:  33%|███▎      | 32856675/100000000 [00:09<00:16, 4020740.49tokens/s]
Shard 31:  33%|███▎      | 33269898/100000000 [00:10<00:18, 3566418.63tokens/s]
Shard 31:  34%|███▎      | 33694744/100000000 [00:10<00:17, 3738717.19tokens/s]
Shard 31:  34%|███▍      | 34125160/100000000 [00:10<00:16, 3888433.81tokens/s]
Shard 31:  35%|███▍      | 34525504/100000000 [00:10<00:17, 3800764.28tokens/s]
Shard 31:  35%|███▌      | 35017385/100000000 [00:10<00:15, 4074978.26tokens/s]
Shard 31:  36%|███▌      | 35550521/100000000 [00:10<00:14, 4413926.40tokens/s]
Shard 31:  36%|███▌      | 35999926/100000000 [00:10<00:18, 3480141.77tokens/s]
Shard 31:  36%|███▋      | 36452384/100000000 [00:10<00:17, 3730674.73tokens/s]
Shard 31:  37%|███▋      | 36888978/100000000 [00:11<00:16, 3884179.62tokens/s]
Shard 31:  37%|███▋      | 37301099/100000000 [00:11<00:19, 3250500.17tokens/s]
Shard 31:  38%|███▊      | 37658165/100000000 [00:11<00:19, 3121397.14tokens/s]
Shard 31:  38%|███▊      | 38103958/100000000 [00:11<00:17, 3443745.12tokens/s]
Shard 31:  39%|███▊      | 38593714/100000000 [00:11<00:16, 3793535.05tokens/s]
Shard 31:  39%|███▉      | 39156534/100000000 [00:11<00:14, 4274111.43tokens/s]
Shard 31:  40%|███▉      | 39604292/100000000 [00:11<00:13, 4314518.94tokens/s]
Shard 31:  40%|████      | 40063109/100000000 [00:11<00:13, 4383546.56tokens/s]
Shard 31:  41%|████      | 40516734/100000000 [00:12<00:15, 3767593.54tokens/s]
Shard 31:  41%|████      | 40939544/100000000 [00:12<00:15, 3886670.61tokens/s]
Shard 31:  41%|████▏     | 41347948/100000000 [00:12<00:16, 3622937.51tokens/s]
Shard 31:  42%|████▏     | 41725843/100000000 [00:12<00:16, 3504156.70tokens/s]
Shard 31:  42%|████▏     | 42086313/100000000 [00:12<00:16, 3501878.21tokens/s]
Shard 31:  43%|████▎     | 42595640/100000000 [00:12<00:14, 3933331.83tokens/s]
Shard 31:  43%|████▎     | 42998583/100000000 [00:12<00:15, 3720253.80tokens/s]
Shard 31:  43%|████▎     | 43492592/100000000 [00:12<00:13, 4043039.03tokens/s]
Shard 31:  44%|████▍     | 43905998/100000000 [00:12<00:17, 3295091.05tokens/s]
Shard 31:  44%|████▍     | 44309234/100000000 [00:13<00:16, 3462418.76tokens/s]
Shard 31:  45%|████▍     | 44720011/100000000 [00:13<00:15, 3624016.71tokens/s]
Shard 31:  45%|████▌     | 45100726/100000000 [00:13<00:15, 3506272.16tokens/s]
Shard 31:  46%|████▌     | 45546838/100000000 [00:13<00:14, 3688970.41tokens/s]
Shard 31:  46%|████▌     | 45926321/100000000 [00:13<00:14, 3709591.76tokens/s]
Shard 31:  46%|████▋     | 46417192/100000000 [00:13<00:13, 4037108.43tokens/s]
Shard 31:  47%|████▋     | 46830390/100000000 [00:13<00:13, 3998691.45tokens/s]
Shard 31:  47%|████▋     | 47235855/100000000 [00:13<00:14, 3645889.82tokens/s]
Shard 31:  48%|████▊     | 47609287/100000000 [00:13<00:14, 3578415.30tokens/s]
Shard 31:  48%|████▊     | 47973164/100000000 [00:14<00:15, 3321361.38tokens/s]
Shard 31:  48%|████▊     | 48312262/100000000 [00:14<00:16, 3121484.89tokens/s]
Shard 31:  49%|████▉     | 48839988/100000000 [00:14<00:13, 3681054.76tokens/s]
Shard 31:  49%|████▉     | 49220326/100000000 [00:14<00:14, 3486315.69tokens/s]
Shard 31:  50%|████▉     | 49646413/100000000 [00:14<00:13, 3691352.20tokens/s]
Shard 31:  50%|█████     | 50051359/100000000 [00:14<00:13, 3782668.31tokens/s]
Shard 31:  51%|█████     | 50590380/100000000 [00:14<00:11, 4233252.55tokens/s]
Shard 31:  51%|█████     | 51023593/100000000 [00:14<00:11, 4163594.62tokens/s]
Shard 31:  51%|█████▏    | 51445413/100000000 [00:14<00:12, 3939304.02tokens/s]
Shard 31:  52%|█████▏    | 51852642/100000000 [00:15<00:12, 3943003.77tokens/s]
Shard 31:  52%|█████▏    | 52301051/100000000 [00:15<00:11, 4093615.88tokens/s]
Shard 31:  53%|█████▎    | 52714387/100000000 [00:15<00:11, 4055860.81tokens/s]
Shard 31:  53%|█████▎    | 53262379/100000000 [00:15<00:10, 4464418.98tokens/s]
Shard 31:  54%|█████▎    | 53713416/100000000 [00:15<00:11, 4151959.52tokens/s]
Shard 31:  54%|█████▍    | 54264376/100000000 [00:15<00:10, 4526150.40tokens/s]
Shard 31:  55%|█████▍    | 54724554/100000000 [00:15<00:11, 3997497.76tokens/s]
Shard 31:  55%|█████▌    | 55359729/100000000 [00:15<00:09, 4614550.02tokens/s]
Shard 31:  56%|█████▌    | 55911198/100000000 [00:15<00:09, 4813061.57tokens/s]
Shard 31:  56%|█████▋    | 56410072/100000000 [00:16<00:09, 4449776.69tokens/s]
Shard 31:  57%|█████▋    | 57044620/100000000 [00:16<00:08, 4954783.69tokens/s]
Shard 31:  58%|█████▊    | 57556672/100000000 [00:16<00:08, 4974984.16tokens/s]
Shard 31:  58%|█████▊    | 58215637/100000000 [00:16<00:07, 5426672.71tokens/s]
Shard 31:  59%|█████▉    | 58769621/100000000 [00:16<00:08, 5091433.76tokens/s]
Shard 31:  59%|█████▉    | 59290035/100000000 [00:16<00:08, 4964130.23tokens/s]
Shard 31:  60%|█████▉    | 59797508/100000000 [00:16<00:08, 4980065.09tokens/s]
Shard 31:  60%|██████    | 60342613/100000000 [00:16<00:07, 5111215.62tokens/s]
Shard 31:  61%|██████    | 60985437/100000000 [00:16<00:07, 5424654.41tokens/s]
Shard 31:  62%|██████▏   | 61531641/100000000 [00:17<00:07, 5011151.13tokens/s]
Shard 31:  62%|██████▏   | 62040923/100000000 [00:17<00:08, 4271137.79tokens/s]
Shard 31:  62%|██████▏   | 62490510/100000000 [00:17<00:10, 3700352.27tokens/s]
Shard 31:  63%|██████▎   | 62885876/100000000 [00:17<00:10, 3709726.32tokens/s]
Shard 31:  63%|██████▎   | 63275850/100000000 [00:17<00:10, 3617750.81tokens/s]
Shard 31:  64%|██████▍   | 63784855/100000000 [00:17<00:09, 3991633.45tokens/s]
Shard 31:  64%|██████▍   | 64344595/100000000 [00:17<00:08, 4406242.95tokens/s]
Shard 31:  65%|██████▍   | 64800920/100000000 [00:17<00:09, 3771826.83tokens/s]
Shard 31:  65%|██████▌   | 65202780/100000000 [00:18<00:09, 3701835.28tokens/s]
Shard 31:  66%|██████▌   | 65768008/100000000 [00:18<00:08, 4190703.71tokens/s]
Shard 31:  66%|██████▌   | 66230207/100000000 [00:18<00:07, 4287662.26tokens/s]
Shard 31:  67%|██████▋   | 66692426/100000000 [00:18<00:07, 4264680.01tokens/s]
Shard 31:  67%|██████▋   | 67129949/100000000 [00:18<00:07, 4185685.26tokens/s]
Shard 31:  68%|██████▊   | 67571535/100000000 [00:18<00:08, 3775209.11tokens/s]
Shard 31:  68%|██████▊   | 67989601/100000000 [00:18<00:08, 3874604.63tokens/s]
Shard 31:  69%|██████▊   | 68683615/100000000 [00:18<00:06, 4687658.84tokens/s]
Shard 31:  70%|██████▉   | 69873958/100000000 [00:18<00:04, 6692717.19tokens/s]
Shard 31:  71%|███████   | 70653615/100000000 [00:19<00:04, 6977426.80tokens/s]
Shard 31:  71%|███████▏  | 71372455/100000000 [00:19<00:04, 6319097.75tokens/s]
Shard 31:  72%|███████▏  | 72029366/100000000 [00:19<00:04, 5856431.25tokens/s]
Shard 31:  73%|███████▎  | 72656224/100000000 [00:19<00:04, 5950357.44tokens/s]
Shard 31:  73%|███████▎  | 73267769/100000000 [00:19<00:04, 5590304.75tokens/s]
Shard 31:  74%|███████▍  | 73840838/100000000 [00:19<00:04, 5309799.98tokens/s]
Shard 31:  74%|███████▍  | 74451314/100000000 [00:19<00:04, 5368715.21tokens/s]
Shard 31:  75%|███████▍  | 74997387/100000000 [00:19<00:04, 5284706.28tokens/s]
Shard 31:  76%|███████▌  | 75568985/100000000 [00:19<00:04, 5386782.62tokens/s]
Shard 31:  76%|███████▌  | 76112141/100000000 [00:20<00:04, 5262643.36tokens/s]
Shard 31:  77%|███████▋  | 76641748/100000000 [00:20<00:04, 5128027.71tokens/s]
Shard 31:  77%|███████▋  | 77166777/100000000 [00:20<00:04, 4991035.99tokens/s]
Shard 31:  78%|███████▊  | 77668661/100000000 [00:20<00:04, 4794677.96tokens/s]
Shard 31:  78%|███████▊  | 78255729/100000000 [00:20<00:04, 5075970.92tokens/s]
Shard 31:  79%|███████▉  | 79173923/100000000 [00:20<00:03, 6236381.73tokens/s]
Shard 31:  80%|███████▉  | 79807234/100000000 [00:20<00:03, 5631476.87tokens/s]
Shard 31:  80%|████████  | 80387227/100000000 [00:20<00:04, 4645552.63tokens/s]
Shard 31:  81%|████████  | 80891839/100000000 [00:21<00:04, 4034128.90tokens/s]
Shard 31:  81%|████████▏ | 81330383/100000000 [00:21<00:04, 3862852.50tokens/s]
Shard 31:  82%|████████▏ | 81745197/100000000 [00:21<00:04, 3721619.25tokens/s]
Shard 31:  82%|████████▏ | 82131884/100000000 [00:21<00:05, 3537667.23tokens/s]
Shard 31:  83%|████████▎ | 83305865/100000000 [00:21<00:03, 5523731.86tokens/s]
Shard 31:  84%|████████▍ | 83913427/100000000 [00:21<00:03, 4715549.66tokens/s]
Shard 31:  84%|████████▍ | 84438079/100000000 [00:21<00:03, 4718840.60tokens/s]
Shard 31:  85%|████████▌ | 85059758/100000000 [00:22<00:02, 5085252.82tokens/s]
Shard 31:  86%|████████▌ | 85602442/100000000 [00:22<00:02, 4800470.04tokens/s]
Shard 31:  86%|████████▌ | 86108739/100000000 [00:22<00:02, 4778156.00tokens/s]
Shard 31:  87%|████████▋ | 86604021/100000000 [00:22<00:02, 4722258.05tokens/s]
Shard 31:  87%|████████▋ | 87088098/100000000 [00:22<00:02, 4470430.04tokens/s]
Shard 31:  88%|████████▊ | 87545453/100000000 [00:22<00:02, 4192094.37tokens/s]
Shard 31:  88%|████████▊ | 88071449/100000000 [00:22<00:02, 4468515.89tokens/s]
Shard 31:  89%|████████▊ | 88532383/100000000 [00:22<00:02, 4263883.86tokens/s]
Shard 31:  89%|████████▉ | 89099301/100000000 [00:22<00:02, 4631479.60tokens/s]
Shard 31:  90%|████████▉ | 89728401/100000000 [00:23<00:02, 5089959.72tokens/s]
Shard 31:  91%|█████████ | 90504891/100000000 [00:23<00:01, 5821237.14tokens/s]
Shard 31:  91%|█████████ | 91156444/100000000 [00:23<00:01, 6002243.29tokens/s]
Shard 31:  92%|█████████▏| 91765448/100000000 [00:23<00:01, 5493007.47tokens/s]
Shard 31:  92%|█████████▏| 92346445/100000000 [00:23<00:01, 5152501.26tokens/s]
Shard 31:  93%|█████████▎| 92876674/100000000 [00:23<00:01, 5153216.18tokens/s]
Shard 31:  93%|█████████▎| 93421123/100000000 [00:23<00:01, 5223361.71tokens/s]
Shard 31:  94%|█████████▍| 93960063/100000000 [00:23<00:01, 5120133.61tokens/s]
Shard 31:  95%|█████████▍| 94624494/100000000 [00:23<00:00, 5511271.44tokens/s]
Shard 31:  95%|█████████▌| 95242382/100000000 [00:24<00:00, 5683698.46tokens/s]
Shard 31:  96%|█████████▌| 95826906/100000000 [00:24<00:00, 5724764.10tokens/s]
Shard 31:  96%|█████████▋| 96403601/100000000 [00:24<00:00, 4964588.84tokens/s]
Shard 31:  97%|█████████▋| 96920221/100000000 [00:24<00:00, 4588762.38tokens/s]
Shard 31:  97%|█████████▋| 97395962/100000000 [00:24<00:00, 4434965.34tokens/s]
Shard 31:  98%|█████████▊| 97850866/100000000 [00:24<00:00, 4400512.05tokens/s]
Shard 31:  98%|█████████▊| 98301404/100000000 [00:24<00:00, 4389750.71tokens/s]
Shard 31:  99%|█████████▉| 98768578/100000000 [00:24<00:00, 4457268.10tokens/s]
Shard 31:  99%|█████████▉| 99231906/100000000 [00:24<00:00, 4427680.75tokens/s]
Shard 31: 100%|█████████▉| 99678079/100000000 [00:25<00:00, 3588840.21tokens/s]
Shard 31: 100%|█████████▉| 99999641/100000000 [00:33<00:00, 2991051.97tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000031.bin (200,001,024 bytes) in the gpt-2 format

Shard 32:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 32:   0%|          | 259474/100000000 [00:00<00:39, 2514109.70tokens/s]
Shard 32:   1%|          | 631638/100000000 [00:00<00:31, 3198866.67tokens/s]
Shard 32:   1%|          | 970240/100000000 [00:00<00:30, 3271376.10tokens/s]
Shard 32:   1%|▏         | 1318254/100000000 [00:00<00:30, 3289367.16tokens/s]
Shard 32:   2%|▏         | 1746391/100000000 [00:00<00:27, 3571947.20tokens/s]
Shard 32:   2%|▏         | 2103961/100000000 [00:00<00:27, 3569892.18tokens/s]
Shard 32:   3%|▎         | 2643040/100000000 [00:00<00:23, 4128295.22tokens/s]
Shard 32:   3%|▎         | 3056484/100000000 [00:00<00:24, 3878030.36tokens/s]
Shard 32:   3%|▎         | 3448550/100000000 [00:00<00:28, 3358279.49tokens/s]
Shard 32:   4%|▍         | 3797640/100000000 [00:01<00:29, 3291252.52tokens/s]
Shard 32:   4%|▍         | 4169980/100000000 [00:01<00:28, 3397142.04tokens/s]
Shard 32:   5%|▍         | 4518304/100000000 [00:01<00:28, 3404873.97tokens/s]
Shard 32:   5%|▍         | 4863791/100000000 [00:01<00:28, 3298336.55tokens/s]
Shard 32:   5%|▌         | 5286899/100000000 [00:01<00:26, 3535208.55tokens/s]
Shard 32:   6%|▌         | 5645064/100000000 [00:01<00:29, 3221161.17tokens/s]
Shard 32:   6%|▌         | 5975943/100000000 [00:01<00:30, 3105100.33tokens/s]
Shard 32:   6%|▋         | 6305874/100000000 [00:01<00:29, 3151859.70tokens/s]
Shard 32:   7%|▋         | 6625125/100000000 [00:01<00:30, 3112356.95tokens/s]
Shard 32:   7%|▋         | 7008227/100000000 [00:02<00:28, 3295265.03tokens/s]
Shard 32:   7%|▋         | 7491296/100000000 [00:02<00:24, 3727533.67tokens/s]
Shard 32:   8%|▊         | 7868330/100000000 [00:02<00:26, 3527569.55tokens/s]
Shard 32:   8%|▊         | 8231622/100000000 [00:02<00:28, 3167274.62tokens/s]
Shard 32:   9%|▊         | 8748515/100000000 [00:02<00:24, 3693325.95tokens/s]
Shard 32:   9%|▉         | 9136154/100000000 [00:02<00:24, 3683191.07tokens/s]
Shard 32:  10%|▉         | 9517418/100000000 [00:02<00:24, 3623382.20tokens/s]
Shard 32:  10%|▉         | 9931750/100000000 [00:02<00:23, 3756444.87tokens/s]
Shard 32:  10%|█         | 10313262/100000000 [00:02<00:25, 3512347.77tokens/s]
Shard 32:  11%|█         | 10671054/100000000 [00:03<00:25, 3449987.09tokens/s]
Shard 32:  11%|█         | 11020917/100000000 [00:03<00:26, 3388741.70tokens/s]
Shard 32:  11%|█▏        | 11406696/100000000 [00:03<00:25, 3519569.66tokens/s]
Shard 32:  12%|█▏        | 11887359/100000000 [00:03<00:22, 3880682.13tokens/s]
Shard 32:  12%|█▏        | 12280034/100000000 [00:03<00:22, 3890809.31tokens/s]
Shard 32:  13%|█▎        | 12795788/100000000 [00:03<00:20, 4231052.49tokens/s]
Shard 32:  13%|█▎        | 13325563/100000000 [00:03<00:19, 4499666.94tokens/s]
Shard 32:  14%|█▍        | 13806315/100000000 [00:03<00:18, 4585623.60tokens/s]
Shard 32:  14%|█▍        | 14270633/100000000 [00:03<00:19, 4432968.20tokens/s]
Shard 32:  15%|█▍        | 14742163/100000000 [00:04<00:18, 4514202.84tokens/s]
Shard 32:  15%|█▌        | 15315875/100000000 [00:04<00:17, 4842328.37tokens/s]
Shard 32:  16%|█▌        | 15827022/100000000 [00:04<00:17, 4882479.76tokens/s]
Shard 32:  16%|█▋        | 16321741/100000000 [00:04<00:19, 4327701.85tokens/s]
Shard 32:  17%|█▋        | 16766671/100000000 [00:04<00:19, 4174295.04tokens/s]
Shard 32:  17%|█▋        | 17194358/100000000 [00:04<00:19, 4163239.14tokens/s]
Shard 32:  18%|█▊        | 17618425/100000000 [00:04<00:22, 3740739.67tokens/s]
Shard 32:  18%|█▊        | 18003354/100000000 [00:04<00:22, 3624300.15tokens/s]
Shard 32:  18%|█▊        | 18373057/100000000 [00:04<00:25, 3260768.84tokens/s]
Shard 32:  19%|█▊        | 18708515/100000000 [00:05<00:24, 3271334.58tokens/s]
Shard 32:  19%|█▉        | 19174525/100000000 [00:05<00:22, 3632513.87tokens/s]
Shard 32:  20%|██        | 20015530/100000000 [00:05<00:16, 4918856.80tokens/s]
Shard 32:  21%|██        | 20525340/100000000 [00:05<00:18, 4389778.27tokens/s]
Shard 32:  21%|██        | 20986122/100000000 [00:05<00:18, 4226372.55tokens/s]
Shard 32:  22%|██▏       | 21632481/100000000 [00:05<00:16, 4804220.19tokens/s]
Shard 32:  22%|██▏       | 22180408/100000000 [00:05<00:15, 4958295.94tokens/s]
Shard 32:  23%|██▎       | 22691513/100000000 [00:05<00:17, 4415317.31tokens/s]
Shard 32:  23%|██▎       | 23152621/100000000 [00:06<00:19, 3964133.20tokens/s]
Shard 32:  24%|██▎       | 23573305/100000000 [00:06<00:21, 3565111.03tokens/s]
Shard 32:  24%|██▍       | 23969946/100000000 [00:06<00:20, 3645960.75tokens/s]
Shard 32:  24%|██▍       | 24349271/100000000 [00:06<00:22, 3300418.76tokens/s]
Shard 32:  25%|██▍       | 24693411/100000000 [00:06<00:22, 3320812.29tokens/s]
Shard 32:  25%|██▌       | 25462318/100000000 [00:06<00:16, 4445705.96tokens/s]
Shard 32:  26%|██▌       | 25931761/100000000 [00:06<00:18, 4068440.32tokens/s]
Shard 32:  26%|██▋       | 26374391/100000000 [00:06<00:20, 3676880.83tokens/s]
Shard 32:  27%|██▋       | 26763389/100000000 [00:07<00:22, 3284924.15tokens/s]
Shard 32:  27%|██▋       | 27110295/100000000 [00:07<00:22, 3240590.34tokens/s]
Shard 32:  27%|██▋       | 27446452/100000000 [00:07<00:22, 3161252.09tokens/s]
Shard 32:  28%|██▊       | 27770563/100000000 [00:07<00:22, 3177730.44tokens/s]
Shard 32:  28%|██▊       | 28095015/100000000 [00:07<00:23, 3042658.00tokens/s]
Shard 32:  28%|██▊       | 28403350/100000000 [00:07<00:24, 2929184.79tokens/s]
Shard 32:  29%|██▊       | 28703435/100000000 [00:07<00:24, 2928212.44tokens/s]
Shard 32:  29%|██▉       | 29051878/100000000 [00:07<00:23, 3070119.41tokens/s]
Shard 32:  29%|██▉       | 29496733/100000000 [00:07<00:20, 3446265.24tokens/s]
Shard 32:  30%|██▉       | 29881314/100000000 [00:08<00:19, 3544540.39tokens/s]
Shard 32:  30%|███       | 30240426/100000000 [00:08<00:19, 3495298.73tokens/s]
Shard 32:  31%|███       | 30640562/100000000 [00:08<00:19, 3619725.38tokens/s]
Shard 32:  31%|███       | 31042875/100000000 [00:08<00:18, 3733957.37tokens/s]
Shard 32:  31%|███▏      | 31440005/100000000 [00:08<00:18, 3788869.37tokens/s]
Shard 32:  32%|███▏      | 31821323/100000000 [00:08<00:19, 3500690.52tokens/s]
Shard 32:  32%|███▏      | 32259945/100000000 [00:08<00:18, 3738285.47tokens/s]
Shard 32:  33%|███▎      | 32639306/100000000 [00:08<00:18, 3597653.31tokens/s]
Shard 32:  33%|███▎      | 33003318/100000000 [00:08<00:19, 3508920.63tokens/s]
Shard 32:  33%|███▎      | 33357511/100000000 [00:09<00:19, 3492051.76tokens/s]
Shard 32:  34%|███▎      | 33708906/100000000 [00:09<00:19, 3452863.96tokens/s]
Shard 32:  34%|███▍      | 34065993/100000000 [00:09<00:19, 3466933.28tokens/s]
Shard 32:  35%|███▍      | 34604576/100000000 [00:09<00:16, 3984558.53tokens/s]
Shard 32:  35%|███▌      | 35005612/100000000 [00:09<00:17, 3702492.37tokens/s]
Shard 32:  35%|███▌      | 35384748/100000000 [00:09<00:17, 3637215.46tokens/s]
Shard 32:  36%|███▌      | 35804493/100000000 [00:09<00:16, 3788870.11tokens/s]
Shard 32:  36%|███▌      | 36186610/100000000 [00:09<00:17, 3620938.00tokens/s]
Shard 32:  37%|███▋      | 36730012/100000000 [00:09<00:15, 4114681.02tokens/s]
Shard 32:  37%|███▋      | 37310071/100000000 [00:10<00:13, 4562092.98tokens/s]
Shard 32:  38%|███▊      | 37772543/100000000 [00:10<00:13, 4537542.00tokens/s]
Shard 32:  38%|███▊      | 38233053/100000000 [00:10<00:13, 4553983.70tokens/s]
Shard 32:  39%|███▊      | 38699617/100000000 [00:10<00:13, 4447676.82tokens/s]
Shard 32:  39%|███▉      | 39188017/100000000 [00:10<00:13, 4571780.54tokens/s]
Shard 32:  40%|███▉      | 39647643/100000000 [00:10<00:13, 4533310.42tokens/s]
Shard 32:  40%|████      | 40102898/100000000 [00:10<00:15, 3878681.74tokens/s]
Shard 32:  41%|████      | 40508014/100000000 [00:10<00:15, 3919466.29tokens/s]
Shard 32:  41%|████      | 41104379/100000000 [00:10<00:13, 4438676.98tokens/s]
Shard 32:  42%|████▏     | 41771116/100000000 [00:10<00:11, 5058304.06tokens/s]
Shard 32:  42%|████▏     | 42290451/100000000 [00:11<00:11, 4922445.15tokens/s]
Shard 32:  43%|████▎     | 42793168/100000000 [00:11<00:12, 4560989.65tokens/s]
Shard 32:  43%|████▎     | 43260698/100000000 [00:11<00:12, 4450614.28tokens/s]
Shard 32:  44%|████▍     | 43841875/100000000 [00:11<00:11, 4810343.87tokens/s]
Shard 32:  44%|████▍     | 44392367/100000000 [00:11<00:11, 4991471.91tokens/s]
Shard 32:  45%|████▍     | 44898975/100000000 [00:11<00:12, 4446265.69tokens/s]
Shard 32:  45%|████▌     | 45361383/100000000 [00:11<00:12, 4408477.09tokens/s]
Shard 32:  46%|████▌     | 45812846/100000000 [00:11<00:12, 4263290.87tokens/s]
Shard 32:  46%|████▌     | 46247285/100000000 [00:12<00:12, 4144859.40tokens/s]
Shard 32:  47%|████▋     | 46667184/100000000 [00:12<00:15, 3538129.38tokens/s]
Shard 32:  47%|████▋     | 47051751/100000000 [00:12<00:14, 3610516.97tokens/s]
Shard 32:  48%|████▊     | 47542401/100000000 [00:12<00:13, 3907420.16tokens/s]
Shard 32:  48%|████▊     | 48063963/100000000 [00:12<00:12, 4259058.93tokens/s]
Shard 32:  49%|████▊     | 48502341/100000000 [00:12<00:12, 4062993.45tokens/s]
Shard 32:  49%|████▉     | 49028482/100000000 [00:12<00:11, 4355864.07tokens/s]
Shard 32:  49%|████▉     | 49475256/100000000 [00:12<00:11, 4377451.95tokens/s]
Shard 32:  51%|█████     | 50500323/100000000 [00:12<00:08, 6048652.28tokens/s]
Shard 32:  51%|█████     | 51120310/100000000 [00:13<00:08, 5922358.90tokens/s]
Shard 32:  52%|█████▏    | 51758781/100000000 [00:13<00:08, 5676161.06tokens/s]
Shard 32:  52%|█████▏    | 52336330/100000000 [00:13<00:09, 4769845.49tokens/s]
Shard 32:  53%|█████▎    | 53068849/100000000 [00:13<00:08, 5381314.70tokens/s]
Shard 32:  54%|█████▎    | 53638596/100000000 [00:13<00:09, 4887480.96tokens/s]
Shard 32:  54%|█████▍    | 54155651/100000000 [00:13<00:09, 4642644.50tokens/s]
Shard 32:  55%|█████▍    | 54639424/100000000 [00:13<00:11, 3952441.38tokens/s]
Shard 32:  55%|█████▌    | 55063294/100000000 [00:13<00:11, 3994101.85tokens/s]
Shard 32:  56%|█████▌    | 55513833/100000000 [00:14<00:10, 4120231.46tokens/s]
Shard 32:  56%|█████▌    | 55942640/100000000 [00:14<00:11, 3924281.64tokens/s]
Shard 32:  56%|█████▋    | 56355387/100000000 [00:14<00:11, 3691040.83tokens/s]
Shard 32:  57%|█████▋    | 56733765/100000000 [00:14<00:12, 3432386.64tokens/s]
Shard 32:  57%|█████▋    | 57098699/100000000 [00:14<00:12, 3480715.54tokens/s]
Shard 32:  57%|█████▋    | 57470635/100000000 [00:14<00:12, 3525430.75tokens/s]
Shard 32:  58%|█████▊    | 57857026/100000000 [00:14<00:11, 3581236.30tokens/s]
Shard 32:  58%|█████▊    | 58220181/100000000 [00:14<00:11, 3591007.88tokens/s]
Shard 32:  59%|█████▊    | 58592086/100000000 [00:14<00:11, 3587771.61tokens/s]
Shard 32:  59%|█████▉    | 59029763/100000000 [00:15<00:10, 3808523.41tokens/s]
Shard 32:  60%|█████▉    | 59532719/100000000 [00:15<00:09, 4148901.91tokens/s]
Shard 32:  60%|██████    | 60040912/100000000 [00:15<00:09, 4393741.48tokens/s]
Shard 32:  60%|██████    | 60482413/100000000 [00:15<00:09, 4059524.55tokens/s]
Shard 32:  61%|██████    | 60894420/100000000 [00:15<00:10, 3805580.98tokens/s]
Shard 32:  61%|██████▏   | 61281490/100000000 [00:15<00:11, 3427928.44tokens/s]
Shard 32:  62%|██████▏   | 61639781/100000000 [00:15<00:11, 3307668.03tokens/s]
Shard 32:  62%|██████▏   | 62060342/100000000 [00:15<00:10, 3529784.41tokens/s]
Shard 32:  62%|██████▏   | 62422661/100000000 [00:16<00:11, 3147231.76tokens/s]
Shard 32:  63%|██████▎   | 62748618/100000000 [00:16<00:11, 3159948.15tokens/s]
Shard 32:  63%|██████▎   | 63217302/100000000 [00:16<00:10, 3560239.64tokens/s]
Shard 32:  64%|██████▍   | 63754281/100000000 [00:16<00:08, 4028462.40tokens/s]
Shard 32:  64%|██████▍   | 64350096/100000000 [00:16<00:07, 4539337.27tokens/s]
Shard 32:  65%|██████▍   | 64814242/100000000 [00:16<00:08, 4319505.72tokens/s]
Shard 32:  65%|██████▌   | 65255421/100000000 [00:16<00:09, 3814533.69tokens/s]
Shard 32:  66%|██████▌   | 65652415/100000000 [00:16<00:10, 3269880.30tokens/s]
Shard 32:  66%|██████▌   | 66000971/100000000 [00:17<00:11, 2976712.82tokens/s]
Shard 32:  66%|██████▋   | 66321766/100000000 [00:17<00:11, 3022346.48tokens/s]
Shard 32:  67%|██████▋   | 66638033/100000000 [00:17<00:10, 3056800.40tokens/s]
Shard 32:  67%|██████▋   | 66953518/100000000 [00:17<00:11, 2995237.02tokens/s]
Shard 32:  67%|██████▋   | 67302385/100000000 [00:17<00:10, 3121855.10tokens/s]
Shard 32:  68%|██████▊   | 67635628/100000000 [00:17<00:10, 3161290.85tokens/s]
Shard 32:  68%|██████▊   | 67956352/100000000 [00:17<00:10, 2971512.15tokens/s]
Shard 32:  68%|██████▊   | 68296874/100000000 [00:17<00:10, 3084928.45tokens/s]
Shard 32:  69%|██████▊   | 68611256/100000000 [00:17<00:10, 2995742.70tokens/s]
Shard 32:  69%|██████▉   | 68914397/100000000 [00:18<00:10, 2892635.45tokens/s]
Shard 32:  69%|██████▉   | 69206240/100000000 [00:18<00:10, 2810287.51tokens/s]
Shard 32:  70%|██████▉   | 69626091/100000000 [00:18<00:09, 3194543.03tokens/s]
Shard 32:  70%|██████▉   | 69978408/100000000 [00:18<00:09, 3237212.53tokens/s]
Shard 32:  70%|███████   | 70433658/100000000 [00:18<00:08, 3590761.66tokens/s]
Shard 32:  71%|███████   | 70796297/100000000 [00:18<00:08, 3539311.01tokens/s]
Shard 32:  71%|███████   | 71212505/100000000 [00:18<00:07, 3687865.08tokens/s]
Shard 32:  72%|███████▏  | 71688609/100000000 [00:18<00:07, 3987286.21tokens/s]
Shard 32:  72%|███████▏  | 72145040/100000000 [00:18<00:06, 4120120.71tokens/s]
Shard 32:  73%|███████▎  | 72569341/100000000 [00:18<00:06, 4138393.50tokens/s]
Shard 32:  73%|███████▎  | 73170704/100000000 [00:19<00:05, 4683165.29tokens/s]
Shard 32:  74%|███████▎  | 73671159/100000000 [00:19<00:05, 4729069.14tokens/s]
Shard 32:  74%|███████▍  | 74146606/100000000 [00:19<00:05, 4464308.82tokens/s]
Shard 32:  75%|███████▍  | 74710330/100000000 [00:19<00:05, 4764924.75tokens/s]
Shard 32:  75%|███████▌  | 75195569/100000000 [00:19<00:05, 4243505.71tokens/s]
Shard 32:  76%|███████▌  | 75632708/100000000 [00:19<00:06, 3991656.10tokens/s]
Shard 32:  76%|███████▌  | 76043550/100000000 [00:19<00:06, 3967959.66tokens/s]
Shard 32:  76%|███████▋  | 76481032/100000000 [00:19<00:05, 4069362.22tokens/s]
Shard 32:  77%|███████▋  | 76968801/100000000 [00:19<00:05, 4285338.19tokens/s]
Shard 32:  77%|███████▋  | 77403095/100000000 [00:20<00:05, 3961683.10tokens/s]
Shard 32:  78%|███████▊  | 77813441/100000000 [00:20<00:05, 3982705.91tokens/s]
Shard 32:  78%|███████▊  | 78218356/100000000 [00:20<00:06, 3524349.05tokens/s]
Shard 32:  79%|███████▊  | 78664172/100000000 [00:20<00:05, 3747721.89tokens/s]
Shard 32:  79%|███████▉  | 79052374/100000000 [00:20<00:06, 3472824.96tokens/s]
Shard 32:  79%|███████▉  | 79411092/100000000 [00:20<00:05, 3486660.81tokens/s]
Shard 32:  80%|███████▉  | 79976863/100000000 [00:20<00:04, 4070329.64tokens/s]
Shard 32:  80%|████████  | 80395383/100000000 [00:20<00:04, 3995890.50tokens/s]
Shard 32:  81%|████████  | 80803555/100000000 [00:21<00:05, 3754925.56tokens/s]
Shard 32:  81%|████████  | 81186751/100000000 [00:21<00:05, 3620163.04tokens/s]
Shard 32:  82%|████████▏ | 81554266/100000000 [00:21<00:05, 3423551.50tokens/s]
Shard 32:  82%|████████▏ | 81901778/100000000 [00:21<00:05, 3188367.00tokens/s]
Shard 32:  82%|████████▏ | 82225876/100000000 [00:21<00:05, 3100902.49tokens/s]
Shard 32:  83%|████████▎ | 82544914/100000000 [00:21<00:05, 3122754.99tokens/s]
Shard 32:  83%|████████▎ | 82859698/100000000 [00:21<00:05, 2958021.50tokens/s]
Shard 32:  83%|████████▎ | 83354309/100000000 [00:21<00:04, 3494071.42tokens/s]
Shard 32:  84%|████████▍ | 83761693/100000000 [00:21<00:04, 3651135.54tokens/s]
Shard 32:  84%|████████▍ | 84161377/100000000 [00:22<00:04, 3736063.33tokens/s]
Shard 32:  85%|████████▍ | 84543440/100000000 [00:22<00:04, 3233968.65tokens/s]
Shard 32:  85%|████████▍ | 84886205/100000000 [00:22<00:04, 3273881.04tokens/s]
Shard 32:  85%|████████▌ | 85225473/100000000 [00:22<00:04, 2965067.61tokens/s]
Shard 32:  86%|████████▌ | 85534610/100000000 [00:22<00:06, 2410247.68tokens/s]
Shard 32:  86%|████████▌ | 85800742/100000000 [00:22<00:05, 2375948.23tokens/s]
Shard 32:  86%|████████▌ | 86053534/100000000 [00:22<00:06, 2212627.08tokens/s]
Shard 32:  86%|████████▋ | 86317162/100000000 [00:22<00:05, 2309749.38tokens/s]
Shard 32:  87%|████████▋ | 86581320/100000000 [00:23<00:05, 2388138.51tokens/s]
Shard 32:  87%|████████▋ | 86884385/100000000 [00:23<00:05, 2551917.89tokens/s]
Shard 32:  87%|████████▋ | 87162127/100000000 [00:23<00:04, 2579791.38tokens/s]
Shard 32:  88%|████████▊ | 87516936/100000000 [00:23<00:04, 2844601.81tokens/s]
Shard 32:  88%|████████▊ | 88066684/100000000 [00:23<00:03, 3588241.76tokens/s]
Shard 32:  89%|████████▊ | 88552550/100000000 [00:23<00:02, 3949068.73tokens/s]
Shard 32:  89%|████████▉ | 88955143/100000000 [00:23<00:03, 3465988.00tokens/s]
Shard 32:  90%|████████▉ | 89676864/100000000 [00:23<00:02, 4434668.04tokens/s]
Shard 32:  90%|█████████ | 90143654/100000000 [00:23<00:02, 3921019.31tokens/s]
Shard 32:  91%|█████████ | 90562540/100000000 [00:24<00:02, 3913485.98tokens/s]
Shard 32:  91%|█████████ | 90971121/100000000 [00:24<00:02, 3606888.08tokens/s]
Shard 32:  91%|█████████▏| 91396645/100000000 [00:24<00:02, 3733425.96tokens/s]
Shard 32:  92%|█████████▏| 91783543/100000000 [00:24<00:02, 3305960.38tokens/s]
Shard 32:  92%|█████████▏| 92130874/100000000 [00:24<00:02, 3075759.20tokens/s]
Shard 32:  92%|█████████▏| 92451615/100000000 [00:24<00:02, 3029427.56tokens/s]
Shard 32:  93%|█████████▎| 92890949/100000000 [00:24<00:02, 3349182.76tokens/s]
Shard 32:  93%|█████████▎| 93237389/100000000 [00:24<00:02, 2823948.91tokens/s]
Shard 32:  94%|█████████▎| 93595964/100000000 [00:25<00:02, 2965679.35tokens/s]
Shard 32:  94%|█████████▍| 93908978/100000000 [00:25<00:02, 2894706.40tokens/s]
Shard 32:  94%|█████████▍| 94209063/100000000 [00:25<00:02, 2794193.37tokens/s]
Shard 32:  95%|█████████▍| 94573756/100000000 [00:25<00:01, 3002166.03tokens/s]
Shard 32:  95%|█████████▍| 94882009/100000000 [00:25<00:01, 2782133.52tokens/s]
Shard 32:  95%|█████████▌| 95283446/100000000 [00:25<00:01, 3100158.12tokens/s]
Shard 32:  96%|█████████▌| 95627607/100000000 [00:25<00:01, 3180224.54tokens/s]
Shard 32:  96%|█████████▌| 95955450/100000000 [00:25<00:01, 3198780.18tokens/s]
Shard 32:  96%|█████████▋| 96317850/100000000 [00:25<00:01, 3315677.67tokens/s]
Shard 32:  97%|█████████▋| 97237371/100000000 [00:26<00:00, 5006654.82tokens/s]
Shard 32:  98%|█████████▊| 97749702/100000000 [00:26<00:00, 4962390.94tokens/s]
Shard 32:  98%|█████████▊| 98252396/100000000 [00:26<00:00, 4596146.63tokens/s]
Shard 32:  99%|█████████▊| 98721539/100000000 [00:26<00:00, 4272561.52tokens/s]
Shard 32:  99%|█████████▉| 99158729/100000000 [00:26<00:00, 4191351.02tokens/s]
Shard 32: 100%|█████████▉| 99585128/100000000 [00:26<00:00, 3895999.84tokens/s]
Shard 32: 100%|█████████▉| 99981556/100000000 [00:26<00:00, 3125776.07tokens/s]
Shard 32: 100%|█████████▉| 99999107/100000000 [00:35<00:00, 2814170.41tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000032.bin (200,001,024 bytes) in the gpt-2 format

Shard 33:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 33:   0%|          | 217116/100000000 [00:00<00:46, 2133875.34tokens/s]
Shard 33:   1%|          | 575423/100000000 [00:00<00:33, 2959428.32tokens/s]
Shard 33:   1%|          | 1193656/100000000 [00:00<00:22, 4328274.41tokens/s]
Shard 33:   2%|▏         | 1629909/100000000 [00:00<00:24, 4002335.57tokens/s]
Shard 33:   2%|▏         | 2032512/100000000 [00:00<00:24, 3937908.60tokens/s]
Shard 33:   2%|▏         | 2427251/100000000 [00:00<00:32, 3029812.16tokens/s]
Shard 33:   3%|▎         | 2757906/100000000 [00:00<00:33, 2939361.63tokens/s]
Shard 33:   3%|▎         | 3070839/100000000 [00:00<00:35, 2731952.74tokens/s]
Shard 33:   3%|▎         | 3449796/100000000 [00:01<00:32, 2997426.24tokens/s]
Shard 33:   4%|▍         | 3764076/100000000 [00:01<00:32, 2925786.27tokens/s]
Shard 33:   4%|▍         | 4073979/100000000 [00:01<00:32, 2970432.94tokens/s]
Shard 33:   4%|▍         | 4447162/100000000 [00:01<00:30, 3180474.31tokens/s]
Shard 33:   5%|▍         | 4904524/100000000 [00:01<00:26, 3568916.55tokens/s]
Shard 33:   5%|▌         | 5271968/100000000 [00:01<00:27, 3392952.52tokens/s]
Shard 33:   6%|▌         | 5617907/100000000 [00:01<00:29, 3152508.48tokens/s]
Shard 33:   6%|▌         | 5994562/100000000 [00:01<00:28, 3307712.50tokens/s]
Shard 33:   6%|▋         | 6334640/100000000 [00:01<00:29, 3175952.61tokens/s]
Shard 33:   7%|▋         | 6659911/100000000 [00:02<00:33, 2777155.47tokens/s]
Shard 33:   7%|▋         | 6948897/100000000 [00:02<00:34, 2688952.97tokens/s]
Shard 33:   7%|▋         | 7325774/100000000 [00:02<00:31, 2929157.31tokens/s]
Shard 33:   8%|▊         | 7626943/100000000 [00:02<00:31, 2932953.65tokens/s]
Shard 33:   8%|▊         | 7970253/100000000 [00:02<00:30, 3060060.40tokens/s]
Shard 33:   8%|▊         | 8401576/100000000 [00:02<00:26, 3407206.82tokens/s]
Shard 33:   9%|▉         | 9043305/100000000 [00:02<00:21, 4255838.97tokens/s]
Shard 33:   9%|▉         | 9476751/100000000 [00:02<00:21, 4184349.27tokens/s]
Shard 33:  10%|█         | 10161922/100000000 [00:02<00:18, 4934461.98tokens/s]
Shard 33:  11%|█         | 10662580/100000000 [00:03<00:20, 4342356.90tokens/s]
Shard 33:  11%|█         | 11114533/100000000 [00:03<00:22, 3976449.78tokens/s]
Shard 33:  12%|█▏        | 11571028/100000000 [00:03<00:21, 4122290.85tokens/s]
Shard 33:  15%|█▍        | 14584369/100000000 [00:03<00:07, 11036786.69tokens/s]
Shard 33:  16%|█▌        | 15769720/100000000 [00:03<00:10, 7690614.29tokens/s] 
Shard 33:  17%|█▋        | 16732279/100000000 [00:04<00:14, 5762417.94tokens/s]
Shard 33:  18%|█▊        | 17504720/100000000 [00:04<00:14, 5667843.08tokens/s]
Shard 33:  18%|█▊        | 18206189/100000000 [00:04<00:15, 5239443.38tokens/s]
Shard 33:  19%|█▉        | 18934075/100000000 [00:04<00:14, 5604231.61tokens/s]
Shard 33:  20%|█▉        | 19577641/100000000 [00:04<00:15, 5114119.37tokens/s]
Shard 33:  20%|██        | 20149016/100000000 [00:04<00:17, 4588581.08tokens/s]
Shard 33:  21%|██        | 20652032/100000000 [00:04<00:17, 4624755.89tokens/s]
Shard 33:  21%|██        | 21230086/100000000 [00:04<00:16, 4882631.44tokens/s]
Shard 33:  22%|██▏       | 21748361/100000000 [00:05<00:16, 4803836.31tokens/s]
Shard 33:  22%|██▏       | 22250603/100000000 [00:05<00:16, 4593322.48tokens/s]
Shard 33:  23%|██▎       | 22724500/100000000 [00:05<00:17, 4458806.15tokens/s]
Shard 33:  23%|██▎       | 23179354/100000000 [00:05<00:18, 4110308.60tokens/s]
Shard 33:  24%|██▎       | 23657146/100000000 [00:05<00:19, 3898664.73tokens/s]
Shard 33:  24%|██▍       | 24093953/100000000 [00:05<00:18, 4013785.17tokens/s]
Shard 33:  25%|██▍       | 24502995/100000000 [00:05<00:21, 3584274.40tokens/s]
Shard 33:  25%|██▍       | 24873594/100000000 [00:05<00:21, 3543795.63tokens/s]
Shard 33:  25%|██▌       | 25240233/100000000 [00:06<00:22, 3339375.09tokens/s]
Shard 33:  26%|██▌       | 25580727/100000000 [00:06<00:24, 3041914.88tokens/s]
Shard 33:  26%|██▌       | 25891883/100000000 [00:06<00:24, 3034334.58tokens/s]
Shard 33:  26%|██▌       | 26246979/100000000 [00:06<00:23, 3158797.48tokens/s]
Shard 33:  27%|██▋       | 26697779/100000000 [00:06<00:20, 3515319.55tokens/s]
Shard 33:  27%|██▋       | 27059079/100000000 [00:06<00:21, 3344681.01tokens/s]
Shard 33:  27%|██▋       | 27400691/100000000 [00:06<00:22, 3266896.02tokens/s]
Shard 33:  28%|██▊       | 27732173/100000000 [00:06<00:22, 3244254.07tokens/s]
Shard 33:  28%|██▊       | 28070664/100000000 [00:06<00:21, 3271001.26tokens/s]
Shard 33:  28%|██▊       | 28400299/100000000 [00:07<00:23, 3043782.65tokens/s]
Shard 33:  29%|██▉       | 28785696/100000000 [00:07<00:21, 3265320.70tokens/s]
Shard 33:  29%|██▉       | 29136219/100000000 [00:07<00:21, 3297237.37tokens/s]
Shard 33:  29%|██▉       | 29484600/100000000 [00:07<00:21, 3346290.65tokens/s]
Shard 33:  30%|██▉       | 29829536/100000000 [00:07<00:20, 3354047.62tokens/s]
Shard 33:  30%|███       | 30166897/100000000 [00:07<00:21, 3250970.69tokens/s]
Shard 33:  30%|███       | 30497946/100000000 [00:07<00:21, 3219920.29tokens/s]
Shard 33:  31%|███       | 30821452/100000000 [00:07<00:22, 3029975.19tokens/s]
Shard 33:  31%|███       | 31127518/100000000 [00:07<00:23, 2990495.44tokens/s]
Shard 33:  31%|███▏      | 31428441/100000000 [00:08<00:23, 2917741.15tokens/s]
Shard 33:  32%|███▏      | 31766467/100000000 [00:08<00:22, 3044008.91tokens/s]
Shard 33:  32%|███▏      | 32072547/100000000 [00:08<00:22, 2974480.69tokens/s]
Shard 33:  32%|███▏      | 32371340/100000000 [00:08<00:23, 2877748.41tokens/s]
Shard 33:  33%|███▎      | 32660375/100000000 [00:08<00:23, 2832933.71tokens/s]
Shard 33:  33%|███▎      | 33110904/100000000 [00:08<00:20, 3302956.34tokens/s]
Shard 33:  34%|███▎      | 33575549/100000000 [00:08<00:18, 3680901.45tokens/s]
Shard 33:  34%|███▍      | 33952149/100000000 [00:08<00:18, 3613590.28tokens/s]
Shard 33:  34%|███▍      | 34316161/100000000 [00:08<00:18, 3463597.76tokens/s]
Shard 33:  35%|███▍      | 34665526/100000000 [00:09<00:20, 3146274.55tokens/s]
Shard 33:  35%|███▍      | 34987994/100000000 [00:09<00:22, 2879634.08tokens/s]
Shard 33:  35%|███▌      | 35286484/100000000 [00:09<00:22, 2860428.96tokens/s]
Shard 33:  36%|███▌      | 35592614/100000000 [00:09<00:22, 2904662.61tokens/s]
Shard 33:  36%|███▌      | 35889420/100000000 [00:09<00:22, 2841706.56tokens/s]
Shard 33:  36%|███▋      | 36293923/100000000 [00:09<00:20, 3149900.52tokens/s]
Shard 33:  37%|███▋      | 36613356/100000000 [00:09<00:20, 3095299.30tokens/s]
Shard 33:  37%|███▋      | 37010507/100000000 [00:09<00:18, 3339934.48tokens/s]
Shard 33:  37%|███▋      | 37348018/100000000 [00:09<00:20, 3092084.97tokens/s]
Shard 33:  38%|███▊      | 37664544/100000000 [00:10<00:20, 3041729.60tokens/s]
Shard 33:  38%|███▊      | 37972646/100000000 [00:10<00:20, 3050140.14tokens/s]
Shard 33:  38%|███▊      | 38322360/100000000 [00:10<00:19, 3176846.53tokens/s]
Shard 33:  39%|███▊      | 38704512/100000000 [00:10<00:18, 3361964.93tokens/s]
Shard 33:  39%|███▉      | 39064782/100000000 [00:10<00:17, 3426593.52tokens/s]
Shard 33:  39%|███▉      | 39409604/100000000 [00:10<00:18, 3191068.48tokens/s]
Shard 33:  40%|███▉      | 39875632/100000000 [00:10<00:16, 3601449.35tokens/s]
Shard 33:  40%|████      | 40241834/100000000 [00:10<00:18, 3199043.46tokens/s]
Shard 33:  41%|████      | 40573803/100000000 [00:10<00:19, 3021011.06tokens/s]
Shard 33:  41%|████      | 40900921/100000000 [00:11<00:19, 3081875.94tokens/s]
Shard 33:  41%|████      | 41217755/100000000 [00:11<00:20, 2929609.71tokens/s]
Shard 33:  42%|████▏     | 41516988/100000000 [00:11<00:21, 2777105.34tokens/s]
Shard 33:  42%|████▏     | 41880310/100000000 [00:11<00:19, 3002390.57tokens/s]
Shard 33:  42%|████▏     | 42242227/100000000 [00:11<00:18, 3171687.54tokens/s]
Shard 33:  43%|████▎     | 42619424/100000000 [00:11<00:17, 3334854.86tokens/s]
Shard 33:  43%|████▎     | 42958244/100000000 [00:11<00:17, 3215496.58tokens/s]
Shard 33:  43%|████▎     | 43388074/100000000 [00:11<00:16, 3470304.96tokens/s]
Shard 33:  44%|████▎     | 43738632/100000000 [00:11<00:16, 3460618.33tokens/s]
Shard 33:  44%|████▍     | 44116480/100000000 [00:12<00:15, 3518850.27tokens/s]
Shard 33:  44%|████▍     | 44470193/100000000 [00:12<00:16, 3296630.37tokens/s]
Shard 33:  45%|████▍     | 44822897/100000000 [00:12<00:16, 3360822.70tokens/s]
Shard 33:  45%|████▌     | 45212732/100000000 [00:12<00:15, 3500276.50tokens/s]
Shard 33:  46%|████▌     | 45597388/100000000 [00:12<00:15, 3592523.47tokens/s]
Shard 33:  46%|████▌     | 45975495/100000000 [00:12<00:14, 3613510.77tokens/s]
Shard 33:  46%|████▋     | 46396785/100000000 [00:12<00:14, 3780212.71tokens/s]
Shard 33:  47%|████▋     | 46843098/100000000 [00:12<00:13, 3950520.09tokens/s]
Shard 33:  47%|████▋     | 47241428/100000000 [00:12<00:14, 3624660.12tokens/s]
Shard 33:  48%|████▊     | 47621683/100000000 [00:12<00:14, 3654671.49tokens/s]
Shard 33:  48%|████▊     | 47991354/100000000 [00:13<00:15, 3420837.51tokens/s]
Shard 33:  48%|████▊     | 48340651/100000000 [00:13<00:15, 3415308.90tokens/s]
Shard 33:  49%|████▉     | 48918416/100000000 [00:13<00:12, 4070941.09tokens/s]
Shard 33:  49%|████▉     | 49359526/100000000 [00:13<00:12, 4147017.38tokens/s]
Shard 33:  50%|████▉     | 49840033/100000000 [00:13<00:11, 4327553.82tokens/s]
Shard 33:  50%|█████     | 50277563/100000000 [00:13<00:11, 4168346.49tokens/s]
Shard 33:  51%|█████     | 51028841/100000000 [00:13<00:09, 5112840.04tokens/s]
Shard 33:  52%|█████▏    | 51547589/100000000 [00:13<00:10, 4539646.81tokens/s]
Shard 33:  52%|█████▏    | 52017843/100000000 [00:13<00:10, 4410416.53tokens/s]
Shard 33:  53%|█████▎    | 52534604/100000000 [00:14<00:10, 4605537.59tokens/s]
Shard 33:  53%|█████▎    | 53006119/100000000 [00:14<00:11, 4207571.95tokens/s]
Shard 33:  53%|█████▎    | 53439735/100000000 [00:14<00:11, 4053257.07tokens/s]
Shard 33:  54%|█████▍    | 53853903/100000000 [00:14<00:11, 4056330.94tokens/s]
Shard 33:  54%|█████▍    | 54269911/100000000 [00:14<00:11, 4071331.36tokens/s]
Shard 33:  55%|█████▍    | 54795020/100000000 [00:14<00:10, 4402197.90tokens/s]
Shard 33:  55%|█████▌    | 55292212/100000000 [00:14<00:09, 4561256.15tokens/s]
Shard 33:  56%|█████▌    | 55760712/100000000 [00:14<00:09, 4596197.21tokens/s]
Shard 33:  56%|█████▌    | 56224536/100000000 [00:14<00:10, 4328881.21tokens/s]
Shard 33:  57%|█████▋    | 56663126/100000000 [00:15<00:10, 4164986.69tokens/s]
Shard 33:  57%|█████▋    | 57083869/100000000 [00:15<00:10, 4057706.97tokens/s]
Shard 33:  58%|█████▊    | 57505476/100000000 [00:15<00:10, 4098449.84tokens/s]
Shard 33:  58%|█████▊    | 57935166/100000000 [00:15<00:10, 4121515.16tokens/s]
Shard 33:  59%|█████▊    | 58535860/100000000 [00:15<00:08, 4623385.79tokens/s]
Shard 33:  59%|█████▉    | 59036380/100000000 [00:15<00:08, 4730642.95tokens/s]
Shard 33:  60%|█████▉    | 59511422/100000000 [00:15<00:09, 4368253.48tokens/s]
Shard 33:  60%|█████▉    | 59954937/100000000 [00:15<00:09, 4014798.25tokens/s]
Shard 33:  60%|██████    | 60365392/100000000 [00:15<00:10, 3772605.53tokens/s]
Shard 33:  61%|██████    | 60750745/100000000 [00:16<00:10, 3605204.27tokens/s]
Shard 33:  61%|██████▏   | 61292824/100000000 [00:16<00:09, 4051956.90tokens/s]
Shard 33:  62%|██████▏   | 61851483/100000000 [00:16<00:08, 4453185.82tokens/s]
Shard 33:  62%|██████▏   | 62308308/100000000 [00:16<00:08, 4324772.13tokens/s]
Shard 33:  63%|██████▎   | 62748176/100000000 [00:16<00:08, 4229390.89tokens/s]
Shard 33:  63%|██████▎   | 63176683/100000000 [00:16<00:08, 4200533.46tokens/s]
Shard 33:  64%|██████▎   | 63600077/100000000 [00:16<00:10, 3579632.84tokens/s]
Shard 33:  64%|██████▍   | 63974902/100000000 [00:16<00:10, 3455891.41tokens/s]
Shard 33:  64%|██████▍   | 64342217/100000000 [00:17<00:10, 3499253.57tokens/s]
Shard 33:  65%|██████▍   | 64799747/100000000 [00:17<00:09, 3786552.09tokens/s]
Shard 33:  65%|██████▌   | 65189580/100000000 [00:17<00:10, 3292820.98tokens/s]
Shard 33:  66%|██████▌   | 65601575/100000000 [00:17<00:09, 3492248.28tokens/s]
Shard 33:  66%|██████▌   | 65966806/100000000 [00:17<00:10, 3328407.28tokens/s]
Shard 33:  66%|██████▋   | 66310699/100000000 [00:17<00:11, 2923782.61tokens/s]
Shard 33:  67%|██████▋   | 66617710/100000000 [00:17<00:11, 2932535.31tokens/s]
Shard 33:  67%|██████▋   | 66922056/100000000 [00:17<00:11, 2881131.62tokens/s]
Shard 33:  67%|██████▋   | 67349951/100000000 [00:17<00:10, 3241399.60tokens/s]
Shard 33:  68%|██████▊   | 67684406/100000000 [00:18<00:10, 2961907.82tokens/s]
Shard 33:  68%|██████▊   | 68205600/100000000 [00:18<00:08, 3550169.63tokens/s]
Shard 33:  69%|██████▊   | 68576440/100000000 [00:18<00:09, 3424061.73tokens/s]
Shard 33:  69%|██████▉   | 69059554/100000000 [00:18<00:08, 3804302.45tokens/s]
Shard 33:  70%|██████▉   | 69913853/100000000 [00:18<00:05, 5112753.05tokens/s]
Shard 33:  70%|███████   | 70443203/100000000 [00:18<00:06, 4837834.97tokens/s]
Shard 33:  71%|███████   | 70942507/100000000 [00:18<00:06, 4691720.68tokens/s]
Shard 33:  71%|███████▏  | 71432390/100000000 [00:18<00:06, 4743634.32tokens/s]
Shard 33:  72%|███████▏  | 71989303/100000000 [00:18<00:05, 4972629.79tokens/s]
Shard 33:  73%|███████▎  | 72559391/100000000 [00:19<00:05, 5180847.14tokens/s]
Shard 33:  73%|███████▎  | 73083647/100000000 [00:19<00:05, 5093002.18tokens/s]
Shard 33:  74%|███████▎  | 73713120/100000000 [00:19<00:04, 5394681.77tokens/s]
Shard 33:  74%|███████▍  | 74262645/100000000 [00:19<00:04, 5409578.97tokens/s]
Shard 33:  75%|███████▍  | 74916267/100000000 [00:19<00:04, 5716716.98tokens/s]
Shard 33:  76%|███████▌  | 75536399/100000000 [00:19<00:04, 5839117.53tokens/s]
Shard 33:  76%|███████▌  | 76122222/100000000 [00:19<00:05, 4685942.86tokens/s]
Shard 33:  77%|███████▋  | 76648645/100000000 [00:19<00:04, 4820830.31tokens/s]
Shard 33:  77%|███████▋  | 77160114/100000000 [00:19<00:04, 4750129.23tokens/s]
Shard 33:  78%|███████▊  | 77654969/100000000 [00:20<00:05, 3938030.92tokens/s]
Shard 33:  78%|███████▊  | 78083194/100000000 [00:20<00:05, 3790263.50tokens/s]
Shard 33:  78%|███████▊  | 78485103/100000000 [00:20<00:05, 3727241.62tokens/s]
Shard 33:  79%|███████▉  | 78873080/100000000 [00:20<00:05, 3559615.18tokens/s]
Shard 33:  79%|███████▉  | 79239418/100000000 [00:20<00:06, 3458992.60tokens/s]
Shard 33:  80%|███████▉  | 79647967/100000000 [00:20<00:05, 3608561.28tokens/s]
Shard 33:  80%|████████  | 80016180/100000000 [00:20<00:05, 3611295.61tokens/s]
Shard 33:  80%|████████  | 80465571/100000000 [00:20<00:05, 3842756.43tokens/s]
Shard 33:  81%|████████  | 80855204/100000000 [00:21<00:05, 3726625.78tokens/s]
Shard 33:  81%|████████  | 81233717/100000000 [00:21<00:05, 3479667.74tokens/s]
Shard 33:  82%|████████▏ | 81591774/100000000 [00:21<00:05, 3286191.31tokens/s]
Shard 33:  82%|████████▏ | 81939592/100000000 [00:21<00:05, 3332203.66tokens/s]
Shard 33:  82%|████████▏ | 82308097/100000000 [00:21<00:05, 3429018.01tokens/s]
Shard 33:  83%|████████▎ | 82655222/100000000 [00:21<00:05, 3371224.34tokens/s]
Shard 33:  83%|████████▎ | 82995754/100000000 [00:21<00:05, 3249874.20tokens/s]
Shard 33:  83%|████████▎ | 83322888/100000000 [00:21<00:05, 3229198.85tokens/s]
Shard 33:  84%|████████▍ | 83755324/100000000 [00:21<00:04, 3539328.45tokens/s]
Shard 33:  84%|████████▍ | 84175351/100000000 [00:22<00:04, 3729842.04tokens/s]
Shard 33:  85%|████████▍ | 84647305/100000000 [00:22<00:03, 4015198.71tokens/s]
Shard 33:  85%|████████▌ | 85051467/100000000 [00:22<00:03, 3972486.18tokens/s]
Shard 33:  85%|████████▌ | 85450845/100000000 [00:22<00:03, 3941841.41tokens/s]
Shard 33:  86%|████████▌ | 85848330/100000000 [00:22<00:03, 3703502.72tokens/s]
Shard 33:  86%|████████▋ | 86351179/100000000 [00:22<00:03, 4048124.97tokens/s]
Shard 33:  87%|████████▋ | 86760754/100000000 [00:22<00:03, 4043450.05tokens/s]
Shard 33:  87%|████████▋ | 87169705/100000000 [00:22<00:03, 3583400.68tokens/s]
Shard 33:  88%|████████▊ | 87585757/100000000 [00:22<00:03, 3729014.64tokens/s]
Shard 33:  89%|████████▊ | 88567587/100000000 [00:23<00:02, 5401150.88tokens/s]
Shard 33:  89%|████████▉ | 89129089/100000000 [00:23<00:02, 4917646.94tokens/s]
Shard 33:  90%|████████▉ | 89642789/100000000 [00:23<00:02, 4318060.77tokens/s]
Shard 33:  90%|█████████ | 90100583/100000000 [00:23<00:02, 3819339.25tokens/s]
Shard 33:  91%|█████████ | 90507987/100000000 [00:23<00:02, 3206284.71tokens/s]
Shard 33:  91%|█████████ | 90885694/100000000 [00:23<00:02, 3317654.64tokens/s]
Shard 33:  91%|█████████ | 91241717/100000000 [00:23<00:02, 3081399.24tokens/s]
Shard 33:  92%|█████████▏| 91567390/100000000 [00:24<00:03, 2675480.60tokens/s]
Shard 33:  92%|█████████▏| 91852212/100000000 [00:24<00:03, 2616868.52tokens/s]
Shard 33:  92%|█████████▏| 92163459/100000000 [00:24<00:02, 2727256.44tokens/s]
Shard 33:  93%|█████████▎| 92550169/100000000 [00:24<00:02, 3007509.81tokens/s]
Shard 33:  93%|█████████▎| 92872843/100000000 [00:24<00:02, 3065219.79tokens/s]
Shard 33:  93%|█████████▎| 93332357/100000000 [00:24<00:01, 3484041.60tokens/s]
Shard 33:  94%|█████████▍| 93757460/100000000 [00:24<00:01, 3677988.99tokens/s]
Shard 33:  94%|█████████▍| 94169243/100000000 [00:24<00:01, 3803145.05tokens/s]
Shard 33:  95%|█████████▍| 94586118/100000000 [00:24<00:01, 3885169.92tokens/s]
Shard 33:  95%|█████████▌| 95070672/100000000 [00:24<00:01, 4155360.79tokens/s]
Shard 33:  95%|█████████▌| 95490489/100000000 [00:25<00:01, 3857099.07tokens/s]
Shard 33:  96%|█████████▌| 95883052/100000000 [00:25<00:01, 3553322.93tokens/s]
Shard 33:  96%|█████████▋| 96470783/100000000 [00:25<00:00, 4155493.73tokens/s]
Shard 33:  97%|█████████▋| 96908525/100000000 [00:25<00:00, 4175429.98tokens/s]
Shard 33:  97%|█████████▋| 97335070/100000000 [00:25<00:00, 3867290.13tokens/s]
Shard 33:  98%|█████████▊| 97864685/100000000 [00:25<00:00, 4227008.50tokens/s]
Shard 33:  98%|█████████▊| 98306691/100000000 [00:25<00:00, 4047698.82tokens/s]
Shard 33:  99%|█████████▊| 98719460/100000000 [00:25<00:00, 3719480.62tokens/s]
Shard 33:  99%|█████████▉| 99101894/100000000 [00:26<00:00, 3654062.96tokens/s]
Shard 33: 100%|█████████▉| 99652574/100000000 [00:26<00:00, 4149766.12tokens/s]
Shard 33: 100%|█████████▉| 99998289/100000000 [00:35<00:00, 2839203.35tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000033.bin (200,001,024 bytes) in the gpt-2 format

Shard 34:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 34:   0%|          | 374838/100000000 [00:00<00:26, 3701391.44tokens/s]
Shard 34:   1%|          | 745723/100000000 [00:00<00:34, 2885218.51tokens/s]
Shard 34:   1%|          | 1060267/100000000 [00:00<00:38, 2571093.06tokens/s]
Shard 34:   1%|▏         | 1324809/100000000 [00:00<00:39, 2509936.11tokens/s]
Shard 34:   2%|▏         | 1712888/100000000 [00:00<00:33, 2898706.99tokens/s]
Shard 34:   2%|▏         | 2029754/100000000 [00:00<00:33, 2963172.43tokens/s]
Shard 34:   2%|▏         | 2404235/100000000 [00:00<00:30, 3187393.95tokens/s]
Shard 34:   3%|▎         | 2856086/100000000 [00:00<00:27, 3583650.45tokens/s]
Shard 34:   3%|▎         | 3307678/100000000 [00:01<00:25, 3846201.22tokens/s]
Shard 34:   4%|▎         | 3697223/100000000 [00:01<00:26, 3663566.46tokens/s]
Shard 34:   4%|▍         | 4189666/100000000 [00:01<00:23, 4026798.53tokens/s]
Shard 34:   5%|▍         | 4600471/100000000 [00:01<00:24, 3840304.80tokens/s]
Shard 34:   5%|▍         | 4989989/100000000 [00:01<00:26, 3627697.70tokens/s]
Shard 34:   5%|▌         | 5360007/100000000 [00:01<00:27, 3482399.63tokens/s]
Shard 34:   6%|▌         | 5847623/100000000 [00:01<00:24, 3857127.52tokens/s]
Shard 34:   6%|▋         | 6282111/100000000 [00:01<00:23, 3963849.83tokens/s]
Shard 34:   7%|▋         | 6685592/100000000 [00:01<00:23, 3922997.98tokens/s]
Shard 34:   7%|▋         | 7203902/100000000 [00:01<00:21, 4266813.14tokens/s]
Shard 34:   8%|▊         | 7634563/100000000 [00:02<00:21, 4263563.56tokens/s]
Shard 34:   8%|▊         | 8063812/100000000 [00:02<00:23, 3868979.20tokens/s]
Shard 34:   8%|▊         | 8464907/100000000 [00:02<00:24, 3758666.59tokens/s]
Shard 34:   9%|▉         | 9085220/100000000 [00:02<00:20, 4330380.47tokens/s]
Shard 34:  10%|▉         | 9656336/100000000 [00:02<00:19, 4696852.19tokens/s]
Shard 34:  10%|█         | 10229446/100000000 [00:02<00:18, 4983548.13tokens/s]
Shard 34:  11%|█         | 10736641/100000000 [00:02<00:18, 4835488.62tokens/s]
Shard 34:  11%|█▏        | 11298111/100000000 [00:02<00:17, 5056245.99tokens/s]
Shard 34:  12%|█▏        | 11808685/100000000 [00:02<00:17, 4919327.88tokens/s]
Shard 34:  12%|█▏        | 12429690/100000000 [00:03<00:16, 5271840.45tokens/s]
Shard 34:  13%|█▎        | 13423540/100000000 [00:03<00:13, 6599537.79tokens/s]
Shard 34:  14%|█▍        | 14091789/100000000 [00:03<00:16, 5254041.43tokens/s]
Shard 34:  15%|█▍        | 14664543/100000000 [00:03<00:17, 4818011.85tokens/s]
Shard 34:  15%|█▌        | 15183441/100000000 [00:03<00:17, 4899771.66tokens/s]
Shard 34:  16%|█▌        | 15701347/100000000 [00:03<00:18, 4602826.76tokens/s]
Shard 34:  16%|█▌        | 16182069/100000000 [00:03<00:19, 4352486.13tokens/s]
Shard 34:  17%|█▋        | 16632791/100000000 [00:04<00:20, 4143742.89tokens/s]
Shard 34:  17%|█▋        | 17059529/100000000 [00:04<00:21, 3884859.97tokens/s]
Shard 34:  17%|█▋        | 17483877/100000000 [00:04<00:20, 3969341.59tokens/s]
Shard 34:  18%|█▊        | 17887613/100000000 [00:04<00:22, 3574852.51tokens/s]
Shard 34:  18%|█▊        | 18256404/100000000 [00:04<00:23, 3449121.06tokens/s]
Shard 34:  19%|█▊        | 18709721/100000000 [00:04<00:21, 3716232.76tokens/s]
Shard 34:  19%|█▉        | 19153548/100000000 [00:04<00:20, 3899424.18tokens/s]
Shard 34:  20%|█▉        | 19561609/100000000 [00:04<00:20, 3948625.83tokens/s]
Shard 34:  20%|█▉        | 19966009/100000000 [00:04<00:22, 3595443.27tokens/s]
Shard 34:  20%|██        | 20335442/100000000 [00:05<00:24, 3249916.43tokens/s]
Shard 34:  21%|██        | 20671327/100000000 [00:05<00:28, 2805548.53tokens/s]
Shard 34:  21%|██        | 20979697/100000000 [00:05<00:29, 2685871.73tokens/s]
Shard 34:  21%|██▏       | 21329127/100000000 [00:05<00:27, 2872354.20tokens/s]
Shard 34:  22%|██▏       | 21628153/100000000 [00:05<00:27, 2810664.56tokens/s]
Shard 34:  22%|██▏       | 21997469/100000000 [00:05<00:25, 3037591.27tokens/s]
Shard 34:  22%|██▏       | 22314511/100000000 [00:05<00:25, 3068402.10tokens/s]
Shard 34:  23%|██▎       | 22704546/100000000 [00:05<00:23, 3292448.71tokens/s]
Shard 34:  23%|██▎       | 23246143/100000000 [00:05<00:19, 3890789.37tokens/s]
Shard 34:  24%|██▎       | 23643309/100000000 [00:06<00:20, 3799537.86tokens/s]
Shard 34:  24%|██▍       | 24031116/100000000 [00:06<00:22, 3379568.48tokens/s]
Shard 34:  25%|██▍       | 24645271/100000000 [00:06<00:18, 4101769.43tokens/s]
Shard 34:  25%|██▌       | 25084514/100000000 [00:06<00:18, 3986829.07tokens/s]
Shard 34:  26%|██▌       | 25504398/100000000 [00:06<00:18, 4032254.30tokens/s]
Shard 34:  26%|██▌       | 25987354/100000000 [00:06<00:17, 4254870.86tokens/s]
Shard 34:  27%|██▋       | 26530397/100000000 [00:06<00:16, 4577011.82tokens/s]
Shard 34:  27%|██▋       | 27403988/100000000 [00:06<00:12, 5778375.30tokens/s]
Shard 34:  28%|██▊       | 27991334/100000000 [00:06<00:13, 5533590.87tokens/s]
Shard 34:  29%|██▊       | 28553723/100000000 [00:07<00:13, 5387525.96tokens/s]
Shard 34:  29%|██▉       | 29103899/100000000 [00:07<00:14, 4923706.54tokens/s]
Shard 34:  30%|██▉       | 29609017/100000000 [00:07<00:14, 4721705.34tokens/s]
Shard 34:  30%|███       | 30090150/100000000 [00:07<00:16, 4229135.91tokens/s]
Shard 34:  31%|███       | 30547618/100000000 [00:07<00:16, 4204035.39tokens/s]
Shard 34:  31%|███       | 30976367/100000000 [00:07<00:17, 3946534.81tokens/s]
Shard 34:  31%|███▏      | 31403824/100000000 [00:07<00:17, 3924945.66tokens/s]
Shard 34:  32%|███▏      | 31839088/100000000 [00:07<00:16, 4026537.15tokens/s]
Shard 34:  32%|███▏      | 32255931/100000000 [00:08<00:16, 4064158.56tokens/s]
Shard 34:  33%|███▎      | 32750673/100000000 [00:08<00:15, 4291779.58tokens/s]
Shard 34:  33%|███▎      | 33184429/100000000 [00:08<00:18, 3645643.83tokens/s]
Shard 34:  34%|███▎      | 33571545/100000000 [00:08<00:17, 3703356.15tokens/s]
Shard 34:  34%|███▍      | 33955515/100000000 [00:08<00:18, 3551155.38tokens/s]
Shard 34:  34%|███▍      | 34339582/100000000 [00:08<00:18, 3605199.57tokens/s]
Shard 34:  35%|███▍      | 34711663/100000000 [00:08<00:17, 3634359.12tokens/s]
Shard 34:  35%|███▌      | 35123633/100000000 [00:08<00:17, 3729864.11tokens/s]
Shard 34:  36%|███▌      | 35500820/100000000 [00:08<00:18, 3524185.25tokens/s]
Shard 34:  36%|███▌      | 35906513/100000000 [00:09<00:17, 3671348.45tokens/s]
Shard 34:  36%|███▋      | 36278556/100000000 [00:09<00:18, 3535145.19tokens/s]
Shard 34:  37%|███▋      | 36635746/100000000 [00:09<00:18, 3436890.10tokens/s]
Shard 34:  37%|███▋      | 37338794/100000000 [00:09<00:14, 4437687.95tokens/s]
Shard 34:  38%|███▊      | 38021990/100000000 [00:09<00:12, 5120086.61tokens/s]
Shard 34:  39%|███▊      | 38547562/100000000 [00:09<00:11, 5152418.69tokens/s]
Shard 34:  39%|███▉      | 39069948/100000000 [00:09<00:11, 5103088.73tokens/s]
Shard 34:  40%|███▉      | 39585486/100000000 [00:09<00:11, 5106166.20tokens/s]
Shard 34:  40%|████      | 40103796/100000000 [00:09<00:12, 4940011.64tokens/s]
Shard 34:  41%|████      | 40645000/100000000 [00:10<00:11, 5063075.78tokens/s]
Shard 34:  41%|████      | 41154265/100000000 [00:10<00:12, 4788421.51tokens/s]
Shard 34:  42%|████▏     | 41638270/100000000 [00:10<00:12, 4579736.69tokens/s]
Shard 34:  42%|████▏     | 42231314/100000000 [00:10<00:11, 4939546.79tokens/s]
Shard 34:  43%|████▎     | 42730916/100000000 [00:10<00:12, 4692940.34tokens/s]
Shard 34:  44%|████▎     | 43594033/100000000 [00:10<00:09, 5736889.94tokens/s]
Shard 34:  44%|████▍     | 44376006/100000000 [00:10<00:08, 6323765.40tokens/s]
Shard 34:  45%|████▌     | 45086333/100000000 [00:10<00:08, 6545562.89tokens/s]
Shard 34:  46%|████▌     | 45749552/100000000 [00:10<00:09, 5836417.17tokens/s]
Shard 34:  46%|████▋     | 46352688/100000000 [00:11<00:09, 5861902.25tokens/s]
Shard 34:  47%|████▋     | 46953578/100000000 [00:11<00:10, 5017643.90tokens/s]
Shard 34:  47%|████▋     | 47484231/100000000 [00:11<00:11, 4749550.23tokens/s]
Shard 34:  48%|████▊     | 47979318/100000000 [00:11<00:11, 4691226.03tokens/s]
Shard 34:  48%|████▊     | 48462300/100000000 [00:11<00:11, 4522966.02tokens/s]
Shard 34:  49%|████▉     | 49340010/100000000 [00:11<00:09, 5616756.70tokens/s]
Shard 34:  50%|████▉     | 49922534/100000000 [00:11<00:09, 5125450.02tokens/s]
Shard 34:  50%|█████     | 50457799/100000000 [00:11<00:10, 4893107.58tokens/s]
Shard 34:  51%|█████     | 50966650/100000000 [00:11<00:09, 4940557.12tokens/s]
Shard 34:  51%|█████▏    | 51471722/100000000 [00:12<00:11, 4309863.50tokens/s]
Shard 34:  52%|█████▏    | 52019503/100000000 [00:12<00:10, 4600726.57tokens/s]
Shard 34:  53%|█████▎    | 52502959/100000000 [00:12<00:10, 4651057.16tokens/s]
Shard 34:  53%|█████▎    | 52983356/100000000 [00:12<00:10, 4599611.66tokens/s]
Shard 34:  54%|█████▎    | 53605694/100000000 [00:12<00:09, 5019717.46tokens/s]
Shard 34:  54%|█████▍    | 54380847/100000000 [00:12<00:07, 5785598.87tokens/s]
Shard 34:  55%|█████▍    | 54970896/100000000 [00:12<00:08, 5003741.99tokens/s]
Shard 34:  55%|█████▌    | 55496960/100000000 [00:12<00:09, 4618194.35tokens/s]
Shard 34:  56%|█████▌    | 55980324/100000000 [00:13<00:10, 4381347.36tokens/s]
Shard 34:  56%|█████▋    | 56433444/100000000 [00:13<00:10, 4022503.46tokens/s]
Shard 34:  57%|█████▋    | 56868959/100000000 [00:13<00:10, 4063086.16tokens/s]
Shard 34:  57%|█████▋    | 57289891/100000000 [00:13<00:10, 4044880.38tokens/s]
Shard 34:  58%|█████▊    | 57701971/100000000 [00:13<00:11, 3818786.19tokens/s]
Shard 34:  58%|█████▊    | 58089277/100000000 [00:13<00:11, 3716851.17tokens/s]
Shard 34:  58%|█████▊    | 58464403/100000000 [00:13<00:11, 3607522.37tokens/s]
Shard 34:  59%|█████▉    | 58929206/100000000 [00:13<00:10, 3854239.39tokens/s]
Shard 34:  59%|█████▉    | 59346583/100000000 [00:13<00:10, 3935719.96tokens/s]
Shard 34:  60%|█████▉    | 59743332/100000000 [00:14<00:11, 3453010.14tokens/s]
Shard 34:  60%|██████    | 60100212/100000000 [00:14<00:13, 2929409.69tokens/s]
Shard 34:  60%|██████    | 60414595/100000000 [00:14<00:14, 2821034.33tokens/s]
Shard 34:  61%|██████    | 60725365/100000000 [00:14<00:13, 2881768.60tokens/s]
Shard 34:  61%|██████    | 61024213/100000000 [00:14<00:13, 2816272.44tokens/s]
Shard 34:  61%|██████▏   | 61477416/100000000 [00:14<00:11, 3261306.95tokens/s]
Shard 34:  62%|██████▏   | 61815168/100000000 [00:14<00:11, 3261786.57tokens/s]
Shard 34:  62%|██████▏   | 62149544/100000000 [00:15<00:13, 2842192.83tokens/s]
Shard 34:  62%|██████▏   | 62447741/100000000 [00:15<00:13, 2813411.55tokens/s]
Shard 34:  63%|██████▎   | 62738798/100000000 [00:15<00:14, 2518818.23tokens/s]
Shard 34:  63%|██████▎   | 63223203/100000000 [00:15<00:11, 3087124.18tokens/s]
Shard 34:  64%|██████▎   | 63550308/100000000 [00:15<00:13, 2785403.09tokens/s]
Shard 34:  64%|██████▍   | 63909890/100000000 [00:15<00:12, 2980578.88tokens/s]
Shard 34:  64%|██████▍   | 64498705/100000000 [00:15<00:09, 3685064.64tokens/s]
Shard 34:  65%|██████▍   | 64885168/100000000 [00:15<00:09, 3534743.15tokens/s]
Shard 34:  65%|██████▌   | 65251631/100000000 [00:15<00:09, 3561419.23tokens/s]
Shard 34:  66%|██████▌   | 65829337/100000000 [00:16<00:08, 4153755.21tokens/s]
Shard 34:  66%|██████▋   | 66255633/100000000 [00:16<00:08, 3986257.80tokens/s]
Shard 34:  67%|██████▋   | 66827747/100000000 [00:16<00:07, 4446257.91tokens/s]
Shard 34:  67%|██████▋   | 67281654/100000000 [00:16<00:08, 4082825.36tokens/s]
Shard 34:  68%|██████▊   | 67710077/100000000 [00:16<00:07, 4110845.65tokens/s]
Shard 34:  68%|██████▊   | 68129308/100000000 [00:16<00:07, 4109221.10tokens/s]
Shard 34:  69%|██████▊   | 68572767/100000000 [00:16<00:07, 4196242.69tokens/s]
Shard 34:  69%|██████▉   | 69029117/100000000 [00:16<00:07, 4298366.81tokens/s]
Shard 34:  69%|██████▉   | 69462723/100000000 [00:16<00:07, 3922325.95tokens/s]
Shard 34:  70%|██████▉   | 69863429/100000000 [00:17<00:07, 3774431.62tokens/s]
Shard 34:  70%|███████   | 70247041/100000000 [00:17<00:08, 3435588.66tokens/s]
Shard 34:  71%|███████   | 70600036/100000000 [00:17<00:08, 3383427.46tokens/s]
Shard 34:  71%|███████   | 71008957/100000000 [00:17<00:08, 3497837.01tokens/s]
Shard 34:  71%|███████▏  | 71363296/100000000 [00:17<00:08, 3304544.60tokens/s]
Shard 34:  72%|███████▏  | 71770201/100000000 [00:17<00:08, 3504933.25tokens/s]
Shard 34:  72%|███████▏  | 72127660/100000000 [00:17<00:07, 3493731.21tokens/s]
Shard 34:  72%|███████▏  | 72488321/100000000 [00:17<00:07, 3500938.86tokens/s]
Shard 34:  73%|███████▎  | 73068335/100000000 [00:17<00:06, 4143873.65tokens/s]
Shard 34:  74%|███████▎  | 73708661/100000000 [00:18<00:05, 4789568.04tokens/s]
Shard 34:  74%|███████▍  | 74274763/100000000 [00:18<00:05, 5033109.87tokens/s]
Shard 34:  75%|███████▍  | 74783043/100000000 [00:18<00:05, 4596921.88tokens/s]
Shard 34:  75%|███████▌  | 75253544/100000000 [00:18<00:05, 4609826.38tokens/s]
Shard 34:  76%|███████▌  | 75722880/100000000 [00:18<00:05, 4086123.20tokens/s]
Shard 34:  76%|███████▌  | 76146449/100000000 [00:18<00:06, 3964448.24tokens/s]
Shard 34:  77%|███████▋  | 76552905/100000000 [00:18<00:06, 3586831.30tokens/s]
Shard 34:  77%|███████▋  | 76923959/100000000 [00:18<00:06, 3346630.33tokens/s]
Shard 34:  77%|███████▋  | 77267974/100000000 [00:19<00:07, 3191719.91tokens/s]
Shard 34:  78%|███████▊  | 77638189/100000000 [00:19<00:06, 3312586.26tokens/s]
Shard 34:  78%|███████▊  | 77975486/100000000 [00:19<00:07, 3022449.98tokens/s]
Shard 34:  78%|███████▊  | 78364963/100000000 [00:19<00:06, 3235724.27tokens/s]
Shard 34:  79%|███████▊  | 78706393/100000000 [00:19<00:06, 3243341.72tokens/s]
Shard 34:  79%|███████▉  | 79036800/100000000 [00:19<00:06, 3137668.71tokens/s]
Shard 34:  79%|███████▉  | 79409405/100000000 [00:19<00:06, 3281907.32tokens/s]
Shard 34:  80%|███████▉  | 79741767/100000000 [00:19<00:06, 3094351.72tokens/s]
Shard 34:  80%|████████  | 80055661/100000000 [00:19<00:06, 2945820.53tokens/s]
Shard 34:  80%|████████  | 80400934/100000000 [00:20<00:06, 3068757.75tokens/s]
Shard 34:  81%|████████  | 80792465/100000000 [00:20<00:05, 3303603.01tokens/s]
Shard 34:  81%|████████  | 81131450/100000000 [00:20<00:05, 3310685.92tokens/s]
Shard 34:  81%|████████▏ | 81465807/100000000 [00:20<00:05, 3206860.96tokens/s]
Shard 34:  82%|████████▏ | 81944713/100000000 [00:20<00:04, 3651832.17tokens/s]
Shard 34:  82%|████████▏ | 82335141/100000000 [00:20<00:04, 3713308.61tokens/s]
Shard 34:  83%|████████▎ | 82736828/100000000 [00:20<00:04, 3782781.52tokens/s]
Shard 34:  83%|████████▎ | 83278731/100000000 [00:20<00:03, 4237827.25tokens/s]
Shard 34:  84%|████████▎ | 83705385/100000000 [00:20<00:04, 3752150.45tokens/s]
Shard 34:  84%|████████▍ | 84312676/100000000 [00:20<00:03, 4376872.17tokens/s]
Shard 34:  85%|████████▍ | 84765353/100000000 [00:21<00:03, 3945441.55tokens/s]
Shard 34:  85%|████████▌ | 85177836/100000000 [00:21<00:03, 3955392.30tokens/s]
Shard 34:  86%|████████▌ | 85585555/100000000 [00:21<00:03, 3705336.94tokens/s]
Shard 34:  86%|████████▌ | 85966554/100000000 [00:21<00:03, 3686188.57tokens/s]
Shard 34:  87%|████████▋ | 86876556/100000000 [00:21<00:02, 5049521.73tokens/s]
Shard 34:  87%|████████▋ | 87393029/100000000 [00:21<00:02, 5014526.01tokens/s]
Shard 34:  88%|████████▊ | 87902308/100000000 [00:21<00:02, 4987689.55tokens/s]
Shard 34:  88%|████████▊ | 88412956/100000000 [00:21<00:02, 4536964.76tokens/s]
Shard 34:  89%|████████▉ | 88880955/100000000 [00:22<00:02, 4070045.75tokens/s]
Shard 34:  89%|████████▉ | 89302534/100000000 [00:22<00:02, 3992222.17tokens/s]
Shard 34:  90%|████████▉ | 89710948/100000000 [00:22<00:03, 3333769.89tokens/s]
Shard 34:  90%|█████████ | 90209335/100000000 [00:22<00:02, 3691087.98tokens/s]
Shard 34:  91%|█████████ | 90761704/100000000 [00:22<00:02, 4124307.30tokens/s]
Shard 34:  91%|█████████ | 91198846/100000000 [00:22<00:02, 3817110.98tokens/s]
Shard 34:  92%|█████████▏| 91600080/100000000 [00:22<00:02, 3559175.38tokens/s]
Shard 34:  92%|█████████▏| 91979763/100000000 [00:22<00:02, 3591257.20tokens/s]
Shard 34:  92%|█████████▏| 92350563/100000000 [00:23<00:02, 3313582.94tokens/s]
Shard 34:  93%|█████████▎| 92692272/100000000 [00:23<00:02, 3234575.92tokens/s]
Shard 34:  93%|█████████▎| 93070885/100000000 [00:23<00:02, 3378533.54tokens/s]
Shard 34:  94%|█████████▎| 93595434/100000000 [00:23<00:01, 3876939.99tokens/s]
Shard 34:  94%|█████████▍| 94138443/100000000 [00:23<00:01, 4280846.70tokens/s]
Shard 34:  95%|█████████▍| 94575990/100000000 [00:23<00:01, 4301320.38tokens/s]
Shard 34:  95%|█████████▌| 95429135/100000000 [00:23<00:00, 5519773.51tokens/s]
Shard 34:  96%|█████████▌| 95990864/100000000 [00:23<00:00, 4584091.07tokens/s]
Shard 34:  96%|█████████▋| 96482157/100000000 [00:23<00:00, 4382836.03tokens/s]
Shard 34:  97%|█████████▋| 96996079/100000000 [00:24<00:00, 4553200.18tokens/s]
Shard 34:  97%|█████████▋| 97470072/100000000 [00:24<00:00, 3562079.73tokens/s]
Shard 34:  98%|█████████▊| 97870468/100000000 [00:24<00:00, 3509791.99tokens/s]
Shard 34:  98%|█████████▊| 98273219/100000000 [00:24<00:00, 3628500.30tokens/s]
Shard 34:  99%|█████████▉| 98763146/100000000 [00:24<00:00, 3928767.76tokens/s]
Shard 34:  99%|█████████▉| 99177792/100000000 [00:24<00:00, 3542007.38tokens/s]
Shard 34: 100%|█████████▉| 99557229/100000000 [00:24<00:00, 3520634.85tokens/s]
Shard 34: 100%|█████████▉| 99924927/100000000 [00:25<00:00, 3283612.60tokens/s]
Shard 34: 100%|█████████▉| 99999778/100000000 [00:33<00:00, 2997097.50tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000034.bin (200,001,024 bytes) in the gpt-2 format

Shard 35:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 35:   0%|          | 272929/100000000 [00:00<00:36, 2727523.81tokens/s]
Shard 35:   1%|          | 560392/100000000 [00:00<00:35, 2797736.65tokens/s]
Shard 35:   1%|          | 919202/100000000 [00:00<00:31, 3106567.09tokens/s]
Shard 35:   1%|          | 1231961/100000000 [00:00<00:35, 2783389.04tokens/s]
Shard 35:   2%|▏         | 1734655/100000000 [00:00<00:28, 3491421.85tokens/s]
Shard 35:   2%|▏         | 2090739/100000000 [00:00<00:35, 2729349.55tokens/s]
Shard 35:   3%|▎         | 2544194/100000000 [00:00<00:30, 3179513.24tokens/s]
Shard 35:   3%|▎         | 3048631/100000000 [00:00<00:26, 3682771.54tokens/s]
Shard 35:   4%|▎         | 3609781/100000000 [00:01<00:23, 4184185.54tokens/s]
Shard 35:   5%|▌         | 5128920/100000000 [00:01<00:13, 7290649.93tokens/s]
Shard 35:   6%|▌         | 5901313/100000000 [00:01<00:17, 5426409.35tokens/s]
Shard 35:   7%|▋         | 6542064/100000000 [00:01<00:17, 5195075.55tokens/s]
Shard 35:   7%|▋         | 7128812/100000000 [00:01<00:19, 4735443.36tokens/s]
Shard 35:   8%|▊         | 7656502/100000000 [00:01<00:19, 4834057.98tokens/s]
Shard 35:   8%|▊         | 8261018/100000000 [00:01<00:17, 5127234.00tokens/s]
Shard 35:   9%|▉         | 8823674/100000000 [00:02<00:19, 4601743.81tokens/s]
Shard 35:   9%|▉         | 9314691/100000000 [00:02<00:20, 4372876.33tokens/s]
Shard 35:  10%|▉         | 9890635/100000000 [00:02<00:19, 4704451.78tokens/s]
Shard 35:  10%|█         | 10382781/100000000 [00:02<00:20, 4290793.68tokens/s]
Shard 35:  11%|█         | 10903250/100000000 [00:02<00:19, 4499856.83tokens/s]
Shard 35:  11%|█▏        | 11415746/100000000 [00:02<00:19, 4659780.66tokens/s]
Shard 35:  12%|█▏        | 11896336/100000000 [00:02<00:19, 4563109.93tokens/s]
Shard 35:  12%|█▏        | 12368122/100000000 [00:02<00:21, 4008304.32tokens/s]
Shard 35:  13%|█▎        | 12786344/100000000 [00:03<00:25, 3476929.34tokens/s]
Shard 35:  13%|█▎        | 13181646/100000000 [00:03<00:26, 3283754.80tokens/s]
Shard 35:  14%|█▎        | 13524876/100000000 [00:03<00:28, 3066342.84tokens/s]
Shard 35:  14%|█▍        | 13841559/100000000 [00:03<00:34, 2507131.89tokens/s]
Shard 35:  14%|█▍        | 14113278/100000000 [00:03<00:37, 2310042.03tokens/s]
Shard 35:  14%|█▍        | 14356986/100000000 [00:03<00:38, 2242983.27tokens/s]
Shard 35:  15%|█▍        | 14669938/100000000 [00:03<00:34, 2441769.65tokens/s]
Shard 35:  15%|█▍        | 14998670/100000000 [00:03<00:32, 2649838.41tokens/s]
Shard 35:  15%|█▌        | 15498325/100000000 [00:04<00:25, 3258571.79tokens/s]
Shard 35:  16%|█▌        | 15844152/100000000 [00:04<00:28, 2925050.73tokens/s]
Shard 35:  16%|█▌        | 16159922/100000000 [00:04<00:28, 2982940.41tokens/s]
Shard 35:  17%|█▋        | 16645600/100000000 [00:04<00:23, 3477214.83tokens/s]
Shard 35:  18%|█▊        | 17685648/100000000 [00:04<00:15, 5379425.30tokens/s]
Shard 35:  18%|█▊        | 18250595/100000000 [00:04<00:16, 4887496.87tokens/s]
Shard 35:  19%|█▉        | 18765989/100000000 [00:04<00:19, 4266072.25tokens/s]
Shard 35:  19%|█▉        | 19224893/100000000 [00:04<00:18, 4329598.90tokens/s]
Shard 35:  20%|██        | 20135827/100000000 [00:05<00:14, 5548225.42tokens/s]
Shard 35:  21%|██        | 20727118/100000000 [00:05<00:16, 4668101.99tokens/s]
Shard 35:  21%|██        | 21239818/100000000 [00:05<00:17, 4404301.36tokens/s]
Shard 35:  22%|██▏       | 21711627/100000000 [00:05<00:19, 4089220.01tokens/s]
Shard 35:  22%|██▏       | 22143933/100000000 [00:05<00:20, 3858215.23tokens/s]
Shard 35:  23%|██▎       | 22664349/100000000 [00:05<00:18, 4176239.92tokens/s]
Shard 35:  23%|██▎       | 23100691/100000000 [00:05<00:18, 4134881.02tokens/s]
Shard 35:  24%|██▎       | 23526809/100000000 [00:06<00:39, 1933265.18tokens/s]
Shard 35:  24%|██▍       | 23895554/100000000 [00:06<00:34, 2193083.36tokens/s]
Shard 35:  24%|██▍       | 24257231/100000000 [00:06<00:31, 2437534.50tokens/s]
Shard 35:  25%|██▍       | 24599476/100000000 [00:06<00:29, 2532930.68tokens/s]
Shard 35:  25%|██▌       | 25072005/100000000 [00:06<00:24, 2998388.32tokens/s]
Shard 35:  26%|██▌       | 25725541/100000000 [00:06<00:19, 3819613.85tokens/s]
Shard 35:  26%|██▌       | 26181858/100000000 [00:06<00:18, 3997394.23tokens/s]
Shard 35:  27%|██▋       | 26869797/100000000 [00:07<00:15, 4740329.68tokens/s]
Shard 35:  27%|██▋       | 27420569/100000000 [00:07<00:14, 4932203.22tokens/s]
Shard 35:  28%|██▊       | 27948206/100000000 [00:07<00:15, 4516082.69tokens/s]
Shard 35:  28%|██▊       | 28431454/100000000 [00:07<00:16, 4436988.93tokens/s]
Shard 35:  29%|██▉       | 28895953/100000000 [00:07<00:19, 3729337.26tokens/s]
Shard 35:  29%|██▉       | 29300495/100000000 [00:07<00:19, 3668209.38tokens/s]
Shard 35:  30%|██▉       | 29688400/100000000 [00:07<00:19, 3690787.02tokens/s]
Shard 35:  30%|███       | 30072624/100000000 [00:07<00:21, 3252258.09tokens/s]
Shard 35:  31%|███       | 30800031/100000000 [00:08<00:16, 4237215.45tokens/s]
Shard 35:  31%|███▏      | 31300278/100000000 [00:08<00:15, 4398834.13tokens/s]
Shard 35:  32%|███▏      | 31766441/100000000 [00:08<00:15, 4326608.29tokens/s]
Shard 35:  32%|███▏      | 32217277/100000000 [00:08<00:16, 4229172.15tokens/s]
Shard 35:  33%|███▎      | 32653460/100000000 [00:08<00:18, 3699729.76tokens/s]
Shard 35:  33%|███▎      | 33188067/100000000 [00:08<00:16, 4109707.28tokens/s]
Shard 35:  34%|███▎      | 33620099/100000000 [00:08<00:16, 4004503.99tokens/s]
Shard 35:  34%|███▍      | 34253807/100000000 [00:08<00:14, 4603523.41tokens/s]
Shard 35:  35%|███▍      | 34731525/100000000 [00:09<00:14, 4397145.35tokens/s]
Shard 35:  35%|███▌      | 35183834/100000000 [00:09<00:15, 4261597.84tokens/s]
Shard 35:  36%|███▌      | 35765275/100000000 [00:09<00:13, 4674445.02tokens/s]
Shard 35:  36%|███▌      | 36244502/100000000 [00:09<00:14, 4391387.72tokens/s]
Shard 35:  37%|███▋      | 36693332/100000000 [00:09<00:14, 4224357.10tokens/s]
Shard 35:  37%|███▋      | 37123113/100000000 [00:09<00:16, 3782092.65tokens/s]
Shard 35:  38%|███▊      | 37578435/100000000 [00:09<00:15, 3979343.18tokens/s]
Shard 35:  38%|███▊      | 37988334/100000000 [00:09<00:16, 3707511.05tokens/s]
Shard 35:  38%|███▊      | 38369259/100000000 [00:09<00:17, 3498229.13tokens/s]
Shard 35:  39%|███▊      | 38729740/100000000 [00:10<00:20, 3042824.77tokens/s]
Shard 35:  39%|███▉      | 39126646/100000000 [00:10<00:18, 3260208.35tokens/s]
Shard 35:  39%|███▉      | 39466734/100000000 [00:10<00:19, 3107308.51tokens/s]
Shard 35:  40%|███▉      | 39787626/100000000 [00:10<00:19, 3043312.07tokens/s]
Shard 35:  40%|████      | 40098483/100000000 [00:10<00:23, 2523178.13tokens/s]
Shard 35:  40%|████      | 40367619/100000000 [00:10<00:24, 2448315.73tokens/s]
Shard 35:  41%|████      | 40623961/100000000 [00:10<00:24, 2471377.28tokens/s]
Shard 35:  41%|████      | 41020528/100000000 [00:10<00:20, 2850376.32tokens/s]
Shard 35:  41%|████▏     | 41360417/100000000 [00:11<00:19, 2986271.20tokens/s]
Shard 35:  42%|████▏     | 41715944/100000000 [00:11<00:18, 3126788.04tokens/s]
Shard 35:  42%|████▏     | 42036526/100000000 [00:11<00:18, 3081426.26tokens/s]
Shard 35:  42%|████▏     | 42350291/100000000 [00:11<00:18, 3091481.56tokens/s]
Shard 35:  43%|████▎     | 42663703/100000000 [00:11<00:19, 3007573.81tokens/s]
Shard 35:  43%|████▎     | 43085979/100000000 [00:11<00:17, 3297740.95tokens/s]
Shard 35:  43%|████▎     | 43418611/100000000 [00:11<00:18, 3097720.37tokens/s]
Shard 35:  44%|████▎     | 43732327/100000000 [00:11<00:18, 3002245.87tokens/s]
Shard 35:  44%|████▍     | 44039159/100000000 [00:11<00:18, 2962780.82tokens/s]
Shard 35:  44%|████▍     | 44359350/100000000 [00:12<00:18, 3025749.60tokens/s]
Shard 35:  45%|████▍     | 44673742/100000000 [00:12<00:18, 3015867.76tokens/s]
Shard 35:  45%|████▌     | 45010989/100000000 [00:12<00:17, 3117808.53tokens/s]
Shard 35:  45%|████▌     | 45324570/100000000 [00:12<00:18, 3020091.86tokens/s]
Shard 35:  46%|████▌     | 45628018/100000000 [00:12<00:21, 2499968.59tokens/s]
Shard 35:  46%|████▌     | 45893280/100000000 [00:12<00:24, 2244934.74tokens/s]
Shard 35:  46%|████▌     | 46131254/100000000 [00:12<00:25, 2097145.53tokens/s]
Shard 35:  46%|████▋     | 46463070/100000000 [00:12<00:22, 2387289.37tokens/s]
Shard 35:  47%|████▋     | 46759750/100000000 [00:13<00:21, 2526893.73tokens/s]
Shard 35:  47%|████▋     | 47024289/100000000 [00:13<00:21, 2505681.39tokens/s]
Shard 35:  47%|████▋     | 47297975/100000000 [00:13<00:21, 2456956.41tokens/s]
Shard 35:  48%|████▊     | 47554295/100000000 [00:13<00:24, 2152898.80tokens/s]
Shard 35:  48%|████▊     | 47853589/100000000 [00:13<00:22, 2345917.47tokens/s]
Shard 35:  48%|████▊     | 48108958/100000000 [00:13<00:21, 2389811.40tokens/s]
Shard 35:  48%|████▊     | 48451678/100000000 [00:13<00:19, 2670840.01tokens/s]
Shard 35:  49%|████▊     | 48746033/100000000 [00:13<00:18, 2746564.16tokens/s]
Shard 35:  49%|████▉     | 49027152/100000000 [00:13<00:18, 2764687.73tokens/s]
Shard 35:  49%|████▉     | 49393887/100000000 [00:14<00:16, 3023989.73tokens/s]
Shard 35:  50%|████▉     | 49823208/100000000 [00:14<00:14, 3387490.53tokens/s]
Shard 35:  50%|█████     | 50166984/100000000 [00:14<00:15, 3272080.26tokens/s]
Shard 35:  50%|█████     | 50498233/100000000 [00:14<00:15, 3119872.56tokens/s]
Shard 35:  51%|█████     | 50892121/100000000 [00:14<00:14, 3343989.98tokens/s]
Shard 35:  51%|█████▏    | 51353171/100000000 [00:14<00:13, 3700970.41tokens/s]
Shard 35:  52%|█████▏    | 52093212/100000000 [00:14<00:10, 4772105.23tokens/s]
Shard 35:  53%|█████▎    | 52578499/100000000 [00:14<00:10, 4559938.12tokens/s]
Shard 35:  53%|█████▎    | 53235464/100000000 [00:14<00:09, 5129200.12tokens/s]
Shard 35:  54%|█████▍    | 53756368/100000000 [00:14<00:09, 4775536.29tokens/s]
Shard 35:  54%|█████▍    | 54245892/100000000 [00:15<00:09, 4798809.27tokens/s]
Shard 35:  55%|█████▍    | 54732958/100000000 [00:15<00:10, 4357770.87tokens/s]
Shard 35:  55%|█████▌    | 55180201/100000000 [00:15<00:10, 4095489.34tokens/s]
Shard 35:  56%|█████▌    | 55600374/100000000 [00:15<00:11, 3813350.45tokens/s]
Shard 35:  56%|█████▌    | 55989992/100000000 [00:15<00:12, 3389342.03tokens/s]
Shard 35:  57%|█████▋    | 56610226/100000000 [00:15<00:10, 4080093.40tokens/s]
Shard 35:  57%|█████▋    | 57042646/100000000 [00:15<00:11, 3750431.14tokens/s]
Shard 35:  57%|█████▋    | 57437438/100000000 [00:15<00:11, 3670409.04tokens/s]
Shard 35:  58%|█████▊    | 57817942/100000000 [00:16<00:12, 3281781.16tokens/s]
Shard 35:  58%|█████▊    | 58337598/100000000 [00:16<00:11, 3749069.82tokens/s]
Shard 35:  59%|█████▉    | 59088215/100000000 [00:16<00:08, 4708514.40tokens/s]
Shard 35:  60%|█████▉    | 59587974/100000000 [00:16<00:08, 4713700.81tokens/s]
Shard 35:  60%|██████    | 60086055/100000000 [00:16<00:08, 4787136.60tokens/s]
Shard 35:  61%|██████    | 60580375/100000000 [00:16<00:09, 4316750.11tokens/s]
Shard 35:  61%|██████    | 61029764/100000000 [00:16<00:09, 4135162.48tokens/s]
Shard 35:  61%|██████▏   | 61455942/100000000 [00:16<00:11, 3487133.69tokens/s]
Shard 35:  62%|██████▏   | 61828661/100000000 [00:17<00:11, 3441794.65tokens/s]
Shard 35:  62%|██████▏   | 62198807/100000000 [00:17<00:10, 3506104.55tokens/s]
Shard 35:  63%|██████▎   | 62595484/100000000 [00:17<00:10, 3616843.42tokens/s]
Shard 35:  63%|██████▎   | 63040145/100000000 [00:17<00:09, 3809113.99tokens/s]
Shard 35:  63%|██████▎   | 63429492/100000000 [00:17<00:11, 3182664.27tokens/s]
Shard 35:  64%|██████▍   | 63864108/100000000 [00:17<00:10, 3454093.90tokens/s]
Shard 35:  64%|██████▍   | 64230060/100000000 [00:17<00:10, 3476035.31tokens/s]
Shard 35:  65%|██████▍   | 64656262/100000000 [00:17<00:09, 3654410.39tokens/s]
Shard 35:  65%|██████▌   | 65034638/100000000 [00:17<00:09, 3566845.24tokens/s]
Shard 35:  65%|██████▌   | 65475336/100000000 [00:18<00:09, 3788192.13tokens/s]
Shard 35:  66%|██████▌   | 66019179/100000000 [00:18<00:08, 4207639.65tokens/s]
Shard 35:  66%|██████▋   | 66446912/100000000 [00:18<00:08, 4090205.28tokens/s]
Shard 35:  67%|██████▋   | 67493095/100000000 [00:18<00:05, 5836234.99tokens/s]
Shard 35:  68%|██████▊   | 68086710/100000000 [00:18<00:06, 5165730.90tokens/s]
Shard 35:  69%|██████▊   | 68623357/100000000 [00:18<00:06, 4590742.86tokens/s]
Shard 35:  69%|██████▉   | 69134753/100000000 [00:18<00:06, 4720384.28tokens/s]
Shard 35:  70%|██████▉   | 69720669/100000000 [00:18<00:06, 5008304.91tokens/s]
Shard 35:  70%|███████   | 70238474/100000000 [00:19<00:06, 4804711.99tokens/s]
Shard 35:  71%|███████   | 70732462/100000000 [00:19<00:06, 4561524.74tokens/s]
Shard 35:  71%|███████   | 71198311/100000000 [00:19<00:06, 4564286.30tokens/s]
Shard 35:  72%|███████▏  | 71822488/100000000 [00:19<00:05, 4963448.71tokens/s]
Shard 35:  72%|███████▏  | 72359369/100000000 [00:19<00:05, 5036572.07tokens/s]
Shard 35:  73%|███████▎  | 72868352/100000000 [00:19<00:06, 4457092.48tokens/s]
Shard 35:  73%|███████▎  | 73329534/100000000 [00:19<00:06, 4313447.61tokens/s]
Shard 35:  74%|███████▍  | 73771356/100000000 [00:19<00:06, 4168399.56tokens/s]
Shard 35:  74%|███████▍  | 74194898/100000000 [00:19<00:06, 4155987.55tokens/s]
Shard 35:  75%|███████▍  | 74756977/100000000 [00:20<00:05, 4548524.06tokens/s]
Shard 35:  75%|███████▌  | 75329309/100000000 [00:20<00:05, 4873313.38tokens/s]
Shard 35:  76%|███████▌  | 75824035/100000000 [00:20<00:05, 4808372.66tokens/s]
Shard 35:  76%|███████▋  | 76309938/100000000 [00:20<00:05, 4036725.73tokens/s]
Shard 35:  77%|███████▋  | 76854687/100000000 [00:20<00:05, 4395587.74tokens/s]
Shard 35:  77%|███████▋  | 77317894/100000000 [00:20<00:05, 4330208.78tokens/s]
Shard 35:  78%|███████▊  | 77784521/100000000 [00:20<00:05, 4421011.34tokens/s]
Shard 35:  78%|███████▊  | 78238487/100000000 [00:20<00:04, 4424509.32tokens/s]
Shard 35:  79%|███████▉  | 78750845/100000000 [00:20<00:04, 4591431.83tokens/s]
Shard 35:  79%|███████▉  | 79227950/100000000 [00:21<00:04, 4638739.66tokens/s]
Shard 35:  80%|███████▉  | 79696752/100000000 [00:21<00:04, 4533601.25tokens/s]
Shard 35:  80%|████████  | 80215924/100000000 [00:21<00:04, 4653561.33tokens/s]
Shard 35:  81%|████████  | 80721455/100000000 [00:21<00:04, 4767317.55tokens/s]
Shard 35:  81%|████████▏ | 81383370/100000000 [00:21<00:03, 5306327.11tokens/s]
Shard 35:  82%|████████▏ | 81917810/100000000 [00:21<00:03, 4861138.60tokens/s]
Shard 35:  82%|████████▏ | 82415334/100000000 [00:21<00:03, 4444241.19tokens/s]
Shard 35:  83%|████████▎ | 82965868/100000000 [00:21<00:03, 4716482.60tokens/s]
Shard 35:  83%|████████▎ | 83453543/100000000 [00:21<00:03, 4749265.00tokens/s]
Shard 35:  84%|████████▍ | 83969788/100000000 [00:22<00:03, 4865144.84tokens/s]
Shard 35:  85%|████████▍ | 84643020/100000000 [00:22<00:02, 5376869.51tokens/s]
Shard 35:  85%|████████▌ | 85228903/100000000 [00:22<00:02, 5514661.87tokens/s]
Shard 35:  86%|████████▌ | 85789918/100000000 [00:22<00:02, 5024392.94tokens/s]
Shard 35:  87%|████████▋ | 86502561/100000000 [00:22<00:02, 5594916.40tokens/s]
Shard 35:  87%|████████▋ | 87468953/100000000 [00:22<00:01, 6737798.27tokens/s]
Shard 35:  88%|████████▊ | 88167199/100000000 [00:22<00:02, 5473273.34tokens/s]
Shard 35:  89%|████████▉ | 88764104/100000000 [00:22<00:02, 5055440.93tokens/s]
Shard 35:  89%|████████▉ | 89306751/100000000 [00:22<00:02, 5027935.86tokens/s]
Shard 35:  90%|████████▉ | 89835475/100000000 [00:23<00:02, 5008551.98tokens/s]
Shard 35:  90%|█████████ | 90353952/100000000 [00:23<00:01, 4972288.35tokens/s]
Shard 35:  91%|█████████ | 90863189/100000000 [00:23<00:02, 4407701.61tokens/s]
Shard 35:  91%|█████████▏| 91321624/100000000 [00:23<00:01, 4386538.75tokens/s]
Shard 35:  92%|█████████▏| 91772703/100000000 [00:23<00:01, 4383178.12tokens/s]
Shard 35:  92%|█████████▏| 92226243/100000000 [00:23<00:01, 4275530.70tokens/s]
Shard 35:  93%|█████████▎| 92659750/100000000 [00:23<00:01, 4059630.23tokens/s]
Shard 35:  93%|█████████▎| 93070707/100000000 [00:23<00:01, 4029321.92tokens/s]
Shard 35:  94%|█████████▎| 93591622/100000000 [00:24<00:01, 4348263.16tokens/s]
Shard 35:  94%|█████████▍| 94031977/100000000 [00:24<00:01, 3785845.89tokens/s]
Shard 35:  94%|█████████▍| 94431625/100000000 [00:24<00:01, 3814436.65tokens/s]
Shard 35:  95%|█████████▍| 94827611/100000000 [00:24<00:01, 3848062.79tokens/s]
Shard 35:  95%|█████████▌| 95220497/100000000 [00:24<00:01, 3730964.14tokens/s]
Shard 35:  96%|█████████▌| 95614104/100000000 [00:24<00:01, 3760117.65tokens/s]
Shard 35:  96%|█████████▌| 96009886/100000000 [00:24<00:01, 3786376.30tokens/s]
Shard 35:  96%|█████████▋| 96392065/100000000 [00:24<00:01, 3424375.37tokens/s]
Shard 35:  97%|█████████▋| 96958966/100000000 [00:24<00:00, 4027565.40tokens/s]
Shard 35:  97%|█████████▋| 97374187/100000000 [00:25<00:00, 3767215.50tokens/s]
Shard 35:  98%|█████████▊| 97922355/100000000 [00:25<00:00, 4213408.72tokens/s]
Shard 35:  98%|█████████▊| 98362580/100000000 [00:25<00:00, 4260746.00tokens/s]
Shard 35:  99%|█████████▉| 98798327/100000000 [00:25<00:00, 4101810.32tokens/s]
Shard 35:  99%|█████████▉| 99216608/100000000 [00:25<00:00, 3860996.08tokens/s]
Shard 35: 100%|█████████▉| 99614724/100000000 [00:25<00:00, 3890420.79tokens/s]
Shard 35: 100%|█████████▉| 99999944/100000000 [00:33<00:00, 2965779.95tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000035.bin (200,001,024 bytes) in the gpt-2 format

Shard 36:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 36:   0%|          | 246810/100000000 [00:00<00:40, 2458115.87tokens/s]
Shard 36:   0%|          | 493112/100000000 [00:00<00:44, 2218647.14tokens/s]
Shard 36:   1%|          | 737210/100000000 [00:00<00:43, 2308425.24tokens/s]
Shard 36:   1%|          | 969900/100000000 [00:00<00:44, 2218207.45tokens/s]
Shard 36:   1%|▏         | 1261889/100000000 [00:00<00:40, 2458377.93tokens/s]
Shard 36:   2%|▏         | 1510676/100000000 [00:00<00:43, 2286493.26tokens/s]
Shard 36:   2%|▏         | 1876574/100000000 [00:00<00:36, 2692923.37tokens/s]
Shard 36:   2%|▏         | 2171735/100000000 [00:00<00:35, 2746445.18tokens/s]
Shard 36:   3%|▎         | 2591326/100000000 [00:00<00:30, 3156709.85tokens/s]
Shard 36:   3%|▎         | 2927562/100000000 [00:01<00:30, 3210010.72tokens/s]
Shard 36:   3%|▎         | 3290525/100000000 [00:01<00:29, 3315455.73tokens/s]
Shard 36:   4%|▎         | 3625187/100000000 [00:01<00:30, 3192751.58tokens/s]
Shard 36:   4%|▍         | 3946721/100000000 [00:01<00:30, 3196658.13tokens/s]
Shard 36:   4%|▍         | 4268387/100000000 [00:01<00:30, 3112800.10tokens/s]
Shard 36:   5%|▍         | 4581093/100000000 [00:01<00:31, 3016469.42tokens/s]
Shard 36:   5%|▌         | 5037300/100000000 [00:01<00:27, 3452800.94tokens/s]
Shard 36:   6%|▌         | 5585888/100000000 [00:01<00:23, 4026549.07tokens/s]
Shard 36:   6%|▌         | 5992535/100000000 [00:01<00:24, 3797153.66tokens/s]
Shard 36:   6%|▋         | 6385197/100000000 [00:02<00:24, 3763663.20tokens/s]
Shard 36:   7%|▋         | 6765200/100000000 [00:02<00:25, 3649830.90tokens/s]
Shard 36:   7%|▋         | 7151601/100000000 [00:02<00:25, 3704594.31tokens/s]
Shard 36:   8%|▊         | 7524553/100000000 [00:02<00:24, 3710778.03tokens/s]
Shard 36:   8%|▊         | 7897614/100000000 [00:02<00:26, 3436329.68tokens/s]
Shard 36:   8%|▊         | 8275330/100000000 [00:02<00:26, 3489335.71tokens/s]
Shard 36:   9%|▊         | 8658511/100000000 [00:02<00:25, 3567397.42tokens/s]
Shard 36:   9%|▉         | 9169728/100000000 [00:02<00:22, 4006303.60tokens/s]
Shard 36:  10%|▉         | 9691111/100000000 [00:02<00:20, 4349086.86tokens/s]
Shard 36:  10%|█         | 10149195/100000000 [00:02<00:21, 4239941.86tokens/s]
Shard 36:  11%|█         | 10607248/100000000 [00:03<00:20, 4328689.32tokens/s]
Shard 36:  11%|█         | 11055163/100000000 [00:03<00:20, 4300754.27tokens/s]
Shard 36:  12%|█▏        | 11562853/100000000 [00:03<00:19, 4453428.22tokens/s]
Shard 36:  12%|█▏        | 12030134/100000000 [00:03<00:19, 4505173.36tokens/s]
Shard 36:  12%|█▏        | 12482401/100000000 [00:03<00:22, 3858958.13tokens/s]
Shard 36:  13%|█▎        | 12884689/100000000 [00:03<00:24, 3602943.61tokens/s]
Shard 36:  13%|█▎        | 13257946/100000000 [00:03<00:26, 3240138.94tokens/s]
Shard 36:  14%|█▎        | 13598845/100000000 [00:03<00:28, 3010099.93tokens/s]
Shard 36:  14%|█▍        | 13910820/100000000 [00:04<00:29, 2962345.63tokens/s]
Shard 36:  14%|█▍        | 14213982/100000000 [00:04<00:28, 2975520.80tokens/s]
Shard 36:  15%|█▍        | 14519984/100000000 [00:04<00:30, 2802562.85tokens/s]
Shard 36:  15%|█▍        | 14898349/100000000 [00:04<00:28, 3039295.54tokens/s]
Shard 36:  15%|█▌        | 15222589/100000000 [00:04<00:27, 3091322.62tokens/s]
Shard 36:  16%|█▌        | 15699073/100000000 [00:04<00:23, 3557255.07tokens/s]
Shard 36:  16%|█▌        | 16062750/100000000 [00:04<00:25, 3284096.00tokens/s]
Shard 36:  16%|█▋        | 16401074/100000000 [00:04<00:27, 3031193.77tokens/s]
Shard 36:  17%|█▋        | 16713056/100000000 [00:05<00:30, 2710240.25tokens/s]
Shard 36:  17%|█▋        | 17072028/100000000 [00:05<00:28, 2929205.83tokens/s]
Shard 36:  17%|█▋        | 17468197/100000000 [00:05<00:25, 3200237.63tokens/s]
Shard 36:  18%|█▊        | 17865222/100000000 [00:05<00:24, 3389559.19tokens/s]
Shard 36:  18%|█▊        | 18217155/100000000 [00:05<00:26, 3059447.97tokens/s]
Shard 36:  19%|█▊        | 18535611/100000000 [00:05<00:31, 2589534.73tokens/s]
Shard 36:  19%|█▉        | 18813614/100000000 [00:05<00:32, 2507983.58tokens/s]
Shard 36:  19%|█▉        | 19077394/100000000 [00:05<00:32, 2478658.13tokens/s]
Shard 36:  19%|█▉        | 19333871/100000000 [00:05<00:32, 2484092.53tokens/s]
Shard 36:  20%|█▉        | 19592522/100000000 [00:06<00:32, 2437694.22tokens/s]
Shard 36:  20%|█▉        | 19840819/100000000 [00:06<00:33, 2367998.43tokens/s]
Shard 36:  20%|██        | 20084214/100000000 [00:06<00:34, 2324492.98tokens/s]
Shard 36:  20%|██        | 20351689/100000000 [00:06<00:33, 2412884.54tokens/s]
Shard 36:  21%|██        | 20595476/100000000 [00:06<00:33, 2351791.89tokens/s]
Shard 36:  21%|██        | 20832429/100000000 [00:06<00:36, 2173023.04tokens/s]
Shard 36:  21%|██        | 21053377/100000000 [00:06<00:36, 2179524.36tokens/s]
Shard 36:  21%|██▏       | 21321012/100000000 [00:06<00:34, 2307617.11tokens/s]
Shard 36:  22%|██▏       | 21600680/100000000 [00:06<00:32, 2431472.68tokens/s]
Shard 36:  22%|██▏       | 21881733/100000000 [00:07<00:30, 2538818.96tokens/s]
Shard 36:  22%|██▏       | 22173756/100000000 [00:07<00:29, 2633412.39tokens/s]
Shard 36:  23%|██▎       | 22664772/100000000 [00:07<00:23, 3298100.44tokens/s]
Shard 36:  23%|██▎       | 22997671/100000000 [00:07<00:23, 3305786.18tokens/s]
Shard 36:  23%|██▎       | 23330593/100000000 [00:07<00:24, 3152395.90tokens/s]
Shard 36:  24%|██▎       | 23674840/100000000 [00:07<00:23, 3215640.26tokens/s]
Shard 36:  24%|██▍       | 24164526/100000000 [00:07<00:20, 3701234.81tokens/s]
Shard 36:  25%|██▍       | 24538678/100000000 [00:07<00:21, 3568014.27tokens/s]
Shard 36:  25%|██▌       | 25012880/100000000 [00:07<00:19, 3897724.12tokens/s]
Shard 36:  26%|██▌       | 25530980/100000000 [00:08<00:17, 4265605.45tokens/s]
Shard 36:  26%|██▌       | 25978621/100000000 [00:08<00:17, 4320252.95tokens/s]
Shard 36:  26%|██▋       | 26436623/100000000 [00:08<00:16, 4387725.64tokens/s]
Shard 36:  27%|██▋       | 26908818/100000000 [00:08<00:16, 4452444.44tokens/s]
Shard 36:  27%|██▋       | 27405082/100000000 [00:08<00:15, 4589628.87tokens/s]
Shard 36:  28%|██▊       | 27889591/100000000 [00:08<00:15, 4659082.89tokens/s]
Shard 36:  28%|██▊       | 28361714/100000000 [00:08<00:15, 4675546.80tokens/s]
Shard 36:  29%|██▉       | 28830062/100000000 [00:08<00:16, 4336395.18tokens/s]
Shard 36:  29%|██▉       | 29346088/100000000 [00:08<00:15, 4559159.31tokens/s]
Shard 36:  30%|██▉       | 29813006/100000000 [00:08<00:15, 4576801.22tokens/s]
Shard 36:  30%|███       | 30276052/100000000 [00:09<00:17, 3956675.76tokens/s]
Shard 36:  31%|███       | 30715784/100000000 [00:09<00:17, 4049937.21tokens/s]
Shard 36:  31%|███       | 31133783/100000000 [00:09<00:17, 3997924.08tokens/s]
Shard 36:  32%|███▏      | 31798967/100000000 [00:09<00:14, 4705417.52tokens/s]
Shard 36:  32%|███▏      | 32283646/100000000 [00:09<00:15, 4265170.54tokens/s]
Shard 36:  33%|███▎      | 32752309/100000000 [00:09<00:15, 4344065.03tokens/s]
Shard 36:  33%|███▎      | 33198317/100000000 [00:09<00:15, 4274767.20tokens/s]
Shard 36:  34%|███▎      | 33743920/100000000 [00:09<00:14, 4582831.98tokens/s]
Shard 36:  34%|███▍      | 34212150/100000000 [00:09<00:14, 4599061.55tokens/s]
Shard 36:  35%|███▍      | 34678266/100000000 [00:10<00:14, 4574501.85tokens/s]
Shard 36:  35%|███▌      | 35139522/100000000 [00:10<00:14, 4550307.66tokens/s]
Shard 36:  36%|███▌      | 35598411/100000000 [00:10<00:15, 4165224.14tokens/s]
Shard 36:  36%|███▌      | 36169494/100000000 [00:10<00:13, 4587131.48tokens/s]
Shard 36:  37%|███▋      | 36637697/100000000 [00:10<00:15, 4099580.73tokens/s]
Shard 36:  37%|███▋      | 37135353/100000000 [00:10<00:14, 4319778.20tokens/s]
Shard 36:  38%|███▊      | 37580702/100000000 [00:10<00:15, 4145086.79tokens/s]
Shard 36:  38%|███▊      | 38273402/100000000 [00:10<00:12, 4766738.36tokens/s]
Shard 36:  39%|███▉      | 38905866/100000000 [00:10<00:11, 5143330.94tokens/s]
Shard 36:  40%|███▉      | 39548174/100000000 [00:11<00:10, 5500149.48tokens/s]
Shard 36:  40%|████      | 40107547/100000000 [00:11<00:10, 5450926.82tokens/s]
Shard 36:  41%|████      | 40659566/100000000 [00:11<00:10, 5441277.48tokens/s]
Shard 36:  41%|████      | 41207965/100000000 [00:11<00:12, 4752653.12tokens/s]
Shard 36:  42%|████▏     | 41701141/100000000 [00:11<00:12, 4494648.10tokens/s]
Shard 36:  42%|████▏     | 42164076/100000000 [00:11<00:13, 4308251.18tokens/s]
Shard 36:  43%|████▎     | 42653787/100000000 [00:11<00:12, 4436624.83tokens/s]
Shard 36:  43%|████▎     | 43106192/100000000 [00:11<00:12, 4458449.63tokens/s]
Shard 36:  44%|████▎     | 43656891/100000000 [00:11<00:11, 4743606.78tokens/s]
Shard 36:  44%|████▍     | 44137743/100000000 [00:12<00:13, 4137664.13tokens/s]
Shard 36:  45%|████▍     | 44568826/100000000 [00:12<00:13, 4102963.23tokens/s]
Shard 36:  45%|████▌     | 45071160/100000000 [00:12<00:12, 4324237.49tokens/s]
Shard 36:  46%|████▌     | 45579011/100000000 [00:12<00:12, 4525095.92tokens/s]
Shard 36:  46%|████▌     | 46245950/100000000 [00:12<00:10, 5027595.23tokens/s]
Shard 36:  47%|████▋     | 46757127/100000000 [00:12<00:11, 4735733.57tokens/s]
Shard 36:  47%|████▋     | 47239118/100000000 [00:12<00:11, 4492521.46tokens/s]
Shard 36:  48%|████▊     | 47715551/100000000 [00:12<00:11, 4545873.41tokens/s]
Shard 36:  48%|████▊     | 48193175/100000000 [00:12<00:11, 4609128.77tokens/s]
Shard 36:  49%|████▊     | 48712980/100000000 [00:13<00:10, 4760716.72tokens/s]
Shard 36:  49%|████▉     | 49270226/100000000 [00:13<00:10, 4977609.33tokens/s]
Shard 36:  50%|████▉     | 49774747/100000000 [00:13<00:11, 4211699.11tokens/s]
Shard 36:  50%|█████     | 50217757/100000000 [00:13<00:11, 4169142.84tokens/s]
Shard 36:  51%|█████     | 50650134/100000000 [00:13<00:12, 4022116.50tokens/s]
Shard 36:  52%|█████▏    | 51781624/100000000 [00:13<00:08, 5950086.23tokens/s]
Shard 36:  52%|█████▏    | 52426676/100000000 [00:13<00:07, 6073571.02tokens/s]
Shard 36:  53%|█████▎    | 53056740/100000000 [00:13<00:08, 5395081.05tokens/s]
Shard 36:  54%|█████▎    | 53625446/100000000 [00:14<00:09, 5130573.57tokens/s]
Shard 36:  54%|█████▍    | 54159335/100000000 [00:14<00:09, 5070111.27tokens/s]
Shard 36:  55%|█████▍    | 54690605/100000000 [00:14<00:08, 5074398.55tokens/s]
Shard 36:  55%|█████▌    | 55208100/100000000 [00:14<00:08, 5082370.41tokens/s]
Shard 36:  56%|█████▌    | 55724190/100000000 [00:14<00:09, 4840091.78tokens/s]
Shard 36:  56%|█████▌    | 56230651/100000000 [00:14<00:08, 4882289.27tokens/s]
Shard 36:  57%|█████▋    | 56723588/100000000 [00:14<00:08, 4842343.41tokens/s]
Shard 36:  57%|█████▋    | 57210982/100000000 [00:14<00:09, 4562203.19tokens/s]
Shard 36:  58%|█████▊    | 57674757/100000000 [00:14<00:09, 4572432.90tokens/s]
Shard 36:  58%|█████▊    | 58232669/100000000 [00:15<00:08, 4819753.61tokens/s]
Shard 36:  59%|█████▊    | 58717911/100000000 [00:15<00:09, 4506678.54tokens/s]
Shard 36:  59%|█████▉    | 59262313/100000000 [00:15<00:08, 4764922.90tokens/s]
Shard 36:  60%|█████▉    | 59744916/100000000 [00:15<00:08, 4627571.10tokens/s]
Shard 36:  60%|██████    | 60438723/100000000 [00:15<00:07, 5256876.67tokens/s]
Shard 36:  61%|██████    | 60972903/100000000 [00:15<00:07, 5185518.44tokens/s]
Shard 36:  62%|██████▏   | 61636917/100000000 [00:15<00:06, 5601458.77tokens/s]
Shard 36:  62%|██████▏   | 62202720/100000000 [00:15<00:06, 5565237.84tokens/s]
Shard 36:  63%|██████▎   | 62855245/100000000 [00:15<00:06, 5829705.75tokens/s]
Shard 36:  63%|██████▎   | 63447470/100000000 [00:16<00:07, 5117843.13tokens/s]
Shard 36:  64%|██████▍   | 63978370/100000000 [00:16<00:08, 4420171.02tokens/s]
Shard 36:  65%|██████▍   | 64561532/100000000 [00:16<00:07, 4763291.38tokens/s]
Shard 36:  65%|██████▌   | 65154202/100000000 [00:16<00:06, 5048153.67tokens/s]
Shard 36:  66%|██████▌   | 65681765/100000000 [00:16<00:07, 4849930.70tokens/s]
Shard 36:  66%|██████▌   | 66187628/100000000 [00:16<00:07, 4441684.82tokens/s]
Shard 36:  67%|██████▋   | 66647420/100000000 [00:16<00:07, 4170455.30tokens/s]
Shard 36:  67%|██████▋   | 67076788/100000000 [00:16<00:07, 4120545.08tokens/s]
Shard 36:  67%|██████▋   | 67496896/100000000 [00:17<00:08, 3897544.49tokens/s]
Shard 36:  68%|██████▊   | 67922110/100000000 [00:17<00:08, 3982869.27tokens/s]
Shard 36:  68%|██████▊   | 68325905/100000000 [00:17<00:08, 3679243.93tokens/s]
Shard 36:  69%|██████▊   | 68724538/100000000 [00:17<00:08, 3750248.21tokens/s]
Shard 36:  69%|██████▉   | 69110502/100000000 [00:17<00:08, 3751786.69tokens/s]
Shard 36:  70%|██████▉   | 69727780/100000000 [00:17<00:06, 4426317.15tokens/s]
Shard 36:  70%|███████   | 70242476/100000000 [00:17<00:06, 4623234.96tokens/s]
Shard 36:  71%|███████   | 70907020/100000000 [00:17<00:05, 5174918.78tokens/s]
Shard 36:  71%|███████▏  | 71430861/100000000 [00:17<00:05, 5138281.38tokens/s]
Shard 36:  72%|███████▏  | 71950838/100000000 [00:18<00:06, 4541862.18tokens/s]
Shard 36:  72%|███████▏  | 72420743/100000000 [00:18<00:06, 4105328.59tokens/s]
Shard 36:  73%|███████▎  | 72848452/100000000 [00:18<00:06, 4077235.16tokens/s]
Shard 36:  73%|███████▎  | 73267357/100000000 [00:18<00:07, 3468318.05tokens/s]
Shard 36:  74%|███████▎  | 73649398/100000000 [00:18<00:07, 3550420.93tokens/s]
Shard 36:  74%|███████▍  | 74050956/100000000 [00:18<00:07, 3631044.08tokens/s]
Shard 36:  74%|███████▍  | 74426970/100000000 [00:18<00:07, 3457746.37tokens/s]
Shard 36:  75%|███████▍  | 74882812/100000000 [00:18<00:06, 3700085.72tokens/s]
Shard 36:  75%|███████▌  | 75315807/100000000 [00:18<00:06, 3836151.95tokens/s]
Shard 36:  76%|███████▌  | 75777902/100000000 [00:19<00:05, 4041078.20tokens/s]
Shard 36:  76%|███████▌  | 76188833/100000000 [00:19<00:06, 3723661.85tokens/s]
Shard 36:  77%|███████▋  | 76570667/100000000 [00:19<00:06, 3506210.36tokens/s]
Shard 36:  77%|███████▋  | 76928530/100000000 [00:19<00:07, 3165786.44tokens/s]
Shard 36:  77%|███████▋  | 77261742/100000000 [00:19<00:07, 3204450.97tokens/s]
Shard 36:  78%|███████▊  | 77592219/100000000 [00:19<00:06, 3230008.69tokens/s]
Shard 36:  78%|███████▊  | 77963063/100000000 [00:19<00:06, 3342065.74tokens/s]
Shard 36:  78%|███████▊  | 78377963/100000000 [00:19<00:06, 3556785.05tokens/s]
Shard 36:  79%|███████▊  | 78738960/100000000 [00:19<00:06, 3325047.54tokens/s]
Shard 36:  79%|███████▉  | 79153282/100000000 [00:20<00:05, 3547656.42tokens/s]
Shard 36:  80%|███████▉  | 79533695/100000000 [00:20<00:05, 3619655.85tokens/s]
Shard 36:  80%|███████▉  | 79900308/100000000 [00:20<00:05, 3375870.90tokens/s]
Shard 36:  80%|████████  | 80383893/100000000 [00:20<00:05, 3733656.96tokens/s]
Shard 36:  81%|████████  | 80802218/100000000 [00:20<00:05, 3775933.34tokens/s]
Shard 36:  81%|████████  | 81184152/100000000 [00:20<00:05, 3222181.25tokens/s]
Shard 36:  82%|████████▏ | 81524394/100000000 [00:20<00:05, 3113727.00tokens/s]
Shard 36:  82%|████████▏ | 82007469/100000000 [00:20<00:05, 3545001.63tokens/s]
Shard 36:  82%|████████▏ | 82376231/100000000 [00:21<00:05, 3489259.84tokens/s]
Shard 36:  83%|████████▎ | 82737416/100000000 [00:21<00:05, 3426676.04tokens/s]
Shard 36:  83%|████████▎ | 83134199/100000000 [00:21<00:04, 3549118.20tokens/s]
Shard 36:  83%|████████▎ | 83494731/100000000 [00:21<00:04, 3520167.41tokens/s]
Shard 36:  84%|████████▍ | 83850762/100000000 [00:21<00:04, 3384647.13tokens/s]
Shard 36:  84%|████████▍ | 84196040/100000000 [00:21<00:05, 3072397.29tokens/s]
Shard 36:  85%|████████▍ | 84532177/100000000 [00:21<00:04, 3146469.71tokens/s]
Shard 36:  85%|████████▍ | 84852748/100000000 [00:21<00:05, 2705776.83tokens/s]
Shard 36:  85%|████████▌ | 85136851/100000000 [00:21<00:05, 2491501.35tokens/s]
Shard 36:  85%|████████▌ | 85396556/100000000 [00:22<00:06, 2404858.19tokens/s]
Shard 36:  86%|████████▌ | 85643805/100000000 [00:22<00:06, 2195977.12tokens/s]
Shard 36:  86%|████████▌ | 85877686/100000000 [00:22<00:06, 2057043.15tokens/s]
Shard 36:  86%|████████▌ | 86089884/100000000 [00:22<00:07, 1817267.64tokens/s]
Shard 36:  86%|████████▋ | 86375699/100000000 [00:22<00:06, 2055222.20tokens/s]
Shard 36:  87%|████████▋ | 86799077/100000000 [00:22<00:05, 2600017.23tokens/s]
Shard 36:  87%|████████▋ | 87077252/100000000 [00:22<00:04, 2620219.51tokens/s]
Shard 36:  87%|████████▋ | 87417917/100000000 [00:22<00:04, 2831943.10tokens/s]
Shard 36:  88%|████████▊ | 87715861/100000000 [00:23<00:04, 2785617.69tokens/s]
Shard 36:  88%|████████▊ | 88044415/100000000 [00:23<00:04, 2917749.99tokens/s]
Shard 36:  88%|████████▊ | 88391905/100000000 [00:23<00:03, 3069601.17tokens/s]
Shard 36:  89%|████████▊ | 88704314/100000000 [00:23<00:03, 3057550.68tokens/s]
Shard 36:  89%|████████▉ | 89105967/100000000 [00:23<00:03, 3335503.70tokens/s]
Shard 36:  90%|████████▉ | 89779406/100000000 [00:23<00:02, 4322360.94tokens/s]
Shard 36:  90%|█████████ | 90216254/100000000 [00:23<00:02, 3653928.67tokens/s]
Shard 36:  91%|█████████ | 90602233/100000000 [00:23<00:02, 3523717.37tokens/s]
Shard 36:  91%|█████████ | 91085196/100000000 [00:23<00:02, 3858399.68tokens/s]
Shard 36:  92%|█████████▏| 91520773/100000000 [00:24<00:02, 3990934.12tokens/s]
Shard 36:  92%|█████████▏| 91964885/100000000 [00:24<00:01, 4040942.48tokens/s]
Shard 36:  92%|█████████▏| 92479848/100000000 [00:24<00:01, 4352505.77tokens/s]
Shard 36:  93%|█████████▎| 92925795/100000000 [00:24<00:01, 4176189.02tokens/s]
Shard 36:  93%|█████████▎| 93383874/100000000 [00:24<00:01, 4284488.77tokens/s]
Shard 36:  94%|█████████▍| 93817797/100000000 [00:24<00:01, 3883145.23tokens/s]
Shard 36:  94%|█████████▍| 94216169/100000000 [00:24<00:01, 3561658.11tokens/s]
Shard 36:  95%|█████████▍| 94583843/100000000 [00:24<00:01, 3414103.09tokens/s]
Shard 36:  95%|█████████▍| 94933472/100000000 [00:24<00:01, 3244315.95tokens/s]
Shard 36:  96%|█████████▌| 95584435/100000000 [00:25<00:01, 4093436.76tokens/s]
Shard 36:  96%|█████████▌| 96026695/100000000 [00:25<00:00, 4176118.29tokens/s]
Shard 36:  97%|█████████▋| 96515798/100000000 [00:25<00:00, 4358335.83tokens/s]
Shard 36:  97%|█████████▋| 96961539/100000000 [00:25<00:00, 4329767.88tokens/s]
Shard 36:  97%|█████████▋| 97402034/100000000 [00:25<00:00, 3977582.29tokens/s]
Shard 36:  98%|█████████▊| 97810289/100000000 [00:25<00:00, 3803384.10tokens/s]
Shard 36:  98%|█████████▊| 98197974/100000000 [00:25<00:00, 3684469.98tokens/s]
Shard 36:  99%|█████████▊| 98571429/100000000 [00:25<00:00, 3549580.78tokens/s]
Shard 36:  99%|█████████▉| 98929983/100000000 [00:26<00:00, 3157619.10tokens/s]
Shard 36:  99%|█████████▉| 99293656/100000000 [00:26<00:00, 3279913.08tokens/s]
Shard 36: 100%|█████████▉| 99632385/100000000 [00:26<00:00, 3305956.62tokens/s]
Shard 36: 100%|█████████▉| 99968980/100000000 [00:26<00:00, 3088962.54tokens/s]
Shard 36: 100%|█████████▉| 99996419/100000000 [00:34<00:00, 2889208.91tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000036.bin (200,001,024 bytes) in the gpt-2 format

Shard 37:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 37:   0%|          | 260300/100000000 [00:00<00:38, 2601830.07tokens/s]
Shard 37:   1%|          | 521034/100000000 [00:00<00:39, 2520053.00tokens/s]
Shard 37:   1%|          | 791698/100000000 [00:00<00:38, 2581623.07tokens/s]
Shard 37:   1%|          | 1050097/100000000 [00:00<00:41, 2381641.54tokens/s]
Shard 37:   1%|▏         | 1337638/100000000 [00:00<00:38, 2542704.09tokens/s]
Shard 37:   2%|▏         | 1594880/100000000 [00:00<00:40, 2401887.77tokens/s]
Shard 37:   2%|▏         | 2168093/100000000 [00:00<00:28, 3399541.02tokens/s]
Shard 37:   3%|▎         | 2521889/100000000 [00:00<00:29, 3294923.20tokens/s]
Shard 37:   3%|▎         | 2857229/100000000 [00:00<00:29, 3261385.64tokens/s]
Shard 37:   3%|▎         | 3230170/100000000 [00:01<00:28, 3396795.27tokens/s]
Shard 37:   4%|▎         | 3599818/100000000 [00:01<00:27, 3444612.01tokens/s]
Shard 37:   4%|▍         | 3947112/100000000 [00:01<00:30, 3154865.09tokens/s]
Shard 37:   4%|▍         | 4269787/100000000 [00:01<00:30, 3128443.86tokens/s]
Shard 37:   5%|▍         | 4586882/100000000 [00:01<00:31, 3025117.50tokens/s]
Shard 37:   5%|▌         | 5030861/100000000 [00:01<00:28, 3383931.96tokens/s]
Shard 37:   5%|▌         | 5406788/100000000 [00:01<00:27, 3417023.33tokens/s]
Shard 37:   6%|▋         | 6256285/100000000 [00:01<00:19, 4849367.28tokens/s]
Shard 37:   7%|▋         | 6846483/100000000 [00:01<00:18, 5093541.33tokens/s]
Shard 37:   7%|▋         | 7376360/100000000 [00:02<00:21, 4346876.98tokens/s]
Shard 37:   8%|▊         | 7835086/100000000 [00:02<00:21, 4310756.26tokens/s]
Shard 37:   9%|▊         | 8663452/100000000 [00:02<00:17, 5352260.73tokens/s]
Shard 37:   9%|▉         | 9228140/100000000 [00:02<00:17, 5207691.98tokens/s]
Shard 37:  10%|▉         | 9766452/100000000 [00:02<00:17, 5212689.86tokens/s]
Shard 37:  10%|█         | 10300335/100000000 [00:02<00:20, 4458060.34tokens/s]
Shard 37:  11%|█         | 10771999/100000000 [00:02<00:21, 4072160.59tokens/s]
Shard 37:  11%|█         | 11201317/100000000 [00:02<00:21, 4083399.16tokens/s]
Shard 37:  12%|█▏        | 11624656/100000000 [00:03<00:21, 4051411.27tokens/s]
Shard 37:  12%|█▏        | 12040418/100000000 [00:03<00:25, 3514529.49tokens/s]
Shard 37:  12%|█▏        | 12408839/100000000 [00:03<00:29, 2994073.14tokens/s]
Shard 37:  13%|█▎        | 12729903/100000000 [00:03<00:29, 2990087.53tokens/s]
Shard 37:  13%|█▎        | 13105906/100000000 [00:03<00:27, 3152144.99tokens/s]
Shard 37:  13%|█▎        | 13480727/100000000 [00:03<00:26, 3295544.46tokens/s]
Shard 37:  14%|█▍        | 13822419/100000000 [00:03<00:26, 3286150.25tokens/s]
Shard 37:  14%|█▍        | 14187584/100000000 [00:03<00:25, 3377643.58tokens/s]
Shard 37:  15%|█▍        | 14533328/100000000 [00:04<00:25, 3350551.97tokens/s]
Shard 37:  15%|█▍        | 14935873/100000000 [00:04<00:24, 3538881.88tokens/s]
Shard 37:  15%|█▌        | 15384345/100000000 [00:04<00:22, 3796946.05tokens/s]
Shard 37:  16%|█▌        | 15767940/100000000 [00:04<00:24, 3428933.47tokens/s]
Shard 37:  16%|█▌        | 16119798/100000000 [00:04<00:24, 3443347.31tokens/s]
Shard 37:  17%|█▋        | 16588799/100000000 [00:04<00:22, 3779976.35tokens/s]
Shard 37:  17%|█▋        | 16974311/100000000 [00:04<00:22, 3732745.62tokens/s]
Shard 37:  17%|█▋        | 17356623/100000000 [00:04<00:22, 3754563.81tokens/s]
Shard 37:  18%|█▊        | 17736156/100000000 [00:04<00:22, 3701310.96tokens/s]
Shard 37:  18%|█▊        | 18108725/100000000 [00:05<00:24, 3377705.71tokens/s]
Shard 37:  18%|█▊        | 18453492/100000000 [00:05<00:25, 3244818.41tokens/s]
Shard 37:  19%|█▉        | 18783531/100000000 [00:05<00:26, 3020617.88tokens/s]
Shard 37:  19%|█▉        | 19091041/100000000 [00:05<00:27, 2909546.87tokens/s]
Shard 37:  19%|█▉        | 19385588/100000000 [00:05<00:27, 2914420.63tokens/s]
Shard 37:  20%|█▉        | 19708564/100000000 [00:05<00:26, 2986647.72tokens/s]
Shard 37:  20%|██        | 20009698/100000000 [00:05<00:27, 2889408.08tokens/s]
Shard 37:  20%|██        | 20301396/100000000 [00:05<00:27, 2860601.22tokens/s]
Shard 37:  21%|██        | 20690654/100000000 [00:05<00:25, 3147577.90tokens/s]
Shard 37:  21%|██        | 21084878/100000000 [00:05<00:23, 3374720.93tokens/s]
Shard 37:  22%|██▏       | 21532963/100000000 [00:06<00:21, 3672948.42tokens/s]
Shard 37:  23%|██▎       | 22675612/100000000 [00:06<00:13, 5937096.85tokens/s]
Shard 37:  23%|██▎       | 23277836/100000000 [00:06<00:16, 4623636.50tokens/s]
Shard 37:  24%|██▍       | 23790607/100000000 [00:06<00:16, 4555400.41tokens/s]
Shard 37:  24%|██▍       | 24280308/100000000 [00:06<00:19, 3969333.87tokens/s]
Shard 37:  25%|██▍       | 24712375/100000000 [00:06<00:24, 3044959.13tokens/s]
Shard 37:  25%|██▌       | 25068499/100000000 [00:07<00:24, 3050431.06tokens/s]
Shard 37:  25%|██▌       | 25409402/100000000 [00:07<00:24, 3006424.35tokens/s]
Shard 37:  26%|██▌       | 25734215/100000000 [00:07<00:25, 2905269.95tokens/s]
Shard 37:  26%|██▌       | 26053934/100000000 [00:07<00:24, 2965938.58tokens/s]
Shard 37:  26%|██▋       | 26363522/100000000 [00:07<00:28, 2621268.35tokens/s]
Shard 37:  27%|██▋       | 26781917/100000000 [00:07<00:24, 2992807.82tokens/s]
Shard 37:  27%|██▋       | 27156532/100000000 [00:07<00:23, 3155194.01tokens/s]
Shard 37:  27%|██▋       | 27487973/100000000 [00:07<00:23, 3032412.29tokens/s]
Shard 37:  28%|██▊       | 27803268/100000000 [00:07<00:25, 2856893.14tokens/s]
Shard 37:  28%|██▊       | 28150604/100000000 [00:08<00:23, 3008591.42tokens/s]
Shard 37:  28%|██▊       | 28464081/100000000 [00:08<00:25, 2814974.65tokens/s]
Shard 37:  29%|██▉       | 28867262/100000000 [00:08<00:22, 3116411.37tokens/s]
Shard 37:  29%|██▉       | 29200094/100000000 [00:08<00:22, 3161376.90tokens/s]
Shard 37:  30%|██▉       | 29523427/100000000 [00:08<00:22, 3162003.63tokens/s]
Shard 37:  30%|███       | 30074662/100000000 [00:08<00:18, 3825029.73tokens/s]
Shard 37:  30%|███       | 30463912/100000000 [00:08<00:18, 3695670.13tokens/s]
Shard 37:  31%|███       | 30839634/100000000 [00:08<00:20, 3361268.82tokens/s]
Shard 37:  31%|███       | 31190383/100000000 [00:08<00:21, 3241788.18tokens/s]
Shard 37:  32%|███▏      | 31890447/100000000 [00:09<00:16, 4238100.63tokens/s]
Shard 37:  32%|███▏      | 32330572/100000000 [00:09<00:16, 4022617.88tokens/s]
Shard 37:  33%|███▎      | 32753424/100000000 [00:09<00:16, 4047677.03tokens/s]
Shard 37:  33%|███▎      | 33167069/100000000 [00:09<00:16, 3981095.64tokens/s]
Shard 37:  34%|███▎      | 33571355/100000000 [00:09<00:17, 3833175.69tokens/s]
Shard 37:  34%|███▍      | 33959873/100000000 [00:09<00:19, 3451294.22tokens/s]
Shard 37:  34%|███▍      | 34321448/100000000 [00:09<00:20, 3235667.35tokens/s]
Shard 37:  35%|███▍      | 34654020/100000000 [00:09<00:20, 3239163.84tokens/s]
Shard 37:  35%|███▌      | 35017162/100000000 [00:10<00:19, 3333885.22tokens/s]
Shard 37:  35%|███▌      | 35355490/100000000 [00:10<00:19, 3299961.30tokens/s]
Shard 37:  36%|███▌      | 35877305/100000000 [00:10<00:16, 3819945.97tokens/s]
Shard 37:  36%|███▋      | 36325408/100000000 [00:10<00:15, 4003715.94tokens/s]
Shard 37:  37%|███▋      | 36787702/100000000 [00:10<00:15, 4182056.81tokens/s]
Shard 37:  37%|███▋      | 37210388/100000000 [00:10<00:15, 4154503.80tokens/s]
Shard 37:  38%|███▊      | 37628853/100000000 [00:10<00:15, 4057648.86tokens/s]
Shard 37:  38%|███▊      | 38067248/100000000 [00:10<00:15, 4033856.08tokens/s]
Shard 37:  38%|███▊      | 38472793/100000000 [00:10<00:17, 3518078.68tokens/s]
Shard 37:  39%|███▉      | 38836881/100000000 [00:11<00:17, 3426311.52tokens/s]
Shard 37:  39%|███▉      | 39299922/100000000 [00:11<00:16, 3720606.31tokens/s]
Shard 37:  40%|███▉      | 39682423/100000000 [00:11<00:16, 3623110.64tokens/s]
Shard 37:  40%|████      | 40083128/100000000 [00:11<00:16, 3723157.70tokens/s]
Shard 37:  40%|████      | 40460685/100000000 [00:11<00:16, 3618500.72tokens/s]
Shard 37:  41%|████      | 40879321/100000000 [00:11<00:15, 3776025.27tokens/s]
Shard 37:  41%|████▏     | 41276734/100000000 [00:11<00:15, 3832275.88tokens/s]
Shard 37:  42%|████▏     | 41663576/100000000 [00:11<00:15, 3736365.25tokens/s]
Shard 37:  42%|████▏     | 42039831/100000000 [00:11<00:16, 3527621.92tokens/s]
Shard 37:  42%|████▏     | 42398238/100000000 [00:11<00:16, 3533762.54tokens/s]
Shard 37:  43%|████▎     | 42808493/100000000 [00:12<00:15, 3674921.36tokens/s]
Shard 37:  43%|████▎     | 43234840/100000000 [00:12<00:14, 3842027.66tokens/s]
Shard 37:  44%|████▎     | 43621802/100000000 [00:12<00:15, 3603484.57tokens/s]
Shard 37:  44%|████▍     | 44034447/100000000 [00:12<00:14, 3747344.06tokens/s]
Shard 37:  44%|████▍     | 44413263/100000000 [00:12<00:15, 3568479.43tokens/s]
Shard 37:  45%|████▍     | 44774232/100000000 [00:12<00:16, 3365425.94tokens/s]
Shard 37:  45%|████▌     | 45115253/100000000 [00:12<00:18, 2948643.89tokens/s]
Shard 37:  45%|████▌     | 45426551/100000000 [00:12<00:18, 2968046.10tokens/s]
Shard 37:  46%|████▌     | 45731153/100000000 [00:13<00:19, 2764882.36tokens/s]
Shard 37:  46%|████▌     | 46014749/100000000 [00:13<00:22, 2431873.69tokens/s]
Shard 37:  46%|████▋     | 46267599/100000000 [00:13<00:24, 2236690.84tokens/s]
Shard 37:  47%|████▋     | 46503662/100000000 [00:13<00:26, 2028176.03tokens/s]
Shard 37:  47%|████▋     | 46713198/100000000 [00:13<00:26, 1978438.89tokens/s]
Shard 37:  47%|████▋     | 46915003/100000000 [00:13<00:27, 1965747.46tokens/s]
Shard 37:  47%|████▋     | 47114054/100000000 [00:13<00:28, 1867231.97tokens/s]
Shard 37:  47%|████▋     | 47308540/100000000 [00:13<00:28, 1861214.69tokens/s]
Shard 37:  48%|████▊     | 47515393/100000000 [00:14<00:27, 1915617.38tokens/s]
Shard 37:  48%|████▊     | 47782552/100000000 [00:14<00:24, 2124110.30tokens/s]
Shard 37:  48%|████▊     | 47999027/100000000 [00:14<00:24, 2082642.89tokens/s]
Shard 37:  48%|████▊     | 48209825/100000000 [00:14<00:25, 2038100.84tokens/s]
Shard 37:  48%|████▊     | 48460363/100000000 [00:14<00:23, 2159058.41tokens/s]
Shard 37:  49%|████▉     | 48761298/100000000 [00:14<00:21, 2367110.36tokens/s]
Shard 37:  49%|████▉     | 49174103/100000000 [00:14<00:17, 2869551.64tokens/s]
Shard 37:  50%|████▉     | 49602472/100000000 [00:14<00:15, 3268248.50tokens/s]
Shard 37:  50%|████▉     | 49970769/100000000 [00:14<00:15, 3333686.19tokens/s]
Shard 37:  50%|█████     | 50331704/100000000 [00:14<00:14, 3413282.25tokens/s]
Shard 37:  51%|█████     | 50755282/100000000 [00:15<00:13, 3644067.23tokens/s]
Shard 37:  51%|█████     | 51121918/100000000 [00:15<00:13, 3650512.59tokens/s]
Shard 37:  51%|█████▏    | 51488182/100000000 [00:15<00:14, 3307452.00tokens/s]
Shard 37:  52%|█████▏    | 51882747/100000000 [00:15<00:13, 3474142.87tokens/s]
Shard 37:  52%|█████▏    | 52344436/100000000 [00:15<00:12, 3795753.48tokens/s]
Shard 37:  53%|█████▎    | 52730045/100000000 [00:15<00:12, 3737418.37tokens/s]
Shard 37:  53%|█████▎    | 53109244/100000000 [00:15<00:12, 3632300.04tokens/s]
Shard 37:  53%|█████▎    | 53476114/100000000 [00:15<00:13, 3364916.71tokens/s]
Shard 37:  54%|█████▍    | 53877619/100000000 [00:15<00:13, 3521145.94tokens/s]
Shard 37:  54%|█████▍    | 54236596/100000000 [00:16<00:13, 3304848.48tokens/s]
Shard 37:  55%|█████▍    | 54572753/100000000 [00:16<00:14, 3089322.09tokens/s]
Shard 37:  55%|█████▍    | 54888123/100000000 [00:16<00:15, 2937482.09tokens/s]
Shard 37:  55%|█████▌    | 55185901/100000000 [00:16<00:16, 2657305.88tokens/s]
Shard 37:  56%|█████▌    | 55504166/100000000 [00:16<00:16, 2780960.01tokens/s]
Shard 37:  56%|█████▌    | 55795814/100000000 [00:16<00:15, 2814413.32tokens/s]
Shard 37:  56%|█████▌    | 56082060/100000000 [00:16<00:15, 2826791.44tokens/s]
Shard 37:  56%|█████▋    | 56448529/100000000 [00:16<00:14, 2995588.36tokens/s]
Shard 37:  57%|█████▋    | 56751288/100000000 [00:16<00:14, 2970948.24tokens/s]
Shard 37:  57%|█████▋    | 57079983/100000000 [00:17<00:14, 3050370.01tokens/s]
Shard 37:  57%|█████▋    | 57444512/100000000 [00:17<00:13, 3206354.02tokens/s]
Shard 37:  58%|█████▊    | 57766873/100000000 [00:17<00:14, 2912815.55tokens/s]
Shard 37:  58%|█████▊    | 58065063/100000000 [00:17<00:14, 2877980.04tokens/s]
Shard 37:  58%|█████▊    | 58415378/100000000 [00:17<00:13, 3049712.24tokens/s]
Shard 37:  59%|█████▉    | 58763874/100000000 [00:17<00:13, 3171452.67tokens/s]
Shard 37:  59%|█████▉    | 59084620/100000000 [00:17<00:13, 2999783.32tokens/s]
Shard 37:  59%|█████▉    | 59389036/100000000 [00:17<00:13, 2985046.30tokens/s]
Shard 37:  60%|█████▉    | 59708715/100000000 [00:17<00:13, 3042261.79tokens/s]
Shard 37:  60%|██████    | 60114117/100000000 [00:18<00:12, 3305218.81tokens/s]
Shard 37:  60%|██████    | 60448149/100000000 [00:18<00:12, 3110177.46tokens/s]
Shard 37:  61%|██████    | 60762786/100000000 [00:18<00:13, 2961597.78tokens/s]
Shard 37:  61%|██████    | 61074391/100000000 [00:18<00:12, 3003754.14tokens/s]
Shard 37:  61%|██████▏   | 61384770/100000000 [00:18<00:12, 3028779.70tokens/s]
Shard 37:  62%|██████▏   | 61690526/100000000 [00:18<00:13, 2788563.84tokens/s]
Shard 37:  62%|██████▏   | 61973952/100000000 [00:18<00:13, 2769613.91tokens/s]
Shard 37:  62%|██████▏   | 62391211/100000000 [00:18<00:11, 3160427.61tokens/s]
Shard 37:  63%|██████▎   | 63258797/100000000 [00:18<00:07, 4721107.40tokens/s]
Shard 37:  64%|██████▎   | 63743888/100000000 [00:19<00:08, 4362073.77tokens/s]
Shard 37:  64%|██████▍   | 64196548/100000000 [00:19<00:09, 3977142.15tokens/s]
Shard 37:  65%|██████▍   | 64728852/100000000 [00:19<00:08, 4328556.84tokens/s]
Shard 37:  65%|██████▌   | 65177027/100000000 [00:19<00:08, 4229297.38tokens/s]
Shard 37:  66%|██████▌   | 65610354/100000000 [00:19<00:09, 3795013.35tokens/s]
Shard 37:  66%|██████▌   | 66003827/100000000 [00:19<00:09, 3639840.61tokens/s]
Shard 37:  66%|██████▋   | 66377476/100000000 [00:19<00:09, 3363847.07tokens/s]
Shard 37:  67%|██████▋   | 66722381/100000000 [00:19<00:09, 3382923.29tokens/s]
Shard 37:  67%|██████▋   | 67105388/100000000 [00:19<00:09, 3500489.31tokens/s]
Shard 37:  68%|██████▊   | 67518201/100000000 [00:20<00:08, 3673107.81tokens/s]
Shard 37:  68%|██████▊   | 67892813/100000000 [00:20<00:09, 3293175.30tokens/s]
Shard 37:  68%|██████▊   | 68270221/100000000 [00:20<00:09, 3408296.01tokens/s]
Shard 37:  69%|██████▊   | 68717596/100000000 [00:20<00:08, 3693955.68tokens/s]
Shard 37:  69%|██████▉   | 69096887/100000000 [00:20<00:09, 3374530.28tokens/s]
Shard 37:  69%|██████▉   | 69474485/100000000 [00:20<00:08, 3467736.80tokens/s]
Shard 37:  70%|██████▉   | 69831198/100000000 [00:20<00:09, 3143091.28tokens/s]
Shard 37:  70%|███████   | 70155878/100000000 [00:20<00:09, 2984626.17tokens/s]
Shard 37:  71%|███████   | 70669778/100000000 [00:21<00:08, 3537415.93tokens/s]
Shard 37:  71%|███████▏  | 71336861/100000000 [00:21<00:06, 4371247.13tokens/s]
Shard 37:  72%|███████▏  | 71965921/100000000 [00:21<00:05, 4898125.72tokens/s]
Shard 37:  73%|███████▎  | 72666061/100000000 [00:21<00:04, 5495171.06tokens/s]
Shard 37:  73%|███████▎  | 73230479/100000000 [00:21<00:05, 5111683.40tokens/s]
Shard 37:  74%|███████▍  | 73757006/100000000 [00:21<00:05, 4948005.48tokens/s]
Shard 37:  74%|███████▍  | 74263626/100000000 [00:21<00:06, 4203872.47tokens/s]
Shard 37:  75%|███████▍  | 74708550/100000000 [00:21<00:06, 4045015.14tokens/s]
Shard 37:  75%|███████▌  | 75129997/100000000 [00:22<00:06, 3837753.24tokens/s]
Shard 37:  76%|███████▌  | 75660778/100000000 [00:22<00:05, 4204921.34tokens/s]
Shard 37:  76%|███████▌  | 76096898/100000000 [00:22<00:05, 4151741.28tokens/s]
Shard 37:  77%|███████▋  | 76523006/100000000 [00:22<00:05, 3955389.29tokens/s]
Shard 37:  77%|███████▋  | 76946349/100000000 [00:22<00:05, 3989523.04tokens/s]
Shard 37:  77%|███████▋  | 77418944/100000000 [00:22<00:05, 4179382.43tokens/s]
Shard 37:  78%|███████▊  | 77878785/100000000 [00:22<00:05, 4264444.68tokens/s]
Shard 37:  78%|███████▊  | 78310206/100000000 [00:22<00:05, 3899650.95tokens/s]
Shard 37:  79%|███████▊  | 78708505/100000000 [00:22<00:06, 3328179.09tokens/s]
Shard 37:  79%|███████▉  | 79059416/100000000 [00:23<00:06, 3267289.67tokens/s]
Shard 37:  79%|███████▉  | 79397945/100000000 [00:23<00:06, 3013926.50tokens/s]
Shard 37:  80%|███████▉  | 79709241/100000000 [00:23<00:07, 2868404.55tokens/s]
Shard 37:  80%|████████  | 80034732/100000000 [00:23<00:06, 2924830.77tokens/s]
Shard 37:  80%|████████  | 80332772/100000000 [00:23<00:06, 2922347.41tokens/s]
Shard 37:  81%|████████  | 80635341/100000000 [00:23<00:07, 2592180.78tokens/s]
Shard 37:  81%|████████  | 80902630/100000000 [00:23<00:07, 2513983.92tokens/s]
Shard 37:  81%|████████  | 81174003/100000000 [00:23<00:07, 2546702.13tokens/s]
Shard 37:  81%|████████▏ | 81482218/100000000 [00:23<00:06, 2688843.45tokens/s]
Shard 37:  82%|████████▏ | 81757457/100000000 [00:24<00:06, 2623363.85tokens/s]
Shard 37:  82%|████████▏ | 82025069/100000000 [00:24<00:07, 2469388.89tokens/s]
Shard 37:  82%|████████▏ | 82345803/100000000 [00:24<00:06, 2665733.20tokens/s]
Shard 37:  83%|████████▎ | 82667423/100000000 [00:24<00:06, 2818889.06tokens/s]
Shard 37:  83%|████████▎ | 82964246/100000000 [00:24<00:05, 2849110.51tokens/s]
Shard 37:  83%|████████▎ | 83311971/100000000 [00:24<00:05, 3020282.12tokens/s]
Shard 37:  84%|████████▎ | 83616744/100000000 [00:24<00:05, 2745498.15tokens/s]
Shard 37:  84%|████████▍ | 83915537/100000000 [00:24<00:05, 2803549.48tokens/s]
Shard 37:  84%|████████▍ | 84201336/100000000 [00:24<00:05, 2786382.99tokens/s]
Shard 37:  84%|████████▍ | 84490763/100000000 [00:25<00:05, 2811826.69tokens/s]
Shard 37:  85%|████████▍ | 84799952/100000000 [00:25<00:05, 2887691.80tokens/s]
Shard 37:  85%|████████▌ | 85164107/100000000 [00:25<00:04, 3086979.89tokens/s]
Shard 37:  86%|████████▌ | 85519913/100000000 [00:25<00:04, 3223898.76tokens/s]
Shard 37:  86%|████████▌ | 85844152/100000000 [00:25<00:04, 3045287.41tokens/s]
Shard 37:  86%|████████▌ | 86195393/100000000 [00:25<00:04, 3169443.79tokens/s]
Shard 37:  87%|████████▋ | 86516963/100000000 [00:25<00:04, 2994351.30tokens/s]
Shard 37:  87%|████████▋ | 86821598/100000000 [00:25<00:04, 2884111.53tokens/s]
Shard 37:  87%|████████▋ | 87239298/100000000 [00:25<00:03, 3219252.63tokens/s]
Shard 37:  88%|████████▊ | 87590199/100000000 [00:26<00:03, 3297933.53tokens/s]
Shard 37:  88%|████████▊ | 87964464/100000000 [00:26<00:03, 3422961.25tokens/s]
Shard 37:  88%|████████▊ | 88309739/100000000 [00:26<00:03, 3425418.99tokens/s]
Shard 37:  89%|████████▊ | 88654812/100000000 [00:26<00:03, 3224785.18tokens/s]
Shard 37:  89%|████████▉ | 88982062/100000000 [00:26<00:03, 2945756.89tokens/s]
Shard 37:  89%|████████▉ | 89294710/100000000 [00:26<00:03, 2983583.57tokens/s]
Shard 37:  90%|████████▉ | 89725171/100000000 [00:26<00:03, 3344862.36tokens/s]
Shard 37:  90%|█████████ | 90098982/100000000 [00:26<00:02, 3455980.73tokens/s]
Shard 37:  90%|█████████ | 90449694/100000000 [00:26<00:02, 3193041.95tokens/s]
Shard 37:  91%|█████████ | 90781898/100000000 [00:27<00:02, 3212255.24tokens/s]
Shard 37:  91%|█████████ | 91109570/100000000 [00:27<00:02, 3193302.63tokens/s]
Shard 37:  91%|█████████▏| 91436110/100000000 [00:27<00:03, 2838722.18tokens/s]
Shard 37:  92%|█████████▏| 91732320/100000000 [00:27<00:02, 2858463.03tokens/s]
Shard 37:  92%|█████████▏| 92043915/100000000 [00:27<00:02, 2896183.31tokens/s]
Shard 37:  92%|█████████▏| 92414944/100000000 [00:27<00:02, 3118928.76tokens/s]
Shard 37:  93%|█████████▎| 92779955/100000000 [00:27<00:02, 3193803.25tokens/s]
Shard 37:  93%|█████████▎| 93103114/100000000 [00:27<00:02, 3096099.04tokens/s]
Shard 37:  93%|█████████▎| 93431339/100000000 [00:27<00:02, 3145697.81tokens/s]
Shard 37:  94%|█████████▎| 93748306/100000000 [00:28<00:02, 2925602.85tokens/s]
Shard 37:  94%|█████████▍| 94045270/100000000 [00:28<00:02, 2686023.71tokens/s]
Shard 37:  94%|█████████▍| 94382452/100000000 [00:28<00:01, 2866926.57tokens/s]
Shard 37:  95%|█████████▍| 94887468/100000000 [00:28<00:01, 3457621.53tokens/s]
Shard 37:  95%|█████████▌| 95309380/100000000 [00:28<00:01, 3670422.05tokens/s]
Shard 37:  96%|█████████▌| 95708707/100000000 [00:28<00:01, 3757757.87tokens/s]
Shard 37:  96%|█████████▌| 96094814/100000000 [00:28<00:01, 3572961.29tokens/s]
Shard 37:  96%|█████████▋| 96459587/100000000 [00:28<00:01, 3307357.21tokens/s]
Shard 37:  97%|█████████▋| 96820055/100000000 [00:28<00:00, 3378759.52tokens/s]
Shard 37:  97%|█████████▋| 97169749/100000000 [00:29<00:00, 3115983.12tokens/s]
Shard 37:  97%|█████████▋| 97491937/100000000 [00:29<00:00, 2910427.97tokens/s]
Shard 37:  98%|█████████▊| 97789378/100000000 [00:29<00:00, 2916890.43tokens/s]
Shard 37:  98%|█████████▊| 98086445/100000000 [00:29<00:00, 2597511.95tokens/s]
Shard 37:  98%|█████████▊| 98473122/100000000 [00:29<00:00, 2920535.66tokens/s]
Shard 37:  99%|█████████▉| 98875304/100000000 [00:29<00:00, 3211608.66tokens/s]
Shard 37:  99%|█████████▉| 99208250/100000000 [00:29<00:00, 3056681.74tokens/s]
Shard 37: 100%|█████████▉| 99533863/100000000 [00:29<00:00, 3076923.89tokens/s]
Shard 37: 100%|█████████▉| 99917677/100000000 [00:29<00:00, 3286054.18tokens/s]
Shard 37: 100%|█████████▉| 99999962/100000000 [00:38<00:00, 2606312.75tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000037.bin (200,001,024 bytes) in the gpt-2 format

Shard 38:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 38:   0%|          | 227561/100000000 [00:00<00:44, 2260687.53tokens/s]
Shard 38:   1%|          | 502682/100000000 [00:00<00:39, 2546817.99tokens/s]
Shard 38:   1%|          | 773000/100000000 [00:00<00:37, 2616924.09tokens/s]
Shard 38:   1%|          | 1035383/100000000 [00:00<00:39, 2503274.32tokens/s]
Shard 38:   1%|▏         | 1357704/100000000 [00:00<00:36, 2727820.74tokens/s]
Shard 38:   2%|▏         | 1669360/100000000 [00:00<00:34, 2853337.45tokens/s]
Shard 38:   2%|▏         | 2163603/100000000 [00:00<00:27, 3518412.34tokens/s]
Shard 38:   3%|▎         | 2860955/100000000 [00:00<00:21, 4543926.39tokens/s]
Shard 38:   3%|▎         | 3315312/100000000 [00:00<00:26, 3701430.43tokens/s]
Shard 38:   4%|▎         | 3709712/100000000 [00:01<00:27, 3472722.41tokens/s]
Shard 38:   4%|▍         | 4086741/100000000 [00:01<00:27, 3542266.84tokens/s]
Shard 38:   4%|▍         | 4454980/100000000 [00:01<00:30, 3118933.42tokens/s]
Shard 38:   5%|▍         | 4782920/100000000 [00:01<00:30, 3101475.32tokens/s]
Shard 38:   5%|▌         | 5103998/100000000 [00:01<00:30, 3099661.59tokens/s]
Shard 38:   5%|▌         | 5421538/100000000 [00:01<00:33, 2835928.07tokens/s]
Shard 38:   6%|▌         | 5713870/100000000 [00:01<00:33, 2845622.89tokens/s]
Shard 38:   6%|▌         | 6074181/100000000 [00:01<00:31, 3025273.76tokens/s]
Shard 38:   6%|▋         | 6382598/100000000 [00:02<00:30, 3023721.76tokens/s]
Shard 38:   7%|▋         | 6753897/100000000 [00:02<00:29, 3213995.68tokens/s]
Shard 38:   7%|▋         | 7147240/100000000 [00:02<00:27, 3390914.74tokens/s]
Shard 38:   8%|▊         | 7552589/100000000 [00:02<00:25, 3578670.33tokens/s]
Shard 38:   8%|▊         | 8121887/100000000 [00:02<00:21, 4189179.46tokens/s]
Shard 38:   9%|▊         | 8544904/100000000 [00:02<00:22, 4064032.53tokens/s]
Shard 38:   9%|▉         | 8954660/100000000 [00:02<00:23, 3936150.47tokens/s]
Shard 38:   9%|▉         | 9403799/100000000 [00:02<00:22, 4085095.60tokens/s]
Shard 38:  10%|▉         | 9815603/100000000 [00:02<00:23, 3853864.28tokens/s]
Shard 38:  10%|█         | 10244180/100000000 [00:02<00:22, 3941001.58tokens/s]
Shard 38:  11%|█         | 10743260/100000000 [00:03<00:21, 4238863.01tokens/s]
Shard 38:  11%|█         | 11171348/100000000 [00:03<00:26, 3358383.05tokens/s]
Shard 38:  12%|█▏        | 11548965/100000000 [00:03<00:25, 3457443.28tokens/s]
Shard 38:  12%|█▏        | 11924123/100000000 [00:03<00:24, 3527311.52tokens/s]
Shard 38:  12%|█▏        | 12315014/100000000 [00:03<00:24, 3613746.43tokens/s]
Shard 38:  13%|█▎        | 12689853/100000000 [00:03<00:27, 3209897.50tokens/s]
Shard 38:  13%|█▎        | 13027434/100000000 [00:03<00:30, 2861496.75tokens/s]
Shard 38:  13%|█▎        | 13356839/100000000 [00:03<00:29, 2964543.48tokens/s]
Shard 38:  14%|█▎        | 13667628/100000000 [00:04<00:30, 2821011.37tokens/s]
Shard 38:  14%|█▍        | 13959617/100000000 [00:04<00:33, 2573433.75tokens/s]
Shard 38:  14%|█▍        | 14236455/100000000 [00:04<00:32, 2618622.45tokens/s]
Shard 38:  15%|█▍        | 14505935/100000000 [00:04<00:32, 2630220.50tokens/s]
Shard 38:  15%|█▍        | 14775253/100000000 [00:04<00:33, 2512059.56tokens/s]
Shard 38:  15%|█▌        | 15051348/100000000 [00:04<00:32, 2577913.34tokens/s]
Shard 38:  15%|█▌        | 15315235/100000000 [00:04<00:32, 2574194.39tokens/s]
Shard 38:  16%|█▌        | 15600487/100000000 [00:04<00:32, 2636502.52tokens/s]
Shard 38:  16%|█▌        | 15866869/100000000 [00:05<00:35, 2369052.73tokens/s]
Shard 38:  16%|█▌        | 16110218/100000000 [00:05<00:36, 2322256.87tokens/s]
Shard 38:  16%|█▋        | 16347147/100000000 [00:05<00:38, 2198421.61tokens/s]
Shard 38:  17%|█▋        | 16596403/100000000 [00:05<00:36, 2269506.54tokens/s]
Shard 38:  17%|█▋        | 16828736/100000000 [00:05<00:43, 1911213.69tokens/s]
Shard 38:  17%|█▋        | 17030958/100000000 [00:05<00:44, 1864444.97tokens/s]
Shard 38:  18%|█▊        | 17575302/100000000 [00:05<00:29, 2771870.40tokens/s]
Shard 38:  18%|█▊        | 17941355/100000000 [00:05<00:27, 3008195.67tokens/s]
Shard 38:  18%|█▊        | 18260072/100000000 [00:05<00:28, 2901314.62tokens/s]
Shard 38:  19%|█▊        | 18620555/100000000 [00:06<00:26, 3080161.77tokens/s]
Shard 38:  19%|█▉        | 18969501/100000000 [00:06<00:25, 3184087.86tokens/s]
Shard 38:  19%|█▉        | 19296571/100000000 [00:06<00:28, 2848556.13tokens/s]
Shard 38:  20%|█▉        | 19656155/100000000 [00:06<00:26, 3041328.43tokens/s]
Shard 38:  20%|█▉        | 19971125/100000000 [00:06<00:27, 2897377.65tokens/s]
Shard 38:  20%|██        | 20269468/100000000 [00:06<00:28, 2834517.67tokens/s]
Shard 38:  21%|██        | 20559346/100000000 [00:06<00:28, 2811310.90tokens/s]
Shard 38:  21%|██        | 20844241/100000000 [00:06<00:30, 2610929.55tokens/s]
Shard 38:  21%|██        | 21110279/100000000 [00:06<00:30, 2617873.60tokens/s]
Shard 38:  21%|██▏       | 21375293/100000000 [00:07<00:32, 2451416.79tokens/s]
Shard 38:  22%|██▏       | 21630238/100000000 [00:07<00:32, 2422161.43tokens/s]
Shard 38:  22%|██▏       | 21957817/100000000 [00:07<00:29, 2620870.97tokens/s]
Shard 38:  22%|██▏       | 22233375/100000000 [00:07<00:31, 2501656.30tokens/s]
Shard 38:  23%|██▎       | 22596411/100000000 [00:07<00:27, 2803363.46tokens/s]
Shard 38:  23%|██▎       | 22987824/100000000 [00:07<00:24, 3099835.46tokens/s]
Shard 38:  23%|██▎       | 23303742/100000000 [00:07<00:24, 3108737.89tokens/s]
Shard 38:  24%|██▎       | 23618428/100000000 [00:07<00:24, 3080689.62tokens/s]
Shard 38:  24%|██▍       | 23929079/100000000 [00:07<00:25, 2926614.82tokens/s]
Shard 38:  24%|██▍       | 24410841/100000000 [00:08<00:21, 3447214.34tokens/s]
Shard 38:  25%|██▍       | 24760791/100000000 [00:08<00:24, 3091539.75tokens/s]
Shard 38:  25%|██▌       | 25080032/100000000 [00:08<00:24, 3097971.13tokens/s]
Shard 38:  25%|██▌       | 25468368/100000000 [00:08<00:22, 3300372.83tokens/s]
Shard 38:  26%|██▌       | 25919236/100000000 [00:08<00:20, 3639922.58tokens/s]
Shard 38:  26%|██▋       | 26418115/100000000 [00:08<00:18, 4026490.04tokens/s]
Shard 38:  27%|██▋       | 26827580/100000000 [00:08<00:18, 4014621.27tokens/s]
Shard 38:  27%|██▋       | 27236178/100000000 [00:08<00:18, 3900464.73tokens/s]
Shard 38:  28%|██▊       | 27634814/100000000 [00:08<00:21, 3330850.54tokens/s]
Shard 38:  28%|██▊       | 27984642/100000000 [00:09<00:22, 3196562.99tokens/s]
Shard 38:  28%|██▊       | 28320248/100000000 [00:09<00:22, 3197222.57tokens/s]
Shard 38:  29%|██▊       | 28648407/100000000 [00:09<00:23, 3021351.01tokens/s]
Shard 38:  29%|██▉       | 29087183/100000000 [00:09<00:20, 3381495.65tokens/s]
Shard 38:  29%|██▉       | 29434670/100000000 [00:09<00:23, 2953175.56tokens/s]
Shard 38:  30%|██▉       | 29744816/100000000 [00:09<00:24, 2909796.41tokens/s]
Shard 38:  30%|███       | 30121452/100000000 [00:09<00:22, 3125613.87tokens/s]
Shard 38:  30%|███       | 30453507/100000000 [00:09<00:21, 3164029.25tokens/s]
Shard 38:  31%|███       | 30808092/100000000 [00:10<00:21, 3266914.42tokens/s]
Shard 38:  31%|███       | 31142187/100000000 [00:10<00:22, 3092056.74tokens/s]
Shard 38:  31%|███▏      | 31457055/100000000 [00:10<00:23, 2942002.23tokens/s]
Shard 38:  32%|███▏      | 31882075/100000000 [00:10<00:20, 3273619.38tokens/s]
Shard 38:  32%|███▏      | 32242216/100000000 [00:10<00:20, 3356917.61tokens/s]
Shard 38:  33%|███▎      | 32651811/100000000 [00:10<00:18, 3556075.26tokens/s]
Shard 38:  33%|███▎      | 33013111/100000000 [00:10<00:19, 3480159.95tokens/s]
Shard 38:  33%|███▎      | 33376547/100000000 [00:10<00:18, 3516524.93tokens/s]
Shard 38:  34%|███▍      | 33809406/100000000 [00:10<00:17, 3730905.07tokens/s]
Shard 38:  34%|███▍      | 34184981/100000000 [00:11<00:19, 3310465.48tokens/s]
Shard 38:  35%|███▍      | 34715948/100000000 [00:11<00:16, 3842312.93tokens/s]
Shard 38:  35%|███▌      | 35113633/100000000 [00:11<00:20, 3157130.86tokens/s]
Shard 38:  35%|███▌      | 35493458/100000000 [00:11<00:19, 3287947.35tokens/s]
Shard 38:  36%|███▌      | 35933966/100000000 [00:11<00:18, 3549443.28tokens/s]
Shard 38:  36%|███▋      | 36308077/100000000 [00:11<00:20, 3172781.24tokens/s]
Shard 38:  37%|███▋      | 36694253/100000000 [00:11<00:18, 3341466.34tokens/s]
Shard 38:  37%|███▋      | 37056927/100000000 [00:11<00:18, 3384707.69tokens/s]
Shard 38:  37%|███▋      | 37411611/100000000 [00:11<00:18, 3423106.53tokens/s]
Shard 38:  38%|███▊      | 37762763/100000000 [00:12<00:18, 3402565.35tokens/s]
Shard 38:  38%|███▊      | 38109800/100000000 [00:12<00:18, 3290542.10tokens/s]
Shard 38:  39%|███▊      | 38605910/100000000 [00:12<00:16, 3735929.17tokens/s]
Shard 38:  39%|███▉      | 38986023/100000000 [00:12<00:17, 3395061.60tokens/s]
Shard 38:  39%|███▉      | 39338090/100000000 [00:12<00:17, 3424401.36tokens/s]
Shard 38:  40%|███▉      | 39755902/100000000 [00:12<00:16, 3600304.60tokens/s]
Shard 38:  40%|████      | 40357867/100000000 [00:12<00:13, 4274009.66tokens/s]
Shard 38:  41%|████      | 40805354/100000000 [00:12<00:13, 4310271.03tokens/s]
Shard 38:  41%|████▏     | 41331344/100000000 [00:12<00:12, 4533678.53tokens/s]
Shard 38:  42%|████▏     | 41793620/100000000 [00:13<00:13, 4389897.51tokens/s]
Shard 38:  42%|████▏     | 42236941/100000000 [00:13<00:15, 3611648.75tokens/s]
Shard 38:  43%|████▎     | 42624889/100000000 [00:13<00:15, 3645629.07tokens/s]
Shard 38:  43%|████▎     | 43028125/100000000 [00:13<00:15, 3729113.38tokens/s]
Shard 38:  43%|████▎     | 43414441/100000000 [00:13<00:15, 3559885.86tokens/s]
Shard 38:  44%|████▍     | 43780536/100000000 [00:13<00:15, 3575220.57tokens/s]
Shard 38:  44%|████▍     | 44145762/100000000 [00:13<00:17, 3275799.83tokens/s]
Shard 38:  44%|████▍     | 44482538/100000000 [00:13<00:17, 3112221.96tokens/s]
Shard 38:  45%|████▍     | 44888772/100000000 [00:14<00:16, 3354558.58tokens/s]
Shard 38:  45%|████▌     | 45324818/100000000 [00:14<00:15, 3611671.00tokens/s]
Shard 38:  46%|████▌     | 45796279/100000000 [00:14<00:13, 3906241.46tokens/s]
Shard 38:  46%|████▌     | 46195283/100000000 [00:14<00:14, 3601445.36tokens/s]
Shard 38:  47%|████▋     | 46565276/100000000 [00:14<00:15, 3505130.27tokens/s]
Shard 38:  47%|████▋     | 46922173/100000000 [00:14<00:15, 3506019.93tokens/s]
Shard 38:  47%|████▋     | 47278429/100000000 [00:14<00:15, 3428960.31tokens/s]
Shard 38:  48%|████▊     | 47624594/100000000 [00:14<00:15, 3407996.28tokens/s]
Shard 38:  48%|████▊     | 48031772/100000000 [00:14<00:14, 3596324.69tokens/s]
Shard 38:  48%|████▊     | 48436428/100000000 [00:14<00:13, 3704271.89tokens/s]
Shard 38:  49%|████▉     | 48809240/100000000 [00:15<00:15, 3370271.57tokens/s]
Shard 38:  49%|████▉     | 49153470/100000000 [00:15<00:15, 3178041.81tokens/s]
Shard 38:  49%|████▉     | 49477203/100000000 [00:15<00:16, 3095437.03tokens/s]
Shard 38:  50%|████▉     | 49827620/100000000 [00:15<00:15, 3199870.78tokens/s]
Shard 38:  50%|█████     | 50151869/100000000 [00:15<00:16, 3102974.59tokens/s]
Shard 38:  50%|█████     | 50466459/100000000 [00:15<00:16, 3094539.88tokens/s]
Shard 38:  51%|█████     | 50938066/100000000 [00:15<00:13, 3552949.32tokens/s]
Shard 38:  51%|█████▏    | 51298727/100000000 [00:15<00:13, 3530385.00tokens/s]
Shard 38:  52%|█████▏    | 51757832/100000000 [00:15<00:12, 3806820.81tokens/s]
Shard 38:  52%|█████▏    | 52141298/100000000 [00:16<00:13, 3665727.23tokens/s]
Shard 38:  53%|█████▎    | 52511652/100000000 [00:16<00:13, 3398498.20tokens/s]
Shard 38:  53%|█████▎    | 52858222/100000000 [00:16<00:14, 3205676.57tokens/s]
Shard 38:  53%|█████▎    | 53203356/100000000 [00:16<00:14, 3271501.92tokens/s]
Shard 38:  54%|█████▎    | 53534493/100000000 [00:16<00:16, 2805487.23tokens/s]
Shard 38:  54%|█████▍    | 53917605/100000000 [00:16<00:15, 3064566.60tokens/s]
Shard 38:  54%|█████▍    | 54240042/100000000 [00:16<00:14, 3096778.75tokens/s]
Shard 38:  55%|█████▍    | 54592041/100000000 [00:16<00:14, 3212655.42tokens/s]
Shard 38:  55%|█████▌    | 55055269/100000000 [00:16<00:12, 3583692.28tokens/s]
Shard 38:  55%|█████▌    | 55476831/100000000 [00:17<00:11, 3733312.01tokens/s]
Shard 38:  56%|█████▌    | 56085578/100000000 [00:17<00:09, 4393826.43tokens/s]
Shard 38:  57%|█████▋    | 56531173/100000000 [00:17<00:10, 4236124.00tokens/s]
Shard 38:  57%|█████▋    | 56962182/100000000 [00:17<00:11, 3599006.24tokens/s]
Shard 38:  57%|█████▋    | 57341868/100000000 [00:17<00:12, 3354558.55tokens/s]
Shard 38:  58%|█████▊    | 57692273/100000000 [00:17<00:12, 3282731.63tokens/s]
Shard 38:  58%|█████▊    | 58030147/100000000 [00:17<00:13, 3129247.96tokens/s]
Shard 38:  58%|█████▊    | 58366040/100000000 [00:17<00:13, 3179331.63tokens/s]
Shard 38:  59%|█████▊    | 58689652/100000000 [00:18<00:13, 3162517.37tokens/s]
Shard 38:  59%|█████▉    | 59010348/100000000 [00:18<00:13, 2990221.29tokens/s]
Shard 38:  59%|█████▉    | 59345911/100000000 [00:18<00:13, 3078174.96tokens/s]
Shard 38:  60%|█████▉    | 59668595/100000000 [00:18<00:12, 3112124.74tokens/s]
Shard 38:  60%|██████    | 60071355/100000000 [00:18<00:11, 3370554.01tokens/s]
Shard 38:  60%|██████    | 60411800/100000000 [00:18<00:11, 3314015.10tokens/s]
Shard 38:  61%|██████    | 60746902/100000000 [00:18<00:12, 3204327.23tokens/s]
Shard 38:  61%|██████    | 61076569/100000000 [00:18<00:12, 3227710.78tokens/s]
Shard 38:  61%|██████▏   | 61447551/100000000 [00:18<00:11, 3366482.16tokens/s]
Shard 38:  62%|██████▏   | 61786516/100000000 [00:19<00:11, 3216222.86tokens/s]
Shard 38:  62%|██████▏   | 62111084/100000000 [00:19<00:12, 2956219.57tokens/s]
Shard 38:  62%|██████▏   | 62411655/100000000 [00:19<00:13, 2809891.71tokens/s]
Shard 38:  63%|██████▎   | 62696603/100000000 [00:19<00:15, 2465382.19tokens/s]
Shard 38:  63%|██████▎   | 62951863/100000000 [00:19<00:15, 2445737.28tokens/s]
Shard 38:  63%|██████▎   | 63230642/100000000 [00:19<00:14, 2531392.57tokens/s]
Shard 38:  64%|██████▎   | 63556100/100000000 [00:19<00:13, 2712482.61tokens/s]
Shard 38:  64%|██████▍   | 63842892/100000000 [00:19<00:13, 2728340.18tokens/s]
Shard 38:  64%|██████▍   | 64181107/100000000 [00:19<00:12, 2904847.00tokens/s]
Shard 38:  65%|██████▍   | 64529181/100000000 [00:20<00:11, 3069203.27tokens/s]
Shard 38:  65%|██████▍   | 64841637/100000000 [00:20<00:11, 3034939.53tokens/s]
Shard 38:  65%|██████▌   | 65148264/100000000 [00:20<00:12, 2746060.34tokens/s]
Shard 38:  65%|██████▌   | 65480868/100000000 [00:20<00:11, 2896706.57tokens/s]
Shard 38:  66%|██████▌   | 65826114/100000000 [00:20<00:11, 3033018.36tokens/s]
Shard 38:  66%|██████▌   | 66184514/100000000 [00:20<00:10, 3178468.12tokens/s]
Shard 38:  67%|██████▋   | 66526062/100000000 [00:20<00:10, 3240094.98tokens/s]
Shard 38:  67%|██████▋   | 66920919/100000000 [00:20<00:09, 3405375.80tokens/s]
Shard 38:  67%|██████▋   | 67265037/100000000 [00:20<00:10, 3235327.27tokens/s]
Shard 38:  68%|██████▊   | 67624196/100000000 [00:21<00:09, 3328418.20tokens/s]
Shard 38:  68%|██████▊   | 67962383/100000000 [00:21<00:10, 3086137.93tokens/s]
Shard 38:  68%|██████▊   | 68276056/100000000 [00:21<00:10, 3076012.66tokens/s]
Shard 38:  69%|██████▊   | 68587286/100000000 [00:21<00:10, 3064476.04tokens/s]
Shard 38:  69%|██████▉   | 68964210/100000000 [00:21<00:09, 3205410.92tokens/s]
Shard 38:  69%|██████▉   | 69331582/100000000 [00:21<00:09, 3331484.34tokens/s]
Shard 38:  70%|██████▉   | 69710566/100000000 [00:21<00:08, 3460420.28tokens/s]
Shard 38:  70%|███████   | 70058510/100000000 [00:21<00:08, 3427056.67tokens/s]
Shard 38:  70%|███████   | 70477345/100000000 [00:21<00:08, 3645238.61tokens/s]
Shard 38:  71%|███████   | 70905201/100000000 [00:21<00:07, 3825585.16tokens/s]
Shard 38:  71%|███████▏  | 71405885/100000000 [00:22<00:06, 4174588.50tokens/s]
Shard 38:  72%|███████▏  | 71880372/100000000 [00:22<00:06, 4327202.20tokens/s]
Shard 38:  72%|███████▏  | 72314194/100000000 [00:22<00:06, 4244236.05tokens/s]
Shard 38:  73%|███████▎  | 72739650/100000000 [00:22<00:06, 4085037.60tokens/s]
Shard 38:  73%|███████▎  | 73149952/100000000 [00:22<00:06, 3866048.04tokens/s]
Shard 38:  74%|███████▎  | 73619061/100000000 [00:22<00:06, 4010032.56tokens/s]
Shard 38:  74%|███████▍  | 74030177/100000000 [00:22<00:06, 4037913.15tokens/s]
Shard 38:  75%|███████▍  | 74789687/100000000 [00:22<00:04, 5043700.63tokens/s]
Shard 38:  75%|███████▌  | 75299929/100000000 [00:22<00:05, 4824134.95tokens/s]
Shard 38:  76%|███████▌  | 75849812/100000000 [00:23<00:04, 4992464.05tokens/s]
Shard 38:  76%|███████▋  | 76355377/100000000 [00:23<00:04, 4780646.50tokens/s]
Shard 38:  77%|███████▋  | 76838479/100000000 [00:23<00:05, 3952033.21tokens/s]
Shard 38:  77%|███████▋  | 77270101/100000000 [00:23<00:06, 3673381.71tokens/s]
Shard 38:  78%|███████▊  | 77657458/100000000 [00:23<00:06, 3458168.94tokens/s]
Shard 38:  78%|███████▊  | 78020179/100000000 [00:23<00:06, 3231252.49tokens/s]
Shard 38:  79%|███████▊  | 78582832/100000000 [00:23<00:05, 3815308.72tokens/s]
Shard 38:  79%|███████▉  | 78984360/100000000 [00:23<00:06, 3429895.61tokens/s]
Shard 38:  79%|███████▉  | 79346273/100000000 [00:24<00:06, 3249589.99tokens/s]
Shard 38:  80%|███████▉  | 79684112/100000000 [00:24<00:06, 3239998.58tokens/s]
Shard 38:  80%|████████  | 80017350/100000000 [00:24<00:06, 3005001.65tokens/s]
Shard 38:  80%|████████  | 80379260/100000000 [00:24<00:06, 3160002.50tokens/s]
Shard 38:  81%|████████  | 80734720/100000000 [00:24<00:05, 3260415.43tokens/s]
Shard 38:  81%|████████  | 81193052/100000000 [00:24<00:05, 3601795.86tokens/s]
Shard 38:  82%|████████▏ | 81560541/100000000 [00:24<00:05, 3515356.24tokens/s]
Shard 38:  82%|████████▏ | 81917294/100000000 [00:24<00:05, 3426237.74tokens/s]
Shard 38:  82%|████████▏ | 82266025/100000000 [00:24<00:05, 3129991.61tokens/s]
Shard 38:  83%|████████▎ | 82590911/100000000 [00:25<00:05, 3156794.93tokens/s]
Shard 38:  83%|████████▎ | 82911871/100000000 [00:25<00:05, 3026668.45tokens/s]
Shard 38:  83%|████████▎ | 83220111/100000000 [00:25<00:05, 2904162.89tokens/s]
Shard 38:  84%|████████▎ | 83514370/100000000 [00:25<00:05, 2802261.49tokens/s]
Shard 38:  84%|████████▍ | 83983056/100000000 [00:25<00:04, 3294579.86tokens/s]
Shard 38:  84%|████████▍ | 84432859/100000000 [00:25<00:04, 3622572.02tokens/s]
Shard 38:  85%|████████▍ | 84903539/100000000 [00:25<00:03, 3910483.90tokens/s]
Shard 38:  85%|████████▌ | 85301027/100000000 [00:25<00:03, 3752456.15tokens/s]
Shard 38:  86%|████████▌ | 85681317/100000000 [00:26<00:04, 3124619.84tokens/s]
Shard 38:  86%|████████▌ | 86014544/100000000 [00:26<00:05, 2773487.80tokens/s]
Shard 38:  86%|████████▋ | 86317280/100000000 [00:26<00:05, 2593383.38tokens/s]
Shard 38:  87%|████████▋ | 86593766/100000000 [00:26<00:05, 2586338.33tokens/s]
Shard 38:  87%|████████▋ | 86962178/100000000 [00:26<00:04, 2856915.93tokens/s]
Shard 38:  87%|████████▋ | 87259707/100000000 [00:26<00:04, 2882414.96tokens/s]
Shard 38:  88%|████████▊ | 87618525/100000000 [00:26<00:04, 3065549.64tokens/s]
Shard 38:  89%|████████▊ | 88636783/100000000 [00:26<00:02, 5055359.33tokens/s]
Shard 38:  89%|████████▉ | 89162819/100000000 [00:26<00:02, 5018077.78tokens/s]
Shard 38:  90%|████████▉ | 89678837/100000000 [00:27<00:02, 4708016.30tokens/s]
Shard 38:  90%|█████████ | 90162713/100000000 [00:27<00:02, 4015028.49tokens/s]
Shard 38:  91%|█████████ | 90588936/100000000 [00:27<00:02, 3861563.66tokens/s]
Shard 38:  91%|█████████ | 90991830/100000000 [00:27<00:02, 3573419.03tokens/s]
Shard 38:  91%|█████████▏| 91362509/100000000 [00:27<00:02, 3327596.76tokens/s]
Shard 38:  92%|█████████▏| 91760571/100000000 [00:27<00:02, 3485962.58tokens/s]
Shard 38:  92%|█████████▏| 92119384/100000000 [00:27<00:02, 3373516.83tokens/s]
Shard 38:  92%|█████████▏| 92463881/100000000 [00:27<00:02, 3255203.88tokens/s]
Shard 38:  93%|█████████▎| 92906134/100000000 [00:28<00:02, 3545193.81tokens/s]
Shard 38:  93%|█████████▎| 93267200/100000000 [00:28<00:02, 2994723.59tokens/s]
Shard 38:  94%|█████████▎| 93588073/100000000 [00:28<00:02, 2785412.26tokens/s]
Shard 38:  94%|█████████▍| 93974316/100000000 [00:28<00:01, 3013460.54tokens/s]
Shard 38:  94%|█████████▍| 94289632/100000000 [00:28<00:01, 2992589.41tokens/s]
Shard 38:  95%|█████████▍| 94600331/100000000 [00:28<00:01, 2986486.86tokens/s]
Shard 38:  95%|█████████▍| 94916113/100000000 [00:28<00:01, 3027034.52tokens/s]
Shard 38:  95%|█████████▌| 95223750/100000000 [00:28<00:01, 2950232.59tokens/s]
Shard 38:  96%|█████████▌| 95556918/100000000 [00:29<00:01, 3047402.35tokens/s]
Shard 38:  96%|█████████▌| 95920907/100000000 [00:29<00:01, 3214747.61tokens/s]
Shard 38:  96%|█████████▋| 96328465/100000000 [00:29<00:01, 3462110.79tokens/s]
Shard 38:  97%|█████████▋| 96678328/100000000 [00:29<00:01, 3123007.12tokens/s]
Shard 38:  97%|█████████▋| 97168027/100000000 [00:29<00:00, 3606732.14tokens/s]
Shard 38:  98%|█████████▊| 97539397/100000000 [00:29<00:00, 3582401.06tokens/s]
Shard 38:  98%|█████████▊| 97905124/100000000 [00:29<00:00, 3448499.94tokens/s]
Shard 38:  98%|█████████▊| 98256215/100000000 [00:29<00:00, 3417466.62tokens/s]
Shard 38:  99%|█████████▊| 98602013/100000000 [00:29<00:00, 3211578.49tokens/s]
Shard 38:  99%|█████████▉| 98930359/100000000 [00:30<00:00, 3194759.73tokens/s]
Shard 38:  99%|█████████▉| 99253542/100000000 [00:30<00:00, 3186707.12tokens/s]
Shard 38: 100%|█████████▉| 99606453/100000000 [00:30<00:00, 3274577.49tokens/s]
Shard 38: 100%|█████████▉| 99938432/100000000 [00:30<00:00, 2996061.30tokens/s]
Shard 38: 100%|█████████▉| 99999591/100000000 [00:38<00:00, 2599293.03tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000038.bin (200,001,024 bytes) in the gpt-2 format

Shard 39:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 39:   0%|          | 183242/100000000 [00:00<00:55, 1803805.48tokens/s]
Shard 39:   0%|          | 410597/100000000 [00:00<00:48, 2057761.53tokens/s]
Shard 39:   1%|          | 616487/100000000 [00:00<00:48, 2034574.01tokens/s]
Shard 39:   1%|          | 824176/100000000 [00:00<00:54, 1833303.71tokens/s]
Shard 39:   1%|▏         | 1259709/100000000 [00:00<00:37, 2654863.39tokens/s]
Shard 39:   2%|▏         | 1595876/100000000 [00:00<00:34, 2865064.24tokens/s]
Shard 39:   2%|▏         | 1944072/100000000 [00:00<00:32, 3052559.38tokens/s]
Shard 39:   2%|▏         | 2265871/100000000 [00:00<00:32, 3043432.76tokens/s]
Shard 39:   3%|▎         | 2574174/100000000 [00:00<00:32, 2978133.43tokens/s]
Shard 39:   3%|▎         | 2950269/100000000 [00:01<00:30, 3166253.44tokens/s]
Shard 39:   3%|▎         | 3268884/100000000 [00:01<00:35, 2692474.29tokens/s]
Shard 39:   4%|▍         | 3759694/100000000 [00:01<00:29, 3272364.94tokens/s]
Shard 39:   4%|▍         | 4272942/100000000 [00:01<00:25, 3764640.02tokens/s]
Shard 39:   5%|▍         | 4667418/100000000 [00:01<00:29, 3183446.27tokens/s]
Shard 39:   5%|▌         | 5037387/100000000 [00:01<00:28, 3291178.77tokens/s]
Shard 39:   5%|▌         | 5386894/100000000 [00:01<00:29, 3231680.72tokens/s]
Shard 39:   6%|▌         | 5746927/100000000 [00:01<00:28, 3325606.72tokens/s]
Shard 39:   6%|▌         | 6214726/100000000 [00:02<00:25, 3694355.59tokens/s]
Shard 39:   7%|▋         | 6595496/100000000 [00:02<00:28, 3278713.11tokens/s]
Shard 39:   7%|▋         | 6960599/100000000 [00:02<00:27, 3375420.90tokens/s]
Shard 39:   7%|▋         | 7311760/100000000 [00:02<00:27, 3366775.84tokens/s]
Shard 39:   8%|▊         | 7686868/100000000 [00:02<00:26, 3461147.48tokens/s]
Shard 39:   8%|▊         | 8040304/100000000 [00:02<00:29, 3069063.99tokens/s]
Shard 39:   8%|▊         | 8359712/100000000 [00:02<00:31, 2919938.57tokens/s]
Shard 39:   9%|▊         | 8739465/100000000 [00:02<00:29, 3139415.46tokens/s]
Shard 39:   9%|▉         | 9248786/100000000 [00:02<00:24, 3663609.03tokens/s]
Shard 39:  10%|▉         | 9688856/100000000 [00:03<00:23, 3861596.89tokens/s]
Shard 39:  10%|█         | 10160060/100000000 [00:03<00:21, 4094715.62tokens/s]
Shard 39:  11%|█         | 10578732/100000000 [00:03<00:25, 3573126.50tokens/s]
Shard 39:  11%|█         | 10965784/100000000 [00:03<00:24, 3642173.74tokens/s]
Shard 39:  11%|█▏        | 11361581/100000000 [00:03<00:23, 3713423.29tokens/s]
Shard 39:  12%|█▏        | 11744761/100000000 [00:03<00:25, 3494801.15tokens/s]
Shard 39:  12%|█▏        | 12103080/100000000 [00:03<00:26, 3371355.97tokens/s]
Shard 39:  13%|█▎        | 12515104/100000000 [00:03<00:24, 3566062.99tokens/s]
Shard 39:  13%|█▎        | 12921711/100000000 [00:03<00:23, 3696975.90tokens/s]
Shard 39:  14%|█▎        | 13574402/100000000 [00:04<00:19, 4486300.04tokens/s]
Shard 39:  14%|█▍        | 14098543/100000000 [00:04<00:18, 4684103.50tokens/s]
Shard 39:  15%|█▍        | 14574170/100000000 [00:04<00:18, 4556105.28tokens/s]
Shard 39:  15%|█▌        | 15034575/100000000 [00:04<00:20, 4132792.99tokens/s]
Shard 39:  15%|█▌        | 15463632/100000000 [00:04<00:20, 4147847.36tokens/s]
Shard 39:  16%|█▌        | 15885410/100000000 [00:04<00:24, 3453376.00tokens/s]
Shard 39:  16%|█▋        | 16323609/100000000 [00:04<00:22, 3671126.56tokens/s]
Shard 39:  17%|█▋        | 16710925/100000000 [00:04<00:24, 3461029.23tokens/s]
Shard 39:  17%|█▋        | 17163263/100000000 [00:05<00:22, 3671801.51tokens/s]
Shard 39:  18%|█▊        | 17567812/100000000 [00:05<00:21, 3761994.91tokens/s]
Shard 39:  18%|█▊        | 17954183/100000000 [00:05<00:23, 3449948.12tokens/s]
Shard 39:  18%|█▊        | 18334962/100000000 [00:05<00:23, 3539356.20tokens/s]
Shard 39:  19%|█▊        | 18697994/100000000 [00:05<00:23, 3482818.65tokens/s]
Shard 39:  19%|█▉        | 19160920/100000000 [00:05<00:21, 3789459.18tokens/s]
Shard 39:  20%|█▉        | 19546480/100000000 [00:05<00:21, 3682946.91tokens/s]
Shard 39:  20%|█▉        | 19920131/100000000 [00:05<00:21, 3670576.95tokens/s]
Shard 39:  20%|██        | 20290573/100000000 [00:05<00:21, 3666998.22tokens/s]
Shard 39:  21%|██        | 20660200/100000000 [00:05<00:23, 3384241.12tokens/s]
Shard 39:  21%|██▏       | 21470378/100000000 [00:06<00:16, 4681208.04tokens/s]
Shard 39:  22%|██▏       | 21955457/100000000 [00:06<00:17, 4471645.87tokens/s]
Shard 39:  22%|██▏       | 22415568/100000000 [00:06<00:18, 4286610.28tokens/s]
Shard 39:  23%|██▎       | 23040456/100000000 [00:06<00:16, 4809061.15tokens/s]
Shard 39:  24%|██▎       | 23532709/100000000 [00:06<00:16, 4595767.93tokens/s]
Shard 39:  24%|██▍       | 24266878/100000000 [00:06<00:14, 5344410.63tokens/s]
Shard 39:  25%|██▍       | 24814351/100000000 [00:06<00:15, 4954310.74tokens/s]
Shard 39:  25%|██▌       | 25323371/100000000 [00:06<00:17, 4339570.53tokens/s]
Shard 39:  26%|██▌       | 25783147/100000000 [00:07<00:17, 4240838.39tokens/s]
Shard 39:  26%|██▌       | 26228328/100000000 [00:07<00:18, 3924407.97tokens/s]
Shard 39:  27%|██▋       | 26632797/100000000 [00:07<00:19, 3722692.13tokens/s]
Shard 39:  27%|██▋       | 27013353/100000000 [00:07<00:20, 3582543.71tokens/s]
Shard 39:  27%|██▋       | 27409239/100000000 [00:07<00:19, 3670089.76tokens/s]
Shard 39:  28%|██▊       | 27781165/100000000 [00:07<00:20, 3448473.31tokens/s]
Shard 39:  28%|██▊       | 28187963/100000000 [00:07<00:19, 3603347.68tokens/s]
Shard 39:  29%|██▊       | 28688788/100000000 [00:07<00:17, 3963719.10tokens/s]
Shard 39:  29%|██▉       | 29299868/100000000 [00:07<00:15, 4495574.58tokens/s]
Shard 39:  30%|██▉       | 29849089/100000000 [00:08<00:14, 4763723.37tokens/s]
Shard 39:  30%|███       | 30332161/100000000 [00:08<00:14, 4778398.46tokens/s]
Shard 39:  31%|███       | 30858476/100000000 [00:08<00:14, 4907679.39tokens/s]
Shard 39:  31%|███▏      | 31352951/100000000 [00:08<00:15, 4355280.08tokens/s]
Shard 39:  32%|███▏      | 31803177/100000000 [00:08<00:15, 4339205.13tokens/s]
Shard 39:  32%|███▏      | 32248556/100000000 [00:08<00:16, 4112862.07tokens/s]
Shard 39:  33%|███▎      | 32672306/100000000 [00:08<00:17, 3939888.92tokens/s]
Shard 39:  33%|███▎      | 33096677/100000000 [00:08<00:16, 4019359.05tokens/s]
Shard 39:  34%|███▎      | 33612730/100000000 [00:08<00:15, 4325263.04tokens/s]
Shard 39:  34%|███▍      | 34051377/100000000 [00:09<00:16, 3936810.41tokens/s]
Shard 39:  34%|███▍      | 34464980/100000000 [00:09<00:16, 3977899.46tokens/s]
Shard 39:  35%|███▍      | 34873907/100000000 [00:09<00:17, 3678780.59tokens/s]
Shard 39:  35%|███▌      | 35359276/100000000 [00:09<00:16, 3958696.24tokens/s]
Shard 39:  36%|███▌      | 35764317/100000000 [00:09<00:16, 3962051.32tokens/s]
Shard 39:  36%|███▌      | 36241465/100000000 [00:09<00:15, 4173396.23tokens/s]
Shard 39:  37%|███▋      | 37052544/100000000 [00:09<00:11, 5284045.86tokens/s]
Shard 39:  38%|███▊      | 37635459/100000000 [00:09<00:11, 5424860.38tokens/s]
Shard 39:  38%|███▊      | 38194575/100000000 [00:09<00:11, 5473212.74tokens/s]
Shard 39:  39%|███▉      | 38824775/100000000 [00:10<00:10, 5646135.25tokens/s]
Shard 39:  39%|███▉      | 39453943/100000000 [00:10<00:10, 5637017.72tokens/s]
Shard 39:  40%|████      | 40020306/100000000 [00:10<00:11, 5220467.46tokens/s]
Shard 39:  41%|████      | 40551954/100000000 [00:10<00:11, 5029657.56tokens/s]
Shard 39:  41%|████      | 41090293/100000000 [00:10<00:11, 5098031.93tokens/s]
Shard 39:  42%|████▏     | 41604652/100000000 [00:10<00:11, 4903534.46tokens/s]
Shard 39:  42%|████▏     | 42175215/100000000 [00:10<00:11, 5110994.21tokens/s]
Shard 39:  43%|████▎     | 42690449/100000000 [00:10<00:11, 5078097.10tokens/s]
Shard 39:  43%|████▎     | 43211622/100000000 [00:10<00:11, 5110021.01tokens/s]
Shard 39:  44%|████▎     | 43726882/100000000 [00:11<00:12, 4557546.01tokens/s]
Shard 39:  44%|████▍     | 44202063/100000000 [00:11<00:12, 4516064.86tokens/s]
Shard 39:  45%|████▍     | 44661701/100000000 [00:11<00:13, 4252821.64tokens/s]
Shard 39:  45%|████▌     | 45094378/100000000 [00:11<00:14, 3872600.77tokens/s]
Shard 39:  45%|████▌     | 45490831/100000000 [00:11<00:14, 3875145.27tokens/s]
Shard 39:  46%|████▌     | 45926730/100000000 [00:11<00:13, 3980145.96tokens/s]
Shard 39:  46%|████▋     | 46330624/100000000 [00:11<00:13, 3866284.15tokens/s]
Shard 39:  47%|████▋     | 46824952/100000000 [00:11<00:12, 4154654.64tokens/s]
Shard 39:  47%|████▋     | 47252106/100000000 [00:11<00:12, 4157901.36tokens/s]
Shard 39:  48%|████▊     | 47674873/100000000 [00:12<00:12, 4153467.20tokens/s]
Shard 39:  48%|████▊     | 48092767/100000000 [00:12<00:12, 4030493.09tokens/s]
Shard 39:  49%|████▊     | 48503248/100000000 [00:12<00:12, 4011959.03tokens/s]
Shard 39:  49%|████▉     | 48973565/100000000 [00:12<00:12, 4190366.37tokens/s]
Shard 39:  49%|████▉     | 49394819/100000000 [00:12<00:12, 4049566.67tokens/s]
Shard 39:  50%|████▉     | 49973039/100000000 [00:12<00:11, 4527337.82tokens/s]
Shard 39:  50%|█████     | 50429354/100000000 [00:12<00:11, 4291967.78tokens/s]
Shard 39:  51%|█████     | 50862902/100000000 [00:12<00:11, 4150576.82tokens/s]
Shard 39:  51%|█████▏    | 51281366/100000000 [00:12<00:13, 3609475.94tokens/s]
Shard 39:  52%|█████▏    | 51657402/100000000 [00:13<00:13, 3562041.46tokens/s]
Shard 39:  52%|█████▏    | 52142932/100000000 [00:13<00:12, 3845711.37tokens/s]
Shard 39:  53%|█████▎    | 52536876/100000000 [00:13<00:12, 3762011.48tokens/s]
Shard 39:  53%|█████▎    | 52969100/100000000 [00:13<00:12, 3900848.13tokens/s]
Shard 39:  54%|█████▎    | 53536720/100000000 [00:13<00:10, 4392755.44tokens/s]
Shard 39:  54%|█████▍    | 54100959/100000000 [00:13<00:09, 4730854.93tokens/s]
Shard 39:  55%|█████▍    | 54704447/100000000 [00:13<00:08, 5090728.60tokens/s]
Shard 39:  55%|█████▌    | 55223647/100000000 [00:13<00:08, 5115666.71tokens/s]
Shard 39:  56%|█████▌    | 55774279/100000000 [00:13<00:08, 5230230.75tokens/s]
Shard 39:  56%|█████▋    | 56311560/100000000 [00:14<00:08, 5263780.31tokens/s]
Shard 39:  57%|█████▋    | 56840395/100000000 [00:14<00:08, 5202314.49tokens/s]
Shard 39:  57%|█████▋    | 57362420/100000000 [00:14<00:09, 4565228.08tokens/s]
Shard 39:  58%|█████▊    | 57868347/100000000 [00:14<00:09, 4485756.24tokens/s]
Shard 39:  58%|█████▊    | 58327650/100000000 [00:14<00:10, 4088254.00tokens/s]
Shard 39:  59%|█████▉    | 58899678/100000000 [00:14<00:09, 4499347.19tokens/s]
Shard 39:  59%|█████▉    | 59364389/100000000 [00:14<00:09, 4457307.95tokens/s]
Shard 39:  60%|█████▉    | 59820217/100000000 [00:14<00:09, 4446570.99tokens/s]
Shard 39:  60%|██████    | 60273017/100000000 [00:14<00:08, 4460769.00tokens/s]
Shard 39:  61%|██████    | 60833841/100000000 [00:15<00:08, 4771878.26tokens/s]
Shard 39:  61%|██████▏   | 61316127/100000000 [00:15<00:08, 4544062.72tokens/s]
Shard 39:  62%|██████▏   | 61775680/100000000 [00:15<00:09, 4096867.09tokens/s]
Shard 39:  62%|██████▏   | 62267850/100000000 [00:15<00:08, 4310874.66tokens/s]
Shard 39:  63%|██████▎   | 62709397/100000000 [00:15<00:08, 4332683.15tokens/s]
Shard 39:  63%|██████▎   | 63150008/100000000 [00:15<00:08, 4109202.66tokens/s]
Shard 39:  64%|██████▎   | 63704269/100000000 [00:15<00:08, 4489512.12tokens/s]
Shard 39:  64%|██████▍   | 64173326/100000000 [00:15<00:08, 4367367.60tokens/s]
Shard 39:  65%|██████▍   | 64618831/100000000 [00:15<00:08, 4342144.12tokens/s]
Shard 39:  65%|██████▌   | 65059607/100000000 [00:16<00:08, 4206750.01tokens/s]
Shard 39:  65%|██████▌   | 65483569/100000000 [00:16<00:08, 4038354.73tokens/s]
Shard 39:  66%|██████▌   | 65902176/100000000 [00:16<00:08, 4067503.43tokens/s]
Shard 39:  66%|██████▋   | 66311735/100000000 [00:16<00:08, 3945760.98tokens/s]
Shard 39:  67%|██████▋   | 66708470/100000000 [00:16<00:08, 3809517.22tokens/s]
Shard 39:  67%|██████▋   | 67095927/100000000 [00:16<00:08, 3767902.10tokens/s]
Shard 39:  68%|██████▊   | 67624295/100000000 [00:16<00:07, 4171579.72tokens/s]
Shard 39:  68%|██████▊   | 68303076/100000000 [00:16<00:06, 4911199.20tokens/s]
Shard 39:  69%|██████▉   | 68799396/100000000 [00:16<00:06, 4881607.67tokens/s]
Shard 39:  69%|██████▉   | 69291175/100000000 [00:17<00:06, 4753194.27tokens/s]
Shard 39:  70%|██████▉   | 69769674/100000000 [00:17<00:07, 4190111.54tokens/s]
Shard 39:  70%|███████   | 70315414/100000000 [00:17<00:06, 4508614.71tokens/s]
Shard 39:  71%|███████   | 70874568/100000000 [00:17<00:06, 4708980.73tokens/s]
Shard 39:  71%|███████▏  | 71355912/100000000 [00:17<00:06, 4397393.90tokens/s]
Shard 39:  72%|███████▏  | 72131659/100000000 [00:17<00:05, 5287159.76tokens/s]
Shard 39:  73%|███████▎  | 72676754/100000000 [00:17<00:06, 4387421.79tokens/s]
Shard 39:  73%|███████▎  | 73150304/100000000 [00:17<00:06, 4359016.72tokens/s]
Shard 39:  74%|███████▍  | 73841823/100000000 [00:17<00:05, 5009454.20tokens/s]
Shard 39:  74%|███████▍  | 74371313/100000000 [00:18<00:05, 4999785.77tokens/s]
Shard 39:  75%|███████▍  | 74891723/100000000 [00:18<00:05, 4781678.29tokens/s]
Shard 39:  75%|███████▌  | 75386405/100000000 [00:18<00:05, 4506853.49tokens/s]
Shard 39:  76%|███████▌  | 75850149/100000000 [00:18<00:05, 4193602.28tokens/s]
Shard 39:  76%|███████▋  | 76280098/100000000 [00:18<00:06, 3634605.87tokens/s]
Shard 39:  77%|███████▋  | 76751365/100000000 [00:18<00:05, 3892708.12tokens/s]
Shard 39:  77%|███████▋  | 77158773/100000000 [00:18<00:06, 3641542.23tokens/s]
Shard 39:  78%|███████▊  | 77536695/100000000 [00:18<00:06, 3447402.55tokens/s]
Shard 39:  78%|███████▊  | 77911852/100000000 [00:19<00:06, 3490354.28tokens/s]
Shard 39:  78%|███████▊  | 78273196/100000000 [00:19<00:06, 3336315.97tokens/s]
Shard 39:  79%|███████▊  | 78612367/100000000 [00:19<00:06, 3164255.98tokens/s]
Shard 39:  79%|███████▉  | 79049424/100000000 [00:19<00:06, 3471990.98tokens/s]
Shard 39:  79%|███████▉  | 79404200/100000000 [00:19<00:06, 3293036.49tokens/s]
Shard 39:  80%|███████▉  | 79739172/100000000 [00:19<00:06, 3249096.36tokens/s]
Shard 39:  80%|████████  | 80069155/100000000 [00:19<00:06, 3143179.68tokens/s]
Shard 39:  80%|████████  | 80448163/100000000 [00:19<00:05, 3308081.90tokens/s]
Shard 39:  81%|████████  | 80870920/100000000 [00:19<00:05, 3560693.85tokens/s]
Shard 39:  81%|████████▏ | 81282147/100000000 [00:20<00:05, 3714977.66tokens/s]
Shard 39:  82%|████████▏ | 81657353/100000000 [00:20<00:05, 3584517.76tokens/s]
Shard 39:  82%|████████▏ | 82071070/100000000 [00:20<00:04, 3741662.18tokens/s]
Shard 39:  82%|████████▏ | 82449097/100000000 [00:20<00:04, 3731360.65tokens/s]
Shard 39:  83%|████████▎ | 82865206/100000000 [00:20<00:04, 3851553.03tokens/s]
Shard 39:  83%|████████▎ | 83252211/100000000 [00:20<00:04, 3495629.86tokens/s]
Shard 39:  84%|████████▎ | 83680399/100000000 [00:20<00:04, 3681189.08tokens/s]
Shard 39:  84%|████████▍ | 84139598/100000000 [00:20<00:04, 3914878.27tokens/s]
Shard 39:  85%|████████▍ | 84543520/100000000 [00:20<00:03, 3947214.87tokens/s]
Shard 39:  85%|████████▍ | 84942507/100000000 [00:21<00:03, 3852436.24tokens/s]
Shard 39:  85%|████████▌ | 85331257/100000000 [00:21<00:03, 3821537.89tokens/s]
Shard 39:  86%|████████▌ | 85858689/100000000 [00:21<00:03, 4231803.36tokens/s]
Shard 39:  86%|████████▋ | 86439174/100000000 [00:21<00:02, 4689201.68tokens/s]
Shard 39:  87%|████████▋ | 86911562/100000000 [00:21<00:03, 4336441.25tokens/s]
Shard 39:  87%|████████▋ | 87352790/100000000 [00:21<00:03, 4212082.81tokens/s]
Shard 39:  88%|████████▊ | 87782717/100000000 [00:21<00:02, 4184848.97tokens/s]
Shard 39:  88%|████████▊ | 88237070/100000000 [00:21<00:02, 4278684.94tokens/s]
Shard 39:  89%|████████▉ | 88820064/100000000 [00:21<00:02, 4705427.22tokens/s]
Shard 39:  89%|████████▉ | 89296564/100000000 [00:22<00:02, 4119680.72tokens/s]
Shard 39:  90%|████████▉ | 89723887/100000000 [00:22<00:02, 3996743.87tokens/s]
Shard 39:  90%|█████████ | 90135157/100000000 [00:22<00:02, 3453288.29tokens/s]
Shard 39:  90%|█████████ | 90498961/100000000 [00:22<00:02, 3427822.24tokens/s]
Shard 39:  91%|█████████ | 90854497/100000000 [00:22<00:02, 3363778.55tokens/s]
Shard 39:  91%|█████████ | 91199214/100000000 [00:22<00:02, 3313057.37tokens/s]
Shard 39:  92%|█████████▏| 91535629/100000000 [00:22<00:02, 2923480.27tokens/s]
Shard 39:  92%|█████████▏| 91841164/100000000 [00:22<00:02, 2874001.52tokens/s]
Shard 39:  92%|█████████▏| 92169495/100000000 [00:23<00:02, 2975037.11tokens/s]
Shard 39:  92%|█████████▏| 92473696/100000000 [00:23<00:02, 2811887.20tokens/s]
Shard 39:  93%|█████████▎| 92829859/100000000 [00:23<00:02, 2999878.57tokens/s]
Shard 39:  93%|█████████▎| 93139171/100000000 [00:23<00:02, 2918449.04tokens/s]
Shard 39:  94%|█████████▎| 93741951/100000000 [00:23<00:01, 3756744.87tokens/s]
Shard 39:  94%|█████████▍| 94193097/100000000 [00:23<00:01, 3955078.68tokens/s]
Shard 39:  95%|█████████▍| 94596588/100000000 [00:23<00:01, 3716689.74tokens/s]
Shard 39:  95%|█████████▌| 95056795/100000000 [00:23<00:01, 3955298.76tokens/s]
Shard 39:  96%|█████████▌| 95869673/100000000 [00:23<00:00, 5132549.56tokens/s]
Shard 39:  96%|█████████▋| 96395981/100000000 [00:24<00:00, 4612934.32tokens/s]
Shard 39:  97%|█████████▋| 96875409/100000000 [00:24<00:00, 4079920.86tokens/s]
Shard 39:  97%|█████████▋| 97305519/100000000 [00:24<00:00, 3472359.18tokens/s]
Shard 39:  98%|█████████▊| 97679328/100000000 [00:24<00:00, 3532910.21tokens/s]
Shard 39:  98%|█████████▊| 98053227/100000000 [00:24<00:00, 3245549.13tokens/s]
Shard 39:  98%|█████████▊| 98394055/100000000 [00:24<00:00, 3113080.00tokens/s]
Shard 39:  99%|█████████▉| 98841350/100000000 [00:24<00:00, 3417651.68tokens/s]
Shard 39:  99%|█████████▉| 99196829/100000000 [00:24<00:00, 3308194.74tokens/s]
Shard 39: 100%|█████████▉| 99553041/100000000 [00:25<00:00, 3364912.77tokens/s]
Shard 39: 100%|█████████▉| 99896345/100000000 [00:25<00:00, 2868189.32tokens/s]
Shard 39: 100%|█████████▉| 99999286/100000000 [00:33<00:00, 3016216.39tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000039.bin (200,001,024 bytes) in the gpt-2 format

Shard 40:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 40:   0%|          | 324953/100000000 [00:00<00:30, 3232324.55tokens/s]
Shard 40:   1%|          | 648321/100000000 [00:00<00:32, 3084420.23tokens/s]
Shard 40:   1%|          | 957499/100000000 [00:00<00:32, 3030953.64tokens/s]
Shard 40:   1%|▏         | 1390671/100000000 [00:00<00:28, 3521358.45tokens/s]
Shard 40:   2%|▏         | 1810012/100000000 [00:00<00:26, 3734511.35tokens/s]
Shard 40:   2%|▏         | 2385073/100000000 [00:00<00:22, 4395066.63tokens/s]
Shard 40:   3%|▎         | 2932891/100000000 [00:00<00:20, 4731572.69tokens/s]
Shard 40:   3%|▎         | 3407641/100000000 [00:00<00:22, 4269062.55tokens/s]
Shard 40:   4%|▍         | 3843175/100000000 [00:01<00:28, 3381137.63tokens/s]
Shard 40:   4%|▍         | 4212874/100000000 [00:01<00:28, 3369183.39tokens/s]
Shard 40:   5%|▍         | 4572175/100000000 [00:01<00:29, 3256217.24tokens/s]
Shard 40:   5%|▍         | 4914196/100000000 [00:01<00:29, 3257644.08tokens/s]
Shard 40:   5%|▌         | 5250790/100000000 [00:01<00:31, 3045789.40tokens/s]
Shard 40:   6%|▌         | 5564658/100000000 [00:01<00:33, 2848258.63tokens/s]
Shard 40:   6%|▌         | 5855976/100000000 [00:01<00:33, 2771809.15tokens/s]
Shard 40:   6%|▌         | 6228758/100000000 [00:01<00:31, 3016658.48tokens/s]
Shard 40:   7%|▋         | 6537079/100000000 [00:01<00:31, 2969130.65tokens/s]
Shard 40:   7%|▋         | 6919299/100000000 [00:02<00:29, 3201215.59tokens/s]
Shard 40:   7%|▋         | 7323501/100000000 [00:02<00:27, 3424695.84tokens/s]
Shard 40:   8%|▊         | 7671011/100000000 [00:02<00:31, 2969337.93tokens/s]
Shard 40:   8%|▊         | 7981267/100000000 [00:02<00:30, 2999491.11tokens/s]
Shard 40:   8%|▊         | 8291063/100000000 [00:02<00:31, 2928048.22tokens/s]
Shard 40:   9%|▊         | 8607392/100000000 [00:02<00:30, 2983306.55tokens/s]
Shard 40:   9%|▉         | 8962232/100000000 [00:02<00:29, 3130821.93tokens/s]
Shard 40:   9%|▉         | 9360027/100000000 [00:02<00:26, 3363112.29tokens/s]
Shard 40:  10%|▉         | 9704775/100000000 [00:02<00:26, 3375986.12tokens/s]
Shard 40:  10%|█         | 10153668/100000000 [00:03<00:24, 3687506.24tokens/s]
Shard 40:  11%|█         | 10525535/100000000 [00:03<00:25, 3556790.69tokens/s]
Shard 40:  11%|█         | 10925907/100000000 [00:03<00:24, 3652119.67tokens/s]
Shard 40:  11%|█▏        | 11293411/100000000 [00:03<00:26, 3393437.02tokens/s]
Shard 40:  12%|█▏        | 11638215/100000000 [00:03<00:26, 3357799.22tokens/s]
Shard 40:  12%|█▏        | 12043259/100000000 [00:03<00:24, 3537256.86tokens/s]
Shard 40:  13%|█▎        | 12561385/100000000 [00:03<00:21, 4006092.40tokens/s]
Shard 40:  13%|█▎        | 12966970/100000000 [00:03<00:22, 3799045.19tokens/s]
Shard 40:  13%|█▎        | 13405000/100000000 [00:03<00:21, 3955925.15tokens/s]
Shard 40:  14%|█▍        | 13805765/100000000 [00:04<00:22, 3875593.31tokens/s]
Shard 40:  14%|█▍        | 14208761/100000000 [00:04<00:21, 3918688.20tokens/s]
Shard 40:  15%|█▍        | 14765517/100000000 [00:04<00:19, 4381116.55tokens/s]
Shard 40:  15%|█▌        | 15206758/100000000 [00:04<00:20, 4074653.78tokens/s]
Shard 40:  16%|█▌        | 15624610/100000000 [00:04<00:22, 3763020.52tokens/s]
Shard 40:  16%|█▌        | 16008751/100000000 [00:04<00:23, 3608546.61tokens/s]
Shard 40:  16%|█▋        | 16381662/100000000 [00:04<00:26, 3155064.12tokens/s]
Shard 40:  17%|█▋        | 16767190/100000000 [00:04<00:25, 3327339.13tokens/s]
Shard 40:  17%|█▋        | 17144439/100000000 [00:04<00:24, 3432426.50tokens/s]
Shard 40:  18%|█▊        | 17516255/100000000 [00:05<00:23, 3505666.46tokens/s]
Shard 40:  18%|█▊        | 18033643/100000000 [00:05<00:20, 3963305.54tokens/s]
Shard 40:  18%|█▊        | 18456194/100000000 [00:05<00:20, 4032927.27tokens/s]
Shard 40:  19%|█▉        | 18865759/100000000 [00:05<00:20, 3933088.00tokens/s]
Shard 40:  19%|█▉        | 19328090/100000000 [00:05<00:19, 4130341.23tokens/s]
Shard 40:  20%|█▉        | 19828908/100000000 [00:05<00:18, 4369190.35tokens/s]
Shard 40:  20%|██        | 20270169/100000000 [00:05<00:20, 3945521.18tokens/s]
Shard 40:  21%|██        | 20674606/100000000 [00:05<00:22, 3542030.08tokens/s]
Shard 40:  21%|██        | 21095760/100000000 [00:05<00:21, 3694936.39tokens/s]
Shard 40:  21%|██▏       | 21476566/100000000 [00:06<00:21, 3702240.64tokens/s]
Shard 40:  22%|██▏       | 21855033/100000000 [00:06<00:21, 3619973.62tokens/s]
Shard 40:  22%|██▏       | 22223910/100000000 [00:06<00:26, 2946824.54tokens/s]
Shard 40:  23%|██▎       | 22541121/100000000 [00:06<00:25, 2999033.53tokens/s]
Shard 40:  23%|██▎       | 22894444/100000000 [00:06<00:24, 3124048.10tokens/s]
Shard 40:  24%|██▎       | 23602435/100000000 [00:06<00:18, 4174725.40tokens/s]
Shard 40:  24%|██▍       | 24041693/100000000 [00:06<00:20, 3619179.59tokens/s]
Shard 40:  24%|██▍       | 24430971/100000000 [00:06<00:23, 3245772.62tokens/s]
Shard 40:  25%|██▍       | 24779117/100000000 [00:07<00:23, 3217864.75tokens/s]
Shard 40:  25%|██▌       | 25190029/100000000 [00:07<00:21, 3436341.18tokens/s]
Shard 40:  26%|██▌       | 25608583/100000000 [00:07<00:20, 3623097.53tokens/s]
Shard 40:  26%|██▌       | 26020945/100000000 [00:07<00:19, 3752956.76tokens/s]
Shard 40:  26%|██▋       | 26410171/100000000 [00:07<00:20, 3631203.71tokens/s]
Shard 40:  27%|██▋       | 26824481/100000000 [00:07<00:19, 3765292.28tokens/s]
Shard 40:  27%|██▋       | 27207498/100000000 [00:07<00:20, 3510077.37tokens/s]
Shard 40:  28%|██▊       | 27566448/100000000 [00:07<00:23, 3126275.47tokens/s]
Shard 40:  28%|██▊       | 27939089/100000000 [00:07<00:22, 3261286.24tokens/s]
Shard 40:  28%|██▊       | 28370370/100000000 [00:08<00:20, 3532095.85tokens/s]
Shard 40:  29%|██▊       | 28734429/100000000 [00:08<00:21, 3279196.50tokens/s]
Shard 40:  29%|██▉       | 29072091/100000000 [00:08<00:21, 3234089.87tokens/s]
Shard 40:  29%|██▉       | 29401885/100000000 [00:08<00:22, 3159938.13tokens/s]
Shard 40:  30%|██▉       | 29748283/100000000 [00:08<00:21, 3221630.38tokens/s]
Shard 40:  30%|███       | 30073933/100000000 [00:08<00:23, 2970636.57tokens/s]
Shard 40:  30%|███       | 30377943/100000000 [00:08<00:24, 2799996.61tokens/s]
Shard 40:  31%|███       | 30663088/100000000 [00:08<00:25, 2727991.64tokens/s]
Shard 40:  31%|███       | 30963954/100000000 [00:09<00:24, 2788592.93tokens/s]
Shard 40:  31%|███       | 31245820/100000000 [00:09<00:25, 2733016.10tokens/s]
Shard 40:  32%|███▏      | 31683996/100000000 [00:09<00:21, 3189777.17tokens/s]
Shard 40:  32%|███▏      | 32008332/100000000 [00:09<00:22, 2974065.71tokens/s]
Shard 40:  32%|███▏      | 32314949/100000000 [00:09<00:24, 2784316.59tokens/s]
Shard 40:  33%|███▎      | 32700096/100000000 [00:09<00:21, 3061963.63tokens/s]
Shard 40:  33%|███▎      | 33013722/100000000 [00:09<00:22, 2916149.64tokens/s]
Shard 40:  34%|███▎      | 33530312/100000000 [00:09<00:18, 3520202.45tokens/s]
Shard 40:  34%|███▍      | 33892906/100000000 [00:09<00:19, 3394801.52tokens/s]
Shard 40:  34%|███▍      | 34242596/100000000 [00:10<00:20, 3261144.07tokens/s]
Shard 40:  35%|███▍      | 34580410/100000000 [00:10<00:21, 3094785.88tokens/s]
Shard 40:  35%|███▍      | 34895169/100000000 [00:10<00:21, 2993402.26tokens/s]
Shard 40:  35%|███▌      | 35278296/100000000 [00:10<00:20, 3204322.42tokens/s]
Shard 40:  36%|███▌      | 35643797/100000000 [00:10<00:19, 3324623.35tokens/s]
Shard 40:  36%|███▌      | 36000406/100000000 [00:10<00:18, 3388686.61tokens/s]
Shard 40:  36%|███▋      | 36342959/100000000 [00:10<00:18, 3382205.06tokens/s]
Shard 40:  37%|███▋      | 36685825/100000000 [00:10<00:18, 3366180.41tokens/s]
Shard 40:  37%|███▋      | 37114983/100000000 [00:10<00:17, 3629516.42tokens/s]
Shard 40:  37%|███▋      | 37479687/100000000 [00:10<00:17, 3619954.66tokens/s]
Shard 40:  38%|███▊      | 37847009/100000000 [00:11<00:17, 3528075.87tokens/s]
Shard 40:  38%|███▊      | 38201396/100000000 [00:11<00:18, 3283342.55tokens/s]
Shard 40:  39%|███▊      | 38571820/100000000 [00:11<00:18, 3371516.13tokens/s]
Shard 40:  39%|███▉      | 38942504/100000000 [00:11<00:17, 3423797.37tokens/s]
Shard 40:  39%|███▉      | 39321020/100000000 [00:11<00:17, 3511676.09tokens/s]
Shard 40:  40%|███▉      | 39674323/100000000 [00:11<00:18, 3325640.93tokens/s]
Shard 40:  40%|████      | 40010130/100000000 [00:11<00:19, 3039878.45tokens/s]
Shard 40:  40%|████      | 40319989/100000000 [00:11<00:21, 2774320.68tokens/s]
Shard 40:  41%|████      | 40620049/100000000 [00:12<00:20, 2830424.26tokens/s]
Shard 40:  41%|████      | 40908904/100000000 [00:12<00:23, 2520539.90tokens/s]
Shard 40:  41%|████▏     | 41320264/100000000 [00:12<00:20, 2907826.39tokens/s]
Shard 40:  42%|████▏     | 41631196/100000000 [00:12<00:19, 2956877.74tokens/s]
Shard 40:  42%|████▏     | 41936160/100000000 [00:12<00:19, 2945527.31tokens/s]
Shard 40:  42%|████▏     | 42259541/100000000 [00:12<00:19, 2890539.74tokens/s]
Shard 40:  43%|████▎     | 42679191/100000000 [00:12<00:17, 3247291.23tokens/s]
Shard 40:  43%|████▎     | 43073867/100000000 [00:12<00:16, 3443692.99tokens/s]
Shard 40:  43%|████▎     | 43425172/100000000 [00:12<00:16, 3393418.43tokens/s]
Shard 40:  44%|████▍     | 43768586/100000000 [00:13<00:18, 3091626.80tokens/s]
Shard 40:  44%|████▍     | 44151326/100000000 [00:13<00:16, 3290508.73tokens/s]
Shard 40:  45%|████▍     | 44740737/100000000 [00:13<00:13, 4002523.95tokens/s]
Shard 40:  45%|████▌     | 45151338/100000000 [00:13<00:14, 3855134.39tokens/s]
Shard 40:  46%|████▌     | 45682477/100000000 [00:13<00:12, 4244900.94tokens/s]
Shard 40:  46%|████▌     | 46121139/100000000 [00:13<00:12, 4283768.33tokens/s]
Shard 40:  47%|████▋     | 46555547/100000000 [00:13<00:13, 4031160.50tokens/s]
Shard 40:  47%|████▋     | 46965241/100000000 [00:13<00:15, 3534016.38tokens/s]
Shard 40:  47%|████▋     | 47399883/100000000 [00:13<00:14, 3743439.76tokens/s]
Shard 40:  48%|████▊     | 47823112/100000000 [00:14<00:13, 3866617.92tokens/s]
Shard 40:  48%|████▊     | 48221756/100000000 [00:14<00:13, 3815726.46tokens/s]
Shard 40:  49%|████▊     | 48669676/100000000 [00:14<00:12, 3992717.02tokens/s]
Shard 40:  49%|████▉     | 49076562/100000000 [00:14<00:13, 3643931.80tokens/s]
Shard 40:  49%|████▉     | 49450675/100000000 [00:14<00:14, 3587332.27tokens/s]
Shard 40:  50%|████▉     | 49936558/100000000 [00:14<00:12, 3931445.56tokens/s]
Shard 40:  50%|█████     | 50419600/100000000 [00:14<00:12, 4122667.38tokens/s]
Shard 40:  51%|█████     | 50838514/100000000 [00:14<00:12, 3891464.36tokens/s]
Shard 40:  51%|█████     | 51233707/100000000 [00:14<00:13, 3628089.36tokens/s]
Shard 40:  52%|█████▏    | 51604995/100000000 [00:15<00:13, 3474965.09tokens/s]
Shard 40:  52%|█████▏    | 51957287/100000000 [00:15<00:14, 3237898.71tokens/s]
Shard 40:  52%|█████▏    | 52311037/100000000 [00:15<00:14, 3308080.09tokens/s]
Shard 40:  53%|█████▎    | 52750144/100000000 [00:15<00:13, 3582743.89tokens/s]
Shard 40:  53%|█████▎    | 53143840/100000000 [00:15<00:12, 3681044.69tokens/s]
Shard 40:  54%|█████▎    | 53516955/100000000 [00:15<00:13, 3568146.44tokens/s]
Shard 40:  54%|█████▍    | 53878543/100000000 [00:15<00:13, 3361737.86tokens/s]
Shard 40:  54%|█████▍    | 54218847/100000000 [00:15<00:14, 3150259.27tokens/s]
Shard 40:  55%|█████▍    | 54538397/100000000 [00:15<00:14, 3143530.91tokens/s]
Shard 40:  55%|█████▍    | 54875779/100000000 [00:16<00:14, 3203956.00tokens/s]
Shard 40:  55%|█████▌    | 55199605/100000000 [00:16<00:14, 3047183.33tokens/s]
Shard 40:  56%|█████▌    | 55536285/100000000 [00:16<00:14, 3129835.42tokens/s]
Shard 40:  56%|█████▌    | 55851885/100000000 [00:16<00:15, 2877900.31tokens/s]
Shard 40:  56%|█████▌    | 56186031/100000000 [00:16<00:14, 3000600.17tokens/s]
Shard 40:  57%|█████▋    | 56890343/100000000 [00:16<00:10, 4122664.24tokens/s]
Shard 40:  57%|█████▋    | 57494354/100000000 [00:16<00:09, 4653146.23tokens/s]
Shard 40:  58%|█████▊    | 58119888/100000000 [00:16<00:08, 5094491.39tokens/s]
Shard 40:  59%|█████▊    | 58638450/100000000 [00:16<00:08, 4699286.73tokens/s]
Shard 40:  59%|█████▉    | 59120177/100000000 [00:17<00:09, 4283932.07tokens/s]
Shard 40:  60%|█████▉    | 59562377/100000000 [00:17<00:10, 3896655.91tokens/s]
Shard 40:  60%|█████▉    | 59965827/100000000 [00:17<00:10, 3789097.10tokens/s]
Shard 40:  60%|██████    | 60379009/100000000 [00:17<00:11, 3473558.52tokens/s]
Shard 40:  61%|██████    | 60824482/100000000 [00:17<00:10, 3672669.43tokens/s]
Shard 40:  61%|██████    | 61201960/100000000 [00:17<00:11, 3475112.81tokens/s]
Shard 40:  62%|██████▏   | 61619704/100000000 [00:17<00:10, 3655662.55tokens/s]
Shard 40:  62%|██████▏   | 61993248/100000000 [00:17<00:10, 3515255.84tokens/s]
Shard 40:  62%|██████▏   | 62350636/100000000 [00:18<00:11, 3379967.80tokens/s]
Shard 40:  63%|██████▎   | 62699331/100000000 [00:18<00:12, 2982924.71tokens/s]
Shard 40:  63%|██████▎   | 63008623/100000000 [00:18<00:12, 2956820.16tokens/s]
Shard 40:  63%|██████▎   | 63381307/100000000 [00:18<00:11, 3149527.56tokens/s]
Shard 40:  64%|██████▎   | 63709149/100000000 [00:18<00:12, 3016956.69tokens/s]
Shard 40:  64%|██████▍   | 64016918/100000000 [00:18<00:12, 2937886.47tokens/s]
Shard 40:  64%|██████▍   | 64387440/100000000 [00:18<00:11, 3129753.97tokens/s]
Shard 40:  65%|██████▍   | 64718909/100000000 [00:18<00:11, 3181025.28tokens/s]
Shard 40:  65%|██████▌   | 65041019/100000000 [00:18<00:12, 2840932.71tokens/s]
Shard 40:  65%|██████▌   | 65333394/100000000 [00:19<00:12, 2847140.39tokens/s]
Shard 40:  66%|██████▌   | 65624291/100000000 [00:19<00:12, 2658858.99tokens/s]
Shard 40:  66%|██████▌   | 65895994/100000000 [00:19<00:12, 2646101.35tokens/s]
Shard 40:  66%|██████▌   | 66164700/100000000 [00:19<00:13, 2564393.65tokens/s]
Shard 40:  66%|██████▋   | 66423827/100000000 [00:19<00:13, 2442613.50tokens/s]
Shard 40:  67%|██████▋   | 66725833/100000000 [00:19<00:12, 2592263.59tokens/s]
Shard 40:  67%|██████▋   | 66988785/100000000 [00:19<00:14, 2280515.64tokens/s]
Shard 40:  67%|██████▋   | 67495939/100000000 [00:19<00:10, 3001892.86tokens/s]
Shard 40:  68%|██████▊   | 67813267/100000000 [00:20<00:11, 2773583.89tokens/s]
Shard 40:  68%|██████▊   | 68113126/100000000 [00:20<00:11, 2831924.16tokens/s]
Shard 40:  69%|██████▊   | 68513416/100000000 [00:20<00:10, 3142978.88tokens/s]
Shard 40:  69%|██████▉   | 68926753/100000000 [00:20<00:09, 3400211.99tokens/s]
Shard 40:  69%|██████▉   | 69276197/100000000 [00:20<00:09, 3247954.38tokens/s]
Shard 40:  70%|██████▉   | 69745267/100000000 [00:20<00:08, 3637214.60tokens/s]
Shard 40:  70%|███████   | 70117863/100000000 [00:20<00:09, 3244596.26tokens/s]
Shard 40:  70%|███████   | 70458269/100000000 [00:20<00:09, 3157492.58tokens/s]
Shard 40:  71%|███████   | 70782657/100000000 [00:20<00:09, 3098733.60tokens/s]
Shard 40:  71%|███████   | 71131283/100000000 [00:21<00:09, 3195436.53tokens/s]
Shard 40:  71%|███████▏  | 71475411/100000000 [00:21<00:08, 3263604.24tokens/s]
Shard 40:  72%|███████▏  | 71805891/100000000 [00:21<00:09, 2870683.58tokens/s]
Shard 40:  72%|███████▏  | 72103476/100000000 [00:21<00:10, 2721472.08tokens/s]
Shard 40:  72%|███████▏  | 72383861/100000000 [00:21<00:10, 2708362.80tokens/s]
Shard 40:  73%|███████▎  | 72827703/100000000 [00:21<00:08, 3153159.15tokens/s]
Shard 40:  73%|███████▎  | 73153671/100000000 [00:21<00:08, 3064204.50tokens/s]
Shard 40:  73%|███████▎  | 73495026/100000000 [00:21<00:08, 3158302.40tokens/s]
Shard 40:  74%|███████▍  | 73816195/100000000 [00:21<00:08, 3019051.40tokens/s]
Shard 40:  74%|███████▍  | 74174025/100000000 [00:22<00:08, 3158568.22tokens/s]
Shard 40:  75%|███████▍  | 74781784/100000000 [00:22<00:06, 3969852.76tokens/s]
Shard 40:  75%|███████▌  | 75260032/100000000 [00:22<00:05, 4200649.77tokens/s]
Shard 40:  76%|███████▌  | 75717248/100000000 [00:22<00:05, 4305409.35tokens/s]
Shard 40:  76%|███████▌  | 76153448/100000000 [00:22<00:05, 4101230.27tokens/s]
Shard 40:  77%|███████▋  | 76569023/100000000 [00:22<00:06, 3665620.21tokens/s]
Shard 40:  77%|███████▋  | 76983358/100000000 [00:22<00:06, 3770775.01tokens/s]
Shard 40:  77%|███████▋  | 77370150/100000000 [00:22<00:06, 3503094.91tokens/s]
Shard 40:  78%|███████▊  | 77730395/100000000 [00:22<00:07, 3079072.90tokens/s]
Shard 40:  78%|███████▊  | 78101237/100000000 [00:23<00:06, 3230786.85tokens/s]
Shard 40:  78%|███████▊  | 78437229/100000000 [00:23<00:06, 3091505.60tokens/s]
Shard 40:  79%|███████▉  | 78755281/100000000 [00:23<00:07, 2855594.44tokens/s]
Shard 40:  79%|███████▉  | 79065892/100000000 [00:23<00:07, 2910951.80tokens/s]
Shard 40:  79%|███████▉  | 79363539/100000000 [00:23<00:07, 2865742.16tokens/s]
Shard 40:  80%|███████▉  | 79841578/100000000 [00:23<00:05, 3373860.27tokens/s]
Shard 40:  80%|████████  | 80220009/100000000 [00:23<00:05, 3487001.14tokens/s]
Shard 40:  81%|████████  | 80575408/100000000 [00:23<00:05, 3490904.17tokens/s]
Shard 40:  81%|████████  | 80933897/100000000 [00:23<00:05, 3497154.21tokens/s]
Shard 40:  81%|████████▏ | 81286775/100000000 [00:24<00:05, 3405369.66tokens/s]
Shard 40:  82%|████████▏ | 81662349/100000000 [00:24<00:05, 3495792.23tokens/s]
Shard 40:  82%|████████▏ | 82014213/100000000 [00:24<00:05, 3213121.67tokens/s]
Shard 40:  82%|████████▏ | 82341085/100000000 [00:24<00:05, 3045865.45tokens/s]
Shard 40:  83%|████████▎ | 82664377/100000000 [00:24<00:05, 3089223.47tokens/s]
Shard 40:  83%|████████▎ | 82981564/100000000 [00:24<00:05, 3041730.33tokens/s]
Shard 40:  83%|████████▎ | 83345167/100000000 [00:24<00:05, 3206269.66tokens/s]
Shard 40:  84%|████████▎ | 83675328/100000000 [00:24<00:05, 3228453.74tokens/s]
Shard 40:  84%|████████▍ | 84026778/100000000 [00:24<00:04, 3274758.83tokens/s]
Shard 40:  84%|████████▍ | 84356680/100000000 [00:25<00:05, 2987068.66tokens/s]
Shard 40:  85%|████████▍ | 84670081/100000000 [00:25<00:05, 3020011.14tokens/s]
Shard 40:  85%|████████▌ | 85085267/100000000 [00:25<00:04, 3321545.46tokens/s]
Shard 40:  86%|████████▌ | 85605956/100000000 [00:25<00:03, 3859438.25tokens/s]
Shard 40:  86%|████████▌ | 85998369/100000000 [00:25<00:03, 3875915.98tokens/s]
Shard 40:  86%|████████▋ | 86401291/100000000 [00:25<00:03, 3913909.91tokens/s]
Shard 40:  87%|████████▋ | 86795860/100000000 [00:25<00:03, 3558930.68tokens/s]
Shard 40:  87%|████████▋ | 87163321/100000000 [00:25<00:04, 3085786.36tokens/s]
Shard 40:  88%|████████▊ | 87561054/100000000 [00:25<00:03, 3299639.56tokens/s]
Shard 40:  88%|████████▊ | 87905785/100000000 [00:26<00:03, 3068130.09tokens/s]
Shard 40:  88%|████████▊ | 88224616/100000000 [00:26<00:03, 2997591.41tokens/s]
Shard 40:  89%|████████▊ | 88677013/100000000 [00:26<00:03, 3386030.58tokens/s]
Shard 40:  89%|████████▉ | 89129662/100000000 [00:26<00:02, 3688167.15tokens/s]
Shard 40:  90%|████████▉ | 89510353/100000000 [00:26<00:03, 3424425.12tokens/s]
Shard 40:  90%|████████▉ | 89862950/100000000 [00:26<00:02, 3439817.75tokens/s]
Shard 40:  90%|█████████ | 90214236/100000000 [00:26<00:02, 3370934.75tokens/s]
Shard 40:  91%|█████████ | 90558010/100000000 [00:26<00:02, 3386224.23tokens/s]
Shard 40:  91%|█████████ | 90923337/100000000 [00:26<00:02, 3427922.39tokens/s]
Shard 40:  91%|█████████▏| 91270415/100000000 [00:27<00:02, 3309862.17tokens/s]
Shard 40:  92%|█████████▏| 91683498/100000000 [00:27<00:02, 3537635.26tokens/s]
Shard 40:  92%|█████████▏| 92040791/100000000 [00:27<00:02, 3397878.23tokens/s]
Shard 40:  92%|█████████▏| 92383516/100000000 [00:27<00:02, 3393573.01tokens/s]
Shard 40:  93%|█████████▎| 92786604/100000000 [00:27<00:02, 3573234.74tokens/s]
Shard 40:  93%|█████████▎| 93193182/100000000 [00:27<00:01, 3714693.73tokens/s]
Shard 40:  94%|█████████▎| 93608713/100000000 [00:27<00:01, 3821688.44tokens/s]
Shard 40:  94%|█████████▍| 93994204/100000000 [00:27<00:01, 3564774.22tokens/s]
Shard 40:  94%|█████████▍| 94419150/100000000 [00:27<00:01, 3754603.02tokens/s]
Shard 40:  95%|█████████▍| 94855826/100000000 [00:28<00:01, 3908826.28tokens/s]
Shard 40:  95%|█████████▌| 95258002/100000000 [00:28<00:01, 3494211.47tokens/s]
Shard 40:  96%|█████████▌| 95618376/100000000 [00:28<00:01, 3323010.56tokens/s]
Shard 40:  96%|█████████▌| 95958819/100000000 [00:28<00:01, 3231002.55tokens/s]
Shard 40:  96%|█████████▋| 96378953/100000000 [00:28<00:01, 3486362.63tokens/s]
Shard 40:  97%|█████████▋| 96813644/100000000 [00:28<00:00, 3719440.84tokens/s]
Shard 40:  97%|█████████▋| 97205237/100000000 [00:28<00:00, 3772351.74tokens/s]
Shard 40:  98%|█████████▊| 97796358/100000000 [00:28<00:00, 4381271.91tokens/s]
Shard 40:  98%|█████████▊| 98240294/100000000 [00:28<00:00, 4344915.03tokens/s]
Shard 40:  99%|█████████▊| 98680196/100000000 [00:29<00:00, 4122124.34tokens/s]
Shard 40:  99%|█████████▉| 99097340/100000000 [00:29<00:00, 3830741.93tokens/s]
Shard 40:  99%|█████████▉| 99486877/100000000 [00:29<00:00, 3630732.90tokens/s]
Shard 40: 100%|█████████▉| 99858841/100000000 [00:29<00:00, 3247024.46tokens/s]
Shard 40: 100%|█████████▉| 99999529/100000000 [00:37<00:00, 2651986.93tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000040.bin (200,001,024 bytes) in the gpt-2 format

Shard 41:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 41:   0%|          | 260968/100000000 [00:00<00:40, 2490685.64tokens/s]
Shard 41:   1%|          | 511147/100000000 [00:00<00:41, 2385254.19tokens/s]
Shard 41:   1%|          | 750204/100000000 [00:00<00:45, 2166057.87tokens/s]
Shard 41:   1%|          | 977732/100000000 [00:00<00:46, 2148396.29tokens/s]
Shard 41:   1%|          | 1194044/100000000 [00:00<00:48, 2026616.91tokens/s]
Shard 41:   2%|▏         | 1534365/100000000 [00:00<00:40, 2442005.56tokens/s]
Shard 41:   2%|▏         | 1785429/100000000 [00:00<00:42, 2285310.78tokens/s]
Shard 41:   2%|▏         | 2120905/100000000 [00:00<00:37, 2583056.55tokens/s]
Shard 41:   3%|▎         | 2782684/100000000 [00:00<00:26, 3704972.25tokens/s]
Shard 41:   3%|▎         | 3181609/100000000 [00:01<00:29, 3253675.86tokens/s]
Shard 41:   4%|▍         | 3813762/100000000 [00:01<00:23, 4047114.22tokens/s]
Shard 41:   4%|▍         | 4241384/100000000 [00:01<00:26, 3609525.55tokens/s]
Shard 41:   5%|▍         | 4625765/100000000 [00:01<00:30, 3152692.47tokens/s]
Shard 41:   5%|▍         | 4964898/100000000 [00:01<00:31, 3030825.01tokens/s]
Shard 41:   5%|▌         | 5368747/100000000 [00:01<00:29, 3257798.33tokens/s]
Shard 41:   6%|▌         | 5710057/100000000 [00:01<00:32, 2915507.51tokens/s]
Shard 41:   6%|▌         | 6016952/100000000 [00:02<00:32, 2922502.49tokens/s]
Shard 41:   6%|▋         | 6320060/100000000 [00:02<00:34, 2733331.87tokens/s]
Shard 41:   7%|▋         | 6905447/100000000 [00:02<00:26, 3518406.49tokens/s]
Shard 41:   7%|▋         | 7276784/100000000 [00:02<00:27, 3371394.14tokens/s]
Shard 41:   8%|▊         | 7645437/100000000 [00:02<00:26, 3437191.75tokens/s]
Shard 41:   8%|▊         | 8054174/100000000 [00:02<00:25, 3611843.08tokens/s]
Shard 41:   8%|▊         | 8484455/100000000 [00:02<00:24, 3802881.98tokens/s]
Shard 41:   9%|▉         | 8904795/100000000 [00:02<00:23, 3913938.80tokens/s]
Shard 41:   9%|▉         | 9302325/100000000 [00:02<00:24, 3650700.03tokens/s]
Shard 41:  10%|▉         | 9675559/100000000 [00:03<00:25, 3593975.22tokens/s]
Shard 41:  10%|█         | 10039936/100000000 [00:03<00:28, 3206803.64tokens/s]
Shard 41:  10%|█         | 10382749/100000000 [00:03<00:27, 3242369.29tokens/s]
Shard 41:  11%|█         | 10715766/100000000 [00:03<00:27, 3236290.60tokens/s]
Shard 41:  11%|█         | 11106467/100000000 [00:03<00:26, 3403822.68tokens/s]
Shard 41:  11%|█▏        | 11466206/100000000 [00:03<00:25, 3457517.41tokens/s]
Shard 41:  12%|█▏        | 11815985/100000000 [00:03<00:27, 3254608.90tokens/s]
Shard 41:  12%|█▏        | 12237008/100000000 [00:03<00:24, 3518336.26tokens/s]
Shard 41:  13%|█▎        | 12594619/100000000 [00:03<00:25, 3470878.57tokens/s]
Shard 41:  13%|█▎        | 12945781/100000000 [00:04<00:25, 3371984.29tokens/s]
Shard 41:  13%|█▎        | 13337834/100000000 [00:04<00:24, 3519147.16tokens/s]
Shard 41:  14%|█▎        | 13692597/100000000 [00:04<00:25, 3425389.41tokens/s]
Shard 41:  14%|█▍        | 14074976/100000000 [00:04<00:24, 3526453.72tokens/s]
Shard 41:  14%|█▍        | 14431015/100000000 [00:04<00:26, 3250573.83tokens/s]
Shard 41:  15%|█▍        | 14844187/100000000 [00:04<00:24, 3471582.67tokens/s]
Shard 41:  15%|█▌        | 15370515/100000000 [00:04<00:21, 3961196.93tokens/s]
Shard 41:  16%|█▌        | 15850576/100000000 [00:04<00:20, 4193907.83tokens/s]
Shard 41:  16%|█▋        | 16277659/100000000 [00:04<00:19, 4195161.25tokens/s]
Shard 41:  17%|█▋        | 16701512/100000000 [00:05<00:21, 3938082.62tokens/s]
Shard 41:  17%|█▋        | 17104418/100000000 [00:05<00:21, 3840991.02tokens/s]
Shard 41:  17%|█▋        | 17492828/100000000 [00:05<00:21, 3841708.08tokens/s]
Shard 41:  18%|█▊        | 17879817/100000000 [00:05<00:22, 3592499.17tokens/s]
Shard 41:  18%|█▊        | 18252705/100000000 [00:05<00:22, 3619189.51tokens/s]
Shard 41:  19%|█▊        | 18739331/100000000 [00:05<00:20, 3948674.25tokens/s]
Shard 41:  19%|█▉        | 19160936/100000000 [00:05<00:20, 4017519.46tokens/s]
Shard 41:  20%|█▉        | 19566020/100000000 [00:05<00:20, 3987209.11tokens/s]
Shard 41:  20%|█▉        | 19977356/100000000 [00:05<00:20, 3957585.05tokens/s]
Shard 41:  20%|██        | 20388706/100000000 [00:05<00:20, 3977726.47tokens/s]
Shard 41:  21%|██        | 20796917/100000000 [00:06<00:20, 3776902.03tokens/s]
Shard 41:  21%|██        | 21177357/100000000 [00:06<00:20, 3767539.18tokens/s]
Shard 41:  22%|██▏       | 21556085/100000000 [00:06<00:21, 3634587.41tokens/s]
Shard 41:  22%|██▏       | 21922048/100000000 [00:06<00:22, 3478053.84tokens/s]
Shard 41:  22%|██▏       | 22274159/100000000 [00:06<00:24, 3218254.64tokens/s]
Shard 41:  23%|██▎       | 22600488/100000000 [00:06<00:24, 3168426.87tokens/s]
Shard 41:  23%|██▎       | 23014586/100000000 [00:06<00:22, 3430451.26tokens/s]
Shard 41:  23%|██▎       | 23363962/100000000 [00:06<00:23, 3279450.51tokens/s]
Shard 41:  24%|██▎       | 23731236/100000000 [00:06<00:22, 3386574.13tokens/s]
Shard 41:  24%|██▍       | 24074065/100000000 [00:07<00:23, 3212895.92tokens/s]
Shard 41:  24%|██▍       | 24399292/100000000 [00:07<00:24, 3065994.99tokens/s]
Shard 41:  25%|██▍       | 24709045/100000000 [00:07<00:24, 3063513.02tokens/s]
Shard 41:  25%|██▌       | 25092100/100000000 [00:07<00:22, 3265448.31tokens/s]
Shard 41:  26%|██▌       | 25520400/100000000 [00:07<00:21, 3539017.87tokens/s]
Shard 41:  26%|██▌       | 25881502/100000000 [00:07<00:20, 3541344.92tokens/s]
Shard 41:  26%|██▌       | 26237861/100000000 [00:07<00:21, 3361978.70tokens/s]
Shard 41:  27%|██▋       | 26586906/100000000 [00:07<00:21, 3392090.28tokens/s]
Shard 41:  27%|██▋       | 26929753/100000000 [00:07<00:21, 3393175.39tokens/s]
Shard 41:  27%|██▋       | 27348098/100000000 [00:08<00:20, 3608132.73tokens/s]
Shard 41:  28%|██▊       | 27788308/100000000 [00:08<00:18, 3816837.54tokens/s]
Shard 41:  28%|██▊       | 28246147/100000000 [00:08<00:17, 3991177.62tokens/s]
Shard 41:  29%|██▊       | 28647105/100000000 [00:08<00:18, 3838262.12tokens/s]
Shard 41:  29%|██▉       | 29136653/100000000 [00:08<00:17, 4141091.48tokens/s]
Shard 41:  30%|██▉       | 29664303/100000000 [00:08<00:15, 4469970.26tokens/s]
Shard 41:  30%|███       | 30114963/100000000 [00:08<00:17, 4099079.04tokens/s]
Shard 41:  31%|███       | 30532586/100000000 [00:08<00:18, 3808687.88tokens/s]
Shard 41:  31%|███       | 30921614/100000000 [00:08<00:18, 3719682.36tokens/s]
Shard 41:  31%|███▏      | 31299121/100000000 [00:09<00:18, 3643687.93tokens/s]
Shard 41:  32%|███▏      | 31667070/100000000 [00:09<00:18, 3611013.32tokens/s]
Shard 41:  32%|███▏      | 32455854/100000000 [00:09<00:14, 4793338.89tokens/s]
Shard 41:  33%|███▎      | 32947033/100000000 [00:09<00:14, 4479596.90tokens/s]
Shard 41:  33%|███▎      | 33405616/100000000 [00:09<00:16, 4045253.91tokens/s]
Shard 41:  34%|███▍      | 33823538/100000000 [00:09<00:17, 3793404.98tokens/s]
Shard 41:  34%|███▍      | 34213258/100000000 [00:09<00:17, 3766409.45tokens/s]
Shard 41:  35%|███▍      | 34845187/100000000 [00:09<00:14, 4435569.29tokens/s]
Shard 41:  35%|███▌      | 35302650/100000000 [00:09<00:15, 4167725.62tokens/s]
Shard 41:  36%|███▌      | 35738581/100000000 [00:10<00:15, 4218460.39tokens/s]
Shard 41:  36%|███▌      | 36169406/100000000 [00:10<00:15, 4006166.33tokens/s]
Shard 41:  37%|███▋      | 36735516/100000000 [00:10<00:14, 4450255.73tokens/s]
Shard 41:  37%|███▋      | 37242234/100000000 [00:10<00:13, 4609776.26tokens/s]
Shard 41:  38%|███▊      | 37821851/100000000 [00:10<00:12, 4937414.23tokens/s]
Shard 41:  38%|███▊      | 38467232/100000000 [00:10<00:11, 5339455.56tokens/s]
Shard 41:  39%|███▉      | 39056684/100000000 [00:10<00:11, 5484802.90tokens/s]
Shard 41:  40%|███▉      | 39843446/100000000 [00:10<00:09, 6124067.84tokens/s]
Shard 41:  40%|████      | 40464829/100000000 [00:10<00:11, 5402028.20tokens/s]
Shard 41:  41%|████      | 41022698/100000000 [00:11<00:12, 4609057.84tokens/s]
Shard 41:  42%|████▏     | 41528444/100000000 [00:11<00:13, 4457672.25tokens/s]
Shard 41:  42%|████▏     | 42062952/100000000 [00:11<00:12, 4666499.11tokens/s]
Shard 41:  43%|████▎     | 42575487/100000000 [00:11<00:12, 4783624.84tokens/s]
Shard 41:  43%|████▎     | 43069296/100000000 [00:11<00:11, 4767549.55tokens/s]
Shard 41:  44%|████▎     | 43701881/100000000 [00:11<00:10, 5185460.50tokens/s]
Shard 41:  44%|████▍     | 44232471/100000000 [00:11<00:11, 4973953.05tokens/s]
Shard 41:  45%|████▍     | 44771776/100000000 [00:11<00:11, 4991252.65tokens/s]
Shard 41:  45%|████▌     | 45354997/100000000 [00:11<00:10, 5226831.26tokens/s]
Shard 41:  46%|████▌     | 45883618/100000000 [00:12<00:12, 4415878.39tokens/s]
Shard 41:  46%|████▋     | 46393181/100000000 [00:12<00:11, 4585353.45tokens/s]
Shard 41:  47%|████▋     | 46871842/100000000 [00:12<00:11, 4483409.90tokens/s]
Shard 41:  47%|████▋     | 47334908/100000000 [00:12<00:11, 4495880.94tokens/s]
Shard 41:  48%|████▊     | 47794088/100000000 [00:12<00:11, 4473263.21tokens/s]
Shard 41:  48%|████▊     | 48288608/100000000 [00:12<00:11, 4593364.11tokens/s]
Shard 41:  49%|████▉     | 48760807/100000000 [00:12<00:12, 3942632.92tokens/s]
Shard 41:  49%|████▉     | 49174335/100000000 [00:13<00:15, 3288924.27tokens/s]
Shard 41:  50%|████▉     | 49558912/100000000 [00:13<00:14, 3415570.28tokens/s]
Shard 41:  50%|████▉     | 49945380/100000000 [00:13<00:14, 3527285.05tokens/s]
Shard 41:  50%|█████     | 50316721/100000000 [00:13<00:14, 3348505.86tokens/s]
Shard 41:  51%|█████     | 50665275/100000000 [00:13<00:19, 2530741.64tokens/s]
Shard 41:  51%|█████     | 51015198/100000000 [00:13<00:17, 2738038.58tokens/s]
Shard 41:  51%|█████▏    | 51322577/100000000 [00:13<00:19, 2497684.25tokens/s]
Shard 41:  52%|█████▏    | 51597491/100000000 [00:13<00:19, 2449694.00tokens/s]
Shard 41:  52%|█████▏    | 51878804/100000000 [00:14<00:19, 2530084.04tokens/s]
Shard 41:  52%|█████▏    | 52178917/100000000 [00:14<00:18, 2641595.65tokens/s]
Shard 41:  52%|█████▏    | 52456129/100000000 [00:14<00:19, 2466354.06tokens/s]
Shard 41:  53%|█████▎    | 52719570/100000000 [00:14<00:18, 2500036.26tokens/s]
Shard 41:  53%|█████▎    | 52976870/100000000 [00:14<00:19, 2382076.75tokens/s]
Shard 41:  53%|█████▎    | 53273150/100000000 [00:14<00:18, 2532218.28tokens/s]
Shard 41:  54%|█████▎    | 53643156/100000000 [00:14<00:16, 2846015.42tokens/s]
Shard 41:  54%|█████▍    | 54087464/100000000 [00:14<00:13, 3284713.27tokens/s]
Shard 41:  54%|█████▍    | 54423040/100000000 [00:14<00:14, 3237689.52tokens/s]
Shard 41:  55%|█████▍    | 54936104/100000000 [00:15<00:11, 3766814.20tokens/s]
Shard 41:  55%|█████▌    | 55324156/100000000 [00:15<00:12, 3708528.03tokens/s]
Shard 41:  56%|█████▋    | 56263341/100000000 [00:15<00:08, 5318916.68tokens/s]
Shard 41:  57%|█████▋    | 56803828/100000000 [00:15<00:08, 4968230.86tokens/s]
Shard 41:  57%|█████▋    | 57310662/100000000 [00:15<00:09, 4372933.38tokens/s]
Shard 41:  58%|█████▊    | 57866774/100000000 [00:15<00:09, 4666595.55tokens/s]
Shard 41:  58%|█████▊    | 58351092/100000000 [00:15<00:10, 4140025.42tokens/s]
Shard 41:  59%|█████▉    | 58786837/100000000 [00:15<00:10, 3784842.32tokens/s]
Shard 41:  59%|█████▉    | 59183052/100000000 [00:16<00:11, 3510097.63tokens/s]
Shard 41:  60%|█████▉    | 59546879/100000000 [00:16<00:12, 3316714.88tokens/s]
Shard 41:  60%|█████▉    | 59907636/100000000 [00:16<00:11, 3376369.98tokens/s]
Shard 41:  60%|██████    | 60286530/100000000 [00:16<00:11, 3474815.93tokens/s]
Shard 41:  61%|██████    | 60743203/100000000 [00:16<00:10, 3727658.88tokens/s]
Shard 41:  61%|██████▏   | 61361515/100000000 [00:16<00:08, 4392255.92tokens/s]
Shard 41:  62%|██████▏   | 61812558/100000000 [00:16<00:08, 4293884.94tokens/s]
Shard 41:  62%|██████▏   | 62249075/100000000 [00:16<00:09, 4010279.80tokens/s]
Shard 41:  63%|██████▎   | 62714129/100000000 [00:16<00:08, 4156158.85tokens/s]
Shard 41:  63%|██████▎   | 63137402/100000000 [00:17<00:09, 3933545.08tokens/s]
Shard 41:  64%|██████▎   | 63537397/100000000 [00:17<00:09, 3696613.36tokens/s]
Shard 41:  64%|██████▍   | 63913120/100000000 [00:17<00:10, 3342230.68tokens/s]
Shard 41:  64%|██████▍   | 64255763/100000000 [00:17<00:10, 3315908.83tokens/s]
Shard 41:  65%|██████▍   | 64593169/100000000 [00:17<00:10, 3279623.71tokens/s]
Shard 41:  65%|██████▍   | 64924855/100000000 [00:17<00:10, 3257480.72tokens/s]
Shard 41:  65%|██████▌   | 65344423/100000000 [00:17<00:09, 3514282.99tokens/s]
Shard 41:  66%|██████▌   | 65699840/100000000 [00:17<00:10, 3179140.70tokens/s]
Shard 41:  66%|██████▌   | 66025495/100000000 [00:17<00:10, 3183953.29tokens/s]
Shard 41:  66%|██████▋   | 66349444/100000000 [00:18<00:10, 3169479.15tokens/s]
Shard 41:  67%|██████▋   | 66922005/100000000 [00:18<00:08, 3873589.34tokens/s]
Shard 41:  67%|██████▋   | 67353025/100000000 [00:18<00:08, 3975089.18tokens/s]
Shard 41:  68%|██████▊   | 67797829/100000000 [00:18<00:07, 4093744.05tokens/s]
Shard 41:  68%|██████▊   | 68246254/100000000 [00:18<00:07, 4180520.58tokens/s]
Shard 41:  69%|██████▊   | 68669631/100000000 [00:18<00:07, 4194883.41tokens/s]
Shard 41:  69%|██████▉   | 69091952/100000000 [00:18<00:08, 3603464.81tokens/s]
Shard 41:  69%|██████▉   | 69469098/100000000 [00:18<00:08, 3552279.02tokens/s]
Shard 41:  70%|██████▉   | 69835083/100000000 [00:18<00:08, 3443978.49tokens/s]
Shard 41:  70%|███████   | 70187500/100000000 [00:19<00:08, 3436484.50tokens/s]
Shard 41:  71%|███████   | 70540539/100000000 [00:19<00:08, 3304425.74tokens/s]
Shard 41:  71%|███████   | 70878518/100000000 [00:19<00:08, 3261577.63tokens/s]
Shard 41:  71%|███████   | 71207812/100000000 [00:19<00:08, 3204091.70tokens/s]
Shard 41:  72%|███████▏  | 71530352/100000000 [00:19<00:09, 3153637.54tokens/s]
Shard 41:  72%|███████▏  | 71847770/100000000 [00:19<00:08, 3151676.71tokens/s]
Shard 41:  72%|███████▏  | 72168552/100000000 [00:19<00:08, 3156418.70tokens/s]
Shard 41:  72%|███████▏  | 72485001/100000000 [00:20<00:21, 1290863.33tokens/s]
Shard 41:  73%|███████▎  | 72852075/100000000 [00:20<00:16, 1633366.68tokens/s]
Shard 41:  73%|███████▎  | 73129134/100000000 [00:20<00:14, 1824687.66tokens/s]
Shard 41:  74%|███████▎  | 73503383/100000000 [00:20<00:12, 2193183.73tokens/s]
Shard 41:  74%|███████▍  | 73885823/100000000 [00:20<00:10, 2534662.82tokens/s]
Shard 41:  74%|███████▍  | 74218663/100000000 [00:20<00:09, 2722342.90tokens/s]
Shard 41:  75%|███████▍  | 74695605/100000000 [00:20<00:07, 3236912.05tokens/s]
Shard 41:  75%|███████▌  | 75066802/100000000 [00:21<00:07, 3140528.46tokens/s]
Shard 41:  75%|███████▌  | 75426561/100000000 [00:21<00:07, 3256748.03tokens/s]
Shard 41:  76%|███████▌  | 75807598/100000000 [00:21<00:07, 3402498.00tokens/s]
Shard 41:  77%|███████▋  | 76570738/100000000 [00:21<00:05, 4583405.91tokens/s]
Shard 41:  77%|███████▋  | 77063558/100000000 [00:21<00:05, 4481300.11tokens/s]
Shard 41:  78%|███████▊  | 77640076/100000000 [00:21<00:04, 4789795.88tokens/s]
Shard 41:  78%|███████▊  | 78133353/100000000 [00:21<00:04, 4656282.38tokens/s]
Shard 41:  79%|███████▊  | 78627303/100000000 [00:21<00:04, 4717678.12tokens/s]
Shard 41:  79%|███████▉  | 79106724/100000000 [00:21<00:04, 4642991.15tokens/s]
Shard 41:  80%|███████▉  | 79576157/100000000 [00:22<00:05, 3766001.04tokens/s]
Shard 41:  80%|███████▉  | 79984271/100000000 [00:22<00:05, 3395659.38tokens/s]
Shard 41:  80%|████████  | 80351352/100000000 [00:22<00:06, 3097005.93tokens/s]
Shard 41:  81%|████████  | 80694857/100000000 [00:22<00:06, 3125938.04tokens/s]
Shard 41:  81%|████████  | 81021948/100000000 [00:22<00:06, 2811828.83tokens/s]
Shard 41:  82%|████████▏ | 81636728/100000000 [00:22<00:05, 3600270.90tokens/s]
Shard 41:  82%|████████▏ | 82174013/100000000 [00:22<00:04, 4044944.69tokens/s]
Shard 41:  83%|████████▎ | 82606382/100000000 [00:22<00:04, 3854254.81tokens/s]
Shard 41:  83%|████████▎ | 83015373/100000000 [00:23<00:04, 3746896.50tokens/s]
Shard 41:  83%|████████▎ | 83404497/100000000 [00:23<00:04, 3573914.71tokens/s]
Shard 41:  84%|████████▍ | 83864623/100000000 [00:23<00:04, 3833701.60tokens/s]
Shard 41:  84%|████████▍ | 84293954/100000000 [00:23<00:03, 3954416.72tokens/s]
Shard 41:  85%|████████▍ | 84701778/100000000 [00:23<00:03, 3981336.71tokens/s]
Shard 41:  86%|████████▌ | 85616414/100000000 [00:23<00:02, 5449686.27tokens/s]
Shard 41:  86%|████████▌ | 86173489/100000000 [00:23<00:03, 3925440.50tokens/s]
Shard 41:  87%|████████▋ | 86710009/100000000 [00:23<00:03, 4253202.98tokens/s]
Shard 41:  87%|████████▋ | 87193964/100000000 [00:24<00:03, 3471211.56tokens/s]
Shard 41:  88%|████████▊ | 87601897/100000000 [00:24<00:03, 3509877.31tokens/s]
Shard 41:  88%|████████▊ | 88007423/100000000 [00:24<00:03, 3624653.22tokens/s]
Shard 41:  88%|████████▊ | 88403710/100000000 [00:24<00:03, 3470040.63tokens/s]
Shard 41:  89%|████████▉ | 88774675/100000000 [00:24<00:03, 3442035.62tokens/s]
Shard 41:  89%|████████▉ | 89139000/100000000 [00:24<00:03, 3098651.04tokens/s]
Shard 41:  90%|████████▉ | 89538693/100000000 [00:24<00:03, 3304702.24tokens/s]
Shard 41:  90%|████████▉ | 89884247/100000000 [00:24<00:03, 3071370.69tokens/s]
Shard 41:  91%|█████████ | 90680959/100000000 [00:25<00:02, 4303770.61tokens/s]
Shard 41:  91%|█████████ | 91145466/100000000 [00:25<00:02, 4292802.78tokens/s]
Shard 41:  92%|█████████▏| 91596532/100000000 [00:25<00:02, 3671327.04tokens/s]
Shard 41:  92%|█████████▏| 91992039/100000000 [00:25<00:02, 3508793.56tokens/s]
Shard 41:  92%|█████████▏| 92362546/100000000 [00:25<00:02, 3250774.66tokens/s]
Shard 41:  93%|█████████▎| 92702414/100000000 [00:25<00:02, 2964003.19tokens/s]
Shard 41:  93%|█████████▎| 93373381/100000000 [00:25<00:01, 3857514.13tokens/s]
Shard 41:  94%|█████████▍| 93790336/100000000 [00:25<00:01, 3583988.38tokens/s]
Shard 41:  94%|█████████▍| 94271198/100000000 [00:26<00:01, 3877372.90tokens/s]
Shard 41:  95%|█████████▍| 94711262/100000000 [00:26<00:01, 4002465.51tokens/s]
Shard 41:  95%|█████████▌| 95278628/100000000 [00:26<00:01, 4449095.84tokens/s]
Shard 41:  96%|█████████▌| 95740352/100000000 [00:26<00:01, 4216437.76tokens/s]
Shard 41:  96%|█████████▌| 96176885/100000000 [00:26<00:01, 3289681.95tokens/s]
Shard 41:  97%|█████████▋| 96544379/100000000 [00:26<00:01, 2973862.25tokens/s]
Shard 41:  97%|█████████▋| 96871431/100000000 [00:26<00:01, 2535703.56tokens/s]
Shard 41:  97%|█████████▋| 97153584/100000000 [00:27<00:01, 2378633.10tokens/s]
Shard 41:  97%|█████████▋| 97409602/100000000 [00:27<00:01, 2417976.07tokens/s]
Shard 41:  98%|█████████▊| 97667800/100000000 [00:27<00:01, 2270256.31tokens/s]
Shard 41:  98%|█████████▊| 97904364/100000000 [00:27<00:00, 2278758.31tokens/s]
Shard 41:  98%|█████████▊| 98158158/100000000 [00:27<00:00, 2332085.18tokens/s]
Shard 41:  98%|█████████▊| 98397101/100000000 [00:27<00:00, 2081182.77tokens/s]
Shard 41:  99%|█████████▊| 98613186/100000000 [00:27<00:00, 1982315.24tokens/s]
Shard 41:  99%|█████████▉| 98925941/100000000 [00:27<00:00, 2260305.47tokens/s]
Shard 41:  99%|█████████▉| 99201817/100000000 [00:28<00:00, 2387991.69tokens/s]
Shard 41:  99%|█████████▉| 99448184/100000000 [00:28<00:00, 2236365.89tokens/s]
Shard 41: 100%|█████████▉| 99678217/100000000 [00:28<00:00, 2124913.27tokens/s]
Shard 41: 100%|█████████▉| 99895538/100000000 [00:28<00:00, 2078872.08tokens/s]
Shard 41: 100%|█████████▉| 99999319/100000000 [00:36<00:00, 2724005.47tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000041.bin (200,001,024 bytes) in the gpt-2 format

Shard 42:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 42:   0%|          | 282889/100000000 [00:00<00:35, 2825665.71tokens/s]
Shard 42:   1%|          | 565536/100000000 [00:00<00:38, 2563702.28tokens/s]
Shard 42:   1%|          | 823876/100000000 [00:00<00:39, 2518741.02tokens/s]
Shard 42:   1%|          | 1076520/100000000 [00:00<00:46, 2143141.36tokens/s]
Shard 42:   1%|▏         | 1457643/100000000 [00:00<00:36, 2665881.58tokens/s]
Shard 42:   2%|▏         | 1835352/100000000 [00:00<00:32, 3009626.25tokens/s]
Shard 42:   2%|▏         | 2221506/100000000 [00:00<00:29, 3267758.55tokens/s]
Shard 42:   3%|▎         | 2587626/100000000 [00:00<00:28, 3387500.15tokens/s]
Shard 42:   3%|▎         | 2933486/100000000 [00:00<00:29, 3293274.73tokens/s]
Shard 42:   3%|▎         | 3322413/100000000 [00:01<00:27, 3466798.98tokens/s]
Shard 42:   4%|▎         | 3673635/100000000 [00:01<00:31, 3019721.54tokens/s]
Shard 42:   4%|▍         | 3996670/100000000 [00:01<00:33, 2891686.18tokens/s]
Shard 42:   4%|▍         | 4338217/100000000 [00:01<00:31, 2996382.20tokens/s]
Shard 42:   5%|▍         | 4645405/100000000 [00:01<00:33, 2842956.43tokens/s]
Shard 42:   5%|▍         | 4936400/100000000 [00:01<00:40, 2346072.48tokens/s]
Shard 42:   5%|▌         | 5472097/100000000 [00:01<00:30, 3053713.10tokens/s]
Shard 42:   6%|▌         | 5827604/100000000 [00:01<00:29, 3178269.58tokens/s]
Shard 42:   6%|▌         | 6167912/100000000 [00:02<00:31, 3013962.72tokens/s]
Shard 42:   6%|▋         | 6486027/100000000 [00:02<00:33, 2767411.89tokens/s]
Shard 42:   7%|▋         | 7254282/100000000 [00:02<00:23, 3999115.18tokens/s]
Shard 42:   8%|▊         | 7691634/100000000 [00:02<00:23, 4002765.84tokens/s]
Shard 42:   8%|▊         | 8114462/100000000 [00:02<00:25, 3661642.00tokens/s]
Shard 42:   9%|▊         | 8500494/100000000 [00:02<00:28, 3191847.96tokens/s]
Shard 42:   9%|▉         | 8842324/100000000 [00:02<00:36, 2474235.83tokens/s]
Shard 42:   9%|▉         | 9125890/100000000 [00:03<00:36, 2475888.77tokens/s]
Shard 42:  10%|▉         | 9600451/100000000 [00:03<00:30, 2982896.12tokens/s]
Shard 42:  10%|▉         | 9933928/100000000 [00:03<00:31, 2860618.49tokens/s]
Shard 42:  10%|█         | 10246175/100000000 [00:03<00:30, 2900052.13tokens/s]
Shard 42:  11%|█         | 10689682/100000000 [00:03<00:27, 3289536.24tokens/s]
Shard 42:  11%|█         | 11141983/100000000 [00:03<00:24, 3612229.98tokens/s]
Shard 42:  12%|█▏        | 11585408/100000000 [00:03<00:23, 3838903.42tokens/s]
Shard 42:  12%|█▏        | 12453461/100000000 [00:03<00:16, 5209254.46tokens/s]
Shard 42:  13%|█▎        | 12992662/100000000 [00:03<00:18, 4692603.64tokens/s]
Shard 42:  14%|█▎        | 13632129/100000000 [00:04<00:16, 5123944.00tokens/s]
Shard 42:  14%|█▍        | 14167985/100000000 [00:04<00:17, 4888576.33tokens/s]
Shard 42:  15%|█▍        | 14672439/100000000 [00:04<00:19, 4304919.13tokens/s]
Shard 42:  15%|█▌        | 15123748/100000000 [00:04<00:20, 4080608.07tokens/s]
Shard 42:  16%|█▌        | 15588989/100000000 [00:04<00:20, 4219894.65tokens/s]
Shard 42:  16%|█▌        | 16025024/100000000 [00:04<00:20, 4086236.00tokens/s]
Shard 42:  16%|█▋        | 16442414/100000000 [00:04<00:20, 4074529.42tokens/s]
Shard 42:  17%|█▋        | 16912556/100000000 [00:04<00:19, 4238092.45tokens/s]
Shard 42:  17%|█▋        | 17378720/100000000 [00:04<00:18, 4356876.67tokens/s]
Shard 42:  18%|█▊        | 17847175/100000000 [00:05<00:18, 4419044.31tokens/s]
Shard 42:  18%|█▊        | 18292679/100000000 [00:05<00:18, 4400103.34tokens/s]
Shard 42:  19%|█▊        | 18735151/100000000 [00:05<00:21, 3853070.65tokens/s]
Shard 42:  19%|█▉        | 19136001/100000000 [00:05<00:21, 3849785.14tokens/s]
Shard 42:  20%|█▉        | 19530621/100000000 [00:05<00:22, 3592781.79tokens/s]
Shard 42:  20%|█▉        | 19898841/100000000 [00:05<00:24, 3288162.59tokens/s]
Shard 42:  20%|██        | 20237647/100000000 [00:05<00:24, 3291729.76tokens/s]
Shard 42:  21%|██        | 20573707/100000000 [00:05<00:25, 3166523.60tokens/s]
Shard 42:  21%|██        | 21005292/100000000 [00:06<00:22, 3467534.44tokens/s]
Shard 42:  21%|██▏       | 21379229/100000000 [00:06<00:22, 3531297.48tokens/s]
Shard 42:  22%|██▏       | 21820691/100000000 [00:06<00:20, 3768266.33tokens/s]
Shard 42:  22%|██▏       | 22344369/100000000 [00:06<00:18, 4175003.09tokens/s]
Shard 42:  23%|██▎       | 22767128/100000000 [00:06<00:20, 3830391.74tokens/s]
Shard 42:  23%|██▎       | 23175430/100000000 [00:06<00:19, 3845967.72tokens/s]
Shard 42:  24%|██▎       | 23587877/100000000 [00:06<00:19, 3923638.40tokens/s]
Shard 42:  24%|██▍       | 24158656/100000000 [00:06<00:17, 4431940.28tokens/s]
Shard 42:  25%|██▍       | 24798318/100000000 [00:06<00:15, 4963202.33tokens/s]
Shard 42:  25%|██▌       | 25300380/100000000 [00:07<00:17, 4156947.81tokens/s]
Shard 42:  26%|██▌       | 25742695/100000000 [00:07<00:19, 3764802.87tokens/s]
Shard 42:  26%|██▌       | 26141457/100000000 [00:07<00:19, 3791374.42tokens/s]
Shard 42:  27%|██▋       | 26536619/100000000 [00:07<00:20, 3650156.50tokens/s]
Shard 42:  27%|██▋       | 27185238/100000000 [00:07<00:16, 4388502.58tokens/s]
Shard 42:  28%|██▊       | 27662741/100000000 [00:07<00:16, 4491613.15tokens/s]
Shard 42:  28%|██▊       | 28127505/100000000 [00:07<00:18, 3794865.97tokens/s]
Shard 42:  29%|██▊       | 28560824/100000000 [00:07<00:18, 3916123.53tokens/s]
Shard 42:  29%|██▉       | 28973550/100000000 [00:08<00:19, 3607752.35tokens/s]
Shard 42:  30%|██▉       | 29589320/100000000 [00:08<00:16, 4225184.51tokens/s]
Shard 42:  30%|███       | 30036356/100000000 [00:08<00:17, 3998334.03tokens/s]
Shard 42:  30%|███       | 30477831/100000000 [00:08<00:16, 4102873.18tokens/s]
Shard 42:  31%|███       | 30925566/100000000 [00:08<00:16, 4200579.36tokens/s]
Shard 42:  31%|███▏      | 31356156/100000000 [00:08<00:18, 3653356.07tokens/s]
Shard 42:  32%|███▏      | 31740072/100000000 [00:08<00:19, 3542646.74tokens/s]
Shard 42:  32%|███▏      | 32107695/100000000 [00:08<00:21, 3178563.05tokens/s]
Shard 42:  32%|███▏      | 32461057/100000000 [00:08<00:20, 3256360.06tokens/s]
Shard 42:  33%|███▎      | 32797601/100000000 [00:09<00:21, 3140333.73tokens/s]
Shard 42:  33%|███▎      | 33119651/100000000 [00:09<00:21, 3093434.79tokens/s]
Shard 42:  33%|███▎      | 33434219/100000000 [00:09<00:22, 2960442.39tokens/s]
Shard 42:  34%|███▍      | 33905062/100000000 [00:09<00:19, 3405422.76tokens/s]
Shard 42:  34%|███▍      | 34252584/100000000 [00:09<00:20, 3241728.28tokens/s]
Shard 42:  35%|███▍      | 34585748/100000000 [00:09<00:20, 3255139.78tokens/s]
Shard 42:  35%|███▍      | 34982681/100000000 [00:09<00:18, 3453651.51tokens/s]
Shard 42:  35%|███▌      | 35332615/100000000 [00:09<00:19, 3381294.86tokens/s]
Shard 42:  36%|███▌      | 35921287/100000000 [00:09<00:15, 4067234.64tokens/s]
Shard 42:  36%|███▋      | 36372937/100000000 [00:10<00:15, 4164359.20tokens/s]
Shard 42:  37%|███▋      | 36793072/100000000 [00:10<00:17, 3595699.24tokens/s]
Shard 42:  37%|███▋      | 37297607/100000000 [00:10<00:15, 3953923.71tokens/s]
Shard 42:  38%|███▊      | 37748663/100000000 [00:10<00:15, 4102903.33tokens/s]
Shard 42:  38%|███▊      | 38171748/100000000 [00:10<00:15, 4027527.31tokens/s]
Shard 42:  40%|███▉      | 39803317/100000000 [00:10<00:08, 7444678.25tokens/s]
Shard 42:  41%|████      | 40575788/100000000 [00:10<00:11, 5382844.62tokens/s]
Shard 42:  41%|████▏     | 41414074/100000000 [00:10<00:09, 6042580.33tokens/s]
Shard 42:  42%|████▏     | 42108740/100000000 [00:11<00:11, 5250871.19tokens/s]
Shard 42:  43%|████▎     | 42711581/100000000 [00:11<00:11, 5103788.84tokens/s]
Shard 42:  43%|████▎     | 43275337/100000000 [00:11<00:10, 5210275.68tokens/s]
Shard 42:  44%|████▍     | 43837677/100000000 [00:11<00:10, 5312852.66tokens/s]
Shard 42:  44%|████▍     | 44399299/100000000 [00:11<00:12, 4584657.86tokens/s]
Shard 42:  45%|████▍     | 44941502/100000000 [00:11<00:11, 4767614.13tokens/s]
Shard 42:  45%|████▌     | 45447834/100000000 [00:11<00:11, 4837825.77tokens/s]
Shard 42:  46%|████▌     | 45954311/100000000 [00:11<00:12, 4405712.68tokens/s]
Shard 42:  47%|████▋     | 46698634/100000000 [00:12<00:10, 5173572.76tokens/s]
Shard 42:  47%|████▋     | 47243773/100000000 [00:12<00:11, 4612724.16tokens/s]
Shard 42:  48%|████▊     | 47736920/100000000 [00:12<00:11, 4658792.54tokens/s]
Shard 42:  48%|████▊     | 48223065/100000000 [00:12<00:11, 4432035.63tokens/s]
Shard 42:  49%|████▊     | 48681014/100000000 [00:12<00:12, 4000500.81tokens/s]
Shard 42:  49%|████▉     | 49097450/100000000 [00:12<00:14, 3602724.65tokens/s]
Shard 42:  49%|████▉     | 49494895/100000000 [00:12<00:13, 3691425.63tokens/s]
Shard 42:  50%|████▉     | 49876941/100000000 [00:12<00:13, 3686227.92tokens/s]
Shard 42:  50%|█████     | 50255988/100000000 [00:13<00:13, 3713477.45tokens/s]
Shard 42:  51%|█████     | 50633949/100000000 [00:13<00:14, 3477729.23tokens/s]
Shard 42:  51%|█████     | 50994439/100000000 [00:13<00:14, 3397357.99tokens/s]
Shard 42:  51%|█████▏    | 51338804/100000000 [00:13<00:14, 3358351.63tokens/s]
Shard 42:  52%|█████▏    | 51740463/100000000 [00:13<00:13, 3509062.11tokens/s]
Shard 42:  52%|█████▏    | 52139293/100000000 [00:13<00:13, 3598150.40tokens/s]
Shard 42:  53%|█████▎    | 52503759/100000000 [00:13<00:15, 3133184.99tokens/s]
Shard 42:  53%|█████▎    | 52829128/100000000 [00:13<00:15, 3081010.35tokens/s]
Shard 42:  53%|█████▎    | 53145010/100000000 [00:13<00:15, 3095833.57tokens/s]
Shard 42:  53%|█████▎    | 53475392/100000000 [00:14<00:14, 3149239.81tokens/s]
Shard 42:  54%|█████▍    | 53844877/100000000 [00:14<00:13, 3300687.43tokens/s]
Shard 42:  54%|█████▍    | 54180346/100000000 [00:14<00:15, 2937382.41tokens/s]
Shard 42:  55%|█████▍    | 54534885/100000000 [00:14<00:14, 3093817.86tokens/s]
Shard 42:  55%|█████▍    | 54853157/100000000 [00:14<00:15, 2974413.00tokens/s]
Shard 42:  55%|█████▌    | 55200814/100000000 [00:14<00:14, 3091670.99tokens/s]
Shard 42:  56%|█████▌    | 55543303/100000000 [00:14<00:13, 3181473.82tokens/s]
Shard 42:  56%|█████▌    | 55866800/100000000 [00:14<00:15, 2937824.94tokens/s]
Shard 42:  56%|█████▌    | 56169926/100000000 [00:14<00:14, 2950373.88tokens/s]
Shard 42:  57%|█████▋    | 56563882/100000000 [00:15<00:13, 3212999.42tokens/s]
Shard 42:  57%|█████▋    | 57088440/100000000 [00:15<00:11, 3774250.63tokens/s]
Shard 42:  57%|█████▋    | 57491214/100000000 [00:15<00:11, 3759360.45tokens/s]
Shard 42:  58%|█████▊    | 57871274/100000000 [00:15<00:13, 3145212.72tokens/s]
Shard 42:  58%|█████▊    | 58298559/100000000 [00:15<00:12, 3425082.77tokens/s]
Shard 42:  59%|█████▊    | 58672108/100000000 [00:15<00:11, 3473426.18tokens/s]
Shard 42:  59%|█████▉    | 59032729/100000000 [00:15<00:13, 3095915.49tokens/s]
Shard 42:  59%|█████▉    | 59358122/100000000 [00:15<00:14, 2839170.05tokens/s]
Shard 42:  60%|█████▉    | 59655336/100000000 [00:16<00:14, 2774694.47tokens/s]
Shard 42:  60%|█████▉    | 59941610/100000000 [00:16<00:15, 2604752.99tokens/s]
Shard 42:  60%|██████    | 60317354/100000000 [00:16<00:13, 2888568.54tokens/s]
Shard 42:  61%|██████    | 60665621/100000000 [00:16<00:12, 3035672.24tokens/s]
Shard 42:  61%|██████    | 60977512/100000000 [00:16<00:14, 2710618.95tokens/s]
Shard 42:  61%|██████▏   | 61261178/100000000 [00:16<00:15, 2461474.59tokens/s]
Shard 42:  62%|██████▏   | 61517873/100000000 [00:16<00:15, 2450639.49tokens/s]
Shard 42:  62%|██████▏   | 61819155/100000000 [00:16<00:14, 2587447.35tokens/s]
Shard 42:  62%|██████▏   | 62085134/100000000 [00:17<00:14, 2563236.87tokens/s]
Shard 42:  62%|██████▏   | 62394878/100000000 [00:17<00:13, 2705108.49tokens/s]
Shard 42:  63%|██████▎   | 62739973/100000000 [00:17<00:12, 2909481.79tokens/s]
Shard 42:  63%|██████▎   | 63118402/100000000 [00:17<00:11, 3159812.38tokens/s]
Shard 42:  63%|██████▎   | 63438734/100000000 [00:17<00:11, 3062980.01tokens/s]
Shard 42:  64%|██████▎   | 63748567/100000000 [00:17<00:14, 2540916.31tokens/s]
Shard 42:  64%|██████▍   | 64019115/100000000 [00:17<00:14, 2446322.83tokens/s]
Shard 42:  64%|██████▍   | 64275488/100000000 [00:17<00:15, 2357562.68tokens/s]
Shard 42:  65%|██████▍   | 64520611/100000000 [00:17<00:14, 2365704.26tokens/s]
Shard 42:  65%|██████▍   | 64762853/100000000 [00:18<00:15, 2319684.00tokens/s]
Shard 42:  65%|██████▌   | 65045567/100000000 [00:18<00:14, 2450438.23tokens/s]
Shard 42:  65%|██████▌   | 65295061/100000000 [00:18<00:15, 2252605.53tokens/s]
Shard 42:  66%|██████▌   | 65525535/100000000 [00:18<00:16, 2143184.47tokens/s]
Shard 42:  66%|██████▌   | 65794449/100000000 [00:18<00:15, 2278350.63tokens/s]
Shard 42:  66%|██████▌   | 66026985/100000000 [00:18<00:16, 2072401.25tokens/s]
Shard 42:  67%|██████▋   | 66839637/100000000 [00:18<00:09, 3617690.27tokens/s]
Shard 42:  67%|██████▋   | 67223322/100000000 [00:18<00:08, 3676212.95tokens/s]
Shard 42:  68%|██████▊   | 67607073/100000000 [00:19<00:10, 3207135.35tokens/s]
Shard 42:  68%|██████▊   | 67953659/100000000 [00:19<00:10, 3140091.38tokens/s]
Shard 42:  68%|██████▊   | 68285601/100000000 [00:19<00:09, 3184774.11tokens/s]
Shard 42:  69%|██████▊   | 68711825/100000000 [00:19<00:09, 3474373.77tokens/s]
Shard 42:  69%|██████▉   | 69070425/100000000 [00:19<00:08, 3481875.05tokens/s]
Shard 42:  69%|██████▉   | 69427429/100000000 [00:19<00:09, 3290188.50tokens/s]
Shard 42:  70%|██████▉   | 69763425/100000000 [00:19<00:09, 3118975.80tokens/s]
Shard 42:  70%|███████   | 70104600/100000000 [00:19<00:09, 3170100.35tokens/s]
Shard 42:  70%|███████   | 70426640/100000000 [00:19<00:09, 3113391.60tokens/s]
Shard 42:  71%|███████   | 70784245/100000000 [00:19<00:09, 3203805.42tokens/s]
Shard 42:  71%|███████   | 71146967/100000000 [00:20<00:08, 3322035.43tokens/s]
Shard 42:  72%|███████▏  | 71663354/100000000 [00:20<00:07, 3838949.10tokens/s]
Shard 42:  72%|███████▏  | 72118583/100000000 [00:20<00:06, 4039613.37tokens/s]
Shard 42:  73%|███████▎  | 72572455/100000000 [00:20<00:06, 4151691.45tokens/s]
Shard 42:  73%|███████▎  | 72989867/100000000 [00:20<00:06, 4033849.91tokens/s]
Shard 42:  73%|███████▎  | 73396757/100000000 [00:20<00:07, 3793122.02tokens/s]
Shard 42:  74%|███████▍  | 73780074/100000000 [00:20<00:07, 3533704.92tokens/s]
Shard 42:  74%|███████▍  | 74138768/100000000 [00:20<00:07, 3372986.76tokens/s]
Shard 42:  74%|███████▍  | 74489516/100000000 [00:21<00:08, 3182129.10tokens/s]
Shard 42:  75%|███████▍  | 74811546/100000000 [00:21<00:08, 3085186.90tokens/s]
Shard 42:  75%|███████▌  | 75253855/100000000 [00:21<00:07, 3430868.78tokens/s]
Shard 42:  76%|███████▌  | 75745936/100000000 [00:21<00:06, 3840969.14tokens/s]
Shard 42:  76%|███████▌  | 76153806/100000000 [00:21<00:06, 3879454.06tokens/s]
Shard 42:  77%|███████▋  | 76627784/100000000 [00:21<00:05, 4123467.95tokens/s]
Shard 42:  77%|███████▋  | 77045455/100000000 [00:21<00:07, 3267839.35tokens/s]
Shard 42:  77%|███████▋  | 77403207/100000000 [00:21<00:07, 3116978.22tokens/s]
Shard 42:  78%|███████▊  | 78029901/100000000 [00:21<00:05, 3863784.06tokens/s]
Shard 42:  78%|███████▊  | 78445421/100000000 [00:22<00:06, 3174794.84tokens/s]
Shard 42:  79%|███████▉  | 78800708/100000000 [00:22<00:06, 3113071.38tokens/s]
Shard 42:  79%|███████▉  | 79295462/100000000 [00:22<00:05, 3545382.67tokens/s]
Shard 42:  80%|███████▉  | 79741502/100000000 [00:22<00:05, 3772726.55tokens/s]
Shard 42:  80%|████████  | 80143945/100000000 [00:22<00:05, 3568038.55tokens/s]
Shard 42:  81%|████████  | 80626371/100000000 [00:22<00:05, 3871938.62tokens/s]
Shard 42:  81%|████████  | 81116363/100000000 [00:22<00:04, 4140330.19tokens/s]
Shard 42:  82%|████████▏ | 81545903/100000000 [00:22<00:04, 4151259.22tokens/s]
Shard 42:  82%|████████▏ | 82071413/100000000 [00:23<00:04, 4419618.62tokens/s]
Shard 42:  83%|████████▎ | 82521602/100000000 [00:23<00:04, 4307013.50tokens/s]
Shard 42:  83%|████████▎ | 83019237/100000000 [00:23<00:03, 4483175.68tokens/s]
Shard 42:  83%|████████▎ | 83474286/100000000 [00:23<00:03, 4408391.93tokens/s]
Shard 42:  84%|████████▍ | 83918863/100000000 [00:23<00:04, 3729445.03tokens/s]
Shard 42:  84%|████████▍ | 84406147/100000000 [00:23<00:03, 3992441.42tokens/s]
Shard 42:  85%|████████▍ | 84823054/100000000 [00:23<00:03, 3996775.32tokens/s]
Shard 42:  85%|████████▌ | 85235154/100000000 [00:23<00:03, 3878403.52tokens/s]
Shard 42:  86%|████████▌ | 85632206/100000000 [00:23<00:03, 3748585.65tokens/s]
Shard 42:  86%|████████▌ | 86139576/100000000 [00:24<00:03, 4099843.36tokens/s]
Shard 42:  87%|████████▋ | 86557131/100000000 [00:24<00:03, 4046635.68tokens/s]
Shard 42:  87%|████████▋ | 86968163/100000000 [00:24<00:03, 4043393.81tokens/s]
Shard 42:  87%|████████▋ | 87475691/100000000 [00:24<00:02, 4335944.77tokens/s]
Shard 42:  88%|████████▊ | 87913468/100000000 [00:24<00:02, 4295845.97tokens/s]
Shard 42:  88%|████████▊ | 88346111/100000000 [00:24<00:02, 4159180.10tokens/s]
Shard 42:  89%|████████▉ | 88776434/100000000 [00:24<00:02, 4088093.94tokens/s]
Shard 42:  89%|████████▉ | 89187952/100000000 [00:24<00:02, 4012166.16tokens/s]
Shard 42:  90%|████████▉ | 89594064/100000000 [00:24<00:02, 4015438.18tokens/s]
Shard 42:  90%|████████▉ | 89997032/100000000 [00:24<00:02, 3973703.30tokens/s]
Shard 42:  90%|█████████ | 90395109/100000000 [00:25<00:02, 3974590.15tokens/s]
Shard 42:  91%|█████████ | 90811709/100000000 [00:25<00:02, 4022796.24tokens/s]
Shard 42:  91%|█████████ | 91214801/100000000 [00:25<00:02, 3514396.06tokens/s]
Shard 42:  92%|█████████▏| 91577717/100000000 [00:25<00:02, 3463387.68tokens/s]
Shard 42:  92%|█████████▏| 92002125/100000000 [00:25<00:02, 3669811.53tokens/s]
Shard 42:  92%|█████████▏| 92376696/100000000 [00:25<00:02, 3166325.82tokens/s]
Shard 42:  93%|█████████▎| 92710088/100000000 [00:25<00:02, 3084164.67tokens/s]
Shard 42:  93%|█████████▎| 93029792/100000000 [00:25<00:02, 3027547.91tokens/s]
Shard 42:  93%|█████████▎| 93340270/100000000 [00:26<00:02, 2952355.10tokens/s]
Shard 42:  94%|█████████▍| 93781880/100000000 [00:26<00:01, 3340560.64tokens/s]
Shard 42:  94%|█████████▍| 94123650/100000000 [00:26<00:01, 3289184.23tokens/s]
Shard 42:  94%|█████████▍| 94487481/100000000 [00:26<00:01, 3386396.96tokens/s]
Shard 42:  95%|█████████▍| 94874986/100000000 [00:26<00:01, 3510688.02tokens/s]
Shard 42:  95%|█████████▌| 95229757/100000000 [00:26<00:01, 3292502.99tokens/s]
Shard 42:  96%|█████████▌| 95564072/100000000 [00:26<00:01, 3112411.38tokens/s]
Shard 42:  96%|█████████▌| 95879981/100000000 [00:26<00:01, 3086315.88tokens/s]
Shard 42:  96%|█████████▋| 96269807/100000000 [00:26<00:01, 3291514.45tokens/s]
Shard 42:  97%|█████████▋| 96602441/100000000 [00:27<00:01, 3169324.25tokens/s]
Shard 42:  97%|█████████▋| 96922524/100000000 [00:27<00:01, 3055374.47tokens/s]
Shard 42:  97%|█████████▋| 97230460/100000000 [00:27<00:01, 2673286.28tokens/s]
Shard 42:  98%|█████████▊| 97664612/100000000 [00:27<00:00, 3084510.05tokens/s]
Shard 42:  98%|█████████▊| 97985311/100000000 [00:27<00:00, 2786132.90tokens/s]
Shard 42:  98%|█████████▊| 98279461/100000000 [00:27<00:00, 2782467.99tokens/s]
Shard 42:  99%|█████████▊| 98566714/100000000 [00:27<00:00, 2676845.55tokens/s]
Shard 42:  99%|█████████▉| 98950345/100000000 [00:27<00:00, 2957564.76tokens/s]
Shard 42:  99%|█████████▉| 99253365/100000000 [00:27<00:00, 2905494.16tokens/s]
Shard 42: 100%|█████████▉| 99698235/100000000 [00:28<00:00, 3323768.65tokens/s]
Shard 42: 100%|█████████▉| 99999847/100000000 [00:37<00:00, 2660889.63tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000042.bin (200,001,024 bytes) in the gpt-2 format

Shard 43:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 43:   0%|          | 155878/100000000 [00:00<01:04, 1552142.61tokens/s]
Shard 43:   1%|          | 541540/100000000 [00:00<00:34, 2899438.42tokens/s]
Shard 43:   1%|          | 919358/100000000 [00:00<00:30, 3286251.87tokens/s]
Shard 43:   1%|          | 1248248/100000000 [00:00<00:30, 3255760.18tokens/s]
Shard 43:   2%|▏         | 1697111/100000000 [00:00<00:26, 3695620.34tokens/s]
Shard 43:   2%|▏         | 2158828/100000000 [00:00<00:24, 4003604.97tokens/s]
Shard 43:   3%|▎         | 2559642/100000000 [00:00<00:25, 3871773.19tokens/s]
Shard 43:   3%|▎         | 3011749/100000000 [00:00<00:23, 4072517.17tokens/s]
Shard 43:   3%|▎         | 3434345/100000000 [00:00<00:23, 4095005.17tokens/s]
Shard 43:   4%|▍         | 3844857/100000000 [00:01<00:24, 3890183.85tokens/s]
Shard 43:   4%|▍         | 4269930/100000000 [00:01<00:23, 3995099.09tokens/s]
Shard 43:   5%|▍         | 4672090/100000000 [00:01<00:31, 3050000.42tokens/s]
Shard 43:   5%|▌         | 5051382/100000000 [00:01<00:29, 3230007.71tokens/s]
Shard 43:   5%|▌         | 5403181/100000000 [00:01<00:31, 3004232.80tokens/s]
Shard 43:   6%|▌         | 5724857/100000000 [00:01<00:34, 2702759.99tokens/s]
Shard 43:   6%|▌         | 6013239/100000000 [00:01<00:36, 2564546.72tokens/s]
Shard 43:   6%|▋         | 6325300/100000000 [00:01<00:34, 2699513.11tokens/s]
Shard 43:   7%|▋         | 6629705/100000000 [00:02<00:33, 2782078.45tokens/s]
Shard 43:   7%|▋         | 7005297/100000000 [00:02<00:30, 3015508.47tokens/s]
Shard 43:   7%|▋         | 7316475/100000000 [00:02<00:30, 2992569.17tokens/s]
Shard 43:   8%|▊         | 7621788/100000000 [00:02<00:34, 2706774.93tokens/s]
Shard 43:   8%|▊         | 7959520/100000000 [00:02<00:32, 2874012.86tokens/s]
Shard 43:   8%|▊         | 8348475/100000000 [00:02<00:29, 3141249.15tokens/s]
Shard 43:   9%|▊         | 8670612/100000000 [00:02<00:29, 3142822.86tokens/s]
Shard 43:   9%|▉         | 9061062/100000000 [00:02<00:27, 3354327.16tokens/s]
Shard 43:   9%|▉         | 9405673/100000000 [00:02<00:28, 3186988.20tokens/s]
Shard 43:  10%|▉         | 9730096/100000000 [00:03<00:30, 2971202.23tokens/s]
Shard 43:  10%|█         | 10032823/100000000 [00:03<00:31, 2902101.99tokens/s]
Shard 43:  10%|█         | 10326818/100000000 [00:03<00:35, 2529307.63tokens/s]
Shard 43:  11%|█         | 10676389/100000000 [00:03<00:32, 2766824.61tokens/s]
Shard 43:  11%|█         | 10964149/100000000 [00:03<00:33, 2650049.22tokens/s]
Shard 43:  11%|█▏        | 11305305/100000000 [00:03<00:31, 2845778.64tokens/s]
Shard 43:  12%|█▏        | 11597850/100000000 [00:03<00:30, 2858021.29tokens/s]
Shard 43:  12%|█▏        | 11896709/100000000 [00:03<00:31, 2834612.55tokens/s]
Shard 43:  12%|█▏        | 12194433/100000000 [00:03<00:32, 2690699.93tokens/s]
Shard 43:  12%|█▏        | 12467637/100000000 [00:04<00:34, 2572490.91tokens/s]
Shard 43:  13%|█▎        | 12728259/100000000 [00:04<00:34, 2502358.25tokens/s]
Shard 43:  13%|█▎        | 13063299/100000000 [00:04<00:31, 2731656.55tokens/s]
Shard 43:  13%|█▎        | 13378358/100000000 [00:04<00:30, 2841028.23tokens/s]
Shard 43:  14%|█▎        | 13666223/100000000 [00:04<00:32, 2692885.05tokens/s]
Shard 43:  14%|█▍        | 14020363/100000000 [00:04<00:29, 2906857.74tokens/s]
Shard 43:  14%|█▍        | 14314954/100000000 [00:04<00:30, 2838288.43tokens/s]
Shard 43:  15%|█▍        | 14621795/100000000 [00:04<00:29, 2892365.63tokens/s]
Shard 43:  15%|█▍        | 14913678/100000000 [00:04<00:30, 2799711.19tokens/s]
Shard 43:  15%|█▌        | 15205916/100000000 [00:05<00:30, 2746957.68tokens/s]
Shard 43:  16%|█▌        | 15561808/100000000 [00:05<00:28, 2963023.17tokens/s]
Shard 43:  16%|█▌        | 15941488/100000000 [00:05<00:26, 3193083.55tokens/s]
Shard 43:  16%|█▋        | 16337442/100000000 [00:05<00:24, 3414885.80tokens/s]
Shard 43:  17%|█▋        | 17344015/100000000 [00:05<00:15, 5346867.31tokens/s]
Shard 43:  18%|█▊        | 17885397/100000000 [00:05<00:18, 4431037.67tokens/s]
Shard 43:  18%|█▊        | 18358128/100000000 [00:05<00:19, 4118386.93tokens/s]
Shard 43:  19%|█▉        | 18792835/100000000 [00:05<00:22, 3596040.51tokens/s]
Shard 43:  19%|█▉        | 19184690/100000000 [00:06<00:23, 3439722.28tokens/s]
Shard 43:  20%|█▉        | 19545297/100000000 [00:06<00:24, 3226531.11tokens/s]
Shard 43:  20%|██        | 20015437/100000000 [00:06<00:22, 3576591.61tokens/s]
Shard 43:  21%|██        | 20529863/100000000 [00:06<00:19, 3976301.81tokens/s]
Shard 43:  21%|██        | 20998367/100000000 [00:06<00:19, 4123871.44tokens/s]
Shard 43:  21%|██▏       | 21424849/100000000 [00:06<00:20, 3837200.99tokens/s]
Shard 43:  22%|██▏       | 21829533/100000000 [00:06<00:20, 3886271.91tokens/s]
Shard 43:  22%|██▏       | 22227708/100000000 [00:06<00:21, 3593110.00tokens/s]
Shard 43:  23%|██▎       | 22789358/100000000 [00:06<00:18, 4102098.80tokens/s]
Shard 43:  23%|██▎       | 23212482/100000000 [00:07<00:18, 4130272.38tokens/s]
Shard 43:  24%|██▎       | 23635634/100000000 [00:07<00:18, 4123643.56tokens/s]
Shard 43:  24%|██▍       | 24085512/100000000 [00:07<00:17, 4228398.83tokens/s]
Shard 43:  25%|██▍       | 24513427/100000000 [00:07<00:21, 3471993.88tokens/s]
Shard 43:  25%|██▍       | 24885633/100000000 [00:07<00:22, 3303636.67tokens/s]
Shard 43:  25%|██▌       | 25260310/100000000 [00:07<00:21, 3411921.30tokens/s]
Shard 43:  26%|██▌       | 25615897/100000000 [00:07<00:23, 3230040.34tokens/s]
Shard 43:  26%|██▌       | 25967605/100000000 [00:07<00:22, 3284439.94tokens/s]
Shard 43:  26%|██▋       | 26321114/100000000 [00:08<00:21, 3351824.88tokens/s]
Shard 43:  27%|██▋       | 26672527/100000000 [00:08<00:21, 3377305.24tokens/s]
Shard 43:  27%|██▋       | 27082413/100000000 [00:08<00:20, 3581752.03tokens/s]
Shard 43:  28%|██▊       | 27619375/100000000 [00:08<00:17, 4086297.71tokens/s]
Shard 43:  28%|██▊       | 28034260/100000000 [00:08<00:19, 3664857.86tokens/s]
Shard 43:  28%|██▊       | 28412414/100000000 [00:08<00:20, 3489053.31tokens/s]
Shard 43:  29%|██▉       | 28771018/100000000 [00:08<00:21, 3388531.39tokens/s]
Shard 43:  29%|██▉       | 29116589/100000000 [00:08<00:21, 3311865.72tokens/s]
Shard 43:  29%|██▉       | 29452876/100000000 [00:08<00:21, 3288148.50tokens/s]
Shard 43:  30%|██▉       | 29784311/100000000 [00:09<00:21, 3211512.13tokens/s]
Shard 43:  30%|███       | 30107114/100000000 [00:09<00:24, 2879877.70tokens/s]
Shard 43:  30%|███       | 30436351/100000000 [00:09<00:24, 2896495.41tokens/s]
Shard 43:  31%|███       | 30850332/100000000 [00:09<00:21, 3219111.57tokens/s]
Shard 43:  31%|███       | 31179652/100000000 [00:09<00:21, 3178523.05tokens/s]
Shard 43:  32%|███▏      | 31619414/100000000 [00:09<00:19, 3507307.10tokens/s]
Shard 43:  32%|███▏      | 32009620/100000000 [00:09<00:18, 3613618.10tokens/s]
Shard 43:  32%|███▏      | 32377411/100000000 [00:09<00:20, 3339171.76tokens/s]
Shard 43:  33%|███▎      | 32718692/100000000 [00:09<00:20, 3358332.94tokens/s]
Shard 43:  33%|███▎      | 33059958/100000000 [00:10<00:20, 3328559.48tokens/s]
Shard 43:  33%|███▎      | 33397063/100000000 [00:10<00:21, 3135881.94tokens/s]
Shard 43:  34%|███▎      | 33748140/100000000 [00:10<00:20, 3227822.24tokens/s]
Shard 43:  34%|███▍      | 34074825/100000000 [00:10<00:21, 3087141.70tokens/s]
Shard 43:  34%|███▍      | 34386700/100000000 [00:10<00:22, 2862032.42tokens/s]
Shard 43:  35%|███▍      | 34692065/100000000 [00:10<00:22, 2912568.05tokens/s]
Shard 43:  35%|███▍      | 34987588/100000000 [00:10<00:22, 2913805.90tokens/s]
Shard 43:  35%|███▌      | 35388440/100000000 [00:10<00:20, 3205713.90tokens/s]
Shard 43:  36%|███▌      | 35719446/100000000 [00:10<00:19, 3235600.90tokens/s]
Shard 43:  36%|███▌      | 36084726/100000000 [00:11<00:19, 3356547.71tokens/s]
Shard 43:  36%|███▋      | 36422470/100000000 [00:11<00:18, 3352822.70tokens/s]
Shard 43:  37%|███▋      | 36759787/100000000 [00:11<00:20, 3048613.64tokens/s]
Shard 43:  37%|███▋      | 37150798/100000000 [00:11<00:19, 3280557.50tokens/s]
Shard 43:  37%|███▋      | 37487072/100000000 [00:11<00:19, 3159838.65tokens/s]
Shard 43:  38%|███▊      | 37859878/100000000 [00:11<00:18, 3300164.88tokens/s]
Shard 43:  38%|███▊      | 38406455/100000000 [00:11<00:15, 3909687.48tokens/s]
Shard 43:  39%|███▉      | 38804055/100000000 [00:11<00:15, 3828518.80tokens/s]
Shard 43:  39%|███▉      | 39192004/100000000 [00:11<00:17, 3539648.43tokens/s]
Shard 43:  41%|████      | 40612605/100000000 [00:11<00:09, 6448003.28tokens/s]
Shard 43:  41%|████▏     | 41291691/100000000 [00:12<00:11, 5011497.64tokens/s]
Shard 43:  42%|████▏     | 41863645/100000000 [00:12<00:12, 4556249.09tokens/s]
Shard 43:  42%|████▏     | 42371971/100000000 [00:12<00:15, 3601928.88tokens/s]
Shard 43:  43%|████▎     | 42794642/100000000 [00:12<00:16, 3406225.61tokens/s]
Shard 43:  43%|████▎     | 43176638/100000000 [00:12<00:17, 3315950.38tokens/s]
Shard 43:  44%|████▎     | 43535992/100000000 [00:13<00:18, 2989111.98tokens/s]
Shard 43:  44%|████▍     | 43855546/100000000 [00:13<00:18, 2956761.67tokens/s]
Shard 43:  44%|████▍     | 44167599/100000000 [00:13<00:18, 2955941.61tokens/s]
Shard 43:  45%|████▍     | 44544001/100000000 [00:13<00:17, 3150904.75tokens/s]
Shard 43:  45%|████▍     | 44870356/100000000 [00:13<00:17, 3110817.11tokens/s]
Shard 43:  45%|████▌     | 45199175/100000000 [00:13<00:17, 3134285.31tokens/s]
Shard 43:  46%|████▌     | 45517881/100000000 [00:13<00:17, 3078134.87tokens/s]
Shard 43:  46%|████▌     | 45841990/100000000 [00:13<00:17, 3118943.28tokens/s]
Shard 43:  46%|████▋     | 46293258/100000000 [00:13<00:15, 3506876.79tokens/s]
Shard 43:  47%|████▋     | 46655017/100000000 [00:13<00:15, 3391939.06tokens/s]
Shard 43:  47%|████▋     | 47184145/100000000 [00:14<00:13, 3915384.90tokens/s]
Shard 43:  48%|████▊     | 47627356/100000000 [00:14<00:12, 4048834.16tokens/s]
Shard 43:  48%|████▊     | 48036715/100000000 [00:14<00:14, 3629874.56tokens/s]
Shard 43:  48%|████▊     | 48410095/100000000 [00:14<00:18, 2813903.71tokens/s]
Shard 43:  49%|████▉     | 48795665/100000000 [00:14<00:16, 3029722.75tokens/s]
Shard 43:  49%|████▉     | 49127804/100000000 [00:14<00:16, 3090143.96tokens/s]
Shard 43:  49%|████▉     | 49458757/100000000 [00:14<00:16, 3063878.75tokens/s]
Shard 43:  50%|█████     | 50039740/100000000 [00:14<00:13, 3787944.81tokens/s]
Shard 43:  50%|█████     | 50438759/100000000 [00:15<00:13, 3714560.45tokens/s]
Shard 43:  51%|█████     | 50860434/100000000 [00:15<00:12, 3852451.21tokens/s]
Shard 43:  51%|█████▏    | 51333362/100000000 [00:15<00:11, 4095226.39tokens/s]
Shard 43:  52%|█████▏    | 51810065/100000000 [00:15<00:11, 4277288.19tokens/s]
Shard 43:  52%|█████▏    | 52245196/100000000 [00:15<00:11, 4086252.77tokens/s]
Shard 43:  53%|█████▎    | 52660420/100000000 [00:15<00:12, 3711512.69tokens/s]
Shard 43:  53%|█████▎    | 53184759/100000000 [00:15<00:11, 4116580.44tokens/s]
Shard 43:  54%|█████▎    | 53608539/100000000 [00:15<00:12, 3654776.99tokens/s]
Shard 43:  54%|█████▍    | 53990450/100000000 [00:15<00:12, 3562095.71tokens/s]
Shard 43:  54%|█████▍    | 54357667/100000000 [00:16<00:13, 3358168.66tokens/s]
Shard 43:  55%|█████▍    | 54702651/100000000 [00:16<00:14, 3094634.70tokens/s]
Shard 43:  56%|█████▌    | 55515574/100000000 [00:16<00:10, 4364579.95tokens/s]
Shard 43:  56%|█████▋    | 56301900/100000000 [00:16<00:08, 5293007.91tokens/s]
Shard 43:  57%|█████▋    | 56863603/100000000 [00:16<00:08, 4845686.61tokens/s]
Shard 43:  57%|█████▋    | 57384260/100000000 [00:16<00:08, 4756624.45tokens/s]
Shard 43:  58%|█████▊    | 57879666/100000000 [00:16<00:10, 4192928.90tokens/s]
Shard 43:  58%|█████▊    | 58327045/100000000 [00:16<00:10, 4052288.30tokens/s]
Shard 43:  59%|█████▊    | 58748376/100000000 [00:17<00:10, 4013136.96tokens/s]
Shard 43:  59%|█████▉    | 59159988/100000000 [00:17<00:10, 3956417.64tokens/s]
Shard 43:  60%|█████▉    | 59563020/100000000 [00:17<00:10, 3821601.29tokens/s]
Shard 43:  60%|█████▉    | 59950024/100000000 [00:17<00:12, 3173650.33tokens/s]
Shard 43:  60%|██████    | 60327415/100000000 [00:17<00:11, 3316874.91tokens/s]
Shard 43:  61%|██████    | 60736256/100000000 [00:17<00:11, 3510253.11tokens/s]
Shard 43:  61%|██████    | 61104443/100000000 [00:17<00:11, 3294837.58tokens/s]
Shard 43:  62%|██████▏   | 61517761/100000000 [00:17<00:11, 3449333.71tokens/s]
Shard 43:  62%|██████▏   | 61873039/100000000 [00:18<00:11, 3300062.18tokens/s]
Shard 43:  62%|██████▏   | 62281868/100000000 [00:18<00:10, 3432672.54tokens/s]
Shard 43:  63%|██████▎   | 62644399/100000000 [00:18<00:10, 3476744.51tokens/s]
Shard 43:  63%|██████▎   | 62999542/100000000 [00:18<00:10, 3399330.99tokens/s]
Shard 43:  63%|██████▎   | 63343523/100000000 [00:18<00:12, 2855387.72tokens/s]
Shard 43:  64%|██████▎   | 63671997/100000000 [00:18<00:12, 2954774.25tokens/s]
Shard 43:  64%|██████▍   | 63980808/100000000 [00:18<00:13, 2742167.22tokens/s]
Shard 43:  64%|██████▍   | 64321553/100000000 [00:18<00:12, 2895603.61tokens/s]
Shard 43:  65%|██████▍   | 64659258/100000000 [00:18<00:11, 3024040.84tokens/s]
Shard 43:  65%|██████▍   | 64970116/100000000 [00:19<00:13, 2684687.51tokens/s]
Shard 43:  65%|██████▌   | 65250337/100000000 [00:19<00:13, 2504459.90tokens/s]
Shard 43:  66%|██████▌   | 65509946/100000000 [00:19<00:13, 2510486.84tokens/s]
Shard 43:  66%|██████▌   | 65967240/100000000 [00:19<00:11, 3055548.22tokens/s]
Shard 43:  66%|██████▋   | 66285026/100000000 [00:19<00:11, 2966114.35tokens/s]
Shard 43:  67%|██████▋   | 66777196/100000000 [00:19<00:09, 3492139.50tokens/s]
Shard 43:  67%|██████▋   | 67186803/100000000 [00:19<00:09, 3643128.90tokens/s]
Shard 43:  68%|██████▊   | 68215676/100000000 [00:19<00:05, 5522914.56tokens/s]
Shard 43:  69%|██████▉   | 68782523/100000000 [00:20<00:06, 4566914.00tokens/s]
Shard 43:  69%|██████▉   | 69277322/100000000 [00:20<00:06, 4478654.29tokens/s]
Shard 43:  70%|██████▉   | 69750627/100000000 [00:20<00:07, 3873110.94tokens/s]
Shard 43:  70%|███████   | 70166906/100000000 [00:20<00:07, 3940152.60tokens/s]
Shard 43:  71%|███████   | 70592814/100000000 [00:20<00:07, 3866645.87tokens/s]
Shard 43:  71%|███████   | 70994391/100000000 [00:20<00:07, 3905054.37tokens/s]
Shard 43:  71%|███████▏  | 71474425/100000000 [00:20<00:06, 4135463.08tokens/s]
Shard 43:  72%|███████▏  | 71899080/100000000 [00:20<00:07, 3850103.52tokens/s]
Shard 43:  72%|███████▏  | 72416555/100000000 [00:20<00:06, 4203172.05tokens/s]
Shard 43:  73%|███████▎  | 72849370/100000000 [00:21<00:06, 4199299.00tokens/s]
Shard 43:  73%|███████▎  | 73355260/100000000 [00:21<00:06, 4432412.15tokens/s]
Shard 43:  74%|███████▍  | 73805387/100000000 [00:21<00:06, 3879937.79tokens/s]
Shard 43:  74%|███████▍  | 74209908/100000000 [00:21<00:07, 3615488.54tokens/s]
Shard 43:  75%|███████▍  | 74584567/100000000 [00:21<00:07, 3402523.57tokens/s]
Shard 43:  75%|███████▍  | 74992321/100000000 [00:21<00:07, 3565335.01tokens/s]
Shard 43:  76%|███████▌  | 75525222/100000000 [00:21<00:06, 3988197.84tokens/s]
Shard 43:  76%|███████▌  | 76075091/100000000 [00:21<00:05, 4283622.15tokens/s]
Shard 43:  77%|███████▋  | 76511584/100000000 [00:22<00:06, 3823476.44tokens/s]
Shard 43:  77%|███████▋  | 76907191/100000000 [00:22<00:06, 3727931.65tokens/s]
Shard 43:  77%|███████▋  | 77288600/100000000 [00:22<00:06, 3265605.32tokens/s]
Shard 43:  78%|███████▊  | 77629478/100000000 [00:22<00:07, 3145296.47tokens/s]
Shard 43:  78%|███████▊  | 77955204/100000000 [00:22<00:07, 3139078.63tokens/s]
Shard 43:  78%|███████▊  | 78336550/100000000 [00:22<00:06, 3311634.56tokens/s]
Shard 43:  79%|███████▉  | 78752689/100000000 [00:22<00:06, 3536813.56tokens/s]
Shard 43:  79%|███████▉  | 79114236/100000000 [00:22<00:06, 3471062.84tokens/s]
Shard 43:  79%|███████▉  | 79466531/100000000 [00:22<00:06, 3328204.55tokens/s]
Shard 43:  80%|███████▉  | 79829130/100000000 [00:23<00:05, 3396347.66tokens/s]
Shard 43:  80%|████████  | 80287839/100000000 [00:23<00:05, 3709416.75tokens/s]
Shard 43:  81%|████████  | 80995139/100000000 [00:23<00:04, 4647995.41tokens/s]
Shard 43:  81%|████████▏ | 81466971/100000000 [00:23<00:04, 4401760.83tokens/s]
Shard 43:  82%|████████▏ | 82147818/100000000 [00:23<00:03, 5045298.25tokens/s]
Shard 43:  83%|████████▎ | 82730339/100000000 [00:23<00:03, 5232892.31tokens/s]
Shard 43:  83%|████████▎ | 83259935/100000000 [00:23<00:03, 5130690.66tokens/s]
Shard 43:  84%|████████▍ | 83777552/100000000 [00:23<00:03, 4359458.06tokens/s]
Shard 43:  84%|████████▍ | 84238582/100000000 [00:24<00:03, 4129149.21tokens/s]
Shard 43:  85%|████████▍ | 84669387/100000000 [00:24<00:03, 4111809.78tokens/s]
Shard 43:  85%|████████▌ | 85091433/100000000 [00:24<00:03, 4115876.20tokens/s]
Shard 43:  86%|████████▌ | 85510694/100000000 [00:24<00:03, 3911411.82tokens/s]
Shard 43:  86%|████████▌ | 85908455/100000000 [00:24<00:04, 3505464.06tokens/s]
Shard 43:  86%|████████▋ | 86383887/100000000 [00:24<00:03, 3788746.71tokens/s]
Shard 43:  87%|████████▋ | 86773915/100000000 [00:24<00:03, 3527365.23tokens/s]
Shard 43:  87%|████████▋ | 87212999/100000000 [00:24<00:03, 3718008.46tokens/s]
Shard 43:  88%|████████▊ | 87595195/100000000 [00:24<00:03, 3453392.73tokens/s]
Shard 43:  88%|████████▊ | 88035436/100000000 [00:25<00:03, 3680692.97tokens/s]
Shard 43:  88%|████████▊ | 88412374/100000000 [00:25<00:03, 3653514.62tokens/s]
Shard 43:  89%|████████▉ | 88784117/100000000 [00:25<00:03, 3651741.63tokens/s]
Shard 43:  89%|████████▉ | 89248196/100000000 [00:25<00:02, 3917559.38tokens/s]
Shard 43:  90%|████████▉ | 89728254/100000000 [00:25<00:02, 4170183.43tokens/s]
Shard 43:  90%|█████████ | 90179434/100000000 [00:25<00:02, 4266404.29tokens/s]
Shard 43:  91%|█████████ | 90610264/100000000 [00:25<00:02, 3779789.79tokens/s]
Shard 43:  91%|█████████ | 91000576/100000000 [00:25<00:02, 3728059.40tokens/s]
Shard 43:  91%|█████████▏| 91485077/100000000 [00:25<00:02, 4030238.40tokens/s]
Shard 43:  92%|█████████▏| 91902255/100000000 [00:26<00:01, 4065898.33tokens/s]
Shard 43:  93%|█████████▎| 92701196/100000000 [00:26<00:01, 5188689.91tokens/s]
Shard 43:  93%|█████████▎| 93229846/100000000 [00:26<00:01, 4747686.94tokens/s]
Shard 43:  94%|█████████▎| 93718363/100000000 [00:26<00:01, 4256545.71tokens/s]
Shard 43:  94%|█████████▍| 94203532/100000000 [00:26<00:01, 4390223.08tokens/s]
Shard 43:  95%|█████████▍| 94656780/100000000 [00:26<00:01, 4035744.74tokens/s]
Shard 43:  95%|█████████▌| 95378680/100000000 [00:26<00:00, 4819299.38tokens/s]
Shard 43:  96%|█████████▌| 95881171/100000000 [00:26<00:01, 3648563.70tokens/s]
Shard 43:  96%|█████████▋| 96299843/100000000 [00:27<00:01, 3265239.02tokens/s]
Shard 43:  97%|█████████▋| 96667476/100000000 [00:27<00:01, 3223352.74tokens/s]
Shard 43:  97%|█████████▋| 97017350/100000000 [00:27<00:00, 3163594.25tokens/s]
Shard 43:  97%|█████████▋| 97361342/100000000 [00:27<00:00, 3009838.84tokens/s]
Shard 43:  98%|█████████▊| 97700103/100000000 [00:27<00:00, 3081695.43tokens/s]
Shard 43:  98%|█████████▊| 98018943/100000000 [00:27<00:00, 2829728.50tokens/s]
Shard 43:  98%|█████████▊| 98310894/100000000 [00:27<00:00, 2714173.49tokens/s]
Shard 43:  99%|█████████▊| 98655260/100000000 [00:27<00:00, 2895761.47tokens/s]
Shard 43:  99%|█████████▉| 98952478/100000000 [00:28<00:00, 2776792.39tokens/s]
Shard 43:  99%|█████████▉| 99252444/100000000 [00:28<00:00, 2827944.32tokens/s]
Shard 43: 100%|█████████▉| 99539542/100000000 [00:28<00:00, 2797907.32tokens/s]
Shard 43: 100%|█████████▉| 99917590/100000000 [00:28<00:00, 3051322.64tokens/s]
Shard 43: 100%|█████████▉| 99999881/100000000 [00:36<00:00, 2759999.88tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000043.bin (200,001,024 bytes) in the gpt-2 format

Shard 44:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 44:   0%|          | 199640/100000000 [00:00<00:50, 1991805.03tokens/s]
Shard 44:   1%|          | 502176/100000000 [00:00<00:38, 2592345.39tokens/s]
Shard 44:   1%|          | 790491/100000000 [00:00<00:36, 2719621.05tokens/s]
Shard 44:   1%|          | 1072515/100000000 [00:00<00:36, 2747086.52tokens/s]
Shard 44:   1%|▏         | 1444705/100000000 [00:00<00:31, 3094366.54tokens/s]
Shard 44:   2%|▏         | 1778594/100000000 [00:00<00:31, 3152751.68tokens/s]
Shard 44:   2%|▏         | 2094620/100000000 [00:00<00:32, 3010816.27tokens/s]
Shard 44:   2%|▏         | 2420804/100000000 [00:00<00:31, 3081373.53tokens/s]
Shard 44:   3%|▎         | 2731417/100000000 [00:00<00:32, 2972135.04tokens/s]
Shard 44:   3%|▎         | 3302065/100000000 [00:01<00:25, 3776734.17tokens/s]
Shard 44:   4%|▎         | 3684376/100000000 [00:01<00:27, 3484661.24tokens/s]
Shard 44:   4%|▍         | 4139123/100000000 [00:01<00:25, 3775756.06tokens/s]
Shard 44:   5%|▍         | 4527235/100000000 [00:01<00:28, 3353738.78tokens/s]
Shard 44:   5%|▍         | 4875955/100000000 [00:01<00:28, 3335985.88tokens/s]
Shard 44:   6%|▌         | 5502037/100000000 [00:01<00:22, 4128024.24tokens/s]
Shard 44:   6%|▌         | 5932473/100000000 [00:01<00:22, 4156503.88tokens/s]
Shard 44:   6%|▋         | 6361629/100000000 [00:01<00:24, 3898719.91tokens/s]
Shard 44:   7%|▋         | 6837206/100000000 [00:01<00:22, 4125445.50tokens/s]
Shard 44:   7%|▋         | 7259889/100000000 [00:02<00:24, 3814978.50tokens/s]
Shard 44:   8%|▊         | 7678518/100000000 [00:02<00:23, 3875862.18tokens/s]
Shard 44:   8%|▊         | 8073906/100000000 [00:02<00:24, 3825432.30tokens/s]
Shard 44:   8%|▊         | 8467056/100000000 [00:02<00:23, 3831964.73tokens/s]
Shard 44:   9%|▉         | 8856871/100000000 [00:02<00:23, 3823013.09tokens/s]
Shard 44:   9%|▉         | 9242517/100000000 [00:02<00:25, 3505744.11tokens/s]
Shard 44:  10%|▉         | 9599963/100000000 [00:02<00:25, 3485322.18tokens/s]
Shard 44:  10%|▉         | 9952895/100000000 [00:02<00:27, 3281639.33tokens/s]
Shard 44:  10%|█         | 10298888/100000000 [00:02<00:27, 3249556.11tokens/s]
Shard 44:  11%|█         | 10689672/100000000 [00:03<00:26, 3420621.85tokens/s]
Shard 44:  11%|█         | 11071868/100000000 [00:03<00:25, 3523074.18tokens/s]
Shard 44:  11%|█▏        | 11427662/100000000 [00:03<00:28, 3079022.02tokens/s]
Shard 44:  12%|█▏        | 12005084/100000000 [00:03<00:23, 3779237.00tokens/s]
Shard 44:  12%|█▏        | 12400502/100000000 [00:03<00:24, 3538534.46tokens/s]
Shard 44:  13%|█▎        | 12768963/100000000 [00:03<00:24, 3560367.73tokens/s]
Shard 44:  13%|█▎        | 13231276/100000000 [00:03<00:22, 3849253.34tokens/s]
Shard 44:  14%|█▎        | 13626219/100000000 [00:03<00:25, 3434144.12tokens/s]
Shard 44:  14%|█▍        | 14094135/100000000 [00:04<00:23, 3685448.29tokens/s]
Shard 44:  14%|█▍        | 14475871/100000000 [00:04<00:24, 3464375.70tokens/s]
Shard 44:  15%|█▍        | 14833769/100000000 [00:04<00:24, 3471549.13tokens/s]
Shard 44:  15%|█▌        | 15349719/100000000 [00:04<00:21, 3922909.53tokens/s]
Shard 44:  16%|█▌        | 15836155/100000000 [00:04<00:20, 4162743.79tokens/s]
Shard 44:  16%|█▋        | 16261289/100000000 [00:04<00:20, 4186674.84tokens/s]
Shard 44:  17%|█▋        | 16686034/100000000 [00:04<00:19, 4184739.01tokens/s]
Shard 44:  17%|█▋        | 17137642/100000000 [00:04<00:19, 4269492.78tokens/s]
Shard 44:  18%|█▊        | 17567863/100000000 [00:04<00:21, 3772208.14tokens/s]
Shard 44:  18%|█▊        | 17960190/100000000 [00:05<00:22, 3574734.38tokens/s]
Shard 44:  18%|█▊        | 18328187/100000000 [00:05<00:23, 3479834.11tokens/s]
Shard 44:  19%|█▉        | 18992008/100000000 [00:05<00:18, 4321487.57tokens/s]
Shard 44:  19%|█▉        | 19438397/100000000 [00:05<00:19, 4188801.28tokens/s]
Shard 44:  20%|██        | 20167679/100000000 [00:05<00:15, 5005665.87tokens/s]
Shard 44:  21%|██        | 20681879/100000000 [00:05<00:16, 4865888.30tokens/s]
Shard 44:  21%|██        | 21189431/100000000 [00:05<00:17, 4546649.44tokens/s]
Shard 44:  22%|██▏       | 21655699/100000000 [00:05<00:20, 3782217.00tokens/s]
Shard 44:  22%|██▏       | 22060864/100000000 [00:06<00:23, 3309267.75tokens/s]
Shard 44:  22%|██▏       | 22457110/100000000 [00:06<00:22, 3424691.72tokens/s]
Shard 44:  23%|██▎       | 22820257/100000000 [00:06<00:23, 3328068.99tokens/s]
Shard 44:  23%|██▎       | 23166448/100000000 [00:06<00:22, 3359181.30tokens/s]
Shard 44:  24%|██▎       | 23512460/100000000 [00:06<00:22, 3384539.92tokens/s]
Shard 44:  24%|██▍       | 23860563/100000000 [00:06<00:22, 3364754.94tokens/s]
Shard 44:  24%|██▍       | 24201998/100000000 [00:06<00:24, 3138201.58tokens/s]
Shard 44:  25%|██▍       | 24521676/100000000 [00:06<00:24, 3030081.50tokens/s]
Shard 44:  25%|██▍       | 24947509/100000000 [00:06<00:22, 3320410.68tokens/s]
Shard 44:  25%|██▌       | 25285411/100000000 [00:07<00:22, 3261180.22tokens/s]
Shard 44:  26%|██▌       | 25614840/100000000 [00:07<00:23, 3157783.07tokens/s]
Shard 44:  26%|██▌       | 25933203/100000000 [00:07<00:24, 3066308.75tokens/s]
Shard 44:  26%|██▋       | 26373405/100000000 [00:07<00:21, 3436534.83tokens/s]
Shard 44:  27%|██▋       | 26753021/100000000 [00:07<00:20, 3520982.81tokens/s]
Shard 44:  27%|██▋       | 27150545/100000000 [00:07<00:19, 3647284.51tokens/s]
Shard 44:  28%|██▊       | 27518601/100000000 [00:07<00:20, 3591025.13tokens/s]
Shard 44:  28%|██▊       | 27883396/100000000 [00:07<00:20, 3584419.49tokens/s]
Shard 44:  28%|██▊       | 28246029/100000000 [00:07<00:20, 3576910.89tokens/s]
Shard 44:  29%|██▊       | 28605362/100000000 [00:08<00:23, 2993215.76tokens/s]
Shard 44:  29%|██▉       | 28921997/100000000 [00:08<00:26, 2663643.88tokens/s]
Shard 44:  29%|██▉       | 29204812/100000000 [00:08<00:26, 2645151.29tokens/s]
Shard 44:  29%|██▉       | 29499719/100000000 [00:08<00:25, 2721632.06tokens/s]
Shard 44:  30%|██▉       | 29780928/100000000 [00:08<00:27, 2578025.50tokens/s]
Shard 44:  30%|███       | 30135162/100000000 [00:08<00:24, 2811005.77tokens/s]
Shard 44:  31%|███       | 30503471/100000000 [00:08<00:22, 3033272.75tokens/s]
Shard 44:  31%|███       | 30941955/100000000 [00:08<00:20, 3408980.31tokens/s]
Shard 44:  31%|███▏      | 31378850/100000000 [00:08<00:18, 3678257.91tokens/s]
Shard 44:  32%|███▏      | 31754012/100000000 [00:09<00:20, 3336253.93tokens/s]
Shard 44:  32%|███▏      | 32098794/100000000 [00:09<00:21, 3158842.73tokens/s]
Shard 44:  32%|███▏      | 32423051/100000000 [00:09<00:21, 3161588.77tokens/s]
Shard 44:  33%|███▎      | 32759374/100000000 [00:09<00:20, 3215543.98tokens/s]
Shard 44:  33%|███▎      | 33086787/100000000 [00:09<00:22, 2939982.12tokens/s]
Shard 44:  34%|███▎      | 33558672/100000000 [00:09<00:19, 3412895.44tokens/s]
Shard 44:  34%|███▍      | 34293720/100000000 [00:09<00:14, 4479962.39tokens/s]
Shard 44:  35%|███▍      | 34756548/100000000 [00:09<00:15, 4314113.65tokens/s]
Shard 44:  35%|███▌      | 35450423/100000000 [00:09<00:12, 5043070.28tokens/s]
Shard 44:  36%|███▌      | 35968937/100000000 [00:10<00:13, 4864101.84tokens/s]
Shard 44:  36%|███▋      | 36466285/100000000 [00:10<00:14, 4528369.35tokens/s]
Shard 44:  37%|███▋      | 36930580/100000000 [00:10<00:14, 4429661.48tokens/s]
Shard 44:  37%|███▋      | 37381283/100000000 [00:10<00:15, 4000576.20tokens/s]
Shard 44:  38%|███▊      | 37856265/100000000 [00:10<00:14, 4182115.81tokens/s]
Shard 44:  39%|███▊      | 38653395/100000000 [00:10<00:11, 5192967.14tokens/s]
Shard 44:  39%|███▉      | 39191174/100000000 [00:10<00:12, 4842958.52tokens/s]
Shard 44:  40%|███▉      | 39691718/100000000 [00:10<00:13, 4593137.40tokens/s]
Shard 44:  40%|████      | 40171693/100000000 [00:10<00:12, 4643871.83tokens/s]
Shard 44:  41%|████      | 40692182/100000000 [00:11<00:12, 4774357.18tokens/s]
Shard 44:  41%|████      | 41177174/100000000 [00:11<00:13, 4326600.06tokens/s]
Shard 44:  42%|████▏     | 41623266/100000000 [00:11<00:14, 3997389.91tokens/s]
Shard 44:  42%|████▏     | 42138586/100000000 [00:11<00:13, 4276650.91tokens/s]
Shard 44:  43%|████▎     | 42628093/100000000 [00:11<00:12, 4423426.49tokens/s]
Shard 44:  43%|████▎     | 43080339/100000000 [00:11<00:13, 4259388.12tokens/s]
Shard 44:  44%|████▎     | 43514098/100000000 [00:11<00:13, 4279073.02tokens/s]
Shard 44:  44%|████▍     | 43948774/100000000 [00:11<00:13, 4129895.04tokens/s]
Shard 44:  45%|████▍     | 44559132/100000000 [00:11<00:12, 4573923.67tokens/s]
Shard 44:  45%|████▌     | 45020606/100000000 [00:12<00:13, 4111663.57tokens/s]
Shard 44:  45%|████▌     | 45441542/100000000 [00:12<00:14, 3749569.71tokens/s]
Shard 44:  46%|████▌     | 45827809/100000000 [00:12<00:14, 3649724.40tokens/s]
Shard 44:  46%|████▋     | 46481816/100000000 [00:12<00:12, 4398240.82tokens/s]
Shard 44:  47%|████▋     | 46937558/100000000 [00:12<00:14, 3763720.71tokens/s]
Shard 44:  47%|████▋     | 47338852/100000000 [00:12<00:14, 3635315.75tokens/s]
Shard 44:  48%|████▊     | 47984300/100000000 [00:12<00:11, 4336198.24tokens/s]
Shard 44:  48%|████▊     | 48442960/100000000 [00:13<00:13, 3902563.46tokens/s]
Shard 44:  49%|████▉     | 48856568/100000000 [00:13<00:14, 3587880.26tokens/s]
Shard 44:  49%|████▉     | 49233826/100000000 [00:13<00:15, 3323861.84tokens/s]
Shard 44:  50%|████▉     | 49582084/100000000 [00:13<00:15, 3233035.07tokens/s]
Shard 44:  50%|█████     | 50066438/100000000 [00:13<00:13, 3618211.31tokens/s]
Shard 44:  50%|█████     | 50441538/100000000 [00:13<00:13, 3552195.71tokens/s]
Shard 44:  51%|█████     | 50805536/100000000 [00:13<00:15, 3171447.68tokens/s]
Shard 44:  51%|█████     | 51170636/100000000 [00:13<00:14, 3287449.92tokens/s]
Shard 44:  52%|█████▏    | 51511578/100000000 [00:14<00:15, 3195033.46tokens/s]
Shard 44:  52%|█████▏    | 51966783/100000000 [00:14<00:13, 3554359.00tokens/s]
Shard 44:  52%|█████▏    | 52336304/100000000 [00:14<00:13, 3518785.44tokens/s]
Shard 44:  53%|█████▎    | 52695588/100000000 [00:14<00:14, 3258627.25tokens/s]
Shard 44:  53%|█████▎    | 53056368/100000000 [00:14<00:14, 3339810.25tokens/s]
Shard 44:  54%|█████▎    | 53625552/100000000 [00:14<00:11, 3988906.78tokens/s]
Shard 44:  55%|█████▍    | 54567110/100000000 [00:14<00:08, 5508112.72tokens/s]
Shard 44:  55%|█████▌    | 55132779/100000000 [00:14<00:08, 5129634.47tokens/s]
Shard 44:  56%|█████▌    | 55660878/100000000 [00:14<00:08, 5024954.45tokens/s]
Shard 44:  56%|█████▌    | 56179036/100000000 [00:15<00:09, 4442841.55tokens/s]
Shard 44:  57%|█████▋    | 56641223/100000000 [00:15<00:10, 4125811.85tokens/s]
Shard 44:  57%|█████▋    | 57070379/100000000 [00:15<00:10, 4059752.45tokens/s]
Shard 44:  58%|█████▊    | 57533521/100000000 [00:15<00:10, 4193527.43tokens/s]
Shard 44:  58%|█████▊    | 57963115/100000000 [00:15<00:10, 4028765.60tokens/s]
Shard 44:  58%|█████▊    | 58455647/100000000 [00:15<00:09, 4252885.58tokens/s]
Shard 44:  59%|█████▉    | 58887953/100000000 [00:15<00:10, 3826406.13tokens/s]
Shard 44:  59%|█████▉    | 59338055/100000000 [00:15<00:10, 3971828.12tokens/s]
Shard 44:  60%|█████▉    | 59746846/100000000 [00:15<00:10, 3958211.78tokens/s]
Shard 44:  60%|██████    | 60149431/100000000 [00:16<00:10, 3844993.53tokens/s]
Shard 44:  61%|██████    | 60540005/100000000 [00:16<00:10, 3805856.78tokens/s]
Shard 44:  61%|██████    | 61006207/100000000 [00:16<00:09, 4045431.14tokens/s]
Shard 44:  61%|██████▏   | 61454148/100000000 [00:16<00:09, 4160595.40tokens/s]
Shard 44:  62%|██████▏   | 61875310/100000000 [00:16<00:09, 4143539.79tokens/s]
Shard 44:  62%|██████▏   | 62292123/100000000 [00:16<00:09, 3974697.42tokens/s]
Shard 44:  63%|██████▎   | 62692559/100000000 [00:16<00:10, 3710651.99tokens/s]
Shard 44:  63%|██████▎   | 63183844/100000000 [00:16<00:09, 4040819.31tokens/s]
Shard 44:  64%|██████▎   | 63594383/100000000 [00:16<00:09, 3819911.56tokens/s]
Shard 44:  64%|██████▍   | 63983153/100000000 [00:17<00:09, 3689495.03tokens/s]
Shard 44:  64%|██████▍   | 64356868/100000000 [00:17<00:10, 3398155.53tokens/s]
Shard 44:  65%|██████▍   | 64749353/100000000 [00:17<00:09, 3537040.79tokens/s]
Shard 44:  65%|██████▌   | 65109243/100000000 [00:17<00:11, 3032366.92tokens/s]
Shard 44:  65%|██████▌   | 65428005/100000000 [00:17<00:11, 3044223.59tokens/s]
Shard 44:  66%|██████▌   | 65755909/100000000 [00:17<00:11, 3071331.29tokens/s]
Shard 44:  66%|██████▌   | 66093116/100000000 [00:17<00:10, 3141257.49tokens/s]
Shard 44:  66%|██████▋   | 66413974/100000000 [00:17<00:11, 2870435.78tokens/s]
Shard 44:  67%|██████▋   | 66708875/100000000 [00:18<00:13, 2515348.43tokens/s]
Shard 44:  67%|██████▋   | 67086474/100000000 [00:18<00:11, 2827385.53tokens/s]
Shard 44:  68%|██████▊   | 67626949/100000000 [00:18<00:09, 3495992.44tokens/s]
Shard 44:  68%|██████▊   | 68126894/100000000 [00:18<00:08, 3892110.83tokens/s]
Shard 44:  69%|██████▊   | 68589422/100000000 [00:18<00:07, 4091504.88tokens/s]
Shard 44:  69%|██████▉   | 69040568/100000000 [00:18<00:07, 4207097.31tokens/s]
Shard 44:  69%|██████▉   | 69472112/100000000 [00:18<00:07, 4105724.49tokens/s]
Shard 44:  70%|██████▉   | 69890252/100000000 [00:18<00:08, 3751524.88tokens/s]
Shard 44:  71%|███████   | 70608704/100000000 [00:18<00:06, 4671148.65tokens/s]
Shard 44:  71%|███████   | 71092704/100000000 [00:19<00:06, 4181736.16tokens/s]
Shard 44:  72%|███████▏  | 71640552/100000000 [00:19<00:06, 4514048.51tokens/s]
Shard 44:  72%|███████▏  | 72222965/100000000 [00:19<00:05, 4867968.33tokens/s]
Shard 44:  73%|███████▎  | 72729349/100000000 [00:19<00:06, 4257324.86tokens/s]
Shard 44:  73%|███████▎  | 73179896/100000000 [00:19<00:06, 4177326.78tokens/s]
Shard 44:  74%|███████▎  | 73620367/100000000 [00:19<00:06, 3858074.11tokens/s]
Shard 44:  74%|███████▍  | 74025849/100000000 [00:19<00:07, 3476867.59tokens/s]
Shard 44:  74%|███████▍  | 74389725/100000000 [00:19<00:07, 3320375.17tokens/s]
Shard 44:  75%|███████▍  | 74772074/100000000 [00:20<00:07, 3433653.95tokens/s]
Shard 44:  75%|███████▌  | 75126772/100000000 [00:20<00:07, 3304747.25tokens/s]
Shard 44:  76%|███████▌  | 75669799/100000000 [00:20<00:06, 3857696.13tokens/s]
Shard 44:  76%|███████▌  | 76071067/100000000 [00:20<00:07, 3347327.11tokens/s]
Shard 44:  77%|███████▋  | 76542518/100000000 [00:20<00:06, 3684463.71tokens/s]
Shard 44:  77%|███████▋  | 76930776/100000000 [00:20<00:06, 3363716.09tokens/s]
Shard 44:  77%|███████▋  | 77284719/100000000 [00:20<00:07, 3178556.64tokens/s]
Shard 44:  78%|███████▊  | 77616786/100000000 [00:20<00:07, 2801760.74tokens/s]
Shard 44:  78%|███████▊  | 78040745/100000000 [00:21<00:06, 3138973.42tokens/s]
Shard 44:  79%|███████▊  | 78725443/100000000 [00:21<00:05, 4078746.86tokens/s]
Shard 44:  79%|███████▉  | 79316455/100000000 [00:21<00:04, 4556192.89tokens/s]
Shard 44:  80%|███████▉  | 79798704/100000000 [00:21<00:05, 3968280.00tokens/s]
Shard 44:  80%|████████  | 80230726/100000000 [00:21<00:05, 3758016.16tokens/s]
Shard 44:  81%|████████  | 80632512/100000000 [00:21<00:05, 3581760.81tokens/s]
Shard 44:  81%|████████  | 81005921/100000000 [00:21<00:05, 3600818.25tokens/s]
Shard 44:  81%|████████▏ | 81400392/100000000 [00:21<00:05, 3676183.96tokens/s]
Shard 44:  82%|████████▏ | 81776703/100000000 [00:22<00:05, 3120003.06tokens/s]
Shard 44:  82%|████████▏ | 82107554/100000000 [00:22<00:06, 2854203.50tokens/s]
Shard 44:  83%|████████▎ | 82592340/100000000 [00:22<00:05, 3284483.07tokens/s]
Shard 44:  83%|████████▎ | 82939343/100000000 [00:22<00:05, 3320887.07tokens/s]
Shard 44:  83%|████████▎ | 83327375/100000000 [00:22<00:04, 3465420.30tokens/s]
Shard 44:  84%|████████▎ | 83685383/100000000 [00:22<00:04, 3275891.98tokens/s]
Shard 44:  84%|████████▍ | 84022958/100000000 [00:22<00:04, 3260541.42tokens/s]
Shard 44:  84%|████████▍ | 84360574/100000000 [00:22<00:04, 3286993.48tokens/s]
Shard 44:  85%|████████▍ | 84694022/100000000 [00:22<00:04, 3262207.52tokens/s]
Shard 44:  85%|████████▌ | 85023386/100000000 [00:23<00:04, 3041506.65tokens/s]
Shard 44:  85%|████████▌ | 85423827/100000000 [00:23<00:04, 3299826.36tokens/s]
Shard 44:  86%|████████▌ | 85809856/100000000 [00:23<00:04, 3454421.17tokens/s]
Shard 44:  86%|████████▌ | 86160327/100000000 [00:23<00:04, 3305486.44tokens/s]
Shard 44:  87%|████████▋ | 86716811/100000000 [00:23<00:03, 3909392.18tokens/s]
Shard 44:  87%|████████▋ | 87185230/100000000 [00:23<00:03, 4097789.82tokens/s]
Shard 44:  88%|████████▊ | 87661569/100000000 [00:23<00:02, 4276029.41tokens/s]
Shard 44:  88%|████████▊ | 88093659/100000000 [00:23<00:02, 4024378.86tokens/s]
Shard 44:  89%|████████▊ | 88744341/100000000 [00:23<00:02, 4703667.77tokens/s]
Shard 44:  89%|████████▉ | 89222791/100000000 [00:24<00:02, 4373904.54tokens/s]
Shard 44:  90%|████████▉ | 89669524/100000000 [00:24<00:02, 4074563.36tokens/s]
Shard 44:  90%|█████████ | 90089054/100000000 [00:24<00:02, 4095599.74tokens/s]
Shard 44:  91%|█████████ | 90547961/100000000 [00:24<00:02, 4187729.45tokens/s]
Shard 44:  91%|█████████ | 91065923/100000000 [00:24<00:02, 4459270.91tokens/s]
Shard 44:  92%|█████████▏| 91837342/100000000 [00:24<00:01, 5376379.55tokens/s]
Shard 44:  92%|█████████▏| 92383540/100000000 [00:24<00:01, 5138764.70tokens/s]
Shard 44:  93%|█████████▎| 92904833/100000000 [00:24<00:01, 4532331.21tokens/s]
Shard 44:  93%|█████████▎| 93374929/100000000 [00:24<00:01, 4424466.16tokens/s]
Shard 44:  94%|█████████▍| 93828627/100000000 [00:25<00:01, 4201216.30tokens/s]
Shard 44:  94%|█████████▍| 94257579/100000000 [00:25<00:01, 4162786.26tokens/s]
Shard 44:  95%|█████████▍| 94726189/100000000 [00:25<00:01, 4301179.45tokens/s]
Shard 44:  95%|█████████▌| 95161665/100000000 [00:25<00:01, 3732639.30tokens/s]
Shard 44:  96%|█████████▌| 95755192/100000000 [00:25<00:00, 4274179.19tokens/s]
Shard 44:  96%|█████████▌| 96201915/100000000 [00:25<00:00, 4121823.57tokens/s]
Shard 44:  97%|█████████▋| 96628278/100000000 [00:25<00:00, 4048962.42tokens/s]
Shard 44:  97%|█████████▋| 97042346/100000000 [00:25<00:00, 3697826.46tokens/s]
Shard 44:  98%|█████████▊| 97585972/100000000 [00:25<00:00, 4127139.39tokens/s]
Shard 44:  98%|█████████▊| 98012564/100000000 [00:26<00:00, 4050290.46tokens/s]
Shard 44:  98%|█████████▊| 98429583/100000000 [00:26<00:00, 3999753.67tokens/s]
Shard 44:  99%|█████████▉| 98835675/100000000 [00:26<00:00, 3908131.84tokens/s]
Shard 44:  99%|█████████▉| 99328871/100000000 [00:26<00:00, 4182657.36tokens/s]
Shard 44: 100%|█████████▉| 99755609/100000000 [00:26<00:00, 4118002.34tokens/s]
Shard 44: 100%|█████████▉| 99998785/100000000 [00:34<00:00, 2914802.35tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000044.bin (200,001,024 bytes) in the gpt-2 format

Shard 45:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 45:   0%|          | 156048/100000000 [00:00<01:05, 1513712.77tokens/s]
Shard 45:   0%|          | 329875/100000000 [00:00<01:02, 1582145.17tokens/s]
Shard 45:   1%|          | 616001/100000000 [00:00<00:46, 2142761.39tokens/s]
Shard 45:   1%|          | 909804/100000000 [00:00<00:40, 2428173.75tokens/s]
Shard 45:   1%|          | 1153990/100000000 [00:00<00:40, 2421385.50tokens/s]
Shard 45:   2%|▏         | 1858390/100000000 [00:00<00:24, 3967465.94tokens/s]
Shard 45:   2%|▏         | 2259431/100000000 [00:00<00:26, 3711456.75tokens/s]
Shard 45:   3%|▎         | 2635744/100000000 [00:00<00:27, 3479080.49tokens/s]
Shard 45:   3%|▎         | 3059969/100000000 [00:00<00:26, 3681567.26tokens/s]
Shard 45:   3%|▎         | 3433507/100000000 [00:01<00:29, 3245914.44tokens/s]
Shard 45:   4%|▍         | 3769951/100000000 [00:01<00:29, 3247344.25tokens/s]
Shard 45:   4%|▍         | 4223622/100000000 [00:01<00:26, 3580057.28tokens/s]
Shard 45:   5%|▍         | 4591470/100000000 [00:01<00:27, 3407795.20tokens/s]
Shard 45:   5%|▍         | 4939815/100000000 [00:01<00:29, 3221133.09tokens/s]
Shard 45:   5%|▌         | 5268144/100000000 [00:01<00:34, 2727915.72tokens/s]
Shard 45:   6%|▌         | 5555846/100000000 [00:01<00:38, 2443733.22tokens/s]
Shard 45:   6%|▌         | 5826552/100000000 [00:01<00:37, 2503037.08tokens/s]
Shard 45:   6%|▌         | 6125729/100000000 [00:02<00:35, 2616446.37tokens/s]
Shard 45:   7%|▋         | 6501399/100000000 [00:02<00:32, 2916951.46tokens/s]
Shard 45:   7%|▋         | 6804504/100000000 [00:02<00:33, 2806794.73tokens/s]
Shard 45:   7%|▋         | 7093290/100000000 [00:02<00:35, 2595981.86tokens/s]
Shard 45:   7%|▋         | 7462520/100000000 [00:02<00:32, 2883609.05tokens/s]
Shard 45:   8%|▊         | 7892681/100000000 [00:02<00:28, 3262282.96tokens/s]
Shard 45:   8%|▊         | 8498647/100000000 [00:02<00:22, 3991915.57tokens/s]
Shard 45:   9%|▉         | 8911064/100000000 [00:02<00:24, 3737702.52tokens/s]
Shard 45:   9%|▉         | 9298422/100000000 [00:02<00:24, 3653697.94tokens/s]
Shard 45:  10%|▉         | 9702833/100000000 [00:03<00:24, 3748346.50tokens/s]
Shard 45:  10%|█         | 10083888/100000000 [00:03<00:24, 3738265.15tokens/s]
Shard 45:  10%|█         | 10467949/100000000 [00:03<00:23, 3767379.41tokens/s]
Shard 45:  11%|█         | 10847482/100000000 [00:03<00:23, 3760602.04tokens/s]
Shard 45:  11%|█▏        | 11293945/100000000 [00:03<00:22, 3944280.39tokens/s]
Shard 45:  12%|█▏        | 11690828/100000000 [00:03<00:25, 3458465.61tokens/s]
Shard 45:  12%|█▏        | 12050393/100000000 [00:03<00:27, 3146172.71tokens/s]
Shard 45:  12%|█▏        | 12438626/100000000 [00:03<00:26, 3316708.59tokens/s]
Shard 45:  13%|█▎        | 12781178/100000000 [00:03<00:26, 3330040.68tokens/s]
Shard 45:  13%|█▎        | 13125883/100000000 [00:04<00:27, 3112886.71tokens/s]
Shard 45:  13%|█▎        | 13445354/100000000 [00:04<00:28, 3051255.99tokens/s]
Shard 45:  14%|█▍        | 13756022/100000000 [00:04<00:28, 3050776.26tokens/s]
Shard 45:  14%|█▍        | 14112934/100000000 [00:04<00:26, 3185934.67tokens/s]
Shard 45:  15%|█▍        | 14549485/100000000 [00:04<00:24, 3507939.81tokens/s]
Shard 45:  15%|█▍        | 14904857/100000000 [00:04<00:25, 3366904.14tokens/s]
Shard 45:  15%|█▌        | 15401272/100000000 [00:04<00:22, 3817362.61tokens/s]
Shard 45:  16%|█▌        | 15788663/100000000 [00:04<00:22, 3819066.86tokens/s]
Shard 45:  16%|█▌        | 16174629/100000000 [00:04<00:22, 3751134.38tokens/s]
Shard 45:  17%|█▋        | 16559513/100000000 [00:05<00:23, 3500442.72tokens/s]
Shard 45:  17%|█▋        | 16988056/100000000 [00:05<00:22, 3704449.59tokens/s]
Shard 45:  18%|█▊        | 17591156/100000000 [00:05<00:18, 4352635.53tokens/s]
Shard 45:  18%|█▊        | 18034050/100000000 [00:05<00:19, 4200868.45tokens/s]
Shard 45:  18%|█▊        | 18477486/100000000 [00:05<00:19, 4260157.29tokens/s]
Shard 45:  19%|█▉        | 18935181/100000000 [00:05<00:18, 4339582.59tokens/s]
Shard 45:  20%|█▉        | 19695542/100000000 [00:05<00:15, 5287231.61tokens/s]
Shard 45:  20%|██        | 20229756/100000000 [00:05<00:17, 4681322.89tokens/s]
Shard 45:  21%|██        | 20727779/100000000 [00:05<00:16, 4745960.75tokens/s]
Shard 45:  21%|██        | 21214565/100000000 [00:06<00:17, 4503430.76tokens/s]
Shard 45:  22%|██▏       | 21675819/100000000 [00:06<00:17, 4500427.79tokens/s]
Shard 45:  22%|██▏       | 22141805/100000000 [00:06<00:19, 3947203.13tokens/s]
Shard 45:  23%|██▎       | 22552219/100000000 [00:06<00:19, 3920810.54tokens/s]
Shard 45:  23%|██▎       | 23194642/100000000 [00:06<00:16, 4581773.67tokens/s]
Shard 45:  24%|██▎       | 23669506/100000000 [00:06<00:16, 4613242.58tokens/s]
Shard 45:  24%|██▍       | 24147267/100000000 [00:06<00:18, 4103370.93tokens/s]
Shard 45:  25%|██▍       | 24741901/100000000 [00:06<00:16, 4529720.52tokens/s]
Shard 45:  25%|██▌       | 25250860/100000000 [00:06<00:16, 4664307.11tokens/s]
Shard 45:  26%|██▌       | 25734030/100000000 [00:07<00:16, 4428939.41tokens/s]
Shard 45:  26%|██▌       | 26188137/100000000 [00:07<00:19, 3802634.80tokens/s]
Shard 45:  27%|██▋       | 26653942/100000000 [00:07<00:18, 4014950.74tokens/s]
Shard 45:  27%|██▋       | 27077425/100000000 [00:07<00:20, 3614092.54tokens/s]
Shard 45:  28%|██▊       | 27531291/100000000 [00:07<00:18, 3830289.04tokens/s]
Shard 45:  28%|██▊       | 27931842/100000000 [00:07<00:19, 3705990.55tokens/s]
Shard 45:  28%|██▊       | 28314474/100000000 [00:07<00:19, 3621593.19tokens/s]
Shard 45:  29%|██▊       | 28684582/100000000 [00:07<00:20, 3564194.04tokens/s]
Shard 45:  29%|██▉       | 29046226/100000000 [00:08<00:20, 3438356.02tokens/s]
Shard 45:  29%|██▉       | 29394315/100000000 [00:08<00:22, 3143372.28tokens/s]
Shard 45:  30%|██▉       | 29844385/100000000 [00:08<00:20, 3493137.26tokens/s]
Shard 45:  30%|███       | 30203989/100000000 [00:08<00:20, 3355108.68tokens/s]
Shard 45:  31%|███       | 30651702/100000000 [00:08<00:18, 3657764.18tokens/s]
Shard 45:  31%|███       | 31025931/100000000 [00:08<00:19, 3543899.34tokens/s]
Shard 45:  31%|███▏      | 31386428/100000000 [00:08<00:20, 3318993.86tokens/s]
Shard 45:  32%|███▏      | 31740761/100000000 [00:08<00:20, 3370464.93tokens/s]
Shard 45:  32%|███▏      | 32083135/100000000 [00:08<00:21, 3182802.11tokens/s]
Shard 45:  32%|███▏      | 32406946/100000000 [00:09<00:21, 3186881.73tokens/s]
Shard 45:  33%|███▎      | 32857304/100000000 [00:09<00:18, 3535865.62tokens/s]
Shard 45:  33%|███▎      | 33238587/100000000 [00:09<00:18, 3597427.65tokens/s]
Shard 45:  34%|███▎      | 33618701/100000000 [00:09<00:18, 3643308.86tokens/s]
Shard 45:  34%|███▍      | 34126383/100000000 [00:09<00:16, 4058750.00tokens/s]
Shard 45:  35%|███▍      | 34706862/100000000 [00:09<00:14, 4557510.19tokens/s]
Shard 45:  35%|███▌      | 35244540/100000000 [00:09<00:13, 4798848.11tokens/s]
Shard 45:  36%|███▌      | 35782826/100000000 [00:09<00:12, 4971556.23tokens/s]
Shard 45:  36%|███▋      | 36331843/100000000 [00:09<00:12, 5058713.15tokens/s]
Shard 45:  37%|███▋      | 36933577/100000000 [00:10<00:11, 5327355.68tokens/s]
Shard 45:  37%|███▋      | 37467683/100000000 [00:10<00:11, 5315164.65tokens/s]
Shard 45:  38%|███▊      | 38077273/100000000 [00:10<00:11, 5533743.89tokens/s]
Shard 45:  39%|███▊      | 38692127/100000000 [00:10<00:10, 5714785.46tokens/s]
Shard 45:  39%|███▉      | 39264652/100000000 [00:10<00:12, 5009036.73tokens/s]
Shard 45:  40%|███▉      | 39782311/100000000 [00:10<00:13, 4316353.27tokens/s]
Shard 45:  40%|████      | 40240409/100000000 [00:10<00:13, 4367140.00tokens/s]
Shard 45:  41%|████      | 40733216/100000000 [00:10<00:13, 4513677.36tokens/s]
Shard 45:  41%|████      | 41208296/100000000 [00:10<00:12, 4555862.94tokens/s]
Shard 45:  42%|████▏     | 41675877/100000000 [00:11<00:13, 4307012.97tokens/s]
Shard 45:  42%|████▏     | 42164824/100000000 [00:11<00:12, 4463637.62tokens/s]
Shard 45:  43%|████▎     | 42621809/100000000 [00:11<00:13, 4402609.51tokens/s]
Shard 45:  43%|████▎     | 43067888/100000000 [00:11<00:13, 4328068.92tokens/s]
Shard 45:  44%|████▎     | 43505183/100000000 [00:11<00:13, 4172558.83tokens/s]
Shard 45:  44%|████▍     | 44208231/100000000 [00:11<00:11, 4961017.70tokens/s]
Shard 45:  45%|████▍     | 44712196/100000000 [00:11<00:12, 4305130.69tokens/s]
Shard 45:  45%|████▌     | 45162155/100000000 [00:11<00:13, 3968595.87tokens/s]
Shard 45:  46%|████▌     | 45591110/100000000 [00:11<00:13, 4047631.59tokens/s]
Shard 45:  46%|████▌     | 46072684/100000000 [00:12<00:12, 4246850.08tokens/s]
Shard 45:  47%|████▋     | 46509012/100000000 [00:12<00:13, 4077553.58tokens/s]
Shard 45:  47%|████▋     | 47065126/100000000 [00:12<00:11, 4454460.51tokens/s]
Shard 45:  48%|████▊     | 47977588/100000000 [00:12<00:09, 5752027.28tokens/s]
Shard 45:  49%|████▊     | 48569598/100000000 [00:12<00:10, 4809318.65tokens/s]
Shard 45:  49%|████▉     | 49086646/100000000 [00:12<00:10, 4644524.78tokens/s]
Shard 45:  50%|████▉     | 49579032/100000000 [00:12<00:11, 4291700.53tokens/s]
Shard 45:  50%|█████     | 50029551/100000000 [00:12<00:11, 4308436.65tokens/s]
Shard 45:  51%|█████     | 50654243/100000000 [00:13<00:10, 4803202.77tokens/s]
Shard 45:  51%|█████     | 51154166/100000000 [00:13<00:11, 4305746.10tokens/s]
Shard 45:  52%|█████▏    | 51655315/100000000 [00:13<00:10, 4455627.66tokens/s]
Shard 45:  52%|█████▏    | 52116907/100000000 [00:13<00:10, 4417717.69tokens/s]
Shard 45:  53%|█████▎    | 52572544/100000000 [00:13<00:10, 4413321.36tokens/s]
Shard 45:  53%|█████▎    | 53052877/100000000 [00:13<00:10, 4501941.33tokens/s]
Shard 45:  54%|█████▎    | 53509131/100000000 [00:13<00:10, 4350969.44tokens/s]
Shard 45:  54%|█████▍    | 53949466/100000000 [00:13<00:10, 4307127.03tokens/s]
Shard 45:  55%|█████▍    | 54560303/100000000 [00:13<00:09, 4813277.26tokens/s]
Shard 45:  55%|█████▌    | 55049814/100000000 [00:14<00:10, 4494572.32tokens/s]
Shard 45:  56%|█████▌    | 55628099/100000000 [00:14<00:09, 4848058.07tokens/s]
Shard 45:  56%|█████▋    | 56282833/100000000 [00:14<00:08, 5293595.87tokens/s]
Shard 45:  57%|█████▋    | 56819783/100000000 [00:14<00:08, 5312981.08tokens/s]
Shard 45:  57%|█████▋    | 57385054/100000000 [00:14<00:07, 5411616.83tokens/s]
Shard 45:  58%|█████▊    | 57930287/100000000 [00:14<00:07, 5356247.01tokens/s]
Shard 45:  58%|█████▊    | 58476460/100000000 [00:14<00:07, 5383759.74tokens/s]
Shard 45:  59%|█████▉    | 59018500/100000000 [00:14<00:08, 4626785.10tokens/s]
Shard 45:  60%|█████▉    | 59500952/100000000 [00:14<00:08, 4540230.46tokens/s]
Shard 45:  60%|██████    | 60055166/100000000 [00:15<00:08, 4795053.51tokens/s]
Shard 45:  61%|██████    | 60558415/100000000 [00:15<00:08, 4846479.25tokens/s]
Shard 45:  61%|██████    | 61053999/100000000 [00:15<00:08, 4631829.45tokens/s]
Shard 45:  62%|██████▏   | 61530362/100000000 [00:15<00:08, 4665624.74tokens/s]
Shard 45:  62%|██████▏   | 62002472/100000000 [00:15<00:09, 4076427.73tokens/s]
Shard 45:  62%|██████▏   | 62467350/100000000 [00:15<00:08, 4215314.89tokens/s]
Shard 45:  63%|██████▎   | 63026410/100000000 [00:15<00:08, 4572079.29tokens/s]
Shard 45:  64%|██████▎   | 63503465/100000000 [00:15<00:07, 4619567.92tokens/s]
Shard 45:  64%|██████▍   | 63974656/100000000 [00:15<00:08, 4372564.68tokens/s]
Shard 45:  64%|██████▍   | 64420456/100000000 [00:16<00:08, 4133662.43tokens/s]
Shard 45:  65%|██████▍   | 64841544/100000000 [00:16<00:09, 3902603.49tokens/s]
Shard 45:  66%|██████▌   | 65859912/100000000 [00:16<00:06, 5527212.09tokens/s]
Shard 45:  66%|██████▋   | 66433346/100000000 [00:16<00:06, 5391556.78tokens/s]
Shard 45:  67%|██████▋   | 66988720/100000000 [00:16<00:06, 5230716.46tokens/s]
Shard 45:  68%|██████▊   | 67522283/100000000 [00:16<00:06, 5251889.71tokens/s]
Shard 45:  68%|██████▊   | 68086202/100000000 [00:16<00:06, 5299999.76tokens/s]
Shard 45:  69%|██████▊   | 68623001/100000000 [00:16<00:06, 5089748.64tokens/s]
Shard 45:  69%|██████▉   | 69138314/100000000 [00:16<00:06, 5029231.73tokens/s]
Shard 45:  70%|██████▉   | 69644458/100000000 [00:17<00:06, 4865397.49tokens/s]
Shard 45:  70%|███████   | 70133764/100000000 [00:17<00:06, 4589754.20tokens/s]
Shard 45:  71%|███████   | 70596658/100000000 [00:17<00:07, 4171092.12tokens/s]
Shard 45:  71%|███████   | 71146247/100000000 [00:17<00:06, 4511134.80tokens/s]
Shard 45:  72%|███████▏  | 71607549/100000000 [00:17<00:06, 4078368.57tokens/s]
Shard 45:  72%|███████▏  | 72028691/100000000 [00:17<00:07, 3909400.47tokens/s]
Shard 45:  72%|███████▏  | 72434767/100000000 [00:17<00:06, 3945050.03tokens/s]
Shard 45:  73%|███████▎  | 72892955/100000000 [00:17<00:06, 4107045.75tokens/s]
Shard 45:  73%|███████▎  | 73329964/100000000 [00:17<00:06, 4160136.99tokens/s]
Shard 45:  74%|███████▍  | 73892714/100000000 [00:18<00:05, 4563064.76tokens/s]
Shard 45:  74%|███████▍  | 74355909/100000000 [00:18<00:05, 4484331.35tokens/s]
Shard 45:  75%|███████▍  | 74808612/100000000 [00:18<00:05, 4416389.39tokens/s]
Shard 45:  75%|███████▌  | 75460875/100000000 [00:18<00:04, 5004810.63tokens/s]
Shard 45:  77%|███████▋  | 76517736/100000000 [00:18<00:03, 6618324.65tokens/s]
Shard 45:  77%|███████▋  | 77187970/100000000 [00:18<00:03, 6522212.31tokens/s]
Shard 45:  78%|███████▊  | 77996912/100000000 [00:18<00:03, 6965415.07tokens/s]
Shard 45:  79%|███████▊  | 78699050/100000000 [00:18<00:03, 5468653.48tokens/s]
Shard 45:  79%|███████▉  | 79298966/100000000 [00:19<00:04, 5017404.13tokens/s]
Shard 45:  80%|███████▉  | 79841746/100000000 [00:19<00:03, 5047661.26tokens/s]
Shard 45:  80%|████████  | 80379096/100000000 [00:19<00:04, 4901509.45tokens/s]
Shard 45:  81%|████████  | 80888992/100000000 [00:19<00:04, 4454935.95tokens/s]
Shard 45:  81%|████████▏ | 81352956/100000000 [00:19<00:04, 4065117.22tokens/s]
Shard 45:  82%|████████▏ | 81776862/100000000 [00:19<00:04, 4082992.01tokens/s]
Shard 45:  82%|████████▏ | 82196409/100000000 [00:19<00:04, 4027208.65tokens/s]
Shard 45:  83%|████████▎ | 82607828/100000000 [00:19<00:04, 3819339.42tokens/s]
Shard 45:  83%|████████▎ | 82996499/100000000 [00:19<00:04, 3720165.00tokens/s]
Shard 45:  84%|████████▎ | 83646502/100000000 [00:20<00:03, 4437167.66tokens/s]
Shard 45:  84%|████████▍ | 84242494/100000000 [00:20<00:03, 4840289.02tokens/s]
Shard 45:  85%|████████▌ | 85006128/100000000 [00:20<00:02, 5625927.71tokens/s]
Shard 45:  86%|████████▌ | 85581374/100000000 [00:20<00:02, 5477045.48tokens/s]
Shard 45:  86%|████████▌ | 86149787/100000000 [00:20<00:02, 5003453.05tokens/s]
Shard 45:  87%|████████▋ | 86663702/100000000 [00:20<00:03, 4223704.88tokens/s]
Shard 45:  87%|████████▋ | 87137556/100000000 [00:20<00:02, 4306030.06tokens/s]
Shard 45:  88%|████████▊ | 87589656/100000000 [00:20<00:03, 3948174.61tokens/s]
Shard 45:  88%|████████▊ | 88061037/100000000 [00:21<00:02, 4101894.90tokens/s]
Shard 45:  88%|████████▊ | 88493585/100000000 [00:21<00:03, 3821757.36tokens/s]
Shard 45:  89%|████████▉ | 88888084/100000000 [00:21<00:02, 3795676.88tokens/s]
Shard 45:  89%|████████▉ | 89399944/100000000 [00:21<00:02, 4141259.98tokens/s]
Shard 45:  90%|████████▉ | 89824523/100000000 [00:21<00:02, 3964189.13tokens/s]
Shard 45:  90%|█████████ | 90245565/100000000 [00:21<00:02, 3926125.08tokens/s]
Shard 45:  91%|█████████ | 90813057/100000000 [00:21<00:02, 4357371.44tokens/s]
Shard 45:  91%|█████████▏| 91255671/100000000 [00:21<00:02, 4348195.60tokens/s]
Shard 45:  92%|█████████▏| 91696420/100000000 [00:21<00:01, 4353291.96tokens/s]
Shard 45:  92%|█████████▏| 92135213/100000000 [00:22<00:01, 4345942.50tokens/s]
Shard 45:  93%|█████████▎| 92702832/100000000 [00:22<00:01, 4722906.21tokens/s]
Shard 45:  94%|█████████▎| 93570404/100000000 [00:22<00:01, 5870614.44tokens/s]
Shard 45:  94%|█████████▍| 94460630/100000000 [00:22<00:00, 6721121.31tokens/s]
Shard 45:  95%|█████████▌| 95139019/100000000 [00:22<00:00, 6258076.41tokens/s]
Shard 45:  96%|█████████▌| 95776942/100000000 [00:22<00:00, 6188907.34tokens/s]
Shard 45:  96%|█████████▋| 96401593/100000000 [00:22<00:00, 5101500.19tokens/s]
Shard 45:  97%|█████████▋| 96945105/100000000 [00:22<00:00, 4970198.54tokens/s]
Shard 45:  97%|█████████▋| 97469722/100000000 [00:22<00:00, 4568056.02tokens/s]
Shard 45:  98%|█████████▊| 97945249/100000000 [00:23<00:00, 4087535.29tokens/s]
Shard 45:  99%|█████████▊| 98560532/100000000 [00:23<00:00, 4572444.67tokens/s]
Shard 45:  99%|█████████▉| 99266778/100000000 [00:23<00:00, 5201317.62tokens/s]
Shard 45: 100%|█████████▉| 99925553/100000000 [00:23<00:00, 5566575.02tokens/s]
Shard 45: 100%|█████████▉| 99999835/100000000 [00:31<00:00, 3125614.95tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000045.bin (200,001,024 bytes) in the gpt-2 format

Shard 46:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 46:   0%|          | 247803/100000000 [00:00<00:40, 2445872.09tokens/s]
Shard 46:   1%|          | 524792/100000000 [00:00<00:38, 2617079.88tokens/s]
Shard 46:   1%|          | 817379/100000000 [00:00<00:36, 2723991.62tokens/s]
Shard 46:   1%|          | 1089768/100000000 [00:00<00:38, 2543545.96tokens/s]
Shard 46:   1%|▏         | 1494352/100000000 [00:00<00:32, 3012925.46tokens/s]
Shard 46:   2%|▏         | 1952562/100000000 [00:00<00:27, 3516700.69tokens/s]
Shard 46:   2%|▏         | 2308029/100000000 [00:00<00:27, 3504520.75tokens/s]
Shard 46:   3%|▎         | 2897410/100000000 [00:00<00:22, 4240509.16tokens/s]
Shard 46:   3%|▎         | 3324913/100000000 [00:00<00:23, 4188656.07tokens/s]
Shard 46:   4%|▍         | 3772313/100000000 [00:01<00:22, 4266302.82tokens/s]
Shard 46:   4%|▍         | 4201821/100000000 [00:01<00:23, 4151437.46tokens/s]
Shard 46:   5%|▍         | 4620262/100000000 [00:01<00:23, 4090127.70tokens/s]
Shard 46:   5%|▌         | 5031213/100000000 [00:01<00:27, 3418899.02tokens/s]
Shard 46:   5%|▌         | 5435149/100000000 [00:01<00:26, 3566798.89tokens/s]
Shard 46:   6%|▌         | 5848124/100000000 [00:01<00:25, 3694631.84tokens/s]
Shard 46:   6%|▋         | 6294158/100000000 [00:01<00:24, 3889190.54tokens/s]
Shard 46:   7%|▋         | 6696009/100000000 [00:01<00:23, 3903107.66tokens/s]
Shard 46:   7%|▋         | 7094060/100000000 [00:01<00:25, 3598177.28tokens/s]
Shard 46:   8%|▊         | 7521022/100000000 [00:02<00:24, 3728806.54tokens/s]
Shard 46:   8%|▊         | 7922584/100000000 [00:02<00:24, 3805720.66tokens/s]
Shard 46:   8%|▊         | 8360032/100000000 [00:02<00:23, 3943368.03tokens/s]
Shard 46:   9%|▉         | 8807740/100000000 [00:02<00:22, 4091415.80tokens/s]
Shard 46:   9%|▉         | 9293959/100000000 [00:02<00:21, 4302595.39tokens/s]
Shard 46:  10%|▉         | 9807775/100000000 [00:02<00:19, 4539465.22tokens/s]
Shard 46:  10%|█         | 10264921/100000000 [00:02<00:19, 4538506.60tokens/s]
Shard 46:  11%|█         | 10720716/100000000 [00:02<00:21, 4121345.68tokens/s]
Shard 46:  11%|█▏        | 11296842/100000000 [00:02<00:19, 4541431.16tokens/s]
Shard 46:  12%|█▏        | 12032534/100000000 [00:03<00:16, 5329903.11tokens/s]
Shard 46:  13%|█▎        | 12578582/100000000 [00:03<00:18, 4607011.46tokens/s]
Shard 46:  13%|█▎        | 13063396/100000000 [00:03<00:18, 4647606.10tokens/s]
Shard 46:  14%|█▎        | 13558043/100000000 [00:03<00:19, 4497295.35tokens/s]
Shard 46:  14%|█▍        | 14091720/100000000 [00:03<00:18, 4716363.58tokens/s]
Shard 46:  15%|█▍        | 14991399/100000000 [00:03<00:14, 5882453.39tokens/s]
Shard 46:  16%|█▌        | 15596107/100000000 [00:03<00:18, 4633442.22tokens/s]
Shard 46:  16%|█▌        | 16110884/100000000 [00:03<00:19, 4409969.26tokens/s]
Shard 46:  17%|█▋        | 16587628/100000000 [00:04<00:19, 4238936.25tokens/s]
Shard 46:  17%|█▋        | 17035765/100000000 [00:04<00:22, 3750171.21tokens/s]
Shard 46:  17%|█▋        | 17449656/100000000 [00:04<00:21, 3835795.50tokens/s]
Shard 46:  18%|█▊        | 17851738/100000000 [00:04<00:21, 3766806.24tokens/s]
Shard 46:  18%|█▊        | 18240837/100000000 [00:04<00:22, 3606043.22tokens/s]
Shard 46:  19%|█▊        | 18610086/100000000 [00:04<00:23, 3485860.90tokens/s]
Shard 46:  19%|█▉        | 18997631/100000000 [00:04<00:22, 3529237.85tokens/s]
Shard 46:  19%|█▉        | 19354988/100000000 [00:04<00:24, 3261312.29tokens/s]
Shard 46:  20%|█▉        | 19821248/100000000 [00:04<00:22, 3621026.57tokens/s]
Shard 46:  20%|██        | 20204352/100000000 [00:05<00:21, 3677861.06tokens/s]
Shard 46:  21%|██        | 20578953/100000000 [00:05<00:21, 3682711.32tokens/s]
Shard 46:  21%|██▏       | 21252756/100000000 [00:05<00:17, 4544640.77tokens/s]
Shard 46:  22%|██▏       | 21772895/100000000 [00:05<00:16, 4725639.45tokens/s]
Shard 46:  22%|██▏       | 22251509/100000000 [00:05<00:19, 4051394.04tokens/s]
Shard 46:  23%|██▎       | 22677089/100000000 [00:05<00:22, 3465313.23tokens/s]
Shard 46:  23%|██▎       | 23151264/100000000 [00:05<00:20, 3765610.51tokens/s]
Shard 46:  24%|██▎       | 23553744/100000000 [00:05<00:20, 3674649.72tokens/s]
Shard 46:  24%|██▍       | 23938774/100000000 [00:06<00:20, 3711145.69tokens/s]
Shard 46:  24%|██▍       | 24333336/100000000 [00:06<00:20, 3652946.57tokens/s]
Shard 46:  25%|██▍       | 24708390/100000000 [00:06<00:20, 3618793.38tokens/s]
Shard 46:  25%|██▌       | 25077235/100000000 [00:06<00:22, 3380600.41tokens/s]
Shard 46:  25%|██▌       | 25422204/100000000 [00:06<00:22, 3331601.42tokens/s]
Shard 46:  26%|██▌       | 26010109/100000000 [00:06<00:18, 4024061.18tokens/s]
Shard 46:  26%|██▋       | 26422157/100000000 [00:06<00:20, 3559705.72tokens/s]
Shard 46:  27%|██▋       | 26793421/100000000 [00:06<00:20, 3571450.46tokens/s]
Shard 46:  27%|██▋       | 27189339/100000000 [00:06<00:20, 3571259.29tokens/s]
Shard 46:  28%|██▊       | 27671029/100000000 [00:07<00:18, 3899290.40tokens/s]
Shard 46:  28%|██▊       | 28069479/100000000 [00:07<00:19, 3732616.08tokens/s]
Shard 46:  29%|██▊       | 28500944/100000000 [00:07<00:18, 3859723.07tokens/s]
Shard 46:  29%|██▉       | 28940407/100000000 [00:07<00:17, 4003932.39tokens/s]
Shard 46:  30%|██▉       | 29581575/100000000 [00:07<00:15, 4649669.28tokens/s]
Shard 46:  30%|███       | 30053574/100000000 [00:07<00:15, 4660938.32tokens/s]
Shard 46:  31%|███       | 30523319/100000000 [00:07<00:15, 4388064.01tokens/s]
Shard 46:  31%|███       | 30970121/100000000 [00:07<00:16, 4191147.15tokens/s]
Shard 46:  31%|███▏      | 31394165/100000000 [00:07<00:17, 3946509.12tokens/s]
Shard 46:  32%|███▏      | 31794433/100000000 [00:08<00:19, 3533201.25tokens/s]
Shard 46:  32%|███▏      | 32187060/100000000 [00:08<00:18, 3621430.48tokens/s]
Shard 46:  33%|███▎      | 32561006/100000000 [00:08<00:18, 3650959.80tokens/s]
Shard 46:  33%|███▎      | 32932054/100000000 [00:08<00:18, 3607974.43tokens/s]
Shard 46:  33%|███▎      | 33297442/100000000 [00:08<00:20, 3191655.72tokens/s]
Shard 46:  34%|███▎      | 33639527/100000000 [00:08<00:21, 3062580.76tokens/s]
Shard 46:  34%|███▍      | 34091018/100000000 [00:08<00:19, 3433345.25tokens/s]
Shard 46:  34%|███▍      | 34445482/100000000 [00:08<00:20, 3171974.38tokens/s]
Shard 46:  35%|███▍      | 34772842/100000000 [00:09<00:21, 3013600.83tokens/s]
Shard 46:  35%|███▌      | 35085120/100000000 [00:09<00:22, 2913583.43tokens/s]
Shard 46:  35%|███▌      | 35384021/100000000 [00:09<00:22, 2908113.71tokens/s]
Shard 46:  36%|███▌      | 35739894/100000000 [00:09<00:20, 3084402.07tokens/s]
Shard 46:  36%|███▌      | 36082440/100000000 [00:09<00:20, 3173296.01tokens/s]
Shard 46:  36%|███▋      | 36403570/100000000 [00:09<00:20, 3120021.39tokens/s]
Shard 46:  37%|███▋      | 36720534/100000000 [00:09<00:20, 3131721.33tokens/s]
Shard 46:  37%|███▋      | 37070231/100000000 [00:09<00:19, 3236117.54tokens/s]
Shard 46:  37%|███▋      | 37453530/100000000 [00:09<00:18, 3395694.46tokens/s]
Shard 46:  38%|███▊      | 37794553/100000000 [00:09<00:18, 3371015.98tokens/s]
Shard 46:  38%|███▊      | 38133127/100000000 [00:10<00:19, 3158369.14tokens/s]
Shard 46:  39%|███▊      | 38646825/100000000 [00:10<00:16, 3715405.65tokens/s]
Shard 46:  39%|███▉      | 39024799/100000000 [00:10<00:16, 3595119.26tokens/s]
Shard 46:  39%|███▉      | 39389423/100000000 [00:10<00:17, 3419291.75tokens/s]
Shard 46:  40%|███▉      | 39738639/100000000 [00:10<00:20, 2953509.36tokens/s]
Shard 46:  40%|████      | 40047729/100000000 [00:10<00:20, 2959420.50tokens/s]
Shard 46:  40%|████      | 40417379/100000000 [00:10<00:18, 3152480.27tokens/s]
Shard 46:  41%|████      | 40851648/100000000 [00:10<00:17, 3457394.39tokens/s]
Shard 46:  41%|████      | 41224354/100000000 [00:11<00:17, 3301388.74tokens/s]
Shard 46:  42%|████▏     | 41597664/100000000 [00:11<00:17, 3392787.18tokens/s]
Shard 46:  42%|████▏     | 41943393/100000000 [00:11<00:17, 3327665.70tokens/s]
Shard 46:  42%|████▏     | 42280281/100000000 [00:11<00:17, 3316802.62tokens/s]
Shard 46:  43%|████▎     | 42665612/100000000 [00:11<00:16, 3442190.50tokens/s]
Shard 46:  43%|████▎     | 43017387/100000000 [00:11<00:16, 3402090.90tokens/s]
Shard 46:  43%|████▎     | 43359368/100000000 [00:11<00:16, 3353955.81tokens/s]
Shard 46:  44%|████▎     | 43696972/100000000 [00:11<00:17, 3184789.55tokens/s]
Shard 46:  44%|████▍     | 44093800/100000000 [00:11<00:16, 3388106.18tokens/s]
Shard 46:  44%|████▍     | 44435488/100000000 [00:11<00:16, 3299601.93tokens/s]
Shard 46:  45%|████▍     | 44771868/100000000 [00:12<00:16, 3300289.32tokens/s]
Shard 46:  45%|████▌     | 45219405/100000000 [00:12<00:15, 3587549.16tokens/s]
Shard 46:  46%|████▌     | 45640021/100000000 [00:12<00:14, 3732095.68tokens/s]
Shard 46:  46%|████▌     | 46017017/100000000 [00:12<00:14, 3742503.12tokens/s]
Shard 46:  46%|████▋     | 46394594/100000000 [00:12<00:14, 3630965.73tokens/s]
Shard 46:  47%|████▋     | 46769747/100000000 [00:12<00:14, 3564862.60tokens/s]
Shard 46:  47%|████▋     | 47189724/100000000 [00:12<00:14, 3739443.27tokens/s]
Shard 46:  48%|████▊     | 47565281/100000000 [00:12<00:14, 3669375.54tokens/s]
Shard 46:  48%|████▊     | 47933501/100000000 [00:12<00:15, 3328661.76tokens/s]
Shard 46:  48%|████▊     | 48323941/100000000 [00:13<00:14, 3478915.28tokens/s]
Shard 46:  49%|████▉     | 48841531/100000000 [00:13<00:12, 3944102.93tokens/s]
Shard 46:  49%|████▉     | 49244292/100000000 [00:13<00:14, 3547442.71tokens/s]
Shard 46:  50%|████▉     | 49610746/100000000 [00:13<00:16, 3011258.94tokens/s]
Shard 46:  50%|████▉     | 49932565/100000000 [00:13<00:16, 3006673.43tokens/s]
Shard 46:  50%|█████     | 50250504/100000000 [00:13<00:19, 2507029.03tokens/s]
Shard 46:  51%|█████     | 50523858/100000000 [00:13<00:22, 2225290.47tokens/s]
Shard 46:  51%|█████     | 50777334/100000000 [00:14<00:21, 2293930.10tokens/s]
Shard 46:  51%|█████     | 51223847/100000000 [00:14<00:17, 2809093.55tokens/s]
Shard 46:  52%|█████▏    | 51604496/100000000 [00:14<00:15, 3044833.19tokens/s]
Shard 46:  52%|█████▏    | 51952185/100000000 [00:14<00:15, 3153186.90tokens/s]
Shard 46:  52%|█████▏    | 52299421/100000000 [00:14<00:14, 3235578.89tokens/s]
Shard 46:  53%|█████▎    | 52635163/100000000 [00:14<00:15, 3143878.72tokens/s]
Shard 46:  53%|█████▎    | 52957615/100000000 [00:14<00:15, 2957744.53tokens/s]
Shard 46:  53%|█████▎    | 53260962/100000000 [00:14<00:17, 2663165.74tokens/s]
Shard 46:  54%|█████▎    | 53644745/100000000 [00:14<00:15, 2960635.96tokens/s]
Shard 46:  54%|█████▍    | 54032268/100000000 [00:15<00:14, 3206185.34tokens/s]
Shard 46:  54%|█████▍    | 54364689/100000000 [00:15<00:14, 3057601.13tokens/s]
Shard 46:  55%|█████▍    | 54679392/100000000 [00:15<00:16, 2831832.33tokens/s]
Shard 46:  55%|█████▌    | 55112094/100000000 [00:15<00:14, 3191709.57tokens/s]
Shard 46:  55%|█████▌    | 55441659/100000000 [00:15<00:14, 3178293.49tokens/s]
Shard 46:  56%|█████▌    | 55777199/100000000 [00:15<00:14, 3069012.26tokens/s]
Shard 46:  56%|█████▌    | 56089340/100000000 [00:15<00:14, 3069030.97tokens/s]
Shard 46:  56%|█████▋    | 56399910/100000000 [00:15<00:15, 2898976.86tokens/s]
Shard 46:  57%|█████▋    | 56725046/100000000 [00:15<00:14, 2985042.77tokens/s]
Shard 46:  57%|█████▋    | 57124001/100000000 [00:16<00:13, 3261366.83tokens/s]
Shard 46:  57%|█████▋    | 57454346/100000000 [00:16<00:13, 3222897.39tokens/s]
Shard 46:  58%|█████▊    | 57779610/100000000 [00:16<00:13, 3060984.54tokens/s]
Shard 46:  58%|█████▊    | 58144283/100000000 [00:16<00:12, 3221450.74tokens/s]
Shard 46:  58%|█████▊    | 58469876/100000000 [00:16<00:13, 3146534.37tokens/s]
Shard 46:  59%|█████▉    | 58822391/100000000 [00:16<00:12, 3242781.87tokens/s]
Shard 46:  59%|█████▉    | 59242444/100000000 [00:16<00:11, 3464307.63tokens/s]
Shard 46:  60%|█████▉    | 59591098/100000000 [00:16<00:11, 3425749.56tokens/s]
Shard 46:  60%|█████▉    | 59934768/100000000 [00:16<00:11, 3399795.56tokens/s]
Shard 46:  60%|██████    | 60327590/100000000 [00:16<00:11, 3541561.47tokens/s]
Shard 46:  61%|██████    | 60682630/100000000 [00:17<00:11, 3446923.68tokens/s]
Shard 46:  61%|██████    | 61028482/100000000 [00:17<00:11, 3299672.90tokens/s]
Shard 46:  61%|██████▏   | 61365556/100000000 [00:17<00:12, 3065503.19tokens/s]
Shard 46:  62%|██████▏   | 61715676/100000000 [00:17<00:12, 3179762.74tokens/s]
Shard 46:  62%|██████▏   | 62041432/100000000 [00:17<00:11, 3194424.92tokens/s]
Shard 46:  62%|██████▏   | 62364068/100000000 [00:17<00:12, 3090297.17tokens/s]
Shard 46:  63%|██████▎   | 62675736/100000000 [00:17<00:13, 2750795.52tokens/s]
Shard 46:  63%|██████▎   | 62959103/100000000 [00:17<00:14, 2593019.43tokens/s]
Shard 46:  63%|██████▎   | 63286189/100000000 [00:18<00:13, 2769455.58tokens/s]
Shard 46:  64%|██████▎   | 63574319/100000000 [00:18<00:13, 2798608.16tokens/s]
Shard 46:  64%|██████▍   | 64064309/100000000 [00:18<00:10, 3384959.25tokens/s]
Shard 46:  65%|██████▍   | 64591624/100000000 [00:18<00:09, 3915646.34tokens/s]
Shard 46:  65%|██████▍   | 64991672/100000000 [00:18<00:09, 3561987.65tokens/s]
Shard 46:  65%|██████▌   | 65360211/100000000 [00:18<00:10, 3207260.47tokens/s]
Shard 46:  66%|██████▌   | 65693983/100000000 [00:18<00:11, 3010924.90tokens/s]
Shard 46:  66%|██████▌   | 66004821/100000000 [00:18<00:11, 3002363.74tokens/s]
Shard 46:  66%|██████▋   | 66312260/100000000 [00:18<00:11, 2994147.01tokens/s]
Shard 46:  67%|██████▋   | 66848499/100000000 [00:19<00:09, 3637174.90tokens/s]
Shard 46:  67%|██████▋   | 67316211/100000000 [00:19<00:08, 3924089.10tokens/s]
Shard 46:  68%|██████▊   | 67721323/100000000 [00:19<00:08, 3910372.16tokens/s]
Shard 46:  68%|██████▊   | 68144498/100000000 [00:19<00:07, 3988075.46tokens/s]
Shard 46:  69%|██████▊   | 68590408/100000000 [00:19<00:07, 4106665.86tokens/s]
Shard 46:  69%|██████▉   | 69068714/100000000 [00:19<00:07, 4291372.80tokens/s]
Shard 46:  70%|██████▉   | 69912468/100000000 [00:19<00:05, 5489600.12tokens/s]
Shard 46:  70%|███████   | 70465105/100000000 [00:19<00:06, 4408702.11tokens/s]
Shard 46:  71%|███████   | 70942304/100000000 [00:19<00:06, 4152410.25tokens/s]
Shard 46:  71%|███████▏  | 71383529/100000000 [00:20<00:07, 3937190.83tokens/s]
Shard 46:  72%|███████▏  | 71795426/100000000 [00:20<00:07, 3644511.50tokens/s]
Shard 46:  72%|███████▏  | 72175102/100000000 [00:20<00:07, 3508632.92tokens/s]
Shard 46:  73%|███████▎  | 72535398/100000000 [00:20<00:08, 3393047.66tokens/s]
Shard 46:  73%|███████▎  | 72880208/100000000 [00:20<00:08, 3352438.24tokens/s]
Shard 46:  73%|███████▎  | 73219170/100000000 [00:20<00:08, 3309672.67tokens/s]
Shard 46:  74%|███████▎  | 73654738/100000000 [00:20<00:07, 3585482.62tokens/s]
Shard 46:  74%|███████▍  | 74070202/100000000 [00:20<00:06, 3736048.68tokens/s]
Shard 46:  75%|███████▍  | 74541974/100000000 [00:20<00:06, 4011325.93tokens/s]
Shard 46:  75%|███████▍  | 74947521/100000000 [00:21<00:06, 3948686.38tokens/s]
Shard 46:  75%|███████▌  | 75346431/100000000 [00:21<00:07, 3455676.55tokens/s]
Shard 46:  76%|███████▌  | 75767667/100000000 [00:21<00:06, 3649378.91tokens/s]
Shard 46:  76%|███████▌  | 76144155/100000000 [00:21<00:06, 3639095.32tokens/s]
Shard 46:  77%|███████▋  | 76516295/100000000 [00:21<00:06, 3405188.19tokens/s]
Shard 46:  77%|███████▋  | 76906315/100000000 [00:21<00:06, 3512185.35tokens/s]
Shard 46:  77%|███████▋  | 77264337/100000000 [00:21<00:06, 3348337.48tokens/s]
Shard 46:  78%|███████▊  | 77631838/100000000 [00:21<00:06, 3431234.77tokens/s]
Shard 46:  78%|███████▊  | 78055206/100000000 [00:21<00:06, 3632477.26tokens/s]
Shard 46:  79%|███████▊  | 78500442/100000000 [00:22<00:05, 3852409.52tokens/s]
Shard 46:  79%|███████▉  | 78889908/100000000 [00:22<00:05, 3673430.67tokens/s]
Shard 46:  79%|███████▉  | 79261548/100000000 [00:22<00:05, 3662502.59tokens/s]
Shard 46:  80%|███████▉  | 79642144/100000000 [00:22<00:05, 3701140.63tokens/s]
Shard 46:  80%|████████  | 80014677/100000000 [00:22<00:05, 3447254.63tokens/s]
Shard 46:  80%|████████  | 80396752/100000000 [00:22<00:05, 3550932.78tokens/s]
Shard 46:  81%|████████  | 80801119/100000000 [00:22<00:05, 3683991.12tokens/s]
Shard 46:  81%|████████  | 81189209/100000000 [00:22<00:05, 3727497.96tokens/s]
Shard 46:  82%|████████▏ | 81644038/100000000 [00:22<00:04, 3952544.15tokens/s]
Shard 46:  82%|████████▏ | 82041675/100000000 [00:23<00:05, 3578641.23tokens/s]
Shard 46:  82%|████████▏ | 82408118/100000000 [00:23<00:04, 3559592.27tokens/s]
Shard 46:  83%|████████▎ | 82769958/100000000 [00:23<00:04, 3505758.71tokens/s]
Shard 46:  83%|████████▎ | 83134101/100000000 [00:23<00:05, 3153078.12tokens/s]
Shard 46:  84%|████████▎ | 83635310/100000000 [00:23<00:04, 3630796.49tokens/s]
Shard 46:  84%|████████▍ | 84010712/100000000 [00:23<00:04, 3223849.55tokens/s]
Shard 46:  84%|████████▍ | 84349625/100000000 [00:23<00:04, 3210904.63tokens/s]
Shard 46:  85%|████████▍ | 84681281/100000000 [00:23<00:04, 3166687.74tokens/s]
Shard 46:  85%|████████▌ | 85005881/100000000 [00:24<00:04, 3128479.63tokens/s]
Shard 46:  85%|████████▌ | 85324194/100000000 [00:24<00:05, 2934302.81tokens/s]
Shard 46:  86%|████████▌ | 85681993/100000000 [00:24<00:04, 3091587.47tokens/s]
Shard 46:  86%|████████▌ | 86050739/100000000 [00:24<00:04, 3214289.05tokens/s]
Shard 46:  86%|████████▋ | 86376703/100000000 [00:24<00:04, 3023169.75tokens/s]
Shard 46:  87%|████████▋ | 86684931/100000000 [00:24<00:04, 2926604.08tokens/s]
Shard 46:  87%|████████▋ | 87076370/100000000 [00:24<00:04, 3183543.82tokens/s]
Shard 46:  87%|████████▋ | 87402335/100000000 [00:24<00:04, 3058116.49tokens/s]
Shard 46:  88%|████████▊ | 87719033/100000000 [00:24<00:03, 3087684.41tokens/s]
Shard 46:  88%|████████▊ | 88083339/100000000 [00:24<00:03, 3217266.35tokens/s]
Shard 46:  88%|████████▊ | 88408132/100000000 [00:25<00:03, 3186530.46tokens/s]
Shard 46:  89%|████████▊ | 88735336/100000000 [00:25<00:03, 3192523.08tokens/s]
Shard 46:  89%|████████▉ | 89141277/100000000 [00:25<00:03, 3439140.63tokens/s]
Shard 46:  90%|████████▉ | 89576795/100000000 [00:25<00:02, 3704348.74tokens/s]
Shard 46:  90%|█████████ | 90088624/100000000 [00:25<00:02, 4120719.45tokens/s]
Shard 46:  91%|█████████ | 90502713/100000000 [00:25<00:02, 4099193.88tokens/s]
Shard 46:  91%|█████████ | 90938368/100000000 [00:25<00:02, 4163629.06tokens/s]
Shard 46:  92%|█████████▏| 91736728/100000000 [00:25<00:01, 5262058.27tokens/s]
Shard 46:  92%|█████████▏| 92268845/100000000 [00:26<00:01, 4097623.24tokens/s]
Shard 46:  93%|█████████▎| 92718300/100000000 [00:26<00:02, 3579724.35tokens/s]
Shard 46:  93%|█████████▎| 93112659/100000000 [00:26<00:01, 3524575.12tokens/s]
Shard 46:  94%|█████████▎| 93500679/100000000 [00:26<00:01, 3597379.19tokens/s]
Shard 46:  94%|█████████▍| 93879776/100000000 [00:26<00:01, 3516841.23tokens/s]
Shard 46:  94%|█████████▍| 94245551/100000000 [00:26<00:01, 3548764.49tokens/s]
Shard 46:  95%|█████████▍| 94641156/100000000 [00:26<00:01, 3638959.50tokens/s]
Shard 46:  95%|█████████▌| 95045975/100000000 [00:26<00:01, 3748853.19tokens/s]
Shard 46:  95%|█████████▌| 95490866/100000000 [00:26<00:01, 3855788.56tokens/s]
Shard 46:  96%|█████████▌| 96022670/100000000 [00:27<00:00, 4267441.28tokens/s]
Shard 46:  96%|█████████▋| 96455084/100000000 [00:27<00:00, 3832390.55tokens/s]
Shard 46:  97%|█████████▋| 96850449/100000000 [00:27<00:00, 3823590.56tokens/s]
Shard 46:  97%|█████████▋| 97240879/100000000 [00:27<00:00, 3291914.51tokens/s]
Shard 46:  98%|█████████▊| 97664639/100000000 [00:27<00:00, 3526695.05tokens/s]
Shard 46:  98%|█████████▊| 98040842/100000000 [00:27<00:00, 3119012.03tokens/s]
Shard 46:  98%|█████████▊| 98372848/100000000 [00:27<00:00, 3106642.90tokens/s]
Shard 46:  99%|█████████▉| 98902935/100000000 [00:27<00:00, 3652456.09tokens/s]
Shard 46:  99%|█████████▉| 99285596/100000000 [00:27<00:00, 3641159.37tokens/s]
Shard 46: 100%|█████████▉| 99721805/100000000 [00:28<00:00, 3796054.86tokens/s]
Shard 46: 100%|█████████▉| 99999353/100000000 [00:36<00:00, 2743458.58tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000046.bin (200,001,024 bytes) in the gpt-2 format

Shard 47:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 47:   0%|          | 165330/100000000 [00:00<01:00, 1649012.25tokens/s]
Shard 47:   0%|          | 361023/100000000 [00:00<00:54, 1823084.84tokens/s]
Shard 47:   1%|          | 639687/100000000 [00:00<00:43, 2260201.06tokens/s]
Shard 47:   1%|          | 866162/100000000 [00:00<00:43, 2253934.02tokens/s]
Shard 47:   1%|▏         | 1321360/100000000 [00:00<00:32, 3079373.46tokens/s]
Shard 47:   2%|▏         | 1648620/100000000 [00:00<00:31, 3135542.99tokens/s]
Shard 47:   2%|▏         | 1962363/100000000 [00:00<00:31, 3066253.91tokens/s]
Shard 47:   2%|▏         | 2321039/100000000 [00:00<00:30, 3193222.29tokens/s]
Shard 47:   3%|▎         | 2641376/100000000 [00:00<00:30, 3158307.47tokens/s]
Shard 47:   3%|▎         | 3202238/100000000 [00:01<00:24, 3897941.20tokens/s]
Shard 47:   4%|▍         | 3775469/100000000 [00:01<00:21, 4408570.85tokens/s]
Shard 47:   4%|▍         | 4217681/100000000 [00:01<00:29, 3278073.34tokens/s]
Shard 47:   5%|▍         | 4588712/100000000 [00:01<00:30, 3175229.44tokens/s]
Shard 47:   5%|▍         | 4935061/100000000 [00:01<00:30, 3164047.34tokens/s]
Shard 47:   5%|▌         | 5271358/100000000 [00:01<00:30, 3076533.73tokens/s]
Shard 47:   6%|▌         | 5661149/100000000 [00:01<00:28, 3276586.39tokens/s]
Shard 47:   6%|▌         | 6171634/100000000 [00:01<00:24, 3763780.93tokens/s]
Shard 47:   7%|▋         | 6562676/100000000 [00:01<00:24, 3799031.90tokens/s]
Shard 47:   7%|▋         | 7092533/100000000 [00:02<00:22, 4213319.03tokens/s]
Shard 47:   8%|▊         | 7523430/100000000 [00:02<00:22, 4118107.56tokens/s]
Shard 47:   8%|▊         | 8015625/100000000 [00:02<00:21, 4339156.75tokens/s]
Shard 47:   8%|▊         | 8456764/100000000 [00:02<00:21, 4356742.95tokens/s]
Shard 47:   9%|▉         | 8897272/100000000 [00:02<00:20, 4350690.48tokens/s]
Shard 47:   9%|▉         | 9335915/100000000 [00:02<00:21, 4214917.68tokens/s]
Shard 47:  10%|▉         | 9760245/100000000 [00:02<00:22, 3950541.60tokens/s]
Shard 47:  10%|█         | 10164015/100000000 [00:02<00:23, 3751188.49tokens/s]
Shard 47:  11%|█         | 10586083/100000000 [00:02<00:23, 3860567.38tokens/s]
Shard 47:  11%|█         | 11046761/100000000 [00:03<00:21, 4062467.98tokens/s]
Shard 47:  12%|█▏        | 11530183/100000000 [00:03<00:20, 4265778.96tokens/s]
Shard 47:  12%|█▏        | 11960890/100000000 [00:03<00:21, 4089446.08tokens/s]
Shard 47:  12%|█▏        | 12440816/100000000 [00:03<00:20, 4270991.69tokens/s]
Shard 47:  13%|█▎        | 12871541/100000000 [00:03<00:22, 3885123.56tokens/s]
Shard 47:  13%|█▎        | 13303880/100000000 [00:03<00:23, 3696516.07tokens/s]
Shard 47:  14%|█▎        | 13680883/100000000 [00:03<00:26, 3236755.34tokens/s]
Shard 47:  14%|█▍        | 14017375/100000000 [00:03<00:26, 3236895.73tokens/s]
Shard 47:  15%|█▍        | 14562287/100000000 [00:04<00:22, 3798818.15tokens/s]
Shard 47:  15%|█▍        | 14958352/100000000 [00:04<00:23, 3620564.20tokens/s]
Shard 47:  15%|█▌        | 15331620/100000000 [00:04<00:27, 3040474.93tokens/s]
Shard 47:  16%|█▌        | 15668788/100000000 [00:04<00:30, 2751352.10tokens/s]
Shard 47:  16%|█▌        | 15962357/100000000 [00:04<00:30, 2751245.22tokens/s]
Shard 47:  16%|█▌        | 16249657/100000000 [00:04<00:33, 2523531.68tokens/s]
Shard 47:  17%|█▋        | 16593670/100000000 [00:04<00:30, 2736626.50tokens/s]
Shard 47:  17%|█▋        | 17103113/100000000 [00:04<00:24, 3332703.50tokens/s]
Shard 47:  18%|█▊        | 17544299/100000000 [00:05<00:22, 3618656.47tokens/s]
Shard 47:  18%|█▊        | 17924476/100000000 [00:05<00:23, 3467136.89tokens/s]
Shard 47:  18%|█▊        | 18305868/100000000 [00:05<00:23, 3551536.90tokens/s]
Shard 47:  19%|█▊        | 18676402/100000000 [00:05<00:23, 3410321.47tokens/s]
Shard 47:  19%|█▉        | 19025175/100000000 [00:05<00:24, 3261792.16tokens/s]
Shard 47:  19%|█▉        | 19436681/100000000 [00:05<00:23, 3489669.77tokens/s]
Shard 47:  20%|█▉        | 19792613/100000000 [00:05<00:23, 3426449.92tokens/s]
Shard 47:  20%|██        | 20323606/100000000 [00:05<00:20, 3941673.62tokens/s]
Shard 47:  21%|██        | 20779818/100000000 [00:05<00:19, 4109594.72tokens/s]
Shard 47:  21%|██        | 21200309/100000000 [00:05<00:19, 4101976.41tokens/s]
Shard 47:  22%|██▏       | 21621092/100000000 [00:06<00:19, 4103186.63tokens/s]
Shard 47:  22%|██▏       | 22037866/100000000 [00:06<00:21, 3678978.04tokens/s]
Shard 47:  22%|██▏       | 22415984/100000000 [00:06<00:21, 3690005.61tokens/s]
Shard 47:  23%|██▎       | 22830838/100000000 [00:06<00:20, 3792884.21tokens/s]
Shard 47:  23%|██▎       | 23277312/100000000 [00:06<00:19, 3974289.64tokens/s]
Shard 47:  24%|██▎       | 23680919/100000000 [00:06<00:23, 3232124.14tokens/s]
Shard 47:  25%|██▍       | 24500732/100000000 [00:06<00:16, 4451304.79tokens/s]
Shard 47:  25%|██▍       | 24986059/100000000 [00:06<00:19, 3831330.17tokens/s]
Shard 47:  25%|██▌       | 25410349/100000000 [00:07<00:21, 3413165.19tokens/s]
Shard 47:  26%|██▌       | 25983674/100000000 [00:07<00:18, 3934674.32tokens/s]
Shard 47:  26%|██▋       | 26416821/100000000 [00:07<00:19, 3710401.42tokens/s]
Shard 47:  27%|██▋       | 26816777/100000000 [00:07<00:21, 3353491.05tokens/s]
Shard 47:  27%|██▋       | 27175184/100000000 [00:07<00:22, 3244654.83tokens/s]
Shard 47:  28%|██▊       | 27553812/100000000 [00:07<00:21, 3369210.61tokens/s]
Shard 47:  28%|██▊       | 27992371/100000000 [00:07<00:19, 3615104.62tokens/s]
Shard 47:  29%|██▊       | 28513561/100000000 [00:07<00:17, 4029634.31tokens/s]
Shard 47:  29%|██▉       | 28930639/100000000 [00:08<00:19, 3615198.21tokens/s]
Shard 47:  29%|██▉       | 29308299/100000000 [00:08<00:20, 3366779.53tokens/s]
Shard 47:  30%|██▉       | 29658527/100000000 [00:08<00:21, 3198380.53tokens/s]
Shard 47:  30%|██▉       | 29987723/100000000 [00:08<00:23, 3040689.75tokens/s]
Shard 47:  30%|███       | 30297917/100000000 [00:08<00:24, 2853039.27tokens/s]
Shard 47:  31%|███▏      | 31386779/100000000 [00:08<00:14, 4896100.68tokens/s]
Shard 47:  32%|███▏      | 31916514/100000000 [00:08<00:15, 4357737.05tokens/s]
Shard 47:  33%|███▎      | 32768056/100000000 [00:08<00:12, 5359491.50tokens/s]
Shard 47:  33%|███▎      | 33352641/100000000 [00:09<00:14, 4721006.28tokens/s]
Shard 47:  34%|███▍      | 33866971/100000000 [00:09<00:16, 3956128.67tokens/s]
Shard 47:  34%|███▍      | 34312606/100000000 [00:09<00:16, 4060054.17tokens/s]
Shard 47:  35%|███▍      | 34754257/100000000 [00:09<00:18, 3555905.89tokens/s]
Shard 47:  35%|███▌      | 35227537/100000000 [00:09<00:16, 3817251.47tokens/s]
Shard 47:  36%|███▌      | 35644847/100000000 [00:09<00:17, 3752791.68tokens/s]
Shard 47:  36%|███▌      | 36044440/100000000 [00:09<00:17, 3744965.18tokens/s]
Shard 47:  36%|███▋      | 36433342/100000000 [00:10<00:17, 3619999.80tokens/s]
Shard 47:  37%|███▋      | 36816313/100000000 [00:10<00:17, 3648496.46tokens/s]
Shard 47:  37%|███▋      | 37392782/100000000 [00:10<00:14, 4178296.89tokens/s]
Shard 47:  38%|███▊      | 37825848/100000000 [00:10<00:14, 4216687.60tokens/s]
Shard 47:  38%|███▊      | 38254651/100000000 [00:10<00:15, 4045189.84tokens/s]
Shard 47:  39%|███▊      | 38664826/100000000 [00:10<00:16, 3611676.82tokens/s]
Shard 47:  39%|███▉      | 39055811/100000000 [00:10<00:16, 3679343.18tokens/s]
Shard 47:  39%|███▉      | 39470618/100000000 [00:10<00:16, 3771187.46tokens/s]
Shard 47:  40%|███▉      | 39990319/100000000 [00:10<00:14, 4147741.29tokens/s]
Shard 47:  40%|████      | 40465118/100000000 [00:11<00:13, 4290981.98tokens/s]
Shard 47:  41%|████      | 40935629/100000000 [00:11<00:13, 4246288.97tokens/s]
Shard 47:  41%|████▏     | 41365347/100000000 [00:11<00:13, 4230075.71tokens/s]
Shard 47:  42%|████▏     | 41802112/100000000 [00:11<00:14, 4082857.52tokens/s]
Shard 47:  42%|████▏     | 42213377/100000000 [00:11<00:15, 3818560.09tokens/s]
Shard 47:  43%|████▎     | 42599714/100000000 [00:11<00:15, 3735400.23tokens/s]
Shard 47:  43%|████▎     | 42976812/100000000 [00:11<00:15, 3618284.93tokens/s]
Shard 47:  43%|████▎     | 43351510/100000000 [00:11<00:16, 3504842.81tokens/s]
Shard 47:  44%|████▎     | 43703566/100000000 [00:11<00:16, 3494694.76tokens/s]
Shard 47:  44%|████▍     | 44054570/100000000 [00:12<00:17, 3165292.10tokens/s]
Shard 47:  45%|████▍     | 44541969/100000000 [00:12<00:15, 3611248.72tokens/s]
Shard 47:  45%|████▌     | 45082965/100000000 [00:12<00:13, 4095581.15tokens/s]
Shard 47:  46%|████▌     | 45530590/100000000 [00:12<00:13, 4162203.96tokens/s]
Shard 47:  46%|████▌     | 45954792/100000000 [00:12<00:13, 4082865.33tokens/s]
Shard 47:  47%|████▋     | 46688252/100000000 [00:12<00:10, 4978644.95tokens/s]
Shard 47:  47%|████▋     | 47193596/100000000 [00:12<00:11, 4749862.91tokens/s]
Shard 47:  48%|████▊     | 47675746/100000000 [00:12<00:13, 3919138.50tokens/s]
Shard 47:  48%|████▊     | 48094836/100000000 [00:12<00:13, 3818896.39tokens/s]
Shard 47:  48%|████▊     | 48495603/100000000 [00:13<00:14, 3573350.07tokens/s]
Shard 47:  49%|████▉     | 48866712/100000000 [00:13<00:16, 3146470.11tokens/s]
Shard 47:  49%|████▉     | 49325280/100000000 [00:13<00:14, 3472989.82tokens/s]
Shard 47:  50%|████▉     | 49691821/100000000 [00:13<00:15, 3342572.54tokens/s]
Shard 47:  50%|█████     | 50136467/100000000 [00:13<00:13, 3609013.77tokens/s]
Shard 47:  51%|█████     | 50510693/100000000 [00:13<00:15, 3226569.72tokens/s]
Shard 47:  51%|█████     | 50848108/100000000 [00:13<00:15, 3227970.88tokens/s]
Shard 47:  51%|█████▏    | 51383012/100000000 [00:13<00:12, 3779320.46tokens/s]
Shard 47:  52%|█████▏    | 51792907/100000000 [00:14<00:12, 3864497.14tokens/s]
Shard 47:  52%|█████▏    | 52190726/100000000 [00:14<00:14, 3315390.87tokens/s]
Shard 47:  53%|█████▎    | 52570148/100000000 [00:14<00:13, 3433279.47tokens/s]
Shard 47:  53%|█████▎    | 52929906/100000000 [00:14<00:14, 3227789.87tokens/s]
Shard 47:  53%|█████▎    | 53447874/100000000 [00:14<00:12, 3728093.54tokens/s]
Shard 47:  54%|█████▍    | 53872915/100000000 [00:14<00:12, 3832688.21tokens/s]
Shard 47:  54%|█████▍    | 54268978/100000000 [00:14<00:12, 3690310.45tokens/s]
Shard 47:  55%|█████▍    | 54841294/100000000 [00:14<00:10, 4233523.13tokens/s]
Shard 47:  55%|█████▌    | 55276176/100000000 [00:14<00:10, 4232328.98tokens/s]
Shard 47:  56%|█████▌    | 55707120/100000000 [00:15<00:11, 3920971.40tokens/s]
Shard 47:  56%|█████▌    | 56206734/100000000 [00:15<00:10, 4187341.07tokens/s]
Shard 47:  57%|█████▋    | 56640313/100000000 [00:15<00:10, 4222273.01tokens/s]
Shard 47:  57%|█████▋    | 57091472/100000000 [00:15<00:09, 4295552.26tokens/s]
Shard 47:  58%|█████▊    | 57525955/100000000 [00:15<00:10, 4092544.91tokens/s]
Shard 47:  58%|█████▊    | 57940156/100000000 [00:15<00:11, 3794018.17tokens/s]
Shard 47:  58%|█████▊    | 58326273/100000000 [00:15<00:11, 3726535.96tokens/s]
Shard 47:  59%|█████▉    | 58781008/100000000 [00:15<00:10, 3940144.34tokens/s]
Shard 47:  59%|█████▉    | 59190717/100000000 [00:15<00:10, 3983938.99tokens/s]
Shard 47:  60%|█████▉    | 59650058/100000000 [00:16<00:09, 4112765.59tokens/s]
Shard 47:  60%|██████    | 60098032/100000000 [00:16<00:09, 4118420.67tokens/s]
Shard 47:  61%|██████    | 60538094/100000000 [00:16<00:09, 4184375.81tokens/s]
Shard 47:  61%|██████    | 60987814/100000000 [00:16<00:09, 4272780.19tokens/s]
Shard 47:  61%|██████▏   | 61417279/100000000 [00:16<00:10, 3835124.18tokens/s]
Shard 47:  62%|██████▏   | 61809749/100000000 [00:16<00:10, 3774555.78tokens/s]
Shard 47:  62%|██████▏   | 62193748/100000000 [00:16<00:10, 3692342.12tokens/s]
Shard 47:  63%|██████▎   | 62568170/100000000 [00:16<00:10, 3692258.13tokens/s]
Shard 47:  63%|██████▎   | 63048674/100000000 [00:16<00:09, 3997139.37tokens/s]
Shard 47:  64%|██████▎   | 63506367/100000000 [00:17<00:08, 4135503.65tokens/s]
Shard 47:  64%|██████▍   | 63923517/100000000 [00:17<00:09, 3939105.36tokens/s]
Shard 47:  64%|██████▍   | 64479157/100000000 [00:17<00:08, 4395438.25tokens/s]
Shard 47:  65%|██████▍   | 64957778/100000000 [00:17<00:07, 4497104.72tokens/s]
Shard 47:  65%|██████▌   | 65412149/100000000 [00:17<00:08, 4281537.57tokens/s]
Shard 47:  66%|██████▌   | 65845363/100000000 [00:17<00:08, 4042404.62tokens/s]
Shard 47:  66%|██████▋   | 66254913/100000000 [00:17<00:08, 3974240.60tokens/s]
Shard 47:  67%|██████▋   | 66864924/100000000 [00:17<00:07, 4556857.87tokens/s]
Shard 47:  67%|██████▋   | 67332873/100000000 [00:17<00:07, 4315895.80tokens/s]
Shard 47:  68%|██████▊   | 67829495/100000000 [00:18<00:07, 4308506.52tokens/s]
Shard 47:  68%|██████▊   | 68266358/100000000 [00:18<00:08, 3872201.05tokens/s]
Shard 47:  69%|██████▉   | 68950425/100000000 [00:18<00:06, 4642148.14tokens/s]
Shard 47:  69%|██████▉   | 69433298/100000000 [00:18<00:06, 4436387.00tokens/s]
Shard 47:  70%|██████▉   | 69959794/100000000 [00:18<00:06, 4626016.03tokens/s]
Shard 47:  70%|███████   | 70433892/100000000 [00:18<00:06, 4501513.90tokens/s]
Shard 47:  71%|███████   | 70972277/100000000 [00:18<00:06, 4742966.11tokens/s]
Shard 47:  71%|███████▏  | 71454712/100000000 [00:18<00:06, 4472819.15tokens/s]
Shard 47:  72%|███████▏  | 71909662/100000000 [00:18<00:06, 4450650.78tokens/s]
Shard 47:  72%|███████▏  | 72395879/100000000 [00:19<00:06, 4541600.99tokens/s]
Shard 47:  73%|███████▎  | 72862427/100000000 [00:19<00:06, 4451321.28tokens/s]
Shard 47:  73%|███████▎  | 73310800/100000000 [00:19<00:06, 4410741.88tokens/s]
Shard 47:  74%|███████▍  | 73803165/100000000 [00:19<00:05, 4546196.66tokens/s]
Shard 47:  74%|███████▍  | 74260723/100000000 [00:19<00:05, 4423016.52tokens/s]
Shard 47:  75%|███████▍  | 74771060/100000000 [00:19<00:05, 4606887.32tokens/s]
Shard 47:  75%|███████▌  | 75248465/100000000 [00:19<00:05, 4552652.65tokens/s]
Shard 47:  76%|███████▌  | 75705890/100000000 [00:19<00:05, 4436085.48tokens/s]
Shard 47:  76%|███████▋  | 76255815/100000000 [00:19<00:05, 4740044.74tokens/s]
Shard 47:  77%|███████▋  | 76733586/100000000 [00:20<00:05, 4272614.23tokens/s]
Shard 47:  77%|███████▋  | 77170847/100000000 [00:20<00:05, 4264050.14tokens/s]
Shard 47:  78%|███████▊  | 77627528/100000000 [00:20<00:05, 4337674.53tokens/s]
Shard 47:  78%|███████▊  | 78217122/100000000 [00:20<00:04, 4759731.98tokens/s]
Shard 47:  79%|███████▊  | 78698948/100000000 [00:20<00:04, 4646061.68tokens/s]
Shard 47:  79%|███████▉  | 79167863/100000000 [00:20<00:04, 4282521.09tokens/s]
Shard 47:  80%|███████▉  | 79603774/100000000 [00:20<00:04, 4192551.13tokens/s]
Shard 47:  80%|████████  | 80202127/100000000 [00:20<00:04, 4684440.58tokens/s]
Shard 47:  81%|████████  | 80766054/100000000 [00:20<00:03, 4953514.82tokens/s]
Shard 47:  81%|████████▏ | 81269330/100000000 [00:21<00:04, 4663302.33tokens/s]
Shard 47:  82%|████████▏ | 81833702/100000000 [00:21<00:03, 4930686.27tokens/s]
Shard 47:  82%|████████▏ | 82334514/100000000 [00:21<00:04, 4219583.64tokens/s]
Shard 47:  83%|████████▎ | 82780408/100000000 [00:21<00:04, 3890075.19tokens/s]
Shard 47:  83%|████████▎ | 83187287/100000000 [00:21<00:05, 3322218.53tokens/s]
Shard 47:  84%|████████▎ | 83546375/100000000 [00:21<00:05, 3011675.89tokens/s]
Shard 47:  84%|████████▍ | 83865818/100000000 [00:21<00:05, 3031493.16tokens/s]
Shard 47:  84%|████████▍ | 84413798/100000000 [00:21<00:04, 3626171.18tokens/s]
Shard 47:  85%|████████▍ | 84798923/100000000 [00:22<00:04, 3343726.71tokens/s]
Shard 47:  85%|████████▌ | 85151837/100000000 [00:22<00:04, 3341474.08tokens/s]
Shard 47:  85%|████████▌ | 85498699/100000000 [00:22<00:04, 3341914.97tokens/s]
Shard 47:  86%|████████▌ | 85891501/100000000 [00:22<00:04, 3476404.54tokens/s]
Shard 47:  86%|████████▋ | 86318007/100000000 [00:22<00:03, 3685615.91tokens/s]
Shard 47:  87%|████████▋ | 86739801/100000000 [00:22<00:03, 3836072.19tokens/s]
Shard 47:  87%|████████▋ | 87129488/100000000 [00:22<00:03, 3312032.34tokens/s]
Shard 47:  88%|████████▊ | 87627648/100000000 [00:22<00:03, 3744742.65tokens/s]
Shard 47:  88%|████████▊ | 88137674/100000000 [00:22<00:02, 4061048.34tokens/s]
Shard 47:  89%|████████▊ | 88559506/100000000 [00:23<00:02, 3826394.90tokens/s]
Shard 47:  89%|████████▉ | 88977148/100000000 [00:23<00:02, 3896811.10tokens/s]
Shard 47:  90%|████████▉ | 89502251/100000000 [00:23<00:02, 4257911.26tokens/s]
Shard 47:  90%|████████▉ | 89937320/100000000 [00:23<00:02, 4104626.13tokens/s]
Shard 47:  90%|█████████ | 90355196/100000000 [00:23<00:02, 3861900.62tokens/s]
Shard 47:  91%|█████████ | 90832406/100000000 [00:23<00:02, 4105565.52tokens/s]
Shard 47:  91%|█████████▏| 91344586/100000000 [00:23<00:01, 4385060.90tokens/s]
Shard 47:  92%|█████████▏| 91851646/100000000 [00:23<00:01, 4580383.22tokens/s]
Shard 47:  92%|█████████▏| 92342354/100000000 [00:23<00:01, 4668392.28tokens/s]
Shard 47:  93%|█████████▎| 92813656/100000000 [00:24<00:01, 4506751.42tokens/s]
Shard 47:  93%|█████████▎| 93433120/100000000 [00:24<00:01, 4956578.86tokens/s]
Shard 47:  94%|█████████▍| 93933885/100000000 [00:24<00:01, 4694309.39tokens/s]
Shard 47:  94%|█████████▍| 94452981/100000000 [00:24<00:01, 4829679.00tokens/s]
Shard 47:  95%|█████████▍| 94942430/100000000 [00:24<00:01, 4367004.56tokens/s]
Shard 47:  95%|█████████▌| 95389715/100000000 [00:24<00:01, 3968830.52tokens/s]
Shard 47:  96%|█████████▌| 95798783/100000000 [00:24<00:01, 3678874.63tokens/s]
Shard 47:  96%|█████████▋| 96277541/100000000 [00:24<00:00, 3932792.44tokens/s]
Shard 47:  97%|█████████▋| 96884619/100000000 [00:24<00:00, 4487036.38tokens/s]
Shard 47:  97%|█████████▋| 97366232/100000000 [00:25<00:00, 4537333.31tokens/s]
Shard 47:  98%|█████████▊| 97903866/100000000 [00:25<00:00, 4769067.28tokens/s]
Shard 47:  99%|█████████▉| 98867609/100000000 [00:25<00:00, 6151245.98tokens/s]
Shard 47:  99%|█████████▉| 99496465/100000000 [00:25<00:00, 5343213.91tokens/s]
Shard 47: 100%|█████████▉| 99999629/100000000 [00:33<00:00, 2988262.34tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000047.bin (200,001,024 bytes) in the gpt-2 format

Shard 48:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 48:   0%|          | 218157/100000000 [00:00<00:46, 2165356.30tokens/s]
Shard 48:   0%|          | 436395/100000000 [00:00<00:47, 2113553.14tokens/s]
Shard 48:   1%|          | 705065/100000000 [00:00<00:42, 2363776.92tokens/s]
Shard 48:   1%|          | 942864/100000000 [00:00<00:47, 2098579.38tokens/s]
Shard 48:   1%|          | 1157697/100000000 [00:00<00:47, 2068636.25tokens/s]
Shard 48:   1%|▏         | 1374860/100000000 [00:00<00:47, 2097590.92tokens/s]
Shard 48:   3%|▎         | 2905545/100000000 [00:00<00:15, 6198629.35tokens/s]
Shard 48:   4%|▎         | 3548919/100000000 [00:00<00:17, 5437806.57tokens/s]
Shard 48:   4%|▍         | 4124154/100000000 [00:00<00:17, 5505495.73tokens/s]
Shard 48:   5%|▌         | 5210613/100000000 [00:01<00:13, 7001879.25tokens/s]
Shard 48:   6%|▌         | 5940192/100000000 [00:01<00:16, 5753034.68tokens/s]
Shard 48:   7%|▋         | 6569824/100000000 [00:01<00:16, 5690108.24tokens/s]
Shard 48:   7%|▋         | 7176265/100000000 [00:01<00:21, 4336832.26tokens/s]
Shard 48:   8%|▊         | 7678391/100000000 [00:01<00:23, 3997101.30tokens/s]
Shard 48:   8%|▊         | 8125786/100000000 [00:01<00:24, 3793149.21tokens/s]
Shard 48:   9%|▊         | 8536765/100000000 [00:02<00:24, 3749430.34tokens/s]
Shard 48:   9%|▉         | 8932693/100000000 [00:02<00:25, 3607593.06tokens/s]
Shard 48:   9%|▉         | 9306704/100000000 [00:02<00:25, 3582301.76tokens/s]
Shard 48:  10%|▉         | 9674538/100000000 [00:02<00:25, 3586049.03tokens/s]
Shard 48:  10%|█         | 10119610/100000000 [00:02<00:24, 3718912.77tokens/s]
Shard 48:  10%|█         | 10497080/100000000 [00:02<00:27, 3309085.11tokens/s]
Shard 48:  11%|█         | 10924894/100000000 [00:02<00:25, 3548415.90tokens/s]
Shard 48:  11%|█▏        | 11295856/100000000 [00:02<00:25, 3432317.36tokens/s]
Shard 48:  12%|█▏        | 11791454/100000000 [00:02<00:23, 3823288.35tokens/s]
Shard 48:  12%|█▏        | 12183558/100000000 [00:03<00:24, 3598513.47tokens/s]
Shard 48:  13%|█▎        | 12586494/100000000 [00:03<00:23, 3713912.75tokens/s]
Shard 48:  13%|█▎        | 13014276/100000000 [00:03<00:22, 3851673.15tokens/s]
Shard 48:  13%|█▎        | 13436919/100000000 [00:03<00:21, 3953836.72tokens/s]
Shard 48:  15%|█▍        | 14562332/100000000 [00:03<00:14, 6045818.24tokens/s]
Shard 48:  15%|█▌        | 15179545/100000000 [00:03<00:15, 5523059.33tokens/s]
Shard 48:  16%|█▌        | 15760180/100000000 [00:03<00:16, 5008880.84tokens/s]
Shard 48:  16%|█▋        | 16280219/100000000 [00:03<00:17, 4697669.81tokens/s]
Shard 48:  17%|█▋        | 16767261/100000000 [00:04<00:20, 4109917.43tokens/s]
Shard 48:  17%|█▋        | 17197697/100000000 [00:04<00:21, 3869786.26tokens/s]
Shard 48:  18%|█▊        | 17597628/100000000 [00:04<00:24, 3367195.91tokens/s]
Shard 48:  18%|█▊        | 18474043/100000000 [00:04<00:17, 4596611.26tokens/s]
Shard 48:  19%|█▉        | 18981023/100000000 [00:04<00:20, 3926516.66tokens/s]
Shard 48:  19%|█▉        | 19419555/100000000 [00:04<00:22, 3554660.59tokens/s]
Shard 48:  20%|█▉        | 19844920/100000000 [00:04<00:22, 3595864.15tokens/s]
Shard 48:  20%|██        | 20309704/100000000 [00:04<00:20, 3836759.80tokens/s]
Shard 48:  21%|██        | 20734341/100000000 [00:05<00:20, 3929845.63tokens/s]
Shard 48:  21%|██        | 21147206/100000000 [00:05<00:21, 3729371.35tokens/s]
Shard 48:  22%|██▏       | 21534002/100000000 [00:05<00:21, 3725573.99tokens/s]
Shard 48:  22%|██▏       | 21921303/100000000 [00:05<00:20, 3762527.96tokens/s]
Shard 48:  22%|██▏       | 22304706/100000000 [00:05<00:21, 3567890.69tokens/s]
Shard 48:  23%|██▎       | 22731911/100000000 [00:05<00:20, 3756849.23tokens/s]
Shard 48:  23%|██▎       | 23220538/100000000 [00:05<00:18, 4062717.19tokens/s]
Shard 48:  24%|██▎       | 23674915/100000000 [00:05<00:18, 4196562.12tokens/s]
Shard 48:  24%|██▍       | 24185078/100000000 [00:05<00:17, 4411460.81tokens/s]
Shard 48:  25%|██▍       | 24632615/100000000 [00:06<00:18, 4128770.90tokens/s]
Shard 48:  25%|██▌       | 25051558/100000000 [00:06<00:18, 3995427.45tokens/s]
Shard 48:  25%|██▌       | 25455717/100000000 [00:06<00:20, 3692734.14tokens/s]
Shard 48:  26%|██▌       | 25996680/100000000 [00:06<00:17, 4138961.39tokens/s]
Shard 48:  26%|██▋       | 26470221/100000000 [00:06<00:17, 4292692.41tokens/s]
Shard 48:  27%|██▋       | 27014021/100000000 [00:06<00:15, 4615130.71tokens/s]
Shard 48:  28%|██▊       | 27916503/100000000 [00:06<00:12, 5811027.15tokens/s]
Shard 48:  29%|██▊       | 28504859/100000000 [00:06<00:15, 4557782.13tokens/s]
Shard 48:  29%|██▉       | 29007093/100000000 [00:07<00:17, 3965982.31tokens/s]
Shard 48:  29%|██▉       | 29446677/100000000 [00:07<00:17, 3948819.62tokens/s]
Shard 48:  30%|██▉       | 29870689/100000000 [00:07<00:20, 3364045.35tokens/s]
Shard 48:  30%|███       | 30237822/100000000 [00:07<00:20, 3355510.73tokens/s]
Shard 48:  31%|███       | 30621218/100000000 [00:07<00:20, 3458104.98tokens/s]
Shard 48:  31%|███       | 31015872/100000000 [00:07<00:19, 3574679.53tokens/s]
Shard 48:  31%|███▏      | 31388261/100000000 [00:07<00:19, 3570212.16tokens/s]
Shard 48:  32%|███▏      | 31755859/100000000 [00:07<00:19, 3416501.83tokens/s]
Shard 48:  32%|███▏      | 32110812/100000000 [00:08<00:19, 3449100.80tokens/s]
Shard 48:  32%|███▏      | 32475879/100000000 [00:08<00:19, 3486536.59tokens/s]
Shard 48:  33%|███▎      | 32828970/100000000 [00:08<00:19, 3365475.91tokens/s]
Shard 48:  33%|███▎      | 33181578/100000000 [00:08<00:19, 3409520.65tokens/s]
Shard 48:  34%|███▎      | 33557483/100000000 [00:08<00:18, 3497335.13tokens/s]
Shard 48:  34%|███▍      | 33910919/100000000 [00:08<00:21, 3103263.34tokens/s]
Shard 48:  34%|███▍      | 34275023/100000000 [00:08<00:20, 3228246.43tokens/s]
Shard 48:  35%|███▍      | 34650991/100000000 [00:08<00:19, 3369647.54tokens/s]
Shard 48:  35%|███▍      | 34994681/100000000 [00:08<00:20, 3125516.43tokens/s]
Shard 48:  35%|███▌      | 35314956/100000000 [00:09<00:22, 2819127.29tokens/s]
Shard 48:  36%|███▌      | 35608050/100000000 [00:09<00:22, 2823743.29tokens/s]
Shard 48:  36%|███▌      | 35897145/100000000 [00:09<00:26, 2449071.91tokens/s]
Shard 48:  36%|███▌      | 36165743/100000000 [00:09<00:25, 2476937.54tokens/s]
Shard 48:  36%|███▋      | 36484451/100000000 [00:09<00:23, 2649094.16tokens/s]
Shard 48:  37%|███▋      | 36758223/100000000 [00:09<00:26, 2367389.54tokens/s]
Shard 48:  37%|███▋      | 37005142/100000000 [00:09<00:27, 2325569.93tokens/s]
Shard 48:  37%|███▋      | 37285955/100000000 [00:09<00:25, 2437217.12tokens/s]
Shard 48:  38%|███▊      | 37535972/100000000 [00:10<00:30, 2053154.22tokens/s]
Shard 48:  38%|███▊      | 37754836/100000000 [00:10<00:32, 1936705.44tokens/s]
Shard 48:  38%|███▊      | 38114119/100000000 [00:10<00:26, 2339472.12tokens/s]
Shard 48:  38%|███▊      | 38418652/100000000 [00:10<00:24, 2501698.00tokens/s]
Shard 48:  39%|███▉      | 38827667/100000000 [00:10<00:20, 2921132.53tokens/s]
Shard 48:  39%|███▉      | 39168513/100000000 [00:10<00:19, 3055901.08tokens/s]
Shard 48:  40%|███▉      | 39576584/100000000 [00:10<00:18, 3345126.01tokens/s]
Shard 48:  40%|███▉      | 39920307/100000000 [00:10<00:20, 2962446.18tokens/s]
Shard 48:  40%|████      | 40233048/100000000 [00:10<00:21, 2802000.37tokens/s]
Shard 48:  41%|████      | 40562406/100000000 [00:11<00:20, 2845460.80tokens/s]
Shard 48:  41%|████      | 40855170/100000000 [00:11<00:22, 2654282.26tokens/s]
Shard 48:  41%|████      | 41127313/100000000 [00:11<00:22, 2561212.70tokens/s]
Shard 48:  41%|████▏     | 41474117/100000000 [00:11<00:20, 2797700.52tokens/s]
Shard 48:  42%|████▏     | 41763215/100000000 [00:11<00:20, 2822423.39tokens/s]
Shard 48:  42%|████▏     | 42095094/100000000 [00:11<00:19, 2961484.11tokens/s]
Shard 48:  42%|████▏     | 42395649/100000000 [00:11<00:20, 2774899.44tokens/s]
Shard 48:  43%|████▎     | 42678119/100000000 [00:11<00:21, 2656699.94tokens/s]
Shard 48:  43%|████▎     | 43048360/100000000 [00:11<00:19, 2932901.56tokens/s]
Shard 48:  43%|████▎     | 43347392/100000000 [00:12<00:20, 2757531.04tokens/s]
Shard 48:  44%|████▎     | 43628628/100000000 [00:12<00:21, 2633827.10tokens/s]
Shard 48:  44%|████▍     | 43921029/100000000 [00:12<00:20, 2708748.52tokens/s]
Shard 48:  44%|████▍     | 44324793/100000000 [00:12<00:18, 3071393.29tokens/s]
Shard 48:  45%|████▍     | 44710589/100000000 [00:12<00:16, 3282202.82tokens/s]
Shard 48:  45%|████▌     | 45043576/100000000 [00:12<00:17, 3107224.81tokens/s]
Shard 48:  45%|████▌     | 45360019/100000000 [00:12<00:17, 3117054.94tokens/s]
Shard 48:  46%|████▌     | 45695452/100000000 [00:12<00:17, 3180215.28tokens/s]
Shard 48:  46%|████▌     | 46091248/100000000 [00:12<00:15, 3400105.79tokens/s]
Shard 48:  46%|████▋     | 46433974/100000000 [00:13<00:15, 3386705.39tokens/s]
Shard 48:  47%|████▋     | 46856053/100000000 [00:13<00:14, 3614092.16tokens/s]
Shard 48:  47%|████▋     | 47219462/100000000 [00:13<00:14, 3591358.84tokens/s]
Shard 48:  48%|████▊     | 47580733/100000000 [00:13<00:14, 3534314.07tokens/s]
Shard 48:  48%|████▊     | 47936788/100000000 [00:13<00:14, 3540111.80tokens/s]
Shard 48:  49%|████▊     | 48664713/100000000 [00:13<00:11, 4632583.62tokens/s]
Shard 48:  49%|████▉     | 49130033/100000000 [00:13<00:11, 4243325.38tokens/s]
Shard 48:  50%|████▉     | 49562635/100000000 [00:13<00:12, 3919909.64tokens/s]
Shard 48:  50%|████▉     | 49963526/100000000 [00:13<00:14, 3345716.92tokens/s]
Shard 48:  50%|█████     | 50430855/100000000 [00:14<00:13, 3624335.78tokens/s]
Shard 48:  51%|█████     | 51150418/100000000 [00:14<00:10, 4506084.07tokens/s]
Shard 48:  52%|█████▏    | 51650026/100000000 [00:14<00:10, 4459178.74tokens/s]
Shard 48:  52%|█████▏    | 52114555/100000000 [00:14<00:12, 3977063.79tokens/s]
Shard 48:  53%|█████▎    | 52531884/100000000 [00:14<00:12, 3815206.21tokens/s]
Shard 48:  53%|█████▎    | 53016778/100000000 [00:14<00:11, 4066649.52tokens/s]
Shard 48:  53%|█████▎    | 53437105/100000000 [00:14<00:11, 4094649.25tokens/s]
Shard 48:  54%|█████▍    | 53860159/100000000 [00:14<00:12, 3811602.90tokens/s]
Shard 48:  54%|█████▍    | 54251381/100000000 [00:15<00:12, 3592148.78tokens/s]
Shard 48:  55%|█████▍    | 54618402/100000000 [00:15<00:13, 3435898.69tokens/s]
Shard 48:  55%|█████▍    | 54968708/100000000 [00:15<00:13, 3336095.36tokens/s]
Shard 48:  55%|█████▌    | 55305936/100000000 [00:15<00:13, 3196381.63tokens/s]
Shard 48:  56%|█████▌    | 55627969/100000000 [00:15<00:13, 3200100.63tokens/s]
Shard 48:  56%|█████▌    | 55979468/100000000 [00:15<00:13, 3286138.80tokens/s]
Shard 48:  56%|█████▋    | 56359039/100000000 [00:15<00:12, 3390020.47tokens/s]
Shard 48:  57%|█████▋    | 56718785/100000000 [00:15<00:12, 3437788.62tokens/s]
Shard 48:  58%|█████▊    | 57877346/100000000 [00:15<00:07, 5786711.94tokens/s]
Shard 48:  58%|█████▊    | 58465930/100000000 [00:16<00:08, 4942097.78tokens/s]
Shard 48:  59%|█████▉    | 58998137/100000000 [00:16<00:09, 4478160.75tokens/s]
Shard 48:  59%|█████▉    | 59471633/100000000 [00:16<00:09, 4378947.34tokens/s]
Shard 48:  60%|█████▉    | 59926894/100000000 [00:16<00:10, 3865154.47tokens/s]
Shard 48:  60%|██████    | 60336162/100000000 [00:16<00:10, 3617450.13tokens/s]
Shard 48:  61%|██████    | 60711387/100000000 [00:16<00:11, 3569418.64tokens/s]
Shard 48:  61%|██████    | 61184839/100000000 [00:16<00:10, 3848557.97tokens/s]
Shard 48:  62%|██████▏   | 61689140/100000000 [00:16<00:09, 4156394.28tokens/s]
Shard 48:  62%|██████▏   | 62130520/100000000 [00:17<00:09, 4186639.24tokens/s]
Shard 48:  63%|██████▎   | 62588406/100000000 [00:17<00:08, 4272817.71tokens/s]
Shard 48:  63%|██████▎   | 63022572/100000000 [00:17<00:10, 3677982.99tokens/s]
Shard 48:  63%|██████▎   | 63496732/100000000 [00:17<00:09, 3898212.60tokens/s]
Shard 48:  64%|██████▍   | 63901845/100000000 [00:17<00:09, 3655385.51tokens/s]
Shard 48:  64%|██████▍   | 64323834/100000000 [00:17<00:09, 3798425.23tokens/s]
Shard 48:  65%|██████▍   | 64714261/100000000 [00:17<00:09, 3721983.46tokens/s]
Shard 48:  65%|██████▌   | 65145639/100000000 [00:17<00:08, 3877973.76tokens/s]
Shard 48:  66%|██████▌   | 65901040/100000000 [00:17<00:07, 4870802.61tokens/s]
Shard 48:  66%|██████▋   | 66400485/100000000 [00:18<00:07, 4280971.29tokens/s]
Shard 48:  67%|██████▋   | 66849399/100000000 [00:18<00:08, 3756916.67tokens/s]
Shard 48:  67%|██████▋   | 67272391/100000000 [00:18<00:08, 3827415.42tokens/s]
Shard 48:  68%|██████▊   | 67673115/100000000 [00:18<00:08, 3815920.88tokens/s]
Shard 48:  68%|██████▊   | 68067022/100000000 [00:18<00:09, 3537648.72tokens/s]
Shard 48:  68%|██████▊   | 68447213/100000000 [00:18<00:08, 3605457.26tokens/s]
Shard 48:  69%|██████▉   | 69028212/100000000 [00:18<00:07, 4179516.18tokens/s]
Shard 48:  69%|██████▉   | 69458721/100000000 [00:18<00:07, 4030032.84tokens/s]
Shard 48:  70%|███████   | 70135956/100000000 [00:19<00:06, 4763671.44tokens/s]
Shard 48:  71%|███████   | 70755488/100000000 [00:19<00:05, 5163019.83tokens/s]
Shard 48:  71%|███████▏  | 71304156/100000000 [00:19<00:05, 5244106.98tokens/s]
Shard 48:  72%|███████▏  | 71836853/100000000 [00:19<00:05, 4907025.20tokens/s]
Shard 48:  72%|███████▏  | 72337363/100000000 [00:19<00:05, 4920810.71tokens/s]
Shard 48:  73%|███████▎  | 72836263/100000000 [00:19<00:05, 4769924.53tokens/s]
Shard 48:  73%|███████▎  | 73351358/100000000 [00:19<00:05, 4832300.25tokens/s]
Shard 48:  74%|███████▍  | 73839029/100000000 [00:19<00:05, 4551259.78tokens/s]
Shard 48:  74%|███████▍  | 74307878/100000000 [00:19<00:05, 4572161.63tokens/s]
Shard 48:  75%|███████▍  | 74817918/100000000 [00:19<00:05, 4702835.71tokens/s]
Shard 48:  75%|███████▌  | 75292048/100000000 [00:20<00:05, 4561393.75tokens/s]
Shard 48:  76%|███████▌  | 75759432/100000000 [00:20<00:05, 4563218.52tokens/s]
Shard 48:  76%|███████▋  | 76293058/100000000 [00:20<00:04, 4752381.31tokens/s]
Shard 48:  77%|███████▋  | 76794712/100000000 [00:20<00:04, 4821656.42tokens/s]
Shard 48:  77%|███████▋  | 77368855/100000000 [00:20<00:04, 5053606.51tokens/s]
Shard 48:  78%|███████▊  | 77954351/100000000 [00:20<00:04, 5287554.30tokens/s]
Shard 48:  78%|███████▊  | 78485038/100000000 [00:20<00:04, 5107918.36tokens/s]
Shard 48:  79%|███████▉  | 78998216/100000000 [00:20<00:04, 4204629.60tokens/s]
Shard 48:  79%|███████▉  | 79466562/100000000 [00:20<00:04, 4319263.84tokens/s]
Shard 48:  80%|███████▉  | 79976274/100000000 [00:21<00:04, 4522596.95tokens/s]
Shard 48:  81%|████████  | 80542284/100000000 [00:21<00:04, 4815084.57tokens/s]
Shard 48:  81%|████████  | 81038963/100000000 [00:21<00:04, 4646951.15tokens/s]
Shard 48:  82%|████████▏ | 81559539/100000000 [00:21<00:03, 4794720.28tokens/s]
Shard 48:  82%|████████▏ | 82048163/100000000 [00:21<00:03, 4528233.56tokens/s]
Shard 48:  83%|████████▎ | 82608643/100000000 [00:21<00:03, 4810586.37tokens/s]
Shard 48:  83%|████████▎ | 83097981/100000000 [00:21<00:03, 4522143.45tokens/s]
Shard 48:  84%|████████▎ | 83559578/100000000 [00:21<00:03, 4515276.25tokens/s]
Shard 48:  84%|████████▍ | 84166210/100000000 [00:21<00:03, 4946075.26tokens/s]
Shard 48:  85%|████████▌ | 85053518/100000000 [00:22<00:02, 6061515.55tokens/s]
Shard 48:  86%|████████▌ | 85670389/100000000 [00:22<00:02, 5884906.10tokens/s]
Shard 48:  86%|████████▋ | 86336072/100000000 [00:22<00:02, 6075418.20tokens/s]
Shard 48:  87%|████████▋ | 86984176/100000000 [00:22<00:02, 6189355.01tokens/s]
Shard 48:  88%|████████▊ | 87610745/100000000 [00:22<00:02, 5999987.82tokens/s]
Shard 48:  88%|████████▊ | 88216271/100000000 [00:22<00:02, 4972070.98tokens/s]
Shard 48:  89%|████████▊ | 88744938/100000000 [00:22<00:02, 4838454.34tokens/s]
Shard 48:  89%|████████▉ | 89311517/100000000 [00:22<00:02, 5021028.73tokens/s]
Shard 48:  90%|█████████ | 90068437/100000000 [00:22<00:01, 5671202.51tokens/s]
Shard 48:  91%|█████████ | 90665438/100000000 [00:23<00:01, 5264114.55tokens/s]
Shard 48:  91%|█████████ | 91241504/100000000 [00:23<00:01, 5388722.32tokens/s]
Shard 48:  92%|█████████▏| 92042962/100000000 [00:23<00:01, 6105639.46tokens/s]
Shard 48:  93%|█████████▎| 92683614/100000000 [00:23<00:01, 6156211.45tokens/s]
Shard 48:  93%|█████████▎| 93311413/100000000 [00:23<00:01, 5831055.53tokens/s]
Shard 48:  94%|█████████▍| 93905508/100000000 [00:23<00:01, 5252726.67tokens/s]
Shard 48:  94%|█████████▍| 94452884/100000000 [00:23<00:01, 5292614.33tokens/s]
Shard 48:  95%|█████████▌| 95174644/100000000 [00:23<00:00, 5798476.91tokens/s]
Shard 48:  96%|█████████▌| 95767770/100000000 [00:24<00:00, 5537050.92tokens/s]
Shard 48:  96%|█████████▋| 96332185/100000000 [00:24<00:00, 5143029.63tokens/s]
Shard 48:  97%|█████████▋| 96869998/100000000 [00:24<00:00, 4672623.15tokens/s]
Shard 48:  97%|█████████▋| 97350634/100000000 [00:24<00:00, 4589430.58tokens/s]
Shard 48:  98%|█████████▊| 97818129/100000000 [00:24<00:00, 4443073.53tokens/s]
Shard 48:  98%|█████████▊| 98269248/100000000 [00:24<00:00, 3765085.41tokens/s]
Shard 48:  99%|█████████▊| 98704349/100000000 [00:24<00:00, 3904504.84tokens/s]
Shard 48:  99%|█████████▉| 99428454/100000000 [00:24<00:00, 4722127.59tokens/s]
Shard 48: 100%|█████████▉| 99927338/100000000 [00:25<00:00, 4279510.48tokens/s]
Shard 48: 100%|█████████▉| 99999938/100000000 [00:32<00:00, 3057853.91tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000048.bin (200,001,024 bytes) in the gpt-2 format

Shard 49:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 49:   0%|          | 264421/100000000 [00:00<00:38, 2585979.73tokens/s]
Shard 49:   1%|          | 523087/100000000 [00:00<00:41, 2416842.60tokens/s]
Shard 49:   1%|          | 774500/100000000 [00:00<00:40, 2437764.07tokens/s]
Shard 49:   1%|          | 1032410/100000000 [00:00<00:39, 2478230.22tokens/s]
Shard 49:   1%|▏         | 1381286/100000000 [00:00<00:35, 2810661.29tokens/s]
Shard 49:   2%|▏         | 1669250/100000000 [00:00<00:34, 2815190.61tokens/s]
Shard 49:   2%|▏         | 2026671/100000000 [00:00<00:32, 3051420.83tokens/s]
Shard 49:   2%|▏         | 2332520/100000000 [00:00<00:34, 2818843.82tokens/s]
Shard 49:   3%|▎         | 2729814/100000000 [00:00<00:30, 3154170.82tokens/s]
Shard 49:   3%|▎         | 3099471/100000000 [00:01<00:29, 3309991.79tokens/s]
Shard 49:   4%|▎         | 3504465/100000000 [00:01<00:27, 3528893.04tokens/s]
Shard 49:   4%|▍         | 3897733/100000000 [00:01<00:26, 3642097.49tokens/s]
Shard 49:   4%|▍         | 4357895/100000000 [00:01<00:24, 3915964.78tokens/s]
Shard 49:   5%|▍         | 4835838/100000000 [00:01<00:22, 4171636.37tokens/s]
Shard 49:   5%|▌         | 5259932/100000000 [00:01<00:25, 3659679.25tokens/s]
Shard 49:   6%|▌         | 5764775/100000000 [00:01<00:23, 4022562.86tokens/s]
Shard 49:   6%|▋         | 6292033/100000000 [00:01<00:21, 4310822.55tokens/s]
Shard 49:   7%|▋         | 6733356/100000000 [00:01<00:22, 4083384.37tokens/s]
Shard 49:   7%|▋         | 7150398/100000000 [00:02<00:23, 3927866.90tokens/s]
Shard 49:   8%|▊         | 7689537/100000000 [00:02<00:21, 4316086.84tokens/s]
Shard 49:   8%|▊         | 8130589/100000000 [00:02<00:22, 4173700.75tokens/s]
Shard 49:   9%|▊         | 8651363/100000000 [00:02<00:20, 4457419.10tokens/s]
Shard 49:   9%|▉         | 9105933/100000000 [00:02<00:20, 4363697.93tokens/s]
Shard 49:  10%|▉         | 9547551/100000000 [00:02<00:22, 4107704.91tokens/s]
Shard 49:  10%|█         | 10058551/100000000 [00:02<00:20, 4370010.34tokens/s]
Shard 49:  11%|█         | 10501698/100000000 [00:02<00:21, 4169756.52tokens/s]
Shard 49:  11%|█▏        | 11474144/100000000 [00:02<00:15, 5691503.39tokens/s]
Shard 49:  12%|█▏        | 12132401/100000000 [00:03<00:14, 5937249.22tokens/s]
Shard 49:  13%|█▎        | 12739109/100000000 [00:03<00:16, 5296883.80tokens/s]
Shard 49:  13%|█▎        | 13289748/100000000 [00:03<00:21, 4017785.99tokens/s]
Shard 49:  14%|█▍        | 13753066/100000000 [00:03<00:23, 3749768.73tokens/s]
Shard 49:  14%|█▍        | 14167850/100000000 [00:03<00:25, 3406373.18tokens/s]
Shard 49:  15%|█▍        | 14546043/100000000 [00:03<00:24, 3460645.03tokens/s]
Shard 49:  15%|█▍        | 14917266/100000000 [00:03<00:24, 3513927.46tokens/s]
Shard 49:  15%|█▌        | 15285653/100000000 [00:04<00:24, 3495927.17tokens/s]
Shard 49:  16%|█▌        | 15668562/100000000 [00:04<00:23, 3583319.65tokens/s]
Shard 49:  16%|█▌        | 16171082/100000000 [00:04<00:21, 3971596.10tokens/s]
Shard 49:  17%|█▋        | 16578605/100000000 [00:04<00:21, 3913466.48tokens/s]
Shard 49:  17%|█▋        | 17080055/100000000 [00:04<00:19, 4216751.10tokens/s]
Shard 49:  18%|█▊        | 17557048/100000000 [00:04<00:18, 4346134.32tokens/s]
Shard 49:  18%|█▊        | 17997086/100000000 [00:04<00:20, 3993410.22tokens/s]
Shard 49:  18%|█▊        | 18405118/100000000 [00:04<00:20, 3909046.95tokens/s]
Shard 49:  19%|█▉        | 18802103/100000000 [00:04<00:21, 3840045.83tokens/s]
Shard 49:  19%|█▉        | 19242633/100000000 [00:04<00:20, 3974944.01tokens/s]
Shard 49:  20%|█▉        | 19762254/100000000 [00:05<00:18, 4313902.50tokens/s]
Shard 49:  20%|██        | 20198359/100000000 [00:05<00:21, 3732267.86tokens/s]
Shard 49:  21%|██        | 20587727/100000000 [00:05<00:21, 3665588.64tokens/s]
Shard 49:  21%|██        | 21028223/100000000 [00:05<00:20, 3839711.17tokens/s]
Shard 49:  21%|██▏       | 21493331/100000000 [00:05<00:19, 4062372.83tokens/s]
Shard 49:  22%|██▏       | 21933917/100000000 [00:05<00:18, 4158930.62tokens/s]
Shard 49:  22%|██▏       | 22368014/100000000 [00:05<00:19, 3929456.51tokens/s]
Shard 49:  23%|██▎       | 22769292/100000000 [00:05<00:20, 3815358.47tokens/s]
Shard 49:  23%|██▎       | 23252215/100000000 [00:05<00:18, 4088704.33tokens/s]
Shard 49:  24%|██▍       | 23821778/100000000 [00:06<00:16, 4533130.08tokens/s]
Shard 49:  24%|██▍       | 24371065/100000000 [00:06<00:15, 4800329.28tokens/s]
Shard 49:  25%|██▍       | 24856839/100000000 [00:06<00:16, 4515099.15tokens/s]
Shard 49:  25%|██▌       | 25367294/100000000 [00:06<00:16, 4652078.74tokens/s]
Shard 49:  26%|██▌       | 25970894/100000000 [00:06<00:14, 5042895.68tokens/s]
Shard 49:  26%|██▋       | 26481446/100000000 [00:06<00:16, 4458528.02tokens/s]
Shard 49:  27%|██▋       | 26943411/100000000 [00:06<00:16, 4348313.72tokens/s]
Shard 49:  27%|██▋       | 27388950/100000000 [00:06<00:19, 3796733.38tokens/s]
Shard 49:  28%|██▊       | 28042041/100000000 [00:07<00:16, 4468747.69tokens/s]
Shard 49:  29%|██▊       | 28519446/100000000 [00:07<00:15, 4541725.54tokens/s]
Shard 49:  29%|██▉       | 28992377/100000000 [00:07<00:15, 4460929.86tokens/s]
Shard 49:  29%|██▉       | 29451494/100000000 [00:07<00:17, 3961395.55tokens/s]
Shard 49:  30%|██▉       | 29875983/100000000 [00:07<00:17, 4023860.74tokens/s]
Shard 49:  31%|███       | 30638527/100000000 [00:07<00:13, 4986445.46tokens/s]
Shard 49:  31%|███       | 31223472/100000000 [00:07<00:13, 5224710.87tokens/s]
Shard 49:  32%|███▏      | 31781727/100000000 [00:07<00:12, 5325151.12tokens/s]
Shard 49:  32%|███▏      | 32349402/100000000 [00:07<00:12, 5426080.31tokens/s]
Shard 49:  33%|███▎      | 32900956/100000000 [00:07<00:12, 5297277.51tokens/s]
Shard 49:  34%|███▍      | 33853711/100000000 [00:08<00:10, 6496185.55tokens/s]
Shard 49:  35%|███▍      | 34513363/100000000 [00:08<00:10, 6212891.75tokens/s]
Shard 49:  35%|███▌      | 35143632/100000000 [00:08<00:10, 6019102.16tokens/s]
Shard 49:  36%|███▌      | 35754125/100000000 [00:08<00:11, 5725287.90tokens/s]
Shard 49:  36%|███▋      | 36469494/100000000 [00:08<00:10, 6086873.93tokens/s]
Shard 49:  37%|███▋      | 37085222/100000000 [00:08<00:10, 5826826.34tokens/s]
Shard 49:  38%|███▊      | 37677765/100000000 [00:08<00:11, 5358265.26tokens/s]
Shard 49:  38%|███▊      | 38262664/100000000 [00:08<00:11, 5484616.41tokens/s]
Shard 49:  39%|███▉      | 38963041/100000000 [00:08<00:10, 5871673.18tokens/s]
Shard 49:  40%|███▉      | 39607762/100000000 [00:09<00:10, 6018673.52tokens/s]
Shard 49:  40%|████      | 40217087/100000000 [00:09<00:10, 5603659.35tokens/s]
Shard 49:  41%|████      | 40787451/100000000 [00:09<00:11, 5256767.48tokens/s]
Shard 49:  41%|████▏     | 41323726/100000000 [00:09<00:11, 5129045.72tokens/s]
Shard 49:  42%|████▏     | 41845784/100000000 [00:09<00:11, 5126172.72tokens/s]
Shard 49:  42%|████▏     | 42363593/100000000 [00:09<00:12, 4772131.19tokens/s]
Shard 49:  43%|████▎     | 42847128/100000000 [00:09<00:12, 4663674.35tokens/s]
Shard 49:  43%|████▎     | 43317558/100000000 [00:09<00:12, 4519027.40tokens/s]
Shard 49:  44%|████▍     | 43776626/100000000 [00:10<00:12, 4372858.97tokens/s]
Shard 49:  44%|████▍     | 44215977/100000000 [00:10<00:12, 4377511.96tokens/s]
Shard 49:  45%|████▍     | 44796608/100000000 [00:10<00:11, 4758024.68tokens/s]
Shard 49:  45%|████▌     | 45276483/100000000 [00:10<00:11, 4762970.72tokens/s]
Shard 49:  46%|████▌     | 46085550/100000000 [00:10<00:09, 5719857.33tokens/s]
Shard 49:  47%|████▋     | 46662626/100000000 [00:10<00:10, 4865745.03tokens/s]
Shard 49:  47%|████▋     | 47174433/100000000 [00:10<00:11, 4779352.41tokens/s]
Shard 49:  48%|████▊     | 47671340/100000000 [00:10<00:11, 4540039.80tokens/s]
Shard 49:  48%|████▊     | 48259771/100000000 [00:10<00:10, 4827418.57tokens/s]
Shard 49:  49%|████▉     | 48754551/100000000 [00:11<00:11, 4330422.91tokens/s]
Shard 49:  49%|████▉     | 49254137/100000000 [00:11<00:11, 4500714.88tokens/s]
Shard 49:  50%|████▉     | 49717837/100000000 [00:11<00:11, 4510587.61tokens/s]
Shard 49:  50%|█████     | 50179466/100000000 [00:11<00:11, 4309707.27tokens/s]
Shard 49:  51%|█████     | 50762551/100000000 [00:11<00:10, 4710413.57tokens/s]
Shard 49:  51%|█████     | 51242652/100000000 [00:11<00:10, 4573840.28tokens/s]
Shard 49:  52%|█████▏    | 51827152/100000000 [00:11<00:09, 4922575.85tokens/s]
Shard 49:  52%|█████▏    | 52327533/100000000 [00:11<00:10, 4717251.88tokens/s]
Shard 49:  53%|█████▎    | 52822390/100000000 [00:11<00:10, 4443710.35tokens/s]
Shard 49:  53%|█████▎    | 53425185/100000000 [00:12<00:09, 4813548.57tokens/s]
Shard 49:  54%|█████▍    | 53914107/100000000 [00:12<00:10, 4210899.71tokens/s]
Shard 49:  54%|█████▍    | 54352226/100000000 [00:12<00:11, 3823739.93tokens/s]
Shard 49:  55%|█████▍    | 54750503/100000000 [00:12<00:12, 3720762.25tokens/s]
Shard 49:  55%|█████▌    | 55139760/100000000 [00:12<00:12, 3705038.08tokens/s]
Shard 49:  56%|█████▌    | 55518058/100000000 [00:12<00:12, 3549934.23tokens/s]
Shard 49:  56%|█████▌    | 55877814/100000000 [00:12<00:12, 3467966.46tokens/s]
Shard 49:  56%|█████▌    | 56228458/100000000 [00:12<00:14, 3122851.01tokens/s]
Shard 49:  57%|█████▋    | 56558999/100000000 [00:13<00:13, 3166442.64tokens/s]
Shard 49:  57%|█████▋    | 56918323/100000000 [00:13<00:13, 3242740.77tokens/s]
Shard 49:  57%|█████▋    | 57365713/100000000 [00:13<00:11, 3571956.13tokens/s]
Shard 49:  58%|█████▊    | 57729030/100000000 [00:13<00:11, 3542497.63tokens/s]
Shard 49:  58%|█████▊    | 58149630/100000000 [00:13<00:11, 3728705.47tokens/s]
Shard 49:  59%|█████▊    | 58663648/100000000 [00:13<00:10, 4122049.74tokens/s]
Shard 49:  59%|█████▉    | 59173300/100000000 [00:13<00:09, 4391072.58tokens/s]
Shard 49:  60%|█████▉    | 59616316/100000000 [00:13<00:09, 4380670.56tokens/s]
Shard 49:  60%|██████    | 60281348/100000000 [00:13<00:07, 5038623.85tokens/s]
Shard 49:  61%|██████    | 60991610/100000000 [00:13<00:06, 5643344.02tokens/s]
Shard 49:  62%|██████▏   | 61559896/100000000 [00:14<00:07, 5180438.48tokens/s]
Shard 49:  62%|██████▏   | 62088387/100000000 [00:14<00:07, 5094526.24tokens/s]
Shard 49:  63%|██████▎   | 62605050/100000000 [00:14<00:07, 4871664.65tokens/s]
Shard 49:  63%|██████▎   | 63099773/100000000 [00:14<00:07, 4859976.08tokens/s]
Shard 49:  64%|██████▎   | 63589761/100000000 [00:14<00:07, 4797887.60tokens/s]
Shard 49:  64%|██████▍   | 64072010/100000000 [00:14<00:08, 4451055.76tokens/s]
Shard 49:  65%|██████▍   | 64553969/100000000 [00:14<00:07, 4536126.27tokens/s]
Shard 49:  65%|██████▌   | 65183245/100000000 [00:14<00:06, 5018482.62tokens/s]
Shard 49:  66%|██████▌   | 65822072/100000000 [00:14<00:06, 5376399.19tokens/s]
Shard 49:  66%|██████▋   | 66366732/100000000 [00:15<00:07, 4713812.46tokens/s]
Shard 49:  68%|██████▊   | 67904403/100000000 [00:15<00:04, 7509516.16tokens/s]
Shard 49:  69%|██████▊   | 68703281/100000000 [00:15<00:04, 6601382.29tokens/s]
Shard 49:  69%|██████▉   | 69414766/100000000 [00:15<00:05, 5608453.65tokens/s]
Shard 49:  70%|███████   | 70031385/100000000 [00:15<00:05, 5319633.85tokens/s]
Shard 49:  71%|███████   | 70601489/100000000 [00:15<00:05, 5395410.20tokens/s]
Shard 49:  71%|███████   | 71169595/100000000 [00:15<00:05, 4856456.79tokens/s]
Shard 49:  72%|███████▏  | 71680957/100000000 [00:16<00:06, 4574938.78tokens/s]
Shard 49:  72%|███████▏  | 72188833/100000000 [00:16<00:05, 4685272.58tokens/s]
Shard 49:  73%|███████▎  | 72672652/100000000 [00:16<00:05, 4590109.69tokens/s]
Shard 49:  73%|███████▎  | 73141874/100000000 [00:16<00:06, 4228192.55tokens/s]
Shard 49:  74%|███████▎  | 73592922/100000000 [00:16<00:06, 4292884.78tokens/s]
Shard 49:  74%|███████▍  | 74031618/100000000 [00:16<00:06, 4251477.55tokens/s]
Shard 49:  74%|███████▍  | 74494321/100000000 [00:16<00:05, 4340114.76tokens/s]
Shard 49:  75%|███████▍  | 74953851/100000000 [00:16<00:05, 4396087.33tokens/s]
Shard 49:  76%|███████▌  | 75574332/100000000 [00:16<00:05, 4822333.19tokens/s]
Shard 49:  76%|███████▌  | 76062854/100000000 [00:17<00:05, 4602262.63tokens/s]
Shard 49:  77%|███████▋  | 76621556/100000000 [00:17<00:04, 4874008.28tokens/s]
Shard 49:  77%|███████▋  | 77114937/100000000 [00:17<00:04, 4732427.35tokens/s]
Shard 49:  78%|███████▊  | 77591860/100000000 [00:17<00:04, 4499802.54tokens/s]
Shard 49:  78%|███████▊  | 78065960/100000000 [00:17<00:04, 4562795.12tokens/s]
Shard 49:  79%|███████▉  | 78846453/100000000 [00:17<00:03, 5440226.75tokens/s]
Shard 49:  79%|███████▉  | 79396162/100000000 [00:17<00:04, 4903941.64tokens/s]
Shard 49:  80%|███████▉  | 79926757/100000000 [00:17<00:04, 4934971.60tokens/s]
Shard 49:  81%|████████  | 80558927/100000000 [00:17<00:03, 5302877.40tokens/s]
Shard 49:  81%|████████  | 81224948/100000000 [00:18<00:03, 5642052.52tokens/s]
Shard 49:  82%|████████▏ | 81801768/100000000 [00:18<00:03, 5657136.45tokens/s]
Shard 49:  82%|████████▏ | 82374649/100000000 [00:18<00:03, 5633868.73tokens/s]
Shard 49:  83%|████████▎ | 82942127/100000000 [00:18<00:03, 5185177.73tokens/s]
Shard 49:  83%|████████▎ | 83499811/100000000 [00:18<00:03, 5206626.32tokens/s]
Shard 49:  84%|████████▍ | 84027542/100000000 [00:18<00:03, 4872772.50tokens/s]
Shard 49:  85%|████████▍ | 84522979/100000000 [00:18<00:03, 4147800.80tokens/s]
Shard 49:  85%|████████▍ | 84959366/100000000 [00:18<00:04, 3744066.25tokens/s]
Shard 49:  85%|████████▌ | 85352442/100000000 [00:19<00:04, 3622256.60tokens/s]
Shard 49:  86%|████████▌ | 85726395/100000000 [00:19<00:03, 3591315.95tokens/s]
Shard 49:  86%|████████▌ | 86093082/100000000 [00:19<00:04, 3375765.03tokens/s]
Shard 49:  86%|████████▋ | 86437413/100000000 [00:19<00:04, 3009537.44tokens/s]
Shard 49:  87%|████████▋ | 86763900/100000000 [00:19<00:04, 3067707.75tokens/s]
Shard 49:  87%|████████▋ | 87077832/100000000 [00:19<00:04, 2940373.70tokens/s]
Shard 49:  88%|████████▊ | 87534017/100000000 [00:19<00:03, 3362295.86tokens/s]
Shard 49:  88%|████████▊ | 88031199/100000000 [00:19<00:03, 3792372.90tokens/s]
Shard 49:  88%|████████▊ | 88422664/100000000 [00:19<00:03, 3651390.41tokens/s]
Shard 49:  89%|████████▉ | 88796434/100000000 [00:20<00:03, 3352616.85tokens/s]
Shard 49:  89%|████████▉ | 89141211/100000000 [00:20<00:03, 3357306.23tokens/s]
Shard 49:  90%|████████▉ | 89505103/100000000 [00:20<00:03, 3433494.42tokens/s]
Shard 49:  90%|████████▉ | 89887868/100000000 [00:20<00:02, 3531753.38tokens/s]
Shard 49:  90%|█████████ | 90369410/100000000 [00:20<00:02, 3897604.49tokens/s]
Shard 49:  91%|█████████ | 90764451/100000000 [00:20<00:02, 3830028.72tokens/s]
Shard 49:  91%|█████████ | 91152134/100000000 [00:20<00:02, 3834380.40tokens/s]
Shard 49:  92%|█████████▏| 91550123/100000000 [00:20<00:02, 3864685.93tokens/s]
Shard 49:  92%|█████████▏| 92097886/100000000 [00:20<00:01, 4332893.16tokens/s]
Shard 49:  93%|█████████▎| 92533805/100000000 [00:21<00:01, 4113096.47tokens/s]
Shard 49:  93%|█████████▎| 92949594/100000000 [00:21<00:01, 4087754.89tokens/s]
Shard 49:  93%|█████████▎| 93486129/100000000 [00:21<00:01, 4455188.09tokens/s]
Shard 49:  94%|█████████▍| 93935946/100000000 [00:21<00:01, 4456258.53tokens/s]
Shard 49:  94%|█████████▍| 94463081/100000000 [00:21<00:01, 4690873.18tokens/s]
Shard 49:  95%|█████████▍| 94938223/100000000 [00:21<00:01, 4542472.17tokens/s]
Shard 49:  95%|█████████▌| 95395401/100000000 [00:21<00:01, 4452962.15tokens/s]
Shard 49:  96%|█████████▌| 95842640/100000000 [00:21<00:01, 3973212.58tokens/s]
Shard 49:  96%|█████████▋| 96277765/100000000 [00:21<00:00, 4014520.39tokens/s]
Shard 49:  97%|█████████▋| 96689314/100000000 [00:21<00:00, 4025802.75tokens/s]
Shard 49:  97%|█████████▋| 97144308/100000000 [00:22<00:00, 4172765.83tokens/s]
Shard 49:  98%|█████████▊| 97566736/100000000 [00:22<00:00, 3680554.79tokens/s]
Shard 49:  98%|█████████▊| 98303296/100000000 [00:22<00:00, 4650053.47tokens/s]
Shard 49:  99%|█████████▉| 98790699/100000000 [00:22<00:00, 4566646.03tokens/s]
Shard 49:  99%|█████████▉| 99413480/100000000 [00:22<00:00, 5021389.18tokens/s]
Shard 49: 100%|█████████▉| 99999911/100000000 [00:30<00:00, 3312480.44tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000049.bin (200,001,024 bytes) in the gpt-2 format

Shard 50:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 50:   0%|          | 353670/100000000 [00:00<03:17, 504623.22tokens/s]
Shard 50:   1%|          | 688895/100000000 [00:00<01:40, 986758.77tokens/s]
Shard 50:   1%|          | 1041928/100000000 [00:00<01:06, 1480728.76tokens/s]
Shard 50:   1%|▏         | 1472675/100000000 [00:01<00:47, 2093306.77tokens/s]
Shard 50:   2%|▏         | 1945955/100000000 [00:01<00:36, 2709409.69tokens/s]
Shard 50:   2%|▏         | 2330460/100000000 [00:01<00:33, 2927135.27tokens/s]
Shard 50:   3%|▎         | 3094393/100000000 [00:01<00:23, 4148206.92tokens/s]
Shard 50:   4%|▎         | 3594342/100000000 [00:01<00:23, 4020686.53tokens/s]
Shard 50:   4%|▍         | 4054806/100000000 [00:01<00:26, 3634350.00tokens/s]
Shard 50:   4%|▍         | 4464210/100000000 [00:01<00:26, 3638858.31tokens/s]
Shard 50:   5%|▌         | 5136162/100000000 [00:01<00:21, 4413907.55tokens/s]
Shard 50:   6%|▌         | 5624311/100000000 [00:01<00:20, 4525360.65tokens/s]
Shard 50:   6%|▌         | 6105438/100000000 [00:02<00:23, 3955371.65tokens/s]
Shard 50:   7%|▋         | 6536183/100000000 [00:02<00:25, 3694434.01tokens/s]
Shard 50:   7%|▋         | 6928274/100000000 [00:02<00:26, 3520411.65tokens/s]
Shard 50:   7%|▋         | 7297296/100000000 [00:02<00:27, 3372497.49tokens/s]
Shard 50:   8%|▊         | 7647018/100000000 [00:02<00:27, 3348782.52tokens/s]
Shard 50:   8%|▊         | 7988825/100000000 [00:02<00:27, 3296098.67tokens/s]
Shard 50:   8%|▊         | 8323109/100000000 [00:02<00:28, 3172738.53tokens/s]
Shard 50:   9%|▉         | 8789819/100000000 [00:02<00:25, 3572108.89tokens/s]
Shard 50:   9%|▉         | 9154736/100000000 [00:03<00:26, 3493693.28tokens/s]
Shard 50:  10%|▉         | 9508979/100000000 [00:03<00:26, 3391769.33tokens/s]
Shard 50:  10%|▉         | 9852145/100000000 [00:03<00:28, 3109899.92tokens/s]
Shard 50:  10%|█         | 10168723/100000000 [00:03<00:30, 2968064.86tokens/s]
Shard 50:  10%|█         | 10469717/100000000 [00:03<00:30, 2926626.86tokens/s]
Shard 50:  11%|█         | 11016031/100000000 [00:03<00:24, 3607973.36tokens/s]
Shard 50:  12%|█▏        | 11512535/100000000 [00:03<00:22, 3936976.77tokens/s]
Shard 50:  12%|█▏        | 11925817/100000000 [00:03<00:31, 2792185.33tokens/s]
Shard 50:  12%|█▏        | 12258320/100000000 [00:04<00:31, 2820827.85tokens/s]
Shard 50:  13%|█▎        | 12655323/100000000 [00:04<00:28, 3083790.55tokens/s]
Shard 50:  13%|█▎        | 12996555/100000000 [00:04<00:28, 3077916.47tokens/s]
Shard 50:  13%|█▎        | 13327656/100000000 [00:04<00:28, 3065785.45tokens/s]
Shard 50:  14%|█▎        | 13653648/100000000 [00:04<00:27, 3098871.60tokens/s]
Shard 50:  14%|█▍        | 13976557/100000000 [00:04<00:29, 2885580.26tokens/s]
Shard 50:  14%|█▍        | 14275605/100000000 [00:04<00:30, 2783105.60tokens/s]
Shard 50:  15%|█▍        | 14780211/100000000 [00:04<00:25, 3382153.49tokens/s]
Shard 50:  15%|█▌        | 15131769/100000000 [00:04<00:27, 3058872.56tokens/s]
Shard 50:  16%|█▌        | 15505509/100000000 [00:05<00:26, 3212916.90tokens/s]
Shard 50:  16%|█▌        | 16024650/100000000 [00:05<00:22, 3730336.15tokens/s]
Shard 50:  16%|█▋        | 16411369/100000000 [00:05<00:25, 3317462.13tokens/s]
Shard 50:  17%|█▋        | 16776022/100000000 [00:05<00:24, 3396197.74tokens/s]
Shard 50:  17%|█▋        | 17130968/100000000 [00:05<00:24, 3405613.11tokens/s]
Shard 50:  18%|█▊        | 17545464/100000000 [00:05<00:22, 3601013.52tokens/s]
Shard 50:  18%|█▊        | 17914863/100000000 [00:05<00:23, 3475698.06tokens/s]
Shard 50:  18%|█▊        | 18314352/100000000 [00:05<00:22, 3618851.12tokens/s]
Shard 50:  19%|█▊        | 18681578/100000000 [00:05<00:22, 3594418.30tokens/s]
Shard 50:  19%|█▉        | 19073634/100000000 [00:06<00:22, 3639076.30tokens/s]
Shard 50:  19%|█▉        | 19440171/100000000 [00:06<00:24, 3298074.37tokens/s]
Shard 50:  20%|█▉        | 19877947/100000000 [00:06<00:22, 3585404.81tokens/s]
Shard 50:  20%|██        | 20247070/100000000 [00:06<00:24, 3320730.35tokens/s]
Shard 50:  21%|██        | 20628491/100000000 [00:06<00:23, 3423269.95tokens/s]
Shard 50:  21%|██        | 20979003/100000000 [00:06<00:24, 3279129.58tokens/s]
Shard 50:  21%|██▏       | 21451437/100000000 [00:06<00:21, 3658686.94tokens/s]
Shard 50:  22%|██▏       | 21860993/100000000 [00:06<00:20, 3774793.45tokens/s]
Shard 50:  22%|██▏       | 22355268/100000000 [00:06<00:19, 4078450.17tokens/s]
Shard 50:  23%|██▎       | 22768916/100000000 [00:07<00:19, 3878034.40tokens/s]
Shard 50:  23%|██▎       | 23209015/100000000 [00:07<00:19, 4023541.79tokens/s]
Shard 50:  24%|██▎       | 23616022/100000000 [00:07<00:21, 3605175.60tokens/s]
Shard 50:  24%|██▍       | 23987475/100000000 [00:07<00:21, 3611252.52tokens/s]
Shard 50:  24%|██▍       | 24355995/100000000 [00:07<00:20, 3630795.74tokens/s]
Shard 50:  25%|██▍       | 24765067/100000000 [00:07<00:20, 3760197.60tokens/s]
Shard 50:  25%|██▌       | 25145820/100000000 [00:07<00:21, 3532180.18tokens/s]
Shard 50:  26%|██▌       | 25504929/100000000 [00:07<00:21, 3431440.21tokens/s]
Shard 50:  26%|██▌       | 25862859/100000000 [00:07<00:22, 3357777.50tokens/s]
Shard 50:  26%|██▋       | 26351524/100000000 [00:08<00:19, 3727386.40tokens/s]
Shard 50:  27%|██▋       | 26748907/100000000 [00:08<00:19, 3788482.71tokens/s]
Shard 50:  27%|██▋       | 27158617/100000000 [00:08<00:18, 3863200.07tokens/s]
Shard 50:  28%|██▊       | 27551377/100000000 [00:08<00:18, 3871864.25tokens/s]
Shard 50:  28%|██▊       | 27944578/100000000 [00:08<00:18, 3888587.54tokens/s]
Shard 50:  28%|██▊       | 28336887/100000000 [00:08<00:19, 3704886.20tokens/s]
Shard 50:  29%|██▊       | 28710164/100000000 [00:08<00:19, 3619451.07tokens/s]
Shard 50:  29%|██▉       | 29073963/100000000 [00:08<00:20, 3433550.11tokens/s]
Shard 50:  29%|██▉       | 29420434/100000000 [00:08<00:20, 3403628.34tokens/s]
Shard 50:  30%|██▉       | 29805580/100000000 [00:09<00:19, 3515862.14tokens/s]
Shard 50:  30%|███       | 30161943/100000000 [00:09<00:21, 3223551.13tokens/s]
Shard 50:  30%|███       | 30492311/100000000 [00:09<00:22, 3070075.99tokens/s]
Shard 50:  31%|███       | 30880296/100000000 [00:09<00:21, 3287316.16tokens/s]
Shard 50:  31%|███       | 31214643/100000000 [00:09<00:20, 3281633.26tokens/s]
Shard 50:  32%|███▏      | 31611025/100000000 [00:09<00:19, 3434693.41tokens/s]
Shard 50:  32%|███▏      | 31957857/100000000 [00:09<00:19, 3406472.80tokens/s]
Shard 50:  32%|███▏      | 32416311/100000000 [00:09<00:18, 3740639.79tokens/s]
Shard 50:  33%|███▎      | 32837000/100000000 [00:09<00:17, 3863975.14tokens/s]
Shard 50:  33%|███▎      | 33230748/100000000 [00:09<00:17, 3865231.56tokens/s]
Shard 50:  34%|███▎      | 33620008/100000000 [00:10<00:18, 3669642.09tokens/s]
Shard 50:  34%|███▍      | 33992375/100000000 [00:10<00:19, 3469737.69tokens/s]
Shard 50:  34%|███▍      | 34365119/100000000 [00:10<00:18, 3530176.52tokens/s]
Shard 50:  35%|███▍      | 34732180/100000000 [00:10<00:18, 3569847.58tokens/s]
Shard 50:  35%|███▌      | 35091887/100000000 [00:10<00:18, 3468386.87tokens/s]
Shard 50:  35%|███▌      | 35446165/100000000 [00:10<00:18, 3408746.82tokens/s]
Shard 50:  36%|███▌      | 35788562/100000000 [00:10<00:20, 3153953.33tokens/s]
Shard 50:  36%|███▌      | 36107979/100000000 [00:10<00:20, 3094739.71tokens/s]
Shard 50:  36%|███▋      | 36420307/100000000 [00:11<00:22, 2863444.91tokens/s]
Shard 50:  37%|███▋      | 36907280/100000000 [00:11<00:18, 3390931.33tokens/s]
Shard 50:  37%|███▋      | 37260824/100000000 [00:11<00:18, 3402016.03tokens/s]
Shard 50:  38%|███▊      | 37608215/100000000 [00:11<00:19, 3243174.88tokens/s]
Shard 50:  38%|███▊      | 37992911/100000000 [00:11<00:18, 3370549.64tokens/s]
Shard 50:  38%|███▊      | 38334558/100000000 [00:11<00:19, 3174713.87tokens/s]
Shard 50:  39%|███▊      | 38690251/100000000 [00:11<00:18, 3273573.22tokens/s]
Shard 50:  39%|███▉      | 39022205/100000000 [00:11<00:19, 3159351.97tokens/s]
Shard 50:  39%|███▉      | 39381614/100000000 [00:11<00:18, 3269004.53tokens/s]
Shard 50:  40%|███▉      | 39786743/100000000 [00:11<00:17, 3486534.05tokens/s]
Shard 50:  40%|████      | 40139277/100000000 [00:12<00:17, 3364511.41tokens/s]
Shard 50:  40%|████      | 40480288/100000000 [00:12<00:17, 3375553.87tokens/s]
Shard 50:  41%|████      | 40877630/100000000 [00:12<00:16, 3539099.82tokens/s]
Shard 50:  41%|████      | 41237885/100000000 [00:12<00:17, 3350845.66tokens/s]
Shard 50:  42%|████▏     | 41595713/100000000 [00:12<00:17, 3389484.97tokens/s]
Shard 50:  42%|████▏     | 41938591/100000000 [00:12<00:17, 3232379.15tokens/s]
Shard 50:  42%|████▏     | 42398942/100000000 [00:12<00:16, 3589201.88tokens/s]
Shard 50:  43%|████▎     | 42783759/100000000 [00:12<00:15, 3622163.54tokens/s]
Shard 50:  43%|████▎     | 43313912/100000000 [00:12<00:14, 4015868.97tokens/s]
Shard 50:  44%|████▍     | 44055512/100000000 [00:13<00:11, 4977776.65tokens/s]
Shard 50:  45%|████▍     | 44559811/100000000 [00:13<00:11, 4831230.74tokens/s]
Shard 50:  45%|████▌     | 45047800/100000000 [00:13<00:13, 4217837.72tokens/s]
Shard 50:  46%|████▌     | 45606255/100000000 [00:13<00:11, 4576045.44tokens/s]
Shard 50:  46%|████▌     | 46082476/100000000 [00:13<00:13, 4040123.87tokens/s]
Shard 50:  47%|████▋     | 46508199/100000000 [00:13<00:13, 4054151.39tokens/s]
Shard 50:  47%|████▋     | 46929069/100000000 [00:13<00:14, 3690103.40tokens/s]
Shard 50:  47%|████▋     | 47312842/100000000 [00:13<00:14, 3623891.39tokens/s]
Shard 50:  48%|████▊     | 47684668/100000000 [00:14<00:14, 3550893.22tokens/s]
Shard 50:  48%|████▊     | 48104763/100000000 [00:14<00:13, 3723656.92tokens/s]
Shard 50:  48%|████▊     | 48484317/100000000 [00:14<00:14, 3547440.29tokens/s]
Shard 50:  49%|████▉     | 48950370/100000000 [00:14<00:13, 3849634.62tokens/s]
Shard 50:  49%|████▉     | 49345022/100000000 [00:14<00:14, 3582708.97tokens/s]
Shard 50:  50%|████▉     | 49712202/100000000 [00:14<00:14, 3498062.51tokens/s]
Shard 50:  50%|█████     | 50116023/100000000 [00:14<00:13, 3603954.26tokens/s]
Shard 50:  50%|█████     | 50481162/100000000 [00:14<00:14, 3336962.39tokens/s]
Shard 50:  51%|█████     | 50840828/100000000 [00:14<00:14, 3398824.61tokens/s]
Shard 50:  51%|█████     | 51186388/100000000 [00:15<00:14, 3375666.90tokens/s]
Shard 50:  52%|█████▏    | 52180176/100000000 [00:15<00:09, 5217301.54tokens/s]
Shard 50:  53%|█████▎    | 52716874/100000000 [00:15<00:09, 4822200.00tokens/s]
Shard 50:  53%|█████▎    | 53214869/100000000 [00:15<00:11, 3940793.55tokens/s]
Shard 50:  54%|█████▎    | 53726701/100000000 [00:15<00:10, 4219282.20tokens/s]
Shard 50:  54%|█████▍    | 54182522/100000000 [00:15<00:11, 4026963.63tokens/s]
Shard 50:  55%|█████▌    | 55170416/100000000 [00:15<00:08, 5508330.48tokens/s]
Shard 50:  56%|█████▌    | 55762970/100000000 [00:15<00:08, 5398820.36tokens/s]
Shard 50:  56%|█████▋    | 56331955/100000000 [00:16<00:09, 4816443.12tokens/s]
Shard 50:  57%|█████▋    | 56844104/100000000 [00:16<00:10, 3948383.39tokens/s]
Shard 50:  58%|█████▊    | 57806537/100000000 [00:16<00:08, 5217467.70tokens/s]
Shard 50:  58%|█████▊    | 58402568/100000000 [00:16<00:07, 5366809.35tokens/s]
Shard 50:  59%|█████▉    | 58992195/100000000 [00:16<00:08, 5105173.05tokens/s]
Shard 50:  60%|█████▉    | 59714343/100000000 [00:16<00:07, 5630201.91tokens/s]
Shard 50:  60%|██████    | 60313366/100000000 [00:16<00:07, 5469758.90tokens/s]
Shard 50:  61%|██████    | 60885506/100000000 [00:16<00:07, 5529111.45tokens/s]
Shard 50:  61%|██████▏   | 61456984/100000000 [00:17<00:07, 5216903.02tokens/s]
Shard 50:  62%|██████▏   | 61993614/100000000 [00:17<00:07, 4883185.39tokens/s]
Shard 50:  62%|██████▏   | 62494646/100000000 [00:17<00:08, 4401853.37tokens/s]
Shard 50:  63%|██████▎   | 62949393/100000000 [00:17<00:08, 4251957.72tokens/s]
Shard 50:  64%|██████▎   | 63607444/100000000 [00:17<00:07, 4844802.58tokens/s]
Shard 50:  64%|██████▍   | 64234216/100000000 [00:17<00:06, 5224289.21tokens/s]
Shard 50:  65%|██████▌   | 65326289/100000000 [00:17<00:05, 6793022.69tokens/s]
Shard 50:  66%|██████▌   | 66029827/100000000 [00:17<00:04, 6824693.20tokens/s]
Shard 50:  67%|██████▋   | 66732615/100000000 [00:17<00:05, 6233547.49tokens/s]
Shard 50:  67%|██████▋   | 67377074/100000000 [00:18<00:05, 5487127.83tokens/s]
Shard 50:  68%|██████▊   | 67953905/100000000 [00:18<00:06, 5296877.97tokens/s]
Shard 50:  69%|██████▊   | 68610154/100000000 [00:18<00:05, 5593932.89tokens/s]
Shard 50:  69%|██████▉   | 69187348/100000000 [00:18<00:05, 5226087.05tokens/s]
Shard 50:  70%|██████▉   | 69770795/100000000 [00:18<00:05, 5344244.26tokens/s]
Shard 50:  70%|███████   | 70324198/100000000 [00:18<00:05, 5395278.90tokens/s]
Shard 50:  71%|███████   | 70872806/100000000 [00:18<00:05, 5356643.44tokens/s]
Shard 50:  71%|███████▏  | 71414487/100000000 [00:18<00:05, 5311797.46tokens/s]
Shard 50:  72%|███████▏  | 71953076/100000000 [00:19<00:06, 4332878.31tokens/s]
Shard 50:  72%|███████▏  | 72418029/100000000 [00:19<00:06, 4062185.40tokens/s]
Shard 50:  73%|███████▎  | 72847876/100000000 [00:19<00:06, 3994159.76tokens/s]
Shard 50:  73%|███████▎  | 73393273/100000000 [00:19<00:06, 4365427.75tokens/s]
Shard 50:  74%|███████▍  | 73847038/100000000 [00:19<00:05, 4374572.89tokens/s]
Shard 50:  74%|███████▍  | 74341787/100000000 [00:19<00:05, 4529165.60tokens/s]
Shard 50:  75%|███████▍  | 74806005/100000000 [00:19<00:06, 4118516.46tokens/s]
Shard 50:  75%|███████▌  | 75327779/100000000 [00:19<00:05, 4400771.54tokens/s]
Shard 50:  76%|███████▌  | 75780335/100000000 [00:19<00:05, 4248747.73tokens/s]
Shard 50:  76%|███████▋  | 76415173/100000000 [00:20<00:04, 4809669.67tokens/s]
Shard 50:  77%|███████▋  | 76907411/100000000 [00:20<00:05, 4366063.94tokens/s]
Shard 50:  77%|███████▋  | 77360874/100000000 [00:20<00:05, 4211394.33tokens/s]
Shard 50:  78%|███████▊  | 77818309/100000000 [00:20<00:05, 4301186.88tokens/s]
Shard 50:  78%|███████▊  | 78480333/100000000 [00:20<00:04, 4938475.38tokens/s]
Shard 50:  79%|███████▉  | 78989273/100000000 [00:20<00:05, 4092407.28tokens/s]
Shard 50:  79%|███████▉  | 79433701/100000000 [00:20<00:04, 4169676.67tokens/s]
Shard 50:  80%|███████▉  | 79874956/100000000 [00:20<00:05, 4016656.66tokens/s]
Shard 50:  80%|████████  | 80296453/100000000 [00:21<00:04, 3950014.89tokens/s]
Shard 50:  81%|████████  | 80702394/100000000 [00:21<00:04, 3864960.61tokens/s]
Shard 50:  81%|████████  | 81182634/100000000 [00:21<00:04, 4068405.32tokens/s]
Shard 50:  82%|████████▏ | 81596172/100000000 [00:21<00:05, 3583637.47tokens/s]
Shard 50:  82%|████████▏ | 81968419/100000000 [00:21<00:05, 3234087.17tokens/s]
Shard 50:  82%|████████▏ | 82305490/100000000 [00:21<00:05, 3030082.56tokens/s]
Shard 50:  83%|████████▎ | 82695289/100000000 [00:21<00:05, 3231362.42tokens/s]
Shard 50:  83%|████████▎ | 83029041/100000000 [00:21<00:05, 3072777.38tokens/s]
Shard 50:  83%|████████▎ | 83442860/100000000 [00:21<00:04, 3347321.91tokens/s]
Shard 50:  84%|████████▍ | 83789908/100000000 [00:22<00:04, 3373840.32tokens/s]
Shard 50:  84%|████████▍ | 84136689/100000000 [00:22<00:04, 3379576.28tokens/s]
Shard 50:  84%|████████▍ | 84481458/100000000 [00:22<00:05, 3091888.94tokens/s]
Shard 50:  85%|████████▍ | 84805690/100000000 [00:22<00:04, 3131413.87tokens/s]
Shard 50:  85%|████████▌ | 85124689/100000000 [00:22<00:04, 3051083.09tokens/s]
Shard 50:  85%|████████▌ | 85440084/100000000 [00:22<00:04, 3068502.95tokens/s]
Shard 50:  86%|████████▌ | 85796567/100000000 [00:22<00:04, 3208770.80tokens/s]
Shard 50:  86%|████████▌ | 86134130/100000000 [00:22<00:04, 3231264.92tokens/s]
Shard 50:  87%|████████▋ | 86652398/100000000 [00:22<00:03, 3785593.80tokens/s]
Shard 50:  87%|████████▋ | 87034439/100000000 [00:23<00:03, 3550344.15tokens/s]
Shard 50:  88%|████████▊ | 87587371/100000000 [00:23<00:03, 4102386.86tokens/s]
Shard 50:  88%|████████▊ | 88099421/100000000 [00:23<00:02, 4393557.12tokens/s]
Shard 50:  89%|████████▊ | 88545815/100000000 [00:23<00:02, 3936953.34tokens/s]
Shard 50:  89%|████████▉ | 88952503/100000000 [00:23<00:02, 3785710.09tokens/s]
Shard 50:  89%|████████▉ | 89340348/100000000 [00:23<00:03, 3516200.68tokens/s]
Shard 50:  90%|████████▉ | 89736327/100000000 [00:23<00:02, 3609731.99tokens/s]
Shard 50:  90%|█████████ | 90232614/100000000 [00:23<00:02, 3968929.53tokens/s]
Shard 50:  91%|█████████ | 91209199/100000000 [00:23<00:01, 5529345.03tokens/s]
Shard 50:  92%|█████████▏| 91777092/100000000 [00:24<00:01, 5020037.34tokens/s]
Shard 50:  92%|█████████▏| 92346504/100000000 [00:24<00:01, 5159801.44tokens/s]
Shard 50:  93%|█████████▎| 92876291/100000000 [00:24<00:01, 4364157.24tokens/s]
Shard 50:  93%|█████████▎| 93340888/100000000 [00:24<00:01, 4254171.65tokens/s]
Shard 50:  94%|█████████▍| 93788917/100000000 [00:24<00:01, 3909328.61tokens/s]
Shard 50:  94%|█████████▍| 94201354/100000000 [00:24<00:01, 3541027.83tokens/s]
Shard 50:  95%|█████████▍| 94570558/100000000 [00:24<00:01, 3518242.04tokens/s]
Shard 50:  95%|█████████▍| 94931912/100000000 [00:25<00:01, 3463322.90tokens/s]
Shard 50:  95%|█████████▌| 95284717/100000000 [00:25<00:01, 3379067.15tokens/s]
Shard 50:  96%|█████████▌| 95779722/100000000 [00:25<00:01, 3776824.54tokens/s]
Shard 50:  96%|█████████▌| 96165045/100000000 [00:25<00:01, 3754012.59tokens/s]
Shard 50:  97%|█████████▋| 96618371/100000000 [00:25<00:00, 3969236.12tokens/s]
Shard 50:  97%|█████████▋| 97020397/100000000 [00:25<00:00, 3884490.49tokens/s]
Shard 50:  98%|█████████▊| 97511292/100000000 [00:25<00:00, 4169564.86tokens/s]
Shard 50:  98%|█████████▊| 98055120/100000000 [00:25<00:00, 4483580.46tokens/s]
Shard 50:  99%|█████████▊| 98507681/100000000 [00:25<00:00, 3976517.41tokens/s]
Shard 50:  99%|█████████▉| 99121787/100000000 [00:25<00:00, 4555057.62tokens/s]
Shard 50: 100%|█████████▉| 99748375/100000000 [00:26<00:00, 5022522.28tokens/s]
Shard 50: 100%|█████████▉| 99999869/100000000 [00:34<00:00, 2883629.79tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000050.bin (200,001,024 bytes) in the gpt-2 format

Shard 51:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 51:   0%|          | 276311/100000000 [00:00<00:36, 2726764.87tokens/s]
Shard 51:   1%|          | 552940/100000000 [00:00<00:36, 2710178.80tokens/s]
Shard 51:   1%|          | 826138/100000000 [00:00<00:47, 2080050.98tokens/s]
Shard 51:   1%|          | 1047695/100000000 [00:00<00:53, 1861386.07tokens/s]
Shard 51:   1%|          | 1244957/100000000 [00:00<00:53, 1848485.09tokens/s]
Shard 51:   1%|▏         | 1478396/100000000 [00:00<00:49, 1977564.67tokens/s]
Shard 51:   2%|▏         | 1706124/100000000 [00:00<00:47, 2060875.78tokens/s]
Shard 51:   2%|▏         | 2060343/100000000 [00:00<00:39, 2495599.82tokens/s]
Shard 51:   2%|▏         | 2484623/100000000 [00:01<00:32, 2977722.63tokens/s]
Shard 51:   3%|▎         | 2837495/100000000 [00:01<00:31, 3129651.72tokens/s]
Shard 51:   3%|▎         | 3156124/100000000 [00:01<00:31, 3050394.20tokens/s]
Shard 51:   3%|▎         | 3464969/100000000 [00:01<00:32, 2940422.65tokens/s]
Shard 51:   4%|▍         | 3763784/100000000 [00:01<00:32, 2947673.57tokens/s]
Shard 51:   4%|▍         | 4061076/100000000 [00:01<00:36, 2641449.68tokens/s]
Shard 51:   4%|▍         | 4332664/100000000 [00:01<00:39, 2448052.91tokens/s]
Shard 51:   5%|▍         | 4584391/100000000 [00:01<00:43, 2190971.40tokens/s]
Shard 51:   5%|▍         | 4813076/100000000 [00:01<00:43, 2182842.59tokens/s]
Shard 51:   5%|▌         | 5036570/100000000 [00:02<00:43, 2186797.32tokens/s]
Shard 51:   5%|▌         | 5304710/100000000 [00:02<00:40, 2316335.54tokens/s]
Shard 51:   6%|▌         | 5555738/100000000 [00:02<00:40, 2352597.23tokens/s]
Shard 51:   6%|▌         | 5794306/100000000 [00:02<00:43, 2190490.09tokens/s]
Shard 51:   6%|▌         | 6111516/100000000 [00:02<00:38, 2425049.24tokens/s]
Shard 51:   6%|▋         | 6400089/100000000 [00:02<00:36, 2529920.37tokens/s]
Shard 51:   7%|▋         | 6851304/100000000 [00:02<00:30, 3085092.91tokens/s]
Shard 51:   7%|▋         | 7165653/100000000 [00:02<00:30, 2999409.26tokens/s]
Shard 51:   8%|▊         | 7573396/100000000 [00:02<00:27, 3303990.55tokens/s]
Shard 51:   8%|▊         | 8375711/100000000 [00:03<00:19, 4663068.14tokens/s]
Shard 51:   9%|▉         | 8851898/100000000 [00:03<00:25, 3617623.54tokens/s]
Shard 51:   9%|▉         | 9256034/100000000 [00:03<00:26, 3430272.34tokens/s]
Shard 51:  10%|▉         | 9732644/100000000 [00:03<00:24, 3743947.87tokens/s]
Shard 51:  10%|█         | 10137871/100000000 [00:03<00:23, 3760953.56tokens/s]
Shard 51:  11%|█         | 10533525/100000000 [00:03<00:24, 3583592.61tokens/s]
Shard 51:  11%|█         | 10906525/100000000 [00:03<00:26, 3362416.79tokens/s]
Shard 51:  11%|█▏        | 11287496/100000000 [00:03<00:25, 3470056.06tokens/s]
Shard 51:  12%|█▏        | 11762830/100000000 [00:04<00:23, 3789368.60tokens/s]
Shard 51:  12%|█▏        | 12213447/100000000 [00:04<00:22, 3985403.51tokens/s]
Shard 51:  13%|█▎        | 12756026/100000000 [00:04<00:20, 4358519.24tokens/s]
Shard 51:  13%|█▎        | 13199321/100000000 [00:04<00:22, 3810366.12tokens/s]
Shard 51:  14%|█▎        | 13607162/100000000 [00:04<00:22, 3865574.28tokens/s]
Shard 51:  14%|█▍        | 14029783/100000000 [00:04<00:22, 3858809.59tokens/s]
Shard 51:  14%|█▍        | 14424605/100000000 [00:04<00:23, 3705793.68tokens/s]
Shard 51:  15%|█▍        | 14802766/100000000 [00:04<00:22, 3707263.95tokens/s]
Shard 51:  15%|█▌        | 15178037/100000000 [00:04<00:24, 3490529.11tokens/s]
Shard 51:  16%|█▌        | 15533107/100000000 [00:05<00:27, 3065890.22tokens/s]
Shard 51:  16%|█▌        | 15860161/100000000 [00:05<00:27, 3104492.21tokens/s]
Shard 51:  16%|█▌        | 16178779/100000000 [00:05<00:27, 3076911.89tokens/s]
Shard 51:  16%|█▋        | 16493610/100000000 [00:05<00:28, 2903263.14tokens/s]
Shard 51:  17%|█▋        | 16888211/100000000 [00:05<00:26, 3173035.18tokens/s]
Shard 51:  17%|█▋        | 17213192/100000000 [00:05<00:27, 3020273.04tokens/s]
Shard 51:  18%|█▊        | 17520495/100000000 [00:05<00:27, 3028647.96tokens/s]
Shard 51:  18%|█▊        | 17871456/100000000 [00:05<00:26, 3151846.30tokens/s]
Shard 51:  18%|█▊        | 18227370/100000000 [00:05<00:25, 3255370.96tokens/s]
Shard 51:  19%|█▊        | 18612832/100000000 [00:06<00:24, 3375721.39tokens/s]
Shard 51:  19%|█▉        | 19385826/100000000 [00:06<00:17, 4612672.90tokens/s]
Shard 51:  20%|█▉        | 19863035/100000000 [00:06<00:17, 4657009.69tokens/s]
Shard 51:  20%|██        | 20369067/100000000 [00:06<00:16, 4732268.25tokens/s]
Shard 51:  21%|██        | 20845153/100000000 [00:06<00:17, 4482229.66tokens/s]
Shard 51:  21%|██▏       | 21318727/100000000 [00:06<00:17, 4544439.59tokens/s]
Shard 51:  22%|██▏       | 21835856/100000000 [00:06<00:16, 4712677.87tokens/s]
Shard 51:  22%|██▏       | 22310328/100000000 [00:06<00:17, 4561871.00tokens/s]
Shard 51:  23%|██▎       | 22986470/100000000 [00:06<00:14, 5168575.87tokens/s]
Shard 51:  24%|██▎       | 23507751/100000000 [00:07<00:18, 4209007.14tokens/s]
Shard 51:  24%|██▍       | 23959435/100000000 [00:07<00:20, 3694062.22tokens/s]
Shard 51:  24%|██▍       | 24358325/100000000 [00:07<00:22, 3404924.42tokens/s]
Shard 51:  25%|██▍       | 24818714/100000000 [00:07<00:20, 3681569.40tokens/s]
Shard 51:  25%|██▌       | 25209820/100000000 [00:07<00:21, 3546731.62tokens/s]
Shard 51:  26%|██▌       | 25580355/100000000 [00:07<00:21, 3513358.59tokens/s]
Shard 51:  26%|██▌       | 25943959/100000000 [00:07<00:20, 3544421.71tokens/s]
Shard 51:  26%|██▋       | 26306431/100000000 [00:07<00:21, 3446536.07tokens/s]
Shard 51:  27%|██▋       | 26657309/100000000 [00:08<00:22, 3192667.91tokens/s]
Shard 51:  27%|██▋       | 26982635/100000000 [00:08<00:25, 2869678.68tokens/s]
Shard 51:  27%|██▋       | 27396584/100000000 [00:08<00:22, 3172846.96tokens/s]
Shard 51:  28%|██▊       | 27868897/100000000 [00:08<00:20, 3577967.61tokens/s]
Shard 51:  28%|██▊       | 28251068/100000000 [00:08<00:20, 3583408.27tokens/s]
Shard 51:  29%|██▊       | 28618674/100000000 [00:08<00:20, 3487842.47tokens/s]
Shard 51:  29%|██▉       | 28990480/100000000 [00:08<00:20, 3541928.92tokens/s]
Shard 51:  29%|██▉       | 29385641/100000000 [00:08<00:19, 3657158.79tokens/s]
Shard 51:  30%|██▉       | 29755697/100000000 [00:08<00:19, 3656836.05tokens/s]
Shard 51:  30%|███       | 30126320/100000000 [00:09<00:19, 3638521.42tokens/s]
Shard 51:  30%|███       | 30492824/100000000 [00:09<00:19, 3616054.27tokens/s]
Shard 51:  31%|███       | 30856061/100000000 [00:09<00:20, 3303605.43tokens/s]
Shard 51:  31%|███       | 31236898/100000000 [00:09<00:20, 3433579.00tokens/s]
Shard 51:  32%|███▏      | 31917000/100000000 [00:09<00:15, 4361290.98tokens/s]
Shard 51:  33%|███▎      | 32544819/100000000 [00:09<00:13, 4896139.43tokens/s]
Shard 51:  33%|███▎      | 33043143/100000000 [00:09<00:13, 4816446.04tokens/s]
Shard 51:  34%|███▎      | 33576936/100000000 [00:09<00:13, 4923598.52tokens/s]
Shard 51:  34%|███▍      | 34133204/100000000 [00:09<00:12, 5099894.22tokens/s]
Shard 51:  35%|███▍      | 34683340/100000000 [00:09<00:12, 5208886.80tokens/s]
Shard 51:  35%|███▌      | 35208497/100000000 [00:10<00:13, 4705127.55tokens/s]
Shard 51:  36%|███▌      | 35690097/100000000 [00:10<00:15, 4216767.64tokens/s]
Shard 51:  36%|███▌      | 36128363/100000000 [00:10<00:16, 3967560.09tokens/s]
Shard 51:  37%|███▋      | 36646395/100000000 [00:10<00:14, 4252101.21tokens/s]
Shard 51:  37%|███▋      | 37083771/100000000 [00:10<00:17, 3657229.23tokens/s]
Shard 51:  37%|███▋      | 37470469/100000000 [00:10<00:17, 3599197.81tokens/s]
Shard 51:  38%|███▊      | 37931082/100000000 [00:10<00:16, 3822710.62tokens/s]
Shard 51:  38%|███▊      | 38449260/100000000 [00:10<00:14, 4181420.13tokens/s]
Shard 51:  39%|███▉      | 38882052/100000000 [00:11<00:14, 4157783.34tokens/s]
Shard 51:  39%|███▉      | 39307798/100000000 [00:11<00:15, 3975997.96tokens/s]
Shard 51:  40%|███▉      | 39715167/100000000 [00:11<00:15, 3807381.49tokens/s]
Shard 51:  40%|████      | 40101841/100000000 [00:11<00:17, 3334565.28tokens/s]
Shard 51:  40%|████      | 40448443/100000000 [00:11<00:18, 3166800.30tokens/s]
Shard 51:  41%|████      | 40773765/100000000 [00:11<00:19, 3028185.13tokens/s]
Shard 51:  41%|████      | 41122515/100000000 [00:11<00:18, 3145175.72tokens/s]
Shard 51:  42%|████▏     | 41522091/100000000 [00:11<00:17, 3368053.33tokens/s]
Shard 51:  42%|████▏     | 41866408/100000000 [00:12<00:17, 3333189.07tokens/s]
Shard 51:  42%|████▏     | 42299135/100000000 [00:12<00:16, 3604234.40tokens/s]
Shard 51:  43%|████▎     | 42664704/100000000 [00:12<00:16, 3583326.07tokens/s]
Shard 51:  43%|████▎     | 43332317/100000000 [00:12<00:12, 4466753.77tokens/s]
Shard 51:  44%|████▍     | 43803183/100000000 [00:12<00:12, 4346103.90tokens/s]
Shard 51:  44%|████▍     | 44243071/100000000 [00:12<00:13, 4123928.35tokens/s]
Shard 51:  45%|████▍     | 44661099/100000000 [00:12<00:13, 4097865.37tokens/s]
Shard 51:  45%|████▌     | 45076357/100000000 [00:12<00:14, 3738739.27tokens/s]
Shard 51:  45%|████▌     | 45458340/100000000 [00:12<00:14, 3746104.57tokens/s]
Shard 51:  46%|████▌     | 45917299/100000000 [00:13<00:13, 3969203.68tokens/s]
Shard 51:  47%|████▋     | 46573587/100000000 [00:13<00:11, 4650313.12tokens/s]
Shard 51:  47%|████▋     | 47045549/100000000 [00:13<00:11, 4420855.58tokens/s]
Shard 51:  47%|████▋     | 47493580/100000000 [00:13<00:14, 3688519.70tokens/s]
Shard 51:  48%|████▊     | 47885658/100000000 [00:13<00:14, 3625347.24tokens/s]
Shard 51:  48%|████▊     | 48288140/100000000 [00:13<00:13, 3717795.99tokens/s]
Shard 51:  49%|████▉     | 48767981/100000000 [00:13<00:13, 3917158.09tokens/s]
Shard 51:  49%|████▉     | 49205297/100000000 [00:13<00:12, 4030644.86tokens/s]
Shard 51:  50%|████▉     | 49616403/100000000 [00:13<00:14, 3520731.76tokens/s]
Shard 51:  50%|████▉     | 49985052/100000000 [00:14<00:15, 3207975.61tokens/s]
Shard 51:  50%|█████     | 50426687/100000000 [00:14<00:14, 3506169.04tokens/s]
Shard 51:  51%|█████     | 50893242/100000000 [00:14<00:12, 3800630.19tokens/s]
Shard 51:  51%|█████▏    | 51337907/100000000 [00:14<00:12, 3976213.66tokens/s]
Shard 51:  52%|█████▏    | 51921743/100000000 [00:14<00:10, 4495152.23tokens/s]
Shard 51:  52%|█████▏    | 52390892/100000000 [00:14<00:10, 4442761.68tokens/s]
Shard 51:  53%|█████▎    | 52851245/100000000 [00:14<00:10, 4486188.65tokens/s]
Shard 51:  53%|█████▎    | 53422247/100000000 [00:14<00:09, 4831579.17tokens/s]
Shard 51:  54%|█████▍    | 53911633/100000000 [00:14<00:09, 4729998.21tokens/s]
Shard 51:  54%|█████▍    | 54497664/100000000 [00:15<00:09, 5051851.98tokens/s]
Shard 51:  55%|█████▌    | 55253678/100000000 [00:15<00:07, 5778041.90tokens/s]
Shard 51:  56%|█████▌    | 55837085/100000000 [00:15<00:08, 5295020.89tokens/s]
Shard 51:  56%|█████▋    | 56378268/100000000 [00:15<00:09, 4820983.11tokens/s]
Shard 51:  57%|█████▋    | 56874810/100000000 [00:15<00:09, 4655957.93tokens/s]
Shard 51:  57%|█████▋    | 57350530/100000000 [00:15<00:10, 4149040.99tokens/s]
Shard 51:  58%|█████▊    | 57779447/100000000 [00:15<00:10, 3996779.58tokens/s]
Shard 51:  58%|█████▊    | 58197791/100000000 [00:15<00:10, 3806965.35tokens/s]
Shard 51:  59%|█████▊    | 58585084/100000000 [00:16<00:11, 3720526.71tokens/s]
Shard 51:  59%|█████▉    | 59007585/100000000 [00:16<00:10, 3836702.07tokens/s]
Shard 51:  59%|█████▉    | 59461705/100000000 [00:16<00:10, 4019157.75tokens/s]
Shard 51:  60%|█████▉    | 59876043/100000000 [00:16<00:10, 3943353.10tokens/s]
Shard 51:  60%|██████    | 60273736/100000000 [00:16<00:10, 3859190.63tokens/s]
Shard 51:  61%|██████    | 60666696/100000000 [00:16<00:10, 3806256.59tokens/s]
Shard 51:  61%|██████▏   | 61308363/100000000 [00:16<00:08, 4483361.98tokens/s]
Shard 51:  62%|██████▏   | 61759809/100000000 [00:16<00:08, 4256691.39tokens/s]
Shard 51:  62%|██████▏   | 62257346/100000000 [00:16<00:08, 4449121.61tokens/s]
Shard 51:  63%|██████▎   | 62707138/100000000 [00:17<00:08, 4319942.86tokens/s]
Shard 51:  63%|██████▎   | 63142971/100000000 [00:17<00:08, 4329479.16tokens/s]
Shard 51:  64%|██████▎   | 63610006/100000000 [00:17<00:08, 4378266.29tokens/s]
Shard 51:  64%|██████▍   | 64049867/100000000 [00:17<00:08, 4036964.70tokens/s]
Shard 51:  64%|██████▍   | 64459288/100000000 [00:17<00:09, 3782099.50tokens/s]
Shard 51:  65%|██████▍   | 64846326/100000000 [00:17<00:10, 3494602.28tokens/s]
Shard 51:  65%|██████▌   | 65202948/100000000 [00:17<00:10, 3396961.74tokens/s]
Shard 51:  66%|██████▌   | 65546800/100000000 [00:17<00:10, 3390856.22tokens/s]
Shard 51:  66%|██████▌   | 65990856/100000000 [00:17<00:09, 3662194.64tokens/s]
Shard 51:  66%|██████▋   | 66361909/100000000 [00:18<00:09, 3650000.17tokens/s]
Shard 51:  67%|██████▋   | 66906446/100000000 [00:18<00:07, 4160120.12tokens/s]
Shard 51:  67%|██████▋   | 67363827/100000000 [00:18<00:07, 4279578.27tokens/s]
Shard 51:  68%|██████▊   | 67798757/100000000 [00:18<00:08, 3963882.64tokens/s]
Shard 51:  68%|██████▊   | 68229358/100000000 [00:18<00:07, 4020851.63tokens/s]
Shard 51:  69%|██████▉   | 68774399/100000000 [00:18<00:07, 4416514.92tokens/s]
Shard 51:  69%|██████▉   | 69222087/100000000 [00:18<00:07, 4192727.76tokens/s]
Shard 51:  70%|██████▉   | 69647366/100000000 [00:18<00:07, 4168922.45tokens/s]
Shard 51:  70%|███████   | 70151498/100000000 [00:18<00:06, 4397273.34tokens/s]
Shard 51:  71%|███████   | 70638581/100000000 [00:18<00:06, 4532488.22tokens/s]
Shard 51:  71%|███████   | 71246097/100000000 [00:19<00:05, 4936685.92tokens/s]
Shard 51:  72%|███████▏  | 71761315/100000000 [00:19<00:05, 4993381.08tokens/s]
Shard 51:  72%|███████▏  | 72263210/100000000 [00:19<00:05, 4655988.01tokens/s]
Shard 51:  73%|███████▎  | 72736271/100000000 [00:19<00:06, 4540866.80tokens/s]
Shard 51:  73%|███████▎  | 73204067/100000000 [00:19<00:05, 4574806.34tokens/s]
Shard 51:  74%|███████▎  | 73664777/100000000 [00:19<00:05, 4399222.92tokens/s]
Shard 51:  74%|███████▍  | 74238153/100000000 [00:19<00:05, 4747996.90tokens/s]
Shard 51:  75%|███████▍  | 74727191/100000000 [00:19<00:05, 4780167.88tokens/s]
Shard 51:  75%|███████▌  | 75208536/100000000 [00:19<00:05, 4521340.50tokens/s]
Shard 51:  76%|███████▌  | 75665288/100000000 [00:20<00:05, 4493702.04tokens/s]
Shard 51:  76%|███████▌  | 76124717/100000000 [00:20<00:05, 4423848.13tokens/s]
Shard 51:  77%|███████▋  | 76572391/100000000 [00:20<00:06, 3718532.58tokens/s]
Shard 51:  77%|███████▋  | 77143663/100000000 [00:20<00:05, 4213906.04tokens/s]
Shard 51:  78%|███████▊  | 77588121/100000000 [00:20<00:05, 4015493.64tokens/s]
Shard 51:  78%|███████▊  | 78006587/100000000 [00:20<00:05, 3883641.55tokens/s]
Shard 51:  78%|███████▊  | 78406551/100000000 [00:20<00:05, 3632765.11tokens/s]
Shard 51:  79%|███████▉  | 78863129/100000000 [00:20<00:05, 3860475.62tokens/s]
Shard 51:  79%|███████▉  | 79392157/100000000 [00:21<00:04, 4245602.02tokens/s]
Shard 51:  80%|███████▉  | 79828954/100000000 [00:21<00:04, 4266681.84tokens/s]
Shard 51:  80%|████████  | 80264930/100000000 [00:21<00:04, 4064345.40tokens/s]
Shard 51:  81%|████████  | 80679586/100000000 [00:21<00:05, 3829161.87tokens/s]
Shard 51:  81%|████████  | 81069582/100000000 [00:21<00:05, 3520709.17tokens/s]
Shard 51:  82%|████████▏ | 81514949/100000000 [00:21<00:04, 3760590.04tokens/s]
Shard 51:  82%|████████▏ | 81900677/100000000 [00:21<00:05, 3337048.68tokens/s]
Shard 51:  82%|████████▏ | 82247520/100000000 [00:21<00:05, 3358903.98tokens/s]
Shard 51:  83%|████████▎ | 82593050/100000000 [00:21<00:05, 3263999.16tokens/s]
Shard 51:  83%|████████▎ | 83000137/100000000 [00:22<00:04, 3460283.14tokens/s]
Shard 51:  83%|████████▎ | 83356733/100000000 [00:22<00:04, 3488765.14tokens/s]
Shard 51:  84%|████████▍ | 83786707/100000000 [00:22<00:04, 3703597.93tokens/s]
Shard 51:  85%|████████▍ | 84514787/100000000 [00:22<00:03, 4703431.71tokens/s]
Shard 51:  85%|████████▍ | 84993055/100000000 [00:22<00:03, 4505785.36tokens/s]
Shard 51:  86%|████████▌ | 85549425/100000000 [00:22<00:03, 4795609.56tokens/s]
Shard 51:  86%|████████▋ | 86277839/100000000 [00:22<00:02, 5490909.77tokens/s]
Shard 51:  87%|████████▋ | 86850056/100000000 [00:22<00:02, 5530283.22tokens/s]
Shard 51:  87%|████████▋ | 87410869/100000000 [00:22<00:03, 4167770.89tokens/s]
Shard 51:  88%|████████▊ | 87880513/100000000 [00:23<00:03, 3263319.89tokens/s]
Shard 51:  88%|████████▊ | 88269867/100000000 [00:23<00:03, 3222273.61tokens/s]
Shard 51:  89%|████████▊ | 88635670/100000000 [00:23<00:04, 2773933.53tokens/s]
Shard 51:  89%|████████▉ | 88949136/100000000 [00:23<00:04, 2342058.41tokens/s]
Shard 51:  89%|████████▉ | 89215048/100000000 [00:23<00:04, 2332020.91tokens/s]
Shard 51:  89%|████████▉ | 89470812/100000000 [00:23<00:04, 2263219.86tokens/s]
Shard 51:  90%|████████▉ | 89900504/100000000 [00:24<00:03, 2711861.93tokens/s]
Shard 51:  90%|█████████ | 90196165/100000000 [00:24<00:03, 2585111.26tokens/s]
Shard 51:  90%|█████████ | 90472343/100000000 [00:24<00:04, 2240859.96tokens/s]
Shard 51:  91%|█████████ | 90731718/100000000 [00:24<00:04, 2279102.19tokens/s]
Shard 51:  91%|█████████ | 91077274/100000000 [00:24<00:03, 2560248.18tokens/s]
Shard 51:  91%|█████████▏| 91348715/100000000 [00:24<00:03, 2471100.68tokens/s]
Shard 51:  92%|█████████▏| 91606602/100000000 [00:24<00:03, 2287132.79tokens/s]
Shard 51:  92%|█████████▏| 91850324/100000000 [00:24<00:03, 2320566.62tokens/s]
Shard 51:  92%|█████████▏| 92099220/100000000 [00:25<00:03, 2364450.83tokens/s]
Shard 51:  92%|█████████▏| 92371431/100000000 [00:25<00:03, 2462272.59tokens/s]
Shard 51:  93%|█████████▎| 92722696/100000000 [00:25<00:02, 2718177.10tokens/s]
Shard 51:  93%|█████████▎| 93233446/100000000 [00:25<00:01, 3396372.81tokens/s]
Shard 51:  94%|█████████▎| 93579777/100000000 [00:25<00:01, 3360478.10tokens/s]
Shard 51:  94%|█████████▍| 93920588/100000000 [00:25<00:01, 3055906.92tokens/s]
Shard 51:  94%|█████████▍| 94243166/100000000 [00:25<00:01, 3045141.02tokens/s]
Shard 51:  95%|█████████▍| 94610346/100000000 [00:25<00:01, 3210663.60tokens/s]
Shard 51:  95%|█████████▍| 94937308/100000000 [00:25<00:01, 2960887.71tokens/s]
Shard 51:  95%|█████████▌| 95240615/100000000 [00:26<00:01, 2597087.91tokens/s]
Shard 51:  96%|█████████▌| 95512013/100000000 [00:26<00:01, 2617223.09tokens/s]
Shard 51:  96%|█████████▌| 95818033/100000000 [00:26<00:01, 2724839.89tokens/s]
Shard 51:  96%|█████████▌| 96104245/100000000 [00:26<00:01, 2698478.57tokens/s]
Shard 51:  96%|█████████▋| 96430800/100000000 [00:26<00:01, 2851106.51tokens/s]
Shard 51:  97%|█████████▋| 96721310/100000000 [00:26<00:01, 2659186.51tokens/s]
Shard 51:  97%|█████████▋| 97163363/100000000 [00:26<00:00, 3133298.59tokens/s]
Shard 51:  97%|█████████▋| 97485292/100000000 [00:26<00:00, 2880772.51tokens/s]
Shard 51:  98%|█████████▊| 97858240/100000000 [00:26<00:00, 3061091.27tokens/s]
Shard 51:  98%|█████████▊| 98172136/100000000 [00:27<00:00, 3019618.46tokens/s]
Shard 51:  98%|█████████▊| 98480958/100000000 [00:27<00:00, 2919353.06tokens/s]
Shard 51:  99%|█████████▉| 98894211/100000000 [00:27<00:00, 3246043.09tokens/s]
Shard 51:  99%|█████████▉| 99224256/100000000 [00:27<00:00, 3174819.58tokens/s]
Shard 51: 100%|█████████▉| 99545687/100000000 [00:27<00:00, 2912203.84tokens/s]
Shard 51: 100%|█████████▉| 99843013/100000000 [00:27<00:00, 2734060.76tokens/s]
Shard 51: 100%|█████████▉| 99999514/100000000 [00:36<00:00, 2748409.77tokens/s]
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000051.bin (200,001,024 bytes) in the gpt-2 format

Shard 52:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 52:   0%|          | 163938/100000000 [00:00<01:03, 1584166.32tokens/s]
Shard 52:   0%|          | 357007/100000000 [00:00<00:55, 1783808.71tokens/s]
Shard 52:   1%|          | 567806/100000000 [00:00<00:51, 1913835.90tokens/s]
Shard 52:   1%|          | 770172/100000000 [00:00<00:51, 1943964.95tokens/s]
Shard 52:   1%|          | 1067954/100000000 [00:00<00:43, 2291532.63tokens/s]
Shard 52:   1%|▏         | 1352181/100000000 [00:00<00:40, 2462723.22tokens/s]
Shard 52:   2%|▏         | 1638375/100000000 [00:00<00:38, 2578548.25tokens/s]
Shard 52:   2%|▏         | 2260984/100000000 [00:00<00:26, 3709036.93tokens/s]
Shard 52:   3%|▎         | 2632403/100000000 [00:00<00:26, 3693035.28tokens/s]
Shard 52:   3%|▎         | 3027549/100000000 [00:01<00:25, 3761397.67tokens/s]
Shard 52:   3%|▎         | 3404279/100000000 [00:01<00:26, 3589222.99tokens/s]
Shard 52:   4%|▍         | 4242618/100000000 [00:01<00:19, 4973486.85tokens/s]
Shard 52:   5%|▍         | 4745762/100000000 [00:01<00:24, 3812515.53tokens/s]
Shard 52:   5%|▌         | 5170880/100000000 [00:01<00:24, 3869320.77tokens/s]
Shard 52:   6%|▌         | 5703418/100000000 [00:01<00:22, 4232316.30tokens/s]
Shard 52:   6%|▌         | 6156964/100000000 [00:01<00:24, 3795142.03tokens/s]
Shard 52:   7%|▋         | 6564511/100000000 [00:01<00:28, 3309733.29tokens/s]
Shard 52:   7%|▋         | 6921905/100000000 [00:02<00:29, 3197556.49tokens/s]
Shard 52:   7%|▋         | 7303641/100000000 [00:02<00:27, 3336911.89tokens/s]
Shard 52:   8%|▊         | 7655167/100000000 [00:02<00:30, 2991230.80tokens/s]
Shard 52:   8%|▊         | 7969405/100000000 [00:02<00:32, 2812444.18tokens/s]
Shard 52:   9%|▊         | 8550262/100000000 [00:02<00:25, 3542852.00tokens/s]
Shard 52:   9%|▉         | 8988479/100000000 [00:02<00:24, 3753894.28tokens/s]
Shard 52:   9%|▉         | 9432795/100000000 [00:02<00:23, 3923163.21tokens/s]
Shard 52:  10%|▉         | 9840904/100000000 [00:02<00:23, 3913358.19tokens/s]
Shard 52:  10%|█         | 10243312/100000000 [00:03<00:25, 3545033.16tokens/s]
Shard 52:  11%|█         | 10732377/100000000 [00:03<00:22, 3887947.66tokens/s]
Shard 52:  11%|█         | 11139842/100000000 [00:03<00:22, 3882920.14tokens/s]
Shard 52:  12%|█▏        | 11557291/100000000 [00:03<00:22, 3953839.48tokens/s]
Shard 52:  12%|█▏        | 11960101/100000000 [00:03<00:23, 3726439.45tokens/s]
Shard 52:  12%|█▏        | 12368657/100000000 [00:03<00:22, 3810637.23tokens/s]
Shard 52:  13%|█▎        | 12755749/100000000 [00:03<00:24, 3519484.49tokens/s]
Shard 52:  13%|█▎        | 13115457/100000000 [00:03<00:30, 2813929.33tokens/s]
Shard 52:  13%|█▎        | 13426754/100000000 [00:03<00:30, 2878007.32tokens/s]
Shard 52:  14%|█▎        | 13748404/100000000 [00:04<00:30, 2860455.70tokens/s]
Shard 52:  14%|█▍        | 14083412/100000000 [00:04<00:28, 2982300.16tokens/s]
Shard 52:  14%|█▍        | 14396197/100000000 [00:04<00:28, 3012281.41tokens/s]
Shard 52:  15%|█▍        | 14705917/100000000 [00:04<00:29, 2935208.73tokens/s]
Shard 52:  15%|█▌        | 15178714/100000000 [00:04<00:24, 3427412.28tokens/s]
Shard 52:  16%|█▌        | 15603175/100000000 [00:04<00:23, 3644582.71tokens/s]
Shard 52:  16%|█▌        | 15974592/100000000 [00:04<00:23, 3593092.71tokens/s]
Shard 52:  16%|█▋        | 16461509/100000000 [00:04<00:21, 3942448.70tokens/s]
Shard 52:  17%|█▋        | 16860842/100000000 [00:04<00:22, 3771127.29tokens/s]
Shard 52:  17%|█▋        | 17242742/100000000 [00:05<00:22, 3609635.43tokens/s]
Shard 52:  18%|█▊        | 17609887/100000000 [00:05<00:26, 3062825.59tokens/s]
Shard 52:  18%|█▊        | 18050585/100000000 [00:05<00:24, 3399288.55tokens/s]
Shard 52:  18%|█▊        | 18407806/100000000 [00:05<00:25, 3155666.26tokens/s]
Shard 52:  19%|█▉        | 18928532/100000000 [00:05<00:22, 3679180.04tokens/s]
Shard 52:  19%|█▉        | 19318689/100000000 [00:05<00:25, 3143864.13tokens/s]
Shard 52:  20%|█▉        | 19670156/100000000 [00:05<00:25, 3213014.32tokens/s]
Shard 52:  20%|██        | 20010644/100000000 [00:05<00:24, 3221190.99tokens/s]
Shard 52:  20%|██        | 20345987/100000000 [00:06<00:26, 2993833.10tokens/s]
Shard 52:  21%|██        | 20657987/100000000 [00:06<00:27, 2893763.29tokens/s]
Shard 52:  21%|██        | 21041364/100000000 [00:06<00:25, 3114568.12tokens/s]
Shard 52:  21%|██▏       | 21361602/100000000 [00:06<00:25, 3117051.11tokens/s]
Shard 52:  22%|██▏       | 21764698/100000000 [00:06<00:23, 3354996.63tokens/s]
Shard 52:  22%|██▏       | 22169899/100000000 [00:06<00:21, 3541082.85tokens/s]
Shard 52:  23%|██▎       | 22528991/100000000 [00:06<00:25, 3090428.90tokens/s]
Shard 52:  23%|██▎       | 22851401/100000000 [00:06<00:26, 2922969.22tokens/s]
Shard 52:  23%|██▎       | 23272403/100000000 [00:06<00:23, 3253068.96tokens/s]
Shard 52:  24%|██▎       | 23609846/100000000 [00:07<00:24, 3178386.01tokens/s]
Shard 52:  24%|██▍       | 23938616/100000000 [00:07<00:26, 2904363.23tokens/s]
Shard 52:  24%|██▍       | 24239072/100000000 [00:07<00:26, 2862340.14tokens/s]
Shard 52:  25%|██▍       | 24531658/100000000 [00:07<00:27, 2749482.16tokens/s]
Shard 52:  25%|██▍       | 24853142/100000000 [00:07<00:26, 2867359.19tokens/s]
Shard 52:  25%|██▌       | 25177581/100000000 [00:07<00:25, 2940951.58tokens/s]
Shard 52:  25%|██▌       | 25475214/100000000 [00:07<00:25, 2880395.56tokens/s]
Shard 52:  26%|██▌       | 25766371/100000000 [00:07<00:26, 2772348.12tokens/s]
Shard 52:  26%|██▌       | 26136832/100000000 [00:07<00:24, 2996944.03tokens/s]
Shard 52:  26%|██▋       | 26453196/100000000 [00:08<00:24, 3035817.59tokens/s]
Shard 52:  27%|██▋       | 26758811/100000000 [00:08<00:24, 2935309.70tokens/s]
Shard 52:  27%|██▋       | 27054169/100000000 [00:08<00:25, 2844103.64tokens/s]
Shard 52:  28%|██▊       | 27585330/100000000 [00:08<00:20, 3513028.65tokens/s]
Shard 52:  28%|██▊       | 27941062/100000000 [00:08<00:21, 3427493.33tokens/s]
Shard 52:  28%|██▊       | 28287046/100000000 [00:08<00:23, 3075341.23tokens/s]
Shard 52:  29%|██▊       | 28602944/100000000 [00:08<00:26, 2726401.65tokens/s]
Shard 52:  29%|██▉       | 28886719/100000000 [00:08<00:27, 2560638.01tokens/s]
Shard 52:  29%|██▉       | 29161976/100000000 [00:09<00:27, 2608409.57tokens/s]
Shard 52:  30%|██▉       | 29552211/100000000 [00:09<00:23, 2940293.62tokens/s]
Shard 52:  30%|██▉       | 29883763/100000000 [00:09<00:23, 3031217.33tokens/s]
Shard 52:  30%|███       | 30276890/100000000 [00:09<00:21, 3276924.03tokens/s]
Shard 52:  31%|███       | 30660304/100000000 [00:09<00:20, 3406322.23tokens/s]
Shard 52:  31%|███       | 31129080/100000000 [00:09<00:18, 3770790.59tokens/s]
Shard 52:  32%|███▏      | 31511175/100000000 [00:09<00:19, 3575375.38tokens/s]
Shard 52:  32%|███▏      | 31941482/100000000 [00:09<00:18, 3775869.29tokens/s]
Shard 52:  32%|███▏      | 32324021/100000000 [00:09<00:21, 3077300.61tokens/s]
Shard 52:  33%|███▎      | 32655607/100000000 [00:10<00:23, 2904195.44tokens/s]
Shard 52:  33%|███▎      | 32980377/100000000 [00:10<00:23, 2906209.09tokens/s]
Shard 52:  33%|███▎      | 33284580/100000000 [00:10<00:23, 2805320.11tokens/s]
Shard 52:  34%|███▎      | 33594375/100000000 [00:10<00:23, 2868076.33tokens/s]
Shard 52:  34%|███▍      | 33973844/100000000 [00:10<00:21, 3113535.55tokens/s]
Shard 52:  34%|███▍      | 34315427/100000000 [00:10<00:20, 3186150.54tokens/s]
Shard 52:  35%|███▍      | 34639947/100000000 [00:10<00:23, 2770226.96tokens/s]
Shard 52:  35%|███▌      | 35238477/100000000 [00:10<00:17, 3603424.85tokens/s]
Shard 52:  36%|███▌      | 35621162/100000000 [00:10<00:18, 3495113.45tokens/s]
Shard 52:  36%|███▌      | 35997176/100000000 [00:11<00:20, 3090300.64tokens/s]
Shard 52:  37%|███▋      | 37449025/100000000 [00:11<00:10, 5931584.11tokens/s]
Shard 52:  38%|███▊      | 38110598/100000000 [00:11<00:10, 6022006.45tokens/s]
Shard 52:  39%|███▉      | 38763462/100000000 [00:11<00:10, 5808424.92tokens/s]
Shard 52:  39%|███▉      | 39403444/100000000 [00:11<00:10, 5966088.38tokens/s]
Shard 52:  40%|████      | 40027566/100000000 [00:11<00:11, 5275511.03tokens/s]
Shard 52:  41%|████      | 40591182/100000000 [00:11<00:11, 5035131.16tokens/s]
Shard 52:  41%|████      | 41116674/100000000 [00:11<00:12, 4637492.15tokens/s]
Shard 52:  42%|████▏     | 41599094/100000000 [00:12<00:12, 4654927.46tokens/s]
Shard 52:  42%|████▏     | 42077611/100000000 [00:12<00:13, 4355254.42tokens/s]
Shard 52:  43%|████▎     | 42722552/100000000 [00:12<00:11, 4877569.95tokens/s]
Shard 52:  43%|████▎     | 43226365/100000000 [00:12<00:12, 4708623.74tokens/s]
Shard 52:  44%|████▎     | 43708384/100000000 [00:12<00:12, 4608488.82tokens/s]
Shard 52:  44%|████▍     | 44177773/100000000 [00:12<00:12, 4413708.54tokens/s]
Shard 52:  45%|████▍     | 44627532/100000000 [00:12<00:12, 4434345.69tokens/s]
Shard 52:  45%|████▌     | 45075025/100000000 [00:12<00:12, 4325131.14tokens/s]
Shard 52:  46%|████▌     | 45510229/100000000 [00:13<00:14, 3845911.61tokens/s]
Shard 52:  46%|████▌     | 45989357/100000000 [00:13<00:13, 4088036.47tokens/s]
Shard 52:  46%|████▋     | 46408782/100000000 [00:13<00:14, 3782464.94tokens/s]
Shard 52:  47%|████▋     | 46802139/100000000 [00:13<00:13, 3808160.64tokens/s]
Shard 52:  47%|████▋     | 47204265/100000000 [00:13<00:13, 3849634.29tokens/s]
Shard 52:  48%|████▊     | 47605235/100000000 [00:13<00:14, 3642760.47tokens/s]
Shard 52:  48%|████▊     | 47989626/100000000 [00:13<00:14, 3622674.38tokens/s]
Shard 52:  48%|████▊     | 48356193/100000000 [00:13<00:14, 3563321.94tokens/s]
Shard 52:  49%|████▊     | 48715588/100000000 [00:13<00:15, 3392696.18tokens/s]
Shard 52:  49%|████▉     | 49255919/100000000 [00:14<00:12, 3934015.21tokens/s]
Shard 52:  50%|████▉     | 49656387/100000000 [00:14<00:13, 3811539.26tokens/s]
Shard 52:  50%|█████     | 50043803/100000000 [00:14<00:13, 3827785.08tokens/s]
Shard 52:  50%|█████     | 50430103/100000000 [00:14<00:14, 3493821.08tokens/s]
Shard 52:  51%|█████     | 50857782/100000000 [00:14<00:13, 3661714.89tokens/s]
Shard 52:  51%|█████     | 51243405/100000000 [00:14<00:13, 3713110.92tokens/s]
Shard 52:  52%|█████▏    | 51619662/100000000 [00:14<00:14, 3450778.87tokens/s]
Shard 52:  52%|█████▏    | 51980310/100000000 [00:14<00:13, 3469316.57tokens/s]
Shard 52:  52%|█████▏    | 52475368/100000000 [00:14<00:12, 3880951.78tokens/s]
Shard 52:  53%|█████▎    | 52875447/100000000 [00:15<00:12, 3910742.46tokens/s]
Shard 52:  53%|█████▎    | 53271498/100000000 [00:15<00:12, 3802941.87tokens/s]
Shard 52:  54%|█████▍    | 53801574/100000000 [00:15<00:10, 4222475.19tokens/s]
Shard 52:  54%|█████▍    | 54283380/100000000 [00:15<00:10, 4393485.18tokens/s]
Shard 52:  55%|█████▍    | 54727278/100000000 [00:15<00:11, 3943024.57tokens/s]
Shard 52:  55%|█████▌    | 55134394/100000000 [00:15<00:13, 3443687.31tokens/s]
Shard 52:  55%|█████▌    | 55498573/100000000 [00:15<00:13, 3235749.31tokens/s]
Shard 52:  56%|█████▌    | 55877033/100000000 [00:15<00:13, 3364350.87tokens/s]
Shard 52:  56%|█████▌    | 56235108/100000000 [00:15<00:14, 3123750.04tokens/s]
Shard 52:  57%|█████▋    | 56626314/100000000 [00:16<00:13, 3284030.85tokens/s]
Shard 52:  57%|█████▋    | 56965230/100000000 [00:16<00:14, 2990570.17tokens/s]
Shard 52:  57%|█████▋    | 57274169/100000000 [00:16<00:14, 2919340.41tokens/s]
Shard 52:  58%|█████▊    | 57589765/100000000 [00:16<00:14, 2978726.67tokens/s]
Shard 52:  58%|█████▊    | 57892810/100000000 [00:16<00:15, 2671748.54tokens/s]
Shard 52:  58%|█████▊    | 58196563/100000000 [00:16<00:15, 2734932.91tokens/s]
Shard 52:  59%|█████▊    | 58514679/100000000 [00:16<00:14, 2829062.67tokens/s]
Shard 52:  59%|█████▉    | 58803097/100000000 [00:16<00:15, 2675103.84tokens/s]
Shard 52:  59%|█████▉    | 59075276/100000000 [00:17<00:19, 2151129.91tokens/s]
Shard 52:  59%|█████▉    | 59414934/100000000 [00:17<00:16, 2431627.19tokens/s]
Shard 52:  60%|█████▉    | 59735943/100000000 [00:17<00:15, 2605893.63tokens/s]
Shard 52:  60%|██████    | 60029461/100000000 [00:17<00:14, 2690271.75tokens/s]
Shard 52:  60%|██████    | 60363064/100000000 [00:17<00:13, 2859658.14tokens/s]
Shard 52:  61%|██████    | 60659758/100000000 [00:17<00:14, 2704716.76tokens/s]
Shard 52:  61%|██████    | 61008988/100000000 [00:17<00:13, 2913915.73tokens/s]
Shard 52:  61%|██████▏   | 61410380/100000000 [00:17<00:11, 3215835.43tokens/s]
Shard 52:  62%|██████▏   | 61739732/100000000 [00:17<00:12, 3005444.33tokens/s]
Shard 52:  62%|██████▏   | 62190898/100000000 [00:18<00:11, 3418395.95tokens/s]
Shard 52:  63%|██████▎   | 62575240/100000000 [00:18<00:10, 3536266.79tokens/s]
Shard 52:  63%|██████▎   | 62946221/100000000 [00:18<00:10, 3498258.30tokens/s]
Shard 52:  63%|██████▎   | 63410130/100000000 [00:18<00:09, 3817156.04tokens/s]
Shard 52:  64%|██████▍   | 63797222/100000000 [00:18<00:09, 3809850.82tokens/s]
Shard 52:  64%|██████▍   | 64248766/100000000 [00:18<00:08, 4015306.64tokens/s]
Shard 52:  65%|██████▍   | 64654073/100000000 [00:18<00:09, 3847473.43tokens/s]
Shard 52:  65%|██████▌   | 65042313/100000000 [00:18<00:09, 3690913.24tokens/s]
Shard 52:  65%|██████▌   | 65498089/100000000 [00:18<00:08, 3932357.43tokens/s]
Shard 52:  66%|██████▌   | 65895418/100000000 [00:19<00:08, 3900543.34tokens/s]
Shard 52:  66%|██████▋   | 66289367/100000000 [00:19<00:09, 3390261.93tokens/s]
Shard 52:  67%|██████▋   | 66641376/100000000 [00:19<00:10, 3271593.33tokens/s]
Shard 52:  67%|██████▋   | 67111016/100000000 [00:19<00:09, 3640329.88tokens/s]
Shard 52:  67%|██████▋   | 67486064/100000000 [00:19<00:09, 3420150.48tokens/s]
Shard 52:  68%|██████▊   | 67838526/100000000 [00:19<00:09, 3311979.15tokens/s]
Shard 52:  68%|██████▊   | 68336455/100000000 [00:19<00:08, 3754677.17tokens/s]
Shard 52:  69%|██████▊   | 68721723/100000000 [00:19<00:08, 3525970.37tokens/s]
Shard 52:  69%|██████▉   | 69082809/100000000 [00:20<00:10, 3084942.13tokens/s]
Shard 52:  69%|██████▉   | 69487981/100000000 [00:20<00:09, 3325750.88tokens/s]
Shard 52:  70%|██████▉   | 69835703/100000000 [00:20<00:10, 2901075.45tokens/s]
Shard 52:  70%|███████   | 70223794/100000000 [00:20<00:09, 3133138.62tokens/s]
Shard 52:  71%|███████   | 70608803/100000000 [00:20<00:08, 3296100.52tokens/s]
Shard 52:  71%|███████   | 70954242/100000000 [00:20<00:08, 3275866.15tokens/s]
Shard 52:  72%|███████▏  | 71533492/100000000 [00:20<00:07, 3956367.92tokens/s]
Shard 52:  72%|███████▏  | 71942350/100000000 [00:20<00:07, 3810279.46tokens/s]
Shard 52:  72%|███████▏  | 72333389/100000000 [00:20<00:08, 3378192.32tokens/s]
Shard 52:  73%|███████▎  | 72685776/100000000 [00:21<00:09, 2872883.91tokens/s]
Shard 52:  73%|███████▎  | 73013606/100000000 [00:21<00:09, 2962494.71tokens/s]
Shard 52:  73%|███████▎  | 73354371/100000000 [00:21<00:08, 3068951.83tokens/s]
Shard 52:  74%|███████▍  | 73765856/100000000 [00:21<00:07, 3343699.71tokens/s]
Shard 52:  74%|███████▍  | 74113532/100000000 [00:21<00:08, 3206256.61tokens/s]
Shard 52:  74%|███████▍  | 74444962/100000000 [00:21<00:08, 2990745.11tokens/s]
Shard 52:  75%|███████▍  | 74753070/100000000 [00:21<00:08, 2863625.16tokens/s]
Shard 52:  75%|███████▌  | 75204144/100000000 [00:21<00:07, 3297307.10tokens/s]
Shard 52:  82%|████████▏ | 81611713/100000000 [00:21<00:00, 19912368.72tokens/s]
Shard 52: 100%|█████████▉| 99999657/100000000 [00:30<00:00, 3301955.65tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000052.bin (200,001,024 bytes) in the gpt-2 format

Shard 53:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 53:  23%|██▎       | 23103573/100000000 [00:00<00:00, 231032645.38tokens/s]
Shard 53:  52%|█████▏    | 52153355/100000000 [00:00<00:00, 266011370.65tokens/s]
Shard 53:  82%|████████▏ | 82287826/100000000 [00:00<00:00, 282144141.91tokens/s]
Shard 53: 100%|█████████▉| 99999810/100000000 [00:08<00:00, 11485895.06tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000053.bin (200,001,024 bytes) in the gpt-2 format

Shard 54:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 54:  24%|██▍       | 24385631/100000000 [00:00<00:00, 243844915.12tokens/s]
Shard 54:  52%|█████▏    | 51794682/100000000 [00:00<00:00, 261635879.03tokens/s]
Shard 54:  81%|████████  | 80792984/100000000 [00:00<00:00, 274575572.47tokens/s]
Shard 54: 100%|█████████▉| 99992797/100000000 [00:08<00:00, 11561430.86tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000054.bin (200,001,024 bytes) in the gpt-2 format

Shard 55:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 55:  25%|██▍       | 24740930/100000000 [00:00<00:00, 247390661.52tokens/s]
Shard 55:  52%|█████▏    | 51742781/100000000 [00:00<00:00, 260700528.35tokens/s]
Shard 55:  79%|███████▉  | 79257419/100000000 [00:00<00:00, 267293899.93tokens/s]
Shard 55: 100%|█████████▉| 99999829/100000000 [00:08<00:00, 11585892.67tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000055.bin (200,001,024 bytes) in the gpt-2 format

Shard 56:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 56:  25%|██▌       | 25245371/100000000 [00:00<00:00, 252449737.55tokens/s]
Shard 56:  50%|█████     | 50491447/100000000 [00:00<00:00, 240317726.91tokens/s]
Shard 56:  76%|███████▌  | 76186779/100000000 [00:00<00:00, 247710525.26tokens/s]
Shard 56: 100%|█████████▉| 99999879/100000000 [00:08<00:00, 11595633.20tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000056.bin (200,001,024 bytes) in the gpt-2 format

Shard 57:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 57:  25%|██▌       | 25332541/100000000 [00:00<00:00, 253324443.64tokens/s]
Shard 57:  53%|█████▎    | 53080395/100000000 [00:00<00:00, 267525359.51tokens/s]
Shard 57:  80%|████████  | 80183223/100000000 [00:00<00:00, 269122907.65tokens/s]
Shard 57: 100%|█████████▉| 99999584/100000000 [00:08<00:00, 11835405.29tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000057.bin (200,001,024 bytes) in the gpt-2 format

Shard 58:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 58:  25%|██▍       | 24828346/100000000 [00:00<00:00, 248279553.17tokens/s]
Shard 58:  51%|█████     | 51107122/100000000 [00:00<00:00, 256810639.77tokens/s]
Shard 58:  79%|███████▊  | 78642114/100000000 [00:00<00:00, 265272629.27tokens/s]
Shard 58: 100%|█████████▉| 99998870/100000000 [00:08<00:00, 11379019.73tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000058.bin (200,001,024 bytes) in the gpt-2 format

Shard 59:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 59:  28%|██▊       | 27706935/100000000 [00:00<00:00, 277068293.07tokens/s]
Shard 59:  57%|█████▋    | 56997080/100000000 [00:00<00:00, 286374028.25tokens/s]
Shard 59:  86%|████████▌ | 85634858/100000000 [00:00<00:00, 273415398.89tokens/s]
Shard 59: 100%|█████████▉| 99998106/100000000 [00:08<00:00, 11314153.73tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000059.bin (200,001,024 bytes) in the gpt-2 format

Shard 60:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 60:  24%|██▍       | 23783526/100000000 [00:00<00:00, 237832651.63tokens/s]
Shard 60:  48%|████▊     | 48249688/100000000 [00:00<00:00, 241840312.60tokens/s]
Shard 60:  75%|███████▍  | 74649952/100000000 [00:00<00:00, 251959145.96tokens/s]
Shard 60: 100%|█████████▉| 99999491/100000000 [00:08<00:00, 11187224.17tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000060.bin (200,001,024 bytes) in the gpt-2 format

Shard 61:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 61:  25%|██▍       | 24724615/100000000 [00:00<00:00, 247242848.95tokens/s]
Shard 61:  52%|█████▏    | 52448599/100000000 [00:00<00:00, 264882617.10tokens/s]
Shard 61:  80%|████████  | 80024685/100000000 [00:00<00:00, 269848136.08tokens/s]
Shard 61: 100%|█████████▉| 99999631/100000000 [00:08<00:00, 11276622.14tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000061.bin (200,001,024 bytes) in the gpt-2 format

Shard 62:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 62:  25%|██▍       | 24929378/100000000 [00:00<00:00, 249289857.27tokens/s]
Shard 62:  53%|█████▎    | 52584391/100000000 [00:00<00:00, 265320891.56tokens/s]
Shard 62:  81%|████████  | 80813121/100000000 [00:00<00:00, 273067241.34tokens/s]
Shard 62: 100%|█████████▉| 99999099/100000000 [00:08<00:00, 11365844.78tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000062.bin (200,001,024 bytes) in the gpt-2 format

Shard 63:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 63:  28%|██▊       | 28444922/100000000 [00:00<00:00, 284444065.93tokens/s]
Shard 63:  59%|█████▊    | 58625431/100000000 [00:00<00:00, 294649910.41tokens/s]
Shard 63:  88%|████████▊ | 88090544/100000000 [00:00<00:00, 290973853.27tokens/s]
Shard 63: 100%|█████████▉| 99997321/100000000 [00:08<00:00, 11396919.21tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000063.bin (200,001,024 bytes) in the gpt-2 format

Shard 64:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 64:  26%|██▌       | 25983159/100000000 [00:00<00:00, 259828740.39tokens/s]
Shard 64:  54%|█████▍    | 53902356/100000000 [00:00<00:00, 271218595.80tokens/s]
Shard 64:  83%|████████▎ | 82557716/100000000 [00:00<00:00, 278213409.55tokens/s]
Shard 64: 100%|█████████▉| 99999940/100000000 [00:08<00:00, 11429371.79tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000064.bin (200,001,024 bytes) in the gpt-2 format

Shard 65:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 65:  23%|██▎       | 22936503/100000000 [00:00<00:00, 229360327.20tokens/s]
Shard 65:  51%|█████     | 50635534/100000000 [00:00<00:00, 257375345.28tokens/s]
Shard 65:  78%|███████▊  | 77823888/100000000 [00:00<00:00, 263998418.55tokens/s]
Shard 65: 100%|█████████▉| 99999958/100000000 [00:08<00:00, 11311533.76tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000065.bin (200,001,024 bytes) in the gpt-2 format

Shard 66:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 66:  25%|██▌       | 25298749/100000000 [00:00<00:00, 252987128.10tokens/s]
Shard 66:  52%|█████▏    | 52389943/100000000 [00:00<00:00, 263530537.54tokens/s]
Shard 66:  80%|████████  | 80180307/100000000 [00:00<00:00, 270093411.50tokens/s]
Shard 66: 100%|█████████▉| 99999782/100000000 [00:08<00:00, 11303056.95tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000066.bin (200,001,024 bytes) in the gpt-2 format

Shard 67:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 67:  24%|██▍       | 24012658/100000000 [00:00<00:00, 240121656.55tokens/s]
Shard 67:  51%|█████     | 50903504/100000000 [00:00<00:00, 257054711.07tokens/s]
Shard 67:  78%|███████▊  | 78219898/100000000 [00:00<00:00, 264408298.59tokens/s]
Shard 67: 100%|█████████▉| 99999613/100000000 [00:08<00:00, 11387698.29tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000067.bin (200,001,024 bytes) in the gpt-2 format

Shard 68:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 68:  26%|██▋       | 26313314/100000000 [00:00<00:00, 263132136.23tokens/s]
Shard 68:  54%|█████▍    | 54437277/100000000 [00:00<00:00, 273781436.20tokens/s]
Shard 68:  83%|████████▎ | 82523916/100000000 [00:00<00:00, 277012155.39tokens/s]
Shard 68: 100%|█████████▉| 99999497/100000000 [00:08<00:00, 11557022.43tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000068.bin (200,001,024 bytes) in the gpt-2 format

Shard 69:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 69:  24%|██▍       | 23899381/100000000 [00:00<00:00, 238983781.84tokens/s]
Shard 69:  50%|█████     | 50295514/100000000 [00:00<00:00, 253673855.34tokens/s]
Shard 69:  78%|███████▊  | 78209333/100000000 [00:00<00:00, 265300775.65tokens/s]
Shard 69: 100%|█████████▉| 99998839/100000000 [00:08<00:00, 11361696.96tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000069.bin (200,001,024 bytes) in the gpt-2 format

Shard 70:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 70:  23%|██▎       | 23145297/100000000 [00:00<00:00, 231449879.81tokens/s]
Shard 70:  52%|█████▏    | 51805997/100000000 [00:00<00:00, 263893733.90tokens/s]
Shard 70:  83%|████████▎ | 82764566/100000000 [00:00<00:00, 284755475.21tokens/s]
Shard 70: 100%|█████████▉| 99987271/100000000 [00:08<00:00, 11528775.86tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000070.bin (200,001,024 bytes) in the gpt-2 format

Shard 71:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 71:  29%|██▉       | 28814116/100000000 [00:00<00:00, 288140060.83tokens/s]
Shard 71:  60%|█████▉    | 59954202/100000000 [00:00<00:00, 301821761.82tokens/s]
Shard 71:  90%|█████████ | 90136907/100000000 [00:00<00:00, 297880584.75tokens/s]
Shard 71: 100%|█████████▉| 99999877/100000000 [00:08<00:00, 11430223.93tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000071.bin (200,001,024 bytes) in the gpt-2 format

Shard 72:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 72:  26%|██▌       | 25567775/100000000 [00:00<00:00, 255675555.52tokens/s]
Shard 72:  53%|█████▎    | 52994842/100000000 [00:00<00:00, 266613595.15tokens/s]
Shard 72:  80%|███████▉  | 79709963/100000000 [00:00<00:00, 266856872.86tokens/s]
Shard 72: 100%|█████████▉| 99998147/100000000 [00:08<00:00, 11337675.33tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000072.bin (200,001,024 bytes) in the gpt-2 format

Shard 73:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 73:  27%|██▋       | 26672440/100000000 [00:00<00:00, 266724018.45tokens/s]
Shard 73:  57%|█████▋    | 57278146/100000000 [00:00<00:00, 289860031.14tokens/s]
Shard 73:  86%|████████▋ | 86267377/100000000 [00:00<00:00, 281072062.37tokens/s]
Shard 73: 100%|█████████▉| 99999965/100000000 [00:08<00:00, 11383743.59tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000073.bin (200,001,024 bytes) in the gpt-2 format

Shard 74:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 74:  25%|██▌       | 25018469/100000000 [00:00<00:00, 250180753.25tokens/s]
Shard 74:  50%|█████     | 50037494/100000000 [00:00<00:00, 239525929.19tokens/s]
Shard 74:  77%|███████▋  | 77377780/100000000 [00:00<00:00, 254628555.25tokens/s]
Shard 74: 100%|█████████▉| 99999896/100000000 [00:08<00:00, 11415644.68tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000074.bin (200,001,024 bytes) in the gpt-2 format

Shard 75:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 75:  25%|██▍       | 24617459/100000000 [00:00<00:00, 246172477.08tokens/s]
Shard 75:  49%|████▉     | 49322739/100000000 [00:00<00:00, 246683531.39tokens/s]
Shard 75:  76%|███████▌  | 75622222/100000000 [00:00<00:00, 254131044.74tokens/s]
Shard 75: 100%|█████████▉| 99999110/100000000 [00:08<00:00, 11436799.22tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000075.bin (200,001,024 bytes) in the gpt-2 format

Shard 76:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 76:  24%|██▍       | 23859226/100000000 [00:00<00:00, 238448655.41tokens/s]
Shard 76:  51%|█████     | 50906393/100000000 [00:00<00:00, 257277950.39tokens/s]
Shard 76:  79%|███████▊  | 78570144/100000000 [00:00<00:00, 266113033.19tokens/s]
Shard 76: 100%|█████████▉| 99999612/100000000 [00:08<00:00, 11491258.12tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000076.bin (200,001,024 bytes) in the gpt-2 format

Shard 77:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 77:  26%|██▋       | 26399044/100000000 [00:00<00:00, 263987544.78tokens/s]
Shard 77:  57%|█████▋    | 57049871/100000000 [00:00<00:00, 288998164.03tokens/s]
Shard 77:  88%|████████▊ | 87723843/100000000 [00:00<00:00, 297097477.77tokens/s]
Shard 77: 100%|█████████▉| 99999973/100000000 [00:08<00:00, 11551366.07tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000077.bin (200,001,024 bytes) in the gpt-2 format

Shard 78:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 78:  26%|██▋       | 26288028/100000000 [00:00<00:00, 262878023.70tokens/s]
Shard 78:  55%|█████▌    | 55433218/100000000 [00:00<00:00, 279677262.31tokens/s]
Shard 78:  84%|████████▍ | 84006037/100000000 [00:00<00:00, 282439698.13tokens/s]
Shard 78: 100%|█████████▉| 99999677/100000000 [00:08<00:00, 11530488.41tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000078.bin (200,001,024 bytes) in the gpt-2 format

Shard 79:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 79:  25%|██▌       | 25078518/100000000 [00:00<00:00, 250781831.70tokens/s]
Shard 79:  52%|█████▏    | 51676872/100000000 [00:00<00:00, 259719738.65tokens/s]
Shard 79:  79%|███████▉  | 78921074/100000000 [00:00<00:00, 265323099.39tokens/s]
Shard 79: 100%|█████████▉| 99998455/100000000 [00:08<00:00, 11703192.59tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000079.bin (200,001,024 bytes) in the gpt-2 format

Shard 80:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 80:  24%|██▍       | 24129642/100000000 [00:00<00:00, 241295499.53tokens/s]
Shard 80:  51%|█████     | 51061414/100000000 [00:00<00:00, 257771685.54tokens/s]
Shard 80:  77%|███████▋  | 76838552/100000000 [00:00<00:00, 256663768.03tokens/s]
Shard 80: 100%|█████████▉| 99999874/100000000 [00:08<00:00, 11526921.44tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000080.bin (200,001,024 bytes) in the gpt-2 format

Shard 81:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 81:  25%|██▌       | 25274279/100000000 [00:00<00:00, 252730979.87tokens/s]
Shard 81:  53%|█████▎    | 52518152/100000000 [00:00<00:00, 264319992.36tokens/s]
Shard 81:  81%|████████  | 80590813/100000000 [00:00<00:00, 271800339.79tokens/s]
Shard 81: 100%|█████████▉| 99999895/100000000 [00:08<00:00, 11528846.45tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000081.bin (200,001,024 bytes) in the gpt-2 format

Shard 82:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 82:  24%|██▍       | 23854145/100000000 [00:00<00:00, 238515518.86tokens/s]
Shard 82:  50%|████▉     | 49950175/100000000 [00:00<00:00, 251715763.44tokens/s]
Shard 82:  77%|███████▋  | 77179135/100000000 [00:00<00:00, 261108348.94tokens/s]
Shard 82: 100%|█████████▉| 99998474/100000000 [00:08<00:00, 11565306.66tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000082.bin (200,001,024 bytes) in the gpt-2 format

Shard 83:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 83:  23%|██▎       | 23061183/100000000 [00:00<00:00, 230598854.94tokens/s]
Shard 83:  48%|████▊     | 47867100/100000000 [00:00<00:00, 240868512.27tokens/s]
Shard 83:  74%|███████▍  | 73850557/100000000 [00:00<00:00, 249519016.04tokens/s]
Shard 83: 100%|█████████▉| 99985188/100000000 [00:08<00:00, 12419524.60tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000083.bin (200,001,024 bytes) in the gpt-2 format

Shard 84:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 84:  25%|██▍       | 24763131/100000000 [00:00<00:00, 247627413.43tokens/s]
Shard 84:  52%|█████▏    | 51733695/100000000 [00:00<00:00, 260610647.14tokens/s]
Shard 84:  79%|███████▊  | 78662823/100000000 [00:00<00:00, 264570731.33tokens/s]
Shard 84: 100%|█████████▉| 99993539/100000000 [00:08<00:00, 11788758.93tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000084.bin (200,001,024 bytes) in the gpt-2 format

Shard 85:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 85:  24%|██▍       | 24182459/100000000 [00:00<00:00, 241817325.63tokens/s]
Shard 85:  51%|█████▏    | 51343643/100000000 [00:00<00:00, 259343075.53tokens/s]
Shard 85:  79%|███████▉  | 79001035/100000000 [00:00<00:00, 267210228.22tokens/s]
Shard 85: 100%|█████████▉| 99999989/100000000 [00:08<00:00, 11723147.30tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000085.bin (200,001,024 bytes) in the gpt-2 format

Shard 86:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 86:  25%|██▍       | 24791735/100000000 [00:00<00:00, 247914631.06tokens/s]
Shard 86:  52%|█████▏    | 52243388/100000000 [00:00<00:00, 263560296.67tokens/s]
Shard 86:  79%|███████▊  | 78599649/100000000 [00:00<00:00, 261165495.89tokens/s]
Shard 86: 100%|█████████▉| 99995434/100000000 [00:08<00:00, 12135172.61tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000086.bin (200,001,024 bytes) in the gpt-2 format

Shard 87:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 87:  24%|██▍       | 24421879/100000000 [00:00<00:00, 244216111.62tokens/s]
Shard 87:  52%|█████▏    | 51540134/100000000 [00:00<00:00, 260077702.45tokens/s]
Shard 87:  80%|████████  | 80096579/100000000 [00:00<00:00, 271713186.99tokens/s]
Shard 87: 100%|█████████▉| 99999863/100000000 [00:08<00:00, 11637487.40tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000087.bin (200,001,024 bytes) in the gpt-2 format

Shard 88:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 88:  26%|██▌       | 26075560/100000000 [00:00<00:00, 260748388.60tokens/s]
Shard 88:  56%|█████▌    | 55648569/100000000 [00:00<00:00, 281323810.79tokens/s]
Shard 88:  84%|████████▍ | 84365714/100000000 [00:00<00:00, 283987261.83tokens/s]
Shard 88: 100%|█████████▉| 99998930/100000000 [00:08<00:00, 11693309.41tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000088.bin (200,001,024 bytes) in the gpt-2 format

Shard 89:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 89:  25%|██▍       | 24749237/100000000 [00:00<00:00, 247485525.40tokens/s]
Shard 89:  51%|█████▏    | 51482736/100000000 [00:00<00:00, 259160966.55tokens/s]
Shard 89:  79%|███████▉  | 79308193/100000000 [00:00<00:00, 267878975.75tokens/s]
Shard 89: 100%|█████████▉| 99999102/100000000 [00:08<00:00, 12059328.98tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000089.bin (200,001,024 bytes) in the gpt-2 format

Shard 90:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 90:  26%|██▌       | 25526970/100000000 [00:00<00:00, 255266900.42tokens/s]
Shard 90:  54%|█████▍    | 54381936/100000000 [00:00<00:00, 274843903.69tokens/s]
Shard 90:  84%|████████▍ | 83872097/100000000 [00:00<00:00, 284001520.02tokens/s]
Shard 90: 100%|█████████▉| 99999112/100000000 [00:08<00:00, 11802107.60tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000090.bin (200,001,024 bytes) in the gpt-2 format

Shard 91:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 91:  29%|██▉       | 28971481/100000000 [00:00<00:00, 289708869.81tokens/s]
Shard 91:  61%|██████    | 60659967/100000000 [00:00<00:00, 305694354.19tokens/s]
Shard 91:  91%|█████████ | 91229679/100000000 [00:00<00:00, 295159667.72tokens/s]
Shard 91: 100%|█████████▉| 99994603/100000000 [00:08<00:00, 11607674.81tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000091.bin (200,001,024 bytes) in the gpt-2 format

Shard 92:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 92:  23%|██▎       | 22982426/100000000 [00:00<00:00, 229823931.23tokens/s]
Shard 92:  50%|█████     | 50199087/100000000 [00:00<00:00, 254731160.31tokens/s]
Shard 92:  77%|███████▋  | 76738666/100000000 [00:00<00:00, 259600398.26tokens/s]
Shard 92: 100%|█████████▉| 99997359/100000000 [00:08<00:00, 11742974.83tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000092.bin (200,001,024 bytes) in the gpt-2 format

Shard 93:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 93:  26%|██▋       | 26391990/100000000 [00:00<00:00, 263896242.92tokens/s]
Shard 93:  55%|█████▌    | 55350502/100000000 [00:00<00:00, 279001859.25tokens/s]
Shard 93:  84%|████████▍ | 83801707/100000000 [00:00<00:00, 281513379.06tokens/s]
Shard 93: 100%|█████████▉| 99999602/100000000 [00:08<00:00, 12011820.69tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000093.bin (200,001,024 bytes) in the gpt-2 format

Shard 94:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 94:  25%|██▌       | 25334351/100000000 [00:00<00:00, 253335295.62tokens/s]
Shard 94:  53%|█████▎    | 53393089/100000000 [00:00<00:00, 269364359.50tokens/s]
Shard 94:  82%|████████▏ | 82085182/100000000 [00:00<00:00, 277379287.16tokens/s]
Shard 94: 100%|█████████▉| 99999467/100000000 [00:08<00:00, 12033011.63tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000094.bin (200,001,024 bytes) in the gpt-2 format

Shard 95:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 95:  25%|██▍       | 24591620/100000000 [00:00<00:00, 245912916.71tokens/s]
Shard 95:  51%|█████     | 50659004/100000000 [00:00<00:00, 254592337.31tokens/s]
Shard 95:  77%|███████▋  | 76591548/100000000 [00:00<00:00, 256748629.74tokens/s]
Shard 95: 100%|█████████▉| 99998259/100000000 [00:08<00:00, 12174003.71tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000095.bin (200,001,024 bytes) in the gpt-2 format

Shard 96:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 96:  26%|██▋       | 26292907/100000000 [00:00<00:00, 262927440.14tokens/s]
Shard 96:  56%|█████▌    | 56108448/100000000 [00:00<00:00, 283645505.77tokens/s]
Shard 96:  87%|████████▋ | 87289799/100000000 [00:00<00:00, 296503742.04tokens/s]
Shard 96: 100%|█████████▉| 99999540/100000000 [00:08<00:00, 12453193.71tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000096.bin (200,001,024 bytes) in the gpt-2 format

Shard 97:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 97:  27%|██▋       | 26994857/100000000 [00:00<00:00, 269946896.63tokens/s]
Shard 97:  59%|█████▉    | 58986106/100000000 [00:00<00:00, 299338095.24tokens/s]
Shard 97:  91%|█████████ | 90573565/100000000 [00:00<00:00, 306888123.50tokens/s]
Shard 97: 100%|█████████▉| 99999716/100000000 [00:08<00:00, 12034392.92tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000097.bin (200,001,024 bytes) in the gpt-2 format

Shard 98:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 98:  27%|██▋       | 26510311/100000000 [00:00<00:00, 265088826.33tokens/s]
Shard 98:  57%|█████▋    | 56646648/100000000 [00:00<00:00, 286425275.38tokens/s]
Shard 98:  85%|████████▌ | 85325904/100000000 [00:00<00:00, 286586243.84tokens/s]
Shard 98: 100%|█████████▉| 99999866/100000000 [00:08<00:00, 11945819.22tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000098.bin (200,001,024 bytes) in the gpt-2 format

Shard 99:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 99:  25%|██▍       | 24583784/100000000 [00:00<00:00, 245835729.98tokens/s]
Shard 99:  52%|█████▏    | 52039634/100000000 [00:00<00:00, 262728617.00tokens/s]
Shard 99:  78%|███████▊  | 78328419/100000000 [00:00<00:00, 262799437.31tokens/s]
Shard 99: 100%|█████████▉| 99999493/100000000 [00:08<00:00, 11908844.07tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000099.bin (200,001,024 bytes) in the gpt-2 format

Shard 100:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 100:  24%|██▍       | 24001199/100000000 [00:00<00:00, 240008785.54tokens/s]
Shard 100:  52%|█████▏    | 52255471/100000000 [00:00<00:00, 265028386.42tokens/s]
Shard 100:  80%|████████  | 80380918/100000000 [00:00<00:00, 272434684.22tokens/s]
Shard 100: 100%|█████████▉| 99999604/100000000 [00:08<00:00, 12219119.04tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000100.bin (200,001,024 bytes) in the gpt-2 format

Shard 101:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 101:  24%|██▎       | 23560423/100000000 [00:00<00:00, 235601646.10tokens/s]
Shard 101:  49%|████▉     | 49432896/100000000 [00:00<00:00, 249202490.04tokens/s]
Shard 101:  74%|███████▍  | 74353378/100000000 [00:00<00:00, 235514893.13tokens/s]
Shard 101: 100%|█████████▉| 99999883/100000000 [00:08<00:00, 12197306.75tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000101.bin (200,001,024 bytes) in the gpt-2 format

Shard 102:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 102:  24%|██▎       | 23529604/100000000 [00:00<00:00, 235283362.32tokens/s]
Shard 102:  50%|████▉     | 49764458/100000000 [00:00<00:00, 251201729.01tokens/s]
Shard 102:  78%|███████▊  | 77840521/100000000 [00:00<00:00, 264697398.20tokens/s]
Shard 102: 100%|█████████▉| 99999885/100000000 [00:07<00:00, 12913580.09tokens/s] 
writing 100,000,000 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000102.bin (200,001,024 bytes) in the gpt-2 format

Shard 103:   0%|          | 0/100000000 [00:00<?, ?tokens/s]
Shard 103:  23%|██▎       | 23432937/100000000 [00:00<00:00, 234327917.43tokens/s]
Shard 103:  51%|█████     | 51128169/100000000 [00:00<00:00, 259399357.95tokens/s]writing 55,324,043 tokens to /home/ubuntu/llm.c/dev/data/fineweb10B/fineweb_train_000103.bin (110,649,110 bytes) in the gpt-2 format

Shard 103:  55%|█████▌    | 55324038/100000000 [00:07<00:05, 7498607.99tokens/s]  
start Wed Nov 20 05:26:06 UTC 2024
+-----------------------+----------------------------------------------------+
| Parameter             | Value                                              |
+-----------------------+----------------------------------------------------+
| train data pattern    | dev/data/fineweb10B/fineweb_train_*.bin            |
| val data pattern      | dev/data/fineweb10B/fineweb_val_*.bin              |
| output log dir        | log124M                                            |
| checkpoint_every      | 5000                                               |
| resume                | 0                                                  |
| micro batch size B    | 64                                                 |
| sequence length T     | 1024                                               |
| total batch size      | 524288                                             |
| LR scheduler          | cosine                                             |
| learning rate (LR)    | 6.000000e-04                                       |
| warmup iterations     | 700                                                |
| final LR fraction     | 0.000000e+00                                       |
| weight decay          | 1.000000e-01                                       |
| skip update lossz     | 0.000000                                           |
| skip update gradz     | 0.000000                                           |
| max_steps             | -1                                                 |
| val_loss_every        | 250                                                |
| val_max_steps         | 20                                                 |
| sample_every          | 20000                                              |
| genT                  | 64                                                 |
| overfit_single_batch  | 0                                                  |
| use_master_weights    | enabled                                            |
| gelu_fusion           | 0                                                  |
| recompute             | 1                                                  |
+-----------------------+----------------------------------------------------+
| device                | NVIDIA A100-SXM4-40GB                              |
| peak TFlops           | 312.0                                              |
| precision             | BF16                                               |
+-----------------------+----------------------------------------------------+
| weight init method    | d12                                                |
| max_sequence_length T | 1024                                               |
| vocab_size V          | 50257                                              |
| padded_vocab_size Vp  | 50304                                              |
| num_layers L          | 12                                                 |
| num_heads NH          | 12                                                 |
| channels C            | 768                                                |
| num_parameters        | 124475904                                          |
+-----------------------+----------------------------------------------------+
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
| train_num_batches     | 19560                                              |
| val_num_batches       | 20                                                 |
+-----------------------+----------------------------------------------------+
| run hellaswag         | yes                                                |
+-----------------------+----------------------------------------------------+
| num_processes         | 8                                                  |
| zero_stage            | 1                                                  |
+-----------------------+----------------------------------------------------+
num_parameters: 124475904 => bytes: 248951808
allocated 237 MiB for model parameters
batch_size B=64 * seq_len T=1024 * num_processes=8 and total_batch_size=524288
=> setting grad_accum_steps=1
created directory: log124M
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
allocating 237 MiB for parameter gradients
allocating 21216 MiB for activations
allocating 59 MiB for AdamW optimizer state m
allocating 59 MiB for AdamW optimizer state v
allocating 59 MiB for master copy of params
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
device memory usage: 23129 MiB / 40326 MiB
memory per sequence: 331 MiB
 -> estimated maximum batch size: 115
val loss 11.009204
step    1/19560 | loss 11.009109 (+nanz)| norm 15.3717 (+nanz)| lr 8.57e-07 | 694.02 ms | 24.3% bf16 MFU | 755432 tok/s
step    2/19560 | loss 10.959894 (+nanz)| norm 15.2375 (+nanz)| lr 1.71e-06 | 314.23 ms | 53.7% bf16 MFU | 1668471 tok/s
step    3/19560 | loss 10.855607 (+nanz)| norm 14.8389 (+nanz)| lr 2.57e-06 | 314.03 ms | 53.7% bf16 MFU | 1669030 tok/s
step    4/19560 | loss 10.715733 (+nanz)| norm 13.1195 (+nanz)| lr 3.43e-06 | 315.18 ms | 53.5% bf16 MFU | 1667084 tok/s
step    5/19560 | loss 10.568422 (+nanz)| norm 10.4406 (+nanz)| lr 4.29e-06 | 314.67 ms | 53.6% bf16 MFU | 1666839 tok/s
step    6/19560 | loss 10.423462 (+nanz)| norm 8.4943 (+nanz)| lr 5.14e-06 | 314.80 ms | 53.6% bf16 MFU | 1666536 tok/s
step    7/19560 | loss 10.291794 (+nanz)| norm 7.2414 (+nanz)| lr 6.00e-06 | 314.67 ms | 53.6% bf16 MFU | 1666467 tok/s
step    8/19560 | loss 10.187210 (+nanz)| norm 6.1835 (+nanz)| lr 6.86e-06 | 313.99 ms | 53.8% bf16 MFU | 1667013 tok/s
step    9/19560 | loss 10.096747 (+nanz)| norm 5.2947 (+nanz)| lr 7.71e-06 | 314.87 ms | 53.6% bf16 MFU | 1666731 tok/s
step   10/19560 | loss 9.976242 (+nanz)| norm 4.6636 (+nanz)| lr 8.57e-06 | 313.82 ms | 53.8% bf16 MFU | 1667259 tok/s
step   11/19560 | loss 9.935363 (+nanz)| norm 3.8501 (+nanz)| lr 9.43e-06 | 314.47 ms | 53.7% bf16 MFU | 1667250 tok/s
step   12/19560 | loss 9.853886 (+nanz)| norm 3.3981 (+nanz)| lr 1.03e-05 | 314.46 ms | 53.7% bf16 MFU | 1667249 tok/s
step   13/19560 | loss 9.792469 (+nanz)| norm 3.0057 (+nanz)| lr 1.11e-05 | 314.79 ms | 53.6% bf16 MFU | 1667061 tok/s
step   14/19560 | loss 9.758893 (+nanz)| norm 2.6958 (+nanz)| lr 1.20e-05 | 314.43 ms | 53.7% bf16 MFU | 1667098 tok/s
step   15/19560 | loss 9.702515 (+nanz)| norm 2.5212 (+nanz)| lr 1.29e-05 | 314.33 ms | 53.7% bf16 MFU | 1667180 tok/s
step   16/19560 | loss 9.701741 (+nanz)| norm 2.3030 (+nanz)| lr 1.37e-05 | 314.86 ms | 53.6% bf16 MFU | 1666991 tok/s
step   17/19560 | loss 9.655608 (+nanz)| norm 2.2412 (+nanz)| lr 1.46e-05 | 314.08 ms | 53.7% bf16 MFU | 1667196 tok/s
step   18/19560 | loss 9.636610 (+nanz)| norm 2.1886 (+nanz)| lr 1.54e-05 | 314.75 ms | 53.6% bf16 MFU | 1667068 tok/s
step   19/19560 | loss 9.603570 (+nanz)| norm 2.2160 (+nanz)| lr 1.63e-05 | 314.42 ms | 53.7% bf16 MFU | 1667100 tok/s
step   20/19560 | loss 9.596474 (+nanz)| norm 2.1545 (+nanz)| lr 1.71e-05 | 314.56 ms | 53.7% bf16 MFU | 1667070 tok/s
step   21/19560 | loss 9.562174 (+nanz)| norm 2.1811 (+nanz)| lr 1.80e-05 | 314.04 ms | 53.7% bf16 MFU | 1667257 tok/s
step   22/19560 | loss 9.537735 (+nanz)| norm 2.1489 (+nanz)| lr 1.89e-05 | 315.56 ms | 53.5% bf16 MFU | 1666816 tok/s
step   23/19560 | loss 9.533236 (+nanz)| norm 2.0896 (+nanz)| lr 1.97e-05 | 315.00 ms | 53.6% bf16 MFU | 1666637 tok/s
step   24/19560 | loss 9.476654 (+nanz)| norm 2.1037 (+nanz)| lr 2.06e-05 | 314.74 ms | 53.6% bf16 MFU | 1666576 tok/s
step   25/19560 | loss 9.458330 (+nanz)| norm 2.0526 (+nanz)| lr 2.14e-05 | 315.12 ms | 53.6% bf16 MFU | 1666377 tok/s
step   26/19560 | loss 9.422844 (+nanz)| norm 2.0944 (+nanz)| lr 2.23e-05 | 315.20 ms | 53.5% bf16 MFU | 1666166 tok/s
step   27/19560 | loss 9.377932 (+nanz)| norm 2.1533 (+nanz)| lr 2.31e-05 | 315.00 ms | 53.6% bf16 MFU | 1666048 tok/s
step   28/19560 | loss 9.388620 (+nanz)| norm 1.9650 (+nanz)| lr 2.40e-05 | 315.06 ms | 53.6% bf16 MFU | 1665918 tok/s
step   29/19560 | loss 9.357556 (+nanz)| norm 2.4479 (+nanz)| lr 2.49e-05 | 316.05 ms | 53.4% bf16 MFU | 1665455 tok/s
step   30/19560 | loss 9.306395 (+nanz)| norm 1.9511 (+nanz)| lr 2.57e-05 | 314.97 ms | 53.6% bf16 MFU | 1665397 tok/s
step   31/19560 | loss 9.244958 (+nanz)| norm 2.2945 (+nanz)| lr 2.66e-05 | 315.26 ms | 53.5% bf16 MFU | 1665248 tok/s
step   32/19560 | loss 9.211145 (+nanz)| norm 2.2606 (+nanz)| lr 2.74e-05 | 315.26 ms | 53.5% bf16 MFU | 1665108 tok/s
step   33/19560 | loss 9.235586 (+nanz)| norm 2.5847 (+nanz)| lr 2.83e-05 | 316.22 ms | 53.4% bf16 MFU | 1664665 tok/s
step   34/19560 | loss 9.159502 (+nanz)| norm 2.4417 (+nanz)| lr 2.91e-05 | 315.45 ms | 53.5% bf16 MFU | 1664503 tok/s
step   35/19560 | loss 9.200704 (+nanz)| norm 1.9541 (+nanz)| lr 3.00e-05 | 315.33 ms | 53.5% bf16 MFU | 1664392 tok/s
step   36/19560 | loss 9.080119 (+nanz)| norm 2.4101 (+nanz)| lr 3.09e-05 | 316.02 ms | 53.4% bf16 MFU | 1664069 tok/s
step   37/19560 | loss 9.029286 (+nanz)| norm 1.8689 (+nanz)| lr 3.17e-05 | 315.40 ms | 53.5% bf16 MFU | 1663964 tok/s
step   38/19560 | loss 9.039859 (+nanz)| norm 1.9748 (+nanz)| lr 3.26e-05 | 315.42 ms | 53.5% bf16 MFU | 1663861 tok/s
step   39/19560 | loss 8.977991 (+nanz)| norm 1.8097 (+nanz)| lr 3.34e-05 | 317.05 ms | 53.2% bf16 MFU | 1663265 tok/s
step   40/19560 | loss 8.961273 (+nanz)| norm 2.0400 (+nanz)| lr 3.43e-05 | 315.88 ms | 53.4% bf16 MFU | 1663062 tok/s
step   41/19560 | loss 8.906675 (+nanz)| norm 1.8695 (+nanz)| lr 3.51e-05 | 316.29 ms | 53.4% bf16 MFU | 1662749 tok/s
step   42/19560 | loss 8.860172 (+nanz)| norm 1.6710 (+nanz)| lr 3.60e-05 | 316.54 ms | 53.3% bf16 MFU | 1662382 tok/s
step   43/19560 | loss 8.839678 (+nanz)| norm 1.7310 (+nanz)| lr 3.69e-05 | 315.59 ms | 53.5% bf16 MFU | 1662320 tok/s
step   44/19560 | loss 8.856478 (+nanz)| norm 1.6672 (+nanz)| lr 3.77e-05 | 316.37 ms | 53.3% bf16 MFU | 1662033 tok/s
step   45/19560 | loss 8.748159 (+nanz)| norm 1.6705 (+nanz)| lr 3.86e-05 | 316.51 ms | 53.3% bf16 MFU | 1661722 tok/s
step   46/19560 | loss 8.692660 (+nanz)| norm 1.6797 (+nanz)| lr 3.94e-05 | 316.05 ms | 53.4% bf16 MFU | 1661564 tok/s
step   47/19560 | loss 8.712541 (+nanz)| norm 1.5855 (+nanz)| lr 4.03e-05 | 316.11 ms | 53.4% bf16 MFU | 1661398 tok/s
step   48/19560 | loss 8.646988 (+nanz)| norm 1.5953 (+nanz)| lr 4.11e-05 | 316.75 ms | 53.3% bf16 MFU | 1661059 tok/s
step   49/19560 | loss 8.607601 (+nanz)| norm 1.5653 (+nanz)| lr 4.20e-05 | 316.04 ms | 53.4% bf16 MFU | 1660943 tok/s
step   50/19560 | loss 8.586852 (+nanz)| norm 1.5617 (+nanz)| lr 4.29e-05 | 316.96 ms | 53.2% bf16 MFU | 1660572 tok/s
step   51/19560 | loss 8.536360 (+nanz)| norm 1.6164 (+nanz)| lr 4.37e-05 | 316.65 ms | 53.3% bf16 MFU | 1660310 tok/s
step   52/19560 | loss 8.493007 (+nanz)| norm 1.7058 (+nanz)| lr 4.46e-05 | 316.76 ms | 53.3% bf16 MFU | 1660032 tok/s
step   53/19560 | loss 8.489213 (+nanz)| norm 1.7822 (+nanz)| lr 4.54e-05 | 316.78 ms | 53.3% bf16 MFU | 1659766 tok/s
step   54/19560 | loss 8.421474 (+nanz)| norm 1.5619 (+nanz)| lr 4.63e-05 | 317.03 ms | 53.2% bf16 MFU | 1659443 tok/s
step   55/19560 | loss 8.446768 (+nanz)| norm 1.4457 (+nanz)| lr 4.71e-05 | 316.30 ms | 53.4% bf16 MFU | 1659343 tok/s
step   56/19560 | loss 8.378421 (+nanz)| norm 1.6670 (+nanz)| lr 4.80e-05 | 317.06 ms | 53.2% bf16 MFU | 1659038 tok/s
step   57/19560 | loss 8.326710 (+nanz)| norm 1.6516 (+nanz)| lr 4.89e-05 | 316.82 ms | 53.3% bf16 MFU | 1658817 tok/s
step   58/19560 | loss 8.262535 (+nanz)| norm 1.6611 (+nanz)| lr 4.97e-05 | 316.83 ms | 53.3% bf16 MFU | 1658603 tok/s
step   59/19560 | loss 8.225919 (+nanz)| norm 1.7677 (+nanz)| lr 5.06e-05 | 316.05 ms | 53.4% bf16 MFU | 1658617 tok/s
step   60/19560 | loss 8.181098 (+nanz)| norm 1.4496 (+nanz)| lr 5.14e-05 | 317.47 ms | 53.2% bf16 MFU | 1658242 tok/s
step   61/19560 | loss 8.197227 (+nanz)| norm 1.7004 (+nanz)| lr 5.23e-05 | 317.43 ms | 53.2% bf16 MFU | 1657897 tok/s
step   62/19560 | loss 8.186023 (+nanz)| norm 1.3861 (+nanz)| lr 5.31e-05 | 317.40 ms | 53.2% bf16 MFU | 1657579 tok/s
step   63/19560 | loss 8.078353 (+nanz)| norm 1.3741 (+nanz)| lr 5.40e-05 | 317.43 ms | 53.2% bf16 MFU | 1657270 tok/s
step   64/19560 | loss 8.078118 (+nanz)| norm 1.4731 (+nanz)| lr 5.49e-05 | 317.50 ms | 53.2% bf16 MFU | 1656958 tok/s
step   65/19560 | loss 8.054794 (+nanz)| norm 1.4474 (+nanz)| lr 5.57e-05 | 317.05 ms | 53.2% bf16 MFU | 1656788 tok/s
step   66/19560 | loss 7.981112 (+nanz)| norm 1.6258 (+nanz)| lr 5.66e-05 | 316.69 ms | 53.3% bf16 MFU | 1656722 tok/s
step   67/19560 | loss 7.928320 (+nanz)| norm 1.3561 (+nanz)| lr 5.74e-05 | 317.43 ms | 53.2% bf16 MFU | 1656461 tok/s
step   68/19560 | loss 7.929897 (+nanz)| norm 1.3077 (+nanz)| lr 5.83e-05 | 317.57 ms | 53.1% bf16 MFU | 1656176 tok/s
step   69/19560 | loss 7.937150 (+nanz)| norm 1.2623 (+nanz)| lr 5.91e-05 | 317.29 ms | 53.2% bf16 MFU | 1655981 tok/s
step   70/19560 | loss 7.888319 (+nanz)| norm 1.2202 (+nanz)| lr 6.00e-05 | 317.32 ms | 53.2% bf16 MFU | 1655788 tok/s
step   71/19560 | loss 7.836940 (+nanz)| norm 1.3426 (+nanz)| lr 6.09e-05 | 317.24 ms | 53.2% bf16 MFU | 1655628 tok/s
step   72/19560 | loss 7.792568 (+nanz)| norm 1.1690 (+nanz)| lr 6.17e-05 | 316.99 ms | 53.2% bf16 MFU | 1655542 tok/s
step   73/19560 | loss 7.773895 (+nanz)| norm 1.1610 (+nanz)| lr 6.26e-05 | 317.74 ms | 53.1% bf16 MFU | 1655260 tok/s
step   74/19560 | loss 7.792548 (+nanz)| norm 1.4956 (+nanz)| lr 6.34e-05 | 317.85 ms | 53.1% bf16 MFU | 1654965 tok/s
step   75/19560 | loss 7.717053 (+nanz)| norm 1.3791 (+nanz)| lr 6.43e-05 | 318.58 ms | 53.0% bf16 MFU | 1654491 tok/s
step   76/19560 | loss 7.568538 (+nanz)| norm 1.0961 (+nanz)| lr 6.51e-05 | 318.38 ms | 53.0% bf16 MFU | 1654095 tok/s
step   77/19560 | loss 7.679242 (+nanz)| norm 1.7162 (+nanz)| lr 6.60e-05 | 317.44 ms | 53.2% bf16 MFU | 1653968 tok/s
step   78/19560 | loss 7.632957 (+nanz)| norm 0.9554 (+nanz)| lr 6.69e-05 | 318.68 ms | 53.0% bf16 MFU | 1653519 tok/s
step   79/19560 | loss 7.592442 (+nanz)| norm 1.2417 (+nanz)| lr 6.77e-05 | 318.08 ms | 53.1% bf16 MFU | 1653253 tok/s
step   80/19560 | loss 7.615833 (+nanz)| norm 0.9376 (+nanz)| lr 6.86e-05 | 318.47 ms | 53.0% bf16 MFU | 1652898 tok/s
step   81/19560 | loss 7.423759 (+nanz)| norm 1.4176 (+nanz)| lr 6.94e-05 | 318.70 ms | 53.0% bf16 MFU | 1652501 tok/s
step   82/19560 | loss 7.515634 (+nanz)| norm 1.5691 (+nanz)| lr 7.03e-05 | 318.52 ms | 53.0% bf16 MFU | 1652172 tok/s
step   83/19560 | loss 7.442426 (+nanz)| norm 0.9844 (+nanz)| lr 7.11e-05 | 318.12 ms | 53.1% bf16 MFU | 1651964 tok/s
step   84/19560 | loss 7.463370 (+nanz)| norm 1.0315 (+nanz)| lr 7.20e-05 | 318.07 ms | 53.1% bf16 MFU | 1651779 tok/s
step   85/19560 | loss 7.399691 (+nanz)| norm 1.3543 (+nanz)| lr 7.29e-05 | 319.53 ms | 52.8% bf16 MFU | 1651224 tok/s
step   86/19560 | loss 7.403323 (+nanz)| norm 0.9910 (+nanz)| lr 7.37e-05 | 318.51 ms | 53.0% bf16 MFU | 1650963 tok/s
step   87/19560 | loss 7.372291 (+nanz)| norm 0.9624 (+nanz)| lr 7.46e-05 | 318.49 ms | 53.0% bf16 MFU | 1650720 tok/s
step   88/19560 | loss 7.304594 (+nanz)| norm 1.0400 (+nanz)| lr 7.54e-05 | 318.36 ms | 53.0% bf16 MFU | 1650523 tok/s
step   89/19560 | loss 7.340849 (+nanz)| norm 1.0088 (+nanz)| lr 7.63e-05 | 318.75 ms | 52.9% bf16 MFU | 1650235 tok/s
step   90/19560 | loss 7.323218 (+nanz)| norm 0.9783 (+nanz)| lr 7.71e-05 | 318.66 ms | 53.0% bf16 MFU | 1649985 tok/s
step   91/19560 | loss 7.380167 (+nanz)| norm 0.8986 (+nanz)| lr 7.80e-05 | 318.62 ms | 53.0% bf16 MFU | 1649759 tok/s
step   92/19560 | loss 7.239905 (+nanz)| norm 1.3216 (+nanz)| lr 7.89e-05 | 319.01 ms | 52.9% bf16 MFU | 1649442 tok/s
step   93/19560 | loss 7.279544 (+nanz)| norm 1.1597 (+nanz)| lr 7.97e-05 | 318.16 ms | 53.0% bf16 MFU | 1649364 tok/s
step   94/19560 | loss 7.172609 (+nanz)| norm 0.6085 (+nanz)| lr 8.06e-05 | 318.64 ms | 53.0% bf16 MFU | 1649165 tok/s
step   95/19560 | loss 7.207014 (+nanz)| norm 1.1218 (+nanz)| lr 8.14e-05 | 319.32 ms | 52.9% bf16 MFU | 1648799 tok/s
step   96/19560 | loss 7.126600 (+nanz)| norm 1.1004 (+nanz)| lr 8.23e-05 | 319.01 ms | 52.9% bf16 MFU | 1648530 tok/s
step   97/19560 | loss 7.140619 (+nanz)| norm 0.7407 (+nanz)| lr 8.31e-05 | 319.12 ms | 52.9% bf16 MFU | 1648247 tok/s
step   98/19560 | loss 7.246284 (+nanz)| norm 0.8143 (+nanz)| lr 8.40e-05 | 319.28 ms | 52.9% bf16 MFU | 1647939 tok/s
step   99/19560 | loss 7.150189 (+nanz)| norm 0.9513 (+nanz)| lr 8.49e-05 | 320.06 ms | 52.7% bf16 MFU | 1647443 tok/s
step  100/19560 | loss 7.145782 (+nanz)| norm 0.7225 (+nanz)| lr 8.57e-05 | 319.18 ms | 52.9% bf16 MFU | 1647199 tok/s
step  101/19560 | loss 7.090214 (+nanz)| norm 0.9221 (+nanz)| lr 8.66e-05 | 319.96 ms | 52.7% bf16 MFU | 1646766 tok/s
step  102/19560 | loss 7.049090 (+nanz)| norm 0.8677 (+nanz)| lr 8.74e-05 | 319.36 ms | 52.8% bf16 MFU | 1646510 tok/s
step  103/19560 | loss 7.089582 (+nanz)| norm 0.9838 (+nanz)| lr 8.83e-05 | 319.40 ms | 52.8% bf16 MFU | 1646258 tok/s
step  104/19560 | loss 7.151857 (+nanz)| norm 1.0523 (+nanz)| lr 8.91e-05 | 319.61 ms | 52.8% bf16 MFU | 1645963 tok/s
step  105/19560 | loss 7.035015 (+nanz)| norm 0.7944 (+nanz)| lr 9.00e-05 | 319.10 ms | 52.9% bf16 MFU | 1645816 tok/s
step  106/19560 | loss 7.067171 (+nanz)| norm 0.8416 (+nanz)| lr 9.09e-05 | 320.25 ms | 52.7% bf16 MFU | 1645378 tok/s
step  107/19560 | loss 6.894728 (+nanz)| norm 0.9262 (+nanz)| lr 9.17e-05 | 319.67 ms | 52.8% bf16 MFU | 1645112 tok/s
step  108/19560 | loss 7.045878 (+nanz)| norm 0.8536 (+nanz)| lr 9.26e-05 | 319.66 ms | 52.8% bf16 MFU | 1644861 tok/s
step  109/19560 | loss 6.967838 (+nanz)| norm 0.8406 (+nanz)| lr 9.34e-05 | 319.72 ms | 52.8% bf16 MFU | 1644609 tok/s
step  110/19560 | loss 7.057088 (+nanz)| norm 0.9238 (+nanz)| lr 9.43e-05 | 319.93 ms | 52.8% bf16 MFU | 1644315 tok/s
step  111/19560 | loss 6.920825 (+nanz)| norm 1.0072 (+nanz)| lr 9.51e-05 | 320.17 ms | 52.7% bf16 MFU | 1643976 tok/s
step  112/19560 | loss 6.909135 (+nanz)| norm 1.0471 (+nanz)| lr 9.60e-05 | 320.07 ms | 52.7% bf16 MFU | 1643677 tok/s
step  113/19560 | loss 7.005341 (+nanz)| norm 0.7607 (+nanz)| lr 9.69e-05 | 320.24 ms | 52.7% bf16 MFU | 1643351 tok/s
step  114/19560 | loss 6.882371 (+nanz)| norm 1.3039 (+nanz)| lr 9.77e-05 | 322.02 ms | 52.4% bf16 MFU | 1642588 tok/s
step  115/19560 | loss 6.914182 (+nanz)| norm 1.3692 (+nanz)| lr 9.86e-05 | 319.71 ms | 52.8% bf16 MFU | 1642453 tok/s
step  116/19560 | loss 6.932863 (+nanz)| norm 0.8334 (+nanz)| lr 9.94e-05 | 322.12 ms | 52.4% bf16 MFU | 1641710 tok/s
step  117/19560 | loss 6.998663 (+nanz)| norm 1.9052 (+nanz)| lr 1.00e-04 | 321.63 ms | 52.5% bf16 MFU | 1641128 tok/s
step  118/19560 | loss 6.875816 (+nanz)| norm 0.8842 (+nanz)| lr 1.01e-04 | 320.07 ms | 52.7% bf16 MFU | 1640972 tok/s
step  119/19560 | loss 6.886818 (+nanz)| norm 1.1607 (+nanz)| lr 1.02e-04 | 319.87 ms | 52.8% bf16 MFU | 1640876 tok/s
step  120/19560 | loss 6.917952 (+nanz)| norm 1.0263 (+nanz)| lr 1.03e-04 | 320.55 ms | 52.7% bf16 MFU | 1640612 tok/s
step  121/19560 | loss 6.934100 (+nanz)| norm 1.0135 (+nanz)| lr 1.04e-04 | 320.85 ms | 52.6% bf16 MFU | 1640284 tok/s
step  122/19560 | loss 6.870656 (+nanz)| norm 1.0886 (+nanz)| lr 1.05e-04 | 320.78 ms | 52.6% bf16 MFU | 1639989 tok/s
step  123/19560 | loss 6.941378 (+nanz)| norm 0.9241 (+nanz)| lr 1.05e-04 | 320.65 ms | 52.6% bf16 MFU | 1639743 tok/s
step  124/19560 | loss 6.821346 (+nanz)| norm 0.9488 (+nanz)| lr 1.06e-04 | 320.34 ms | 52.7% bf16 MFU | 1639588 tok/s
step  125/19560 | loss 6.788586 (+nanz)| norm 1.2955 (+nanz)| lr 1.07e-04 | 320.52 ms | 52.7% bf16 MFU | 1639396 tok/s
step  126/19560 | loss 6.886657 (+nanz)| norm 0.6732 (+nanz)| lr 1.08e-04 | 320.43 ms | 52.7% bf16 MFU | 1639236 tok/s
step  127/19560 | loss 6.848351 (+nanz)| norm 0.7708 (+nanz)| lr 1.09e-04 | 320.73 ms | 52.6% bf16 MFU | 1639008 tok/s
step  128/19560 | loss 6.757625 (+nanz)| norm 0.9592 (+nanz)| lr 1.10e-04 | 322.94 ms | 52.3% bf16 MFU | 1638231 tok/s
step  129/19560 | loss 6.738553 (-1.32z)| norm 1.1891 (-0.37z)| lr 1.11e-04 | 320.51 ms | 52.7% bf16 MFU | 1638110 tok/s
step  130/19560 | loss 6.805323 (-1.25z)| norm 0.7411 (-0.59z)| lr 1.11e-04 | 320.87 ms | 52.6% bf16 MFU | 1637903 tok/s
step  131/19560 | loss 6.733213 (-1.31z)| norm 0.7565 (-0.64z)| lr 1.12e-04 | 320.75 ms | 52.6% bf16 MFU | 1637735 tok/s
step  132/19560 | loss 6.692756 (-1.34z)| norm 0.7235 (-0.74z)| lr 1.13e-04 | 320.80 ms | 52.6% bf16 MFU | 1637565 tok/s
step  133/19560 | loss 6.714043 (-1.31z)| norm 0.6848 (-0.86z)| lr 1.14e-04 | 320.76 ms | 52.6% bf16 MFU | 1637412 tok/s
step  134/19560 | loss 6.744080 (-1.27z)| norm 0.7027 (-0.93z)| lr 1.15e-04 | 320.60 ms | 52.6% bf16 MFU | 1637308 tok/s
step  135/19560 | loss 6.686044 (-1.31z)| norm 0.8869 (-0.80z)| lr 1.16e-04 | 321.07 ms | 52.6% bf16 MFU | 1637090 tok/s
step  136/19560 | loss 6.698227 (-1.29z)| norm 1.3588 (-0.22z)| lr 1.17e-04 | 321.55 ms | 52.5% bf16 MFU | 1636759 tok/s
step  137/19560 | loss 6.701592 (-1.27z)| norm 0.7831 (-1.07z)| lr 1.17e-04 | 320.66 ms | 52.6% bf16 MFU | 1636673 tok/s
step  138/19560 | loss 6.695870 (-1.26z)| norm 0.8640 (-0.99z)| lr 1.18e-04 | 320.81 ms | 52.6% bf16 MFU | 1636552 tok/s
step  139/19560 | loss 6.701926 (-1.24z)| norm 1.0766 (-0.64z)| lr 1.19e-04 | 321.63 ms | 52.5% bf16 MFU | 1636228 tok/s
step  140/19560 | loss 6.657983 (-1.27z)| norm 1.2112 (-0.39z)| lr 1.20e-04 | 320.67 ms | 52.6% bf16 MFU | 1636166 tok/s
step  141/19560 | loss 6.638037 (-1.28z)| norm 0.6936 (-1.36z)| lr 1.21e-04 | 321.14 ms | 52.6% bf16 MFU | 1635986 tok/s
step  142/19560 | loss 6.660120 (-1.24z)| norm 1.1571 (-0.46z)| lr 1.22e-04 | 321.25 ms | 52.5% bf16 MFU | 1635788 tok/s
step  143/19560 | loss 6.659147 (-1.23z)| norm 0.9675 (-0.82z)| lr 1.23e-04 | 321.54 ms | 52.5% bf16 MFU | 1635526 tok/s
step  144/19560 | loss 6.660672 (-1.21z)| norm 0.8242 (-1.10z)| lr 1.23e-04 | 322.29 ms | 52.4% bf16 MFU | 1635087 tok/s
step  145/19560 | loss 6.631635 (-1.23z)| norm 1.0779 (-0.57z)| lr 1.24e-04 | 321.48 ms | 52.5% bf16 MFU | 1634876 tok/s
step  146/19560 | loss 6.685111 (-1.16z)| norm 1.2781 (-0.14z)| lr 1.25e-04 | 321.03 ms | 52.6% bf16 MFU | 1634789 tok/s
step  147/19560 | loss 6.644342 (-1.19z)| norm 0.9396 (-0.84z)| lr 1.26e-04 | 321.46 ms | 52.5% bf16 MFU | 1634598 tok/s
step  148/19560 | loss 6.640083 (-1.18z)| norm 0.8838 (-0.94z)| lr 1.27e-04 | 321.11 ms | 52.6% bf16 MFU | 1634504 tok/s
step  149/19560 | loss 6.611271 (-1.20z)| norm 0.5423 (-1.64z)| lr 1.28e-04 | 321.16 ms | 52.6% bf16 MFU | 1634402 tok/s
step  150/19560 | loss 6.676477 (-1.12z)| norm 0.9108 (-0.85z)| lr 1.29e-04 | 321.63 ms | 52.5% bf16 MFU | 1634188 tok/s
step  151/19560 | loss 6.588881 (-1.20z)| norm 0.9762 (-0.69z)| lr 1.29e-04 | 322.06 ms | 52.4% bf16 MFU | 1633875 tok/s
step  152/19560 | loss 6.512462 (-1.28z)| norm 0.6910 (-1.30z)| lr 1.30e-04 | 321.87 ms | 52.4% bf16 MFU | 1633624 tok/s
step  153/19560 | loss 6.654555 (-1.10z)| norm 0.7826 (-1.08z)| lr 1.31e-04 | 320.97 ms | 52.6% bf16 MFU | 1633615 tok/s
step  154/19560 | loss 6.591102 (-1.16z)| norm 0.9325 (-0.74z)| lr 1.32e-04 | 321.26 ms | 52.5% bf16 MFU | 1633534 tok/s
step  155/19560 | loss 6.576769 (-1.17z)| norm 1.0483 (-0.47z)| lr 1.33e-04 | 321.67 ms | 52.5% bf16 MFU | 1633352 tok/s
step  156/19560 | loss 6.631476 (-1.09z)| norm 1.1559 (-0.22z)| lr 1.34e-04 | 322.17 ms | 52.4% bf16 MFU | 1633052 tok/s
step  157/19560 | loss 6.553662 (-1.18z)| norm 0.8778 (-0.84z)| lr 1.35e-04 | 320.93 ms | 52.6% bf16 MFU | 1633082 tok/s
step  158/19560 | loss 6.565734 (-1.15z)| norm 1.0544 (-0.42z)| lr 1.35e-04 | 321.59 ms | 52.5% bf16 MFU | 1632942 tok/s
step  159/19560 | loss 6.564290 (-1.14z)| norm 1.2492 (+0.06z)| lr 1.36e-04 | 322.38 ms | 52.4% bf16 MFU | 1632609 tok/s
step  160/19560 | loss 6.537234 (-1.17z)| norm 0.8371 (-0.93z)| lr 1.37e-04 | 321.88 ms | 52.4% bf16 MFU | 1632421 tok/s
step  161/19560 | loss 6.581266 (-1.10z)| norm 0.8292 (-0.96z)| lr 1.38e-04 | 321.20 ms | 52.5% bf16 MFU | 1632413 tok/s
step  162/19560 | loss 6.515292 (-1.18z)| norm 0.9193 (-0.72z)| lr 1.39e-04 | 323.29 ms | 52.2% bf16 MFU | 1631880 tok/s
step  163/19560 | loss 6.477805 (-1.22z)| norm 0.8065 (-1.01z)| lr 1.40e-04 | 322.12 ms | 52.4% bf16 MFU | 1631666 tok/s
step  164/19560 | loss 6.463096 (-1.23z)| norm 0.7481 (-1.19z)| lr 1.41e-04 | 322.09 ms | 52.4% bf16 MFU | 1631471 tok/s
step  165/19560 | loss 6.489305 (-1.18z)| norm 0.8020 (-1.02z)| lr 1.41e-04 | 321.79 ms | 52.4% bf16 MFU | 1631362 tok/s
step  166/19560 | loss 6.481836 (-1.18z)| norm 1.1173 (-0.10z)| lr 1.42e-04 | 322.28 ms | 52.4% bf16 MFU | 1631136 tok/s
step  167/19560 | loss 6.484381 (-1.17z)| norm 0.8161 (-0.97z)| lr 1.43e-04 | 322.11 ms | 52.4% bf16 MFU | 1630962 tok/s
step  168/19560 | loss 6.440076 (-1.22z)| norm 0.7304 (-1.22z)| lr 1.44e-04 | 321.60 ms | 52.5% bf16 MFU | 1630926 tok/s
step  169/19560 | loss 6.499251 (-1.13z)| norm 0.6188 (-1.55z)| lr 1.45e-04 | 322.89 ms | 52.3% bf16 MFU | 1630567 tok/s
step  170/19560 | loss 6.445863 (-1.20z)| norm 0.5297 (-1.79z)| lr 1.46e-04 | 321.74 ms | 52.5% bf16 MFU | 1630515 tok/s
step  171/19560 | loss 6.423327 (-1.22z)| norm 0.7079 (-1.23z)| lr 1.47e-04 | 321.59 ms | 52.5% bf16 MFU | 1630504 tok/s
step  172/19560 | loss 6.527288 (-1.06z)| norm 0.8409 (-0.81z)| lr 1.47e-04 | 321.88 ms | 52.4% bf16 MFU | 1630420 tok/s
step  173/19560 | loss 6.439231 (-1.19z)| norm 1.2091 (+0.36z)| lr 1.48e-04 | 321.71 ms | 52.5% bf16 MFU | 1630382 tok/s
step  174/19560 | loss 6.411086 (-1.22z)| norm 1.2758 (+0.59z)| lr 1.49e-04 | 321.60 ms | 52.5% bf16 MFU | 1630375 tok/s
step  175/19560 | loss 6.422555 (-1.20z)| norm 0.8457 (-0.78z)| lr 1.50e-04 | 321.57 ms | 52.5% bf16 MFU | 1630377 tok/s
step  176/19560 | loss 6.430792 (-1.18z)| norm 0.9278 (-0.50z)| lr 1.51e-04 | 321.56 ms | 52.5% bf16 MFU | 1630381 tok/s
step  177/19560 | loss 6.482769 (-1.08z)| norm 1.0105 (-0.22z)| lr 1.52e-04 | 322.20 ms | 52.4% bf16 MFU | 1630222 tok/s
step  178/19560 | loss 6.405682 (-1.20z)| norm 1.2120 (+0.46z)| lr 1.53e-04 | 321.21 ms | 52.5% bf16 MFU | 1630323 tok/s
step  179/19560 | loss 6.443990 (-1.13z)| norm 0.9910 (-0.26z)| lr 1.53e-04 | 321.80 ms | 52.4% bf16 MFU | 1630269 tok/s
step  180/19560 | loss 6.446496 (-1.12z)| norm 1.2475 (+0.62z)| lr 1.54e-04 | 321.94 ms | 52.4% bf16 MFU | 1630183 tok/s
step  181/19560 | loss 6.426447 (-1.15z)| norm 1.1282 (+0.24z)| lr 1.55e-04 | 321.83 ms | 52.4% bf16 MFU | 1630129 tok/s
step  182/19560 | loss 6.540252 (-0.94z)| norm 1.2299 (+0.61z)| lr 1.56e-04 | 321.45 ms | 52.5% bf16 MFU | 1630173 tok/s
step  183/19560 | loss 6.466063 (-1.07z)| norm 1.2860 (+0.82z)| lr 1.57e-04 | 322.16 ms | 52.4% bf16 MFU | 1630036 tok/s
step  184/19560 | loss 6.410929 (-1.17z)| norm 1.2490 (+0.71z)| lr 1.58e-04 | 321.83 ms | 52.4% bf16 MFU | 1629988 tok/s
step  185/19560 | loss 6.398338 (-1.18z)| norm 0.9078 (-0.52z)| lr 1.59e-04 | 322.53 ms | 52.3% bf16 MFU | 1629765 tok/s
step  186/19560 | loss 6.421873 (-1.13z)| norm 1.0038 (-0.15z)| lr 1.59e-04 | 321.51 ms | 52.5% bf16 MFU | 1629811 tok/s
step  187/19560 | loss 6.434301 (-1.10z)| norm 1.0095 (-0.11z)| lr 1.60e-04 | 321.78 ms | 52.4% bf16 MFU | 1629788 tok/s
step  188/19560 | loss 6.347479 (-1.27z)| norm 0.9490 (-0.33z)| lr 1.61e-04 | 322.35 ms | 52.4% bf16 MFU | 1629620 tok/s
step  189/19560 | loss 6.346868 (-1.26z)| norm 0.9825 (-0.18z)| lr 1.62e-04 | 322.48 ms | 52.3% bf16 MFU | 1629430 tok/s
step  190/19560 | loss 6.367538 (-1.22z)| norm 0.9460 (-0.32z)| lr 1.63e-04 | 322.03 ms | 52.4% bf16 MFU | 1629361 tok/s
step  191/19560 | loss 6.332886 (-1.28z)| norm 0.9006 (-0.49z)| lr 1.64e-04 | 321.77 ms | 52.5% bf16 MFU | 1629362 tok/s
step  192/19560 | loss 6.339782 (-1.26z)| norm 0.8391 (-0.73z)| lr 1.65e-04 | 322.33 ms | 52.4% bf16 MFU | 1629220 tok/s
step  193/19560 | loss 6.369072 (-1.19z)| norm 0.8594 (-0.63z)| lr 1.65e-04 | 322.57 ms | 52.3% bf16 MFU | 1629027 tok/s
step  194/19560 | loss 6.322015 (-1.29z)| norm 0.9760 (-0.13z)| lr 1.66e-04 | 322.70 ms | 52.3% bf16 MFU | 1628810 tok/s
step  195/19560 | loss 6.264630 (-1.42z)| norm 0.9491 (-0.23z)| lr 1.67e-04 | 322.20 ms | 52.4% bf16 MFU | 1628730 tok/s
step  196/19560 | loss 6.391530 (-1.11z)| norm 0.6910 (-1.33z)| lr 1.68e-04 | 321.82 ms | 52.4% bf16 MFU | 1628752 tok/s
step  197/19560 | loss 6.303662 (-1.32z)| norm 0.5944 (-1.71z)| lr 1.69e-04 | 321.90 ms | 52.4% bf16 MFU | 1628751 tok/s
step  198/19560 | loss 6.346094 (-1.21z)| norm 0.9871 (-0.01z)| lr 1.70e-04 | 322.37 ms | 52.4% bf16 MFU | 1628631 tok/s
step  199/19560 | loss 6.332290 (-1.24z)| norm 1.0545 (+0.29z)| lr 1.71e-04 | 321.87 ms | 52.4% bf16 MFU | 1628645 tok/s
step  200/19560 | loss 6.309676 (-1.29z)| norm 0.9253 (-0.27z)| lr 1.71e-04 | 321.49 ms | 52.5% bf16 MFU | 1628753 tok/s
step  201/19560 | loss 6.360720 (-1.15z)| norm 0.6984 (-1.24z)| lr 1.72e-04 | 322.18 ms | 52.4% bf16 MFU | 1628681 tok/s
step  202/19560 | loss 6.334215 (-1.22z)| norm 0.8400 (-0.61z)| lr 1.73e-04 | 321.22 ms | 52.5% bf16 MFU | 1628855 tok/s
step  203/19560 | loss 6.384963 (-1.08z)| norm 0.9009 (-0.33z)| lr 1.74e-04 | 321.74 ms | 52.5% bf16 MFU | 1628889 tok/s
step  204/19560 | loss 6.334068 (-1.21z)| norm 0.6918 (-1.24z)| lr 1.75e-04 | 322.00 ms | 52.4% bf16 MFU | 1628857 tok/s
step  205/19560 | loss 6.385949 (-1.06z)| norm 0.7777 (-0.86z)| lr 1.76e-04 | 321.77 ms | 52.5% bf16 MFU | 1628884 tok/s
step  206/19560 | loss 6.327003 (-1.22z)| norm 0.7041 (-1.19z)| lr 1.77e-04 | 321.50 ms | 52.5% bf16 MFU | 1628977 tok/s
step  207/19560 | loss 6.258727 (-1.41z)| norm 0.9408 (-0.09z)| lr 1.77e-04 | 321.65 ms | 52.5% bf16 MFU | 1629029 tok/s
step  208/19560 | loss 6.339325 (-1.17z)| norm 1.0069 (+0.22z)| lr 1.78e-04 | 322.30 ms | 52.4% bf16 MFU | 1628912 tok/s
step  209/19560 | loss 6.319749 (-1.22z)| norm 0.9581 (+0.01z)| lr 1.79e-04 | 322.47 ms | 52.3% bf16 MFU | 1628758 tok/s
step  210/19560 | loss 6.312520 (-1.23z)| norm 0.8478 (-0.50z)| lr 1.80e-04 | 321.33 ms | 52.5% bf16 MFU | 1628901 tok/s
step  211/19560 | loss 6.282964 (-1.31z)| norm 0.8311 (-0.58z)| lr 1.81e-04 | 322.39 ms | 52.4% bf16 MFU | 1628768 tok/s
step  212/19560 | loss 6.352109 (-1.09z)| norm 0.7342 (-1.04z)| lr 1.82e-04 | 321.41 ms | 52.5% bf16 MFU | 1628892 tok/s
step  213/19560 | loss 6.293660 (-1.27z)| norm 1.0657 (+0.60z)| lr 1.83e-04 | 322.23 ms | 52.4% bf16 MFU | 1628800 tok/s
step  214/19560 | loss 6.373910 (-1.00z)| norm 1.2671 (+1.57z)| lr 1.83e-04 | 323.61 ms | 52.2% bf16 MFU | 1628366 tok/s
step  215/19560 | loss 6.274603 (-1.32z)| norm 0.9181 (-0.14z)| lr 1.84e-04 | 322.12 ms | 52.4% bf16 MFU | 1628329 tok/s
step  216/19560 | loss 6.326112 (-1.13z)| norm 0.9230 (-0.11z)| lr 1.85e-04 | 322.02 ms | 52.4% bf16 MFU | 1628318 tok/s
step  217/19560 | loss 6.305054 (-1.20z)| norm 0.9743 (+0.14z)| lr 1.86e-04 | 321.81 ms | 52.4% bf16 MFU | 1628362 tok/s
step  218/19560 | loss 6.303480 (-1.19z)| norm 1.0917 (+0.71z)| lr 1.87e-04 | 322.63 ms | 52.3% bf16 MFU | 1628196 tok/s
step  219/19560 | loss 6.276670 (-1.28z)| norm 0.8728 (-0.36z)| lr 1.88e-04 | 321.76 ms | 52.5% bf16 MFU | 1628259 tok/s
step  220/19560 | loss 6.187314 (-1.59z)| norm 0.8676 (-0.37z)| lr 1.89e-04 | 323.23 ms | 52.2% bf16 MFU | 1627948 tok/s
step  221/19560 | loss 6.289708 (-1.21z)| norm 0.7918 (-0.74z)| lr 1.89e-04 | 322.76 ms | 52.3% bf16 MFU | 1627771 tok/s
step  222/19560 | loss 6.290948 (-1.19z)| norm 0.8122 (-0.65z)| lr 1.90e-04 | 321.64 ms | 52.5% bf16 MFU | 1627885 tok/s
step  223/19560 | loss 6.231311 (-1.41z)| norm 0.6466 (-1.45z)| lr 1.91e-04 | 323.03 ms | 52.2% bf16 MFU | 1627641 tok/s
step  224/19560 | loss 6.299545 (-1.13z)| norm 0.6262 (-1.53z)| lr 1.92e-04 | 321.72 ms | 52.5% bf16 MFU | 1627740 tok/s
step  225/19560 | loss 6.268791 (-1.24z)| norm 0.6307 (-1.49z)| lr 1.93e-04 | 322.17 ms | 52.4% bf16 MFU | 1627722 tok/s
step  226/19560 | loss 6.170896 (-1.61z)| norm 0.7372 (-0.96z)| lr 1.94e-04 | 321.82 ms | 52.4% bf16 MFU | 1627792 tok/s
step  227/19560 | loss 6.202180 (-1.48z)| norm 0.8456 (-0.42z)| lr 1.95e-04 | 321.92 ms | 52.4% bf16 MFU | 1627834 tok/s
step  228/19560 | loss 6.255916 (-1.25z)| norm 0.9567 (+0.12z)| lr 1.95e-04 | 321.96 ms | 52.4% bf16 MFU | 1627863 tok/s
step  229/19560 | loss 6.189298 (-1.51z)| norm 0.7889 (-0.71z)| lr 1.96e-04 | 321.91 ms | 52.4% bf16 MFU | 1627904 tok/s
step  230/19560 | loss 6.202960 (-1.44z)| norm 0.7400 (-0.94z)| lr 1.97e-04 | 322.27 ms | 52.4% bf16 MFU | 1627853 tok/s
step  231/19560 | loss 6.205101 (-1.41z)| norm 0.8333 (-0.48z)| lr 1.98e-04 | 321.67 ms | 52.5% bf16 MFU | 1627955 tok/s
step  232/19560 | loss 6.190864 (-1.47z)| norm 0.7611 (-0.82z)| lr 1.99e-04 | 322.41 ms | 52.3% bf16 MFU | 1627866 tok/s
step  233/19560 | loss 6.116053 (-1.76z)| norm 0.7484 (-0.88z)| lr 2.00e-04 | 322.03 ms | 52.4% bf16 MFU | 1627877 tok/s
step  234/19560 | loss 6.208245 (-1.36z)| norm 0.7113 (-1.05z)| lr 2.01e-04 | 321.31 ms | 52.5% bf16 MFU | 1628070 tok/s
step  235/19560 | loss 6.189609 (-1.42z)| norm 0.5289 (-1.90z)| lr 2.01e-04 | 323.50 ms | 52.2% bf16 MFU | 1627700 tok/s
step  236/19560 | loss 6.183286 (-1.44z)| norm 0.5708 (-1.67z)| lr 2.02e-04 | 321.70 ms | 52.5% bf16 MFU | 1627802 tok/s
step  237/19560 | loss 6.233494 (-1.20z)| norm 0.6833 (-1.12z)| lr 2.03e-04 | 322.52 ms | 52.3% bf16 MFU | 1627690 tok/s
step  238/19560 | loss 6.152956 (-1.55z)| norm 1.1028 (+0.86z)| lr 2.04e-04 | 322.12 ms | 52.4% bf16 MFU | 1627687 tok/s
step  239/19560 | loss 6.141744 (-1.58z)| norm 1.2986 (+1.76z)| lr 2.05e-04 | 322.86 ms | 52.3% bf16 MFU | 1627498 tok/s
step  240/19560 | loss 6.122013 (-1.64z)| norm 0.9029 (-0.09z)| lr 2.06e-04 | 321.89 ms | 52.4% bf16 MFU | 1627562 tok/s
step  241/19560 | loss 6.203560 (-1.27z)| norm 0.9829 (+0.28z)| lr 2.07e-04 | 322.17 ms | 52.4% bf16 MFU | 1627551 tok/s
step  242/19560 | loss 6.089408 (-1.76z)| norm 0.9447 (+0.11z)| lr 2.07e-04 | 322.21 ms | 52.4% bf16 MFU | 1627531 tok/s
step  243/19560 | loss 6.097892 (-1.70z)| norm 1.3246 (+1.93z)| lr 2.08e-04 | 322.17 ms | 52.4% bf16 MFU | 1627523 tok/s
step  244/19560 | loss 6.157339 (-1.42z)| norm 0.7777 (-0.68z)| lr 2.09e-04 | 322.96 ms | 52.3% bf16 MFU | 1627316 tok/s
step  245/19560 | loss 6.222884 (-1.11z)| norm 1.1242 (+1.09z)| lr 2.10e-04 | 321.89 ms | 52.4% bf16 MFU | 1627388 tok/s
step  246/19560 | loss 6.168555 (-1.35z)| norm 1.0572 (+0.74z)| lr 2.11e-04 | 322.56 ms | 52.3% bf16 MFU | 1627289 tok/s
step  247/19560 | loss 6.219739 (-1.09z)| norm 1.3531 (+2.24z)| lr 2.12e-04 | 323.00 ms | 52.3% bf16 MFU | 1627083 tok/s
step  248/19560 | loss 6.193687 (-1.21z)| norm 0.8183 (-0.50z)| lr 2.13e-04 | 321.80 ms | 52.4% bf16 MFU | 1627191 tok/s
step  249/19560 | loss 6.098156 (-1.67z)| norm 0.7166 (-1.01z)| lr 2.13e-04 | 322.11 ms | 52.4% bf16 MFU | 1627215 tok/s
step  250/19560 | loss 6.191140 (-1.20z)| norm 0.7039 (-1.06z)| lr 2.14e-04 | 322.71 ms | 52.3% bf16 MFU | 1627086 tok/s
val loss 6.161226
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2474/10042 = 0.246365
step  251/19560 | loss 6.170714 (-1.30z)| norm 0.7247 (-0.94z)| lr 2.15e-04 | 321.51 ms | 52.5% bf16 MFU | 1627265 tok/s
step  252/19560 | loss 6.139436 (-1.44z)| norm 0.7882 (-0.61z)| lr 2.16e-04 | 321.50 ms | 52.5% bf16 MFU | 1627440 tok/s
step  253/19560 | loss 6.186738 (-1.18z)| norm 1.0084 (+0.53z)| lr 2.17e-04 | 321.97 ms | 52.4% bf16 MFU | 1627488 tok/s
step  254/19560 | loss 6.098423 (-1.64z)| norm 1.5318 (+3.09z)| lr 2.18e-04 | 322.31 ms | 52.4% bf16 MFU | 1627446 tok/s
step  255/19560 | loss 6.163700 (-1.28z)| norm 0.8270 (-0.43z)| lr 2.19e-04 | 322.50 ms | 52.3% bf16 MFU | 1627360 tok/s
step  256/19560 | loss 6.178758 (-1.19z)| norm 0.8859 (-0.13z)| lr 2.19e-04 | 322.01 ms | 52.4% bf16 MFU | 1627400 tok/s
step  257/19560 | loss 6.131074 (-1.43z)| norm 0.9603 (+0.25z)| lr 2.20e-04 | 322.54 ms | 52.3% bf16 MFU | 1627304 tok/s
step  258/19560 | loss 6.132967 (-1.41z)| norm 0.7425 (-0.84z)| lr 2.21e-04 | 322.39 ms | 52.4% bf16 MFU | 1627252 tok/s
step  259/19560 | loss 6.120650 (-1.46z)| norm 0.6974 (-1.07z)| lr 2.22e-04 | 323.25 ms | 52.2% bf16 MFU | 1626984 tok/s
step  260/19560 | loss 6.065009 (-1.74z)| norm 0.7579 (-0.76z)| lr 2.23e-04 | 322.82 ms | 52.3% bf16 MFU | 1626838 tok/s
step  261/19560 | loss 6.168475 (-1.15z)| norm 0.8033 (-0.54z)| lr 2.24e-04 | 322.58 ms | 52.3% bf16 MFU | 1626760 tok/s
step  262/19560 | loss 6.122702 (-1.39z)| norm 0.8633 (-0.25z)| lr 2.25e-04 | 321.76 ms | 52.5% bf16 MFU | 1626895 tok/s
step  263/19560 | loss 6.077872 (-1.63z)| norm 1.0526 (+0.71z)| lr 2.25e-04 | 322.72 ms | 52.3% bf16 MFU | 1626780 tok/s
step  264/19560 | loss 6.072188 (-1.64z)| norm 0.9287 (+0.10z)| lr 2.26e-04 | 322.54 ms | 52.3% bf16 MFU | 1626716 tok/s
step  265/19560 | loss 6.078524 (-1.58z)| norm 0.7649 (-0.75z)| lr 2.27e-04 | 322.69 ms | 52.3% bf16 MFU | 1626618 tok/s
step  266/19560 | loss 6.099412 (-1.45z)| norm 0.6816 (-1.17z)| lr 2.28e-04 | 322.21 ms | 52.4% bf16 MFU | 1626646 tok/s
step  267/19560 | loss 6.132026 (-1.24z)| norm 0.5822 (-1.64z)| lr 2.29e-04 | 321.75 ms | 52.5% bf16 MFU | 1626788 tok/s
step  268/19560 | loss 6.046096 (-1.72z)| norm 0.7515 (-0.77z)| lr 2.30e-04 | 323.18 ms | 52.2% bf16 MFU | 1626563 tok/s
step  269/19560 | loss 6.147350 (-1.11z)| norm 0.8301 (-0.37z)| lr 2.31e-04 | 322.66 ms | 52.3% bf16 MFU | 1626480 tok/s
step  270/19560 | loss 6.132587 (-1.19z)| norm 0.9729 (+0.38z)| lr 2.31e-04 | 322.43 ms | 52.3% bf16 MFU | 1626458 tok/s
step  271/19560 | loss 6.052064 (-1.65z)| norm 0.7361 (-0.84z)| lr 2.32e-04 | 323.07 ms | 52.2% bf16 MFU | 1626277 tok/s
step  272/19560 | loss 6.098472 (-1.36z)| norm 0.5770 (-1.64z)| lr 2.33e-04 | 322.25 ms | 52.4% bf16 MFU | 1626310 tok/s
step  273/19560 | loss 6.043133 (-1.67z)| norm 0.6163 (-1.41z)| lr 2.34e-04 | 321.81 ms | 52.4% bf16 MFU | 1626453 tok/s
step  274/19560 | loss 6.140728 (-1.07z)| norm 0.7127 (-0.91z)| lr 2.35e-04 | 322.79 ms | 52.3% bf16 MFU | 1626342 tok/s
step  275/19560 | loss 6.042943 (-1.65z)| norm 0.8290 (-0.30z)| lr 2.36e-04 | 322.80 ms | 52.3% bf16 MFU | 1626234 tok/s
step  276/19560 | loss 6.087063 (-1.36z)| norm 0.7367 (-0.77z)| lr 2.37e-04 | 322.05 ms | 52.4% bf16 MFU | 1626322 tok/s
step  277/19560 | loss 6.012643 (-1.80z)| norm 0.9222 (+0.17z)| lr 2.37e-04 | 322.79 ms | 52.3% bf16 MFU | 1626218 tok/s
step  278/19560 | loss 6.073081 (-1.42z)| norm 0.7074 (-0.94z)| lr 2.38e-04 | 321.93 ms | 52.4% bf16 MFU | 1626338 tok/s
step  279/19560 | loss 6.085942 (-1.32z)| norm 0.6929 (-1.00z)| lr 2.39e-04 | 322.47 ms | 52.3% bf16 MFU | 1626313 tok/s
step  280/19560 | loss 6.014268 (-1.75z)| norm 0.8564 (-0.16z)| lr 2.40e-04 | 323.03 ms | 52.2% bf16 MFU | 1626150 tok/s
step  281/19560 | loss 6.064736 (-1.41z)| norm 1.0737 (+0.96z)| lr 2.41e-04 | 322.45 ms | 52.3% bf16 MFU | 1626139 tok/s
step  282/19560 | loss 6.163796 (-0.76z)| norm 1.3107 (+2.13z)| lr 2.42e-04 | 322.57 ms | 52.3% bf16 MFU | 1626100 tok/s
step  283/19560 | loss 6.092479 (-1.22z)| norm 0.7165 (-0.88z)| lr 2.43e-04 | 321.97 ms | 52.4% bf16 MFU | 1626213 tok/s
step  284/19560 | loss 6.059585 (-1.43z)| norm 0.7752 (-0.57z)| lr 2.43e-04 | 323.46 ms | 52.2% bf16 MFU | 1625946 tok/s
step  285/19560 | loss 6.051013 (-1.47z)| norm 0.8881 (+0.01z)| lr 2.44e-04 | 322.11 ms | 52.4% bf16 MFU | 1626033 tok/s
step  286/19560 | loss 6.031407 (-1.58z)| norm 0.9707 (+0.43z)| lr 2.45e-04 | 322.47 ms | 52.3% bf16 MFU | 1626025 tok/s
step  287/19560 | loss 6.037878 (-1.52z)| norm 0.7730 (-0.57z)| lr 2.46e-04 | 322.26 ms | 52.4% bf16 MFU | 1626068 tok/s
step  288/19560 | loss 6.010933 (-1.68z)| norm 0.7018 (-0.93z)| lr 2.47e-04 | 322.21 ms | 52.4% bf16 MFU | 1626122 tok/s
step  289/19560 | loss 6.050262 (-1.40z)| norm 0.7083 (-0.89z)| lr 2.48e-04 | 322.31 ms | 52.4% bf16 MFU | 1626149 tok/s
step  290/19560 | loss 6.014580 (-1.63z)| norm 0.6835 (-1.01z)| lr 2.49e-04 | 322.67 ms | 52.3% bf16 MFU | 1626084 tok/s
step  291/19560 | loss 5.997790 (-1.72z)| norm 0.7134 (-0.85z)| lr 2.49e-04 | 322.36 ms | 52.4% bf16 MFU | 1626099 tok/s
step  292/19560 | loss 6.056323 (-1.29z)| norm 0.7873 (-0.47z)| lr 2.50e-04 | 323.06 ms | 52.2% bf16 MFU | 1625937 tok/s
step  293/19560 | loss 5.936280 (-2.09z)| norm 0.9420 (+0.32z)| lr 2.51e-04 | 321.84 ms | 52.4% bf16 MFU | 1626092 tok/s
step  294/19560 | loss 6.031710 (-1.41z)| norm 1.1393 (+1.33z)| lr 2.52e-04 | 322.05 ms | 52.4% bf16 MFU | 1626185 tok/s
step  295/19560 | loss 6.076549 (-1.08z)| norm 0.7521 (-0.65z)| lr 2.53e-04 | 322.85 ms | 52.3% bf16 MFU | 1626072 tok/s
step  296/19560 | loss 5.974367 (-1.77z)| norm 0.8062 (-0.38z)| lr 2.54e-04 | 321.97 ms | 52.4% bf16 MFU | 1626187 tok/s
step  297/19560 | loss 6.017417 (-1.45z)| norm 0.7973 (-0.43z)| lr 2.55e-04 | 322.59 ms | 52.3% bf16 MFU | 1626141 tok/s
step  298/19560 | loss 5.929180 (-2.03z)| norm 0.8796 (-0.02z)| lr 2.55e-04 | 322.22 ms | 52.4% bf16 MFU | 1626189 tok/s
step  299/19560 | loss 6.032049 (-1.29z)| norm 1.0782 (+1.00z)| lr 2.56e-04 | 323.28 ms | 52.2% bf16 MFU | 1625969 tok/s
step  300/19560 | loss 6.004977 (-1.47z)| norm 1.1486 (+1.35z)| lr 2.57e-04 | 322.81 ms | 52.3% bf16 MFU | 1625877 tok/s
step  301/19560 | loss 6.012208 (-1.40z)| norm 1.1050 (+1.13z)| lr 2.58e-04 | 321.97 ms | 52.4% bf16 MFU | 1626002 tok/s
step  302/19560 | loss 6.007414 (-1.41z)| norm 0.9424 (+0.30z)| lr 2.59e-04 | 322.55 ms | 52.3% bf16 MFU | 1625975 tok/s
step  303/19560 | loss 5.927119 (-1.95z)| norm 0.7586 (-0.67z)| lr 2.60e-04 | 322.98 ms | 52.3% bf16 MFU | 1625841 tok/s
step  304/19560 | loss 5.979395 (-1.55z)| norm 0.7463 (-0.73z)| lr 2.61e-04 | 322.26 ms | 52.4% bf16 MFU | 1625895 tok/s
step  305/19560 | loss 6.026130 (-1.20z)| norm 0.8494 (-0.17z)| lr 2.61e-04 | 323.28 ms | 52.2% bf16 MFU | 1625688 tok/s
step  306/19560 | loss 5.999972 (-1.37z)| norm 0.9095 (+0.16z)| lr 2.62e-04 | 322.94 ms | 52.3% bf16 MFU | 1625579 tok/s
step  307/19560 | loss 5.983938 (-1.47z)| norm 0.8459 (-0.18z)| lr 2.63e-04 | 322.08 ms | 52.4% bf16 MFU | 1625692 tok/s
step  308/19560 | loss 5.999636 (-1.34z)| norm 0.8304 (-0.25z)| lr 2.64e-04 | 322.08 ms | 52.4% bf16 MFU | 1625798 tok/s
step  309/19560 | loss 5.988765 (-1.40z)| norm 1.0922 (+1.19z)| lr 2.65e-04 | 323.04 ms | 52.2% bf16 MFU | 1625657 tok/s
step  310/19560 | loss 6.181597 (+0.05z)| norm 1.0049 (+0.73z)| lr 2.66e-04 | 322.07 ms | 52.4% bf16 MFU | 1625768 tok/s
step  311/19560 | loss 5.978450 (-1.49z)| norm 1.1352 (+1.48z)| lr 2.67e-04 | 322.48 ms | 52.3% bf16 MFU | 1625770 tok/s
step  312/19560 | loss 6.019770 (-1.16z)| norm 1.3545 (+2.67z)| lr 2.67e-04 | 322.09 ms | 52.4% bf16 MFU | 1625870 tok/s
step  313/19560 | loss 6.015505 (-1.17z)| norm 0.9328 (+0.33z)| lr 2.68e-04 | 323.16 ms | 52.2% bf16 MFU | 1625696 tok/s
step  314/19560 | loss 6.024765 (-1.09z)| norm 0.9984 (+0.69z)| lr 2.69e-04 | 322.34 ms | 52.4% bf16 MFU | 1625736 tok/s
step  315/19560 | loss 6.004093 (-1.24z)| norm 1.1876 (+1.72z)| lr 2.70e-04 | 322.18 ms | 52.4% bf16 MFU | 1625815 tok/s
step  316/19560 | loss 5.937592 (-1.74z)| norm 0.9990 (+0.68z)| lr 2.71e-04 | 323.30 ms | 52.2% bf16 MFU | 1625610 tok/s
step  317/19560 | loss 5.930367 (-1.77z)| norm 1.0054 (+0.71z)| lr 2.72e-04 | 322.43 ms | 52.3% bf16 MFU | 1625632 tok/s
step  318/19560 | loss 5.957980 (-1.53z)| norm 0.7862 (-0.48z)| lr 2.73e-04 | 321.84 ms | 52.4% bf16 MFU | 1625802 tok/s
step  319/19560 | loss 5.962630 (-1.47z)| norm 0.7464 (-0.69z)| lr 2.73e-04 | 322.41 ms | 52.3% bf16 MFU | 1625819 tok/s
step  320/19560 | loss 5.891017 (-2.00z)| norm 0.6738 (-1.08z)| lr 2.74e-04 | 322.71 ms | 52.3% bf16 MFU | 1625760 tok/s
step  321/19560 | loss 5.948895 (-1.52z)| norm 0.6873 (-0.99z)| lr 2.75e-04 | 322.73 ms | 52.3% bf16 MFU | 1625699 tok/s
step  322/19560 | loss 5.896472 (-1.90z)| norm 0.6523 (-1.17z)| lr 2.76e-04 | 322.94 ms | 52.3% bf16 MFU | 1625588 tok/s
step  323/19560 | loss 5.866193 (-2.08z)| norm 0.7516 (-0.62z)| lr 2.77e-04 | 322.30 ms | 52.4% bf16 MFU | 1625643 tok/s
step  324/19560 | loss 5.929397 (-1.57z)| norm 1.0871 (+1.18z)| lr 2.78e-04 | 322.23 ms | 52.4% bf16 MFU | 1625714 tok/s
step  325/19560 | loss 5.824729 (-2.33z)| norm 1.1047 (+1.25z)| lr 2.79e-04 | 322.82 ms | 52.3% bf16 MFU | 1625632 tok/s
step  326/19560 | loss 5.920951 (-1.56z)| norm 1.0379 (+0.89z)| lr 2.79e-04 | 322.31 ms | 52.4% bf16 MFU | 1625683 tok/s
step  327/19560 | loss 5.907260 (-1.64z)| norm 1.3754 (+2.63z)| lr 2.80e-04 | 322.56 ms | 52.3% bf16 MFU | 1625670 tok/s
step  328/19560 | loss 5.982334 (-1.05z)| norm 0.7945 (-0.42z)| lr 2.81e-04 | 322.75 ms | 52.3% bf16 MFU | 1625608 tok/s
step  329/19560 | loss 5.892969 (-1.72z)| norm 0.9413 (+0.34z)| lr 2.82e-04 | 322.32 ms | 52.4% bf16 MFU | 1625656 tok/s
step  330/19560 | loss 5.908260 (-1.58z)| norm 0.8929 (+0.08z)| lr 2.83e-04 | 322.86 ms | 52.3% bf16 MFU | 1625568 tok/s
step  331/19560 | loss 5.958205 (-1.18z)| norm 0.8705 (-0.04z)| lr 2.84e-04 | 322.95 ms | 52.3% bf16 MFU | 1625460 tok/s
step  332/19560 | loss 5.932256 (-1.36z)| norm 0.8339 (-0.24z)| lr 2.85e-04 | 323.18 ms | 52.2% bf16 MFU | 1625301 tok/s
step  333/19560 | loss 5.904912 (-1.57z)| norm 0.8774 (-0.01z)| lr 2.85e-04 | 322.49 ms | 52.3% bf16 MFU | 1625324 tok/s
step  334/19560 | loss 5.852131 (-1.96z)| norm 0.9555 (+0.40z)| lr 2.86e-04 | 323.42 ms | 52.2% bf16 MFU | 1625112 tok/s
step  335/19560 | loss 5.889353 (-1.63z)| norm 1.1322 (+1.32z)| lr 2.87e-04 | 322.11 ms | 52.4% bf16 MFU | 1625241 tok/s
step  336/19560 | loss 5.898538 (-1.54z)| norm 1.2622 (+1.97z)| lr 2.88e-04 | 322.67 ms | 52.3% bf16 MFU | 1625220 tok/s
step  337/19560 | loss 5.893711 (-1.56z)| norm 0.9089 (+0.13z)| lr 2.89e-04 | 322.84 ms | 52.3% bf16 MFU | 1625159 tok/s
step  338/19560 | loss 5.882050 (-1.63z)| norm 0.8151 (-0.36z)| lr 2.90e-04 | 321.95 ms | 52.4% bf16 MFU | 1625324 tok/s
step  339/19560 | loss 5.934131 (-1.19z)| norm 0.7395 (-0.75z)| lr 2.91e-04 | 322.30 ms | 52.4% bf16 MFU | 1625393 tok/s
step  340/19560 | loss 5.838213 (-1.95z)| norm 0.7837 (-0.52z)| lr 2.91e-04 | 322.02 ms | 52.4% bf16 MFU | 1625530 tok/s
step  341/19560 | loss 5.845649 (-1.86z)| norm 0.7655 (-0.60z)| lr 2.92e-04 | 323.55 ms | 52.2% bf16 MFU | 1625273 tok/s
step  342/19560 | loss 5.851879 (-1.79z)| norm 0.7371 (-0.74z)| lr 2.93e-04 | 322.98 ms | 52.3% bf16 MFU | 1625173 tok/s
step  343/19560 | loss 5.873928 (-1.59z)| norm 0.7046 (-0.90z)| lr 2.94e-04 | 322.34 ms | 52.4% bf16 MFU | 1625238 tok/s
step  344/19560 | loss 5.815110 (-2.04z)| norm 0.7610 (-0.60z)| lr 2.95e-04 | 322.46 ms | 52.3% bf16 MFU | 1625272 tok/s
step  345/19560 | loss 5.890047 (-1.41z)| norm 0.8427 (-0.16z)| lr 2.96e-04 | 323.01 ms | 52.2% bf16 MFU | 1625164 tok/s
step  346/19560 | loss 5.877178 (-1.50z)| norm 0.8441 (-0.14z)| lr 2.97e-04 | 322.15 ms | 52.4% bf16 MFU | 1625279 tok/s
step  347/19560 | loss 5.843349 (-1.76z)| norm 0.9460 (+0.39z)| lr 2.97e-04 | 323.14 ms | 52.2% bf16 MFU | 1625138 tok/s
step  348/19560 | loss 5.834387 (-1.80z)| norm 1.0741 (+1.06z)| lr 2.98e-04 | 322.26 ms | 52.4% bf16 MFU | 1625226 tok/s
step  349/19560 | loss 5.835071 (-1.77z)| norm 1.1507 (+1.44z)| lr 2.99e-04 | 322.67 ms | 52.3% bf16 MFU | 1625206 tok/s
step  350/19560 | loss 5.887681 (-1.31z)| norm 1.2230 (+1.78z)| lr 3.00e-04 | 322.94 ms | 52.3% bf16 MFU | 1625121 tok/s
step  351/19560 | loss 5.855923 (-1.56z)| norm 1.2046 (+1.65z)| lr 3.01e-04 | 322.98 ms | 52.3% bf16 MFU | 1625030 tok/s
step  352/19560 | loss 5.869227 (-1.43z)| norm 1.0849 (+1.02z)| lr 3.02e-04 | 322.53 ms | 52.3% bf16 MFU | 1625055 tok/s
step  353/19560 | loss 5.876354 (-1.36z)| norm 0.6274 (-1.34z)| lr 3.03e-04 | 322.62 ms | 52.3% bf16 MFU | 1625058 tok/s
step  354/19560 | loss 5.750834 (-2.38z)| norm 0.7555 (-0.68z)| lr 3.03e-04 | 322.37 ms | 52.4% bf16 MFU | 1625122 tok/s
step  355/19560 | loss 5.808078 (-1.85z)| norm 0.8335 (-0.28z)| lr 3.04e-04 | 322.60 ms | 52.3% bf16 MFU | 1625126 tok/s
step  356/19560 | loss 5.734389 (-2.42z)| norm 0.9844 (+0.50z)| lr 3.05e-04 | 323.26 ms | 52.2% bf16 MFU | 1624963 tok/s
step  357/19560 | loss 5.861800 (-1.33z)| norm 1.0258 (+0.70z)| lr 3.06e-04 | 323.08 ms | 52.2% bf16 MFU | 1624853 tok/s
step  358/19560 | loss 5.850970 (-1.40z)| norm 0.8918 (+0.01z)| lr 3.07e-04 | 322.31 ms | 52.4% bf16 MFU | 1624944 tok/s
step  359/19560 | loss 5.794242 (-1.85z)| norm 0.9129 (+0.11z)| lr 3.08e-04 | 322.03 ms | 52.4% bf16 MFU | 1625101 tok/s
step  360/19560 | loss 5.836368 (-1.47z)| norm 1.0078 (+0.59z)| lr 3.09e-04 | 322.63 ms | 52.3% bf16 MFU | 1625097 tok/s
step  361/19560 | loss 5.874473 (-1.13z)| norm 1.1199 (+1.16z)| lr 3.09e-04 | 321.84 ms | 52.4% bf16 MFU | 1625293 tok/s
step  362/19560 | loss 5.823833 (-1.53z)| norm 1.1025 (+1.05z)| lr 3.10e-04 | 323.18 ms | 52.2% bf16 MFU | 1625143 tok/s
step  363/19560 | loss 5.816920 (-1.57z)| norm 0.9826 (+0.42z)| lr 3.11e-04 | 322.45 ms | 52.3% bf16 MFU | 1625183 tok/s
step  364/19560 | loss 5.808570 (-1.61z)| norm 0.7347 (-0.89z)| lr 3.12e-04 | 321.78 ms | 52.4% bf16 MFU | 1625392 tok/s
step  365/19560 | loss 5.763203 (-1.96z)| norm 0.6808 (-1.18z)| lr 3.13e-04 | 322.60 ms | 52.3% bf16 MFU | 1625382 tok/s
step  366/19560 | loss 5.778937 (-1.79z)| norm 0.6525 (-1.31z)| lr 3.14e-04 | 322.96 ms | 52.3% bf16 MFU | 1625283 tok/s
step  367/19560 | loss 5.710406 (-2.30z)| norm 0.7673 (-0.69z)| lr 3.15e-04 | 322.25 ms | 52.4% bf16 MFU | 1625366 tok/s
step  368/19560 | loss 5.700305 (-2.32z)| norm 0.7359 (-0.85z)| lr 3.15e-04 | 322.63 ms | 52.3% bf16 MFU | 1625350 tok/s
step  369/19560 | loss 5.741160 (-1.95z)| norm 0.6710 (-1.18z)| lr 3.16e-04 | 322.21 ms | 52.4% bf16 MFU | 1625440 tok/s
step  370/19560 | loss 5.794682 (-1.49z)| norm 0.7766 (-0.61z)| lr 3.17e-04 | 322.61 ms | 52.3% bf16 MFU | 1625427 tok/s
step  371/19560 | loss 5.798728 (-1.43z)| norm 0.8420 (-0.25z)| lr 3.18e-04 | 323.51 ms | 52.2% bf16 MFU | 1625186 tok/s
step  372/19560 | loss 5.753415 (-1.76z)| norm 1.4364 (+2.85z)| lr 3.19e-04 | 322.42 ms | 52.3% bf16 MFU | 1625232 tok/s
step  373/19560 | loss 5.780749 (-1.53z)| norm 0.8169 (-0.39z)| lr 3.20e-04 | 323.39 ms | 52.2% bf16 MFU | 1625032 tok/s
step  374/19560 | loss 5.804579 (-1.32z)| norm 0.7360 (-0.80z)| lr 3.21e-04 | 322.23 ms | 52.4% bf16 MFU | 1625133 tok/s
step  375/19560 | loss 5.704128 (-2.09z)| norm 0.8319 (-0.28z)| lr 3.21e-04 | 322.31 ms | 52.4% bf16 MFU | 1625209 tok/s
step  376/19560 | loss 5.771087 (-1.53z)| norm 1.0515 (+0.90z)| lr 3.22e-04 | 322.87 ms | 52.3% bf16 MFU | 1625139 tok/s
step  377/19560 | loss 5.762410 (-1.57z)| norm 1.1884 (+1.60z)| lr 3.23e-04 | 322.73 ms | 52.3% bf16 MFU | 1625110 tok/s
step  378/19560 | loss 5.763268 (-1.55z)| norm 0.8333 (-0.31z)| lr 3.24e-04 | 322.14 ms | 52.4% bf16 MFU | 1625230 tok/s
step  379/19560 | loss 5.745463 (-1.66z)| norm 0.8395 (-0.28z)| lr 3.25e-04 | 322.92 ms | 52.3% bf16 MFU | 1625146 tok/s
step  380/19560 | loss 5.750226 (-1.60z)| norm 0.7997 (-0.49z)| lr 3.26e-04 | 322.41 ms | 52.3% bf16 MFU | 1625196 tok/s
step  381/19560 | loss 5.736010 (-1.69z)| norm 0.9710 (+0.43z)| lr 3.27e-04 | 323.11 ms | 52.2% bf16 MFU | 1625068 tok/s
step  382/19560 | loss 5.749287 (-1.55z)| norm 1.0330 (+0.83z)| lr 3.27e-04 | 322.53 ms | 52.3% bf16 MFU | 1625093 tok/s
step  383/19560 | loss 5.698438 (-1.93z)| norm 1.0423 (+0.87z)| lr 3.28e-04 | 323.13 ms | 52.2% bf16 MFU | 1624964 tok/s
step  384/19560 | loss 5.784569 (-1.23z)| norm 0.9105 (+0.12z)| lr 3.29e-04 | 322.21 ms | 52.4% bf16 MFU | 1625073 tok/s
step  385/19560 | loss 5.764462 (-1.37z)| norm 1.1218 (+1.30z)| lr 3.30e-04 | 322.63 ms | 52.3% bf16 MFU | 1625072 tok/s
step  386/19560 | loss 5.696859 (-1.88z)| norm 0.8176 (-0.41z)| lr 3.31e-04 | 323.06 ms | 52.2% bf16 MFU | 1624963 tok/s
step  387/19560 | loss 5.697162 (-1.84z)| norm 0.7666 (-0.70z)| lr 3.32e-04 | 322.61 ms | 52.3% bf16 MFU | 1624972 tok/s
step  388/19560 | loss 5.683781 (-1.91z)| norm 0.6920 (-1.12z)| lr 3.33e-04 | 323.19 ms | 52.2% bf16 MFU | 1624834 tok/s
step  389/19560 | loss 5.739658 (-1.45z)| norm 0.7381 (-0.85z)| lr 3.33e-04 | 323.14 ms | 52.2% bf16 MFU | 1624716 tok/s
step  390/19560 | loss 5.701883 (-1.72z)| norm 0.8968 (+0.04z)| lr 3.34e-04 | 322.34 ms | 52.4% bf16 MFU | 1624806 tok/s
step  391/19560 | loss 5.746603 (-1.34z)| norm 1.2453 (+1.96z)| lr 3.35e-04 | 322.76 ms | 52.3% bf16 MFU | 1624786 tok/s
step  392/19560 | loss 5.709377 (-1.61z)| norm 1.1252 (+1.28z)| lr 3.36e-04 | 322.78 ms | 52.3% bf16 MFU | 1624762 tok/s
step  393/19560 | loss 5.773208 (-1.09z)| norm 1.5690 (+3.51z)| lr 3.37e-04 | 322.15 ms | 52.4% bf16 MFU | 1624898 tok/s
step  394/19560 | loss 5.736277 (-1.36z)| norm 1.0398 (+0.72z)| lr 3.38e-04 | 322.40 ms | 52.3% bf16 MFU | 1624963 tok/s
step  395/19560 | loss 5.697823 (-1.64z)| norm 0.7677 (-0.72z)| lr 3.39e-04 | 321.79 ms | 52.4% bf16 MFU | 1625180 tok/s
step  396/19560 | loss 5.719881 (-1.44z)| norm 0.6493 (-1.34z)| lr 3.39e-04 | 323.02 ms | 52.2% bf16 MFU | 1625075 tok/s
step  397/19560 | loss 5.676668 (-1.76z)| norm 0.7460 (-0.82z)| lr 3.40e-04 | 322.49 ms | 52.3% bf16 MFU | 1625110 tok/s
step  398/19560 | loss 5.699487 (-1.56z)| norm 0.8371 (-0.34z)| lr 3.41e-04 | 322.50 ms | 52.3% bf16 MFU | 1625140 tok/s
step  399/19560 | loss 5.721134 (-1.36z)| norm 1.4316 (+2.69z)| lr 3.42e-04 | 322.62 ms | 52.3% bf16 MFU | 1625138 tok/s
step  400/19560 | loss 5.747796 (-1.13z)| norm 0.9091 (-0.00z)| lr 3.43e-04 | 322.58 ms | 52.3% bf16 MFU | 1625146 tok/s
step  401/19560 | loss 5.645664 (-1.92z)| norm 0.9908 (+0.41z)| lr 3.44e-04 | 323.21 ms | 52.2% bf16 MFU | 1624994 tok/s
step  402/19560 | loss 5.732313 (-1.21z)| norm 1.5178 (+3.03z)| lr 3.45e-04 | 322.52 ms | 52.3% bf16 MFU | 1625025 tok/s
step  403/19560 | loss 5.699378 (-1.45z)| norm 0.8086 (-0.56z)| lr 3.45e-04 | 322.40 ms | 52.3% bf16 MFU | 1625083 tok/s
step  404/19560 | loss 5.608023 (-2.14z)| norm 1.0548 (+0.68z)| lr 3.46e-04 | 321.87 ms | 52.4% bf16 MFU | 1625273 tok/s
step  405/19560 | loss 5.708708 (-1.32z)| norm 0.9836 (+0.31z)| lr 3.47e-04 | 322.54 ms | 52.3% bf16 MFU | 1625285 tok/s
step  406/19560 | loss 5.652881 (-1.73z)| norm 0.8873 (-0.18z)| lr 3.48e-04 | 322.63 ms | 52.3% bf16 MFU | 1625273 tok/s
step  407/19560 | loss 5.627907 (-1.90z)| norm 0.7628 (-0.82z)| lr 3.49e-04 | 322.04 ms | 52.4% bf16 MFU | 1625410 tok/s
step  408/19560 | loss 5.711045 (-1.22z)| norm 0.7968 (-0.64z)| lr 3.50e-04 | 322.55 ms | 52.3% bf16 MFU | 1625410 tok/s
step  409/19560 | loss 5.664342 (-1.56z)| norm 0.7079 (-1.08z)| lr 3.51e-04 | 322.55 ms | 52.3% bf16 MFU | 1625411 tok/s
step  410/19560 | loss 5.633265 (-1.80z)| norm 0.7716 (-0.74z)| lr 3.51e-04 | 322.59 ms | 52.3% bf16 MFU | 1625403 tok/s
step  411/19560 | loss 5.588402 (-2.12z)| norm 0.8159 (-0.52z)| lr 3.52e-04 | 322.73 ms | 52.3% bf16 MFU | 1625360 tok/s
step  412/19560 | loss 5.705221 (-1.17z)| norm 0.9973 (+0.41z)| lr 3.53e-04 | 322.65 ms | 52.3% bf16 MFU | 1625339 tok/s
step  413/19560 | loss 5.640998 (-1.66z)| norm 0.8342 (-0.43z)| lr 3.54e-04 | 322.74 ms | 52.3% bf16 MFU | 1625298 tok/s
step  414/19560 | loss 5.599078 (-1.95z)| norm 0.8024 (-0.59z)| lr 3.55e-04 | 322.95 ms | 52.3% bf16 MFU | 1625205 tok/s
step  415/19560 | loss 5.605090 (-1.87z)| norm 0.8830 (-0.18z)| lr 3.56e-04 | 323.21 ms | 52.2% bf16 MFU | 1625051 tok/s
step  416/19560 | loss 5.651168 (-1.48z)| norm 1.2966 (+1.92z)| lr 3.57e-04 | 322.87 ms | 52.3% bf16 MFU | 1624991 tok/s
step  417/19560 | loss 5.683066 (-1.21z)| norm 1.0212 (+0.50z)| lr 3.57e-04 | 322.12 ms | 52.4% bf16 MFU | 1625122 tok/s
step  418/19560 | loss 5.687293 (-1.16z)| norm 1.0925 (+0.85z)| lr 3.58e-04 | 322.51 ms | 52.3% bf16 MFU | 1625149 tok/s
step  419/19560 | loss 5.656088 (-1.39z)| norm 1.0495 (+0.62z)| lr 3.59e-04 | 322.91 ms | 52.3% bf16 MFU | 1625074 tok/s
step  420/19560 | loss 5.648342 (-1.43z)| norm 1.2059 (+1.40z)| lr 3.60e-04 | 323.38 ms | 52.2% bf16 MFU | 1624884 tok/s
step  421/19560 | loss 5.696465 (-1.03z)| norm 0.9834 (+0.25z)| lr 3.61e-04 | 322.54 ms | 52.3% bf16 MFU | 1624914 tok/s
step  422/19560 | loss 5.581553 (-1.92z)| norm 0.7401 (-0.98z)| lr 3.62e-04 | 323.15 ms | 52.2% bf16 MFU | 1624790 tok/s
step  423/19560 | loss 5.582055 (-1.89z)| norm 0.6630 (-1.37z)| lr 3.63e-04 | 323.05 ms | 52.2% bf16 MFU | 1624696 tok/s
step  424/19560 | loss 5.627133 (-1.50z)| norm 0.8760 (-0.28z)| lr 3.63e-04 | 323.14 ms | 52.2% bf16 MFU | 1624586 tok/s
step  425/19560 | loss 5.599741 (-1.69z)| norm 1.0957 (+0.84z)| lr 3.64e-04 | 322.65 ms | 52.3% bf16 MFU | 1624604 tok/s
step  426/19560 | loss 5.640201 (-1.34z)| norm 1.1179 (+0.94z)| lr 3.65e-04 | 322.48 ms | 52.3% bf16 MFU | 1624665 tok/s
step  427/19560 | loss 5.644914 (-1.29z)| norm 0.9126 (-0.11z)| lr 3.66e-04 | 323.02 ms | 52.2% bf16 MFU | 1624584 tok/s
step  428/19560 | loss 5.586823 (-1.73z)| norm 0.8980 (-0.17z)| lr 3.67e-04 | 322.88 ms | 52.3% bf16 MFU | 1624544 tok/s
step  429/19560 | loss 5.594231 (-1.64z)| norm 0.7729 (-0.81z)| lr 3.68e-04 | 322.63 ms | 52.3% bf16 MFU | 1624568 tok/s
step  430/19560 | loss 5.581963 (-1.71z)| norm 0.7826 (-0.75z)| lr 3.69e-04 | 322.55 ms | 52.3% bf16 MFU | 1624613 tok/s
step  431/19560 | loss 5.592231 (-1.60z)| norm 0.8492 (-0.41z)| lr 3.69e-04 | 323.10 ms | 52.2% bf16 MFU | 1624516 tok/s
step  432/19560 | loss 5.555316 (-1.86z)| norm 0.7208 (-1.07z)| lr 3.70e-04 | 322.17 ms | 52.4% bf16 MFU | 1624658 tok/s
step  433/19560 | loss 5.549290 (-1.88z)| norm 0.7836 (-0.74z)| lr 3.71e-04 | 322.08 ms | 52.4% bf16 MFU | 1624815 tok/s
step  434/19560 | loss 5.597592 (-1.47z)| norm 0.9018 (-0.13z)| lr 3.72e-04 | 323.02 ms | 52.2% bf16 MFU | 1624728 tok/s
step  435/19560 | loss 5.595928 (-1.46z)| norm 1.1269 (+1.01z)| lr 3.73e-04 | 322.68 ms | 52.3% bf16 MFU | 1624731 tok/s
step  436/19560 | loss 5.604266 (-1.38z)| norm 1.0030 (+0.37z)| lr 3.74e-04 | 322.75 ms | 52.3% bf16 MFU | 1624717 tok/s
step  437/19560 | loss 5.503093 (-2.14z)| norm 0.8563 (-0.38z)| lr 3.75e-04 | 322.81 ms | 52.3% bf16 MFU | 1624687 tok/s
step  438/19560 | loss 5.637400 (-1.08z)| norm 0.9635 (+0.18z)| lr 3.75e-04 | 323.93 ms | 52.1% bf16 MFU | 1624379 tok/s
step  439/19560 | loss 5.548215 (-1.79z)| norm 1.1969 (+1.37z)| lr 3.76e-04 | 322.81 ms | 52.3% bf16 MFU | 1624368 tok/s
step  440/19560 | loss 5.570685 (-1.59z)| norm 0.8225 (-0.54z)| lr 3.77e-04 | 323.64 ms | 52.1% bf16 MFU | 1624150 tok/s
step  441/19560 | loss 5.535388 (-1.86z)| norm 0.8110 (-0.59z)| lr 3.78e-04 | 321.95 ms | 52.4% bf16 MFU | 1624365 tok/s
step  442/19560 | loss 5.551450 (-1.71z)| norm 0.8799 (-0.23z)| lr 3.79e-04 | 322.70 ms | 52.3% bf16 MFU | 1624382 tok/s
step  443/19560 | loss 5.588181 (-1.38z)| norm 0.7774 (-0.75z)| lr 3.80e-04 | 322.85 ms | 52.3% bf16 MFU | 1624359 tok/s
step  444/19560 | loss 5.493528 (-2.14z)| norm 0.7301 (-0.98z)| lr 3.81e-04 | 322.32 ms | 52.4% bf16 MFU | 1624472 tok/s
step  445/19560 | loss 5.515854 (-1.92z)| norm 0.9461 (+0.15z)| lr 3.81e-04 | 322.63 ms | 52.3% bf16 MFU | 1624500 tok/s
step  446/19560 | loss 5.569407 (-1.45z)| norm 1.1722 (+1.31z)| lr 3.82e-04 | 322.84 ms | 52.3% bf16 MFU | 1624474 tok/s
step  447/19560 | loss 5.520355 (-1.83z)| norm 0.7020 (-1.14z)| lr 3.83e-04 | 322.31 ms | 52.4% bf16 MFU | 1624584 tok/s
step  448/19560 | loss 5.540468 (-1.63z)| norm 0.7891 (-0.69z)| lr 3.84e-04 | 322.80 ms | 52.3% bf16 MFU | 1624565 tok/s
step  449/19560 | loss 5.607126 (-1.06z)| norm 0.8632 (-0.31z)| lr 3.85e-04 | 322.83 ms | 52.3% bf16 MFU | 1624539 tok/s
step  450/19560 | loss 5.553960 (-1.49z)| norm 0.9792 (+0.28z)| lr 3.86e-04 | 321.82 ms | 52.4% bf16 MFU | 1624769 tok/s
step  451/19560 | loss 5.591267 (-1.15z)| norm 1.0273 (+0.53z)| lr 3.87e-04 | 322.78 ms | 52.3% bf16 MFU | 1624745 tok/s
step  452/19560 | loss 5.571222 (-1.30z)| norm 0.8060 (-0.63z)| lr 3.87e-04 | 322.75 ms | 52.3% bf16 MFU | 1624730 tok/s
step  453/19560 | loss 5.544106 (-1.51z)| norm 0.7530 (-0.90z)| lr 3.88e-04 | 322.62 ms | 52.3% bf16 MFU | 1624748 tok/s
step  454/19560 | loss 5.467613 (-2.11z)| norm 0.7107 (-1.11z)| lr 3.89e-04 | 323.97 ms | 52.1% bf16 MFU | 1624427 tok/s
step  455/19560 | loss 5.529677 (-1.56z)| norm 0.7287 (-1.01z)| lr 3.90e-04 | 321.97 ms | 52.4% bf16 MFU | 1624623 tok/s
step  456/19560 | loss 5.523317 (-1.60z)| norm 0.8351 (-0.43z)| lr 3.91e-04 | 322.79 ms | 52.3% bf16 MFU | 1624605 tok/s
step  457/19560 | loss 5.563190 (-1.25z)| norm 0.8202 (-0.51z)| lr 3.92e-04 | 322.71 ms | 52.3% bf16 MFU | 1624608 tok/s
step  458/19560 | loss 5.500129 (-1.75z)| norm 0.8308 (-0.45z)| lr 3.93e-04 | 322.10 ms | 52.4% bf16 MFU | 1624762 tok/s
step  459/19560 | loss 5.558644 (-1.25z)| norm 1.1058 (+1.02z)| lr 3.93e-04 | 322.46 ms | 52.3% bf16 MFU | 1624819 tok/s
step  460/19560 | loss 5.532295 (-1.45z)| norm 0.9210 (+0.02z)| lr 3.94e-04 | 322.83 ms | 52.3% bf16 MFU | 1624781 tok/s
step  461/19560 | loss 5.520588 (-1.53z)| norm 1.1248 (+1.11z)| lr 3.95e-04 | 323.08 ms | 52.2% bf16 MFU | 1624680 tok/s
step  462/19560 | loss 5.520423 (-1.51z)| norm 1.0647 (+0.78z)| lr 3.96e-04 | 322.86 ms | 52.3% bf16 MFU | 1624639 tok/s
step  463/19560 | loss 5.649081 (-0.39z)| norm 0.9920 (+0.40z)| lr 3.97e-04 | 323.10 ms | 52.2% bf16 MFU | 1624541 tok/s
step  464/19560 | loss 5.549935 (-1.24z)| norm 1.0482 (+0.72z)| lr 3.98e-04 | 322.02 ms | 52.4% bf16 MFU | 1624721 tok/s
step  465/19560 | loss 5.462357 (-1.97z)| norm 0.9506 (+0.18z)| lr 3.99e-04 | 322.85 ms | 52.3% bf16 MFU | 1624682 tok/s
step  466/19560 | loss 5.453300 (-2.01z)| norm 0.8609 (-0.31z)| lr 3.99e-04 | 323.04 ms | 52.2% bf16 MFU | 1624598 tok/s
step  467/19560 | loss 5.465189 (-1.89z)| norm 0.8461 (-0.39z)| lr 4.00e-04 | 322.37 ms | 52.4% bf16 MFU | 1624685 tok/s
step  468/19560 | loss 5.469641 (-1.81z)| norm 0.7119 (-1.12z)| lr 4.01e-04 | 322.45 ms | 52.3% bf16 MFU | 1624748 tok/s
step  469/19560 | loss 5.452538 (-1.92z)| norm 0.7696 (-0.80z)| lr 4.02e-04 | 322.76 ms | 52.3% bf16 MFU | 1624730 tok/s
step  470/19560 | loss 5.530628 (-1.23z)| norm 0.8799 (-0.21z)| lr 4.03e-04 | 322.49 ms | 52.3% bf16 MFU | 1624780 tok/s
step  471/19560 | loss 5.481446 (-1.63z)| norm 0.7972 (-0.67z)| lr 4.04e-04 | 323.68 ms | 52.1% bf16 MFU | 1624530 tok/s
step  472/19560 | loss 5.426326 (-2.06z)| norm 0.8624 (-0.32z)| lr 4.05e-04 | 323.12 ms | 52.2% bf16 MFU | 1624432 tok/s
step  473/19560 | loss 5.470809 (-1.66z)| norm 1.0298 (+0.59z)| lr 4.05e-04 | 322.15 ms | 52.4% bf16 MFU | 1624583 tok/s
step  474/19560 | loss 5.450632 (-1.81z)| norm 0.9982 (+0.41z)| lr 4.06e-04 | 322.81 ms | 52.3% bf16 MFU | 1624560 tok/s
step  475/19560 | loss 5.466815 (-1.64z)| norm 1.1457 (+1.21z)| lr 4.07e-04 | 322.47 ms | 52.3% bf16 MFU | 1624624 tok/s
step  476/19560 | loss 5.444256 (-1.80z)| norm 0.7032 (-1.19z)| lr 4.08e-04 | 322.35 ms | 52.4% bf16 MFU | 1624716 tok/s
step  477/19560 | loss 5.514423 (-1.18z)| norm 0.7536 (-0.90z)| lr 4.09e-04 | 323.15 ms | 52.2% bf16 MFU | 1624603 tok/s
step  478/19560 | loss 5.500505 (-1.29z)| norm 0.7322 (-1.00z)| lr 4.10e-04 | 322.61 ms | 52.3% bf16 MFU | 1624631 tok/s
step  479/19560 | loss 5.428710 (-1.88z)| norm 0.8018 (-0.61z)| lr 4.11e-04 | 322.65 ms | 52.3% bf16 MFU | 1624646 tok/s
step  480/19560 | loss 5.412281 (-2.00z)| norm 0.8902 (-0.11z)| lr 4.11e-04 | 322.85 ms | 52.3% bf16 MFU | 1624609 tok/s
step  481/19560 | loss 5.470517 (-1.47z)| norm 0.7932 (-0.66z)| lr 4.12e-04 | 322.61 ms | 52.3% bf16 MFU | 1624636 tok/s
step  482/19560 | loss 5.462602 (-1.51z)| norm 0.7694 (-0.79z)| lr 4.13e-04 | 322.75 ms | 52.3% bf16 MFU | 1624625 tok/s
step  483/19560 | loss 5.455883 (-1.55z)| norm 0.9316 (+0.11z)| lr 4.14e-04 | 322.65 ms | 52.3% bf16 MFU | 1624641 tok/s
step  484/19560 | loss 5.427275 (-1.76z)| norm 0.9896 (+0.44z)| lr 4.15e-04 | 322.91 ms | 52.3% bf16 MFU | 1624591 tok/s
step  485/19560 | loss 5.453872 (-1.51z)| norm 1.0460 (+0.75z)| lr 4.16e-04 | 323.00 ms | 52.3% bf16 MFU | 1624520 tok/s
step  486/19560 | loss 5.480731 (-1.26z)| norm 0.9344 (+0.12z)| lr 4.17e-04 | 321.80 ms | 52.4% bf16 MFU | 1624755 tok/s
step  487/19560 | loss 5.421312 (-1.76z)| norm 0.7281 (-1.02z)| lr 4.17e-04 | 322.83 ms | 52.3% bf16 MFU | 1624718 tok/s
step  488/19560 | loss 5.495105 (-1.10z)| norm 0.7017 (-1.15z)| lr 4.18e-04 | 323.71 ms | 52.1% bf16 MFU | 1624464 tok/s
step  489/19560 | loss 5.379706 (-2.09z)| norm 0.6671 (-1.32z)| lr 4.19e-04 | 322.86 ms | 52.3% bf16 MFU | 1624434 tok/s
step  490/19560 | loss 5.452560 (-1.43z)| norm 0.7446 (-0.88z)| lr 4.20e-04 | 321.79 ms | 52.4% bf16 MFU | 1624678 tok/s
step  491/19560 | loss 5.411655 (-1.77z)| norm 0.7727 (-0.71z)| lr 4.21e-04 | 322.73 ms | 52.3% bf16 MFU | 1624671 tok/s
step  492/19560 | loss 5.403941 (-1.81z)| norm 0.8306 (-0.39z)| lr 4.22e-04 | 322.88 ms | 52.3% bf16 MFU | 1624626 tok/s
step  493/19560 | loss 5.394545 (-1.85z)| norm 0.8389 (-0.36z)| lr 4.23e-04 | 323.19 ms | 52.2% bf16 MFU | 1624506 tok/s
step  494/19560 | loss 5.409517 (-1.69z)| norm 1.0450 (+0.78z)| lr 4.23e-04 | 322.41 ms | 52.3% bf16 MFU | 1624587 tok/s
step  495/19560 | loss 5.418550 (-1.58z)| norm 1.1251 (+1.22z)| lr 4.24e-04 | 322.23 ms | 52.4% bf16 MFU | 1624710 tok/s
step  496/19560 | loss 5.394727 (-1.76z)| norm 0.8615 (-0.27z)| lr 4.25e-04 | 322.73 ms | 52.3% bf16 MFU | 1624701 tok/s
step  497/19560 | loss 5.406218 (-1.63z)| norm 0.8084 (-0.58z)| lr 4.26e-04 | 323.01 ms | 52.2% bf16 MFU | 1624623 tok/s
step  498/19560 | loss 5.419038 (-1.49z)| norm 0.8683 (-0.24z)| lr 4.27e-04 | 322.52 ms | 52.3% bf16 MFU | 1624671 tok/s
step  499/19560 | loss 5.415681 (-1.50z)| norm 0.8377 (-0.42z)| lr 4.28e-04 | 322.47 ms | 52.3% bf16 MFU | 1624730 tok/s
step  500/19560 | loss 5.391510 (-1.69z)| norm 0.6522 (-1.48z)| lr 4.29e-04 | 322.08 ms | 52.4% bf16 MFU | 1624883 tok/s
val loss 5.364819
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2455/10042 = 0.244473
step  501/19560 | loss 5.498502 (-0.73z)| norm 0.5839 (-1.84z)| lr 4.29e-04 | 321.88 ms | 52.4% bf16 MFU | 1625080 tok/s
step  502/19560 | loss 5.364617 (-1.89z)| norm 0.5607 (-1.94z)| lr 4.30e-04 | 322.13 ms | 52.4% bf16 MFU | 1625205 tok/s
step  503/19560 | loss 5.347536 (-2.00z)| norm 0.6226 (-1.57z)| lr 4.31e-04 | 323.36 ms | 52.2% bf16 MFU | 1625015 tok/s
step  504/19560 | loss 5.347878 (-1.96z)| norm 0.7357 (-0.91z)| lr 4.32e-04 | 322.41 ms | 52.3% bf16 MFU | 1625073 tok/s
step  505/19560 | loss 5.418486 (-1.32z)| norm 0.8774 (-0.10z)| lr 4.33e-04 | 322.33 ms | 52.4% bf16 MFU | 1625147 tok/s
step  506/19560 | loss 5.340372 (-1.97z)| norm 0.9101 (+0.08z)| lr 4.34e-04 | 322.40 ms | 52.3% bf16 MFU | 1625201 tok/s
step  507/19560 | loss 5.350870 (-1.85z)| norm 1.0830 (+1.05z)| lr 4.35e-04 | 322.33 ms | 52.4% bf16 MFU | 1625268 tok/s
step  508/19560 | loss 5.441101 (-1.05z)| norm 1.2948 (+2.19z)| lr 4.35e-04 | 321.97 ms | 52.4% bf16 MFU | 1625423 tok/s
step  509/19560 | loss 5.356126 (-1.76z)| norm 0.8207 (-0.45z)| lr 4.36e-04 | 322.38 ms | 52.4% bf16 MFU | 1625466 tok/s
step  510/19560 | loss 5.409202 (-1.28z)| norm 0.7187 (-1.00z)| lr 4.37e-04 | 322.53 ms | 52.3% bf16 MFU | 1625470 tok/s
step  511/19560 | loss 5.337696 (-1.87z)| norm 0.7167 (-0.99z)| lr 4.38e-04 | 322.30 ms | 52.4% bf16 MFU | 1625532 tok/s
step  512/19560 | loss 5.302006 (-2.14z)| norm 0.8005 (-0.52z)| lr 4.39e-04 | 322.52 ms | 52.3% bf16 MFU | 1625535 tok/s
step  513/19560 | loss 5.340942 (-1.78z)| norm 0.9039 (+0.06z)| lr 4.40e-04 | 322.43 ms | 52.3% bf16 MFU | 1625562 tok/s
step  514/19560 | loss 5.346510 (-1.70z)| norm 0.8360 (-0.32z)| lr 4.41e-04 | 323.08 ms | 52.2% bf16 MFU | 1625423 tok/s
step  515/19560 | loss 5.381519 (-1.37z)| norm 0.9962 (+0.57z)| lr 4.41e-04 | 322.35 ms | 52.4% bf16 MFU | 1625474 tok/s
step  516/19560 | loss 5.420273 (-1.02z)| norm 1.5913 (+3.67z)| lr 4.42e-04 | 322.00 ms | 52.4% bf16 MFU | 1625612 tok/s
step  517/19560 | loss 5.352194 (-1.59z)| norm 0.9089 (+0.03z)| lr 4.43e-04 | 322.75 ms | 52.3% bf16 MFU | 1625554 tok/s
step  518/19560 | loss 5.368319 (-1.43z)| norm 0.8914 (-0.06z)| lr 4.44e-04 | 322.93 ms | 52.3% bf16 MFU | 1625452 tok/s
step  519/19560 | loss 5.309851 (-1.91z)| norm 0.9037 (+0.02z)| lr 4.45e-04 | 322.38 ms | 52.4% bf16 MFU | 1625493 tok/s
step  520/19560 | loss 5.326802 (-1.73z)| norm 0.8177 (-0.44z)| lr 4.46e-04 | 322.36 ms | 52.4% bf16 MFU | 1625539 tok/s
step  521/19560 | loss 5.380197 (-1.25z)| norm 0.8519 (-0.23z)| lr 4.47e-04 | 322.04 ms | 52.4% bf16 MFU | 1625664 tok/s
step  522/19560 | loss 5.286824 (-2.04z)| norm 0.8887 (-0.02z)| lr 4.47e-04 | 322.57 ms | 52.3% bf16 MFU | 1625648 tok/s
step  523/19560 | loss 5.339777 (-1.55z)| norm 0.9630 (+0.40z)| lr 4.48e-04 | 323.13 ms | 52.2% bf16 MFU | 1625493 tok/s
step  524/19560 | loss 5.276851 (-2.06z)| norm 1.1618 (+1.53z)| lr 4.49e-04 | 323.45 ms | 52.2% bf16 MFU | 1625264 tok/s
step  525/19560 | loss 5.331431 (-1.56z)| norm 0.8211 (-0.44z)| lr 4.50e-04 | 321.93 ms | 52.4% bf16 MFU | 1625430 tok/s
step  526/19560 | loss 5.305256 (-1.76z)| norm 0.7615 (-0.78z)| lr 4.51e-04 | 322.70 ms | 52.3% bf16 MFU | 1625394 tok/s
step  527/19560 | loss 5.337534 (-1.46z)| norm 0.7519 (-0.83z)| lr 4.52e-04 | 323.07 ms | 52.2% bf16 MFU | 1625265 tok/s
step  528/19560 | loss 5.365855 (-1.20z)| norm 0.6454 (-1.45z)| lr 4.53e-04 | 323.05 ms | 52.2% bf16 MFU | 1625149 tok/s
step  529/19560 | loss 5.215557 (-2.46z)| norm 0.6524 (-1.38z)| lr 4.53e-04 | 322.72 ms | 52.3% bf16 MFU | 1625121 tok/s
step  530/19560 | loss 5.355121 (-1.23z)| norm 0.6029 (-1.71z)| lr 4.54e-04 | 322.60 ms | 52.3% bf16 MFU | 1625124 tok/s
step  531/19560 | loss 5.309175 (-1.61z)| norm 0.6232 (-1.56z)| lr 4.55e-04 | 322.21 ms | 52.4% bf16 MFU | 1625226 tok/s
step  532/19560 | loss 5.286662 (-1.78z)| norm 0.6796 (-1.19z)| lr 4.56e-04 | 322.45 ms | 52.3% bf16 MFU | 1625262 tok/s
step  533/19560 | loss 5.276320 (-1.84z)| norm 0.8257 (-0.30z)| lr 4.57e-04 | 322.93 ms | 52.3% bf16 MFU | 1625176 tok/s
step  534/19560 | loss 5.301159 (-1.59z)| norm 1.0123 (+0.84z)| lr 4.58e-04 | 323.05 ms | 52.2% bf16 MFU | 1625064 tok/s
step  535/19560 | loss 5.258131 (-1.93z)| norm 0.9188 (+0.26z)| lr 4.59e-04 | 322.45 ms | 52.3% bf16 MFU | 1625109 tok/s
step  536/19560 | loss 5.260259 (-1.88z)| norm 0.8472 (-0.18z)| lr 4.59e-04 | 323.31 ms | 52.2% bf16 MFU | 1624935 tok/s
step  537/19560 | loss 5.205267 (-2.31z)| norm 0.8731 (-0.03z)| lr 4.60e-04 | 322.57 ms | 52.3% bf16 MFU | 1624955 tok/s
step  538/19560 | loss 5.237278 (-1.99z)| norm 0.9583 (+0.48z)| lr 4.61e-04 | 322.99 ms | 52.3% bf16 MFU | 1624869 tok/s
step  539/19560 | loss 5.273420 (-1.65z)| norm 0.7446 (-0.82z)| lr 4.62e-04 | 322.69 ms | 52.3% bf16 MFU | 1624863 tok/s
step  540/19560 | loss 5.300308 (-1.40z)| norm 0.6498 (-1.38z)| lr 4.63e-04 | 322.80 ms | 52.3% bf16 MFU | 1624828 tok/s
step  541/19560 | loss 5.199187 (-2.22z)| norm 0.6230 (-1.52z)| lr 4.64e-04 | 322.95 ms | 52.3% bf16 MFU | 1624758 tok/s
step  542/19560 | loss 5.224133 (-1.96z)| norm 0.5510 (-1.92z)| lr 4.65e-04 | 322.75 ms | 52.3% bf16 MFU | 1624742 tok/s
step  543/19560 | loss 5.257647 (-1.65z)| norm 0.5791 (-1.71z)| lr 4.65e-04 | 322.90 ms | 52.3% bf16 MFU | 1624688 tok/s
step  544/19560 | loss 5.188994 (-2.17z)| norm 0.7344 (-0.79z)| lr 4.66e-04 | 322.86 ms | 52.3% bf16 MFU | 1624648 tok/s
step  545/19560 | loss 5.323084 (-1.05z)| norm 1.1945 (+1.95z)| lr 4.67e-04 | 323.01 ms | 52.3% bf16 MFU | 1624573 tok/s
step  546/19560 | loss 5.234902 (-1.77z)| norm 1.0489 (+1.09z)| lr 4.68e-04 | 322.20 ms | 52.4% bf16 MFU | 1624706 tok/s
step  547/19560 | loss 5.212219 (-1.92z)| norm 0.9813 (+0.69z)| lr 4.69e-04 | 322.61 ms | 52.3% bf16 MFU | 1624728 tok/s
step  548/19560 | loss 5.345063 (-0.80z)| norm 0.9062 (+0.26z)| lr 4.70e-04 | 322.78 ms | 52.3% bf16 MFU | 1624706 tok/s
step  549/19560 | loss 5.245448 (-1.62z)| norm 0.9232 (+0.36z)| lr 4.71e-04 | 321.98 ms | 52.4% bf16 MFU | 1624886 tok/s
step  550/19560 | loss 5.260484 (-1.47z)| norm 0.9321 (+0.41z)| lr 4.71e-04 | 323.61 ms | 52.2% bf16 MFU | 1624648 tok/s
step  551/19560 | loss 5.314529 (-1.00z)| norm 0.8480 (-0.11z)| lr 4.72e-04 | 322.58 ms | 52.3% bf16 MFU | 1624680 tok/s
step  552/19560 | loss 5.238575 (-1.62z)| norm 0.8240 (-0.26z)| lr 4.73e-04 | 323.08 ms | 52.2% bf16 MFU | 1624585 tok/s
step  553/19560 | loss 5.231218 (-1.65z)| norm 0.8802 (+0.10z)| lr 4.74e-04 | 322.30 ms | 52.4% bf16 MFU | 1624692 tok/s
step  554/19560 | loss 5.267057 (-1.33z)| norm 0.8607 (-0.01z)| lr 4.75e-04 | 322.95 ms | 52.3% bf16 MFU | 1624628 tok/s
step  555/19560 | loss 5.220951 (-1.70z)| norm 0.7869 (-0.46z)| lr 4.76e-04 | 322.96 ms | 52.3% bf16 MFU | 1624566 tok/s
step  556/19560 | loss 5.239261 (-1.52z)| norm 0.8243 (-0.23z)| lr 4.77e-04 | 322.62 ms | 52.3% bf16 MFU | 1624592 tok/s
step  557/19560 | loss 5.230048 (-1.57z)| norm 1.0265 (+1.02z)| lr 4.77e-04 | 322.48 ms | 52.3% bf16 MFU | 1624653 tok/s
step  558/19560 | loss 5.130980 (-2.36z)| norm 0.7765 (-0.54z)| lr 4.78e-04 | 323.15 ms | 52.2% bf16 MFU | 1624543 tok/s
step  559/19560 | loss 5.240596 (-1.41z)| norm 0.8398 (-0.14z)| lr 4.79e-04 | 322.94 ms | 52.3% bf16 MFU | 1624489 tok/s
step  560/19560 | loss 5.223023 (-1.53z)| norm 0.7135 (-0.93z)| lr 4.80e-04 | 322.40 ms | 52.3% bf16 MFU | 1624574 tok/s
step  561/19560 | loss 5.253553 (-1.26z)| norm 0.6145 (-1.52z)| lr 4.81e-04 | 322.89 ms | 52.3% bf16 MFU | 1624531 tok/s
step  562/19560 | loss 5.229512 (-1.44z)| norm 0.6276 (-1.42z)| lr 4.82e-04 | 323.56 ms | 52.2% bf16 MFU | 1624322 tok/s
step  563/19560 | loss 5.245800 (-1.28z)| norm 0.6034 (-1.54z)| lr 4.83e-04 | 322.43 ms | 52.3% bf16 MFU | 1624408 tok/s
step  564/19560 | loss 5.212085 (-1.55z)| norm 0.9239 (+0.43z)| lr 4.83e-04 | 322.47 ms | 52.3% bf16 MFU | 1624480 tok/s
step  565/19560 | loss 5.237493 (-1.31z)| norm 1.2477 (+2.35z)| lr 4.84e-04 | 323.87 ms | 52.1% bf16 MFU | 1624198 tok/s
step  566/19560 | loss 5.189203 (-1.70z)| norm 0.8181 (-0.23z)| lr 4.85e-04 | 323.11 ms | 52.2% bf16 MFU | 1624118 tok/s
step  567/19560 | loss 5.222032 (-1.40z)| norm 0.8756 (+0.13z)| lr 4.86e-04 | 322.67 ms | 52.3% bf16 MFU | 1624155 tok/s
step  568/19560 | loss 5.208960 (-1.49z)| norm 0.8070 (-0.29z)| lr 4.87e-04 | 322.80 ms | 52.3% bf16 MFU | 1624155 tok/s
step  569/19560 | loss 5.227763 (-1.31z)| norm 0.6797 (-1.05z)| lr 4.88e-04 | 322.35 ms | 52.4% bf16 MFU | 1624271 tok/s
step  570/19560 | loss 5.199591 (-1.52z)| norm 0.7698 (-0.50z)| lr 4.89e-04 | 323.25 ms | 52.2% bf16 MFU | 1624154 tok/s
step  571/19560 | loss 5.223817 (-1.30z)| norm 0.7924 (-0.36z)| lr 4.89e-04 | 322.38 ms | 52.4% bf16 MFU | 1624262 tok/s
step  572/19560 | loss 5.183462 (-1.62z)| norm 0.7297 (-0.74z)| lr 4.90e-04 | 323.16 ms | 52.2% bf16 MFU | 1624167 tok/s
step  573/19560 | loss 5.145094 (-1.90z)| norm 0.8880 (+0.22z)| lr 4.91e-04 | 322.73 ms | 52.3% bf16 MFU | 1624186 tok/s
step  574/19560 | loss 5.153665 (-1.80z)| norm 0.8206 (-0.17z)| lr 4.92e-04 | 322.98 ms | 52.3% bf16 MFU | 1624141 tok/s
step  575/19560 | loss 5.144991 (-1.84z)| norm 0.6598 (-1.17z)| lr 4.93e-04 | 323.05 ms | 52.2% bf16 MFU | 1624081 tok/s
step  576/19560 | loss 5.180297 (-1.52z)| norm 0.6297 (-1.33z)| lr 4.94e-04 | 322.84 ms | 52.3% bf16 MFU | 1624076 tok/s
step  577/19560 | loss 5.174501 (-1.55z)| norm 0.7124 (-0.82z)| lr 4.95e-04 | 322.93 ms | 52.3% bf16 MFU | 1624050 tok/s
step  578/19560 | loss 5.201692 (-1.30z)| norm 0.7491 (-0.58z)| lr 4.95e-04 | 322.73 ms | 52.3% bf16 MFU | 1624073 tok/s
step  579/19560 | loss 5.227182 (-1.07z)| norm 0.8329 (-0.06z)| lr 4.96e-04 | 323.23 ms | 52.2% bf16 MFU | 1623972 tok/s
step  580/19560 | loss 5.207070 (-1.23z)| norm 0.7012 (-0.86z)| lr 4.97e-04 | 322.71 ms | 52.3% bf16 MFU | 1624004 tok/s
step  581/19560 | loss 5.105211 (-2.07z)| norm 0.6148 (-1.38z)| lr 4.98e-04 | 322.66 ms | 52.3% bf16 MFU | 1624048 tok/s
step  582/19560 | loss 5.142806 (-1.71z)| norm 0.6275 (-1.29z)| lr 4.99e-04 | 323.08 ms | 52.2% bf16 MFU | 1623985 tok/s
step  583/19560 | loss 5.175977 (-1.41z)| norm 0.7438 (-0.59z)| lr 5.00e-04 | 322.85 ms | 52.3% bf16 MFU | 1623983 tok/s
step  584/19560 | loss 5.148100 (-1.62z)| norm 0.9102 (+0.42z)| lr 5.01e-04 | 322.62 ms | 52.3% bf16 MFU | 1624040 tok/s
step  585/19560 | loss 5.153560 (-1.56z)| norm 0.9601 (+0.72z)| lr 5.01e-04 | 322.80 ms | 52.3% bf16 MFU | 1624047 tok/s
step  586/19560 | loss 5.154264 (-1.53z)| norm 0.7470 (-0.57z)| lr 5.02e-04 | 323.02 ms | 52.2% bf16 MFU | 1623998 tok/s
step  587/19560 | loss 5.078993 (-2.14z)| norm 0.6308 (-1.26z)| lr 5.03e-04 | 323.06 ms | 52.2% bf16 MFU | 1623942 tok/s
step  588/19560 | loss 5.163724 (-1.39z)| norm 0.8836 (+0.28z)| lr 5.04e-04 | 323.09 ms | 52.2% bf16 MFU | 1623882 tok/s
step  589/19560 | loss 5.170393 (-1.31z)| norm 0.9619 (+0.77z)| lr 5.05e-04 | 322.74 ms | 52.3% bf16 MFU | 1623913 tok/s
step  590/19560 | loss 5.067248 (-2.16z)| norm 0.6952 (-0.85z)| lr 5.06e-04 | 322.46 ms | 52.3% bf16 MFU | 1624012 tok/s
step  591/19560 | loss 5.150664 (-1.45z)| norm 0.8111 (-0.13z)| lr 5.07e-04 | 323.34 ms | 52.2% bf16 MFU | 1623884 tok/s
step  592/19560 | loss 5.208635 (-0.92z)| norm 0.7689 (-0.38z)| lr 5.07e-04 | 322.35 ms | 52.4% bf16 MFU | 1624013 tok/s
step  593/19560 | loss 5.115358 (-1.73z)| norm 0.6106 (-1.34z)| lr 5.08e-04 | 323.02 ms | 52.2% bf16 MFU | 1623966 tok/s
step  594/19560 | loss 5.123180 (-1.63z)| norm 0.6507 (-1.08z)| lr 5.09e-04 | 323.03 ms | 52.2% bf16 MFU | 1623919 tok/s
step  595/19560 | loss 5.123184 (-1.60z)| norm 0.7032 (-0.75z)| lr 5.10e-04 | 322.96 ms | 52.3% bf16 MFU | 1623891 tok/s
step  596/19560 | loss 5.132890 (-1.49z)| norm 0.7276 (-0.60z)| lr 5.11e-04 | 322.46 ms | 52.3% bf16 MFU | 1623991 tok/s
step  597/19560 | loss 5.200877 (-0.87z)| norm 0.6916 (-0.81z)| lr 5.12e-04 | 323.59 ms | 52.2% bf16 MFU | 1623803 tok/s
step  598/19560 | loss 5.109354 (-1.67z)| norm 0.7024 (-0.74z)| lr 5.13e-04 | 322.33 ms | 52.4% bf16 MFU | 1623940 tok/s
step  599/19560 | loss 5.097095 (-1.75z)| norm 0.7637 (-0.36z)| lr 5.13e-04 | 322.58 ms | 52.3% bf16 MFU | 1624009 tok/s
step  600/19560 | loss 5.075758 (-1.90z)| norm 0.7418 (-0.49z)| lr 5.14e-04 | 323.54 ms | 52.2% bf16 MFU | 1623832 tok/s
step  601/19560 | loss 5.197525 (-0.81z)| norm 0.7172 (-0.63z)| lr 5.15e-04 | 323.35 ms | 52.2% bf16 MFU | 1623711 tok/s
step  602/19560 | loss 5.135144 (-1.35z)| norm 0.7520 (-0.40z)| lr 5.16e-04 | 322.95 ms | 52.3% bf16 MFU | 1623699 tok/s
step  603/19560 | loss 5.131846 (-1.36z)| norm 0.8895 (+0.47z)| lr 5.17e-04 | 323.13 ms | 52.2% bf16 MFU | 1623640 tok/s
step  604/19560 | loss 5.101715 (-1.60z)| norm 0.8449 (+0.18z)| lr 5.18e-04 | 322.29 ms | 52.4% bf16 MFU | 1623795 tok/s
step  605/19560 | loss 5.074262 (-1.82z)| norm 0.6858 (-0.82z)| lr 5.19e-04 | 322.85 ms | 52.3% bf16 MFU | 1623801 tok/s
step  606/19560 | loss 5.116573 (-1.43z)| norm 0.6640 (-0.95z)| lr 5.19e-04 | 323.40 ms | 52.2% bf16 MFU | 1623670 tok/s
step  607/19560 | loss 5.073135 (-1.79z)| norm 0.6640 (-0.94z)| lr 5.20e-04 | 323.92 ms | 52.1% bf16 MFU | 1623416 tok/s
step  608/19560 | loss 5.094081 (-1.57z)| norm 0.7195 (-0.58z)| lr 5.21e-04 | 322.62 ms | 52.3% bf16 MFU | 1623500 tok/s
step  609/19560 | loss 5.102177 (-1.48z)| norm 0.7199 (-0.57z)| lr 5.22e-04 | 322.69 ms | 52.3% bf16 MFU | 1623561 tok/s
step  610/19560 | loss 5.069654 (-1.75z)| norm 0.7280 (-0.52z)| lr 5.23e-04 | 322.83 ms | 52.3% bf16 MFU | 1623584 tok/s
step  611/19560 | loss 5.083541 (-1.60z)| norm 0.8477 (+0.23z)| lr 5.24e-04 | 322.98 ms | 52.3% bf16 MFU | 1623568 tok/s
step  612/19560 | loss 5.091235 (-1.50z)| norm 0.7500 (-0.37z)| lr 5.25e-04 | 322.98 ms | 52.3% bf16 MFU | 1623555 tok/s
step  613/19560 | loss 5.080273 (-1.58z)| norm 0.7085 (-0.62z)| lr 5.25e-04 | 323.44 ms | 52.2% bf16 MFU | 1623427 tok/s
step  614/19560 | loss 5.067672 (-1.68z)| norm 0.6610 (-0.91z)| lr 5.26e-04 | 323.37 ms | 52.2% bf16 MFU | 1623322 tok/s
step  615/19560 | loss 5.057717 (-1.74z)| norm 0.6492 (-0.97z)| lr 5.27e-04 | 323.33 ms | 52.2% bf16 MFU | 1623232 tok/s
step  616/19560 | loss 5.016659 (-2.08z)| norm 0.6081 (-1.22z)| lr 5.28e-04 | 322.92 ms | 52.3% bf16 MFU | 1623249 tok/s
step  617/19560 | loss 5.003469 (-2.15z)| norm 0.6443 (-0.99z)| lr 5.29e-04 | 323.26 ms | 52.2% bf16 MFU | 1623180 tok/s
step  618/19560 | loss 5.125568 (-1.03z)| norm 0.7191 (-0.52z)| lr 5.30e-04 | 322.66 ms | 52.3% bf16 MFU | 1623264 tok/s
step  619/19560 | loss 5.062023 (-1.59z)| norm 0.8893 (+0.54z)| lr 5.31e-04 | 323.14 ms | 52.2% bf16 MFU | 1623225 tok/s
step  620/19560 | loss 5.051396 (-1.66z)| norm 0.9007 (+0.61z)| lr 5.31e-04 | 323.00 ms | 52.3% bf16 MFU | 1623222 tok/s
step  621/19560 | loss 5.030831 (-1.81z)| norm 0.6909 (-0.70z)| lr 5.32e-04 | 322.33 ms | 52.4% bf16 MFU | 1623389 tok/s
step  622/19560 | loss 4.978292 (-2.24z)| norm 0.6073 (-1.20z)| lr 5.33e-04 | 323.03 ms | 52.2% bf16 MFU | 1623372 tok/s
step  623/19560 | loss 4.993465 (-2.06z)| norm 0.5829 (-1.34z)| lr 5.34e-04 | 322.82 ms | 52.3% bf16 MFU | 1623408 tok/s
step  624/19560 | loss 5.010024 (-1.88z)| norm 0.6075 (-1.17z)| lr 5.35e-04 | 322.95 ms | 52.3% bf16 MFU | 1623408 tok/s
step  625/19560 | loss 5.060742 (-1.41z)| norm 0.6416 (-0.94z)| lr 5.36e-04 | 322.86 ms | 52.3% bf16 MFU | 1623433 tok/s
step  626/19560 | loss 5.031288 (-1.64z)| norm 0.7701 (-0.13z)| lr 5.37e-04 | 322.73 ms | 52.3% bf16 MFU | 1623487 tok/s
step  627/19560 | loss 5.102491 (-0.99z)| norm 0.9538 (+1.01z)| lr 5.37e-04 | 322.95 ms | 52.3% bf16 MFU | 1623484 tok/s
step  628/19560 | loss 5.092971 (-1.07z)| norm 0.9394 (+0.91z)| lr 5.38e-04 | 322.86 ms | 52.3% bf16 MFU | 1623504 tok/s
step  629/19560 | loss 5.068263 (-1.29z)| norm 0.9414 (+0.91z)| lr 5.39e-04 | 322.72 ms | 52.3% bf16 MFU | 1623559 tok/s
step  630/19560 | loss 5.060086 (-1.34z)| norm 0.9682 (+1.06z)| lr 5.40e-04 | 322.85 ms | 52.3% bf16 MFU | 1623578 tok/s
step  631/19560 | loss 5.093626 (-1.01z)| norm 0.9878 (+1.17z)| lr 5.41e-04 | 323.05 ms | 52.2% bf16 MFU | 1623544 tok/s
step  632/19560 | loss 5.079383 (-1.13z)| norm 0.8824 (+0.49z)| lr 5.42e-04 | 322.90 ms | 52.3% bf16 MFU | 1623551 tok/s
step  633/19560 | loss 5.052071 (-1.37z)| norm 0.7613 (-0.26z)| lr 5.43e-04 | 322.98 ms | 52.3% bf16 MFU | 1623538 tok/s
step  634/19560 | loss 5.123904 (-0.68z)| norm 0.6617 (-0.88z)| lr 5.43e-04 | 322.95 ms | 52.3% bf16 MFU | 1623532 tok/s
step  635/19560 | loss 4.997602 (-1.84z)| norm 0.6105 (-1.19z)| lr 5.44e-04 | 322.39 ms | 52.3% bf16 MFU | 1623667 tok/s
step  636/19560 | loss 5.064015 (-1.21z)| norm 0.6564 (-0.90z)| lr 5.45e-04 | 322.08 ms | 52.4% bf16 MFU | 1623874 tok/s
step  637/19560 | loss 4.984893 (-1.93z)| norm 0.6587 (-0.87z)| lr 5.46e-04 | 322.98 ms | 52.3% bf16 MFU | 1623845 tok/s
step  638/19560 | loss 5.020777 (-1.57z)| norm 0.7053 (-0.56z)| lr 5.47e-04 | 322.31 ms | 52.4% bf16 MFU | 1623986 tok/s
step  639/19560 | loss 5.058457 (-1.19z)| norm 0.5669 (-1.45z)| lr 5.48e-04 | 321.67 ms | 52.5% bf16 MFU | 1624281 tok/s
step  640/19560 | loss 5.027593 (-1.47z)| norm 0.5760 (-1.37z)| lr 5.49e-04 | 322.94 ms | 52.3% bf16 MFU | 1624241 tok/s
step  641/19560 | loss 5.004543 (-1.66z)| norm 0.6891 (-0.63z)| lr 5.49e-04 | 323.27 ms | 52.2% bf16 MFU | 1624121 tok/s
step  642/19560 | loss 4.970813 (-1.95z)| norm 0.7922 (+0.04z)| lr 5.50e-04 | 322.78 ms | 52.3% bf16 MFU | 1624130 tok/s
step  643/19560 | loss 5.007246 (-1.58z)| norm 0.6525 (-0.85z)| lr 5.51e-04 | 323.02 ms | 52.2% bf16 MFU | 1624078 tok/s
step  644/19560 | loss 5.031816 (-1.33z)| norm 0.6939 (-0.61z)| lr 5.52e-04 | 322.30 ms | 52.4% bf16 MFU | 1624210 tok/s
step  645/19560 | loss 5.025863 (-1.38z)| norm 0.8993 (+0.90z)| lr 5.53e-04 | 322.10 ms | 52.4% bf16 MFU | 1624387 tok/s
step  646/19560 | loss 4.972078 (-1.88z)| norm 0.7775 (+0.01z)| lr 5.54e-04 | 322.91 ms | 52.3% bf16 MFU | 1624348 tok/s
step  647/19560 | loss 4.995323 (-1.62z)| norm 0.6490 (-0.92z)| lr 5.55e-04 | 323.18 ms | 52.2% bf16 MFU | 1624245 tok/s
step  648/19560 | loss 5.004746 (-1.51z)| norm 0.7387 (-0.25z)| lr 5.55e-04 | 322.64 ms | 52.3% bf16 MFU | 1624281 tok/s
step  649/19560 | loss 5.017095 (-1.37z)| norm 0.9046 (+0.96z)| lr 5.56e-04 | 323.11 ms | 52.2% bf16 MFU | 1624198 tok/s
step  650/19560 | loss 5.013813 (-1.38z)| norm 0.7492 (-0.17z)| lr 5.57e-04 | 322.26 ms | 52.4% bf16 MFU | 1624334 tok/s
step  651/19560 | loss 4.993725 (-1.56z)| norm 0.7661 (-0.03z)| lr 5.58e-04 | 322.56 ms | 52.3% bf16 MFU | 1624387 tok/s
step  652/19560 | loss 4.974297 (-1.72z)| norm 1.3601 (+4.19z)| lr 5.59e-04 | 323.34 ms | 52.2% bf16 MFU | 1624240 tok/s
step  653/19560 | loss 5.069190 (-0.77z)| norm 0.8768 (+0.74z)| lr 5.60e-04 | 322.58 ms | 52.3% bf16 MFU | 1624294 tok/s
step  654/19560 | loss 4.943919 (-1.99z)| norm 0.8171 (+0.31z)| lr 5.61e-04 | 322.97 ms | 52.3% bf16 MFU | 1624246 tok/s
step  655/19560 | loss 4.945743 (-1.94z)| norm 0.8294 (+0.40z)| lr 5.61e-04 | 322.86 ms | 52.3% bf16 MFU | 1624228 tok/s
step  656/19560 | loss 5.040781 (-0.98z)| norm 1.1191 (+2.38z)| lr 5.62e-04 | 322.74 ms | 52.3% bf16 MFU | 1624241 tok/s
step  657/19560 | loss 4.969687 (-1.67z)| norm 0.8635 (+0.59z)| lr 5.63e-04 | 322.68 ms | 52.3% bf16 MFU | 1624267 tok/s
step  658/19560 | loss 5.039181 (-0.96z)| norm 0.7252 (-0.39z)| lr 5.64e-04 | 323.26 ms | 52.2% bf16 MFU | 1624148 tok/s
step  659/19560 | loss 5.063507 (-0.70z)| norm 0.6768 (-0.73z)| lr 5.65e-04 | 322.81 ms | 52.3% bf16 MFU | 1624147 tok/s
step  660/19560 | loss 4.980377 (-1.54z)| norm 0.6036 (-1.24z)| lr 5.66e-04 | 322.13 ms | 52.4% bf16 MFU | 1624318 tok/s
step  661/19560 | loss 5.276470 (+1.54z)| norm 0.5489 (-1.59z)| lr 5.67e-04 | 323.31 ms | 52.2% bf16 MFU | 1624182 tok/s
step  662/19560 | loss 5.026884 (-1.04z)| norm 0.7274 (-0.34z)| lr 5.67e-04 | 323.38 ms | 52.2% bf16 MFU | 1624038 tok/s
step  663/19560 | loss 4.992612 (-1.38z)| norm 0.9472 (+1.20z)| lr 5.68e-04 | 322.80 ms | 52.3% bf16 MFU | 1624045 tok/s
step  664/19560 | loss 5.018263 (-1.09z)| norm 0.9637 (+1.30z)| lr 5.69e-04 | 323.05 ms | 52.2% bf16 MFU | 1623990 tok/s
step  665/19560 | loss 5.104635 (-0.18z)| norm 0.9886 (+1.46z)| lr 5.70e-04 | 322.52 ms | 52.3% bf16 MFU | 1624072 tok/s
step  666/19560 | loss 5.028912 (-0.96z)| norm 1.1072 (+2.24z)| lr 5.71e-04 | 322.96 ms | 52.3% bf16 MFU | 1624037 tok/s
step  667/19560 | loss 5.020986 (-1.03z)| norm 1.0803 (+2.01z)| lr 5.72e-04 | 323.13 ms | 52.2% bf16 MFU | 1623961 tok/s
step  668/19560 | loss 4.959846 (-1.66z)| norm 0.8126 (+0.20z)| lr 5.73e-04 | 322.52 ms | 52.3% bf16 MFU | 1624043 tok/s
step  669/19560 | loss 5.019249 (-1.01z)| norm 0.8732 (+0.60z)| lr 5.73e-04 | 322.46 ms | 52.3% bf16 MFU | 1624135 tok/s
step  670/19560 | loss 4.995850 (-1.24z)| norm 0.8977 (+0.75z)| lr 5.74e-04 | 323.05 ms | 52.2% bf16 MFU | 1624075 tok/s
step  671/19560 | loss 5.004800 (-1.13z)| norm 0.9138 (+0.85z)| lr 5.75e-04 | 323.29 ms | 52.2% bf16 MFU | 1623958 tok/s
step  672/19560 | loss 5.029220 (-0.85z)| norm 0.9062 (+0.79z)| lr 5.76e-04 | 322.89 ms | 52.3% bf16 MFU | 1623946 tok/s
step  673/19560 | loss 5.001417 (-1.14z)| norm 0.8267 (+0.27z)| lr 5.77e-04 | 322.66 ms | 52.3% bf16 MFU | 1623992 tok/s
step  674/19560 | loss 5.011970 (-1.01z)| norm 0.7735 (-0.09z)| lr 5.78e-04 | 323.36 ms | 52.2% bf16 MFU | 1623862 tok/s
step  675/19560 | loss 4.978489 (-1.35z)| norm 0.6499 (-0.96z)| lr 5.79e-04 | 322.58 ms | 52.3% bf16 MFU | 1623932 tok/s
step  676/19560 | loss 4.974800 (-1.39z)| norm 0.7067 (-0.54z)| lr 5.79e-04 | 322.93 ms | 52.3% bf16 MFU | 1623913 tok/s
step  677/19560 | loss 5.000626 (-1.09z)| norm 0.7027 (-0.56z)| lr 5.80e-04 | 322.89 ms | 52.3% bf16 MFU | 1623904 tok/s
step  678/19560 | loss 4.925295 (-1.90z)| norm 0.6024 (-1.26z)| lr 5.81e-04 | 322.55 ms | 52.3% bf16 MFU | 1623981 tok/s
step  679/19560 | loss 4.878237 (-2.39z)| norm 0.5471 (-1.63z)| lr 5.82e-04 | 322.45 ms | 52.3% bf16 MFU | 1624080 tok/s
step  680/19560 | loss 4.988664 (-1.14z)| norm 0.4570 (-2.21z)| lr 5.83e-04 | 322.24 ms | 52.4% bf16 MFU | 1624228 tok/s
step  681/19560 | loss 4.897344 (-2.12z)| norm 0.5565 (-1.48z)| lr 5.84e-04 | 323.09 ms | 52.2% bf16 MFU | 1624152 tok/s
step  682/19560 | loss 4.926800 (-1.77z)| norm 0.6114 (-1.08z)| lr 5.85e-04 | 322.88 ms | 52.3% bf16 MFU | 1624133 tok/s
step  683/19560 | loss 4.894254 (-2.09z)| norm 0.7927 (+0.17z)| lr 5.85e-04 | 323.89 ms | 52.1% bf16 MFU | 1623863 tok/s
step  684/19560 | loss 5.002707 (-0.87z)| norm 0.9107 (+0.98z)| lr 5.86e-04 | 322.24 ms | 52.4% bf16 MFU | 1624020 tok/s
step  685/19560 | loss 4.993126 (-0.96z)| norm 0.8631 (+0.67z)| lr 5.87e-04 | 322.93 ms | 52.3% bf16 MFU | 1623997 tok/s
step  686/19560 | loss 4.872915 (-2.25z)| norm 0.8211 (+0.37z)| lr 5.88e-04 | 323.28 ms | 52.2% bf16 MFU | 1623887 tok/s
step  687/19560 | loss 4.909283 (-1.82z)| norm 0.7737 (+0.04z)| lr 5.89e-04 | 323.13 ms | 52.2% bf16 MFU | 1623818 tok/s
step  688/19560 | loss 4.967854 (-1.16z)| norm 0.8200 (+0.36z)| lr 5.90e-04 | 322.45 ms | 52.3% bf16 MFU | 1623926 tok/s
step  689/19560 | loss 4.874228 (-2.16z)| norm 0.6533 (-0.81z)| lr 5.91e-04 | 322.77 ms | 52.3% bf16 MFU | 1623948 tok/s
step  690/19560 | loss 4.892570 (-1.93z)| norm 0.6775 (-0.64z)| lr 5.91e-04 | 322.90 ms | 52.3% bf16 MFU | 1623934 tok/s
step  691/19560 | loss 4.980680 (-0.94z)| norm 0.7240 (-0.32z)| lr 5.92e-04 | 322.77 ms | 52.3% bf16 MFU | 1623954 tok/s
step  692/19560 | loss 4.958344 (-1.17z)| norm 0.8255 (+0.40z)| lr 5.93e-04 | 322.64 ms | 52.3% bf16 MFU | 1624005 tok/s
step  693/19560 | loss 4.879052 (-2.03z)| norm 0.6773 (-0.65z)| lr 5.94e-04 | 323.28 ms | 52.2% bf16 MFU | 1623894 tok/s
step  694/19560 | loss 4.937470 (-1.35z)| norm 0.6570 (-0.79z)| lr 5.95e-04 | 322.55 ms | 52.3% bf16 MFU | 1623972 tok/s
step  695/19560 | loss 4.904213 (-1.71z)| norm 0.5991 (-1.19z)| lr 5.96e-04 | 323.05 ms | 52.2% bf16 MFU | 1623920 tok/s
step  696/19560 | loss 4.871541 (-2.04z)| norm 0.5632 (-1.43z)| lr 5.97e-04 | 322.96 ms | 52.3% bf16 MFU | 1623894 tok/s
step  697/19560 | loss 4.841018 (-2.33z)| norm 0.5918 (-1.21z)| lr 5.97e-04 | 322.82 ms | 52.3% bf16 MFU | 1623904 tok/s
step  698/19560 | loss 4.933313 (-1.28z)| norm 0.5891 (-1.22z)| lr 5.98e-04 | 323.25 ms | 52.2% bf16 MFU | 1623805 tok/s
step  699/19560 | loss 4.882298 (-1.83z)| norm 0.5889 (-1.20z)| lr 5.99e-04 | 323.41 ms | 52.2% bf16 MFU | 1623670 tok/s
step  700/19560 | loss 4.842571 (-2.22z)| norm 0.5787 (-1.26z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1623737 tok/s
step  701/19560 | loss 4.935590 (-1.17z)| norm 0.6714 (-0.58z)| lr 6.00e-04 | 323.00 ms | 52.3% bf16 MFU | 1623710 tok/s
step  702/19560 | loss 4.868848 (-1.87z)| norm 0.8171 (+0.46z)| lr 6.00e-04 | 322.69 ms | 52.3% bf16 MFU | 1623762 tok/s
step  703/19560 | loss 4.943333 (-1.03z)| norm 0.8086 (+0.39z)| lr 6.00e-04 | 323.04 ms | 52.2% bf16 MFU | 1623723 tok/s
step  704/19560 | loss 4.923669 (-1.23z)| norm 0.7288 (-0.19z)| lr 6.00e-04 | 322.58 ms | 52.3% bf16 MFU | 1623801 tok/s
step  705/19560 | loss 4.881377 (-1.68z)| norm 0.6616 (-0.67z)| lr 6.00e-04 | 323.10 ms | 52.2% bf16 MFU | 1623744 tok/s
step  706/19560 | loss 4.857975 (-1.91z)| norm 0.6513 (-0.73z)| lr 6.00e-04 | 323.50 ms | 52.2% bf16 MFU | 1623590 tok/s
step  707/19560 | loss 4.885143 (-1.59z)| norm 0.6598 (-0.66z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1623644 tok/s
step  708/19560 | loss 4.906756 (-1.33z)| norm 0.7385 (-0.10z)| lr 6.00e-04 | 323.25 ms | 52.2% bf16 MFU | 1623558 tok/s
step  709/19560 | loss 4.795980 (-2.50z)| norm 0.7501 (-0.02z)| lr 6.00e-04 | 321.98 ms | 52.4% bf16 MFU | 1623796 tok/s
step  710/19560 | loss 4.859131 (-1.77z)| norm 0.7155 (-0.28z)| lr 6.00e-04 | 322.97 ms | 52.3% bf16 MFU | 1623772 tok/s
step  711/19560 | loss 4.908017 (-1.22z)| norm 0.6768 (-0.55z)| lr 6.00e-04 | 322.31 ms | 52.4% bf16 MFU | 1623917 tok/s
step  712/19560 | loss 4.825622 (-2.08z)| norm 0.7052 (-0.34z)| lr 6.00e-04 | 322.65 ms | 52.3% bf16 MFU | 1623969 tok/s
step  713/19560 | loss 4.805757 (-2.25z)| norm 0.7129 (-0.27z)| lr 6.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1623995 tok/s
step  714/19560 | loss 4.819892 (-2.05z)| norm 0.7852 (+0.25z)| lr 6.00e-04 | 323.13 ms | 52.2% bf16 MFU | 1623922 tok/s
step  715/19560 | loss 4.852966 (-1.66z)| norm 0.8758 (+0.90z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1623960 tok/s
step  716/19560 | loss 4.859012 (-1.57z)| norm 0.9480 (+1.42z)| lr 6.00e-04 | 323.20 ms | 52.2% bf16 MFU | 1623872 tok/s
step  717/19560 | loss 4.856486 (-1.57z)| norm 0.8429 (+0.67z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1623892 tok/s
step  718/19560 | loss 4.747184 (-2.65z)| norm 0.7563 (+0.03z)| lr 6.00e-04 | 323.04 ms | 52.2% bf16 MFU | 1623846 tok/s
step  719/19560 | loss 4.854129 (-1.51z)| norm 0.6280 (-0.90z)| lr 6.00e-04 | 323.07 ms | 52.2% bf16 MFU | 1623796 tok/s
step  720/19560 | loss 4.924534 (-0.76z)| norm 0.6322 (-0.86z)| lr 6.00e-04 | 322.46 ms | 52.3% bf16 MFU | 1623902 tok/s
step  721/19560 | loss 4.841887 (-1.61z)| norm 0.6348 (-0.84z)| lr 6.00e-04 | 322.82 ms | 52.3% bf16 MFU | 1623911 tok/s
step  722/19560 | loss 4.870138 (-1.29z)| norm 0.6908 (-0.44z)| lr 6.00e-04 | 322.18 ms | 52.4% bf16 MFU | 1624082 tok/s
step  723/19560 | loss 4.821965 (-1.77z)| norm 0.7505 (-0.00z)| lr 6.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1624073 tok/s
step  724/19560 | loss 4.799098 (-1.97z)| norm 0.7012 (-0.36z)| lr 6.00e-04 | 322.71 ms | 52.3% bf16 MFU | 1624100 tok/s
step  725/19560 | loss 4.806869 (-1.87z)| norm 0.5692 (-1.31z)| lr 6.00e-04 | 322.34 ms | 52.4% bf16 MFU | 1624220 tok/s
step  726/19560 | loss 4.754896 (-2.35z)| norm 0.5899 (-1.15z)| lr 6.00e-04 | 322.99 ms | 52.3% bf16 MFU | 1624169 tok/s
step  727/19560 | loss 4.865260 (-1.19z)| norm 0.6771 (-0.51z)| lr 6.00e-04 | 322.71 ms | 52.3% bf16 MFU | 1624193 tok/s
step  728/19560 | loss 4.821895 (-1.61z)| norm 0.7336 (-0.10z)| lr 6.00e-04 | 323.05 ms | 52.2% bf16 MFU | 1624129 tok/s
step  729/19560 | loss 4.770300 (-2.11z)| norm 0.7784 (+0.22z)| lr 6.00e-04 | 323.07 ms | 52.2% bf16 MFU | 1624065 tok/s
step  730/19560 | loss 4.790355 (-1.87z)| norm 0.5867 (-1.15z)| lr 6.00e-04 | 322.41 ms | 52.3% bf16 MFU | 1624168 tok/s
step  731/19560 | loss 4.799998 (-1.74z)| norm 0.6848 (-0.44z)| lr 6.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1624155 tok/s
step  732/19560 | loss 4.793441 (-1.78z)| norm 0.7803 (+0.26z)| lr 6.00e-04 | 323.08 ms | 52.2% bf16 MFU | 1624087 tok/s
step  733/19560 | loss 4.854555 (-1.13z)| norm 0.7823 (+0.26z)| lr 6.00e-04 | 322.91 ms | 52.3% bf16 MFU | 1624065 tok/s
step  734/19560 | loss 4.720920 (-2.43z)| norm 0.7072 (-0.28z)| lr 6.00e-04 | 322.83 ms | 52.3% bf16 MFU | 1624063 tok/s
step  735/19560 | loss 4.779951 (-1.80z)| norm 0.7608 (+0.10z)| lr 6.00e-04 | 322.44 ms | 52.3% bf16 MFU | 1624160 tok/s
step  736/19560 | loss 4.766173 (-1.90z)| norm 0.8045 (+0.41z)| lr 6.00e-04 | 322.80 ms | 52.3% bf16 MFU | 1624163 tok/s
step  737/19560 | loss 4.793047 (-1.60z)| norm 0.9228 (+1.25z)| lr 6.00e-04 | 322.99 ms | 52.3% bf16 MFU | 1624115 tok/s
step  738/19560 | loss 4.761183 (-1.88z)| norm 0.7274 (-0.16z)| lr 6.00e-04 | 323.34 ms | 52.2% bf16 MFU | 1623984 tok/s
step  739/19560 | loss 4.824106 (-1.24z)| norm 0.8541 (+0.76z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1624035 tok/s
step  740/19560 | loss 4.771642 (-1.73z)| norm 0.8130 (+0.46z)| lr 6.00e-04 | 322.59 ms | 52.3% bf16 MFU | 1624095 tok/s
step  741/19560 | loss 4.793971 (-1.48z)| norm 0.6912 (-0.42z)| lr 6.00e-04 | 322.66 ms | 52.3% bf16 MFU | 1624135 tok/s
step  742/19560 | loss 4.758155 (-1.80z)| norm 0.6010 (-1.06z)| lr 6.00e-04 | 322.87 ms | 52.3% bf16 MFU | 1624120 tok/s
step  743/19560 | loss 4.757328 (-1.77z)| norm 0.6363 (-0.81z)| lr 6.00e-04 | 322.88 ms | 52.3% bf16 MFU | 1624102 tok/s
step  744/19560 | loss 4.754287 (-1.76z)| norm 0.5118 (-1.68z)| lr 6.00e-04 | 323.07 ms | 52.2% bf16 MFU | 1624040 tok/s
step  745/19560 | loss 4.766960 (-1.61z)| norm 0.5423 (-1.45z)| lr 6.00e-04 | 322.63 ms | 52.3% bf16 MFU | 1624090 tok/s
step  746/19560 | loss 4.747696 (-1.76z)| norm 0.4927 (-1.77z)| lr 6.00e-04 | 323.48 ms | 52.2% bf16 MFU | 1623925 tok/s
step  747/19560 | loss 4.769404 (-1.53z)| norm 0.5390 (-1.42z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1623978 tok/s
step  748/19560 | loss 4.739942 (-1.77z)| norm 0.5319 (-1.44z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1623994 tok/s
step  749/19560 | loss 4.697782 (-2.11z)| norm 0.5500 (-1.30z)| lr 6.00e-04 | 322.51 ms | 52.3% bf16 MFU | 1624076 tok/s
step  750/19560 | loss 4.706387 (-1.98z)| norm 0.5522 (-1.28z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1624111 tok/s
val loss 4.731819
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2523/10042 = 0.251245
step  751/19560 | loss 4.752688 (-1.53z)| norm 0.7793 (+0.27z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624125 tok/s
step  752/19560 | loss 4.767797 (-1.37z)| norm 0.8375 (+0.66z)| lr 6.00e-04 | 323.47 ms | 52.2% bf16 MFU | 1623960 tok/s
step  753/19560 | loss 4.823668 (-0.86z)| norm 0.7592 (+0.11z)| lr 6.00e-04 | 323.22 ms | 52.2% bf16 MFU | 1623867 tok/s
step  754/19560 | loss 4.769236 (-1.32z)| norm 0.6913 (-0.35z)| lr 6.00e-04 | 322.92 ms | 52.3% bf16 MFU | 1623852 tok/s
step  755/19560 | loss 4.733784 (-1.62z)| norm 0.5884 (-1.05z)| lr 6.00e-04 | 322.91 ms | 52.3% bf16 MFU | 1623841 tok/s
step  756/19560 | loss 4.714999 (-1.76z)| norm 0.5645 (-1.20z)| lr 6.00e-04 | 323.86 ms | 52.1% bf16 MFU | 1623594 tok/s
step  757/19560 | loss 4.746745 (-1.45z)| norm 0.5690 (-1.15z)| lr 6.00e-04 | 322.44 ms | 52.3% bf16 MFU | 1623713 tok/s
step  758/19560 | loss 4.846592 (-0.54z)| norm 0.6310 (-0.70z)| lr 6.00e-04 | 322.55 ms | 52.3% bf16 MFU | 1623799 tok/s
step  759/19560 | loss 4.756117 (-1.34z)| norm 0.9392 (+1.48z)| lr 6.00e-04 | 323.39 ms | 52.2% bf16 MFU | 1623671 tok/s
step  760/19560 | loss 4.679823 (-1.99z)| norm 0.7017 (-0.19z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1623708 tok/s
step  761/19560 | loss 4.685313 (-1.90z)| norm 0.6757 (-0.37z)| lr 6.00e-04 | 322.90 ms | 52.3% bf16 MFU | 1623706 tok/s
step  762/19560 | loss 4.701752 (-1.74z)| norm 0.5762 (-1.07z)| lr 6.00e-04 | 323.83 ms | 52.1% bf16 MFU | 1623472 tok/s
step  763/19560 | loss 4.740846 (-1.36z)| norm 0.5438 (-1.29z)| lr 6.00e-04 | 322.14 ms | 52.4% bf16 MFU | 1623675 tok/s
step  764/19560 | loss 4.641892 (-2.20z)| norm 0.5347 (-1.34z)| lr 6.00e-04 | 323.30 ms | 52.2% bf16 MFU | 1623576 tok/s
step  765/19560 | loss 4.689033 (-1.74z)| norm 0.6166 (-0.76z)| lr 6.00e-04 | 323.15 ms | 52.2% bf16 MFU | 1623518 tok/s
step  766/19560 | loss 4.729656 (-1.36z)| norm 0.7154 (-0.07z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1623556 tok/s
step  767/19560 | loss 4.718853 (-1.43z)| norm 0.9485 (+1.53z)| lr 6.00e-04 | 322.52 ms | 52.3% bf16 MFU | 1623658 tok/s
step  768/19560 | loss 4.701714 (-1.56z)| norm 0.9646 (+1.61z)| lr 6.00e-04 | 322.93 ms | 52.3% bf16 MFU | 1623653 tok/s
step  769/19560 | loss 4.744864 (-1.16z)| norm 0.8986 (+1.14z)| lr 6.00e-04 | 323.15 ms | 52.2% bf16 MFU | 1623591 tok/s
step  770/19560 | loss 4.725748 (-1.31z)| norm 0.8321 (+0.68z)| lr 6.00e-04 | 323.53 ms | 52.2% bf16 MFU | 1623438 tok/s
step  771/19560 | loss 4.677504 (-1.69z)| norm 0.5429 (-1.30z)| lr 6.00e-04 | 323.25 ms | 52.2% bf16 MFU | 1623362 tok/s
step  772/19560 | loss 4.727492 (-1.24z)| norm 0.5423 (-1.29z)| lr 6.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1623389 tok/s
step  773/19560 | loss 4.650577 (-1.87z)| norm 0.5354 (-1.32z)| lr 6.00e-04 | 322.88 ms | 52.3% bf16 MFU | 1623409 tok/s
step  774/19560 | loss 4.737201 (-1.11z)| norm 0.6554 (-0.49z)| lr 6.00e-04 | 323.18 ms | 52.2% bf16 MFU | 1623353 tok/s
step  775/19560 | loss 4.727294 (-1.17z)| norm 0.6655 (-0.43z)| lr 6.00e-04 | 323.03 ms | 52.2% bf16 MFU | 1623336 tok/s
step  776/19560 | loss 4.681262 (-1.54z)| norm 0.6185 (-0.74z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1623388 tok/s
step  777/19560 | loss 4.697464 (-1.38z)| norm 0.6033 (-0.83z)| lr 6.00e-04 | 322.91 ms | 52.3% bf16 MFU | 1623400 tok/s
step  778/19560 | loss 4.758095 (-0.85z)| norm 0.6096 (-0.77z)| lr 6.00e-04 | 323.04 ms | 52.2% bf16 MFU | 1623377 tok/s
step  779/19560 | loss 4.692084 (-1.39z)| norm 0.6559 (-0.45z)| lr 6.00e-04 | 323.30 ms | 52.2% bf16 MFU | 1623291 tok/s
step  780/19560 | loss 4.683318 (-1.44z)| norm 0.7188 (+0.01z)| lr 6.00e-04 | 322.66 ms | 52.3% bf16 MFU | 1623372 tok/s
step  781/19560 | loss 4.644392 (-1.75z)| norm 0.6525 (-0.47z)| lr 6.00e-04 | 322.75 ms | 52.3% bf16 MFU | 1623424 tok/s
step  782/19560 | loss 4.671682 (-1.49z)| norm 0.6190 (-0.71z)| lr 6.00e-04 | 323.30 ms | 52.2% bf16 MFU | 1623337 tok/s
step  783/19560 | loss 4.646279 (-1.67z)| norm 0.8445 (+0.96z)| lr 6.00e-04 | 323.00 ms | 52.3% bf16 MFU | 1623331 tok/s
step  784/19560 | loss 4.667816 (-1.47z)| norm 0.6836 (-0.21z)| lr 6.00e-04 | 323.15 ms | 52.2% bf16 MFU | 1623284 tok/s
step  785/19560 | loss 4.668583 (-1.44z)| norm 0.7206 (+0.08z)| lr 6.00e-04 | 322.81 ms | 52.3% bf16 MFU | 1623328 tok/s
step  786/19560 | loss 4.607783 (-1.91z)| norm 0.6913 (-0.14z)| lr 6.00e-04 | 322.96 ms | 52.3% bf16 MFU | 1623331 tok/s
step  787/19560 | loss 4.635625 (-1.66z)| norm 0.5621 (-1.12z)| lr 6.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1623360 tok/s
step  788/19560 | loss 4.681964 (-1.25z)| norm 0.6446 (-0.50z)| lr 6.00e-04 | 323.37 ms | 52.2% bf16 MFU | 1623258 tok/s
step  789/19560 | loss 4.638859 (-1.65z)| norm 0.7375 (+0.21z)| lr 6.00e-04 | 323.40 ms | 52.2% bf16 MFU | 1623155 tok/s
step  790/19560 | loss 4.685817 (-1.22z)| norm 0.5858 (-0.95z)| lr 6.00e-04 | 323.51 ms | 52.2% bf16 MFU | 1623029 tok/s
step  791/19560 | loss 4.700580 (-1.07z)| norm 0.4689 (-1.82z)| lr 6.00e-04 | 323.18 ms | 52.2% bf16 MFU | 1622991 tok/s
step  792/19560 | loss 4.611324 (-1.84z)| norm 0.6815 (-0.17z)| lr 6.00e-04 | 323.25 ms | 52.2% bf16 MFU | 1622938 tok/s
step  793/19560 | loss 4.677313 (-1.24z)| norm 0.8066 (+0.83z)| lr 6.00e-04 | 323.42 ms | 52.2% bf16 MFU | 1622846 tok/s
step  794/19560 | loss 4.616054 (-1.78z)| norm 0.8201 (+0.99z)| lr 6.00e-04 | 322.93 ms | 52.3% bf16 MFU | 1622881 tok/s
step  795/19560 | loss 4.669495 (-1.27z)| norm 0.6950 (-0.02z)| lr 6.00e-04 | 323.82 ms | 52.1% bf16 MFU | 1622690 tok/s
step  796/19560 | loss 4.689667 (-1.07z)| norm 0.8209 (+1.06z)| lr 6.00e-04 | 322.98 ms | 52.3% bf16 MFU | 1622719 tok/s
step  797/19560 | loss 4.633152 (-1.58z)| norm 0.8728 (+1.50z)| lr 6.00e-04 | 323.21 ms | 52.2% bf16 MFU | 1622691 tok/s
step  798/19560 | loss 4.706203 (-0.88z)| norm 0.9022 (+1.76z)| lr 6.00e-04 | 322.19 ms | 52.4% bf16 MFU | 1622920 tok/s
step  799/19560 | loss 4.655054 (-1.35z)| norm 0.9404 (+2.07z)| lr 6.00e-04 | 323.19 ms | 52.2% bf16 MFU | 1622885 tok/s
step  800/19560 | loss 4.616530 (-1.70z)| norm 0.7385 (+0.37z)| lr 6.00e-04 | 323.12 ms | 52.2% bf16 MFU | 1622869 tok/s
step  801/19560 | loss 4.653017 (-1.33z)| norm 0.6143 (-0.69z)| lr 6.00e-04 | 323.18 ms | 52.2% bf16 MFU | 1622840 tok/s
step  802/19560 | loss 4.598925 (-1.83z)| norm 0.5159 (-1.52z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1622938 tok/s
step  803/19560 | loss 4.596166 (-1.83z)| norm 0.4487 (-2.05z)| lr 6.00e-04 | 322.65 ms | 52.3% bf16 MFU | 1623038 tok/s
step  804/19560 | loss 4.736272 (-0.46z)| norm 0.4405 (-2.07z)| lr 6.00e-04 | 323.12 ms | 52.2% bf16 MFU | 1623016 tok/s
step  805/19560 | loss 4.605661 (-1.72z)| norm 0.4042 (-2.30z)| lr 6.00e-04 | 323.22 ms | 52.2% bf16 MFU | 1622970 tok/s
step  806/19560 | loss 4.608259 (-1.67z)| norm 0.4162 (-2.15z)| lr 6.00e-04 | 323.42 ms | 52.2% bf16 MFU | 1622875 tok/s
step  807/19560 | loss 4.612420 (-1.60z)| norm 0.4671 (-1.73z)| lr 6.00e-04 | 322.95 ms | 52.3% bf16 MFU | 1622904 tok/s
step  808/19560 | loss 4.615067 (-1.55z)| norm 0.5502 (-1.08z)| lr 6.00e-04 | 322.32 ms | 52.4% bf16 MFU | 1623089 tok/s
step  809/19560 | loss 4.611504 (-1.56z)| norm 0.7799 (+0.75z)| lr 6.00e-04 | 323.46 ms | 52.2% bf16 MFU | 1622979 tok/s
step  810/19560 | loss 4.583096 (-1.81z)| norm 0.8123 (+0.99z)| lr 6.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1623072 tok/s
step  811/19560 | loss 4.577732 (-1.82z)| norm 0.7363 (+0.39z)| lr 6.00e-04 | 323.19 ms | 52.2% bf16 MFU | 1623030 tok/s
step  812/19560 | loss 4.601267 (-1.58z)| norm 0.5560 (-1.05z)| lr 6.00e-04 | 323.29 ms | 52.2% bf16 MFU | 1622965 tok/s
step  813/19560 | loss 4.647179 (-1.12z)| norm 0.5833 (-0.81z)| lr 6.00e-04 | 322.97 ms | 52.3% bf16 MFU | 1622984 tok/s
step  814/19560 | loss 4.625286 (-1.32z)| norm 0.5765 (-0.85z)| lr 6.00e-04 | 323.42 ms | 52.2% bf16 MFU | 1622890 tok/s
step  815/19560 | loss 4.554557 (-1.98z)| norm 0.6459 (-0.28z)| lr 6.00e-04 | 322.31 ms | 52.4% bf16 MFU | 1623077 tok/s
step  816/19560 | loss 4.628315 (-1.24z)| norm 0.6117 (-0.55z)| lr 6.00e-04 | 322.09 ms | 52.4% bf16 MFU | 1623311 tok/s
step  817/19560 | loss 4.581597 (-1.68z)| norm 0.4984 (-1.46z)| lr 6.00e-04 | 322.93 ms | 52.3% bf16 MFU | 1623323 tok/s
step  818/19560 | loss 4.583045 (-1.63z)| norm 0.4709 (-1.65z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1623406 tok/s
step  819/19560 | loss 4.599386 (-1.46z)| norm 0.4307 (-1.93z)| lr 6.00e-04 | 322.93 ms | 52.3% bf16 MFU | 1623413 tok/s
step  820/19560 | loss 4.586987 (-1.57z)| norm 0.3871 (-2.22z)| lr 6.00e-04 | 322.15 ms | 52.4% bf16 MFU | 1623615 tok/s
step  821/19560 | loss 4.530159 (-2.10z)| norm 0.4596 (-1.62z)| lr 6.00e-04 | 322.11 ms | 52.4% bf16 MFU | 1623817 tok/s
step  822/19560 | loss 4.556902 (-1.80z)| norm 0.4253 (-1.84z)| lr 6.00e-04 | 322.66 ms | 52.3% bf16 MFU | 1623872 tok/s
step  823/19560 | loss 4.564120 (-1.70z)| norm 0.4978 (-1.28z)| lr 6.00e-04 | 323.06 ms | 52.2% bf16 MFU | 1623822 tok/s
step  824/19560 | loss 4.547585 (-1.84z)| norm 0.5812 (-0.64z)| lr 6.00e-04 | 322.97 ms | 52.3% bf16 MFU | 1623797 tok/s
step  825/19560 | loss 4.557037 (-1.71z)| norm 0.6368 (-0.22z)| lr 6.00e-04 | 322.69 ms | 52.3% bf16 MFU | 1623844 tok/s
step  826/19560 | loss 4.563113 (-1.63z)| norm 0.6022 (-0.49z)| lr 6.00e-04 | 322.94 ms | 52.3% bf16 MFU | 1623826 tok/s
step  827/19560 | loss 4.651984 (-0.73z)| norm 0.5418 (-0.94z)| lr 6.00e-04 | 322.91 ms | 52.3% bf16 MFU | 1623817 tok/s
step  828/19560 | loss 4.569741 (-1.53z)| norm 0.6107 (-0.42z)| lr 6.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1623820 tok/s
step  829/19560 | loss 4.549089 (-1.72z)| norm 0.5649 (-0.76z)| lr 6.00e-04 | 323.55 ms | 52.2% bf16 MFU | 1623649 tok/s
step  830/19560 | loss 4.525521 (-1.92z)| norm 0.5604 (-0.78z)| lr 6.00e-04 | 322.44 ms | 52.3% bf16 MFU | 1623766 tok/s
step  831/19560 | loss 4.571943 (-1.44z)| norm 0.5655 (-0.73z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1623794 tok/s
step  832/19560 | loss 4.590589 (-1.24z)| norm 0.5755 (-0.65z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1623804 tok/s
step  833/19560 | loss 4.621037 (-0.91z)| norm 0.5677 (-0.70z)| lr 6.00e-04 | 322.52 ms | 52.3% bf16 MFU | 1623892 tok/s
step  834/19560 | loss 4.571955 (-1.40z)| norm 0.5425 (-0.88z)| lr 6.00e-04 | 322.80 ms | 52.3% bf16 MFU | 1623906 tok/s
step  835/19560 | loss 4.559008 (-1.52z)| norm 0.6194 (-0.30z)| lr 6.00e-04 | 322.75 ms | 52.3% bf16 MFU | 1623933 tok/s
step  836/19560 | loss 4.592214 (-1.16z)| norm 0.6047 (-0.40z)| lr 6.00e-04 | 323.24 ms | 52.2% bf16 MFU | 1623836 tok/s
step  837/19560 | loss 4.509272 (-1.99z)| norm 0.5429 (-0.86z)| lr 6.00e-04 | 322.75 ms | 52.3% bf16 MFU | 1623866 tok/s
step  838/19560 | loss 4.550372 (-1.54z)| norm 0.5858 (-0.52z)| lr 6.00e-04 | 322.82 ms | 52.3% bf16 MFU | 1623876 tok/s
step  839/19560 | loss 4.723170 (+0.30z)| norm 0.6180 (-0.27z)| lr 6.00e-04 | 322.91 ms | 52.3% bf16 MFU | 1623865 tok/s
step  840/19560 | loss 4.552167 (-1.51z)| norm 0.7232 (+0.52z)| lr 6.00e-04 | 323.10 ms | 52.2% bf16 MFU | 1623804 tok/s
step  841/19560 | loss 4.554356 (-1.46z)| norm 0.5647 (-0.67z)| lr 6.00e-04 | 322.33 ms | 52.4% bf16 MFU | 1623942 tok/s
step  842/19560 | loss 4.614858 (-0.80z)| norm 0.4958 (-1.17z)| lr 6.00e-04 | 322.51 ms | 52.3% bf16 MFU | 1624028 tok/s
step  843/19560 | loss 4.535882 (-1.62z)| norm 0.4649 (-1.39z)| lr 6.00e-04 | 322.79 ms | 52.3% bf16 MFU | 1624038 tok/s
step  844/19560 | loss 4.492441 (-2.05z)| norm 0.5210 (-0.95z)| lr 6.00e-04 | 322.58 ms | 52.3% bf16 MFU | 1624101 tok/s
step  845/19560 | loss 4.573414 (-1.17z)| norm 0.6421 (-0.01z)| lr 6.00e-04 | 322.98 ms | 52.3% bf16 MFU | 1624060 tok/s
step  846/19560 | loss 4.543926 (-1.46z)| norm 0.6277 (-0.11z)| lr 6.00e-04 | 322.81 ms | 52.3% bf16 MFU | 1624064 tok/s
step  847/19560 | loss 4.516595 (-1.73z)| norm 0.5525 (-0.69z)| lr 6.00e-04 | 323.03 ms | 52.2% bf16 MFU | 1624013 tok/s
step  848/19560 | loss 4.525113 (-1.64z)| norm 0.5189 (-0.95z)| lr 6.00e-04 | 322.80 ms | 52.3% bf16 MFU | 1624021 tok/s
step  849/19560 | loss 4.548756 (-1.36z)| norm 0.4917 (-1.14z)| lr 6.00e-04 | 323.13 ms | 52.2% bf16 MFU | 1623946 tok/s
step  850/19560 | loss 4.456058 (-2.34z)| norm 0.5447 (-0.72z)| lr 6.00e-04 | 322.80 ms | 52.3% bf16 MFU | 1623957 tok/s
step  851/19560 | loss 4.556492 (-1.22z)| norm 0.5563 (-0.62z)| lr 6.00e-04 | 322.55 ms | 52.3% bf16 MFU | 1624031 tok/s
step  852/19560 | loss 4.470996 (-2.12z)| norm 0.5314 (-0.80z)| lr 6.00e-04 | 322.69 ms | 52.3% bf16 MFU | 1624067 tok/s
step  853/19560 | loss 4.521953 (-1.53z)| norm 0.6179 (-0.14z)| lr 6.00e-04 | 323.88 ms | 52.1% bf16 MFU | 1623802 tok/s
step  854/19560 | loss 4.503319 (-1.70z)| norm 0.6026 (-0.26z)| lr 6.00e-04 | 322.50 ms | 52.3% bf16 MFU | 1623898 tok/s
step  855/19560 | loss 4.516375 (-1.55z)| norm 0.5844 (-0.39z)| lr 6.00e-04 | 322.92 ms | 52.3% bf16 MFU | 1623881 tok/s
step  856/19560 | loss 4.477001 (-1.95z)| norm 0.5539 (-0.62z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1623905 tok/s
step  857/19560 | loss 4.524323 (-1.41z)| norm 0.5265 (-0.82z)| lr 6.00e-04 | 322.87 ms | 52.3% bf16 MFU | 1623902 tok/s
step  858/19560 | loss 4.509416 (-1.54z)| norm 0.5588 (-0.56z)| lr 6.00e-04 | 322.34 ms | 52.4% bf16 MFU | 1624032 tok/s
step  859/19560 | loss 4.605366 (-0.48z)| norm 0.5723 (-0.45z)| lr 6.00e-04 | 323.07 ms | 52.2% bf16 MFU | 1623972 tok/s
step  860/19560 | loss 4.533696 (-1.26z)| norm 0.5541 (-0.58z)| lr 6.00e-04 | 322.47 ms | 52.3% bf16 MFU | 1624066 tok/s
step  861/19560 | loss 4.560771 (-0.95z)| norm 0.5803 (-0.37z)| lr 6.00e-04 | 323.10 ms | 52.2% bf16 MFU | 1623997 tok/s
step  862/19560 | loss 4.456868 (-2.07z)| norm 0.4696 (-1.21z)| lr 6.00e-04 | 322.94 ms | 52.3% bf16 MFU | 1623971 tok/s
step  863/19560 | loss 4.580936 (-0.68z)| norm 0.4137 (-1.62z)| lr 6.00e-04 | 322.36 ms | 52.4% bf16 MFU | 1624093 tok/s
step  864/19560 | loss 4.527455 (-1.25z)| norm 0.4502 (-1.32z)| lr 6.00e-04 | 323.05 ms | 52.2% bf16 MFU | 1624036 tok/s
step  865/19560 | loss 4.507093 (-1.46z)| norm 0.4017 (-1.68z)| lr 6.00e-04 | 323.04 ms | 52.2% bf16 MFU | 1623984 tok/s
step  866/19560 | loss 4.500328 (-1.51z)| norm 0.4538 (-1.25z)| lr 6.00e-04 | 322.45 ms | 52.3% bf16 MFU | 1624082 tok/s
step  867/19560 | loss 4.565885 (-0.77z)| norm 0.5075 (-0.82z)| lr 6.00e-04 | 322.44 ms | 52.3% bf16 MFU | 1624178 tok/s
step  868/19560 | loss 4.471168 (-1.82z)| norm 0.6120 (+0.02z)| lr 6.00e-04 | 323.01 ms | 52.2% bf16 MFU | 1624125 tok/s
start Wed Nov 20 05:52:53 UTC 2024
+-----------------------+----------------------------------------------------+
| Parameter             | Value                                              |
+-----------------------+----------------------------------------------------+
| train data pattern    | dev/data/fineweb10B/fineweb_train_*.bin            |
| val data pattern      | dev/data/fineweb10B/fineweb_val_*.bin              |
| output log dir        | log124M                                            |
| checkpoint_every      | 5000                                               |
| resume                | 0                                                  |
| micro batch size B    | 64                                                 |
| sequence length T     | 1024                                               |
| total batch size      | 524288                                             |
| LR scheduler          | cosine                                             |
| learning rate (LR)    | 6.000000e-04                                       |
| warmup iterations     | 700                                                |
| final LR fraction     | 0.000000e+00                                       |
| weight decay          | 1.000000e-01                                       |
| skip update lossz     | 0.000000                                           |
| skip update gradz     | 0.000000                                           |
| max_steps             | -1                                                 |
| val_loss_every        | 250                                                |
| val_max_steps         | 20                                                 |
| sample_every          | 20000                                              |
| genT                  | 64                                                 |
| overfit_single_batch  | 0                                                  |
| use_master_weights    | enabled                                            |
| gelu_fusion           | 0                                                  |
| recompute             | 1                                                  |
+-----------------------+----------------------------------------------------+
| device                | NVIDIA A100-SXM4-40GB                              |
| peak TFlops           | 312.0                                              |
| precision             | BF16                                               |
+-----------------------+----------------------------------------------------+
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
| weight init method    | d12                                                |
| max_sequence_length T | 1024                                               |
| vocab_size V          | 50257                                              |
| padded_vocab_size Vp  | 50304                                              |
| num_layers L          | 12                                                 |
| num_heads NH          | 12                                                 |
| channels C            | 768                                                |
| num_parameters        | 124475904                                          |
+-----------------------+----------------------------------------------------+
| train_num_batches     | 19560                                              |
| val_num_batches       | 20                                                 |
+-----------------------+----------------------------------------------------+
| run hellaswag         | yes                                                |
+-----------------------+----------------------------------------------------+
| num_processes         | 8                                                  |
| zero_stage            | 1                                                  |
+-----------------------+----------------------------------------------------+
num_parameters: 124475904 => bytes: 248951808
allocated 237 MiB for model parameters
batch_size B=64 * seq_len T=1024 * num_processes=8 and total_batch_size=524288
=> setting grad_accum_steps=1
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
allocating 237 MiB for parameter gradients
allocating 21216 MiB for activations
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
allocating 59 MiB for AdamW optimizer state m
allocating 59 MiB for AdamW optimizer state v
allocating 59 MiB for master copy of params
device memory usage: 23129 MiB / 40326 MiB
memory per sequence: 331 MiB
 -> estimated maximum batch size: 115
val loss 11.009204
step    1/19560 | loss 11.009109 (+nanz)| norm 15.3717 (+nanz)| lr 8.57e-07 | 694.62 ms | 24.3% bf16 MFU | 754784 tok/s
step    2/19560 | loss 10.959906 (+nanz)| norm 15.2377 (+nanz)| lr 1.71e-06 | 314.81 ms | 53.6% bf16 MFU | 1665435 tok/s
step    3/19560 | loss 10.855613 (+nanz)| norm 14.8389 (+nanz)| lr 2.57e-06 | 315.55 ms | 53.5% bf16 MFU | 1663410 tok/s
step    4/19560 | loss 10.715732 (+nanz)| norm 13.1195 (+nanz)| lr 3.43e-06 | 315.63 ms | 53.5% bf16 MFU | 1662603 tok/s
step    5/19560 | loss 10.568424 (+nanz)| norm 10.4405 (+nanz)| lr 4.29e-06 | 314.80 ms | 53.6% bf16 MFU | 1663372 tok/s
step    6/19560 | loss 10.423451 (+nanz)| norm 8.4943 (+nanz)| lr 5.14e-06 | 316.25 ms | 53.4% bf16 MFU | 1662141 tok/s
step    7/19560 | loss 10.291803 (+nanz)| norm 7.2415 (+nanz)| lr 6.00e-06 | 314.33 ms | 53.7% bf16 MFU | 1663243 tok/s
step    8/19560 | loss 10.187215 (+nanz)| norm 6.1835 (+nanz)| lr 6.86e-06 | 314.61 ms | 53.6% bf16 MFU | 1663778 tok/s
step    9/19560 | loss 10.096741 (+nanz)| norm 5.2947 (+nanz)| lr 7.71e-06 | 314.69 ms | 53.6% bf16 MFU | 1664113 tok/s
step   10/19560 | loss 9.976251 (+nanz)| norm 4.6635 (+nanz)| lr 8.57e-06 | 314.25 ms | 53.7% bf16 MFU | 1664688 tok/s
step   11/19560 | loss 9.935358 (+nanz)| norm 3.8501 (+nanz)| lr 9.43e-06 | 314.42 ms | 53.7% bf16 MFU | 1665038 tok/s
step   12/19560 | loss 9.853889 (+nanz)| norm 3.3980 (+nanz)| lr 1.03e-05 | 314.53 ms | 53.7% bf16 MFU | 1665252 tok/s
step   13/19560 | loss 9.792458 (+nanz)| norm 3.0057 (+nanz)| lr 1.11e-05 | 313.95 ms | 53.8% bf16 MFU | 1665768 tok/s
step   14/19560 | loss 9.758890 (+nanz)| norm 2.6957 (+nanz)| lr 1.20e-05 | 314.46 ms | 53.7% bf16 MFU | 1665920 tok/s
step   15/19560 | loss 9.702516 (+nanz)| norm 2.5212 (+nanz)| lr 1.29e-05 | 314.34 ms | 53.7% bf16 MFU | 1666115 tok/s
step   16/19560 | loss 9.701745 (+nanz)| norm 2.3030 (+nanz)| lr 1.37e-05 | 314.66 ms | 53.6% bf16 MFU | 1666125 tok/s
step   17/19560 | loss 9.655603 (+nanz)| norm 2.2412 (+nanz)| lr 1.46e-05 | 313.97 ms | 53.8% bf16 MFU | 1666458 tok/s
step   18/19560 | loss 9.636615 (+nanz)| norm 2.1886 (+nanz)| lr 1.54e-05 | 314.98 ms | 53.6% bf16 MFU | 1666292 tok/s
step   19/19560 | loss 9.603565 (+nanz)| norm 2.2160 (+nanz)| lr 1.63e-05 | 314.25 ms | 53.7% bf16 MFU | 1666464 tok/s
step   20/19560 | loss 9.596481 (+nanz)| norm 2.1544 (+nanz)| lr 1.71e-05 | 314.16 ms | 53.7% bf16 MFU | 1666656 tok/s
step   21/19560 | loss 9.562174 (+nanz)| norm 2.1811 (+nanz)| lr 1.80e-05 | 314.54 ms | 53.7% bf16 MFU | 1666672 tok/s
step   22/19560 | loss 9.537735 (+nanz)| norm 2.1489 (+nanz)| lr 1.89e-05 | 313.90 ms | 53.8% bf16 MFU | 1666944 tok/s
step   23/19560 | loss 9.533234 (+nanz)| norm 2.0896 (+nanz)| lr 1.97e-05 | 314.80 ms | 53.6% bf16 MFU | 1666835 tok/s
step   24/19560 | loss 9.476656 (+nanz)| norm 2.1037 (+nanz)| lr 2.06e-05 | 314.22 ms | 53.7% bf16 MFU | 1666959 tok/s
step   25/19560 | loss 9.458328 (+nanz)| norm 2.0526 (+nanz)| lr 2.14e-05 | 314.46 ms | 53.7% bf16 MFU | 1666982 tok/s
step   26/19560 | loss 9.422851 (+nanz)| norm 2.0944 (+nanz)| lr 2.23e-05 | 315.15 ms | 53.6% bf16 MFU | 1666748 tok/s
step   27/19560 | loss 9.377934 (+nanz)| norm 2.1533 (+nanz)| lr 2.31e-05 | 314.62 ms | 53.6% bf16 MFU | 1666724 tok/s
step   28/19560 | loss 9.388617 (+nanz)| norm 1.9649 (+nanz)| lr 2.40e-05 | 315.59 ms | 53.5% bf16 MFU | 1666363 tok/s
step   29/19560 | loss 9.357564 (+nanz)| norm 2.4482 (+nanz)| lr 2.49e-05 | 314.29 ms | 53.7% bf16 MFU | 1666483 tok/s
step   30/19560 | loss 9.306392 (+nanz)| norm 1.9512 (+nanz)| lr 2.57e-05 | 315.52 ms | 53.5% bf16 MFU | 1666172 tok/s
step   31/19560 | loss 9.244952 (+nanz)| norm 2.2942 (+nanz)| lr 2.66e-05 | 315.59 ms | 53.5% bf16 MFU | 1665862 tok/s
step   32/19560 | loss 9.211153 (+nanz)| norm 2.2607 (+nanz)| lr 2.74e-05 | 315.30 ms | 53.5% bf16 MFU | 1665672 tok/s
step   33/19560 | loss 9.235577 (+nanz)| norm 2.5835 (+nanz)| lr 2.83e-05 | 315.60 ms | 53.5% bf16 MFU | 1665396 tok/s
step   34/19560 | loss 9.159492 (+nanz)| norm 2.4417 (+nanz)| lr 2.91e-05 | 315.31 ms | 53.5% bf16 MFU | 1665234 tok/s
step   35/19560 | loss 9.200701 (+nanz)| norm 1.9539 (+nanz)| lr 3.00e-05 | 315.66 ms | 53.5% bf16 MFU | 1664972 tok/s
step   36/19560 | loss 9.080112 (+nanz)| norm 2.4098 (+nanz)| lr 3.09e-05 | 315.95 ms | 53.4% bf16 MFU | 1664639 tok/s
step   37/19560 | loss 9.029287 (+nanz)| norm 1.8690 (+nanz)| lr 3.17e-05 | 315.30 ms | 53.5% bf16 MFU | 1664530 tok/s
step   38/19560 | loss 9.039859 (+nanz)| norm 1.9751 (+nanz)| lr 3.26e-05 | 315.53 ms | 53.5% bf16 MFU | 1664360 tok/s
step   39/19560 | loss 8.977989 (+nanz)| norm 1.8097 (+nanz)| lr 3.34e-05 | 315.59 ms | 53.5% bf16 MFU | 1664182 tok/s
step   40/19560 | loss 8.961264 (+nanz)| norm 2.0398 (+nanz)| lr 3.43e-05 | 315.81 ms | 53.4% bf16 MFU | 1663950 tok/s
step   41/19560 | loss 8.906665 (+nanz)| norm 1.8697 (+nanz)| lr 3.51e-05 | 317.02 ms | 53.2% bf16 MFU | 1663368 tok/s
step   42/19560 | loss 8.860152 (+nanz)| norm 1.6709 (+nanz)| lr 3.60e-05 | 316.21 ms | 53.4% bf16 MFU | 1663064 tok/s
step   43/19560 | loss 8.839649 (+nanz)| norm 1.7306 (+nanz)| lr 3.69e-05 | 316.50 ms | 53.3% bf16 MFU | 1662694 tok/s
step   44/19560 | loss 8.856464 (+nanz)| norm 1.6668 (+nanz)| lr 3.77e-05 | 316.67 ms | 53.3% bf16 MFU | 1662297 tok/s
step   45/19560 | loss 8.748160 (+nanz)| norm 1.6705 (+nanz)| lr 3.86e-05 | 316.52 ms | 53.3% bf16 MFU | 1661970 tok/s
step   46/19560 | loss 8.692651 (+nanz)| norm 1.6793 (+nanz)| lr 3.94e-05 | 316.42 ms | 53.3% bf16 MFU | 1661691 tok/s
step   47/19560 | loss 8.712533 (+nanz)| norm 1.5849 (+nanz)| lr 4.03e-05 | 316.61 ms | 53.3% bf16 MFU | 1661374 tok/s
step   48/19560 | loss 8.646973 (+nanz)| norm 1.5954 (+nanz)| lr 4.11e-05 | 316.15 ms | 53.4% bf16 MFU | 1661206 tok/s
step   49/19560 | loss 8.607582 (+nanz)| norm 1.5651 (+nanz)| lr 4.20e-05 | 317.01 ms | 53.2% bf16 MFU | 1660803 tok/s
step   50/19560 | loss 8.586839 (+nanz)| norm 1.5604 (+nanz)| lr 4.29e-05 | 317.10 ms | 53.2% bf16 MFU | 1660398 tok/s
step   51/19560 | loss 8.536323 (+nanz)| norm 1.6132 (+nanz)| lr 4.37e-05 | 316.83 ms | 53.3% bf16 MFU | 1660095 tok/s
step   52/19560 | loss 8.492963 (+nanz)| norm 1.7036 (+nanz)| lr 4.46e-05 | 316.48 ms | 53.3% bf16 MFU | 1659908 tok/s
step   53/19560 | loss 8.489202 (+nanz)| norm 1.7864 (+nanz)| lr 4.54e-05 | 316.81 ms | 53.3% bf16 MFU | 1659640 tok/s
step   54/19560 | loss 8.421484 (+nanz)| norm 1.5661 (+nanz)| lr 4.63e-05 | 316.76 ms | 53.3% bf16 MFU | 1659400 tok/s
step   55/19560 | loss 8.446745 (+nanz)| norm 1.4467 (+nanz)| lr 4.71e-05 | 316.79 ms | 53.3% bf16 MFU | 1659167 tok/s
step   56/19560 | loss 8.378387 (+nanz)| norm 1.6649 (+nanz)| lr 4.80e-05 | 316.73 ms | 53.3% bf16 MFU | 1658961 tok/s
step   57/19560 | loss 8.326707 (+nanz)| norm 1.6512 (+nanz)| lr 4.89e-05 | 317.15 ms | 53.2% bf16 MFU | 1658651 tok/s
step   58/19560 | loss 8.262538 (+nanz)| norm 1.6633 (+nanz)| lr 4.97e-05 | 317.71 ms | 53.1% bf16 MFU | 1658204 tok/s
step   59/19560 | loss 8.225888 (+nanz)| norm 1.7655 (+nanz)| lr 5.06e-05 | 316.92 ms | 53.3% bf16 MFU | 1658000 tok/s
step   60/19560 | loss 8.181088 (+nanz)| norm 1.4495 (+nanz)| lr 5.14e-05 | 317.11 ms | 53.2% bf16 MFU | 1657754 tok/s
step   61/19560 | loss 8.197222 (+nanz)| norm 1.7008 (+nanz)| lr 5.23e-05 | 317.48 ms | 53.2% bf16 MFU | 1657421 tok/s
step   62/19560 | loss 8.186007 (+nanz)| norm 1.3840 (+nanz)| lr 5.31e-05 | 317.33 ms | 53.2% bf16 MFU | 1657148 tok/s
step   63/19560 | loss 8.078360 (+nanz)| norm 1.3752 (+nanz)| lr 5.40e-05 | 316.38 ms | 53.3% bf16 MFU | 1657147 tok/s
step   64/19560 | loss 8.078139 (+nanz)| norm 1.4739 (+nanz)| lr 5.49e-05 | 317.47 ms | 53.2% bf16 MFU | 1656850 tok/s
step   65/19560 | loss 8.054733 (+nanz)| norm 1.4442 (+nanz)| lr 5.57e-05 | 317.15 ms | 53.2% bf16 MFU | 1656657 tok/s
step   66/19560 | loss 7.981145 (+nanz)| norm 1.6291 (+nanz)| lr 5.66e-05 | 317.78 ms | 53.1% bf16 MFU | 1656304 tok/s
step   67/19560 | loss 7.928283 (+nanz)| norm 1.3553 (+nanz)| lr 5.74e-05 | 317.21 ms | 53.2% bf16 MFU | 1656124 tok/s
step   68/19560 | loss 7.929861 (+nanz)| norm 1.3051 (+nanz)| lr 5.83e-05 | 317.44 ms | 53.2% bf16 MFU | 1655891 tok/s
step   69/19560 | loss 7.937125 (+nanz)| norm 1.2620 (+nanz)| lr 5.91e-05 | 317.23 ms | 53.2% bf16 MFU | 1655727 tok/s
step   70/19560 | loss 7.888303 (+nanz)| norm 1.2230 (+nanz)| lr 6.00e-05 | 318.91 ms | 52.9% bf16 MFU | 1655123 tok/s
step   71/19560 | loss 7.836903 (+nanz)| norm 1.3416 (+nanz)| lr 6.09e-05 | 317.28 ms | 53.2% bf16 MFU | 1654985 tok/s
step   72/19560 | loss 7.792533 (+nanz)| norm 1.1669 (+nanz)| lr 6.17e-05 | 317.11 ms | 53.2% bf16 MFU | 1654899 tok/s
step   73/19560 | loss 7.773809 (+nanz)| norm 1.1522 (+nanz)| lr 6.26e-05 | 318.22 ms | 53.0% bf16 MFU | 1654524 tok/s
step   74/19560 | loss 7.792335 (+nanz)| norm 1.4781 (+nanz)| lr 6.34e-05 | 318.10 ms | 53.1% bf16 MFU | 1654199 tok/s
step   75/19560 | loss 7.717278 (+nanz)| norm 1.3997 (+nanz)| lr 6.43e-05 | 317.26 ms | 53.2% bf16 MFU | 1654116 tok/s
step   76/19560 | loss 7.568539 (+nanz)| norm 1.0994 (+nanz)| lr 6.51e-05 | 317.61 ms | 53.1% bf16 MFU | 1653942 tok/s
step   77/19560 | loss 7.679488 (+nanz)| norm 1.7398 (+nanz)| lr 6.60e-05 | 318.15 ms | 53.0% bf16 MFU | 1653634 tok/s
step   78/19560 | loss 7.633045 (+nanz)| norm 0.9579 (+nanz)| lr 6.69e-05 | 317.74 ms | 53.1% bf16 MFU | 1653452 tok/s
step   79/19560 | loss 7.592651 (+nanz)| norm 1.2439 (+nanz)| lr 6.77e-05 | 317.91 ms | 53.1% bf16 MFU | 1653233 tok/s
step   80/19560 | loss 7.616010 (+nanz)| norm 0.9443 (+nanz)| lr 6.86e-05 | 318.23 ms | 53.0% bf16 MFU | 1652942 tok/s
step   81/19560 | loss 7.424067 (+nanz)| norm 1.4287 (+nanz)| lr 6.94e-05 | 318.45 ms | 53.0% bf16 MFU | 1652608 tok/s
step   82/19560 | loss 7.515984 (+nanz)| norm 1.5817 (+nanz)| lr 7.03e-05 | 318.30 ms | 53.0% bf16 MFU | 1652331 tok/s
step   83/19560 | loss 7.442465 (+nanz)| norm 0.9805 (+nanz)| lr 7.11e-05 | 318.05 ms | 53.1% bf16 MFU | 1652134 tok/s
step   84/19560 | loss 7.463772 (+nanz)| norm 1.0387 (+nanz)| lr 7.20e-05 | 318.95 ms | 52.9% bf16 MFU | 1651712 tok/s
step   85/19560 | loss 7.399567 (+nanz)| norm 1.3155 (+nanz)| lr 7.29e-05 | 317.95 ms | 53.1% bf16 MFU | 1651572 tok/s
step   86/19560 | loss 7.403726 (+nanz)| norm 1.0048 (+nanz)| lr 7.37e-05 | 318.22 ms | 53.0% bf16 MFU | 1651368 tok/s
step   87/19560 | loss 7.372361 (+nanz)| norm 0.9639 (+nanz)| lr 7.46e-05 | 318.97 ms | 52.9% bf16 MFU | 1650979 tok/s
step   88/19560 | loss 7.304776 (+nanz)| norm 1.0455 (+nanz)| lr 7.54e-05 | 318.20 ms | 53.0% bf16 MFU | 1650810 tok/s
step   89/19560 | loss 7.340996 (+nanz)| norm 0.9987 (+nanz)| lr 7.63e-05 | 318.88 ms | 52.9% bf16 MFU | 1650474 tok/s
step   90/19560 | loss 7.323092 (+nanz)| norm 0.9632 (+nanz)| lr 7.71e-05 | 318.92 ms | 52.9% bf16 MFU | 1650144 tok/s
step   91/19560 | loss 7.380370 (+nanz)| norm 0.9188 (+nanz)| lr 7.80e-05 | 318.48 ms | 53.0% bf16 MFU | 1649947 tok/s
step   92/19560 | loss 7.240514 (+nanz)| norm 1.3798 (+nanz)| lr 7.89e-05 | 318.65 ms | 53.0% bf16 MFU | 1649716 tok/s
step   93/19560 | loss 7.278755 (+nanz)| norm 1.0515 (+nanz)| lr 7.97e-05 | 318.62 ms | 53.0% bf16 MFU | 1649502 tok/s
step   94/19560 | loss 7.172446 (+nanz)| norm 0.6091 (+nanz)| lr 8.06e-05 | 319.39 ms | 52.8% bf16 MFU | 1649101 tok/s
step   95/19560 | loss 7.206691 (+nanz)| norm 1.0823 (+nanz)| lr 8.14e-05 | 318.61 ms | 53.0% bf16 MFU | 1648922 tok/s
step   96/19560 | loss 7.126145 (+nanz)| norm 1.0807 (+nanz)| lr 8.23e-05 | 319.09 ms | 52.9% bf16 MFU | 1648628 tok/s
step   97/19560 | loss 7.140677 (+nanz)| norm 0.7696 (+nanz)| lr 8.31e-05 | 318.52 ms | 53.0% bf16 MFU | 1648495 tok/s
step   98/19560 | loss 7.245871 (+nanz)| norm 0.7834 (+nanz)| lr 8.40e-05 | 319.75 ms | 52.8% bf16 MFU | 1648051 tok/s
step   99/19560 | loss 7.150112 (+nanz)| norm 0.9784 (+nanz)| lr 8.49e-05 | 319.48 ms | 52.8% bf16 MFU | 1647701 tok/s
step  100/19560 | loss 7.146232 (+nanz)| norm 0.7532 (+nanz)| lr 8.57e-05 | 319.01 ms | 52.9% bf16 MFU | 1647488 tok/s
step  101/19560 | loss 7.090318 (+nanz)| norm 0.9386 (+nanz)| lr 8.66e-05 | 318.51 ms | 53.0% bf16 MFU | 1647415 tok/s
step  102/19560 | loss 7.049257 (+nanz)| norm 0.8929 (+nanz)| lr 8.74e-05 | 319.68 ms | 52.8% bf16 MFU | 1647043 tok/s
step  103/19560 | loss 7.089221 (+nanz)| norm 0.9470 (+nanz)| lr 8.83e-05 | 319.13 ms | 52.9% bf16 MFU | 1646833 tok/s
step  104/19560 | loss 7.151436 (+nanz)| norm 1.0774 (+nanz)| lr 8.91e-05 | 319.46 ms | 52.8% bf16 MFU | 1646550 tok/s
step  105/19560 | loss 7.035321 (+nanz)| norm 0.8694 (+nanz)| lr 9.00e-05 | 320.22 ms | 52.7% bf16 MFU | 1646082 tok/s
step  106/19560 | loss 7.067957 (+nanz)| norm 0.9092 (+nanz)| lr 9.09e-05 | 319.38 ms | 52.8% bf16 MFU | 1645857 tok/s
step  107/19560 | loss 6.896207 (+nanz)| norm 0.9970 (+nanz)| lr 9.17e-05 | 319.57 ms | 52.8% bf16 MFU | 1645594 tok/s
step  108/19560 | loss 7.045938 (+nanz)| norm 0.8555 (+nanz)| lr 9.26e-05 | 320.33 ms | 52.7% bf16 MFU | 1645148 tok/s
step  109/19560 | loss 6.968539 (+nanz)| norm 0.8375 (+nanz)| lr 9.34e-05 | 319.29 ms | 52.9% bf16 MFU | 1644992 tok/s
step  110/19560 | loss 7.056652 (+nanz)| norm 0.8642 (+nanz)| lr 9.43e-05 | 320.58 ms | 52.6% bf16 MFU | 1644513 tok/s
step  111/19560 | loss 6.921691 (+nanz)| norm 1.0757 (+nanz)| lr 9.51e-05 | 320.35 ms | 52.7% bf16 MFU | 1644117 tok/s
step  112/19560 | loss 6.911484 (+nanz)| norm 1.1640 (+nanz)| lr 9.60e-05 | 320.26 ms | 52.7% bf16 MFU | 1643764 tok/s
step  113/19560 | loss 7.007264 (+nanz)| norm 0.8044 (+nanz)| lr 9.69e-05 | 319.60 ms | 52.8% bf16 MFU | 1643597 tok/s
step  114/19560 | loss 6.890769 (+nanz)| norm 1.9847 (+nanz)| lr 9.77e-05 | 320.29 ms | 52.7% bf16 MFU | 1643263 tok/s
step  115/19560 | loss 6.908181 (+nanz)| norm 1.0685 (+nanz)| lr 9.86e-05 | 319.79 ms | 52.8% bf16 MFU | 1643073 tok/s
step  116/19560 | loss 6.931115 (+nanz)| norm 0.7416 (+nanz)| lr 9.94e-05 | 319.62 ms | 52.8% bf16 MFU | 1642938 tok/s
step  117/19560 | loss 6.991655 (+nanz)| norm 1.4740 (+nanz)| lr 1.00e-04 | 320.40 ms | 52.7% bf16 MFU | 1642607 tok/s
step  118/19560 | loss 6.873771 (+nanz)| norm 1.0094 (+nanz)| lr 1.01e-04 | 320.56 ms | 52.6% bf16 MFU | 1642253 tok/s
step  119/19560 | loss 6.878549 (+nanz)| norm 0.8534 (+nanz)| lr 1.02e-04 | 320.33 ms | 52.7% bf16 MFU | 1641975 tok/s
step  120/19560 | loss 6.912476 (+nanz)| norm 0.7953 (+nanz)| lr 1.03e-04 | 320.25 ms | 52.7% bf16 MFU | 1641732 tok/s
step  121/19560 | loss 6.926718 (+nanz)| norm 0.6222 (+nanz)| lr 1.04e-04 | 320.77 ms | 52.6% bf16 MFU | 1641369 tok/s
step  122/19560 | loss 6.864659 (+nanz)| norm 0.7715 (+nanz)| lr 1.05e-04 | 319.45 ms | 52.8% bf16 MFU | 1641362 tok/s
step  123/19560 | loss 6.937659 (+nanz)| norm 0.8921 (+nanz)| lr 1.05e-04 | 319.90 ms | 52.8% bf16 MFU | 1641241 tok/s
step  124/19560 | loss 6.817866 (+nanz)| norm 0.8085 (+nanz)| lr 1.06e-04 | 320.50 ms | 52.7% bf16 MFU | 1640970 tok/s
step  125/19560 | loss 6.784693 (+nanz)| norm 1.1932 (+nanz)| lr 1.07e-04 | 320.04 ms | 52.7% bf16 MFU | 1640831 tok/s
step  126/19560 | loss 6.884774 (+nanz)| norm 1.0859 (+nanz)| lr 1.08e-04 | 319.83 ms | 52.8% bf16 MFU | 1640753 tok/s
step  127/19560 | loss 6.848583 (+nanz)| norm 0.9801 (+nanz)| lr 1.09e-04 | 320.86 ms | 52.6% bf16 MFU | 1640416 tok/s
step  128/19560 | loss 6.754868 (+nanz)| norm 0.9551 (+nanz)| lr 1.10e-04 | 320.56 ms | 52.6% bf16 MFU | 1640172 tok/s
step  129/19560 | loss 6.737210 (-1.32z)| norm 1.2265 (-0.35z)| lr 1.11e-04 | 320.73 ms | 52.6% bf16 MFU | 1639897 tok/s
step  130/19560 | loss 6.802297 (-1.25z)| norm 0.6604 (-0.63z)| lr 1.11e-04 | 320.29 ms | 52.7% bf16 MFU | 1639748 tok/s
step  131/19560 | loss 6.728189 (-1.31z)| norm 0.6409 (-0.70z)| lr 1.12e-04 | 320.14 ms | 52.7% bf16 MFU | 1639645 tok/s
step  132/19560 | loss 6.695301 (-1.33z)| norm 1.0297 (-0.52z)| lr 1.13e-04 | 320.54 ms | 52.7% bf16 MFU | 1639444 tok/s
step  133/19560 | loss 6.712407 (-1.31z)| norm 0.8658 (-0.70z)| lr 1.14e-04 | 320.53 ms | 52.7% bf16 MFU | 1639257 tok/s
step  134/19560 | loss 6.745340 (-1.26z)| norm 0.9275 (-0.70z)| lr 1.15e-04 | 320.54 ms | 52.7% bf16 MFU | 1639076 tok/s
step  135/19560 | loss 6.685360 (-1.31z)| norm 1.0226 (-0.64z)| lr 1.16e-04 | 320.51 ms | 52.7% bf16 MFU | 1638911 tok/s
step  136/19560 | loss 6.695441 (-1.29z)| norm 1.2460 (-0.37z)| lr 1.17e-04 | 320.55 ms | 52.7% bf16 MFU | 1638746 tok/s
step  137/19560 | loss 6.700593 (-1.27z)| norm 0.9276 (-0.85z)| lr 1.17e-04 | 321.39 ms | 52.5% bf16 MFU | 1638373 tok/s
step  138/19560 | loss 6.694285 (-1.26z)| norm 0.9143 (-0.91z)| lr 1.18e-04 | 320.98 ms | 52.6% bf16 MFU | 1638124 tok/s
step  139/19560 | loss 6.701507 (-1.24z)| norm 1.0674 (-0.66z)| lr 1.19e-04 | 321.38 ms | 52.5% bf16 MFU | 1637785 tok/s
step  140/19560 | loss 6.649533 (-1.28z)| norm 0.8993 (-0.97z)| lr 1.20e-04 | 320.71 ms | 52.6% bf16 MFU | 1637633 tok/s
step  141/19560 | loss 6.643652 (-1.27z)| norm 1.2593 (-0.28z)| lr 1.21e-04 | 320.40 ms | 52.7% bf16 MFU | 1637570 tok/s
step  142/19560 | loss 6.653690 (-1.25z)| norm 0.7099 (-1.34z)| lr 1.22e-04 | 320.37 ms | 52.7% bf16 MFU | 1637516 tok/s
step  143/19560 | loss 6.653822 (-1.23z)| norm 0.7694 (-1.22z)| lr 1.23e-04 | 321.36 ms | 52.5% bf16 MFU | 1637212 tok/s
step  144/19560 | loss 6.654366 (-1.22z)| norm 0.5592 (-1.62z)| lr 1.23e-04 | 320.58 ms | 52.6% bf16 MFU | 1637122 tok/s
step  145/19560 | loss 6.625488 (-1.23z)| norm 0.7303 (-1.26z)| lr 1.24e-04 | 321.52 ms | 52.5% bf16 MFU | 1636799 tok/s
step  146/19560 | loss 6.673954 (-1.17z)| norm 0.6119 (-1.48z)| lr 1.25e-04 | 321.15 ms | 52.6% bf16 MFU | 1636585 tok/s
step  147/19560 | loss 6.635135 (-1.20z)| norm 0.7076 (-1.27z)| lr 1.26e-04 | 320.76 ms | 52.6% bf16 MFU | 1636481 tok/s
step  148/19560 | loss 6.636708 (-1.18z)| norm 1.0659 (-0.52z)| lr 1.27e-04 | 321.21 ms | 52.5% bf16 MFU | 1636269 tok/s
step  149/19560 | loss 6.620650 (-1.19z)| norm 1.1442 (-0.34z)| lr 1.28e-04 | 321.65 ms | 52.5% bf16 MFU | 1635955 tok/s
step  150/19560 | loss 6.670755 (-1.12z)| norm 0.8805 (-0.88z)| lr 1.29e-04 | 321.00 ms | 52.6% bf16 MFU | 1635822 tok/s
step  151/19560 | loss 6.596502 (-1.19z)| norm 1.6190 (+0.69z)| lr 1.29e-04 | 321.24 ms | 52.5% bf16 MFU | 1635634 tok/s
step  152/19560 | loss 6.519027 (-1.27z)| norm 0.9438 (-0.74z)| lr 1.30e-04 | 320.94 ms | 52.6% bf16 MFU | 1635532 tok/s
step  153/19560 | loss 6.655558 (-1.10z)| norm 0.9484 (-0.72z)| lr 1.31e-04 | 320.74 ms | 52.6% bf16 MFU | 1635486 tok/s
step  154/19560 | loss 6.590478 (-1.16z)| norm 1.0792 (-0.42z)| lr 1.32e-04 | 321.28 ms | 52.5% bf16 MFU | 1635305 tok/s
step  155/19560 | loss 6.582931 (-1.16z)| norm 1.3952 (+0.30z)| lr 1.33e-04 | 321.80 ms | 52.4% bf16 MFU | 1635000 tok/s
step  156/19560 | loss 6.626073 (-1.10z)| norm 0.8860 (-0.83z)| lr 1.34e-04 | 320.62 ms | 52.6% bf16 MFU | 1635011 tok/s
step  157/19560 | loss 6.556036 (-1.17z)| norm 1.0843 (-0.37z)| lr 1.35e-04 | 320.80 ms | 52.6% bf16 MFU | 1634976 tok/s
step  158/19560 | loss 6.570563 (-1.14z)| norm 1.1266 (-0.26z)| lr 1.35e-04 | 321.20 ms | 52.5% bf16 MFU | 1634842 tok/s
step  159/19560 | loss 6.558052 (-1.15z)| norm 0.9948 (-0.56z)| lr 1.36e-04 | 321.81 ms | 52.4% bf16 MFU | 1634560 tok/s
step  160/19560 | loss 6.537641 (-1.16z)| norm 0.8370 (-0.93z)| lr 1.37e-04 | 321.36 ms | 52.5% bf16 MFU | 1634405 tok/s
step  161/19560 | loss 6.579946 (-1.10z)| norm 0.7336 (-1.20z)| lr 1.38e-04 | 321.32 ms | 52.5% bf16 MFU | 1634269 tok/s
step  162/19560 | loss 6.512710 (-1.18z)| norm 0.9114 (-0.74z)| lr 1.39e-04 | 322.31 ms | 52.4% bf16 MFU | 1633888 tok/s
step  163/19560 | loss 6.482497 (-1.21z)| norm 1.1708 (-0.03z)| lr 1.40e-04 | 321.78 ms | 52.4% bf16 MFU | 1633660 tok/s
step  164/19560 | loss 6.473069 (-1.21z)| norm 1.4080 (+0.66z)| lr 1.41e-04 | 321.06 ms | 52.6% bf16 MFU | 1633626 tok/s
step  165/19560 | loss 6.495692 (-1.17z)| norm 1.0809 (-0.25z)| lr 1.41e-04 | 321.50 ms | 52.5% bf16 MFU | 1633481 tok/s
step  166/19560 | loss 6.482262 (-1.18z)| norm 0.8868 (-0.80z)| lr 1.42e-04 | 321.67 ms | 52.5% bf16 MFU | 1633303 tok/s
start Wed Nov 20 05:55:47 UTC 2024
+-----------------------+----------------------------------------------------+
| Parameter             | Value                                              |
+-----------------------+----------------------------------------------------+
| train data pattern    | dev/data/fineweb10B/fineweb_train_*.bin            |
| val data pattern      | dev/data/fineweb10B/fineweb_val_*.bin              |
| output log dir        | log124M                                            |
| checkpoint_every      | 5000                                               |
| resume                | 0                                                  |
| micro batch size B    | 64                                                 |
| sequence length T     | 1024                                               |
| total batch size      | 524288                                             |
| LR scheduler          | cosine                                             |
| learning rate (LR)    | 6.000000e-04                                       |
| warmup iterations     | 700                                                |
| final LR fraction     | 0.000000e+00                                       |
| weight decay          | 1.000000e-01                                       |
| skip update lossz     | 0.000000                                           |
| skip update gradz     | 0.000000                                           |
| max_steps             | -1                                                 |
| val_loss_every        | 250                                                |
| val_max_steps         | 20                                                 |
| sample_every          | 20000                                              |
| genT                  | 64                                                 |
| overfit_single_batch  | 0                                                  |
| use_master_weights    | enabled                                            |
| gelu_fusion           | 0                                                  |
| recompute             | 1                                                  |
+-----------------------+----------------------------------------------------+
| device                | NVIDIA A100-SXM4-40GB                              |
| peak TFlops           | 312.0                                              |
| precision             | BF16                                               |
+-----------------------+----------------------------------------------------+
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
| weight init method    | d12                                                |
| max_sequence_length T | 1024                                               |
| vocab_size V          | 50257                                              |
| padded_vocab_size Vp  | 50304                                              |
| num_layers L          | 12                                                 |
| num_heads NH          | 12                                                 |
| channels C            | 768                                                |
| num_parameters        | 124475904                                          |
+-----------------------+----------------------------------------------------+
| train_num_batches     | 19560                                              |
| val_num_batches       | 20                                                 |
+-----------------------+----------------------------------------------------+
| run hellaswag         | yes                                                |
+-----------------------+----------------------------------------------------+
| num_processes         | 8                                                  |
| zero_stage            | 1                                                  |
+-----------------------+----------------------------------------------------+
num_parameters: 124475904 => bytes: 248951808
allocated 237 MiB for model parameters
batch_size B=64 * seq_len T=1024 * num_processes=8 and total_batch_size=524288
=> setting grad_accum_steps=1
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
allocating 237 MiB for parameter gradients
allocating 21216 MiB for activations
allocating 59 MiB for AdamW optimizer state m
allocating 59 MiB for AdamW optimizer state v
allocating 59 MiB for master copy of params
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
---
WARNING: Failed to open the tokenizer file gpt2_tokenizer.bin
The Tokenizer is a new feature added April 14 2024.
Re-run `python train_gpt2.py` to write it
---
device memory usage: 23129 MiB / 40326 MiB
memory per sequence: 331 MiB
 -> estimated maximum batch size: 115
val loss 11.009204
step    1/19560 | loss 11.009109 (+nanz)| norm 15.3717 (+nanz)| lr 8.57e-07 | 696.92 ms | 24.2% bf16 MFU | 752294 tok/s
step    2/19560 | loss 10.959850 (+nanz)| norm 15.2377 (+nanz)| lr 1.71e-06 | 313.72 ms | 53.8% bf16 MFU | 1671212 tok/s
step    3/19560 | loss 10.855611 (+nanz)| norm 14.8389 (+nanz)| lr 2.57e-06 | 315.14 ms | 53.6% bf16 MFU | 1667356 tok/s
step    4/19560 | loss 10.715738 (+nanz)| norm 13.1195 (+nanz)| lr 3.43e-06 | 315.67 ms | 53.5% bf16 MFU | 1665085 tok/s
step    5/19560 | loss 10.568426 (+nanz)| norm 10.4405 (+nanz)| lr 4.29e-06 | 314.86 ms | 53.6% bf16 MFU | 1665101 tok/s
step    6/19560 | loss 10.423454 (+nanz)| norm 8.4942 (+nanz)| lr 5.14e-06 | 315.10 ms | 53.6% bf16 MFU | 1664830 tok/s
step    7/19560 | loss 10.291792 (+nanz)| norm 7.2415 (+nanz)| lr 6.00e-06 | 315.63 ms | 53.5% bf16 MFU | 1664128 tok/s
step    8/19560 | loss 10.187212 (+nanz)| norm 6.1835 (+nanz)| lr 6.86e-06 | 314.60 ms | 53.6% bf16 MFU | 1664521 tok/s
step    9/19560 | loss 10.096745 (+nanz)| norm 5.2947 (+nanz)| lr 7.71e-06 | 315.47 ms | 53.5% bf16 MFU | 1664136 tok/s
step   10/19560 | loss 9.976257 (+nanz)| norm 4.6636 (+nanz)| lr 8.57e-06 | 315.13 ms | 53.6% bf16 MFU | 1664076 tok/s
step   11/19560 | loss 9.935361 (+nanz)| norm 3.8501 (+nanz)| lr 9.43e-06 | 314.56 ms | 53.7% bf16 MFU | 1664410 tok/s
step   12/19560 | loss 9.853884 (+nanz)| norm 3.3980 (+nanz)| lr 1.03e-05 | 315.67 ms | 53.5% bf16 MFU | 1663998 tok/s
step   13/19560 | loss 9.792458 (+nanz)| norm 3.0057 (+nanz)| lr 1.11e-05 | 315.94 ms | 53.4% bf16 MFU | 1663503 tok/s
step   14/19560 | loss 9.758899 (+nanz)| norm 2.6957 (+nanz)| lr 1.20e-05 | 315.41 ms | 53.5% bf16 MFU | 1663371 tok/s
step   15/19560 | loss 9.702504 (+nanz)| norm 2.5212 (+nanz)| lr 1.29e-05 | 315.25 ms | 53.5% bf16 MFU | 1663343 tok/s
step   16/19560 | loss 9.701744 (+nanz)| norm 2.3030 (+nanz)| lr 1.37e-05 | 316.39 ms | 53.3% bf16 MFU | 1662760 tok/s
step   17/19560 | loss 9.655616 (+nanz)| norm 2.2412 (+nanz)| lr 1.46e-05 | 314.88 ms | 53.6% bf16 MFU | 1662965 tok/s
step   18/19560 | loss 9.636614 (+nanz)| norm 2.1886 (+nanz)| lr 1.54e-05 | 314.89 ms | 53.6% bf16 MFU | 1663138 tok/s
step   19/19560 | loss 9.603564 (+nanz)| norm 2.2161 (+nanz)| lr 1.63e-05 | 314.83 ms | 53.6% bf16 MFU | 1663318 tok/s
step   20/19560 | loss 9.596482 (+nanz)| norm 2.1545 (+nanz)| lr 1.71e-05 | 315.22 ms | 53.5% bf16 MFU | 1663312 tok/s
step   21/19560 | loss 9.562165 (+nanz)| norm 2.1811 (+nanz)| lr 1.80e-05 | 315.59 ms | 53.5% bf16 MFU | 1663155 tok/s
step   22/19560 | loss 9.537733 (+nanz)| norm 2.1489 (+nanz)| lr 1.89e-05 | 315.39 ms | 53.5% bf16 MFU | 1663095 tok/s
step   23/19560 | loss 9.533237 (+nanz)| norm 2.0896 (+nanz)| lr 1.97e-05 | 315.02 ms | 53.6% bf16 MFU | 1663183 tok/s
step   24/19560 | loss 9.476657 (+nanz)| norm 2.1037 (+nanz)| lr 2.06e-05 | 315.23 ms | 53.5% bf16 MFU | 1663182 tok/s
step   25/19560 | loss 9.458346 (+nanz)| norm 2.0526 (+nanz)| lr 2.14e-05 | 315.10 ms | 53.6% bf16 MFU | 1663232 tok/s
step   26/19560 | loss 9.422850 (+nanz)| norm 2.0943 (+nanz)| lr 2.23e-05 | 315.80 ms | 53.4% bf16 MFU | 1663022 tok/s
step   27/19560 | loss 9.377939 (+nanz)| norm 2.1531 (+nanz)| lr 2.31e-05 | 315.89 ms | 53.4% bf16 MFU | 1662799 tok/s
step   28/19560 | loss 9.388626 (+nanz)| norm 1.9649 (+nanz)| lr 2.40e-05 | 315.54 ms | 53.5% bf16 MFU | 1662716 tok/s
step   29/19560 | loss 9.357561 (+nanz)| norm 2.4480 (+nanz)| lr 2.49e-05 | 315.81 ms | 53.4% bf16 MFU | 1662546 tok/s
step   30/19560 | loss 9.306387 (+nanz)| norm 1.9511 (+nanz)| lr 2.57e-05 | 316.32 ms | 53.4% bf16 MFU | 1662217 tok/s
step   31/19560 | loss 9.244957 (+nanz)| norm 2.2944 (+nanz)| lr 2.66e-05 | 315.64 ms | 53.5% bf16 MFU | 1662143 tok/s
step   32/19560 | loss 9.211149 (+nanz)| norm 2.2605 (+nanz)| lr 2.74e-05 | 317.27 ms | 53.2% bf16 MFU | 1661538 tok/s
step   33/19560 | loss 9.235588 (+nanz)| norm 2.5857 (+nanz)| lr 2.83e-05 | 316.16 ms | 53.4% bf16 MFU | 1661335 tok/s
step   34/19560 | loss 9.159510 (+nanz)| norm 2.4415 (+nanz)| lr 2.91e-05 | 316.08 ms | 53.4% bf16 MFU | 1661174 tok/s
step   35/19560 | loss 9.200710 (+nanz)| norm 1.9544 (+nanz)| lr 3.00e-05 | 316.53 ms | 53.3% bf16 MFU | 1660882 tok/s
step   36/19560 | loss 9.080112 (+nanz)| norm 2.4098 (+nanz)| lr 3.09e-05 | 316.35 ms | 53.3% bf16 MFU | 1660666 tok/s
step   37/19560 | loss 9.029282 (+nanz)| norm 1.8688 (+nanz)| lr 3.17e-05 | 316.29 ms | 53.4% bf16 MFU | 1660485 tok/s
step   38/19560 | loss 9.039858 (+nanz)| norm 1.9748 (+nanz)| lr 3.26e-05 | 316.99 ms | 53.2% bf16 MFU | 1660102 tok/s
step   39/19560 | loss 8.977991 (+nanz)| norm 1.8095 (+nanz)| lr 3.34e-05 | 317.55 ms | 53.1% bf16 MFU | 1659574 tok/s
step   40/19560 | loss 8.961290 (+nanz)| norm 2.0396 (+nanz)| lr 3.43e-05 | 316.64 ms | 53.3% bf16 MFU | 1659356 tok/s
step   41/19560 | loss 8.906673 (+nanz)| norm 1.8697 (+nanz)| lr 3.51e-05 | 317.42 ms | 53.2% bf16 MFU | 1658919 tok/s
step   42/19560 | loss 8.860168 (+nanz)| norm 1.6712 (+nanz)| lr 3.60e-05 | 317.66 ms | 53.1% bf16 MFU | 1658437 tok/s
step   43/19560 | loss 8.839666 (+nanz)| norm 1.7301 (+nanz)| lr 3.69e-05 | 316.28 ms | 53.4% bf16 MFU | 1658392 tok/s
step   44/19560 | loss 8.856472 (+nanz)| norm 1.6667 (+nanz)| lr 3.77e-05 | 317.10 ms | 53.2% bf16 MFU | 1658112 tok/s
step   45/19560 | loss 8.748165 (+nanz)| norm 1.6708 (+nanz)| lr 3.86e-05 | 318.71 ms | 53.0% bf16 MFU | 1657382 tok/s
step   46/19560 | loss 8.692653 (+nanz)| norm 1.6775 (+nanz)| lr 3.94e-05 | 317.29 ms | 53.2% bf16 MFU | 1657104 tok/s
step   47/19560 | loss 8.712533 (+nanz)| norm 1.5835 (+nanz)| lr 4.03e-05 | 316.96 ms | 53.2% bf16 MFU | 1656940 tok/s
step   48/19560 | loss 8.646987 (+nanz)| norm 1.5961 (+nanz)| lr 4.11e-05 | 317.18 ms | 53.2% bf16 MFU | 1656720 tok/s
step   49/19560 | loss 8.607571 (+nanz)| norm 1.5648 (+nanz)| lr 4.20e-05 | 317.69 ms | 53.1% bf16 MFU | 1656370 tok/s
step   50/19560 | loss 8.586817 (+nanz)| norm 1.5581 (+nanz)| lr 4.29e-05 | 317.18 ms | 53.2% bf16 MFU | 1656184 tok/s
step   51/19560 | loss 8.536273 (+nanz)| norm 1.6072 (+nanz)| lr 4.37e-05 | 316.89 ms | 53.3% bf16 MFU | 1656091 tok/s
step   52/19560 | loss 8.492901 (+nanz)| norm 1.7007 (+nanz)| lr 4.46e-05 | 317.45 ms | 53.2% bf16 MFU | 1655846 tok/s
step   53/19560 | loss 8.489247 (+nanz)| norm 1.7998 (+nanz)| lr 4.54e-05 | 317.93 ms | 53.1% bf16 MFU | 1655483 tok/s
step   54/19560 | loss 8.421532 (+nanz)| norm 1.5751 (+nanz)| lr 4.63e-05 | 317.67 ms | 53.1% bf16 MFU | 1655212 tok/s
step   55/19560 | loss 8.446787 (+nanz)| norm 1.4480 (+nanz)| lr 4.71e-05 | 316.55 ms | 53.3% bf16 MFU | 1655267 tok/s
step   56/19560 | loss 8.378431 (+nanz)| norm 1.6670 (+nanz)| lr 4.80e-05 | 317.54 ms | 53.1% bf16 MFU | 1655044 tok/s
step   57/19560 | loss 8.326777 (+nanz)| norm 1.6541 (+nanz)| lr 4.89e-05 | 317.91 ms | 53.1% bf16 MFU | 1654733 tok/s
step   58/19560 | loss 8.262592 (+nanz)| norm 1.6627 (+nanz)| lr 4.97e-05 | 317.74 ms | 53.1% bf16 MFU | 1654486 tok/s
step   59/19560 | loss 8.225939 (+nanz)| norm 1.7647 (+nanz)| lr 5.06e-05 | 318.23 ms | 53.0% bf16 MFU | 1654118 tok/s
step   60/19560 | loss 8.181162 (+nanz)| norm 1.4505 (+nanz)| lr 5.14e-05 | 317.53 ms | 53.2% bf16 MFU | 1653962 tok/s
step   61/19560 | loss 8.197295 (+nanz)| norm 1.6991 (+nanz)| lr 5.23e-05 | 317.84 ms | 53.1% bf16 MFU | 1653730 tok/s
step   62/19560 | loss 8.186028 (+nanz)| norm 1.3791 (+nanz)| lr 5.31e-05 | 318.47 ms | 53.0% bf16 MFU | 1653340 tok/s
step   63/19560 | loss 8.078424 (+nanz)| norm 1.3744 (+nanz)| lr 5.40e-05 | 318.06 ms | 53.1% bf16 MFU | 1653083 tok/s
step   64/19560 | loss 8.078199 (+nanz)| norm 1.4735 (+nanz)| lr 5.49e-05 | 316.94 ms | 53.3% bf16 MFU | 1653141 tok/s
step   65/19560 | loss 8.054811 (+nanz)| norm 1.4451 (+nanz)| lr 5.57e-05 | 318.27 ms | 53.0% bf16 MFU | 1652838 tok/s
step   66/19560 | loss 7.981157 (+nanz)| norm 1.6246 (+nanz)| lr 5.66e-05 | 318.65 ms | 53.0% bf16 MFU | 1652449 tok/s
step   67/19560 | loss 7.928317 (+nanz)| norm 1.3552 (+nanz)| lr 5.74e-05 | 317.91 ms | 53.1% bf16 MFU | 1652278 tok/s
step   68/19560 | loss 7.929958 (+nanz)| norm 1.3094 (+nanz)| lr 5.83e-05 | 317.97 ms | 53.1% bf16 MFU | 1652101 tok/s
step   69/19560 | loss 7.937172 (+nanz)| norm 1.2620 (+nanz)| lr 5.91e-05 | 319.02 ms | 52.9% bf16 MFU | 1651654 tok/s
step   70/19560 | loss 7.888276 (+nanz)| norm 1.2156 (+nanz)| lr 6.00e-05 | 318.55 ms | 53.0% bf16 MFU | 1651357 tok/s
step   71/19560 | loss 7.836855 (+nanz)| norm 1.3333 (+nanz)| lr 6.09e-05 | 318.25 ms | 53.0% bf16 MFU | 1651155 tok/s
step   72/19560 | loss 7.792550 (+nanz)| norm 1.1686 (+nanz)| lr 6.17e-05 | 318.10 ms | 53.1% bf16 MFU | 1651003 tok/s
step   73/19560 | loss 7.773930 (+nanz)| norm 1.1652 (+nanz)| lr 6.26e-05 | 319.02 ms | 52.9% bf16 MFU | 1650615 tok/s
step   74/19560 | loss 7.792465 (+nanz)| norm 1.4856 (+nanz)| lr 6.34e-05 | 318.27 ms | 53.0% bf16 MFU | 1650447 tok/s
step   75/19560 | loss 7.716968 (+nanz)| norm 1.3789 (+nanz)| lr 6.43e-05 | 317.49 ms | 53.2% bf16 MFU | 1650492 tok/s
step   76/19560 | loss 7.568437 (+nanz)| norm 1.0922 (+nanz)| lr 6.51e-05 | 318.67 ms | 53.0% bf16 MFU | 1650224 tok/s
step   77/19560 | loss 7.679068 (+nanz)| norm 1.7059 (+nanz)| lr 6.60e-05 | 319.03 ms | 52.9% bf16 MFU | 1649873 tok/s
step   78/19560 | loss 7.632849 (+nanz)| norm 0.9554 (+nanz)| lr 6.69e-05 | 318.88 ms | 52.9% bf16 MFU | 1649582 tok/s
step   79/19560 | loss 7.592070 (+nanz)| norm 1.2280 (+nanz)| lr 6.77e-05 | 318.49 ms | 53.0% bf16 MFU | 1649409 tok/s
step   80/19560 | loss 7.615588 (+nanz)| norm 0.9336 (+nanz)| lr 6.86e-05 | 318.47 ms | 53.0% bf16 MFU | 1649250 tok/s
step   81/19560 | loss 7.423123 (+nanz)| norm 1.3849 (+nanz)| lr 6.94e-05 | 318.67 ms | 53.0% bf16 MFU | 1649047 tok/s
step   82/19560 | loss 7.515193 (+nanz)| norm 1.5655 (+nanz)| lr 7.03e-05 | 319.08 ms | 52.9% bf16 MFU | 1648747 tok/s
step   83/19560 | loss 7.442195 (+nanz)| norm 0.9895 (+nanz)| lr 7.11e-05 | 319.01 ms | 52.9% bf16 MFU | 1648479 tok/s
step   84/19560 | loss 7.462711 (+nanz)| norm 1.0160 (+nanz)| lr 7.20e-05 | 319.62 ms | 52.8% bf16 MFU | 1648068 tok/s
step   85/19560 | loss 7.399843 (+nanz)| norm 1.4101 (+nanz)| lr 7.29e-05 | 318.71 ms | 53.0% bf16 MFU | 1647913 tok/s
step   86/19560 | loss 7.402619 (+nanz)| norm 0.9672 (+nanz)| lr 7.37e-05 | 318.41 ms | 53.0% bf16 MFU | 1647846 tok/s
step   87/19560 | loss 7.372180 (+nanz)| norm 0.9722 (+nanz)| lr 7.46e-05 | 319.19 ms | 52.9% bf16 MFU | 1647578 tok/s
step   88/19560 | loss 7.304500 (+nanz)| norm 1.0659 (+nanz)| lr 7.54e-05 | 319.43 ms | 52.8% bf16 MFU | 1647262 tok/s
step   89/19560 | loss 7.340682 (+nanz)| norm 1.0230 (+nanz)| lr 7.63e-05 | 319.15 ms | 52.9% bf16 MFU | 1647035 tok/s
step   90/19560 | loss 7.322897 (+nanz)| norm 0.9668 (+nanz)| lr 7.71e-05 | 319.81 ms | 52.8% bf16 MFU | 1646649 tok/s
step   91/19560 | loss 7.380168 (+nanz)| norm 0.9023 (+nanz)| lr 7.80e-05 | 318.77 ms | 52.9% bf16 MFU | 1646552 tok/s
step   92/19560 | loss 7.239254 (+nanz)| norm 1.2497 (+nanz)| lr 7.89e-05 | 319.17 ms | 52.9% bf16 MFU | 1646356 tok/s
step   93/19560 | loss 7.280473 (+nanz)| norm 1.2754 (+nanz)| lr 7.97e-05 | 319.14 ms | 52.9% bf16 MFU | 1646176 tok/s
step   94/19560 | loss 7.173059 (+nanz)| norm 0.6341 (+nanz)| lr 8.06e-05 | 319.84 ms | 52.8% bf16 MFU | 1645826 tok/s
step   95/19560 | loss 7.207663 (+nanz)| norm 1.1749 (+nanz)| lr 8.14e-05 | 319.74 ms | 52.8% bf16 MFU | 1645518 tok/s
step   96/19560 | loss 7.126929 (+nanz)| norm 1.1039 (+nanz)| lr 8.23e-05 | 319.82 ms | 52.8% bf16 MFU | 1645205 tok/s
step   97/19560 | loss 7.140967 (+nanz)| norm 0.7372 (+nanz)| lr 8.31e-05 | 319.02 ms | 52.9% bf16 MFU | 1645117 tok/s
step   98/19560 | loss 7.246431 (+nanz)| norm 0.8139 (+nanz)| lr 8.40e-05 | 320.12 ms | 52.7% bf16 MFU | 1644747 tok/s
step   99/19560 | loss 7.150344 (+nanz)| norm 0.9218 (+nanz)| lr 8.49e-05 | 319.66 ms | 52.8% bf16 MFU | 1644515 tok/s
step  100/19560 | loss 7.145090 (+nanz)| norm 0.6691 (+nanz)| lr 8.57e-05 | 319.76 ms | 52.8% bf16 MFU | 1644268 tok/s
step  101/19560 | loss 7.090052 (+nanz)| norm 0.8795 (+nanz)| lr 8.66e-05 | 320.24 ms | 52.7% bf16 MFU | 1643911 tok/s
step  102/19560 | loss 7.048390 (+nanz)| norm 0.7897 (+nanz)| lr 8.74e-05 | 319.96 ms | 52.7% bf16 MFU | 1643644 tok/s
step  103/19560 | loss 7.089171 (+nanz)| norm 0.9258 (+nanz)| lr 8.83e-05 | 319.52 ms | 52.8% bf16 MFU | 1643504 tok/s
step  104/19560 | loss 7.151195 (+nanz)| norm 1.0183 (+nanz)| lr 8.91e-05 | 319.80 ms | 52.8% bf16 MFU | 1643300 tok/s
step  105/19560 | loss 7.035669 (+nanz)| norm 0.8900 (+nanz)| lr 9.00e-05 | 320.26 ms | 52.7% bf16 MFU | 1642986 tok/s
step  106/19560 | loss 7.065207 (+nanz)| norm 0.6486 (+nanz)| lr 9.09e-05 | 320.05 ms | 52.7% bf16 MFU | 1642742 tok/s
step  107/19560 | loss 6.892458 (+nanz)| norm 0.8898 (+nanz)| lr 9.17e-05 | 320.26 ms | 52.7% bf16 MFU | 1642457 tok/s
step  108/19560 | loss 7.045056 (+nanz)| norm 0.8603 (+nanz)| lr 9.26e-05 | 320.70 ms | 52.6% bf16 MFU | 1642074 tok/s
step  109/19560 | loss 6.967371 (+nanz)| norm 0.8506 (+nanz)| lr 9.34e-05 | 320.54 ms | 52.7% bf16 MFU | 1641750 tok/s
step  110/19560 | loss 7.060790 (+nanz)| norm 1.1379 (+nanz)| lr 9.43e-05 | 319.62 ms | 52.8% bf16 MFU | 1641681 tok/s
step  111/19560 | loss 6.921129 (+nanz)| norm 0.9117 (+nanz)| lr 9.51e-05 | 320.24 ms | 52.7% bf16 MFU | 1641454 tok/s
step  112/19560 | loss 6.903554 (+nanz)| norm 0.6963 (+nanz)| lr 9.60e-05 | 320.72 ms | 52.6% bf16 MFU | 1641116 tok/s
step  113/19560 | loss 7.002238 (+nanz)| norm 0.6131 (+nanz)| lr 9.69e-05 | 322.25 ms | 52.4% bf16 MFU | 1640406 tok/s
step  114/19560 | loss 6.877550 (+nanz)| norm 0.9659 (+nanz)| lr 9.77e-05 | 321.01 ms | 52.6% bf16 MFU | 1640048 tok/s
step  115/19560 | loss 6.912578 (+nanz)| norm 1.3991 (+nanz)| lr 9.86e-05 | 321.66 ms | 52.5% bf16 MFU | 1639542 tok/s
step  116/19560 | loss 6.930488 (+nanz)| norm 0.7822 (+nanz)| lr 9.94e-05 | 320.56 ms | 52.6% bf16 MFU | 1639341 tok/s
step  117/19560 | loss 6.994165 (+nanz)| norm 1.4629 (+nanz)| lr 1.00e-04 | 320.58 ms | 52.6% bf16 MFU | 1639146 tok/s
step  118/19560 | loss 6.870842 (+nanz)| norm 0.8994 (+nanz)| lr 1.01e-04 | 321.30 ms | 52.5% bf16 MFU | 1638777 tok/s
step  119/19560 | loss 6.878057 (+nanz)| norm 0.8905 (+nanz)| lr 1.02e-04 | 320.99 ms | 52.6% bf16 MFU | 1638505 tok/s
step  120/19560 | loss 6.911891 (+nanz)| norm 0.9541 (+nanz)| lr 1.03e-04 | 321.44 ms | 52.5% bf16 MFU | 1638132 tok/s
step  121/19560 | loss 6.925647 (+nanz)| norm 0.7611 (+nanz)| lr 1.04e-04 | 320.86 ms | 52.6% bf16 MFU | 1637924 tok/s
step  122/19560 | loss 6.864729 (+nanz)| norm 0.8617 (+nanz)| lr 1.05e-04 | 321.47 ms | 52.5% bf16 MFU | 1637571 tok/s
step  123/19560 | loss 6.934637 (+nanz)| norm 1.0621 (+nanz)| lr 1.05e-04 | 321.70 ms | 52.5% bf16 MFU | 1637179 tok/s
step  124/19560 | loss 6.830661 (+nanz)| norm 1.4283 (+nanz)| lr 1.06e-04 | 320.95 ms | 52.6% bf16 MFU | 1636998 tok/s
step  125/19560 | loss 6.785739 (+nanz)| norm 1.0351 (+nanz)| lr 1.07e-04 | 321.16 ms | 52.6% bf16 MFU | 1636772 tok/s
step  126/19560 | loss 6.888448 (+nanz)| norm 1.2989 (+nanz)| lr 1.08e-04 | 321.26 ms | 52.5% bf16 MFU | 1636531 tok/s
step  127/19560 | loss 6.854775 (+nanz)| norm 1.5645 (+nanz)| lr 1.09e-04 | 321.76 ms | 52.5% bf16 MFU | 1636175 tok/s
step  128/19560 | loss 6.757652 (+nanz)| norm 0.8796 (+nanz)| lr 1.10e-04 | 321.22 ms | 52.5% bf16 MFU | 1635976 tok/s
step  129/19560 | loss 6.734069 (-1.32z)| norm 0.9832 (-0.46z)| lr 1.11e-04 | 321.40 ms | 52.5% bf16 MFU | 1635740 tok/s
step  130/19560 | loss 6.804447 (-1.25z)| norm 0.9410 (-0.49z)| lr 1.11e-04 | 322.35 ms | 52.4% bf16 MFU | 1635274 tok/s
step  131/19560 | loss 6.736572 (-1.30z)| norm 1.1193 (-0.42z)| lr 1.12e-04 | 321.83 ms | 52.4% bf16 MFU | 1634964 tok/s
step  132/19560 | loss 6.690763 (-1.34z)| norm 0.8417 (-0.65z)| lr 1.13e-04 | 320.95 ms | 52.6% bf16 MFU | 1634894 tok/s
step  133/19560 | loss 6.716677 (-1.30z)| norm 0.8852 (-0.68z)| lr 1.14e-04 | 321.11 ms | 52.6% bf16 MFU | 1634785 tok/s
step  134/19560 | loss 6.746459 (-1.26z)| norm 0.8890 (-0.74z)| lr 1.15e-04 | 321.30 ms | 52.5% bf16 MFU | 1634636 tok/s
step  135/19560 | loss 6.686617 (-1.31z)| norm 0.9627 (-0.71z)| lr 1.16e-04 | 321.82 ms | 52.4% bf16 MFU | 1634360 tok/s
step  136/19560 | loss 6.694745 (-1.29z)| norm 0.9746 (-0.74z)| lr 1.17e-04 | 321.61 ms | 52.5% bf16 MFU | 1634152 tok/s
step  137/19560 | loss 6.701411 (-1.27z)| norm 0.8374 (-0.99z)| lr 1.17e-04 | 322.18 ms | 52.4% bf16 MFU | 1633810 tok/s
step  138/19560 | loss 6.695772 (-1.26z)| norm 0.8120 (-1.08z)| lr 1.18e-04 | 322.23 ms | 52.4% bf16 MFU | 1633474 tok/s
step  139/19560 | loss 6.699075 (-1.24z)| norm 0.9601 (-0.85z)| lr 1.19e-04 | 321.53 ms | 52.5% bf16 MFU | 1633330 tok/s
step  140/19560 | loss 6.652607 (-1.28z)| norm 0.9474 (-0.88z)| lr 1.20e-04 | 321.25 ms | 52.5% bf16 MFU | 1633266 tok/s
step  141/19560 | loss 6.644627 (-1.27z)| norm 1.1908 (-0.41z)| lr 1.21e-04 | 322.03 ms | 52.4% bf16 MFU | 1633006 tok/s
step  142/19560 | loss 6.658344 (-1.24z)| norm 1.1292 (-0.52z)| lr 1.22e-04 | 321.54 ms | 52.5% bf16 MFU | 1632883 tok/s
step  143/19560 | loss 6.652598 (-1.23z)| norm 0.5062 (-1.75z)| lr 1.23e-04 | 321.09 ms | 52.6% bf16 MFU | 1632880 tok/s
step  144/19560 | loss 6.658814 (-1.21z)| norm 0.7841 (-1.17z)| lr 1.23e-04 | 321.29 ms | 52.5% bf16 MFU | 1632826 tok/s
step  145/19560 | loss 6.629550 (-1.23z)| norm 0.8909 (-0.94z)| lr 1.24e-04 | 320.94 ms | 52.6% bf16 MFU | 1632864 tok/s
step  146/19560 | loss 6.678525 (-1.16z)| norm 0.9792 (-0.75z)| lr 1.25e-04 | 321.88 ms | 52.4% bf16 MFU | 1632663 tok/s
step  147/19560 | loss 6.642830 (-1.19z)| norm 1.1263 (-0.43z)| lr 1.26e-04 | 321.43 ms | 52.5% bf16 MFU | 1632584 tok/s
step  148/19560 | loss 6.639793 (-1.18z)| norm 1.0252 (-0.63z)| lr 1.27e-04 | 322.25 ms | 52.4% bf16 MFU | 1632303 tok/s
step  149/19560 | loss 6.615000 (-1.19z)| norm 0.7926 (-1.11z)| lr 1.28e-04 | 321.73 ms | 52.5% bf16 MFU | 1632168 tok/s
step  150/19560 | loss 6.670747 (-1.12z)| norm 0.7933 (-1.09z)| lr 1.29e-04 | 321.68 ms | 52.5% bf16 MFU | 1632052 tok/s
step  151/19560 | loss 6.594179 (-1.19z)| norm 1.4163 (+0.26z)| lr 1.29e-04 | 322.05 ms | 52.4% bf16 MFU | 1631847 tok/s
step  152/19560 | loss 6.517328 (-1.27z)| norm 1.1646 (-0.27z)| lr 1.30e-04 | 321.29 ms | 52.5% bf16 MFU | 1631845 tok/s
step  153/19560 | loss 6.656227 (-1.10z)| norm 1.0052 (-0.61z)| lr 1.31e-04 | 320.75 ms | 52.6% bf16 MFU | 1631982 tok/s
step  154/19560 | loss 6.594466 (-1.16z)| norm 1.1414 (-0.30z)| lr 1.32e-04 | 321.37 ms | 52.5% bf16 MFU | 1631954 tok/s
step  155/19560 | loss 6.581488 (-1.16z)| norm 1.1147 (-0.34z)| lr 1.33e-04 | 321.14 ms | 52.6% bf16 MFU | 1631985 tok/s
step  156/19560 | loss 6.626822 (-1.10z)| norm 0.8736 (-0.88z)| lr 1.34e-04 | 321.82 ms | 52.4% bf16 MFU | 1631842 tok/s
step  157/19560 | loss 6.552612 (-1.18z)| norm 0.7725 (-1.11z)| lr 1.35e-04 | 322.66 ms | 52.3% bf16 MFU | 1631495 tok/s
step  158/19560 | loss 6.562229 (-1.16z)| norm 0.6758 (-1.33z)| lr 1.35e-04 | 321.14 ms | 52.6% bf16 MFU | 1631550 tok/s
step  159/19560 | loss 6.556561 (-1.15z)| norm 0.8551 (-0.89z)| lr 1.36e-04 | 322.11 ms | 52.4% bf16 MFU | 1631356 tok/s
step  160/19560 | loss 6.534228 (-1.17z)| norm 0.7214 (-1.21z)| lr 1.37e-04 | 322.40 ms | 52.3% bf16 MFU | 1631098 tok/s
step  161/19560 | loss 6.581291 (-1.10z)| norm 0.7718 (-1.10z)| lr 1.38e-04 | 321.83 ms | 52.4% bf16 MFU | 1630997 tok/s
step  162/19560 | loss 6.509456 (-1.18z)| norm 0.6473 (-1.43z)| lr 1.39e-04 | 321.96 ms | 52.4% bf16 MFU | 1630868 tok/s
step  163/19560 | loss 6.475272 (-1.22z)| norm 0.7170 (-1.23z)| lr 1.40e-04 | 323.06 ms | 52.2% bf16 MFU | 1630468 tok/s
step  164/19560 | loss 6.458817 (-1.23z)| norm 0.5773 (-1.63z)| lr 1.41e-04 | 321.15 ms | 52.6% bf16 MFU | 1630570 tok/s
step  165/19560 | loss 6.487965 (-1.18z)| norm 0.6924 (-1.29z)| lr 1.41e-04 | 322.18 ms | 52.4% bf16 MFU | 1630408 tok/s
step  166/19560 | loss 6.474494 (-1.19z)| norm 0.6740 (-1.34z)| lr 1.42e-04 | 321.59 ms | 52.5% bf16 MFU | 1630404 tok/s
step  167/19560 | loss 6.484888 (-1.17z)| norm 1.0290 (-0.30z)| lr 1.43e-04 | 321.52 ms | 52.5% bf16 MFU | 1630416 tok/s
step  168/19560 | loss 6.452101 (-1.20z)| norm 1.4963 (+1.10z)| lr 1.44e-04 | 321.99 ms | 52.4% bf16 MFU | 1630309 tok/s
step  169/19560 | loss 6.499334 (-1.13z)| norm 0.6267 (-1.49z)| lr 1.45e-04 | 321.46 ms | 52.5% bf16 MFU | 1630342 tok/s
step  170/19560 | loss 6.452170 (-1.19z)| norm 1.0261 (-0.27z)| lr 1.46e-04 | 321.12 ms | 52.6% bf16 MFU | 1630458 tok/s
step  171/19560 | loss 6.440082 (-1.20z)| norm 1.7258 (+1.87z)| lr 1.47e-04 | 321.48 ms | 52.5% bf16 MFU | 1630477 tok/s
step  172/19560 | loss 6.531271 (-1.05z)| norm 0.8970 (-0.65z)| lr 1.47e-04 | 322.19 ms | 52.4% bf16 MFU | 1630316 tok/s
step  173/19560 | loss 6.446108 (-1.18z)| norm 1.4760 (+1.15z)| lr 1.48e-04 | 321.31 ms | 52.5% bf16 MFU | 1630385 tok/s
step  174/19560 | loss 6.417402 (-1.21z)| norm 1.2472 (+0.45z)| lr 1.49e-04 | 322.12 ms | 52.4% bf16 MFU | 1630246 tok/s
step  175/19560 | loss 6.424203 (-1.19z)| norm 0.6903 (-1.28z)| lr 1.50e-04 | 322.23 ms | 52.4% bf16 MFU | 1630088 tok/s
step  176/19560 | loss 6.437121 (-1.17z)| norm 0.9103 (-0.57z)| lr 1.51e-04 | 321.07 ms | 52.6% bf16 MFU | 1630230 tok/s
step  177/19560 | loss 6.487861 (-1.07z)| norm 1.0471 (-0.13z)| lr 1.52e-04 | 322.70 ms | 52.3% bf16 MFU | 1629952 tok/s
step  178/19560 | loss 6.400074 (-1.21z)| norm 0.9577 (-0.40z)| lr 1.53e-04 | 321.99 ms | 52.4% bf16 MFU | 1629867 tok/s
step  179/19560 | loss 6.445080 (-1.13z)| norm 1.0410 (-0.12z)| lr 1.53e-04 | 322.78 ms | 52.3% bf16 MFU | 1629589 tok/s
step  180/19560 | loss 6.441687 (-1.13z)| norm 0.9112 (-0.53z)| lr 1.54e-04 | 322.27 ms | 52.4% bf16 MFU | 1629451 tok/s
step  181/19560 | loss 6.424420 (-1.15z)| norm 0.9778 (-0.30z)| lr 1.55e-04 | 321.78 ms | 52.4% bf16 MFU | 1629445 tok/s
step  182/19560 | loss 6.539721 (-0.94z)| norm 1.1865 (+0.43z)| lr 1.56e-04 | 322.22 ms | 52.4% bf16 MFU | 1629329 tok/s
step  183/19560 | loss 6.459341 (-1.08z)| norm 1.0446 (-0.05z)| lr 1.57e-04 | 322.00 ms | 52.4% bf16 MFU | 1629275 tok/s
step  184/19560 | loss 6.407334 (-1.17z)| norm 1.1060 (+0.18z)| lr 1.58e-04 | 322.13 ms | 52.4% bf16 MFU | 1629190 tok/s
step  185/19560 | loss 6.395887 (-1.19z)| norm 0.8576 (-0.68z)| lr 1.59e-04 | 322.12 ms | 52.4% bf16 MFU | 1629112 tok/s
step  186/19560 | loss 6.423013 (-1.13z)| norm 1.1102 (+0.24z)| lr 1.59e-04 | 322.30 ms | 52.4% bf16 MFU | 1628992 tok/s
step  187/19560 | loss 6.426555 (-1.11z)| norm 0.8073 (-0.86z)| lr 1.60e-04 | 322.12 ms | 52.4% bf16 MFU | 1628924 tok/s
step  188/19560 | loss 6.348249 (-1.27z)| norm 1.0249 (-0.03z)| lr 1.61e-04 | 322.08 ms | 52.4% bf16 MFU | 1628869 tok/s
step  189/19560 | loss 6.347377 (-1.26z)| norm 1.0403 (+0.05z)| lr 1.62e-04 | 321.72 ms | 52.5% bf16 MFU | 1628907 tok/s
step  190/19560 | loss 6.365944 (-1.22z)| norm 0.9139 (-0.43z)| lr 1.63e-04 | 323.12 ms | 52.2% bf16 MFU | 1628590 tok/s
step  191/19560 | loss 6.328228 (-1.29z)| norm 0.8223 (-0.78z)| lr 1.64e-04 | 322.48 ms | 52.3% bf16 MFU | 1628450 tok/s
step  192/19560 | loss 6.335621 (-1.27z)| norm 0.8242 (-0.76z)| lr 1.65e-04 | 321.83 ms | 52.4% bf16 MFU | 1628481 tok/s
step  193/19560 | loss 6.367826 (-1.20z)| norm 0.8237 (-0.75z)| lr 1.65e-04 | 321.93 ms | 52.4% bf16 MFU | 1628485 tok/s
step  194/19560 | loss 6.320416 (-1.30z)| norm 0.9747 (-0.12z)| lr 1.66e-04 | 321.48 ms | 52.5% bf16 MFU | 1628604 tok/s
step  195/19560 | loss 6.265522 (-1.41z)| norm 0.9921 (-0.04z)| lr 1.67e-04 | 322.17 ms | 52.4% bf16 MFU | 1628542 tok/s
step  196/19560 | loss 6.387854 (-1.12z)| norm 0.8391 (-0.66z)| lr 1.68e-04 | 321.37 ms | 52.5% bf16 MFU | 1628685 tok/s
step  197/19560 | loss 6.310363 (-1.30z)| norm 0.8593 (-0.57z)| lr 1.69e-04 | 321.74 ms | 52.5% bf16 MFU | 1628727 tok/s
step  198/19560 | loss 6.339876 (-1.23z)| norm 0.7897 (-0.84z)| lr 1.70e-04 | 321.21 ms | 52.5% bf16 MFU | 1628901 tok/s
step  199/19560 | loss 6.329884 (-1.25z)| norm 0.8923 (-0.40z)| lr 1.71e-04 | 322.06 ms | 52.4% bf16 MFU | 1628851 tok/s
step  200/19560 | loss 6.306967 (-1.30z)| norm 0.9885 (+0.01z)| lr 1.71e-04 | 321.59 ms | 52.5% bf16 MFU | 1628925 tok/s
step  201/19560 | loss 6.368891 (-1.13z)| norm 1.2945 (+1.29z)| lr 1.72e-04 | 321.45 ms | 52.5% bf16 MFU | 1629029 tok/s
step  202/19560 | loss 6.333399 (-1.22z)| norm 0.7737 (-0.89z)| lr 1.73e-04 | 321.73 ms | 52.5% bf16 MFU | 1629058 tok/s
step  203/19560 | loss 6.379382 (-1.09z)| norm 0.7853 (-0.83z)| lr 1.74e-04 | 321.36 ms | 52.5% bf16 MFU | 1629178 tok/s
step  204/19560 | loss 6.334266 (-1.20z)| norm 0.9555 (-0.09z)| lr 1.75e-04 | 321.97 ms | 52.4% bf16 MFU | 1629138 tok/s
step  205/19560 | loss 6.385811 (-1.06z)| norm 1.0454 (+0.33z)| lr 1.76e-04 | 321.52 ms | 52.5% bf16 MFU | 1629215 tok/s
step  206/19560 | loss 6.331663 (-1.20z)| norm 1.0411 (+0.31z)| lr 1.77e-04 | 322.27 ms | 52.4% bf16 MFU | 1629098 tok/s
step  207/19560 | loss 6.254904 (-1.42z)| norm 0.8483 (-0.54z)| lr 1.77e-04 | 322.25 ms | 52.4% bf16 MFU | 1628990 tok/s
step  208/19560 | loss 6.333177 (-1.18z)| norm 0.7313 (-1.06z)| lr 1.78e-04 | 322.78 ms | 52.3% bf16 MFU | 1628754 tok/s
step  209/19560 | loss 6.311634 (-1.24z)| norm 0.7074 (-1.15z)| lr 1.79e-04 | 321.87 ms | 52.4% bf16 MFU | 1628760 tok/s
step  210/19560 | loss 6.310075 (-1.24z)| norm 0.8357 (-0.56z)| lr 1.80e-04 | 322.22 ms | 52.4% bf16 MFU | 1628676 tok/s
step  211/19560 | loss 6.281957 (-1.31z)| norm 0.9084 (-0.22z)| lr 1.81e-04 | 322.38 ms | 52.4% bf16 MFU | 1628558 tok/s
step  212/19560 | loss 6.355189 (-1.08z)| norm 0.9187 (-0.17z)| lr 1.82e-04 | 322.39 ms | 52.4% bf16 MFU | 1628443 tok/s
step  213/19560 | loss 6.285794 (-1.29z)| norm 0.9755 (+0.11z)| lr 1.83e-04 | 322.19 ms | 52.4% bf16 MFU | 1628385 tok/s
step  214/19560 | loss 6.374501 (-0.99z)| norm 1.1681 (+1.01z)| lr 1.83e-04 | 322.23 ms | 52.4% bf16 MFU | 1628318 tok/s
step  215/19560 | loss 6.268120 (-1.34z)| norm 0.7327 (-1.03z)| lr 1.84e-04 | 321.87 ms | 52.4% bf16 MFU | 1628346 tok/s
step  216/19560 | loss 6.324082 (-1.14z)| norm 0.8923 (-0.27z)| lr 1.85e-04 | 322.16 ms | 52.4% bf16 MFU | 1628299 tok/s
step  217/19560 | loss 6.301116 (-1.21z)| norm 0.8987 (-0.24z)| lr 1.86e-04 | 322.55 ms | 52.3% bf16 MFU | 1628157 tok/s
step  218/19560 | loss 6.297749 (-1.21z)| norm 0.9448 (-0.02z)| lr 1.87e-04 | 322.24 ms | 52.4% bf16 MFU | 1628101 tok/s
step  219/19560 | loss 6.276949 (-1.28z)| norm 0.9657 (+0.08z)| lr 1.88e-04 | 322.04 ms | 52.4% bf16 MFU | 1628098 tok/s
step  220/19560 | loss 6.189223 (-1.58z)| norm 0.8820 (-0.31z)| lr 1.89e-04 | 322.11 ms | 52.4% bf16 MFU | 1628075 tok/s
step  221/19560 | loss 6.288687 (-1.21z)| norm 0.7327 (-1.00z)| lr 1.89e-04 | 322.37 ms | 52.4% bf16 MFU | 1627990 tok/s
step  222/19560 | loss 6.287546 (-1.20z)| norm 0.7229 (-1.05z)| lr 1.90e-04 | 321.98 ms | 52.4% bf16 MFU | 1628006 tok/s
step  223/19560 | loss 6.232458 (-1.40z)| norm 0.7927 (-0.71z)| lr 1.91e-04 | 322.53 ms | 52.3% bf16 MFU | 1627882 tok/s
step  224/19560 | loss 6.302462 (-1.12z)| norm 0.9380 (-0.00z)| lr 1.92e-04 | 321.88 ms | 52.4% bf16 MFU | 1627929 tok/s
step  225/19560 | loss 6.274525 (-1.22z)| norm 0.9228 (-0.08z)| lr 1.93e-04 | 321.83 ms | 52.4% bf16 MFU | 1627986 tok/s
step  226/19560 | loss 6.173187 (-1.60z)| norm 0.8746 (-0.32z)| lr 1.94e-04 | 322.07 ms | 52.4% bf16 MFU | 1627981 tok/s
step  227/19560 | loss 6.202115 (-1.48z)| norm 0.8800 (-0.29z)| lr 1.95e-04 | 321.38 ms | 52.5% bf16 MFU | 1628149 tok/s
step  228/19560 | loss 6.251428 (-1.27z)| norm 0.6701 (-1.31z)| lr 1.95e-04 | 322.09 ms | 52.4% bf16 MFU | 1628129 tok/s
step  229/19560 | loss 6.182144 (-1.53z)| norm 0.5626 (-1.79z)| lr 1.96e-04 | 323.25 ms | 52.2% bf16 MFU | 1627820 tok/s
step  230/19560 | loss 6.197141 (-1.46z)| norm 0.7991 (-0.66z)| lr 1.97e-04 | 321.94 ms | 52.4% bf16 MFU | 1627856 tok/s
step  231/19560 | loss 6.207821 (-1.40z)| norm 1.1295 (+0.90z)| lr 1.98e-04 | 321.42 ms | 52.5% bf16 MFU | 1628021 tok/s
step  232/19560 | loss 6.206169 (-1.40z)| norm 1.2466 (+1.44z)| lr 1.99e-04 | 321.84 ms | 52.4% bf16 MFU | 1628073 tok/s
step  233/19560 | loss 6.121982 (-1.74z)| norm 0.9204 (-0.10z)| lr 2.00e-04 | 321.96 ms | 52.4% bf16 MFU | 1628090 tok/s
step  234/19560 | loss 6.214460 (-1.33z)| norm 0.9283 (-0.07z)| lr 2.01e-04 | 321.81 ms | 52.4% bf16 MFU | 1628145 tok/s
step  235/19560 | loss 6.196240 (-1.39z)| norm 0.8991 (-0.21z)| lr 2.01e-04 | 321.47 ms | 52.5% bf16 MFU | 1628282 tok/s
step  236/19560 | loss 6.193775 (-1.39z)| norm 0.8129 (-0.62z)| lr 2.02e-04 | 322.15 ms | 52.4% bf16 MFU | 1628240 tok/s
step  237/19560 | loss 6.242153 (-1.17z)| norm 0.8246 (-0.56z)| lr 2.03e-04 | 322.26 ms | 52.4% bf16 MFU | 1628173 tok/s
step  238/19560 | loss 6.159338 (-1.53z)| norm 1.0368 (+0.45z)| lr 2.04e-04 | 322.55 ms | 52.3% bf16 MFU | 1628036 tok/s
step  239/19560 | loss 6.141267 (-1.59z)| norm 1.0644 (+0.58z)| lr 2.05e-04 | 322.32 ms | 52.4% bf16 MFU | 1627964 tok/s
step  240/19560 | loss 6.128356 (-1.62z)| norm 1.0487 (+0.49z)| lr 2.06e-04 | 322.02 ms | 52.4% bf16 MFU | 1627972 tok/s
step  241/19560 | loss 6.204836 (-1.27z)| norm 0.8554 (-0.45z)| lr 2.07e-04 | 322.34 ms | 52.4% bf16 MFU | 1627897 tok/s
step  242/19560 | loss 6.093307 (-1.75z)| norm 0.9986 (+0.24z)| lr 2.07e-04 | 322.04 ms | 52.4% bf16 MFU | 1627904 tok/s
step  243/19560 | loss 6.099175 (-1.71z)| norm 1.0532 (+0.53z)| lr 2.08e-04 | 321.54 ms | 52.5% bf16 MFU | 1628037 tok/s
step  244/19560 | loss 6.157618 (-1.42z)| norm 0.7963 (-0.73z)| lr 2.09e-04 | 322.37 ms | 52.4% bf16 MFU | 1627954 tok/s
step  245/19560 | loss 6.210825 (-1.17z)| norm 0.6617 (-1.39z)| lr 2.10e-04 | 322.94 ms | 52.3% bf16 MFU | 1627731 tok/s
step  246/19560 | loss 6.159844 (-1.40z)| norm 0.6581 (-1.39z)| lr 2.11e-04 | 322.41 ms | 52.3% bf16 MFU | 1627652 tok/s
step  247/19560 | loss 6.195800 (-1.21z)| norm 0.5860 (-1.72z)| lr 2.12e-04 | 322.29 ms | 52.4% bf16 MFU | 1627608 tok/s
step  248/19560 | loss 6.181733 (-1.27z)| norm 0.6444 (-1.41z)| lr 2.13e-04 | 321.94 ms | 52.4% bf16 MFU | 1627653 tok/s
step  249/19560 | loss 6.091333 (-1.70z)| norm 0.6953 (-1.15z)| lr 2.13e-04 | 322.52 ms | 52.3% bf16 MFU | 1627550 tok/s
step  250/19560 | loss 6.188012 (-1.21z)| norm 0.8369 (-0.46z)| lr 2.14e-04 | 321.96 ms | 52.4% bf16 MFU | 1627595 tok/s
val loss 6.173676
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2475/10042 = 0.246465
step  251/19560 | loss 6.181686 (-1.24z)| norm 1.3707 (+2.09z)| lr 2.15e-04 | 321.77 ms | 52.5% bf16 MFU | 1627686 tok/s
step  252/19560 | loss 6.139812 (-1.44z)| norm 0.8852 (-0.22z)| lr 2.16e-04 | 322.01 ms | 52.4% bf16 MFU | 1627709 tok/s
step  253/19560 | loss 6.184560 (-1.19z)| norm 0.9596 (+0.15z)| lr 2.17e-04 | 322.66 ms | 52.3% bf16 MFU | 1627569 tok/s
step  254/19560 | loss 6.075264 (-1.75z)| norm 0.8222 (-0.51z)| lr 2.18e-04 | 321.45 ms | 52.5% bf16 MFU | 1627741 tok/s
step  255/19560 | loss 6.159996 (-1.30z)| norm 0.8925 (-0.15z)| lr 2.19e-04 | 322.19 ms | 52.4% bf16 MFU | 1627716 tok/s
step  256/19560 | loss 6.182535 (-1.16z)| norm 1.0928 (+0.88z)| lr 2.19e-04 | 322.73 ms | 52.3% bf16 MFU | 1627556 tok/s
step  257/19560 | loss 6.129143 (-1.43z)| norm 1.1002 (+0.91z)| lr 2.20e-04 | 321.68 ms | 52.5% bf16 MFU | 1627670 tok/s
step  258/19560 | loss 6.128068 (-1.43z)| norm 0.8288 (-0.48z)| lr 2.21e-04 | 322.04 ms | 52.4% bf16 MFU | 1627689 tok/s
step  259/19560 | loss 6.116661 (-1.48z)| norm 0.6254 (-1.50z)| lr 2.22e-04 | 322.20 ms | 52.4% bf16 MFU | 1627666 tok/s
step  260/19560 | loss 6.054629 (-1.79z)| norm 0.5884 (-1.66z)| lr 2.23e-04 | 322.45 ms | 52.3% bf16 MFU | 1627580 tok/s
step  261/19560 | loss 6.157582 (-1.20z)| norm 0.5724 (-1.71z)| lr 2.24e-04 | 322.17 ms | 52.4% bf16 MFU | 1627570 tok/s
step  262/19560 | loss 6.116055 (-1.42z)| norm 0.6775 (-1.17z)| lr 2.25e-04 | 321.80 ms | 52.4% bf16 MFU | 1627652 tok/s
step  263/19560 | loss 6.060465 (-1.72z)| norm 0.6570 (-1.25z)| lr 2.25e-04 | 323.25 ms | 52.2% bf16 MFU | 1627367 tok/s
step  264/19560 | loss 6.059302 (-1.70z)| norm 0.6723 (-1.16z)| lr 2.26e-04 | 322.49 ms | 52.3% bf16 MFU | 1627286 tok/s
step  265/19560 | loss 6.071833 (-1.61z)| norm 0.7637 (-0.70z)| lr 2.27e-04 | 322.53 ms | 52.3% bf16 MFU | 1627199 tok/s
step  266/19560 | loss 6.091773 (-1.48z)| norm 0.7208 (-0.91z)| lr 2.28e-04 | 321.96 ms | 52.4% bf16 MFU | 1627259 tok/s
step  267/19560 | loss 6.132460 (-1.23z)| norm 0.8663 (-0.19z)| lr 2.29e-04 | 321.92 ms | 52.4% bf16 MFU | 1627328 tok/s
step  268/19560 | loss 6.049953 (-1.69z)| norm 1.1055 (+0.97z)| lr 2.30e-04 | 322.40 ms | 52.3% bf16 MFU | 1627272 tok/s
step  269/19560 | loss 6.144701 (-1.12z)| norm 0.8932 (-0.06z)| lr 2.31e-04 | 321.92 ms | 52.4% bf16 MFU | 1627340 tok/s
step  270/19560 | loss 6.120766 (-1.24z)| norm 0.7558 (-0.72z)| lr 2.31e-04 | 321.71 ms | 52.5% bf16 MFU | 1627458 tok/s
step  271/19560 | loss 6.054239 (-1.62z)| norm 0.9668 (+0.31z)| lr 2.32e-04 | 322.40 ms | 52.3% bf16 MFU | 1627394 tok/s
step  272/19560 | loss 6.109072 (-1.28z)| norm 1.0310 (+0.62z)| lr 2.33e-04 | 321.41 ms | 52.5% bf16 MFU | 1627584 tok/s
step  273/19560 | loss 6.048799 (-1.62z)| norm 0.9505 (+0.21z)| lr 2.34e-04 | 322.86 ms | 52.3% bf16 MFU | 1627400 tok/s
step  274/19560 | loss 6.159984 (-0.94z)| norm 1.2466 (+1.67z)| lr 2.35e-04 | 322.22 ms | 52.4% bf16 MFU | 1627384 tok/s
step  275/19560 | loss 6.051655 (-1.59z)| norm 1.0267 (+0.58z)| lr 2.36e-04 | 322.84 ms | 52.3% bf16 MFU | 1627216 tok/s
step  276/19560 | loss 6.100168 (-1.27z)| norm 0.9614 (+0.26z)| lr 2.37e-04 | 322.05 ms | 52.4% bf16 MFU | 1627254 tok/s
step  277/19560 | loss 6.010573 (-1.81z)| norm 0.8595 (-0.25z)| lr 2.37e-04 | 321.57 ms | 52.5% bf16 MFU | 1627410 tok/s
step  278/19560 | loss 6.079008 (-1.37z)| norm 0.7606 (-0.74z)| lr 2.38e-04 | 322.03 ms | 52.4% bf16 MFU | 1627444 tok/s
step  279/19560 | loss 6.088156 (-1.30z)| norm 0.7010 (-1.03z)| lr 2.39e-04 | 322.63 ms | 52.3% bf16 MFU | 1627323 tok/s
step  280/19560 | loss 6.010752 (-1.76z)| norm 0.6840 (-1.10z)| lr 2.40e-04 | 322.44 ms | 52.3% bf16 MFU | 1627258 tok/s
step  281/19560 | loss 6.057764 (-1.45z)| norm 0.7041 (-0.98z)| lr 2.41e-04 | 322.77 ms | 52.3% bf16 MFU | 1627113 tok/s
step  282/19560 | loss 6.135691 (-0.93z)| norm 0.7332 (-0.82z)| lr 2.42e-04 | 321.71 ms | 52.5% bf16 MFU | 1627241 tok/s
step  283/19560 | loss 6.088119 (-1.23z)| norm 0.7690 (-0.62z)| lr 2.43e-04 | 321.92 ms | 52.4% bf16 MFU | 1627311 tok/s
step  284/19560 | loss 6.051684 (-1.47z)| norm 0.7286 (-0.82z)| lr 2.43e-04 | 322.81 ms | 52.3% bf16 MFU | 1627152 tok/s
step  285/19560 | loss 6.045553 (-1.49z)| norm 0.7824 (-0.55z)| lr 2.44e-04 | 322.45 ms | 52.3% bf16 MFU | 1627093 tok/s
step  286/19560 | loss 6.019584 (-1.64z)| norm 0.7575 (-0.68z)| lr 2.45e-04 | 322.46 ms | 52.3% bf16 MFU | 1627034 tok/s
step  287/19560 | loss 6.027895 (-1.57z)| norm 0.6522 (-1.20z)| lr 2.46e-04 | 322.36 ms | 52.4% bf16 MFU | 1627002 tok/s
step  288/19560 | loss 6.005795 (-1.70z)| norm 0.6869 (-1.03z)| lr 2.47e-04 | 322.12 ms | 52.4% bf16 MFU | 1627034 tok/s
step  289/19560 | loss 6.048473 (-1.39z)| norm 0.8421 (-0.24z)| lr 2.48e-04 | 322.61 ms | 52.3% bf16 MFU | 1626940 tok/s
step  290/19560 | loss 6.013555 (-1.61z)| norm 0.8975 (+0.03z)| lr 2.49e-04 | 322.72 ms | 52.3% bf16 MFU | 1626822 tok/s
step  291/19560 | loss 6.002988 (-1.66z)| norm 1.0161 (+0.63z)| lr 2.49e-04 | 323.01 ms | 52.2% bf16 MFU | 1626637 tok/s
step  292/19560 | loss 6.070134 (-1.18z)| norm 1.2183 (+1.64z)| lr 2.50e-04 | 322.58 ms | 52.3% bf16 MFU | 1626569 tok/s
step  293/19560 | loss 5.941104 (-2.04z)| norm 0.9722 (+0.37z)| lr 2.51e-04 | 322.26 ms | 52.4% bf16 MFU | 1626586 tok/s
step  294/19560 | loss 6.023878 (-1.44z)| norm 0.9237 (+0.11z)| lr 2.52e-04 | 322.48 ms | 52.3% bf16 MFU | 1626547 tok/s
step  295/19560 | loss 6.084309 (-1.01z)| norm 1.0126 (+0.57z)| lr 2.53e-04 | 322.92 ms | 52.3% bf16 MFU | 1626400 tok/s
step  296/19560 | loss 5.997696 (-1.59z)| norm 1.4799 (+2.99z)| lr 2.54e-04 | 322.07 ms | 52.4% bf16 MFU | 1626474 tok/s
step  297/19560 | loss 6.020942 (-1.41z)| norm 0.8512 (-0.28z)| lr 2.55e-04 | 322.16 ms | 52.4% bf16 MFU | 1626520 tok/s
step  298/19560 | loss 5.929161 (-2.02z)| norm 0.7992 (-0.54z)| lr 2.55e-04 | 323.61 ms | 52.2% bf16 MFU | 1626200 tok/s
step  299/19560 | loss 6.036056 (-1.25z)| norm 1.1161 (+1.23z)| lr 2.56e-04 | 321.86 ms | 52.4% bf16 MFU | 1626336 tok/s
step  300/19560 | loss 6.007178 (-1.44z)| norm 0.9531 (+0.31z)| lr 2.57e-04 | 322.77 ms | 52.3% bf16 MFU | 1626236 tok/s
step  301/19560 | loss 5.997185 (-1.50z)| norm 0.7145 (-1.04z)| lr 2.58e-04 | 322.96 ms | 52.3% bf16 MFU | 1626094 tok/s
step  302/19560 | loss 6.000340 (-1.45z)| norm 0.6340 (-1.49z)| lr 2.59e-04 | 322.14 ms | 52.4% bf16 MFU | 1626166 tok/s
step  303/19560 | loss 5.917272 (-2.01z)| norm 0.5613 (-1.90z)| lr 2.60e-04 | 322.68 ms | 52.3% bf16 MFU | 1626098 tok/s
step  304/19560 | loss 5.971196 (-1.60z)| norm 0.6630 (-1.28z)| lr 2.61e-04 | 322.64 ms | 52.3% bf16 MFU | 1626043 tok/s
step  305/19560 | loss 6.011260 (-1.30z)| norm 0.5642 (-1.82z)| lr 2.61e-04 | 322.08 ms | 52.4% bf16 MFU | 1626131 tok/s
step  306/19560 | loss 5.988555 (-1.44z)| norm 0.7029 (-1.01z)| lr 2.62e-04 | 322.52 ms | 52.3% bf16 MFU | 1626104 tok/s
step  307/19560 | loss 5.982687 (-1.47z)| norm 1.0402 (+0.92z)| lr 2.63e-04 | 323.67 ms | 52.1% bf16 MFU | 1625789 tok/s
step  308/19560 | loss 6.012085 (-1.24z)| norm 1.2939 (+2.31z)| lr 2.64e-04 | 322.17 ms | 52.4% bf16 MFU | 1625868 tok/s
step  309/19560 | loss 5.988081 (-1.40z)| norm 1.0314 (+0.83z)| lr 2.65e-04 | 322.46 ms | 52.3% bf16 MFU | 1625869 tok/s
step  310/19560 | loss 6.205850 (+0.24z)| norm 1.4755 (+3.21z)| lr 2.66e-04 | 322.96 ms | 52.3% bf16 MFU | 1625745 tok/s
step  311/19560 | loss 5.974777 (-1.50z)| norm 0.9991 (+0.62z)| lr 2.67e-04 | 322.30 ms | 52.4% bf16 MFU | 1625794 tok/s
step  312/19560 | loss 6.027915 (-1.08z)| norm 1.4537 (+2.99z)| lr 2.67e-04 | 322.09 ms | 52.4% bf16 MFU | 1625892 tok/s
step  313/19560 | loss 6.027864 (-1.07z)| norm 1.0915 (+1.06z)| lr 2.68e-04 | 322.48 ms | 52.3% bf16 MFU | 1625886 tok/s
step  314/19560 | loss 6.054132 (-0.85z)| norm 1.5860 (+3.48z)| lr 2.69e-04 | 322.28 ms | 52.4% bf16 MFU | 1625933 tok/s
step  315/19560 | loss 6.008661 (-1.20z)| norm 1.1098 (+1.07z)| lr 2.70e-04 | 322.37 ms | 52.4% bf16 MFU | 1625954 tok/s
step  316/19560 | loss 5.953825 (-1.61z)| norm 1.2184 (+1.60z)| lr 2.71e-04 | 322.21 ms | 52.4% bf16 MFU | 1626015 tok/s
step  317/19560 | loss 5.939453 (-1.70z)| norm 0.9028 (+0.04z)| lr 2.72e-04 | 321.88 ms | 52.4% bf16 MFU | 1626156 tok/s
step  318/19560 | loss 5.965785 (-1.47z)| norm 0.8877 (-0.04z)| lr 2.73e-04 | 321.81 ms | 52.4% bf16 MFU | 1626307 tok/s
step  319/19560 | loss 5.965137 (-1.45z)| norm 0.6572 (-1.17z)| lr 2.73e-04 | 321.67 ms | 52.5% bf16 MFU | 1626487 tok/s
step  320/19560 | loss 5.891699 (-2.00z)| norm 0.6046 (-1.41z)| lr 2.74e-04 | 321.98 ms | 52.4% bf16 MFU | 1626578 tok/s
step  321/19560 | loss 5.950617 (-1.51z)| norm 0.6766 (-1.05z)| lr 2.75e-04 | 322.31 ms | 52.4% bf16 MFU | 1626581 tok/s
step  322/19560 | loss 5.905738 (-1.83z)| norm 0.8218 (-0.33z)| lr 2.76e-04 | 321.98 ms | 52.4% bf16 MFU | 1626669 tok/s
step  323/19560 | loss 5.877674 (-2.01z)| norm 1.0206 (+0.64z)| lr 2.77e-04 | 322.89 ms | 52.3% bf16 MFU | 1626522 tok/s
step  324/19560 | loss 5.943759 (-1.47z)| norm 1.2408 (+1.68z)| lr 2.78e-04 | 322.46 ms | 52.3% bf16 MFU | 1626492 tok/s
step  325/19560 | loss 5.825142 (-2.35z)| norm 1.0034 (+0.53z)| lr 2.79e-04 | 322.08 ms | 52.4% bf16 MFU | 1626558 tok/s
step  326/19560 | loss 5.931548 (-1.50z)| norm 1.1801 (+1.36z)| lr 2.79e-04 | 322.97 ms | 52.3% bf16 MFU | 1626397 tok/s
step  327/19560 | loss 5.898881 (-1.72z)| norm 0.9803 (+0.39z)| lr 2.80e-04 | 322.37 ms | 52.4% bf16 MFU | 1626394 tok/s
step  328/19560 | loss 5.984981 (-1.03z)| norm 0.9396 (+0.20z)| lr 2.81e-04 | 323.40 ms | 52.2% bf16 MFU | 1626133 tok/s
step  329/19560 | loss 5.902948 (-1.66z)| norm 1.0167 (+0.59z)| lr 2.82e-04 | 322.47 ms | 52.3% bf16 MFU | 1626119 tok/s
step  330/19560 | loss 5.909916 (-1.58z)| norm 0.7868 (-0.53z)| lr 2.83e-04 | 322.83 ms | 52.3% bf16 MFU | 1626015 tok/s
step  331/19560 | loss 5.963213 (-1.15z)| norm 0.8201 (-0.37z)| lr 2.84e-04 | 322.59 ms | 52.3% bf16 MFU | 1625976 tok/s
step  332/19560 | loss 5.933629 (-1.37z)| norm 0.7109 (-0.89z)| lr 2.85e-04 | 321.78 ms | 52.4% bf16 MFU | 1626144 tok/s
step  333/19560 | loss 5.905851 (-1.58z)| norm 0.7739 (-0.57z)| lr 2.85e-04 | 321.82 ms | 52.4% bf16 MFU | 1626294 tok/s
step  334/19560 | loss 5.841239 (-2.07z)| norm 0.7540 (-0.66z)| lr 2.86e-04 | 321.70 ms | 52.5% bf16 MFU | 1626466 tok/s
step  335/19560 | loss 5.874666 (-1.76z)| norm 0.7758 (-0.55z)| lr 2.87e-04 | 321.89 ms | 52.4% bf16 MFU | 1626582 tok/s
step  336/19560 | loss 5.880956 (-1.69z)| norm 0.9425 (+0.25z)| lr 2.88e-04 | 322.07 ms | 52.4% bf16 MFU | 1626646 tok/s
step  337/19560 | loss 5.901791 (-1.50z)| norm 1.2648 (+1.78z)| lr 2.89e-04 | 322.73 ms | 52.3% bf16 MFU | 1626542 tok/s
step  338/19560 | loss 5.881930 (-1.64z)| norm 0.9167 (+0.10z)| lr 2.90e-04 | 322.92 ms | 52.3% bf16 MFU | 1626395 tok/s
step  339/19560 | loss 5.938679 (-1.16z)| norm 0.8419 (-0.26z)| lr 2.91e-04 | 322.04 ms | 52.4% bf16 MFU | 1626477 tok/s
step  340/19560 | loss 5.840579 (-1.94z)| norm 0.9368 (+0.20z)| lr 2.91e-04 | 322.22 ms | 52.4% bf16 MFU | 1626508 tok/s
step  341/19560 | loss 5.866185 (-1.70z)| norm 1.2356 (+1.61z)| lr 2.92e-04 | 322.86 ms | 52.3% bf16 MFU | 1626378 tok/s
step  342/19560 | loss 5.862097 (-1.72z)| norm 0.7736 (-0.58z)| lr 2.93e-04 | 322.58 ms | 52.3% bf16 MFU | 1626324 tok/s
step  343/19560 | loss 5.884152 (-1.52z)| norm 0.6846 (-1.00z)| lr 2.94e-04 | 322.41 ms | 52.3% bf16 MFU | 1626315 tok/s
step  344/19560 | loss 5.827714 (-1.96z)| norm 0.9660 (+0.34z)| lr 2.95e-04 | 322.72 ms | 52.3% bf16 MFU | 1626229 tok/s
step  345/19560 | loss 5.905993 (-1.29z)| norm 1.0111 (+0.55z)| lr 2.96e-04 | 322.11 ms | 52.4% bf16 MFU | 1626300 tok/s
step  346/19560 | loss 5.890816 (-1.41z)| norm 0.7848 (-0.52z)| lr 2.97e-04 | 322.47 ms | 52.3% bf16 MFU | 1626277 tok/s
step  347/19560 | loss 5.849085 (-1.74z)| norm 0.7704 (-0.58z)| lr 2.97e-04 | 323.83 ms | 52.1% bf16 MFU | 1625914 tok/s
step  348/19560 | loss 5.842230 (-1.76z)| norm 1.0372 (+0.68z)| lr 2.98e-04 | 322.54 ms | 52.3% bf16 MFU | 1625893 tok/s
step  349/19560 | loss 5.839679 (-1.76z)| norm 1.0092 (+0.54z)| lr 2.99e-04 | 323.64 ms | 52.1% bf16 MFU | 1625596 tok/s
step  350/19560 | loss 5.883446 (-1.37z)| norm 0.9541 (+0.27z)| lr 3.00e-04 | 322.59 ms | 52.3% bf16 MFU | 1625579 tok/s
step  351/19560 | loss 5.845253 (-1.68z)| norm 0.7711 (-0.60z)| lr 3.01e-04 | 322.29 ms | 52.4% bf16 MFU | 1625637 tok/s
step  352/19560 | loss 5.859211 (-1.54z)| norm 0.8240 (-0.35z)| lr 3.02e-04 | 322.74 ms | 52.3% bf16 MFU | 1625578 tok/s
step  353/19560 | loss 5.884883 (-1.31z)| norm 0.7642 (-0.62z)| lr 3.03e-04 | 322.33 ms | 52.4% bf16 MFU | 1625628 tok/s
step  354/19560 | loss 5.743085 (-2.48z)| norm 0.6273 (-1.26z)| lr 3.03e-04 | 322.08 ms | 52.4% bf16 MFU | 1625737 tok/s
step  355/19560 | loss 5.807248 (-1.89z)| norm 0.7283 (-0.77z)| lr 3.04e-04 | 322.54 ms | 52.3% bf16 MFU | 1625724 tok/s
step  356/19560 | loss 5.727894 (-2.50z)| norm 0.7883 (-0.50z)| lr 3.05e-04 | 322.95 ms | 52.3% bf16 MFU | 1625609 tok/s
step  357/19560 | loss 5.849351 (-1.45z)| norm 0.9012 (+0.03z)| lr 3.06e-04 | 323.01 ms | 52.2% bf16 MFU | 1625486 tok/s
step  358/19560 | loss 5.873186 (-1.23z)| norm 1.7094 (+3.65z)| lr 3.07e-04 | 322.21 ms | 52.4% bf16 MFU | 1625569 tok/s
step  359/19560 | loss 5.796082 (-1.85z)| norm 1.0025 (+0.46z)| lr 3.08e-04 | 322.69 ms | 52.3% bf16 MFU | 1625528 tok/s
step  360/19560 | loss 5.839208 (-1.47z)| norm 0.9790 (+0.36z)| lr 3.09e-04 | 322.26 ms | 52.4% bf16 MFU | 1625596 tok/s
step  361/19560 | loss 5.858850 (-1.28z)| norm 0.8327 (-0.30z)| lr 3.09e-04 | 322.58 ms | 52.3% bf16 MFU | 1625580 tok/s
step  362/19560 | loss 5.808546 (-1.68z)| norm 0.7634 (-0.62z)| lr 3.10e-04 | 322.28 ms | 52.4% bf16 MFU | 1625641 tok/s
step  363/19560 | loss 5.806164 (-1.67z)| norm 0.9428 (+0.20z)| lr 3.11e-04 | 322.76 ms | 52.3% bf16 MFU | 1625578 tok/s
step  364/19560 | loss 5.817901 (-1.55z)| norm 1.0145 (+0.53z)| lr 3.12e-04 | 322.21 ms | 52.4% bf16 MFU | 1625656 tok/s
step  365/19560 | loss 5.770277 (-1.92z)| norm 0.9715 (+0.32z)| lr 3.13e-04 | 322.30 ms | 52.4% bf16 MFU | 1625710 tok/s
step  366/19560 | loss 5.792306 (-1.71z)| norm 1.0671 (+0.76z)| lr 3.14e-04 | 322.15 ms | 52.4% bf16 MFU | 1625799 tok/s
step  367/19560 | loss 5.720038 (-2.26z)| norm 0.9619 (+0.28z)| lr 3.15e-04 | 322.75 ms | 52.3% bf16 MFU | 1625730 tok/s
step  368/19560 | loss 5.718963 (-2.21z)| norm 0.9665 (+0.31z)| lr 3.15e-04 | 323.02 ms | 52.2% bf16 MFU | 1625598 tok/s
step  369/19560 | loss 5.764118 (-1.81z)| norm 1.0256 (+0.57z)| lr 3.16e-04 | 322.17 ms | 52.4% bf16 MFU | 1625685 tok/s
step  370/19560 | loss 5.827168 (-1.27z)| norm 1.1665 (+1.21z)| lr 3.17e-04 | 322.54 ms | 52.3% bf16 MFU | 1625677 tok/s
step  371/19560 | loss 5.822650 (-1.28z)| norm 0.8900 (-0.05z)| lr 3.18e-04 | 321.91 ms | 52.4% bf16 MFU | 1625828 tok/s
step  372/19560 | loss 5.750629 (-1.84z)| norm 0.8913 (-0.05z)| lr 3.19e-04 | 322.61 ms | 52.3% bf16 MFU | 1625793 tok/s
step  373/19560 | loss 5.785405 (-1.53z)| norm 0.6851 (-0.99z)| lr 3.20e-04 | 322.91 ms | 52.3% bf16 MFU | 1625686 tok/s
step  374/19560 | loss 5.819055 (-1.24z)| norm 0.8535 (-0.23z)| lr 3.21e-04 | 322.57 ms | 52.3% bf16 MFU | 1625670 tok/s
step  375/19560 | loss 5.713826 (-2.07z)| norm 0.8625 (-0.20z)| lr 3.21e-04 | 322.31 ms | 52.4% bf16 MFU | 1625718 tok/s
step  376/19560 | loss 5.776162 (-1.54z)| norm 0.8518 (-0.26z)| lr 3.22e-04 | 322.94 ms | 52.3% bf16 MFU | 1625607 tok/s
step  377/19560 | loss 5.761419 (-1.62z)| norm 0.9883 (+0.37z)| lr 3.23e-04 | 323.24 ms | 52.2% bf16 MFU | 1625427 tok/s
step  378/19560 | loss 5.783203 (-1.43z)| norm 1.2551 (+1.59z)| lr 3.24e-04 | 322.50 ms | 52.3% bf16 MFU | 1625439 tok/s
step  379/19560 | loss 5.756898 (-1.62z)| norm 0.8764 (-0.15z)| lr 3.25e-04 | 322.35 ms | 52.4% bf16 MFU | 1625491 tok/s
step  380/19560 | loss 5.770188 (-1.49z)| norm 0.9272 (+0.08z)| lr 3.26e-04 | 322.32 ms | 52.4% bf16 MFU | 1625547 tok/s
step  381/19560 | loss 5.745415 (-1.67z)| norm 0.8312 (-0.36z)| lr 3.27e-04 | 323.07 ms | 52.2% bf16 MFU | 1625411 tok/s
step  382/19560 | loss 5.750505 (-1.60z)| norm 0.8161 (-0.43z)| lr 3.27e-04 | 322.34 ms | 52.4% bf16 MFU | 1625464 tok/s
step  383/19560 | loss 5.695450 (-2.01z)| norm 0.9185 (+0.05z)| lr 3.28e-04 | 323.01 ms | 52.2% bf16 MFU | 1625348 tok/s
step  384/19560 | loss 5.786231 (-1.26z)| norm 0.8738 (-0.16z)| lr 3.29e-04 | 323.15 ms | 52.2% bf16 MFU | 1625201 tok/s
step  385/19560 | loss 5.757025 (-1.48z)| norm 1.0157 (+0.52z)| lr 3.30e-04 | 321.90 ms | 52.4% bf16 MFU | 1625378 tok/s
step  386/19560 | loss 5.692902 (-1.96z)| norm 0.7604 (-0.69z)| lr 3.31e-04 | 322.68 ms | 52.3% bf16 MFU | 1625348 tok/s
step  387/19560 | loss 5.694768 (-1.91z)| norm 0.7658 (-0.67z)| lr 3.32e-04 | 322.84 ms | 52.3% bf16 MFU | 1625281 tok/s
step  388/19560 | loss 5.681092 (-1.98z)| norm 0.6569 (-1.20z)| lr 3.33e-04 | 322.73 ms | 52.3% bf16 MFU | 1625244 tok/s
step  389/19560 | loss 5.742258 (-1.47z)| norm 0.6893 (-1.05z)| lr 3.33e-04 | 322.25 ms | 52.4% bf16 MFU | 1625329 tok/s
step  390/19560 | loss 5.695709 (-1.81z)| norm 0.7322 (-0.85z)| lr 3.34e-04 | 322.07 ms | 52.4% bf16 MFU | 1625457 tok/s
step  391/19560 | loss 5.731614 (-1.49z)| norm 1.2843 (+1.77z)| lr 3.35e-04 | 322.82 ms | 52.3% bf16 MFU | 1625389 tok/s
step  392/19560 | loss 5.736551 (-1.43z)| norm 1.4277 (+2.39z)| lr 3.36e-04 | 322.86 ms | 52.3% bf16 MFU | 1625313 tok/s
step  393/19560 | loss 5.755140 (-1.26z)| norm 0.9903 (+0.33z)| lr 3.37e-04 | 322.46 ms | 52.3% bf16 MFU | 1625343 tok/s
step  394/19560 | loss 5.735332 (-1.40z)| norm 1.0733 (+0.70z)| lr 3.38e-04 | 322.47 ms | 52.3% bf16 MFU | 1625367 tok/s
step  395/19560 | loss 5.721637 (-1.49z)| norm 1.1927 (+1.25z)| lr 3.39e-04 | 322.45 ms | 52.3% bf16 MFU | 1625397 tok/s
step  396/19560 | loss 5.736702 (-1.34z)| norm 0.9543 (+0.14z)| lr 3.39e-04 | 322.63 ms | 52.3% bf16 MFU | 1625380 tok/s
step  397/19560 | loss 5.687119 (-1.72z)| norm 0.9149 (-0.05z)| lr 3.40e-04 | 321.90 ms | 52.4% bf16 MFU | 1625548 tok/s
step  398/19560 | loss 5.716406 (-1.46z)| norm 1.0600 (+0.62z)| lr 3.41e-04 | 322.98 ms | 52.3% bf16 MFU | 1625435 tok/s
step  399/19560 | loss 5.719507 (-1.41z)| norm 1.0972 (+0.79z)| lr 3.42e-04 | 322.98 ms | 52.3% bf16 MFU | 1625327 tok/s
step  400/19560 | loss 5.766946 (-1.02z)| norm 1.1407 (+0.99z)| lr 3.43e-04 | 322.73 ms | 52.3% bf16 MFU | 1625288 tok/s
step  401/19560 | loss 5.669549 (-1.78z)| norm 1.1828 (+1.17z)| lr 3.44e-04 | 322.61 ms | 52.3% bf16 MFU | 1625280 tok/s
step  402/19560 | loss 5.742557 (-1.18z)| norm 1.4134 (+2.21z)| lr 3.45e-04 | 322.74 ms | 52.3% bf16 MFU | 1625240 tok/s
step  403/19560 | loss 5.729059 (-1.27z)| norm 1.1337 (+0.92z)| lr 3.45e-04 | 322.97 ms | 52.3% bf16 MFU | 1625145 tok/s
step  404/19560 | loss 5.622725 (-2.10z)| norm 0.9593 (+0.12z)| lr 3.46e-04 | 322.44 ms | 52.3% bf16 MFU | 1625188 tok/s
step  405/19560 | loss 5.725780 (-1.24z)| norm 0.9325 (-0.01z)| lr 3.47e-04 | 322.86 ms | 52.3% bf16 MFU | 1625122 tok/s
step  406/19560 | loss 5.670414 (-1.66z)| norm 1.0246 (+0.41z)| lr 3.48e-04 | 323.08 ms | 52.2% bf16 MFU | 1625004 tok/s
step  407/19560 | loss 5.655542 (-1.75z)| norm 1.0075 (+0.32z)| lr 3.49e-04 | 321.73 ms | 52.5% bf16 MFU | 1625233 tok/s
step  408/19560 | loss 5.720356 (-1.21z)| norm 0.7679 (-0.79z)| lr 3.50e-04 | 322.78 ms | 52.3% bf16 MFU | 1625187 tok/s
step  409/19560 | loss 5.689933 (-1.43z)| norm 0.8612 (-0.37z)| lr 3.51e-04 | 322.82 ms | 52.3% bf16 MFU | 1625133 tok/s
step  410/19560 | loss 5.647253 (-1.76z)| norm 0.8234 (-0.55z)| lr 3.51e-04 | 321.96 ms | 52.4% bf16 MFU | 1625298 tok/s
step  411/19560 | loss 5.608929 (-2.04z)| norm 0.8368 (-0.49z)| lr 3.52e-04 | 322.87 ms | 52.3% bf16 MFU | 1625225 tok/s
step  412/19560 | loss 5.703447 (-1.25z)| norm 0.6916 (-1.17z)| lr 3.53e-04 | 322.43 ms | 52.3% bf16 MFU | 1625266 tok/s
step  413/19560 | loss 5.643915 (-1.71z)| norm 0.7164 (-1.05z)| lr 3.54e-04 | 322.49 ms | 52.3% bf16 MFU | 1625290 tok/s
step  414/19560 | loss 5.622111 (-1.85z)| norm 1.1186 (+0.81z)| lr 3.55e-04 | 322.58 ms | 52.3% bf16 MFU | 1625289 tok/s
step  415/19560 | loss 5.636696 (-1.70z)| norm 1.1089 (+0.76z)| lr 3.56e-04 | 322.34 ms | 52.4% bf16 MFU | 1625351 tok/s
step  416/19560 | loss 5.644901 (-1.61z)| norm 0.7677 (-0.85z)| lr 3.57e-04 | 322.44 ms | 52.3% bf16 MFU | 1625384 tok/s
step  417/19560 | loss 5.676686 (-1.33z)| norm 0.8204 (-0.60z)| lr 3.57e-04 | 322.27 ms | 52.4% bf16 MFU | 1625458 tok/s
step  418/19560 | loss 5.672420 (-1.35z)| norm 0.6397 (-1.43z)| lr 3.58e-04 | 322.33 ms | 52.4% bf16 MFU | 1625512 tok/s
step  419/19560 | loss 5.638844 (-1.59z)| norm 0.7051 (-1.10z)| lr 3.59e-04 | 322.61 ms | 52.3% bf16 MFU | 1625494 tok/s
step  420/19560 | loss 5.629757 (-1.64z)| norm 0.6911 (-1.15z)| lr 3.60e-04 | 322.47 ms | 52.3% bf16 MFU | 1625510 tok/s
step  421/19560 | loss 5.674884 (-1.25z)| norm 0.6832 (-1.17z)| lr 3.61e-04 | 322.22 ms | 52.4% bf16 MFU | 1625591 tok/s
step  422/19560 | loss 5.586718 (-1.93z)| norm 0.9527 (+0.07z)| lr 3.62e-04 | 322.21 ms | 52.4% bf16 MFU | 1625668 tok/s
step  423/19560 | loss 5.597835 (-1.82z)| norm 1.0148 (+0.36z)| lr 3.63e-04 | 322.80 ms | 52.3% bf16 MFU | 1625595 tok/s
step  424/19560 | loss 5.625693 (-1.57z)| norm 0.8374 (-0.45z)| lr 3.63e-04 | 322.62 ms | 52.3% bf16 MFU | 1625570 tok/s
step  425/19560 | loss 5.593684 (-1.80z)| norm 0.8765 (-0.26z)| lr 3.64e-04 | 322.23 ms | 52.4% bf16 MFU | 1625644 tok/s
step  426/19560 | loss 5.619625 (-1.56z)| norm 0.7717 (-0.76z)| lr 3.65e-04 | 322.62 ms | 52.3% bf16 MFU | 1625616 tok/s
step  427/19560 | loss 5.646550 (-1.32z)| norm 1.1069 (+0.83z)| lr 3.66e-04 | 322.74 ms | 52.3% bf16 MFU | 1625560 tok/s
step  428/19560 | loss 5.589303 (-1.76z)| norm 1.1361 (+0.96z)| lr 3.67e-04 | 322.43 ms | 52.3% bf16 MFU | 1625584 tok/s
step  429/19560 | loss 5.599124 (-1.65z)| norm 0.9718 (+0.17z)| lr 3.68e-04 | 322.70 ms | 52.3% bf16 MFU | 1625538 tok/s
step  430/19560 | loss 5.584253 (-1.74z)| norm 0.9013 (-0.17z)| lr 3.69e-04 | 322.45 ms | 52.3% bf16 MFU | 1625559 tok/s
step  431/19560 | loss 5.599386 (-1.59z)| norm 1.0663 (+0.60z)| lr 3.69e-04 | 323.12 ms | 52.2% bf16 MFU | 1625410 tok/s
step  432/19560 | loss 5.568501 (-1.80z)| norm 0.9260 (-0.09z)| lr 3.70e-04 | 322.22 ms | 52.4% bf16 MFU | 1625496 tok/s
step  433/19560 | loss 5.555615 (-1.87z)| norm 0.8499 (-0.47z)| lr 3.71e-04 | 322.58 ms | 52.3% bf16 MFU | 1625488 tok/s
step  434/19560 | loss 5.597670 (-1.51z)| norm 0.8140 (-0.66z)| lr 3.72e-04 | 322.73 ms | 52.3% bf16 MFU | 1625441 tok/s
step  435/19560 | loss 5.579070 (-1.63z)| norm 0.6902 (-1.25z)| lr 3.73e-04 | 322.76 ms | 52.3% bf16 MFU | 1625388 tok/s
step  436/19560 | loss 5.590956 (-1.52z)| norm 0.7445 (-0.97z)| lr 3.74e-04 | 322.03 ms | 52.4% bf16 MFU | 1625522 tok/s
step  437/19560 | loss 5.501616 (-2.18z)| norm 0.9231 (-0.08z)| lr 3.75e-04 | 322.84 ms | 52.3% bf16 MFU | 1625446 tok/s
step  438/19560 | loss 5.645470 (-1.06z)| norm 1.1164 (+0.92z)| lr 3.75e-04 | 323.03 ms | 52.2% bf16 MFU | 1625326 tok/s
step  439/19560 | loss 5.533571 (-1.94z)| norm 0.9254 (-0.05z)| lr 3.76e-04 | 322.22 ms | 52.4% bf16 MFU | 1625415 tok/s
step  440/19560 | loss 5.561307 (-1.69z)| norm 0.8409 (-0.47z)| lr 3.77e-04 | 322.38 ms | 52.4% bf16 MFU | 1625461 tok/s
step  441/19560 | loss 5.518496 (-2.02z)| norm 0.6693 (-1.34z)| lr 3.78e-04 | 322.44 ms | 52.3% bf16 MFU | 1625488 tok/s
step  442/19560 | loss 5.538939 (-1.83z)| norm 0.7303 (-1.04z)| lr 3.79e-04 | 322.32 ms | 52.4% bf16 MFU | 1625544 tok/s
step  443/19560 | loss 5.580063 (-1.48z)| norm 0.7631 (-0.84z)| lr 3.80e-04 | 323.51 ms | 52.2% bf16 MFU | 1625299 tok/s
step  444/19560 | loss 5.485353 (-2.22z)| norm 0.7597 (-0.85z)| lr 3.81e-04 | 322.33 ms | 52.4% bf16 MFU | 1625362 tok/s
step  445/19560 | loss 5.500202 (-2.06z)| norm 0.6991 (-1.17z)| lr 3.81e-04 | 323.23 ms | 52.2% bf16 MFU | 1625196 tok/s
step  446/19560 | loss 5.536902 (-1.73z)| norm 0.7388 (-0.94z)| lr 3.82e-04 | 323.12 ms | 52.2% bf16 MFU | 1625065 tok/s
step  447/19560 | loss 5.518736 (-1.85z)| norm 0.9274 (+0.07z)| lr 3.83e-04 | 322.64 ms | 52.3% bf16 MFU | 1625061 tok/s
step  448/19560 | loss 5.525872 (-1.75z)| norm 0.8873 (-0.16z)| lr 3.84e-04 | 322.63 ms | 52.3% bf16 MFU | 1625059 tok/s
step  449/19560 | loss 5.598598 (-1.14z)| norm 0.8607 (-0.32z)| lr 3.85e-04 | 322.49 ms | 52.3% bf16 MFU | 1625093 tok/s
step  450/19560 | loss 5.555230 (-1.47z)| norm 0.9887 (+0.39z)| lr 3.86e-04 | 322.27 ms | 52.4% bf16 MFU | 1625182 tok/s
step  451/19560 | loss 5.584246 (-1.22z)| norm 0.8414 (-0.42z)| lr 3.87e-04 | 322.76 ms | 52.3% bf16 MFU | 1625143 tok/s
step  452/19560 | loss 5.559985 (-1.40z)| norm 0.7951 (-0.67z)| lr 3.87e-04 | 321.82 ms | 52.4% bf16 MFU | 1625342 tok/s
step  453/19560 | loss 5.543053 (-1.51z)| norm 0.9667 (+0.30z)| lr 3.88e-04 | 322.43 ms | 52.3% bf16 MFU | 1625376 tok/s
step  454/19560 | loss 5.481050 (-1.99z)| norm 1.0480 (+0.77z)| lr 3.89e-04 | 322.66 ms | 52.3% bf16 MFU | 1625352 tok/s
step  455/19560 | loss 5.553469 (-1.37z)| norm 1.1745 (+1.47z)| lr 3.90e-04 | 322.75 ms | 52.3% bf16 MFU | 1625305 tok/s
step  456/19560 | loss 5.536829 (-1.49z)| norm 1.0369 (+0.69z)| lr 3.91e-04 | 322.56 ms | 52.3% bf16 MFU | 1625310 tok/s
step  457/19560 | loss 5.584871 (-1.08z)| norm 1.0562 (+0.80z)| lr 3.92e-04 | 322.84 ms | 52.3% bf16 MFU | 1625244 tok/s
step  458/19560 | loss 5.514379 (-1.64z)| norm 0.8486 (-0.38z)| lr 3.93e-04 | 322.47 ms | 52.3% bf16 MFU | 1625274 tok/s
step  459/19560 | loss 5.549839 (-1.33z)| norm 0.8395 (-0.43z)| lr 3.93e-04 | 322.33 ms | 52.4% bf16 MFU | 1625339 tok/s
step  460/19560 | loss 5.530938 (-1.47z)| norm 0.7881 (-0.73z)| lr 3.94e-04 | 322.82 ms | 52.3% bf16 MFU | 1625278 tok/s
step  461/19560 | loss 5.496747 (-1.74z)| norm 0.6855 (-1.30z)| lr 3.95e-04 | 322.35 ms | 52.4% bf16 MFU | 1625335 tok/s
step  462/19560 | loss 5.499086 (-1.69z)| norm 0.7505 (-0.93z)| lr 3.96e-04 | 322.86 ms | 52.3% bf16 MFU | 1625263 tok/s
step  463/19560 | loss 5.628045 (-0.58z)| norm 0.8172 (-0.56z)| lr 3.97e-04 | 322.62 ms | 52.3% bf16 MFU | 1625254 tok/s
step  464/19560 | loss 5.545953 (-1.26z)| norm 1.1557 (+1.34z)| lr 3.98e-04 | 322.56 ms | 52.3% bf16 MFU | 1625262 tok/s
step  465/19560 | loss 5.453753 (-2.02z)| norm 1.0178 (+0.58z)| lr 3.99e-04 | 322.56 ms | 52.3% bf16 MFU | 1625269 tok/s
step  466/19560 | loss 5.455788 (-1.96z)| norm 1.0387 (+0.69z)| lr 3.99e-04 | 323.14 ms | 52.2% bf16 MFU | 1625128 tok/s
step  467/19560 | loss 5.472921 (-1.79z)| norm 0.9023 (-0.08z)| lr 4.00e-04 | 322.52 ms | 52.3% bf16 MFU | 1625152 tok/s
step  468/19560 | loss 5.470323 (-1.78z)| norm 0.7840 (-0.75z)| lr 4.01e-04 | 322.77 ms | 52.3% bf16 MFU | 1625110 tok/s
step  469/19560 | loss 5.455796 (-1.87z)| norm 0.8661 (-0.27z)| lr 4.02e-04 | 322.33 ms | 52.4% bf16 MFU | 1625182 tok/s
step  470/19560 | loss 5.540091 (-1.14z)| norm 1.0270 (+0.64z)| lr 4.03e-04 | 323.31 ms | 52.2% bf16 MFU | 1625003 tok/s
step  471/19560 | loss 5.507588 (-1.40z)| norm 1.2436 (+1.85z)| lr 4.04e-04 | 322.07 ms | 52.4% bf16 MFU | 1625148 tok/s
step  472/19560 | loss 5.447962 (-1.87z)| norm 0.9871 (+0.39z)| lr 4.05e-04 | 322.29 ms | 52.4% bf16 MFU | 1625228 tok/s
step  473/19560 | loss 5.456339 (-1.78z)| norm 0.7876 (-0.74z)| lr 4.05e-04 | 322.71 ms | 52.3% bf16 MFU | 1625199 tok/s
step  474/19560 | loss 5.448184 (-1.82z)| norm 0.9719 (+0.30z)| lr 4.06e-04 | 323.03 ms | 52.2% bf16 MFU | 1625090 tok/s
step  475/19560 | loss 5.447782 (-1.79z)| norm 0.9177 (-0.02z)| lr 4.07e-04 | 321.99 ms | 52.4% bf16 MFU | 1625250 tok/s
step  476/19560 | loss 5.435426 (-1.86z)| norm 0.6918 (-1.29z)| lr 4.08e-04 | 322.50 ms | 52.3% bf16 MFU | 1625272 tok/s
step  477/19560 | loss 5.514537 (-1.18z)| norm 0.8658 (-0.29z)| lr 4.09e-04 | 322.55 ms | 52.3% bf16 MFU | 1625281 tok/s
step  478/19560 | loss 5.498700 (-1.30z)| norm 0.8963 (-0.11z)| lr 4.10e-04 | 322.41 ms | 52.3% bf16 MFU | 1625326 tok/s
step  479/19560 | loss 5.435882 (-1.80z)| norm 0.9105 (-0.04z)| lr 4.11e-04 | 322.26 ms | 52.4% bf16 MFU | 1625404 tok/s
step  480/19560 | loss 5.426637 (-1.85z)| norm 1.2213 (+1.70z)| lr 4.11e-04 | 322.19 ms | 52.4% bf16 MFU | 1625497 tok/s
step  481/19560 | loss 5.496995 (-1.24z)| norm 1.1932 (+1.52z)| lr 4.12e-04 | 322.48 ms | 52.3% bf16 MFU | 1625512 tok/s
step  482/19560 | loss 5.484913 (-1.32z)| norm 0.9583 (+0.18z)| lr 4.13e-04 | 322.77 ms | 52.3% bf16 MFU | 1625454 tok/s
step  483/19560 | loss 5.466426 (-1.46z)| norm 0.8764 (-0.29z)| lr 4.14e-04 | 322.33 ms | 52.4% bf16 MFU | 1625508 tok/s
step  484/19560 | loss 5.423511 (-1.79z)| norm 0.8038 (-0.71z)| lr 4.15e-04 | 322.28 ms | 52.4% bf16 MFU | 1625573 tok/s
step  485/19560 | loss 5.438282 (-1.64z)| norm 0.7041 (-1.26z)| lr 4.16e-04 | 322.35 ms | 52.4% bf16 MFU | 1625617 tok/s
step  486/19560 | loss 5.460457 (-1.43z)| norm 0.5657 (-2.13z)| lr 4.17e-04 | 322.80 ms | 52.3% bf16 MFU | 1625544 tok/s
step  487/19560 | loss 5.409263 (-1.84z)| norm 0.6027 (-1.87z)| lr 4.17e-04 | 322.38 ms | 52.4% bf16 MFU | 1625583 tok/s
step  488/19560 | loss 5.488084 (-1.15z)| norm 0.6170 (-1.74z)| lr 4.18e-04 | 322.50 ms | 52.3% bf16 MFU | 1625588 tok/s
step  489/19560 | loss 5.378205 (-2.07z)| norm 0.8146 (-0.57z)| lr 4.19e-04 | 323.13 ms | 52.2% bf16 MFU | 1625435 tok/s
step  490/19560 | loss 5.468954 (-1.27z)| norm 1.0960 (+1.08z)| lr 4.20e-04 | 322.18 ms | 52.4% bf16 MFU | 1625530 tok/s
step  491/19560 | loss 5.427242 (-1.60z)| norm 0.9624 (+0.29z)| lr 4.21e-04 | 322.48 ms | 52.3% bf16 MFU | 1625542 tok/s
step  492/19560 | loss 5.411309 (-1.72z)| norm 0.8421 (-0.42z)| lr 4.22e-04 | 322.86 ms | 52.3% bf16 MFU | 1625459 tok/s
step  493/19560 | loss 5.391879 (-1.85z)| norm 0.6382 (-1.59z)| lr 4.23e-04 | 322.17 ms | 52.4% bf16 MFU | 1625553 tok/s
step  494/19560 | loss 5.397700 (-1.77z)| norm 0.7292 (-1.04z)| lr 4.23e-04 | 322.54 ms | 52.3% bf16 MFU | 1625551 tok/s
step  495/19560 | loss 5.402930 (-1.69z)| norm 0.9433 (+0.21z)| lr 4.24e-04 | 322.73 ms | 52.3% bf16 MFU | 1625502 tok/s
step  496/19560 | loss 5.391466 (-1.75z)| norm 0.8844 (-0.13z)| lr 4.25e-04 | 322.48 ms | 52.3% bf16 MFU | 1625516 tok/s
step  497/19560 | loss 5.403422 (-1.62z)| norm 0.8047 (-0.59z)| lr 4.26e-04 | 322.35 ms | 52.4% bf16 MFU | 1625564 tok/s
step  498/19560 | loss 5.416341 (-1.50z)| norm 0.9743 (+0.42z)| lr 4.27e-04 | 322.39 ms | 52.4% bf16 MFU | 1625598 tok/s
step  499/19560 | loss 5.419875 (-1.45z)| norm 1.0150 (+0.66z)| lr 4.28e-04 | 322.48 ms | 52.3% bf16 MFU | 1625609 tok/s
step  500/19560 | loss 5.402827 (-1.57z)| norm 0.8915 (-0.07z)| lr 4.29e-04 | 322.95 ms | 52.3% bf16 MFU | 1625501 tok/s
val loss 5.389525
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2388/10042 = 0.237801
step  501/19560 | loss 5.524487 (-0.51z)| norm 1.0309 (+0.74z)| lr 4.29e-04 | 321.91 ms | 52.4% bf16 MFU | 1625661 tok/s
step  502/19560 | loss 5.402844 (-1.55z)| norm 1.0707 (+0.96z)| lr 4.30e-04 | 323.14 ms | 52.2% bf16 MFU | 1625503 tok/s
step  503/19560 | loss 5.378118 (-1.73z)| norm 0.8648 (-0.26z)| lr 4.31e-04 | 322.31 ms | 52.4% bf16 MFU | 1625560 tok/s
step  504/19560 | loss 5.369720 (-1.78z)| norm 0.7505 (-0.93z)| lr 4.32e-04 | 322.24 ms | 52.4% bf16 MFU | 1625632 tok/s
step  505/19560 | loss 5.428893 (-1.25z)| norm 0.7723 (-0.79z)| lr 4.33e-04 | 322.32 ms | 52.4% bf16 MFU | 1625680 tok/s
step  506/19560 | loss 5.347645 (-1.92z)| norm 0.8359 (-0.40z)| lr 4.34e-04 | 323.25 ms | 52.2% bf16 MFU | 1625492 tok/s
step  507/19560 | loss 5.348750 (-1.88z)| norm 0.9073 (+0.03z)| lr 4.35e-04 | 323.14 ms | 52.2% bf16 MFU | 1625340 tok/s
step  508/19560 | loss 5.432893 (-1.13z)| norm 0.8907 (-0.07z)| lr 4.35e-04 | 323.01 ms | 52.2% bf16 MFU | 1625230 tok/s
step  509/19560 | loss 5.358836 (-1.75z)| norm 0.8624 (-0.24z)| lr 4.36e-04 | 323.96 ms | 52.1% bf16 MFU | 1624888 tok/s
step  510/19560 | loss 5.419590 (-1.21z)| norm 0.8713 (-0.19z)| lr 4.37e-04 | 322.46 ms | 52.3% bf16 MFU | 1624938 tok/s
step  511/19560 | loss 5.351375 (-1.77z)| norm 0.8790 (-0.14z)| lr 4.38e-04 | 322.79 ms | 52.3% bf16 MFU | 1624901 tok/s
step  512/19560 | loss 5.317251 (-2.04z)| norm 0.9398 (+0.22z)| lr 4.39e-04 | 322.91 ms | 52.3% bf16 MFU | 1624839 tok/s
step  513/19560 | loss 5.337533 (-1.83z)| norm 0.7361 (-0.99z)| lr 4.40e-04 | 322.53 ms | 52.3% bf16 MFU | 1624874 tok/s
step  514/19560 | loss 5.347617 (-1.71z)| norm 0.7874 (-0.68z)| lr 4.41e-04 | 322.30 ms | 52.4% bf16 MFU | 1624966 tok/s
step  515/19560 | loss 5.370241 (-1.49z)| norm 0.7819 (-0.72z)| lr 4.41e-04 | 323.20 ms | 52.2% bf16 MFU | 1624827 tok/s
step  516/19560 | loss 5.361914 (-1.53z)| norm 0.7619 (-0.85z)| lr 4.42e-04 | 322.99 ms | 52.3% bf16 MFU | 1624747 tok/s
step  517/19560 | loss 5.332770 (-1.76z)| norm 0.8869 (-0.10z)| lr 4.43e-04 | 322.56 ms | 52.3% bf16 MFU | 1624780 tok/s
step  518/19560 | loss 5.362351 (-1.48z)| norm 1.0179 (+0.68z)| lr 4.44e-04 | 322.56 ms | 52.3% bf16 MFU | 1624810 tok/s
step  519/19560 | loss 5.306779 (-1.92z)| norm 0.9885 (+0.52z)| lr 4.45e-04 | 323.32 ms | 52.2% bf16 MFU | 1624648 tok/s
step  520/19560 | loss 5.336686 (-1.64z)| norm 1.0889 (+1.21z)| lr 4.46e-04 | 322.96 ms | 52.3% bf16 MFU | 1624583 tok/s
step  521/19560 | loss 5.382787 (-1.23z)| norm 0.9118 (+0.07z)| lr 4.47e-04 | 322.79 ms | 52.3% bf16 MFU | 1624566 tok/s
step  522/19560 | loss 5.291169 (-1.99z)| norm 0.9392 (+0.26z)| lr 4.47e-04 | 323.29 ms | 52.2% bf16 MFU | 1624423 tok/s
step  523/19560 | loss 5.337360 (-1.57z)| norm 0.9163 (+0.12z)| lr 4.48e-04 | 323.08 ms | 52.2% bf16 MFU | 1624340 tok/s
step  524/19560 | loss 5.252901 (-2.26z)| norm 0.7830 (-0.74z)| lr 4.49e-04 | 322.74 ms | 52.3% bf16 MFU | 1624348 tok/s
step  525/19560 | loss 5.320742 (-1.64z)| norm 0.6455 (-1.61z)| lr 4.50e-04 | 322.69 ms | 52.3% bf16 MFU | 1624368 tok/s
step  526/19560 | loss 5.293087 (-1.85z)| norm 0.6213 (-1.74z)| lr 4.51e-04 | 322.36 ms | 52.4% bf16 MFU | 1624470 tok/s
step  527/19560 | loss 5.325662 (-1.55z)| norm 0.6578 (-1.48z)| lr 4.52e-04 | 322.52 ms | 52.3% bf16 MFU | 1624526 tok/s
step  528/19560 | loss 5.362967 (-1.22z)| norm 0.7042 (-1.16z)| lr 4.53e-04 | 323.27 ms | 52.2% bf16 MFU | 1624392 tok/s
step  529/19560 | loss 5.212873 (-2.46z)| norm 0.7884 (-0.60z)| lr 4.53e-04 | 322.32 ms | 52.4% bf16 MFU | 1624503 tok/s
step  530/19560 | loss 5.369731 (-1.10z)| norm 0.9738 (+0.66z)| lr 4.54e-04 | 322.80 ms | 52.3% bf16 MFU | 1624488 tok/s
step  531/19560 | loss 5.325718 (-1.47z)| norm 0.8640 (-0.08z)| lr 4.55e-04 | 322.91 ms | 52.3% bf16 MFU | 1624444 tok/s
step  532/19560 | loss 5.299959 (-1.66z)| norm 0.7597 (-0.79z)| lr 4.56e-04 | 322.36 ms | 52.4% bf16 MFU | 1624542 tok/s
step  533/19560 | loss 5.282382 (-1.79z)| norm 0.8405 (-0.23z)| lr 4.57e-04 | 322.76 ms | 52.3% bf16 MFU | 1624534 tok/s
step  534/19560 | loss 5.308837 (-1.54z)| norm 0.9572 (+0.59z)| lr 4.58e-04 | 322.25 ms | 52.4% bf16 MFU | 1624656 tok/s
step  535/19560 | loss 5.254652 (-1.97z)| norm 0.8290 (-0.29z)| lr 4.59e-04 | 322.80 ms | 52.3% bf16 MFU | 1624633 tok/s
step  536/19560 | loss 5.262820 (-1.88z)| norm 0.8172 (-0.38z)| lr 4.59e-04 | 323.17 ms | 52.2% bf16 MFU | 1624517 tok/s
step  537/19560 | loss 5.207742 (-2.31z)| norm 0.9359 (+0.45z)| lr 4.60e-04 | 322.30 ms | 52.4% bf16 MFU | 1624625 tok/s
step  538/19560 | loss 5.228716 (-2.08z)| norm 0.8486 (-0.17z)| lr 4.61e-04 | 322.94 ms | 52.3% bf16 MFU | 1624568 tok/s
step  539/19560 | loss 5.280213 (-1.61z)| norm 0.7906 (-0.57z)| lr 4.62e-04 | 322.47 ms | 52.3% bf16 MFU | 1624632 tok/s
step  540/19560 | loss 5.303538 (-1.39z)| norm 0.7402 (-0.93z)| lr 4.63e-04 | 323.21 ms | 52.2% bf16 MFU | 1624506 tok/s
step  541/19560 | loss 5.209216 (-2.16z)| norm 0.8289 (-0.31z)| lr 4.64e-04 | 322.89 ms | 52.3% bf16 MFU | 1624468 tok/s
step  542/19560 | loss 5.247682 (-1.80z)| norm 0.8443 (-0.19z)| lr 4.65e-04 | 323.28 ms | 52.2% bf16 MFU | 1624334 tok/s
step  543/19560 | loss 5.272687 (-1.56z)| norm 0.7364 (-0.95z)| lr 4.65e-04 | 322.32 ms | 52.4% bf16 MFU | 1624446 tok/s
step  544/19560 | loss 5.200600 (-2.13z)| norm 0.6822 (-1.33z)| lr 4.66e-04 | 322.37 ms | 52.4% bf16 MFU | 1624540 tok/s
step  545/19560 | loss 5.296463 (-1.30z)| norm 0.5700 (-2.08z)| lr 4.67e-04 | 322.31 ms | 52.4% bf16 MFU | 1624646 tok/s
step  546/19560 | loss 5.208014 (-2.02z)| norm 0.5812 (-1.99z)| lr 4.68e-04 | 322.71 ms | 52.3% bf16 MFU | 1624645 tok/s
step  547/19560 | loss 5.178508 (-2.23z)| norm 0.6208 (-1.69z)| lr 4.69e-04 | 322.78 ms | 52.3% bf16 MFU | 1624628 tok/s
step  548/19560 | loss 5.329017 (-0.94z)| norm 0.8896 (+0.17z)| lr 4.70e-04 | 322.59 ms | 52.3% bf16 MFU | 1624658 tok/s
step  549/19560 | loss 5.254537 (-1.55z)| norm 1.3744 (+3.38z)| lr 4.71e-04 | 322.24 ms | 52.4% bf16 MFU | 1624775 tok/s
step  550/19560 | loss 5.273184 (-1.37z)| norm 1.1152 (+1.62z)| lr 4.71e-04 | 322.61 ms | 52.3% bf16 MFU | 1624794 tok/s
step  551/19560 | loss 5.319035 (-0.96z)| norm 0.9714 (+0.66z)| lr 4.72e-04 | 322.81 ms | 52.3% bf16 MFU | 1624761 tok/s
step  552/19560 | loss 5.250103 (-1.53z)| norm 1.0021 (+0.86z)| lr 4.73e-04 | 323.25 ms | 52.2% bf16 MFU | 1624620 tok/s
step  553/19560 | loss 5.222087 (-1.74z)| norm 0.9173 (+0.29z)| lr 4.74e-04 | 322.77 ms | 52.3% bf16 MFU | 1624606 tok/s
step  554/19560 | loss 5.268061 (-1.33z)| norm 0.8835 (+0.06z)| lr 4.75e-04 | 322.23 ms | 52.4% bf16 MFU | 1624728 tok/s
step  555/19560 | loss 5.226632 (-1.67z)| norm 0.8722 (-0.00z)| lr 4.76e-04 | 322.84 ms | 52.3% bf16 MFU | 1624692 tok/s
step  556/19560 | loss 5.238420 (-1.54z)| norm 0.8402 (-0.21z)| lr 4.77e-04 | 322.72 ms | 52.3% bf16 MFU | 1624687 tok/s
step  557/19560 | loss 5.238579 (-1.52z)| norm 1.1119 (+1.62z)| lr 4.77e-04 | 322.29 ms | 52.4% bf16 MFU | 1624791 tok/s
step  558/19560 | loss 5.134130 (-2.36z)| norm 0.6978 (-1.16z)| lr 4.78e-04 | 322.55 ms | 52.3% bf16 MFU | 1624824 tok/s
step  559/19560 | loss 5.227890 (-1.53z)| norm 0.6225 (-1.63z)| lr 4.79e-04 | 322.82 ms | 52.3% bf16 MFU | 1624788 tok/s
step  560/19560 | loss 5.213434 (-1.63z)| norm 0.5709 (-1.93z)| lr 4.80e-04 | 323.26 ms | 52.2% bf16 MFU | 1624642 tok/s
step  561/19560 | loss 5.256575 (-1.24z)| norm 0.6584 (-1.33z)| lr 4.81e-04 | 322.55 ms | 52.3% bf16 MFU | 1624683 tok/s
step  562/19560 | loss 5.221104 (-1.52z)| norm 0.6159 (-1.59z)| lr 4.82e-04 | 322.84 ms | 52.3% bf16 MFU | 1624648 tok/s
step  563/19560 | loss 5.249693 (-1.26z)| norm 0.7306 (-0.85z)| lr 4.83e-04 | 322.57 ms | 52.3% bf16 MFU | 1624682 tok/s
step  564/19560 | loss 5.215561 (-1.53z)| norm 0.8677 (+0.04z)| lr 4.83e-04 | 322.92 ms | 52.3% bf16 MFU | 1624626 tok/s
step  565/19560 | loss 5.207976 (-1.57z)| norm 0.8878 (+0.17z)| lr 4.84e-04 | 322.48 ms | 52.3% bf16 MFU | 1624686 tok/s
step  566/19560 | loss 5.194946 (-1.66z)| norm 0.9942 (+0.88z)| lr 4.85e-04 | 322.46 ms | 52.3% bf16 MFU | 1624746 tok/s
step  567/19560 | loss 5.227783 (-1.36z)| norm 0.9163 (+0.37z)| lr 4.86e-04 | 322.90 ms | 52.3% bf16 MFU | 1624692 tok/s
step  568/19560 | loss 5.199757 (-1.57z)| norm 0.6972 (-1.06z)| lr 4.87e-04 | 322.60 ms | 52.3% bf16 MFU | 1624718 tok/s
step  569/19560 | loss 5.228430 (-1.30z)| norm 0.7097 (-0.98z)| lr 4.88e-04 | 322.25 ms | 52.4% bf16 MFU | 1624830 tok/s
step  570/19560 | loss 5.198398 (-1.54z)| norm 0.7512 (-0.71z)| lr 4.89e-04 | 322.82 ms | 52.3% bf16 MFU | 1624793 tok/s
step  571/19560 | loss 5.212082 (-1.40z)| norm 0.7198 (-0.92z)| lr 4.89e-04 | 322.79 ms | 52.3% bf16 MFU | 1624764 tok/s
step  572/19560 | loss 5.179855 (-1.64z)| norm 0.7626 (-0.64z)| lr 4.90e-04 | 322.76 ms | 52.3% bf16 MFU | 1624745 tok/s
step  573/19560 | loss 5.130253 (-2.02z)| norm 0.7789 (-0.54z)| lr 4.91e-04 | 322.91 ms | 52.3% bf16 MFU | 1624689 tok/s
step  574/19560 | loss 5.137377 (-1.92z)| norm 0.7269 (-0.88z)| lr 4.92e-04 | 323.11 ms | 52.2% bf16 MFU | 1624587 tok/s
step  575/19560 | loss 5.140460 (-1.86z)| norm 0.6764 (-1.19z)| lr 4.93e-04 | 322.86 ms | 52.3% bf16 MFU | 1624552 tok/s
step  576/19560 | loss 5.175934 (-1.53z)| norm 0.6887 (-1.09z)| lr 4.94e-04 | 322.10 ms | 52.4% bf16 MFU | 1624709 tok/s
step  577/19560 | loss 5.175227 (-1.52z)| norm 0.8266 (-0.19z)| lr 4.95e-04 | 322.57 ms | 52.3% bf16 MFU | 1624741 tok/s
step  578/19560 | loss 5.202324 (-1.28z)| norm 0.8131 (-0.27z)| lr 4.95e-04 | 323.05 ms | 52.2% bf16 MFU | 1624651 tok/s
step  579/19560 | loss 5.226425 (-1.06z)| norm 0.8043 (-0.33z)| lr 4.96e-04 | 322.88 ms | 52.3% bf16 MFU | 1624607 tok/s
step  580/19560 | loss 5.211358 (-1.18z)| norm 0.8189 (-0.24z)| lr 4.97e-04 | 322.41 ms | 52.3% bf16 MFU | 1624685 tok/s
step  581/19560 | loss 5.119787 (-1.92z)| norm 0.8870 (+0.21z)| lr 4.98e-04 | 322.33 ms | 52.4% bf16 MFU | 1624779 tok/s
step  582/19560 | loss 5.162151 (-1.54z)| norm 0.8209 (-0.21z)| lr 4.99e-04 | 323.11 ms | 52.2% bf16 MFU | 1624670 tok/s
step  583/19560 | loss 5.181075 (-1.36z)| norm 0.7294 (-0.80z)| lr 5.00e-04 | 322.62 ms | 52.3% bf16 MFU | 1624690 tok/s
step  584/19560 | loss 5.152989 (-1.57z)| norm 0.9127 (+0.43z)| lr 5.01e-04 | 322.44 ms | 52.3% bf16 MFU | 1624756 tok/s
step  585/19560 | loss 5.163503 (-1.47z)| norm 0.9945 (+0.99z)| lr 5.01e-04 | 322.57 ms | 52.3% bf16 MFU | 1624785 tok/s
step  586/19560 | loss 5.181536 (-1.30z)| norm 1.0547 (+1.37z)| lr 5.02e-04 | 323.00 ms | 52.3% bf16 MFU | 1624706 tok/s
step  587/19560 | loss 5.106391 (-1.92z)| norm 0.8645 (+0.10z)| lr 5.03e-04 | 322.37 ms | 52.4% bf16 MFU | 1624787 tok/s
step  588/19560 | loss 5.168844 (-1.36z)| norm 0.7727 (-0.51z)| lr 5.04e-04 | 323.09 ms | 52.2% bf16 MFU | 1624683 tok/s
step  589/19560 | loss 5.157025 (-1.44z)| norm 0.7311 (-0.79z)| lr 5.05e-04 | 322.60 ms | 52.3% bf16 MFU | 1624708 tok/s
step  590/19560 | loss 5.073137 (-2.12z)| norm 0.6870 (-1.08z)| lr 5.06e-04 | 322.72 ms | 52.3% bf16 MFU | 1624703 tok/s
step  591/19560 | loss 5.136032 (-1.57z)| norm 0.6021 (-1.62z)| lr 5.07e-04 | 322.43 ms | 52.3% bf16 MFU | 1624769 tok/s
step  592/19560 | loss 5.194335 (-1.05z)| norm 0.5910 (-1.68z)| lr 5.07e-04 | 322.41 ms | 52.3% bf16 MFU | 1624840 tok/s
step  593/19560 | loss 5.107955 (-1.78z)| norm 0.5316 (-2.02z)| lr 5.08e-04 | 322.99 ms | 52.3% bf16 MFU | 1624760 tok/s
step  594/19560 | loss 5.113731 (-1.70z)| norm 0.5490 (-1.87z)| lr 5.09e-04 | 322.83 ms | 52.3% bf16 MFU | 1624724 tok/s
step  595/19560 | loss 5.116246 (-1.65z)| norm 0.6390 (-1.26z)| lr 5.10e-04 | 322.89 ms | 52.3% bf16 MFU | 1624674 tok/s
step  596/19560 | loss 5.130189 (-1.50z)| norm 0.8921 (+0.38z)| lr 5.11e-04 | 322.34 ms | 52.4% bf16 MFU | 1624765 tok/s
step  597/19560 | loss 5.237840 (-0.55z)| norm 1.1005 (+1.70z)| lr 5.12e-04 | 322.56 ms | 52.3% bf16 MFU | 1624796 tok/s
step  598/19560 | loss 5.121524 (-1.56z)| norm 0.7647 (-0.45z)| lr 5.13e-04 | 322.96 ms | 52.3% bf16 MFU | 1624725 tok/s
step  599/19560 | loss 5.108318 (-1.65z)| norm 0.7731 (-0.38z)| lr 5.13e-04 | 322.81 ms | 52.3% bf16 MFU | 1624695 tok/s
step  600/19560 | loss 5.082011 (-1.85z)| norm 0.6581 (-1.13z)| lr 5.14e-04 | 322.94 ms | 52.3% bf16 MFU | 1624635 tok/s
step  601/19560 | loss 5.203005 (-0.76z)| norm 0.5965 (-1.51z)| lr 5.15e-04 | 322.70 ms | 52.3% bf16 MFU | 1624638 tok/s
step  602/19560 | loss 5.128351 (-1.41z)| norm 0.5401 (-1.84z)| lr 5.16e-04 | 321.89 ms | 52.4% bf16 MFU | 1624846 tok/s
step  603/19560 | loss 5.117797 (-1.48z)| norm 0.5554 (-1.71z)| lr 5.17e-04 | 322.56 ms | 52.3% bf16 MFU | 1624873 tok/s
step  604/19560 | loss 5.081254 (-1.77z)| norm 0.5858 (-1.50z)| lr 5.18e-04 | 322.75 ms | 52.3% bf16 MFU | 1624850 tok/s
step  605/19560 | loss 5.069843 (-1.85z)| norm 0.7113 (-0.68z)| lr 5.19e-04 | 322.52 ms | 52.3% bf16 MFU | 1624888 tok/s
step  606/19560 | loss 5.118257 (-1.40z)| norm 0.7893 (-0.18z)| lr 5.19e-04 | 322.32 ms | 52.4% bf16 MFU | 1624973 tok/s
step  607/19560 | loss 5.081005 (-1.71z)| norm 0.8457 (+0.19z)| lr 5.20e-04 | 322.38 ms | 52.4% bf16 MFU | 1625040 tok/s
step  608/19560 | loss 5.107182 (-1.45z)| norm 0.9137 (+0.65z)| lr 5.21e-04 | 322.99 ms | 52.3% bf16 MFU | 1624948 tok/s
step  609/19560 | loss 5.131974 (-1.21z)| norm 0.9982 (+1.24z)| lr 5.22e-04 | 323.11 ms | 52.2% bf16 MFU | 1624832 tok/s
step  610/19560 | loss 5.106199 (-1.43z)| norm 0.9657 (+1.02z)| lr 5.23e-04 | 322.91 ms | 52.3% bf16 MFU | 1624772 tok/s
step  611/19560 | loss 5.098745 (-1.48z)| norm 0.8403 (+0.19z)| lr 5.24e-04 | 322.65 ms | 52.3% bf16 MFU | 1624780 tok/s
step  612/19560 | loss 5.103807 (-1.41z)| norm 0.7037 (-0.72z)| lr 5.25e-04 | 322.54 ms | 52.3% bf16 MFU | 1624815 tok/s
step  613/19560 | loss 5.090555 (-1.51z)| norm 0.7163 (-0.64z)| lr 5.25e-04 | 322.94 ms | 52.3% bf16 MFU | 1624749 tok/s
step  614/19560 | loss 5.085859 (-1.54z)| norm 0.7535 (-0.40z)| lr 5.26e-04 | 322.84 ms | 52.3% bf16 MFU | 1624710 tok/s
step  615/19560 | loss 5.074931 (-1.61z)| norm 0.7368 (-0.53z)| lr 5.27e-04 | 322.62 ms | 52.3% bf16 MFU | 1624730 tok/s
step  616/19560 | loss 5.035066 (-1.95z)| norm 0.7201 (-0.65z)| lr 5.28e-04 | 322.55 ms | 52.3% bf16 MFU | 1624765 tok/s
step  617/19560 | loss 5.019628 (-2.05z)| norm 0.6617 (-1.04z)| lr 5.29e-04 | 322.63 ms | 52.3% bf16 MFU | 1624780 tok/s
step  618/19560 | loss 5.136543 (-0.97z)| norm 0.7097 (-0.70z)| lr 5.30e-04 | 322.64 ms | 52.3% bf16 MFU | 1624790 tok/s
step  619/19560 | loss 5.069710 (-1.56z)| norm 0.8442 (+0.24z)| lr 5.31e-04 | 322.86 ms | 52.3% bf16 MFU | 1624744 tok/s
step  620/19560 | loss 5.057464 (-1.65z)| norm 0.8453 (+0.24z)| lr 5.31e-04 | 322.35 ms | 52.4% bf16 MFU | 1624831 tok/s
step  621/19560 | loss 5.044183 (-1.74z)| norm 0.7773 (-0.24z)| lr 5.32e-04 | 322.35 ms | 52.4% bf16 MFU | 1624913 tok/s
step  622/19560 | loss 4.990043 (-2.19z)| norm 0.6644 (-1.02z)| lr 5.33e-04 | 322.84 ms | 52.3% bf16 MFU | 1624866 tok/s
step  623/19560 | loss 5.012901 (-1.94z)| norm 0.6626 (-1.01z)| lr 5.34e-04 | 322.57 ms | 52.3% bf16 MFU | 1624890 tok/s
step  624/19560 | loss 5.017928 (-1.86z)| norm 0.6281 (-1.23z)| lr 5.35e-04 | 322.42 ms | 52.3% bf16 MFU | 1624951 tok/s
step  625/19560 | loss 5.068626 (-1.38z)| norm 0.5539 (-1.71z)| lr 5.36e-04 | 322.50 ms | 52.3% bf16 MFU | 1624989 tok/s
step  626/19560 | loss 5.031133 (-1.70z)| norm 0.5444 (-1.74z)| lr 5.37e-04 | 323.10 ms | 52.2% bf16 MFU | 1624874 tok/s
step  627/19560 | loss 5.077384 (-1.26z)| norm 0.5904 (-1.41z)| lr 5.37e-04 | 322.76 ms | 52.3% bf16 MFU | 1624851 tok/s
step  628/19560 | loss 5.064902 (-1.36z)| norm 0.6532 (-0.97z)| lr 5.38e-04 | 322.41 ms | 52.3% bf16 MFU | 1624916 tok/s
step  629/19560 | loss 5.035083 (-1.64z)| norm 0.6408 (-1.04z)| lr 5.39e-04 | 323.11 ms | 52.2% bf16 MFU | 1624801 tok/s
step  630/19560 | loss 5.014622 (-1.80z)| norm 0.6388 (-1.04z)| lr 5.40e-04 | 322.66 ms | 52.3% bf16 MFU | 1624805 tok/s
step  631/19560 | loss 5.048150 (-1.46z)| norm 0.6796 (-0.75z)| lr 5.41e-04 | 322.43 ms | 52.3% bf16 MFU | 1624867 tok/s
step  632/19560 | loss 5.036717 (-1.55z)| norm 0.7330 (-0.38z)| lr 5.42e-04 | 322.86 ms | 52.3% bf16 MFU | 1624818 tok/s
step  633/19560 | loss 5.043062 (-1.47z)| norm 0.9825 (+1.32z)| lr 5.43e-04 | 322.71 ms | 52.3% bf16 MFU | 1624809 tok/s
step  634/19560 | loss 5.149160 (-0.45z)| norm 1.1988 (+2.69z)| lr 5.43e-04 | 322.75 ms | 52.3% bf16 MFU | 1624790 tok/s
step  635/19560 | loss 5.007878 (-1.78z)| norm 0.7625 (-0.19z)| lr 5.44e-04 | 322.57 ms | 52.3% bf16 MFU | 1624819 tok/s
step  636/19560 | loss 5.065555 (-1.21z)| norm 0.6432 (-0.97z)| lr 5.45e-04 | 322.34 ms | 52.4% bf16 MFU | 1624904 tok/s
step  637/19560 | loss 4.986655 (-1.95z)| norm 0.6298 (-1.04z)| lr 5.46e-04 | 322.83 ms | 52.3% bf16 MFU | 1624862 tok/s
step  638/19560 | loss 5.013972 (-1.67z)| norm 0.6211 (-1.08z)| lr 5.47e-04 | 323.14 ms | 52.2% bf16 MFU | 1624743 tok/s
step  639/19560 | loss 5.062648 (-1.17z)| norm 0.6861 (-0.65z)| lr 5.48e-04 | 322.84 ms | 52.3% bf16 MFU | 1624705 tok/s
step  640/19560 | loss 5.039643 (-1.38z)| norm 0.7980 (+0.10z)| lr 5.49e-04 | 322.40 ms | 52.3% bf16 MFU | 1624779 tok/s
step  641/19560 | loss 5.029319 (-1.46z)| norm 0.9868 (+1.33z)| lr 5.49e-04 | 322.91 ms | 52.3% bf16 MFU | 1624722 tok/s
step  642/19560 | loss 4.993726 (-1.78z)| norm 1.0527 (+1.72z)| lr 5.50e-04 | 322.89 ms | 52.3% bf16 MFU | 1624674 tok/s
step  643/19560 | loss 5.049856 (-1.21z)| norm 0.9163 (+0.83z)| lr 5.51e-04 | 322.89 ms | 52.3% bf16 MFU | 1624627 tok/s
step  644/19560 | loss 5.062680 (-1.07z)| norm 0.7868 (-0.01z)| lr 5.52e-04 | 322.81 ms | 52.3% bf16 MFU | 1624603 tok/s
step  645/19560 | loss 5.028902 (-1.39z)| norm 0.6895 (-0.63z)| lr 5.53e-04 | 322.80 ms | 52.3% bf16 MFU | 1624582 tok/s
step  646/19560 | loss 4.973598 (-1.92z)| norm 0.7215 (-0.41z)| lr 5.54e-04 | 322.99 ms | 52.3% bf16 MFU | 1624515 tok/s
step  647/19560 | loss 5.006144 (-1.56z)| norm 0.7853 (+0.02z)| lr 5.55e-04 | 322.72 ms | 52.3% bf16 MFU | 1624518 tok/s
step  648/19560 | loss 5.010554 (-1.50z)| norm 0.6757 (-0.69z)| lr 5.55e-04 | 322.42 ms | 52.3% bf16 MFU | 1624597 tok/s
step  649/19560 | loss 4.993208 (-1.66z)| norm 0.5831 (-1.29z)| lr 5.56e-04 | 322.93 ms | 52.3% bf16 MFU | 1624544 tok/s
step  650/19560 | loss 4.999944 (-1.56z)| norm 0.6699 (-0.70z)| lr 5.57e-04 | 323.05 ms | 52.2% bf16 MFU | 1624464 tok/s
step  651/19560 | loss 4.994024 (-1.60z)| norm 0.8102 (+0.24z)| lr 5.58e-04 | 322.51 ms | 52.3% bf16 MFU | 1624522 tok/s
step  652/19560 | loss 4.952317 (-1.98z)| norm 0.8145 (+0.27z)| lr 5.59e-04 | 322.58 ms | 52.3% bf16 MFU | 1624561 tok/s
step  653/19560 | loss 5.064122 (-0.84z)| norm 0.9658 (+1.26z)| lr 5.60e-04 | 322.76 ms | 52.3% bf16 MFU | 1624552 tok/s
step  654/19560 | loss 4.946109 (-2.00z)| norm 0.8448 (+0.44z)| lr 5.61e-04 | 322.92 ms | 52.3% bf16 MFU | 1624504 tok/s
step  655/19560 | loss 4.934239 (-2.08z)| norm 0.7075 (-0.48z)| lr 5.61e-04 | 323.45 ms | 52.2% bf16 MFU | 1624325 tok/s
step  656/19560 | loss 4.991432 (-1.49z)| norm 0.6353 (-0.96z)| lr 5.62e-04 | 322.74 ms | 52.3% bf16 MFU | 1624333 tok/s
step  657/19560 | loss 4.938716 (-1.98z)| norm 0.6217 (-1.03z)| lr 5.63e-04 | 323.25 ms | 52.2% bf16 MFU | 1624212 tok/s
step  658/19560 | loss 5.006645 (-1.29z)| norm 0.5430 (-1.53z)| lr 5.64e-04 | 322.81 ms | 52.3% bf16 MFU | 1624209 tok/s
step  659/19560 | loss 5.036720 (-0.97z)| norm 0.5862 (-1.22z)| lr 5.65e-04 | 323.65 ms | 52.1% bf16 MFU | 1623994 tok/s
step  660/19560 | loss 4.961880 (-1.72z)| norm 0.6142 (-1.03z)| lr 5.66e-04 | 322.49 ms | 52.3% bf16 MFU | 1624082 tok/s
step  661/19560 | loss 5.270241 (+1.46z)| norm 0.7808 (+0.07z)| lr 5.67e-04 | 322.92 ms | 52.3% bf16 MFU | 1624058 tok/s
step  662/19560 | loss 5.036720 (-0.93z)| norm 1.0769 (+1.99z)| lr 5.67e-04 | 322.78 ms | 52.3% bf16 MFU | 1624070 tok/s
step  663/19560 | loss 4.994245 (-1.35z)| norm 0.9730 (+1.30z)| lr 5.68e-04 | 322.92 ms | 52.3% bf16 MFU | 1624046 tok/s
step  664/19560 | loss 4.989199 (-1.39z)| norm 0.7031 (-0.44z)| lr 5.69e-04 | 322.82 ms | 52.3% bf16 MFU | 1624049 tok/s
step  665/19560 | loss 5.085757 (-0.37z)| norm 0.8159 (+0.30z)| lr 5.70e-04 | 323.21 ms | 52.2% bf16 MFU | 1623953 tok/s
step  666/19560 | loss 4.998095 (-1.27z)| norm 0.8673 (+0.63z)| lr 5.71e-04 | 322.40 ms | 52.3% bf16 MFU | 1624066 tok/s
step  667/19560 | loss 4.984738 (-1.39z)| norm 0.8970 (+0.81z)| lr 5.72e-04 | 322.80 ms | 52.3% bf16 MFU | 1624072 tok/s
step  668/19560 | loss 4.964344 (-1.58z)| norm 1.0919 (+2.03z)| lr 5.73e-04 | 323.09 ms | 52.2% bf16 MFU | 1624005 tok/s
step  669/19560 | loss 5.018040 (-1.00z)| norm 1.0096 (+1.48z)| lr 5.73e-04 | 322.88 ms | 52.3% bf16 MFU | 1623995 tok/s
step  670/19560 | loss 4.987677 (-1.30z)| norm 0.8449 (+0.44z)| lr 5.74e-04 | 322.63 ms | 52.3% bf16 MFU | 1624046 tok/s
step  671/19560 | loss 4.984091 (-1.32z)| norm 0.8880 (+0.70z)| lr 5.75e-04 | 322.98 ms | 52.3% bf16 MFU | 1624008 tok/s
step  672/19560 | loss 5.033219 (-0.79z)| norm 1.1347 (+2.20z)| lr 5.76e-04 | 322.62 ms | 52.3% bf16 MFU | 1624063 tok/s
step  673/19560 | loss 5.003238 (-1.09z)| norm 0.8940 (+0.69z)| lr 5.77e-04 | 322.81 ms | 52.3% bf16 MFU | 1624067 tok/s
step  674/19560 | loss 5.015086 (-0.95z)| norm 0.7828 (-0.01z)| lr 5.78e-04 | 322.83 ms | 52.3% bf16 MFU | 1624065 tok/s
step  675/19560 | loss 4.988688 (-1.21z)| norm 0.7909 (+0.03z)| lr 5.79e-04 | 323.04 ms | 52.2% bf16 MFU | 1624011 tok/s
step  676/19560 | loss 4.986254 (-1.23z)| norm 0.8312 (+0.29z)| lr 5.79e-04 | 322.67 ms | 52.3% bf16 MFU | 1624051 tok/s
step  677/19560 | loss 5.023244 (-0.82z)| norm 0.9255 (+0.96z)| lr 5.80e-04 | 322.45 ms | 52.3% bf16 MFU | 1624145 tok/s
step  678/19560 | loss 4.936784 (-1.74z)| norm 0.6732 (-0.71z)| lr 5.81e-04 | 323.00 ms | 52.3% bf16 MFU | 1624097 tok/s
step  679/19560 | loss 4.886338 (-2.27z)| norm 0.5921 (-1.24z)| lr 5.82e-04 | 323.13 ms | 52.2% bf16 MFU | 1624019 tok/s
step  680/19560 | loss 4.999411 (-1.01z)| norm 0.5438 (-1.54z)| lr 5.83e-04 | 322.93 ms | 52.3% bf16 MFU | 1623994 tok/s
step  681/19560 | loss 4.899426 (-2.08z)| norm 0.5390 (-1.55z)| lr 5.84e-04 | 323.12 ms | 52.2% bf16 MFU | 1623923 tok/s
step  682/19560 | loss 4.927174 (-1.75z)| norm 0.5453 (-1.48z)| lr 5.85e-04 | 322.58 ms | 52.3% bf16 MFU | 1623992 tok/s
step  683/19560 | loss 4.873587 (-2.29z)| norm 0.4864 (-1.83z)| lr 5.85e-04 | 322.87 ms | 52.3% bf16 MFU | 1623984 tok/s
step  684/19560 | loss 4.971961 (-1.19z)| norm 0.5311 (-1.51z)| lr 5.86e-04 | 323.19 ms | 52.2% bf16 MFU | 1623897 tok/s
step  685/19560 | loss 4.963676 (-1.27z)| norm 0.6455 (-0.75z)| lr 5.87e-04 | 322.91 ms | 52.3% bf16 MFU | 1623885 tok/s
step  686/19560 | loss 4.864251 (-2.30z)| norm 0.9424 (+1.22z)| lr 5.88e-04 | 322.73 ms | 52.3% bf16 MFU | 1623918 tok/s
step  687/19560 | loss 4.922982 (-1.64z)| norm 0.9146 (+1.02z)| lr 5.89e-04 | 323.14 ms | 52.2% bf16 MFU | 1623846 tok/s
step  688/19560 | loss 4.955513 (-1.26z)| norm 0.7357 (-0.18z)| lr 5.90e-04 | 322.81 ms | 52.3% bf16 MFU | 1623860 tok/s
step  689/19560 | loss 4.878910 (-2.07z)| norm 0.7903 (+0.18z)| lr 5.91e-04 | 322.56 ms | 52.3% bf16 MFU | 1623936 tok/s
step  690/19560 | loss 4.897873 (-1.83z)| norm 0.7225 (-0.28z)| lr 5.91e-04 | 322.85 ms | 52.3% bf16 MFU | 1623937 tok/s
step  691/19560 | loss 4.975124 (-0.97z)| norm 0.6568 (-0.72z)| lr 5.92e-04 | 323.08 ms | 52.2% bf16 MFU | 1623878 tok/s
step  692/19560 | loss 4.945214 (-1.29z)| norm 0.6408 (-0.81z)| lr 5.93e-04 | 322.59 ms | 52.3% bf16 MFU | 1623946 tok/s
step  693/19560 | loss 4.871378 (-2.06z)| norm 0.6512 (-0.73z)| lr 5.94e-04 | 322.48 ms | 52.3% bf16 MFU | 1624038 tok/s
step  694/19560 | loss 4.927730 (-1.42z)| norm 0.6274 (-0.88z)| lr 5.95e-04 | 322.63 ms | 52.3% bf16 MFU | 1624089 tok/s
step  695/19560 | loss 4.903580 (-1.67z)| norm 0.6795 (-0.51z)| lr 5.96e-04 | 323.14 ms | 52.2% bf16 MFU | 1624007 tok/s
step  696/19560 | loss 4.871758 (-1.98z)| norm 0.6566 (-0.67z)| lr 5.97e-04 | 323.09 ms | 52.2% bf16 MFU | 1623944 tok/s
step  697/19560 | loss 4.842562 (-2.26z)| norm 0.7150 (-0.27z)| lr 5.97e-04 | 322.67 ms | 52.3% bf16 MFU | 1623988 tok/s
step  698/19560 | loss 4.941879 (-1.15z)| norm 0.7540 (-0.01z)| lr 5.98e-04 | 323.42 ms | 52.2% bf16 MFU | 1623843 tok/s
step  699/19560 | loss 4.894193 (-1.65z)| norm 0.7439 (-0.08z)| lr 5.99e-04 | 323.39 ms | 52.2% bf16 MFU | 1623712 tok/s
step  700/19560 | loss 4.862970 (-1.96z)| norm 0.7967 (+0.28z)| lr 6.00e-04 | 323.18 ms | 52.2% bf16 MFU | 1623640 tok/s
step  701/19560 | loss 4.964519 (-0.83z)| norm 1.0231 (+1.78z)| lr 6.00e-04 | 322.89 ms | 52.3% bf16 MFU | 1623646 tok/s
step  702/19560 | loss 4.896083 (-1.55z)| norm 0.9381 (+1.19z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1623712 tok/s
step  703/19560 | loss 4.946777 (-0.98z)| norm 0.7163 (-0.29z)| lr 6.00e-04 | 323.21 ms | 52.2% bf16 MFU | 1623634 tok/s
step  704/19560 | loss 4.921443 (-1.24z)| norm 0.6400 (-0.79z)| lr 6.00e-04 | 322.19 ms | 52.4% bf16 MFU | 1623816 tok/s
step  705/19560 | loss 4.883496 (-1.63z)| norm 0.6028 (-1.02z)| lr 6.00e-04 | 322.85 ms | 52.3% bf16 MFU | 1623822 tok/s
step  706/19560 | loss 4.858153 (-1.88z)| norm 0.5906 (-1.09z)| lr 6.00e-04 | 322.56 ms | 52.3% bf16 MFU | 1623901 tok/s
step  707/19560 | loss 4.883925 (-1.58z)| norm 0.5978 (-1.03z)| lr 6.00e-04 | 323.05 ms | 52.2% bf16 MFU | 1623852 tok/s
step  708/19560 | loss 4.902527 (-1.36z)| norm 0.6575 (-0.63z)| lr 6.00e-04 | 322.38 ms | 52.4% bf16 MFU | 1623976 tok/s
step  709/19560 | loss 4.788436 (-2.55z)| norm 0.6490 (-0.67z)| lr 6.00e-04 | 322.51 ms | 52.3% bf16 MFU | 1624058 tok/s
step  710/19560 | loss 4.858588 (-1.75z)| norm 0.7027 (-0.31z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1624105 tok/s
step  711/19560 | loss 4.900668 (-1.28z)| norm 0.6287 (-0.79z)| lr 6.00e-04 | 322.30 ms | 52.4% bf16 MFU | 1624234 tok/s
step  712/19560 | loss 4.812255 (-2.20z)| norm 0.6168 (-0.86z)| lr 6.00e-04 | 323.10 ms | 52.2% bf16 MFU | 1624156 tok/s
step  713/19560 | loss 4.789722 (-2.38z)| norm 0.6140 (-0.86z)| lr 6.00e-04 | 323.09 ms | 52.2% bf16 MFU | 1624085 tok/s
step  714/19560 | loss 4.813803 (-2.08z)| norm 0.8224 (+0.54z)| lr 6.00e-04 | 322.71 ms | 52.3% bf16 MFU | 1624113 tok/s
step  715/19560 | loss 4.856848 (-1.59z)| norm 0.9713 (+1.53z)| lr 6.00e-04 | 323.54 ms | 52.2% bf16 MFU | 1623931 tok/s
step  716/19560 | loss 4.843964 (-1.70z)| norm 0.7927 (+0.33z)| lr 6.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1623975 tok/s
step  717/19560 | loss 4.830743 (-1.81z)| norm 0.6847 (-0.39z)| lr 6.00e-04 | 323.44 ms | 52.2% bf16 MFU | 1623824 tok/s
step  718/19560 | loss 4.736825 (-2.70z)| norm 0.8112 (+0.45z)| lr 6.00e-04 | 323.18 ms | 52.2% bf16 MFU | 1623747 tok/s
step  719/19560 | loss 4.860819 (-1.40z)| norm 0.7532 (+0.06z)| lr 6.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1623826 tok/s
step  720/19560 | loss 4.927311 (-0.71z)| norm 0.7661 (+0.13z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1623893 tok/s
step  721/19560 | loss 4.845044 (-1.54z)| norm 0.7537 (+0.04z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1623933 tok/s
step  722/19560 | loss 4.887622 (-1.08z)| norm 0.8110 (+0.42z)| lr 6.00e-04 | 322.37 ms | 52.4% bf16 MFU | 1624053 tok/s
step  723/19560 | loss 4.841437 (-1.53z)| norm 0.8767 (+0.85z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624069 tok/s
step  724/19560 | loss 4.820416 (-1.72z)| norm 0.8706 (+0.81z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1624080 tok/s
step  725/19560 | loss 4.830868 (-1.61z)| norm 0.7195 (-0.20z)| lr 6.00e-04 | 322.85 ms | 52.3% bf16 MFU | 1624072 tok/s
step  726/19560 | loss 4.766219 (-2.23z)| norm 0.5976 (-1.04z)| lr 6.00e-04 | 322.66 ms | 52.3% bf16 MFU | 1624114 tok/s
step  727/19560 | loss 4.871646 (-1.12z)| norm 0.5606 (-1.28z)| lr 6.00e-04 | 323.16 ms | 52.2% bf16 MFU | 1624026 tok/s
step  728/19560 | loss 4.815707 (-1.67z)| norm 0.4959 (-1.70z)| lr 6.00e-04 | 322.80 ms | 52.3% bf16 MFU | 1624035 tok/s
step  729/19560 | loss 4.751202 (-2.30z)| norm 0.4953 (-1.69z)| lr 6.00e-04 | 323.50 ms | 52.2% bf16 MFU | 1623867 tok/s
step  730/19560 | loss 4.781316 (-1.95z)| norm 0.4700 (-1.85z)| lr 6.00e-04 | 322.17 ms | 52.4% bf16 MFU | 1624041 tok/s
step  731/19560 | loss 4.783220 (-1.90z)| norm 0.5161 (-1.53z)| lr 6.00e-04 | 323.17 ms | 52.2% bf16 MFU | 1623954 tok/s
step  732/19560 | loss 4.766355 (-2.02z)| norm 0.4855 (-1.72z)| lr 6.00e-04 | 322.88 ms | 52.3% bf16 MFU | 1623947 tok/s
step  733/19560 | loss 4.824569 (-1.41z)| norm 0.5339 (-1.37z)| lr 6.00e-04 | 323.06 ms | 52.2% bf16 MFU | 1623893 tok/s
step  734/19560 | loss 4.694496 (-2.64z)| norm 0.6102 (-0.86z)| lr 6.00e-04 | 323.36 ms | 52.2% bf16 MFU | 1623766 tok/s
step  735/19560 | loss 4.752392 (-2.02z)| norm 0.6103 (-0.84z)| lr 6.00e-04 | 322.87 ms | 52.3% bf16 MFU | 1623771 tok/s
step  736/19560 | loss 4.744374 (-2.05z)| norm 0.6483 (-0.58z)| lr 6.00e-04 | 322.95 ms | 52.3% bf16 MFU | 1623754 tok/s
step  737/19560 | loss 4.756208 (-1.90z)| norm 0.7648 (+0.21z)| lr 6.00e-04 | 322.41 ms | 52.3% bf16 MFU | 1623873 tok/s
step  738/19560 | loss 4.742828 (-1.99z)| norm 0.7986 (+0.45z)| lr 6.00e-04 | 323.28 ms | 52.2% bf16 MFU | 1623770 tok/s
step  739/19560 | loss 4.790648 (-1.51z)| norm 0.7069 (-0.17z)| lr 6.00e-04 | 322.89 ms | 52.3% bf16 MFU | 1623768 tok/s
step  740/19560 | loss 4.737453 (-1.98z)| norm 0.6920 (-0.27z)| lr 6.00e-04 | 322.82 ms | 52.3% bf16 MFU | 1623784 tok/s
step  741/19560 | loss 4.776419 (-1.58z)| norm 0.7704 (+0.26z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1623793 tok/s
step  742/19560 | loss 4.752944 (-1.77z)| norm 0.8000 (+0.46z)| lr 6.00e-04 | 322.41 ms | 52.3% bf16 MFU | 1623910 tok/s
step  743/19560 | loss 4.743009 (-1.83z)| norm 0.7097 (-0.15z)| lr 6.00e-04 | 323.44 ms | 52.2% bf16 MFU | 1623764 tok/s
step  744/19560 | loss 4.751386 (-1.71z)| norm 0.6410 (-0.61z)| lr 6.00e-04 | 323.15 ms | 52.2% bf16 MFU | 1623696 tok/s
step  745/19560 | loss 4.764440 (-1.56z)| norm 0.7278 (-0.03z)| lr 6.00e-04 | 322.92 ms | 52.3% bf16 MFU | 1623690 tok/s
step  746/19560 | loss 4.767522 (-1.51z)| norm 0.8383 (+0.72z)| lr 6.00e-04 | 322.24 ms | 52.4% bf16 MFU | 1623856 tok/s
step  747/19560 | loss 4.795957 (-1.23z)| norm 0.8131 (+0.55z)| lr 6.00e-04 | 323.16 ms | 52.2% bf16 MFU | 1623782 tok/s
step  748/19560 | loss 4.753600 (-1.60z)| norm 0.7053 (-0.18z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1623832 tok/s
step  749/19560 | loss 4.709216 (-1.96z)| norm 0.6796 (-0.35z)| lr 6.00e-04 | 322.79 ms | 52.3% bf16 MFU | 1623853 tok/s
step  750/19560 | loss 4.724714 (-1.78z)| norm 0.7019 (-0.20z)| lr 6.00e-04 | 323.29 ms | 52.2% bf16 MFU | 1623748 tok/s
val loss 4.731918
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2504/10042 = 0.249353
step  751/19560 | loss 4.750709 (-1.52z)| norm 0.7485 (+0.12z)| lr 6.00e-04 | 321.27 ms | 52.5% bf16 MFU | 1624155 tok/s
step  752/19560 | loss 4.758900 (-1.42z)| norm 0.7801 (+0.32z)| lr 6.00e-04 | 321.99 ms | 52.4% bf16 MFU | 1624361 tok/s
step  753/19560 | loss 4.810132 (-0.94z)| norm 0.6522 (-0.56z)| lr 6.00e-04 | 322.75 ms | 52.3% bf16 MFU | 1624366 tok/s
step  754/19560 | loss 4.756321 (-1.40z)| norm 0.5733 (-1.10z)| lr 6.00e-04 | 322.55 ms | 52.3% bf16 MFU | 1624421 tok/s
step  755/19560 | loss 4.724876 (-1.66z)| norm 0.5487 (-1.26z)| lr 6.00e-04 | 322.32 ms | 52.4% bf16 MFU | 1624530 tok/s
step  756/19560 | loss 4.702960 (-1.82z)| norm 0.4971 (-1.59z)| lr 6.00e-04 | 322.16 ms | 52.4% bf16 MFU | 1624674 tok/s
step  757/19560 | loss 4.733330 (-1.52z)| norm 0.4862 (-1.64z)| lr 6.00e-04 | 322.39 ms | 52.4% bf16 MFU | 1624754 tok/s
step  758/19560 | loss 4.826579 (-0.68z)| norm 0.5437 (-1.25z)| lr 6.00e-04 | 322.48 ms | 52.3% bf16 MFU | 1624806 tok/s
step  759/19560 | loss 4.734684 (-1.48z)| norm 0.8831 (+1.01z)| lr 6.00e-04 | 322.65 ms | 52.3% bf16 MFU | 1624813 tok/s
step  760/19560 | loss 4.668581 (-2.02z)| norm 0.6775 (-0.36z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1624811 tok/s
step  761/19560 | loss 4.682612 (-1.86z)| norm 0.8361 (+0.71z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1624786 tok/s
step  762/19560 | loss 4.712783 (-1.58z)| norm 0.8224 (+0.67z)| lr 6.00e-04 | 322.26 ms | 52.4% bf16 MFU | 1624892 tok/s
step  763/19560 | loss 4.757513 (-1.16z)| norm 0.7292 (+0.02z)| lr 6.00e-04 | 322.59 ms | 52.3% bf16 MFU | 1624909 tok/s
step  764/19560 | loss 4.650088 (-2.07z)| norm 0.5842 (-0.99z)| lr 6.00e-04 | 322.71 ms | 52.3% bf16 MFU | 1624894 tok/s
step  765/19560 | loss 4.698924 (-1.61z)| norm 0.6550 (-0.50z)| lr 6.00e-04 | 322.40 ms | 52.3% bf16 MFU | 1624961 tok/s
step  766/19560 | loss 4.732685 (-1.29z)| norm 0.7272 (-0.00z)| lr 6.00e-04 | 322.38 ms | 52.4% bf16 MFU | 1625027 tok/s
step  767/19560 | loss 4.713754 (-1.44z)| norm 0.8933 (+1.14z)| lr 6.00e-04 | 322.44 ms | 52.3% bf16 MFU | 1625075 tok/s
step  768/19560 | loss 4.704225 (-1.50z)| norm 1.0007 (+1.86z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1625071 tok/s
step  769/19560 | loss 4.752849 (-1.06z)| norm 0.9058 (+1.22z)| lr 6.00e-04 | 322.56 ms | 52.3% bf16 MFU | 1625086 tok/s
step  770/19560 | loss 4.725540 (-1.27z)| norm 0.8220 (+0.66z)| lr 6.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1625090 tok/s
step  771/19560 | loss 4.686444 (-1.59z)| norm 0.6726 (-0.38z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1625075 tok/s
step  772/19560 | loss 4.723881 (-1.25z)| norm 0.5360 (-1.33z)| lr 6.00e-04 | 322.72 ms | 52.3% bf16 MFU | 1625052 tok/s
step  773/19560 | loss 4.645856 (-1.89z)| norm 0.4452 (-1.93z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1624998 tok/s
step  774/19560 | loss 4.724556 (-1.19z)| norm 0.4756 (-1.68z)| lr 6.00e-04 | 323.19 ms | 52.2% bf16 MFU | 1624860 tok/s
step  775/19560 | loss 4.713929 (-1.26z)| norm 0.4888 (-1.56z)| lr 6.00e-04 | 322.13 ms | 52.4% bf16 MFU | 1624996 tok/s
step  776/19560 | loss 4.669775 (-1.62z)| norm 0.6016 (-0.79z)| lr 6.00e-04 | 323.01 ms | 52.2% bf16 MFU | 1624903 tok/s
step  777/19560 | loss 4.708864 (-1.26z)| norm 0.8484 (+0.87z)| lr 6.00e-04 | 322.53 ms | 52.3% bf16 MFU | 1624936 tok/s
step  778/19560 | loss 4.784009 (-0.59z)| norm 0.8905 (+1.14z)| lr 6.00e-04 | 323.03 ms | 52.2% bf16 MFU | 1624842 tok/s
step  779/19560 | loss 4.701509 (-1.29z)| norm 0.7122 (-0.06z)| lr 6.00e-04 | 322.16 ms | 52.4% bf16 MFU | 1624971 tok/s
step  780/19560 | loss 4.695317 (-1.32z)| norm 0.6949 (-0.17z)| lr 6.00e-04 | 322.20 ms | 52.4% bf16 MFU | 1625083 tok/s
step  781/19560 | loss 4.646690 (-1.72z)| norm 0.5945 (-0.84z)| lr 6.00e-04 | 322.75 ms | 52.3% bf16 MFU | 1625051 tok/s
step  782/19560 | loss 4.670248 (-1.49z)| norm 0.5197 (-1.33z)| lr 6.00e-04 | 322.22 ms | 52.4% bf16 MFU | 1625154 tok/s
step  783/19560 | loss 4.641622 (-1.70z)| norm 0.6750 (-0.27z)| lr 6.00e-04 | 323.39 ms | 52.2% bf16 MFU | 1624958 tok/s
step  784/19560 | loss 4.671340 (-1.42z)| norm 0.6059 (-0.73z)| lr 6.00e-04 | 322.14 ms | 52.4% bf16 MFU | 1625087 tok/s
step  785/19560 | loss 4.662946 (-1.47z)| norm 0.6321 (-0.56z)| lr 6.00e-04 | 322.96 ms | 52.3% bf16 MFU | 1625002 tok/s
step  786/19560 | loss 4.606144 (-1.91z)| norm 0.6847 (-0.21z)| lr 6.00e-04 | 323.17 ms | 52.2% bf16 MFU | 1624870 tok/s
step  787/19560 | loss 4.646077 (-1.55z)| norm 0.7391 (+0.16z)| lr 6.00e-04 | 322.48 ms | 52.3% bf16 MFU | 1624916 tok/s
step  788/19560 | loss 4.676457 (-1.28z)| norm 0.6007 (-0.79z)| lr 6.00e-04 | 322.98 ms | 52.3% bf16 MFU | 1624836 tok/s
step  789/19560 | loss 4.627562 (-1.73z)| norm 0.6312 (-0.57z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1624834 tok/s
step  790/19560 | loss 4.681332 (-1.23z)| norm 0.6403 (-0.50z)| lr 6.00e-04 | 322.39 ms | 52.4% bf16 MFU | 1624906 tok/s
step  791/19560 | loss 4.702680 (-1.03z)| norm 0.6070 (-0.72z)| lr 6.00e-04 | 323.07 ms | 52.2% bf16 MFU | 1624801 tok/s
step  792/19560 | loss 4.603000 (-1.89z)| norm 0.6346 (-0.52z)| lr 6.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1624829 tok/s
step  793/19560 | loss 4.656129 (-1.41z)| norm 0.5581 (-1.05z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1624848 tok/s
step  794/19560 | loss 4.586333 (-2.00z)| norm 0.5650 (-0.98z)| lr 6.00e-04 | 323.13 ms | 52.2% bf16 MFU | 1624732 tok/s
step  795/19560 | loss 4.649553 (-1.41z)| norm 0.5602 (-1.00z)| lr 6.00e-04 | 322.01 ms | 52.4% bf16 MFU | 1624904 tok/s
step  796/19560 | loss 4.658714 (-1.31z)| norm 0.5545 (-1.04z)| lr 6.00e-04 | 323.06 ms | 52.2% bf16 MFU | 1624802 tok/s
step  797/19560 | loss 4.588563 (-1.92z)| norm 0.5655 (-0.95z)| lr 6.00e-04 | 322.88 ms | 52.3% bf16 MFU | 1624751 tok/s
step  798/19560 | loss 4.658260 (-1.26z)| norm 0.5544 (-1.02z)| lr 6.00e-04 | 322.73 ms | 52.3% bf16 MFU | 1624740 tok/s
step  799/19560 | loss 4.592325 (-1.84z)| norm 0.5368 (-1.13z)| lr 6.00e-04 | 322.79 ms | 52.3% bf16 MFU | 1624716 tok/s
step  800/19560 | loss 4.567824 (-2.03z)| norm 0.4869 (-1.52z)| lr 6.00e-04 | 322.42 ms | 52.3% bf16 MFU | 1624786 tok/s
step  801/19560 | loss 4.609706 (-1.63z)| norm 0.5433 (-1.07z)| lr 6.00e-04 | 322.48 ms | 52.3% bf16 MFU | 1624837 tok/s
step  802/19560 | loss 4.573147 (-1.94z)| norm 0.5992 (-0.62z)| lr 6.00e-04 | 323.09 ms | 52.2% bf16 MFU | 1624732 tok/s
step  803/19560 | loss 4.572931 (-1.91z)| norm 0.5501 (-0.99z)| lr 6.00e-04 | 322.59 ms | 52.3% bf16 MFU | 1624758 tok/s
step  804/19560 | loss 4.718993 (-0.55z)| norm 0.5840 (-0.72z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624741 tok/s
step  805/19560 | loss 4.594267 (-1.70z)| norm 0.5874 (-0.68z)| lr 6.00e-04 | 322.03 ms | 52.4% bf16 MFU | 1624907 tok/s
step  806/19560 | loss 4.601640 (-1.61z)| norm 0.5890 (-0.66z)| lr 6.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1624920 tok/s
step  807/19560 | loss 4.602854 (-1.57z)| norm 0.5708 (-0.80z)| lr 6.00e-04 | 323.64 ms | 52.1% bf16 MFU | 1624674 tok/s
step  808/19560 | loss 4.610412 (-1.48z)| norm 0.7077 (+0.27z)| lr 6.00e-04 | 322.40 ms | 52.3% bf16 MFU | 1624750 tok/s
step  809/19560 | loss 4.604239 (-1.51z)| norm 0.7287 (+0.43z)| lr 6.00e-04 | 322.89 ms | 52.3% bf16 MFU | 1624700 tok/s
step  810/19560 | loss 4.546172 (-2.02z)| norm 0.5808 (-0.75z)| lr 6.00e-04 | 323.09 ms | 52.2% bf16 MFU | 1624601 tok/s
step  811/19560 | loss 4.540331 (-2.02z)| norm 0.5491 (-1.01z)| lr 6.00e-04 | 322.53 ms | 52.3% bf16 MFU | 1624647 tok/s
step  812/19560 | loss 4.581498 (-1.62z)| norm 0.5331 (-1.14z)| lr 6.00e-04 | 322.36 ms | 52.4% bf16 MFU | 1624734 tok/s
step  813/19560 | loss 4.624102 (-1.21z)| norm 0.5494 (-1.00z)| lr 6.00e-04 | 322.88 ms | 52.3% bf16 MFU | 1624687 tok/s
step  814/19560 | loss 4.606969 (-1.35z)| norm 0.5397 (-1.07z)| lr 6.00e-04 | 323.03 ms | 52.2% bf16 MFU | 1624603 tok/s
step  815/19560 | loss 4.528615 (-2.04z)| norm 0.5090 (-1.30z)| lr 6.00e-04 | 322.43 ms | 52.3% bf16 MFU | 1624676 tok/s
step  816/19560 | loss 4.601659 (-1.35z)| norm 0.5698 (-0.79z)| lr 6.00e-04 | 322.40 ms | 52.3% bf16 MFU | 1624752 tok/s
step  817/19560 | loss 4.568259 (-1.63z)| norm 0.5680 (-0.80z)| lr 6.00e-04 | 323.40 ms | 52.2% bf16 MFU | 1624574 tok/s
step  818/19560 | loss 4.580789 (-1.49z)| norm 0.6613 (-0.03z)| lr 6.00e-04 | 322.47 ms | 52.3% bf16 MFU | 1624637 tok/s
step  819/19560 | loss 4.603157 (-1.27z)| norm 0.6944 (+0.24z)| lr 6.00e-04 | 322.09 ms | 52.4% bf16 MFU | 1624793 tok/s
step  820/19560 | loss 4.594363 (-1.34z)| norm 0.6182 (-0.38z)| lr 6.00e-04 | 323.65 ms | 52.1% bf16 MFU | 1624548 tok/s
step  821/19560 | loss 4.527572 (-1.93z)| norm 0.5325 (-1.07z)| lr 6.00e-04 | 322.99 ms | 52.3% bf16 MFU | 1624482 tok/s
step  822/19560 | loss 4.554987 (-1.65z)| norm 0.5277 (-1.10z)| lr 6.00e-04 | 322.54 ms | 52.3% bf16 MFU | 1624533 tok/s
step  823/19560 | loss 4.559875 (-1.58z)| norm 0.5441 (-0.95z)| lr 6.00e-04 | 323.12 ms | 52.2% bf16 MFU | 1624435 tok/s
step  824/19560 | loss 4.542778 (-1.71z)| norm 0.6095 (-0.42z)| lr 6.00e-04 | 322.88 ms | 52.3% bf16 MFU | 1624403 tok/s
step  825/19560 | loss 4.549355 (-1.62z)| norm 0.5806 (-0.65z)| lr 6.00e-04 | 322.92 ms | 52.3% bf16 MFU | 1624361 tok/s
step  826/19560 | loss 4.546217 (-1.63z)| norm 0.4836 (-1.40z)| lr 6.00e-04 | 322.83 ms | 52.3% bf16 MFU | 1624345 tok/s
step  827/19560 | loss 4.642519 (-0.71z)| norm 0.5038 (-1.22z)| lr 6.00e-04 | 322.52 ms | 52.3% bf16 MFU | 1624406 tok/s
step  828/19560 | loss 4.553758 (-1.53z)| norm 0.5305 (-0.99z)| lr 6.00e-04 | 322.56 ms | 52.3% bf16 MFU | 1624455 tok/s
step  829/19560 | loss 4.534232 (-1.70z)| norm 0.4886 (-1.33z)| lr 6.00e-04 | 322.88 ms | 52.3% bf16 MFU | 1624420 tok/s
step  830/19560 | loss 4.507692 (-1.93z)| norm 0.4488 (-1.65z)| lr 6.00e-04 | 322.81 ms | 52.3% bf16 MFU | 1624406 tok/s
step  831/19560 | loss 4.556407 (-1.45z)| norm 0.4433 (-1.66z)| lr 6.00e-04 | 323.26 ms | 52.2% bf16 MFU | 1624281 tok/s
step  832/19560 | loss 4.568287 (-1.32z)| norm 0.4338 (-1.70z)| lr 6.00e-04 | 322.10 ms | 52.4% bf16 MFU | 1624452 tok/s
step  833/19560 | loss 4.608150 (-0.91z)| norm 0.5693 (-0.60z)| lr 6.00e-04 | 323.35 ms | 52.2% bf16 MFU | 1624301 tok/s
step  834/19560 | loss 4.575103 (-1.22z)| norm 0.7558 (+0.91z)| lr 6.00e-04 | 323.01 ms | 52.2% bf16 MFU | 1624242 tok/s
step  835/19560 | loss 4.559517 (-1.36z)| norm 0.7067 (+0.50z)| lr 6.00e-04 | 322.59 ms | 52.3% bf16 MFU | 1624293 tok/s
step  836/19560 | loss 4.593911 (-1.01z)| norm 0.6912 (+0.37z)| lr 6.00e-04 | 323.06 ms | 52.2% bf16 MFU | 1624223 tok/s
step  837/19560 | loss 4.529801 (-1.63z)| norm 0.7482 (+0.83z)| lr 6.00e-04 | 322.93 ms | 52.3% bf16 MFU | 1624188 tok/s
step  838/19560 | loss 4.556854 (-1.34z)| norm 0.6626 (+0.14z)| lr 6.00e-04 | 322.44 ms | 52.3% bf16 MFU | 1624278 tok/s
step  839/19560 | loss 4.727320 (+0.42z)| norm 0.6642 (+0.15z)| lr 6.00e-04 | 322.69 ms | 52.3% bf16 MFU | 1624300 tok/s
step  840/19560 | loss 4.547984 (-1.42z)| norm 0.5160 (-1.04z)| lr 6.00e-04 | 322.87 ms | 52.3% bf16 MFU | 1624278 tok/s
step  841/19560 | loss 4.543496 (-1.44z)| norm 0.4255 (-1.74z)| lr 6.00e-04 | 323.86 ms | 52.1% bf16 MFU | 1624007 tok/s
step  842/19560 | loss 4.604276 (-0.80z)| norm 0.4637 (-1.41z)| lr 6.00e-04 | 323.09 ms | 52.2% bf16 MFU | 1623944 tok/s
step  843/19560 | loss 4.533867 (-1.51z)| norm 0.4794 (-1.28z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1623987 tok/s
step  844/19560 | loss 4.484531 (-1.98z)| norm 0.4853 (-1.22z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1624028 tok/s
step  845/19560 | loss 4.554555 (-1.24z)| norm 0.4964 (-1.11z)| lr 6.00e-04 | 322.93 ms | 52.3% bf16 MFU | 1624002 tok/s
step  846/19560 | loss 4.527771 (-1.49z)| norm 0.6121 (-0.16z)| lr 6.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1624070 tok/s
step  847/19560 | loss 4.514639 (-1.61z)| norm 0.6127 (-0.15z)| lr 6.00e-04 | 323.11 ms | 52.2% bf16 MFU | 1623997 tok/s
step  848/19560 | loss 4.520532 (-1.54z)| norm 0.5448 (-0.69z)| lr 6.00e-04 | 322.71 ms | 52.3% bf16 MFU | 1624030 tok/s
step  849/19560 | loss 4.551583 (-1.20z)| norm 0.5476 (-0.66z)| lr 6.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1624086 tok/s
step  850/19560 | loss 4.454568 (-2.21z)| norm 0.5909 (-0.29z)| lr 6.00e-04 | 323.57 ms | 52.2% bf16 MFU | 1623898 tok/s
step  851/19560 | loss 4.559063 (-1.08z)| norm 0.6485 (+0.21z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1623937 tok/s
step  852/19560 | loss 4.474381 (-1.96z)| norm 0.6051 (-0.14z)| lr 6.00e-04 | 323.12 ms | 52.2% bf16 MFU | 1623868 tok/s
step  853/19560 | loss 4.519273 (-1.45z)| norm 0.5834 (-0.32z)| lr 6.00e-04 | 323.18 ms | 52.2% bf16 MFU | 1623788 tok/s
step  854/19560 | loss 4.499451 (-1.64z)| norm 0.5967 (-0.21z)| lr 6.00e-04 | 322.65 ms | 52.3% bf16 MFU | 1623845 tok/s
step  855/19560 | loss 4.513379 (-1.48z)| norm 0.5701 (-0.44z)| lr 6.00e-04 | 323.33 ms | 52.2% bf16 MFU | 1623729 tok/s
step  856/19560 | loss 4.469506 (-1.93z)| norm 0.5184 (-0.89z)| lr 6.00e-04 | 322.75 ms | 52.3% bf16 MFU | 1623766 tok/s
step  857/19560 | loss 4.516726 (-1.39z)| norm 0.4881 (-1.15z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1623838 tok/s
step  858/19560 | loss 4.501809 (-1.53z)| norm 0.4874 (-1.16z)| lr 6.00e-04 | 323.67 ms | 52.1% bf16 MFU | 1623636 tok/s
step  859/19560 | loss 4.596187 (-0.48z)| norm 0.5102 (-0.96z)| lr 6.00e-04 | 322.58 ms | 52.3% bf16 MFU | 1623720 tok/s
step  860/19560 | loss 4.523275 (-1.27z)| norm 0.5262 (-0.82z)| lr 6.00e-04 | 322.98 ms | 52.3% bf16 MFU | 1623699 tok/s
step  861/19560 | loss 4.548597 (-0.98z)| norm 0.5660 (-0.48z)| lr 6.00e-04 | 323.28 ms | 52.2% bf16 MFU | 1623602 tok/s
step  862/19560 | loss 4.460073 (-1.92z)| norm 0.5790 (-0.37z)| lr 6.00e-04 | 323.00 ms | 52.3% bf16 MFU | 1623581 tok/s
step  863/19560 | loss 4.587985 (-0.50z)| norm 0.6137 (-0.06z)| lr 6.00e-04 | 323.23 ms | 52.2% bf16 MFU | 1623505 tok/s
step  864/19560 | loss 4.545689 (-0.95z)| norm 0.7424 (+1.04z)| lr 6.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1623554 tok/s
step  865/19560 | loss 4.530947 (-1.10z)| norm 0.6848 (+0.55z)| lr 6.00e-04 | 323.12 ms | 52.2% bf16 MFU | 1623506 tok/s
step  866/19560 | loss 4.529347 (-1.10z)| norm 0.7610 (+1.22z)| lr 6.00e-04 | 323.10 ms | 52.2% bf16 MFU | 1623464 tok/s
step  867/19560 | loss 4.592486 (-0.38z)| norm 0.6393 (+0.17z)| lr 6.00e-04 | 322.45 ms | 52.3% bf16 MFU | 1623589 tok/s
step  868/19560 | loss 4.479220 (-1.63z)| norm 0.5195 (-0.87z)| lr 6.00e-04 | 322.59 ms | 52.3% bf16 MFU | 1623672 tok/s
step  869/19560 | loss 4.501421 (-1.37z)| norm 0.4768 (-1.22z)| lr 6.00e-04 | 323.61 ms | 52.2% bf16 MFU | 1623494 tok/s
step  870/19560 | loss 4.479470 (-1.59z)| norm 0.4399 (-1.52z)| lr 6.00e-04 | 323.09 ms | 52.2% bf16 MFU | 1623455 tok/s
step  871/19560 | loss 4.455842 (-1.82z)| norm 0.3992 (-1.84z)| lr 6.00e-04 | 322.96 ms | 52.3% bf16 MFU | 1623450 tok/s
step  872/19560 | loss 4.486840 (-1.45z)| norm 0.3829 (-1.94z)| lr 6.00e-04 | 323.38 ms | 52.2% bf16 MFU | 1623341 tok/s
step  873/19560 | loss 4.453401 (-1.79z)| norm 0.3922 (-1.82z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1623373 tok/s
step  874/19560 | loss 4.377589 (-2.57z)| norm 0.4649 (-1.19z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1623404 tok/s
step  875/19560 | loss 4.456231 (-1.68z)| norm 0.4779 (-1.06z)| lr 6.00e-04 | 323.26 ms | 52.2% bf16 MFU | 1623329 tok/s
step  876/19560 | loss 4.519008 (-0.97z)| norm 0.4777 (-1.05z)| lr 6.00e-04 | 323.23 ms | 52.2% bf16 MFU | 1623264 tok/s
step  877/19560 | loss 4.517462 (-0.97z)| norm 0.4947 (-0.89z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1623300 tok/s
step  878/19560 | loss 4.509140 (-1.05z)| norm 0.5601 (-0.32z)| lr 6.00e-04 | 322.77 ms | 52.3% bf16 MFU | 1623352 tok/s
step  879/19560 | loss 4.636855 (+0.40z)| norm 0.5959 (+0.00z)| lr 6.00e-04 | 322.52 ms | 52.3% bf16 MFU | 1623465 tok/s
step  880/19560 | loss 4.536485 (-0.73z)| norm 0.8146 (+1.89z)| lr 6.00e-04 | 323.67 ms | 52.1% bf16 MFU | 1623284 tok/s
step  881/19560 | loss 4.504397 (-1.09z)| norm 0.9071 (+2.60z)| lr 6.00e-04 | 322.63 ms | 52.3% bf16 MFU | 1623373 tok/s
step  882/19560 | loss 4.567525 (-0.34z)| norm 0.7612 (+1.35z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1623453 tok/s
step  883/19560 | loss 4.501540 (-1.11z)| norm 0.6687 (+0.57z)| lr 6.00e-04 | 322.49 ms | 52.3% bf16 MFU | 1623568 tok/s
step  884/19560 | loss 4.465900 (-1.50z)| norm 0.5694 (-0.26z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1623604 tok/s
step  885/19560 | loss 4.508203 (-0.99z)| norm 0.5253 (-0.64z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1623658 tok/s
step  886/19560 | loss 4.474409 (-1.39z)| norm 0.5023 (-0.83z)| lr 6.00e-04 | 323.07 ms | 52.2% bf16 MFU | 1623617 tok/s
step  887/19560 | loss 4.518575 (-0.84z)| norm 0.4296 (-1.43z)| lr 6.00e-04 | 323.15 ms | 52.2% bf16 MFU | 1623557 tok/s
step  888/19560 | loss 4.425880 (-1.94z)| norm 0.4267 (-1.43z)| lr 6.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1623647 tok/s
step  889/19560 | loss 4.480249 (-1.26z)| norm 0.4374 (-1.32z)| lr 6.00e-04 | 323.53 ms | 52.2% bf16 MFU | 1623491 tok/s
step  890/19560 | loss 4.411825 (-2.05z)| norm 0.4264 (-1.40z)| lr 6.00e-04 | 322.05 ms | 52.4% bf16 MFU | 1623714 tok/s
step  891/19560 | loss 4.487944 (-1.12z)| norm 0.3899 (-1.68z)| lr 6.00e-04 | 323.27 ms | 52.2% bf16 MFU | 1623618 tok/s
step  892/19560 | loss 4.467730 (-1.34z)| norm 0.3721 (-1.80z)| lr 6.00e-04 | 322.81 ms | 52.3% bf16 MFU | 1623644 tok/s
step  893/19560 | loss 4.387103 (-2.27z)| norm 0.3451 (-1.98z)| lr 6.00e-04 | 323.55 ms | 52.2% bf16 MFU | 1623484 tok/s
step  894/19560 | loss 4.432099 (-1.70z)| norm 0.4041 (-1.46z)| lr 6.00e-04 | 323.25 ms | 52.2% bf16 MFU | 1623407 tok/s
step  895/19560 | loss 4.486924 (-1.02z)| norm 0.4702 (-0.91z)| lr 6.00e-04 | 322.47 ms | 52.3% bf16 MFU | 1623528 tok/s
step  896/19560 | loss 4.572297 (+0.03z)| norm 0.5171 (-0.50z)| lr 6.00e-04 | 323.38 ms | 52.2% bf16 MFU | 1623416 tok/s
step  897/19560 | loss 4.494519 (-0.92z)| norm 0.6089 (+0.35z)| lr 6.00e-04 | 323.24 ms | 52.2% bf16 MFU | 1623343 tok/s
step  898/19560 | loss 4.444746 (-1.53z)| norm 0.6813 (+1.06z)| lr 6.00e-04 | 322.90 ms | 52.3% bf16 MFU | 1623361 tok/s
step  899/19560 | loss 4.427084 (-1.72z)| norm 0.6165 (+0.45z)| lr 6.00e-04 | 323.71 ms | 52.1% bf16 MFU | 1623174 tok/s
step  900/19560 | loss 4.448463 (-1.44z)| norm 0.6340 (+0.61z)| lr 6.00e-04 | 322.32 ms | 52.4% bf16 MFU | 1623346 tok/s
step  901/19560 | loss 4.493427 (-0.85z)| norm 0.6015 (+0.29z)| lr 6.00e-04 | 322.99 ms | 52.3% bf16 MFU | 1623341 tok/s
step  902/19560 | loss 4.491237 (-0.87z)| norm 0.5868 (+0.14z)| lr 6.00e-04 | 323.35 ms | 52.2% bf16 MFU | 1623244 tok/s
step  903/19560 | loss 4.455154 (-1.32z)| norm 0.5297 (-0.41z)| lr 6.00e-04 | 323.04 ms | 52.2% bf16 MFU | 1623232 tok/s
step  904/19560 | loss 4.451303 (-1.35z)| norm 0.4985 (-0.70z)| lr 6.00e-04 | 323.38 ms | 52.2% bf16 MFU | 1623135 tok/s
step  905/19560 | loss 4.391316 (-2.10z)| norm 0.5159 (-0.52z)| lr 6.00e-04 | 322.32 ms | 52.4% bf16 MFU | 1623308 tok/s
step  906/19560 | loss 4.433712 (-1.56z)| norm 0.4424 (-1.25z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1623402 tok/s
step  907/19560 | loss 4.450596 (-1.31z)| norm 0.4416 (-1.24z)| lr 6.00e-04 | 323.03 ms | 52.2% bf16 MFU | 1623382 tok/s
step  908/19560 | loss 4.391743 (-2.08z)| norm 0.4435 (-1.20z)| lr 6.00e-04 | 323.16 ms | 52.2% bf16 MFU | 1623332 tok/s
step  909/19560 | loss 4.467037 (-1.04z)| norm 0.4097 (-1.52z)| lr 6.00e-04 | 322.34 ms | 52.4% bf16 MFU | 1623491 tok/s
step  910/19560 | loss 4.426228 (-1.58z)| norm 0.4409 (-1.19z)| lr 6.00e-04 | 322.69 ms | 52.3% bf16 MFU | 1623553 tok/s
step  911/19560 | loss 4.384405 (-2.10z)| norm 0.4533 (-1.05z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1623635 tok/s
step  912/19560 | loss 4.432291 (-1.43z)| norm 0.5030 (-0.54z)| lr 6.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1623678 tok/s
step  913/19560 | loss 4.387957 (-2.00z)| norm 0.5230 (-0.33z)| lr 6.00e-04 | 322.80 ms | 52.3% bf16 MFU | 1623704 tok/s
step  914/19560 | loss 4.421427 (-1.52z)| norm 0.5007 (-0.54z)| lr 6.00e-04 | 322.62 ms | 52.3% bf16 MFU | 1623774 tok/s
step  915/19560 | loss 4.427155 (-1.42z)| norm 0.4845 (-0.70z)| lr 6.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1623852 tok/s
step  916/19560 | loss 4.426624 (-1.41z)| norm 0.4647 (-0.89z)| lr 6.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1623917 tok/s
step  917/19560 | loss 4.364565 (-2.19z)| norm 0.4699 (-0.82z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1623920 tok/s
step  918/19560 | loss 4.415363 (-1.50z)| norm 0.5083 (-0.42z)| lr 6.00e-04 | 322.94 ms | 52.3% bf16 MFU | 1623899 tok/s
step  919/19560 | loss 4.372352 (-2.06z)| norm 0.5103 (-0.39z)| lr 6.00e-04 | 322.46 ms | 52.3% bf16 MFU | 1624000 tok/s
step  920/19560 | loss 4.485586 (-0.51z)| norm 0.5489 (+0.01z)| lr 6.00e-04 | 322.81 ms | 52.3% bf16 MFU | 1624006 tok/s
step  921/19560 | loss 4.420200 (-1.38z)| norm 0.5793 (+0.32z)| lr 6.00e-04 | 322.91 ms | 52.3% bf16 MFU | 1623986 tok/s
step  922/19560 | loss 4.448822 (-0.97z)| norm 0.5836 (+0.37z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1624028 tok/s
step  923/19560 | loss 4.456485 (-0.86z)| norm 0.5008 (-0.48z)| lr 6.00e-04 | 322.51 ms | 52.3% bf16 MFU | 1624110 tok/s
step  924/19560 | loss 4.425251 (-1.27z)| norm 0.4745 (-0.74z)| lr 6.00e-04 | 322.33 ms | 52.4% bf16 MFU | 1624232 tok/s
step  925/19560 | loss 4.515070 (-0.01z)| norm 0.4778 (-0.70z)| lr 6.00e-04 | 322.97 ms | 52.3% bf16 MFU | 1624186 tok/s
step  926/19560 | loss 4.410030 (-1.47z)| norm 0.5401 (-0.06z)| lr 6.00e-04 | 322.90 ms | 52.3% bf16 MFU | 1624160 tok/s
step  927/19560 | loss 4.404501 (-1.52z)| norm 0.4812 (-0.66z)| lr 6.00e-04 | 322.58 ms | 52.3% bf16 MFU | 1624217 tok/s
step  928/19560 | loss 4.431636 (-1.12z)| norm 0.4799 (-0.67z)| lr 6.00e-04 | 322.55 ms | 52.3% bf16 MFU | 1624280 tok/s
step  929/19560 | loss 4.576412 (+0.92z)| norm 0.6022 (+0.57z)| lr 6.00e-04 | 322.36 ms | 52.4% bf16 MFU | 1624385 tok/s
step  930/19560 | loss 4.432632 (-1.09z)| norm 0.6696 (+1.25z)| lr 6.00e-04 | 323.04 ms | 52.2% bf16 MFU | 1624315 tok/s
step  931/19560 | loss 4.451801 (-0.81z)| norm 0.6462 (+1.00z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624320 tok/s
step  932/19560 | loss 4.421003 (-1.24z)| norm 0.5632 (+0.16z)| lr 6.00e-04 | 323.01 ms | 52.2% bf16 MFU | 1624260 tok/s
step  933/19560 | loss 4.415336 (-1.30z)| norm 0.4687 (-0.78z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1624261 tok/s
step  934/19560 | loss 4.434638 (-1.01z)| norm 0.4074 (-1.38z)| lr 6.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1624242 tok/s
step  935/19560 | loss 4.375488 (-1.83z)| norm 0.4156 (-1.28z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1624245 tok/s
step  936/19560 | loss 4.456660 (-0.65z)| norm 0.4230 (-1.19z)| lr 6.00e-04 | 322.85 ms | 52.3% bf16 MFU | 1624230 tok/s
step  937/19560 | loss 4.417194 (-1.21z)| norm 0.5059 (-0.34z)| lr 6.00e-04 | 322.22 ms | 52.4% bf16 MFU | 1624374 tok/s
step  938/19560 | loss 4.459547 (-0.58z)| norm 0.5765 (+0.38z)| lr 6.00e-04 | 322.85 ms | 52.3% bf16 MFU | 1624352 tok/s
step  939/19560 | loss 4.408288 (-1.30z)| norm 0.5991 (+0.60z)| lr 6.00e-04 | 323.26 ms | 52.2% bf16 MFU | 1624229 tok/s
step  940/19560 | loss 4.373230 (-1.78z)| norm 0.5430 (+0.03z)| lr 6.00e-04 | 322.31 ms | 52.4% bf16 MFU | 1624350 tok/s
step  941/19560 | loss 4.379801 (-1.66z)| norm 0.5271 (-0.13z)| lr 6.00e-04 | 322.41 ms | 52.3% bf16 MFU | 1624442 tok/s
step  942/19560 | loss 4.409242 (-1.22z)| norm 0.5201 (-0.20z)| lr 6.00e-04 | 322.34 ms | 52.4% bf16 MFU | 1624544 tok/s
step  943/19560 | loss 4.446931 (-0.66z)| norm 0.4840 (-0.56z)| lr 6.00e-04 | 323.08 ms | 52.2% bf16 MFU | 1624457 tok/s
step  944/19560 | loss 4.464184 (-0.40z)| norm 0.4879 (-0.52z)| lr 6.00e-04 | 322.51 ms | 52.3% bf16 MFU | 1624517 tok/s
step  945/19560 | loss 4.401361 (-1.30z)| norm 0.4788 (-0.60z)| lr 6.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1624557 tok/s
step  946/19560 | loss 4.367875 (-1.76z)| norm 0.4487 (-0.89z)| lr 6.00e-04 | 322.85 ms | 52.3% bf16 MFU | 1624526 tok/s
step  947/19560 | loss 4.399831 (-1.27z)| norm 0.4133 (-1.24z)| lr 6.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1624526 tok/s
step  948/19560 | loss 4.397146 (-1.29z)| norm 0.4093 (-1.26z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1624533 tok/s
step  949/19560 | loss 4.422815 (-0.90z)| norm 0.4580 (-0.75z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1624565 tok/s
step  950/19560 | loss 4.474222 (-0.14z)| norm 0.4285 (-1.04z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1624572 tok/s
step  951/19560 | loss 4.389612 (-1.36z)| norm 0.4395 (-0.92z)| lr 6.00e-04 | 322.80 ms | 52.3% bf16 MFU | 1624554 tok/s
step  952/19560 | loss 4.360703 (-1.75z)| norm 0.4810 (-0.49z)| lr 6.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1624584 tok/s
step  953/19560 | loss 4.321119 (-2.26z)| norm 0.4401 (-0.89z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1624588 tok/s
step  954/19560 | loss 4.403850 (-1.06z)| norm 0.4896 (-0.39z)| lr 6.00e-04 | 322.25 ms | 52.4% bf16 MFU | 1624708 tok/s
step  955/19560 | loss 4.425549 (-0.74z)| norm 0.5754 (+0.47z)| lr 6.00e-04 | 323.27 ms | 52.2% bf16 MFU | 1624564 tok/s
step  956/19560 | loss 4.368651 (-1.55z)| norm 0.5164 (-0.13z)| lr 6.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1624559 tok/s
step  957/19560 | loss 4.351866 (-1.75z)| norm 0.4522 (-0.77z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1624591 tok/s
step  958/19560 | loss 4.359515 (-1.61z)| norm 0.4542 (-0.75z)| lr 6.00e-04 | 322.53 ms | 52.3% bf16 MFU | 1624640 tok/s
step  959/19560 | loss 4.370401 (-1.43z)| norm 0.4046 (-1.24z)| lr 6.00e-04 | 322.53 ms | 52.3% bf16 MFU | 1624686 tok/s
step  960/19560 | loss 4.415242 (-0.78z)| norm 0.4209 (-1.08z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624670 tok/s
step  961/19560 | loss 4.354748 (-1.63z)| norm 0.4342 (-0.93z)| lr 6.00e-04 | 322.32 ms | 52.4% bf16 MFU | 1624766 tok/s
step  962/19560 | loss 4.435390 (-0.45z)| norm 0.4271 (-0.99z)| lr 6.00e-04 | 323.28 ms | 52.2% bf16 MFU | 1624618 tok/s
step  963/19560 | loss 4.371108 (-1.36z)| norm 0.5329 (+0.10z)| lr 6.00e-04 | 323.13 ms | 52.2% bf16 MFU | 1624513 tok/s
step  964/19560 | loss 4.364655 (-1.44z)| norm 0.6184 (+0.99z)| lr 6.00e-04 | 322.38 ms | 52.4% bf16 MFU | 1624603 tok/s
step  965/19560 | loss 4.363682 (-1.43z)| norm 0.5986 (+0.81z)| lr 6.00e-04 | 323.00 ms | 52.3% bf16 MFU | 1624532 tok/s
step  966/19560 | loss 4.383604 (-1.12z)| norm 0.6175 (+1.02z)| lr 6.00e-04 | 322.38 ms | 52.4% bf16 MFU | 1624621 tok/s
step  967/19560 | loss 4.351465 (-1.63z)| norm 0.5749 (+0.58z)| lr 6.00e-04 | 323.03 ms | 52.2% bf16 MFU | 1624541 tok/s
step  968/19560 | loss 4.487774 (+0.47z)| norm 0.4916 (-0.31z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1624574 tok/s
step  969/19560 | loss 4.346553 (-1.68z)| norm 0.4743 (-0.50z)| lr 6.00e-04 | 322.42 ms | 52.3% bf16 MFU | 1624651 tok/s
step  970/19560 | loss 4.336484 (-1.82z)| norm 0.4271 (-1.00z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1624654 tok/s
step  971/19560 | loss 4.435225 (-0.27z)| norm 0.4410 (-0.85z)| lr 6.00e-04 | 322.48 ms | 52.3% bf16 MFU | 1624712 tok/s
step  972/19560 | loss 4.333393 (-1.83z)| norm 0.4309 (-0.95z)| lr 6.00e-04 | 322.49 ms | 52.3% bf16 MFU | 1624765 tok/s
step  973/19560 | loss 4.358710 (-1.41z)| norm 0.4381 (-0.86z)| lr 6.00e-04 | 322.25 ms | 52.4% bf16 MFU | 1624875 tok/s
step  974/19560 | loss 4.304809 (-2.19z)| norm 0.4118 (-1.12z)| lr 6.00e-04 | 323.37 ms | 52.2% bf16 MFU | 1624697 tok/s
step  975/19560 | loss 4.340498 (-1.62z)| norm 0.4034 (-1.19z)| lr 6.00e-04 | 322.85 ms | 52.3% bf16 MFU | 1624660 tok/s
step  976/19560 | loss 4.345835 (-1.51z)| norm 0.4029 (-1.18z)| lr 6.00e-04 | 322.53 ms | 52.3% bf16 MFU | 1624706 tok/s
step  977/19560 | loss 4.387686 (-0.86z)| norm 0.3893 (-1.30z)| lr 6.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1624713 tok/s
step  978/19560 | loss 4.248606 (-2.86z)| norm 0.4698 (-0.45z)| lr 6.00e-04 | 323.03 ms | 52.2% bf16 MFU | 1624628 tok/s
step  979/19560 | loss 4.306850 (-1.96z)| norm 0.5072 (-0.05z)| lr 6.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1624591 tok/s
step  980/19560 | loss 4.310771 (-1.86z)| norm 0.5280 (+0.18z)| lr 6.00e-04 | 322.54 ms | 52.3% bf16 MFU | 1624637 tok/s
step  981/19560 | loss 4.389370 (-0.71z)| norm 0.5381 (+0.29z)| lr 6.00e-04 | 323.09 ms | 52.2% bf16 MFU | 1624542 tok/s
step  982/19560 | loss 4.329943 (-1.55z)| norm 0.5678 (+0.61z)| lr 6.00e-04 | 322.96 ms | 52.3% bf16 MFU | 1624485 tok/s
step  983/19560 | loss 4.327624 (-1.55z)| norm 0.4728 (-0.39z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1624476 tok/s
step  984/19560 | loss 4.453816 (+0.26z)| norm 0.4585 (-0.54z)| lr 6.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1624508 tok/s
step  985/19560 | loss 4.333447 (-1.44z)| norm 0.4562 (-0.56z)| lr 6.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1624532 tok/s
step  986/19560 | loss 4.336009 (-1.38z)| norm 0.4903 (-0.20z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1624546 tok/s
step  987/19560 | loss 4.390206 (-0.60z)| norm 0.5948 (+0.90z)| lr 6.00e-04 | 322.46 ms | 52.3% bf16 MFU | 1624614 tok/s
step  988/19560 | loss 4.361480 (-1.00z)| norm 0.5287 (+0.20z)| lr 6.00e-04 | 322.95 ms | 52.3% bf16 MFU | 1624553 tok/s
step  989/19560 | loss 4.316251 (-1.64z)| norm 0.3977 (-1.17z)| lr 6.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1624549 tok/s
step  990/19560 | loss 4.301621 (-1.81z)| norm 0.4308 (-0.81z)| lr 6.00e-04 | 322.81 ms | 52.3% bf16 MFU | 1624528 tok/s
step  991/19560 | loss 4.359831 (-0.96z)| norm 0.4747 (-0.33z)| lr 6.00e-04 | 323.26 ms | 52.2% bf16 MFU | 1624397 tok/s
step  992/19560 | loss 4.295407 (-1.88z)| norm 0.5028 (-0.02z)| lr 6.00e-04 | 322.43 ms | 52.3% bf16 MFU | 1624480 tok/s
step  993/19560 | loss 4.308767 (-1.65z)| norm 0.4657 (-0.41z)| lr 6.00e-04 | 322.99 ms | 52.3% bf16 MFU | 1624418 tok/s
step  994/19560 | loss 4.314485 (-1.55z)| norm 0.4382 (-0.71z)| lr 6.00e-04 | 322.89 ms | 52.3% bf16 MFU | 1624384 tok/s
step  995/19560 | loss 4.326447 (-1.37z)| norm 0.4220 (-0.88z)| lr 6.00e-04 | 322.65 ms | 52.3% bf16 MFU | 1624411 tok/s
step  996/19560 | loss 4.343665 (-1.09z)| norm 0.4428 (-0.63z)| lr 6.00e-04 | 322.78 ms | 52.3% bf16 MFU | 1624406 tok/s
step  997/19560 | loss 4.299064 (-1.72z)| norm 0.4405 (-0.65z)| lr 6.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1624444 tok/s
step  998/19560 | loss 4.366317 (-0.71z)| norm 0.4143 (-0.95z)| lr 6.00e-04 | 323.32 ms | 52.2% bf16 MFU | 1624300 tok/s
step  999/19560 | loss 4.357787 (-0.83z)| norm 0.4375 (-0.69z)| lr 6.00e-04 | 322.89 ms | 52.3% bf16 MFU | 1624273 tok/s
step 1000/19560 | loss 4.334036 (-1.16z)| norm 0.3980 (-1.15z)| lr 6.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1624327 tok/s
val loss 4.327087
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2575/10042 = 0.256423
step 1001/19560 | loss 4.315507 (-1.41z)| norm 0.5049 (+0.07z)| lr 6.00e-04 | 322.35 ms | 52.4% bf16 MFU | 1624434 tok/s
step 1002/19560 | loss 4.332686 (-1.15z)| norm 0.5759 (+0.88z)| lr 6.00e-04 | 322.62 ms | 52.3% bf16 MFU | 1624468 tok/s
step 1003/19560 | loss 4.334732 (-1.10z)| norm 0.6145 (+1.31z)| lr 6.00e-04 | 322.55 ms | 52.3% bf16 MFU | 1624518 tok/s
step 1004/19560 | loss 4.358151 (-0.75z)| norm 0.6296 (+1.45z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624512 tok/s
step 1005/19560 | loss 4.355356 (-0.77z)| norm 0.4870 (-0.17z)| lr 6.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1624522 tok/s
step 1006/19560 | loss 4.254376 (-2.22z)| norm 0.4357 (-0.74z)| lr 6.00e-04 | 323.18 ms | 52.2% bf16 MFU | 1624409 tok/s
step 1007/19560 | loss 4.336263 (-1.03z)| norm 0.4272 (-0.82z)| lr 6.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1624430 tok/s
step 1008/19560 | loss 4.264089 (-2.10z)| norm 0.4253 (-0.85z)| lr 6.00e-04 | 322.56 ms | 52.3% bf16 MFU | 1624479 tok/s
step 1009/19560 | loss 4.317988 (-1.26z)| norm 0.4082 (-1.11z)| lr 6.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1624479 tok/s
step 1010/19560 | loss 4.364633 (-0.53z)| norm 0.3745 (-1.59z)| lr 6.00e-04 | 322.47 ms | 52.3% bf16 MFU | 1624546 tok/s
step 1011/19560 | loss 4.310432 (-1.37z)| norm 0.3969 (-1.27z)| lr 6.00e-04 | 323.10 ms | 52.2% bf16 MFU | 1624454 tok/s
step 1012/19560 | loss 4.334487 (-0.97z)| norm 0.4199 (-0.93z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624451 tok/s
step 1013/19560 | loss 4.319251 (-1.20z)| norm 0.4474 (-0.54z)| lr 6.00e-04 | 323.02 ms | 52.2% bf16 MFU | 1624382 tok/s
step 1014/19560 | loss 4.316617 (-1.22z)| norm 0.4709 (-0.20z)| lr 6.00e-04 | 322.65 ms | 52.3% bf16 MFU | 1624411 tok/s
step 1015/19560 | loss 4.281258 (-1.76z)| norm 0.4508 (-0.49z)| lr 6.00e-04 | 322.82 ms | 52.3% bf16 MFU | 1624394 tok/s
step 1016/19560 | loss 4.321610 (-1.10z)| norm 0.4984 (+0.18z)| lr 6.00e-04 | 322.94 ms | 52.3% bf16 MFU | 1624349 tok/s
step 1017/19560 | loss 4.318653 (-1.13z)| norm 0.5479 (+0.86z)| lr 6.00e-04 | 322.81 ms | 52.3% bf16 MFU | 1624337 tok/s
step 1018/19560 | loss 4.262259 (-1.99z)| norm 0.4924 (+0.07z)| lr 6.00e-04 | 322.94 ms | 52.3% bf16 MFU | 1624296 tok/s
step 1019/19560 | loss 4.287188 (-1.57z)| norm 0.4157 (-1.02z)| lr 6.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624300 tok/s
step 1020/19560 | loss 4.266711 (-1.85z)| norm 0.4812 (-0.10z)| lr 6.00e-04 | 322.66 ms | 52.3% bf16 MFU | 1624329 tok/s
step 1021/19560 | loss 4.342735 (-0.65z)| norm 0.4838 (-0.08z)| lr 6.00e-04 | 323.77 ms | 52.1% bf16 MFU | 1624079 tok/s
step 1022/19560 | loss 4.333303 (-0.79z)| norm 0.4454 (-0.65z)| lr 6.00e-04 | 322.82 ms | 52.3% bf16 MFU | 1624080 tok/s
step 1023/19560 | loss 4.331410 (-0.80z)| norm 0.4235 (-0.96z)| lr 6.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1624133 tok/s
step 1024/19560 | loss 4.325337 (-0.90z)| norm 0.4276 (-0.89z)| lr 6.00e-04 | 322.82 ms | 52.3% bf16 MFU | 1624131 tok/s
step 1025/19560 | loss 4.341902 (-0.61z)| norm 0.4362 (-0.75z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1624124 tok/s
step 1026/19560 | loss 4.336498 (-0.69z)| norm 0.4801 (-0.09z)| lr 6.00e-04 | 323.11 ms | 52.2% bf16 MFU | 1624048 tok/s
step 1027/19560 | loss 4.346272 (-0.52z)| norm 0.4709 (-0.21z)| lr 6.00e-04 | 322.83 ms | 52.3% bf16 MFU | 1624047 tok/s
step 1028/19560 | loss 4.263344 (-1.87z)| norm 0.3693 (-1.77z)| lr 6.00e-04 | 323.13 ms | 52.2% bf16 MFU | 1623972 tok/s
step 1029/19560 | loss 4.227961 (-2.40z)| norm 0.4241 (-0.90z)| lr 6.00e-04 | 322.83 ms | 52.3% bf16 MFU | 1623976 tok/s
step 1030/19560 | loss 4.328561 (-0.73z)| norm 0.4084 (-1.13z)| lr 6.00e-04 | 322.79 ms | 52.3% bf16 MFU | 1623989 tok/s
step 1031/19560 | loss 4.285833 (-1.42z)| norm 0.4547 (-0.39z)| lr 6.00e-04 | 322.92 ms | 52.3% bf16 MFU | 1623970 tok/s
step 1032/19560 | loss 4.272451 (-1.62z)| norm 0.4401 (-0.61z)| lr 6.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1624011 tok/s
step 1033/19560 | loss 4.352414 (-0.28z)| norm 0.3992 (-1.24z)| lr 6.00e-04 | 322.66 ms | 52.3% bf16 MFU | 1624056 tok/s
step 1034/19560 | loss 4.298450 (-1.16z)| norm 0.3842 (-1.46z)| lr 6.00e-04 | 322.32 ms | 52.4% bf16 MFU | 1624183 tok/s
step 1035/19560 | loss 4.294179 (-1.21z)| norm 0.4080 (-1.08z)| lr 6.00e-04 | 323.10 ms | 52.2% bf16 MFU | 1624109 tok/s
step 1036/19560 | loss 4.210465 (-2.52z)| norm 0.4292 (-0.75z)| lr 6.00e-04 | 323.14 ms | 52.2% bf16 MFU | 1624029 tok/s
step 1037/19560 | loss 4.293723 (-1.15z)| norm 0.4928 (+0.24z)| lr 6.00e-04 | 322.50 ms | 52.3% bf16 MFU | 1624112 tok/s
step 1038/19560 | loss 4.262871 (-1.62z)| norm 0.4634 (-0.23z)| lr 6.00e-04 | 322.99 ms | 52.3% bf16 MFU | 1624070 tok/s
step 1039/19560 | loss 4.237586 (-1.98z)| norm 0.4094 (-1.07z)| lr 6.00e-04 | 322.84 ms | 52.3% bf16 MFU | 1624064 tok/s
step 1040/19560 | loss 4.250121 (-1.75z)| norm 0.4154 (-0.96z)| lr 6.00e-04 | 323.06 ms | 52.2% bf16 MFU | 1624006 tok/s
step 1041/19560 | loss 4.346634 (-0.22z)| norm 0.4455 (-0.48z)| lr 6.00e-04 | 323.38 ms | 52.2% bf16 MFU | 1623870 tok/s
step 1042/19560 | loss 4.300263 (-0.94z)| norm 0.5610 (+1.31z)| lr 6.00e-04 | 322.73 ms | 52.3% bf16 MFU | 1623904 tok/s
step 1043/19560 | loss 4.335783 (-0.36z)| norm 0.5796 (+1.57z)| lr 6.00e-04 | 323.46 ms | 52.2% bf16 MFU | 1623752 tok/s
step 1044/19560 | loss 4.319640 (-0.61z)| norm 0.5098 (+0.49z)| lr 6.00e-04 | 322.89 ms | 52.3% bf16 MFU | 1623752 tok/s
step 1045/19560 | loss 4.341962 (-0.25z)| norm 0.4671 (-0.16z)| lr 6.00e-04 | 323.07 ms | 52.2% bf16 MFU | 1623707 tok/s
step 1046/19560 | loss 4.307075 (-0.79z)| norm 0.4322 (-0.69z)| lr 6.00e-04 | 322.36 ms | 52.4% bf16 MFU | 1623842 tok/s
step 1047/19560 | loss 4.291709 (-1.02z)| norm 0.5139 (+0.56z)| lr 6.00e-04 | 324.04 ms | 52.1% bf16 MFU | 1623547 tok/s
step 1048/19560 | loss 4.231095 (-1.96z)| norm 0.4769 (+0.00z)| lr 5.99e-04 | 322.93 ms | 52.3% bf16 MFU | 1623546 tok/s
step 1049/19560 | loss 4.230667 (-1.92z)| norm 0.3938 (-1.26z)| lr 5.99e-04 | 322.55 ms | 52.3% bf16 MFU | 1623640 tok/s
step 1050/19560 | loss 4.257368 (-1.48z)| norm 0.4180 (-0.87z)| lr 5.99e-04 | 323.80 ms | 52.1% bf16 MFU | 1623417 tok/s
step 1051/19560 | loss 4.284750 (-1.03z)| norm 0.4304 (-0.67z)| lr 5.99e-04 | 323.09 ms | 52.2% bf16 MFU | 1623383 tok/s
step 1052/19560 | loss 4.273417 (-1.19z)| norm 0.4154 (-0.89z)| lr 5.99e-04 | 322.73 ms | 52.3% bf16 MFU | 1623440 tok/s
step 1053/19560 | loss 4.284092 (-1.02z)| norm 0.3919 (-1.24z)| lr 5.99e-04 | 323.13 ms | 52.2% bf16 MFU | 1623394 tok/s
step 1054/19560 | loss 4.277356 (-1.11z)| norm 0.3778 (-1.43z)| lr 5.99e-04 | 323.51 ms | 52.2% bf16 MFU | 1623254 tok/s
step 1055/19560 | loss 4.308005 (-0.60z)| norm 0.4021 (-1.05z)| lr 5.99e-04 | 323.12 ms | 52.2% bf16 MFU | 1623220 tok/s
step 1056/19560 | loss 4.254036 (-1.46z)| norm 0.4492 (-0.32z)| lr 5.99e-04 | 322.76 ms | 52.3% bf16 MFU | 1623280 tok/s
step 1057/19560 | loss 4.285842 (-0.96z)| norm 0.4369 (-0.50z)| lr 5.99e-04 | 323.57 ms | 52.2% bf16 MFU | 1623132 tok/s
step 1058/19560 | loss 4.272183 (-1.17z)| norm 0.4513 (-0.26z)| lr 5.99e-04 | 322.90 ms | 52.3% bf16 MFU | 1623161 tok/s
step 1059/19560 | loss 4.250378 (-1.53z)| norm 0.4449 (-0.35z)| lr 5.99e-04 | 322.25 ms | 52.4% bf16 MFU | 1623350 tok/s
step 1060/19560 | loss 4.318236 (-0.34z)| norm 0.4776 (+0.22z)| lr 5.99e-04 | 322.99 ms | 52.3% bf16 MFU | 1623345 tok/s
step 1061/19560 | loss 4.461792 (+2.14z)| norm 0.5335 (+1.15z)| lr 5.99e-04 | 322.58 ms | 52.3% bf16 MFU | 1623441 tok/s
step 1062/19560 | loss 4.275778 (-1.07z)| norm 0.6488 (+2.96z)| lr 5.99e-04 | 322.49 ms | 52.3% bf16 MFU | 1623556 tok/s
step 1063/19560 | loss 4.300944 (-0.62z)| norm 0.5553 (+1.41z)| lr 5.99e-04 | 323.07 ms | 52.2% bf16 MFU | 1623521 tok/s
step 1064/19560 | loss 4.341844 (+0.11z)| norm 0.4979 (+0.47z)| lr 5.99e-04 | 322.97 ms | 52.3% bf16 MFU | 1623512 tok/s
step 1065/19560 | loss 4.287944 (-0.83z)| norm 0.5022 (+0.54z)| lr 5.99e-04 | 322.55 ms | 52.3% bf16 MFU | 1623608 tok/s
step 1066/19560 | loss 4.300617 (-0.59z)| norm 0.4693 (+0.02z)| lr 5.99e-04 | 323.20 ms | 52.2% bf16 MFU | 1623537 tok/s
step 1067/19560 | loss 4.264319 (-1.23z)| norm 0.4487 (-0.31z)| lr 5.99e-04 | 323.02 ms | 52.2% bf16 MFU | 1623514 tok/s
step 1068/19560 | loss 4.302049 (-0.54z)| norm 0.4347 (-0.53z)| lr 5.99e-04 | 323.32 ms | 52.2% bf16 MFU | 1623418 tok/s
step 1069/19560 | loss 4.425255 (+1.69z)| norm 0.4280 (-0.63z)| lr 5.99e-04 | 323.03 ms | 52.2% bf16 MFU | 1623399 tok/s
step 1070/19560 | loss 4.301131 (-0.54z)| norm 0.4736 (+0.15z)| lr 5.99e-04 | 322.76 ms | 52.3% bf16 MFU | 1623449 tok/s
step 1071/19560 | loss 4.246578 (-1.52z)| norm 0.4479 (-0.28z)| lr 5.99e-04 | 323.25 ms | 52.2% bf16 MFU | 1623371 tok/s
step 1072/19560 | loss 4.244065 (-1.56z)| norm 0.4306 (-0.57z)| lr 5.99e-04 | 323.30 ms | 52.2% bf16 MFU | 1623286 tok/s
step 1073/19560 | loss 4.311873 (-0.28z)| norm 0.3979 (-1.10z)| lr 5.99e-04 | 322.61 ms | 52.3% bf16 MFU | 1623380 tok/s
step 1074/19560 | loss 4.271628 (-1.02z)| norm 0.4051 (-0.98z)| lr 5.99e-04 | 322.72 ms | 52.3% bf16 MFU | 1623441 tok/s
step 1075/19560 | loss 4.265193 (-1.13z)| norm 0.3738 (-1.49z)| lr 5.99e-04 | 322.31 ms | 52.4% bf16 MFU | 1623601 tok/s
step 1076/19560 | loss 4.273385 (-0.96z)| norm 0.3771 (-1.42z)| lr 5.99e-04 | 322.97 ms | 52.3% bf16 MFU | 1623588 tok/s
step 1077/19560 | loss 4.254282 (-1.30z)| norm 0.3587 (-1.69z)| lr 5.99e-04 | 322.56 ms | 52.3% bf16 MFU | 1623678 tok/s
step 1078/19560 | loss 4.305501 (-0.32z)| norm 0.3851 (-1.25z)| lr 5.99e-04 | 323.05 ms | 52.2% bf16 MFU | 1623641 tok/s
step 1079/19560 | loss 4.242988 (-1.52z)| norm 0.4177 (-0.71z)| lr 5.99e-04 | 323.08 ms | 52.2% bf16 MFU | 1623598 tok/s
step 1080/19560 | loss 4.306591 (-0.26z)| norm 0.4312 (-0.49z)| lr 5.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1623697 tok/s
step 1081/19560 | loss 4.261479 (-1.13z)| norm 0.4131 (-0.78z)| lr 5.99e-04 | 322.70 ms | 52.3% bf16 MFU | 1623746 tok/s
step 1082/19560 | loss 4.298052 (-0.41z)| norm 0.3856 (-1.20z)| lr 5.99e-04 | 322.90 ms | 52.3% bf16 MFU | 1623743 tok/s
step 1083/19560 | loss 4.323348 (+0.11z)| norm 0.4195 (-0.64z)| lr 5.99e-04 | 322.77 ms | 52.3% bf16 MFU | 1623771 tok/s
step 1084/19560 | loss 4.303419 (-0.28z)| norm 0.4543 (-0.07z)| lr 5.99e-04 | 322.93 ms | 52.3% bf16 MFU | 1623759 tok/s
step 1085/19560 | loss 4.267123 (-1.00z)| norm 0.5108 (+0.85z)| lr 5.99e-04 | 322.79 ms | 52.3% bf16 MFU | 1623784 tok/s
step 1086/19560 | loss 4.254716 (-1.23z)| norm 0.5396 (+1.31z)| lr 5.99e-04 | 322.89 ms | 52.3% bf16 MFU | 1623780 tok/s
step 1087/19560 | loss 4.268510 (-0.94z)| norm 0.5929 (+2.12z)| lr 5.99e-04 | 322.66 ms | 52.3% bf16 MFU | 1623835 tok/s
step 1088/19560 | loss 4.273470 (-0.83z)| norm 0.4767 (+0.25z)| lr 5.99e-04 | 322.96 ms | 52.3% bf16 MFU | 1623812 tok/s
step 1089/19560 | loss 4.303243 (-0.21z)| norm 0.4317 (-0.48z)| lr 5.99e-04 | 322.92 ms | 52.3% bf16 MFU | 1623800 tok/s
step 1090/19560 | loss 4.215114 (-1.99z)| norm 0.4072 (-0.87z)| lr 5.99e-04 | 322.78 ms | 52.3% bf16 MFU | 1623825 tok/s
step 1091/19560 | loss 4.316307 (+0.10z)| norm 0.4040 (-0.90z)| lr 5.99e-04 | 322.82 ms | 52.3% bf16 MFU | 1623838 tok/s
step 1092/19560 | loss 4.308203 (-0.06z)| norm 0.3978 (-1.00z)| lr 5.99e-04 | 322.93 ms | 52.3% bf16 MFU | 1623822 tok/s
step 1093/19560 | loss 4.262244 (-1.00z)| norm 0.3936 (-1.06z)| lr 5.99e-04 | 323.06 ms | 52.2% bf16 MFU | 1623775 tok/s
step 1094/19560 | loss 4.310332 (+0.01z)| norm 0.3832 (-1.23z)| lr 5.99e-04 | 322.83 ms | 52.3% bf16 MFU | 1623788 tok/s
step 1095/19560 | loss 4.252660 (-1.18z)| norm 0.3649 (-1.52z)| lr 5.99e-04 | 322.85 ms | 52.3% bf16 MFU | 1623796 tok/s
step 1096/19560 | loss 4.225327 (-1.79z)| norm 0.3816 (-1.22z)| lr 5.99e-04 | 322.82 ms | 52.3% bf16 MFU | 1623811 tok/s
step 1097/19560 | loss 4.180476 (-2.67z)| norm 0.3770 (-1.28z)| lr 5.99e-04 | 322.44 ms | 52.3% bf16 MFU | 1623920 tok/s
step 1098/19560 | loss 4.259275 (-0.97z)| norm 0.3877 (-1.08z)| lr 5.99e-04 | 322.73 ms | 52.3% bf16 MFU | 1623952 tok/s
step 1099/19560 | loss 4.285997 (-0.39z)| norm 0.4428 (-0.15z)| lr 5.99e-04 | 322.55 ms | 52.3% bf16 MFU | 1624027 tok/s
step 1100/19560 | loss 4.211704 (-1.98z)| norm 0.4856 (+0.57z)| lr 5.99e-04 | 322.51 ms | 52.3% bf16 MFU | 1624107 tok/s
step 1101/19560 | loss 4.267924 (-0.74z)| norm 0.5066 (+0.92z)| lr 5.99e-04 | 322.89 ms | 52.3% bf16 MFU | 1624089 tok/s
step 1102/19560 | loss 4.250597 (-1.11z)| norm 0.5486 (+1.60z)| lr 5.99e-04 | 322.91 ms | 52.3% bf16 MFU | 1624067 tok/s
step 1103/19560 | loss 4.256037 (-0.97z)| norm 0.5318 (+1.29z)| lr 5.99e-04 | 322.94 ms | 52.3% bf16 MFU | 1624038 tok/s
step 1104/19560 | loss 4.249403 (-1.10z)| norm 0.4415 (-0.22z)| lr 5.99e-04 | 322.34 ms | 52.4% bf16 MFU | 1624161 tok/s
step 1105/19560 | loss 4.289483 (-0.22z)| norm 0.4712 (+0.27z)| lr 5.99e-04 | 322.82 ms | 52.3% bf16 MFU | 1624156 tok/s
step 1106/19560 | loss 4.276387 (-0.51z)| norm 0.4701 (+0.25z)| lr 5.99e-04 | 322.87 ms | 52.3% bf16 MFU | 1624139 tok/s
step 1107/19560 | loss 4.407222 (+2.29z)| norm 0.4354 (-0.33z)| lr 5.99e-04 | 322.12 ms | 52.4% bf16 MFU | 1624314 tok/s
step 1108/19560 | loss 4.230106 (-1.49z)| norm 0.4870 (+0.55z)| lr 5.99e-04 | 323.43 ms | 52.2% bf16 MFU | 1624148 tok/s
step 1109/19560 | loss 4.229639 (-1.48z)| norm 0.5265 (+1.23z)| lr 5.99e-04 | 322.70 ms | 52.3% bf16 MFU | 1624174 tok/s
step 1110/19560 | loss 4.256308 (-0.90z)| norm 0.4725 (+0.33z)| lr 5.99e-04 | 322.61 ms | 52.3% bf16 MFU | 1624223 tok/s
step 1111/19560 | loss 4.163010 (-2.78z)| norm 0.4820 (+0.49z)| lr 5.99e-04 | 322.86 ms | 52.3% bf16 MFU | 1624207 tok/s
step 1112/19560 | loss 4.228174 (-1.44z)| norm 0.4962 (+0.73z)| lr 5.99e-04 | 322.87 ms | 52.3% bf16 MFU | 1624187 tok/s
step 1113/19560 | loss 4.245527 (-1.05z)| norm 0.3770 (-1.31z)| lr 5.99e-04 | 322.65 ms | 52.3% bf16 MFU | 1624224 tok/s
step 1114/19560 | loss 4.251103 (-0.92z)| norm 0.3307 (-2.04z)| lr 5.99e-04 | 322.64 ms | 52.3% bf16 MFU | 1624261 tok/s
step 1115/19560 | loss 4.246608 (-1.00z)| norm 0.3441 (-1.80z)| lr 5.99e-04 | 322.47 ms | 52.3% bf16 MFU | 1624340 tok/s
step 1116/19560 | loss 4.240746 (-1.11z)| norm 0.3235 (-2.11z)| lr 5.99e-04 | 322.72 ms | 52.3% bf16 MFU | 1624352 tok/s
step 1117/19560 | loss 4.226913 (-1.39z)| norm 0.3296 (-1.97z)| lr 5.99e-04 | 322.39 ms | 52.4% bf16 MFU | 1624447 tok/s
step 1118/19560 | loss 4.224158 (-1.43z)| norm 0.3120 (-2.20z)| lr 5.99e-04 | 323.09 ms | 52.2% bf16 MFU | 1624361 tok/s
step 1119/19560 | loss 4.217641 (-1.54z)| norm 0.3155 (-2.09z)| lr 5.99e-04 | 322.75 ms | 52.3% bf16 MFU | 1624365 tok/s
step 1120/19560 | loss 4.233857 (-1.18z)| norm 0.3431 (-1.62z)| lr 5.99e-04 | 322.64 ms | 52.3% bf16 MFU | 1624396 tok/s
step 1121/19560 | loss 4.207540 (-1.71z)| norm 0.4048 (-0.63z)| lr 5.99e-04 | 323.19 ms | 52.2% bf16 MFU | 1624287 tok/s
step 1122/19560 | loss 4.174316 (-2.34z)| norm 0.4092 (-0.55z)| lr 5.99e-04 | 322.26 ms | 52.4% bf16 MFU | 1624417 tok/s
step 1123/19560 | loss 4.294195 (+0.16z)| norm 0.4419 (-0.04z)| lr 5.99e-04 | 323.07 ms | 52.2% bf16 MFU | 1624339 tok/s
step 1124/19560 | loss 4.201818 (-1.73z)| norm 0.4715 (+0.43z)| lr 5.99e-04 | 322.42 ms | 52.3% bf16 MFU | 1624428 tok/s
step 1125/19560 | loss 4.230196 (-1.13z)| norm 0.4194 (-0.39z)| lr 5.99e-04 | 322.65 ms | 52.3% bf16 MFU | 1624454 tok/s
step 1126/19560 | loss 4.312085 (+0.57z)| norm 0.4336 (-0.17z)| lr 5.99e-04 | 322.98 ms | 52.3% bf16 MFU | 1624396 tok/s
step 1127/19560 | loss 4.208316 (-1.56z)| norm 0.4685 (+0.38z)| lr 5.99e-04 | 322.63 ms | 52.3% bf16 MFU | 1624428 tok/s
step 1128/19560 | loss 4.180327 (-2.09z)| norm 0.5129 (+1.07z)| lr 5.99e-04 | 322.74 ms | 52.3% bf16 MFU | 1624433 tok/s
step 1129/19560 | loss 4.261365 (-0.42z)| norm 0.5181 (+1.15z)| lr 5.99e-04 | 322.70 ms | 52.3% bf16 MFU | 1624446 tok/s
step 1130/19560 | loss 4.251108 (-0.62z)| norm 0.4000 (-0.71z)| lr 5.99e-04 | 322.21 ms | 52.4% bf16 MFU | 1624581 tok/s
step 1131/19560 | loss 4.149376 (-2.62z)| norm 0.3458 (-1.58z)| lr 5.99e-04 | 322.95 ms | 52.3% bf16 MFU | 1624522 tok/s
step 1132/19560 | loss 4.223258 (-1.12z)| norm 0.3727 (-1.14z)| lr 5.99e-04 | 323.04 ms | 52.2% bf16 MFU | 1624446 tok/s
step 1133/19560 | loss 4.221910 (-1.13z)| norm 0.3783 (-1.03z)| lr 5.99e-04 | 323.12 ms | 52.2% bf16 MFU | 1624353 tok/s
step 1134/19560 | loss 4.259343 (-0.37z)| norm 0.4631 (+0.40z)| lr 5.99e-04 | 322.91 ms | 52.3% bf16 MFU | 1624316 tok/s
step 1135/19560 | loss 4.229073 (-0.97z)| norm 0.5217 (+1.37z)| lr 5.99e-04 | 322.75 ms | 52.3% bf16 MFU | 1624322 tok/s
step 1136/19560 | loss 4.231712 (-0.91z)| norm 0.4970 (+0.94z)| lr 5.99e-04 | 322.16 ms | 52.4% bf16 MFU | 1624475 tok/s
step 1137/19560 | loss 4.188697 (-1.75z)| norm 0.4085 (-0.54z)| lr 5.99e-04 | 322.59 ms | 52.3% bf16 MFU | 1624513 tok/s
step 1138/19560 | loss 4.307250 (+0.65z)| norm 0.3533 (-1.45z)| lr 5.99e-04 | 322.99 ms | 52.3% bf16 MFU | 1624448 tok/s
step 1139/19560 | loss 4.266775 (-0.16z)| norm 0.3798 (-1.01z)| lr 5.99e-04 | 322.73 ms | 52.3% bf16 MFU | 1624454 tok/s
step 1140/19560 | loss 4.197397 (-1.55z)| norm 0.3659 (-1.23z)| lr 5.99e-04 | 322.47 ms | 52.3% bf16 MFU | 1624524 tok/s
step 1141/19560 | loss 4.206427 (-1.34z)| norm 0.3752 (-1.06z)| lr 5.99e-04 | 322.55 ms | 52.3% bf16 MFU | 1624569 tok/s
step 1142/19560 | loss 4.160388 (-2.21z)| norm 0.3856 (-0.87z)| lr 5.99e-04 | 322.73 ms | 52.3% bf16 MFU | 1624568 tok/s
step 1143/19560 | loss 4.164186 (-2.08z)| norm 0.4105 (-0.46z)| lr 5.99e-04 | 323.27 ms | 52.2% bf16 MFU | 1624431 tok/s
step 1144/19560 | loss 4.187081 (-1.60z)| norm 0.4165 (-0.35z)| lr 5.99e-04 | 322.64 ms | 52.3% bf16 MFU | 1624460 tok/s
step 1145/19560 | loss 4.190776 (-1.50z)| norm 0.4206 (-0.27z)| lr 5.99e-04 | 322.77 ms | 52.3% bf16 MFU | 1624454 tok/s
step 1146/19560 | loss 4.249236 (-0.37z)| norm 0.4281 (-0.14z)| lr 5.99e-04 | 322.50 ms | 52.3% bf16 MFU | 1624518 tok/s
step 1147/19560 | loss 4.394010 (+2.36z)| norm 0.4151 (-0.36z)| lr 5.99e-04 | 322.63 ms | 52.3% bf16 MFU | 1624545 tok/s
step 1148/19560 | loss 4.235040 (-0.64z)| norm 0.3994 (-0.61z)| lr 5.99e-04 | 322.83 ms | 52.3% bf16 MFU | 1624520 tok/s
step 1149/19560 | loss 4.156266 (-2.08z)| norm 0.4114 (-0.40z)| lr 5.99e-04 | 322.65 ms | 52.3% bf16 MFU | 1624542 tok/s
step 1150/19560 | loss 4.146527 (-2.21z)| norm 0.3886 (-0.77z)| lr 5.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1624596 tok/s
step 1151/19560 | loss 4.257714 (-0.15z)| norm 0.4006 (-0.57z)| lr 5.99e-04 | 322.67 ms | 52.3% bf16 MFU | 1624607 tok/s
step 1152/19560 | loss 4.190435 (-1.37z)| norm 0.4374 (+0.05z)| lr 5.99e-04 | 322.65 ms | 52.3% bf16 MFU | 1624623 tok/s
step 1153/19560 | loss 4.185427 (-1.44z)| norm 0.4423 (+0.13z)| lr 5.99e-04 | 323.35 ms | 52.2% bf16 MFU | 1624462 tok/s
step 1154/19560 | loss 4.216591 (-0.85z)| norm 0.3881 (-0.77z)| lr 5.99e-04 | 322.76 ms | 52.3% bf16 MFU | 1624458 tok/s
step 1155/19560 | loss 4.192917 (-1.27z)| norm 0.4513 (+0.29z)| lr 5.99e-04 | 322.62 ms | 52.3% bf16 MFU | 1624490 tok/s
step 1156/19560 | loss 4.238899 (-0.41z)| norm 0.3659 (-1.14z)| lr 5.99e-04 | 323.10 ms | 52.2% bf16 MFU | 1624400 tok/s
step 1157/19560 | loss 4.170979 (-1.65z)| norm 0.3431 (-1.49z)| lr 5.99e-04 | 322.86 ms | 52.3% bf16 MFU | 1624374 tok/s
step 1158/19560 | loss 4.281308 (+0.39z)| norm 0.3549 (-1.28z)| lr 5.99e-04 | 322.85 ms | 52.3% bf16 MFU | 1624353 tok/s
step 1159/19560 | loss 4.189448 (-1.29z)| norm 0.3662 (-1.08z)| lr 5.99e-04 | 322.78 ms | 52.3% bf16 MFU | 1624349 tok/s
step 1160/19560 | loss 4.308475 (+0.90z)| norm 0.4213 (-0.17z)| lr 5.99e-04 | 322.27 ms | 52.4% bf16 MFU | 1624474 tok/s
step 1161/19560 | loss 4.204869 (-1.00z)| norm 0.4216 (-0.17z)| lr 5.99e-04 | 322.57 ms | 52.3% bf16 MFU | 1624516 tok/s
step 1162/19560 | loss 4.204150 (-0.99z)| norm 0.4771 (+0.73z)| lr 5.99e-04 | 322.83 ms | 52.3% bf16 MFU | 1624492 tok/s
step 1163/19560 | loss 4.286435 (+0.53z)| norm 0.5112 (+1.27z)| lr 5.99e-04 | 322.79 ms | 52.3% bf16 MFU | 1624479 tok/s
step 1164/19560 | loss 4.167003 (-1.66z)| norm 0.4828 (+0.80z)| lr 5.99e-04 | 322.49 ms | 52.3% bf16 MFU | 1624542 tok/s
step 1165/19560 | loss 4.211086 (-0.84z)| norm 0.4589 (+0.41z)| lr 5.99e-04 | 322.36 ms | 52.4% bf16 MFU | 1624636 tok/s
step 1166/19560 | loss 4.246637 (-0.18z)| norm 0.4763 (+0.70z)| lr 5.99e-04 | 322.84 ms | 52.3% bf16 MFU | 1624603 tok/s
step 1167/19560 | loss 4.237340 (-0.35z)| norm 0.4727 (+0.63z)| lr 5.99e-04 | 322.85 ms | 52.3% bf16 MFU | 1624570 tok/s
step 1168/19560 | loss 4.207323 (-0.90z)| norm 0.4088 (-0.42z)| lr 5.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1624623 tok/s
step 1169/19560 | loss 4.159014 (-1.75z)| norm 0.4054 (-0.47z)| lr 5.99e-04 | 322.66 ms | 52.3% bf16 MFU | 1624636 tok/s
step 1170/19560 | loss 4.247252 (-0.13z)| norm 0.4304 (-0.04z)| lr 5.99e-04 | 323.01 ms | 52.2% bf16 MFU | 1624559 tok/s
step 1171/19560 | loss 4.221545 (-0.59z)| norm 0.4400 (+0.14z)| lr 5.99e-04 | 322.82 ms | 52.3% bf16 MFU | 1624535 tok/s
step 1172/19560 | loss 4.189423 (-1.17z)| norm 0.4713 (+0.68z)| lr 5.99e-04 | 322.64 ms | 52.3% bf16 MFU | 1624558 tok/s
step 1173/19560 | loss 4.216667 (-0.65z)| norm 0.4143 (-0.29z)| lr 5.99e-04 | 323.32 ms | 52.2% bf16 MFU | 1624410 tok/s
step 1174/19560 | loss 4.354784 (+1.90z)| norm 0.4666 (+0.60z)| lr 5.99e-04 | 322.68 ms | 52.3% bf16 MFU | 1624428 tok/s
step 1175/19560 | loss 4.254039 (+0.04z)| norm 0.5576 (+2.13z)| lr 5.99e-04 | 322.68 ms | 52.3% bf16 MFU | 1624446 tok/s
step 1176/19560 | loss 4.213186 (-0.71z)| norm 0.4859 (+0.91z)| lr 5.99e-04 | 322.54 ms | 52.3% bf16 MFU | 1624500 tok/s
step 1177/19560 | loss 4.179469 (-1.31z)| norm 0.4027 (-0.49z)| lr 5.99e-04 | 323.03 ms | 52.2% bf16 MFU | 1624426 tok/s
step 1178/19560 | loss 4.164054 (-1.57z)| norm 0.3798 (-0.87z)| lr 5.99e-04 | 322.55 ms | 52.3% bf16 MFU | 1624476 tok/s
step 1179/19560 | loss 4.239907 (-0.18z)| norm 0.4072 (-0.41z)| lr 5.99e-04 | 322.50 ms | 52.3% bf16 MFU | 1624537 tok/s
step 1180/19560 | loss 4.159409 (-1.62z)| norm 0.3927 (-0.65z)| lr 5.99e-04 | 322.48 ms | 52.3% bf16 MFU | 1624600 tok/s
step 1181/19560 | loss 4.197785 (-0.91z)| norm 0.4038 (-0.46z)| lr 5.99e-04 | 323.17 ms | 52.2% bf16 MFU | 1624488 tok/s
step 1182/19560 | loss 4.244324 (-0.07z)| norm 0.3581 (-1.23z)| lr 5.99e-04 | 322.55 ms | 52.3% bf16 MFU | 1624534 tok/s
step 1183/19560 | loss 4.216019 (-0.57z)| norm 0.3500 (-1.35z)| lr 5.99e-04 | 322.69 ms | 52.3% bf16 MFU | 1624544 tok/s
step 1184/19560 | loss 4.193949 (-0.95z)| norm 0.3448 (-1.41z)| lr 5.99e-04 | 322.35 ms | 52.4% bf16 MFU | 1624639 tok/s
step 1185/19560 | loss 4.198360 (-0.86z)| norm 0.3800 (-0.82z)| lr 5.99e-04 | 322.76 ms | 52.3% bf16 MFU | 1624626 tok/s
step 1186/19560 | loss 4.183338 (-1.12z)| norm 0.3955 (-0.55z)| lr 5.99e-04 | 322.64 ms | 52.3% bf16 MFU | 1624644 tok/s
step 1187/19560 | loss 4.197066 (-0.86z)| norm 0.4053 (-0.39z)| lr 5.99e-04 | 322.67 ms | 52.3% bf16 MFU | 1624654 tok/s
step 1188/19560 | loss 4.169842 (-1.32z)| norm 0.4409 (+0.21z)| lr 5.99e-04 | 322.90 ms | 52.3% bf16 MFU | 1624606 tok/s
step 1189/19560 | loss 4.191823 (-0.95z)| norm 0.5074 (+1.32z)| lr 5.99e-04 | 322.59 ms | 52.3% bf16 MFU | 1624636 tok/s
step 1190/19560 | loss 4.181055 (-1.14z)| norm 0.4292 (+0.05z)| lr 5.99e-04 | 322.26 ms | 52.4% bf16 MFU | 1624750 tok/s
step 1191/19560 | loss 4.212391 (-0.53z)| norm 0.4028 (-0.40z)| lr 5.99e-04 | 322.49 ms | 52.3% bf16 MFU | 1624799 tok/s
step 1192/19560 | loss 4.204677 (-0.67z)| norm 0.3985 (-0.47z)| lr 5.99e-04 | 322.93 ms | 52.3% bf16 MFU | 1624737 tok/s
step 1193/19560 | loss 4.218983 (-0.38z)| norm 0.3900 (-0.61z)| lr 5.99e-04 | 322.86 ms | 52.3% bf16 MFU | 1624694 tok/s
step 1194/19560 | loss 4.151640 (-1.65z)| norm 0.3780 (-0.82z)| lr 5.99e-04 | 322.66 ms | 52.3% bf16 MFU | 1624703 tok/s
step 1195/19560 | loss 4.176857 (-1.15z)| norm 0.4004 (-0.40z)| lr 5.99e-04 | 322.85 ms | 52.3% bf16 MFU | 1624665 tok/s
step 1196/19560 | loss 4.158461 (-1.47z)| norm 0.4460 (+0.42z)| lr 5.99e-04 | 323.16 ms | 52.2% bf16 MFU | 1624551 tok/s
step 1197/19560 | loss 4.193521 (-0.81z)| norm 0.4861 (+1.14z)| lr 5.99e-04 | 322.95 ms | 52.3% bf16 MFU | 1624494 tok/s
step 1198/19560 | loss 4.156698 (-1.53z)| norm 0.4028 (-0.36z)| lr 5.99e-04 | 322.76 ms | 52.3% bf16 MFU | 1624490 tok/s
step 1199/19560 | loss 4.206083 (-0.53z)| norm 0.3915 (-0.55z)| lr 5.99e-04 | 322.73 ms | 52.3% bf16 MFU | 1624491 tok/s
step 1200/19560 | loss 4.146915 (-1.68z)| norm 0.3655 (-1.01z)| lr 5.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1624547 tok/s
step 1201/19560 | loss 4.151252 (-1.57z)| norm 0.3462 (-1.34z)| lr 5.99e-04 | 322.63 ms | 52.3% bf16 MFU | 1624573 tok/s
step 1202/19560 | loss 4.188396 (-0.82z)| norm 0.3154 (-1.86z)| lr 5.99e-04 | 323.26 ms | 52.2% bf16 MFU | 1624439 tok/s
step 1203/19560 | loss 4.370740 (+2.69z)| norm 0.3537 (-1.18z)| lr 5.99e-04 | 323.57 ms | 52.2% bf16 MFU | 1624232 tok/s
step 1204/19560 | loss 4.082326 (-2.75z)| norm 0.3673 (-0.93z)| lr 5.99e-04 | 322.05 ms | 52.4% bf16 MFU | 1624419 tok/s
step 1205/19560 | loss 4.185234 (-0.81z)| norm 0.4121 (-0.15z)| lr 5.99e-04 | 323.04 ms | 52.2% bf16 MFU | 1624349 tok/s
step 1206/19560 | loss 4.179572 (-0.90z)| norm 0.4715 (+0.89z)| lr 5.99e-04 | 323.60 ms | 52.2% bf16 MFU | 1624139 tok/s
step 1207/19560 | loss 4.119399 (-1.98z)| norm 0.4467 (+0.44z)| lr 5.99e-04 | 322.64 ms | 52.3% bf16 MFU | 1624181 tok/s
step 1208/19560 | loss 4.150757 (-1.38z)| norm 0.4527 (+0.54z)| lr 5.99e-04 | 322.29 ms | 52.4% bf16 MFU | 1624310 tok/s
step 1209/19560 | loss 4.132619 (-1.68z)| norm 0.4262 (+0.08z)| lr 5.99e-04 | 323.32 ms | 52.2% bf16 MFU | 1624174 tok/s
step 1210/19560 | loss 4.202093 (-0.40z)| norm 0.3774 (-0.78z)| lr 5.99e-04 | 322.99 ms | 52.3% bf16 MFU | 1624127 tok/s
step 1211/19560 | loss 4.176745 (-0.86z)| norm 0.3610 (-1.06z)| lr 5.99e-04 | 322.67 ms | 52.3% bf16 MFU | 1624163 tok/s
step 1212/19560 | loss 4.076228 (-2.64z)| norm 0.3604 (-1.05z)| lr 5.99e-04 | 322.77 ms | 52.3% bf16 MFU | 1624173 tok/s
step 1213/19560 | loss 4.153026 (-1.22z)| norm 0.3481 (-1.25z)| lr 5.99e-04 | 323.74 ms | 52.1% bf16 MFU | 1623937 tok/s
step 1214/19560 | loss 4.123189 (-1.73z)| norm 0.3645 (-0.95z)| lr 5.99e-04 | 322.61 ms | 52.3% bf16 MFU | 1623998 tok/s
step 1215/19560 | loss 4.152661 (-1.18z)| norm 0.3753 (-0.76z)| lr 5.99e-04 | 322.81 ms | 52.3% bf16 MFU | 1624006 tok/s
step 1216/19560 | loss 4.151632 (-1.18z)| norm 0.3810 (-0.64z)| lr 5.99e-04 | 322.97 ms | 52.3% bf16 MFU | 1623972 tok/s
step 1217/19560 | loss 4.126484 (-1.60z)| norm 0.4298 (+0.26z)| lr 5.99e-04 | 323.57 ms | 52.2% bf16 MFU | 1623790 tok/s
step 1218/19560 | loss 4.173442 (-0.75z)| norm 0.5062 (+1.65z)| lr 5.99e-04 | 323.09 ms | 52.2% bf16 MFU | 1623737 tok/s
step 1219/19560 | loss 4.129362 (-1.52z)| norm 0.4997 (+1.50z)| lr 5.99e-04 | 322.83 ms | 52.3% bf16 MFU | 1623751 tok/s
step 1220/19560 | loss 4.154597 (-1.05z)| norm 0.4004 (-0.30z)| lr 5.99e-04 | 323.08 ms | 52.2% bf16 MFU | 1623704 tok/s
step 1221/19560 | loss 4.100323 (-1.99z)| norm 0.3834 (-0.61z)| lr 5.99e-04 | 322.89 ms | 52.3% bf16 MFU | 1623704 tok/s
step 1222/19560 | loss 4.210634 (-0.00z)| norm 0.3712 (-0.83z)| lr 5.99e-04 | 322.80 ms | 52.3% bf16 MFU | 1623730 tok/s
step 1223/19560 | loss 4.254615 (+0.79z)| norm 0.3962 (-0.38z)| lr 5.99e-04 | 322.89 ms | 52.3% bf16 MFU | 1623730 tok/s
step 1224/19560 | loss 4.226801 (+0.29z)| norm 0.4601 (+0.77z)| lr 5.99e-04 | 323.15 ms | 52.2% bf16 MFU | 1623666 tok/s
step 1225/19560 | loss 4.172290 (-0.69z)| norm 0.4747 (+1.02z)| lr 5.99e-04 | 323.22 ms | 52.2% bf16 MFU | 1623587 tok/s
step 1226/19560 | loss 4.167961 (-0.76z)| norm 0.4690 (+0.90z)| lr 5.99e-04 | 323.27 ms | 52.2% bf16 MFU | 1623500 tok/s
step 1227/19560 | loss 4.262054 (+0.95z)| norm 0.4267 (+0.14z)| lr 5.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1623605 tok/s
step 1228/19560 | loss 4.160992 (-0.88z)| norm 0.3813 (-0.67z)| lr 5.99e-04 | 322.84 ms | 52.3% bf16 MFU | 1623624 tok/s
step 1229/19560 | loss 4.116798 (-1.65z)| norm 0.3538 (-1.16z)| lr 5.99e-04 | 323.23 ms | 52.2% bf16 MFU | 1623544 tok/s
step 1230/19560 | loss 4.140265 (-1.20z)| norm 0.3383 (-1.43z)| lr 5.99e-04 | 322.85 ms | 52.3% bf16 MFU | 1623564 tok/s
step 1231/19560 | loss 4.133938 (-1.29z)| norm 0.3151 (-1.84z)| lr 5.99e-04 | 323.32 ms | 52.2% bf16 MFU | 1623464 tok/s
step 1232/19560 | loss 4.323032 (+2.04z)| norm 0.3224 (-1.67z)| lr 5.99e-04 | 322.41 ms | 52.3% bf16 MFU | 1623599 tok/s
step 1233/19560 | loss 4.166764 (-0.70z)| norm 0.3389 (-1.34z)| lr 5.99e-04 | 322.99 ms | 52.3% bf16 MFU | 1623582 tok/s
step 1234/19560 | loss 4.115427 (-1.58z)| norm 0.3846 (-0.49z)| lr 5.99e-04 | 323.34 ms | 52.2% bf16 MFU | 1623476 tok/s
step 1235/19560 | loss 4.129014 (-1.36z)| norm 0.4297 (+0.35z)| lr 5.99e-04 | 323.19 ms | 52.2% bf16 MFU | 1623413 tok/s
step 1236/19560 | loss 4.135087 (-1.23z)| norm 0.4543 (+0.81z)| lr 5.99e-04 | 323.22 ms | 52.2% bf16 MFU | 1623346 tok/s
step 1237/19560 | loss 4.187058 (-0.27z)| norm 0.3752 (-0.65z)| lr 5.99e-04 | 322.20 ms | 52.4% bf16 MFU | 1623538 tok/s
step 1238/19560 | loss 4.170949 (-0.55z)| norm 0.3508 (-1.10z)| lr 5.99e-04 | 322.56 ms | 52.3% bf16 MFU | 1623632 tok/s
step 1239/19560 | loss 4.118270 (-1.51z)| norm 0.3300 (-1.47z)| lr 5.99e-04 | 323.65 ms | 52.1% bf16 MFU | 1623448 tok/s
step 1240/19560 | loss 4.161664 (-0.70z)| norm 0.3355 (-1.34z)| lr 5.99e-04 | 322.97 ms | 52.3% bf16 MFU | 1623443 tok/s
step 1241/19560 | loss 4.134306 (-1.18z)| norm 0.3053 (-1.88z)| lr 5.99e-04 | 322.93 ms | 52.3% bf16 MFU | 1623448 tok/s
step 1242/19560 | loss 4.110388 (-1.59z)| norm 0.3232 (-1.54z)| lr 5.99e-04 | 322.99 ms | 52.3% bf16 MFU | 1623438 tok/s
step 1243/19560 | loss 4.183714 (-0.25z)| norm 0.3611 (-0.84z)| lr 5.99e-04 | 322.96 ms | 52.3% bf16 MFU | 1623435 tok/s
step 1244/19560 | loss 4.148710 (-0.87z)| norm 0.3617 (-0.84z)| lr 5.99e-04 | 322.81 ms | 52.3% bf16 MFU | 1623471 tok/s
step 1245/19560 | loss 4.092630 (-1.85z)| norm 0.3575 (-0.93z)| lr 5.99e-04 | 323.26 ms | 52.2% bf16 MFU | 1623390 tok/s
step 1246/19560 | loss 4.123858 (-1.27z)| norm 0.4315 (+0.47z)| lr 5.99e-04 | 322.94 ms | 52.3% bf16 MFU | 1623395 tok/s
step 1247/19560 | loss 4.141810 (-0.94z)| norm 0.4624 (+1.05z)| lr 5.99e-04 | 322.84 ms | 52.3% bf16 MFU | 1623426 tok/s
step 1248/19560 | loss 4.124043 (-1.23z)| norm 0.5388 (+2.46z)| lr 5.99e-04 | 322.59 ms | 52.3% bf16 MFU | 1623517 tok/s
step 1249/19560 | loss 4.121623 (-1.26z)| norm 0.5292 (+2.21z)| lr 5.99e-04 | 322.56 ms | 52.3% bf16 MFU | 1623611 tok/s
step 1250/19560 | loss 4.124686 (-1.19z)| norm 0.4793 (+1.26z)| lr 5.99e-04 | 322.83 ms | 52.3% bf16 MFU | 1623631 tok/s
val loss 4.141542
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2595/10042 = 0.258415
step 1251/19560 | loss 4.167761 (-0.42z)| norm 0.4587 (+0.87z)| lr 5.99e-04 | 321.67 ms | 52.5% bf16 MFU | 1623945 tok/s
step 1252/19560 | loss 4.118954 (-1.27z)| norm 0.4477 (+0.68z)| lr 5.99e-04 | 323.16 ms | 52.2% bf16 MFU | 1623866 tok/s
step 1253/19560 | loss 4.119660 (-1.24z)| norm 0.3975 (-0.25z)| lr 5.99e-04 | 324.18 ms | 52.1% bf16 MFU | 1623536 tok/s
step 1254/19560 | loss 4.112191 (-1.35z)| norm 0.3505 (-1.11z)| lr 5.99e-04 | 323.31 ms | 52.2% bf16 MFU | 1623441 tok/s
step 1255/19560 | loss 4.105109 (-1.45z)| norm 0.4028 (-0.13z)| lr 5.99e-04 | 323.52 ms | 52.2% bf16 MFU | 1623296 tok/s
step 1256/19560 | loss 4.075395 (-1.93z)| norm 0.3462 (-1.17z)| lr 5.99e-04 | 322.78 ms | 52.3% bf16 MFU | 1623346 tok/s
step 1257/19560 | loss 4.106080 (-1.38z)| norm 0.3317 (-1.43z)| lr 5.99e-04 | 323.28 ms | 52.2% bf16 MFU | 1623268 tok/s
step 1258/19560 | loss 4.114724 (-1.21z)| norm 0.3162 (-1.69z)| lr 5.99e-04 | 323.07 ms | 52.2% bf16 MFU | 1623247 tok/s
step 1259/19560 | loss 4.124422 (-1.04z)| norm 0.3345 (-1.34z)| lr 5.99e-04 | 322.96 ms | 52.3% bf16 MFU | 1623255 tok/s
step 1260/19560 | loss 4.155513 (-0.49z)| norm 0.3676 (-0.72z)| lr 5.99e-04 | 322.94 ms | 52.3% bf16 MFU | 1623266 tok/s
step 1261/19560 | loss 4.165601 (-0.31z)| norm 0.3917 (-0.28z)| lr 5.99e-04 | 323.05 ms | 52.2% bf16 MFU | 1623250 tok/s
step 1262/19560 | loss 4.090282 (-1.58z)| norm 0.3847 (-0.40z)| lr 5.99e-04 | 323.72 ms | 52.1% bf16 MFU | 1623067 tok/s
step 1263/19560 | loss 4.119903 (-1.06z)| norm 0.3584 (-0.88z)| lr 5.99e-04 | 322.93 ms | 52.3% bf16 MFU | 1623091 tok/s
step 1264/19560 | loss 4.151977 (-0.49z)| norm 0.3081 (-1.81z)| lr 5.99e-04 | 322.99 ms | 52.3% bf16 MFU | 1623097 tok/s
step 1265/19560 | loss 4.190174 (+0.17z)| norm 0.3364 (-1.25z)| lr 5.99e-04 | 322.98 ms | 52.3% bf16 MFU | 1623108 tok/s
step 1266/19560 | loss 4.214524 (+0.61z)| norm 0.4458 (+0.80z)| lr 5.99e-04 | 322.95 ms | 52.3% bf16 MFU | 1623123 tok/s
step 1267/19560 | loss 4.137581 (-0.73z)| norm 0.4704 (+1.25z)| lr 5.99e-04 | 322.64 ms | 52.3% bf16 MFU | 1623216 tok/s
step 1268/19560 | loss 4.109784 (-1.20z)| norm 0.4521 (+0.89z)| lr 5.99e-04 | 323.14 ms | 52.2% bf16 MFU | 1623179 tok/s
step 1269/19560 | loss 4.184509 (+0.11z)| norm 0.5158 (+2.04z)| lr 5.99e-04 | 322.82 ms | 52.3% bf16 MFU | 1623224 tok/s
step 1270/19560 | loss 4.108706 (-1.21z)| norm 0.4424 (+0.67z)| lr 5.99e-04 | 323.28 ms | 52.2% bf16 MFU | 1623150 tok/s
step 1271/19560 | loss 4.090984 (-1.50z)| norm 0.4492 (+0.79z)| lr 5.99e-04 | 322.50 ms | 52.3% bf16 MFU | 1623279 tok/s
step 1272/19560 | loss 4.077159 (-1.70z)| norm 0.3861 (-0.37z)| lr 5.99e-04 | 323.06 ms | 52.2% bf16 MFU | 1623258 tok/s
step 1273/19560 | loss 4.078413 (-1.65z)| norm 0.3513 (-1.00z)| lr 5.99e-04 | 322.86 ms | 52.3% bf16 MFU | 1623290 tok/s
step 1274/19560 | loss 4.066900 (-1.81z)| norm 0.3423 (-1.15z)| lr 5.99e-04 | 322.62 ms | 52.3% bf16 MFU | 1623380 tok/s
step 1275/19560 | loss 4.076345 (-1.68z)| norm 0.3608 (-0.80z)| lr 5.99e-04 | 323.01 ms | 52.2% bf16 MFU | 1623367 tok/s
step 1276/19560 | loss 4.089949 (-1.42z)| norm 0.4125 (+0.14z)| lr 5.99e-04 | 322.96 ms | 52.3% bf16 MFU | 1623368 tok/s
step 1277/19560 | loss 4.099973 (-1.22z)| norm 0.4324 (+0.50z)| lr 5.99e-04 | 323.07 ms | 52.2% bf16 MFU | 1623342 tok/s
step 1278/19560 | loss 4.062058 (-1.85z)| norm 0.4132 (+0.15z)| lr 5.99e-04 | 322.83 ms | 52.3% bf16 MFU | 1623377 tok/s
step 1279/19560 | loss 4.157190 (-0.20z)| norm 0.4424 (+0.67z)| lr 5.99e-04 | 322.71 ms | 52.3% bf16 MFU | 1623439 tok/s
step 1280/19560 | loss 4.127111 (-0.71z)| norm 0.4409 (+0.64z)| lr 5.99e-04 | 322.89 ms | 52.3% bf16 MFU | 1623455 tok/s
step 1281/19560 | loss 4.146239 (-0.37z)| norm 0.4010 (-0.08z)| lr 5.99e-04 | 322.64 ms | 52.3% bf16 MFU | 1623533 tok/s
step 1282/19560 | loss 4.144344 (-0.40z)| norm 0.3913 (-0.25z)| lr 5.99e-04 | 322.98 ms | 52.3% bf16 MFU | 1623519 tok/s
step 1283/19560 | loss 4.133632 (-0.57z)| norm 0.3562 (-0.88z)| lr 5.99e-04 | 323.03 ms | 52.2% bf16 MFU | 1623495 tok/s
step 1284/19560 | loss 4.103918 (-1.08z)| norm 0.3510 (-0.97z)| lr 5.99e-04 | 322.66 ms | 52.3% bf16 MFU | 1623565 tok/s
step 1285/19560 | loss 4.075294 (-1.55z)| norm 0.3543 (-0.91z)| lr 5.99e-04 | 322.80 ms | 52.3% bf16 MFU | 1623596 tok/s
step 1286/19560 | loss 4.119056 (-0.78z)| norm 0.3443 (-1.09z)| lr 5.99e-04 | 322.46 ms | 52.3% bf16 MFU | 1623712 tok/s
step 1287/19560 | loss 4.129856 (-0.58z)| norm 0.3679 (-0.66z)| lr 5.99e-04 | 323.06 ms | 52.2% bf16 MFU | 1623671 tok/s
step 1288/19560 | loss 4.138741 (-0.41z)| norm 0.4137 (+0.17z)| lr 5.99e-04 | 322.84 ms | 52.3% bf16 MFU | 1623686 tok/s
step 1289/19560 | loss 4.090109 (-1.27z)| norm 0.4167 (+0.23z)| lr 5.99e-04 | 322.86 ms | 52.3% bf16 MFU | 1623695 tok/s
step 1290/19560 | loss 4.094909 (-1.16z)| norm 0.3594 (-0.80z)| lr 5.99e-04 | 322.72 ms | 52.3% bf16 MFU | 1623741 tok/s
step 1291/19560 | loss 4.094496 (-1.16z)| norm 0.3439 (-1.08z)| lr 5.99e-04 | 322.72 ms | 52.3% bf16 MFU | 1623782 tok/s
step 1292/19560 | loss 4.115676 (-0.77z)| norm 0.3359 (-1.21z)| lr 5.99e-04 | 322.97 ms | 52.3% bf16 MFU | 1623759 tok/s
step 1293/19560 | loss 4.090409 (-1.21z)| norm 0.3191 (-1.49z)| lr 5.99e-04 | 322.49 ms | 52.3% bf16 MFU | 1623860 tok/s
step 1294/19560 | loss 4.104644 (-0.94z)| norm 0.3570 (-0.78z)| lr 5.99e-04 | 322.60 ms | 52.3% bf16 MFU | 1623927 tok/s
step 1295/19560 | loss 4.132895 (-0.41z)| norm 0.3929 (-0.10z)| lr 5.99e-04 | 323.17 ms | 52.2% bf16 MFU | 1623846 tok/s
step 1296/19560 | loss 4.103261 (-0.94z)| norm 0.4353 (+0.69z)| lr 5.99e-04 | 322.65 ms | 52.3% bf16 MFU | 1623902 tok/s
step 1297/19560 | loss 4.082335 (-1.30z)| norm 0.4010 (+0.05z)| lr 5.99e-04 | 322.98 ms | 52.3% bf16 MFU | 1623870 tok/s
step 1298/19560 | loss 4.123477 (-0.54z)| norm 0.4824 (+1.55z)| lr 5.99e-04 | 322.84 ms | 52.3% bf16 MFU | 1623876 tok/s
step 1299/19560 | loss 4.008973 (-2.57z)| norm 0.4999 (+1.84z)| lr 5.99e-04 | 322.56 ms | 52.3% bf16 MFU | 1623952 tok/s
step 1300/19560 | loss 4.041705 (-1.93z)| norm 0.3914 (-0.13z)| lr 5.99e-04 | 322.88 ms | 52.3% bf16 MFU | 1623943 tok/s
step 1301/19560 | loss 4.153322 (+0.07z)| norm 0.3732 (-0.46z)| lr 5.99e-04 | 322.76 ms | 52.3% bf16 MFU | 1623964 tok/s
step 1302/19560 | loss 4.140512 (-0.14z)| norm 0.4270 (+0.54z)| lr 5.98e-04 | 323.48 ms | 52.2% bf16 MFU | 1623805 tok/s
step 1303/19560 | loss 4.045087 (-1.92z)| norm 0.3682 (-0.54z)| lr 5.98e-04 | 322.56 ms | 52.3% bf16 MFU | 1623885 tok/s
step 1304/19560 | loss 4.153109 (+0.14z)| norm 0.3322 (-1.22z)| lr 5.98e-04 | 322.63 ms | 52.3% bf16 MFU | 1623943 tok/s
step 1305/19560 | loss 4.059356 (-1.62z)| norm 0.3242 (-1.35z)| lr 5.98e-04 | 322.81 ms | 52.3% bf16 MFU | 1623953 tok/s
step 1306/19560 | loss 4.160428 (+0.30z)| norm 0.3477 (-0.89z)| lr 5.98e-04 | 322.86 ms | 52.3% bf16 MFU | 1623949 tok/s
step 1307/19560 | loss 4.023816 (-2.25z)| norm 0.3568 (-0.71z)| lr 5.98e-04 | 322.76 ms | 52.3% bf16 MFU | 1623970 tok/s
step 1308/19560 | loss 4.151257 (+0.16z)| norm 0.3477 (-0.88z)| lr 5.98e-04 | 323.12 ms | 52.2% bf16 MFU | 1623900 tok/s
step 1309/19560 | loss 4.043431 (-1.84z)| norm 0.3621 (-0.59z)| lr 5.98e-04 | 322.62 ms | 52.3% bf16 MFU | 1623959 tok/s
step 1310/19560 | loss 4.118425 (-0.42z)| norm 0.3308 (-1.18z)| lr 5.98e-04 | 322.78 ms | 52.3% bf16 MFU | 1623976 tok/s
step 1311/19560 | loss 4.113226 (-0.51z)| norm 0.3447 (-0.92z)| lr 5.98e-04 | 323.06 ms | 52.2% bf16 MFU | 1623922 tok/s
step 1312/19560 | loss 4.087688 (-0.98z)| norm 0.3935 (-0.00z)| lr 5.98e-04 | 322.79 ms | 52.3% bf16 MFU | 1623937 tok/s
step 1313/19560 | loss 4.141487 (+0.05z)| norm 0.4112 (+0.33z)| lr 5.98e-04 | 322.95 ms | 52.3% bf16 MFU | 1623911 tok/s
step 1314/19560 | loss 4.116852 (-0.41z)| norm 0.3485 (-0.85z)| lr 5.98e-04 | 322.84 ms | 52.3% bf16 MFU | 1623916 tok/s
step 1315/19560 | loss 4.046134 (-1.73z)| norm 0.3045 (-1.65z)| lr 5.98e-04 | 323.00 ms | 52.3% bf16 MFU | 1623880 tok/s
step 1316/19560 | loss 4.070950 (-1.24z)| norm 0.3306 (-1.14z)| lr 5.98e-04 | 323.03 ms | 52.2% bf16 MFU | 1623838 tok/s
step 1317/19560 | loss 4.052610 (-1.56z)| norm 0.3357 (-1.04z)| lr 5.98e-04 | 322.96 ms | 52.3% bf16 MFU | 1623814 tok/s
step 1318/19560 | loss 4.061085 (-1.38z)| norm 0.3646 (-0.48z)| lr 5.98e-04 | 322.64 ms | 52.3% bf16 MFU | 1623874 tok/s
step 1319/19560 | loss 4.111064 (-0.42z)| norm 0.3909 (+0.02z)| lr 5.98e-04 | 322.63 ms | 52.3% bf16 MFU | 1623932 tok/s
step 1320/19560 | loss 4.139692 (+0.13z)| norm 0.4405 (+0.95z)| lr 5.98e-04 | 323.17 ms | 52.2% bf16 MFU | 1623852 tok/s
step 1321/19560 | loss 4.136234 (+0.08z)| norm 0.4441 (+1.01z)| lr 5.98e-04 | 322.89 ms | 52.3% bf16 MFU | 1623845 tok/s
step 1322/19560 | loss 4.084999 (-0.90z)| norm 0.4603 (+1.29z)| lr 5.98e-04 | 322.91 ms | 52.3% bf16 MFU | 1623834 tok/s
step 1323/19560 | loss 4.118074 (-0.25z)| norm 0.4515 (+1.12z)| lr 5.98e-04 | 322.88 ms | 52.3% bf16 MFU | 1623831 tok/s
step 1324/19560 | loss 4.192485 (+1.17z)| norm 0.4036 (+0.23z)| lr 5.98e-04 | 322.12 ms | 52.4% bf16 MFU | 1624020 tok/s
step 1325/19560 | loss 4.107422 (-0.45z)| norm 0.3844 (-0.11z)| lr 5.98e-04 | 322.92 ms | 52.3% bf16 MFU | 1623997 tok/s
step 1326/19560 | loss 4.092796 (-0.72z)| norm 0.3788 (-0.22z)| lr 5.98e-04 | 322.59 ms | 52.3% bf16 MFU | 1624060 tok/s
step 1327/19560 | loss 4.093022 (-0.71z)| norm 0.3935 (+0.06z)| lr 5.98e-04 | 323.07 ms | 52.2% bf16 MFU | 1623999 tok/s
step 1328/19560 | loss 4.181535 (+1.00z)| norm 0.3441 (-0.87z)| lr 5.98e-04 | 322.65 ms | 52.3% bf16 MFU | 1624047 tok/s
step 1329/19560 | loss 4.076842 (-1.01z)| norm 0.3170 (-1.37z)| lr 5.98e-04 | 322.58 ms | 52.3% bf16 MFU | 1624110 tok/s
step 1330/19560 | loss 4.204367 (+1.44z)| norm 0.3193 (-1.33z)| lr 5.98e-04 | 322.52 ms | 52.3% bf16 MFU | 1624185 tok/s
step 1331/19560 | loss 4.084752 (-0.89z)| norm 0.3360 (-1.01z)| lr 5.98e-04 | 322.96 ms | 52.3% bf16 MFU | 1624145 tok/s
step 1332/19560 | loss 4.051969 (-1.56z)| norm 0.3196 (-1.30z)| lr 5.98e-04 | 322.86 ms | 52.3% bf16 MFU | 1624131 tok/s
step 1333/19560 | loss 4.088660 (-0.78z)| norm 0.2938 (-1.75z)| lr 5.98e-04 | 322.64 ms | 52.3% bf16 MFU | 1624175 tok/s
step 1334/19560 | loss 4.167230 (+0.87z)| norm 0.2979 (-1.64z)| lr 5.98e-04 | 324.58 ms | 52.0% bf16 MFU | 1623731 tok/s
step 1335/19560 | loss 4.110883 (-0.32z)| norm 0.3637 (-0.42z)| lr 5.98e-04 | 322.24 ms | 52.4% bf16 MFU | 1623894 tok/s
step 1336/19560 | loss 4.129498 (+0.08z)| norm 0.4022 (+0.30z)| lr 5.98e-04 | 322.59 ms | 52.3% bf16 MFU | 1623962 tok/s
step 1337/19560 | loss 4.120827 (-0.10z)| norm 0.3838 (-0.04z)| lr 5.98e-04 | 322.58 ms | 52.3% bf16 MFU | 1624028 tok/s
step 1338/19560 | loss 4.054112 (-1.48z)| norm 0.4041 (+0.34z)| lr 5.98e-04 | 322.22 ms | 52.4% bf16 MFU | 1624182 tok/s
step 1339/19560 | loss 4.232796 (+2.24z)| norm 0.4791 (+1.70z)| lr 5.98e-04 | 323.32 ms | 52.2% bf16 MFU | 1624051 tok/s
step 1340/19560 | loss 4.150126 (+0.51z)| norm 0.4515 (+1.17z)| lr 5.98e-04 | 322.66 ms | 52.3% bf16 MFU | 1624092 tok/s
step 1341/19560 | loss 4.093121 (-0.66z)| norm 0.3791 (-0.16z)| lr 5.98e-04 | 322.46 ms | 52.3% bf16 MFU | 1624183 tok/s
step 1342/19560 | loss 4.145670 (+0.43z)| norm 0.3880 (-0.00z)| lr 5.98e-04 | 323.28 ms | 52.2% bf16 MFU | 1624064 tok/s
step 1343/19560 | loss 4.129520 (+0.09z)| norm 0.3465 (-0.76z)| lr 5.98e-04 | 322.79 ms | 52.3% bf16 MFU | 1624074 tok/s
step 1344/19560 | loss 4.078375 (-0.96z)| norm 0.3154 (-1.31z)| lr 5.98e-04 | 322.94 ms | 52.3% bf16 MFU | 1624043 tok/s
step 1345/19560 | loss 4.041599 (-1.69z)| norm 0.3466 (-0.73z)| lr 5.98e-04 | 322.46 ms | 52.3% bf16 MFU | 1624137 tok/s
step 1346/19560 | loss 4.041125 (-1.66z)| norm 0.3604 (-0.47z)| lr 5.98e-04 | 323.21 ms | 52.2% bf16 MFU | 1624038 tok/s
step 1347/19560 | loss 4.024932 (-1.95z)| norm 0.3829 (-0.03z)| lr 5.98e-04 | 322.83 ms | 52.3% bf16 MFU | 1624038 tok/s
step 1348/19560 | loss 4.075709 (-0.91z)| norm 0.3752 (-0.17z)| lr 5.98e-04 | 322.38 ms | 52.4% bf16 MFU | 1624152 tok/s
step 1349/19560 | loss 4.117517 (-0.08z)| norm 0.3306 (-1.01z)| lr 5.98e-04 | 322.71 ms | 52.3% bf16 MFU | 1624176 tok/s
step 1350/19560 | loss 4.082644 (-0.77z)| norm 0.3231 (-1.14z)| lr 5.98e-04 | 322.90 ms | 52.3% bf16 MFU | 1624151 tok/s
step 1351/19560 | loss 4.052288 (-1.38z)| norm 0.3147 (-1.27z)| lr 5.98e-04 | 322.64 ms | 52.3% bf16 MFU | 1624193 tok/s
step 1352/19560 | loss 4.080681 (-0.78z)| norm 0.3262 (-1.04z)| lr 5.98e-04 | 323.14 ms | 52.2% bf16 MFU | 1624106 tok/s
step 1353/19560 | loss 4.098711 (-0.39z)| norm 0.3673 (-0.26z)| lr 5.98e-04 | 322.84 ms | 52.3% bf16 MFU | 1624100 tok/s
step 1354/19560 | loss 4.166344 (+1.04z)| norm 0.4942 (+2.12z)| lr 5.98e-04 | 322.71 ms | 52.3% bf16 MFU | 1624128 tok/s
step 1355/19560 | loss 4.194375 (+1.69z)| norm 0.6202 (+4.16z)| lr 5.98e-04 | 322.93 ms | 52.3% bf16 MFU | 1624099 tok/s
step 1356/19560 | loss 4.134597 (+0.40z)| norm 0.5152 (+2.26z)| lr 5.98e-04 | 323.11 ms | 52.2% bf16 MFU | 1624025 tok/s
step 1357/19560 | loss 4.058218 (-1.25z)| norm 0.4679 (+1.42z)| lr 5.98e-04 | 323.13 ms | 52.2% bf16 MFU | 1623949 tok/s
step 1358/19560 | loss 4.114557 (-0.03z)| norm 0.3947 (+0.16z)| lr 5.98e-04 | 322.25 ms | 52.4% bf16 MFU | 1624100 tok/s
step 1359/19560 | loss 4.055529 (-1.29z)| norm 0.4255 (+0.68z)| lr 5.98e-04 | 322.71 ms | 52.3% bf16 MFU | 1624127 tok/s
step 1360/19560 | loss 4.065694 (-1.11z)| norm 0.4300 (+0.74z)| lr 5.98e-04 | 323.00 ms | 52.3% bf16 MFU | 1624080 tok/s
step 1361/19560 | loss 4.025440 (-2.01z)| norm 0.3630 (-0.41z)| lr 5.98e-04 | 322.69 ms | 52.3% bf16 MFU | 1624114 tok/s
step 1362/19560 | loss 4.110214 (-0.04z)| norm 0.3514 (-0.61z)| lr 5.98e-04 | 322.72 ms | 52.3% bf16 MFU | 1624138 tok/s
step 1363/19560 | loss 4.093547 (-0.42z)| norm 0.3261 (-1.03z)| lr 5.98e-04 | 323.04 ms | 52.2% bf16 MFU | 1624082 tok/s
step 1364/19560 | loss 4.030174 (-1.85z)| norm 0.3229 (-1.07z)| lr 5.98e-04 | 323.06 ms | 52.2% bf16 MFU | 1624020 tok/s
step 1365/19560 | loss 4.066306 (-1.01z)| norm 0.2807 (-1.76z)| lr 5.98e-04 | 322.75 ms | 52.3% bf16 MFU | 1624042 tok/s
step 1366/19560 | loss 4.096484 (-0.30z)| norm 0.2627 (-2.02z)| lr 5.98e-04 | 322.56 ms | 52.3% bf16 MFU | 1624110 tok/s
step 1367/19560 | loss 4.088656 (-0.48z)| norm 0.2834 (-1.66z)| lr 5.98e-04 | 322.79 ms | 52.3% bf16 MFU | 1624116 tok/s
step 1368/19560 | loss 4.076690 (-0.74z)| norm 0.2850 (-1.61z)| lr 5.98e-04 | 322.74 ms | 52.3% bf16 MFU | 1624135 tok/s
step 1369/19560 | loss 4.090102 (-0.42z)| norm 0.3470 (-0.60z)| lr 5.98e-04 | 322.62 ms | 52.3% bf16 MFU | 1624182 tok/s
step 1370/19560 | loss 4.050742 (-1.32z)| norm 0.3974 (+0.23z)| lr 5.98e-04 | 322.86 ms | 52.3% bf16 MFU | 1624167 tok/s
step 1371/19560 | loss 4.039556 (-1.56z)| norm 0.3369 (-0.77z)| lr 5.98e-04 | 323.04 ms | 52.2% bf16 MFU | 1624106 tok/s
step 1372/19560 | loss 4.020722 (-1.95z)| norm 0.3300 (-0.88z)| lr 5.98e-04 | 322.77 ms | 52.3% bf16 MFU | 1624118 tok/s
step 1373/19560 | loss 4.053628 (-1.18z)| norm 0.3144 (-1.13z)| lr 5.98e-04 | 322.87 ms | 52.3% bf16 MFU | 1624105 tok/s
step 1374/19560 | loss 4.081979 (-0.52z)| norm 0.3489 (-0.55z)| lr 5.98e-04 | 322.25 ms | 52.4% bf16 MFU | 1624247 tok/s
step 1375/19560 | loss 4.049324 (-1.25z)| norm 0.3638 (-0.29z)| lr 5.98e-04 | 322.97 ms | 52.3% bf16 MFU | 1624203 tok/s
step 1376/19560 | loss 4.049805 (-1.22z)| norm 0.3475 (-0.55z)| lr 5.98e-04 | 323.16 ms | 52.2% bf16 MFU | 1624112 tok/s
step 1377/19560 | loss 4.036278 (-1.50z)| norm 0.3441 (-0.60z)| lr 5.98e-04 | 322.69 ms | 52.3% bf16 MFU | 1624142 tok/s
step 1378/19560 | loss 4.055214 (-1.06z)| norm 0.3613 (-0.29z)| lr 5.98e-04 | 322.64 ms | 52.3% bf16 MFU | 1624185 tok/s
step 1379/19560 | loss 4.144552 (+0.95z)| norm 0.3874 (+0.18z)| lr 5.98e-04 | 323.68 ms | 52.1% bf16 MFU | 1623966 tok/s
step 1380/19560 | loss 4.039429 (-1.39z)| norm 0.3790 (+0.04z)| lr 5.98e-04 | 322.94 ms | 52.3% bf16 MFU | 1623942 tok/s
step 1381/19560 | loss 4.011459 (-1.97z)| norm 0.4386 (+1.10z)| lr 5.98e-04 | 322.87 ms | 52.3% bf16 MFU | 1623936 tok/s
step 1382/19560 | loss 4.121147 (+0.45z)| norm 0.3692 (-0.14z)| lr 5.98e-04 | 322.52 ms | 52.3% bf16 MFU | 1624020 tok/s
step 1383/19560 | loss 3.972503 (-2.72z)| norm 0.3237 (-0.94z)| lr 5.98e-04 | 322.47 ms | 52.3% bf16 MFU | 1624113 tok/s
step 1384/19560 | loss 4.084218 (-0.33z)| norm 0.3589 (-0.31z)| lr 5.98e-04 | 322.84 ms | 52.3% bf16 MFU | 1624106 tok/s
step 1385/19560 | loss 4.048861 (-1.08z)| norm 0.3473 (-0.52z)| lr 5.98e-04 | 322.88 ms | 52.3% bf16 MFU | 1624089 tok/s
step 1386/19560 | loss 4.030810 (-1.44z)| norm 0.3942 (+0.31z)| lr 5.98e-04 | 322.89 ms | 52.3% bf16 MFU | 1624071 tok/s
step 1387/19560 | loss 4.126639 (+0.59z)| norm 0.4331 (+0.99z)| lr 5.98e-04 | 323.28 ms | 52.2% bf16 MFU | 1623957 tok/s
step 1388/19560 | loss 4.097718 (-0.01z)| norm 0.4786 (+1.76z)| lr 5.98e-04 | 322.51 ms | 52.3% bf16 MFU | 1624042 tok/s
step 1389/19560 | loss 4.198313 (+2.10z)| norm 0.4884 (+1.90z)| lr 5.98e-04 | 323.01 ms | 52.2% bf16 MFU | 1623996 tok/s
step 1390/19560 | loss 4.086090 (-0.26z)| norm 0.4371 (+0.99z)| lr 5.98e-04 | 322.88 ms | 52.3% bf16 MFU | 1623985 tok/s
step 1391/19560 | loss 4.140803 (+0.88z)| norm 0.3961 (+0.27z)| lr 5.98e-04 | 323.14 ms | 52.2% bf16 MFU | 1623910 tok/s
step 1392/19560 | loss 4.092006 (-0.13z)| norm 0.3768 (-0.07z)| lr 5.98e-04 | 322.68 ms | 52.3% bf16 MFU | 1623955 tok/s
step 1393/19560 | loss 4.000178 (-2.04z)| norm 0.3900 (+0.15z)| lr 5.98e-04 | 323.26 ms | 52.2% bf16 MFU | 1623850 tok/s
step 1394/19560 | loss 4.012664 (-1.76z)| norm 0.3640 (-0.29z)| lr 5.98e-04 | 323.02 ms | 52.2% bf16 MFU | 1623811 tok/s
step 1395/19560 | loss 4.069551 (-0.54z)| norm 0.3438 (-0.64z)| lr 5.98e-04 | 323.07 ms | 52.2% bf16 MFU | 1623762 tok/s
step 1396/19560 | loss 4.064982 (-0.63z)| norm 0.3348 (-0.78z)| lr 5.98e-04 | 323.56 ms | 52.2% bf16 MFU | 1623593 tok/s
step 1397/19560 | loss 4.069489 (-0.52z)| norm 0.3399 (-0.68z)| lr 5.98e-04 | 322.94 ms | 52.3% bf16 MFU | 1623587 tok/s
step 1398/19560 | loss 4.035017 (-1.24z)| norm 0.3265 (-0.91z)| lr 5.98e-04 | 323.62 ms | 52.2% bf16 MFU | 1623412 tok/s
step 1399/19560 | loss 4.104888 (+0.26z)| norm 0.3471 (-0.52z)| lr 5.98e-04 | 322.27 ms | 52.4% bf16 MFU | 1623586 tok/s
step 1400/19560 | loss 4.065950 (-0.58z)| norm 0.3542 (-0.39z)| lr 5.98e-04 | 322.62 ms | 52.3% bf16 MFU | 1623661 tok/s
step 1401/19560 | loss 4.116630 (+0.50z)| norm 0.3496 (-0.47z)| lr 5.98e-04 | 323.01 ms | 52.3% bf16 MFU | 1623635 tok/s
step 1402/19560 | loss 4.074270 (-0.41z)| norm 0.3626 (-0.24z)| lr 5.98e-04 | 322.77 ms | 52.3% bf16 MFU | 1623671 tok/s
step 1403/19560 | loss 4.112826 (+0.42z)| norm 0.3429 (-0.60z)| lr 5.98e-04 | 323.44 ms | 52.2% bf16 MFU | 1623535 tok/s
step 1404/19560 | loss 4.049541 (-0.93z)| norm 0.3584 (-0.31z)| lr 5.98e-04 | 322.67 ms | 52.3% bf16 MFU | 1623601 tok/s
step 1405/19560 | loss 4.082209 (-0.23z)| norm 0.3372 (-0.68z)| lr 5.98e-04 | 323.40 ms | 52.2% bf16 MFU | 1623479 tok/s
step 1406/19560 | loss 4.017365 (-1.60z)| norm 0.3251 (-0.89z)| lr 5.98e-04 | 323.62 ms | 52.2% bf16 MFU | 1623310 tok/s
step 1407/19560 | loss 4.034731 (-1.21z)| norm 0.3272 (-0.84z)| lr 5.98e-04 | 323.13 ms | 52.2% bf16 MFU | 1623270 tok/s
step 1408/19560 | loss 4.031169 (-1.27z)| norm 0.3282 (-0.81z)| lr 5.98e-04 | 322.89 ms | 52.3% bf16 MFU | 1623293 tok/s
step 1409/19560 | loss 4.041810 (-1.03z)| norm 0.3275 (-0.81z)| lr 5.98e-04 | 322.94 ms | 52.3% bf16 MFU | 1623302 tok/s
step 1410/19560 | loss 4.037273 (-1.10z)| norm 0.3499 (-0.39z)| lr 5.98e-04 | 323.49 ms | 52.2% bf16 MFU | 1623173 tok/s
step 1411/19560 | loss 4.044871 (-0.93z)| norm 0.3925 (+0.40z)| lr 5.98e-04 | 322.80 ms | 52.3% bf16 MFU | 1623223 tok/s
step 1412/19560 | loss 4.093290 (+0.10z)| norm 0.4013 (+0.55z)| lr 5.98e-04 | 322.93 ms | 52.3% bf16 MFU | 1623238 tok/s
step 1413/19560 | loss 4.053787 (-0.73z)| norm 0.3894 (+0.32z)| lr 5.98e-04 | 324.32 ms | 52.0% bf16 MFU | 1622905 tok/s
step 1414/19560 | loss 4.025778 (-1.31z)| norm 0.3633 (-0.16z)| lr 5.98e-04 | 322.74 ms | 52.3% bf16 MFU | 1622984 tok/s
step 1415/19560 | loss 4.055253 (-0.67z)| norm 0.3640 (-0.15z)| lr 5.98e-04 | 323.33 ms | 52.2% bf16 MFU | 1622910 tok/s
step 1416/19560 | loss 4.088930 (+0.05z)| norm 0.3820 (+0.19z)| lr 5.98e-04 | 323.03 ms | 52.2% bf16 MFU | 1622916 tok/s
step 1417/19560 | loss 4.172957 (+1.79z)| norm 0.4069 (+0.66z)| lr 5.98e-04 | 322.49 ms | 52.3% bf16 MFU | 1623056 tok/s
step 1418/19560 | loss 4.097576 (+0.21z)| norm 0.4509 (+1.45z)| lr 5.98e-04 | 322.88 ms | 52.3% bf16 MFU | 1623093 tok/s
step 1419/19560 | loss 4.076322 (-0.23z)| norm 0.5171 (+2.58z)| lr 5.98e-04 | 322.92 ms | 52.3% bf16 MFU | 1623118 tok/s
step 1420/19560 | loss 4.049673 (-0.77z)| norm 0.4778 (+1.83z)| lr 5.98e-04 | 323.10 ms | 52.2% bf16 MFU | 1623097 tok/s
step 1421/19560 | loss 4.100173 (+0.28z)| norm 0.4188 (+0.77z)| lr 5.98e-04 | 322.89 ms | 52.3% bf16 MFU | 1623130 tok/s
step 1422/19560 | loss 4.099229 (+0.26z)| norm 0.4073 (+0.56z)| lr 5.98e-04 | 322.83 ms | 52.3% bf16 MFU | 1623174 tok/s
step 1423/19560 | loss 4.024128 (-1.29z)| norm 0.4275 (+0.91z)| lr 5.98e-04 | 322.29 ms | 52.4% bf16 MFU | 1623353 tok/s
step 1424/19560 | loss 4.095939 (+0.21z)| norm 0.4045 (+0.51z)| lr 5.98e-04 | 323.05 ms | 52.2% bf16 MFU | 1623332 tok/s
step 1425/19560 | loss 4.161427 (+1.55z)| norm 0.3384 (-0.66z)| lr 5.98e-04 | 323.07 ms | 52.2% bf16 MFU | 1623308 tok/s
step 1426/19560 | loss 4.020894 (-1.33z)| norm 0.2972 (-1.38z)| lr 5.98e-04 | 323.05 ms | 52.2% bf16 MFU | 1623288 tok/s
step 1427/19560 | loss 4.061159 (-0.52z)| norm 0.3158 (-1.03z)| lr 5.98e-04 | 322.42 ms | 52.3% bf16 MFU | 1623428 tok/s
step 1428/19560 | loss 4.013459 (-1.49z)| norm 0.3193 (-0.96z)| lr 5.98e-04 | 322.63 ms | 52.3% bf16 MFU | 1623508 tok/s
step 1429/19560 | loss 3.995669 (-1.83z)| norm 0.2979 (-1.33z)| lr 5.98e-04 | 323.46 ms | 52.2% bf16 MFU | 1623377 tok/s
step 1430/19560 | loss 4.048530 (-0.73z)| norm 0.2955 (-1.35z)| lr 5.98e-04 | 322.80 ms | 52.3% bf16 MFU | 1623418 tok/s
step 1431/19560 | loss 4.142686 (+1.19z)| norm 0.3280 (-0.76z)| lr 5.98e-04 | 322.90 ms | 52.3% bf16 MFU | 1623431 tok/s
step 1432/19560 | loss 3.989061 (-1.92z)| norm 0.3451 (-0.45z)| lr 5.98e-04 | 323.38 ms | 52.2% bf16 MFU | 1623323 tok/s
step 1433/19560 | loss 4.030345 (-1.07z)| norm 0.4008 (+0.54z)| lr 5.98e-04 | 323.30 ms | 52.2% bf16 MFU | 1623240 tok/s
step 1434/19560 | loss 4.122947 (+0.82z)| norm 0.3422 (-0.51z)| lr 5.98e-04 | 322.85 ms | 52.3% bf16 MFU | 1623275 tok/s
step 1435/19560 | loss 4.144711 (+1.25z)| norm 0.3605 (-0.18z)| lr 5.98e-04 | 322.85 ms | 52.3% bf16 MFU | 1623308 tok/s
step 1436/19560 | loss 4.085480 (+0.05z)| norm 0.4550 (+1.49z)| lr 5.98e-04 | 322.81 ms | 52.3% bf16 MFU | 1623349 tok/s
step 1437/19560 | loss 4.086898 (+0.07z)| norm 0.4354 (+1.12z)| lr 5.98e-04 | 323.23 ms | 52.2% bf16 MFU | 1623284 tok/s
step 1438/19560 | loss 4.036263 (-0.96z)| norm 0.3667 (-0.10z)| lr 5.98e-04 | 323.13 ms | 52.2% bf16 MFU | 1623247 tok/s
step 1439/19560 | loss 4.014958 (-1.38z)| norm 0.3274 (-0.80z)| lr 5.98e-04 | 323.05 ms | 52.2% bf16 MFU | 1623230 tok/s
step 1440/19560 | loss 4.069127 (-0.26z)| norm 0.3423 (-0.53z)| lr 5.98e-04 | 322.34 ms | 52.4% bf16 MFU | 1623393 tok/s
step 1441/19560 | loss 4.089343 (+0.16z)| norm 0.3433 (-0.50z)| lr 5.98e-04 | 322.38 ms | 52.4% bf16 MFU | 1623540 tok/s
step 1442/19560 | loss 4.026896 (-1.11z)| norm 0.3446 (-0.48z)| lr 5.98e-04 | 322.87 ms | 52.3% bf16 MFU | 1623554 tok/s
step 1443/19560 | loss 4.005475 (-1.53z)| norm 0.3834 (+0.20z)| lr 5.98e-04 | 322.63 ms | 52.3% bf16 MFU | 1623629 tok/s
step 1444/19560 | loss 4.056977 (-0.48z)| norm 0.3827 (+0.18z)| lr 5.98e-04 | 322.42 ms | 52.3% bf16 MFU | 1623752 tok/s
step 1445/19560 | loss 3.998985 (-1.64z)| norm 0.4263 (+0.95z)| lr 5.98e-04 | 322.99 ms | 52.3% bf16 MFU | 1623726 tok/s
step 1446/19560 | loss 4.069960 (-0.21z)| norm 0.3501 (-0.41z)| lr 5.98e-04 | 322.83 ms | 52.3% bf16 MFU | 1623741 tok/s
step 1447/19560 | loss 4.022071 (-1.16z)| norm 0.3219 (-0.90z)| lr 5.98e-04 | 323.13 ms | 52.2% bf16 MFU | 1623680 tok/s
step 1448/19560 | loss 4.057993 (-0.42z)| norm 0.3109 (-1.08z)| lr 5.98e-04 | 322.47 ms | 52.3% bf16 MFU | 1623788 tok/s
step 1449/19560 | loss 4.038275 (-0.81z)| norm 0.3231 (-0.85z)| lr 5.98e-04 | 322.57 ms | 52.3% bf16 MFU | 1623867 tok/s
step 1450/19560 | loss 4.122305 (+0.89z)| norm 0.3322 (-0.67z)| lr 5.98e-04 | 323.10 ms | 52.2% bf16 MFU | 1623807 tok/s
step 1451/19560 | loss 4.086774 (+0.18z)| norm 0.3861 (+0.31z)| lr 5.98e-04 | 323.02 ms | 52.2% bf16 MFU | 1623770 tok/s
step 1452/19560 | loss 4.056293 (-0.43z)| norm 0.4185 (+0.89z)| lr 5.98e-04 | 322.90 ms | 52.3% bf16 MFU | 1623765 tok/s
step 1453/19560 | loss 4.079787 (+0.06z)| norm 0.3935 (+0.44z)| lr 5.98e-04 | 323.24 ms | 52.2% bf16 MFU | 1623675 tok/s
step 1454/19560 | loss 4.055875 (-0.43z)| norm 0.3668 (-0.04z)| lr 5.98e-04 | 323.49 ms | 52.2% bf16 MFU | 1623528 tok/s
step 1455/19560 | loss 4.110554 (+0.70z)| norm 0.3166 (-0.94z)| lr 5.98e-04 | 323.40 ms | 52.2% bf16 MFU | 1623411 tok/s
step 1456/19560 | loss 4.008396 (-1.40z)| norm 0.2852 (-1.49z)| lr 5.98e-04 | 323.33 ms | 52.2% bf16 MFU | 1623317 tok/s
step 1457/19560 | loss 3.987010 (-1.81z)| norm 0.3016 (-1.19z)| lr 5.98e-04 | 323.49 ms | 52.2% bf16 MFU | 1623187 tok/s
step 1458/19560 | loss 4.071054 (-0.05z)| norm 0.3242 (-0.78z)| lr 5.98e-04 | 323.02 ms | 52.2% bf16 MFU | 1623183 tok/s
step 1459/19560 | loss 4.056961 (-0.35z)| norm 0.3164 (-0.92z)| lr 5.98e-04 | 323.08 ms | 52.2% bf16 MFU | 1623162 tok/s
step 1460/19560 | loss 4.102508 (+0.61z)| norm 0.3122 (-0.99z)| lr 5.98e-04 | 323.19 ms | 52.2% bf16 MFU | 1623115 tok/s
step 1461/19560 | loss 4.049548 (-0.51z)| norm 0.3162 (-0.93z)| lr 5.98e-04 | 322.84 ms | 52.3% bf16 MFU | 1623159 tok/s
step 1462/19560 | loss 4.086233 (+0.29z)| norm 0.3198 (-0.87z)| lr 5.98e-04 | 322.65 ms | 52.3% bf16 MFU | 1623247 tok/s
step 1463/19560 | loss 4.070814 (-0.04z)| norm 0.3511 (-0.30z)| lr 5.98e-04 | 322.69 ms | 52.3% bf16 MFU | 1623323 tok/s
step 1464/19560 | loss 4.051999 (-0.43z)| norm 0.3698 (+0.04z)| lr 5.98e-04 | 322.74 ms | 52.3% bf16 MFU | 1623381 tok/s
step 1465/19560 | loss 4.029665 (-0.90z)| norm 0.3699 (+0.04z)| lr 5.98e-04 | 322.94 ms | 52.3% bf16 MFU | 1623386 tok/s
step 1466/19560 | loss 4.006220 (-1.40z)| norm 0.3672 (-0.00z)| lr 5.98e-04 | 322.75 ms | 52.3% bf16 MFU | 1623438 tok/s
step 1467/19560 | loss 4.065162 (-0.10z)| norm 0.3552 (-0.21z)| lr 5.98e-04 | 323.18 ms | 52.2% bf16 MFU | 1623381 tok/s
step 1468/19560 | loss 4.017833 (-1.16z)| norm 0.3782 (+0.23z)| lr 5.98e-04 | 322.54 ms | 52.3% bf16 MFU | 1623486 tok/s
step 1469/19560 | loss 4.015823 (-1.19z)| norm 0.3675 (+0.03z)| lr 5.98e-04 | 323.14 ms | 52.2% bf16 MFU | 1623436 tok/s
step 1470/19560 | loss 4.057711 (-0.22z)| norm 0.3569 (-0.16z)| lr 5.98e-04 | 322.95 ms | 52.3% bf16 MFU | 1623436 tok/s
step 1471/19560 | loss 4.067285 (+0.01z)| norm 0.3411 (-0.45z)| lr 5.98e-04 | 323.43 ms | 52.2% bf16 MFU | 1623317 tok/s
step 1472/19560 | loss 3.983083 (-1.90z)| norm 0.3473 (-0.34z)| lr 5.98e-04 | 322.96 ms | 52.3% bf16 MFU | 1623319 tok/s
step 1473/19560 | loss 3.933501 (-2.93z)| norm 0.3269 (-0.72z)| lr 5.98e-04 | 322.69 ms | 52.3% bf16 MFU | 1623389 tok/s
step 1474/19560 | loss 4.079915 (+0.32z)| norm 0.3188 (-0.86z)| lr 5.98e-04 | 322.54 ms | 52.3% bf16 MFU | 1623494 tok/s
step 1475/19560 | loss 4.026858 (-0.86z)| norm 0.3192 (-0.84z)| lr 5.98e-04 | 322.45 ms | 52.3% bf16 MFU | 1623615 tok/s
step 1476/19560 | loss 4.079443 (+0.31z)| norm 0.3454 (-0.35z)| lr 5.98e-04 | 323.03 ms | 52.2% bf16 MFU | 1623586 tok/s
step 1477/19560 | loss 4.021117 (-0.97z)| norm 0.3780 (+0.24z)| lr 5.97e-04 | 322.73 ms | 52.3% bf16 MFU | 1623635 tok/s
step 1478/19560 | loss 4.009720 (-1.21z)| norm 0.3820 (+0.31z)| lr 5.97e-04 | 322.43 ms | 52.3% bf16 MFU | 1623756 tok/s
step 1479/19560 | loss 4.036293 (-0.61z)| norm 0.3748 (+0.17z)| lr 5.97e-04 | 322.71 ms | 52.3% bf16 MFU | 1623801 tok/s
step 1480/19560 | loss 4.027168 (-0.80z)| norm 0.3744 (+0.15z)| lr 5.97e-04 | 323.15 ms | 52.2% bf16 MFU | 1623731 tok/s
step 1481/19560 | loss 3.981533 (-1.77z)| norm 0.3479 (-0.34z)| lr 5.97e-04 | 322.84 ms | 52.3% bf16 MFU | 1623743 tok/s
step 1482/19560 | loss 4.061494 (-0.01z)| norm 0.3821 (+0.32z)| lr 5.97e-04 | 323.20 ms | 52.2% bf16 MFU | 1623664 tok/s
step 1483/19560 | loss 3.972362 (-2.00z)| norm 0.4244 (+1.27z)| lr 5.97e-04 | 322.72 ms | 52.3% bf16 MFU | 1623709 tok/s
step 1484/19560 | loss 4.001307 (-1.32z)| norm 0.4823 (+2.53z)| lr 5.97e-04 | 322.31 ms | 52.4% bf16 MFU | 1623856 tok/s
step 1485/19560 | loss 4.052230 (-0.16z)| norm 0.4281 (+1.40z)| lr 5.97e-04 | 322.59 ms | 52.3% bf16 MFU | 1623924 tok/s
step 1486/19560 | loss 4.071679 (+0.30z)| norm 0.3311 (-0.68z)| lr 5.97e-04 | 322.84 ms | 52.3% bf16 MFU | 1623928 tok/s
step 1487/19560 | loss 4.108179 (+1.12z)| norm 0.3507 (-0.24z)| lr 5.97e-04 | 322.58 ms | 52.3% bf16 MFU | 1623998 tok/s
step 1488/19560 | loss 4.038028 (-0.48z)| norm 0.3453 (-0.35z)| lr 5.97e-04 | 323.11 ms | 52.2% bf16 MFU | 1623928 tok/s
step 1489/19560 | loss 4.084297 (+0.57z)| norm 0.3554 (-0.13z)| lr 5.97e-04 | 322.58 ms | 52.3% bf16 MFU | 1623998 tok/s
step 1490/19560 | loss 4.053097 (-0.13z)| norm 0.4125 (+1.10z)| lr 5.97e-04 | 322.89 ms | 52.3% bf16 MFU | 1623984 tok/s
step 1491/19560 | loss 4.010375 (-1.10z)| norm 0.4080 (+0.99z)| lr 5.97e-04 | 322.79 ms | 52.3% bf16 MFU | 1623996 tok/s
step 1492/19560 | loss 4.094645 (+0.82z)| norm 0.3581 (-0.10z)| lr 5.97e-04 | 322.31 ms | 52.4% bf16 MFU | 1624128 tok/s
step 1493/19560 | loss 4.067115 (+0.19z)| norm 0.3631 (-0.00z)| lr 5.97e-04 | 322.86 ms | 52.3% bf16 MFU | 1624116 tok/s
step 1494/19560 | loss 4.044972 (-0.31z)| norm 0.3130 (-1.13z)| lr 5.97e-04 | 322.62 ms | 52.3% bf16 MFU | 1624165 tok/s
step 1495/19560 | loss 4.085721 (+0.63z)| norm 0.3105 (-1.20z)| lr 5.97e-04 | 322.57 ms | 52.3% bf16 MFU | 1624224 tok/s
step 1496/19560 | loss 4.057797 (-0.01z)| norm 0.3263 (-0.86z)| lr 5.97e-04 | 322.78 ms | 52.3% bf16 MFU | 1624226 tok/s
step 1497/19560 | loss 4.053786 (-0.09z)| norm 0.3339 (-0.69z)| lr 5.97e-04 | 322.78 ms | 52.3% bf16 MFU | 1624228 tok/s
step 1498/19560 | loss 4.042284 (-0.36z)| norm 0.3315 (-0.73z)| lr 5.97e-04 | 322.73 ms | 52.3% bf16 MFU | 1624243 tok/s
step 1499/19560 | loss 4.054980 (-0.07z)| norm 0.3127 (-1.15z)| lr 5.97e-04 | 322.64 ms | 52.3% bf16 MFU | 1624281 tok/s
step 1500/19560 | loss 4.055475 (-0.06z)| norm 0.2910 (-1.62z)| lr 5.97e-04 | 322.79 ms | 52.3% bf16 MFU | 1624278 tok/s
val loss 4.011433
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2592/10042 = 0.258116
step 1501/19560 | loss 4.159521 (+2.28z)| norm 0.4034 (+0.89z)| lr 5.97e-04 | 322.09 ms | 52.4% bf16 MFU | 1624451 tok/s
step 1502/19560 | loss 3.990705 (-1.52z)| norm 0.3291 (-0.77z)| lr 5.97e-04 | 323.34 ms | 52.2% bf16 MFU | 1624302 tok/s
step 1503/19560 | loss 4.041358 (-0.38z)| norm 0.3859 (+0.50z)| lr 5.97e-04 | 323.17 ms | 52.2% bf16 MFU | 1624203 tok/s
step 1504/19560 | loss 3.992959 (-1.45z)| norm 0.4343 (+1.55z)| lr 5.97e-04 | 323.87 ms | 52.1% bf16 MFU | 1623933 tok/s
step 1505/19560 | loss 3.961343 (-2.10z)| norm 0.5339 (+3.55z)| lr 5.97e-04 | 323.16 ms | 52.2% bf16 MFU | 1623856 tok/s
step 1506/19560 | loss 4.059728 (+0.05z)| norm 0.5041 (+2.80z)| lr 5.97e-04 | 322.74 ms | 52.3% bf16 MFU | 1623887 tok/s
step 1507/19560 | loss 4.048226 (-0.18z)| norm 0.4055 (+0.78z)| lr 5.97e-04 | 323.18 ms | 52.2% bf16 MFU | 1623806 tok/s
step 1508/19560 | loss 4.049907 (-0.15z)| norm 0.4110 (+0.89z)| lr 5.97e-04 | 323.01 ms | 52.3% bf16 MFU | 1623774 tok/s
step 1509/19560 | loss 4.042882 (-0.31z)| norm 0.3609 (-0.12z)| lr 5.97e-04 | 323.35 ms | 52.2% bf16 MFU | 1623658 tok/s
step 1510/19560 | loss 3.994079 (-1.38z)| norm 0.3147 (-1.06z)| lr 5.97e-04 | 323.06 ms | 52.2% bf16 MFU | 1623619 tok/s
step 1511/19560 | loss 3.996654 (-1.34z)| norm 0.3265 (-0.82z)| lr 5.97e-04 | 322.48 ms | 52.3% bf16 MFU | 1623729 tok/s
step 1512/19560 | loss 4.015581 (-0.90z)| norm 0.3089 (-1.16z)| lr 5.97e-04 | 322.96 ms | 52.3% bf16 MFU | 1623711 tok/s
step 1513/19560 | loss 4.018122 (-0.83z)| norm 0.2922 (-1.48z)| lr 5.97e-04 | 322.98 ms | 52.3% bf16 MFU | 1623690 tok/s
step 1514/19560 | loss 4.085457 (+0.67z)| norm 0.2788 (-1.72z)| lr 5.97e-04 | 322.33 ms | 52.4% bf16 MFU | 1623834 tok/s
step 1515/19560 | loss 3.973032 (-1.82z)| norm 0.3051 (-1.17z)| lr 5.97e-04 | 322.92 ms | 52.3% bf16 MFU | 1623821 tok/s
step 1516/19560 | loss 4.023579 (-0.68z)| norm 0.3237 (-0.79z)| lr 5.97e-04 | 323.63 ms | 52.1% bf16 MFU | 1623630 tok/s
step 1517/19560 | loss 4.125345 (+1.66z)| norm 0.3714 (+0.20z)| lr 5.97e-04 | 322.42 ms | 52.3% bf16 MFU | 1623755 tok/s
step 1518/19560 | loss 4.001955 (-1.17z)| norm 0.3693 (+0.17z)| lr 5.97e-04 | 323.06 ms | 52.2% bf16 MFU | 1623712 tok/s
step 1519/19560 | loss 4.035736 (-0.38z)| norm 0.3158 (-0.94z)| lr 5.97e-04 | 323.10 ms | 52.2% bf16 MFU | 1623660 tok/s
step 1520/19560 | loss 3.977811 (-1.70z)| norm 0.3328 (-0.58z)| lr 5.97e-04 | 322.89 ms | 52.3% bf16 MFU | 1623664 tok/s
step 1521/19560 | loss 3.966904 (-1.93z)| norm 0.3748 (+0.31z)| lr 5.97e-04 | 322.45 ms | 52.3% bf16 MFU | 1623778 tok/s
step 1522/19560 | loss 4.022779 (-0.65z)| norm 0.4269 (+1.39z)| lr 5.97e-04 | 322.75 ms | 52.3% bf16 MFU | 1623811 tok/s
step 1523/19560 | loss 4.022706 (-0.64z)| norm 0.4721 (+2.27z)| lr 5.97e-04 | 322.63 ms | 52.3% bf16 MFU | 1623873 tok/s
step 1524/19560 | loss 4.026923 (-0.53z)| norm 0.4967 (+2.68z)| lr 5.97e-04 | 322.29 ms | 52.4% bf16 MFU | 1624016 tok/s
step 1525/19560 | loss 4.003592 (-1.05z)| norm 0.4110 (+0.95z)| lr 5.97e-04 | 323.23 ms | 52.2% bf16 MFU | 1623916 tok/s
step 1526/19560 | loss 4.021257 (-0.65z)| norm 0.3451 (-0.37z)| lr 5.97e-04 | 322.93 ms | 52.3% bf16 MFU | 1623898 tok/s
step 1527/19560 | loss 4.031202 (-0.41z)| norm 0.3179 (-0.90z)| lr 5.97e-04 | 322.78 ms | 52.3% bf16 MFU | 1623917 tok/s
step 1528/19560 | loss 4.041647 (-0.16z)| norm 0.3257 (-0.74z)| lr 5.97e-04 | 322.88 ms | 52.3% bf16 MFU | 1623910 tok/s
step 1529/19560 | loss 4.028530 (-0.45z)| norm 0.3403 (-0.45z)| lr 5.97e-04 | 323.10 ms | 52.2% bf16 MFU | 1623849 tok/s
step 1530/19560 | loss 4.014847 (-0.76z)| norm 0.3172 (-0.90z)| lr 5.97e-04 | 323.17 ms | 52.2% bf16 MFU | 1623774 tok/s
step 1531/19560 | loss 3.942752 (-2.37z)| norm 0.2906 (-1.40z)| lr 5.97e-04 | 323.04 ms | 52.2% bf16 MFU | 1623734 tok/s
step 1532/19560 | loss 4.049927 (+0.08z)| norm 0.2939 (-1.32z)| lr 5.97e-04 | 322.85 ms | 52.3% bf16 MFU | 1623745 tok/s
step 1533/19560 | loss 4.017544 (-0.65z)| norm 0.2962 (-1.26z)| lr 5.97e-04 | 322.61 ms | 52.3% bf16 MFU | 1623815 tok/s
step 1534/19560 | loss 4.004326 (-0.95z)| norm 0.2933 (-1.31z)| lr 5.97e-04 | 322.40 ms | 52.3% bf16 MFU | 1623934 tok/s
step 1535/19560 | loss 4.084575 (+0.88z)| norm 0.3083 (-1.01z)| lr 5.97e-04 | 322.63 ms | 52.3% bf16 MFU | 1623988 tok/s
step 1536/19560 | loss 4.048764 (+0.06z)| norm 0.2864 (-1.42z)| lr 5.97e-04 | 322.69 ms | 52.3% bf16 MFU | 1624025 tok/s
step 1537/19560 | loss 4.078271 (+0.72z)| norm 0.3809 (+0.38z)| lr 5.97e-04 | 322.70 ms | 52.3% bf16 MFU | 1624058 tok/s
step 1538/19560 | loss 3.988653 (-1.30z)| norm 0.3347 (-0.50z)| lr 5.97e-04 | 322.98 ms | 52.3% bf16 MFU | 1624018 tok/s
step 1539/19560 | loss 4.074364 (+0.63z)| norm 0.3536 (-0.13z)| lr 5.97e-04 | 322.56 ms | 52.3% bf16 MFU | 1624086 tok/s
step 1540/19560 | loss 4.009256 (-0.82z)| norm 0.3984 (+0.72z)| lr 5.97e-04 | 323.20 ms | 52.2% bf16 MFU | 1623991 tok/s
step 1541/19560 | loss 4.024399 (-0.48z)| norm 0.4513 (+1.71z)| lr 5.97e-04 | 322.73 ms | 52.3% bf16 MFU | 1624019 tok/s
step 1542/19560 | loss 4.016889 (-0.64z)| norm 0.4290 (+1.27z)| lr 5.97e-04 | 322.40 ms | 52.3% bf16 MFU | 1624128 tok/s
step 1543/19560 | loss 4.030420 (-0.33z)| norm 0.3966 (+0.65z)| lr 5.97e-04 | 322.62 ms | 52.3% bf16 MFU | 1624177 tok/s
step 1544/19560 | loss 4.065173 (+0.46z)| norm 0.4079 (+0.86z)| lr 5.97e-04 | 322.95 ms | 52.3% bf16 MFU | 1624139 tok/s
step 1545/19560 | loss 4.024431 (-0.45z)| norm 0.3410 (-0.39z)| lr 5.97e-04 | 323.23 ms | 52.2% bf16 MFU | 1624033 tok/s
step 1546/19560 | loss 3.985888 (-1.34z)| norm 0.3409 (-0.38z)| lr 5.97e-04 | 322.99 ms | 52.3% bf16 MFU | 1623992 tok/s
step 1547/19560 | loss 3.944701 (-2.24z)| norm 0.3020 (-1.12z)| lr 5.97e-04 | 322.28 ms | 52.4% bf16 MFU | 1624133 tok/s
step 1548/19560 | loss 4.028181 (-0.31z)| norm 0.3132 (-0.89z)| lr 5.97e-04 | 322.74 ms | 52.3% bf16 MFU | 1624152 tok/s
step 1549/19560 | loss 3.957532 (-1.90z)| norm 0.3497 (-0.15z)| lr 5.97e-04 | 323.11 ms | 52.2% bf16 MFU | 1624076 tok/s
step 1550/19560 | loss 4.085234 (+1.02z)| norm 0.3437 (-0.26z)| lr 5.97e-04 | 323.57 ms | 52.2% bf16 MFU | 1623888 tok/s
step 1551/19560 | loss 4.008932 (-0.72z)| norm 0.3551 (-0.02z)| lr 5.97e-04 | 322.92 ms | 52.3% bf16 MFU | 1623872 tok/s
step 1552/19560 | loss 4.042807 (+0.06z)| norm 0.4116 (+1.13z)| lr 5.97e-04 | 322.74 ms | 52.3% bf16 MFU | 1623904 tok/s
step 1553/19560 | loss 4.068636 (+0.70z)| norm 0.3655 (+0.18z)| lr 5.97e-04 | 322.19 ms | 52.4% bf16 MFU | 1624072 tok/s
step 1554/19560 | loss 4.005545 (-0.80z)| norm 0.3091 (-0.96z)| lr 5.97e-04 | 323.25 ms | 52.2% bf16 MFU | 1623965 tok/s
step 1555/19560 | loss 3.998467 (-0.95z)| norm 0.3148 (-0.85z)| lr 5.97e-04 | 322.84 ms | 52.3% bf16 MFU | 1623967 tok/s
step 1556/19560 | loss 3.952214 (-2.00z)| norm 0.2859 (-1.42z)| lr 5.97e-04 | 322.85 ms | 52.3% bf16 MFU | 1623965 tok/s
step 1557/19560 | loss 3.932813 (-2.39z)| norm 0.2675 (-1.78z)| lr 5.97e-04 | 322.54 ms | 52.3% bf16 MFU | 1624042 tok/s
step 1558/19560 | loss 4.036528 (-0.02z)| norm 0.3168 (-0.79z)| lr 5.97e-04 | 322.68 ms | 52.3% bf16 MFU | 1624080 tok/s
step 1559/19560 | loss 3.983883 (-1.22z)| norm 0.3149 (-0.83z)| lr 5.97e-04 | 322.91 ms | 52.3% bf16 MFU | 1624057 tok/s
step 1560/19560 | loss 3.932744 (-2.35z)| norm 0.3162 (-0.80z)| lr 5.97e-04 | 323.59 ms | 52.2% bf16 MFU | 1623864 tok/s
step 1561/19560 | loss 3.994828 (-0.93z)| norm 0.3051 (-1.00z)| lr 5.97e-04 | 322.58 ms | 52.3% bf16 MFU | 1623936 tok/s
step 1562/19560 | loss 4.020146 (-0.34z)| norm 0.3052 (-0.99z)| lr 5.97e-04 | 322.71 ms | 52.3% bf16 MFU | 1623972 tok/s
step 1563/19560 | loss 3.982113 (-1.21z)| norm 0.3182 (-0.72z)| lr 5.97e-04 | 323.50 ms | 52.2% bf16 MFU | 1623806 tok/s
step 1564/19560 | loss 4.003666 (-0.69z)| norm 0.3073 (-0.93z)| lr 5.97e-04 | 322.89 ms | 52.3% bf16 MFU | 1623802 tok/s
step 1565/19560 | loss 4.124558 (+2.13z)| norm 0.2763 (-1.53z)| lr 5.97e-04 | 322.77 ms | 52.3% bf16 MFU | 1623829 tok/s
step 1566/19560 | loss 3.965366 (-1.56z)| norm 0.3157 (-0.73z)| lr 5.97e-04 | 322.80 ms | 52.3% bf16 MFU | 1623847 tok/s
step 1567/19560 | loss 4.020566 (-0.28z)| norm 0.3424 (-0.19z)| lr 5.97e-04 | 322.85 ms | 52.3% bf16 MFU | 1623852 tok/s
step 1568/19560 | loss 4.053872 (+0.49z)| norm 0.3425 (-0.19z)| lr 5.97e-04 | 323.14 ms | 52.2% bf16 MFU | 1623784 tok/s
step 1569/19560 | loss 3.970419 (-1.42z)| norm 0.3323 (-0.39z)| lr 5.97e-04 | 322.52 ms | 52.3% bf16 MFU | 1623873 tok/s
step 1570/19560 | loss 3.997555 (-0.78z)| norm 0.3288 (-0.46z)| lr 5.97e-04 | 322.66 ms | 52.3% bf16 MFU | 1623924 tok/s
step 1571/19560 | loss 4.006727 (-0.57z)| norm 0.3697 (+0.37z)| lr 5.97e-04 | 323.49 ms | 52.2% bf16 MFU | 1623764 tok/s
step 1572/19560 | loss 4.033582 (+0.05z)| norm 0.4407 (+1.78z)| lr 5.97e-04 | 322.91 ms | 52.3% bf16 MFU | 1623758 tok/s
step 1573/19560 | loss 3.952165 (-1.80z)| norm 0.4248 (+1.46z)| lr 5.97e-04 | 322.78 ms | 52.3% bf16 MFU | 1623786 tok/s
step 1574/19560 | loss 4.069812 (+0.89z)| norm 0.3524 (+0.01z)| lr 5.97e-04 | 323.31 ms | 52.2% bf16 MFU | 1623678 tok/s
step 1575/19560 | loss 4.009238 (-0.49z)| norm 0.3859 (+0.67z)| lr 5.97e-04 | 323.25 ms | 52.2% bf16 MFU | 1623590 tok/s
step 1576/19560 | loss 3.968860 (-1.39z)| norm 0.3509 (-0.04z)| lr 5.97e-04 | 322.62 ms | 52.3% bf16 MFU | 1623664 tok/s
step 1577/19560 | loss 4.010723 (-0.44z)| norm 0.2999 (-1.05z)| lr 5.97e-04 | 322.85 ms | 52.3% bf16 MFU | 1623677 tok/s
step 1578/19560 | loss 3.984411 (-1.02z)| norm 0.3143 (-0.76z)| lr 5.97e-04 | 322.98 ms | 52.3% bf16 MFU | 1623658 tok/s
step 1579/19560 | loss 3.948206 (-1.82z)| norm 0.3101 (-0.83z)| lr 5.97e-04 | 322.96 ms | 52.3% bf16 MFU | 1623645 tok/s
step 1580/19560 | loss 4.013085 (-0.33z)| norm 0.3308 (-0.41z)| lr 5.97e-04 | 322.86 ms | 52.3% bf16 MFU | 1623657 tok/s
step 1581/19560 | loss 3.995130 (-0.73z)| norm 0.3425 (-0.17z)| lr 5.97e-04 | 323.62 ms | 52.2% bf16 MFU | 1623479 tok/s
step 1582/19560 | loss 3.936509 (-2.03z)| norm 0.3673 (+0.33z)| lr 5.97e-04 | 322.44 ms | 52.3% bf16 MFU | 1623604 tok/s
step 1583/19560 | loss 4.022608 (-0.06z)| norm 0.3388 (-0.24z)| lr 5.97e-04 | 323.37 ms | 52.2% bf16 MFU | 1623492 tok/s
step 1584/19560 | loss 3.897214 (-2.83z)| norm 0.3388 (-0.25z)| lr 5.97e-04 | 323.48 ms | 52.2% bf16 MFU | 1623355 tok/s
step 1585/19560 | loss 4.025544 (+0.02z)| norm 0.3767 (+0.51z)| lr 5.97e-04 | 322.99 ms | 52.3% bf16 MFU | 1623350 tok/s
step 1586/19560 | loss 3.974672 (-1.10z)| norm 0.3850 (+0.66z)| lr 5.97e-04 | 322.69 ms | 52.3% bf16 MFU | 1623419 tok/s
step 1587/19560 | loss 3.973110 (-1.11z)| norm 0.3574 (+0.10z)| lr 5.97e-04 | 322.87 ms | 52.3% bf16 MFU | 1623440 tok/s
step 1588/19560 | loss 3.984173 (-0.86z)| norm 0.3848 (+0.64z)| lr 5.97e-04 | 323.02 ms | 52.2% bf16 MFU | 1623421 tok/s
step 1589/19560 | loss 4.098431 (+1.69z)| norm 0.3483 (-0.11z)| lr 5.97e-04 | 323.76 ms | 52.1% bf16 MFU | 1623220 tok/s
step 1590/19560 | loss 4.027652 (+0.12z)| norm 0.3505 (-0.07z)| lr 5.97e-04 | 322.89 ms | 52.3% bf16 MFU | 1623245 tok/s
step 1591/19560 | loss 3.976610 (-1.01z)| norm 0.3291 (-0.50z)| lr 5.97e-04 | 322.77 ms | 52.3% bf16 MFU | 1623300 tok/s
step 1592/19560 | loss 3.997116 (-0.54z)| norm 0.3109 (-0.86z)| lr 5.97e-04 | 322.74 ms | 52.3% bf16 MFU | 1623360 tok/s
step 1593/19560 | loss 3.969239 (-1.15z)| norm 0.2914 (-1.24z)| lr 5.97e-04 | 322.86 ms | 52.3% bf16 MFU | 1623388 tok/s
step 1594/19560 | loss 4.030815 (+0.23z)| norm 0.3115 (-0.82z)| lr 5.97e-04 | 323.13 ms | 52.2% bf16 MFU | 1623344 tok/s
step 1595/19560 | loss 3.998285 (-0.49z)| norm 0.3084 (-0.88z)| lr 5.97e-04 | 323.13 ms | 52.2% bf16 MFU | 1623304 tok/s
step 1596/19560 | loss 3.970103 (-1.11z)| norm 0.3005 (-1.02z)| lr 5.97e-04 | 323.39 ms | 52.2% bf16 MFU | 1623201 tok/s
step 1597/19560 | loss 3.962514 (-1.26z)| norm 0.3020 (-0.98z)| lr 5.97e-04 | 322.53 ms | 52.3% bf16 MFU | 1623318 tok/s
step 1598/19560 | loss 4.003498 (-0.34z)| norm 0.3245 (-0.52z)| lr 5.97e-04 | 322.83 ms | 52.3% bf16 MFU | 1623353 tok/s
step 1599/19560 | loss 4.048811 (+0.67z)| norm 0.3091 (-0.82z)| lr 5.97e-04 | 322.95 ms | 52.3% bf16 MFU | 1623358 tok/s
step 1600/19560 | loss 3.958730 (-1.33z)| norm 0.3082 (-0.83z)| lr 5.97e-04 | 323.28 ms | 52.2% bf16 MFU | 1623279 tok/s
step 1601/19560 | loss 3.988135 (-0.70z)| norm 0.2961 (-1.06z)| lr 5.97e-04 | 322.80 ms | 52.3% bf16 MFU | 1623325 tok/s
step 1602/19560 | loss 3.984780 (-0.76z)| norm 0.3078 (-0.83z)| lr 5.97e-04 | 322.77 ms | 52.3% bf16 MFU | 1623375 tok/s
step 1603/19560 | loss 4.010561 (-0.17z)| norm 0.3218 (-0.55z)| lr 5.97e-04 | 323.75 ms | 52.1% bf16 MFU | 1623178 tok/s
step 1604/19560 | loss 4.020858 (+0.07z)| norm 0.3680 (+0.36z)| lr 5.97e-04 | 322.96 ms | 52.3% bf16 MFU | 1623190 tok/s
step 1605/19560 | loss 3.999475 (-0.41z)| norm 0.3460 (-0.07z)| lr 5.97e-04 | 322.82 ms | 52.3% bf16 MFU | 1623236 tok/s
step 1606/19560 | loss 3.953088 (-1.45z)| norm 0.3502 (+0.02z)| lr 5.97e-04 | 323.50 ms | 52.2% bf16 MFU | 1623107 tok/s
step 1607/19560 | loss 4.024491 (+0.17z)| norm 0.3539 (+0.10z)| lr 5.97e-04 | 322.84 ms | 52.3% bf16 MFU | 1623150 tok/s
step 1608/19560 | loss 3.951769 (-1.45z)| norm 0.3833 (+0.68z)| lr 5.97e-04 | 323.27 ms | 52.2% bf16 MFU | 1623084 tok/s
step 1609/19560 | loss 4.013806 (-0.06z)| norm 0.3936 (+0.88z)| lr 5.97e-04 | 323.26 ms | 52.2% bf16 MFU | 1623023 tok/s
step 1610/19560 | loss 3.975573 (-0.91z)| norm 0.3950 (+0.90z)| lr 5.97e-04 | 323.15 ms | 52.2% bf16 MFU | 1622993 tok/s
step 1611/19560 | loss 4.029557 (+0.30z)| norm 0.3967 (+0.95z)| lr 5.97e-04 | 322.85 ms | 52.3% bf16 MFU | 1623040 tok/s
step 1612/19560 | loss 4.010869 (-0.13z)| norm 0.3250 (-0.47z)| lr 5.97e-04 | 323.41 ms | 52.2% bf16 MFU | 1622944 tok/s
step 1613/19560 | loss 3.943769 (-1.61z)| norm 0.2905 (-1.16z)| lr 5.97e-04 | 322.82 ms | 52.3% bf16 MFU | 1623001 tok/s
step 1614/19560 | loss 3.991206 (-0.54z)| norm 0.3072 (-0.81z)| lr 5.97e-04 | 323.11 ms | 52.2% bf16 MFU | 1622982 tok/s
step 1615/19560 | loss 3.957269 (-1.29z)| norm 0.2884 (-1.18z)| lr 5.97e-04 | 323.77 ms | 52.1% bf16 MFU | 1622800 tok/s
step 1616/19560 | loss 4.023487 (+0.22z)| norm 0.2787 (-1.36z)| lr 5.97e-04 | 323.05 ms | 52.2% bf16 MFU | 1622807 tok/s
step 1617/19560 | loss 3.986166 (-0.62z)| norm 0.3023 (-0.87z)| lr 5.97e-04 | 322.91 ms | 52.3% bf16 MFU | 1622847 tok/s
step 1618/19560 | loss 4.073366 (+1.38z)| norm 0.3012 (-0.88z)| lr 5.97e-04 | 323.98 ms | 52.1% bf16 MFU | 1622619 tok/s
step 1619/19560 | loss 3.983947 (-0.66z)| norm 0.3256 (-0.37z)| lr 5.96e-04 | 323.50 ms | 52.2% bf16 MFU | 1622521 tok/s
step 1620/19560 | loss 3.931635 (-1.83z)| norm 0.3206 (-0.47z)| lr 5.96e-04 | 323.36 ms | 52.2% bf16 MFU | 1622462 tok/s
step 1621/19560 | loss 4.033907 (+0.52z)| norm 0.3491 (+0.12z)| lr 5.96e-04 | 323.66 ms | 52.1% bf16 MFU | 1622334 tok/s
step 1622/19560 | loss 3.982190 (-0.66z)| norm 0.3873 (+0.88z)| lr 5.96e-04 | 323.16 ms | 52.2% bf16 MFU | 1622335 tok/s
step 1623/19560 | loss 3.973523 (-0.85z)| norm 0.4696 (+2.48z)| lr 5.96e-04 | 323.16 ms | 52.2% bf16 MFU | 1622338 tok/s
step 1624/19560 | loss 3.940369 (-1.59z)| norm 0.4894 (+2.76z)| lr 5.96e-04 | 322.05 ms | 52.4% bf16 MFU | 1622620 tok/s
step 1625/19560 | loss 3.978081 (-0.70z)| norm 0.4086 (+1.18z)| lr 5.96e-04 | 322.97 ms | 52.3% bf16 MFU | 1622656 tok/s
step 1626/19560 | loss 3.967330 (-0.94z)| norm 0.3592 (+0.23z)| lr 5.96e-04 | 323.21 ms | 52.2% bf16 MFU | 1622628 tok/s
step 1627/19560 | loss 3.956390 (-1.17z)| norm 0.3250 (-0.43z)| lr 5.96e-04 | 322.56 ms | 52.3% bf16 MFU | 1622766 tok/s
step 1628/19560 | loss 3.981700 (-0.58z)| norm 0.3358 (-0.23z)| lr 5.96e-04 | 322.39 ms | 52.4% bf16 MFU | 1622941 tok/s
step 1629/19560 | loss 4.023356 (+0.43z)| norm 0.3289 (-0.36z)| lr 5.96e-04 | 322.38 ms | 52.4% bf16 MFU | 1623108 tok/s
step 1630/19560 | loss 3.968607 (-0.89z)| norm 0.3100 (-0.72z)| lr 5.96e-04 | 322.53 ms | 52.3% bf16 MFU | 1623230 tok/s
step 1631/19560 | loss 3.978902 (-0.63z)| norm 0.3035 (-0.83z)| lr 5.96e-04 | 323.04 ms | 52.2% bf16 MFU | 1623217 tok/s
step 1632/19560 | loss 3.949059 (-1.34z)| norm 0.2993 (-0.90z)| lr 5.96e-04 | 323.19 ms | 52.2% bf16 MFU | 1623168 tok/s
step 1633/19560 | loss 4.007390 (+0.06z)| norm 0.2798 (-1.31z)| lr 5.96e-04 | 322.90 ms | 52.3% bf16 MFU | 1623194 tok/s
step 1634/19560 | loss 3.980777 (-0.57z)| norm 0.3227 (-0.41z)| lr 5.96e-04 | 322.30 ms | 52.4% bf16 MFU | 1623368 tok/s
step 1635/19560 | loss 4.029692 (+0.63z)| norm 0.3390 (-0.05z)| lr 5.96e-04 | 323.18 ms | 52.2% bf16 MFU | 1623314 tok/s
step 1636/19560 | loss 4.027854 (+0.59z)| norm 0.3384 (-0.05z)| lr 5.96e-04 | 322.47 ms | 52.3% bf16 MFU | 1623441 tok/s
step 1637/19560 | loss 3.940619 (-1.53z)| norm 0.3717 (+0.67z)| lr 5.96e-04 | 323.38 ms | 52.2% bf16 MFU | 1623332 tok/s
step 1638/19560 | loss 4.008698 (+0.13z)| norm 0.4178 (+1.64z)| lr 5.96e-04 | 322.88 ms | 52.3% bf16 MFU | 1623354 tok/s
step 1639/19560 | loss 3.901658 (-2.41z)| norm 0.4008 (+1.26z)| lr 5.96e-04 | 322.81 ms | 52.3% bf16 MFU | 1623394 tok/s
step 1640/19560 | loss 3.983058 (-0.46z)| norm 0.3526 (+0.21z)| lr 5.96e-04 | 323.18 ms | 52.2% bf16 MFU | 1623338 tok/s
step 1641/19560 | loss 4.200288 (+4.34z)| norm 0.3666 (+0.50z)| lr 5.96e-04 | 323.25 ms | 52.2% bf16 MFU | 1623266 tok/s
step 1642/19560 | loss 3.935713 (-1.48z)| norm 0.3090 (-0.75z)| lr 5.96e-04 | 322.78 ms | 52.3% bf16 MFU | 1623319 tok/s
step 1643/19560 | loss 4.013534 (+0.24z)| norm 0.3405 (-0.07z)| lr 5.96e-04 | 322.75 ms | 52.3% bf16 MFU | 1623375 tok/s
step 1644/19560 | loss 3.966593 (-0.79z)| norm 0.3027 (-0.89z)| lr 5.96e-04 | 323.05 ms | 52.2% bf16 MFU | 1623353 tok/s
step 1645/19560 | loss 4.058642 (+1.29z)| norm 0.3842 (+0.88z)| lr 5.96e-04 | 323.12 ms | 52.2% bf16 MFU | 1623316 tok/s
step 1646/19560 | loss 3.960019 (-0.94z)| norm 0.3023 (-0.89z)| lr 5.96e-04 | 322.78 ms | 52.3% bf16 MFU | 1623364 tok/s
step 1647/19560 | loss 3.986022 (-0.34z)| norm 0.3045 (-0.84z)| lr 5.96e-04 | 322.62 ms | 52.3% bf16 MFU | 1623450 tok/s
step 1648/19560 | loss 3.977446 (-0.54z)| norm 0.3117 (-0.68z)| lr 5.96e-04 | 322.39 ms | 52.4% bf16 MFU | 1623591 tok/s
step 1649/19560 | loss 4.036729 (+0.79z)| norm 0.3265 (-0.35z)| lr 5.96e-04 | 323.03 ms | 52.2% bf16 MFU | 1623564 tok/s
step 1650/19560 | loss 4.039316 (+0.85z)| norm 0.3486 (+0.15z)| lr 5.96e-04 | 322.95 ms | 52.3% bf16 MFU | 1623556 tok/s
step 1651/19560 | loss 3.995047 (-0.15z)| norm 0.4114 (+1.57z)| lr 5.96e-04 | 322.53 ms | 52.3% bf16 MFU | 1623657 tok/s
step 1652/19560 | loss 3.965794 (-0.80z)| norm 0.4220 (+1.89z)| lr 5.96e-04 | 322.45 ms | 52.3% bf16 MFU | 1623772 tok/s
step 1653/19560 | loss 3.975358 (-0.58z)| norm 0.4498 (+2.49z)| lr 5.96e-04 | 323.23 ms | 52.2% bf16 MFU | 1623685 tok/s
step 1654/19560 | loss 3.953948 (-1.04z)| norm 0.4460 (+2.33z)| lr 5.96e-04 | 322.28 ms | 52.4% bf16 MFU | 1623840 tok/s
step 1655/19560 | loss 4.025266 (+0.56z)| norm 0.3799 (+0.84z)| lr 5.96e-04 | 323.06 ms | 52.2% bf16 MFU | 1623793 tok/s
step 1656/19560 | loss 4.040297 (+0.90z)| norm 0.3859 (+0.96z)| lr 5.96e-04 | 322.20 ms | 52.4% bf16 MFU | 1623963 tok/s
step 1657/19560 | loss 3.969197 (-0.69z)| norm 0.3242 (-0.41z)| lr 5.96e-04 | 323.31 ms | 52.2% bf16 MFU | 1623847 tok/s
step 1658/19560 | loss 4.037359 (+0.84z)| norm 0.2946 (-1.07z)| lr 5.96e-04 | 323.19 ms | 52.2% bf16 MFU | 1623766 tok/s
step 1659/19560 | loss 4.005958 (+0.12z)| norm 0.2933 (-1.10z)| lr 5.96e-04 | 322.76 ms | 52.3% bf16 MFU | 1623798 tok/s
step 1660/19560 | loss 4.047435 (+1.06z)| norm 0.3147 (-0.63z)| lr 5.96e-04 | 322.68 ms | 52.3% bf16 MFU | 1623848 tok/s
step 1661/19560 | loss 4.005736 (+0.12z)| norm 0.3496 (+0.14z)| lr 5.96e-04 | 322.52 ms | 52.3% bf16 MFU | 1623936 tok/s
step 1662/19560 | loss 4.209295 (+4.33z)| norm 0.3669 (+0.52z)| lr 5.96e-04 | 322.66 ms | 52.3% bf16 MFU | 1623984 tok/s
step 1663/19560 | loss 3.986740 (-0.31z)| norm 0.3797 (+0.80z)| lr 5.96e-04 | 322.33 ms | 52.4% bf16 MFU | 1624112 tok/s
step 1664/19560 | loss 4.013573 (+0.27z)| norm 0.3281 (-0.38z)| lr 5.96e-04 | 322.44 ms | 52.3% bf16 MFU | 1624208 tok/s
step 1665/19560 | loss 3.989362 (-0.23z)| norm 0.2948 (-1.11z)| lr 5.96e-04 | 322.77 ms | 52.3% bf16 MFU | 1624214 tok/s
step 1666/19560 | loss 3.928823 (-1.51z)| norm 0.3292 (-0.33z)| lr 5.96e-04 | 323.21 ms | 52.2% bf16 MFU | 1624110 tok/s
step 1667/19560 | loss 3.965026 (-0.73z)| norm 0.3149 (-0.65z)| lr 5.96e-04 | 322.55 ms | 52.3% bf16 MFU | 1624176 tok/s
step 1668/19560 | loss 3.982849 (-0.34z)| norm 0.2976 (-1.02z)| lr 5.96e-04 | 322.33 ms | 52.4% bf16 MFU | 1624294 tok/s
step 1669/19560 | loss 4.001373 (+0.06z)| norm 0.2872 (-1.25z)| lr 5.96e-04 | 323.04 ms | 52.2% bf16 MFU | 1624228 tok/s
step 1670/19560 | loss 3.954597 (-0.93z)| norm 0.3406 (-0.01z)| lr 5.96e-04 | 322.97 ms | 52.3% bf16 MFU | 1624184 tok/s
step 1671/19560 | loss 3.995460 (-0.05z)| norm 0.3653 (+0.58z)| lr 5.96e-04 | 322.69 ms | 52.3% bf16 MFU | 1624213 tok/s
step 1672/19560 | loss 3.946181 (-1.09z)| norm 0.3702 (+0.71z)| lr 5.96e-04 | 322.18 ms | 52.4% bf16 MFU | 1624369 tok/s
step 1673/19560 | loss 3.939324 (-1.22z)| norm 0.3331 (-0.17z)| lr 5.96e-04 | 322.92 ms | 52.3% bf16 MFU | 1624330 tok/s
step 1674/19560 | loss 3.945903 (-1.07z)| norm 0.3285 (-0.28z)| lr 5.96e-04 | 322.54 ms | 52.3% bf16 MFU | 1624388 tok/s
step 1675/19560 | loss 4.016770 (+0.43z)| norm 0.3218 (-0.44z)| lr 5.96e-04 | 322.58 ms | 52.3% bf16 MFU | 1624433 tok/s
step 1676/19560 | loss 3.982213 (-0.30z)| norm 0.3783 (+0.89z)| lr 5.96e-04 | 322.27 ms | 52.4% bf16 MFU | 1624554 tok/s
step 1677/19560 | loss 3.935833 (-1.28z)| norm 0.3172 (-0.56z)| lr 5.96e-04 | 322.96 ms | 52.3% bf16 MFU | 1624496 tok/s
step 1678/19560 | loss 3.907446 (-1.86z)| norm 0.3104 (-0.71z)| lr 5.96e-04 | 322.61 ms | 52.3% bf16 MFU | 1624528 tok/s
step 1679/19560 | loss 3.887442 (-2.22z)| norm 0.3098 (-0.71z)| lr 5.96e-04 | 322.68 ms | 52.3% bf16 MFU | 1624542 tok/s
step 1680/19560 | loss 4.061501 (+1.41z)| norm 0.2991 (-0.95z)| lr 5.96e-04 | 322.48 ms | 52.3% bf16 MFU | 1624605 tok/s
step 1681/19560 | loss 3.983044 (-0.21z)| norm 0.3036 (-0.83z)| lr 5.96e-04 | 322.29 ms | 52.4% bf16 MFU | 1624712 tok/s
step 1682/19560 | loss 3.982893 (-0.21z)| norm 0.3371 (-0.04z)| lr 5.96e-04 | 322.76 ms | 52.3% bf16 MFU | 1624696 tok/s
step 1683/19560 | loss 3.904428 (-1.82z)| norm 0.3266 (-0.30z)| lr 5.96e-04 | 322.36 ms | 52.4% bf16 MFU | 1624782 tok/s
step 1684/19560 | loss 3.947376 (-0.93z)| norm 0.3155 (-0.57z)| lr 5.96e-04 | 322.94 ms | 52.3% bf16 MFU | 1624716 tok/s
step 1685/19560 | loss 3.989165 (-0.07z)| norm 0.2847 (-1.31z)| lr 5.96e-04 | 322.85 ms | 52.3% bf16 MFU | 1624678 tok/s
step 1686/19560 | loss 4.164039 (+3.40z)| norm 0.3062 (-0.79z)| lr 5.96e-04 | 322.29 ms | 52.4% bf16 MFU | 1624781 tok/s
step 1687/19560 | loss 3.964754 (-0.57z)| norm 0.3215 (-0.43z)| lr 5.96e-04 | 322.69 ms | 52.3% bf16 MFU | 1624778 tok/s
step 1688/19560 | loss 3.944545 (-0.98z)| norm 0.2993 (-0.96z)| lr 5.96e-04 | 322.75 ms | 52.3% bf16 MFU | 1624762 tok/s
step 1689/19560 | loss 3.911718 (-1.61z)| norm 0.3027 (-0.87z)| lr 5.96e-04 | 322.98 ms | 52.3% bf16 MFU | 1624687 tok/s
step 1690/19560 | loss 3.981247 (-0.22z)| norm 0.3025 (-0.88z)| lr 5.96e-04 | 322.61 ms | 52.3% bf16 MFU | 1624709 tok/s
step 1691/19560 | loss 3.959027 (-0.66z)| norm 0.3044 (-0.83z)| lr 5.96e-04 | 322.40 ms | 52.3% bf16 MFU | 1624784 tok/s
step 1692/19560 | loss 3.964952 (-0.54z)| norm 0.3057 (-0.80z)| lr 5.96e-04 | 322.73 ms | 52.3% bf16 MFU | 1624770 tok/s
step 1693/19560 | loss 3.961044 (-0.60z)| norm 0.3389 (-0.02z)| lr 5.96e-04 | 322.63 ms | 52.3% bf16 MFU | 1624784 tok/s
step 1694/19560 | loss 3.966521 (-0.49z)| norm 0.3542 (+0.35z)| lr 5.96e-04 | 322.57 ms | 52.3% bf16 MFU | 1624811 tok/s
step 1695/19560 | loss 3.990681 (+0.00z)| norm 0.3797 (+0.95z)| lr 5.96e-04 | 322.74 ms | 52.3% bf16 MFU | 1624795 tok/s
step 1696/19560 | loss 4.024958 (+0.71z)| norm 0.3754 (+0.84z)| lr 5.96e-04 | 322.91 ms | 52.3% bf16 MFU | 1624738 tok/s
step 1697/19560 | loss 3.977563 (-0.26z)| norm 0.3555 (+0.36z)| lr 5.96e-04 | 322.43 ms | 52.3% bf16 MFU | 1624804 tok/s
step 1698/19560 | loss 3.971622 (-0.38z)| norm 0.3278 (-0.31z)| lr 5.96e-04 | 322.87 ms | 52.3% bf16 MFU | 1624754 tok/s
step 1699/19560 | loss 3.952085 (-0.77z)| norm 0.3207 (-0.47z)| lr 5.96e-04 | 322.87 ms | 52.3% bf16 MFU | 1624709 tok/s
step 1700/19560 | loss 3.932538 (-1.15z)| norm 0.3110 (-0.69z)| lr 5.96e-04 | 322.59 ms | 52.3% bf16 MFU | 1624737 tok/s
step 1701/19560 | loss 3.897227 (-1.84z)| norm 0.3154 (-0.57z)| lr 5.96e-04 | 322.69 ms | 52.3% bf16 MFU | 1624737 tok/s
step 1702/19560 | loss 4.001628 (+0.28z)| norm 0.3179 (-0.50z)| lr 5.96e-04 | 322.88 ms | 52.3% bf16 MFU | 1624690 tok/s
step 1703/19560 | loss 3.922822 (-1.30z)| norm 0.3271 (-0.26z)| lr 5.96e-04 | 322.76 ms | 52.3% bf16 MFU | 1624673 tok/s
step 1704/19560 | loss 3.975940 (-0.23z)| norm 0.3281 (-0.23z)| lr 5.96e-04 | 323.14 ms | 52.2% bf16 MFU | 1624562 tok/s
step 1705/19560 | loss 3.962553 (-0.49z)| norm 0.3716 (+0.84z)| lr 5.96e-04 | 322.77 ms | 52.3% bf16 MFU | 1624552 tok/s
step 1706/19560 | loss 3.956603 (-0.61z)| norm 0.3726 (+0.86z)| lr 5.96e-04 | 322.99 ms | 52.3% bf16 MFU | 1624487 tok/s
step 1707/19560 | loss 3.948868 (-0.77z)| norm 0.3376 (-0.02z)| lr 5.96e-04 | 323.01 ms | 52.2% bf16 MFU | 1624418 tok/s
step 1708/19560 | loss 3.971791 (-0.30z)| norm 0.3265 (-0.30z)| lr 5.96e-04 | 322.71 ms | 52.3% bf16 MFU | 1624429 tok/s
step 1709/19560 | loss 3.948129 (-0.77z)| norm 0.3272 (-0.28z)| lr 5.96e-04 | 323.45 ms | 52.2% bf16 MFU | 1624253 tok/s
step 1710/19560 | loss 3.974335 (-0.24z)| norm 0.3181 (-0.50z)| lr 5.96e-04 | 322.51 ms | 52.3% bf16 MFU | 1624322 tok/s
step 1711/19560 | loss 4.030900 (+0.90z)| norm 0.3418 (+0.09z)| lr 5.96e-04 | 323.44 ms | 52.2% bf16 MFU | 1624154 tok/s
step 1712/19560 | loss 3.956485 (-0.62z)| norm 0.3346 (-0.09z)| lr 5.96e-04 | 322.80 ms | 52.3% bf16 MFU | 1624155 tok/s
step 1713/19560 | loss 3.945597 (-0.83z)| norm 0.3096 (-0.70z)| lr 5.96e-04 | 322.68 ms | 52.3% bf16 MFU | 1624186 tok/s
step 1714/19560 | loss 3.966821 (-0.40z)| norm 0.3189 (-0.46z)| lr 5.96e-04 | 323.39 ms | 52.2% bf16 MFU | 1624038 tok/s
step 1715/19560 | loss 3.933908 (-1.06z)| norm 0.2935 (-1.08z)| lr 5.96e-04 | 322.81 ms | 52.3% bf16 MFU | 1624042 tok/s
step 1716/19560 | loss 3.923087 (-1.26z)| norm 0.2691 (-1.67z)| lr 5.96e-04 | 322.12 ms | 52.4% bf16 MFU | 1624221 tok/s
step 1717/19560 | loss 3.919364 (-1.33z)| norm 0.2898 (-1.13z)| lr 5.96e-04 | 322.87 ms | 52.3% bf16 MFU | 1624202 tok/s
step 1718/19560 | loss 3.939133 (-0.91z)| norm 0.2798 (-1.36z)| lr 5.96e-04 | 323.07 ms | 52.2% bf16 MFU | 1624132 tok/s
step 1719/19560 | loss 3.951615 (-0.65z)| norm 0.3092 (-0.62z)| lr 5.96e-04 | 323.17 ms | 52.2% bf16 MFU | 1624042 tok/s
step 1720/19560 | loss 3.954625 (-0.58z)| norm 0.2833 (-1.25z)| lr 5.96e-04 | 322.79 ms | 52.3% bf16 MFU | 1624051 tok/s
step 1721/19560 | loss 3.901308 (-1.64z)| norm 0.2874 (-1.15z)| lr 5.96e-04 | 323.37 ms | 52.2% bf16 MFU | 1623915 tok/s
step 1722/19560 | loss 3.955902 (-0.52z)| norm 0.2998 (-0.84z)| lr 5.96e-04 | 322.29 ms | 52.4% bf16 MFU | 1624058 tok/s
step 1723/19560 | loss 3.911971 (-1.39z)| norm 0.3074 (-0.66z)| lr 5.96e-04 | 322.48 ms | 52.3% bf16 MFU | 1624146 tok/s
step 1724/19560 | loss 3.869134 (-2.19z)| norm 0.3143 (-0.49z)| lr 5.96e-04 | 323.08 ms | 52.2% bf16 MFU | 1624078 tok/s
step 1725/19560 | loss 3.977250 (-0.06z)| norm 0.3322 (-0.06z)| lr 5.96e-04 | 322.55 ms | 52.3% bf16 MFU | 1624145 tok/s
step 1726/19560 | loss 3.953190 (-0.53z)| norm 0.3428 (+0.20z)| lr 5.96e-04 | 322.99 ms | 52.3% bf16 MFU | 1624100 tok/s
step 1727/19560 | loss 3.952786 (-0.53z)| norm 0.3804 (+1.11z)| lr 5.96e-04 | 322.69 ms | 52.3% bf16 MFU | 1624132 tok/s
step 1728/19560 | loss 3.955787 (-0.47z)| norm 0.3658 (+0.74z)| lr 5.96e-04 | 322.65 ms | 52.3% bf16 MFU | 1624173 tok/s
step 1729/19560 | loss 3.859198 (-2.31z)| norm 0.3443 (+0.21z)| lr 5.96e-04 | 322.81 ms | 52.3% bf16 MFU | 1624170 tok/s
step 1730/19560 | loss 4.001730 (+0.45z)| norm 0.3605 (+0.59z)| lr 5.96e-04 | 322.62 ms | 52.3% bf16 MFU | 1624217 tok/s
step 1731/19560 | loss 3.985606 (+0.14z)| norm 0.3784 (+1.02z)| lr 5.96e-04 | 323.04 ms | 52.2% bf16 MFU | 1624156 tok/s
step 1732/19560 | loss 4.035286 (+1.11z)| norm 0.3443 (+0.19z)| lr 5.96e-04 | 322.78 ms | 52.3% bf16 MFU | 1624162 tok/s
step 1733/19560 | loss 3.917593 (-1.16z)| norm 0.3304 (-0.15z)| lr 5.96e-04 | 322.63 ms | 52.3% bf16 MFU | 1624205 tok/s
step 1734/19560 | loss 3.992927 (+0.29z)| norm 0.3264 (-0.25z)| lr 5.96e-04 | 323.02 ms | 52.2% bf16 MFU | 1624149 tok/s
step 1735/19560 | loss 3.938751 (-0.75z)| norm 0.3576 (+0.52z)| lr 5.96e-04 | 322.50 ms | 52.3% bf16 MFU | 1624227 tok/s
step 1736/19560 | loss 4.001000 (+0.45z)| norm 0.3323 (-0.09z)| lr 5.96e-04 | 323.08 ms | 52.2% bf16 MFU | 1624154 tok/s
step 1737/19560 | loss 3.943174 (-0.66z)| norm 0.3183 (-0.43z)| lr 5.96e-04 | 322.75 ms | 52.3% bf16 MFU | 1624169 tok/s
step 1738/19560 | loss 3.925190 (-0.99z)| norm 0.3005 (-0.86z)| lr 5.96e-04 | 322.57 ms | 52.3% bf16 MFU | 1624228 tok/s
step 1739/19560 | loss 3.994501 (+0.35z)| norm 0.3321 (-0.05z)| lr 5.96e-04 | 322.49 ms | 52.3% bf16 MFU | 1624304 tok/s
step 1740/19560 | loss 3.994535 (+0.35z)| norm 0.3039 (-0.76z)| lr 5.96e-04 | 322.94 ms | 52.3% bf16 MFU | 1624263 tok/s
step 1741/19560 | loss 3.909254 (-1.29z)| norm 0.3109 (-0.59z)| lr 5.96e-04 | 322.67 ms | 52.3% bf16 MFU | 1624291 tok/s
step 1742/19560 | loss 3.933182 (-0.82z)| norm 0.2984 (-0.90z)| lr 5.96e-04 | 322.50 ms | 52.3% bf16 MFU | 1624362 tok/s
step 1743/19560 | loss 3.964703 (-0.21z)| norm 0.2680 (-1.66z)| lr 5.95e-04 | 323.20 ms | 52.2% bf16 MFU | 1624252 tok/s
step 1744/19560 | loss 3.917544 (-1.10z)| norm 0.3057 (-0.72z)| lr 5.95e-04 | 322.79 ms | 52.3% bf16 MFU | 1624252 tok/s
step 1745/19560 | loss 4.047411 (+1.37z)| norm 0.3056 (-0.72z)| lr 5.95e-04 | 322.45 ms | 52.3% bf16 MFU | 1624337 tok/s
step 1746/19560 | loss 3.919715 (-1.05z)| norm 0.3095 (-0.63z)| lr 5.95e-04 | 323.02 ms | 52.2% bf16 MFU | 1624274 tok/s
step 1747/19560 | loss 3.946610 (-0.52z)| norm 0.3109 (-0.59z)| lr 5.95e-04 | 322.68 ms | 52.3% bf16 MFU | 1624300 tok/s
step 1748/19560 | loss 4.028149 (+1.03z)| norm 0.3109 (-0.59z)| lr 5.95e-04 | 322.23 ms | 52.4% bf16 MFU | 1624438 tok/s
step 1749/19560 | loss 4.002176 (+0.54z)| norm 0.3373 (+0.08z)| lr 5.95e-04 | 322.77 ms | 52.3% bf16 MFU | 1624434 tok/s
step 1750/19560 | loss 3.914291 (-1.14z)| norm 0.3826 (+1.24z)| lr 5.95e-04 | 323.36 ms | 52.2% bf16 MFU | 1624282 tok/s
val loss 3.932409
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2574/10042 = 0.256323
step 1751/19560 | loss 3.973516 (-0.01z)| norm 0.3537 (+0.55z)| lr 5.95e-04 | 321.96 ms | 52.4% bf16 MFU | 1624490 tok/s
step 1752/19560 | loss 4.047817 (+1.40z)| norm 0.3508 (+0.54z)| lr 5.95e-04 | 322.62 ms | 52.3% bf16 MFU | 1624519 tok/s
step 1753/19560 | loss 3.914773 (-1.13z)| norm 0.3371 (+0.17z)| lr 5.95e-04 | 323.73 ms | 52.1% bf16 MFU | 1624270 tok/s
step 1754/19560 | loss 3.913014 (-1.15z)| norm 0.3311 (+0.00z)| lr 5.95e-04 | 323.02 ms | 52.2% bf16 MFU | 1624210 tok/s
step 1755/19560 | loss 3.929302 (-0.83z)| norm 0.3238 (-0.21z)| lr 5.95e-04 | 322.85 ms | 52.3% bf16 MFU | 1624198 tok/s
step 1756/19560 | loss 3.968888 (-0.09z)| norm 0.3441 (+0.38z)| lr 5.95e-04 | 322.72 ms | 52.3% bf16 MFU | 1624218 tok/s
step 1757/19560 | loss 3.887860 (-1.58z)| norm 0.3715 (+1.16z)| lr 5.95e-04 | 322.83 ms | 52.3% bf16 MFU | 1624210 tok/s
step 1758/19560 | loss 3.918562 (-1.00z)| norm 0.3692 (+1.08z)| lr 5.95e-04 | 322.87 ms | 52.3% bf16 MFU | 1624190 tok/s
step 1759/19560 | loss 3.916976 (-1.01z)| norm 0.3794 (+1.35z)| lr 5.95e-04 | 323.20 ms | 52.2% bf16 MFU | 1624089 tok/s
step 1760/19560 | loss 3.972547 (+0.02z)| norm 0.3431 (+0.29z)| lr 5.95e-04 | 322.78 ms | 52.3% bf16 MFU | 1624100 tok/s
step 1761/19560 | loss 4.004533 (+0.61z)| norm 0.2857 (-1.37z)| lr 5.95e-04 | 323.39 ms | 52.2% bf16 MFU | 1623957 tok/s
step 1762/19560 | loss 3.910335 (-1.12z)| norm 0.2769 (-1.60z)| lr 5.95e-04 | 322.64 ms | 52.3% bf16 MFU | 1624008 tok/s
step 1763/19560 | loss 3.939862 (-0.57z)| norm 0.3111 (-0.61z)| lr 5.95e-04 | 323.19 ms | 52.2% bf16 MFU | 1623919 tok/s
step 1764/19560 | loss 3.935099 (-0.64z)| norm 0.3156 (-0.48z)| lr 5.95e-04 | 323.34 ms | 52.2% bf16 MFU | 1623796 tok/s
step 1765/19560 | loss 3.947445 (-0.41z)| norm 0.3149 (-0.48z)| lr 5.95e-04 | 322.86 ms | 52.3% bf16 MFU | 1623800 tok/s
step 1766/19560 | loss 3.875075 (-1.73z)| norm 0.3069 (-0.71z)| lr 5.95e-04 | 323.30 ms | 52.2% bf16 MFU | 1623693 tok/s
step 1767/19560 | loss 4.003951 (+0.64z)| norm 0.2931 (-1.10z)| lr 5.95e-04 | 323.29 ms | 52.2% bf16 MFU | 1623595 tok/s
step 1768/19560 | loss 3.938674 (-0.56z)| norm 0.3179 (-0.35z)| lr 5.95e-04 | 322.54 ms | 52.3% bf16 MFU | 1623691 tok/s
step 1769/19560 | loss 3.908736 (-1.16z)| norm 0.3408 (+0.34z)| lr 5.95e-04 | 322.73 ms | 52.3% bf16 MFU | 1623733 tok/s
step 1770/19560 | loss 3.906401 (-1.19z)| norm 0.3180 (-0.35z)| lr 5.95e-04 | 322.96 ms | 52.3% bf16 MFU | 1623716 tok/s
step 1771/19560 | loss 3.928017 (-0.75z)| norm 0.2968 (-0.97z)| lr 5.95e-04 | 323.31 ms | 52.2% bf16 MFU | 1623613 tok/s
step 1772/19560 | loss 3.925140 (-0.80z)| norm 0.3211 (-0.25z)| lr 5.95e-04 | 322.77 ms | 52.3% bf16 MFU | 1623650 tok/s
step 1773/19560 | loss 3.971457 (+0.13z)| norm 0.3311 (+0.06z)| lr 5.95e-04 | 323.21 ms | 52.2% bf16 MFU | 1623574 tok/s
step 1774/19560 | loss 3.937717 (-0.54z)| norm 0.3388 (+0.29z)| lr 5.95e-04 | 322.49 ms | 52.3% bf16 MFU | 1623681 tok/s
step 1775/19560 | loss 3.907554 (-1.13z)| norm 0.3488 (+0.58z)| lr 5.95e-04 | 322.94 ms | 52.3% bf16 MFU | 1623672 tok/s
step 1776/19560 | loss 3.918024 (-0.91z)| norm 0.3103 (-0.59z)| lr 5.95e-04 | 322.95 ms | 52.3% bf16 MFU | 1623661 tok/s
step 1777/19560 | loss 3.901931 (-1.21z)| norm 0.3781 (+1.45z)| lr 5.95e-04 | 322.46 ms | 52.3% bf16 MFU | 1623774 tok/s
step 1778/19560 | loss 3.960621 (-0.03z)| norm 0.3323 (+0.07z)| lr 5.95e-04 | 323.46 ms | 52.2% bf16 MFU | 1623628 tok/s
step 1779/19560 | loss 3.944809 (-0.34z)| norm 0.2961 (-1.01z)| lr 5.95e-04 | 322.33 ms | 52.4% bf16 MFU | 1623776 tok/s
step 1780/19560 | loss 3.919430 (-0.84z)| norm 0.2841 (-1.38z)| lr 5.95e-04 | 323.57 ms | 52.2% bf16 MFU | 1623603 tok/s
step 1781/19560 | loss 3.924814 (-0.72z)| norm 0.2901 (-1.22z)| lr 5.95e-04 | 323.36 ms | 52.2% bf16 MFU | 1623491 tok/s
step 1782/19560 | loss 3.948969 (-0.24z)| norm 0.3001 (-0.90z)| lr 5.95e-04 | 323.02 ms | 52.2% bf16 MFU | 1623470 tok/s
step 1783/19560 | loss 3.906819 (-1.07z)| norm 0.3273 (+0.08z)| lr 5.95e-04 | 323.52 ms | 52.2% bf16 MFU | 1623324 tok/s
step 1784/19560 | loss 3.946754 (-0.25z)| norm 0.3747 (+1.80z)| lr 5.95e-04 | 322.63 ms | 52.3% bf16 MFU | 1623411 tok/s
step 1785/19560 | loss 3.949878 (-0.19z)| norm 0.4170 (+3.18z)| lr 5.95e-04 | 323.11 ms | 52.2% bf16 MFU | 1623372 tok/s
step 1786/19560 | loss 3.925636 (-0.67z)| norm 0.4112 (+2.86z)| lr 5.95e-04 | 322.96 ms | 52.3% bf16 MFU | 1623372 tok/s
step 1787/19560 | loss 3.944310 (-0.27z)| norm 0.3702 (+1.45z)| lr 5.95e-04 | 322.58 ms | 52.3% bf16 MFU | 1623469 tok/s
step 1788/19560 | loss 3.881239 (-1.55z)| norm 0.3612 (+1.13z)| lr 5.95e-04 | 322.93 ms | 52.3% bf16 MFU | 1623472 tok/s
step 1789/19560 | loss 3.895774 (-1.23z)| norm 0.3244 (-0.10z)| lr 5.95e-04 | 322.79 ms | 52.3% bf16 MFU | 1623509 tok/s
step 1790/19560 | loss 3.885234 (-1.56z)| norm 0.2961 (-1.04z)| lr 5.95e-04 | 322.80 ms | 52.3% bf16 MFU | 1623544 tok/s
step 1791/19560 | loss 3.911340 (-0.94z)| norm 0.2881 (-1.29z)| lr 5.95e-04 | 322.56 ms | 52.3% bf16 MFU | 1623637 tok/s
step 1792/19560 | loss 3.899828 (-1.19z)| norm 0.2816 (-1.49z)| lr 5.95e-04 | 322.41 ms | 52.3% bf16 MFU | 1623762 tok/s
step 1793/19560 | loss 3.929955 (-0.49z)| norm 0.2855 (-1.35z)| lr 5.95e-04 | 322.52 ms | 52.3% bf16 MFU | 1623854 tok/s
step 1794/19560 | loss 3.934066 (-0.39z)| norm 0.3154 (-0.34z)| lr 5.95e-04 | 323.28 ms | 52.2% bf16 MFU | 1623750 tok/s
step 1795/19560 | loss 3.872092 (-1.79z)| norm 0.3173 (-0.28z)| lr 5.95e-04 | 323.37 ms | 52.2% bf16 MFU | 1623629 tok/s
step 1796/19560 | loss 3.878901 (-1.60z)| norm 0.3227 (-0.11z)| lr 5.95e-04 | 323.06 ms | 52.2% bf16 MFU | 1623590 tok/s
step 1797/19560 | loss 3.881090 (-1.52z)| norm 0.3074 (-0.63z)| lr 5.95e-04 | 322.29 ms | 52.4% bf16 MFU | 1623749 tok/s
step 1798/19560 | loss 3.941275 (-0.16z)| norm 0.2911 (-1.16z)| lr 5.95e-04 | 322.84 ms | 52.3% bf16 MFU | 1623762 tok/s
step 1799/19560 | loss 3.941587 (-0.15z)| norm 0.3070 (-0.62z)| lr 5.95e-04 | 322.99 ms | 52.3% bf16 MFU | 1623734 tok/s
step 1800/19560 | loss 3.958652 (+0.24z)| norm 0.3189 (-0.20z)| lr 5.95e-04 | 323.10 ms | 52.2% bf16 MFU | 1623682 tok/s
step 1801/19560 | loss 3.909759 (-0.86z)| norm 0.3194 (-0.18z)| lr 5.95e-04 | 323.12 ms | 52.2% bf16 MFU | 1623627 tok/s
step 1802/19560 | loss 3.953363 (+0.12z)| norm 0.3429 (+0.62z)| lr 5.95e-04 | 323.06 ms | 52.2% bf16 MFU | 1623591 tok/s
step 1803/19560 | loss 3.929342 (-0.41z)| norm 0.3287 (+0.13z)| lr 5.95e-04 | 322.85 ms | 52.3% bf16 MFU | 1623608 tok/s
step 1804/19560 | loss 3.978669 (+0.71z)| norm 0.3234 (-0.03z)| lr 5.95e-04 | 323.14 ms | 52.2% bf16 MFU | 1623551 tok/s
step 1805/19560 | loss 3.915376 (-0.72z)| norm 0.3418 (+0.59z)| lr 5.95e-04 | 323.27 ms | 52.2% bf16 MFU | 1623464 tok/s
step 1806/19560 | loss 3.851506 (-2.13z)| norm 0.3077 (-0.58z)| lr 5.95e-04 | 322.74 ms | 52.3% bf16 MFU | 1623514 tok/s
step 1807/19560 | loss 3.945408 (-0.04z)| norm 0.2872 (-1.28z)| lr 5.95e-04 | 322.62 ms | 52.3% bf16 MFU | 1623592 tok/s
step 1808/19560 | loss 3.960146 (+0.32z)| norm 0.3020 (-0.77z)| lr 5.95e-04 | 323.17 ms | 52.2% bf16 MFU | 1623530 tok/s
step 1809/19560 | loss 3.900874 (-1.04z)| norm 0.3000 (-0.84z)| lr 5.95e-04 | 323.40 ms | 52.2% bf16 MFU | 1623411 tok/s
step 1810/19560 | loss 3.917804 (-0.64z)| norm 0.2875 (-1.25z)| lr 5.95e-04 | 323.66 ms | 52.1% bf16 MFU | 1623233 tok/s
step 1811/19560 | loss 3.930100 (-0.36z)| norm 0.2934 (-1.03z)| lr 5.95e-04 | 322.50 ms | 52.3% bf16 MFU | 1623357 tok/s
step 1812/19560 | loss 3.942937 (-0.06z)| norm 0.3335 (+0.33z)| lr 5.95e-04 | 322.54 ms | 52.3% bf16 MFU | 1623464 tok/s
step 1813/19560 | loss 3.872732 (-1.66z)| norm 0.3165 (-0.26z)| lr 5.95e-04 | 322.90 ms | 52.3% bf16 MFU | 1623476 tok/s
step 1814/19560 | loss 4.034369 (+2.30z)| norm 0.3144 (-0.34z)| lr 5.95e-04 | 323.22 ms | 52.2% bf16 MFU | 1623405 tok/s
step 1815/19560 | loss 3.940471 (-0.07z)| norm 0.3441 (+0.68z)| lr 5.95e-04 | 323.44 ms | 52.2% bf16 MFU | 1623284 tok/s
step 1816/19560 | loss 3.980687 (+0.94z)| norm 0.3537 (+0.99z)| lr 5.95e-04 | 323.32 ms | 52.2% bf16 MFU | 1623200 tok/s
step 1817/19560 | loss 3.907422 (-0.91z)| norm 0.3382 (+0.45z)| lr 5.95e-04 | 323.09 ms | 52.2% bf16 MFU | 1623177 tok/s
step 1818/19560 | loss 3.943439 (+0.01z)| norm 0.3894 (+2.15z)| lr 5.95e-04 | 323.25 ms | 52.2% bf16 MFU | 1623114 tok/s
step 1819/19560 | loss 3.958316 (+0.38z)| norm 0.4237 (+3.15z)| lr 5.95e-04 | 322.94 ms | 52.3% bf16 MFU | 1623133 tok/s
step 1820/19560 | loss 3.938196 (-0.12z)| norm 0.3917 (+2.06z)| lr 5.95e-04 | 323.62 ms | 52.2% bf16 MFU | 1622980 tok/s
step 1821/19560 | loss 3.941909 (-0.02z)| norm 0.3587 (+0.99z)| lr 5.95e-04 | 322.93 ms | 52.3% bf16 MFU | 1623008 tok/s
step 1822/19560 | loss 3.894263 (-1.21z)| norm 0.3793 (+1.63z)| lr 5.95e-04 | 322.80 ms | 52.3% bf16 MFU | 1623068 tok/s
step 1823/19560 | loss 3.882454 (-1.49z)| norm 0.3259 (-0.04z)| lr 5.95e-04 | 323.46 ms | 52.2% bf16 MFU | 1622960 tok/s
step 1824/19560 | loss 3.882908 (-1.46z)| norm 0.3015 (-0.81z)| lr 5.95e-04 | 323.28 ms | 52.2% bf16 MFU | 1622901 tok/s
step 1825/19560 | loss 3.910388 (-0.75z)| norm 0.2777 (-1.55z)| lr 5.95e-04 | 323.24 ms | 52.2% bf16 MFU | 1622855 tok/s
step 1826/19560 | loss 3.876657 (-1.58z)| norm 0.2910 (-1.11z)| lr 5.95e-04 | 322.68 ms | 52.3% bf16 MFU | 1622953 tok/s
step 1827/19560 | loss 3.914424 (-0.61z)| norm 0.3273 (+0.05z)| lr 5.95e-04 | 322.97 ms | 52.3% bf16 MFU | 1622971 tok/s
step 1828/19560 | loss 3.902818 (-0.90z)| norm 0.3375 (+0.36z)| lr 5.95e-04 | 323.04 ms | 52.2% bf16 MFU | 1622971 tok/s
step 1829/19560 | loss 3.912383 (-0.66z)| norm 0.3121 (-0.45z)| lr 5.95e-04 | 323.06 ms | 52.2% bf16 MFU | 1622967 tok/s
step 1830/19560 | loss 3.931211 (-0.18z)| norm 0.2855 (-1.27z)| lr 5.95e-04 | 322.82 ms | 52.3% bf16 MFU | 1623022 tok/s
step 1831/19560 | loss 3.938343 (+0.00z)| norm 0.3069 (-0.59z)| lr 5.95e-04 | 323.03 ms | 52.2% bf16 MFU | 1623023 tok/s
step 1832/19560 | loss 3.935064 (-0.07z)| norm 0.3185 (-0.22z)| lr 5.95e-04 | 322.87 ms | 52.3% bf16 MFU | 1623064 tok/s
step 1833/19560 | loss 3.909375 (-0.72z)| norm 0.3151 (-0.32z)| lr 5.95e-04 | 322.86 ms | 52.3% bf16 MFU | 1623106 tok/s
step 1834/19560 | loss 3.903347 (-0.86z)| norm 0.3105 (-0.45z)| lr 5.95e-04 | 322.73 ms | 52.3% bf16 MFU | 1623177 tok/s
step 1835/19560 | loss 3.906613 (-0.77z)| norm 0.3454 (+0.66z)| lr 5.95e-04 | 323.22 ms | 52.2% bf16 MFU | 1623121 tok/s
step 1836/19560 | loss 3.926259 (-0.26z)| norm 0.3150 (-0.31z)| lr 5.95e-04 | 322.76 ms | 52.3% bf16 MFU | 1623185 tok/s
step 1837/19560 | loss 3.965204 (+0.73z)| norm 0.2931 (-1.00z)| lr 5.95e-04 | 322.52 ms | 52.3% bf16 MFU | 1623306 tok/s
step 1838/19560 | loss 3.891656 (-1.13z)| norm 0.2976 (-0.85z)| lr 5.95e-04 | 322.94 ms | 52.3% bf16 MFU | 1623316 tok/s
step 1839/19560 | loss 3.899467 (-0.92z)| norm 0.3045 (-0.62z)| lr 5.95e-04 | 323.09 ms | 52.2% bf16 MFU | 1623286 tok/s
step 1840/19560 | loss 3.867416 (-1.72z)| norm 0.3417 (+0.57z)| lr 5.95e-04 | 322.94 ms | 52.3% bf16 MFU | 1623297 tok/s
step 1841/19560 | loss 3.910877 (-0.59z)| norm 0.3445 (+0.65z)| lr 5.95e-04 | 322.45 ms | 52.3% bf16 MFU | 1623430 tok/s
step 1842/19560 | loss 3.967297 (+0.86z)| norm 0.3743 (+1.56z)| lr 5.95e-04 | 322.63 ms | 52.3% bf16 MFU | 1623512 tok/s
step 1843/19560 | loss 3.869354 (-1.63z)| norm 0.3459 (+0.66z)| lr 5.95e-04 | 323.13 ms | 52.2% bf16 MFU | 1623462 tok/s
step 1844/19560 | loss 3.904262 (-0.74z)| norm 0.2904 (-1.11z)| lr 5.95e-04 | 323.24 ms | 52.2% bf16 MFU | 1623386 tok/s
step 1845/19560 | loss 3.925210 (-0.21z)| norm 0.2721 (-1.67z)| lr 5.95e-04 | 322.93 ms | 52.3% bf16 MFU | 1623394 tok/s
step 1846/19560 | loss 3.882395 (-1.27z)| norm 0.2917 (-1.06z)| lr 5.95e-04 | 323.01 ms | 52.2% bf16 MFU | 1623381 tok/s
step 1847/19560 | loss 3.865370 (-1.67z)| norm 0.2919 (-1.05z)| lr 5.95e-04 | 322.69 ms | 52.3% bf16 MFU | 1623449 tok/s
step 1848/19560 | loss 3.910219 (-0.54z)| norm 0.3008 (-0.77z)| lr 5.95e-04 | 322.85 ms | 52.3% bf16 MFU | 1623473 tok/s
step 1849/19560 | loss 3.945799 (+0.34z)| norm 0.3083 (-0.54z)| lr 5.95e-04 | 322.78 ms | 52.3% bf16 MFU | 1623513 tok/s
step 1850/19560 | loss 3.850249 (-2.00z)| norm 0.3277 (+0.07z)| lr 5.95e-04 | 323.18 ms | 52.2% bf16 MFU | 1623452 tok/s
step 1851/19560 | loss 3.881564 (-1.21z)| norm 0.3264 (+0.02z)| lr 5.95e-04 | 322.45 ms | 52.3% bf16 MFU | 1623578 tok/s
step 1852/19560 | loss 3.885060 (-1.14z)| norm 0.3092 (-0.53z)| lr 5.95e-04 | 322.51 ms | 52.3% bf16 MFU | 1623682 tok/s
step 1853/19560 | loss 3.992932 (+1.51z)| norm 0.3211 (-0.14z)| lr 5.94e-04 | 322.61 ms | 52.3% bf16 MFU | 1623754 tok/s
step 1854/19560 | loss 3.916271 (-0.36z)| norm 0.3303 (+0.15z)| lr 5.94e-04 | 322.81 ms | 52.3% bf16 MFU | 1623772 tok/s
step 1855/19560 | loss 3.855841 (-1.81z)| norm 0.3004 (-0.79z)| lr 5.94e-04 | 323.05 ms | 52.2% bf16 MFU | 1623731 tok/s
step 1856/19560 | loss 3.975829 (+1.10z)| norm 0.2803 (-1.42z)| lr 5.94e-04 | 323.12 ms | 52.2% bf16 MFU | 1623673 tok/s
step 1857/19560 | loss 3.874674 (-1.36z)| norm 0.2767 (-1.51z)| lr 5.94e-04 | 323.02 ms | 52.2% bf16 MFU | 1623642 tok/s
step 1858/19560 | loss 3.930558 (+0.01z)| norm 0.2973 (-0.83z)| lr 5.94e-04 | 322.94 ms | 52.3% bf16 MFU | 1623634 tok/s
step 1859/19560 | loss 3.876902 (-1.29z)| norm 0.3214 (-0.04z)| lr 5.94e-04 | 323.25 ms | 52.2% bf16 MFU | 1623548 tok/s
step 1860/19560 | loss 3.867698 (-1.51z)| norm 0.3491 (+0.86z)| lr 5.94e-04 | 322.44 ms | 52.3% bf16 MFU | 1623670 tok/s
step 1861/19560 | loss 3.833389 (-2.30z)| norm 0.3681 (+1.46z)| lr 5.94e-04 | 322.83 ms | 52.3% bf16 MFU | 1623689 tok/s
step 1862/19560 | loss 3.882550 (-1.08z)| norm 0.3496 (+0.85z)| lr 5.94e-04 | 322.51 ms | 52.3% bf16 MFU | 1623788 tok/s
step 1863/19560 | loss 3.846069 (-1.93z)| norm 0.3735 (+1.60z)| lr 5.94e-04 | 322.60 ms | 52.3% bf16 MFU | 1623858 tok/s
step 1864/19560 | loss 3.919854 (-0.13z)| norm 0.3609 (+1.19z)| lr 5.94e-04 | 322.88 ms | 52.3% bf16 MFU | 1623854 tok/s
step 1865/19560 | loss 3.910881 (-0.34z)| norm 0.3345 (+0.34z)| lr 5.94e-04 | 322.98 ms | 52.3% bf16 MFU | 1623826 tok/s
step 1866/19560 | loss 3.873638 (-1.24z)| norm 0.3429 (+0.60z)| lr 5.94e-04 | 322.45 ms | 52.3% bf16 MFU | 1623934 tok/s
step 1867/19560 | loss 3.949182 (+0.62z)| norm 0.3498 (+0.81z)| lr 5.94e-04 | 323.16 ms | 52.2% bf16 MFU | 1623857 tok/s
step 1868/19560 | loss 3.921269 (-0.05z)| norm 0.3486 (+0.76z)| lr 5.94e-04 | 323.00 ms | 52.3% bf16 MFU | 1623824 tok/s
step 1869/19560 | loss 3.873145 (-1.24z)| norm 0.3156 (-0.29z)| lr 5.94e-04 | 322.97 ms | 52.3% bf16 MFU | 1623800 tok/s
step 1870/19560 | loss 3.806360 (-2.79z)| norm 0.3117 (-0.41z)| lr 5.94e-04 | 322.86 ms | 52.3% bf16 MFU | 1623803 tok/s
step 1871/19560 | loss 3.860638 (-1.46z)| norm 0.3132 (-0.38z)| lr 5.94e-04 | 322.99 ms | 52.3% bf16 MFU | 1623775 tok/s
step 1872/19560 | loss 3.903416 (-0.43z)| norm 0.3187 (-0.21z)| lr 5.94e-04 | 322.68 ms | 52.3% bf16 MFU | 1623825 tok/s
step 1873/19560 | loss 3.929187 (+0.22z)| norm 0.3516 (+0.84z)| lr 5.94e-04 | 322.73 ms | 52.3% bf16 MFU | 1623860 tok/s
step 1874/19560 | loss 3.961948 (+1.03z)| norm 0.3337 (+0.26z)| lr 5.94e-04 | 323.06 ms | 52.2% bf16 MFU | 1623812 tok/s
step 1875/19560 | loss 3.973602 (+1.30z)| norm 0.3173 (-0.27z)| lr 5.94e-04 | 322.67 ms | 52.3% bf16 MFU | 1623864 tok/s
step 1876/19560 | loss 3.916710 (-0.08z)| norm 0.3349 (+0.29z)| lr 5.94e-04 | 322.59 ms | 52.3% bf16 MFU | 1623932 tok/s
step 1877/19560 | loss 3.941557 (+0.57z)| norm 0.2899 (-1.15z)| lr 5.94e-04 | 323.38 ms | 52.2% bf16 MFU | 1623800 tok/s
step 1878/19560 | loss 3.937272 (+0.45z)| norm 0.2808 (-1.42z)| lr 5.94e-04 | 322.53 ms | 52.3% bf16 MFU | 1623887 tok/s
step 1879/19560 | loss 3.902525 (-0.43z)| norm 0.2754 (-1.57z)| lr 5.94e-04 | 322.45 ms | 52.3% bf16 MFU | 1623991 tok/s
step 1880/19560 | loss 3.886342 (-0.85z)| norm 0.2637 (-1.90z)| lr 5.94e-04 | 322.74 ms | 52.3% bf16 MFU | 1624017 tok/s
step 1881/19560 | loss 3.929120 (+0.30z)| norm 0.2692 (-1.69z)| lr 5.94e-04 | 323.08 ms | 52.2% bf16 MFU | 1623955 tok/s
step 1882/19560 | loss 4.052195 (+3.44z)| norm 0.3070 (-0.50z)| lr 5.94e-04 | 322.69 ms | 52.3% bf16 MFU | 1623995 tok/s
step 1883/19560 | loss 3.831885 (-2.19z)| norm 0.2975 (-0.78z)| lr 5.94e-04 | 322.50 ms | 52.3% bf16 MFU | 1624079 tok/s
step 1884/19560 | loss 3.848399 (-1.74z)| norm 0.2941 (-0.88z)| lr 5.94e-04 | 322.96 ms | 52.3% bf16 MFU | 1624044 tok/s
step 1885/19560 | loss 3.863045 (-1.36z)| norm 0.2962 (-0.80z)| lr 5.94e-04 | 322.64 ms | 52.3% bf16 MFU | 1624092 tok/s
step 1886/19560 | loss 3.847478 (-1.71z)| norm 0.2802 (-1.28z)| lr 5.94e-04 | 323.00 ms | 52.3% bf16 MFU | 1624047 tok/s
step 1887/19560 | loss 3.913686 (-0.07z)| norm 0.3163 (-0.13z)| lr 5.94e-04 | 322.82 ms | 52.3% bf16 MFU | 1624050 tok/s
step 1888/19560 | loss 3.931793 (+0.39z)| norm 0.3306 (+0.33z)| lr 5.94e-04 | 322.39 ms | 52.4% bf16 MFU | 1624161 tok/s
step 1889/19560 | loss 3.921515 (+0.15z)| norm 0.2931 (-0.87z)| lr 5.94e-04 | 323.07 ms | 52.2% bf16 MFU | 1624095 tok/s
step 1890/19560 | loss 3.814682 (-2.49z)| norm 0.3132 (-0.24z)| lr 5.94e-04 | 322.74 ms | 52.3% bf16 MFU | 1624114 tok/s
step 1891/19560 | loss 3.859995 (-1.34z)| norm 0.3508 (+0.96z)| lr 5.94e-04 | 322.55 ms | 52.3% bf16 MFU | 1624180 tok/s
step 1892/19560 | loss 3.896742 (-0.42z)| norm 0.3154 (-0.18z)| lr 5.94e-04 | 322.62 ms | 52.3% bf16 MFU | 1624226 tok/s
step 1893/19560 | loss 3.858229 (-1.35z)| norm 0.3100 (-0.35z)| lr 5.94e-04 | 322.46 ms | 52.3% bf16 MFU | 1624310 tok/s
step 1894/19560 | loss 3.887149 (-0.64z)| norm 0.2902 (-0.98z)| lr 5.94e-04 | 323.17 ms | 52.2% bf16 MFU | 1624212 tok/s
step 1895/19560 | loss 3.868073 (-1.11z)| norm 0.2875 (-1.07z)| lr 5.94e-04 | 322.56 ms | 52.3% bf16 MFU | 1624271 tok/s
step 1896/19560 | loss 3.946312 (+0.85z)| norm 0.2790 (-1.32z)| lr 5.94e-04 | 322.81 ms | 52.3% bf16 MFU | 1624264 tok/s
step 1897/19560 | loss 3.832264 (-1.95z)| norm 0.2664 (-1.69z)| lr 5.94e-04 | 322.18 ms | 52.4% bf16 MFU | 1624415 tok/s
step 1898/19560 | loss 4.018414 (+2.54z)| norm 0.3098 (-0.32z)| lr 5.94e-04 | 322.75 ms | 52.3% bf16 MFU | 1624417 tok/s
step 1899/19560 | loss 3.825441 (-2.04z)| norm 0.3599 (+1.25z)| lr 5.94e-04 | 323.15 ms | 52.2% bf16 MFU | 1624318 tok/s
step 1900/19560 | loss 3.947583 (+0.84z)| norm 0.3844 (+1.97z)| lr 5.94e-04 | 323.24 ms | 52.2% bf16 MFU | 1624202 tok/s
step 1901/19560 | loss 3.908469 (-0.07z)| norm 0.3731 (+1.60z)| lr 5.94e-04 | 322.38 ms | 52.4% bf16 MFU | 1624308 tok/s
step 1902/19560 | loss 3.913713 (+0.06z)| norm 0.7522 (+8.55z)| lr 5.94e-04 | 322.32 ms | 52.4% bf16 MFU | 1624423 tok/s
step 1903/19560 | loss 3.955807 (+1.04z)| norm 0.6184 (+5.19z)| lr 5.94e-04 | 322.85 ms | 52.3% bf16 MFU | 1624397 tok/s
step 1904/19560 | loss 4.034506 (+2.79z)| norm 0.6207 (+4.71z)| lr 5.94e-04 | 323.14 ms | 52.2% bf16 MFU | 1624301 tok/s
step 1905/19560 | loss 4.047697 (+2.96z)| norm 0.3706 (+0.67z)| lr 5.94e-04 | 322.88 ms | 52.3% bf16 MFU | 1624276 tok/s
step 1906/19560 | loss 3.823776 (-1.94z)| norm 0.4209 (+1.46z)| lr 5.94e-04 | 322.80 ms | 52.3% bf16 MFU | 1624271 tok/s
step 1907/19560 | loss 3.913228 (+0.02z)| norm 0.3394 (+0.15z)| lr 5.94e-04 | 322.44 ms | 52.3% bf16 MFU | 1624358 tok/s
step 1908/19560 | loss 3.914479 (+0.05z)| norm 0.3340 (+0.06z)| lr 5.94e-04 | 322.55 ms | 52.3% bf16 MFU | 1624414 tok/s
step 1909/19560 | loss 3.866066 (-1.00z)| norm 0.3941 (+1.01z)| lr 5.94e-04 | 323.02 ms | 52.2% bf16 MFU | 1624347 tok/s
step 1910/19560 | loss 3.924031 (+0.27z)| norm 0.4175 (+1.36z)| lr 5.94e-04 | 322.78 ms | 52.3% bf16 MFU | 1624343 tok/s
step 1911/19560 | loss 3.933537 (+0.47z)| norm 0.3850 (+0.84z)| lr 5.94e-04 | 322.78 ms | 52.3% bf16 MFU | 1624340 tok/s
step 1912/19560 | loss 3.926575 (+0.32z)| norm 0.3258 (-0.10z)| lr 5.94e-04 | 322.66 ms | 52.3% bf16 MFU | 1624367 tok/s
step 1913/19560 | loss 3.931008 (+0.43z)| norm 0.3397 (+0.13z)| lr 5.94e-04 | 322.63 ms | 52.3% bf16 MFU | 1624401 tok/s
step 1914/19560 | loss 3.888435 (-0.50z)| norm 0.3404 (+0.15z)| lr 5.94e-04 | 322.67 ms | 52.3% bf16 MFU | 1624424 tok/s
step 1915/19560 | loss 3.927322 (+0.35z)| norm 0.3227 (-0.13z)| lr 5.94e-04 | 323.75 ms | 52.1% bf16 MFU | 1624173 tok/s
step 1916/19560 | loss 3.895730 (-0.34z)| norm 0.2983 (-0.51z)| lr 5.94e-04 | 322.53 ms | 52.3% bf16 MFU | 1624243 tok/s
step 1917/19560 | loss 3.820195 (-1.96z)| norm 0.3190 (-0.18z)| lr 5.94e-04 | 322.42 ms | 52.3% bf16 MFU | 1624336 tok/s
step 1918/19560 | loss 3.908283 (-0.06z)| norm 0.3015 (-0.46z)| lr 5.94e-04 | 323.00 ms | 52.3% bf16 MFU | 1624278 tok/s
step 1919/19560 | loss 3.893671 (-0.37z)| norm 0.3264 (-0.06z)| lr 5.94e-04 | 323.26 ms | 52.2% bf16 MFU | 1624158 tok/s
step 1920/19560 | loss 3.863209 (-1.02z)| norm 0.3086 (-0.36z)| lr 5.94e-04 | 322.89 ms | 52.3% bf16 MFU | 1624136 tok/s
step 1921/19560 | loss 3.905039 (-0.11z)| norm 0.3029 (-0.45z)| lr 5.94e-04 | 322.64 ms | 52.3% bf16 MFU | 1624180 tok/s
step 1922/19560 | loss 3.849390 (-1.29z)| norm 0.2809 (-0.80z)| lr 5.94e-04 | 322.34 ms | 52.4% bf16 MFU | 1624296 tok/s
step 1923/19560 | loss 3.922589 (+0.27z)| norm 0.3059 (-0.39z)| lr 5.94e-04 | 322.57 ms | 52.3% bf16 MFU | 1624350 tok/s
step 1924/19560 | loss 3.964665 (+1.16z)| norm 0.3497 (+0.31z)| lr 5.94e-04 | 322.94 ms | 52.3% bf16 MFU | 1624307 tok/s
step 1925/19560 | loss 3.906454 (-0.10z)| norm 0.3261 (-0.07z)| lr 5.94e-04 | 322.87 ms | 52.3% bf16 MFU | 1624284 tok/s
step 1926/19560 | loss 4.056831 (+3.01z)| norm 0.3007 (-0.49z)| lr 5.94e-04 | 322.50 ms | 52.3% bf16 MFU | 1624355 tok/s
step 1927/19560 | loss 3.935499 (+0.49z)| norm 0.3091 (-0.35z)| lr 5.94e-04 | 322.94 ms | 52.3% bf16 MFU | 1624312 tok/s
step 1928/19560 | loss 3.847380 (-1.31z)| norm 0.3241 (-0.11z)| lr 5.94e-04 | 323.00 ms | 52.3% bf16 MFU | 1624257 tok/s
step 1929/19560 | loss 3.935276 (+0.50z)| norm 0.2967 (-0.55z)| lr 5.94e-04 | 322.49 ms | 52.3% bf16 MFU | 1624331 tok/s
step 1930/19560 | loss 3.861609 (-1.01z)| norm 0.2861 (-0.71z)| lr 5.94e-04 | 323.42 ms | 52.2% bf16 MFU | 1624167 tok/s
step 1931/19560 | loss 3.916395 (+0.13z)| norm 0.2787 (-0.82z)| lr 5.94e-04 | 323.26 ms | 52.2% bf16 MFU | 1624053 tok/s
step 1932/19560 | loss 3.874068 (-0.74z)| norm 0.2779 (-0.83z)| lr 5.94e-04 | 322.48 ms | 52.3% bf16 MFU | 1624140 tok/s
step 1933/19560 | loss 3.866444 (-0.88z)| norm 0.2869 (-0.68z)| lr 5.94e-04 | 322.68 ms | 52.3% bf16 MFU | 1624171 tok/s
step 1934/19560 | loss 3.821479 (-1.80z)| norm 0.2624 (-1.06z)| lr 5.94e-04 | 323.22 ms | 52.2% bf16 MFU | 1624066 tok/s
step 1935/19560 | loss 3.867861 (-0.83z)| norm 0.2800 (-0.77z)| lr 5.94e-04 | 322.93 ms | 52.3% bf16 MFU | 1624040 tok/s
step 1936/19560 | loss 3.894511 (-0.27z)| norm 0.3080 (-0.33z)| lr 5.94e-04 | 322.42 ms | 52.3% bf16 MFU | 1624145 tok/s
step 1937/19560 | loss 3.916310 (+0.18z)| norm 0.3102 (-0.30z)| lr 5.94e-04 | 322.94 ms | 52.3% bf16 MFU | 1624112 tok/s
step 1938/19560 | loss 3.899676 (-0.17z)| norm 0.3072 (-0.35z)| lr 5.94e-04 | 323.25 ms | 52.2% bf16 MFU | 1624003 tok/s
step 1939/19560 | loss 3.867883 (-0.81z)| norm 0.2845 (-0.71z)| lr 5.94e-04 | 322.93 ms | 52.3% bf16 MFU | 1623980 tok/s
step 1940/19560 | loss 3.816818 (-1.82z)| norm 0.2908 (-0.60z)| lr 5.94e-04 | 323.07 ms | 52.2% bf16 MFU | 1623922 tok/s
step 1941/19560 | loss 3.863210 (-0.87z)| norm 0.3208 (-0.12z)| lr 5.94e-04 | 323.22 ms | 52.2% bf16 MFU | 1623829 tok/s
step 1942/19560 | loss 3.909719 (+0.10z)| norm 0.3400 (+0.18z)| lr 5.94e-04 | 322.70 ms | 52.3% bf16 MFU | 1623871 tok/s
step 1943/19560 | loss 3.934447 (+0.61z)| norm 0.3093 (-0.31z)| lr 5.94e-04 | 323.36 ms | 52.2% bf16 MFU | 1623746 tok/s
step 1944/19560 | loss 3.887289 (-0.36z)| norm 0.2886 (-0.63z)| lr 5.94e-04 | 323.09 ms | 52.2% bf16 MFU | 1623696 tok/s
step 1945/19560 | loss 3.892253 (-0.25z)| norm 0.3019 (-0.41z)| lr 5.94e-04 | 322.50 ms | 52.3% bf16 MFU | 1623796 tok/s
step 1946/19560 | loss 3.982024 (+1.63z)| norm 0.3231 (-0.06z)| lr 5.94e-04 | 323.45 ms | 52.2% bf16 MFU | 1623653 tok/s
step 1947/19560 | loss 3.894596 (-0.20z)| norm 0.3210 (-0.09z)| lr 5.94e-04 | 322.73 ms | 52.3% bf16 MFU | 1623697 tok/s
step 1948/19560 | loss 3.866687 (-0.78z)| norm 0.3029 (-0.37z)| lr 5.94e-04 | 322.91 ms | 52.3% bf16 MFU | 1623693 tok/s
step 1949/19560 | loss 3.836724 (-1.38z)| norm 0.2899 (-0.57z)| lr 5.94e-04 | 322.98 ms | 52.3% bf16 MFU | 1623673 tok/s
step 1950/19560 | loss 3.896240 (-0.14z)| norm 0.2706 (-0.87z)| lr 5.94e-04 | 323.03 ms | 52.2% bf16 MFU | 1623641 tok/s
step 1951/19560 | loss 3.858839 (-0.91z)| norm 0.2670 (-0.92z)| lr 5.94e-04 | 322.44 ms | 52.3% bf16 MFU | 1623758 tok/s
step 1952/19560 | loss 3.903010 (+0.01z)| norm 0.3332 (+0.15z)| lr 5.94e-04 | 323.72 ms | 52.1% bf16 MFU | 1623549 tok/s
step 1953/19560 | loss 4.008102 (+2.15z)| norm 0.3828 (+0.94z)| lr 5.93e-04 | 322.63 ms | 52.3% bf16 MFU | 1623624 tok/s
step 1954/19560 | loss 3.861987 (-0.85z)| norm 0.3595 (+0.55z)| lr 5.93e-04 | 323.77 ms | 52.1% bf16 MFU | 1623408 tok/s
step 1955/19560 | loss 3.878477 (-0.50z)| norm 0.3167 (-0.14z)| lr 5.93e-04 | 322.82 ms | 52.3% bf16 MFU | 1623442 tok/s
step 1956/19560 | loss 3.833225 (-1.41z)| norm 0.3062 (-0.30z)| lr 5.93e-04 | 322.95 ms | 52.3% bf16 MFU | 1623443 tok/s
step 1957/19560 | loss 3.872427 (-0.60z)| norm 0.3203 (-0.08z)| lr 5.93e-04 | 322.80 ms | 52.3% bf16 MFU | 1623479 tok/s
step 1958/19560 | loss 3.861245 (-0.82z)| norm 0.2985 (-0.43z)| lr 5.93e-04 | 322.85 ms | 52.3% bf16 MFU | 1623502 tok/s
step 1959/19560 | loss 3.839693 (-1.24z)| norm 0.3030 (-0.36z)| lr 5.93e-04 | 323.52 ms | 52.2% bf16 MFU | 1623357 tok/s
step 1960/19560 | loss 3.889944 (-0.21z)| norm 0.3019 (-0.37z)| lr 5.93e-04 | 322.89 ms | 52.3% bf16 MFU | 1623376 tok/s
step 1961/19560 | loss 3.875113 (-0.51z)| norm 0.3018 (-0.37z)| lr 5.93e-04 | 322.69 ms | 52.3% bf16 MFU | 1623444 tok/s
step 1962/19560 | loss 3.869118 (-0.62z)| norm 0.2917 (-0.53z)| lr 5.93e-04 | 322.83 ms | 52.3% bf16 MFU | 1623474 tok/s
step 1963/19560 | loss 3.826785 (-1.45z)| norm 0.2993 (-0.40z)| lr 5.93e-04 | 323.01 ms | 52.2% bf16 MFU | 1623455 tok/s
step 1964/19560 | loss 3.858535 (-0.81z)| norm 0.3175 (-0.11z)| lr 5.93e-04 | 323.41 ms | 52.2% bf16 MFU | 1623338 tok/s
step 1965/19560 | loss 3.854115 (-0.88z)| norm 0.3116 (-0.21z)| lr 5.93e-04 | 322.97 ms | 52.3% bf16 MFU | 1623338 tok/s
step 1966/19560 | loss 3.851043 (-0.93z)| norm 0.3385 (+0.22z)| lr 5.93e-04 | 322.65 ms | 52.3% bf16 MFU | 1623418 tok/s
step 1967/19560 | loss 3.847920 (-0.98z)| norm 0.3483 (+0.37z)| lr 5.93e-04 | 322.69 ms | 52.3% bf16 MFU | 1623483 tok/s
step 1968/19560 | loss 3.885864 (-0.23z)| norm 0.3274 (+0.04z)| lr 5.93e-04 | 323.22 ms | 52.2% bf16 MFU | 1623414 tok/s
step 1969/19560 | loss 3.904807 (+0.15z)| norm 0.3404 (+0.25z)| lr 5.93e-04 | 323.07 ms | 52.2% bf16 MFU | 1623386 tok/s
step 1970/19560 | loss 3.905075 (+0.17z)| norm 0.3189 (-0.09z)| lr 5.93e-04 | 322.89 ms | 52.3% bf16 MFU | 1623403 tok/s
step 1971/19560 | loss 3.876718 (-0.41z)| norm 0.3411 (+0.27z)| lr 5.93e-04 | 323.45 ms | 52.2% bf16 MFU | 1623278 tok/s
step 1972/19560 | loss 3.840563 (-1.12z)| norm 0.3519 (+0.43z)| lr 5.93e-04 | 323.12 ms | 52.2% bf16 MFU | 1623242 tok/s
step 1973/19560 | loss 3.836907 (-1.17z)| norm 0.3308 (+0.09z)| lr 5.93e-04 | 322.91 ms | 52.3% bf16 MFU | 1623260 tok/s
step 1974/19560 | loss 3.875861 (-0.40z)| norm 0.3076 (-0.29z)| lr 5.93e-04 | 322.66 ms | 52.3% bf16 MFU | 1623342 tok/s
step 1975/19560 | loss 3.887004 (-0.18z)| norm 0.3034 (-0.36z)| lr 5.93e-04 | 323.16 ms | 52.2% bf16 MFU | 1623295 tok/s
step 1976/19560 | loss 3.876280 (-0.39z)| norm 0.2872 (-0.63z)| lr 5.93e-04 | 322.73 ms | 52.3% bf16 MFU | 1623358 tok/s
step 1977/19560 | loss 3.840518 (-1.08z)| norm 0.2749 (-0.82z)| lr 5.93e-04 | 323.60 ms | 52.2% bf16 MFU | 1623198 tok/s
step 1978/19560 | loss 3.926577 (+0.62z)| norm 0.2913 (-0.55z)| lr 5.93e-04 | 323.25 ms | 52.2% bf16 MFU | 1623134 tok/s
step 1979/19560 | loss 3.883982 (-0.23z)| norm 0.2924 (-0.53z)| lr 5.93e-04 | 323.06 ms | 52.2% bf16 MFU | 1623121 tok/s
step 1980/19560 | loss 3.901013 (+0.11z)| norm 0.2993 (-0.41z)| lr 5.93e-04 | 323.23 ms | 52.2% bf16 MFU | 1623066 tok/s
step 1981/19560 | loss 3.866685 (-0.56z)| norm 0.2943 (-0.49z)| lr 5.93e-04 | 322.72 ms | 52.3% bf16 MFU | 1623144 tok/s
step 1982/19560 | loss 3.910547 (+0.33z)| norm 0.2879 (-0.59z)| lr 5.93e-04 | 322.73 ms | 52.3% bf16 MFU | 1623213 tok/s
step 1983/19560 | loss 3.940312 (+0.92z)| norm 0.2759 (-0.78z)| lr 5.93e-04 | 323.13 ms | 52.2% bf16 MFU | 1623179 tok/s
step 1984/19560 | loss 3.823101 (-1.44z)| norm 0.3136 (-0.17z)| lr 5.93e-04 | 323.19 ms | 52.2% bf16 MFU | 1623132 tok/s
step 1985/19560 | loss 3.898646 (+0.09z)| norm 0.3475 (+0.37z)| lr 5.93e-04 | 323.39 ms | 52.2% bf16 MFU | 1623036 tok/s
step 1986/19560 | loss 3.897266 (+0.07z)| norm 0.3420 (+0.27z)| lr 5.93e-04 | 322.82 ms | 52.3% bf16 MFU | 1623089 tok/s
step 1987/19560 | loss 3.900568 (+0.13z)| norm 0.2936 (-0.51z)| lr 5.93e-04 | 322.98 ms | 52.3% bf16 MFU | 1623099 tok/s
step 1988/19560 | loss 3.941552 (+0.95z)| norm 0.2669 (-0.93z)| lr 5.93e-04 | 323.12 ms | 52.2% bf16 MFU | 1623074 tok/s
step 1989/19560 | loss 3.851890 (-0.87z)| norm 0.3024 (-0.35z)| lr 5.93e-04 | 322.73 ms | 52.3% bf16 MFU | 1623148 tok/s
step 1990/19560 | loss 3.836714 (-1.17z)| norm 0.3143 (-0.15z)| lr 5.93e-04 | 322.96 ms | 52.3% bf16 MFU | 1623160 tok/s
step 1991/19560 | loss 4.094044 (+3.79z)| norm 0.3531 (+0.48z)| lr 5.93e-04 | 322.99 ms | 52.3% bf16 MFU | 1623163 tok/s
step 1992/19560 | loss 3.924194 (+0.53z)| norm 0.4741 (+2.38z)| lr 5.93e-04 | 322.87 ms | 52.3% bf16 MFU | 1623196 tok/s
step 1993/19560 | loss 3.864517 (-0.60z)| norm 0.4408 (+1.81z)| lr 5.93e-04 | 323.20 ms | 52.2% bf16 MFU | 1623144 tok/s
step 1994/19560 | loss 3.896804 (+0.01z)| norm 0.4129 (+1.36z)| lr 5.93e-04 | 323.26 ms | 52.2% bf16 MFU | 1623082 tok/s
step 1995/19560 | loss 3.870105 (-0.49z)| norm 0.3603 (+0.54z)| lr 5.93e-04 | 322.76 ms | 52.3% bf16 MFU | 1623146 tok/s
step 1996/19560 | loss 3.870321 (-0.48z)| norm 0.3313 (+0.09z)| lr 5.93e-04 | 323.05 ms | 52.2% bf16 MFU | 1623137 tok/s
step 1997/19560 | loss 3.960653 (+1.24z)| norm 0.3060 (-0.30z)| lr 5.93e-04 | 323.67 ms | 52.1% bf16 MFU | 1622970 tok/s
step 1998/19560 | loss 3.821547 (-1.43z)| norm 0.2950 (-0.47z)| lr 5.93e-04 | 323.08 ms | 52.2% bf16 MFU | 1622961 tok/s
step 1999/19560 | loss 3.897863 (+0.03z)| norm 0.2681 (-0.88z)| lr 5.93e-04 | 322.80 ms | 52.3% bf16 MFU | 1623021 tok/s
step 2000/19560 | loss 3.996846 (+1.89z)| norm 0.2901 (-0.54z)| lr 5.93e-04 | 323.66 ms | 52.1% bf16 MFU | 1622865 tok/s
val loss 3.872994
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2615/10042 = 0.260406
step 2001/19560 | loss 3.838551 (-1.09z)| norm 0.3036 (-0.32z)| lr 5.93e-04 | 322.66 ms | 52.3% bf16 MFU | 1622966 tok/s
step 2002/19560 | loss 3.884687 (-0.21z)| norm 0.2866 (-0.58z)| lr 5.93e-04 | 323.42 ms | 52.2% bf16 MFU | 1622873 tok/s
step 2003/19560 | loss 3.879128 (-0.30z)| norm 0.2736 (-0.77z)| lr 5.93e-04 | 322.83 ms | 52.3% bf16 MFU | 1622930 tok/s
step 2004/19560 | loss 3.855515 (-0.75z)| norm 0.2690 (-0.83z)| lr 5.93e-04 | 323.23 ms | 52.2% bf16 MFU | 1622885 tok/s
step 2005/19560 | loss 3.896732 (+0.05z)| norm 0.2715 (-0.79z)| lr 5.93e-04 | 322.48 ms | 52.3% bf16 MFU | 1623031 tok/s
step 2006/19560 | loss 3.877994 (-0.30z)| norm 0.2844 (-0.59z)| lr 5.93e-04 | 323.04 ms | 52.2% bf16 MFU | 1623028 tok/s
step 2007/19560 | loss 3.872452 (-0.40z)| norm 0.2988 (-0.38z)| lr 5.93e-04 | 323.00 ms | 52.3% bf16 MFU | 1623035 tok/s
step 2008/19560 | loss 3.822048 (-1.35z)| norm 0.2720 (-0.79z)| lr 5.93e-04 | 322.91 ms | 52.3% bf16 MFU | 1623064 tok/s
step 2009/19560 | loss 3.858433 (-0.65z)| norm 0.2906 (-0.51z)| lr 5.93e-04 | 323.09 ms | 52.2% bf16 MFU | 1623048 tok/s
step 2010/19560 | loss 3.829288 (-1.21z)| norm 0.2953 (-0.43z)| lr 5.93e-04 | 323.12 ms | 52.2% bf16 MFU | 1623024 tok/s
step 2011/19560 | loss 3.839217 (-1.02z)| norm 0.2938 (-0.46z)| lr 5.93e-04 | 323.02 ms | 52.2% bf16 MFU | 1623027 tok/s
step 2012/19560 | loss 3.918465 (+0.54z)| norm 0.3124 (-0.17z)| lr 5.93e-04 | 322.41 ms | 52.3% bf16 MFU | 1623184 tok/s
step 2013/19560 | loss 3.840703 (-0.99z)| norm 0.2846 (-0.60z)| lr 5.93e-04 | 322.73 ms | 52.3% bf16 MFU | 1623251 tok/s
step 2014/19560 | loss 3.854673 (-0.72z)| norm 0.2798 (-0.67z)| lr 5.93e-04 | 323.04 ms | 52.2% bf16 MFU | 1623237 tok/s
step 2015/19560 | loss 3.903294 (+0.24z)| norm 0.3095 (-0.21z)| lr 5.93e-04 | 323.07 ms | 52.2% bf16 MFU | 1623216 tok/s
step 2016/19560 | loss 3.861357 (-0.58z)| norm 0.2857 (-0.57z)| lr 5.93e-04 | 323.11 ms | 52.2% bf16 MFU | 1623186 tok/s
step 2017/19560 | loss 3.866676 (-0.46z)| norm 0.2877 (-0.54z)| lr 5.93e-04 | 322.60 ms | 52.3% bf16 MFU | 1623288 tok/s
step 2018/19560 | loss 3.859258 (-0.62z)| norm 0.3158 (-0.11z)| lr 5.93e-04 | 323.18 ms | 52.2% bf16 MFU | 1623236 tok/s
step 2019/19560 | loss 3.926078 (+0.70z)| norm 0.3870 (+0.98z)| lr 5.93e-04 | 322.66 ms | 52.3% bf16 MFU | 1623319 tok/s
step 2020/19560 | loss 3.903627 (+0.25z)| norm 0.3906 (+1.02z)| lr 5.93e-04 | 322.72 ms | 52.3% bf16 MFU | 1623382 tok/s
step 2021/19560 | loss 3.849583 (-0.82z)| norm 0.3324 (+0.13z)| lr 5.93e-04 | 323.02 ms | 52.2% bf16 MFU | 1623366 tok/s
step 2022/19560 | loss 3.799003 (-1.79z)| norm 0.3376 (+0.20z)| lr 5.93e-04 | 323.24 ms | 52.2% bf16 MFU | 1623297 tok/s
step 2023/19560 | loss 3.969550 (+1.53z)| norm 0.2994 (-0.38z)| lr 5.93e-04 | 322.33 ms | 52.4% bf16 MFU | 1623460 tok/s
step 2024/19560 | loss 3.819745 (-1.36z)| norm 0.2658 (-0.90z)| lr 5.93e-04 | 322.85 ms | 52.3% bf16 MFU | 1623484 tok/s
step 2025/19560 | loss 3.894693 (+0.08z)| norm 0.3056 (-0.29z)| lr 5.93e-04 | 323.15 ms | 52.2% bf16 MFU | 1623432 tok/s
step 2026/19560 | loss 3.859398 (-0.60z)| norm 0.3050 (-0.30z)| lr 5.93e-04 | 322.92 ms | 52.3% bf16 MFU | 1623439 tok/s
step 2027/19560 | loss 3.910039 (+0.40z)| norm 0.3072 (-0.26z)| lr 5.93e-04 | 322.60 ms | 52.3% bf16 MFU | 1623526 tok/s
step 2028/19560 | loss 3.872269 (-0.34z)| norm 0.3475 (+0.36z)| lr 5.93e-04 | 322.91 ms | 52.3% bf16 MFU | 1623530 tok/s
step 2029/19560 | loss 3.831859 (-1.14z)| norm 0.4520 (+1.93z)| lr 5.93e-04 | 323.02 ms | 52.2% bf16 MFU | 1623509 tok/s
step 2030/19560 | loss 3.885267 (-0.07z)| norm 0.4174 (+1.75z)| lr 5.93e-04 | 323.20 ms | 52.2% bf16 MFU | 1623441 tok/s
step 2031/19560 | loss 3.875842 (-0.24z)| norm 0.3900 (+1.45z)| lr 5.93e-04 | 322.91 ms | 52.3% bf16 MFU | 1623450 tok/s
step 2032/19560 | loss 3.902372 (+0.33z)| norm 0.3707 (+1.30z)| lr 5.93e-04 | 322.48 ms | 52.3% bf16 MFU | 1623567 tok/s
step 2033/19560 | loss 3.887965 (+0.05z)| norm 0.2847 (-0.82z)| lr 5.93e-04 | 323.05 ms | 52.2% bf16 MFU | 1623534 tok/s
step 2034/19560 | loss 3.883620 (-0.05z)| norm 0.3262 (+0.24z)| lr 5.93e-04 | 322.53 ms | 52.3% bf16 MFU | 1623635 tok/s
step 2035/19560 | loss 3.973438 (+1.90z)| norm 0.2853 (-0.79z)| lr 5.93e-04 | 323.74 ms | 52.1% bf16 MFU | 1623426 tok/s
step 2036/19560 | loss 3.838825 (-1.03z)| norm 0.2970 (-0.49z)| lr 5.93e-04 | 322.61 ms | 52.3% bf16 MFU | 1623511 tok/s
step 2037/19560 | loss 3.824172 (-1.33z)| norm 0.2999 (-0.40z)| lr 5.93e-04 | 322.93 ms | 52.3% bf16 MFU | 1623512 tok/s
step 2038/19560 | loss 3.877923 (-0.16z)| norm 0.2930 (-0.57z)| lr 5.93e-04 | 322.24 ms | 52.4% bf16 MFU | 1623687 tok/s
step 2039/19560 | loss 3.868313 (-0.36z)| norm 0.3200 (+0.16z)| lr 5.93e-04 | 323.04 ms | 52.2% bf16 MFU | 1623651 tok/s
step 2040/19560 | loss 3.916786 (+0.70z)| norm 0.3583 (+1.18z)| lr 5.93e-04 | 323.36 ms | 52.2% bf16 MFU | 1623536 tok/s
step 2041/19560 | loss 3.895571 (+0.24z)| norm 0.3228 (+0.24z)| lr 5.93e-04 | 322.47 ms | 52.3% bf16 MFU | 1623652 tok/s
step 2042/19560 | loss 3.945998 (+1.33z)| norm 0.3114 (-0.06z)| lr 5.93e-04 | 322.88 ms | 52.3% bf16 MFU | 1623659 tok/s
step 2043/19560 | loss 3.891880 (+0.16z)| norm 0.2996 (-0.38z)| lr 5.93e-04 | 322.97 ms | 52.3% bf16 MFU | 1623643 tok/s
step 2044/19560 | loss 3.952746 (+1.47z)| norm 0.2717 (-1.12z)| lr 5.93e-04 | 323.02 ms | 52.2% bf16 MFU | 1623614 tok/s
step 2045/19560 | loss 3.866626 (-0.41z)| norm 0.2773 (-0.96z)| lr 5.93e-04 | 322.27 ms | 52.4% bf16 MFU | 1623776 tok/s
step 2046/19560 | loss 3.868048 (-0.37z)| norm 0.2789 (-0.91z)| lr 5.93e-04 | 322.54 ms | 52.3% bf16 MFU | 1623862 tok/s
step 2047/19560 | loss 3.841606 (-0.94z)| norm 0.2721 (-1.07z)| lr 5.92e-04 | 323.55 ms | 52.2% bf16 MFU | 1623690 tok/s
step 2048/19560 | loss 3.878742 (-0.13z)| norm 0.2756 (-0.97z)| lr 5.92e-04 | 323.39 ms | 52.2% bf16 MFU | 1623568 tok/s
step 2049/19560 | loss 3.815015 (-1.49z)| norm 0.2705 (-1.09z)| lr 5.92e-04 | 322.53 ms | 52.3% bf16 MFU | 1623668 tok/s
step 2050/19560 | loss 4.051280 (+3.42z)| norm 0.2838 (-0.74z)| lr 5.92e-04 | 322.23 ms | 52.4% bf16 MFU | 1623837 tok/s
step 2051/19560 | loss 3.842019 (-0.89z)| norm 0.2749 (-0.97z)| lr 5.92e-04 | 323.56 ms | 52.2% bf16 MFU | 1623665 tok/s
step 2052/19560 | loss 3.861690 (-0.47z)| norm 0.3038 (-0.20z)| lr 5.92e-04 | 322.52 ms | 52.3% bf16 MFU | 1623760 tok/s
step 2053/19560 | loss 3.868600 (-0.32z)| norm 0.2982 (-0.34z)| lr 5.92e-04 | 322.65 ms | 52.3% bf16 MFU | 1623819 tok/s
step 2054/19560 | loss 3.872980 (-0.21z)| norm 0.3303 (+0.50z)| lr 5.92e-04 | 322.41 ms | 52.3% bf16 MFU | 1623936 tok/s
step 2055/19560 | loss 3.864410 (-0.39z)| norm 0.3373 (+0.67z)| lr 5.92e-04 | 322.53 ms | 52.3% bf16 MFU | 1624016 tok/s
step 2056/19560 | loss 3.891681 (+0.21z)| norm 0.3099 (-0.04z)| lr 5.92e-04 | 322.75 ms | 52.3% bf16 MFU | 1624038 tok/s
step 2057/19560 | loss 3.951708 (+1.53z)| norm 0.2908 (-0.54z)| lr 5.92e-04 | 322.79 ms | 52.3% bf16 MFU | 1624047 tok/s
step 2058/19560 | loss 3.916007 (+0.73z)| norm 0.2722 (-1.03z)| lr 5.92e-04 | 322.87 ms | 52.3% bf16 MFU | 1624038 tok/s
step 2059/19560 | loss 3.894839 (+0.27z)| norm 0.3103 (-0.03z)| lr 5.92e-04 | 322.51 ms | 52.3% bf16 MFU | 1624118 tok/s
step 2060/19560 | loss 3.864342 (-0.40z)| norm 0.3012 (-0.28z)| lr 5.92e-04 | 323.21 ms | 52.2% bf16 MFU | 1624019 tok/s
step 2061/19560 | loss 3.848931 (-0.74z)| norm 0.3078 (-0.11z)| lr 5.92e-04 | 322.60 ms | 52.3% bf16 MFU | 1624077 tok/s
step 2062/19560 | loss 3.796524 (-1.88z)| norm 0.3276 (+0.40z)| lr 5.92e-04 | 323.05 ms | 52.2% bf16 MFU | 1624019 tok/s
step 2063/19560 | loss 3.831339 (-1.10z)| norm 0.3342 (+0.57z)| lr 5.92e-04 | 322.72 ms | 52.3% bf16 MFU | 1624048 tok/s
step 2064/19560 | loss 3.836989 (-0.97z)| norm 0.2842 (-0.76z)| lr 5.92e-04 | 322.48 ms | 52.3% bf16 MFU | 1624135 tok/s
step 2065/19560 | loss 3.867804 (-0.29z)| norm 0.2892 (-0.62z)| lr 5.92e-04 | 322.69 ms | 52.3% bf16 MFU | 1624166 tok/s
step 2066/19560 | loss 3.889737 (+0.19z)| norm 0.3007 (-0.31z)| lr 5.92e-04 | 322.66 ms | 52.3% bf16 MFU | 1624203 tok/s
step 2067/19560 | loss 3.899692 (+0.40z)| norm 0.3419 (+0.77z)| lr 5.92e-04 | 322.96 ms | 52.3% bf16 MFU | 1624161 tok/s
step 2068/19560 | loss 3.847639 (-0.74z)| norm 0.3475 (+0.90z)| lr 5.92e-04 | 322.60 ms | 52.3% bf16 MFU | 1624212 tok/s
step 2069/19560 | loss 3.928022 (+1.00z)| norm 0.3372 (+0.62z)| lr 5.92e-04 | 322.63 ms | 52.3% bf16 MFU | 1624253 tok/s
step 2070/19560 | loss 3.859410 (-0.49z)| norm 0.3707 (+1.49z)| lr 5.92e-04 | 322.84 ms | 52.3% bf16 MFU | 1624240 tok/s
step 2071/19560 | loss 3.914354 (+0.72z)| norm 0.3597 (+1.19z)| lr 5.92e-04 | 322.68 ms | 52.3% bf16 MFU | 1624266 tok/s
step 2072/19560 | loss 3.849498 (-0.69z)| norm 0.3617 (+1.22z)| lr 5.92e-04 | 322.26 ms | 52.4% bf16 MFU | 1624397 tok/s
step 2073/19560 | loss 3.884192 (+0.07z)| norm 0.3370 (+0.57z)| lr 5.92e-04 | 322.79 ms | 52.3% bf16 MFU | 1624390 tok/s
step 2074/19560 | loss 3.825682 (-1.20z)| norm 0.2898 (-0.65z)| lr 5.92e-04 | 323.13 ms | 52.2% bf16 MFU | 1624296 tok/s
step 2075/19560 | loss 3.839286 (-0.89z)| norm 0.2864 (-0.73z)| lr 5.92e-04 | 323.04 ms | 52.2% bf16 MFU | 1624229 tok/s
step 2076/19560 | loss 3.837759 (-0.91z)| norm 0.2955 (-0.49z)| lr 5.92e-04 | 322.90 ms | 52.3% bf16 MFU | 1624202 tok/s
step 2077/19560 | loss 3.856917 (-0.50z)| norm 0.2942 (-0.52z)| lr 5.92e-04 | 322.71 ms | 52.3% bf16 MFU | 1624224 tok/s
step 2078/19560 | loss 3.841416 (-0.83z)| norm 0.2997 (-0.39z)| lr 5.92e-04 | 322.37 ms | 52.4% bf16 MFU | 1624331 tok/s
step 2079/19560 | loss 3.831806 (-1.03z)| norm 0.2599 (-1.42z)| lr 5.92e-04 | 323.08 ms | 52.2% bf16 MFU | 1624254 tok/s
step 2080/19560 | loss 3.873190 (-0.12z)| norm 0.2700 (-1.14z)| lr 5.92e-04 | 323.04 ms | 52.2% bf16 MFU | 1624192 tok/s
step 2081/19560 | loss 3.897665 (+0.45z)| norm 0.2685 (-1.16z)| lr 5.92e-04 | 322.38 ms | 52.4% bf16 MFU | 1624296 tok/s
step 2082/19560 | loss 3.818328 (-1.33z)| norm 0.2893 (-0.61z)| lr 5.92e-04 | 323.32 ms | 52.2% bf16 MFU | 1624161 tok/s
step 2083/19560 | loss 3.850338 (-0.60z)| norm 0.2869 (-0.67z)| lr 5.92e-04 | 322.52 ms | 52.3% bf16 MFU | 1624233 tok/s
step 2084/19560 | loss 3.822008 (-1.24z)| norm 0.2992 (-0.34z)| lr 5.92e-04 | 323.34 ms | 52.2% bf16 MFU | 1624096 tok/s
step 2085/19560 | loss 3.867054 (-0.22z)| norm 0.3383 (+0.67z)| lr 5.92e-04 | 322.38 ms | 52.4% bf16 MFU | 1624206 tok/s
step 2086/19560 | loss 3.850869 (-0.59z)| norm 0.3143 (+0.04z)| lr 5.92e-04 | 323.08 ms | 52.2% bf16 MFU | 1624135 tok/s
step 2087/19560 | loss 3.859416 (-0.40z)| norm 0.2774 (-0.91z)| lr 5.92e-04 | 322.37 ms | 52.4% bf16 MFU | 1624246 tok/s
step 2088/19560 | loss 3.809022 (-1.50z)| norm 0.3016 (-0.28z)| lr 5.92e-04 | 322.49 ms | 52.3% bf16 MFU | 1624322 tok/s
step 2089/19560 | loss 3.820754 (-1.23z)| norm 0.2917 (-0.54z)| lr 5.92e-04 | 322.77 ms | 52.3% bf16 MFU | 1624322 tok/s
step 2090/19560 | loss 3.833531 (-0.93z)| norm 0.2882 (-0.63z)| lr 5.92e-04 | 323.08 ms | 52.2% bf16 MFU | 1624245 tok/s
step 2091/19560 | loss 3.842823 (-0.73z)| norm 0.3069 (-0.14z)| lr 5.92e-04 | 322.58 ms | 52.3% bf16 MFU | 1624297 tok/s
step 2092/19560 | loss 3.901645 (+0.56z)| norm 0.3368 (+0.63z)| lr 5.92e-04 | 322.40 ms | 52.3% bf16 MFU | 1624393 tok/s
step 2093/19560 | loss 3.868407 (-0.18z)| norm 0.3960 (+2.11z)| lr 5.92e-04 | 322.85 ms | 52.3% bf16 MFU | 1624369 tok/s
step 2094/19560 | loss 3.840100 (-0.80z)| norm 0.3756 (+1.57z)| lr 5.92e-04 | 322.40 ms | 52.3% bf16 MFU | 1624461 tok/s
step 2095/19560 | loss 3.840324 (-0.79z)| norm 0.3463 (+0.83z)| lr 5.92e-04 | 323.21 ms | 52.2% bf16 MFU | 1624344 tok/s
step 2096/19560 | loss 3.824178 (-1.13z)| norm 0.3116 (-0.05z)| lr 5.92e-04 | 322.94 ms | 52.3% bf16 MFU | 1624301 tok/s
step 2097/19560 | loss 3.875939 (+0.01z)| norm 0.3089 (-0.11z)| lr 5.92e-04 | 322.63 ms | 52.3% bf16 MFU | 1624339 tok/s
step 2098/19560 | loss 3.860743 (-0.32z)| norm 0.2829 (-0.76z)| lr 5.92e-04 | 322.65 ms | 52.3% bf16 MFU | 1624370 tok/s
step 2099/19560 | loss 3.896521 (+0.47z)| norm 0.2845 (-0.70z)| lr 5.92e-04 | 322.69 ms | 52.3% bf16 MFU | 1624389 tok/s
step 2100/19560 | loss 3.885157 (+0.21z)| norm 0.3058 (-0.16z)| lr 5.92e-04 | 322.67 ms | 52.3% bf16 MFU | 1624411 tok/s
step 2101/19560 | loss 3.868900 (-0.15z)| norm 0.3125 (+0.01z)| lr 5.92e-04 | 322.57 ms | 52.3% bf16 MFU | 1624458 tok/s
step 2102/19560 | loss 3.790573 (-1.85z)| norm 0.2842 (-0.70z)| lr 5.92e-04 | 322.62 ms | 52.3% bf16 MFU | 1624491 tok/s
step 2103/19560 | loss 3.798814 (-1.64z)| norm 0.2830 (-0.72z)| lr 5.92e-04 | 322.96 ms | 52.3% bf16 MFU | 1624435 tok/s
step 2104/19560 | loss 3.860708 (-0.30z)| norm 0.2667 (-1.13z)| lr 5.92e-04 | 322.67 ms | 52.3% bf16 MFU | 1624455 tok/s
step 2105/19560 | loss 3.814388 (-1.29z)| norm 0.2666 (-1.13z)| lr 5.92e-04 | 322.61 ms | 52.3% bf16 MFU | 1624489 tok/s
step 2106/19560 | loss 3.853290 (-0.44z)| norm 0.2745 (-0.92z)| lr 5.92e-04 | 322.55 ms | 52.3% bf16 MFU | 1624537 tok/s
step 2107/19560 | loss 3.865197 (-0.18z)| norm 0.3138 (+0.06z)| lr 5.92e-04 | 323.27 ms | 52.2% bf16 MFU | 1624402 tok/s
step 2108/19560 | loss 3.862202 (-0.24z)| norm 0.3069 (-0.11z)| lr 5.92e-04 | 322.41 ms | 52.3% bf16 MFU | 1624489 tok/s
step 2109/19560 | loss 3.837365 (-0.77z)| norm 0.3066 (-0.12z)| lr 5.92e-04 | 322.88 ms | 52.3% bf16 MFU | 1624455 tok/s
step 2110/19560 | loss 3.806658 (-1.41z)| norm 0.2807 (-0.77z)| lr 5.92e-04 | 323.10 ms | 52.2% bf16 MFU | 1624367 tok/s
step 2111/19560 | loss 4.015674 (+2.99z)| norm 0.3194 (+0.19z)| lr 5.92e-04 | 322.77 ms | 52.3% bf16 MFU | 1624366 tok/s
step 2112/19560 | loss 3.901825 (+0.60z)| norm 0.3531 (+1.03z)| lr 5.92e-04 | 323.15 ms | 52.2% bf16 MFU | 1624269 tok/s
step 2113/19560 | loss 3.866952 (-0.13z)| norm 0.3801 (+1.68z)| lr 5.92e-04 | 322.77 ms | 52.3% bf16 MFU | 1624272 tok/s
step 2114/19560 | loss 3.843457 (-0.61z)| norm 0.3482 (+0.89z)| lr 5.92e-04 | 323.33 ms | 52.2% bf16 MFU | 1624136 tok/s
step 2115/19560 | loss 3.815472 (-1.18z)| norm 0.2966 (-0.39z)| lr 5.92e-04 | 323.40 ms | 52.2% bf16 MFU | 1623989 tok/s
step 2116/19560 | loss 3.853722 (-0.37z)| norm 0.2727 (-0.99z)| lr 5.92e-04 | 322.31 ms | 52.4% bf16 MFU | 1624123 tok/s
step 2117/19560 | loss 3.877978 (+0.14z)| norm 0.2683 (-1.09z)| lr 5.92e-04 | 322.59 ms | 52.3% bf16 MFU | 1624178 tok/s
step 2118/19560 | loss 3.810423 (-1.28z)| norm 0.2859 (-0.65z)| lr 5.92e-04 | 323.29 ms | 52.2% bf16 MFU | 1624055 tok/s
step 2119/19560 | loss 3.877544 (+0.18z)| norm 0.3037 (-0.20z)| lr 5.92e-04 | 322.60 ms | 52.3% bf16 MFU | 1624111 tok/s
step 2120/19560 | loss 3.799153 (-1.59z)| norm 0.2989 (-0.30z)| lr 5.92e-04 | 322.95 ms | 52.3% bf16 MFU | 1624076 tok/s
step 2121/19560 | loss 3.793444 (-1.69z)| norm 0.3016 (-0.21z)| lr 5.92e-04 | 323.00 ms | 52.3% bf16 MFU | 1624032 tok/s
step 2122/19560 | loss 3.887735 (+0.45z)| norm 0.3016 (-0.19z)| lr 5.92e-04 | 322.93 ms | 52.3% bf16 MFU | 1624008 tok/s
step 2123/19560 | loss 3.844864 (-0.52z)| norm 0.3206 (+0.37z)| lr 5.92e-04 | 323.16 ms | 52.2% bf16 MFU | 1623927 tok/s
step 2124/19560 | loss 3.889866 (+0.50z)| norm 0.3261 (+0.53z)| lr 5.92e-04 | 322.91 ms | 52.3% bf16 MFU | 1623912 tok/s
step 2125/19560 | loss 3.820311 (-1.07z)| norm 0.3112 (+0.09z)| lr 5.92e-04 | 323.16 ms | 52.2% bf16 MFU | 1623834 tok/s
step 2126/19560 | loss 3.860945 (-0.14z)| norm 0.2929 (-0.44z)| lr 5.92e-04 | 323.10 ms | 52.2% bf16 MFU | 1623776 tok/s
step 2127/19560 | loss 3.856990 (-0.23z)| norm 0.2847 (-0.68z)| lr 5.92e-04 | 322.68 ms | 52.3% bf16 MFU | 1623826 tok/s
step 2128/19560 | loss 3.856740 (-0.22z)| norm 0.3230 (+0.43z)| lr 5.92e-04 | 323.46 ms | 52.2% bf16 MFU | 1623678 tok/s
step 2129/19560 | loss 3.840557 (-0.60z)| norm 0.3158 (+0.21z)| lr 5.92e-04 | 322.84 ms | 52.3% bf16 MFU | 1623693 tok/s
step 2130/19560 | loss 3.929665 (+1.51z)| norm 0.2956 (-0.38z)| lr 5.92e-04 | 323.34 ms | 52.2% bf16 MFU | 1623582 tok/s
step 2131/19560 | loss 3.841668 (-0.57z)| norm 0.2958 (-0.38z)| lr 5.92e-04 | 322.75 ms | 52.3% bf16 MFU | 1623626 tok/s
step 2132/19560 | loss 3.845230 (-0.49z)| norm 0.2925 (-0.48z)| lr 5.92e-04 | 323.11 ms | 52.2% bf16 MFU | 1623575 tok/s
step 2133/19560 | loss 3.830211 (-0.83z)| norm 0.2586 (-1.47z)| lr 5.92e-04 | 322.61 ms | 52.3% bf16 MFU | 1623652 tok/s
step 2134/19560 | loss 3.821730 (-1.02z)| norm 0.2641 (-1.30z)| lr 5.91e-04 | 323.30 ms | 52.2% bf16 MFU | 1623554 tok/s
step 2135/19560 | loss 3.865021 (+0.01z)| norm 0.2747 (-0.98z)| lr 5.91e-04 | 322.99 ms | 52.3% bf16 MFU | 1623538 tok/s
step 2136/19560 | loss 3.844307 (-0.49z)| norm 0.2777 (-0.90z)| lr 5.91e-04 | 323.02 ms | 52.2% bf16 MFU | 1623516 tok/s
step 2137/19560 | loss 3.906429 (+0.97z)| norm 0.2828 (-0.75z)| lr 5.91e-04 | 323.29 ms | 52.2% bf16 MFU | 1623425 tok/s
step 2138/19560 | loss 3.955154 (+2.07z)| norm 0.2948 (-0.40z)| lr 5.91e-04 | 322.70 ms | 52.3% bf16 MFU | 1623488 tok/s
step 2139/19560 | loss 3.857857 (-0.20z)| norm 0.3637 (+1.58z)| lr 5.91e-04 | 322.72 ms | 52.3% bf16 MFU | 1623543 tok/s
step 2140/19560 | loss 3.859879 (-0.14z)| norm 0.4356 (+3.46z)| lr 5.91e-04 | 322.74 ms | 52.3% bf16 MFU | 1623591 tok/s
step 2141/19560 | loss 3.858809 (-0.17z)| norm 0.4112 (+2.68z)| lr 5.91e-04 | 322.78 ms | 52.3% bf16 MFU | 1623626 tok/s
step 2142/19560 | loss 3.899968 (+0.78z)| norm 0.3572 (+1.22z)| lr 5.91e-04 | 323.02 ms | 52.2% bf16 MFU | 1623599 tok/s
step 2143/19560 | loss 3.835336 (-0.72z)| norm 0.3359 (+0.64z)| lr 5.91e-04 | 323.08 ms | 52.2% bf16 MFU | 1623558 tok/s
step 2144/19560 | loss 3.842565 (-0.54z)| norm 0.3126 (+0.02z)| lr 5.91e-04 | 322.51 ms | 52.3% bf16 MFU | 1623662 tok/s
step 2145/19560 | loss 3.865440 (-0.01z)| norm 0.3476 (+0.94z)| lr 5.91e-04 | 322.84 ms | 52.3% bf16 MFU | 1623678 tok/s
step 2146/19560 | loss 3.805069 (-1.40z)| norm 0.3116 (-0.02z)| lr 5.91e-04 | 323.06 ms | 52.2% bf16 MFU | 1623637 tok/s
step 2147/19560 | loss 3.893787 (+0.67z)| norm 0.2892 (-0.61z)| lr 5.91e-04 | 322.78 ms | 52.3% bf16 MFU | 1623669 tok/s
step 2148/19560 | loss 3.867718 (+0.07z)| norm 0.2897 (-0.58z)| lr 5.91e-04 | 323.30 ms | 52.2% bf16 MFU | 1623570 tok/s
step 2149/19560 | loss 3.859550 (-0.13z)| norm 0.2994 (-0.31z)| lr 5.91e-04 | 322.80 ms | 52.3% bf16 MFU | 1623602 tok/s
step 2150/19560 | loss 3.829487 (-0.84z)| norm 0.3010 (-0.26z)| lr 5.91e-04 | 322.84 ms | 52.3% bf16 MFU | 1623620 tok/s
step 2151/19560 | loss 3.839760 (-0.59z)| norm 0.2844 (-0.71z)| lr 5.91e-04 | 323.27 ms | 52.2% bf16 MFU | 1623531 tok/s
step 2152/19560 | loss 3.799077 (-1.56z)| norm 0.2761 (-0.94z)| lr 5.91e-04 | 322.81 ms | 52.3% bf16 MFU | 1623561 tok/s
step 2153/19560 | loss 3.897567 (+0.81z)| norm 0.2845 (-0.70z)| lr 5.91e-04 | 323.40 ms | 52.2% bf16 MFU | 1623441 tok/s
step 2154/19560 | loss 3.795753 (-1.61z)| norm 0.2799 (-0.82z)| lr 5.91e-04 | 322.93 ms | 52.3% bf16 MFU | 1623445 tok/s
step 2155/19560 | loss 3.899330 (+0.86z)| norm 0.2608 (-1.33z)| lr 5.91e-04 | 322.86 ms | 52.3% bf16 MFU | 1623468 tok/s
step 2156/19560 | loss 3.841872 (-0.51z)| norm 0.3035 (-0.16z)| lr 5.91e-04 | 323.16 ms | 52.2% bf16 MFU | 1623413 tok/s
step 2157/19560 | loss 3.851319 (-0.29z)| norm 0.2970 (-0.32z)| lr 5.91e-04 | 323.03 ms | 52.2% bf16 MFU | 1623394 tok/s
step 2158/19560 | loss 3.863230 (+0.00z)| norm 0.2926 (-0.44z)| lr 5.91e-04 | 323.84 ms | 52.1% bf16 MFU | 1623174 tok/s
step 2159/19560 | loss 3.828242 (-0.82z)| norm 0.2865 (-0.61z)| lr 5.91e-04 | 322.85 ms | 52.3% bf16 MFU | 1623212 tok/s
step 2160/19560 | loss 3.860061 (-0.06z)| norm 0.2600 (-1.42z)| lr 5.91e-04 | 322.94 ms | 52.3% bf16 MFU | 1623226 tok/s
step 2161/19560 | loss 3.889133 (+0.64z)| norm 0.2669 (-1.20z)| lr 5.91e-04 | 322.76 ms | 52.3% bf16 MFU | 1623284 tok/s
step 2162/19560 | loss 3.862992 (+0.02z)| norm 0.2622 (-1.32z)| lr 5.91e-04 | 323.14 ms | 52.2% bf16 MFU | 1623243 tok/s
step 2163/19560 | loss 3.896971 (+0.87z)| norm 0.2703 (-1.06z)| lr 5.91e-04 | 322.62 ms | 52.3% bf16 MFU | 1623336 tok/s
step 2164/19560 | loss 3.865563 (+0.09z)| norm 0.2937 (-0.34z)| lr 5.91e-04 | 323.12 ms | 52.2% bf16 MFU | 1623297 tok/s
step 2165/19560 | loss 3.873277 (+0.27z)| norm 0.3053 (+0.02z)| lr 5.91e-04 | 323.03 ms | 52.2% bf16 MFU | 1623283 tok/s
step 2166/19560 | loss 3.893630 (+0.77z)| norm 0.3662 (+1.87z)| lr 5.91e-04 | 323.63 ms | 52.1% bf16 MFU | 1623121 tok/s
step 2167/19560 | loss 3.893423 (+0.76z)| norm 0.4119 (+3.12z)| lr 5.91e-04 | 322.99 ms | 52.3% bf16 MFU | 1623127 tok/s
step 2168/19560 | loss 3.871274 (+0.22z)| norm 0.3406 (+1.03z)| lr 5.91e-04 | 323.53 ms | 52.2% bf16 MFU | 1622996 tok/s
step 2169/19560 | loss 3.853224 (-0.22z)| norm 0.3140 (+0.25z)| lr 5.91e-04 | 322.83 ms | 52.3% bf16 MFU | 1623048 tok/s
step 2170/19560 | loss 3.835666 (-0.64z)| norm 0.3093 (+0.11z)| lr 5.91e-04 | 322.75 ms | 52.3% bf16 MFU | 1623116 tok/s
step 2171/19560 | loss 3.850403 (-0.26z)| norm 0.3010 (-0.14z)| lr 5.91e-04 | 323.03 ms | 52.2% bf16 MFU | 1623112 tok/s
step 2172/19560 | loss 3.788124 (-1.82z)| norm 0.2666 (-1.15z)| lr 5.91e-04 | 323.63 ms | 52.2% bf16 MFU | 1622959 tok/s
step 2173/19560 | loss 3.828601 (-0.78z)| norm 0.2808 (-0.74z)| lr 5.91e-04 | 323.17 ms | 52.2% bf16 MFU | 1622928 tok/s
step 2174/19560 | loss 3.790626 (-1.71z)| norm 0.3061 (+0.01z)| lr 5.91e-04 | 322.90 ms | 52.3% bf16 MFU | 1622966 tok/s
step 2175/19560 | loss 3.886532 (+0.69z)| norm 0.3035 (-0.08z)| lr 5.91e-04 | 323.16 ms | 52.2% bf16 MFU | 1622936 tok/s
step 2176/19560 | loss 3.772528 (-2.11z)| norm 0.2872 (-0.57z)| lr 5.91e-04 | 323.30 ms | 52.2% bf16 MFU | 1622874 tok/s
step 2177/19560 | loss 3.846866 (-0.28z)| norm 0.3145 (+0.24z)| lr 5.91e-04 | 322.89 ms | 52.3% bf16 MFU | 1622916 tok/s
step 2178/19560 | loss 3.813419 (-1.17z)| norm 0.3080 (+0.04z)| lr 5.91e-04 | 323.67 ms | 52.1% bf16 MFU | 1622761 tok/s
step 2179/19560 | loss 3.853672 (-0.08z)| norm 0.2896 (-0.52z)| lr 5.91e-04 | 323.53 ms | 52.2% bf16 MFU | 1622648 tok/s
step 2180/19560 | loss 3.813949 (-1.14z)| norm 0.2865 (-0.61z)| lr 5.91e-04 | 323.18 ms | 52.2% bf16 MFU | 1622630 tok/s
step 2181/19560 | loss 3.866473 (+0.28z)| norm 0.2920 (-0.44z)| lr 5.91e-04 | 323.11 ms | 52.2% bf16 MFU | 1622629 tok/s
step 2182/19560 | loss 3.895161 (+1.05z)| norm 0.2800 (-0.79z)| lr 5.91e-04 | 323.97 ms | 52.1% bf16 MFU | 1622414 tok/s
step 2183/19560 | loss 3.817982 (-1.02z)| norm 0.2697 (-1.08z)| lr 5.91e-04 | 322.85 ms | 52.3% bf16 MFU | 1622489 tok/s
step 2184/19560 | loss 3.789605 (-1.75z)| norm 0.2787 (-0.80z)| lr 5.91e-04 | 323.27 ms | 52.2% bf16 MFU | 1622456 tok/s
step 2185/19560 | loss 3.860663 (+0.17z)| norm 0.2843 (-0.63z)| lr 5.91e-04 | 323.01 ms | 52.3% bf16 MFU | 1622490 tok/s
step 2186/19560 | loss 3.832847 (-0.58z)| norm 0.2921 (-0.40z)| lr 5.91e-04 | 323.09 ms | 52.2% bf16 MFU | 1622503 tok/s
step 2187/19560 | loss 3.922686 (+1.88z)| norm 0.3194 (+0.41z)| lr 5.91e-04 | 322.56 ms | 52.3% bf16 MFU | 1622648 tok/s
step 2188/19560 | loss 3.848661 (-0.14z)| norm 0.3100 (+0.13z)| lr 5.91e-04 | 322.64 ms | 52.3% bf16 MFU | 1622765 tok/s
step 2189/19560 | loss 3.900276 (+1.25z)| norm 0.2838 (-0.65z)| lr 5.91e-04 | 322.54 ms | 52.3% bf16 MFU | 1622902 tok/s
step 2190/19560 | loss 3.804972 (-1.35z)| norm 0.2723 (-0.98z)| lr 5.91e-04 | 323.45 ms | 52.2% bf16 MFU | 1622803 tok/s
step 2191/19560 | loss 3.813070 (-1.12z)| norm 0.2898 (-0.45z)| lr 5.91e-04 | 323.16 ms | 52.2% bf16 MFU | 1622781 tok/s
step 2192/19560 | loss 3.817869 (-0.98z)| norm 0.3110 (+0.18z)| lr 5.91e-04 | 323.13 ms | 52.2% bf16 MFU | 1622768 tok/s
step 2193/19560 | loss 3.833385 (-0.55z)| norm 0.3090 (+0.11z)| lr 5.91e-04 | 323.33 ms | 52.2% bf16 MFU | 1622707 tok/s
step 2194/19560 | loss 3.885960 (+0.87z)| norm 0.3153 (+0.30z)| lr 5.91e-04 | 323.10 ms | 52.2% bf16 MFU | 1622704 tok/s
step 2195/19560 | loss 3.833728 (-0.53z)| norm 0.3120 (+0.21z)| lr 5.91e-04 | 323.18 ms | 52.2% bf16 MFU | 1622684 tok/s
step 2196/19560 | loss 3.783632 (-1.86z)| norm 0.3393 (+1.04z)| lr 5.91e-04 | 323.19 ms | 52.2% bf16 MFU | 1622660 tok/s
step 2197/19560 | loss 3.837052 (-0.41z)| norm 0.3180 (+0.40z)| lr 5.91e-04 | 322.69 ms | 52.3% bf16 MFU | 1622763 tok/s
step 2198/19560 | loss 3.813322 (-1.04z)| norm 0.3516 (+1.43z)| lr 5.91e-04 | 323.41 ms | 52.2% bf16 MFU | 1622681 tok/s
step 2199/19560 | loss 3.815918 (-0.96z)| norm 0.2921 (-0.37z)| lr 5.91e-04 | 322.25 ms | 52.4% bf16 MFU | 1622894 tok/s
step 2200/19560 | loss 3.883500 (+0.88z)| norm 0.2934 (-0.32z)| lr 5.91e-04 | 323.13 ms | 52.2% bf16 MFU | 1622876 tok/s
step 2201/19560 | loss 3.892226 (+1.12z)| norm 0.3303 (+0.84z)| lr 5.91e-04 | 323.00 ms | 52.3% bf16 MFU | 1622891 tok/s
step 2202/19560 | loss 3.824568 (-0.73z)| norm 0.3401 (+1.13z)| lr 5.91e-04 | 323.61 ms | 52.2% bf16 MFU | 1622753 tok/s
step 2203/19560 | loss 3.891141 (+1.07z)| norm 0.3224 (+0.57z)| lr 5.91e-04 | 323.16 ms | 52.2% bf16 MFU | 1622734 tok/s
step 2204/19560 | loss 3.853151 (+0.04z)| norm 0.3159 (+0.36z)| lr 5.91e-04 | 322.77 ms | 52.3% bf16 MFU | 1622815 tok/s
step 2205/19560 | loss 3.838854 (-0.35z)| norm 0.3355 (+0.96z)| lr 5.91e-04 | 322.87 ms | 52.3% bf16 MFU | 1622867 tok/s
step 2206/19560 | loss 3.879784 (+0.76z)| norm 0.3489 (+1.35z)| lr 5.91e-04 | 323.38 ms | 52.2% bf16 MFU | 1622787 tok/s
step 2207/19560 | loss 3.903367 (+1.37z)| norm 0.3190 (+0.42z)| lr 5.91e-04 | 322.94 ms | 52.3% bf16 MFU | 1622822 tok/s
step 2208/19560 | loss 3.885102 (+0.87z)| norm 0.3130 (+0.22z)| lr 5.91e-04 | 323.37 ms | 52.2% bf16 MFU | 1622748 tok/s
step 2209/19560 | loss 3.888732 (+0.98z)| norm 0.2874 (-0.58z)| lr 5.91e-04 | 322.87 ms | 52.3% bf16 MFU | 1622802 tok/s
step 2210/19560 | loss 3.836384 (-0.44z)| norm 0.2731 (-1.02z)| lr 5.91e-04 | 323.11 ms | 52.2% bf16 MFU | 1622795 tok/s
step 2211/19560 | loss 3.810434 (-1.13z)| norm 0.3021 (-0.12z)| lr 5.91e-04 | 323.36 ms | 52.2% bf16 MFU | 1622725 tok/s
step 2212/19560 | loss 3.858227 (+0.15z)| norm 0.3233 (+0.53z)| lr 5.91e-04 | 322.94 ms | 52.3% bf16 MFU | 1622762 tok/s
step 2213/19560 | loss 3.808764 (-1.17z)| norm 0.3266 (+0.64z)| lr 5.91e-04 | 323.08 ms | 52.2% bf16 MFU | 1622762 tok/s
step 2214/19560 | loss 3.828126 (-0.64z)| norm 0.3016 (-0.14z)| lr 5.91e-04 | 322.66 ms | 52.3% bf16 MFU | 1622870 tok/s
step 2215/19560 | loss 3.865067 (+0.35z)| norm 0.2749 (-0.97z)| lr 5.91e-04 | 322.82 ms | 52.3% bf16 MFU | 1622931 tok/s
step 2216/19560 | loss 3.887672 (+0.94z)| norm 0.2910 (-0.47z)| lr 5.90e-04 | 322.58 ms | 52.3% bf16 MFU | 1623049 tok/s
step 2217/19560 | loss 3.870350 (+0.47z)| norm 0.3133 (+0.23z)| lr 5.90e-04 | 323.27 ms | 52.2% bf16 MFU | 1622987 tok/s
step 2218/19560 | loss 3.780512 (-1.91z)| norm 0.2884 (-0.55z)| lr 5.90e-04 | 322.62 ms | 52.3% bf16 MFU | 1623092 tok/s
step 2219/19560 | loss 3.793497 (-1.54z)| norm 0.2782 (-0.86z)| lr 5.90e-04 | 322.64 ms | 52.3% bf16 MFU | 1623187 tok/s
step 2220/19560 | loss 3.793355 (-1.52z)| norm 0.3107 (+0.16z)| lr 5.90e-04 | 322.88 ms | 52.3% bf16 MFU | 1623217 tok/s
step 2221/19560 | loss 3.835339 (-0.41z)| norm 0.3114 (+0.21z)| lr 5.90e-04 | 323.22 ms | 52.2% bf16 MFU | 1623161 tok/s
step 2222/19560 | loss 3.825064 (-0.68z)| norm 0.2591 (-1.47z)| lr 5.90e-04 | 323.30 ms | 52.2% bf16 MFU | 1623088 tok/s
step 2223/19560 | loss 3.879581 (+0.74z)| norm 0.3004 (-0.11z)| lr 5.90e-04 | 323.10 ms | 52.2% bf16 MFU | 1623068 tok/s
step 2224/19560 | loss 3.832315 (-0.50z)| norm 0.2996 (-0.13z)| lr 5.90e-04 | 323.06 ms | 52.2% bf16 MFU | 1623058 tok/s
step 2225/19560 | loss 3.814955 (-0.94z)| norm 0.2778 (-0.84z)| lr 5.90e-04 | 322.59 ms | 52.3% bf16 MFU | 1623168 tok/s
step 2226/19560 | loss 3.843934 (-0.18z)| norm 0.3219 (+0.60z)| lr 5.90e-04 | 322.62 ms | 52.3% bf16 MFU | 1623264 tok/s
step 2227/19560 | loss 3.792624 (-1.49z)| norm 0.3071 (+0.11z)| lr 5.90e-04 | 323.44 ms | 52.2% bf16 MFU | 1623150 tok/s
step 2228/19560 | loss 3.802410 (-1.22z)| norm 0.3037 (-0.01z)| lr 5.90e-04 | 322.07 ms | 52.4% bf16 MFU | 1623387 tok/s
step 2229/19560 | loss 3.841020 (-0.21z)| norm 0.3148 (+0.36z)| lr 5.90e-04 | 322.62 ms | 52.3% bf16 MFU | 1623472 tok/s
step 2230/19560 | loss 3.818304 (-0.81z)| norm 0.3011 (-0.10z)| lr 5.90e-04 | 322.92 ms | 52.3% bf16 MFU | 1623476 tok/s
step 2231/19560 | loss 3.832697 (-0.44z)| norm 0.3087 (+0.15z)| lr 5.90e-04 | 322.92 ms | 52.3% bf16 MFU | 1623483 tok/s
step 2232/19560 | loss 3.797839 (-1.34z)| norm 0.3200 (+0.51z)| lr 5.90e-04 | 322.48 ms | 52.3% bf16 MFU | 1623599 tok/s
step 2233/19560 | loss 3.809361 (-1.04z)| norm 0.3040 (-0.03z)| lr 5.90e-04 | 322.88 ms | 52.3% bf16 MFU | 1623607 tok/s
step 2234/19560 | loss 4.021517 (+4.16z)| norm 0.2956 (-0.32z)| lr 5.90e-04 | 322.59 ms | 52.3% bf16 MFU | 1623689 tok/s
step 2235/19560 | loss 3.837027 (-0.32z)| norm 0.3024 (-0.09z)| lr 5.90e-04 | 322.38 ms | 52.4% bf16 MFU | 1623819 tok/s
step 2236/19560 | loss 3.808284 (-1.00z)| norm 0.2687 (-1.20z)| lr 5.90e-04 | 323.07 ms | 52.2% bf16 MFU | 1623771 tok/s
step 2237/19560 | loss 3.845932 (-0.09z)| norm 0.2962 (-0.28z)| lr 5.90e-04 | 322.43 ms | 52.3% bf16 MFU | 1623886 tok/s
step 2238/19560 | loss 3.837286 (-0.31z)| norm 0.3048 (+0.00z)| lr 5.90e-04 | 322.98 ms | 52.3% bf16 MFU | 1623856 tok/s
step 2239/19560 | loss 3.904487 (+1.43z)| norm 0.3178 (+0.44z)| lr 5.90e-04 | 322.71 ms | 52.3% bf16 MFU | 1623894 tok/s
step 2240/19560 | loss 3.854264 (+0.14z)| norm 0.3365 (+1.07z)| lr 5.90e-04 | 322.49 ms | 52.3% bf16 MFU | 1623986 tok/s
step 2241/19560 | loss 3.818579 (-0.77z)| norm 0.3069 (+0.10z)| lr 5.90e-04 | 322.40 ms | 52.3% bf16 MFU | 1624096 tok/s
step 2242/19560 | loss 3.793741 (-1.40z)| norm 0.2556 (-1.65z)| lr 5.90e-04 | 323.01 ms | 52.2% bf16 MFU | 1624047 tok/s
step 2243/19560 | loss 3.784155 (-1.63z)| norm 0.2929 (-0.36z)| lr 5.90e-04 | 322.84 ms | 52.3% bf16 MFU | 1624044 tok/s
step 2244/19560 | loss 3.882524 (+0.88z)| norm 0.2817 (-0.75z)| lr 5.90e-04 | 322.61 ms | 52.3% bf16 MFU | 1624100 tok/s
step 2245/19560 | loss 3.794253 (-1.35z)| norm 0.2976 (-0.21z)| lr 5.90e-04 | 322.87 ms | 52.3% bf16 MFU | 1624087 tok/s
step 2246/19560 | loss 3.858137 (+0.26z)| norm 0.3059 (+0.07z)| lr 5.90e-04 | 322.48 ms | 52.3% bf16 MFU | 1624172 tok/s
step 2247/19560 | loss 3.869950 (+0.57z)| norm 0.2976 (-0.21z)| lr 5.90e-04 | 323.32 ms | 52.2% bf16 MFU | 1624044 tok/s
step 2248/19560 | loss 3.840655 (-0.19z)| norm 0.3077 (+0.14z)| lr 5.90e-04 | 322.16 ms | 52.4% bf16 MFU | 1624211 tok/s
step 2249/19560 | loss 3.831407 (-0.44z)| norm 0.2830 (-0.72z)| lr 5.90e-04 | 322.79 ms | 52.3% bf16 MFU | 1624212 tok/s
step 2250/19560 | loss 3.810333 (-0.96z)| norm 0.3563 (+1.79z)| lr 5.90e-04 | 322.85 ms | 52.3% bf16 MFU | 1624197 tok/s
val loss 3.819037
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2609/10042 = 0.259809
step 2251/19560 | loss 3.845198 (-0.06z)| norm 0.3583 (+1.83z)| lr 5.90e-04 | 323.30 ms | 52.2% bf16 MFU | 1624071 tok/s
step 2252/19560 | loss 3.795984 (-1.31z)| norm 0.3113 (+0.24z)| lr 5.90e-04 | 323.14 ms | 52.2% bf16 MFU | 1623992 tok/s
step 2253/19560 | loss 3.800017 (-1.20z)| norm 0.3121 (+0.27z)| lr 5.90e-04 | 322.75 ms | 52.3% bf16 MFU | 1624013 tok/s
step 2254/19560 | loss 3.877542 (+0.78z)| norm 0.2932 (-0.38z)| lr 5.90e-04 | 323.23 ms | 52.2% bf16 MFU | 1623914 tok/s
step 2255/19560 | loss 3.796699 (-1.27z)| norm 0.3094 (+0.17z)| lr 5.90e-04 | 322.87 ms | 52.3% bf16 MFU | 1623910 tok/s
step 2256/19560 | loss 3.833616 (-0.32z)| norm 0.3144 (+0.34z)| lr 5.90e-04 | 323.00 ms | 52.3% bf16 MFU | 1623873 tok/s
step 2257/19560 | loss 3.777520 (-1.72z)| norm 0.2840 (-0.69z)| lr 5.90e-04 | 322.71 ms | 52.3% bf16 MFU | 1623910 tok/s
step 2258/19560 | loss 3.782048 (-1.59z)| norm 0.2626 (-1.39z)| lr 5.90e-04 | 322.94 ms | 52.3% bf16 MFU | 1623890 tok/s
step 2259/19560 | loss 3.946272 (+2.49z)| norm 0.2724 (-1.05z)| lr 5.90e-04 | 322.93 ms | 52.3% bf16 MFU | 1623871 tok/s
step 2260/19560 | loss 3.777153 (-1.66z)| norm 0.3275 (+0.79z)| lr 5.90e-04 | 322.71 ms | 52.3% bf16 MFU | 1623910 tok/s
step 2261/19560 | loss 3.831545 (-0.33z)| norm 0.4035 (+3.20z)| lr 5.90e-04 | 322.94 ms | 52.3% bf16 MFU | 1623889 tok/s
step 2262/19560 | loss 3.900838 (+1.34z)| norm 0.4147 (+3.38z)| lr 5.90e-04 | 322.91 ms | 52.3% bf16 MFU | 1623875 tok/s
step 2263/19560 | loss 3.835582 (-0.24z)| norm 0.4569 (+4.32z)| lr 5.90e-04 | 322.60 ms | 52.3% bf16 MFU | 1623942 tok/s
step 2264/19560 | loss 3.810823 (-0.83z)| norm 0.3638 (+1.59z)| lr 5.90e-04 | 323.29 ms | 52.2% bf16 MFU | 1623832 tok/s
step 2265/19560 | loss 3.789056 (-1.34z)| norm 0.3416 (+0.94z)| lr 5.90e-04 | 323.22 ms | 52.2% bf16 MFU | 1623745 tok/s
step 2266/19560 | loss 3.814459 (-0.71z)| norm 0.2626 (-1.31z)| lr 5.90e-04 | 322.85 ms | 52.3% bf16 MFU | 1623755 tok/s
step 2267/19560 | loss 3.777929 (-1.60z)| norm 0.2796 (-0.81z)| lr 5.90e-04 | 322.65 ms | 52.3% bf16 MFU | 1623815 tok/s
step 2268/19560 | loss 3.848052 (+0.14z)| norm 0.2564 (-1.51z)| lr 5.90e-04 | 322.54 ms | 52.3% bf16 MFU | 1623899 tok/s
step 2269/19560 | loss 3.996967 (+3.60z)| norm 0.2641 (-1.28z)| lr 5.90e-04 | 322.56 ms | 52.3% bf16 MFU | 1623974 tok/s
step 2270/19560 | loss 3.850561 (+0.18z)| norm 0.2670 (-1.18z)| lr 5.90e-04 | 323.14 ms | 52.2% bf16 MFU | 1623900 tok/s
step 2271/19560 | loss 3.839243 (-0.09z)| norm 0.2436 (-1.87z)| lr 5.90e-04 | 322.97 ms | 52.3% bf16 MFU | 1623872 tok/s
step 2272/19560 | loss 3.744771 (-2.26z)| norm 0.2511 (-1.61z)| lr 5.90e-04 | 322.78 ms | 52.3% bf16 MFU | 1623894 tok/s
step 2273/19560 | loss 3.814921 (-0.62z)| norm 0.2397 (-1.92z)| lr 5.90e-04 | 323.09 ms | 52.2% bf16 MFU | 1623836 tok/s
step 2274/19560 | loss 3.805678 (-0.84z)| norm 0.2497 (-1.58z)| lr 5.90e-04 | 322.56 ms | 52.3% bf16 MFU | 1623913 tok/s
step 2275/19560 | loss 3.886713 (+1.04z)| norm 0.2717 (-0.91z)| lr 5.90e-04 | 322.64 ms | 52.3% bf16 MFU | 1623967 tok/s
step 2276/19560 | loss 3.830768 (-0.25z)| norm 0.2593 (-1.27z)| lr 5.90e-04 | 322.72 ms | 52.3% bf16 MFU | 1623997 tok/s
step 2277/19560 | loss 3.899174 (+1.32z)| norm 0.3019 (+0.00z)| lr 5.90e-04 | 323.03 ms | 52.2% bf16 MFU | 1623949 tok/s
step 2278/19560 | loss 3.820830 (-0.48z)| norm 0.2966 (-0.16z)| lr 5.90e-04 | 322.50 ms | 52.3% bf16 MFU | 1624035 tok/s
step 2279/19560 | loss 3.784799 (-1.30z)| norm 0.3386 (+1.09z)| lr 5.90e-04 | 322.61 ms | 52.3% bf16 MFU | 1624091 tok/s
step 2280/19560 | loss 3.864542 (+0.52z)| norm 0.3116 (+0.27z)| lr 5.90e-04 | 322.86 ms | 52.3% bf16 MFU | 1624080 tok/s
step 2281/19560 | loss 3.848513 (+0.16z)| norm 0.2841 (-0.55z)| lr 5.90e-04 | 323.02 ms | 52.2% bf16 MFU | 1624031 tok/s
step 2282/19560 | loss 3.806200 (-0.82z)| norm 0.2934 (-0.27z)| lr 5.90e-04 | 323.06 ms | 52.2% bf16 MFU | 1623972 tok/s
step 2283/19560 | loss 3.811474 (-0.68z)| norm 0.2777 (-0.75z)| lr 5.90e-04 | 322.97 ms | 52.3% bf16 MFU | 1623942 tok/s
step 2284/19560 | loss 3.780030 (-1.39z)| norm 0.2924 (-0.31z)| lr 5.90e-04 | 323.37 ms | 52.2% bf16 MFU | 1623811 tok/s
step 2285/19560 | loss 3.782219 (-1.32z)| norm 0.3282 (+0.76z)| lr 5.90e-04 | 322.56 ms | 52.3% bf16 MFU | 1623892 tok/s
step 2286/19560 | loss 3.812474 (-0.62z)| norm 0.2943 (-0.26z)| lr 5.90e-04 | 322.86 ms | 52.3% bf16 MFU | 1623892 tok/s
step 2287/19560 | loss 3.839877 (+0.01z)| norm 0.2852 (-0.53z)| lr 5.90e-04 | 323.11 ms | 52.2% bf16 MFU | 1623828 tok/s
step 2288/19560 | loss 3.827256 (-0.28z)| norm 0.3392 (+1.07z)| lr 5.90e-04 | 322.61 ms | 52.3% bf16 MFU | 1623893 tok/s
step 2289/19560 | loss 3.826686 (-0.28z)| norm 0.3448 (+1.22z)| lr 5.90e-04 | 322.88 ms | 52.3% bf16 MFU | 1623887 tok/s
step 2290/19560 | loss 3.840833 (+0.05z)| norm 0.2791 (-0.75z)| lr 5.90e-04 | 323.56 ms | 52.2% bf16 MFU | 1623711 tok/s
step 2291/19560 | loss 3.809868 (-0.65z)| norm 0.2886 (-0.48z)| lr 5.90e-04 | 322.58 ms | 52.3% bf16 MFU | 1623790 tok/s
step 2292/19560 | loss 3.820103 (-0.41z)| norm 0.2697 (-1.04z)| lr 5.90e-04 | 322.72 ms | 52.3% bf16 MFU | 1623832 tok/s
step 2293/19560 | loss 3.836496 (-0.02z)| norm 0.2784 (-0.77z)| lr 5.90e-04 | 323.47 ms | 52.2% bf16 MFU | 1623681 tok/s
step 2294/19560 | loss 3.848221 (+0.26z)| norm 0.2602 (-1.30z)| lr 5.90e-04 | 322.68 ms | 52.3% bf16 MFU | 1623737 tok/s
step 2295/19560 | loss 3.815930 (-0.48z)| norm 0.2736 (-0.90z)| lr 5.89e-04 | 323.45 ms | 52.2% bf16 MFU | 1623596 tok/s
step 2296/19560 | loss 3.748985 (-2.01z)| norm 0.2628 (-1.22z)| lr 5.89e-04 | 322.84 ms | 52.3% bf16 MFU | 1623615 tok/s
step 2297/19560 | loss 3.873532 (+0.88z)| norm 0.2983 (-0.10z)| lr 5.89e-04 | 323.03 ms | 52.2% bf16 MFU | 1623586 tok/s
step 2298/19560 | loss 3.789026 (-1.07z)| norm 0.2778 (-0.73z)| lr 5.89e-04 | 322.95 ms | 52.3% bf16 MFU | 1623578 tok/s
step 2299/19560 | loss 3.798212 (-0.84z)| norm 0.2902 (-0.34z)| lr 5.89e-04 | 322.22 ms | 52.4% bf16 MFU | 1623754 tok/s
step 2300/19560 | loss 3.852995 (+0.41z)| norm 0.2905 (-0.34z)| lr 5.89e-04 | 323.70 ms | 52.1% bf16 MFU | 1623550 tok/s
step 2301/19560 | loss 3.803387 (-0.73z)| norm 0.2798 (-0.67z)| lr 5.89e-04 | 323.45 ms | 52.2% bf16 MFU | 1623419 tok/s
step 2302/19560 | loss 3.835579 (+0.00z)| norm 0.2946 (-0.21z)| lr 5.89e-04 | 323.08 ms | 52.2% bf16 MFU | 1623387 tok/s
step 2303/19560 | loss 3.765421 (-1.60z)| norm 0.3149 (+0.43z)| lr 5.89e-04 | 323.24 ms | 52.2% bf16 MFU | 1623317 tok/s
step 2304/19560 | loss 3.788161 (-1.08z)| norm 0.3428 (+1.29z)| lr 5.89e-04 | 322.83 ms | 52.3% bf16 MFU | 1623352 tok/s
step 2305/19560 | loss 3.824896 (-0.22z)| norm 0.3241 (+0.70z)| lr 5.89e-04 | 322.50 ms | 52.3% bf16 MFU | 1623470 tok/s
step 2306/19560 | loss 3.799635 (-0.80z)| norm 0.3065 (+0.15z)| lr 5.89e-04 | 323.19 ms | 52.2% bf16 MFU | 1623408 tok/s
step 2307/19560 | loss 3.820621 (-0.31z)| norm 0.2820 (-0.61z)| lr 5.89e-04 | 323.01 ms | 52.2% bf16 MFU | 1623393 tok/s
step 2308/19560 | loss 3.860807 (+0.61z)| norm 0.2687 (-1.02z)| lr 5.89e-04 | 323.01 ms | 52.3% bf16 MFU | 1623381 tok/s
step 2309/19560 | loss 3.809770 (-0.56z)| norm 0.2728 (-0.89z)| lr 5.89e-04 | 322.90 ms | 52.3% bf16 MFU | 1623397 tok/s
step 2310/19560 | loss 3.809581 (-0.55z)| norm 0.2755 (-0.80z)| lr 5.89e-04 | 323.13 ms | 52.2% bf16 MFU | 1623354 tok/s
step 2311/19560 | loss 3.738820 (-2.15z)| norm 0.2704 (-0.96z)| lr 5.89e-04 | 323.14 ms | 52.2% bf16 MFU | 1623310 tok/s
step 2312/19560 | loss 3.718814 (-2.54z)| norm 0.2817 (-0.61z)| lr 5.89e-04 | 322.80 ms | 52.3% bf16 MFU | 1623354 tok/s
step 2313/19560 | loss 3.778263 (-1.19z)| norm 0.2793 (-0.68z)| lr 5.89e-04 | 322.87 ms | 52.3% bf16 MFU | 1623378 tok/s
step 2314/19560 | loss 3.836004 (+0.10z)| norm 0.2859 (-0.48z)| lr 5.89e-04 | 323.15 ms | 52.2% bf16 MFU | 1623330 tok/s
step 2315/19560 | loss 3.871470 (+0.91z)| norm 0.2781 (-0.71z)| lr 5.89e-04 | 323.01 ms | 52.3% bf16 MFU | 1623321 tok/s
step 2316/19560 | loss 3.855022 (+0.54z)| norm 0.2909 (-0.31z)| lr 5.89e-04 | 323.11 ms | 52.2% bf16 MFU | 1623286 tok/s
step 2317/19560 | loss 3.893150 (+1.40z)| norm 0.3008 (-0.00z)| lr 5.89e-04 | 322.67 ms | 52.3% bf16 MFU | 1623364 tok/s
step 2318/19560 | loss 3.776022 (-1.24z)| norm 0.2944 (-0.21z)| lr 5.89e-04 | 323.09 ms | 52.2% bf16 MFU | 1623332 tok/s
step 2319/19560 | loss 3.793742 (-0.83z)| norm 0.2874 (-0.42z)| lr 5.89e-04 | 322.99 ms | 52.3% bf16 MFU | 1623327 tok/s
step 2320/19560 | loss 3.753922 (-1.70z)| norm 0.3050 (+0.12z)| lr 5.89e-04 | 322.64 ms | 52.3% bf16 MFU | 1623412 tok/s
step 2321/19560 | loss 3.800998 (-0.64z)| norm 0.2829 (-0.56z)| lr 5.89e-04 | 323.58 ms | 52.2% bf16 MFU | 1623255 tok/s
step 2322/19560 | loss 3.851397 (+0.48z)| norm 0.2946 (-0.19z)| lr 5.89e-04 | 322.97 ms | 52.3% bf16 MFU | 1623258 tok/s
step 2323/19560 | loss 3.804776 (-0.55z)| norm 0.3291 (+0.88z)| lr 5.89e-04 | 323.23 ms | 52.2% bf16 MFU | 1623198 tok/s
step 2324/19560 | loss 3.802111 (-0.62z)| norm 0.3297 (+0.90z)| lr 5.89e-04 | 323.03 ms | 52.2% bf16 MFU | 1623190 tok/s
step 2325/19560 | loss 3.917960 (+1.93z)| norm 0.3151 (+0.45z)| lr 5.89e-04 | 322.96 ms | 52.3% bf16 MFU | 1623199 tok/s
step 2326/19560 | loss 3.851144 (+0.45z)| norm 0.3080 (+0.24z)| lr 5.89e-04 | 323.02 ms | 52.2% bf16 MFU | 1623193 tok/s
step 2327/19560 | loss 3.824095 (-0.14z)| norm 0.3502 (+1.54z)| lr 5.89e-04 | 323.23 ms | 52.2% bf16 MFU | 1623133 tok/s
step 2328/19560 | loss 3.834930 (+0.10z)| norm 0.3183 (+0.54z)| lr 5.89e-04 | 323.19 ms | 52.2% bf16 MFU | 1623089 tok/s
step 2329/19560 | loss 3.797446 (-0.71z)| norm 0.2935 (-0.22z)| lr 5.89e-04 | 322.68 ms | 52.3% bf16 MFU | 1623176 tok/s
step 2330/19560 | loss 3.797718 (-0.70z)| norm 0.2921 (-0.26z)| lr 5.89e-04 | 322.93 ms | 52.3% bf16 MFU | 1623193 tok/s
step 2331/19560 | loss 3.775640 (-1.18z)| norm 0.2734 (-0.83z)| lr 5.89e-04 | 323.44 ms | 52.2% bf16 MFU | 1623083 tok/s
step 2332/19560 | loss 3.833792 (+0.12z)| norm 0.2921 (-0.24z)| lr 5.89e-04 | 322.67 ms | 52.3% bf16 MFU | 1623170 tok/s
step 2333/19560 | loss 3.828542 (+0.01z)| norm 0.2754 (-0.75z)| lr 5.89e-04 | 323.52 ms | 52.2% bf16 MFU | 1623042 tok/s
step 2334/19560 | loss 3.775408 (-1.16z)| norm 0.2730 (-0.81z)| lr 5.89e-04 | 322.88 ms | 52.3% bf16 MFU | 1623080 tok/s
step 2335/19560 | loss 3.787076 (-0.89z)| norm 0.2938 (-0.15z)| lr 5.89e-04 | 322.60 ms | 52.3% bf16 MFU | 1623186 tok/s
step 2336/19560 | loss 3.817269 (-0.20z)| norm 0.2830 (-0.48z)| lr 5.89e-04 | 323.27 ms | 52.2% bf16 MFU | 1623118 tok/s
step 2337/19560 | loss 3.751443 (-1.66z)| norm 0.2774 (-0.66z)| lr 5.89e-04 | 322.80 ms | 52.3% bf16 MFU | 1623171 tok/s
step 2338/19560 | loss 3.786089 (-0.86z)| norm 0.3036 (+0.17z)| lr 5.89e-04 | 323.30 ms | 52.2% bf16 MFU | 1623097 tok/s
step 2339/19560 | loss 3.784770 (-0.89z)| norm 0.3057 (+0.23z)| lr 5.89e-04 | 322.71 ms | 52.3% bf16 MFU | 1623174 tok/s
step 2340/19560 | loss 3.799827 (-0.54z)| norm 0.2784 (-0.63z)| lr 5.89e-04 | 322.89 ms | 52.3% bf16 MFU | 1623202 tok/s
step 2341/19560 | loss 3.802803 (-0.47z)| norm 0.2891 (-0.28z)| lr 5.89e-04 | 324.41 ms | 52.0% bf16 MFU | 1622848 tok/s
step 2342/19560 | loss 3.833994 (+0.23z)| norm 0.3075 (+0.31z)| lr 5.89e-04 | 322.19 ms | 52.4% bf16 MFU | 1623069 tok/s
step 2343/19560 | loss 3.844184 (+0.46z)| norm 0.2989 (+0.03z)| lr 5.89e-04 | 322.96 ms | 52.3% bf16 MFU | 1623085 tok/s
step 2344/19560 | loss 3.799263 (-0.54z)| norm 0.2915 (-0.21z)| lr 5.89e-04 | 323.66 ms | 52.1% bf16 MFU | 1622925 tok/s
step 2345/19560 | loss 3.785121 (-0.84z)| norm 0.3078 (+0.31z)| lr 5.89e-04 | 322.95 ms | 52.3% bf16 MFU | 1622950 tok/s
step 2346/19560 | loss 3.766305 (-1.27z)| norm 0.3555 (+1.80z)| lr 5.89e-04 | 323.39 ms | 52.2% bf16 MFU | 1622862 tok/s
step 2347/19560 | loss 3.826642 (+0.10z)| norm 0.3450 (+1.44z)| lr 5.89e-04 | 323.18 ms | 52.2% bf16 MFU | 1622833 tok/s
step 2348/19560 | loss 3.747540 (-1.68z)| norm 0.3019 (+0.09z)| lr 5.89e-04 | 322.82 ms | 52.3% bf16 MFU | 1622896 tok/s
step 2349/19560 | loss 3.791402 (-0.68z)| norm 0.3018 (+0.09z)| lr 5.89e-04 | 322.89 ms | 52.3% bf16 MFU | 1622938 tok/s
step 2350/19560 | loss 3.812991 (-0.19z)| norm 0.2945 (-0.15z)| lr 5.89e-04 | 323.51 ms | 52.2% bf16 MFU | 1622821 tok/s
step 2351/19560 | loss 3.773593 (-1.06z)| norm 0.2711 (-0.88z)| lr 5.89e-04 | 322.81 ms | 52.3% bf16 MFU | 1622888 tok/s
step 2352/19560 | loss 3.791281 (-0.66z)| norm 0.2837 (-0.47z)| lr 5.89e-04 | 322.77 ms | 52.3% bf16 MFU | 1622961 tok/s
step 2353/19560 | loss 3.775766 (-0.99z)| norm 0.2745 (-0.76z)| lr 5.89e-04 | 322.47 ms | 52.3% bf16 MFU | 1623104 tok/s
step 2354/19560 | loss 3.837745 (+0.39z)| norm 0.5705 (+6.79z)| lr 5.89e-04 | 322.92 ms | 52.3% bf16 MFU | 1623128 tok/s
step 2355/19560 | loss 3.787466 (-0.73z)| norm 0.4111 (+2.68z)| lr 5.89e-04 | 322.33 ms | 52.4% bf16 MFU | 1623299 tok/s
step 2356/19560 | loss 3.764937 (-1.22z)| norm 0.4760 (+3.96z)| lr 5.89e-04 | 323.57 ms | 52.2% bf16 MFU | 1623151 tok/s
step 2357/19560 | loss 3.837166 (+0.39z)| norm 0.3706 (+1.53z)| lr 5.89e-04 | 322.50 ms | 52.3% bf16 MFU | 1623278 tok/s
step 2358/19560 | loss 3.838485 (+0.41z)| norm 0.3141 (+0.24z)| lr 5.89e-04 | 322.82 ms | 52.3% bf16 MFU | 1623319 tok/s
step 2359/19560 | loss 3.788975 (-0.68z)| norm 0.3170 (+0.31z)| lr 5.89e-04 | 322.90 ms | 52.3% bf16 MFU | 1623337 tok/s
step 2360/19560 | loss 3.770358 (-1.08z)| norm 0.2954 (-0.18z)| lr 5.89e-04 | 322.73 ms | 52.3% bf16 MFU | 1623398 tok/s
step 2361/19560 | loss 3.830543 (+0.24z)| norm 0.2908 (-0.28z)| lr 5.89e-04 | 322.75 ms | 52.3% bf16 MFU | 1623451 tok/s
step 2362/19560 | loss 3.790244 (-0.66z)| norm 0.2812 (-0.49z)| lr 5.89e-04 | 323.16 ms | 52.2% bf16 MFU | 1623398 tok/s
step 2363/19560 | loss 3.826760 (+0.22z)| norm 0.2603 (-0.96z)| lr 5.89e-04 | 323.06 ms | 52.2% bf16 MFU | 1623374 tok/s
step 2364/19560 | loss 3.809740 (-0.19z)| norm 0.2552 (-1.07z)| lr 5.89e-04 | 323.10 ms | 52.2% bf16 MFU | 1623340 tok/s
step 2365/19560 | loss 3.823277 (+0.14z)| norm 0.2717 (-0.69z)| lr 5.89e-04 | 322.55 ms | 52.3% bf16 MFU | 1623445 tok/s
step 2366/19560 | loss 3.809357 (-0.19z)| norm 0.3656 (+1.40z)| lr 5.89e-04 | 323.49 ms | 52.2% bf16 MFU | 1623308 tok/s
step 2367/19560 | loss 3.872110 (+1.34z)| norm 0.2986 (-0.09z)| lr 5.89e-04 | 323.32 ms | 52.2% bf16 MFU | 1623221 tok/s
step 2368/19560 | loss 3.824281 (+0.18z)| norm 0.3327 (+0.67z)| lr 5.89e-04 | 322.92 ms | 52.3% bf16 MFU | 1623240 tok/s
step 2369/19560 | loss 3.863683 (+1.13z)| norm 0.3029 (+0.00z)| lr 5.88e-04 | 323.01 ms | 52.3% bf16 MFU | 1623235 tok/s
step 2370/19560 | loss 3.846398 (+0.70z)| norm 0.3277 (+0.55z)| lr 5.88e-04 | 322.59 ms | 52.3% bf16 MFU | 1623336 tok/s
step 2371/19560 | loss 3.859132 (+1.00z)| norm 0.2999 (-0.08z)| lr 5.88e-04 | 322.96 ms | 52.3% bf16 MFU | 1623338 tok/s
step 2372/19560 | loss 3.838985 (+0.52z)| norm 0.2880 (-0.34z)| lr 5.88e-04 | 323.11 ms | 52.2% bf16 MFU | 1623303 tok/s
step 2373/19560 | loss 3.785880 (-0.78z)| norm 0.3224 (+0.42z)| lr 5.88e-04 | 323.25 ms | 52.2% bf16 MFU | 1623234 tok/s
step 2374/19560 | loss 3.821207 (+0.09z)| norm 0.3339 (+0.67z)| lr 5.88e-04 | 322.71 ms | 52.3% bf16 MFU | 1623304 tok/s
step 2375/19560 | loss 3.796314 (-0.51z)| norm 0.3029 (-0.02z)| lr 5.88e-04 | 323.33 ms | 52.2% bf16 MFU | 1623216 tok/s
step 2376/19560 | loss 3.814389 (-0.06z)| norm 0.2543 (-1.09z)| lr 5.88e-04 | 323.05 ms | 52.2% bf16 MFU | 1623202 tok/s
step 2377/19560 | loss 3.787122 (-0.72z)| norm 0.2588 (-0.99z)| lr 5.88e-04 | 322.96 ms | 52.3% bf16 MFU | 1623211 tok/s
step 2378/19560 | loss 3.823843 (+0.18z)| norm 0.2405 (-1.37z)| lr 5.88e-04 | 323.02 ms | 52.2% bf16 MFU | 1623205 tok/s
step 2379/19560 | loss 3.746772 (-1.69z)| norm 0.2749 (-0.60z)| lr 5.88e-04 | 322.53 ms | 52.3% bf16 MFU | 1623321 tok/s
step 2380/19560 | loss 3.818555 (+0.07z)| norm 0.2667 (-0.77z)| lr 5.88e-04 | 323.41 ms | 52.2% bf16 MFU | 1623211 tok/s
step 2381/19560 | loss 3.800004 (-0.39z)| norm 0.2770 (-0.53z)| lr 5.88e-04 | 323.18 ms | 52.2% bf16 MFU | 1623165 tok/s
step 2382/19560 | loss 3.845452 (+0.74z)| norm 0.2671 (-0.75z)| lr 5.88e-04 | 323.30 ms | 52.2% bf16 MFU | 1623092 tok/s
step 2383/19560 | loss 3.795957 (-0.48z)| norm 0.2675 (-0.73z)| lr 5.88e-04 | 322.86 ms | 52.3% bf16 MFU | 1623131 tok/s
step 2384/19560 | loss 3.842443 (+0.66z)| norm 0.2994 (-0.02z)| lr 5.88e-04 | 323.01 ms | 52.2% bf16 MFU | 1623130 tok/s
step 2385/19560 | loss 3.821863 (+0.15z)| norm 0.2847 (-0.35z)| lr 5.88e-04 | 322.79 ms | 52.3% bf16 MFU | 1623185 tok/s
step 2386/19560 | loss 3.793858 (-0.55z)| norm 0.2771 (-0.52z)| lr 5.88e-04 | 322.89 ms | 52.3% bf16 MFU | 1623212 tok/s
step 2387/19560 | loss 3.761661 (-1.36z)| norm 0.3401 (+0.86z)| lr 5.88e-04 | 322.73 ms | 52.3% bf16 MFU | 1623278 tok/s
step 2388/19560 | loss 3.780739 (-0.87z)| norm 0.3011 (+0.01z)| lr 5.88e-04 | 322.80 ms | 52.3% bf16 MFU | 1623323 tok/s
step 2389/19560 | loss 3.786953 (-0.70z)| norm 0.2989 (-0.03z)| lr 5.88e-04 | 323.09 ms | 52.2% bf16 MFU | 1623294 tok/s
step 2390/19560 | loss 3.771441 (-1.09z)| norm 0.2935 (-0.13z)| lr 5.88e-04 | 322.69 ms | 52.3% bf16 MFU | 1623366 tok/s
step 2391/19560 | loss 3.807467 (-0.15z)| norm 0.3219 (+0.59z)| lr 5.88e-04 | 322.74 ms | 52.3% bf16 MFU | 1623421 tok/s
step 2392/19560 | loss 3.784799 (-0.73z)| norm 0.2858 (-0.29z)| lr 5.88e-04 | 323.48 ms | 52.2% bf16 MFU | 1623287 tok/s
step 2393/19560 | loss 3.703033 (-2.76z)| norm 0.3578 (+1.49z)| lr 5.88e-04 | 322.78 ms | 52.3% bf16 MFU | 1623338 tok/s
step 2394/19560 | loss 3.806237 (-0.15z)| norm 0.3625 (+1.57z)| lr 5.88e-04 | 322.51 ms | 52.3% bf16 MFU | 1623454 tok/s
step 2395/19560 | loss 3.721497 (-2.24z)| norm 0.3354 (+0.90z)| lr 5.88e-04 | 323.32 ms | 52.2% bf16 MFU | 1623360 tok/s
step 2396/19560 | loss 3.891065 (+1.94z)| norm 0.3359 (+0.89z)| lr 5.88e-04 | 322.93 ms | 52.3% bf16 MFU | 1623368 tok/s
step 2397/19560 | loss 3.799025 (-0.31z)| norm 0.2990 (-0.02z)| lr 5.88e-04 | 322.85 ms | 52.3% bf16 MFU | 1623397 tok/s
step 2398/19560 | loss 3.798177 (-0.32z)| norm 0.3009 (+0.02z)| lr 5.88e-04 | 323.25 ms | 52.2% bf16 MFU | 1623323 tok/s
step 2399/19560 | loss 3.811796 (+0.05z)| norm 0.3016 (+0.03z)| lr 5.88e-04 | 323.45 ms | 52.2% bf16 MFU | 1623202 tok/s
step 2400/19560 | loss 3.813511 (+0.08z)| norm 0.2987 (-0.05z)| lr 5.88e-04 | 322.97 ms | 52.3% bf16 MFU | 1623208 tok/s
step 2401/19560 | loss 3.821557 (+0.30z)| norm 0.3088 (+0.19z)| lr 5.88e-04 | 322.40 ms | 52.3% bf16 MFU | 1623358 tok/s
step 2402/19560 | loss 3.801061 (-0.26z)| norm 0.2818 (-0.50z)| lr 5.88e-04 | 322.58 ms | 52.3% bf16 MFU | 1623454 tok/s
step 2403/19560 | loss 3.803312 (-0.18z)| norm 0.2773 (-0.61z)| lr 5.88e-04 | 322.91 ms | 52.3% bf16 MFU | 1623463 tok/s
step 2404/19560 | loss 3.736094 (-2.00z)| norm 0.2821 (-0.50z)| lr 5.88e-04 | 323.07 ms | 52.2% bf16 MFU | 1623432 tok/s
step 2405/19560 | loss 3.805214 (-0.09z)| norm 0.2872 (-0.37z)| lr 5.88e-04 | 322.88 ms | 52.3% bf16 MFU | 1623450 tok/s
step 2406/19560 | loss 3.777059 (-0.86z)| norm 0.2879 (-0.35z)| lr 5.88e-04 | 322.79 ms | 52.3% bf16 MFU | 1623489 tok/s
step 2407/19560 | loss 3.828737 (+0.57z)| norm 0.2694 (-0.80z)| lr 5.88e-04 | 323.12 ms | 52.2% bf16 MFU | 1623444 tok/s
step 2408/19560 | loss 3.743880 (-1.77z)| norm 0.2904 (-0.27z)| lr 5.88e-04 | 322.35 ms | 52.4% bf16 MFU | 1623596 tok/s
step 2409/19560 | loss 3.807560 (+0.01z)| norm 0.2850 (-0.40z)| lr 5.88e-04 | 322.51 ms | 52.3% bf16 MFU | 1623698 tok/s
step 2410/19560 | loss 3.777904 (-0.81z)| norm 0.2821 (-0.47z)| lr 5.88e-04 | 322.82 ms | 52.3% bf16 MFU | 1623718 tok/s
step 2411/19560 | loss 3.816594 (+0.27z)| norm 0.2554 (-1.15z)| lr 5.88e-04 | 323.27 ms | 52.2% bf16 MFU | 1623623 tok/s
step 2412/19560 | loss 3.746941 (-1.65z)| norm 0.2679 (-0.82z)| lr 5.88e-04 | 322.84 ms | 52.3% bf16 MFU | 1623640 tok/s
step 2413/19560 | loss 3.733947 (-1.98z)| norm 0.2846 (-0.39z)| lr 5.88e-04 | 322.67 ms | 52.3% bf16 MFU | 1623701 tok/s
step 2414/19560 | loss 3.731327 (-2.00z)| norm 0.2999 (-0.00z)| lr 5.88e-04 | 323.22 ms | 52.2% bf16 MFU | 1623620 tok/s
step 2415/19560 | loss 3.788291 (-0.46z)| norm 0.3147 (+0.36z)| lr 5.88e-04 | 322.42 ms | 52.3% bf16 MFU | 1623743 tok/s
step 2416/19560 | loss 3.796772 (-0.22z)| norm 0.3365 (+0.91z)| lr 5.88e-04 | 322.29 ms | 52.4% bf16 MFU | 1623894 tok/s
step 2417/19560 | loss 3.781199 (-0.63z)| norm 0.3116 (+0.29z)| lr 5.88e-04 | 323.42 ms | 52.2% bf16 MFU | 1623751 tok/s
step 2418/19560 | loss 3.810256 (+0.16z)| norm 0.3031 (+0.07z)| lr 5.88e-04 | 322.47 ms | 52.3% bf16 MFU | 1623856 tok/s
step 2419/19560 | loss 3.768143 (-0.97z)| norm 0.2855 (-0.37z)| lr 5.88e-04 | 322.99 ms | 52.3% bf16 MFU | 1623825 tok/s
step 2420/19560 | loss 3.745945 (-1.54z)| norm 0.2908 (-0.25z)| lr 5.88e-04 | 323.04 ms | 52.2% bf16 MFU | 1623782 tok/s
step 2421/19560 | loss 3.760942 (-1.12z)| norm 0.2766 (-0.61z)| lr 5.88e-04 | 322.72 ms | 52.3% bf16 MFU | 1623823 tok/s
step 2422/19560 | loss 3.802832 (+0.01z)| norm 0.2711 (-0.75z)| lr 5.88e-04 | 322.96 ms | 52.3% bf16 MFU | 1623802 tok/s
step 2423/19560 | loss 3.718697 (-2.19z)| norm 0.2896 (-0.28z)| lr 5.88e-04 | 322.63 ms | 52.3% bf16 MFU | 1623864 tok/s
step 2424/19560 | loss 3.781273 (-0.55z)| norm 0.2681 (-0.83z)| lr 5.88e-04 | 323.06 ms | 52.2% bf16 MFU | 1623815 tok/s
step 2425/19560 | loss 3.829536 (+0.75z)| norm 0.3037 (+0.08z)| lr 5.88e-04 | 322.95 ms | 52.3% bf16 MFU | 1623797 tok/s
step 2426/19560 | loss 3.792250 (-0.25z)| norm 0.3200 (+0.49z)| lr 5.88e-04 | 322.57 ms | 52.3% bf16 MFU | 1623874 tok/s
step 2427/19560 | loss 3.777773 (-0.64z)| norm 0.3545 (+1.35z)| lr 5.88e-04 | 323.00 ms | 52.3% bf16 MFU | 1623839 tok/s
step 2428/19560 | loss 3.848796 (+1.27z)| norm 0.3890 (+2.17z)| lr 5.88e-04 | 322.86 ms | 52.3% bf16 MFU | 1623842 tok/s
step 2429/19560 | loss 3.702137 (-2.58z)| norm 0.3351 (+0.81z)| lr 5.88e-04 | 323.09 ms | 52.2% bf16 MFU | 1623786 tok/s
step 2430/19560 | loss 3.786343 (-0.37z)| norm 0.3211 (+0.45z)| lr 5.88e-04 | 322.51 ms | 52.3% bf16 MFU | 1623880 tok/s
step 2431/19560 | loss 3.892754 (+2.35z)| norm 0.3752 (+1.77z)| lr 5.88e-04 | 322.94 ms | 52.3% bf16 MFU | 1623861 tok/s
step 2432/19560 | loss 3.822172 (+0.53z)| norm 0.3024 (-0.02z)| lr 5.88e-04 | 323.45 ms | 52.2% bf16 MFU | 1623714 tok/s
step 2433/19560 | loss 3.783959 (-0.45z)| norm 0.2761 (-0.66z)| lr 5.88e-04 | 322.62 ms | 52.3% bf16 MFU | 1623783 tok/s
step 2434/19560 | loss 3.843686 (+1.08z)| norm 0.2674 (-0.86z)| lr 5.88e-04 | 323.48 ms | 52.2% bf16 MFU | 1623632 tok/s
step 2435/19560 | loss 3.753422 (-1.22z)| norm 0.3129 (+0.25z)| lr 5.88e-04 | 322.81 ms | 52.3% bf16 MFU | 1623658 tok/s
step 2436/19560 | loss 3.817996 (+0.44z)| norm 0.3133 (+0.26z)| lr 5.88e-04 | 322.71 ms | 52.3% bf16 MFU | 1623707 tok/s
step 2437/19560 | loss 3.898489 (+2.44z)| norm 0.3149 (+0.29z)| lr 5.88e-04 | 322.83 ms | 52.3% bf16 MFU | 1623725 tok/s
step 2438/19560 | loss 3.801486 (+0.00z)| norm 0.3374 (+0.83z)| lr 5.88e-04 | 322.99 ms | 52.3% bf16 MFU | 1623699 tok/s
step 2439/19560 | loss 3.836968 (+0.88z)| norm 0.3077 (+0.09z)| lr 5.88e-04 | 323.26 ms | 52.2% bf16 MFU | 1623608 tok/s
step 2440/19560 | loss 3.835354 (+0.83z)| norm 0.2971 (-0.18z)| lr 5.88e-04 | 322.96 ms | 52.3% bf16 MFU | 1623598 tok/s
step 2441/19560 | loss 3.762598 (-1.03z)| norm 0.3062 (+0.05z)| lr 5.87e-04 | 322.05 ms | 52.4% bf16 MFU | 1623817 tok/s
step 2442/19560 | loss 3.747158 (-1.40z)| norm 0.2832 (-0.53z)| lr 5.87e-04 | 322.90 ms | 52.3% bf16 MFU | 1623812 tok/s
step 2443/19560 | loss 3.814126 (+0.32z)| norm 0.2765 (-0.69z)| lr 5.87e-04 | 322.90 ms | 52.3% bf16 MFU | 1623804 tok/s
step 2444/19560 | loss 3.816314 (+0.39z)| norm 0.2924 (-0.30z)| lr 5.87e-04 | 322.71 ms | 52.3% bf16 MFU | 1623845 tok/s
step 2445/19560 | loss 3.792135 (-0.23z)| norm 0.2857 (-0.46z)| lr 5.87e-04 | 322.90 ms | 52.3% bf16 MFU | 1623837 tok/s
step 2446/19560 | loss 3.821135 (+0.53z)| norm 0.2627 (-1.02z)| lr 5.87e-04 | 322.45 ms | 52.3% bf16 MFU | 1623944 tok/s
step 2447/19560 | loss 3.817029 (+0.42z)| norm 0.2672 (-0.90z)| lr 5.87e-04 | 323.19 ms | 52.2% bf16 MFU | 1623859 tok/s
step 2448/19560 | loss 3.804526 (+0.08z)| norm 0.2972 (-0.16z)| lr 5.87e-04 | 322.58 ms | 52.3% bf16 MFU | 1623930 tok/s
step 2449/19560 | loss 3.799471 (-0.06z)| norm 0.3367 (+0.80z)| lr 5.87e-04 | 323.20 ms | 52.2% bf16 MFU | 1623842 tok/s
step 2450/19560 | loss 3.764297 (-0.98z)| norm 0.3217 (+0.43z)| lr 5.87e-04 | 322.38 ms | 52.4% bf16 MFU | 1623965 tok/s
step 2451/19560 | loss 3.801799 (+0.02z)| norm 0.3449 (+0.99z)| lr 5.87e-04 | 322.56 ms | 52.3% bf16 MFU | 1624036 tok/s
step 2452/19560 | loss 3.752546 (-1.28z)| norm 0.2941 (-0.25z)| lr 5.87e-04 | 322.83 ms | 52.3% bf16 MFU | 1624035 tok/s
step 2453/19560 | loss 3.839753 (+1.10z)| norm 0.2628 (-1.01z)| lr 5.87e-04 | 323.27 ms | 52.2% bf16 MFU | 1623923 tok/s
step 2454/19560 | loss 3.790058 (-0.26z)| norm 0.2620 (-1.01z)| lr 5.87e-04 | 322.71 ms | 52.3% bf16 MFU | 1623959 tok/s
step 2455/19560 | loss 3.747726 (-1.41z)| norm 0.2911 (-0.29z)| lr 5.87e-04 | 323.03 ms | 52.2% bf16 MFU | 1623913 tok/s
step 2456/19560 | loss 3.799814 (+0.03z)| norm 0.2902 (-0.31z)| lr 5.87e-04 | 322.50 ms | 52.3% bf16 MFU | 1624003 tok/s
step 2457/19560 | loss 3.787413 (-0.31z)| norm 0.2768 (-0.63z)| lr 5.87e-04 | 322.80 ms | 52.3% bf16 MFU | 1624012 tok/s
step 2458/19560 | loss 3.826602 (+0.77z)| norm 0.3063 (+0.09z)| lr 5.87e-04 | 322.77 ms | 52.3% bf16 MFU | 1624029 tok/s
step 2459/19560 | loss 3.799251 (+0.01z)| norm 0.2843 (-0.46z)| lr 5.87e-04 | 322.80 ms | 52.3% bf16 MFU | 1624037 tok/s
step 2460/19560 | loss 3.815897 (+0.48z)| norm 0.2573 (-1.11z)| lr 5.87e-04 | 322.68 ms | 52.3% bf16 MFU | 1624075 tok/s
step 2461/19560 | loss 3.819885 (+0.59z)| norm 0.2567 (-1.12z)| lr 5.87e-04 | 322.98 ms | 52.3% bf16 MFU | 1624036 tok/s
step 2462/19560 | loss 3.760215 (-1.06z)| norm 0.2641 (-0.93z)| lr 5.87e-04 | 322.64 ms | 52.3% bf16 MFU | 1624085 tok/s
step 2463/19560 | loss 3.754056 (-1.22z)| norm 0.2947 (-0.19z)| lr 5.87e-04 | 323.16 ms | 52.2% bf16 MFU | 1624000 tok/s
step 2464/19560 | loss 3.745152 (-1.44z)| norm 0.2828 (-0.48z)| lr 5.87e-04 | 322.61 ms | 52.3% bf16 MFU | 1624058 tok/s
step 2465/19560 | loss 3.758494 (-1.08z)| norm 0.2614 (-0.99z)| lr 5.87e-04 | 322.36 ms | 52.4% bf16 MFU | 1624175 tok/s
step 2466/19560 | loss 3.754497 (-1.18z)| norm 0.2682 (-0.82z)| lr 5.87e-04 | 323.27 ms | 52.2% bf16 MFU | 1624059 tok/s
step 2467/19560 | loss 3.766269 (-0.85z)| norm 0.2851 (-0.40z)| lr 5.87e-04 | 323.02 ms | 52.2% bf16 MFU | 1624010 tok/s
step 2468/19560 | loss 3.829383 (+0.86z)| norm 0.3111 (+0.22z)| lr 5.87e-04 | 322.78 ms | 52.3% bf16 MFU | 1624023 tok/s
step 2469/19560 | loss 3.764190 (-0.90z)| norm 0.2768 (-0.61z)| lr 5.87e-04 | 322.66 ms | 52.3% bf16 MFU | 1624065 tok/s
step 2470/19560 | loss 3.774196 (-0.62z)| norm 0.2573 (-1.07z)| lr 5.87e-04 | 322.72 ms | 52.3% bf16 MFU | 1624091 tok/s
step 2471/19560 | loss 3.791071 (-0.15z)| norm 0.3126 (+0.26z)| lr 5.87e-04 | 322.43 ms | 52.3% bf16 MFU | 1624189 tok/s
step 2472/19560 | loss 3.786976 (-0.26z)| norm 0.2780 (-0.57z)| lr 5.87e-04 | 323.10 ms | 52.2% bf16 MFU | 1624113 tok/s
step 2473/19560 | loss 3.762369 (-0.92z)| norm 0.2931 (-0.20z)| lr 5.87e-04 | 322.69 ms | 52.3% bf16 MFU | 1624145 tok/s
step 2474/19560 | loss 3.805726 (+0.25z)| norm 0.3322 (+0.75z)| lr 5.87e-04 | 322.72 ms | 52.3% bf16 MFU | 1624167 tok/s
step 2475/19560 | loss 3.750921 (-1.22z)| norm 0.3430 (+1.01z)| lr 5.87e-04 | 322.67 ms | 52.3% bf16 MFU | 1624200 tok/s
step 2476/19560 | loss 3.787066 (-0.25z)| norm 0.3554 (+1.29z)| lr 5.87e-04 | 322.62 ms | 52.3% bf16 MFU | 1624245 tok/s
step 2477/19560 | loss 3.816283 (+0.54z)| norm 0.3069 (+0.12z)| lr 5.87e-04 | 322.98 ms | 52.3% bf16 MFU | 1624196 tok/s
step 2478/19560 | loss 3.862461 (+1.78z)| norm 0.3078 (+0.14z)| lr 5.87e-04 | 322.94 ms | 52.3% bf16 MFU | 1624159 tok/s
step 2479/19560 | loss 3.752958 (-1.18z)| norm 0.3202 (+0.43z)| lr 5.87e-04 | 322.82 ms | 52.3% bf16 MFU | 1624155 tok/s
step 2480/19560 | loss 3.908223 (+2.88z)| norm 0.3426 (+0.95z)| lr 5.87e-04 | 322.50 ms | 52.3% bf16 MFU | 1624232 tok/s
step 2481/19560 | loss 3.769568 (-0.73z)| norm 0.2594 (-1.03z)| lr 5.87e-04 | 322.44 ms | 52.3% bf16 MFU | 1624320 tok/s
step 2482/19560 | loss 3.828462 (+0.81z)| norm 0.3101 (+0.28z)| lr 5.87e-04 | 323.13 ms | 52.2% bf16 MFU | 1624230 tok/s
step 2483/19560 | loss 3.857288 (+1.53z)| norm 0.3293 (+0.89z)| lr 5.87e-04 | 323.02 ms | 52.2% bf16 MFU | 1624173 tok/s
step 2484/19560 | loss 3.874381 (+1.93z)| norm 0.3151 (+0.56z)| lr 5.87e-04 | 323.00 ms | 52.3% bf16 MFU | 1624123 tok/s
step 2485/19560 | loss 3.779583 (-0.48z)| norm 0.2735 (-0.85z)| lr 5.87e-04 | 322.60 ms | 52.3% bf16 MFU | 1624177 tok/s
step 2486/19560 | loss 3.779656 (-0.47z)| norm 0.2786 (-0.66z)| lr 5.87e-04 | 322.39 ms | 52.3% bf16 MFU | 1624280 tok/s
step 2487/19560 | loss 3.789139 (-0.23z)| norm 0.2780 (-0.68z)| lr 5.87e-04 | 323.12 ms | 52.2% bf16 MFU | 1624195 tok/s
step 2488/19560 | loss 3.850091 (+1.32z)| norm 0.2988 (+0.05z)| lr 5.87e-04 | 323.49 ms | 52.2% bf16 MFU | 1624021 tok/s
step 2489/19560 | loss 3.758869 (-1.00z)| norm 0.2623 (-1.21z)| lr 5.87e-04 | 322.56 ms | 52.3% bf16 MFU | 1624091 tok/s
step 2490/19560 | loss 3.851647 (+1.35z)| norm 0.2770 (-0.70z)| lr 5.87e-04 | 322.83 ms | 52.3% bf16 MFU | 1624088 tok/s
step 2491/19560 | loss 3.816258 (+0.45z)| norm 0.2856 (-0.41z)| lr 5.87e-04 | 322.94 ms | 52.3% bf16 MFU | 1624057 tok/s
step 2492/19560 | loss 3.778680 (-0.49z)| norm 0.2740 (-0.82z)| lr 5.87e-04 | 322.45 ms | 52.3% bf16 MFU | 1624152 tok/s
step 2493/19560 | loss 3.769901 (-0.71z)| norm 0.2901 (-0.26z)| lr 5.87e-04 | 323.12 ms | 52.2% bf16 MFU | 1624075 tok/s
step 2494/19560 | loss 3.799367 (+0.04z)| norm 0.2715 (-0.91z)| lr 5.87e-04 | 323.09 ms | 52.2% bf16 MFU | 1624007 tok/s
step 2495/19560 | loss 3.751661 (-1.15z)| norm 0.2786 (-0.65z)| lr 5.87e-04 | 322.74 ms | 52.3% bf16 MFU | 1624031 tok/s
step 2496/19560 | loss 3.812238 (+0.40z)| norm 0.3121 (+0.56z)| lr 5.87e-04 | 322.66 ms | 52.3% bf16 MFU | 1624074 tok/s
step 2497/19560 | loss 3.781444 (-0.38z)| norm 0.3179 (+0.76z)| lr 5.87e-04 | 322.49 ms | 52.3% bf16 MFU | 1624158 tok/s
step 2498/19560 | loss 3.815129 (+0.51z)| norm 0.3127 (+0.58z)| lr 5.87e-04 | 322.90 ms | 52.3% bf16 MFU | 1624135 tok/s
step 2499/19560 | loss 3.825183 (+0.78z)| norm 0.3556 (+2.08z)| lr 5.87e-04 | 322.41 ms | 52.3% bf16 MFU | 1624236 tok/s
step 2500/19560 | loss 3.776311 (-0.49z)| norm 0.3219 (+0.87z)| lr 5.87e-04 | 322.70 ms | 52.3% bf16 MFU | 1624260 tok/s
val loss 3.772850
 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Helevaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2674/10042 = 0.266282
step 2501/19560 | loss 3.722334 (-1.88z)| norm 0.2911 (-0.21z)| lr 5.87e-04 | 322.79 ms | 52.3% bf16 MFU | 1624260 tok/s
step 2502/19560 | loss 3.795560 (+0.03z)| norm 0.2827 (-0.50z)| lr 5.87e-04 | 323.15 ms | 52.2% bf16 MFU | 1624168 tok/s
step 2503/19560 | loss 3.767299 (-0.70z)| norm 0.2758 (-0.73z)| lr 5.87e-04 | 323.17 ms | 52.2% bf16 MFU | 1624077 tok/s
step 2504/19560 | loss 3.769343 (-0.63z)| norm 0.2702 (-0.94z)| lr 5.87e-04 | 323.45 ms | 52.2% bf16 MFU | 1623919 tok/s
step 2505/19560 | loss 3.815243 (+0.55z)| norm 0.2762 (-0.74z)| lr 5.87e-04 | 322.71 ms | 52.3% bf16 MFU | 1623954 tok/s
step 2506/19560 | loss 3.802663 (+0.23z)| norm 0.2666 (-1.10z)| lr 5.87e-04 | 322.96 ms | 52.3% bf16 MFU | 1623926 tok/s
step 2507/19560 | loss 3.794680 (+0.01z)| norm 0.2615 (-1.28z)| lr 5.87e-04 | 323.17 ms | 52.2% bf16 MFU | 1623846 tok/s
step 2508/19560 | loss 3.724365 (-1.79z)| norm 0.2712 (-0.93z)| lr 5.87e-04 | 323.13 ms | 52.2% bf16 MFU | 1623780 tok/s
step 2509/19560 | loss 3.802876 (+0.25z)| norm 0.2658 (-1.12z)| lr 5.86e-04 | 323.05 ms | 52.2% bf16 MFU | 1623737 tok/s
step 2510/19560 | loss 3.786878 (-0.16z)| norm 0.3197 (+0.82z)| lr 5.86e-04 | 322.66 ms | 52.3% bf16 MFU | 1623795 tok/s
step 2511/19560 | loss 3.743241 (-1.28z)| norm 0.2840 (-0.48z)| lr 5.86e-04 | 323.68 ms | 52.1% bf16 MFU | 1623593 tok/s
step 2512/19560 | loss 3.775994 (-0.42z)| norm 0.3025 (+0.19z)| lr 5.86e-04 | 322.66 ms | 52.3% bf16 MFU | 1623658 tok/s
step 2513/19560 | loss 3.739677 (-1.34z)| norm 0.2648 (-1.17z)| lr 5.86e-04 | 322.62 ms | 52.3% bf16 MFU | 1623729 tok/s
step 2514/19560 | loss 3.749341 (-1.08z)| norm 0.2524 (-1.60z)| lr 5.86e-04 | 323.13 ms | 52.2% bf16 MFU | 1623670 tok/s
step 2515/19560 | loss 3.768361 (-0.59z)| norm 0.2587 (-1.36z)| lr 5.86e-04 | 322.97 ms | 52.3% bf16 MFU | 1623654 tok/s
step 2516/19560 | loss 3.800784 (+0.25z)| norm 0.2819 (-0.51z)| lr 5.86e-04 | 323.30 ms | 52.2% bf16 MFU | 1623554 tok/s
step 2517/19560 | loss 3.740984 (-1.28z)| norm 0.2843 (-0.42z)| lr 5.86e-04 | 323.42 ms | 52.2% bf16 MFU | 1623430 tok/s
step 2518/19560 | loss 3.728158 (-1.59z)| norm 0.2667 (-1.04z)| lr 5.86e-04 | 323.01 ms | 52.2% bf16 MFU | 1623414 tok/s
step 2519/19560 | loss 3.761664 (-0.73z)| norm 0.2780 (-0.63z)| lr 5.86e-04 | 322.99 ms | 52.3% bf16 MFU | 1623406 tok/s
step 2520/19560 | loss 3.794909 (+0.12z)| norm 0.3408 (+1.60z)| lr 5.86e-04 | 323.52 ms | 52.2% bf16 MFU | 1623264 tok/s
step 2521/19560 | loss 3.767959 (-0.59z)| norm 0.3395 (+1.57z)| lr 5.86e-04 | 323.15 ms | 52.2% bf16 MFU | 1623221 tok/s
step 2522/19560 | loss 3.774223 (-0.42z)| norm 0.3187 (+0.85z)| lr 5.86e-04 | 323.65 ms | 52.1% bf16 MFU | 1623056 tok/s
step 2523/19560 | loss 3.752446 (-1.00z)| norm 0.2893 (-0.21z)| lr 5.86e-04 | 323.26 ms | 52.2% bf16 MFU | 1622997 tok/s
step 2524/19560 | loss 3.843059 (+1.40z)| norm 0.3304 (+1.31z)| lr 5.86e-04 | 322.81 ms | 52.3% bf16 MFU | 1623054 tok/s
step 2525/19560 | loss 3.894445 (+2.68z)| norm 0.3435 (+1.76z)| lr 5.86e-04 | 322.93 ms | 52.3% bf16 MFU | 1623077 tok/s
step 2526/19560 | loss 3.827628 (+0.94z)| norm 0.3492 (+1.93z)| lr 5.86e-04 | 322.91 ms | 52.3% bf16 MFU | 1623106 tok/s
step 2527/19560 | loss 3.791441 (+0.01z)| norm 0.3024 (+0.24z)| lr 5.86e-04 | 322.99 ms | 52.3% bf16 MFU | 1623112 tok/s
step 2528/19560 | loss 3.743499 (-1.21z)| norm 0.2571 (-1.37z)| lr 5.86e-04 | 323.16 ms | 52.2% bf16 MFU | 1623074 tok/s
step 2529/19560 | loss 3.826960 (+0.93z)| norm 0.2831 (-0.43z)| lr 5.86e-04 | 323.07 ms | 52.2% bf16 MFU | 1623062 tok/s
step 2530/19560 | loss 3.759873 (-0.78z)| norm 0.2599 (-1.25z)| lr 5.86e-04 | 322.71 ms | 52.3% bf16 MFU | 1623140 tok/s
step 2531/19560 | loss 3.737898 (-1.32z)| norm 0.2398 (-1.93z)| lr 5.86e-04 | 322.51 ms | 52.3% bf16 MFU | 1623266 tok/s
step 2532/19560 | loss 3.830634 (+1.02z)| norm 0.2851 (-0.34z)| lr 5.86e-04 | 323.00 ms | 52.3% bf16 MFU | 1623262 tok/s
step 2533/19560 | loss 3.770744 (-0.50z)| norm 0.2748 (-0.70z)| lr 5.86e-04 | 322.84 ms | 52.3% bf16 MFU | 1623298 tok/s
step 2534/19560 | loss 3.766336 (-0.61z)| norm 0.2988 (+0.14z)| lr 5.86e-04 | 323.06 ms | 52.2% bf16 MFU | 1623278 tok/s
step 2535/19560 | loss 3.823014 (+0.84z)| norm 0.3177 (+0.79z)| lr 5.86e-04 | 322.60 ms | 52.3% bf16 MFU | 1623375 tok/s
step 2536/19560 | loss 3.756813 (-0.86z)| norm 0.3016 (+0.22z)| lr 5.86e-04 | 322.81 ms | 52.3% bf16 MFU | 1623413 tok/s
step 2537/19560 | loss 3.774621 (-0.40z)| norm 0.2418 (-1.84z)| lr 5.86e-04 | 322.78 ms | 52.3% bf16 MFU | 1623457 tok/s
step 2538/19560 | loss 3.827524 (+0.95z)| norm 0.3084 (+0.46z)| lr 5.86e-04 | 323.05 ms | 52.2% bf16 MFU | 1623430 tok/s
step 2539/19560 | loss 3.769520 (-0.53z)| norm 0.3935 (+3.25z)| lr 5.86e-04 | 323.38 ms | 52.2% bf16 MFU | 1623323 tok/s
step 2540/19560 | loss 3.771699 (-0.48z)| norm 0.4029 (+3.38z)| lr 5.86e-04 | 322.77 ms | 52.3% bf16 MFU | 1623375 tok/s
step 2541/19560 | loss 3.886908 (+2.41z)| norm 0.4037 (+3.24z)| lr 5.86e-04 | 323.42 ms | 52.2% bf16 MFU | 1623260 tok/s
step 2542/19560 | loss 3.762893 (-0.73z)| norm 0.3164 (+0.56z)| lr 5.86e-04 | 323.44 ms | 52.2% bf16 MFU | 1623145 tok/s
step 2543/19560 | loss 3.759208 (-0.82z)| norm 0.2976 (-0.02z)| lr 5.86e-04 | 322.67 ms | 52.3% bf16 MFU | 1623228 tok/s
step 2544/19560 | loss 3.858714 (+1.68z)| norm 0.2980 (+0.00z)| lr 5.86e-04 | 322.67 ms | 52.3% bf16 MFU | 1623310 tok/s
step 2545/19560 | loss 3.749262 (-1.06z)| norm 0.2696 (-0.86z)| lr 5.86e-04 | 322.97 ms | 52.3% bf16 MFU | 1623310 tok/s
step 2546/19560 | loss 3.896489 (+2.54z)| norm 0.2777 (-0.60z)| lr 5.86e-04 | 322.96 ms | 52.3% bf16 MFU | 1623315 tok/s
step 2547/19560 | loss 3.686396 (-2.51z)| norm 0.2882 (-0.28z)| lr 5.86e-04 | 322.94 ms | 52.3% bf16 MFU | 1623323 tok/s
step 2548/19560 | loss 3.794493 (+0.06z)| norm 0.2913 (-0.18z)| lr 5.86e-04 | 322.54 ms | 52.3% bf16 MFU | 1623432 tok/s
step 2549/19560 | loss 3.784196 (-0.19z)| norm 0.2705 (-0.82z)| lr 5.86e-04 | 322.76 ms | 52.3% bf16 MFU | 1623480 tok/s
step 2550/19560 | loss 3.830750 (+0.91z)| norm 0.2860 (-0.35z)| lr 5.86e-04 | 322.67 ms | 52.3% bf16 MFU | 1623547 tok/s
step 2551/19560 | loss 3.865186 (+1.71z)| norm 0.2655 (-0.97z)| lr 5.86e-04 | 323.24 ms | 52.2% bf16 MFU | 1623468 tok/s
step 2552/19560 | loss 3.774605 (-0.45z)| norm 0.2634 (-1.04z)| lr 5.86e-04 | 323.05 ms | 52.2% bf16 MFU | 1623442 tok/s
step 2553/19560 | loss 3.774303 (-0.45z)| norm 0.2746 (-0.68z)| lr 5.86e-04 | 323.27 ms | 52.2% bf16 MFU | 1623361 tok/s
step 2554/19560 | loss 3.786272 (-0.16z)| norm 0.2620 (-1.05z)| lr 5.86e-04 | 322.62 ms | 52.3% bf16 MFU | 1623447 tok/s
step 2555/19560 | loss 3.826340 (+0.78z)| norm 0.2778 (-0.56z)| lr 5.86e-04 | 322.39 ms | 52.3% bf16 MFU | 1623586 tok/s
step 2556/19560 | loss 3.744342 (-1.16z)| norm 0.2715 (-0.75z)| lr 5.86e-04 | 323.66 ms | 52.1% bf16 MFU | 1623399 tok/s
step 2557/19560 | loss 3.691392 (-2.41z)| norm 0.2952 (+0.02z)| lr 5.86e-04 | 322.83 ms | 52.3% bf16 MFU | 1623432 tok/s
step 2558/19560 | loss 3.779439 (-0.31z)| norm 0.2815 (-0.41z)| lr 5.86e-04 | 323.01 ms | 52.2% bf16 MFU | 1623417 tok/s
step 2559/19560 | loss 3.708851 (-1.97z)| norm 0.2813 (-0.40z)| lr 5.86e-04 | 322.92 ms | 52.3% bf16 MFU | 1623425 tok/s
step 2560/19560 | loss 3.773670 (-0.41z)| norm 0.2633 (-0.99z)| lr 5.86e-04 | 323.08 ms | 52.2% bf16 MFU | 1623392 tok/s
step 2561/19560 | loss 3.755945 (-0.83z)| norm 0.2724 (-0.69z)| lr 5.86e-04 | 323.10 ms | 52.2% bf16 MFU | 1623356 tok/s
step 2562/19560 | loss 3.709023 (-1.91z)| norm 0.2666 (-0.88z)| lr 5.86e-04 | 323.49 ms | 52.2% bf16 MFU | 1623226 tok/s
step 2563/19560 | loss 3.808296 (+0.44z)| norm 0.2635 (-0.96z)| lr 5.86e-04 | 322.98 ms | 52.3% bf16 MFU | 1623230 tok/s
step 2564/19560 | loss 3.723370 (-1.56z)| norm 0.2674 (-0.82z)| lr 5.86e-04 | 323.03 ms | 52.2% bf16 MFU | 1623221 tok/s
step 2565/19560 | loss 3.780755 (-0.18z)| norm 0.2965 (+0.13z)| lr 5.86e-04 | 322.77 ms | 52.3% bf16 MFU | 1623276 tok/s
step 2566/19560 | loss 3.777951 (-0.25z)| norm 0.2943 (+0.07z)| lr 5.86e-04 | 322.56 ms | 52.3% bf16 MFU | 1623382 tok/s
step 2567/19560 | loss 3.770980 (-0.40z)| norm 0.3471 (+1.79z)| lr 5.86e-04 | 322.93 ms | 52.3% bf16 MFU | 1623390 tok/s
step 2568/19560 | loss 3.761218 (-0.63z)| norm 0.2960 (+0.12z)| lr 5.86e-04 | 323.27 ms | 52.2% bf16 MFU | 1623310 tok/s
step 2569/19560 | loss 3.724352 (-1.52z)| norm 0.2638 (-0.92z)| lr 5.86e-04 | 322.51 ms | 52.3% bf16 MFU | 1623429 tok/s
step 2570/19560 | loss 3.772637 (-0.35z)| norm 0.2868 (-0.17z)| lr 5.86e-04 | 322.94 ms | 52.3% bf16 MFU | 1623432 tok/s
step 2571/19560 | loss 3.824644 (+0.92z)| norm 0.3117 (+0.63z)| lr 5.86e-04 | 323.34 ms | 52.2% bf16 MFU | 1623335 tok/s
step 2572/19560 | loss 3.718022 (-1.65z)| norm 0.3293 (+1.19z)| lr 5.86e-04 | 322.78 ms | 52.3% bf16 MFU | 1623381 tok/s
step 2573/19560 | loss 3.828629 (+1.01z)| norm 0.2860 (-0.22z)| lr 5.86e-04 | 322.77 ms | 52.3% bf16 MFU | 1623429 tok/s
step 2574/19560 | loss 3.752099 (-0.81z)| norm 0.3100 (+0.55z)| lr 5.86e-04 | 323.18 ms | 52.2% bf16 MFU | 1623371 tok/s
step 2575/19560 | loss 3.842842 (+1.36z)| norm 0.5265 (+6.27z)| lr 5.86e-04 | 322.64 ms | 52.3% bf16 MFU | 1623452 tok/s
step 2576/19560 | loss 3.771036 (-0.35z)| norm 0.3815 (+2.28z)| lr 5.85e-04 | 323.14 ms | 52.2% bf16 MFU | 1623404 tok/s
step 2577/19560 | loss 3.797517 (+0.28z)| norm 0.3438 (+1.27z)| lr 5.85e-04 | 323.32 ms | 52.2% bf16 MFU | 1623313 tok/s
step 2578/19560 | loss 3.859514 (+1.73z)| norm 0.3311 (+0.93z)| lr 5.85e-04 | 322.81 ms | 52.3% bf16 MFU | 1623355 tok/s
step 2579/19560 | loss 3.775180 (-0.27z)| norm 0.3699 (+1.94z)| lr 5.85e-04 | 322.35 ms | 52.4% bf16 MFU | 1623510 tok/s
step 2580/19560 | loss 3.853699 (+1.56z)| norm 0.3580 (+1.60z)| lr 5.85e-04 | 323.17 ms | 52.2% bf16 MFU | 1623451 tok/s
step 2581/19560 | loss 3.851680 (+1.51z)| norm 0.3552 (+1.50z)| lr 5.85e-04 | 322.70 ms | 52.3% bf16 MFU | 1623512 tok/s
step 2582/19560 | loss 3.778661 (-0.20z)| norm 0.3428 (+1.16z)| lr 5.85e-04 | 323.31 ms | 52.2% bf16 MFU | 1623418 tok/s
step 2583/19560 | loss 3.757520 (-0.70z)| norm 0.3567 (+1.49z)| lr 5.85e-04 | 322.76 ms | 52.3% bf16 MFU | 1623466 tok/s
step 2584/19560 | loss 3.735546 (-1.20z)| norm 0.3457 (+1.19z)| lr 5.85e-04 | 322.45 ms | 52.3% bf16 MFU | 1623589 tok/s
step 2585/19560 | loss 3.788047 (+0.03z)| norm 0.2738 (-0.63z)| lr 5.85e-04 | 323.23 ms | 52.2% bf16 MFU | 1623511 tok/s
step 2586/19560 | loss 3.766297 (-0.47z)| norm 0.2658 (-0.83z)| lr 5.85e-04 | 322.51 ms | 52.3% bf16 MFU | 1623617 tok/s
step 2587/19560 | loss 3.718997 (-1.55z)| norm 0.2909 (-0.19z)| lr 5.85e-04 | 322.94 ms | 52.3% bf16 MFU | 1623612 tok/s
step 2588/19560 | loss 3.840653 (+1.27z)| norm 0.2595 (-0.99z)| lr 5.85e-04 | 322.59 ms | 52.3% bf16 MFU | 1623692 tok/s
step 2589/19560 | loss 3.744183 (-0.95z)| norm 0.2687 (-0.76z)| lr 5.85e-04 | 322.97 ms | 52.3% bf16 MFU | 1623676 tok/s
step 2590/19560 | loss 3.798287 (+0.29z)| norm 0.2639 (-0.88z)| lr 5.85e-04 | 322.75 ms | 52.3% bf16 MFU | 1623714 tok/s
step 2591/19560 | loss 3.765145 (-0.47z)| norm 0.2696 (-0.73z)| lr 5.85e-04 | 322.64 ms | 52.3% bf16 MFU | 1623777 tok/s
step 2592/19560 | loss 3.742227 (-1.00z)| norm 0.2750 (-0.59z)| lr 5.85e-04 | 322.87 ms | 52.3% bf16 MFU | 1623778 tok/s
step 2593/19560 | loss 3.779512 (-0.15z)| norm 0.2660 (-0.82z)| lr 5.85e-04 | 322.47 ms | 52.3% bf16 MFU | 1623882 tok/s
step 2594/19560 | loss 3.801584 (+0.36z)| norm 0.2956 (-0.08z)| lr 5.85e-04 | 323.13 ms | 52.2% bf16 MFU | 1623816 tok/s
step 2595/19560 | loss 3.793080 (+0.16z)| norm 0.2754 (-0.59z)| lr 5.85e-04 | 323.10 ms | 52.2% bf16 MFU | 1623760 tok/s
step 2596/19560 | loss 3.753289 (-0.75z)| norm 0.2539 (-1.12z)| lr 5.85e-04 | 322.71 ms | 52.3% bf16 MFU | 1623804 tok/s
step 2597/19560 | loss 3.796544 (+0.24z)| norm 0.2878 (-0.26z)| lr 5.85e-04 | 323.19 ms | 52.2% bf16 MFU | 1623725 tok/s
step 2598/19560 | loss 3.786915 (+0.02z)| norm 0.2469 (-1.29z)| lr 5.85e-04 | 323.05 ms | 52.2% bf16 MFU | 1623685 tok/s
step 2599/19560 | loss 3.854761 (+1.57z)| norm 0.2550 (-1.07z)| lr 5.85e-04 | 322.81 ms | 52.3% bf16 MFU | 1623707 tok/s
step 2600/19560 | loss 3.776588 (-0.23z)| norm 0.2955 (-0.06z)| lr 5.85e-04 | 322.16 ms | 52.4% bf16 MFU | 1623893 tok/s
step 2601/19560 | loss 3.734419 (-1.19z)| norm 0.2892 (-0.21z)| lr 5.85e-04 | 322.81 ms | 52.3% bf16 MFU | 1623905 tok/s
step 2602/19560 | loss 3.778741 (-0.17z)| norm 0.2831 (-0.36z)| lr 5.85e-04 | 322.55 ms | 52.3% bf16 MFU | 1623982 tok/s
step 2603/19560 | loss 3.771581 (-0.34z)| norm 0.2774 (-0.49z)| lr 5.85e-04 | 323.22 ms | 52.2% bf16 MFU | 1623888 tok/s
step 2604/19560 | loss 3.757559 (-0.65z)| norm 0.2688 (-0.70z)| lr 5.85e-04 | 322.59 ms | 52.3% bf16 MFU | 1623955 tok/s
step 2605/19560 | loss 3.796062 (+0.23z)| norm 0.2731 (-0.58z)| lr 5.85e-04 | 322.66 ms | 52.3% bf16 MFU | 1624002 tok/s
step 2606/19560 | loss 3.700093 (-1.94z)| norm 0.2431 (-1.32z)| lr 5.85e-04 | 322.29 ms | 52.4% bf16 MFU | 1624140 tok/s
step 2607/19560 | loss 3.797020 (+0.28z)| norm 0.2480 (-1.18z)| lr 5.85e-04 | 322.66 ms | 52.3% bf16 MFU | 1624177 tok/s
step 2608/19560 | loss 3.764773 (-0.45z)| norm 0.2611 (-0.84z)| lr 5.85e-04 | 322.47 ms | 52.3% bf16 MFU | 1624261 tok/s
step 2609/19560 | loss 3.765101 (-0.44z)| norm 0.2845 (-0.25z)| lr 5.85e-04 | 322.75 ms | 52.3% bf16 MFU | 1624271 tok/s
step 2610/19560 | loss 3.773821 (-0.23z)| norm 0.3102 (+0.40z)| lr 5.85e-04 | 322.64 ms | 52.3% bf16 MFU | 1624307 tok/s
step 2611/19560 | loss 3.754994 (-0.66z)| norm 0.3350 (+1.03z)| lr 5.85e-04 | 322.91 ms | 52.3% bf16 MFU | 1624273 tok/s
step 2612/19560 | loss 3.693582 (-2.11z)| norm 0.3281 (+0.85z)| lr 5.85e-04 | 322.94 ms | 52.3% bf16 MFU | 1624234 tok/s
step 2613/19560 | loss 3.705609 (-1.78z)| norm 0.3085 (+0.34z)| lr 5.85e-04 | 322.86 ms | 52.3% bf16 MFU | 1624216 tok/s
step 2614/19560 | loss 3.755591 (-0.59z)| norm 0.2855 (-0.24z)| lr 5.85e-04 | 323.40 ms | 52.2% bf16 MFU | 1624063 tok/s
step 2615/19560 | loss 3.773777 (-0.15z)| norm 0.2858 (-0.23z)| lr 5.85e-04 | 322.75 ms | 52.3% bf16 MFU | 1624081 tok/s
step 2616/19560 | loss 3.769355 (-0.25z)| norm 0.2870 (-0.20z)| lr 5.85e-04 | 322.46 ms | 52.3% bf16 MFU | 1624171 tok/s
step 2617/19560 | loss 3.776553 (-0.08z)| norm 0.2764 (-0.47z)| lr 5.85e-04 | 322.91 ms | 52.3% bf16 MFU | 1624144 tok/s
step 2618/19560 | loss 3.791133 (+0.29z)| norm 0.2820 (-0.33z)| lr 5.85e-04 | 322.27 ms | 52.4% bf16 MFU | 1624281 tok/s
step 2619/19560 | loss 3.816413 (+0.90z)| norm 0.2948 (-0.01z)| lr 5.85e-04 | 323.30 ms | 52.2% bf16 MFU | 1624151 tok/s
step 2620/19560 | loss 3.745913 (-0.80z)| norm 0.3202 (+0.63z)| lr 5.85e-04 | 322.90 ms | 52.3% bf16 MFU | 1624128 tok/s
step 2621/19560 | loss 3.774220 (-0.12z)| norm 0.3026 (+0.18z)| lr 5.85e-04 | 322.63 ms | 52.3% bf16 MFU | 1624173 tok/s
step 2622/19560 | loss 3.727488 (-1.23z)| norm 0.2621 (-0.85z)| lr 5.85e-04 | 322.56 ms | 52.3% bf16 MFU | 1624234 tok/s
step 2623/19560 | loss 3.808542 (+0.71z)| norm 0.2942 (-0.03z)| lr 5.85e-04 | 323.03 ms | 52.2% bf16 MFU | 1624174 tok/s
step 2624/19560 | loss 3.792736 (+0.34z)| norm 0.2948 (-0.02z)| lr 5.85e-04 | 322.13 ms | 52.4% bf16 MFU | 1624344 tok/s
step 2625/19560 | loss 3.768869 (-0.24z)| norm 0.3070 (+0.29z)| lr 5.85e-04 | 322.66 ms | 52.3% bf16 MFU | 1624372 tok/s
step 2626/19560 | loss 3.724691 (-1.28z)| norm 0.2756 (-0.50z)| lr 5.85e-04 | 322.71 ms | 52.3% bf16 MFU | 1624386 tok/s
step 2627/19560 | loss 3.817633 (+0.96z)| norm 0.2917 (-0.07z)| lr 5.85e-04 | 322.41 ms | 52.3% bf16 MFU | 1624474 tok/s
step 2628/19560 | loss 3.806212 (+0.68z)| norm 0.2950 (+0.02z)| lr 5.85e-04 | 322.56 ms | 52.3% bf16 MFU | 1624520 tok/s
step 2629/19560 | loss 3.802204 (+0.57z)| norm 0.2923 (-0.05z)| lr 5.85e-04 | 323.56 ms | 52.2% bf16 MFU | 1624312 tok/s
step 2630/19560 | loss 3.821895 (+1.04z)| norm 0.2882 (-0.16z)| lr 5.85e-04 | 322.85 ms | 52.3% bf16 MFU | 1624293 tok/s
step 2631/19560 | loss 3.796062 (+0.41z)| norm 0.2858 (-0.22z)| lr 5.85e-04 | 322.68 ms | 52.3% bf16 MFU | 1624318 tok/s
step 2632/19560 | loss 3.742098 (-0.89z)| norm 0.2800 (-0.37z)| lr 5.85e-04 | 322.90 ms | 52.3% bf16 MFU | 1624286 tok/s
step 2633/19560 | loss 3.907814 (+2.98z)| norm 0.2788 (-0.41z)| lr 5.85e-04 | 322.83 ms | 52.3% bf16 MFU | 1624275 tok/s
step 2634/19560 | loss 3.795686 (+0.37z)| norm 0.2671 (-0.71z)| lr 5.85e-04 | 322.93 ms | 52.3% bf16 MFU | 1624238 tok/s
step 2635/19560 | loss 3.753474 (-0.60z)| norm 0.2645 (-0.78z)| lr 5.85e-04 | 322.89 ms | 52.3% bf16 MFU | 1624214 tok/s
step 2636/19560 | loss 3.765162 (-0.34z)| norm 0.2791 (-0.40z)| lr 5.85e-04 | 322.52 ms | 52.3% bf16 MFU | 1624284 tok/s
step 2637/19560 | loss 3.719581 (-1.38z)| norm 0.2829 (-0.31z)| lr 5.85e-04 | 322.60 ms | 52.3% bf16 MFU | 1624329 tok/s
step 2638/19560 | loss 3.752830 (-0.60z)| norm 0.2555 (-1.00z)| lr 5.85e-04 | 322.95 ms | 52.3% bf16 MFU | 1624285 tok/s
step 2639/19560 | loss 3.785881 (+0.16z)| norm 0.2757 (-0.48z)| lr 5.85e-04 | 322.52 ms | 52.3% bf16 MFU | 1624350 tok/s
step 2640/19560 | loss 3.804918 (+0.60z)| norm 0.3036 (+0.24z)| lr 5.84e-04 | 323.01 ms | 52.2% bf16 MFU | 1624289 tok/s
step 2641/19560 | loss 3.799178 (+0.45z)| norm 0.3035 (+0.23z)| lr 5.84e-04 | 322.43 ms | 52.3% bf16 MFU | 1624377 tok/s
step 2642/19560 | loss 3.816880 (+0.85z)| norm 0.3268 (+0.82z)| lr 5.84e-04 | 322.67 ms | 52.3% bf16 MFU | 1624399 tok/s
step 2643/19560 | loss 3.814546 (+0.79z)| norm 0.2817 (-0.35z)| lr 5.84e-04 | 322.69 ms | 52.3% bf16 MFU | 1624417 tok/s
step 2644/19560 | loss 3.842889 (+1.43z)| norm 0.2701 (-0.65z)| lr 5.84e-04 | 323.40 ms | 52.2% bf16 MFU | 1624254 tok/s
step 2645/19560 | loss 3.768966 (-0.28z)| norm 0.2945 (-0.02z)| lr 5.84e-04 | 322.57 ms | 52.3% bf16 MFU | 1624310 tok/s
step 2646/19560 | loss 3.705042 (-1.75z)| norm 0.2978 (+0.06z)| lr 5.84e-04 | 322.27 ms | 52.4% bf16 MFU | 1624438 tok/s
step 2647/19560 | loss 3.708200 (-1.65z)| norm 0.2731 (-0.58z)| lr 5.84e-04 | 322.55 ms | 52.3% bf16 MFU | 1624489 tok/s
step 2648/19560 | loss 3.691811 (-1.97z)| norm 0.2654 (-0.77z)| lr 5.84e-04 | 322.66 ms | 52.3% bf16 MFU | 1624509 tok/s
step 2649/19560 | loss 3.719347 (-1.34z)| norm 0.2830 (-0.30z)| lr 5.84e-04 | 323.11 ms | 52.2% bf16 MFU | 1624416 tok/s
step 2650/19560 | loss 3.830149 (+1.11z)| norm 0.2566 (-0.98z)| lr 5.84e-04 | 322.54 ms | 52.3% bf16 MFU | 1624470 tok/s
step 2651/19560 | loss 3.740602 (-0.87z)| norm 0.2744 (-0.51z)| lr 5.84e-04 | 322.19 ms | 52.4% bf16 MFU | 1624610 tok/s
step 2652/19560 | loss 3.757222 (-0.49z)| norm 0.2688 (-0.64z)| lr 5.84e-04 | 323.05 ms | 52.2% bf16 MFU | 1624526 tok/s
step 2653/19560 | loss 3.749304 (-0.66z)| norm 0.2798 (-0.34z)| lr 5.84e-04 | 322.97 ms | 52.3% bf16 MFU | 1624466 tok/s
step 2654/19560 | loss 3.769788 (-0.18z)| norm 0.3237 (+0.82z)| lr 5.84e-04 | 323.16 ms | 52.2% bf16 MFU | 1624361 tok/s
step 2655/19560 | loss 3.744332 (-0.75z)| norm 0.3197 (+0.71z)| lr 5.84e-04 | 322.76 ms | 52.3% bf16 MFU | 1624362 tok/s
step 2656/19560 | loss 3.766641 (-0.25z)| norm 0.2971 (+0.10z)| lr 5.84e-04 | 322.55 ms | 52.3% bf16 MFU | 1624417 tok/s
step 2657/19560 | loss 3.810688 (+0.77z)| norm 0.2955 (+0.06z)| lr 5.84e-04 | 322.98 ms | 52.3% bf16 MFU | 1624360 tok/s
step 2658/19560 | loss 3.735004 (-0.97z)| norm 0.2918 (-0.04z)| lr 5.84e-04 | 324.82 ms | 52.0% bf16 MFU | 1623846 tok/s
step 2659/19560 | loss 3.736697 (-0.93z)| norm 0.2501 (-1.16z)| lr 5.84e-04 | 321.85 ms | 52.4% bf16 MFU | 1624103 tok/s
step 2660/19560 | loss 3.782734 (+0.14z)| norm 0.2835 (-0.27z)| lr 5.84e-04 | 322.79 ms | 52.3% bf16 MFU | 1624111 tok/s
step 2661/19560 | loss 3.758779 (-0.41z)| norm 0.3272 (+0.88z)| lr 5.84e-04 | 324.11 ms | 52.1% bf16 MFU | 1623788 tok/s
step 2662/19560 | loss 3.780130 (+0.08z)| norm 0.3283 (+0.90z)| lr 5.84e-04 | 323.06 ms | 52.2% bf16 MFU | 1623742 tok/s
step 2663/19560 | loss 3.822543 (+1.06z)| norm 0.2999 (+0.15z)| lr 5.84e-04 | 322.41 ms | 52.3% bf16 MFU | 1623862 tok/s
step 2664/19560 | loss 3.849291 (+1.64z)| norm 0.2927 (-0.03z)| lr 5.84e-04 | 322.32 ms | 52.4% bf16 MFU | 1623998 tok/s
step 2665/19560 | loss 3.763551 (-0.31z)| norm 0.2765 (-0.47z)| lr 5.84e-04 | 322.92 ms | 52.3% bf16 MFU | 1623978 tok/s
step 2666/19560 | loss 3.835428 (+1.33z)| norm 0.2681 (-0.69z)| lr 5.84e-04 | 322.25 ms | 52.4% bf16 MFU | 1624126 tok/s
step 2667/19560 | loss 3.733766 (-0.98z)| norm 0.3127 (+0.53z)| lr 5.84e-04 | 323.26 ms | 52.2% bf16 MFU | 1624013 tok/s
step 2668/19560 | loss 3.747480 (-0.67z)| norm 0.3009 (+0.24z)| lr 5.84e-04 | 323.20 ms | 52.2% bf16 MFU | 1623922 tok/s
step 2669/19560 | loss 3.738165 (-0.87z)| norm 0.3301 (+1.13z)| lr 5.84e-04 | 323.07 ms | 52.2% bf16 MFU | 1623867 tok/s
step 2670/19560 | loss 3.703669 (-1.64z)| norm 0.3345 (+1.25z)| lr 5.84e-04 | 322.28 ms | 52.4% bf16 MFU | 1624014 tok/s
step 2671/19560 | loss 3.847188 (+1.62z)| norm 0.3225 (+0.89z)| lr 5.84e-04 | 323.18 ms | 52.2% bf16 MFU | 1623928 tok/s
step 2672/19560 | loss 3.732490 (-0.98z)| norm 0.2996 (+0.22z)| lr 5.84e-04 | 323.17 ms | 52.2% bf16 MFU | 1623848 tok/s
step 2673/19560 | loss 3.766713 (-0.19z)| norm 0.3005 (+0.23z)| lr 5.84e-04 | 323.27 ms | 52.2% bf16 MFU | 1623746 tok/s
step 2674/19560 | loss 3.739775 (-0.81z)| norm 0.2781 (-0.42z)| lr 5.84e-04 | 322.96 ms | 52.3% bf16 MFU | 1623727 tok/s
step 2675/19560 | loss 3.773391 (-0.03z)| norm 0.2794 (-0.38z)| lr 5.84e-04 | 322.80 ms | 52.3% bf16 MFU | 1623749 tok/s
step 2676/19560 | loss 3.718816 (-1.32z)| norm 0.2675 (-0.72z)| lr 5.84e-04 | 322.78 ms | 52.3% bf16 MFU | 1623776 tok/s
step 2677/19560 | loss 3.795676 (+0.52z)| norm 0.2863 (-0.18z)| lr 5.84e-04 | 322.70 ms | 52.3% bf16 MFU | 1623821 tok/s
step 2678/19560 | loss 3.677610 (-2.25z)| norm 0.2649 (-0.80z)| lr 5.84e-04 | 322.79 ms | 52.3% bf16 MFU | 1623842 tok/s
step 2679/19560 | loss 3.824577 (+1.25z)| norm 0.3052 (+0.37z)| lr 5.84e-04 | 322.94 ms | 52.3% bf16 MFU | 1623824 tok/s
step 2680/19560 | loss 3.718174 (-1.28z)| norm 0.3086 (+0.46z)| lr 5.84e-04 | 322.93 ms | 52.3% bf16 MFU | 1623809 tok/s
step 2681/19560 | loss 3.757776 (-0.34z)| norm 0.3013 (+0.24z)| lr 5.84e-04 | 322.78 ms | 52.3% bf16 MFU | 1623832 tok/s
step 2682/19560 | loss 3.756133 (-0.37z)| norm 0.2983 (+0.15z)| lr 5.84e-04 | 322.80 ms | 52.3% bf16 MFU | 1623849 tok/s
step 2683/19560 | loss 3.802548 (+0.74z)| norm 0.2878 (-0.17z)| lr 5.84e-04 | 323.23 ms | 52.2% bf16 MFU | 1623758 tok/s
step 2684/19560 | loss 3.757248 (-0.34z)| norm 0.2534 (-1.17z)| lr 5.84e-04 | 322.98 ms | 52.3% bf16 MFU | 1623734 tok/s
step 2685/19560 | loss 3.764011 (-0.20z)| norm 0.2525 (-1.18z)| lr 5.84e-04 | 322.23 ms | 52.4% bf16 MFU | 1623899 tok/s
step 2686/19560 | loss 3.736913 (-0.85z)| norm 0.2624 (-0.89z)| lr 5.84e-04 | 323.35 ms | 52.2% bf16 MFU | 1623776 tok/s
step 2687/19560 | loss 3.788915 (+0.40z)| norm 0.2790 (-0.40z)| lr 5.84e-04 | 322.77 ms | 52.3% bf16 MFU | 1623803 tok/s
step 2688/19560 | loss 3.717040 (-1.33z)| norm 0.2799 (-0.38z)| lr 5.84e-04 | 323.15 ms | 52.2% bf16 MFU | 1623735 tok/s
step 2689/19560 | loss 3.723266 (-1.17z)| norm 0.2862 (-0.20z)| lr 5.84e-04 | 323.20 ms | 52.2% bf16 MFU | 1623657 tok/s
step 2690/19560 | loss 3.783320 (+0.27z)| norm 0.2976 (+0.12z)| lr 5.84e-04 | 322.51 ms | 52.3% bf16 MFU | 1623758 tok/s
step 2691/19560 | loss 3.727760 (-1.07z)| norm 0.2767 (-0.49z)| lr 5.84e-04 | 322.74 ms | 52.3% bf16 MFU | 1623794 tok/s
step 2692/19560 | loss 3.720562 (-1.24z)| norm 0.2791 (-0.42z)| lr 5.84e-04 | 323.69 ms | 52.1% bf16 MFU | 1623590 tok/s
step 2693/19560 | loss 3.746448 (-0.61z)| norm 0.2933 (-0.01z)| lr 5.84e-04 | 323.00 ms | 52.3% bf16 MFU | 1623569 tok/s
step 2694/19560 | loss 3.810347 (+0.93z)| norm 0.3174 (+0.70z)| lr 5.84e-04 | 322.69 ms | 52.3% bf16 MFU | 1623629 tok/s
step 2695/19560 | loss 3.769260 (-0.06z)| norm 0.3644 (+2.06z)| lr 5.84e-04 | 323.00 ms | 52.3% bf16 MFU | 1623607 tok/s
step 2696/19560 | loss 3.839933 (+1.62z)| norm 0.3639 (+2.00z)| lr 5.84e-04 | 322.81 ms | 52.3% bf16 MFU | 1623633 tok/s
step 2697/19560 | loss 3.750230 (-0.54z)| norm 0.3380 (+1.23z)| lr 5.84e-04 | 323.01 ms | 52.2% bf16 MFU | 1623607 tok/s
step 2698/19560 | loss 3.731635 (-0.97z)| norm 0.2983 (+0.09z)| lr 5.84e-04 | 322.68 ms | 52.3% bf16 MFU | 1623667 tok/s
step 2699/19560 | loss 3.762305 (-0.23z)| norm 0.3019 (+0.20z)| lr 5.84e-04 | 322.80 ms | 52.3% bf16 MFU | 1623694 tok/s
step 2700/19560 | loss 3.724539 (-1.14z)| norm 0.2820 (-0.36z)| lr 5.84e-04 | 323.16 ms | 52.2% bf16 MFU | 1623627 tok/s
step 2701/19560 | loss 3.823443 (+1.25z)| norm 0.2893 (-0.15z)| lr 5.84e-04 | 323.34 ms | 52.2% bf16 MFU | 1623520 tok/s
step 2702/19560 | loss 3.746871 (-0.60z)| norm 0.2825 (-0.34z)| lr 5.83e-04 | 323.23 ms | 52.2% bf16 MFU | 1623446 tok/s
step 2703/19560 | loss 3.794259 (+0.56z)| norm 0.3196 (+0.95z)| lr 5.83e-04 | 323.01 ms | 52.2% bf16 MFU | 1623430 tok/s
step 2704/19560 | loss 3.685556 (-2.04z)| norm 0.3290 (+1.34z)| lr 5.83e-04 | 322.97 ms | 52.3% bf16 MFU | 1623425 tok/s
step 2705/19560 | loss 3.747341 (-0.55z)| norm 0.3277 (+1.31z)| lr 5.83e-04 | 323.11 ms | 52.2% bf16 MFU | 1623385 tok/s
step 2706/19560 | loss 3.732234 (-0.90z)| norm 0.2974 (+0.20z)| lr 5.83e-04 | 322.59 ms | 52.3% bf16 MFU | 1623477 tok/s
step 2707/19560 | loss 3.868419 (+2.34z)| norm 0.3247 (+1.27z)| lr 5.83e-04 | 323.13 ms | 52.2% bf16 MFU | 1623430 tok/s
step 2708/19560 | loss 3.744354 (-0.60z)| norm 0.2904 (-0.03z)| lr 5.83e-04 | 323.01 ms | 52.2% bf16 MFU | 1623414 tok/s
step 2709/19560 | loss 3.739653 (-0.70z)| norm 0.2750 (-0.62z)| lr 5.83e-04 | 322.91 ms | 52.3% bf16 MFU | 1623425 tok/s
step 2710/19560 | loss 3.700207 (-1.64z)| norm 0.2680 (-0.89z)| lr 5.83e-04 | 322.60 ms | 52.3% bf16 MFU | 1623514 tok/s
step 2711/19560 | loss 3.690817 (-1.83z)| norm 0.2732 (-0.67z)| lr 5.83e-04 | 322.59 ms | 52.3% bf16 MFU | 1623599 tok/s
step 2712/19560 | loss 3.788317 (+0.50z)| norm 0.2518 (-1.56z)| lr 5.83e-04 | 323.22 ms | 52.2% bf16 MFU | 1623523 tok/s
step 2713/19560 | loss 3.777300 (+0.24z)| norm 0.2606 (-1.18z)| lr 5.83e-04 | 322.86 ms | 52.3% bf16 MFU | 1623541 tok/s
step 2714/19560 | loss 3.719654 (-1.13z)| norm 0.2964 (+0.33z)| lr 5.83e-04 | 322.95 ms | 52.3% bf16 MFU | 1623535 tok/s
step 2715/19560 | loss 3.748741 (-0.45z)| norm 0.2572 (-1.32z)| lr 5.83e-04 | 323.03 ms | 52.2% bf16 MFU | 1623511 tok/s
step 2716/19560 | loss 3.693574 (-1.74z)| norm 0.2572 (-1.31z)| lr 5.83e-04 | 323.39 ms | 52.2% bf16 MFU | 1623398 tok/s
step 2717/19560 | loss 3.760852 (-0.13z)| norm 0.2599 (-1.20z)| lr 5.83e-04 | 322.85 ms | 52.3% bf16 MFU | 1623424 tok/s
step 2718/19560 | loss 3.785863 (+0.47z)| norm 0.2824 (-0.26z)| lr 5.83e-04 | 323.40 ms | 52.2% bf16 MFU | 1623312 tok/s
step 2719/19560 | loss 3.759660 (-0.16z)| norm 0.2761 (-0.53z)| lr 5.83e-04 | 322.87 ms | 52.3% bf16 MFU | 1623339 tok/s
step 2720/19560 | loss 3.767508 (+0.03z)| norm 0.2876 (-0.04z)| lr 5.83e-04 | 322.99 ms | 52.3% bf16 MFU | 1623334 tok/s
step 2721/19560 | loss 3.811087 (+1.07z)| norm 0.2893 (+0.02z)| lr 5.83e-04 | 323.20 ms | 52.2% bf16 MFU | 1623277 tok/s
step 2722/19560 | loss 3.722528 (-1.04z)| norm 0.2689 (-0.83z)| lr 5.83e-04 | 323.23 ms | 52.2% bf16 MFU | 1623215 tok/s
step 2723/19560 | loss 3.795530 (+0.71z)| norm 0.2860 (-0.11z)| lr 5.83e-04 | 322.74 ms | 52.3% bf16 MFU | 1623279 tok/s
step 2724/19560 | loss 3.723794 (-1.00z)| norm 0.2954 (+0.28z)| lr 5.83e-04 | 323.20 ms | 52.2% bf16 MFU | 1623223 tok/s
step 2725/19560 | loss 3.733434 (-0.76z)| norm 0.3330 (+1.85z)| lr 5.83e-04 | 323.45 ms | 52.2% bf16 MFU | 1623109 tok/s
step 2726/19560 | loss 3.753031 (-0.29z)| norm 0.3204 (+1.30z)| lr 5.83e-04 | 322.78 ms | 52.3% bf16 MFU | 1623167 tok/s
step 2727/19560 | loss 3.751918 (-0.30z)| norm 0.2999 (+0.42z)| lr 5.83e-04 | 323.01 ms | 52.3% bf16 MFU | 1623166 tok/s
step 2728/19560 | loss 3.771125 (+0.17z)| norm 0.3253 (+1.48z)| lr 5.83e-04 | 323.29 ms | 52.2% bf16 MFU | 1623093 tok/s
step 2729/19560 | loss 3.743728 (-0.50z)| norm 0.3100 (+0.83z)| lr 5.83e-04 | 323.07 ms | 52.2% bf16 MFU | 1623080 tok/s
step 2730/19560 | loss 3.723992 (-0.97z)| norm 0.3189 (+1.18z)| lr 5.83e-04 | 322.67 ms | 52.3% bf16 MFU | 1623167 tok/s
step 2731/19560 | loss 3.727577 (-0.87z)| norm 0.3084 (+0.73z)| lr 5.83e-04 | 323.01 ms | 52.3% bf16 MFU | 1623166 tok/s
step 2732/19560 | loss 3.852580 (+2.10z)| norm 0.2952 (+0.16z)| lr 5.83e-04 | 322.58 ms | 52.3% bf16 MFU | 1623272 tok/s
step 2733/19560 | loss 3.779510 (+0.37z)| norm 0.2818 (-0.41z)| lr 5.83e-04 | 324.23 ms | 52.1% bf16 MFU | 1622960 tok/s
step 2734/19560 | loss 3.749238 (-0.37z)| norm 0.2785 (-0.57z)| lr 5.83e-04 | 323.58 ms | 52.2% bf16 MFU | 1622826 tok/s
step 2735/19560 | loss 3.768931 (+0.11z)| norm 0.2847 (-0.32z)| lr 5.83e-04 | 323.26 ms | 52.2% bf16 MFU | 1622777 tok/s
step 2736/19560 | loss 3.746121 (-0.43z)| norm 0.2511 (-1.77z)| lr 5.83e-04 | 323.35 ms | 52.2% bf16 MFU | 1622710 tok/s
step 2737/19560 | loss 3.796119 (+0.76z)| norm 0.2754 (-0.71z)| lr 5.83e-04 | 323.12 ms | 52.2% bf16 MFU | 1622704 tok/s
step 2738/19560 | loss 3.784251 (+0.47z)| norm 0.2677 (-1.03z)| lr 5.83e-04 | 322.75 ms | 52.3% bf16 MFU | 1622790 tok/s
step 2739/19560 | loss 3.764137 (-0.01z)| norm 0.2459 (-1.94z)| lr 5.83e-04 | 322.82 ms | 52.3% bf16 MFU | 1622855 tok/s
step 2740/19560 | loss 3.724480 (-0.97z)| norm 0.2550 (-1.53z)| lr 5.83e-04 | 322.94 ms | 52.3% bf16 MFU | 1622888 tok/s
step 2741/19560 | loss 3.689567 (-1.80z)| norm 0.2391 (-2.16z)| lr 5.83e-04 | 322.67 ms | 52.3% bf16 MFU | 1622985 tok/s
step 2742/19560 | loss 3.825511 (+1.44z)| norm 0.2819 (-0.33z)| lr 5.83e-04 | 322.53 ms | 52.3% bf16 MFU | 1623114 tok/s
step 2743/19560 | loss 3.701335 (-1.50z)| norm 0.3311 (+1.73z)| lr 5.83e-04 | 322.91 ms | 52.3% bf16 MFU | 1623141 tok/s
step 2744/19560 | loss 3.809417 (+1.05z)| norm 0.3355 (+1.87z)| lr 5.83e-04 | 322.56 ms | 52.3% bf16 MFU | 1623253 tok/s
step 2745/19560 | loss 3.727582 (-0.87z)| norm 0.3408 (+2.04z)| lr 5.83e-04 | 323.21 ms | 52.2% bf16 MFU | 1623197 tok/s
step 2746/19560 | loss 3.701106 (-1.46z)| norm 0.3072 (+0.66z)| lr 5.83e-04 | 322.86 ms | 52.3% bf16 MFU | 1623231 tok/s
step 2747/19560 | loss 3.739161 (-0.56z)| norm 0.3124 (+0.86z)| lr 5.83e-04 | 322.59 ms | 52.3% bf16 MFU | 1623332 tok/s
step 2748/19560 | loss 3.903172 (+3.12z)| norm 0.2764 (-0.59z)| lr 5.83e-04 | 322.74 ms | 52.3% bf16 MFU | 1623389 tok/s
step 2749/19560 | loss 3.699767 (-1.43z)| norm 0.2917 (+0.04z)| lr 5.83e-04 | 323.22 ms | 52.2% bf16 MFU | 1623324 tok/s
step 2750/19560 | loss 3.823112 (+1.30z)| norm 0.3000 (+0.37z)| lr 5.83e-04 | 322.96 ms | 52.3% bf16 MFU | 1623328 tok/s
val loss 3.739780
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2645/10042 = 0.263394
step 2751/19560 | loss 3.771020 (+0.15z)| norm 0.3100 (+0.77z)| lr 5.83e-04 | 322.62 ms | 52.3% bf16 MFU | 1623416 tok/s
step 2752/19560 | loss 3.746324 (-0.39z)| norm 0.2942 (+0.12z)| lr 5.83e-04 | 322.98 ms | 52.3% bf16 MFU | 1623409 tok/s
step 2753/19560 | loss 3.734823 (-0.64z)| norm 0.2805 (-0.43z)| lr 5.83e-04 | 323.23 ms | 52.2% bf16 MFU | 1623340 tok/s
step 2754/19560 | loss 3.793948 (+0.66z)| norm 0.2784 (-0.52z)| lr 5.83e-04 | 322.87 ms | 52.3% bf16 MFU | 1623365 tok/s
step 2755/19560 | loss 3.744542 (-0.43z)| norm 0.2580 (-1.33z)| lr 5.83e-04 | 323.70 ms | 52.1% bf16 MFU | 1623181 tok/s
step 2756/19560 | loss 3.758360 (-0.11z)| norm 0.2727 (-0.73z)| lr 5.83e-04 | 322.75 ms | 52.3% bf16 MFU | 1623244 tok/s
step 2757/19560 | loss 3.720559 (-0.95z)| norm 0.2505 (-1.60z)| lr 5.83e-04 | 322.99 ms | 52.3% bf16 MFU | 1623244 tok/s
step 2758/19560 | loss 3.645978 (-2.55z)| norm 0.2625 (-1.11z)| lr 5.83e-04 | 323.18 ms | 52.2% bf16 MFU | 1623195 tok/s
step 2759/19560 | loss 3.684530 (-1.66z)| norm 0.3170 (+1.07z)| lr 5.83e-04 | 322.88 ms | 52.3% bf16 MFU | 1623224 tok/s
step 2760/19560 | loss 3.790644 (+0.65z)| norm 0.3459 (+2.16z)| lr 5.83e-04 | 323.08 ms | 52.2% bf16 MFU | 1623203 tok/s
step 2761/19560 | loss 3.762017 (+0.05z)| norm 0.3278 (+1.43z)| lr 5.83e-04 | 322.74 ms | 52.3% bf16 MFU | 1623267 tok/s
step 2762/19560 | loss 3.765527 (+0.14z)| norm 0.3082 (+0.65z)| lr 5.82e-04 | 322.64 ms | 52.3% bf16 MFU | 1623355 tok/s
step 2763/19560 | loss 3.719879 (-0.90z)| norm 0.3215 (+1.15z)| lr 5.82e-04 | 323.28 ms | 52.2% bf16 MFU | 1623275 tok/s
step 2764/19560 | loss 3.748742 (-0.23z)| norm 0.3060 (+0.54z)| lr 5.82e-04 | 322.80 ms | 52.3% bf16 MFU | 1623322 tok/s
step 2765/19560 | loss 3.750151 (-0.21z)| norm 0.2690 (-0.90z)| lr 5.82e-04 | 323.28 ms | 52.2% bf16 MFU | 1623244 tok/s
step 2766/19560 | loss 3.743104 (-0.37z)| norm 0.2835 (-0.34z)| lr 5.82e-04 | 322.67 ms | 52.3% bf16 MFU | 1623325 tok/s
step 2767/19560 | loss 3.756400 (-0.06z)| norm 0.2705 (-0.85z)| lr 5.82e-04 | 322.98 ms | 52.3% bf16 MFU | 1623322 tok/s
step 2768/19560 | loss 3.747297 (-0.26z)| norm 0.2914 (-0.03z)| lr 5.82e-04 | 323.23 ms | 52.2% bf16 MFU | 1623258 tok/s
step 2769/19560 | loss 3.824649 (+1.51z)| norm 0.2953 (+0.13z)| lr 5.82e-04 | 323.08 ms | 52.2% bf16 MFU | 1623235 tok/s
step 2770/19560 | loss 3.754273 (-0.09z)| norm 0.2938 (+0.08z)| lr 5.82e-04 | 323.05 ms | 52.2% bf16 MFU | 1623221 tok/s
step 2771/19560 | loss 3.783105 (+0.58z)| norm 0.2662 (-1.00z)| lr 5.82e-04 | 323.12 ms | 52.2% bf16 MFU | 1623189 tok/s
step 2772/19560 | loss 3.725506 (-0.74z)| norm 0.3088 (+0.66z)| lr 5.82e-04 | 323.67 ms | 52.1% bf16 MFU | 1623022 tok/s
step 2773/19560 | loss 3.778456 (+0.50z)| norm 0.3554 (+2.42z)| lr 5.82e-04 | 322.06 ms | 52.4% bf16 MFU | 1623266 tok/s
step 2774/19560 | loss 3.686740 (-1.64z)| norm 0.3359 (+1.64z)| lr 5.82e-04 | 322.69 ms | 52.3% bf16 MFU | 1623340 tok/s
step 2775/19560 | loss 3.714483 (-0.99z)| norm 0.3137 (+0.78z)| lr 5.82e-04 | 323.31 ms | 52.2% bf16 MFU | 1623253 tok/s
step 2776/19560 | loss 3.723993 (-0.78z)| norm 0.3099 (+0.63z)| lr 5.82e-04 | 323.24 ms | 52.2% bf16 MFU | 1623190 tok/s
step 2777/19560 | loss 3.785683 (+0.66z)| norm 0.3611 (+2.50z)| lr 5.82e-04 | 322.80 ms | 52.3% bf16 MFU | 1623238 tok/s
step 2778/19560 | loss 3.783050 (+0.61z)| norm 0.3299 (+1.32z)| lr 5.82e-04 | 323.20 ms | 52.2% bf16 MFU | 1623186 tok/s
step 2779/19560 | loss 3.784127 (+0.63z)| norm 0.2826 (-0.45z)| lr 5.82e-04 | 323.13 ms | 52.2% bf16 MFU | 1623154 tok/s
step 2780/19560 | loss 3.657238 (-2.32z)| norm 0.3179 (+0.85z)| lr 5.82e-04 | 323.12 ms | 52.2% bf16 MFU | 1623125 tok/s
step 2781/19560 | loss 3.685107 (-1.64z)| norm 0.3303 (+1.30z)| lr 5.82e-04 | 323.16 ms | 52.2% bf16 MFU | 1623088 tok/s
step 2782/19560 | loss 3.752918 (-0.08z)| norm 0.2967 (+0.05z)| lr 5.82e-04 | 322.27 ms | 52.4% bf16 MFU | 1623276 tok/s
step 2783/19560 | loss 3.754962 (-0.03z)| norm 0.3173 (+0.82z)| lr 5.82e-04 | 322.64 ms | 52.3% bf16 MFU | 1623362 tok/s
step 2784/19560 | loss 3.762016 (+0.13z)| norm 0.3185 (+0.86z)| lr 5.82e-04 | 323.03 ms | 52.2% bf16 MFU | 1623345 tok/s
step 2785/19560 | loss 3.711478 (-1.02z)| norm 0.3231 (+1.02z)| lr 5.82e-04 | 323.12 ms | 52.2% bf16 MFU | 1623307 tok/s
step 2786/19560 | loss 3.768352 (+0.29z)| norm 0.2995 (+0.14z)| lr 5.82e-04 | 322.39 ms | 52.3% bf16 MFU | 1623453 tok/s
step 2787/19560 | loss 3.783440 (+0.63z)| norm 0.3165 (+0.76z)| lr 5.82e-04 | 323.37 ms | 52.2% bf16 MFU | 1623348 tok/s
step 2788/19560 | loss 3.851493 (+2.15z)| norm 0.3297 (+1.23z)| lr 5.82e-04 | 323.02 ms | 52.2% bf16 MFU | 1623334 tok/s
step 2789/19560 | loss 3.733188 (-0.53z)| norm 0.2696 (-0.99z)| lr 5.82e-04 | 322.92 ms | 52.3% bf16 MFU | 1623346 tok/s
step 2790/19560 | loss 3.839342 (+1.84z)| norm 0.3001 (+0.16z)| lr 5.82e-04 | 322.56 ms | 52.3% bf16 MFU | 1623449 tok/s
step 2791/19560 | loss 3.666276 (-1.99z)| norm 0.2962 (+0.01z)| lr 5.82e-04 | 322.59 ms | 52.3% bf16 MFU | 1623539 tok/s
step 2792/19560 | loss 3.663516 (-2.02z)| norm 0.2874 (-0.31z)| lr 5.82e-04 | 323.06 ms | 52.2% bf16 MFU | 1623507 tok/s
step 2793/19560 | loss 3.739332 (-0.33z)| norm 0.2647 (-1.16z)| lr 5.82e-04 | 323.00 ms | 52.3% bf16 MFU | 1623492 tok/s
step 2794/19560 | loss 3.743019 (-0.24z)| norm 0.2772 (-0.70z)| lr 5.82e-04 | 322.58 ms | 52.3% bf16 MFU | 1623581 tok/s
step 2795/19560 | loss 3.759586 (+0.13z)| norm 0.2685 (-1.00z)| lr 5.82e-04 | 323.20 ms | 52.2% bf16 MFU | 1623512 tok/s
step 2796/19560 | loss 3.753053 (-0.02z)| norm 0.2673 (-1.04z)| lr 5.82e-04 | 322.69 ms | 52.3% bf16 MFU | 1623574 tok/s
step 2797/19560 | loss 3.692133 (-1.37z)| norm 0.2709 (-0.89z)| lr 5.82e-04 | 323.07 ms | 52.2% bf16 MFU | 1623537 tok/s
step 2798/19560 | loss 3.728142 (-0.57z)| norm 0.2484 (-1.70z)| lr 5.82e-04 | 322.99 ms | 52.3% bf16 MFU | 1623523 tok/s
step 2799/19560 | loss 3.911849 (+3.44z)| norm 0.2594 (-1.27z)| lr 5.82e-04 | 323.10 ms | 52.2% bf16 MFU | 1623481 tok/s
step 2800/19560 | loss 3.722550 (-0.68z)| norm 0.2888 (-0.17z)| lr 5.82e-04 | 322.56 ms | 52.3% bf16 MFU | 1623578 tok/s
step 2801/19560 | loss 3.727488 (-0.57z)| norm 0.3073 (+0.51z)| lr 5.82e-04 | 322.50 ms | 52.3% bf16 MFU | 1623684 tok/s
step 2802/19560 | loss 3.749223 (-0.10z)| norm 0.2721 (-0.79z)| lr 5.82e-04 | 323.27 ms | 52.2% bf16 MFU | 1623592 tok/s
step 2803/19560 | loss 3.759329 (+0.12z)| norm 0.2785 (-0.55z)| lr 5.82e-04 | 322.43 ms | 52.3% bf16 MFU | 1623716 tok/s
step 2804/19560 | loss 3.719505 (-0.74z)| norm 0.2553 (-1.41z)| lr 5.82e-04 | 322.74 ms | 52.3% bf16 MFU | 1623754 tok/s
step 2805/19560 | loss 3.740845 (-0.27z)| norm 0.2566 (-1.34z)| lr 5.82e-04 | 322.97 ms | 52.3% bf16 MFU | 1623732 tok/s
step 2806/19560 | loss 3.731171 (-0.50z)| norm 0.2611 (-1.17z)| lr 5.82e-04 | 322.50 ms | 52.3% bf16 MFU | 1623832 tok/s
step 2807/19560 | loss 3.742512 (-0.23z)| norm 0.2403 (-1.89z)| lr 5.82e-04 | 322.65 ms | 52.3% bf16 MFU | 1623887 tok/s
step 2808/19560 | loss 3.772869 (+0.43z)| norm 0.2524 (-1.43z)| lr 5.82e-04 | 322.66 ms | 52.3% bf16 MFU | 1623939 tok/s
step 2809/19560 | loss 3.711201 (-0.93z)| norm 0.2648 (-0.97z)| lr 5.82e-04 | 322.34 ms | 52.4% bf16 MFU | 1624067 tok/s
step 2810/19560 | loss 3.742381 (-0.23z)| norm 0.2844 (-0.27z)| lr 5.82e-04 | 322.86 ms | 52.3% bf16 MFU | 1624058 tok/s
step 2811/19560 | loss 3.764773 (+0.27z)| norm 0.3203 (+1.01z)| lr 5.82e-04 | 323.14 ms | 52.2% bf16 MFU | 1623980 tok/s
step 2812/19560 | loss 3.699181 (-1.18z)| norm 0.3103 (+0.64z)| lr 5.82e-04 | 322.57 ms | 52.3% bf16 MFU | 1624048 tok/s
step 2813/19560 | loss 3.774446 (+0.49z)| norm 0.2745 (-0.65z)| lr 5.82e-04 | 322.58 ms | 52.3% bf16 MFU | 1624110 tok/s
step 2814/19560 | loss 3.745945 (-0.14z)| norm 0.3336 (+1.45z)| lr 5.82e-04 | 322.56 ms | 52.3% bf16 MFU | 1624174 tok/s
step 2815/19560 | loss 3.690374 (-1.35z)| norm 0.3153 (+0.78z)| lr 5.82e-04 | 322.88 ms | 52.3% bf16 MFU | 1624155 tok/s
step 2816/19560 | loss 3.717651 (-0.75z)| norm 0.3048 (+0.40z)| lr 5.82e-04 | 323.18 ms | 52.2% bf16 MFU | 1624062 tok/s
step 2817/19560 | loss 3.760496 (+0.19z)| norm 0.3194 (+0.91z)| lr 5.82e-04 | 322.69 ms | 52.3% bf16 MFU | 1624096 tok/s
step 2818/19560 | loss 3.727116 (-0.54z)| norm 0.2704 (-0.83z)| lr 5.82e-04 | 322.44 ms | 52.3% bf16 MFU | 1624191 tok/s
step 2819/19560 | loss 3.779703 (+0.62z)| norm 0.2768 (-0.60z)| lr 5.82e-04 | 322.69 ms | 52.3% bf16 MFU | 1624218 tok/s
step 2820/19560 | loss 3.871721 (+2.56z)| norm 0.2660 (-0.98z)| lr 5.82e-04 | 322.44 ms | 52.3% bf16 MFU | 1624307 tok/s
step 2821/19560 | loss 3.703601 (-1.06z)| norm 0.2826 (-0.39z)| lr 5.81e-04 | 323.47 ms | 52.2% bf16 MFU | 1624133 tok/s
step 2822/19560 | loss 3.747183 (-0.11z)| norm 0.2873 (-0.21z)| lr 5.81e-04 | 322.88 ms | 52.3% bf16 MFU | 1624115 tok/s
step 2823/19560 | loss 3.713122 (-0.83z)| norm 0.2759 (-0.61z)| lr 5.81e-04 | 322.82 ms | 52.3% bf16 MFU | 1624114 tok/s
step 2824/19560 | loss 3.782605 (+0.68z)| norm 0.3098 (+0.66z)| lr 5.81e-04 | 322.69 ms | 52.3% bf16 MFU | 1624147 tok/s
step 2825/19560 | loss 3.776878 (+0.55z)| norm 0.3032 (+0.43z)| lr 5.81e-04 | 322.48 ms | 52.3% bf16 MFU | 1624229 tok/s
step 2826/19560 | loss 3.757584 (+0.13z)| norm 0.2869 (-0.19z)| lr 5.81e-04 | 323.04 ms | 52.2% bf16 MFU | 1624166 tok/s
step 2827/19560 | loss 3.699047 (-1.13z)| norm 0.2443 (-1.76z)| lr 5.81e-04 | 323.01 ms | 52.2% bf16 MFU | 1624114 tok/s
step 2828/19560 | loss 3.687774 (-1.36z)| norm 0.2718 (-0.73z)| lr 5.81e-04 | 322.66 ms | 52.3% bf16 MFU | 1624154 tok/s
step 2829/19560 | loss 3.764784 (+0.31z)| norm 0.3207 (+1.09z)| lr 5.81e-04 | 322.61 ms | 52.3% bf16 MFU | 1624203 tok/s
step 2830/19560 | loss 3.737046 (-0.29z)| norm 0.3395 (+1.75z)| lr 5.81e-04 | 323.03 ms | 52.2% bf16 MFU | 1624145 tok/s
step 2831/19560 | loss 3.793384 (+0.94z)| norm 0.3203 (+1.04z)| lr 5.81e-04 | 323.35 ms | 52.2% bf16 MFU | 1624008 tok/s
step 2832/19560 | loss 3.713360 (-0.82z)| norm 0.2637 (-1.03z)| lr 5.81e-04 | 322.59 ms | 52.3% bf16 MFU | 1624071 tok/s
step 2833/19560 | loss 3.754216 (+0.08z)| norm 0.2785 (-0.47z)| lr 5.81e-04 | 322.52 ms | 52.3% bf16 MFU | 1624148 tok/s
step 2834/19560 | loss 3.690750 (-1.30z)| norm 0.2881 (-0.11z)| lr 5.81e-04 | 323.46 ms | 52.2% bf16 MFU | 1623984 tok/s
step 2835/19560 | loss 3.702110 (-1.05z)| norm 0.2502 (-1.49z)| lr 5.81e-04 | 322.51 ms | 52.3% bf16 MFU | 1624067 tok/s
step 2836/19560 | loss 3.720030 (-0.64z)| norm 0.2824 (-0.30z)| lr 5.81e-04 | 323.40 ms | 52.2% bf16 MFU | 1623923 tok/s
step 2837/19560 | loss 3.746528 (-0.05z)| norm 0.2614 (-1.07z)| lr 5.81e-04 | 322.74 ms | 52.3% bf16 MFU | 1623953 tok/s
step 2838/19560 | loss 3.734298 (-0.33z)| norm 0.2925 (+0.08z)| lr 5.81e-04 | 322.42 ms | 52.3% bf16 MFU | 1624060 tok/s
step 2839/19560 | loss 3.713338 (-0.81z)| norm 0.3145 (+0.87z)| lr 5.81e-04 | 322.92 ms | 52.3% bf16 MFU | 1624035 tok/s
step 2840/19560 | loss 3.678433 (-1.56z)| norm 0.2921 (+0.04z)| lr 5.81e-04 | 323.19 ms | 52.2% bf16 MFU | 1623944 tok/s
step 2841/19560 | loss 3.754615 (+0.14z)| norm 0.2734 (-0.67z)| lr 5.81e-04 | 323.15 ms | 52.2% bf16 MFU | 1623868 tok/s
step 2842/19560 | loss 3.702112 (-1.03z)| norm 0.2906 (-0.02z)| lr 5.81e-04 | 322.99 ms | 52.3% bf16 MFU | 1623836 tok/s
step 2843/19560 | loss 3.728867 (-0.43z)| norm 0.2454 (-1.70z)| lr 5.81e-04 | 322.34 ms | 52.4% bf16 MFU | 1623968 tok/s
step 2844/19560 | loss 3.785123 (+0.81z)| norm 0.2749 (-0.61z)| lr 5.81e-04 | 323.22 ms | 52.2% bf16 MFU | 1623873 tok/s
step 2845/19560 | loss 3.682388 (-1.46z)| norm 0.3167 (+0.94z)| lr 5.81e-04 | 323.13 ms | 52.2% bf16 MFU | 1623806 tok/s
step 2846/19560 | loss 3.746674 (-0.03z)| norm 0.3134 (+0.81z)| lr 5.81e-04 | 323.04 ms | 52.2% bf16 MFU | 1623765 tok/s
step 2847/19560 | loss 3.723579 (-0.53z)| norm 0.2888 (-0.12z)| lr 5.81e-04 | 322.73 ms | 52.3% bf16 MFU | 1623805 tok/s
step 2848/19560 | loss 3.747608 (+0.01z)| norm 0.2881 (-0.14z)| lr 5.81e-04 | 323.06 ms | 52.2% bf16 MFU | 1623759 tok/s
step 2849/19560 | loss 3.686808 (-1.32z)| norm 0.3079 (+0.59z)| lr 5.81e-04 | 322.56 ms | 52.3% bf16 MFU | 1623840 tok/s
step 2850/19560 | loss 3.764825 (+0.40z)| norm 0.3378 (+1.67z)| lr 5.81e-04 | 323.39 ms | 52.2% bf16 MFU | 1623710 tok/s
step 2851/19560 | loss 3.689434 (-1.25z)| norm 0.3336 (+1.49z)| lr 5.81e-04 | 322.53 ms | 52.3% bf16 MFU | 1623801 tok/s
step 2852/19560 | loss 3.701958 (-0.97z)| norm 0.2832 (-0.36z)| lr 5.81e-04 | 322.75 ms | 52.3% bf16 MFU | 1623832 tok/s
step 2853/19560 | loss 3.725263 (-0.45z)| norm 0.2299 (-2.26z)| lr 5.81e-04 | 322.97 ms | 52.3% bf16 MFU | 1623807 tok/s
step 2854/19560 | loss 3.650445 (-2.05z)| norm 0.2514 (-1.45z)| lr 5.81e-04 | 323.29 ms | 52.2% bf16 MFU | 1623703 tok/s
step 2855/19560 | loss 3.818886 (+1.58z)| norm 0.2938 (+0.08z)| lr 5.81e-04 | 322.62 ms | 52.3% bf16 MFU | 1623772 tok/s
step 2856/19560 | loss 3.726985 (-0.39z)| norm 0.3041 (+0.46z)| lr 5.81e-04 | 323.28 ms | 52.2% bf16 MFU | 1623672 tok/s
step 2857/19560 | loss 3.710419 (-0.74z)| norm 0.2821 (-0.33z)| lr 5.81e-04 | 323.99 ms | 52.1% bf16 MFU | 1623400 tok/s
step 2858/19560 | loss 3.739505 (-0.12z)| norm 0.2709 (-0.73z)| lr 5.81e-04 | 322.30 ms | 52.4% bf16 MFU | 1623565 tok/s
step 2859/19560 | loss 3.825562 (+1.70z)| norm 0.3024 (+0.42z)| lr 5.81e-04 | 322.79 ms | 52.3% bf16 MFU | 1623599 tok/s
step 2860/19560 | loss 3.694658 (-1.08z)| norm 0.3107 (+0.72z)| lr 5.81e-04 | 323.11 ms | 52.2% bf16 MFU | 1623551 tok/s
step 2861/19560 | loss 3.713740 (-0.65z)| norm 0.3076 (+0.60z)| lr 5.81e-04 | 323.42 ms | 52.2% bf16 MFU | 1623427 tok/s
step 2862/19560 | loss 3.711244 (-0.70z)| norm 0.2763 (-0.54z)| lr 5.81e-04 | 322.66 ms | 52.3% bf16 MFU | 1623499 tok/s
step 2863/19560 | loss 3.791805 (+1.03z)| norm 0.2604 (-1.10z)| lr 5.81e-04 | 323.17 ms | 52.2% bf16 MFU | 1623441 tok/s
step 2864/19560 | loss 3.743989 (+0.00z)| norm 0.2866 (-0.17z)| lr 5.81e-04 | 322.98 ms | 52.3% bf16 MFU | 1623432 tok/s
step 2865/19560 | loss 3.695203 (-1.03z)| norm 0.2653 (-0.94z)| lr 5.81e-04 | 322.70 ms | 52.3% bf16 MFU | 1623496 tok/s
step 2866/19560 | loss 3.694207 (-1.04z)| norm 0.3042 (+0.47z)| lr 5.81e-04 | 323.66 ms | 52.1% bf16 MFU | 1623314 tok/s
step 2867/19560 | loss 3.745579 (+0.07z)| norm 0.3131 (+0.78z)| lr 5.81e-04 | 323.52 ms | 52.2% bf16 MFU | 1623176 tok/s
step 2868/19560 | loss 3.709235 (-0.71z)| norm 0.2934 (+0.04z)| lr 5.81e-04 | 322.59 ms | 52.3% bf16 MFU | 1623280 tok/s
step 2869/19560 | loss 3.709957 (-0.70z)| norm 0.3350 (+1.57z)| lr 5.81e-04 | 322.65 ms | 52.3% bf16 MFU | 1623362 tok/s
step 2870/19560 | loss 3.757650 (+0.35z)| norm 0.3408 (+1.75z)| lr 5.81e-04 | 322.77 ms | 52.3% bf16 MFU | 1623411 tok/s
step 2871/19560 | loss 3.731882 (-0.22z)| norm 0.3417 (+1.77z)| lr 5.81e-04 | 322.45 ms | 52.3% bf16 MFU | 1623538 tok/s
step 2872/19560 | loss 3.691308 (-1.09z)| norm 0.2885 (-0.17z)| lr 5.81e-04 | 322.80 ms | 52.3% bf16 MFU | 1623571 tok/s
step 2873/19560 | loss 3.750730 (+0.21z)| norm 0.2719 (-0.77z)| lr 5.81e-04 | 323.39 ms | 52.2% bf16 MFU | 1623454 tok/s
step 2874/19560 | loss 3.746755 (+0.12z)| norm 0.3060 (+0.50z)| lr 5.81e-04 | 323.22 ms | 52.2% bf16 MFU | 1623386 tok/s
step 2875/19560 | loss 3.670473 (-1.54z)| norm 0.2594 (-1.22z)| lr 5.81e-04 | 322.99 ms | 52.3% bf16 MFU | 1623377 tok/s
step 2876/19560 | loss 3.720363 (-0.44z)| norm 0.2495 (-1.57z)| lr 5.81e-04 | 322.71 ms | 52.3% bf16 MFU | 1623439 tok/s
step 2877/19560 | loss 3.735169 (-0.11z)| norm 0.2707 (-0.78z)| lr 5.81e-04 | 323.60 ms | 52.2% bf16 MFU | 1623275 tok/s
step 2878/19560 | loss 3.757975 (+0.44z)| norm 0.2869 (-0.18z)| lr 5.80e-04 | 322.48 ms | 52.3% bf16 MFU | 1623401 tok/s
step 2879/19560 | loss 3.781589 (+0.99z)| norm 0.2687 (-0.83z)| lr 5.80e-04 | 322.80 ms | 52.3% bf16 MFU | 1623440 tok/s
step 2880/19560 | loss 3.662468 (-1.76z)| norm 0.2571 (-1.24z)| lr 5.80e-04 | 323.88 ms | 52.1% bf16 MFU | 1623206 tok/s
step 2881/19560 | loss 3.749111 (+0.24z)| norm 0.2582 (-1.19z)| lr 5.80e-04 | 323.31 ms | 52.2% bf16 MFU | 1623128 tok/s
step 2882/19560 | loss 3.752007 (+0.31z)| norm 0.2618 (-1.05z)| lr 5.80e-04 | 322.40 ms | 52.3% bf16 MFU | 1623281 tok/s
step 2883/19560 | loss 3.740732 (+0.05z)| norm 0.2820 (-0.33z)| lr 5.80e-04 | 323.06 ms | 52.2% bf16 MFU | 1623261 tok/s
step 2884/19560 | loss 3.779354 (+0.94z)| norm 0.2746 (-0.60z)| lr 5.80e-04 | 322.50 ms | 52.3% bf16 MFU | 1623383 tok/s
step 2885/19560 | loss 3.702133 (-0.84z)| norm 0.2666 (-0.90z)| lr 5.80e-04 | 322.96 ms | 52.3% bf16 MFU | 1623384 tok/s
step 2886/19560 | loss 3.746030 (+0.16z)| norm 0.2768 (-0.53z)| lr 5.80e-04 | 322.70 ms | 52.3% bf16 MFU | 1623448 tok/s
step 2887/19560 | loss 3.720223 (-0.46z)| norm 0.2751 (-0.58z)| lr 5.80e-04 | 322.61 ms | 52.3% bf16 MFU | 1623532 tok/s
step 2888/19560 | loss 3.680507 (-1.38z)| norm 0.2749 (-0.58z)| lr 5.80e-04 | 323.18 ms | 52.2% bf16 MFU | 1623471 tok/s
step 2889/19560 | loss 3.757445 (+0.44z)| norm 0.2681 (-0.82z)| lr 5.80e-04 | 322.82 ms | 52.3% bf16 MFU | 1623502 tok/s
step 2890/19560 | loss 3.675918 (-1.46z)| norm 0.3047 (+0.56z)| lr 5.80e-04 | 322.73 ms | 52.3% bf16 MFU | 1623554 tok/s
step 2891/19560 | loss 3.691824 (-1.08z)| norm 0.3079 (+0.69z)| lr 5.80e-04 | 322.84 ms | 52.3% bf16 MFU | 1623575 tok/s
step 2892/19560 | loss 3.868124 (+2.93z)| norm 0.3173 (+1.04z)| lr 5.80e-04 | 323.19 ms | 52.2% bf16 MFU | 1623508 tok/s
step 2893/19560 | loss 3.744205 (+0.13z)| norm 0.3417 (+1.92z)| lr 5.80e-04 | 322.69 ms | 52.3% bf16 MFU | 1623569 tok/s
step 2894/19560 | loss 3.773355 (+0.78z)| norm 0.3171 (+0.98z)| lr 5.80e-04 | 323.06 ms | 52.2% bf16 MFU | 1623534 tok/s
step 2895/19560 | loss 3.663753 (-1.66z)| norm 0.3407 (+1.82z)| lr 5.80e-04 | 323.10 ms | 52.2% bf16 MFU | 1623491 tok/s
step 2896/19560 | loss 3.741116 (+0.07z)| norm 0.3109 (+0.72z)| lr 5.80e-04 | 323.09 ms | 52.2% bf16 MFU | 1623453 tok/s
step 2897/19560 | loss 3.765071 (+0.62z)| norm 0.2887 (-0.10z)| lr 5.80e-04 | 323.50 ms | 52.2% bf16 MFU | 1623314 tok/s
step 2898/19560 | loss 3.713593 (-0.54z)| norm 0.2782 (-0.47z)| lr 5.80e-04 | 322.63 ms | 52.3% bf16 MFU | 1623400 tok/s
step 2899/19560 | loss 3.716004 (-0.47z)| norm 0.2867 (-0.17z)| lr 5.80e-04 | 323.09 ms | 52.2% bf16 MFU | 1623367 tok/s
step 2900/19560 | loss 3.683182 (-1.20z)| norm 0.2396 (-1.86z)| lr 5.80e-04 | 323.03 ms | 52.2% bf16 MFU | 1623350 tok/s
step 2901/19560 | loss 3.731867 (-0.10z)| norm 0.2606 (-1.09z)| lr 5.80e-04 | 323.35 ms | 52.2% bf16 MFU | 1623255 tok/s
step 2902/19560 | loss 3.690414 (-1.04z)| norm 0.3047 (+0.56z)| lr 5.80e-04 | 323.16 ms | 52.2% bf16 MFU | 1623210 tok/s
step 2903/19560 | loss 3.674229 (-1.39z)| norm 0.2745 (-0.56z)| lr 5.80e-04 | 323.55 ms | 52.2% bf16 MFU | 1623070 tok/s
step 2904/19560 | loss 3.716165 (-0.44z)| norm 0.2745 (-0.55z)| lr 5.80e-04 | 324.23 ms | 52.1% bf16 MFU | 1622767 tok/s
step 2905/19560 | loss 3.688177 (-1.05z)| norm 0.2751 (-0.52z)| lr 5.80e-04 | 323.01 ms | 52.2% bf16 MFU | 1622785 tok/s
step 2906/19560 | loss 3.755404 (+0.47z)| norm 0.2841 (-0.16z)| lr 5.80e-04 | 323.26 ms | 52.2% bf16 MFU | 1622739 tok/s
step 2907/19560 | loss 3.755227 (+0.47z)| norm 0.2678 (-0.79z)| lr 5.80e-04 | 322.96 ms | 52.3% bf16 MFU | 1622771 tok/s
step 2908/19560 | loss 3.695385 (-0.90z)| norm 0.2565 (-1.21z)| lr 5.80e-04 | 323.34 ms | 52.2% bf16 MFU | 1622705 tok/s
step 2909/19560 | loss 3.721891 (-0.31z)| norm 0.2749 (-0.48z)| lr 5.80e-04 | 323.68 ms | 52.1% bf16 MFU | 1622558 tok/s
step 2910/19560 | loss 3.712732 (-0.51z)| norm 0.2823 (-0.19z)| lr 5.80e-04 | 323.20 ms | 52.2% bf16 MFU | 1622539 tok/s
step 2911/19560 | loss 3.841750 (+2.39z)| norm 0.2729 (-0.54z)| lr 5.80e-04 | 323.29 ms | 52.2% bf16 MFU | 1622498 tok/s
step 2912/19560 | loss 3.806421 (+1.57z)| norm 0.2797 (-0.27z)| lr 5.80e-04 | 323.49 ms | 52.2% bf16 MFU | 1622409 tok/s
step 2913/19560 | loss 3.888103 (+3.23z)| norm 0.3031 (+0.67z)| lr 5.80e-04 | 322.70 ms | 52.3% bf16 MFU | 1622523 tok/s
step 2914/19560 | loss 3.741333 (+0.09z)| norm 0.3639 (+2.97z)| lr 5.80e-04 | 322.16 ms | 52.4% bf16 MFU | 1622768 tok/s
step 2915/19560 | loss 3.703396 (-0.71z)| norm 0.3550 (+2.56z)| lr 5.80e-04 | 322.96 ms | 52.3% bf16 MFU | 1622799 tok/s
step 2916/19560 | loss 3.686385 (-1.07z)| norm 0.3386 (+1.93z)| lr 5.80e-04 | 322.92 ms | 52.3% bf16 MFU | 1622837 tok/s
step 2917/19560 | loss 3.741388 (+0.14z)| norm 0.2758 (-0.43z)| lr 5.80e-04 | 322.32 ms | 52.4% bf16 MFU | 1623027 tok/s
step 2918/19560 | loss 3.727317 (-0.16z)| norm 0.3167 (+1.10z)| lr 5.80e-04 | 322.62 ms | 52.3% bf16 MFU | 1623129 tok/s
step 2919/19560 | loss 3.759880 (+0.56z)| norm 0.3303 (+1.58z)| lr 5.80e-04 | 322.88 ms | 52.3% bf16 MFU | 1623162 tok/s
step 2920/19560 | loss 3.699652 (-0.81z)| norm 0.3062 (+0.68z)| lr 5.80e-04 | 322.55 ms | 52.3% bf16 MFU | 1623277 tok/s
step 2921/19560 | loss 3.734389 (-0.02z)| norm 0.2778 (-0.37z)| lr 5.80e-04 | 323.05 ms | 52.2% bf16 MFU | 1623259 tok/s
step 2922/19560 | loss 3.794522 (+1.33z)| norm 0.2874 (-0.02z)| lr 5.80e-04 | 323.12 ms | 52.2% bf16 MFU | 1623226 tok/s
step 2923/19560 | loss 3.742994 (+0.17z)| norm 0.2848 (-0.12z)| lr 5.80e-04 | 323.37 ms | 52.2% bf16 MFU | 1623131 tok/s
step 2924/19560 | loss 3.678155 (-1.28z)| norm 0.3035 (+0.56z)| lr 5.80e-04 | 322.94 ms | 52.3% bf16 MFU | 1623148 tok/s
step 2925/19560 | loss 3.675274 (-1.33z)| norm 0.3171 (+1.06z)| lr 5.80e-04 | 322.97 ms | 52.3% bf16 MFU | 1623156 tok/s
step 2926/19560 | loss 3.723248 (-0.26z)| norm 0.2696 (-0.72z)| lr 5.80e-04 | 323.38 ms | 52.2% bf16 MFU | 1623063 tok/s
step 2927/19560 | loss 3.690573 (-1.01z)| norm 0.2823 (-0.25z)| lr 5.80e-04 | 322.96 ms | 52.3% bf16 MFU | 1623080 tok/s
step 2928/19560 | loss 3.747028 (+0.33z)| norm 0.2721 (-0.63z)| lr 5.80e-04 | 323.37 ms | 52.2% bf16 MFU | 1622992 tok/s
step 2929/19560 | loss 3.701383 (-0.75z)| norm 0.2607 (-1.04z)| lr 5.80e-04 | 323.14 ms | 52.2% bf16 MFU | 1622966 tok/s
step 2930/19560 | loss 3.661227 (-1.67z)| norm 0.2582 (-1.13z)| lr 5.80e-04 | 323.39 ms | 52.2% bf16 MFU | 1622880 tok/s
step 2931/19560 | loss 3.712629 (-0.46z)| norm 0.2725 (-0.59z)| lr 5.80e-04 | 322.81 ms | 52.3% bf16 MFU | 1622943 tok/s
step 2932/19560 | loss 3.667543 (-1.49z)| norm 0.2769 (-0.44z)| lr 5.80e-04 | 322.96 ms | 52.3% bf16 MFU | 1622964 tok/s
step 2933/19560 | loss 3.721946 (-0.22z)| norm 0.2405 (-1.78z)| lr 5.80e-04 | 323.07 ms | 52.2% bf16 MFU | 1622957 tok/s
step 2934/19560 | loss 3.713171 (-0.42z)| norm 0.2690 (-0.72z)| lr 5.79e-04 | 322.49 ms | 52.3% bf16 MFU | 1623097 tok/s
step 2935/19560 | loss 3.719267 (-0.28z)| norm 0.2763 (-0.47z)| lr 5.79e-04 | 323.24 ms | 52.2% bf16 MFU | 1623042 tok/s
step 2936/19560 | loss 3.678944 (-1.20z)| norm 0.2506 (-1.44z)| lr 5.79e-04 | 322.98 ms | 52.3% bf16 MFU | 1623053 tok/s
step 2937/19560 | loss 3.699280 (-0.72z)| norm 0.2480 (-1.52z)| lr 5.79e-04 | 322.59 ms | 52.3% bf16 MFU | 1623163 tok/s
step 2938/19560 | loss 3.704854 (-0.58z)| norm 0.2510 (-1.39z)| lr 5.79e-04 | 323.03 ms | 52.2% bf16 MFU | 1623157 tok/s
step 2939/19560 | loss 3.772979 (+0.99z)| norm 0.2713 (-0.62z)| lr 5.79e-04 | 323.23 ms | 52.2% bf16 MFU | 1623100 tok/s
step 2940/19560 | loss 3.718306 (-0.28z)| norm 0.2720 (-0.59z)| lr 5.79e-04 | 323.08 ms | 52.2% bf16 MFU | 1623084 tok/s
step 2941/19560 | loss 3.737068 (+0.17z)| norm 0.2582 (-1.09z)| lr 5.79e-04 | 322.57 ms | 52.3% bf16 MFU | 1623198 tok/s
step 2942/19560 | loss 3.855021 (+2.81z)| norm 0.3075 (+0.76z)| lr 5.79e-04 | 322.67 ms | 52.3% bf16 MFU | 1623280 tok/s
step 2943/19560 | loss 3.741509 (+0.23z)| norm 0.3066 (+0.73z)| lr 5.79e-04 | 322.61 ms | 52.3% bf16 MFU | 1623374 tok/s
step 2944/19560 | loss 3.772003 (+0.91z)| norm 0.3260 (+1.44z)| lr 5.79e-04 | 322.83 ms | 52.3% bf16 MFU | 1623407 tok/s
step 2945/19560 | loss 3.707462 (-0.54z)| norm 0.3010 (+0.51z)| lr 5.79e-04 | 322.92 ms | 52.3% bf16 MFU | 1623416 tok/s
step 2946/19560 | loss 3.744477 (+0.30z)| norm 0.3134 (+0.97z)| lr 5.79e-04 | 322.77 ms | 52.3% bf16 MFU | 1623463 tok/s
step 2947/19560 | loss 3.719668 (-0.25z)| norm 0.3030 (+0.57z)| lr 5.79e-04 | 322.79 ms | 52.3% bf16 MFU | 1623501 tok/s
step 2948/19560 | loss 3.743231 (+0.32z)| norm 0.2930 (+0.19z)| lr 5.79e-04 | 323.03 ms | 52.2% bf16 MFU | 1623477 tok/s
step 2949/19560 | loss 3.700984 (-0.68z)| norm 0.2834 (-0.18z)| lr 5.79e-04 | 322.40 ms | 52.3% bf16 MFU | 1623614 tok/s
step 2950/19560 | loss 3.792044 (+1.45z)| norm 0.3132 (+0.93z)| lr 5.79e-04 | 322.69 ms | 52.3% bf16 MFU | 1623671 tok/s
step 2951/19560 | loss 3.760403 (+0.70z)| norm 0.2756 (-0.47z)| lr 5.79e-04 | 323.08 ms | 52.2% bf16 MFU | 1623626 tok/s
step 2952/19560 | loss 3.733781 (+0.09z)| norm 0.2454 (-1.57z)| lr 5.79e-04 | 322.81 ms | 52.3% bf16 MFU | 1623651 tok/s
step 2953/19560 | loss 3.649804 (-1.86z)| norm 0.2716 (-0.59z)| lr 5.79e-04 | 322.54 ms | 52.3% bf16 MFU | 1623742 tok/s
step 2954/19560 | loss 3.731605 (+0.06z)| norm 0.2847 (-0.10z)| lr 5.79e-04 | 322.57 ms | 52.3% bf16 MFU | 1623822 tok/s
step 2955/19560 | loss 3.686919 (-0.98z)| norm 0.2570 (-1.14z)| lr 5.79e-04 | 322.98 ms | 52.3% bf16 MFU | 1623795 tok/s
step 2956/19560 | loss 3.779972 (+1.18z)| norm 0.2769 (-0.40z)| lr 5.79e-04 | 322.22 ms | 52.4% bf16 MFU | 1623960 tok/s
step 2957/19560 | loss 3.658932 (-1.62z)| norm 0.2830 (-0.16z)| lr 5.79e-04 | 322.49 ms | 52.3% bf16 MFU | 1624050 tok/s
step 2958/19560 | loss 3.781906 (+1.22z)| norm 0.3039 (+0.64z)| lr 5.79e-04 | 322.67 ms | 52.3% bf16 MFU | 1624090 tok/s
step 2959/19560 | loss 3.707869 (-0.48z)| norm 0.2836 (-0.12z)| lr 5.79e-04 | 322.31 ms | 52.4% bf16 MFU | 1624219 tok/s
step 2960/19560 | loss 3.679030 (-1.14z)| norm 0.2777 (-0.35z)| lr 5.79e-04 | 322.74 ms | 52.3% bf16 MFU | 1624231 tok/s
step 2961/19560 | loss 3.667477 (-1.38z)| norm 0.2974 (+0.40z)| lr 5.79e-04 | 322.51 ms | 52.3% bf16 MFU | 1624301 tok/s
step 2962/19560 | loss 3.672512 (-1.26z)| norm 0.3079 (+0.80z)| lr 5.79e-04 | 322.49 ms | 52.3% bf16 MFU | 1624372 tok/s
step 2963/19560 | loss 3.682366 (-1.02z)| norm 0.3031 (+0.60z)| lr 5.79e-04 | 322.70 ms | 52.3% bf16 MFU | 1624388 tok/s
step 2964/19560 | loss 3.694747 (-0.73z)| norm 0.3049 (+0.66z)| lr 5.79e-04 | 322.78 ms | 52.3% bf16 MFU | 1624384 tok/s
step 2965/19560 | loss 3.689155 (-0.85z)| norm 0.2641 (-0.91z)| lr 5.79e-04 | 322.86 ms | 52.3% bf16 MFU | 1624360 tok/s
step 2966/19560 | loss 3.631439 (-2.11z)| norm 0.2729 (-0.57z)| lr 5.79e-04 | 323.03 ms | 52.2% bf16 MFU | 1624294 tok/s
step 2967/19560 | loss 3.717647 (-0.18z)| norm 0.2787 (-0.33z)| lr 5.79e-04 | 322.64 ms | 52.3% bf16 MFU | 1624328 tok/s
step 2968/19560 | loss 3.769530 (+0.96z)| norm 0.3033 (+0.61z)| lr 5.79e-04 | 322.50 ms | 52.3% bf16 MFU | 1624396 tok/s
step 2969/19560 | loss 3.703057 (-0.52z)| norm 0.3237 (+1.38z)| lr 5.79e-04 | 322.80 ms | 52.3% bf16 MFU | 1624385 tok/s
step 2970/19560 | loss 3.748514 (+0.49z)| norm 0.3010 (+0.50z)| lr 5.79e-04 | 323.23 ms | 52.2% bf16 MFU | 1624268 tok/s
step 2971/19560 | loss 3.708203 (-0.40z)| norm 0.2959 (+0.30z)| lr 5.79e-04 | 322.57 ms | 52.3% bf16 MFU | 1624322 tok/s
step 2972/19560 | loss 3.724105 (-0.04z)| norm 0.3013 (+0.50z)| lr 5.79e-04 | 322.83 ms | 52.3% bf16 MFU | 1624308 tok/s
step 2973/19560 | loss 3.671284 (-1.22z)| norm 0.2840 (-0.17z)| lr 5.79e-04 | 322.89 ms | 52.3% bf16 MFU | 1624279 tok/s
step 2974/19560 | loss 3.690986 (-0.77z)| norm 0.2858 (-0.09z)| lr 5.79e-04 | 323.32 ms | 52.2% bf16 MFU | 1624144 tok/s
step 2975/19560 | loss 3.775170 (+1.10z)| norm 0.2754 (-0.49z)| lr 5.79e-04 | 322.48 ms | 52.3% bf16 MFU | 1624227 tok/s
step 2976/19560 | loss 3.750643 (+0.56z)| norm 0.2869 (-0.04z)| lr 5.79e-04 | 322.84 ms | 52.3% bf16 MFU | 1624215 tok/s
step 2977/19560 | loss 3.726292 (+0.01z)| norm 0.2566 (-1.21z)| lr 5.79e-04 | 323.00 ms | 52.3% bf16 MFU | 1624163 tok/s
step 2978/19560 | loss 3.691401 (-0.76z)| norm 0.2797 (-0.29z)| lr 5.79e-04 | 322.89 ms | 52.3% bf16 MFU | 1624140 tok/s
step 2979/19560 | loss 3.763691 (+0.84z)| norm 0.2922 (+0.22z)| lr 5.79e-04 | 322.81 ms | 52.3% bf16 MFU | 1624140 tok/s
step 2980/19560 | loss 3.689962 (-0.80z)| norm 0.2570 (-1.18z)| lr 5.79e-04 | 322.98 ms | 52.3% bf16 MFU | 1624097 tok/s
step 2981/19560 | loss 3.768402 (+0.94z)| norm 0.2695 (-0.70z)| lr 5.79e-04 | 322.83 ms | 52.3% bf16 MFU | 1624094 tok/s
step 2982/19560 | loss 3.748224 (+0.48z)| norm 0.2844 (-0.11z)| lr 5.79e-04 | 322.85 ms | 52.3% bf16 MFU | 1624087 tok/s
step 2983/19560 | loss 3.691540 (-0.79z)| norm 0.2966 (+0.39z)| lr 5.79e-04 | 323.02 ms | 52.2% bf16 MFU | 1624036 tok/s
step 2984/19560 | loss 3.765627 (+0.89z)| norm 0.2693 (-0.72z)| lr 5.79e-04 | 322.67 ms | 52.3% bf16 MFU | 1624077 tok/s
step 2985/19560 | loss 3.651749 (-1.67z)| norm 0.2848 (-0.08z)| lr 5.79e-04 | 322.42 ms | 52.3% bf16 MFU | 1624178 tok/s
step 2986/19560 | loss 3.675994 (-1.11z)| norm 0.2585 (-1.15z)| lr 5.79e-04 | 322.96 ms | 52.3% bf16 MFU | 1624140 tok/s
step 2987/19560 | loss 3.702832 (-0.49z)| norm 0.2686 (-0.73z)| lr 5.79e-04 | 322.81 ms | 52.3% bf16 MFU | 1624138 tok/s
step 2988/19560 | loss 3.766839 (+0.95z)| norm 0.2908 (+0.18z)| lr 5.78e-04 | 323.29 ms | 52.2% bf16 MFU | 1624018 tok/s
step 2989/19560 | loss 3.777137 (+1.17z)| norm 0.2799 (-0.26z)| lr 5.78e-04 | 322.64 ms | 52.3% bf16 MFU | 1624067 tok/s
step 2990/19560 | loss 3.837072 (+2.45z)| norm 0.2675 (-0.76z)| lr 5.78e-04 | 322.72 ms | 52.3% bf16 MFU | 1624092 tok/s
step 2991/19560 | loss 3.752678 (+0.59z)| norm 0.2850 (-0.05z)| lr 5.78e-04 | 322.65 ms | 52.3% bf16 MFU | 1624135 tok/s
step 2992/19560 | loss 3.702930 (-0.51z)| norm 0.2768 (-0.39z)| lr 5.78e-04 | 323.27 ms | 52.2% bf16 MFU | 1624019 tok/s
step 2993/19560 | loss 3.731890 (+0.13z)| norm 0.3196 (+1.35z)| lr 5.78e-04 | 322.50 ms | 52.3% bf16 MFU | 1624102 tok/s
step 2994/19560 | loss 3.738987 (+0.28z)| norm 0.3051 (+0.76z)| lr 5.78e-04 | 322.87 ms | 52.3% bf16 MFU | 1624088 tok/s
step 2995/19560 | loss 3.710408 (-0.35z)| norm 0.2963 (+0.40z)| lr 5.78e-04 | 322.58 ms | 52.3% bf16 MFU | 1624147 tok/s
step 2996/19560 | loss 3.716156 (-0.23z)| norm 0.3166 (+1.22z)| lr 5.78e-04 | 322.71 ms | 52.3% bf16 MFU | 1624173 tok/s
step 2997/19560 | loss 3.698983 (-0.61z)| norm 0.3024 (+0.66z)| lr 5.78e-04 | 323.14 ms | 52.2% bf16 MFU | 1624087 tok/s
step 2998/19560 | loss 3.720685 (-0.12z)| norm 0.3095 (+0.98z)| lr 5.78e-04 | 322.48 ms | 52.3% bf16 MFU | 1624172 tok/s
step 2999/19560 | loss 3.735607 (+0.22z)| norm 0.3274 (+1.76z)| lr 5.78e-04 | 322.44 ms | 52.3% bf16 MFU | 1624262 tok/s
step 3000/19560 | loss 3.728986 (+0.06z)| norm 0.2829 (-0.13z)| lr 5.78e-04 | 322.83 ms | 52.3% bf16 MFU | 1624251 tok/s
val loss 3.705422
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2663/10042 = 0.265186
step 3001/19560 | loss 3.615312 (-2.42z)| norm 0.2612 (-1.05z)| lr 5.78e-04 | 322.77 ms | 52.3% bf16 MFU | 1624256 tok/s
step 3002/19560 | loss 3.705487 (-0.43z)| norm 0.2489 (-1.54z)| lr 5.78e-04 | 323.30 ms | 52.2% bf16 MFU | 1624128 tok/s
step 3003/19560 | loss 3.706055 (-0.42z)| norm 0.2694 (-0.68z)| lr 5.78e-04 | 322.98 ms | 52.3% bf16 MFU | 1624087 tok/s
step 3004/19560 | loss 3.725370 (+0.00z)| norm 0.2756 (-0.43z)| lr 5.78e-04 | 322.97 ms | 52.3% bf16 MFU | 1624049 tok/s
step 3005/19560 | loss 3.753065 (+0.61z)| norm 0.2657 (-0.85z)| lr 5.78e-04 | 323.18 ms | 52.2% bf16 MFU | 1623960 tok/s
step 3006/19560 | loss 3.701484 (-0.52z)| norm 0.2891 (+0.14z)| lr 5.78e-04 | 324.03 ms | 52.1% bf16 MFU | 1623663 tok/s
step 3007/19560 | loss 3.780184 (+1.22z)| norm 0.2993 (+0.57z)| lr 5.78e-04 | 323.21 ms | 52.2% bf16 MFU | 1623586 tok/s
step 3008/19560 | loss 3.733690 (+0.18z)| norm 0.3004 (+0.60z)| lr 5.78e-04 | 322.94 ms | 52.3% bf16 MFU | 1623581 tok/s
step 3009/19560 | loss 3.761492 (+0.80z)| norm 0.2967 (+0.44z)| lr 5.78e-04 | 323.11 ms | 52.2% bf16 MFU | 1623533 tok/s
step 3010/19560 | loss 3.770710 (+1.00z)| norm 0.2614 (-1.09z)| lr 5.78e-04 | 323.23 ms | 52.2% bf16 MFU | 1623458 tok/s
step 3011/19560 | loss 3.768864 (+0.95z)| norm 0.2824 (-0.18z)| lr 5.78e-04 | 323.37 ms | 52.2% bf16 MFU | 1623352 tok/s
step 3012/19560 | loss 3.734732 (+0.20z)| norm 0.2883 (+0.07z)| lr 5.78e-04 | 322.59 ms | 52.3% bf16 MFU | 1623447 tok/s
step 3013/19560 | loss 3.706581 (-0.42z)| norm 0.2744 (-0.53z)| lr 5.78e-04 | 322.92 ms | 52.3% bf16 MFU | 1623455 tok/s
step 3014/19560 | loss 3.751149 (+0.57z)| norm 0.2840 (-0.12z)| lr 5.78e-04 | 323.10 ms | 52.2% bf16 MFU | 1623417 tok/s
step 3015/19560 | loss 3.775253 (+1.09z)| norm 0.2956 (+0.37z)| lr 5.78e-04 | 323.17 ms | 52.2% bf16 MFU | 1623363 tok/s
step 3016/19560 | loss 3.722136 (-0.09z)| norm 0.2889 (+0.08z)| lr 5.78e-04 | 323.08 ms | 52.2% bf16 MFU | 1623333 tok/s
step 3017/19560 | loss 3.692103 (-0.75z)| norm 0.2672 (-0.86z)| lr 5.78e-04 | 322.72 ms | 52.3% bf16 MFU | 1623396 tok/s
step 3018/19560 | loss 3.730867 (+0.10z)| norm 0.2908 (+0.16z)| lr 5.78e-04 | 323.33 ms | 52.2% bf16 MFU | 1623302 tok/s
step 3019/19560 | loss 3.716696 (-0.22z)| norm 0.2949 (+0.35z)| lr 5.78e-04 | 322.93 ms | 52.3% bf16 MFU | 1623315 tok/s
step 3020/19560 | loss 3.682963 (-0.98z)| norm 0.2632 (-1.01z)| lr 5.78e-04 | 323.70 ms | 52.1% bf16 MFU | 1623132 tok/s
step 3021/19560 | loss 3.735471 (+0.24z)| norm 0.2692 (-0.74z)| lr 5.78e-04 | 323.03 ms | 52.2% bf16 MFU | 1623126 tok/s
step 3022/19560 | loss 3.721995 (-0.06z)| norm 0.2896 (+0.18z)| lr 5.78e-04 | 323.11 ms | 52.2% bf16 MFU | 1623101 tok/s
step 3023/19560 | loss 3.758198 (+0.77z)| norm 0.3094 (+1.09z)| lr 5.78e-04 | 322.04 ms | 52.4% bf16 MFU | 1623346 tok/s
step 3024/19560 | loss 3.706384 (-0.44z)| norm 0.3076 (+1.02z)| lr 5.78e-04 | 323.53 ms | 52.2% bf16 MFU | 1623205 tok/s
step 3025/19560 | loss 3.718027 (-0.16z)| norm 0.3093 (+1.08z)| lr 5.78e-04 | 323.09 ms | 52.2% bf16 MFU | 1623181 tok/s
step 3026/19560 | loss 3.726819 (+0.05z)| norm 0.3345 (+2.17z)| lr 5.78e-04 | 323.55 ms | 52.2% bf16 MFU | 1623044 tok/s
step 3027/19560 | loss 3.689518 (-0.82z)| norm 0.3112 (+1.11z)| lr 5.78e-04 | 322.38 ms | 52.4% bf16 MFU | 1623206 tok/s
step 3028/19560 | loss 3.850797 (+2.85z)| norm 0.2949 (+0.37z)| lr 5.78e-04 | 322.97 ms | 52.3% bf16 MFU | 1623213 tok/s
step 3029/19560 | loss 3.776739 (+1.15z)| norm 0.2680 (-0.85z)| lr 5.78e-04 | 323.27 ms | 52.2% bf16 MFU | 1623143 tok/s
step 3030/19560 | loss 3.721025 (-0.12z)| norm 0.2620 (-1.10z)| lr 5.78e-04 | 323.34 ms | 52.2% bf16 MFU | 1623059 tok/s
step 3031/19560 | loss 3.689968 (-0.84z)| norm 0.2912 (+0.22z)| lr 5.78e-04 | 323.37 ms | 52.2% bf16 MFU | 1622972 tok/s
step 3032/19560 | loss 3.708446 (-0.41z)| norm 0.2696 (-0.76z)| lr 5.78e-04 | 322.47 ms | 52.3% bf16 MFU | 1623116 tok/s
step 3033/19560 | loss 3.640597 (-1.93z)| norm 0.2758 (-0.48z)| lr 5.78e-04 | 322.45 ms | 52.3% bf16 MFU | 1623259 tok/s
step 3034/19560 | loss 3.732359 (+0.14z)| norm 0.2776 (-0.40z)| lr 5.78e-04 | 323.02 ms | 52.2% bf16 MFU | 1623251 tok/s
step 3035/19560 | loss 3.664042 (-1.37z)| norm 0.3274 (+1.81z)| lr 5.78e-04 | 323.00 ms | 52.3% bf16 MFU | 1623247 tok/s
step 3036/19560 | loss 3.678757 (-1.04z)| norm 0.2708 (-0.73z)| lr 5.78e-04 | 322.60 ms | 52.3% bf16 MFU | 1623344 tok/s
step 3037/19560 | loss 3.708128 (-0.38z)| norm 0.2836 (-0.16z)| lr 5.78e-04 | 323.19 ms | 52.2% bf16 MFU | 1623287 tok/s
step 3038/19560 | loss 3.664842 (-1.33z)| norm 0.3423 (+2.41z)| lr 5.78e-04 | 322.98 ms | 52.3% bf16 MFU | 1623287 tok/s
step 3039/19560 | loss 3.693046 (-0.69z)| norm 0.3332 (+1.96z)| lr 5.78e-04 | 323.46 ms | 52.2% bf16 MFU | 1623167 tok/s
step 3040/19560 | loss 3.755420 (+0.75z)| norm 0.3121 (+1.03z)| lr 5.78e-04 | 323.38 ms | 52.2% bf16 MFU | 1623073 tok/s
step 3041/19560 | loss 3.708917 (-0.31z)| norm 0.3212 (+1.41z)| lr 5.77e-04 | 322.75 ms | 52.3% bf16 MFU | 1623142 tok/s
step 3042/19560 | loss 3.684100 (-0.91z)| norm 0.3114 (+1.05z)| lr 5.77e-04 | 322.94 ms | 52.3% bf16 MFU | 1623159 tok/s
step 3043/19560 | loss 3.652668 (-1.65z)| norm 0.2885 (+0.05z)| lr 5.77e-04 | 322.75 ms | 52.3% bf16 MFU | 1623222 tok/s
step 3044/19560 | loss 3.793298 (+1.72z)| norm 0.2838 (-0.15z)| lr 5.77e-04 | 323.05 ms | 52.2% bf16 MFU | 1623206 tok/s
step 3045/19560 | loss 3.741689 (+0.48z)| norm 0.3190 (+1.49z)| lr 5.77e-04 | 323.29 ms | 52.2% bf16 MFU | 1623131 tok/s
step 3046/19560 | loss 3.698350 (-0.55z)| norm 0.2966 (+0.44z)| lr 5.77e-04 | 322.70 ms | 52.3% bf16 MFU | 1623210 tok/s
step 3047/19560 | loss 3.679331 (-0.99z)| norm 0.2807 (-0.29z)| lr 5.77e-04 | 323.28 ms | 52.2% bf16 MFU | 1623140 tok/s
step 3048/19560 | loss 3.749581 (+0.68z)| norm 0.2641 (-1.08z)| lr 5.77e-04 | 322.59 ms | 52.3% bf16 MFU | 1623246 tok/s
step 3049/19560 | loss 3.765181 (+1.04z)| norm 0.2933 (+0.32z)| lr 5.77e-04 | 323.16 ms | 52.2% bf16 MFU | 1623202 tok/s
step 3050/19560 | loss 3.645940 (-1.77z)| norm 0.3165 (+1.42z)| lr 5.77e-04 | 323.36 ms | 52.2% bf16 MFU | 1623110 tok/s
step 3051/19560 | loss 3.731107 (+0.26z)| norm 0.2721 (-0.70z)| lr 5.77e-04 | 322.78 ms | 52.3% bf16 MFU | 1623168 tok/s
step 3052/19560 | loss 3.694185 (-0.62z)| norm 0.2880 (+0.07z)| lr 5.77e-04 | 322.63 ms | 52.3% bf16 MFU | 1623263 tok/s
step 3053/19560 | loss 3.698130 (-0.54z)| norm 0.2613 (-1.19z)| lr 5.77e-04 | 322.84 ms | 52.3% bf16 MFU | 1623300 tok/s
step 3054/19560 | loss 3.749548 (+0.69z)| norm 0.2548 (-1.49z)| lr 5.77e-04 | 322.76 ms | 52.3% bf16 MFU | 1623354 tok/s
step 3055/19560 | loss 3.655128 (-1.55z)| norm 0.2616 (-1.15z)| lr 5.77e-04 | 323.21 ms | 52.2% bf16 MFU | 1623292 tok/s
step 3056/19560 | loss 3.722019 (+0.04z)| norm 0.2623 (-1.11z)| lr 5.77e-04 | 322.69 ms | 52.3% bf16 MFU | 1623365 tok/s
step 3057/19560 | loss 3.679040 (-0.97z)| norm 0.2674 (-0.87z)| lr 5.77e-04 | 323.18 ms | 52.2% bf16 MFU | 1623310 tok/s
step 3058/19560 | loss 3.663565 (-1.34z)| norm 0.2673 (-0.89z)| lr 5.77e-04 | 323.14 ms | 52.2% bf16 MFU | 1623268 tok/s
step 3059/19560 | loss 3.743421 (+0.55z)| norm 0.2643 (-1.02z)| lr 5.77e-04 | 323.03 ms | 52.2% bf16 MFU | 1623255 tok/s
step 3060/19560 | loss 3.757103 (+0.86z)| norm 0.2542 (-1.48z)| lr 5.77e-04 | 323.04 ms | 52.2% bf16 MFU | 1623240 tok/s
step 3061/19560 | loss 3.709077 (-0.28z)| norm 0.2637 (-1.06z)| lr 5.77e-04 | 323.02 ms | 52.2% bf16 MFU | 1623234 tok/s
step 3062/19560 | loss 3.662332 (-1.37z)| norm 0.2713 (-0.70z)| lr 5.77e-04 | 323.05 ms | 52.2% bf16 MFU | 1623218 tok/s
step 3063/19560 | loss 3.711976 (-0.20z)| norm 0.2670 (-0.90z)| lr 5.77e-04 | 322.77 ms | 52.3% bf16 MFU | 1623274 tok/s
step 3064/19560 | loss 3.688640 (-0.75z)| norm 0.2660 (-0.96z)| lr 5.77e-04 | 323.45 ms | 52.2% bf16 MFU | 1623158 tok/s
step 3065/19560 | loss 3.722360 (+0.04z)| norm 0.2757 (-0.51z)| lr 5.77e-04 | 323.21 ms | 52.2% bf16 MFU | 1623107 tok/s
step 3066/19560 | loss 3.700021 (-0.49z)| norm 0.2810 (-0.27z)| lr 5.77e-04 | 322.75 ms | 52.3% bf16 MFU | 1623175 tok/s
step 3067/19560 | loss 3.708552 (-0.28z)| norm 0.6634 (+9.61z)| lr 5.77e-04 | 322.23 ms | 52.4% bf16 MFU | 1623370 tok/s
step 3068/19560 | loss 3.761899 (+0.98z)| norm 0.3724 (+2.08z)| lr 5.77e-04 | 323.29 ms | 52.2% bf16 MFU | 1623290 tok/s
step 3069/19560 | loss 3.707692 (-0.30z)| norm 0.3847 (+2.32z)| lr 5.77e-04 | 322.80 ms | 52.3% bf16 MFU | 1623334 tok/s
step 3070/19560 | loss 3.718183 (-0.03z)| norm 0.3910 (+2.40z)| lr 5.77e-04 | 323.16 ms | 52.2% bf16 MFU | 1623287 tok/s
step 3071/19560 | loss 3.696507 (-0.55z)| norm 0.3297 (+0.91z)| lr 5.77e-04 | 323.08 ms | 52.2% bf16 MFU | 1623262 tok/s
step 3072/19560 | loss 3.749359 (+0.76z)| norm 0.2897 (-0.05z)| lr 5.77e-04 | 322.16 ms | 52.4% bf16 MFU | 1623470 tok/s
step 3073/19560 | loss 3.734599 (+0.39z)| norm 0.3137 (+0.53z)| lr 5.77e-04 | 323.59 ms | 52.2% bf16 MFU | 1623308 tok/s
step 3074/19560 | loss 3.613250 (-2.54z)| norm 0.2811 (-0.25z)| lr 5.77e-04 | 323.54 ms | 52.2% bf16 MFU | 1623165 tok/s
step 3075/19560 | loss 3.737735 (+0.48z)| norm 0.2945 (+0.07z)| lr 5.77e-04 | 323.14 ms | 52.2% bf16 MFU | 1623131 tok/s
step 3076/19560 | loss 3.701699 (-0.39z)| norm 0.3158 (+0.58z)| lr 5.77e-04 | 322.41 ms | 52.3% bf16 MFU | 1623282 tok/s
step 3077/19560 | loss 3.703601 (-0.34z)| norm 0.3650 (+1.74z)| lr 5.77e-04 | 323.45 ms | 52.2% bf16 MFU | 1623163 tok/s
step 3078/19560 | loss 3.704483 (-0.31z)| norm 0.3143 (+0.52z)| lr 5.77e-04 | 323.31 ms | 52.2% bf16 MFU | 1623085 tok/s
step 3079/19560 | loss 3.784996 (+1.65z)| norm 0.2720 (-0.49z)| lr 5.77e-04 | 323.09 ms | 52.2% bf16 MFU | 1623067 tok/s
step 3080/19560 | loss 3.735521 (+0.45z)| norm 0.2980 (+0.12z)| lr 5.77e-04 | 323.27 ms | 52.2% bf16 MFU | 1623006 tok/s
step 3081/19560 | loss 3.770789 (+1.29z)| norm 0.2720 (-0.50z)| lr 5.77e-04 | 323.38 ms | 52.2% bf16 MFU | 1622919 tok/s
step 3082/19560 | loss 3.695321 (-0.55z)| norm 0.2708 (-0.53z)| lr 5.77e-04 | 323.45 ms | 52.2% bf16 MFU | 1622820 tok/s
step 3083/19560 | loss 3.755412 (+0.90z)| norm 0.2785 (-0.34z)| lr 5.77e-04 | 323.08 ms | 52.2% bf16 MFU | 1622819 tok/s
step 3084/19560 | loss 3.798028 (+1.93z)| norm 0.2900 (-0.07z)| lr 5.77e-04 | 322.01 ms | 52.4% bf16 MFU | 1623085 tok/s
step 3085/19560 | loss 3.737390 (+0.45z)| norm 0.2747 (-0.44z)| lr 5.77e-04 | 323.06 ms | 52.2% bf16 MFU | 1623075 tok/s
step 3086/19560 | loss 3.766998 (+1.18z)| norm 0.2637 (-0.69z)| lr 5.77e-04 | 322.61 ms | 52.3% bf16 MFU | 1623177 tok/s
step 3087/19560 | loss 3.704152 (-0.37z)| norm 0.3163 (+0.56z)| lr 5.77e-04 | 322.76 ms | 52.3% bf16 MFU | 1623239 tok/s
step 3088/19560 | loss 3.671156 (-1.17z)| norm 0.3130 (+0.48z)| lr 5.77e-04 | 322.60 ms | 52.3% bf16 MFU | 1623336 tok/s
step 3089/19560 | loss 3.675415 (-1.07z)| norm 0.2629 (-0.72z)| lr 5.77e-04 | 322.68 ms | 52.3% bf16 MFU | 1623408 tok/s
step 3090/19560 | loss 3.676587 (-1.05z)| norm 0.2541 (-0.91z)| lr 5.77e-04 | 322.97 ms | 52.3% bf16 MFU | 1623405 tok/s
step 3091/19560 | loss 3.719886 (+0.01z)| norm 0.2609 (-0.74z)| lr 5.77e-04 | 322.65 ms | 52.3% bf16 MFU | 1623482 tok/s
step 3092/19560 | loss 3.667376 (-1.27z)| norm 0.2628 (-0.69z)| lr 5.77e-04 | 323.00 ms | 52.3% bf16 MFU | 1623466 tok/s
step 3093/19560 | loss 3.703721 (-0.38z)| norm 0.2686 (-0.55z)| lr 5.76e-04 | 322.99 ms | 52.3% bf16 MFU | 1623456 tok/s
step 3094/19560 | loss 3.731722 (+0.29z)| norm 0.2757 (-0.38z)| lr 5.76e-04 | 322.88 ms | 52.3% bf16 MFU | 1623472 tok/s
step 3095/19560 | loss 3.672190 (-1.18z)| norm 0.2703 (-0.51z)| lr 5.76e-04 | 322.90 ms | 52.3% bf16 MFU | 1623481 tok/s
step 3096/19560 | loss 3.642269 (-1.89z)| norm 0.2596 (-0.76z)| lr 5.76e-04 | 323.27 ms | 52.2% bf16 MFU | 1623399 tok/s
step 3097/19560 | loss 3.730669 (+0.29z)| norm 0.2562 (-0.82z)| lr 5.76e-04 | 322.95 ms | 52.3% bf16 MFU | 1623400 tok/s
step 3098/19560 | loss 3.683877 (-0.85z)| norm 0.2557 (-0.83z)| lr 5.76e-04 | 323.03 ms | 52.2% bf16 MFU | 1623381 tok/s
step 3099/19560 | loss 3.689938 (-0.70z)| norm 0.2509 (-0.93z)| lr 5.76e-04 | 323.21 ms | 52.2% bf16 MFU | 1623320 tok/s
step 3100/19560 | loss 3.673412 (-1.09z)| norm 0.2462 (-1.03z)| lr 5.76e-04 | 323.43 ms | 52.2% bf16 MFU | 1623205 tok/s
step 3101/19560 | loss 3.674001 (-1.08z)| norm 0.2529 (-0.86z)| lr 5.76e-04 | 323.19 ms | 52.2% bf16 MFU | 1623157 tok/s
step 3102/19560 | loss 3.712869 (-0.13z)| norm 0.2407 (-1.13z)| lr 5.76e-04 | 323.05 ms | 52.2% bf16 MFU | 1623145 tok/s
step 3103/19560 | loss 3.689792 (-0.68z)| norm 0.2694 (-0.46z)| lr 5.76e-04 | 323.18 ms | 52.2% bf16 MFU | 1623101 tok/s
step 3104/19560 | loss 3.741635 (+0.60z)| norm 0.2829 (-0.14z)| lr 5.76e-04 | 323.50 ms | 52.2% bf16 MFU | 1622979 tok/s
step 3105/19560 | loss 3.668437 (-1.19z)| norm 0.3068 (+0.40z)| lr 5.76e-04 | 322.80 ms | 52.3% bf16 MFU | 1623038 tok/s
step 3106/19560 | loss 3.704102 (-0.32z)| norm 0.3210 (+0.73z)| lr 5.76e-04 | 323.21 ms | 52.2% bf16 MFU | 1622992 tok/s
step 3107/19560 | loss 3.696824 (-0.49z)| norm 0.2829 (-0.16z)| lr 5.76e-04 | 323.16 ms | 52.2% bf16 MFU | 1622960 tok/s
step 3108/19560 | loss 3.689852 (-0.66z)| norm 0.2882 (-0.04z)| lr 5.76e-04 | 323.16 ms | 52.2% bf16 MFU | 1622932 tok/s
step 3109/19560 | loss 3.687230 (-0.71z)| norm 0.2737 (-0.38z)| lr 5.76e-04 | 323.64 ms | 52.1% bf16 MFU | 1622784 tok/s
step 3110/19560 | loss 3.741080 (+0.63z)| norm 0.2857 (-0.10z)| lr 5.76e-04 | 323.19 ms | 52.2% bf16 MFU | 1622755 tok/s
step 3111/19560 | loss 3.789825 (+1.81z)| norm 0.3749 (+1.94z)| lr 5.76e-04 | 322.76 ms | 52.3% bf16 MFU | 1622838 tok/s
step 3112/19560 | loss 3.711464 (-0.12z)| norm 0.2779 (-0.29z)| lr 5.76e-04 | 323.24 ms | 52.2% bf16 MFU | 1622795 tok/s
step 3113/19560 | loss 3.678874 (-0.93z)| norm 0.2669 (-0.54z)| lr 5.76e-04 | 323.11 ms | 52.2% bf16 MFU | 1622788 tok/s
step 3114/19560 | loss 3.774331 (+1.42z)| norm 0.2905 (-0.01z)| lr 5.76e-04 | 322.34 ms | 52.4% bf16 MFU | 1622975 tok/s
step 3115/19560 | loss 3.697179 (-0.49z)| norm 0.3158 (+0.57z)| lr 5.76e-04 | 323.34 ms | 52.2% bf16 MFU | 1622900 tok/s
step 3116/19560 | loss 3.664237 (-1.29z)| norm 0.2974 (+0.14z)| lr 5.76e-04 | 323.00 ms | 52.3% bf16 MFU | 1622913 tok/s
step 3117/19560 | loss 3.708980 (-0.17z)| norm 0.2786 (-0.29z)| lr 5.76e-04 | 323.10 ms | 52.2% bf16 MFU | 1622901 tok/s
step 3118/19560 | loss 3.646459 (-1.74z)| norm 0.2753 (-0.37z)| lr 5.76e-04 | 323.02 ms | 52.2% bf16 MFU | 1622909 tok/s
step 3119/19560 | loss 3.663316 (-1.29z)| norm 0.3010 (+0.22z)| lr 5.76e-04 | 323.11 ms | 52.2% bf16 MFU | 1622895 tok/s
step 3120/19560 | loss 3.743481 (+0.76z)| norm 0.3324 (+0.93z)| lr 5.76e-04 | 322.91 ms | 52.3% bf16 MFU | 1622931 tok/s
step 3121/19560 | loss 3.851073 (+3.33z)| norm 0.3438 (+1.19z)| lr 5.76e-04 | 322.57 ms | 52.3% bf16 MFU | 1623052 tok/s
step 3122/19560 | loss 3.697652 (-0.41z)| norm 0.3249 (+0.75z)| lr 5.76e-04 | 323.12 ms | 52.2% bf16 MFU | 1623028 tok/s
step 3123/19560 | loss 3.638912 (-1.81z)| norm 0.2948 (+0.06z)| lr 5.76e-04 | 322.87 ms | 52.3% bf16 MFU | 1623068 tok/s
step 3124/19560 | loss 3.701250 (-0.30z)| norm 0.2772 (-0.33z)| lr 5.76e-04 | 323.21 ms | 52.2% bf16 MFU | 1623021 tok/s
step 3125/19560 | loss 3.760973 (+1.12z)| norm 0.2914 (-0.01z)| lr 5.76e-04 | 323.24 ms | 52.2% bf16 MFU | 1622969 tok/s
step 3126/19560 | loss 3.730271 (+0.38z)| norm 0.2886 (-0.07z)| lr 5.76e-04 | 322.89 ms | 52.3% bf16 MFU | 1623009 tok/s
step 3127/19560 | loss 3.649679 (-1.52z)| norm 0.2605 (-0.70z)| lr 5.76e-04 | 322.57 ms | 52.3% bf16 MFU | 1623125 tok/s
step 3128/19560 | loss 3.838799 (+2.87z)| norm 0.2933 (+0.05z)| lr 5.76e-04 | 322.85 ms | 52.3% bf16 MFU | 1623164 tok/s
step 3129/19560 | loss 3.704227 (-0.26z)| norm 0.2974 (+0.14z)| lr 5.76e-04 | 322.90 ms | 52.3% bf16 MFU | 1623190 tok/s
step 3130/19560 | loss 3.703085 (-0.28z)| norm 0.2558 (-0.82z)| lr 5.76e-04 | 322.62 ms | 52.3% bf16 MFU | 1623284 tok/s
step 3131/19560 | loss 3.767364 (+1.21z)| norm 0.2671 (-0.56z)| lr 5.76e-04 | 323.09 ms | 52.2% bf16 MFU | 1623257 tok/s
step 3132/19560 | loss 3.715461 (-0.00z)| norm 0.2716 (-0.46z)| lr 5.76e-04 | 322.57 ms | 52.3% bf16 MFU | 1623360 tok/s
step 3133/19560 | loss 3.700706 (-0.34z)| norm 0.2672 (-0.56z)| lr 5.76e-04 | 322.57 ms | 52.3% bf16 MFU | 1623458 tok/s
step 3134/19560 | loss 3.793281 (+1.79z)| norm 0.3018 (+0.24z)| lr 5.76e-04 | 323.06 ms | 52.2% bf16 MFU | 1623429 tok/s
step 3135/19560 | loss 3.687685 (-0.64z)| norm 0.2739 (-0.40z)| lr 5.76e-04 | 322.66 ms | 52.3% bf16 MFU | 1623501 tok/s
step 3136/19560 | loss 3.679383 (-0.82z)| norm 0.3006 (+0.21z)| lr 5.76e-04 | 322.55 ms | 52.3% bf16 MFU | 1623599 tok/s
step 3137/19560 | loss 3.696192 (-0.42z)| norm 0.3152 (+0.54z)| lr 5.76e-04 | 322.87 ms | 52.3% bf16 MFU | 1623612 tok/s
step 3138/19560 | loss 3.657652 (-1.30z)| norm 0.3327 (+0.93z)| lr 5.76e-04 | 322.55 ms | 52.3% bf16 MFU | 1623703 tok/s
step 3139/19560 | loss 3.699987 (-0.30z)| norm 0.3231 (+0.70z)| lr 5.76e-04 | 323.06 ms | 52.2% bf16 MFU | 1623661 tok/s
step 3140/19560 | loss 3.668320 (-1.03z)| norm 0.3270 (+0.78z)| lr 5.76e-04 | 322.80 ms | 52.3% bf16 MFU | 1623689 tok/s
step 3141/19560 | loss 3.703471 (-0.21z)| norm 0.2996 (+0.15z)| lr 5.76e-04 | 323.06 ms | 52.2% bf16 MFU | 1623649 tok/s
step 3142/19560 | loss 3.741819 (+0.70z)| norm 0.3137 (+0.47z)| lr 5.76e-04 | 322.79 ms | 52.3% bf16 MFU | 1623678 tok/s
step 3143/19560 | loss 3.683063 (-0.67z)| norm 0.3126 (+0.44z)| lr 5.76e-04 | 323.00 ms | 52.3% bf16 MFU | 1623654 tok/s
step 3144/19560 | loss 3.666512 (-1.05z)| norm 0.2970 (+0.08z)| lr 5.76e-04 | 323.25 ms | 52.2% bf16 MFU | 1623568 tok/s
step 3145/19560 | loss 3.683398 (-0.65z)| norm 0.3004 (+0.16z)| lr 5.75e-04 | 322.62 ms | 52.3% bf16 MFU | 1623644 tok/s
step 3146/19560 | loss 3.729162 (+0.43z)| norm 0.2642 (-0.66z)| lr 5.75e-04 | 323.21 ms | 52.2% bf16 MFU | 1623568 tok/s
step 3147/19560 | loss 3.718845 (+0.18z)| norm 0.2923 (-0.02z)| lr 5.75e-04 | 323.13 ms | 52.2% bf16 MFU | 1623517 tok/s
step 3148/19560 | loss 3.668958 (-0.99z)| norm 0.2715 (-0.50z)| lr 5.75e-04 | 322.53 ms | 52.3% bf16 MFU | 1623618 tok/s
step 3149/19560 | loss 3.681247 (-0.69z)| norm 0.2512 (-0.96z)| lr 5.75e-04 | 322.84 ms | 52.3% bf16 MFU | 1623636 tok/s
step 3150/19560 | loss 3.656999 (-1.24z)| norm 0.2677 (-0.58z)| lr 5.75e-04 | 322.88 ms | 52.3% bf16 MFU | 1623644 tok/s
step 3151/19560 | loss 3.814508 (+2.39z)| norm 0.2910 (-0.04z)| lr 5.75e-04 | 322.91 ms | 52.3% bf16 MFU | 1623643 tok/s
step 3152/19560 | loss 3.620931 (-2.00z)| norm 0.2708 (-0.50z)| lr 5.75e-04 | 322.72 ms | 52.3% bf16 MFU | 1623691 tok/s
step 3153/19560 | loss 3.682433 (-0.61z)| norm 0.2682 (-0.55z)| lr 5.75e-04 | 322.91 ms | 52.3% bf16 MFU | 1623687 tok/s
step 3154/19560 | loss 3.735785 (+0.59z)| norm 0.2772 (-0.33z)| lr 5.75e-04 | 323.03 ms | 52.2% bf16 MFU | 1623654 tok/s
step 3155/19560 | loss 3.689265 (-0.46z)| norm 0.2682 (-0.53z)| lr 5.75e-04 | 322.56 ms | 52.3% bf16 MFU | 1623741 tok/s
step 3156/19560 | loss 3.713635 (+0.12z)| norm 0.2919 (+0.01z)| lr 5.75e-04 | 323.25 ms | 52.2% bf16 MFU | 1623651 tok/s
step 3157/19560 | loss 3.645545 (-1.46z)| norm 0.2667 (-0.56z)| lr 5.75e-04 | 322.69 ms | 52.3% bf16 MFU | 1623706 tok/s
step 3158/19560 | loss 3.681895 (-0.59z)| norm 0.2873 (-0.10z)| lr 5.75e-04 | 322.92 ms | 52.3% bf16 MFU | 1623700 tok/s
step 3159/19560 | loss 3.661126 (-1.07z)| norm 0.3038 (+0.27z)| lr 5.75e-04 | 323.54 ms | 52.2% bf16 MFU | 1623538 tok/s
step 3160/19560 | loss 3.761309 (+1.26z)| norm 0.2462 (-1.03z)| lr 5.75e-04 | 322.40 ms | 52.3% bf16 MFU | 1623672 tok/s
step 3161/19560 | loss 3.708754 (+0.02z)| norm 0.2878 (-0.09z)| lr 5.75e-04 | 323.31 ms | 52.2% bf16 MFU | 1623570 tok/s
step 3162/19560 | loss 3.679554 (-0.65z)| norm 0.2752 (-0.38z)| lr 5.75e-04 | 322.83 ms | 52.3% bf16 MFU | 1623595 tok/s
step 3163/19560 | loss 3.703671 (-0.10z)| norm 0.2823 (-0.21z)| lr 5.75e-04 | 322.71 ms | 52.3% bf16 MFU | 1623647 tok/s
step 3164/19560 | loss 3.674170 (-0.79z)| norm 0.2716 (-0.45z)| lr 5.75e-04 | 322.94 ms | 52.3% bf16 MFU | 1623638 tok/s
step 3165/19560 | loss 3.690962 (-0.39z)| norm 0.2896 (-0.04z)| lr 5.75e-04 | 322.73 ms | 52.3% bf16 MFU | 1623684 tok/s
step 3166/19560 | loss 3.669657 (-0.89z)| norm 0.3024 (+0.26z)| lr 5.75e-04 | 322.75 ms | 52.3% bf16 MFU | 1623721 tok/s
step 3167/19560 | loss 3.695136 (-0.29z)| norm 0.3007 (+0.23z)| lr 5.75e-04 | 322.84 ms | 52.3% bf16 MFU | 1623734 tok/s
step 3168/19560 | loss 3.708281 (+0.02z)| norm 0.3105 (+0.45z)| lr 5.75e-04 | 322.88 ms | 52.3% bf16 MFU | 1623736 tok/s
step 3169/19560 | loss 3.655650 (-1.21z)| norm 0.2939 (+0.08z)| lr 5.75e-04 | 323.23 ms | 52.2% bf16 MFU | 1623650 tok/s
step 3170/19560 | loss 3.691179 (-0.37z)| norm 0.2857 (-0.11z)| lr 5.75e-04 | 322.33 ms | 52.4% bf16 MFU | 1623794 tok/s
step 3171/19560 | loss 3.678432 (-0.68z)| norm 0.3284 (+0.87z)| lr 5.75e-04 | 323.38 ms | 52.2% bf16 MFU | 1623667 tok/s
step 3172/19560 | loss 3.665226 (-0.98z)| norm 0.2892 (-0.04z)| lr 5.75e-04 | 322.38 ms | 52.4% bf16 MFU | 1623799 tok/s
step 3173/19560 | loss 3.740482 (+0.83z)| norm 0.2845 (-0.14z)| lr 5.75e-04 | 323.04 ms | 52.2% bf16 MFU | 1623758 tok/s
step 3174/19560 | loss 3.696718 (-0.22z)| norm 0.2976 (+0.16z)| lr 5.75e-04 | 322.30 ms | 52.4% bf16 MFU | 1623906 tok/s
step 3175/19560 | loss 3.595816 (-2.56z)| norm 0.2573 (-0.76z)| lr 5.75e-04 | 322.72 ms | 52.3% bf16 MFU | 1623939 tok/s
step 3176/19560 | loss 3.634440 (-1.63z)| norm 0.2912 (+0.02z)| lr 5.75e-04 | 323.09 ms | 52.2% bf16 MFU | 1623880 tok/s
step 3177/19560 | loss 3.663131 (-0.95z)| norm 0.2839 (-0.15z)| lr 5.75e-04 | 323.07 ms | 52.2% bf16 MFU | 1623828 tok/s
step 3178/19560 | loss 3.691218 (-0.30z)| norm 0.2721 (-0.41z)| lr 5.75e-04 | 322.31 ms | 52.4% bf16 MFU | 1623968 tok/s
step 3179/19560 | loss 3.693584 (-0.24z)| norm 0.2671 (-0.53z)| lr 5.75e-04 | 322.83 ms | 52.3% bf16 MFU | 1623972 tok/s
step 3180/19560 | loss 3.687956 (-0.37z)| norm 0.2707 (-0.44z)| lr 5.75e-04 | 322.74 ms | 52.3% bf16 MFU | 1623998 tok/s
step 3181/19560 | loss 3.680115 (-0.55z)| norm 0.2802 (-0.23z)| lr 5.75e-04 | 323.08 ms | 52.2% bf16 MFU | 1623938 tok/s
step 3182/19560 | loss 3.704582 (+0.03z)| norm 0.3015 (+0.25z)| lr 5.75e-04 | 322.85 ms | 52.3% bf16 MFU | 1623937 tok/s
step 3183/19560 | loss 3.657591 (-1.08z)| norm 0.2666 (-0.55z)| lr 5.75e-04 | 322.48 ms | 52.3% bf16 MFU | 1624030 tok/s
step 3184/19560 | loss 3.686415 (-0.39z)| norm 0.2514 (-0.90z)| lr 5.75e-04 | 323.16 ms | 52.2% bf16 MFU | 1623946 tok/s
step 3185/19560 | loss 3.632821 (-1.63z)| norm 0.2794 (-0.26z)| lr 5.75e-04 | 322.29 ms | 52.4% bf16 MFU | 1624086 tok/s
step 3186/19560 | loss 3.674569 (-0.66z)| norm 0.2775 (-0.30z)| lr 5.75e-04 | 323.36 ms | 52.2% bf16 MFU | 1623951 tok/s
step 3187/19560 | loss 3.711024 (+0.20z)| norm 0.2877 (-0.07z)| lr 5.75e-04 | 322.38 ms | 52.4% bf16 MFU | 1624068 tok/s
step 3188/19560 | loss 3.691062 (-0.26z)| norm 0.3095 (+0.42z)| lr 5.75e-04 | 322.68 ms | 52.3% bf16 MFU | 1624104 tok/s
step 3189/19560 | loss 3.699739 (-0.05z)| norm 0.2802 (-0.26z)| lr 5.75e-04 | 323.15 ms | 52.2% bf16 MFU | 1624020 tok/s
step 3190/19560 | loss 3.678455 (-0.56z)| norm 0.3144 (+0.53z)| lr 5.75e-04 | 322.95 ms | 52.3% bf16 MFU | 1623991 tok/s
step 3191/19560 | loss 3.715856 (+0.33z)| norm 0.3324 (+0.93z)| lr 5.75e-04 | 322.76 ms | 52.3% bf16 MFU | 1624012 tok/s
step 3192/19560 | loss 3.751519 (+1.16z)| norm 0.3127 (+0.47z)| lr 5.75e-04 | 322.80 ms | 52.3% bf16 MFU | 1624020 tok/s
step 3193/19560 | loss 3.712334 (+0.23z)| norm 0.2874 (-0.12z)| lr 5.75e-04 | 322.78 ms | 52.3% bf16 MFU | 1624033 tok/s
step 3194/19560 | loss 3.664166 (-0.89z)| norm 0.2553 (-0.86z)| lr 5.75e-04 | 322.85 ms | 52.3% bf16 MFU | 1624029 tok/s
step 3195/19560 | loss 3.609713 (-2.12z)| norm 0.2793 (-0.36z)| lr 5.74e-04 | 322.53 ms | 52.3% bf16 MFU | 1624105 tok/s
step 3196/19560 | loss 3.707566 (+0.15z)| norm 0.2843 (-0.16z)| lr 5.74e-04 | 322.82 ms | 52.3% bf16 MFU | 1624104 tok/s
step 3197/19560 | loss 3.639666 (-1.41z)| norm 0.2453 (-1.63z)| lr 5.74e-04 | 322.50 ms | 52.3% bf16 MFU | 1624184 tok/s
step 3198/19560 | loss 3.713089 (+0.29z)| norm 0.2436 (-1.74z)| lr 5.74e-04 | 322.86 ms | 52.3% bf16 MFU | 1624168 tok/s
step 3199/19560 | loss 3.645424 (-1.25z)| norm 0.2652 (-0.86z)| lr 5.74e-04 | 322.61 ms | 52.3% bf16 MFU | 1624216 tok/s
step 3200/19560 | loss 3.682593 (-0.39z)| norm 0.2506 (-1.43z)| lr 5.74e-04 | 322.86 ms | 52.3% bf16 MFU | 1624199 tok/s
step 3201/19560 | loss 3.631286 (-1.55z)| norm 0.2334 (-2.08z)| lr 5.74e-04 | 322.78 ms | 52.3% bf16 MFU | 1624203 tok/s
step 3202/19560 | loss 3.672474 (-0.62z)| norm 0.2414 (-1.72z)| lr 5.74e-04 | 322.88 ms | 52.3% bf16 MFU | 1624183 tok/s
step 3203/19560 | loss 3.700020 (+0.03z)| norm 0.2450 (-1.55z)| lr 5.74e-04 | 322.88 ms | 52.3% bf16 MFU | 1624162 tok/s
step 3204/19560 | loss 3.725207 (+0.61z)| norm 0.2829 (-0.05z)| lr 5.74e-04 | 322.79 ms | 52.3% bf16 MFU | 1624165 tok/s
step 3205/19560 | loss 3.726603 (+0.64z)| norm 0.2872 (+0.15z)| lr 5.74e-04 | 322.57 ms | 52.3% bf16 MFU | 1624224 tok/s
step 3206/19560 | loss 3.667387 (-0.73z)| norm 0.2800 (-0.13z)| lr 5.74e-04 | 322.93 ms | 52.3% bf16 MFU | 1624188 tok/s
step 3207/19560 | loss 3.667254 (-0.72z)| norm 0.2905 (+0.30z)| lr 5.74e-04 | 323.03 ms | 52.2% bf16 MFU | 1624130 tok/s
step 3208/19560 | loss 3.727790 (+0.70z)| norm 0.2895 (+0.26z)| lr 5.74e-04 | 322.85 ms | 52.3% bf16 MFU | 1624121 tok/s
step 3209/19560 | loss 3.684154 (-0.31z)| norm 0.2826 (-0.03z)| lr 5.74e-04 | 322.89 ms | 52.3% bf16 MFU | 1624101 tok/s
step 3210/19560 | loss 3.692698 (-0.11z)| norm 0.2786 (-0.20z)| lr 5.74e-04 | 323.10 ms | 52.2% bf16 MFU | 1624031 tok/s
step 3211/19560 | loss 3.621390 (-1.77z)| norm 0.2673 (-0.67z)| lr 5.74e-04 | 322.80 ms | 52.3% bf16 MFU | 1624039 tok/s
step 3212/19560 | loss 3.691805 (-0.09z)| norm 0.2384 (-1.83z)| lr 5.74e-04 | 322.87 ms | 52.3% bf16 MFU | 1624030 tok/s
step 3213/19560 | loss 3.728368 (+0.81z)| norm 0.2556 (-1.12z)| lr 5.74e-04 | 322.66 ms | 52.3% bf16 MFU | 1624073 tok/s
step 3214/19560 | loss 3.701577 (+0.17z)| norm 0.2359 (-1.89z)| lr 5.74e-04 | 322.78 ms | 52.3% bf16 MFU | 1624084 tok/s
step 3215/19560 | loss 3.642787 (-1.26z)| norm 0.2546 (-1.12z)| lr 5.74e-04 | 322.92 ms | 52.3% bf16 MFU | 1624058 tok/s
step 3216/19560 | loss 3.680385 (-0.34z)| norm 0.2608 (-0.85z)| lr 5.74e-04 | 323.03 ms | 52.2% bf16 MFU | 1624008 tok/s
step 3217/19560 | loss 3.756570 (+1.50z)| norm 0.2978 (+0.64z)| lr 5.74e-04 | 322.82 ms | 52.3% bf16 MFU | 1624011 tok/s
step 3218/19560 | loss 3.683888 (-0.27z)| norm 0.3179 (+1.44z)| lr 5.74e-04 | 322.78 ms | 52.3% bf16 MFU | 1624024 tok/s
step 3219/19560 | loss 3.689615 (-0.13z)| norm 0.3216 (+1.56z)| lr 5.74e-04 | 322.97 ms | 52.3% bf16 MFU | 1623989 tok/s
step 3220/19560 | loss 3.662769 (-0.78z)| norm 0.2888 (+0.23z)| lr 5.74e-04 | 323.08 ms | 52.2% bf16 MFU | 1623929 tok/s
step 3221/19560 | loss 3.700569 (+0.14z)| norm 0.2489 (-1.37z)| lr 5.74e-04 | 323.33 ms | 52.2% bf16 MFU | 1623809 tok/s
step 3222/19560 | loss 3.715732 (+0.51z)| norm 0.2649 (-0.72z)| lr 5.74e-04 | 322.72 ms | 52.3% bf16 MFU | 1623848 tok/s
step 3223/19560 | loss 3.688141 (-0.16z)| norm 0.2863 (+0.13z)| lr 5.74e-04 | 322.69 ms | 52.3% bf16 MFU | 1623892 tok/s
step 3224/19560 | loss 3.654345 (-0.99z)| norm 0.2856 (+0.09z)| lr 5.74e-04 | 322.69 ms | 52.3% bf16 MFU | 1623935 tok/s
step 3225/19560 | loss 3.696919 (+0.06z)| norm 0.2824 (-0.04z)| lr 5.74e-04 | 323.36 ms | 52.2% bf16 MFU | 1623807 tok/s
step 3226/19560 | loss 3.668742 (-0.63z)| norm 0.2755 (-0.33z)| lr 5.74e-04 | 323.50 ms | 52.2% bf16 MFU | 1623650 tok/s
step 3227/19560 | loss 3.695070 (+0.01z)| norm 0.3123 (+1.15z)| lr 5.74e-04 | 322.49 ms | 52.3% bf16 MFU | 1623756 tok/s
step 3228/19560 | loss 3.723274 (+0.70z)| norm 0.3223 (+1.53z)| lr 5.74e-04 | 322.62 ms | 52.3% bf16 MFU | 1623822 tok/s
step 3229/19560 | loss 3.613341 (-1.96z)| norm 0.3216 (+1.48z)| lr 5.74e-04 | 322.70 ms | 52.3% bf16 MFU | 1623864 tok/s
step 3230/19560 | loss 3.764103 (+1.66z)| norm 0.3207 (+1.43z)| lr 5.74e-04 | 323.39 ms | 52.2% bf16 MFU | 1623731 tok/s
step 3231/19560 | loss 3.759909 (+1.53z)| norm 0.2970 (+0.45z)| lr 5.74e-04 | 322.98 ms | 52.3% bf16 MFU | 1623708 tok/s
step 3232/19560 | loss 3.706000 (+0.26z)| norm 0.2940 (+0.32z)| lr 5.74e-04 | 323.03 ms | 52.2% bf16 MFU | 1623675 tok/s
step 3233/19560 | loss 3.689104 (-0.15z)| norm 0.2726 (-0.54z)| lr 5.74e-04 | 323.22 ms | 52.2% bf16 MFU | 1623597 tok/s
step 3234/19560 | loss 3.747208 (+1.22z)| norm 0.2732 (-0.51z)| lr 5.74e-04 | 323.01 ms | 52.2% bf16 MFU | 1623574 tok/s
step 3235/19560 | loss 3.644205 (-1.20z)| norm 0.2835 (-0.08z)| lr 5.74e-04 | 323.33 ms | 52.2% bf16 MFU | 1623472 tok/s
step 3236/19560 | loss 3.755828 (+1.41z)| norm 0.2781 (-0.30z)| lr 5.74e-04 | 322.58 ms | 52.3% bf16 MFU | 1623563 tok/s
step 3237/19560 | loss 3.729925 (+0.79z)| norm 0.2985 (+0.53z)| lr 5.74e-04 | 322.65 ms | 52.3% bf16 MFU | 1623633 tok/s
step 3238/19560 | loss 3.794547 (+2.25z)| norm 0.3131 (+1.12z)| lr 5.74e-04 | 323.49 ms | 52.2% bf16 MFU | 1623487 tok/s
step 3239/19560 | loss 3.729255 (+0.78z)| norm 0.2598 (-1.09z)| lr 5.74e-04 | 323.11 ms | 52.2% bf16 MFU | 1623445 tok/s
step 3240/19560 | loss 3.701605 (+0.13z)| norm 0.2714 (-0.58z)| lr 5.74e-04 | 322.71 ms | 52.3% bf16 MFU | 1623504 tok/s
step 3241/19560 | loss 3.666129 (-0.69z)| norm 0.3031 (+0.77z)| lr 5.74e-04 | 322.72 ms | 52.3% bf16 MFU | 1623560 tok/s
step 3242/19560 | loss 3.668031 (-0.63z)| norm 0.3104 (+1.08z)| lr 5.74e-04 | 323.20 ms | 52.2% bf16 MFU | 1623491 tok/s
step 3243/19560 | loss 3.696621 (+0.04z)| norm 0.2985 (+0.57z)| lr 5.74e-04 | 322.50 ms | 52.3% bf16 MFU | 1623602 tok/s
step 3244/19560 | loss 3.711979 (+0.39z)| norm 0.2989 (+0.59z)| lr 5.73e-04 | 322.98 ms | 52.3% bf16 MFU | 1623586 tok/s
step 3245/19560 | loss 3.775847 (+1.86z)| norm 0.2633 (-0.94z)| lr 5.73e-04 | 322.87 ms | 52.3% bf16 MFU | 1623597 tok/s
step 3246/19560 | loss 3.706805 (+0.25z)| norm 0.2680 (-0.73z)| lr 5.73e-04 | 322.75 ms | 52.3% bf16 MFU | 1623639 tok/s
step 3247/19560 | loss 3.709409 (+0.30z)| norm 0.2639 (-0.90z)| lr 5.73e-04 | 323.42 ms | 52.2% bf16 MFU | 1623512 tok/s
step 3248/19560 | loss 3.634936 (-1.42z)| norm 0.2419 (-1.82z)| lr 5.73e-04 | 322.98 ms | 52.3% bf16 MFU | 1623501 tok/s
step 3249/19560 | loss 3.740180 (+1.11z)| norm 0.2559 (-1.21z)| lr 5.73e-04 | 323.46 ms | 52.2% bf16 MFU | 1623370 tok/s
step 3250/19560 | loss 3.688173 (-0.16z)| norm 0.2729 (-0.45z)| lr 5.73e-04 | 322.96 ms | 52.3% bf16 MFU | 1623370 tok/s
val loss 3.678645
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2683/10042 = 0.267178
step 3251/19560 | loss 3.696390 (+0.03z)| norm 0.2589 (-1.05z)| lr 5.73e-04 | 322.11 ms | 52.4% bf16 MFU | 1623586 tok/s
step 3252/19560 | loss 3.702756 (+0.18z)| norm 0.2548 (-1.22z)| lr 5.73e-04 | 323.02 ms | 52.2% bf16 MFU | 1623561 tok/s
step 3253/19560 | loss 3.764822 (+1.72z)| norm 0.4818 (+6.91z)| lr 5.73e-04 | 323.66 ms | 52.1% bf16 MFU | 1623376 tok/s
step 3254/19560 | loss 3.742220 (+1.15z)| norm 0.3282 (+1.52z)| lr 5.73e-04 | 323.36 ms | 52.2% bf16 MFU | 1623275 tok/s
step 3255/19560 | loss 3.713723 (+0.44z)| norm 0.2985 (+0.48z)| lr 5.73e-04 | 323.22 ms | 52.2% bf16 MFU | 1623216 tok/s
step 3256/19560 | loss 3.655802 (-1.01z)| norm 0.2845 (+0.00z)| lr 5.73e-04 | 323.51 ms | 52.2% bf16 MFU | 1623088 tok/s
step 3257/19560 | loss 3.627860 (-1.70z)| norm 0.2892 (+0.17z)| lr 5.73e-04 | 323.49 ms | 52.2% bf16 MFU | 1622969 tok/s
step 3258/19560 | loss 3.671021 (-0.58z)| norm 0.2831 (-0.06z)| lr 5.73e-04 | 323.51 ms | 52.2% bf16 MFU | 1622851 tok/s
step 3259/19560 | loss 3.824357 (+3.25z)| norm 0.2858 (+0.04z)| lr 5.73e-04 | 323.41 ms | 52.2% bf16 MFU | 1622765 tok/s
step 3260/19560 | loss 3.655288 (-0.95z)| norm 0.2963 (+0.40z)| lr 5.73e-04 | 323.15 ms | 52.2% bf16 MFU | 1622748 tok/s
step 3261/19560 | loss 3.693375 (-0.01z)| norm 0.2779 (-0.25z)| lr 5.73e-04 | 323.53 ms | 52.2% bf16 MFU | 1622638 tok/s
step 3262/19560 | loss 3.683002 (-0.25z)| norm 0.2697 (-0.53z)| lr 5.73e-04 | 322.96 ms | 52.3% bf16 MFU | 1622676 tok/s
step 3263/19560 | loss 3.783340 (+2.24z)| norm 0.3003 (+0.53z)| lr 5.73e-04 | 323.82 ms | 52.1% bf16 MFU | 1622495 tok/s
step 3264/19560 | loss 3.681397 (-0.30z)| norm 0.2974 (+0.43z)| lr 5.73e-04 | 323.26 ms | 52.2% bf16 MFU | 1622463 tok/s
step 3265/19560 | loss 3.689314 (-0.10z)| norm 0.2855 (+0.02z)| lr 5.73e-04 | 322.75 ms | 52.3% bf16 MFU | 1622561 tok/s
step 3266/19560 | loss 3.763644 (+1.72z)| norm 0.3186 (+1.20z)| lr 5.73e-04 | 322.72 ms | 52.3% bf16 MFU | 1622662 tok/s
step 3267/19560 | loss 3.694453 (+0.01z)| norm 0.3338 (+1.72z)| lr 5.73e-04 | 323.55 ms | 52.2% bf16 MFU | 1622551 tok/s
step 3268/19560 | loss 3.729711 (+0.87z)| norm 0.3129 (+1.00z)| lr 5.73e-04 | 322.80 ms | 52.3% bf16 MFU | 1622632 tok/s
step 3269/19560 | loss 3.695463 (+0.02z)| norm 0.2616 (-0.80z)| lr 5.73e-04 | 322.63 ms | 52.3% bf16 MFU | 1622752 tok/s
step 3270/19560 | loss 3.710131 (+0.39z)| norm 0.2604 (-0.83z)| lr 5.73e-04 | 322.51 ms | 52.3% bf16 MFU | 1622897 tok/s
step 3271/19560 | loss 3.678530 (-0.39z)| norm 0.2728 (-0.38z)| lr 5.73e-04 | 322.84 ms | 52.3% bf16 MFU | 1622950 tok/s
step 3272/19560 | loss 3.666641 (-0.69z)| norm 0.2730 (-0.37z)| lr 5.73e-04 | 322.71 ms | 52.3% bf16 MFU | 1623034 tok/s
step 3273/19560 | loss 3.723111 (+0.71z)| norm 0.2390 (-1.54z)| lr 5.73e-04 | 322.99 ms | 52.3% bf16 MFU | 1623044 tok/s
step 3274/19560 | loss 3.701970 (+0.19z)| norm 0.2647 (-0.64z)| lr 5.73e-04 | 323.23 ms | 52.2% bf16 MFU | 1622994 tok/s
step 3275/19560 | loss 3.760213 (+1.61z)| norm 0.2770 (-0.20z)| lr 5.73e-04 | 322.80 ms | 52.3% bf16 MFU | 1623054 tok/s
step 3276/19560 | loss 3.698072 (+0.08z)| norm 0.2742 (-0.31z)| lr 5.73e-04 | 322.50 ms | 52.3% bf16 MFU | 1623187 tok/s
step 3277/19560 | loss 3.624154 (-1.72z)| norm 0.2896 (+0.23z)| lr 5.73e-04 | 323.11 ms | 52.2% bf16 MFU | 1623158 tok/s
step 3278/19560 | loss 3.760365 (+1.58z)| norm 0.2846 (+0.05z)| lr 5.73e-04 | 322.72 ms | 52.3% bf16 MFU | 1623231 tok/s
step 3279/19560 | loss 3.684399 (-0.25z)| norm 0.2832 (-0.00z)| lr 5.73e-04 | 322.75 ms | 52.3% bf16 MFU | 1623292 tok/s
step 3280/19560 | loss 3.649263 (-1.15z)| norm 0.2755 (-0.28z)| lr 5.73e-04 | 323.27 ms | 52.2% bf16 MFU | 1623220 tok/s
step 3281/19560 | loss 3.687056 (-0.19z)| norm 0.2907 (+0.26z)| lr 5.73e-04 | 322.92 ms | 52.3% bf16 MFU | 1623238 tok/s
step 3282/19560 | loss 3.753514 (+1.48z)| norm 0.3137 (+1.06z)| lr 5.73e-04 | 323.89 ms | 52.1% bf16 MFU | 1623013 tok/s
step 3283/19560 | loss 3.743462 (+1.21z)| norm 0.2928 (+0.31z)| lr 5.73e-04 | 322.79 ms | 52.3% bf16 MFU | 1623073 tok/s
step 3284/19560 | loss 3.705314 (+0.26z)| norm 0.2542 (-1.04z)| lr 5.73e-04 | 323.00 ms | 52.3% bf16 MFU | 1623080 tok/s
step 3285/19560 | loss 3.699313 (+0.09z)| norm 0.2625 (-0.74z)| lr 5.73e-04 | 322.63 ms | 52.3% bf16 MFU | 1623178 tok/s
step 3286/19560 | loss 3.721731 (+0.65z)| norm 0.2794 (-0.14z)| lr 5.73e-04 | 322.78 ms | 52.3% bf16 MFU | 1623233 tok/s
step 3287/19560 | loss 3.670036 (-0.65z)| norm 0.2723 (-0.39z)| lr 5.73e-04 | 322.92 ms | 52.3% bf16 MFU | 1623251 tok/s
step 3288/19560 | loss 3.671425 (-0.61z)| norm 0.2600 (-0.83z)| lr 5.73e-04 | 322.77 ms | 52.3% bf16 MFU | 1623306 tok/s
step 3289/19560 | loss 3.685275 (-0.25z)| norm 0.2526 (-1.08z)| lr 5.73e-04 | 322.79 ms | 52.3% bf16 MFU | 1623354 tok/s
step 3290/19560 | loss 3.684609 (-0.27z)| norm 0.2772 (-0.21z)| lr 5.73e-04 | 322.83 ms | 52.3% bf16 MFU | 1623387 tok/s
step 3291/19560 | loss 3.718636 (+0.60z)| norm 0.2931 (+0.35z)| lr 5.73e-04 | 322.69 ms | 52.3% bf16 MFU | 1623456 tok/s
step 3292/19560 | loss 3.725147 (+0.75z)| norm 0.2709 (-0.43z)| lr 5.72e-04 | 323.13 ms | 52.2% bf16 MFU | 1623409 tok/s
step 3293/19560 | loss 3.655217 (-1.02z)| norm 0.2700 (-0.46z)| lr 5.72e-04 | 322.91 ms | 52.3% bf16 MFU | 1623420 tok/s
step 3294/19560 | loss 3.673260 (-0.56z)| norm 0.2636 (-0.67z)| lr 5.72e-04 | 322.80 ms | 52.3% bf16 MFU | 1623459 tok/s
step 3295/19560 | loss 3.704463 (+0.23z)| norm 0.2607 (-0.76z)| lr 5.72e-04 | 322.70 ms | 52.3% bf16 MFU | 1623522 tok/s
step 3296/19560 | loss 3.643950 (-1.28z)| norm 0.3078 (+0.89z)| lr 5.72e-04 | 322.98 ms | 52.3% bf16 MFU | 1623510 tok/s
step 3297/19560 | loss 3.719840 (+0.62z)| norm 0.3690 (+2.92z)| lr 5.72e-04 | 322.59 ms | 52.3% bf16 MFU | 1623596 tok/s
step 3298/19560 | loss 3.706294 (+0.27z)| norm 0.3527 (+2.30z)| lr 5.72e-04 | 323.24 ms | 52.2% bf16 MFU | 1623516 tok/s
step 3299/19560 | loss 3.690620 (-0.13z)| norm 0.2869 (+0.12z)| lr 5.72e-04 | 322.74 ms | 52.3% bf16 MFU | 1623564 tok/s
step 3300/19560 | loss 3.587313 (-2.65z)| norm 0.2759 (-0.24z)| lr 5.72e-04 | 322.92 ms | 52.3% bf16 MFU | 1623565 tok/s
step 3301/19560 | loss 3.681112 (-0.33z)| norm 0.3202 (+1.23z)| lr 5.72e-04 | 322.56 ms | 52.3% bf16 MFU | 1623658 tok/s
step 3302/19560 | loss 3.667115 (-0.67z)| norm 0.3034 (+0.67z)| lr 5.72e-04 | 322.84 ms | 52.3% bf16 MFU | 1623675 tok/s
step 3303/19560 | loss 3.694960 (-0.00z)| norm 0.3061 (+0.74z)| lr 5.72e-04 | 322.82 ms | 52.3% bf16 MFU | 1623695 tok/s
step 3304/19560 | loss 3.714496 (+0.48z)| norm 0.3406 (+1.86z)| lr 5.72e-04 | 322.40 ms | 52.3% bf16 MFU | 1623821 tok/s
step 3305/19560 | loss 3.651899 (-1.11z)| norm 0.2892 (+0.16z)| lr 5.72e-04 | 323.26 ms | 52.2% bf16 MFU | 1623725 tok/s
step 3306/19560 | loss 3.647086 (-1.22z)| norm 0.3721 (+2.78z)| lr 5.72e-04 | 322.69 ms | 52.3% bf16 MFU | 1623775 tok/s
step 3307/19560 | loss 3.692151 (-0.08z)| norm 0.3296 (+1.40z)| lr 5.72e-04 | 323.17 ms | 52.2% bf16 MFU | 1623703 tok/s
step 3308/19560 | loss 3.682999 (-0.31z)| norm 0.3513 (+2.04z)| lr 5.72e-04 | 322.54 ms | 52.3% bf16 MFU | 1623793 tok/s
step 3309/19560 | loss 3.677485 (-0.45z)| norm 0.3093 (+0.71z)| lr 5.72e-04 | 322.72 ms | 52.3% bf16 MFU | 1623834 tok/s
step 3310/19560 | loss 3.641174 (-1.34z)| norm 0.3087 (+0.70z)| lr 5.72e-04 | 323.21 ms | 52.2% bf16 MFU | 1623748 tok/s
step 3311/19560 | loss 3.739415 (+1.10z)| norm 0.2619 (-0.76z)| lr 5.72e-04 | 322.54 ms | 52.3% bf16 MFU | 1623836 tok/s
step 3312/19560 | loss 3.741284 (+1.13z)| norm 0.2649 (-0.67z)| lr 5.72e-04 | 323.58 ms | 52.2% bf16 MFU | 1623659 tok/s
step 3313/19560 | loss 3.618570 (-1.91z)| norm 0.2602 (-0.81z)| lr 5.72e-04 | 322.77 ms | 52.3% bf16 MFU | 1623692 tok/s
step 3314/19560 | loss 3.689782 (-0.15z)| norm 0.2528 (-1.03z)| lr 5.72e-04 | 323.12 ms | 52.2% bf16 MFU | 1623635 tok/s
step 3315/19560 | loss 3.740806 (+1.11z)| norm 0.2673 (-0.58z)| lr 5.72e-04 | 322.98 ms | 52.3% bf16 MFU | 1623617 tok/s
step 3316/19560 | loss 3.683110 (-0.32z)| norm 0.2726 (-0.41z)| lr 5.72e-04 | 322.77 ms | 52.3% bf16 MFU | 1623652 tok/s
step 3317/19560 | loss 3.684201 (-0.29z)| norm 0.2774 (-0.26z)| lr 5.72e-04 | 322.94 ms | 52.3% bf16 MFU | 1623643 tok/s
step 3318/19560 | loss 3.683357 (-0.31z)| norm 0.2395 (-1.41z)| lr 5.72e-04 | 322.57 ms | 52.3% bf16 MFU | 1623728 tok/s
step 3319/19560 | loss 3.650646 (-1.10z)| norm 0.2357 (-1.50z)| lr 5.72e-04 | 323.49 ms | 52.2% bf16 MFU | 1623577 tok/s
step 3320/19560 | loss 3.657915 (-0.91z)| norm 0.2517 (-0.99z)| lr 5.72e-04 | 322.49 ms | 52.3% bf16 MFU | 1623686 tok/s
step 3321/19560 | loss 3.633368 (-1.49z)| norm 0.2355 (-1.46z)| lr 5.72e-04 | 322.62 ms | 52.3% bf16 MFU | 1623757 tok/s
step 3322/19560 | loss 3.605494 (-2.13z)| norm 0.2562 (-0.83z)| lr 5.72e-04 | 322.84 ms | 52.3% bf16 MFU | 1623768 tok/s
step 3323/19560 | loss 3.604757 (-2.14z)| norm 0.2550 (-0.86z)| lr 5.72e-04 | 323.36 ms | 52.2% bf16 MFU | 1623650 tok/s
step 3324/19560 | loss 3.674684 (-0.45z)| norm 0.2698 (-0.41z)| lr 5.72e-04 | 322.29 ms | 52.4% bf16 MFU | 1623804 tok/s
step 3325/19560 | loss 3.732823 (+0.94z)| norm 0.3040 (+0.62z)| lr 5.72e-04 | 322.74 ms | 52.3% bf16 MFU | 1623839 tok/s
step 3326/19560 | loss 3.755291 (+1.46z)| norm 0.2798 (-0.12z)| lr 5.72e-04 | 323.14 ms | 52.2% bf16 MFU | 1623770 tok/s
step 3327/19560 | loss 3.661168 (-0.80z)| norm 0.2632 (-0.63z)| lr 5.72e-04 | 323.03 ms | 52.2% bf16 MFU | 1623732 tok/s
step 3328/19560 | loss 3.615398 (-1.86z)| norm 0.2719 (-0.37z)| lr 5.72e-04 | 322.67 ms | 52.3% bf16 MFU | 1623788 tok/s
step 3329/19560 | loss 3.672793 (-0.51z)| norm 0.2785 (-0.18z)| lr 5.72e-04 | 323.02 ms | 52.2% bf16 MFU | 1623752 tok/s
step 3330/19560 | loss 3.731031 (+0.87z)| norm 0.2493 (-1.10z)| lr 5.72e-04 | 322.61 ms | 52.3% bf16 MFU | 1623823 tok/s
step 3331/19560 | loss 3.624873 (-1.64z)| norm 0.2445 (-1.25z)| lr 5.72e-04 | 322.81 ms | 52.3% bf16 MFU | 1623838 tok/s
step 3332/19560 | loss 3.652117 (-0.98z)| norm 0.2559 (-0.88z)| lr 5.72e-04 | 323.22 ms | 52.2% bf16 MFU | 1623751 tok/s
step 3333/19560 | loss 3.644719 (-1.13z)| norm 0.2552 (-0.89z)| lr 5.72e-04 | 322.91 ms | 52.3% bf16 MFU | 1623744 tok/s
step 3334/19560 | loss 3.758904 (+1.52z)| norm 0.2711 (-0.40z)| lr 5.72e-04 | 322.39 ms | 52.3% bf16 MFU | 1623869 tok/s
step 3335/19560 | loss 3.656636 (-0.86z)| norm 0.2773 (-0.20z)| lr 5.72e-04 | 322.98 ms | 52.3% bf16 MFU | 1623840 tok/s
step 3336/19560 | loss 3.710903 (+0.41z)| norm 0.2866 (+0.09z)| lr 5.72e-04 | 323.10 ms | 52.2% bf16 MFU | 1623782 tok/s
step 3337/19560 | loss 3.660943 (-0.75z)| norm 0.2874 (+0.11z)| lr 5.72e-04 | 322.87 ms | 52.3% bf16 MFU | 1623785 tok/s
step 3338/19560 | loss 3.684973 (-0.19z)| norm 0.2969 (+0.40z)| lr 5.72e-04 | 323.27 ms | 52.2% bf16 MFU | 1623688 tok/s
step 3339/19560 | loss 3.736895 (+1.00z)| norm 0.2979 (+0.42z)| lr 5.71e-04 | 322.79 ms | 52.3% bf16 MFU | 1623716 tok/s
step 3340/19560 | loss 3.680922 (-0.31z)| norm 0.3163 (+0.98z)| lr 5.71e-04 | 323.06 ms | 52.2% bf16 MFU | 1623676 tok/s
step 3341/19560 | loss 3.721260 (+0.64z)| norm 0.3133 (+0.87z)| lr 5.71e-04 | 322.61 ms | 52.3% bf16 MFU | 1623748 tok/s
step 3342/19560 | loss 3.651060 (-0.99z)| norm 0.3242 (+1.19z)| lr 5.71e-04 | 322.95 ms | 52.3% bf16 MFU | 1623734 tok/s
step 3343/19560 | loss 3.656392 (-0.87z)| norm 0.3022 (+0.50z)| lr 5.71e-04 | 322.11 ms | 52.4% bf16 MFU | 1623931 tok/s
step 3344/19560 | loss 3.685981 (-0.18z)| norm 0.2756 (-0.34z)| lr 5.71e-04 | 323.17 ms | 52.2% bf16 MFU | 1623851 tok/s
step 3345/19560 | loss 3.724131 (+0.72z)| norm 0.2734 (-0.40z)| lr 5.71e-04 | 322.29 ms | 52.4% bf16 MFU | 1623996 tok/s
step 3346/19560 | loss 3.696491 (+0.07z)| norm 0.2453 (-1.27z)| lr 5.71e-04 | 322.35 ms | 52.4% bf16 MFU | 1624120 tok/s
step 3347/19560 | loss 3.706540 (+0.30z)| norm 0.2556 (-0.93z)| lr 5.71e-04 | 323.54 ms | 52.2% bf16 MFU | 1623938 tok/s
step 3348/19560 | loss 3.712831 (+0.44z)| norm 0.2636 (-0.67z)| lr 5.71e-04 | 322.56 ms | 52.3% bf16 MFU | 1624011 tok/s
step 3349/19560 | loss 3.681122 (-0.30z)| norm 0.2726 (-0.39z)| lr 5.71e-04 | 322.26 ms | 52.4% bf16 MFU | 1624156 tok/s
step 3350/19560 | loss 3.713390 (+0.46z)| norm 0.2814 (-0.12z)| lr 5.71e-04 | 323.24 ms | 52.2% bf16 MFU | 1624048 tok/s
step 3351/19560 | loss 3.668691 (-0.59z)| norm 0.2715 (-0.43z)| lr 5.71e-04 | 322.84 ms | 52.3% bf16 MFU | 1624044 tok/s
step 3352/19560 | loss 3.655758 (-0.90z)| norm 0.2568 (-0.88z)| lr 5.71e-04 | 322.93 ms | 52.3% bf16 MFU | 1624018 tok/s
step 3353/19560 | loss 3.700764 (+0.16z)| norm 0.2471 (-1.17z)| lr 5.71e-04 | 322.84 ms | 52.3% bf16 MFU | 1624016 tok/s
step 3354/19560 | loss 3.657199 (-0.86z)| norm 0.2408 (-1.35z)| lr 5.71e-04 | 322.33 ms | 52.4% bf16 MFU | 1624144 tok/s
step 3355/19560 | loss 3.684429 (-0.22z)| norm 0.2465 (-1.16z)| lr 5.71e-04 | 322.95 ms | 52.3% bf16 MFU | 1624109 tok/s
step 3356/19560 | loss 3.725117 (+0.74z)| norm 0.2512 (-1.00z)| lr 5.71e-04 | 322.65 ms | 52.3% bf16 MFU | 1624150 tok/s
step 3357/19560 | loss 3.607476 (-2.02z)| norm 0.2821 (-0.03z)| lr 5.71e-04 | 322.77 ms | 52.3% bf16 MFU | 1624160 tok/s
step 3358/19560 | loss 3.744815 (+1.21z)| norm 0.3170 (+1.06z)| lr 5.71e-04 | 322.46 ms | 52.3% bf16 MFU | 1624246 tok/s
step 3359/19560 | loss 3.660413 (-0.77z)| norm 0.3319 (+1.50z)| lr 5.71e-04 | 323.05 ms | 52.2% bf16 MFU | 1624180 tok/s
step 3360/19560 | loss 3.669435 (-0.55z)| norm 0.2865 (+0.10z)| lr 5.71e-04 | 322.64 ms | 52.3% bf16 MFU | 1624221 tok/s
step 3361/19560 | loss 3.676309 (-0.38z)| norm 0.2898 (+0.20z)| lr 5.71e-04 | 322.18 ms | 52.4% bf16 MFU | 1624376 tok/s
step 3362/19560 | loss 3.694860 (+0.07z)| norm 0.3438 (+1.83z)| lr 5.71e-04 | 322.62 ms | 52.3% bf16 MFU | 1624411 tok/s
step 3363/19560 | loss 3.695092 (+0.07z)| norm 0.3205 (+1.10z)| lr 5.71e-04 | 322.61 ms | 52.3% bf16 MFU | 1624448 tok/s
step 3364/19560 | loss 3.646910 (-1.07z)| norm 0.2998 (+0.47z)| lr 5.71e-04 | 322.46 ms | 52.3% bf16 MFU | 1624522 tok/s
step 3365/19560 | loss 3.637894 (-1.27z)| norm 0.2907 (+0.19z)| lr 5.71e-04 | 322.55 ms | 52.3% bf16 MFU | 1624568 tok/s
step 3366/19560 | loss 3.657141 (-0.80z)| norm 0.2873 (+0.10z)| lr 5.71e-04 | 322.55 ms | 52.3% bf16 MFU | 1624612 tok/s
step 3367/19560 | loss 3.675282 (-0.34z)| norm 0.3052 (+0.63z)| lr 5.71e-04 | 322.97 ms | 52.3% bf16 MFU | 1624549 tok/s
step 3368/19560 | loss 3.741784 (+1.28z)| norm 0.3287 (+1.33z)| lr 5.71e-04 | 322.56 ms | 52.3% bf16 MFU | 1624590 tok/s
step 3369/19560 | loss 3.651825 (-0.92z)| norm 0.3198 (+1.05z)| lr 5.71e-04 | 322.42 ms | 52.3% bf16 MFU | 1624667 tok/s
step 3370/19560 | loss 3.677831 (-0.29z)| norm 0.3098 (+0.75z)| lr 5.71e-04 | 322.64 ms | 52.3% bf16 MFU | 1624684 tok/s
step 3371/19560 | loss 3.698433 (+0.22z)| norm 0.2655 (-0.58z)| lr 5.71e-04 | 322.88 ms | 52.3% bf16 MFU | 1624639 tok/s
step 3372/19560 | loss 3.731972 (+1.04z)| norm 0.2596 (-0.75z)| lr 5.71e-04 | 323.06 ms | 52.2% bf16 MFU | 1624552 tok/s
step 3373/19560 | loss 3.663380 (-0.63z)| norm 0.2685 (-0.48z)| lr 5.71e-04 | 322.34 ms | 52.4% bf16 MFU | 1624651 tok/s
step 3374/19560 | loss 3.697206 (+0.21z)| norm 0.2462 (-1.15z)| lr 5.71e-04 | 323.00 ms | 52.3% bf16 MFU | 1624577 tok/s
step 3375/19560 | loss 3.738912 (+1.24z)| norm 0.2479 (-1.09z)| lr 5.71e-04 | 322.88 ms | 52.3% bf16 MFU | 1624538 tok/s
step 3376/19560 | loss 3.643558 (-1.13z)| norm 0.2658 (-0.56z)| lr 5.71e-04 | 322.88 ms | 52.3% bf16 MFU | 1624500 tok/s
step 3377/19560 | loss 3.659608 (-0.72z)| norm 0.2527 (-0.95z)| lr 5.71e-04 | 322.95 ms | 52.3% bf16 MFU | 1624447 tok/s
step 3378/19560 | loss 3.649061 (-0.97z)| norm 0.2780 (-0.19z)| lr 5.71e-04 | 323.20 ms | 52.2% bf16 MFU | 1624332 tok/s
step 3379/19560 | loss 3.704080 (+0.40z)| norm 0.2944 (+0.29z)| lr 5.71e-04 | 322.74 ms | 52.3% bf16 MFU | 1624339 tok/s
step 3380/19560 | loss 3.649323 (-0.95z)| norm 0.2755 (-0.28z)| lr 5.71e-04 | 322.72 ms | 52.3% bf16 MFU | 1624351 tok/s
step 3381/19560 | loss 3.683176 (-0.10z)| norm 0.2933 (+0.35z)| lr 5.71e-04 | 322.74 ms | 52.3% bf16 MFU | 1624358 tok/s
step 3382/19560 | loss 3.700035 (+0.34z)| norm 0.3057 (+0.81z)| lr 5.71e-04 | 323.08 ms | 52.2% bf16 MFU | 1624279 tok/s
step 3383/19560 | loss 3.676276 (-0.26z)| norm 0.3207 (+1.33z)| lr 5.71e-04 | 322.64 ms | 52.3% bf16 MFU | 1624315 tok/s
step 3384/19560 | loss 3.665138 (-0.54z)| norm 0.3060 (+0.80z)| lr 5.71e-04 | 322.76 ms | 52.3% bf16 MFU | 1624318 tok/s
step 3385/19560 | loss 3.663056 (-0.61z)| norm 0.2916 (+0.29z)| lr 5.71e-04 | 323.35 ms | 52.2% bf16 MFU | 1624174 tok/s
step 3386/19560 | loss 3.653588 (-0.84z)| norm 0.2633 (-0.71z)| lr 5.70e-04 | 322.71 ms | 52.3% bf16 MFU | 1624197 tok/s
step 3387/19560 | loss 3.621528 (-1.68z)| norm 0.2580 (-0.89z)| lr 5.70e-04 | 323.13 ms | 52.2% bf16 MFU | 1624114 tok/s
step 3388/19560 | loss 3.668841 (-0.43z)| norm 0.2669 (-0.57z)| lr 5.70e-04 | 322.91 ms | 52.3% bf16 MFU | 1624092 tok/s
step 3389/19560 | loss 3.663946 (-0.56z)| norm 0.2502 (-1.15z)| lr 5.70e-04 | 323.01 ms | 52.2% bf16 MFU | 1624043 tok/s
step 3390/19560 | loss 3.680719 (-0.11z)| norm 0.2395 (-1.50z)| lr 5.70e-04 | 322.41 ms | 52.3% bf16 MFU | 1624149 tok/s
step 3391/19560 | loss 3.668332 (-0.43z)| norm 0.2614 (-0.73z)| lr 5.70e-04 | 322.59 ms | 52.3% bf16 MFU | 1624205 tok/s
step 3392/19560 | loss 3.623150 (-1.63z)| norm 0.2557 (-0.91z)| lr 5.70e-04 | 322.70 ms | 52.3% bf16 MFU | 1624231 tok/s
step 3393/19560 | loss 3.677561 (-0.16z)| norm 0.2690 (-0.44z)| lr 5.70e-04 | 322.46 ms | 52.3% bf16 MFU | 1624313 tok/s
step 3394/19560 | loss 3.692694 (+0.27z)| norm 0.2579 (-0.82z)| lr 5.70e-04 | 323.11 ms | 52.2% bf16 MFU | 1624230 tok/s
step 3395/19560 | loss 3.638581 (-1.20z)| norm 0.2860 (+0.18z)| lr 5.70e-04 | 322.94 ms | 52.3% bf16 MFU | 1624193 tok/s
step 3396/19560 | loss 3.719738 (+1.02z)| norm 0.2760 (-0.16z)| lr 5.70e-04 | 322.80 ms | 52.3% bf16 MFU | 1624192 tok/s
step 3397/19560 | loss 3.666102 (-0.44z)| norm 0.2765 (-0.15z)| lr 5.70e-04 | 322.90 ms | 52.3% bf16 MFU | 1624167 tok/s
step 3398/19560 | loss 3.644530 (-1.01z)| norm 0.2669 (-0.49z)| lr 5.70e-04 | 322.33 ms | 52.4% bf16 MFU | 1624286 tok/s
step 3399/19560 | loss 3.714964 (+0.90z)| norm 0.2797 (-0.04z)| lr 5.70e-04 | 323.55 ms | 52.2% bf16 MFU | 1624093 tok/s
step 3400/19560 | loss 3.693952 (+0.32z)| norm 0.2861 (+0.19z)| lr 5.70e-04 | 323.44 ms | 52.2% bf16 MFU | 1623938 tok/s
step 3401/19560 | loss 3.659525 (-0.61z)| norm 0.2823 (+0.04z)| lr 5.70e-04 | 322.91 ms | 52.3% bf16 MFU | 1623921 tok/s
step 3402/19560 | loss 3.605315 (-2.04z)| norm 0.3053 (+0.85z)| lr 5.70e-04 | 322.64 ms | 52.3% bf16 MFU | 1623976 tok/s
step 3403/19560 | loss 3.717487 (+1.01z)| norm 0.3475 (+2.30z)| lr 5.70e-04 | 322.82 ms | 52.3% bf16 MFU | 1623981 tok/s
step 3404/19560 | loss 3.659146 (-0.58z)| norm 0.2878 (+0.20z)| lr 5.70e-04 | 322.98 ms | 52.3% bf16 MFU | 1623947 tok/s
step 3405/19560 | loss 3.658441 (-0.61z)| norm 0.2763 (-0.21z)| lr 5.70e-04 | 323.13 ms | 52.2% bf16 MFU | 1623875 tok/s
step 3406/19560 | loss 3.666961 (-0.36z)| norm 0.3150 (+1.14z)| lr 5.70e-04 | 323.76 ms | 52.1% bf16 MFU | 1623650 tok/s
step 3407/19560 | loss 3.734056 (+1.50z)| norm 0.2674 (-0.52z)| lr 5.70e-04 | 322.84 ms | 52.3% bf16 MFU | 1623666 tok/s
step 3408/19560 | loss 3.638472 (-1.15z)| norm 0.2803 (-0.07z)| lr 5.70e-04 | 322.99 ms | 52.3% bf16 MFU | 1623645 tok/s
step 3409/19560 | loss 3.676239 (-0.10z)| norm 0.2984 (+0.56z)| lr 5.70e-04 | 322.63 ms | 52.3% bf16 MFU | 1623715 tok/s
step 3410/19560 | loss 3.723595 (+1.23z)| norm 0.3046 (+0.78z)| lr 5.70e-04 | 323.17 ms | 52.2% bf16 MFU | 1623645 tok/s
step 3411/19560 | loss 3.600152 (-2.19z)| norm 0.2651 (-0.59z)| lr 5.70e-04 | 323.20 ms | 52.2% bf16 MFU | 1623572 tok/s
step 3412/19560 | loss 3.605825 (-1.98z)| norm 0.2758 (-0.22z)| lr 5.70e-04 | 322.68 ms | 52.3% bf16 MFU | 1623632 tok/s
step 3413/19560 | loss 3.665535 (-0.33z)| norm 0.2920 (+0.33z)| lr 5.70e-04 | 322.98 ms | 52.3% bf16 MFU | 1623614 tok/s
step 3414/19560 | loss 3.724487 (+1.29z)| norm 0.3020 (+0.68z)| lr 5.70e-04 | 323.15 ms | 52.2% bf16 MFU | 1623556 tok/s
step 3415/19560 | loss 3.691626 (+0.38z)| norm 0.2919 (+0.32z)| lr 5.70e-04 | 322.58 ms | 52.3% bf16 MFU | 1623642 tok/s
step 3416/19560 | loss 3.659560 (-0.50z)| norm 0.2862 (+0.11z)| lr 5.70e-04 | 322.86 ms | 52.3% bf16 MFU | 1623656 tok/s
step 3417/19560 | loss 3.716787 (+1.06z)| norm 0.2962 (+0.46z)| lr 5.70e-04 | 323.16 ms | 52.2% bf16 MFU | 1623593 tok/s
step 3418/19560 | loss 3.693579 (+0.42z)| norm 0.2586 (-0.87z)| lr 5.70e-04 | 323.16 ms | 52.2% bf16 MFU | 1623532 tok/s
step 3419/19560 | loss 3.676621 (-0.03z)| norm 0.2727 (-0.36z)| lr 5.70e-04 | 323.39 ms | 52.2% bf16 MFU | 1623416 tok/s
step 3420/19560 | loss 3.670767 (-0.18z)| norm 0.2627 (-0.71z)| lr 5.70e-04 | 322.62 ms | 52.3% bf16 MFU | 1623500 tok/s
step 3421/19560 | loss 3.681642 (+0.11z)| norm 0.3189 (+1.24z)| lr 5.70e-04 | 322.39 ms | 52.3% bf16 MFU | 1623637 tok/s
step 3422/19560 | loss 3.737501 (+1.63z)| norm 0.2539 (-1.03z)| lr 5.70e-04 | 323.59 ms | 52.2% bf16 MFU | 1623466 tok/s
step 3423/19560 | loss 3.737751 (+1.62z)| norm 0.3045 (+0.73z)| lr 5.70e-04 | 323.63 ms | 52.1% bf16 MFU | 1623294 tok/s
step 3424/19560 | loss 3.669468 (-0.24z)| norm 0.2957 (+0.43z)| lr 5.70e-04 | 322.72 ms | 52.3% bf16 MFU | 1623359 tok/s
step 3425/19560 | loss 3.612000 (-1.77z)| norm 0.2670 (-0.57z)| lr 5.70e-04 | 322.42 ms | 52.3% bf16 MFU | 1623495 tok/s
step 3426/19560 | loss 3.694709 (+0.47z)| norm 0.2793 (-0.10z)| lr 5.70e-04 | 323.78 ms | 52.1% bf16 MFU | 1623284 tok/s
step 3427/19560 | loss 3.663936 (-0.36z)| norm 0.2763 (-0.21z)| lr 5.70e-04 | 322.72 ms | 52.3% bf16 MFU | 1623349 tok/s
step 3428/19560 | loss 3.666409 (-0.32z)| norm 0.2643 (-0.66z)| lr 5.70e-04 | 322.98 ms | 52.3% bf16 MFU | 1623346 tok/s
step 3429/19560 | loss 3.645461 (-0.89z)| norm 0.2587 (-0.85z)| lr 5.70e-04 | 323.39 ms | 52.2% bf16 MFU | 1623239 tok/s
step 3430/19560 | loss 3.708720 (+0.85z)| norm 0.2401 (-1.52z)| lr 5.70e-04 | 323.40 ms | 52.2% bf16 MFU | 1623135 tok/s
step 3431/19560 | loss 3.661163 (-0.46z)| norm 0.2566 (-0.89z)| lr 5.70e-04 | 322.51 ms | 52.3% bf16 MFU | 1623262 tok/s
step 3432/19560 | loss 3.709402 (+0.88z)| norm 0.2749 (-0.20z)| lr 5.69e-04 | 322.83 ms | 52.3% bf16 MFU | 1623300 tok/s
step 3433/19560 | loss 3.650425 (-0.75z)| norm 0.2737 (-0.24z)| lr 5.69e-04 | 324.01 ms | 52.1% bf16 MFU | 1623042 tok/s
step 3434/19560 | loss 3.688077 (+0.28z)| norm 0.2863 (+0.28z)| lr 5.69e-04 | 323.39 ms | 52.2% bf16 MFU | 1622950 tok/s
step 3435/19560 | loss 3.634910 (-1.17z)| norm 0.4376 (+5.55z)| lr 5.69e-04 | 323.08 ms | 52.2% bf16 MFU | 1622942 tok/s
step 3436/19560 | loss 3.766613 (+2.38z)| norm 0.2590 (-0.74z)| lr 5.69e-04 | 322.69 ms | 52.3% bf16 MFU | 1623031 tok/s
step 3437/19560 | loss 3.719837 (+1.11z)| norm 0.2738 (-0.20z)| lr 5.69e-04 | 323.46 ms | 52.2% bf16 MFU | 1622922 tok/s
step 3438/19560 | loss 3.668324 (-0.28z)| norm 0.2796 (+0.02z)| lr 5.69e-04 | 323.01 ms | 52.3% bf16 MFU | 1622934 tok/s
step 3439/19560 | loss 3.667648 (-0.29z)| norm 0.3000 (+0.75z)| lr 5.69e-04 | 322.88 ms | 52.3% bf16 MFU | 1622977 tok/s
step 3440/19560 | loss 3.631783 (-1.24z)| norm 0.3087 (+1.06z)| lr 5.69e-04 | 322.80 ms | 52.3% bf16 MFU | 1623038 tok/s
step 3441/19560 | loss 3.656391 (-0.58z)| norm 0.3161 (+1.30z)| lr 5.69e-04 | 322.89 ms | 52.3% bf16 MFU | 1623072 tok/s
step 3442/19560 | loss 3.726743 (+1.34z)| norm 0.2674 (-0.46z)| lr 5.69e-04 | 322.90 ms | 52.3% bf16 MFU | 1623102 tok/s
step 3443/19560 | loss 3.709272 (+0.87z)| norm 0.2450 (-1.26z)| lr 5.69e-04 | 322.70 ms | 52.3% bf16 MFU | 1623182 tok/s
step 3444/19560 | loss 3.676770 (-0.02z)| norm 0.2527 (-0.97z)| lr 5.69e-04 | 323.09 ms | 52.2% bf16 MFU | 1623158 tok/s
step 3445/19560 | loss 3.765381 (+2.35z)| norm 0.4190 (+4.53z)| lr 5.69e-04 | 323.19 ms | 52.2% bf16 MFU | 1623112 tok/s
step 3446/19560 | loss 3.680556 (+0.06z)| norm 0.3657 (+2.69z)| lr 5.69e-04 | 322.24 ms | 52.4% bf16 MFU | 1623306 tok/s
step 3447/19560 | loss 3.668627 (-0.26z)| norm 0.3449 (+1.98z)| lr 5.69e-04 | 322.93 ms | 52.3% bf16 MFU | 1623318 tok/s
step 3448/19560 | loss 3.707021 (+0.76z)| norm 0.2991 (+0.51z)| lr 5.69e-04 | 322.92 ms | 52.3% bf16 MFU | 1623331 tok/s
step 3449/19560 | loss 3.682602 (+0.09z)| norm 0.2799 (-0.11z)| lr 5.69e-04 | 322.95 ms | 52.3% bf16 MFU | 1623336 tok/s
step 3450/19560 | loss 3.711821 (+0.88z)| norm 0.3065 (+0.73z)| lr 5.69e-04 | 322.81 ms | 52.3% bf16 MFU | 1623375 tok/s
step 3451/19560 | loss 3.724052 (+1.20z)| norm 0.2980 (+0.45z)| lr 5.69e-04 | 322.77 ms | 52.3% bf16 MFU | 1623423 tok/s
step 3452/19560 | loss 3.618874 (-1.70z)| norm 0.2766 (-0.25z)| lr 5.69e-04 | 322.76 ms | 52.3% bf16 MFU | 1623471 tok/s
step 3453/19560 | loss 3.631230 (-1.34z)| norm 0.2544 (-0.95z)| lr 5.69e-04 | 323.39 ms | 52.2% bf16 MFU | 1623360 tok/s
step 3454/19560 | loss 3.652442 (-0.74z)| norm 0.2848 (+0.03z)| lr 5.69e-04 | 323.01 ms | 52.3% bf16 MFU | 1623349 tok/s
step 3455/19560 | loss 3.676032 (-0.08z)| norm 0.2730 (-0.35z)| lr 5.69e-04 | 322.88 ms | 52.3% bf16 MFU | 1623372 tok/s
step 3456/19560 | loss 3.733888 (+1.52z)| norm 0.2937 (+0.31z)| lr 5.69e-04 | 322.76 ms | 52.3% bf16 MFU | 1623423 tok/s
step 3457/19560 | loss 3.647980 (-0.89z)| norm 0.2757 (-0.27z)| lr 5.69e-04 | 322.88 ms | 52.3% bf16 MFU | 1623442 tok/s
step 3458/19560 | loss 3.681678 (+0.07z)| norm 0.2795 (-0.16z)| lr 5.69e-04 | 323.12 ms | 52.2% bf16 MFU | 1623398 tok/s
step 3459/19560 | loss 3.691869 (+0.34z)| norm 0.3028 (+0.59z)| lr 5.69e-04 | 323.35 ms | 52.2% bf16 MFU | 1623298 tok/s
step 3460/19560 | loss 3.702503 (+0.64z)| norm 0.3048 (+0.64z)| lr 5.69e-04 | 323.18 ms | 52.2% bf16 MFU | 1623247 tok/s
step 3461/19560 | loss 3.666767 (-0.39z)| norm 0.3252 (+1.29z)| lr 5.69e-04 | 323.40 ms | 52.2% bf16 MFU | 1623143 tok/s
step 3462/19560 | loss 3.689463 (+0.28z)| norm 0.2940 (+0.27z)| lr 5.69e-04 | 323.34 ms | 52.2% bf16 MFU | 1623059 tok/s
step 3463/19560 | loss 3.688694 (+0.25z)| norm 0.3160 (+0.97z)| lr 5.69e-04 | 323.14 ms | 52.2% bf16 MFU | 1623029 tok/s
step 3464/19560 | loss 3.655601 (-0.71z)| norm 0.2963 (+0.33z)| lr 5.69e-04 | 323.20 ms | 52.2% bf16 MFU | 1622986 tok/s
step 3465/19560 | loss 3.723301 (+1.26z)| norm 0.2736 (-0.41z)| lr 5.69e-04 | 324.36 ms | 52.0% bf16 MFU | 1622655 tok/s
step 3466/19560 | loss 3.701477 (+0.62z)| norm 0.2864 (+0.01z)| lr 5.69e-04 | 323.03 ms | 52.2% bf16 MFU | 1622673 tok/s
step 3467/19560 | loss 3.689270 (+0.28z)| norm 0.2939 (+0.26z)| lr 5.69e-04 | 323.35 ms | 52.2% bf16 MFU | 1622610 tok/s
step 3468/19560 | loss 3.716680 (+1.07z)| norm 0.2884 (+0.08z)| lr 5.69e-04 | 323.82 ms | 52.1% bf16 MFU | 1622432 tok/s
step 3469/19560 | loss 3.649822 (-0.88z)| norm 0.2929 (+0.24z)| lr 5.69e-04 | 322.57 ms | 52.3% bf16 MFU | 1622576 tok/s
step 3470/19560 | loss 3.635066 (-1.30z)| norm 0.3050 (+0.64z)| lr 5.69e-04 | 322.87 ms | 52.3% bf16 MFU | 1622639 tok/s
step 3471/19560 | loss 3.692979 (+0.39z)| norm 0.2636 (-0.71z)| lr 5.69e-04 | 323.45 ms | 52.2% bf16 MFU | 1622554 tok/s
step 3472/19560 | loss 3.629451 (-1.45z)| norm 0.2594 (-0.84z)| lr 5.69e-04 | 322.54 ms | 52.3% bf16 MFU | 1622702 tok/s
step 3473/19560 | loss 3.735872 (+1.63z)| norm 0.3274 (+1.37z)| lr 5.69e-04 | 322.73 ms | 52.3% bf16 MFU | 1622793 tok/s
step 3474/19560 | loss 3.686114 (+0.20z)| norm 0.2928 (+0.23z)| lr 5.69e-04 | 322.75 ms | 52.3% bf16 MFU | 1622875 tok/s
step 3475/19560 | loss 3.651187 (-0.80z)| norm 0.2682 (-0.59z)| lr 5.69e-04 | 322.98 ms | 52.3% bf16 MFU | 1622894 tok/s
step 3476/19560 | loss 3.651219 (-0.79z)| norm 0.2526 (-1.09z)| lr 5.69e-04 | 322.44 ms | 52.3% bf16 MFU | 1623050 tok/s
step 3477/19560 | loss 3.638497 (-1.14z)| norm 0.2958 (+0.32z)| lr 5.68e-04 | 322.76 ms | 52.3% bf16 MFU | 1623117 tok/s
step 3478/19560 | loss 3.597924 (-2.25z)| norm 0.2808 (-0.17z)| lr 5.68e-04 | 323.17 ms | 52.2% bf16 MFU | 1623078 tok/s
step 3479/19560 | loss 3.625063 (-1.46z)| norm 0.2817 (-0.15z)| lr 5.68e-04 | 323.02 ms | 52.2% bf16 MFU | 1623078 tok/s
step 3480/19560 | loss 3.635489 (-1.16z)| norm 0.2602 (-0.86z)| lr 5.68e-04 | 322.66 ms | 52.3% bf16 MFU | 1623169 tok/s
step 3481/19560 | loss 3.642164 (-0.96z)| norm 0.2709 (-0.51z)| lr 5.68e-04 | 322.74 ms | 52.3% bf16 MFU | 1623234 tok/s
step 3482/19560 | loss 3.615742 (-1.67z)| norm 0.2921 (+0.18z)| lr 5.68e-04 | 322.84 ms | 52.3% bf16 MFU | 1623272 tok/s
step 3483/19560 | loss 3.696653 (+0.57z)| norm 0.2824 (-0.16z)| lr 5.68e-04 | 323.61 ms | 52.2% bf16 MFU | 1623114 tok/s
step 3484/19560 | loss 3.643339 (-0.89z)| norm 0.2892 (+0.06z)| lr 5.68e-04 | 322.64 ms | 52.3% bf16 MFU | 1623208 tok/s
step 3485/19560 | loss 3.673242 (-0.08z)| norm 0.3113 (+0.80z)| lr 5.68e-04 | 323.30 ms | 52.2% bf16 MFU | 1623132 tok/s
step 3486/19560 | loss 3.635597 (-1.12z)| norm 0.2836 (-0.13z)| lr 5.68e-04 | 323.33 ms | 52.2% bf16 MFU | 1623051 tok/s
step 3487/19560 | loss 3.661675 (-0.38z)| norm 0.2849 (-0.07z)| lr 5.68e-04 | 323.28 ms | 52.2% bf16 MFU | 1622986 tok/s
step 3488/19560 | loss 3.641285 (-0.95z)| norm 0.2954 (+0.29z)| lr 5.68e-04 | 323.03 ms | 52.2% bf16 MFU | 1622989 tok/s
step 3489/19560 | loss 3.725842 (+1.43z)| norm 0.2868 (-0.01z)| lr 5.68e-04 | 322.86 ms | 52.3% bf16 MFU | 1623033 tok/s
step 3490/19560 | loss 3.602313 (-2.00z)| norm 0.2882 (+0.06z)| lr 5.68e-04 | 323.53 ms | 52.2% bf16 MFU | 1622908 tok/s
step 3491/19560 | loss 3.735977 (+1.68z)| norm 0.2924 (+0.21z)| lr 5.68e-04 | 323.02 ms | 52.2% bf16 MFU | 1622916 tok/s
step 3492/19560 | loss 3.672425 (-0.07z)| norm 0.2907 (+0.15z)| lr 5.68e-04 | 322.89 ms | 52.3% bf16 MFU | 1622958 tok/s
step 3493/19560 | loss 3.634572 (-1.11z)| norm 0.2714 (-0.51z)| lr 5.68e-04 | 323.51 ms | 52.2% bf16 MFU | 1622843 tok/s
step 3494/19560 | loss 3.656166 (-0.52z)| norm 0.2764 (-0.34z)| lr 5.68e-04 | 322.76 ms | 52.3% bf16 MFU | 1622919 tok/s
step 3495/19560 | loss 3.658243 (-0.46z)| norm 0.2682 (-0.61z)| lr 5.68e-04 | 323.07 ms | 52.2% bf16 MFU | 1622915 tok/s
step 3496/19560 | loss 3.752013 (+2.11z)| norm 0.2605 (-0.87z)| lr 5.68e-04 | 323.21 ms | 52.2% bf16 MFU | 1622874 tok/s
step 3497/19560 | loss 3.649086 (-0.71z)| norm 0.2598 (-0.88z)| lr 5.68e-04 | 322.85 ms | 52.3% bf16 MFU | 1622928 tok/s
step 3498/19560 | loss 3.629924 (-1.21z)| norm 0.2487 (-1.25z)| lr 5.68e-04 | 322.97 ms | 52.3% bf16 MFU | 1622948 tok/s
step 3499/19560 | loss 3.648370 (-0.70z)| norm 0.2588 (-0.89z)| lr 5.68e-04 | 323.10 ms | 52.2% bf16 MFU | 1622935 tok/s
step 3500/19560 | loss 3.765308 (+2.44z)| norm 0.2523 (-1.11z)| lr 5.68e-04 | 322.48 ms | 52.3% bf16 MFU | 1623078 tok/s
val loss 3.657746
laSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluaevaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2687/10042 = 0.267576
step 3501/19560 | loss 3.617394 (-1.51z)| norm 0.2761 (-0.28z)| lr 5.68e-04 | 322.41 ms | 52.3% bf16 MFU | 1623231 tok/s
step 3502/19560 | loss 3.660687 (-0.35z)| norm 0.2808 (-0.13z)| lr 5.68e-04 | 323.01 ms | 52.3% bf16 MFU | 1623227 tok/s
step 3503/19560 | loss 3.743374 (+1.85z)| norm 0.3100 (+0.88z)| lr 5.68e-04 | 323.01 ms | 52.2% bf16 MFU | 1623222 tok/s
step 3504/19560 | loss 3.712737 (+1.02z)| norm 0.2813 (-0.13z)| lr 5.68e-04 | 322.17 ms | 52.4% bf16 MFU | 1623429 tok/s
step 3505/19560 | loss 3.631363 (-1.13z)| norm 0.2797 (-0.20z)| lr 5.68e-04 | 323.43 ms | 52.2% bf16 MFU | 1623309 tok/s
step 3506/19560 | loss 3.638841 (-0.93z)| norm 0.3008 (+0.54z)| lr 5.68e-04 | 323.26 ms | 52.2% bf16 MFU | 1623238 tok/s
step 3507/19560 | loss 3.645107 (-0.75z)| norm 0.2956 (+0.36z)| lr 5.68e-04 | 322.88 ms | 52.3% bf16 MFU | 1623266 tok/s
step 3508/19560 | loss 3.731641 (+1.50z)| norm 0.3056 (+0.70z)| lr 5.68e-04 | 322.75 ms | 52.3% bf16 MFU | 1623325 tok/s
step 3509/19560 | loss 3.728935 (+1.41z)| norm 0.3303 (+1.56z)| lr 5.68e-04 | 323.45 ms | 52.2% bf16 MFU | 1623204 tok/s
step 3510/19560 | loss 3.672247 (-0.06z)| norm 0.2865 (+0.02z)| lr 5.68e-04 | 323.17 ms | 52.2% bf16 MFU | 1623159 tok/s
step 3511/19560 | loss 3.664184 (-0.26z)| norm 0.3180 (+1.13z)| lr 5.68e-04 | 322.70 ms | 52.3% bf16 MFU | 1623235 tok/s
step 3512/19560 | loss 3.719232 (+1.15z)| norm 0.3224 (+1.28z)| lr 5.68e-04 | 322.97 ms | 52.3% bf16 MFU | 1623240 tok/s
step 3513/19560 | loss 3.695384 (+0.53z)| norm 0.2719 (-0.49z)| lr 5.68e-04 | 322.48 ms | 52.3% bf16 MFU | 1623368 tok/s
step 3514/19560 | loss 3.695477 (+0.52z)| norm 0.2990 (+0.45z)| lr 5.68e-04 | 323.02 ms | 52.2% bf16 MFU | 1623354 tok/s
step 3515/19560 | loss 3.709172 (+0.86z)| norm 0.3266 (+1.40z)| lr 5.68e-04 | 322.82 ms | 52.3% bf16 MFU | 1623391 tok/s
step 3516/19560 | loss 3.670799 (-0.13z)| norm 0.5283 (+6.74z)| lr 5.68e-04 | 322.32 ms | 52.4% bf16 MFU | 1623552 tok/s
step 3517/19560 | loss 3.624639 (-1.32z)| norm 0.3147 (+0.72z)| lr 5.68e-04 | 323.13 ms | 52.2% bf16 MFU | 1623500 tok/s
step 3518/19560 | loss 3.712430 (+0.94z)| norm 0.3078 (+0.51z)| lr 5.68e-04 | 322.60 ms | 52.3% bf16 MFU | 1623586 tok/s
step 3519/19560 | loss 3.594564 (-2.05z)| norm 0.2812 (-0.25z)| lr 5.68e-04 | 322.97 ms | 52.3% bf16 MFU | 1623574 tok/s
step 3520/19560 | loss 3.683169 (+0.19z)| norm 0.2806 (-0.27z)| lr 5.68e-04 | 322.89 ms | 52.3% bf16 MFU | 1623582 tok/s
step 3521/19560 | loss 3.641939 (-0.85z)| norm 0.3698 (+2.21z)| lr 5.68e-04 | 322.77 ms | 52.3% bf16 MFU | 1623621 tok/s
step 3522/19560 | loss 3.717300 (+1.05z)| norm 0.3326 (+1.15z)| lr 5.67e-04 | 322.67 ms | 52.3% bf16 MFU | 1623682 tok/s
step 3523/19560 | loss 3.701891 (+0.65z)| norm 0.3460 (+1.50z)| lr 5.67e-04 | 322.53 ms | 52.3% bf16 MFU | 1623776 tok/s
step 3524/19560 | loss 3.688797 (+0.33z)| norm 0.3089 (+0.46z)| lr 5.67e-04 | 322.67 ms | 52.3% bf16 MFU | 1623829 tok/s
step 3525/19560 | loss 3.649810 (-0.66z)| norm 0.3113 (+0.52z)| lr 5.67e-04 | 323.18 ms | 52.2% bf16 MFU | 1623752 tok/s
step 3526/19560 | loss 3.728164 (+1.31z)| norm 0.2870 (-0.16z)| lr 5.67e-04 | 322.94 ms | 52.3% bf16 MFU | 1623739 tok/s
step 3527/19560 | loss 3.761778 (+2.12z)| norm 0.2685 (-0.67z)| lr 5.67e-04 | 322.59 ms | 52.3% bf16 MFU | 1623814 tok/s
step 3528/19560 | loss 3.731305 (+1.34z)| norm 0.2671 (-0.70z)| lr 5.67e-04 | 322.63 ms | 52.3% bf16 MFU | 1623875 tok/s
step 3529/19560 | loss 3.693401 (+0.40z)| norm 0.2567 (-0.98z)| lr 5.67e-04 | 322.79 ms | 52.3% bf16 MFU | 1623894 tok/s
step 3530/19560 | loss 3.609279 (-1.69z)| norm 0.2658 (-0.72z)| lr 5.67e-04 | 323.18 ms | 52.2% bf16 MFU | 1623813 tok/s
step 3531/19560 | loss 3.803757 (+3.02z)| norm 0.2976 (+0.17z)| lr 5.67e-04 | 323.17 ms | 52.2% bf16 MFU | 1623740 tok/s
step 3532/19560 | loss 3.732564 (+1.29z)| norm 0.3191 (+0.76z)| lr 5.67e-04 | 322.89 ms | 52.3% bf16 MFU | 1623741 tok/s
step 3533/19560 | loss 3.658366 (-0.49z)| norm 0.2767 (-0.42z)| lr 5.67e-04 | 322.54 ms | 52.3% bf16 MFU | 1623828 tok/s
step 3534/19560 | loss 3.708018 (+0.69z)| norm 0.2912 (-0.01z)| lr 5.67e-04 | 322.73 ms | 52.3% bf16 MFU | 1623863 tok/s
step 3535/19560 | loss 3.711666 (+0.79z)| norm 0.2676 (-0.67z)| lr 5.67e-04 | 322.87 ms | 52.3% bf16 MFU | 1623863 tok/s
step 3536/19560 | loss 3.688311 (+0.22z)| norm 0.2460 (-1.25z)| lr 5.67e-04 | 322.96 ms | 52.3% bf16 MFU | 1623839 tok/s
step 3537/19560 | loss 3.675902 (-0.08z)| norm 0.2901 (-0.03z)| lr 5.67e-04 | 323.24 ms | 52.2% bf16 MFU | 1623746 tok/s
step 3538/19560 | loss 3.651445 (-0.66z)| norm 0.2730 (-0.50z)| lr 5.67e-04 | 322.94 ms | 52.3% bf16 MFU | 1623733 tok/s
step 3539/19560 | loss 3.715102 (+0.87z)| norm 0.2611 (-0.82z)| lr 5.67e-04 | 322.34 ms | 52.4% bf16 MFU | 1623872 tok/s
step 3540/19560 | loss 3.697473 (+0.42z)| norm 0.2866 (-0.12z)| lr 5.67e-04 | 322.68 ms | 52.3% bf16 MFU | 1623919 tok/s
step 3541/19560 | loss 3.645177 (-0.86z)| norm 0.2859 (-0.14z)| lr 5.67e-04 | 323.15 ms | 52.2% bf16 MFU | 1623844 tok/s
step 3542/19560 | loss 3.774791 (+2.28z)| norm 0.3062 (+0.42z)| lr 5.67e-04 | 322.55 ms | 52.3% bf16 MFU | 1623923 tok/s
step 3543/19560 | loss 3.694628 (+0.34z)| norm 0.3054 (+0.39z)| lr 5.67e-04 | 322.75 ms | 52.3% bf16 MFU | 1623949 tok/s
step 3544/19560 | loss 3.680361 (-0.01z)| norm 0.2902 (-0.03z)| lr 5.67e-04 | 322.91 ms | 52.3% bf16 MFU | 1623932 tok/s
step 3545/19560 | loss 3.747041 (+1.59z)| norm 0.2945 (+0.09z)| lr 5.67e-04 | 322.90 ms | 52.3% bf16 MFU | 1623920 tok/s
step 3546/19560 | loss 3.589914 (-2.14z)| norm 0.2887 (-0.07z)| lr 5.67e-04 | 322.87 ms | 52.3% bf16 MFU | 1623915 tok/s
step 3547/19560 | loss 3.688187 (+0.19z)| norm 0.3356 (+1.21z)| lr 5.67e-04 | 322.47 ms | 52.3% bf16 MFU | 1624011 tok/s
step 3548/19560 | loss 3.677904 (-0.06z)| norm 0.3337 (+1.13z)| lr 5.67e-04 | 323.29 ms | 52.2% bf16 MFU | 1623896 tok/s
step 3549/19560 | loss 3.754322 (+1.72z)| norm 0.3261 (+0.92z)| lr 5.67e-04 | 322.70 ms | 52.3% bf16 MFU | 1623937 tok/s
step 3550/19560 | loss 3.673514 (-0.16z)| norm 0.2718 (-0.57z)| lr 5.67e-04 | 323.48 ms | 52.2% bf16 MFU | 1623780 tok/s
step 3551/19560 | loss 3.601372 (-1.82z)| norm 0.3187 (+0.71z)| lr 5.67e-04 | 323.14 ms | 52.2% bf16 MFU | 1623714 tok/s
step 3552/19560 | loss 3.689033 (+0.22z)| norm 0.2841 (-0.23z)| lr 5.67e-04 | 322.51 ms | 52.3% bf16 MFU | 1623810 tok/s
step 3553/19560 | loss 3.679593 (-0.01z)| norm 0.2709 (-0.60z)| lr 5.67e-04 | 323.08 ms | 52.2% bf16 MFU | 1623760 tok/s
step 3554/19560 | loss 3.667016 (-0.30z)| norm 0.2676 (-0.69z)| lr 5.67e-04 | 322.50 ms | 52.3% bf16 MFU | 1623858 tok/s
step 3555/19560 | loss 3.718745 (+0.91z)| norm 0.2711 (-0.59z)| lr 5.67e-04 | 323.04 ms | 52.2% bf16 MFU | 1623816 tok/s
step 3556/19560 | loss 3.641137 (-0.91z)| norm 0.2663 (-0.72z)| lr 5.67e-04 | 322.95 ms | 52.3% bf16 MFU | 1623798 tok/s
step 3557/19560 | loss 3.656965 (-0.54z)| norm 0.2828 (-0.27z)| lr 5.67e-04 | 322.69 ms | 52.3% bf16 MFU | 1623844 tok/s
step 3558/19560 | loss 3.684798 (+0.12z)| norm 0.2573 (-0.98z)| lr 5.67e-04 | 323.18 ms | 52.2% bf16 MFU | 1623765 tok/s
step 3559/19560 | loss 3.705243 (+0.59z)| norm 0.2510 (-1.15z)| lr 5.67e-04 | 322.79 ms | 52.3% bf16 MFU | 1623788 tok/s
step 3560/19560 | loss 3.720216 (+0.94z)| norm 0.2633 (-0.81z)| lr 5.67e-04 | 322.81 ms | 52.3% bf16 MFU | 1623805 tok/s
step 3561/19560 | loss 3.611530 (-1.60z)| norm 0.2625 (-0.83z)| lr 5.67e-04 | 322.98 ms | 52.3% bf16 MFU | 1623779 tok/s
step 3562/19560 | loss 3.618780 (-1.41z)| norm 0.2635 (-0.79z)| lr 5.67e-04 | 322.46 ms | 52.3% bf16 MFU | 1623885 tok/s
step 3563/19560 | loss 3.731630 (+1.19z)| norm 0.2718 (-0.57z)| lr 5.67e-04 | 322.70 ms | 52.3% bf16 MFU | 1623925 tok/s
step 3564/19560 | loss 3.667905 (-0.27z)| norm 0.2556 (-1.04z)| lr 5.67e-04 | 323.00 ms | 52.3% bf16 MFU | 1623887 tok/s
step 3565/19560 | loss 3.658940 (-0.47z)| norm 0.2675 (-0.69z)| lr 5.67e-04 | 323.10 ms | 52.2% bf16 MFU | 1623827 tok/s
step 3566/19560 | loss 3.750008 (+1.64z)| norm 0.2526 (-1.12z)| lr 5.66e-04 | 322.71 ms | 52.3% bf16 MFU | 1623866 tok/s
step 3567/19560 | loss 3.665475 (-0.33z)| norm 0.2658 (-0.72z)| lr 5.66e-04 | 322.28 ms | 52.4% bf16 MFU | 1624015 tok/s
step 3568/19560 | loss 3.668571 (-0.27z)| norm 0.2882 (-0.07z)| lr 5.66e-04 | 322.62 ms | 52.3% bf16 MFU | 1624069 tok/s
step 3569/19560 | loss 3.562182 (-2.66z)| norm 0.2964 (+0.18z)| lr 5.66e-04 | 323.15 ms | 52.2% bf16 MFU | 1623986 tok/s
step 3570/19560 | loss 3.681739 (+0.07z)| norm 0.2702 (-0.59z)| lr 5.66e-04 | 322.75 ms | 52.3% bf16 MFU | 1624008 tok/s
step 3571/19560 | loss 3.621551 (-1.29z)| norm 0.2781 (-0.37z)| lr 5.66e-04 | 323.78 ms | 52.1% bf16 MFU | 1623771 tok/s
step 3572/19560 | loss 3.654600 (-0.53z)| norm 0.2856 (-0.15z)| lr 5.66e-04 | 322.46 ms | 52.3% bf16 MFU | 1623877 tok/s
step 3573/19560 | loss 3.691231 (+0.32z)| norm 0.2890 (-0.03z)| lr 5.66e-04 | 322.73 ms | 52.3% bf16 MFU | 1623910 tok/s
step 3574/19560 | loss 3.660940 (-0.38z)| norm 0.2695 (-0.63z)| lr 5.66e-04 | 322.90 ms | 52.3% bf16 MFU | 1623899 tok/s
step 3575/19560 | loss 3.648249 (-0.66z)| norm 0.2588 (-0.95z)| lr 5.66e-04 | 322.68 ms | 52.3% bf16 MFU | 1623944 tok/s
step 3576/19560 | loss 3.683464 (+0.15z)| norm 0.3101 (+0.70z)| lr 5.66e-04 | 323.54 ms | 52.2% bf16 MFU | 1623769 tok/s
step 3577/19560 | loss 3.753423 (+1.74z)| norm 0.3228 (+1.09z)| lr 5.66e-04 | 322.52 ms | 52.3% bf16 MFU | 1623862 tok/s
step 3578/19560 | loss 3.629082 (-1.09z)| norm 0.3057 (+0.54z)| lr 5.66e-04 | 322.30 ms | 52.4% bf16 MFU | 1624004 tok/s
step 3579/19560 | loss 3.606296 (-1.58z)| norm 0.2431 (-1.44z)| lr 5.66e-04 | 323.00 ms | 52.3% bf16 MFU | 1623964 tok/s
step 3580/19560 | loss 3.621768 (-1.23z)| norm 0.2918 (+0.10z)| lr 5.66e-04 | 322.16 ms | 52.4% bf16 MFU | 1624136 tok/s
step 3581/19560 | loss 3.711468 (+0.79z)| norm 0.2553 (-1.06z)| lr 5.66e-04 | 322.76 ms | 52.3% bf16 MFU | 1624148 tok/s
step 3582/19560 | loss 3.697450 (+0.47z)| norm 0.2706 (-0.57z)| lr 5.66e-04 | 322.50 ms | 52.3% bf16 MFU | 1624226 tok/s
step 3583/19560 | loss 3.629316 (-1.07z)| norm 0.2599 (-0.90z)| lr 5.66e-04 | 323.12 ms | 52.2% bf16 MFU | 1624144 tok/s
step 3584/19560 | loss 3.724694 (+1.09z)| norm 0.2558 (-1.02z)| lr 5.66e-04 | 323.23 ms | 52.2% bf16 MFU | 1624039 tok/s
step 3585/19560 | loss 3.638425 (-0.86z)| norm 0.2717 (-0.52z)| lr 5.66e-04 | 322.91 ms | 52.3% bf16 MFU | 1624019 tok/s
step 3586/19560 | loss 3.699886 (+0.53z)| norm 0.2735 (-0.46z)| lr 5.66e-04 | 322.85 ms | 52.3% bf16 MFU | 1624015 tok/s
step 3587/19560 | loss 3.722135 (+1.02z)| norm 0.2808 (-0.22z)| lr 5.66e-04 | 322.61 ms | 52.3% bf16 MFU | 1624072 tok/s
step 3588/19560 | loss 3.693297 (+0.38z)| norm 0.2790 (-0.27z)| lr 5.66e-04 | 323.25 ms | 52.2% bf16 MFU | 1623966 tok/s
step 3589/19560 | loss 3.621727 (-1.22z)| norm 0.2724 (-0.47z)| lr 5.66e-04 | 323.17 ms | 52.2% bf16 MFU | 1623883 tok/s
step 3590/19560 | loss 3.662674 (-0.30z)| norm 0.3224 (+1.11z)| lr 5.66e-04 | 322.93 ms | 52.3% bf16 MFU | 1623865 tok/s
step 3591/19560 | loss 3.810460 (+2.89z)| norm 0.3493 (+1.93z)| lr 5.66e-04 | 322.76 ms | 52.3% bf16 MFU | 1623892 tok/s
step 3592/19560 | loss 3.661386 (-0.34z)| norm 0.2617 (-0.80z)| lr 5.66e-04 | 323.01 ms | 52.3% bf16 MFU | 1623854 tok/s
step 3593/19560 | loss 3.629669 (-1.01z)| norm 0.2761 (-0.35z)| lr 5.66e-04 | 322.62 ms | 52.3% bf16 MFU | 1623917 tok/s
step 3594/19560 | loss 3.643033 (-0.71z)| norm 0.2640 (-0.72z)| lr 5.66e-04 | 323.58 ms | 52.2% bf16 MFU | 1623734 tok/s
step 3595/19560 | loss 3.644464 (-0.67z)| norm 0.2709 (-0.50z)| lr 5.66e-04 | 322.86 ms | 52.3% bf16 MFU | 1623741 tok/s
step 3596/19560 | loss 3.652606 (-0.49z)| norm 0.2726 (-0.45z)| lr 5.66e-04 | 322.87 ms | 52.3% bf16 MFU | 1623746 tok/s
step 3597/19560 | loss 3.660758 (-0.31z)| norm 0.2625 (-0.75z)| lr 5.66e-04 | 322.83 ms | 52.3% bf16 MFU | 1623761 tok/s
step 3598/19560 | loss 3.685028 (+0.21z)| norm 0.2623 (-0.75z)| lr 5.66e-04 | 323.08 ms | 52.2% bf16 MFU | 1623712 tok/s
step 3599/19560 | loss 3.635247 (-0.86z)| norm 0.2708 (-0.48z)| lr 5.66e-04 | 323.44 ms | 52.2% bf16 MFU | 1623574 tok/s
step 3600/19560 | loss 3.618408 (-1.23z)| norm 0.2927 (+0.19z)| lr 5.66e-04 | 323.11 ms | 52.2% bf16 MFU | 1623527 tok/s
step 3601/19560 | loss 3.653492 (-0.45z)| norm 0.2898 (+0.11z)| lr 5.66e-04 | 322.73 ms | 52.3% bf16 MFU | 1623577 tok/s
step 3602/19560 | loss 3.704870 (+0.66z)| norm 0.2690 (-0.54z)| lr 5.66e-04 | 322.57 ms | 52.3% bf16 MFU | 1623664 tok/s
step 3603/19560 | loss 3.634503 (-0.87z)| norm 0.2749 (-0.35z)| lr 5.66e-04 | 323.40 ms | 52.2% bf16 MFU | 1623538 tok/s
step 3604/19560 | loss 3.766853 (+1.96z)| norm 0.3156 (+0.91z)| lr 5.66e-04 | 323.16 ms | 52.2% bf16 MFU | 1623480 tok/s
step 3605/19560 | loss 3.737541 (+1.31z)| norm 0.2970 (+0.32z)| lr 5.66e-04 | 323.08 ms | 52.2% bf16 MFU | 1623444 tok/s
step 3606/19560 | loss 3.698489 (+0.47z)| norm 0.2983 (+0.36z)| lr 5.66e-04 | 322.65 ms | 52.3% bf16 MFU | 1623520 tok/s
step 3607/19560 | loss 3.729347 (+1.11z)| norm 0.2999 (+0.41z)| lr 5.66e-04 | 323.00 ms | 52.3% bf16 MFU | 1623502 tok/s
step 3608/19560 | loss 3.672470 (-0.12z)| norm 0.2846 (-0.08z)| lr 5.66e-04 | 324.12 ms | 52.1% bf16 MFU | 1623205 tok/s
step 3609/19560 | loss 3.619717 (-1.25z)| norm 0.2813 (-0.19z)| lr 5.65e-04 | 322.91 ms | 52.3% bf16 MFU | 1623226 tok/s
step 3610/19560 | loss 3.702425 (+0.52z)| norm 0.2633 (-0.74z)| lr 5.65e-04 | 322.82 ms | 52.3% bf16 MFU | 1623270 tok/s
step 3611/19560 | loss 3.725451 (+1.01z)| norm 0.2826 (-0.14z)| lr 5.65e-04 | 322.81 ms | 52.3% bf16 MFU | 1623313 tok/s
step 3612/19560 | loss 3.667240 (-0.25z)| norm 0.2722 (-0.46z)| lr 5.65e-04 | 323.10 ms | 52.2% bf16 MFU | 1623282 tok/s
step 3613/19560 | loss 3.660913 (-0.39z)| norm 0.2591 (-0.86z)| lr 5.65e-04 | 323.14 ms | 52.2% bf16 MFU | 1623241 tok/s
step 3614/19560 | loss 3.627745 (-1.10z)| norm 0.2685 (-0.56z)| lr 5.65e-04 | 322.83 ms | 52.3% bf16 MFU | 1623281 tok/s
step 3615/19560 | loss 3.610760 (-1.45z)| norm 0.2519 (-1.07z)| lr 5.65e-04 | 322.73 ms | 52.3% bf16 MFU | 1623345 tok/s
step 3616/19560 | loss 3.705536 (+0.57z)| norm 0.2474 (-1.19z)| lr 5.65e-04 | 323.88 ms | 52.1% bf16 MFU | 1623116 tok/s
step 3617/19560 | loss 3.658176 (-0.43z)| norm 0.2495 (-1.11z)| lr 5.65e-04 | 322.41 ms | 52.3% bf16 MFU | 1623267 tok/s
step 3618/19560 | loss 3.680024 (+0.03z)| norm 0.2673 (-0.56z)| lr 5.65e-04 | 322.43 ms | 52.3% bf16 MFU | 1623407 tok/s
step 3619/19560 | loss 3.645299 (-0.72z)| norm 0.2367 (-1.47z)| lr 5.65e-04 | 323.13 ms | 52.2% bf16 MFU | 1623364 tok/s
step 3620/19560 | loss 3.657030 (-0.46z)| norm 0.2327 (-1.57z)| lr 5.65e-04 | 322.91 ms | 52.3% bf16 MFU | 1623378 tok/s
step 3621/19560 | loss 3.660207 (-0.39z)| norm 0.2503 (-1.02z)| lr 5.65e-04 | 322.91 ms | 52.3% bf16 MFU | 1623391 tok/s
step 3622/19560 | loss 3.628361 (-1.08z)| norm 0.2438 (-1.21z)| lr 5.65e-04 | 322.70 ms | 52.3% bf16 MFU | 1623456 tok/s
step 3623/19560 | loss 3.705199 (+0.58z)| norm 0.2577 (-0.78z)| lr 5.65e-04 | 322.95 ms | 52.3% bf16 MFU | 1623456 tok/s
step 3624/19560 | loss 3.711195 (+0.73z)| norm 0.2543 (-0.88z)| lr 5.65e-04 | 323.11 ms | 52.2% bf16 MFU | 1623415 tok/s
step 3625/19560 | loss 3.756629 (+1.69z)| norm 0.2792 (-0.14z)| lr 5.65e-04 | 322.88 ms | 52.3% bf16 MFU | 1623433 tok/s
step 3626/19560 | loss 3.667435 (-0.26z)| norm 0.3051 (+0.62z)| lr 5.65e-04 | 322.94 ms | 52.3% bf16 MFU | 1623434 tok/s
step 3627/19560 | loss 3.648014 (-0.68z)| norm 0.3087 (+0.72z)| lr 5.65e-04 | 323.22 ms | 52.2% bf16 MFU | 1623366 tok/s
step 3628/19560 | loss 3.725416 (+1.03z)| norm 0.2761 (-0.27z)| lr 5.65e-04 | 322.52 ms | 52.3% bf16 MFU | 1623477 tok/s
step 3629/19560 | loss 3.671014 (-0.18z)| norm 0.2797 (-0.16z)| lr 5.65e-04 | 323.72 ms | 52.1% bf16 MFU | 1623282 tok/s
step 3630/19560 | loss 3.685443 (+0.13z)| norm 0.2769 (-0.25z)| lr 5.65e-04 | 322.73 ms | 52.3% bf16 MFU | 1623345 tok/s
step 3631/19560 | loss 3.709129 (+0.67z)| norm 0.2915 (+0.20z)| lr 5.65e-04 | 323.23 ms | 52.2% bf16 MFU | 1623278 tok/s
step 3632/19560 | loss 3.669146 (-0.22z)| norm 0.2772 (-0.23z)| lr 5.65e-04 | 323.25 ms | 52.2% bf16 MFU | 1623211 tok/s
step 3633/19560 | loss 3.642221 (-0.82z)| norm 0.2972 (+0.37z)| lr 5.65e-04 | 323.27 ms | 52.2% bf16 MFU | 1623141 tok/s
step 3634/19560 | loss 3.642330 (-0.82z)| norm 0.3009 (+0.48z)| lr 5.65e-04 | 323.24 ms | 52.2% bf16 MFU | 1623081 tok/s
step 3635/19560 | loss 3.700344 (+0.47z)| norm 0.2689 (-0.48z)| lr 5.65e-04 | 323.56 ms | 52.2% bf16 MFU | 1622945 tok/s
step 3636/19560 | loss 3.722425 (+0.97z)| norm 0.2685 (-0.48z)| lr 5.65e-04 | 322.90 ms | 52.3% bf16 MFU | 1622982 tok/s
step 3637/19560 | loss 3.647052 (-0.72z)| norm 0.2725 (-0.35z)| lr 5.65e-04 | 323.41 ms | 52.2% bf16 MFU | 1622888 tok/s
step 3638/19560 | loss 3.640153 (-0.86z)| norm 0.2706 (-0.41z)| lr 5.65e-04 | 323.70 ms | 52.1% bf16 MFU | 1622727 tok/s
step 3639/19560 | loss 3.584045 (-2.08z)| norm 0.2570 (-0.80z)| lr 5.65e-04 | 323.12 ms | 52.2% bf16 MFU | 1622718 tok/s
step 3640/19560 | loss 3.699117 (+0.48z)| norm 0.2950 (+0.36z)| lr 5.65e-04 | 323.80 ms | 52.1% bf16 MFU | 1622541 tok/s
step 3641/19560 | loss 3.697595 (+0.44z)| norm 0.3348 (+1.55z)| lr 5.65e-04 | 323.77 ms | 52.1% bf16 MFU | 1622379 tok/s
step 3642/19560 | loss 3.658769 (-0.41z)| norm 0.3344 (+1.52z)| lr 5.65e-04 | 323.02 ms | 52.2% bf16 MFU | 1622413 tok/s
step 3643/19560 | loss 3.858369 (+3.77z)| norm 0.3591 (+2.23z)| lr 5.65e-04 | 323.22 ms | 52.2% bf16 MFU | 1622395 tok/s
step 3644/19560 | loss 3.736358 (+1.19z)| norm 0.3588 (+2.85z)| lr 5.65e-04 | 323.36 ms | 52.2% bf16 MFU | 1622345 tok/s
step 3645/19560 | loss 3.682651 (+0.06z)| norm 0.3525 (+2.55z)| lr 5.65e-04 | 323.27 ms | 52.2% bf16 MFU | 1622318 tok/s
step 3646/19560 | loss 3.607031 (-1.49z)| norm 0.3214 (+1.40z)| lr 5.65e-04 | 323.11 ms | 52.2% bf16 MFU | 1622333 tok/s
step 3647/19560 | loss 3.679637 (+0.01z)| norm 0.3023 (+0.69z)| lr 5.65e-04 | 322.58 ms | 52.3% bf16 MFU | 1622481 tok/s
step 3648/19560 | loss 3.661587 (-0.37z)| norm 0.3285 (+1.62z)| lr 5.65e-04 | 322.49 ms | 52.3% bf16 MFU | 1622644 tok/s
step 3649/19560 | loss 3.655878 (-0.49z)| norm 0.3011 (+0.67z)| lr 5.65e-04 | 322.07 ms | 52.4% bf16 MFU | 1622904 tok/s
step 3650/19560 | loss 3.655916 (-0.48z)| norm 0.3053 (+0.84z)| lr 5.65e-04 | 323.04 ms | 52.2% bf16 MFU | 1622909 tok/s
step 3651/19560 | loss 3.636170 (-0.89z)| norm 0.2611 (-0.83z)| lr 5.65e-04 | 323.15 ms | 52.2% bf16 MFU | 1622885 tok/s
step 3652/19560 | loss 3.674745 (-0.07z)| norm 0.2988 (+0.64z)| lr 5.64e-04 | 323.03 ms | 52.2% bf16 MFU | 1622894 tok/s
step 3653/19560 | loss 3.635424 (-0.90z)| norm 0.2820 (-0.00z)| lr 5.64e-04 | 323.06 ms | 52.2% bf16 MFU | 1622892 tok/s
step 3654/19560 | loss 3.756694 (+1.64z)| norm 0.3042 (+0.86z)| lr 5.64e-04 | 322.81 ms | 52.3% bf16 MFU | 1622956 tok/s
step 3655/19560 | loss 3.687736 (+0.21z)| norm 0.2798 (-0.10z)| lr 5.64e-04 | 322.69 ms | 52.3% bf16 MFU | 1623046 tok/s
step 3656/19560 | loss 3.658149 (-0.40z)| norm 0.2647 (-0.69z)| lr 5.64e-04 | 323.31 ms | 52.2% bf16 MFU | 1622976 tok/s
step 3657/19560 | loss 3.712457 (+0.74z)| norm 0.2538 (-1.11z)| lr 5.64e-04 | 322.91 ms | 52.3% bf16 MFU | 1623008 tok/s
step 3658/19560 | loss 3.665302 (-0.27z)| norm 0.2448 (-1.45z)| lr 5.64e-04 | 322.43 ms | 52.3% bf16 MFU | 1623159 tok/s
step 3659/19560 | loss 3.660909 (-0.35z)| norm 0.2713 (-0.41z)| lr 5.64e-04 | 323.29 ms | 52.2% bf16 MFU | 1623087 tok/s
step 3660/19560 | loss 3.702199 (+0.57z)| norm 0.2742 (-0.29z)| lr 5.64e-04 | 323.00 ms | 52.3% bf16 MFU | 1623091 tok/s
step 3661/19560 | loss 3.720089 (+0.95z)| norm 0.2907 (+0.35z)| lr 5.64e-04 | 323.72 ms | 52.1% bf16 MFU | 1622914 tok/s
step 3662/19560 | loss 3.722805 (+1.01z)| norm 0.2606 (-0.81z)| lr 5.64e-04 | 323.27 ms | 52.2% bf16 MFU | 1622859 tok/s
step 3663/19560 | loss 3.673752 (-0.07z)| norm 0.2587 (-0.88z)| lr 5.64e-04 | 323.10 ms | 52.2% bf16 MFU | 1622851 tok/s
step 3664/19560 | loss 3.719105 (+0.93z)| norm 0.2987 (+0.66z)| lr 5.64e-04 | 322.84 ms | 52.3% bf16 MFU | 1622908 tok/s
step 3665/19560 | loss 3.708408 (+0.68z)| norm 0.2750 (-0.26z)| lr 5.64e-04 | 323.37 ms | 52.2% bf16 MFU | 1622828 tok/s
step 3666/19560 | loss 3.633447 (-0.96z)| norm 0.2533 (-1.10z)| lr 5.64e-04 | 322.88 ms | 52.3% bf16 MFU | 1622876 tok/s
step 3667/19560 | loss 3.630573 (-1.00z)| norm 0.2570 (-0.95z)| lr 5.64e-04 | 322.62 ms | 52.3% bf16 MFU | 1622986 tok/s
step 3668/19560 | loss 3.631576 (-0.97z)| norm 0.2459 (-1.36z)| lr 5.64e-04 | 323.13 ms | 52.2% bf16 MFU | 1622964 tok/s
step 3669/19560 | loss 3.593766 (-1.76z)| norm 0.2572 (-0.91z)| lr 5.64e-04 | 323.01 ms | 52.2% bf16 MFU | 1622972 tok/s
step 3670/19560 | loss 3.736791 (+1.34z)| norm 0.2693 (-0.44z)| lr 5.64e-04 | 323.14 ms | 52.2% bf16 MFU | 1622948 tok/s
step 3671/19560 | loss 3.725659 (+1.09z)| norm 0.2908 (+0.40z)| lr 5.64e-04 | 323.30 ms | 52.2% bf16 MFU | 1622886 tok/s
step 3672/19560 | loss 3.720199 (+0.96z)| norm 0.2777 (-0.11z)| lr 5.64e-04 | 322.79 ms | 52.3% bf16 MFU | 1622952 tok/s
step 3673/19560 | loss 3.636162 (-0.84z)| norm 0.3067 (+1.01z)| lr 5.64e-04 | 322.99 ms | 52.3% bf16 MFU | 1622966 tok/s
step 3674/19560 | loss 3.589453 (-1.86z)| norm 0.3102 (+1.13z)| lr 5.64e-04 | 322.74 ms | 52.3% bf16 MFU | 1623041 tok/s
step 3675/19560 | loss 3.669109 (-0.12z)| norm 0.2780 (-0.09z)| lr 5.64e-04 | 323.55 ms | 52.2% bf16 MFU | 1622910 tok/s
step 3676/19560 | loss 3.661311 (-0.29z)| norm 0.3017 (+0.86z)| lr 5.64e-04 | 322.91 ms | 52.3% bf16 MFU | 1622946 tok/s
step 3677/19560 | loss 3.643021 (-0.68z)| norm 0.2964 (+0.67z)| lr 5.64e-04 | 322.98 ms | 52.3% bf16 MFU | 1622963 tok/s
step 3678/19560 | loss 3.451085 (-4.47z)| norm 0.5565 (+7.88z)| lr 5.64e-04 | 322.69 ms | 52.3% bf16 MFU | 1623053 tok/s
step 3679/19560 | loss 3.614644 (-1.16z)| norm 0.2728 (-0.25z)| lr 5.64e-04 | 322.77 ms | 52.3% bf16 MFU | 1623116 tok/s
step 3680/19560 | loss 3.650567 (-0.43z)| norm 0.2712 (-0.30z)| lr 5.64e-04 | 322.47 ms | 52.3% bf16 MFU | 1623253 tok/s
step 3681/19560 | loss 3.639132 (-0.65z)| norm 0.3228 (+1.17z)| lr 5.64e-04 | 323.13 ms | 52.2% bf16 MFU | 1623217 tok/s
step 3682/19560 | loss 3.636233 (-0.71z)| norm 0.4445 (+4.28z)| lr 5.64e-04 | 322.82 ms | 52.3% bf16 MFU | 1623261 tok/s
step 3683/19560 | loss 3.665777 (-0.10z)| norm 0.2599 (-0.62z)| lr 5.64e-04 | 322.63 ms | 52.3% bf16 MFU | 1623351 tok/s
step 3684/19560 | loss 3.706988 (+0.72z)| norm 0.3273 (+1.15z)| lr 5.64e-04 | 323.07 ms | 52.2% bf16 MFU | 1623326 tok/s
step 3685/19560 | loss 3.687790 (+0.33z)| norm 0.2955 (+0.31z)| lr 5.64e-04 | 322.87 ms | 52.3% bf16 MFU | 1623351 tok/s
step 3686/19560 | loss 3.663200 (-0.17z)| norm 0.2841 (+0.00z)| lr 5.64e-04 | 322.96 ms | 52.3% bf16 MFU | 1623352 tok/s
step 3687/19560 | loss 3.711071 (+0.80z)| norm 0.2551 (-0.77z)| lr 5.64e-04 | 323.07 ms | 52.2% bf16 MFU | 1623325 tok/s
step 3688/19560 | loss 3.676479 (+0.11z)| norm 0.2798 (-0.12z)| lr 5.64e-04 | 323.35 ms | 52.2% bf16 MFU | 1623230 tok/s
step 3689/19560 | loss 3.672940 (+0.03z)| norm 0.2629 (-0.56z)| lr 5.64e-04 | 322.80 ms | 52.3% bf16 MFU | 1623277 tok/s
step 3690/19560 | loss 3.635348 (-0.75z)| norm 0.2676 (-0.44z)| lr 5.64e-04 | 322.64 ms | 52.3% bf16 MFU | 1623362 tok/s
step 3691/19560 | loss 3.703012 (+0.65z)| norm 0.2532 (-0.82z)| lr 5.64e-04 | 322.86 ms | 52.3% bf16 MFU | 1623388 tok/s
step 3692/19560 | loss 3.739142 (+1.37z)| norm 0.2967 (+0.33z)| lr 5.64e-04 | 322.59 ms | 52.3% bf16 MFU | 1623480 tok/s
step 3693/19560 | loss 3.668287 (-0.08z)| norm 0.2874 (+0.08z)| lr 5.64e-04 | 323.24 ms | 52.2% bf16 MFU | 1623406 tok/s
step 3694/19560 | loss 3.626527 (-0.92z)| norm 0.2650 (-0.52z)| lr 5.63e-04 | 322.71 ms | 52.3% bf16 MFU | 1623468 tok/s
step 3695/19560 | loss 3.685738 (+0.30z)| norm 0.2677 (-0.45z)| lr 5.63e-04 | 322.71 ms | 52.3% bf16 MFU | 1623527 tok/s
step 3696/19560 | loss 3.697922 (+0.54z)| norm 0.2818 (-0.08z)| lr 5.63e-04 | 322.84 ms | 52.3% bf16 MFU | 1623549 tok/s
step 3697/19560 | loss 3.652001 (-0.42z)| norm 0.3198 (+0.93z)| lr 5.63e-04 | 323.43 ms | 52.2% bf16 MFU | 1623422 tok/s
step 3698/19560 | loss 3.676887 (+0.10z)| norm 0.3076 (+0.60z)| lr 5.63e-04 | 323.22 ms | 52.2% bf16 MFU | 1623356 tok/s
step 3699/19560 | loss 3.691283 (+0.39z)| norm 0.2769 (-0.22z)| lr 5.63e-04 | 322.77 ms | 52.3% bf16 MFU | 1623405 tok/s
step 3700/19560 | loss 3.624268 (-1.01z)| norm 0.2811 (-0.10z)| lr 5.63e-04 | 322.58 ms | 52.3% bf16 MFU | 1623500 tok/s
step 3701/19560 | loss 3.714941 (+0.88z)| norm 0.2771 (-0.21z)| lr 5.63e-04 | 323.16 ms | 52.2% bf16 MFU | 1623444 tok/s
step 3702/19560 | loss 3.651188 (-0.45z)| norm 0.2808 (-0.11z)| lr 5.63e-04 | 322.70 ms | 52.3% bf16 MFU | 1623506 tok/s
step 3703/19560 | loss 3.704969 (+0.67z)| norm 0.2839 (-0.03z)| lr 5.63e-04 | 322.72 ms | 52.3% bf16 MFU | 1623560 tok/s
step 3704/19560 | loss 3.680736 (+0.16z)| norm 0.3323 (+1.24z)| lr 5.63e-04 | 322.16 ms | 52.4% bf16 MFU | 1623752 tok/s
step 3705/19560 | loss 3.600591 (-1.49z)| norm 0.3206 (+0.93z)| lr 5.63e-04 | 322.78 ms | 52.3% bf16 MFU | 1623780 tok/s
step 3706/19560 | loss 3.631083 (-0.85z)| norm 0.2733 (-0.31z)| lr 5.63e-04 | 322.99 ms | 52.3% bf16 MFU | 1623753 tok/s
step 3707/19560 | loss 3.667774 (-0.10z)| norm 0.2603 (-0.66z)| lr 5.63e-04 | 322.35 ms | 52.4% bf16 MFU | 1623889 tok/s
step 3708/19560 | loss 3.710727 (+0.80z)| norm 0.2590 (-0.69z)| lr 5.63e-04 | 322.87 ms | 52.3% bf16 MFU | 1623886 tok/s
step 3709/19560 | loss 3.698312 (+0.54z)| norm 0.2618 (-0.62z)| lr 5.63e-04 | 322.93 ms | 52.3% bf16 MFU | 1623869 tok/s
step 3710/19560 | loss 3.670493 (-0.05z)| norm 0.2636 (-0.57z)| lr 5.63e-04 | 322.67 ms | 52.3% bf16 MFU | 1623917 tok/s
step 3711/19560 | loss 3.628466 (-0.94z)| norm 0.3260 (+1.07z)| lr 5.63e-04 | 322.55 ms | 52.3% bf16 MFU | 1623993 tok/s
step 3712/19560 | loss 3.627172 (-0.95z)| norm 0.3341 (+1.26z)| lr 5.63e-04 | 322.96 ms | 52.3% bf16 MFU | 1623962 tok/s
step 3713/19560 | loss 3.662467 (-0.21z)| norm 0.2973 (+0.29z)| lr 5.63e-04 | 322.89 ms | 52.3% bf16 MFU | 1623951 tok/s
step 3714/19560 | loss 3.702573 (+0.65z)| norm 0.2977 (+0.29z)| lr 5.63e-04 | 322.56 ms | 52.3% bf16 MFU | 1624024 tok/s
step 3715/19560 | loss 3.722736 (+1.07z)| norm 0.2810 (-0.15z)| lr 5.63e-04 | 323.02 ms | 52.2% bf16 MFU | 1623978 tok/s
step 3716/19560 | loss 3.711448 (+0.83z)| norm 0.3093 (+0.59z)| lr 5.63e-04 | 323.25 ms | 52.2% bf16 MFU | 1623876 tok/s
step 3717/19560 | loss 3.726911 (+1.14z)| norm 0.2797 (-0.19z)| lr 5.63e-04 | 322.92 ms | 52.3% bf16 MFU | 1623861 tok/s
step 3718/19560 | loss 3.606706 (-1.39z)| norm 0.2872 (+0.02z)| lr 5.63e-04 | 322.83 ms | 52.3% bf16 MFU | 1623870 tok/s
step 3719/19560 | loss 3.641021 (-0.66z)| norm 0.2893 (+0.09z)| lr 5.63e-04 | 322.78 ms | 52.3% bf16 MFU | 1623890 tok/s
step 3720/19560 | loss 3.669548 (-0.04z)| norm 0.2699 (-0.43z)| lr 5.63e-04 | 322.82 ms | 52.3% bf16 MFU | 1623900 tok/s
step 3721/19560 | loss 3.737176 (+1.40z)| norm 0.2755 (-0.28z)| lr 5.63e-04 | 322.97 ms | 52.3% bf16 MFU | 1623872 tok/s
step 3722/19560 | loss 3.641972 (-0.66z)| norm 0.2489 (-0.99z)| lr 5.63e-04 | 322.91 ms | 52.3% bf16 MFU | 1623860 tok/s
step 3723/19560 | loss 3.673823 (+0.03z)| norm 0.2945 (+0.22z)| lr 5.63e-04 | 322.75 ms | 52.3% bf16 MFU | 1623888 tok/s
step 3724/19560 | loss 3.637520 (-0.75z)| norm 0.2823 (-0.11z)| lr 5.63e-04 | 322.64 ms | 52.3% bf16 MFU | 1623945 tok/s
step 3725/19560 | loss 3.625556 (-1.00z)| norm 0.2592 (-0.72z)| lr 5.63e-04 | 322.98 ms | 52.3% bf16 MFU | 1623911 tok/s
step 3726/19560 | loss 3.659479 (-0.27z)| norm 0.2639 (-0.60z)| lr 5.63e-04 | 322.43 ms | 52.3% bf16 MFU | 1624017 tok/s
step 3727/19560 | loss 3.660875 (-0.24z)| norm 0.2686 (-0.47z)| lr 5.63e-04 | 322.57 ms | 52.3% bf16 MFU | 1624084 tok/s
step 3728/19560 | loss 3.683945 (+0.25z)| norm 0.2719 (-0.38z)| lr 5.63e-04 | 323.25 ms | 52.2% bf16 MFU | 1623975 tok/s
step 3729/19560 | loss 3.620582 (-1.12z)| norm 0.2481 (-1.00z)| lr 5.63e-04 | 322.83 ms | 52.3% bf16 MFU | 1623977 tok/s
step 3730/19560 | loss 3.645226 (-0.58z)| norm 0.2767 (-0.24z)| lr 5.63e-04 | 322.90 ms | 52.3% bf16 MFU | 1623963 tok/s
step 3731/19560 | loss 3.627522 (-0.96z)| norm 0.2651 (-0.55z)| lr 5.63e-04 | 323.36 ms | 52.2% bf16 MFU | 1623834 tok/s
step 3732/19560 | loss 3.643894 (-0.59z)| norm 0.2974 (+0.31z)| lr 5.63e-04 | 322.79 ms | 52.3% bf16 MFU | 1623855 tok/s
step 3733/19560 | loss 3.649383 (-0.46z)| norm 0.3014 (+0.42z)| lr 5.63e-04 | 322.45 ms | 52.3% bf16 MFU | 1623959 tok/s
step 3734/19560 | loss 3.671402 (+0.03z)| norm 0.2809 (-0.12z)| lr 5.63e-04 | 322.90 ms | 52.3% bf16 MFU | 1623944 tok/s
step 3735/19560 | loss 3.665178 (-0.10z)| norm 0.2760 (-0.25z)| lr 5.62e-04 | 322.86 ms | 52.3% bf16 MFU | 1623942 tok/s
step 3736/19560 | loss 3.682496 (+0.29z)| norm 0.2695 (-0.42z)| lr 5.62e-04 | 322.58 ms | 52.3% bf16 MFU | 1624011 tok/s
step 3737/19560 | loss 3.684072 (+0.31z)| norm 0.2831 (-0.06z)| lr 5.62e-04 | 322.75 ms | 52.3% bf16 MFU | 1624033 tok/s
step 3738/19560 | loss 3.692793 (+0.51z)| norm 0.2703 (-0.40z)| lr 5.62e-04 | 322.71 ms | 52.3% bf16 MFU | 1624064 tok/s
step 3739/19560 | loss 3.653551 (-0.36z)| norm 0.2752 (-0.27z)| lr 5.62e-04 | 322.81 ms | 52.3% bf16 MFU | 1624068 tok/s
step 3740/19560 | loss 3.617956 (-1.15z)| norm 0.2604 (-0.66z)| lr 5.62e-04 | 323.04 ms | 52.2% bf16 MFU | 1624014 tok/s
step 3741/19560 | loss 3.623093 (-1.02z)| norm 0.2681 (-0.46z)| lr 5.62e-04 | 323.30 ms | 52.2% bf16 MFU | 1623897 tok/s
step 3742/19560 | loss 3.667682 (-0.03z)| norm 0.2630 (-0.59z)| lr 5.62e-04 | 322.28 ms | 52.4% bf16 MFU | 1624043 tok/s
step 3743/19560 | loss 3.655007 (-0.32z)| norm 0.2619 (-0.62z)| lr 5.62e-04 | 322.74 ms | 52.3% bf16 MFU | 1624066 tok/s
step 3744/19560 | loss 3.575800 (-2.06z)| norm 0.2388 (-1.23z)| lr 5.62e-04 | 322.99 ms | 52.3% bf16 MFU | 1624025 tok/s
step 3745/19560 | loss 3.717788 (+1.08z)| norm 0.2913 (+0.15z)| lr 5.62e-04 | 322.55 ms | 52.3% bf16 MFU | 1624096 tok/s
step 3746/19560 | loss 3.734930 (+1.44z)| norm 0.2945 (+0.23z)| lr 5.62e-04 | 322.76 ms | 52.3% bf16 MFU | 1624109 tok/s
step 3747/19560 | loss 3.679275 (+0.21z)| norm 0.2899 (+0.10z)| lr 5.62e-04 | 322.88 ms | 52.3% bf16 MFU | 1624092 tok/s
step 3748/19560 | loss 3.677579 (+0.17z)| norm 0.2642 (-0.60z)| lr 5.62e-04 | 322.88 ms | 52.3% bf16 MFU | 1624078 tok/s
step 3749/19560 | loss 3.640811 (-0.63z)| norm 0.2678 (-0.51z)| lr 5.62e-04 | 322.69 ms | 52.3% bf16 MFU | 1624111 tok/s
step 3750/19560 | loss 3.647289 (-0.49z)| norm 0.2803 (-0.18z)| lr 5.62e-04 | 323.52 ms | 52.2% bf16 MFU | 1623935 tok/s
val loss 3.639310
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2717/10042 = 0.270564
step 3751/19560 | loss 3.637692 (-0.69z)| norm 0.3625 (+2.01z)| lr 5.62e-04 | 322.59 ms | 52.3% bf16 MFU | 1624001 tok/s
step 3752/19560 | loss 3.617714 (-1.11z)| norm 0.3324 (+1.18z)| lr 5.62e-04 | 322.83 ms | 52.3% bf16 MFU | 1624002 tok/s
step 3753/19560 | loss 3.613930 (-1.18z)| norm 0.2959 (+0.20z)| lr 5.62e-04 | 323.10 ms | 52.2% bf16 MFU | 1623935 tok/s
step 3754/19560 | loss 3.663908 (-0.08z)| norm 0.3005 (+0.33z)| lr 5.62e-04 | 323.48 ms | 52.2% bf16 MFU | 1623778 tok/s
step 3755/19560 | loss 3.649072 (-0.40z)| norm 0.3041 (+0.42z)| lr 5.62e-04 | 322.73 ms | 52.3% bf16 MFU | 1623815 tok/s
step 3756/19560 | loss 3.679752 (+0.29z)| norm 0.2448 (-1.15z)| lr 5.62e-04 | 322.29 ms | 52.4% bf16 MFU | 1623962 tok/s
step 3757/19560 | loss 3.624314 (-0.94z)| norm 0.2897 (+0.04z)| lr 5.62e-04 | 323.12 ms | 52.2% bf16 MFU | 1623892 tok/s
step 3758/19560 | loss 3.632936 (-0.74z)| norm 0.2900 (+0.05z)| lr 5.62e-04 | 322.92 ms | 52.3% bf16 MFU | 1623876 tok/s
step 3759/19560 | loss 3.666486 (+0.01z)| norm 0.3511 (+1.65z)| lr 5.62e-04 | 322.66 ms | 52.3% bf16 MFU | 1623925 tok/s
step 3760/19560 | loss 3.671982 (+0.14z)| norm 0.3178 (+0.76z)| lr 5.62e-04 | 323.04 ms | 52.2% bf16 MFU | 1623878 tok/s
step 3761/19560 | loss 3.635179 (-0.68z)| norm 0.2849 (-0.11z)| lr 5.62e-04 | 322.62 ms | 52.3% bf16 MFU | 1623939 tok/s
step 3762/19560 | loss 3.585231 (-1.76z)| norm 0.3368 (+1.25z)| lr 5.62e-04 | 322.88 ms | 52.3% bf16 MFU | 1623932 tok/s
step 3763/19560 | loss 3.609230 (-1.22z)| norm 0.3236 (+0.89z)| lr 5.62e-04 | 323.00 ms | 52.3% bf16 MFU | 1623894 tok/s
step 3764/19560 | loss 3.674091 (+0.22z)| norm 0.3261 (+0.94z)| lr 5.62e-04 | 323.08 ms | 52.2% bf16 MFU | 1623839 tok/s
step 3765/19560 | loss 3.532030 (-2.81z)| norm 0.3303 (+1.04z)| lr 5.62e-04 | 322.46 ms | 52.3% bf16 MFU | 1623943 tok/s
step 3766/19560 | loss 3.652826 (-0.23z)| norm 0.3531 (+1.60z)| lr 5.62e-04 | 323.06 ms | 52.2% bf16 MFU | 1623890 tok/s
step 3767/19560 | loss 3.625540 (-0.83z)| norm 0.3455 (+1.38z)| lr 5.62e-04 | 322.75 ms | 52.3% bf16 MFU | 1623918 tok/s
step 3768/19560 | loss 3.696029 (+0.70z)| norm 0.2693 (-0.57z)| lr 5.62e-04 | 322.60 ms | 52.3% bf16 MFU | 1623982 tok/s
step 3769/19560 | loss 3.636214 (-0.59z)| norm 0.2831 (-0.21z)| lr 5.62e-04 | 323.52 ms | 52.2% bf16 MFU | 1623812 tok/s
step 3770/19560 | loss 3.661832 (-0.03z)| norm 0.2673 (-0.61z)| lr 5.62e-04 | 323.27 ms | 52.2% bf16 MFU | 1623712 tok/s
step 3771/19560 | loss 3.620548 (-0.95z)| norm 0.2802 (-0.26z)| lr 5.62e-04 | 322.56 ms | 52.3% bf16 MFU | 1623796 tok/s
step 3772/19560 | loss 3.655952 (-0.12z)| norm 0.2681 (-0.56z)| lr 5.62e-04 | 322.55 ms | 52.3% bf16 MFU | 1623879 tok/s
step 3773/19560 | loss 3.707346 (+1.08z)| norm 0.2394 (-1.31z)| lr 5.62e-04 | 323.61 ms | 52.2% bf16 MFU | 1623692 tok/s
step 3774/19560 | loss 3.661674 (+0.00z)| norm 0.2655 (-0.60z)| lr 5.62e-04 | 322.80 ms | 52.3% bf16 MFU | 1623717 tok/s
step 3775/19560 | loss 3.650508 (-0.25z)| norm 0.2508 (-0.98z)| lr 5.62e-04 | 322.97 ms | 52.3% bf16 MFU | 1623698 tok/s
step 3776/19560 | loss 3.684157 (+0.54z)| norm 0.2694 (-0.47z)| lr 5.61e-04 | 322.62 ms | 52.3% bf16 MFU | 1623768 tok/s
step 3777/19560 | loss 3.751866 (+2.08z)| norm 0.2754 (-0.31z)| lr 5.61e-04 | 322.95 ms | 52.3% bf16 MFU | 1623751 tok/s
step 3778/19560 | loss 3.675031 (+0.29z)| norm 0.2318 (-1.44z)| lr 5.61e-04 | 322.79 ms | 52.3% bf16 MFU | 1623774 tok/s
step 3779/19560 | loss 3.670704 (+0.19z)| norm 0.3181 (+0.83z)| lr 5.61e-04 | 322.78 ms | 52.3% bf16 MFU | 1623799 tok/s
step 3780/19560 | loss 3.624263 (-0.88z)| norm 0.2894 (+0.07z)| lr 5.61e-04 | 323.27 ms | 52.2% bf16 MFU | 1623701 tok/s
step 3781/19560 | loss 3.678533 (+0.37z)| norm 0.3298 (+1.12z)| lr 5.61e-04 | 323.37 ms | 52.2% bf16 MFU | 1623581 tok/s
step 3782/19560 | loss 3.673080 (+0.26z)| norm 0.3467 (+1.54z)| lr 5.61e-04 | 322.95 ms | 52.3% bf16 MFU | 1623574 tok/s
step 3783/19560 | loss 3.667310 (+0.13z)| norm 0.2838 (-0.10z)| lr 5.61e-04 | 323.43 ms | 52.2% bf16 MFU | 1623446 tok/s
step 3784/19560 | loss 3.613449 (-1.13z)| norm 0.2582 (-0.76z)| lr 5.61e-04 | 323.13 ms | 52.2% bf16 MFU | 1623401 tok/s
step 3785/19560 | loss 3.683435 (+0.53z)| norm 0.2821 (-0.15z)| lr 5.61e-04 | 323.38 ms | 52.2% bf16 MFU | 1623294 tok/s
step 3786/19560 | loss 3.664803 (+0.09z)| norm 0.2824 (-0.15z)| lr 5.61e-04 | 323.43 ms | 52.2% bf16 MFU | 1623180 tok/s
step 3787/19560 | loss 3.633282 (-0.65z)| norm 0.2984 (+0.27z)| lr 5.61e-04 | 322.70 ms | 52.3% bf16 MFU | 1623256 tok/s
step 3788/19560 | loss 3.685585 (+0.59z)| norm 0.2690 (-0.50z)| lr 5.61e-04 | 323.41 ms | 52.2% bf16 MFU | 1623150 tok/s
step 3789/19560 | loss 3.658369 (-0.05z)| norm 0.2611 (-0.70z)| lr 5.61e-04 | 322.86 ms | 52.3% bf16 MFU | 1623186 tok/s
step 3790/19560 | loss 3.635900 (-0.57z)| norm 0.2752 (-0.34z)| lr 5.61e-04 | 323.18 ms | 52.2% bf16 MFU | 1623141 tok/s
step 3791/19560 | loss 3.645669 (-0.33z)| norm 0.2590 (-0.76z)| lr 5.61e-04 | 323.04 ms | 52.2% bf16 MFU | 1623134 tok/s
step 3792/19560 | loss 3.658548 (-0.01z)| norm 0.2636 (-0.63z)| lr 5.61e-04 | 322.90 ms | 52.3% bf16 MFU | 1623161 tok/s
step 3793/19560 | loss 3.674253 (+0.38z)| norm 0.2743 (-0.35z)| lr 5.61e-04 | 322.84 ms | 52.3% bf16 MFU | 1623204 tok/s
step 3794/19560 | loss 3.697062 (+0.92z)| norm 0.2773 (-0.28z)| lr 5.61e-04 | 323.19 ms | 52.2% bf16 MFU | 1623154 tok/s
step 3795/19560 | loss 3.704190 (+1.08z)| norm 0.2832 (-0.13z)| lr 5.61e-04 | 323.08 ms | 52.2% bf16 MFU | 1623135 tok/s
step 3796/19560 | loss 3.692914 (+0.79z)| norm 0.2539 (-0.91z)| lr 5.61e-04 | 322.55 ms | 52.3% bf16 MFU | 1623251 tok/s
step 3797/19560 | loss 3.680575 (+0.48z)| norm 0.3011 (+0.33z)| lr 5.61e-04 | 322.83 ms | 52.3% bf16 MFU | 1623290 tok/s
step 3798/19560 | loss 3.647364 (-0.32z)| norm 0.2783 (-0.27z)| lr 5.61e-04 | 323.01 ms | 52.2% bf16 MFU | 1623281 tok/s
step 3799/19560 | loss 3.708030 (+1.19z)| norm 0.3240 (+0.93z)| lr 5.61e-04 | 323.30 ms | 52.2% bf16 MFU | 1623200 tok/s
step 3800/19560 | loss 3.637475 (-0.55z)| norm 0.3251 (+0.94z)| lr 5.61e-04 | 323.06 ms | 52.2% bf16 MFU | 1623184 tok/s
step 3801/19560 | loss 3.739770 (+1.96z)| norm 0.2830 (-0.16z)| lr 5.61e-04 | 322.81 ms | 52.3% bf16 MFU | 1623232 tok/s
step 3802/19560 | loss 3.655882 (-0.12z)| norm 0.2713 (-0.46z)| lr 5.61e-04 | 323.14 ms | 52.2% bf16 MFU | 1623193 tok/s
step 3803/19560 | loss 3.646423 (-0.35z)| norm 0.2617 (-0.71z)| lr 5.61e-04 | 322.96 ms | 52.3% bf16 MFU | 1623203 tok/s
step 3804/19560 | loss 3.652139 (-0.21z)| norm 0.2629 (-0.67z)| lr 5.61e-04 | 323.14 ms | 52.2% bf16 MFU | 1623166 tok/s
step 3805/19560 | loss 3.654244 (-0.16z)| norm 0.2892 (+0.02z)| lr 5.61e-04 | 322.79 ms | 52.3% bf16 MFU | 1623219 tok/s
step 3806/19560 | loss 3.625976 (-1.01z)| norm 0.2799 (-0.21z)| lr 5.61e-04 | 322.65 ms | 52.3% bf16 MFU | 1623305 tok/s
step 3807/19560 | loss 3.675209 (+0.36z)| norm 0.2621 (-0.81z)| lr 5.61e-04 | 323.66 ms | 52.1% bf16 MFU | 1623132 tok/s
step 3808/19560 | loss 3.587528 (-2.07z)| norm 0.2764 (-0.33z)| lr 5.61e-04 | 322.49 ms | 52.3% bf16 MFU | 1623262 tok/s
step 3809/19560 | loss 3.637567 (-0.68z)| norm 0.2661 (-0.66z)| lr 5.61e-04 | 323.69 ms | 52.1% bf16 MFU | 1623086 tok/s
step 3810/19560 | loss 3.700073 (+1.05z)| norm 0.2445 (-1.51z)| lr 5.61e-04 | 322.71 ms | 52.3% bf16 MFU | 1623164 tok/s
step 3811/19560 | loss 3.640188 (-0.61z)| norm 0.2481 (-1.36z)| lr 5.61e-04 | 323.08 ms | 52.2% bf16 MFU | 1623144 tok/s
step 3812/19560 | loss 3.615474 (-1.28z)| norm 0.2570 (-1.01z)| lr 5.61e-04 | 324.36 ms | 52.0% bf16 MFU | 1622807 tok/s
step 3813/19560 | loss 3.652056 (-0.25z)| norm 0.3029 (+0.74z)| lr 5.61e-04 | 323.36 ms | 52.2% bf16 MFU | 1622735 tok/s
step 3814/19560 | loss 3.723362 (+1.69z)| norm 0.3787 (+3.43z)| lr 5.61e-04 | 323.26 ms | 52.2% bf16 MFU | 1622692 tok/s
step 3815/19560 | loss 3.691061 (+0.81z)| norm 0.3938 (+3.73z)| lr 5.61e-04 | 323.14 ms | 52.2% bf16 MFU | 1622681 tok/s
step 3816/19560 | loss 3.676440 (+0.41z)| norm 0.3275 (+1.43z)| lr 5.61e-04 | 322.94 ms | 52.3% bf16 MFU | 1622722 tok/s
step 3817/19560 | loss 3.664554 (+0.08z)| norm 0.3285 (+1.44z)| lr 5.60e-04 | 323.96 ms | 52.1% bf16 MFU | 1622505 tok/s
step 3818/19560 | loss 3.611359 (-1.37z)| norm 0.3031 (+0.56z)| lr 5.60e-04 | 322.35 ms | 52.4% bf16 MFU | 1622704 tok/s
step 3819/19560 | loss 3.632732 (-0.77z)| norm 0.2810 (-0.19z)| lr 5.60e-04 | 323.90 ms | 52.1% bf16 MFU | 1622502 tok/s
step 3820/19560 | loss 3.627015 (-0.92z)| norm 0.2783 (-0.28z)| lr 5.60e-04 | 323.80 ms | 52.1% bf16 MFU | 1622337 tok/s
step 3821/19560 | loss 3.679521 (+0.55z)| norm 0.2784 (-0.27z)| lr 5.60e-04 | 323.32 ms | 52.2% bf16 MFU | 1622298 tok/s
step 3822/19560 | loss 3.617105 (-1.19z)| norm 0.3094 (+0.77z)| lr 5.60e-04 | 322.86 ms | 52.3% bf16 MFU | 1622378 tok/s
step 3823/19560 | loss 3.608981 (-1.39z)| norm 0.2715 (-0.52z)| lr 5.60e-04 | 323.84 ms | 52.1% bf16 MFU | 1622207 tok/s
step 3824/19560 | loss 3.640787 (-0.50z)| norm 0.2629 (-0.81z)| lr 5.60e-04 | 322.79 ms | 52.3% bf16 MFU | 1622310 tok/s
step 3825/19560 | loss 3.635431 (-0.65z)| norm 0.2614 (-0.85z)| lr 5.60e-04 | 322.81 ms | 52.3% bf16 MFU | 1622402 tok/s
step 3826/19560 | loss 3.642083 (-0.45z)| norm 0.2880 (+0.06z)| lr 5.60e-04 | 323.48 ms | 52.2% bf16 MFU | 1622320 tok/s
step 3827/19560 | loss 3.655002 (-0.09z)| norm 0.2599 (-0.89z)| lr 5.60e-04 | 323.30 ms | 52.2% bf16 MFU | 1622287 tok/s
step 3828/19560 | loss 3.617495 (-1.13z)| norm 0.2421 (-1.47z)| lr 5.60e-04 | 323.93 ms | 52.1% bf16 MFU | 1622100 tok/s
step 3829/19560 | loss 3.680330 (+0.63z)| norm 0.2624 (-0.78z)| lr 5.60e-04 | 322.58 ms | 52.3% bf16 MFU | 1622258 tok/s
step 3830/19560 | loss 3.838093 (+4.58z)| norm 0.3137 (+0.94z)| lr 5.60e-04 | 323.18 ms | 52.2% bf16 MFU | 1622260 tok/s
step 3831/19560 | loss 3.542315 (-2.88z)| norm 0.7182 (+8.88z)| lr 5.60e-04 | 323.43 ms | 52.2% bf16 MFU | 1622199 tok/s
step 3832/19560 | loss 3.640293 (-0.43z)| norm 0.3268 (+0.78z)| lr 5.60e-04 | 323.08 ms | 52.2% bf16 MFU | 1622229 tok/s
step 3833/19560 | loss 3.632845 (-0.63z)| norm 0.3411 (+1.07z)| lr 5.60e-04 | 322.79 ms | 52.3% bf16 MFU | 1622331 tok/s
step 3834/19560 | loss 3.677379 (+0.48z)| norm 0.3298 (+0.82z)| lr 5.60e-04 | 323.67 ms | 52.1% bf16 MFU | 1622205 tok/s
step 3835/19560 | loss 3.657063 (-0.03z)| norm 0.3076 (+0.36z)| lr 5.60e-04 | 322.51 ms | 52.3% bf16 MFU | 1622378 tok/s
step 3836/19560 | loss 3.613303 (-1.11z)| norm 0.2965 (+0.13z)| lr 5.60e-04 | 322.99 ms | 52.3% bf16 MFU | 1622421 tok/s
step 3837/19560 | loss 3.598778 (-1.45z)| norm 0.2980 (+0.15z)| lr 5.60e-04 | 323.28 ms | 52.2% bf16 MFU | 1622389 tok/s
step 3838/19560 | loss 3.665151 (+0.21z)| norm 0.3028 (+0.24z)| lr 5.60e-04 | 323.12 ms | 52.2% bf16 MFU | 1622398 tok/s
step 3839/19560 | loss 3.616221 (-1.01z)| norm 0.3080 (+0.35z)| lr 5.60e-04 | 322.93 ms | 52.3% bf16 MFU | 1622455 tok/s
step 3840/19560 | loss 3.586105 (-1.74z)| norm 0.3292 (+0.80z)| lr 5.60e-04 | 322.57 ms | 52.3% bf16 MFU | 1622599 tok/s
step 3841/19560 | loss 3.630872 (-0.62z)| norm 0.2843 (-0.13z)| lr 5.60e-04 | 323.27 ms | 52.2% bf16 MFU | 1622560 tok/s
step 3842/19560 | loss 3.687037 (+0.77z)| norm 0.2857 (-0.10z)| lr 5.60e-04 | 323.02 ms | 52.2% bf16 MFU | 1622586 tok/s
step 3843/19560 | loss 3.640893 (-0.36z)| norm 0.2797 (-0.23z)| lr 5.60e-04 | 322.51 ms | 52.3% bf16 MFU | 1622739 tok/s
step 3844/19560 | loss 3.673693 (+0.47z)| norm 0.2934 (+0.06z)| lr 5.60e-04 | 323.13 ms | 52.2% bf16 MFU | 1622727 tok/s
step 3845/19560 | loss 3.656204 (+0.05z)| norm 0.2918 (+0.03z)| lr 5.60e-04 | 322.59 ms | 52.3% bf16 MFU | 1622854 tok/s
step 3846/19560 | loss 3.616329 (-0.98z)| norm 0.2658 (-0.51z)| lr 5.60e-04 | 322.79 ms | 52.3% bf16 MFU | 1622923 tok/s
step 3847/19560 | loss 3.703316 (+1.23z)| norm 0.2859 (-0.09z)| lr 5.60e-04 | 323.00 ms | 52.3% bf16 MFU | 1622935 tok/s
step 3848/19560 | loss 3.658825 (+0.10z)| norm 0.2806 (-0.21z)| lr 5.60e-04 | 322.94 ms | 52.3% bf16 MFU | 1622963 tok/s
step 3849/19560 | loss 3.705988 (+1.32z)| norm 0.2858 (-0.10z)| lr 5.60e-04 | 323.21 ms | 52.2% bf16 MFU | 1622921 tok/s
step 3850/19560 | loss 3.681422 (+0.68z)| norm 0.2480 (-0.88z)| lr 5.60e-04 | 322.69 ms | 52.3% bf16 MFU | 1623011 tok/s
step 3851/19560 | loss 3.606838 (-1.22z)| norm 0.2621 (-0.59z)| lr 5.60e-04 | 322.40 ms | 52.3% bf16 MFU | 1623170 tok/s
step 3852/19560 | loss 3.617697 (-0.93z)| norm 0.2654 (-0.51z)| lr 5.60e-04 | 323.09 ms | 52.2% bf16 MFU | 1623148 tok/s
step 3853/19560 | loss 3.735027 (+2.01z)| norm 0.2783 (-0.25z)| lr 5.60e-04 | 322.99 ms | 52.3% bf16 MFU | 1623152 tok/s
step 3854/19560 | loss 3.628605 (-0.66z)| norm 0.2632 (-0.56z)| lr 5.60e-04 | 323.22 ms | 52.2% bf16 MFU | 1623098 tok/s
step 3855/19560 | loss 3.628300 (-0.66z)| norm 0.2427 (-0.98z)| lr 5.60e-04 | 322.95 ms | 52.3% bf16 MFU | 1623116 tok/s
step 3856/19560 | loss 3.662941 (+0.21z)| norm 0.2669 (-0.48z)| lr 5.60e-04 | 322.49 ms | 52.3% bf16 MFU | 1623248 tok/s
step 3857/19560 | loss 3.705261 (+1.26z)| norm 0.2886 (-0.04z)| lr 5.59e-04 | 322.92 ms | 52.3% bf16 MFU | 1623265 tok/s
step 3858/19560 | loss 3.749468 (+2.30z)| norm 0.2531 (-0.77z)| lr 5.59e-04 | 322.79 ms | 52.3% bf16 MFU | 1623312 tok/s
step 3859/19560 | loss 3.694173 (+0.92z)| norm 0.2796 (-0.22z)| lr 5.59e-04 | 322.90 ms | 52.3% bf16 MFU | 1623330 tok/s
step 3860/19560 | loss 3.625353 (-0.76z)| norm 0.2976 (+0.15z)| lr 5.59e-04 | 322.63 ms | 52.3% bf16 MFU | 1623415 tok/s
step 3861/19560 | loss 3.664726 (+0.20z)| norm 0.2620 (-0.58z)| lr 5.59e-04 | 322.91 ms | 52.3% bf16 MFU | 1623426 tok/s
step 3862/19560 | loss 3.807526 (+3.49z)| norm 0.2712 (-0.39z)| lr 5.59e-04 | 323.31 ms | 52.2% bf16 MFU | 1623337 tok/s
step 3863/19560 | loss 3.612859 (-1.02z)| norm 0.3044 (+0.29z)| lr 5.59e-04 | 322.85 ms | 52.3% bf16 MFU | 1623368 tok/s
step 3864/19560 | loss 3.627631 (-0.67z)| norm 0.2825 (-0.16z)| lr 5.59e-04 | 323.06 ms | 52.2% bf16 MFU | 1623344 tok/s
step 3865/19560 | loss 3.680854 (+0.56z)| norm 0.2767 (-0.28z)| lr 5.59e-04 | 322.86 ms | 52.3% bf16 MFU | 1623370 tok/s
step 3866/19560 | loss 3.667696 (+0.26z)| norm 0.2824 (-0.16z)| lr 5.59e-04 | 322.42 ms | 52.3% bf16 MFU | 1623506 tok/s
step 3867/19560 | loss 3.648294 (-0.19z)| norm 0.2755 (-0.31z)| lr 5.59e-04 | 323.48 ms | 52.2% bf16 MFU | 1623368 tok/s
step 3868/19560 | loss 3.575311 (-1.86z)| norm 0.2699 (-0.42z)| lr 5.59e-04 | 322.94 ms | 52.3% bf16 MFU | 1623374 tok/s
step 3869/19560 | loss 3.663209 (+0.16z)| norm 0.2521 (-0.79z)| lr 5.59e-04 | 322.93 ms | 52.3% bf16 MFU | 1623381 tok/s
step 3870/19560 | loss 3.654096 (-0.05z)| norm 0.2933 (+0.06z)| lr 5.59e-04 | 322.90 ms | 52.3% bf16 MFU | 1623396 tok/s
step 3871/19560 | loss 3.628921 (-0.62z)| norm 0.3078 (+0.35z)| lr 5.59e-04 | 323.08 ms | 52.2% bf16 MFU | 1623366 tok/s
step 3872/19560 | loss 3.632710 (-0.55z)| norm 0.2631 (-0.58z)| lr 5.59e-04 | 323.15 ms | 52.2% bf16 MFU | 1623319 tok/s
step 3873/19560 | loss 3.764026 (+2.46z)| norm 0.2900 (-0.02z)| lr 5.59e-04 | 322.90 ms | 52.3% bf16 MFU | 1623338 tok/s
step 3874/19560 | loss 3.633051 (-0.53z)| norm 0.2800 (-0.23z)| lr 5.59e-04 | 323.17 ms | 52.2% bf16 MFU | 1623288 tok/s
step 3875/19560 | loss 3.603438 (-1.20z)| norm 0.2717 (-0.40z)| lr 5.59e-04 | 322.94 ms | 52.3% bf16 MFU | 1623299 tok/s
step 3876/19560 | loss 3.584350 (-1.61z)| norm 0.2639 (-0.56z)| lr 5.59e-04 | 323.09 ms | 52.2% bf16 MFU | 1623270 tok/s
step 3877/19560 | loss 3.617919 (-0.84z)| norm 0.2678 (-0.48z)| lr 5.59e-04 | 322.93 ms | 52.3% bf16 MFU | 1623284 tok/s
step 3878/19560 | loss 3.605305 (-1.11z)| norm 0.2778 (-0.27z)| lr 5.59e-04 | 322.55 ms | 52.3% bf16 MFU | 1623392 tok/s
step 3879/19560 | loss 3.790177 (+2.95z)| norm 0.3577 (+1.39z)| lr 5.59e-04 | 323.22 ms | 52.2% bf16 MFU | 1623327 tok/s
step 3880/19560 | loss 3.611782 (-0.95z)| norm 0.3866 (+1.96z)| lr 5.59e-04 | 323.01 ms | 52.2% bf16 MFU | 1623316 tok/s
step 3881/19560 | loss 3.776150 (+2.56z)| norm 0.5175 (+4.27z)| lr 5.59e-04 | 322.88 ms | 52.3% bf16 MFU | 1623340 tok/s
step 3882/19560 | loss 3.627346 (-0.62z)| norm 0.3364 (+0.82z)| lr 5.59e-04 | 322.82 ms | 52.3% bf16 MFU | 1623377 tok/s
step 3883/19560 | loss 3.702940 (+0.98z)| norm 0.3670 (+1.38z)| lr 5.59e-04 | 322.76 ms | 52.3% bf16 MFU | 1623429 tok/s
step 3884/19560 | loss 3.641616 (-0.32z)| norm 0.3082 (+0.26z)| lr 5.59e-04 | 323.10 ms | 52.2% bf16 MFU | 1623391 tok/s
step 3885/19560 | loss 3.571486 (-1.78z)| norm 0.2632 (-0.58z)| lr 5.59e-04 | 322.40 ms | 52.3% bf16 MFU | 1623532 tok/s
step 3886/19560 | loss 3.572664 (-1.73z)| norm 0.9843 (+8.49z)| lr 5.59e-04 | 322.69 ms | 52.3% bf16 MFU | 1623594 tok/s
step 3887/19560 | loss 3.637591 (-0.37z)| norm 0.4306 (+1.61z)| lr 5.59e-04 | 322.98 ms | 52.3% bf16 MFU | 1623579 tok/s
step 3888/19560 | loss 3.679749 (+0.51z)| norm 0.4372 (+1.66z)| lr 5.59e-04 | 322.64 ms | 52.3% bf16 MFU | 1623651 tok/s
step 3889/19560 | loss 3.658504 (+0.06z)| norm 0.3161 (+0.18z)| lr 5.59e-04 | 322.96 ms | 52.3% bf16 MFU | 1623637 tok/s
step 3890/19560 | loss 3.618050 (-0.79z)| norm 0.3313 (+0.37z)| lr 5.59e-04 | 323.49 ms | 52.2% bf16 MFU | 1623492 tok/s
step 3891/19560 | loss 3.623894 (-0.67z)| norm 0.3100 (+0.11z)| lr 5.59e-04 | 322.78 ms | 52.3% bf16 MFU | 1623532 tok/s
step 3892/19560 | loss 3.646134 (-0.20z)| norm 0.3057 (+0.06z)| lr 5.59e-04 | 322.36 ms | 52.4% bf16 MFU | 1623675 tok/s
step 3893/19560 | loss 3.651131 (-0.12z)| norm 0.2932 (-0.09z)| lr 5.59e-04 | 322.92 ms | 52.3% bf16 MFU | 1623670 tok/s
step 3894/19560 | loss 3.618277 (-0.82z)| norm 0.2621 (-0.46z)| lr 5.59e-04 | 323.05 ms | 52.2% bf16 MFU | 1623633 tok/s
step 3895/19560 | loss 3.657338 (+0.01z)| norm 0.2758 (-0.29z)| lr 5.59e-04 | 322.89 ms | 52.3% bf16 MFU | 1623638 tok/s
step 3896/19560 | loss 3.647271 (-0.20z)| norm 0.2607 (-0.47z)| lr 5.59e-04 | 322.94 ms | 52.3% bf16 MFU | 1623630 tok/s
step 3897/19560 | loss 3.616188 (-0.86z)| norm 0.2754 (-0.29z)| lr 5.58e-04 | 322.38 ms | 52.4% bf16 MFU | 1623764 tok/s
step 3898/19560 | loss 3.573119 (-1.76z)| norm 0.2565 (-0.52z)| lr 5.58e-04 | 322.54 ms | 52.3% bf16 MFU | 1623851 tok/s
step 3899/19560 | loss 3.658716 (+0.06z)| norm 0.2661 (-0.40z)| lr 5.58e-04 | 323.46 ms | 52.2% bf16 MFU | 1623701 tok/s
step 3900/19560 | loss 3.804778 (+3.04z)| norm 0.2913 (-0.10z)| lr 5.58e-04 | 322.98 ms | 52.3% bf16 MFU | 1623680 tok/s
step 3901/19560 | loss 3.633183 (-0.48z)| norm 0.2902 (-0.11z)| lr 5.58e-04 | 322.65 ms | 52.3% bf16 MFU | 1623743 tok/s
step 3902/19560 | loss 3.692009 (+0.73z)| norm 0.2493 (-0.61z)| lr 5.58e-04 | 322.24 ms | 52.4% bf16 MFU | 1623906 tok/s
step 3903/19560 | loss 3.644405 (-0.25z)| norm 0.2755 (-0.29z)| lr 5.58e-04 | 323.11 ms | 52.2% bf16 MFU | 1623843 tok/s
step 3904/19560 | loss 3.672253 (+0.33z)| norm 0.2852 (-0.18z)| lr 5.58e-04 | 323.08 ms | 52.2% bf16 MFU | 1623791 tok/s
step 3905/19560 | loss 3.647147 (-0.18z)| norm 0.2818 (-0.22z)| lr 5.58e-04 | 323.83 ms | 52.1% bf16 MFU | 1623553 tok/s
step 3906/19560 | loss 3.659176 (+0.08z)| norm 0.2643 (-0.44z)| lr 5.58e-04 | 322.53 ms | 52.3% bf16 MFU | 1623652 tok/s
step 3907/19560 | loss 3.695656 (+0.84z)| norm 0.2552 (-0.54z)| lr 5.58e-04 | 322.19 ms | 52.4% bf16 MFU | 1623833 tok/s
step 3908/19560 | loss 3.667979 (+0.25z)| norm 0.2774 (-0.27z)| lr 5.58e-04 | 323.24 ms | 52.2% bf16 MFU | 1623739 tok/s
step 3909/19560 | loss 3.635420 (-0.42z)| norm 0.2816 (-0.21z)| lr 5.58e-04 | 322.56 ms | 52.3% bf16 MFU | 1623821 tok/s
step 3910/19560 | loss 3.649520 (-0.13z)| norm 0.2806 (-0.22z)| lr 5.58e-04 | 322.53 ms | 52.3% bf16 MFU | 1623908 tok/s
step 3911/19560 | loss 3.647323 (-0.17z)| norm 0.2740 (-0.30z)| lr 5.58e-04 | 322.45 ms | 52.3% bf16 MFU | 1624011 tok/s
step 3912/19560 | loss 3.549495 (-2.17z)| norm 0.2872 (-0.14z)| lr 5.58e-04 | 322.90 ms | 52.3% bf16 MFU | 1623995 tok/s
step 3913/19560 | loss 3.596874 (-1.18z)| norm 0.3055 (+0.08z)| lr 5.58e-04 | 322.54 ms | 52.3% bf16 MFU | 1624069 tok/s
step 3914/19560 | loss 3.727256 (+1.48z)| norm 0.2862 (-0.16z)| lr 5.58e-04 | 322.60 ms | 52.3% bf16 MFU | 1624125 tok/s
step 3915/19560 | loss 3.592384 (-1.25z)| norm 0.2648 (-0.41z)| lr 5.58e-04 | 322.68 ms | 52.3% bf16 MFU | 1624157 tok/s
step 3916/19560 | loss 3.571592 (-1.64z)| norm 0.2383 (-0.74z)| lr 5.58e-04 | 322.84 ms | 52.3% bf16 MFU | 1624150 tok/s
step 3917/19560 | loss 3.616771 (-0.73z)| norm 0.2661 (-0.40z)| lr 5.58e-04 | 322.69 ms | 52.3% bf16 MFU | 1624180 tok/s
step 3918/19560 | loss 3.629938 (-0.46z)| norm 0.2777 (-0.25z)| lr 5.58e-04 | 323.09 ms | 52.2% bf16 MFU | 1624108 tok/s
step 3919/19560 | loss 3.637212 (-0.32z)| norm 0.2581 (-0.49z)| lr 5.58e-04 | 323.13 ms | 52.2% bf16 MFU | 1624029 tok/s
step 3920/19560 | loss 3.662820 (+0.20z)| norm 0.2911 (-0.09z)| lr 5.58e-04 | 322.51 ms | 52.3% bf16 MFU | 1624111 tok/s
step 3921/19560 | loss 3.597711 (-1.09z)| norm 0.3110 (+0.15z)| lr 5.58e-04 | 322.66 ms | 52.3% bf16 MFU | 1624151 tok/s
step 3922/19560 | loss 3.627851 (-0.48z)| norm 0.3146 (+0.19z)| lr 5.58e-04 | 323.00 ms | 52.3% bf16 MFU | 1624102 tok/s
step 3923/19560 | loss 3.641779 (-0.19z)| norm 0.3052 (+0.07z)| lr 5.58e-04 | 323.44 ms | 52.2% bf16 MFU | 1623945 tok/s
step 3924/19560 | loss 3.581506 (-1.38z)| norm 0.2727 (-0.33z)| lr 5.58e-04 | 322.68 ms | 52.3% bf16 MFU | 1623987 tok/s
step 3925/19560 | loss 3.625554 (-0.49z)| norm 0.2892 (-0.13z)| lr 5.58e-04 | 322.92 ms | 52.3% bf16 MFU | 1623968 tok/s
step 3926/19560 | loss 3.625952 (-0.48z)| norm 0.2647 (-0.43z)| lr 5.58e-04 | 322.56 ms | 52.3% bf16 MFU | 1624039 tok/s
step 3927/19560 | loss 3.636221 (-0.26z)| norm 0.2509 (-0.59z)| lr 5.58e-04 | 322.70 ms | 52.3% bf16 MFU | 1624072 tok/s
step 3928/19560 | loss 3.592215 (-1.13z)| norm 0.2495 (-0.60z)| lr 5.58e-04 | 323.45 ms | 52.2% bf16 MFU | 1623914 tok/s
step 3929/19560 | loss 3.605160 (-0.86z)| norm 0.2386 (-0.72z)| lr 5.58e-04 | 323.07 ms | 52.2% bf16 MFU | 1623861 tok/s
step 3930/19560 | loss 3.640105 (-0.16z)| norm 0.2448 (-0.64z)| lr 5.58e-04 | 322.72 ms | 52.3% bf16 MFU | 1623899 tok/s
step 3931/19560 | loss 3.679430 (+0.63z)| norm 0.2521 (-0.55z)| lr 5.58e-04 | 322.89 ms | 52.3% bf16 MFU | 1623890 tok/s
step 3932/19560 | loss 3.676025 (+0.56z)| norm 0.2466 (-0.62z)| lr 5.58e-04 | 322.52 ms | 52.3% bf16 MFU | 1623976 tok/s
step 3933/19560 | loss 3.593951 (-1.08z)| norm 0.2450 (-0.63z)| lr 5.58e-04 | 322.50 ms | 52.3% bf16 MFU | 1624061 tok/s
step 3934/19560 | loss 3.646760 (-0.02z)| norm 0.2639 (-0.40z)| lr 5.58e-04 | 322.92 ms | 52.3% bf16 MFU | 1624038 tok/s
step 3935/19560 | loss 3.668716 (+0.42z)| norm 0.2792 (-0.22z)| lr 5.58e-04 | 322.87 ms | 52.3% bf16 MFU | 1624029 tok/s
step 3936/19560 | loss 3.674552 (+0.52z)| norm 0.2706 (-0.32z)| lr 5.57e-04 | 322.75 ms | 52.3% bf16 MFU | 1624049 tok/s
step 3937/19560 | loss 3.583191 (-1.30z)| norm 0.2917 (-0.07z)| lr 5.57e-04 | 322.46 ms | 52.3% bf16 MFU | 1624141 tok/s
step 3938/19560 | loss 3.658196 (+0.21z)| norm 0.2747 (-0.28z)| lr 5.57e-04 | 323.14 ms | 52.2% bf16 MFU | 1624058 tok/s
step 3939/19560 | loss 3.615187 (-0.65z)| norm 0.2818 (-0.20z)| lr 5.57e-04 | 322.71 ms | 52.3% bf16 MFU | 1624088 tok/s
step 3940/19560 | loss 3.629362 (-0.37z)| norm 0.3175 (+0.24z)| lr 5.57e-04 | 322.72 ms | 52.3% bf16 MFU | 1624114 tok/s
step 3941/19560 | loss 3.605606 (-0.84z)| norm 0.3116 (+0.16z)| lr 5.57e-04 | 322.67 ms | 52.3% bf16 MFU | 1624150 tok/s
step 3942/19560 | loss 3.625524 (-0.43z)| norm 0.2894 (-0.10z)| lr 5.57e-04 | 322.68 ms | 52.3% bf16 MFU | 1624181 tok/s
step 3943/19560 | loss 3.598871 (-0.95z)| norm 0.2495 (-0.58z)| lr 5.57e-04 | 323.11 ms | 52.2% bf16 MFU | 1624102 tok/s
step 3944/19560 | loss 3.626048 (-0.39z)| norm 0.2651 (-0.38z)| lr 5.57e-04 | 322.56 ms | 52.3% bf16 MFU | 1624167 tok/s
step 3945/19560 | loss 3.607510 (-0.76z)| norm 0.2640 (-0.39z)| lr 5.57e-04 | 322.68 ms | 52.3% bf16 MFU | 1624197 tok/s
step 3946/19560 | loss 3.676498 (+0.62z)| norm 0.2730 (-0.27z)| lr 5.57e-04 | 322.68 ms | 52.3% bf16 MFU | 1624225 tok/s
step 3947/19560 | loss 3.708236 (+1.24z)| norm 0.2663 (-0.35z)| lr 5.57e-04 | 323.52 ms | 52.2% bf16 MFU | 1624042 tok/s
step 3948/19560 | loss 3.680824 (+0.69z)| norm 0.2777 (-0.22z)| lr 5.57e-04 | 323.02 ms | 52.2% bf16 MFU | 1623994 tok/s
step 3949/19560 | loss 3.546893 (-1.95z)| norm 0.3378 (+0.52z)| lr 5.57e-04 | 322.91 ms | 52.3% bf16 MFU | 1623976 tok/s
step 3950/19560 | loss 3.628535 (-0.34z)| norm 0.3264 (+0.38z)| lr 5.57e-04 | 322.24 ms | 52.4% bf16 MFU | 1624129 tok/s
step 3951/19560 | loss 3.646073 (+0.00z)| norm 0.2812 (-0.18z)| lr 5.57e-04 | 322.95 ms | 52.3% bf16 MFU | 1624096 tok/s
step 3952/19560 | loss 3.640282 (-0.11z)| norm 0.3044 (+0.10z)| lr 5.57e-04 | 322.66 ms | 52.3% bf16 MFU | 1624135 tok/s
step 3953/19560 | loss 3.624948 (-0.41z)| norm 0.3123 (+0.19z)| lr 5.57e-04 | 322.43 ms | 52.3% bf16 MFU | 1624231 tok/s
step 3954/19560 | loss 3.628107 (-0.35z)| norm 0.2701 (-0.32z)| lr 5.57e-04 | 323.32 ms | 52.2% bf16 MFU | 1624099 tok/s
step 3955/19560 | loss 3.610330 (-0.69z)| norm 0.2740 (-0.28z)| lr 5.57e-04 | 322.94 ms | 52.3% bf16 MFU | 1624068 tok/s
step 3956/19560 | loss 3.659822 (+0.28z)| norm 0.2643 (-0.40z)| lr 5.57e-04 | 322.77 ms | 52.3% bf16 MFU | 1624081 tok/s
step 3957/19560 | loss 3.597030 (-0.95z)| norm 0.2478 (-0.60z)| lr 5.57e-04 | 322.69 ms | 52.3% bf16 MFU | 1624113 tok/s
step 3958/19560 | loss 3.623905 (-0.41z)| norm 0.2401 (-0.69z)| lr 5.57e-04 | 322.92 ms | 52.3% bf16 MFU | 1624087 tok/s
step 3959/19560 | loss 3.591009 (-1.12z)| norm 0.2882 (-0.06z)| lr 5.57e-04 | 323.42 ms | 52.2% bf16 MFU | 1623937 tok/s
step 3960/19560 | loss 3.636445 (-0.16z)| norm 0.2562 (-0.50z)| lr 5.57e-04 | 323.00 ms | 52.3% bf16 MFU | 1623900 tok/s
step 3961/19560 | loss 3.660935 (+0.36z)| norm 0.2640 (-0.38z)| lr 5.57e-04 | 323.13 ms | 52.2% bf16 MFU | 1623830 tok/s
step 3962/19560 | loss 3.629664 (-0.30z)| norm 0.2803 (-0.15z)| lr 5.57e-04 | 323.23 ms | 52.2% bf16 MFU | 1623739 tok/s
step 3963/19560 | loss 3.627267 (-0.34z)| norm 0.2790 (-0.16z)| lr 5.57e-04 | 322.40 ms | 52.3% bf16 MFU | 1623862 tok/s
step 3964/19560 | loss 3.607651 (-0.76z)| norm 0.2640 (-0.37z)| lr 5.57e-04 | 322.79 ms | 52.3% bf16 MFU | 1623881 tok/s
step 3965/19560 | loss 3.600327 (-0.91z)| norm 0.2709 (-0.27z)| lr 5.57e-04 | 323.28 ms | 52.2% bf16 MFU | 1623776 tok/s
step 3966/19560 | loss 3.612387 (-0.65z)| norm 0.2598 (-0.42z)| lr 5.57e-04 | 322.53 ms | 52.3% bf16 MFU | 1623864 tok/s
step 3967/19560 | loss 3.602875 (-0.85z)| norm 0.2711 (-0.26z)| lr 5.57e-04 | 323.22 ms | 52.2% bf16 MFU | 1623775 tok/s
step 3968/19560 | loss 3.678384 (+0.74z)| norm 0.2589 (-0.42z)| lr 5.57e-04 | 322.51 ms | 52.3% bf16 MFU | 1623869 tok/s
step 3969/19560 | loss 3.725144 (+1.70z)| norm 0.2705 (-0.26z)| lr 5.57e-04 | 322.65 ms | 52.3% bf16 MFU | 1623923 tok/s
step 3970/19560 | loss 3.677567 (+0.70z)| norm 0.2834 (-0.08z)| lr 5.57e-04 | 323.39 ms | 52.2% bf16 MFU | 1623789 tok/s
step 3971/19560 | loss 3.658990 (+0.31z)| norm 0.2975 (+0.11z)| lr 5.57e-04 | 323.23 ms | 52.2% bf16 MFU | 1623700 tok/s
step 3972/19560 | loss 3.703323 (+1.23z)| norm 0.2941 (+0.07z)| lr 5.57e-04 | 323.03 ms | 52.2% bf16 MFU | 1623665 tok/s
step 3973/19560 | loss 3.644889 (+0.01z)| norm 0.2802 (-0.12z)| lr 5.57e-04 | 323.31 ms | 52.2% bf16 MFU | 1623562 tok/s
step 3974/19560 | loss 3.609233 (-0.74z)| norm 0.2491 (-0.55z)| lr 5.57e-04 | 322.55 ms | 52.3% bf16 MFU | 1623656 tok/s
step 3975/19560 | loss 3.659533 (+0.33z)| norm 0.2922 (+0.04z)| lr 5.56e-04 | 323.06 ms | 52.2% bf16 MFU | 1623616 tok/s
step 3976/19560 | loss 3.613551 (-0.64z)| norm 0.2988 (+0.13z)| lr 5.56e-04 | 323.33 ms | 52.2% bf16 MFU | 1623512 tok/s
step 3977/19560 | loss 3.639420 (-0.08z)| norm 0.3196 (+0.42z)| lr 5.56e-04 | 322.85 ms | 52.3% bf16 MFU | 1623533 tok/s
step 3978/19560 | loss 3.619028 (-0.50z)| norm 0.2959 (+0.08z)| lr 5.56e-04 | 323.41 ms | 52.2% bf16 MFU | 1623412 tok/s
step 3979/19560 | loss 3.595455 (-1.00z)| norm 0.3093 (+0.26z)| lr 5.56e-04 | 322.83 ms | 52.3% bf16 MFU | 1623444 tok/s
step 3980/19560 | loss 3.688497 (+0.96z)| norm 0.2929 (+0.03z)| lr 5.56e-04 | 322.87 ms | 52.3% bf16 MFU | 1623463 tok/s
step 3981/19560 | loss 3.573047 (-1.47z)| norm 0.3087 (+0.25z)| lr 5.56e-04 | 323.96 ms | 52.1% bf16 MFU | 1623209 tok/s
step 3982/19560 | loss 3.639865 (-0.05z)| norm 0.3175 (+0.37z)| lr 5.56e-04 | 322.94 ms | 52.3% bf16 MFU | 1623223 tok/s
step 3983/19560 | loss 3.656484 (+0.30z)| norm 0.3295 (+0.52z)| lr 5.56e-04 | 323.16 ms | 52.2% bf16 MFU | 1623181 tok/s
step 3984/19560 | loss 3.606674 (-0.75z)| norm 0.2987 (+0.09z)| lr 5.56e-04 | 323.03 ms | 52.2% bf16 MFU | 1623174 tok/s
step 3985/19560 | loss 3.618655 (-0.48z)| norm 0.2974 (+0.07z)| lr 5.56e-04 | 322.42 ms | 52.3% bf16 MFU | 1623319 tok/s
step 3986/19560 | loss 3.613683 (-0.58z)| norm 0.2553 (-0.51z)| lr 5.56e-04 | 323.61 ms | 52.2% bf16 MFU | 1623159 tok/s
step 3987/19560 | loss 3.662063 (+0.49z)| norm 0.2772 (-0.21z)| lr 5.56e-04 | 322.56 ms | 52.3% bf16 MFU | 1623272 tok/s
step 3988/19560 | loss 3.731322 (+1.96z)| norm 0.2626 (-0.41z)| lr 5.56e-04 | 323.15 ms | 52.2% bf16 MFU | 1623231 tok/s
step 3989/19560 | loss 3.703114 (+1.33z)| norm 0.2755 (-0.23z)| lr 5.56e-04 | 323.08 ms | 52.2% bf16 MFU | 1623209 tok/s
step 3990/19560 | loss 3.586591 (-1.18z)| norm 0.2805 (-0.16z)| lr 5.56e-04 | 322.63 ms | 52.3% bf16 MFU | 1623302 tok/s
step 3991/19560 | loss 3.595653 (-0.98z)| norm 0.2536 (-0.53z)| lr 5.56e-04 | 322.79 ms | 52.3% bf16 MFU | 1623348 tok/s
step 3992/19560 | loss 3.642853 (+0.08z)| norm 0.2980 (+0.09z)| lr 5.56e-04 | 322.89 ms | 52.3% bf16 MFU | 1623366 tok/s
step 3993/19560 | loss 3.638317 (-0.01z)| norm 0.2961 (+0.06z)| lr 5.56e-04 | 323.67 ms | 52.1% bf16 MFU | 1623190 tok/s
step 3994/19560 | loss 3.666627 (+0.63z)| norm 0.2734 (-0.25z)| lr 5.56e-04 | 323.15 ms | 52.2% bf16 MFU | 1623151 tok/s
step 3995/19560 | loss 3.742148 (+2.27z)| norm 0.3110 (+0.26z)| lr 5.56e-04 | 322.32 ms | 52.4% bf16 MFU | 1623324 tok/s
step 3996/19560 | loss 3.589920 (-1.10z)| norm 0.3364 (+0.61z)| lr 5.56e-04 | 323.07 ms | 52.2% bf16 MFU | 1623299 tok/s
step 3997/19560 | loss 3.690125 (+1.11z)| norm 0.3116 (+0.26z)| lr 5.56e-04 | 323.24 ms | 52.2% bf16 MFU | 1623232 tok/s
step 3998/19560 | loss 3.637177 (-0.06z)| norm 0.3201 (+0.37z)| lr 5.56e-04 | 322.66 ms | 52.3% bf16 MFU | 1623316 tok/s
step 3999/19560 | loss 3.663365 (+0.51z)| norm 0.2875 (-0.08z)| lr 5.56e-04 | 322.83 ms | 52.3% bf16 MFU | 1623351 tok/s
step 4000/19560 | loss 3.665452 (+0.55z)| norm 0.2955 (+0.03z)| lr 5.56e-04 | 322.88 ms | 52.3% bf16 MFU | 1623372 tok/s
val loss 3.624114
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2745/10042 = 0.273352
step 4001/19560 | loss 3.593017 (-1.04z)| norm 0.2969 (+0.05z)| lr 5.56e-04 | 322.27 ms | 52.4% bf16 MFU | 1623546 tok/s
step 4002/19560 | loss 3.655041 (+0.36z)| norm 0.2813 (-0.17z)| lr 5.56e-04 | 322.97 ms | 52.3% bf16 MFU | 1623535 tok/s
step 4003/19560 | loss 3.594423 (-1.01z)| norm 0.2633 (-0.42z)| lr 5.56e-04 | 323.16 ms | 52.2% bf16 MFU | 1623476 tok/s
step 4004/19560 | loss 3.614021 (-0.57z)| norm 0.2821 (-0.16z)| lr 5.56e-04 | 322.54 ms | 52.3% bf16 MFU | 1623578 tok/s
step 4005/19560 | loss 3.593353 (-1.03z)| norm 0.3035 (+0.13z)| lr 5.56e-04 | 322.69 ms | 52.3% bf16 MFU | 1623636 tok/s
step 4006/19560 | loss 3.650147 (+0.24z)| norm 0.2715 (-0.31z)| lr 5.56e-04 | 323.30 ms | 52.2% bf16 MFU | 1623537 tok/s
step 4007/19560 | loss 3.629933 (-0.20z)| norm 0.2393 (-0.74z)| lr 5.56e-04 | 322.71 ms | 52.3% bf16 MFU | 1623593 tok/s
step 4008/19560 | loss 3.642234 (+0.09z)| norm 0.2895 (-0.04z)| lr 5.56e-04 | 323.05 ms | 52.2% bf16 MFU | 1623560 tok/s
step 4009/19560 | loss 3.608412 (-0.71z)| norm 0.2626 (-0.40z)| lr 5.56e-04 | 323.03 ms | 52.2% bf16 MFU | 1623534 tok/s
step 4010/19560 | loss 3.601538 (-0.88z)| norm 0.2632 (-0.38z)| lr 5.56e-04 | 323.34 ms | 52.2% bf16 MFU | 1623431 tok/s
step 4011/19560 | loss 3.587490 (-1.21z)| norm 0.2492 (-0.58z)| lr 5.56e-04 | 322.44 ms | 52.3% bf16 MFU | 1623560 tok/s
step 4012/19560 | loss 3.615353 (-0.51z)| norm 0.2425 (-0.66z)| lr 5.56e-04 | 323.22 ms | 52.2% bf16 MFU | 1623486 tok/s
step 4013/19560 | loss 3.591149 (-1.12z)| norm 0.2634 (-0.36z)| lr 5.55e-04 | 322.95 ms | 52.3% bf16 MFU | 1623483 tok/s
step 4014/19560 | loss 3.713456 (+1.90z)| norm 0.3150 (+1.08z)| lr 5.55e-04 | 323.28 ms | 52.2% bf16 MFU | 1623399 tok/s
step 4015/19560 | loss 3.644744 (+0.19z)| norm 0.2763 (-0.20z)| lr 5.55e-04 | 322.40 ms | 52.3% bf16 MFU | 1623539 tok/s
step 4016/19560 | loss 3.650062 (+0.33z)| norm 0.2735 (-0.30z)| lr 5.55e-04 | 323.50 ms | 52.2% bf16 MFU | 1623395 tok/s
step 4017/19560 | loss 3.611217 (-0.63z)| norm 0.2889 (+0.38z)| lr 5.55e-04 | 322.76 ms | 52.3% bf16 MFU | 1623445 tok/s
step 4018/19560 | loss 3.580933 (-1.37z)| norm 0.2496 (-1.33z)| lr 5.55e-04 | 323.15 ms | 52.2% bf16 MFU | 1623395 tok/s
step 4019/19560 | loss 3.637861 (+0.04z)| norm 0.2490 (-1.34z)| lr 5.55e-04 | 322.77 ms | 52.3% bf16 MFU | 1623443 tok/s
step 4020/19560 | loss 3.607496 (-0.71z)| norm 0.2617 (-0.76z)| lr 5.55e-04 | 323.01 ms | 52.2% bf16 MFU | 1623427 tok/s
step 4021/19560 | loss 3.628456 (-0.18z)| norm 0.2423 (-1.59z)| lr 5.55e-04 | 323.07 ms | 52.2% bf16 MFU | 1623396 tok/s
step 4022/19560 | loss 3.667736 (+0.78z)| norm 0.2518 (-1.17z)| lr 5.55e-04 | 323.17 ms | 52.2% bf16 MFU | 1623344 tok/s
step 4023/19560 | loss 3.672827 (+0.90z)| norm 0.2572 (-0.92z)| lr 5.55e-04 | 322.87 ms | 52.3% bf16 MFU | 1623368 tok/s
step 4024/19560 | loss 3.638294 (+0.05z)| norm 0.2877 (+0.41z)| lr 5.55e-04 | 322.72 ms | 52.3% bf16 MFU | 1623430 tok/s
step 4025/19560 | loss 3.616563 (-0.49z)| norm 0.2563 (-0.96z)| lr 5.55e-04 | 322.80 ms | 52.3% bf16 MFU | 1623469 tok/s
step 4026/19560 | loss 3.665863 (+0.72z)| norm 0.2940 (+0.68z)| lr 5.55e-04 | 322.99 ms | 52.3% bf16 MFU | 1623456 tok/s
step 4027/19560 | loss 3.585206 (-1.27z)| norm 0.2986 (+0.87z)| lr 5.55e-04 | 322.82 ms | 52.3% bf16 MFU | 1623487 tok/s
step 4028/19560 | loss 3.613826 (-0.56z)| norm 0.2726 (-0.26z)| lr 5.55e-04 | 322.60 ms | 52.3% bf16 MFU | 1623572 tok/s
step 4029/19560 | loss 3.616753 (-0.48z)| norm 0.2598 (-0.81z)| lr 5.55e-04 | 322.24 ms | 52.4% bf16 MFU | 1623743 tok/s
step 4030/19560 | loss 3.682773 (+1.28z)| norm 0.2974 (+0.82z)| lr 5.55e-04 | 322.98 ms | 52.3% bf16 MFU | 1623721 tok/s
step 4031/19560 | loss 3.654122 (+0.51z)| norm 0.3191 (+1.74z)| lr 5.55e-04 | 322.91 ms | 52.3% bf16 MFU | 1623716 tok/s
step 4032/19560 | loss 3.645009 (+0.28z)| norm 0.2729 (-0.26z)| lr 5.55e-04 | 322.52 ms | 52.3% bf16 MFU | 1623811 tok/s
step 4033/19560 | loss 3.702865 (+1.79z)| norm 0.2976 (+0.80z)| lr 5.55e-04 | 323.02 ms | 52.2% bf16 MFU | 1623775 tok/s
step 4034/19560 | loss 3.665070 (+0.79z)| norm 0.3243 (+1.92z)| lr 5.55e-04 | 322.76 ms | 52.3% bf16 MFU | 1623805 tok/s
step 4035/19560 | loss 3.565159 (-1.81z)| norm 0.3134 (+1.43z)| lr 5.55e-04 | 322.61 ms | 52.3% bf16 MFU | 1623872 tok/s
step 4036/19560 | loss 3.618596 (-0.40z)| norm 0.3074 (+1.15z)| lr 5.55e-04 | 322.54 ms | 52.3% bf16 MFU | 1623954 tok/s
step 4037/19560 | loss 3.643104 (+0.25z)| norm 0.2772 (-0.13z)| lr 5.55e-04 | 322.99 ms | 52.3% bf16 MFU | 1623918 tok/s
step 4038/19560 | loss 3.616293 (-0.45z)| norm 0.3027 (+0.94z)| lr 5.55e-04 | 322.36 ms | 52.4% bf16 MFU | 1624041 tok/s
step 4039/19560 | loss 3.653923 (+0.54z)| norm 0.3027 (+0.93z)| lr 5.55e-04 | 322.32 ms | 52.4% bf16 MFU | 1624171 tok/s
step 4040/19560 | loss 3.650120 (+0.42z)| norm 0.2928 (+0.51z)| lr 5.55e-04 | 323.33 ms | 52.2% bf16 MFU | 1624038 tok/s
step 4041/19560 | loss 3.620551 (-0.38z)| norm 0.2627 (-0.74z)| lr 5.55e-04 | 322.34 ms | 52.4% bf16 MFU | 1624161 tok/s
step 4042/19560 | loss 3.641786 (+0.22z)| norm 0.3145 (+1.42z)| lr 5.55e-04 | 322.90 ms | 52.3% bf16 MFU | 1624139 tok/s
step 4043/19560 | loss 3.576379 (-1.58z)| norm 0.2728 (-0.32z)| lr 5.55e-04 | 322.88 ms | 52.3% bf16 MFU | 1624122 tok/s
step 4044/19560 | loss 3.602767 (-0.87z)| norm 0.2997 (+0.79z)| lr 5.55e-04 | 322.57 ms | 52.3% bf16 MFU | 1624184 tok/s
step 4045/19560 | loss 3.544895 (-2.40z)| norm 0.2456 (-1.48z)| lr 5.55e-04 | 322.64 ms | 52.3% bf16 MFU | 1624224 tok/s
step 4046/19560 | loss 3.595658 (-1.01z)| norm 0.2607 (-0.84z)| lr 5.55e-04 | 322.76 ms | 52.3% bf16 MFU | 1624232 tok/s
step 4047/19560 | loss 3.628889 (-0.11z)| norm 0.2854 (+0.19z)| lr 5.55e-04 | 323.09 ms | 52.2% bf16 MFU | 1624156 tok/s
step 4048/19560 | loss 3.711565 (+2.08z)| norm 0.2841 (+0.13z)| lr 5.55e-04 | 322.89 ms | 52.3% bf16 MFU | 1624136 tok/s
step 4049/19560 | loss 3.613978 (-0.52z)| norm 0.2467 (-1.42z)| lr 5.55e-04 | 322.54 ms | 52.3% bf16 MFU | 1624205 tok/s
step 4050/19560 | loss 3.687603 (+1.42z)| norm 0.2742 (-0.25z)| lr 5.55e-04 | 322.71 ms | 52.3% bf16 MFU | 1624226 tok/s
step 4051/19560 | loss 3.619924 (-0.37z)| norm 0.3114 (+1.32z)| lr 5.54e-04 | 323.07 ms | 52.2% bf16 MFU | 1624156 tok/s
step 4052/19560 | loss 3.645320 (+0.29z)| norm 0.2692 (-0.46z)| lr 5.54e-04 | 322.82 ms | 52.3% bf16 MFU | 1624152 tok/s
step 4053/19560 | loss 3.621313 (-0.35z)| norm 0.2559 (-1.01z)| lr 5.54e-04 | 323.05 ms | 52.2% bf16 MFU | 1624090 tok/s
step 4054/19560 | loss 3.656263 (+0.58z)| norm 0.2672 (-0.53z)| lr 5.54e-04 | 322.25 ms | 52.4% bf16 MFU | 1624234 tok/s
step 4055/19560 | loss 3.651044 (+0.43z)| norm 0.2782 (-0.08z)| lr 5.54e-04 | 322.58 ms | 52.3% bf16 MFU | 1624287 tok/s
step 4056/19560 | loss 3.664264 (+0.77z)| norm 0.3634 (+3.35z)| lr 5.54e-04 | 322.81 ms | 52.3% bf16 MFU | 1624280 tok/s
step 4057/19560 | loss 3.571424 (-1.68z)| norm 0.3098 (+1.16z)| lr 5.54e-04 | 322.68 ms | 52.3% bf16 MFU | 1624306 tok/s
step 4058/19560 | loss 3.644574 (+0.25z)| norm 0.3031 (+0.87z)| lr 5.54e-04 | 323.11 ms | 52.2% bf16 MFU | 1624221 tok/s
step 4059/19560 | loss 3.642491 (+0.21z)| norm 0.2965 (+0.59z)| lr 5.54e-04 | 323.11 ms | 52.2% bf16 MFU | 1624140 tok/s
step 4060/19560 | loss 3.653914 (+0.52z)| norm 0.2872 (+0.19z)| lr 5.54e-04 | 322.77 ms | 52.3% bf16 MFU | 1624150 tok/s
step 4061/19560 | loss 3.618897 (-0.43z)| norm 0.2947 (+0.49z)| lr 5.54e-04 | 323.06 ms | 52.2% bf16 MFU | 1624086 tok/s
step 4062/19560 | loss 3.627296 (-0.20z)| norm 0.2899 (+0.28z)| lr 5.54e-04 | 323.08 ms | 52.2% bf16 MFU | 1624021 tok/s
step 4063/19560 | loss 3.608376 (-0.69z)| norm 0.2725 (-0.45z)| lr 5.54e-04 | 322.34 ms | 52.4% bf16 MFU | 1624144 tok/s
step 4064/19560 | loss 3.590637 (-1.15z)| norm 0.2811 (-0.09z)| lr 5.54e-04 | 323.32 ms | 52.2% bf16 MFU | 1624016 tok/s
step 4065/19560 | loss 3.580518 (-1.42z)| norm 0.2655 (-0.74z)| lr 5.54e-04 | 322.09 ms | 52.4% bf16 MFU | 1624204 tok/s
step 4066/19560 | loss 3.727915 (+2.46z)| norm 0.3258 (+1.77z)| lr 5.54e-04 | 322.94 ms | 52.3% bf16 MFU | 1624166 tok/s
step 4067/19560 | loss 3.679946 (+1.18z)| norm 0.2959 (+0.51z)| lr 5.54e-04 | 323.07 ms | 52.2% bf16 MFU | 1624098 tok/s
step 4068/19560 | loss 3.588843 (-1.18z)| norm 0.2962 (+0.54z)| lr 5.54e-04 | 322.54 ms | 52.3% bf16 MFU | 1624169 tok/s
step 4069/19560 | loss 3.629129 (-0.14z)| norm 0.2968 (+0.57z)| lr 5.54e-04 | 323.10 ms | 52.2% bf16 MFU | 1624096 tok/s
step 4070/19560 | loss 3.626384 (-0.21z)| norm 0.3028 (+0.81z)| lr 5.54e-04 | 322.80 ms | 52.3% bf16 MFU | 1624101 tok/s
step 4071/19560 | loss 3.702760 (+1.74z)| norm 0.3383 (+2.25z)| lr 5.54e-04 | 322.71 ms | 52.3% bf16 MFU | 1624127 tok/s
step 4072/19560 | loss 3.549523 (-2.15z)| norm 0.2866 (+0.10z)| lr 5.54e-04 | 323.04 ms | 52.2% bf16 MFU | 1624070 tok/s
step 4073/19560 | loss 3.563224 (-1.78z)| norm 0.2937 (+0.38z)| lr 5.54e-04 | 323.29 ms | 52.2% bf16 MFU | 1623954 tok/s
step 4074/19560 | loss 3.646889 (+0.32z)| norm 0.2792 (-0.22z)| lr 5.54e-04 | 322.70 ms | 52.3% bf16 MFU | 1623990 tok/s
step 4075/19560 | loss 3.608414 (-0.63z)| norm 0.2676 (-0.71z)| lr 5.54e-04 | 322.61 ms | 52.3% bf16 MFU | 1624046 tok/s
step 4076/19560 | loss 3.604089 (-0.73z)| norm 0.2779 (-0.28z)| lr 5.54e-04 | 322.58 ms | 52.3% bf16 MFU | 1624108 tok/s
step 4077/19560 | loss 3.597394 (-0.93z)| norm 0.2933 (+0.39z)| lr 5.54e-04 | 323.14 ms | 52.2% bf16 MFU | 1624027 tok/s
step 4078/19560 | loss 3.612101 (-0.54z)| norm 0.2720 (-0.51z)| lr 5.54e-04 | 322.44 ms | 52.3% bf16 MFU | 1624125 tok/s
step 4079/19560 | loss 3.581243 (-1.32z)| norm 0.2474 (-1.54z)| lr 5.54e-04 | 322.58 ms | 52.3% bf16 MFU | 1624183 tok/s
step 4080/19560 | loss 3.630858 (-0.04z)| norm 0.2585 (-1.05z)| lr 5.54e-04 | 322.94 ms | 52.3% bf16 MFU | 1624148 tok/s
step 4081/19560 | loss 3.624035 (-0.21z)| norm 0.2609 (-0.93z)| lr 5.54e-04 | 322.36 ms | 52.4% bf16 MFU | 1624261 tok/s
step 4082/19560 | loss 3.597451 (-0.89z)| norm 0.2489 (-1.43z)| lr 5.54e-04 | 323.08 ms | 52.2% bf16 MFU | 1624188 tok/s
step 4083/19560 | loss 3.639673 (+0.19z)| norm 0.2660 (-0.70z)| lr 5.54e-04 | 322.52 ms | 52.3% bf16 MFU | 1624259 tok/s
step 4084/19560 | loss 3.626272 (-0.15z)| norm 0.2638 (-0.79z)| lr 5.54e-04 | 322.65 ms | 52.3% bf16 MFU | 1624293 tok/s
step 4085/19560 | loss 3.641899 (+0.24z)| norm 0.2809 (-0.08z)| lr 5.54e-04 | 322.88 ms | 52.3% bf16 MFU | 1624269 tok/s
step 4086/19560 | loss 3.630301 (-0.06z)| norm 0.2863 (+0.14z)| lr 5.54e-04 | 322.86 ms | 52.3% bf16 MFU | 1624250 tok/s
step 4087/19560 | loss 3.638008 (+0.13z)| norm 0.2599 (-0.99z)| lr 5.54e-04 | 322.44 ms | 52.3% bf16 MFU | 1624337 tok/s
step 4088/19560 | loss 3.568530 (-1.64z)| norm 0.2638 (-0.83z)| lr 5.54e-04 | 322.50 ms | 52.3% bf16 MFU | 1624404 tok/s
step 4089/19560 | loss 3.585500 (-1.18z)| norm 0.2721 (-0.47z)| lr 5.53e-04 | 323.50 ms | 52.2% bf16 MFU | 1624218 tok/s
step 4090/19560 | loss 3.594686 (-0.94z)| norm 0.2594 (-1.01z)| lr 5.53e-04 | 322.56 ms | 52.3% bf16 MFU | 1624278 tok/s
step 4091/19560 | loss 3.649438 (+0.45z)| norm 0.2629 (-0.85z)| lr 5.53e-04 | 322.63 ms | 52.3% bf16 MFU | 1624316 tok/s
step 4092/19560 | loss 3.605963 (-0.65z)| norm 0.2731 (-0.42z)| lr 5.53e-04 | 323.18 ms | 52.2% bf16 MFU | 1624214 tok/s
step 4093/19560 | loss 3.589080 (-1.08z)| norm 0.2532 (-1.26z)| lr 5.53e-04 | 322.62 ms | 52.3% bf16 MFU | 1624258 tok/s
step 4094/19560 | loss 3.612295 (-0.49z)| norm 0.2507 (-1.36z)| lr 5.53e-04 | 322.41 ms | 52.3% bf16 MFU | 1624353 tok/s
step 4095/19560 | loss 3.616704 (-0.38z)| norm 0.2703 (-0.52z)| lr 5.53e-04 | 322.88 ms | 52.3% bf16 MFU | 1624324 tok/s
step 4096/19560 | loss 3.634635 (+0.09z)| norm 0.2627 (-0.85z)| lr 5.53e-04 | 323.06 ms | 52.2% bf16 MFU | 1624252 tok/s
step 4097/19560 | loss 3.609053 (-0.56z)| norm 0.2419 (-1.71z)| lr 5.53e-04 | 323.14 ms | 52.2% bf16 MFU | 1624162 tok/s
step 4098/19560 | loss 3.557957 (-1.86z)| norm 0.2742 (-0.35z)| lr 5.53e-04 | 322.88 ms | 52.3% bf16 MFU | 1624145 tok/s
step 4099/19560 | loss 3.572682 (-1.45z)| norm 0.2359 (-1.92z)| lr 5.53e-04 | 323.50 ms | 52.2% bf16 MFU | 1623971 tok/s
step 4100/19560 | loss 3.604041 (-0.63z)| norm 0.2469 (-1.43z)| lr 5.53e-04 | 322.42 ms | 52.3% bf16 MFU | 1624079 tok/s
step 4101/19560 | loss 3.579966 (-1.24z)| norm 0.2688 (-0.52z)| lr 5.53e-04 | 322.70 ms | 52.3% bf16 MFU | 1624109 tok/s
step 4102/19560 | loss 3.605384 (-0.57z)| norm 0.2736 (-0.33z)| lr 5.53e-04 | 322.92 ms | 52.3% bf16 MFU | 1624082 tok/s
step 4103/19560 | loss 3.641677 (+0.37z)| norm 0.3008 (+0.80z)| lr 5.53e-04 | 322.42 ms | 52.3% bf16 MFU | 1624183 tok/s
step 4104/19560 | loss 3.587112 (-1.04z)| norm 0.2789 (-0.11z)| lr 5.53e-04 | 322.92 ms | 52.3% bf16 MFU | 1624153 tok/s
step 4105/19560 | loss 3.593425 (-0.86z)| norm 0.2579 (-0.97z)| lr 5.53e-04 | 322.77 ms | 52.3% bf16 MFU | 1624163 tok/s
step 4106/19560 | loss 3.634370 (+0.19z)| norm 0.2990 (+0.75z)| lr 5.53e-04 | 322.47 ms | 52.3% bf16 MFU | 1624247 tok/s
step 4107/19560 | loss 3.632100 (+0.13z)| norm 0.3096 (+1.20z)| lr 5.53e-04 | 322.78 ms | 52.3% bf16 MFU | 1624250 tok/s
step 4108/19560 | loss 3.646509 (+0.52z)| norm 0.2906 (+0.40z)| lr 5.53e-04 | 323.11 ms | 52.2% bf16 MFU | 1624169 tok/s
step 4109/19560 | loss 3.511515 (-2.93z)| norm 0.2697 (-0.47z)| lr 5.53e-04 | 322.76 ms | 52.3% bf16 MFU | 1624179 tok/s
step 4110/19560 | loss 3.597666 (-0.72z)| norm 0.2470 (-1.40z)| lr 5.53e-04 | 322.59 ms | 52.3% bf16 MFU | 1624234 tok/s
step 4111/19560 | loss 3.578649 (-1.19z)| norm 0.2943 (+0.62z)| lr 5.53e-04 | 323.01 ms | 52.2% bf16 MFU | 1624179 tok/s
step 4112/19560 | loss 3.559188 (-1.65z)| norm 0.2547 (-1.06z)| lr 5.53e-04 | 322.93 ms | 52.3% bf16 MFU | 1624148 tok/s
step 4113/19560 | loss 3.516365 (-2.63z)| norm 0.2969 (+0.74z)| lr 5.53e-04 | 323.24 ms | 52.2% bf16 MFU | 1624040 tok/s
step 4114/19560 | loss 3.614078 (-0.25z)| norm 0.3136 (+1.43z)| lr 5.53e-04 | 322.80 ms | 52.3% bf16 MFU | 1624048 tok/s
step 4115/19560 | loss 3.643498 (+0.47z)| norm 0.2928 (+0.54z)| lr 5.53e-04 | 322.55 ms | 52.3% bf16 MFU | 1624118 tok/s
step 4116/19560 | loss 3.813227 (+4.37z)| norm 0.2974 (+0.73z)| lr 5.53e-04 | 322.70 ms | 52.3% bf16 MFU | 1624146 tok/s
step 4117/19560 | loss 3.583923 (-0.93z)| norm 0.3049 (+1.03z)| lr 5.53e-04 | 322.74 ms | 52.3% bf16 MFU | 1624163 tok/s
step 4118/19560 | loss 3.550241 (-1.70z)| norm 0.2954 (+0.62z)| lr 5.53e-04 | 322.83 ms | 52.3% bf16 MFU | 1624156 tok/s
step 4119/19560 | loss 3.629765 (+0.14z)| norm 0.3321 (+2.12z)| lr 5.53e-04 | 322.59 ms | 52.3% bf16 MFU | 1624212 tok/s
step 4120/19560 | loss 3.624552 (+0.02z)| norm 0.2791 (-0.09z)| lr 5.53e-04 | 322.39 ms | 52.3% bf16 MFU | 1624313 tok/s
step 4121/19560 | loss 3.607483 (-0.37z)| norm 0.2877 (+0.27z)| lr 5.53e-04 | 322.21 ms | 52.4% bf16 MFU | 1624457 tok/s
step 4122/19560 | loss 3.599349 (-0.55z)| norm 0.2958 (+0.61z)| lr 5.53e-04 | 322.91 ms | 52.3% bf16 MFU | 1624416 tok/s
step 4123/19560 | loss 3.569139 (-1.25z)| norm 0.2762 (-0.20z)| lr 5.53e-04 | 322.42 ms | 52.3% bf16 MFU | 1624500 tok/s
step 4124/19560 | loss 3.611614 (-0.24z)| norm 0.2638 (-0.72z)| lr 5.53e-04 | 323.10 ms | 52.2% bf16 MFU | 1624408 tok/s
step 4125/19560 | loss 3.563298 (-1.38z)| norm 0.2702 (-0.43z)| lr 5.53e-04 | 322.92 ms | 52.3% bf16 MFU | 1624368 tok/s
step 4126/19560 | loss 3.587513 (-0.79z)| norm 0.2755 (-0.19z)| lr 5.52e-04 | 322.26 ms | 52.4% bf16 MFU | 1624494 tok/s
step 4127/19560 | loss 3.593126 (-0.64z)| norm 0.2853 (+0.24z)| lr 5.52e-04 | 323.09 ms | 52.2% bf16 MFU | 1624405 tok/s
step 4128/19560 | loss 3.687693 (+1.62z)| norm 0.2788 (-0.04z)| lr 5.52e-04 | 322.82 ms | 52.3% bf16 MFU | 1624388 tok/s
step 4129/19560 | loss 3.573114 (-1.11z)| norm 0.2681 (-0.50z)| lr 5.52e-04 | 323.08 ms | 52.2% bf16 MFU | 1624307 tok/s
step 4130/19560 | loss 3.703206 (+1.96z)| norm 0.2942 (+0.64z)| lr 5.52e-04 | 322.56 ms | 52.3% bf16 MFU | 1624361 tok/s
step 4131/19560 | loss 3.726842 (+2.44z)| norm 0.3054 (+1.12z)| lr 5.52e-04 | 322.35 ms | 52.4% bf16 MFU | 1624465 tok/s
step 4132/19560 | loss 3.580533 (-0.93z)| norm 0.2941 (+0.62z)| lr 5.52e-04 | 323.10 ms | 52.2% bf16 MFU | 1624376 tok/s
step 4133/19560 | loss 3.583084 (-0.87z)| norm 0.2588 (-0.91z)| lr 5.52e-04 | 323.74 ms | 52.1% bf16 MFU | 1624132 tok/s
step 4134/19560 | loss 3.584373 (-0.82z)| norm 0.2694 (-0.44z)| lr 5.52e-04 | 322.63 ms | 52.3% bf16 MFU | 1624177 tok/s
step 4135/19560 | loss 3.681573 (+1.38z)| norm 0.2756 (-0.19z)| lr 5.52e-04 | 322.44 ms | 52.3% bf16 MFU | 1624268 tok/s
step 4136/19560 | loss 3.662638 (+0.95z)| norm 0.2739 (-0.26z)| lr 5.52e-04 | 323.07 ms | 52.2% bf16 MFU | 1624197 tok/s
step 4137/19560 | loss 3.628286 (+0.16z)| norm 0.2699 (-0.44z)| lr 5.52e-04 | 323.06 ms | 52.2% bf16 MFU | 1624132 tok/s
step 4138/19560 | loss 3.608376 (-0.29z)| norm 0.2631 (-0.74z)| lr 5.52e-04 | 322.48 ms | 52.3% bf16 MFU | 1624214 tok/s
step 4139/19560 | loss 3.649589 (+0.64z)| norm 0.2695 (-0.47z)| lr 5.52e-04 | 322.70 ms | 52.3% bf16 MFU | 1624239 tok/s
step 4140/19560 | loss 3.640210 (+0.42z)| norm 0.2719 (-0.37z)| lr 5.52e-04 | 322.93 ms | 52.3% bf16 MFU | 1624204 tok/s
step 4141/19560 | loss 3.704181 (+1.83z)| norm 0.2737 (-0.30z)| lr 5.52e-04 | 323.31 ms | 52.2% bf16 MFU | 1624075 tok/s
step 4142/19560 | loss 3.573242 (-1.10z)| norm 0.2548 (-1.13z)| lr 5.52e-04 | 322.99 ms | 52.3% bf16 MFU | 1624033 tok/s
step 4143/19560 | loss 3.639866 (+0.42z)| norm 0.2686 (-0.50z)| lr 5.52e-04 | 322.78 ms | 52.3% bf16 MFU | 1624045 tok/s
step 4144/19560 | loss 3.598634 (-0.51z)| norm 0.2962 (+0.74z)| lr 5.52e-04 | 322.91 ms | 52.3% bf16 MFU | 1624024 tok/s
step 4145/19560 | loss 3.597631 (-0.53z)| norm 0.2870 (+0.32z)| lr 5.52e-04 | 323.67 ms | 52.1% bf16 MFU | 1623814 tok/s
step 4146/19560 | loss 3.564604 (-1.28z)| norm 0.2580 (-1.00z)| lr 5.52e-04 | 322.03 ms | 52.4% bf16 MFU | 1624027 tok/s
step 4147/19560 | loss 3.643326 (+0.51z)| norm 0.3021 (+0.99z)| lr 5.52e-04 | 322.89 ms | 52.3% bf16 MFU | 1624012 tok/s
step 4148/19560 | loss 3.564430 (-1.26z)| norm 0.2838 (+0.15z)| lr 5.52e-04 | 323.11 ms | 52.2% bf16 MFU | 1623941 tok/s
step 4149/19560 | loss 3.641378 (+0.46z)| norm 0.2611 (-0.91z)| lr 5.52e-04 | 322.84 ms | 52.3% bf16 MFU | 1623944 tok/s
step 4150/19560 | loss 3.544142 (-1.69z)| norm 0.2869 (+0.27z)| lr 5.52e-04 | 322.82 ms | 52.3% bf16 MFU | 1623952 tok/s
step 4151/19560 | loss 3.581851 (-0.83z)| norm 0.2795 (-0.08z)| lr 5.52e-04 | 322.78 ms | 52.3% bf16 MFU | 1623968 tok/s
step 4152/19560 | loss 3.638568 (+0.44z)| norm 0.2725 (-0.40z)| lr 5.52e-04 | 322.84 ms | 52.3% bf16 MFU | 1623970 tok/s
step 4153/19560 | loss 3.532677 (-1.89z)| norm 0.2742 (-0.33z)| lr 5.52e-04 | 322.75 ms | 52.3% bf16 MFU | 1623993 tok/s
step 4154/19560 | loss 3.601058 (-0.37z)| norm 0.2769 (-0.19z)| lr 5.52e-04 | 323.06 ms | 52.2% bf16 MFU | 1623938 tok/s
step 4155/19560 | loss 3.650210 (+0.71z)| norm 0.2992 (+0.85z)| lr 5.52e-04 | 323.24 ms | 52.2% bf16 MFU | 1623840 tok/s
step 4156/19560 | loss 3.593151 (-0.56z)| norm 0.3412 (+2.71z)| lr 5.52e-04 | 322.54 ms | 52.3% bf16 MFU | 1623923 tok/s
step 4157/19560 | loss 3.554747 (-1.38z)| norm 0.2733 (-0.39z)| lr 5.52e-04 | 323.13 ms | 52.2% bf16 MFU | 1623853 tok/s
step 4158/19560 | loss 3.647127 (+0.66z)| norm 0.2837 (+0.09z)| lr 5.52e-04 | 322.93 ms | 52.3% bf16 MFU | 1623838 tok/s
step 4159/19560 | loss 3.644315 (+0.60z)| norm 0.2691 (-0.56z)| lr 5.52e-04 | 322.64 ms | 52.3% bf16 MFU | 1623896 tok/s
step 4160/19560 | loss 3.560921 (-1.23z)| norm 0.2448 (-1.66z)| lr 5.52e-04 | 322.69 ms | 52.3% bf16 MFU | 1623938 tok/s
step 4161/19560 | loss 3.572039 (-0.97z)| norm 0.2661 (-0.67z)| lr 5.52e-04 | 322.91 ms | 52.3% bf16 MFU | 1623923 tok/s
step 4162/19560 | loss 3.627092 (+0.26z)| norm 0.2523 (-1.29z)| lr 5.52e-04 | 323.03 ms | 52.2% bf16 MFU | 1623879 tok/s
step 4163/19560 | loss 3.744982 (+2.79z)| norm 0.2740 (-0.28z)| lr 5.51e-04 | 323.04 ms | 52.2% bf16 MFU | 1623834 tok/s
step 4164/19560 | loss 3.638915 (+0.48z)| norm 0.2573 (-1.04z)| lr 5.51e-04 | 322.85 ms | 52.3% bf16 MFU | 1623839 tok/s
step 4165/19560 | loss 3.601285 (-0.33z)| norm 0.2874 (+0.37z)| lr 5.51e-04 | 322.97 ms | 52.3% bf16 MFU | 1623813 tok/s
step 4166/19560 | loss 3.597833 (-0.41z)| norm 0.2493 (-1.39z)| lr 5.51e-04 | 323.00 ms | 52.3% bf16 MFU | 1623782 tok/s
step 4167/19560 | loss 3.589050 (-0.59z)| norm 0.2794 (+0.02z)| lr 5.51e-04 | 323.22 ms | 52.2% bf16 MFU | 1623697 tok/s
step 4168/19560 | loss 3.627823 (+0.26z)| norm 0.2817 (+0.13z)| lr 5.51e-04 | 323.10 ms | 52.2% bf16 MFU | 1623645 tok/s
step 4169/19560 | loss 3.666702 (+1.10z)| norm 0.2396 (-1.81z)| lr 5.51e-04 | 322.84 ms | 52.3% bf16 MFU | 1623661 tok/s
step 4170/19560 | loss 3.612550 (-0.07z)| norm 0.2746 (-0.18z)| lr 5.51e-04 | 322.73 ms | 52.3% bf16 MFU | 1623706 tok/s
step 4171/19560 | loss 3.636220 (+0.43z)| norm 0.2834 (+0.23z)| lr 5.51e-04 | 322.76 ms | 52.3% bf16 MFU | 1623741 tok/s
step 4172/19560 | loss 3.704060 (+1.87z)| norm 0.2916 (+0.62z)| lr 5.51e-04 | 323.48 ms | 52.2% bf16 MFU | 1623592 tok/s
step 4173/19560 | loss 3.598729 (-0.41z)| norm 0.2903 (+0.55z)| lr 5.51e-04 | 323.98 ms | 52.1% bf16 MFU | 1623326 tok/s
step 4174/19560 | loss 3.666920 (+1.05z)| norm 0.2686 (-0.48z)| lr 5.51e-04 | 322.20 ms | 52.4% bf16 MFU | 1623522 tok/s
step 4175/19560 | loss 3.627440 (+0.20z)| norm 0.3054 (+1.25z)| lr 5.51e-04 | 323.16 ms | 52.2% bf16 MFU | 1623464 tok/s
step 4176/19560 | loss 3.558077 (-1.29z)| norm 0.3142 (+1.63z)| lr 5.51e-04 | 323.08 ms | 52.2% bf16 MFU | 1623428 tok/s
step 4177/19560 | loss 3.660540 (+0.94z)| norm 0.3146 (+1.63z)| lr 5.51e-04 | 322.77 ms | 52.3% bf16 MFU | 1623473 tok/s
step 4178/19560 | loss 3.563473 (-1.16z)| norm 0.2900 (+0.47z)| lr 5.51e-04 | 323.11 ms | 52.2% bf16 MFU | 1623431 tok/s
step 4179/19560 | loss 3.632526 (+0.35z)| norm 0.2794 (-0.01z)| lr 5.51e-04 | 322.64 ms | 52.3% bf16 MFU | 1623510 tok/s
step 4180/19560 | loss 3.674088 (+1.25z)| norm 0.2890 (+0.44z)| lr 5.51e-04 | 322.82 ms | 52.3% bf16 MFU | 1623539 tok/s
step 4181/19560 | loss 3.567482 (-1.06z)| norm 0.2827 (+0.13z)| lr 5.51e-04 | 323.37 ms | 52.2% bf16 MFU | 1623428 tok/s
step 4182/19560 | loss 3.595584 (-0.44z)| norm 0.2962 (+0.76z)| lr 5.51e-04 | 322.76 ms | 52.3% bf16 MFU | 1623477 tok/s
step 4183/19560 | loss 3.643428 (+0.60z)| norm 0.2891 (+0.42z)| lr 5.51e-04 | 322.62 ms | 52.3% bf16 MFU | 1623557 tok/s
step 4184/19560 | loss 3.615085 (-0.01z)| norm 0.2909 (+0.56z)| lr 5.51e-04 | 323.01 ms | 52.2% bf16 MFU | 1623535 tok/s
step 4185/19560 | loss 3.576340 (-0.85z)| norm 0.2546 (-1.25z)| lr 5.51e-04 | 323.11 ms | 52.2% bf16 MFU | 1623491 tok/s
step 4186/19560 | loss 3.644481 (+0.63z)| norm 0.2705 (-0.43z)| lr 5.51e-04 | 322.92 ms | 52.3% bf16 MFU | 1623495 tok/s
step 4187/19560 | loss 3.574047 (-0.89z)| norm 0.2869 (+0.41z)| lr 5.51e-04 | 322.72 ms | 52.3% bf16 MFU | 1623550 tok/s
step 4188/19560 | loss 3.622020 (+0.16z)| norm 0.2436 (-1.76z)| lr 5.51e-04 | 322.96 ms | 52.3% bf16 MFU | 1623541 tok/s
step 4189/19560 | loss 3.571959 (-0.92z)| norm 0.2899 (+0.57z)| lr 5.51e-04 | 323.08 ms | 52.2% bf16 MFU | 1623502 tok/s
step 4190/19560 | loss 3.555382 (-1.26z)| norm 0.2695 (-0.45z)| lr 5.51e-04 | 322.97 ms | 52.3% bf16 MFU | 1623493 tok/s
step 4191/19560 | loss 3.630452 (+0.36z)| norm 0.2668 (-0.59z)| lr 5.51e-04 | 323.14 ms | 52.2% bf16 MFU | 1623442 tok/s
step 4192/19560 | loss 3.601705 (-0.27z)| norm 0.2674 (-0.54z)| lr 5.51e-04 | 323.11 ms | 52.2% bf16 MFU | 1623403 tok/s
step 4193/19560 | loss 3.692235 (+1.66z)| norm 0.2803 (+0.10z)| lr 5.51e-04 | 323.90 ms | 52.1% bf16 MFU | 1623166 tok/s
step 4194/19560 | loss 3.597083 (-0.37z)| norm 0.2724 (-0.29z)| lr 5.51e-04 | 322.93 ms | 52.3% bf16 MFU | 1623184 tok/s
step 4195/19560 | loss 3.709996 (+2.09z)| norm 0.2763 (-0.08z)| lr 5.51e-04 | 323.42 ms | 52.2% bf16 MFU | 1623080 tok/s
step 4196/19560 | loss 3.652454 (+0.82z)| norm 0.2667 (-0.56z)| lr 5.51e-04 | 323.65 ms | 52.1% bf16 MFU | 1622923 tok/s
step 4197/19560 | loss 3.582052 (-0.70z)| norm 0.2667 (-0.55z)| lr 5.51e-04 | 323.05 ms | 52.2% bf16 MFU | 1622922 tok/s
step 4198/19560 | loss 3.628586 (+0.31z)| norm 0.2564 (-1.07z)| lr 5.51e-04 | 324.29 ms | 52.0% bf16 MFU | 1622613 tok/s
step 4199/19560 | loss 3.596868 (-0.36z)| norm 0.2727 (-0.20z)| lr 5.50e-04 | 322.84 ms | 52.3% bf16 MFU | 1622683 tok/s
step 4200/19560 | loss 3.611749 (-0.05z)| norm 0.2428 (-1.79z)| lr 5.50e-04 | 322.37 ms | 52.4% bf16 MFU | 1622866 tok/s
step 4201/19560 | loss 3.562517 (-1.14z)| norm 0.2606 (-0.82z)| lr 5.50e-04 | 323.14 ms | 52.2% bf16 MFU | 1622847 tok/s
step 4202/19560 | loss 3.523859 (-1.95z)| norm 0.2811 (+0.28z)| lr 5.50e-04 | 323.95 ms | 52.1% bf16 MFU | 1622625 tok/s
step 4203/19560 | loss 3.606107 (-0.15z)| norm 0.2812 (+0.28z)| lr 5.50e-04 | 323.31 ms | 52.2% bf16 MFU | 1622574 tok/s
step 4204/19560 | loss 3.554815 (-1.26z)| norm 0.2573 (-0.99z)| lr 5.50e-04 | 323.16 ms | 52.2% bf16 MFU | 1622564 tok/s
step 4205/19560 | loss 3.613616 (+0.02z)| norm 0.2582 (-0.93z)| lr 5.50e-04 | 323.12 ms | 52.2% bf16 MFU | 1622565 tok/s
step 4206/19560 | loss 3.623657 (+0.24z)| norm 0.2805 (+0.26z)| lr 5.50e-04 | 324.08 ms | 52.1% bf16 MFU | 1622325 tok/s
step 4207/19560 | loss 3.655248 (+0.91z)| norm 0.2869 (+0.59z)| lr 5.50e-04 | 322.80 ms | 52.3% bf16 MFU | 1622417 tok/s
step 4208/19560 | loss 3.626151 (+0.28z)| norm 0.3135 (+1.98z)| lr 5.50e-04 | 322.81 ms | 52.3% bf16 MFU | 1622503 tok/s
step 4209/19560 | loss 3.584778 (-0.61z)| norm 0.3019 (+1.34z)| lr 5.50e-04 | 323.57 ms | 52.2% bf16 MFU | 1622394 tok/s
step 4210/19560 | loss 3.585268 (-0.60z)| norm 0.2965 (+1.04z)| lr 5.50e-04 | 323.02 ms | 52.2% bf16 MFU | 1622429 tok/s
step 4211/19560 | loss 3.576159 (-0.78z)| norm 0.2615 (-0.82z)| lr 5.50e-04 | 322.83 ms | 52.3% bf16 MFU | 1622509 tok/s
step 4212/19560 | loss 3.577580 (-0.74z)| norm 0.2660 (-0.59z)| lr 5.50e-04 | 323.59 ms | 52.2% bf16 MFU | 1622395 tok/s
step 4213/19560 | loss 3.566657 (-0.97z)| norm 0.2728 (-0.22z)| lr 5.50e-04 | 323.31 ms | 52.2% bf16 MFU | 1622356 tok/s
step 4214/19560 | loss 3.522942 (-1.87z)| norm 0.2533 (-1.24z)| lr 5.50e-04 | 323.34 ms | 52.2% bf16 MFU | 1622312 tok/s
step 4215/19560 | loss 3.546978 (-1.33z)| norm 0.2836 (+0.35z)| lr 5.50e-04 | 323.29 ms | 52.2% bf16 MFU | 1622283 tok/s
step 4216/19560 | loss 3.575742 (-0.72z)| norm 0.3001 (+1.21z)| lr 5.50e-04 | 323.50 ms | 52.2% bf16 MFU | 1622201 tok/s
step 4217/19560 | loss 3.620598 (+0.22z)| norm 0.2959 (+0.98z)| lr 5.50e-04 | 323.82 ms | 52.1% bf16 MFU | 1622044 tok/s
step 4218/19560 | loss 3.603521 (-0.14z)| norm 0.2851 (+0.40z)| lr 5.50e-04 | 322.72 ms | 52.3% bf16 MFU | 1622172 tok/s
step 4219/19560 | loss 3.600808 (-0.19z)| norm 0.2722 (-0.29z)| lr 5.50e-04 | 322.83 ms | 52.3% bf16 MFU | 1622265 tok/s
step 4220/19560 | loss 3.566720 (-0.91z)| norm 0.2919 (+0.74z)| lr 5.50e-04 | 323.40 ms | 52.2% bf16 MFU | 1622210 tok/s
step 4221/19560 | loss 3.711264 (+2.10z)| norm 0.3233 (+2.34z)| lr 5.50e-04 | 323.11 ms | 52.2% bf16 MFU | 1622232 tok/s
step 4222/19560 | loss 3.600213 (-0.21z)| norm 0.3254 (+2.39z)| lr 5.50e-04 | 323.21 ms | 52.2% bf16 MFU | 1622225 tok/s
step 4223/19560 | loss 3.547559 (-1.29z)| norm 0.3079 (+1.46z)| lr 5.50e-04 | 322.84 ms | 52.3% bf16 MFU | 1622312 tok/s
step 4224/19560 | loss 3.629761 (+0.41z)| norm 0.3244 (+2.24z)| lr 5.50e-04 | 322.88 ms | 52.3% bf16 MFU | 1622387 tok/s
step 4225/19560 | loss 3.586833 (-0.47z)| norm 0.2967 (+0.84z)| lr 5.50e-04 | 323.30 ms | 52.2% bf16 MFU | 1622352 tok/s
step 4226/19560 | loss 3.539608 (-1.44z)| norm 0.2739 (-0.31z)| lr 5.50e-04 | 322.45 ms | 52.3% bf16 MFU | 1622531 tok/s
step 4227/19560 | loss 3.600838 (-0.18z)| norm 0.2671 (-0.68z)| lr 5.50e-04 | 323.46 ms | 52.2% bf16 MFU | 1622447 tok/s
step 4228/19560 | loss 3.697851 (+1.78z)| norm 0.2971 (+0.85z)| lr 5.50e-04 | 323.10 ms | 52.2% bf16 MFU | 1622460 tok/s
step 4229/19560 | loss 3.636356 (+0.52z)| norm 0.2607 (-1.04z)| lr 5.50e-04 | 322.84 ms | 52.3% bf16 MFU | 1622536 tok/s
step 4230/19560 | loss 3.676650 (+1.32z)| norm 0.2742 (-0.34z)| lr 5.50e-04 | 322.69 ms | 52.3% bf16 MFU | 1622646 tok/s
step 4231/19560 | loss 3.611029 (-0.00z)| norm 0.2652 (-0.79z)| lr 5.50e-04 | 323.46 ms | 52.2% bf16 MFU | 1622557 tok/s
step 4232/19560 | loss 3.525924 (-1.70z)| norm 0.2778 (-0.14z)| lr 5.50e-04 | 322.93 ms | 52.3% bf16 MFU | 1622605 tok/s
step 4233/19560 | loss 3.610830 (-0.00z)| norm 0.2677 (-0.67z)| lr 5.50e-04 | 323.10 ms | 52.2% bf16 MFU | 1622608 tok/s
step 4234/19560 | loss 3.523885 (-1.71z)| norm 0.9589 (+10.73z)| lr 5.50e-04 | 322.41 ms | 52.3% bf16 MFU | 1622786 tok/s
step 4235/19560 | loss 3.665288 (+1.09z)| norm 0.3516 (+1.04z)| lr 5.50e-04 | 322.79 ms | 52.3% bf16 MFU | 1622859 tok/s
step 4236/19560 | loss 3.707662 (+1.90z)| norm 0.3345 (+0.76z)| lr 5.49e-04 | 323.25 ms | 52.2% bf16 MFU | 1622811 tok/s
step 4237/19560 | loss 3.547343 (-1.26z)| norm 0.3123 (+0.41z)| lr 5.49e-04 | 323.06 ms | 52.2% bf16 MFU | 1622815 tok/s
step 4238/19560 | loss 3.579733 (-0.61z)| norm 0.2893 (+0.04z)| lr 5.49e-04 | 322.26 ms | 52.4% bf16 MFU | 1623019 tok/s
step 4239/19560 | loss 3.590334 (-0.41z)| norm 0.3225 (+0.56z)| lr 5.49e-04 | 322.57 ms | 52.3% bf16 MFU | 1623136 tok/s
step 4240/19560 | loss 3.619441 (+0.16z)| norm 0.2887 (+0.02z)| lr 5.49e-04 | 323.68 ms | 52.1% bf16 MFU | 1622968 tok/s
step 4241/19560 | loss 3.639861 (+0.55z)| norm 0.2837 (-0.06z)| lr 5.49e-04 | 322.27 ms | 52.4% bf16 MFU | 1623162 tok/s
step 4242/19560 | loss 3.650142 (+0.75z)| norm 0.2769 (-0.16z)| lr 5.49e-04 | 322.68 ms | 52.3% bf16 MFU | 1623244 tok/s
step 4243/19560 | loss 3.678531 (+1.31z)| norm 0.2791 (-0.12z)| lr 5.49e-04 | 323.96 ms | 52.1% bf16 MFU | 1623000 tok/s
step 4244/19560 | loss 3.586909 (-0.52z)| norm 0.2927 (+0.09z)| lr 5.49e-04 | 322.32 ms | 52.4% bf16 MFU | 1623180 tok/s
step 4245/19560 | loss 3.597649 (-0.29z)| norm 0.2908 (+0.06z)| lr 5.49e-04 | 323.20 ms | 52.2% bf16 MFU | 1623130 tok/s
step 4246/19560 | loss 3.683958 (+1.52z)| norm 0.2891 (+0.04z)| lr 5.49e-04 | 323.00 ms | 52.3% bf16 MFU | 1623134 tok/s
step 4247/19560 | loss 3.609339 (-0.06z)| norm 0.2636 (-0.36z)| lr 5.49e-04 | 322.41 ms | 52.3% bf16 MFU | 1623285 tok/s
step 4248/19560 | loss 3.606239 (-0.12z)| norm 0.2791 (-0.11z)| lr 5.49e-04 | 322.53 ms | 52.3% bf16 MFU | 1623398 tok/s
step 4249/19560 | loss 3.612806 (+0.02z)| norm 0.2877 (+0.02z)| lr 5.49e-04 | 322.86 ms | 52.3% bf16 MFU | 1623422 tok/s
step 4250/19560 | loss 3.643720 (+0.66z)| norm 0.3073 (+0.33z)| lr 5.49e-04 | 323.10 ms | 52.2% bf16 MFU | 1623386 tok/s
val loss 3.606812
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2756/10042 = 0.274447
step 4251/19560 | loss 3.657717 (+0.95z)| norm 0.3025 (+0.25z)| lr 5.49e-04 | 322.33 ms | 52.4% bf16 MFU | 1623544 tok/s
step 4252/19560 | loss 3.633719 (+0.43z)| norm 0.2784 (-0.13z)| lr 5.49e-04 | 322.96 ms | 52.3% bf16 MFU | 1623536 tok/s
step 4253/19560 | loss 3.515887 (-2.03z)| norm 0.2628 (-0.38z)| lr 5.49e-04 | 322.70 ms | 52.3% bf16 MFU | 1623594 tok/s
step 4254/19560 | loss 3.603077 (-0.21z)| norm 0.2782 (-0.13z)| lr 5.49e-04 | 322.56 ms | 52.3% bf16 MFU | 1623684 tok/s
step 4255/19560 | loss 3.534624 (-1.62z)| norm 0.2785 (-0.13z)| lr 5.49e-04 | 323.56 ms | 52.2% bf16 MFU | 1623519 tok/s
step 4256/19560 | loss 3.624863 (+0.27z)| norm 0.2354 (-0.81z)| lr 5.49e-04 | 323.15 ms | 52.2% bf16 MFU | 1623465 tok/s
step 4257/19560 | loss 3.629890 (+0.36z)| norm 0.2715 (-0.23z)| lr 5.49e-04 | 323.00 ms | 52.3% bf16 MFU | 1623452 tok/s
step 4258/19560 | loss 3.670438 (+1.23z)| norm 0.2597 (-0.42z)| lr 5.49e-04 | 322.79 ms | 52.3% bf16 MFU | 1623492 tok/s
step 4259/19560 | loss 3.604557 (-0.15z)| norm 0.2503 (-0.56z)| lr 5.49e-04 | 323.14 ms | 52.2% bf16 MFU | 1623440 tok/s
step 4260/19560 | loss 3.636955 (+0.55z)| norm 0.2665 (-0.30z)| lr 5.49e-04 | 322.26 ms | 52.4% bf16 MFU | 1623614 tok/s
step 4261/19560 | loss 3.661551 (+1.07z)| norm 0.2683 (-0.27z)| lr 5.49e-04 | 323.42 ms | 52.2% bf16 MFU | 1623487 tok/s
step 4262/19560 | loss 3.585674 (-0.58z)| norm 0.2496 (-0.56z)| lr 5.49e-04 | 323.09 ms | 52.2% bf16 MFU | 1623449 tok/s
step 4263/19560 | loss 3.566324 (-0.98z)| norm 0.2599 (-0.40z)| lr 5.49e-04 | 323.06 ms | 52.2% bf16 MFU | 1623420 tok/s
step 4264/19560 | loss 3.618252 (+0.16z)| norm 0.2507 (-0.54z)| lr 5.49e-04 | 322.74 ms | 52.3% bf16 MFU | 1623474 tok/s
step 4265/19560 | loss 3.591113 (-0.43z)| norm 0.2588 (-0.41z)| lr 5.49e-04 | 323.49 ms | 52.2% bf16 MFU | 1623336 tok/s
step 4266/19560 | loss 3.635366 (+0.53z)| norm 0.2817 (-0.05z)| lr 5.49e-04 | 322.52 ms | 52.3% bf16 MFU | 1623449 tok/s
step 4267/19560 | loss 3.679836 (+1.49z)| norm 0.2879 (+0.04z)| lr 5.49e-04 | 323.14 ms | 52.2% bf16 MFU | 1623400 tok/s
step 4268/19560 | loss 3.727754 (+2.46z)| norm 0.2627 (-0.35z)| lr 5.49e-04 | 323.38 ms | 52.2% bf16 MFU | 1623294 tok/s
step 4269/19560 | loss 3.619023 (+0.17z)| norm 0.2581 (-0.42z)| lr 5.49e-04 | 323.51 ms | 52.2% bf16 MFU | 1623160 tok/s
step 4270/19560 | loss 3.710155 (+2.08z)| norm 0.2616 (-0.37z)| lr 5.49e-04 | 323.26 ms | 52.2% bf16 MFU | 1623097 tok/s
step 4271/19560 | loss 3.587511 (-0.52z)| norm 0.2637 (-0.34z)| lr 5.48e-04 | 322.76 ms | 52.3% bf16 MFU | 1623163 tok/s
step 4272/19560 | loss 3.576011 (-0.76z)| norm 0.2824 (-0.04z)| lr 5.48e-04 | 323.04 ms | 52.2% bf16 MFU | 1623155 tok/s
step 4273/19560 | loss 3.591442 (-0.43z)| norm 0.2808 (-0.06z)| lr 5.48e-04 | 323.01 ms | 52.3% bf16 MFU | 1623155 tok/s
step 4274/19560 | loss 3.563662 (-1.02z)| norm 0.2715 (-0.21z)| lr 5.48e-04 | 322.84 ms | 52.3% bf16 MFU | 1623196 tok/s
step 4275/19560 | loss 3.679440 (+1.42z)| norm 0.2967 (+0.19z)| lr 5.48e-04 | 322.64 ms | 52.3% bf16 MFU | 1623286 tok/s
step 4276/19560 | loss 3.642288 (+0.63z)| norm 0.2843 (-0.01z)| lr 5.48e-04 | 323.40 ms | 52.2% bf16 MFU | 1623181 tok/s
step 4277/19560 | loss 3.581959 (-0.64z)| norm 0.2723 (-0.20z)| lr 5.48e-04 | 322.92 ms | 52.3% bf16 MFU | 1623202 tok/s
step 4278/19560 | loss 3.643831 (+0.66z)| norm 0.2789 (-0.09z)| lr 5.48e-04 | 323.11 ms | 52.2% bf16 MFU | 1623173 tok/s
step 4279/19560 | loss 3.608216 (-0.11z)| norm 0.2640 (-0.33z)| lr 5.48e-04 | 323.20 ms | 52.2% bf16 MFU | 1623124 tok/s
step 4280/19560 | loss 3.537209 (-1.59z)| norm 0.3384 (+0.84z)| lr 5.48e-04 | 322.87 ms | 52.3% bf16 MFU | 1623159 tok/s
step 4281/19560 | loss 3.562867 (-1.06z)| norm 0.3518 (+1.03z)| lr 5.48e-04 | 323.54 ms | 52.2% bf16 MFU | 1623024 tok/s
step 4282/19560 | loss 3.586231 (-0.56z)| norm 0.3467 (+0.94z)| lr 5.48e-04 | 322.67 ms | 52.3% bf16 MFU | 1623116 tok/s
step 4283/19560 | loss 3.649903 (+0.79z)| norm 0.3307 (+0.69z)| lr 5.48e-04 | 322.97 ms | 52.3% bf16 MFU | 1623127 tok/s
step 4284/19560 | loss 3.584245 (-0.60z)| norm 0.3466 (+0.93z)| lr 5.48e-04 | 323.51 ms | 52.2% bf16 MFU | 1623003 tok/s
step 4285/19560 | loss 3.644145 (+0.66z)| norm 0.3004 (+0.21z)| lr 5.48e-04 | 322.84 ms | 52.3% bf16 MFU | 1623053 tok/s
step 4286/19560 | loss 3.623230 (+0.22z)| norm 0.3185 (+0.49z)| lr 5.48e-04 | 322.41 ms | 52.3% bf16 MFU | 1623207 tok/s
step 4287/19560 | loss 3.653535 (+0.86z)| norm 0.3339 (+0.72z)| lr 5.48e-04 | 323.05 ms | 52.2% bf16 MFU | 1623192 tok/s
step 4288/19560 | loss 3.606359 (-0.15z)| norm 0.2942 (+0.09z)| lr 5.48e-04 | 322.81 ms | 52.3% bf16 MFU | 1623240 tok/s
step 4289/19560 | loss 3.577531 (-0.77z)| norm 0.2887 (+0.01z)| lr 5.48e-04 | 322.92 ms | 52.3% bf16 MFU | 1623257 tok/s
step 4290/19560 | loss 3.632134 (+0.40z)| norm 0.2968 (+0.13z)| lr 5.48e-04 | 322.88 ms | 52.3% bf16 MFU | 1623284 tok/s
step 4291/19560 | loss 3.552191 (-1.31z)| norm 0.2796 (-0.14z)| lr 5.48e-04 | 322.87 ms | 52.3% bf16 MFU | 1623312 tok/s
step 4292/19560 | loss 3.608504 (-0.07z)| norm 0.3051 (+0.25z)| lr 5.48e-04 | 323.14 ms | 52.2% bf16 MFU | 1623271 tok/s
step 4293/19560 | loss 3.608592 (-0.07z)| norm 0.2989 (+0.15z)| lr 5.48e-04 | 322.59 ms | 52.3% bf16 MFU | 1623369 tok/s
step 4294/19560 | loss 3.675426 (+1.38z)| norm 0.3083 (+0.29z)| lr 5.48e-04 | 322.94 ms | 52.3% bf16 MFU | 1623374 tok/s
step 4295/19560 | loss 3.515800 (-2.06z)| norm 0.2854 (-0.07z)| lr 5.48e-04 | 323.29 ms | 52.2% bf16 MFU | 1623291 tok/s
step 4296/19560 | loss 3.555757 (-1.19z)| norm 0.2833 (-0.10z)| lr 5.48e-04 | 323.14 ms | 52.2% bf16 MFU | 1623249 tok/s
step 4297/19560 | loss 3.551528 (-1.26z)| norm 0.2792 (-0.17z)| lr 5.48e-04 | 322.49 ms | 52.3% bf16 MFU | 1623374 tok/s
step 4298/19560 | loss 3.692727 (+1.73z)| norm 0.2744 (-0.24z)| lr 5.48e-04 | 323.11 ms | 52.2% bf16 MFU | 1623336 tok/s
step 4299/19560 | loss 3.658694 (+1.00z)| norm 0.2795 (-0.16z)| lr 5.48e-04 | 322.57 ms | 52.3% bf16 MFU | 1623435 tok/s
step 4300/19560 | loss 3.567279 (-0.91z)| norm 0.3043 (+0.22z)| lr 5.48e-04 | 322.89 ms | 52.3% bf16 MFU | 1623449 tok/s
step 4301/19560 | loss 3.602048 (-0.17z)| norm 0.2813 (-0.14z)| lr 5.48e-04 | 323.08 ms | 52.2% bf16 MFU | 1623416 tok/s
step 4302/19560 | loss 3.619236 (+0.20z)| norm 0.2776 (-0.20z)| lr 5.48e-04 | 322.90 ms | 52.3% bf16 MFU | 1623428 tok/s
step 4303/19560 | loss 3.600095 (-0.20z)| norm 0.2632 (-0.42z)| lr 5.48e-04 | 322.40 ms | 52.3% bf16 MFU | 1623568 tok/s
step 4304/19560 | loss 3.594891 (-0.32z)| norm 0.2762 (-0.21z)| lr 5.48e-04 | 323.21 ms | 52.2% bf16 MFU | 1623496 tok/s
step 4305/19560 | loss 3.651468 (+0.90z)| norm 0.2699 (-0.30z)| lr 5.48e-04 | 322.55 ms | 52.3% bf16 MFU | 1623594 tok/s
step 4306/19560 | loss 3.550086 (-1.28z)| norm 0.2581 (-0.48z)| lr 5.48e-04 | 323.03 ms | 52.2% bf16 MFU | 1623565 tok/s
step 4307/19560 | loss 3.621755 (+0.26z)| norm 0.2812 (-0.12z)| lr 5.47e-04 | 323.70 ms | 52.1% bf16 MFU | 1623371 tok/s
step 4308/19560 | loss 3.597847 (-0.24z)| norm 0.2770 (-0.18z)| lr 5.47e-04 | 322.40 ms | 52.3% bf16 MFU | 1623513 tok/s
step 4309/19560 | loss 3.551745 (-1.23z)| norm 0.2918 (+0.05z)| lr 5.47e-04 | 323.42 ms | 52.2% bf16 MFU | 1623392 tok/s
step 4310/19560 | loss 3.637038 (+0.60z)| norm 0.2697 (-0.30z)| lr 5.47e-04 | 323.39 ms | 52.2% bf16 MFU | 1623284 tok/s
step 4311/19560 | loss 3.647645 (+0.83z)| norm 0.2574 (-0.48z)| lr 5.47e-04 | 322.85 ms | 52.3% bf16 MFU | 1623316 tok/s
step 4312/19560 | loss 3.597891 (-0.24z)| norm 0.2709 (-0.27z)| lr 5.47e-04 | 322.94 ms | 52.3% bf16 MFU | 1623324 tok/s
step 4313/19560 | loss 3.596636 (-0.27z)| norm 0.2713 (-0.27z)| lr 5.47e-04 | 322.91 ms | 52.3% bf16 MFU | 1623339 tok/s
step 4314/19560 | loss 3.596867 (-0.26z)| norm 0.2493 (-0.61z)| lr 5.47e-04 | 322.58 ms | 52.3% bf16 MFU | 1623438 tok/s
step 4315/19560 | loss 3.620206 (+0.24z)| norm 0.2821 (-0.09z)| lr 5.47e-04 | 323.16 ms | 52.2% bf16 MFU | 1623384 tok/s
step 4316/19560 | loss 3.594434 (-0.32z)| norm 0.2960 (+0.12z)| lr 5.47e-04 | 322.50 ms | 52.3% bf16 MFU | 1623500 tok/s
step 4317/19560 | loss 3.640478 (+0.67z)| norm 0.2725 (-0.25z)| lr 5.47e-04 | 323.02 ms | 52.2% bf16 MFU | 1623479 tok/s
step 4318/19560 | loss 3.555356 (-1.18z)| norm 0.2756 (-0.20z)| lr 5.47e-04 | 322.75 ms | 52.3% bf16 MFU | 1623528 tok/s
step 4319/19560 | loss 3.642560 (+0.72z)| norm 0.2557 (-0.51z)| lr 5.47e-04 | 322.87 ms | 52.3% bf16 MFU | 1623544 tok/s
step 4320/19560 | loss 3.594782 (-0.32z)| norm 0.2732 (-0.24z)| lr 5.47e-04 | 323.20 ms | 52.2% bf16 MFU | 1623476 tok/s
step 4321/19560 | loss 3.607779 (-0.02z)| norm 0.2603 (-0.44z)| lr 5.47e-04 | 322.87 ms | 52.3% bf16 MFU | 1623494 tok/s
step 4322/19560 | loss 3.530621 (-1.69z)| norm 0.2708 (-0.27z)| lr 5.47e-04 | 322.61 ms | 52.3% bf16 MFU | 1623578 tok/s
step 4323/19560 | loss 3.641984 (+0.76z)| norm 0.3295 (+0.64z)| lr 5.47e-04 | 322.58 ms | 52.3% bf16 MFU | 1623664 tok/s
step 4324/19560 | loss 3.617965 (+0.23z)| norm 0.2864 (-0.04z)| lr 5.47e-04 | 323.39 ms | 52.2% bf16 MFU | 1623542 tok/s
step 4325/19560 | loss 3.614511 (+0.15z)| norm 0.2685 (-0.32z)| lr 5.47e-04 | 322.52 ms | 52.3% bf16 MFU | 1623644 tok/s
step 4326/19560 | loss 3.711533 (+2.25z)| norm 0.2639 (-0.39z)| lr 5.47e-04 | 323.07 ms | 52.2% bf16 MFU | 1623604 tok/s
step 4327/19560 | loss 3.533474 (-1.61z)| norm 0.2701 (-0.29z)| lr 5.47e-04 | 323.22 ms | 52.2% bf16 MFU | 1623528 tok/s
step 4328/19560 | loss 3.640454 (+0.70z)| norm 0.2927 (+0.05z)| lr 5.47e-04 | 322.27 ms | 52.4% bf16 MFU | 1623696 tok/s
step 4329/19560 | loss 3.629741 (+0.45z)| norm 0.3160 (+0.41z)| lr 5.47e-04 | 323.18 ms | 52.2% bf16 MFU | 1623625 tok/s
step 4330/19560 | loss 3.624812 (+0.34z)| norm 0.3299 (+0.62z)| lr 5.47e-04 | 323.46 ms | 52.2% bf16 MFU | 1623487 tok/s
step 4331/19560 | loss 3.635322 (+0.56z)| norm 0.3374 (+0.73z)| lr 5.47e-04 | 322.98 ms | 52.3% bf16 MFU | 1623476 tok/s
step 4332/19560 | loss 3.667142 (+1.24z)| norm 0.2808 (-0.15z)| lr 5.47e-04 | 322.92 ms | 52.3% bf16 MFU | 1623480 tok/s
step 4333/19560 | loss 3.681546 (+1.53z)| norm 0.2936 (+0.04z)| lr 5.47e-04 | 322.03 ms | 52.4% bf16 MFU | 1623709 tok/s
step 4334/19560 | loss 3.581997 (-0.62z)| norm 0.2779 (-0.20z)| lr 5.47e-04 | 322.90 ms | 52.3% bf16 MFU | 1623708 tok/s
step 4335/19560 | loss 3.566482 (-0.94z)| norm 0.2905 (-0.01z)| lr 5.47e-04 | 323.30 ms | 52.2% bf16 MFU | 1623607 tok/s
step 4336/19560 | loss 3.595219 (-0.32z)| norm 0.2656 (-0.39z)| lr 5.47e-04 | 323.57 ms | 52.2% bf16 MFU | 1623443 tok/s
step 4337/19560 | loss 3.583574 (-0.57z)| norm 0.2580 (-0.50z)| lr 5.47e-04 | 322.51 ms | 52.3% bf16 MFU | 1623554 tok/s
step 4338/19560 | loss 3.650576 (+0.87z)| norm 0.2685 (-0.34z)| lr 5.47e-04 | 323.20 ms | 52.2% bf16 MFU | 1623486 tok/s
step 4339/19560 | loss 3.604853 (-0.12z)| norm 0.2470 (-0.67z)| lr 5.47e-04 | 323.15 ms | 52.2% bf16 MFU | 1623433 tok/s
step 4340/19560 | loss 3.597119 (-0.30z)| norm 0.2468 (-0.67z)| lr 5.47e-04 | 323.36 ms | 52.2% bf16 MFU | 1623330 tok/s
step 4341/19560 | loss 3.606673 (-0.10z)| norm 0.2747 (-0.23z)| lr 5.47e-04 | 322.99 ms | 52.3% bf16 MFU | 1623325 tok/s
step 4342/19560 | loss 3.612479 (+0.02z)| norm 0.2380 (-0.80z)| lr 5.46e-04 | 323.08 ms | 52.2% bf16 MFU | 1623298 tok/s
step 4343/19560 | loss 3.691236 (+1.72z)| norm 0.2953 (+0.09z)| lr 5.46e-04 | 323.56 ms | 52.2% bf16 MFU | 1623151 tok/s
step 4344/19560 | loss 3.688223 (+1.62z)| norm 0.2882 (-0.02z)| lr 5.46e-04 | 323.23 ms | 52.2% bf16 MFU | 1623094 tok/s
step 4345/19560 | loss 3.586503 (-0.59z)| norm 0.2777 (-0.18z)| lr 5.46e-04 | 322.97 ms | 52.3% bf16 MFU | 1623105 tok/s
step 4346/19560 | loss 3.632637 (+0.41z)| norm 0.2839 (-0.09z)| lr 5.46e-04 | 323.01 ms | 52.3% bf16 MFU | 1623108 tok/s
step 4347/19560 | loss 3.583999 (-0.64z)| norm 0.3033 (+0.21z)| lr 5.46e-04 | 322.50 ms | 52.3% bf16 MFU | 1623236 tok/s
step 4348/19560 | loss 3.639725 (+0.56z)| norm 0.2711 (-0.29z)| lr 5.46e-04 | 323.42 ms | 52.2% bf16 MFU | 1623129 tok/s
step 4349/19560 | loss 3.633917 (+0.45z)| norm 0.2821 (-0.11z)| lr 5.46e-04 | 323.45 ms | 52.2% bf16 MFU | 1623018 tok/s
step 4350/19560 | loss 3.615116 (+0.03z)| norm 0.2609 (-0.43z)| lr 5.46e-04 | 323.08 ms | 52.2% bf16 MFU | 1623006 tok/s
step 4351/19560 | loss 3.549410 (-1.43z)| norm 0.2587 (-0.46z)| lr 5.46e-04 | 322.86 ms | 52.3% bf16 MFU | 1623049 tok/s
step 4352/19560 | loss 3.568752 (-0.98z)| norm 0.2722 (-0.25z)| lr 5.46e-04 | 323.03 ms | 52.2% bf16 MFU | 1623047 tok/s
step 4353/19560 | loss 3.599916 (-0.30z)| norm 0.2479 (-0.62z)| lr 5.46e-04 | 322.61 ms | 52.3% bf16 MFU | 1623153 tok/s
step 4354/19560 | loss 3.570683 (-0.96z)| norm 0.2807 (-0.11z)| lr 5.46e-04 | 322.84 ms | 52.3% bf16 MFU | 1623194 tok/s
step 4355/19560 | loss 3.642940 (+0.65z)| norm 0.2977 (+0.15z)| lr 5.46e-04 | 323.26 ms | 52.2% bf16 MFU | 1623128 tok/s
step 4356/19560 | loss 3.580966 (-0.72z)| norm 0.2809 (-0.11z)| lr 5.46e-04 | 323.09 ms | 52.2% bf16 MFU | 1623107 tok/s
step 4357/19560 | loss 3.681344 (+1.52z)| norm 0.2709 (-0.26z)| lr 5.46e-04 | 322.57 ms | 52.3% bf16 MFU | 1623218 tok/s
step 4358/19560 | loss 3.570584 (-0.94z)| norm 0.2976 (+0.15z)| lr 5.46e-04 | 323.00 ms | 52.3% bf16 MFU | 1623217 tok/s
step 4359/19560 | loss 3.625823 (+0.30z)| norm 0.3057 (+0.27z)| lr 5.46e-04 | 322.84 ms | 52.3% bf16 MFU | 1623255 tok/s
step 4360/19560 | loss 3.639495 (+0.59z)| norm 0.3003 (+0.18z)| lr 5.46e-04 | 322.88 ms | 52.3% bf16 MFU | 1623282 tok/s
step 4361/19560 | loss 3.569885 (-0.98z)| norm 0.2826 (-0.09z)| lr 5.46e-04 | 323.27 ms | 52.2% bf16 MFU | 1623209 tok/s
step 4362/19560 | loss 3.567374 (-1.06z)| norm 0.2767 (-0.28z)| lr 5.46e-04 | 322.95 ms | 52.3% bf16 MFU | 1623219 tok/s
step 4363/19560 | loss 3.578224 (-0.80z)| norm 0.2914 (+0.37z)| lr 5.46e-04 | 322.53 ms | 52.3% bf16 MFU | 1623335 tok/s
step 4364/19560 | loss 3.608848 (-0.08z)| norm 0.2587 (-1.04z)| lr 5.46e-04 | 323.56 ms | 52.2% bf16 MFU | 1623188 tok/s
step 4365/19560 | loss 3.613565 (+0.02z)| norm 0.2640 (-0.79z)| lr 5.46e-04 | 323.08 ms | 52.2% bf16 MFU | 1623167 tok/s
step 4366/19560 | loss 3.690310 (+1.80z)| norm 0.2902 (+0.37z)| lr 5.46e-04 | 323.69 ms | 52.1% bf16 MFU | 1622994 tok/s
step 4367/19560 | loss 3.636428 (+0.53z)| norm 0.3125 (+1.37z)| lr 5.46e-04 | 323.14 ms | 52.2% bf16 MFU | 1622968 tok/s
step 4368/19560 | loss 3.666706 (+1.22z)| norm 0.3514 (+2.97z)| lr 5.46e-04 | 322.66 ms | 52.3% bf16 MFU | 1623064 tok/s
step 4369/19560 | loss 3.608751 (-0.12z)| norm 0.3138 (+1.34z)| lr 5.46e-04 | 323.24 ms | 52.2% bf16 MFU | 1623009 tok/s
step 4370/19560 | loss 3.621764 (+0.19z)| norm 0.2516 (-1.30z)| lr 5.46e-04 | 323.42 ms | 52.2% bf16 MFU | 1622911 tok/s
step 4371/19560 | loss 3.592314 (-0.49z)| norm 0.2921 (+0.41z)| lr 5.46e-04 | 322.74 ms | 52.3% bf16 MFU | 1622991 tok/s
step 4372/19560 | loss 3.616275 (+0.07z)| norm 0.2759 (-0.27z)| lr 5.46e-04 | 322.48 ms | 52.3% bf16 MFU | 1623131 tok/s
step 4373/19560 | loss 3.637085 (+0.56z)| norm 0.2683 (-0.59z)| lr 5.46e-04 | 323.01 ms | 52.3% bf16 MFU | 1623132 tok/s
step 4374/19560 | loss 3.574289 (-0.91z)| norm 0.2536 (-1.19z)| lr 5.46e-04 | 323.67 ms | 52.1% bf16 MFU | 1622966 tok/s
step 4375/19560 | loss 3.665800 (+1.25z)| norm 0.2854 (+0.14z)| lr 5.46e-04 | 323.26 ms | 52.2% bf16 MFU | 1622912 tok/s
step 4376/19560 | loss 3.700242 (+2.01z)| norm 0.2938 (+0.49z)| lr 5.46e-04 | 323.99 ms | 52.1% bf16 MFU | 1622677 tok/s
step 4377/19560 | loss 3.611730 (-0.05z)| norm 0.2664 (-0.66z)| lr 5.45e-04 | 323.10 ms | 52.2% bf16 MFU | 1622676 tok/s
step 4378/19560 | loss 3.599649 (-0.32z)| norm 0.2730 (-0.37z)| lr 5.45e-04 | 323.23 ms | 52.2% bf16 MFU | 1622643 tok/s
step 4379/19560 | loss 3.701562 (+2.02z)| norm 0.2595 (-0.93z)| lr 5.45e-04 | 323.78 ms | 52.1% bf16 MFU | 1622474 tok/s
step 4380/19560 | loss 3.698237 (+1.90z)| norm 0.2842 (+0.12z)| lr 5.45e-04 | 323.10 ms | 52.2% bf16 MFU | 1622484 tok/s
step 4381/19560 | loss 3.610438 (-0.11z)| norm 0.2796 (-0.08z)| lr 5.45e-04 | 322.90 ms | 52.3% bf16 MFU | 1622544 tok/s
step 4382/19560 | loss 3.590709 (-0.56z)| norm 0.2563 (-1.06z)| lr 5.45e-04 | 323.80 ms | 52.1% bf16 MFU | 1622376 tok/s
step 4383/19560 | loss 3.727215 (+2.53z)| norm 0.3034 (+0.92z)| lr 5.45e-04 | 322.62 ms | 52.3% bf16 MFU | 1622513 tok/s
step 4384/19560 | loss 3.606011 (-0.24z)| norm 0.3207 (+1.63z)| lr 5.45e-04 | 322.33 ms | 52.4% bf16 MFU | 1622716 tok/s
step 4385/19560 | loss 3.620506 (+0.10z)| norm 0.2760 (-0.26z)| lr 5.45e-04 | 323.19 ms | 52.2% bf16 MFU | 1622691 tok/s
step 4386/19560 | loss 3.554870 (-1.38z)| norm 0.3062 (+1.00z)| lr 5.45e-04 | 322.72 ms | 52.3% bf16 MFU | 1622786 tok/s
step 4387/19560 | loss 3.657111 (+0.94z)| norm 0.2767 (-0.26z)| lr 5.45e-04 | 322.66 ms | 52.3% bf16 MFU | 1622893 tok/s
step 4388/19560 | loss 3.636754 (+0.48z)| norm 0.2856 (+0.11z)| lr 5.45e-04 | 322.51 ms | 52.3% bf16 MFU | 1623030 tok/s
step 4389/19560 | loss 3.685083 (+1.56z)| norm 0.2897 (+0.28z)| lr 5.45e-04 | 323.12 ms | 52.2% bf16 MFU | 1623008 tok/s
step 4390/19560 | loss 3.590594 (-0.58z)| norm 0.2767 (-0.29z)| lr 5.45e-04 | 323.03 ms | 52.2% bf16 MFU | 1623009 tok/s
step 4391/19560 | loss 3.680339 (+1.43z)| norm 0.2741 (-0.41z)| lr 5.45e-04 | 323.05 ms | 52.2% bf16 MFU | 1623006 tok/s
step 4392/19560 | loss 3.567895 (-1.09z)| norm 0.2742 (-0.41z)| lr 5.45e-04 | 322.97 ms | 52.3% bf16 MFU | 1623022 tok/s
step 4393/19560 | loss 3.573573 (-0.96z)| norm 0.2451 (-1.66z)| lr 5.45e-04 | 323.49 ms | 52.2% bf16 MFU | 1622906 tok/s
step 4394/19560 | loss 3.605349 (-0.24z)| norm 0.2789 (-0.20z)| lr 5.45e-04 | 323.37 ms | 52.2% bf16 MFU | 1622826 tok/s
step 4395/19560 | loss 3.713892 (+2.16z)| norm 0.2803 (-0.14z)| lr 5.45e-04 | 322.87 ms | 52.3% bf16 MFU | 1622876 tok/s
step 4396/19560 | loss 3.659828 (+1.00z)| norm 0.2714 (-0.53z)| lr 5.45e-04 | 322.94 ms | 52.3% bf16 MFU | 1622907 tok/s
step 4397/19560 | loss 3.489590 (-2.76z)| norm 0.2673 (-0.71z)| lr 5.45e-04 | 323.25 ms | 52.2% bf16 MFU | 1622858 tok/s
step 4398/19560 | loss 3.594442 (-0.44z)| norm 0.2700 (-0.60z)| lr 5.45e-04 | 322.82 ms | 52.3% bf16 MFU | 1622920 tok/s
step 4399/19560 | loss 3.595922 (-0.41z)| norm 0.2905 (+0.29z)| lr 5.45e-04 | 322.73 ms | 52.3% bf16 MFU | 1623001 tok/s
step 4400/19560 | loss 3.552989 (-1.36z)| norm 0.2551 (-1.24z)| lr 5.45e-04 | 322.96 ms | 52.3% bf16 MFU | 1623019 tok/s
step 4401/19560 | loss 3.644774 (+0.68z)| norm 0.2854 (+0.07z)| lr 5.45e-04 | 322.88 ms | 52.3% bf16 MFU | 1623057 tok/s
step 4402/19560 | loss 3.661746 (+1.04z)| norm 0.2962 (+0.54z)| lr 5.45e-04 | 323.77 ms | 52.1% bf16 MFU | 1622871 tok/s
step 4403/19560 | loss 3.608547 (-0.13z)| norm 0.2820 (-0.08z)| lr 5.45e-04 | 323.00 ms | 52.3% bf16 MFU | 1622886 tok/s
step 4404/19560 | loss 3.611626 (-0.06z)| norm 0.2819 (-0.08z)| lr 5.45e-04 | 323.19 ms | 52.2% bf16 MFU | 1622853 tok/s
step 4405/19560 | loss 3.573374 (-0.92z)| norm 0.2635 (-0.88z)| lr 5.45e-04 | 322.66 ms | 52.3% bf16 MFU | 1622954 tok/s
step 4406/19560 | loss 3.590078 (-0.53z)| norm 0.2660 (-0.76z)| lr 5.45e-04 | 322.69 ms | 52.3% bf16 MFU | 1623044 tok/s
step 4407/19560 | loss 3.554733 (-1.31z)| norm 0.2606 (-0.99z)| lr 5.45e-04 | 322.98 ms | 52.3% bf16 MFU | 1623056 tok/s
step 4408/19560 | loss 3.563469 (-1.13z)| norm 0.2596 (-1.03z)| lr 5.45e-04 | 323.27 ms | 52.2% bf16 MFU | 1622995 tok/s
step 4409/19560 | loss 3.583587 (-0.68z)| norm 0.2550 (-1.24z)| lr 5.45e-04 | 322.98 ms | 52.3% bf16 MFU | 1623009 tok/s
step 4410/19560 | loss 3.605933 (-0.18z)| norm 0.2561 (-1.19z)| lr 5.45e-04 | 323.17 ms | 52.2% bf16 MFU | 1622976 tok/s
step 4411/19560 | loss 3.623275 (+0.22z)| norm 0.2581 (-1.09z)| lr 5.45e-04 | 323.17 ms | 52.2% bf16 MFU | 1622942 tok/s
step 4412/19560 | loss 3.584075 (-0.67z)| norm 0.2617 (-0.92z)| lr 5.44e-04 | 322.67 ms | 52.3% bf16 MFU | 1623038 tok/s
step 4413/19560 | loss 3.597902 (-0.35z)| norm 0.2620 (-0.89z)| lr 5.44e-04 | 323.30 ms | 52.2% bf16 MFU | 1622969 tok/s
step 4414/19560 | loss 3.564673 (-1.09z)| norm 0.2815 (+0.09z)| lr 5.44e-04 | 322.94 ms | 52.3% bf16 MFU | 1622995 tok/s
step 4415/19560 | loss 3.644712 (+0.72z)| norm 0.2834 (+0.21z)| lr 5.44e-04 | 323.32 ms | 52.2% bf16 MFU | 1622924 tok/s
step 4416/19560 | loss 3.560727 (-1.16z)| norm 0.2962 (+0.87z)| lr 5.44e-04 | 322.70 ms | 52.3% bf16 MFU | 1623012 tok/s
step 4417/19560 | loss 3.610704 (-0.04z)| norm 0.2989 (+1.00z)| lr 5.44e-04 | 323.06 ms | 52.2% bf16 MFU | 1623004 tok/s
step 4418/19560 | loss 3.586375 (-0.58z)| norm 0.2886 (+0.48z)| lr 5.44e-04 | 322.74 ms | 52.3% bf16 MFU | 1623080 tok/s
step 4419/19560 | loss 3.632085 (+0.43z)| norm 0.3118 (+1.65z)| lr 5.44e-04 | 322.83 ms | 52.3% bf16 MFU | 1623127 tok/s
step 4420/19560 | loss 3.656071 (+0.96z)| norm 0.3217 (+2.12z)| lr 5.44e-04 | 322.71 ms | 52.3% bf16 MFU | 1623203 tok/s
step 4421/19560 | loss 3.600680 (-0.28z)| norm 0.3060 (+1.32z)| lr 5.44e-04 | 322.64 ms | 52.3% bf16 MFU | 1623293 tok/s
step 4422/19560 | loss 3.566115 (-1.05z)| norm 0.2942 (+0.74z)| lr 5.44e-04 | 322.82 ms | 52.3% bf16 MFU | 1623333 tok/s
step 4423/19560 | loss 3.630291 (+0.39z)| norm 0.2807 (+0.06z)| lr 5.44e-04 | 322.96 ms | 52.3% bf16 MFU | 1623335 tok/s
step 4424/19560 | loss 3.600591 (-0.30z)| norm 0.3208 (+2.04z)| lr 5.44e-04 | 322.64 ms | 52.3% bf16 MFU | 1623418 tok/s
step 4425/19560 | loss 3.557308 (-1.31z)| norm 0.2785 (-0.07z)| lr 5.44e-04 | 322.94 ms | 52.3% bf16 MFU | 1623422 tok/s
step 4426/19560 | loss 3.622908 (+0.23z)| norm 0.2699 (-0.49z)| lr 5.44e-04 | 322.79 ms | 52.3% bf16 MFU | 1623463 tok/s
step 4427/19560 | loss 3.560084 (-1.23z)| norm 0.2695 (-0.51z)| lr 5.44e-04 | 322.52 ms | 52.3% bf16 MFU | 1623570 tok/s
step 4428/19560 | loss 3.609362 (-0.08z)| norm 0.2755 (-0.20z)| lr 5.44e-04 | 322.44 ms | 52.3% bf16 MFU | 1623690 tok/s
step 4429/19560 | loss 3.614216 (+0.03z)| norm 0.2805 (+0.05z)| lr 5.44e-04 | 323.14 ms | 52.2% bf16 MFU | 1623629 tok/s
step 4430/19560 | loss 3.617361 (+0.11z)| norm 0.2348 (-2.18z)| lr 5.44e-04 | 322.96 ms | 52.3% bf16 MFU | 1623616 tok/s
step 4431/19560 | loss 3.554557 (-1.36z)| norm 0.2671 (-0.60z)| lr 5.44e-04 | 322.68 ms | 52.3% bf16 MFU | 1623676 tok/s
step 4432/19560 | loss 3.601496 (-0.26z)| norm 0.2643 (-0.73z)| lr 5.44e-04 | 322.50 ms | 52.3% bf16 MFU | 1623778 tok/s
step 4433/19560 | loss 3.561870 (-1.17z)| norm 0.2540 (-1.22z)| lr 5.44e-04 | 323.07 ms | 52.2% bf16 MFU | 1623731 tok/s
step 4434/19560 | loss 3.549912 (-1.45z)| norm 0.2573 (-1.06z)| lr 5.44e-04 | 323.12 ms | 52.2% bf16 MFU | 1623672 tok/s
step 4435/19560 | loss 3.597879 (-0.32z)| norm 0.2531 (-1.25z)| lr 5.44e-04 | 322.54 ms | 52.3% bf16 MFU | 1623765 tok/s
step 4436/19560 | loss 3.629691 (+0.42z)| norm 0.2667 (-0.58z)| lr 5.44e-04 | 322.88 ms | 52.3% bf16 MFU | 1623766 tok/s
step 4437/19560 | loss 3.646283 (+0.79z)| norm 0.2788 (+0.01z)| lr 5.44e-04 | 322.50 ms | 52.3% bf16 MFU | 1623861 tok/s
step 4438/19560 | loss 3.591426 (-0.49z)| norm 0.2974 (+0.90z)| lr 5.44e-04 | 322.73 ms | 52.3% bf16 MFU | 1623896 tok/s
step 4439/19560 | loss 3.596687 (-0.36z)| norm 0.3076 (+1.37z)| lr 5.44e-04 | 322.45 ms | 52.3% bf16 MFU | 1623999 tok/s
step 4440/19560 | loss 3.583553 (-0.66z)| norm 0.2899 (+0.51z)| lr 5.44e-04 | 322.40 ms | 52.3% bf16 MFU | 1624110 tok/s
step 4441/19560 | loss 3.570989 (-0.95z)| norm 0.2627 (-0.80z)| lr 5.44e-04 | 322.51 ms | 52.3% bf16 MFU | 1624187 tok/s
step 4442/19560 | loss 3.598618 (-0.30z)| norm 0.2590 (-0.98z)| lr 5.44e-04 | 323.10 ms | 52.2% bf16 MFU | 1624112 tok/s
step 4443/19560 | loss 3.629704 (+0.43z)| norm 0.2648 (-0.69z)| lr 5.44e-04 | 323.53 ms | 52.2% bf16 MFU | 1623932 tok/s
step 4444/19560 | loss 3.577414 (-0.80z)| norm 0.2693 (-0.47z)| lr 5.44e-04 | 322.25 ms | 52.4% bf16 MFU | 1624084 tok/s
step 4445/19560 | loss 3.590708 (-0.48z)| norm 0.2700 (-0.43z)| lr 5.44e-04 | 322.38 ms | 52.4% bf16 MFU | 1624195 tok/s
step 4446/19560 | loss 3.593846 (-0.41z)| norm 0.2829 (+0.19z)| lr 5.43e-04 | 322.84 ms | 52.3% bf16 MFU | 1624185 tok/s
step 4447/19560 | loss 3.581846 (-0.69z)| norm 0.2670 (-0.59z)| lr 5.43e-04 | 322.94 ms | 52.3% bf16 MFU | 1624149 tok/s
step 4448/19560 | loss 3.644226 (+0.78z)| norm 0.2489 (-1.44z)| lr 5.43e-04 | 322.95 ms | 52.3% bf16 MFU | 1624114 tok/s
step 4449/19560 | loss 3.581000 (-0.71z)| norm 0.2518 (-1.30z)| lr 5.43e-04 | 323.13 ms | 52.2% bf16 MFU | 1624033 tok/s
step 4450/19560 | loss 3.549918 (-1.45z)| norm 0.2634 (-0.74z)| lr 5.43e-04 | 322.70 ms | 52.3% bf16 MFU | 1624067 tok/s
step 4451/19560 | loss 3.587421 (-0.55z)| norm 0.2905 (+0.58z)| lr 5.43e-04 | 322.95 ms | 52.3% bf16 MFU | 1624035 tok/s
step 4452/19560 | loss 3.603090 (-0.18z)| norm 0.3017 (+1.12z)| lr 5.43e-04 | 322.65 ms | 52.3% bf16 MFU | 1624080 tok/s
step 4453/19560 | loss 3.628360 (+0.42z)| norm 0.2891 (+0.50z)| lr 5.43e-04 | 322.97 ms | 52.3% bf16 MFU | 1624042 tok/s
step 4454/19560 | loss 3.707264 (+2.29z)| norm 0.2908 (+0.57z)| lr 5.43e-04 | 322.64 ms | 52.3% bf16 MFU | 1624090 tok/s
step 4455/19560 | loss 3.524452 (-2.04z)| norm 0.2775 (-0.08z)| lr 5.43e-04 | 322.57 ms | 52.3% bf16 MFU | 1624154 tok/s
step 4456/19560 | loss 3.657982 (+1.11z)| norm 0.2535 (-1.23z)| lr 5.43e-04 | 322.67 ms | 52.3% bf16 MFU | 1624189 tok/s
step 4457/19560 | loss 3.606180 (-0.11z)| norm 0.2632 (-0.74z)| lr 5.43e-04 | 323.27 ms | 52.2% bf16 MFU | 1624072 tok/s
step 4458/19560 | loss 3.668894 (+1.36z)| norm 0.2875 (+0.48z)| lr 5.43e-04 | 322.57 ms | 52.3% bf16 MFU | 1624135 tok/s
step 4459/19560 | loss 3.594274 (-0.39z)| norm 0.2991 (+1.11z)| lr 5.43e-04 | 323.18 ms | 52.2% bf16 MFU | 1624043 tok/s
step 4460/19560 | loss 3.634315 (+0.56z)| norm 0.2921 (+0.74z)| lr 5.43e-04 | 322.97 ms | 52.3% bf16 MFU | 1624008 tok/s
step 4461/19560 | loss 3.626019 (+0.38z)| norm 0.2678 (-0.51z)| lr 5.43e-04 | 322.68 ms | 52.3% bf16 MFU | 1624048 tok/s
step 4462/19560 | loss 3.612043 (+0.04z)| norm 0.2895 (+0.61z)| lr 5.43e-04 | 322.91 ms | 52.3% bf16 MFU | 1624028 tok/s
step 4463/19560 | loss 3.682787 (+1.70z)| norm 0.3009 (+1.19z)| lr 5.43e-04 | 323.11 ms | 52.2% bf16 MFU | 1623958 tok/s
step 4464/19560 | loss 3.663871 (+1.23z)| norm 0.2706 (-0.37z)| lr 5.43e-04 | 322.78 ms | 52.3% bf16 MFU | 1623975 tok/s
step 4465/19560 | loss 3.618473 (+0.15z)| norm 0.3176 (+2.01z)| lr 5.43e-04 | 323.05 ms | 52.2% bf16 MFU | 1623922 tok/s
step 4466/19560 | loss 3.652853 (+0.97z)| norm 0.3236 (+2.25z)| lr 5.43e-04 | 322.42 ms | 52.3% bf16 MFU | 1624030 tok/s
step 4467/19560 | loss 3.576649 (-0.83z)| norm 0.3046 (+1.28z)| lr 5.43e-04 | 323.50 ms | 52.2% bf16 MFU | 1623862 tok/s
step 4468/19560 | loss 3.594845 (-0.40z)| norm 0.3021 (+1.14z)| lr 5.43e-04 | 322.77 ms | 52.3% bf16 MFU | 1623886 tok/s
step 4469/19560 | loss 3.580420 (-0.73z)| norm 0.2770 (-0.13z)| lr 5.43e-04 | 323.04 ms | 52.2% bf16 MFU | 1623842 tok/s
step 4470/19560 | loss 3.568639 (-1.00z)| norm 0.2750 (-0.25z)| lr 5.43e-04 | 322.87 ms | 52.3% bf16 MFU | 1623840 tok/s
step 4471/19560 | loss 3.681537 (+1.65z)| norm 0.2851 (+0.27z)| lr 5.43e-04 | 322.57 ms | 52.3% bf16 MFU | 1623916 tok/s
step 4472/19560 | loss 3.565122 (-1.07z)| norm 0.2633 (-0.84z)| lr 5.43e-04 | 323.02 ms | 52.2% bf16 MFU | 1623873 tok/s
step 4473/19560 | loss 3.680251 (+1.63z)| norm 0.2515 (-1.43z)| lr 5.43e-04 | 322.39 ms | 52.3% bf16 MFU | 1623991 tok/s
step 4474/19560 | loss 3.591791 (-0.44z)| norm 0.2750 (-0.23z)| lr 5.43e-04 | 322.46 ms | 52.3% bf16 MFU | 1624087 tok/s
step 4475/19560 | loss 3.590223 (-0.48z)| norm 0.2494 (-1.50z)| lr 5.43e-04 | 323.16 ms | 52.2% bf16 MFU | 1624000 tok/s
step 4476/19560 | loss 3.636388 (+0.61z)| norm 0.2626 (-0.83z)| lr 5.43e-04 | 322.63 ms | 52.3% bf16 MFU | 1624053 tok/s
step 4477/19560 | loss 3.541068 (-1.60z)| norm 0.2400 (-1.93z)| lr 5.43e-04 | 323.03 ms | 52.2% bf16 MFU | 1624001 tok/s
step 4478/19560 | loss 3.614717 (+0.11z)| norm 0.2689 (-0.48z)| lr 5.43e-04 | 323.41 ms | 52.2% bf16 MFU | 1623858 tok/s
step 4479/19560 | loss 3.648349 (+0.88z)| norm 0.2785 (-0.01z)| lr 5.43e-04 | 322.67 ms | 52.3% bf16 MFU | 1623906 tok/s
step 4480/19560 | loss 3.597796 (-0.31z)| norm 0.2857 (+0.34z)| lr 5.42e-04 | 322.79 ms | 52.3% bf16 MFU | 1623923 tok/s
step 4481/19560 | loss 3.603783 (-0.17z)| norm 0.2593 (-0.99z)| lr 5.42e-04 | 323.23 ms | 52.2% bf16 MFU | 1623827 tok/s
step 4482/19560 | loss 3.605877 (-0.12z)| norm 0.2867 (+0.39z)| lr 5.42e-04 | 322.92 ms | 52.3% bf16 MFU | 1623815 tok/s
step 4483/19560 | loss 3.526031 (-1.96z)| norm 0.3140 (+1.75z)| lr 5.42e-04 | 322.52 ms | 52.3% bf16 MFU | 1623905 tok/s
step 4484/19560 | loss 3.592550 (-0.41z)| norm 0.3081 (+1.43z)| lr 5.42e-04 | 322.22 ms | 52.4% bf16 MFU | 1624064 tok/s
step 4485/19560 | loss 3.615191 (+0.13z)| norm 0.3201 (+1.98z)| lr 5.42e-04 | 323.13 ms | 52.2% bf16 MFU | 1623989 tok/s
step 4486/19560 | loss 3.643631 (+0.78z)| norm 0.3148 (+1.70z)| lr 5.42e-04 | 322.82 ms | 52.3% bf16 MFU | 1623994 tok/s
step 4487/19560 | loss 3.627628 (+0.41z)| norm 0.3616 (+3.75z)| lr 5.42e-04 | 322.84 ms | 52.3% bf16 MFU | 1623994 tok/s
step 4488/19560 | loss 3.562008 (-1.12z)| norm 0.3097 (+1.35z)| lr 5.42e-04 | 323.35 ms | 52.2% bf16 MFU | 1623865 tok/s
step 4489/19560 | loss 3.604702 (-0.13z)| norm 0.2929 (+0.57z)| lr 5.42e-04 | 322.62 ms | 52.3% bf16 MFU | 1623927 tok/s
step 4490/19560 | loss 3.547265 (-1.47z)| norm 0.2676 (-0.59z)| lr 5.42e-04 | 322.68 ms | 52.3% bf16 MFU | 1623970 tok/s
step 4491/19560 | loss 3.664709 (+1.26z)| norm 0.3075 (+1.23z)| lr 5.42e-04 | 323.39 ms | 52.2% bf16 MFU | 1623833 tok/s
step 4492/19560 | loss 3.601287 (-0.22z)| norm 0.2736 (-0.32z)| lr 5.42e-04 | 322.83 ms | 52.3% bf16 MFU | 1623843 tok/s
step 4493/19560 | loss 3.609632 (-0.02z)| norm 0.2593 (-0.97z)| lr 5.42e-04 | 323.05 ms | 52.2% bf16 MFU | 1623799 tok/s
step 4494/19560 | loss 3.548989 (-1.42z)| norm 0.2792 (-0.06z)| lr 5.42e-04 | 323.09 ms | 52.2% bf16 MFU | 1623746 tok/s
step 4495/19560 | loss 3.653472 (+1.03z)| norm 0.2782 (-0.09z)| lr 5.42e-04 | 322.36 ms | 52.4% bf16 MFU | 1623879 tok/s
step 4496/19560 | loss 3.559134 (-1.16z)| norm 0.2619 (-0.85z)| lr 5.42e-04 | 322.89 ms | 52.3% bf16 MFU | 1623873 tok/s
step 4497/19560 | loss 3.623951 (+0.36z)| norm 0.2778 (-0.07z)| lr 5.42e-04 | 323.29 ms | 52.2% bf16 MFU | 1623766 tok/s
step 4498/19560 | loss 3.564142 (-1.03z)| norm 0.2760 (-0.17z)| lr 5.42e-04 | 322.81 ms | 52.3% bf16 MFU | 1623785 tok/s
step 4499/19560 | loss 3.560209 (-1.11z)| norm 0.2849 (+0.27z)| lr 5.42e-04 | 323.03 ms | 52.2% bf16 MFU | 1623748 tok/s
step 4500/19560 | loss 3.545851 (-1.42z)| norm 0.2559 (-1.14z)| lr 5.42e-04 | 322.53 ms | 52.3% bf16 MFU | 1623838 tok/s
val loss 3.586649
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2780/10042 = 0.276837
step 4501/19560 | loss 3.610704 (+0.08z)| norm 0.2612 (-0.87z)| lr 5.42e-04 | 322.65 ms | 52.3% bf16 MFU | 1623893 tok/s
step 4502/19560 | loss 3.562644 (-1.03z)| norm 0.2655 (-0.67z)| lr 5.42e-04 | 322.44 ms | 52.3% bf16 MFU | 1623998 tok/s
step 4503/19560 | loss 3.549914 (-1.30z)| norm 0.2536 (-1.23z)| lr 5.42e-04 | 323.31 ms | 52.2% bf16 MFU | 1623879 tok/s
step 4504/19560 | loss 3.640954 (+0.82z)| norm 0.2476 (-1.50z)| lr 5.42e-04 | 323.00 ms | 52.3% bf16 MFU | 1623846 tok/s
step 4505/19560 | loss 3.627991 (+0.52z)| norm 0.2656 (-0.63z)| lr 5.42e-04 | 323.87 ms | 52.1% bf16 MFU | 1623594 tok/s
step 4506/19560 | loss 3.571037 (-0.81z)| norm 0.2667 (-0.57z)| lr 5.42e-04 | 322.81 ms | 52.3% bf16 MFU | 1623621 tok/s
step 4507/19560 | loss 3.565683 (-0.93z)| norm 0.2500 (-1.37z)| lr 5.42e-04 | 322.72 ms | 52.3% bf16 MFU | 1623670 tok/s
step 4508/19560 | loss 3.658538 (+1.30z)| norm 0.2848 (+0.30z)| lr 5.42e-04 | 322.88 ms | 52.3% bf16 MFU | 1623676 tok/s
step 4509/19560 | loss 3.524763 (-1.88z)| norm 0.2797 (+0.06z)| lr 5.42e-04 | 322.95 ms | 52.3% bf16 MFU | 1623663 tok/s
step 4510/19560 | loss 3.576801 (-0.64z)| norm 0.2682 (-0.50z)| lr 5.42e-04 | 322.66 ms | 52.3% bf16 MFU | 1623725 tok/s
step 4511/19560 | loss 3.561192 (-1.01z)| norm 0.2632 (-0.73z)| lr 5.42e-04 | 322.54 ms | 52.3% bf16 MFU | 1623813 tok/s
step 4512/19560 | loss 3.531725 (-1.70z)| norm 0.2635 (-0.71z)| lr 5.42e-04 | 322.86 ms | 52.3% bf16 MFU | 1623817 tok/s
step 4513/19560 | loss 3.570914 (-0.74z)| norm 0.2575 (-0.99z)| lr 5.42e-04 | 323.13 ms | 52.2% bf16 MFU | 1623754 tok/s
step 4514/19560 | loss 3.632274 (+0.73z)| norm 0.2557 (-1.06z)| lr 5.41e-04 | 323.37 ms | 52.2% bf16 MFU | 1623634 tok/s
step 4515/19560 | loss 3.595145 (-0.16z)| norm 0.3011 (+1.15z)| lr 5.41e-04 | 322.85 ms | 52.3% bf16 MFU | 1623648 tok/s
step 4516/19560 | loss 3.559551 (-1.01z)| norm 0.2782 (+0.04z)| lr 5.41e-04 | 322.61 ms | 52.3% bf16 MFU | 1623724 tok/s
step 4517/19560 | loss 3.629785 (+0.73z)| norm 0.2755 (-0.09z)| lr 5.41e-04 | 323.18 ms | 52.2% bf16 MFU | 1623650 tok/s
step 4518/19560 | loss 3.717923 (+2.79z)| norm 0.2925 (+0.73z)| lr 5.41e-04 | 322.75 ms | 52.3% bf16 MFU | 1623691 tok/s
step 4519/19560 | loss 3.631183 (+0.73z)| norm 0.2835 (+0.29z)| lr 5.41e-04 | 322.79 ms | 52.3% bf16 MFU | 1623719 tok/s
step 4520/19560 | loss 3.656678 (+1.33z)| norm 0.2949 (+0.84z)| lr 5.41e-04 | 322.85 ms | 52.3% bf16 MFU | 1623729 tok/s
step 4521/19560 | loss 3.724595 (+2.85z)| norm 0.3036 (+1.24z)| lr 5.41e-04 | 322.92 ms | 52.3% bf16 MFU | 1623722 tok/s
step 4522/19560 | loss 3.611003 (+0.19z)| norm 0.3012 (+1.11z)| lr 5.41e-04 | 322.91 ms | 52.3% bf16 MFU | 1623718 tok/s
step 4523/19560 | loss 3.604650 (+0.06z)| norm 0.2625 (-0.76z)| lr 5.41e-04 | 323.30 ms | 52.2% bf16 MFU | 1623615 tok/s
step 4524/19560 | loss 3.597713 (-0.10z)| norm 0.2747 (-0.17z)| lr 5.41e-04 | 322.41 ms | 52.3% bf16 MFU | 1623741 tok/s
step 4525/19560 | loss 3.650746 (+1.19z)| norm 0.2856 (+0.35z)| lr 5.41e-04 | 323.01 ms | 52.2% bf16 MFU | 1623710 tok/s
step 4526/19560 | loss 3.596760 (-0.15z)| norm 0.2660 (-0.60z)| lr 5.41e-04 | 323.70 ms | 52.1% bf16 MFU | 1623508 tok/s
step 4527/19560 | loss 3.631533 (+0.70z)| norm 0.2719 (-0.31z)| lr 5.41e-04 | 322.16 ms | 52.4% bf16 MFU | 1623704 tok/s
step 4528/19560 | loss 3.531225 (-1.77z)| norm 0.2928 (+0.70z)| lr 5.41e-04 | 323.39 ms | 52.2% bf16 MFU | 1623579 tok/s
step 4529/19560 | loss 3.632918 (+0.74z)| norm 0.3101 (+1.52z)| lr 5.41e-04 | 323.10 ms | 52.2% bf16 MFU | 1623535 tok/s
step 4530/19560 | loss 3.569797 (-0.81z)| norm 0.2824 (+0.18z)| lr 5.41e-04 | 322.92 ms | 52.3% bf16 MFU | 1623537 tok/s
step 4531/19560 | loss 3.615904 (+0.34z)| norm 0.3127 (+1.62z)| lr 5.41e-04 | 323.23 ms | 52.2% bf16 MFU | 1623462 tok/s
step 4532/19560 | loss 3.643363 (+1.01z)| norm 0.3001 (+1.01z)| lr 5.41e-04 | 322.59 ms | 52.3% bf16 MFU | 1623550 tok/s
step 4533/19560 | loss 3.659714 (+1.39z)| norm 0.2779 (-0.06z)| lr 5.41e-04 | 322.95 ms | 52.3% bf16 MFU | 1623545 tok/s
step 4534/19560 | loss 3.548027 (-1.34z)| norm 0.2732 (-0.28z)| lr 5.41e-04 | 322.89 ms | 52.3% bf16 MFU | 1623554 tok/s
step 4535/19560 | loss 3.620009 (+0.41z)| norm 0.3023 (+1.09z)| lr 5.41e-04 | 323.13 ms | 52.2% bf16 MFU | 1623503 tok/s
step 4536/19560 | loss 3.666755 (+1.53z)| norm 0.2822 (+0.13z)| lr 5.41e-04 | 323.01 ms | 52.3% bf16 MFU | 1623486 tok/s
step 4537/19560 | loss 3.598349 (-0.15z)| norm 0.3109 (+1.47z)| lr 5.41e-04 | 322.72 ms | 52.3% bf16 MFU | 1623541 tok/s
step 4538/19560 | loss 3.518523 (-2.05z)| norm 0.2851 (+0.23z)| lr 5.41e-04 | 322.58 ms | 52.3% bf16 MFU | 1623629 tok/s
step 4539/19560 | loss 3.560149 (-1.03z)| norm 0.2910 (+0.50z)| lr 5.41e-04 | 323.66 ms | 52.1% bf16 MFU | 1623441 tok/s
step 4540/19560 | loss 3.786277 (+4.06z)| norm 0.2942 (+0.65z)| lr 5.41e-04 | 322.36 ms | 52.4% bf16 MFU | 1623589 tok/s
step 4541/19560 | loss 3.543461 (-1.35z)| norm 0.3136 (+1.56z)| lr 5.41e-04 | 323.10 ms | 52.2% bf16 MFU | 1623543 tok/s
step 4542/19560 | loss 3.540149 (-1.41z)| norm 0.2799 (-0.06z)| lr 5.41e-04 | 323.08 ms | 52.2% bf16 MFU | 1623506 tok/s
step 4543/19560 | loss 3.559802 (-0.96z)| norm 0.2916 (+0.50z)| lr 5.41e-04 | 322.56 ms | 52.3% bf16 MFU | 1623600 tok/s
step 4544/19560 | loss 3.600287 (-0.08z)| norm 0.3605 (+3.59z)| lr 5.41e-04 | 323.71 ms | 52.1% bf16 MFU | 1623400 tok/s
step 4545/19560 | loss 3.575628 (-0.62z)| norm 0.2917 (+0.46z)| lr 5.41e-04 | 323.44 ms | 52.2% bf16 MFU | 1623279 tok/s
step 4546/19560 | loss 3.585833 (-0.39z)| norm 0.2681 (-0.61z)| lr 5.41e-04 | 322.49 ms | 52.3% bf16 MFU | 1623402 tok/s
step 4547/19560 | loss 3.577451 (-0.57z)| norm 0.2614 (-0.90z)| lr 5.41e-04 | 323.19 ms | 52.2% bf16 MFU | 1623343 tok/s
step 4548/19560 | loss 3.617676 (+0.33z)| norm 0.2770 (-0.18z)| lr 5.40e-04 | 323.50 ms | 52.2% bf16 MFU | 1623210 tok/s
step 4549/19560 | loss 3.605289 (+0.06z)| norm 0.2912 (+0.49z)| lr 5.40e-04 | 323.34 ms | 52.2% bf16 MFU | 1623124 tok/s
step 4550/19560 | loss 3.559408 (-0.96z)| norm 0.2649 (-0.72z)| lr 5.40e-04 | 322.73 ms | 52.3% bf16 MFU | 1623193 tok/s
step 4551/19560 | loss 3.576767 (-0.57z)| norm 0.2808 (+0.02z)| lr 5.40e-04 | 322.57 ms | 52.3% bf16 MFU | 1623300 tok/s
step 4552/19560 | loss 3.538781 (-1.39z)| norm 0.3079 (+1.29z)| lr 5.40e-04 | 322.91 ms | 52.3% bf16 MFU | 1623316 tok/s
step 4553/19560 | loss 3.754114 (+3.20z)| norm 0.2705 (-0.46z)| lr 5.40e-04 | 322.80 ms | 52.3% bf16 MFU | 1623359 tok/s
step 4554/19560 | loss 3.565832 (-0.79z)| norm 0.3080 (+1.28z)| lr 5.40e-04 | 322.42 ms | 52.3% bf16 MFU | 1623497 tok/s
step 4555/19560 | loss 3.625802 (+0.48z)| norm 0.2869 (+0.29z)| lr 5.40e-04 | 322.90 ms | 52.3% bf16 MFU | 1623507 tok/s
step 4556/19560 | loss 3.623303 (+0.42z)| norm 0.3147 (+1.56z)| lr 5.40e-04 | 322.84 ms | 52.3% bf16 MFU | 1623531 tok/s
step 4557/19560 | loss 3.574011 (-0.62z)| norm 0.2946 (+0.62z)| lr 5.40e-04 | 322.39 ms | 52.4% bf16 MFU | 1623668 tok/s
step 4558/19560 | loss 3.573852 (-0.62z)| norm 0.2518 (-1.37z)| lr 5.40e-04 | 323.59 ms | 52.2% bf16 MFU | 1623496 tok/s
step 4559/19560 | loss 3.585269 (-0.38z)| norm 0.2666 (-0.68z)| lr 5.40e-04 | 322.67 ms | 52.3% bf16 MFU | 1623564 tok/s
step 4560/19560 | loss 3.674439 (+1.49z)| norm 0.2551 (-1.21z)| lr 5.40e-04 | 322.99 ms | 52.3% bf16 MFU | 1623548 tok/s
step 4561/19560 | loss 3.585932 (-0.38z)| norm 0.2652 (-0.75z)| lr 5.40e-04 | 322.50 ms | 52.3% bf16 MFU | 1623656 tok/s
step 4562/19560 | loss 3.587245 (-0.36z)| norm 0.2715 (-0.47z)| lr 5.40e-04 | 323.14 ms | 52.2% bf16 MFU | 1623596 tok/s
step 4563/19560 | loss 3.563575 (-0.85z)| norm 0.2436 (-1.76z)| lr 5.40e-04 | 322.85 ms | 52.3% bf16 MFU | 1623613 tok/s
step 4564/19560 | loss 3.693384 (+1.86z)| norm 0.2470 (-1.58z)| lr 5.40e-04 | 322.81 ms | 52.3% bf16 MFU | 1623640 tok/s
step 4565/19560 | loss 3.552518 (-1.07z)| norm 0.2494 (-1.45z)| lr 5.40e-04 | 323.11 ms | 52.2% bf16 MFU | 1623590 tok/s
step 4566/19560 | loss 3.611734 (+0.17z)| norm 0.2633 (-0.80z)| lr 5.40e-04 | 322.82 ms | 52.3% bf16 MFU | 1623616 tok/s
step 4567/19560 | loss 3.625111 (+0.44z)| norm 0.2523 (-1.28z)| lr 5.40e-04 | 323.51 ms | 52.2% bf16 MFU | 1623466 tok/s
step 4568/19560 | loss 3.615442 (+0.23z)| norm 0.2577 (-1.02z)| lr 5.40e-04 | 323.35 ms | 52.2% bf16 MFU | 1623363 tok/s
step 4569/19560 | loss 3.635651 (+0.64z)| norm 0.2772 (-0.13z)| lr 5.40e-04 | 323.05 ms | 52.2% bf16 MFU | 1623341 tok/s
step 4570/19560 | loss 3.573121 (-0.66z)| norm 0.3177 (+1.69z)| lr 5.40e-04 | 322.95 ms | 52.3% bf16 MFU | 1623344 tok/s
step 4571/19560 | loss 3.564879 (-0.82z)| norm 0.3172 (+1.63z)| lr 5.40e-04 | 323.40 ms | 52.2% bf16 MFU | 1623237 tok/s
step 4572/19560 | loss 3.549466 (-1.13z)| norm 0.2716 (-0.42z)| lr 5.40e-04 | 322.84 ms | 52.3% bf16 MFU | 1623275 tok/s
step 4573/19560 | loss 3.548316 (-1.14z)| norm 0.2519 (-1.30z)| lr 5.40e-04 | 323.12 ms | 52.2% bf16 MFU | 1623239 tok/s
step 4574/19560 | loss 3.610386 (+0.14z)| norm 0.2647 (-0.71z)| lr 5.40e-04 | 322.52 ms | 52.3% bf16 MFU | 1623357 tok/s
step 4575/19560 | loss 3.615915 (+0.25z)| norm 0.2436 (-1.64z)| lr 5.40e-04 | 323.25 ms | 52.2% bf16 MFU | 1623286 tok/s
step 4576/19560 | loss 3.547756 (-1.14z)| norm 0.2576 (-1.02z)| lr 5.40e-04 | 322.83 ms | 52.3% bf16 MFU | 1623325 tok/s
step 4577/19560 | loss 3.611266 (+0.16z)| norm 0.2377 (-1.89z)| lr 5.40e-04 | 322.89 ms | 52.3% bf16 MFU | 1623346 tok/s
step 4578/19560 | loss 3.590649 (-0.27z)| norm 0.2571 (-1.03z)| lr 5.40e-04 | 322.88 ms | 52.3% bf16 MFU | 1623368 tok/s
step 4579/19560 | loss 3.555427 (-0.99z)| norm 0.2540 (-1.15z)| lr 5.40e-04 | 322.92 ms | 52.3% bf16 MFU | 1623380 tok/s
step 4580/19560 | loss 3.616011 (+0.26z)| norm 0.2825 (+0.11z)| lr 5.40e-04 | 323.25 ms | 52.2% bf16 MFU | 1623307 tok/s
step 4581/19560 | loss 3.645131 (+0.85z)| norm 0.2697 (-0.45z)| lr 5.39e-04 | 322.36 ms | 52.4% bf16 MFU | 1623461 tok/s
step 4582/19560 | loss 3.598383 (-0.09z)| norm 0.2503 (-1.28z)| lr 5.39e-04 | 323.29 ms | 52.2% bf16 MFU | 1623374 tok/s
step 4583/19560 | loss 3.693352 (+1.86z)| norm 0.6999 (+9.59z)| lr 5.39e-04 | 322.82 ms | 52.3% bf16 MFU | 1623409 tok/s
step 4584/19560 | loss 3.567060 (-0.76z)| norm 0.3848 (+2.28z)| lr 5.39e-04 | 322.78 ms | 52.3% bf16 MFU | 1623452 tok/s
step 4585/19560 | loss 3.591753 (-0.24z)| norm 0.3535 (+1.54z)| lr 5.39e-04 | 322.62 ms | 52.3% bf16 MFU | 1623534 tok/s
step 4586/19560 | loss 3.620050 (+0.36z)| norm 0.2943 (+0.22z)| lr 5.39e-04 | 323.11 ms | 52.2% bf16 MFU | 1623488 tok/s
step 4587/19560 | loss 3.577180 (-0.54z)| norm 0.2970 (+0.28z)| lr 5.39e-04 | 323.24 ms | 52.2% bf16 MFU | 1623412 tok/s
step 4588/19560 | loss 3.560382 (-0.88z)| norm 0.3126 (+0.62z)| lr 5.39e-04 | 323.08 ms | 52.2% bf16 MFU | 1623380 tok/s
step 4589/19560 | loss 3.587319 (-0.31z)| norm 0.3124 (+0.61z)| lr 5.39e-04 | 322.42 ms | 52.3% bf16 MFU | 1623516 tok/s
step 4590/19560 | loss 3.594256 (-0.16z)| norm 0.3010 (+0.35z)| lr 5.39e-04 | 323.04 ms | 52.2% bf16 MFU | 1623489 tok/s
step 4591/19560 | loss 3.582680 (-0.39z)| norm 0.2712 (-0.31z)| lr 5.39e-04 | 322.54 ms | 52.3% bf16 MFU | 1623589 tok/s
step 4592/19560 | loss 3.599794 (-0.02z)| norm 0.2763 (-0.19z)| lr 5.39e-04 | 322.57 ms | 52.3% bf16 MFU | 1623676 tok/s
step 4593/19560 | loss 3.587986 (-0.26z)| norm 0.3152 (+0.67z)| lr 5.39e-04 | 322.44 ms | 52.3% bf16 MFU | 1623793 tok/s
step 4594/19560 | loss 3.603683 (+0.08z)| norm 0.3127 (+0.62z)| lr 5.39e-04 | 322.97 ms | 52.3% bf16 MFU | 1623770 tok/s
step 4595/19560 | loss 3.706323 (+2.23z)| norm 0.2690 (-0.35z)| lr 5.39e-04 | 322.33 ms | 52.4% bf16 MFU | 1623909 tok/s
step 4596/19560 | loss 3.609588 (+0.18z)| norm 0.2596 (-0.55z)| lr 5.39e-04 | 322.76 ms | 52.3% bf16 MFU | 1623934 tok/s
step 4597/19560 | loss 3.660240 (+1.23z)| norm 0.2659 (-0.41z)| lr 5.39e-04 | 323.04 ms | 52.2% bf16 MFU | 1623887 tok/s
step 4598/19560 | loss 3.573137 (-0.60z)| norm 0.2666 (-0.39z)| lr 5.39e-04 | 323.27 ms | 52.2% bf16 MFU | 1623785 tok/s
step 4599/19560 | loss 3.587550 (-0.29z)| norm 0.2593 (-0.55z)| lr 5.39e-04 | 322.72 ms | 52.3% bf16 MFU | 1623825 tok/s
step 4600/19560 | loss 3.578443 (-0.48z)| norm 0.2605 (-0.52z)| lr 5.39e-04 | 322.63 ms | 52.3% bf16 MFU | 1623885 tok/s
step 4601/19560 | loss 3.623243 (+0.49z)| norm 0.2753 (-0.20z)| lr 5.39e-04 | 323.27 ms | 52.2% bf16 MFU | 1623783 tok/s
step 4602/19560 | loss 3.617306 (+0.35z)| norm 0.2745 (-0.22z)| lr 5.39e-04 | 322.43 ms | 52.3% bf16 MFU | 1623897 tok/s
step 4603/19560 | loss 3.598840 (-0.04z)| norm 0.2718 (-0.28z)| lr 5.39e-04 | 322.72 ms | 52.3% bf16 MFU | 1623932 tok/s
step 4604/19560 | loss 3.583447 (-0.37z)| norm 0.2698 (-0.32z)| lr 5.39e-04 | 323.04 ms | 52.2% bf16 MFU | 1623884 tok/s
step 4605/19560 | loss 3.567991 (-0.71z)| norm 0.2690 (-0.35z)| lr 5.39e-04 | 322.72 ms | 52.3% bf16 MFU | 1623918 tok/s
step 4606/19560 | loss 3.665652 (+1.39z)| norm 0.2620 (-0.50z)| lr 5.39e-04 | 323.01 ms | 52.2% bf16 MFU | 1623878 tok/s
step 4607/19560 | loss 3.587442 (-0.28z)| norm 0.2528 (-0.71z)| lr 5.39e-04 | 322.56 ms | 52.3% bf16 MFU | 1623954 tok/s
step 4608/19560 | loss 3.596278 (-0.09z)| norm 0.2836 (-0.02z)| lr 5.39e-04 | 322.96 ms | 52.3% bf16 MFU | 1623924 tok/s
step 4609/19560 | loss 3.589587 (-0.24z)| norm 0.2800 (-0.10z)| lr 5.39e-04 | 323.59 ms | 52.2% bf16 MFU | 1623740 tok/s
step 4610/19560 | loss 3.625787 (+0.54z)| norm 0.3069 (+0.50z)| lr 5.39e-04 | 322.63 ms | 52.3% bf16 MFU | 1623806 tok/s
step 4611/19560 | loss 3.602898 (+0.04z)| norm 0.2716 (-0.28z)| lr 5.39e-04 | 322.89 ms | 52.3% bf16 MFU | 1623803 tok/s
step 4612/19560 | loss 3.707010 (+2.24z)| norm 0.2892 (+0.11z)| lr 5.39e-04 | 322.80 ms | 52.3% bf16 MFU | 1623821 tok/s
step 4613/19560 | loss 3.539931 (-1.31z)| norm 0.2710 (-0.29z)| lr 5.39e-04 | 322.69 ms | 52.3% bf16 MFU | 1623868 tok/s
step 4614/19560 | loss 3.601176 (-0.00z)| norm 0.3022 (+0.42z)| lr 5.38e-04 | 322.96 ms | 52.3% bf16 MFU | 1623844 tok/s
step 4615/19560 | loss 3.572410 (-0.60z)| norm 0.2555 (-0.62z)| lr 5.38e-04 | 322.83 ms | 52.3% bf16 MFU | 1623854 tok/s
step 4616/19560 | loss 3.608420 (+0.15z)| norm 0.3016 (+0.43z)| lr 5.38e-04 | 322.75 ms | 52.3% bf16 MFU | 1623884 tok/s
step 4617/19560 | loss 3.563900 (-0.79z)| norm 0.3124 (+0.67z)| lr 5.38e-04 | 322.61 ms | 52.3% bf16 MFU | 1623946 tok/s
step 4618/19560 | loss 3.554888 (-0.98z)| norm 0.2809 (-0.05z)| lr 5.38e-04 | 322.42 ms | 52.3% bf16 MFU | 1624054 tok/s
step 4619/19560 | loss 3.569369 (-0.66z)| norm 0.2613 (-0.49z)| lr 5.38e-04 | 322.56 ms | 52.3% bf16 MFU | 1624122 tok/s
step 4620/19560 | loss 3.651784 (+1.09z)| norm 0.2546 (-0.63z)| lr 5.38e-04 | 323.53 ms | 52.2% bf16 MFU | 1623940 tok/s
step 4621/19560 | loss 3.597777 (-0.06z)| norm 0.2801 (-0.06z)| lr 5.38e-04 | 322.71 ms | 52.3% bf16 MFU | 1623975 tok/s
step 4622/19560 | loss 3.584711 (-0.34z)| norm 0.2881 (+0.12z)| lr 5.38e-04 | 322.67 ms | 52.3% bf16 MFU | 1624018 tok/s
step 4623/19560 | loss 3.562605 (-0.80z)| norm 0.2878 (+0.11z)| lr 5.38e-04 | 323.16 ms | 52.2% bf16 MFU | 1623936 tok/s
step 4624/19560 | loss 3.553930 (-0.99z)| norm 0.2920 (+0.20z)| lr 5.38e-04 | 322.59 ms | 52.3% bf16 MFU | 1624001 tok/s
step 4625/19560 | loss 3.593531 (-0.13z)| norm 0.2537 (-0.66z)| lr 5.38e-04 | 322.54 ms | 52.3% bf16 MFU | 1624076 tok/s
step 4626/19560 | loss 3.621909 (+0.47z)| norm 0.2729 (-0.23z)| lr 5.38e-04 | 323.07 ms | 52.2% bf16 MFU | 1624015 tok/s
step 4627/19560 | loss 3.561170 (-0.84z)| norm 0.2583 (-0.55z)| lr 5.38e-04 | 323.36 ms | 52.2% bf16 MFU | 1623884 tok/s
step 4628/19560 | loss 3.570371 (-0.65z)| norm 0.2617 (-0.48z)| lr 5.38e-04 | 322.70 ms | 52.3% bf16 MFU | 1623924 tok/s
step 4629/19560 | loss 3.588876 (-0.25z)| norm 0.2746 (-0.19z)| lr 5.38e-04 | 322.78 ms | 52.3% bf16 MFU | 1623942 tok/s
step 4630/19560 | loss 3.641197 (+0.87z)| norm 0.2621 (-0.47z)| lr 5.38e-04 | 322.86 ms | 52.3% bf16 MFU | 1623940 tok/s
step 4631/19560 | loss 3.651772 (+1.08z)| norm 0.2637 (-0.44z)| lr 5.38e-04 | 322.06 ms | 52.4% bf16 MFU | 1624139 tok/s
step 4632/19560 | loss 3.726856 (+2.62z)| norm 0.2837 (+0.01z)| lr 5.38e-04 | 322.65 ms | 52.3% bf16 MFU | 1624180 tok/s
step 4633/19560 | loss 3.625121 (+0.48z)| norm 0.2691 (-0.32z)| lr 5.38e-04 | 323.23 ms | 52.2% bf16 MFU | 1624074 tok/s
step 4634/19560 | loss 3.562155 (-0.84z)| norm 0.2612 (-0.50z)| lr 5.38e-04 | 322.64 ms | 52.3% bf16 MFU | 1624119 tok/s
step 4635/19560 | loss 3.546630 (-1.16z)| norm 0.2851 (+0.04z)| lr 5.38e-04 | 322.79 ms | 52.3% bf16 MFU | 1624124 tok/s
step 4636/19560 | loss 3.852825 (+4.77z)| norm 0.3309 (+1.08z)| lr 5.38e-04 | 322.84 ms | 52.3% bf16 MFU | 1624116 tok/s
step 4637/19560 | loss 3.524935 (-1.50z)| norm 0.3165 (+0.74z)| lr 5.38e-04 | 322.99 ms | 52.3% bf16 MFU | 1624072 tok/s
step 4638/19560 | loss 3.562304 (-0.79z)| norm 0.3084 (+0.55z)| lr 5.38e-04 | 322.79 ms | 52.3% bf16 MFU | 1624081 tok/s
step 4639/19560 | loss 3.612137 (+0.16z)| norm 0.3152 (+0.69z)| lr 5.38e-04 | 322.45 ms | 52.3% bf16 MFU | 1624174 tok/s
step 4640/19560 | loss 3.575202 (-0.56z)| norm 0.3538 (+1.54z)| lr 5.38e-04 | 323.00 ms | 52.3% bf16 MFU | 1624123 tok/s
step 4641/19560 | loss 3.582076 (-0.43z)| norm 0.3379 (+1.16z)| lr 5.38e-04 | 322.64 ms | 52.3% bf16 MFU | 1624167 tok/s
step 4642/19560 | loss 3.577291 (-0.51z)| norm 0.3396 (+1.18z)| lr 5.38e-04 | 323.01 ms | 52.3% bf16 MFU | 1624117 tok/s
step 4643/19560 | loss 3.528325 (-1.44z)| norm 0.3045 (+0.40z)| lr 5.38e-04 | 322.56 ms | 52.3% bf16 MFU | 1624180 tok/s
step 4644/19560 | loss 3.548182 (-1.05z)| norm 0.2889 (+0.05z)| lr 5.38e-04 | 322.48 ms | 52.3% bf16 MFU | 1624261 tok/s
step 4645/19560 | loss 3.510258 (-1.74z)| norm 0.2740 (-0.29z)| lr 5.38e-04 | 323.06 ms | 52.2% bf16 MFU | 1624191 tok/s
step 4646/19560 | loss 3.512586 (-1.68z)| norm 0.2735 (-0.29z)| lr 5.38e-04 | 323.63 ms | 52.1% bf16 MFU | 1623983 tok/s
step 4647/19560 | loss 3.599269 (-0.02z)| norm 0.3181 (+0.70z)| lr 5.37e-04 | 322.93 ms | 52.3% bf16 MFU | 1623961 tok/s
step 4648/19560 | loss 3.548632 (-0.97z)| norm 0.2447 (-0.93z)| lr 5.37e-04 | 322.39 ms | 52.3% bf16 MFU | 1624075 tok/s
step 4649/19560 | loss 3.531399 (-1.29z)| norm 0.2749 (-0.25z)| lr 5.37e-04 | 322.24 ms | 52.4% bf16 MFU | 1624220 tok/s
step 4650/19560 | loss 3.766064 (+3.11z)| norm 0.2686 (-0.39z)| lr 5.37e-04 | 323.44 ms | 52.2% bf16 MFU | 1624058 tok/s
step 4651/19560 | loss 3.602116 (+0.05z)| norm 0.2815 (-0.10z)| lr 5.37e-04 | 322.49 ms | 52.3% bf16 MFU | 1624143 tok/s
step 4652/19560 | loss 3.562711 (-0.68z)| norm 0.2855 (-0.02z)| lr 5.37e-04 | 322.99 ms | 52.3% bf16 MFU | 1624097 tok/s
step 4653/19560 | loss 3.577845 (-0.39z)| norm 0.3185 (+0.71z)| lr 5.37e-04 | 322.49 ms | 52.3% bf16 MFU | 1624179 tok/s
step 4654/19560 | loss 3.548726 (-0.92z)| norm 0.2727 (-0.31z)| lr 5.37e-04 | 322.53 ms | 52.3% bf16 MFU | 1624247 tok/s
step 4655/19560 | loss 3.609303 (+0.21z)| norm 0.2472 (-0.87z)| lr 5.37e-04 | 323.20 ms | 52.2% bf16 MFU | 1624144 tok/s
step 4656/19560 | loss 3.621356 (+0.43z)| norm 0.2629 (-0.52z)| lr 5.37e-04 | 322.98 ms | 52.3% bf16 MFU | 1624100 tok/s
step 4657/19560 | loss 3.607958 (+0.18z)| norm 0.2776 (-0.18z)| lr 5.37e-04 | 323.01 ms | 52.2% bf16 MFU | 1624051 tok/s
step 4658/19560 | loss 3.537790 (-1.13z)| norm 0.2419 (-0.96z)| lr 5.37e-04 | 323.25 ms | 52.2% bf16 MFU | 1623945 tok/s
step 4659/19560 | loss 3.594902 (-0.06z)| norm 0.2510 (-0.75z)| lr 5.37e-04 | 322.72 ms | 52.3% bf16 MFU | 1623978 tok/s
step 4660/19560 | loss 3.580780 (-0.31z)| norm 0.2554 (-0.65z)| lr 5.37e-04 | 322.42 ms | 52.3% bf16 MFU | 1624085 tok/s
step 4661/19560 | loss 3.622022 (+0.47z)| norm 0.2540 (-0.67z)| lr 5.37e-04 | 322.62 ms | 52.3% bf16 MFU | 1624137 tok/s
step 4662/19560 | loss 3.563051 (-0.65z)| norm 0.2784 (-0.14z)| lr 5.37e-04 | 323.02 ms | 52.2% bf16 MFU | 1624085 tok/s
step 4663/19560 | loss 3.563556 (-0.63z)| norm 0.2566 (-0.61z)| lr 5.37e-04 | 322.67 ms | 52.3% bf16 MFU | 1624122 tok/s
step 4664/19560 | loss 3.522380 (-1.38z)| norm 0.2568 (-0.60z)| lr 5.37e-04 | 322.61 ms | 52.3% bf16 MFU | 1624173 tok/s
step 4665/19560 | loss 3.567149 (-0.53z)| norm 0.2799 (-0.09z)| lr 5.37e-04 | 322.31 ms | 52.4% bf16 MFU | 1624297 tok/s
step 4666/19560 | loss 3.556747 (-0.74z)| norm 0.3240 (+0.87z)| lr 5.37e-04 | 322.92 ms | 52.3% bf16 MFU | 1624262 tok/s
step 4667/19560 | loss 3.595470 (-0.01z)| norm 0.3111 (+0.59z)| lr 5.37e-04 | 323.39 ms | 52.2% bf16 MFU | 1624109 tok/s
step 4668/19560 | loss 3.625587 (+0.61z)| norm 0.3029 (+0.41z)| lr 5.37e-04 | 322.61 ms | 52.3% bf16 MFU | 1624161 tok/s
step 4669/19560 | loss 3.536304 (-1.17z)| norm 0.2863 (+0.05z)| lr 5.37e-04 | 322.72 ms | 52.3% bf16 MFU | 1624181 tok/s
step 4670/19560 | loss 3.608769 (+0.27z)| norm 0.3637 (+1.71z)| lr 5.37e-04 | 322.43 ms | 52.3% bf16 MFU | 1624274 tok/s
step 4671/19560 | loss 3.681275 (+1.69z)| norm 0.3148 (+0.64z)| lr 5.37e-04 | 323.49 ms | 52.2% bf16 MFU | 1624097 tok/s
step 4672/19560 | loss 3.623213 (+0.53z)| norm 0.3123 (+0.60z)| lr 5.37e-04 | 322.79 ms | 52.3% bf16 MFU | 1624104 tok/s
step 4673/19560 | loss 3.598323 (+0.03z)| norm 0.3122 (+0.60z)| lr 5.37e-04 | 322.93 ms | 52.3% bf16 MFU | 1624076 tok/s
step 4674/19560 | loss 3.599467 (+0.05z)| norm 0.2756 (-0.20z)| lr 5.37e-04 | 323.04 ms | 52.2% bf16 MFU | 1624022 tok/s
step 4675/19560 | loss 3.626198 (+0.58z)| norm 0.2923 (+0.16z)| lr 5.37e-04 | 322.90 ms | 52.3% bf16 MFU | 1624005 tok/s
step 4676/19560 | loss 3.598665 (+0.03z)| norm 0.2646 (-0.44z)| lr 5.37e-04 | 322.81 ms | 52.3% bf16 MFU | 1624011 tok/s
step 4677/19560 | loss 3.627992 (+0.61z)| norm 0.2884 (+0.08z)| lr 5.37e-04 | 323.34 ms | 52.2% bf16 MFU | 1623883 tok/s
step 4678/19560 | loss 3.533018 (-1.26z)| norm 0.2985 (+0.29z)| lr 5.37e-04 | 323.03 ms | 52.2% bf16 MFU | 1623840 tok/s
step 4679/19560 | loss 3.587343 (-0.19z)| norm 0.2649 (-0.44z)| lr 5.37e-04 | 322.68 ms | 52.3% bf16 MFU | 1623887 tok/s
step 4680/19560 | loss 3.581693 (-0.31z)| norm 0.2575 (-0.59z)| lr 5.36e-04 | 322.11 ms | 52.4% bf16 MFU | 1624077 tok/s
step 4681/19560 | loss 3.633577 (+0.77z)| norm 0.2623 (-0.49z)| lr 5.36e-04 | 323.45 ms | 52.2% bf16 MFU | 1623920 tok/s
step 4682/19560 | loss 3.596406 (-0.01z)| norm 0.2497 (-0.75z)| lr 5.36e-04 | 323.15 ms | 52.2% bf16 MFU | 1623845 tok/s
step 4683/19560 | loss 3.459914 (-2.72z)| norm 0.2606 (-0.51z)| lr 5.36e-04 | 322.91 ms | 52.3% bf16 MFU | 1623835 tok/s
step 4684/19560 | loss 3.528020 (-1.33z)| norm 0.2869 (+0.07z)| lr 5.36e-04 | 322.80 ms | 52.3% bf16 MFU | 1623853 tok/s
step 4685/19560 | loss 3.570405 (-0.48z)| norm 0.2785 (-0.11z)| lr 5.36e-04 | 322.79 ms | 52.3% bf16 MFU | 1623873 tok/s
step 4686/19560 | loss 3.557185 (-0.74z)| norm 0.2579 (-0.56z)| lr 5.36e-04 | 323.19 ms | 52.2% bf16 MFU | 1623790 tok/s
step 4687/19560 | loss 3.587110 (-0.15z)| norm 0.3125 (+0.62z)| lr 5.36e-04 | 323.04 ms | 52.2% bf16 MFU | 1623750 tok/s
step 4688/19560 | loss 3.618086 (+0.48z)| norm 0.3069 (+0.49z)| lr 5.36e-04 | 322.72 ms | 52.3% bf16 MFU | 1623792 tok/s
step 4689/19560 | loss 3.576736 (-0.35z)| norm 0.4042 (+2.52z)| lr 5.36e-04 | 322.41 ms | 52.3% bf16 MFU | 1623910 tok/s
step 4690/19560 | loss 3.532905 (-1.21z)| norm 0.2738 (-0.25z)| lr 5.36e-04 | 322.75 ms | 52.3% bf16 MFU | 1623937 tok/s
step 4691/19560 | loss 3.515144 (-1.55z)| norm 0.2763 (-0.20z)| lr 5.36e-04 | 323.59 ms | 52.2% bf16 MFU | 1623752 tok/s
step 4692/19560 | loss 3.589289 (-0.06z)| norm 0.2847 (-0.03z)| lr 5.36e-04 | 323.19 ms | 52.2% bf16 MFU | 1623675 tok/s
step 4693/19560 | loss 3.619322 (+0.53z)| norm 0.2644 (-0.47z)| lr 5.36e-04 | 322.57 ms | 52.3% bf16 MFU | 1623759 tok/s
step 4694/19560 | loss 3.554792 (-0.76z)| norm 0.2802 (-0.13z)| lr 5.36e-04 | 322.62 ms | 52.3% bf16 MFU | 1623825 tok/s
step 4695/19560 | loss 3.624643 (+0.65z)| norm 0.3000 (+0.29z)| lr 5.36e-04 | 322.91 ms | 52.3% bf16 MFU | 1623816 tok/s
step 4696/19560 | loss 3.589857 (-0.05z)| norm 0.2628 (-0.51z)| lr 5.36e-04 | 323.30 ms | 52.2% bf16 MFU | 1623709 tok/s
step 4697/19560 | loss 3.559165 (-0.65z)| norm 0.2651 (-0.46z)| lr 5.36e-04 | 323.06 ms | 52.2% bf16 MFU | 1623667 tok/s
step 4698/19560 | loss 3.610356 (+0.37z)| norm 0.2778 (-0.18z)| lr 5.36e-04 | 322.53 ms | 52.3% bf16 MFU | 1623762 tok/s
step 4699/19560 | loss 3.564275 (-0.56z)| norm 0.2676 (-0.39z)| lr 5.36e-04 | 322.83 ms | 52.3% bf16 MFU | 1623775 tok/s
step 4700/19560 | loss 3.522334 (-1.39z)| norm 0.2681 (-0.38z)| lr 5.36e-04 | 323.30 ms | 52.2% bf16 MFU | 1623671 tok/s
step 4701/19560 | loss 3.582036 (-0.20z)| norm 0.2782 (-0.17z)| lr 5.36e-04 | 323.44 ms | 52.2% bf16 MFU | 1623537 tok/s
step 4702/19560 | loss 3.561688 (-0.60z)| norm 0.2908 (+0.10z)| lr 5.36e-04 | 322.84 ms | 52.3% bf16 MFU | 1623560 tok/s
step 4703/19560 | loss 3.673246 (+1.62z)| norm 0.2747 (-0.26z)| lr 5.36e-04 | 322.57 ms | 52.3% bf16 MFU | 1623650 tok/s
step 4704/19560 | loss 3.534267 (-1.15z)| norm 0.2663 (-0.44z)| lr 5.36e-04 | 322.39 ms | 52.3% bf16 MFU | 1623779 tok/s
step 4705/19560 | loss 3.597525 (+0.11z)| norm 0.2729 (-0.30z)| lr 5.36e-04 | 322.88 ms | 52.3% bf16 MFU | 1623781 tok/s
step 4706/19560 | loss 3.651686 (+1.17z)| norm 0.2748 (-0.27z)| lr 5.36e-04 | 323.29 ms | 52.2% bf16 MFU | 1623678 tok/s
step 4707/19560 | loss 3.558134 (-0.68z)| norm 0.2603 (-0.58z)| lr 5.36e-04 | 322.45 ms | 52.3% bf16 MFU | 1623792 tok/s
step 4708/19560 | loss 3.576828 (-0.30z)| norm 0.2854 (-0.04z)| lr 5.36e-04 | 323.16 ms | 52.2% bf16 MFU | 1623721 tok/s
step 4709/19560 | loss 3.902802 (+5.39z)| norm 0.2819 (-0.12z)| lr 5.36e-04 | 322.77 ms | 52.3% bf16 MFU | 1623751 tok/s
step 4710/19560 | loss 3.585321 (-0.15z)| norm 0.2799 (-0.16z)| lr 5.36e-04 | 323.10 ms | 52.2% bf16 MFU | 1623698 tok/s
step 4711/19560 | loss 3.627463 (+0.60z)| norm 0.2736 (-0.38z)| lr 5.36e-04 | 322.94 ms | 52.3% bf16 MFU | 1623687 tok/s
step 4712/19560 | loss 3.612420 (+0.33z)| norm 0.3457 (+2.32z)| lr 5.35e-04 | 322.57 ms | 52.3% bf16 MFU | 1623770 tok/s
step 4713/19560 | loss 3.635081 (+0.72z)| norm 0.3161 (+1.25z)| lr 5.35e-04 | 322.91 ms | 52.3% bf16 MFU | 1623764 tok/s
step 4714/19560 | loss 3.588962 (-0.09z)| norm 0.2657 (-0.68z)| lr 5.35e-04 | 322.50 ms | 52.3% bf16 MFU | 1623862 tok/s
step 4715/19560 | loss 3.610085 (+0.28z)| norm 0.2865 (+0.12z)| lr 5.35e-04 | 323.43 ms | 52.2% bf16 MFU | 1623720 tok/s
step 4716/19560 | loss 3.592628 (-0.03z)| norm 0.2853 (+0.09z)| lr 5.35e-04 | 322.88 ms | 52.3% bf16 MFU | 1623724 tok/s
step 4717/19560 | loss 3.562859 (-0.55z)| norm 0.2897 (+0.27z)| lr 5.35e-04 | 322.81 ms | 52.3% bf16 MFU | 1623746 tok/s
step 4718/19560 | loss 3.636452 (+0.74z)| norm 0.2511 (-1.21z)| lr 5.35e-04 | 323.18 ms | 52.2% bf16 MFU | 1623674 tok/s
step 4719/19560 | loss 3.623971 (+0.51z)| norm 0.2877 (+0.20z)| lr 5.35e-04 | 322.86 ms | 52.3% bf16 MFU | 1623684 tok/s
step 4720/19560 | loss 3.586424 (-0.15z)| norm 0.2944 (+0.45z)| lr 5.35e-04 | 323.30 ms | 52.2% bf16 MFU | 1623584 tok/s
step 4721/19560 | loss 3.578290 (-0.29z)| norm 0.2687 (-0.53z)| lr 5.35e-04 | 322.48 ms | 52.3% bf16 MFU | 1623696 tok/s
step 4722/19560 | loss 3.633223 (+0.67z)| norm 0.2785 (-0.14z)| lr 5.35e-04 | 322.77 ms | 52.3% bf16 MFU | 1623727 tok/s
step 4723/19560 | loss 3.566336 (-0.49z)| norm 0.2718 (-0.40z)| lr 5.35e-04 | 323.26 ms | 52.2% bf16 MFU | 1623636 tok/s
step 4724/19560 | loss 3.616079 (+0.40z)| norm 0.2621 (-0.78z)| lr 5.35e-04 | 322.80 ms | 52.3% bf16 MFU | 1623664 tok/s
step 4725/19560 | loss 3.666106 (+1.28z)| norm 0.2648 (-0.68z)| lr 5.35e-04 | 322.85 ms | 52.3% bf16 MFU | 1623676 tok/s
step 4726/19560 | loss 3.535304 (-1.03z)| norm 0.2477 (-1.33z)| lr 5.35e-04 | 323.09 ms | 52.2% bf16 MFU | 1623629 tok/s
step 4727/19560 | loss 3.592214 (-0.03z)| norm 0.2572 (-0.96z)| lr 5.35e-04 | 323.23 ms | 52.2% bf16 MFU | 1623548 tok/s
step 4728/19560 | loss 3.537985 (-0.98z)| norm 0.2685 (-0.53z)| lr 5.35e-04 | 323.41 ms | 52.2% bf16 MFU | 1623426 tok/s
step 4729/19560 | loss 3.551153 (-0.73z)| norm 0.2659 (-0.62z)| lr 5.35e-04 | 322.61 ms | 52.3% bf16 MFU | 1623512 tok/s
step 4730/19560 | loss 3.570170 (-0.39z)| norm 0.2780 (-0.16z)| lr 5.35e-04 | 323.05 ms | 52.2% bf16 MFU | 1623482 tok/s
step 4731/19560 | loss 3.598907 (+0.12z)| norm 0.2923 (+0.40z)| lr 5.35e-04 | 322.77 ms | 52.3% bf16 MFU | 1623527 tok/s
step 4732/19560 | loss 3.675697 (+1.44z)| norm 0.2654 (-0.65z)| lr 5.35e-04 | 322.73 ms | 52.3% bf16 MFU | 1623578 tok/s
step 4733/19560 | loss 3.557533 (-0.62z)| norm 0.2940 (+0.45z)| lr 5.35e-04 | 323.14 ms | 52.2% bf16 MFU | 1623523 tok/s
step 4734/19560 | loss 3.592129 (-0.01z)| norm 0.3045 (+0.85z)| lr 5.35e-04 | 322.77 ms | 52.3% bf16 MFU | 1623565 tok/s
step 4735/19560 | loss 3.586573 (-0.10z)| norm 0.2946 (+0.45z)| lr 5.35e-04 | 323.87 ms | 52.1% bf16 MFU | 1623329 tok/s
step 4736/19560 | loss 3.563667 (-0.50z)| norm 0.2618 (-0.82z)| lr 5.35e-04 | 323.20 ms | 52.2% bf16 MFU | 1623272 tok/s
step 4737/19560 | loss 3.597226 (+0.09z)| norm 0.2572 (-0.98z)| lr 5.35e-04 | 322.46 ms | 52.3% bf16 MFU | 1623403 tok/s
step 4738/19560 | loss 3.635536 (+0.76z)| norm 0.2746 (-0.30z)| lr 5.35e-04 | 323.09 ms | 52.2% bf16 MFU | 1623369 tok/s
step 4739/19560 | loss 3.606118 (+0.24z)| norm 0.2942 (+0.45z)| lr 5.35e-04 | 323.49 ms | 52.2% bf16 MFU | 1623238 tok/s
step 4740/19560 | loss 3.571105 (-0.36z)| norm 0.2543 (-1.08z)| lr 5.35e-04 | 323.07 ms | 52.2% bf16 MFU | 1623218 tok/s
step 4741/19560 | loss 3.586225 (-0.10z)| norm 0.2899 (+0.29z)| lr 5.35e-04 | 323.30 ms | 52.2% bf16 MFU | 1623140 tok/s
step 4742/19560 | loss 3.598306 (+0.12z)| norm 0.2693 (-0.50z)| lr 5.35e-04 | 322.82 ms | 52.3% bf16 MFU | 1623188 tok/s
step 4743/19560 | loss 3.564715 (-0.48z)| norm 0.2753 (-0.27z)| lr 5.35e-04 | 323.00 ms | 52.3% bf16 MFU | 1623188 tok/s
step 4744/19560 | loss 3.584325 (-0.13z)| norm 0.2644 (-0.69z)| lr 5.35e-04 | 323.08 ms | 52.2% bf16 MFU | 1623169 tok/s
step 4745/19560 | loss 3.572302 (-0.34z)| norm 0.2665 (-0.59z)| lr 5.34e-04 | 322.86 ms | 52.3% bf16 MFU | 1623204 tok/s
step 4746/19560 | loss 3.584930 (-0.12z)| norm 0.2733 (-0.32z)| lr 5.34e-04 | 323.78 ms | 52.1% bf16 MFU | 1623008 tok/s
step 4747/19560 | loss 3.523077 (-1.22z)| norm 0.2612 (-0.80z)| lr 5.34e-04 | 323.05 ms | 52.2% bf16 MFU | 1623004 tok/s
step 4748/19560 | loss 3.597201 (+0.11z)| norm 0.2648 (-0.66z)| lr 5.34e-04 | 322.46 ms | 52.3% bf16 MFU | 1623150 tok/s
step 4749/19560 | loss 3.564998 (-0.46z)| norm 0.2756 (-0.24z)| lr 5.34e-04 | 323.28 ms | 52.2% bf16 MFU | 1623082 tok/s
step 4750/19560 | loss 3.595903 (+0.09z)| norm 0.2752 (-0.25z)| lr 5.34e-04 | 323.67 ms | 52.1% bf16 MFU | 1622918 tok/s
val loss 3.571055
ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2776/10042 = 0.276439
step 4751/19560 | loss 3.575073 (-0.28z)| norm 0.3090 (+1.06z)| lr 5.34e-04 | 322.20 ms | 52.4% bf16 MFU | 1623132 tok/s
step 4752/19560 | loss 3.558228 (-0.58z)| norm 0.2912 (+0.37z)| lr 5.34e-04 | 323.23 ms | 52.2% bf16 MFU | 1623077 tok/s
step 4753/19560 | loss 3.642257 (+0.91z)| norm 0.2460 (-1.39z)| lr 5.34e-04 | 322.98 ms | 52.3% bf16 MFU | 1623088 tok/s
step 4754/19560 | loss 3.607160 (+0.29z)| norm 0.2812 (-0.02z)| lr 5.34e-04 | 322.83 ms | 52.3% bf16 MFU | 1623135 tok/s
step 4755/19560 | loss 3.620661 (+0.52z)| norm 0.2783 (-0.14z)| lr 5.34e-04 | 322.74 ms | 52.3% bf16 MFU | 1623202 tok/s
step 4756/19560 | loss 3.572160 (-0.35z)| norm 0.2917 (+0.38z)| lr 5.34e-04 | 322.97 ms | 52.3% bf16 MFU | 1623208 tok/s
step 4757/19560 | loss 3.639639 (+0.85z)| norm 0.3120 (+1.15z)| lr 5.34e-04 | 322.83 ms | 52.3% bf16 MFU | 1623250 tok/s
step 4758/19560 | loss 3.630886 (+0.70z)| norm 0.3233 (+1.56z)| lr 5.34e-04 | 322.52 ms | 52.3% bf16 MFU | 1623367 tok/s
step 4759/19560 | loss 3.625693 (+0.61z)| norm 0.3157 (+1.25z)| lr 5.34e-04 | 322.84 ms | 52.3% bf16 MFU | 1623399 tok/s
step 4760/19560 | loss 3.578603 (-0.22z)| norm 0.3269 (+1.65z)| lr 5.34e-04 | 322.65 ms | 52.3% bf16 MFU | 1623477 tok/s
step 4761/19560 | loss 3.589500 (-0.01z)| norm 0.3108 (+1.02z)| lr 5.34e-04 | 322.61 ms | 52.3% bf16 MFU | 1623561 tok/s
step 4762/19560 | loss 3.611462 (+0.38z)| norm 0.3386 (+2.03z)| lr 5.34e-04 | 322.88 ms | 52.3% bf16 MFU | 1623573 tok/s
step 4763/19560 | loss 3.626578 (+0.65z)| norm 0.3531 (+2.48z)| lr 5.34e-04 | 322.35 ms | 52.4% bf16 MFU | 1623718 tok/s
step 4764/19560 | loss 3.571175 (-0.36z)| norm 0.2984 (+0.50z)| lr 5.34e-04 | 322.48 ms | 52.3% bf16 MFU | 1623822 tok/s
step 4765/19560 | loss 3.553120 (-0.74z)| norm 0.3110 (+0.97z)| lr 5.34e-04 | 322.64 ms | 52.3% bf16 MFU | 1623880 tok/s
step 4766/19560 | loss 3.649274 (+1.20z)| norm 0.3117 (+0.99z)| lr 5.34e-04 | 322.95 ms | 52.3% bf16 MFU | 1623858 tok/s
step 4767/19560 | loss 3.585071 (-0.10z)| norm 0.2700 (-0.54z)| lr 5.34e-04 | 322.60 ms | 52.3% bf16 MFU | 1623926 tok/s
step 4768/19560 | loss 3.670307 (+1.61z)| norm 0.2777 (-0.23z)| lr 5.34e-04 | 323.23 ms | 52.2% bf16 MFU | 1623831 tok/s
step 4769/19560 | loss 3.588175 (-0.05z)| norm 0.2878 (+0.17z)| lr 5.34e-04 | 322.76 ms | 52.3% bf16 MFU | 1623858 tok/s
step 4770/19560 | loss 3.640055 (+0.98z)| norm 0.2655 (-0.69z)| lr 5.34e-04 | 321.98 ms | 52.4% bf16 MFU | 1624080 tok/s
step 4771/19560 | loss 3.574090 (-0.35z)| norm 0.2752 (-0.30z)| lr 5.34e-04 | 322.38 ms | 52.4% bf16 MFU | 1624190 tok/s
step 4772/19560 | loss 3.583602 (-0.16z)| norm 0.2573 (-0.99z)| lr 5.34e-04 | 323.35 ms | 52.2% bf16 MFU | 1624053 tok/s
step 4773/19560 | loss 3.590667 (-0.03z)| norm 0.2672 (-0.59z)| lr 5.34e-04 | 322.90 ms | 52.3% bf16 MFU | 1624034 tok/s
step 4774/19560 | loss 3.595508 (+0.05z)| norm 0.2641 (-0.71z)| lr 5.34e-04 | 322.94 ms | 52.3% bf16 MFU | 1624006 tok/s
step 4775/19560 | loss 3.675673 (+1.68z)| norm 0.2569 (-0.98z)| lr 5.34e-04 | 322.34 ms | 52.4% bf16 MFU | 1624131 tok/s
step 4776/19560 | loss 3.594363 (+0.01z)| norm 0.2800 (-0.08z)| lr 5.33e-04 | 322.74 ms | 52.3% bf16 MFU | 1624149 tok/s
step 4777/19560 | loss 3.634792 (+0.82z)| norm 0.2667 (-0.61z)| lr 5.33e-04 | 322.79 ms | 52.3% bf16 MFU | 1624152 tok/s
step 4778/19560 | loss 3.583088 (-0.22z)| norm 0.2705 (-0.46z)| lr 5.33e-04 | 322.61 ms | 52.3% bf16 MFU | 1624203 tok/s
step 4779/19560 | loss 3.575545 (-0.38z)| norm 0.2535 (-1.12z)| lr 5.33e-04 | 322.75 ms | 52.3% bf16 MFU | 1624216 tok/s
step 4780/19560 | loss 3.541058 (-1.12z)| norm 0.2578 (-0.94z)| lr 5.33e-04 | 322.88 ms | 52.3% bf16 MFU | 1624193 tok/s
step 4781/19560 | loss 3.611009 (+0.39z)| norm 0.2687 (-0.49z)| lr 5.33e-04 | 322.76 ms | 52.3% bf16 MFU | 1624202 tok/s
step 4782/19560 | loss 3.697007 (+2.18z)| norm 0.2931 (+0.46z)| lr 5.33e-04 | 322.39 ms | 52.4% bf16 MFU | 1624306 tok/s
step 4783/19560 | loss 3.598027 (+0.08z)| norm 0.2650 (-0.66z)| lr 5.33e-04 | 322.83 ms | 52.3% bf16 MFU | 1624293 tok/s
step 4784/19560 | loss 3.525195 (-1.44z)| norm 0.3570 (+2.89z)| lr 5.33e-04 | 322.76 ms | 52.3% bf16 MFU | 1624297 tok/s
step 4785/19560 | loss 3.559966 (-0.70z)| norm 0.2866 (+0.17z)| lr 5.33e-04 | 322.58 ms | 52.3% bf16 MFU | 1624347 tok/s
step 4786/19560 | loss 3.631635 (+0.80z)| norm 0.3078 (+0.97z)| lr 5.33e-04 | 321.91 ms | 52.4% bf16 MFU | 1624564 tok/s
step 4787/19560 | loss 3.515207 (-1.63z)| norm 0.2804 (-0.10z)| lr 5.33e-04 | 323.16 ms | 52.2% bf16 MFU | 1624456 tok/s
step 4788/19560 | loss 3.584501 (-0.18z)| norm 0.2666 (-0.65z)| lr 5.33e-04 | 322.76 ms | 52.3% bf16 MFU | 1624451 tok/s
step 4789/19560 | loss 3.575223 (-0.37z)| norm 0.2584 (-0.97z)| lr 5.33e-04 | 323.19 ms | 52.2% bf16 MFU | 1624340 tok/s
step 4790/19560 | loss 3.590783 (-0.05z)| norm 0.2829 (-0.01z)| lr 5.33e-04 | 323.02 ms | 52.2% bf16 MFU | 1624277 tok/s
step 4791/19560 | loss 3.621015 (+0.58z)| norm 0.3143 (+1.20z)| lr 5.33e-04 | 322.32 ms | 52.4% bf16 MFU | 1624395 tok/s
step 4792/19560 | loss 3.582456 (-0.24z)| norm 0.3102 (+1.03z)| lr 5.33e-04 | 322.67 ms | 52.3% bf16 MFU | 1624418 tok/s
step 4793/19560 | loss 3.636764 (+0.89z)| norm 0.2840 (-0.00z)| lr 5.33e-04 | 322.59 ms | 52.3% bf16 MFU | 1624459 tok/s
step 4794/19560 | loss 3.599574 (+0.10z)| norm 0.2708 (-0.51z)| lr 5.33e-04 | 323.38 ms | 52.2% bf16 MFU | 1624300 tok/s
step 4795/19560 | loss 3.633637 (+0.81z)| norm 0.2742 (-0.37z)| lr 5.33e-04 | 322.31 ms | 52.4% bf16 MFU | 1624418 tok/s
step 4796/19560 | loss 3.677675 (+1.72z)| norm 0.3109 (+1.09z)| lr 5.33e-04 | 322.63 ms | 52.3% bf16 MFU | 1624448 tok/s
step 4797/19560 | loss 3.592599 (-0.07z)| norm 0.2942 (+0.42z)| lr 5.33e-04 | 323.28 ms | 52.2% bf16 MFU | 1624314 tok/s
step 4798/19560 | loss 3.654103 (+1.21z)| norm 0.2588 (-0.98z)| lr 5.33e-04 | 322.63 ms | 52.3% bf16 MFU | 1624350 tok/s
step 4799/19560 | loss 3.627180 (+0.66z)| norm 0.2722 (-0.42z)| lr 5.33e-04 | 322.77 ms | 52.3% bf16 MFU | 1624349 tok/s
step 4800/19560 | loss 3.582022 (-0.29z)| norm 0.3125 (+1.25z)| lr 5.33e-04 | 323.01 ms | 52.2% bf16 MFU | 1624289 tok/s
step 4801/19560 | loss 3.531198 (-1.34z)| norm 0.2833 (+0.05z)| lr 5.33e-04 | 322.61 ms | 52.3% bf16 MFU | 1624332 tok/s
step 4802/19560 | loss 3.531682 (-1.31z)| norm 0.2799 (-0.10z)| lr 5.33e-04 | 323.37 ms | 52.2% bf16 MFU | 1624181 tok/s
step 4803/19560 | loss 3.554588 (-0.82z)| norm 0.2802 (-0.08z)| lr 5.33e-04 | 322.64 ms | 52.3% bf16 MFU | 1624222 tok/s
step 4804/19560 | loss 3.530538 (-1.30z)| norm 0.3035 (+0.88z)| lr 5.33e-04 | 322.56 ms | 52.3% bf16 MFU | 1624280 tok/s
step 4805/19560 | loss 3.602895 (+0.20z)| norm 0.2840 (+0.07z)| lr 5.33e-04 | 322.73 ms | 52.3% bf16 MFU | 1624293 tok/s
step 4806/19560 | loss 3.532785 (-1.26z)| norm 0.2583 (-0.99z)| lr 5.33e-04 | 323.34 ms | 52.2% bf16 MFU | 1624151 tok/s
step 4807/19560 | loss 3.618541 (+0.52z)| norm 0.2816 (-0.02z)| lr 5.33e-04 | 322.93 ms | 52.3% bf16 MFU | 1624121 tok/s
step 4808/19560 | loss 3.557525 (-0.74z)| norm 0.2852 (+0.12z)| lr 5.32e-04 | 322.90 ms | 52.3% bf16 MFU | 1624098 tok/s
step 4809/19560 | loss 3.578798 (-0.29z)| norm 0.2538 (-1.19z)| lr 5.32e-04 | 322.86 ms | 52.3% bf16 MFU | 1624086 tok/s
step 4810/19560 | loss 3.635619 (+0.88z)| norm 0.2641 (-0.77z)| lr 5.32e-04 | 323.32 ms | 52.2% bf16 MFU | 1623961 tok/s
step 4811/19560 | loss 3.656896 (+1.32z)| norm 0.3102 (+1.15z)| lr 5.32e-04 | 322.35 ms | 52.4% bf16 MFU | 1624085 tok/s
step 4812/19560 | loss 3.568115 (-0.58z)| norm 0.2579 (-1.03z)| lr 5.32e-04 | 322.42 ms | 52.3% bf16 MFU | 1624186 tok/s
step 4813/19560 | loss 3.616278 (+0.44z)| norm 0.2725 (-0.42z)| lr 5.32e-04 | 322.95 ms | 52.3% bf16 MFU | 1624149 tok/s
step 4814/19560 | loss 3.586123 (-0.20z)| norm 0.2629 (-0.82z)| lr 5.32e-04 | 322.59 ms | 52.3% bf16 MFU | 1624203 tok/s
step 4815/19560 | loss 3.593143 (-0.06z)| norm 0.2706 (-0.49z)| lr 5.32e-04 | 323.02 ms | 52.2% bf16 MFU | 1624146 tok/s
step 4816/19560 | loss 3.611032 (+0.33z)| norm 0.2698 (-0.51z)| lr 5.32e-04 | 322.93 ms | 52.3% bf16 MFU | 1624116 tok/s
step 4817/19560 | loss 3.572990 (-0.48z)| norm 0.2863 (+0.25z)| lr 5.32e-04 | 322.87 ms | 52.3% bf16 MFU | 1624102 tok/s
step 4818/19560 | loss 3.606188 (+0.21z)| norm 0.2710 (-0.47z)| lr 5.32e-04 | 322.36 ms | 52.4% bf16 MFU | 1624218 tok/s
step 4819/19560 | loss 3.525672 (-1.53z)| norm 0.2726 (-0.40z)| lr 5.32e-04 | 322.96 ms | 52.3% bf16 MFU | 1624175 tok/s
step 4820/19560 | loss 3.553436 (-0.92z)| norm 0.2647 (-0.76z)| lr 5.32e-04 | 322.77 ms | 52.3% bf16 MFU | 1624184 tok/s
step 4821/19560 | loss 3.595619 (-0.00z)| norm 0.2594 (-1.01z)| lr 5.32e-04 | 322.32 ms | 52.4% bf16 MFU | 1624304 tok/s
step 4822/19560 | loss 3.583164 (-0.28z)| norm 0.2568 (-1.12z)| lr 5.32e-04 | 322.79 ms | 52.3% bf16 MFU | 1624300 tok/s
step 4823/19560 | loss 3.551986 (-0.94z)| norm 0.2512 (-1.36z)| lr 5.32e-04 | 322.68 ms | 52.3% bf16 MFU | 1624324 tok/s
step 4824/19560 | loss 3.625840 (+0.65z)| norm 0.3168 (+1.67z)| lr 5.32e-04 | 322.46 ms | 52.3% bf16 MFU | 1624403 tok/s
step 4825/19560 | loss 3.578736 (-0.37z)| norm 0.3070 (+1.20z)| lr 5.32e-04 | 322.95 ms | 52.3% bf16 MFU | 1624356 tok/s
step 4826/19560 | loss 3.536916 (-1.25z)| norm 0.2985 (+0.80z)| lr 5.32e-04 | 323.60 ms | 52.2% bf16 MFU | 1624146 tok/s
step 4827/19560 | loss 3.572714 (-0.49z)| norm 0.2757 (-0.25z)| lr 5.32e-04 | 322.57 ms | 52.3% bf16 MFU | 1624204 tok/s
step 4828/19560 | loss 3.645069 (+1.05z)| norm 0.2701 (-0.51z)| lr 5.32e-04 | 322.37 ms | 52.4% bf16 MFU | 1624312 tok/s
step 4829/19560 | loss 3.575351 (-0.45z)| norm 0.2958 (+0.66z)| lr 5.32e-04 | 322.76 ms | 52.3% bf16 MFU | 1624315 tok/s
step 4830/19560 | loss 3.622540 (+0.56z)| norm 0.2442 (-1.68z)| lr 5.32e-04 | 322.98 ms | 52.3% bf16 MFU | 1624263 tok/s
step 4831/19560 | loss 3.570714 (-0.55z)| norm 0.2682 (-0.58z)| lr 5.32e-04 | 323.05 ms | 52.2% bf16 MFU | 1624196 tok/s
step 4832/19560 | loss 3.588078 (-0.18z)| norm 0.2709 (-0.46z)| lr 5.32e-04 | 322.75 ms | 52.3% bf16 MFU | 1624208 tok/s
step 4833/19560 | loss 3.576826 (-0.43z)| norm 0.2962 (+0.68z)| lr 5.32e-04 | 322.70 ms | 52.3% bf16 MFU | 1624233 tok/s
step 4834/19560 | loss 3.601878 (+0.13z)| norm 0.3379 (+2.49z)| lr 5.32e-04 | 322.18 ms | 52.4% bf16 MFU | 1624386 tok/s
step 4835/19560 | loss 3.583302 (-0.28z)| norm 0.3163 (+1.50z)| lr 5.32e-04 | 323.14 ms | 52.2% bf16 MFU | 1624291 tok/s
step 4836/19560 | loss 3.690921 (+2.05z)| norm 0.3014 (+0.84z)| lr 5.32e-04 | 322.86 ms | 52.3% bf16 MFU | 1624270 tok/s
step 4837/19560 | loss 3.577185 (-0.46z)| norm 0.3328 (+2.16z)| lr 5.32e-04 | 322.46 ms | 52.3% bf16 MFU | 1624352 tok/s
step 4838/19560 | loss 3.482328 (-2.90z)| norm 0.2749 (-0.33z)| lr 5.32e-04 | 322.68 ms | 52.3% bf16 MFU | 1624373 tok/s
step 4839/19560 | loss 3.508560 (-2.16z)| norm 0.3002 (+0.75z)| lr 5.32e-04 | 322.78 ms | 52.3% bf16 MFU | 1624368 tok/s
step 4840/19560 | loss 3.654976 (+1.58z)| norm 0.2979 (+0.68z)| lr 5.31e-04 | 322.69 ms | 52.3% bf16 MFU | 1624387 tok/s
step 4841/19560 | loss 3.595369 (+0.07z)| norm 0.2631 (-0.84z)| lr 5.31e-04 | 323.46 ms | 52.2% bf16 MFU | 1624212 tok/s
step 4842/19560 | loss 3.590986 (-0.04z)| norm 0.2904 (+0.36z)| lr 5.31e-04 | 322.91 ms | 52.3% bf16 MFU | 1624182 tok/s
step 4843/19560 | loss 3.549397 (-1.09z)| norm 0.2632 (-0.84z)| lr 5.31e-04 | 322.63 ms | 52.3% bf16 MFU | 1624225 tok/s
step 4844/19560 | loss 3.629720 (+0.94z)| norm 0.2787 (-0.15z)| lr 5.31e-04 | 322.54 ms | 52.3% bf16 MFU | 1624290 tok/s
step 4845/19560 | loss 3.613577 (+0.52z)| norm 0.3253 (+1.89z)| lr 5.31e-04 | 322.33 ms | 52.4% bf16 MFU | 1624403 tok/s
step 4846/19560 | loss 3.636283 (+1.10z)| norm 0.3328 (+2.17z)| lr 5.31e-04 | 322.90 ms | 52.3% bf16 MFU | 1624368 tok/s
step 4847/19560 | loss 3.566817 (-0.65z)| norm 0.2790 (-0.17z)| lr 5.31e-04 | 323.36 ms | 52.2% bf16 MFU | 1624220 tok/s
step 4848/19560 | loss 3.635456 (+1.08z)| norm 0.2910 (+0.36z)| lr 5.31e-04 | 322.65 ms | 52.3% bf16 MFU | 1624256 tok/s
step 4849/19560 | loss 3.528602 (-1.60z)| norm 0.2986 (+0.67z)| lr 5.31e-04 | 322.59 ms | 52.3% bf16 MFU | 1624306 tok/s
step 4850/19560 | loss 3.582729 (-0.24z)| norm 0.3256 (+1.81z)| lr 5.31e-04 | 323.27 ms | 52.2% bf16 MFU | 1624182 tok/s
step 4851/19560 | loss 3.567109 (-0.63z)| norm 0.3098 (+1.11z)| lr 5.31e-04 | 322.35 ms | 52.4% bf16 MFU | 1624296 tok/s
step 4852/19560 | loss 3.553664 (-0.95z)| norm 0.2984 (+0.61z)| lr 5.31e-04 | 323.38 ms | 52.2% bf16 MFU | 1624146 tok/s
step 4853/19560 | loss 3.569546 (-0.54z)| norm 0.2939 (+0.42z)| lr 5.31e-04 | 322.52 ms | 52.3% bf16 MFU | 1624218 tok/s
step 4854/19560 | loss 3.611784 (+0.52z)| norm 0.2717 (-0.55z)| lr 5.31e-04 | 322.40 ms | 52.3% bf16 MFU | 1624318 tok/s
step 4855/19560 | loss 3.606586 (+0.38z)| norm 0.2761 (-0.37z)| lr 5.31e-04 | 322.94 ms | 52.3% bf16 MFU | 1624276 tok/s
step 4856/19560 | loss 3.573113 (-0.48z)| norm 0.2789 (-0.25z)| lr 5.31e-04 | 322.95 ms | 52.3% bf16 MFU | 1624234 tok/s
step 4857/19560 | loss 3.583066 (-0.23z)| norm 0.2653 (-0.84z)| lr 5.31e-04 | 322.69 ms | 52.3% bf16 MFU | 1624260 tok/s
step 4858/19560 | loss 3.570724 (-0.55z)| norm 0.2769 (-0.33z)| lr 5.31e-04 | 322.68 ms | 52.3% bf16 MFU | 1624286 tok/s
step 4859/19560 | loss 3.528383 (-1.62z)| norm 0.2762 (-0.36z)| lr 5.31e-04 | 322.37 ms | 52.4% bf16 MFU | 1624390 tok/s
step 4860/19560 | loss 3.605096 (+0.37z)| norm 0.2633 (-0.92z)| lr 5.31e-04 | 322.41 ms | 52.3% bf16 MFU | 1624479 tok/s
step 4861/19560 | loss 3.565886 (-0.66z)| norm 0.3282 (+1.86z)| lr 5.31e-04 | 322.91 ms | 52.3% bf16 MFU | 1624436 tok/s
step 4862/19560 | loss 3.547728 (-1.12z)| norm 0.2931 (+0.36z)| lr 5.31e-04 | 323.45 ms | 52.2% bf16 MFU | 1624260 tok/s
step 4863/19560 | loss 3.781799 (+4.51z)| norm 0.2726 (-0.51z)| lr 5.31e-04 | 322.36 ms | 52.4% bf16 MFU | 1624368 tok/s
step 4864/19560 | loss 3.571758 (-0.49z)| norm 0.3034 (+0.80z)| lr 5.31e-04 | 322.71 ms | 52.3% bf16 MFU | 1624382 tok/s
step 4865/19560 | loss 3.638919 (+1.10z)| norm 0.2823 (-0.12z)| lr 5.31e-04 | 322.99 ms | 52.3% bf16 MFU | 1624324 tok/s
step 4866/19560 | loss 3.566359 (-0.61z)| norm 0.3097 (+1.05z)| lr 5.31e-04 | 323.25 ms | 52.2% bf16 MFU | 1624205 tok/s
step 4867/19560 | loss 3.570832 (-0.50z)| norm 0.3382 (+2.22z)| lr 5.31e-04 | 322.64 ms | 52.3% bf16 MFU | 1624244 tok/s
step 4868/19560 | loss 3.550492 (-0.97z)| norm 0.3252 (+1.64z)| lr 5.31e-04 | 322.17 ms | 52.4% bf16 MFU | 1624400 tok/s
step 4869/19560 | loss 3.532954 (-1.37z)| norm 0.2892 (+0.13z)| lr 5.31e-04 | 323.11 ms | 52.2% bf16 MFU | 1624311 tok/s
step 4870/19560 | loss 3.539689 (-1.19z)| norm 0.2864 (+0.01z)| lr 5.31e-04 | 323.27 ms | 52.2% bf16 MFU | 1624188 tok/s
step 4871/19560 | loss 3.608091 (+0.40z)| norm 0.2877 (+0.06z)| lr 5.30e-04 | 322.52 ms | 52.3% bf16 MFU | 1624259 tok/s
step 4872/19560 | loss 3.636342 (+1.04z)| norm 0.3107 (+1.01z)| lr 5.30e-04 | 322.57 ms | 52.3% bf16 MFU | 1624313 tok/s
step 4873/19560 | loss 3.561223 (-0.70z)| norm 0.3029 (+0.67z)| lr 5.30e-04 | 324.91 ms | 51.9% bf16 MFU | 1623780 tok/s
step 4874/19560 | loss 3.588705 (-0.06z)| norm 0.3252 (+1.58z)| lr 5.30e-04 | 322.19 ms | 52.4% bf16 MFU | 1623956 tok/s
step 4875/19560 | loss 3.645760 (+1.25z)| norm 0.3588 (+2.87z)| lr 5.30e-04 | 322.73 ms | 52.3% bf16 MFU | 1623985 tok/s
step 4876/19560 | loss 3.550030 (-0.98z)| norm 0.2793 (-0.37z)| lr 5.30e-04 | 322.07 ms | 52.4% bf16 MFU | 1624178 tok/s
step 4877/19560 | loss 3.604089 (+0.27z)| norm 0.2982 (+0.40z)| lr 5.30e-04 | 323.40 ms | 52.2% bf16 MFU | 1624028 tok/s
step 4878/19560 | loss 3.529286 (-1.45z)| norm 0.2880 (-0.03z)| lr 5.30e-04 | 323.02 ms | 52.2% bf16 MFU | 1623980 tok/s
step 4879/19560 | loss 3.546999 (-1.03z)| norm 0.2849 (-0.14z)| lr 5.30e-04 | 323.15 ms | 52.2% bf16 MFU | 1623903 tok/s
step 4880/19560 | loss 3.637097 (+1.03z)| norm 0.2527 (-1.44z)| lr 5.30e-04 | 323.60 ms | 52.2% bf16 MFU | 1623716 tok/s
step 4881/19560 | loss 3.698140 (+2.38z)| norm 0.2833 (-0.21z)| lr 5.30e-04 | 323.23 ms | 52.2% bf16 MFU | 1623633 tok/s
step 4882/19560 | loss 3.580644 (-0.27z)| norm 0.2635 (-1.01z)| lr 5.30e-04 | 323.15 ms | 52.2% bf16 MFU | 1623574 tok/s
step 4883/19560 | loss 3.549857 (-0.95z)| norm 0.2525 (-1.45z)| lr 5.30e-04 | 322.99 ms | 52.3% bf16 MFU | 1623558 tok/s
step 4884/19560 | loss 3.614168 (+0.49z)| norm 0.2953 (+0.29z)| lr 5.30e-04 | 323.51 ms | 52.2% bf16 MFU | 1623412 tok/s
step 4885/19560 | loss 3.579690 (-0.27z)| norm 0.3137 (+1.04z)| lr 5.30e-04 | 323.25 ms | 52.2% bf16 MFU | 1623338 tok/s
step 4886/19560 | loss 3.504712 (-1.93z)| norm 0.2746 (-0.54z)| lr 5.30e-04 | 322.60 ms | 52.3% bf16 MFU | 1623432 tok/s
step 4887/19560 | loss 3.491343 (-2.17z)| norm 0.2709 (-0.68z)| lr 5.30e-04 | 323.19 ms | 52.2% bf16 MFU | 1623370 tok/s
step 4888/19560 | loss 3.621322 (+0.69z)| norm 0.3132 (+1.07z)| lr 5.30e-04 | 322.84 ms | 52.3% bf16 MFU | 1623401 tok/s
step 4889/19560 | loss 3.585600 (-0.10z)| norm 0.2963 (+0.38z)| lr 5.30e-04 | 323.09 ms | 52.2% bf16 MFU | 1623368 tok/s
step 4890/19560 | loss 3.515809 (-1.60z)| norm 0.2879 (+0.05z)| lr 5.30e-04 | 322.91 ms | 52.3% bf16 MFU | 1623382 tok/s
step 4891/19560 | loss 3.586554 (-0.05z)| norm 0.2624 (-1.02z)| lr 5.30e-04 | 323.13 ms | 52.2% bf16 MFU | 1623338 tok/s
step 4892/19560 | loss 3.612346 (+0.50z)| norm 0.2828 (-0.13z)| lr 5.30e-04 | 323.18 ms | 52.2% bf16 MFU | 1623286 tok/s
step 4893/19560 | loss 3.622996 (+0.72z)| norm 0.2977 (+0.52z)| lr 5.30e-04 | 323.01 ms | 52.3% bf16 MFU | 1623279 tok/s
step 4894/19560 | loss 3.604001 (+0.32z)| norm 0.2916 (+0.26z)| lr 5.30e-04 | 322.86 ms | 52.3% bf16 MFU | 1623308 tok/s
step 4895/19560 | loss 3.551603 (-0.82z)| norm 0.2967 (+0.47z)| lr 5.30e-04 | 323.75 ms | 52.1% bf16 MFU | 1623115 tok/s
step 4896/19560 | loss 3.616151 (+0.61z)| norm 0.2706 (-0.67z)| lr 5.30e-04 | 323.39 ms | 52.2% bf16 MFU | 1623021 tok/s
step 4897/19560 | loss 3.623251 (+0.76z)| norm 0.3030 (+0.75z)| lr 5.30e-04 | 322.35 ms | 52.4% bf16 MFU | 1623194 tok/s
step 4898/19560 | loss 3.534936 (-1.18z)| norm 0.2693 (-0.73z)| lr 5.30e-04 | 323.51 ms | 52.2% bf16 MFU | 1623066 tok/s
step 4899/19560 | loss 3.639417 (+1.11z)| norm 0.3103 (+1.05z)| lr 5.30e-04 | 322.87 ms | 52.3% bf16 MFU | 1623105 tok/s
step 4900/19560 | loss 3.576063 (-0.28z)| norm 0.2657 (-0.90z)| lr 5.30e-04 | 322.60 ms | 52.3% bf16 MFU | 1623211 tok/s
step 4901/19560 | loss 3.596001 (+0.16z)| norm 0.2800 (-0.28z)| lr 5.30e-04 | 322.97 ms | 52.3% bf16 MFU | 1623218 tok/s
step 4902/19560 | loss 3.577723 (-0.24z)| norm 0.2661 (-0.89z)| lr 5.29e-04 | 323.55 ms | 52.2% bf16 MFU | 1623079 tok/s
step 4903/19560 | loss 3.601409 (+0.30z)| norm 0.2749 (-0.52z)| lr 5.29e-04 | 323.13 ms | 52.2% bf16 MFU | 1623052 tok/s
step 4904/19560 | loss 3.501549 (-1.89z)| norm 0.2572 (-1.28z)| lr 5.29e-04 | 323.12 ms | 52.2% bf16 MFU | 1623029 tok/s
step 4905/19560 | loss 3.568003 (-0.41z)| norm 0.2802 (-0.28z)| lr 5.29e-04 | 323.61 ms | 52.2% bf16 MFU | 1622883 tok/s
step 4906/19560 | loss 3.507767 (-1.71z)| norm 0.2595 (-1.18z)| lr 5.29e-04 | 322.70 ms | 52.3% bf16 MFU | 1622975 tok/s
step 4907/19560 | loss 3.513027 (-1.57z)| norm 0.2774 (-0.40z)| lr 5.29e-04 | 323.33 ms | 52.2% bf16 MFU | 1622901 tok/s
step 4908/19560 | loss 3.551007 (-0.75z)| norm 0.2970 (+0.45z)| lr 5.29e-04 | 322.97 ms | 52.3% bf16 MFU | 1622924 tok/s
step 4909/19560 | loss 3.606787 (+0.46z)| norm 0.2700 (-0.75z)| lr 5.29e-04 | 323.22 ms | 52.2% bf16 MFU | 1622881 tok/s
step 4910/19560 | loss 3.547951 (-0.81z)| norm 0.2727 (-0.62z)| lr 5.29e-04 | 323.16 ms | 52.2% bf16 MFU | 1622856 tok/s
step 4911/19560 | loss 3.627520 (+0.94z)| norm 0.2455 (-1.81z)| lr 5.29e-04 | 323.30 ms | 52.2% bf16 MFU | 1622796 tok/s
step 4912/19560 | loss 3.528666 (-1.24z)| norm 0.2505 (-1.59z)| lr 5.29e-04 | 322.93 ms | 52.3% bf16 MFU | 1622834 tok/s
step 4913/19560 | loss 3.524654 (-1.31z)| norm 0.2703 (-0.69z)| lr 5.29e-04 | 322.61 ms | 52.3% bf16 MFU | 1622950 tok/s
step 4914/19560 | loss 3.600785 (+0.36z)| norm 0.2673 (-0.82z)| lr 5.29e-04 | 322.56 ms | 52.3% bf16 MFU | 1623071 tok/s
step 4915/19560 | loss 3.591660 (+0.15z)| norm 0.2730 (-0.55z)| lr 5.29e-04 | 323.10 ms | 52.2% bf16 MFU | 1623052 tok/s
step 4916/19560 | loss 3.528792 (-1.23z)| norm 0.2494 (-1.60z)| lr 5.29e-04 | 323.15 ms | 52.2% bf16 MFU | 1623021 tok/s
step 4917/19560 | loss 3.677633 (+2.01z)| norm 0.2758 (-0.42z)| lr 5.29e-04 | 322.49 ms | 52.3% bf16 MFU | 1623158 tok/s
step 4918/19560 | loss 3.613519 (+0.61z)| norm 0.2883 (+0.14z)| lr 5.29e-04 | 322.99 ms | 52.3% bf16 MFU | 1623162 tok/s
step 4919/19560 | loss 3.529027 (-1.20z)| norm 0.2978 (+0.57z)| lr 5.29e-04 | 323.15 ms | 52.2% bf16 MFU | 1623125 tok/s
step 4920/19560 | loss 3.607602 (+0.49z)| norm 0.2684 (-0.75z)| lr 5.29e-04 | 322.61 ms | 52.3% bf16 MFU | 1623226 tok/s
step 4921/19560 | loss 3.640103 (+1.19z)| norm 0.2953 (+0.47z)| lr 5.29e-04 | 323.21 ms | 52.2% bf16 MFU | 1623171 tok/s
step 4922/19560 | loss 3.544150 (-0.87z)| norm 0.3098 (+1.11z)| lr 5.29e-04 | 323.28 ms | 52.2% bf16 MFU | 1623102 tok/s
step 4923/19560 | loss 3.587632 (+0.07z)| norm 0.2967 (+0.51z)| lr 5.29e-04 | 322.82 ms | 52.3% bf16 MFU | 1623152 tok/s
step 4924/19560 | loss 3.568313 (-0.33z)| norm 0.3032 (+0.81z)| lr 5.29e-04 | 322.66 ms | 52.3% bf16 MFU | 1623238 tok/s
step 4925/19560 | loss 3.586571 (+0.07z)| norm 0.2881 (+0.13z)| lr 5.29e-04 | 323.00 ms | 52.3% bf16 MFU | 1623234 tok/s
step 4926/19560 | loss 3.582288 (-0.01z)| norm 0.2916 (+0.28z)| lr 5.29e-04 | 323.79 ms | 52.1% bf16 MFU | 1623033 tok/s
step 4927/19560 | loss 3.599340 (+0.38z)| norm 0.3238 (+1.71z)| lr 5.29e-04 | 323.53 ms | 52.2% bf16 MFU | 1622907 tok/s
step 4928/19560 | loss 3.582597 (+0.00z)| norm 0.3206 (+1.55z)| lr 5.29e-04 | 323.15 ms | 52.2% bf16 MFU | 1622884 tok/s
step 4929/19560 | loss 3.575550 (-0.16z)| norm 0.3549 (+2.96z)| lr 5.29e-04 | 323.04 ms | 52.2% bf16 MFU | 1622888 tok/s
step 4930/19560 | loss 3.627229 (+0.98z)| norm 0.3499 (+2.65z)| lr 5.29e-04 | 323.91 ms | 52.1% bf16 MFU | 1622676 tok/s
step 4931/19560 | loss 3.652840 (+1.52z)| norm 0.3404 (+2.18z)| lr 5.29e-04 | 322.92 ms | 52.3% bf16 MFU | 1622721 tok/s
step 4932/19560 | loss 3.452798 (-2.83z)| norm 0.3814 (+3.66z)| lr 5.29e-04 | 322.86 ms | 52.3% bf16 MFU | 1622778 tok/s
step 4933/19560 | loss 3.565119 (-0.40z)| norm 0.3202 (+1.24z)| lr 5.28e-04 | 323.93 ms | 52.1% bf16 MFU | 1622565 tok/s
step 4934/19560 | loss 3.621637 (+0.81z)| norm 0.3314 (+1.64z)| lr 5.28e-04 | 322.66 ms | 52.3% bf16 MFU | 1622682 tok/s
step 4935/19560 | loss 3.598449 (+0.31z)| norm 0.3030 (+0.53z)| lr 5.28e-04 | 323.13 ms | 52.2% bf16 MFU | 1622674 tok/s
step 4936/19560 | loss 3.559046 (-0.54z)| norm 0.3319 (+1.62z)| lr 5.28e-04 | 323.75 ms | 52.1% bf16 MFU | 1622513 tok/s
step 4937/19560 | loss 3.601956 (+0.39z)| norm 0.3170 (+1.04z)| lr 5.28e-04 | 323.11 ms | 52.2% bf16 MFU | 1622519 tok/s
step 4938/19560 | loss 3.601838 (+0.39z)| norm 0.3353 (+1.71z)| lr 5.28e-04 | 322.98 ms | 52.3% bf16 MFU | 1622558 tok/s
step 4939/19560 | loss 3.568251 (-0.33z)| norm 0.2823 (-0.31z)| lr 5.28e-04 | 323.68 ms | 52.1% bf16 MFU | 1622420 tok/s
step 4940/19560 | loss 3.554327 (-0.63z)| norm 0.2985 (+0.30z)| lr 5.28e-04 | 322.95 ms | 52.3% bf16 MFU | 1622469 tok/s
step 4941/19560 | loss 3.617737 (+0.76z)| norm 0.2706 (-0.78z)| lr 5.28e-04 | 323.05 ms | 52.2% bf16 MFU | 1622492 tok/s
step 4942/19560 | loss 3.534097 (-1.07z)| norm 0.2505 (-1.53z)| lr 5.28e-04 | 322.66 ms | 52.3% bf16 MFU | 1622612 tok/s
step 4943/19560 | loss 3.505379 (-1.66z)| norm 0.2566 (-1.29z)| lr 5.28e-04 | 323.49 ms | 52.2% bf16 MFU | 1622519 tok/s
step 4944/19560 | loss 3.621428 (+0.85z)| norm 0.2606 (-1.13z)| lr 5.28e-04 | 322.99 ms | 52.3% bf16 MFU | 1622555 tok/s
step 4945/19560 | loss 3.558269 (-0.51z)| norm 0.2568 (-1.26z)| lr 5.28e-04 | 323.06 ms | 52.2% bf16 MFU | 1622571 tok/s
step 4946/19560 | loss 3.674625 (+1.96z)| norm 0.2649 (-0.95z)| lr 5.28e-04 | 323.42 ms | 52.2% bf16 MFU | 1622497 tok/s
step 4947/19560 | loss 3.589201 (+0.13z)| norm 0.2550 (-1.32z)| lr 5.28e-04 | 324.27 ms | 52.0% bf16 MFU | 1622213 tok/s
step 4948/19560 | loss 3.517303 (-1.39z)| norm 0.2487 (-1.54z)| lr 5.28e-04 | 323.43 ms | 52.2% bf16 MFU | 1622154 tok/s
step 4949/19560 | loss 3.673353 (+1.89z)| norm 0.2674 (-0.85z)| lr 5.28e-04 | 322.54 ms | 52.3% bf16 MFU | 1622321 tok/s
step 4950/19560 | loss 3.529061 (-1.13z)| norm 0.2805 (-0.36z)| lr 5.28e-04 | 323.18 ms | 52.2% bf16 MFU | 1622320 tok/s
step 4951/19560 | loss 3.579705 (-0.07z)| norm 0.2550 (-1.32z)| lr 5.28e-04 | 323.20 ms | 52.2% bf16 MFU | 1622314 tok/s
step 4952/19560 | loss 3.611620 (+0.60z)| norm 0.2553 (-1.29z)| lr 5.28e-04 | 323.15 ms | 52.2% bf16 MFU | 1622320 tok/s
step 4953/19560 | loss 3.537588 (-0.94z)| norm 0.2549 (-1.28z)| lr 5.28e-04 | 323.35 ms | 52.2% bf16 MFU | 1622275 tok/s
step 4954/19560 | loss 3.671439 (+1.81z)| norm 0.2678 (-0.79z)| lr 5.28e-04 | 323.54 ms | 52.2% bf16 MFU | 1622186 tok/s
step 4955/19560 | loss 3.555200 (-0.59z)| norm 0.2645 (-0.91z)| lr 5.28e-04 | 323.28 ms | 52.2% bf16 MFU | 1622165 tok/s
step 4956/19560 | loss 3.566313 (-0.35z)| norm 0.2934 (+0.16z)| lr 5.28e-04 | 322.61 ms | 52.3% bf16 MFU | 1622313 tok/s
step 4957/19560 | loss 3.547706 (-0.73z)| norm 0.2588 (-1.11z)| lr 5.28e-04 | 322.95 ms | 52.3% bf16 MFU | 1622369 tok/s
step 4958/19560 | loss 3.519458 (-1.29z)| norm 0.2779 (-0.42z)| lr 5.28e-04 | 323.75 ms | 52.1% bf16 MFU | 1622222 tok/s
step 4959/19560 | loss 3.628613 (+0.95z)| norm 0.2694 (-0.74z)| lr 5.28e-04 | 322.70 ms | 52.3% bf16 MFU | 1622346 tok/s
step 4960/19560 | loss 3.746482 (+3.22z)| norm 0.2522 (-1.37z)| lr 5.28e-04 | 322.74 ms | 52.3% bf16 MFU | 1622452 tok/s
step 4961/19560 | loss 3.558087 (-0.50z)| norm 0.2758 (-0.49z)| lr 5.28e-04 | 322.56 ms | 52.3% bf16 MFU | 1622599 tok/s
step 4962/19560 | loss 3.608388 (+0.49z)| norm 0.2553 (-1.23z)| lr 5.28e-04 | 323.01 ms | 52.2% bf16 MFU | 1622626 tok/s
step 4963/19560 | loss 3.525233 (-1.14z)| norm 0.2755 (-0.47z)| lr 5.28e-04 | 322.93 ms | 52.3% bf16 MFU | 1622672 tok/s
step 4964/19560 | loss 3.525616 (-1.12z)| norm 0.2585 (-1.09z)| lr 5.27e-04 | 323.23 ms | 52.2% bf16 MFU | 1622639 tok/s
step 4965/19560 | loss 3.571973 (-0.20z)| norm 0.2629 (-0.91z)| lr 5.27e-04 | 323.04 ms | 52.2% bf16 MFU | 1622656 tok/s
step 4966/19560 | loss 3.533950 (-0.97z)| norm 0.2883 (+0.04z)| lr 5.27e-04 | 322.85 ms | 52.3% bf16 MFU | 1622721 tok/s
step 4967/19560 | loss 3.591981 (+0.19z)| norm 0.2456 (-1.54z)| lr 5.27e-04 | 322.73 ms | 52.3% bf16 MFU | 1622811 tok/s
step 4968/19560 | loss 3.588772 (+0.13z)| norm 0.2707 (-0.59z)| lr 5.27e-04 | 322.71 ms | 52.3% bf16 MFU | 1622904 tok/s
step 4969/19560 | loss 3.529004 (-1.08z)| norm 0.2896 (+0.11z)| lr 5.27e-04 | 323.05 ms | 52.2% bf16 MFU | 1622906 tok/s
step 4970/19560 | loss 3.593068 (+0.23z)| norm 0.2907 (+0.15z)| lr 5.27e-04 | 322.84 ms | 52.3% bf16 MFU | 1622960 tok/s
step 4971/19560 | loss 3.477177 (-2.09z)| norm 0.2517 (-1.31z)| lr 5.27e-04 | 323.32 ms | 52.2% bf16 MFU | 1622890 tok/s
step 4972/19560 | loss 3.516619 (-1.28z)| norm 0.2732 (-0.50z)| lr 5.27e-04 | 322.88 ms | 52.3% bf16 MFU | 1622934 tok/s
step 4973/19560 | loss 3.540050 (-0.79z)| norm 0.3163 (+1.12z)| lr 5.27e-04 | 322.49 ms | 52.3% bf16 MFU | 1623075 tok/s
step 4974/19560 | loss 3.608448 (+0.58z)| norm 0.2803 (-0.22z)| lr 5.27e-04 | 322.78 ms | 52.3% bf16 MFU | 1623136 tok/s
step 4975/19560 | loss 3.516687 (-1.25z)| norm 0.2773 (-0.33z)| lr 5.27e-04 | 323.71 ms | 52.1% bf16 MFU | 1622961 tok/s
step 4976/19560 | loss 3.537732 (-0.81z)| norm 0.3302 (+1.65z)| lr 5.27e-04 | 322.52 ms | 52.3% bf16 MFU | 1623093 tok/s
step 4977/19560 | loss 3.524551 (-1.08z)| norm 0.3224 (+1.34z)| lr 5.27e-04 | 322.82 ms | 52.3% bf16 MFU | 1623143 tok/s
step 4978/19560 | loss 3.510547 (-1.34z)| norm 0.2729 (-0.50z)| lr 5.27e-04 | 323.45 ms | 52.2% bf16 MFU | 1623034 tok/s
step 4979/19560 | loss 3.550963 (-0.53z)| norm 0.2845 (-0.05z)| lr 5.27e-04 | 322.91 ms | 52.3% bf16 MFU | 1623064 tok/s
step 4980/19560 | loss 3.577769 (-0.00z)| norm 0.2955 (+0.36z)| lr 5.27e-04 | 322.69 ms | 52.3% bf16 MFU | 1623147 tok/s
step 4981/19560 | loss 3.633533 (+1.09z)| norm 0.2766 (-0.35z)| lr 5.27e-04 | 322.90 ms | 52.3% bf16 MFU | 1623173 tok/s
step 4982/19560 | loss 3.624046 (+0.90z)| norm 0.2950 (+0.34z)| lr 5.27e-04 | 323.31 ms | 52.2% bf16 MFU | 1623096 tok/s
step 4983/19560 | loss 3.618081 (+0.78z)| norm 0.2840 (-0.08z)| lr 5.27e-04 | 322.58 ms | 52.3% bf16 MFU | 1623206 tok/s
step 4984/19560 | loss 3.559576 (-0.37z)| norm 0.2791 (-0.26z)| lr 5.27e-04 | 323.16 ms | 52.2% bf16 MFU | 1623164 tok/s
step 4985/19560 | loss 3.547417 (-0.61z)| norm 0.2783 (-0.29z)| lr 5.27e-04 | 322.67 ms | 52.3% bf16 MFU | 1623247 tok/s
step 4986/19560 | loss 3.499551 (-1.52z)| norm 0.2727 (-0.51z)| lr 5.27e-04 | 323.03 ms | 52.2% bf16 MFU | 1623235 tok/s
step 4987/19560 | loss 3.570168 (-0.15z)| norm 0.2519 (-1.28z)| lr 5.27e-04 | 322.55 ms | 52.3% bf16 MFU | 1623347 tok/s
step 4988/19560 | loss 3.474462 (-1.97z)| norm 0.2602 (-0.97z)| lr 5.27e-04 | 323.61 ms | 52.2% bf16 MFU | 1623185 tok/s
step 4989/19560 | loss 3.518886 (-1.11z)| norm 0.2536 (-1.20z)| lr 5.27e-04 | 322.43 ms | 52.3% bf16 MFU | 1623328 tok/s
step 4990/19560 | loss 3.598417 (+0.41z)| norm 0.2596 (-0.96z)| lr 5.27e-04 | 322.99 ms | 52.3% bf16 MFU | 1623322 tok/s
step 4991/19560 | loss 3.611427 (+0.73z)| norm 0.2547 (-1.13z)| lr 5.27e-04 | 323.10 ms | 52.2% bf16 MFU | 1623291 tok/s
step 4992/19560 | loss 3.600289 (+0.50z)| norm 0.2612 (-0.87z)| lr 5.27e-04 | 323.18 ms | 52.2% bf16 MFU | 1623241 tok/s
step 4993/19560 | loss 3.554197 (-0.43z)| norm 0.2897 (+0.19z)| lr 5.27e-04 | 323.00 ms | 52.3% bf16 MFU | 1623238 tok/s
step 4994/19560 | loss 3.540402 (-0.71z)| norm 0.3165 (+1.19z)| lr 5.27e-04 | 322.64 ms | 52.3% bf16 MFU | 1623325 tok/s
step 4995/19560 | loss 3.554491 (-0.42z)| norm 0.2996 (+0.58z)| lr 5.26e-04 | 322.44 ms | 52.3% bf16 MFU | 1623460 tok/s
step 4996/19560 | loss 3.582642 (+0.15z)| norm 0.2930 (+0.34z)| lr 5.26e-04 | 322.94 ms | 52.3% bf16 MFU | 1623462 tok/s
step 4997/19560 | loss 3.515320 (-1.22z)| norm 0.2476 (-1.38z)| lr 5.26e-04 | 323.43 ms | 52.2% bf16 MFU | 1623339 tok/s
step 4998/19560 | loss 3.513011 (-1.26z)| norm 0.2700 (-0.52z)| lr 5.26e-04 | 323.19 ms | 52.2% bf16 MFU | 1623284 tok/s
step 4999/19560 | loss 3.578470 (+0.08z)| norm 0.2655 (-0.68z)| lr 5.26e-04 | 323.57 ms | 52.2% bf16 MFU | 1623135 tok/s
step 5000/19560 | loss 3.579595 (+0.11z)| norm 0.2620 (-0.80z)| lr 5.26e-04 | 322.47 ms | 52.3% bf16 MFU | 1623271 tok/s
val loss 3.558775

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00005000_00001.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00005000_00004.bin
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2761/10042 = 0.274945
Writing checkpoint at step 5000
Writing model to log124M/model_00005000.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00005000_00003.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00005000_00002.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00005000_00005.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00005000_00007.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00005000_00006.bin
Writing state to log124M/state_00005000_00000.bin
step 5001/19560 | loss 3.563595 (-0.22z)| norm 0.2856 (+0.10z)| lr 5.26e-04 | 318.87 ms | 52.9% bf16 MFU | 1624317 tok/s
step 5002/19560 | loss 3.536031 (-0.77z)| norm 0.2709 (-0.45z)| lr 5.26e-04 | 320.99 ms | 52.6% bf16 MFU | 1624768 tok/s
step 5003/19560 | loss 3.514350 (-1.20z)| norm 0.2677 (-0.56z)| lr 5.26e-04 | 322.17 ms | 52.4% bf16 MFU | 1624898 tok/s
step 5004/19560 | loss 3.658193 (+1.72z)| norm 0.2699 (-0.47z)| lr 5.26e-04 | 321.72 ms | 52.5% bf16 MFU | 1625134 tok/s
step 5005/19560 | loss 3.565235 (-0.16z)| norm 0.2928 (+0.44z)| lr 5.26e-04 | 321.92 ms | 52.4% bf16 MFU | 1625309 tok/s
step 5006/19560 | loss 3.576202 (+0.05z)| norm 0.2744 (-0.29z)| lr 5.26e-04 | 322.74 ms | 52.3% bf16 MFU | 1625268 tok/s
step 5007/19560 | loss 3.535709 (-0.77z)| norm 0.2998 (+0.72z)| lr 5.26e-04 | 321.84 ms | 52.4% bf16 MFU | 1625458 tok/s
step 5008/19560 | loss 3.595331 (+0.45z)| norm 0.2771 (-0.19z)| lr 5.26e-04 | 322.17 ms | 52.4% bf16 MFU | 1625552 tok/s
step 5009/19560 | loss 3.576960 (+0.10z)| norm 0.2869 (+0.20z)| lr 5.26e-04 | 322.25 ms | 52.4% bf16 MFU | 1625624 tok/s
step 5010/19560 | loss 3.511806 (-1.25z)| norm 0.2900 (+0.32z)| lr 5.26e-04 | 322.71 ms | 52.3% bf16 MFU | 1625574 tok/s
step 5011/19560 | loss 3.577993 (+0.13z)| norm 0.3069 (+0.97z)| lr 5.26e-04 | 322.48 ms | 52.3% bf16 MFU | 1625586 tok/s
step 5012/19560 | loss 3.610289 (+0.80z)| norm 0.2695 (-0.51z)| lr 5.26e-04 | 322.10 ms | 52.4% bf16 MFU | 1625694 tok/s
step 5013/19560 | loss 3.542117 (-0.62z)| norm 0.2747 (-0.30z)| lr 5.26e-04 | 322.69 ms | 52.3% bf16 MFU | 1625645 tok/s
step 5014/19560 | loss 3.552410 (-0.41z)| norm 0.2806 (-0.06z)| lr 5.26e-04 | 322.51 ms | 52.3% bf16 MFU | 1625645 tok/s
step 5015/19560 | loss 3.575680 (+0.06z)| norm 0.2677 (-0.58z)| lr 5.26e-04 | 322.26 ms | 52.4% bf16 MFU | 1625710 tok/s
step 5016/19560 | loss 3.494785 (-1.63z)| norm 0.2646 (-0.69z)| lr 5.26e-04 | 322.19 ms | 52.4% bf16 MFU | 1625786 tok/s
step 5017/19560 | loss 3.573226 (+0.04z)| norm 0.2681 (-0.54z)| lr 5.26e-04 | 323.52 ms | 52.2% bf16 MFU | 1625525 tok/s
step 5018/19560 | loss 3.523031 (-1.03z)| norm 0.2777 (-0.15z)| lr 5.26e-04 | 322.47 ms | 52.3% bf16 MFU | 1625542 tok/s
step 5019/19560 | loss 3.626008 (+1.14z)| norm 0.2919 (+0.41z)| lr 5.26e-04 | 322.46 ms | 52.3% bf16 MFU | 1625561 tok/s
step 5020/19560 | loss 3.536155 (-0.74z)| norm 0.2874 (+0.23z)| lr 5.26e-04 | 322.80 ms | 52.3% bf16 MFU | 1625492 tok/s
step 5021/19560 | loss 3.512032 (-1.23z)| norm 0.2754 (-0.25z)| lr 5.26e-04 | 322.77 ms | 52.3% bf16 MFU | 1625435 tok/s
step 5022/19560 | loss 3.561490 (-0.18z)| norm 0.2946 (+0.53z)| lr 5.26e-04 | 322.54 ms | 52.3% bf16 MFU | 1625437 tok/s
step 5023/19560 | loss 3.587629 (+0.36z)| norm 0.3035 (+0.89z)| lr 5.26e-04 | 322.78 ms | 52.3% bf16 MFU | 1625379 tok/s
step 5024/19560 | loss 3.474690 (-1.98z)| norm 0.3271 (+1.81z)| lr 5.26e-04 | 322.50 ms | 52.3% bf16 MFU | 1625394 tok/s
step 5025/19560 | loss 3.630334 (+1.27z)| norm 0.2975 (+0.62z)| lr 5.25e-04 | 322.36 ms | 52.4% bf16 MFU | 1625445 tok/s
step 5026/19560 | loss 3.506300 (-1.30z)| norm 0.2726 (-0.37z)| lr 5.25e-04 | 322.90 ms | 52.3% bf16 MFU | 1625358 tok/s
step 5027/19560 | loss 3.523304 (-0.94z)| norm 0.2772 (-0.18z)| lr 5.25e-04 | 322.48 ms | 52.3% bf16 MFU | 1625380 tok/s
step 5028/19560 | loss 3.572634 (+0.09z)| norm 0.2502 (-1.26z)| lr 5.25e-04 | 323.27 ms | 52.2% bf16 MFU | 1625202 tok/s
step 5029/19560 | loss 3.549759 (-0.38z)| norm 0.2812 (-0.02z)| lr 5.25e-04 | 322.05 ms | 52.4% bf16 MFU | 1625340 tok/s
step 5030/19560 | loss 3.603853 (+0.75z)| norm 0.2843 (+0.10z)| lr 5.25e-04 | 323.11 ms | 52.2% bf16 MFU | 1625204 tok/s
step 5031/19560 | loss 3.566382 (-0.03z)| norm 0.2820 (+0.01z)| lr 5.25e-04 | 323.23 ms | 52.2% bf16 MFU | 1625045 tok/s
step 5032/19560 | loss 3.549970 (-0.38z)| norm 0.2563 (-1.02z)| lr 5.25e-04 | 322.49 ms | 52.3% bf16 MFU | 1625081 tok/s
step 5033/19560 | loss 3.572929 (+0.10z)| norm 0.2679 (-0.55z)| lr 5.25e-04 | 322.72 ms | 52.3% bf16 MFU | 1625055 tok/s
step 5034/19560 | loss 3.529986 (-0.81z)| norm 0.2679 (-0.56z)| lr 5.25e-04 | 322.70 ms | 52.3% bf16 MFU | 1625038 tok/s
step 5035/19560 | loss 3.540280 (-0.60z)| norm 0.2741 (-0.31z)| lr 5.25e-04 | 322.53 ms | 52.3% bf16 MFU | 1625065 tok/s
step 5036/19560 | loss 3.537796 (-0.65z)| norm 0.2785 (-0.13z)| lr 5.25e-04 | 322.80 ms | 52.3% bf16 MFU | 1625021 tok/s
step 5037/19560 | loss 3.563147 (-0.11z)| norm 0.2692 (-0.50z)| lr 5.25e-04 | 322.56 ms | 52.3% bf16 MFU | 1625039 tok/s
step 5038/19560 | loss 3.583719 (+0.33z)| norm 0.2555 (-1.04z)| lr 5.25e-04 | 322.85 ms | 52.3% bf16 MFU | 1624984 tok/s
step 5039/19560 | loss 3.557359 (-0.22z)| norm 0.2653 (-0.66z)| lr 5.25e-04 | 322.54 ms | 52.3% bf16 MFU | 1625010 tok/s
step 5040/19560 | loss 3.551732 (-0.35z)| norm 0.2726 (-0.37z)| lr 5.25e-04 | 322.62 ms | 52.3% bf16 MFU | 1625014 tok/s
step 5041/19560 | loss 3.587013 (+0.40z)| norm 0.2455 (-1.45z)| lr 5.25e-04 | 323.18 ms | 52.2% bf16 MFU | 1624876 tok/s
step 5042/19560 | loss 3.589133 (+0.44z)| norm 0.3088 (+1.08z)| lr 5.25e-04 | 322.32 ms | 52.4% bf16 MFU | 1624961 tok/s
step 5043/19560 | loss 3.562057 (-0.13z)| norm 0.3172 (+1.39z)| lr 5.25e-04 | 323.23 ms | 52.2% bf16 MFU | 1624813 tok/s
step 5044/19560 | loss 3.562728 (-0.12z)| norm 0.2918 (+0.37z)| lr 5.25e-04 | 322.60 ms | 52.3% bf16 MFU | 1624832 tok/s
step 5045/19560 | loss 3.547813 (-0.43z)| norm 0.2669 (-0.63z)| lr 5.25e-04 | 322.37 ms | 52.4% bf16 MFU | 1624907 tok/s
step 5046/19560 | loss 3.583993 (+0.37z)| norm 0.2696 (-0.51z)| lr 5.25e-04 | 323.05 ms | 52.2% bf16 MFU | 1624809 tok/s
step 5047/19560 | loss 3.661667 (+2.04z)| norm 0.2655 (-0.66z)| lr 5.25e-04 | 322.79 ms | 52.3% bf16 MFU | 1624780 tok/s
step 5048/19560 | loss 3.569735 (+0.04z)| norm 0.2786 (-0.14z)| lr 5.25e-04 | 323.43 ms | 52.2% bf16 MFU | 1624593 tok/s
step 5049/19560 | loss 3.519613 (-1.04z)| norm 0.2904 (+0.33z)| lr 5.25e-04 | 323.00 ms | 52.3% bf16 MFU | 1624521 tok/s
step 5050/19560 | loss 3.534418 (-0.71z)| norm 0.2684 (-0.54z)| lr 5.25e-04 | 322.38 ms | 52.4% bf16 MFU | 1624611 tok/s
step 5051/19560 | loss 3.622623 (+1.21z)| norm 0.2828 (+0.04z)| lr 5.25e-04 | 322.19 ms | 52.4% bf16 MFU | 1624744 tok/s
step 5052/19560 | loss 3.516656 (-1.09z)| norm 0.2592 (-0.89z)| lr 5.25e-04 | 323.61 ms | 52.2% bf16 MFU | 1624514 tok/s
step 5053/19560 | loss 3.883112 (+5.85z)| norm 0.2807 (-0.03z)| lr 5.25e-04 | 323.21 ms | 52.2% bf16 MFU | 1624396 tok/s
step 5054/19560 | loss 3.590499 (+0.40z)| norm 0.3009 (+0.78z)| lr 5.25e-04 | 322.73 ms | 52.3% bf16 MFU | 1624404 tok/s
step 5055/19560 | loss 3.548207 (-0.38z)| norm 0.2760 (-0.21z)| lr 5.24e-04 | 322.64 ms | 52.3% bf16 MFU | 1624433 tok/s
step 5056/19560 | loss 3.528412 (-0.74z)| norm 0.2772 (-0.14z)| lr 5.24e-04 | 323.17 ms | 52.2% bf16 MFU | 1624328 tok/s
step 5057/19560 | loss 3.539780 (-0.53z)| norm 0.2565 (-1.00z)| lr 5.24e-04 | 322.97 ms | 52.3% bf16 MFU | 1624279 tok/s
step 5058/19560 | loss 3.598954 (+0.58z)| norm 0.2895 (+0.44z)| lr 5.24e-04 | 322.46 ms | 52.3% bf16 MFU | 1624360 tok/s
step 5059/19560 | loss 3.597025 (+0.56z)| norm 0.2853 (+0.28z)| lr 5.24e-04 | 323.41 ms | 52.2% bf16 MFU | 1624200 tok/s
step 5060/19560 | loss 3.586210 (+0.34z)| norm 0.2722 (-0.30z)| lr 5.24e-04 | 322.66 ms | 52.3% bf16 MFU | 1624234 tok/s
step 5061/19560 | loss 3.650757 (+1.55z)| norm 0.2689 (-0.45z)| lr 5.24e-04 | 322.93 ms | 52.3% bf16 MFU | 1624198 tok/s
step 5062/19560 | loss 3.505530 (-1.19z)| norm 0.2499 (-1.41z)| lr 5.24e-04 | 323.33 ms | 52.2% bf16 MFU | 1624064 tok/s
step 5063/19560 | loss 3.502185 (-1.23z)| norm 0.2907 (+0.71z)| lr 5.24e-04 | 322.61 ms | 52.3% bf16 MFU | 1624118 tok/s
step 5064/19560 | loss 3.525702 (-0.78z)| norm 0.2789 (+0.12z)| lr 5.24e-04 | 323.21 ms | 52.2% bf16 MFU | 1624018 tok/s
step 5065/19560 | loss 3.583863 (+0.32z)| norm 0.2684 (-0.43z)| lr 5.24e-04 | 322.80 ms | 52.3% bf16 MFU | 1624026 tok/s
step 5066/19560 | loss 3.523441 (-0.81z)| norm 0.2743 (-0.09z)| lr 5.24e-04 | 322.68 ms | 52.3% bf16 MFU | 1624066 tok/s
step 5067/19560 | loss 3.593980 (+0.51z)| norm 0.2774 (+0.09z)| lr 5.24e-04 | 323.38 ms | 52.2% bf16 MFU | 1623925 tok/s
step 5068/19560 | loss 3.609179 (+0.79z)| norm 0.2902 (+0.83z)| lr 5.24e-04 | 322.15 ms | 52.4% bf16 MFU | 1624102 tok/s
step 5069/19560 | loss 3.551098 (-0.29z)| norm 0.2914 (+0.89z)| lr 5.24e-04 | 322.88 ms | 52.3% bf16 MFU | 1624086 tok/s
step 5070/19560 | loss 3.560156 (-0.12z)| norm 0.3000 (+1.36z)| lr 5.24e-04 | 322.85 ms | 52.3% bf16 MFU | 1624078 tok/s
step 5071/19560 | loss 3.603966 (+0.69z)| norm 0.2595 (-0.96z)| lr 5.24e-04 | 323.05 ms | 52.2% bf16 MFU | 1624020 tok/s
step 5072/19560 | loss 3.598878 (+0.60z)| norm 0.2786 (+0.13z)| lr 5.24e-04 | 323.47 ms | 52.2% bf16 MFU | 1623859 tok/s
step 5073/19560 | loss 3.558131 (-0.18z)| norm 0.2662 (-0.59z)| lr 5.24e-04 | 322.47 ms | 52.3% bf16 MFU | 1623960 tok/s
step 5074/19560 | loss 3.536629 (-0.57z)| norm 0.2952 (+1.07z)| lr 5.24e-04 | 322.62 ms | 52.3% bf16 MFU | 1624018 tok/s
step 5075/19560 | loss 3.526793 (-0.75z)| norm 0.2836 (+0.39z)| lr 5.24e-04 | 323.27 ms | 52.2% bf16 MFU | 1623907 tok/s
step 5076/19560 | loss 3.557763 (-0.16z)| norm 0.2827 (+0.32z)| lr 5.24e-04 | 323.31 ms | 52.2% bf16 MFU | 1623794 tok/s
step 5077/19560 | loss 3.533645 (-0.61z)| norm 0.2974 (+1.16z)| lr 5.24e-04 | 323.27 ms | 52.2% bf16 MFU | 1623696 tok/s
step 5078/19560 | loss 3.542974 (-0.43z)| norm 0.2810 (+0.21z)| lr 5.24e-04 | 323.05 ms | 52.2% bf16 MFU | 1623658 tok/s
step 5079/19560 | loss 3.542827 (-0.43z)| norm 0.2667 (-0.63z)| lr 5.24e-04 | 322.84 ms | 52.3% bf16 MFU | 1623675 tok/s
step 5080/19560 | loss 3.543820 (-0.40z)| norm 0.2828 (+0.30z)| lr 5.24e-04 | 322.86 ms | 52.3% bf16 MFU | 1623686 tok/s
step 5081/19560 | loss 3.598244 (+0.66z)| norm 0.2717 (-0.36z)| lr 5.24e-04 | 322.61 ms | 52.3% bf16 MFU | 1623760 tok/s
step 5082/19560 | loss 3.588640 (+0.49z)| norm 0.2915 (+0.80z)| lr 5.24e-04 | 322.65 ms | 52.3% bf16 MFU | 1623820 tok/s
step 5083/19560 | loss 3.553211 (-0.22z)| norm 0.2450 (-1.92z)| lr 5.24e-04 | 323.20 ms | 52.2% bf16 MFU | 1623739 tok/s
step 5084/19560 | loss 3.566316 (+0.04z)| norm 0.2685 (-0.54z)| lr 5.24e-04 | 322.28 ms | 52.4% bf16 MFU | 1623892 tok/s
step 5085/19560 | loss 3.542235 (-0.44z)| norm 0.2665 (-0.66z)| lr 5.23e-04 | 322.36 ms | 52.4% bf16 MFU | 1624019 tok/s
step 5086/19560 | loss 3.586257 (+0.43z)| norm 0.2863 (+0.49z)| lr 5.23e-04 | 322.79 ms | 52.3% bf16 MFU | 1624030 tok/s
step 5087/19560 | loss 3.552439 (-0.23z)| norm 0.2812 (+0.19z)| lr 5.23e-04 | 322.98 ms | 52.3% bf16 MFU | 1623994 tok/s
step 5088/19560 | loss 3.584903 (+0.47z)| norm 0.2750 (-0.18z)| lr 5.23e-04 | 322.63 ms | 52.3% bf16 MFU | 1624045 tok/s
step 5089/19560 | loss 3.587380 (+0.52z)| norm 0.2789 (+0.05z)| lr 5.23e-04 | 322.69 ms | 52.3% bf16 MFU | 1624079 tok/s
step 5090/19560 | loss 3.599436 (+0.78z)| norm 0.2921 (+0.82z)| lr 5.23e-04 | 323.11 ms | 52.2% bf16 MFU | 1624008 tok/s
step 5091/19560 | loss 3.612947 (+1.05z)| norm 0.2517 (-1.57z)| lr 5.23e-04 | 322.57 ms | 52.3% bf16 MFU | 1624074 tok/s
step 5092/19560 | loss 3.572202 (+0.18z)| norm 0.2868 (+0.49z)| lr 5.23e-04 | 322.91 ms | 52.3% bf16 MFU | 1624052 tok/s
step 5093/19560 | loss 3.551220 (-0.27z)| norm 0.2857 (+0.42z)| lr 5.23e-04 | 323.35 ms | 52.2% bf16 MFU | 1623922 tok/s
step 5094/19560 | loss 3.554911 (-0.19z)| norm 0.2558 (-1.33z)| lr 5.23e-04 | 322.43 ms | 52.3% bf16 MFU | 1624029 tok/s
step 5095/19560 | loss 3.524089 (-0.84z)| norm 0.2760 (-0.16z)| lr 5.23e-04 | 322.79 ms | 52.3% bf16 MFU | 1624038 tok/s
step 5096/19560 | loss 3.552409 (-0.23z)| norm 0.2625 (-0.96z)| lr 5.23e-04 | 322.63 ms | 52.3% bf16 MFU | 1624089 tok/s
step 5097/19560 | loss 3.545948 (-0.37z)| norm 0.2694 (-0.54z)| lr 5.23e-04 | 324.05 ms | 52.1% bf16 MFU | 1623782 tok/s
step 5098/19560 | loss 3.642319 (+1.66z)| norm 0.2540 (-1.44z)| lr 5.23e-04 | 323.01 ms | 52.2% bf16 MFU | 1623749 tok/s
step 5099/19560 | loss 3.701152 (+2.82z)| norm 0.3938 (+5.87z)| lr 5.23e-04 | 322.72 ms | 52.3% bf16 MFU | 1623792 tok/s
step 5100/19560 | loss 3.566303 (+0.01z)| norm 0.3267 (+2.36z)| lr 5.23e-04 | 323.40 ms | 52.2% bf16 MFU | 1623661 tok/s
step 5101/19560 | loss 3.547609 (-0.38z)| norm 0.2932 (+0.70z)| lr 5.23e-04 | 322.96 ms | 52.3% bf16 MFU | 1623648 tok/s
step 5102/19560 | loss 3.595885 (+0.63z)| norm 0.3146 (+1.75z)| lr 5.23e-04 | 322.56 ms | 52.3% bf16 MFU | 1623735 tok/s
step 5103/19560 | loss 3.640934 (+1.54z)| norm 0.2978 (+0.90z)| lr 5.23e-04 | 323.32 ms | 52.2% bf16 MFU | 1623628 tok/s
step 5104/19560 | loss 3.567411 (+0.01z)| norm 0.2927 (+0.67z)| lr 5.23e-04 | 322.99 ms | 52.3% bf16 MFU | 1623608 tok/s
step 5105/19560 | loss 3.553794 (-0.28z)| norm 0.2976 (+0.95z)| lr 5.23e-04 | 322.82 ms | 52.3% bf16 MFU | 1623633 tok/s
step 5106/19560 | loss 3.656878 (+1.83z)| norm 0.4049 (+5.61z)| lr 5.23e-04 | 323.18 ms | 52.2% bf16 MFU | 1623566 tok/s
step 5107/19560 | loss 3.512159 (-1.15z)| norm 0.3752 (+3.97z)| lr 5.23e-04 | 322.44 ms | 52.3% bf16 MFU | 1623687 tok/s
step 5108/19560 | loss 3.583725 (+0.32z)| norm 0.3334 (+2.16z)| lr 5.23e-04 | 322.82 ms | 52.3% bf16 MFU | 1623707 tok/s
step 5109/19560 | loss 3.547131 (-0.42z)| norm 0.3251 (+1.77z)| lr 5.23e-04 | 322.68 ms | 52.3% bf16 MFU | 1623761 tok/s
step 5110/19560 | loss 3.507401 (-1.22z)| norm 0.3088 (+1.10z)| lr 5.23e-04 | 323.03 ms | 52.2% bf16 MFU | 1623726 tok/s
step 5111/19560 | loss 3.589016 (+0.47z)| norm 0.2886 (+0.27z)| lr 5.23e-04 | 322.57 ms | 52.3% bf16 MFU | 1623806 tok/s
step 5112/19560 | loss 3.539262 (-0.56z)| norm 0.2812 (-0.03z)| lr 5.23e-04 | 322.52 ms | 52.3% bf16 MFU | 1623895 tok/s
step 5113/19560 | loss 3.488577 (-1.59z)| norm 0.2721 (-0.40z)| lr 5.23e-04 | 322.61 ms | 52.3% bf16 MFU | 1623958 tok/s
step 5114/19560 | loss 3.574961 (+0.18z)| norm 0.2614 (-0.83z)| lr 5.23e-04 | 322.87 ms | 52.3% bf16 MFU | 1623952 tok/s
step 5115/19560 | loss 3.631826 (+1.34z)| norm 0.2718 (-0.42z)| lr 5.22e-04 | 322.88 ms | 52.3% bf16 MFU | 1623943 tok/s
step 5116/19560 | loss 3.589478 (+0.46z)| norm 0.2857 (+0.14z)| lr 5.22e-04 | 323.10 ms | 52.2% bf16 MFU | 1623879 tok/s
step 5117/19560 | loss 3.651671 (+1.72z)| norm 0.3049 (+0.92z)| lr 5.22e-04 | 323.27 ms | 52.2% bf16 MFU | 1623777 tok/s
step 5118/19560 | loss 3.550671 (-0.37z)| norm 0.2942 (+0.47z)| lr 5.22e-04 | 322.75 ms | 52.3% bf16 MFU | 1623809 tok/s
step 5119/19560 | loss 3.612156 (+0.91z)| norm 0.3136 (+1.25z)| lr 5.22e-04 | 323.30 ms | 52.2% bf16 MFU | 1623703 tok/s
step 5120/19560 | loss 3.563345 (-0.10z)| norm 0.2800 (-0.14z)| lr 5.22e-04 | 322.51 ms | 52.3% bf16 MFU | 1623800 tok/s
step 5121/19560 | loss 3.521057 (-0.97z)| norm 0.2799 (-0.14z)| lr 5.22e-04 | 322.71 ms | 52.3% bf16 MFU | 1623843 tok/s
step 5122/19560 | loss 3.602790 (+0.71z)| norm 0.2636 (-0.80z)| lr 5.22e-04 | 322.47 ms | 52.3% bf16 MFU | 1623942 tok/s
step 5123/19560 | loss 3.577318 (+0.18z)| norm 0.2582 (-1.01z)| lr 5.22e-04 | 322.61 ms | 52.3% bf16 MFU | 1624002 tok/s
step 5124/19560 | loss 3.562851 (-0.11z)| norm 0.2672 (-0.63z)| lr 5.22e-04 | 322.79 ms | 52.3% bf16 MFU | 1624014 tok/s
step 5125/19560 | loss 3.510437 (-1.19z)| norm 0.2759 (-0.28z)| lr 5.22e-04 | 322.49 ms | 52.3% bf16 MFU | 1624100 tok/s
step 5126/19560 | loss 3.542228 (-0.54z)| norm 0.2588 (-0.99z)| lr 5.22e-04 | 322.87 ms | 52.3% bf16 MFU | 1624088 tok/s
step 5127/19560 | loss 3.538454 (-0.61z)| norm 0.2646 (-0.75z)| lr 5.22e-04 | 322.64 ms | 52.3% bf16 MFU | 1624133 tok/s
step 5128/19560 | loss 3.538321 (-0.61z)| norm 0.2483 (-1.42z)| lr 5.22e-04 | 322.74 ms | 52.3% bf16 MFU | 1624150 tok/s
step 5129/19560 | loss 3.536960 (-0.63z)| norm 0.2629 (-0.80z)| lr 5.22e-04 | 323.03 ms | 52.2% bf16 MFU | 1624094 tok/s
step 5130/19560 | loss 3.500163 (-1.38z)| norm 0.2566 (-1.05z)| lr 5.22e-04 | 322.34 ms | 52.4% bf16 MFU | 1624216 tok/s
step 5131/19560 | loss 3.613039 (+0.92z)| norm 0.2685 (-0.56z)| lr 5.22e-04 | 322.95 ms | 52.3% bf16 MFU | 1624176 tok/s
step 5132/19560 | loss 3.580036 (+0.26z)| norm 0.2811 (-0.05z)| lr 5.22e-04 | 322.52 ms | 52.3% bf16 MFU | 1624248 tok/s
step 5133/19560 | loss 3.560100 (-0.15z)| norm 0.2775 (-0.19z)| lr 5.22e-04 | 322.16 ms | 52.4% bf16 MFU | 1624407 tok/s
step 5134/19560 | loss 3.562082 (-0.11z)| norm 0.2691 (-0.53z)| lr 5.22e-04 | 322.64 ms | 52.3% bf16 MFU | 1624438 tok/s
step 5135/19560 | loss 3.555401 (-0.25z)| norm 0.2628 (-0.78z)| lr 5.22e-04 | 322.61 ms | 52.3% bf16 MFU | 1624473 tok/s
step 5136/19560 | loss 3.589890 (+0.47z)| norm 0.2743 (-0.31z)| lr 5.22e-04 | 323.20 ms | 52.2% bf16 MFU | 1624358 tok/s
step 5137/19560 | loss 3.535004 (-0.67z)| norm 0.2780 (-0.15z)| lr 5.22e-04 | 322.59 ms | 52.3% bf16 MFU | 1624403 tok/s
step 5138/19560 | loss 3.509778 (-1.19z)| norm 0.2843 (+0.11z)| lr 5.22e-04 | 322.32 ms | 52.4% bf16 MFU | 1624514 tok/s
step 5139/19560 | loss 3.536853 (-0.62z)| norm 0.2746 (-0.28z)| lr 5.22e-04 | 322.57 ms | 52.3% bf16 MFU | 1624557 tok/s
step 5140/19560 | loss 3.494505 (-1.48z)| norm 0.2578 (-0.97z)| lr 5.22e-04 | 322.95 ms | 52.3% bf16 MFU | 1624502 tok/s
step 5141/19560 | loss 3.490445 (-1.54z)| norm 0.2654 (-0.65z)| lr 5.22e-04 | 322.70 ms | 52.3% bf16 MFU | 1624510 tok/s
step 5142/19560 | loss 3.564527 (-0.02z)| norm 0.2441 (-1.51z)| lr 5.22e-04 | 322.38 ms | 52.4% bf16 MFU | 1624600 tok/s
step 5143/19560 | loss 3.635724 (+1.42z)| norm 0.2708 (-0.42z)| lr 5.22e-04 | 322.65 ms | 52.3% bf16 MFU | 1624616 tok/s
step 5144/19560 | loss 3.530494 (-0.73z)| norm 0.2766 (-0.18z)| lr 5.22e-04 | 323.13 ms | 52.2% bf16 MFU | 1624512 tok/s
step 5145/19560 | loss 3.596481 (+0.61z)| norm 0.3289 (+1.91z)| lr 5.21e-04 | 322.91 ms | 52.3% bf16 MFU | 1624468 tok/s
step 5146/19560 | loss 3.626298 (+1.20z)| norm 0.3279 (+1.83z)| lr 5.21e-04 | 322.76 ms | 52.3% bf16 MFU | 1624465 tok/s
step 5147/19560 | loss 3.592811 (+0.53z)| norm 0.3032 (+0.84z)| lr 5.21e-04 | 322.10 ms | 52.4% bf16 MFU | 1624626 tok/s
step 5148/19560 | loss 3.561825 (-0.11z)| norm 0.2631 (-0.74z)| lr 5.21e-04 | 322.13 ms | 52.4% bf16 MFU | 1624773 tok/s
step 5149/19560 | loss 3.527388 (-0.82z)| norm 0.3008 (+0.74z)| lr 5.21e-04 | 323.49 ms | 52.2% bf16 MFU | 1624571 tok/s
step 5150/19560 | loss 3.554258 (-0.27z)| norm 0.2944 (+0.49z)| lr 5.21e-04 | 322.62 ms | 52.3% bf16 MFU | 1624596 tok/s
step 5151/19560 | loss 3.503586 (-1.29z)| norm 0.2623 (-0.77z)| lr 5.21e-04 | 322.62 ms | 52.3% bf16 MFU | 1624622 tok/s
step 5152/19560 | loss 3.584303 (+0.35z)| norm 0.2735 (-0.31z)| lr 5.21e-04 | 322.88 ms | 52.3% bf16 MFU | 1624579 tok/s
step 5153/19560 | loss 3.499645 (-1.39z)| norm 0.2536 (-1.10z)| lr 5.21e-04 | 323.13 ms | 52.2% bf16 MFU | 1624476 tok/s
step 5154/19560 | loss 3.539051 (-0.58z)| norm 0.2879 (+0.27z)| lr 5.21e-04 | 322.79 ms | 52.3% bf16 MFU | 1624464 tok/s
step 5155/19560 | loss 3.556036 (-0.23z)| norm 0.2715 (-0.38z)| lr 5.21e-04 | 322.78 ms | 52.3% bf16 MFU | 1624456 tok/s
step 5156/19560 | loss 3.526185 (-0.84z)| norm 0.3007 (+0.77z)| lr 5.21e-04 | 322.75 ms | 52.3% bf16 MFU | 1624455 tok/s
step 5157/19560 | loss 3.531451 (-0.73z)| norm 0.3203 (+1.53z)| lr 5.21e-04 | 323.06 ms | 52.2% bf16 MFU | 1624375 tok/s
step 5158/19560 | loss 3.520626 (-0.94z)| norm 0.2667 (-0.59z)| lr 5.21e-04 | 322.52 ms | 52.3% bf16 MFU | 1624437 tok/s
step 5159/19560 | loss 3.565184 (-0.01z)| norm 0.2590 (-0.89z)| lr 5.21e-04 | 323.21 ms | 52.2% bf16 MFU | 1624321 tok/s
step 5160/19560 | loss 3.594676 (+0.59z)| norm 0.2531 (-1.12z)| lr 5.21e-04 | 322.86 ms | 52.3% bf16 MFU | 1624299 tok/s
step 5161/19560 | loss 3.538949 (-0.56z)| norm 0.2658 (-0.62z)| lr 5.21e-04 | 322.78 ms | 52.3% bf16 MFU | 1624299 tok/s
step 5162/19560 | loss 3.574415 (+0.17z)| norm 0.2407 (-1.58z)| lr 5.21e-04 | 322.76 ms | 52.3% bf16 MFU | 1624304 tok/s
step 5163/19560 | loss 3.599506 (+0.68z)| norm 0.2647 (-0.64z)| lr 5.21e-04 | 322.58 ms | 52.3% bf16 MFU | 1624352 tok/s
step 5164/19560 | loss 3.499092 (-1.39z)| norm 0.2514 (-1.15z)| lr 5.21e-04 | 322.61 ms | 52.3% bf16 MFU | 1624391 tok/s
step 5165/19560 | loss 3.574523 (+0.16z)| norm 0.2734 (-0.30z)| lr 5.21e-04 | 322.90 ms | 52.3% bf16 MFU | 1624355 tok/s
step 5166/19560 | loss 3.528922 (-0.77z)| norm 0.2704 (-0.42z)| lr 5.21e-04 | 322.73 ms | 52.3% bf16 MFU | 1624364 tok/s
step 5167/19560 | loss 3.518542 (-0.97z)| norm 0.2663 (-0.58z)| lr 5.21e-04 | 322.68 ms | 52.3% bf16 MFU | 1624385 tok/s
step 5168/19560 | loss 3.562917 (-0.06z)| norm 0.2814 (+0.01z)| lr 5.21e-04 | 323.35 ms | 52.2% bf16 MFU | 1624236 tok/s
step 5169/19560 | loss 3.550551 (-0.31z)| norm 0.2674 (-0.55z)| lr 5.21e-04 | 322.63 ms | 52.3% bf16 MFU | 1624276 tok/s
step 5170/19560 | loss 3.513635 (-1.05z)| norm 0.2544 (-1.04z)| lr 5.21e-04 | 322.53 ms | 52.3% bf16 MFU | 1624339 tok/s
step 5171/19560 | loss 3.484232 (-1.62z)| norm 0.2786 (-0.08z)| lr 5.21e-04 | 323.28 ms | 52.2% bf16 MFU | 1624210 tok/s
step 5172/19560 | loss 3.553542 (-0.22z)| norm 0.3196 (+1.53z)| lr 5.21e-04 | 322.65 ms | 52.3% bf16 MFU | 1624247 tok/s
step 5173/19560 | loss 3.553214 (-0.23z)| norm 0.3161 (+1.37z)| lr 5.21e-04 | 322.58 ms | 52.3% bf16 MFU | 1624299 tok/s
step 5174/19560 | loss 3.520158 (-0.88z)| norm 0.2635 (-0.69z)| lr 5.21e-04 | 322.69 ms | 52.3% bf16 MFU | 1624322 tok/s
step 5175/19560 | loss 3.526005 (-0.75z)| norm 0.3037 (+0.87z)| lr 5.20e-04 | 322.87 ms | 52.3% bf16 MFU | 1624298 tok/s
step 5176/19560 | loss 3.525905 (-0.75z)| norm 0.2909 (+0.36z)| lr 5.20e-04 | 322.15 ms | 52.4% bf16 MFU | 1624456 tok/s
step 5177/19560 | loss 3.549495 (-0.27z)| norm 0.2848 (+0.13z)| lr 5.20e-04 | 322.83 ms | 52.3% bf16 MFU | 1624435 tok/s
step 5178/19560 | loss 3.495093 (-1.37z)| norm 0.2835 (+0.07z)| lr 5.20e-04 | 322.88 ms | 52.3% bf16 MFU | 1624402 tok/s
step 5179/19560 | loss 3.554199 (-0.16z)| norm 0.2820 (+0.01z)| lr 5.20e-04 | 322.52 ms | 52.3% bf16 MFU | 1624461 tok/s
step 5180/19560 | loss 3.531236 (-0.63z)| norm 0.2901 (+0.32z)| lr 5.20e-04 | 322.68 ms | 52.3% bf16 MFU | 1624478 tok/s
step 5181/19560 | loss 3.552514 (-0.17z)| norm 0.2922 (+0.40z)| lr 5.20e-04 | 322.44 ms | 52.3% bf16 MFU | 1624554 tok/s
step 5182/19560 | loss 3.537111 (-0.55z)| norm 0.2753 (-0.25z)| lr 5.20e-04 | 322.43 ms | 52.3% bf16 MFU | 1624629 tok/s
step 5183/19560 | loss 3.565777 (+0.17z)| norm 0.2756 (-0.24z)| lr 5.20e-04 | 323.40 ms | 52.2% bf16 MFU | 1624456 tok/s
step 5184/19560 | loss 3.549966 (-0.24z)| norm 0.2688 (-0.50z)| lr 5.20e-04 | 322.59 ms | 52.3% bf16 MFU | 1624494 tok/s
step 5185/19560 | loss 3.560349 (+0.02z)| norm 0.2601 (-0.85z)| lr 5.20e-04 | 322.60 ms | 52.3% bf16 MFU | 1624528 tok/s
step 5186/19560 | loss 3.538330 (-0.53z)| norm 0.2405 (-1.58z)| lr 5.20e-04 | 322.69 ms | 52.3% bf16 MFU | 1624538 tok/s
step 5187/19560 | loss 3.610138 (+1.30z)| norm 0.2700 (-0.43z)| lr 5.20e-04 | 322.73 ms | 52.3% bf16 MFU | 1624537 tok/s
step 5188/19560 | loss 3.538123 (-0.52z)| norm 0.2719 (-0.36z)| lr 5.20e-04 | 322.45 ms | 52.3% bf16 MFU | 1624608 tok/s
step 5189/19560 | loss 3.514983 (-1.10z)| norm 0.2510 (-1.16z)| lr 5.20e-04 | 322.80 ms | 52.3% bf16 MFU | 1624587 tok/s
step 5190/19560 | loss 3.543308 (-0.38z)| norm 0.2680 (-0.51z)| lr 5.20e-04 | 322.82 ms | 52.3% bf16 MFU | 1624562 tok/s
step 5191/19560 | loss 3.569236 (+0.28z)| norm 0.2702 (-0.42z)| lr 5.20e-04 | 322.72 ms | 52.3% bf16 MFU | 1624562 tok/s
step 5192/19560 | loss 3.589444 (+0.80z)| norm 0.2834 (+0.09z)| lr 5.20e-04 | 322.28 ms | 52.4% bf16 MFU | 1624676 tok/s
step 5193/19560 | loss 3.533970 (-0.65z)| norm 0.2925 (+0.44z)| lr 5.20e-04 | 322.57 ms | 52.3% bf16 MFU | 1624708 tok/s
step 5194/19560 | loss 3.578761 (+0.52z)| norm 0.2893 (+0.31z)| lr 5.20e-04 | 322.66 ms | 52.3% bf16 MFU | 1624717 tok/s
step 5195/19560 | loss 3.578291 (+0.51z)| norm 0.2647 (-0.64z)| lr 5.20e-04 | 323.04 ms | 52.2% bf16 MFU | 1624631 tok/s
step 5196/19560 | loss 3.547928 (-0.28z)| norm 0.2609 (-0.78z)| lr 5.20e-04 | 323.25 ms | 52.2% bf16 MFU | 1624495 tok/s
step 5197/19560 | loss 3.524135 (-0.90z)| norm 0.2937 (+0.49z)| lr 5.20e-04 | 322.47 ms | 52.3% bf16 MFU | 1624563 tok/s
step 5198/19560 | loss 3.516896 (-1.08z)| norm 0.2523 (-1.09z)| lr 5.20e-04 | 322.49 ms | 52.3% bf16 MFU | 1624623 tok/s
step 5199/19560 | loss 3.543178 (-0.38z)| norm 0.2591 (-0.83z)| lr 5.20e-04 | 322.88 ms | 52.3% bf16 MFU | 1624581 tok/s
step 5200/19560 | loss 3.533157 (-0.63z)| norm 0.2528 (-1.06z)| lr 5.20e-04 | 322.60 ms | 52.3% bf16 MFU | 1624613 tok/s
step 5201/19560 | loss 3.539741 (-0.45z)| norm 0.2980 (+0.66z)| lr 5.20e-04 | 322.22 ms | 52.4% bf16 MFU | 1624737 tok/s
step 5202/19560 | loss 3.557936 (+0.03z)| norm 0.2638 (-0.64z)| lr 5.20e-04 | 323.25 ms | 52.2% bf16 MFU | 1624597 tok/s
step 5203/19560 | loss 3.707136 (+3.74z)| norm 0.3202 (+1.50z)| lr 5.20e-04 | 322.68 ms | 52.3% bf16 MFU | 1624607 tok/s
step 5204/19560 | loss 3.524845 (-0.83z)| norm 0.4245 (+4.88z)| lr 5.19e-04 | 322.49 ms | 52.3% bf16 MFU | 1624663 tok/s
step 5205/19560 | loss 3.562383 (+0.10z)| norm 0.3121 (+1.03z)| lr 5.19e-04 | 322.58 ms | 52.3% bf16 MFU | 1624694 tok/s
step 5206/19560 | loss 3.551236 (-0.18z)| norm 0.2801 (-0.06z)| lr 5.19e-04 | 322.98 ms | 52.3% bf16 MFU | 1624624 tok/s
step 5207/19560 | loss 3.572062 (+0.34z)| norm 0.2777 (-0.15z)| lr 5.19e-04 | 322.15 ms | 52.4% bf16 MFU | 1624765 tok/s
step 5208/19560 | loss 3.464305 (-2.30z)| norm 0.2609 (-0.71z)| lr 5.19e-04 | 322.76 ms | 52.3% bf16 MFU | 1624746 tok/s
step 5209/19560 | loss 3.535965 (-0.53z)| norm 0.2721 (-0.33z)| lr 5.19e-04 | 322.52 ms | 52.3% bf16 MFU | 1624788 tok/s
step 5210/19560 | loss 3.509251 (-1.17z)| norm 0.2547 (-0.92z)| lr 5.19e-04 | 324.18 ms | 52.1% bf16 MFU | 1624413 tok/s
step 5211/19560 | loss 3.516793 (-0.97z)| norm 0.2736 (-0.28z)| lr 5.19e-04 | 322.44 ms | 52.3% bf16 MFU | 1624493 tok/s
step 5212/19560 | loss 3.569086 (+0.31z)| norm 0.2747 (-0.24z)| lr 5.19e-04 | 322.13 ms | 52.4% bf16 MFU | 1624648 tok/s
step 5213/19560 | loss 3.506691 (-1.21z)| norm 0.2777 (-0.15z)| lr 5.19e-04 | 322.59 ms | 52.3% bf16 MFU | 1624678 tok/s
step 5214/19560 | loss 3.493516 (-1.50z)| norm 0.2917 (+0.33z)| lr 5.19e-04 | 322.51 ms | 52.3% bf16 MFU | 1624725 tok/s
step 5215/19560 | loss 3.607897 (+1.25z)| norm 0.2578 (-0.82z)| lr 5.19e-04 | 322.50 ms | 52.3% bf16 MFU | 1624774 tok/s
step 5216/19560 | loss 3.559024 (+0.08z)| norm 0.2752 (-0.23z)| lr 5.19e-04 | 322.83 ms | 52.3% bf16 MFU | 1624737 tok/s
step 5217/19560 | loss 3.537531 (-0.43z)| norm 0.2636 (-0.62z)| lr 5.19e-04 | 322.32 ms | 52.4% bf16 MFU | 1624830 tok/s
step 5218/19560 | loss 3.507312 (-1.14z)| norm 0.2619 (-0.67z)| lr 5.19e-04 | 322.69 ms | 52.3% bf16 MFU | 1624826 tok/s
step 5219/19560 | loss 3.510864 (-1.04z)| norm 0.2704 (-0.38z)| lr 5.19e-04 | 322.77 ms | 52.3% bf16 MFU | 1624801 tok/s
step 5220/19560 | loss 3.637280 (+1.98z)| norm 0.2790 (-0.09z)| lr 5.19e-04 | 322.55 ms | 52.3% bf16 MFU | 1624832 tok/s
step 5221/19560 | loss 3.567577 (+0.31z)| norm 0.2862 (+0.16z)| lr 5.19e-04 | 322.75 ms | 52.3% bf16 MFU | 1624813 tok/s
step 5222/19560 | loss 3.527090 (-0.65z)| norm 0.2770 (-0.16z)| lr 5.19e-04 | 322.19 ms | 52.4% bf16 MFU | 1624936 tok/s
step 5223/19560 | loss 3.576400 (+0.52z)| norm 0.2946 (+0.44z)| lr 5.19e-04 | 323.26 ms | 52.2% bf16 MFU | 1624782 tok/s
step 5224/19560 | loss 3.544830 (-0.23z)| norm 0.2794 (-0.09z)| lr 5.19e-04 | 322.62 ms | 52.3% bf16 MFU | 1624797 tok/s
step 5225/19560 | loss 3.542746 (-0.28z)| norm 0.2694 (-0.43z)| lr 5.19e-04 | 322.56 ms | 52.3% bf16 MFU | 1624827 tok/s
step 5226/19560 | loss 3.516887 (-0.89z)| norm 0.2738 (-0.29z)| lr 5.19e-04 | 322.22 ms | 52.4% bf16 MFU | 1624941 tok/s
step 5227/19560 | loss 3.554927 (+0.06z)| norm 0.2988 (+0.64z)| lr 5.19e-04 | 322.53 ms | 52.3% bf16 MFU | 1624970 tok/s
step 5228/19560 | loss 3.577560 (+0.64z)| norm 0.2878 (+0.25z)| lr 5.19e-04 | 322.29 ms | 52.4% bf16 MFU | 1625060 tok/s
step 5229/19560 | loss 3.622991 (+1.75z)| norm 0.2838 (+0.10z)| lr 5.19e-04 | 323.10 ms | 52.2% bf16 MFU | 1624941 tok/s
step 5230/19560 | loss 3.489702 (-1.57z)| norm 0.2768 (-0.15z)| lr 5.19e-04 | 322.38 ms | 52.4% bf16 MFU | 1625009 tok/s
step 5231/19560 | loss 3.545860 (-0.15z)| norm 0.3087 (+1.03z)| lr 5.19e-04 | 322.59 ms | 52.3% bf16 MFU | 1625020 tok/s
step 5232/19560 | loss 3.517608 (-0.85z)| norm 0.2844 (+0.14z)| lr 5.19e-04 | 323.60 ms | 52.2% bf16 MFU | 1624777 tok/s
step 5233/19560 | loss 3.506004 (-1.13z)| norm 0.2781 (-0.09z)| lr 5.18e-04 | 322.38 ms | 52.4% bf16 MFU | 1624855 tok/s
step 5234/19560 | loss 3.496230 (-1.38z)| norm 0.2594 (-0.82z)| lr 5.18e-04 | 322.25 ms | 52.4% bf16 MFU | 1624959 tok/s
step 5235/19560 | loss 3.568862 (+0.49z)| norm 0.2653 (-0.58z)| lr 5.18e-04 | 323.20 ms | 52.2% bf16 MFU | 1624820 tok/s
step 5236/19560 | loss 3.542883 (-0.18z)| norm 0.2654 (-0.56z)| lr 5.18e-04 | 322.56 ms | 52.3% bf16 MFU | 1624848 tok/s
step 5237/19560 | loss 3.559570 (+0.25z)| norm 0.2758 (-0.09z)| lr 5.18e-04 | 322.82 ms | 52.3% bf16 MFU | 1624810 tok/s
step 5238/19560 | loss 3.550585 (+0.01z)| norm 0.2431 (-1.53z)| lr 5.18e-04 | 322.53 ms | 52.3% bf16 MFU | 1624847 tok/s
step 5239/19560 | loss 3.564058 (+0.37z)| norm 0.2532 (-1.06z)| lr 5.18e-04 | 323.07 ms | 52.2% bf16 MFU | 1624746 tok/s
step 5240/19560 | loss 3.590827 (+1.06z)| norm 0.2535 (-1.04z)| lr 5.18e-04 | 322.31 ms | 52.4% bf16 MFU | 1624842 tok/s
step 5241/19560 | loss 3.584331 (+0.87z)| norm 0.2823 (+0.25z)| lr 5.18e-04 | 322.51 ms | 52.3% bf16 MFU | 1624883 tok/s
step 5242/19560 | loss 3.603237 (+1.36z)| norm 0.2508 (-1.15z)| lr 5.18e-04 | 323.46 ms | 52.2% bf16 MFU | 1624684 tok/s
step 5243/19560 | loss 3.505053 (-1.20z)| norm 0.2610 (-0.69z)| lr 5.18e-04 | 322.43 ms | 52.3% bf16 MFU | 1624751 tok/s
step 5244/19560 | loss 3.631899 (+2.12z)| norm 0.2602 (-0.72z)| lr 5.18e-04 | 322.55 ms | 52.3% bf16 MFU | 1624785 tok/s
step 5245/19560 | loss 3.640508 (+2.36z)| norm 0.2757 (-0.02z)| lr 5.18e-04 | 323.09 ms | 52.2% bf16 MFU | 1624682 tok/s
step 5246/19560 | loss 3.575975 (+0.66z)| norm 0.2502 (-1.14z)| lr 5.18e-04 | 323.13 ms | 52.2% bf16 MFU | 1624573 tok/s
step 5247/19560 | loss 3.487313 (-1.64z)| norm 0.2703 (-0.23z)| lr 5.18e-04 | 322.95 ms | 52.3% bf16 MFU | 1624515 tok/s
step 5248/19560 | loss 3.541054 (-0.22z)| norm 0.2653 (-0.45z)| lr 5.18e-04 | 322.73 ms | 52.3% bf16 MFU | 1624517 tok/s
step 5249/19560 | loss 3.588393 (+1.00z)| norm 0.2653 (-0.45z)| lr 5.18e-04 | 322.71 ms | 52.3% bf16 MFU | 1624524 tok/s
step 5250/19560 | loss 3.493776 (-1.45z)| norm 0.2719 (-0.16z)| lr 5.18e-04 | 323.20 ms | 52.2% bf16 MFU | 1624408 tok/s
val loss 3.545979
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2754/10042 = 0.274248
step 5251/19560 | loss 3.517582 (-0.82z)| norm 0.2879 (+0.55z)| lr 5.18e-04 | 321.76 ms | 52.5% bf16 MFU | 1624660 tok/s
step 5252/19560 | loss 3.498264 (-1.30z)| norm 0.2846 (+0.40z)| lr 5.18e-04 | 322.67 ms | 52.3% bf16 MFU | 1624668 tok/s
step 5253/19560 | loss 3.599802 (+1.31z)| norm 0.2538 (-0.97z)| lr 5.18e-04 | 323.33 ms | 52.2% bf16 MFU | 1624511 tok/s
step 5254/19560 | loss 3.535271 (-0.35z)| norm 0.2912 (+0.69z)| lr 5.18e-04 | 322.81 ms | 52.3% bf16 MFU | 1624492 tok/s
step 5255/19560 | loss 3.552275 (+0.08z)| norm 0.2813 (+0.24z)| lr 5.18e-04 | 322.91 ms | 52.3% bf16 MFU | 1624450 tok/s
step 5256/19560 | loss 3.578015 (+0.74z)| norm 0.2806 (+0.20z)| lr 5.18e-04 | 322.58 ms | 52.3% bf16 MFU | 1624491 tok/s
step 5257/19560 | loss 3.580423 (+0.79z)| norm 0.2834 (+0.32z)| lr 5.18e-04 | 323.09 ms | 52.2% bf16 MFU | 1624403 tok/s
step 5258/19560 | loss 3.584064 (+0.87z)| norm 0.2797 (+0.15z)| lr 5.18e-04 | 323.01 ms | 52.2% bf16 MFU | 1624340 tok/s
step 5259/19560 | loss 3.567622 (+0.46z)| norm 0.2590 (-0.79z)| lr 5.18e-04 | 323.39 ms | 52.2% bf16 MFU | 1624184 tok/s
step 5260/19560 | loss 3.542408 (-0.19z)| norm 0.2690 (-0.33z)| lr 5.18e-04 | 322.86 ms | 52.3% bf16 MFU | 1624170 tok/s
step 5261/19560 | loss 3.571701 (+0.57z)| norm 0.2812 (+0.22z)| lr 5.18e-04 | 324.03 ms | 52.1% bf16 MFU | 1623861 tok/s
step 5262/19560 | loss 3.522394 (-0.71z)| norm 0.2729 (-0.16z)| lr 5.18e-04 | 322.88 ms | 52.3% bf16 MFU | 1623856 tok/s
step 5263/19560 | loss 3.677195 (+3.18z)| norm 0.2783 (+0.08z)| lr 5.17e-04 | 322.12 ms | 52.4% bf16 MFU | 1624044 tok/s
step 5264/19560 | loss 3.549039 (-0.03z)| norm 0.2902 (+0.61z)| lr 5.17e-04 | 323.11 ms | 52.2% bf16 MFU | 1623973 tok/s
step 5265/19560 | loss 3.536417 (-0.35z)| norm 0.2802 (+0.16z)| lr 5.17e-04 | 322.42 ms | 52.3% bf16 MFU | 1624081 tok/s
step 5266/19560 | loss 3.594908 (+1.11z)| norm 0.2827 (+0.28z)| lr 5.17e-04 | 322.93 ms | 52.3% bf16 MFU | 1624053 tok/s
step 5267/19560 | loss 3.516133 (-0.87z)| norm 0.2503 (-1.18z)| lr 5.17e-04 | 322.95 ms | 52.3% bf16 MFU | 1624022 tok/s
step 5268/19560 | loss 3.529371 (-0.54z)| norm 0.2582 (-0.82z)| lr 5.17e-04 | 322.92 ms | 52.3% bf16 MFU | 1624001 tok/s
step 5269/19560 | loss 3.553147 (+0.04z)| norm 0.2843 (+0.35z)| lr 5.17e-04 | 323.25 ms | 52.2% bf16 MFU | 1623897 tok/s
step 5270/19560 | loss 3.554713 (+0.09z)| norm 0.2785 (+0.07z)| lr 5.17e-04 | 322.57 ms | 52.3% bf16 MFU | 1623970 tok/s
step 5271/19560 | loss 3.517354 (-0.86z)| norm 0.2559 (-0.94z)| lr 5.17e-04 | 322.38 ms | 52.4% bf16 MFU | 1624088 tok/s
step 5272/19560 | loss 3.581236 (+0.79z)| norm 0.2618 (-0.67z)| lr 5.17e-04 | 322.74 ms | 52.3% bf16 MFU | 1624108 tok/s
step 5273/19560 | loss 3.531739 (-0.48z)| norm 0.2619 (-0.65z)| lr 5.17e-04 | 323.51 ms | 52.2% bf16 MFU | 1623934 tok/s
step 5274/19560 | loss 3.587431 (+0.98z)| norm 0.2464 (-1.36z)| lr 5.17e-04 | 322.89 ms | 52.3% bf16 MFU | 1623925 tok/s
step 5275/19560 | loss 3.559848 (+0.27z)| norm 0.2534 (-1.02z)| lr 5.17e-04 | 322.54 ms | 52.3% bf16 MFU | 1624004 tok/s
step 5276/19560 | loss 3.530156 (-0.51z)| norm 0.2654 (-0.46z)| lr 5.17e-04 | 323.47 ms | 52.2% bf16 MFU | 1623846 tok/s
step 5277/19560 | loss 3.539078 (-0.28z)| norm 0.2592 (-0.73z)| lr 5.17e-04 | 322.37 ms | 52.4% bf16 MFU | 1623972 tok/s
step 5278/19560 | loss 3.518686 (-0.81z)| norm 0.2675 (-0.33z)| lr 5.17e-04 | 323.42 ms | 52.2% bf16 MFU | 1623828 tok/s
step 5279/19560 | loss 3.499359 (-1.31z)| norm 0.2904 (+0.74z)| lr 5.17e-04 | 323.21 ms | 52.2% bf16 MFU | 1623744 tok/s
step 5280/19560 | loss 3.599795 (+1.32z)| norm 0.3025 (+1.29z)| lr 5.17e-04 | 322.68 ms | 52.3% bf16 MFU | 1623796 tok/s
step 5281/19560 | loss 3.497254 (-1.37z)| norm 0.3162 (+1.89z)| lr 5.17e-04 | 322.87 ms | 52.3% bf16 MFU | 1623797 tok/s
step 5282/19560 | loss 3.506152 (-1.12z)| norm 0.3042 (+1.32z)| lr 5.17e-04 | 322.50 ms | 52.3% bf16 MFU | 1623892 tok/s
step 5283/19560 | loss 3.568954 (+0.51z)| norm 0.2829 (+0.33z)| lr 5.17e-04 | 323.70 ms | 52.1% bf16 MFU | 1623681 tok/s
step 5284/19560 | loss 3.605422 (+1.44z)| norm 0.3209 (+2.05z)| lr 5.17e-04 | 323.07 ms | 52.2% bf16 MFU | 1623637 tok/s
step 5285/19560 | loss 3.540723 (-0.24z)| norm 0.3118 (+1.65z)| lr 5.17e-04 | 322.93 ms | 52.3% bf16 MFU | 1623633 tok/s
step 5286/19560 | loss 3.548955 (-0.03z)| norm 0.2846 (+0.40z)| lr 5.17e-04 | 322.90 ms | 52.3% bf16 MFU | 1623635 tok/s
step 5287/19560 | loss 3.511089 (-1.00z)| norm 0.3351 (+2.62z)| lr 5.17e-04 | 322.64 ms | 52.3% bf16 MFU | 1623704 tok/s
step 5288/19560 | loss 3.694086 (+3.55z)| norm 0.3125 (+1.58z)| lr 5.17e-04 | 323.96 ms | 52.1% bf16 MFU | 1623436 tok/s
step 5289/19560 | loss 3.571779 (+0.52z)| norm 0.3262 (+2.13z)| lr 5.17e-04 | 323.14 ms | 52.2% bf16 MFU | 1623388 tok/s
step 5290/19560 | loss 3.635437 (+2.05z)| norm 0.3445 (+2.84z)| lr 5.17e-04 | 322.46 ms | 52.3% bf16 MFU | 1623514 tok/s
step 5291/19560 | loss 3.590143 (+0.95z)| norm 0.3054 (+1.14z)| lr 5.17e-04 | 322.31 ms | 52.4% bf16 MFU | 1623670 tok/s
step 5292/19560 | loss 3.749637 (+4.43z)| norm 0.2755 (-0.14z)| lr 5.16e-04 | 322.88 ms | 52.3% bf16 MFU | 1623675 tok/s
step 5293/19560 | loss 3.571384 (+0.41z)| norm 0.2962 (+0.74z)| lr 5.16e-04 | 323.11 ms | 52.2% bf16 MFU | 1623624 tok/s
step 5294/19560 | loss 3.509710 (-0.97z)| norm 0.2630 (-0.68z)| lr 5.16e-04 | 322.90 ms | 52.3% bf16 MFU | 1623626 tok/s
step 5295/19560 | loss 3.556040 (+0.06z)| norm 0.2790 (-0.00z)| lr 5.16e-04 | 322.72 ms | 52.3% bf16 MFU | 1623674 tok/s
step 5296/19560 | loss 3.574795 (+0.48z)| norm 0.2631 (-0.67z)| lr 5.16e-04 | 322.52 ms | 52.3% bf16 MFU | 1623770 tok/s
step 5297/19560 | loss 3.562970 (+0.21z)| norm 0.2545 (-1.03z)| lr 5.16e-04 | 322.45 ms | 52.3% bf16 MFU | 1623880 tok/s
step 5298/19560 | loss 3.555919 (+0.05z)| norm 0.2651 (-0.59z)| lr 5.16e-04 | 322.52 ms | 52.3% bf16 MFU | 1623966 tok/s
step 5299/19560 | loss 3.529009 (-0.57z)| norm 0.2447 (-1.43z)| lr 5.16e-04 | 322.89 ms | 52.3% bf16 MFU | 1623953 tok/s
step 5300/19560 | loss 3.501764 (-1.18z)| norm 0.2678 (-0.45z)| lr 5.16e-04 | 322.89 ms | 52.3% bf16 MFU | 1623941 tok/s
step 5301/19560 | loss 3.515114 (-0.86z)| norm 0.2660 (-0.51z)| lr 5.16e-04 | 322.46 ms | 52.3% bf16 MFU | 1624040 tok/s
step 5302/19560 | loss 3.527977 (-0.58z)| norm 0.2483 (-1.27z)| lr 5.16e-04 | 322.06 ms | 52.4% bf16 MFU | 1624235 tok/s
step 5303/19560 | loss 3.557565 (+0.09z)| norm 0.2855 (+0.34z)| lr 5.16e-04 | 323.43 ms | 52.2% bf16 MFU | 1624074 tok/s
step 5304/19560 | loss 3.614842 (+1.36z)| norm 0.3040 (+1.13z)| lr 5.16e-04 | 322.29 ms | 52.4% bf16 MFU | 1624209 tok/s
step 5305/19560 | loss 3.557415 (+0.07z)| norm 0.3460 (+2.83z)| lr 5.16e-04 | 322.25 ms | 52.4% bf16 MFU | 1624347 tok/s
step 5306/19560 | loss 3.487227 (-1.51z)| norm 0.2899 (+0.49z)| lr 5.16e-04 | 322.10 ms | 52.4% bf16 MFU | 1624516 tok/s
step 5307/19560 | loss 3.631088 (+1.69z)| norm 0.2663 (-0.49z)| lr 5.16e-04 | 322.78 ms | 52.3% bf16 MFU | 1624504 tok/s
step 5308/19560 | loss 3.524769 (-0.67z)| norm 0.2808 (+0.12z)| lr 5.16e-04 | 322.87 ms | 52.3% bf16 MFU | 1624472 tok/s
step 5309/19560 | loss 3.556198 (+0.03z)| norm 0.2927 (+0.61z)| lr 5.16e-04 | 322.48 ms | 52.3% bf16 MFU | 1624539 tok/s
step 5310/19560 | loss 3.505163 (-1.10z)| norm 0.2813 (+0.14z)| lr 5.16e-04 | 322.63 ms | 52.3% bf16 MFU | 1624564 tok/s
step 5311/19560 | loss 3.633034 (+1.70z)| norm 0.2835 (+0.22z)| lr 5.16e-04 | 322.73 ms | 52.3% bf16 MFU | 1624564 tok/s
step 5312/19560 | loss 3.575373 (+0.44z)| norm 0.2718 (-0.26z)| lr 5.16e-04 | 322.48 ms | 52.3% bf16 MFU | 1624626 tok/s
step 5313/19560 | loss 3.564428 (+0.20z)| norm 0.2746 (-0.15z)| lr 5.16e-04 | 322.84 ms | 52.3% bf16 MFU | 1624594 tok/s
step 5314/19560 | loss 3.546864 (-0.19z)| norm 0.2865 (+0.33z)| lr 5.16e-04 | 322.71 ms | 52.3% bf16 MFU | 1624597 tok/s
step 5315/19560 | loss 3.520399 (-0.76z)| norm 0.2581 (-0.86z)| lr 5.16e-04 | 322.65 ms | 52.3% bf16 MFU | 1624615 tok/s
step 5316/19560 | loss 3.544314 (-0.23z)| norm 0.2545 (-1.00z)| lr 5.16e-04 | 323.03 ms | 52.2% bf16 MFU | 1624537 tok/s
step 5317/19560 | loss 3.539090 (-0.35z)| norm 0.2802 (+0.07z)| lr 5.16e-04 | 322.77 ms | 52.3% bf16 MFU | 1624526 tok/s
step 5318/19560 | loss 3.492039 (-1.37z)| norm 0.2636 (-0.63z)| lr 5.16e-04 | 322.72 ms | 52.3% bf16 MFU | 1624528 tok/s
step 5319/19560 | loss 3.595263 (+0.88z)| norm 0.2699 (-0.36z)| lr 5.16e-04 | 323.31 ms | 52.2% bf16 MFU | 1624384 tok/s
step 5320/19560 | loss 3.540812 (-0.30z)| norm 0.2886 (+0.42z)| lr 5.15e-04 | 322.85 ms | 52.3% bf16 MFU | 1624361 tok/s
step 5321/19560 | loss 3.494916 (-1.29z)| norm 0.2701 (-0.35z)| lr 5.15e-04 | 322.39 ms | 52.4% bf16 MFU | 1624456 tok/s
step 5322/19560 | loss 3.490009 (-1.37z)| norm 0.2942 (+0.66z)| lr 5.15e-04 | 322.83 ms | 52.3% bf16 MFU | 1624436 tok/s
step 5323/19560 | loss 3.578983 (+0.55z)| norm 0.2860 (+0.31z)| lr 5.15e-04 | 322.93 ms | 52.3% bf16 MFU | 1624392 tok/s
step 5324/19560 | loss 3.545892 (-0.16z)| norm 0.3041 (+1.05z)| lr 5.15e-04 | 322.59 ms | 52.3% bf16 MFU | 1624434 tok/s
step 5325/19560 | loss 3.565512 (+0.25z)| norm 0.3348 (+2.29z)| lr 5.15e-04 | 323.01 ms | 52.2% bf16 MFU | 1624369 tok/s
step 5326/19560 | loss 3.599744 (+0.98z)| norm 0.3249 (+1.84z)| lr 5.15e-04 | 322.82 ms | 52.3% bf16 MFU | 1624356 tok/s
step 5327/19560 | loss 3.507960 (-0.99z)| norm 0.2735 (-0.26z)| lr 5.15e-04 | 322.71 ms | 52.3% bf16 MFU | 1624370 tok/s
step 5328/19560 | loss 3.583802 (+0.63z)| norm 0.2692 (-0.45z)| lr 5.15e-04 | 322.56 ms | 52.3% bf16 MFU | 1624420 tok/s
step 5329/19560 | loss 3.610614 (+1.18z)| norm 0.2962 (+0.66z)| lr 5.15e-04 | 323.10 ms | 52.2% bf16 MFU | 1624333 tok/s
step 5330/19560 | loss 3.604243 (+1.04z)| norm 0.2790 (-0.05z)| lr 5.15e-04 | 322.54 ms | 52.3% bf16 MFU | 1624391 tok/s
step 5331/19560 | loss 3.558762 (+0.10z)| norm 0.2708 (-0.38z)| lr 5.15e-04 | 323.05 ms | 52.2% bf16 MFU | 1624319 tok/s
step 5332/19560 | loss 3.560011 (+0.12z)| norm 0.2711 (-0.37z)| lr 5.15e-04 | 322.63 ms | 52.3% bf16 MFU | 1624356 tok/s
step 5333/19560 | loss 3.550981 (-0.08z)| norm 0.2681 (-0.50z)| lr 5.15e-04 | 322.37 ms | 52.4% bf16 MFU | 1624455 tok/s
step 5334/19560 | loss 3.452497 (-2.21z)| norm 0.2674 (-0.54z)| lr 5.15e-04 | 323.45 ms | 52.2% bf16 MFU | 1624278 tok/s
step 5335/19560 | loss 3.623730 (+1.50z)| norm 0.2504 (-1.36z)| lr 5.15e-04 | 322.19 ms | 52.4% bf16 MFU | 1624428 tok/s
step 5336/19560 | loss 3.588418 (+0.73z)| norm 0.2859 (+0.39z)| lr 5.15e-04 | 322.66 ms | 52.3% bf16 MFU | 1624450 tok/s
step 5337/19560 | loss 3.569559 (+0.31z)| norm 0.2683 (-0.49z)| lr 5.15e-04 | 322.18 ms | 52.4% bf16 MFU | 1624593 tok/s
step 5338/19560 | loss 3.601763 (+1.00z)| norm 0.2971 (+0.92z)| lr 5.15e-04 | 323.25 ms | 52.2% bf16 MFU | 1624460 tok/s
step 5339/19560 | loss 3.527221 (-0.64z)| norm 0.2794 (+0.05z)| lr 5.15e-04 | 322.85 ms | 52.3% bf16 MFU | 1624435 tok/s
step 5340/19560 | loss 3.595975 (+0.87z)| norm 0.2727 (-0.29z)| lr 5.15e-04 | 322.84 ms | 52.3% bf16 MFU | 1624414 tok/s
step 5341/19560 | loss 3.562923 (+0.13z)| norm 0.2955 (+0.83z)| lr 5.15e-04 | 322.72 ms | 52.3% bf16 MFU | 1624422 tok/s
step 5342/19560 | loss 3.557790 (+0.01z)| norm 0.2865 (+0.39z)| lr 5.15e-04 | 322.77 ms | 52.3% bf16 MFU | 1624418 tok/s
step 5343/19560 | loss 3.496271 (-1.33z)| norm 0.2553 (-1.15z)| lr 5.15e-04 | 322.27 ms | 52.4% bf16 MFU | 1624540 tok/s
step 5344/19560 | loss 3.553119 (-0.07z)| norm 0.2562 (-1.09z)| lr 5.15e-04 | 322.92 ms | 52.3% bf16 MFU | 1624493 tok/s
step 5345/19560 | loss 3.581805 (+0.55z)| norm 0.2973 (+0.91z)| lr 5.15e-04 | 323.14 ms | 52.2% bf16 MFU | 1624393 tok/s
step 5346/19560 | loss 3.550881 (-0.14z)| norm 0.2538 (-1.21z)| lr 5.15e-04 | 322.76 ms | 52.3% bf16 MFU | 1624393 tok/s
step 5347/19560 | loss 3.608925 (+1.13z)| norm 0.2450 (-1.62z)| lr 5.15e-04 | 322.68 ms | 52.3% bf16 MFU | 1624412 tok/s
step 5348/19560 | loss 3.519993 (-0.83z)| norm 0.2727 (-0.27z)| lr 5.15e-04 | 323.09 ms | 52.2% bf16 MFU | 1624327 tok/s
step 5349/19560 | loss 3.572505 (+0.35z)| norm 0.2537 (-1.18z)| lr 5.14e-04 | 322.22 ms | 52.4% bf16 MFU | 1624466 tok/s
step 5350/19560 | loss 3.554004 (-0.07z)| norm 0.2426 (-1.68z)| lr 5.14e-04 | 322.51 ms | 52.3% bf16 MFU | 1624525 tok/s
step 5351/19560 | loss 3.562165 (+0.11z)| norm 0.2796 (+0.09z)| lr 5.14e-04 | 322.77 ms | 52.3% bf16 MFU | 1624516 tok/s
step 5352/19560 | loss 3.557684 (+0.01z)| norm 0.2517 (-1.22z)| lr 5.14e-04 | 322.53 ms | 52.3% bf16 MFU | 1624569 tok/s
step 5353/19560 | loss 3.563207 (+0.13z)| norm 0.2479 (-1.39z)| lr 5.14e-04 | 322.40 ms | 52.3% bf16 MFU | 1624651 tok/s
step 5354/19560 | loss 3.543724 (-0.31z)| norm 0.2622 (-0.71z)| lr 5.14e-04 | 323.36 ms | 52.2% bf16 MFU | 1624487 tok/s
step 5355/19560 | loss 3.513421 (-0.98z)| norm 0.2644 (-0.59z)| lr 5.14e-04 | 322.98 ms | 52.3% bf16 MFU | 1624427 tok/s
step 5356/19560 | loss 3.535829 (-0.47z)| norm 0.2768 (-0.00z)| lr 5.14e-04 | 322.66 ms | 52.3% bf16 MFU | 1624451 tok/s
step 5357/19560 | loss 3.594517 (+0.85z)| norm 0.2661 (-0.50z)| lr 5.14e-04 | 322.71 ms | 52.3% bf16 MFU | 1624461 tok/s
step 5358/19560 | loss 3.568552 (+0.26z)| norm 0.2690 (-0.36z)| lr 5.14e-04 | 322.29 ms | 52.4% bf16 MFU | 1624575 tok/s
step 5359/19560 | loss 3.505526 (-1.17z)| norm 0.2825 (+0.29z)| lr 5.14e-04 | 323.02 ms | 52.2% bf16 MFU | 1624501 tok/s
step 5360/19560 | loss 3.562422 (+0.11z)| norm 0.2652 (-0.53z)| lr 5.14e-04 | 322.40 ms | 52.3% bf16 MFU | 1624586 tok/s
step 5361/19560 | loss 3.589415 (+0.72z)| norm 0.2641 (-0.58z)| lr 5.14e-04 | 322.80 ms | 52.3% bf16 MFU | 1624565 tok/s
step 5362/19560 | loss 3.537037 (-0.49z)| norm 0.2761 (-0.01z)| lr 5.14e-04 | 322.69 ms | 52.3% bf16 MFU | 1624574 tok/s
step 5363/19560 | loss 3.733758 (+3.77z)| norm 0.2835 (+0.33z)| lr 5.14e-04 | 322.98 ms | 52.3% bf16 MFU | 1624509 tok/s
step 5364/19560 | loss 3.634745 (+1.59z)| norm 0.3058 (+1.37z)| lr 5.14e-04 | 322.77 ms | 52.3% bf16 MFU | 1624500 tok/s
step 5365/19560 | loss 3.554918 (-0.12z)| norm 0.3011 (+1.13z)| lr 5.14e-04 | 322.70 ms | 52.3% bf16 MFU | 1624510 tok/s
step 5366/19560 | loss 3.557214 (-0.07z)| norm 0.3186 (+1.92z)| lr 5.14e-04 | 322.40 ms | 52.3% bf16 MFU | 1624593 tok/s
step 5367/19560 | loss 3.515231 (-0.96z)| norm 0.3083 (+1.41z)| lr 5.14e-04 | 322.76 ms | 52.3% bf16 MFU | 1624582 tok/s
step 5368/19560 | loss 3.572930 (+0.28z)| norm 0.3041 (+1.20z)| lr 5.14e-04 | 322.75 ms | 52.3% bf16 MFU | 1624576 tok/s
step 5369/19560 | loss 3.535794 (-0.51z)| norm 0.2813 (+0.13z)| lr 5.14e-04 | 322.97 ms | 52.3% bf16 MFU | 1624513 tok/s
step 5370/19560 | loss 3.547271 (-0.25z)| norm 0.2828 (+0.19z)| lr 5.14e-04 | 323.01 ms | 52.2% bf16 MFU | 1624444 tok/s
step 5371/19560 | loss 3.553489 (-0.13z)| norm 0.2873 (+0.40z)| lr 5.14e-04 | 322.57 ms | 52.3% bf16 MFU | 1624488 tok/s
step 5372/19560 | loss 3.614343 (+1.19z)| norm 0.2755 (-0.16z)| lr 5.14e-04 | 322.83 ms | 52.3% bf16 MFU | 1624466 tok/s
step 5373/19560 | loss 3.601526 (+0.93z)| norm 0.2710 (-0.38z)| lr 5.14e-04 | 322.43 ms | 52.3% bf16 MFU | 1624545 tok/s
step 5374/19560 | loss 3.526575 (-0.70z)| norm 0.2906 (+0.54z)| lr 5.14e-04 | 322.41 ms | 52.3% bf16 MFU | 1624626 tok/s
step 5375/19560 | loss 3.480446 (-1.70z)| norm 0.2630 (-0.77z)| lr 5.14e-04 | 323.00 ms | 52.3% bf16 MFU | 1624554 tok/s
step 5376/19560 | loss 3.522349 (-0.78z)| norm 0.3115 (+1.50z)| lr 5.14e-04 | 322.29 ms | 52.4% bf16 MFU | 1624663 tok/s
step 5377/19560 | loss 3.572608 (+0.31z)| norm 0.2982 (+0.86z)| lr 5.14e-04 | 322.89 ms | 52.3% bf16 MFU | 1624617 tok/s
step 5378/19560 | loss 3.611979 (+1.15z)| norm 0.2845 (+0.22z)| lr 5.13e-04 | 323.42 ms | 52.2% bf16 MFU | 1624439 tok/s
step 5379/19560 | loss 3.510165 (-1.07z)| norm 0.2918 (+0.56z)| lr 5.13e-04 | 322.17 ms | 52.4% bf16 MFU | 1624585 tok/s
step 5380/19560 | loss 3.575166 (+0.34z)| norm 0.2829 (+0.14z)| lr 5.13e-04 | 322.50 ms | 52.3% bf16 MFU | 1624640 tok/s
step 5381/19560 | loss 3.534879 (-0.53z)| norm 0.2615 (-0.87z)| lr 5.13e-04 | 323.30 ms | 52.2% bf16 MFU | 1624491 tok/s
step 5382/19560 | loss 3.553035 (-0.14z)| norm 0.3047 (+1.15z)| lr 5.13e-04 | 322.73 ms | 52.3% bf16 MFU | 1624492 tok/s
step 5383/19560 | loss 3.539091 (-0.44z)| norm 0.2797 (-0.02z)| lr 5.13e-04 | 322.58 ms | 52.3% bf16 MFU | 1624532 tok/s
step 5384/19560 | loss 3.563011 (+0.09z)| norm 0.2697 (-0.48z)| lr 5.13e-04 | 322.70 ms | 52.3% bf16 MFU | 1624539 tok/s
step 5385/19560 | loss 3.517781 (-0.90z)| norm 0.2732 (-0.32z)| lr 5.13e-04 | 322.27 ms | 52.4% bf16 MFU | 1624656 tok/s
step 5386/19560 | loss 3.551480 (-0.15z)| norm 0.2870 (+0.33z)| lr 5.13e-04 | 322.58 ms | 52.3% bf16 MFU | 1624689 tok/s
step 5387/19560 | loss 3.520817 (-0.82z)| norm 0.2836 (+0.16z)| lr 5.13e-04 | 322.74 ms | 52.3% bf16 MFU | 1624679 tok/s
step 5388/19560 | loss 3.534656 (-0.51z)| norm 0.3048 (+1.14z)| lr 5.13e-04 | 323.02 ms | 52.2% bf16 MFU | 1624599 tok/s
step 5389/19560 | loss 3.564748 (+0.15z)| norm 0.2820 (+0.07z)| lr 5.13e-04 | 322.81 ms | 52.3% bf16 MFU | 1624575 tok/s
step 5390/19560 | loss 3.597627 (+0.86z)| norm 0.2560 (-1.14z)| lr 5.13e-04 | 322.67 ms | 52.3% bf16 MFU | 1624590 tok/s
step 5391/19560 | loss 3.558505 (+0.02z)| norm 0.2898 (+0.44z)| lr 5.13e-04 | 322.29 ms | 52.4% bf16 MFU | 1624698 tok/s
step 5392/19560 | loss 3.539473 (-0.41z)| norm 0.2933 (+0.60z)| lr 5.13e-04 | 322.22 ms | 52.4% bf16 MFU | 1624818 tok/s
step 5393/19560 | loss 3.571787 (+0.32z)| norm 0.2573 (-1.07z)| lr 5.13e-04 | 323.03 ms | 52.2% bf16 MFU | 1624729 tok/s
step 5394/19560 | loss 3.532529 (-0.56z)| norm 0.2888 (+0.39z)| lr 5.13e-04 | 322.51 ms | 52.3% bf16 MFU | 1624774 tok/s
step 5395/19560 | loss 3.540651 (-0.38z)| norm 0.2764 (-0.19z)| lr 5.13e-04 | 323.45 ms | 52.2% bf16 MFU | 1624582 tok/s
step 5396/19560 | loss 3.512108 (-1.02z)| norm 0.2616 (-0.89z)| lr 5.13e-04 | 322.23 ms | 52.4% bf16 MFU | 1624706 tok/s
step 5397/19560 | loss 3.649284 (+2.02z)| norm 0.2996 (+0.88z)| lr 5.13e-04 | 323.13 ms | 52.2% bf16 MFU | 1624597 tok/s
step 5398/19560 | loss 3.578979 (+0.46z)| norm 0.2798 (-0.04z)| lr 5.13e-04 | 322.90 ms | 52.3% bf16 MFU | 1624552 tok/s
step 5399/19560 | loss 3.498211 (-1.32z)| norm 0.2754 (-0.25z)| lr 5.13e-04 | 322.62 ms | 52.3% bf16 MFU | 1624580 tok/s
step 5400/19560 | loss 3.500764 (-1.25z)| norm 0.2557 (-1.18z)| lr 5.13e-04 | 323.00 ms | 52.3% bf16 MFU | 1624509 tok/s
step 5401/19560 | loss 3.478700 (-1.70z)| norm 0.2512 (-1.37z)| lr 5.13e-04 | 322.60 ms | 52.3% bf16 MFU | 1624544 tok/s
step 5402/19560 | loss 3.472888 (-1.79z)| norm 0.2610 (-0.93z)| lr 5.13e-04 | 322.55 ms | 52.3% bf16 MFU | 1624590 tok/s
step 5403/19560 | loss 3.530012 (-0.56z)| norm 0.2596 (-1.00z)| lr 5.13e-04 | 322.93 ms | 52.3% bf16 MFU | 1624538 tok/s
step 5404/19560 | loss 3.597949 (+0.89z)| norm 0.2815 (+0.02z)| lr 5.13e-04 | 322.42 ms | 52.3% bf16 MFU | 1624616 tok/s
step 5405/19560 | loss 3.514081 (-0.90z)| norm 0.2670 (-0.66z)| lr 5.13e-04 | 322.85 ms | 52.3% bf16 MFU | 1624582 tok/s
step 5406/19560 | loss 3.540176 (-0.35z)| norm 0.2672 (-0.66z)| lr 5.12e-04 | 322.90 ms | 52.3% bf16 MFU | 1624536 tok/s
step 5407/19560 | loss 3.602530 (+0.97z)| norm 0.2867 (+0.27z)| lr 5.12e-04 | 322.74 ms | 52.3% bf16 MFU | 1624534 tok/s
step 5408/19560 | loss 3.521799 (-0.75z)| norm 0.2749 (-0.28z)| lr 5.12e-04 | 322.29 ms | 52.4% bf16 MFU | 1624645 tok/s
step 5409/19560 | loss 3.503317 (-1.15z)| norm 0.2625 (-0.85z)| lr 5.12e-04 | 323.36 ms | 52.2% bf16 MFU | 1624480 tok/s
step 5410/19560 | loss 3.498151 (-1.26z)| norm 0.2623 (-0.85z)| lr 5.12e-04 | 322.62 ms | 52.3% bf16 MFU | 1624511 tok/s
step 5411/19560 | loss 3.610260 (+1.14z)| norm 0.2605 (-0.92z)| lr 5.12e-04 | 322.96 ms | 52.3% bf16 MFU | 1624454 tok/s
step 5412/19560 | loss 3.525589 (-0.66z)| norm 0.2522 (-1.31z)| lr 5.12e-04 | 322.60 ms | 52.3% bf16 MFU | 1624491 tok/s
step 5413/19560 | loss 3.516598 (-0.85z)| norm 0.2954 (+0.78z)| lr 5.12e-04 | 322.14 ms | 52.4% bf16 MFU | 1624642 tok/s
step 5414/19560 | loss 3.551395 (-0.10z)| norm 0.3158 (+1.74z)| lr 5.12e-04 | 323.52 ms | 52.2% bf16 MFU | 1624439 tok/s
step 5415/19560 | loss 3.564439 (+0.17z)| norm 0.2663 (-0.62z)| lr 5.12e-04 | 322.68 ms | 52.3% bf16 MFU | 1624456 tok/s
step 5416/19560 | loss 3.551075 (-0.10z)| norm 0.2627 (-0.79z)| lr 5.12e-04 | 322.63 ms | 52.3% bf16 MFU | 1624486 tok/s
step 5417/19560 | loss 3.562599 (+0.16z)| norm 0.3069 (+1.44z)| lr 5.12e-04 | 322.52 ms | 52.3% bf16 MFU | 1624542 tok/s
step 5418/19560 | loss 3.622739 (+1.51z)| norm 0.2798 (+0.10z)| lr 5.12e-04 | 322.11 ms | 52.4% bf16 MFU | 1624698 tok/s
step 5419/19560 | loss 3.545404 (-0.21z)| norm 0.2518 (-1.35z)| lr 5.12e-04 | 322.56 ms | 52.3% bf16 MFU | 1624734 tok/s
step 5420/19560 | loss 3.599513 (+1.10z)| norm 0.2517 (-1.34z)| lr 5.12e-04 | 322.49 ms | 52.3% bf16 MFU | 1624785 tok/s
step 5421/19560 | loss 3.512062 (-0.99z)| norm 0.2653 (-0.62z)| lr 5.12e-04 | 322.65 ms | 52.3% bf16 MFU | 1624793 tok/s
step 5422/19560 | loss 3.567056 (+0.32z)| norm 0.2547 (-1.17z)| lr 5.12e-04 | 322.99 ms | 52.3% bf16 MFU | 1624715 tok/s
step 5423/19560 | loss 3.555243 (+0.04z)| norm 0.2786 (+0.09z)| lr 5.12e-04 | 322.56 ms | 52.3% bf16 MFU | 1624749 tok/s
step 5424/19560 | loss 3.548721 (-0.12z)| norm 0.2780 (+0.05z)| lr 5.12e-04 | 322.14 ms | 52.4% bf16 MFU | 1624887 tok/s
step 5425/19560 | loss 3.551072 (-0.06z)| norm 0.2937 (+0.86z)| lr 5.12e-04 | 322.76 ms | 52.3% bf16 MFU | 1624860 tok/s
step 5426/19560 | loss 3.504177 (-1.18z)| norm 0.2741 (-0.18z)| lr 5.12e-04 | 322.66 ms | 52.3% bf16 MFU | 1624862 tok/s
step 5427/19560 | loss 3.547531 (-0.14z)| norm 0.3202 (+2.21z)| lr 5.12e-04 | 322.83 ms | 52.3% bf16 MFU | 1624821 tok/s
step 5428/19560 | loss 3.582510 (+0.69z)| norm 0.3165 (+1.96z)| lr 5.12e-04 | 322.71 ms | 52.3% bf16 MFU | 1624812 tok/s
step 5429/19560 | loss 3.523992 (-0.72z)| norm 0.2926 (+0.72z)| lr 5.12e-04 | 322.35 ms | 52.4% bf16 MFU | 1624895 tok/s
step 5430/19560 | loss 3.544384 (-0.23z)| norm 0.2898 (+0.56z)| lr 5.12e-04 | 323.43 ms | 52.2% bf16 MFU | 1624701 tok/s
step 5431/19560 | loss 3.539718 (-0.34z)| norm 0.3218 (+2.17z)| lr 5.12e-04 | 322.73 ms | 52.3% bf16 MFU | 1624694 tok/s
step 5432/19560 | loss 3.527532 (-0.63z)| norm 0.2948 (+0.80z)| lr 5.12e-04 | 322.70 ms | 52.3% bf16 MFU | 1624694 tok/s
step 5433/19560 | loss 3.561379 (+0.20z)| norm 0.3062 (+1.46z)| lr 5.12e-04 | 322.49 ms | 52.3% bf16 MFU | 1624748 tok/s
step 5434/19560 | loss 3.682755 (+3.04z)| norm 0.3063 (+1.45z)| lr 5.11e-04 | 322.82 ms | 52.3% bf16 MFU | 1624714 tok/s
step 5435/19560 | loss 3.525420 (-0.68z)| norm 0.2957 (+0.87z)| lr 5.11e-04 | 323.25 ms | 52.2% bf16 MFU | 1624576 tok/s
step 5436/19560 | loss 3.513022 (-0.98z)| norm 0.2910 (+0.62z)| lr 5.11e-04 | 322.71 ms | 52.3% bf16 MFU | 1624581 tok/s
step 5437/19560 | loss 3.625833 (+1.69z)| norm 0.3069 (+1.44z)| lr 5.11e-04 | 323.12 ms | 52.2% bf16 MFU | 1624481 tok/s
step 5438/19560 | loss 3.572232 (+0.41z)| norm 0.2917 (+0.64z)| lr 5.11e-04 | 323.57 ms | 52.2% bf16 MFU | 1624273 tok/s
step 5439/19560 | loss 3.583199 (+0.69z)| norm 0.2578 (-1.12z)| lr 5.11e-04 | 323.23 ms | 52.2% bf16 MFU | 1624161 tok/s
step 5440/19560 | loss 3.575111 (+0.50z)| norm 0.2982 (+0.97z)| lr 5.11e-04 | 322.67 ms | 52.3% bf16 MFU | 1624196 tok/s
step 5441/19560 | loss 3.506401 (-1.14z)| norm 0.2867 (+0.37z)| lr 5.11e-04 | 322.05 ms | 52.4% bf16 MFU | 1624384 tok/s
step 5442/19560 | loss 3.523521 (-0.73z)| norm 0.2742 (-0.27z)| lr 5.11e-04 | 322.84 ms | 52.3% bf16 MFU | 1624365 tok/s
step 5443/19560 | loss 3.528146 (-0.62z)| norm 0.2459 (-1.73z)| lr 5.11e-04 | 322.92 ms | 52.3% bf16 MFU | 1624326 tok/s
step 5444/19560 | loss 3.509380 (-1.06z)| norm 0.2669 (-0.66z)| lr 5.11e-04 | 322.70 ms | 52.3% bf16 MFU | 1624344 tok/s
step 5445/19560 | loss 3.506470 (-1.12z)| norm 0.2630 (-0.85z)| lr 5.11e-04 | 322.60 ms | 52.3% bf16 MFU | 1624387 tok/s
step 5446/19560 | loss 3.594720 (+0.97z)| norm 0.2741 (-0.28z)| lr 5.11e-04 | 322.79 ms | 52.3% bf16 MFU | 1624378 tok/s
step 5447/19560 | loss 3.529777 (-0.57z)| norm 0.2442 (-1.80z)| lr 5.11e-04 | 323.07 ms | 52.2% bf16 MFU | 1624300 tok/s
step 5448/19560 | loss 3.559403 (+0.13z)| norm 0.2809 (+0.09z)| lr 5.11e-04 | 322.88 ms | 52.3% bf16 MFU | 1624274 tok/s
step 5449/19560 | loss 3.526876 (-0.66z)| norm 0.2922 (+0.66z)| lr 5.11e-04 | 322.91 ms | 52.3% bf16 MFU | 1624242 tok/s
step 5450/19560 | loss 3.491418 (-1.51z)| norm 0.2625 (-0.85z)| lr 5.11e-04 | 322.39 ms | 52.3% bf16 MFU | 1624342 tok/s
step 5451/19560 | loss 3.573553 (+0.47z)| norm 0.2656 (-0.68z)| lr 5.11e-04 | 322.91 ms | 52.3% bf16 MFU | 1624306 tok/s
step 5452/19560 | loss 3.528135 (-0.62z)| norm 0.2683 (-0.54z)| lr 5.11e-04 | 323.06 ms | 52.2% bf16 MFU | 1624236 tok/s
step 5453/19560 | loss 3.628042 (+1.75z)| norm 0.2618 (-0.86z)| lr 5.11e-04 | 324.21 ms | 52.1% bf16 MFU | 1623880 tok/s
step 5454/19560 | loss 3.546920 (-0.17z)| norm 0.2642 (-0.73z)| lr 5.11e-04 | 322.70 ms | 52.3% bf16 MFU | 1623922 tok/s
step 5455/19560 | loss 3.601409 (+1.12z)| norm 0.2633 (-0.77z)| lr 5.11e-04 | 322.59 ms | 52.3% bf16 MFU | 1623989 tok/s
step 5456/19560 | loss 3.532445 (-0.52z)| norm 0.2627 (-0.80z)| lr 5.11e-04 | 322.82 ms | 52.3% bf16 MFU | 1623995 tok/s
step 5457/19560 | loss 3.639817 (+2.02z)| norm 0.2717 (-0.30z)| lr 5.11e-04 | 322.53 ms | 52.3% bf16 MFU | 1624072 tok/s
step 5458/19560 | loss 3.543013 (-0.26z)| norm 0.2825 (+0.28z)| lr 5.11e-04 | 323.11 ms | 52.2% bf16 MFU | 1623999 tok/s
step 5459/19560 | loss 3.547168 (-0.16z)| norm 0.2518 (-1.37z)| lr 5.11e-04 | 322.67 ms | 52.3% bf16 MFU | 1624041 tok/s
step 5460/19560 | loss 3.556522 (+0.06z)| norm 0.2951 (+0.95z)| lr 5.11e-04 | 322.95 ms | 52.3% bf16 MFU | 1624010 tok/s
step 5461/19560 | loss 3.544346 (-0.23z)| norm 0.2737 (-0.20z)| lr 5.11e-04 | 322.96 ms | 52.3% bf16 MFU | 1623978 tok/s
step 5462/19560 | loss 3.509470 (-1.09z)| norm 0.2541 (-1.24z)| lr 5.11e-04 | 323.56 ms | 52.2% bf16 MFU | 1623798 tok/s
step 5463/19560 | loss 3.487976 (-1.59z)| norm 0.2792 (+0.09z)| lr 5.10e-04 | 322.70 ms | 52.3% bf16 MFU | 1623842 tok/s
step 5464/19560 | loss 3.526941 (-0.63z)| norm 0.2673 (-0.54z)| lr 5.10e-04 | 322.19 ms | 52.4% bf16 MFU | 1624012 tok/s
step 5465/19560 | loss 3.538555 (-0.34z)| norm 0.2700 (-0.40z)| lr 5.10e-04 | 322.89 ms | 52.3% bf16 MFU | 1623999 tok/s
step 5466/19560 | loss 3.678081 (+2.95z)| norm 0.2696 (-0.41z)| lr 5.10e-04 | 322.90 ms | 52.3% bf16 MFU | 1623984 tok/s
step 5467/19560 | loss 3.528852 (-0.57z)| norm 0.2782 (+0.05z)| lr 5.10e-04 | 322.85 ms | 52.3% bf16 MFU | 1623982 tok/s
step 5468/19560 | loss 3.637142 (+1.95z)| norm 0.2817 (+0.24z)| lr 5.10e-04 | 322.67 ms | 52.3% bf16 MFU | 1624024 tok/s
step 5469/19560 | loss 3.566304 (+0.30z)| norm 0.2822 (+0.27z)| lr 5.10e-04 | 322.87 ms | 52.3% bf16 MFU | 1624014 tok/s
step 5470/19560 | loss 3.460992 (-2.11z)| norm 0.2576 (-1.05z)| lr 5.10e-04 | 322.77 ms | 52.3% bf16 MFU | 1624031 tok/s
step 5471/19560 | loss 3.495056 (-1.32z)| norm 0.2989 (+1.17z)| lr 5.10e-04 | 322.81 ms | 52.3% bf16 MFU | 1624035 tok/s
step 5472/19560 | loss 3.550556 (-0.05z)| norm 0.3008 (+1.26z)| lr 5.10e-04 | 322.55 ms | 52.3% bf16 MFU | 1624106 tok/s
step 5473/19560 | loss 3.506293 (-1.05z)| norm 0.3151 (+2.00z)| lr 5.10e-04 | 322.71 ms | 52.3% bf16 MFU | 1624132 tok/s
step 5474/19560 | loss 3.610292 (+1.31z)| norm 0.2640 (-0.74z)| lr 5.10e-04 | 322.91 ms | 52.3% bf16 MFU | 1624106 tok/s
step 5475/19560 | loss 3.483529 (-1.54z)| norm 0.3172 (+2.07z)| lr 5.10e-04 | 322.35 ms | 52.4% bf16 MFU | 1624222 tok/s
step 5476/19560 | loss 3.564998 (+0.30z)| norm 0.2915 (+0.69z)| lr 5.10e-04 | 322.79 ms | 52.3% bf16 MFU | 1624222 tok/s
step 5477/19560 | loss 3.588181 (+0.82z)| norm 0.2831 (+0.24z)| lr 5.10e-04 | 322.97 ms | 52.3% bf16 MFU | 1624179 tok/s
step 5478/19560 | loss 3.561527 (+0.21z)| norm 0.3087 (+1.59z)| lr 5.10e-04 | 322.80 ms | 52.3% bf16 MFU | 1624178 tok/s
step 5479/19560 | loss 3.543291 (-0.20z)| norm 0.2935 (+0.76z)| lr 5.10e-04 | 323.16 ms | 52.2% bf16 MFU | 1624089 tok/s
step 5480/19560 | loss 3.499234 (-1.18z)| norm 0.3015 (+1.18z)| lr 5.10e-04 | 322.80 ms | 52.3% bf16 MFU | 1624094 tok/s
step 5481/19560 | loss 3.522523 (-0.65z)| norm 0.3026 (+1.22z)| lr 5.10e-04 | 322.62 ms | 52.3% bf16 MFU | 1624144 tok/s
step 5482/19560 | loss 3.604449 (+1.18z)| norm 0.3410 (+3.15z)| lr 5.10e-04 | 323.36 ms | 52.2% bf16 MFU | 1624005 tok/s
step 5483/19560 | loss 3.504721 (-1.05z)| norm 0.2927 (+0.61z)| lr 5.10e-04 | 322.92 ms | 52.3% bf16 MFU | 1623983 tok/s
step 5484/19560 | loss 3.550860 (-0.02z)| norm 0.2657 (-0.80z)| lr 5.10e-04 | 322.82 ms | 52.3% bf16 MFU | 1623988 tok/s
step 5485/19560 | loss 3.522622 (-0.64z)| norm 0.2677 (-0.70z)| lr 5.10e-04 | 323.00 ms | 52.3% bf16 MFU | 1623949 tok/s
step 5486/19560 | loss 3.529866 (-0.47z)| norm 0.2724 (-0.45z)| lr 5.10e-04 | 322.76 ms | 52.3% bf16 MFU | 1623972 tok/s
step 5487/19560 | loss 3.471596 (-1.76z)| norm 0.2720 (-0.47z)| lr 5.10e-04 | 322.69 ms | 52.3% bf16 MFU | 1624010 tok/s
step 5488/19560 | loss 3.548548 (-0.04z)| norm 0.2624 (-0.97z)| lr 5.10e-04 | 322.93 ms | 52.3% bf16 MFU | 1623987 tok/s
step 5489/19560 | loss 3.580507 (+0.67z)| norm 0.2807 (-0.02z)| lr 5.10e-04 | 322.64 ms | 52.3% bf16 MFU | 1624037 tok/s
step 5490/19560 | loss 3.526921 (-0.52z)| norm 0.2987 (+0.92z)| lr 5.10e-04 | 323.16 ms | 52.2% bf16 MFU | 1623955 tok/s
step 5491/19560 | loss 3.561368 (+0.29z)| norm 0.2613 (-1.03z)| lr 5.09e-04 | 322.36 ms | 52.4% bf16 MFU | 1624078 tok/s
step 5492/19560 | loss 3.539690 (-0.21z)| norm 0.2830 (+0.11z)| lr 5.09e-04 | 322.29 ms | 52.4% bf16 MFU | 1624212 tok/s
step 5493/19560 | loss 3.530181 (-0.43z)| norm 0.2625 (-0.95z)| lr 5.09e-04 | 321.90 ms | 52.4% bf16 MFU | 1624438 tok/s
step 5494/19560 | loss 3.558466 (+0.25z)| norm 0.2803 (+0.00z)| lr 5.09e-04 | 322.58 ms | 52.3% bf16 MFU | 1624482 tok/s
step 5495/19560 | loss 3.529996 (-0.44z)| norm 0.2694 (-0.57z)| lr 5.09e-04 | 322.29 ms | 52.4% bf16 MFU | 1624596 tok/s
step 5496/19560 | loss 3.498839 (-1.18z)| norm 0.2896 (+0.53z)| lr 5.09e-04 | 322.20 ms | 52.4% bf16 MFU | 1624727 tok/s
step 5497/19560 | loss 3.542816 (-0.12z)| norm 0.2903 (+0.56z)| lr 5.09e-04 | 322.38 ms | 52.4% bf16 MFU | 1624806 tok/s
step 5498/19560 | loss 3.559383 (+0.28z)| norm 0.2789 (-0.05z)| lr 5.09e-04 | 322.73 ms | 52.3% bf16 MFU | 1624792 tok/s
step 5499/19560 | loss 3.568217 (+0.49z)| norm 0.2961 (+0.87z)| lr 5.09e-04 | 322.84 ms | 52.3% bf16 MFU | 1624751 tok/s
step 5500/19560 | loss 3.568776 (+0.52z)| norm 0.2859 (+0.32z)| lr 5.09e-04 | 322.81 ms | 52.3% bf16 MFU | 1624720 tok/s
val loss 3.535431
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2806/10042 = 0.279426
step 5501/19560 | loss 3.587350 (+0.98z)| norm 0.2744 (-0.31z)| lr 5.09e-04 | 322.00 ms | 52.4% bf16 MFU | 1624896 tok/s
step 5502/19560 | loss 3.689809 (+3.30z)| norm 0.2624 (-0.94z)| lr 5.09e-04 | 322.31 ms | 52.4% bf16 MFU | 1624984 tok/s
step 5503/19560 | loss 3.602938 (+1.25z)| norm 0.2970 (+0.91z)| lr 5.09e-04 | 323.18 ms | 52.2% bf16 MFU | 1624848 tok/s
step 5504/19560 | loss 3.439763 (-2.51z)| norm 0.2701 (-0.52z)| lr 5.09e-04 | 323.06 ms | 52.2% bf16 MFU | 1624750 tok/s
step 5505/19560 | loss 3.605810 (+1.29z)| norm 0.2783 (-0.07z)| lr 5.09e-04 | 322.90 ms | 52.3% bf16 MFU | 1624696 tok/s
step 5506/19560 | loss 3.573493 (+0.56z)| norm 0.2730 (-0.36z)| lr 5.09e-04 | 322.65 ms | 52.3% bf16 MFU | 1624708 tok/s
step 5507/19560 | loss 3.551366 (+0.05z)| norm 0.2730 (-0.35z)| lr 5.09e-04 | 322.92 ms | 52.3% bf16 MFU | 1624653 tok/s
step 5508/19560 | loss 3.595881 (+1.07z)| norm 0.2680 (-0.61z)| lr 5.09e-04 | 323.51 ms | 52.2% bf16 MFU | 1624451 tok/s
step 5509/19560 | loss 3.542668 (-0.16z)| norm 0.2577 (-1.17z)| lr 5.09e-04 | 322.88 ms | 52.3% bf16 MFU | 1624416 tok/s
step 5510/19560 | loss 3.535421 (-0.32z)| norm 0.2754 (-0.20z)| lr 5.09e-04 | 323.14 ms | 52.2% bf16 MFU | 1624320 tok/s
step 5511/19560 | loss 3.556803 (+0.17z)| norm 0.2858 (+0.37z)| lr 5.09e-04 | 323.27 ms | 52.2% bf16 MFU | 1624194 tok/s
step 5512/19560 | loss 3.570205 (+0.47z)| norm 0.2706 (-0.46z)| lr 5.09e-04 | 322.54 ms | 52.3% bf16 MFU | 1624258 tok/s
step 5513/19560 | loss 3.603978 (+1.23z)| norm 0.2806 (+0.08z)| lr 5.09e-04 | 323.31 ms | 52.2% bf16 MFU | 1624126 tok/s
step 5514/19560 | loss 3.558006 (+0.18z)| norm 0.2611 (-0.98z)| lr 5.09e-04 | 323.00 ms | 52.3% bf16 MFU | 1624080 tok/s
step 5515/19560 | loss 3.534722 (-0.36z)| norm 0.2641 (-0.80z)| lr 5.09e-04 | 323.35 ms | 52.2% bf16 MFU | 1623946 tok/s
step 5516/19560 | loss 3.555783 (+0.12z)| norm 0.2482 (-1.64z)| lr 5.09e-04 | 322.90 ms | 52.3% bf16 MFU | 1623933 tok/s
step 5517/19560 | loss 3.572726 (+0.51z)| norm 0.2572 (-1.14z)| lr 5.09e-04 | 322.50 ms | 52.3% bf16 MFU | 1624021 tok/s
step 5518/19560 | loss 3.566573 (+0.37z)| norm 0.2579 (-1.10z)| lr 5.08e-04 | 323.14 ms | 52.2% bf16 MFU | 1623944 tok/s
step 5519/19560 | loss 3.532855 (-0.40z)| norm 0.2724 (-0.30z)| lr 5.08e-04 | 322.17 ms | 52.4% bf16 MFU | 1624116 tok/s
step 5520/19560 | loss 3.588879 (+0.88z)| norm 0.2920 (+0.76z)| lr 5.08e-04 | 322.60 ms | 52.3% bf16 MFU | 1624170 tok/s
step 5521/19560 | loss 3.561538 (+0.25z)| norm 0.2732 (-0.27z)| lr 5.08e-04 | 322.83 ms | 52.3% bf16 MFU | 1624162 tok/s
step 5522/19560 | loss 3.566983 (+0.37z)| norm 0.2777 (-0.02z)| lr 5.08e-04 | 322.64 ms | 52.3% bf16 MFU | 1624204 tok/s
step 5523/19560 | loss 3.562703 (+0.27z)| norm 0.2549 (-1.25z)| lr 5.08e-04 | 323.34 ms | 52.2% bf16 MFU | 1624067 tok/s
step 5524/19560 | loss 3.502635 (-1.10z)| norm 0.2738 (-0.23z)| lr 5.08e-04 | 322.07 ms | 52.4% bf16 MFU | 1624257 tok/s
step 5525/19560 | loss 3.519829 (-0.70z)| norm 0.2805 (+0.15z)| lr 5.08e-04 | 322.62 ms | 52.3% bf16 MFU | 1624298 tok/s
step 5526/19560 | loss 3.552129 (+0.06z)| norm 0.2683 (-0.51z)| lr 5.08e-04 | 322.67 ms | 52.3% bf16 MFU | 1624325 tok/s
step 5527/19560 | loss 3.584273 (+0.80z)| norm 0.2412 (-1.95z)| lr 5.08e-04 | 322.29 ms | 52.4% bf16 MFU | 1624446 tok/s
step 5528/19560 | loss 3.486130 (-1.50z)| norm 0.2727 (-0.26z)| lr 5.08e-04 | 323.48 ms | 52.2% bf16 MFU | 1624263 tok/s
step 5529/19560 | loss 3.523917 (-0.63z)| norm 0.2926 (+0.80z)| lr 5.08e-04 | 322.83 ms | 52.3% bf16 MFU | 1624251 tok/s
step 5530/19560 | loss 3.485593 (-1.54z)| norm 0.2851 (+0.38z)| lr 5.08e-04 | 322.65 ms | 52.3% bf16 MFU | 1624286 tok/s
step 5531/19560 | loss 3.588271 (+0.88z)| norm 0.2723 (-0.32z)| lr 5.08e-04 | 323.27 ms | 52.2% bf16 MFU | 1624164 tok/s
step 5532/19560 | loss 3.522126 (-0.67z)| norm 0.2774 (-0.04z)| lr 5.08e-04 | 322.37 ms | 52.4% bf16 MFU | 1624274 tok/s
step 5533/19560 | loss 3.494080 (-1.33z)| norm 0.2653 (-0.70z)| lr 5.08e-04 | 322.73 ms | 52.3% bf16 MFU | 1624286 tok/s
step 5534/19560 | loss 3.518134 (-0.76z)| norm 0.2763 (-0.11z)| lr 5.08e-04 | 322.47 ms | 52.3% bf16 MFU | 1624365 tok/s
step 5535/19560 | loss 3.524988 (-0.58z)| norm 0.2776 (-0.03z)| lr 5.08e-04 | 323.00 ms | 52.3% bf16 MFU | 1624307 tok/s
step 5536/19560 | loss 3.524267 (-0.60z)| norm 0.2519 (-1.42z)| lr 5.08e-04 | 322.32 ms | 52.4% bf16 MFU | 1624422 tok/s
step 5537/19560 | loss 3.518286 (-0.75z)| norm 0.3046 (+1.42z)| lr 5.08e-04 | 322.45 ms | 52.3% bf16 MFU | 1624498 tok/s
step 5538/19560 | loss 3.562814 (+0.30z)| norm 0.2855 (+0.38z)| lr 5.08e-04 | 323.39 ms | 52.2% bf16 MFU | 1624334 tok/s
step 5539/19560 | loss 3.545013 (-0.11z)| norm 0.2569 (-1.17z)| lr 5.08e-04 | 322.88 ms | 52.3% bf16 MFU | 1624306 tok/s
step 5540/19560 | loss 3.539993 (-0.24z)| norm 0.2977 (+1.03z)| lr 5.08e-04 | 322.87 ms | 52.3% bf16 MFU | 1624283 tok/s
step 5541/19560 | loss 3.580481 (+0.73z)| norm 0.2580 (-1.12z)| lr 5.08e-04 | 322.43 ms | 52.3% bf16 MFU | 1624372 tok/s
step 5542/19560 | loss 3.622410 (+1.70z)| norm 0.2961 (+0.98z)| lr 5.08e-04 | 322.43 ms | 52.3% bf16 MFU | 1624456 tok/s
step 5543/19560 | loss 3.597130 (+1.09z)| norm 0.2705 (-0.43z)| lr 5.08e-04 | 323.12 ms | 52.2% bf16 MFU | 1624361 tok/s
step 5544/19560 | loss 3.545496 (-0.13z)| norm 0.2764 (-0.12z)| lr 5.08e-04 | 322.77 ms | 52.3% bf16 MFU | 1624360 tok/s
step 5545/19560 | loss 3.473356 (-1.80z)| norm 0.2852 (+0.38z)| lr 5.08e-04 | 322.37 ms | 52.4% bf16 MFU | 1624459 tok/s
step 5546/19560 | loss 3.554437 (+0.11z)| norm 0.2741 (-0.24z)| lr 5.07e-04 | 323.16 ms | 52.2% bf16 MFU | 1624356 tok/s
step 5547/19560 | loss 3.522269 (-0.65z)| norm 0.2598 (-1.04z)| lr 5.07e-04 | 322.85 ms | 52.3% bf16 MFU | 1624336 tok/s
step 5548/19560 | loss 3.552979 (+0.09z)| norm 0.2703 (-0.47z)| lr 5.07e-04 | 322.39 ms | 52.4% bf16 MFU | 1624432 tok/s
step 5549/19560 | loss 3.502609 (-1.11z)| norm 0.3328 (+2.94z)| lr 5.07e-04 | 323.22 ms | 52.2% bf16 MFU | 1624315 tok/s
step 5550/19560 | loss 3.510655 (-0.90z)| norm 0.3236 (+2.37z)| lr 5.07e-04 | 322.80 ms | 52.3% bf16 MFU | 1624309 tok/s
step 5551/19560 | loss 3.536957 (-0.28z)| norm 0.2646 (-0.80z)| lr 5.07e-04 | 322.77 ms | 52.3% bf16 MFU | 1624310 tok/s
step 5552/19560 | loss 3.543479 (-0.12z)| norm 0.2709 (-0.46z)| lr 5.07e-04 | 323.19 ms | 52.2% bf16 MFU | 1624207 tok/s
step 5553/19560 | loss 3.548745 (+0.00z)| norm 0.2812 (+0.10z)| lr 5.07e-04 | 322.77 ms | 52.3% bf16 MFU | 1624213 tok/s
step 5554/19560 | loss 3.545292 (-0.09z)| norm 0.2706 (-0.47z)| lr 5.07e-04 | 322.44 ms | 52.3% bf16 MFU | 1624304 tok/s
step 5555/19560 | loss 3.518772 (-0.71z)| norm 0.2681 (-0.59z)| lr 5.07e-04 | 322.73 ms | 52.3% bf16 MFU | 1624315 tok/s
step 5556/19560 | loss 3.677022 (+2.93z)| norm 0.2665 (-0.67z)| lr 5.07e-04 | 322.47 ms | 52.3% bf16 MFU | 1624392 tok/s
step 5557/19560 | loss 3.576635 (+0.61z)| norm 0.2634 (-0.82z)| lr 5.07e-04 | 322.79 ms | 52.3% bf16 MFU | 1624384 tok/s
step 5558/19560 | loss 3.529827 (-0.46z)| norm 0.3300 (+2.77z)| lr 5.07e-04 | 322.68 ms | 52.3% bf16 MFU | 1624403 tok/s
step 5559/19560 | loss 3.537335 (-0.28z)| norm 0.3015 (+1.26z)| lr 5.07e-04 | 322.41 ms | 52.3% bf16 MFU | 1624490 tok/s
step 5560/19560 | loss 3.535274 (-0.33z)| norm 0.2717 (-0.36z)| lr 5.07e-04 | 323.03 ms | 52.2% bf16 MFU | 1624417 tok/s
step 5561/19560 | loss 3.560294 (+0.24z)| norm 0.2682 (-0.54z)| lr 5.07e-04 | 322.76 ms | 52.3% bf16 MFU | 1624415 tok/s
step 5562/19560 | loss 3.464063 (-1.97z)| norm 0.2713 (-0.36z)| lr 5.07e-04 | 322.55 ms | 52.3% bf16 MFU | 1624466 tok/s
step 5563/19560 | loss 3.538057 (-0.24z)| norm 0.2887 (+0.62z)| lr 5.07e-04 | 322.54 ms | 52.3% bf16 MFU | 1624518 tok/s
step 5564/19560 | loss 3.497082 (-1.19z)| norm 0.2821 (+0.26z)| lr 5.07e-04 | 322.66 ms | 52.3% bf16 MFU | 1624537 tok/s
step 5565/19560 | loss 3.597974 (+1.18z)| norm 0.3019 (+1.37z)| lr 5.07e-04 | 322.59 ms | 52.3% bf16 MFU | 1624573 tok/s
step 5566/19560 | loss 3.537189 (-0.25z)| norm 0.3425 (+3.47z)| lr 5.07e-04 | 322.71 ms | 52.3% bf16 MFU | 1624578 tok/s
step 5567/19560 | loss 3.597078 (+1.16z)| norm 0.3079 (+1.58z)| lr 5.07e-04 | 323.08 ms | 52.2% bf16 MFU | 1624488 tok/s
step 5568/19560 | loss 3.521240 (-0.61z)| norm 0.2949 (+0.89z)| lr 5.07e-04 | 322.11 ms | 52.4% bf16 MFU | 1624646 tok/s
step 5569/19560 | loss 3.674544 (+2.88z)| norm 0.2890 (+0.57z)| lr 5.07e-04 | 322.31 ms | 52.4% bf16 MFU | 1624748 tok/s
step 5570/19560 | loss 3.537106 (-0.26z)| norm 0.2948 (+0.87z)| lr 5.07e-04 | 323.28 ms | 52.2% bf16 MFU | 1624600 tok/s
step 5571/19560 | loss 3.553171 (+0.10z)| norm 0.3012 (+1.20z)| lr 5.07e-04 | 322.89 ms | 52.3% bf16 MFU | 1624557 tok/s
step 5572/19560 | loss 3.522363 (-0.61z)| norm 0.3283 (+2.56z)| lr 5.07e-04 | 322.40 ms | 52.3% bf16 MFU | 1624639 tok/s
step 5573/19560 | loss 3.532572 (-0.38z)| norm 0.2936 (+0.73z)| lr 5.07e-04 | 323.09 ms | 52.2% bf16 MFU | 1624544 tok/s
step 5574/19560 | loss 3.544080 (-0.11z)| norm 0.2798 (+0.01z)| lr 5.06e-04 | 322.21 ms | 52.4% bf16 MFU | 1624676 tok/s
step 5575/19560 | loss 3.531888 (-0.39z)| norm 0.2797 (-0.01z)| lr 5.06e-04 | 322.18 ms | 52.4% bf16 MFU | 1624807 tok/s
step 5576/19560 | loss 3.537797 (-0.25z)| norm 0.3099 (+1.57z)| lr 5.06e-04 | 322.88 ms | 52.3% bf16 MFU | 1624756 tok/s
step 5577/19560 | loss 3.491370 (-1.31z)| norm 0.2852 (+0.27z)| lr 5.06e-04 | 323.13 ms | 52.2% bf16 MFU | 1624644 tok/s
step 5578/19560 | loss 3.608209 (+1.36z)| norm 0.2754 (-0.25z)| lr 5.06e-04 | 322.45 ms | 52.3% bf16 MFU | 1624710 tok/s
step 5579/19560 | loss 3.553614 (+0.10z)| norm 0.2787 (-0.08z)| lr 5.06e-04 | 322.70 ms | 52.3% bf16 MFU | 1624709 tok/s
step 5580/19560 | loss 3.533902 (-0.35z)| norm 0.2583 (-1.15z)| lr 5.06e-04 | 322.87 ms | 52.3% bf16 MFU | 1624665 tok/s
step 5581/19560 | loss 3.498250 (-1.16z)| norm 0.2516 (-1.50z)| lr 5.06e-04 | 322.31 ms | 52.4% bf16 MFU | 1624763 tok/s
step 5582/19560 | loss 3.584560 (+0.84z)| norm 0.2587 (-1.12z)| lr 5.06e-04 | 322.88 ms | 52.3% bf16 MFU | 1624715 tok/s
step 5583/19560 | loss 3.517011 (-0.71z)| norm 0.2589 (-1.11z)| lr 5.06e-04 | 323.09 ms | 52.2% bf16 MFU | 1624615 tok/s
step 5584/19560 | loss 3.538913 (-0.21z)| norm 0.2686 (-0.60z)| lr 5.06e-04 | 322.84 ms | 52.3% bf16 MFU | 1624584 tok/s
step 5585/19560 | loss 3.573328 (+0.62z)| norm 0.2569 (-1.20z)| lr 5.06e-04 | 322.87 ms | 52.3% bf16 MFU | 1624548 tok/s
step 5586/19560 | loss 3.617868 (+1.64z)| norm 0.2614 (-0.95z)| lr 5.06e-04 | 322.44 ms | 52.3% bf16 MFU | 1624620 tok/s
step 5587/19560 | loss 3.515764 (-0.74z)| norm 0.2789 (-0.06z)| lr 5.06e-04 | 322.60 ms | 52.3% bf16 MFU | 1624649 tok/s
step 5588/19560 | loss 3.488220 (-1.36z)| norm 0.2330 (-2.39z)| lr 5.06e-04 | 322.90 ms | 52.3% bf16 MFU | 1624602 tok/s
step 5589/19560 | loss 3.506207 (-0.94z)| norm 0.2672 (-0.63z)| lr 5.06e-04 | 322.43 ms | 52.3% bf16 MFU | 1624673 tok/s
step 5590/19560 | loss 3.566453 (+0.44z)| norm 0.2726 (-0.36z)| lr 5.06e-04 | 322.44 ms | 52.3% bf16 MFU | 1624739 tok/s
step 5591/19560 | loss 3.443221 (-2.36z)| norm 0.2560 (-1.20z)| lr 5.06e-04 | 322.84 ms | 52.3% bf16 MFU | 1624702 tok/s
step 5592/19560 | loss 3.564350 (+0.39z)| norm 0.2985 (+0.96z)| lr 5.06e-04 | 322.74 ms | 52.3% bf16 MFU | 1624692 tok/s
step 5593/19560 | loss 3.573084 (+0.58z)| norm 0.2591 (-1.04z)| lr 5.06e-04 | 322.83 ms | 52.3% bf16 MFU | 1624659 tok/s
step 5594/19560 | loss 3.538115 (-0.19z)| norm 0.2860 (+0.32z)| lr 5.06e-04 | 322.97 ms | 52.3% bf16 MFU | 1624592 tok/s
step 5595/19560 | loss 3.517065 (-0.69z)| norm 0.3021 (+1.13z)| lr 5.06e-04 | 323.51 ms | 52.2% bf16 MFU | 1624393 tok/s
step 5596/19560 | loss 3.562371 (+0.40z)| norm 0.3131 (+1.65z)| lr 5.06e-04 | 322.75 ms | 52.3% bf16 MFU | 1624396 tok/s
step 5597/19560 | loss 3.511175 (-0.82z)| norm 0.2956 (+0.77z)| lr 5.06e-04 | 322.64 ms | 52.3% bf16 MFU | 1624426 tok/s
step 5598/19560 | loss 3.521746 (-0.58z)| norm 0.3065 (+1.29z)| lr 5.06e-04 | 322.66 ms | 52.3% bf16 MFU | 1624451 tok/s
step 5599/19560 | loss 3.550453 (+0.10z)| norm 0.2792 (-0.06z)| lr 5.06e-04 | 323.55 ms | 52.2% bf16 MFU | 1624250 tok/s
step 5600/19560 | loss 3.564370 (+0.44z)| norm 0.2719 (-0.42z)| lr 5.06e-04 | 322.38 ms | 52.4% bf16 MFU | 1624353 tok/s
step 5601/19560 | loss 3.580525 (+0.82z)| norm 0.2533 (-1.34z)| lr 5.05e-04 | 322.15 ms | 52.4% bf16 MFU | 1624508 tok/s
step 5602/19560 | loss 3.533915 (-0.30z)| norm 0.2682 (-0.59z)| lr 5.05e-04 | 323.01 ms | 52.2% bf16 MFU | 1624439 tok/s
step 5603/19560 | loss 3.504232 (-1.05z)| norm 0.2989 (+0.99z)| lr 5.05e-04 | 322.86 ms | 52.3% bf16 MFU | 1624410 tok/s
step 5604/19560 | loss 3.561188 (+0.37z)| norm 0.2844 (+0.24z)| lr 5.05e-04 | 322.79 ms | 52.3% bf16 MFU | 1624401 tok/s
step 5605/19560 | loss 3.529927 (-0.40z)| norm 0.2394 (-2.01z)| lr 5.05e-04 | 323.93 ms | 52.1% bf16 MFU | 1624107 tok/s
step 5606/19560 | loss 3.518752 (-0.67z)| norm 0.2753 (-0.19z)| lr 5.05e-04 | 323.38 ms | 52.2% bf16 MFU | 1623964 tok/s
step 5607/19560 | loss 3.639977 (+2.28z)| norm 0.3063 (+1.37z)| lr 5.05e-04 | 322.18 ms | 52.4% bf16 MFU | 1624131 tok/s
step 5608/19560 | loss 3.525391 (-0.52z)| norm 0.2584 (-1.03z)| lr 5.05e-04 | 322.42 ms | 52.3% bf16 MFU | 1624229 tok/s
step 5609/19560 | loss 3.523923 (-0.55z)| norm 0.2759 (-0.14z)| lr 5.05e-04 | 323.49 ms | 52.2% bf16 MFU | 1624053 tok/s
step 5610/19560 | loss 3.537649 (-0.21z)| norm 0.2526 (-1.33z)| lr 5.05e-04 | 322.65 ms | 52.3% bf16 MFU | 1624097 tok/s
step 5611/19560 | loss 3.509822 (-0.90z)| norm 0.2672 (-0.55z)| lr 5.05e-04 | 322.08 ms | 52.4% bf16 MFU | 1624284 tok/s
step 5612/19560 | loss 3.513795 (-0.79z)| norm 0.2687 (-0.48z)| lr 5.05e-04 | 323.29 ms | 52.2% bf16 MFU | 1624155 tok/s
step 5613/19560 | loss 3.551083 (+0.12z)| norm 0.2687 (-0.48z)| lr 5.05e-04 | 323.36 ms | 52.2% bf16 MFU | 1624017 tok/s
step 5614/19560 | loss 3.500224 (-1.12z)| norm 0.2830 (+0.28z)| lr 5.05e-04 | 322.87 ms | 52.3% bf16 MFU | 1624007 tok/s
step 5615/19560 | loss 3.548241 (+0.05z)| norm 0.2938 (+0.84z)| lr 5.05e-04 | 323.08 ms | 52.2% bf16 MFU | 1623946 tok/s
step 5616/19560 | loss 3.561930 (+0.38z)| norm 0.2807 (+0.14z)| lr 5.05e-04 | 322.11 ms | 52.4% bf16 MFU | 1624131 tok/s
step 5617/19560 | loss 3.571746 (+0.63z)| norm 0.3031 (+1.30z)| lr 5.05e-04 | 323.06 ms | 52.2% bf16 MFU | 1624069 tok/s
step 5618/19560 | loss 3.518059 (-0.70z)| norm 0.3019 (+1.24z)| lr 5.05e-04 | 322.65 ms | 52.3% bf16 MFU | 1624114 tok/s
step 5619/19560 | loss 3.794532 (+5.39z)| norm 0.3656 (+4.22z)| lr 5.05e-04 | 322.66 ms | 52.3% bf16 MFU | 1624154 tok/s
step 5620/19560 | loss 3.546651 (-0.03z)| norm 0.3104 (+1.50z)| lr 5.05e-04 | 322.77 ms | 52.3% bf16 MFU | 1624162 tok/s
step 5621/19560 | loss 3.515968 (-0.70z)| norm 0.3302 (+2.38z)| lr 5.05e-04 | 323.41 ms | 52.2% bf16 MFU | 1624010 tok/s
step 5622/19560 | loss 3.595445 (+1.02z)| norm 0.3117 (+1.48z)| lr 5.05e-04 | 322.51 ms | 52.3% bf16 MFU | 1624092 tok/s
step 5623/19560 | loss 3.545611 (-0.06z)| norm 0.2854 (+0.24z)| lr 5.05e-04 | 323.02 ms | 52.2% bf16 MFU | 1624040 tok/s
step 5624/19560 | loss 3.580284 (+0.68z)| norm 0.2797 (-0.02z)| lr 5.05e-04 | 323.06 ms | 52.2% bf16 MFU | 1623984 tok/s
step 5625/19560 | loss 3.530408 (-0.41z)| norm 0.2804 (+0.01z)| lr 5.05e-04 | 322.92 ms | 52.3% bf16 MFU | 1623963 tok/s
step 5626/19560 | loss 3.492110 (-1.22z)| norm 0.2751 (-0.23z)| lr 5.05e-04 | 322.84 ms | 52.3% bf16 MFU | 1623965 tok/s
step 5627/19560 | loss 3.484244 (-1.37z)| norm 0.2776 (-0.11z)| lr 5.05e-04 | 322.80 ms | 52.3% bf16 MFU | 1623976 tok/s
step 5628/19560 | loss 3.603228 (+1.18z)| norm 0.2584 (-1.00z)| lr 5.05e-04 | 322.82 ms | 52.3% bf16 MFU | 1623982 tok/s
step 5629/19560 | loss 3.595483 (+1.01z)| norm 0.2752 (-0.21z)| lr 5.04e-04 | 322.98 ms | 52.3% bf16 MFU | 1623948 tok/s
step 5630/19560 | loss 3.533176 (-0.31z)| norm 0.2719 (-0.37z)| lr 5.04e-04 | 322.75 ms | 52.3% bf16 MFU | 1623973 tok/s
step 5631/19560 | loss 3.543188 (-0.07z)| norm 0.2753 (-0.20z)| lr 5.04e-04 | 322.74 ms | 52.3% bf16 MFU | 1624000 tok/s
step 5632/19560 | loss 3.540103 (-0.17z)| norm 0.2782 (-0.07z)| lr 5.04e-04 | 322.64 ms | 52.3% bf16 MFU | 1624050 tok/s
step 5633/19560 | loss 3.568120 (+0.49z)| norm 0.2729 (-0.32z)| lr 5.04e-04 | 322.96 ms | 52.3% bf16 MFU | 1624018 tok/s
step 5634/19560 | loss 3.539543 (-0.17z)| norm 0.2854 (+0.27z)| lr 5.04e-04 | 322.81 ms | 52.3% bf16 MFU | 1624024 tok/s
step 5635/19560 | loss 3.503392 (-0.99z)| norm 0.2535 (-1.22z)| lr 5.04e-04 | 322.97 ms | 52.3% bf16 MFU | 1623990 tok/s
step 5636/19560 | loss 3.548148 (+0.05z)| norm 0.2824 (+0.13z)| lr 5.04e-04 | 322.79 ms | 52.3% bf16 MFU | 1624002 tok/s
step 5637/19560 | loss 3.549762 (+0.09z)| norm 0.3426 (+2.84z)| lr 5.04e-04 | 322.77 ms | 52.3% bf16 MFU | 1624020 tok/s
step 5638/19560 | loss 3.655515 (+2.45z)| norm 0.2700 (-0.47z)| lr 5.04e-04 | 323.36 ms | 52.2% bf16 MFU | 1623888 tok/s
step 5639/19560 | loss 3.492695 (-1.21z)| norm 0.2775 (-0.12z)| lr 5.04e-04 | 322.90 ms | 52.3% bf16 MFU | 1623877 tok/s
step 5640/19560 | loss 3.586693 (+0.90z)| norm 0.2781 (-0.10z)| lr 5.04e-04 | 322.43 ms | 52.3% bf16 MFU | 1623985 tok/s
step 5641/19560 | loss 3.510386 (-0.80z)| norm 0.2736 (-0.30z)| lr 5.04e-04 | 323.40 ms | 52.2% bf16 MFU | 1623843 tok/s
step 5642/19560 | loss 3.528214 (-0.39z)| norm 0.3074 (+1.22z)| lr 5.04e-04 | 322.45 ms | 52.3% bf16 MFU | 1623948 tok/s
step 5643/19560 | loss 3.522720 (-0.51z)| norm 0.2794 (-0.06z)| lr 5.04e-04 | 322.88 ms | 52.3% bf16 MFU | 1623940 tok/s
step 5644/19560 | loss 3.521676 (-0.53z)| norm 0.2528 (-1.28z)| lr 5.04e-04 | 322.66 ms | 52.3% bf16 MFU | 1623988 tok/s
step 5645/19560 | loss 3.522490 (-0.50z)| norm 0.2549 (-1.18z)| lr 5.04e-04 | 322.57 ms | 52.3% bf16 MFU | 1624056 tok/s
step 5646/19560 | loss 3.556815 (+0.27z)| norm 0.2750 (-0.27z)| lr 5.04e-04 | 322.91 ms | 52.3% bf16 MFU | 1624034 tok/s
step 5647/19560 | loss 3.552042 (+0.16z)| norm 0.3028 (+0.99z)| lr 5.04e-04 | 322.83 ms | 52.3% bf16 MFU | 1624034 tok/s
step 5648/19560 | loss 3.592956 (+1.08z)| norm 0.2973 (+0.74z)| lr 5.04e-04 | 322.46 ms | 52.3% bf16 MFU | 1624127 tok/s
step 5649/19560 | loss 3.507677 (-0.83z)| norm 0.2661 (-0.68z)| lr 5.04e-04 | 323.30 ms | 52.2% bf16 MFU | 1624004 tok/s
step 5650/19560 | loss 3.576226 (+0.71z)| norm 0.2788 (-0.10z)| lr 5.04e-04 | 322.71 ms | 52.3% bf16 MFU | 1624036 tok/s
step 5651/19560 | loss 3.550003 (+0.12z)| norm 0.2997 (+0.83z)| lr 5.04e-04 | 322.81 ms | 52.3% bf16 MFU | 1624040 tok/s
step 5652/19560 | loss 3.499047 (-1.02z)| norm 0.2993 (+0.81z)| lr 5.04e-04 | 322.77 ms | 52.3% bf16 MFU | 1624054 tok/s
step 5653/19560 | loss 3.564889 (+0.45z)| norm 0.2633 (-0.83z)| lr 5.04e-04 | 323.12 ms | 52.2% bf16 MFU | 1623980 tok/s
step 5654/19560 | loss 3.461927 (-1.82z)| norm 0.2744 (-0.33z)| lr 5.04e-04 | 323.43 ms | 52.2% bf16 MFU | 1623833 tok/s
step 5655/19560 | loss 3.544269 (+0.01z)| norm 0.2572 (-1.12z)| lr 5.04e-04 | 322.43 ms | 52.3% bf16 MFU | 1623943 tok/s
step 5656/19560 | loss 3.560527 (+0.36z)| norm 0.2567 (-1.14z)| lr 5.03e-04 | 323.22 ms | 52.2% bf16 MFU | 1623851 tok/s
step 5657/19560 | loss 3.529962 (-0.32z)| norm 0.2713 (-0.46z)| lr 5.03e-04 | 323.34 ms | 52.2% bf16 MFU | 1623732 tok/s
step 5658/19560 | loss 3.501465 (-0.97z)| norm 0.2480 (-1.50z)| lr 5.03e-04 | 323.11 ms | 52.2% bf16 MFU | 1623677 tok/s
step 5659/19560 | loss 3.502812 (-0.92z)| norm 0.2658 (-0.69z)| lr 5.03e-04 | 323.42 ms | 52.2% bf16 MFU | 1623547 tok/s
step 5660/19560 | loss 3.559983 (+0.35z)| norm 0.2450 (-1.61z)| lr 5.03e-04 | 322.94 ms | 52.3% bf16 MFU | 1623544 tok/s
step 5661/19560 | loss 3.550565 (+0.13z)| norm 0.2581 (-1.01z)| lr 5.03e-04 | 322.59 ms | 52.3% bf16 MFU | 1623629 tok/s
step 5662/19560 | loss 3.507322 (-0.84z)| norm 0.2710 (-0.43z)| lr 5.03e-04 | 323.06 ms | 52.2% bf16 MFU | 1623591 tok/s
step 5663/19560 | loss 3.585789 (+0.91z)| norm 0.2745 (-0.28z)| lr 5.03e-04 | 323.04 ms | 52.2% bf16 MFU | 1623561 tok/s
step 5664/19560 | loss 3.496573 (-1.08z)| norm 0.2632 (-0.79z)| lr 5.03e-04 | 322.62 ms | 52.3% bf16 MFU | 1623638 tok/s
step 5665/19560 | loss 3.571286 (+0.58z)| norm 0.2807 (+0.00z)| lr 5.03e-04 | 323.34 ms | 52.2% bf16 MFU | 1623530 tok/s
step 5666/19560 | loss 3.492299 (-1.16z)| norm 0.2931 (+0.56z)| lr 5.03e-04 | 322.85 ms | 52.3% bf16 MFU | 1623549 tok/s
step 5667/19560 | loss 3.496222 (-1.06z)| norm 0.3138 (+1.47z)| lr 5.03e-04 | 322.98 ms | 52.3% bf16 MFU | 1623537 tok/s
step 5668/19560 | loss 3.510243 (-0.75z)| norm 0.2675 (-0.60z)| lr 5.03e-04 | 322.80 ms | 52.3% bf16 MFU | 1623571 tok/s
step 5669/19560 | loss 3.582397 (+0.84z)| norm 0.2953 (+0.64z)| lr 5.03e-04 | 322.80 ms | 52.3% bf16 MFU | 1623602 tok/s
step 5670/19560 | loss 3.560243 (+0.37z)| norm 0.3338 (+2.31z)| lr 5.03e-04 | 323.82 ms | 52.1% bf16 MFU | 1623375 tok/s
step 5671/19560 | loss 3.509292 (-0.75z)| norm 0.2793 (-0.10z)| lr 5.03e-04 | 323.00 ms | 52.3% bf16 MFU | 1623365 tok/s
step 5672/19560 | loss 3.595326 (+1.16z)| norm 0.3084 (+1.17z)| lr 5.03e-04 | 322.50 ms | 52.3% bf16 MFU | 1623481 tok/s
step 5673/19560 | loss 3.511297 (-0.73z)| norm 0.2530 (-1.24z)| lr 5.03e-04 | 322.29 ms | 52.4% bf16 MFU | 1623645 tok/s
step 5674/19560 | loss 3.545919 (+0.05z)| norm 0.2698 (-0.51z)| lr 5.03e-04 | 322.54 ms | 52.3% bf16 MFU | 1623737 tok/s
step 5675/19560 | loss 3.563416 (+0.44z)| norm 0.2687 (-0.56z)| lr 5.03e-04 | 322.72 ms | 52.3% bf16 MFU | 1623781 tok/s
step 5676/19560 | loss 3.567522 (+0.53z)| norm 0.2997 (+0.78z)| lr 5.03e-04 | 322.76 ms | 52.3% bf16 MFU | 1623812 tok/s
step 5677/19560 | loss 3.509952 (-0.77z)| norm 0.2661 (-0.67z)| lr 5.03e-04 | 322.10 ms | 52.4% bf16 MFU | 1624006 tok/s
step 5678/19560 | loss 3.560049 (+0.35z)| norm 0.2626 (-0.82z)| lr 5.03e-04 | 322.26 ms | 52.4% bf16 MFU | 1624151 tok/s
step 5679/19560 | loss 3.556067 (+0.26z)| norm 0.2819 (+0.05z)| lr 5.03e-04 | 323.15 ms | 52.2% bf16 MFU | 1624064 tok/s
step 5680/19560 | loss 3.566732 (+0.49z)| norm 0.2777 (-0.15z)| lr 5.03e-04 | 323.40 ms | 52.2% bf16 MFU | 1623919 tok/s
step 5681/19560 | loss 3.488246 (-1.25z)| norm 0.2756 (-0.24z)| lr 5.03e-04 | 322.71 ms | 52.3% bf16 MFU | 1623954 tok/s
step 5682/19560 | loss 3.571944 (+0.61z)| norm 0.2927 (+0.52z)| lr 5.03e-04 | 322.48 ms | 52.3% bf16 MFU | 1624047 tok/s
step 5683/19560 | loss 3.454887 (-1.96z)| norm 0.3305 (+2.17z)| lr 5.02e-04 | 322.52 ms | 52.3% bf16 MFU | 1624125 tok/s
step 5684/19560 | loss 3.590594 (+1.07z)| norm 0.2719 (-0.43z)| lr 5.02e-04 | 322.99 ms | 52.3% bf16 MFU | 1624081 tok/s
step 5685/19560 | loss 3.633410 (+2.00z)| norm 0.3285 (+2.03z)| lr 5.02e-04 | 323.10 ms | 52.2% bf16 MFU | 1624012 tok/s
step 5686/19560 | loss 3.529132 (-0.33z)| norm 0.2980 (+0.72z)| lr 5.02e-04 | 322.95 ms | 52.3% bf16 MFU | 1623983 tok/s
step 5687/19560 | loss 3.573321 (+0.65z)| norm 0.2765 (-0.23z)| lr 5.02e-04 | 322.93 ms | 52.3% bf16 MFU | 1623962 tok/s
step 5688/19560 | loss 3.706463 (+3.42z)| norm 0.3118 (+1.32z)| lr 5.02e-04 | 322.50 ms | 52.3% bf16 MFU | 1624050 tok/s
step 5689/19560 | loss 3.513038 (-0.68z)| norm 0.3053 (+1.02z)| lr 5.02e-04 | 322.48 ms | 52.3% bf16 MFU | 1624138 tok/s
step 5690/19560 | loss 3.638873 (+1.96z)| norm 0.2878 (+0.24z)| lr 5.02e-04 | 322.63 ms | 52.3% bf16 MFU | 1624183 tok/s
step 5691/19560 | loss 3.544356 (-0.04z)| norm 0.2771 (-0.23z)| lr 5.02e-04 | 323.37 ms | 52.2% bf16 MFU | 1624040 tok/s
step 5692/19560 | loss 3.539525 (-0.15z)| norm 0.2831 (+0.03z)| lr 5.02e-04 | 322.34 ms | 52.4% bf16 MFU | 1624165 tok/s
step 5693/19560 | loss 3.538520 (-0.17z)| norm 0.2714 (-0.47z)| lr 5.02e-04 | 322.39 ms | 52.4% bf16 MFU | 1624269 tok/s
step 5694/19560 | loss 3.499393 (-0.99z)| norm 0.2421 (-1.77z)| lr 5.02e-04 | 323.23 ms | 52.2% bf16 MFU | 1624156 tok/s
step 5695/19560 | loss 3.503807 (-0.88z)| norm 0.2768 (-0.19z)| lr 5.02e-04 | 322.88 ms | 52.3% bf16 MFU | 1624138 tok/s
step 5696/19560 | loss 3.572204 (+0.56z)| norm 0.2667 (-0.64z)| lr 5.02e-04 | 322.90 ms | 52.3% bf16 MFU | 1624116 tok/s
step 5697/19560 | loss 3.579941 (+0.77z)| norm 0.2499 (-1.38z)| lr 5.02e-04 | 322.45 ms | 52.3% bf16 MFU | 1624207 tok/s
step 5698/19560 | loss 3.556056 (+0.24z)| norm 0.2687 (-0.53z)| lr 5.02e-04 | 322.69 ms | 52.3% bf16 MFU | 1624234 tok/s
step 5699/19560 | loss 3.536323 (-0.19z)| norm 0.2386 (-1.84z)| lr 5.02e-04 | 322.24 ms | 52.4% bf16 MFU | 1624372 tok/s
step 5700/19560 | loss 3.595500 (+1.09z)| norm 0.2521 (-1.22z)| lr 5.02e-04 | 323.18 ms | 52.2% bf16 MFU | 1624269 tok/s
step 5701/19560 | loss 3.543591 (-0.04z)| norm 0.2742 (-0.22z)| lr 5.02e-04 | 322.63 ms | 52.3% bf16 MFU | 1624307 tok/s
step 5702/19560 | loss 3.594902 (+1.06z)| norm 0.2849 (+0.26z)| lr 5.02e-04 | 322.66 ms | 52.3% bf16 MFU | 1624337 tok/s
step 5703/19560 | loss 3.542629 (-0.08z)| norm 0.2680 (-0.50z)| lr 5.02e-04 | 323.32 ms | 52.2% bf16 MFU | 1624200 tok/s
step 5704/19560 | loss 3.546777 (+0.01z)| norm 0.2539 (-1.12z)| lr 5.02e-04 | 323.11 ms | 52.2% bf16 MFU | 1624120 tok/s
step 5705/19560 | loss 3.599096 (+1.13z)| norm 0.2795 (+0.04z)| lr 5.02e-04 | 322.27 ms | 52.4% bf16 MFU | 1624257 tok/s
step 5706/19560 | loss 3.567874 (+0.46z)| norm 0.2659 (-0.57z)| lr 5.02e-04 | 322.80 ms | 52.3% bf16 MFU | 1624253 tok/s
step 5707/19560 | loss 3.531075 (-0.34z)| norm 0.3032 (+1.10z)| lr 5.02e-04 | 322.71 ms | 52.3% bf16 MFU | 1624272 tok/s
step 5708/19560 | loss 3.554488 (+0.17z)| norm 0.2706 (-0.37z)| lr 5.02e-04 | 322.66 ms | 52.3% bf16 MFU | 1624303 tok/s
step 5709/19560 | loss 3.602003 (+1.19z)| norm 0.2859 (+0.31z)| lr 5.02e-04 | 323.25 ms | 52.2% bf16 MFU | 1624183 tok/s
step 5710/19560 | loss 3.545082 (-0.05z)| norm 0.3240 (+1.99z)| lr 5.01e-04 | 322.44 ms | 52.3% bf16 MFU | 1624275 tok/s
step 5711/19560 | loss 3.470096 (-1.67z)| norm 0.3026 (+1.02z)| lr 5.01e-04 | 323.34 ms | 52.2% bf16 MFU | 1624134 tok/s
step 5712/19560 | loss 3.471175 (-1.61z)| norm 0.2623 (-0.79z)| lr 5.01e-04 | 322.75 ms | 52.3% bf16 MFU | 1624150 tok/s
step 5713/19560 | loss 3.582667 (+0.78z)| norm 0.2957 (+0.69z)| lr 5.01e-04 | 322.27 ms | 52.4% bf16 MFU | 1624286 tok/s
step 5714/19560 | loss 3.518944 (-0.58z)| norm 0.2774 (-0.13z)| lr 5.01e-04 | 322.38 ms | 52.4% bf16 MFU | 1624388 tok/s
step 5715/19560 | loss 3.606868 (+1.30z)| norm 0.2752 (-0.23z)| lr 5.01e-04 | 323.04 ms | 52.2% bf16 MFU | 1624319 tok/s
step 5716/19560 | loss 3.595792 (+1.05z)| norm 0.2703 (-0.47z)| lr 5.01e-04 | 322.80 ms | 52.3% bf16 MFU | 1624313 tok/s
step 5717/19560 | loss 3.512728 (-0.74z)| norm 0.2725 (-0.37z)| lr 5.01e-04 | 322.77 ms | 52.3% bf16 MFU | 1624315 tok/s
step 5718/19560 | loss 3.542065 (-0.11z)| norm 0.2729 (-0.35z)| lr 5.01e-04 | 322.54 ms | 52.3% bf16 MFU | 1624374 tok/s
step 5719/19560 | loss 3.559468 (+0.25z)| norm 0.2753 (-0.25z)| lr 5.01e-04 | 323.11 ms | 52.2% bf16 MFU | 1624286 tok/s
step 5720/19560 | loss 3.540909 (-0.15z)| norm 0.2549 (-1.17z)| lr 5.01e-04 | 322.98 ms | 52.3% bf16 MFU | 1624237 tok/s
step 5721/19560 | loss 3.613412 (+1.43z)| norm 0.2724 (-0.37z)| lr 5.01e-04 | 322.54 ms | 52.3% bf16 MFU | 1624299 tok/s
step 5722/19560 | loss 3.542602 (-0.12z)| norm 0.2519 (-1.30z)| lr 5.01e-04 | 323.03 ms | 52.2% bf16 MFU | 1624235 tok/s
step 5723/19560 | loss 3.519557 (-0.63z)| norm 0.2765 (-0.16z)| lr 5.01e-04 | 323.10 ms | 52.2% bf16 MFU | 1624157 tok/s
step 5724/19560 | loss 3.630843 (+1.78z)| norm 0.2517 (-1.28z)| lr 5.01e-04 | 322.68 ms | 52.3% bf16 MFU | 1624188 tok/s
step 5725/19560 | loss 3.533533 (-0.33z)| norm 0.2708 (-0.40z)| lr 5.01e-04 | 322.83 ms | 52.3% bf16 MFU | 1624181 tok/s
step 5726/19560 | loss 3.600836 (+1.11z)| norm 0.2711 (-0.37z)| lr 5.01e-04 | 322.99 ms | 52.3% bf16 MFU | 1624133 tok/s
step 5727/19560 | loss 3.488751 (-1.29z)| norm 0.2806 (+0.07z)| lr 5.01e-04 | 322.08 ms | 52.4% bf16 MFU | 1624317 tok/s
step 5728/19560 | loss 3.541285 (-0.16z)| norm 0.3002 (+0.96z)| lr 5.01e-04 | 322.26 ms | 52.4% bf16 MFU | 1624446 tok/s
step 5729/19560 | loss 3.526868 (-0.46z)| norm 0.2871 (+0.35z)| lr 5.01e-04 | 323.38 ms | 52.2% bf16 MFU | 1624289 tok/s
step 5730/19560 | loss 3.505353 (-0.92z)| norm 0.2612 (-0.85z)| lr 5.01e-04 | 322.31 ms | 52.4% bf16 MFU | 1624408 tok/s
step 5731/19560 | loss 3.506777 (-0.89z)| norm 0.3012 (+1.00z)| lr 5.01e-04 | 323.01 ms | 52.3% bf16 MFU | 1624344 tok/s
step 5732/19560 | loss 3.538265 (-0.21z)| norm 0.2840 (+0.21z)| lr 5.01e-04 | 322.32 ms | 52.4% bf16 MFU | 1624458 tok/s
step 5733/19560 | loss 3.580223 (+0.68z)| norm 0.2533 (-1.23z)| lr 5.01e-04 | 322.74 ms | 52.3% bf16 MFU | 1624460 tok/s
step 5734/19560 | loss 3.456889 (-1.92z)| norm 0.2448 (-1.60z)| lr 5.01e-04 | 323.06 ms | 52.2% bf16 MFU | 1624380 tok/s
step 5735/19560 | loss 3.542108 (-0.11z)| norm 0.2598 (-0.89z)| lr 5.01e-04 | 322.85 ms | 52.3% bf16 MFU | 1624358 tok/s
step 5736/19560 | loss 3.529030 (-0.39z)| norm 0.2576 (-0.99z)| lr 5.01e-04 | 322.55 ms | 52.3% bf16 MFU | 1624414 tok/s
step 5737/19560 | loss 3.520097 (-0.58z)| norm 0.2958 (+0.77z)| lr 5.00e-04 | 322.97 ms | 52.3% bf16 MFU | 1624359 tok/s
step 5738/19560 | loss 3.500521 (-0.99z)| norm 0.3682 (+3.85z)| lr 5.00e-04 | 322.46 ms | 52.3% bf16 MFU | 1624435 tok/s
step 5739/19560 | loss 3.541598 (-0.12z)| norm 0.4009 (+4.75z)| lr 5.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1624433 tok/s
step 5740/19560 | loss 3.570470 (+0.49z)| norm 0.3070 (+1.01z)| lr 5.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1624453 tok/s
step 5741/19560 | loss 3.592205 (+0.95z)| norm 0.2825 (+0.04z)| lr 5.00e-04 | 322.56 ms | 52.3% bf16 MFU | 1624501 tok/s
step 5742/19560 | loss 3.521357 (-0.57z)| norm 0.3330 (+1.99z)| lr 5.00e-04 | 322.90 ms | 52.3% bf16 MFU | 1624461 tok/s
step 5743/19560 | loss 3.503341 (-0.95z)| norm 0.2634 (-0.71z)| lr 5.00e-04 | 323.28 ms | 52.2% bf16 MFU | 1624327 tok/s
step 5744/19560 | loss 3.606695 (+1.25z)| norm 0.3370 (+2.10z)| lr 5.00e-04 | 323.17 ms | 52.2% bf16 MFU | 1624228 tok/s
step 5745/19560 | loss 3.570370 (+0.48z)| norm 0.2714 (-0.40z)| lr 5.00e-04 | 322.51 ms | 52.3% bf16 MFU | 1624298 tok/s
step 5746/19560 | loss 3.526376 (-0.46z)| norm 0.2838 (+0.08z)| lr 5.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1624318 tok/s
step 5747/19560 | loss 3.511241 (-0.83z)| norm 0.2631 (-0.72z)| lr 5.00e-04 | 323.42 ms | 52.2% bf16 MFU | 1624156 tok/s
step 5748/19560 | loss 3.535874 (-0.24z)| norm 0.2794 (-0.05z)| lr 5.00e-04 | 322.77 ms | 52.3% bf16 MFU | 1624164 tok/s
step 5749/19560 | loss 3.492636 (-1.26z)| norm 0.2629 (-0.71z)| lr 5.00e-04 | 322.88 ms | 52.3% bf16 MFU | 1624146 tok/s
step 5750/19560 | loss 3.537158 (-0.19z)| norm 0.2914 (+0.47z)| lr 5.00e-04 | 323.33 ms | 52.2% bf16 MFU | 1624015 tok/s
val loss 3.524983
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2774/10042 = 0.276240
step 5751/19560 | loss 3.504372 (-0.97z)| norm 0.2621 (-0.73z)| lr 5.00e-04 | 321.81 ms | 52.4% bf16 MFU | 1624273 tok/s
step 5752/19560 | loss 3.593611 (+1.16z)| norm 0.2688 (-0.45z)| lr 5.00e-04 | 322.20 ms | 52.4% bf16 MFU | 1624421 tok/s
step 5753/19560 | loss 3.508547 (-0.86z)| norm 0.2773 (-0.10z)| lr 5.00e-04 | 322.62 ms | 52.3% bf16 MFU | 1624454 tok/s
step 5754/19560 | loss 3.498852 (-1.09z)| norm 0.2655 (-0.58z)| lr 5.00e-04 | 322.99 ms | 52.3% bf16 MFU | 1624392 tok/s
step 5755/19560 | loss 3.548357 (+0.07z)| norm 0.2906 (+0.44z)| lr 5.00e-04 | 322.38 ms | 52.4% bf16 MFU | 1624487 tok/s
step 5756/19560 | loss 3.542691 (-0.05z)| norm 0.2730 (-0.28z)| lr 5.00e-04 | 322.42 ms | 52.3% bf16 MFU | 1624568 tok/s
step 5757/19560 | loss 3.468046 (-1.82z)| norm 0.2410 (-1.57z)| lr 5.00e-04 | 323.09 ms | 52.2% bf16 MFU | 1624476 tok/s
step 5758/19560 | loss 3.619132 (+1.77z)| norm 0.2551 (-0.99z)| lr 5.00e-04 | 322.87 ms | 52.3% bf16 MFU | 1624445 tok/s
step 5759/19560 | loss 3.475362 (-1.61z)| norm 0.2864 (+0.28z)| lr 5.00e-04 | 322.96 ms | 52.3% bf16 MFU | 1624391 tok/s
step 5760/19560 | loss 3.506507 (-0.87z)| norm 0.3325 (+2.08z)| lr 5.00e-04 | 323.44 ms | 52.2% bf16 MFU | 1624221 tok/s
step 5761/19560 | loss 3.561348 (+0.41z)| norm 0.2810 (+0.04z)| lr 5.00e-04 | 322.20 ms | 52.4% bf16 MFU | 1624371 tok/s
step 5762/19560 | loss 3.497765 (-1.06z)| norm 0.2757 (-0.17z)| lr 5.00e-04 | 322.62 ms | 52.3% bf16 MFU | 1624408 tok/s
step 5763/19560 | loss 3.512290 (-0.73z)| norm 0.2558 (-0.96z)| lr 5.00e-04 | 323.53 ms | 52.2% bf16 MFU | 1624215 tok/s
step 5764/19560 | loss 3.567520 (+0.56z)| norm 0.2857 (+0.23z)| lr 4.99e-04 | 322.51 ms | 52.3% bf16 MFU | 1624286 tok/s
step 5765/19560 | loss 3.525950 (-0.41z)| norm 0.2830 (+0.14z)| lr 4.99e-04 | 324.77 ms | 52.0% bf16 MFU | 1623790 tok/s
step 5766/19560 | loss 3.565522 (+0.55z)| norm 0.3030 (+0.94z)| lr 4.99e-04 | 322.34 ms | 52.4% bf16 MFU | 1623924 tok/s
step 5767/19560 | loss 3.446872 (-2.25z)| norm 0.2723 (-0.30z)| lr 4.99e-04 | 323.04 ms | 52.2% bf16 MFU | 1623878 tok/s
step 5768/19560 | loss 3.449064 (-2.14z)| norm 0.2787 (-0.04z)| lr 4.99e-04 | 322.82 ms | 52.3% bf16 MFU | 1623888 tok/s
step 5769/19560 | loss 3.555623 (+0.33z)| norm 0.2987 (+0.76z)| lr 4.99e-04 | 322.75 ms | 52.3% bf16 MFU | 1623916 tok/s
step 5770/19560 | loss 3.609354 (+1.55z)| norm 0.2685 (-0.46z)| lr 4.99e-04 | 322.32 ms | 52.4% bf16 MFU | 1624051 tok/s
step 5771/19560 | loss 3.525099 (-0.40z)| norm 0.2603 (-0.78z)| lr 4.99e-04 | 322.76 ms | 52.3% bf16 MFU | 1624067 tok/s
step 5772/19560 | loss 3.524408 (-0.41z)| norm 0.2685 (-0.46z)| lr 4.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1624143 tok/s
step 5773/19560 | loss 3.519845 (-0.52z)| norm 0.2680 (-0.48z)| lr 4.99e-04 | 322.67 ms | 52.3% bf16 MFU | 1624179 tok/s
step 5774/19560 | loss 3.505251 (-0.84z)| norm 0.2433 (-1.47z)| lr 4.99e-04 | 322.58 ms | 52.3% bf16 MFU | 1624234 tok/s
step 5775/19560 | loss 3.446459 (-2.14z)| norm 0.2717 (-0.31z)| lr 4.99e-04 | 322.51 ms | 52.3% bf16 MFU | 1624304 tok/s
step 5776/19560 | loss 3.542271 (+0.04z)| norm 0.2524 (-1.08z)| lr 4.99e-04 | 322.75 ms | 52.3% bf16 MFU | 1624312 tok/s
step 5777/19560 | loss 3.635333 (+2.10z)| norm 0.2613 (-0.71z)| lr 4.99e-04 | 323.06 ms | 52.2% bf16 MFU | 1624240 tok/s
step 5778/19560 | loss 3.564378 (+0.51z)| norm 0.2891 (+0.41z)| lr 4.99e-04 | 322.32 ms | 52.4% bf16 MFU | 1624358 tok/s
step 5779/19560 | loss 3.529796 (-0.26z)| norm 0.2486 (-1.21z)| lr 4.99e-04 | 322.84 ms | 52.3% bf16 MFU | 1624340 tok/s
step 5780/19560 | loss 3.603652 (+1.37z)| norm 0.2607 (-0.71z)| lr 4.99e-04 | 322.96 ms | 52.3% bf16 MFU | 1624293 tok/s
step 5781/19560 | loss 3.484716 (-1.26z)| norm 0.2695 (-0.36z)| lr 4.99e-04 | 322.63 ms | 52.3% bf16 MFU | 1624330 tok/s
step 5782/19560 | loss 3.501682 (-0.90z)| norm 0.2636 (-0.59z)| lr 4.99e-04 | 323.37 ms | 52.2% bf16 MFU | 1624180 tok/s
step 5783/19560 | loss 3.504729 (-0.82z)| norm 0.2714 (-0.28z)| lr 4.99e-04 | 322.54 ms | 52.3% bf16 MFU | 1624247 tok/s
step 5784/19560 | loss 3.518870 (-0.50z)| norm 0.2624 (-0.65z)| lr 4.99e-04 | 322.38 ms | 52.4% bf16 MFU | 1624350 tok/s
step 5785/19560 | loss 3.523270 (-0.40z)| norm 0.2826 (+0.17z)| lr 4.99e-04 | 322.78 ms | 52.3% bf16 MFU | 1624347 tok/s
step 5786/19560 | loss 3.521429 (-0.45z)| norm 0.2519 (-1.08z)| lr 4.99e-04 | 322.67 ms | 52.3% bf16 MFU | 1624372 tok/s
step 5787/19560 | loss 3.513407 (-0.63z)| norm 0.2683 (-0.42z)| lr 4.99e-04 | 323.20 ms | 52.2% bf16 MFU | 1624262 tok/s
step 5788/19560 | loss 3.508074 (-0.74z)| norm 0.3102 (+1.27z)| lr 4.99e-04 | 322.41 ms | 52.3% bf16 MFU | 1624358 tok/s
step 5789/19560 | loss 3.519129 (-0.48z)| norm 0.2982 (+0.77z)| lr 4.99e-04 | 322.73 ms | 52.3% bf16 MFU | 1624367 tok/s
step 5790/19560 | loss 3.535779 (-0.12z)| norm 0.2665 (-0.52z)| lr 4.99e-04 | 322.51 ms | 52.3% bf16 MFU | 1624432 tok/s
step 5791/19560 | loss 3.602765 (+1.38z)| norm 0.2713 (-0.33z)| lr 4.98e-04 | 322.76 ms | 52.3% bf16 MFU | 1624430 tok/s
step 5792/19560 | loss 3.509217 (-0.72z)| norm 0.2531 (-1.06z)| lr 4.98e-04 | 322.73 ms | 52.3% bf16 MFU | 1624435 tok/s
step 5793/19560 | loss 3.553810 (+0.29z)| norm 0.2820 (+0.11z)| lr 4.98e-04 | 322.72 ms | 52.3% bf16 MFU | 1624444 tok/s
step 5794/19560 | loss 3.528567 (-0.29z)| norm 0.2836 (+0.18z)| lr 4.98e-04 | 322.98 ms | 52.3% bf16 MFU | 1624387 tok/s
step 5795/19560 | loss 3.512312 (-0.66z)| norm 0.2715 (-0.30z)| lr 4.98e-04 | 323.15 ms | 52.2% bf16 MFU | 1624289 tok/s
step 5796/19560 | loss 3.555901 (+0.32z)| norm 0.2643 (-0.59z)| lr 4.98e-04 | 322.60 ms | 52.3% bf16 MFU | 1624335 tok/s
step 5797/19560 | loss 3.646858 (+2.32z)| norm 0.2918 (+0.54z)| lr 4.98e-04 | 322.49 ms | 52.3% bf16 MFU | 1624405 tok/s
step 5798/19560 | loss 3.569967 (+0.61z)| norm 0.3086 (+1.25z)| lr 4.98e-04 | 322.97 ms | 52.3% bf16 MFU | 1624352 tok/s
step 5799/19560 | loss 3.542630 (-0.00z)| norm 0.3017 (+0.95z)| lr 4.98e-04 | 322.39 ms | 52.4% bf16 MFU | 1624448 tok/s
step 5800/19560 | loss 3.526597 (-0.35z)| norm 0.3178 (+1.61z)| lr 4.98e-04 | 322.86 ms | 52.3% bf16 MFU | 1624420 tok/s
step 5801/19560 | loss 3.509332 (-0.73z)| norm 0.2803 (+0.05z)| lr 4.98e-04 | 323.12 ms | 52.2% bf16 MFU | 1624327 tok/s
step 5802/19560 | loss 3.523178 (-0.42z)| norm 0.2850 (+0.24z)| lr 4.98e-04 | 322.63 ms | 52.3% bf16 MFU | 1624364 tok/s
step 5803/19560 | loss 3.507357 (-0.76z)| norm 0.2645 (-0.60z)| lr 4.98e-04 | 322.96 ms | 52.3% bf16 MFU | 1624316 tok/s
step 5804/19560 | loss 3.554576 (+0.29z)| norm 0.2910 (+0.50z)| lr 4.98e-04 | 322.41 ms | 52.3% bf16 MFU | 1624407 tok/s
step 5805/19560 | loss 3.517968 (-0.52z)| norm 0.2997 (+0.85z)| lr 4.98e-04 | 323.17 ms | 52.2% bf16 MFU | 1624303 tok/s
step 5806/19560 | loss 3.567641 (+0.58z)| norm 0.2484 (-1.27z)| lr 4.98e-04 | 322.60 ms | 52.3% bf16 MFU | 1624348 tok/s
step 5807/19560 | loss 3.520411 (-0.46z)| norm 0.2670 (-0.50z)| lr 4.98e-04 | 322.55 ms | 52.3% bf16 MFU | 1624402 tok/s
step 5808/19560 | loss 3.510997 (-0.66z)| norm 0.2576 (-0.87z)| lr 4.98e-04 | 322.88 ms | 52.3% bf16 MFU | 1624372 tok/s
step 5809/19560 | loss 3.548375 (+0.16z)| norm 0.2664 (-0.51z)| lr 4.98e-04 | 322.49 ms | 52.3% bf16 MFU | 1624442 tok/s
step 5810/19560 | loss 3.517336 (-0.53z)| norm 0.2486 (-1.22z)| lr 4.98e-04 | 322.82 ms | 52.3% bf16 MFU | 1624423 tok/s
step 5811/19560 | loss 3.750514 (+4.35z)| norm 0.3091 (+1.27z)| lr 4.98e-04 | 323.14 ms | 52.2% bf16 MFU | 1624327 tok/s
step 5812/19560 | loss 3.567928 (+0.52z)| norm 0.3126 (+1.39z)| lr 4.98e-04 | 322.95 ms | 52.3% bf16 MFU | 1624281 tok/s
step 5813/19560 | loss 3.535763 (-0.14z)| norm 0.2986 (+0.84z)| lr 4.98e-04 | 322.52 ms | 52.3% bf16 MFU | 1624346 tok/s
step 5814/19560 | loss 3.552363 (+0.21z)| norm 0.2797 (+0.06z)| lr 4.98e-04 | 322.99 ms | 52.3% bf16 MFU | 1624292 tok/s
step 5815/19560 | loss 3.559419 (+0.37z)| norm 0.3148 (+1.49z)| lr 4.98e-04 | 322.50 ms | 52.3% bf16 MFU | 1624362 tok/s
step 5816/19560 | loss 3.488314 (-1.17z)| norm 0.2657 (-0.52z)| lr 4.98e-04 | 323.08 ms | 52.2% bf16 MFU | 1624282 tok/s
step 5817/19560 | loss 3.564167 (+0.52z)| norm 0.3141 (+1.48z)| lr 4.97e-04 | 322.49 ms | 52.3% bf16 MFU | 1624356 tok/s
step 5818/19560 | loss 3.593755 (+1.21z)| norm 0.2524 (-1.05z)| lr 4.97e-04 | 322.49 ms | 52.3% bf16 MFU | 1624426 tok/s
step 5819/19560 | loss 3.492993 (-1.07z)| norm 0.2662 (-0.48z)| lr 4.97e-04 | 323.27 ms | 52.2% bf16 MFU | 1624296 tok/s
step 5820/19560 | loss 3.563544 (+0.52z)| norm 0.2594 (-0.75z)| lr 4.97e-04 | 322.77 ms | 52.3% bf16 MFU | 1624298 tok/s
step 5821/19560 | loss 3.493435 (-1.05z)| norm 0.2912 (+0.55z)| lr 4.97e-04 | 323.06 ms | 52.2% bf16 MFU | 1624227 tok/s
step 5822/19560 | loss 3.542490 (+0.05z)| norm 0.2600 (-0.75z)| lr 4.97e-04 | 322.66 ms | 52.3% bf16 MFU | 1624260 tok/s
step 5823/19560 | loss 3.455739 (-1.88z)| norm 0.2830 (+0.20z)| lr 4.97e-04 | 322.86 ms | 52.3% bf16 MFU | 1624241 tok/s
step 5824/19560 | loss 3.500601 (-0.87z)| norm 0.2826 (+0.18z)| lr 4.97e-04 | 323.18 ms | 52.2% bf16 MFU | 1624143 tok/s
step 5825/19560 | loss 3.477642 (-1.35z)| norm 0.2815 (+0.13z)| lr 4.97e-04 | 323.05 ms | 52.2% bf16 MFU | 1624084 tok/s
step 5826/19560 | loss 3.573573 (+0.77z)| norm 0.2862 (+0.32z)| lr 4.97e-04 | 322.57 ms | 52.3% bf16 MFU | 1624146 tok/s
step 5827/19560 | loss 3.527065 (-0.26z)| norm 0.3023 (+0.97z)| lr 4.97e-04 | 322.18 ms | 52.4% bf16 MFU | 1624305 tok/s
step 5828/19560 | loss 3.569427 (+0.69z)| norm 0.2853 (+0.25z)| lr 4.97e-04 | 322.52 ms | 52.3% bf16 MFU | 1624370 tok/s
step 5829/19560 | loss 3.525578 (-0.28z)| norm 0.2663 (-0.55z)| lr 4.97e-04 | 323.41 ms | 52.2% bf16 MFU | 1624209 tok/s
step 5830/19560 | loss 3.523003 (-0.33z)| norm 0.2646 (-0.61z)| lr 4.97e-04 | 323.01 ms | 52.3% bf16 MFU | 1624155 tok/s
step 5831/19560 | loss 3.547089 (+0.21z)| norm 0.2677 (-0.48z)| lr 4.97e-04 | 322.68 ms | 52.3% bf16 MFU | 1624186 tok/s
step 5832/19560 | loss 3.532481 (-0.12z)| norm 0.2644 (-0.62z)| lr 4.97e-04 | 323.02 ms | 52.2% bf16 MFU | 1624132 tok/s
step 5833/19560 | loss 3.515849 (-0.48z)| norm 0.2675 (-0.49z)| lr 4.97e-04 | 322.70 ms | 52.3% bf16 MFU | 1624158 tok/s
step 5834/19560 | loss 3.510398 (-0.59z)| norm 0.2859 (+0.28z)| lr 4.97e-04 | 323.19 ms | 52.2% bf16 MFU | 1624063 tok/s
step 5835/19560 | loss 3.535010 (-0.04z)| norm 0.2742 (-0.21z)| lr 4.97e-04 | 322.93 ms | 52.3% bf16 MFU | 1624037 tok/s
step 5836/19560 | loss 3.543676 (+0.16z)| norm 0.2871 (+0.34z)| lr 4.97e-04 | 323.55 ms | 52.2% bf16 MFU | 1623857 tok/s
step 5837/19560 | loss 3.583549 (+1.06z)| norm 0.2767 (-0.10z)| lr 4.97e-04 | 323.25 ms | 52.2% bf16 MFU | 1623759 tok/s
step 5838/19560 | loss 3.519485 (-0.38z)| norm 0.2913 (+0.53z)| lr 4.97e-04 | 322.53 ms | 52.3% bf16 MFU | 1623850 tok/s
step 5839/19560 | loss 3.501975 (-0.79z)| norm 0.2810 (+0.10z)| lr 4.97e-04 | 323.45 ms | 52.2% bf16 MFU | 1623703 tok/s
step 5840/19560 | loss 3.496423 (-0.92z)| norm 0.2631 (-0.67z)| lr 4.97e-04 | 323.30 ms | 52.2% bf16 MFU | 1623602 tok/s
step 5841/19560 | loss 3.593250 (+1.29z)| norm 0.3205 (+1.77z)| lr 4.97e-04 | 322.31 ms | 52.4% bf16 MFU | 1623756 tok/s
step 5842/19560 | loss 3.494437 (-0.96z)| norm 0.3380 (+2.44z)| lr 4.97e-04 | 322.58 ms | 52.3% bf16 MFU | 1623832 tok/s
step 5843/19560 | loss 3.491259 (-1.02z)| norm 0.3351 (+2.26z)| lr 4.97e-04 | 323.14 ms | 52.2% bf16 MFU | 1623765 tok/s
step 5844/19560 | loss 3.536764 (+0.04z)| norm 0.2946 (+0.59z)| lr 4.96e-04 | 323.07 ms | 52.2% bf16 MFU | 1623717 tok/s
step 5845/19560 | loss 3.531086 (-0.10z)| norm 0.2746 (-0.22z)| lr 4.96e-04 | 323.27 ms | 52.2% bf16 MFU | 1623622 tok/s
step 5846/19560 | loss 3.523231 (-0.28z)| norm 0.3069 (+1.08z)| lr 4.96e-04 | 322.57 ms | 52.3% bf16 MFU | 1623709 tok/s
step 5847/19560 | loss 3.472752 (-1.42z)| norm 0.2658 (-0.58z)| lr 4.96e-04 | 322.82 ms | 52.3% bf16 MFU | 1623729 tok/s
step 5848/19560 | loss 3.479836 (-1.24z)| norm 0.2743 (-0.25z)| lr 4.96e-04 | 322.50 ms | 52.3% bf16 MFU | 1623827 tok/s
step 5849/19560 | loss 3.475021 (-1.33z)| norm 0.2645 (-0.64z)| lr 4.96e-04 | 322.82 ms | 52.3% bf16 MFU | 1623841 tok/s
step 5850/19560 | loss 3.534080 (+0.03z)| norm 0.2608 (-0.80z)| lr 4.96e-04 | 323.25 ms | 52.2% bf16 MFU | 1623746 tok/s
step 5851/19560 | loss 3.531141 (-0.04z)| norm 0.2549 (-1.03z)| lr 4.96e-04 | 322.88 ms | 52.3% bf16 MFU | 1623747 tok/s
step 5852/19560 | loss 3.506083 (-0.61z)| norm 0.2366 (-1.75z)| lr 4.96e-04 | 322.91 ms | 52.3% bf16 MFU | 1623742 tok/s
step 5853/19560 | loss 3.497403 (-0.80z)| norm 0.2427 (-1.49z)| lr 4.96e-04 | 322.77 ms | 52.3% bf16 MFU | 1623770 tok/s
step 5854/19560 | loss 3.566204 (+0.82z)| norm 0.2635 (-0.65z)| lr 4.96e-04 | 322.89 ms | 52.3% bf16 MFU | 1623768 tok/s
step 5855/19560 | loss 3.525188 (-0.16z)| norm 0.2755 (-0.17z)| lr 4.96e-04 | 323.29 ms | 52.2% bf16 MFU | 1623665 tok/s
step 5856/19560 | loss 3.483101 (-1.13z)| norm 0.2475 (-1.27z)| lr 4.96e-04 | 322.38 ms | 52.4% bf16 MFU | 1623797 tok/s
step 5857/19560 | loss 3.568392 (+0.86z)| norm 0.2714 (-0.31z)| lr 4.96e-04 | 322.75 ms | 52.3% bf16 MFU | 1623830 tok/s
step 5858/19560 | loss 3.535365 (+0.08z)| norm 0.2740 (-0.21z)| lr 4.96e-04 | 323.05 ms | 52.2% bf16 MFU | 1623784 tok/s
step 5859/19560 | loss 3.494559 (-0.87z)| norm 0.2894 (+0.40z)| lr 4.96e-04 | 322.78 ms | 52.3% bf16 MFU | 1623809 tok/s
step 5860/19560 | loss 3.540977 (+0.21z)| norm 0.2823 (+0.12z)| lr 4.96e-04 | 323.00 ms | 52.3% bf16 MFU | 1623776 tok/s
step 5861/19560 | loss 3.581802 (+1.17z)| norm 0.2570 (-0.89z)| lr 4.96e-04 | 322.39 ms | 52.4% bf16 MFU | 1623901 tok/s
step 5862/19560 | loss 3.559906 (+0.65z)| norm 0.2702 (-0.37z)| lr 4.96e-04 | 322.68 ms | 52.3% bf16 MFU | 1623947 tok/s
step 5863/19560 | loss 3.524808 (-0.18z)| norm 0.2700 (-0.38z)| lr 4.96e-04 | 323.00 ms | 52.3% bf16 MFU | 1623908 tok/s
step 5864/19560 | loss 3.480418 (-1.22z)| norm 0.2894 (+0.39z)| lr 4.96e-04 | 322.47 ms | 52.3% bf16 MFU | 1624004 tok/s
step 5865/19560 | loss 3.522665 (-0.22z)| norm 0.2728 (-0.28z)| lr 4.96e-04 | 322.67 ms | 52.3% bf16 MFU | 1624047 tok/s
step 5866/19560 | loss 3.528188 (-0.10z)| norm 0.2695 (-0.40z)| lr 4.96e-04 | 322.70 ms | 52.3% bf16 MFU | 1624079 tok/s
step 5867/19560 | loss 3.490055 (-0.98z)| norm 0.2967 (+0.90z)| lr 4.96e-04 | 322.75 ms | 52.3% bf16 MFU | 1624096 tok/s
step 5868/19560 | loss 3.560101 (+0.66z)| norm 0.2595 (-0.88z)| lr 4.96e-04 | 322.70 ms | 52.3% bf16 MFU | 1624127 tok/s
step 5869/19560 | loss 3.525265 (-0.14z)| norm 0.2771 (-0.03z)| lr 4.96e-04 | 322.35 ms | 52.4% bf16 MFU | 1624243 tok/s
step 5870/19560 | loss 3.616351 (+1.97z)| norm 0.2956 (+0.91z)| lr 4.95e-04 | 322.21 ms | 52.4% bf16 MFU | 1624388 tok/s
step 5871/19560 | loss 3.532564 (+0.01z)| norm 0.2892 (+0.58z)| lr 4.95e-04 | 322.60 ms | 52.3% bf16 MFU | 1624428 tok/s
step 5872/19560 | loss 3.497577 (-0.80z)| norm 0.2901 (+0.66z)| lr 4.95e-04 | 322.43 ms | 52.3% bf16 MFU | 1624508 tok/s
step 5873/19560 | loss 3.494828 (-0.85z)| norm 0.2967 (+0.99z)| lr 4.95e-04 | 322.91 ms | 52.3% bf16 MFU | 1624464 tok/s
step 5874/19560 | loss 3.533055 (+0.05z)| norm 0.3096 (+1.62z)| lr 4.95e-04 | 322.77 ms | 52.3% bf16 MFU | 1624456 tok/s
step 5875/19560 | loss 3.550386 (+0.45z)| norm 0.2777 (+0.00z)| lr 4.95e-04 | 322.92 ms | 52.3% bf16 MFU | 1624413 tok/s
step 5876/19560 | loss 3.506880 (-0.57z)| norm 0.2582 (-0.97z)| lr 4.95e-04 | 322.77 ms | 52.3% bf16 MFU | 1624410 tok/s
step 5877/19560 | loss 3.593691 (+1.45z)| norm 0.2668 (-0.54z)| lr 4.95e-04 | 322.53 ms | 52.3% bf16 MFU | 1624466 tok/s
step 5878/19560 | loss 3.497147 (-0.80z)| norm 0.2829 (+0.27z)| lr 4.95e-04 | 322.45 ms | 52.3% bf16 MFU | 1624540 tok/s
step 5879/19560 | loss 3.577707 (+1.07z)| norm 0.2563 (-1.07z)| lr 4.95e-04 | 322.33 ms | 52.4% bf16 MFU | 1624641 tok/s
step 5880/19560 | loss 3.505780 (-0.60z)| norm 0.2772 (-0.01z)| lr 4.95e-04 | 322.56 ms | 52.3% bf16 MFU | 1624678 tok/s
step 5881/19560 | loss 3.543015 (+0.27z)| norm 0.2669 (-0.53z)| lr 4.95e-04 | 322.61 ms | 52.3% bf16 MFU | 1624702 tok/s
step 5882/19560 | loss 3.488867 (-1.00z)| norm 0.2657 (-0.59z)| lr 4.95e-04 | 322.24 ms | 52.4% bf16 MFU | 1624818 tok/s
step 5883/19560 | loss 3.553995 (+0.53z)| norm 0.2851 (+0.39z)| lr 4.95e-04 | 322.34 ms | 52.4% bf16 MFU | 1624904 tok/s
step 5884/19560 | loss 3.478456 (-1.23z)| norm 0.2787 (+0.06z)| lr 4.95e-04 | 322.40 ms | 52.3% bf16 MFU | 1624968 tok/s
step 5885/19560 | loss 3.480732 (-1.18z)| norm 0.2560 (-1.09z)| lr 4.95e-04 | 322.73 ms | 52.3% bf16 MFU | 1624947 tok/s
step 5886/19560 | loss 3.531143 (+0.02z)| norm 0.2831 (+0.28z)| lr 4.95e-04 | 322.75 ms | 52.3% bf16 MFU | 1624921 tok/s
step 5887/19560 | loss 3.535535 (+0.11z)| norm 0.2652 (-0.63z)| lr 4.95e-04 | 322.63 ms | 52.3% bf16 MFU | 1624928 tok/s
step 5888/19560 | loss 3.495725 (-0.84z)| norm 0.2477 (-1.53z)| lr 4.95e-04 | 322.25 ms | 52.4% bf16 MFU | 1625030 tok/s
step 5889/19560 | loss 3.535809 (+0.12z)| norm 0.2721 (-0.25z)| lr 4.95e-04 | 322.36 ms | 52.4% bf16 MFU | 1625099 tok/s
step 5890/19560 | loss 3.496213 (-0.83z)| norm 0.2639 (-0.67z)| lr 4.95e-04 | 323.02 ms | 52.2% bf16 MFU | 1624998 tok/s
step 5891/19560 | loss 3.472025 (-1.39z)| norm 0.2871 (+0.53z)| lr 4.95e-04 | 322.40 ms | 52.3% bf16 MFU | 1625058 tok/s
step 5892/19560 | loss 3.535020 (+0.12z)| norm 0.2875 (+0.55z)| lr 4.95e-04 | 322.65 ms | 52.3% bf16 MFU | 1625051 tok/s
step 5893/19560 | loss 3.541480 (+0.27z)| norm 0.2479 (-1.50z)| lr 4.95e-04 | 322.53 ms | 52.3% bf16 MFU | 1625075 tok/s
step 5894/19560 | loss 3.489318 (-0.96z)| norm 0.3013 (+1.28z)| lr 4.95e-04 | 323.10 ms | 52.2% bf16 MFU | 1624955 tok/s
step 5895/19560 | loss 3.538837 (+0.21z)| norm 0.3064 (+1.52z)| lr 4.95e-04 | 322.63 ms | 52.3% bf16 MFU | 1624959 tok/s
step 5896/19560 | loss 3.451047 (-1.92z)| norm 0.2830 (+0.31z)| lr 4.95e-04 | 322.45 ms | 52.3% bf16 MFU | 1625009 tok/s
step 5897/19560 | loss 3.491419 (-0.93z)| norm 0.2852 (+0.43z)| lr 4.94e-04 | 322.18 ms | 52.4% bf16 MFU | 1625124 tok/s
step 5898/19560 | loss 3.504430 (-0.60z)| norm 0.2949 (+0.92z)| lr 4.94e-04 | 322.81 ms | 52.3% bf16 MFU | 1625074 tok/s
step 5899/19560 | loss 3.510805 (-0.44z)| norm 0.2637 (-0.70z)| lr 4.94e-04 | 322.48 ms | 52.3% bf16 MFU | 1625110 tok/s
step 5900/19560 | loss 3.519631 (-0.23z)| norm 0.2939 (+0.85z)| lr 4.94e-04 | 322.79 ms | 52.3% bf16 MFU | 1625065 tok/s
step 5901/19560 | loss 3.483026 (-1.11z)| norm 0.2848 (+0.38z)| lr 4.94e-04 | 322.65 ms | 52.3% bf16 MFU | 1625060 tok/s
step 5902/19560 | loss 3.577681 (+1.18z)| norm 0.2474 (-1.56z)| lr 4.94e-04 | 322.11 ms | 52.4% bf16 MFU | 1625190 tok/s
step 5903/19560 | loss 3.460845 (-1.67z)| norm 0.3022 (+1.26z)| lr 4.94e-04 | 323.21 ms | 52.2% bf16 MFU | 1625036 tok/s
step 5904/19560 | loss 3.565181 (+0.87z)| norm 0.3353 (+2.85z)| lr 4.94e-04 | 322.61 ms | 52.3% bf16 MFU | 1625041 tok/s
step 5905/19560 | loss 3.517899 (-0.27z)| norm 0.2785 (-0.00z)| lr 4.94e-04 | 322.92 ms | 52.3% bf16 MFU | 1624968 tok/s
step 5906/19560 | loss 3.554256 (+0.65z)| norm 0.2774 (-0.05z)| lr 4.94e-04 | 322.46 ms | 52.3% bf16 MFU | 1625015 tok/s
step 5907/19560 | loss 3.561752 (+0.83z)| norm 0.2799 (+0.06z)| lr 4.94e-04 | 322.63 ms | 52.3% bf16 MFU | 1625016 tok/s
step 5908/19560 | loss 3.537766 (+0.24z)| norm 0.2623 (-0.83z)| lr 4.94e-04 | 322.21 ms | 52.4% bf16 MFU | 1625122 tok/s
step 5909/19560 | loss 3.542649 (+0.36z)| norm 0.2522 (-1.33z)| lr 4.94e-04 | 322.89 ms | 52.3% bf16 MFU | 1625053 tok/s
step 5910/19560 | loss 3.506055 (-0.58z)| norm 0.2608 (-0.89z)| lr 4.94e-04 | 322.88 ms | 52.3% bf16 MFU | 1624991 tok/s
step 5911/19560 | loss 3.534041 (+0.13z)| norm 0.2490 (-1.47z)| lr 4.94e-04 | 322.28 ms | 52.4% bf16 MFU | 1625081 tok/s
step 5912/19560 | loss 3.515672 (-0.34z)| norm 0.2457 (-1.61z)| lr 4.94e-04 | 322.93 ms | 52.3% bf16 MFU | 1625005 tok/s
step 5913/19560 | loss 3.483792 (-1.14z)| norm 0.2475 (-1.50z)| lr 4.94e-04 | 322.23 ms | 52.4% bf16 MFU | 1625108 tok/s
step 5914/19560 | loss 3.512931 (-0.40z)| norm 0.2617 (-0.81z)| lr 4.94e-04 | 322.70 ms | 52.3% bf16 MFU | 1625087 tok/s
step 5915/19560 | loss 3.471915 (-1.42z)| norm 0.2728 (-0.26z)| lr 4.94e-04 | 323.17 ms | 52.2% bf16 MFU | 1624949 tok/s
step 5916/19560 | loss 3.464450 (-1.58z)| norm 0.2625 (-0.76z)| lr 4.94e-04 | 323.15 ms | 52.2% bf16 MFU | 1624824 tok/s
step 5917/19560 | loss 3.468393 (-1.46z)| norm 0.2579 (-0.97z)| lr 4.94e-04 | 322.95 ms | 52.3% bf16 MFU | 1624755 tok/s
step 5918/19560 | loss 3.541689 (+0.35z)| norm 0.2798 (+0.12z)| lr 4.94e-04 | 322.25 ms | 52.4% bf16 MFU | 1624866 tok/s
step 5919/19560 | loss 3.532879 (+0.15z)| norm 0.2672 (-0.51z)| lr 4.94e-04 | 323.04 ms | 52.2% bf16 MFU | 1624772 tok/s
step 5920/19560 | loss 3.474389 (-1.30z)| norm 0.2862 (+0.42z)| lr 4.94e-04 | 322.12 ms | 52.4% bf16 MFU | 1624914 tok/s
step 5921/19560 | loss 3.572506 (+1.13z)| norm 0.2831 (+0.27z)| lr 4.94e-04 | 322.67 ms | 52.3% bf16 MFU | 1624910 tok/s
step 5922/19560 | loss 3.482916 (-1.08z)| norm 0.2848 (+0.35z)| lr 4.94e-04 | 322.57 ms | 52.3% bf16 MFU | 1624933 tok/s
step 5923/19560 | loss 3.540843 (+0.35z)| norm 0.2769 (-0.04z)| lr 4.93e-04 | 322.84 ms | 52.3% bf16 MFU | 1624887 tok/s
step 5924/19560 | loss 3.568248 (+1.02z)| norm 0.2659 (-0.59z)| lr 4.93e-04 | 322.41 ms | 52.3% bf16 MFU | 1624950 tok/s
step 5925/19560 | loss 3.509773 (-0.41z)| norm 0.2779 (+0.01z)| lr 4.93e-04 | 323.32 ms | 52.2% bf16 MFU | 1624780 tok/s
step 5926/19560 | loss 3.544495 (+0.49z)| norm 0.2885 (+0.56z)| lr 4.93e-04 | 322.22 ms | 52.4% bf16 MFU | 1624897 tok/s
step 5927/19560 | loss 3.464214 (-1.54z)| norm 0.2671 (-0.52z)| lr 4.93e-04 | 322.54 ms | 52.3% bf16 MFU | 1624928 tok/s
step 5928/19560 | loss 3.558372 (+0.84z)| norm 0.2837 (+0.35z)| lr 4.93e-04 | 323.32 ms | 52.2% bf16 MFU | 1624761 tok/s
step 5929/19560 | loss 3.574977 (+1.24z)| norm 0.2934 (+0.84z)| lr 4.93e-04 | 322.78 ms | 52.3% bf16 MFU | 1624737 tok/s
step 5930/19560 | loss 3.519397 (-0.16z)| norm 0.2948 (+0.91z)| lr 4.93e-04 | 322.33 ms | 52.4% bf16 MFU | 1624828 tok/s
step 5931/19560 | loss 3.537176 (+0.28z)| norm 0.2595 (-0.91z)| lr 4.93e-04 | 322.69 ms | 52.3% bf16 MFU | 1624823 tok/s
step 5932/19560 | loss 3.540595 (+0.37z)| norm 0.2714 (-0.28z)| lr 4.93e-04 | 322.86 ms | 52.3% bf16 MFU | 1624776 tok/s
step 5933/19560 | loss 3.524101 (-0.04z)| norm 0.2820 (+0.27z)| lr 4.93e-04 | 322.27 ms | 52.4% bf16 MFU | 1624880 tok/s
step 5934/19560 | loss 3.525645 (+0.00z)| norm 0.2707 (-0.33z)| lr 4.93e-04 | 322.53 ms | 52.3% bf16 MFU | 1624914 tok/s
step 5935/19560 | loss 3.482669 (-1.07z)| norm 0.2849 (+0.40z)| lr 4.93e-04 | 322.98 ms | 52.3% bf16 MFU | 1624833 tok/s
step 5936/19560 | loss 3.533634 (+0.21z)| norm 0.2610 (-0.85z)| lr 4.93e-04 | 322.54 ms | 52.3% bf16 MFU | 1624866 tok/s
step 5937/19560 | loss 3.559695 (+0.86z)| norm 0.2657 (-0.60z)| lr 4.93e-04 | 323.16 ms | 52.2% bf16 MFU | 1624741 tok/s
step 5938/19560 | loss 3.560961 (+0.88z)| norm 0.2654 (-0.63z)| lr 4.93e-04 | 322.50 ms | 52.3% bf16 MFU | 1624790 tok/s
step 5939/19560 | loss 3.414486 (-3.04z)| norm 0.2684 (-0.45z)| lr 4.93e-04 | 322.94 ms | 52.3% bf16 MFU | 1624726 tok/s
step 5940/19560 | loss 3.498421 (-0.68z)| norm 0.2677 (-0.48z)| lr 4.93e-04 | 322.88 ms | 52.3% bf16 MFU | 1624677 tok/s
step 5941/19560 | loss 3.537693 (+0.42z)| norm 0.2763 (-0.01z)| lr 4.93e-04 | 323.46 ms | 52.2% bf16 MFU | 1624487 tok/s
step 5942/19560 | loss 3.550923 (+0.80z)| norm 0.2788 (+0.13z)| lr 4.93e-04 | 322.54 ms | 52.3% bf16 MFU | 1624538 tok/s
step 5943/19560 | loss 3.490911 (-0.88z)| norm 0.2864 (+0.56z)| lr 4.93e-04 | 322.28 ms | 52.4% bf16 MFU | 1624651 tok/s
step 5944/19560 | loss 3.504229 (-0.51z)| norm 0.2676 (-0.48z)| lr 4.93e-04 | 322.37 ms | 52.4% bf16 MFU | 1624736 tok/s
step 5945/19560 | loss 3.490545 (-0.88z)| norm 0.2586 (-0.96z)| lr 4.93e-04 | 322.50 ms | 52.3% bf16 MFU | 1624784 tok/s
step 5946/19560 | loss 3.512280 (-0.25z)| norm 0.2784 (+0.13z)| lr 4.93e-04 | 322.58 ms | 52.3% bf16 MFU | 1624809 tok/s
step 5947/19560 | loss 3.508130 (-0.38z)| norm 0.2790 (+0.16z)| lr 4.93e-04 | 322.28 ms | 52.4% bf16 MFU | 1624909 tok/s
step 5948/19560 | loss 3.535526 (+0.42z)| norm 0.2764 (+0.01z)| lr 4.93e-04 | 322.75 ms | 52.3% bf16 MFU | 1624885 tok/s
step 5949/19560 | loss 3.541368 (+0.58z)| norm 0.2755 (-0.03z)| lr 4.92e-04 | 322.38 ms | 52.4% bf16 MFU | 1624956 tok/s
step 5950/19560 | loss 3.597895 (+2.17z)| norm 0.2622 (-0.79z)| lr 4.92e-04 | 322.48 ms | 52.3% bf16 MFU | 1624997 tok/s
step 5951/19560 | loss 3.643533 (+3.33z)| norm 0.2652 (-0.61z)| lr 4.92e-04 | 322.74 ms | 52.3% bf16 MFU | 1624973 tok/s
step 5952/19560 | loss 3.514839 (-0.23z)| norm 0.2851 (+0.52z)| lr 4.92e-04 | 322.55 ms | 52.3% bf16 MFU | 1624996 tok/s
step 5953/19560 | loss 3.517778 (-0.16z)| norm 0.2886 (+0.71z)| lr 4.92e-04 | 322.64 ms | 52.3% bf16 MFU | 1624996 tok/s
step 5954/19560 | loss 3.528296 (+0.14z)| norm 0.2719 (-0.23z)| lr 4.92e-04 | 322.43 ms | 52.3% bf16 MFU | 1625049 tok/s
step 5955/19560 | loss 3.490710 (-0.90z)| norm 0.2917 (+0.90z)| lr 4.92e-04 | 322.71 ms | 52.3% bf16 MFU | 1625028 tok/s
step 5956/19560 | loss 3.502033 (-0.57z)| norm 0.2782 (+0.14z)| lr 4.92e-04 | 322.88 ms | 52.3% bf16 MFU | 1624967 tok/s
step 5957/19560 | loss 3.576561 (+1.50z)| norm 0.2643 (-0.65z)| lr 4.92e-04 | 323.02 ms | 52.2% bf16 MFU | 1624872 tok/s
step 5958/19560 | loss 3.521957 (-0.02z)| norm 0.2855 (+0.54z)| lr 4.92e-04 | 322.88 ms | 52.3% bf16 MFU | 1624818 tok/s
step 5959/19560 | loss 3.517151 (-0.15z)| norm 0.3157 (+2.20z)| lr 4.92e-04 | 322.22 ms | 52.4% bf16 MFU | 1624931 tok/s
step 5960/19560 | loss 3.501932 (-0.57z)| norm 0.3307 (+2.91z)| lr 4.92e-04 | 322.74 ms | 52.3% bf16 MFU | 1624908 tok/s
step 5961/19560 | loss 3.413793 (-2.91z)| norm 0.2906 (+0.73z)| lr 4.92e-04 | 322.12 ms | 52.4% bf16 MFU | 1625044 tok/s
step 5962/19560 | loss 3.539547 (+0.48z)| norm 0.2872 (+0.55z)| lr 4.92e-04 | 322.56 ms | 52.3% bf16 MFU | 1625061 tok/s
step 5963/19560 | loss 3.472621 (-1.30z)| norm 0.3118 (+1.84z)| lr 4.92e-04 | 323.10 ms | 52.2% bf16 MFU | 1624943 tok/s
step 5964/19560 | loss 3.500244 (-0.56z)| norm 0.2808 (+0.19z)| lr 4.92e-04 | 322.45 ms | 52.3% bf16 MFU | 1624993 tok/s
step 5965/19560 | loss 3.523507 (+0.08z)| norm 0.2855 (+0.44z)| lr 4.92e-04 | 322.53 ms | 52.3% bf16 MFU | 1625020 tok/s
step 5966/19560 | loss 3.563248 (+1.14z)| norm 0.2995 (+1.17z)| lr 4.92e-04 | 322.32 ms | 52.4% bf16 MFU | 1625100 tok/s
step 5967/19560 | loss 3.489478 (-0.84z)| norm 0.2845 (+0.37z)| lr 4.92e-04 | 322.76 ms | 52.3% bf16 MFU | 1625064 tok/s
step 5968/19560 | loss 3.547944 (+0.72z)| norm 0.2802 (+0.14z)| lr 4.92e-04 | 322.58 ms | 52.3% bf16 MFU | 1625075 tok/s
step 5969/19560 | loss 3.495433 (-0.68z)| norm 0.2893 (+0.65z)| lr 4.92e-04 | 322.46 ms | 52.3% bf16 MFU | 1625116 tok/s
step 5970/19560 | loss 3.595084 (+1.99z)| norm 0.2759 (-0.06z)| lr 4.92e-04 | 322.89 ms | 52.3% bf16 MFU | 1625047 tok/s
step 5971/19560 | loss 3.573549 (+1.38z)| norm 0.2844 (+0.47z)| lr 4.92e-04 | 322.73 ms | 52.3% bf16 MFU | 1625022 tok/s
step 5972/19560 | loss 3.508233 (-0.36z)| norm 0.2622 (-0.83z)| lr 4.92e-04 | 322.36 ms | 52.4% bf16 MFU | 1625093 tok/s
step 5973/19560 | loss 3.438251 (-2.17z)| norm 0.2833 (+0.42z)| lr 4.92e-04 | 323.33 ms | 52.2% bf16 MFU | 1624915 tok/s
step 5974/19560 | loss 3.587581 (+1.72z)| norm 0.2690 (-0.42z)| lr 4.92e-04 | 322.38 ms | 52.4% bf16 MFU | 1624984 tok/s
step 5975/19560 | loss 3.504165 (-0.46z)| norm 0.3003 (+1.44z)| lr 4.91e-04 | 323.29 ms | 52.2% bf16 MFU | 1624821 tok/s
step 5976/19560 | loss 3.482122 (-1.03z)| norm 0.2659 (-0.61z)| lr 4.91e-04 | 322.28 ms | 52.4% bf16 MFU | 1624919 tok/s
step 5977/19560 | loss 3.475552 (-1.20z)| norm 0.2800 (+0.22z)| lr 4.91e-04 | 322.64 ms | 52.3% bf16 MFU | 1624924 tok/s
step 5978/19560 | loss 3.561479 (+1.03z)| norm 0.2986 (+1.31z)| lr 4.91e-04 | 322.83 ms | 52.3% bf16 MFU | 1624880 tok/s
step 5979/19560 | loss 3.479750 (-1.08z)| norm 0.2892 (+0.74z)| lr 4.91e-04 | 323.01 ms | 52.2% bf16 MFU | 1624793 tok/s
step 5980/19560 | loss 3.464725 (-1.45z)| norm 0.3088 (+1.89z)| lr 4.91e-04 | 322.63 ms | 52.3% bf16 MFU | 1624806 tok/s
step 5981/19560 | loss 3.538651 (+0.44z)| norm 0.2568 (-1.27z)| lr 4.91e-04 | 322.60 ms | 52.3% bf16 MFU | 1624824 tok/s
step 5982/19560 | loss 3.578002 (+1.45z)| norm 0.3403 (+3.60z)| lr 4.91e-04 | 322.74 ms | 52.3% bf16 MFU | 1624807 tok/s
step 5983/19560 | loss 3.608797 (+2.18z)| norm 0.2866 (+0.49z)| lr 4.91e-04 | 322.72 ms | 52.3% bf16 MFU | 1624795 tok/s
step 5984/19560 | loss 3.607148 (+2.08z)| norm 0.3354 (+3.18z)| lr 4.91e-04 | 322.83 ms | 52.3% bf16 MFU | 1624757 tok/s
step 5985/19560 | loss 3.510692 (-0.30z)| norm 0.3140 (+1.93z)| lr 4.91e-04 | 322.55 ms | 52.3% bf16 MFU | 1624792 tok/s
step 5986/19560 | loss 3.485442 (-0.92z)| norm 0.2681 (-0.62z)| lr 4.91e-04 | 322.36 ms | 52.4% bf16 MFU | 1624871 tok/s
step 5987/19560 | loss 3.524290 (+0.04z)| norm 0.2805 (+0.07z)| lr 4.91e-04 | 323.29 ms | 52.2% bf16 MFU | 1624715 tok/s
step 5988/19560 | loss 3.487153 (-0.87z)| norm 0.2768 (-0.13z)| lr 4.91e-04 | 322.68 ms | 52.3% bf16 MFU | 1624719 tok/s
step 5989/19560 | loss 3.544226 (+0.56z)| norm 0.2711 (-0.45z)| lr 4.91e-04 | 322.67 ms | 52.3% bf16 MFU | 1624726 tok/s
step 5990/19560 | loss 3.518535 (-0.08z)| norm 0.2656 (-0.75z)| lr 4.91e-04 | 322.85 ms | 52.3% bf16 MFU | 1624686 tok/s
step 5991/19560 | loss 3.512186 (-0.23z)| norm 0.2618 (-0.96z)| lr 4.91e-04 | 322.49 ms | 52.3% bf16 MFU | 1624740 tok/s
step 5992/19560 | loss 3.433489 (-2.17z)| norm 0.2577 (-1.17z)| lr 4.91e-04 | 323.32 ms | 52.2% bf16 MFU | 1624583 tok/s
step 5993/19560 | loss 3.533838 (+0.31z)| norm 0.2752 (-0.20z)| lr 4.91e-04 | 322.56 ms | 52.3% bf16 MFU | 1624624 tok/s
step 5994/19560 | loss 3.584208 (+1.53z)| norm 0.2459 (-1.80z)| lr 4.91e-04 | 323.02 ms | 52.2% bf16 MFU | 1624546 tok/s
step 5995/19560 | loss 3.532179 (+0.25z)| norm 0.2810 (+0.14z)| lr 4.91e-04 | 322.57 ms | 52.3% bf16 MFU | 1624585 tok/s
step 5996/19560 | loss 3.460047 (-1.49z)| norm 0.2461 (-1.76z)| lr 4.91e-04 | 322.69 ms | 52.3% bf16 MFU | 1624593 tok/s
step 5997/19560 | loss 3.492780 (-0.69z)| norm 0.2661 (-0.67z)| lr 4.91e-04 | 323.57 ms | 52.2% bf16 MFU | 1624378 tok/s
step 5998/19560 | loss 3.476058 (-1.08z)| norm 0.2689 (-0.50z)| lr 4.91e-04 | 322.23 ms | 52.4% bf16 MFU | 1624513 tok/s
step 5999/19560 | loss 3.488585 (-0.76z)| norm 0.2635 (-0.79z)| lr 4.91e-04 | 323.01 ms | 52.2% bf16 MFU | 1624444 tok/s
step 6000/19560 | loss 3.540606 (+0.52z)| norm 0.2717 (-0.34z)| lr 4.91e-04 | 323.28 ms | 52.2% bf16 MFU | 1624309 tok/s
val loss 3.510486
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2792/10042 = 0.278032
step 6001/19560 | loss 3.484311 (-0.87z)| norm 0.2468 (-1.66z)| lr 4.90e-04 | 322.89 ms | 52.3% bf16 MFU | 1624281 tok/s
step 6002/19560 | loss 3.530425 (+0.27z)| norm 0.2703 (-0.37z)| lr 4.90e-04 | 322.34 ms | 52.4% bf16 MFU | 1624391 tok/s
step 6003/19560 | loss 3.574306 (+1.34z)| norm 0.2598 (-0.94z)| lr 4.90e-04 | 325.07 ms | 51.9% bf16 MFU | 1623814 tok/s
step 6004/19560 | loss 3.541531 (+0.53z)| norm 0.2902 (+0.71z)| lr 4.90e-04 | 322.46 ms | 52.3% bf16 MFU | 1623918 tok/s
step 6005/19560 | loss 3.564774 (+1.11z)| norm 0.2952 (+0.97z)| lr 4.90e-04 | 323.21 ms | 52.2% bf16 MFU | 1623828 tok/s
step 6006/19560 | loss 3.486656 (-0.82z)| norm 0.3002 (+1.23z)| lr 4.90e-04 | 322.28 ms | 52.4% bf16 MFU | 1623978 tok/s
step 6007/19560 | loss 3.531697 (+0.31z)| norm 0.3198 (+2.23z)| lr 4.90e-04 | 322.63 ms | 52.3% bf16 MFU | 1624032 tok/s
step 6008/19560 | loss 3.496106 (-0.58z)| norm 0.2557 (-1.18z)| lr 4.90e-04 | 323.15 ms | 52.2% bf16 MFU | 1623953 tok/s
step 6009/19560 | loss 3.544112 (+0.62z)| norm 0.2929 (+0.78z)| lr 4.90e-04 | 322.67 ms | 52.3% bf16 MFU | 1623999 tok/s
step 6010/19560 | loss 3.524221 (+0.11z)| norm 0.2668 (-0.60z)| lr 4.90e-04 | 323.00 ms | 52.3% bf16 MFU | 1623958 tok/s
step 6011/19560 | loss 3.627606 (+2.62z)| norm 0.2939 (+0.83z)| lr 4.90e-04 | 322.88 ms | 52.3% bf16 MFU | 1623949 tok/s
step 6012/19560 | loss 3.545660 (+0.61z)| norm 0.3469 (+3.44z)| lr 4.90e-04 | 322.73 ms | 52.3% bf16 MFU | 1623980 tok/s
step 6013/19560 | loss 3.521091 (+0.00z)| norm 0.3324 (+2.62z)| lr 4.90e-04 | 322.68 ms | 52.3% bf16 MFU | 1624020 tok/s
step 6014/19560 | loss 3.488998 (-0.78z)| norm 0.3171 (+1.83z)| lr 4.90e-04 | 323.05 ms | 52.2% bf16 MFU | 1623964 tok/s
step 6015/19560 | loss 3.464579 (-1.35z)| norm 0.3092 (+1.42z)| lr 4.90e-04 | 323.05 ms | 52.2% bf16 MFU | 1623912 tok/s
step 6016/19560 | loss 3.516647 (-0.09z)| norm 0.2982 (+0.87z)| lr 4.90e-04 | 322.92 ms | 52.3% bf16 MFU | 1623895 tok/s
step 6017/19560 | loss 3.495374 (-0.60z)| norm 0.3006 (+0.97z)| lr 4.90e-04 | 322.87 ms | 52.3% bf16 MFU | 1623892 tok/s
step 6018/19560 | loss 3.500168 (-0.48z)| norm 0.2823 (+0.08z)| lr 4.90e-04 | 323.33 ms | 52.2% bf16 MFU | 1623773 tok/s
step 6019/19560 | loss 3.523855 (+0.08z)| norm 0.2909 (+0.50z)| lr 4.90e-04 | 322.39 ms | 52.3% bf16 MFU | 1623897 tok/s
step 6020/19560 | loss 3.543225 (+0.55z)| norm 0.2840 (+0.16z)| lr 4.90e-04 | 323.39 ms | 52.2% bf16 MFU | 1623764 tok/s
step 6021/19560 | loss 3.552980 (+0.79z)| norm 0.2711 (-0.48z)| lr 4.90e-04 | 323.22 ms | 52.2% bf16 MFU | 1623680 tok/s
step 6022/19560 | loss 3.521562 (+0.02z)| norm 0.2760 (-0.23z)| lr 4.90e-04 | 323.21 ms | 52.2% bf16 MFU | 1623603 tok/s
step 6023/19560 | loss 3.660158 (+3.24z)| norm 0.3285 (+2.32z)| lr 4.90e-04 | 322.70 ms | 52.3% bf16 MFU | 1623657 tok/s
step 6024/19560 | loss 3.506659 (-0.37z)| norm 0.2815 (+0.03z)| lr 4.90e-04 | 323.51 ms | 52.2% bf16 MFU | 1623506 tok/s
step 6025/19560 | loss 3.454045 (-1.60z)| norm 0.2638 (-0.82z)| lr 4.90e-04 | 322.80 ms | 52.3% bf16 MFU | 1623541 tok/s
step 6026/19560 | loss 3.443453 (-1.81z)| norm 0.2837 (+0.15z)| lr 4.90e-04 | 322.79 ms | 52.3% bf16 MFU | 1623575 tok/s
step 6027/19560 | loss 3.539989 (+0.42z)| norm 0.2808 (+0.00z)| lr 4.89e-04 | 323.33 ms | 52.2% bf16 MFU | 1623472 tok/s
step 6028/19560 | loss 3.595517 (+1.68z)| norm 0.2697 (-0.53z)| lr 4.89e-04 | 323.13 ms | 52.2% bf16 MFU | 1623425 tok/s
step 6029/19560 | loss 3.483003 (-0.90z)| norm 0.3070 (+1.27z)| lr 4.89e-04 | 322.98 ms | 52.3% bf16 MFU | 1623416 tok/s
step 6030/19560 | loss 3.498702 (-0.53z)| norm 0.2830 (+0.10z)| lr 4.89e-04 | 323.18 ms | 52.2% bf16 MFU | 1623360 tok/s
step 6031/19560 | loss 3.495098 (-0.62z)| norm 0.2832 (+0.12z)| lr 4.89e-04 | 322.86 ms | 52.3% bf16 MFU | 1623388 tok/s
step 6032/19560 | loss 3.554514 (+0.76z)| norm 0.2751 (-0.27z)| lr 4.89e-04 | 323.13 ms | 52.2% bf16 MFU | 1623344 tok/s
step 6033/19560 | loss 3.508757 (-0.30z)| norm 0.3128 (+1.61z)| lr 4.89e-04 | 322.70 ms | 52.3% bf16 MFU | 1623411 tok/s
step 6034/19560 | loss 3.544198 (+0.52z)| norm 0.2770 (-0.18z)| lr 4.89e-04 | 323.24 ms | 52.2% bf16 MFU | 1623340 tok/s
step 6035/19560 | loss 3.538201 (+0.39z)| norm 0.2465 (-1.68z)| lr 4.89e-04 | 323.40 ms | 52.2% bf16 MFU | 1623230 tok/s
step 6036/19560 | loss 3.555290 (+0.78z)| norm 0.2591 (-1.05z)| lr 4.89e-04 | 322.80 ms | 52.3% bf16 MFU | 1623279 tok/s
step 6037/19560 | loss 3.505181 (-0.38z)| norm 0.2715 (-0.45z)| lr 4.89e-04 | 322.85 ms | 52.3% bf16 MFU | 1623313 tok/s
step 6038/19560 | loss 3.546719 (+0.58z)| norm 0.2422 (-1.88z)| lr 4.89e-04 | 322.91 ms | 52.3% bf16 MFU | 1623330 tok/s
step 6039/19560 | loss 3.535565 (+0.32z)| norm 0.2445 (-1.76z)| lr 4.89e-04 | 322.95 ms | 52.3% bf16 MFU | 1623336 tok/s
step 6040/19560 | loss 3.509073 (-0.29z)| norm 0.2371 (-2.11z)| lr 4.89e-04 | 323.13 ms | 52.2% bf16 MFU | 1623296 tok/s
step 6041/19560 | loss 3.541135 (+0.44z)| norm 0.2383 (-2.03z)| lr 4.89e-04 | 322.95 ms | 52.3% bf16 MFU | 1623303 tok/s
step 6042/19560 | loss 3.471799 (-1.16z)| norm 0.2679 (-0.60z)| lr 4.89e-04 | 323.16 ms | 52.2% bf16 MFU | 1623257 tok/s
step 6043/19560 | loss 3.551459 (+0.68z)| norm 0.2515 (-1.38z)| lr 4.89e-04 | 322.76 ms | 52.3% bf16 MFU | 1623314 tok/s
step 6044/19560 | loss 3.567906 (+1.04z)| norm 0.2896 (+0.45z)| lr 4.89e-04 | 322.85 ms | 52.3% bf16 MFU | 1623344 tok/s
step 6045/19560 | loss 3.526599 (+0.07z)| norm 0.2819 (+0.07z)| lr 4.89e-04 | 322.71 ms | 52.3% bf16 MFU | 1623408 tok/s
step 6046/19560 | loss 3.530402 (+0.16z)| norm 0.2563 (-1.16z)| lr 4.89e-04 | 323.01 ms | 52.2% bf16 MFU | 1623395 tok/s
step 6047/19560 | loss 3.506407 (-0.40z)| norm 0.2949 (+0.70z)| lr 4.89e-04 | 322.98 ms | 52.3% bf16 MFU | 1623389 tok/s
step 6048/19560 | loss 3.522407 (-0.03z)| norm 0.2992 (+0.90z)| lr 4.89e-04 | 322.81 ms | 52.3% bf16 MFU | 1623427 tok/s
step 6049/19560 | loss 3.479282 (-1.04z)| norm 0.2476 (-1.56z)| lr 4.89e-04 | 322.44 ms | 52.3% bf16 MFU | 1623556 tok/s
step 6050/19560 | loss 3.507765 (-0.37z)| norm 0.2952 (+0.71z)| lr 4.89e-04 | 322.71 ms | 52.3% bf16 MFU | 1623609 tok/s
step 6051/19560 | loss 3.516262 (-0.16z)| norm 0.2753 (-0.24z)| lr 4.89e-04 | 322.90 ms | 52.3% bf16 MFU | 1623612 tok/s
step 6052/19560 | loss 3.528806 (+0.14z)| norm 0.3107 (+1.42z)| lr 4.89e-04 | 322.56 ms | 52.3% bf16 MFU | 1623702 tok/s
step 6053/19560 | loss 3.538189 (+0.36z)| norm 0.3226 (+1.93z)| lr 4.88e-04 | 322.42 ms | 52.3% bf16 MFU | 1623822 tok/s
step 6054/19560 | loss 3.519213 (-0.08z)| norm 0.3047 (+1.09z)| lr 4.88e-04 | 322.95 ms | 52.3% bf16 MFU | 1623804 tok/s
step 6055/19560 | loss 3.481010 (-1.00z)| norm 0.2721 (-0.42z)| lr 4.88e-04 | 323.36 ms | 52.2% bf16 MFU | 1623682 tok/s
step 6056/19560 | loss 3.532065 (+0.23z)| norm 0.2567 (-1.13z)| lr 4.88e-04 | 323.12 ms | 52.2% bf16 MFU | 1623627 tok/s
step 6057/19560 | loss 3.479688 (-1.02z)| norm 0.2594 (-0.99z)| lr 4.88e-04 | 322.26 ms | 52.4% bf16 MFU | 1623791 tok/s
step 6058/19560 | loss 3.511237 (-0.26z)| norm 0.2680 (-0.58z)| lr 4.88e-04 | 322.39 ms | 52.4% bf16 MFU | 1623914 tok/s
step 6059/19560 | loss 3.454922 (-1.58z)| norm 0.2503 (-1.39z)| lr 4.88e-04 | 322.71 ms | 52.3% bf16 MFU | 1623950 tok/s
step 6060/19560 | loss 3.523625 (+0.06z)| norm 0.2497 (-1.40z)| lr 4.88e-04 | 323.19 ms | 52.2% bf16 MFU | 1623864 tok/s
step 6061/19560 | loss 3.520831 (-0.01z)| norm 0.2644 (-0.72z)| lr 4.88e-04 | 322.64 ms | 52.3% bf16 MFU | 1623920 tok/s
step 6062/19560 | loss 3.523437 (+0.06z)| norm 0.2666 (-0.62z)| lr 4.88e-04 | 322.59 ms | 52.3% bf16 MFU | 1623988 tok/s
step 6063/19560 | loss 3.582136 (+1.43z)| norm 0.2627 (-0.78z)| lr 4.88e-04 | 322.34 ms | 52.4% bf16 MFU | 1624114 tok/s
step 6064/19560 | loss 3.533760 (+0.28z)| norm 0.2386 (-1.85z)| lr 4.88e-04 | 322.50 ms | 52.3% bf16 MFU | 1624193 tok/s
step 6065/19560 | loss 3.586009 (+1.51z)| norm 0.2799 (-0.00z)| lr 4.88e-04 | 322.71 ms | 52.3% bf16 MFU | 1624215 tok/s
step 6066/19560 | loss 3.564393 (+1.00z)| norm 0.2749 (-0.23z)| lr 4.88e-04 | 323.15 ms | 52.2% bf16 MFU | 1624125 tok/s
step 6067/19560 | loss 3.572976 (+1.20z)| norm 0.2849 (+0.22z)| lr 4.88e-04 | 322.54 ms | 52.3% bf16 MFU | 1624195 tok/s
step 6068/19560 | loss 3.532608 (+0.22z)| norm 0.2550 (-1.12z)| lr 4.88e-04 | 322.48 ms | 52.3% bf16 MFU | 1624275 tok/s
step 6069/19560 | loss 3.551728 (+0.68z)| norm 0.2482 (-1.41z)| lr 4.88e-04 | 322.73 ms | 52.3% bf16 MFU | 1624288 tok/s
step 6070/19560 | loss 3.541960 (+0.44z)| norm 0.2539 (-1.14z)| lr 4.88e-04 | 322.19 ms | 52.4% bf16 MFU | 1624438 tok/s
step 6071/19560 | loss 3.524829 (+0.02z)| norm 0.2609 (-0.82z)| lr 4.88e-04 | 322.47 ms | 52.3% bf16 MFU | 1624508 tok/s
step 6072/19560 | loss 3.519537 (-0.11z)| norm 0.2630 (-0.73z)| lr 4.88e-04 | 323.04 ms | 52.2% bf16 MFU | 1624431 tok/s
step 6073/19560 | loss 3.506227 (-0.43z)| norm 0.2750 (-0.20z)| lr 4.88e-04 | 322.62 ms | 52.3% bf16 MFU | 1624463 tok/s
step 6074/19560 | loss 3.692810 (+3.82z)| norm 0.3136 (+1.49z)| lr 4.88e-04 | 322.57 ms | 52.3% bf16 MFU | 1624508 tok/s
step 6075/19560 | loss 3.491546 (-0.77z)| norm 0.2870 (+0.32z)| lr 4.88e-04 | 322.37 ms | 52.4% bf16 MFU | 1624599 tok/s
step 6076/19560 | loss 3.512904 (-0.28z)| norm 0.2750 (-0.21z)| lr 4.88e-04 | 322.68 ms | 52.3% bf16 MFU | 1624608 tok/s
step 6077/19560 | loss 3.538340 (+0.30z)| norm 0.2958 (+0.70z)| lr 4.88e-04 | 322.75 ms | 52.3% bf16 MFU | 1624599 tok/s
step 6078/19560 | loss 3.479317 (-1.03z)| norm 0.2632 (-0.74z)| lr 4.87e-04 | 322.85 ms | 52.3% bf16 MFU | 1624566 tok/s
step 6079/19560 | loss 3.578030 (+1.27z)| norm 0.2651 (-0.66z)| lr 4.87e-04 | 322.71 ms | 52.3% bf16 MFU | 1624569 tok/s
step 6080/19560 | loss 3.518061 (-0.13z)| norm 0.2958 (+0.69z)| lr 4.87e-04 | 322.13 ms | 52.4% bf16 MFU | 1624719 tok/s
step 6081/19560 | loss 3.515830 (-0.19z)| norm 0.2912 (+0.48z)| lr 4.87e-04 | 322.60 ms | 52.3% bf16 MFU | 1624743 tok/s
step 6082/19560 | loss 3.533210 (+0.22z)| norm 0.2451 (-1.51z)| lr 4.87e-04 | 322.71 ms | 52.3% bf16 MFU | 1624739 tok/s
step 6083/19560 | loss 3.472730 (-1.19z)| norm 0.2650 (-0.64z)| lr 4.87e-04 | 322.58 ms | 52.3% bf16 MFU | 1624766 tok/s
step 6084/19560 | loss 3.511689 (-0.28z)| norm 0.2660 (-0.59z)| lr 4.87e-04 | 322.45 ms | 52.3% bf16 MFU | 1624824 tok/s
step 6085/19560 | loss 3.547532 (+0.56z)| norm 0.2687 (-0.48z)| lr 4.87e-04 | 322.45 ms | 52.3% bf16 MFU | 1624880 tok/s
step 6086/19560 | loss 3.563497 (+0.93z)| norm 0.2711 (-0.37z)| lr 4.87e-04 | 322.57 ms | 52.3% bf16 MFU | 1624903 tok/s
step 6087/19560 | loss 3.567352 (+1.01z)| norm 0.2640 (-0.66z)| lr 4.87e-04 | 322.45 ms | 52.3% bf16 MFU | 1624954 tok/s
step 6088/19560 | loss 3.511268 (-0.30z)| norm 0.2839 (+0.23z)| lr 4.87e-04 | 322.86 ms | 52.3% bf16 MFU | 1624901 tok/s
step 6089/19560 | loss 3.565090 (+0.95z)| norm 0.3276 (+2.13z)| lr 4.87e-04 | 322.17 ms | 52.4% bf16 MFU | 1625025 tok/s
step 6090/19560 | loss 3.528689 (+0.08z)| norm 0.2877 (+0.38z)| lr 4.87e-04 | 322.63 ms | 52.3% bf16 MFU | 1625026 tok/s
step 6091/19560 | loss 3.506853 (-0.45z)| norm 0.2785 (-0.01z)| lr 4.87e-04 | 322.75 ms | 52.3% bf16 MFU | 1624996 tok/s
step 6092/19560 | loss 3.492683 (-0.79z)| norm 0.2933 (+0.64z)| lr 4.87e-04 | 322.70 ms | 52.3% bf16 MFU | 1624980 tok/s
step 6093/19560 | loss 3.488347 (-0.89z)| norm 0.2683 (-0.46z)| lr 4.87e-04 | 322.58 ms | 52.3% bf16 MFU | 1624997 tok/s
step 6094/19560 | loss 3.455925 (-1.63z)| norm 0.2564 (-0.97z)| lr 4.87e-04 | 322.33 ms | 52.4% bf16 MFU | 1625074 tok/s
step 6095/19560 | loss 3.512374 (-0.29z)| norm 0.2674 (-0.48z)| lr 4.87e-04 | 323.12 ms | 52.2% bf16 MFU | 1624951 tok/s
step 6096/19560 | loss 3.520465 (-0.10z)| norm 0.2720 (-0.27z)| lr 4.87e-04 | 323.11 ms | 52.2% bf16 MFU | 1624835 tok/s
step 6097/19560 | loss 3.487828 (-0.87z)| norm 0.2610 (-0.75z)| lr 4.87e-04 | 322.65 ms | 52.3% bf16 MFU | 1624840 tok/s
step 6098/19560 | loss 3.663235 (+3.20z)| norm 0.2826 (+0.20z)| lr 4.87e-04 | 322.80 ms | 52.3% bf16 MFU | 1624808 tok/s
step 6099/19560 | loss 3.433664 (-2.06z)| norm 0.2674 (-0.46z)| lr 4.87e-04 | 322.66 ms | 52.3% bf16 MFU | 1624811 tok/s
step 6100/19560 | loss 3.495999 (-0.63z)| norm 0.2644 (-0.59z)| lr 4.87e-04 | 322.44 ms | 52.3% bf16 MFU | 1624872 tok/s
step 6101/19560 | loss 3.513325 (-0.26z)| norm 0.2766 (-0.05z)| lr 4.87e-04 | 322.69 ms | 52.3% bf16 MFU | 1624866 tok/s
step 6102/19560 | loss 3.491906 (-0.74z)| norm 0.2586 (-0.84z)| lr 4.87e-04 | 322.33 ms | 52.4% bf16 MFU | 1624951 tok/s
step 6103/19560 | loss 3.505099 (-0.43z)| norm 0.2901 (+0.55z)| lr 4.87e-04 | 322.85 ms | 52.3% bf16 MFU | 1624900 tok/s
step 6104/19560 | loss 3.477809 (-1.07z)| norm 0.3650 (+3.61z)| lr 4.86e-04 | 322.71 ms | 52.3% bf16 MFU | 1624887 tok/s
step 6105/19560 | loss 3.542732 (+0.44z)| norm 0.4952 (+7.02z)| lr 4.86e-04 | 322.12 ms | 52.4% bf16 MFU | 1625022 tok/s
step 6106/19560 | loss 3.581515 (+1.34z)| norm 0.3726 (+2.90z)| lr 4.86e-04 | 322.90 ms | 52.3% bf16 MFU | 1624956 tok/s
step 6107/19560 | loss 3.534658 (+0.23z)| norm 0.3385 (+1.79z)| lr 4.86e-04 | 322.35 ms | 52.4% bf16 MFU | 1625031 tok/s
step 6108/19560 | loss 3.509223 (-0.37z)| norm 0.3320 (+1.57z)| lr 4.86e-04 | 322.76 ms | 52.3% bf16 MFU | 1624998 tok/s
step 6109/19560 | loss 3.513317 (-0.27z)| norm 0.3109 (+0.90z)| lr 4.86e-04 | 322.73 ms | 52.3% bf16 MFU | 1624976 tok/s
step 6110/19560 | loss 3.554829 (+0.71z)| norm 0.3328 (+1.59z)| lr 4.86e-04 | 322.28 ms | 52.4% bf16 MFU | 1625068 tok/s
step 6111/19560 | loss 3.545760 (+0.52z)| norm 0.3108 (+0.89z)| lr 4.86e-04 | 322.62 ms | 52.3% bf16 MFU | 1625068 tok/s
step 6112/19560 | loss 3.605615 (+1.96z)| norm 0.2857 (+0.13z)| lr 4.86e-04 | 322.39 ms | 52.4% bf16 MFU | 1625128 tok/s
step 6113/19560 | loss 3.510928 (-0.32z)| norm 0.3148 (+1.04z)| lr 4.86e-04 | 322.83 ms | 52.3% bf16 MFU | 1625074 tok/s
step 6114/19560 | loss 3.479756 (-1.07z)| norm 0.2712 (-0.32z)| lr 4.86e-04 | 322.57 ms | 52.3% bf16 MFU | 1625087 tok/s
step 6115/19560 | loss 3.497251 (-0.64z)| norm 0.2739 (-0.24z)| lr 4.86e-04 | 322.54 ms | 52.3% bf16 MFU | 1625108 tok/s
step 6116/19560 | loss 3.489020 (-0.84z)| norm 0.2664 (-0.47z)| lr 4.86e-04 | 322.69 ms | 52.3% bf16 MFU | 1625088 tok/s
step 6117/19560 | loss 3.521870 (-0.04z)| norm 0.2631 (-0.57z)| lr 4.86e-04 | 322.18 ms | 52.4% bf16 MFU | 1625199 tok/s
step 6118/19560 | loss 3.663100 (+3.18z)| norm 0.2599 (-0.67z)| lr 4.86e-04 | 322.46 ms | 52.3% bf16 MFU | 1625234 tok/s
step 6119/19560 | loss 3.535633 (+0.24z)| norm 0.3014 (+0.62z)| lr 4.86e-04 | 323.71 ms | 52.1% bf16 MFU | 1624953 tok/s
step 6120/19560 | loss 3.534025 (+0.19z)| norm 0.2736 (-0.25z)| lr 4.86e-04 | 322.28 ms | 52.4% bf16 MFU | 1625047 tok/s
step 6121/19560 | loss 3.548064 (+0.52z)| norm 0.2727 (-0.28z)| lr 4.86e-04 | 322.56 ms | 52.3% bf16 MFU | 1625064 tok/s
step 6122/19560 | loss 3.577471 (+1.21z)| norm 0.2626 (-0.60z)| lr 4.86e-04 | 322.81 ms | 52.3% bf16 MFU | 1625018 tok/s
step 6123/19560 | loss 3.463645 (-1.44z)| norm 0.2485 (-1.03z)| lr 4.86e-04 | 322.64 ms | 52.3% bf16 MFU | 1625016 tok/s
step 6124/19560 | loss 3.471851 (-1.25z)| norm 0.2873 (+0.17z)| lr 4.86e-04 | 322.88 ms | 52.3% bf16 MFU | 1624955 tok/s
step 6125/19560 | loss 3.499713 (-0.60z)| norm 0.2491 (-1.02z)| lr 4.86e-04 | 322.37 ms | 52.4% bf16 MFU | 1625026 tok/s
step 6126/19560 | loss 3.520275 (-0.13z)| norm 0.2813 (-0.02z)| lr 4.86e-04 | 322.19 ms | 52.4% bf16 MFU | 1625137 tok/s
step 6127/19560 | loss 3.452787 (-1.70z)| norm 0.2768 (-0.16z)| lr 4.86e-04 | 322.60 ms | 52.3% bf16 MFU | 1625140 tok/s
step 6128/19560 | loss 3.489867 (-0.82z)| norm 0.2609 (-0.66z)| lr 4.86e-04 | 322.93 ms | 52.3% bf16 MFU | 1625060 tok/s
step 6129/19560 | loss 3.561852 (+0.84z)| norm 0.2627 (-0.61z)| lr 4.86e-04 | 322.56 ms | 52.3% bf16 MFU | 1625077 tok/s
step 6130/19560 | loss 3.490465 (-0.81z)| norm 0.2560 (-0.81z)| lr 4.85e-04 | 322.47 ms | 52.3% bf16 MFU | 1625115 tok/s
step 6131/19560 | loss 3.472311 (-1.21z)| norm 0.2613 (-0.64z)| lr 4.85e-04 | 323.54 ms | 52.2% bf16 MFU | 1624884 tok/s
step 6132/19560 | loss 3.498892 (-0.59z)| norm 0.2664 (-0.48z)| lr 4.85e-04 | 322.20 ms | 52.4% bf16 MFU | 1624999 tok/s
step 6133/19560 | loss 3.443300 (-1.84z)| norm 0.2758 (-0.18z)| lr 4.85e-04 | 323.00 ms | 52.3% bf16 MFU | 1624908 tok/s
step 6134/19560 | loss 3.526862 (+0.07z)| norm 0.2558 (-0.80z)| lr 4.85e-04 | 322.43 ms | 52.3% bf16 MFU | 1624965 tok/s
step 6135/19560 | loss 3.517921 (-0.13z)| norm 0.2575 (-0.73z)| lr 4.85e-04 | 322.33 ms | 52.4% bf16 MFU | 1625044 tok/s
step 6136/19560 | loss 3.447248 (-1.73z)| norm 0.2681 (-0.40z)| lr 4.85e-04 | 322.47 ms | 52.3% bf16 MFU | 1625086 tok/s
step 6137/19560 | loss 3.421906 (-2.24z)| norm 0.2646 (-0.50z)| lr 4.85e-04 | 323.07 ms | 52.2% bf16 MFU | 1624972 tok/s
step 6138/19560 | loss 3.535302 (+0.29z)| norm 0.2682 (-0.39z)| lr 4.85e-04 | 322.70 ms | 52.3% bf16 MFU | 1624959 tok/s
step 6139/19560 | loss 3.501842 (-0.44z)| norm 0.2836 (+0.10z)| lr 4.85e-04 | 322.79 ms | 52.3% bf16 MFU | 1624922 tok/s
step 6140/19560 | loss 3.488800 (-0.73z)| norm 0.2597 (-0.64z)| lr 4.85e-04 | 322.33 ms | 52.4% bf16 MFU | 1625003 tok/s
step 6141/19560 | loss 3.509117 (-0.27z)| norm 0.2536 (-0.83z)| lr 4.85e-04 | 322.36 ms | 52.4% bf16 MFU | 1625073 tok/s
step 6142/19560 | loss 3.477624 (-0.98z)| norm 0.2519 (-0.87z)| lr 4.85e-04 | 322.74 ms | 52.3% bf16 MFU | 1625045 tok/s
step 6143/19560 | loss 3.521912 (+0.02z)| norm 0.2748 (-0.12z)| lr 4.85e-04 | 322.51 ms | 52.3% bf16 MFU | 1625075 tok/s
step 6144/19560 | loss 3.449893 (-1.60z)| norm 0.2810 (+0.09z)| lr 4.85e-04 | 322.90 ms | 52.3% bf16 MFU | 1625005 tok/s
step 6145/19560 | loss 3.495197 (-0.58z)| norm 0.2549 (-0.75z)| lr 4.85e-04 | 322.69 ms | 52.3% bf16 MFU | 1624991 tok/s
step 6146/19560 | loss 3.522421 (+0.04z)| norm 0.2667 (-0.36z)| lr 4.85e-04 | 322.82 ms | 52.3% bf16 MFU | 1624945 tok/s
step 6147/19560 | loss 3.533541 (+0.29z)| norm 0.2770 (-0.02z)| lr 4.85e-04 | 322.52 ms | 52.3% bf16 MFU | 1624977 tok/s
step 6148/19560 | loss 3.557308 (+0.82z)| norm 0.2968 (+0.62z)| lr 4.85e-04 | 322.92 ms | 52.3% bf16 MFU | 1624908 tok/s
step 6149/19560 | loss 3.514586 (-0.14z)| norm 0.3096 (+1.02z)| lr 4.85e-04 | 322.69 ms | 52.3% bf16 MFU | 1624901 tok/s
step 6150/19560 | loss 3.509228 (-0.26z)| norm 0.2847 (+0.21z)| lr 4.85e-04 | 322.51 ms | 52.3% bf16 MFU | 1624938 tok/s
step 6151/19560 | loss 3.502858 (-0.39z)| norm 0.2631 (-0.48z)| lr 4.85e-04 | 322.64 ms | 52.3% bf16 MFU | 1624941 tok/s
step 6152/19560 | loss 3.475156 (-1.03z)| norm 0.2754 (-0.07z)| lr 4.85e-04 | 322.72 ms | 52.3% bf16 MFU | 1624923 tok/s
step 6153/19560 | loss 3.497189 (-0.53z)| norm 0.3109 (+1.07z)| lr 4.85e-04 | 322.75 ms | 52.3% bf16 MFU | 1624898 tok/s
step 6154/19560 | loss 3.560340 (+0.96z)| norm 0.2779 (-0.00z)| lr 4.85e-04 | 322.28 ms | 52.4% bf16 MFU | 1624993 tok/s
step 6155/19560 | loss 3.517431 (-0.07z)| norm 0.2632 (-0.47z)| lr 4.84e-04 | 322.73 ms | 52.3% bf16 MFU | 1624972 tok/s
step 6156/19560 | loss 3.603163 (+1.98z)| norm 0.2918 (+0.45z)| lr 4.84e-04 | 322.33 ms | 52.4% bf16 MFU | 1625052 tok/s
step 6157/19560 | loss 3.539005 (+0.44z)| norm 0.2906 (+0.42z)| lr 4.84e-04 | 323.39 ms | 52.2% bf16 MFU | 1624860 tok/s
step 6158/19560 | loss 3.502486 (-0.44z)| norm 0.3004 (+0.73z)| lr 4.84e-04 | 322.25 ms | 52.4% bf16 MFU | 1624965 tok/s
step 6159/19560 | loss 3.484146 (-0.87z)| norm 0.2917 (+0.44z)| lr 4.84e-04 | 322.63 ms | 52.3% bf16 MFU | 1624968 tok/s
step 6160/19560 | loss 3.508214 (-0.29z)| norm 0.2869 (+0.28z)| lr 4.84e-04 | 323.18 ms | 52.2% bf16 MFU | 1624834 tok/s
step 6161/19560 | loss 3.522713 (+0.06z)| norm 0.2958 (+0.58z)| lr 4.84e-04 | 322.64 ms | 52.3% bf16 MFU | 1624842 tok/s
step 6162/19560 | loss 3.569642 (+1.17z)| norm 0.2779 (-0.01z)| lr 4.84e-04 | 322.63 ms | 52.3% bf16 MFU | 1624851 tok/s
step 6163/19560 | loss 3.566295 (+1.08z)| norm 0.2789 (+0.02z)| lr 4.84e-04 | 322.70 ms | 52.3% bf16 MFU | 1624842 tok/s
step 6164/19560 | loss 3.492146 (-0.67z)| norm 0.2751 (-0.11z)| lr 4.84e-04 | 322.95 ms | 52.3% bf16 MFU | 1624771 tok/s
step 6165/19560 | loss 3.475609 (-1.05z)| norm 0.2931 (+0.48z)| lr 4.84e-04 | 322.42 ms | 52.3% bf16 MFU | 1624839 tok/s
step 6166/19560 | loss 3.486301 (-0.79z)| norm 0.2566 (-0.73z)| lr 4.84e-04 | 322.51 ms | 52.3% bf16 MFU | 1624879 tok/s
step 6167/19560 | loss 3.482877 (-0.86z)| norm 0.2783 (-0.02z)| lr 4.84e-04 | 323.02 ms | 52.2% bf16 MFU | 1624789 tok/s
step 6168/19560 | loss 3.555391 (+0.84z)| norm 0.2702 (-0.30z)| lr 4.84e-04 | 322.35 ms | 52.4% bf16 MFU | 1624873 tok/s
step 6169/19560 | loss 3.476210 (-1.01z)| norm 0.2544 (-0.84z)| lr 4.84e-04 | 323.33 ms | 52.2% bf16 MFU | 1624706 tok/s
step 6170/19560 | loss 3.548679 (+0.68z)| norm 0.2647 (-0.49z)| lr 4.84e-04 | 322.43 ms | 52.3% bf16 MFU | 1624774 tok/s
step 6171/19560 | loss 3.457679 (-1.44z)| norm 0.2868 (+0.24z)| lr 4.84e-04 | 322.57 ms | 52.3% bf16 MFU | 1624803 tok/s
step 6172/19560 | loss 3.542548 (+0.56z)| norm 0.2475 (-1.06z)| lr 4.84e-04 | 323.50 ms | 52.2% bf16 MFU | 1624597 tok/s
step 6173/19560 | loss 3.496663 (-0.51z)| norm 0.2604 (-0.62z)| lr 4.84e-04 | 322.66 ms | 52.3% bf16 MFU | 1624611 tok/s
step 6174/19560 | loss 3.504820 (-0.32z)| norm 0.2619 (-0.57z)| lr 4.84e-04 | 322.81 ms | 52.3% bf16 MFU | 1624588 tok/s
step 6175/19560 | loss 3.630444 (+2.55z)| norm 0.3272 (+1.59z)| lr 4.84e-04 | 322.47 ms | 52.3% bf16 MFU | 1624651 tok/s
step 6176/19560 | loss 3.489400 (-0.68z)| norm 0.3308 (+1.68z)| lr 4.84e-04 | 322.43 ms | 52.3% bf16 MFU | 1624722 tok/s
step 6177/19560 | loss 3.509143 (-0.23z)| norm 0.3045 (+0.80z)| lr 4.84e-04 | 323.42 ms | 52.2% bf16 MFU | 1624539 tok/s
step 6178/19560 | loss 3.493599 (-0.58z)| norm 0.2602 (-0.64z)| lr 4.84e-04 | 323.09 ms | 52.2% bf16 MFU | 1624449 tok/s
step 6179/19560 | loss 3.484330 (-0.79z)| norm 0.3004 (+0.67z)| lr 4.84e-04 | 322.29 ms | 52.4% bf16 MFU | 1624564 tok/s
step 6180/19560 | loss 3.566733 (+1.08z)| norm 0.2727 (-0.23z)| lr 4.83e-04 | 322.82 ms | 52.3% bf16 MFU | 1624540 tok/s
step 6181/19560 | loss 3.496519 (-0.51z)| norm 0.2810 (+0.05z)| lr 4.83e-04 | 322.46 ms | 52.3% bf16 MFU | 1624609 tok/s
step 6182/19560 | loss 3.529753 (+0.25z)| norm 0.3011 (+0.72z)| lr 4.83e-04 | 322.37 ms | 52.4% bf16 MFU | 1624697 tok/s
step 6183/19560 | loss 3.549799 (+0.69z)| norm 0.3132 (+1.11z)| lr 4.83e-04 | 323.12 ms | 52.2% bf16 MFU | 1624591 tok/s
step 6184/19560 | loss 3.561198 (+0.94z)| norm 0.2850 (+0.17z)| lr 4.83e-04 | 323.12 ms | 52.2% bf16 MFU | 1624491 tok/s
step 6185/19560 | loss 3.480786 (-0.88z)| norm 0.2835 (+0.11z)| lr 4.83e-04 | 322.83 ms | 52.3% bf16 MFU | 1624467 tok/s
step 6186/19560 | loss 3.501924 (-0.40z)| norm 0.2744 (-0.19z)| lr 4.83e-04 | 322.88 ms | 52.3% bf16 MFU | 1624432 tok/s
step 6187/19560 | loss 3.483122 (-0.84z)| norm 0.2901 (+0.32z)| lr 4.83e-04 | 322.83 ms | 52.3% bf16 MFU | 1624413 tok/s
step 6188/19560 | loss 3.542738 (+0.52z)| norm 0.2889 (+0.27z)| lr 4.83e-04 | 322.42 ms | 52.3% bf16 MFU | 1624497 tok/s
step 6189/19560 | loss 3.541184 (+0.48z)| norm 0.2880 (+0.24z)| lr 4.83e-04 | 323.05 ms | 52.2% bf16 MFU | 1624418 tok/s
step 6190/19560 | loss 3.487460 (-0.74z)| norm 0.2855 (+0.15z)| lr 4.83e-04 | 322.96 ms | 52.3% bf16 MFU | 1624366 tok/s
step 6191/19560 | loss 3.524841 (+0.12z)| norm 0.2811 (-0.01z)| lr 4.83e-04 | 322.74 ms | 52.3% bf16 MFU | 1624373 tok/s
step 6192/19560 | loss 3.473091 (-1.05z)| norm 0.2756 (-0.20z)| lr 4.83e-04 | 322.59 ms | 52.3% bf16 MFU | 1624416 tok/s
step 6193/19560 | loss 3.567389 (+1.11z)| norm 0.2974 (+0.53z)| lr 4.83e-04 | 322.41 ms | 52.3% bf16 MFU | 1624504 tok/s
step 6194/19560 | loss 3.487311 (-0.71z)| norm 0.2933 (+0.39z)| lr 4.83e-04 | 323.01 ms | 52.3% bf16 MFU | 1624437 tok/s
step 6195/19560 | loss 3.517513 (-0.01z)| norm 0.3001 (+0.61z)| lr 4.83e-04 | 323.07 ms | 52.2% bf16 MFU | 1624357 tok/s
step 6196/19560 | loss 3.563147 (+1.04z)| norm 0.3030 (+0.70z)| lr 4.83e-04 | 322.69 ms | 52.3% bf16 MFU | 1624375 tok/s
step 6197/19560 | loss 3.483327 (-0.79z)| norm 0.2855 (+0.10z)| lr 4.83e-04 | 323.04 ms | 52.2% bf16 MFU | 1624305 tok/s
step 6198/19560 | loss 3.444286 (-1.65z)| norm 0.2683 (-0.49z)| lr 4.83e-04 | 322.60 ms | 52.3% bf16 MFU | 1624350 tok/s
step 6199/19560 | loss 3.460839 (-1.26z)| norm 0.3199 (+1.25z)| lr 4.83e-04 | 322.41 ms | 52.3% bf16 MFU | 1624440 tok/s
step 6200/19560 | loss 3.461489 (-1.22z)| norm 0.2665 (-0.57z)| lr 4.83e-04 | 322.90 ms | 52.3% bf16 MFU | 1624401 tok/s
step 6201/19560 | loss 3.518786 (+0.07z)| norm 0.2646 (-0.63z)| lr 4.83e-04 | 323.39 ms | 52.2% bf16 MFU | 1624242 tok/s
step 6202/19560 | loss 3.493136 (-0.51z)| norm 0.2929 (+0.34z)| lr 4.83e-04 | 322.56 ms | 52.3% bf16 MFU | 1624299 tok/s
step 6203/19560 | loss 3.557180 (+1.02z)| norm 0.2761 (-0.23z)| lr 4.83e-04 | 323.37 ms | 52.2% bf16 MFU | 1624151 tok/s
step 6204/19560 | loss 3.525204 (+0.25z)| norm 0.2842 (+0.04z)| lr 4.83e-04 | 323.13 ms | 52.2% bf16 MFU | 1624070 tok/s
step 6205/19560 | loss 3.508545 (-0.15z)| norm 0.2781 (-0.16z)| lr 4.83e-04 | 322.51 ms | 52.3% bf16 MFU | 1624149 tok/s
step 6206/19560 | loss 3.522210 (+0.17z)| norm 0.3141 (+1.05z)| lr 4.82e-04 | 322.64 ms | 52.3% bf16 MFU | 1624190 tok/s
step 6207/19560 | loss 3.466894 (-1.14z)| norm 0.3471 (+2.12z)| lr 4.82e-04 | 323.05 ms | 52.2% bf16 MFU | 1624126 tok/s
step 6208/19560 | loss 3.503570 (-0.25z)| norm 0.2756 (-0.27z)| lr 4.82e-04 | 322.77 ms | 52.3% bf16 MFU | 1624136 tok/s
step 6209/19560 | loss 3.500726 (-0.32z)| norm 0.3107 (+0.89z)| lr 4.82e-04 | 322.89 ms | 52.3% bf16 MFU | 1624117 tok/s
step 6210/19560 | loss 3.578284 (+1.53z)| norm 0.3527 (+2.24z)| lr 4.82e-04 | 323.06 ms | 52.2% bf16 MFU | 1624055 tok/s
step 6211/19560 | loss 3.456916 (-1.37z)| norm 0.3071 (+0.73z)| lr 4.82e-04 | 323.21 ms | 52.2% bf16 MFU | 1623960 tok/s
step 6212/19560 | loss 3.533046 (+0.45z)| norm 0.2918 (+0.22z)| lr 4.82e-04 | 322.62 ms | 52.3% bf16 MFU | 1624017 tok/s
step 6213/19560 | loss 3.482838 (-0.74z)| norm 0.3084 (+0.75z)| lr 4.82e-04 | 322.68 ms | 52.3% bf16 MFU | 1624055 tok/s
step 6214/19560 | loss 3.422364 (-2.13z)| norm 0.3157 (+0.98z)| lr 4.82e-04 | 323.27 ms | 52.2% bf16 MFU | 1623945 tok/s
step 6215/19560 | loss 3.528778 (+0.39z)| norm 0.2682 (-0.58z)| lr 4.82e-04 | 322.80 ms | 52.3% bf16 MFU | 1623957 tok/s
step 6216/19560 | loss 3.546431 (+0.80z)| norm 0.3234 (+1.21z)| lr 4.82e-04 | 322.76 ms | 52.3% bf16 MFU | 1623980 tok/s
step 6217/19560 | loss 3.529563 (+0.41z)| norm 0.2756 (-0.33z)| lr 4.82e-04 | 322.85 ms | 52.3% bf16 MFU | 1623976 tok/s
step 6218/19560 | loss 3.531831 (+0.46z)| norm 0.2657 (-0.65z)| lr 4.82e-04 | 323.17 ms | 52.2% bf16 MFU | 1623895 tok/s
step 6219/19560 | loss 3.459624 (-1.24z)| norm 0.3066 (+0.68z)| lr 4.82e-04 | 322.96 ms | 52.3% bf16 MFU | 1623868 tok/s
step 6220/19560 | loss 3.544886 (+0.76z)| norm 0.2843 (-0.05z)| lr 4.82e-04 | 323.01 ms | 52.2% bf16 MFU | 1623831 tok/s
step 6221/19560 | loss 3.530892 (+0.43z)| norm 0.3066 (+0.67z)| lr 4.82e-04 | 323.12 ms | 52.2% bf16 MFU | 1623769 tok/s
step 6222/19560 | loss 3.501812 (-0.27z)| norm 0.2873 (+0.03z)| lr 4.82e-04 | 323.41 ms | 52.2% bf16 MFU | 1623636 tok/s
step 6223/19560 | loss 3.511832 (-0.03z)| norm 0.3153 (+0.94z)| lr 4.82e-04 | 322.93 ms | 52.3% bf16 MFU | 1623630 tok/s
step 6224/19560 | loss 3.521857 (+0.21z)| norm 0.3094 (+0.74z)| lr 4.82e-04 | 322.70 ms | 52.3% bf16 MFU | 1623682 tok/s
step 6225/19560 | loss 3.463304 (-1.18z)| norm 0.2961 (+0.29z)| lr 4.82e-04 | 323.43 ms | 52.2% bf16 MFU | 1623548 tok/s
step 6226/19560 | loss 3.548068 (+0.90z)| norm 0.3118 (+0.80z)| lr 4.82e-04 | 323.44 ms | 52.2% bf16 MFU | 1623420 tok/s
step 6227/19560 | loss 3.598430 (+2.11z)| norm 0.3154 (+0.90z)| lr 4.82e-04 | 322.64 ms | 52.3% bf16 MFU | 1623499 tok/s
step 6228/19560 | loss 3.501799 (-0.29z)| norm 0.2630 (-0.81z)| lr 4.82e-04 | 322.72 ms | 52.3% bf16 MFU | 1623553 tok/s
step 6229/19560 | loss 3.553926 (+0.99z)| norm 0.3265 (+1.25z)| lr 4.82e-04 | 322.79 ms | 52.3% bf16 MFU | 1623589 tok/s
step 6230/19560 | loss 3.515609 (+0.04z)| norm 0.2613 (-0.88z)| lr 4.82e-04 | 322.44 ms | 52.3% bf16 MFU | 1623710 tok/s
step 6231/19560 | loss 3.497813 (-0.40z)| norm 0.2674 (-0.67z)| lr 4.81e-04 | 322.72 ms | 52.3% bf16 MFU | 1623754 tok/s
step 6232/19560 | loss 3.510133 (-0.10z)| norm 0.2753 (-0.40z)| lr 4.81e-04 | 322.96 ms | 52.3% bf16 MFU | 1623737 tok/s
step 6233/19560 | loss 3.632555 (+2.83z)| norm 0.2589 (-1.12z)| lr 4.81e-04 | 323.04 ms | 52.2% bf16 MFU | 1623700 tok/s
step 6234/19560 | loss 3.499293 (-0.36z)| norm 0.2939 (+0.40z)| lr 4.81e-04 | 323.15 ms | 52.2% bf16 MFU | 1623637 tok/s
step 6235/19560 | loss 3.485709 (-0.68z)| norm 0.2557 (-1.29z)| lr 4.81e-04 | 323.02 ms | 52.2% bf16 MFU | 1623608 tok/s
step 6236/19560 | loss 3.481407 (-0.78z)| norm 0.2733 (-0.48z)| lr 4.81e-04 | 323.20 ms | 52.2% bf16 MFU | 1623536 tok/s
step 6237/19560 | loss 3.498694 (-0.36z)| norm 0.2573 (-1.20z)| lr 4.81e-04 | 323.54 ms | 52.2% bf16 MFU | 1623381 tok/s
step 6238/19560 | loss 3.482422 (-0.74z)| norm 0.2708 (-0.57z)| lr 4.81e-04 | 323.06 ms | 52.2% bf16 MFU | 1623357 tok/s
step 6239/19560 | loss 3.539457 (+0.65z)| norm 0.2971 (+0.67z)| lr 4.81e-04 | 323.82 ms | 52.1% bf16 MFU | 1623142 tok/s
step 6240/19560 | loss 3.551489 (+0.96z)| norm 0.3045 (+1.01z)| lr 4.81e-04 | 323.08 ms | 52.2% bf16 MFU | 1623123 tok/s
step 6241/19560 | loss 3.557109 (+1.09z)| norm 0.2665 (-0.76z)| lr 4.81e-04 | 322.80 ms | 52.3% bf16 MFU | 1623177 tok/s
step 6242/19560 | loss 3.540012 (+0.66z)| norm 0.2618 (-0.97z)| lr 4.81e-04 | 323.59 ms | 52.2% bf16 MFU | 1623028 tok/s
step 6243/19560 | loss 3.431859 (-1.96z)| norm 0.2677 (-0.69z)| lr 4.81e-04 | 322.90 ms | 52.3% bf16 MFU | 1623061 tok/s
step 6244/19560 | loss 3.464074 (-1.17z)| norm 0.2811 (-0.07z)| lr 4.81e-04 | 323.25 ms | 52.2% bf16 MFU | 1623005 tok/s
step 6245/19560 | loss 3.685606 (+3.89z)| norm 0.2709 (-0.55z)| lr 4.81e-04 | 322.78 ms | 52.3% bf16 MFU | 1623069 tok/s
step 6246/19560 | loss 3.492090 (-0.48z)| norm 0.2918 (+0.42z)| lr 4.81e-04 | 323.55 ms | 52.2% bf16 MFU | 1622937 tok/s
step 6247/19560 | loss 3.499373 (-0.30z)| norm 0.2497 (-1.54z)| lr 4.81e-04 | 323.48 ms | 52.2% bf16 MFU | 1622828 tok/s
step 6248/19560 | loss 3.463372 (-1.14z)| norm 0.2872 (+0.22z)| lr 4.81e-04 | 323.57 ms | 52.2% bf16 MFU | 1622702 tok/s
step 6249/19560 | loss 3.483726 (-0.65z)| norm 0.2401 (-1.96z)| lr 4.81e-04 | 322.81 ms | 52.3% bf16 MFU | 1622773 tok/s
step 6250/19560 | loss 3.507961 (-0.06z)| norm 0.2687 (-0.63z)| lr 4.81e-04 | 322.54 ms | 52.3% bf16 MFU | 1622909 tok/s
val loss 3.503370
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2794/10042 = 0.278231
step 6251/19560 | loss 3.579242 (+1.61z)| norm 0.2371 (-2.08z)| lr 4.81e-04 | 323.85 ms | 52.1% bf16 MFU | 1622709 tok/s
step 6252/19560 | loss 3.442970 (-1.61z)| norm 0.3622 (+3.48z)| lr 4.81e-04 | 322.18 ms | 52.4% bf16 MFU | 1622939 tok/s
step 6253/19560 | loss 3.506087 (-0.12z)| norm 0.3077 (+1.08z)| lr 4.81e-04 | 322.61 ms | 52.3% bf16 MFU | 1623049 tok/s
step 6254/19560 | loss 3.560266 (+1.14z)| norm 0.3157 (+1.40z)| lr 4.81e-04 | 322.55 ms | 52.3% bf16 MFU | 1623170 tok/s
step 6255/19560 | loss 3.490836 (-0.50z)| norm 0.2822 (-0.06z)| lr 4.81e-04 | 322.19 ms | 52.4% bf16 MFU | 1623374 tok/s
step 6256/19560 | loss 3.476043 (-0.85z)| norm 0.2842 (+0.02z)| lr 4.80e-04 | 322.09 ms | 52.4% bf16 MFU | 1623593 tok/s
step 6257/19560 | loss 3.492730 (-0.44z)| norm 0.2785 (-0.24z)| lr 4.80e-04 | 322.54 ms | 52.3% bf16 MFU | 1623688 tok/s
step 6258/19560 | loss 3.510886 (-0.01z)| norm 0.2720 (-0.53z)| lr 4.80e-04 | 322.32 ms | 52.4% bf16 MFU | 1623835 tok/s
step 6259/19560 | loss 3.518633 (+0.16z)| norm 0.2564 (-1.22z)| lr 4.80e-04 | 322.69 ms | 52.3% bf16 MFU | 1623882 tok/s
step 6260/19560 | loss 3.477183 (-0.82z)| norm 0.2573 (-1.17z)| lr 4.80e-04 | 322.61 ms | 52.3% bf16 MFU | 1623945 tok/s
step 6261/19560 | loss 3.644023 (+3.03z)| norm 0.2911 (+0.31z)| lr 4.80e-04 | 322.75 ms | 52.3% bf16 MFU | 1623968 tok/s
step 6262/19560 | loss 3.529314 (+0.37z)| norm 0.3598 (+3.18z)| lr 4.80e-04 | 322.41 ms | 52.3% bf16 MFU | 1624079 tok/s
step 6263/19560 | loss 3.510019 (-0.07z)| norm 0.2862 (+0.05z)| lr 4.80e-04 | 322.83 ms | 52.3% bf16 MFU | 1624077 tok/s
step 6264/19560 | loss 3.410561 (-2.34z)| norm 0.2669 (-0.78z)| lr 4.80e-04 | 322.63 ms | 52.3% bf16 MFU | 1624125 tok/s
step 6265/19560 | loss 3.529505 (+0.37z)| norm 0.2527 (-1.37z)| lr 4.80e-04 | 322.29 ms | 52.4% bf16 MFU | 1624256 tok/s
step 6266/19560 | loss 3.498094 (-0.36z)| norm 0.2963 (+0.47z)| lr 4.80e-04 | 322.58 ms | 52.3% bf16 MFU | 1624308 tok/s
step 6267/19560 | loss 3.497514 (-0.37z)| norm 0.2666 (-0.78z)| lr 4.80e-04 | 322.66 ms | 52.3% bf16 MFU | 1624337 tok/s
step 6268/19560 | loss 3.471319 (-0.97z)| norm 0.2893 (+0.17z)| lr 4.80e-04 | 322.65 ms | 52.3% bf16 MFU | 1624367 tok/s
step 6269/19560 | loss 3.501080 (-0.28z)| norm 0.2613 (-1.03z)| lr 4.80e-04 | 322.55 ms | 52.3% bf16 MFU | 1624421 tok/s
step 6270/19560 | loss 3.461044 (-1.20z)| norm 0.2730 (-0.54z)| lr 4.80e-04 | 322.59 ms | 52.3% bf16 MFU | 1624462 tok/s
step 6271/19560 | loss 3.834426 (+6.17z)| norm 0.3057 (+0.86z)| lr 4.80e-04 | 322.89 ms | 52.3% bf16 MFU | 1624425 tok/s
step 6272/19560 | loss 3.558135 (+0.81z)| norm 0.2900 (+0.18z)| lr 4.80e-04 | 322.93 ms | 52.3% bf16 MFU | 1624380 tok/s
step 6273/19560 | loss 3.484955 (-0.61z)| norm 0.2566 (-1.26z)| lr 4.80e-04 | 322.50 ms | 52.3% bf16 MFU | 1624447 tok/s
step 6274/19560 | loss 3.464944 (-0.98z)| norm 0.2474 (-1.64z)| lr 4.80e-04 | 322.91 ms | 52.3% bf16 MFU | 1624406 tok/s
step 6275/19560 | loss 3.488451 (-0.52z)| norm 0.2744 (-0.48z)| lr 4.80e-04 | 322.50 ms | 52.3% bf16 MFU | 1624472 tok/s
step 6276/19560 | loss 3.519331 (+0.08z)| norm 0.2512 (-1.45z)| lr 4.80e-04 | 322.55 ms | 52.3% bf16 MFU | 1624519 tok/s
step 6277/19560 | loss 3.640235 (+2.35z)| norm 0.3289 (+1.82z)| lr 4.80e-04 | 322.94 ms | 52.3% bf16 MFU | 1624467 tok/s
step 6278/19560 | loss 3.530055 (+0.26z)| norm 0.2817 (-0.16z)| lr 4.80e-04 | 322.50 ms | 52.3% bf16 MFU | 1624529 tok/s
step 6279/19560 | loss 3.493374 (-0.43z)| norm 0.2446 (-1.69z)| lr 4.80e-04 | 322.18 ms | 52.4% bf16 MFU | 1624669 tok/s
step 6280/19560 | loss 3.484412 (-0.60z)| norm 0.2752 (-0.42z)| lr 4.80e-04 | 322.93 ms | 52.3% bf16 MFU | 1624611 tok/s
step 6281/19560 | loss 3.499402 (-0.32z)| norm 0.3374 (+2.13z)| lr 4.79e-04 | 322.74 ms | 52.3% bf16 MFU | 1624604 tok/s
step 6282/19560 | loss 3.551251 (+0.66z)| norm 0.2897 (+0.17z)| lr 4.79e-04 | 322.21 ms | 52.4% bf16 MFU | 1624732 tok/s
step 6283/19560 | loss 3.510391 (-0.11z)| norm 0.2629 (-0.93z)| lr 4.79e-04 | 322.35 ms | 52.4% bf16 MFU | 1624818 tok/s
step 6284/19560 | loss 3.466023 (-0.94z)| norm 0.2653 (-0.82z)| lr 4.79e-04 | 322.45 ms | 52.3% bf16 MFU | 1624875 tok/s
step 6285/19560 | loss 3.555554 (+0.77z)| norm 0.2844 (-0.04z)| lr 4.79e-04 | 322.48 ms | 52.3% bf16 MFU | 1624922 tok/s
step 6286/19560 | loss 3.515291 (-0.00z)| norm 0.3000 (+0.60z)| lr 4.79e-04 | 322.59 ms | 52.3% bf16 MFU | 1624937 tok/s
step 6287/19560 | loss 3.484585 (-0.59z)| norm 0.3020 (+0.68z)| lr 4.79e-04 | 322.66 ms | 52.3% bf16 MFU | 1624935 tok/s
step 6288/19560 | loss 3.504854 (-0.20z)| norm 0.2927 (+0.30z)| lr 4.79e-04 | 322.27 ms | 52.4% bf16 MFU | 1625030 tok/s
step 6289/19560 | loss 3.506565 (-0.17z)| norm 0.2872 (+0.07z)| lr 4.79e-04 | 322.32 ms | 52.4% bf16 MFU | 1625110 tok/s
step 6290/19560 | loss 3.447981 (-1.26z)| norm 0.2931 (+0.31z)| lr 4.79e-04 | 322.96 ms | 52.3% bf16 MFU | 1625024 tok/s
step 6291/19560 | loss 3.518585 (+0.09z)| norm 0.2668 (-0.76z)| lr 4.79e-04 | 322.89 ms | 52.3% bf16 MFU | 1624961 tok/s
step 6292/19560 | loss 3.474599 (-0.75z)| norm 0.3097 (+0.97z)| lr 4.79e-04 | 323.10 ms | 52.2% bf16 MFU | 1624847 tok/s
step 6293/19560 | loss 3.553777 (+0.75z)| norm 0.2841 (-0.06z)| lr 4.79e-04 | 322.28 ms | 52.4% bf16 MFU | 1624946 tok/s
step 6294/19560 | loss 3.481963 (-0.62z)| norm 0.2793 (-0.27z)| lr 4.79e-04 | 323.20 ms | 52.2% bf16 MFU | 1624808 tok/s
step 6295/19560 | loss 3.527758 (+0.25z)| norm 0.3033 (+0.71z)| lr 4.79e-04 | 322.40 ms | 52.3% bf16 MFU | 1624878 tok/s
step 6296/19560 | loss 3.506857 (-0.14z)| norm 0.2780 (-0.33z)| lr 4.79e-04 | 322.60 ms | 52.3% bf16 MFU | 1624894 tok/s
step 6297/19560 | loss 3.404366 (-2.06z)| norm 0.2648 (-0.88z)| lr 4.79e-04 | 321.94 ms | 52.4% bf16 MFU | 1625075 tok/s
step 6298/19560 | loss 3.521998 (+0.16z)| norm 0.2797 (-0.27z)| lr 4.79e-04 | 323.00 ms | 52.3% bf16 MFU | 1624980 tok/s
step 6299/19560 | loss 3.531684 (+0.33z)| norm 0.2565 (-1.21z)| lr 4.79e-04 | 322.66 ms | 52.3% bf16 MFU | 1624976 tok/s
step 6300/19560 | loss 3.454922 (-1.11z)| norm 0.2364 (-2.02z)| lr 4.79e-04 | 322.64 ms | 52.3% bf16 MFU | 1624976 tok/s
step 6301/19560 | loss 3.593203 (+1.48z)| norm 0.2832 (-0.12z)| lr 4.79e-04 | 322.90 ms | 52.3% bf16 MFU | 1624912 tok/s
step 6302/19560 | loss 3.440724 (-1.36z)| norm 0.2899 (+0.15z)| lr 4.79e-04 | 322.18 ms | 52.4% bf16 MFU | 1625033 tok/s
step 6303/19560 | loss 3.482370 (-0.57z)| norm 0.2736 (-0.51z)| lr 4.79e-04 | 322.58 ms | 52.3% bf16 MFU | 1625046 tok/s
step 6304/19560 | loss 3.484906 (-0.52z)| norm 0.2826 (-0.12z)| lr 4.79e-04 | 322.93 ms | 52.3% bf16 MFU | 1624970 tok/s
step 6305/19560 | loss 3.529806 (+0.32z)| norm 0.2598 (-1.06z)| lr 4.79e-04 | 322.58 ms | 52.3% bf16 MFU | 1624987 tok/s
step 6306/19560 | loss 3.455774 (-1.07z)| norm 0.2697 (-0.66z)| lr 4.78e-04 | 322.28 ms | 52.4% bf16 MFU | 1625078 tok/s
step 6307/19560 | loss 3.511266 (-0.02z)| norm 0.2896 (+0.18z)| lr 4.78e-04 | 323.25 ms | 52.2% bf16 MFU | 1624920 tok/s
step 6308/19560 | loss 3.508466 (-0.07z)| norm 0.2400 (-1.86z)| lr 4.78e-04 | 322.49 ms | 52.3% bf16 MFU | 1624961 tok/s
step 6309/19560 | loss 3.486229 (-0.49z)| norm 0.3005 (+0.64z)| lr 4.78e-04 | 322.39 ms | 52.4% bf16 MFU | 1625026 tok/s
step 6310/19560 | loss 3.550902 (+0.73z)| norm 0.2547 (-1.24z)| lr 4.78e-04 | 322.72 ms | 52.3% bf16 MFU | 1625004 tok/s
step 6311/19560 | loss 3.556291 (+0.83z)| norm 0.2708 (-0.56z)| lr 4.78e-04 | 322.57 ms | 52.3% bf16 MFU | 1625022 tok/s
step 6312/19560 | loss 3.544097 (+0.61z)| norm 0.3052 (+0.86z)| lr 4.78e-04 | 322.29 ms | 52.4% bf16 MFU | 1625108 tok/s
step 6313/19560 | loss 3.529960 (+0.33z)| norm 0.2680 (-0.68z)| lr 4.78e-04 | 322.61 ms | 52.3% bf16 MFU | 1625111 tok/s
step 6314/19560 | loss 3.557943 (+0.85z)| norm 0.2828 (-0.07z)| lr 4.78e-04 | 322.49 ms | 52.3% bf16 MFU | 1625142 tok/s
step 6315/19560 | loss 3.471315 (-0.78z)| norm 0.3099 (+1.04z)| lr 4.78e-04 | 323.02 ms | 52.2% bf16 MFU | 1625038 tok/s
step 6316/19560 | loss 3.474493 (-0.71z)| norm 0.2693 (-0.62z)| lr 4.78e-04 | 322.89 ms | 52.3% bf16 MFU | 1624972 tok/s
step 6317/19560 | loss 3.445765 (-1.23z)| norm 0.2577 (-1.09z)| lr 4.78e-04 | 323.11 ms | 52.2% bf16 MFU | 1624855 tok/s
step 6318/19560 | loss 3.504561 (-0.13z)| norm 0.2786 (-0.23z)| lr 4.78e-04 | 322.89 ms | 52.3% bf16 MFU | 1624799 tok/s
step 6319/19560 | loss 3.477028 (-0.64z)| norm 0.2865 (+0.09z)| lr 4.78e-04 | 322.37 ms | 52.4% bf16 MFU | 1624878 tok/s
step 6320/19560 | loss 3.532232 (+0.38z)| norm 0.2718 (-0.51z)| lr 4.78e-04 | 322.19 ms | 52.4% bf16 MFU | 1624996 tok/s
step 6321/19560 | loss 3.507910 (-0.06z)| norm 0.2926 (+0.34z)| lr 4.78e-04 | 322.32 ms | 52.4% bf16 MFU | 1625076 tok/s
step 6322/19560 | loss 3.591430 (+1.48z)| norm 0.2840 (-0.00z)| lr 4.78e-04 | 323.02 ms | 52.2% bf16 MFU | 1624976 tok/s
step 6323/19560 | loss 3.527610 (+0.29z)| norm 0.3034 (+0.79z)| lr 4.78e-04 | 322.80 ms | 52.3% bf16 MFU | 1624935 tok/s
step 6324/19560 | loss 3.484008 (-0.52z)| norm 0.3241 (+1.61z)| lr 4.78e-04 | 322.52 ms | 52.3% bf16 MFU | 1624969 tok/s
step 6325/19560 | loss 3.529647 (+0.33z)| norm 0.3012 (+0.68z)| lr 4.78e-04 | 322.55 ms | 52.3% bf16 MFU | 1624993 tok/s
step 6326/19560 | loss 3.512085 (-0.01z)| norm 0.3076 (+0.92z)| lr 4.78e-04 | 323.10 ms | 52.2% bf16 MFU | 1624876 tok/s
step 6327/19560 | loss 3.557331 (+0.83z)| norm 0.2926 (+0.33z)| lr 4.78e-04 | 322.74 ms | 52.3% bf16 MFU | 1624856 tok/s
step 6328/19560 | loss 3.536378 (+0.43z)| norm 0.2823 (-0.10z)| lr 4.78e-04 | 322.24 ms | 52.4% bf16 MFU | 1624962 tok/s
step 6329/19560 | loss 3.529413 (+0.29z)| norm 0.2643 (-0.83z)| lr 4.78e-04 | 322.46 ms | 52.3% bf16 MFU | 1625010 tok/s
step 6330/19560 | loss 3.522672 (+0.16z)| norm 0.2959 (+0.46z)| lr 4.78e-04 | 322.96 ms | 52.3% bf16 MFU | 1624928 tok/s
step 6331/19560 | loss 3.475789 (-0.72z)| norm 0.2983 (+0.55z)| lr 4.77e-04 | 322.91 ms | 52.3% bf16 MFU | 1624865 tok/s
step 6332/19560 | loss 3.522699 (+0.17z)| norm 0.2539 (-1.24z)| lr 4.77e-04 | 323.04 ms | 52.2% bf16 MFU | 1624769 tok/s
step 6333/19560 | loss 3.412520 (-1.88z)| norm 0.2777 (-0.28z)| lr 4.77e-04 | 323.13 ms | 52.2% bf16 MFU | 1624656 tok/s
step 6334/19560 | loss 3.522545 (+0.18z)| norm 0.2480 (-1.46z)| lr 4.77e-04 | 322.45 ms | 52.3% bf16 MFU | 1624722 tok/s
step 6335/19560 | loss 3.504967 (-0.15z)| norm 0.2700 (-0.56z)| lr 4.77e-04 | 322.37 ms | 52.4% bf16 MFU | 1624805 tok/s
step 6336/19560 | loss 3.483689 (-0.55z)| norm 0.2960 (+0.51z)| lr 4.77e-04 | 322.81 ms | 52.3% bf16 MFU | 1624771 tok/s
step 6337/19560 | loss 3.531753 (+0.35z)| norm 0.2521 (-1.28z)| lr 4.77e-04 | 323.23 ms | 52.2% bf16 MFU | 1624634 tok/s
step 6338/19560 | loss 3.608474 (+1.77z)| norm 0.2768 (-0.25z)| lr 4.77e-04 | 322.99 ms | 52.3% bf16 MFU | 1624563 tok/s
step 6339/19560 | loss 3.464659 (-0.91z)| norm 0.2662 (-0.69z)| lr 4.77e-04 | 322.53 ms | 52.3% bf16 MFU | 1624612 tok/s
step 6340/19560 | loss 3.522377 (+0.17z)| norm 0.3090 (+1.14z)| lr 4.77e-04 | 322.92 ms | 52.3% bf16 MFU | 1624562 tok/s
step 6341/19560 | loss 3.468266 (-0.84z)| norm 0.2556 (-1.13z)| lr 4.77e-04 | 322.71 ms | 52.3% bf16 MFU | 1624566 tok/s
step 6342/19560 | loss 3.492636 (-0.40z)| norm 0.2709 (-0.46z)| lr 4.77e-04 | 323.13 ms | 52.2% bf16 MFU | 1624464 tok/s
step 6343/19560 | loss 3.462322 (-0.96z)| norm 0.2590 (-0.97z)| lr 4.77e-04 | 322.98 ms | 52.3% bf16 MFU | 1624405 tok/s
step 6344/19560 | loss 3.527515 (+0.27z)| norm 0.2444 (-1.57z)| lr 4.77e-04 | 322.85 ms | 52.3% bf16 MFU | 1624381 tok/s
step 6345/19560 | loss 3.442552 (-1.30z)| norm 0.2616 (-0.83z)| lr 4.77e-04 | 323.56 ms | 52.2% bf16 MFU | 1624180 tok/s
step 6346/19560 | loss 3.504592 (-0.14z)| norm 0.2792 (-0.07z)| lr 4.77e-04 | 322.35 ms | 52.4% bf16 MFU | 1624295 tok/s
step 6347/19560 | loss 3.517442 (+0.09z)| norm 0.2729 (-0.33z)| lr 4.77e-04 | 322.71 ms | 52.3% bf16 MFU | 1624312 tok/s
step 6348/19560 | loss 3.512998 (+0.01z)| norm 0.2414 (-1.66z)| lr 4.77e-04 | 323.63 ms | 52.1% bf16 MFU | 1624098 tok/s
step 6349/19560 | loss 3.518306 (+0.11z)| norm 0.2642 (-0.68z)| lr 4.77e-04 | 322.31 ms | 52.4% bf16 MFU | 1624226 tok/s
step 6350/19560 | loss 3.460321 (-0.97z)| norm 0.2560 (-1.01z)| lr 4.77e-04 | 323.07 ms | 52.2% bf16 MFU | 1624157 tok/s
step 6351/19560 | loss 3.544518 (+0.60z)| norm 0.2555 (-1.02z)| lr 4.77e-04 | 322.89 ms | 52.3% bf16 MFU | 1624135 tok/s
step 6352/19560 | loss 3.512131 (-0.00z)| norm 0.2812 (+0.09z)| lr 4.77e-04 | 322.85 ms | 52.3% bf16 MFU | 1624126 tok/s
step 6353/19560 | loss 3.523556 (+0.20z)| norm 0.2494 (-1.26z)| lr 4.77e-04 | 322.38 ms | 52.4% bf16 MFU | 1624236 tok/s
step 6354/19560 | loss 3.478415 (-0.63z)| norm 0.2654 (-0.56z)| lr 4.77e-04 | 322.46 ms | 52.3% bf16 MFU | 1624318 tok/s
step 6355/19560 | loss 3.490077 (-0.40z)| norm 0.2699 (-0.35z)| lr 4.76e-04 | 323.16 ms | 52.2% bf16 MFU | 1624220 tok/s
step 6356/19560 | loss 3.502719 (-0.16z)| norm 0.2594 (-0.81z)| lr 4.76e-04 | 322.99 ms | 52.3% bf16 MFU | 1624170 tok/s
step 6357/19560 | loss 3.509610 (-0.02z)| norm 0.2836 (+0.27z)| lr 4.76e-04 | 323.57 ms | 52.2% bf16 MFU | 1623978 tok/s
step 6358/19560 | loss 3.447659 (-1.18z)| norm 0.2561 (-0.95z)| lr 4.76e-04 | 322.72 ms | 52.3% bf16 MFU | 1624008 tok/s
step 6359/19560 | loss 3.528549 (+0.34z)| norm 0.2796 (+0.08z)| lr 4.76e-04 | 323.06 ms | 52.2% bf16 MFU | 1623950 tok/s
step 6360/19560 | loss 3.485134 (-0.48z)| norm 0.2599 (-0.78z)| lr 4.76e-04 | 322.86 ms | 52.3% bf16 MFU | 1623947 tok/s
step 6361/19560 | loss 3.500717 (-0.17z)| norm 0.2701 (-0.34z)| lr 4.76e-04 | 323.59 ms | 52.2% bf16 MFU | 1623762 tok/s
step 6362/19560 | loss 3.468061 (-0.79z)| norm 0.2732 (-0.19z)| lr 4.76e-04 | 322.49 ms | 52.3% bf16 MFU | 1623861 tok/s
step 6363/19560 | loss 3.689054 (+3.28z)| norm 0.3375 (+2.58z)| lr 4.76e-04 | 322.79 ms | 52.3% bf16 MFU | 1623881 tok/s
step 6364/19560 | loss 3.504886 (-0.11z)| norm 0.3427 (+2.70z)| lr 4.76e-04 | 322.77 ms | 52.3% bf16 MFU | 1623904 tok/s
step 6365/19560 | loss 3.497630 (-0.24z)| norm 0.3080 (+1.22z)| lr 4.76e-04 | 322.68 ms | 52.3% bf16 MFU | 1623949 tok/s
step 6366/19560 | loss 3.501081 (-0.18z)| norm 0.2563 (-0.95z)| lr 4.76e-04 | 323.06 ms | 52.2% bf16 MFU | 1623894 tok/s
step 6367/19560 | loss 3.475090 (-0.65z)| norm 0.3575 (+3.15z)| lr 4.76e-04 | 322.51 ms | 52.3% bf16 MFU | 1623983 tok/s
step 6368/19560 | loss 3.512761 (+0.05z)| norm 0.3382 (+2.32z)| lr 4.76e-04 | 323.08 ms | 52.2% bf16 MFU | 1623923 tok/s
step 6369/19560 | loss 3.551306 (+0.76z)| norm 0.3931 (+4.15z)| lr 4.76e-04 | 323.20 ms | 52.2% bf16 MFU | 1623834 tok/s
step 6370/19560 | loss 3.481289 (-0.53z)| norm 0.2713 (-0.35z)| lr 4.76e-04 | 322.76 ms | 52.3% bf16 MFU | 1623863 tok/s
step 6371/19560 | loss 3.517744 (+0.14z)| norm 0.3176 (+1.34z)| lr 4.76e-04 | 323.41 ms | 52.2% bf16 MFU | 1623726 tok/s
step 6372/19560 | loss 3.516292 (+0.10z)| norm 0.2922 (+0.40z)| lr 4.76e-04 | 322.69 ms | 52.3% bf16 MFU | 1623778 tok/s
step 6373/19560 | loss 3.520826 (+0.22z)| norm 0.2904 (+0.33z)| lr 4.76e-04 | 323.38 ms | 52.2% bf16 MFU | 1623652 tok/s
step 6374/19560 | loss 3.549261 (+0.77z)| norm 0.3013 (+0.73z)| lr 4.76e-04 | 322.70 ms | 52.3% bf16 MFU | 1623703 tok/s
step 6375/19560 | loss 3.491912 (-0.35z)| norm 0.2660 (-0.58z)| lr 4.76e-04 | 323.14 ms | 52.2% bf16 MFU | 1623643 tok/s
step 6376/19560 | loss 3.569279 (+1.14z)| norm 0.2790 (-0.09z)| lr 4.76e-04 | 323.12 ms | 52.2% bf16 MFU | 1623590 tok/s
step 6377/19560 | loss 3.475944 (-0.68z)| norm 0.2675 (-0.53z)| lr 4.76e-04 | 323.11 ms | 52.2% bf16 MFU | 1623542 tok/s
step 6378/19560 | loss 3.497231 (-0.26z)| norm 0.2477 (-1.25z)| lr 4.76e-04 | 323.14 ms | 52.2% bf16 MFU | 1623489 tok/s
step 6379/19560 | loss 3.556972 (+0.91z)| norm 0.2649 (-0.63z)| lr 4.76e-04 | 322.44 ms | 52.3% bf16 MFU | 1623615 tok/s
step 6380/19560 | loss 3.475874 (-0.68z)| norm 0.2641 (-0.65z)| lr 4.75e-04 | 322.49 ms | 52.3% bf16 MFU | 1623722 tok/s
step 6381/19560 | loss 3.494365 (-0.32z)| norm 0.2586 (-0.85z)| lr 4.75e-04 | 322.70 ms | 52.3% bf16 MFU | 1623770 tok/s
step 6382/19560 | loss 3.519013 (+0.17z)| norm 0.2534 (-1.04z)| lr 4.75e-04 | 322.54 ms | 52.3% bf16 MFU | 1623855 tok/s
step 6383/19560 | loss 3.508252 (-0.04z)| norm 0.2550 (-0.96z)| lr 4.75e-04 | 323.54 ms | 52.2% bf16 MFU | 1623686 tok/s
step 6384/19560 | loss 3.418260 (-1.78z)| norm 0.2533 (-1.02z)| lr 4.75e-04 | 322.71 ms | 52.3% bf16 MFU | 1623732 tok/s
step 6385/19560 | loss 3.521356 (+0.22z)| norm 0.2553 (-0.93z)| lr 4.75e-04 | 322.75 ms | 52.3% bf16 MFU | 1623769 tok/s
step 6386/19560 | loss 3.465423 (-0.86z)| norm 0.2559 (-0.90z)| lr 4.75e-04 | 322.34 ms | 52.4% bf16 MFU | 1623906 tok/s
step 6387/19560 | loss 3.552522 (+0.82z)| norm 0.2725 (-0.27z)| lr 4.75e-04 | 322.83 ms | 52.3% bf16 MFU | 1623911 tok/s
step 6388/19560 | loss 3.538181 (+0.53z)| norm 0.2917 (+0.46z)| lr 4.75e-04 | 323.88 ms | 52.1% bf16 MFU | 1623655 tok/s
step 6389/19560 | loss 3.489854 (-0.39z)| norm 0.2709 (-0.33z)| lr 4.75e-04 | 322.79 ms | 52.3% bf16 MFU | 1623684 tok/s
step 6390/19560 | loss 3.553474 (+0.87z)| norm 0.2485 (-1.20z)| lr 4.75e-04 | 323.13 ms | 52.2% bf16 MFU | 1623626 tok/s
step 6391/19560 | loss 3.483795 (-0.51z)| norm 0.2483 (-1.19z)| lr 4.75e-04 | 322.70 ms | 52.3% bf16 MFU | 1623680 tok/s
step 6392/19560 | loss 3.499374 (-0.21z)| norm 0.2622 (-0.64z)| lr 4.75e-04 | 323.04 ms | 52.2% bf16 MFU | 1623646 tok/s
step 6393/19560 | loss 3.514546 (+0.09z)| norm 0.2596 (-0.75z)| lr 4.75e-04 | 323.09 ms | 52.2% bf16 MFU | 1623601 tok/s
step 6394/19560 | loss 3.561776 (+1.03z)| norm 0.2457 (-1.27z)| lr 4.75e-04 | 322.99 ms | 52.3% bf16 MFU | 1623583 tok/s
step 6395/19560 | loss 3.413295 (-1.91z)| norm 0.2946 (+0.64z)| lr 4.75e-04 | 323.04 ms | 52.2% bf16 MFU | 1623554 tok/s
step 6396/19560 | loss 3.521156 (+0.22z)| norm 0.3163 (+1.48z)| lr 4.75e-04 | 323.29 ms | 52.2% bf16 MFU | 1623462 tok/s
step 6397/19560 | loss 3.471858 (-0.75z)| norm 0.3055 (+1.04z)| lr 4.75e-04 | 322.67 ms | 52.3% bf16 MFU | 1623532 tok/s
step 6398/19560 | loss 3.503160 (-0.14z)| norm 0.3050 (+1.01z)| lr 4.75e-04 | 323.71 ms | 52.1% bf16 MFU | 1623336 tok/s
step 6399/19560 | loss 3.559131 (+1.22z)| norm 0.2817 (+0.11z)| lr 4.75e-04 | 322.65 ms | 52.3% bf16 MFU | 1623416 tok/s
step 6400/19560 | loss 3.461855 (-1.09z)| norm 0.2930 (+0.55z)| lr 4.75e-04 | 322.56 ms | 52.3% bf16 MFU | 1623515 tok/s
step 6401/19560 | loss 3.481793 (-0.61z)| norm 0.2646 (-0.56z)| lr 4.75e-04 | 322.82 ms | 52.3% bf16 MFU | 1623544 tok/s
step 6402/19560 | loss 3.532154 (+0.59z)| norm 0.2892 (+0.39z)| lr 4.75e-04 | 322.62 ms | 52.3% bf16 MFU | 1623622 tok/s
step 6403/19560 | loss 3.488174 (-0.47z)| norm 0.2575 (-0.85z)| lr 4.75e-04 | 322.72 ms | 52.3% bf16 MFU | 1623671 tok/s
step 6404/19560 | loss 3.466388 (-0.99z)| norm 0.2538 (-1.00z)| lr 4.75e-04 | 323.05 ms | 52.2% bf16 MFU | 1623634 tok/s
step 6405/19560 | loss 3.459461 (-1.16z)| norm 0.2661 (-0.50z)| lr 4.74e-04 | 322.92 ms | 52.3% bf16 MFU | 1623633 tok/s
step 6406/19560 | loss 3.501045 (-0.12z)| norm 0.2805 (+0.07z)| lr 4.74e-04 | 322.99 ms | 52.3% bf16 MFU | 1623613 tok/s
step 6407/19560 | loss 3.473480 (-0.80z)| norm 0.2608 (-0.72z)| lr 4.74e-04 | 323.13 ms | 52.2% bf16 MFU | 1623558 tok/s
step 6408/19560 | loss 3.584207 (+1.91z)| norm 0.3056 (+1.06z)| lr 4.74e-04 | 323.33 ms | 52.2% bf16 MFU | 1623458 tok/s
step 6409/19560 | loss 3.516081 (+0.23z)| norm 0.2735 (-0.21z)| lr 4.74e-04 | 322.62 ms | 52.3% bf16 MFU | 1623540 tok/s
step 6410/19560 | loss 3.519814 (+0.33z)| norm 0.3081 (+1.19z)| lr 4.74e-04 | 322.95 ms | 52.3% bf16 MFU | 1623536 tok/s
step 6411/19560 | loss 3.525673 (+0.47z)| norm 0.2922 (+0.54z)| lr 4.74e-04 | 322.74 ms | 52.3% bf16 MFU | 1623583 tok/s
step 6412/19560 | loss 3.540098 (+0.82z)| norm 0.2946 (+0.62z)| lr 4.74e-04 | 322.93 ms | 52.3% bf16 MFU | 1623581 tok/s
step 6413/19560 | loss 3.499604 (-0.17z)| norm 0.2801 (+0.04z)| lr 4.74e-04 | 323.14 ms | 52.2% bf16 MFU | 1623525 tok/s
step 6414/19560 | loss 3.458833 (-1.17z)| norm 0.2822 (+0.13z)| lr 4.74e-04 | 323.19 ms | 52.2% bf16 MFU | 1623461 tok/s
step 6415/19560 | loss 3.514653 (+0.20z)| norm 0.3057 (+1.08z)| lr 4.74e-04 | 322.27 ms | 52.4% bf16 MFU | 1623630 tok/s
step 6416/19560 | loss 3.525124 (+0.46z)| norm 0.2812 (+0.09z)| lr 4.74e-04 | 322.53 ms | 52.3% bf16 MFU | 1623726 tok/s
step 6417/19560 | loss 3.478228 (-0.69z)| norm 0.2837 (+0.20z)| lr 4.74e-04 | 322.76 ms | 52.3% bf16 MFU | 1623759 tok/s
step 6418/19560 | loss 3.545582 (+0.95z)| norm 0.2675 (-0.46z)| lr 4.74e-04 | 322.60 ms | 52.3% bf16 MFU | 1623831 tok/s
step 6419/19560 | loss 3.534852 (+0.68z)| norm 0.3255 (+1.85z)| lr 4.74e-04 | 322.57 ms | 52.3% bf16 MFU | 1623907 tok/s
step 6420/19560 | loss 3.439663 (-1.65z)| norm 0.2615 (-0.70z)| lr 4.74e-04 | 322.32 ms | 52.4% bf16 MFU | 1624043 tok/s
step 6421/19560 | loss 3.530633 (+0.59z)| norm 0.2881 (+0.37z)| lr 4.74e-04 | 322.46 ms | 52.3% bf16 MFU | 1624136 tok/s
step 6422/19560 | loss 3.525791 (+0.46z)| norm 0.2793 (+0.02z)| lr 4.74e-04 | 322.54 ms | 52.3% bf16 MFU | 1624203 tok/s
step 6423/19560 | loss 3.470103 (-0.90z)| norm 0.2748 (-0.15z)| lr 4.74e-04 | 322.48 ms | 52.3% bf16 MFU | 1624283 tok/s
step 6424/19560 | loss 3.461529 (-1.09z)| norm 0.2546 (-0.96z)| lr 4.74e-04 | 323.04 ms | 52.2% bf16 MFU | 1624217 tok/s
step 6425/19560 | loss 3.450467 (-1.40z)| norm 0.2531 (-1.01z)| lr 4.74e-04 | 322.59 ms | 52.3% bf16 MFU | 1624269 tok/s
step 6426/19560 | loss 3.514693 (+0.20z)| norm 0.2542 (-0.96z)| lr 4.74e-04 | 322.52 ms | 52.3% bf16 MFU | 1624335 tok/s
step 6427/19560 | loss 3.524664 (+0.45z)| norm 0.2561 (-0.88z)| lr 4.74e-04 | 322.39 ms | 52.4% bf16 MFU | 1624431 tok/s
step 6428/19560 | loss 3.515229 (+0.21z)| norm 0.2747 (-0.15z)| lr 4.74e-04 | 323.26 ms | 52.2% bf16 MFU | 1624303 tok/s
step 6429/19560 | loss 3.469795 (-0.92z)| norm 0.2541 (-0.97z)| lr 4.73e-04 | 322.14 ms | 52.4% bf16 MFU | 1624462 tok/s
step 6430/19560 | loss 3.447809 (-1.49z)| norm 0.2603 (-0.71z)| lr 4.73e-04 | 322.70 ms | 52.3% bf16 MFU | 1624473 tok/s
step 6431/19560 | loss 3.462637 (-1.10z)| norm 0.2744 (-0.14z)| lr 4.73e-04 | 322.67 ms | 52.3% bf16 MFU | 1624490 tok/s
step 6432/19560 | loss 3.501885 (-0.11z)| norm 0.3132 (+1.40z)| lr 4.73e-04 | 322.49 ms | 52.3% bf16 MFU | 1624553 tok/s
step 6433/19560 | loss 3.520355 (+0.37z)| norm 0.2818 (+0.14z)| lr 4.73e-04 | 323.15 ms | 52.2% bf16 MFU | 1624447 tok/s
step 6434/19560 | loss 3.490232 (-0.41z)| norm 0.3042 (+1.02z)| lr 4.73e-04 | 322.40 ms | 52.3% bf16 MFU | 1624536 tok/s
step 6435/19560 | loss 3.489789 (-0.42z)| norm 0.2735 (-0.20z)| lr 4.73e-04 | 322.90 ms | 52.3% bf16 MFU | 1624494 tok/s
step 6436/19560 | loss 3.515174 (+0.23z)| norm 0.2713 (-0.30z)| lr 4.73e-04 | 322.76 ms | 52.3% bf16 MFU | 1624488 tok/s
step 6437/19560 | loss 3.492805 (-0.34z)| norm 0.2981 (+0.78z)| lr 4.73e-04 | 322.53 ms | 52.3% bf16 MFU | 1624541 tok/s
step 6438/19560 | loss 3.569281 (+1.60z)| norm 0.2786 (-0.01z)| lr 4.73e-04 | 322.57 ms | 52.3% bf16 MFU | 1624582 tok/s
step 6439/19560 | loss 3.454220 (-1.31z)| norm 0.2950 (+0.64z)| lr 4.73e-04 | 322.80 ms | 52.3% bf16 MFU | 1624562 tok/s
step 6440/19560 | loss 3.591343 (+2.14z)| norm 0.2797 (+0.03z)| lr 4.73e-04 | 322.52 ms | 52.3% bf16 MFU | 1624614 tok/s
step 6441/19560 | loss 3.536117 (+0.76z)| norm 0.2635 (-0.62z)| lr 4.73e-04 | 322.70 ms | 52.3% bf16 MFU | 1624618 tok/s
step 6442/19560 | loss 3.502386 (-0.08z)| norm 0.2768 (-0.08z)| lr 4.73e-04 | 322.14 ms | 52.4% bf16 MFU | 1624761 tok/s
step 6443/19560 | loss 3.553759 (+1.20z)| norm 0.2660 (-0.51z)| lr 4.73e-04 | 323.19 ms | 52.2% bf16 MFU | 1624635 tok/s
step 6444/19560 | loss 3.491665 (-0.37z)| norm 0.2975 (+0.76z)| lr 4.73e-04 | 322.67 ms | 52.3% bf16 MFU | 1624647 tok/s
step 6445/19560 | loss 3.443342 (-1.59z)| norm 0.2667 (-0.49z)| lr 4.73e-04 | 322.34 ms | 52.4% bf16 MFU | 1624739 tok/s
step 6446/19560 | loss 3.413198 (-2.28z)| norm 0.2659 (-0.52z)| lr 4.73e-04 | 322.62 ms | 52.3% bf16 MFU | 1624758 tok/s
step 6447/19560 | loss 3.530772 (+0.61z)| norm 0.2643 (-0.58z)| lr 4.73e-04 | 322.46 ms | 52.3% bf16 MFU | 1624815 tok/s
step 6448/19560 | loss 3.535524 (+0.73z)| norm 0.2945 (+0.64z)| lr 4.73e-04 | 322.62 ms | 52.3% bf16 MFU | 1624829 tok/s
step 6449/19560 | loss 3.532621 (+0.65z)| norm 0.2839 (+0.21z)| lr 4.73e-04 | 322.65 ms | 52.3% bf16 MFU | 1624835 tok/s
step 6450/19560 | loss 3.599233 (+2.28z)| norm 0.2764 (-0.09z)| lr 4.73e-04 | 322.56 ms | 52.3% bf16 MFU | 1624862 tok/s
step 6451/19560 | loss 3.494525 (-0.28z)| norm 0.2801 (+0.07z)| lr 4.73e-04 | 322.50 ms | 52.3% bf16 MFU | 1624904 tok/s
step 6452/19560 | loss 3.547541 (+1.01z)| norm 0.2717 (-0.26z)| lr 4.73e-04 | 322.59 ms | 52.3% bf16 MFU | 1624919 tok/s
step 6453/19560 | loss 3.509308 (+0.07z)| norm 0.2666 (-0.46z)| lr 4.73e-04 | 322.55 ms | 52.3% bf16 MFU | 1624946 tok/s
step 6454/19560 | loss 3.495454 (-0.26z)| norm 0.2661 (-0.47z)| lr 4.72e-04 | 322.10 ms | 52.4% bf16 MFU | 1625084 tok/s
step 6455/19560 | loss 3.513277 (+0.18z)| norm 0.2507 (-1.10z)| lr 4.72e-04 | 322.56 ms | 52.3% bf16 MFU | 1625099 tok/s
step 6456/19560 | loss 3.508008 (+0.06z)| norm 0.2522 (-1.02z)| lr 4.72e-04 | 322.18 ms | 52.4% bf16 MFU | 1625209 tok/s
step 6457/19560 | loss 3.631407 (+2.99z)| norm 0.2899 (+0.53z)| lr 4.72e-04 | 322.62 ms | 52.3% bf16 MFU | 1625204 tok/s
step 6458/19560 | loss 3.555541 (+1.16z)| norm 0.3072 (+1.24z)| lr 4.72e-04 | 322.52 ms | 52.3% bf16 MFU | 1625222 tok/s
step 6459/19560 | loss 3.550219 (+1.02z)| norm 0.2652 (-0.48z)| lr 4.72e-04 | 322.45 ms | 52.3% bf16 MFU | 1625260 tok/s
step 6460/19560 | loss 3.504487 (-0.06z)| norm 0.2878 (+0.44z)| lr 4.72e-04 | 322.57 ms | 52.3% bf16 MFU | 1625265 tok/s
step 6461/19560 | loss 3.457102 (-1.21z)| norm 0.2608 (-0.67z)| lr 4.72e-04 | 322.22 ms | 52.4% bf16 MFU | 1625357 tok/s
step 6462/19560 | loss 3.476606 (-0.73z)| norm 0.2880 (+0.44z)| lr 4.72e-04 | 322.70 ms | 52.3% bf16 MFU | 1625323 tok/s
step 6463/19560 | loss 3.542258 (+0.84z)| norm 0.3231 (+1.86z)| lr 4.72e-04 | 322.74 ms | 52.3% bf16 MFU | 1625282 tok/s
step 6464/19560 | loss 3.524570 (+0.40z)| norm 0.2984 (+0.85z)| lr 4.72e-04 | 322.50 ms | 52.3% bf16 MFU | 1625303 tok/s
step 6465/19560 | loss 3.458667 (-1.16z)| norm 0.2672 (-0.44z)| lr 4.72e-04 | 322.44 ms | 52.3% bf16 MFU | 1625338 tok/s
step 6466/19560 | loss 3.409258 (-2.31z)| norm 0.2972 (+0.79z)| lr 4.72e-04 | 322.54 ms | 52.3% bf16 MFU | 1625347 tok/s
step 6467/19560 | loss 3.471888 (-0.81z)| norm 0.2742 (-0.16z)| lr 4.72e-04 | 322.82 ms | 52.3% bf16 MFU | 1625283 tok/s
step 6468/19560 | loss 3.532907 (+0.65z)| norm 0.2744 (-0.14z)| lr 4.72e-04 | 322.76 ms | 52.3% bf16 MFU | 1625238 tok/s
step 6469/19560 | loss 3.455320 (-1.20z)| norm 0.2898 (+0.49z)| lr 4.72e-04 | 322.37 ms | 52.4% bf16 MFU | 1625295 tok/s
step 6470/19560 | loss 3.435313 (-1.65z)| norm 0.2529 (-1.03z)| lr 4.72e-04 | 322.50 ms | 52.3% bf16 MFU | 1625316 tok/s
step 6471/19560 | loss 3.524921 (+0.46z)| norm 0.2825 (+0.18z)| lr 4.72e-04 | 322.74 ms | 52.3% bf16 MFU | 1625274 tok/s
step 6472/19560 | loss 3.413302 (-2.13z)| norm 0.2652 (-0.54z)| lr 4.72e-04 | 322.37 ms | 52.4% bf16 MFU | 1625328 tok/s
step 6473/19560 | loss 3.502580 (-0.06z)| norm 0.2616 (-0.70z)| lr 4.72e-04 | 322.53 ms | 52.3% bf16 MFU | 1625340 tok/s
step 6474/19560 | loss 3.454141 (-1.19z)| norm 0.2487 (-1.22z)| lr 4.72e-04 | 322.89 ms | 52.3% bf16 MFU | 1625260 tok/s
step 6475/19560 | loss 3.468341 (-0.84z)| norm 0.2746 (-0.14z)| lr 4.72e-04 | 322.45 ms | 52.3% bf16 MFU | 1625294 tok/s
step 6476/19560 | loss 3.465165 (-0.91z)| norm 0.3201 (+1.71z)| lr 4.72e-04 | 322.59 ms | 52.3% bf16 MFU | 1625292 tok/s
step 6477/19560 | loss 3.447645 (-1.29z)| norm 0.2697 (-0.37z)| lr 4.72e-04 | 322.98 ms | 52.3% bf16 MFU | 1625191 tok/s
step 6478/19560 | loss 3.550739 (+1.07z)| norm 0.2546 (-1.00z)| lr 4.71e-04 | 322.77 ms | 52.3% bf16 MFU | 1625148 tok/s
step 6479/19560 | loss 3.546536 (+0.98z)| norm 0.2700 (-0.37z)| lr 4.71e-04 | 322.56 ms | 52.3% bf16 MFU | 1625162 tok/s
step 6480/19560 | loss 3.483613 (-0.47z)| norm 0.2611 (-0.73z)| lr 4.71e-04 | 322.33 ms | 52.4% bf16 MFU | 1625231 tok/s
step 6481/19560 | loss 3.489116 (-0.34z)| norm 0.2863 (+0.31z)| lr 4.71e-04 | 322.68 ms | 52.3% bf16 MFU | 1625210 tok/s
step 6482/19560 | loss 3.485727 (-0.42z)| norm 0.2626 (-0.68z)| lr 4.71e-04 | 322.39 ms | 52.3% bf16 MFU | 1625262 tok/s
step 6483/19560 | loss 3.536664 (+0.75z)| norm 0.3015 (+0.92z)| lr 4.71e-04 | 322.63 ms | 52.3% bf16 MFU | 1625251 tok/s
step 6484/19560 | loss 3.485210 (-0.44z)| norm 0.2919 (+0.52z)| lr 4.71e-04 | 322.46 ms | 52.3% bf16 MFU | 1625282 tok/s
step 6485/19560 | loss 3.443942 (-1.36z)| norm 0.2783 (-0.04z)| lr 4.71e-04 | 322.83 ms | 52.3% bf16 MFU | 1625219 tok/s
step 6486/19560 | loss 3.515218 (+0.26z)| norm 0.2922 (+0.52z)| lr 4.71e-04 | 322.99 ms | 52.3% bf16 MFU | 1625120 tok/s
step 6487/19560 | loss 3.520895 (+0.39z)| norm 0.2796 (-0.00z)| lr 4.71e-04 | 322.49 ms | 52.3% bf16 MFU | 1625151 tok/s
step 6488/19560 | loss 3.482985 (-0.48z)| norm 0.2987 (+0.78z)| lr 4.71e-04 | 322.74 ms | 52.3% bf16 MFU | 1625118 tok/s
step 6489/19560 | loss 3.459921 (-1.00z)| norm 0.3127 (+1.34z)| lr 4.71e-04 | 323.18 ms | 52.2% bf16 MFU | 1624975 tok/s
step 6490/19560 | loss 3.526147 (+0.50z)| norm 0.2744 (-0.24z)| lr 4.71e-04 | 322.90 ms | 52.3% bf16 MFU | 1624910 tok/s
step 6491/19560 | loss 3.526846 (+0.59z)| norm 0.2669 (-0.54z)| lr 4.71e-04 | 322.42 ms | 52.3% bf16 MFU | 1624971 tok/s
step 6492/19560 | loss 3.517663 (+0.36z)| norm 0.2555 (-1.02z)| lr 4.71e-04 | 322.94 ms | 52.3% bf16 MFU | 1624897 tok/s
step 6493/19560 | loss 3.484746 (-0.45z)| norm 0.2696 (-0.40z)| lr 4.71e-04 | 322.93 ms | 52.3% bf16 MFU | 1624828 tok/s
step 6494/19560 | loss 3.468108 (-0.85z)| norm 0.2589 (-0.87z)| lr 4.71e-04 | 322.80 ms | 52.3% bf16 MFU | 1624794 tok/s
step 6495/19560 | loss 3.494842 (-0.20z)| norm 0.2422 (-1.62z)| lr 4.71e-04 | 322.83 ms | 52.3% bf16 MFU | 1624758 tok/s
step 6496/19560 | loss 3.483396 (-0.47z)| norm 0.2726 (-0.22z)| lr 4.71e-04 | 322.47 ms | 52.3% bf16 MFU | 1624813 tok/s
step 6497/19560 | loss 3.452151 (-1.23z)| norm 0.2710 (-0.29z)| lr 4.71e-04 | 322.92 ms | 52.3% bf16 MFU | 1624751 tok/s
step 6498/19560 | loss 3.487114 (-0.36z)| norm 0.2706 (-0.31z)| lr 4.71e-04 | 322.54 ms | 52.3% bf16 MFU | 1624788 tok/s
step 6499/19560 | loss 3.543884 (+1.03z)| norm 0.2563 (-1.06z)| lr 4.71e-04 | 322.81 ms | 52.3% bf16 MFU | 1624756 tok/s
step 6500/19560 | loss 3.563413 (+1.49z)| norm 0.3553 (+3.99z)| lr 4.71e-04 | 322.26 ms | 52.4% bf16 MFU | 1624865 tok/s
val loss 3.497015
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2820/10042 = 0.280821
step 6501/19560 | loss 3.558582 (+1.35z)| norm 0.3088 (+1.62z)| lr 4.71e-04 | 323.24 ms | 52.2% bf16 MFU | 1624721 tok/s
step 6502/19560 | loss 3.508764 (+0.16z)| norm 0.2917 (+0.77z)| lr 4.71e-04 | 322.18 ms | 52.4% bf16 MFU | 1624851 tok/s
step 6503/19560 | loss 3.439563 (-1.51z)| norm 0.2967 (+1.00z)| lr 4.70e-04 | 322.27 ms | 52.4% bf16 MFU | 1624951 tok/s
step 6504/19560 | loss 3.500138 (-0.03z)| norm 0.3068 (+1.48z)| lr 4.70e-04 | 322.32 ms | 52.4% bf16 MFU | 1625033 tok/s
step 6505/19560 | loss 3.428964 (-1.74z)| norm 0.2887 (+0.57z)| lr 4.70e-04 | 322.69 ms | 52.3% bf16 MFU | 1625018 tok/s
step 6506/19560 | loss 3.448035 (-1.26z)| norm 0.2841 (+0.33z)| lr 4.70e-04 | 323.33 ms | 52.2% bf16 MFU | 1624843 tok/s
step 6507/19560 | loss 3.519491 (+0.46z)| norm 0.2843 (+0.34z)| lr 4.70e-04 | 321.96 ms | 52.4% bf16 MFU | 1625021 tok/s
step 6508/19560 | loss 3.523232 (+0.54z)| norm 0.2641 (-0.68z)| lr 4.70e-04 | 322.78 ms | 52.3% bf16 MFU | 1624984 tok/s
step 6509/19560 | loss 3.517900 (+0.41z)| norm 0.2666 (-0.55z)| lr 4.70e-04 | 321.87 ms | 52.4% bf16 MFU | 1625180 tok/s
step 6510/19560 | loss 3.528118 (+0.65z)| norm 0.2696 (-0.41z)| lr 4.70e-04 | 322.54 ms | 52.3% bf16 MFU | 1625196 tok/s
step 6511/19560 | loss 3.585454 (+1.99z)| norm 0.2862 (+0.42z)| lr 4.70e-04 | 322.69 ms | 52.3% bf16 MFU | 1625172 tok/s
step 6512/19560 | loss 3.479215 (-0.55z)| norm 0.2904 (+0.62z)| lr 4.70e-04 | 322.88 ms | 52.3% bf16 MFU | 1625104 tok/s
step 6513/19560 | loss 3.454189 (-1.14z)| norm 0.2622 (-0.83z)| lr 4.70e-04 | 322.39 ms | 52.4% bf16 MFU | 1625162 tok/s
step 6514/19560 | loss 3.463359 (-0.92z)| norm 0.2823 (+0.19z)| lr 4.70e-04 | 323.12 ms | 52.2% bf16 MFU | 1625032 tok/s
step 6515/19560 | loss 3.509010 (+0.19z)| norm 0.2919 (+0.68z)| lr 4.70e-04 | 322.30 ms | 52.4% bf16 MFU | 1625114 tok/s
step 6516/19560 | loss 3.474294 (-0.64z)| norm 0.2769 (-0.09z)| lr 4.70e-04 | 322.75 ms | 52.3% bf16 MFU | 1625081 tok/s
step 6517/19560 | loss 3.471639 (-0.70z)| norm 0.2796 (+0.05z)| lr 4.70e-04 | 323.69 ms | 52.1% bf16 MFU | 1624813 tok/s
step 6518/19560 | loss 3.474386 (-0.62z)| norm 0.2514 (-1.41z)| lr 4.70e-04 | 322.95 ms | 52.3% bf16 MFU | 1624745 tok/s
step 6519/19560 | loss 3.492502 (-0.18z)| norm 0.2499 (-1.49z)| lr 4.70e-04 | 322.63 ms | 52.3% bf16 MFU | 1624759 tok/s
step 6520/19560 | loss 3.468044 (-0.77z)| norm 0.2778 (-0.05z)| lr 4.70e-04 | 322.58 ms | 52.3% bf16 MFU | 1624786 tok/s
step 6521/19560 | loss 3.479229 (-0.49z)| norm 0.2812 (+0.11z)| lr 4.70e-04 | 322.39 ms | 52.3% bf16 MFU | 1624859 tok/s
step 6522/19560 | loss 3.458899 (-0.97z)| norm 0.2667 (-0.66z)| lr 4.70e-04 | 323.22 ms | 52.2% bf16 MFU | 1624721 tok/s
step 6523/19560 | loss 3.535202 (+0.88z)| norm 0.2888 (+0.51z)| lr 4.70e-04 | 322.61 ms | 52.3% bf16 MFU | 1624741 tok/s
step 6524/19560 | loss 3.507007 (+0.18z)| norm 0.2838 (+0.26z)| lr 4.70e-04 | 322.81 ms | 52.3% bf16 MFU | 1624710 tok/s
step 6525/19560 | loss 3.496344 (-0.08z)| norm 0.2721 (-0.35z)| lr 4.70e-04 | 323.10 ms | 52.2% bf16 MFU | 1624609 tok/s
step 6526/19560 | loss 3.448396 (-1.25z)| norm 0.2631 (-0.82z)| lr 4.70e-04 | 322.48 ms | 52.3% bf16 MFU | 1624670 tok/s
step 6527/19560 | loss 3.524749 (+0.64z)| norm 0.2900 (+0.63z)| lr 4.69e-04 | 322.74 ms | 52.3% bf16 MFU | 1624662 tok/s
step 6528/19560 | loss 3.510035 (+0.26z)| norm 0.2675 (-0.57z)| lr 4.69e-04 | 322.34 ms | 52.4% bf16 MFU | 1624754 tok/s
step 6529/19560 | loss 3.487035 (-0.31z)| norm 0.2890 (+0.58z)| lr 4.69e-04 | 323.17 ms | 52.2% bf16 MFU | 1624634 tok/s
step 6530/19560 | loss 3.508197 (+0.22z)| norm 0.2865 (+0.45z)| lr 4.69e-04 | 322.85 ms | 52.3% bf16 MFU | 1624599 tok/s
step 6531/19560 | loss 3.529490 (+0.74z)| norm 0.2870 (+0.46z)| lr 4.69e-04 | 322.35 ms | 52.4% bf16 MFU | 1624690 tok/s
step 6532/19560 | loss 3.513547 (+0.34z)| norm 0.3283 (+2.62z)| lr 4.69e-04 | 323.26 ms | 52.2% bf16 MFU | 1624549 tok/s
step 6533/19560 | loss 3.533015 (+0.81z)| norm 0.3055 (+1.38z)| lr 4.69e-04 | 322.76 ms | 52.3% bf16 MFU | 1624540 tok/s
step 6534/19560 | loss 3.557156 (+1.39z)| norm 0.2650 (-0.76z)| lr 4.69e-04 | 323.11 ms | 52.2% bf16 MFU | 1624446 tok/s
step 6535/19560 | loss 3.545856 (+1.09z)| norm 0.2737 (-0.30z)| lr 4.69e-04 | 322.43 ms | 52.3% bf16 MFU | 1624528 tok/s
step 6536/19560 | loss 3.535966 (+0.87z)| norm 0.2744 (-0.26z)| lr 4.69e-04 | 322.51 ms | 52.3% bf16 MFU | 1624585 tok/s
step 6537/19560 | loss 3.505132 (+0.10z)| norm 0.3270 (+2.48z)| lr 4.69e-04 | 323.78 ms | 52.1% bf16 MFU | 1624319 tok/s
step 6538/19560 | loss 3.458930 (-1.04z)| norm 0.2826 (+0.17z)| lr 4.69e-04 | 322.35 ms | 52.4% bf16 MFU | 1624425 tok/s
step 6539/19560 | loss 3.465077 (-0.87z)| norm 0.2643 (-0.78z)| lr 4.69e-04 | 322.86 ms | 52.3% bf16 MFU | 1624398 tok/s
step 6540/19560 | loss 3.470192 (-0.73z)| norm 0.2845 (+0.29z)| lr 4.69e-04 | 322.86 ms | 52.3% bf16 MFU | 1624372 tok/s
step 6541/19560 | loss 3.490594 (-0.22z)| norm 0.2664 (-0.66z)| lr 4.69e-04 | 323.08 ms | 52.2% bf16 MFU | 1624292 tok/s
step 6542/19560 | loss 3.448600 (-1.27z)| norm 0.2828 (+0.20z)| lr 4.69e-04 | 323.07 ms | 52.2% bf16 MFU | 1624218 tok/s
step 6543/19560 | loss 3.504993 (+0.14z)| norm 0.2447 (-1.77z)| lr 4.69e-04 | 322.59 ms | 52.3% bf16 MFU | 1624270 tok/s
step 6544/19560 | loss 3.537560 (+0.95z)| norm 0.2927 (+0.74z)| lr 4.69e-04 | 322.78 ms | 52.3% bf16 MFU | 1624270 tok/s
step 6545/19560 | loss 3.518069 (+0.46z)| norm 0.2328 (-2.32z)| lr 4.69e-04 | 323.36 ms | 52.2% bf16 MFU | 1624126 tok/s
step 6546/19560 | loss 3.560839 (+1.51z)| norm 0.2843 (+0.31z)| lr 4.69e-04 | 322.36 ms | 52.4% bf16 MFU | 1624241 tok/s
step 6547/19560 | loss 3.464539 (-0.86z)| norm 0.2751 (-0.15z)| lr 4.69e-04 | 322.80 ms | 52.3% bf16 MFU | 1624237 tok/s
step 6548/19560 | loss 3.532404 (+0.81z)| norm 0.2673 (-0.56z)| lr 4.69e-04 | 322.60 ms | 52.3% bf16 MFU | 1624286 tok/s
step 6549/19560 | loss 3.527705 (+0.69z)| norm 0.2948 (+0.88z)| lr 4.69e-04 | 322.94 ms | 52.3% bf16 MFU | 1624247 tok/s
step 6550/19560 | loss 3.543763 (+1.08z)| norm 0.3232 (+2.30z)| lr 4.69e-04 | 322.38 ms | 52.4% bf16 MFU | 1624351 tok/s
step 6551/19560 | loss 3.551081 (+1.24z)| norm 0.2797 (+0.06z)| lr 4.68e-04 | 322.94 ms | 52.3% bf16 MFU | 1624307 tok/s
step 6552/19560 | loss 3.480423 (-0.51z)| norm 0.2895 (+0.56z)| lr 4.68e-04 | 322.53 ms | 52.3% bf16 MFU | 1624370 tok/s
step 6553/19560 | loss 3.512870 (+0.29z)| norm 0.2923 (+0.69z)| lr 4.68e-04 | 323.07 ms | 52.2% bf16 MFU | 1624292 tok/s
step 6554/19560 | loss 3.529078 (+0.69z)| norm 0.2662 (-0.67z)| lr 4.68e-04 | 322.87 ms | 52.3% bf16 MFU | 1624269 tok/s
step 6555/19560 | loss 3.523256 (+0.54z)| norm 0.2766 (-0.14z)| lr 4.68e-04 | 322.57 ms | 52.3% bf16 MFU | 1624323 tok/s
step 6556/19560 | loss 3.480772 (-0.51z)| norm 0.2722 (-0.37z)| lr 4.68e-04 | 322.60 ms | 52.3% bf16 MFU | 1624367 tok/s
step 6557/19560 | loss 3.459044 (-1.05z)| norm 0.2590 (-1.06z)| lr 4.68e-04 | 322.57 ms | 52.3% bf16 MFU | 1624416 tok/s
step 6558/19560 | loss 3.532449 (+0.76z)| norm 0.2826 (+0.17z)| lr 4.68e-04 | 323.48 ms | 52.2% bf16 MFU | 1624234 tok/s
step 6559/19560 | loss 3.515288 (+0.33z)| norm 0.2908 (+0.59z)| lr 4.68e-04 | 323.01 ms | 52.2% bf16 MFU | 1624179 tok/s
step 6560/19560 | loss 3.444659 (-1.42z)| norm 0.2713 (-0.42z)| lr 4.68e-04 | 322.59 ms | 52.3% bf16 MFU | 1624232 tok/s
step 6561/19560 | loss 3.501350 (-0.01z)| norm 0.2645 (-0.77z)| lr 4.68e-04 | 322.72 ms | 52.3% bf16 MFU | 1624250 tok/s
step 6562/19560 | loss 3.594924 (+2.26z)| norm 0.2728 (-0.32z)| lr 4.68e-04 | 323.05 ms | 52.2% bf16 MFU | 1624184 tok/s
step 6563/19560 | loss 3.448598 (-1.30z)| norm 0.2655 (-0.71z)| lr 4.68e-04 | 323.10 ms | 52.2% bf16 MFU | 1624108 tok/s
step 6564/19560 | loss 3.472637 (-0.71z)| norm 0.2481 (-1.61z)| lr 4.68e-04 | 322.44 ms | 52.3% bf16 MFU | 1624202 tok/s
step 6565/19560 | loss 3.505350 (+0.08z)| norm 0.2473 (-1.62z)| lr 4.68e-04 | 323.05 ms | 52.2% bf16 MFU | 1624139 tok/s
step 6566/19560 | loss 3.451577 (-1.20z)| norm 0.2514 (-1.39z)| lr 4.68e-04 | 322.60 ms | 52.3% bf16 MFU | 1624191 tok/s
step 6567/19560 | loss 3.477405 (-0.58z)| norm 0.2495 (-1.46z)| lr 4.68e-04 | 323.18 ms | 52.2% bf16 MFU | 1624096 tok/s
step 6568/19560 | loss 3.529045 (+0.70z)| norm 0.2640 (-0.70z)| lr 4.68e-04 | 323.03 ms | 52.2% bf16 MFU | 1624043 tok/s
step 6569/19560 | loss 3.463958 (-0.90z)| norm 0.2508 (-1.37z)| lr 4.68e-04 | 322.82 ms | 52.3% bf16 MFU | 1624046 tok/s
step 6570/19560 | loss 3.559810 (+1.46z)| norm 0.2624 (-0.77z)| lr 4.68e-04 | 322.86 ms | 52.3% bf16 MFU | 1624037 tok/s
step 6571/19560 | loss 3.487891 (-0.30z)| norm 0.2541 (-1.18z)| lr 4.68e-04 | 322.53 ms | 52.3% bf16 MFU | 1624112 tok/s
step 6572/19560 | loss 3.502866 (+0.07z)| norm 0.2519 (-1.27z)| lr 4.68e-04 | 322.25 ms | 52.4% bf16 MFU | 1624255 tok/s
step 6573/19560 | loss 3.526915 (+0.65z)| norm 0.2614 (-0.79z)| lr 4.68e-04 | 323.12 ms | 52.2% bf16 MFU | 1624171 tok/s
step 6574/19560 | loss 3.480379 (-0.53z)| norm 0.2563 (-1.04z)| lr 4.68e-04 | 323.12 ms | 52.2% bf16 MFU | 1624092 tok/s
step 6575/19560 | loss 3.515765 (+0.37z)| norm 0.2372 (-1.97z)| lr 4.67e-04 | 322.53 ms | 52.3% bf16 MFU | 1624164 tok/s
step 6576/19560 | loss 3.479016 (-0.56z)| norm 0.3084 (+1.58z)| lr 4.67e-04 | 322.73 ms | 52.3% bf16 MFU | 1624183 tok/s
step 6577/19560 | loss 3.537131 (+0.93z)| norm 0.3031 (+1.30z)| lr 4.67e-04 | 322.88 ms | 52.3% bf16 MFU | 1624164 tok/s
step 6578/19560 | loss 3.452691 (-1.22z)| norm 0.2911 (+0.70z)| lr 4.67e-04 | 323.75 ms | 52.1% bf16 MFU | 1623927 tok/s
step 6579/19560 | loss 3.574175 (+1.89z)| norm 0.2667 (-0.50z)| lr 4.67e-04 | 322.59 ms | 52.3% bf16 MFU | 1623994 tok/s
step 6580/19560 | loss 3.524738 (+0.63z)| norm 0.2893 (+0.61z)| lr 4.67e-04 | 322.67 ms | 52.3% bf16 MFU | 1624035 tok/s
step 6581/19560 | loss 3.472053 (-0.71z)| norm 0.2854 (+0.41z)| lr 4.67e-04 | 322.93 ms | 52.3% bf16 MFU | 1624009 tok/s
step 6582/19560 | loss 3.553572 (+1.36z)| norm 0.2931 (+0.78z)| lr 4.67e-04 | 322.72 ms | 52.3% bf16 MFU | 1624038 tok/s
step 6583/19560 | loss 3.497040 (-0.08z)| norm 0.2917 (+0.70z)| lr 4.67e-04 | 323.27 ms | 52.2% bf16 MFU | 1623928 tok/s
step 6584/19560 | loss 3.453845 (-1.16z)| norm 0.2734 (-0.22z)| lr 4.67e-04 | 323.04 ms | 52.2% bf16 MFU | 1623881 tok/s
step 6585/19560 | loss 3.487979 (-0.28z)| norm 0.2513 (-1.29z)| lr 4.67e-04 | 323.09 ms | 52.2% bf16 MFU | 1623824 tok/s
step 6586/19560 | loss 3.442163 (-1.48z)| norm 0.2789 (+0.08z)| lr 4.67e-04 | 322.84 ms | 52.3% bf16 MFU | 1623832 tok/s
step 6587/19560 | loss 3.463895 (-0.89z)| norm 0.2593 (-0.89z)| lr 4.67e-04 | 322.03 ms | 52.4% bf16 MFU | 1624045 tok/s
step 6588/19560 | loss 3.546850 (+1.31z)| norm 0.2768 (-0.01z)| lr 4.67e-04 | 323.25 ms | 52.2% bf16 MFU | 1623940 tok/s
step 6589/19560 | loss 3.467142 (-0.81z)| norm 0.2584 (-0.93z)| lr 4.67e-04 | 323.09 ms | 52.2% bf16 MFU | 1623880 tok/s
step 6590/19560 | loss 3.491081 (-0.17z)| norm 0.2850 (+0.39z)| lr 4.67e-04 | 321.89 ms | 52.4% bf16 MFU | 1624126 tok/s
step 6591/19560 | loss 3.490320 (-0.18z)| norm 0.2560 (-1.04z)| lr 4.67e-04 | 322.86 ms | 52.3% bf16 MFU | 1624115 tok/s
step 6592/19560 | loss 3.480217 (-0.44z)| norm 0.2650 (-0.57z)| lr 4.67e-04 | 322.04 ms | 52.4% bf16 MFU | 1624310 tok/s
step 6593/19560 | loss 3.577281 (+2.10z)| norm 0.2979 (+1.08z)| lr 4.67e-04 | 322.99 ms | 52.3% bf16 MFU | 1624257 tok/s
step 6594/19560 | loss 3.478175 (-0.54z)| norm 0.3153 (+1.94z)| lr 4.67e-04 | 322.89 ms | 52.3% bf16 MFU | 1624231 tok/s
step 6595/19560 | loss 3.525792 (+0.73z)| norm 0.2583 (-0.91z)| lr 4.67e-04 | 322.54 ms | 52.3% bf16 MFU | 1624295 tok/s
step 6596/19560 | loss 3.493429 (-0.13z)| norm 0.2756 (-0.05z)| lr 4.67e-04 | 322.81 ms | 52.3% bf16 MFU | 1624288 tok/s
step 6597/19560 | loss 3.473796 (-0.67z)| norm 0.3084 (+1.57z)| lr 4.67e-04 | 322.88 ms | 52.3% bf16 MFU | 1624263 tok/s
step 6598/19560 | loss 3.491648 (-0.20z)| norm 0.3022 (+1.25z)| lr 4.67e-04 | 322.69 ms | 52.3% bf16 MFU | 1624288 tok/s
step 6599/19560 | loss 3.538761 (+1.09z)| norm 0.3038 (+1.31z)| lr 4.66e-04 | 322.66 ms | 52.3% bf16 MFU | 1624319 tok/s
step 6600/19560 | loss 3.480623 (-0.53z)| norm 0.3418 (+3.04z)| lr 4.66e-04 | 322.46 ms | 52.3% bf16 MFU | 1624397 tok/s
step 6601/19560 | loss 3.510003 (+0.29z)| norm 0.3007 (+1.07z)| lr 4.66e-04 | 322.44 ms | 52.3% bf16 MFU | 1624478 tok/s
step 6602/19560 | loss 3.518906 (+0.53z)| norm 0.2907 (+0.58z)| lr 4.66e-04 | 322.25 ms | 52.4% bf16 MFU | 1624600 tok/s
step 6603/19560 | loss 3.554514 (+1.50z)| norm 0.3026 (+1.13z)| lr 4.66e-04 | 322.49 ms | 52.3% bf16 MFU | 1624658 tok/s
step 6604/19560 | loss 3.469921 (-0.87z)| norm 0.2952 (+0.80z)| lr 4.66e-04 | 322.22 ms | 52.4% bf16 MFU | 1624781 tok/s
step 6605/19560 | loss 3.509401 (+0.23z)| norm 0.3035 (+1.18z)| lr 4.66e-04 | 322.12 ms | 52.4% bf16 MFU | 1624922 tok/s
step 6606/19560 | loss 3.508693 (+0.22z)| norm 0.2693 (-0.46z)| lr 4.66e-04 | 322.61 ms | 52.3% bf16 MFU | 1624932 tok/s
step 6607/19560 | loss 3.510992 (+0.29z)| norm 0.2790 (-0.00z)| lr 4.66e-04 | 322.80 ms | 52.3% bf16 MFU | 1624895 tok/s
step 6608/19560 | loss 3.467982 (-0.93z)| norm 0.2645 (-0.70z)| lr 4.66e-04 | 322.26 ms | 52.4% bf16 MFU | 1624995 tok/s
step 6609/19560 | loss 3.542950 (+1.19z)| norm 0.2634 (-0.74z)| lr 4.66e-04 | 322.60 ms | 52.3% bf16 MFU | 1625006 tok/s
step 6610/19560 | loss 3.504114 (+0.08z)| norm 0.2866 (+0.37z)| lr 4.66e-04 | 322.56 ms | 52.3% bf16 MFU | 1625026 tok/s
step 6611/19560 | loss 3.479142 (-0.62z)| norm 0.2759 (-0.14z)| lr 4.66e-04 | 322.49 ms | 52.3% bf16 MFU | 1625062 tok/s
step 6612/19560 | loss 3.552871 (+1.46z)| norm 0.2749 (-0.18z)| lr 4.66e-04 | 322.49 ms | 52.3% bf16 MFU | 1625097 tok/s
step 6613/19560 | loss 3.490554 (-0.32z)| norm 0.3900 (+4.83z)| lr 4.66e-04 | 322.39 ms | 52.3% bf16 MFU | 1625153 tok/s
step 6614/19560 | loss 3.519622 (+0.51z)| norm 0.2920 (+0.54z)| lr 4.66e-04 | 322.67 ms | 52.3% bf16 MFU | 1625139 tok/s
step 6615/19560 | loss 3.505332 (+0.11z)| norm 0.2649 (-0.63z)| lr 4.66e-04 | 322.74 ms | 52.3% bf16 MFU | 1625107 tok/s
step 6616/19560 | loss 3.513598 (+0.34z)| norm 0.2739 (-0.23z)| lr 4.66e-04 | 322.90 ms | 52.3% bf16 MFU | 1625036 tok/s
step 6617/19560 | loss 3.464465 (-1.07z)| norm 0.2855 (+0.28z)| lr 4.66e-04 | 321.99 ms | 52.4% bf16 MFU | 1625199 tok/s
step 6618/19560 | loss 3.540899 (+1.11z)| norm 0.2548 (-1.06z)| lr 4.66e-04 | 322.55 ms | 52.3% bf16 MFU | 1625212 tok/s
step 6619/19560 | loss 3.458830 (-1.21z)| norm 0.2677 (-0.49z)| lr 4.66e-04 | 322.67 ms | 52.3% bf16 MFU | 1625194 tok/s
step 6620/19560 | loss 3.521378 (+0.57z)| norm 0.2895 (+0.46z)| lr 4.66e-04 | 322.70 ms | 52.3% bf16 MFU | 1625170 tok/s
step 6621/19560 | loss 3.490653 (-0.31z)| norm 0.2727 (-0.29z)| lr 4.66e-04 | 322.58 ms | 52.3% bf16 MFU | 1625175 tok/s
step 6622/19560 | loss 3.487088 (-0.42z)| norm 0.2648 (-0.64z)| lr 4.66e-04 | 322.30 ms | 52.4% bf16 MFU | 1625251 tok/s
step 6623/19560 | loss 3.452518 (-1.38z)| norm 0.2424 (-1.63z)| lr 4.65e-04 | 322.53 ms | 52.3% bf16 MFU | 1625264 tok/s
step 6624/19560 | loss 3.552236 (+1.42z)| norm 0.2886 (+0.41z)| lr 4.65e-04 | 322.66 ms | 52.3% bf16 MFU | 1625246 tok/s
step 6625/19560 | loss 3.472018 (-0.85z)| norm 0.2823 (+0.13z)| lr 4.65e-04 | 322.29 ms | 52.4% bf16 MFU | 1625321 tok/s
step 6626/19560 | loss 3.500343 (-0.05z)| norm 0.2621 (-0.76z)| lr 4.65e-04 | 322.86 ms | 52.3% bf16 MFU | 1625250 tok/s
step 6627/19560 | loss 3.481896 (-0.56z)| norm 0.2624 (-0.75z)| lr 4.65e-04 | 322.80 ms | 52.3% bf16 MFU | 1625198 tok/s
step 6628/19560 | loss 3.526955 (+0.73z)| norm 0.2524 (-1.21z)| lr 4.65e-04 | 322.51 ms | 52.3% bf16 MFU | 1625220 tok/s
step 6629/19560 | loss 3.456405 (-1.27z)| norm 0.2471 (-1.43z)| lr 4.65e-04 | 322.60 ms | 52.3% bf16 MFU | 1625219 tok/s
step 6630/19560 | loss 3.535326 (+0.99z)| norm 0.2642 (-0.63z)| lr 4.65e-04 | 322.10 ms | 52.4% bf16 MFU | 1625343 tok/s
step 6631/19560 | loss 3.475835 (-0.73z)| norm 0.2388 (-1.76z)| lr 4.65e-04 | 322.88 ms | 52.3% bf16 MFU | 1625264 tok/s
step 6632/19560 | loss 3.476428 (-0.71z)| norm 0.2430 (-1.54z)| lr 4.65e-04 | 322.52 ms | 52.3% bf16 MFU | 1625282 tok/s
step 6633/19560 | loss 3.551034 (+1.44z)| norm 0.2595 (-0.78z)| lr 4.65e-04 | 322.60 ms | 52.3% bf16 MFU | 1625277 tok/s
step 6634/19560 | loss 3.445080 (-1.65z)| norm 0.2640 (-0.57z)| lr 4.65e-04 | 323.19 ms | 52.2% bf16 MFU | 1625125 tok/s
step 6635/19560 | loss 3.458560 (-1.24z)| norm 0.3393 (+2.75z)| lr 4.65e-04 | 322.37 ms | 52.4% bf16 MFU | 1625188 tok/s
step 6636/19560 | loss 3.560980 (+1.71z)| norm 0.3043 (+1.18z)| lr 4.65e-04 | 322.25 ms | 52.4% bf16 MFU | 1625276 tok/s
step 6637/19560 | loss 3.492843 (-0.25z)| norm 0.2756 (-0.08z)| lr 4.65e-04 | 323.08 ms | 52.2% bf16 MFU | 1625150 tok/s
step 6638/19560 | loss 3.522826 (+0.62z)| norm 0.3149 (+1.61z)| lr 4.65e-04 | 322.39 ms | 52.3% bf16 MFU | 1625205 tok/s
step 6639/19560 | loss 3.508227 (+0.22z)| norm 0.2695 (-0.35z)| lr 4.65e-04 | 322.22 ms | 52.4% bf16 MFU | 1625300 tok/s
step 6640/19560 | loss 3.482026 (-0.55z)| norm 0.2828 (+0.23z)| lr 4.65e-04 | 323.14 ms | 52.2% bf16 MFU | 1625159 tok/s
step 6641/19560 | loss 3.514511 (+0.39z)| norm 0.2557 (-0.95z)| lr 4.65e-04 | 322.21 ms | 52.4% bf16 MFU | 1625259 tok/s
step 6642/19560 | loss 3.467980 (-0.99z)| norm 0.2781 (+0.03z)| lr 4.65e-04 | 322.58 ms | 52.3% bf16 MFU | 1625261 tok/s
step 6643/19560 | loss 3.507825 (+0.19z)| norm 0.2518 (-1.10z)| lr 4.65e-04 | 323.02 ms | 52.2% bf16 MFU | 1625152 tok/s
step 6644/19560 | loss 3.596210 (+2.72z)| norm 0.2659 (-0.48z)| lr 4.65e-04 | 322.33 ms | 52.4% bf16 MFU | 1625222 tok/s
step 6645/19560 | loss 3.518582 (+0.46z)| norm 0.2854 (+0.36z)| lr 4.65e-04 | 322.70 ms | 52.3% bf16 MFU | 1625194 tok/s
step 6646/19560 | loss 3.512366 (+0.28z)| norm 0.2756 (-0.08z)| lr 4.65e-04 | 322.78 ms | 52.3% bf16 MFU | 1625148 tok/s
step 6647/19560 | loss 3.487982 (-0.43z)| norm 0.3076 (+1.30z)| lr 4.64e-04 | 322.57 ms | 52.3% bf16 MFU | 1625159 tok/s
step 6648/19560 | loss 3.518211 (+0.44z)| norm 0.2736 (-0.18z)| lr 4.64e-04 | 322.37 ms | 52.4% bf16 MFU | 1625220 tok/s
step 6649/19560 | loss 3.536519 (+0.95z)| norm 0.2881 (+0.45z)| lr 4.64e-04 | 322.57 ms | 52.3% bf16 MFU | 1625227 tok/s
step 6650/19560 | loss 3.459710 (-1.28z)| norm 0.2889 (+0.47z)| lr 4.64e-04 | 322.42 ms | 52.3% bf16 MFU | 1625271 tok/s
step 6651/19560 | loss 3.480239 (-0.67z)| norm 0.2795 (+0.07z)| lr 4.64e-04 | 322.32 ms | 52.4% bf16 MFU | 1625337 tok/s
step 6652/19560 | loss 3.475178 (-0.81z)| norm 0.2674 (-0.45z)| lr 4.64e-04 | 322.86 ms | 52.3% bf16 MFU | 1625265 tok/s
step 6653/19560 | loss 3.576542 (+2.08z)| norm 0.2826 (+0.21z)| lr 4.64e-04 | 322.39 ms | 52.3% bf16 MFU | 1625314 tok/s
step 6654/19560 | loss 3.430833 (-2.06z)| norm 0.2695 (-0.37z)| lr 4.64e-04 | 322.99 ms | 52.3% bf16 MFU | 1625210 tok/s
step 6655/19560 | loss 3.488024 (-0.43z)| norm 0.2644 (-0.58z)| lr 4.64e-04 | 322.39 ms | 52.4% bf16 MFU | 1625263 tok/s
step 6656/19560 | loss 3.502698 (-0.01z)| norm 0.2581 (-0.84z)| lr 4.64e-04 | 322.28 ms | 52.4% bf16 MFU | 1625340 tok/s
step 6657/19560 | loss 3.525173 (+0.62z)| norm 0.2891 (+0.50z)| lr 4.64e-04 | 322.56 ms | 52.3% bf16 MFU | 1625342 tok/s
step 6658/19560 | loss 3.427463 (-2.10z)| norm 0.2775 (-0.00z)| lr 4.64e-04 | 322.51 ms | 52.3% bf16 MFU | 1625358 tok/s
step 6659/19560 | loss 3.458046 (-1.23z)| norm 0.2410 (-1.55z)| lr 4.64e-04 | 322.43 ms | 52.3% bf16 MFU | 1625392 tok/s
step 6660/19560 | loss 3.400176 (-2.73z)| norm 0.2604 (-0.71z)| lr 4.64e-04 | 322.33 ms | 52.4% bf16 MFU | 1625451 tok/s
step 6661/19560 | loss 3.460627 (-1.08z)| norm 0.2981 (+0.94z)| lr 4.64e-04 | 322.38 ms | 52.4% bf16 MFU | 1625495 tok/s
step 6662/19560 | loss 3.596516 (+2.52z)| norm 0.2846 (+0.35z)| lr 4.64e-04 | 322.54 ms | 52.3% bf16 MFU | 1625495 tok/s
step 6663/19560 | loss 3.449653 (-1.34z)| norm 0.2987 (+0.95z)| lr 4.64e-04 | 322.61 ms | 52.3% bf16 MFU | 1625478 tok/s
step 6664/19560 | loss 3.580180 (+2.07z)| norm 0.2931 (+0.70z)| lr 4.64e-04 | 322.51 ms | 52.3% bf16 MFU | 1625485 tok/s
step 6665/19560 | loss 3.491798 (-0.23z)| norm 0.2653 (-0.50z)| lr 4.64e-04 | 322.38 ms | 52.4% bf16 MFU | 1625527 tok/s
step 6666/19560 | loss 3.486781 (-0.37z)| norm 0.2791 (+0.11z)| lr 4.64e-04 | 322.59 ms | 52.3% bf16 MFU | 1625513 tok/s
step 6667/19560 | loss 3.500940 (-0.00z)| norm 0.2983 (+0.95z)| lr 4.64e-04 | 322.69 ms | 52.3% bf16 MFU | 1625474 tok/s
step 6668/19560 | loss 3.488102 (-0.35z)| norm 0.2680 (-0.39z)| lr 4.64e-04 | 322.78 ms | 52.3% bf16 MFU | 1625414 tok/s
step 6669/19560 | loss 3.495736 (-0.15z)| norm 0.2580 (-0.82z)| lr 4.64e-04 | 322.20 ms | 52.4% bf16 MFU | 1625503 tok/s
step 6670/19560 | loss 3.538178 (+0.96z)| norm 0.2780 (+0.06z)| lr 4.64e-04 | 322.76 ms | 52.3% bf16 MFU | 1625447 tok/s
step 6671/19560 | loss 3.504640 (+0.07z)| norm 0.2812 (+0.19z)| lr 4.63e-04 | 322.93 ms | 52.3% bf16 MFU | 1625352 tok/s
step 6672/19560 | loss 3.517670 (+0.42z)| norm 0.2859 (+0.40z)| lr 4.63e-04 | 322.45 ms | 52.3% bf16 MFU | 1625382 tok/s
step 6673/19560 | loss 3.516538 (+0.39z)| norm 0.2773 (+0.01z)| lr 4.63e-04 | 322.52 ms | 52.3% bf16 MFU | 1625392 tok/s
step 6674/19560 | loss 3.514565 (+0.35z)| norm 0.2588 (-0.82z)| lr 4.63e-04 | 322.29 ms | 52.4% bf16 MFU | 1625460 tok/s
step 6675/19560 | loss 3.519954 (+0.48z)| norm 0.2852 (+0.37z)| lr 4.63e-04 | 322.69 ms | 52.3% bf16 MFU | 1625423 tok/s
step 6676/19560 | loss 3.467175 (-0.92z)| norm 0.2642 (-0.58z)| lr 4.63e-04 | 322.40 ms | 52.3% bf16 MFU | 1625461 tok/s
step 6677/19560 | loss 3.517790 (+0.44z)| norm 0.3000 (+1.03z)| lr 4.63e-04 | 322.58 ms | 52.3% bf16 MFU | 1625454 tok/s
step 6678/19560 | loss 3.470116 (-0.82z)| norm 0.2750 (-0.08z)| lr 4.63e-04 | 323.35 ms | 52.2% bf16 MFU | 1625253 tok/s
step 6679/19560 | loss 3.558372 (+1.55z)| norm 0.2586 (-0.82z)| lr 4.63e-04 | 322.13 ms | 52.4% bf16 MFU | 1625369 tok/s
step 6680/19560 | loss 3.460495 (-1.07z)| norm 0.3080 (+1.43z)| lr 4.63e-04 | 322.20 ms | 52.4% bf16 MFU | 1625461 tok/s
step 6681/19560 | loss 3.519598 (+0.51z)| norm 0.3121 (+1.59z)| lr 4.63e-04 | 322.85 ms | 52.3% bf16 MFU | 1625384 tok/s
step 6682/19560 | loss 3.514651 (+0.38z)| norm 0.2948 (+0.80z)| lr 4.63e-04 | 322.38 ms | 52.4% bf16 MFU | 1625431 tok/s
step 6683/19560 | loss 3.467777 (-0.87z)| norm 0.2591 (-0.80z)| lr 4.63e-04 | 322.36 ms | 52.4% bf16 MFU | 1625480 tok/s
step 6684/19560 | loss 3.503370 (+0.08z)| norm 0.3141 (+1.64z)| lr 4.63e-04 | 322.57 ms | 52.3% bf16 MFU | 1625473 tok/s
step 6685/19560 | loss 3.499237 (-0.04z)| norm 0.2942 (+0.74z)| lr 4.63e-04 | 322.41 ms | 52.3% bf16 MFU | 1625507 tok/s
step 6686/19560 | loss 3.488815 (-0.31z)| norm 0.2754 (-0.09z)| lr 4.63e-04 | 322.22 ms | 52.4% bf16 MFU | 1625586 tok/s
step 6687/19560 | loss 3.400768 (-2.59z)| norm 0.2961 (+0.83z)| lr 4.63e-04 | 322.49 ms | 52.3% bf16 MFU | 1625595 tok/s
step 6688/19560 | loss 3.439278 (-1.58z)| norm 0.3080 (+1.33z)| lr 4.63e-04 | 322.45 ms | 52.3% bf16 MFU | 1625612 tok/s
step 6689/19560 | loss 3.508993 (+0.25z)| norm 0.2675 (-0.46z)| lr 4.63e-04 | 322.68 ms | 52.3% bf16 MFU | 1625570 tok/s
step 6690/19560 | loss 3.471644 (-0.72z)| norm 0.2834 (+0.25z)| lr 4.63e-04 | 322.56 ms | 52.3% bf16 MFU | 1625561 tok/s
step 6691/19560 | loss 3.442619 (-1.49z)| norm 0.2685 (-0.42z)| lr 4.63e-04 | 322.42 ms | 52.3% bf16 MFU | 1625588 tok/s
step 6692/19560 | loss 3.517848 (+0.51z)| norm 0.2871 (+0.39z)| lr 4.63e-04 | 322.63 ms | 52.3% bf16 MFU | 1625560 tok/s
step 6693/19560 | loss 3.466387 (-0.86z)| norm 0.2854 (+0.31z)| lr 4.63e-04 | 322.85 ms | 52.3% bf16 MFU | 1625478 tok/s
step 6694/19560 | loss 3.510281 (+0.31z)| norm 0.2907 (+0.53z)| lr 4.63e-04 | 323.18 ms | 52.2% bf16 MFU | 1625318 tok/s
step 6695/19560 | loss 3.539831 (+1.08z)| norm 0.2976 (+0.83z)| lr 4.62e-04 | 322.37 ms | 52.4% bf16 MFU | 1625369 tok/s
step 6696/19560 | loss 3.438366 (-1.60z)| norm 0.3046 (+1.13z)| lr 4.62e-04 | 322.71 ms | 52.3% bf16 MFU | 1625333 tok/s
step 6697/19560 | loss 3.489522 (-0.25z)| norm 0.3109 (+1.39z)| lr 4.62e-04 | 322.21 ms | 52.4% bf16 MFU | 1625423 tok/s
step 6698/19560 | loss 3.466406 (-0.85z)| norm 0.3245 (+1.96z)| lr 4.62e-04 | 322.80 ms | 52.3% bf16 MFU | 1625361 tok/s
step 6699/19560 | loss 3.509037 (+0.29z)| norm 0.2873 (+0.29z)| lr 4.62e-04 | 322.55 ms | 52.3% bf16 MFU | 1625365 tok/s
step 6700/19560 | loss 3.482016 (-0.43z)| norm 0.2790 (-0.09z)| lr 4.62e-04 | 322.43 ms | 52.3% bf16 MFU | 1625399 tok/s
step 6701/19560 | loss 3.518151 (+0.54z)| norm 0.2726 (-0.38z)| lr 4.62e-04 | 322.51 ms | 52.3% bf16 MFU | 1625413 tok/s
step 6702/19560 | loss 3.522736 (+0.65z)| norm 0.2809 (-0.02z)| lr 4.62e-04 | 322.43 ms | 52.3% bf16 MFU | 1625445 tok/s
step 6703/19560 | loss 3.485003 (-0.35z)| norm 0.2635 (-0.82z)| lr 4.62e-04 | 322.56 ms | 52.3% bf16 MFU | 1625444 tok/s
step 6704/19560 | loss 3.514797 (+0.44z)| norm 0.2768 (-0.20z)| lr 4.62e-04 | 322.70 ms | 52.3% bf16 MFU | 1625408 tok/s
step 6705/19560 | loss 3.491994 (-0.17z)| norm 0.2550 (-1.19z)| lr 4.62e-04 | 322.29 ms | 52.4% bf16 MFU | 1625475 tok/s
step 6706/19560 | loss 3.645883 (+3.75z)| norm 0.2852 (+0.21z)| lr 4.62e-04 | 322.57 ms | 52.3% bf16 MFU | 1625470 tok/s
step 6707/19560 | loss 3.509345 (+0.27z)| norm 0.2958 (+0.68z)| lr 4.62e-04 | 322.58 ms | 52.3% bf16 MFU | 1625461 tok/s
step 6708/19560 | loss 3.559676 (+1.55z)| norm 0.2407 (-1.82z)| lr 4.62e-04 | 322.88 ms | 52.3% bf16 MFU | 1625377 tok/s
step 6709/19560 | loss 3.417052 (-2.08z)| norm 0.3019 (+0.96z)| lr 4.62e-04 | 322.48 ms | 52.3% bf16 MFU | 1625399 tok/s
step 6710/19560 | loss 3.484537 (-0.36z)| norm 0.3014 (+0.93z)| lr 4.62e-04 | 322.43 ms | 52.3% bf16 MFU | 1625432 tok/s
step 6711/19560 | loss 3.500098 (+0.04z)| norm 0.2453 (-1.58z)| lr 4.62e-04 | 322.31 ms | 52.4% bf16 MFU | 1625493 tok/s
step 6712/19560 | loss 3.474078 (-0.63z)| norm 0.2664 (-0.63z)| lr 4.62e-04 | 322.87 ms | 52.3% bf16 MFU | 1625410 tok/s
step 6713/19560 | loss 3.465206 (-0.85z)| norm 0.2678 (-0.57z)| lr 4.62e-04 | 322.38 ms | 52.4% bf16 MFU | 1625454 tok/s
step 6714/19560 | loss 3.499906 (+0.03z)| norm 0.2566 (-1.07z)| lr 4.62e-04 | 322.86 ms | 52.3% bf16 MFU | 1625376 tok/s
step 6715/19560 | loss 3.519318 (+0.52z)| norm 0.2503 (-1.34z)| lr 4.62e-04 | 322.64 ms | 52.3% bf16 MFU | 1625358 tok/s
step 6716/19560 | loss 3.468877 (-0.77z)| norm 0.2573 (-1.02z)| lr 4.62e-04 | 322.65 ms | 52.3% bf16 MFU | 1625338 tok/s
step 6717/19560 | loss 3.524401 (+0.66z)| norm 0.2574 (-1.01z)| lr 4.62e-04 | 322.83 ms | 52.3% bf16 MFU | 1625272 tok/s
step 6718/19560 | loss 3.536846 (+0.97z)| norm 0.2594 (-0.91z)| lr 4.62e-04 | 322.97 ms | 52.3% bf16 MFU | 1625174 tok/s
step 6719/19560 | loss 3.466706 (-0.84z)| norm 0.2664 (-0.60z)| lr 4.61e-04 | 322.64 ms | 52.3% bf16 MFU | 1625164 tok/s
step 6720/19560 | loss 3.457237 (-1.08z)| norm 0.2763 (-0.17z)| lr 4.61e-04 | 322.72 ms | 52.3% bf16 MFU | 1625136 tok/s
step 6721/19560 | loss 3.637910 (+3.44z)| norm 0.2663 (-0.61z)| lr 4.61e-04 | 322.61 ms | 52.3% bf16 MFU | 1625137 tok/s
step 6722/19560 | loss 3.455959 (-1.08z)| norm 0.3117 (+1.42z)| lr 4.61e-04 | 322.59 ms | 52.3% bf16 MFU | 1625144 tok/s
step 6723/19560 | loss 3.432995 (-1.62z)| norm 0.2587 (-0.95z)| lr 4.61e-04 | 322.55 ms | 52.3% bf16 MFU | 1625159 tok/s
step 6724/19560 | loss 3.420255 (-1.89z)| norm 0.2852 (+0.24z)| lr 4.61e-04 | 322.84 ms | 52.3% bf16 MFU | 1625101 tok/s
step 6725/19560 | loss 3.461946 (-0.87z)| norm 0.2884 (+0.39z)| lr 4.61e-04 | 322.62 ms | 52.3% bf16 MFU | 1625100 tok/s
step 6726/19560 | loss 3.552720 (+1.30z)| norm 0.2973 (+0.79z)| lr 4.61e-04 | 322.67 ms | 52.3% bf16 MFU | 1625086 tok/s
step 6727/19560 | loss 3.527181 (+0.69z)| norm 0.2766 (-0.13z)| lr 4.61e-04 | 323.17 ms | 52.2% bf16 MFU | 1624947 tok/s
step 6728/19560 | loss 3.494922 (-0.09z)| norm 0.2931 (+0.66z)| lr 4.61e-04 | 323.34 ms | 52.2% bf16 MFU | 1624775 tok/s
step 6729/19560 | loss 3.463733 (-0.83z)| norm 0.3517 (+3.24z)| lr 4.61e-04 | 322.50 ms | 52.3% bf16 MFU | 1624820 tok/s
step 6730/19560 | loss 3.435173 (-1.49z)| norm 0.3142 (+1.53z)| lr 4.61e-04 | 322.62 ms | 52.3% bf16 MFU | 1624834 tok/s
step 6731/19560 | loss 3.479147 (-0.43z)| norm 0.2760 (-0.16z)| lr 4.61e-04 | 323.00 ms | 52.3% bf16 MFU | 1624751 tok/s
step 6732/19560 | loss 3.446381 (-1.21z)| norm 0.2917 (+0.55z)| lr 4.61e-04 | 322.35 ms | 52.4% bf16 MFU | 1624836 tok/s
step 6733/19560 | loss 3.443881 (-1.25z)| norm 0.2751 (-0.18z)| lr 4.61e-04 | 322.67 ms | 52.3% bf16 MFU | 1624835 tok/s
step 6734/19560 | loss 3.507665 (+0.27z)| norm 0.2670 (-0.55z)| lr 4.61e-04 | 322.83 ms | 52.3% bf16 MFU | 1624796 tok/s
step 6735/19560 | loss 3.513796 (+0.42z)| norm 0.2716 (-0.34z)| lr 4.61e-04 | 322.57 ms | 52.3% bf16 MFU | 1624823 tok/s
step 6736/19560 | loss 3.468586 (-0.66z)| norm 0.2826 (+0.15z)| lr 4.61e-04 | 323.07 ms | 52.2% bf16 MFU | 1624723 tok/s
step 6737/19560 | loss 3.449000 (-1.11z)| norm 0.2366 (-1.89z)| lr 4.61e-04 | 323.62 ms | 52.2% bf16 MFU | 1624490 tok/s
step 6738/19560 | loss 3.561591 (+1.55z)| norm 0.2850 (+0.26z)| lr 4.61e-04 | 322.85 ms | 52.3% bf16 MFU | 1624462 tok/s
step 6739/19560 | loss 3.544661 (+1.13z)| norm 0.2950 (+0.70z)| lr 4.61e-04 | 322.56 ms | 52.3% bf16 MFU | 1624510 tok/s
step 6740/19560 | loss 3.518570 (+0.53z)| norm 0.2818 (+0.11z)| lr 4.61e-04 | 322.92 ms | 52.3% bf16 MFU | 1624464 tok/s
step 6741/19560 | loss 3.496872 (+0.01z)| norm 0.3025 (+1.17z)| lr 4.61e-04 | 322.77 ms | 52.3% bf16 MFU | 1624458 tok/s
step 6742/19560 | loss 3.567273 (+1.65z)| norm 0.2731 (-0.26z)| lr 4.61e-04 | 322.54 ms | 52.3% bf16 MFU | 1624511 tok/s
step 6743/19560 | loss 3.503056 (+0.15z)| norm 0.2574 (-1.03z)| lr 4.60e-04 | 322.45 ms | 52.3% bf16 MFU | 1624584 tok/s
step 6744/19560 | loss 3.514887 (+0.43z)| norm 0.2882 (+0.48z)| lr 4.60e-04 | 322.89 ms | 52.3% bf16 MFU | 1624542 tok/s
step 6745/19560 | loss 3.482365 (-0.34z)| norm 0.2652 (-0.64z)| lr 4.60e-04 | 322.81 ms | 52.3% bf16 MFU | 1624523 tok/s
step 6746/19560 | loss 3.469053 (-0.64z)| norm 0.2467 (-1.54z)| lr 4.60e-04 | 322.49 ms | 52.3% bf16 MFU | 1624583 tok/s
step 6747/19560 | loss 3.493753 (-0.06z)| norm 0.2410 (-1.78z)| lr 4.60e-04 | 323.14 ms | 52.2% bf16 MFU | 1624477 tok/s
step 6748/19560 | loss 3.419895 (-1.77z)| norm 0.2628 (-0.72z)| lr 4.60e-04 | 323.48 ms | 52.2% bf16 MFU | 1624292 tok/s
step 6749/19560 | loss 3.481181 (-0.34z)| norm 0.2826 (+0.23z)| lr 4.60e-04 | 322.54 ms | 52.3% bf16 MFU | 1624352 tok/s
step 6750/19560 | loss 3.456567 (-0.90z)| norm 0.2520 (-1.24z)| lr 4.60e-04 | 322.65 ms | 52.3% bf16 MFU | 1624380 tok/s
val loss 3.486723
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2801/10042 = 0.278928
step 6751/19560 | loss 3.475295 (-0.47z)| norm 0.2979 (+0.95z)| lr 4.60e-04 | 322.60 ms | 52.3% bf16 MFU | 1624420 tok/s
step 6752/19560 | loss 3.508076 (+0.30z)| norm 0.2501 (-1.34z)| lr 4.60e-04 | 322.65 ms | 52.3% bf16 MFU | 1624445 tok/s
step 6753/19560 | loss 3.599795 (+2.38z)| norm 0.2825 (+0.22z)| lr 4.60e-04 | 322.94 ms | 52.3% bf16 MFU | 1624398 tok/s
step 6754/19560 | loss 3.508654 (+0.28z)| norm 0.2681 (-0.47z)| lr 4.60e-04 | 322.92 ms | 52.3% bf16 MFU | 1624356 tok/s
step 6755/19560 | loss 3.603796 (+2.39z)| norm 0.2824 (+0.20z)| lr 4.60e-04 | 322.42 ms | 52.3% bf16 MFU | 1624444 tok/s
step 6756/19560 | loss 3.459548 (-0.83z)| norm 0.2637 (-0.70z)| lr 4.60e-04 | 322.92 ms | 52.3% bf16 MFU | 1624400 tok/s
step 6757/19560 | loss 3.505834 (+0.20z)| norm 0.2678 (-0.52z)| lr 4.60e-04 | 322.29 ms | 52.4% bf16 MFU | 1624518 tok/s
step 6758/19560 | loss 3.499484 (+0.06z)| norm 0.2531 (-1.22z)| lr 4.60e-04 | 322.13 ms | 52.4% bf16 MFU | 1624671 tok/s
step 6759/19560 | loss 3.561264 (+1.43z)| norm 0.2764 (-0.11z)| lr 4.60e-04 | 322.48 ms | 52.3% bf16 MFU | 1624728 tok/s
step 6760/19560 | loss 3.466963 (-0.68z)| norm 0.2871 (+0.41z)| lr 4.60e-04 | 322.64 ms | 52.3% bf16 MFU | 1624742 tok/s
step 6761/19560 | loss 3.527918 (+0.69z)| norm 0.2805 (+0.07z)| lr 4.60e-04 | 322.21 ms | 52.4% bf16 MFU | 1624862 tok/s
step 6762/19560 | loss 3.472256 (-0.57z)| norm 0.3053 (+1.28z)| lr 4.60e-04 | 321.94 ms | 52.4% bf16 MFU | 1625045 tok/s
step 6763/19560 | loss 3.492811 (-0.11z)| norm 0.2915 (+0.64z)| lr 4.60e-04 | 322.70 ms | 52.3% bf16 MFU | 1625027 tok/s
step 6764/19560 | loss 3.489198 (-0.18z)| norm 0.2473 (-1.61z)| lr 4.60e-04 | 322.80 ms | 52.3% bf16 MFU | 1624984 tok/s
step 6765/19560 | loss 3.488575 (-0.19z)| norm 0.2915 (+0.65z)| lr 4.60e-04 | 322.56 ms | 52.3% bf16 MFU | 1625003 tok/s
step 6766/19560 | loss 3.483749 (-0.30z)| norm 0.2791 (+0.03z)| lr 4.59e-04 | 322.47 ms | 52.3% bf16 MFU | 1625046 tok/s
step 6767/19560 | loss 3.479025 (-0.40z)| norm 0.2790 (+0.02z)| lr 4.59e-04 | 322.68 ms | 52.3% bf16 MFU | 1625034 tok/s
step 6768/19560 | loss 3.520070 (+0.53z)| norm 0.2857 (+0.37z)| lr 4.59e-04 | 322.35 ms | 52.4% bf16 MFU | 1625105 tok/s
step 6769/19560 | loss 3.489075 (-0.17z)| norm 0.2710 (-0.40z)| lr 4.59e-04 | 323.09 ms | 52.2% bf16 MFU | 1624985 tok/s
step 6770/19560 | loss 3.509100 (+0.28z)| norm 0.2990 (+1.05z)| lr 4.59e-04 | 323.11 ms | 52.2% bf16 MFU | 1624868 tok/s
step 6771/19560 | loss 3.524869 (+0.63z)| norm 0.2648 (-0.74z)| lr 4.59e-04 | 322.25 ms | 52.4% bf16 MFU | 1624972 tok/s
step 6772/19560 | loss 3.543674 (+1.09z)| norm 0.2505 (-1.47z)| lr 4.59e-04 | 322.76 ms | 52.3% bf16 MFU | 1624944 tok/s
step 6773/19560 | loss 3.467055 (-0.68z)| norm 0.2602 (-0.96z)| lr 4.59e-04 | 322.66 ms | 52.3% bf16 MFU | 1624941 tok/s
step 6774/19560 | loss 3.438929 (-1.31z)| norm 0.2504 (-1.44z)| lr 4.59e-04 | 322.90 ms | 52.3% bf16 MFU | 1624879 tok/s
step 6775/19560 | loss 3.520355 (+0.56z)| norm 0.2606 (-0.90z)| lr 4.59e-04 | 322.97 ms | 52.3% bf16 MFU | 1624802 tok/s
step 6776/19560 | loss 3.443587 (-1.18z)| norm 0.2571 (-1.07z)| lr 4.59e-04 | 322.55 ms | 52.3% bf16 MFU | 1624833 tok/s
step 6777/19560 | loss 3.503270 (+0.19z)| norm 0.2666 (-0.58z)| lr 4.59e-04 | 322.95 ms | 52.3% bf16 MFU | 1624763 tok/s
step 6778/19560 | loss 3.456707 (-0.88z)| norm 0.2644 (-0.68z)| lr 4.59e-04 | 322.53 ms | 52.3% bf16 MFU | 1624803 tok/s
step 6779/19560 | loss 3.468144 (-0.62z)| norm 0.2759 (-0.09z)| lr 4.59e-04 | 322.69 ms | 52.3% bf16 MFU | 1624800 tok/s
step 6780/19560 | loss 3.469519 (-0.58z)| norm 0.2867 (+0.46z)| lr 4.59e-04 | 322.74 ms | 52.3% bf16 MFU | 1624784 tok/s
step 6781/19560 | loss 3.520396 (+0.60z)| norm 0.2814 (+0.19z)| lr 4.59e-04 | 322.23 ms | 52.4% bf16 MFU | 1624899 tok/s
step 6782/19560 | loss 3.447641 (-1.09z)| norm 0.3429 (+3.19z)| lr 4.59e-04 | 322.23 ms | 52.4% bf16 MFU | 1625007 tok/s
step 6783/19560 | loss 3.450187 (-1.02z)| norm 0.3069 (+1.39z)| lr 4.59e-04 | 322.39 ms | 52.3% bf16 MFU | 1625069 tok/s
step 6784/19560 | loss 3.500263 (+0.14z)| norm 0.2838 (+0.24z)| lr 4.59e-04 | 323.18 ms | 52.2% bf16 MFU | 1624930 tok/s
step 6785/19560 | loss 3.472871 (-0.49z)| norm 0.2844 (+0.28z)| lr 4.59e-04 | 322.60 ms | 52.3% bf16 MFU | 1624944 tok/s
step 6786/19560 | loss 3.517496 (+0.53z)| norm 0.2688 (-0.49z)| lr 4.59e-04 | 322.46 ms | 52.3% bf16 MFU | 1624992 tok/s
step 6787/19560 | loss 3.488091 (-0.16z)| norm 0.2726 (-0.32z)| lr 4.59e-04 | 322.43 ms | 52.3% bf16 MFU | 1625046 tok/s
step 6788/19560 | loss 3.466285 (-0.70z)| norm 0.2685 (-0.52z)| lr 4.59e-04 | 322.65 ms | 52.3% bf16 MFU | 1625042 tok/s
step 6789/19560 | loss 3.534447 (+0.92z)| norm 0.2635 (-0.76z)| lr 4.59e-04 | 322.65 ms | 52.3% bf16 MFU | 1625036 tok/s
step 6790/19560 | loss 3.501965 (+0.16z)| norm 0.2615 (-0.85z)| lr 4.58e-04 | 322.78 ms | 52.3% bf16 MFU | 1624998 tok/s
step 6791/19560 | loss 3.500459 (+0.12z)| norm 0.2871 (+0.43z)| lr 4.58e-04 | 322.65 ms | 52.3% bf16 MFU | 1624996 tok/s
step 6792/19560 | loss 3.530068 (+0.87z)| norm 0.2712 (-0.36z)| lr 4.58e-04 | 322.38 ms | 52.4% bf16 MFU | 1625061 tok/s
step 6793/19560 | loss 3.435005 (-1.48z)| norm 0.2577 (-1.03z)| lr 4.58e-04 | 322.43 ms | 52.3% bf16 MFU | 1625110 tok/s
step 6794/19560 | loss 3.545527 (+1.23z)| norm 0.2905 (+0.61z)| lr 4.58e-04 | 322.54 ms | 52.3% bf16 MFU | 1625129 tok/s
step 6795/19560 | loss 3.438847 (-1.36z)| norm 0.2928 (+0.73z)| lr 4.58e-04 | 322.72 ms | 52.3% bf16 MFU | 1625103 tok/s
step 6796/19560 | loss 3.539912 (+1.08z)| norm 0.2736 (-0.24z)| lr 4.58e-04 | 322.75 ms | 52.3% bf16 MFU | 1625070 tok/s
step 6797/19560 | loss 3.531914 (+0.88z)| norm 0.2917 (+0.66z)| lr 4.58e-04 | 322.35 ms | 52.4% bf16 MFU | 1625140 tok/s
step 6798/19560 | loss 3.426182 (-1.64z)| norm 0.2724 (-0.31z)| lr 4.58e-04 | 322.64 ms | 52.3% bf16 MFU | 1625133 tok/s
step 6799/19560 | loss 3.459944 (-0.82z)| norm 0.2712 (-0.37z)| lr 4.58e-04 | 322.08 ms | 52.4% bf16 MFU | 1625267 tok/s
step 6800/19560 | loss 3.519532 (+0.60z)| norm 0.2651 (-0.66z)| lr 4.58e-04 | 322.41 ms | 52.3% bf16 MFU | 1625310 tok/s
step 6801/19560 | loss 3.472391 (-0.52z)| norm 0.2704 (-0.39z)| lr 4.58e-04 | 322.74 ms | 52.3% bf16 MFU | 1625270 tok/s
step 6802/19560 | loss 3.441894 (-1.23z)| norm 0.2583 (-1.00z)| lr 4.58e-04 | 322.31 ms | 52.4% bf16 MFU | 1625339 tok/s
step 6803/19560 | loss 3.491915 (-0.03z)| norm 0.2744 (-0.19z)| lr 4.58e-04 | 322.58 ms | 52.3% bf16 MFU | 1625336 tok/s
step 6804/19560 | loss 3.484266 (-0.22z)| norm 0.2685 (-0.49z)| lr 4.58e-04 | 322.56 ms | 52.3% bf16 MFU | 1625340 tok/s
step 6805/19560 | loss 3.496660 (+0.08z)| norm 0.2800 (+0.10z)| lr 4.58e-04 | 322.86 ms | 52.3% bf16 MFU | 1625268 tok/s
step 6806/19560 | loss 3.511967 (+0.44z)| norm 0.2945 (+0.82z)| lr 4.58e-04 | 322.50 ms | 52.3% bf16 MFU | 1625290 tok/s
step 6807/19560 | loss 3.511570 (+0.45z)| norm 0.2899 (+0.58z)| lr 4.58e-04 | 322.45 ms | 52.3% bf16 MFU | 1625324 tok/s
step 6808/19560 | loss 3.476666 (-0.40z)| norm 0.2715 (-0.34z)| lr 4.58e-04 | 322.12 ms | 52.4% bf16 MFU | 1625438 tok/s
step 6809/19560 | loss 3.461906 (-0.75z)| norm 0.2834 (+0.28z)| lr 4.58e-04 | 322.66 ms | 52.3% bf16 MFU | 1625410 tok/s
step 6810/19560 | loss 3.545987 (+1.27z)| norm 0.2491 (-1.46z)| lr 4.58e-04 | 322.46 ms | 52.3% bf16 MFU | 1625434 tok/s
step 6811/19560 | loss 3.551167 (+1.38z)| norm 0.3116 (+1.70z)| lr 4.58e-04 | 322.67 ms | 52.3% bf16 MFU | 1625405 tok/s
step 6812/19560 | loss 3.525974 (+0.77z)| norm 0.3098 (+1.62z)| lr 4.58e-04 | 322.62 ms | 52.3% bf16 MFU | 1625390 tok/s
step 6813/19560 | loss 3.477684 (-0.38z)| norm 0.2746 (-0.16z)| lr 4.57e-04 | 322.44 ms | 52.3% bf16 MFU | 1625419 tok/s
step 6814/19560 | loss 3.488848 (-0.11z)| norm 0.3065 (+1.44z)| lr 4.57e-04 | 322.57 ms | 52.3% bf16 MFU | 1625415 tok/s
step 6815/19560 | loss 3.491323 (-0.07z)| norm 0.2798 (+0.09z)| lr 4.57e-04 | 322.36 ms | 52.4% bf16 MFU | 1625465 tok/s
step 6816/19560 | loss 3.501237 (+0.16z)| norm 0.3180 (+2.02z)| lr 4.57e-04 | 322.88 ms | 52.3% bf16 MFU | 1625382 tok/s
step 6817/19560 | loss 3.526509 (+0.77z)| norm 0.2848 (+0.34z)| lr 4.57e-04 | 322.88 ms | 52.3% bf16 MFU | 1625301 tok/s
step 6818/19560 | loss 3.484483 (-0.26z)| norm 0.2831 (+0.25z)| lr 4.57e-04 | 322.24 ms | 52.4% bf16 MFU | 1625388 tok/s
step 6819/19560 | loss 3.510565 (+0.37z)| norm 0.2878 (+0.48z)| lr 4.57e-04 | 322.73 ms | 52.3% bf16 MFU | 1625346 tok/s
step 6820/19560 | loss 3.478621 (-0.41z)| norm 0.2782 (+0.00z)| lr 4.57e-04 | 322.75 ms | 52.3% bf16 MFU | 1625301 tok/s
step 6821/19560 | loss 3.531646 (+0.88z)| norm 0.2803 (+0.11z)| lr 4.57e-04 | 322.20 ms | 52.4% bf16 MFU | 1625397 tok/s
step 6822/19560 | loss 3.458330 (-0.91z)| norm 0.2810 (+0.15z)| lr 4.57e-04 | 322.85 ms | 52.3% bf16 MFU | 1625324 tok/s
step 6823/19560 | loss 3.518617 (+0.58z)| norm 0.3014 (+1.17z)| lr 4.57e-04 | 322.69 ms | 52.3% bf16 MFU | 1625294 tok/s
step 6824/19560 | loss 3.501267 (+0.14z)| norm 0.2763 (-0.08z)| lr 4.57e-04 | 322.33 ms | 52.4% bf16 MFU | 1625357 tok/s
step 6825/19560 | loss 3.471463 (-0.60z)| norm 0.2744 (-0.17z)| lr 4.57e-04 | 322.78 ms | 52.3% bf16 MFU | 1625304 tok/s
step 6826/19560 | loss 3.556985 (+1.49z)| norm 0.2832 (+0.31z)| lr 4.57e-04 | 322.74 ms | 52.3% bf16 MFU | 1625263 tok/s
step 6827/19560 | loss 3.498891 (+0.07z)| norm 0.2810 (+0.19z)| lr 4.57e-04 | 322.60 ms | 52.3% bf16 MFU | 1625261 tok/s
step 6828/19560 | loss 3.452570 (-1.07z)| norm 0.2531 (-1.26z)| lr 4.57e-04 | 323.18 ms | 52.2% bf16 MFU | 1625112 tok/s
step 6829/19560 | loss 3.441584 (-1.31z)| norm 0.2763 (-0.04z)| lr 4.57e-04 | 322.45 ms | 52.3% bf16 MFU | 1625154 tok/s
step 6830/19560 | loss 3.498032 (+0.07z)| norm 0.2433 (-1.73z)| lr 4.57e-04 | 322.33 ms | 52.4% bf16 MFU | 1625224 tok/s
step 6831/19560 | loss 3.449819 (-1.10z)| norm 0.2680 (-0.46z)| lr 4.57e-04 | 322.17 ms | 52.4% bf16 MFU | 1625332 tok/s
step 6832/19560 | loss 3.505334 (+0.25z)| norm 0.2896 (+0.65z)| lr 4.57e-04 | 322.97 ms | 52.3% bf16 MFU | 1625232 tok/s
step 6833/19560 | loss 3.468842 (-0.63z)| norm 0.2971 (+1.03z)| lr 4.57e-04 | 322.86 ms | 52.3% bf16 MFU | 1625165 tok/s
step 6834/19560 | loss 3.518825 (+0.65z)| norm 0.2719 (-0.27z)| lr 4.57e-04 | 322.41 ms | 52.3% bf16 MFU | 1625214 tok/s
step 6835/19560 | loss 3.476518 (-0.43z)| norm 0.2718 (-0.27z)| lr 4.57e-04 | 322.54 ms | 52.3% bf16 MFU | 1625230 tok/s
step 6836/19560 | loss 3.405436 (-2.21z)| norm 0.2647 (-0.65z)| lr 4.57e-04 | 322.69 ms | 52.3% bf16 MFU | 1625206 tok/s
step 6837/19560 | loss 3.657363 (+3.95z)| norm 0.2932 (+0.85z)| lr 4.56e-04 | 322.89 ms | 52.3% bf16 MFU | 1625132 tok/s
step 6838/19560 | loss 3.541077 (+1.12z)| norm 0.2924 (+0.81z)| lr 4.56e-04 | 322.51 ms | 52.3% bf16 MFU | 1625158 tok/s
step 6839/19560 | loss 3.565532 (+1.68z)| norm 0.2601 (-0.91z)| lr 4.56e-04 | 322.89 ms | 52.3% bf16 MFU | 1625086 tok/s
step 6840/19560 | loss 3.506731 (+0.27z)| norm 0.2720 (-0.28z)| lr 4.56e-04 | 323.00 ms | 52.3% bf16 MFU | 1624991 tok/s
step 6841/19560 | loss 3.481271 (-0.34z)| norm 0.2755 (-0.09z)| lr 4.56e-04 | 322.58 ms | 52.3% bf16 MFU | 1625007 tok/s
step 6842/19560 | loss 3.426930 (-1.60z)| norm 0.2727 (-0.25z)| lr 4.56e-04 | 322.46 ms | 52.3% bf16 MFU | 1625051 tok/s
step 6843/19560 | loss 3.474356 (-0.48z)| norm 0.2736 (-0.21z)| lr 4.56e-04 | 322.48 ms | 52.3% bf16 MFU | 1625090 tok/s
step 6844/19560 | loss 3.439246 (-1.29z)| norm 0.2747 (-0.16z)| lr 4.56e-04 | 322.43 ms | 52.3% bf16 MFU | 1625137 tok/s
step 6845/19560 | loss 3.456505 (-0.87z)| norm 0.2494 (-1.53z)| lr 4.56e-04 | 322.75 ms | 52.3% bf16 MFU | 1625103 tok/s
step 6846/19560 | loss 3.502976 (+0.22z)| norm 0.2623 (-0.83z)| lr 4.56e-04 | 323.54 ms | 52.2% bf16 MFU | 1624871 tok/s
step 6847/19560 | loss 3.481261 (-0.29z)| norm 0.2849 (+0.39z)| lr 4.56e-04 | 322.53 ms | 52.3% bf16 MFU | 1624904 tok/s
step 6848/19560 | loss 3.499225 (+0.13z)| norm 0.2468 (-1.65z)| lr 4.56e-04 | 322.47 ms | 52.3% bf16 MFU | 1624952 tok/s
step 6849/19560 | loss 3.430677 (-1.52z)| norm 0.2719 (-0.30z)| lr 4.56e-04 | 322.29 ms | 52.4% bf16 MFU | 1625041 tok/s
step 6850/19560 | loss 3.531845 (+0.96z)| norm 0.2619 (-0.83z)| lr 4.56e-04 | 322.81 ms | 52.3% bf16 MFU | 1624996 tok/s
step 6851/19560 | loss 3.535801 (+1.04z)| norm 0.2613 (-0.87z)| lr 4.56e-04 | 322.36 ms | 52.4% bf16 MFU | 1625066 tok/s
step 6852/19560 | loss 3.459198 (-0.87z)| norm 0.2504 (-1.43z)| lr 4.56e-04 | 322.66 ms | 52.3% bf16 MFU | 1625056 tok/s
step 6853/19560 | loss 3.545730 (+1.27z)| norm 0.2763 (-0.03z)| lr 4.56e-04 | 322.57 ms | 52.3% bf16 MFU | 1625072 tok/s
step 6854/19560 | loss 3.463933 (-0.75z)| norm 0.2627 (-0.76z)| lr 4.56e-04 | 322.85 ms | 52.3% bf16 MFU | 1625014 tok/s
step 6855/19560 | loss 3.491165 (-0.06z)| norm 0.2966 (+1.07z)| lr 4.56e-04 | 322.56 ms | 52.3% bf16 MFU | 1625032 tok/s
step 6856/19560 | loss 3.503642 (+0.25z)| norm 0.2563 (-1.09z)| lr 4.56e-04 | 322.58 ms | 52.3% bf16 MFU | 1625045 tok/s
step 6857/19560 | loss 3.440865 (-1.32z)| norm 0.2833 (+0.43z)| lr 4.56e-04 | 322.82 ms | 52.3% bf16 MFU | 1624998 tok/s
step 6858/19560 | loss 3.513021 (+0.47z)| norm 0.2736 (-0.12z)| lr 4.56e-04 | 322.35 ms | 52.4% bf16 MFU | 1625071 tok/s
step 6859/19560 | loss 3.493312 (-0.02z)| norm 0.2809 (+0.31z)| lr 4.56e-04 | 322.29 ms | 52.4% bf16 MFU | 1625156 tok/s
step 6860/19560 | loss 3.532228 (+0.94z)| norm 0.2706 (-0.29z)| lr 4.55e-04 | 322.73 ms | 52.3% bf16 MFU | 1625124 tok/s
step 6861/19560 | loss 3.558112 (+1.56z)| norm 0.2693 (-0.36z)| lr 4.55e-04 | 322.46 ms | 52.3% bf16 MFU | 1625164 tok/s
step 6862/19560 | loss 3.495742 (-0.00z)| norm 0.2656 (-0.58z)| lr 4.55e-04 | 322.52 ms | 52.3% bf16 MFU | 1625184 tok/s
step 6863/19560 | loss 3.442302 (-1.32z)| norm 0.2507 (-1.44z)| lr 4.55e-04 | 322.72 ms | 52.3% bf16 MFU | 1625154 tok/s
step 6864/19560 | loss 3.453007 (-1.05z)| norm 0.2644 (-0.63z)| lr 4.55e-04 | 322.46 ms | 52.3% bf16 MFU | 1625191 tok/s
step 6865/19560 | loss 3.438065 (-1.42z)| norm 0.2799 (+0.27z)| lr 4.55e-04 | 322.52 ms | 52.3% bf16 MFU | 1625211 tok/s
step 6866/19560 | loss 3.465520 (-0.72z)| norm 0.2590 (-0.97z)| lr 4.55e-04 | 322.53 ms | 52.3% bf16 MFU | 1625228 tok/s
step 6867/19560 | loss 3.552359 (+1.45z)| norm 0.2760 (+0.06z)| lr 4.55e-04 | 322.57 ms | 52.3% bf16 MFU | 1625233 tok/s
step 6868/19560 | loss 3.550191 (+1.38z)| norm 0.2949 (+1.17z)| lr 4.55e-04 | 322.43 ms | 52.3% bf16 MFU | 1625273 tok/s
step 6869/19560 | loss 3.536535 (+1.03z)| norm 0.2925 (+1.04z)| lr 4.55e-04 | 322.32 ms | 52.4% bf16 MFU | 1625339 tok/s
step 6870/19560 | loss 3.463897 (-0.76z)| norm 0.2667 (-0.50z)| lr 4.55e-04 | 322.50 ms | 52.3% bf16 MFU | 1625358 tok/s
step 6871/19560 | loss 3.511284 (+0.43z)| norm 0.2778 (+0.15z)| lr 4.55e-04 | 322.30 ms | 52.4% bf16 MFU | 1625425 tok/s
step 6872/19560 | loss 3.411741 (-2.01z)| norm 0.2861 (+0.65z)| lr 4.55e-04 | 322.72 ms | 52.3% bf16 MFU | 1625383 tok/s
step 6873/19560 | loss 3.530428 (+0.90z)| norm 0.2745 (-0.05z)| lr 4.55e-04 | 322.54 ms | 52.3% bf16 MFU | 1625388 tok/s
step 6874/19560 | loss 3.451283 (-1.04z)| norm 0.2631 (-0.75z)| lr 4.55e-04 | 322.89 ms | 52.3% bf16 MFU | 1625306 tok/s
step 6875/19560 | loss 3.456066 (-0.91z)| norm 0.2641 (-0.71z)| lr 4.55e-04 | 322.81 ms | 52.3% bf16 MFU | 1625248 tok/s
step 6876/19560 | loss 3.474778 (-0.47z)| norm 0.2834 (+0.47z)| lr 4.55e-04 | 322.42 ms | 52.3% bf16 MFU | 1625292 tok/s
step 6877/19560 | loss 3.484997 (-0.22z)| norm 0.2808 (+0.31z)| lr 4.55e-04 | 322.45 ms | 52.3% bf16 MFU | 1625325 tok/s
step 6878/19560 | loss 3.480564 (-0.33z)| norm 0.2749 (-0.06z)| lr 4.55e-04 | 322.53 ms | 52.3% bf16 MFU | 1625337 tok/s
step 6879/19560 | loss 3.459260 (-0.85z)| norm 0.2942 (+1.14z)| lr 4.55e-04 | 322.25 ms | 52.4% bf16 MFU | 1625418 tok/s
step 6880/19560 | loss 3.546427 (+1.29z)| norm 0.2612 (-0.93z)| lr 4.55e-04 | 322.23 ms | 52.4% bf16 MFU | 1625501 tok/s
step 6881/19560 | loss 3.477955 (-0.38z)| norm 0.2908 (+0.93z)| lr 4.55e-04 | 322.55 ms | 52.3% bf16 MFU | 1625498 tok/s
step 6882/19560 | loss 3.534217 (+1.03z)| norm 0.3027 (+1.64z)| lr 4.55e-04 | 322.98 ms | 52.3% bf16 MFU | 1625388 tok/s
step 6883/19560 | loss 3.475558 (-0.44z)| norm 0.2627 (-0.84z)| lr 4.55e-04 | 322.41 ms | 52.3% bf16 MFU | 1625427 tok/s
step 6884/19560 | loss 3.485410 (-0.19z)| norm 0.2826 (+0.39z)| lr 4.54e-04 | 322.77 ms | 52.3% bf16 MFU | 1625374 tok/s
step 6885/19560 | loss 3.507263 (+0.38z)| norm 0.2718 (-0.28z)| lr 4.54e-04 | 322.82 ms | 52.3% bf16 MFU | 1625309 tok/s
step 6886/19560 | loss 3.554814 (+1.59z)| norm 0.2682 (-0.52z)| lr 4.54e-04 | 322.61 ms | 52.3% bf16 MFU | 1625301 tok/s
step 6887/19560 | loss 3.521684 (+0.76z)| norm 0.2848 (+0.52z)| lr 4.54e-04 | 322.75 ms | 52.3% bf16 MFU | 1625258 tok/s
step 6888/19560 | loss 3.480327 (-0.32z)| norm 0.2899 (+0.83z)| lr 4.54e-04 | 322.93 ms | 52.3% bf16 MFU | 1625172 tok/s
step 6889/19560 | loss 3.435595 (-1.46z)| norm 0.2927 (+1.00z)| lr 4.54e-04 | 323.11 ms | 52.2% bf16 MFU | 1625043 tok/s
step 6890/19560 | loss 3.498550 (+0.16z)| norm 0.2991 (+1.41z)| lr 4.54e-04 | 322.86 ms | 52.3% bf16 MFU | 1624984 tok/s
step 6891/19560 | loss 3.580312 (+2.22z)| norm 0.2753 (-0.07z)| lr 4.54e-04 | 322.31 ms | 52.4% bf16 MFU | 1625068 tok/s
step 6892/19560 | loss 3.453740 (-0.99z)| norm 0.2696 (-0.45z)| lr 4.54e-04 | 322.64 ms | 52.3% bf16 MFU | 1625064 tok/s
step 6893/19560 | loss 3.521354 (+0.72z)| norm 0.2731 (-0.22z)| lr 4.54e-04 | 322.75 ms | 52.3% bf16 MFU | 1625033 tok/s
step 6894/19560 | loss 3.488527 (-0.11z)| norm 0.2986 (+1.39z)| lr 4.54e-04 | 322.54 ms | 52.3% bf16 MFU | 1625056 tok/s
step 6895/19560 | loss 3.497050 (+0.10z)| norm 0.3205 (+2.69z)| lr 4.54e-04 | 322.64 ms | 52.3% bf16 MFU | 1625053 tok/s
step 6896/19560 | loss 3.510036 (+0.43z)| norm 0.3190 (+2.51z)| lr 4.54e-04 | 323.20 ms | 52.2% bf16 MFU | 1624909 tok/s
step 6897/19560 | loss 3.487315 (-0.14z)| norm 0.3420 (+3.66z)| lr 4.54e-04 | 323.35 ms | 52.2% bf16 MFU | 1624734 tok/s
step 6898/19560 | loss 3.473776 (-0.48z)| norm 0.2748 (-0.16z)| lr 4.54e-04 | 322.45 ms | 52.3% bf16 MFU | 1624794 tok/s
step 6899/19560 | loss 3.522583 (+0.76z)| norm 0.3004 (+1.28z)| lr 4.54e-04 | 322.74 ms | 52.3% bf16 MFU | 1624779 tok/s
step 6900/19560 | loss 3.511836 (+0.49z)| norm 0.2700 (-0.46z)| lr 4.54e-04 | 322.99 ms | 52.3% bf16 MFU | 1624703 tok/s
step 6901/19560 | loss 3.479114 (-0.34z)| norm 0.2737 (-0.26z)| lr 4.54e-04 | 322.98 ms | 52.3% bf16 MFU | 1624632 tok/s
step 6902/19560 | loss 3.532627 (+1.01z)| norm 0.2862 (+0.45z)| lr 4.54e-04 | 322.45 ms | 52.3% bf16 MFU | 1624697 tok/s
step 6903/19560 | loss 3.552431 (+1.50z)| norm 0.3007 (+1.28z)| lr 4.54e-04 | 322.74 ms | 52.3% bf16 MFU | 1624688 tok/s
step 6904/19560 | loss 3.612228 (+2.90z)| norm 0.3151 (+2.07z)| lr 4.54e-04 | 322.60 ms | 52.3% bf16 MFU | 1624713 tok/s
step 6905/19560 | loss 3.474136 (-0.51z)| norm 0.3198 (+2.27z)| lr 4.54e-04 | 322.38 ms | 52.4% bf16 MFU | 1624791 tok/s
step 6906/19560 | loss 3.475890 (-0.47z)| norm 0.2936 (+0.78z)| lr 4.54e-04 | 322.98 ms | 52.3% bf16 MFU | 1624715 tok/s
step 6907/19560 | loss 3.508359 (+0.33z)| norm 0.2815 (+0.09z)| lr 4.53e-04 | 322.86 ms | 52.3% bf16 MFU | 1624674 tok/s
step 6908/19560 | loss 3.471355 (-0.59z)| norm 0.2523 (-1.53z)| lr 4.53e-04 | 323.11 ms | 52.2% bf16 MFU | 1624572 tok/s
step 6909/19560 | loss 3.442322 (-1.29z)| norm 0.2799 (+0.02z)| lr 4.53e-04 | 322.89 ms | 52.3% bf16 MFU | 1624530 tok/s
step 6910/19560 | loss 3.531732 (+0.90z)| norm 0.2620 (-1.00z)| lr 4.53e-04 | 322.36 ms | 52.4% bf16 MFU | 1624624 tok/s
step 6911/19560 | loss 3.442884 (-1.29z)| norm 0.2997 (+1.23z)| lr 4.53e-04 | 323.63 ms | 52.1% bf16 MFU | 1624393 tok/s
step 6912/19560 | loss 3.477998 (-0.42z)| norm 0.2704 (-0.49z)| lr 4.53e-04 | 322.57 ms | 52.3% bf16 MFU | 1624442 tok/s
step 6913/19560 | loss 3.527159 (+0.78z)| norm 0.2938 (+0.88z)| lr 4.53e-04 | 322.82 ms | 52.3% bf16 MFU | 1624426 tok/s
step 6914/19560 | loss 3.487436 (-0.19z)| norm 0.2739 (-0.30z)| lr 4.53e-04 | 322.60 ms | 52.3% bf16 MFU | 1624465 tok/s
step 6915/19560 | loss 3.534223 (+0.95z)| norm 0.2702 (-0.51z)| lr 4.53e-04 | 323.22 ms | 52.2% bf16 MFU | 1624345 tok/s
step 6916/19560 | loss 3.516490 (+0.51z)| norm 0.2932 (+0.83z)| lr 4.53e-04 | 322.74 ms | 52.3% bf16 MFU | 1624352 tok/s
step 6917/19560 | loss 3.470546 (-0.61z)| norm 0.2556 (-1.37z)| lr 4.53e-04 | 322.77 ms | 52.3% bf16 MFU | 1624352 tok/s
step 6918/19560 | loss 3.442747 (-1.28z)| norm 0.2643 (-0.86z)| lr 4.53e-04 | 323.12 ms | 52.2% bf16 MFU | 1624263 tok/s
step 6919/19560 | loss 3.612068 (+2.76z)| norm 0.2623 (-0.97z)| lr 4.53e-04 | 322.16 ms | 52.4% bf16 MFU | 1624421 tok/s
step 6920/19560 | loss 3.510791 (+0.36z)| norm 0.2737 (-0.31z)| lr 4.53e-04 | 322.77 ms | 52.3% bf16 MFU | 1624418 tok/s
step 6921/19560 | loss 3.520546 (+0.58z)| norm 0.2516 (-1.59z)| lr 4.53e-04 | 323.30 ms | 52.2% bf16 MFU | 1624282 tok/s
step 6922/19560 | loss 3.475132 (-0.50z)| norm 0.2618 (-0.98z)| lr 4.53e-04 | 322.57 ms | 52.3% bf16 MFU | 1624335 tok/s
step 6923/19560 | loss 3.498110 (+0.05z)| norm 0.2870 (+0.49z)| lr 4.53e-04 | 322.55 ms | 52.3% bf16 MFU | 1624390 tok/s
step 6924/19560 | loss 3.447552 (-1.16z)| norm 0.2858 (+0.41z)| lr 4.53e-04 | 323.37 ms | 52.2% bf16 MFU | 1624238 tok/s
step 6925/19560 | loss 3.445638 (-1.19z)| norm 0.2647 (-0.80z)| lr 4.53e-04 | 322.82 ms | 52.3% bf16 MFU | 1624230 tok/s
step 6926/19560 | loss 3.529849 (+0.83z)| norm 0.2621 (-0.94z)| lr 4.53e-04 | 322.61 ms | 52.3% bf16 MFU | 1624276 tok/s
step 6927/19560 | loss 3.452056 (-1.06z)| norm 0.2548 (-1.35z)| lr 4.53e-04 | 323.25 ms | 52.2% bf16 MFU | 1624159 tok/s
step 6928/19560 | loss 3.442902 (-1.26z)| norm 0.2597 (-1.06z)| lr 4.53e-04 | 322.91 ms | 52.3% bf16 MFU | 1624132 tok/s
step 6929/19560 | loss 3.524874 (+0.71z)| norm 0.2860 (+0.44z)| lr 4.53e-04 | 323.71 ms | 52.1% bf16 MFU | 1623907 tok/s
step 6930/19560 | loss 3.507968 (+0.29z)| norm 0.2641 (-0.82z)| lr 4.52e-04 | 322.50 ms | 52.3% bf16 MFU | 1623996 tok/s
step 6931/19560 | loss 3.474658 (-0.51z)| norm 0.2701 (-0.47z)| lr 4.52e-04 | 322.24 ms | 52.4% bf16 MFU | 1624148 tok/s
step 6932/19560 | loss 3.473806 (-0.53z)| norm 0.2525 (-1.47z)| lr 4.52e-04 | 322.82 ms | 52.3% bf16 MFU | 1624144 tok/s
step 6933/19560 | loss 3.519510 (+0.57z)| norm 0.2745 (-0.21z)| lr 4.52e-04 | 323.01 ms | 52.2% bf16 MFU | 1624093 tok/s
step 6934/19560 | loss 3.532444 (+0.88z)| norm 0.2829 (+0.28z)| lr 4.52e-04 | 322.45 ms | 52.3% bf16 MFU | 1624186 tok/s
step 6935/19560 | loss 3.480659 (-0.37z)| norm 0.2804 (+0.14z)| lr 4.52e-04 | 323.16 ms | 52.2% bf16 MFU | 1624095 tok/s
step 6936/19560 | loss 3.503911 (+0.19z)| norm 0.2725 (-0.32z)| lr 4.52e-04 | 322.68 ms | 52.3% bf16 MFU | 1624130 tok/s
step 6937/19560 | loss 3.504928 (+0.21z)| norm 0.2792 (+0.07z)| lr 4.52e-04 | 322.54 ms | 52.3% bf16 MFU | 1624198 tok/s
step 6938/19560 | loss 3.508514 (+0.31z)| norm 0.2753 (-0.17z)| lr 4.52e-04 | 323.63 ms | 52.1% bf16 MFU | 1623989 tok/s
step 6939/19560 | loss 3.466447 (-0.71z)| norm 0.2908 (+0.75z)| lr 4.52e-04 | 322.59 ms | 52.3% bf16 MFU | 1624052 tok/s
step 6940/19560 | loss 3.576307 (+1.95z)| norm 0.2824 (+0.27z)| lr 4.52e-04 | 322.40 ms | 52.3% bf16 MFU | 1624160 tok/s
step 6941/19560 | loss 3.509759 (+0.33z)| norm 0.2979 (+1.18z)| lr 4.52e-04 | 322.79 ms | 52.3% bf16 MFU | 1624163 tok/s
step 6942/19560 | loss 3.409518 (-2.05z)| norm 0.2572 (-1.22z)| lr 4.52e-04 | 322.91 ms | 52.3% bf16 MFU | 1624137 tok/s
step 6943/19560 | loss 3.521243 (+0.61z)| norm 0.2974 (+1.17z)| lr 4.52e-04 | 322.52 ms | 52.3% bf16 MFU | 1624212 tok/s
step 6944/19560 | loss 3.462069 (-0.79z)| norm 0.2974 (+1.20z)| lr 4.52e-04 | 322.74 ms | 52.3% bf16 MFU | 1624225 tok/s
step 6945/19560 | loss 3.460054 (-0.82z)| norm 0.3064 (+1.71z)| lr 4.52e-04 | 323.14 ms | 52.2% bf16 MFU | 1624138 tok/s
step 6946/19560 | loss 3.544975 (+1.17z)| norm 0.2658 (-0.70z)| lr 4.52e-04 | 322.78 ms | 52.3% bf16 MFU | 1624144 tok/s
step 6947/19560 | loss 3.484695 (-0.24z)| norm 0.3073 (+1.75z)| lr 4.52e-04 | 322.65 ms | 52.3% bf16 MFU | 1624183 tok/s
step 6948/19560 | loss 3.549983 (+1.28z)| norm 0.2587 (-1.11z)| lr 4.52e-04 | 323.19 ms | 52.2% bf16 MFU | 1624085 tok/s
step 6949/19560 | loss 3.498419 (+0.07z)| norm 0.3099 (+1.86z)| lr 4.52e-04 | 322.95 ms | 52.3% bf16 MFU | 1624052 tok/s
step 6950/19560 | loss 3.457528 (-0.89z)| norm 0.2571 (-1.19z)| lr 4.52e-04 | 322.54 ms | 52.3% bf16 MFU | 1624124 tok/s
step 6951/19560 | loss 3.470725 (-0.57z)| norm 0.2518 (-1.47z)| lr 4.52e-04 | 322.38 ms | 52.4% bf16 MFU | 1624232 tok/s
step 6952/19560 | loss 3.421834 (-1.69z)| norm 0.2496 (-1.57z)| lr 4.52e-04 | 322.50 ms | 52.3% bf16 MFU | 1624306 tok/s
step 6953/19560 | loss 3.495889 (+0.03z)| norm 0.2733 (-0.21z)| lr 4.51e-04 | 322.94 ms | 52.3% bf16 MFU | 1624266 tok/s
step 6954/19560 | loss 3.526709 (+0.76z)| norm 0.2452 (-1.78z)| lr 4.51e-04 | 322.37 ms | 52.4% bf16 MFU | 1624370 tok/s
step 6955/19560 | loss 3.458878 (-0.82z)| norm 0.2538 (-1.27z)| lr 4.51e-04 | 322.46 ms | 52.3% bf16 MFU | 1624445 tok/s
step 6956/19560 | loss 3.442696 (-1.20z)| norm 0.2664 (-0.58z)| lr 4.51e-04 | 322.46 ms | 52.3% bf16 MFU | 1624519 tok/s
step 6957/19560 | loss 3.472251 (-0.51z)| norm 0.2575 (-1.07z)| lr 4.51e-04 | 322.17 ms | 52.4% bf16 MFU | 1624660 tok/s
step 6958/19560 | loss 3.474192 (-0.46z)| norm 0.2587 (-1.02z)| lr 4.51e-04 | 323.03 ms | 52.2% bf16 MFU | 1624579 tok/s
step 6959/19560 | loss 3.487190 (-0.17z)| norm 0.2643 (-0.69z)| lr 4.51e-04 | 323.20 ms | 52.2% bf16 MFU | 1624460 tok/s
step 6960/19560 | loss 3.474201 (-0.47z)| norm 0.2515 (-1.39z)| lr 4.51e-04 | 323.03 ms | 52.2% bf16 MFU | 1624389 tok/s
step 6961/19560 | loss 3.486456 (-0.18z)| norm 0.2731 (-0.17z)| lr 4.51e-04 | 322.82 ms | 52.3% bf16 MFU | 1624374 tok/s
step 6962/19560 | loss 3.506605 (+0.30z)| norm 0.3196 (+2.38z)| lr 4.51e-04 | 322.37 ms | 52.4% bf16 MFU | 1624473 tok/s
step 6963/19560 | loss 3.457970 (-0.85z)| norm 0.3301 (+2.85z)| lr 4.51e-04 | 323.54 ms | 52.2% bf16 MFU | 1624272 tok/s
step 6964/19560 | loss 3.473580 (-0.50z)| norm 0.3083 (+1.65z)| lr 4.51e-04 | 322.97 ms | 52.3% bf16 MFU | 1624225 tok/s
step 6965/19560 | loss 3.488000 (-0.13z)| norm 0.2814 (+0.22z)| lr 4.51e-04 | 322.96 ms | 52.3% bf16 MFU | 1624183 tok/s
step 6966/19560 | loss 3.502813 (+0.26z)| norm 0.2973 (+1.06z)| lr 4.51e-04 | 322.74 ms | 52.3% bf16 MFU | 1624199 tok/s
step 6967/19560 | loss 3.466242 (-0.67z)| norm 0.2811 (+0.20z)| lr 4.51e-04 | 322.10 ms | 52.4% bf16 MFU | 1624375 tok/s
step 6968/19560 | loss 3.473089 (-0.49z)| norm 0.2983 (+1.10z)| lr 4.51e-04 | 322.28 ms | 52.4% bf16 MFU | 1624496 tok/s
step 6969/19560 | loss 3.474605 (-0.44z)| norm 0.2547 (-1.20z)| lr 4.51e-04 | 322.18 ms | 52.4% bf16 MFU | 1624636 tok/s
step 6970/19560 | loss 3.485709 (-0.17z)| norm 0.2935 (+0.84z)| lr 4.51e-04 | 322.78 ms | 52.3% bf16 MFU | 1624618 tok/s
step 6971/19560 | loss 3.500283 (+0.21z)| norm 0.2498 (-1.44z)| lr 4.51e-04 | 322.91 ms | 52.3% bf16 MFU | 1624568 tok/s
step 6972/19560 | loss 3.497704 (+0.13z)| norm 0.2722 (-0.27z)| lr 4.51e-04 | 322.71 ms | 52.3% bf16 MFU | 1624572 tok/s
step 6973/19560 | loss 3.502263 (+0.24z)| norm 0.2982 (+1.07z)| lr 4.51e-04 | 322.66 ms | 52.3% bf16 MFU | 1624588 tok/s
step 6974/19560 | loss 3.460302 (-0.86z)| norm 0.2717 (-0.32z)| lr 4.51e-04 | 323.11 ms | 52.2% bf16 MFU | 1624490 tok/s
step 6975/19560 | loss 3.467854 (-0.66z)| norm 0.2964 (+0.97z)| lr 4.51e-04 | 322.70 ms | 52.3% bf16 MFU | 1624501 tok/s
step 6976/19560 | loss 3.524014 (+0.82z)| norm 0.3112 (+1.71z)| lr 4.51e-04 | 322.77 ms | 52.3% bf16 MFU | 1624492 tok/s
step 6977/19560 | loss 3.571095 (+2.02z)| norm 0.2997 (+1.09z)| lr 4.50e-04 | 323.14 ms | 52.2% bf16 MFU | 1624392 tok/s
step 6978/19560 | loss 3.456659 (-0.97z)| norm 0.2885 (+0.50z)| lr 4.50e-04 | 322.43 ms | 52.3% bf16 MFU | 1624476 tok/s
step 6979/19560 | loss 3.525181 (+0.83z)| norm 0.3047 (+1.33z)| lr 4.50e-04 | 322.45 ms | 52.3% bf16 MFU | 1624549 tok/s
step 6980/19560 | loss 3.504424 (+0.28z)| norm 0.3070 (+1.42z)| lr 4.50e-04 | 322.70 ms | 52.3% bf16 MFU | 1624555 tok/s
step 6981/19560 | loss 3.461157 (-0.84z)| norm 0.2695 (-0.52z)| lr 4.50e-04 | 322.65 ms | 52.3% bf16 MFU | 1624574 tok/s
step 6982/19560 | loss 3.493861 (+0.01z)| norm 0.2936 (+0.72z)| lr 4.50e-04 | 322.99 ms | 52.3% bf16 MFU | 1624507 tok/s
step 6983/19560 | loss 3.448590 (-1.17z)| norm 0.2537 (-1.34z)| lr 4.50e-04 | 322.50 ms | 52.3% bf16 MFU | 1624567 tok/s
step 6984/19560 | loss 3.535797 (+1.12z)| norm 0.2677 (-0.62z)| lr 4.50e-04 | 322.56 ms | 52.3% bf16 MFU | 1624608 tok/s
step 6985/19560 | loss 3.535745 (+1.10z)| norm 0.2622 (-0.89z)| lr 4.50e-04 | 323.30 ms | 52.2% bf16 MFU | 1624461 tok/s
step 6986/19560 | loss 3.484138 (-0.25z)| norm 0.2565 (-1.17z)| lr 4.50e-04 | 323.22 ms | 52.2% bf16 MFU | 1624341 tok/s
step 6987/19560 | loss 3.489564 (-0.11z)| norm 0.2692 (-0.52z)| lr 4.50e-04 | 322.37 ms | 52.4% bf16 MFU | 1624441 tok/s
step 6988/19560 | loss 3.471586 (-0.57z)| norm 0.2745 (-0.24z)| lr 4.50e-04 | 322.75 ms | 52.3% bf16 MFU | 1624440 tok/s
step 6989/19560 | loss 3.469873 (-0.61z)| norm 0.2717 (-0.39z)| lr 4.50e-04 | 322.63 ms | 52.3% bf16 MFU | 1624469 tok/s
step 6990/19560 | loss 3.476742 (-0.42z)| norm 0.2625 (-0.86z)| lr 4.50e-04 | 322.86 ms | 52.3% bf16 MFU | 1624439 tok/s
step 6991/19560 | loss 3.510668 (+0.48z)| norm 0.2700 (-0.49z)| lr 4.50e-04 | 322.80 ms | 52.3% bf16 MFU | 1624426 tok/s
step 6992/19560 | loss 3.514968 (+0.58z)| norm 0.2490 (-1.56z)| lr 4.50e-04 | 322.71 ms | 52.3% bf16 MFU | 1624437 tok/s
step 6993/19560 | loss 3.498301 (+0.12z)| norm 0.2864 (+0.36z)| lr 4.50e-04 | 322.75 ms | 52.3% bf16 MFU | 1624438 tok/s
step 6994/19560 | loss 3.486735 (-0.20z)| norm 0.2965 (+0.87z)| lr 4.50e-04 | 322.29 ms | 52.4% bf16 MFU | 1624555 tok/s
step 6995/19560 | loss 3.491066 (-0.07z)| norm 0.2595 (-1.03z)| lr 4.50e-04 | 322.92 ms | 52.3% bf16 MFU | 1624508 tok/s
step 6996/19560 | loss 3.469768 (-0.64z)| norm 0.2595 (-1.01z)| lr 4.50e-04 | 322.57 ms | 52.3% bf16 MFU | 1624550 tok/s
step 6997/19560 | loss 3.483881 (-0.24z)| norm 0.2385 (-2.03z)| lr 4.50e-04 | 322.66 ms | 52.3% bf16 MFU | 1624567 tok/s
step 6998/19560 | loss 3.447558 (-1.25z)| norm 0.2971 (+0.92z)| lr 4.50e-04 | 322.60 ms | 52.3% bf16 MFU | 1624598 tok/s
step 6999/19560 | loss 3.480840 (-0.32z)| norm 0.2874 (+0.42z)| lr 4.50e-04 | 322.59 ms | 52.3% bf16 MFU | 1624631 tok/s
step 7000/19560 | loss 3.499639 (+0.19z)| norm 0.2568 (-1.11z)| lr 4.49e-04 | 322.60 ms | 52.3% bf16 MFU | 1624659 tok/s
val loss 3.476349
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2852/10042 = 0.284007
step 7001/19560 | loss 3.475997 (-0.47z)| norm 0.2837 (+0.24z)| lr 4.49e-04 | 321.93 ms | 52.4% bf16 MFU | 1624855 tok/s
step 7002/19560 | loss 3.462398 (-0.86z)| norm 0.2856 (+0.33z)| lr 4.49e-04 | 322.71 ms | 52.3% bf16 MFU | 1624845 tok/s
step 7003/19560 | loss 3.516041 (+0.66z)| norm 0.2838 (+0.23z)| lr 4.49e-04 | 322.55 ms | 52.3% bf16 MFU | 1624874 tok/s
step 7004/19560 | loss 3.485075 (-0.23z)| norm 0.2794 (+0.01z)| lr 4.49e-04 | 322.67 ms | 52.3% bf16 MFU | 1624872 tok/s
step 7005/19560 | loss 3.515252 (+0.63z)| norm 0.2725 (-0.33z)| lr 4.49e-04 | 322.31 ms | 52.4% bf16 MFU | 1624962 tok/s
step 7006/19560 | loss 3.570453 (+2.15z)| norm 0.2867 (+0.38z)| lr 4.49e-04 | 322.34 ms | 52.4% bf16 MFU | 1625040 tok/s
step 7007/19560 | loss 3.563116 (+1.90z)| norm 0.2853 (+0.31z)| lr 4.49e-04 | 322.07 ms | 52.4% bf16 MFU | 1625182 tok/s
step 7008/19560 | loss 3.555975 (+1.69z)| norm 0.2592 (-1.01z)| lr 4.49e-04 | 322.93 ms | 52.3% bf16 MFU | 1625099 tok/s
step 7009/19560 | loss 3.412119 (-2.23z)| norm 0.2749 (-0.21z)| lr 4.49e-04 | 322.62 ms | 52.3% bf16 MFU | 1625099 tok/s
step 7010/19560 | loss 3.418093 (-2.02z)| norm 0.2664 (-0.63z)| lr 4.49e-04 | 322.15 ms | 52.4% bf16 MFU | 1625218 tok/s
step 7011/19560 | loss 3.541593 (+1.27z)| norm 0.2502 (-1.43z)| lr 4.49e-04 | 323.19 ms | 52.2% bf16 MFU | 1625068 tok/s
step 7012/19560 | loss 3.456484 (-0.99z)| norm 0.2825 (+0.19z)| lr 4.49e-04 | 322.63 ms | 52.3% bf16 MFU | 1625068 tok/s
step 7013/19560 | loss 3.475049 (-0.49z)| norm 0.2688 (-0.50z)| lr 4.49e-04 | 323.43 ms | 52.2% bf16 MFU | 1624866 tok/s
step 7014/19560 | loss 3.493813 (+0.02z)| norm 0.2592 (-0.97z)| lr 4.49e-04 | 322.20 ms | 52.4% bf16 MFU | 1624983 tok/s
step 7015/19560 | loss 3.522416 (+0.79z)| norm 0.2595 (-0.95z)| lr 4.49e-04 | 322.65 ms | 52.3% bf16 MFU | 1624982 tok/s
step 7016/19560 | loss 3.486429 (-0.18z)| norm 0.2350 (-2.11z)| lr 4.49e-04 | 322.36 ms | 52.4% bf16 MFU | 1625052 tok/s
step 7017/19560 | loss 3.495540 (+0.05z)| norm 0.2671 (-0.53z)| lr 4.49e-04 | 323.14 ms | 52.2% bf16 MFU | 1624923 tok/s
step 7018/19560 | loss 3.402888 (-2.39z)| norm 0.2660 (-0.57z)| lr 4.49e-04 | 322.94 ms | 52.3% bf16 MFU | 1624850 tok/s
step 7019/19560 | loss 3.517307 (+0.68z)| norm 0.2659 (-0.57z)| lr 4.49e-04 | 322.73 ms | 52.3% bf16 MFU | 1624836 tok/s
step 7020/19560 | loss 3.500924 (+0.22z)| norm 0.2512 (-1.28z)| lr 4.49e-04 | 322.60 ms | 52.3% bf16 MFU | 1624854 tok/s
step 7021/19560 | loss 3.592958 (+2.64z)| norm 0.2606 (-0.81z)| lr 4.49e-04 | 322.94 ms | 52.3% bf16 MFU | 1624786 tok/s
step 7022/19560 | loss 3.494717 (+0.04z)| norm 0.2541 (-1.11z)| lr 4.49e-04 | 322.49 ms | 52.3% bf16 MFU | 1624834 tok/s
step 7023/19560 | loss 3.516870 (+0.62z)| norm 0.2547 (-1.07z)| lr 4.48e-04 | 322.21 ms | 52.4% bf16 MFU | 1624951 tok/s
step 7024/19560 | loss 3.470592 (-0.60z)| norm 0.2826 (+0.34z)| lr 4.48e-04 | 322.99 ms | 52.3% bf16 MFU | 1624866 tok/s
step 7025/19560 | loss 3.498310 (+0.13z)| norm 0.2676 (-0.41z)| lr 4.48e-04 | 322.94 ms | 52.3% bf16 MFU | 1624796 tok/s
step 7026/19560 | loss 3.522703 (+0.77z)| norm 0.2619 (-0.71z)| lr 4.48e-04 | 322.66 ms | 52.3% bf16 MFU | 1624802 tok/s
step 7027/19560 | loss 3.431662 (-1.60z)| norm 0.2553 (-1.04z)| lr 4.48e-04 | 322.50 ms | 52.3% bf16 MFU | 1624848 tok/s
step 7028/19560 | loss 3.497717 (+0.13z)| norm 0.2647 (-0.54z)| lr 4.48e-04 | 322.15 ms | 52.4% bf16 MFU | 1624978 tok/s
step 7029/19560 | loss 3.511412 (+0.48z)| norm 0.3492 (+3.68z)| lr 4.48e-04 | 322.57 ms | 52.3% bf16 MFU | 1624996 tok/s
step 7030/19560 | loss 3.467401 (-0.66z)| norm 0.3002 (+1.22z)| lr 4.48e-04 | 322.87 ms | 52.3% bf16 MFU | 1624938 tok/s
step 7031/19560 | loss 3.566648 (+1.93z)| norm 0.2526 (-1.13z)| lr 4.48e-04 | 322.65 ms | 52.3% bf16 MFU | 1624940 tok/s
step 7032/19560 | loss 3.456856 (-0.94z)| norm 0.3059 (+1.54z)| lr 4.48e-04 | 322.75 ms | 52.3% bf16 MFU | 1624914 tok/s
step 7033/19560 | loss 3.476744 (-0.40z)| norm 0.2751 (+0.01z)| lr 4.48e-04 | 323.27 ms | 52.2% bf16 MFU | 1624759 tok/s
step 7034/19560 | loss 3.513142 (+0.58z)| norm 0.2635 (-0.57z)| lr 4.48e-04 | 321.60 ms | 52.5% bf16 MFU | 1625033 tok/s
step 7035/19560 | loss 3.548352 (+1.51z)| norm 0.2819 (+0.37z)| lr 4.48e-04 | 321.98 ms | 52.4% bf16 MFU | 1625196 tok/s
step 7036/19560 | loss 3.477593 (-0.39z)| norm 0.2526 (-1.13z)| lr 4.48e-04 | 323.08 ms | 52.2% bf16 MFU | 1625076 tok/s
step 7037/19560 | loss 3.468711 (-0.64z)| norm 0.2778 (+0.17z)| lr 4.48e-04 | 322.29 ms | 52.4% bf16 MFU | 1625162 tok/s
step 7038/19560 | loss 3.488058 (-0.11z)| norm 0.2885 (+0.70z)| lr 4.48e-04 | 322.76 ms | 52.3% bf16 MFU | 1625123 tok/s
step 7039/19560 | loss 3.426331 (-1.77z)| norm 0.3148 (+2.02z)| lr 4.48e-04 | 322.73 ms | 52.3% bf16 MFU | 1625094 tok/s
step 7040/19560 | loss 3.468660 (-0.62z)| norm 0.2469 (-1.40z)| lr 4.48e-04 | 322.34 ms | 52.4% bf16 MFU | 1625163 tok/s
step 7041/19560 | loss 3.502194 (+0.29z)| norm 0.2826 (+0.40z)| lr 4.48e-04 | 322.57 ms | 52.3% bf16 MFU | 1625171 tok/s
step 7042/19560 | loss 3.497350 (+0.15z)| norm 0.2775 (+0.14z)| lr 4.48e-04 | 322.55 ms | 52.3% bf16 MFU | 1625186 tok/s
step 7043/19560 | loss 3.529111 (+1.02z)| norm 0.2830 (+0.41z)| lr 4.48e-04 | 322.92 ms | 52.3% bf16 MFU | 1625105 tok/s
step 7044/19560 | loss 3.454286 (-1.00z)| norm 0.2599 (-0.74z)| lr 4.48e-04 | 322.67 ms | 52.3% bf16 MFU | 1625093 tok/s
step 7045/19560 | loss 3.495002 (+0.10z)| norm 0.2657 (-0.45z)| lr 4.48e-04 | 322.05 ms | 52.4% bf16 MFU | 1625236 tok/s
step 7046/19560 | loss 3.533698 (+1.13z)| norm 0.2556 (-0.96z)| lr 4.47e-04 | 322.82 ms | 52.3% bf16 MFU | 1625178 tok/s
step 7047/19560 | loss 3.518781 (+0.78z)| norm 0.2663 (-0.42z)| lr 4.47e-04 | 322.05 ms | 52.4% bf16 MFU | 1625318 tok/s
step 7048/19560 | loss 3.448436 (-1.19z)| norm 0.2675 (-0.35z)| lr 4.47e-04 | 322.92 ms | 52.3% bf16 MFU | 1625232 tok/s
step 7049/19560 | loss 3.467374 (-0.65z)| norm 0.2501 (-1.23z)| lr 4.47e-04 | 322.74 ms | 52.3% bf16 MFU | 1625194 tok/s
step 7050/19560 | loss 3.490098 (-0.01z)| norm 0.2973 (+1.13z)| lr 4.47e-04 | 322.25 ms | 52.4% bf16 MFU | 1625282 tok/s
step 7051/19560 | loss 3.607737 (+3.15z)| norm 0.2587 (-0.80z)| lr 4.47e-04 | 322.66 ms | 52.3% bf16 MFU | 1625262 tok/s
step 7052/19560 | loss 3.462056 (-0.80z)| norm 0.2536 (-1.04z)| lr 4.47e-04 | 322.45 ms | 52.3% bf16 MFU | 1625296 tok/s
step 7053/19560 | loss 3.475845 (-0.43z)| norm 0.2496 (-1.23z)| lr 4.47e-04 | 322.55 ms | 52.3% bf16 MFU | 1625303 tok/s
step 7054/19560 | loss 3.382007 (-2.88z)| norm 0.2621 (-0.60z)| lr 4.47e-04 | 322.73 ms | 52.3% bf16 MFU | 1625264 tok/s
step 7055/19560 | loss 3.448666 (-1.11z)| norm 0.2724 (-0.10z)| lr 4.47e-04 | 322.23 ms | 52.4% bf16 MFU | 1625353 tok/s
step 7056/19560 | loss 3.562500 (+1.87z)| norm 0.2675 (-0.34z)| lr 4.47e-04 | 322.31 ms | 52.4% bf16 MFU | 1625419 tok/s
step 7057/19560 | loss 3.479932 (-0.30z)| norm 0.2680 (-0.31z)| lr 4.47e-04 | 322.30 ms | 52.4% bf16 MFU | 1625485 tok/s
step 7058/19560 | loss 3.499239 (+0.22z)| norm 0.2956 (+1.06z)| lr 4.47e-04 | 322.97 ms | 52.3% bf16 MFU | 1625376 tok/s
step 7059/19560 | loss 3.444917 (-1.20z)| norm 0.2732 (-0.07z)| lr 4.47e-04 | 322.38 ms | 52.4% bf16 MFU | 1625422 tok/s
step 7060/19560 | loss 3.470477 (-0.53z)| norm 0.2607 (-0.70z)| lr 4.47e-04 | 322.68 ms | 52.3% bf16 MFU | 1625391 tok/s
step 7061/19560 | loss 3.537723 (+1.22z)| norm 0.2802 (+0.28z)| lr 4.47e-04 | 323.08 ms | 52.2% bf16 MFU | 1625260 tok/s
step 7062/19560 | loss 3.485593 (-0.13z)| norm 0.2600 (-0.72z)| lr 4.47e-04 | 322.62 ms | 52.3% bf16 MFU | 1625253 tok/s
step 7063/19560 | loss 3.446059 (-1.16z)| norm 0.2853 (+0.54z)| lr 4.47e-04 | 322.22 ms | 52.4% bf16 MFU | 1625346 tok/s
step 7064/19560 | loss 3.538726 (+1.25z)| norm 0.2647 (-0.49z)| lr 4.47e-04 | 322.70 ms | 52.3% bf16 MFU | 1625314 tok/s
step 7065/19560 | loss 3.463916 (-0.68z)| norm 0.2678 (-0.32z)| lr 4.47e-04 | 322.88 ms | 52.3% bf16 MFU | 1625239 tok/s
step 7066/19560 | loss 3.452459 (-0.97z)| norm 0.2756 (+0.06z)| lr 4.47e-04 | 322.81 ms | 52.3% bf16 MFU | 1625184 tok/s
step 7067/19560 | loss 3.415561 (-1.89z)| norm 0.2984 (+1.20z)| lr 4.47e-04 | 322.03 ms | 52.4% bf16 MFU | 1625328 tok/s
step 7068/19560 | loss 3.476361 (-0.32z)| norm 0.3037 (+1.44z)| lr 4.47e-04 | 323.12 ms | 52.2% bf16 MFU | 1625190 tok/s
step 7069/19560 | loss 3.425318 (-1.62z)| norm 0.2657 (-0.43z)| lr 4.46e-04 | 322.56 ms | 52.3% bf16 MFU | 1625200 tok/s
step 7070/19560 | loss 3.482223 (-0.17z)| norm 0.2926 (+0.89z)| lr 4.46e-04 | 322.64 ms | 52.3% bf16 MFU | 1625189 tok/s
step 7071/19560 | loss 3.551441 (+1.63z)| norm 0.2737 (-0.03z)| lr 4.46e-04 | 322.97 ms | 52.3% bf16 MFU | 1625097 tok/s
step 7072/19560 | loss 3.611722 (+3.06z)| norm 0.3123 (+1.87z)| lr 4.46e-04 | 322.50 ms | 52.3% bf16 MFU | 1625128 tok/s
step 7073/19560 | loss 3.610548 (+2.90z)| norm 0.3146 (+1.97z)| lr 4.46e-04 | 322.62 ms | 52.3% bf16 MFU | 1625127 tok/s
step 7074/19560 | loss 3.515979 (+0.61z)| norm 0.2950 (+0.99z)| lr 4.46e-04 | 323.00 ms | 52.3% bf16 MFU | 1625030 tok/s
step 7075/19560 | loss 3.636874 (+3.37z)| norm 0.3241 (+2.38z)| lr 4.46e-04 | 322.59 ms | 52.3% bf16 MFU | 1625039 tok/s
step 7076/19560 | loss 3.457959 (-0.78z)| norm 0.2972 (+1.06z)| lr 4.46e-04 | 322.65 ms | 52.3% bf16 MFU | 1625034 tok/s
step 7077/19560 | loss 3.470912 (-0.47z)| norm 0.2996 (+1.19z)| lr 4.46e-04 | 322.49 ms | 52.3% bf16 MFU | 1625069 tok/s
step 7078/19560 | loss 3.470374 (-0.49z)| norm 0.2974 (+1.07z)| lr 4.46e-04 | 322.99 ms | 52.3% bf16 MFU | 1624977 tok/s
step 7079/19560 | loss 3.450234 (-0.96z)| norm 0.2825 (+0.33z)| lr 4.46e-04 | 322.55 ms | 52.3% bf16 MFU | 1625002 tok/s
step 7080/19560 | loss 3.468656 (-0.54z)| norm 0.2990 (+1.12z)| lr 4.46e-04 | 322.78 ms | 52.3% bf16 MFU | 1624965 tok/s
step 7081/19560 | loss 3.537258 (+1.07z)| norm 0.2726 (-0.17z)| lr 4.46e-04 | 322.68 ms | 52.3% bf16 MFU | 1624957 tok/s
step 7082/19560 | loss 3.451079 (-0.94z)| norm 0.2967 (+0.99z)| lr 4.46e-04 | 322.40 ms | 52.3% bf16 MFU | 1625020 tok/s
step 7083/19560 | loss 3.470022 (-0.50z)| norm 0.2898 (+0.64z)| lr 4.46e-04 | 322.57 ms | 52.3% bf16 MFU | 1625038 tok/s
step 7084/19560 | loss 3.446525 (-1.05z)| norm 0.2827 (+0.29z)| lr 4.46e-04 | 322.90 ms | 52.3% bf16 MFU | 1624970 tok/s
step 7085/19560 | loss 3.427419 (-1.48z)| norm 0.2694 (-0.38z)| lr 4.46e-04 | 322.45 ms | 52.3% bf16 MFU | 1625020 tok/s
step 7086/19560 | loss 3.469782 (-0.49z)| norm 0.2684 (-0.43z)| lr 4.46e-04 | 323.59 ms | 52.2% bf16 MFU | 1624779 tok/s
step 7087/19560 | loss 3.503857 (+0.30z)| norm 0.2727 (-0.22z)| lr 4.46e-04 | 323.13 ms | 52.2% bf16 MFU | 1624668 tok/s
step 7088/19560 | loss 3.528035 (+0.85z)| norm 0.2627 (-0.72z)| lr 4.46e-04 | 322.72 ms | 52.3% bf16 MFU | 1624663 tok/s
step 7089/19560 | loss 3.472984 (-0.43z)| norm 0.2812 (+0.20z)| lr 4.46e-04 | 322.79 ms | 52.3% bf16 MFU | 1624641 tok/s
step 7090/19560 | loss 3.508574 (+0.40z)| norm 0.2412 (-1.78z)| lr 4.46e-04 | 322.78 ms | 52.3% bf16 MFU | 1624624 tok/s
step 7091/19560 | loss 3.445980 (-1.05z)| norm 0.2616 (-0.75z)| lr 4.46e-04 | 322.53 ms | 52.3% bf16 MFU | 1624670 tok/s
step 7092/19560 | loss 3.501568 (+0.23z)| norm 0.3101 (+1.75z)| lr 4.45e-04 | 323.46 ms | 52.2% bf16 MFU | 1624480 tok/s
step 7093/19560 | loss 3.436006 (-1.27z)| norm 0.3591 (+3.97z)| lr 4.45e-04 | 323.29 ms | 52.2% bf16 MFU | 1624344 tok/s
step 7094/19560 | loss 3.463275 (-0.63z)| norm 0.2889 (+0.59z)| lr 4.45e-04 | 323.11 ms | 52.2% bf16 MFU | 1624258 tok/s
step 7095/19560 | loss 3.489089 (-0.04z)| norm 0.2934 (+0.80z)| lr 4.45e-04 | 323.35 ms | 52.2% bf16 MFU | 1624117 tok/s
step 7096/19560 | loss 3.494730 (+0.08z)| norm 0.3087 (+1.53z)| lr 4.45e-04 | 323.20 ms | 52.2% bf16 MFU | 1624021 tok/s
step 7097/19560 | loss 3.458663 (-0.75z)| norm 0.2584 (-0.89z)| lr 4.45e-04 | 322.44 ms | 52.3% bf16 MFU | 1624119 tok/s
step 7098/19560 | loss 3.468185 (-0.52z)| norm 0.2802 (+0.16z)| lr 4.45e-04 | 322.87 ms | 52.3% bf16 MFU | 1624105 tok/s
step 7099/19560 | loss 3.501913 (+0.25z)| norm 0.2657 (-0.54z)| lr 4.45e-04 | 322.77 ms | 52.3% bf16 MFU | 1624116 tok/s
step 7100/19560 | loss 3.431844 (-1.34z)| norm 0.2815 (+0.22z)| lr 4.45e-04 | 323.47 ms | 52.2% bf16 MFU | 1623951 tok/s
step 7101/19560 | loss 3.455716 (-0.78z)| norm 0.2589 (-0.86z)| lr 4.45e-04 | 323.46 ms | 52.2% bf16 MFU | 1623798 tok/s
step 7102/19560 | loss 3.519160 (+0.65z)| norm 0.2584 (-0.88z)| lr 4.45e-04 | 322.00 ms | 52.4% bf16 MFU | 1624018 tok/s
step 7103/19560 | loss 3.464129 (-0.60z)| norm 0.2518 (-1.18z)| lr 4.45e-04 | 323.21 ms | 52.2% bf16 MFU | 1623924 tok/s
step 7104/19560 | loss 3.428688 (-1.38z)| norm 0.2465 (-1.41z)| lr 4.45e-04 | 322.92 ms | 52.3% bf16 MFU | 1623906 tok/s
step 7105/19560 | loss 3.468863 (-0.46z)| norm 0.2671 (-0.41z)| lr 4.45e-04 | 322.84 ms | 52.3% bf16 MFU | 1623910 tok/s
step 7106/19560 | loss 3.482658 (-0.15z)| norm 0.2584 (-0.82z)| lr 4.45e-04 | 322.74 ms | 52.3% bf16 MFU | 1623937 tok/s
step 7107/19560 | loss 3.491338 (+0.06z)| norm 0.2594 (-0.75z)| lr 4.45e-04 | 322.34 ms | 52.4% bf16 MFU | 1624065 tok/s
step 7108/19560 | loss 3.473000 (-0.36z)| norm 0.2621 (-0.61z)| lr 4.45e-04 | 323.04 ms | 52.2% bf16 MFU | 1624010 tok/s
step 7109/19560 | loss 3.511696 (+0.52z)| norm 0.2686 (-0.29z)| lr 4.45e-04 | 322.49 ms | 52.3% bf16 MFU | 1624098 tok/s
step 7110/19560 | loss 3.455377 (-0.77z)| norm 0.2793 (+0.24z)| lr 4.45e-04 | 322.25 ms | 52.4% bf16 MFU | 1624241 tok/s
step 7111/19560 | loss 3.445338 (-1.00z)| norm 0.2610 (-0.67z)| lr 4.45e-04 | 322.84 ms | 52.3% bf16 MFU | 1624227 tok/s
step 7112/19560 | loss 3.438311 (-1.14z)| norm 0.2516 (-1.12z)| lr 4.45e-04 | 323.63 ms | 52.1% bf16 MFU | 1624016 tok/s
step 7113/19560 | loss 3.466897 (-0.47z)| norm 0.2350 (-1.90z)| lr 4.45e-04 | 322.70 ms | 52.3% bf16 MFU | 1624051 tok/s
step 7114/19560 | loss 3.520052 (+0.75z)| norm 0.2419 (-1.55z)| lr 4.44e-04 | 322.25 ms | 52.4% bf16 MFU | 1624195 tok/s
step 7115/19560 | loss 3.541599 (+1.23z)| norm 0.2747 (+0.03z)| lr 4.44e-04 | 322.55 ms | 52.3% bf16 MFU | 1624257 tok/s
step 7116/19560 | loss 3.498447 (+0.23z)| norm 0.2480 (-1.24z)| lr 4.44e-04 | 322.51 ms | 52.3% bf16 MFU | 1624326 tok/s
step 7117/19560 | loss 3.685386 (+4.17z)| norm 0.2834 (+0.45z)| lr 4.44e-04 | 323.38 ms | 52.2% bf16 MFU | 1624174 tok/s
step 7118/19560 | loss 3.543140 (+1.12z)| norm 0.2975 (+1.11z)| lr 4.44e-04 | 322.43 ms | 52.3% bf16 MFU | 1624267 tok/s
step 7119/19560 | loss 3.500941 (+0.22z)| norm 0.3099 (+1.67z)| lr 4.44e-04 | 322.75 ms | 52.3% bf16 MFU | 1624275 tok/s
step 7120/19560 | loss 3.450876 (-0.83z)| norm 0.3004 (+1.21z)| lr 4.44e-04 | 322.74 ms | 52.3% bf16 MFU | 1624285 tok/s
step 7121/19560 | loss 3.483530 (-0.13z)| norm 0.2986 (+1.11z)| lr 4.44e-04 | 322.75 ms | 52.3% bf16 MFU | 1624293 tok/s
step 7122/19560 | loss 3.491228 (+0.03z)| norm 0.2730 (-0.09z)| lr 4.44e-04 | 323.34 ms | 52.2% bf16 MFU | 1624152 tok/s
step 7123/19560 | loss 3.465630 (-0.51z)| norm 0.2752 (+0.01z)| lr 4.44e-04 | 322.86 ms | 52.3% bf16 MFU | 1624138 tok/s
step 7124/19560 | loss 3.454601 (-0.74z)| norm 0.2799 (+0.23z)| lr 4.44e-04 | 322.85 ms | 52.3% bf16 MFU | 1624127 tok/s
step 7125/19560 | loss 3.488403 (-0.02z)| norm 0.2853 (+0.47z)| lr 4.44e-04 | 323.29 ms | 52.2% bf16 MFU | 1624006 tok/s
step 7126/19560 | loss 3.500342 (+0.22z)| norm 0.3017 (+1.25z)| lr 4.44e-04 | 322.58 ms | 52.3% bf16 MFU | 1624072 tok/s
step 7127/19560 | loss 3.487815 (-0.05z)| norm 0.2863 (+0.52z)| lr 4.44e-04 | 323.12 ms | 52.2% bf16 MFU | 1623997 tok/s
step 7128/19560 | loss 3.465282 (-0.52z)| norm 0.3142 (+1.82z)| lr 4.44e-04 | 322.91 ms | 52.3% bf16 MFU | 1623978 tok/s
step 7129/19560 | loss 3.575362 (+1.78z)| norm 0.2758 (-0.00z)| lr 4.44e-04 | 322.80 ms | 52.3% bf16 MFU | 1623990 tok/s
step 7130/19560 | loss 3.514983 (+0.50z)| norm 0.2990 (+1.09z)| lr 4.44e-04 | 322.96 ms | 52.3% bf16 MFU | 1623960 tok/s
step 7131/19560 | loss 3.520775 (+0.62z)| norm 0.2634 (-0.59z)| lr 4.44e-04 | 322.81 ms | 52.3% bf16 MFU | 1623970 tok/s
step 7132/19560 | loss 3.544139 (+1.10z)| norm 0.2814 (+0.26z)| lr 4.44e-04 | 323.03 ms | 52.2% bf16 MFU | 1623922 tok/s
step 7133/19560 | loss 3.476376 (-0.31z)| norm 0.2748 (-0.05z)| lr 4.44e-04 | 322.82 ms | 52.3% bf16 MFU | 1623931 tok/s
step 7134/19560 | loss 3.538632 (+1.00z)| norm 0.2780 (+0.10z)| lr 4.44e-04 | 322.03 ms | 52.4% bf16 MFU | 1624139 tok/s
step 7135/19560 | loss 3.486558 (-0.08z)| norm 0.2718 (-0.18z)| lr 4.44e-04 | 322.77 ms | 52.3% bf16 MFU | 1624149 tok/s
step 7136/19560 | loss 3.491779 (+0.04z)| norm 0.2664 (-0.44z)| lr 4.44e-04 | 322.84 ms | 52.3% bf16 MFU | 1624141 tok/s
step 7137/19560 | loss 3.506290 (+0.34z)| norm 0.2576 (-0.85z)| lr 4.43e-04 | 322.91 ms | 52.3% bf16 MFU | 1624114 tok/s
step 7138/19560 | loss 3.461262 (-0.64z)| norm 0.2751 (-0.03z)| lr 4.43e-04 | 322.93 ms | 52.3% bf16 MFU | 1624086 tok/s
step 7139/19560 | loss 3.452312 (-0.82z)| norm 0.2854 (+0.45z)| lr 4.43e-04 | 322.57 ms | 52.3% bf16 MFU | 1624149 tok/s
step 7140/19560 | loss 3.447547 (-0.93z)| norm 0.2683 (-0.36z)| lr 4.43e-04 | 322.97 ms | 52.3% bf16 MFU | 1624109 tok/s
step 7141/19560 | loss 3.446848 (-0.93z)| norm 0.2613 (-0.69z)| lr 4.43e-04 | 323.01 ms | 52.3% bf16 MFU | 1624061 tok/s
step 7142/19560 | loss 3.466431 (-0.50z)| norm 0.2484 (-1.29z)| lr 4.43e-04 | 322.92 ms | 52.3% bf16 MFU | 1624038 tok/s
step 7143/19560 | loss 3.557983 (+1.47z)| norm 0.2740 (-0.08z)| lr 4.43e-04 | 322.94 ms | 52.3% bf16 MFU | 1624010 tok/s
step 7144/19560 | loss 3.537006 (+1.00z)| norm 0.2973 (+1.00z)| lr 4.43e-04 | 322.61 ms | 52.3% bf16 MFU | 1624068 tok/s
step 7145/19560 | loss 3.616507 (+2.61z)| norm 0.2911 (+0.70z)| lr 4.43e-04 | 322.44 ms | 52.3% bf16 MFU | 1624163 tok/s
step 7146/19560 | loss 3.505594 (+0.29z)| norm 0.2967 (+0.95z)| lr 4.43e-04 | 323.25 ms | 52.2% bf16 MFU | 1624050 tok/s
step 7147/19560 | loss 3.491524 (-0.01z)| norm 0.3208 (+2.05z)| lr 4.43e-04 | 322.80 ms | 52.3% bf16 MFU | 1624058 tok/s
step 7148/19560 | loss 3.498971 (+0.15z)| norm 0.2735 (-0.18z)| lr 4.43e-04 | 322.76 ms | 52.3% bf16 MFU | 1624074 tok/s
step 7149/19560 | loss 3.505087 (+0.30z)| norm 0.2821 (+0.22z)| lr 4.43e-04 | 321.99 ms | 52.4% bf16 MFU | 1624283 tok/s
step 7150/19560 | loss 3.430464 (-1.29z)| norm 0.2945 (+0.79z)| lr 4.43e-04 | 322.81 ms | 52.3% bf16 MFU | 1624277 tok/s
step 7151/19560 | loss 3.459765 (-0.65z)| norm 0.2813 (+0.16z)| lr 4.43e-04 | 322.75 ms | 52.3% bf16 MFU | 1624285 tok/s
step 7152/19560 | loss 3.478554 (-0.25z)| norm 0.2794 (+0.07z)| lr 4.43e-04 | 323.05 ms | 52.2% bf16 MFU | 1624218 tok/s
step 7153/19560 | loss 3.513167 (+0.49z)| norm 0.2775 (-0.03z)| lr 4.43e-04 | 322.02 ms | 52.4% bf16 MFU | 1624413 tok/s
step 7154/19560 | loss 3.455768 (-0.73z)| norm 0.2517 (-1.25z)| lr 4.43e-04 | 322.56 ms | 52.3% bf16 MFU | 1624463 tok/s
step 7155/19560 | loss 3.487040 (-0.07z)| norm 0.2611 (-0.81z)| lr 4.43e-04 | 322.74 ms | 52.3% bf16 MFU | 1624463 tok/s
step 7156/19560 | loss 3.452472 (-0.80z)| norm 0.2614 (-0.79z)| lr 4.43e-04 | 322.54 ms | 52.3% bf16 MFU | 1624515 tok/s
step 7157/19560 | loss 3.627530 (+2.84z)| norm 0.3981 (+5.29z)| lr 4.43e-04 | 322.06 ms | 52.4% bf16 MFU | 1624687 tok/s
step 7158/19560 | loss 3.487537 (-0.07z)| norm 0.2831 (+0.22z)| lr 4.43e-04 | 322.54 ms | 52.3% bf16 MFU | 1624728 tok/s
step 7159/19560 | loss 3.552338 (+1.28z)| norm 0.3054 (+1.18z)| lr 4.43e-04 | 322.14 ms | 52.4% bf16 MFU | 1624868 tok/s
step 7160/19560 | loss 3.460609 (-0.63z)| norm 0.2800 (+0.07z)| lr 4.42e-04 | 322.50 ms | 52.3% bf16 MFU | 1624909 tok/s
step 7161/19560 | loss 3.480021 (-0.23z)| norm 0.2946 (+0.71z)| lr 4.42e-04 | 322.50 ms | 52.3% bf16 MFU | 1624948 tok/s
step 7162/19560 | loss 3.455851 (-0.72z)| norm 0.3058 (+1.19z)| lr 4.42e-04 | 322.56 ms | 52.3% bf16 MFU | 1624972 tok/s
step 7163/19560 | loss 3.498172 (+0.17z)| norm 0.3250 (+1.99z)| lr 4.42e-04 | 322.57 ms | 52.3% bf16 MFU | 1624991 tok/s
step 7164/19560 | loss 3.489698 (-0.01z)| norm 0.3254 (+1.97z)| lr 4.42e-04 | 321.98 ms | 52.4% bf16 MFU | 1625157 tok/s
step 7165/19560 | loss 3.485497 (-0.10z)| norm 0.3193 (+1.67z)| lr 4.42e-04 | 322.37 ms | 52.4% bf16 MFU | 1625218 tok/s
step 7166/19560 | loss 3.459040 (-0.65z)| norm 0.2949 (+0.63z)| lr 4.42e-04 | 322.76 ms | 52.3% bf16 MFU | 1625176 tok/s
step 7167/19560 | loss 3.455248 (-0.74z)| norm 0.3046 (+1.04z)| lr 4.42e-04 | 322.46 ms | 52.3% bf16 MFU | 1625212 tok/s
step 7168/19560 | loss 3.431167 (-1.23z)| norm 0.2875 (+0.30z)| lr 4.42e-04 | 322.74 ms | 52.3% bf16 MFU | 1625175 tok/s
step 7169/19560 | loss 3.440797 (-1.02z)| norm 0.2685 (-0.51z)| lr 4.42e-04 | 321.96 ms | 52.4% bf16 MFU | 1625336 tok/s
step 7170/19560 | loss 3.470902 (-0.39z)| norm 0.2916 (+0.48z)| lr 4.42e-04 | 323.05 ms | 52.2% bf16 MFU | 1625216 tok/s
step 7171/19560 | loss 3.533939 (+0.93z)| norm 0.2734 (-0.30z)| lr 4.42e-04 | 322.71 ms | 52.3% bf16 MFU | 1625188 tok/s
step 7172/19560 | loss 3.479546 (-0.21z)| norm 0.2848 (+0.18z)| lr 4.42e-04 | 322.31 ms | 52.4% bf16 MFU | 1625263 tok/s
step 7173/19560 | loss 3.412029 (-1.59z)| norm 0.2621 (-0.80z)| lr 4.42e-04 | 322.52 ms | 52.3% bf16 MFU | 1625279 tok/s
step 7174/19560 | loss 3.510180 (+0.44z)| norm 0.3018 (+0.90z)| lr 4.42e-04 | 323.15 ms | 52.2% bf16 MFU | 1625136 tok/s
step 7175/19560 | loss 3.441205 (-0.97z)| norm 0.2676 (-0.58z)| lr 4.42e-04 | 322.52 ms | 52.3% bf16 MFU | 1625160 tok/s
step 7176/19560 | loss 3.451150 (-0.77z)| norm 0.2631 (-0.77z)| lr 4.42e-04 | 322.13 ms | 52.4% bf16 MFU | 1625281 tok/s
step 7177/19560 | loss 3.479210 (-0.19z)| norm 0.2864 (+0.23z)| lr 4.42e-04 | 322.76 ms | 52.3% bf16 MFU | 1625236 tok/s
step 7178/19560 | loss 3.460668 (-0.57z)| norm 0.2427 (-1.64z)| lr 4.42e-04 | 322.44 ms | 52.3% bf16 MFU | 1625274 tok/s
step 7179/19560 | loss 3.480625 (-0.14z)| norm 0.2910 (+0.43z)| lr 4.42e-04 | 322.73 ms | 52.3% bf16 MFU | 1625238 tok/s
step 7180/19560 | loss 3.489852 (+0.06z)| norm 0.2782 (-0.13z)| lr 4.42e-04 | 323.00 ms | 52.3% bf16 MFU | 1625135 tok/s
step 7181/19560 | loss 3.624527 (+2.80z)| norm 0.3002 (+0.81z)| lr 4.42e-04 | 322.29 ms | 52.4% bf16 MFU | 1625217 tok/s
step 7182/19560 | loss 3.462041 (-0.57z)| norm 0.2628 (-0.82z)| lr 4.42e-04 | 321.90 ms | 52.4% bf16 MFU | 1625394 tok/s
step 7183/19560 | loss 3.492702 (+0.07z)| norm 0.2840 (+0.10z)| lr 4.41e-04 | 322.63 ms | 52.3% bf16 MFU | 1625375 tok/s
step 7184/19560 | loss 3.509589 (+0.44z)| norm 0.2659 (-0.69z)| lr 4.41e-04 | 322.49 ms | 52.3% bf16 MFU | 1625394 tok/s
step 7185/19560 | loss 3.463547 (-0.54z)| norm 0.2698 (-0.52z)| lr 4.41e-04 | 322.55 ms | 52.3% bf16 MFU | 1625397 tok/s
step 7186/19560 | loss 3.433472 (-1.16z)| norm 0.2760 (-0.24z)| lr 4.41e-04 | 322.76 ms | 52.3% bf16 MFU | 1625346 tok/s
step 7187/19560 | loss 3.564937 (+1.58z)| norm 0.3212 (+1.70z)| lr 4.41e-04 | 322.32 ms | 52.4% bf16 MFU | 1625408 tok/s
step 7188/19560 | loss 3.442663 (-0.97z)| norm 0.2768 (-0.23z)| lr 4.41e-04 | 322.53 ms | 52.3% bf16 MFU | 1625414 tok/s
step 7189/19560 | loss 3.571009 (+1.69z)| norm 0.2815 (-0.02z)| lr 4.41e-04 | 322.83 ms | 52.3% bf16 MFU | 1625346 tok/s
step 7190/19560 | loss 3.452312 (-0.76z)| norm 0.2520 (-1.30z)| lr 4.41e-04 | 322.96 ms | 52.3% bf16 MFU | 1625247 tok/s
step 7191/19560 | loss 3.564978 (+1.54z)| norm 0.5727 (+8.36z)| lr 4.41e-04 | 322.51 ms | 52.3% bf16 MFU | 1625266 tok/s
step 7192/19560 | loss 3.449044 (-0.83z)| norm 0.3116 (+0.78z)| lr 4.41e-04 | 322.58 ms | 52.3% bf16 MFU | 1625267 tok/s
step 7193/19560 | loss 3.596269 (+2.14z)| norm 0.3158 (+0.89z)| lr 4.41e-04 | 322.58 ms | 52.3% bf16 MFU | 1625268 tok/s
step 7194/19560 | loss 3.430813 (-1.19z)| norm 0.3251 (+1.14z)| lr 4.41e-04 | 322.67 ms | 52.3% bf16 MFU | 1625247 tok/s
step 7195/19560 | loss 3.447108 (-0.88z)| norm 0.3009 (+0.45z)| lr 4.41e-04 | 322.54 ms | 52.3% bf16 MFU | 1625259 tok/s
step 7196/19560 | loss 3.498634 (+0.16z)| norm 0.3122 (+0.77z)| lr 4.41e-04 | 322.03 ms | 52.4% bf16 MFU | 1625400 tok/s
step 7197/19560 | loss 3.522844 (+0.64z)| norm 0.2984 (+0.36z)| lr 4.41e-04 | 322.62 ms | 52.3% bf16 MFU | 1625385 tok/s
step 7198/19560 | loss 3.435575 (-1.12z)| norm 0.2902 (+0.13z)| lr 4.41e-04 | 322.46 ms | 52.3% bf16 MFU | 1625411 tok/s
step 7199/19560 | loss 3.490169 (-0.01z)| norm 0.2758 (-0.29z)| lr 4.41e-04 | 322.72 ms | 52.3% bf16 MFU | 1625370 tok/s
step 7200/19560 | loss 3.390737 (-2.02z)| norm 0.3083 (+0.65z)| lr 4.41e-04 | 322.87 ms | 52.3% bf16 MFU | 1625292 tok/s
step 7201/19560 | loss 3.485691 (-0.04z)| norm 0.2708 (-0.42z)| lr 4.41e-04 | 322.62 ms | 52.3% bf16 MFU | 1625282 tok/s
step 7202/19560 | loss 3.501874 (+0.30z)| norm 0.2896 (+0.12z)| lr 4.41e-04 | 322.30 ms | 52.4% bf16 MFU | 1625353 tok/s
step 7203/19560 | loss 3.431268 (-1.20z)| norm 0.2816 (-0.10z)| lr 4.41e-04 | 322.54 ms | 52.3% bf16 MFU | 1625361 tok/s
step 7204/19560 | loss 3.441157 (-0.98z)| norm 0.2695 (-0.44z)| lr 4.41e-04 | 322.37 ms | 52.4% bf16 MFU | 1625410 tok/s
step 7205/19560 | loss 3.467438 (-0.40z)| norm 0.2603 (-0.70z)| lr 4.40e-04 | 322.84 ms | 52.3% bf16 MFU | 1625340 tok/s
step 7206/19560 | loss 3.474076 (-0.26z)| norm 0.2672 (-0.49z)| lr 4.40e-04 | 322.92 ms | 52.3% bf16 MFU | 1625253 tok/s
step 7207/19560 | loss 3.590276 (+2.21z)| norm 0.3099 (+0.73z)| lr 4.40e-04 | 322.94 ms | 52.3% bf16 MFU | 1625165 tok/s
step 7208/19560 | loss 3.421425 (-1.39z)| norm 0.2888 (+0.13z)| lr 4.40e-04 | 323.00 ms | 52.3% bf16 MFU | 1625065 tok/s
step 7209/19560 | loss 3.533367 (+1.00z)| norm 0.3036 (+0.55z)| lr 4.40e-04 | 322.23 ms | 52.4% bf16 MFU | 1625165 tok/s
step 7210/19560 | loss 3.490002 (+0.07z)| norm 0.2680 (-0.47z)| lr 4.40e-04 | 322.63 ms | 52.3% bf16 MFU | 1625158 tok/s
step 7211/19560 | loss 3.464162 (-0.49z)| norm 0.2639 (-0.58z)| lr 4.40e-04 | 322.59 ms | 52.3% bf16 MFU | 1625162 tok/s
step 7212/19560 | loss 3.427044 (-1.27z)| norm 0.2785 (-0.16z)| lr 4.40e-04 | 322.83 ms | 52.3% bf16 MFU | 1625105 tok/s
step 7213/19560 | loss 3.471140 (-0.34z)| norm 0.2803 (-0.11z)| lr 4.40e-04 | 322.53 ms | 52.3% bf16 MFU | 1625128 tok/s
step 7214/19560 | loss 3.361641 (-2.60z)| norm 0.2509 (-0.95z)| lr 4.40e-04 | 322.29 ms | 52.4% bf16 MFU | 1625210 tok/s
step 7215/19560 | loss 3.492634 (+0.14z)| norm 0.2765 (-0.22z)| lr 4.40e-04 | 322.98 ms | 52.3% bf16 MFU | 1625114 tok/s
step 7216/19560 | loss 3.559509 (+1.51z)| norm 0.2630 (-0.61z)| lr 4.40e-04 | 322.67 ms | 52.3% bf16 MFU | 1625100 tok/s
step 7217/19560 | loss 3.589471 (+2.08z)| norm 0.2697 (-0.41z)| lr 4.40e-04 | 322.38 ms | 52.4% bf16 MFU | 1625162 tok/s
step 7218/19560 | loss 3.433770 (-1.07z)| norm 0.2834 (-0.03z)| lr 4.40e-04 | 322.35 ms | 52.4% bf16 MFU | 1625226 tok/s
step 7219/19560 | loss 3.523649 (+0.74z)| norm 0.2616 (-0.66z)| lr 4.40e-04 | 323.33 ms | 52.2% bf16 MFU | 1625040 tok/s
step 7220/19560 | loss 3.467506 (-0.40z)| norm 0.2678 (-0.47z)| lr 4.40e-04 | 322.41 ms | 52.3% bf16 MFU | 1625096 tok/s
step 7221/19560 | loss 3.469066 (-0.37z)| norm 0.2781 (-0.16z)| lr 4.40e-04 | 322.60 ms | 52.3% bf16 MFU | 1625102 tok/s
step 7222/19560 | loss 3.452058 (-0.72z)| norm 0.2913 (+0.23z)| lr 4.40e-04 | 322.58 ms | 52.3% bf16 MFU | 1625113 tok/s
step 7223/19560 | loss 3.417910 (-1.39z)| norm 0.2501 (-0.97z)| lr 4.40e-04 | 322.21 ms | 52.4% bf16 MFU | 1625214 tok/s
step 7224/19560 | loss 3.473463 (-0.26z)| norm 0.2806 (-0.07z)| lr 4.40e-04 | 322.63 ms | 52.3% bf16 MFU | 1625207 tok/s
step 7225/19560 | loss 3.483101 (-0.07z)| norm 0.2475 (-1.04z)| lr 4.40e-04 | 322.58 ms | 52.3% bf16 MFU | 1625211 tok/s
step 7226/19560 | loss 3.513634 (+0.54z)| norm 0.3037 (+0.61z)| lr 4.40e-04 | 322.34 ms | 52.4% bf16 MFU | 1625277 tok/s
step 7227/19560 | loss 3.405846 (-1.61z)| norm 0.2721 (-0.32z)| lr 4.40e-04 | 322.38 ms | 52.4% bf16 MFU | 1625328 tok/s
step 7228/19560 | loss 3.431983 (-1.09z)| norm 0.2841 (+0.03z)| lr 4.39e-04 | 322.39 ms | 52.3% bf16 MFU | 1625373 tok/s
step 7229/19560 | loss 3.438780 (-0.95z)| norm 0.2639 (-0.56z)| lr 4.39e-04 | 322.36 ms | 52.4% bf16 MFU | 1625425 tok/s
step 7230/19560 | loss 3.479969 (-0.12z)| norm 0.2623 (-0.61z)| lr 4.39e-04 | 322.54 ms | 52.3% bf16 MFU | 1625427 tok/s
step 7231/19560 | loss 3.444539 (-0.82z)| norm 0.2759 (-0.22z)| lr 4.39e-04 | 322.25 ms | 52.4% bf16 MFU | 1625503 tok/s
step 7232/19560 | loss 3.464966 (-0.42z)| norm 0.2488 (-1.02z)| lr 4.39e-04 | 322.49 ms | 52.3% bf16 MFU | 1625516 tok/s
step 7233/19560 | loss 3.437322 (-0.97z)| norm 0.2849 (+0.04z)| lr 4.39e-04 | 322.72 ms | 52.3% bf16 MFU | 1625470 tok/s
step 7234/19560 | loss 3.477724 (-0.16z)| norm 0.3038 (+0.59z)| lr 4.39e-04 | 322.38 ms | 52.4% bf16 MFU | 1625511 tok/s
step 7235/19560 | loss 3.502806 (+0.34z)| norm 0.2886 (+0.14z)| lr 4.39e-04 | 322.53 ms | 52.3% bf16 MFU | 1625512 tok/s
step 7236/19560 | loss 3.474487 (-0.23z)| norm 0.2803 (-0.11z)| lr 4.39e-04 | 322.64 ms | 52.3% bf16 MFU | 1625486 tok/s
step 7237/19560 | loss 3.554421 (+1.35z)| norm 0.3053 (+0.62z)| lr 4.39e-04 | 322.67 ms | 52.3% bf16 MFU | 1625453 tok/s
step 7238/19560 | loss 3.473180 (-0.26z)| norm 0.2761 (-0.25z)| lr 4.39e-04 | 322.27 ms | 52.4% bf16 MFU | 1625524 tok/s
step 7239/19560 | loss 3.466214 (-0.40z)| norm 0.3016 (+0.50z)| lr 4.39e-04 | 323.19 ms | 52.2% bf16 MFU | 1625358 tok/s
step 7240/19560 | loss 3.509569 (+0.45z)| norm 0.3052 (+0.60z)| lr 4.39e-04 | 322.51 ms | 52.3% bf16 MFU | 1625373 tok/s
step 7241/19560 | loss 3.496273 (+0.18z)| norm 0.2809 (-0.14z)| lr 4.39e-04 | 322.53 ms | 52.3% bf16 MFU | 1625382 tok/s
step 7242/19560 | loss 3.457455 (-0.59z)| norm 0.3021 (+0.49z)| lr 4.39e-04 | 322.74 ms | 52.3% bf16 MFU | 1625337 tok/s
step 7243/19560 | loss 3.466427 (-0.40z)| norm 0.2995 (+0.40z)| lr 4.39e-04 | 322.28 ms | 52.4% bf16 MFU | 1625412 tok/s
step 7244/19560 | loss 3.503286 (+0.34z)| norm 0.2703 (-0.49z)| lr 4.39e-04 | 322.76 ms | 52.3% bf16 MFU | 1625359 tok/s
step 7245/19560 | loss 3.458557 (-0.55z)| norm 0.3093 (+0.69z)| lr 4.39e-04 | 322.48 ms | 52.3% bf16 MFU | 1625381 tok/s
step 7246/19560 | loss 3.478098 (-0.13z)| norm 0.2673 (-0.58z)| lr 4.39e-04 | 322.31 ms | 52.4% bf16 MFU | 1625444 tok/s
step 7247/19560 | loss 3.450899 (-0.70z)| norm 0.2923 (+0.18z)| lr 4.39e-04 | 322.61 ms | 52.3% bf16 MFU | 1625429 tok/s
step 7248/19560 | loss 3.447001 (-0.79z)| norm 0.2799 (-0.18z)| lr 4.39e-04 | 322.56 ms | 52.3% bf16 MFU | 1625427 tok/s
step 7249/19560 | loss 3.530072 (+0.99z)| norm 0.2827 (-0.10z)| lr 4.39e-04 | 322.84 ms | 52.3% bf16 MFU | 1625353 tok/s
step 7250/19560 | loss 3.501363 (+0.37z)| norm 0.2924 (+0.19z)| lr 4.39e-04 | 322.19 ms | 52.4% bf16 MFU | 1625449 tok/s
val loss 3.470035
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2831/10042 = 0.281916
step 7251/19560 | loss 3.441615 (-0.90z)| norm 0.3020 (+0.48z)| lr 4.38e-04 | 322.78 ms | 52.3% bf16 MFU | 1625391 tok/s
step 7252/19560 | loss 3.462667 (-0.45z)| norm 0.2803 (-0.18z)| lr 4.38e-04 | 322.37 ms | 52.4% bf16 MFU | 1625438 tok/s
step 7253/19560 | loss 3.520429 (+0.78z)| norm 0.2823 (-0.12z)| lr 4.38e-04 | 322.77 ms | 52.3% bf16 MFU | 1625383 tok/s
step 7254/19560 | loss 3.515687 (+0.67z)| norm 0.2748 (-0.34z)| lr 4.38e-04 | 322.72 ms | 52.3% bf16 MFU | 1625345 tok/s
step 7255/19560 | loss 3.496807 (+0.27z)| norm 0.2491 (-1.11z)| lr 4.38e-04 | 322.70 ms | 52.3% bf16 MFU | 1625311 tok/s
step 7256/19560 | loss 3.490825 (+0.14z)| norm 0.2751 (-0.31z)| lr 4.38e-04 | 322.42 ms | 52.3% bf16 MFU | 1625350 tok/s
step 7257/19560 | loss 3.393240 (-1.91z)| norm 0.2516 (-1.01z)| lr 4.38e-04 | 322.63 ms | 52.3% bf16 MFU | 1625335 tok/s
step 7258/19560 | loss 3.481246 (-0.03z)| norm 0.2722 (-0.39z)| lr 4.38e-04 | 322.93 ms | 52.3% bf16 MFU | 1625245 tok/s
step 7259/19560 | loss 3.475977 (-0.14z)| norm 0.2750 (-0.31z)| lr 4.38e-04 | 322.63 ms | 52.3% bf16 MFU | 1625236 tok/s
step 7260/19560 | loss 3.476937 (-0.11z)| norm 0.2722 (-0.39z)| lr 4.38e-04 | 322.98 ms | 52.3% bf16 MFU | 1625139 tok/s
step 7261/19560 | loss 3.491877 (+0.21z)| norm 0.2838 (-0.04z)| lr 4.38e-04 | 322.59 ms | 52.3% bf16 MFU | 1625144 tok/s
step 7262/19560 | loss 3.476505 (-0.11z)| norm 0.3072 (+0.66z)| lr 4.38e-04 | 322.25 ms | 52.4% bf16 MFU | 1625235 tok/s
step 7263/19560 | loss 3.423966 (-1.23z)| norm 0.2617 (-0.71z)| lr 4.38e-04 | 323.52 ms | 52.2% bf16 MFU | 1625003 tok/s
step 7264/19560 | loss 3.506929 (+0.55z)| norm 0.2686 (-0.50z)| lr 4.38e-04 | 323.39 ms | 52.2% bf16 MFU | 1624814 tok/s
step 7265/19560 | loss 3.463127 (-0.38z)| norm 0.2921 (+0.20z)| lr 4.38e-04 | 322.73 ms | 52.3% bf16 MFU | 1624800 tok/s
step 7266/19560 | loss 3.462936 (-0.39z)| norm 0.2652 (-0.61z)| lr 4.38e-04 | 322.48 ms | 52.3% bf16 MFU | 1624849 tok/s
step 7267/19560 | loss 3.538099 (+1.21z)| norm 0.2762 (-0.28z)| lr 4.38e-04 | 323.10 ms | 52.2% bf16 MFU | 1624741 tok/s
step 7268/19560 | loss 3.482383 (+0.01z)| norm 0.2931 (+0.23z)| lr 4.38e-04 | 323.15 ms | 52.2% bf16 MFU | 1624626 tok/s
step 7269/19560 | loss 3.511497 (+0.63z)| norm 0.3087 (+0.69z)| lr 4.38e-04 | 322.28 ms | 52.4% bf16 MFU | 1624734 tok/s
step 7270/19560 | loss 3.532793 (+1.07z)| norm 0.2977 (+0.34z)| lr 4.38e-04 | 323.16 ms | 52.2% bf16 MFU | 1624616 tok/s
step 7271/19560 | loss 3.454285 (-0.60z)| norm 0.2638 (-0.68z)| lr 4.38e-04 | 322.52 ms | 52.3% bf16 MFU | 1624664 tok/s
step 7272/19560 | loss 3.455448 (-0.56z)| norm 0.2964 (+0.31z)| lr 4.38e-04 | 322.57 ms | 52.3% bf16 MFU | 1624699 tok/s
step 7273/19560 | loss 3.487471 (+0.16z)| norm 0.2530 (-1.00z)| lr 4.37e-04 | 323.18 ms | 52.2% bf16 MFU | 1624578 tok/s
step 7274/19560 | loss 3.472332 (-0.17z)| norm 0.2789 (-0.21z)| lr 4.37e-04 | 322.09 ms | 52.4% bf16 MFU | 1624736 tok/s
step 7275/19560 | loss 3.447038 (-0.73z)| norm 0.2837 (-0.06z)| lr 4.37e-04 | 322.72 ms | 52.3% bf16 MFU | 1624728 tok/s
step 7276/19560 | loss 3.454100 (-0.57z)| norm 0.2835 (-0.06z)| lr 4.37e-04 | 323.10 ms | 52.2% bf16 MFU | 1624625 tok/s
step 7277/19560 | loss 3.471077 (-0.18z)| norm 0.2813 (-0.13z)| lr 4.37e-04 | 322.72 ms | 52.3% bf16 MFU | 1624623 tok/s
step 7278/19560 | loss 3.507125 (+0.62z)| norm 0.2750 (-0.32z)| lr 4.37e-04 | 323.08 ms | 52.2% bf16 MFU | 1624530 tok/s
step 7279/19560 | loss 3.548743 (+1.53z)| norm 0.3322 (+1.39z)| lr 4.37e-04 | 322.66 ms | 52.3% bf16 MFU | 1624549 tok/s
step 7280/19560 | loss 3.509979 (+0.65z)| norm 0.3194 (+1.00z)| lr 4.37e-04 | 323.03 ms | 52.2% bf16 MFU | 1624474 tok/s
step 7281/19560 | loss 3.449338 (-0.69z)| norm 0.3024 (+0.48z)| lr 4.37e-04 | 323.19 ms | 52.2% bf16 MFU | 1624363 tok/s
step 7282/19560 | loss 3.414909 (-1.44z)| norm 0.2727 (-0.42z)| lr 4.37e-04 | 322.57 ms | 52.3% bf16 MFU | 1624412 tok/s
step 7283/19560 | loss 3.445139 (-0.76z)| norm 0.3187 (+0.95z)| lr 4.37e-04 | 322.71 ms | 52.3% bf16 MFU | 1624424 tok/s
step 7284/19560 | loss 3.497700 (+0.39z)| norm 0.3037 (+0.49z)| lr 4.37e-04 | 322.69 ms | 52.3% bf16 MFU | 1624439 tok/s
step 7285/19560 | loss 3.518555 (+0.91z)| norm 0.2911 (+0.15z)| lr 4.37e-04 | 322.40 ms | 52.3% bf16 MFU | 1624526 tok/s
step 7286/19560 | loss 3.461222 (-0.41z)| norm 0.3018 (+0.48z)| lr 4.37e-04 | 322.43 ms | 52.3% bf16 MFU | 1624604 tok/s
step 7287/19560 | loss 3.445940 (-0.75z)| norm 0.2677 (-0.58z)| lr 4.37e-04 | 323.11 ms | 52.2% bf16 MFU | 1624504 tok/s
step 7288/19560 | loss 3.450936 (-0.63z)| norm 0.2732 (-0.41z)| lr 4.37e-04 | 322.83 ms | 52.3% bf16 MFU | 1624481 tok/s
step 7289/19560 | loss 3.480612 (+0.06z)| norm 0.2829 (-0.10z)| lr 4.37e-04 | 323.25 ms | 52.2% bf16 MFU | 1624353 tok/s
step 7290/19560 | loss 3.473442 (-0.11z)| norm 0.2776 (-0.26z)| lr 4.37e-04 | 322.28 ms | 52.4% bf16 MFU | 1624476 tok/s
step 7291/19560 | loss 3.702245 (+4.71z)| norm 0.3427 (+1.77z)| lr 4.37e-04 | 322.79 ms | 52.3% bf16 MFU | 1624463 tok/s
step 7292/19560 | loss 3.506065 (+0.56z)| norm 0.3766 (+2.75z)| lr 4.37e-04 | 323.00 ms | 52.3% bf16 MFU | 1624400 tok/s
step 7293/19560 | loss 3.503928 (+0.51z)| norm 0.2997 (+0.41z)| lr 4.37e-04 | 322.50 ms | 52.3% bf16 MFU | 1624465 tok/s
step 7294/19560 | loss 3.473432 (-0.14z)| norm 0.2747 (-0.35z)| lr 4.37e-04 | 323.27 ms | 52.2% bf16 MFU | 1624333 tok/s
step 7295/19560 | loss 3.460823 (-0.41z)| norm 0.2626 (-0.71z)| lr 4.37e-04 | 322.66 ms | 52.3% bf16 MFU | 1624360 tok/s
step 7296/19560 | loss 3.492409 (+0.25z)| norm 0.2588 (-0.82z)| lr 4.36e-04 | 323.49 ms | 52.2% bf16 MFU | 1624178 tok/s
step 7297/19560 | loss 3.488590 (+0.16z)| norm 0.2587 (-0.82z)| lr 4.36e-04 | 322.54 ms | 52.3% bf16 MFU | 1624245 tok/s
step 7298/19560 | loss 3.472330 (-0.18z)| norm 0.2592 (-0.79z)| lr 4.36e-04 | 322.60 ms | 52.3% bf16 MFU | 1624292 tok/s
step 7299/19560 | loss 3.514491 (+0.72z)| norm 0.2692 (-0.49z)| lr 4.36e-04 | 322.92 ms | 52.3% bf16 MFU | 1624258 tok/s
step 7300/19560 | loss 3.429751 (-1.08z)| norm 0.2757 (-0.29z)| lr 4.36e-04 | 322.96 ms | 52.3% bf16 MFU | 1624214 tok/s
step 7301/19560 | loss 3.467528 (-0.28z)| norm 0.2670 (-0.55z)| lr 4.36e-04 | 322.88 ms | 52.3% bf16 MFU | 1624193 tok/s
step 7302/19560 | loss 3.442820 (-0.80z)| norm 0.2748 (-0.31z)| lr 4.36e-04 | 323.18 ms | 52.2% bf16 MFU | 1624097 tok/s
step 7303/19560 | loss 3.501840 (+0.45z)| norm 0.2815 (-0.11z)| lr 4.36e-04 | 323.05 ms | 52.2% bf16 MFU | 1624038 tok/s
step 7304/19560 | loss 3.462310 (-0.40z)| norm 0.2580 (-0.82z)| lr 4.36e-04 | 322.63 ms | 52.3% bf16 MFU | 1624090 tok/s
step 7305/19560 | loss 3.535858 (+1.16z)| norm 0.2938 (+0.26z)| lr 4.36e-04 | 323.24 ms | 52.2% bf16 MFU | 1623984 tok/s
step 7306/19560 | loss 3.485526 (+0.09z)| norm 0.2746 (-0.33z)| lr 4.36e-04 | 322.20 ms | 52.4% bf16 MFU | 1624145 tok/s
step 7307/19560 | loss 3.512468 (+0.66z)| norm 0.3199 (+1.04z)| lr 4.36e-04 | 322.91 ms | 52.3% bf16 MFU | 1624120 tok/s
step 7308/19560 | loss 3.477933 (-0.08z)| norm 0.2531 (-0.98z)| lr 4.36e-04 | 323.22 ms | 52.2% bf16 MFU | 1624018 tok/s
step 7309/19560 | loss 3.436656 (-0.96z)| norm 0.3533 (+2.02z)| lr 4.36e-04 | 322.95 ms | 52.3% bf16 MFU | 1623990 tok/s
step 7310/19560 | loss 3.460954 (-0.42z)| norm 0.2813 (-0.14z)| lr 4.36e-04 | 322.46 ms | 52.3% bf16 MFU | 1624086 tok/s
step 7311/19560 | loss 3.587653 (+2.31z)| norm 0.3181 (+0.95z)| lr 4.36e-04 | 322.23 ms | 52.4% bf16 MFU | 1624236 tok/s
step 7312/19560 | loss 3.481982 (+0.03z)| norm 0.2918 (+0.16z)| lr 4.36e-04 | 322.53 ms | 52.3% bf16 MFU | 1624301 tok/s
step 7313/19560 | loss 3.504189 (+0.50z)| norm 0.2693 (-0.51z)| lr 4.36e-04 | 323.17 ms | 52.2% bf16 MFU | 1624203 tok/s
step 7314/19560 | loss 3.470157 (-0.24z)| norm 0.2792 (-0.22z)| lr 4.36e-04 | 322.91 ms | 52.3% bf16 MFU | 1624174 tok/s
step 7315/19560 | loss 3.481248 (+0.01z)| norm 0.2702 (-0.48z)| lr 4.36e-04 | 322.31 ms | 52.4% bf16 MFU | 1624298 tok/s
step 7316/19560 | loss 3.497610 (+0.36z)| norm 0.2814 (-0.14z)| lr 4.36e-04 | 322.95 ms | 52.3% bf16 MFU | 1624254 tok/s
step 7317/19560 | loss 3.474633 (-0.13z)| norm 0.2592 (-0.80z)| lr 4.36e-04 | 322.61 ms | 52.3% bf16 MFU | 1624300 tok/s
step 7318/19560 | loss 3.491229 (+0.24z)| norm 0.2944 (+0.24z)| lr 4.35e-04 | 323.76 ms | 52.1% bf16 MFU | 1624053 tok/s
step 7319/19560 | loss 3.470484 (-0.21z)| norm 0.2536 (-1.39z)| lr 4.35e-04 | 322.81 ms | 52.3% bf16 MFU | 1624058 tok/s
step 7320/19560 | loss 3.472198 (-0.18z)| norm 0.2748 (-0.40z)| lr 4.35e-04 | 323.13 ms | 52.2% bf16 MFU | 1623981 tok/s
step 7321/19560 | loss 3.516837 (+0.87z)| norm 0.2766 (-0.31z)| lr 4.35e-04 | 322.97 ms | 52.3% bf16 MFU | 1623950 tok/s
step 7322/19560 | loss 3.556156 (+1.76z)| norm 0.2719 (-0.52z)| lr 4.35e-04 | 323.06 ms | 52.2% bf16 MFU | 1623895 tok/s
step 7323/19560 | loss 3.568581 (+2.00z)| norm 0.2636 (-0.90z)| lr 4.35e-04 | 322.95 ms | 52.3% bf16 MFU | 1623871 tok/s
step 7324/19560 | loss 3.531396 (+1.13z)| norm 0.2753 (-0.33z)| lr 4.35e-04 | 323.03 ms | 52.2% bf16 MFU | 1623829 tok/s
step 7325/19560 | loss 3.471732 (-0.22z)| norm 0.2767 (-0.25z)| lr 4.35e-04 | 322.74 ms | 52.3% bf16 MFU | 1623863 tok/s
step 7326/19560 | loss 3.434312 (-1.07z)| norm 0.2607 (-1.01z)| lr 4.35e-04 | 322.94 ms | 52.3% bf16 MFU | 1623845 tok/s
step 7327/19560 | loss 3.452204 (-0.66z)| norm 0.2806 (-0.06z)| lr 4.35e-04 | 323.18 ms | 52.2% bf16 MFU | 1623767 tok/s
step 7328/19560 | loss 3.506214 (+0.57z)| norm 0.2841 (+0.12z)| lr 4.35e-04 | 323.09 ms | 52.2% bf16 MFU | 1623714 tok/s
step 7329/19560 | loss 3.459551 (-0.51z)| norm 0.2678 (-0.66z)| lr 4.35e-04 | 322.73 ms | 52.3% bf16 MFU | 1623755 tok/s
step 7330/19560 | loss 3.532681 (+1.17z)| norm 0.2597 (-1.04z)| lr 4.35e-04 | 322.69 ms | 52.3% bf16 MFU | 1623803 tok/s
step 7331/19560 | loss 3.523112 (+0.94z)| norm 0.2789 (-0.12z)| lr 4.35e-04 | 322.47 ms | 52.3% bf16 MFU | 1623906 tok/s
step 7332/19560 | loss 3.457210 (-0.59z)| norm 0.2793 (-0.10z)| lr 4.35e-04 | 322.98 ms | 52.3% bf16 MFU | 1623874 tok/s
step 7333/19560 | loss 3.516031 (+0.76z)| norm 0.2634 (-0.87z)| lr 4.35e-04 | 322.92 ms | 52.3% bf16 MFU | 1623860 tok/s
step 7334/19560 | loss 3.445814 (-0.85z)| norm 0.2585 (-1.09z)| lr 4.35e-04 | 322.75 ms | 52.3% bf16 MFU | 1623889 tok/s
step 7335/19560 | loss 3.497120 (+0.35z)| norm 0.2996 (+0.88z)| lr 4.35e-04 | 322.62 ms | 52.3% bf16 MFU | 1623948 tok/s
step 7336/19560 | loss 3.557849 (+1.76z)| norm 0.3764 (+4.20z)| lr 4.35e-04 | 322.79 ms | 52.3% bf16 MFU | 1623962 tok/s
step 7337/19560 | loss 3.462911 (-0.47z)| norm 0.2996 (+0.79z)| lr 4.35e-04 | 322.73 ms | 52.3% bf16 MFU | 1623992 tok/s
step 7338/19560 | loss 3.481480 (-0.03z)| norm 0.2670 (-0.67z)| lr 4.35e-04 | 322.91 ms | 52.3% bf16 MFU | 1623974 tok/s
step 7339/19560 | loss 3.437548 (-1.06z)| norm 0.3129 (+1.36z)| lr 4.35e-04 | 322.47 ms | 52.3% bf16 MFU | 1624067 tok/s
step 7340/19560 | loss 3.434525 (-1.13z)| norm 0.2469 (-1.55z)| lr 4.35e-04 | 322.76 ms | 52.3% bf16 MFU | 1624082 tok/s
step 7341/19560 | loss 3.470320 (-0.29z)| norm 0.2857 (+0.16z)| lr 4.34e-04 | 322.83 ms | 52.3% bf16 MFU | 1624080 tok/s
step 7342/19560 | loss 3.469817 (-0.33z)| norm 0.2764 (-0.26z)| lr 4.34e-04 | 322.29 ms | 52.4% bf16 MFU | 1624213 tok/s
step 7343/19560 | loss 3.508824 (+0.62z)| norm 0.2771 (-0.23z)| lr 4.34e-04 | 322.73 ms | 52.3% bf16 MFU | 1624229 tok/s
step 7344/19560 | loss 3.487556 (+0.12z)| norm 0.2992 (+0.73z)| lr 4.34e-04 | 322.76 ms | 52.3% bf16 MFU | 1624237 tok/s
step 7345/19560 | loss 3.403887 (-1.94z)| norm 0.2966 (+0.61z)| lr 4.34e-04 | 322.28 ms | 52.4% bf16 MFU | 1624366 tok/s
step 7346/19560 | loss 3.584624 (+2.50z)| norm 0.2755 (-0.32z)| lr 4.34e-04 | 322.96 ms | 52.3% bf16 MFU | 1624316 tok/s
step 7347/19560 | loss 3.467800 (-0.35z)| norm 0.2870 (+0.18z)| lr 4.34e-04 | 322.45 ms | 52.3% bf16 MFU | 1624398 tok/s
step 7348/19560 | loss 3.461369 (-0.51z)| norm 0.2840 (+0.04z)| lr 4.34e-04 | 322.60 ms | 52.3% bf16 MFU | 1624439 tok/s
step 7349/19560 | loss 3.521805 (+0.96z)| norm 0.2585 (-1.08z)| lr 4.34e-04 | 322.76 ms | 52.3% bf16 MFU | 1624436 tok/s
step 7350/19560 | loss 3.489720 (+0.17z)| norm 0.2903 (+0.33z)| lr 4.34e-04 | 322.48 ms | 52.3% bf16 MFU | 1624504 tok/s
step 7351/19560 | loss 3.478588 (-0.12z)| norm 0.2727 (-0.47z)| lr 4.34e-04 | 323.03 ms | 52.2% bf16 MFU | 1624429 tok/s
step 7352/19560 | loss 3.525682 (+1.04z)| norm 0.2616 (-0.95z)| lr 4.34e-04 | 322.90 ms | 52.3% bf16 MFU | 1624392 tok/s
step 7353/19560 | loss 3.474037 (-0.24z)| norm 0.2600 (-1.03z)| lr 4.34e-04 | 322.66 ms | 52.3% bf16 MFU | 1624417 tok/s
step 7354/19560 | loss 3.484130 (+0.02z)| norm 0.2416 (-1.82z)| lr 4.34e-04 | 322.54 ms | 52.3% bf16 MFU | 1624472 tok/s
step 7355/19560 | loss 3.479311 (-0.12z)| norm 0.2805 (-0.10z)| lr 4.34e-04 | 322.72 ms | 52.3% bf16 MFU | 1624477 tok/s
step 7356/19560 | loss 3.546118 (+1.54z)| norm 0.2660 (-0.73z)| lr 4.34e-04 | 322.33 ms | 52.4% bf16 MFU | 1624582 tok/s
step 7357/19560 | loss 3.485685 (+0.01z)| norm 0.2356 (-2.04z)| lr 4.34e-04 | 322.52 ms | 52.3% bf16 MFU | 1624632 tok/s
step 7358/19560 | loss 3.517376 (+0.80z)| norm 0.2825 (+0.00z)| lr 4.34e-04 | 322.81 ms | 52.3% bf16 MFU | 1624608 tok/s
step 7359/19560 | loss 3.450349 (-0.89z)| norm 0.2655 (-0.74z)| lr 4.34e-04 | 322.30 ms | 52.4% bf16 MFU | 1624712 tok/s
step 7360/19560 | loss 3.558950 (+1.81z)| norm 0.2669 (-0.69z)| lr 4.34e-04 | 322.87 ms | 52.3% bf16 MFU | 1624668 tok/s
step 7361/19560 | loss 3.526586 (+0.99z)| norm 0.2991 (+0.72z)| lr 4.34e-04 | 322.61 ms | 52.3% bf16 MFU | 1624691 tok/s
step 7362/19560 | loss 3.454883 (-0.79z)| norm 0.2584 (-1.05z)| lr 4.34e-04 | 322.28 ms | 52.4% bf16 MFU | 1624796 tok/s
step 7363/19560 | loss 3.433077 (-1.31z)| norm 0.2439 (-1.65z)| lr 4.33e-04 | 322.18 ms | 52.4% bf16 MFU | 1624922 tok/s
step 7364/19560 | loss 3.471557 (-0.36z)| norm 0.2658 (-0.69z)| lr 4.33e-04 | 322.68 ms | 52.3% bf16 MFU | 1624914 tok/s
step 7365/19560 | loss 3.502150 (+0.41z)| norm 0.2967 (+0.65z)| lr 4.33e-04 | 322.88 ms | 52.3% bf16 MFU | 1624858 tok/s
step 7366/19560 | loss 3.496036 (+0.25z)| norm 0.2756 (-0.27z)| lr 4.33e-04 | 322.54 ms | 52.3% bf16 MFU | 1624891 tok/s
step 7367/19560 | loss 3.471843 (-0.35z)| norm 0.2697 (-0.51z)| lr 4.33e-04 | 322.78 ms | 52.3% bf16 MFU | 1624862 tok/s
step 7368/19560 | loss 3.462815 (-0.57z)| norm 0.3163 (+1.50z)| lr 4.33e-04 | 322.50 ms | 52.3% bf16 MFU | 1624905 tok/s
step 7369/19560 | loss 3.482403 (-0.08z)| norm 0.2667 (-0.64z)| lr 4.33e-04 | 322.03 ms | 52.4% bf16 MFU | 1625064 tok/s
step 7370/19560 | loss 3.458779 (-0.67z)| norm 0.2443 (-1.58z)| lr 4.33e-04 | 322.64 ms | 52.3% bf16 MFU | 1625061 tok/s
step 7371/19560 | loss 3.498421 (+0.31z)| norm 0.2742 (-0.28z)| lr 4.33e-04 | 322.52 ms | 52.3% bf16 MFU | 1625087 tok/s
step 7372/19560 | loss 3.470583 (-0.38z)| norm 0.2466 (-1.45z)| lr 4.33e-04 | 322.70 ms | 52.3% bf16 MFU | 1625067 tok/s
step 7373/19560 | loss 3.482553 (-0.08z)| norm 0.2644 (-0.68z)| lr 4.33e-04 | 322.90 ms | 52.3% bf16 MFU | 1624997 tok/s
step 7374/19560 | loss 3.540284 (+1.34z)| norm 0.2586 (-0.92z)| lr 4.33e-04 | 322.18 ms | 52.4% bf16 MFU | 1625113 tok/s
step 7375/19560 | loss 3.470260 (-0.40z)| norm 0.2626 (-0.74z)| lr 4.33e-04 | 322.53 ms | 52.3% bf16 MFU | 1625134 tok/s
step 7376/19560 | loss 3.440438 (-1.14z)| norm 0.2653 (-0.62z)| lr 4.33e-04 | 322.66 ms | 52.3% bf16 MFU | 1625123 tok/s
step 7377/19560 | loss 3.410913 (-1.84z)| norm 0.2797 (-0.00z)| lr 4.33e-04 | 322.69 ms | 52.3% bf16 MFU | 1625105 tok/s
step 7378/19560 | loss 3.448683 (-0.89z)| norm 0.2651 (-0.62z)| lr 4.33e-04 | 322.07 ms | 52.4% bf16 MFU | 1625244 tok/s
step 7379/19560 | loss 3.528054 (+1.04z)| norm 0.2774 (-0.09z)| lr 4.33e-04 | 322.27 ms | 52.4% bf16 MFU | 1625324 tok/s
step 7380/19560 | loss 3.459162 (-0.65z)| norm 0.2693 (-0.43z)| lr 4.33e-04 | 322.47 ms | 52.3% bf16 MFU | 1625350 tok/s
step 7381/19560 | loss 3.538990 (+1.30z)| norm 0.2855 (+0.26z)| lr 4.33e-04 | 322.21 ms | 52.4% bf16 MFU | 1625441 tok/s
step 7382/19560 | loss 3.511961 (+0.64z)| norm 0.2561 (-0.99z)| lr 4.33e-04 | 322.54 ms | 52.3% bf16 MFU | 1625445 tok/s
step 7383/19560 | loss 3.484202 (-0.04z)| norm 0.2552 (-1.03z)| lr 4.33e-04 | 322.39 ms | 52.4% bf16 MFU | 1625486 tok/s
step 7384/19560 | loss 3.440481 (-1.09z)| norm 0.2611 (-0.77z)| lr 4.33e-04 | 322.64 ms | 52.3% bf16 MFU | 1625462 tok/s
step 7385/19560 | loss 3.492107 (+0.15z)| norm 0.2650 (-0.61z)| lr 4.32e-04 | 322.53 ms | 52.3% bf16 MFU | 1625465 tok/s
step 7386/19560 | loss 3.477168 (-0.22z)| norm 0.2830 (+0.16z)| lr 4.32e-04 | 322.69 ms | 52.3% bf16 MFU | 1625428 tok/s
step 7387/19560 | loss 3.456424 (-0.73z)| norm 0.2714 (-0.34z)| lr 4.32e-04 | 322.30 ms | 52.4% bf16 MFU | 1625492 tok/s
step 7388/19560 | loss 3.450174 (-0.88z)| norm 0.2599 (-0.83z)| lr 4.32e-04 | 322.64 ms | 52.3% bf16 MFU | 1625468 tok/s
step 7389/19560 | loss 3.445964 (-0.97z)| norm 0.2755 (-0.16z)| lr 4.32e-04 | 322.34 ms | 52.4% bf16 MFU | 1625520 tok/s
step 7390/19560 | loss 3.458999 (-0.65z)| norm 0.2471 (-1.35z)| lr 4.32e-04 | 322.28 ms | 52.4% bf16 MFU | 1625584 tok/s
step 7391/19560 | loss 3.472490 (-0.32z)| norm 0.2771 (-0.07z)| lr 4.32e-04 | 322.74 ms | 52.3% bf16 MFU | 1625529 tok/s
step 7392/19560 | loss 3.446589 (-0.96z)| norm 0.2584 (-0.87z)| lr 4.32e-04 | 322.63 ms | 52.3% bf16 MFU | 1625504 tok/s
step 7393/19560 | loss 3.474637 (-0.26z)| norm 0.2736 (-0.21z)| lr 4.32e-04 | 322.70 ms | 52.3% bf16 MFU | 1625464 tok/s
step 7394/19560 | loss 3.428958 (-1.38z)| norm 0.2745 (-0.18z)| lr 4.32e-04 | 322.31 ms | 52.4% bf16 MFU | 1625523 tok/s
step 7395/19560 | loss 3.503986 (+0.48z)| norm 0.2787 (+0.00z)| lr 4.32e-04 | 322.65 ms | 52.3% bf16 MFU | 1625493 tok/s
step 7396/19560 | loss 3.503420 (+0.46z)| norm 0.2919 (+0.57z)| lr 4.32e-04 | 322.50 ms | 52.3% bf16 MFU | 1625504 tok/s
step 7397/19560 | loss 3.518158 (+0.83z)| norm 0.3065 (+1.19z)| lr 4.32e-04 | 323.03 ms | 52.2% bf16 MFU | 1625380 tok/s
step 7398/19560 | loss 3.569457 (+2.07z)| norm 0.3180 (+1.66z)| lr 4.32e-04 | 322.61 ms | 52.3% bf16 MFU | 1625367 tok/s
step 7399/19560 | loss 3.460880 (-0.60z)| norm 0.3436 (+2.65z)| lr 4.32e-04 | 322.35 ms | 52.4% bf16 MFU | 1625422 tok/s
step 7400/19560 | loss 3.486076 (+0.02z)| norm 0.3094 (+1.23z)| lr 4.32e-04 | 322.49 ms | 52.3% bf16 MFU | 1625439 tok/s
step 7401/19560 | loss 3.468163 (-0.42z)| norm 0.2847 (+0.20z)| lr 4.32e-04 | 322.50 ms | 52.3% bf16 MFU | 1625453 tok/s
step 7402/19560 | loss 3.414362 (-1.71z)| norm 0.2639 (-0.65z)| lr 4.32e-04 | 322.73 ms | 52.3% bf16 MFU | 1625406 tok/s
step 7403/19560 | loss 3.454083 (-0.75z)| norm 0.3133 (+1.37z)| lr 4.32e-04 | 322.38 ms | 52.4% bf16 MFU | 1625452 tok/s
step 7404/19560 | loss 3.457326 (-0.67z)| norm 0.2731 (-0.28z)| lr 4.32e-04 | 322.98 ms | 52.3% bf16 MFU | 1625344 tok/s
step 7405/19560 | loss 3.496888 (+0.29z)| norm 0.3079 (+1.14z)| lr 4.32e-04 | 322.23 ms | 52.4% bf16 MFU | 1625428 tok/s
step 7406/19560 | loss 3.433553 (-1.23z)| norm 0.2974 (+0.70z)| lr 4.32e-04 | 322.54 ms | 52.3% bf16 MFU | 1625432 tok/s
step 7407/19560 | loss 3.544193 (+1.45z)| norm 0.2970 (+0.71z)| lr 4.32e-04 | 322.77 ms | 52.3% bf16 MFU | 1625376 tok/s
step 7408/19560 | loss 3.476407 (-0.19z)| norm 0.2926 (+0.54z)| lr 4.31e-04 | 322.58 ms | 52.3% bf16 MFU | 1625373 tok/s
step 7409/19560 | loss 3.576971 (+2.19z)| norm 0.3125 (+1.36z)| lr 4.31e-04 | 322.30 ms | 52.4% bf16 MFU | 1625439 tok/s
step 7410/19560 | loss 3.470149 (-0.37z)| norm 0.2693 (-0.43z)| lr 4.31e-04 | 322.41 ms | 52.3% bf16 MFU | 1625474 tok/s
step 7411/19560 | loss 3.463865 (-0.53z)| norm 0.2721 (-0.30z)| lr 4.31e-04 | 322.65 ms | 52.3% bf16 MFU | 1625446 tok/s
step 7412/19560 | loss 3.535124 (+1.18z)| norm 0.2771 (-0.09z)| lr 4.31e-04 | 322.36 ms | 52.4% bf16 MFU | 1625493 tok/s
step 7413/19560 | loss 3.512844 (+0.65z)| norm 0.3238 (+1.84z)| lr 4.31e-04 | 322.74 ms | 52.3% bf16 MFU | 1625443 tok/s
step 7414/19560 | loss 3.490559 (+0.10z)| norm 0.3159 (+1.50z)| lr 4.31e-04 | 322.34 ms | 52.4% bf16 MFU | 1625498 tok/s
step 7415/19560 | loss 3.527061 (+0.97z)| norm 0.2655 (-0.58z)| lr 4.31e-04 | 322.85 ms | 52.3% bf16 MFU | 1625419 tok/s
step 7416/19560 | loss 3.439564 (-1.14z)| norm 0.3039 (+0.99z)| lr 4.31e-04 | 322.32 ms | 52.4% bf16 MFU | 1625478 tok/s
step 7417/19560 | loss 3.538588 (+1.23z)| norm 0.2727 (-0.29z)| lr 4.31e-04 | 322.53 ms | 52.3% bf16 MFU | 1625482 tok/s
step 7418/19560 | loss 3.470183 (-0.41z)| norm 0.2937 (+0.57z)| lr 4.31e-04 | 322.73 ms | 52.3% bf16 MFU | 1625434 tok/s
step 7419/19560 | loss 3.441031 (-1.18z)| norm 0.2677 (-0.49z)| lr 4.31e-04 | 322.68 ms | 52.3% bf16 MFU | 1625403 tok/s
step 7420/19560 | loss 3.501100 (+0.43z)| norm 0.2851 (+0.30z)| lr 4.31e-04 | 322.42 ms | 52.3% bf16 MFU | 1625437 tok/s
step 7421/19560 | loss 3.446522 (-1.02z)| norm 0.2660 (-0.56z)| lr 4.31e-04 | 322.61 ms | 52.3% bf16 MFU | 1625423 tok/s
step 7422/19560 | loss 3.454422 (-0.80z)| norm 0.3118 (+1.50z)| lr 4.31e-04 | 322.36 ms | 52.4% bf16 MFU | 1625472 tok/s
step 7423/19560 | loss 3.500783 (+0.42z)| norm 0.2883 (+0.43z)| lr 4.31e-04 | 322.82 ms | 52.3% bf16 MFU | 1625402 tok/s
step 7424/19560 | loss 3.476756 (-0.21z)| norm 0.2843 (+0.24z)| lr 4.31e-04 | 322.81 ms | 52.3% bf16 MFU | 1625338 tok/s
step 7425/19560 | loss 3.565748 (+2.10z)| norm 0.3064 (+1.22z)| lr 4.31e-04 | 322.47 ms | 52.3% bf16 MFU | 1625365 tok/s
step 7426/19560 | loss 3.385681 (-2.53z)| norm 0.2623 (-0.77z)| lr 4.31e-04 | 322.56 ms | 52.3% bf16 MFU | 1625368 tok/s
step 7427/19560 | loss 3.500276 (+0.40z)| norm 0.2733 (-0.28z)| lr 4.31e-04 | 322.52 ms | 52.3% bf16 MFU | 1625378 tok/s
step 7428/19560 | loss 3.483820 (-0.03z)| norm 0.2780 (-0.06z)| lr 4.31e-04 | 323.00 ms | 52.3% bf16 MFU | 1625269 tok/s
step 7429/19560 | loss 3.472501 (-0.32z)| norm 0.2514 (-1.25z)| lr 4.31e-04 | 322.50 ms | 52.3% bf16 MFU | 1625289 tok/s
step 7430/19560 | loss 3.494024 (+0.22z)| norm 0.2652 (-0.63z)| lr 4.30e-04 | 322.72 ms | 52.3% bf16 MFU | 1625254 tok/s
step 7431/19560 | loss 3.427742 (-1.47z)| norm 0.2636 (-0.69z)| lr 4.30e-04 | 322.83 ms | 52.3% bf16 MFU | 1625194 tok/s
step 7432/19560 | loss 3.503459 (+0.47z)| norm 0.2582 (-0.94z)| lr 4.30e-04 | 322.51 ms | 52.3% bf16 MFU | 1625216 tok/s
step 7433/19560 | loss 3.471519 (-0.34z)| norm 0.2579 (-0.93z)| lr 4.30e-04 | 322.85 ms | 52.3% bf16 MFU | 1625152 tok/s
step 7434/19560 | loss 3.468566 (-0.41z)| norm 0.2604 (-0.81z)| lr 4.30e-04 | 322.85 ms | 52.3% bf16 MFU | 1625091 tok/s
step 7435/19560 | loss 3.503077 (+0.48z)| norm 0.2714 (-0.31z)| lr 4.30e-04 | 322.54 ms | 52.3% bf16 MFU | 1625112 tok/s
step 7436/19560 | loss 3.730028 (+5.50z)| norm 1.3569 (+10.98z)| lr 4.30e-04 | 322.34 ms | 52.4% bf16 MFU | 1625181 tok/s
step 7437/19560 | loss 3.486502 (-0.01z)| norm 0.3937 (+1.09z)| lr 4.30e-04 | 322.79 ms | 52.3% bf16 MFU | 1625134 tok/s
step 7438/19560 | loss 3.582431 (+2.11z)| norm 0.3232 (+0.36z)| lr 4.30e-04 | 322.78 ms | 52.3% bf16 MFU | 1625092 tok/s
step 7439/19560 | loss 3.551729 (+1.45z)| norm 0.3391 (+0.52z)| lr 4.30e-04 | 322.89 ms | 52.3% bf16 MFU | 1625026 tok/s
step 7440/19560 | loss 3.403364 (-1.86z)| norm 0.3027 (+0.15z)| lr 4.30e-04 | 322.51 ms | 52.3% bf16 MFU | 1625056 tok/s
step 7441/19560 | loss 3.473424 (-0.29z)| norm 0.3042 (+0.17z)| lr 4.30e-04 | 322.57 ms | 52.3% bf16 MFU | 1625070 tok/s
step 7442/19560 | loss 3.472989 (-0.31z)| norm 0.2957 (+0.08z)| lr 4.30e-04 | 323.14 ms | 52.2% bf16 MFU | 1624940 tok/s
step 7443/19560 | loss 3.514832 (+0.62z)| norm 0.3277 (+0.40z)| lr 4.30e-04 | 322.24 ms | 52.4% bf16 MFU | 1625044 tok/s
step 7444/19560 | loss 3.491895 (+0.11z)| norm 0.3154 (+0.27z)| lr 4.30e-04 | 322.93 ms | 52.3% bf16 MFU | 1624970 tok/s
step 7445/19560 | loss 3.458344 (-0.63z)| norm 0.3003 (+0.11z)| lr 4.30e-04 | 322.29 ms | 52.4% bf16 MFU | 1625059 tok/s
step 7446/19560 | loss 3.520060 (+0.73z)| norm 0.3016 (+0.13z)| lr 4.30e-04 | 322.61 ms | 52.3% bf16 MFU | 1625064 tok/s
step 7447/19560 | loss 3.516748 (+0.65z)| norm 0.3141 (+0.25z)| lr 4.30e-04 | 322.49 ms | 52.3% bf16 MFU | 1625098 tok/s
step 7448/19560 | loss 3.467539 (-0.44z)| norm 0.2813 (-0.09z)| lr 4.30e-04 | 322.97 ms | 52.3% bf16 MFU | 1625010 tok/s
step 7449/19560 | loss 3.517983 (+0.68z)| norm 0.2749 (-0.15z)| lr 4.30e-04 | 322.61 ms | 52.3% bf16 MFU | 1625017 tok/s
step 7450/19560 | loss 3.459615 (-0.60z)| norm 0.2684 (-0.22z)| lr 4.30e-04 | 322.26 ms | 52.4% bf16 MFU | 1625112 tok/s
step 7451/19560 | loss 3.464949 (-0.47z)| norm 0.2723 (-0.18z)| lr 4.30e-04 | 322.75 ms | 52.3% bf16 MFU | 1625077 tok/s
step 7452/19560 | loss 3.591568 (+2.34z)| norm 0.2463 (-0.44z)| lr 4.29e-04 | 322.34 ms | 52.4% bf16 MFU | 1625148 tok/s
step 7453/19560 | loss 3.453253 (-0.73z)| norm 0.2840 (-0.06z)| lr 4.29e-04 | 322.57 ms | 52.3% bf16 MFU | 1625158 tok/s
step 7454/19560 | loss 3.494510 (+0.18z)| norm 0.2850 (-0.05z)| lr 4.29e-04 | 322.82 ms | 52.3% bf16 MFU | 1625106 tok/s
step 7455/19560 | loss 3.487755 (+0.02z)| norm 0.2785 (-0.12z)| lr 4.29e-04 | 323.03 ms | 52.2% bf16 MFU | 1625002 tok/s
step 7456/19560 | loss 3.513123 (+0.59z)| norm 0.2761 (-0.14z)| lr 4.29e-04 | 322.87 ms | 52.3% bf16 MFU | 1624943 tok/s
step 7457/19560 | loss 3.558356 (+1.56z)| norm 0.2739 (-0.16z)| lr 4.29e-04 | 322.85 ms | 52.3% bf16 MFU | 1624894 tok/s
step 7458/19560 | loss 3.485378 (-0.04z)| norm 0.2548 (-0.36z)| lr 4.29e-04 | 322.74 ms | 52.3% bf16 MFU | 1624873 tok/s
step 7459/19560 | loss 3.405048 (-1.79z)| norm 0.2647 (-0.26z)| lr 4.29e-04 | 322.53 ms | 52.3% bf16 MFU | 1624907 tok/s
step 7460/19560 | loss 3.463202 (-0.51z)| norm 0.2776 (-0.12z)| lr 4.29e-04 | 322.66 ms | 52.3% bf16 MFU | 1624906 tok/s
step 7461/19560 | loss 3.526364 (+0.88z)| norm 0.2571 (-0.33z)| lr 4.29e-04 | 322.71 ms | 52.3% bf16 MFU | 1624893 tok/s
step 7462/19560 | loss 3.487513 (+0.02z)| norm 0.2691 (-0.21z)| lr 4.29e-04 | 322.77 ms | 52.3% bf16 MFU | 1624865 tok/s
step 7463/19560 | loss 3.494219 (+0.16z)| norm 0.2507 (-0.39z)| lr 4.29e-04 | 322.94 ms | 52.3% bf16 MFU | 1624797 tok/s
step 7464/19560 | loss 3.411032 (-1.64z)| norm 0.2619 (-0.27z)| lr 4.29e-04 | 322.72 ms | 52.3% bf16 MFU | 1624787 tok/s
step 7465/19560 | loss 3.505459 (+0.43z)| norm 1.7556 (+8.99z)| lr 4.29e-04 | 322.64 ms | 52.3% bf16 MFU | 1624798 tok/s
step 7466/19560 | loss 3.468656 (-0.38z)| norm 0.3323 (+0.20z)| lr 4.29e-04 | 322.56 ms | 52.3% bf16 MFU | 1624828 tok/s
step 7467/19560 | loss 3.467438 (-0.41z)| norm 0.3020 (+0.01z)| lr 4.29e-04 | 322.83 ms | 52.3% bf16 MFU | 1624790 tok/s
step 7468/19560 | loss 3.473934 (-0.28z)| norm 0.3157 (+0.09z)| lr 4.29e-04 | 323.00 ms | 52.3% bf16 MFU | 1624710 tok/s
step 7469/19560 | loss 3.572142 (+1.86z)| norm 0.3146 (+0.08z)| lr 4.29e-04 | 323.11 ms | 52.2% bf16 MFU | 1624607 tok/s
step 7470/19560 | loss 3.467366 (-0.43z)| norm 0.2809 (-0.12z)| lr 4.29e-04 | 322.52 ms | 52.3% bf16 MFU | 1624658 tok/s
step 7471/19560 | loss 3.525319 (+0.83z)| norm 0.3189 (+0.11z)| lr 4.29e-04 | 322.76 ms | 52.3% bf16 MFU | 1624644 tok/s
step 7472/19560 | loss 3.453495 (-0.73z)| norm 0.2458 (-0.34z)| lr 4.29e-04 | 323.22 ms | 52.2% bf16 MFU | 1624515 tok/s
step 7473/19560 | loss 3.503307 (+0.34z)| norm 0.2993 (-0.01z)| lr 4.29e-04 | 322.87 ms | 52.3% bf16 MFU | 1624480 tok/s
step 7474/19560 | loss 3.441595 (-1.01z)| norm 0.2806 (-0.13z)| lr 4.28e-04 | 322.70 ms | 52.3% bf16 MFU | 1624491 tok/s
step 7475/19560 | loss 3.468525 (-0.41z)| norm 0.2608 (-0.25z)| lr 4.28e-04 | 322.71 ms | 52.3% bf16 MFU | 1624499 tok/s
step 7476/19560 | loss 3.467577 (-0.43z)| norm 0.2668 (-0.21z)| lr 4.28e-04 | 323.25 ms | 52.2% bf16 MFU | 1624369 tok/s
step 7477/19560 | loss 3.503539 (+0.38z)| norm 0.2518 (-0.30z)| lr 4.28e-04 | 322.33 ms | 52.4% bf16 MFU | 1624480 tok/s
step 7478/19560 | loss 3.461555 (-0.56z)| norm 0.2599 (-0.25z)| lr 4.28e-04 | 322.93 ms | 52.3% bf16 MFU | 1624432 tok/s
step 7479/19560 | loss 3.501419 (+0.33z)| norm 0.2670 (-0.21z)| lr 4.28e-04 | 322.42 ms | 52.3% bf16 MFU | 1624516 tok/s
step 7480/19560 | loss 3.432065 (-1.20z)| norm 0.2679 (-0.20z)| lr 4.28e-04 | 322.97 ms | 52.3% bf16 MFU | 1624456 tok/s
step 7481/19560 | loss 3.446584 (-0.87z)| norm 0.3044 (+0.02z)| lr 4.28e-04 | 323.47 ms | 52.2% bf16 MFU | 1624275 tok/s
step 7482/19560 | loss 3.480419 (-0.12z)| norm 0.3006 (-0.00z)| lr 4.28e-04 | 322.59 ms | 52.3% bf16 MFU | 1624325 tok/s
step 7483/19560 | loss 3.411740 (-1.62z)| norm 0.2779 (-0.14z)| lr 4.28e-04 | 323.08 ms | 52.2% bf16 MFU | 1624246 tok/s
step 7484/19560 | loss 3.541507 (+1.25z)| norm 0.2977 (-0.02z)| lr 4.28e-04 | 322.29 ms | 52.4% bf16 MFU | 1624371 tok/s
step 7485/19560 | loss 3.439955 (-0.99z)| norm 0.2607 (-0.25z)| lr 4.28e-04 | 323.05 ms | 52.2% bf16 MFU | 1624298 tok/s
step 7486/19560 | loss 3.455677 (-0.63z)| norm 0.2978 (-0.02z)| lr 4.28e-04 | 323.49 ms | 52.2% bf16 MFU | 1624120 tok/s
step 7487/19560 | loss 3.465180 (-0.42z)| norm 0.2963 (-0.04z)| lr 4.28e-04 | 322.79 ms | 52.3% bf16 MFU | 1624125 tok/s
step 7488/19560 | loss 3.459656 (-0.53z)| norm 0.2641 (-0.23z)| lr 4.28e-04 | 322.53 ms | 52.3% bf16 MFU | 1624196 tok/s
step 7489/19560 | loss 3.526182 (+0.95z)| norm 0.2957 (-0.04z)| lr 4.28e-04 | 323.52 ms | 52.2% bf16 MFU | 1624014 tok/s
step 7490/19560 | loss 3.398221 (-1.87z)| norm 0.2779 (-0.15z)| lr 4.28e-04 | 322.99 ms | 52.3% bf16 MFU | 1623976 tok/s
step 7491/19560 | loss 3.516058 (+0.71z)| norm 0.2572 (-0.28z)| lr 4.28e-04 | 322.73 ms | 52.3% bf16 MFU | 1624005 tok/s
step 7492/19560 | loss 3.466509 (-0.38z)| norm 0.2781 (-0.15z)| lr 4.28e-04 | 322.81 ms | 52.3% bf16 MFU | 1624011 tok/s
step 7493/19560 | loss 3.513237 (+0.65z)| norm 0.2550 (-0.29z)| lr 4.28e-04 | 322.83 ms | 52.3% bf16 MFU | 1624013 tok/s
step 7494/19560 | loss 3.472831 (-0.24z)| norm 0.2761 (-0.16z)| lr 4.28e-04 | 322.75 ms | 52.3% bf16 MFU | 1624033 tok/s
step 7495/19560 | loss 3.444532 (-0.86z)| norm 0.2692 (-0.20z)| lr 4.28e-04 | 322.49 ms | 52.3% bf16 MFU | 1624120 tok/s
step 7496/19560 | loss 3.424088 (-1.29z)| norm 0.2539 (-0.29z)| lr 4.27e-04 | 323.14 ms | 52.2% bf16 MFU | 1624038 tok/s
step 7497/19560 | loss 3.452343 (-0.67z)| norm 0.2864 (-0.09z)| lr 4.27e-04 | 323.20 ms | 52.2% bf16 MFU | 1623944 tok/s
step 7498/19560 | loss 3.469438 (-0.30z)| norm 0.2612 (-0.25z)| lr 4.27e-04 | 322.89 ms | 52.3% bf16 MFU | 1623933 tok/s
step 7499/19560 | loss 3.416064 (-1.44z)| norm 0.2453 (-0.35z)| lr 4.27e-04 | 322.74 ms | 52.3% bf16 MFU | 1623960 tok/s
step 7500/19560 | loss 3.509397 (+0.58z)| norm 0.2625 (-0.24z)| lr 4.27e-04 | 322.76 ms | 52.3% bf16 MFU | 1623981 tok/s
val loss 3.460540
 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Helevaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2818/10042 = 0.280621
step 7501/19560 | loss 3.488135 (+0.12z)| norm 0.2576 (-0.27z)| lr 4.27e-04 | 321.61 ms | 52.5% bf16 MFU | 1624291 tok/s
step 7502/19560 | loss 3.475910 (-0.14z)| norm 0.2541 (-0.29z)| lr 4.27e-04 | 322.72 ms | 52.3% bf16 MFU | 1624306 tok/s
step 7503/19560 | loss 3.495042 (+0.28z)| norm 0.2683 (-0.21z)| lr 4.27e-04 | 323.15 ms | 52.2% bf16 MFU | 1624213 tok/s
step 7504/19560 | loss 3.504112 (+0.46z)| norm 0.2820 (-0.12z)| lr 4.27e-04 | 322.48 ms | 52.3% bf16 MFU | 1624292 tok/s
step 7505/19560 | loss 3.503468 (+0.44z)| norm 0.2822 (-0.12z)| lr 4.27e-04 | 322.15 ms | 52.4% bf16 MFU | 1624451 tok/s
step 7506/19560 | loss 3.452683 (-0.68z)| norm 0.2665 (-0.22z)| lr 4.27e-04 | 322.65 ms | 52.3% bf16 MFU | 1624476 tok/s
step 7507/19560 | loss 3.515402 (+0.70z)| norm 0.3000 (-0.01z)| lr 4.27e-04 | 322.73 ms | 52.3% bf16 MFU | 1624480 tok/s
step 7508/19560 | loss 3.452843 (-0.68z)| norm 0.2670 (-0.22z)| lr 4.27e-04 | 322.70 ms | 52.3% bf16 MFU | 1624491 tok/s
step 7509/19560 | loss 3.481057 (-0.04z)| norm 0.2867 (-0.09z)| lr 4.27e-04 | 323.13 ms | 52.2% bf16 MFU | 1624393 tok/s
step 7510/19560 | loss 3.541069 (+1.28z)| norm 0.2431 (-0.36z)| lr 4.27e-04 | 323.22 ms | 52.2% bf16 MFU | 1624278 tok/s
step 7511/19560 | loss 3.474930 (-0.18z)| norm 0.3069 (+0.03z)| lr 4.27e-04 | 322.57 ms | 52.3% bf16 MFU | 1624331 tok/s
step 7512/19560 | loss 3.427350 (-1.23z)| norm 0.2569 (-0.28z)| lr 4.27e-04 | 322.53 ms | 52.3% bf16 MFU | 1624392 tok/s
step 7513/19560 | loss 3.449711 (-0.73z)| norm 0.2866 (-0.10z)| lr 4.27e-04 | 323.08 ms | 52.2% bf16 MFU | 1624312 tok/s
step 7514/19560 | loss 3.461193 (-0.47z)| norm 0.3290 (+0.16z)| lr 4.27e-04 | 323.08 ms | 52.2% bf16 MFU | 1624234 tok/s
step 7515/19560 | loss 3.466264 (-0.36z)| norm 0.2907 (-0.08z)| lr 4.27e-04 | 322.86 ms | 52.3% bf16 MFU | 1624218 tok/s
step 7516/19560 | loss 3.491520 (+0.19z)| norm 0.2603 (-0.26z)| lr 4.27e-04 | 323.04 ms | 52.2% bf16 MFU | 1624157 tok/s
step 7517/19560 | loss 3.439269 (-0.96z)| norm 0.2880 (-0.09z)| lr 4.27e-04 | 322.74 ms | 52.3% bf16 MFU | 1624173 tok/s
step 7518/19560 | loss 3.524270 (+0.90z)| norm 0.2886 (-0.09z)| lr 4.26e-04 | 322.76 ms | 52.3% bf16 MFU | 1624185 tok/s
step 7519/19560 | loss 3.524688 (+0.89z)| norm 0.2929 (-0.07z)| lr 4.26e-04 | 322.44 ms | 52.3% bf16 MFU | 1624274 tok/s
step 7520/19560 | loss 3.545145 (+1.32z)| norm 0.2676 (-0.22z)| lr 4.26e-04 | 322.89 ms | 52.3% bf16 MFU | 1624248 tok/s
step 7521/19560 | loss 3.506743 (+0.48z)| norm 0.2667 (-0.23z)| lr 4.26e-04 | 322.58 ms | 52.3% bf16 MFU | 1624300 tok/s
step 7522/19560 | loss 3.471605 (-0.30z)| norm 0.2770 (-0.16z)| lr 4.26e-04 | 322.60 ms | 52.3% bf16 MFU | 1624345 tok/s
step 7523/19560 | loss 3.519191 (+0.74z)| norm 0.2679 (-0.22z)| lr 4.26e-04 | 322.71 ms | 52.3% bf16 MFU | 1624361 tok/s
step 7524/19560 | loss 3.500424 (+0.33z)| norm 0.2538 (-0.30z)| lr 4.26e-04 | 322.83 ms | 52.3% bf16 MFU | 1624344 tok/s
step 7525/19560 | loss 3.479339 (-0.12z)| norm 0.2730 (-0.18z)| lr 4.26e-04 | 323.16 ms | 52.2% bf16 MFU | 1624247 tok/s
step 7526/19560 | loss 3.482734 (-0.04z)| norm 0.2631 (-0.24z)| lr 4.26e-04 | 322.86 ms | 52.3% bf16 MFU | 1624230 tok/s
step 7527/19560 | loss 3.420710 (-1.40z)| norm 0.2395 (-0.38z)| lr 4.26e-04 | 322.27 ms | 52.4% bf16 MFU | 1624360 tok/s
step 7528/19560 | loss 3.470795 (-0.29z)| norm 0.2597 (-0.26z)| lr 4.26e-04 | 322.85 ms | 52.3% bf16 MFU | 1624338 tok/s
step 7529/19560 | loss 3.463962 (-0.44z)| norm 0.2525 (-0.30z)| lr 4.26e-04 | 322.37 ms | 52.4% bf16 MFU | 1624438 tok/s
step 7530/19560 | loss 3.473017 (-0.25z)| norm 0.2632 (-0.23z)| lr 4.26e-04 | 323.22 ms | 52.2% bf16 MFU | 1624320 tok/s
step 7531/19560 | loss 3.444314 (-0.89z)| norm 0.2598 (-0.25z)| lr 4.26e-04 | 322.19 ms | 52.4% bf16 MFU | 1624467 tok/s
step 7532/19560 | loss 3.463984 (-0.45z)| norm 0.2537 (-0.29z)| lr 4.26e-04 | 322.53 ms | 52.3% bf16 MFU | 1624521 tok/s
step 7533/19560 | loss 3.486531 (+0.05z)| norm 0.2691 (-0.19z)| lr 4.26e-04 | 322.42 ms | 52.3% bf16 MFU | 1624600 tok/s
step 7534/19560 | loss 3.478544 (-0.13z)| norm 0.2743 (-0.16z)| lr 4.26e-04 | 322.51 ms | 52.3% bf16 MFU | 1624652 tok/s
step 7535/19560 | loss 3.485429 (+0.03z)| norm 0.2543 (-0.28z)| lr 4.26e-04 | 322.64 ms | 52.3% bf16 MFU | 1624669 tok/s
step 7536/19560 | loss 3.509858 (+0.57z)| norm 0.2771 (-0.14z)| lr 4.26e-04 | 323.47 ms | 52.2% bf16 MFU | 1624476 tok/s
step 7537/19560 | loss 3.499628 (+0.36z)| norm 0.2558 (-0.27z)| lr 4.26e-04 | 322.70 ms | 52.3% bf16 MFU | 1624486 tok/s
step 7538/19560 | loss 3.449165 (-0.79z)| norm 0.2571 (-0.26z)| lr 4.26e-04 | 322.65 ms | 52.3% bf16 MFU | 1624510 tok/s
step 7539/19560 | loss 3.486773 (+0.07z)| norm 0.2650 (-0.21z)| lr 4.26e-04 | 322.44 ms | 52.3% bf16 MFU | 1624585 tok/s
step 7540/19560 | loss 3.494390 (+0.25z)| norm 0.2698 (-0.18z)| lr 4.25e-04 | 322.59 ms | 52.3% bf16 MFU | 1624618 tok/s
step 7541/19560 | loss 3.515757 (+0.74z)| norm 0.2606 (-0.23z)| lr 4.25e-04 | 322.66 ms | 52.3% bf16 MFU | 1624630 tok/s
step 7542/19560 | loss 3.498458 (+0.34z)| norm 0.2788 (-0.12z)| lr 4.25e-04 | 322.15 ms | 52.4% bf16 MFU | 1624773 tok/s
step 7543/19560 | loss 3.639514 (+3.41z)| norm 0.3084 (+0.06z)| lr 4.25e-04 | 323.49 ms | 52.2% bf16 MFU | 1624570 tok/s
step 7544/19560 | loss 3.520510 (+0.78z)| norm 0.2802 (-0.11z)| lr 4.25e-04 | 322.37 ms | 52.4% bf16 MFU | 1624659 tok/s
step 7545/19560 | loss 3.498022 (+0.29z)| norm 0.2737 (-0.15z)| lr 4.25e-04 | 323.51 ms | 52.2% bf16 MFU | 1624456 tok/s
step 7546/19560 | loss 3.423497 (-1.34z)| norm 0.2472 (-0.31z)| lr 4.25e-04 | 322.30 ms | 52.4% bf16 MFU | 1624569 tok/s
step 7547/19560 | loss 3.470941 (-0.30z)| norm 0.2864 (-0.07z)| lr 4.25e-04 | 322.55 ms | 52.3% bf16 MFU | 1624612 tok/s
step 7548/19560 | loss 3.454324 (-0.66z)| norm 0.2995 (+0.01z)| lr 4.25e-04 | 322.81 ms | 52.3% bf16 MFU | 1624589 tok/s
step 7549/19560 | loss 3.509896 (+0.55z)| norm 0.2666 (-0.19z)| lr 4.25e-04 | 323.17 ms | 52.2% bf16 MFU | 1624476 tok/s
step 7550/19560 | loss 3.388880 (-2.07z)| norm 0.2948 (-0.02z)| lr 4.25e-04 | 323.06 ms | 52.2% bf16 MFU | 1624396 tok/s
step 7551/19560 | loss 3.493275 (+0.20z)| norm 0.2912 (-0.04z)| lr 4.25e-04 | 323.44 ms | 52.2% bf16 MFU | 1624224 tok/s
step 7552/19560 | loss 3.501743 (+0.38z)| norm 0.2899 (-0.05z)| lr 4.25e-04 | 321.97 ms | 52.4% bf16 MFU | 1624432 tok/s
step 7553/19560 | loss 3.487569 (+0.08z)| norm 0.3142 (+0.10z)| lr 4.25e-04 | 323.03 ms | 52.2% bf16 MFU | 1624362 tok/s
step 7554/19560 | loss 3.496408 (+0.26z)| norm 0.2929 (-0.03z)| lr 4.25e-04 | 323.02 ms | 52.2% bf16 MFU | 1624298 tok/s
step 7555/19560 | loss 3.491627 (+0.16z)| norm 0.2835 (-0.09z)| lr 4.25e-04 | 323.19 ms | 52.2% bf16 MFU | 1624195 tok/s
step 7556/19560 | loss 3.501615 (+0.38z)| norm 0.2760 (-0.14z)| lr 4.25e-04 | 322.60 ms | 52.3% bf16 MFU | 1624246 tok/s
step 7557/19560 | loss 3.448429 (-0.81z)| norm 0.2829 (-0.10z)| lr 4.25e-04 | 322.55 ms | 52.3% bf16 MFU | 1624305 tok/s
step 7558/19560 | loss 3.425333 (-1.30z)| norm 0.2765 (-0.14z)| lr 4.25e-04 | 322.93 ms | 52.3% bf16 MFU | 1624267 tok/s
step 7559/19560 | loss 3.505175 (+0.46z)| norm 0.2680 (-0.19z)| lr 4.25e-04 | 322.80 ms | 52.3% bf16 MFU | 1624263 tok/s
step 7560/19560 | loss 3.494768 (+0.23z)| norm 0.2835 (-0.09z)| lr 4.25e-04 | 322.29 ms | 52.4% bf16 MFU | 1624388 tok/s
step 7561/19560 | loss 3.443092 (-0.92z)| norm 0.2821 (-0.11z)| lr 4.25e-04 | 322.71 ms | 52.3% bf16 MFU | 1624402 tok/s
step 7562/19560 | loss 3.475733 (-0.19z)| norm 0.2655 (-0.21z)| lr 4.24e-04 | 322.80 ms | 52.3% bf16 MFU | 1624391 tok/s
step 7563/19560 | loss 3.458675 (-0.56z)| norm 0.2660 (-0.20z)| lr 4.24e-04 | 322.50 ms | 52.3% bf16 MFU | 1624457 tok/s
step 7564/19560 | loss 3.525988 (+1.10z)| norm 0.2708 (-0.15z)| lr 4.24e-04 | 322.84 ms | 52.3% bf16 MFU | 1624433 tok/s
step 7565/19560 | loss 3.457080 (-0.63z)| norm 0.2624 (-0.21z)| lr 4.24e-04 | 322.21 ms | 52.4% bf16 MFU | 1624570 tok/s
step 7566/19560 | loss 3.446198 (-0.90z)| norm 0.2810 (-0.06z)| lr 4.24e-04 | 323.53 ms | 52.2% bf16 MFU | 1624367 tok/s
step 7567/19560 | loss 3.410817 (-1.79z)| norm 0.2885 (-0.00z)| lr 4.24e-04 | 322.73 ms | 52.3% bf16 MFU | 1624377 tok/s
step 7568/19560 | loss 3.425370 (-1.43z)| norm 0.2444 (-0.33z)| lr 4.24e-04 | 322.67 ms | 52.3% bf16 MFU | 1624400 tok/s
step 7569/19560 | loss 3.452495 (-0.72z)| norm 0.2568 (-0.24z)| lr 4.24e-04 | 322.99 ms | 52.3% bf16 MFU | 1624343 tok/s
step 7570/19560 | loss 3.464442 (-0.40z)| norm 0.2589 (-0.22z)| lr 4.24e-04 | 322.37 ms | 52.4% bf16 MFU | 1624444 tok/s
step 7571/19560 | loss 3.488703 (+0.23z)| norm 0.2883 (+0.01z)| lr 4.24e-04 | 323.01 ms | 52.3% bf16 MFU | 1624379 tok/s
step 7572/19560 | loss 3.439988 (-1.02z)| norm 0.2772 (-0.08z)| lr 4.24e-04 | 323.55 ms | 52.2% bf16 MFU | 1624182 tok/s
step 7573/19560 | loss 3.495163 (+0.40z)| norm 0.2573 (-0.22z)| lr 4.24e-04 | 322.52 ms | 52.3% bf16 MFU | 1624254 tok/s
step 7574/19560 | loss 3.500288 (+0.54z)| norm 0.2692 (-0.13z)| lr 4.24e-04 | 322.68 ms | 52.3% bf16 MFU | 1624281 tok/s
step 7575/19560 | loss 3.481999 (+0.07z)| norm 0.2757 (-0.08z)| lr 4.24e-04 | 322.14 ms | 52.4% bf16 MFU | 1624442 tok/s
step 7576/19560 | loss 3.416209 (-1.62z)| norm 0.2480 (-0.29z)| lr 4.24e-04 | 322.95 ms | 52.3% bf16 MFU | 1624392 tok/s
step 7577/19560 | loss 3.493524 (+0.39z)| norm 0.2898 (+0.03z)| lr 4.24e-04 | 322.60 ms | 52.3% bf16 MFU | 1624431 tok/s
step 7578/19560 | loss 3.490751 (+0.31z)| norm 0.2738 (-0.09z)| lr 4.24e-04 | 322.63 ms | 52.3% bf16 MFU | 1624461 tok/s
step 7579/19560 | loss 3.461673 (-0.45z)| norm 0.2857 (-0.00z)| lr 4.24e-04 | 322.45 ms | 52.3% bf16 MFU | 1624536 tok/s
step 7580/19560 | loss 3.443441 (-0.92z)| norm 0.2812 (-0.04z)| lr 4.24e-04 | 323.11 ms | 52.2% bf16 MFU | 1624440 tok/s
step 7581/19560 | loss 3.419132 (-1.55z)| norm 0.2868 (+0.00z)| lr 4.24e-04 | 322.94 ms | 52.3% bf16 MFU | 1624393 tok/s
step 7582/19560 | loss 3.461384 (-0.42z)| norm 0.2743 (-0.09z)| lr 4.24e-04 | 322.74 ms | 52.3% bf16 MFU | 1624399 tok/s
step 7583/19560 | loss 3.461581 (-0.41z)| norm 0.2703 (-0.12z)| lr 4.24e-04 | 322.41 ms | 52.3% bf16 MFU | 1624486 tok/s
step 7584/19560 | loss 3.470994 (-0.15z)| norm 0.2773 (-0.07z)| lr 4.23e-04 | 322.71 ms | 52.3% bf16 MFU | 1624494 tok/s
step 7585/19560 | loss 3.508095 (+0.87z)| norm 0.2816 (-0.04z)| lr 4.23e-04 | 322.65 ms | 52.3% bf16 MFU | 1624517 tok/s
step 7586/19560 | loss 3.490369 (+0.38z)| norm 0.2728 (-0.10z)| lr 4.23e-04 | 322.64 ms | 52.3% bf16 MFU | 1624542 tok/s
step 7587/19560 | loss 3.527858 (+1.39z)| norm 0.2885 (+0.01z)| lr 4.23e-04 | 322.59 ms | 52.3% bf16 MFU | 1624576 tok/s
step 7588/19560 | loss 3.451343 (-0.70z)| norm 0.3214 (+0.26z)| lr 4.23e-04 | 322.79 ms | 52.3% bf16 MFU | 1624559 tok/s
step 7589/19560 | loss 3.498439 (+0.59z)| norm 0.2838 (-0.03z)| lr 4.23e-04 | 322.63 ms | 52.3% bf16 MFU | 1624582 tok/s
step 7590/19560 | loss 3.448795 (-0.76z)| norm 0.2806 (-0.05z)| lr 4.23e-04 | 323.11 ms | 52.2% bf16 MFU | 1624485 tok/s
step 7591/19560 | loss 3.515235 (+1.05z)| norm 0.3098 (+0.17z)| lr 4.23e-04 | 322.51 ms | 52.3% bf16 MFU | 1624542 tok/s
step 7592/19560 | loss 3.506035 (+0.79z)| norm 0.2952 (+0.05z)| lr 4.23e-04 | 322.47 ms | 52.3% bf16 MFU | 1624607 tok/s
step 7593/19560 | loss 3.485794 (+0.23z)| norm 0.3026 (+1.36z)| lr 4.23e-04 | 323.18 ms | 52.2% bf16 MFU | 1624490 tok/s
step 7594/19560 | loss 3.445050 (-0.89z)| norm 0.2940 (+0.95z)| lr 4.23e-04 | 322.47 ms | 52.3% bf16 MFU | 1624559 tok/s
step 7595/19560 | loss 3.425329 (-1.41z)| norm 0.2935 (+0.93z)| lr 4.23e-04 | 322.39 ms | 52.4% bf16 MFU | 1624643 tok/s
step 7596/19560 | loss 3.473894 (-0.08z)| norm 0.3152 (+2.13z)| lr 4.23e-04 | 323.16 ms | 52.2% bf16 MFU | 1624531 tok/s
step 7597/19560 | loss 3.498942 (+0.64z)| norm 0.2872 (+0.61z)| lr 4.23e-04 | 322.61 ms | 52.3% bf16 MFU | 1624562 tok/s
step 7598/19560 | loss 3.463452 (-0.36z)| norm 0.2992 (+1.27z)| lr 4.23e-04 | 322.40 ms | 52.3% bf16 MFU | 1624644 tok/s
step 7599/19560 | loss 3.555327 (+2.19z)| norm 0.2943 (+1.02z)| lr 4.23e-04 | 322.92 ms | 52.3% bf16 MFU | 1624592 tok/s
step 7600/19560 | loss 3.483146 (+0.18z)| norm 0.2791 (+0.15z)| lr 4.23e-04 | 322.61 ms | 52.3% bf16 MFU | 1624618 tok/s
step 7601/19560 | loss 3.548363 (+1.96z)| norm 0.2984 (+1.26z)| lr 4.23e-04 | 322.92 ms | 52.3% bf16 MFU | 1624568 tok/s
step 7602/19560 | loss 3.501449 (+0.66z)| norm 0.3434 (+3.60z)| lr 4.23e-04 | 322.62 ms | 52.3% bf16 MFU | 1624594 tok/s
step 7603/19560 | loss 3.409552 (-1.83z)| norm 0.3155 (+2.04z)| lr 4.23e-04 | 322.27 ms | 52.4% bf16 MFU | 1624708 tok/s
step 7604/19560 | loss 3.474319 (-0.08z)| norm 0.2874 (+0.53z)| lr 4.23e-04 | 323.01 ms | 52.2% bf16 MFU | 1624629 tok/s
step 7605/19560 | loss 3.447690 (-0.79z)| norm 0.2962 (+0.98z)| lr 4.23e-04 | 322.72 ms | 52.3% bf16 MFU | 1624626 tok/s
step 7606/19560 | loss 3.443444 (-0.90z)| norm 0.2823 (+0.23z)| lr 4.22e-04 | 322.32 ms | 52.4% bf16 MFU | 1624724 tok/s
step 7607/19560 | loss 3.411807 (-1.72z)| norm 0.2842 (+0.32z)| lr 4.22e-04 | 322.96 ms | 52.3% bf16 MFU | 1624657 tok/s
step 7608/19560 | loss 3.475864 (-0.01z)| norm 0.2906 (+0.66z)| lr 4.22e-04 | 322.22 ms | 52.4% bf16 MFU | 1624779 tok/s
step 7609/19560 | loss 3.487842 (+0.31z)| norm 0.2844 (+0.34z)| lr 4.22e-04 | 322.96 ms | 52.3% bf16 MFU | 1624709 tok/s
step 7610/19560 | loss 3.515569 (+1.04z)| norm 0.2878 (+0.53z)| lr 4.22e-04 | 322.32 ms | 52.4% bf16 MFU | 1624805 tok/s
step 7611/19560 | loss 3.511553 (+0.92z)| norm 0.2659 (-0.66z)| lr 4.22e-04 | 322.76 ms | 52.3% bf16 MFU | 1624784 tok/s
step 7612/19560 | loss 3.455641 (-0.58z)| norm 0.2594 (-1.00z)| lr 4.22e-04 | 322.60 ms | 52.3% bf16 MFU | 1624804 tok/s
step 7613/19560 | loss 3.469336 (-0.21z)| norm 0.2958 (+0.97z)| lr 4.22e-04 | 322.33 ms | 52.4% bf16 MFU | 1624892 tok/s
step 7614/19560 | loss 3.524805 (+1.29z)| norm 0.2541 (-1.28z)| lr 4.22e-04 | 322.31 ms | 52.4% bf16 MFU | 1624980 tok/s
step 7615/19560 | loss 3.442108 (-0.97z)| norm 0.2803 (+0.15z)| lr 4.22e-04 | 322.47 ms | 52.3% bf16 MFU | 1625024 tok/s
step 7616/19560 | loss 3.483118 (+0.15z)| norm 0.2471 (-1.64z)| lr 4.22e-04 | 322.50 ms | 52.3% bf16 MFU | 1625059 tok/s
step 7617/19560 | loss 3.427937 (-1.34z)| norm 0.2934 (+0.87z)| lr 4.22e-04 | 322.72 ms | 52.3% bf16 MFU | 1625034 tok/s
step 7618/19560 | loss 3.491647 (+0.39z)| norm 0.2653 (-0.64z)| lr 4.22e-04 | 322.60 ms | 52.3% bf16 MFU | 1625041 tok/s
step 7619/19560 | loss 3.513757 (+1.01z)| norm 0.2815 (+0.22z)| lr 4.22e-04 | 322.58 ms | 52.3% bf16 MFU | 1625053 tok/s
step 7620/19560 | loss 3.384749 (-2.51z)| norm 0.3048 (+1.46z)| lr 4.22e-04 | 322.38 ms | 52.4% bf16 MFU | 1625115 tok/s
step 7621/19560 | loss 3.486424 (+0.26z)| norm 0.2868 (+0.48z)| lr 4.22e-04 | 323.07 ms | 52.2% bf16 MFU | 1625000 tok/s
step 7622/19560 | loss 3.589976 (+2.95z)| norm 0.2814 (+0.19z)| lr 4.22e-04 | 322.37 ms | 52.4% bf16 MFU | 1625067 tok/s
step 7623/19560 | loss 3.470289 (-0.20z)| norm 0.3369 (+3.04z)| lr 4.22e-04 | 322.85 ms | 52.3% bf16 MFU | 1625011 tok/s
step 7624/19560 | loss 3.435439 (-1.13z)| norm 0.3013 (+1.17z)| lr 4.22e-04 | 322.98 ms | 52.3% bf16 MFU | 1624926 tok/s
step 7625/19560 | loss 3.450749 (-0.72z)| norm 0.2680 (-0.56z)| lr 4.22e-04 | 322.81 ms | 52.3% bf16 MFU | 1624887 tok/s
step 7626/19560 | loss 3.478642 (+0.02z)| norm 0.2788 (+0.00z)| lr 4.22e-04 | 322.86 ms | 52.3% bf16 MFU | 1624837 tok/s
step 7627/19560 | loss 3.533335 (+1.44z)| norm 0.2979 (+0.98z)| lr 4.22e-04 | 322.25 ms | 52.4% bf16 MFU | 1624944 tok/s
step 7628/19560 | loss 3.530472 (+1.36z)| norm 0.2787 (-0.04z)| lr 4.21e-04 | 323.08 ms | 52.2% bf16 MFU | 1624835 tok/s
step 7629/19560 | loss 3.485505 (+0.17z)| norm 0.2901 (+0.55z)| lr 4.21e-04 | 322.82 ms | 52.3% bf16 MFU | 1624799 tok/s
step 7630/19560 | loss 3.448451 (-0.80z)| norm 0.2503 (-1.55z)| lr 4.21e-04 | 322.56 ms | 52.3% bf16 MFU | 1624830 tok/s
step 7631/19560 | loss 3.464247 (-0.38z)| norm 0.2875 (+0.41z)| lr 4.21e-04 | 322.62 ms | 52.3% bf16 MFU | 1624844 tok/s
step 7632/19560 | loss 3.456968 (-0.56z)| norm 0.2749 (-0.25z)| lr 4.21e-04 | 322.71 ms | 52.3% bf16 MFU | 1624834 tok/s
step 7633/19560 | loss 3.466927 (-0.29z)| norm 0.2760 (-0.19z)| lr 4.21e-04 | 322.86 ms | 52.3% bf16 MFU | 1624788 tok/s
step 7634/19560 | loss 3.425634 (-1.37z)| norm 0.2614 (-0.96z)| lr 4.21e-04 | 323.29 ms | 52.2% bf16 MFU | 1624635 tok/s
step 7635/19560 | loss 3.448934 (-0.74z)| norm 0.3008 (+1.12z)| lr 4.21e-04 | 322.99 ms | 52.3% bf16 MFU | 1624564 tok/s
step 7636/19560 | loss 3.472743 (-0.12z)| norm 0.2455 (-1.78z)| lr 4.21e-04 | 322.38 ms | 52.4% bf16 MFU | 1624651 tok/s
step 7637/19560 | loss 3.434659 (-1.11z)| norm 0.8041 (+10.41z)| lr 4.21e-04 | 322.87 ms | 52.3% bf16 MFU | 1624609 tok/s
step 7638/19560 | loss 3.482837 (+0.17z)| norm 0.2990 (+0.30z)| lr 4.21e-04 | 322.81 ms | 52.3% bf16 MFU | 1624585 tok/s
step 7639/19560 | loss 3.466732 (-0.26z)| norm 0.3029 (+0.38z)| lr 4.21e-04 | 322.86 ms | 52.3% bf16 MFU | 1624549 tok/s
step 7640/19560 | loss 3.479941 (+0.08z)| norm 0.3142 (+0.60z)| lr 4.21e-04 | 322.95 ms | 52.3% bf16 MFU | 1624492 tok/s
step 7641/19560 | loss 3.500957 (+0.63z)| norm 0.2874 (+0.06z)| lr 4.21e-04 | 323.60 ms | 52.2% bf16 MFU | 1624278 tok/s
step 7642/19560 | loss 3.457766 (-0.52z)| norm 0.3069 (+0.46z)| lr 4.21e-04 | 323.34 ms | 52.2% bf16 MFU | 1624138 tok/s
step 7643/19560 | loss 3.467585 (-0.26z)| norm 0.2839 (-0.00z)| lr 4.21e-04 | 322.91 ms | 52.3% bf16 MFU | 1624113 tok/s
step 7644/19560 | loss 3.450986 (-0.69z)| norm 0.3045 (+0.40z)| lr 4.21e-04 | 322.51 ms | 52.3% bf16 MFU | 1624191 tok/s
step 7645/19560 | loss 3.501736 (+0.65z)| norm 0.2710 (-0.27z)| lr 4.21e-04 | 323.28 ms | 52.2% bf16 MFU | 1624069 tok/s
step 7646/19560 | loss 3.476072 (-0.03z)| norm 0.2696 (-0.29z)| lr 4.21e-04 | 322.38 ms | 52.4% bf16 MFU | 1624180 tok/s
step 7647/19560 | loss 3.453546 (-0.62z)| norm 0.2755 (-0.17z)| lr 4.21e-04 | 322.60 ms | 52.3% bf16 MFU | 1624232 tok/s
step 7648/19560 | loss 3.475951 (-0.00z)| norm 0.2666 (-0.35z)| lr 4.21e-04 | 322.59 ms | 52.3% bf16 MFU | 1624283 tok/s
step 7649/19560 | loss 3.558747 (+2.22z)| norm 0.2615 (-0.45z)| lr 4.21e-04 | 322.74 ms | 52.3% bf16 MFU | 1624292 tok/s
step 7650/19560 | loss 3.436996 (-1.05z)| norm 0.2561 (-0.56z)| lr 4.20e-04 | 322.43 ms | 52.3% bf16 MFU | 1624380 tok/s
step 7651/19560 | loss 3.491018 (+0.41z)| norm 0.2677 (-0.32z)| lr 4.20e-04 | 322.76 ms | 52.3% bf16 MFU | 1624382 tok/s
step 7652/19560 | loss 3.401136 (-1.97z)| norm 0.2591 (-0.50z)| lr 4.20e-04 | 322.68 ms | 52.3% bf16 MFU | 1624401 tok/s
step 7653/19560 | loss 3.411398 (-1.66z)| norm 0.2622 (-0.43z)| lr 4.20e-04 | 322.93 ms | 52.3% bf16 MFU | 1624357 tok/s
step 7654/19560 | loss 3.356769 (-2.96z)| norm 0.2518 (-0.64z)| lr 4.20e-04 | 323.10 ms | 52.2% bf16 MFU | 1624274 tok/s
step 7655/19560 | loss 3.532209 (+1.46z)| norm 0.2667 (-0.35z)| lr 4.20e-04 | 323.00 ms | 52.3% bf16 MFU | 1624219 tok/s
step 7656/19560 | loss 3.504997 (+0.76z)| norm 0.2942 (+0.20z)| lr 4.20e-04 | 322.79 ms | 52.3% bf16 MFU | 1624220 tok/s
step 7657/19560 | loss 3.461269 (-0.34z)| norm 0.2775 (-0.14z)| lr 4.20e-04 | 322.71 ms | 52.3% bf16 MFU | 1624241 tok/s
step 7658/19560 | loss 3.533179 (+1.45z)| norm 0.2634 (-0.42z)| lr 4.20e-04 | 323.41 ms | 52.2% bf16 MFU | 1624086 tok/s
step 7659/19560 | loss 3.454147 (-0.53z)| norm 0.2796 (-0.10z)| lr 4.20e-04 | 322.47 ms | 52.3% bf16 MFU | 1624173 tok/s
step 7660/19560 | loss 3.546793 (+1.76z)| norm 0.2588 (-0.52z)| lr 4.20e-04 | 323.10 ms | 52.2% bf16 MFU | 1624098 tok/s
step 7661/19560 | loss 3.392974 (-2.01z)| norm 0.2608 (-0.48z)| lr 4.20e-04 | 322.42 ms | 52.3% bf16 MFU | 1624198 tok/s
step 7662/19560 | loss 3.450683 (-0.59z)| norm 0.2543 (-0.61z)| lr 4.20e-04 | 322.82 ms | 52.3% bf16 MFU | 1624192 tok/s
step 7663/19560 | loss 3.372712 (-2.41z)| norm 0.2624 (-0.44z)| lr 4.20e-04 | 322.60 ms | 52.3% bf16 MFU | 1624242 tok/s
step 7664/19560 | loss 3.519462 (+1.08z)| norm 0.3102 (+0.51z)| lr 4.20e-04 | 322.86 ms | 52.3% bf16 MFU | 1624223 tok/s
step 7665/19560 | loss 3.468284 (-0.13z)| norm 0.2873 (+0.05z)| lr 4.20e-04 | 322.36 ms | 52.4% bf16 MFU | 1624334 tok/s
step 7666/19560 | loss 3.432456 (-0.98z)| norm 0.2735 (-0.23z)| lr 4.20e-04 | 322.86 ms | 52.3% bf16 MFU | 1624311 tok/s
step 7667/19560 | loss 3.478229 (+0.11z)| norm 0.2921 (+0.14z)| lr 4.20e-04 | 322.79 ms | 52.3% bf16 MFU | 1624309 tok/s
step 7668/19560 | loss 3.502335 (+0.68z)| norm 0.2807 (-0.09z)| lr 4.20e-04 | 322.90 ms | 52.3% bf16 MFU | 1624277 tok/s
step 7669/19560 | loss 3.487845 (+0.34z)| norm 0.2991 (+0.27z)| lr 4.20e-04 | 323.08 ms | 52.2% bf16 MFU | 1624203 tok/s
step 7670/19560 | loss 3.481047 (+0.18z)| norm 0.3245 (+0.78z)| lr 4.20e-04 | 322.46 ms | 52.3% bf16 MFU | 1624288 tok/s
step 7671/19560 | loss 3.486811 (+0.37z)| norm 0.2854 (-0.01z)| lr 4.20e-04 | 323.00 ms | 52.3% bf16 MFU | 1624234 tok/s
step 7672/19560 | loss 3.539850 (+1.71z)| norm 0.3055 (+0.39z)| lr 4.19e-04 | 323.04 ms | 52.2% bf16 MFU | 1624172 tok/s
step 7673/19560 | loss 3.458583 (-0.34z)| norm 0.3083 (+0.44z)| lr 4.19e-04 | 322.83 ms | 52.3% bf16 MFU | 1624164 tok/s
step 7674/19560 | loss 3.506612 (+0.86z)| norm 0.2906 (+0.08z)| lr 4.19e-04 | 322.72 ms | 52.3% bf16 MFU | 1624185 tok/s
step 7675/19560 | loss 3.465280 (-0.19z)| norm 0.3015 (+0.30z)| lr 4.19e-04 | 322.65 ms | 52.3% bf16 MFU | 1624224 tok/s
step 7676/19560 | loss 3.495165 (+0.56z)| norm 0.3497 (+1.26z)| lr 4.19e-04 | 322.99 ms | 52.3% bf16 MFU | 1624174 tok/s
step 7677/19560 | loss 3.555973 (+2.07z)| norm 0.3005 (+0.26z)| lr 4.19e-04 | 322.94 ms | 52.3% bf16 MFU | 1624139 tok/s
step 7678/19560 | loss 3.461149 (-0.33z)| norm 0.2894 (+0.04z)| lr 4.19e-04 | 323.13 ms | 52.2% bf16 MFU | 1624057 tok/s
step 7679/19560 | loss 3.455684 (-0.46z)| norm 0.2809 (-0.13z)| lr 4.19e-04 | 323.15 ms | 52.2% bf16 MFU | 1623975 tok/s
step 7680/19560 | loss 3.485776 (+0.31z)| norm 0.3286 (+0.82z)| lr 4.19e-04 | 322.87 ms | 52.3% bf16 MFU | 1623968 tok/s
step 7681/19560 | loss 3.492003 (+0.47z)| norm 0.2584 (-0.57z)| lr 4.19e-04 | 323.21 ms | 52.2% bf16 MFU | 1623877 tok/s
step 7682/19560 | loss 3.438343 (-0.89z)| norm 0.2728 (-0.28z)| lr 4.19e-04 | 323.13 ms | 52.2% bf16 MFU | 1623809 tok/s
step 7683/19560 | loss 3.513401 (+1.02z)| norm 0.2670 (-0.40z)| lr 4.19e-04 | 323.15 ms | 52.2% bf16 MFU | 1623740 tok/s
step 7684/19560 | loss 3.470618 (-0.06z)| norm 0.2607 (-0.52z)| lr 4.19e-04 | 322.99 ms | 52.3% bf16 MFU | 1623714 tok/s
step 7685/19560 | loss 3.481942 (+0.22z)| norm 0.2539 (-0.65z)| lr 4.19e-04 | 323.12 ms | 52.2% bf16 MFU | 1623659 tok/s
step 7686/19560 | loss 3.480189 (+0.17z)| norm 0.2554 (-0.62z)| lr 4.19e-04 | 322.99 ms | 52.3% bf16 MFU | 1623637 tok/s
step 7687/19560 | loss 3.524579 (+1.29z)| norm 0.2723 (-0.28z)| lr 4.19e-04 | 323.38 ms | 52.2% bf16 MFU | 1623518 tok/s
step 7688/19560 | loss 3.312791 (-3.84z)| norm 0.6710 (+6.31z)| lr 4.19e-04 | 322.18 ms | 52.4% bf16 MFU | 1623709 tok/s
step 7689/19560 | loss 3.504685 (+0.76z)| norm 0.3762 (+1.41z)| lr 4.19e-04 | 322.74 ms | 52.3% bf16 MFU | 1623747 tok/s
step 7690/19560 | loss 3.457823 (-0.36z)| norm 0.3410 (+0.82z)| lr 4.19e-04 | 323.00 ms | 52.3% bf16 MFU | 1623718 tok/s
step 7691/19560 | loss 3.489522 (+0.40z)| norm 0.2892 (-0.03z)| lr 4.19e-04 | 322.78 ms | 52.3% bf16 MFU | 1623748 tok/s
step 7692/19560 | loss 3.397562 (-1.78z)| norm 0.3170 (+0.42z)| lr 4.19e-04 | 323.14 ms | 52.2% bf16 MFU | 1623685 tok/s
step 7693/19560 | loss 3.440647 (-0.74z)| norm 0.2938 (+0.04z)| lr 4.19e-04 | 322.70 ms | 52.3% bf16 MFU | 1623736 tok/s
step 7694/19560 | loss 3.445387 (-0.63z)| norm 0.2860 (-0.09z)| lr 4.18e-04 | 322.51 ms | 52.3% bf16 MFU | 1623832 tok/s
step 7695/19560 | loss 3.473830 (+0.03z)| norm 0.2673 (-0.40z)| lr 4.18e-04 | 322.60 ms | 52.3% bf16 MFU | 1623900 tok/s
step 7696/19560 | loss 3.447539 (-0.60z)| norm 0.2723 (-0.32z)| lr 4.18e-04 | 322.57 ms | 52.3% bf16 MFU | 1623971 tok/s
step 7697/19560 | loss 3.436345 (-0.87z)| norm 0.2638 (-0.46z)| lr 4.18e-04 | 322.76 ms | 52.3% bf16 MFU | 1623992 tok/s
step 7698/19560 | loss 3.447267 (-0.60z)| norm 0.2743 (-0.29z)| lr 4.18e-04 | 323.20 ms | 52.2% bf16 MFU | 1623902 tok/s
step 7699/19560 | loss 3.476727 (+0.11z)| norm 0.2836 (-0.13z)| lr 4.18e-04 | 322.77 ms | 52.3% bf16 MFU | 1623924 tok/s
step 7700/19560 | loss 3.456969 (-0.37z)| norm 0.2724 (-0.32z)| lr 4.18e-04 | 322.89 ms | 52.3% bf16 MFU | 1623916 tok/s
step 7701/19560 | loss 3.444601 (-0.66z)| norm 0.2794 (-0.21z)| lr 4.18e-04 | 322.74 ms | 52.3% bf16 MFU | 1623944 tok/s
step 7702/19560 | loss 3.436295 (-0.85z)| norm 0.2767 (-0.25z)| lr 4.18e-04 | 323.33 ms | 52.2% bf16 MFU | 1623822 tok/s
step 7703/19560 | loss 3.473963 (+0.06z)| norm 0.2814 (-0.17z)| lr 4.18e-04 | 322.97 ms | 52.3% bf16 MFU | 1623799 tok/s
step 7704/19560 | loss 3.457790 (-0.34z)| norm 0.2617 (-0.50z)| lr 4.18e-04 | 323.14 ms | 52.2% bf16 MFU | 1623734 tok/s
step 7705/19560 | loss 3.395522 (-1.80z)| norm 0.2878 (-0.07z)| lr 4.18e-04 | 323.01 ms | 52.3% bf16 MFU | 1623705 tok/s
step 7706/19560 | loss 3.418830 (-1.23z)| norm 0.2683 (-0.39z)| lr 4.18e-04 | 322.76 ms | 52.3% bf16 MFU | 1623740 tok/s
step 7707/19560 | loss 3.465689 (-0.11z)| norm 0.2605 (-0.52z)| lr 4.18e-04 | 323.31 ms | 52.2% bf16 MFU | 1623635 tok/s
step 7708/19560 | loss 3.630994 (+3.59z)| norm 0.2821 (-0.16z)| lr 4.18e-04 | 322.84 ms | 52.3% bf16 MFU | 1623652 tok/s
step 7709/19560 | loss 3.436283 (-0.81z)| norm 0.2835 (-0.14z)| lr 4.18e-04 | 322.68 ms | 52.3% bf16 MFU | 1623709 tok/s
step 7710/19560 | loss 3.439220 (-0.74z)| norm 0.2473 (-0.73z)| lr 4.18e-04 | 322.86 ms | 52.3% bf16 MFU | 1623716 tok/s
step 7711/19560 | loss 3.421266 (-1.13z)| norm 0.2852 (-0.11z)| lr 4.18e-04 | 322.87 ms | 52.3% bf16 MFU | 1623723 tok/s
step 7712/19560 | loss 3.522003 (+1.12z)| norm 0.2929 (+0.02z)| lr 4.18e-04 | 322.68 ms | 52.3% bf16 MFU | 1623777 tok/s
step 7713/19560 | loss 3.426791 (-1.00z)| norm 0.2762 (-0.26z)| lr 4.18e-04 | 322.81 ms | 52.3% bf16 MFU | 1623795 tok/s
step 7714/19560 | loss 3.541429 (+1.55z)| norm 0.7694 (+6.40z)| lr 4.18e-04 | 322.63 ms | 52.3% bf16 MFU | 1623857 tok/s
step 7715/19560 | loss 3.431389 (-0.88z)| norm 0.4537 (+2.08z)| lr 4.18e-04 | 323.28 ms | 52.2% bf16 MFU | 1623753 tok/s
step 7716/19560 | loss 3.516754 (+1.00z)| norm 0.3188 (+0.29z)| lr 4.17e-04 | 322.48 ms | 52.3% bf16 MFU | 1623855 tok/s
step 7717/19560 | loss 3.553869 (+1.80z)| norm 0.3272 (+0.40z)| lr 4.17e-04 | 322.49 ms | 52.3% bf16 MFU | 1623950 tok/s
step 7718/19560 | loss 3.498801 (+0.58z)| norm 0.2727 (-0.33z)| lr 4.17e-04 | 322.49 ms | 52.3% bf16 MFU | 1624038 tok/s
step 7719/19560 | loss 3.482821 (+0.24z)| norm 0.2914 (-0.08z)| lr 4.17e-04 | 322.17 ms | 52.4% bf16 MFU | 1624204 tok/s
step 7720/19560 | loss 3.489964 (+0.40z)| norm 0.2650 (-0.42z)| lr 4.17e-04 | 322.28 ms | 52.4% bf16 MFU | 1624336 tok/s
step 7721/19560 | loss 3.500844 (+0.63z)| norm 0.2871 (-0.13z)| lr 4.17e-04 | 322.73 ms | 52.3% bf16 MFU | 1624345 tok/s
step 7722/19560 | loss 3.474000 (+0.04z)| norm 0.2802 (-0.22z)| lr 4.17e-04 | 322.00 ms | 52.4% bf16 MFU | 1624540 tok/s
step 7723/19560 | loss 3.531298 (+1.28z)| norm 0.2760 (-0.27z)| lr 4.17e-04 | 322.69 ms | 52.3% bf16 MFU | 1624551 tok/s
step 7724/19560 | loss 3.467746 (-0.12z)| norm 0.2763 (-0.26z)| lr 4.17e-04 | 322.94 ms | 52.3% bf16 MFU | 1624497 tok/s
step 7725/19560 | loss 3.501866 (+0.63z)| norm 0.2764 (-0.26z)| lr 4.17e-04 | 322.53 ms | 52.3% bf16 MFU | 1624549 tok/s
step 7726/19560 | loss 3.559880 (+1.86z)| norm 0.2877 (-0.11z)| lr 4.17e-04 | 322.22 ms | 52.4% bf16 MFU | 1624677 tok/s
step 7727/19560 | loss 3.497446 (+0.53z)| norm 0.2907 (-0.07z)| lr 4.17e-04 | 322.87 ms | 52.3% bf16 MFU | 1624635 tok/s
step 7728/19560 | loss 3.478065 (+0.10z)| norm 0.2615 (-0.46z)| lr 4.17e-04 | 322.69 ms | 52.3% bf16 MFU | 1624639 tok/s
step 7729/19560 | loss 3.430252 (-0.93z)| norm 0.2729 (-0.30z)| lr 4.17e-04 | 322.41 ms | 52.3% bf16 MFU | 1624715 tok/s
step 7730/19560 | loss 3.475800 (+0.08z)| norm 0.2552 (-0.53z)| lr 4.17e-04 | 322.65 ms | 52.3% bf16 MFU | 1624726 tok/s
step 7731/19560 | loss 3.433334 (-0.87z)| norm 0.2782 (-0.22z)| lr 4.17e-04 | 323.31 ms | 52.2% bf16 MFU | 1624572 tok/s
step 7732/19560 | loss 3.502560 (+0.66z)| norm 0.2480 (-0.61z)| lr 4.17e-04 | 322.97 ms | 52.3% bf16 MFU | 1624510 tok/s
step 7733/19560 | loss 3.425704 (-1.03z)| norm 0.7410 (+5.20z)| lr 4.17e-04 | 322.84 ms | 52.3% bf16 MFU | 1624485 tok/s
step 7734/19560 | loss 3.463694 (-0.20z)| norm 0.2738 (-0.28z)| lr 4.17e-04 | 322.31 ms | 52.4% bf16 MFU | 1624595 tok/s
step 7735/19560 | loss 3.443008 (-0.66z)| norm 0.2667 (-0.36z)| lr 4.17e-04 | 322.69 ms | 52.3% bf16 MFU | 1624602 tok/s
step 7736/19560 | loss 3.497499 (+0.54z)| norm 0.2640 (-0.39z)| lr 4.17e-04 | 323.28 ms | 52.2% bf16 MFU | 1624460 tok/s
step 7737/19560 | loss 3.471334 (-0.03z)| norm 0.2606 (-0.43z)| lr 4.16e-04 | 322.48 ms | 52.3% bf16 MFU | 1624526 tok/s
step 7738/19560 | loss 3.511318 (+0.86z)| norm 0.2681 (-0.34z)| lr 4.16e-04 | 322.76 ms | 52.3% bf16 MFU | 1624520 tok/s
step 7739/19560 | loss 3.484562 (+0.27z)| norm 0.2751 (-0.26z)| lr 4.16e-04 | 322.35 ms | 52.4% bf16 MFU | 1624616 tok/s
step 7740/19560 | loss 3.453469 (-0.43z)| norm 0.2720 (-0.30z)| lr 4.16e-04 | 322.72 ms | 52.3% bf16 MFU | 1624615 tok/s
step 7741/19560 | loss 3.476687 (+0.09z)| norm 0.2608 (-0.42z)| lr 4.16e-04 | 322.54 ms | 52.3% bf16 MFU | 1624660 tok/s
step 7742/19560 | loss 3.454341 (-0.40z)| norm 0.2653 (-0.37z)| lr 4.16e-04 | 322.62 ms | 52.3% bf16 MFU | 1624681 tok/s
step 7743/19560 | loss 3.454838 (-0.39z)| norm 0.3052 (+0.09z)| lr 4.16e-04 | 322.77 ms | 52.3% bf16 MFU | 1624663 tok/s
step 7744/19560 | loss 3.507249 (+0.78z)| norm 0.2470 (-0.59z)| lr 4.16e-04 | 323.04 ms | 52.2% bf16 MFU | 1624580 tok/s
step 7745/19560 | loss 3.439893 (-0.73z)| norm 0.2803 (-0.20z)| lr 4.16e-04 | 322.31 ms | 52.4% bf16 MFU | 1624683 tok/s
step 7746/19560 | loss 3.462511 (-0.22z)| norm 0.2656 (-0.37z)| lr 4.16e-04 | 322.67 ms | 52.3% bf16 MFU | 1624691 tok/s
step 7747/19560 | loss 3.424262 (-1.06z)| norm 0.2760 (-0.25z)| lr 4.16e-04 | 322.63 ms | 52.3% bf16 MFU | 1624708 tok/s
step 7748/19560 | loss 3.541608 (+1.55z)| norm 0.3045 (+0.09z)| lr 4.16e-04 | 322.78 ms | 52.3% bf16 MFU | 1624687 tok/s
step 7749/19560 | loss 3.476271 (+0.08z)| norm 0.2677 (-0.34z)| lr 4.16e-04 | 323.17 ms | 52.2% bf16 MFU | 1624569 tok/s
step 7750/19560 | loss 3.493259 (+0.49z)| norm 0.2712 (-0.30z)| lr 4.16e-04 | 322.82 ms | 52.3% bf16 MFU | 1624546 tok/s
val loss 3.458447
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2865/10042 = 0.285302
step 7751/19560 | loss 3.502538 (+0.70z)| norm 0.2727 (-0.28z)| lr 4.16e-04 | 322.92 ms | 52.3% bf16 MFU | 1624497 tok/s
step 7752/19560 | loss 3.468060 (-0.10z)| norm 0.2645 (-0.37z)| lr 4.16e-04 | 322.63 ms | 52.3% bf16 MFU | 1624525 tok/s
step 7753/19560 | loss 3.486884 (+0.33z)| norm 0.2677 (-0.33z)| lr 4.16e-04 | 322.65 ms | 52.3% bf16 MFU | 1624547 tok/s
step 7754/19560 | loss 3.483437 (+0.25z)| norm 0.2563 (-0.46z)| lr 4.16e-04 | 322.72 ms | 52.3% bf16 MFU | 1624550 tok/s
step 7755/19560 | loss 3.545006 (+1.67z)| norm 0.2794 (-0.19z)| lr 4.16e-04 | 322.56 ms | 52.3% bf16 MFU | 1624593 tok/s
step 7756/19560 | loss 3.432822 (-0.91z)| norm 0.3136 (+0.20z)| lr 4.16e-04 | 322.88 ms | 52.3% bf16 MFU | 1624553 tok/s
step 7757/19560 | loss 3.492687 (+0.48z)| norm 0.2701 (-0.30z)| lr 4.16e-04 | 322.70 ms | 52.3% bf16 MFU | 1624560 tok/s
step 7758/19560 | loss 3.478095 (+0.13z)| norm 0.3247 (+0.33z)| lr 4.16e-04 | 322.74 ms | 52.3% bf16 MFU | 1624556 tok/s
step 7759/19560 | loss 3.453982 (-0.43z)| norm 0.2923 (-0.05z)| lr 4.15e-04 | 322.47 ms | 52.3% bf16 MFU | 1624620 tok/s
step 7760/19560 | loss 3.475974 (+0.08z)| norm 0.2660 (-0.36z)| lr 4.15e-04 | 322.64 ms | 52.3% bf16 MFU | 1624639 tok/s
step 7761/19560 | loss 3.401071 (-1.63z)| norm 0.2855 (-0.13z)| lr 4.15e-04 | 322.58 ms | 52.3% bf16 MFU | 1624672 tok/s
step 7762/19560 | loss 3.453789 (-0.42z)| norm 0.2952 (-0.02z)| lr 4.15e-04 | 322.76 ms | 52.3% bf16 MFU | 1624657 tok/s
step 7763/19560 | loss 3.507470 (+0.80z)| norm 0.2747 (-0.26z)| lr 4.15e-04 | 322.13 ms | 52.4% bf16 MFU | 1624802 tok/s
step 7764/19560 | loss 3.529624 (+1.30z)| norm 0.3033 (+0.07z)| lr 4.15e-04 | 322.61 ms | 52.3% bf16 MFU | 1624820 tok/s
step 7765/19560 | loss 3.472114 (-0.03z)| norm 0.2863 (-0.09z)| lr 4.15e-04 | 323.16 ms | 52.2% bf16 MFU | 1624698 tok/s
step 7766/19560 | loss 3.447708 (-0.58z)| norm 0.2540 (-0.53z)| lr 4.15e-04 | 322.15 ms | 52.4% bf16 MFU | 1624838 tok/s
step 7767/19560 | loss 3.426052 (-1.07z)| norm 0.2827 (-0.13z)| lr 4.15e-04 | 322.34 ms | 52.4% bf16 MFU | 1624920 tok/s
step 7768/19560 | loss 3.434054 (-0.87z)| norm 0.2550 (-0.51z)| lr 4.15e-04 | 323.50 ms | 52.2% bf16 MFU | 1624708 tok/s
step 7769/19560 | loss 3.481716 (+0.22z)| norm 0.2837 (-0.11z)| lr 4.15e-04 | 322.65 ms | 52.3% bf16 MFU | 1624719 tok/s
step 7770/19560 | loss 3.440681 (-0.72z)| norm 0.2849 (-0.09z)| lr 4.15e-04 | 322.11 ms | 52.4% bf16 MFU | 1624865 tok/s
step 7771/19560 | loss 3.578872 (+2.36z)| norm 0.2841 (-0.11z)| lr 4.15e-04 | 322.57 ms | 52.3% bf16 MFU | 1624888 tok/s
step 7772/19560 | loss 3.396573 (-1.68z)| norm 0.2913 (-0.01z)| lr 4.15e-04 | 323.25 ms | 52.2% bf16 MFU | 1624740 tok/s
step 7773/19560 | loss 3.449806 (-0.49z)| norm 0.3027 (+0.15z)| lr 4.15e-04 | 322.39 ms | 52.3% bf16 MFU | 1624816 tok/s
step 7774/19560 | loss 3.603040 (+2.78z)| norm 0.2915 (-0.01z)| lr 4.15e-04 | 322.50 ms | 52.3% bf16 MFU | 1624860 tok/s
step 7775/19560 | loss 3.454629 (-0.40z)| norm 0.2942 (+0.03z)| lr 4.15e-04 | 323.08 ms | 52.2% bf16 MFU | 1624757 tok/s
step 7776/19560 | loss 3.481237 (+0.17z)| norm 0.2549 (-0.51z)| lr 4.15e-04 | 322.69 ms | 52.3% bf16 MFU | 1624755 tok/s
step 7777/19560 | loss 3.438710 (-0.73z)| norm 0.2912 (-0.02z)| lr 4.15e-04 | 323.08 ms | 52.2% bf16 MFU | 1624657 tok/s
step 7778/19560 | loss 3.418087 (-1.16z)| norm 0.2924 (-0.00z)| lr 4.15e-04 | 323.01 ms | 52.2% bf16 MFU | 1624580 tok/s
step 7779/19560 | loss 3.440459 (-0.67z)| norm 0.2768 (-0.22z)| lr 4.15e-04 | 322.35 ms | 52.4% bf16 MFU | 1624674 tok/s
step 7780/19560 | loss 3.613685 (+2.94z)| norm 0.2723 (-0.28z)| lr 4.15e-04 | 322.19 ms | 52.4% bf16 MFU | 1624804 tok/s
step 7781/19560 | loss 3.473325 (-0.01z)| norm 0.2723 (-0.29z)| lr 4.14e-04 | 322.37 ms | 52.4% bf16 MFU | 1624881 tok/s
step 7782/19560 | loss 3.468560 (-0.13z)| norm 0.2704 (-0.31z)| lr 4.14e-04 | 322.45 ms | 52.3% bf16 MFU | 1624935 tok/s
step 7783/19560 | loss 3.450373 (-0.52z)| norm 0.2890 (-0.06z)| lr 4.14e-04 | 322.55 ms | 52.3% bf16 MFU | 1624959 tok/s
step 7784/19560 | loss 3.482784 (+0.19z)| norm 0.3178 (+0.34z)| lr 4.14e-04 | 322.31 ms | 52.4% bf16 MFU | 1625044 tok/s
step 7785/19560 | loss 3.400757 (-1.57z)| norm 0.2901 (-0.05z)| lr 4.14e-04 | 322.30 ms | 52.4% bf16 MFU | 1625128 tok/s
step 7786/19560 | loss 3.486933 (+0.30z)| norm 0.3115 (+0.24z)| lr 4.14e-04 | 323.46 ms | 52.2% bf16 MFU | 1624915 tok/s
step 7787/19560 | loss 3.525103 (+1.11z)| norm 0.2839 (-0.14z)| lr 4.14e-04 | 322.11 ms | 52.4% bf16 MFU | 1625054 tok/s
step 7788/19560 | loss 3.478503 (+0.12z)| norm 0.2934 (-0.01z)| lr 4.14e-04 | 322.48 ms | 52.3% bf16 MFU | 1625091 tok/s
step 7789/19560 | loss 3.513923 (+0.88z)| norm 0.2607 (-0.46z)| lr 4.14e-04 | 321.90 ms | 52.4% bf16 MFU | 1625273 tok/s
step 7790/19560 | loss 3.494957 (+0.45z)| norm 0.2870 (-0.10z)| lr 4.14e-04 | 322.52 ms | 52.3% bf16 MFU | 1625289 tok/s
step 7791/19560 | loss 3.482192 (+0.16z)| norm 0.2732 (-0.30z)| lr 4.14e-04 | 323.27 ms | 52.2% bf16 MFU | 1625117 tok/s
step 7792/19560 | loss 3.473770 (-0.03z)| norm 0.2795 (-0.21z)| lr 4.14e-04 | 322.68 ms | 52.3% bf16 MFU | 1625100 tok/s
step 7793/19560 | loss 3.499402 (+0.55z)| norm 0.2698 (-0.34z)| lr 4.14e-04 | 322.27 ms | 52.4% bf16 MFU | 1625190 tok/s
step 7794/19560 | loss 3.422839 (-1.18z)| norm 0.2684 (-0.36z)| lr 4.14e-04 | 322.58 ms | 52.3% bf16 MFU | 1625194 tok/s
step 7795/19560 | loss 3.425026 (-1.11z)| norm 0.2712 (-0.32z)| lr 4.14e-04 | 322.19 ms | 52.4% bf16 MFU | 1625299 tok/s
step 7796/19560 | loss 3.464377 (-0.22z)| norm 0.2796 (-0.20z)| lr 4.14e-04 | 323.04 ms | 52.2% bf16 MFU | 1625183 tok/s
step 7797/19560 | loss 3.441555 (-0.73z)| norm 0.2925 (-0.02z)| lr 4.14e-04 | 322.62 ms | 52.3% bf16 MFU | 1625177 tok/s
step 7798/19560 | loss 3.494361 (+0.45z)| norm 0.2812 (-0.17z)| lr 4.14e-04 | 322.95 ms | 52.3% bf16 MFU | 1625090 tok/s
step 7799/19560 | loss 3.472399 (-0.04z)| norm 0.2741 (-0.27z)| lr 4.14e-04 | 323.37 ms | 52.2% bf16 MFU | 1624900 tok/s
step 7800/19560 | loss 3.444489 (-0.65z)| norm 0.2881 (-0.07z)| lr 4.14e-04 | 323.22 ms | 52.2% bf16 MFU | 1624758 tok/s
step 7801/19560 | loss 3.412124 (-1.36z)| norm 0.2572 (-0.50z)| lr 4.14e-04 | 322.85 ms | 52.3% bf16 MFU | 1624718 tok/s
step 7802/19560 | loss 3.455162 (-0.39z)| norm 0.3209 (+0.38z)| lr 4.13e-04 | 322.51 ms | 52.3% bf16 MFU | 1624765 tok/s
step 7803/19560 | loss 3.462641 (-0.22z)| norm 0.2775 (-0.22z)| lr 4.13e-04 | 322.63 ms | 52.3% bf16 MFU | 1624780 tok/s
step 7804/19560 | loss 3.440581 (-0.70z)| norm 0.2731 (-0.27z)| lr 4.13e-04 | 322.26 ms | 52.4% bf16 MFU | 1624886 tok/s
step 7805/19560 | loss 3.490616 (+0.43z)| norm 0.2751 (-0.24z)| lr 4.13e-04 | 322.76 ms | 52.3% bf16 MFU | 1624861 tok/s
step 7806/19560 | loss 3.457839 (-0.31z)| norm 0.2813 (-0.15z)| lr 4.13e-04 | 322.89 ms | 52.3% bf16 MFU | 1624804 tok/s
step 7807/19560 | loss 3.469260 (-0.05z)| norm 0.2802 (-0.17z)| lr 4.13e-04 | 322.96 ms | 52.3% bf16 MFU | 1624732 tok/s
step 7808/19560 | loss 3.474701 (+0.07z)| norm 0.2909 (-0.01z)| lr 4.13e-04 | 322.39 ms | 52.4% bf16 MFU | 1624808 tok/s
step 7809/19560 | loss 3.449122 (-0.50z)| norm 0.2924 (+0.00z)| lr 4.13e-04 | 322.32 ms | 52.4% bf16 MFU | 1624897 tok/s
step 7810/19560 | loss 3.418583 (-1.19z)| norm 0.2691 (-0.32z)| lr 4.13e-04 | 322.83 ms | 52.3% bf16 MFU | 1624855 tok/s
step 7811/19560 | loss 3.496118 (+0.57z)| norm 0.3147 (+0.31z)| lr 4.13e-04 | 322.59 ms | 52.3% bf16 MFU | 1624876 tok/s
step 7812/19560 | loss 3.444174 (-0.60z)| norm 0.2903 (-0.03z)| lr 4.13e-04 | 322.50 ms | 52.3% bf16 MFU | 1624916 tok/s
step 7813/19560 | loss 3.503360 (+0.74z)| norm 0.2619 (-0.43z)| lr 4.13e-04 | 323.07 ms | 52.2% bf16 MFU | 1624813 tok/s
step 7814/19560 | loss 3.513059 (+0.95z)| norm 0.3018 (+0.12z)| lr 4.13e-04 | 322.30 ms | 52.4% bf16 MFU | 1624907 tok/s
step 7815/19560 | loss 3.465094 (-0.13z)| norm 0.2997 (+0.09z)| lr 4.13e-04 | 322.42 ms | 52.3% bf16 MFU | 1624968 tok/s
step 7816/19560 | loss 3.458105 (-0.33z)| norm 0.2924 (+0.03z)| lr 4.13e-04 | 323.22 ms | 52.2% bf16 MFU | 1624822 tok/s
step 7817/19560 | loss 3.514205 (+1.01z)| norm 0.3312 (+0.65z)| lr 4.13e-04 | 322.34 ms | 52.4% bf16 MFU | 1624907 tok/s
step 7818/19560 | loss 3.478066 (+0.14z)| norm 0.2821 (-0.12z)| lr 4.13e-04 | 322.79 ms | 52.3% bf16 MFU | 1624874 tok/s
step 7819/19560 | loss 3.453506 (-0.44z)| norm 0.3034 (+0.22z)| lr 4.13e-04 | 322.64 ms | 52.3% bf16 MFU | 1624880 tok/s
step 7820/19560 | loss 3.469297 (-0.07z)| norm 0.2784 (-0.17z)| lr 4.13e-04 | 322.63 ms | 52.3% bf16 MFU | 1624888 tok/s
step 7821/19560 | loss 3.464135 (-0.20z)| norm 0.2782 (-0.18z)| lr 4.13e-04 | 322.97 ms | 52.3% bf16 MFU | 1624811 tok/s
step 7822/19560 | loss 3.406914 (-1.57z)| norm 0.2963 (+0.11z)| lr 4.13e-04 | 322.52 ms | 52.3% bf16 MFU | 1624849 tok/s
step 7823/19560 | loss 3.457557 (-0.35z)| norm 0.2522 (-0.59z)| lr 4.13e-04 | 322.56 ms | 52.3% bf16 MFU | 1624876 tok/s
step 7824/19560 | loss 3.453121 (-0.46z)| norm 0.2787 (-0.17z)| lr 4.12e-04 | 322.76 ms | 52.3% bf16 MFU | 1624852 tok/s
step 7825/19560 | loss 3.524683 (+1.24z)| norm 0.2722 (-0.27z)| lr 4.12e-04 | 323.15 ms | 52.2% bf16 MFU | 1624732 tok/s
step 7826/19560 | loss 3.510461 (+0.89z)| norm 0.2789 (-0.17z)| lr 4.12e-04 | 322.63 ms | 52.3% bf16 MFU | 1624747 tok/s
step 7827/19560 | loss 3.452139 (-0.50z)| norm 0.2877 (-0.03z)| lr 4.12e-04 | 322.62 ms | 52.3% bf16 MFU | 1624764 tok/s
step 7828/19560 | loss 3.465417 (-0.18z)| norm 0.2668 (-0.36z)| lr 4.12e-04 | 322.53 ms | 52.3% bf16 MFU | 1624803 tok/s
step 7829/19560 | loss 3.426736 (-1.10z)| norm 0.2776 (-0.19z)| lr 4.12e-04 | 322.62 ms | 52.3% bf16 MFU | 1624817 tok/s
step 7830/19560 | loss 3.480091 (+0.16z)| norm 0.2729 (-0.26z)| lr 4.12e-04 | 322.78 ms | 52.3% bf16 MFU | 1624790 tok/s
step 7831/19560 | loss 3.425339 (-1.13z)| norm 0.2871 (-0.04z)| lr 4.12e-04 | 322.90 ms | 52.3% bf16 MFU | 1624734 tok/s
step 7832/19560 | loss 3.501539 (+0.67z)| norm 0.3392 (+0.78z)| lr 4.12e-04 | 322.61 ms | 52.3% bf16 MFU | 1624755 tok/s
step 7833/19560 | loss 3.457784 (-0.39z)| norm 0.2638 (-0.41z)| lr 4.12e-04 | 322.74 ms | 52.3% bf16 MFU | 1624743 tok/s
step 7834/19560 | loss 3.514471 (+0.96z)| norm 0.2868 (-0.05z)| lr 4.12e-04 | 322.65 ms | 52.3% bf16 MFU | 1624754 tok/s
step 7835/19560 | loss 3.407466 (-1.59z)| norm 0.2634 (-0.42z)| lr 4.12e-04 | 322.84 ms | 52.3% bf16 MFU | 1624715 tok/s
step 7836/19560 | loss 3.466027 (-0.17z)| norm 0.2767 (-0.21z)| lr 4.12e-04 | 322.79 ms | 52.3% bf16 MFU | 1624692 tok/s
step 7837/19560 | loss 3.449988 (-0.58z)| norm 0.2591 (-0.49z)| lr 4.12e-04 | 322.62 ms | 52.3% bf16 MFU | 1624711 tok/s
step 7838/19560 | loss 3.517545 (+1.12z)| norm 0.2647 (-0.40z)| lr 4.12e-04 | 322.69 ms | 52.3% bf16 MFU | 1624713 tok/s
step 7839/19560 | loss 3.431109 (-1.08z)| norm 0.2673 (-0.36z)| lr 4.12e-04 | 322.94 ms | 52.3% bf16 MFU | 1624652 tok/s
step 7840/19560 | loss 3.460984 (-0.31z)| norm 0.2727 (-0.27z)| lr 4.12e-04 | 322.81 ms | 52.3% bf16 MFU | 1624625 tok/s
step 7841/19560 | loss 3.526651 (+1.35z)| norm 0.2639 (-0.40z)| lr 4.12e-04 | 322.36 ms | 52.4% bf16 MFU | 1624714 tok/s
step 7842/19560 | loss 3.451733 (-0.55z)| norm 0.2629 (-0.48z)| lr 4.12e-04 | 323.21 ms | 52.2% bf16 MFU | 1624586 tok/s
step 7843/19560 | loss 3.486268 (+0.33z)| norm 0.2770 (-0.16z)| lr 4.12e-04 | 323.02 ms | 52.2% bf16 MFU | 1624512 tok/s
step 7844/19560 | loss 3.492863 (+0.50z)| norm 0.2903 (+0.15z)| lr 4.12e-04 | 322.68 ms | 52.3% bf16 MFU | 1624526 tok/s
step 7845/19560 | loss 3.465366 (-0.20z)| norm 0.2508 (-0.74z)| lr 4.11e-04 | 322.66 ms | 52.3% bf16 MFU | 1624544 tok/s
step 7846/19560 | loss 3.484667 (+0.32z)| norm 0.2658 (-0.40z)| lr 4.11e-04 | 322.79 ms | 52.3% bf16 MFU | 1624528 tok/s
step 7847/19560 | loss 3.590823 (+3.00z)| norm 0.2627 (-0.46z)| lr 4.11e-04 | 323.10 ms | 52.2% bf16 MFU | 1624435 tok/s
step 7848/19560 | loss 3.531599 (+1.46z)| norm 0.2948 (+0.26z)| lr 4.11e-04 | 322.22 ms | 52.4% bf16 MFU | 1624568 tok/s
step 7849/19560 | loss 3.458889 (-0.37z)| norm 0.3025 (+0.43z)| lr 4.11e-04 | 322.75 ms | 52.3% bf16 MFU | 1624561 tok/s
step 7850/19560 | loss 3.436411 (-0.93z)| norm 0.2779 (-0.12z)| lr 4.11e-04 | 323.14 ms | 52.2% bf16 MFU | 1624457 tok/s
step 7851/19560 | loss 3.457641 (-0.38z)| norm 0.2688 (-0.33z)| lr 4.11e-04 | 322.92 ms | 52.3% bf16 MFU | 1624413 tok/s
step 7852/19560 | loss 3.415754 (-1.43z)| norm 0.2954 (+0.27z)| lr 4.11e-04 | 322.80 ms | 52.3% bf16 MFU | 1624401 tok/s
step 7853/19560 | loss 3.483060 (+0.28z)| norm 0.2926 (+0.21z)| lr 4.11e-04 | 322.88 ms | 52.3% bf16 MFU | 1624370 tok/s
step 7854/19560 | loss 3.491656 (+0.52z)| norm 0.2860 (+0.06z)| lr 4.11e-04 | 322.51 ms | 52.3% bf16 MFU | 1624433 tok/s
step 7855/19560 | loss 3.466112 (-0.13z)| norm 0.2772 (-0.14z)| lr 4.11e-04 | 322.57 ms | 52.3% bf16 MFU | 1624478 tok/s
step 7856/19560 | loss 3.452226 (-0.49z)| norm 0.2628 (-0.47z)| lr 4.11e-04 | 322.65 ms | 52.3% bf16 MFU | 1624502 tok/s
step 7857/19560 | loss 3.809952 (+6.89z)| norm 0.3638 (+1.78z)| lr 4.11e-04 | 322.37 ms | 52.4% bf16 MFU | 1624593 tok/s
step 7858/19560 | loss 3.392169 (-1.65z)| norm 0.3247 (+0.89z)| lr 4.11e-04 | 322.80 ms | 52.3% bf16 MFU | 1624572 tok/s
step 7859/19560 | loss 3.528068 (+1.09z)| norm 0.2883 (+0.08z)| lr 4.11e-04 | 322.93 ms | 52.3% bf16 MFU | 1624519 tok/s
step 7860/19560 | loss 3.503512 (+0.59z)| norm 0.2962 (+0.25z)| lr 4.11e-04 | 322.90 ms | 52.3% bf16 MFU | 1624477 tok/s
step 7861/19560 | loss 3.444061 (-0.61z)| norm 0.2966 (+0.80z)| lr 4.11e-04 | 322.48 ms | 52.3% bf16 MFU | 1624543 tok/s
step 7862/19560 | loss 3.470542 (-0.08z)| norm 0.3367 (+2.82z)| lr 4.11e-04 | 322.85 ms | 52.3% bf16 MFU | 1624512 tok/s
step 7863/19560 | loss 3.470750 (-0.08z)| norm 0.2870 (+0.24z)| lr 4.11e-04 | 323.62 ms | 52.2% bf16 MFU | 1624290 tok/s
step 7864/19560 | loss 3.474771 (+0.01z)| norm 0.2689 (-0.70z)| lr 4.11e-04 | 322.50 ms | 52.3% bf16 MFU | 1624361 tok/s
step 7865/19560 | loss 3.494049 (+0.40z)| norm 0.2945 (+0.62z)| lr 4.11e-04 | 322.30 ms | 52.4% bf16 MFU | 1624479 tok/s
step 7866/19560 | loss 3.463101 (-0.23z)| norm 0.2958 (+0.67z)| lr 4.11e-04 | 322.82 ms | 52.3% bf16 MFU | 1624458 tok/s
step 7867/19560 | loss 3.477680 (+0.07z)| norm 0.2969 (+0.72z)| lr 4.10e-04 | 323.91 ms | 52.1% bf16 MFU | 1624166 tok/s
step 7868/19560 | loss 3.430127 (-0.89z)| norm 0.2804 (-0.14z)| lr 4.10e-04 | 323.20 ms | 52.2% bf16 MFU | 1624068 tok/s
step 7869/19560 | loss 3.487322 (+0.27z)| norm 0.3252 (+2.14z)| lr 4.10e-04 | 322.79 ms | 52.3% bf16 MFU | 1624076 tok/s
step 7870/19560 | loss 3.458160 (-0.32z)| norm 0.2760 (-0.40z)| lr 4.10e-04 | 322.74 ms | 52.3% bf16 MFU | 1624096 tok/s
step 7871/19560 | loss 3.478211 (+0.08z)| norm 0.2827 (-0.04z)| lr 4.10e-04 | 323.32 ms | 52.2% bf16 MFU | 1623970 tok/s
step 7872/19560 | loss 3.429508 (-0.90z)| norm 0.3070 (+1.20z)| lr 4.10e-04 | 323.24 ms | 52.2% bf16 MFU | 1623872 tok/s
step 7873/19560 | loss 3.613600 (+2.74z)| norm 0.2671 (-0.88z)| lr 4.10e-04 | 322.94 ms | 52.3% bf16 MFU | 1623852 tok/s
step 7874/19560 | loss 3.433341 (-0.82z)| norm 0.3014 (+0.90z)| lr 4.10e-04 | 322.82 ms | 52.3% bf16 MFU | 1623863 tok/s
step 7875/19560 | loss 3.525707 (+0.99z)| norm 0.2751 (-0.47z)| lr 4.10e-04 | 322.81 ms | 52.3% bf16 MFU | 1623878 tok/s
step 7876/19560 | loss 3.457846 (-0.34z)| norm 0.2822 (-0.09z)| lr 4.10e-04 | 322.85 ms | 52.3% bf16 MFU | 1623882 tok/s
step 7877/19560 | loss 3.409043 (-1.29z)| norm 0.2772 (-0.36z)| lr 4.10e-04 | 322.95 ms | 52.3% bf16 MFU | 1623859 tok/s
step 7878/19560 | loss 3.417546 (-1.10z)| norm 0.2761 (-0.42z)| lr 4.10e-04 | 323.23 ms | 52.2% bf16 MFU | 1623767 tok/s
step 7879/19560 | loss 3.508371 (+0.68z)| norm 0.2736 (-0.55z)| lr 4.10e-04 | 323.23 ms | 52.2% bf16 MFU | 1623681 tok/s
step 7880/19560 | loss 3.470830 (-0.06z)| norm 0.2695 (-0.77z)| lr 4.10e-04 | 323.04 ms | 52.2% bf16 MFU | 1623647 tok/s
step 7881/19560 | loss 3.413951 (-1.16z)| norm 0.2663 (-0.94z)| lr 4.10e-04 | 322.46 ms | 52.3% bf16 MFU | 1623758 tok/s
step 7882/19560 | loss 3.525878 (+1.02z)| norm 0.2705 (-0.73z)| lr 4.10e-04 | 322.71 ms | 52.3% bf16 MFU | 1623803 tok/s
step 7883/19560 | loss 3.478911 (+0.11z)| norm 0.2556 (-1.49z)| lr 4.10e-04 | 323.07 ms | 52.2% bf16 MFU | 1623756 tok/s
step 7884/19560 | loss 3.449029 (-0.47z)| norm 0.2716 (-0.64z)| lr 4.10e-04 | 322.76 ms | 52.3% bf16 MFU | 1623787 tok/s
step 7885/19560 | loss 3.470592 (-0.05z)| norm 0.2590 (-1.30z)| lr 4.10e-04 | 323.06 ms | 52.2% bf16 MFU | 1623741 tok/s
step 7886/19560 | loss 3.488428 (+0.30z)| norm 0.2683 (-0.80z)| lr 4.10e-04 | 322.36 ms | 52.4% bf16 MFU | 1623874 tok/s
step 7887/19560 | loss 3.554693 (+1.57z)| norm 0.2681 (-0.80z)| lr 4.10e-04 | 322.99 ms | 52.3% bf16 MFU | 1623843 tok/s
step 7888/19560 | loss 3.462258 (-0.22z)| norm 0.2784 (-0.25z)| lr 4.09e-04 | 323.59 ms | 52.2% bf16 MFU | 1623662 tok/s
step 7889/19560 | loss 3.480033 (+0.11z)| norm 0.2683 (-0.78z)| lr 4.09e-04 | 322.41 ms | 52.3% bf16 MFU | 1623788 tok/s
step 7890/19560 | loss 3.436007 (-0.75z)| norm 0.2737 (-0.49z)| lr 4.09e-04 | 322.78 ms | 52.3% bf16 MFU | 1623814 tok/s
step 7891/19560 | loss 3.369410 (-2.00z)| norm 0.2557 (-1.43z)| lr 4.09e-04 | 322.79 ms | 52.3% bf16 MFU | 1623835 tok/s
step 7892/19560 | loss 3.550978 (+1.49z)| norm 0.2738 (-0.46z)| lr 4.09e-04 | 322.47 ms | 52.3% bf16 MFU | 1623936 tok/s
step 7893/19560 | loss 3.462432 (-0.21z)| norm 0.2626 (-1.05z)| lr 4.09e-04 | 323.19 ms | 52.2% bf16 MFU | 1623850 tok/s
step 7894/19560 | loss 3.461431 (-0.23z)| norm 0.2847 (+0.12z)| lr 4.09e-04 | 323.08 ms | 52.2% bf16 MFU | 1623796 tok/s
step 7895/19560 | loss 3.466129 (-0.15z)| norm 0.2701 (-0.66z)| lr 4.09e-04 | 322.69 ms | 52.3% bf16 MFU | 1623843 tok/s
step 7896/19560 | loss 3.454420 (-0.38z)| norm 0.2959 (+0.71z)| lr 4.09e-04 | 322.98 ms | 52.3% bf16 MFU | 1623816 tok/s
step 7897/19560 | loss 3.483143 (+0.18z)| norm 0.2787 (-0.21z)| lr 4.09e-04 | 322.96 ms | 52.3% bf16 MFU | 1623793 tok/s
step 7898/19560 | loss 3.461788 (-0.24z)| norm 0.2661 (-0.88z)| lr 4.09e-04 | 322.72 ms | 52.3% bf16 MFU | 1623833 tok/s
step 7899/19560 | loss 3.443616 (-0.58z)| norm 0.2935 (+0.59z)| lr 4.09e-04 | 322.91 ms | 52.3% bf16 MFU | 1623822 tok/s
step 7900/19560 | loss 3.434251 (-0.77z)| norm 0.3012 (+0.99z)| lr 4.09e-04 | 322.60 ms | 52.3% bf16 MFU | 1623892 tok/s
step 7901/19560 | loss 3.458102 (-0.30z)| norm 0.3112 (+1.51z)| lr 4.09e-04 | 322.43 ms | 52.3% bf16 MFU | 1624001 tok/s
step 7902/19560 | loss 3.506633 (+0.69z)| norm 0.2812 (-0.08z)| lr 4.09e-04 | 322.46 ms | 52.3% bf16 MFU | 1624096 tok/s
step 7903/19560 | loss 3.564315 (+1.82z)| norm 0.2962 (+0.72z)| lr 4.09e-04 | 323.10 ms | 52.2% bf16 MFU | 1624027 tok/s
step 7904/19560 | loss 3.462658 (-0.21z)| norm 0.2831 (+0.01z)| lr 4.09e-04 | 322.79 ms | 52.3% bf16 MFU | 1624038 tok/s
step 7905/19560 | loss 3.497016 (+0.47z)| norm 0.2686 (-0.75z)| lr 4.09e-04 | 322.82 ms | 52.3% bf16 MFU | 1624041 tok/s
step 7906/19560 | loss 3.439740 (-0.69z)| norm 0.2970 (+0.76z)| lr 4.09e-04 | 322.39 ms | 52.3% bf16 MFU | 1624151 tok/s
step 7907/19560 | loss 3.494539 (+0.41z)| norm 0.2495 (-1.75z)| lr 4.09e-04 | 322.71 ms | 52.3% bf16 MFU | 1624174 tok/s
step 7908/19560 | loss 3.452461 (-0.43z)| norm 0.2792 (-0.18z)| lr 4.09e-04 | 323.00 ms | 52.3% bf16 MFU | 1624124 tok/s
step 7909/19560 | loss 3.483327 (+0.21z)| norm 0.2671 (-0.82z)| lr 4.09e-04 | 322.67 ms | 52.3% bf16 MFU | 1624160 tok/s
step 7910/19560 | loss 3.436212 (-0.76z)| norm 0.2578 (-1.30z)| lr 4.08e-04 | 322.82 ms | 52.3% bf16 MFU | 1624155 tok/s
step 7911/19560 | loss 3.451707 (-0.44z)| norm 0.2757 (-0.35z)| lr 4.08e-04 | 322.84 ms | 52.3% bf16 MFU | 1624146 tok/s
step 7912/19560 | loss 3.489891 (+0.35z)| norm 0.2515 (-1.60z)| lr 4.08e-04 | 322.18 ms | 52.4% bf16 MFU | 1624304 tok/s
step 7913/19560 | loss 3.462612 (-0.23z)| norm 0.2772 (-0.24z)| lr 4.08e-04 | 322.75 ms | 52.3% bf16 MFU | 1624311 tok/s
step 7914/19560 | loss 3.396452 (-1.58z)| norm 0.2385 (-2.23z)| lr 4.08e-04 | 322.50 ms | 52.3% bf16 MFU | 1624380 tok/s
step 7915/19560 | loss 3.452547 (-0.41z)| norm 0.2587 (-1.16z)| lr 4.08e-04 | 322.28 ms | 52.4% bf16 MFU | 1624503 tok/s
step 7916/19560 | loss 3.461627 (-0.22z)| norm 0.2555 (-1.30z)| lr 4.08e-04 | 322.56 ms | 52.3% bf16 MFU | 1624548 tok/s
step 7917/19560 | loss 3.451636 (-0.41z)| norm 0.2905 (+0.50z)| lr 4.08e-04 | 323.01 ms | 52.3% bf16 MFU | 1624478 tok/s
step 7918/19560 | loss 3.502649 (+0.64z)| norm 0.3042 (+1.20z)| lr 4.08e-04 | 322.44 ms | 52.3% bf16 MFU | 1624554 tok/s
step 7919/19560 | loss 3.441085 (-0.63z)| norm 0.2663 (-0.76z)| lr 4.08e-04 | 324.15 ms | 52.1% bf16 MFU | 1624198 tok/s
step 7920/19560 | loss 3.485659 (+0.30z)| norm 0.3004 (+0.99z)| lr 4.08e-04 | 322.01 ms | 52.4% bf16 MFU | 1624396 tok/s
step 7921/19560 | loss 3.450046 (-0.44z)| norm 0.2835 (+0.11z)| lr 4.08e-04 | 322.78 ms | 52.3% bf16 MFU | 1624392 tok/s
step 7922/19560 | loss 3.452549 (-0.39z)| norm 0.2796 (-0.09z)| lr 4.08e-04 | 322.87 ms | 52.3% bf16 MFU | 1624365 tok/s
step 7923/19560 | loss 3.523495 (+1.07z)| norm 0.2937 (+0.63z)| lr 4.08e-04 | 322.67 ms | 52.3% bf16 MFU | 1624389 tok/s
step 7924/19560 | loss 3.420128 (-1.07z)| norm 0.2852 (+0.19z)| lr 4.08e-04 | 322.41 ms | 52.3% bf16 MFU | 1624478 tok/s
step 7925/19560 | loss 3.426541 (-0.93z)| norm 0.2849 (+0.17z)| lr 4.08e-04 | 322.95 ms | 52.3% bf16 MFU | 1624427 tok/s
step 7926/19560 | loss 3.494754 (+0.48z)| norm 0.2657 (-0.81z)| lr 4.08e-04 | 322.50 ms | 52.3% bf16 MFU | 1624490 tok/s
step 7927/19560 | loss 3.502648 (+0.64z)| norm 0.2692 (-0.62z)| lr 4.08e-04 | 322.19 ms | 52.4% bf16 MFU | 1624629 tok/s
step 7928/19560 | loss 3.576786 (+2.11z)| norm 0.2775 (-0.19z)| lr 4.08e-04 | 322.48 ms | 52.3% bf16 MFU | 1624688 tok/s
step 7929/19560 | loss 3.497065 (+0.48z)| norm 0.2579 (-1.20z)| lr 4.08e-04 | 322.68 ms | 52.3% bf16 MFU | 1624692 tok/s
step 7930/19560 | loss 3.515600 (+0.85z)| norm 0.2502 (-1.58z)| lr 4.08e-04 | 322.25 ms | 52.4% bf16 MFU | 1624806 tok/s
step 7931/19560 | loss 3.415571 (-1.18z)| norm 0.2661 (-0.75z)| lr 4.07e-04 | 322.59 ms | 52.3% bf16 MFU | 1624828 tok/s
step 7932/19560 | loss 3.469397 (-0.09z)| norm 0.3004 (+1.01z)| lr 4.07e-04 | 322.19 ms | 52.4% bf16 MFU | 1624950 tok/s
step 7933/19560 | loss 3.453298 (-0.41z)| norm 0.2800 (-0.04z)| lr 4.07e-04 | 322.20 ms | 52.4% bf16 MFU | 1625063 tok/s
step 7934/19560 | loss 3.431337 (-0.85z)| norm 0.2805 (-0.02z)| lr 4.07e-04 | 323.13 ms | 52.2% bf16 MFU | 1624936 tok/s
step 7935/19560 | loss 3.431833 (-0.83z)| norm 0.2961 (+0.78z)| lr 4.07e-04 | 322.61 ms | 52.3% bf16 MFU | 1624946 tok/s
step 7936/19560 | loss 3.506372 (+0.67z)| norm 0.3053 (+1.24z)| lr 4.07e-04 | 322.75 ms | 52.3% bf16 MFU | 1624921 tok/s
step 7937/19560 | loss 3.485136 (+0.23z)| norm 0.2698 (-0.57z)| lr 4.07e-04 | 322.72 ms | 52.3% bf16 MFU | 1624905 tok/s
step 7938/19560 | loss 3.441585 (-0.65z)| norm 0.2870 (+0.30z)| lr 4.07e-04 | 322.21 ms | 52.4% bf16 MFU | 1625017 tok/s
step 7939/19560 | loss 3.414375 (-1.18z)| norm 0.2657 (-0.78z)| lr 4.07e-04 | 322.87 ms | 52.3% bf16 MFU | 1624957 tok/s
step 7940/19560 | loss 3.528814 (+1.10z)| norm 0.2869 (+0.32z)| lr 4.07e-04 | 322.73 ms | 52.3% bf16 MFU | 1624936 tok/s
step 7941/19560 | loss 3.497042 (+0.47z)| norm 0.2728 (-0.41z)| lr 4.07e-04 | 322.88 ms | 52.3% bf16 MFU | 1624879 tok/s
step 7942/19560 | loss 3.354467 (-2.32z)| norm 0.2570 (-1.21z)| lr 4.07e-04 | 322.09 ms | 52.4% bf16 MFU | 1625023 tok/s
step 7943/19560 | loss 3.471577 (-0.02z)| norm 0.2652 (-0.78z)| lr 4.07e-04 | 322.49 ms | 52.3% bf16 MFU | 1625058 tok/s
step 7944/19560 | loss 3.479426 (+0.13z)| norm 0.2793 (-0.04z)| lr 4.07e-04 | 322.55 ms | 52.3% bf16 MFU | 1625077 tok/s
step 7945/19560 | loss 3.451165 (-0.42z)| norm 0.2937 (+0.74z)| lr 4.07e-04 | 322.66 ms | 52.3% bf16 MFU | 1625069 tok/s
step 7946/19560 | loss 3.495407 (+0.45z)| norm 0.2924 (+0.67z)| lr 4.07e-04 | 322.60 ms | 52.3% bf16 MFU | 1625075 tok/s
step 7947/19560 | loss 3.425227 (-0.92z)| norm 0.2755 (-0.22z)| lr 4.07e-04 | 322.59 ms | 52.3% bf16 MFU | 1625084 tok/s
step 7948/19560 | loss 3.574897 (+1.97z)| norm 0.2912 (+0.62z)| lr 4.07e-04 | 322.55 ms | 52.3% bf16 MFU | 1625103 tok/s
step 7949/19560 | loss 3.468737 (-0.08z)| norm 0.2702 (-0.50z)| lr 4.07e-04 | 323.11 ms | 52.2% bf16 MFU | 1624979 tok/s
step 7950/19560 | loss 3.518071 (+0.86z)| norm 0.2798 (+0.02z)| lr 4.07e-04 | 323.07 ms | 52.2% bf16 MFU | 1624873 tok/s
step 7951/19560 | loss 3.508594 (+0.67z)| norm 0.2667 (-0.69z)| lr 4.07e-04 | 323.28 ms | 52.2% bf16 MFU | 1624717 tok/s
step 7952/19560 | loss 3.419104 (-1.06z)| norm 0.2933 (+0.73z)| lr 4.07e-04 | 322.31 ms | 52.4% bf16 MFU | 1624814 tok/s
step 7953/19560 | loss 3.493788 (+0.39z)| norm 0.3063 (+1.41z)| lr 4.06e-04 | 322.28 ms | 52.4% bf16 MFU | 1624914 tok/s
step 7954/19560 | loss 3.477740 (+0.08z)| norm 0.2651 (-0.79z)| lr 4.06e-04 | 322.27 ms | 52.4% bf16 MFU | 1625011 tok/s
step 7955/19560 | loss 3.494067 (+0.39z)| norm 0.2955 (+0.83z)| lr 4.06e-04 | 322.84 ms | 52.3% bf16 MFU | 1624960 tok/s
step 7956/19560 | loss 3.483289 (+0.18z)| norm 0.2760 (-0.21z)| lr 4.06e-04 | 322.47 ms | 52.3% bf16 MFU | 1625006 tok/s
step 7957/19560 | loss 3.498377 (+0.46z)| norm 0.2883 (+0.44z)| lr 4.06e-04 | 322.61 ms | 52.3% bf16 MFU | 1625012 tok/s
step 7958/19560 | loss 3.498023 (+0.45z)| norm 0.2912 (+0.59z)| lr 4.06e-04 | 322.78 ms | 52.3% bf16 MFU | 1624976 tok/s
step 7959/19560 | loss 3.463426 (-0.23z)| norm 0.2719 (-0.44z)| lr 4.06e-04 | 322.39 ms | 52.4% bf16 MFU | 1625040 tok/s
step 7960/19560 | loss 3.470291 (-0.09z)| norm 0.2823 (+0.14z)| lr 4.06e-04 | 322.47 ms | 52.3% bf16 MFU | 1625081 tok/s
step 7961/19560 | loss 3.424570 (-0.97z)| norm 0.2887 (+0.49z)| lr 4.06e-04 | 322.12 ms | 52.4% bf16 MFU | 1625207 tok/s
step 7962/19560 | loss 3.515980 (+0.81z)| norm 0.2987 (+1.04z)| lr 4.06e-04 | 322.24 ms | 52.4% bf16 MFU | 1625297 tok/s
step 7963/19560 | loss 3.496860 (+0.43z)| norm 0.2802 (+0.00z)| lr 4.06e-04 | 323.14 ms | 52.2% bf16 MFU | 1625157 tok/s
step 7964/19560 | loss 3.473211 (-0.04z)| norm 0.2782 (-0.11z)| lr 4.06e-04 | 322.54 ms | 52.3% bf16 MFU | 1625174 tok/s
step 7965/19560 | loss 3.432247 (-0.84z)| norm 0.2781 (-0.12z)| lr 4.06e-04 | 322.67 ms | 52.3% bf16 MFU | 1625157 tok/s
step 7966/19560 | loss 3.518589 (+0.85z)| norm 0.2946 (+0.79z)| lr 4.06e-04 | 322.44 ms | 52.3% bf16 MFU | 1625199 tok/s
step 7967/19560 | loss 3.424242 (-0.99z)| norm 0.3130 (+1.78z)| lr 4.06e-04 | 322.71 ms | 52.3% bf16 MFU | 1625172 tok/s
step 7968/19560 | loss 3.525758 (+0.98z)| norm 0.2934 (+0.68z)| lr 4.06e-04 | 322.50 ms | 52.3% bf16 MFU | 1625199 tok/s
step 7969/19560 | loss 3.457010 (-0.35z)| norm 0.2797 (-0.08z)| lr 4.06e-04 | 322.76 ms | 52.3% bf16 MFU | 1625159 tok/s
step 7970/19560 | loss 3.550543 (+1.45z)| norm 0.3051 (+1.31z)| lr 4.06e-04 | 322.88 ms | 52.3% bf16 MFU | 1625091 tok/s
step 7971/19560 | loss 3.499519 (+0.46z)| norm 0.2934 (+0.65z)| lr 4.06e-04 | 322.47 ms | 52.3% bf16 MFU | 1625128 tok/s
step 7972/19560 | loss 3.456284 (-0.37z)| norm 0.2842 (+0.14z)| lr 4.06e-04 | 322.33 ms | 52.4% bf16 MFU | 1625201 tok/s
step 7973/19560 | loss 3.463039 (-0.24z)| norm 0.2632 (-1.03z)| lr 4.06e-04 | 322.68 ms | 52.3% bf16 MFU | 1625180 tok/s
step 7974/19560 | loss 3.473336 (-0.04z)| norm 0.2976 (+0.88z)| lr 4.05e-04 | 322.16 ms | 52.4% bf16 MFU | 1625292 tok/s
step 7975/19560 | loss 3.454445 (-0.39z)| norm 0.2814 (-0.04z)| lr 4.05e-04 | 322.52 ms | 52.3% bf16 MFU | 1625307 tok/s
step 7976/19560 | loss 3.381644 (-1.80z)| norm 0.2617 (-1.12z)| lr 4.05e-04 | 322.39 ms | 52.4% bf16 MFU | 1625355 tok/s
step 7977/19560 | loss 3.511580 (+0.74z)| norm 0.2896 (+0.45z)| lr 4.05e-04 | 322.58 ms | 52.3% bf16 MFU | 1625351 tok/s
step 7978/19560 | loss 3.415625 (-1.13z)| norm 0.2800 (-0.09z)| lr 4.05e-04 | 322.57 ms | 52.3% bf16 MFU | 1625350 tok/s
step 7979/19560 | loss 3.441562 (-0.62z)| norm 0.2828 (+0.06z)| lr 4.05e-04 | 322.53 ms | 52.3% bf16 MFU | 1625359 tok/s
step 7980/19560 | loss 3.509588 (+0.69z)| norm 0.3176 (+1.97z)| lr 4.05e-04 | 322.24 ms | 52.4% bf16 MFU | 1625443 tok/s
step 7981/19560 | loss 3.482393 (+0.16z)| norm 0.3089 (+1.47z)| lr 4.05e-04 | 322.45 ms | 52.3% bf16 MFU | 1625469 tok/s
step 7982/19560 | loss 3.436014 (-0.73z)| norm 0.2972 (+0.82z)| lr 4.05e-04 | 322.38 ms | 52.4% bf16 MFU | 1625511 tok/s
step 7983/19560 | loss 3.460808 (-0.25z)| norm 0.2776 (-0.25z)| lr 4.05e-04 | 322.30 ms | 52.4% bf16 MFU | 1625570 tok/s
step 7984/19560 | loss 3.514491 (+0.79z)| norm 0.2624 (-1.08z)| lr 4.05e-04 | 322.77 ms | 52.3% bf16 MFU | 1625507 tok/s
step 7985/19560 | loss 3.517333 (+1.08z)| norm 0.2709 (-0.63z)| lr 4.05e-04 | 322.85 ms | 52.3% bf16 MFU | 1625429 tok/s
step 7986/19560 | loss 3.461009 (-0.27z)| norm 0.2658 (-0.93z)| lr 4.05e-04 | 322.85 ms | 52.3% bf16 MFU | 1625354 tok/s
step 7987/19560 | loss 3.513745 (+1.01z)| norm 0.3021 (+1.28z)| lr 4.05e-04 | 322.56 ms | 52.3% bf16 MFU | 1625355 tok/s
step 7988/19560 | loss 3.488988 (+0.41z)| norm 0.2646 (-0.99z)| lr 4.05e-04 | 322.70 ms | 52.3% bf16 MFU | 1625321 tok/s
step 7989/19560 | loss 3.471036 (-0.03z)| norm 0.2722 (-0.51z)| lr 4.05e-04 | 322.19 ms | 52.4% bf16 MFU | 1625420 tok/s
step 7990/19560 | loss 3.456442 (-0.38z)| norm 0.2828 (+0.16z)| lr 4.05e-04 | 322.64 ms | 52.3% bf16 MFU | 1625397 tok/s
step 7991/19560 | loss 3.420627 (-1.24z)| norm 0.3316 (+3.13z)| lr 4.05e-04 | 322.70 ms | 52.3% bf16 MFU | 1625361 tok/s
step 7992/19560 | loss 3.410456 (-1.46z)| norm 0.2974 (+1.01z)| lr 4.05e-04 | 322.86 ms | 52.3% bf16 MFU | 1625287 tok/s
step 7993/19560 | loss 3.444955 (-0.62z)| norm 0.3073 (+1.60z)| lr 4.05e-04 | 322.77 ms | 52.3% bf16 MFU | 1625239 tok/s
step 7994/19560 | loss 3.463541 (-0.18z)| norm 0.2643 (-1.00z)| lr 4.05e-04 | 322.63 ms | 52.3% bf16 MFU | 1625231 tok/s
step 7995/19560 | loss 3.501474 (+0.72z)| norm 0.2860 (+0.33z)| lr 4.05e-04 | 322.92 ms | 52.3% bf16 MFU | 1625148 tok/s
step 7996/19560 | loss 3.452245 (-0.46z)| norm 0.2744 (-0.37z)| lr 4.04e-04 | 322.30 ms | 52.4% bf16 MFU | 1625225 tok/s
step 7997/19560 | loss 3.500881 (+0.71z)| norm 0.2603 (-1.23z)| lr 4.04e-04 | 322.64 ms | 52.3% bf16 MFU | 1625214 tok/s
step 7998/19560 | loss 3.494665 (+0.55z)| norm 0.2544 (-1.58z)| lr 4.04e-04 | 322.55 ms | 52.3% bf16 MFU | 1625226 tok/s
step 7999/19560 | loss 3.515371 (+1.03z)| norm 0.2709 (-0.55z)| lr 4.04e-04 | 322.26 ms | 52.4% bf16 MFU | 1625311 tok/s
step 8000/19560 | loss 3.477132 (+0.11z)| norm 0.2598 (-1.21z)| lr 4.04e-04 | 322.29 ms | 52.4% bf16 MFU | 1625382 tok/s
val loss 3.448355
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2846/10042 = 0.283410
step 8001/19560 | loss 3.452466 (-0.47z)| norm 0.2498 (-1.81z)| lr 4.04e-04 | 322.16 ms | 52.4% bf16 MFU | 1625485 tok/s
step 8002/19560 | loss 3.538170 (+1.65z)| norm 0.2909 (+0.73z)| lr 4.04e-04 | 322.43 ms | 52.3% bf16 MFU | 1625514 tok/s
step 8003/19560 | loss 3.476079 (+0.11z)| norm 0.2951 (+0.97z)| lr 4.04e-04 | 322.61 ms | 52.3% bf16 MFU | 1625496 tok/s
step 8004/19560 | loss 3.437048 (-0.86z)| norm 0.2767 (-0.16z)| lr 4.04e-04 | 322.67 ms | 52.3% bf16 MFU | 1625464 tok/s
step 8005/19560 | loss 3.526115 (+1.35z)| norm 0.2934 (+0.86z)| lr 4.04e-04 | 322.77 ms | 52.3% bf16 MFU | 1625407 tok/s
step 8006/19560 | loss 3.506467 (+0.84z)| norm 0.2755 (-0.24z)| lr 4.04e-04 | 322.62 ms | 52.3% bf16 MFU | 1625390 tok/s
step 8007/19560 | loss 3.478049 (+0.13z)| norm 0.2800 (+0.03z)| lr 4.04e-04 | 322.48 ms | 52.3% bf16 MFU | 1625409 tok/s
step 8008/19560 | loss 3.493196 (+0.51z)| norm 0.2489 (-1.85z)| lr 4.04e-04 | 322.76 ms | 52.3% bf16 MFU | 1625359 tok/s
step 8009/19560 | loss 3.432867 (-1.02z)| norm 0.2821 (+0.16z)| lr 4.04e-04 | 322.61 ms | 52.3% bf16 MFU | 1625348 tok/s
step 8010/19560 | loss 3.402009 (-1.77z)| norm 0.2598 (-1.19z)| lr 4.04e-04 | 322.32 ms | 52.4% bf16 MFU | 1625412 tok/s
step 8011/19560 | loss 3.450325 (-0.54z)| norm 0.2756 (-0.24z)| lr 4.04e-04 | 322.16 ms | 52.4% bf16 MFU | 1625513 tok/s
step 8012/19560 | loss 3.479403 (+0.18z)| norm 0.2761 (-0.21z)| lr 4.04e-04 | 322.50 ms | 52.3% bf16 MFU | 1625524 tok/s
step 8013/19560 | loss 3.542377 (+1.74z)| norm 0.2973 (+1.07z)| lr 4.04e-04 | 322.73 ms | 52.3% bf16 MFU | 1625474 tok/s
step 8014/19560 | loss 3.468172 (-0.11z)| norm 0.2647 (-0.93z)| lr 4.04e-04 | 322.59 ms | 52.3% bf16 MFU | 1625463 tok/s
step 8015/19560 | loss 3.423878 (-1.20z)| norm 0.3008 (+1.26z)| lr 4.04e-04 | 322.29 ms | 52.4% bf16 MFU | 1625528 tok/s
step 8016/19560 | loss 3.417363 (-1.35z)| norm 0.2784 (-0.10z)| lr 4.04e-04 | 322.59 ms | 52.3% bf16 MFU | 1625514 tok/s
step 8017/19560 | loss 3.457446 (-0.34z)| norm 0.3094 (+1.75z)| lr 4.03e-04 | 322.65 ms | 52.3% bf16 MFU | 1625486 tok/s
step 8018/19560 | loss 3.460347 (-0.27z)| norm 0.2514 (-1.72z)| lr 4.03e-04 | 322.77 ms | 52.3% bf16 MFU | 1625430 tok/s
step 8019/19560 | loss 3.497928 (+0.66z)| norm 0.3159 (+2.08z)| lr 4.03e-04 | 322.93 ms | 52.3% bf16 MFU | 1625336 tok/s
step 8020/19560 | loss 3.491471 (+0.52z)| norm 0.2789 (-0.11z)| lr 4.03e-04 | 322.08 ms | 52.4% bf16 MFU | 1625460 tok/s
step 8021/19560 | loss 3.432832 (-1.00z)| norm 0.2923 (+0.67z)| lr 4.03e-04 | 322.36 ms | 52.4% bf16 MFU | 1625506 tok/s
step 8022/19560 | loss 3.493039 (+0.55z)| norm 0.3088 (+1.62z)| lr 4.03e-04 | 323.06 ms | 52.2% bf16 MFU | 1625375 tok/s
step 8023/19560 | loss 3.436702 (-0.90z)| norm 0.2731 (-0.47z)| lr 4.03e-04 | 322.58 ms | 52.3% bf16 MFU | 1625372 tok/s
step 8024/19560 | loss 3.439655 (-0.82z)| norm 0.2671 (-0.81z)| lr 4.03e-04 | 322.12 ms | 52.4% bf16 MFU | 1625485 tok/s
step 8025/19560 | loss 3.432504 (-0.99z)| norm 0.2834 (+0.14z)| lr 4.03e-04 | 322.74 ms | 52.3% bf16 MFU | 1625435 tok/s
step 8026/19560 | loss 3.442844 (-0.72z)| norm 0.2536 (-1.59z)| lr 4.03e-04 | 322.53 ms | 52.3% bf16 MFU | 1625440 tok/s
step 8027/19560 | loss 3.420719 (-1.28z)| norm 0.4697 (+7.85z)| lr 4.03e-04 | 322.70 ms | 52.3% bf16 MFU | 1625402 tok/s
step 8028/19560 | loss 3.489376 (+0.47z)| norm 0.2728 (-0.39z)| lr 4.03e-04 | 322.95 ms | 52.3% bf16 MFU | 1625304 tok/s
step 8029/19560 | loss 3.528022 (+1.44z)| norm 0.2971 (+0.64z)| lr 4.03e-04 | 322.52 ms | 52.3% bf16 MFU | 1625317 tok/s
step 8030/19560 | loss 3.424855 (-1.17z)| norm 0.2678 (-0.59z)| lr 4.03e-04 | 322.67 ms | 52.3% bf16 MFU | 1625294 tok/s
step 8031/19560 | loss 3.447334 (-0.59z)| norm 0.2688 (-0.54z)| lr 4.03e-04 | 322.16 ms | 52.4% bf16 MFU | 1625399 tok/s
step 8032/19560 | loss 3.475389 (+0.14z)| norm 0.2676 (-0.59z)| lr 4.03e-04 | 322.76 ms | 52.3% bf16 MFU | 1625348 tok/s
step 8033/19560 | loss 3.469728 (-0.01z)| norm 0.2814 (-0.01z)| lr 4.03e-04 | 322.66 ms | 52.3% bf16 MFU | 1625326 tok/s
step 8034/19560 | loss 3.437891 (-0.83z)| norm 0.2680 (-0.56z)| lr 4.03e-04 | 322.94 ms | 52.3% bf16 MFU | 1625235 tok/s
step 8035/19560 | loss 3.451470 (-0.47z)| norm 0.2691 (-0.52z)| lr 4.03e-04 | 322.73 ms | 52.3% bf16 MFU | 1625199 tok/s
step 8036/19560 | loss 3.436028 (-0.87z)| norm 0.2542 (-1.14z)| lr 4.03e-04 | 323.28 ms | 52.2% bf16 MFU | 1625029 tok/s
step 8037/19560 | loss 3.428103 (-1.06z)| norm 0.2667 (-0.62z)| lr 4.03e-04 | 322.64 ms | 52.3% bf16 MFU | 1625028 tok/s
step 8038/19560 | loss 3.497617 (+0.73z)| norm 0.5088 (+7.28z)| lr 4.02e-04 | 323.51 ms | 52.2% bf16 MFU | 1624807 tok/s
step 8039/19560 | loss 3.444383 (-0.65z)| norm 0.2998 (+0.53z)| lr 4.02e-04 | 322.49 ms | 52.3% bf16 MFU | 1624855 tok/s
step 8040/19560 | loss 3.479012 (+0.25z)| norm 0.2769 (-0.22z)| lr 4.02e-04 | 322.99 ms | 52.3% bf16 MFU | 1624774 tok/s
step 8041/19560 | loss 3.477558 (+0.21z)| norm 0.2847 (+0.03z)| lr 4.02e-04 | 322.75 ms | 52.3% bf16 MFU | 1624758 tok/s
step 8042/19560 | loss 3.447442 (-0.59z)| norm 0.2732 (-0.35z)| lr 4.02e-04 | 322.51 ms | 52.3% bf16 MFU | 1624803 tok/s
step 8043/19560 | loss 3.379241 (-2.31z)| norm 0.2724 (-0.38z)| lr 4.02e-04 | 323.06 ms | 52.2% bf16 MFU | 1624706 tok/s
step 8044/19560 | loss 3.413341 (-1.42z)| norm 0.2670 (-0.56z)| lr 4.02e-04 | 322.47 ms | 52.3% bf16 MFU | 1624763 tok/s
step 8045/19560 | loss 3.526695 (+1.44z)| norm 0.6381 (+8.06z)| lr 4.02e-04 | 322.26 ms | 52.4% bf16 MFU | 1624871 tok/s
step 8046/19560 | loss 3.447789 (-0.54z)| norm 0.2692 (-0.40z)| lr 4.02e-04 | 323.67 ms | 52.1% bf16 MFU | 1624618 tok/s
step 8047/19560 | loss 3.448811 (-0.52z)| norm 0.2490 (-0.86z)| lr 4.02e-04 | 322.89 ms | 52.3% bf16 MFU | 1624573 tok/s
step 8048/19560 | loss 3.462379 (-0.17z)| norm 0.2662 (-0.46z)| lr 4.02e-04 | 322.60 ms | 52.3% bf16 MFU | 1624604 tok/s
step 8049/19560 | loss 3.492195 (+0.58z)| norm 0.2785 (-0.18z)| lr 4.02e-04 | 322.96 ms | 52.3% bf16 MFU | 1624542 tok/s
step 8050/19560 | loss 3.416892 (-1.31z)| norm 0.2666 (-0.45z)| lr 4.02e-04 | 322.62 ms | 52.3% bf16 MFU | 1624571 tok/s
step 8051/19560 | loss 3.441258 (-0.69z)| norm 0.2732 (-0.29z)| lr 4.02e-04 | 323.17 ms | 52.2% bf16 MFU | 1624459 tok/s
step 8052/19560 | loss 3.448405 (-0.51z)| norm 0.2523 (-0.76z)| lr 4.02e-04 | 322.38 ms | 52.4% bf16 MFU | 1624550 tok/s
step 8053/19560 | loss 3.499894 (+0.78z)| norm 0.2951 (+0.21z)| lr 4.02e-04 | 323.23 ms | 52.2% bf16 MFU | 1624424 tok/s
step 8054/19560 | loss 3.561571 (+2.30z)| norm 0.3005 (+0.33z)| lr 4.02e-04 | 322.91 ms | 52.3% bf16 MFU | 1624385 tok/s
step 8055/19560 | loss 3.445504 (-0.60z)| norm 0.2812 (-0.11z)| lr 4.02e-04 | 322.46 ms | 52.3% bf16 MFU | 1624461 tok/s
step 8056/19560 | loss 3.478900 (+0.27z)| norm 0.2844 (-0.04z)| lr 4.02e-04 | 323.02 ms | 52.2% bf16 MFU | 1624392 tok/s
step 8057/19560 | loss 3.494650 (+0.67z)| norm 0.3039 (+0.40z)| lr 4.02e-04 | 322.60 ms | 52.3% bf16 MFU | 1624431 tok/s
step 8058/19560 | loss 3.553986 (+2.17z)| norm 0.2631 (-0.54z)| lr 4.02e-04 | 322.49 ms | 52.3% bf16 MFU | 1624496 tok/s
step 8059/19560 | loss 3.445965 (-0.59z)| norm 0.2970 (+0.23z)| lr 4.01e-04 | 322.93 ms | 52.3% bf16 MFU | 1624447 tok/s
step 8060/19560 | loss 3.445854 (-0.59z)| norm 0.2716 (-0.34z)| lr 4.01e-04 | 323.01 ms | 52.2% bf16 MFU | 1624380 tok/s
step 8061/19560 | loss 3.431827 (-0.94z)| norm 0.2735 (-0.30z)| lr 4.01e-04 | 322.81 ms | 52.3% bf16 MFU | 1624368 tok/s
step 8062/19560 | loss 3.492123 (+0.59z)| norm 0.2833 (-0.08z)| lr 4.01e-04 | 322.25 ms | 52.4% bf16 MFU | 1624496 tok/s
step 8063/19560 | loss 3.444322 (-0.64z)| norm 0.3031 (+0.38z)| lr 4.01e-04 | 323.53 ms | 52.2% bf16 MFU | 1624297 tok/s
step 8064/19560 | loss 3.468634 (-0.01z)| norm 0.2555 (-0.70z)| lr 4.01e-04 | 322.68 ms | 52.3% bf16 MFU | 1624321 tok/s
step 8065/19560 | loss 3.413982 (-1.39z)| norm 0.2842 (-0.05z)| lr 4.01e-04 | 322.56 ms | 52.3% bf16 MFU | 1624375 tok/s
step 8066/19560 | loss 3.484256 (+0.40z)| norm 0.2788 (-0.17z)| lr 4.01e-04 | 322.67 ms | 52.3% bf16 MFU | 1624398 tok/s
step 8067/19560 | loss 3.503905 (+0.89z)| norm 0.2657 (-0.47z)| lr 4.01e-04 | 322.27 ms | 52.4% bf16 MFU | 1624521 tok/s
step 8068/19560 | loss 3.496096 (+0.70z)| norm 0.2974 (+0.25z)| lr 4.01e-04 | 322.54 ms | 52.3% bf16 MFU | 1624571 tok/s
step 8069/19560 | loss 3.460140 (-0.23z)| norm 0.2768 (-0.22z)| lr 4.01e-04 | 322.38 ms | 52.4% bf16 MFU | 1624656 tok/s
step 8070/19560 | loss 3.460537 (-0.25z)| norm 0.2635 (-0.53z)| lr 4.01e-04 | 322.49 ms | 52.3% bf16 MFU | 1624710 tok/s
step 8071/19560 | loss 3.465949 (-0.10z)| norm 0.2706 (-0.37z)| lr 4.01e-04 | 322.57 ms | 52.3% bf16 MFU | 1624742 tok/s
step 8072/19560 | loss 3.438629 (-0.82z)| norm 0.2426 (-1.00z)| lr 4.01e-04 | 322.55 ms | 52.3% bf16 MFU | 1624777 tok/s
step 8073/19560 | loss 3.391223 (-2.05z)| norm 0.2608 (-0.57z)| lr 4.01e-04 | 322.45 ms | 52.3% bf16 MFU | 1624836 tok/s
step 8074/19560 | loss 3.431296 (-0.98z)| norm 0.2468 (-0.88z)| lr 4.01e-04 | 322.44 ms | 52.3% bf16 MFU | 1624894 tok/s
step 8075/19560 | loss 3.499957 (+0.82z)| norm 0.2521 (-0.76z)| lr 4.01e-04 | 322.79 ms | 52.3% bf16 MFU | 1624860 tok/s
step 8076/19560 | loss 3.457575 (-0.29z)| norm 0.2401 (-1.01z)| lr 4.01e-04 | 321.82 ms | 52.4% bf16 MFU | 1625073 tok/s
step 8077/19560 | loss 3.443516 (-0.66z)| norm 0.2721 (-0.29z)| lr 4.01e-04 | 322.71 ms | 52.3% bf16 MFU | 1625051 tok/s
step 8078/19560 | loss 3.523494 (+1.51z)| norm 0.2581 (-0.60z)| lr 4.01e-04 | 322.77 ms | 52.3% bf16 MFU | 1625015 tok/s
step 8079/19560 | loss 3.409162 (-1.56z)| norm 0.2535 (-0.70z)| lr 4.01e-04 | 322.57 ms | 52.3% bf16 MFU | 1625033 tok/s
step 8080/19560 | loss 3.389805 (-2.06z)| norm 0.2553 (-0.66z)| lr 4.01e-04 | 322.30 ms | 52.4% bf16 MFU | 1625116 tok/s
step 8081/19560 | loss 3.437672 (-0.77z)| norm 0.2628 (-0.48z)| lr 4.00e-04 | 322.43 ms | 52.3% bf16 MFU | 1625164 tok/s
step 8082/19560 | loss 3.433413 (-0.87z)| norm 0.3414 (+1.27z)| lr 4.00e-04 | 322.66 ms | 52.3% bf16 MFU | 1625150 tok/s
step 8083/19560 | loss 3.423380 (-1.12z)| norm 0.2818 (-0.06z)| lr 4.00e-04 | 322.55 ms | 52.3% bf16 MFU | 1625164 tok/s
step 8084/19560 | loss 3.462369 (-0.08z)| norm 0.2665 (-0.40z)| lr 4.00e-04 | 323.17 ms | 52.2% bf16 MFU | 1625024 tok/s
step 8085/19560 | loss 3.466531 (+0.04z)| norm 0.2691 (-0.34z)| lr 4.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1625014 tok/s
step 8086/19560 | loss 3.445737 (-0.51z)| norm 0.2621 (-0.49z)| lr 4.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1624998 tok/s
step 8087/19560 | loss 3.407174 (-1.51z)| norm 0.2860 (+0.04z)| lr 4.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1625005 tok/s
step 8088/19560 | loss 3.506322 (+1.10z)| norm 0.2704 (-0.31z)| lr 4.00e-04 | 322.68 ms | 52.3% bf16 MFU | 1624994 tok/s
step 8089/19560 | loss 3.426090 (-1.02z)| norm 0.2547 (-0.65z)| lr 4.00e-04 | 322.23 ms | 52.4% bf16 MFU | 1625098 tok/s
step 8090/19560 | loss 3.444021 (-0.53z)| norm 0.2800 (-0.08z)| lr 4.00e-04 | 322.34 ms | 52.4% bf16 MFU | 1625169 tok/s
step 8091/19560 | loss 3.463555 (-0.01z)| norm 0.2673 (-0.36z)| lr 4.00e-04 | 322.64 ms | 52.3% bf16 MFU | 1625161 tok/s
step 8092/19560 | loss 3.528061 (+1.68z)| norm 0.2905 (+0.15z)| lr 4.00e-04 | 323.03 ms | 52.2% bf16 MFU | 1625053 tok/s
step 8093/19560 | loss 3.482784 (+0.48z)| norm 0.2814 (-0.05z)| lr 4.00e-04 | 322.69 ms | 52.3% bf16 MFU | 1625039 tok/s
step 8094/19560 | loss 3.523967 (+1.56z)| norm 0.3148 (+0.68z)| lr 4.00e-04 | 322.54 ms | 52.3% bf16 MFU | 1625061 tok/s
step 8095/19560 | loss 3.449972 (-0.39z)| norm 0.3383 (+1.20z)| lr 4.00e-04 | 322.44 ms | 52.3% bf16 MFU | 1625109 tok/s
step 8096/19560 | loss 3.498468 (+0.90z)| norm 0.3150 (+0.68z)| lr 4.00e-04 | 322.22 ms | 52.4% bf16 MFU | 1625209 tok/s
step 8097/19560 | loss 3.500377 (+0.94z)| norm 0.2798 (-0.10z)| lr 4.00e-04 | 323.23 ms | 52.2% bf16 MFU | 1625050 tok/s
step 8098/19560 | loss 3.457263 (-0.19z)| norm 0.3030 (+0.41z)| lr 4.00e-04 | 322.69 ms | 52.3% bf16 MFU | 1625034 tok/s
step 8099/19560 | loss 3.426305 (-1.01z)| norm 0.2839 (-0.01z)| lr 4.00e-04 | 322.89 ms | 52.3% bf16 MFU | 1624969 tok/s
step 8100/19560 | loss 3.476431 (+0.34z)| norm 0.2989 (+0.32z)| lr 4.00e-04 | 322.11 ms | 52.4% bf16 MFU | 1625105 tok/s
step 8101/19560 | loss 3.479382 (+0.42z)| norm 0.2906 (+0.13z)| lr 4.00e-04 | 322.60 ms | 52.3% bf16 MFU | 1625108 tok/s
step 8102/19560 | loss 3.508260 (+1.18z)| norm 0.2934 (+0.20z)| lr 3.99e-04 | 322.74 ms | 52.3% bf16 MFU | 1625077 tok/s
step 8103/19560 | loss 3.486901 (+0.60z)| norm 0.2804 (-0.09z)| lr 3.99e-04 | 322.47 ms | 52.3% bf16 MFU | 1625114 tok/s
step 8104/19560 | loss 3.489400 (+0.66z)| norm 0.3162 (+0.69z)| lr 3.99e-04 | 322.54 ms | 52.3% bf16 MFU | 1625133 tok/s
step 8105/19560 | loss 3.474257 (+0.25z)| norm 0.2978 (+0.28z)| lr 3.99e-04 | 322.80 ms | 52.3% bf16 MFU | 1625087 tok/s
step 8106/19560 | loss 3.453030 (-0.34z)| norm 0.2583 (-0.59z)| lr 3.99e-04 | 322.36 ms | 52.4% bf16 MFU | 1625152 tok/s
step 8107/19560 | loss 3.430432 (-0.96z)| norm 0.2748 (-0.22z)| lr 3.99e-04 | 322.14 ms | 52.4% bf16 MFU | 1625270 tok/s
step 8108/19560 | loss 3.408405 (-1.54z)| norm 0.2630 (-0.47z)| lr 3.99e-04 | 322.88 ms | 52.3% bf16 MFU | 1625196 tok/s
step 8109/19560 | loss 3.460008 (-0.12z)| norm 0.2592 (-0.55z)| lr 3.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1625217 tok/s
step 8110/19560 | loss 3.499227 (+0.95z)| norm 0.2607 (-0.51z)| lr 3.99e-04 | 322.76 ms | 52.3% bf16 MFU | 1625176 tok/s
step 8111/19560 | loss 3.460575 (-0.11z)| norm 0.2670 (-0.37z)| lr 3.99e-04 | 323.03 ms | 52.2% bf16 MFU | 1625068 tok/s
step 8112/19560 | loss 3.452050 (-0.34z)| norm 0.2769 (-0.15z)| lr 3.99e-04 | 322.11 ms | 52.4% bf16 MFU | 1625198 tok/s
step 8113/19560 | loss 3.419296 (-1.23z)| norm 0.2544 (-0.64z)| lr 3.99e-04 | 322.40 ms | 52.3% bf16 MFU | 1625249 tok/s
step 8114/19560 | loss 3.408864 (-1.50z)| norm 0.2602 (-0.52z)| lr 3.99e-04 | 322.96 ms | 52.3% bf16 MFU | 1625157 tok/s
step 8115/19560 | loss 3.429491 (-0.91z)| norm 0.2940 (+0.23z)| lr 3.99e-04 | 322.63 ms | 52.3% bf16 MFU | 1625152 tok/s
step 8116/19560 | loss 3.476048 (+0.38z)| norm 0.2634 (-0.44z)| lr 3.99e-04 | 322.40 ms | 52.3% bf16 MFU | 1625205 tok/s
step 8117/19560 | loss 3.494967 (+0.90z)| norm 0.2663 (-0.38z)| lr 3.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1625223 tok/s
step 8118/19560 | loss 3.474776 (+0.34z)| norm 0.2964 (+0.28z)| lr 3.99e-04 | 322.77 ms | 52.3% bf16 MFU | 1625180 tok/s
step 8119/19560 | loss 3.391868 (-1.93z)| norm 0.2607 (-0.49z)| lr 3.99e-04 | 322.35 ms | 52.4% bf16 MFU | 1625243 tok/s
step 8120/19560 | loss 3.472376 (+0.26z)| norm 0.2917 (+0.19z)| lr 3.99e-04 | 322.66 ms | 52.3% bf16 MFU | 1625225 tok/s
step 8121/19560 | loss 3.408832 (-1.47z)| norm 0.2925 (+0.21z)| lr 3.99e-04 | 322.50 ms | 52.3% bf16 MFU | 1625250 tok/s
step 8122/19560 | loss 3.384706 (-2.08z)| norm 0.2704 (-0.28z)| lr 3.99e-04 | 322.44 ms | 52.3% bf16 MFU | 1625286 tok/s
step 8123/19560 | loss 3.538389 (+2.02z)| norm 0.2737 (-0.20z)| lr 3.98e-04 | 322.44 ms | 52.3% bf16 MFU | 1625322 tok/s
step 8124/19560 | loss 3.459145 (-0.09z)| norm 0.2971 (+0.31z)| lr 3.98e-04 | 322.30 ms | 52.4% bf16 MFU | 1625391 tok/s
step 8125/19560 | loss 3.507561 (+1.20z)| norm 0.2922 (+0.20z)| lr 3.98e-04 | 323.16 ms | 52.2% bf16 MFU | 1625241 tok/s
step 8126/19560 | loss 3.483025 (+0.55z)| norm 0.2717 (-0.26z)| lr 3.98e-04 | 322.63 ms | 52.3% bf16 MFU | 1625232 tok/s
step 8127/19560 | loss 3.411466 (-1.33z)| norm 0.3341 (+1.11z)| lr 3.98e-04 | 322.50 ms | 52.3% bf16 MFU | 1625256 tok/s
step 8128/19560 | loss 3.413821 (-1.25z)| norm 0.2660 (-0.40z)| lr 3.98e-04 | 323.07 ms | 52.2% bf16 MFU | 1625136 tok/s
step 8129/19560 | loss 3.398829 (-1.62z)| norm 0.2733 (-0.24z)| lr 3.98e-04 | 322.15 ms | 52.4% bf16 MFU | 1625251 tok/s
step 8130/19560 | loss 3.438719 (-0.56z)| norm 0.2654 (-0.41z)| lr 3.98e-04 | 322.72 ms | 52.3% bf16 MFU | 1625217 tok/s
step 8131/19560 | loss 3.432166 (-0.73z)| norm 0.2683 (-0.34z)| lr 3.98e-04 | 322.61 ms | 52.3% bf16 MFU | 1625214 tok/s
step 8132/19560 | loss 3.470989 (+0.30z)| norm 0.2634 (-0.45z)| lr 3.98e-04 | 322.60 ms | 52.3% bf16 MFU | 1625213 tok/s
step 8133/19560 | loss 3.531966 (+1.92z)| norm 0.2585 (-0.55z)| lr 3.98e-04 | 322.78 ms | 52.3% bf16 MFU | 1625166 tok/s
step 8134/19560 | loss 3.494095 (+0.92z)| norm 0.2941 (+0.24z)| lr 3.98e-04 | 321.93 ms | 52.4% bf16 MFU | 1625338 tok/s
step 8135/19560 | loss 3.429931 (-0.78z)| norm 0.2725 (-0.24z)| lr 3.98e-04 | 322.60 ms | 52.3% bf16 MFU | 1625330 tok/s
step 8136/19560 | loss 3.456835 (-0.06z)| norm 0.2811 (-0.06z)| lr 3.98e-04 | 322.85 ms | 52.3% bf16 MFU | 1625260 tok/s
step 8137/19560 | loss 3.473119 (+0.37z)| norm 0.2534 (-0.66z)| lr 3.98e-04 | 322.40 ms | 52.3% bf16 MFU | 1625308 tok/s
step 8138/19560 | loss 3.438217 (-0.58z)| norm 0.2815 (-0.05z)| lr 3.98e-04 | 322.45 ms | 52.3% bf16 MFU | 1625340 tok/s
step 8139/19560 | loss 3.388414 (-1.88z)| norm 0.2697 (-0.31z)| lr 3.98e-04 | 322.28 ms | 52.4% bf16 MFU | 1625412 tok/s
step 8140/19560 | loss 3.469594 (+0.28z)| norm 0.2541 (-0.65z)| lr 3.98e-04 | 322.69 ms | 52.3% bf16 MFU | 1625380 tok/s
step 8141/19560 | loss 3.410595 (-1.28z)| norm 0.2896 (+0.14z)| lr 3.98e-04 | 322.87 ms | 52.3% bf16 MFU | 1625302 tok/s
step 8142/19560 | loss 3.521964 (+1.69z)| norm 0.2739 (-0.21z)| lr 3.98e-04 | 322.83 ms | 52.3% bf16 MFU | 1625238 tok/s
step 8143/19560 | loss 3.453334 (-0.14z)| norm 0.2861 (+0.06z)| lr 3.98e-04 | 322.68 ms | 52.3% bf16 MFU | 1625216 tok/s
step 8144/19560 | loss 3.545900 (+2.27z)| norm 0.3006 (+0.38z)| lr 3.97e-04 | 322.67 ms | 52.3% bf16 MFU | 1625198 tok/s
step 8145/19560 | loss 3.427080 (-0.85z)| norm 0.3305 (+1.03z)| lr 3.97e-04 | 323.02 ms | 52.2% bf16 MFU | 1625093 tok/s
step 8146/19560 | loss 3.465783 (+0.16z)| norm 0.2852 (+0.03z)| lr 3.97e-04 | 322.61 ms | 52.3% bf16 MFU | 1625096 tok/s
step 8147/19560 | loss 3.488647 (+0.77z)| norm 0.2979 (+0.31z)| lr 3.97e-04 | 322.16 ms | 52.4% bf16 MFU | 1625211 tok/s
step 8148/19560 | loss 3.486408 (+0.71z)| norm 0.2847 (+0.02z)| lr 3.97e-04 | 322.91 ms | 52.3% bf16 MFU | 1625131 tok/s
step 8149/19560 | loss 3.398932 (-1.57z)| norm 0.3099 (+0.57z)| lr 3.97e-04 | 323.21 ms | 52.2% bf16 MFU | 1624981 tok/s
step 8150/19560 | loss 3.532151 (+1.88z)| norm 0.2736 (-0.22z)| lr 3.97e-04 | 323.04 ms | 52.2% bf16 MFU | 1624881 tok/s
step 8151/19560 | loss 3.425650 (-0.87z)| norm 0.2783 (-0.12z)| lr 3.97e-04 | 322.20 ms | 52.4% bf16 MFU | 1624998 tok/s
step 8152/19560 | loss 3.444342 (-0.39z)| norm 0.3045 (+0.46z)| lr 3.97e-04 | 322.21 ms | 52.4% bf16 MFU | 1625106 tok/s
step 8153/19560 | loss 3.522467 (+1.59z)| norm 0.2843 (+0.01z)| lr 3.97e-04 | 322.65 ms | 52.3% bf16 MFU | 1625098 tok/s
step 8154/19560 | loss 3.467944 (+0.20z)| norm 0.2810 (-0.07z)| lr 3.97e-04 | 322.58 ms | 52.3% bf16 MFU | 1625107 tok/s
step 8155/19560 | loss 3.424740 (-0.91z)| norm 0.2898 (+0.17z)| lr 3.97e-04 | 322.67 ms | 52.3% bf16 MFU | 1625095 tok/s
step 8156/19560 | loss 3.465559 (+0.14z)| norm 0.2694 (-0.32z)| lr 3.97e-04 | 322.29 ms | 52.4% bf16 MFU | 1625178 tok/s
step 8157/19560 | loss 3.489519 (+0.77z)| norm 0.2864 (+0.09z)| lr 3.97e-04 | 322.38 ms | 52.4% bf16 MFU | 1625234 tok/s
step 8158/19560 | loss 3.476874 (+0.43z)| norm 0.2630 (-0.47z)| lr 3.97e-04 | 322.72 ms | 52.3% bf16 MFU | 1625202 tok/s
step 8159/19560 | loss 3.492916 (+0.84z)| norm 0.2789 (-0.09z)| lr 3.97e-04 | 322.88 ms | 52.3% bf16 MFU | 1625132 tok/s
step 8160/19560 | loss 3.445937 (-0.37z)| norm 0.2505 (-0.76z)| lr 3.97e-04 | 322.36 ms | 52.4% bf16 MFU | 1625194 tok/s
step 8161/19560 | loss 3.472691 (+0.32z)| norm 0.2669 (-0.37z)| lr 3.97e-04 | 322.16 ms | 52.4% bf16 MFU | 1625305 tok/s
step 8162/19560 | loss 3.474415 (+0.36z)| norm 0.2798 (-0.07z)| lr 3.97e-04 | 322.58 ms | 52.3% bf16 MFU | 1625305 tok/s
step 8163/19560 | loss 3.391366 (-1.76z)| norm 0.2622 (-0.48z)| lr 3.97e-04 | 322.14 ms | 52.4% bf16 MFU | 1625417 tok/s
step 8164/19560 | loss 3.436774 (-0.60z)| norm 0.2683 (-0.34z)| lr 3.97e-04 | 322.26 ms | 52.4% bf16 MFU | 1625491 tok/s
step 8165/19560 | loss 3.446352 (-0.36z)| norm 0.2812 (-0.04z)| lr 3.96e-04 | 322.72 ms | 52.3% bf16 MFU | 1625446 tok/s
step 8166/19560 | loss 3.518652 (+1.49z)| norm 0.2765 (-0.12z)| lr 3.96e-04 | 322.41 ms | 52.3% bf16 MFU | 1625482 tok/s
step 8167/19560 | loss 3.416748 (-1.11z)| norm 0.2812 (+0.01z)| lr 3.96e-04 | 322.62 ms | 52.3% bf16 MFU | 1625462 tok/s
step 8168/19560 | loss 3.456068 (-0.10z)| norm 0.2982 (+0.47z)| lr 3.96e-04 | 322.40 ms | 52.3% bf16 MFU | 1625499 tok/s
step 8169/19560 | loss 3.416970 (-1.08z)| norm 0.2514 (-0.79z)| lr 3.96e-04 | 322.46 ms | 52.3% bf16 MFU | 1625519 tok/s
step 8170/19560 | loss 3.431715 (-0.70z)| norm 0.2981 (+0.46z)| lr 3.96e-04 | 322.51 ms | 52.3% bf16 MFU | 1625526 tok/s
step 8171/19560 | loss 3.370661 (-2.24z)| norm 0.2576 (-0.63z)| lr 3.96e-04 | 322.72 ms | 52.3% bf16 MFU | 1625479 tok/s
step 8172/19560 | loss 3.565393 (+2.59z)| norm 0.2689 (-0.32z)| lr 3.96e-04 | 322.64 ms | 52.3% bf16 MFU | 1625456 tok/s
step 8173/19560 | loss 3.433549 (-0.66z)| norm 0.2745 (-0.18z)| lr 3.96e-04 | 322.42 ms | 52.3% bf16 MFU | 1625488 tok/s
step 8174/19560 | loss 3.512874 (+1.30z)| norm 0.2851 (+0.37z)| lr 3.96e-04 | 322.60 ms | 52.3% bf16 MFU | 1625473 tok/s
step 8175/19560 | loss 3.481140 (+0.51z)| norm 0.2944 (+0.83z)| lr 3.96e-04 | 322.85 ms | 52.3% bf16 MFU | 1625396 tok/s
step 8176/19560 | loss 3.446451 (-0.35z)| norm 0.2828 (+0.22z)| lr 3.96e-04 | 322.50 ms | 52.3% bf16 MFU | 1625411 tok/s
step 8177/19560 | loss 3.652543 (+4.36z)| norm 0.3201 (+2.12z)| lr 3.96e-04 | 322.42 ms | 52.3% bf16 MFU | 1625445 tok/s
step 8178/19560 | loss 3.410560 (-1.17z)| norm 0.2988 (+1.01z)| lr 3.96e-04 | 322.27 ms | 52.4% bf16 MFU | 1625515 tok/s
step 8179/19560 | loss 3.456918 (-0.11z)| norm 0.3133 (+1.72z)| lr 3.96e-04 | 322.68 ms | 52.3% bf16 MFU | 1625479 tok/s
step 8180/19560 | loss 3.492445 (+0.69z)| norm 0.2930 (+0.67z)| lr 3.96e-04 | 322.68 ms | 52.3% bf16 MFU | 1625444 tok/s
step 8181/19560 | loss 3.485914 (+0.54z)| norm 0.3188 (+1.95z)| lr 3.96e-04 | 322.64 ms | 52.3% bf16 MFU | 1625421 tok/s
step 8182/19560 | loss 3.446149 (-0.35z)| norm 0.3178 (+1.88z)| lr 3.96e-04 | 322.89 ms | 52.3% bf16 MFU | 1625337 tok/s
step 8183/19560 | loss 3.450830 (-0.24z)| norm 0.3219 (+2.03z)| lr 3.96e-04 | 322.36 ms | 52.4% bf16 MFU | 1625390 tok/s
step 8184/19560 | loss 3.406577 (-1.25z)| norm 0.2994 (+0.92z)| lr 3.96e-04 | 323.02 ms | 52.2% bf16 MFU | 1625275 tok/s
step 8185/19560 | loss 3.322572 (-3.06z)| norm 0.3046 (+1.18z)| lr 3.96e-04 | 322.91 ms | 52.3% bf16 MFU | 1625193 tok/s
step 8186/19560 | loss 3.458709 (+0.00z)| norm 0.2681 (-0.61z)| lr 3.96e-04 | 322.74 ms | 52.3% bf16 MFU | 1625157 tok/s
step 8187/19560 | loss 3.431992 (-0.60z)| norm 0.3119 (+1.52z)| lr 3.95e-04 | 322.82 ms | 52.3% bf16 MFU | 1625104 tok/s
step 8188/19560 | loss 3.531547 (+1.63z)| norm 0.2878 (+0.34z)| lr 3.95e-04 | 322.80 ms | 52.3% bf16 MFU | 1625058 tok/s
step 8189/19560 | loss 3.513242 (+1.20z)| norm 0.2823 (+0.07z)| lr 3.95e-04 | 323.26 ms | 52.2% bf16 MFU | 1624900 tok/s
step 8190/19560 | loss 3.446017 (-0.30z)| norm 0.2967 (+0.76z)| lr 3.95e-04 | 322.37 ms | 52.4% bf16 MFU | 1624972 tok/s
step 8191/19560 | loss 3.455307 (-0.09z)| norm 0.2884 (+0.37z)| lr 3.95e-04 | 322.83 ms | 52.3% bf16 MFU | 1624924 tok/s
step 8192/19560 | loss 3.471393 (+0.27z)| norm 0.2761 (-0.24z)| lr 3.95e-04 | 323.19 ms | 52.2% bf16 MFU | 1624789 tok/s
step 8193/19560 | loss 3.496039 (+0.81z)| norm 0.2995 (+0.90z)| lr 3.95e-04 | 322.83 ms | 52.3% bf16 MFU | 1624752 tok/s
step 8194/19560 | loss 3.459618 (-0.01z)| norm 0.2838 (+0.13z)| lr 3.95e-04 | 322.64 ms | 52.3% bf16 MFU | 1624765 tok/s
step 8195/19560 | loss 3.445241 (-0.32z)| norm 0.2619 (-0.94z)| lr 3.95e-04 | 323.06 ms | 52.2% bf16 MFU | 1624671 tok/s
step 8196/19560 | loss 3.451730 (-0.17z)| norm 0.2790 (-0.10z)| lr 3.95e-04 | 322.76 ms | 52.3% bf16 MFU | 1624656 tok/s
step 8197/19560 | loss 3.562030 (+2.27z)| norm 0.3020 (+1.02z)| lr 3.95e-04 | 322.50 ms | 52.3% bf16 MFU | 1624709 tok/s
step 8198/19560 | loss 3.439908 (-0.44z)| norm 0.3012 (+0.96z)| lr 3.95e-04 | 323.03 ms | 52.2% bf16 MFU | 1624625 tok/s
step 8199/19560 | loss 3.452605 (-0.16z)| norm 0.3012 (+0.95z)| lr 3.95e-04 | 323.27 ms | 52.2% bf16 MFU | 1624486 tok/s
step 8200/19560 | loss 3.391875 (-1.49z)| norm 0.3171 (+1.70z)| lr 3.95e-04 | 322.79 ms | 52.3% bf16 MFU | 1624472 tok/s
step 8201/19560 | loss 3.391084 (-1.50z)| norm 0.2809 (-0.08z)| lr 3.95e-04 | 322.45 ms | 52.3% bf16 MFU | 1624547 tok/s
step 8202/19560 | loss 3.444769 (-0.32z)| norm 0.2700 (-0.62z)| lr 3.95e-04 | 323.52 ms | 52.2% bf16 MFU | 1624350 tok/s
step 8203/19560 | loss 3.436304 (-0.50z)| norm 0.2710 (-0.59z)| lr 3.95e-04 | 323.45 ms | 52.2% bf16 MFU | 1624178 tok/s
step 8204/19560 | loss 3.407697 (-1.12z)| norm 0.2585 (-1.23z)| lr 3.95e-04 | 322.67 ms | 52.3% bf16 MFU | 1624212 tok/s
step 8205/19560 | loss 3.468642 (+0.22z)| norm 0.2573 (-1.28z)| lr 3.95e-04 | 322.84 ms | 52.3% bf16 MFU | 1624200 tok/s
step 8206/19560 | loss 3.447823 (-0.23z)| norm 0.2703 (-0.63z)| lr 3.95e-04 | 323.08 ms | 52.2% bf16 MFU | 1624128 tok/s
step 8207/19560 | loss 3.351173 (-2.32z)| norm 0.2798 (-0.17z)| lr 3.95e-04 | 322.83 ms | 52.3% bf16 MFU | 1624122 tok/s
step 8208/19560 | loss 3.575961 (+2.50z)| norm 0.2528 (-1.54z)| lr 3.94e-04 | 322.27 ms | 52.4% bf16 MFU | 1624258 tok/s
step 8209/19560 | loss 3.504668 (+0.96z)| norm 0.2839 (+0.03z)| lr 3.94e-04 | 322.85 ms | 52.3% bf16 MFU | 1624242 tok/s
step 8210/19560 | loss 3.467369 (+0.16z)| norm 0.2764 (-0.34z)| lr 3.94e-04 | 322.37 ms | 52.4% bf16 MFU | 1624348 tok/s
step 8211/19560 | loss 3.468399 (+0.17z)| norm 0.2725 (-0.54z)| lr 3.94e-04 | 323.06 ms | 52.2% bf16 MFU | 1624274 tok/s
step 8212/19560 | loss 3.470813 (+0.22z)| norm 0.2901 (+0.38z)| lr 3.94e-04 | 322.93 ms | 52.3% bf16 MFU | 1624236 tok/s
step 8213/19560 | loss 3.471918 (+0.25z)| norm 0.2830 (+0.00z)| lr 3.94e-04 | 322.90 ms | 52.3% bf16 MFU | 1624209 tok/s
step 8214/19560 | loss 3.497344 (+0.78z)| norm 0.2758 (-0.39z)| lr 3.94e-04 | 321.94 ms | 52.4% bf16 MFU | 1624426 tok/s
step 8215/19560 | loss 3.408718 (-1.11z)| norm 0.2896 (+0.34z)| lr 3.94e-04 | 322.83 ms | 52.3% bf16 MFU | 1624407 tok/s
step 8216/19560 | loss 3.374466 (-1.81z)| norm 0.2922 (+0.47z)| lr 3.94e-04 | 323.32 ms | 52.2% bf16 MFU | 1624266 tok/s
step 8217/19560 | loss 3.451830 (-0.17z)| norm 0.2970 (+0.72z)| lr 3.94e-04 | 322.80 ms | 52.3% bf16 MFU | 1624263 tok/s
step 8218/19560 | loss 3.470942 (+0.23z)| norm 0.3041 (+1.08z)| lr 3.94e-04 | 322.74 ms | 52.3% bf16 MFU | 1624275 tok/s
step 8219/19560 | loss 3.449311 (-0.23z)| norm 0.2993 (+0.81z)| lr 3.94e-04 | 322.86 ms | 52.3% bf16 MFU | 1624256 tok/s
step 8220/19560 | loss 3.442408 (-0.36z)| norm 0.3072 (+1.22z)| lr 3.94e-04 | 323.18 ms | 52.2% bf16 MFU | 1624156 tok/s
step 8221/19560 | loss 3.489245 (+0.64z)| norm 0.2815 (-0.14z)| lr 3.94e-04 | 322.35 ms | 52.4% bf16 MFU | 1624270 tok/s
step 8222/19560 | loss 3.415968 (-0.92z)| norm 0.3331 (+2.55z)| lr 3.94e-04 | 322.83 ms | 52.3% bf16 MFU | 1624260 tok/s
step 8223/19560 | loss 3.469101 (+0.22z)| norm 0.2502 (-1.78z)| lr 3.94e-04 | 323.59 ms | 52.2% bf16 MFU | 1624059 tok/s
step 8224/19560 | loss 3.477499 (+0.41z)| norm 0.2987 (+0.82z)| lr 3.94e-04 | 322.99 ms | 52.3% bf16 MFU | 1624018 tok/s
step 8225/19560 | loss 3.456656 (-0.04z)| norm 0.2444 (-2.06z)| lr 3.94e-04 | 322.81 ms | 52.3% bf16 MFU | 1624024 tok/s
step 8226/19560 | loss 3.507570 (+1.05z)| norm 0.2868 (+0.19z)| lr 3.94e-04 | 322.45 ms | 52.3% bf16 MFU | 1624120 tok/s
step 8227/19560 | loss 3.454659 (-0.09z)| norm 0.2663 (-0.88z)| lr 3.94e-04 | 322.49 ms | 52.3% bf16 MFU | 1624202 tok/s
step 8228/19560 | loss 3.473277 (+0.31z)| norm 0.2671 (-0.83z)| lr 3.94e-04 | 322.45 ms | 52.3% bf16 MFU | 1624290 tok/s
step 8229/19560 | loss 3.470618 (+0.25z)| norm 0.2510 (-1.65z)| lr 3.93e-04 | 322.48 ms | 52.3% bf16 MFU | 1624367 tok/s
step 8230/19560 | loss 3.441821 (-0.36z)| norm 0.2855 (+0.17z)| lr 3.93e-04 | 322.87 ms | 52.3% bf16 MFU | 1624340 tok/s
step 8231/19560 | loss 3.435473 (-0.49z)| norm 0.2688 (-0.71z)| lr 3.93e-04 | 322.78 ms | 52.3% bf16 MFU | 1624338 tok/s
step 8232/19560 | loss 3.468102 (+0.23z)| norm 0.2689 (-0.69z)| lr 3.93e-04 | 322.42 ms | 52.3% bf16 MFU | 1624426 tok/s
step 8233/19560 | loss 3.353908 (-2.19z)| norm 0.2570 (-1.30z)| lr 3.93e-04 | 322.95 ms | 52.3% bf16 MFU | 1624376 tok/s
step 8234/19560 | loss 3.504952 (+1.01z)| norm 0.2643 (-0.91z)| lr 3.93e-04 | 322.75 ms | 52.3% bf16 MFU | 1624380 tok/s
step 8235/19560 | loss 3.433596 (-0.50z)| norm 0.2571 (-1.28z)| lr 3.93e-04 | 322.50 ms | 52.3% bf16 MFU | 1624445 tok/s
step 8236/19560 | loss 3.392683 (-1.36z)| norm 0.2586 (-1.20z)| lr 3.93e-04 | 322.59 ms | 52.3% bf16 MFU | 1624484 tok/s
step 8237/19560 | loss 3.420137 (-0.77z)| norm 0.2714 (-0.54z)| lr 3.93e-04 | 322.61 ms | 52.3% bf16 MFU | 1624516 tok/s
step 8238/19560 | loss 3.458055 (+0.03z)| norm 0.2680 (-0.72z)| lr 3.93e-04 | 323.51 ms | 52.2% bf16 MFU | 1624323 tok/s
step 8239/19560 | loss 3.473243 (+0.35z)| norm 0.2574 (-1.27z)| lr 3.93e-04 | 322.92 ms | 52.3% bf16 MFU | 1624287 tok/s
step 8240/19560 | loss 3.442923 (-0.29z)| norm 0.2868 (+0.27z)| lr 3.93e-04 | 323.22 ms | 52.2% bf16 MFU | 1624176 tok/s
step 8241/19560 | loss 3.415781 (-0.86z)| norm 0.2550 (-1.40z)| lr 3.93e-04 | 323.36 ms | 52.2% bf16 MFU | 1624036 tok/s
step 8242/19560 | loss 3.526121 (+1.45z)| norm 0.2962 (+0.75z)| lr 3.93e-04 | 322.48 ms | 52.3% bf16 MFU | 1624124 tok/s
step 8243/19560 | loss 3.472931 (+0.32z)| norm 0.2672 (-0.77z)| lr 3.93e-04 | 322.70 ms | 52.3% bf16 MFU | 1624152 tok/s
step 8244/19560 | loss 3.436982 (-0.43z)| norm 0.2940 (+0.64z)| lr 3.93e-04 | 323.09 ms | 52.2% bf16 MFU | 1624080 tok/s
step 8245/19560 | loss 3.447840 (-0.19z)| norm 0.3418 (+3.03z)| lr 3.93e-04 | 322.15 ms | 52.4% bf16 MFU | 1624250 tok/s
step 8246/19560 | loss 3.453453 (-0.07z)| norm 0.2846 (+0.11z)| lr 3.93e-04 | 323.28 ms | 52.2% bf16 MFU | 1624126 tok/s
step 8247/19560 | loss 3.415910 (-0.87z)| norm 0.3131 (+1.54z)| lr 3.93e-04 | 322.86 ms | 52.3% bf16 MFU | 1624115 tok/s
step 8248/19560 | loss 3.430942 (-0.55z)| norm 0.3371 (+2.67z)| lr 3.93e-04 | 322.34 ms | 52.4% bf16 MFU | 1624234 tok/s
step 8249/19560 | loss 3.489754 (+0.69z)| norm 0.2824 (-0.03z)| lr 3.93e-04 | 322.94 ms | 52.3% bf16 MFU | 1624196 tok/s
step 8250/19560 | loss 3.484787 (+0.57z)| norm 0.3119 (+1.40z)| lr 3.92e-04 | 322.46 ms | 52.3% bf16 MFU | 1624281 tok/s
val loss 3.443434
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2853/10042 = 0.284107
step 8251/19560 | loss 3.441388 (-0.35z)| norm 0.2853 (+0.09z)| lr 3.92e-04 | 322.24 ms | 52.4% bf16 MFU | 1624418 tok/s
step 8252/19560 | loss 3.451162 (-0.13z)| norm 0.2757 (-0.38z)| lr 3.92e-04 | 322.07 ms | 52.4% bf16 MFU | 1624590 tok/s
step 8253/19560 | loss 3.435404 (-0.46z)| norm 0.2975 (+0.69z)| lr 3.92e-04 | 322.92 ms | 52.3% bf16 MFU | 1624540 tok/s
step 8254/19560 | loss 3.493716 (+0.80z)| norm 0.2902 (+0.33z)| lr 3.92e-04 | 321.77 ms | 52.5% bf16 MFU | 1624781 tok/s
step 8255/19560 | loss 3.476844 (+0.42z)| norm 0.2902 (+0.35z)| lr 3.92e-04 | 322.44 ms | 52.3% bf16 MFU | 1624842 tok/s
step 8256/19560 | loss 3.433516 (-0.52z)| norm 0.2866 (+0.16z)| lr 3.92e-04 | 322.33 ms | 52.4% bf16 MFU | 1624928 tok/s
step 8257/19560 | loss 3.427677 (-0.66z)| norm 0.2818 (-0.08z)| lr 3.92e-04 | 322.90 ms | 52.3% bf16 MFU | 1624867 tok/s
step 8258/19560 | loss 3.479217 (+0.46z)| norm 0.2582 (-1.27z)| lr 3.92e-04 | 322.49 ms | 52.3% bf16 MFU | 1624911 tok/s
step 8259/19560 | loss 3.450292 (-0.17z)| norm 0.3237 (+1.99z)| lr 3.92e-04 | 322.66 ms | 52.3% bf16 MFU | 1624911 tok/s
step 8260/19560 | loss 3.436181 (-0.48z)| norm 0.2526 (-1.55z)| lr 3.92e-04 | 321.68 ms | 52.5% bf16 MFU | 1625156 tok/s
step 8261/19560 | loss 3.496125 (+0.85z)| norm 0.2574 (-1.30z)| lr 3.92e-04 | 322.73 ms | 52.3% bf16 MFU | 1625125 tok/s
step 8262/19560 | loss 3.564317 (+2.30z)| norm 0.2528 (-1.51z)| lr 3.92e-04 | 322.48 ms | 52.3% bf16 MFU | 1625158 tok/s
step 8263/19560 | loss 3.425432 (-0.71z)| norm 0.2811 (-0.12z)| lr 3.92e-04 | 322.17 ms | 52.4% bf16 MFU | 1625269 tok/s
step 8264/19560 | loss 3.445041 (-0.28z)| norm 0.2653 (-0.88z)| lr 3.92e-04 | 322.89 ms | 52.3% bf16 MFU | 1625192 tok/s
step 8265/19560 | loss 3.466116 (+0.18z)| norm 0.2772 (-0.31z)| lr 3.92e-04 | 322.44 ms | 52.3% bf16 MFU | 1625233 tok/s
step 8266/19560 | loss 3.430400 (-0.60z)| norm 0.2539 (-1.44z)| lr 3.92e-04 | 322.67 ms | 52.3% bf16 MFU | 1625214 tok/s
step 8267/19560 | loss 3.488580 (+0.65z)| norm 0.2647 (-0.91z)| lr 3.92e-04 | 322.56 ms | 52.3% bf16 MFU | 1625224 tok/s
step 8268/19560 | loss 3.459253 (+0.01z)| norm 0.2640 (-0.95z)| lr 3.92e-04 | 322.51 ms | 52.3% bf16 MFU | 1625246 tok/s
step 8269/19560 | loss 3.464638 (+0.12z)| norm 0.2582 (-1.22z)| lr 3.92e-04 | 322.36 ms | 52.4% bf16 MFU | 1625305 tok/s
step 8270/19560 | loss 3.457935 (-0.01z)| norm 0.2967 (+0.66z)| lr 3.92e-04 | 322.28 ms | 52.4% bf16 MFU | 1625380 tok/s
step 8271/19560 | loss 3.448553 (-0.22z)| norm 0.2823 (-0.05z)| lr 3.91e-04 | 322.88 ms | 52.3% bf16 MFU | 1625301 tok/s
step 8272/19560 | loss 3.475574 (+0.39z)| norm 0.2959 (+0.62z)| lr 3.91e-04 | 322.62 ms | 52.3% bf16 MFU | 1625292 tok/s
step 8273/19560 | loss 3.433386 (-0.55z)| norm 0.2920 (+0.46z)| lr 3.91e-04 | 322.46 ms | 52.3% bf16 MFU | 1625322 tok/s
step 8274/19560 | loss 3.461739 (+0.08z)| norm 0.2734 (-0.47z)| lr 3.91e-04 | 322.69 ms | 52.3% bf16 MFU | 1625292 tok/s
step 8275/19560 | loss 3.538874 (+1.79z)| norm 0.2931 (+0.52z)| lr 3.91e-04 | 322.80 ms | 52.3% bf16 MFU | 1625236 tok/s
step 8276/19560 | loss 3.419496 (-0.85z)| norm 0.2729 (-0.49z)| lr 3.91e-04 | 322.41 ms | 52.3% bf16 MFU | 1625281 tok/s
step 8277/19560 | loss 3.518191 (+1.32z)| norm 0.2860 (+0.18z)| lr 3.91e-04 | 322.55 ms | 52.3% bf16 MFU | 1625289 tok/s
step 8278/19560 | loss 3.457811 (-0.01z)| norm 0.2856 (+0.15z)| lr 3.91e-04 | 322.08 ms | 52.4% bf16 MFU | 1625414 tok/s
step 8279/19560 | loss 3.435988 (-0.50z)| norm 0.2908 (+0.41z)| lr 3.91e-04 | 322.19 ms | 52.4% bf16 MFU | 1625507 tok/s
step 8280/19560 | loss 3.493035 (+0.77z)| norm 0.2942 (+0.58z)| lr 3.91e-04 | 322.31 ms | 52.4% bf16 MFU | 1625565 tok/s
step 8281/19560 | loss 3.469747 (+0.26z)| norm 0.3451 (+3.02z)| lr 3.91e-04 | 322.68 ms | 52.3% bf16 MFU | 1625525 tok/s
step 8282/19560 | loss 3.453058 (-0.11z)| norm 0.3160 (+1.57z)| lr 3.91e-04 | 322.37 ms | 52.4% bf16 MFU | 1625567 tok/s
step 8283/19560 | loss 3.435590 (-0.51z)| norm 0.2724 (-0.52z)| lr 3.91e-04 | 322.19 ms | 52.4% bf16 MFU | 1625652 tok/s
step 8284/19560 | loss 3.427620 (-0.68z)| norm 0.3598 (+3.49z)| lr 3.91e-04 | 322.71 ms | 52.3% bf16 MFU | 1625602 tok/s
step 8285/19560 | loss 3.425292 (-0.72z)| norm 0.2887 (+0.22z)| lr 3.91e-04 | 322.55 ms | 52.3% bf16 MFU | 1625595 tok/s
step 8286/19560 | loss 3.404144 (-1.18z)| norm 0.2773 (-0.31z)| lr 3.91e-04 | 322.85 ms | 52.3% bf16 MFU | 1625513 tok/s
step 8287/19560 | loss 3.492144 (+0.79z)| norm 0.2990 (+0.68z)| lr 3.91e-04 | 322.83 ms | 52.3% bf16 MFU | 1625438 tok/s
step 8288/19560 | loss 3.394749 (-1.38z)| norm 0.2680 (-0.76z)| lr 3.91e-04 | 322.40 ms | 52.3% bf16 MFU | 1625478 tok/s
step 8289/19560 | loss 3.418315 (-0.84z)| norm 0.2821 (-0.11z)| lr 3.91e-04 | 322.36 ms | 52.4% bf16 MFU | 1625524 tok/s
step 8290/19560 | loss 3.464115 (+0.18z)| norm 0.2726 (-0.55z)| lr 3.91e-04 | 322.57 ms | 52.3% bf16 MFU | 1625515 tok/s
step 8291/19560 | loss 3.432455 (-0.53z)| norm 0.2758 (-0.41z)| lr 3.91e-04 | 322.47 ms | 52.3% bf16 MFU | 1625532 tok/s
step 8292/19560 | loss 3.466100 (+0.21z)| norm 0.2679 (-0.77z)| lr 3.90e-04 | 322.11 ms | 52.4% bf16 MFU | 1625640 tok/s
step 8293/19560 | loss 3.453865 (-0.06z)| norm 0.2797 (-0.22z)| lr 3.90e-04 | 322.65 ms | 52.3% bf16 MFU | 1625606 tok/s
step 8294/19560 | loss 3.436904 (-0.43z)| norm 0.2667 (-0.82z)| lr 3.90e-04 | 322.19 ms | 52.4% bf16 MFU | 1625688 tok/s
step 8295/19560 | loss 3.431001 (-0.57z)| norm 0.2851 (+0.03z)| lr 3.90e-04 | 322.57 ms | 52.3% bf16 MFU | 1625671 tok/s
step 8296/19560 | loss 3.442141 (-0.31z)| norm 0.2752 (-0.42z)| lr 3.90e-04 | 323.22 ms | 52.2% bf16 MFU | 1625492 tok/s
step 8297/19560 | loss 3.424779 (-0.71z)| norm 0.2695 (-0.70z)| lr 3.90e-04 | 322.73 ms | 52.3% bf16 MFU | 1625444 tok/s
step 8298/19560 | loss 3.373734 (-1.83z)| norm 0.2448 (-1.81z)| lr 3.90e-04 | 322.25 ms | 52.4% bf16 MFU | 1625521 tok/s
step 8299/19560 | loss 3.451361 (-0.11z)| norm 0.2745 (-0.45z)| lr 3.90e-04 | 322.64 ms | 52.3% bf16 MFU | 1625494 tok/s
step 8300/19560 | loss 3.366256 (-2.02z)| norm 0.3837 (+4.26z)| lr 3.90e-04 | 322.50 ms | 52.3% bf16 MFU | 1625503 tok/s
step 8301/19560 | loss 3.446693 (-0.18z)| norm 0.3164 (+1.33z)| lr 3.90e-04 | 323.06 ms | 52.2% bf16 MFU | 1625373 tok/s
step 8302/19560 | loss 3.451704 (-0.06z)| norm 0.2883 (+0.12z)| lr 3.90e-04 | 322.63 ms | 52.3% bf16 MFU | 1625356 tok/s
step 8303/19560 | loss 3.488733 (+0.79z)| norm 0.3006 (+0.65z)| lr 3.90e-04 | 322.57 ms | 52.3% bf16 MFU | 1625356 tok/s
step 8304/19560 | loss 3.447015 (-0.17z)| norm 0.2805 (-0.21z)| lr 3.90e-04 | 322.62 ms | 52.3% bf16 MFU | 1625344 tok/s
step 8305/19560 | loss 3.460317 (+0.19z)| norm 0.2831 (-0.09z)| lr 3.90e-04 | 322.54 ms | 52.3% bf16 MFU | 1625350 tok/s
step 8306/19560 | loss 3.548315 (+2.34z)| norm 0.2760 (-0.39z)| lr 3.90e-04 | 322.47 ms | 52.3% bf16 MFU | 1625375 tok/s
step 8307/19560 | loss 3.443582 (-0.25z)| norm 0.2913 (+0.28z)| lr 3.90e-04 | 322.80 ms | 52.3% bf16 MFU | 1625315 tok/s
step 8308/19560 | loss 3.423398 (-0.74z)| norm 0.2818 (-0.12z)| lr 3.90e-04 | 322.53 ms | 52.3% bf16 MFU | 1625326 tok/s
step 8309/19560 | loss 3.495597 (+1.05z)| norm 0.2649 (-0.85z)| lr 3.90e-04 | 322.57 ms | 52.3% bf16 MFU | 1625327 tok/s
step 8310/19560 | loss 3.424668 (-0.70z)| norm 0.3143 (+1.32z)| lr 3.90e-04 | 323.02 ms | 52.2% bf16 MFU | 1625216 tok/s
step 8311/19560 | loss 3.451561 (-0.04z)| norm 0.2890 (+0.22z)| lr 3.90e-04 | 323.30 ms | 52.2% bf16 MFU | 1625038 tok/s
step 8312/19560 | loss 3.447221 (-0.15z)| norm 0.2694 (-0.64z)| lr 3.90e-04 | 321.93 ms | 52.4% bf16 MFU | 1625215 tok/s
step 8313/19560 | loss 3.423456 (-0.80z)| norm 0.3266 (+1.87z)| lr 3.89e-04 | 322.64 ms | 52.3% bf16 MFU | 1625204 tok/s
step 8314/19560 | loss 3.417074 (-0.95z)| norm 0.2729 (-0.48z)| lr 3.89e-04 | 322.64 ms | 52.3% bf16 MFU | 1625193 tok/s
step 8315/19560 | loss 3.401819 (-1.33z)| norm 0.2853 (+0.07z)| lr 3.89e-04 | 322.42 ms | 52.3% bf16 MFU | 1625239 tok/s
step 8316/19560 | loss 3.541657 (+2.24z)| norm 0.2748 (-0.39z)| lr 3.89e-04 | 322.27 ms | 52.4% bf16 MFU | 1625319 tok/s
step 8317/19560 | loss 3.423695 (-0.75z)| norm 0.2877 (+0.18z)| lr 3.89e-04 | 322.45 ms | 52.3% bf16 MFU | 1625350 tok/s
step 8318/19560 | loss 3.485181 (+0.81z)| norm 0.2659 (-0.77z)| lr 3.89e-04 | 322.61 ms | 52.3% bf16 MFU | 1625340 tok/s
step 8319/19560 | loss 3.430441 (-0.58z)| norm 0.2594 (-1.05z)| lr 3.89e-04 | 323.41 ms | 52.2% bf16 MFU | 1625130 tok/s
step 8320/19560 | loss 3.460008 (+0.18z)| norm 0.2706 (-0.55z)| lr 3.89e-04 | 322.69 ms | 52.3% bf16 MFU | 1625111 tok/s
step 8321/19560 | loss 3.464454 (+0.30z)| norm 0.2633 (-0.86z)| lr 3.89e-04 | 322.24 ms | 52.4% bf16 MFU | 1625206 tok/s
step 8322/19560 | loss 3.421218 (-0.80z)| norm 0.2634 (-0.84z)| lr 3.89e-04 | 322.46 ms | 52.3% bf16 MFU | 1625240 tok/s
step 8323/19560 | loss 3.417117 (-0.90z)| norm 0.2666 (-0.71z)| lr 3.89e-04 | 322.66 ms | 52.3% bf16 MFU | 1625224 tok/s
step 8324/19560 | loss 3.481019 (+0.73z)| norm 0.2595 (-1.01z)| lr 3.89e-04 | 322.16 ms | 52.4% bf16 MFU | 1625332 tok/s
step 8325/19560 | loss 3.426050 (-0.67z)| norm 0.2570 (-1.10z)| lr 3.89e-04 | 322.78 ms | 52.3% bf16 MFU | 1625280 tok/s
step 8326/19560 | loss 3.422550 (-0.76z)| norm 0.2497 (-1.39z)| lr 3.89e-04 | 322.48 ms | 52.3% bf16 MFU | 1625306 tok/s
step 8327/19560 | loss 3.470056 (+0.49z)| norm 0.2734 (-0.36z)| lr 3.89e-04 | 322.45 ms | 52.3% bf16 MFU | 1625338 tok/s
step 8328/19560 | loss 3.429636 (-0.59z)| norm 0.2719 (-0.41z)| lr 3.89e-04 | 322.17 ms | 52.4% bf16 MFU | 1625438 tok/s
step 8329/19560 | loss 3.360088 (-2.39z)| norm 0.2865 (+0.23z)| lr 3.89e-04 | 323.33 ms | 52.2% bf16 MFU | 1625242 tok/s
step 8330/19560 | loss 3.481439 (+0.77z)| norm 0.2852 (+0.16z)| lr 3.89e-04 | 323.14 ms | 52.2% bf16 MFU | 1625104 tok/s
step 8331/19560 | loss 3.462125 (+0.26z)| norm 0.2800 (-0.07z)| lr 3.89e-04 | 322.78 ms | 52.3% bf16 MFU | 1625063 tok/s
step 8332/19560 | loss 3.455955 (+0.09z)| norm 0.2711 (-0.46z)| lr 3.89e-04 | 322.27 ms | 52.4% bf16 MFU | 1625152 tok/s
step 8333/19560 | loss 3.474870 (+0.59z)| norm 0.2888 (+0.31z)| lr 3.89e-04 | 322.22 ms | 52.4% bf16 MFU | 1625250 tok/s
step 8334/19560 | loss 3.387517 (-1.67z)| norm 0.2887 (+0.30z)| lr 3.88e-04 | 322.71 ms | 52.3% bf16 MFU | 1625219 tok/s
step 8335/19560 | loss 3.455197 (+0.06z)| norm 0.2809 (-0.05z)| lr 3.88e-04 | 323.22 ms | 52.2% bf16 MFU | 1625062 tok/s
step 8336/19560 | loss 3.446591 (-0.14z)| norm 0.2532 (-1.27z)| lr 3.88e-04 | 322.78 ms | 52.3% bf16 MFU | 1625022 tok/s
step 8337/19560 | loss 3.460354 (+0.25z)| norm 0.2634 (-0.81z)| lr 3.88e-04 | 322.01 ms | 52.4% bf16 MFU | 1625179 tok/s
step 8338/19560 | loss 3.473516 (+0.62z)| norm 0.2571 (-1.08z)| lr 3.88e-04 | 323.09 ms | 52.2% bf16 MFU | 1625057 tok/s
step 8339/19560 | loss 3.478887 (+0.76z)| norm 0.2641 (-0.77z)| lr 3.88e-04 | 323.02 ms | 52.2% bf16 MFU | 1624959 tok/s
step 8340/19560 | loss 3.432971 (-0.51z)| norm 0.2811 (-0.02z)| lr 3.88e-04 | 322.58 ms | 52.3% bf16 MFU | 1624975 tok/s
step 8341/19560 | loss 3.457366 (+0.17z)| norm 0.2757 (-0.25z)| lr 3.88e-04 | 322.30 ms | 52.4% bf16 MFU | 1625061 tok/s
step 8342/19560 | loss 3.459638 (+0.25z)| norm 0.2827 (+0.05z)| lr 3.88e-04 | 322.82 ms | 52.3% bf16 MFU | 1625012 tok/s
step 8343/19560 | loss 3.489995 (+1.09z)| norm 0.2728 (-0.38z)| lr 3.88e-04 | 322.65 ms | 52.3% bf16 MFU | 1625008 tok/s
step 8344/19560 | loss 3.508508 (+1.59z)| norm 0.2579 (-1.02z)| lr 3.88e-04 | 322.26 ms | 52.4% bf16 MFU | 1625102 tok/s
step 8345/19560 | loss 3.443472 (-0.26z)| norm 0.2896 (+0.37z)| lr 3.88e-04 | 322.76 ms | 52.3% bf16 MFU | 1625066 tok/s
step 8346/19560 | loss 3.435963 (-0.46z)| norm 0.2652 (-0.68z)| lr 3.88e-04 | 323.27 ms | 52.2% bf16 MFU | 1624904 tok/s
step 8347/19560 | loss 3.522331 (+1.95z)| norm 0.3056 (+1.08z)| lr 3.88e-04 | 322.66 ms | 52.3% bf16 MFU | 1624904 tok/s
step 8348/19560 | loss 3.464010 (+0.31z)| norm 0.2734 (-0.32z)| lr 3.88e-04 | 322.54 ms | 52.3% bf16 MFU | 1624934 tok/s
step 8349/19560 | loss 3.549232 (+2.63z)| norm 0.2669 (-0.60z)| lr 3.88e-04 | 322.22 ms | 52.4% bf16 MFU | 1625042 tok/s
step 8350/19560 | loss 3.453987 (+0.01z)| norm 0.2788 (-0.05z)| lr 3.88e-04 | 322.66 ms | 52.3% bf16 MFU | 1625034 tok/s
step 8351/19560 | loss 3.399072 (-1.48z)| norm 0.2821 (+0.08z)| lr 3.88e-04 | 322.77 ms | 52.3% bf16 MFU | 1625000 tok/s
step 8352/19560 | loss 3.499687 (+1.26z)| norm 0.2684 (-0.53z)| lr 3.88e-04 | 322.46 ms | 52.3% bf16 MFU | 1625046 tok/s
step 8353/19560 | loss 3.460122 (+0.18z)| norm 0.3415 (+2.70z)| lr 3.88e-04 | 322.61 ms | 52.3% bf16 MFU | 1625050 tok/s
step 8354/19560 | loss 3.436177 (-0.46z)| norm 0.3797 (+4.07z)| lr 3.88e-04 | 322.35 ms | 52.4% bf16 MFU | 1625119 tok/s
step 8355/19560 | loss 3.442657 (-0.28z)| norm 0.3004 (+0.77z)| lr 3.87e-04 | 322.70 ms | 52.3% bf16 MFU | 1625098 tok/s
step 8356/19560 | loss 3.446002 (-0.18z)| norm 0.3599 (+3.09z)| lr 3.87e-04 | 322.87 ms | 52.3% bf16 MFU | 1625034 tok/s
step 8357/19560 | loss 3.438862 (-0.37z)| norm 0.3052 (+0.89z)| lr 3.87e-04 | 322.94 ms | 52.3% bf16 MFU | 1624956 tok/s
step 8358/19560 | loss 3.428980 (-0.63z)| norm 0.3066 (+0.93z)| lr 3.87e-04 | 322.35 ms | 52.4% bf16 MFU | 1625030 tok/s
step 8359/19560 | loss 3.458006 (+0.16z)| norm 0.3317 (+1.89z)| lr 3.87e-04 | 322.88 ms | 52.3% bf16 MFU | 1624968 tok/s
step 8360/19560 | loss 3.473414 (+0.58z)| norm 0.2978 (+0.55z)| lr 3.87e-04 | 322.36 ms | 52.4% bf16 MFU | 1625041 tok/s
step 8361/19560 | loss 3.446155 (-0.20z)| norm 0.3264 (+1.64z)| lr 3.87e-04 | 323.12 ms | 52.2% bf16 MFU | 1624918 tok/s
step 8362/19560 | loss 3.429260 (-0.66z)| norm 0.2723 (-0.48z)| lr 3.87e-04 | 322.39 ms | 52.4% bf16 MFU | 1624985 tok/s
step 8363/19560 | loss 3.558977 (+2.90z)| norm 0.2869 (+0.09z)| lr 3.87e-04 | 322.62 ms | 52.3% bf16 MFU | 1624991 tok/s
step 8364/19560 | loss 3.424964 (-0.80z)| norm 0.2973 (+0.49z)| lr 3.87e-04 | 322.88 ms | 52.3% bf16 MFU | 1624930 tok/s
step 8365/19560 | loss 3.440032 (-0.39z)| norm 0.2777 (-0.29z)| lr 3.87e-04 | 322.48 ms | 52.3% bf16 MFU | 1624974 tok/s
step 8366/19560 | loss 3.442032 (-0.33z)| norm 0.2970 (+0.46z)| lr 3.87e-04 | 322.94 ms | 52.3% bf16 MFU | 1624900 tok/s
step 8367/19560 | loss 3.459357 (+0.16z)| norm 0.2823 (-0.13z)| lr 3.87e-04 | 322.97 ms | 52.3% bf16 MFU | 1624821 tok/s
step 8368/19560 | loss 3.523827 (+1.91z)| norm 0.2950 (+0.38z)| lr 3.87e-04 | 322.53 ms | 52.3% bf16 MFU | 1624858 tok/s
step 8369/19560 | loss 3.507865 (+1.44z)| norm 0.2891 (+0.13z)| lr 3.87e-04 | 322.55 ms | 52.3% bf16 MFU | 1624886 tok/s
step 8370/19560 | loss 3.432399 (-0.61z)| norm 0.2716 (-0.56z)| lr 3.87e-04 | 322.31 ms | 52.4% bf16 MFU | 1624975 tok/s
step 8371/19560 | loss 3.550349 (+2.58z)| norm 0.2833 (-0.10z)| lr 3.87e-04 | 322.50 ms | 52.3% bf16 MFU | 1625012 tok/s
step 8372/19560 | loss 3.495560 (+1.08z)| norm 0.2737 (-0.48z)| lr 3.87e-04 | 322.87 ms | 52.3% bf16 MFU | 1624954 tok/s
step 8373/19560 | loss 3.441838 (-0.36z)| norm 0.2707 (-0.58z)| lr 3.87e-04 | 322.46 ms | 52.3% bf16 MFU | 1625002 tok/s
step 8374/19560 | loss 3.450577 (-0.13z)| norm 0.3016 (+0.67z)| lr 3.87e-04 | 322.89 ms | 52.3% bf16 MFU | 1624939 tok/s
step 8375/19560 | loss 3.460181 (+0.12z)| norm 0.3008 (+0.64z)| lr 3.87e-04 | 322.97 ms | 52.3% bf16 MFU | 1624860 tok/s
step 8376/19560 | loss 3.440199 (-0.42z)| norm 0.2842 (-0.02z)| lr 3.86e-04 | 322.83 ms | 52.3% bf16 MFU | 1624819 tok/s
step 8377/19560 | loss 3.453549 (-0.05z)| norm 0.3026 (+0.74z)| lr 3.86e-04 | 323.32 ms | 52.2% bf16 MFU | 1624657 tok/s
step 8378/19560 | loss 3.417519 (-1.01z)| norm 0.2948 (+0.42z)| lr 3.86e-04 | 322.31 ms | 52.4% bf16 MFU | 1624756 tok/s
step 8379/19560 | loss 3.423790 (-0.84z)| norm 0.2538 (-1.27z)| lr 3.86e-04 | 323.22 ms | 52.2% bf16 MFU | 1624622 tok/s
step 8380/19560 | loss 3.426159 (-0.76z)| norm 0.2910 (+0.27z)| lr 3.86e-04 | 323.22 ms | 52.2% bf16 MFU | 1624494 tok/s
step 8381/19560 | loss 3.455189 (+0.01z)| norm 0.2755 (-0.37z)| lr 3.86e-04 | 322.78 ms | 52.3% bf16 MFU | 1624485 tok/s
step 8382/19560 | loss 3.450104 (-0.12z)| norm 0.2601 (-0.99z)| lr 3.86e-04 | 322.67 ms | 52.3% bf16 MFU | 1624502 tok/s
step 8383/19560 | loss 3.397811 (-1.50z)| norm 0.2652 (-0.77z)| lr 3.86e-04 | 322.37 ms | 52.4% bf16 MFU | 1624596 tok/s
step 8384/19560 | loss 3.462017 (+0.22z)| norm 0.2761 (-0.32z)| lr 3.86e-04 | 323.14 ms | 52.2% bf16 MFU | 1624489 tok/s
step 8385/19560 | loss 3.600127 (+3.69z)| norm 0.2781 (-0.23z)| lr 3.86e-04 | 323.01 ms | 52.3% bf16 MFU | 1624422 tok/s
step 8386/19560 | loss 3.478111 (+0.58z)| norm 0.2736 (-0.43z)| lr 3.86e-04 | 322.39 ms | 52.3% bf16 MFU | 1624513 tok/s
step 8387/19560 | loss 3.343314 (-2.75z)| norm 0.2551 (-1.18z)| lr 3.86e-04 | 322.70 ms | 52.3% bf16 MFU | 1624522 tok/s
step 8388/19560 | loss 3.444894 (-0.24z)| norm 0.2962 (+0.52z)| lr 3.86e-04 | 323.21 ms | 52.2% bf16 MFU | 1624402 tok/s
step 8389/19560 | loss 3.455651 (+0.04z)| norm 0.2813 (-0.11z)| lr 3.86e-04 | 322.86 ms | 52.3% bf16 MFU | 1624377 tok/s
step 8390/19560 | loss 3.444383 (-0.23z)| norm 0.2638 (-0.85z)| lr 3.86e-04 | 322.80 ms | 52.3% bf16 MFU | 1624367 tok/s
step 8391/19560 | loss 3.467017 (+0.34z)| norm 0.3029 (+0.79z)| lr 3.86e-04 | 322.39 ms | 52.4% bf16 MFU | 1624462 tok/s
step 8392/19560 | loss 3.455477 (+0.05z)| norm 0.2886 (+0.18z)| lr 3.86e-04 | 322.65 ms | 52.3% bf16 MFU | 1624485 tok/s
step 8393/19560 | loss 3.512126 (+1.48z)| norm 0.2754 (-0.38z)| lr 3.86e-04 | 323.02 ms | 52.2% bf16 MFU | 1624415 tok/s
step 8394/19560 | loss 3.392346 (-1.55z)| norm 0.2793 (-0.22z)| lr 3.86e-04 | 323.16 ms | 52.2% bf16 MFU | 1624314 tok/s
step 8395/19560 | loss 3.436028 (-0.44z)| norm 0.2992 (+0.61z)| lr 3.86e-04 | 322.98 ms | 52.3% bf16 MFU | 1624263 tok/s
step 8396/19560 | loss 3.472973 (+0.49z)| norm 0.2791 (-0.25z)| lr 3.86e-04 | 322.92 ms | 52.3% bf16 MFU | 1624230 tok/s
step 8397/19560 | loss 3.522881 (+1.72z)| norm 0.2617 (-0.99z)| lr 3.85e-04 | 322.81 ms | 52.3% bf16 MFU | 1624226 tok/s
step 8398/19560 | loss 3.476664 (+0.56z)| norm 0.2639 (-0.89z)| lr 3.85e-04 | 322.95 ms | 52.3% bf16 MFU | 1624187 tok/s
step 8399/19560 | loss 3.471429 (+0.43z)| norm 0.2691 (-0.66z)| lr 3.85e-04 | 322.76 ms | 52.3% bf16 MFU | 1624198 tok/s
step 8400/19560 | loss 3.449655 (-0.11z)| norm 0.2616 (-0.96z)| lr 3.85e-04 | 322.53 ms | 52.3% bf16 MFU | 1624266 tok/s
step 8401/19560 | loss 3.412081 (-1.04z)| norm 0.2614 (-0.96z)| lr 3.85e-04 | 322.62 ms | 52.3% bf16 MFU | 1624308 tok/s
step 8402/19560 | loss 3.400519 (-1.31z)| norm 0.2684 (-0.66z)| lr 3.85e-04 | 322.82 ms | 52.3% bf16 MFU | 1624297 tok/s
step 8403/19560 | loss 3.489544 (+0.92z)| norm 0.3093 (+1.05z)| lr 3.85e-04 | 323.00 ms | 52.3% bf16 MFU | 1624242 tok/s
step 8404/19560 | loss 3.386874 (-1.64z)| norm 0.3016 (+0.72z)| lr 3.85e-04 | 322.70 ms | 52.3% bf16 MFU | 1624263 tok/s
step 8405/19560 | loss 3.449513 (-0.07z)| norm 0.2801 (-0.18z)| lr 3.85e-04 | 322.40 ms | 52.3% bf16 MFU | 1624359 tok/s
step 8406/19560 | loss 3.444527 (-0.19z)| norm 0.2674 (-0.71z)| lr 3.85e-04 | 322.50 ms | 52.3% bf16 MFU | 1624427 tok/s
step 8407/19560 | loss 3.449511 (-0.07z)| norm 0.2639 (-0.84z)| lr 3.85e-04 | 322.97 ms | 52.3% bf16 MFU | 1624373 tok/s
step 8408/19560 | loss 3.501146 (+1.23z)| norm 0.2598 (-1.00z)| lr 3.85e-04 | 323.32 ms | 52.2% bf16 MFU | 1624234 tok/s
step 8409/19560 | loss 3.487082 (+0.87z)| norm 0.2616 (-0.92z)| lr 3.85e-04 | 322.09 ms | 52.4% bf16 MFU | 1624410 tok/s
step 8410/19560 | loss 3.433921 (-0.46z)| norm 0.2848 (+0.08z)| lr 3.85e-04 | 322.53 ms | 52.3% bf16 MFU | 1624467 tok/s
step 8411/19560 | loss 3.409017 (-1.07z)| norm 0.2752 (-0.33z)| lr 3.85e-04 | 322.82 ms | 52.3% bf16 MFU | 1624447 tok/s
step 8412/19560 | loss 3.451704 (-0.01z)| norm 0.3024 (+0.90z)| lr 3.85e-04 | 323.06 ms | 52.2% bf16 MFU | 1624368 tok/s
step 8413/19560 | loss 3.373901 (-1.92z)| norm 0.2526 (-1.32z)| lr 3.85e-04 | 323.34 ms | 52.2% bf16 MFU | 1624223 tok/s
step 8414/19560 | loss 3.413520 (-0.95z)| norm 0.2866 (+0.20z)| lr 3.85e-04 | 322.43 ms | 52.3% bf16 MFU | 1624314 tok/s
step 8415/19560 | loss 3.375419 (-1.85z)| norm 0.2702 (-0.52z)| lr 3.85e-04 | 322.73 ms | 52.3% bf16 MFU | 1624324 tok/s
step 8416/19560 | loss 3.401160 (-1.22z)| norm 0.2901 (+0.35z)| lr 3.85e-04 | 322.82 ms | 52.3% bf16 MFU | 1624313 tok/s
step 8417/19560 | loss 3.438438 (-0.31z)| norm 0.2842 (+0.09z)| lr 3.84e-04 | 322.76 ms | 52.3% bf16 MFU | 1624318 tok/s
step 8418/19560 | loss 3.407290 (-1.06z)| norm 0.2775 (-0.21z)| lr 3.84e-04 | 323.49 ms | 52.2% bf16 MFU | 1624138 tok/s
step 8419/19560 | loss 3.425704 (-0.61z)| norm 0.2793 (-0.13z)| lr 3.84e-04 | 322.34 ms | 52.4% bf16 MFU | 1624256 tok/s
step 8420/19560 | loss 3.431163 (-0.47z)| norm 0.2837 (+0.06z)| lr 3.84e-04 | 322.38 ms | 52.4% bf16 MFU | 1624359 tok/s
step 8421/19560 | loss 3.505784 (+1.33z)| norm 0.2680 (-0.64z)| lr 3.84e-04 | 323.23 ms | 52.2% bf16 MFU | 1624243 tok/s
step 8422/19560 | loss 3.472014 (+0.51z)| norm 0.2785 (-0.17z)| lr 3.84e-04 | 322.22 ms | 52.4% bf16 MFU | 1624386 tok/s
step 8423/19560 | loss 3.482303 (+0.75z)| norm 0.2882 (+0.26z)| lr 3.84e-04 | 322.80 ms | 52.3% bf16 MFU | 1624375 tok/s
step 8424/19560 | loss 3.436960 (-0.35z)| norm 0.2594 (-1.02z)| lr 3.84e-04 | 322.51 ms | 52.3% bf16 MFU | 1624439 tok/s
step 8425/19560 | loss 3.449271 (-0.06z)| norm 0.2793 (-0.14z)| lr 3.84e-04 | 322.42 ms | 52.3% bf16 MFU | 1624521 tok/s
step 8426/19560 | loss 3.451677 (-0.01z)| norm 0.2634 (-0.86z)| lr 3.84e-04 | 323.23 ms | 52.2% bf16 MFU | 1624397 tok/s
step 8427/19560 | loss 3.464553 (+0.30z)| norm 0.2618 (-0.92z)| lr 3.84e-04 | 322.77 ms | 52.3% bf16 MFU | 1624394 tok/s
step 8428/19560 | loss 3.437223 (-0.39z)| norm 0.2717 (-0.48z)| lr 3.84e-04 | 322.23 ms | 52.4% bf16 MFU | 1624528 tok/s
step 8429/19560 | loss 3.438772 (-0.35z)| norm 0.2776 (-0.18z)| lr 3.84e-04 | 322.37 ms | 52.4% bf16 MFU | 1624620 tok/s
step 8430/19560 | loss 3.426370 (-0.66z)| norm 0.3096 (+1.39z)| lr 3.84e-04 | 322.97 ms | 52.3% bf16 MFU | 1624555 tok/s
step 8431/19560 | loss 3.417284 (-0.87z)| norm 0.2634 (-0.87z)| lr 3.84e-04 | 323.21 ms | 52.2% bf16 MFU | 1624434 tok/s
step 8432/19560 | loss 3.479135 (+0.67z)| norm 0.2937 (+0.62z)| lr 3.84e-04 | 322.67 ms | 52.3% bf16 MFU | 1624455 tok/s
step 8433/19560 | loss 3.477282 (+0.62z)| norm 0.2766 (-0.22z)| lr 3.84e-04 | 323.19 ms | 52.2% bf16 MFU | 1624344 tok/s
step 8434/19560 | loss 3.385433 (-1.66z)| norm 0.2723 (-0.43z)| lr 3.84e-04 | 323.02 ms | 52.2% bf16 MFU | 1624280 tok/s
step 8435/19560 | loss 3.406347 (-1.12z)| norm 0.3121 (+1.50z)| lr 3.84e-04 | 322.78 ms | 52.3% bf16 MFU | 1624279 tok/s
step 8436/19560 | loss 3.548350 (+2.37z)| norm 0.3036 (+1.07z)| lr 3.84e-04 | 323.00 ms | 52.3% bf16 MFU | 1624225 tok/s
step 8437/19560 | loss 3.469294 (+0.43z)| norm 0.2984 (+0.81z)| lr 3.84e-04 | 322.46 ms | 52.3% bf16 MFU | 1624309 tok/s
step 8438/19560 | loss 3.451080 (-0.02z)| norm 0.3190 (+1.80z)| lr 3.83e-04 | 322.91 ms | 52.3% bf16 MFU | 1624274 tok/s
step 8439/19560 | loss 3.407965 (-1.07z)| norm 0.2990 (+0.83z)| lr 3.83e-04 | 322.47 ms | 52.3% bf16 MFU | 1624352 tok/s
step 8440/19560 | loss 3.451876 (+0.01z)| norm 0.3165 (+1.64z)| lr 3.83e-04 | 322.78 ms | 52.3% bf16 MFU | 1624348 tok/s
step 8441/19560 | loss 3.411050 (-0.99z)| norm 0.3038 (+1.06z)| lr 3.83e-04 | 322.91 ms | 52.3% bf16 MFU | 1624312 tok/s
step 8442/19560 | loss 3.454168 (+0.06z)| norm 0.2979 (+0.76z)| lr 3.83e-04 | 322.46 ms | 52.3% bf16 MFU | 1624392 tok/s
step 8443/19560 | loss 3.469710 (+0.43z)| norm 0.2939 (+0.56z)| lr 3.83e-04 | 323.39 ms | 52.2% bf16 MFU | 1624234 tok/s
step 8444/19560 | loss 3.366294 (-2.10z)| norm 0.2684 (-0.67z)| lr 3.83e-04 | 322.65 ms | 52.3% bf16 MFU | 1624269 tok/s
step 8445/19560 | loss 3.527098 (+1.84z)| norm 0.3089 (+1.27z)| lr 3.83e-04 | 322.81 ms | 52.3% bf16 MFU | 1624264 tok/s
step 8446/19560 | loss 3.415856 (-0.87z)| norm 0.2683 (-0.68z)| lr 3.83e-04 | 322.35 ms | 52.4% bf16 MFU | 1624373 tok/s
step 8447/19560 | loss 3.431856 (-0.47z)| norm 0.2717 (-0.52z)| lr 3.83e-04 | 322.53 ms | 52.3% bf16 MFU | 1624432 tok/s
step 8448/19560 | loss 3.478153 (+0.65z)| norm 0.2940 (+0.54z)| lr 3.83e-04 | 322.94 ms | 52.3% bf16 MFU | 1624385 tok/s
step 8449/19560 | loss 3.446053 (-0.13z)| norm 0.2688 (-0.67z)| lr 3.83e-04 | 322.43 ms | 52.3% bf16 MFU | 1624468 tok/s
step 8450/19560 | loss 3.407290 (-1.07z)| norm 0.2720 (-0.52z)| lr 3.83e-04 | 322.60 ms | 52.3% bf16 MFU | 1624504 tok/s
step 8451/19560 | loss 3.512158 (+1.46z)| norm 0.2701 (-0.62z)| lr 3.83e-04 | 322.60 ms | 52.3% bf16 MFU | 1624540 tok/s
step 8452/19560 | loss 3.455086 (+0.08z)| norm 0.2902 (+0.35z)| lr 3.83e-04 | 322.34 ms | 52.4% bf16 MFU | 1624638 tok/s
step 8453/19560 | loss 3.418601 (-0.80z)| norm 0.2997 (+0.80z)| lr 3.83e-04 | 322.41 ms | 52.3% bf16 MFU | 1624714 tok/s
step 8454/19560 | loss 3.435163 (-0.40z)| norm 0.2611 (-1.10z)| lr 3.83e-04 | 322.58 ms | 52.3% bf16 MFU | 1624742 tok/s
step 8455/19560 | loss 3.477635 (+0.63z)| norm 0.2786 (-0.24z)| lr 3.83e-04 | 322.26 ms | 52.4% bf16 MFU | 1624850 tok/s
step 8456/19560 | loss 3.451124 (-0.02z)| norm 0.2823 (-0.06z)| lr 3.83e-04 | 322.78 ms | 52.3% bf16 MFU | 1624823 tok/s
step 8457/19560 | loss 3.468943 (+0.40z)| norm 0.2762 (-0.36z)| lr 3.83e-04 | 322.17 ms | 52.4% bf16 MFU | 1624951 tok/s
step 8458/19560 | loss 3.445643 (-0.17z)| norm 0.2755 (-0.39z)| lr 3.83e-04 | 322.52 ms | 52.3% bf16 MFU | 1624984 tok/s
step 8459/19560 | loss 3.375776 (-1.86z)| norm 0.2917 (+0.40z)| lr 3.82e-04 | 323.04 ms | 52.2% bf16 MFU | 1624885 tok/s
step 8460/19560 | loss 3.394618 (-1.38z)| norm 0.2578 (-1.26z)| lr 3.82e-04 | 322.21 ms | 52.4% bf16 MFU | 1624998 tok/s
step 8461/19560 | loss 3.409702 (-1.00z)| norm 0.2860 (+0.12z)| lr 3.82e-04 | 323.01 ms | 52.2% bf16 MFU | 1624905 tok/s
step 8462/19560 | loss 3.471562 (+0.49z)| norm 0.2630 (-0.99z)| lr 3.82e-04 | 322.10 ms | 52.4% bf16 MFU | 1625046 tok/s
step 8463/19560 | loss 3.365463 (-2.05z)| norm 0.2574 (-1.24z)| lr 3.82e-04 | 323.17 ms | 52.2% bf16 MFU | 1624909 tok/s
step 8464/19560 | loss 3.464033 (+0.31z)| norm 0.2734 (-0.48z)| lr 3.82e-04 | 322.04 ms | 52.4% bf16 MFU | 1625063 tok/s
step 8465/19560 | loss 3.423291 (-0.66z)| norm 0.2630 (-0.99z)| lr 3.82e-04 | 322.38 ms | 52.4% bf16 MFU | 1625125 tok/s
step 8466/19560 | loss 3.434638 (-0.38z)| norm 0.2677 (-0.76z)| lr 3.82e-04 | 323.09 ms | 52.2% bf16 MFU | 1625007 tok/s
step 8467/19560 | loss 3.400043 (-1.19z)| norm 0.2700 (-0.65z)| lr 3.82e-04 | 322.36 ms | 52.4% bf16 MFU | 1625076 tok/s
step 8468/19560 | loss 3.408908 (-0.97z)| norm 0.2778 (-0.27z)| lr 3.82e-04 | 322.79 ms | 52.3% bf16 MFU | 1625035 tok/s
step 8469/19560 | loss 3.526629 (+1.80z)| norm 0.2670 (-0.80z)| lr 3.82e-04 | 322.36 ms | 52.4% bf16 MFU | 1625103 tok/s
step 8470/19560 | loss 3.429895 (-0.47z)| norm 0.2781 (-0.25z)| lr 3.82e-04 | 322.64 ms | 52.3% bf16 MFU | 1625097 tok/s
step 8471/19560 | loss 3.431057 (-0.43z)| norm 0.2620 (-1.03z)| lr 3.82e-04 | 322.44 ms | 52.3% bf16 MFU | 1625141 tok/s
step 8472/19560 | loss 3.406031 (-1.01z)| norm 0.2745 (-0.43z)| lr 3.82e-04 | 322.64 ms | 52.3% bf16 MFU | 1625134 tok/s
step 8473/19560 | loss 3.450429 (+0.04z)| norm 0.2665 (-0.81z)| lr 3.82e-04 | 322.40 ms | 52.3% bf16 MFU | 1625189 tok/s
step 8474/19560 | loss 3.432849 (-0.37z)| norm 0.2686 (-0.71z)| lr 3.82e-04 | 323.03 ms | 52.2% bf16 MFU | 1625082 tok/s
step 8475/19560 | loss 3.412231 (-0.85z)| norm 0.2851 (+0.11z)| lr 3.82e-04 | 322.55 ms | 52.3% bf16 MFU | 1625100 tok/s
step 8476/19560 | loss 3.326467 (-2.79z)| norm 0.2521 (-1.50z)| lr 3.82e-04 | 322.70 ms | 52.3% bf16 MFU | 1625080 tok/s
step 8477/19560 | loss 3.433089 (-0.30z)| norm 0.2898 (+0.33z)| lr 3.82e-04 | 322.41 ms | 52.3% bf16 MFU | 1625134 tok/s
step 8478/19560 | loss 3.427930 (-0.42z)| norm 0.2806 (-0.11z)| lr 3.82e-04 | 322.82 ms | 52.3% bf16 MFU | 1625083 tok/s
step 8479/19560 | loss 3.425676 (-0.48z)| norm 0.2891 (+0.30z)| lr 3.82e-04 | 322.22 ms | 52.4% bf16 MFU | 1625184 tok/s
step 8480/19560 | loss 3.422585 (-0.54z)| norm 0.2875 (+0.21z)| lr 3.81e-04 | 322.80 ms | 52.3% bf16 MFU | 1625134 tok/s
step 8481/19560 | loss 3.463497 (+0.44z)| norm 0.2831 (+0.02z)| lr 3.81e-04 | 322.53 ms | 52.3% bf16 MFU | 1625155 tok/s
step 8482/19560 | loss 3.434541 (-0.25z)| norm 0.2940 (+0.67z)| lr 3.81e-04 | 322.68 ms | 52.3% bf16 MFU | 1625137 tok/s
step 8483/19560 | loss 3.429801 (-0.37z)| norm 0.2977 (+0.88z)| lr 3.81e-04 | 322.68 ms | 52.3% bf16 MFU | 1625121 tok/s
step 8484/19560 | loss 3.509105 (+1.50z)| norm 0.2865 (+0.31z)| lr 3.81e-04 | 322.53 ms | 52.3% bf16 MFU | 1625142 tok/s
step 8485/19560 | loss 3.505738 (+1.40z)| norm 0.2649 (-0.99z)| lr 3.81e-04 | 322.43 ms | 52.3% bf16 MFU | 1625188 tok/s
step 8486/19560 | loss 3.459270 (+0.30z)| norm 0.2963 (+0.94z)| lr 3.81e-04 | 322.56 ms | 52.3% bf16 MFU | 1625199 tok/s
step 8487/19560 | loss 3.407431 (-0.90z)| norm 0.2703 (-0.66z)| lr 3.81e-04 | 322.31 ms | 52.4% bf16 MFU | 1625272 tok/s
step 8488/19560 | loss 3.442542 (-0.07z)| norm 0.2909 (+0.67z)| lr 3.81e-04 | 322.81 ms | 52.3% bf16 MFU | 1625216 tok/s
step 8489/19560 | loss 3.464875 (+0.45z)| norm 0.2890 (+0.59z)| lr 3.81e-04 | 322.04 ms | 52.4% bf16 MFU | 1625357 tok/s
step 8490/19560 | loss 3.585331 (+3.12z)| norm 0.2798 (-0.03z)| lr 3.81e-04 | 322.54 ms | 52.3% bf16 MFU | 1625365 tok/s
step 8491/19560 | loss 3.446706 (+0.01z)| norm 0.3209 (+2.62z)| lr 3.81e-04 | 322.72 ms | 52.3% bf16 MFU | 1625325 tok/s
step 8492/19560 | loss 3.451406 (+0.12z)| norm 0.2639 (-1.07z)| lr 3.81e-04 | 322.57 ms | 52.3% bf16 MFU | 1625328 tok/s
step 8493/19560 | loss 3.489359 (+0.98z)| norm 0.2885 (+0.53z)| lr 3.81e-04 | 322.54 ms | 52.3% bf16 MFU | 1625336 tok/s
step 8494/19560 | loss 3.476391 (+0.68z)| norm 0.2687 (-0.74z)| lr 3.81e-04 | 322.41 ms | 52.3% bf16 MFU | 1625376 tok/s
step 8495/19560 | loss 3.490137 (+0.98z)| norm 0.2890 (+0.57z)| lr 3.81e-04 | 322.68 ms | 52.3% bf16 MFU | 1625348 tok/s
step 8496/19560 | loss 3.452627 (+0.14z)| norm 0.2727 (-0.47z)| lr 3.81e-04 | 322.87 ms | 52.3% bf16 MFU | 1625272 tok/s
step 8497/19560 | loss 3.448830 (+0.06z)| norm 0.2535 (-1.69z)| lr 3.81e-04 | 322.10 ms | 52.4% bf16 MFU | 1625395 tok/s
step 8498/19560 | loss 3.449033 (+0.06z)| norm 0.2916 (+0.75z)| lr 3.81e-04 | 322.47 ms | 52.3% bf16 MFU | 1625417 tok/s
step 8499/19560 | loss 3.405685 (-0.95z)| norm 0.2753 (-0.29z)| lr 3.81e-04 | 322.22 ms | 52.4% bf16 MFU | 1625500 tok/s
step 8500/19560 | loss 3.408258 (-0.87z)| norm 0.2776 (-0.15z)| lr 3.81e-04 | 322.96 ms | 52.3% bf16 MFU | 1625394 tok/s
val loss 3.433615
laSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluaevaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2847/10042 = 0.283509
step 8501/19560 | loss 3.397943 (-1.11z)| norm 0.2896 (+0.62z)| lr 3.80e-04 | 323.71 ms | 52.1% bf16 MFU | 1625104 tok/s
step 8502/19560 | loss 3.467796 (+0.56z)| norm 0.2792 (-0.04z)| lr 3.80e-04 | 322.08 ms | 52.4% bf16 MFU | 1625240 tok/s
step 8503/19560 | loss 3.405304 (-0.92z)| norm 0.2462 (-2.13z)| lr 3.80e-04 | 322.70 ms | 52.3% bf16 MFU | 1625213 tok/s
step 8504/19560 | loss 3.464673 (+0.49z)| norm 0.2817 (+0.15z)| lr 3.80e-04 | 322.73 ms | 52.3% bf16 MFU | 1625179 tok/s
step 8505/19560 | loss 3.466044 (+0.52z)| norm 0.2759 (-0.22z)| lr 3.80e-04 | 322.35 ms | 52.4% bf16 MFU | 1625244 tok/s
step 8506/19560 | loss 3.453981 (+0.22z)| norm 0.2770 (-0.13z)| lr 3.80e-04 | 323.00 ms | 52.3% bf16 MFU | 1625141 tok/s
step 8507/19560 | loss 3.457299 (+0.30z)| norm 0.2869 (+0.50z)| lr 3.80e-04 | 322.66 ms | 52.3% bf16 MFU | 1625129 tok/s
step 8508/19560 | loss 3.458462 (+0.32z)| norm 0.2839 (+0.31z)| lr 3.80e-04 | 322.60 ms | 52.3% bf16 MFU | 1625132 tok/s
step 8509/19560 | loss 3.379881 (-1.52z)| norm 0.2684 (-0.71z)| lr 3.80e-04 | 322.49 ms | 52.3% bf16 MFU | 1625162 tok/s
step 8510/19560 | loss 3.453718 (+0.22z)| norm 0.2923 (+0.85z)| lr 3.80e-04 | 321.90 ms | 52.4% bf16 MFU | 1625340 tok/s
step 8511/19560 | loss 3.434658 (-0.24z)| norm 0.3239 (+2.82z)| lr 3.80e-04 | 323.27 ms | 52.2% bf16 MFU | 1625164 tok/s
step 8512/19560 | loss 3.384659 (-1.40z)| norm 0.2821 (+0.14z)| lr 3.80e-04 | 322.30 ms | 52.4% bf16 MFU | 1625242 tok/s
step 8513/19560 | loss 3.437086 (-0.14z)| norm 0.3491 (+4.09z)| lr 3.80e-04 | 322.48 ms | 52.3% bf16 MFU | 1625271 tok/s
step 8514/19560 | loss 3.431070 (-0.29z)| norm 0.3093 (+1.69z)| lr 3.80e-04 | 322.48 ms | 52.3% bf16 MFU | 1625298 tok/s
step 8515/19560 | loss 3.478909 (+0.90z)| norm 0.2994 (+1.08z)| lr 3.80e-04 | 323.05 ms | 52.2% bf16 MFU | 1625180 tok/s
step 8516/19560 | loss 3.454384 (+0.27z)| norm 0.3141 (+1.93z)| lr 3.80e-04 | 322.41 ms | 52.3% bf16 MFU | 1625229 tok/s
step 8517/19560 | loss 3.445824 (+0.06z)| norm 0.2727 (-0.50z)| lr 3.80e-04 | 322.60 ms | 52.3% bf16 MFU | 1625226 tok/s
step 8518/19560 | loss 3.439834 (-0.09z)| norm 0.2928 (+0.67z)| lr 3.80e-04 | 322.81 ms | 52.3% bf16 MFU | 1625172 tok/s
step 8519/19560 | loss 3.413303 (-0.76z)| norm 0.3270 (+2.60z)| lr 3.80e-04 | 322.58 ms | 52.3% bf16 MFU | 1625177 tok/s
step 8520/19560 | loss 3.421385 (-0.55z)| norm 0.2776 (-0.22z)| lr 3.80e-04 | 322.78 ms | 52.3% bf16 MFU | 1625133 tok/s
step 8521/19560 | loss 3.455906 (+0.35z)| norm 0.2835 (+0.11z)| lr 3.79e-04 | 322.38 ms | 52.4% bf16 MFU | 1625193 tok/s
step 8522/19560 | loss 3.414950 (-0.71z)| norm 0.2908 (+0.53z)| lr 3.79e-04 | 322.72 ms | 52.3% bf16 MFU | 1625164 tok/s
step 8523/19560 | loss 3.446068 (+0.09z)| norm 0.2724 (-0.53z)| lr 3.79e-04 | 323.08 ms | 52.2% bf16 MFU | 1625045 tok/s
step 8524/19560 | loss 3.429233 (-0.34z)| norm 0.2904 (+0.51z)| lr 3.79e-04 | 322.94 ms | 52.3% bf16 MFU | 1624967 tok/s
step 8525/19560 | loss 3.444229 (+0.07z)| norm 0.2968 (+0.86z)| lr 3.79e-04 | 322.24 ms | 52.4% bf16 MFU | 1625069 tok/s
step 8526/19560 | loss 3.414863 (-0.70z)| norm 0.2873 (+0.31z)| lr 3.79e-04 | 322.78 ms | 52.3% bf16 MFU | 1625030 tok/s
step 8527/19560 | loss 3.417370 (-0.62z)| norm 0.2685 (-0.78z)| lr 3.79e-04 | 323.41 ms | 52.2% bf16 MFU | 1624835 tok/s
step 8528/19560 | loss 3.420123 (-0.54z)| norm 0.2795 (-0.16z)| lr 3.79e-04 | 322.37 ms | 52.4% bf16 MFU | 1624911 tok/s
step 8529/19560 | loss 3.433796 (-0.18z)| norm 0.2716 (-0.62z)| lr 3.79e-04 | 322.75 ms | 52.3% bf16 MFU | 1624888 tok/s
step 8530/19560 | loss 3.430567 (-0.28z)| norm 0.2914 (+0.53z)| lr 3.79e-04 | 323.04 ms | 52.2% bf16 MFU | 1624793 tok/s
step 8531/19560 | loss 3.435646 (-0.13z)| norm 0.2584 (-1.38z)| lr 3.79e-04 | 322.76 ms | 52.3% bf16 MFU | 1624774 tok/s
step 8532/19560 | loss 3.429736 (-0.30z)| norm 0.2973 (+0.90z)| lr 3.79e-04 | 322.35 ms | 52.4% bf16 MFU | 1624859 tok/s
step 8533/19560 | loss 3.433757 (-0.19z)| norm 0.2817 (-0.02z)| lr 3.79e-04 | 322.52 ms | 52.3% bf16 MFU | 1624896 tok/s
step 8534/19560 | loss 3.413630 (-0.72z)| norm 0.2710 (-0.65z)| lr 3.79e-04 | 322.59 ms | 52.3% bf16 MFU | 1624913 tok/s
step 8535/19560 | loss 3.444435 (+0.11z)| norm 0.2918 (+0.56z)| lr 3.79e-04 | 322.49 ms | 52.3% bf16 MFU | 1624956 tok/s
step 8536/19560 | loss 3.429999 (-0.27z)| norm 0.2700 (-0.73z)| lr 3.79e-04 | 323.25 ms | 52.2% bf16 MFU | 1624804 tok/s
step 8537/19560 | loss 3.436089 (-0.09z)| norm 0.2648 (-1.04z)| lr 3.79e-04 | 322.73 ms | 52.3% bf16 MFU | 1624790 tok/s
step 8538/19560 | loss 3.411350 (-0.76z)| norm 0.2557 (-1.56z)| lr 3.79e-04 | 323.40 ms | 52.2% bf16 MFU | 1624608 tok/s
step 8539/19560 | loss 3.438534 (-0.03z)| norm 0.2687 (-0.79z)| lr 3.79e-04 | 323.39 ms | 52.2% bf16 MFU | 1624440 tok/s
step 8540/19560 | loss 3.509233 (+1.87z)| norm 0.2782 (-0.21z)| lr 3.79e-04 | 322.32 ms | 52.4% bf16 MFU | 1624547 tok/s
step 8541/19560 | loss 3.471739 (+0.84z)| norm 0.2557 (-1.55z)| lr 3.79e-04 | 322.58 ms | 52.3% bf16 MFU | 1624584 tok/s
step 8542/19560 | loss 3.519291 (+2.09z)| norm 0.2702 (-0.68z)| lr 3.78e-04 | 322.34 ms | 52.4% bf16 MFU | 1624680 tok/s
step 8543/19560 | loss 3.461208 (+0.51z)| norm 0.2682 (-0.80z)| lr 3.78e-04 | 322.50 ms | 52.3% bf16 MFU | 1624731 tok/s
step 8544/19560 | loss 3.436311 (-0.17z)| norm 0.2517 (-1.74z)| lr 3.78e-04 | 322.32 ms | 52.4% bf16 MFU | 1624824 tok/s
step 8545/19560 | loss 3.393215 (-1.33z)| norm 0.2493 (-1.84z)| lr 3.78e-04 | 322.40 ms | 52.3% bf16 MFU | 1624892 tok/s
step 8546/19560 | loss 3.434732 (-0.21z)| norm 0.2531 (-1.59z)| lr 3.78e-04 | 322.96 ms | 52.3% bf16 MFU | 1624818 tok/s
step 8547/19560 | loss 3.403682 (-1.05z)| norm 0.2556 (-1.43z)| lr 3.78e-04 | 322.59 ms | 52.3% bf16 MFU | 1624839 tok/s
step 8548/19560 | loss 3.447400 (+0.14z)| norm 0.2601 (-1.16z)| lr 3.78e-04 | 322.96 ms | 52.3% bf16 MFU | 1624767 tok/s
step 8549/19560 | loss 3.414876 (-0.73z)| norm 0.2461 (-1.91z)| lr 3.78e-04 | 322.50 ms | 52.3% bf16 MFU | 1624814 tok/s
step 8550/19560 | loss 3.451184 (+0.26z)| norm 0.2898 (+0.52z)| lr 3.78e-04 | 323.05 ms | 52.2% bf16 MFU | 1624721 tok/s
step 8551/19560 | loss 3.403583 (-1.03z)| norm 0.2931 (+0.70z)| lr 3.78e-04 | 322.64 ms | 52.3% bf16 MFU | 1624735 tok/s
step 8552/19560 | loss 3.417947 (-0.63z)| norm 0.2535 (-1.50z)| lr 3.78e-04 | 322.94 ms | 52.3% bf16 MFU | 1624673 tok/s
step 8553/19560 | loss 3.586523 (+3.74z)| norm 0.3202 (+2.14z)| lr 3.78e-04 | 323.35 ms | 52.2% bf16 MFU | 1624510 tok/s
step 8554/19560 | loss 3.423902 (-0.46z)| norm 0.3035 (+1.21z)| lr 3.78e-04 | 322.68 ms | 52.3% bf16 MFU | 1624526 tok/s
step 8555/19560 | loss 3.475312 (+0.87z)| norm 0.2987 (+0.94z)| lr 3.78e-04 | 323.66 ms | 52.1% bf16 MFU | 1624294 tok/s
step 8556/19560 | loss 3.388183 (-1.36z)| norm 0.2642 (-0.93z)| lr 3.78e-04 | 322.43 ms | 52.3% bf16 MFU | 1624382 tok/s
step 8557/19560 | loss 3.428090 (-0.34z)| norm 0.2903 (+0.48z)| lr 3.78e-04 | 323.22 ms | 52.2% bf16 MFU | 1624266 tok/s
step 8558/19560 | loss 3.385494 (-1.41z)| norm 0.2565 (-1.33z)| lr 3.78e-04 | 322.79 ms | 52.3% bf16 MFU | 1624265 tok/s
step 8559/19560 | loss 3.473921 (+0.83z)| norm 0.2878 (+0.36z)| lr 3.78e-04 | 322.88 ms | 52.3% bf16 MFU | 1624240 tok/s
step 8560/19560 | loss 3.425565 (-0.39z)| norm 0.2676 (-0.73z)| lr 3.78e-04 | 322.93 ms | 52.3% bf16 MFU | 1624204 tok/s
step 8561/19560 | loss 3.446987 (+0.16z)| norm 0.2629 (-0.98z)| lr 3.78e-04 | 322.88 ms | 52.3% bf16 MFU | 1624182 tok/s
step 8562/19560 | loss 3.413682 (-0.70z)| norm 0.2554 (-1.37z)| lr 3.78e-04 | 322.74 ms | 52.3% bf16 MFU | 1624198 tok/s
step 8563/19560 | loss 3.452948 (+0.30z)| norm 0.2818 (+0.07z)| lr 3.77e-04 | 322.71 ms | 52.3% bf16 MFU | 1624220 tok/s
step 8564/19560 | loss 3.440291 (-0.00z)| norm 0.2619 (-1.00z)| lr 3.77e-04 | 322.81 ms | 52.3% bf16 MFU | 1624216 tok/s
step 8565/19560 | loss 3.424538 (-0.41z)| norm 0.2879 (+0.42z)| lr 3.77e-04 | 323.08 ms | 52.2% bf16 MFU | 1624144 tok/s
step 8566/19560 | loss 3.480145 (+1.06z)| norm 0.2778 (-0.11z)| lr 3.77e-04 | 322.70 ms | 52.3% bf16 MFU | 1624172 tok/s
step 8567/19560 | loss 3.409414 (-0.82z)| norm 0.2745 (-0.29z)| lr 3.77e-04 | 322.81 ms | 52.3% bf16 MFU | 1624170 tok/s
step 8568/19560 | loss 3.406154 (-0.90z)| norm 0.2811 (+0.10z)| lr 3.77e-04 | 322.74 ms | 52.3% bf16 MFU | 1624185 tok/s
step 8569/19560 | loss 3.402071 (-1.00z)| norm 0.2930 (+0.78z)| lr 3.77e-04 | 322.81 ms | 52.3% bf16 MFU | 1624183 tok/s
step 8570/19560 | loss 3.431119 (-0.23z)| norm 0.2600 (-1.09z)| lr 3.77e-04 | 322.64 ms | 52.3% bf16 MFU | 1624223 tok/s
step 8571/19560 | loss 3.437090 (-0.06z)| norm 0.3050 (+1.47z)| lr 3.77e-04 | 322.56 ms | 52.3% bf16 MFU | 1624280 tok/s
step 8572/19560 | loss 3.476249 (+0.96z)| norm 0.2860 (+0.39z)| lr 3.77e-04 | 322.49 ms | 52.3% bf16 MFU | 1624354 tok/s
step 8573/19560 | loss 3.440788 (+0.03z)| norm 0.2906 (+0.66z)| lr 3.77e-04 | 322.90 ms | 52.3% bf16 MFU | 1624322 tok/s
step 8574/19560 | loss 3.506953 (+1.80z)| norm 0.2983 (+1.09z)| lr 3.77e-04 | 322.64 ms | 52.3% bf16 MFU | 1624355 tok/s
step 8575/19560 | loss 3.438990 (-0.04z)| norm 0.2866 (+0.41z)| lr 3.77e-04 | 322.85 ms | 52.3% bf16 MFU | 1624335 tok/s
step 8576/19560 | loss 3.426897 (-0.36z)| norm 0.2837 (+0.25z)| lr 3.77e-04 | 322.62 ms | 52.3% bf16 MFU | 1624372 tok/s
step 8577/19560 | loss 3.454490 (+0.39z)| norm 0.2970 (+1.00z)| lr 3.77e-04 | 323.04 ms | 52.2% bf16 MFU | 1624304 tok/s
step 8578/19560 | loss 3.396943 (-1.17z)| norm 0.3297 (+2.76z)| lr 3.77e-04 | 322.62 ms | 52.3% bf16 MFU | 1624344 tok/s
step 8579/19560 | loss 3.464473 (+0.68z)| norm 0.2775 (-0.14z)| lr 3.77e-04 | 322.85 ms | 52.3% bf16 MFU | 1624325 tok/s
step 8580/19560 | loss 3.441208 (+0.05z)| norm 0.3012 (+1.16z)| lr 3.77e-04 | 322.61 ms | 52.3% bf16 MFU | 1624365 tok/s
step 8581/19560 | loss 3.466941 (+0.74z)| norm 0.2756 (-0.24z)| lr 3.77e-04 | 322.34 ms | 52.4% bf16 MFU | 1624471 tok/s
step 8582/19560 | loss 3.515663 (+2.03z)| norm 0.3014 (+1.17z)| lr 3.77e-04 | 323.14 ms | 52.2% bf16 MFU | 1624372 tok/s
step 8583/19560 | loss 3.468759 (+0.77z)| norm 0.2540 (-1.44z)| lr 3.77e-04 | 323.08 ms | 52.2% bf16 MFU | 1624292 tok/s
step 8584/19560 | loss 3.434744 (-0.15z)| norm 0.2848 (+0.26z)| lr 3.76e-04 | 322.37 ms | 52.4% bf16 MFU | 1624395 tok/s
step 8585/19560 | loss 3.409931 (-0.81z)| norm 0.2718 (-0.45z)| lr 3.76e-04 | 323.40 ms | 52.2% bf16 MFU | 1624233 tok/s
step 8586/19560 | loss 3.533392 (+2.45z)| norm 0.2635 (-0.90z)| lr 3.76e-04 | 322.67 ms | 52.3% bf16 MFU | 1624263 tok/s
step 8587/19560 | loss 3.396340 (-1.18z)| norm 0.2734 (-0.35z)| lr 3.76e-04 | 322.74 ms | 52.3% bf16 MFU | 1624275 tok/s
step 8588/19560 | loss 3.477778 (+0.97z)| norm 0.2630 (-0.93z)| lr 3.76e-04 | 323.39 ms | 52.2% bf16 MFU | 1624122 tok/s
step 8589/19560 | loss 3.447051 (+0.14z)| norm 0.4058 (+5.88z)| lr 3.76e-04 | 322.68 ms | 52.3% bf16 MFU | 1624155 tok/s
step 8590/19560 | loss 3.456711 (+0.41z)| norm 0.2853 (+0.20z)| lr 3.76e-04 | 323.04 ms | 52.2% bf16 MFU | 1624097 tok/s
step 8591/19560 | loss 3.451324 (+0.25z)| norm 0.2975 (+0.77z)| lr 3.76e-04 | 323.09 ms | 52.2% bf16 MFU | 1624028 tok/s
step 8592/19560 | loss 3.426154 (-0.43z)| norm 0.2693 (-0.57z)| lr 3.76e-04 | 322.37 ms | 52.4% bf16 MFU | 1624146 tok/s
step 8593/19560 | loss 3.490326 (+1.30z)| norm 0.2854 (+0.19z)| lr 3.76e-04 | 322.71 ms | 52.3% bf16 MFU | 1624170 tok/s
step 8594/19560 | loss 3.394976 (-1.27z)| norm 0.2566 (-1.17z)| lr 3.76e-04 | 322.88 ms | 52.3% bf16 MFU | 1624150 tok/s
step 8595/19560 | loss 3.497386 (+1.46z)| norm 0.2726 (-0.42z)| lr 3.76e-04 | 322.46 ms | 52.3% bf16 MFU | 1624236 tok/s
step 8596/19560 | loss 3.517821 (+1.96z)| norm 0.2754 (-0.28z)| lr 3.76e-04 | 322.88 ms | 52.3% bf16 MFU | 1624214 tok/s
step 8597/19560 | loss 3.462304 (+0.51z)| norm 0.2614 (-0.94z)| lr 3.76e-04 | 322.99 ms | 52.3% bf16 MFU | 1624165 tok/s
step 8598/19560 | loss 3.496711 (+1.42z)| norm 0.2955 (+0.66z)| lr 3.76e-04 | 322.75 ms | 52.3% bf16 MFU | 1624178 tok/s
step 8599/19560 | loss 3.466253 (+0.59z)| norm 0.2569 (-1.15z)| lr 3.76e-04 | 322.96 ms | 52.3% bf16 MFU | 1624139 tok/s
step 8600/19560 | loss 3.437773 (-0.18z)| norm 0.2766 (-0.23z)| lr 3.76e-04 | 322.78 ms | 52.3% bf16 MFU | 1624146 tok/s
step 8601/19560 | loss 3.412980 (-0.83z)| norm 0.2602 (-0.99z)| lr 3.76e-04 | 322.89 ms | 52.3% bf16 MFU | 1624126 tok/s
step 8602/19560 | loss 3.414978 (-0.77z)| norm 0.2759 (-0.26z)| lr 3.76e-04 | 322.54 ms | 52.3% bf16 MFU | 1624194 tok/s
step 8603/19560 | loss 3.395175 (-1.29z)| norm 0.2541 (-1.27z)| lr 3.76e-04 | 322.78 ms | 52.3% bf16 MFU | 1624199 tok/s
step 8604/19560 | loss 3.436646 (-0.22z)| norm 0.2466 (-1.61z)| lr 3.75e-04 | 323.01 ms | 52.3% bf16 MFU | 1624146 tok/s
step 8605/19560 | loss 3.471761 (+0.74z)| norm 0.2625 (-0.86z)| lr 3.75e-04 | 322.75 ms | 52.3% bf16 MFU | 1624161 tok/s
step 8606/19560 | loss 3.434410 (-0.29z)| norm 0.2737 (-0.33z)| lr 3.75e-04 | 322.87 ms | 52.3% bf16 MFU | 1624144 tok/s
step 8607/19560 | loss 3.435482 (-0.26z)| norm 0.2753 (-0.26z)| lr 3.75e-04 | 322.79 ms | 52.3% bf16 MFU | 1624148 tok/s
step 8608/19560 | loss 3.421261 (-0.66z)| norm 0.2716 (-0.42z)| lr 3.75e-04 | 323.02 ms | 52.2% bf16 MFU | 1624095 tok/s
step 8609/19560 | loss 3.371888 (-1.98z)| norm 0.2511 (-1.35z)| lr 3.75e-04 | 322.61 ms | 52.3% bf16 MFU | 1624148 tok/s
step 8610/19560 | loss 3.440477 (-0.11z)| norm 0.2909 (+0.48z)| lr 3.75e-04 | 322.94 ms | 52.3% bf16 MFU | 1624114 tok/s
step 8611/19560 | loss 3.442532 (-0.05z)| norm 0.2570 (-1.06z)| lr 3.75e-04 | 322.90 ms | 52.3% bf16 MFU | 1624093 tok/s
step 8612/19560 | loss 3.479153 (+0.96z)| norm 0.2861 (+0.28z)| lr 3.75e-04 | 322.25 ms | 52.4% bf16 MFU | 1624236 tok/s
step 8613/19560 | loss 3.552805 (+2.92z)| norm 0.2888 (+0.39z)| lr 3.75e-04 | 322.67 ms | 52.3% bf16 MFU | 1624266 tok/s
step 8614/19560 | loss 3.459034 (+0.39z)| norm 0.2878 (+0.35z)| lr 3.75e-04 | 322.64 ms | 52.3% bf16 MFU | 1624302 tok/s
step 8615/19560 | loss 3.501216 (+1.50z)| norm 0.2841 (+0.18z)| lr 3.75e-04 | 322.65 ms | 52.3% bf16 MFU | 1624335 tok/s
step 8616/19560 | loss 3.430509 (-0.39z)| norm 0.2828 (+0.12z)| lr 3.75e-04 | 322.34 ms | 52.4% bf16 MFU | 1624444 tok/s
step 8617/19560 | loss 3.515721 (+1.85z)| norm 0.3052 (+1.14z)| lr 3.75e-04 | 322.08 ms | 52.4% bf16 MFU | 1624612 tok/s
step 8618/19560 | loss 3.373001 (-1.96z)| norm 0.2829 (+0.12z)| lr 3.75e-04 | 322.63 ms | 52.3% bf16 MFU | 1624633 tok/s
step 8619/19560 | loss 3.432909 (-0.30z)| norm 0.2775 (-0.12z)| lr 3.75e-04 | 322.70 ms | 52.3% bf16 MFU | 1624635 tok/s
step 8620/19560 | loss 3.491978 (+1.31z)| norm 0.2802 (+0.00z)| lr 3.75e-04 | 322.30 ms | 52.4% bf16 MFU | 1624738 tok/s
step 8621/19560 | loss 3.411290 (-0.89z)| norm 0.2985 (+0.85z)| lr 3.75e-04 | 322.52 ms | 52.3% bf16 MFU | 1624782 tok/s
step 8622/19560 | loss 3.410945 (-0.88z)| norm 0.2878 (+0.35z)| lr 3.75e-04 | 322.32 ms | 52.4% bf16 MFU | 1624873 tok/s
step 8623/19560 | loss 3.495973 (+1.45z)| norm 0.2911 (+0.50z)| lr 3.75e-04 | 322.37 ms | 52.4% bf16 MFU | 1624946 tok/s
step 8624/19560 | loss 3.454773 (+0.32z)| norm 0.2613 (-0.89z)| lr 3.75e-04 | 322.52 ms | 52.3% bf16 MFU | 1624978 tok/s
step 8625/19560 | loss 3.397222 (-1.24z)| norm 0.3085 (+1.29z)| lr 3.74e-04 | 322.70 ms | 52.3% bf16 MFU | 1624964 tok/s
step 8626/19560 | loss 3.455181 (+0.34z)| norm 0.2706 (-0.46z)| lr 3.74e-04 | 322.88 ms | 52.3% bf16 MFU | 1624906 tok/s
step 8627/19560 | loss 3.373213 (-1.87z)| norm 0.2699 (-0.49z)| lr 3.74e-04 | 322.17 ms | 52.4% bf16 MFU | 1625028 tok/s
step 8628/19560 | loss 3.414404 (-0.76z)| norm 0.3020 (+0.98z)| lr 3.74e-04 | 322.48 ms | 52.3% bf16 MFU | 1625065 tok/s
step 8629/19560 | loss 3.463347 (+0.55z)| norm 0.2588 (-1.00z)| lr 3.74e-04 | 322.58 ms | 52.3% bf16 MFU | 1625077 tok/s
step 8630/19560 | loss 3.436069 (-0.18z)| norm 0.2708 (-0.45z)| lr 3.74e-04 | 322.92 ms | 52.3% bf16 MFU | 1625001 tok/s
step 8631/19560 | loss 3.490862 (+1.28z)| norm 0.2617 (-0.88z)| lr 3.74e-04 | 322.63 ms | 52.3% bf16 MFU | 1625002 tok/s
step 8632/19560 | loss 3.482356 (+1.05z)| norm 0.2724 (-0.38z)| lr 3.74e-04 | 322.59 ms | 52.3% bf16 MFU | 1625015 tok/s
step 8633/19560 | loss 3.440956 (-0.07z)| norm 0.2814 (+0.04z)| lr 3.74e-04 | 322.68 ms | 52.3% bf16 MFU | 1625004 tok/s
step 8634/19560 | loss 3.422919 (-0.55z)| norm 0.2826 (+0.09z)| lr 3.74e-04 | 322.72 ms | 52.3% bf16 MFU | 1624983 tok/s
step 8635/19560 | loss 3.469795 (+0.72z)| norm 0.2723 (-0.38z)| lr 3.74e-04 | 322.66 ms | 52.3% bf16 MFU | 1624978 tok/s
step 8636/19560 | loss 3.411516 (-0.85z)| norm 0.2776 (-0.13z)| lr 3.74e-04 | 322.71 ms | 52.3% bf16 MFU | 1624962 tok/s
step 8637/19560 | loss 3.426938 (-0.44z)| norm 0.2572 (-1.07z)| lr 3.74e-04 | 323.06 ms | 52.2% bf16 MFU | 1624856 tok/s
step 8638/19560 | loss 3.418968 (-0.65z)| norm 0.2687 (-0.53z)| lr 3.74e-04 | 322.62 ms | 52.3% bf16 MFU | 1624868 tok/s
step 8639/19560 | loss 3.485788 (+1.15z)| norm 0.2573 (-1.04z)| lr 3.74e-04 | 322.98 ms | 52.3% bf16 MFU | 1624788 tok/s
step 8640/19560 | loss 3.445615 (+0.05z)| norm 0.2782 (-0.06z)| lr 3.74e-04 | 323.32 ms | 52.2% bf16 MFU | 1624628 tok/s
step 8641/19560 | loss 3.405326 (-1.04z)| norm 0.2915 (+0.60z)| lr 3.74e-04 | 322.33 ms | 52.4% bf16 MFU | 1624724 tok/s
step 8642/19560 | loss 3.451714 (+0.22z)| norm 0.2775 (-0.07z)| lr 3.74e-04 | 322.20 ms | 52.4% bf16 MFU | 1624847 tok/s
step 8643/19560 | loss 3.458167 (+0.40z)| norm 0.2608 (-0.88z)| lr 3.74e-04 | 322.49 ms | 52.3% bf16 MFU | 1624891 tok/s
step 8644/19560 | loss 3.381133 (-1.67z)| norm 0.2610 (-0.85z)| lr 3.74e-04 | 322.80 ms | 52.3% bf16 MFU | 1624857 tok/s
step 8645/19560 | loss 3.384728 (-1.55z)| norm 0.2508 (-1.34z)| lr 3.74e-04 | 322.21 ms | 52.4% bf16 MFU | 1624973 tok/s
step 8646/19560 | loss 3.408725 (-0.90z)| norm 0.2659 (-0.59z)| lr 3.73e-04 | 323.30 ms | 52.2% bf16 MFU | 1624809 tok/s
step 8647/19560 | loss 3.492563 (+1.32z)| norm 0.2762 (-0.06z)| lr 3.73e-04 | 322.25 ms | 52.4% bf16 MFU | 1624917 tok/s
step 8648/19560 | loss 3.464746 (+0.57z)| norm 0.2835 (+0.31z)| lr 3.73e-04 | 322.66 ms | 52.3% bf16 MFU | 1624917 tok/s
step 8649/19560 | loss 3.470128 (+0.71z)| norm 0.2957 (+0.92z)| lr 3.73e-04 | 322.16 ms | 52.4% bf16 MFU | 1625041 tok/s
step 8650/19560 | loss 3.485414 (+1.10z)| norm 0.2834 (+0.30z)| lr 3.73e-04 | 322.73 ms | 52.3% bf16 MFU | 1625015 tok/s
step 8651/19560 | loss 3.433724 (-0.27z)| norm 0.2786 (+0.05z)| lr 3.73e-04 | 322.77 ms | 52.3% bf16 MFU | 1624982 tok/s
step 8652/19560 | loss 3.405980 (-0.99z)| norm 0.2725 (-0.25z)| lr 3.73e-04 | 322.24 ms | 52.4% bf16 MFU | 1625084 tok/s
step 8653/19560 | loss 3.397414 (-1.20z)| norm 0.2805 (+0.17z)| lr 3.73e-04 | 322.28 ms | 52.4% bf16 MFU | 1625169 tok/s
step 8654/19560 | loss 3.398101 (-1.18z)| norm 0.2600 (-0.87z)| lr 3.73e-04 | 322.43 ms | 52.3% bf16 MFU | 1625212 tok/s
step 8655/19560 | loss 3.453868 (+0.27z)| norm 0.2925 (+0.78z)| lr 3.73e-04 | 322.90 ms | 52.3% bf16 MFU | 1625135 tok/s
step 8656/19560 | loss 3.435668 (-0.21z)| norm 0.2938 (+0.83z)| lr 3.73e-04 | 322.64 ms | 52.3% bf16 MFU | 1625127 tok/s
step 8657/19560 | loss 3.406382 (-0.97z)| norm 0.3053 (+1.39z)| lr 3.73e-04 | 322.63 ms | 52.3% bf16 MFU | 1625122 tok/s
step 8658/19560 | loss 3.403144 (-1.04z)| norm 0.2883 (+0.54z)| lr 3.73e-04 | 322.48 ms | 52.3% bf16 MFU | 1625157 tok/s
step 8659/19560 | loss 3.412189 (-0.80z)| norm 0.2971 (+0.97z)| lr 3.73e-04 | 322.63 ms | 52.3% bf16 MFU | 1625151 tok/s
step 8660/19560 | loss 3.418075 (-0.64z)| norm 0.2629 (-0.74z)| lr 3.73e-04 | 322.30 ms | 52.4% bf16 MFU | 1625228 tok/s
step 8661/19560 | loss 3.477532 (+0.89z)| norm 0.2879 (+0.51z)| lr 3.73e-04 | 323.05 ms | 52.2% bf16 MFU | 1625114 tok/s
step 8662/19560 | loss 3.486359 (+1.10z)| norm 0.2906 (+0.64z)| lr 3.73e-04 | 322.48 ms | 52.3% bf16 MFU | 1625148 tok/s
step 8663/19560 | loss 3.461837 (+0.46z)| norm 0.2865 (+0.44z)| lr 3.73e-04 | 322.43 ms | 52.3% bf16 MFU | 1625194 tok/s
step 8664/19560 | loss 3.437306 (-0.17z)| norm 0.2919 (+0.70z)| lr 3.73e-04 | 322.97 ms | 52.3% bf16 MFU | 1625100 tok/s
step 8665/19560 | loss 3.393809 (-1.28z)| norm 0.2701 (-0.39z)| lr 3.73e-04 | 322.64 ms | 52.3% bf16 MFU | 1625094 tok/s
step 8666/19560 | loss 3.434295 (-0.24z)| norm 0.2864 (+0.41z)| lr 3.72e-04 | 322.35 ms | 52.4% bf16 MFU | 1625163 tok/s
step 8667/19560 | loss 3.377280 (-1.68z)| norm 0.2713 (-0.35z)| lr 3.72e-04 | 323.05 ms | 52.2% bf16 MFU | 1625052 tok/s
step 8668/19560 | loss 3.385245 (-1.45z)| norm 0.2651 (-0.66z)| lr 3.72e-04 | 322.34 ms | 52.4% bf16 MFU | 1625126 tok/s
step 8669/19560 | loss 3.498227 (+1.41z)| norm 0.2774 (-0.05z)| lr 3.72e-04 | 322.60 ms | 52.3% bf16 MFU | 1625130 tok/s
step 8670/19560 | loss 3.535441 (+2.33z)| norm 0.2559 (-1.12z)| lr 3.72e-04 | 322.76 ms | 52.3% bf16 MFU | 1625093 tok/s
step 8671/19560 | loss 3.416380 (-0.65z)| norm 0.2915 (+0.66z)| lr 3.72e-04 | 322.57 ms | 52.3% bf16 MFU | 1625105 tok/s
step 8672/19560 | loss 3.395637 (-1.16z)| norm 0.2451 (-1.67z)| lr 3.72e-04 | 322.36 ms | 52.4% bf16 MFU | 1625170 tok/s
step 8673/19560 | loss 3.432580 (-0.25z)| norm 0.2966 (+0.90z)| lr 3.72e-04 | 322.71 ms | 52.3% bf16 MFU | 1625144 tok/s
step 8674/19560 | loss 3.435187 (-0.18z)| norm 0.2662 (-0.64z)| lr 3.72e-04 | 322.49 ms | 52.3% bf16 MFU | 1625174 tok/s
step 8675/19560 | loss 3.459862 (+0.43z)| norm 0.2896 (+0.53z)| lr 3.72e-04 | 322.65 ms | 52.3% bf16 MFU | 1625163 tok/s
step 8676/19560 | loss 3.423178 (-0.49z)| norm 0.2769 (-0.12z)| lr 3.72e-04 | 322.96 ms | 52.3% bf16 MFU | 1625074 tok/s
step 8677/19560 | loss 3.485338 (+1.06z)| norm 0.2973 (+0.91z)| lr 3.72e-04 | 322.42 ms | 52.3% bf16 MFU | 1625125 tok/s
step 8678/19560 | loss 3.421507 (-0.54z)| norm 0.2828 (+0.17z)| lr 3.72e-04 | 322.65 ms | 52.3% bf16 MFU | 1625117 tok/s
step 8679/19560 | loss 3.426128 (-0.43z)| norm 0.2850 (+0.29z)| lr 3.72e-04 | 322.78 ms | 52.3% bf16 MFU | 1625076 tok/s
step 8680/19560 | loss 3.322441 (-2.92z)| norm 0.4592 (+7.16z)| lr 3.72e-04 | 322.80 ms | 52.3% bf16 MFU | 1625033 tok/s
step 8681/19560 | loss 3.407261 (-0.86z)| norm 0.3218 (+1.63z)| lr 3.72e-04 | 322.79 ms | 52.3% bf16 MFU | 1624994 tok/s
step 8682/19560 | loss 3.415633 (-0.64z)| norm 0.2907 (+0.39z)| lr 3.72e-04 | 322.27 ms | 52.4% bf16 MFU | 1625086 tok/s
step 8683/19560 | loss 3.511847 (+1.79z)| norm 0.2687 (-0.49z)| lr 3.72e-04 | 322.38 ms | 52.4% bf16 MFU | 1625147 tok/s
step 8684/19560 | loss 3.443531 (+0.05z)| norm 0.2961 (+0.61z)| lr 3.72e-04 | 323.01 ms | 52.3% bf16 MFU | 1625047 tok/s
step 8685/19560 | loss 3.417397 (-0.61z)| norm 0.2586 (-0.89z)| lr 3.72e-04 | 322.34 ms | 52.4% bf16 MFU | 1625120 tok/s
step 8686/19560 | loss 3.499760 (+1.46z)| norm 0.2939 (+0.52z)| lr 3.72e-04 | 322.74 ms | 52.3% bf16 MFU | 1625089 tok/s
step 8687/19560 | loss 3.403970 (-0.96z)| norm 0.2558 (-1.01z)| lr 3.71e-04 | 322.60 ms | 52.3% bf16 MFU | 1625093 tok/s
step 8688/19560 | loss 3.434091 (-0.20z)| norm 0.2643 (-0.66z)| lr 3.71e-04 | 322.70 ms | 52.3% bf16 MFU | 1625072 tok/s
step 8689/19560 | loss 3.467283 (+0.64z)| norm 0.2866 (+0.23z)| lr 3.71e-04 | 321.89 ms | 52.4% bf16 MFU | 1625257 tok/s
step 8690/19560 | loss 3.403867 (-0.97z)| norm 0.2649 (-0.65z)| lr 3.71e-04 | 322.92 ms | 52.3% bf16 MFU | 1625172 tok/s
step 8691/19560 | loss 3.469811 (+0.70z)| norm 0.2825 (+0.06z)| lr 3.71e-04 | 322.83 ms | 52.3% bf16 MFU | 1625116 tok/s
step 8692/19560 | loss 3.442722 (+0.01z)| norm 0.2445 (-1.46z)| lr 3.71e-04 | 322.95 ms | 52.3% bf16 MFU | 1625032 tok/s
step 8693/19560 | loss 3.438106 (-0.11z)| norm 0.2753 (-0.22z)| lr 3.71e-04 | 322.85 ms | 52.3% bf16 MFU | 1624979 tok/s
step 8694/19560 | loss 3.454275 (+0.31z)| norm 0.2508 (-1.19z)| lr 3.71e-04 | 322.31 ms | 52.4% bf16 MFU | 1625062 tok/s
step 8695/19560 | loss 3.464174 (+0.55z)| norm 0.2605 (-0.80z)| lr 3.71e-04 | 322.65 ms | 52.3% bf16 MFU | 1625057 tok/s
step 8696/19560 | loss 3.499264 (+1.42z)| norm 0.2587 (-0.86z)| lr 3.71e-04 | 323.35 ms | 52.2% bf16 MFU | 1624874 tok/s
step 8697/19560 | loss 3.412081 (-0.79z)| norm 0.2474 (-1.28z)| lr 3.71e-04 | 322.30 ms | 52.4% bf16 MFU | 1624966 tok/s
step 8698/19560 | loss 3.437270 (-0.15z)| norm 0.2726 (-0.30z)| lr 3.71e-04 | 322.96 ms | 52.3% bf16 MFU | 1624887 tok/s
step 8699/19560 | loss 3.454540 (+0.28z)| norm 0.2529 (-1.06z)| lr 3.71e-04 | 322.63 ms | 52.3% bf16 MFU | 1624894 tok/s
step 8700/19560 | loss 3.474526 (+0.79z)| norm 0.2656 (-0.55z)| lr 3.71e-04 | 322.84 ms | 52.3% bf16 MFU | 1624850 tok/s
step 8701/19560 | loss 3.472495 (+0.73z)| norm 0.2709 (-0.33z)| lr 3.71e-04 | 322.66 ms | 52.3% bf16 MFU | 1624851 tok/s
step 8702/19560 | loss 3.432425 (-0.27z)| norm 0.2673 (-0.47z)| lr 3.71e-04 | 322.50 ms | 52.3% bf16 MFU | 1624894 tok/s
step 8703/19560 | loss 3.441824 (-0.03z)| norm 0.2656 (-0.53z)| lr 3.71e-04 | 322.69 ms | 52.3% bf16 MFU | 1624886 tok/s
step 8704/19560 | loss 3.428979 (-0.36z)| norm 0.2826 (+0.14z)| lr 3.71e-04 | 322.21 ms | 52.4% bf16 MFU | 1625001 tok/s
step 8705/19560 | loss 3.379802 (-1.59z)| norm 0.2998 (+0.82z)| lr 3.71e-04 | 322.56 ms | 52.3% bf16 MFU | 1625020 tok/s
step 8706/19560 | loss 3.553038 (+2.70z)| norm 0.3310 (+2.05z)| lr 3.71e-04 | 322.59 ms | 52.3% bf16 MFU | 1625031 tok/s
step 8707/19560 | loss 3.449639 (+0.15z)| norm 0.2985 (+0.76z)| lr 3.70e-04 | 322.37 ms | 52.4% bf16 MFU | 1625098 tok/s
step 8708/19560 | loss 3.456744 (+0.32z)| norm 0.2872 (+0.32z)| lr 3.70e-04 | 322.74 ms | 52.3% bf16 MFU | 1625069 tok/s
step 8709/19560 | loss 3.473979 (+0.74z)| norm 0.2836 (+0.18z)| lr 3.70e-04 | 322.61 ms | 52.3% bf16 MFU | 1625072 tok/s
step 8710/19560 | loss 3.494655 (+1.27z)| norm 0.2841 (+0.20z)| lr 3.70e-04 | 322.62 ms | 52.3% bf16 MFU | 1625072 tok/s
step 8711/19560 | loss 3.510301 (+1.63z)| norm 0.2962 (+0.67z)| lr 3.70e-04 | 322.81 ms | 52.3% bf16 MFU | 1625025 tok/s
step 8712/19560 | loss 3.380667 (-1.54z)| norm 0.2831 (+0.15z)| lr 3.70e-04 | 322.69 ms | 52.3% bf16 MFU | 1625012 tok/s
step 8713/19560 | loss 3.456641 (+0.31z)| norm 0.2786 (-0.03z)| lr 3.70e-04 | 322.62 ms | 52.3% bf16 MFU | 1625016 tok/s
step 8714/19560 | loss 3.435593 (-0.19z)| norm 0.2819 (+0.10z)| lr 3.70e-04 | 322.53 ms | 52.3% bf16 MFU | 1625042 tok/s
step 8715/19560 | loss 3.511577 (+1.67z)| norm 0.2931 (+0.54z)| lr 3.70e-04 | 322.56 ms | 52.3% bf16 MFU | 1625059 tok/s
step 8716/19560 | loss 3.480107 (+0.89z)| norm 0.2839 (+0.16z)| lr 3.70e-04 | 322.51 ms | 52.3% bf16 MFU | 1625088 tok/s
step 8717/19560 | loss 3.365321 (-1.91z)| norm 0.2820 (+0.14z)| lr 3.70e-04 | 322.24 ms | 52.4% bf16 MFU | 1625186 tok/s
step 8718/19560 | loss 3.502020 (+1.41z)| norm 0.2742 (-0.20z)| lr 3.70e-04 | 322.09 ms | 52.4% bf16 MFU | 1625315 tok/s
step 8719/19560 | loss 3.441332 (-0.06z)| norm 0.2753 (-0.15z)| lr 3.70e-04 | 322.56 ms | 52.3% bf16 MFU | 1625318 tok/s
step 8720/19560 | loss 3.485180 (+0.99z)| norm 0.2655 (-0.58z)| lr 3.70e-04 | 322.45 ms | 52.3% bf16 MFU | 1625349 tok/s
step 8721/19560 | loss 3.431324 (-0.30z)| norm 0.2732 (-0.23z)| lr 3.70e-04 | 322.42 ms | 52.3% bf16 MFU | 1625387 tok/s
step 8722/19560 | loss 3.514557 (+1.69z)| norm 0.2526 (-1.15z)| lr 3.70e-04 | 322.95 ms | 52.3% bf16 MFU | 1625288 tok/s
step 8723/19560 | loss 3.421293 (-0.55z)| norm 0.2585 (-0.88z)| lr 3.70e-04 | 322.70 ms | 52.3% bf16 MFU | 1625258 tok/s
step 8724/19560 | loss 3.423205 (-0.50z)| norm 0.2582 (-0.88z)| lr 3.70e-04 | 322.63 ms | 52.3% bf16 MFU | 1625248 tok/s
step 8725/19560 | loss 3.430664 (-0.31z)| norm 0.2543 (-1.05z)| lr 3.70e-04 | 322.75 ms | 52.3% bf16 MFU | 1625207 tok/s
step 8726/19560 | loss 3.456556 (+0.34z)| norm 0.2600 (-0.79z)| lr 3.70e-04 | 322.37 ms | 52.4% bf16 MFU | 1625264 tok/s
step 8727/19560 | loss 3.502815 (+1.47z)| norm 0.2565 (-0.94z)| lr 3.70e-04 | 322.22 ms | 52.4% bf16 MFU | 1625357 tok/s
step 8728/19560 | loss 3.464571 (+0.52z)| norm 0.2749 (-0.13z)| lr 3.69e-04 | 322.57 ms | 52.3% bf16 MFU | 1625356 tok/s
step 8729/19560 | loss 3.461939 (+0.45z)| norm 0.2574 (-0.90z)| lr 3.69e-04 | 322.37 ms | 52.4% bf16 MFU | 1625408 tok/s
step 8730/19560 | loss 3.406213 (-0.92z)| norm 0.2597 (-0.79z)| lr 3.69e-04 | 323.17 ms | 52.2% bf16 MFU | 1625254 tok/s
step 8731/19560 | loss 3.413454 (-0.75z)| norm 0.2650 (-0.56z)| lr 3.69e-04 | 322.63 ms | 52.3% bf16 MFU | 1625244 tok/s
step 8732/19560 | loss 3.492554 (+1.18z)| norm 0.2834 (+0.24z)| lr 3.69e-04 | 322.51 ms | 52.3% bf16 MFU | 1625264 tok/s
step 8733/19560 | loss 3.426785 (-0.42z)| norm 0.2376 (-1.77z)| lr 3.69e-04 | 322.87 ms | 52.3% bf16 MFU | 1625192 tok/s
step 8734/19560 | loss 3.415202 (-0.70z)| norm 0.2730 (-0.21z)| lr 3.69e-04 | 322.55 ms | 52.3% bf16 MFU | 1625205 tok/s
step 8735/19560 | loss 3.469146 (+0.62z)| norm 0.2477 (-1.31z)| lr 3.69e-04 | 323.44 ms | 52.2% bf16 MFU | 1624993 tok/s
step 8736/19560 | loss 3.440381 (-0.09z)| norm 0.2527 (-1.08z)| lr 3.69e-04 | 322.62 ms | 52.3% bf16 MFU | 1624999 tok/s
step 8737/19560 | loss 3.391716 (-1.29z)| norm 0.2751 (-0.11z)| lr 3.69e-04 | 322.58 ms | 52.3% bf16 MFU | 1625015 tok/s
step 8738/19560 | loss 3.384222 (-1.45z)| norm 0.2726 (-0.21z)| lr 3.69e-04 | 322.42 ms | 52.3% bf16 MFU | 1625070 tok/s
step 8739/19560 | loss 3.437695 (-0.15z)| norm 0.2672 (-0.45z)| lr 3.69e-04 | 322.66 ms | 52.3% bf16 MFU | 1625062 tok/s
step 8740/19560 | loss 3.411296 (-0.78z)| norm 0.2809 (+0.15z)| lr 3.69e-04 | 322.61 ms | 52.3% bf16 MFU | 1625067 tok/s
step 8741/19560 | loss 3.489194 (+1.16z)| norm 0.2733 (-0.18z)| lr 3.69e-04 | 323.20 ms | 52.2% bf16 MFU | 1624922 tok/s
step 8742/19560 | loss 3.400721 (-1.04z)| norm 0.2866 (+0.40z)| lr 3.69e-04 | 322.28 ms | 52.4% bf16 MFU | 1625016 tok/s
step 8743/19560 | loss 3.363865 (-1.91z)| norm 0.2524 (-1.08z)| lr 3.69e-04 | 322.76 ms | 52.3% bf16 MFU | 1624985 tok/s
step 8744/19560 | loss 3.559370 (+2.81z)| norm 0.2963 (+0.83z)| lr 3.69e-04 | 322.57 ms | 52.3% bf16 MFU | 1625004 tok/s
step 8745/19560 | loss 3.425246 (-0.40z)| norm 0.2766 (-0.02z)| lr 3.69e-04 | 322.73 ms | 52.3% bf16 MFU | 1624981 tok/s
step 8746/19560 | loss 3.441261 (-0.02z)| norm 0.2723 (-0.20z)| lr 3.69e-04 | 322.59 ms | 52.3% bf16 MFU | 1624994 tok/s
step 8747/19560 | loss 3.439434 (-0.07z)| norm 0.2733 (-0.16z)| lr 3.69e-04 | 322.56 ms | 52.3% bf16 MFU | 1625013 tok/s
step 8748/19560 | loss 3.482214 (+0.99z)| norm 0.2849 (+0.35z)| lr 3.69e-04 | 322.93 ms | 52.3% bf16 MFU | 1624940 tok/s
step 8749/19560 | loss 3.460958 (+0.46z)| norm 0.2574 (-0.84z)| lr 3.68e-04 | 322.54 ms | 52.3% bf16 MFU | 1624967 tok/s
step 8750/19560 | loss 3.427425 (-0.37z)| norm 0.2872 (+0.46z)| lr 3.68e-04 | 322.60 ms | 52.3% bf16 MFU | 1624979 tok/s
val loss 3.428181
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2876/10042 = 0.286397
step 8751/19560 | loss 3.407825 (-0.84z)| norm 0.2922 (+0.68z)| lr 3.68e-04 | 322.18 ms | 52.4% bf16 MFU | 1625095 tok/s
step 8752/19560 | loss 3.471816 (+0.74z)| norm 0.3002 (+1.02z)| lr 3.68e-04 | 322.61 ms | 52.3% bf16 MFU | 1625097 tok/s
step 8753/19560 | loss 3.425977 (-0.40z)| norm 0.2710 (-0.25z)| lr 3.68e-04 | 322.29 ms | 52.4% bf16 MFU | 1625180 tok/s
step 8754/19560 | loss 3.435452 (-0.16z)| norm 0.2863 (+0.42z)| lr 3.68e-04 | 323.33 ms | 52.2% bf16 MFU | 1624998 tok/s
step 8755/19560 | loss 3.424991 (-0.44z)| norm 0.2684 (-0.37z)| lr 3.68e-04 | 323.04 ms | 52.2% bf16 MFU | 1624897 tok/s
step 8756/19560 | loss 3.469100 (+0.66z)| norm 0.2937 (+0.75z)| lr 3.68e-04 | 322.59 ms | 52.3% bf16 MFU | 1624916 tok/s
step 8757/19560 | loss 3.408222 (-0.86z)| norm 0.2731 (-0.16z)| lr 3.68e-04 | 322.74 ms | 52.3% bf16 MFU | 1624894 tok/s
step 8758/19560 | loss 3.427280 (-0.38z)| norm 0.2686 (-0.36z)| lr 3.68e-04 | 322.44 ms | 52.3% bf16 MFU | 1624949 tok/s
step 8759/19560 | loss 3.445343 (+0.08z)| norm 0.2811 (+0.18z)| lr 3.68e-04 | 322.87 ms | 52.3% bf16 MFU | 1624894 tok/s
step 8760/19560 | loss 3.392836 (-1.22z)| norm 0.2583 (-0.82z)| lr 3.68e-04 | 322.53 ms | 52.3% bf16 MFU | 1624928 tok/s
step 8761/19560 | loss 3.408125 (-0.83z)| norm 0.2912 (+0.63z)| lr 3.68e-04 | 322.18 ms | 52.4% bf16 MFU | 1625046 tok/s
step 8762/19560 | loss 3.378864 (-1.54z)| norm 0.2499 (-1.17z)| lr 3.68e-04 | 322.42 ms | 52.3% bf16 MFU | 1625100 tok/s
step 8763/19560 | loss 3.384586 (-1.37z)| norm 0.2968 (+0.87z)| lr 3.68e-04 | 322.24 ms | 52.4% bf16 MFU | 1625196 tok/s
step 8764/19560 | loss 3.365040 (-1.83z)| norm 0.3355 (+2.49z)| lr 3.68e-04 | 323.57 ms | 52.2% bf16 MFU | 1624953 tok/s
step 8765/19560 | loss 3.450251 (+0.25z)| norm 0.2981 (+0.87z)| lr 3.68e-04 | 323.05 ms | 52.2% bf16 MFU | 1624852 tok/s
step 8766/19560 | loss 3.494607 (+1.32z)| norm 0.3414 (+2.63z)| lr 3.68e-04 | 322.50 ms | 52.3% bf16 MFU | 1624894 tok/s
step 8767/19560 | loss 3.462483 (+0.54z)| norm 0.2741 (-0.18z)| lr 3.68e-04 | 322.88 ms | 52.3% bf16 MFU | 1624839 tok/s
step 8768/19560 | loss 3.417066 (-0.56z)| norm 0.3120 (+1.38z)| lr 3.68e-04 | 322.80 ms | 52.3% bf16 MFU | 1624807 tok/s
step 8769/19560 | loss 3.460970 (+0.50z)| norm 0.2842 (+0.23z)| lr 3.67e-04 | 323.28 ms | 52.2% bf16 MFU | 1624655 tok/s
step 8770/19560 | loss 3.511663 (+1.71z)| norm 0.2764 (-0.09z)| lr 3.67e-04 | 322.42 ms | 52.3% bf16 MFU | 1624728 tok/s
step 8771/19560 | loss 3.444846 (+0.10z)| norm 0.2785 (-0.01z)| lr 3.67e-04 | 322.91 ms | 52.3% bf16 MFU | 1624674 tok/s
step 8772/19560 | loss 3.483955 (+1.03z)| norm 0.2863 (+0.31z)| lr 3.67e-04 | 322.32 ms | 52.4% bf16 MFU | 1624772 tok/s
step 8773/19560 | loss 3.509178 (+1.61z)| norm 0.3006 (+0.89z)| lr 3.67e-04 | 322.56 ms | 52.3% bf16 MFU | 1624804 tok/s
step 8774/19560 | loss 3.451959 (+0.22z)| norm 0.3280 (+1.98z)| lr 3.67e-04 | 322.32 ms | 52.4% bf16 MFU | 1624895 tok/s
step 8775/19560 | loss 3.384933 (-1.39z)| norm 0.2513 (-1.15z)| lr 3.67e-04 | 322.43 ms | 52.3% bf16 MFU | 1624952 tok/s
step 8776/19560 | loss 3.483584 (+1.00z)| norm 0.3264 (+1.87z)| lr 3.67e-04 | 322.08 ms | 52.4% bf16 MFU | 1625096 tok/s
step 8777/19560 | loss 3.413937 (-0.67z)| norm 0.2745 (-0.21z)| lr 3.67e-04 | 322.92 ms | 52.3% bf16 MFU | 1625019 tok/s
step 8778/19560 | loss 3.408093 (-0.80z)| norm 0.2865 (+0.27z)| lr 3.67e-04 | 322.46 ms | 52.3% bf16 MFU | 1625063 tok/s
step 8779/19560 | loss 3.466022 (+0.59z)| norm 0.2631 (-0.66z)| lr 3.67e-04 | 322.27 ms | 52.4% bf16 MFU | 1625153 tok/s
step 8780/19560 | loss 3.438791 (-0.07z)| norm 0.2841 (+0.18z)| lr 3.67e-04 | 323.06 ms | 52.2% bf16 MFU | 1625039 tok/s
step 8781/19560 | loss 3.466761 (+0.59z)| norm 0.2797 (+0.00z)| lr 3.67e-04 | 322.75 ms | 52.3% bf16 MFU | 1625009 tok/s
step 8782/19560 | loss 3.371473 (-1.70z)| norm 0.3156 (+1.42z)| lr 3.67e-04 | 322.53 ms | 52.3% bf16 MFU | 1625036 tok/s
step 8783/19560 | loss 3.427069 (-0.36z)| norm 0.2754 (-0.18z)| lr 3.67e-04 | 322.78 ms | 52.3% bf16 MFU | 1624998 tok/s
step 8784/19560 | loss 3.503256 (+1.46z)| norm 0.2941 (+0.56z)| lr 3.67e-04 | 322.45 ms | 52.3% bf16 MFU | 1625045 tok/s
step 8785/19560 | loss 3.487238 (+1.06z)| norm 0.2845 (+0.19z)| lr 3.67e-04 | 323.00 ms | 52.3% bf16 MFU | 1624952 tok/s
step 8786/19560 | loss 3.497473 (+1.28z)| norm 0.2821 (+0.09z)| lr 3.67e-04 | 322.90 ms | 52.3% bf16 MFU | 1624889 tok/s
step 8787/19560 | loss 3.477722 (+0.80z)| norm 0.2908 (+0.45z)| lr 3.67e-04 | 322.91 ms | 52.3% bf16 MFU | 1624827 tok/s
step 8788/19560 | loss 3.478541 (+0.80z)| norm 0.2616 (-0.73z)| lr 3.67e-04 | 322.87 ms | 52.3% bf16 MFU | 1624779 tok/s
step 8789/19560 | loss 3.342118 (-2.37z)| norm 0.2694 (-0.41z)| lr 3.67e-04 | 322.95 ms | 52.3% bf16 MFU | 1624710 tok/s
step 8790/19560 | loss 3.470212 (+0.62z)| norm 0.2722 (-0.29z)| lr 3.66e-04 | 323.18 ms | 52.2% bf16 MFU | 1624590 tok/s
step 8791/19560 | loss 3.393988 (-1.14z)| norm 0.2596 (-0.79z)| lr 3.66e-04 | 322.53 ms | 52.3% bf16 MFU | 1624638 tok/s
step 8792/19560 | loss 3.497056 (+1.24z)| norm 0.2735 (-0.22z)| lr 3.66e-04 | 322.56 ms | 52.3% bf16 MFU | 1624677 tok/s
step 8793/19560 | loss 3.441606 (-0.05z)| norm 0.2946 (+0.61z)| lr 3.66e-04 | 322.96 ms | 52.3% bf16 MFU | 1624612 tok/s
step 8794/19560 | loss 3.452300 (+0.19z)| norm 0.2784 (-0.03z)| lr 3.66e-04 | 323.04 ms | 52.2% bf16 MFU | 1624530 tok/s
step 8795/19560 | loss 3.470620 (+0.61z)| norm 0.2742 (-0.20z)| lr 3.66e-04 | 322.89 ms | 52.3% bf16 MFU | 1624491 tok/s
step 8796/19560 | loss 3.470849 (+0.60z)| norm 0.2983 (+0.75z)| lr 3.66e-04 | 322.80 ms | 52.3% bf16 MFU | 1624476 tok/s
step 8797/19560 | loss 3.409572 (-0.83z)| norm 0.2888 (+0.37z)| lr 3.66e-04 | 323.02 ms | 52.2% bf16 MFU | 1624407 tok/s
step 8798/19560 | loss 3.465658 (+0.52z)| norm 0.2793 (-0.02z)| lr 3.66e-04 | 322.68 ms | 52.3% bf16 MFU | 1624426 tok/s
step 8799/19560 | loss 3.395717 (-1.16z)| norm 0.2711 (-0.34z)| lr 3.66e-04 | 322.95 ms | 52.3% bf16 MFU | 1624377 tok/s
step 8800/19560 | loss 3.395290 (-1.17z)| norm 0.2750 (-0.20z)| lr 3.66e-04 | 322.48 ms | 52.3% bf16 MFU | 1624447 tok/s
step 8801/19560 | loss 3.435674 (-0.20z)| norm 0.2820 (+0.09z)| lr 3.66e-04 | 322.35 ms | 52.4% bf16 MFU | 1624547 tok/s
step 8802/19560 | loss 3.433220 (-0.26z)| norm 0.2990 (+0.77z)| lr 3.66e-04 | 322.26 ms | 52.4% bf16 MFU | 1624664 tok/s
step 8803/19560 | loss 3.415141 (-0.68z)| norm 0.2976 (+0.71z)| lr 3.66e-04 | 322.81 ms | 52.3% bf16 MFU | 1624639 tok/s
step 8804/19560 | loss 3.413068 (-0.73z)| norm 0.2987 (+0.75z)| lr 3.66e-04 | 322.34 ms | 52.4% bf16 MFU | 1624732 tok/s
step 8805/19560 | loss 3.473010 (+0.71z)| norm 0.2909 (+0.44z)| lr 3.66e-04 | 322.62 ms | 52.3% bf16 MFU | 1624751 tok/s
step 8806/19560 | loss 3.476702 (+0.78z)| norm 0.2779 (-0.09z)| lr 3.66e-04 | 322.74 ms | 52.3% bf16 MFU | 1624738 tok/s
step 8807/19560 | loss 3.460063 (+0.38z)| norm 0.2884 (+0.33z)| lr 3.66e-04 | 322.30 ms | 52.4% bf16 MFU | 1624836 tok/s
step 8808/19560 | loss 3.439105 (-0.15z)| norm 0.2869 (+0.43z)| lr 3.66e-04 | 322.56 ms | 52.3% bf16 MFU | 1624864 tok/s
step 8809/19560 | loss 3.434112 (-0.28z)| norm 0.2797 (+0.07z)| lr 3.66e-04 | 322.26 ms | 52.4% bf16 MFU | 1624966 tok/s
step 8810/19560 | loss 3.468459 (+0.57z)| norm 0.4048 (+5.82z)| lr 3.65e-04 | 323.36 ms | 52.2% bf16 MFU | 1624787 tok/s
step 8811/19560 | loss 3.436867 (-0.21z)| norm 0.3710 (+3.95z)| lr 3.65e-04 | 322.87 ms | 52.3% bf16 MFU | 1624739 tok/s
step 8812/19560 | loss 3.445322 (+0.00z)| norm 0.3041 (+1.03z)| lr 3.65e-04 | 322.66 ms | 52.3% bf16 MFU | 1624748 tok/s
step 8813/19560 | loss 3.348622 (-2.37z)| norm 0.3340 (+2.27z)| lr 3.65e-04 | 322.54 ms | 52.3% bf16 MFU | 1624785 tok/s
step 8814/19560 | loss 3.441570 (-0.07z)| norm 0.2927 (+0.51z)| lr 3.65e-04 | 322.23 ms | 52.4% bf16 MFU | 1624898 tok/s
step 8815/19560 | loss 3.354689 (-2.18z)| norm 0.2729 (-0.35z)| lr 3.65e-04 | 323.01 ms | 52.2% bf16 MFU | 1624810 tok/s
step 8816/19560 | loss 3.469795 (+0.63z)| norm 0.2725 (-0.36z)| lr 3.65e-04 | 322.44 ms | 52.3% bf16 MFU | 1624871 tok/s
step 8817/19560 | loss 3.479502 (+0.86z)| norm 0.2682 (-0.54z)| lr 3.65e-04 | 323.10 ms | 52.2% bf16 MFU | 1624762 tok/s
step 8818/19560 | loss 3.528237 (+2.00z)| norm 0.2731 (-0.34z)| lr 3.65e-04 | 322.09 ms | 52.4% bf16 MFU | 1624912 tok/s
step 8819/19560 | loss 3.400769 (-1.05z)| norm 0.2699 (-0.47z)| lr 3.65e-04 | 322.32 ms | 52.4% bf16 MFU | 1624997 tok/s
step 8820/19560 | loss 3.461381 (+0.40z)| norm 0.2986 (+0.75z)| lr 3.65e-04 | 322.45 ms | 52.3% bf16 MFU | 1625046 tok/s
step 8821/19560 | loss 3.470102 (+0.60z)| norm 0.2736 (-0.33z)| lr 3.65e-04 | 322.41 ms | 52.3% bf16 MFU | 1625102 tok/s
step 8822/19560 | loss 3.464617 (+0.47z)| norm 0.2709 (-0.46z)| lr 3.65e-04 | 322.79 ms | 52.3% bf16 MFU | 1625059 tok/s
step 8823/19560 | loss 3.433441 (-0.27z)| norm 0.3123 (+1.32z)| lr 3.65e-04 | 323.13 ms | 52.2% bf16 MFU | 1624932 tok/s
step 8824/19560 | loss 3.589712 (+3.32z)| norm 0.2722 (-0.42z)| lr 3.65e-04 | 323.03 ms | 52.2% bf16 MFU | 1624837 tok/s
step 8825/19560 | loss 3.460890 (+0.35z)| norm 0.2781 (-0.18z)| lr 3.65e-04 | 322.47 ms | 52.3% bf16 MFU | 1624887 tok/s
step 8826/19560 | loss 3.417469 (-0.65z)| norm 0.2527 (-1.27z)| lr 3.65e-04 | 322.52 ms | 52.3% bf16 MFU | 1624922 tok/s
step 8827/19560 | loss 3.521033 (+1.70z)| norm 0.2747 (-0.33z)| lr 3.65e-04 | 322.47 ms | 52.3% bf16 MFU | 1624968 tok/s
step 8828/19560 | loss 3.412377 (-0.76z)| norm 0.2780 (-0.19z)| lr 3.65e-04 | 322.28 ms | 52.4% bf16 MFU | 1625060 tok/s
step 8829/19560 | loss 3.420574 (-0.57z)| norm 0.2717 (-0.46z)| lr 3.65e-04 | 322.72 ms | 52.3% bf16 MFU | 1625037 tok/s
step 8830/19560 | loss 3.393234 (-1.18z)| norm 0.2478 (-1.49z)| lr 3.65e-04 | 322.67 ms | 52.3% bf16 MFU | 1625027 tok/s
step 8831/19560 | loss 3.448792 (+0.08z)| norm 0.2687 (-0.58z)| lr 3.64e-04 | 322.46 ms | 52.3% bf16 MFU | 1625071 tok/s
step 8832/19560 | loss 3.413512 (-0.71z)| norm 0.2806 (-0.07z)| lr 3.64e-04 | 322.62 ms | 52.3% bf16 MFU | 1625073 tok/s
step 8833/19560 | loss 3.431435 (-0.32z)| norm 0.2918 (+0.43z)| lr 3.64e-04 | 322.60 ms | 52.3% bf16 MFU | 1625078 tok/s
step 8834/19560 | loss 3.385299 (-1.37z)| norm 0.2718 (-0.44z)| lr 3.64e-04 | 322.62 ms | 52.3% bf16 MFU | 1625079 tok/s
step 8835/19560 | loss 3.521108 (+1.75z)| norm 0.3257 (+1.93z)| lr 3.64e-04 | 323.05 ms | 52.2% bf16 MFU | 1624972 tok/s
step 8836/19560 | loss 3.539125 (+2.11z)| norm 0.2948 (+0.57z)| lr 3.64e-04 | 322.83 ms | 52.3% bf16 MFU | 1624926 tok/s
step 8837/19560 | loss 3.408901 (-0.81z)| norm 0.2834 (+0.07z)| lr 3.64e-04 | 322.38 ms | 52.4% bf16 MFU | 1624995 tok/s
step 8838/19560 | loss 3.465849 (+0.48z)| norm 0.2991 (+0.75z)| lr 3.64e-04 | 322.23 ms | 52.4% bf16 MFU | 1625099 tok/s
step 8839/19560 | loss 3.431461 (-0.29z)| norm 0.2756 (-0.27z)| lr 3.64e-04 | 322.55 ms | 52.3% bf16 MFU | 1625116 tok/s
step 8840/19560 | loss 3.412944 (-0.72z)| norm 0.2790 (-0.12z)| lr 3.64e-04 | 323.83 ms | 52.1% bf16 MFU | 1624812 tok/s
step 8841/19560 | loss 3.470102 (+0.59z)| norm 0.2769 (-0.21z)| lr 3.64e-04 | 322.13 ms | 52.4% bf16 MFU | 1624950 tok/s
step 8842/19560 | loss 3.477692 (+0.76z)| norm 0.2689 (-0.56z)| lr 3.64e-04 | 322.01 ms | 52.4% bf16 MFU | 1625111 tok/s
step 8843/19560 | loss 3.411615 (-0.74z)| norm 0.2640 (-0.76z)| lr 3.64e-04 | 322.58 ms | 52.3% bf16 MFU | 1625119 tok/s
step 8844/19560 | loss 3.414520 (-0.67z)| norm 0.2866 (+0.23z)| lr 3.64e-04 | 322.51 ms | 52.3% bf16 MFU | 1625146 tok/s
step 8845/19560 | loss 3.395822 (-1.11z)| norm 0.2669 (-0.63z)| lr 3.64e-04 | 322.38 ms | 52.4% bf16 MFU | 1625204 tok/s
step 8846/19560 | loss 3.454199 (+0.26z)| norm 0.2785 (-0.13z)| lr 3.64e-04 | 322.60 ms | 52.3% bf16 MFU | 1625205 tok/s
step 8847/19560 | loss 3.413811 (-0.68z)| norm 0.2957 (+0.62z)| lr 3.64e-04 | 322.23 ms | 52.4% bf16 MFU | 1625297 tok/s
step 8848/19560 | loss 3.480820 (+0.88z)| norm 0.2716 (-0.43z)| lr 3.64e-04 | 322.55 ms | 52.3% bf16 MFU | 1625304 tok/s
step 8849/19560 | loss 3.501454 (+1.35z)| norm 0.2718 (-0.43z)| lr 3.64e-04 | 322.90 ms | 52.3% bf16 MFU | 1625223 tok/s
step 8850/19560 | loss 3.475128 (+0.75z)| norm 0.2965 (+0.64z)| lr 3.64e-04 | 322.89 ms | 52.3% bf16 MFU | 1625149 tok/s
step 8851/19560 | loss 3.409016 (-0.80z)| norm 0.2791 (-0.13z)| lr 3.63e-04 | 322.84 ms | 52.3% bf16 MFU | 1625091 tok/s
step 8852/19560 | loss 3.495074 (+1.20z)| norm 0.2607 (-0.94z)| lr 3.63e-04 | 322.56 ms | 52.3% bf16 MFU | 1625107 tok/s
step 8853/19560 | loss 3.462538 (+0.43z)| norm 0.3030 (+0.91z)| lr 3.63e-04 | 322.36 ms | 52.4% bf16 MFU | 1625172 tok/s
step 8854/19560 | loss 3.383745 (-1.38z)| norm 0.2418 (-1.78z)| lr 3.63e-04 | 322.61 ms | 52.3% bf16 MFU | 1625171 tok/s
step 8855/19560 | loss 3.397996 (-1.03z)| norm 0.2855 (+0.13z)| lr 3.63e-04 | 322.39 ms | 52.3% bf16 MFU | 1625225 tok/s
step 8856/19560 | loss 3.388087 (-1.24z)| norm 0.2421 (-1.75z)| lr 3.63e-04 | 322.29 ms | 52.4% bf16 MFU | 1625301 tok/s
step 8857/19560 | loss 3.392888 (-1.11z)| norm 0.2765 (-0.26z)| lr 3.63e-04 | 322.65 ms | 52.3% bf16 MFU | 1625284 tok/s
step 8858/19560 | loss 3.495675 (+1.23z)| norm 0.2776 (-0.22z)| lr 3.63e-04 | 322.55 ms | 52.3% bf16 MFU | 1625293 tok/s
step 8859/19560 | loss 3.480313 (+0.86z)| norm 0.2779 (-0.21z)| lr 3.63e-04 | 322.40 ms | 52.3% bf16 MFU | 1625339 tok/s
step 8860/19560 | loss 3.425961 (-0.37z)| norm 0.2828 (+0.00z)| lr 3.63e-04 | 322.72 ms | 52.3% bf16 MFU | 1625300 tok/s
step 8861/19560 | loss 3.493790 (+1.17z)| norm 0.2713 (-0.52z)| lr 3.63e-04 | 322.58 ms | 52.3% bf16 MFU | 1625299 tok/s
step 8862/19560 | loss 3.438894 (-0.09z)| norm 0.2711 (-0.53z)| lr 3.63e-04 | 322.57 ms | 52.3% bf16 MFU | 1625300 tok/s
step 8863/19560 | loss 3.425257 (-0.39z)| norm 0.2818 (-0.07z)| lr 3.63e-04 | 322.42 ms | 52.3% bf16 MFU | 1625341 tok/s
step 8864/19560 | loss 3.440145 (-0.05z)| norm 0.2625 (-0.94z)| lr 3.63e-04 | 322.65 ms | 52.3% bf16 MFU | 1625320 tok/s
step 8865/19560 | loss 3.443184 (+0.01z)| norm 0.2723 (-0.50z)| lr 3.63e-04 | 322.37 ms | 52.4% bf16 MFU | 1625372 tok/s
step 8866/19560 | loss 3.460804 (+0.40z)| norm 0.2480 (-1.57z)| lr 3.63e-04 | 322.65 ms | 52.3% bf16 MFU | 1625350 tok/s
step 8867/19560 | loss 3.433975 (-0.22z)| norm 0.2550 (-1.25z)| lr 3.63e-04 | 322.73 ms | 52.3% bf16 MFU | 1625309 tok/s
step 8868/19560 | loss 3.418096 (-0.59z)| norm 0.2865 (+0.15z)| lr 3.63e-04 | 322.19 ms | 52.4% bf16 MFU | 1625406 tok/s
step 8869/19560 | loss 3.405058 (-0.88z)| norm 0.2732 (-0.44z)| lr 3.63e-04 | 323.14 ms | 52.2% bf16 MFU | 1625260 tok/s
step 8870/19560 | loss 3.404686 (-0.89z)| norm 0.2723 (-0.48z)| lr 3.63e-04 | 322.98 ms | 52.3% bf16 MFU | 1625161 tok/s
step 8871/19560 | loss 3.409397 (-0.80z)| norm 0.2753 (-0.35z)| lr 3.63e-04 | 322.62 ms | 52.3% bf16 MFU | 1625157 tok/s
step 8872/19560 | loss 3.402478 (-0.95z)| norm 0.2761 (-0.31z)| lr 3.62e-04 | 322.40 ms | 52.3% bf16 MFU | 1625210 tok/s
step 8873/19560 | loss 3.383002 (-1.41z)| norm 0.2676 (-0.68z)| lr 3.62e-04 | 322.58 ms | 52.3% bf16 MFU | 1625214 tok/s
step 8874/19560 | loss 3.442644 (+0.02z)| norm 0.2719 (-0.49z)| lr 3.62e-04 | 323.08 ms | 52.2% bf16 MFU | 1625092 tok/s
step 8875/19560 | loss 3.366915 (-1.76z)| norm 0.2701 (-0.57z)| lr 3.62e-04 | 322.04 ms | 52.4% bf16 MFU | 1625239 tok/s
step 8876/19560 | loss 3.424840 (-0.38z)| norm 0.2849 (+0.09z)| lr 3.62e-04 | 322.42 ms | 52.3% bf16 MFU | 1625282 tok/s
step 8877/19560 | loss 3.452245 (+0.28z)| norm 0.2743 (-0.39z)| lr 3.62e-04 | 323.03 ms | 52.2% bf16 MFU | 1625169 tok/s
step 8878/19560 | loss 3.432686 (-0.19z)| norm 0.2624 (-0.92z)| lr 3.62e-04 | 322.16 ms | 52.4% bf16 MFU | 1625282 tok/s
step 8879/19560 | loss 3.452299 (+0.27z)| norm 0.2645 (-0.81z)| lr 3.62e-04 | 322.35 ms | 52.4% bf16 MFU | 1625341 tok/s
step 8880/19560 | loss 3.453877 (+0.31z)| norm 0.2634 (-0.84z)| lr 3.62e-04 | 323.14 ms | 52.2% bf16 MFU | 1625197 tok/s
step 8881/19560 | loss 3.461178 (+0.48z)| norm 0.2586 (-1.05z)| lr 3.62e-04 | 322.44 ms | 52.3% bf16 MFU | 1625238 tok/s
step 8882/19560 | loss 3.416324 (-0.59z)| norm 0.2625 (-0.87z)| lr 3.62e-04 | 322.13 ms | 52.4% bf16 MFU | 1625354 tok/s
step 8883/19560 | loss 3.389269 (-1.22z)| norm 0.2645 (-0.78z)| lr 3.62e-04 | 322.31 ms | 52.4% bf16 MFU | 1625420 tok/s
step 8884/19560 | loss 3.498570 (+1.36z)| norm 0.3017 (+0.87z)| lr 3.62e-04 | 322.62 ms | 52.3% bf16 MFU | 1625404 tok/s
step 8885/19560 | loss 3.391124 (-1.17z)| norm 0.2686 (-0.60z)| lr 3.62e-04 | 322.26 ms | 52.4% bf16 MFU | 1625481 tok/s
step 8886/19560 | loss 3.403497 (-0.87z)| norm 0.2683 (-0.61z)| lr 3.62e-04 | 322.67 ms | 52.3% bf16 MFU | 1625450 tok/s
step 8887/19560 | loss 3.475668 (+0.81z)| norm 0.2694 (-0.56z)| lr 3.62e-04 | 322.60 ms | 52.3% bf16 MFU | 1625436 tok/s
step 8888/19560 | loss 3.508762 (+1.56z)| norm 0.2610 (-0.93z)| lr 3.62e-04 | 322.51 ms | 52.3% bf16 MFU | 1625448 tok/s
step 8889/19560 | loss 3.418881 (-0.54z)| norm 0.2652 (-0.73z)| lr 3.62e-04 | 322.41 ms | 52.3% bf16 MFU | 1625484 tok/s
step 8890/19560 | loss 3.405162 (-0.86z)| norm 0.2884 (+0.28z)| lr 3.62e-04 | 322.44 ms | 52.3% bf16 MFU | 1625510 tok/s
step 8891/19560 | loss 3.349230 (-2.14z)| norm 0.2861 (+0.18z)| lr 3.62e-04 | 322.68 ms | 52.3% bf16 MFU | 1625473 tok/s
step 8892/19560 | loss 3.415139 (-0.63z)| norm 0.3081 (+1.20z)| lr 3.61e-04 | 322.48 ms | 52.3% bf16 MFU | 1625488 tok/s
step 8893/19560 | loss 3.410538 (-0.73z)| norm 0.2601 (-0.97z)| lr 3.61e-04 | 322.81 ms | 52.3% bf16 MFU | 1625420 tok/s
step 8894/19560 | loss 3.443739 (+0.05z)| norm 0.2787 (-0.10z)| lr 3.61e-04 | 322.78 ms | 52.3% bf16 MFU | 1625362 tok/s
step 8895/19560 | loss 3.432724 (-0.20z)| norm 0.2847 (+0.17z)| lr 3.61e-04 | 322.18 ms | 52.4% bf16 MFU | 1625459 tok/s
step 8896/19560 | loss 3.439173 (-0.05z)| norm 0.2786 (-0.10z)| lr 3.61e-04 | 322.67 ms | 52.3% bf16 MFU | 1625428 tok/s
step 8897/19560 | loss 3.469169 (+0.65z)| norm 0.2517 (-1.35z)| lr 3.61e-04 | 322.78 ms | 52.3% bf16 MFU | 1625371 tok/s
step 8898/19560 | loss 3.424594 (-0.39z)| norm 0.2591 (-0.99z)| lr 3.61e-04 | 322.53 ms | 52.3% bf16 MFU | 1625379 tok/s
step 8899/19560 | loss 3.407840 (-0.78z)| norm 0.2719 (-0.39z)| lr 3.61e-04 | 322.55 ms | 52.3% bf16 MFU | 1625382 tok/s
step 8900/19560 | loss 3.434444 (-0.14z)| norm 0.2517 (-1.31z)| lr 3.61e-04 | 323.05 ms | 52.2% bf16 MFU | 1625258 tok/s
step 8901/19560 | loss 3.416077 (-0.56z)| norm 0.2661 (-0.64z)| lr 3.61e-04 | 322.41 ms | 52.3% bf16 MFU | 1625302 tok/s
step 8902/19560 | loss 3.473243 (+0.81z)| norm 0.2705 (-0.42z)| lr 3.61e-04 | 322.26 ms | 52.4% bf16 MFU | 1625382 tok/s
step 8903/19560 | loss 3.387339 (-1.25z)| norm 0.2516 (-1.31z)| lr 3.61e-04 | 322.46 ms | 52.3% bf16 MFU | 1625408 tok/s
step 8904/19560 | loss 3.475152 (+0.86z)| norm 0.2777 (-0.06z)| lr 3.61e-04 | 323.98 ms | 52.1% bf16 MFU | 1625052 tok/s
step 8905/19560 | loss 3.426559 (-0.31z)| norm 0.2583 (-0.99z)| lr 3.61e-04 | 321.77 ms | 52.5% bf16 MFU | 1625269 tok/s
step 8906/19560 | loss 3.518939 (+1.87z)| norm 0.2683 (-0.50z)| lr 3.61e-04 | 323.02 ms | 52.2% bf16 MFU | 1625160 tok/s
step 8907/19560 | loss 3.461851 (+0.51z)| norm 0.2509 (-1.32z)| lr 3.61e-04 | 322.30 ms | 52.4% bf16 MFU | 1625238 tok/s
step 8908/19560 | loss 3.407568 (-0.78z)| norm 0.2789 (+0.02z)| lr 3.61e-04 | 323.26 ms | 52.2% bf16 MFU | 1625070 tok/s
step 8909/19560 | loss 3.405862 (-0.80z)| norm 0.2419 (-1.72z)| lr 3.61e-04 | 322.54 ms | 52.3% bf16 MFU | 1625091 tok/s
step 8910/19560 | loss 3.471332 (+0.74z)| norm 0.2491 (-1.36z)| lr 3.61e-04 | 322.35 ms | 52.4% bf16 MFU | 1625159 tok/s
step 8911/19560 | loss 3.448135 (+0.18z)| norm 0.2575 (-0.95z)| lr 3.61e-04 | 322.32 ms | 52.4% bf16 MFU | 1625231 tok/s
step 8912/19560 | loss 3.588362 (+3.39z)| norm 0.3025 (+1.17z)| lr 3.60e-04 | 322.38 ms | 52.4% bf16 MFU | 1625284 tok/s
step 8913/19560 | loss 3.444009 (+0.07z)| norm 0.2622 (-0.72z)| lr 3.60e-04 | 322.51 ms | 52.3% bf16 MFU | 1625303 tok/s
step 8914/19560 | loss 3.366094 (-1.70z)| norm 0.2878 (+0.48z)| lr 3.60e-04 | 322.28 ms | 52.4% bf16 MFU | 1625377 tok/s
step 8915/19560 | loss 3.446282 (+0.15z)| norm 0.2810 (+0.17z)| lr 3.60e-04 | 322.42 ms | 52.3% bf16 MFU | 1625413 tok/s
step 8916/19560 | loss 3.405958 (-0.77z)| norm 0.2943 (+0.78z)| lr 3.60e-04 | 322.73 ms | 52.3% bf16 MFU | 1625371 tok/s
step 8917/19560 | loss 3.406327 (-0.78z)| norm 0.2910 (+0.62z)| lr 3.60e-04 | 323.37 ms | 52.2% bf16 MFU | 1625169 tok/s
step 8918/19560 | loss 3.377916 (-1.43z)| norm 0.2757 (-0.10z)| lr 3.60e-04 | 322.87 ms | 52.3% bf16 MFU | 1625102 tok/s
step 8919/19560 | loss 3.503366 (+1.48z)| norm 0.2695 (-0.40z)| lr 3.60e-04 | 322.39 ms | 52.3% bf16 MFU | 1625159 tok/s
step 8920/19560 | loss 3.558239 (+2.69z)| norm 0.2752 (-0.13z)| lr 3.60e-04 | 323.57 ms | 52.2% bf16 MFU | 1624917 tok/s
step 8921/19560 | loss 3.440918 (+0.02z)| norm 0.2844 (+0.31z)| lr 3.60e-04 | 322.74 ms | 52.3% bf16 MFU | 1624896 tok/s
step 8922/19560 | loss 3.433374 (-0.15z)| norm 0.3127 (+1.62z)| lr 3.60e-04 | 322.11 ms | 52.4% bf16 MFU | 1625034 tok/s
step 8923/19560 | loss 3.442149 (+0.05z)| norm 0.2793 (+0.05z)| lr 3.60e-04 | 322.97 ms | 52.3% bf16 MFU | 1624949 tok/s
step 8924/19560 | loss 3.499680 (+1.35z)| norm 0.3119 (+1.56z)| lr 3.60e-04 | 323.03 ms | 52.2% bf16 MFU | 1624854 tok/s
step 8925/19560 | loss 3.450813 (+0.24z)| norm 0.3154 (+1.70z)| lr 3.60e-04 | 323.51 ms | 52.2% bf16 MFU | 1624643 tok/s
step 8926/19560 | loss 3.442741 (+0.06z)| norm 0.2681 (-0.47z)| lr 3.60e-04 | 322.66 ms | 52.3% bf16 MFU | 1624655 tok/s
step 8927/19560 | loss 3.439507 (-0.02z)| norm 0.2933 (+0.68z)| lr 3.60e-04 | 322.64 ms | 52.3% bf16 MFU | 1624673 tok/s
step 8928/19560 | loss 3.380825 (-1.36z)| norm 0.2802 (+0.07z)| lr 3.60e-04 | 322.62 ms | 52.3% bf16 MFU | 1624693 tok/s
step 8929/19560 | loss 3.386668 (-1.21z)| norm 0.2945 (+0.72z)| lr 3.60e-04 | 322.52 ms | 52.3% bf16 MFU | 1624739 tok/s
step 8930/19560 | loss 3.443421 (+0.07z)| norm 0.2580 (-0.94z)| lr 3.60e-04 | 322.82 ms | 52.3% bf16 MFU | 1624706 tok/s
step 8931/19560 | loss 3.407630 (-0.74z)| norm 0.3157 (+1.69z)| lr 3.60e-04 | 322.71 ms | 52.3% bf16 MFU | 1624702 tok/s
step 8932/19560 | loss 3.439260 (-0.02z)| norm 0.2683 (-0.46z)| lr 3.60e-04 | 322.75 ms | 52.3% bf16 MFU | 1624689 tok/s
step 8933/19560 | loss 3.419868 (-0.46z)| norm 0.2733 (-0.22z)| lr 3.59e-04 | 322.98 ms | 52.3% bf16 MFU | 1624619 tok/s
step 8934/19560 | loss 3.416588 (-0.52z)| norm 0.2729 (-0.24z)| lr 3.59e-04 | 322.47 ms | 52.3% bf16 MFU | 1624681 tok/s
step 8935/19560 | loss 3.507649 (+1.53z)| norm 0.3247 (+2.08z)| lr 3.59e-04 | 322.63 ms | 52.3% bf16 MFU | 1624698 tok/s
step 8936/19560 | loss 3.408108 (-0.71z)| norm 0.2862 (+0.35z)| lr 3.59e-04 | 322.63 ms | 52.3% bf16 MFU | 1624716 tok/s
step 8937/19560 | loss 3.409893 (-0.66z)| norm 0.3084 (+1.33z)| lr 3.59e-04 | 323.38 ms | 52.2% bf16 MFU | 1624544 tok/s
step 8938/19560 | loss 3.413041 (-0.58z)| norm 0.2851 (+0.38z)| lr 3.59e-04 | 323.61 ms | 52.2% bf16 MFU | 1624322 tok/s
step 8939/19560 | loss 3.506184 (+1.49z)| norm 0.2778 (+0.05z)| lr 3.59e-04 | 323.68 ms | 52.1% bf16 MFU | 1624095 tok/s
step 8940/19560 | loss 3.427557 (-0.26z)| norm 0.2794 (+0.15z)| lr 3.59e-04 | 322.97 ms | 52.3% bf16 MFU | 1624058 tok/s
step 8941/19560 | loss 3.389699 (-1.13z)| norm 0.2984 (+1.30z)| lr 3.59e-04 | 323.14 ms | 52.2% bf16 MFU | 1623978 tok/s
step 8942/19560 | loss 3.398412 (-0.92z)| norm 0.2920 (+0.92z)| lr 3.59e-04 | 322.44 ms | 52.3% bf16 MFU | 1624078 tok/s
step 8943/19560 | loss 3.412790 (-0.61z)| norm 0.2722 (-0.26z)| lr 3.59e-04 | 322.89 ms | 52.3% bf16 MFU | 1624062 tok/s
step 8944/19560 | loss 3.446712 (+0.16z)| norm 0.2873 (+0.63z)| lr 3.59e-04 | 323.06 ms | 52.2% bf16 MFU | 1624002 tok/s
step 8945/19560 | loss 3.436824 (-0.06z)| norm 0.2953 (+1.10z)| lr 3.59e-04 | 322.17 ms | 52.4% bf16 MFU | 1624170 tok/s
step 8946/19560 | loss 3.410948 (-0.64z)| norm 0.2754 (-0.09z)| lr 3.59e-04 | 323.72 ms | 52.1% bf16 MFU | 1623941 tok/s
step 8947/19560 | loss 3.417913 (-0.48z)| norm 0.2821 (+0.31z)| lr 3.59e-04 | 323.05 ms | 52.2% bf16 MFU | 1623892 tok/s
step 8948/19560 | loss 3.407659 (-0.71z)| norm 0.2665 (-0.61z)| lr 3.59e-04 | 322.89 ms | 52.3% bf16 MFU | 1623884 tok/s
step 8949/19560 | loss 3.468031 (+0.70z)| norm 0.2637 (-0.77z)| lr 3.59e-04 | 322.41 ms | 52.3% bf16 MFU | 1623997 tok/s
step 8950/19560 | loss 3.478948 (+0.95z)| norm 0.2736 (-0.18z)| lr 3.59e-04 | 322.62 ms | 52.3% bf16 MFU | 1624052 tok/s
step 8951/19560 | loss 3.424880 (-0.31z)| norm 0.2835 (+0.43z)| lr 3.59e-04 | 322.61 ms | 52.3% bf16 MFU | 1624106 tok/s
step 8952/19560 | loss 3.400236 (-0.89z)| norm 0.2595 (-1.02z)| lr 3.59e-04 | 323.11 ms | 52.2% bf16 MFU | 1624031 tok/s
step 8953/19560 | loss 3.477831 (+1.00z)| norm 0.2721 (-0.25z)| lr 3.58e-04 | 322.93 ms | 52.3% bf16 MFU | 1624006 tok/s
step 8954/19560 | loss 3.422528 (-0.35z)| norm 0.2786 (+0.13z)| lr 3.58e-04 | 322.63 ms | 52.3% bf16 MFU | 1624058 tok/s
step 8955/19560 | loss 3.415101 (-0.51z)| norm 0.2467 (-1.77z)| lr 3.58e-04 | 322.67 ms | 52.3% bf16 MFU | 1624098 tok/s
step 8956/19560 | loss 3.409843 (-0.64z)| norm 0.2741 (-0.13z)| lr 3.58e-04 | 322.55 ms | 52.3% bf16 MFU | 1624165 tok/s
step 8957/19560 | loss 3.392240 (-1.07z)| norm 0.2470 (-1.72z)| lr 3.58e-04 | 323.35 ms | 52.2% bf16 MFU | 1624027 tok/s
step 8958/19560 | loss 3.485990 (+1.22z)| norm 0.2685 (-0.46z)| lr 3.58e-04 | 322.65 ms | 52.3% bf16 MFU | 1624074 tok/s
step 8959/19560 | loss 3.403657 (-0.79z)| norm 0.3094 (+1.95z)| lr 3.58e-04 | 323.43 ms | 52.2% bf16 MFU | 1623923 tok/s
step 8960/19560 | loss 3.371307 (-1.56z)| norm 0.2811 (+0.27z)| lr 3.58e-04 | 322.58 ms | 52.3% bf16 MFU | 1623992 tok/s
step 8961/19560 | loss 3.444086 (+0.20z)| norm 0.2581 (-1.07z)| lr 3.58e-04 | 322.99 ms | 52.3% bf16 MFU | 1623953 tok/s
step 8962/19560 | loss 3.406603 (-0.72z)| norm 0.2793 (+0.18z)| lr 3.58e-04 | 322.58 ms | 52.3% bf16 MFU | 1624020 tok/s
step 8963/19560 | loss 3.425295 (-0.25z)| norm 0.2571 (-1.14z)| lr 3.58e-04 | 322.71 ms | 52.3% bf16 MFU | 1624050 tok/s
step 8964/19560 | loss 3.449435 (+0.38z)| norm 0.2661 (-0.58z)| lr 3.58e-04 | 323.20 ms | 52.2% bf16 MFU | 1623957 tok/s
step 8965/19560 | loss 3.452588 (+0.45z)| norm 0.2507 (-1.49z)| lr 3.58e-04 | 322.19 ms | 52.4% bf16 MFU | 1624123 tok/s
step 8966/19560 | loss 3.486574 (+1.31z)| norm 0.2696 (-0.34z)| lr 3.58e-04 | 322.53 ms | 52.3% bf16 MFU | 1624195 tok/s
step 8967/19560 | loss 3.362250 (-1.81z)| norm 0.2610 (-0.85z)| lr 3.58e-04 | 322.99 ms | 52.3% bf16 MFU | 1624148 tok/s
step 8968/19560 | loss 3.465017 (+0.75z)| norm 0.2626 (-0.74z)| lr 3.58e-04 | 322.66 ms | 52.3% bf16 MFU | 1624185 tok/s
step 8969/19560 | loss 3.370696 (-1.58z)| norm 0.2732 (-0.10z)| lr 3.58e-04 | 323.58 ms | 52.2% bf16 MFU | 1623990 tok/s
step 8970/19560 | loss 3.417867 (-0.39z)| norm 0.2765 (+0.10z)| lr 3.58e-04 | 322.28 ms | 52.4% bf16 MFU | 1624131 tok/s
step 8971/19560 | loss 3.419619 (-0.35z)| norm 0.2701 (-0.29z)| lr 3.58e-04 | 322.68 ms | 52.3% bf16 MFU | 1624165 tok/s
step 8972/19560 | loss 3.387263 (-1.15z)| norm 0.2677 (-0.43z)| lr 3.58e-04 | 322.73 ms | 52.3% bf16 MFU | 1624184 tok/s
step 8973/19560 | loss 3.391004 (-1.05z)| norm 0.2743 (-0.03z)| lr 3.58e-04 | 322.87 ms | 52.3% bf16 MFU | 1624166 tok/s
step 8974/19560 | loss 3.499007 (+1.60z)| norm 0.2801 (+0.32z)| lr 3.57e-04 | 323.07 ms | 52.2% bf16 MFU | 1624098 tok/s
step 8975/19560 | loss 3.453792 (+0.48z)| norm 0.2671 (-0.46z)| lr 3.57e-04 | 323.12 ms | 52.2% bf16 MFU | 1624022 tok/s
step 8976/19560 | loss 3.366905 (-1.62z)| norm 0.2773 (+0.16z)| lr 3.57e-04 | 322.50 ms | 52.3% bf16 MFU | 1624106 tok/s
step 8977/19560 | loss 3.399822 (-0.81z)| norm 0.2666 (-0.49z)| lr 3.57e-04 | 323.46 ms | 52.2% bf16 MFU | 1623944 tok/s
step 8978/19560 | loss 3.423532 (-0.21z)| norm 0.2726 (-0.11z)| lr 3.57e-04 | 322.88 ms | 52.3% bf16 MFU | 1623935 tok/s
step 8979/19560 | loss 3.408410 (-0.59z)| norm 0.2806 (+0.38z)| lr 3.57e-04 | 322.99 ms | 52.3% bf16 MFU | 1623900 tok/s
step 8980/19560 | loss 3.413599 (-0.45z)| norm 0.3158 (+2.47z)| lr 3.57e-04 | 323.08 ms | 52.2% bf16 MFU | 1623843 tok/s
step 8981/19560 | loss 3.423694 (-0.19z)| norm 0.3273 (+3.07z)| lr 3.57e-04 | 323.43 ms | 52.2% bf16 MFU | 1623703 tok/s
step 8982/19560 | loss 3.501943 (+1.74z)| norm 0.2908 (+0.91z)| lr 3.57e-04 | 323.27 ms | 52.2% bf16 MFU | 1623609 tok/s
step 8983/19560 | loss 3.414044 (-0.45z)| norm 0.3042 (+1.69z)| lr 3.57e-04 | 322.95 ms | 52.3% bf16 MFU | 1623599 tok/s
step 8984/19560 | loss 3.417197 (-0.38z)| norm 0.3075 (+1.86z)| lr 3.57e-04 | 323.24 ms | 52.2% bf16 MFU | 1623518 tok/s
step 8985/19560 | loss 3.530268 (+2.38z)| norm 0.3297 (+3.02z)| lr 3.57e-04 | 322.93 ms | 52.3% bf16 MFU | 1623520 tok/s
step 8986/19560 | loss 3.388352 (-1.09z)| norm 0.3203 (+2.42z)| lr 3.57e-04 | 322.95 ms | 52.3% bf16 MFU | 1623517 tok/s
step 8987/19560 | loss 3.547812 (+2.76z)| norm 0.2938 (+0.93z)| lr 3.57e-04 | 323.30 ms | 52.2% bf16 MFU | 1623424 tok/s
step 8988/19560 | loss 3.447506 (+0.34z)| norm 0.2789 (+0.11z)| lr 3.57e-04 | 322.94 ms | 52.3% bf16 MFU | 1623428 tok/s
step 8989/19560 | loss 3.440611 (+0.19z)| norm 0.2760 (-0.06z)| lr 3.57e-04 | 322.74 ms | 52.3% bf16 MFU | 1623480 tok/s
step 8990/19560 | loss 3.382853 (-1.20z)| norm 0.2740 (-0.16z)| lr 3.57e-04 | 322.33 ms | 52.4% bf16 MFU | 1623635 tok/s
step 8991/19560 | loss 3.384134 (-1.15z)| norm 0.2786 (+0.09z)| lr 3.57e-04 | 323.24 ms | 52.2% bf16 MFU | 1623552 tok/s
step 8992/19560 | loss 3.442890 (+0.26z)| norm 0.2707 (-0.35z)| lr 3.57e-04 | 323.46 ms | 52.2% bf16 MFU | 1623417 tok/s
step 8993/19560 | loss 3.457104 (+0.59z)| norm 0.2570 (-1.10z)| lr 3.57e-04 | 322.77 ms | 52.3% bf16 MFU | 1623463 tok/s
step 8994/19560 | loss 3.430238 (-0.04z)| norm 0.2815 (+0.24z)| lr 3.56e-04 | 323.36 ms | 52.2% bf16 MFU | 1623360 tok/s
step 8995/19560 | loss 3.347271 (-1.99z)| norm 0.2774 (+0.00z)| lr 3.56e-04 | 323.53 ms | 52.2% bf16 MFU | 1623217 tok/s
step 8996/19560 | loss 3.469496 (+0.89z)| norm 0.2610 (-0.91z)| lr 3.56e-04 | 322.94 ms | 52.3% bf16 MFU | 1623231 tok/s
step 8997/19560 | loss 3.455777 (+0.56z)| norm 0.2808 (+0.20z)| lr 3.56e-04 | 322.66 ms | 52.3% bf16 MFU | 1623313 tok/s
step 8998/19560 | loss 3.359455 (-1.69z)| norm 0.2722 (-0.28z)| lr 3.56e-04 | 323.11 ms | 52.2% bf16 MFU | 1623278 tok/s
step 8999/19560 | loss 3.410799 (-0.49z)| norm 0.2594 (-0.99z)| lr 3.56e-04 | 322.44 ms | 52.3% bf16 MFU | 1623415 tok/s
step 9000/19560 | loss 3.435229 (+0.07z)| norm 0.2708 (-0.35z)| lr 3.56e-04 | 322.95 ms | 52.3% bf16 MFU | 1623416 tok/s
val loss 3.419980
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2860/10042 = 0.284804
step 9001/19560 | loss 3.470970 (+0.89z)| norm 0.2728 (-0.24z)| lr 3.56e-04 | 322.11 ms | 52.4% bf16 MFU | 1623628 tok/s
step 9002/19560 | loss 3.458451 (+0.60z)| norm 0.2611 (-0.89z)| lr 3.56e-04 | 322.51 ms | 52.3% bf16 MFU | 1623728 tok/s
step 9003/19560 | loss 3.430133 (-0.08z)| norm 0.2644 (-0.70z)| lr 3.56e-04 | 322.74 ms | 52.3% bf16 MFU | 1623766 tok/s
step 9004/19560 | loss 3.402656 (-0.72z)| norm 0.2560 (-1.15z)| lr 3.56e-04 | 322.57 ms | 52.3% bf16 MFU | 1623847 tok/s
step 9005/19560 | loss 3.385705 (-1.10z)| norm 0.2578 (-1.04z)| lr 3.56e-04 | 322.85 ms | 52.3% bf16 MFU | 1623850 tok/s
step 9006/19560 | loss 3.411183 (-0.50z)| norm 0.2547 (-1.20z)| lr 3.56e-04 | 322.49 ms | 52.3% bf16 MFU | 1623945 tok/s
step 9007/19560 | loss 3.487280 (+1.27z)| norm 0.4384 (+6.96z)| lr 3.56e-04 | 322.61 ms | 52.3% bf16 MFU | 1624006 tok/s
step 9008/19560 | loss 3.448496 (+0.37z)| norm 0.2983 (+0.87z)| lr 3.56e-04 | 322.69 ms | 52.3% bf16 MFU | 1624044 tok/s
step 9009/19560 | loss 3.488756 (+1.30z)| norm 0.3056 (+1.17z)| lr 3.56e-04 | 322.02 ms | 52.4% bf16 MFU | 1624248 tok/s
step 9010/19560 | loss 3.435748 (+0.06z)| norm 0.2761 (-0.11z)| lr 3.56e-04 | 322.63 ms | 52.3% bf16 MFU | 1624287 tok/s
step 9011/19560 | loss 3.487025 (+1.23z)| norm 0.2778 (-0.04z)| lr 3.56e-04 | 323.03 ms | 52.2% bf16 MFU | 1624225 tok/s
step 9012/19560 | loss 3.433494 (+0.00z)| norm 0.2831 (+0.19z)| lr 3.56e-04 | 321.97 ms | 52.4% bf16 MFU | 1624432 tok/s
step 9013/19560 | loss 3.396920 (-0.86z)| norm 0.2845 (+0.25z)| lr 3.56e-04 | 322.34 ms | 52.4% bf16 MFU | 1624537 tok/s
step 9014/19560 | loss 3.441000 (+0.17z)| norm 0.2641 (-0.63z)| lr 3.55e-04 | 322.96 ms | 52.3% bf16 MFU | 1624479 tok/s
step 9015/19560 | loss 3.398735 (-0.81z)| norm 0.2878 (+0.39z)| lr 3.55e-04 | 322.53 ms | 52.3% bf16 MFU | 1624533 tok/s
step 9016/19560 | loss 3.428229 (-0.10z)| norm 0.2774 (-0.07z)| lr 3.55e-04 | 322.76 ms | 52.3% bf16 MFU | 1624526 tok/s
step 9017/19560 | loss 3.465700 (+0.78z)| norm 0.2668 (-0.53z)| lr 3.55e-04 | 322.67 ms | 52.3% bf16 MFU | 1624542 tok/s
step 9018/19560 | loss 3.446106 (+0.31z)| norm 0.2699 (-0.39z)| lr 3.55e-04 | 322.25 ms | 52.4% bf16 MFU | 1624663 tok/s
step 9019/19560 | loss 3.342871 (-2.14z)| norm 0.2934 (+0.63z)| lr 3.55e-04 | 322.88 ms | 52.3% bf16 MFU | 1624618 tok/s
step 9020/19560 | loss 3.459299 (+0.61z)| norm 0.2682 (-0.45z)| lr 3.55e-04 | 322.98 ms | 52.3% bf16 MFU | 1624552 tok/s
step 9021/19560 | loss 3.448261 (+0.34z)| norm 0.3014 (+0.98z)| lr 3.55e-04 | 323.01 ms | 52.2% bf16 MFU | 1624480 tok/s
step 9022/19560 | loss 3.419070 (-0.34z)| norm 0.2680 (-0.47z)| lr 3.55e-04 | 322.06 ms | 52.4% bf16 MFU | 1624651 tok/s
step 9023/19560 | loss 3.363715 (-1.63z)| norm 0.2758 (-0.13z)| lr 3.55e-04 | 322.74 ms | 52.3% bf16 MFU | 1624643 tok/s
step 9024/19560 | loss 3.434957 (+0.05z)| norm 0.2585 (-0.87z)| lr 3.55e-04 | 322.94 ms | 52.3% bf16 MFU | 1624585 tok/s
step 9025/19560 | loss 3.394803 (-0.88z)| norm 0.2787 (-0.00z)| lr 3.55e-04 | 323.32 ms | 52.2% bf16 MFU | 1624434 tok/s
step 9026/19560 | loss 3.494932 (+1.44z)| norm 0.2775 (-0.06z)| lr 3.55e-04 | 322.53 ms | 52.3% bf16 MFU | 1624490 tok/s
step 9027/19560 | loss 3.464741 (+0.73z)| norm 0.2833 (+0.19z)| lr 3.55e-04 | 322.39 ms | 52.4% bf16 MFU | 1624579 tok/s
step 9028/19560 | loss 3.376192 (-1.31z)| norm 0.2885 (+0.40z)| lr 3.55e-04 | 322.42 ms | 52.3% bf16 MFU | 1624655 tok/s
step 9029/19560 | loss 3.425854 (-0.17z)| norm 0.2985 (+0.83z)| lr 3.55e-04 | 322.79 ms | 52.3% bf16 MFU | 1624635 tok/s
step 9030/19560 | loss 3.417778 (-0.34z)| norm 0.2808 (+0.05z)| lr 3.55e-04 | 322.82 ms | 52.3% bf16 MFU | 1624608 tok/s
step 9031/19560 | loss 3.386141 (-1.08z)| norm 0.2807 (+0.03z)| lr 3.55e-04 | 322.29 ms | 52.4% bf16 MFU | 1624716 tok/s
step 9032/19560 | loss 3.455273 (+0.53z)| norm 0.2829 (+0.13z)| lr 3.55e-04 | 322.75 ms | 52.3% bf16 MFU | 1624701 tok/s
step 9033/19560 | loss 3.438693 (+0.14z)| norm 0.2740 (-0.27z)| lr 3.55e-04 | 322.56 ms | 52.3% bf16 MFU | 1624736 tok/s
step 9034/19560 | loss 3.423903 (-0.19z)| norm 0.2736 (-0.29z)| lr 3.55e-04 | 322.31 ms | 52.4% bf16 MFU | 1624832 tok/s
step 9035/19560 | loss 3.426158 (-0.13z)| norm 0.2436 (-1.62z)| lr 3.54e-04 | 322.89 ms | 52.3% bf16 MFU | 1624777 tok/s
step 9036/19560 | loss 3.407568 (-0.57z)| norm 0.2654 (-0.64z)| lr 3.54e-04 | 323.13 ms | 52.2% bf16 MFU | 1624664 tok/s
step 9037/19560 | loss 3.513742 (+1.90z)| norm 0.2724 (-0.35z)| lr 3.54e-04 | 322.12 ms | 52.4% bf16 MFU | 1624813 tok/s
step 9038/19560 | loss 3.520290 (+2.02z)| norm 0.2952 (+0.66z)| lr 3.54e-04 | 322.83 ms | 52.3% bf16 MFU | 1624775 tok/s
step 9039/19560 | loss 3.388427 (-1.01z)| norm 0.2711 (-0.43z)| lr 3.54e-04 | 322.36 ms | 52.4% bf16 MFU | 1624857 tok/s
step 9040/19560 | loss 3.373554 (-1.37z)| norm 0.2504 (-1.34z)| lr 3.54e-04 | 322.47 ms | 52.3% bf16 MFU | 1624906 tok/s
step 9041/19560 | loss 3.359939 (-1.67z)| norm 0.2770 (-0.15z)| lr 3.54e-04 | 323.30 ms | 52.2% bf16 MFU | 1624744 tok/s
step 9042/19560 | loss 3.411143 (-0.46z)| norm 0.2841 (+0.17z)| lr 3.54e-04 | 322.65 ms | 52.3% bf16 MFU | 1624754 tok/s
step 9043/19560 | loss 3.469146 (+0.93z)| norm 0.2676 (-0.57z)| lr 3.54e-04 | 322.66 ms | 52.3% bf16 MFU | 1624762 tok/s
step 9044/19560 | loss 3.336827 (-2.19z)| norm 0.2897 (+0.43z)| lr 3.54e-04 | 322.16 ms | 52.4% bf16 MFU | 1624896 tok/s
step 9045/19560 | loss 3.378646 (-1.20z)| norm 0.2803 (+0.01z)| lr 3.54e-04 | 322.78 ms | 52.3% bf16 MFU | 1624865 tok/s
step 9046/19560 | loss 3.463053 (+0.77z)| norm 0.2789 (-0.05z)| lr 3.54e-04 | 323.11 ms | 52.2% bf16 MFU | 1624753 tok/s
step 9047/19560 | loss 3.443476 (+0.32z)| norm 0.2906 (+0.47z)| lr 3.54e-04 | 322.75 ms | 52.3% bf16 MFU | 1624737 tok/s
step 9048/19560 | loss 3.432596 (+0.09z)| norm 0.2840 (+0.17z)| lr 3.54e-04 | 322.25 ms | 52.4% bf16 MFU | 1624850 tok/s
step 9049/19560 | loss 3.403382 (-0.62z)| norm 0.2748 (-0.25z)| lr 3.54e-04 | 322.34 ms | 52.4% bf16 MFU | 1624932 tok/s
step 9050/19560 | loss 3.356039 (-1.75z)| norm 0.2795 (-0.02z)| lr 3.54e-04 | 322.71 ms | 52.3% bf16 MFU | 1624917 tok/s
step 9051/19560 | loss 3.464681 (+0.89z)| norm 0.2533 (-1.21z)| lr 3.54e-04 | 322.48 ms | 52.3% bf16 MFU | 1624962 tok/s
step 9052/19560 | loss 3.419859 (-0.19z)| norm 0.2657 (-0.63z)| lr 3.54e-04 | 322.20 ms | 52.4% bf16 MFU | 1625075 tok/s
step 9053/19560 | loss 3.453368 (+0.63z)| norm 0.2616 (-0.81z)| lr 3.54e-04 | 322.93 ms | 52.3% bf16 MFU | 1624997 tok/s
step 9054/19560 | loss 3.399624 (-0.68z)| norm 0.2664 (-0.58z)| lr 3.54e-04 | 322.22 ms | 52.4% bf16 MFU | 1625103 tok/s
step 9055/19560 | loss 3.423254 (-0.10z)| norm 0.2664 (-0.57z)| lr 3.53e-04 | 322.72 ms | 52.3% bf16 MFU | 1625079 tok/s
step 9056/19560 | loss 3.408752 (-0.46z)| norm 0.2765 (-0.10z)| lr 3.53e-04 | 322.49 ms | 52.3% bf16 MFU | 1625112 tok/s
step 9057/19560 | loss 3.466049 (+0.94z)| norm 0.2845 (+0.27z)| lr 3.53e-04 | 322.58 ms | 52.3% bf16 MFU | 1625121 tok/s
step 9058/19560 | loss 3.489961 (+1.51z)| norm 0.2594 (-0.89z)| lr 3.53e-04 | 322.56 ms | 52.3% bf16 MFU | 1625136 tok/s
step 9059/19560 | loss 3.356688 (-1.72z)| norm 0.3006 (+1.02z)| lr 3.53e-04 | 322.46 ms | 52.3% bf16 MFU | 1625174 tok/s
step 9060/19560 | loss 3.420888 (-0.17z)| norm 0.2900 (+0.52z)| lr 3.53e-04 | 323.09 ms | 52.2% bf16 MFU | 1625052 tok/s
step 9061/19560 | loss 3.400290 (-0.66z)| norm 0.2874 (+0.40z)| lr 3.53e-04 | 322.65 ms | 52.3% bf16 MFU | 1625046 tok/s
step 9062/19560 | loss 3.471522 (+1.04z)| norm 0.2670 (-0.55z)| lr 3.53e-04 | 322.08 ms | 52.4% bf16 MFU | 1625185 tok/s
step 9063/19560 | loss 3.421244 (-0.15z)| norm 0.2744 (-0.19z)| lr 3.53e-04 | 322.99 ms | 52.3% bf16 MFU | 1625087 tok/s
step 9064/19560 | loss 3.413169 (-0.35z)| norm 0.2513 (-1.27z)| lr 3.53e-04 | 322.87 ms | 52.3% bf16 MFU | 1625024 tok/s
step 9065/19560 | loss 3.445798 (+0.44z)| norm 0.2713 (-0.31z)| lr 3.53e-04 | 322.33 ms | 52.4% bf16 MFU | 1625101 tok/s
step 9066/19560 | loss 3.413229 (-0.35z)| norm 0.2558 (-1.03z)| lr 3.53e-04 | 322.35 ms | 52.4% bf16 MFU | 1625168 tok/s
step 9067/19560 | loss 3.446687 (+0.48z)| norm 0.2883 (+0.50z)| lr 3.53e-04 | 322.90 ms | 52.3% bf16 MFU | 1625094 tok/s
step 9068/19560 | loss 3.482928 (+1.36z)| norm 0.2604 (-0.81z)| lr 3.53e-04 | 322.21 ms | 52.4% bf16 MFU | 1625196 tok/s
step 9069/19560 | loss 3.447944 (+0.49z)| norm 0.2619 (-0.73z)| lr 3.53e-04 | 322.38 ms | 52.4% bf16 MFU | 1625250 tok/s
step 9070/19560 | loss 3.437739 (+0.23z)| norm 0.2582 (-0.88z)| lr 3.53e-04 | 322.84 ms | 52.3% bf16 MFU | 1625187 tok/s
step 9071/19560 | loss 3.493863 (+1.58z)| norm 0.3029 (+1.20z)| lr 3.53e-04 | 322.50 ms | 52.3% bf16 MFU | 1625212 tok/s
step 9072/19560 | loss 3.376220 (-1.27z)| norm 0.2879 (+0.50z)| lr 3.53e-04 | 322.22 ms | 52.4% bf16 MFU | 1625308 tok/s
step 9073/19560 | loss 3.504932 (+1.82z)| norm 0.2808 (+0.17z)| lr 3.53e-04 | 322.23 ms | 52.4% bf16 MFU | 1625397 tok/s
step 9074/19560 | loss 3.491325 (+1.47z)| norm 0.2786 (+0.06z)| lr 3.53e-04 | 322.56 ms | 52.3% bf16 MFU | 1625397 tok/s
step 9075/19560 | loss 3.409640 (-0.48z)| norm 0.2636 (-0.63z)| lr 3.52e-04 | 322.66 ms | 52.3% bf16 MFU | 1625372 tok/s
step 9076/19560 | loss 3.434485 (+0.11z)| norm 0.2601 (-0.79z)| lr 3.52e-04 | 322.24 ms | 52.4% bf16 MFU | 1625453 tok/s
step 9077/19560 | loss 3.374799 (-1.29z)| norm 0.2722 (-0.23z)| lr 3.52e-04 | 322.41 ms | 52.3% bf16 MFU | 1625489 tok/s
step 9078/19560 | loss 3.368887 (-1.41z)| norm 0.2752 (-0.09z)| lr 3.52e-04 | 322.57 ms | 52.3% bf16 MFU | 1625483 tok/s
step 9079/19560 | loss 3.386582 (-0.98z)| norm 0.2732 (-0.18z)| lr 3.52e-04 | 322.36 ms | 52.4% bf16 MFU | 1625530 tok/s
step 9080/19560 | loss 3.449842 (+0.51z)| norm 0.2823 (+0.24z)| lr 3.52e-04 | 322.53 ms | 52.3% bf16 MFU | 1625531 tok/s
step 9081/19560 | loss 3.406512 (-0.51z)| norm 0.2745 (-0.13z)| lr 3.52e-04 | 322.43 ms | 52.3% bf16 MFU | 1625556 tok/s
step 9082/19560 | loss 3.405335 (-0.53z)| norm 0.2625 (-0.68z)| lr 3.52e-04 | 322.88 ms | 52.3% bf16 MFU | 1625467 tok/s
step 9083/19560 | loss 3.387475 (-0.94z)| norm 0.2755 (-0.08z)| lr 3.52e-04 | 322.51 ms | 52.3% bf16 MFU | 1625476 tok/s
step 9084/19560 | loss 3.441464 (+0.33z)| norm 0.2961 (+0.88z)| lr 3.52e-04 | 322.88 ms | 52.3% bf16 MFU | 1625392 tok/s
step 9085/19560 | loss 3.460471 (+0.76z)| norm 0.2706 (-0.34z)| lr 3.52e-04 | 322.04 ms | 52.4% bf16 MFU | 1625524 tok/s
step 9086/19560 | loss 3.377740 (-1.18z)| norm 0.2753 (-0.11z)| lr 3.52e-04 | 322.76 ms | 52.3% bf16 MFU | 1625466 tok/s
step 9087/19560 | loss 3.442625 (+0.35z)| norm 0.2601 (-0.82z)| lr 3.52e-04 | 323.16 ms | 52.2% bf16 MFU | 1625311 tok/s
step 9088/19560 | loss 3.390009 (-0.90z)| norm 0.2738 (-0.17z)| lr 3.52e-04 | 322.51 ms | 52.3% bf16 MFU | 1625327 tok/s
step 9089/19560 | loss 3.427941 (+0.01z)| norm 0.2879 (+0.50z)| lr 3.52e-04 | 322.39 ms | 52.3% bf16 MFU | 1625372 tok/s
step 9090/19560 | loss 3.405437 (-0.53z)| norm 0.2792 (+0.08z)| lr 3.52e-04 | 322.86 ms | 52.3% bf16 MFU | 1625298 tok/s
step 9091/19560 | loss 3.449969 (+0.52z)| norm 0.2526 (-1.19z)| lr 3.52e-04 | 322.37 ms | 52.4% bf16 MFU | 1625350 tok/s
step 9092/19560 | loss 3.457292 (+0.70z)| norm 0.2596 (-0.85z)| lr 3.52e-04 | 322.54 ms | 52.3% bf16 MFU | 1625358 tok/s
step 9093/19560 | loss 3.401749 (-0.61z)| norm 0.2844 (+0.32z)| lr 3.52e-04 | 322.63 ms | 52.3% bf16 MFU | 1625342 tok/s
step 9094/19560 | loss 3.457084 (+0.71z)| norm 0.3126 (+1.65z)| lr 3.52e-04 | 323.09 ms | 52.2% bf16 MFU | 1625212 tok/s
step 9095/19560 | loss 3.443080 (+0.36z)| norm 0.3060 (+1.31z)| lr 3.52e-04 | 322.79 ms | 52.3% bf16 MFU | 1625164 tok/s
step 9096/19560 | loss 3.502909 (+1.78z)| norm 0.2748 (-0.17z)| lr 3.51e-04 | 322.63 ms | 52.3% bf16 MFU | 1625157 tok/s
step 9097/19560 | loss 3.518124 (+2.10z)| norm 0.3453 (+3.04z)| lr 3.51e-04 | 322.81 ms | 52.3% bf16 MFU | 1625107 tok/s
step 9098/19560 | loss 3.406384 (-0.54z)| norm 0.2671 (-0.54z)| lr 3.51e-04 | 322.53 ms | 52.3% bf16 MFU | 1625129 tok/s
step 9099/19560 | loss 3.469831 (+0.94z)| norm 0.2771 (-0.09z)| lr 3.51e-04 | 322.90 ms | 52.3% bf16 MFU | 1625056 tok/s
step 9100/19560 | loss 3.421618 (-0.20z)| norm 0.2810 (+0.09z)| lr 3.51e-04 | 322.43 ms | 52.3% bf16 MFU | 1625107 tok/s
step 9101/19560 | loss 3.370795 (-1.39z)| norm 0.2734 (-0.26z)| lr 3.51e-04 | 322.76 ms | 52.3% bf16 MFU | 1625072 tok/s
step 9102/19560 | loss 3.442948 (+0.32z)| norm 0.2830 (+0.18z)| lr 3.51e-04 | 323.47 ms | 52.2% bf16 MFU | 1624860 tok/s
step 9103/19560 | loss 3.442721 (+0.32z)| norm 0.2727 (-0.30z)| lr 3.51e-04 | 322.79 ms | 52.3% bf16 MFU | 1624828 tok/s
step 9104/19560 | loss 3.493329 (+1.50z)| norm 0.2668 (-0.56z)| lr 3.51e-04 | 322.55 ms | 52.3% bf16 MFU | 1624860 tok/s
step 9105/19560 | loss 3.408119 (-0.53z)| norm 0.2675 (-0.53z)| lr 3.51e-04 | 322.63 ms | 52.3% bf16 MFU | 1624869 tok/s
step 9106/19560 | loss 3.392351 (-0.90z)| norm 0.2561 (-1.04z)| lr 3.51e-04 | 322.94 ms | 52.3% bf16 MFU | 1624800 tok/s
step 9107/19560 | loss 3.444484 (+0.33z)| norm 0.2648 (-0.64z)| lr 3.51e-04 | 322.70 ms | 52.3% bf16 MFU | 1624795 tok/s
step 9108/19560 | loss 3.360943 (-1.62z)| norm 0.2711 (-0.34z)| lr 3.51e-04 | 323.21 ms | 52.2% bf16 MFU | 1624662 tok/s
step 9109/19560 | loss 3.499036 (+1.59z)| norm 0.2624 (-0.73z)| lr 3.51e-04 | 322.60 ms | 52.3% bf16 MFU | 1624688 tok/s
step 9110/19560 | loss 3.395336 (-0.81z)| norm 0.2703 (-0.35z)| lr 3.51e-04 | 322.37 ms | 52.4% bf16 MFU | 1624772 tok/s
step 9111/19560 | loss 3.414057 (-0.37z)| norm 0.2574 (-0.94z)| lr 3.51e-04 | 322.93 ms | 52.3% bf16 MFU | 1624711 tok/s
step 9112/19560 | loss 3.392458 (-0.87z)| norm 0.2664 (-0.51z)| lr 3.51e-04 | 323.00 ms | 52.3% bf16 MFU | 1624634 tok/s
step 9113/19560 | loss 3.459565 (+0.73z)| norm 0.2608 (-0.77z)| lr 3.51e-04 | 322.86 ms | 52.3% bf16 MFU | 1624596 tok/s
step 9114/19560 | loss 3.379359 (-1.18z)| norm 0.2498 (-1.29z)| lr 3.51e-04 | 322.56 ms | 52.3% bf16 MFU | 1624637 tok/s
step 9115/19560 | loss 3.423426 (-0.11z)| norm 0.2692 (-0.33z)| lr 3.51e-04 | 323.08 ms | 52.2% bf16 MFU | 1624545 tok/s
step 9116/19560 | loss 3.424205 (-0.09z)| norm 0.2639 (-0.58z)| lr 3.50e-04 | 322.24 ms | 52.4% bf16 MFU | 1624668 tok/s
step 9117/19560 | loss 3.400369 (-0.67z)| norm 0.2599 (-0.77z)| lr 3.50e-04 | 323.02 ms | 52.2% bf16 MFU | 1624588 tok/s
step 9118/19560 | loss 3.423598 (-0.10z)| norm 0.2584 (-0.83z)| lr 3.50e-04 | 322.73 ms | 52.3% bf16 MFU | 1624584 tok/s
step 9119/19560 | loss 3.507665 (+1.93z)| norm 0.2643 (-0.54z)| lr 3.50e-04 | 323.03 ms | 52.2% bf16 MFU | 1624506 tok/s
step 9120/19560 | loss 3.435697 (+0.17z)| norm 0.2720 (-0.17z)| lr 3.50e-04 | 322.93 ms | 52.3% bf16 MFU | 1624457 tok/s
step 9121/19560 | loss 3.353178 (-1.81z)| norm 0.2661 (-0.46z)| lr 3.50e-04 | 322.74 ms | 52.3% bf16 MFU | 1624459 tok/s
step 9122/19560 | loss 3.391810 (-0.86z)| norm 0.2725 (-0.14z)| lr 3.50e-04 | 322.64 ms | 52.3% bf16 MFU | 1624485 tok/s
step 9123/19560 | loss 3.433521 (+0.13z)| norm 0.2732 (-0.10z)| lr 3.50e-04 | 322.71 ms | 52.3% bf16 MFU | 1624493 tok/s
step 9124/19560 | loss 3.438283 (+0.25z)| norm 0.2848 (+0.45z)| lr 3.50e-04 | 322.76 ms | 52.3% bf16 MFU | 1624487 tok/s
step 9125/19560 | loss 3.458185 (+0.74z)| norm 0.2798 (+0.21z)| lr 3.50e-04 | 322.45 ms | 52.3% bf16 MFU | 1624562 tok/s
step 9126/19560 | loss 3.462075 (+0.83z)| norm 0.2678 (-0.38z)| lr 3.50e-04 | 323.17 ms | 52.2% bf16 MFU | 1624450 tok/s
step 9127/19560 | loss 3.502004 (+1.78z)| norm 0.2596 (-0.78z)| lr 3.50e-04 | 323.11 ms | 52.2% bf16 MFU | 1624360 tok/s
step 9128/19560 | loss 3.377989 (-1.25z)| norm 0.2520 (-1.13z)| lr 3.50e-04 | 323.54 ms | 52.2% bf16 MFU | 1624166 tok/s
step 9129/19560 | loss 3.425636 (-0.08z)| norm 0.2784 (+0.15z)| lr 3.50e-04 | 322.75 ms | 52.3% bf16 MFU | 1624179 tok/s
step 9130/19560 | loss 3.440199 (+0.29z)| norm 0.2732 (-0.11z)| lr 3.50e-04 | 322.52 ms | 52.3% bf16 MFU | 1624250 tok/s
step 9131/19560 | loss 3.412935 (-0.38z)| norm 0.2665 (-0.44z)| lr 3.50e-04 | 322.74 ms | 52.3% bf16 MFU | 1624262 tok/s
step 9132/19560 | loss 3.377934 (-1.23z)| norm 0.2932 (+0.85z)| lr 3.50e-04 | 322.40 ms | 52.3% bf16 MFU | 1624360 tok/s
step 9133/19560 | loss 3.433290 (+0.11z)| norm 0.2676 (-0.41z)| lr 3.50e-04 | 323.07 ms | 52.2% bf16 MFU | 1624283 tok/s
step 9134/19560 | loss 3.406786 (-0.53z)| norm 0.2872 (+0.55z)| lr 3.50e-04 | 323.16 ms | 52.2% bf16 MFU | 1624188 tok/s
step 9135/19560 | loss 3.415364 (-0.31z)| norm 0.2767 (+0.13z)| lr 3.50e-04 | 322.70 ms | 52.3% bf16 MFU | 1624213 tok/s
step 9136/19560 | loss 3.442364 (+0.36z)| norm 0.2739 (-0.05z)| lr 3.49e-04 | 322.83 ms | 52.3% bf16 MFU | 1624205 tok/s
step 9137/19560 | loss 3.431284 (+0.09z)| norm 0.2654 (-0.64z)| lr 3.49e-04 | 323.48 ms | 52.2% bf16 MFU | 1624035 tok/s
step 9138/19560 | loss 3.436693 (+0.23z)| norm 0.2853 (+0.78z)| lr 3.49e-04 | 323.08 ms | 52.2% bf16 MFU | 1623973 tok/s
step 9139/19560 | loss 3.402441 (-0.61z)| norm 0.2627 (-0.82z)| lr 3.49e-04 | 323.21 ms | 52.2% bf16 MFU | 1623881 tok/s
step 9140/19560 | loss 3.402139 (-0.61z)| norm 0.2563 (-1.26z)| lr 3.49e-04 | 322.33 ms | 52.4% bf16 MFU | 1624014 tok/s
step 9141/19560 | loss 3.344909 (-2.01z)| norm 0.2578 (-1.13z)| lr 3.49e-04 | 322.59 ms | 52.3% bf16 MFU | 1624075 tok/s
step 9142/19560 | loss 3.446065 (+0.49z)| norm 0.2900 (+1.11z)| lr 3.49e-04 | 322.80 ms | 52.3% bf16 MFU | 1624082 tok/s
step 9143/19560 | loss 3.383790 (-1.04z)| norm 0.2682 (-0.41z)| lr 3.49e-04 | 323.12 ms | 52.2% bf16 MFU | 1624006 tok/s
step 9144/19560 | loss 3.446683 (+0.50z)| norm 0.2813 (+0.51z)| lr 3.49e-04 | 322.97 ms | 52.3% bf16 MFU | 1623972 tok/s
step 9145/19560 | loss 3.381692 (-1.08z)| norm 0.2735 (-0.04z)| lr 3.49e-04 | 323.21 ms | 52.2% bf16 MFU | 1623879 tok/s
step 9146/19560 | loss 3.448449 (+0.56z)| norm 0.2679 (-0.43z)| lr 3.49e-04 | 322.40 ms | 52.3% bf16 MFU | 1623995 tok/s
step 9147/19560 | loss 3.448891 (+0.56z)| norm 0.2706 (-0.23z)| lr 3.49e-04 | 323.51 ms | 52.2% bf16 MFU | 1623826 tok/s
step 9148/19560 | loss 3.442932 (+0.42z)| norm 0.2740 (+0.01z)| lr 3.49e-04 | 323.43 ms | 52.2% bf16 MFU | 1623685 tok/s
step 9149/19560 | loss 3.392788 (-0.83z)| norm 0.2912 (+1.25z)| lr 3.49e-04 | 322.90 ms | 52.3% bf16 MFU | 1623686 tok/s
step 9150/19560 | loss 3.479666 (+1.32z)| norm 0.2711 (-0.20z)| lr 3.49e-04 | 322.60 ms | 52.3% bf16 MFU | 1623761 tok/s
step 9151/19560 | loss 3.378798 (-1.19z)| norm 0.2845 (+0.76z)| lr 3.49e-04 | 323.70 ms | 52.1% bf16 MFU | 1623556 tok/s
step 9152/19560 | loss 3.491558 (+1.59z)| norm 0.2577 (-1.16z)| lr 3.49e-04 | 322.70 ms | 52.3% bf16 MFU | 1623612 tok/s
step 9153/19560 | loss 3.463516 (+0.89z)| norm 0.2762 (+0.17z)| lr 3.49e-04 | 322.76 ms | 52.3% bf16 MFU | 1623652 tok/s
step 9154/19560 | loss 3.358200 (-1.68z)| norm 0.2695 (-0.31z)| lr 3.49e-04 | 322.98 ms | 52.3% bf16 MFU | 1623634 tok/s
step 9155/19560 | loss 3.453367 (+0.67z)| norm 0.2782 (+0.32z)| lr 3.49e-04 | 323.31 ms | 52.2% bf16 MFU | 1623534 tok/s
step 9156/19560 | loss 3.498749 (+1.75z)| norm 0.3003 (+1.87z)| lr 3.49e-04 | 322.84 ms | 52.3% bf16 MFU | 1623558 tok/s
step 9157/19560 | loss 3.440573 (+0.32z)| norm 0.3633 (+5.56z)| lr 3.48e-04 | 322.34 ms | 52.4% bf16 MFU | 1623707 tok/s
step 9158/19560 | loss 3.464627 (+0.90z)| norm 0.2874 (+0.81z)| lr 3.48e-04 | 323.32 ms | 52.2% bf16 MFU | 1623599 tok/s
step 9159/19560 | loss 3.459910 (+0.77z)| norm 0.3220 (+2.85z)| lr 3.48e-04 | 322.67 ms | 52.3% bf16 MFU | 1623661 tok/s
step 9160/19560 | loss 3.410057 (-0.44z)| norm 0.2768 (+0.13z)| lr 3.48e-04 | 322.36 ms | 52.4% bf16 MFU | 1623799 tok/s
step 9161/19560 | loss 3.530537 (+2.44z)| norm 0.3003 (+1.52z)| lr 3.48e-04 | 322.85 ms | 52.3% bf16 MFU | 1623805 tok/s
step 9162/19560 | loss 3.438850 (+0.24z)| norm 0.2752 (+0.02z)| lr 3.48e-04 | 322.62 ms | 52.3% bf16 MFU | 1623868 tok/s
step 9163/19560 | loss 3.356667 (-1.70z)| norm 0.2712 (-0.24z)| lr 3.48e-04 | 322.71 ms | 52.3% bf16 MFU | 1623906 tok/s
step 9164/19560 | loss 3.488943 (+1.41z)| norm 0.3060 (+1.84z)| lr 3.48e-04 | 323.14 ms | 52.2% bf16 MFU | 1623835 tok/s
step 9165/19560 | loss 3.429494 (+0.03z)| norm 0.2966 (+1.25z)| lr 3.48e-04 | 322.53 ms | 52.3% bf16 MFU | 1623920 tok/s
step 9166/19560 | loss 3.398105 (-0.71z)| norm 0.2988 (+1.38z)| lr 3.48e-04 | 323.22 ms | 52.2% bf16 MFU | 1623828 tok/s
step 9167/19560 | loss 3.435847 (+0.20z)| norm 0.2997 (+1.41z)| lr 3.48e-04 | 322.55 ms | 52.3% bf16 MFU | 1623909 tok/s
step 9168/19560 | loss 3.446254 (+0.44z)| norm 0.2820 (+0.35z)| lr 3.48e-04 | 323.02 ms | 52.2% bf16 MFU | 1623868 tok/s
step 9169/19560 | loss 3.415394 (-0.33z)| norm 0.2978 (+1.27z)| lr 3.48e-04 | 322.77 ms | 52.3% bf16 MFU | 1623892 tok/s
step 9170/19560 | loss 3.406975 (-0.54z)| norm 0.2873 (+0.65z)| lr 3.48e-04 | 322.50 ms | 52.3% bf16 MFU | 1623982 tok/s
step 9171/19560 | loss 3.510423 (+2.00z)| norm 0.2933 (+0.99z)| lr 3.48e-04 | 323.17 ms | 52.2% bf16 MFU | 1623900 tok/s
step 9172/19560 | loss 3.399031 (-0.76z)| norm 0.2773 (+0.06z)| lr 3.48e-04 | 323.19 ms | 52.2% bf16 MFU | 1623817 tok/s
step 9173/19560 | loss 3.451574 (+0.54z)| norm 0.3073 (+1.79z)| lr 3.48e-04 | 323.03 ms | 52.2% bf16 MFU | 1623778 tok/s
step 9174/19560 | loss 3.393782 (-0.90z)| norm 0.2824 (+0.34z)| lr 3.48e-04 | 322.79 ms | 52.3% bf16 MFU | 1623802 tok/s
step 9175/19560 | loss 3.425767 (-0.09z)| norm 0.3293 (+2.96z)| lr 3.48e-04 | 322.77 ms | 52.3% bf16 MFU | 1623828 tok/s
step 9176/19560 | loss 3.432225 (+0.07z)| norm 0.2999 (+1.28z)| lr 3.48e-04 | 323.36 ms | 52.2% bf16 MFU | 1623705 tok/s
step 9177/19560 | loss 3.350318 (-1.95z)| norm 0.2757 (-0.08z)| lr 3.47e-04 | 322.81 ms | 52.3% bf16 MFU | 1623727 tok/s
step 9178/19560 | loss 3.427531 (-0.05z)| norm 0.2866 (+0.53z)| lr 3.47e-04 | 323.10 ms | 52.2% bf16 MFU | 1623674 tok/s
step 9179/19560 | loss 3.471669 (+1.06z)| norm 0.2734 (-0.22z)| lr 3.47e-04 | 323.05 ms | 52.2% bf16 MFU | 1623636 tok/s
step 9180/19560 | loss 3.397167 (-0.80z)| norm 0.2949 (+0.98z)| lr 3.47e-04 | 322.89 ms | 52.3% bf16 MFU | 1623642 tok/s
step 9181/19560 | loss 3.394390 (-0.86z)| norm 0.2492 (-1.58z)| lr 3.47e-04 | 323.23 ms | 52.2% bf16 MFU | 1623561 tok/s
step 9182/19560 | loss 3.486110 (+1.40z)| norm 0.3102 (+1.79z)| lr 3.47e-04 | 322.60 ms | 52.3% bf16 MFU | 1623642 tok/s
step 9183/19560 | loss 3.445012 (+0.38z)| norm 0.3128 (+1.89z)| lr 3.47e-04 | 323.25 ms | 52.2% bf16 MFU | 1623556 tok/s
step 9184/19560 | loss 3.421500 (-0.21z)| norm 0.2775 (-0.03z)| lr 3.47e-04 | 323.28 ms | 52.2% bf16 MFU | 1623466 tok/s
step 9185/19560 | loss 3.472219 (+1.05z)| norm 0.2921 (+0.76z)| lr 3.47e-04 | 322.82 ms | 52.3% bf16 MFU | 1623497 tok/s
step 9186/19560 | loss 3.459774 (+0.75z)| norm 0.2856 (+0.39z)| lr 3.47e-04 | 322.54 ms | 52.3% bf16 MFU | 1623596 tok/s
step 9187/19560 | loss 3.477762 (+1.19z)| norm 0.2894 (+0.61z)| lr 3.47e-04 | 322.75 ms | 52.3% bf16 MFU | 1623637 tok/s
step 9188/19560 | loss 3.352968 (-1.91z)| norm 0.2828 (+0.25z)| lr 3.47e-04 | 322.90 ms | 52.3% bf16 MFU | 1623641 tok/s
step 9189/19560 | loss 3.473250 (+1.05z)| norm 0.2896 (+0.62z)| lr 3.47e-04 | 322.57 ms | 52.3% bf16 MFU | 1623727 tok/s
step 9190/19560 | loss 3.490620 (+1.47z)| norm 0.2776 (-0.04z)| lr 3.47e-04 | 322.75 ms | 52.3% bf16 MFU | 1623762 tok/s
step 9191/19560 | loss 3.426172 (-0.11z)| norm 0.2893 (+0.60z)| lr 3.47e-04 | 323.01 ms | 52.3% bf16 MFU | 1623731 tok/s
step 9192/19560 | loss 3.453965 (+0.56z)| norm 0.2883 (+0.53z)| lr 3.47e-04 | 322.75 ms | 52.3% bf16 MFU | 1623768 tok/s
step 9193/19560 | loss 3.368112 (-1.52z)| norm 0.2651 (-0.75z)| lr 3.47e-04 | 322.55 ms | 52.3% bf16 MFU | 1623853 tok/s
step 9194/19560 | loss 3.439085 (+0.20z)| norm 0.2886 (+0.53z)| lr 3.47e-04 | 322.23 ms | 52.4% bf16 MFU | 1624014 tok/s
step 9195/19560 | loss 3.470748 (+0.97z)| norm 0.2486 (-1.66z)| lr 3.47e-04 | 323.35 ms | 52.2% bf16 MFU | 1623886 tok/s
step 9196/19560 | loss 3.406990 (-0.57z)| norm 0.3026 (+1.29z)| lr 3.47e-04 | 323.22 ms | 52.2% bf16 MFU | 1623796 tok/s
step 9197/19560 | loss 3.467245 (+0.90z)| norm 0.2815 (+0.13z)| lr 3.46e-04 | 322.82 ms | 52.3% bf16 MFU | 1623809 tok/s
step 9198/19560 | loss 3.343799 (-2.06z)| norm 0.2606 (-1.02z)| lr 3.46e-04 | 322.85 ms | 52.3% bf16 MFU | 1623815 tok/s
step 9199/19560 | loss 3.426730 (-0.06z)| norm 0.2753 (-0.21z)| lr 3.46e-04 | 322.50 ms | 52.3% bf16 MFU | 1623908 tok/s
step 9200/19560 | loss 3.363559 (-1.58z)| norm 0.2946 (+0.86z)| lr 3.46e-04 | 322.47 ms | 52.3% bf16 MFU | 1624006 tok/s
step 9201/19560 | loss 3.418890 (-0.23z)| norm 0.2998 (+1.14z)| lr 3.46e-04 | 322.61 ms | 52.3% bf16 MFU | 1624062 tok/s
step 9202/19560 | loss 3.439525 (+0.28z)| norm 0.2888 (+0.53z)| lr 3.46e-04 | 322.85 ms | 52.3% bf16 MFU | 1624057 tok/s
step 9203/19560 | loss 3.414042 (-0.35z)| norm 0.2686 (-0.59z)| lr 3.46e-04 | 322.94 ms | 52.3% bf16 MFU | 1624028 tok/s
step 9204/19560 | loss 3.406036 (-0.54z)| norm 0.2788 (-0.04z)| lr 3.46e-04 | 322.71 ms | 52.3% bf16 MFU | 1624058 tok/s
step 9205/19560 | loss 3.423432 (-0.12z)| norm 0.2773 (-0.12z)| lr 3.46e-04 | 322.52 ms | 52.3% bf16 MFU | 1624136 tok/s
step 9206/19560 | loss 3.380184 (-1.20z)| norm 0.2788 (-0.04z)| lr 3.46e-04 | 323.12 ms | 52.2% bf16 MFU | 1624057 tok/s
step 9207/19560 | loss 3.426051 (-0.07z)| norm 0.2665 (-0.72z)| lr 3.46e-04 | 322.72 ms | 52.3% bf16 MFU | 1624085 tok/s
step 9208/19560 | loss 3.425822 (-0.07z)| norm 0.2747 (-0.26z)| lr 3.46e-04 | 322.80 ms | 52.3% bf16 MFU | 1624089 tok/s
step 9209/19560 | loss 3.418406 (-0.25z)| norm 0.2588 (-1.13z)| lr 3.46e-04 | 322.58 ms | 52.3% bf16 MFU | 1624151 tok/s
step 9210/19560 | loss 3.412421 (-0.41z)| norm 0.2486 (-1.67z)| lr 3.46e-04 | 322.81 ms | 52.3% bf16 MFU | 1624150 tok/s
step 9211/19560 | loss 3.436935 (+0.20z)| norm 0.2663 (-0.70z)| lr 3.46e-04 | 323.05 ms | 52.2% bf16 MFU | 1624088 tok/s
step 9212/19560 | loss 3.340471 (-2.17z)| norm 0.2795 (+0.03z)| lr 3.46e-04 | 322.20 ms | 52.4% bf16 MFU | 1624244 tok/s
step 9213/19560 | loss 3.302354 (-2.98z)| norm 0.2718 (-0.39z)| lr 3.46e-04 | 322.92 ms | 52.3% bf16 MFU | 1624210 tok/s
step 9214/19560 | loss 3.398011 (-0.70z)| norm 0.2561 (-1.24z)| lr 3.46e-04 | 322.34 ms | 52.4% bf16 MFU | 1624324 tok/s
step 9215/19560 | loss 3.399969 (-0.64z)| norm 0.2794 (+0.03z)| lr 3.46e-04 | 322.96 ms | 52.3% bf16 MFU | 1624278 tok/s
step 9216/19560 | loss 3.470179 (+1.02z)| norm 0.2988 (+1.07z)| lr 3.46e-04 | 322.56 ms | 52.3% bf16 MFU | 1624334 tok/s
step 9217/19560 | loss 3.455409 (+0.66z)| norm 0.3007 (+1.16z)| lr 3.45e-04 | 322.79 ms | 52.3% bf16 MFU | 1624331 tok/s
step 9218/19560 | loss 3.389890 (-0.90z)| norm 0.2926 (+0.72z)| lr 3.45e-04 | 322.49 ms | 52.3% bf16 MFU | 1624402 tok/s
step 9219/19560 | loss 3.370745 (-1.33z)| norm 0.2767 (-0.15z)| lr 3.45e-04 | 323.09 ms | 52.2% bf16 MFU | 1624319 tok/s
step 9220/19560 | loss 3.361611 (-1.52z)| norm 0.2935 (+0.75z)| lr 3.45e-04 | 322.81 ms | 52.3% bf16 MFU | 1624309 tok/s
step 9221/19560 | loss 3.433234 (+0.16z)| norm 0.3187 (+2.08z)| lr 3.45e-04 | 322.54 ms | 52.3% bf16 MFU | 1624368 tok/s
step 9222/19560 | loss 3.399155 (-0.63z)| norm 0.2757 (-0.22z)| lr 3.45e-04 | 322.87 ms | 52.3% bf16 MFU | 1624342 tok/s
step 9223/19560 | loss 3.412042 (-0.32z)| norm 0.3015 (+1.19z)| lr 3.45e-04 | 323.18 ms | 52.2% bf16 MFU | 1624239 tok/s
step 9224/19560 | loss 3.441200 (+0.38z)| norm 0.2888 (+0.49z)| lr 3.45e-04 | 322.85 ms | 52.3% bf16 MFU | 1624223 tok/s
step 9225/19560 | loss 3.420488 (-0.10z)| norm 0.2865 (+0.41z)| lr 3.45e-04 | 322.18 ms | 52.4% bf16 MFU | 1624378 tok/s
step 9226/19560 | loss 3.371516 (-1.28z)| norm 0.2719 (-0.43z)| lr 3.45e-04 | 322.47 ms | 52.3% bf16 MFU | 1624452 tok/s
step 9227/19560 | loss 3.458061 (+0.82z)| norm 0.2587 (-1.18z)| lr 3.45e-04 | 322.36 ms | 52.4% bf16 MFU | 1624551 tok/s
step 9228/19560 | loss 3.434796 (+0.26z)| norm 0.2782 (-0.06z)| lr 3.45e-04 | 323.30 ms | 52.2% bf16 MFU | 1624408 tok/s
step 9229/19560 | loss 3.356945 (-1.62z)| norm 0.2713 (-0.46z)| lr 3.45e-04 | 322.34 ms | 52.4% bf16 MFU | 1624513 tok/s
step 9230/19560 | loss 3.465318 (+0.99z)| norm 0.3009 (+1.22z)| lr 3.45e-04 | 322.85 ms | 52.3% bf16 MFU | 1624483 tok/s
step 9231/19560 | loss 3.464561 (+0.96z)| norm 0.2739 (-0.31z)| lr 3.45e-04 | 322.60 ms | 52.3% bf16 MFU | 1624519 tok/s
step 9232/19560 | loss 3.432316 (+0.20z)| norm 0.2865 (+0.40z)| lr 3.45e-04 | 322.63 ms | 52.3% bf16 MFU | 1624546 tok/s
step 9233/19560 | loss 3.483938 (+1.43z)| norm 0.2602 (-1.09z)| lr 3.45e-04 | 322.48 ms | 52.3% bf16 MFU | 1624607 tok/s
step 9234/19560 | loss 3.399992 (-0.59z)| norm 0.2908 (+0.63z)| lr 3.45e-04 | 322.72 ms | 52.3% bf16 MFU | 1624608 tok/s
step 9235/19560 | loss 3.456805 (+0.77z)| norm 0.2794 (-0.03z)| lr 3.45e-04 | 322.35 ms | 52.4% bf16 MFU | 1624700 tok/s
step 9236/19560 | loss 3.443612 (+0.44z)| norm 0.2751 (-0.28z)| lr 3.45e-04 | 322.51 ms | 52.3% bf16 MFU | 1624747 tok/s
step 9237/19560 | loss 3.416724 (-0.20z)| norm 0.2673 (-0.73z)| lr 3.45e-04 | 322.34 ms | 52.4% bf16 MFU | 1624835 tok/s
step 9238/19560 | loss 3.496682 (+1.73z)| norm 0.3120 (+1.80z)| lr 3.44e-04 | 323.10 ms | 52.2% bf16 MFU | 1624728 tok/s
step 9239/19560 | loss 3.389639 (-0.87z)| norm 0.2661 (-0.81z)| lr 3.44e-04 | 322.32 ms | 52.4% bf16 MFU | 1624822 tok/s
step 9240/19560 | loss 3.417375 (-0.20z)| norm 0.2972 (+0.94z)| lr 3.44e-04 | 322.78 ms | 52.3% bf16 MFU | 1624795 tok/s
step 9241/19560 | loss 3.372125 (-1.28z)| norm 0.2559 (-1.40z)| lr 3.44e-04 | 322.56 ms | 52.3% bf16 MFU | 1624825 tok/s
step 9242/19560 | loss 3.405973 (-0.46z)| norm 0.3103 (+1.67z)| lr 3.44e-04 | 322.39 ms | 52.3% bf16 MFU | 1624896 tok/s
step 9243/19560 | loss 3.454005 (+0.70z)| norm 0.2957 (+0.82z)| lr 3.44e-04 | 322.38 ms | 52.4% bf16 MFU | 1624966 tok/s
step 9244/19560 | loss 3.424214 (-0.03z)| norm 0.2876 (+0.35z)| lr 3.44e-04 | 322.53 ms | 52.3% bf16 MFU | 1624996 tok/s
step 9245/19560 | loss 3.403584 (-0.53z)| norm 0.2734 (-0.47z)| lr 3.44e-04 | 322.82 ms | 52.3% bf16 MFU | 1624950 tok/s
step 9246/19560 | loss 3.446127 (+0.50z)| norm 0.2910 (+0.53z)| lr 3.44e-04 | 322.30 ms | 52.4% bf16 MFU | 1625038 tok/s
step 9247/19560 | loss 3.462542 (+0.92z)| norm 0.2921 (+0.59z)| lr 3.44e-04 | 322.51 ms | 52.3% bf16 MFU | 1625069 tok/s
step 9248/19560 | loss 3.366416 (-1.42z)| norm 0.2867 (+0.27z)| lr 3.44e-04 | 322.28 ms | 52.4% bf16 MFU | 1625156 tok/s
step 9249/19560 | loss 3.458063 (+0.81z)| norm 0.2573 (-1.43z)| lr 3.44e-04 | 322.72 ms | 52.3% bf16 MFU | 1625128 tok/s
step 9250/19560 | loss 3.393614 (-0.78z)| norm 0.2679 (-0.81z)| lr 3.44e-04 | 322.43 ms | 52.3% bf16 MFU | 1625175 tok/s
val loss 3.413161
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2870/10042 = 0.285800
step 9251/19560 | loss 3.445570 (+0.49z)| norm 0.2453 (-2.06z)| lr 3.44e-04 | 322.37 ms | 52.4% bf16 MFU | 1625233 tok/s
step 9252/19560 | loss 3.399996 (-0.62z)| norm 0.2691 (-0.71z)| lr 3.44e-04 | 322.64 ms | 52.3% bf16 MFU | 1625222 tok/s
step 9253/19560 | loss 3.416083 (-0.22z)| norm 0.2600 (-1.21z)| lr 3.44e-04 | 322.92 ms | 52.3% bf16 MFU | 1625140 tok/s
step 9254/19560 | loss 3.399903 (-0.61z)| norm 0.2719 (-0.54z)| lr 3.44e-04 | 322.68 ms | 52.3% bf16 MFU | 1625122 tok/s
step 9255/19560 | loss 3.414478 (-0.23z)| norm 0.2579 (-1.33z)| lr 3.44e-04 | 322.81 ms | 52.3% bf16 MFU | 1625074 tok/s
step 9256/19560 | loss 3.407038 (-0.43z)| norm 0.2789 (-0.16z)| lr 3.44e-04 | 322.82 ms | 52.3% bf16 MFU | 1625026 tok/s
step 9257/19560 | loss 3.389021 (-0.87z)| norm 0.2743 (-0.42z)| lr 3.44e-04 | 322.89 ms | 52.3% bf16 MFU | 1624960 tok/s
step 9258/19560 | loss 3.447663 (+0.60z)| norm 0.2799 (-0.10z)| lr 3.43e-04 | 322.75 ms | 52.3% bf16 MFU | 1624935 tok/s
step 9259/19560 | loss 3.470957 (+1.17z)| norm 0.2639 (-1.01z)| lr 3.43e-04 | 322.89 ms | 52.3% bf16 MFU | 1624876 tok/s
step 9260/19560 | loss 3.382469 (-1.04z)| norm 0.2933 (+0.66z)| lr 3.43e-04 | 322.59 ms | 52.3% bf16 MFU | 1624895 tok/s
step 9261/19560 | loss 3.421422 (-0.07z)| norm 0.2697 (-0.68z)| lr 3.43e-04 | 322.73 ms | 52.3% bf16 MFU | 1624878 tok/s
step 9262/19560 | loss 3.392023 (-0.80z)| norm 0.2837 (+0.11z)| lr 3.43e-04 | 322.72 ms | 52.3% bf16 MFU | 1624863 tok/s
step 9263/19560 | loss 3.494771 (+1.73z)| norm 0.2643 (-0.98z)| lr 3.43e-04 | 322.68 ms | 52.3% bf16 MFU | 1624860 tok/s
step 9264/19560 | loss 3.344530 (-1.93z)| norm 0.2889 (+0.40z)| lr 3.43e-04 | 322.64 ms | 52.3% bf16 MFU | 1624868 tok/s
step 9265/19560 | loss 3.465139 (+0.99z)| norm 0.2751 (-0.38z)| lr 3.43e-04 | 322.63 ms | 52.3% bf16 MFU | 1624875 tok/s
step 9266/19560 | loss 3.412289 (-0.28z)| norm 0.2984 (+0.93z)| lr 3.43e-04 | 323.07 ms | 52.2% bf16 MFU | 1624774 tok/s
step 9267/19560 | loss 3.382241 (-1.00z)| norm 0.2907 (+0.49z)| lr 3.43e-04 | 322.93 ms | 52.3% bf16 MFU | 1624711 tok/s
step 9268/19560 | loss 3.425657 (+0.04z)| norm 0.2943 (+0.68z)| lr 3.43e-04 | 322.26 ms | 52.4% bf16 MFU | 1624822 tok/s
step 9269/19560 | loss 3.438115 (+0.33z)| norm 0.2716 (-0.63z)| lr 3.43e-04 | 322.86 ms | 52.3% bf16 MFU | 1624774 tok/s
step 9270/19560 | loss 3.449979 (+0.62z)| norm 0.2712 (-0.64z)| lr 3.43e-04 | 323.14 ms | 52.2% bf16 MFU | 1624658 tok/s
step 9271/19560 | loss 3.374284 (-1.23z)| norm 0.2584 (-1.37z)| lr 3.43e-04 | 322.66 ms | 52.3% bf16 MFU | 1624669 tok/s
step 9272/19560 | loss 3.409103 (-0.37z)| norm 0.2674 (-0.85z)| lr 3.43e-04 | 323.03 ms | 52.2% bf16 MFU | 1624588 tok/s
step 9273/19560 | loss 3.348776 (-1.83z)| norm 0.2713 (-0.62z)| lr 3.43e-04 | 323.08 ms | 52.2% bf16 MFU | 1624498 tok/s
step 9274/19560 | loss 3.442684 (+0.45z)| norm 0.2719 (-0.59z)| lr 3.43e-04 | 322.69 ms | 52.3% bf16 MFU | 1624511 tok/s
step 9275/19560 | loss 3.440725 (+0.41z)| norm 0.2706 (-0.66z)| lr 3.43e-04 | 322.86 ms | 52.3% bf16 MFU | 1624479 tok/s
step 9276/19560 | loss 3.467121 (+1.04z)| norm 0.2512 (-1.74z)| lr 3.43e-04 | 322.46 ms | 52.3% bf16 MFU | 1624551 tok/s
step 9277/19560 | loss 3.432765 (+0.20z)| norm 0.2533 (-1.59z)| lr 3.43e-04 | 322.94 ms | 52.3% bf16 MFU | 1624497 tok/s
step 9278/19560 | loss 3.435764 (+0.28z)| norm 0.3047 (+1.26z)| lr 3.42e-04 | 322.83 ms | 52.3% bf16 MFU | 1624474 tok/s
step 9279/19560 | loss 3.470587 (+1.12z)| norm 0.3236 (+2.25z)| lr 3.42e-04 | 322.72 ms | 52.3% bf16 MFU | 1624479 tok/s
step 9280/19560 | loss 3.436845 (+0.31z)| norm 0.2571 (-1.37z)| lr 3.42e-04 | 322.63 ms | 52.3% bf16 MFU | 1624508 tok/s
step 9281/19560 | loss 3.485362 (+1.49z)| norm 0.2842 (+0.10z)| lr 3.42e-04 | 323.30 ms | 52.2% bf16 MFU | 1624366 tok/s
step 9282/19560 | loss 3.443853 (+0.46z)| norm 0.2977 (+0.82z)| lr 3.42e-04 | 322.70 ms | 52.3% bf16 MFU | 1624383 tok/s
step 9283/19560 | loss 3.419714 (-0.13z)| norm 0.2495 (-1.77z)| lr 3.42e-04 | 323.48 ms | 52.2% bf16 MFU | 1624203 tok/s
step 9284/19560 | loss 3.389152 (-0.88z)| norm 0.3105 (+1.50z)| lr 3.42e-04 | 322.59 ms | 52.3% bf16 MFU | 1624256 tok/s
step 9285/19560 | loss 3.424518 (+0.01z)| norm 0.2972 (+0.88z)| lr 3.42e-04 | 322.48 ms | 52.3% bf16 MFU | 1624332 tok/s
step 9286/19560 | loss 3.380416 (-1.08z)| norm 0.2609 (-1.20z)| lr 3.42e-04 | 323.00 ms | 52.3% bf16 MFU | 1624274 tok/s
step 9287/19560 | loss 3.424868 (+0.05z)| norm 0.2949 (+0.78z)| lr 3.42e-04 | 323.82 ms | 52.1% bf16 MFU | 1624013 tok/s
step 9288/19560 | loss 3.389166 (-0.85z)| norm 0.2502 (-1.80z)| lr 3.42e-04 | 322.31 ms | 52.4% bf16 MFU | 1624146 tok/s
step 9289/19560 | loss 3.423180 (+0.03z)| norm 0.3037 (+1.29z)| lr 3.42e-04 | 323.13 ms | 52.2% bf16 MFU | 1624064 tok/s
step 9290/19560 | loss 3.397919 (-0.62z)| norm 0.2612 (-1.15z)| lr 3.42e-04 | 322.83 ms | 52.3% bf16 MFU | 1624064 tok/s
step 9291/19560 | loss 3.409080 (-0.34z)| norm 0.2662 (-0.86z)| lr 3.42e-04 | 322.75 ms | 52.3% bf16 MFU | 1624081 tok/s
step 9292/19560 | loss 3.354329 (-1.75z)| norm 0.2740 (-0.40z)| lr 3.42e-04 | 322.80 ms | 52.3% bf16 MFU | 1624087 tok/s
step 9293/19560 | loss 3.453323 (+0.83z)| norm 0.2734 (-0.43z)| lr 3.42e-04 | 323.25 ms | 52.2% bf16 MFU | 1623980 tok/s
step 9294/19560 | loss 3.447568 (+0.67z)| norm 0.2576 (-1.32z)| lr 3.42e-04 | 322.67 ms | 52.3% bf16 MFU | 1624022 tok/s
step 9295/19560 | loss 3.389339 (-0.83z)| norm 0.2697 (-0.61z)| lr 3.42e-04 | 322.56 ms | 52.3% bf16 MFU | 1624090 tok/s
step 9296/19560 | loss 3.409102 (-0.31z)| norm 0.2635 (-0.96z)| lr 3.42e-04 | 323.24 ms | 52.2% bf16 MFU | 1623984 tok/s
step 9297/19560 | loss 3.377214 (-1.13z)| norm 0.2709 (-0.52z)| lr 3.42e-04 | 322.47 ms | 52.3% bf16 MFU | 1624077 tok/s
step 9298/19560 | loss 3.419411 (-0.04z)| norm 0.2507 (-1.66z)| lr 3.41e-04 | 322.83 ms | 52.3% bf16 MFU | 1624075 tok/s
step 9299/19560 | loss 3.419945 (-0.00z)| norm 0.2495 (-1.69z)| lr 3.41e-04 | 323.22 ms | 52.2% bf16 MFU | 1623976 tok/s
step 9300/19560 | loss 3.478252 (+1.51z)| norm 0.2645 (-0.83z)| lr 3.41e-04 | 322.86 ms | 52.3% bf16 MFU | 1623970 tok/s
step 9301/19560 | loss 3.397611 (-0.60z)| norm 0.2483 (-1.72z)| lr 3.41e-04 | 323.24 ms | 52.2% bf16 MFU | 1623870 tok/s
step 9302/19560 | loss 3.426388 (+0.15z)| norm 0.2756 (-0.17z)| lr 3.41e-04 | 322.24 ms | 52.4% bf16 MFU | 1624026 tok/s
step 9303/19560 | loss 3.312884 (-2.73z)| norm 0.2788 (+0.04z)| lr 3.41e-04 | 322.63 ms | 52.3% bf16 MFU | 1624077 tok/s
step 9304/19560 | loss 3.379971 (-1.00z)| norm 0.2783 (+0.01z)| lr 3.41e-04 | 323.72 ms | 52.1% bf16 MFU | 1623853 tok/s
step 9305/19560 | loss 3.415244 (-0.12z)| norm 0.2700 (-0.47z)| lr 3.41e-04 | 322.57 ms | 52.3% bf16 MFU | 1623927 tok/s
step 9306/19560 | loss 3.395101 (-0.63z)| norm 0.2928 (+0.87z)| lr 3.41e-04 | 323.10 ms | 52.2% bf16 MFU | 1623864 tok/s
step 9307/19560 | loss 3.533846 (+2.86z)| norm 0.2440 (-1.96z)| lr 3.41e-04 | 322.74 ms | 52.3% bf16 MFU | 1623896 tok/s
step 9308/19560 | loss 3.431159 (+0.27z)| norm 0.3143 (+2.08z)| lr 3.41e-04 | 322.48 ms | 52.3% bf16 MFU | 1623990 tok/s
step 9309/19560 | loss 3.417886 (-0.07z)| norm 0.2892 (+0.63z)| lr 3.41e-04 | 322.30 ms | 52.4% bf16 MFU | 1624126 tok/s
step 9310/19560 | loss 3.422184 (+0.06z)| norm 0.2912 (+0.76z)| lr 3.41e-04 | 322.73 ms | 52.3% bf16 MFU | 1624148 tok/s
step 9311/19560 | loss 3.485607 (+1.65z)| norm 0.3011 (+1.36z)| lr 3.41e-04 | 323.08 ms | 52.2% bf16 MFU | 1624079 tok/s
step 9312/19560 | loss 3.553445 (+3.19z)| norm 0.2779 (-0.01z)| lr 3.41e-04 | 322.72 ms | 52.3% bf16 MFU | 1624104 tok/s
step 9313/19560 | loss 3.422571 (+0.04z)| norm 0.2992 (+1.24z)| lr 3.41e-04 | 322.61 ms | 52.3% bf16 MFU | 1624155 tok/s
step 9314/19560 | loss 3.472648 (+1.25z)| norm 0.2728 (-0.30z)| lr 3.41e-04 | 322.99 ms | 52.3% bf16 MFU | 1624109 tok/s
step 9315/19560 | loss 3.450166 (+0.72z)| norm 0.2952 (+1.01z)| lr 3.41e-04 | 322.32 ms | 52.4% bf16 MFU | 1624235 tok/s
step 9316/19560 | loss 3.455505 (+0.83z)| norm 0.2913 (+0.77z)| lr 3.41e-04 | 323.12 ms | 52.2% bf16 MFU | 1624152 tok/s
step 9317/19560 | loss 3.423570 (+0.06z)| norm 0.2869 (+0.52z)| lr 3.41e-04 | 322.58 ms | 52.3% bf16 MFU | 1624209 tok/s
step 9318/19560 | loss 3.428918 (+0.20z)| norm 0.2774 (-0.04z)| lr 3.41e-04 | 322.65 ms | 52.3% bf16 MFU | 1624246 tok/s
step 9319/19560 | loss 3.570322 (+3.53z)| norm 0.2864 (+0.49z)| lr 3.40e-04 | 323.03 ms | 52.2% bf16 MFU | 1624186 tok/s
step 9320/19560 | loss 3.482838 (+1.44z)| norm 0.2759 (-0.12z)| lr 3.40e-04 | 323.48 ms | 52.2% bf16 MFU | 1624016 tok/s
step 9321/19560 | loss 3.464703 (+0.99z)| norm 0.2972 (+1.11z)| lr 3.40e-04 | 322.55 ms | 52.3% bf16 MFU | 1624088 tok/s
step 9322/19560 | loss 3.459089 (+0.85z)| norm 0.3077 (+1.70z)| lr 3.40e-04 | 322.95 ms | 52.3% bf16 MFU | 1624055 tok/s
step 9323/19560 | loss 3.409983 (-0.30z)| norm 0.3035 (+1.44z)| lr 3.40e-04 | 323.28 ms | 52.2% bf16 MFU | 1623941 tok/s
step 9324/19560 | loss 3.383781 (-0.91z)| norm 0.3057 (+1.56z)| lr 3.40e-04 | 322.86 ms | 52.3% bf16 MFU | 1623939 tok/s
step 9325/19560 | loss 3.470131 (+1.13z)| norm 0.2943 (+0.89z)| lr 3.40e-04 | 323.24 ms | 52.2% bf16 MFU | 1623841 tok/s
step 9326/19560 | loss 3.467571 (+1.06z)| norm 0.2957 (+0.96z)| lr 3.40e-04 | 322.90 ms | 52.3% bf16 MFU | 1623832 tok/s
step 9327/19560 | loss 3.431810 (+0.20z)| norm 0.2625 (-0.96z)| lr 3.40e-04 | 322.60 ms | 52.3% bf16 MFU | 1623901 tok/s
step 9328/19560 | loss 3.405968 (-0.43z)| norm 0.3096 (+1.73z)| lr 3.40e-04 | 323.61 ms | 52.2% bf16 MFU | 1623712 tok/s
step 9329/19560 | loss 3.451692 (+0.67z)| norm 0.2535 (-1.45z)| lr 3.40e-04 | 322.51 ms | 52.3% bf16 MFU | 1623810 tok/s
step 9330/19560 | loss 3.391675 (-0.77z)| norm 0.2816 (+0.16z)| lr 3.40e-04 | 322.84 ms | 52.3% bf16 MFU | 1623818 tok/s
step 9331/19560 | loss 3.436102 (+0.30z)| norm 0.2723 (-0.37z)| lr 3.40e-04 | 323.57 ms | 52.2% bf16 MFU | 1623643 tok/s
step 9332/19560 | loss 3.454785 (+0.73z)| norm 0.2711 (-0.44z)| lr 3.40e-04 | 323.17 ms | 52.2% bf16 MFU | 1623576 tok/s
step 9333/19560 | loss 3.399625 (-0.58z)| norm 0.2883 (+0.54z)| lr 3.40e-04 | 322.87 ms | 52.3% bf16 MFU | 1623590 tok/s
step 9334/19560 | loss 3.446181 (+0.52z)| norm 0.2572 (-1.22z)| lr 3.40e-04 | 322.48 ms | 52.3% bf16 MFU | 1623699 tok/s
step 9335/19560 | loss 3.440565 (+0.38z)| norm 0.2810 (+0.12z)| lr 3.40e-04 | 322.77 ms | 52.3% bf16 MFU | 1623732 tok/s
step 9336/19560 | loss 3.418458 (-0.15z)| norm 0.2833 (+0.25z)| lr 3.40e-04 | 323.24 ms | 52.2% bf16 MFU | 1623644 tok/s
step 9337/19560 | loss 3.403785 (-0.49z)| norm 0.2877 (+0.49z)| lr 3.40e-04 | 322.87 ms | 52.3% bf16 MFU | 1623653 tok/s
step 9338/19560 | loss 3.408731 (-0.38z)| norm 0.2692 (-0.58z)| lr 3.40e-04 | 322.55 ms | 52.3% bf16 MFU | 1623742 tok/s
step 9339/19560 | loss 3.459961 (+0.85z)| norm 0.2673 (-0.69z)| lr 3.39e-04 | 322.94 ms | 52.3% bf16 MFU | 1623728 tok/s
step 9340/19560 | loss 3.417983 (-0.17z)| norm 0.2861 (+0.39z)| lr 3.39e-04 | 322.47 ms | 52.3% bf16 MFU | 1623834 tok/s
step 9341/19560 | loss 3.452569 (+0.66z)| norm 0.2835 (+0.24z)| lr 3.39e-04 | 322.70 ms | 52.3% bf16 MFU | 1623876 tok/s
step 9342/19560 | loss 3.395315 (-0.78z)| norm 0.2862 (+0.38z)| lr 3.39e-04 | 322.78 ms | 52.3% bf16 MFU | 1623898 tok/s
step 9343/19560 | loss 3.474441 (+1.19z)| norm 0.2919 (+0.71z)| lr 3.39e-04 | 322.57 ms | 52.3% bf16 MFU | 1623971 tok/s
step 9344/19560 | loss 3.414359 (-0.30z)| norm 0.2647 (-0.86z)| lr 3.39e-04 | 322.33 ms | 52.4% bf16 MFU | 1624099 tok/s
step 9345/19560 | loss 3.389789 (-0.91z)| norm 0.2799 (+0.03z)| lr 3.39e-04 | 323.72 ms | 52.1% bf16 MFU | 1623873 tok/s
step 9346/19560 | loss 3.456420 (+0.75z)| norm 0.2665 (-0.74z)| lr 3.39e-04 | 323.14 ms | 52.2% bf16 MFU | 1623804 tok/s
step 9347/19560 | loss 3.457751 (+0.77z)| norm 0.2839 (+0.27z)| lr 3.39e-04 | 323.35 ms | 52.2% bf16 MFU | 1623684 tok/s
step 9348/19560 | loss 3.455708 (+0.71z)| norm 0.2876 (+0.50z)| lr 3.39e-04 | 323.12 ms | 52.2% bf16 MFU | 1623628 tok/s
step 9349/19560 | loss 3.373840 (-1.36z)| norm 0.2708 (-0.47z)| lr 3.39e-04 | 323.24 ms | 52.2% bf16 MFU | 1623545 tok/s
step 9350/19560 | loss 3.441908 (+0.36z)| norm 0.2816 (+0.17z)| lr 3.39e-04 | 323.16 ms | 52.2% bf16 MFU | 1623486 tok/s
step 9351/19560 | loss 3.381757 (-1.16z)| norm 0.2806 (+0.12z)| lr 3.39e-04 | 322.62 ms | 52.3% bf16 MFU | 1623568 tok/s
step 9352/19560 | loss 3.416733 (-0.27z)| norm 0.2697 (-0.53z)| lr 3.39e-04 | 323.56 ms | 52.2% bf16 MFU | 1623408 tok/s
step 9353/19560 | loss 3.409210 (-0.46z)| norm 0.2755 (-0.17z)| lr 3.39e-04 | 322.87 ms | 52.3% bf16 MFU | 1623429 tok/s
step 9354/19560 | loss 3.422941 (-0.12z)| norm 0.2769 (-0.09z)| lr 3.39e-04 | 323.16 ms | 52.2% bf16 MFU | 1623376 tok/s
step 9355/19560 | loss 3.361411 (-1.65z)| norm 0.2803 (+0.10z)| lr 3.39e-04 | 323.16 ms | 52.2% bf16 MFU | 1623325 tok/s
step 9356/19560 | loss 3.409941 (-0.42z)| norm 0.3015 (+1.36z)| lr 3.39e-04 | 322.51 ms | 52.3% bf16 MFU | 1623441 tok/s
step 9357/19560 | loss 3.412963 (-0.36z)| norm 0.2787 (-0.01z)| lr 3.39e-04 | 323.34 ms | 52.2% bf16 MFU | 1623341 tok/s
step 9358/19560 | loss 3.401350 (-0.65z)| norm 0.2948 (+0.97z)| lr 3.39e-04 | 323.19 ms | 52.2% bf16 MFU | 1623287 tok/s
step 9359/19560 | loss 3.406418 (-0.51z)| norm 0.3045 (+1.52z)| lr 3.38e-04 | 322.38 ms | 52.4% bf16 MFU | 1623438 tok/s
step 9360/19560 | loss 3.482491 (+1.42z)| norm 0.2540 (-1.47z)| lr 3.38e-04 | 323.06 ms | 52.2% bf16 MFU | 1623411 tok/s
step 9361/19560 | loss 3.415249 (-0.28z)| norm 0.2976 (+1.10z)| lr 3.38e-04 | 323.34 ms | 52.2% bf16 MFU | 1623313 tok/s
step 9362/19560 | loss 3.458653 (+0.82z)| norm 0.2604 (-1.09z)| lr 3.38e-04 | 322.92 ms | 52.3% bf16 MFU | 1623327 tok/s
step 9363/19560 | loss 3.468327 (+1.07z)| norm 0.2734 (-0.32z)| lr 3.38e-04 | 323.02 ms | 52.2% bf16 MFU | 1623315 tok/s
step 9364/19560 | loss 3.516856 (+2.25z)| norm 0.2843 (+0.32z)| lr 3.38e-04 | 323.32 ms | 52.2% bf16 MFU | 1623227 tok/s
step 9365/19560 | loss 3.435175 (+0.20z)| norm 0.2723 (-0.39z)| lr 3.38e-04 | 323.13 ms | 52.2% bf16 MFU | 1623193 tok/s
step 9366/19560 | loss 3.392496 (-0.86z)| norm 0.2853 (+0.40z)| lr 3.38e-04 | 323.03 ms | 52.2% bf16 MFU | 1623184 tok/s
step 9367/19560 | loss 3.390326 (-0.91z)| norm 0.2549 (-1.41z)| lr 3.38e-04 | 322.70 ms | 52.3% bf16 MFU | 1623261 tok/s
step 9368/19560 | loss 3.473814 (+1.18z)| norm 0.2647 (-0.82z)| lr 3.38e-04 | 322.54 ms | 52.3% bf16 MFU | 1623372 tok/s
step 9369/19560 | loss 3.458562 (+0.78z)| norm 0.2694 (-0.54z)| lr 3.38e-04 | 323.03 ms | 52.2% bf16 MFU | 1623355 tok/s
step 9370/19560 | loss 3.462722 (+0.87z)| norm 0.2637 (-0.88z)| lr 3.38e-04 | 323.62 ms | 52.2% bf16 MFU | 1623190 tok/s
step 9371/19560 | loss 3.389178 (-0.97z)| norm 0.2624 (-0.94z)| lr 3.38e-04 | 322.64 ms | 52.3% bf16 MFU | 1623281 tok/s
step 9372/19560 | loss 3.401764 (-0.64z)| norm 0.2531 (-1.48z)| lr 3.38e-04 | 322.36 ms | 52.4% bf16 MFU | 1623437 tok/s
step 9373/19560 | loss 3.531305 (+2.53z)| norm 0.2926 (+0.90z)| lr 3.38e-04 | 323.30 ms | 52.2% bf16 MFU | 1623350 tok/s
step 9374/19560 | loss 3.499157 (+1.71z)| norm 0.2692 (-0.50z)| lr 3.38e-04 | 322.95 ms | 52.3% bf16 MFU | 1623353 tok/s
step 9375/19560 | loss 3.446391 (+0.43z)| norm 0.2590 (-1.10z)| lr 3.38e-04 | 322.97 ms | 52.3% bf16 MFU | 1623352 tok/s
step 9376/19560 | loss 3.399298 (-0.73z)| norm 0.2674 (-0.59z)| lr 3.38e-04 | 322.76 ms | 52.3% bf16 MFU | 1623405 tok/s
step 9377/19560 | loss 3.394077 (-0.84z)| norm 0.2645 (-0.77z)| lr 3.38e-04 | 322.49 ms | 52.3% bf16 MFU | 1623522 tok/s
step 9378/19560 | loss 3.365987 (-1.51z)| norm 0.2751 (-0.13z)| lr 3.38e-04 | 322.65 ms | 52.3% bf16 MFU | 1623594 tok/s
step 9379/19560 | loss 3.392663 (-0.85z)| norm 0.2570 (-1.25z)| lr 3.37e-04 | 322.76 ms | 52.3% bf16 MFU | 1623633 tok/s
step 9380/19560 | loss 3.421810 (-0.15z)| norm 0.2831 (+0.35z)| lr 3.37e-04 | 322.29 ms | 52.4% bf16 MFU | 1623790 tok/s
step 9381/19560 | loss 3.430404 (+0.06z)| norm 0.2790 (+0.08z)| lr 3.37e-04 | 322.53 ms | 52.3% bf16 MFU | 1623877 tok/s
step 9382/19560 | loss 3.417909 (-0.25z)| norm 0.2601 (-1.07z)| lr 3.37e-04 | 323.57 ms | 52.2% bf16 MFU | 1623699 tok/s
step 9383/19560 | loss 3.405516 (-0.55z)| norm 0.2620 (-0.95z)| lr 3.37e-04 | 322.59 ms | 52.3% bf16 MFU | 1623777 tok/s
step 9384/19560 | loss 3.417656 (-0.26z)| norm 0.2822 (+0.28z)| lr 3.37e-04 | 322.86 ms | 52.3% bf16 MFU | 1623783 tok/s
step 9385/19560 | loss 3.398551 (-0.73z)| norm 0.2791 (+0.10z)| lr 3.37e-04 | 322.70 ms | 52.3% bf16 MFU | 1623829 tok/s
step 9386/19560 | loss 3.366122 (-1.49z)| norm 0.2790 (+0.09z)| lr 3.37e-04 | 322.70 ms | 52.3% bf16 MFU | 1623872 tok/s
step 9387/19560 | loss 3.399670 (-0.67z)| norm 0.2779 (+0.01z)| lr 3.37e-04 | 323.52 ms | 52.2% bf16 MFU | 1623707 tok/s
step 9388/19560 | loss 3.408010 (-0.47z)| norm 0.2697 (-0.48z)| lr 3.37e-04 | 322.75 ms | 52.3% bf16 MFU | 1623743 tok/s
step 9389/19560 | loss 3.368790 (-1.40z)| norm 0.2706 (-0.43z)| lr 3.37e-04 | 322.55 ms | 52.3% bf16 MFU | 1623829 tok/s
step 9390/19560 | loss 3.385008 (-1.01z)| norm 0.2707 (-0.41z)| lr 3.37e-04 | 322.55 ms | 52.3% bf16 MFU | 1623910 tok/s
step 9391/19560 | loss 3.479974 (+1.29z)| norm 0.2598 (-1.09z)| lr 3.37e-04 | 322.21 ms | 52.4% bf16 MFU | 1624073 tok/s
step 9392/19560 | loss 3.413160 (-0.35z)| norm 0.2688 (-0.52z)| lr 3.37e-04 | 322.96 ms | 52.3% bf16 MFU | 1624037 tok/s
step 9393/19560 | loss 3.408783 (-0.45z)| norm 0.2395 (-2.26z)| lr 3.37e-04 | 322.77 ms | 52.3% bf16 MFU | 1624052 tok/s
step 9394/19560 | loss 3.439651 (+0.31z)| norm 0.2867 (+0.60z)| lr 3.37e-04 | 322.45 ms | 52.3% bf16 MFU | 1624148 tok/s
step 9395/19560 | loss 3.398242 (-0.72z)| norm 0.2630 (-0.83z)| lr 3.37e-04 | 322.98 ms | 52.3% bf16 MFU | 1624106 tok/s
step 9396/19560 | loss 3.384566 (-1.04z)| norm 0.2377 (-2.30z)| lr 3.37e-04 | 322.57 ms | 52.3% bf16 MFU | 1624167 tok/s
step 9397/19560 | loss 3.398888 (-0.68z)| norm 0.2698 (-0.38z)| lr 3.37e-04 | 323.17 ms | 52.2% bf16 MFU | 1624076 tok/s
step 9398/19560 | loss 3.446718 (+0.50z)| norm 0.2464 (-1.74z)| lr 3.37e-04 | 322.45 ms | 52.3% bf16 MFU | 1624170 tok/s
step 9399/19560 | loss 3.483299 (+1.38z)| norm 0.2461 (-1.74z)| lr 3.36e-04 | 322.45 ms | 52.3% bf16 MFU | 1624260 tok/s
step 9400/19560 | loss 3.433453 (+0.15z)| norm 0.2692 (-0.39z)| lr 3.36e-04 | 322.78 ms | 52.3% bf16 MFU | 1624261 tok/s
step 9401/19560 | loss 3.425845 (-0.06z)| norm 0.2363 (-2.26z)| lr 3.36e-04 | 323.02 ms | 52.2% bf16 MFU | 1624202 tok/s
step 9402/19560 | loss 3.444388 (+0.41z)| norm 0.2742 (-0.08z)| lr 3.36e-04 | 322.89 ms | 52.3% bf16 MFU | 1624179 tok/s
step 9403/19560 | loss 3.391920 (-0.89z)| norm 0.2726 (-0.18z)| lr 3.36e-04 | 322.10 ms | 52.4% bf16 MFU | 1624355 tok/s
step 9404/19560 | loss 3.391119 (-0.90z)| norm 0.2521 (-1.36z)| lr 3.36e-04 | 322.41 ms | 52.3% bf16 MFU | 1624444 tok/s
step 9405/19560 | loss 3.478980 (+1.27z)| norm 0.2788 (+0.17z)| lr 3.36e-04 | 322.54 ms | 52.3% bf16 MFU | 1624498 tok/s
step 9406/19560 | loss 3.472588 (+1.10z)| norm 0.2843 (+0.50z)| lr 3.36e-04 | 322.43 ms | 52.3% bf16 MFU | 1624577 tok/s
step 9407/19560 | loss 3.441433 (+0.34z)| norm 0.2930 (+1.06z)| lr 3.36e-04 | 322.36 ms | 52.4% bf16 MFU | 1624668 tok/s
step 9408/19560 | loss 3.421423 (-0.15z)| norm 0.2808 (+0.31z)| lr 3.36e-04 | 322.49 ms | 52.3% bf16 MFU | 1624723 tok/s
step 9409/19560 | loss 3.420683 (-0.16z)| norm 0.2779 (+0.14z)| lr 3.36e-04 | 322.55 ms | 52.3% bf16 MFU | 1624758 tok/s
step 9410/19560 | loss 3.418828 (-0.20z)| norm 0.3056 (+1.80z)| lr 3.36e-04 | 322.61 ms | 52.3% bf16 MFU | 1624777 tok/s
step 9411/19560 | loss 3.454445 (+0.68z)| norm 0.2952 (+1.16z)| lr 3.36e-04 | 322.43 ms | 52.3% bf16 MFU | 1624841 tok/s
step 9412/19560 | loss 3.457003 (+0.73z)| norm 0.2788 (+0.19z)| lr 3.36e-04 | 322.62 ms | 52.3% bf16 MFU | 1624854 tok/s
step 9413/19560 | loss 3.340709 (-2.11z)| norm 0.3122 (+2.20z)| lr 3.36e-04 | 323.16 ms | 52.2% bf16 MFU | 1624730 tok/s
step 9414/19560 | loss 3.435354 (+0.20z)| norm 0.2526 (-1.40z)| lr 3.36e-04 | 322.63 ms | 52.3% bf16 MFU | 1624747 tok/s
step 9415/19560 | loss 3.374478 (-1.28z)| norm 0.2839 (+0.50z)| lr 3.36e-04 | 322.74 ms | 52.3% bf16 MFU | 1624735 tok/s
step 9416/19560 | loss 3.395993 (-0.76z)| norm 0.2609 (-0.91z)| lr 3.36e-04 | 322.48 ms | 52.3% bf16 MFU | 1624787 tok/s
step 9417/19560 | loss 3.441001 (+0.34z)| norm 0.2832 (+0.46z)| lr 3.36e-04 | 322.36 ms | 52.4% bf16 MFU | 1624867 tok/s
step 9418/19560 | loss 3.459995 (+0.79z)| norm 0.2900 (+0.87z)| lr 3.36e-04 | 322.56 ms | 52.3% bf16 MFU | 1624894 tok/s
step 9419/19560 | loss 3.443259 (+0.38z)| norm 0.2689 (-0.43z)| lr 3.35e-04 | 322.89 ms | 52.3% bf16 MFU | 1624835 tok/s
step 9420/19560 | loss 3.370189 (-1.42z)| norm 0.2564 (-1.19z)| lr 3.35e-04 | 322.65 ms | 52.3% bf16 MFU | 1624840 tok/s
step 9421/19560 | loss 3.402342 (-0.62z)| norm 0.2702 (-0.34z)| lr 3.35e-04 | 322.50 ms | 52.3% bf16 MFU | 1624883 tok/s
step 9422/19560 | loss 3.385766 (-1.02z)| norm 0.2843 (+0.51z)| lr 3.35e-04 | 322.61 ms | 52.3% bf16 MFU | 1624895 tok/s
step 9423/19560 | loss 3.429618 (+0.05z)| norm 0.2556 (-1.24z)| lr 3.35e-04 | 322.32 ms | 52.4% bf16 MFU | 1624980 tok/s
step 9424/19560 | loss 3.502450 (+1.81z)| norm 0.2836 (+0.47z)| lr 3.35e-04 | 322.93 ms | 52.3% bf16 MFU | 1624906 tok/s
step 9425/19560 | loss 3.564057 (+3.16z)| norm 0.2613 (-0.89z)| lr 3.35e-04 | 322.80 ms | 52.3% bf16 MFU | 1624869 tok/s
step 9426/19560 | loss 3.432077 (+0.05z)| norm 0.2718 (-0.27z)| lr 3.35e-04 | 322.54 ms | 52.3% bf16 MFU | 1624901 tok/s
step 9427/19560 | loss 3.393736 (-0.84z)| norm 0.2678 (-0.52z)| lr 3.35e-04 | 322.59 ms | 52.3% bf16 MFU | 1624917 tok/s
step 9428/19560 | loss 3.428361 (-0.02z)| norm 0.2731 (-0.20z)| lr 3.35e-04 | 322.67 ms | 52.3% bf16 MFU | 1624914 tok/s
step 9429/19560 | loss 3.431750 (+0.05z)| norm 0.2528 (-1.48z)| lr 3.35e-04 | 322.52 ms | 52.3% bf16 MFU | 1624947 tok/s
step 9430/19560 | loss 3.443682 (+0.33z)| norm 0.2759 (-0.03z)| lr 3.35e-04 | 322.50 ms | 52.3% bf16 MFU | 1624985 tok/s
step 9431/19560 | loss 3.410707 (-0.48z)| norm 0.2359 (-2.45z)| lr 3.35e-04 | 322.52 ms | 52.3% bf16 MFU | 1625017 tok/s
step 9432/19560 | loss 3.477991 (+1.14z)| norm 0.2963 (+1.22z)| lr 3.35e-04 | 322.47 ms | 52.3% bf16 MFU | 1625059 tok/s
step 9433/19560 | loss 3.358592 (-1.73z)| norm 0.2626 (-0.81z)| lr 3.35e-04 | 322.63 ms | 52.3% bf16 MFU | 1625059 tok/s
step 9434/19560 | loss 3.444680 (+0.33z)| norm 0.2774 (+0.08z)| lr 3.35e-04 | 323.12 ms | 52.2% bf16 MFU | 1624935 tok/s
step 9435/19560 | loss 3.425455 (-0.12z)| norm 0.2741 (-0.13z)| lr 3.35e-04 | 322.17 ms | 52.4% bf16 MFU | 1625057 tok/s
step 9436/19560 | loss 3.422276 (-0.19z)| norm 0.2813 (+0.34z)| lr 3.35e-04 | 322.83 ms | 52.3% bf16 MFU | 1625006 tok/s
step 9437/19560 | loss 3.421096 (-0.22z)| norm 0.2694 (-0.41z)| lr 3.35e-04 | 322.68 ms | 52.3% bf16 MFU | 1624996 tok/s
step 9438/19560 | loss 3.432518 (+0.06z)| norm 0.3164 (+2.50z)| lr 3.35e-04 | 322.31 ms | 52.4% bf16 MFU | 1625080 tok/s
step 9439/19560 | loss 3.445301 (+0.38z)| norm 0.2706 (-0.32z)| lr 3.35e-04 | 322.75 ms | 52.3% bf16 MFU | 1625049 tok/s
step 9440/19560 | loss 3.462773 (+0.87z)| norm 0.2674 (-0.51z)| lr 3.34e-04 | 323.13 ms | 52.2% bf16 MFU | 1624924 tok/s
step 9441/19560 | loss 3.484311 (+1.40z)| norm 0.3059 (+1.88z)| lr 3.34e-04 | 322.33 ms | 52.4% bf16 MFU | 1625006 tok/s
step 9442/19560 | loss 3.398172 (-0.79z)| norm 0.2740 (-0.11z)| lr 3.34e-04 | 322.39 ms | 52.3% bf16 MFU | 1625068 tok/s
step 9443/19560 | loss 3.488088 (+1.49z)| norm 0.2973 (+1.34z)| lr 3.34e-04 | 322.81 ms | 52.3% bf16 MFU | 1625022 tok/s
step 9444/19560 | loss 3.411378 (-0.45z)| norm 0.3081 (+1.97z)| lr 3.34e-04 | 322.77 ms | 52.3% bf16 MFU | 1624987 tok/s
step 9445/19560 | loss 3.364311 (-1.62z)| norm 0.2872 (+0.70z)| lr 3.34e-04 | 322.83 ms | 52.3% bf16 MFU | 1624940 tok/s
step 9446/19560 | loss 3.414347 (-0.36z)| norm 0.3146 (+2.31z)| lr 3.34e-04 | 322.73 ms | 52.3% bf16 MFU | 1624921 tok/s
step 9447/19560 | loss 3.487561 (+1.57z)| norm 0.3160 (+2.33z)| lr 3.34e-04 | 322.81 ms | 52.3% bf16 MFU | 1624881 tok/s
step 9448/19560 | loss 3.450596 (+0.61z)| norm 0.2810 (+0.27z)| lr 3.34e-04 | 322.77 ms | 52.3% bf16 MFU | 1624854 tok/s
step 9449/19560 | loss 3.452591 (+0.67z)| norm 0.2977 (+1.25z)| lr 3.34e-04 | 322.31 ms | 52.4% bf16 MFU | 1624944 tok/s
step 9450/19560 | loss 3.432281 (+0.13z)| norm 0.2647 (-0.68z)| lr 3.34e-04 | 322.68 ms | 52.3% bf16 MFU | 1624936 tok/s
step 9451/19560 | loss 3.429868 (+0.06z)| norm 0.3088 (+1.93z)| lr 3.34e-04 | 322.95 ms | 52.3% bf16 MFU | 1624862 tok/s
step 9452/19560 | loss 3.467342 (+1.05z)| norm 0.2773 (+0.08z)| lr 3.34e-04 | 322.47 ms | 52.3% bf16 MFU | 1624912 tok/s
step 9453/19560 | loss 3.381235 (-1.23z)| norm 0.2795 (+0.22z)| lr 3.34e-04 | 322.92 ms | 52.3% bf16 MFU | 1624846 tok/s
step 9454/19560 | loss 3.419372 (-0.20z)| norm 0.2797 (+0.24z)| lr 3.34e-04 | 322.98 ms | 52.3% bf16 MFU | 1624769 tok/s
step 9455/19560 | loss 3.433297 (+0.17z)| norm 0.2793 (+0.21z)| lr 3.34e-04 | 322.58 ms | 52.3% bf16 MFU | 1624796 tok/s
step 9456/19560 | loss 3.448240 (+0.56z)| norm 0.2668 (-0.54z)| lr 3.34e-04 | 322.34 ms | 52.4% bf16 MFU | 1624880 tok/s
step 9457/19560 | loss 3.472896 (+1.21z)| norm 0.2819 (+0.38z)| lr 3.34e-04 | 322.93 ms | 52.3% bf16 MFU | 1624812 tok/s
step 9458/19560 | loss 3.389090 (-1.02z)| norm 0.3017 (+1.59z)| lr 3.34e-04 | 322.82 ms | 52.3% bf16 MFU | 1624776 tok/s
step 9459/19560 | loss 3.414068 (-0.35z)| norm 0.2844 (+0.52z)| lr 3.34e-04 | 322.26 ms | 52.4% bf16 MFU | 1624883 tok/s
step 9460/19560 | loss 3.422533 (-0.12z)| norm 0.3049 (+1.74z)| lr 3.33e-04 | 322.77 ms | 52.3% bf16 MFU | 1624855 tok/s
step 9461/19560 | loss 3.446362 (+0.51z)| norm 0.2727 (-0.21z)| lr 3.33e-04 | 322.69 ms | 52.3% bf16 MFU | 1624848 tok/s
step 9462/19560 | loss 3.412710 (-0.39z)| norm 0.2787 (+0.15z)| lr 3.33e-04 | 322.47 ms | 52.3% bf16 MFU | 1624898 tok/s
step 9463/19560 | loss 3.506064 (+2.06z)| norm 0.2792 (+0.18z)| lr 3.33e-04 | 323.10 ms | 52.2% bf16 MFU | 1624788 tok/s
step 9464/19560 | loss 3.395244 (-0.85z)| norm 0.2961 (+1.20z)| lr 3.33e-04 | 322.39 ms | 52.3% bf16 MFU | 1624860 tok/s
step 9465/19560 | loss 3.402099 (-0.67z)| norm 0.2477 (-1.71z)| lr 3.33e-04 | 322.63 ms | 52.3% bf16 MFU | 1624869 tok/s
step 9466/19560 | loss 3.391122 (-0.95z)| norm 0.3037 (+1.63z)| lr 3.33e-04 | 322.53 ms | 52.3% bf16 MFU | 1624902 tok/s
step 9467/19560 | loss 3.465262 (+0.99z)| norm 0.2829 (+0.39z)| lr 3.33e-04 | 322.75 ms | 52.3% bf16 MFU | 1624880 tok/s
step 9468/19560 | loss 3.398269 (-0.76z)| norm 0.2879 (+0.68z)| lr 3.33e-04 | 322.75 ms | 52.3% bf16 MFU | 1624859 tok/s
step 9469/19560 | loss 3.430142 (+0.08z)| norm 0.2687 (-0.45z)| lr 3.33e-04 | 322.57 ms | 52.3% bf16 MFU | 1624882 tok/s
step 9470/19560 | loss 3.417952 (-0.24z)| norm 0.2757 (-0.03z)| lr 3.33e-04 | 322.78 ms | 52.3% bf16 MFU | 1624853 tok/s
step 9471/19560 | loss 3.451258 (+0.64z)| norm 0.2632 (-0.77z)| lr 3.33e-04 | 323.50 ms | 52.2% bf16 MFU | 1624645 tok/s
step 9472/19560 | loss 3.421206 (-0.16z)| norm 0.2822 (+0.36z)| lr 3.33e-04 | 322.81 ms | 52.3% bf16 MFU | 1624621 tok/s
step 9473/19560 | loss 3.405468 (-0.57z)| norm 0.2669 (-0.55z)| lr 3.33e-04 | 323.29 ms | 52.2% bf16 MFU | 1624475 tok/s
step 9474/19560 | loss 3.392671 (-0.90z)| norm 0.2717 (-0.26z)| lr 3.33e-04 | 322.85 ms | 52.3% bf16 MFU | 1624447 tok/s
step 9475/19560 | loss 3.420556 (-0.16z)| norm 0.2725 (-0.21z)| lr 3.33e-04 | 322.96 ms | 52.3% bf16 MFU | 1624395 tok/s
step 9476/19560 | loss 3.370253 (-1.46z)| norm 0.3004 (+1.44z)| lr 3.33e-04 | 322.56 ms | 52.3% bf16 MFU | 1624446 tok/s
step 9477/19560 | loss 3.523122 (+2.48z)| norm 0.2938 (+1.04z)| lr 3.33e-04 | 322.47 ms | 52.3% bf16 MFU | 1624516 tok/s
step 9478/19560 | loss 3.456463 (+0.76z)| norm 0.3110 (+2.01z)| lr 3.33e-04 | 322.31 ms | 52.4% bf16 MFU | 1624624 tok/s
step 9479/19560 | loss 3.396101 (-0.80z)| norm 0.2805 (+0.23z)| lr 3.33e-04 | 322.71 ms | 52.3% bf16 MFU | 1624624 tok/s
step 9480/19560 | loss 3.379694 (-1.21z)| norm 0.2859 (+0.54z)| lr 3.32e-04 | 322.48 ms | 52.3% bf16 MFU | 1624684 tok/s
step 9481/19560 | loss 3.370990 (-1.42z)| norm 0.2971 (+1.18z)| lr 3.32e-04 | 323.10 ms | 52.2% bf16 MFU | 1624582 tok/s
step 9482/19560 | loss 3.437275 (+0.27z)| norm 0.2822 (+0.31z)| lr 3.32e-04 | 322.87 ms | 52.3% bf16 MFU | 1624545 tok/s
step 9483/19560 | loss 3.503578 (+1.92z)| norm 0.3011 (+1.39z)| lr 3.32e-04 | 322.46 ms | 52.3% bf16 MFU | 1624614 tok/s
step 9484/19560 | loss 3.432920 (+0.12z)| norm 0.2929 (+0.92z)| lr 3.32e-04 | 322.55 ms | 52.3% bf16 MFU | 1624654 tok/s
step 9485/19560 | loss 3.457134 (+0.73z)| norm 0.3087 (+1.80z)| lr 3.32e-04 | 323.06 ms | 52.2% bf16 MFU | 1624566 tok/s
step 9486/19560 | loss 3.386091 (-1.07z)| norm 0.2799 (+0.16z)| lr 3.32e-04 | 322.12 ms | 52.4% bf16 MFU | 1624718 tok/s
step 9487/19560 | loss 3.378986 (-1.24z)| norm 0.2861 (+0.53z)| lr 3.32e-04 | 322.39 ms | 52.3% bf16 MFU | 1624794 tok/s
step 9488/19560 | loss 3.395498 (-0.81z)| norm 0.2801 (+0.17z)| lr 3.32e-04 | 322.82 ms | 52.3% bf16 MFU | 1624758 tok/s
step 9489/19560 | loss 3.416463 (-0.28z)| norm 0.2705 (-0.38z)| lr 3.32e-04 | 323.39 ms | 52.2% bf16 MFU | 1624581 tok/s
step 9490/19560 | loss 3.479631 (+1.31z)| norm 0.2818 (+0.28z)| lr 3.32e-04 | 322.58 ms | 52.3% bf16 MFU | 1624616 tok/s
step 9491/19560 | loss 3.437658 (+0.26z)| norm 0.2676 (-0.55z)| lr 3.32e-04 | 322.81 ms | 52.3% bf16 MFU | 1624592 tok/s
step 9492/19560 | loss 3.362331 (-1.63z)| norm 0.2812 (+0.25z)| lr 3.32e-04 | 322.64 ms | 52.3% bf16 MFU | 1624612 tok/s
step 9493/19560 | loss 3.424560 (-0.04z)| norm 0.2568 (-1.17z)| lr 3.32e-04 | 322.96 ms | 52.3% bf16 MFU | 1624550 tok/s
step 9494/19560 | loss 3.413147 (-0.33z)| norm 0.2741 (-0.16z)| lr 3.32e-04 | 322.86 ms | 52.3% bf16 MFU | 1624516 tok/s
step 9495/19560 | loss 3.348840 (-1.95z)| norm 0.3289 (+2.93z)| lr 3.32e-04 | 323.19 ms | 52.2% bf16 MFU | 1624401 tok/s
step 9496/19560 | loss 3.489193 (+1.60z)| norm 0.3127 (+1.96z)| lr 3.32e-04 | 322.79 ms | 52.3% bf16 MFU | 1624392 tok/s
step 9497/19560 | loss 3.370372 (-1.38z)| norm 0.2914 (+0.75z)| lr 3.32e-04 | 322.65 ms | 52.3% bf16 MFU | 1624418 tok/s
step 9498/19560 | loss 3.414044 (-0.27z)| norm 0.2753 (-0.15z)| lr 3.32e-04 | 322.67 ms | 52.3% bf16 MFU | 1624438 tok/s
step 9499/19560 | loss 3.340934 (-2.07z)| norm 0.2967 (+1.03z)| lr 3.32e-04 | 322.23 ms | 52.4% bf16 MFU | 1624570 tok/s
step 9500/19560 | loss 3.457988 (+0.82z)| norm 0.2829 (+0.25z)| lr 3.31e-04 | 323.65 ms | 52.1% bf16 MFU | 1624338 tok/s
val loss 3.408037
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2870/10042 = 0.285800
step 9501/19560 | loss 3.398712 (-0.64z)| norm 0.2971 (+1.04z)| lr 3.31e-04 | 322.53 ms | 52.3% bf16 MFU | 1624398 tok/s
step 9502/19560 | loss 3.445105 (+0.56z)| norm 0.2628 (-0.88z)| lr 3.31e-04 | 322.60 ms | 52.3% bf16 MFU | 1624438 tok/s
step 9503/19560 | loss 3.410753 (-0.32z)| norm 0.2675 (-0.63z)| lr 3.31e-04 | 323.18 ms | 52.2% bf16 MFU | 1624329 tok/s
step 9504/19560 | loss 3.390606 (-0.84z)| norm 0.2527 (-1.44z)| lr 3.31e-04 | 323.72 ms | 52.1% bf16 MFU | 1624091 tok/s
step 9505/19560 | loss 3.410942 (-0.32z)| norm 0.2714 (-0.40z)| lr 3.31e-04 | 322.98 ms | 52.3% bf16 MFU | 1624050 tok/s
step 9506/19560 | loss 3.357731 (-1.69z)| norm 0.2807 (+0.12z)| lr 3.31e-04 | 323.17 ms | 52.2% bf16 MFU | 1623963 tok/s
step 9507/19560 | loss 3.389888 (-0.86z)| norm 0.2544 (-1.35z)| lr 3.31e-04 | 322.80 ms | 52.3% bf16 MFU | 1623974 tok/s
step 9508/19560 | loss 3.437574 (+0.37z)| norm 0.2530 (-1.41z)| lr 3.31e-04 | 323.01 ms | 52.2% bf16 MFU | 1623932 tok/s
step 9509/19560 | loss 3.419964 (-0.08z)| norm 0.2771 (-0.07z)| lr 3.31e-04 | 323.19 ms | 52.2% bf16 MFU | 1623848 tok/s
step 9510/19560 | loss 3.413815 (-0.24z)| norm 0.2705 (-0.44z)| lr 3.31e-04 | 322.96 ms | 52.3% bf16 MFU | 1623824 tok/s
step 9511/19560 | loss 3.431769 (+0.22z)| norm 0.2834 (+0.27z)| lr 3.31e-04 | 323.27 ms | 52.2% bf16 MFU | 1623725 tok/s
step 9512/19560 | loss 3.430604 (+0.18z)| norm 0.2881 (+0.53z)| lr 3.31e-04 | 323.14 ms | 52.2% bf16 MFU | 1623662 tok/s
step 9513/19560 | loss 3.381630 (-1.07z)| norm 0.2529 (-1.41z)| lr 3.31e-04 | 323.10 ms | 52.2% bf16 MFU | 1623614 tok/s
step 9514/19560 | loss 3.487438 (+1.62z)| norm 0.2912 (+0.70z)| lr 3.31e-04 | 323.11 ms | 52.2% bf16 MFU | 1623565 tok/s
step 9515/19560 | loss 3.356934 (-1.70z)| norm 0.2852 (+0.37z)| lr 3.31e-04 | 322.93 ms | 52.3% bf16 MFU | 1623563 tok/s
step 9516/19560 | loss 3.425980 (+0.05z)| norm 0.2887 (+0.55z)| lr 3.31e-04 | 322.67 ms | 52.3% bf16 MFU | 1623628 tok/s
step 9517/19560 | loss 3.378067 (-1.17z)| norm 0.3037 (+1.36z)| lr 3.31e-04 | 322.79 ms | 52.3% bf16 MFU | 1623658 tok/s
step 9518/19560 | loss 3.417153 (-0.19z)| norm 0.2831 (+0.22z)| lr 3.31e-04 | 322.97 ms | 52.3% bf16 MFU | 1623641 tok/s
step 9519/19560 | loss 3.363410 (-1.53z)| norm 0.2638 (-0.84z)| lr 3.31e-04 | 323.29 ms | 52.2% bf16 MFU | 1623545 tok/s
step 9520/19560 | loss 3.470274 (+1.18z)| norm 0.2994 (+1.10z)| lr 3.30e-04 | 323.38 ms | 52.2% bf16 MFU | 1623431 tok/s
step 9521/19560 | loss 3.402656 (-0.54z)| norm 0.2699 (-0.54z)| lr 3.30e-04 | 322.39 ms | 52.4% bf16 MFU | 1623573 tok/s
step 9522/19560 | loss 3.420963 (-0.07z)| norm 0.2783 (-0.07z)| lr 3.30e-04 | 322.99 ms | 52.3% bf16 MFU | 1623557 tok/s
step 9523/19560 | loss 3.404759 (-0.48z)| norm 0.2726 (-0.39z)| lr 3.30e-04 | 323.75 ms | 52.1% bf16 MFU | 1623351 tok/s
step 9524/19560 | loss 3.417156 (-0.18z)| norm 0.2863 (+0.36z)| lr 3.30e-04 | 323.12 ms | 52.2% bf16 MFU | 1623311 tok/s
step 9525/19560 | loss 3.452929 (+0.72z)| norm 0.2840 (+0.23z)| lr 3.30e-04 | 322.81 ms | 52.3% bf16 MFU | 1623353 tok/s
step 9526/19560 | loss 3.382871 (-1.05z)| norm 0.2656 (-0.85z)| lr 3.30e-04 | 322.90 ms | 52.3% bf16 MFU | 1623369 tok/s
step 9527/19560 | loss 3.452681 (+0.74z)| norm 0.2880 (+0.44z)| lr 3.30e-04 | 323.31 ms | 52.2% bf16 MFU | 1623282 tok/s
step 9528/19560 | loss 3.415507 (-0.21z)| norm 0.2782 (-0.14z)| lr 3.30e-04 | 322.65 ms | 52.3% bf16 MFU | 1623366 tok/s
step 9529/19560 | loss 3.358057 (-1.65z)| norm 0.3546 (+4.13z)| lr 3.30e-04 | 322.86 ms | 52.3% bf16 MFU | 1623391 tok/s
step 9530/19560 | loss 3.426531 (+0.09z)| norm 0.3087 (+1.51z)| lr 3.30e-04 | 323.10 ms | 52.2% bf16 MFU | 1623355 tok/s
step 9531/19560 | loss 3.471906 (+1.22z)| norm 0.2744 (-0.42z)| lr 3.30e-04 | 322.90 ms | 52.3% bf16 MFU | 1623372 tok/s
step 9532/19560 | loss 3.513239 (+2.20z)| norm 0.2928 (+0.60z)| lr 3.30e-04 | 322.35 ms | 52.4% bf16 MFU | 1623526 tok/s
step 9533/19560 | loss 3.499510 (+1.85z)| norm 0.2848 (+0.15z)| lr 3.30e-04 | 322.78 ms | 52.3% bf16 MFU | 1623564 tok/s
step 9534/19560 | loss 3.420678 (-0.09z)| norm 0.2687 (-0.75z)| lr 3.30e-04 | 322.77 ms | 52.3% bf16 MFU | 1623602 tok/s
step 9535/19560 | loss 3.407461 (-0.41z)| norm 0.2589 (-1.28z)| lr 3.30e-04 | 322.81 ms | 52.3% bf16 MFU | 1623630 tok/s
step 9536/19560 | loss 3.378646 (-1.11z)| norm 0.2682 (-0.75z)| lr 3.30e-04 | 322.81 ms | 52.3% bf16 MFU | 1623656 tok/s
step 9537/19560 | loss 3.447454 (+0.58z)| norm 0.2508 (-1.70z)| lr 3.30e-04 | 323.17 ms | 52.2% bf16 MFU | 1623590 tok/s
step 9538/19560 | loss 3.448501 (+0.60z)| norm 0.2877 (+0.36z)| lr 3.30e-04 | 322.98 ms | 52.3% bf16 MFU | 1623574 tok/s
step 9539/19560 | loss 3.383805 (-0.98z)| norm 0.2726 (-0.48z)| lr 3.30e-04 | 322.44 ms | 52.3% bf16 MFU | 1623695 tok/s
step 9540/19560 | loss 3.386996 (-0.89z)| norm 0.2851 (+0.22z)| lr 3.29e-04 | 322.47 ms | 52.3% bf16 MFU | 1623801 tok/s
step 9541/19560 | loss 3.405843 (-0.44z)| norm 0.2619 (-1.06z)| lr 3.29e-04 | 323.20 ms | 52.2% bf16 MFU | 1623721 tok/s
step 9542/19560 | loss 3.408195 (-0.38z)| norm 0.3054 (+1.36z)| lr 3.29e-04 | 322.83 ms | 52.3% bf16 MFU | 1623736 tok/s
step 9543/19560 | loss 3.497842 (+1.82z)| norm 0.2833 (+0.12z)| lr 3.29e-04 | 322.95 ms | 52.3% bf16 MFU | 1623721 tok/s
step 9544/19560 | loss 3.473863 (+1.21z)| norm 0.2939 (+0.70z)| lr 3.29e-04 | 322.90 ms | 52.3% bf16 MFU | 1623719 tok/s
step 9545/19560 | loss 3.450199 (+0.62z)| norm 0.2683 (-0.74z)| lr 3.29e-04 | 322.36 ms | 52.4% bf16 MFU | 1623853 tok/s
step 9546/19560 | loss 3.388040 (-0.90z)| norm 0.2980 (+0.93z)| lr 3.29e-04 | 322.72 ms | 52.3% bf16 MFU | 1623890 tok/s
step 9547/19560 | loss 3.427848 (+0.09z)| norm 0.3167 (+1.94z)| lr 3.29e-04 | 322.79 ms | 52.3% bf16 MFU | 1623908 tok/s
step 9548/19560 | loss 3.439544 (+0.37z)| norm 0.2899 (+0.44z)| lr 3.29e-04 | 323.39 ms | 52.2% bf16 MFU | 1623774 tok/s
step 9549/19560 | loss 3.352725 (-1.76z)| norm 0.2819 (-0.01z)| lr 3.29e-04 | 322.71 ms | 52.3% bf16 MFU | 1623818 tok/s
step 9550/19560 | loss 3.443189 (+0.45z)| norm 0.3039 (+1.21z)| lr 3.29e-04 | 322.73 ms | 52.3% bf16 MFU | 1623855 tok/s
step 9551/19560 | loss 3.378059 (-1.14z)| norm 0.2841 (+0.09z)| lr 3.29e-04 | 322.84 ms | 52.3% bf16 MFU | 1623862 tok/s
step 9552/19560 | loss 3.483960 (+1.47z)| norm 0.3115 (+1.60z)| lr 3.29e-04 | 322.51 ms | 52.3% bf16 MFU | 1623951 tok/s
step 9553/19560 | loss 3.404797 (-0.47z)| norm 0.2963 (+0.74z)| lr 3.29e-04 | 322.64 ms | 52.3% bf16 MFU | 1624004 tok/s
step 9554/19560 | loss 3.389563 (-0.86z)| norm 0.2833 (+0.01z)| lr 3.29e-04 | 322.65 ms | 52.3% bf16 MFU | 1624052 tok/s
step 9555/19560 | loss 3.433996 (+0.28z)| norm 0.2644 (-1.05z)| lr 3.29e-04 | 322.86 ms | 52.3% bf16 MFU | 1624044 tok/s
step 9556/19560 | loss 3.442869 (+0.51z)| norm 0.2988 (+0.87z)| lr 3.29e-04 | 322.45 ms | 52.3% bf16 MFU | 1624139 tok/s
step 9557/19560 | loss 3.474622 (+1.31z)| norm 0.2587 (-1.38z)| lr 3.29e-04 | 322.97 ms | 52.3% bf16 MFU | 1624098 tok/s
step 9558/19560 | loss 3.495613 (+1.82z)| norm 0.2848 (+0.08z)| lr 3.29e-04 | 322.49 ms | 52.3% bf16 MFU | 1624179 tok/s
step 9559/19560 | loss 3.334553 (-2.21z)| norm 0.2526 (-1.76z)| lr 3.29e-04 | 322.65 ms | 52.3% bf16 MFU | 1624218 tok/s
step 9560/19560 | loss 3.402310 (-0.51z)| norm 0.2607 (-1.28z)| lr 3.28e-04 | 322.43 ms | 52.3% bf16 MFU | 1624310 tok/s
step 9561/19560 | loss 3.378043 (-1.13z)| norm 0.2859 (+0.14z)| lr 3.28e-04 | 322.47 ms | 52.3% bf16 MFU | 1624387 tok/s
step 9562/19560 | loss 3.457706 (+0.87z)| norm 0.2830 (-0.03z)| lr 3.28e-04 | 323.06 ms | 52.2% bf16 MFU | 1624312 tok/s
step 9563/19560 | loss 3.415602 (-0.18z)| norm 0.2741 (-0.53z)| lr 3.28e-04 | 322.16 ms | 52.4% bf16 MFU | 1624466 tok/s
step 9564/19560 | loss 3.451457 (+0.71z)| norm 0.2803 (-0.18z)| lr 3.28e-04 | 322.10 ms | 52.4% bf16 MFU | 1624630 tok/s
step 9565/19560 | loss 3.379339 (-1.09z)| norm 0.2815 (-0.12z)| lr 3.28e-04 | 322.62 ms | 52.3% bf16 MFU | 1624652 tok/s
step 9566/19560 | loss 3.456500 (+0.83z)| norm 0.2536 (-1.69z)| lr 3.28e-04 | 322.86 ms | 52.3% bf16 MFU | 1624614 tok/s
step 9567/19560 | loss 3.421494 (-0.03z)| norm 0.2576 (-1.45z)| lr 3.28e-04 | 322.47 ms | 52.3% bf16 MFU | 1624676 tok/s
step 9568/19560 | loss 3.437414 (+0.37z)| norm 0.2525 (-1.72z)| lr 3.28e-04 | 322.32 ms | 52.4% bf16 MFU | 1624773 tok/s
step 9569/19560 | loss 3.432430 (+0.26z)| norm 0.2607 (-1.23z)| lr 3.28e-04 | 322.35 ms | 52.4% bf16 MFU | 1624856 tok/s
step 9570/19560 | loss 3.503955 (+2.01z)| norm 0.2553 (-1.52z)| lr 3.28e-04 | 322.45 ms | 52.3% bf16 MFU | 1624911 tok/s
step 9571/19560 | loss 3.390919 (-0.79z)| norm 0.2698 (-0.69z)| lr 3.28e-04 | 322.32 ms | 52.4% bf16 MFU | 1624996 tok/s
step 9572/19560 | loss 3.393704 (-0.71z)| norm 0.2661 (-0.89z)| lr 3.28e-04 | 322.86 ms | 52.3% bf16 MFU | 1624941 tok/s
step 9573/19560 | loss 3.419688 (-0.07z)| norm 0.2611 (-1.15z)| lr 3.28e-04 | 322.44 ms | 52.3% bf16 MFU | 1624994 tok/s
step 9574/19560 | loss 3.440633 (+0.45z)| norm 0.2997 (+1.03z)| lr 3.28e-04 | 322.67 ms | 52.3% bf16 MFU | 1624987 tok/s
step 9575/19560 | loss 3.408379 (-0.35z)| norm 0.2487 (-1.83z)| lr 3.28e-04 | 322.32 ms | 52.4% bf16 MFU | 1625069 tok/s
step 9576/19560 | loss 3.450395 (+0.72z)| norm 0.2820 (+0.06z)| lr 3.28e-04 | 322.22 ms | 52.4% bf16 MFU | 1625171 tok/s
step 9577/19560 | loss 3.389889 (-0.81z)| norm 0.2592 (-1.22z)| lr 3.28e-04 | 322.37 ms | 52.4% bf16 MFU | 1625230 tok/s
step 9578/19560 | loss 3.357328 (-1.61z)| norm 0.2739 (-0.39z)| lr 3.28e-04 | 322.49 ms | 52.3% bf16 MFU | 1625255 tok/s
step 9579/19560 | loss 3.438962 (+0.45z)| norm 0.2754 (-0.29z)| lr 3.28e-04 | 322.96 ms | 52.3% bf16 MFU | 1625162 tok/s
step 9580/19560 | loss 3.403343 (-0.44z)| norm 0.3030 (+1.27z)| lr 3.27e-04 | 323.16 ms | 52.2% bf16 MFU | 1625023 tok/s
step 9581/19560 | loss 3.392036 (-0.73z)| norm 0.2803 (-0.02z)| lr 3.27e-04 | 322.34 ms | 52.4% bf16 MFU | 1625098 tok/s
step 9582/19560 | loss 3.433764 (+0.33z)| norm 0.2541 (-1.49z)| lr 3.27e-04 | 322.82 ms | 52.3% bf16 MFU | 1625048 tok/s
step 9583/19560 | loss 3.405005 (-0.40z)| norm 0.2921 (+0.65z)| lr 3.27e-04 | 322.52 ms | 52.3% bf16 MFU | 1625075 tok/s
step 9584/19560 | loss 3.427612 (+0.18z)| norm 0.2695 (-0.63z)| lr 3.27e-04 | 322.48 ms | 52.3% bf16 MFU | 1625112 tok/s
step 9585/19560 | loss 3.419753 (-0.01z)| norm 0.2765 (-0.23z)| lr 3.27e-04 | 322.31 ms | 52.4% bf16 MFU | 1625189 tok/s
step 9586/19560 | loss 3.376671 (-1.10z)| norm 0.2745 (-0.33z)| lr 3.27e-04 | 322.62 ms | 52.3% bf16 MFU | 1625184 tok/s
step 9587/19560 | loss 3.430777 (+0.27z)| norm 0.2897 (+0.53z)| lr 3.27e-04 | 322.72 ms | 52.3% bf16 MFU | 1625155 tok/s
step 9588/19560 | loss 3.386595 (-0.84z)| norm 0.2476 (-1.82z)| lr 3.27e-04 | 323.10 ms | 52.2% bf16 MFU | 1625031 tok/s
step 9589/19560 | loss 3.412277 (-0.19z)| norm 0.2861 (+0.34z)| lr 3.27e-04 | 322.28 ms | 52.4% bf16 MFU | 1625120 tok/s
step 9590/19560 | loss 3.412431 (-0.18z)| norm 0.2816 (+0.09z)| lr 3.27e-04 | 322.19 ms | 52.4% bf16 MFU | 1625226 tok/s
step 9591/19560 | loss 3.411419 (-0.19z)| norm 0.2841 (+0.22z)| lr 3.27e-04 | 322.69 ms | 52.3% bf16 MFU | 1625203 tok/s
step 9592/19560 | loss 3.439967 (+0.54z)| norm 0.2766 (-0.19z)| lr 3.27e-04 | 322.73 ms | 52.3% bf16 MFU | 1625171 tok/s
step 9593/19560 | loss 3.431129 (+0.31z)| norm 0.2789 (-0.07z)| lr 3.27e-04 | 322.63 ms | 52.3% bf16 MFU | 1625164 tok/s
step 9594/19560 | loss 3.416606 (-0.08z)| norm 0.2683 (-0.67z)| lr 3.27e-04 | 322.77 ms | 52.3% bf16 MFU | 1625123 tok/s
step 9595/19560 | loss 3.357013 (-1.60z)| norm 0.2670 (-0.74z)| lr 3.27e-04 | 322.55 ms | 52.3% bf16 MFU | 1625140 tok/s
step 9596/19560 | loss 3.469311 (+1.29z)| norm 0.3049 (+1.43z)| lr 3.27e-04 | 322.29 ms | 52.4% bf16 MFU | 1625222 tok/s
step 9597/19560 | loss 3.319048 (-2.50z)| norm 0.2748 (-0.30z)| lr 3.27e-04 | 322.93 ms | 52.3% bf16 MFU | 1625136 tok/s
step 9598/19560 | loss 3.426394 (+0.20z)| norm 0.2720 (-0.45z)| lr 3.27e-04 | 322.51 ms | 52.3% bf16 MFU | 1625162 tok/s
step 9599/19560 | loss 3.398706 (-0.49z)| norm 0.2724 (-0.44z)| lr 3.27e-04 | 322.29 ms | 52.4% bf16 MFU | 1625243 tok/s
step 9600/19560 | loss 3.516841 (+2.41z)| norm 0.2780 (-0.11z)| lr 3.27e-04 | 322.74 ms | 52.3% bf16 MFU | 1625206 tok/s
step 9601/19560 | loss 3.467596 (+1.18z)| norm 0.2989 (+1.07z)| lr 3.26e-04 | 322.48 ms | 52.3% bf16 MFU | 1625235 tok/s
step 9602/19560 | loss 3.374290 (-1.10z)| norm 0.2793 (-0.06z)| lr 3.26e-04 | 322.68 ms | 52.3% bf16 MFU | 1625213 tok/s
step 9603/19560 | loss 3.390387 (-0.70z)| norm 0.2762 (-0.24z)| lr 3.26e-04 | 322.42 ms | 52.3% bf16 MFU | 1625256 tok/s
step 9604/19560 | loss 3.341550 (-1.86z)| norm 0.2918 (+0.67z)| lr 3.26e-04 | 322.54 ms | 52.3% bf16 MFU | 1625269 tok/s
step 9605/19560 | loss 3.390992 (-0.66z)| norm 0.3027 (+1.28z)| lr 3.26e-04 | 323.32 ms | 52.2% bf16 MFU | 1625084 tok/s
step 9606/19560 | loss 3.572043 (+3.61z)| norm 0.2770 (-0.18z)| lr 3.26e-04 | 322.11 ms | 52.4% bf16 MFU | 1625213 tok/s
step 9607/19560 | loss 3.410288 (-0.20z)| norm 0.3096 (+1.68z)| lr 3.26e-04 | 322.81 ms | 52.3% bf16 MFU | 1625159 tok/s
step 9608/19560 | loss 3.407198 (-0.28z)| norm 0.2905 (+0.58z)| lr 3.26e-04 | 322.48 ms | 52.3% bf16 MFU | 1625191 tok/s
step 9609/19560 | loss 3.412926 (-0.15z)| norm 0.2706 (-0.54z)| lr 3.26e-04 | 322.49 ms | 52.3% bf16 MFU | 1625218 tok/s
step 9610/19560 | loss 3.496989 (+1.81z)| norm 0.3049 (+1.40z)| lr 3.26e-04 | 322.72 ms | 52.3% bf16 MFU | 1625186 tok/s
step 9611/19560 | loss 3.463747 (+1.05z)| norm 0.2823 (+0.12z)| lr 3.26e-04 | 322.26 ms | 52.4% bf16 MFU | 1625271 tok/s
step 9612/19560 | loss 3.417464 (-0.04z)| norm 0.2945 (+0.82z)| lr 3.26e-04 | 322.62 ms | 52.3% bf16 MFU | 1625261 tok/s
step 9613/19560 | loss 3.419121 (+0.00z)| norm 0.2960 (+0.92z)| lr 3.26e-04 | 322.87 ms | 52.3% bf16 MFU | 1625191 tok/s
step 9614/19560 | loss 3.437354 (+0.43z)| norm 0.2916 (+0.66z)| lr 3.26e-04 | 322.59 ms | 52.3% bf16 MFU | 1625194 tok/s
step 9615/19560 | loss 3.409099 (-0.25z)| norm 0.2623 (-1.01z)| lr 3.26e-04 | 322.54 ms | 52.3% bf16 MFU | 1625209 tok/s
step 9616/19560 | loss 3.368961 (-1.20z)| norm 0.2766 (-0.19z)| lr 3.26e-04 | 322.57 ms | 52.3% bf16 MFU | 1625216 tok/s
step 9617/19560 | loss 3.448697 (+0.69z)| norm 0.2681 (-0.68z)| lr 3.26e-04 | 323.11 ms | 52.2% bf16 MFU | 1625086 tok/s
step 9618/19560 | loss 3.414952 (-0.10z)| norm 0.2377 (-2.34z)| lr 3.26e-04 | 322.85 ms | 52.3% bf16 MFU | 1625028 tok/s
step 9619/19560 | loss 3.419129 (+0.00z)| norm 0.2704 (-0.52z)| lr 3.26e-04 | 322.51 ms | 52.3% bf16 MFU | 1625060 tok/s
step 9620/19560 | loss 3.391438 (-0.67z)| norm 0.2648 (-0.82z)| lr 3.26e-04 | 322.71 ms | 52.3% bf16 MFU | 1625039 tok/s
step 9621/19560 | loss 3.486199 (+1.59z)| norm 0.2896 (+0.55z)| lr 3.25e-04 | 322.54 ms | 52.3% bf16 MFU | 1625061 tok/s
step 9622/19560 | loss 3.402745 (-0.40z)| norm 0.2591 (-1.15z)| lr 3.25e-04 | 322.88 ms | 52.3% bf16 MFU | 1624998 tok/s
step 9623/19560 | loss 3.464547 (+1.06z)| norm 0.2860 (+0.39z)| lr 3.25e-04 | 322.58 ms | 52.3% bf16 MFU | 1625012 tok/s
step 9624/19560 | loss 3.343124 (-1.83z)| norm 0.2689 (-0.59z)| lr 3.25e-04 | 323.18 ms | 52.2% bf16 MFU | 1624875 tok/s
step 9625/19560 | loss 3.628503 (+4.56z)| norm 0.2757 (-0.18z)| lr 3.25e-04 | 322.59 ms | 52.3% bf16 MFU | 1624893 tok/s
step 9626/19560 | loss 3.359375 (-1.35z)| norm 0.3009 (+1.27z)| lr 3.25e-04 | 322.64 ms | 52.3% bf16 MFU | 1624899 tok/s
step 9627/19560 | loss 3.435404 (+0.30z)| norm 0.2887 (+0.57z)| lr 3.25e-04 | 322.62 ms | 52.3% bf16 MFU | 1624909 tok/s
step 9628/19560 | loss 3.428100 (+0.15z)| norm 0.2860 (+0.41z)| lr 3.25e-04 | 322.82 ms | 52.3% bf16 MFU | 1624869 tok/s
step 9629/19560 | loss 3.367707 (-1.18z)| norm 0.3015 (+1.31z)| lr 3.25e-04 | 322.41 ms | 52.3% bf16 MFU | 1624934 tok/s
step 9630/19560 | loss 3.447815 (+0.59z)| norm 0.2877 (+0.50z)| lr 3.25e-04 | 322.41 ms | 52.3% bf16 MFU | 1624996 tok/s
step 9631/19560 | loss 3.406980 (-0.31z)| norm 0.2917 (+0.72z)| lr 3.25e-04 | 322.42 ms | 52.3% bf16 MFU | 1625052 tok/s
step 9632/19560 | loss 3.407991 (-0.30z)| norm 0.2939 (+0.83z)| lr 3.25e-04 | 322.67 ms | 52.3% bf16 MFU | 1625042 tok/s
step 9633/19560 | loss 3.389342 (-0.70z)| norm 0.2712 (-0.50z)| lr 3.25e-04 | 322.64 ms | 52.3% bf16 MFU | 1625038 tok/s
step 9634/19560 | loss 3.400355 (-0.47z)| norm 0.3093 (+1.70z)| lr 3.25e-04 | 322.54 ms | 52.3% bf16 MFU | 1625061 tok/s
step 9635/19560 | loss 3.383321 (-0.85z)| norm 0.2712 (-0.52z)| lr 3.25e-04 | 322.81 ms | 52.3% bf16 MFU | 1625016 tok/s
step 9636/19560 | loss 3.382001 (-0.87z)| norm 0.3639 (+4.49z)| lr 3.25e-04 | 322.84 ms | 52.3% bf16 MFU | 1624964 tok/s
step 9637/19560 | loss 3.516180 (+2.06z)| norm 0.3050 (+1.28z)| lr 3.25e-04 | 322.32 ms | 52.4% bf16 MFU | 1625046 tok/s
step 9638/19560 | loss 3.457141 (+0.76z)| norm 0.3384 (+2.95z)| lr 3.25e-04 | 322.24 ms | 52.4% bf16 MFU | 1625144 tok/s
step 9639/19560 | loss 3.370181 (-1.11z)| norm 0.2743 (-0.38z)| lr 3.25e-04 | 323.46 ms | 52.2% bf16 MFU | 1624931 tok/s
step 9640/19560 | loss 3.413703 (-0.17z)| norm 0.3029 (+1.10z)| lr 3.25e-04 | 322.23 ms | 52.4% bf16 MFU | 1625038 tok/s
step 9641/19560 | loss 3.484541 (+1.34z)| norm 0.2839 (+0.10z)| lr 3.24e-04 | 322.33 ms | 52.4% bf16 MFU | 1625114 tok/s
step 9642/19560 | loss 3.422968 (+0.02z)| norm 0.2885 (+0.34z)| lr 3.24e-04 | 322.21 ms | 52.4% bf16 MFU | 1625217 tok/s
step 9643/19560 | loss 3.493830 (+1.54z)| norm 0.2742 (-0.40z)| lr 3.24e-04 | 322.79 ms | 52.3% bf16 MFU | 1625167 tok/s
step 9644/19560 | loss 3.426471 (+0.08z)| norm 0.2541 (-1.43z)| lr 3.24e-04 | 322.25 ms | 52.4% bf16 MFU | 1625257 tok/s
step 9645/19560 | loss 3.377286 (-0.99z)| norm 0.2498 (-1.62z)| lr 3.24e-04 | 323.40 ms | 52.2% bf16 MFU | 1625053 tok/s
step 9646/19560 | loss 3.345280 (-1.65z)| norm 0.2523 (-1.46z)| lr 3.24e-04 | 322.98 ms | 52.3% bf16 MFU | 1624964 tok/s
step 9647/19560 | loss 3.440675 (+0.38z)| norm 0.2634 (-0.90z)| lr 3.24e-04 | 322.79 ms | 52.3% bf16 MFU | 1624928 tok/s
step 9648/19560 | loss 3.407854 (-0.32z)| norm 0.2807 (-0.01z)| lr 3.24e-04 | 322.80 ms | 52.3% bf16 MFU | 1624891 tok/s
step 9649/19560 | loss 3.383898 (-0.83z)| norm 0.2565 (-1.24z)| lr 3.24e-04 | 322.59 ms | 52.3% bf16 MFU | 1624908 tok/s
step 9650/19560 | loss 3.406762 (-0.33z)| norm 0.2887 (+0.41z)| lr 3.24e-04 | 322.55 ms | 52.3% bf16 MFU | 1624936 tok/s
step 9651/19560 | loss 3.403361 (-0.41z)| norm 0.2572 (-1.19z)| lr 3.24e-04 | 322.99 ms | 52.3% bf16 MFU | 1624850 tok/s
step 9652/19560 | loss 3.411327 (-0.23z)| norm 0.2658 (-0.75z)| lr 3.24e-04 | 322.70 ms | 52.3% bf16 MFU | 1624840 tok/s
step 9653/19560 | loss 3.391505 (-0.65z)| norm 0.2662 (-0.72z)| lr 3.24e-04 | 322.63 ms | 52.3% bf16 MFU | 1624850 tok/s
step 9654/19560 | loss 3.421204 (-0.02z)| norm 0.2601 (-1.02z)| lr 3.24e-04 | 322.87 ms | 52.3% bf16 MFU | 1624799 tok/s
step 9655/19560 | loss 3.348032 (-1.57z)| norm 0.2600 (-1.01z)| lr 3.24e-04 | 322.51 ms | 52.3% bf16 MFU | 1624842 tok/s
step 9656/19560 | loss 3.421371 (+0.00z)| norm 0.2930 (+0.64z)| lr 3.24e-04 | 323.18 ms | 52.2% bf16 MFU | 1624714 tok/s
step 9657/19560 | loss 3.482184 (+1.29z)| norm 0.2802 (+0.03z)| lr 3.24e-04 | 322.19 ms | 52.4% bf16 MFU | 1624840 tok/s
step 9658/19560 | loss 3.466781 (+0.95z)| norm 0.3122 (+1.73z)| lr 3.24e-04 | 323.38 ms | 52.2% bf16 MFU | 1624663 tok/s
step 9659/19560 | loss 3.446273 (+0.52z)| norm 0.2983 (+0.98z)| lr 3.24e-04 | 322.70 ms | 52.3% bf16 MFU | 1624663 tok/s
step 9660/19560 | loss 3.509075 (+1.87z)| norm 0.2971 (+0.91z)| lr 3.24e-04 | 322.57 ms | 52.3% bf16 MFU | 1624698 tok/s
step 9661/19560 | loss 3.387250 (-0.74z)| norm 0.2634 (-0.86z)| lr 3.23e-04 | 323.53 ms | 52.2% bf16 MFU | 1624490 tok/s
step 9662/19560 | loss 3.385260 (-0.78z)| norm 0.2866 (+0.36z)| lr 3.23e-04 | 322.98 ms | 52.3% bf16 MFU | 1624428 tok/s
step 9663/19560 | loss 3.382156 (-0.84z)| norm 0.2481 (-1.66z)| lr 3.23e-04 | 323.63 ms | 52.1% bf16 MFU | 1624208 tok/s
step 9664/19560 | loss 3.544446 (+2.58z)| norm 0.2874 (+0.40z)| lr 3.23e-04 | 322.52 ms | 52.3% bf16 MFU | 1624277 tok/s
step 9665/19560 | loss 3.454575 (+0.68z)| norm 0.2761 (-0.21z)| lr 3.23e-04 | 322.79 ms | 52.3% bf16 MFU | 1624274 tok/s
step 9666/19560 | loss 3.428499 (+0.14z)| norm 0.3032 (+1.21z)| lr 3.23e-04 | 323.85 ms | 52.1% bf16 MFU | 1624006 tok/s
step 9667/19560 | loss 3.432108 (+0.20z)| norm 0.2841 (+0.20z)| lr 3.23e-04 | 322.32 ms | 52.4% bf16 MFU | 1624136 tok/s
step 9668/19560 | loss 3.344786 (-1.62z)| norm 0.3172 (+1.90z)| lr 3.23e-04 | 323.11 ms | 52.2% bf16 MFU | 1624062 tok/s
step 9669/19560 | loss 3.395907 (-0.55z)| norm 0.2625 (-0.94z)| lr 3.23e-04 | 322.67 ms | 52.3% bf16 MFU | 1624100 tok/s
step 9670/19560 | loss 3.527834 (+2.16z)| norm 0.2867 (+0.33z)| lr 3.23e-04 | 323.02 ms | 52.2% bf16 MFU | 1624050 tok/s
step 9671/19560 | loss 3.380484 (-0.86z)| norm 0.2688 (-0.60z)| lr 3.23e-04 | 322.93 ms | 52.3% bf16 MFU | 1624024 tok/s
step 9672/19560 | loss 3.407234 (-0.30z)| norm 0.2720 (-0.43z)| lr 3.23e-04 | 323.24 ms | 52.2% bf16 MFU | 1623923 tok/s
step 9673/19560 | loss 3.430533 (+0.19z)| norm 0.2854 (+0.27z)| lr 3.23e-04 | 322.92 ms | 52.3% bf16 MFU | 1623906 tok/s
step 9674/19560 | loss 3.497948 (+1.56z)| norm 0.2529 (-1.41z)| lr 3.23e-04 | 322.96 ms | 52.3% bf16 MFU | 1623881 tok/s
step 9675/19560 | loss 3.403937 (-0.37z)| norm 0.2775 (-0.11z)| lr 3.23e-04 | 322.42 ms | 52.3% bf16 MFU | 1623992 tok/s
step 9676/19560 | loss 3.420984 (-0.02z)| norm 0.2536 (-1.35z)| lr 3.23e-04 | 322.97 ms | 52.3% bf16 MFU | 1623958 tok/s
step 9677/19560 | loss 3.356939 (-1.35z)| norm 0.2667 (-0.66z)| lr 3.23e-04 | 322.54 ms | 52.3% bf16 MFU | 1624034 tok/s
step 9678/19560 | loss 3.396925 (-0.51z)| norm 0.2757 (-0.17z)| lr 3.23e-04 | 323.22 ms | 52.2% bf16 MFU | 1623936 tok/s
step 9679/19560 | loss 3.410880 (-0.23z)| norm 0.2490 (-1.56z)| lr 3.23e-04 | 322.97 ms | 52.3% bf16 MFU | 1623907 tok/s
step 9680/19560 | loss 3.391210 (-0.62z)| norm 0.2485 (-1.56z)| lr 3.23e-04 | 322.88 ms | 52.3% bf16 MFU | 1623900 tok/s
step 9681/19560 | loss 3.440197 (+0.39z)| norm 0.2706 (-0.39z)| lr 3.22e-04 | 323.18 ms | 52.2% bf16 MFU | 1623819 tok/s
step 9682/19560 | loss 3.419582 (-0.04z)| norm 0.2502 (-1.44z)| lr 3.22e-04 | 323.11 ms | 52.2% bf16 MFU | 1623760 tok/s
step 9683/19560 | loss 3.387608 (-0.70z)| norm 0.2807 (+0.15z)| lr 3.22e-04 | 322.68 ms | 52.3% bf16 MFU | 1623811 tok/s
step 9684/19560 | loss 3.390433 (-0.63z)| norm 0.2535 (-1.26z)| lr 3.22e-04 | 322.59 ms | 52.3% bf16 MFU | 1623882 tok/s
step 9685/19560 | loss 3.405862 (-0.30z)| norm 0.2776 (-0.00z)| lr 3.22e-04 | 323.40 ms | 52.2% bf16 MFU | 1623746 tok/s
step 9686/19560 | loss 3.457834 (+0.80z)| norm 0.2981 (+1.06z)| lr 3.22e-04 | 322.75 ms | 52.3% bf16 MFU | 1623780 tok/s
step 9687/19560 | loss 3.391748 (-0.61z)| norm 0.2680 (-0.52z)| lr 3.22e-04 | 322.61 ms | 52.3% bf16 MFU | 1623850 tok/s
step 9688/19560 | loss 3.372512 (-1.01z)| norm 0.2710 (-0.37z)| lr 3.22e-04 | 323.93 ms | 52.1% bf16 MFU | 1623584 tok/s
step 9689/19560 | loss 3.452368 (+0.67z)| norm 0.2890 (+0.58z)| lr 3.22e-04 | 322.31 ms | 52.4% bf16 MFU | 1623737 tok/s
step 9690/19560 | loss 3.388012 (-0.69z)| norm 0.2825 (+0.24z)| lr 3.22e-04 | 323.05 ms | 52.2% bf16 MFU | 1623697 tok/s
step 9691/19560 | loss 3.331596 (-1.85z)| norm 0.2819 (+0.20z)| lr 3.22e-04 | 322.94 ms | 52.3% bf16 MFU | 1623686 tok/s
step 9692/19560 | loss 3.409013 (-0.22z)| norm 0.2670 (-0.58z)| lr 3.22e-04 | 323.13 ms | 52.2% bf16 MFU | 1623627 tok/s
step 9693/19560 | loss 3.429181 (+0.20z)| norm 0.2715 (-0.33z)| lr 3.22e-04 | 322.90 ms | 52.3% bf16 MFU | 1623630 tok/s
step 9694/19560 | loss 3.402130 (-0.36z)| norm 0.2761 (-0.10z)| lr 3.22e-04 | 322.54 ms | 52.3% bf16 MFU | 1623724 tok/s
step 9695/19560 | loss 3.372706 (-0.97z)| norm 0.2732 (-0.26z)| lr 3.22e-04 | 323.41 ms | 52.2% bf16 MFU | 1623595 tok/s
step 9696/19560 | loss 3.409619 (-0.19z)| norm 0.2725 (-0.31z)| lr 3.22e-04 | 322.92 ms | 52.3% bf16 MFU | 1623594 tok/s
step 9697/19560 | loss 3.382456 (-0.75z)| norm 0.2696 (-0.47z)| lr 3.22e-04 | 323.66 ms | 52.1% bf16 MFU | 1623407 tok/s
step 9698/19560 | loss 3.392912 (-0.52z)| norm 0.2684 (-0.55z)| lr 3.22e-04 | 322.42 ms | 52.3% bf16 MFU | 1623542 tok/s
step 9699/19560 | loss 3.379471 (-0.80z)| norm 0.2678 (-0.58z)| lr 3.22e-04 | 322.66 ms | 52.3% bf16 MFU | 1623610 tok/s
step 9700/19560 | loss 3.389682 (-0.59z)| norm 0.2626 (-0.86z)| lr 3.22e-04 | 322.71 ms | 52.3% bf16 MFU | 1623662 tok/s
step 9701/19560 | loss 3.406093 (-0.24z)| norm 0.2873 (+0.47z)| lr 3.21e-04 | 322.68 ms | 52.3% bf16 MFU | 1623718 tok/s
step 9702/19560 | loss 3.435103 (+0.38z)| norm 0.2516 (-1.44z)| lr 3.21e-04 | 322.90 ms | 52.3% bf16 MFU | 1623716 tok/s
step 9703/19560 | loss 3.409084 (-0.17z)| norm 0.2750 (-0.19z)| lr 3.21e-04 | 322.79 ms | 52.3% bf16 MFU | 1623742 tok/s
step 9704/19560 | loss 3.437378 (+0.43z)| norm 0.2608 (-0.95z)| lr 3.21e-04 | 322.16 ms | 52.4% bf16 MFU | 1623926 tok/s
step 9705/19560 | loss 3.447916 (+0.65z)| norm 0.2825 (+0.22z)| lr 3.21e-04 | 322.79 ms | 52.3% bf16 MFU | 1623941 tok/s
step 9706/19560 | loss 3.406907 (-0.23z)| norm 0.2653 (-0.71z)| lr 3.21e-04 | 323.43 ms | 52.2% bf16 MFU | 1623796 tok/s
step 9707/19560 | loss 3.376355 (-0.88z)| norm 0.2908 (+0.66z)| lr 3.21e-04 | 323.18 ms | 52.2% bf16 MFU | 1623720 tok/s
step 9708/19560 | loss 3.438224 (+0.44z)| norm 0.2794 (+0.06z)| lr 3.21e-04 | 322.48 ms | 52.3% bf16 MFU | 1623824 tok/s
step 9709/19560 | loss 3.457088 (+0.83z)| norm 0.3003 (+1.18z)| lr 3.21e-04 | 322.67 ms | 52.3% bf16 MFU | 1623876 tok/s
step 9710/19560 | loss 3.429690 (+0.25z)| norm 0.2498 (-1.56z)| lr 3.21e-04 | 323.05 ms | 52.2% bf16 MFU | 1623828 tok/s
step 9711/19560 | loss 3.380527 (-0.80z)| norm 0.2702 (-0.44z)| lr 3.21e-04 | 323.07 ms | 52.2% bf16 MFU | 1623779 tok/s
step 9712/19560 | loss 3.469054 (+1.08z)| norm 0.2650 (-0.72z)| lr 3.21e-04 | 322.70 ms | 52.3% bf16 MFU | 1623824 tok/s
step 9713/19560 | loss 3.597787 (+3.58z)| norm 0.2897 (+0.61z)| lr 3.21e-04 | 322.43 ms | 52.3% bf16 MFU | 1623935 tok/s
step 9714/19560 | loss 3.447893 (+0.56z)| norm 0.2594 (-1.02z)| lr 3.21e-04 | 322.60 ms | 52.3% bf16 MFU | 1623999 tok/s
step 9715/19560 | loss 3.379702 (-0.81z)| norm 0.2988 (+1.10z)| lr 3.21e-04 | 322.44 ms | 52.3% bf16 MFU | 1624098 tok/s
step 9716/19560 | loss 3.459443 (+0.78z)| norm 0.2882 (+0.51z)| lr 3.21e-04 | 322.86 ms | 52.3% bf16 MFU | 1624088 tok/s
step 9717/19560 | loss 3.397753 (-0.45z)| norm 0.2888 (+0.55z)| lr 3.21e-04 | 322.42 ms | 52.3% bf16 MFU | 1624189 tok/s
step 9718/19560 | loss 3.401414 (-0.38z)| norm 0.2677 (-0.59z)| lr 3.21e-04 | 323.04 ms | 52.2% bf16 MFU | 1624128 tok/s
step 9719/19560 | loss 3.440849 (+0.41z)| norm 0.2937 (+0.81z)| lr 3.21e-04 | 322.03 ms | 52.4% bf16 MFU | 1624326 tok/s
step 9720/19560 | loss 3.421198 (+0.02z)| norm 0.2625 (-0.86z)| lr 3.21e-04 | 322.46 ms | 52.3% bf16 MFU | 1624406 tok/s
step 9721/19560 | loss 3.382502 (-0.75z)| norm 0.3047 (+1.38z)| lr 3.20e-04 | 323.09 ms | 52.2% bf16 MFU | 1624323 tok/s
step 9722/19560 | loss 3.443902 (+0.48z)| norm 0.2632 (-0.83z)| lr 3.20e-04 | 322.51 ms | 52.3% bf16 MFU | 1624389 tok/s
step 9723/19560 | loss 3.404350 (-0.32z)| norm 0.2906 (+0.62z)| lr 3.20e-04 | 323.13 ms | 52.2% bf16 MFU | 1624297 tok/s
step 9724/19560 | loss 3.427516 (+0.15z)| norm 0.2628 (-0.85z)| lr 3.20e-04 | 322.95 ms | 52.3% bf16 MFU | 1624255 tok/s
step 9725/19560 | loss 3.411612 (-0.19z)| norm 0.2753 (-0.17z)| lr 3.20e-04 | 322.60 ms | 52.3% bf16 MFU | 1624302 tok/s
step 9726/19560 | loss 3.386694 (-0.69z)| norm 0.2727 (-0.31z)| lr 3.20e-04 | 322.81 ms | 52.3% bf16 MFU | 1624293 tok/s
step 9727/19560 | loss 3.452976 (+0.65z)| norm 0.2725 (-0.32z)| lr 3.20e-04 | 323.27 ms | 52.2% bf16 MFU | 1624168 tok/s
step 9728/19560 | loss 3.366651 (-1.10z)| norm 0.2801 (+0.08z)| lr 3.20e-04 | 323.17 ms | 52.2% bf16 MFU | 1624076 tok/s
step 9729/19560 | loss 3.452117 (+0.67z)| norm 0.2694 (-0.48z)| lr 3.20e-04 | 322.73 ms | 52.3% bf16 MFU | 1624098 tok/s
step 9730/19560 | loss 3.385055 (-0.72z)| norm 0.2822 (+0.21z)| lr 3.20e-04 | 322.73 ms | 52.3% bf16 MFU | 1624119 tok/s
step 9731/19560 | loss 3.397027 (-0.47z)| norm 0.3044 (+1.38z)| lr 3.20e-04 | 322.76 ms | 52.3% bf16 MFU | 1624133 tok/s
step 9732/19560 | loss 3.437696 (+0.36z)| norm 0.2668 (-0.62z)| lr 3.20e-04 | 322.82 ms | 52.3% bf16 MFU | 1624131 tok/s
step 9733/19560 | loss 3.407078 (-0.29z)| norm 0.2964 (+0.97z)| lr 3.20e-04 | 323.34 ms | 52.2% bf16 MFU | 1624000 tok/s
step 9734/19560 | loss 3.429811 (+0.22z)| norm 0.2631 (-0.81z)| lr 3.20e-04 | 322.17 ms | 52.4% bf16 MFU | 1624168 tok/s
step 9735/19560 | loss 3.392042 (-0.60z)| norm 0.2616 (-0.88z)| lr 3.20e-04 | 322.30 ms | 52.4% bf16 MFU | 1624296 tok/s
step 9736/19560 | loss 3.362450 (-1.23z)| norm 0.2873 (+0.51z)| lr 3.20e-04 | 322.84 ms | 52.3% bf16 MFU | 1624279 tok/s
step 9737/19560 | loss 3.436856 (+0.38z)| norm 0.2639 (-0.75z)| lr 3.20e-04 | 322.43 ms | 52.3% bf16 MFU | 1624369 tok/s
step 9738/19560 | loss 3.478938 (+1.30z)| norm 0.2924 (+0.80z)| lr 3.20e-04 | 322.76 ms | 52.3% bf16 MFU | 1624369 tok/s
step 9739/19560 | loss 3.462500 (+0.95z)| norm 0.2893 (+0.62z)| lr 3.20e-04 | 322.72 ms | 52.3% bf16 MFU | 1624380 tok/s
step 9740/19560 | loss 3.457505 (+0.83z)| norm 0.2816 (+0.22z)| lr 3.20e-04 | 322.91 ms | 52.3% bf16 MFU | 1624343 tok/s
step 9741/19560 | loss 3.374072 (-0.98z)| norm 0.2576 (-1.07z)| lr 3.19e-04 | 322.84 ms | 52.3% bf16 MFU | 1624325 tok/s
step 9742/19560 | loss 3.470052 (+1.09z)| norm 0.2770 (-0.01z)| lr 3.19e-04 | 322.27 ms | 52.4% bf16 MFU | 1624452 tok/s
step 9743/19560 | loss 3.423038 (+0.08z)| norm 0.3106 (+1.77z)| lr 3.19e-04 | 322.30 ms | 52.4% bf16 MFU | 1624565 tok/s
step 9744/19560 | loss 3.457068 (+0.80z)| norm 0.2994 (+1.16z)| lr 3.19e-04 | 322.98 ms | 52.3% bf16 MFU | 1624502 tok/s
step 9745/19560 | loss 3.375030 (-0.96z)| norm 0.2679 (-0.53z)| lr 3.19e-04 | 322.80 ms | 52.3% bf16 MFU | 1624486 tok/s
step 9746/19560 | loss 3.417289 (-0.05z)| norm 0.2807 (+0.14z)| lr 3.19e-04 | 322.73 ms | 52.3% bf16 MFU | 1624490 tok/s
step 9747/19560 | loss 3.440297 (+0.44z)| norm 0.3131 (+1.86z)| lr 3.19e-04 | 322.32 ms | 52.4% bf16 MFU | 1624597 tok/s
step 9748/19560 | loss 3.471790 (+1.10z)| norm 0.3091 (+1.62z)| lr 3.19e-04 | 322.56 ms | 52.3% bf16 MFU | 1624638 tok/s
step 9749/19560 | loss 3.403504 (-0.35z)| norm 0.2839 (+0.28z)| lr 3.19e-04 | 322.62 ms | 52.3% bf16 MFU | 1624662 tok/s
step 9750/19560 | loss 3.408429 (-0.25z)| norm 0.3074 (+1.50z)| lr 3.19e-04 | 322.43 ms | 52.3% bf16 MFU | 1624731 tok/s
val loss 3.402709
ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2888/10042 = 0.287592
step 9751/19560 | loss 3.484550 (+1.39z)| norm 0.3038 (+1.29z)| lr 3.19e-04 | 322.95 ms | 52.3% bf16 MFU | 1624667 tok/s
step 9752/19560 | loss 3.447915 (+0.59z)| norm 0.3209 (+2.14z)| lr 3.19e-04 | 322.43 ms | 52.3% bf16 MFU | 1624737 tok/s
step 9753/19560 | loss 3.419355 (+0.01z)| norm 0.3153 (+1.81z)| lr 3.19e-04 | 322.50 ms | 52.3% bf16 MFU | 1624785 tok/s
step 9754/19560 | loss 3.411340 (-0.20z)| norm 0.3134 (+1.70z)| lr 3.19e-04 | 322.40 ms | 52.3% bf16 MFU | 1624857 tok/s
step 9755/19560 | loss 3.394213 (-0.60z)| norm 0.2845 (+0.23z)| lr 3.19e-04 | 322.88 ms | 52.3% bf16 MFU | 1624803 tok/s
step 9756/19560 | loss 3.407912 (-0.27z)| norm 0.2915 (+0.58z)| lr 3.19e-04 | 322.87 ms | 52.3% bf16 MFU | 1624756 tok/s
step 9757/19560 | loss 3.397938 (-0.51z)| norm 0.2873 (+0.37z)| lr 3.19e-04 | 323.33 ms | 52.2% bf16 MFU | 1624594 tok/s
step 9758/19560 | loss 3.341270 (-1.84z)| norm 0.2647 (-0.77z)| lr 3.19e-04 | 322.67 ms | 52.3% bf16 MFU | 1624607 tok/s
step 9759/19560 | loss 3.457714 (+0.92z)| norm 0.2775 (-0.11z)| lr 3.19e-04 | 322.24 ms | 52.4% bf16 MFU | 1624729 tok/s
step 9760/19560 | loss 3.384669 (-0.80z)| norm 0.2673 (-0.62z)| lr 3.19e-04 | 322.70 ms | 52.3% bf16 MFU | 1624728 tok/s
step 9761/19560 | loss 3.414160 (-0.11z)| norm 0.2800 (+0.02z)| lr 3.18e-04 | 322.72 ms | 52.3% bf16 MFU | 1624721 tok/s
step 9762/19560 | loss 3.408455 (-0.25z)| norm 0.2669 (-0.63z)| lr 3.18e-04 | 322.75 ms | 52.3% bf16 MFU | 1624707 tok/s
step 9763/19560 | loss 3.404265 (-0.35z)| norm 0.2663 (-0.66z)| lr 3.18e-04 | 322.50 ms | 52.3% bf16 MFU | 1624756 tok/s
step 9764/19560 | loss 3.420811 (+0.03z)| norm 0.2882 (+0.54z)| lr 3.18e-04 | 322.11 ms | 52.4% bf16 MFU | 1624901 tok/s
step 9765/19560 | loss 3.420801 (+0.05z)| norm 0.2962 (+0.99z)| lr 3.18e-04 | 322.54 ms | 52.3% bf16 MFU | 1624932 tok/s
step 9766/19560 | loss 3.420154 (+0.04z)| norm 0.2722 (-0.34z)| lr 3.18e-04 | 322.68 ms | 52.3% bf16 MFU | 1624924 tok/s
step 9767/19560 | loss 3.353555 (-1.57z)| norm 0.2751 (-0.17z)| lr 3.18e-04 | 322.54 ms | 52.3% bf16 MFU | 1624953 tok/s
step 9768/19560 | loss 3.504362 (+2.05z)| norm 0.2488 (-1.68z)| lr 3.18e-04 | 322.52 ms | 52.3% bf16 MFU | 1624986 tok/s
step 9769/19560 | loss 3.431028 (+0.30z)| norm 0.2572 (-1.17z)| lr 3.18e-04 | 322.75 ms | 52.3% bf16 MFU | 1624958 tok/s
step 9770/19560 | loss 3.450794 (+0.77z)| norm 0.2733 (-0.23z)| lr 3.18e-04 | 323.03 ms | 52.2% bf16 MFU | 1624862 tok/s
step 9771/19560 | loss 3.409138 (-0.22z)| norm 0.2550 (-1.28z)| lr 3.18e-04 | 322.23 ms | 52.4% bf16 MFU | 1624973 tok/s
step 9772/19560 | loss 3.530210 (+2.64z)| norm 0.2960 (+1.08z)| lr 3.18e-04 | 322.67 ms | 52.3% bf16 MFU | 1624965 tok/s
step 9773/19560 | loss 3.396685 (-0.53z)| norm 0.2780 (+0.02z)| lr 3.18e-04 | 323.15 ms | 52.2% bf16 MFU | 1624838 tok/s
step 9774/19560 | loss 3.445486 (+0.62z)| norm 0.2920 (+0.83z)| lr 3.18e-04 | 322.93 ms | 52.3% bf16 MFU | 1624774 tok/s
step 9775/19560 | loss 3.424430 (+0.11z)| norm 0.2929 (+0.87z)| lr 3.18e-04 | 322.68 ms | 52.3% bf16 MFU | 1624775 tok/s
step 9776/19560 | loss 3.428991 (+0.22z)| norm 0.2583 (-1.16z)| lr 3.18e-04 | 322.71 ms | 52.3% bf16 MFU | 1624769 tok/s
step 9777/19560 | loss 3.465631 (+1.09z)| norm 0.2702 (-0.47z)| lr 3.18e-04 | 322.73 ms | 52.3% bf16 MFU | 1624759 tok/s
step 9778/19560 | loss 3.369946 (-1.20z)| norm 0.2471 (-1.80z)| lr 3.18e-04 | 322.18 ms | 52.4% bf16 MFU | 1624887 tok/s
step 9779/19560 | loss 3.480680 (+1.42z)| norm 0.2638 (-0.82z)| lr 3.18e-04 | 322.91 ms | 52.3% bf16 MFU | 1624825 tok/s
step 9780/19560 | loss 3.446385 (+0.60z)| norm 0.2617 (-0.94z)| lr 3.18e-04 | 322.55 ms | 52.3% bf16 MFU | 1624857 tok/s
step 9781/19560 | loss 3.441490 (+0.48z)| norm 0.2608 (-0.99z)| lr 3.17e-04 | 322.07 ms | 52.4% bf16 MFU | 1625007 tok/s
step 9782/19560 | loss 3.421106 (-0.01z)| norm 0.2614 (-0.96z)| lr 3.17e-04 | 322.74 ms | 52.3% bf16 MFU | 1624981 tok/s
step 9783/19560 | loss 3.507353 (+2.00z)| norm 0.2726 (-0.31z)| lr 3.17e-04 | 322.58 ms | 52.3% bf16 MFU | 1624995 tok/s
step 9784/19560 | loss 3.364250 (-1.36z)| norm 0.2653 (-0.73z)| lr 3.17e-04 | 322.93 ms | 52.3% bf16 MFU | 1624921 tok/s
step 9785/19560 | loss 3.327962 (-2.16z)| norm 0.2644 (-0.77z)| lr 3.17e-04 | 322.63 ms | 52.3% bf16 MFU | 1624926 tok/s
step 9786/19560 | loss 3.459730 (+0.90z)| norm 0.2409 (-2.11z)| lr 3.17e-04 | 322.33 ms | 52.4% bf16 MFU | 1625008 tok/s
step 9787/19560 | loss 3.374418 (-1.07z)| norm 0.2839 (+0.41z)| lr 3.17e-04 | 322.45 ms | 52.3% bf16 MFU | 1625056 tok/s
step 9788/19560 | loss 3.355455 (-1.49z)| norm 0.2664 (-0.60z)| lr 3.17e-04 | 322.69 ms | 52.3% bf16 MFU | 1625041 tok/s
step 9789/19560 | loss 3.385365 (-0.79z)| norm 0.2596 (-1.01z)| lr 3.17e-04 | 322.76 ms | 52.3% bf16 MFU | 1625009 tok/s
step 9790/19560 | loss 3.398296 (-0.49z)| norm 0.2797 (+0.19z)| lr 3.17e-04 | 322.41 ms | 52.3% bf16 MFU | 1625066 tok/s
step 9791/19560 | loss 3.419753 (+0.00z)| norm 0.2457 (-1.82z)| lr 3.17e-04 | 322.08 ms | 52.4% bf16 MFU | 1625204 tok/s
step 9792/19560 | loss 3.447514 (+0.69z)| norm 0.2841 (+0.45z)| lr 3.17e-04 | 323.17 ms | 52.2% bf16 MFU | 1625059 tok/s
step 9793/19560 | loss 3.483580 (+1.55z)| norm 0.2854 (+0.52z)| lr 3.17e-04 | 322.75 ms | 52.3% bf16 MFU | 1625028 tok/s
step 9794/19560 | loss 3.324063 (-2.23z)| norm 0.2568 (-1.15z)| lr 3.17e-04 | 322.36 ms | 52.4% bf16 MFU | 1625097 tok/s
step 9795/19560 | loss 3.409392 (-0.21z)| norm 0.2708 (-0.32z)| lr 3.17e-04 | 322.81 ms | 52.3% bf16 MFU | 1625048 tok/s
step 9796/19560 | loss 3.352537 (-1.55z)| norm 0.2516 (-1.45z)| lr 3.17e-04 | 322.81 ms | 52.3% bf16 MFU | 1625004 tok/s
step 9797/19560 | loss 3.460135 (+0.98z)| norm 0.2874 (+0.69z)| lr 3.17e-04 | 322.95 ms | 52.3% bf16 MFU | 1624925 tok/s
step 9798/19560 | loss 3.373882 (-1.05z)| norm 0.2722 (-0.21z)| lr 3.17e-04 | 322.40 ms | 52.3% bf16 MFU | 1624990 tok/s
step 9799/19560 | loss 3.382027 (-0.86z)| norm 0.2876 (+0.70z)| lr 3.17e-04 | 322.85 ms | 52.3% bf16 MFU | 1624937 tok/s
step 9800/19560 | loss 3.345006 (-1.72z)| norm 0.2690 (-0.41z)| lr 3.17e-04 | 322.59 ms | 52.3% bf16 MFU | 1624952 tok/s
step 9801/19560 | loss 3.398746 (-0.43z)| norm 0.2673 (-0.51z)| lr 3.16e-04 | 322.46 ms | 52.3% bf16 MFU | 1625000 tok/s
step 9802/19560 | loss 3.458173 (+1.01z)| norm 0.2531 (-1.36z)| lr 3.16e-04 | 322.39 ms | 52.3% bf16 MFU | 1625061 tok/s
step 9803/19560 | loss 3.329202 (-2.06z)| norm 0.3017 (+1.54z)| lr 3.16e-04 | 322.47 ms | 52.3% bf16 MFU | 1625100 tok/s
step 9804/19560 | loss 3.350571 (-1.52z)| norm 0.2669 (-0.54z)| lr 3.16e-04 | 322.61 ms | 52.3% bf16 MFU | 1625103 tok/s
step 9805/19560 | loss 3.352553 (-1.47z)| norm 0.2939 (+1.06z)| lr 3.16e-04 | 322.72 ms | 52.3% bf16 MFU | 1625078 tok/s
step 9806/19560 | loss 3.397637 (-0.41z)| norm 0.3597 (+4.53z)| lr 3.16e-04 | 322.71 ms | 52.3% bf16 MFU | 1625055 tok/s
step 9807/19560 | loss 3.412928 (-0.06z)| norm 0.2681 (-0.49z)| lr 3.16e-04 | 322.82 ms | 52.3% bf16 MFU | 1625007 tok/s
step 9808/19560 | loss 3.397870 (-0.41z)| norm 0.2621 (-0.83z)| lr 3.16e-04 | 322.59 ms | 52.3% bf16 MFU | 1625020 tok/s
step 9809/19560 | loss 3.345418 (-1.61z)| norm 0.2963 (+1.05z)| lr 3.16e-04 | 322.55 ms | 52.3% bf16 MFU | 1625041 tok/s
step 9810/19560 | loss 3.446320 (+0.73z)| norm 0.2492 (-1.55z)| lr 3.16e-04 | 322.44 ms | 52.3% bf16 MFU | 1625090 tok/s
step 9811/19560 | loss 3.423201 (+0.19z)| norm 0.2731 (-0.23z)| lr 3.16e-04 | 322.57 ms | 52.3% bf16 MFU | 1625103 tok/s
step 9812/19560 | loss 3.404634 (-0.25z)| norm 0.2710 (-0.35z)| lr 3.16e-04 | 322.69 ms | 52.3% bf16 MFU | 1625086 tok/s
step 9813/19560 | loss 3.366769 (-1.12z)| norm 0.2806 (+0.18z)| lr 3.16e-04 | 322.16 ms | 52.4% bf16 MFU | 1625202 tok/s
step 9814/19560 | loss 3.413543 (-0.02z)| norm 0.2824 (+0.29z)| lr 3.16e-04 | 323.07 ms | 52.2% bf16 MFU | 1625085 tok/s
step 9815/19560 | loss 3.495379 (+1.84z)| norm 0.2867 (+0.52z)| lr 3.16e-04 | 322.83 ms | 52.3% bf16 MFU | 1625032 tok/s
step 9816/19560 | loss 3.359944 (-1.27z)| norm 0.2649 (-0.70z)| lr 3.16e-04 | 322.75 ms | 52.3% bf16 MFU | 1625002 tok/s
step 9817/19560 | loss 3.432825 (+0.41z)| norm 0.2738 (-0.20z)| lr 3.16e-04 | 322.54 ms | 52.3% bf16 MFU | 1625028 tok/s
step 9818/19560 | loss 3.433241 (+0.41z)| norm 0.2657 (-0.64z)| lr 3.16e-04 | 322.56 ms | 52.3% bf16 MFU | 1625045 tok/s
step 9819/19560 | loss 3.345512 (-1.62z)| norm 0.2729 (-0.23z)| lr 3.16e-04 | 322.26 ms | 52.4% bf16 MFU | 1625138 tok/s
step 9820/19560 | loss 3.421173 (+0.13z)| norm 0.2829 (+0.32z)| lr 3.16e-04 | 322.89 ms | 52.3% bf16 MFU | 1625069 tok/s
step 9821/19560 | loss 3.380313 (-0.81z)| norm 0.2596 (-0.98z)| lr 3.15e-04 | 322.41 ms | 52.3% bf16 MFU | 1625124 tok/s
step 9822/19560 | loss 3.376333 (-0.89z)| norm 0.3140 (+2.01z)| lr 3.15e-04 | 323.70 ms | 52.1% bf16 MFU | 1624851 tok/s
step 9823/19560 | loss 3.414340 (-0.02z)| norm 0.2833 (+0.32z)| lr 3.15e-04 | 322.77 ms | 52.3% bf16 MFU | 1624826 tok/s
step 9824/19560 | loss 3.360106 (-1.26z)| norm 0.2739 (-0.20z)| lr 3.15e-04 | 322.70 ms | 52.3% bf16 MFU | 1624819 tok/s
step 9825/19560 | loss 3.405941 (-0.21z)| norm 0.2893 (+0.64z)| lr 3.15e-04 | 322.75 ms | 52.3% bf16 MFU | 1624801 tok/s
step 9826/19560 | loss 3.448039 (+0.74z)| norm 0.2886 (+0.59z)| lr 3.15e-04 | 322.63 ms | 52.3% bf16 MFU | 1624812 tok/s
step 9827/19560 | loss 3.367552 (-1.10z)| norm 0.2858 (+0.43z)| lr 3.15e-04 | 322.68 ms | 52.3% bf16 MFU | 1624811 tok/s
step 9828/19560 | loss 3.430592 (+0.34z)| norm 0.2803 (+0.12z)| lr 3.15e-04 | 322.47 ms | 52.3% bf16 MFU | 1624864 tok/s
step 9829/19560 | loss 3.426294 (+0.24z)| norm 0.2742 (-0.21z)| lr 3.15e-04 | 323.00 ms | 52.3% bf16 MFU | 1624779 tok/s
step 9830/19560 | loss 3.473379 (+1.30z)| norm 0.3309 (+2.81z)| lr 3.15e-04 | 322.27 ms | 52.4% bf16 MFU | 1624882 tok/s
step 9831/19560 | loss 3.409755 (-0.15z)| norm 0.3597 (+4.04z)| lr 3.15e-04 | 323.02 ms | 52.2% bf16 MFU | 1624793 tok/s
step 9832/19560 | loss 3.417242 (+0.02z)| norm 0.3734 (+4.33z)| lr 3.15e-04 | 322.50 ms | 52.3% bf16 MFU | 1624837 tok/s
step 9833/19560 | loss 3.373582 (-0.96z)| norm 0.3193 (+1.78z)| lr 3.15e-04 | 323.24 ms | 52.2% bf16 MFU | 1624695 tok/s
step 9834/19560 | loss 3.379063 (-0.83z)| norm 0.2822 (+0.08z)| lr 3.15e-04 | 322.62 ms | 52.3% bf16 MFU | 1624715 tok/s
step 9835/19560 | loss 3.409308 (-0.14z)| norm 0.3126 (+1.45z)| lr 3.15e-04 | 323.14 ms | 52.2% bf16 MFU | 1624604 tok/s
step 9836/19560 | loss 3.382964 (-0.73z)| norm 0.2880 (+0.33z)| lr 3.15e-04 | 323.17 ms | 52.2% bf16 MFU | 1624490 tok/s
step 9837/19560 | loss 3.366533 (-1.09z)| norm 0.2927 (+0.55z)| lr 3.15e-04 | 322.54 ms | 52.3% bf16 MFU | 1624542 tok/s
step 9838/19560 | loss 3.385661 (-0.65z)| norm 0.2881 (+0.33z)| lr 3.15e-04 | 322.77 ms | 52.3% bf16 MFU | 1624531 tok/s
step 9839/19560 | loss 3.476488 (+1.39z)| norm 0.3076 (+1.20z)| lr 3.15e-04 | 322.69 ms | 52.3% bf16 MFU | 1624543 tok/s
step 9840/19560 | loss 3.537253 (+2.69z)| norm 0.2756 (-0.27z)| lr 3.15e-04 | 323.03 ms | 52.2% bf16 MFU | 1624468 tok/s
step 9841/19560 | loss 3.398174 (-0.37z)| norm 0.2958 (+0.66z)| lr 3.14e-04 | 323.44 ms | 52.2% bf16 MFU | 1624294 tok/s
step 9842/19560 | loss 3.437152 (+0.55z)| norm 0.2833 (+0.08z)| lr 3.14e-04 | 322.82 ms | 52.3% bf16 MFU | 1624284 tok/s
step 9843/19560 | loss 3.375061 (-0.92z)| norm 0.2824 (+0.04z)| lr 3.14e-04 | 322.37 ms | 52.4% bf16 MFU | 1624388 tok/s
step 9844/19560 | loss 3.430334 (+0.40z)| norm 0.2814 (-0.00z)| lr 3.14e-04 | 322.91 ms | 52.3% bf16 MFU | 1624350 tok/s
step 9845/19560 | loss 3.359562 (-1.27z)| norm 0.2661 (-0.70z)| lr 3.14e-04 | 322.98 ms | 52.3% bf16 MFU | 1624298 tok/s
step 9846/19560 | loss 3.381593 (-0.74z)| norm 0.2854 (+0.19z)| lr 3.14e-04 | 322.44 ms | 52.3% bf16 MFU | 1624384 tok/s
step 9847/19560 | loss 3.400727 (-0.28z)| norm 0.2915 (+0.47z)| lr 3.14e-04 | 322.73 ms | 52.3% bf16 MFU | 1624392 tok/s
step 9848/19560 | loss 3.523227 (+2.52z)| norm 0.3050 (+1.07z)| lr 3.14e-04 | 322.76 ms | 52.3% bf16 MFU | 1624392 tok/s
step 9849/19560 | loss 3.383839 (-0.68z)| norm 0.3067 (+1.15z)| lr 3.14e-04 | 322.98 ms | 52.3% bf16 MFU | 1624336 tok/s
step 9850/19560 | loss 3.387843 (-0.58z)| norm 0.2874 (+0.25z)| lr 3.14e-04 | 322.81 ms | 52.3% bf16 MFU | 1624325 tok/s
step 9851/19560 | loss 3.470280 (+1.30z)| norm 0.2920 (+0.47z)| lr 3.14e-04 | 323.40 ms | 52.2% bf16 MFU | 1624167 tok/s
step 9852/19560 | loss 3.400861 (-0.29z)| norm 0.2889 (+0.31z)| lr 3.14e-04 | 322.53 ms | 52.3% bf16 MFU | 1624236 tok/s
step 9853/19560 | loss 3.404794 (-0.20z)| norm 0.2792 (-0.14z)| lr 3.14e-04 | 322.80 ms | 52.3% bf16 MFU | 1624233 tok/s
step 9854/19560 | loss 3.419583 (+0.14z)| norm 0.2666 (-0.72z)| lr 3.14e-04 | 322.79 ms | 52.3% bf16 MFU | 1624233 tok/s
step 9855/19560 | loss 3.490555 (+1.74z)| norm 0.2751 (-0.32z)| lr 3.14e-04 | 322.65 ms | 52.3% bf16 MFU | 1624269 tok/s
step 9856/19560 | loss 3.356803 (-1.29z)| norm 0.2637 (-0.84z)| lr 3.14e-04 | 322.67 ms | 52.3% bf16 MFU | 1624297 tok/s
step 9857/19560 | loss 3.428078 (+0.33z)| norm 0.2717 (-0.48z)| lr 3.14e-04 | 322.88 ms | 52.3% bf16 MFU | 1624272 tok/s
step 9858/19560 | loss 3.387767 (-0.59z)| norm 0.2711 (-0.50z)| lr 3.14e-04 | 322.79 ms | 52.3% bf16 MFU | 1624269 tok/s
step 9859/19560 | loss 3.425839 (+0.27z)| norm 0.2758 (-0.27z)| lr 3.14e-04 | 322.35 ms | 52.4% bf16 MFU | 1624379 tok/s
step 9860/19560 | loss 3.419418 (+0.13z)| norm 0.2961 (+0.65z)| lr 3.14e-04 | 323.63 ms | 52.2% bf16 MFU | 1624163 tok/s
step 9861/19560 | loss 3.358366 (-1.24z)| norm 0.2915 (+0.44z)| lr 3.13e-04 | 322.62 ms | 52.3% bf16 MFU | 1624208 tok/s
step 9862/19560 | loss 3.374186 (-0.87z)| norm 0.2866 (+0.21z)| lr 3.13e-04 | 322.49 ms | 52.3% bf16 MFU | 1624286 tok/s
step 9863/19560 | loss 3.423794 (+0.24z)| norm 0.3176 (+1.62z)| lr 3.13e-04 | 323.11 ms | 52.2% bf16 MFU | 1624203 tok/s
step 9864/19560 | loss 3.425268 (+0.26z)| norm 0.3007 (+0.83z)| lr 3.13e-04 | 322.83 ms | 52.3% bf16 MFU | 1624196 tok/s
step 9865/19560 | loss 3.444249 (+0.69z)| norm 0.2981 (+0.70z)| lr 3.13e-04 | 322.59 ms | 52.3% bf16 MFU | 1624248 tok/s
step 9866/19560 | loss 3.360647 (-1.19z)| norm 0.3222 (+1.78z)| lr 3.13e-04 | 323.07 ms | 52.2% bf16 MFU | 1624176 tok/s
step 9867/19560 | loss 3.356771 (-1.25z)| norm 0.2841 (+0.04z)| lr 3.13e-04 | 322.58 ms | 52.3% bf16 MFU | 1624233 tok/s
step 9868/19560 | loss 3.410315 (-0.03z)| norm 0.3019 (+0.85z)| lr 3.13e-04 | 322.76 ms | 52.3% bf16 MFU | 1624240 tok/s
step 9869/19560 | loss 3.424704 (+0.29z)| norm 0.2682 (-0.69z)| lr 3.13e-04 | 323.66 ms | 52.1% bf16 MFU | 1624023 tok/s
step 9870/19560 | loss 3.363992 (-1.08z)| norm 0.3106 (+1.22z)| lr 3.13e-04 | 322.57 ms | 52.3% bf16 MFU | 1624090 tok/s
step 9871/19560 | loss 3.420181 (+0.21z)| norm 0.3168 (+1.50z)| lr 3.13e-04 | 322.75 ms | 52.3% bf16 MFU | 1624108 tok/s
step 9872/19560 | loss 3.437461 (+0.61z)| norm 0.2778 (-0.26z)| lr 3.13e-04 | 322.79 ms | 52.3% bf16 MFU | 1624116 tok/s
step 9873/19560 | loss 3.414689 (+0.08z)| norm 0.3314 (+2.11z)| lr 3.13e-04 | 322.60 ms | 52.3% bf16 MFU | 1624169 tok/s
step 9874/19560 | loss 3.457574 (+1.05z)| norm 0.2795 (-0.20z)| lr 3.13e-04 | 323.06 ms | 52.2% bf16 MFU | 1624104 tok/s
step 9875/19560 | loss 3.406516 (-0.11z)| norm 0.3188 (+1.54z)| lr 3.13e-04 | 322.58 ms | 52.3% bf16 MFU | 1624163 tok/s
step 9876/19560 | loss 3.446620 (+0.82z)| norm 0.2873 (+0.15z)| lr 3.13e-04 | 322.65 ms | 52.3% bf16 MFU | 1624202 tok/s
step 9877/19560 | loss 3.464963 (+1.22z)| norm 0.2875 (+0.16z)| lr 3.13e-04 | 323.26 ms | 52.2% bf16 MFU | 1624086 tok/s
step 9878/19560 | loss 3.459225 (+1.08z)| norm 0.2959 (+0.54z)| lr 3.13e-04 | 322.34 ms | 52.4% bf16 MFU | 1624207 tok/s
step 9879/19560 | loss 3.366185 (-1.03z)| norm 0.2875 (+0.17z)| lr 3.13e-04 | 322.88 ms | 52.3% bf16 MFU | 1624186 tok/s
step 9880/19560 | loss 3.466543 (+1.27z)| norm 0.3043 (+0.94z)| lr 3.13e-04 | 323.06 ms | 52.2% bf16 MFU | 1624121 tok/s
step 9881/19560 | loss 3.367527 (-0.99z)| norm 0.2625 (-0.94z)| lr 3.12e-04 | 322.81 ms | 52.3% bf16 MFU | 1624122 tok/s
step 9882/19560 | loss 3.398580 (-0.28z)| norm 0.2801 (-0.12z)| lr 3.12e-04 | 323.43 ms | 52.2% bf16 MFU | 1623968 tok/s
step 9883/19560 | loss 3.461655 (+1.15z)| norm 0.3160 (+1.49z)| lr 3.12e-04 | 322.58 ms | 52.3% bf16 MFU | 1624035 tok/s
step 9884/19560 | loss 3.332651 (-1.75z)| norm 0.2793 (-0.17z)| lr 3.12e-04 | 322.72 ms | 52.3% bf16 MFU | 1624062 tok/s
step 9885/19560 | loss 3.410095 (-0.02z)| norm 0.2909 (+0.36z)| lr 3.12e-04 | 322.98 ms | 52.3% bf16 MFU | 1624022 tok/s
step 9886/19560 | loss 3.347725 (-1.42z)| norm 0.2864 (+0.15z)| lr 3.12e-04 | 322.91 ms | 52.3% bf16 MFU | 1624003 tok/s
step 9887/19560 | loss 3.371966 (-0.86z)| norm 0.2729 (-0.47z)| lr 3.12e-04 | 323.60 ms | 52.2% bf16 MFU | 1623811 tok/s
step 9888/19560 | loss 3.391099 (-0.43z)| norm 0.2780 (-0.24z)| lr 3.12e-04 | 323.11 ms | 52.2% bf16 MFU | 1623753 tok/s
step 9889/19560 | loss 3.347435 (-1.39z)| norm 0.2849 (+0.07z)| lr 3.12e-04 | 323.30 ms | 52.2% bf16 MFU | 1623649 tok/s
step 9890/19560 | loss 3.417593 (+0.17z)| norm 0.3067 (+1.05z)| lr 3.12e-04 | 322.63 ms | 52.3% bf16 MFU | 1623718 tok/s
step 9891/19560 | loss 3.331573 (-1.72z)| norm 0.3136 (+1.34z)| lr 3.12e-04 | 322.79 ms | 52.3% bf16 MFU | 1623743 tok/s
step 9892/19560 | loss 3.397535 (-0.25z)| norm 0.3167 (+1.46z)| lr 3.12e-04 | 323.14 ms | 52.2% bf16 MFU | 1623680 tok/s
step 9893/19560 | loss 3.411121 (+0.05z)| norm 0.3018 (+0.79z)| lr 3.12e-04 | 323.06 ms | 52.2% bf16 MFU | 1623640 tok/s
step 9894/19560 | loss 3.392609 (-0.36z)| norm 0.2778 (-0.29z)| lr 3.12e-04 | 322.79 ms | 52.3% bf16 MFU | 1623671 tok/s
step 9895/19560 | loss 3.437742 (+0.63z)| norm 0.2873 (+0.13z)| lr 3.12e-04 | 322.97 ms | 52.3% bf16 MFU | 1623655 tok/s
step 9896/19560 | loss 3.362039 (-1.04z)| norm 0.2651 (-0.88z)| lr 3.12e-04 | 322.55 ms | 52.3% bf16 MFU | 1623745 tok/s
step 9897/19560 | loss 3.358058 (-1.11z)| norm 0.2921 (+0.33z)| lr 3.12e-04 | 322.26 ms | 52.4% bf16 MFU | 1623904 tok/s
step 9898/19560 | loss 3.398170 (-0.21z)| norm 0.2618 (-1.04z)| lr 3.12e-04 | 322.95 ms | 52.3% bf16 MFU | 1623879 tok/s
step 9899/19560 | loss 3.467883 (+1.34z)| norm 0.2917 (+0.31z)| lr 3.12e-04 | 322.35 ms | 52.4% bf16 MFU | 1624007 tok/s
step 9900/19560 | loss 3.362881 (-1.00z)| norm 0.2575 (-1.23z)| lr 3.12e-04 | 322.95 ms | 52.3% bf16 MFU | 1623979 tok/s
step 9901/19560 | loss 3.380918 (-0.58z)| norm 0.3085 (+1.07z)| lr 3.11e-04 | 322.90 ms | 52.3% bf16 MFU | 1623964 tok/s
step 9902/19560 | loss 3.366138 (-0.91z)| norm 0.2588 (-1.16z)| lr 3.11e-04 | 322.24 ms | 52.4% bf16 MFU | 1624116 tok/s
step 9903/19560 | loss 3.408998 (+0.08z)| norm 0.2855 (+0.04z)| lr 3.11e-04 | 323.30 ms | 52.2% bf16 MFU | 1623994 tok/s
step 9904/19560 | loss 3.390144 (-0.35z)| norm 0.2631 (-0.97z)| lr 3.11e-04 | 322.24 ms | 52.4% bf16 MFU | 1624144 tok/s
step 9905/19560 | loss 3.416725 (+0.27z)| norm 0.2815 (-0.15z)| lr 3.11e-04 | 322.62 ms | 52.3% bf16 MFU | 1624192 tok/s
step 9906/19560 | loss 3.304705 (-2.26z)| norm 0.2643 (-0.93z)| lr 3.11e-04 | 323.17 ms | 52.2% bf16 MFU | 1624100 tok/s
step 9907/19560 | loss 3.365031 (-0.88z)| norm 0.2573 (-1.25z)| lr 3.11e-04 | 322.33 ms | 52.4% bf16 MFU | 1624222 tok/s
step 9908/19560 | loss 3.492005 (+2.00z)| norm 0.2702 (-0.67z)| lr 3.11e-04 | 322.21 ms | 52.4% bf16 MFU | 1624368 tok/s
step 9909/19560 | loss 3.434887 (+0.70z)| norm 0.2847 (-0.02z)| lr 3.11e-04 | 322.59 ms | 52.3% bf16 MFU | 1624410 tok/s
step 9910/19560 | loss 3.339010 (-1.44z)| norm 0.3032 (+0.82z)| lr 3.11e-04 | 323.20 ms | 52.2% bf16 MFU | 1624298 tok/s
step 9911/19560 | loss 3.387230 (-0.34z)| norm 0.2893 (+0.17z)| lr 3.11e-04 | 322.48 ms | 52.3% bf16 MFU | 1624374 tok/s
step 9912/19560 | loss 3.375338 (-0.62z)| norm 0.2883 (+0.12z)| lr 3.11e-04 | 322.40 ms | 52.3% bf16 MFU | 1624465 tok/s
step 9913/19560 | loss 3.398224 (-0.11z)| norm 0.2787 (-0.33z)| lr 3.11e-04 | 322.43 ms | 52.3% bf16 MFU | 1624544 tok/s
step 9914/19560 | loss 3.434472 (+0.74z)| norm 0.3026 (+0.76z)| lr 3.11e-04 | 323.15 ms | 52.2% bf16 MFU | 1624438 tok/s
step 9915/19560 | loss 3.429431 (+0.62z)| norm 0.3264 (+1.84z)| lr 3.11e-04 | 322.29 ms | 52.4% bf16 MFU | 1624554 tok/s
step 9916/19560 | loss 3.381327 (-0.52z)| norm 0.2973 (+0.49z)| lr 3.11e-04 | 322.19 ms | 52.4% bf16 MFU | 1624690 tok/s
step 9917/19560 | loss 3.404878 (+0.03z)| norm 0.3162 (+1.34z)| lr 3.11e-04 | 322.63 ms | 52.3% bf16 MFU | 1624707 tok/s
step 9918/19560 | loss 3.430924 (+0.64z)| norm 0.3068 (+0.89z)| lr 3.11e-04 | 322.70 ms | 52.3% bf16 MFU | 1624707 tok/s
step 9919/19560 | loss 3.493377 (+2.06z)| norm 0.3074 (+0.91z)| lr 3.11e-04 | 322.80 ms | 52.3% bf16 MFU | 1624682 tok/s
step 9920/19560 | loss 3.354851 (-1.12z)| norm 0.3100 (+1.02z)| lr 3.11e-04 | 322.36 ms | 52.4% bf16 MFU | 1624768 tok/s
step 9921/19560 | loss 3.478767 (+1.74z)| norm 0.3436 (+2.50z)| lr 3.10e-04 | 322.75 ms | 52.3% bf16 MFU | 1624752 tok/s
step 9922/19560 | loss 3.416354 (+0.28z)| norm 0.3227 (+1.52z)| lr 3.10e-04 | 323.25 ms | 52.2% bf16 MFU | 1624611 tok/s
step 9923/19560 | loss 3.400590 (-0.08z)| norm 0.3232 (+1.52z)| lr 3.10e-04 | 322.57 ms | 52.3% bf16 MFU | 1624648 tok/s
step 9924/19560 | loss 3.392999 (-0.27z)| norm 0.2985 (+0.39z)| lr 3.10e-04 | 322.25 ms | 52.4% bf16 MFU | 1624763 tok/s
step 9925/19560 | loss 3.424168 (+0.47z)| norm 0.2960 (+0.27z)| lr 3.10e-04 | 322.24 ms | 52.4% bf16 MFU | 1624876 tok/s
step 9926/19560 | loss 3.442568 (+0.89z)| norm 0.3035 (+0.60z)| lr 3.10e-04 | 322.91 ms | 52.3% bf16 MFU | 1624813 tok/s
step 9927/19560 | loss 3.415967 (+0.26z)| norm 0.2726 (-0.80z)| lr 3.10e-04 | 322.60 ms | 52.3% bf16 MFU | 1624831 tok/s
step 9928/19560 | loss 3.379057 (-0.62z)| norm 0.2876 (-0.12z)| lr 3.10e-04 | 322.60 ms | 52.3% bf16 MFU | 1624849 tok/s
step 9929/19560 | loss 3.404902 (-0.01z)| norm 0.2826 (-0.36z)| lr 3.10e-04 | 322.36 ms | 52.4% bf16 MFU | 1624927 tok/s
step 9930/19560 | loss 3.409867 (+0.12z)| norm 0.2644 (-1.21z)| lr 3.10e-04 | 322.24 ms | 52.4% bf16 MFU | 1625032 tok/s
step 9931/19560 | loss 3.400942 (-0.11z)| norm 0.2712 (-0.88z)| lr 3.10e-04 | 323.12 ms | 52.2% bf16 MFU | 1624909 tok/s
step 9932/19560 | loss 3.366341 (-0.96z)| norm 0.2605 (-1.37z)| lr 3.10e-04 | 322.93 ms | 52.3% bf16 MFU | 1624839 tok/s
step 9933/19560 | loss 3.415756 (+0.24z)| norm 0.2722 (-0.82z)| lr 3.10e-04 | 322.39 ms | 52.4% bf16 MFU | 1624910 tok/s
step 9934/19560 | loss 3.416163 (+0.24z)| norm 0.2531 (-1.70z)| lr 3.10e-04 | 322.41 ms | 52.3% bf16 MFU | 1624973 tok/s
step 9935/19560 | loss 3.512588 (+2.52z)| norm 0.2860 (-0.16z)| lr 3.10e-04 | 323.17 ms | 52.2% bf16 MFU | 1624841 tok/s
step 9936/19560 | loss 3.390676 (-0.39z)| norm 0.2707 (-0.89z)| lr 3.10e-04 | 322.78 ms | 52.3% bf16 MFU | 1624814 tok/s
step 9937/19560 | loss 3.428324 (+0.50z)| norm 0.2524 (-1.73z)| lr 3.10e-04 | 322.81 ms | 52.3% bf16 MFU | 1624780 tok/s
step 9938/19560 | loss 3.396899 (-0.25z)| norm 0.2626 (-1.26z)| lr 3.10e-04 | 322.38 ms | 52.4% bf16 MFU | 1624856 tok/s
step 9939/19560 | loss 3.405452 (-0.04z)| norm 0.2511 (-1.78z)| lr 3.10e-04 | 322.68 ms | 52.3% bf16 MFU | 1624852 tok/s
step 9940/19560 | loss 3.362383 (-1.07z)| norm 0.2704 (-0.87z)| lr 3.10e-04 | 322.53 ms | 52.3% bf16 MFU | 1624887 tok/s
step 9941/19560 | loss 3.406262 (-0.02z)| norm 0.2684 (-0.96z)| lr 3.09e-04 | 322.82 ms | 52.3% bf16 MFU | 1624846 tok/s
step 9942/19560 | loss 3.401573 (-0.13z)| norm 0.3061 (+0.80z)| lr 3.09e-04 | 322.12 ms | 52.4% bf16 MFU | 1624984 tok/s
step 9943/19560 | loss 3.523767 (+2.78z)| norm 0.2693 (-0.91z)| lr 3.09e-04 | 322.46 ms | 52.3% bf16 MFU | 1625030 tok/s
step 9944/19560 | loss 3.401977 (-0.13z)| norm 0.3169 (+1.28z)| lr 3.09e-04 | 322.27 ms | 52.4% bf16 MFU | 1625121 tok/s
step 9945/19560 | loss 3.437684 (+0.72z)| norm 0.2748 (-0.68z)| lr 3.09e-04 | 322.59 ms | 52.3% bf16 MFU | 1625127 tok/s
step 9946/19560 | loss 3.424180 (+0.40z)| norm 0.2847 (-0.22z)| lr 3.09e-04 | 322.29 ms | 52.4% bf16 MFU | 1625208 tok/s
step 9947/19560 | loss 3.398819 (-0.22z)| norm 0.2908 (+0.05z)| lr 3.09e-04 | 322.45 ms | 52.3% bf16 MFU | 1625246 tok/s
step 9948/19560 | loss 3.372057 (-0.86z)| norm 0.2727 (-0.79z)| lr 3.09e-04 | 322.41 ms | 52.3% bf16 MFU | 1625291 tok/s
step 9949/19560 | loss 3.420967 (+0.32z)| norm 0.2781 (-0.55z)| lr 3.09e-04 | 322.66 ms | 52.3% bf16 MFU | 1625271 tok/s
step 9950/19560 | loss 3.407738 (-0.01z)| norm 0.3113 (+1.02z)| lr 3.09e-04 | 323.12 ms | 52.2% bf16 MFU | 1625136 tok/s
step 9951/19560 | loss 3.344256 (-1.52z)| norm 0.2911 (+0.06z)| lr 3.09e-04 | 322.45 ms | 52.3% bf16 MFU | 1625176 tok/s
step 9952/19560 | loss 3.418759 (+0.26z)| norm 0.2801 (-0.46z)| lr 3.09e-04 | 322.57 ms | 52.3% bf16 MFU | 1625184 tok/s
step 9953/19560 | loss 3.408945 (+0.02z)| norm 0.2946 (+0.22z)| lr 3.09e-04 | 322.34 ms | 52.4% bf16 MFU | 1625249 tok/s
step 9954/19560 | loss 3.379551 (-0.67z)| norm 0.2709 (-0.89z)| lr 3.09e-04 | 322.66 ms | 52.3% bf16 MFU | 1625231 tok/s
step 9955/19560 | loss 3.356708 (-1.22z)| norm 0.9020 (+10.48z)| lr 3.09e-04 | 322.87 ms | 52.3% bf16 MFU | 1625161 tok/s
step 9956/19560 | loss 3.392318 (-0.36z)| norm 0.3005 (+0.10z)| lr 3.09e-04 | 322.74 ms | 52.3% bf16 MFU | 1625127 tok/s
step 9957/19560 | loss 3.414131 (+0.17z)| norm 0.2877 (-0.12z)| lr 3.09e-04 | 322.16 ms | 52.4% bf16 MFU | 1625240 tok/s
step 9958/19560 | loss 3.365431 (-0.99z)| norm 0.2711 (-0.40z)| lr 3.09e-04 | 322.64 ms | 52.3% bf16 MFU | 1625227 tok/s
step 9959/19560 | loss 3.393036 (-0.32z)| norm 0.2807 (-0.23z)| lr 3.09e-04 | 322.50 ms | 52.3% bf16 MFU | 1625250 tok/s
step 9960/19560 | loss 3.536603 (+3.03z)| norm 0.2787 (-0.25z)| lr 3.09e-04 | 322.30 ms | 52.4% bf16 MFU | 1625323 tok/s
step 9961/19560 | loss 3.421930 (+0.34z)| norm 0.2657 (-0.47z)| lr 3.08e-04 | 322.98 ms | 52.3% bf16 MFU | 1625221 tok/s
step 9962/19560 | loss 3.382137 (-0.59z)| norm 0.2780 (-0.25z)| lr 3.08e-04 | 322.48 ms | 52.3% bf16 MFU | 1625251 tok/s
step 9963/19560 | loss 3.396649 (-0.25z)| norm 0.2628 (-0.51z)| lr 3.08e-04 | 322.62 ms | 52.3% bf16 MFU | 1625244 tok/s
step 9964/19560 | loss 3.333853 (-1.69z)| norm 0.2763 (-0.28z)| lr 3.08e-04 | 322.80 ms | 52.3% bf16 MFU | 1625191 tok/s
step 9965/19560 | loss 3.375028 (-0.74z)| norm 0.2676 (-0.42z)| lr 3.08e-04 | 322.48 ms | 52.3% bf16 MFU | 1625220 tok/s
step 9966/19560 | loss 3.397249 (-0.23z)| norm 0.2780 (-0.24z)| lr 3.08e-04 | 322.74 ms | 52.3% bf16 MFU | 1625183 tok/s
step 9967/19560 | loss 3.418505 (+0.28z)| norm 0.2784 (-0.23z)| lr 3.08e-04 | 322.51 ms | 52.3% bf16 MFU | 1625205 tok/s
step 9968/19560 | loss 3.435609 (+0.73z)| norm 0.2733 (-0.32z)| lr 3.08e-04 | 322.24 ms | 52.4% bf16 MFU | 1625295 tok/s
step 9969/19560 | loss 3.352612 (-1.28z)| norm 0.3023 (+0.19z)| lr 3.08e-04 | 322.48 ms | 52.3% bf16 MFU | 1625319 tok/s
step 9970/19560 | loss 3.352092 (-1.27z)| norm 0.2885 (-0.05z)| lr 3.08e-04 | 322.80 ms | 52.3% bf16 MFU | 1625263 tok/s
step 9971/19560 | loss 3.360571 (-1.06z)| norm 0.2886 (-0.05z)| lr 3.08e-04 | 322.51 ms | 52.3% bf16 MFU | 1625283 tok/s
step 9972/19560 | loss 3.437040 (+0.78z)| norm 0.2819 (-0.17z)| lr 3.08e-04 | 322.18 ms | 52.4% bf16 MFU | 1625385 tok/s
step 9973/19560 | loss 3.343439 (-1.46z)| norm 0.2579 (-0.59z)| lr 3.08e-04 | 322.83 ms | 52.3% bf16 MFU | 1625318 tok/s
step 9974/19560 | loss 3.402462 (-0.06z)| norm 0.2903 (-0.02z)| lr 3.08e-04 | 322.48 ms | 52.3% bf16 MFU | 1625341 tok/s
step 9975/19560 | loss 3.423048 (+0.43z)| norm 0.2708 (-0.36z)| lr 3.08e-04 | 322.37 ms | 52.4% bf16 MFU | 1625392 tok/s
step 9976/19560 | loss 3.428644 (+0.60z)| norm 0.2742 (-0.30z)| lr 3.08e-04 | 322.41 ms | 52.3% bf16 MFU | 1625431 tok/s
step 9977/19560 | loss 3.581136 (+4.04z)| norm 0.3130 (+0.38z)| lr 3.08e-04 | 322.74 ms | 52.3% bf16 MFU | 1625385 tok/s
step 9978/19560 | loss 3.362245 (-1.00z)| norm 0.2759 (-0.27z)| lr 3.08e-04 | 322.42 ms | 52.3% bf16 MFU | 1625422 tok/s
step 9979/19560 | loss 3.403642 (-0.03z)| norm 0.3013 (+0.17z)| lr 3.08e-04 | 322.32 ms | 52.4% bf16 MFU | 1625482 tok/s
step 9980/19560 | loss 3.415594 (+0.24z)| norm 0.2702 (-0.36z)| lr 3.08e-04 | 322.55 ms | 52.3% bf16 MFU | 1625480 tok/s
step 9981/19560 | loss 3.425350 (+0.46z)| norm 0.2484 (-0.74z)| lr 3.07e-04 | 321.95 ms | 52.4% bf16 MFU | 1625629 tok/s
step 9982/19560 | loss 3.392904 (-0.28z)| norm 0.2643 (-0.46z)| lr 3.07e-04 | 322.54 ms | 52.3% bf16 MFU | 1625622 tok/s
step 9983/19560 | loss 3.385045 (-0.45z)| norm 0.2469 (-0.76z)| lr 3.07e-04 | 322.68 ms | 52.3% bf16 MFU | 1625580 tok/s
step 9984/19560 | loss 3.629923 (+4.78z)| norm 0.2690 (-0.38z)| lr 3.07e-04 | 322.50 ms | 52.3% bf16 MFU | 1625587 tok/s
step 9985/19560 | loss 3.426475 (+0.43z)| norm 0.2825 (-0.14z)| lr 3.07e-04 | 322.96 ms | 52.3% bf16 MFU | 1625478 tok/s
step 9986/19560 | loss 3.410400 (+0.08z)| norm 0.2780 (-0.22z)| lr 3.07e-04 | 322.60 ms | 52.3% bf16 MFU | 1625465 tok/s
step 9987/19560 | loss 3.374501 (-0.68z)| norm 0.2708 (-0.35z)| lr 3.07e-04 | 322.10 ms | 52.4% bf16 MFU | 1625578 tok/s
step 9988/19560 | loss 3.425613 (+0.41z)| norm 0.2708 (-0.34z)| lr 3.07e-04 | 322.45 ms | 52.3% bf16 MFU | 1625597 tok/s
step 9989/19560 | loss 3.423953 (+0.37z)| norm 0.2903 (-0.00z)| lr 3.07e-04 | 322.63 ms | 52.3% bf16 MFU | 1625569 tok/s
step 9990/19560 | loss 3.391456 (-0.33z)| norm 0.2650 (-0.44z)| lr 3.07e-04 | 322.99 ms | 52.3% bf16 MFU | 1625451 tok/s
step 9991/19560 | loss 3.378156 (-0.61z)| norm 0.2902 (+0.00z)| lr 3.07e-04 | 322.40 ms | 52.3% bf16 MFU | 1625489 tok/s
step 9992/19560 | loss 3.381907 (-0.52z)| norm 0.2736 (-0.28z)| lr 3.07e-04 | 322.27 ms | 52.4% bf16 MFU | 1625558 tok/s
step 9993/19560 | loss 3.437747 (+0.68z)| norm 0.2677 (-0.38z)| lr 3.07e-04 | 322.51 ms | 52.3% bf16 MFU | 1625562 tok/s
step 9994/19560 | loss 3.405512 (-0.02z)| norm 0.2808 (-0.15z)| lr 3.07e-04 | 322.58 ms | 52.3% bf16 MFU | 1625548 tok/s
step 9995/19560 | loss 3.379626 (-0.58z)| norm 0.2708 (-0.32z)| lr 3.07e-04 | 322.38 ms | 52.4% bf16 MFU | 1625585 tok/s
step 9996/19560 | loss 3.405016 (-0.03z)| norm 0.2650 (-0.42z)| lr 3.07e-04 | 322.63 ms | 52.3% bf16 MFU | 1625558 tok/s
step 9997/19560 | loss 3.406706 (+0.01z)| norm 0.2766 (-0.22z)| lr 3.07e-04 | 322.38 ms | 52.4% bf16 MFU | 1625595 tok/s
step 9998/19560 | loss 3.384115 (-0.49z)| norm 0.2680 (-0.36z)| lr 3.07e-04 | 322.96 ms | 52.3% bf16 MFU | 1625486 tok/s
step 9999/19560 | loss 3.374701 (-0.68z)| norm 0.3041 (+0.27z)| lr 3.07e-04 | 322.52 ms | 52.3% bf16 MFU | 1625492 tok/s
step 10000/19560 | loss 3.419989 (+0.30z)| norm 0.2602 (-0.49z)| lr 3.07e-04 | 322.77 ms | 52.3% bf16 MFU | 1625436 tok/s
val loss 3.396394

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00010000_00007.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00010000_00004.bin
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2880/10042 = 0.286795
Writing checkpoint at step 10000
Writing model to log124M/model_00010000.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00010000_00003.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00010000_00005.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00010000_00002.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00010000_00001.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00010000_00006.bin
Writing state to log124M/state_00010000_00000.bin
step 10001/19560 | loss 3.387290 (-0.40z)| norm 0.3112 (+0.40z)| lr 3.06e-04 | 318.39 ms | 53.0% bf16 MFU | 1626498 tok/s
step 10002/19560 | loss 3.454981 (+1.06z)| norm 0.3007 (+0.21z)| lr 3.06e-04 | 321.28 ms | 52.5% bf16 MFU | 1626766 tok/s
step 10003/19560 | loss 3.372087 (-0.73z)| norm 0.2700 (-0.32z)| lr 3.06e-04 | 323.64 ms | 52.1% bf16 MFU | 1626425 tok/s
step 10004/19560 | loss 3.443544 (+0.82z)| norm 0.3114 (+0.40z)| lr 3.06e-04 | 322.44 ms | 52.3% bf16 MFU | 1626405 tok/s
step 10005/19560 | loss 3.457691 (+1.13z)| norm 0.2748 (-0.23z)| lr 3.06e-04 | 322.32 ms | 52.4% bf16 MFU | 1626416 tok/s
step 10006/19560 | loss 3.385425 (-0.43z)| norm 0.2844 (-0.07z)| lr 3.06e-04 | 323.77 ms | 52.1% bf16 MFU | 1626062 tok/s
step 10007/19560 | loss 3.411654 (+0.14z)| norm 0.2965 (+0.14z)| lr 3.06e-04 | 322.46 ms | 52.3% bf16 MFU | 1626053 tok/s
step 10008/19560 | loss 3.370457 (-0.75z)| norm 0.3042 (+0.28z)| lr 3.06e-04 | 322.65 ms | 52.3% bf16 MFU | 1625998 tok/s
step 10009/19560 | loss 3.442785 (+0.83z)| norm 0.2777 (-0.19z)| lr 3.06e-04 | 322.50 ms | 52.3% bf16 MFU | 1625982 tok/s
step 10010/19560 | loss 3.423753 (+0.40z)| norm 0.2865 (-0.03z)| lr 3.06e-04 | 322.70 ms | 52.3% bf16 MFU | 1625917 tok/s
step 10011/19560 | loss 3.422318 (+0.38z)| norm 0.2755 (-0.22z)| lr 3.06e-04 | 322.45 ms | 52.3% bf16 MFU | 1625918 tok/s
step 10012/19560 | loss 3.437294 (+0.70z)| norm 0.2946 (+0.11z)| lr 3.06e-04 | 322.81 ms | 52.3% bf16 MFU | 1625830 tok/s
step 10013/19560 | loss 3.367734 (-0.84z)| norm 0.2624 (-0.44z)| lr 3.06e-04 | 323.10 ms | 52.2% bf16 MFU | 1625673 tok/s
step 10014/19560 | loss 3.418018 (+0.27z)| norm 0.2853 (-0.05z)| lr 3.06e-04 | 322.65 ms | 52.3% bf16 MFU | 1625636 tok/s
step 10015/19560 | loss 3.367171 (-0.87z)| norm 0.2496 (-0.66z)| lr 3.06e-04 | 323.19 ms | 52.2% bf16 MFU | 1625466 tok/s
step 10016/19560 | loss 3.444044 (+0.84z)| norm 0.2672 (-0.36z)| lr 3.06e-04 | 323.16 ms | 52.2% bf16 MFU | 1625310 tok/s
step 10017/19560 | loss 3.392286 (-0.33z)| norm 0.3000 (+0.21z)| lr 3.06e-04 | 322.81 ms | 52.3% bf16 MFU | 1625251 tok/s
step 10018/19560 | loss 3.422940 (+0.36z)| norm 0.2612 (-0.45z)| lr 3.06e-04 | 322.69 ms | 52.3% bf16 MFU | 1625224 tok/s
step 10019/19560 | loss 3.440383 (+0.74z)| norm 0.2929 (+0.10z)| lr 3.06e-04 | 323.28 ms | 52.2% bf16 MFU | 1625051 tok/s
step 10020/19560 | loss 3.414966 (+0.16z)| norm 0.2794 (-0.13z)| lr 3.06e-04 | 322.93 ms | 52.3% bf16 MFU | 1624975 tok/s
step 10021/19560 | loss 3.444747 (+0.83z)| norm 0.2745 (-0.21z)| lr 3.05e-04 | 322.98 ms | 52.3% bf16 MFU | 1624890 tok/s
step 10022/19560 | loss 3.446858 (+0.86z)| norm 0.2697 (-0.30z)| lr 3.05e-04 | 323.05 ms | 52.2% bf16 MFU | 1624792 tok/s
step 10023/19560 | loss 3.587350 (+3.77z)| norm 0.2952 (+0.15z)| lr 3.05e-04 | 323.10 ms | 52.2% bf16 MFU | 1624687 tok/s
step 10024/19560 | loss 3.371219 (-0.82z)| norm 0.3197 (+0.56z)| lr 3.05e-04 | 323.36 ms | 52.2% bf16 MFU | 1624521 tok/s
step 10025/19560 | loss 3.383323 (-0.57z)| norm 0.2536 (-0.58z)| lr 3.05e-04 | 322.79 ms | 52.3% bf16 MFU | 1624508 tok/s
step 10026/19560 | loss 3.408937 (-0.02z)| norm 0.2836 (-0.06z)| lr 3.05e-04 | 322.90 ms | 52.3% bf16 MFU | 1624466 tok/s
step 10027/19560 | loss 3.450514 (+0.87z)| norm 0.2717 (-0.26z)| lr 3.05e-04 | 323.08 ms | 52.2% bf16 MFU | 1624382 tok/s
step 10028/19560 | loss 3.521168 (+2.31z)| norm 0.2735 (-0.23z)| lr 3.05e-04 | 323.87 ms | 52.1% bf16 MFU | 1624105 tok/s
step 10029/19560 | loss 3.420774 (+0.20z)| norm 0.2532 (-0.58z)| lr 3.05e-04 | 323.30 ms | 52.2% bf16 MFU | 1623985 tok/s
step 10030/19560 | loss 3.370452 (-0.86z)| norm 0.2551 (-0.55z)| lr 3.05e-04 | 322.61 ms | 52.3% bf16 MFU | 1624042 tok/s
step 10031/19560 | loss 3.416344 (+0.10z)| norm 0.2555 (-0.54z)| lr 3.05e-04 | 322.84 ms | 52.3% bf16 MFU | 1624040 tok/s
step 10032/19560 | loss 3.456600 (+0.94z)| norm 0.2553 (-0.54z)| lr 3.05e-04 | 322.91 ms | 52.3% bf16 MFU | 1624021 tok/s
step 10033/19560 | loss 3.351793 (-1.25z)| norm 0.2539 (-0.56z)| lr 3.05e-04 | 323.07 ms | 52.2% bf16 MFU | 1623962 tok/s
step 10034/19560 | loss 3.369354 (-0.91z)| norm 0.2448 (-0.71z)| lr 3.05e-04 | 323.20 ms | 52.2% bf16 MFU | 1623873 tok/s
step 10035/19560 | loss 3.373998 (-0.81z)| norm 0.2538 (-0.55z)| lr 3.05e-04 | 323.14 ms | 52.2% bf16 MFU | 1623804 tok/s
step 10036/19560 | loss 3.396340 (-0.32z)| norm 0.2614 (-0.42z)| lr 3.05e-04 | 322.91 ms | 52.3% bf16 MFU | 1623796 tok/s
step 10037/19560 | loss 3.411421 (+0.00z)| norm 0.2583 (-0.47z)| lr 3.05e-04 | 322.95 ms | 52.3% bf16 MFU | 1623778 tok/s
step 10038/19560 | loss 3.410031 (-0.04z)| norm 0.2619 (-0.40z)| lr 3.05e-04 | 322.90 ms | 52.3% bf16 MFU | 1623775 tok/s
step 10039/19560 | loss 3.385334 (-0.57z)| norm 0.2706 (-0.25z)| lr 3.05e-04 | 322.87 ms | 52.3% bf16 MFU | 1623777 tok/s
step 10040/19560 | loss 3.399068 (-0.28z)| norm 0.2881 (+0.05z)| lr 3.05e-04 | 323.15 ms | 52.2% bf16 MFU | 1623709 tok/s
step 10041/19560 | loss 3.414889 (+0.06z)| norm 0.2555 (-0.51z)| lr 3.04e-04 | 323.47 ms | 52.2% bf16 MFU | 1623565 tok/s
step 10042/19560 | loss 3.387003 (-0.54z)| norm 0.2735 (-0.19z)| lr 3.04e-04 | 322.45 ms | 52.3% bf16 MFU | 1623686 tok/s
step 10043/19560 | loss 3.410193 (-0.03z)| norm 0.2637 (-0.35z)| lr 3.04e-04 | 323.33 ms | 52.2% bf16 MFU | 1623577 tok/s
step 10044/19560 | loss 3.460828 (+1.06z)| norm 0.3017 (+0.30z)| lr 3.04e-04 | 322.32 ms | 52.4% bf16 MFU | 1623729 tok/s
step 10045/19560 | loss 3.390901 (-0.46z)| norm 0.3011 (+0.29z)| lr 3.04e-04 | 324.21 ms | 52.1% bf16 MFU | 1623399 tok/s
step 10046/19560 | loss 3.428874 (+0.37z)| norm 0.2507 (-0.57z)| lr 3.04e-04 | 322.72 ms | 52.3% bf16 MFU | 1623458 tok/s
step 10047/19560 | loss 3.383642 (-0.60z)| norm 0.2968 (+0.23z)| lr 3.04e-04 | 322.79 ms | 52.3% bf16 MFU | 1623496 tok/s
step 10048/19560 | loss 3.467684 (+1.22z)| norm 0.2901 (+0.11z)| lr 3.04e-04 | 323.78 ms | 52.1% bf16 MFU | 1623286 tok/s
step 10049/19560 | loss 3.366791 (-0.98z)| norm 0.2752 (-0.13z)| lr 3.04e-04 | 322.40 ms | 52.3% bf16 MFU | 1623432 tok/s
step 10050/19560 | loss 3.421862 (+0.23z)| norm 0.2925 (+0.17z)| lr 3.04e-04 | 322.67 ms | 52.3% bf16 MFU | 1623503 tok/s
step 10051/19560 | loss 3.395822 (-0.34z)| norm 0.2868 (+0.08z)| lr 3.04e-04 | 323.31 ms | 52.2% bf16 MFU | 1623409 tok/s
step 10052/19560 | loss 3.397180 (-0.31z)| norm 0.2727 (-0.17z)| lr 3.04e-04 | 322.80 ms | 52.3% bf16 MFU | 1623447 tok/s
step 10053/19560 | loss 3.426479 (+0.34z)| norm 0.2773 (-0.08z)| lr 3.04e-04 | 323.26 ms | 52.2% bf16 MFU | 1623369 tok/s
step 10054/19560 | loss 3.421022 (+0.22z)| norm 0.2964 (+0.25z)| lr 3.04e-04 | 322.49 ms | 52.3% bf16 MFU | 1623489 tok/s
step 10055/19560 | loss 3.362828 (-1.05z)| norm 0.2710 (-0.19z)| lr 3.04e-04 | 323.39 ms | 52.2% bf16 MFU | 1623376 tok/s
step 10056/19560 | loss 3.392787 (-0.40z)| norm 0.2570 (-0.43z)| lr 3.04e-04 | 322.43 ms | 52.3% bf16 MFU | 1623509 tok/s
step 10057/19560 | loss 3.380988 (-0.65z)| norm 0.2781 (-0.06z)| lr 3.04e-04 | 323.24 ms | 52.2% bf16 MFU | 1623432 tok/s
step 10058/19560 | loss 3.447933 (+0.81z)| norm 0.3076 (+0.44z)| lr 3.04e-04 | 323.54 ms | 52.2% bf16 MFU | 1623283 tok/s
step 10059/19560 | loss 3.422152 (+0.24z)| norm 0.2989 (+0.29z)| lr 3.04e-04 | 322.93 ms | 52.3% bf16 MFU | 1623295 tok/s
step 10060/19560 | loss 3.405226 (-0.13z)| norm 0.3333 (+0.88z)| lr 3.04e-04 | 323.42 ms | 52.2% bf16 MFU | 1623185 tok/s
step 10061/19560 | loss 3.429962 (+0.41z)| norm 0.2712 (-0.20z)| lr 3.03e-04 | 323.05 ms | 52.2% bf16 MFU | 1623171 tok/s
step 10062/19560 | loss 3.382407 (-0.63z)| norm 0.2940 (+0.19z)| lr 3.03e-04 | 323.05 ms | 52.2% bf16 MFU | 1623159 tok/s
step 10063/19560 | loss 3.480401 (+1.54z)| norm 0.2764 (-0.12z)| lr 3.03e-04 | 323.95 ms | 52.1% bf16 MFU | 1622921 tok/s
step 10064/19560 | loss 3.416005 (+0.11z)| norm 0.2879 (+0.08z)| lr 3.03e-04 | 323.11 ms | 52.2% bf16 MFU | 1622907 tok/s
step 10065/19560 | loss 3.366560 (-0.98z)| norm 0.2992 (+0.27z)| lr 3.03e-04 | 323.26 ms | 52.2% bf16 MFU | 1622856 tok/s
step 10066/19560 | loss 3.387463 (-0.51z)| norm 0.2522 (-0.54z)| lr 3.03e-04 | 322.97 ms | 52.3% bf16 MFU | 1622878 tok/s
step 10067/19560 | loss 3.345057 (-1.43z)| norm 0.3368 (+0.91z)| lr 3.03e-04 | 323.35 ms | 52.2% bf16 MFU | 1622805 tok/s
step 10068/19560 | loss 3.376671 (-0.74z)| norm 0.2683 (-0.28z)| lr 3.03e-04 | 323.51 ms | 52.2% bf16 MFU | 1622697 tok/s
step 10069/19560 | loss 3.385193 (-0.55z)| norm 0.3149 (+0.53z)| lr 3.03e-04 | 322.90 ms | 52.3% bf16 MFU | 1622747 tok/s
step 10070/19560 | loss 3.385457 (-0.54z)| norm 0.2723 (-0.21z)| lr 3.03e-04 | 323.75 ms | 52.1% bf16 MFU | 1622581 tok/s
step 10071/19560 | loss 3.369804 (-0.87z)| norm 0.3106 (+0.45z)| lr 3.03e-04 | 323.51 ms | 52.2% bf16 MFU | 1622484 tok/s
step 10072/19560 | loss 3.389155 (-0.44z)| norm 0.3128 (+0.49z)| lr 3.03e-04 | 322.81 ms | 52.3% bf16 MFU | 1622566 tok/s
step 10073/19560 | loss 3.382064 (-0.59z)| norm 0.2983 (+0.23z)| lr 3.03e-04 | 323.21 ms | 52.2% bf16 MFU | 1622543 tok/s
step 10074/19560 | loss 3.410367 (+0.05z)| norm 0.3139 (+0.50z)| lr 3.03e-04 | 322.20 ms | 52.4% bf16 MFU | 1622776 tok/s
step 10075/19560 | loss 3.402290 (-0.13z)| norm 0.2704 (-0.25z)| lr 3.03e-04 | 324.17 ms | 52.1% bf16 MFU | 1622505 tok/s
step 10076/19560 | loss 3.443523 (+0.78z)| norm 0.3027 (+0.31z)| lr 3.03e-04 | 323.14 ms | 52.2% bf16 MFU | 1622504 tok/s
step 10077/19560 | loss 3.446202 (+0.84z)| norm 0.2619 (-0.40z)| lr 3.03e-04 | 322.96 ms | 52.3% bf16 MFU | 1622547 tok/s
step 10078/19560 | loss 3.420761 (+0.26z)| norm 0.2702 (-0.25z)| lr 3.03e-04 | 323.45 ms | 52.2% bf16 MFU | 1622465 tok/s
step 10079/19560 | loss 3.461098 (+1.15z)| norm 0.2941 (+0.16z)| lr 3.03e-04 | 323.11 ms | 52.2% bf16 MFU | 1622474 tok/s
step 10080/19560 | loss 3.402225 (-0.17z)| norm 0.2449 (-0.68z)| lr 3.03e-04 | 322.96 ms | 52.3% bf16 MFU | 1622518 tok/s
step 10081/19560 | loss 3.382689 (-0.60z)| norm 0.2955 (+0.19z)| lr 3.02e-04 | 322.63 ms | 52.3% bf16 MFU | 1622644 tok/s
step 10082/19560 | loss 3.396438 (-0.30z)| norm 0.2608 (-0.41z)| lr 3.02e-04 | 324.18 ms | 52.1% bf16 MFU | 1622375 tok/s
step 10083/19560 | loss 3.364377 (-1.02z)| norm 0.2641 (-0.81z)| lr 3.02e-04 | 323.06 ms | 52.2% bf16 MFU | 1622400 tok/s
step 10084/19560 | loss 3.367034 (-0.95z)| norm 0.2877 (+0.45z)| lr 3.02e-04 | 322.88 ms | 52.3% bf16 MFU | 1622468 tok/s
step 10085/19560 | loss 3.395104 (-0.32z)| norm 0.2682 (-0.58z)| lr 3.02e-04 | 322.50 ms | 52.3% bf16 MFU | 1622629 tok/s
step 10086/19560 | loss 3.446591 (+0.82z)| norm 0.2666 (-0.66z)| lr 3.02e-04 | 323.11 ms | 52.2% bf16 MFU | 1622630 tok/s
step 10087/19560 | loss 3.411141 (+0.02z)| norm 0.2763 (-0.14z)| lr 3.02e-04 | 323.21 ms | 52.2% bf16 MFU | 1622606 tok/s
step 10088/19560 | loss 3.408317 (-0.02z)| norm 0.2799 (+0.05z)| lr 3.02e-04 | 323.00 ms | 52.3% bf16 MFU | 1622634 tok/s
step 10089/19560 | loss 3.382884 (-0.60z)| norm 0.2798 (+0.04z)| lr 3.02e-04 | 323.67 ms | 52.1% bf16 MFU | 1622492 tok/s
step 10090/19560 | loss 3.357160 (-1.19z)| norm 0.2820 (+0.16z)| lr 3.02e-04 | 323.18 ms | 52.2% bf16 MFU | 1622481 tok/s
step 10091/19560 | loss 3.414435 (+0.13z)| norm 0.2846 (+0.29z)| lr 3.02e-04 | 323.04 ms | 52.2% bf16 MFU | 1622507 tok/s
step 10092/19560 | loss 3.359769 (-1.15z)| norm 0.2912 (+0.63z)| lr 3.02e-04 | 323.08 ms | 52.2% bf16 MFU | 1622521 tok/s
step 10093/19560 | loss 3.429482 (+0.47z)| norm 0.2800 (+0.03z)| lr 3.02e-04 | 322.97 ms | 52.3% bf16 MFU | 1622562 tok/s
step 10094/19560 | loss 3.424188 (+0.34z)| norm 0.2968 (+0.92z)| lr 3.02e-04 | 322.69 ms | 52.3% bf16 MFU | 1622671 tok/s
step 10095/19560 | loss 3.367521 (-0.97z)| norm 0.2823 (+0.14z)| lr 3.02e-04 | 322.98 ms | 52.3% bf16 MFU | 1622701 tok/s
step 10096/19560 | loss 3.373178 (-0.83z)| norm 0.3346 (+2.81z)| lr 3.02e-04 | 322.78 ms | 52.3% bf16 MFU | 1622779 tok/s
step 10097/19560 | loss 3.415446 (+0.14z)| norm 0.2879 (+0.41z)| lr 3.02e-04 | 322.87 ms | 52.3% bf16 MFU | 1622833 tok/s
step 10098/19560 | loss 3.381071 (-0.67z)| norm 0.3358 (+2.79z)| lr 3.02e-04 | 323.21 ms | 52.2% bf16 MFU | 1622799 tok/s
step 10099/19560 | loss 3.397817 (-0.28z)| norm 0.2884 (+0.40z)| lr 3.02e-04 | 323.14 ms | 52.2% bf16 MFU | 1622784 tok/s
step 10100/19560 | loss 3.427428 (+0.42z)| norm 0.3165 (+1.78z)| lr 3.02e-04 | 323.39 ms | 52.2% bf16 MFU | 1622707 tok/s
step 10101/19560 | loss 3.402761 (-0.18z)| norm 0.2745 (-0.31z)| lr 3.01e-04 | 322.36 ms | 52.4% bf16 MFU | 1622892 tok/s
step 10102/19560 | loss 3.400195 (-0.24z)| norm 0.3105 (+1.46z)| lr 3.01e-04 | 323.10 ms | 52.2% bf16 MFU | 1622881 tok/s
step 10103/19560 | loss 3.388227 (-0.52z)| norm 0.2869 (+0.29z)| lr 3.01e-04 | 322.99 ms | 52.3% bf16 MFU | 1622899 tok/s
step 10104/19560 | loss 3.449624 (+0.94z)| norm 0.2592 (-1.08z)| lr 3.01e-04 | 323.39 ms | 52.2% bf16 MFU | 1622815 tok/s
step 10105/19560 | loss 3.356107 (-1.32z)| norm 0.3425 (+2.95z)| lr 3.01e-04 | 323.13 ms | 52.2% bf16 MFU | 1622802 tok/s
step 10106/19560 | loss 3.449329 (+1.02z)| norm 0.2609 (-0.97z)| lr 3.01e-04 | 322.57 ms | 52.3% bf16 MFU | 1622929 tok/s
step 10107/19560 | loss 3.452118 (+1.08z)| norm 0.2998 (+0.90z)| lr 3.01e-04 | 322.98 ms | 52.3% bf16 MFU | 1622947 tok/s
step 10108/19560 | loss 3.439776 (+0.76z)| norm 0.2721 (-0.43z)| lr 3.01e-04 | 322.98 ms | 52.3% bf16 MFU | 1622964 tok/s
step 10109/19560 | loss 3.433910 (+0.61z)| norm 0.2915 (+0.49z)| lr 3.01e-04 | 322.96 ms | 52.3% bf16 MFU | 1622986 tok/s
step 10110/19560 | loss 3.437836 (+0.70z)| norm 0.2781 (-0.17z)| lr 3.01e-04 | 322.73 ms | 52.3% bf16 MFU | 1623064 tok/s
step 10111/19560 | loss 3.459588 (+1.23z)| norm 0.2907 (+0.43z)| lr 3.01e-04 | 322.96 ms | 52.3% bf16 MFU | 1623080 tok/s
step 10112/19560 | loss 3.549150 (+3.75z)| norm 0.3011 (+0.93z)| lr 3.01e-04 | 322.70 ms | 52.3% bf16 MFU | 1623160 tok/s
step 10113/19560 | loss 3.380008 (-0.79z)| norm 0.2681 (-0.68z)| lr 3.01e-04 | 322.51 ms | 52.3% bf16 MFU | 1623284 tok/s
step 10114/19560 | loss 3.437291 (+0.74z)| norm 0.2954 (+0.65z)| lr 3.01e-04 | 322.50 ms | 52.3% bf16 MFU | 1623404 tok/s
step 10115/19560 | loss 3.407177 (-0.08z)| norm 0.2794 (-0.14z)| lr 3.01e-04 | 322.81 ms | 52.3% bf16 MFU | 1623440 tok/s
step 10116/19560 | loss 3.423483 (+0.36z)| norm 0.2870 (+0.23z)| lr 3.01e-04 | 323.12 ms | 52.2% bf16 MFU | 1623396 tok/s
step 10117/19560 | loss 3.380255 (-0.79z)| norm 0.2732 (-0.44z)| lr 3.01e-04 | 322.28 ms | 52.4% bf16 MFU | 1623566 tok/s
step 10118/19560 | loss 3.365291 (-1.18z)| norm 0.2897 (+0.36z)| lr 3.01e-04 | 323.35 ms | 52.2% bf16 MFU | 1623460 tok/s
step 10119/19560 | loss 3.443975 (+0.91z)| norm 0.2874 (+0.25z)| lr 3.01e-04 | 322.46 ms | 52.3% bf16 MFU | 1623581 tok/s
step 10120/19560 | loss 3.449111 (+1.03z)| norm 0.2775 (-0.24z)| lr 3.01e-04 | 323.12 ms | 52.2% bf16 MFU | 1623530 tok/s
step 10121/19560 | loss 3.371733 (-1.02z)| norm 0.2899 (+0.36z)| lr 3.00e-04 | 322.82 ms | 52.3% bf16 MFU | 1623557 tok/s
step 10122/19560 | loss 3.336457 (-1.92z)| norm 0.2802 (-0.12z)| lr 3.00e-04 | 322.24 ms | 52.4% bf16 MFU | 1623730 tok/s
step 10123/19560 | loss 3.341967 (-1.75z)| norm 0.2729 (-0.48z)| lr 3.00e-04 | 322.53 ms | 52.3% bf16 MFU | 1623821 tok/s
step 10124/19560 | loss 3.464136 (+1.41z)| norm 0.2843 (+0.08z)| lr 3.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1623854 tok/s
step 10125/19560 | loss 3.364014 (-1.16z)| norm 0.2546 (-1.37z)| lr 3.00e-04 | 322.66 ms | 52.3% bf16 MFU | 1623907 tok/s
step 10126/19560 | loss 3.402504 (-0.18z)| norm 0.2792 (-0.17z)| lr 3.00e-04 | 322.87 ms | 52.3% bf16 MFU | 1623903 tok/s
step 10127/19560 | loss 3.395495 (-0.36z)| norm 0.2711 (-0.56z)| lr 3.00e-04 | 322.16 ms | 52.4% bf16 MFU | 1624077 tok/s
step 10128/19560 | loss 3.415736 (+0.16z)| norm 0.2793 (-0.16z)| lr 3.00e-04 | 322.83 ms | 52.3% bf16 MFU | 1624076 tok/s
step 10129/19560 | loss 3.366895 (-1.09z)| norm 0.2720 (-0.51z)| lr 3.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1624131 tok/s
step 10130/19560 | loss 3.357682 (-1.31z)| norm 0.2836 (+0.07z)| lr 3.00e-04 | 322.39 ms | 52.3% bf16 MFU | 1624236 tok/s
step 10131/19560 | loss 3.371634 (-0.95z)| norm 0.2471 (-1.72z)| lr 3.00e-04 | 322.87 ms | 52.3% bf16 MFU | 1624216 tok/s
step 10132/19560 | loss 3.408292 (-0.00z)| norm 0.2620 (-0.97z)| lr 3.00e-04 | 322.37 ms | 52.4% bf16 MFU | 1624323 tok/s
step 10133/19560 | loss 3.432796 (+0.64z)| norm 0.2645 (-0.84z)| lr 3.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1624348 tok/s
step 10134/19560 | loss 3.340226 (-1.73z)| norm 0.2865 (+0.25z)| lr 3.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1624389 tok/s
step 10135/19560 | loss 3.395983 (-0.30z)| norm 0.2781 (-0.16z)| lr 3.00e-04 | 322.37 ms | 52.4% bf16 MFU | 1624489 tok/s
step 10136/19560 | loss 3.356447 (-1.30z)| norm 0.2684 (-0.63z)| lr 3.00e-04 | 322.32 ms | 52.4% bf16 MFU | 1624594 tok/s
step 10137/19560 | loss 3.343072 (-1.61z)| norm 0.2778 (-0.16z)| lr 3.00e-04 | 322.91 ms | 52.3% bf16 MFU | 1624547 tok/s
step 10138/19560 | loss 3.467633 (+1.52z)| norm 0.2844 (+0.16z)| lr 3.00e-04 | 322.49 ms | 52.3% bf16 MFU | 1624607 tok/s
step 10139/19560 | loss 3.443596 (+0.91z)| norm 0.2704 (-0.53z)| lr 3.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1624618 tok/s
step 10140/19560 | loss 3.370003 (-0.92z)| norm 0.2696 (-0.56z)| lr 3.00e-04 | 323.12 ms | 52.2% bf16 MFU | 1624516 tok/s
step 10141/19560 | loss 3.394017 (-0.32z)| norm 0.2754 (-0.28z)| lr 3.00e-04 | 322.53 ms | 52.3% bf16 MFU | 1624568 tok/s
step 10142/19560 | loss 3.391545 (-0.38z)| norm 0.3039 (+1.13z)| lr 2.99e-04 | 323.02 ms | 52.2% bf16 MFU | 1624492 tok/s
step 10143/19560 | loss 3.355554 (-1.28z)| norm 0.2654 (-0.79z)| lr 2.99e-04 | 322.61 ms | 52.3% bf16 MFU | 1624525 tok/s
step 10144/19560 | loss 3.492547 (+2.11z)| norm 0.2871 (+0.29z)| lr 2.99e-04 | 323.10 ms | 52.2% bf16 MFU | 1624433 tok/s
step 10145/19560 | loss 3.346537 (-1.47z)| norm 0.2673 (-0.70z)| lr 2.99e-04 | 322.72 ms | 52.3% bf16 MFU | 1624442 tok/s
step 10146/19560 | loss 3.435891 (+0.71z)| norm 0.2580 (-1.16z)| lr 2.99e-04 | 322.71 ms | 52.3% bf16 MFU | 1624453 tok/s
step 10147/19560 | loss 3.404161 (-0.06z)| norm 0.2744 (-0.33z)| lr 2.99e-04 | 322.47 ms | 52.3% bf16 MFU | 1624522 tok/s
step 10148/19560 | loss 3.471923 (+1.58z)| norm 0.2660 (-0.74z)| lr 2.99e-04 | 322.61 ms | 52.3% bf16 MFU | 1624554 tok/s
step 10149/19560 | loss 3.408821 (+0.05z)| norm 0.2730 (-0.39z)| lr 2.99e-04 | 323.27 ms | 52.2% bf16 MFU | 1624417 tok/s
step 10150/19560 | loss 3.449536 (+1.04z)| norm 0.2678 (-0.65z)| lr 2.99e-04 | 322.62 ms | 52.3% bf16 MFU | 1624452 tok/s
step 10151/19560 | loss 3.421459 (+0.42z)| norm 0.2720 (-0.43z)| lr 2.99e-04 | 323.72 ms | 52.1% bf16 MFU | 1624208 tok/s
step 10152/19560 | loss 3.417914 (+0.32z)| norm 0.2747 (-0.28z)| lr 2.99e-04 | 322.30 ms | 52.4% bf16 MFU | 1624334 tok/s
step 10153/19560 | loss 3.386708 (-0.51z)| norm 0.2748 (-0.29z)| lr 2.99e-04 | 322.39 ms | 52.3% bf16 MFU | 1624429 tok/s
step 10154/19560 | loss 3.413061 (+0.19z)| norm 0.2530 (-1.38z)| lr 2.99e-04 | 323.10 ms | 52.2% bf16 MFU | 1624342 tok/s
step 10155/19560 | loss 3.407744 (+0.06z)| norm 0.2719 (-0.42z)| lr 2.99e-04 | 323.00 ms | 52.3% bf16 MFU | 1624285 tok/s
step 10156/19560 | loss 3.424638 (+0.55z)| norm 0.2476 (-1.63z)| lr 2.99e-04 | 323.00 ms | 52.3% bf16 MFU | 1624230 tok/s
step 10157/19560 | loss 3.404423 (-0.00z)| norm 0.2744 (-0.29z)| lr 2.99e-04 | 322.48 ms | 52.3% bf16 MFU | 1624310 tok/s
step 10158/19560 | loss 3.365352 (-1.08z)| norm 0.2588 (-1.09z)| lr 2.99e-04 | 322.50 ms | 52.3% bf16 MFU | 1624380 tok/s
step 10159/19560 | loss 3.404846 (+0.01z)| norm 0.2782 (-0.11z)| lr 2.99e-04 | 323.28 ms | 52.2% bf16 MFU | 1624250 tok/s
step 10160/19560 | loss 3.393644 (-0.29z)| norm 0.2752 (-0.27z)| lr 2.99e-04 | 322.39 ms | 52.4% bf16 MFU | 1624350 tok/s
step 10161/19560 | loss 3.375583 (-0.80z)| norm 0.2770 (-0.19z)| lr 2.99e-04 | 323.48 ms | 52.2% bf16 MFU | 1624171 tok/s
step 10162/19560 | loss 3.469923 (+1.81z)| norm 0.2537 (-1.41z)| lr 2.98e-04 | 322.66 ms | 52.3% bf16 MFU | 1624207 tok/s
step 10163/19560 | loss 3.364522 (-1.12z)| norm 0.2666 (-0.75z)| lr 2.98e-04 | 322.62 ms | 52.3% bf16 MFU | 1624252 tok/s
step 10164/19560 | loss 3.410353 (+0.15z)| norm 0.2318 (-2.51z)| lr 2.98e-04 | 321.82 ms | 52.4% bf16 MFU | 1624497 tok/s
step 10165/19560 | loss 3.400705 (-0.12z)| norm 0.2707 (-0.52z)| lr 2.98e-04 | 323.92 ms | 52.1% bf16 MFU | 1624201 tok/s
step 10166/19560 | loss 3.423496 (+0.51z)| norm 0.2766 (-0.22z)| lr 2.98e-04 | 322.84 ms | 52.3% bf16 MFU | 1624190 tok/s
step 10167/19560 | loss 3.384475 (-0.57z)| norm 0.2679 (-0.67z)| lr 2.98e-04 | 322.40 ms | 52.3% bf16 MFU | 1624290 tok/s
step 10168/19560 | loss 3.403498 (-0.04z)| norm 0.2882 (+0.38z)| lr 2.98e-04 | 322.77 ms | 52.3% bf16 MFU | 1624294 tok/s
step 10169/19560 | loss 3.441831 (+1.01z)| norm 0.2905 (+0.49z)| lr 2.98e-04 | 322.75 ms | 52.3% bf16 MFU | 1624301 tok/s
step 10170/19560 | loss 3.422220 (+0.46z)| norm 0.2995 (+0.95z)| lr 2.98e-04 | 323.10 ms | 52.2% bf16 MFU | 1624219 tok/s
step 10171/19560 | loss 3.366889 (-1.05z)| norm 0.2928 (+0.59z)| lr 2.98e-04 | 322.44 ms | 52.3% bf16 MFU | 1624309 tok/s
step 10172/19560 | loss 3.350686 (-1.48z)| norm 0.2901 (+0.45z)| lr 2.98e-04 | 323.25 ms | 52.2% bf16 MFU | 1624189 tok/s
step 10173/19560 | loss 3.329122 (-2.02z)| norm 0.2792 (-0.11z)| lr 2.98e-04 | 323.06 ms | 52.2% bf16 MFU | 1624123 tok/s
step 10174/19560 | loss 3.446707 (+1.15z)| norm 0.2837 (+0.11z)| lr 2.98e-04 | 322.20 ms | 52.4% bf16 MFU | 1624278 tok/s
step 10175/19560 | loss 3.386755 (-0.47z)| norm 0.2623 (-1.00z)| lr 2.98e-04 | 322.67 ms | 52.3% bf16 MFU | 1624308 tok/s
step 10176/19560 | loss 3.432918 (+0.80z)| norm 0.2907 (+0.50z)| lr 2.98e-04 | 323.09 ms | 52.2% bf16 MFU | 1624229 tok/s
step 10177/19560 | loss 3.406979 (+0.08z)| norm 0.2642 (-0.89z)| lr 2.98e-04 | 323.05 ms | 52.2% bf16 MFU | 1624165 tok/s
step 10178/19560 | loss 3.394552 (-0.25z)| norm 0.2662 (-0.78z)| lr 2.98e-04 | 322.56 ms | 52.3% bf16 MFU | 1624226 tok/s
step 10179/19560 | loss 3.350427 (-1.44z)| norm 0.2804 (-0.03z)| lr 2.98e-04 | 322.68 ms | 52.3% bf16 MFU | 1624254 tok/s
step 10180/19560 | loss 3.476495 (+1.94z)| norm 0.2619 (-0.99z)| lr 2.98e-04 | 322.38 ms | 52.4% bf16 MFU | 1624357 tok/s
step 10181/19560 | loss 3.338516 (-1.72z)| norm 0.2847 (+0.20z)| lr 2.98e-04 | 322.82 ms | 52.3% bf16 MFU | 1624344 tok/s
step 10182/19560 | loss 3.345769 (-1.50z)| norm 0.2797 (-0.05z)| lr 2.97e-04 | 322.81 ms | 52.3% bf16 MFU | 1624334 tok/s
step 10183/19560 | loss 3.429076 (+0.68z)| norm 0.3119 (+1.60z)| lr 2.97e-04 | 322.34 ms | 52.4% bf16 MFU | 1624442 tok/s
step 10184/19560 | loss 3.439487 (+0.94z)| norm 0.2853 (+0.21z)| lr 2.97e-04 | 322.72 ms | 52.3% bf16 MFU | 1624449 tok/s
step 10185/19560 | loss 3.452711 (+1.27z)| norm 0.2759 (-0.28z)| lr 2.97e-04 | 323.22 ms | 52.2% bf16 MFU | 1624330 tok/s
step 10186/19560 | loss 3.512410 (+2.74z)| norm 0.2755 (-0.29z)| lr 2.97e-04 | 322.74 ms | 52.3% bf16 MFU | 1624337 tok/s
step 10187/19560 | loss 3.467059 (+1.56z)| norm 0.2897 (+0.46z)| lr 2.97e-04 | 322.91 ms | 52.3% bf16 MFU | 1624303 tok/s
step 10188/19560 | loss 3.368812 (-0.91z)| norm 0.2679 (-0.68z)| lr 2.97e-04 | 322.39 ms | 52.4% bf16 MFU | 1624401 tok/s
step 10189/19560 | loss 3.383230 (-0.53z)| norm 0.2950 (+0.78z)| lr 2.97e-04 | 322.56 ms | 52.3% bf16 MFU | 1624451 tok/s
step 10190/19560 | loss 3.333957 (-1.75z)| norm 0.2693 (-0.61z)| lr 2.97e-04 | 323.21 ms | 52.2% bf16 MFU | 1624334 tok/s
step 10191/19560 | loss 3.556389 (+3.63z)| norm 0.2750 (-0.30z)| lr 2.97e-04 | 322.91 ms | 52.3% bf16 MFU | 1624298 tok/s
step 10192/19560 | loss 3.401381 (-0.08z)| norm 0.2665 (-0.75z)| lr 2.97e-04 | 322.47 ms | 52.3% bf16 MFU | 1624376 tok/s
step 10193/19560 | loss 3.352057 (-1.25z)| norm 0.2891 (+0.48z)| lr 2.97e-04 | 323.01 ms | 52.2% bf16 MFU | 1624314 tok/s
step 10194/19560 | loss 3.382709 (-0.52z)| norm 0.2817 (+0.07z)| lr 2.97e-04 | 322.91 ms | 52.3% bf16 MFU | 1624280 tok/s
step 10195/19560 | loss 3.402284 (-0.06z)| norm 0.3100 (+1.69z)| lr 2.97e-04 | 322.34 ms | 52.4% bf16 MFU | 1624393 tok/s
step 10196/19560 | loss 3.345845 (-1.40z)| norm 0.2859 (+0.32z)| lr 2.97e-04 | 322.45 ms | 52.3% bf16 MFU | 1624471 tok/s
step 10197/19560 | loss 3.368101 (-0.87z)| norm 0.2908 (+0.61z)| lr 2.97e-04 | 323.64 ms | 52.1% bf16 MFU | 1624245 tok/s
step 10198/19560 | loss 3.423469 (+0.45z)| norm 0.2597 (-1.17z)| lr 2.97e-04 | 322.57 ms | 52.3% bf16 MFU | 1624300 tok/s
step 10199/19560 | loss 3.382286 (-0.54z)| norm 0.2863 (+0.38z)| lr 2.97e-04 | 322.86 ms | 52.3% bf16 MFU | 1624279 tok/s
step 10200/19560 | loss 3.361192 (-1.03z)| norm 0.2673 (-0.72z)| lr 2.97e-04 | 322.95 ms | 52.3% bf16 MFU | 1624237 tok/s
step 10201/19560 | loss 3.381669 (-0.55z)| norm 0.2819 (+0.15z)| lr 2.97e-04 | 322.69 ms | 52.3% bf16 MFU | 1624261 tok/s
step 10202/19560 | loss 3.386502 (-0.43z)| norm 0.2964 (+1.03z)| lr 2.96e-04 | 322.69 ms | 52.3% bf16 MFU | 1624285 tok/s
step 10203/19560 | loss 3.401790 (-0.06z)| norm 0.2767 (-0.16z)| lr 2.96e-04 | 323.12 ms | 52.2% bf16 MFU | 1624199 tok/s
step 10204/19560 | loss 3.440336 (+0.85z)| norm 0.2866 (+0.44z)| lr 2.96e-04 | 322.83 ms | 52.3% bf16 MFU | 1624191 tok/s
step 10205/19560 | loss 3.372369 (-0.75z)| norm 0.2873 (+0.48z)| lr 2.96e-04 | 323.24 ms | 52.2% bf16 MFU | 1624080 tok/s
step 10206/19560 | loss 3.403750 (+0.00z)| norm 0.3538 (+4.14z)| lr 2.96e-04 | 322.39 ms | 52.4% bf16 MFU | 1624188 tok/s
step 10207/19560 | loss 3.411956 (+0.21z)| norm 0.3621 (+4.26z)| lr 2.96e-04 | 322.55 ms | 52.3% bf16 MFU | 1624250 tok/s
step 10208/19560 | loss 3.418391 (+0.36z)| norm 0.2967 (+0.83z)| lr 2.96e-04 | 323.58 ms | 52.2% bf16 MFU | 1624052 tok/s
step 10209/19560 | loss 3.371027 (-0.78z)| norm 0.2754 (-0.28z)| lr 2.96e-04 | 322.42 ms | 52.3% bf16 MFU | 1624154 tok/s
step 10210/19560 | loss 3.410616 (+0.17z)| norm 0.2802 (-0.04z)| lr 2.96e-04 | 322.90 ms | 52.3% bf16 MFU | 1624130 tok/s
step 10211/19560 | loss 3.330790 (-1.72z)| norm 0.2757 (-0.29z)| lr 2.96e-04 | 323.16 ms | 52.2% bf16 MFU | 1624043 tok/s
step 10212/19560 | loss 3.367306 (-0.85z)| norm 0.2772 (-0.20z)| lr 2.96e-04 | 322.35 ms | 52.4% bf16 MFU | 1624163 tok/s
step 10213/19560 | loss 3.421704 (+0.43z)| norm 0.2501 (-1.62z)| lr 2.96e-04 | 322.83 ms | 52.3% bf16 MFU | 1624156 tok/s
step 10214/19560 | loss 3.348931 (-1.27z)| norm 0.2821 (+0.06z)| lr 2.96e-04 | 322.62 ms | 52.3% bf16 MFU | 1624203 tok/s
step 10215/19560 | loss 3.397424 (-0.12z)| norm 0.2660 (-0.78z)| lr 2.96e-04 | 323.33 ms | 52.2% bf16 MFU | 1624070 tok/s
step 10216/19560 | loss 3.403931 (+0.03z)| norm 0.2796 (-0.07z)| lr 2.96e-04 | 322.18 ms | 52.4% bf16 MFU | 1624234 tok/s
step 10217/19560 | loss 3.400023 (-0.06z)| norm 0.2667 (-0.74z)| lr 2.96e-04 | 323.46 ms | 52.2% bf16 MFU | 1624065 tok/s
step 10218/19560 | loss 3.371965 (-0.73z)| norm 0.2595 (-1.11z)| lr 2.96e-04 | 323.45 ms | 52.2% bf16 MFU | 1623908 tok/s
step 10219/19560 | loss 3.379685 (-0.54z)| norm 0.2829 (+0.12z)| lr 2.96e-04 | 322.35 ms | 52.4% bf16 MFU | 1624036 tok/s
step 10220/19560 | loss 3.367248 (-0.84z)| norm 0.2580 (-1.16z)| lr 2.96e-04 | 322.76 ms | 52.3% bf16 MFU | 1624054 tok/s
step 10221/19560 | loss 3.407696 (+0.13z)| norm 0.2825 (+0.11z)| lr 2.96e-04 | 322.96 ms | 52.3% bf16 MFU | 1624022 tok/s
step 10222/19560 | loss 3.396454 (-0.14z)| norm 0.2677 (-0.65z)| lr 2.95e-04 | 322.71 ms | 52.3% bf16 MFU | 1624053 tok/s
step 10223/19560 | loss 3.376299 (-0.62z)| norm 0.2538 (-1.35z)| lr 2.95e-04 | 322.76 ms | 52.3% bf16 MFU | 1624070 tok/s
step 10224/19560 | loss 3.418604 (+0.38z)| norm 0.2669 (-0.66z)| lr 2.95e-04 | 324.21 ms | 52.1% bf16 MFU | 1623722 tok/s
step 10225/19560 | loss 3.403007 (+0.01z)| norm 0.2884 (+0.49z)| lr 2.95e-04 | 322.13 ms | 52.4% bf16 MFU | 1623915 tok/s
step 10226/19560 | loss 3.403640 (+0.02z)| norm 0.2784 (-0.03z)| lr 2.95e-04 | 322.81 ms | 52.3% bf16 MFU | 1623927 tok/s
step 10227/19560 | loss 3.412499 (+0.23z)| norm 0.2753 (-0.19z)| lr 2.95e-04 | 323.12 ms | 52.2% bf16 MFU | 1623860 tok/s
step 10228/19560 | loss 3.423842 (+0.50z)| norm 0.2777 (-0.05z)| lr 2.95e-04 | 323.13 ms | 52.2% bf16 MFU | 1623794 tok/s
step 10229/19560 | loss 3.432160 (+0.70z)| norm 0.2664 (-0.68z)| lr 2.95e-04 | 322.46 ms | 52.3% bf16 MFU | 1623900 tok/s
step 10230/19560 | loss 3.408857 (+0.14z)| norm 0.2733 (-0.28z)| lr 2.95e-04 | 322.74 ms | 52.3% bf16 MFU | 1623928 tok/s
step 10231/19560 | loss 3.416287 (+0.31z)| norm 0.2675 (-0.60z)| lr 2.95e-04 | 323.30 ms | 52.2% bf16 MFU | 1623816 tok/s
step 10232/19560 | loss 3.433891 (+0.74z)| norm 0.2583 (-1.13z)| lr 2.95e-04 | 323.29 ms | 52.2% bf16 MFU | 1623712 tok/s
step 10233/19560 | loss 3.438338 (+0.83z)| norm 0.2664 (-0.67z)| lr 2.95e-04 | 323.18 ms | 52.2% bf16 MFU | 1623640 tok/s
step 10234/19560 | loss 3.425485 (+0.53z)| norm 0.2528 (-1.47z)| lr 2.95e-04 | 322.55 ms | 52.3% bf16 MFU | 1623731 tok/s
step 10235/19560 | loss 3.460366 (+1.36z)| norm 0.2622 (-0.90z)| lr 2.95e-04 | 323.19 ms | 52.2% bf16 MFU | 1623656 tok/s
step 10236/19560 | loss 3.436856 (+0.80z)| norm 0.2610 (-0.96z)| lr 2.95e-04 | 322.68 ms | 52.3% bf16 MFU | 1623713 tok/s
step 10237/19560 | loss 3.513681 (+2.57z)| norm 0.2893 (+0.75z)| lr 2.95e-04 | 322.46 ms | 52.3% bf16 MFU | 1623822 tok/s
step 10238/19560 | loss 3.415410 (+0.27z)| norm 0.2579 (-1.13z)| lr 2.95e-04 | 323.19 ms | 52.2% bf16 MFU | 1623741 tok/s
step 10239/19560 | loss 3.400225 (-0.08z)| norm 0.3091 (+1.91z)| lr 2.95e-04 | 322.98 ms | 52.3% bf16 MFU | 1623718 tok/s
step 10240/19560 | loss 3.402577 (+0.00z)| norm 0.2681 (-0.51z)| lr 2.95e-04 | 323.29 ms | 52.2% bf16 MFU | 1623618 tok/s
step 10241/19560 | loss 3.352726 (-1.23z)| norm 0.2844 (+0.46z)| lr 2.95e-04 | 322.93 ms | 52.3% bf16 MFU | 1623614 tok/s
step 10242/19560 | loss 3.541848 (+3.30z)| norm 0.2753 (-0.08z)| lr 2.94e-04 | 323.26 ms | 52.2% bf16 MFU | 1623527 tok/s
step 10243/19560 | loss 3.385368 (-0.42z)| norm 0.2681 (-0.51z)| lr 2.94e-04 | 323.27 ms | 52.2% bf16 MFU | 1623441 tok/s
step 10244/19560 | loss 3.367690 (-0.83z)| norm 0.2781 (+0.10z)| lr 2.94e-04 | 323.10 ms | 52.2% bf16 MFU | 1623403 tok/s
step 10245/19560 | loss 3.394134 (-0.20z)| norm 0.2792 (+0.16z)| lr 2.94e-04 | 323.23 ms | 52.2% bf16 MFU | 1623335 tok/s
step 10246/19560 | loss 3.359515 (-1.02z)| norm 0.2716 (-0.29z)| lr 2.94e-04 | 322.60 ms | 52.3% bf16 MFU | 1623428 tok/s
step 10247/19560 | loss 3.410253 (+0.19z)| norm 0.2696 (-0.40z)| lr 2.94e-04 | 322.72 ms | 52.3% bf16 MFU | 1623485 tok/s
step 10248/19560 | loss 3.382840 (-0.45z)| norm 0.2829 (+0.40z)| lr 2.94e-04 | 323.23 ms | 52.2% bf16 MFU | 1623412 tok/s
step 10249/19560 | loss 3.399061 (-0.07z)| norm 0.2847 (+0.51z)| lr 2.94e-04 | 323.08 ms | 52.2% bf16 MFU | 1623382 tok/s
step 10250/19560 | loss 3.361068 (-0.99z)| norm 0.3005 (+1.44z)| lr 2.94e-04 | 323.04 ms | 52.2% bf16 MFU | 1623362 tok/s
val loss 3.391018
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2933/10042 = 0.292073
step 10251/19560 | loss 3.392031 (-0.26z)| norm 0.3107 (+2.00z)| lr 2.94e-04 | 322.85 ms | 52.3% bf16 MFU | 1623392 tok/s
step 10252/19560 | loss 3.388530 (-0.33z)| norm 0.3287 (+2.94z)| lr 2.94e-04 | 322.70 ms | 52.3% bf16 MFU | 1623458 tok/s
step 10253/19560 | loss 3.535133 (+3.11z)| norm 0.3302 (+2.91z)| lr 2.94e-04 | 323.48 ms | 52.2% bf16 MFU | 1623324 tok/s
step 10254/19560 | loss 3.453739 (+1.17z)| norm 0.3181 (+2.18z)| lr 2.94e-04 | 323.48 ms | 52.2% bf16 MFU | 1623196 tok/s
step 10255/19560 | loss 3.399994 (-0.09z)| norm 0.3077 (+1.58z)| lr 2.94e-04 | 322.99 ms | 52.3% bf16 MFU | 1623198 tok/s
step 10256/19560 | loss 3.424707 (+0.49z)| norm 0.2824 (+0.22z)| lr 2.94e-04 | 322.49 ms | 52.3% bf16 MFU | 1623326 tok/s
step 10257/19560 | loss 3.381806 (-0.52z)| norm 0.3593 (+4.04z)| lr 2.94e-04 | 323.50 ms | 52.2% bf16 MFU | 1623193 tok/s
step 10258/19560 | loss 3.324053 (-1.86z)| norm 0.2804 (+0.08z)| lr 2.94e-04 | 322.41 ms | 52.3% bf16 MFU | 1623340 tok/s
step 10259/19560 | loss 3.406758 (+0.07z)| norm 0.3127 (+1.67z)| lr 2.94e-04 | 322.55 ms | 52.3% bf16 MFU | 1623444 tok/s
step 10260/19560 | loss 3.429786 (+0.60z)| norm 0.2992 (+0.98z)| lr 2.94e-04 | 322.05 ms | 52.4% bf16 MFU | 1623672 tok/s
step 10261/19560 | loss 3.392919 (-0.25z)| norm 0.2983 (+0.92z)| lr 2.94e-04 | 323.13 ms | 52.2% bf16 MFU | 1623615 tok/s
step 10262/19560 | loss 3.466471 (+1.44z)| norm 0.3024 (+1.11z)| lr 2.93e-04 | 323.28 ms | 52.2% bf16 MFU | 1623523 tok/s
step 10263/19560 | loss 3.338270 (-1.53z)| norm 0.2990 (+0.93z)| lr 2.93e-04 | 323.05 ms | 52.2% bf16 MFU | 1623493 tok/s
step 10264/19560 | loss 3.383070 (-0.50z)| norm 0.2754 (-0.24z)| lr 2.93e-04 | 322.49 ms | 52.3% bf16 MFU | 1623607 tok/s
step 10265/19560 | loss 3.351059 (-1.25z)| norm 0.3098 (+1.44z)| lr 2.93e-04 | 323.29 ms | 52.2% bf16 MFU | 1623513 tok/s
step 10266/19560 | loss 3.342610 (-1.42z)| norm 0.2735 (-0.34z)| lr 2.93e-04 | 323.28 ms | 52.2% bf16 MFU | 1623427 tok/s
step 10267/19560 | loss 3.408967 (+0.13z)| norm 0.3107 (+1.46z)| lr 2.93e-04 | 322.82 ms | 52.3% bf16 MFU | 1623460 tok/s
step 10268/19560 | loss 3.417200 (+0.32z)| norm 0.2811 (+0.01z)| lr 2.93e-04 | 323.23 ms | 52.2% bf16 MFU | 1623390 tok/s
step 10269/19560 | loss 3.385556 (-0.42z)| norm 0.3543 (+3.40z)| lr 2.93e-04 | 323.01 ms | 52.3% bf16 MFU | 1623378 tok/s
step 10270/19560 | loss 3.393845 (-0.23z)| norm 0.3200 (+1.78z)| lr 2.93e-04 | 322.95 ms | 52.3% bf16 MFU | 1623381 tok/s
step 10271/19560 | loss 3.323777 (-1.85z)| norm 0.3798 (+4.19z)| lr 2.93e-04 | 322.73 ms | 52.3% bf16 MFU | 1623438 tok/s
step 10272/19560 | loss 3.448701 (+1.07z)| norm 0.3256 (+1.82z)| lr 2.93e-04 | 322.90 ms | 52.3% bf16 MFU | 1623449 tok/s
step 10273/19560 | loss 3.371027 (-0.76z)| norm 0.3336 (+2.10z)| lr 2.93e-04 | 322.57 ms | 52.3% bf16 MFU | 1623543 tok/s
step 10274/19560 | loss 3.405609 (+0.06z)| norm 0.3236 (+1.65z)| lr 2.93e-04 | 322.77 ms | 52.3% bf16 MFU | 1623584 tok/s
step 10275/19560 | loss 3.400434 (-0.06z)| norm 0.2989 (+0.62z)| lr 2.93e-04 | 323.07 ms | 52.2% bf16 MFU | 1623545 tok/s
step 10276/19560 | loss 3.460319 (+1.36z)| norm 0.3689 (+3.34z)| lr 2.93e-04 | 322.79 ms | 52.3% bf16 MFU | 1623579 tok/s
step 10277/19560 | loss 3.452221 (+1.16z)| norm 0.3199 (+1.37z)| lr 2.93e-04 | 322.54 ms | 52.3% bf16 MFU | 1623674 tok/s
step 10278/19560 | loss 3.425927 (+0.54z)| norm 0.3186 (+1.30z)| lr 2.93e-04 | 322.78 ms | 52.3% bf16 MFU | 1623704 tok/s
step 10279/19560 | loss 3.350157 (-1.24z)| norm 0.2995 (+0.54z)| lr 2.93e-04 | 322.74 ms | 52.3% bf16 MFU | 1623744 tok/s
step 10280/19560 | loss 3.414237 (+0.28z)| norm 0.3004 (+0.57z)| lr 2.93e-04 | 322.98 ms | 52.3% bf16 MFU | 1623720 tok/s
step 10281/19560 | loss 3.412311 (+0.23z)| norm 0.3060 (+0.77z)| lr 2.93e-04 | 323.23 ms | 52.2% bf16 MFU | 1623637 tok/s
step 10282/19560 | loss 3.446305 (+1.02z)| norm 0.2822 (-0.17z)| lr 2.92e-04 | 322.70 ms | 52.3% bf16 MFU | 1623690 tok/s
step 10283/19560 | loss 3.420545 (+0.41z)| norm 0.2922 (+0.22z)| lr 2.92e-04 | 322.90 ms | 52.3% bf16 MFU | 1623688 tok/s
step 10284/19560 | loss 3.372627 (-0.71z)| norm 0.2774 (-0.38z)| lr 2.92e-04 | 322.86 ms | 52.3% bf16 MFU | 1623697 tok/s
step 10285/19560 | loss 3.395025 (-0.18z)| norm 0.2926 (+0.22z)| lr 2.92e-04 | 322.90 ms | 52.3% bf16 MFU | 1623698 tok/s
step 10286/19560 | loss 3.477884 (+1.73z)| norm 0.2920 (+0.19z)| lr 2.92e-04 | 322.89 ms | 52.3% bf16 MFU | 1623698 tok/s
step 10287/19560 | loss 3.331223 (-1.65z)| norm 0.2737 (-0.54z)| lr 2.92e-04 | 322.90 ms | 52.3% bf16 MFU | 1623697 tok/s
step 10288/19560 | loss 3.383301 (-0.45z)| norm 0.2820 (-0.21z)| lr 2.92e-04 | 322.94 ms | 52.3% bf16 MFU | 1623685 tok/s
step 10289/19560 | loss 3.328901 (-1.68z)| norm 0.2648 (-0.90z)| lr 2.92e-04 | 322.61 ms | 52.3% bf16 MFU | 1623759 tok/s
step 10290/19560 | loss 3.423574 (+0.49z)| norm 0.2545 (-1.31z)| lr 2.92e-04 | 323.07 ms | 52.2% bf16 MFU | 1623714 tok/s
step 10291/19560 | loss 3.433373 (+0.71z)| norm 0.2625 (-0.99z)| lr 2.92e-04 | 322.63 ms | 52.3% bf16 MFU | 1623780 tok/s
step 10292/19560 | loss 3.420125 (+0.40z)| norm 0.2567 (-1.24z)| lr 2.92e-04 | 323.00 ms | 52.3% bf16 MFU | 1623751 tok/s
step 10293/19560 | loss 3.357872 (-1.02z)| norm 0.2709 (-0.67z)| lr 2.92e-04 | 322.56 ms | 52.3% bf16 MFU | 1623833 tok/s
step 10294/19560 | loss 3.453434 (+1.16z)| norm 0.2678 (-0.79z)| lr 2.92e-04 | 322.63 ms | 52.3% bf16 MFU | 1623894 tok/s
step 10295/19560 | loss 3.351514 (-1.15z)| norm 0.2536 (-1.35z)| lr 2.92e-04 | 323.14 ms | 52.2% bf16 MFU | 1623824 tok/s
step 10296/19560 | loss 3.382116 (-0.45z)| norm 0.2529 (-1.36z)| lr 2.92e-04 | 322.72 ms | 52.3% bf16 MFU | 1623862 tok/s
step 10297/19560 | loss 3.408166 (+0.14z)| norm 0.2478 (-1.54z)| lr 2.92e-04 | 322.72 ms | 52.3% bf16 MFU | 1623898 tok/s
step 10298/19560 | loss 3.419576 (+0.40z)| norm 0.2595 (-1.06z)| lr 2.92e-04 | 323.03 ms | 52.2% bf16 MFU | 1623855 tok/s
step 10299/19560 | loss 3.384824 (-0.39z)| norm 0.2439 (-1.64z)| lr 2.92e-04 | 322.84 ms | 52.3% bf16 MFU | 1623861 tok/s
step 10300/19560 | loss 3.365147 (-0.85z)| norm 0.2605 (-0.98z)| lr 2.92e-04 | 322.62 ms | 52.3% bf16 MFU | 1623922 tok/s
step 10301/19560 | loss 3.427049 (+0.56z)| norm 0.2461 (-1.51z)| lr 2.92e-04 | 322.71 ms | 52.3% bf16 MFU | 1623958 tok/s
step 10302/19560 | loss 3.351948 (-1.16z)| norm 0.2758 (-0.37z)| lr 2.91e-04 | 323.38 ms | 52.2% bf16 MFU | 1623825 tok/s
step 10303/19560 | loss 3.402617 (+0.01z)| norm 0.2555 (-1.15z)| lr 2.91e-04 | 323.12 ms | 52.2% bf16 MFU | 1623764 tok/s
step 10304/19560 | loss 3.343785 (-1.33z)| norm 0.2619 (-0.89z)| lr 2.91e-04 | 322.76 ms | 52.3% bf16 MFU | 1623795 tok/s
step 10305/19560 | loss 3.404347 (+0.06z)| norm 0.2740 (-0.43z)| lr 2.91e-04 | 322.74 ms | 52.3% bf16 MFU | 1623829 tok/s
step 10306/19560 | loss 3.470975 (+1.56z)| norm 0.2622 (-0.88z)| lr 2.91e-04 | 323.47 ms | 52.2% bf16 MFU | 1623679 tok/s
step 10307/19560 | loss 3.511176 (+2.41z)| norm 0.2779 (-0.28z)| lr 2.91e-04 | 323.12 ms | 52.2% bf16 MFU | 1623624 tok/s
step 10308/19560 | loss 3.396020 (-0.15z)| norm 0.2496 (-1.35z)| lr 2.91e-04 | 323.12 ms | 52.2% bf16 MFU | 1623573 tok/s
step 10309/19560 | loss 3.420400 (+0.39z)| norm 0.2815 (-0.13z)| lr 2.91e-04 | 322.50 ms | 52.3% bf16 MFU | 1623679 tok/s
step 10310/19560 | loss 3.422299 (+0.42z)| norm 0.2518 (-1.25z)| lr 2.91e-04 | 322.59 ms | 52.3% bf16 MFU | 1623758 tok/s
step 10311/19560 | loss 3.387854 (-0.36z)| norm 0.3028 (+0.69z)| lr 2.91e-04 | 323.35 ms | 52.2% bf16 MFU | 1623642 tok/s
step 10312/19560 | loss 3.532535 (+2.84z)| norm 0.2527 (-1.20z)| lr 2.91e-04 | 322.99 ms | 52.3% bf16 MFU | 1623621 tok/s
step 10313/19560 | loss 3.453920 (+1.10z)| norm 0.2799 (-0.17z)| lr 2.91e-04 | 323.03 ms | 52.2% bf16 MFU | 1623592 tok/s
step 10314/19560 | loss 3.404608 (+0.02z)| norm 0.2610 (-0.88z)| lr 2.91e-04 | 323.15 ms | 52.2% bf16 MFU | 1623534 tok/s
step 10315/19560 | loss 3.425292 (+0.50z)| norm 0.2869 (+0.10z)| lr 2.91e-04 | 322.82 ms | 52.3% bf16 MFU | 1623563 tok/s
step 10316/19560 | loss 3.378580 (-0.57z)| norm 0.2883 (+0.15z)| lr 2.91e-04 | 322.52 ms | 52.3% bf16 MFU | 1623665 tok/s
step 10317/19560 | loss 3.378691 (-0.56z)| norm 0.2638 (-0.77z)| lr 2.91e-04 | 323.65 ms | 52.1% bf16 MFU | 1623478 tok/s
step 10318/19560 | loss 3.411563 (+0.18z)| norm 0.3367 (+1.94z)| lr 2.91e-04 | 322.80 ms | 52.3% bf16 MFU | 1623513 tok/s
step 10319/19560 | loss 3.579107 (+3.99z)| norm 0.2974 (+0.46z)| lr 2.91e-04 | 322.43 ms | 52.3% bf16 MFU | 1623640 tok/s
step 10320/19560 | loss 3.439759 (+0.80z)| norm 0.2973 (+0.45z)| lr 2.91e-04 | 323.02 ms | 52.2% bf16 MFU | 1623612 tok/s
step 10321/19560 | loss 3.409502 (+0.11z)| norm 0.3157 (+1.13z)| lr 2.91e-04 | 323.53 ms | 52.2% bf16 MFU | 1623457 tok/s
step 10322/19560 | loss 3.512889 (+2.39z)| norm 0.2808 (-0.17z)| lr 2.90e-04 | 322.25 ms | 52.4% bf16 MFU | 1623632 tok/s
step 10323/19560 | loss 3.519967 (+2.47z)| norm 0.3156 (+1.12z)| lr 2.90e-04 | 322.73 ms | 52.3% bf16 MFU | 1623676 tok/s
step 10324/19560 | loss 3.460085 (+1.14z)| norm 0.2943 (+0.33z)| lr 2.90e-04 | 323.12 ms | 52.2% bf16 MFU | 1623620 tok/s
step 10325/19560 | loss 3.383724 (-0.53z)| norm 0.2958 (+0.38z)| lr 2.90e-04 | 322.59 ms | 52.3% bf16 MFU | 1623702 tok/s
step 10326/19560 | loss 3.459568 (+1.12z)| norm 0.2996 (+0.51z)| lr 2.90e-04 | 323.62 ms | 52.2% bf16 MFU | 1623520 tok/s
step 10327/19560 | loss 3.383298 (-0.54z)| norm 0.3276 (+1.53z)| lr 2.90e-04 | 323.00 ms | 52.3% bf16 MFU | 1623503 tok/s
step 10328/19560 | loss 3.325290 (-1.78z)| norm 0.3196 (+1.21z)| lr 2.90e-04 | 322.97 ms | 52.3% bf16 MFU | 1623496 tok/s
step 10329/19560 | loss 3.509674 (+2.14z)| norm 0.3150 (+1.03z)| lr 2.90e-04 | 322.52 ms | 52.3% bf16 MFU | 1623600 tok/s
step 10330/19560 | loss 3.437422 (+0.60z)| norm 0.3290 (+1.52z)| lr 2.90e-04 | 322.59 ms | 52.3% bf16 MFU | 1623682 tok/s
step 10331/19560 | loss 3.369223 (-0.84z)| norm 0.3117 (+0.88z)| lr 2.90e-04 | 322.76 ms | 52.3% bf16 MFU | 1623718 tok/s
step 10332/19560 | loss 3.349545 (-1.24z)| norm 0.2805 (-0.24z)| lr 2.90e-04 | 323.31 ms | 52.2% bf16 MFU | 1623612 tok/s
step 10333/19560 | loss 3.398796 (-0.20z)| norm 0.2886 (+0.05z)| lr 2.90e-04 | 322.19 ms | 52.4% bf16 MFU | 1623796 tok/s
step 10334/19560 | loss 3.420596 (+0.25z)| norm 0.2954 (+0.31z)| lr 2.90e-04 | 322.76 ms | 52.3% bf16 MFU | 1623826 tok/s
step 10335/19560 | loss 3.376471 (-0.67z)| norm 0.3391 (+1.97z)| lr 2.90e-04 | 322.75 ms | 52.3% bf16 MFU | 1623857 tok/s
step 10336/19560 | loss 3.376290 (-0.67z)| norm 0.2627 (-0.89z)| lr 2.90e-04 | 322.75 ms | 52.3% bf16 MFU | 1623888 tok/s
step 10337/19560 | loss 3.410424 (+0.04z)| norm 0.2803 (-0.23z)| lr 2.90e-04 | 323.01 ms | 52.2% bf16 MFU | 1623848 tok/s
step 10338/19560 | loss 3.381178 (-0.57z)| norm 0.2938 (+0.27z)| lr 2.90e-04 | 322.93 ms | 52.3% bf16 MFU | 1623834 tok/s
step 10339/19560 | loss 3.437498 (+0.61z)| norm 0.2798 (-0.25z)| lr 2.90e-04 | 323.16 ms | 52.2% bf16 MFU | 1623762 tok/s
step 10340/19560 | loss 3.420348 (+0.24z)| norm 0.3010 (+0.53z)| lr 2.90e-04 | 322.56 ms | 52.3% bf16 MFU | 1623844 tok/s
step 10341/19560 | loss 3.387137 (-0.47z)| norm 0.2448 (-1.57z)| lr 2.90e-04 | 322.72 ms | 52.3% bf16 MFU | 1623881 tok/s
step 10342/19560 | loss 3.375831 (-0.72z)| norm 0.3215 (+1.28z)| lr 2.89e-04 | 322.71 ms | 52.3% bf16 MFU | 1623918 tok/s
step 10343/19560 | loss 3.397388 (-0.25z)| norm 0.2816 (-0.21z)| lr 2.89e-04 | 322.95 ms | 52.3% bf16 MFU | 1623893 tok/s
step 10344/19560 | loss 3.391769 (-0.37z)| norm 0.2754 (-0.44z)| lr 2.89e-04 | 322.28 ms | 52.4% bf16 MFU | 1624039 tok/s
step 10345/19560 | loss 3.383533 (-0.55z)| norm 0.2889 (+0.06z)| lr 2.89e-04 | 322.69 ms | 52.3% bf16 MFU | 1624073 tok/s
step 10346/19560 | loss 3.332181 (-1.62z)| norm 0.2534 (-1.26z)| lr 2.89e-04 | 323.16 ms | 52.2% bf16 MFU | 1623987 tok/s
step 10347/19560 | loss 3.413492 (+0.10z)| norm 0.2869 (-0.01z)| lr 2.89e-04 | 322.46 ms | 52.3% bf16 MFU | 1624082 tok/s
step 10348/19560 | loss 3.370379 (-0.82z)| norm 0.2846 (-0.11z)| lr 2.89e-04 | 322.91 ms | 52.3% bf16 MFU | 1624058 tok/s
step 10349/19560 | loss 3.343875 (-1.36z)| norm 0.2885 (+0.04z)| lr 2.89e-04 | 322.70 ms | 52.3% bf16 MFU | 1624091 tok/s
step 10350/19560 | loss 3.429615 (+0.44z)| norm 0.2699 (-0.66z)| lr 2.89e-04 | 322.52 ms | 52.3% bf16 MFU | 1624166 tok/s
step 10351/19560 | loss 3.381390 (-0.58z)| norm 0.3030 (+0.56z)| lr 2.89e-04 | 322.66 ms | 52.3% bf16 MFU | 1624203 tok/s
step 10352/19560 | loss 3.326123 (-1.71z)| norm 0.2647 (-0.87z)| lr 2.89e-04 | 322.55 ms | 52.3% bf16 MFU | 1624264 tok/s
step 10353/19560 | loss 3.393462 (-0.30z)| norm 0.2809 (-0.26z)| lr 2.89e-04 | 322.73 ms | 52.3% bf16 MFU | 1624278 tok/s
step 10354/19560 | loss 3.401007 (-0.15z)| norm 0.2795 (-0.31z)| lr 2.89e-04 | 322.38 ms | 52.4% bf16 MFU | 1624379 tok/s
step 10355/19560 | loss 3.376009 (-0.66z)| norm 0.2827 (-0.19z)| lr 2.89e-04 | 322.29 ms | 52.4% bf16 MFU | 1624498 tok/s
step 10356/19560 | loss 3.374834 (-0.68z)| norm 0.2722 (-0.59z)| lr 2.89e-04 | 322.66 ms | 52.3% bf16 MFU | 1624518 tok/s
step 10357/19560 | loss 3.435853 (+0.59z)| norm 0.2917 (+0.14z)| lr 2.89e-04 | 322.97 ms | 52.3% bf16 MFU | 1624458 tok/s
step 10358/19560 | loss 3.462766 (+1.14z)| norm 0.2655 (-0.85z)| lr 2.89e-04 | 322.67 ms | 52.3% bf16 MFU | 1624478 tok/s
step 10359/19560 | loss 3.363037 (-0.91z)| norm 0.2807 (-0.28z)| lr 2.89e-04 | 322.40 ms | 52.3% bf16 MFU | 1624565 tok/s
step 10360/19560 | loss 3.346876 (-1.23z)| norm 0.2609 (-1.03z)| lr 2.89e-04 | 322.86 ms | 52.3% bf16 MFU | 1624530 tok/s
step 10361/19560 | loss 3.395375 (-0.23z)| norm 0.2697 (-0.69z)| lr 2.89e-04 | 323.02 ms | 52.2% bf16 MFU | 1624456 tok/s
step 10362/19560 | loss 3.409004 (+0.06z)| norm 0.2953 (+0.26z)| lr 2.88e-04 | 321.72 ms | 52.5% bf16 MFU | 1624716 tok/s
step 10363/19560 | loss 3.305111 (-2.03z)| norm 0.2956 (+0.26z)| lr 2.88e-04 | 322.98 ms | 52.3% bf16 MFU | 1624644 tok/s
step 10364/19560 | loss 3.360792 (-0.88z)| norm 0.2778 (-0.43z)| lr 2.88e-04 | 322.91 ms | 52.3% bf16 MFU | 1624594 tok/s
step 10365/19560 | loss 3.421930 (+0.38z)| norm 0.3005 (+0.44z)| lr 2.88e-04 | 322.59 ms | 52.3% bf16 MFU | 1624627 tok/s
step 10366/19560 | loss 3.415609 (+0.25z)| norm 0.2658 (-0.89z)| lr 2.88e-04 | 322.76 ms | 52.3% bf16 MFU | 1624615 tok/s
step 10367/19560 | loss 3.389968 (-0.28z)| norm 0.2797 (-0.35z)| lr 2.88e-04 | 322.52 ms | 52.3% bf16 MFU | 1624666 tok/s
step 10368/19560 | loss 3.421594 (+0.37z)| norm 0.2594 (-1.12z)| lr 2.88e-04 | 322.99 ms | 52.3% bf16 MFU | 1624594 tok/s
step 10369/19560 | loss 3.323472 (-1.64z)| norm 0.2624 (-1.00z)| lr 2.88e-04 | 323.07 ms | 52.2% bf16 MFU | 1624505 tok/s
step 10370/19560 | loss 3.390385 (-0.25z)| norm 0.2701 (-0.70z)| lr 2.88e-04 | 322.77 ms | 52.3% bf16 MFU | 1624496 tok/s
step 10371/19560 | loss 3.335990 (-1.39z)| norm 0.2608 (-1.05z)| lr 2.88e-04 | 322.33 ms | 52.4% bf16 MFU | 1624600 tok/s
step 10372/19560 | loss 3.352979 (-1.03z)| norm 0.2808 (-0.29z)| lr 2.88e-04 | 323.09 ms | 52.2% bf16 MFU | 1624506 tok/s
step 10373/19560 | loss 3.449191 (+0.98z)| norm 0.2916 (+0.11z)| lr 2.88e-04 | 323.26 ms | 52.2% bf16 MFU | 1624374 tok/s
step 10374/19560 | loss 3.386197 (-0.34z)| norm 0.2927 (+0.15z)| lr 2.88e-04 | 322.47 ms | 52.3% bf16 MFU | 1624447 tok/s
step 10375/19560 | loss 3.409414 (+0.15z)| norm 0.2628 (-0.98z)| lr 2.88e-04 | 322.92 ms | 52.3% bf16 MFU | 1624403 tok/s
step 10376/19560 | loss 3.420921 (+0.38z)| norm 0.2738 (-0.56z)| lr 2.88e-04 | 322.60 ms | 52.3% bf16 MFU | 1624441 tok/s
step 10377/19560 | loss 3.351826 (-1.06z)| norm 0.2722 (-0.62z)| lr 2.88e-04 | 322.82 ms | 52.3% bf16 MFU | 1624424 tok/s
step 10378/19560 | loss 3.383608 (-0.40z)| norm 0.2567 (-1.19z)| lr 2.88e-04 | 322.58 ms | 52.3% bf16 MFU | 1624467 tok/s
step 10379/19560 | loss 3.370326 (-0.67z)| norm 0.2521 (-1.34z)| lr 2.88e-04 | 323.13 ms | 52.2% bf16 MFU | 1624369 tok/s
step 10380/19560 | loss 3.405961 (+0.07z)| norm 0.2622 (-0.95z)| lr 2.88e-04 | 322.60 ms | 52.3% bf16 MFU | 1624412 tok/s
step 10381/19560 | loss 3.420540 (+0.41z)| norm 0.2632 (-0.89z)| lr 2.88e-04 | 322.67 ms | 52.3% bf16 MFU | 1624434 tok/s
step 10382/19560 | loss 3.418366 (+0.37z)| norm 0.2749 (-0.44z)| lr 2.87e-04 | 323.09 ms | 52.2% bf16 MFU | 1624349 tok/s
step 10383/19560 | loss 3.386736 (-0.31z)| norm 0.2628 (-0.89z)| lr 2.87e-04 | 322.42 ms | 52.3% bf16 MFU | 1624436 tok/s
step 10384/19560 | loss 3.442666 (+0.89z)| norm 0.3039 (+0.68z)| lr 2.87e-04 | 322.49 ms | 52.3% bf16 MFU | 1624501 tok/s
step 10385/19560 | loss 3.434072 (+0.70z)| norm 0.2836 (-0.08z)| lr 2.87e-04 | 323.10 ms | 52.2% bf16 MFU | 1624411 tok/s
step 10386/19560 | loss 3.433186 (+0.66z)| norm 0.3017 (+0.63z)| lr 2.87e-04 | 322.97 ms | 52.3% bf16 MFU | 1624357 tok/s
step 10387/19560 | loss 3.351513 (-1.10z)| norm 0.2992 (+0.54z)| lr 2.87e-04 | 322.27 ms | 52.4% bf16 MFU | 1624482 tok/s
step 10388/19560 | loss 3.374281 (-0.60z)| norm 0.2935 (+0.31z)| lr 2.87e-04 | 322.97 ms | 52.3% bf16 MFU | 1624424 tok/s
step 10389/19560 | loss 3.478691 (+1.64z)| norm 0.2975 (+0.47z)| lr 2.87e-04 | 322.88 ms | 52.3% bf16 MFU | 1624391 tok/s
step 10390/19560 | loss 3.347431 (-1.16z)| norm 0.2957 (+0.40z)| lr 2.87e-04 | 323.05 ms | 52.2% bf16 MFU | 1624317 tok/s
step 10391/19560 | loss 3.368057 (-0.73z)| norm 0.3108 (+0.99z)| lr 2.87e-04 | 322.87 ms | 52.3% bf16 MFU | 1624294 tok/s
step 10392/19560 | loss 3.404609 (+0.06z)| norm 0.2683 (-0.68z)| lr 2.87e-04 | 322.69 ms | 52.3% bf16 MFU | 1624316 tok/s
step 10393/19560 | loss 3.428813 (+0.57z)| norm 0.2784 (-0.27z)| lr 2.87e-04 | 322.95 ms | 52.3% bf16 MFU | 1624272 tok/s
step 10394/19560 | loss 3.411651 (+0.19z)| norm 0.2597 (-1.01z)| lr 2.87e-04 | 323.69 ms | 52.1% bf16 MFU | 1624045 tok/s
step 10395/19560 | loss 3.395932 (-0.15z)| norm 0.2646 (-0.80z)| lr 2.87e-04 | 322.78 ms | 52.3% bf16 MFU | 1624057 tok/s
step 10396/19560 | loss 3.359146 (-0.95z)| norm 0.2532 (-1.23z)| lr 2.87e-04 | 322.90 ms | 52.3% bf16 MFU | 1624039 tok/s
step 10397/19560 | loss 3.448559 (+0.99z)| norm 0.2614 (-0.91z)| lr 2.87e-04 | 323.03 ms | 52.2% bf16 MFU | 1623990 tok/s
step 10398/19560 | loss 3.446652 (+0.94z)| norm 0.2631 (-0.83z)| lr 2.87e-04 | 322.87 ms | 52.3% bf16 MFU | 1623982 tok/s
step 10399/19560 | loss 3.462704 (+1.27z)| norm 0.2617 (-0.90z)| lr 2.87e-04 | 323.17 ms | 52.2% bf16 MFU | 1623898 tok/s
step 10400/19560 | loss 3.429826 (+0.55z)| norm 0.2639 (-0.79z)| lr 2.87e-04 | 323.59 ms | 52.2% bf16 MFU | 1623714 tok/s
step 10401/19560 | loss 3.402732 (-0.04z)| norm 0.2514 (-1.33z)| lr 2.87e-04 | 323.49 ms | 52.2% bf16 MFU | 1623564 tok/s
step 10402/19560 | loss 3.376420 (-0.61z)| norm 0.2776 (-0.16z)| lr 2.86e-04 | 322.46 ms | 52.3% bf16 MFU | 1623680 tok/s
step 10403/19560 | loss 3.397403 (-0.15z)| norm 0.2652 (-0.70z)| lr 2.86e-04 | 322.39 ms | 52.4% bf16 MFU | 1623809 tok/s
step 10404/19560 | loss 3.394708 (-0.20z)| norm 0.2719 (-0.39z)| lr 2.86e-04 | 323.91 ms | 52.1% bf16 MFU | 1623549 tok/s
step 10405/19560 | loss 3.343357 (-1.31z)| norm 0.2573 (-1.07z)| lr 2.86e-04 | 322.71 ms | 52.3% bf16 MFU | 1623604 tok/s
step 10406/19560 | loss 3.410275 (+0.16z)| norm 0.2821 (+0.14z)| lr 2.86e-04 | 323.07 ms | 52.2% bf16 MFU | 1623565 tok/s
step 10407/19560 | loss 3.388117 (-0.33z)| norm 0.2661 (-0.63z)| lr 2.86e-04 | 323.12 ms | 52.2% bf16 MFU | 1623517 tok/s
step 10408/19560 | loss 3.402112 (-0.02z)| norm 0.2934 (+0.71z)| lr 2.86e-04 | 323.05 ms | 52.2% bf16 MFU | 1623488 tok/s
step 10409/19560 | loss 3.362937 (-0.87z)| norm 0.2612 (-0.86z)| lr 2.86e-04 | 322.93 ms | 52.3% bf16 MFU | 1623490 tok/s
step 10410/19560 | loss 3.410594 (+0.18z)| norm 0.2935 (+0.73z)| lr 2.86e-04 | 323.29 ms | 52.2% bf16 MFU | 1623402 tok/s
step 10411/19560 | loss 3.342582 (-1.30z)| norm 0.2819 (+0.16z)| lr 2.86e-04 | 322.97 ms | 52.3% bf16 MFU | 1623398 tok/s
step 10412/19560 | loss 3.391511 (-0.23z)| norm 0.2721 (-0.32z)| lr 2.86e-04 | 322.92 ms | 52.3% bf16 MFU | 1623406 tok/s
step 10413/19560 | loss 3.411507 (+0.21z)| norm 0.2828 (+0.21z)| lr 2.86e-04 | 323.40 ms | 52.2% bf16 MFU | 1623294 tok/s
step 10414/19560 | loss 3.424620 (+0.51z)| norm 0.2589 (-0.95z)| lr 2.86e-04 | 322.98 ms | 52.3% bf16 MFU | 1623293 tok/s
step 10415/19560 | loss 3.359536 (-0.95z)| norm 0.2617 (-0.81z)| lr 2.86e-04 | 323.18 ms | 52.2% bf16 MFU | 1623241 tok/s
step 10416/19560 | loss 3.395655 (-0.14z)| norm 0.2880 (+0.48z)| lr 2.86e-04 | 323.11 ms | 52.2% bf16 MFU | 1623210 tok/s
step 10417/19560 | loss 3.399389 (-0.07z)| norm 0.2802 (+0.09z)| lr 2.86e-04 | 322.78 ms | 52.3% bf16 MFU | 1623263 tok/s
step 10418/19560 | loss 3.398056 (-0.10z)| norm 0.2768 (-0.09z)| lr 2.86e-04 | 322.75 ms | 52.3% bf16 MFU | 1623321 tok/s
step 10419/19560 | loss 3.399356 (-0.06z)| norm 0.2783 (-0.02z)| lr 2.86e-04 | 323.64 ms | 52.1% bf16 MFU | 1623154 tok/s
step 10420/19560 | loss 3.383693 (-0.41z)| norm 0.2608 (-0.89z)| lr 2.86e-04 | 323.56 ms | 52.2% bf16 MFU | 1623014 tok/s
step 10421/19560 | loss 3.404852 (+0.06z)| norm 0.2590 (-0.97z)| lr 2.86e-04 | 323.21 ms | 52.2% bf16 MFU | 1622970 tok/s
step 10422/19560 | loss 3.352409 (-1.11z)| norm 0.2632 (-0.76z)| lr 2.85e-04 | 322.95 ms | 52.3% bf16 MFU | 1622993 tok/s
step 10423/19560 | loss 3.395986 (-0.13z)| norm 0.2634 (-0.76z)| lr 2.85e-04 | 323.35 ms | 52.2% bf16 MFU | 1622914 tok/s
step 10424/19560 | loss 3.390321 (-0.26z)| norm 0.2656 (-0.66z)| lr 2.85e-04 | 322.53 ms | 52.3% bf16 MFU | 1623045 tok/s
step 10425/19560 | loss 3.407266 (+0.13z)| norm 0.2692 (-0.49z)| lr 2.85e-04 | 322.27 ms | 52.4% bf16 MFU | 1623237 tok/s
step 10426/19560 | loss 3.401907 (+0.01z)| norm 0.2776 (-0.07z)| lr 2.85e-04 | 323.83 ms | 52.1% bf16 MFU | 1623027 tok/s
step 10427/19560 | loss 3.375858 (-0.59z)| norm 0.2901 (+0.55z)| lr 2.85e-04 | 323.45 ms | 52.2% bf16 MFU | 1622922 tok/s
step 10428/19560 | loss 3.432302 (+0.69z)| norm 0.2541 (-1.29z)| lr 2.85e-04 | 322.52 ms | 52.3% bf16 MFU | 1623055 tok/s
step 10429/19560 | loss 3.358898 (-0.97z)| norm 0.2734 (-0.32z)| lr 2.85e-04 | 322.48 ms | 52.3% bf16 MFU | 1623193 tok/s
step 10430/19560 | loss 3.444359 (+0.96z)| norm 0.2910 (+0.58z)| lr 2.85e-04 | 323.26 ms | 52.2% bf16 MFU | 1623127 tok/s
step 10431/19560 | loss 3.400130 (-0.05z)| norm 0.2861 (+0.32z)| lr 2.85e-04 | 323.19 ms | 52.2% bf16 MFU | 1623083 tok/s
step 10432/19560 | loss 3.395468 (-0.17z)| norm 0.3035 (+1.20z)| lr 2.85e-04 | 322.88 ms | 52.3% bf16 MFU | 1623117 tok/s
step 10433/19560 | loss 3.453726 (+1.16z)| norm 0.2801 (-0.01z)| lr 2.85e-04 | 322.96 ms | 52.3% bf16 MFU | 1623131 tok/s
step 10434/19560 | loss 3.368471 (-0.78z)| norm 0.2816 (+0.06z)| lr 2.85e-04 | 322.81 ms | 52.3% bf16 MFU | 1623181 tok/s
step 10435/19560 | loss 3.306954 (-2.17z)| norm 0.2754 (-0.26z)| lr 2.85e-04 | 322.84 ms | 52.3% bf16 MFU | 1623222 tok/s
step 10436/19560 | loss 3.440907 (+0.92z)| norm 0.2646 (-0.83z)| lr 2.85e-04 | 322.64 ms | 52.3% bf16 MFU | 1623311 tok/s
step 10437/19560 | loss 3.433879 (+0.76z)| norm 0.2755 (-0.26z)| lr 2.85e-04 | 323.37 ms | 52.2% bf16 MFU | 1623211 tok/s
step 10438/19560 | loss 3.419759 (+0.43z)| norm 0.2662 (-0.76z)| lr 2.85e-04 | 322.79 ms | 52.3% bf16 MFU | 1623261 tok/s
step 10439/19560 | loss 3.445274 (+1.01z)| norm 0.2607 (-1.03z)| lr 2.85e-04 | 323.93 ms | 52.1% bf16 MFU | 1623024 tok/s
step 10440/19560 | loss 3.353015 (-1.12z)| norm 0.2764 (-0.21z)| lr 2.85e-04 | 323.02 ms | 52.2% bf16 MFU | 1623026 tok/s
step 10441/19560 | loss 3.383676 (-0.38z)| norm 0.2655 (-0.79z)| lr 2.85e-04 | 323.09 ms | 52.2% bf16 MFU | 1623010 tok/s
step 10442/19560 | loss 3.391340 (-0.19z)| norm 0.2872 (+0.35z)| lr 2.84e-04 | 322.97 ms | 52.3% bf16 MFU | 1623028 tok/s
step 10443/19560 | loss 3.335047 (-1.51z)| norm 0.2737 (-0.36z)| lr 2.84e-04 | 323.84 ms | 52.1% bf16 MFU | 1622824 tok/s
step 10444/19560 | loss 3.437990 (+0.92z)| norm 0.2579 (-1.18z)| lr 2.84e-04 | 323.32 ms | 52.2% bf16 MFU | 1622762 tok/s
step 10445/19560 | loss 3.464849 (+1.52z)| norm 0.2758 (-0.24z)| lr 2.84e-04 | 322.99 ms | 52.3% bf16 MFU | 1622786 tok/s
step 10446/19560 | loss 3.384938 (-0.35z)| norm 0.2496 (-1.63z)| lr 2.84e-04 | 323.32 ms | 52.2% bf16 MFU | 1622725 tok/s
step 10447/19560 | loss 3.440897 (+1.06z)| norm 0.2973 (+0.96z)| lr 2.84e-04 | 324.11 ms | 52.1% bf16 MFU | 1622471 tok/s
step 10448/19560 | loss 3.479510 (+2.00z)| norm 0.2669 (-0.68z)| lr 2.84e-04 | 322.54 ms | 52.3% bf16 MFU | 1622623 tok/s
step 10449/19560 | loss 3.340700 (-1.42z)| norm 0.2733 (-0.32z)| lr 2.84e-04 | 324.15 ms | 52.1% bf16 MFU | 1622362 tok/s
step 10450/19560 | loss 3.330201 (-1.68z)| norm 0.2616 (-0.95z)| lr 2.84e-04 | 323.38 ms | 52.2% bf16 MFU | 1622309 tok/s
step 10451/19560 | loss 3.415502 (+0.51z)| norm 0.2740 (-0.26z)| lr 2.84e-04 | 323.30 ms | 52.2% bf16 MFU | 1622278 tok/s
step 10452/19560 | loss 3.421350 (+0.67z)| norm 0.2655 (-0.72z)| lr 2.84e-04 | 324.07 ms | 52.1% bf16 MFU | 1622056 tok/s
step 10453/19560 | loss 3.430697 (+0.91z)| norm 0.2762 (-0.11z)| lr 2.84e-04 | 322.70 ms | 52.3% bf16 MFU | 1622187 tok/s
step 10454/19560 | loss 3.368309 (-0.72z)| norm 0.2784 (+0.02z)| lr 2.84e-04 | 322.99 ms | 52.3% bf16 MFU | 1622239 tok/s
step 10455/19560 | loss 3.388556 (-0.19z)| norm 0.2750 (-0.15z)| lr 2.84e-04 | 323.70 ms | 52.1% bf16 MFU | 1622110 tok/s
step 10456/19560 | loss 3.390249 (-0.16z)| norm 0.3064 (+1.71z)| lr 2.84e-04 | 323.26 ms | 52.2% bf16 MFU | 1622098 tok/s
step 10457/19560 | loss 3.384296 (-0.30z)| norm 0.2641 (-0.79z)| lr 2.84e-04 | 323.17 ms | 52.2% bf16 MFU | 1622111 tok/s
step 10458/19560 | loss 3.403877 (+0.25z)| norm 0.2894 (+0.79z)| lr 2.84e-04 | 322.98 ms | 52.3% bf16 MFU | 1622169 tok/s
step 10459/19560 | loss 3.465446 (+1.94z)| norm 0.2757 (-0.05z)| lr 2.84e-04 | 323.30 ms | 52.2% bf16 MFU | 1622143 tok/s
step 10460/19560 | loss 3.366281 (-0.82z)| norm 0.2820 (+0.35z)| lr 2.84e-04 | 323.15 ms | 52.2% bf16 MFU | 1622158 tok/s
step 10461/19560 | loss 3.375916 (-0.55z)| norm 0.2863 (+0.62z)| lr 2.84e-04 | 323.05 ms | 52.2% bf16 MFU | 1622197 tok/s
step 10462/19560 | loss 3.420750 (+0.70z)| norm 0.2761 (-0.01z)| lr 2.83e-04 | 323.01 ms | 52.3% bf16 MFU | 1622244 tok/s
step 10463/19560 | loss 3.412811 (+0.47z)| norm 0.2491 (-1.80z)| lr 2.83e-04 | 322.69 ms | 52.3% bf16 MFU | 1622369 tok/s
step 10464/19560 | loss 3.457704 (+1.69z)| norm 0.2646 (-0.75z)| lr 2.83e-04 | 323.12 ms | 52.2% bf16 MFU | 1622381 tok/s
step 10465/19560 | loss 3.378021 (-0.50z)| norm 0.2614 (-0.96z)| lr 2.83e-04 | 323.24 ms | 52.2% bf16 MFU | 1622362 tok/s
step 10466/19560 | loss 3.350063 (-1.26z)| norm 0.2585 (-1.14z)| lr 2.83e-04 | 323.23 ms | 52.2% bf16 MFU | 1622345 tok/s
step 10467/19560 | loss 3.379678 (-0.44z)| norm 0.2703 (-0.33z)| lr 2.83e-04 | 322.51 ms | 52.3% bf16 MFU | 1622511 tok/s
step 10468/19560 | loss 3.356075 (-1.07z)| norm 0.2678 (-0.48z)| lr 2.83e-04 | 323.17 ms | 52.2% bf16 MFU | 1622501 tok/s
step 10469/19560 | loss 3.372047 (-0.63z)| norm 0.2769 (+0.12z)| lr 2.83e-04 | 322.87 ms | 52.3% bf16 MFU | 1622568 tok/s
step 10470/19560 | loss 3.386011 (-0.25z)| norm 0.2441 (-2.18z)| lr 2.83e-04 | 322.57 ms | 52.3% bf16 MFU | 1622707 tok/s
step 10471/19560 | loss 3.407826 (+0.35z)| norm 0.2623 (-0.86z)| lr 2.83e-04 | 323.19 ms | 52.2% bf16 MFU | 1622684 tok/s
step 10472/19560 | loss 3.337674 (-1.55z)| norm 0.2522 (-1.56z)| lr 2.83e-04 | 323.05 ms | 52.2% bf16 MFU | 1622696 tok/s
step 10473/19560 | loss 3.451560 (+1.52z)| norm 0.2679 (-0.44z)| lr 2.83e-04 | 323.00 ms | 52.3% bf16 MFU | 1622719 tok/s
step 10474/19560 | loss 3.358530 (-1.00z)| norm 0.2649 (-0.66z)| lr 2.83e-04 | 322.81 ms | 52.3% bf16 MFU | 1622791 tok/s
step 10475/19560 | loss 3.379099 (-0.43z)| norm 0.2932 (+1.35z)| lr 2.83e-04 | 322.80 ms | 52.3% bf16 MFU | 1622861 tok/s
step 10476/19560 | loss 3.479181 (+2.22z)| norm 0.2867 (+0.89z)| lr 2.83e-04 | 322.69 ms | 52.3% bf16 MFU | 1622955 tok/s
step 10477/19560 | loss 3.396650 (+0.01z)| norm 0.2886 (+1.02z)| lr 2.83e-04 | 322.76 ms | 52.3% bf16 MFU | 1623027 tok/s
step 10478/19560 | loss 3.412914 (+0.45z)| norm 0.2678 (-0.46z)| lr 2.83e-04 | 323.31 ms | 52.2% bf16 MFU | 1622957 tok/s
step 10479/19560 | loss 3.380953 (-0.41z)| norm 0.3218 (+3.28z)| lr 2.83e-04 | 322.48 ms | 52.3% bf16 MFU | 1623100 tok/s
step 10480/19560 | loss 3.412162 (+0.42z)| norm 0.2848 (+0.71z)| lr 2.83e-04 | 322.70 ms | 52.3% bf16 MFU | 1623180 tok/s
step 10481/19560 | loss 3.407699 (+0.29z)| norm 0.2852 (+0.74z)| lr 2.83e-04 | 323.56 ms | 52.2% bf16 MFU | 1623040 tok/s
step 10482/19560 | loss 3.372974 (-0.65z)| norm 0.3041 (+2.00z)| lr 2.82e-04 | 322.86 ms | 52.3% bf16 MFU | 1623082 tok/s
step 10483/19560 | loss 3.391312 (-0.15z)| norm 0.3707 (+5.62z)| lr 2.82e-04 | 322.40 ms | 52.3% bf16 MFU | 1623237 tok/s
step 10484/19560 | loss 3.415012 (+0.49z)| norm 0.2829 (+0.44z)| lr 2.82e-04 | 323.27 ms | 52.2% bf16 MFU | 1623167 tok/s
step 10485/19560 | loss 3.403718 (+0.19z)| norm 0.3150 (+2.28z)| lr 2.82e-04 | 322.97 ms | 52.3% bf16 MFU | 1623175 tok/s
step 10486/19560 | loss 3.357488 (-1.07z)| norm 0.3122 (+2.06z)| lr 2.82e-04 | 322.39 ms | 52.4% bf16 MFU | 1623329 tok/s
step 10487/19560 | loss 3.421847 (+0.70z)| norm 0.3097 (+1.88z)| lr 2.82e-04 | 323.12 ms | 52.2% bf16 MFU | 1623291 tok/s
step 10488/19560 | loss 3.371572 (-0.70z)| norm 0.2781 (+0.10z)| lr 2.82e-04 | 322.77 ms | 52.3% bf16 MFU | 1623345 tok/s
step 10489/19560 | loss 3.367743 (-0.80z)| norm 0.3168 (+2.21z)| lr 2.82e-04 | 323.01 ms | 52.2% bf16 MFU | 1623334 tok/s
step 10490/19560 | loss 3.402845 (+0.18z)| norm 0.2715 (-0.28z)| lr 2.82e-04 | 322.85 ms | 52.3% bf16 MFU | 1623364 tok/s
step 10491/19560 | loss 3.376569 (-0.58z)| norm 0.3210 (+2.41z)| lr 2.82e-04 | 323.07 ms | 52.2% bf16 MFU | 1623338 tok/s
step 10492/19560 | loss 3.378131 (-0.55z)| norm 0.2741 (-0.15z)| lr 2.82e-04 | 323.27 ms | 52.2% bf16 MFU | 1623263 tok/s
step 10493/19560 | loss 3.419372 (+0.63z)| norm 0.2708 (-0.31z)| lr 2.82e-04 | 322.65 ms | 52.3% bf16 MFU | 1623348 tok/s
step 10494/19560 | loss 3.351162 (-1.29z)| norm 0.2839 (+0.39z)| lr 2.82e-04 | 323.18 ms | 52.2% bf16 MFU | 1623295 tok/s
step 10495/19560 | loss 3.383709 (-0.37z)| norm 0.2725 (-0.23z)| lr 2.82e-04 | 322.29 ms | 52.4% bf16 MFU | 1623468 tok/s
step 10496/19560 | loss 3.376975 (-0.55z)| norm 0.2668 (-0.54z)| lr 2.82e-04 | 323.18 ms | 52.2% bf16 MFU | 1623410 tok/s
step 10497/19560 | loss 3.334727 (-1.76z)| norm 0.2743 (-0.14z)| lr 2.82e-04 | 322.67 ms | 52.3% bf16 MFU | 1623481 tok/s
step 10498/19560 | loss 3.441298 (+1.26z)| norm 0.2577 (-1.04z)| lr 2.82e-04 | 322.98 ms | 52.3% bf16 MFU | 1623471 tok/s
step 10499/19560 | loss 3.370346 (-0.77z)| norm 0.2615 (-0.83z)| lr 2.82e-04 | 322.60 ms | 52.3% bf16 MFU | 1623558 tok/s
step 10500/19560 | loss 3.413627 (+0.46z)| norm 0.2464 (-1.63z)| lr 2.82e-04 | 322.54 ms | 52.3% bf16 MFU | 1623654 tok/s
val loss 3.384710
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2921/10042 = 0.290878
step 10501/19560 | loss 3.435599 (+1.10z)| norm 0.2781 (+0.10z)| lr 2.82e-04 | 322.10 ms | 52.4% bf16 MFU | 1623857 tok/s
step 10502/19560 | loss 3.385600 (-0.34z)| norm 0.2614 (-0.80z)| lr 2.81e-04 | 322.94 ms | 52.3% bf16 MFU | 1623838 tok/s
step 10503/19560 | loss 3.422592 (+0.72z)| norm 0.2651 (-0.60z)| lr 2.81e-04 | 323.41 ms | 52.2% bf16 MFU | 1623702 tok/s
step 10504/19560 | loss 3.335410 (-1.76z)| norm 0.2615 (-0.79z)| lr 2.81e-04 | 323.10 ms | 52.2% bf16 MFU | 1623651 tok/s
step 10505/19560 | loss 3.414510 (+0.49z)| norm 0.2687 (-0.40z)| lr 2.81e-04 | 322.75 ms | 52.3% bf16 MFU | 1623689 tok/s
step 10506/19560 | loss 3.423794 (+0.75z)| norm 0.2663 (-0.53z)| lr 2.81e-04 | 322.79 ms | 52.3% bf16 MFU | 1623716 tok/s
step 10507/19560 | loss 3.404167 (+0.18z)| norm 0.2661 (-0.55z)| lr 2.81e-04 | 322.90 ms | 52.3% bf16 MFU | 1623715 tok/s
step 10508/19560 | loss 3.349413 (-1.37z)| norm 0.2554 (-1.13z)| lr 2.81e-04 | 323.20 ms | 52.2% bf16 MFU | 1623638 tok/s
step 10509/19560 | loss 3.428109 (+0.87z)| norm 0.2621 (-0.76z)| lr 2.81e-04 | 323.04 ms | 52.2% bf16 MFU | 1623606 tok/s
step 10510/19560 | loss 3.353843 (-1.23z)| norm 0.2651 (-0.59z)| lr 2.81e-04 | 322.91 ms | 52.3% bf16 MFU | 1623607 tok/s
step 10511/19560 | loss 3.362791 (-0.96z)| norm 0.2727 (-0.19z)| lr 2.81e-04 | 322.74 ms | 52.3% bf16 MFU | 1623652 tok/s
step 10512/19560 | loss 3.340265 (-1.57z)| norm 0.2434 (-1.76z)| lr 2.81e-04 | 323.12 ms | 52.2% bf16 MFU | 1623597 tok/s
step 10513/19560 | loss 3.416067 (+0.57z)| norm 0.2680 (-0.41z)| lr 2.81e-04 | 322.68 ms | 52.3% bf16 MFU | 1623656 tok/s
step 10514/19560 | loss 3.314539 (-2.24z)| norm 0.2678 (-0.41z)| lr 2.81e-04 | 322.65 ms | 52.3% bf16 MFU | 1623721 tok/s
step 10515/19560 | loss 3.427630 (+0.90z)| norm 0.2902 (+0.83z)| lr 2.81e-04 | 322.85 ms | 52.3% bf16 MFU | 1623731 tok/s
step 10516/19560 | loss 3.443986 (+1.33z)| norm 0.2972 (+1.22z)| lr 2.81e-04 | 322.95 ms | 52.3% bf16 MFU | 1623717 tok/s
step 10517/19560 | loss 3.417633 (+0.62z)| norm 0.2729 (-0.12z)| lr 2.81e-04 | 322.53 ms | 52.3% bf16 MFU | 1623808 tok/s
step 10518/19560 | loss 3.544463 (+3.94z)| norm 0.3044 (+1.62z)| lr 2.81e-04 | 323.18 ms | 52.2% bf16 MFU | 1623732 tok/s
step 10519/19560 | loss 3.323203 (-1.94z)| norm 0.3002 (+1.41z)| lr 2.81e-04 | 323.14 ms | 52.2% bf16 MFU | 1623669 tok/s
step 10520/19560 | loss 3.349583 (-1.23z)| norm 0.2776 (+0.14z)| lr 2.81e-04 | 322.26 ms | 52.4% bf16 MFU | 1623831 tok/s
step 10521/19560 | loss 3.390671 (-0.14z)| norm 0.2801 (+0.28z)| lr 2.81e-04 | 323.14 ms | 52.2% bf16 MFU | 1623763 tok/s
step 10522/19560 | loss 3.392869 (-0.08z)| norm 0.2803 (+0.28z)| lr 2.80e-04 | 322.65 ms | 52.3% bf16 MFU | 1623822 tok/s
step 10523/19560 | loss 3.325493 (-1.82z)| norm 0.2720 (-0.18z)| lr 2.80e-04 | 322.87 ms | 52.3% bf16 MFU | 1623823 tok/s
step 10524/19560 | loss 3.377193 (-0.48z)| norm 0.2621 (-0.74z)| lr 2.80e-04 | 322.94 ms | 52.3% bf16 MFU | 1623807 tok/s
step 10525/19560 | loss 3.366385 (-0.75z)| norm 0.2830 (+0.42z)| lr 2.80e-04 | 323.11 ms | 52.2% bf16 MFU | 1623748 tok/s
step 10526/19560 | loss 3.542967 (+3.68z)| norm 0.2876 (+0.67z)| lr 2.80e-04 | 322.69 ms | 52.3% bf16 MFU | 1623798 tok/s
step 10527/19560 | loss 3.397644 (+0.06z)| norm 0.2704 (-0.30z)| lr 2.80e-04 | 322.86 ms | 52.3% bf16 MFU | 1623803 tok/s
step 10528/19560 | loss 3.478710 (+2.07z)| norm 0.3018 (+1.44z)| lr 2.80e-04 | 323.38 ms | 52.2% bf16 MFU | 1623675 tok/s
step 10529/19560 | loss 3.420772 (+0.62z)| norm 0.2877 (+0.64z)| lr 2.80e-04 | 322.25 ms | 52.4% bf16 MFU | 1623838 tok/s
step 10530/19560 | loss 3.448711 (+1.30z)| norm 0.3030 (+1.48z)| lr 2.80e-04 | 323.00 ms | 52.3% bf16 MFU | 1623806 tok/s
step 10531/19560 | loss 3.353371 (-1.05z)| norm 0.3382 (+3.27z)| lr 2.80e-04 | 322.37 ms | 52.4% bf16 MFU | 1623935 tok/s
step 10532/19560 | loss 3.426147 (+0.74z)| norm 0.2836 (+0.34z)| lr 2.80e-04 | 322.59 ms | 52.3% bf16 MFU | 1624001 tok/s
step 10533/19560 | loss 3.571831 (+4.01z)| norm 0.3258 (+2.51z)| lr 2.80e-04 | 323.14 ms | 52.2% bf16 MFU | 1623925 tok/s
step 10534/19560 | loss 3.399270 (+0.03z)| norm 0.2719 (-0.30z)| lr 2.80e-04 | 323.01 ms | 52.2% bf16 MFU | 1623886 tok/s
step 10535/19560 | loss 3.403959 (+0.14z)| norm 0.2977 (+1.03z)| lr 2.80e-04 | 322.21 ms | 52.4% bf16 MFU | 1624049 tok/s
step 10536/19560 | loss 3.385666 (-0.28z)| norm 0.2928 (+0.78z)| lr 2.80e-04 | 323.26 ms | 52.2% bf16 MFU | 1623942 tok/s
step 10537/19560 | loss 3.357148 (-0.94z)| norm 0.2952 (+0.89z)| lr 2.80e-04 | 323.25 ms | 52.2% bf16 MFU | 1623841 tok/s
step 10538/19560 | loss 3.405941 (+0.19z)| norm 0.2824 (+0.23z)| lr 2.80e-04 | 322.39 ms | 52.4% bf16 MFU | 1623963 tok/s
step 10539/19560 | loss 3.344558 (-1.23z)| norm 0.2969 (+0.98z)| lr 2.80e-04 | 322.70 ms | 52.3% bf16 MFU | 1624000 tok/s
step 10540/19560 | loss 3.443327 (+1.04z)| norm 0.2883 (+0.52z)| lr 2.80e-04 | 322.30 ms | 52.4% bf16 MFU | 1624136 tok/s
step 10541/19560 | loss 3.371336 (-0.61z)| norm 0.2808 (+0.13z)| lr 2.80e-04 | 322.66 ms | 52.3% bf16 MFU | 1624173 tok/s
step 10542/19560 | loss 3.399459 (+0.04z)| norm 0.2840 (+0.29z)| lr 2.79e-04 | 323.12 ms | 52.2% bf16 MFU | 1624093 tok/s
step 10543/19560 | loss 3.486651 (+2.00z)| norm 0.2890 (+0.54z)| lr 2.79e-04 | 322.88 ms | 52.3% bf16 MFU | 1624078 tok/s
step 10544/19560 | loss 3.428638 (+0.67z)| norm 0.2936 (+0.78z)| lr 2.79e-04 | 322.92 ms | 52.3% bf16 MFU | 1624053 tok/s
step 10545/19560 | loss 3.504817 (+2.33z)| norm 0.3025 (+1.23z)| lr 2.79e-04 | 322.50 ms | 52.3% bf16 MFU | 1624135 tok/s
step 10546/19560 | loss 3.433175 (+0.73z)| norm 0.3043 (+1.30z)| lr 2.79e-04 | 322.83 ms | 52.3% bf16 MFU | 1624129 tok/s
step 10547/19560 | loss 3.416236 (+0.36z)| norm 0.2993 (+1.03z)| lr 2.79e-04 | 322.90 ms | 52.3% bf16 MFU | 1624107 tok/s
step 10548/19560 | loss 3.394439 (-0.13z)| norm 0.2814 (+0.10z)| lr 2.79e-04 | 322.51 ms | 52.3% bf16 MFU | 1624184 tok/s
step 10549/19560 | loss 3.411248 (+0.24z)| norm 0.2706 (-0.47z)| lr 2.79e-04 | 322.87 ms | 52.3% bf16 MFU | 1624166 tok/s
step 10550/19560 | loss 3.453293 (+1.15z)| norm 0.2850 (+0.27z)| lr 2.79e-04 | 322.71 ms | 52.3% bf16 MFU | 1624189 tok/s
step 10551/19560 | loss 3.347005 (-1.18z)| norm 0.2874 (+0.39z)| lr 2.79e-04 | 323.43 ms | 52.2% bf16 MFU | 1624031 tok/s
step 10552/19560 | loss 3.451589 (+1.10z)| norm 0.2703 (-0.50z)| lr 2.79e-04 | 322.37 ms | 52.4% bf16 MFU | 1624147 tok/s
step 10553/19560 | loss 3.520417 (+2.52z)| norm 0.2929 (+0.67z)| lr 2.79e-04 | 322.95 ms | 52.3% bf16 MFU | 1624111 tok/s
step 10554/19560 | loss 3.435887 (+0.71z)| norm 0.2899 (+0.50z)| lr 2.79e-04 | 322.39 ms | 52.3% bf16 MFU | 1624217 tok/s
step 10555/19560 | loss 3.412436 (+0.21z)| norm 0.2697 (-0.54z)| lr 2.79e-04 | 323.16 ms | 52.2% bf16 MFU | 1624126 tok/s
step 10556/19560 | loss 3.396241 (-0.13z)| norm 0.2993 (+0.99z)| lr 2.79e-04 | 322.60 ms | 52.3% bf16 MFU | 1624180 tok/s
step 10557/19560 | loss 3.429553 (+0.57z)| norm 0.2811 (+0.03z)| lr 2.79e-04 | 322.29 ms | 52.4% bf16 MFU | 1624308 tok/s
step 10558/19560 | loss 3.378884 (-0.50z)| norm 0.2939 (+0.70z)| lr 2.79e-04 | 323.33 ms | 52.2% bf16 MFU | 1624170 tok/s
step 10559/19560 | loss 3.396460 (-0.13z)| norm 0.2899 (+0.49z)| lr 2.79e-04 | 323.11 ms | 52.2% bf16 MFU | 1624092 tok/s
step 10560/19560 | loss 3.404680 (+0.05z)| norm 0.2650 (-0.80z)| lr 2.79e-04 | 323.26 ms | 52.2% bf16 MFU | 1623980 tok/s
step 10561/19560 | loss 3.388521 (-0.29z)| norm 0.2871 (+0.36z)| lr 2.79e-04 | 322.07 ms | 52.4% bf16 MFU | 1624174 tok/s
step 10562/19560 | loss 3.346339 (-1.19z)| norm 0.2716 (-0.45z)| lr 2.78e-04 | 323.68 ms | 52.1% bf16 MFU | 1623953 tok/s
step 10563/19560 | loss 3.350066 (-1.13z)| norm 0.2590 (-1.10z)| lr 2.78e-04 | 323.20 ms | 52.2% bf16 MFU | 1623865 tok/s
step 10564/19560 | loss 3.406353 (+0.10z)| norm 0.2567 (-1.21z)| lr 2.78e-04 | 322.84 ms | 52.3% bf16 MFU | 1623871 tok/s
step 10565/19560 | loss 3.341236 (-1.30z)| norm 0.2883 (+0.42z)| lr 2.78e-04 | 322.92 ms | 52.3% bf16 MFU | 1623856 tok/s
step 10566/19560 | loss 3.444220 (+0.93z)| norm 0.2634 (-0.86z)| lr 2.78e-04 | 322.57 ms | 52.3% bf16 MFU | 1623930 tok/s
step 10567/19560 | loss 3.485652 (+1.80z)| norm 0.3305 (+2.53z)| lr 2.78e-04 | 323.17 ms | 52.2% bf16 MFU | 1623850 tok/s
step 10568/19560 | loss 3.380045 (-0.47z)| norm 0.3281 (+2.33z)| lr 2.78e-04 | 323.16 ms | 52.2% bf16 MFU | 1623776 tok/s
step 10569/19560 | loss 3.385063 (-0.36z)| norm 0.2716 (-0.47z)| lr 2.78e-04 | 322.79 ms | 52.3% bf16 MFU | 1623800 tok/s
step 10570/19560 | loss 3.436783 (+0.74z)| norm 0.2883 (+0.36z)| lr 2.78e-04 | 323.07 ms | 52.2% bf16 MFU | 1623751 tok/s
step 10571/19560 | loss 3.387167 (-0.33z)| norm 0.2752 (-0.30z)| lr 2.78e-04 | 322.75 ms | 52.3% bf16 MFU | 1623784 tok/s
step 10572/19560 | loss 3.387872 (-0.31z)| norm 0.3018 (+1.01z)| lr 2.78e-04 | 323.62 ms | 52.2% bf16 MFU | 1623599 tok/s
step 10573/19560 | loss 3.464168 (+1.34z)| norm 0.2926 (+0.54z)| lr 2.78e-04 | 322.86 ms | 52.3% bf16 MFU | 1623613 tok/s
step 10574/19560 | loss 3.396019 (-0.13z)| norm 0.2612 (-1.02z)| lr 2.78e-04 | 322.83 ms | 52.3% bf16 MFU | 1623635 tok/s
step 10575/19560 | loss 3.444334 (+0.91z)| norm 0.3076 (+1.29z)| lr 2.78e-04 | 323.16 ms | 52.2% bf16 MFU | 1623573 tok/s
step 10576/19560 | loss 3.400764 (-0.02z)| norm 0.2519 (-1.47z)| lr 2.78e-04 | 323.19 ms | 52.2% bf16 MFU | 1623506 tok/s
step 10577/19560 | loss 3.409422 (+0.16z)| norm 0.2681 (-0.67z)| lr 2.78e-04 | 322.55 ms | 52.3% bf16 MFU | 1623604 tok/s
step 10578/19560 | loss 3.409169 (+0.14z)| norm 0.2590 (-1.12z)| lr 2.78e-04 | 323.22 ms | 52.2% bf16 MFU | 1623528 tok/s
step 10579/19560 | loss 3.393572 (-0.20z)| norm 0.2695 (-0.60z)| lr 2.78e-04 | 323.02 ms | 52.2% bf16 MFU | 1623504 tok/s
step 10580/19560 | loss 3.357503 (-0.99z)| norm 0.2530 (-1.40z)| lr 2.78e-04 | 323.25 ms | 52.2% bf16 MFU | 1623426 tok/s
step 10581/19560 | loss 3.553186 (+3.20z)| norm 0.2820 (+0.03z)| lr 2.78e-04 | 323.24 ms | 52.2% bf16 MFU | 1623353 tok/s
step 10582/19560 | loss 3.335814 (-1.42z)| norm 0.2620 (-0.95z)| lr 2.77e-04 | 322.39 ms | 52.4% bf16 MFU | 1623498 tok/s
step 10583/19560 | loss 3.383623 (-0.41z)| norm 0.2633 (-0.88z)| lr 2.77e-04 | 322.88 ms | 52.3% bf16 MFU | 1623514 tok/s
step 10584/19560 | loss 3.369281 (-0.71z)| norm 0.2592 (-1.06z)| lr 2.77e-04 | 323.03 ms | 52.2% bf16 MFU | 1623489 tok/s
step 10585/19560 | loss 3.471130 (+1.42z)| norm 0.2818 (+0.04z)| lr 2.77e-04 | 322.80 ms | 52.3% bf16 MFU | 1623525 tok/s
step 10586/19560 | loss 3.391852 (-0.24z)| norm 0.2874 (+0.31z)| lr 2.77e-04 | 323.10 ms | 52.2% bf16 MFU | 1623482 tok/s
step 10587/19560 | loss 3.410735 (+0.17z)| norm 0.2746 (-0.32z)| lr 2.77e-04 | 322.65 ms | 52.3% bf16 MFU | 1623556 tok/s
step 10588/19560 | loss 3.406009 (+0.06z)| norm 0.2792 (-0.09z)| lr 2.77e-04 | 323.05 ms | 52.2% bf16 MFU | 1623524 tok/s
step 10589/19560 | loss 3.431438 (+0.59z)| norm 0.2673 (-0.66z)| lr 2.77e-04 | 323.36 ms | 52.2% bf16 MFU | 1623418 tok/s
step 10590/19560 | loss 3.341713 (-1.29z)| norm 0.2742 (-0.32z)| lr 2.77e-04 | 322.49 ms | 52.3% bf16 MFU | 1623533 tok/s
step 10591/19560 | loss 3.436241 (+0.70z)| norm 0.2697 (-0.56z)| lr 2.77e-04 | 323.15 ms | 52.2% bf16 MFU | 1623477 tok/s
step 10592/19560 | loss 3.437225 (+0.72z)| norm 0.2867 (+0.27z)| lr 2.77e-04 | 323.63 ms | 52.1% bf16 MFU | 1623303 tok/s
step 10593/19560 | loss 3.381245 (-0.46z)| norm 0.2701 (-0.55z)| lr 2.77e-04 | 323.16 ms | 52.2% bf16 MFU | 1623257 tok/s
step 10594/19560 | loss 3.412632 (+0.19z)| norm 0.2760 (-0.27z)| lr 2.77e-04 | 322.79 ms | 52.3% bf16 MFU | 1623305 tok/s
step 10595/19560 | loss 3.377810 (-0.54z)| norm 0.2726 (-0.44z)| lr 2.77e-04 | 323.06 ms | 52.2% bf16 MFU | 1623283 tok/s
step 10596/19560 | loss 3.404307 (+0.01z)| norm 0.2905 (+0.44z)| lr 2.77e-04 | 323.01 ms | 52.2% bf16 MFU | 1623276 tok/s
step 10597/19560 | loss 3.398154 (-0.13z)| norm 0.2818 (+0.01z)| lr 2.77e-04 | 322.67 ms | 52.3% bf16 MFU | 1623353 tok/s
step 10598/19560 | loss 3.471868 (+1.42z)| norm 0.2645 (-0.87z)| lr 2.77e-04 | 323.04 ms | 52.2% bf16 MFU | 1623335 tok/s
step 10599/19560 | loss 3.352904 (-1.08z)| norm 0.2924 (+0.53z)| lr 2.77e-04 | 322.49 ms | 52.3% bf16 MFU | 1623456 tok/s
step 10600/19560 | loss 3.414057 (+0.19z)| norm 0.2698 (-0.63z)| lr 2.77e-04 | 322.68 ms | 52.3% bf16 MFU | 1623522 tok/s
step 10601/19560 | loss 3.401369 (-0.07z)| norm 0.2717 (-0.53z)| lr 2.77e-04 | 322.73 ms | 52.3% bf16 MFU | 1623573 tok/s
step 10602/19560 | loss 3.401865 (-0.06z)| norm 0.2626 (-1.00z)| lr 2.76e-04 | 322.84 ms | 52.3% bf16 MFU | 1623594 tok/s
step 10603/19560 | loss 3.427603 (+0.48z)| norm 0.2801 (-0.10z)| lr 2.76e-04 | 323.48 ms | 52.2% bf16 MFU | 1623453 tok/s
step 10604/19560 | loss 3.402611 (-0.04z)| norm 0.2742 (-0.40z)| lr 2.76e-04 | 322.51 ms | 52.3% bf16 MFU | 1623561 tok/s
step 10605/19560 | loss 3.451191 (+0.99z)| norm 0.2563 (-1.29z)| lr 2.76e-04 | 322.75 ms | 52.3% bf16 MFU | 1623606 tok/s
step 10606/19560 | loss 3.393765 (-0.24z)| norm 0.2783 (-0.18z)| lr 2.76e-04 | 323.12 ms | 52.2% bf16 MFU | 1623554 tok/s
step 10607/19560 | loss 3.404373 (-0.01z)| norm 0.2609 (-1.05z)| lr 2.76e-04 | 322.49 ms | 52.3% bf16 MFU | 1623663 tok/s
step 10608/19560 | loss 3.363288 (-0.89z)| norm 0.2660 (-0.78z)| lr 2.76e-04 | 322.36 ms | 52.4% bf16 MFU | 1623800 tok/s
step 10609/19560 | loss 3.349326 (-1.17z)| norm 0.2646 (-0.84z)| lr 2.76e-04 | 322.93 ms | 52.3% bf16 MFU | 1623787 tok/s
step 10610/19560 | loss 3.442530 (+0.81z)| norm 0.2863 (+0.28z)| lr 2.76e-04 | 322.58 ms | 52.3% bf16 MFU | 1623863 tok/s
step 10611/19560 | loss 3.387735 (-0.36z)| norm 0.2531 (-1.50z)| lr 2.76e-04 | 323.27 ms | 52.2% bf16 MFU | 1623761 tok/s
step 10612/19560 | loss 3.383370 (-0.45z)| norm 0.2725 (-0.41z)| lr 2.76e-04 | 322.80 ms | 52.3% bf16 MFU | 1623783 tok/s
step 10613/19560 | loss 3.339187 (-1.37z)| norm 0.2387 (-2.26z)| lr 2.76e-04 | 323.10 ms | 52.2% bf16 MFU | 1623729 tok/s
step 10614/19560 | loss 3.341651 (-1.31z)| norm 0.2563 (-1.26z)| lr 2.76e-04 | 323.59 ms | 52.2% bf16 MFU | 1623554 tok/s
step 10615/19560 | loss 3.427078 (+0.49z)| norm 0.2569 (-1.21z)| lr 2.76e-04 | 323.30 ms | 52.2% bf16 MFU | 1623460 tok/s
step 10616/19560 | loss 3.382172 (-0.46z)| norm 0.2537 (-1.37z)| lr 2.76e-04 | 323.27 ms | 52.2% bf16 MFU | 1623379 tok/s
step 10617/19560 | loss 3.476957 (+1.51z)| norm 0.2794 (+0.08z)| lr 2.76e-04 | 323.09 ms | 52.2% bf16 MFU | 1623348 tok/s
step 10618/19560 | loss 3.428929 (+0.50z)| norm 0.2928 (+0.84z)| lr 2.76e-04 | 323.00 ms | 52.3% bf16 MFU | 1623340 tok/s
step 10619/19560 | loss 3.412087 (+0.14z)| norm 0.2604 (-1.00z)| lr 2.76e-04 | 323.28 ms | 52.2% bf16 MFU | 1623262 tok/s
step 10620/19560 | loss 3.398341 (-0.15z)| norm 0.2784 (+0.04z)| lr 2.76e-04 | 323.10 ms | 52.2% bf16 MFU | 1623233 tok/s
step 10621/19560 | loss 3.370250 (-0.73z)| norm 0.2708 (-0.40z)| lr 2.76e-04 | 323.67 ms | 52.1% bf16 MFU | 1623063 tok/s
step 10622/19560 | loss 3.388854 (-0.35z)| norm 0.2801 (+0.14z)| lr 2.75e-04 | 323.15 ms | 52.2% bf16 MFU | 1623031 tok/s
step 10623/19560 | loss 3.327197 (-1.62z)| norm 0.2720 (-0.33z)| lr 2.75e-04 | 323.20 ms | 52.2% bf16 MFU | 1622989 tok/s
step 10624/19560 | loss 3.378855 (-0.54z)| norm 0.2718 (-0.34z)| lr 2.75e-04 | 323.52 ms | 52.2% bf16 MFU | 1622867 tok/s
step 10625/19560 | loss 3.413596 (+0.17z)| norm 0.2841 (+0.37z)| lr 2.75e-04 | 323.27 ms | 52.2% bf16 MFU | 1622816 tok/s
step 10626/19560 | loss 3.395108 (-0.21z)| norm 0.2526 (-1.45z)| lr 2.75e-04 | 323.43 ms | 52.2% bf16 MFU | 1622727 tok/s
step 10627/19560 | loss 3.374343 (-0.65z)| norm 0.2838 (+0.34z)| lr 2.75e-04 | 323.09 ms | 52.2% bf16 MFU | 1622728 tok/s
step 10628/19560 | loss 3.440241 (+0.73z)| norm 0.2607 (-1.01z)| lr 2.75e-04 | 323.81 ms | 52.1% bf16 MFU | 1622547 tok/s
step 10629/19560 | loss 3.449539 (+0.92z)| norm 0.2899 (+0.69z)| lr 2.75e-04 | 323.18 ms | 52.2% bf16 MFU | 1622534 tok/s
step 10630/19560 | loss 3.324717 (-1.67z)| norm 0.2511 (-1.56z)| lr 2.75e-04 | 323.62 ms | 52.2% bf16 MFU | 1622410 tok/s
step 10631/19560 | loss 3.404038 (-0.02z)| norm 0.2897 (+0.67z)| lr 2.75e-04 | 323.55 ms | 52.2% bf16 MFU | 1622311 tok/s
step 10632/19560 | loss 3.330280 (-1.55z)| norm 0.2690 (-0.54z)| lr 2.75e-04 | 323.09 ms | 52.2% bf16 MFU | 1622333 tok/s
step 10633/19560 | loss 3.546464 (+2.83z)| norm 0.2920 (+0.79z)| lr 2.75e-04 | 323.63 ms | 52.1% bf16 MFU | 1622217 tok/s
step 10634/19560 | loss 3.398584 (-0.14z)| norm 0.3100 (+1.80z)| lr 2.75e-04 | 322.82 ms | 52.3% bf16 MFU | 1622312 tok/s
step 10635/19560 | loss 3.441256 (+0.71z)| norm 0.3136 (+1.95z)| lr 2.75e-04 | 323.29 ms | 52.2% bf16 MFU | 1622283 tok/s
step 10636/19560 | loss 3.471297 (+1.29z)| norm 0.2924 (+0.74z)| lr 2.75e-04 | 323.23 ms | 52.2% bf16 MFU | 1622271 tok/s
step 10637/19560 | loss 3.466181 (+1.18z)| norm 0.2972 (+1.00z)| lr 2.75e-04 | 322.70 ms | 52.3% bf16 MFU | 1622392 tok/s
step 10638/19560 | loss 3.451528 (+0.87z)| norm 0.3103 (+1.71z)| lr 2.75e-04 | 322.79 ms | 52.3% bf16 MFU | 1622484 tok/s
step 10639/19560 | loss 3.411135 (+0.05z)| norm 0.2742 (-0.33z)| lr 2.75e-04 | 323.51 ms | 52.2% bf16 MFU | 1622391 tok/s
step 10640/19560 | loss 3.392468 (-0.33z)| norm 0.2987 (+1.04z)| lr 2.75e-04 | 323.07 ms | 52.2% bf16 MFU | 1622412 tok/s
step 10641/19560 | loss 3.367826 (-0.82z)| norm 0.2972 (+0.94z)| lr 2.75e-04 | 323.61 ms | 52.2% bf16 MFU | 1622298 tok/s
step 10642/19560 | loss 3.620166 (+4.00z)| norm 0.2702 (-0.60z)| lr 2.74e-04 | 322.74 ms | 52.3% bf16 MFU | 1622409 tok/s
step 10643/19560 | loss 3.464312 (+1.01z)| norm 0.2829 (+0.12z)| lr 2.74e-04 | 323.31 ms | 52.2% bf16 MFU | 1622370 tok/s
step 10644/19560 | loss 3.403600 (-0.14z)| norm 0.2876 (+0.40z)| lr 2.74e-04 | 323.04 ms | 52.2% bf16 MFU | 1622401 tok/s
step 10645/19560 | loss 3.385879 (-0.47z)| norm 0.2756 (-0.29z)| lr 2.74e-04 | 322.56 ms | 52.3% bf16 MFU | 1622550 tok/s
step 10646/19560 | loss 3.346598 (-1.21z)| norm 0.3007 (+1.16z)| lr 2.74e-04 | 323.40 ms | 52.2% bf16 MFU | 1622481 tok/s
step 10647/19560 | loss 3.388628 (-0.41z)| norm 0.2833 (+0.16z)| lr 2.74e-04 | 323.12 ms | 52.2% bf16 MFU | 1622486 tok/s
step 10648/19560 | loss 3.377681 (-0.63z)| norm 0.2819 (+0.08z)| lr 2.74e-04 | 323.04 ms | 52.2% bf16 MFU | 1622510 tok/s
step 10649/19560 | loss 3.525964 (+2.23z)| norm 0.3005 (+1.14z)| lr 2.74e-04 | 323.21 ms | 52.2% bf16 MFU | 1622490 tok/s
step 10650/19560 | loss 3.402624 (-0.16z)| norm 0.3021 (+1.22z)| lr 2.74e-04 | 323.02 ms | 52.2% bf16 MFU | 1622520 tok/s
step 10651/19560 | loss 3.355084 (-1.09z)| norm 0.2587 (-1.25z)| lr 2.74e-04 | 323.08 ms | 52.2% bf16 MFU | 1622533 tok/s
step 10652/19560 | loss 3.405024 (-0.12z)| norm 0.3044 (+1.32z)| lr 2.74e-04 | 322.60 ms | 52.3% bf16 MFU | 1622666 tok/s
step 10653/19560 | loss 3.356997 (-1.06z)| norm 0.2669 (-0.80z)| lr 2.74e-04 | 322.86 ms | 52.3% bf16 MFU | 1622726 tok/s
step 10654/19560 | loss 3.400246 (-0.20z)| norm 0.2924 (+0.65z)| lr 2.74e-04 | 323.41 ms | 52.2% bf16 MFU | 1622646 tok/s
step 10655/19560 | loss 3.400630 (-0.19z)| norm 0.2955 (+0.81z)| lr 2.74e-04 | 322.45 ms | 52.3% bf16 MFU | 1622811 tok/s
step 10656/19560 | loss 3.424180 (+0.29z)| norm 0.2847 (+0.21z)| lr 2.74e-04 | 322.80 ms | 52.3% bf16 MFU | 1622881 tok/s
step 10657/19560 | loss 3.376660 (-0.66z)| norm 0.3108 (+1.67z)| lr 2.74e-04 | 322.77 ms | 52.3% bf16 MFU | 1622953 tok/s
step 10658/19560 | loss 3.401601 (-0.15z)| norm 0.2908 (+0.55z)| lr 2.74e-04 | 323.20 ms | 52.2% bf16 MFU | 1622914 tok/s
step 10659/19560 | loss 3.380474 (-0.58z)| norm 0.2863 (+0.33z)| lr 2.74e-04 | 323.15 ms | 52.2% bf16 MFU | 1622888 tok/s
step 10660/19560 | loss 3.353937 (-1.10z)| norm 0.2674 (-0.77z)| lr 2.74e-04 | 322.30 ms | 52.4% bf16 MFU | 1623080 tok/s
step 10661/19560 | loss 3.427356 (+0.42z)| norm 0.2975 (+1.03z)| lr 2.74e-04 | 322.76 ms | 52.3% bf16 MFU | 1623146 tok/s
step 10662/19560 | loss 3.439938 (+0.67z)| norm 0.2901 (+0.58z)| lr 2.73e-04 | 322.48 ms | 52.3% bf16 MFU | 1623279 tok/s
step 10663/19560 | loss 3.372608 (-0.73z)| norm 0.2999 (+1.16z)| lr 2.73e-04 | 322.73 ms | 52.3% bf16 MFU | 1623343 tok/s
step 10664/19560 | loss 3.423221 (+0.32z)| norm 0.2917 (+0.67z)| lr 2.73e-04 | 322.69 ms | 52.3% bf16 MFU | 1623413 tok/s
step 10665/19560 | loss 3.415965 (+0.16z)| norm 0.2746 (-0.35z)| lr 2.73e-04 | 322.95 ms | 52.3% bf16 MFU | 1623415 tok/s
step 10666/19560 | loss 3.361058 (-0.98z)| norm 0.2887 (+0.50z)| lr 2.73e-04 | 323.15 ms | 52.2% bf16 MFU | 1623365 tok/s
step 10667/19560 | loss 3.461561 (+1.11z)| norm 0.2825 (+0.13z)| lr 2.73e-04 | 322.30 ms | 52.4% bf16 MFU | 1623533 tok/s
step 10668/19560 | loss 3.442847 (+0.71z)| norm 0.2926 (+0.74z)| lr 2.73e-04 | 322.58 ms | 52.3% bf16 MFU | 1623621 tok/s
step 10669/19560 | loss 3.358073 (-1.06z)| norm 0.2732 (-0.43z)| lr 2.73e-04 | 322.61 ms | 52.3% bf16 MFU | 1623698 tok/s
step 10670/19560 | loss 3.385269 (-0.49z)| norm 0.2894 (+0.55z)| lr 2.73e-04 | 323.65 ms | 52.1% bf16 MFU | 1623510 tok/s
step 10671/19560 | loss 3.352719 (-1.16z)| norm 0.2550 (-1.50z)| lr 2.73e-04 | 322.97 ms | 52.3% bf16 MFU | 1623501 tok/s
step 10672/19560 | loss 3.444701 (+0.78z)| norm 0.2926 (+0.75z)| lr 2.73e-04 | 322.75 ms | 52.3% bf16 MFU | 1623548 tok/s
step 10673/19560 | loss 3.373425 (-0.71z)| norm 0.2594 (-1.22z)| lr 2.73e-04 | 322.90 ms | 52.3% bf16 MFU | 1623554 tok/s
step 10674/19560 | loss 3.400724 (-0.12z)| norm 0.2605 (-1.14z)| lr 2.73e-04 | 322.73 ms | 52.3% bf16 MFU | 1623604 tok/s
step 10675/19560 | loss 3.391866 (-0.31z)| norm 0.2802 (+0.06z)| lr 2.73e-04 | 322.95 ms | 52.3% bf16 MFU | 1623596 tok/s
step 10676/19560 | loss 3.401643 (-0.10z)| norm 0.2660 (-0.79z)| lr 2.73e-04 | 322.98 ms | 52.3% bf16 MFU | 1623581 tok/s
step 10677/19560 | loss 3.428911 (+0.48z)| norm 0.2781 (-0.06z)| lr 2.73e-04 | 322.75 ms | 52.3% bf16 MFU | 1623623 tok/s
step 10678/19560 | loss 3.428447 (+0.47z)| norm 0.2708 (-0.50z)| lr 2.73e-04 | 322.51 ms | 52.3% bf16 MFU | 1623724 tok/s
step 10679/19560 | loss 3.377926 (-0.62z)| norm 0.2701 (-0.53z)| lr 2.73e-04 | 322.94 ms | 52.3% bf16 MFU | 1623711 tok/s
step 10680/19560 | loss 3.351926 (-1.16z)| norm 0.2562 (-1.36z)| lr 2.73e-04 | 322.80 ms | 52.3% bf16 MFU | 1623735 tok/s
step 10681/19560 | loss 3.377589 (-0.60z)| norm 0.2622 (-0.99z)| lr 2.73e-04 | 323.06 ms | 52.2% bf16 MFU | 1623692 tok/s
step 10682/19560 | loss 3.365414 (-0.85z)| norm 0.2535 (-1.48z)| lr 2.73e-04 | 322.63 ms | 52.3% bf16 MFU | 1623760 tok/s
step 10683/19560 | loss 3.404235 (+0.00z)| norm 0.2879 (+0.56z)| lr 2.72e-04 | 322.72 ms | 52.3% bf16 MFU | 1623802 tok/s
step 10684/19560 | loss 3.375926 (-0.61z)| norm 0.2656 (-0.75z)| lr 2.72e-04 | 322.87 ms | 52.3% bf16 MFU | 1623804 tok/s
step 10685/19560 | loss 3.362299 (-0.90z)| norm 0.2758 (-0.14z)| lr 2.72e-04 | 322.80 ms | 52.3% bf16 MFU | 1623822 tok/s
step 10686/19560 | loss 3.370076 (-0.73z)| norm 0.2861 (+0.48z)| lr 2.72e-04 | 322.82 ms | 52.3% bf16 MFU | 1623837 tok/s
step 10687/19560 | loss 3.414963 (+0.25z)| norm 0.2672 (-0.64z)| lr 2.72e-04 | 322.67 ms | 52.3% bf16 MFU | 1623886 tok/s
step 10688/19560 | loss 3.388234 (-0.33z)| norm 0.2543 (-1.40z)| lr 2.72e-04 | 323.06 ms | 52.2% bf16 MFU | 1623836 tok/s
step 10689/19560 | loss 3.394121 (-0.20z)| norm 0.2583 (-1.14z)| lr 2.72e-04 | 322.53 ms | 52.3% bf16 MFU | 1623922 tok/s
step 10690/19560 | loss 3.456031 (+1.13z)| norm 0.2657 (-0.70z)| lr 2.72e-04 | 322.99 ms | 52.3% bf16 MFU | 1623887 tok/s
step 10691/19560 | loss 3.352018 (-1.14z)| norm 0.2526 (-1.47z)| lr 2.72e-04 | 322.88 ms | 52.3% bf16 MFU | 1623882 tok/s
step 10692/19560 | loss 3.382294 (-0.48z)| norm 0.2785 (+0.05z)| lr 2.72e-04 | 322.65 ms | 52.3% bf16 MFU | 1623935 tok/s
step 10693/19560 | loss 3.428077 (+0.51z)| norm 0.2636 (-0.82z)| lr 2.72e-04 | 322.64 ms | 52.3% bf16 MFU | 1623988 tok/s
step 10694/19560 | loss 3.377477 (-0.59z)| norm 0.2839 (+0.38z)| lr 2.72e-04 | 322.89 ms | 52.3% bf16 MFU | 1623977 tok/s
step 10695/19560 | loss 3.332013 (-1.57z)| norm 0.2607 (-1.01z)| lr 2.72e-04 | 322.93 ms | 52.3% bf16 MFU | 1623954 tok/s
step 10696/19560 | loss 3.422458 (+0.42z)| norm 0.2915 (+0.95z)| lr 2.72e-04 | 322.72 ms | 52.3% bf16 MFU | 1623986 tok/s
step 10697/19560 | loss 3.383605 (-0.44z)| norm 0.2890 (+0.77z)| lr 2.72e-04 | 322.71 ms | 52.3% bf16 MFU | 1624018 tok/s
step 10698/19560 | loss 3.410459 (+0.16z)| norm 0.2669 (-0.63z)| lr 2.72e-04 | 322.57 ms | 52.3% bf16 MFU | 1624084 tok/s
step 10699/19560 | loss 3.367115 (-0.79z)| norm 0.2656 (-0.71z)| lr 2.72e-04 | 323.44 ms | 52.2% bf16 MFU | 1623928 tok/s
step 10700/19560 | loss 3.415569 (+0.27z)| norm 0.2940 (+1.12z)| lr 2.72e-04 | 322.66 ms | 52.3% bf16 MFU | 1623975 tok/s
step 10701/19560 | loss 3.436249 (+0.74z)| norm 0.3233 (+2.90z)| lr 2.72e-04 | 323.10 ms | 52.2% bf16 MFU | 1623910 tok/s
step 10702/19560 | loss 3.378518 (-0.54z)| norm 0.2839 (+0.43z)| lr 2.72e-04 | 323.20 ms | 52.2% bf16 MFU | 1623823 tok/s
step 10703/19560 | loss 3.455290 (+1.16z)| norm 0.3033 (+1.65z)| lr 2.71e-04 | 322.17 ms | 52.4% bf16 MFU | 1624001 tok/s
step 10704/19560 | loss 3.357970 (-0.99z)| norm 0.2919 (+0.92z)| lr 2.71e-04 | 322.69 ms | 52.3% bf16 MFU | 1624038 tok/s
step 10705/19560 | loss 3.436163 (+0.74z)| norm 0.2991 (+1.36z)| lr 2.71e-04 | 322.88 ms | 52.3% bf16 MFU | 1624025 tok/s
step 10706/19560 | loss 3.441918 (+0.86z)| norm 0.2746 (-0.19z)| lr 2.71e-04 | 322.62 ms | 52.3% bf16 MFU | 1624080 tok/s
step 10707/19560 | loss 3.375228 (-0.61z)| norm 0.2935 (+0.99z)| lr 2.71e-04 | 322.84 ms | 52.3% bf16 MFU | 1624076 tok/s
step 10708/19560 | loss 3.444368 (+0.90z)| norm 0.2766 (-0.09z)| lr 2.71e-04 | 322.71 ms | 52.3% bf16 MFU | 1624105 tok/s
step 10709/19560 | loss 3.400588 (-0.04z)| norm 0.2768 (-0.08z)| lr 2.71e-04 | 322.93 ms | 52.3% bf16 MFU | 1624077 tok/s
step 10710/19560 | loss 3.384720 (-0.42z)| norm 0.2651 (-0.82z)| lr 2.71e-04 | 322.75 ms | 52.3% bf16 MFU | 1624095 tok/s
step 10711/19560 | loss 3.454783 (+1.19z)| norm 0.3207 (+2.62z)| lr 2.71e-04 | 322.90 ms | 52.3% bf16 MFU | 1624076 tok/s
step 10712/19560 | loss 3.488813 (+1.93z)| norm 0.2760 (-0.17z)| lr 2.71e-04 | 322.72 ms | 52.3% bf16 MFU | 1624102 tok/s
step 10713/19560 | loss 3.382797 (-0.48z)| norm 0.2859 (+0.45z)| lr 2.71e-04 | 322.64 ms | 52.3% bf16 MFU | 1624147 tok/s
step 10714/19560 | loss 3.339990 (-1.44z)| norm 0.2694 (-0.57z)| lr 2.71e-04 | 322.17 ms | 52.4% bf16 MFU | 1624307 tok/s
step 10715/19560 | loss 3.362694 (-0.91z)| norm 0.2651 (-0.83z)| lr 2.71e-04 | 323.30 ms | 52.2% bf16 MFU | 1624174 tok/s
step 10716/19560 | loss 3.456952 (+1.22z)| norm 0.2745 (-0.24z)| lr 2.71e-04 | 322.67 ms | 52.3% bf16 MFU | 1624208 tok/s
step 10717/19560 | loss 3.299334 (-2.28z)| norm 0.2687 (-0.61z)| lr 2.71e-04 | 322.66 ms | 52.3% bf16 MFU | 1624243 tok/s
step 10718/19560 | loss 3.407059 (+0.10z)| norm 0.3283 (+2.97z)| lr 2.71e-04 | 322.03 ms | 52.4% bf16 MFU | 1624434 tok/s
step 10719/19560 | loss 3.452195 (+1.10z)| norm 0.2897 (+0.64z)| lr 2.71e-04 | 323.62 ms | 52.2% bf16 MFU | 1624216 tok/s
step 10720/19560 | loss 3.447496 (+0.99z)| norm 0.3097 (+1.81z)| lr 2.71e-04 | 322.81 ms | 52.3% bf16 MFU | 1624213 tok/s
step 10721/19560 | loss 3.378647 (-0.54z)| norm 0.2891 (+0.58z)| lr 2.71e-04 | 322.81 ms | 52.3% bf16 MFU | 1624209 tok/s
step 10722/19560 | loss 3.459506 (+1.24z)| norm 0.2951 (+0.93z)| lr 2.71e-04 | 322.50 ms | 52.3% bf16 MFU | 1624284 tok/s
step 10723/19560 | loss 3.382895 (-0.45z)| norm 0.2849 (+0.32z)| lr 2.70e-04 | 322.31 ms | 52.4% bf16 MFU | 1624402 tok/s
step 10724/19560 | loss 3.314046 (-1.93z)| norm 0.2428 (-2.12z)| lr 2.70e-04 | 322.64 ms | 52.3% bf16 MFU | 1624431 tok/s
step 10725/19560 | loss 3.387422 (-0.33z)| norm 0.2814 (+0.13z)| lr 2.70e-04 | 322.51 ms | 52.3% bf16 MFU | 1624491 tok/s
step 10726/19560 | loss 3.393522 (-0.18z)| norm 0.2703 (-0.52z)| lr 2.70e-04 | 322.52 ms | 52.3% bf16 MFU | 1624547 tok/s
step 10727/19560 | loss 3.346948 (-1.20z)| norm 0.2581 (-1.21z)| lr 2.70e-04 | 322.88 ms | 52.3% bf16 MFU | 1624508 tok/s
step 10728/19560 | loss 3.523655 (+2.58z)| norm 0.2869 (+0.46z)| lr 2.70e-04 | 322.55 ms | 52.3% bf16 MFU | 1624554 tok/s
step 10729/19560 | loss 3.394302 (-0.18z)| norm 0.2971 (+1.03z)| lr 2.70e-04 | 322.41 ms | 52.3% bf16 MFU | 1624633 tok/s
step 10730/19560 | loss 3.314786 (-1.84z)| norm 0.2635 (-0.92z)| lr 2.70e-04 | 322.56 ms | 52.3% bf16 MFU | 1624671 tok/s
step 10731/19560 | loss 3.408947 (+0.15z)| norm 0.3088 (+1.68z)| lr 2.70e-04 | 322.54 ms | 52.3% bf16 MFU | 1624712 tok/s
step 10732/19560 | loss 3.395818 (-0.12z)| norm 0.2898 (+0.58z)| lr 2.70e-04 | 323.29 ms | 52.2% bf16 MFU | 1624563 tok/s
step 10733/19560 | loss 3.393825 (-0.16z)| norm 0.2805 (+0.04z)| lr 2.70e-04 | 322.55 ms | 52.3% bf16 MFU | 1624608 tok/s
step 10734/19560 | loss 3.369970 (-0.66z)| norm 0.2651 (-0.84z)| lr 2.70e-04 | 322.62 ms | 52.3% bf16 MFU | 1624633 tok/s
step 10735/19560 | loss 3.329175 (-1.49z)| norm 0.2592 (-1.18z)| lr 2.70e-04 | 322.47 ms | 52.3% bf16 MFU | 1624694 tok/s
step 10736/19560 | loss 3.358791 (-0.87z)| norm 0.2746 (-0.30z)| lr 2.70e-04 | 323.00 ms | 52.3% bf16 MFU | 1624619 tok/s
step 10737/19560 | loss 3.391263 (-0.20z)| norm 0.2509 (-1.64z)| lr 2.70e-04 | 322.86 ms | 52.3% bf16 MFU | 1624581 tok/s
step 10738/19560 | loss 3.477076 (+1.59z)| norm 0.2631 (-0.94z)| lr 2.70e-04 | 322.95 ms | 52.3% bf16 MFU | 1624523 tok/s
step 10739/19560 | loss 3.399070 (-0.04z)| norm 0.2771 (-0.15z)| lr 2.70e-04 | 323.01 ms | 52.2% bf16 MFU | 1624454 tok/s
step 10740/19560 | loss 3.386901 (-0.30z)| norm 0.2538 (-1.47z)| lr 2.70e-04 | 323.07 ms | 52.2% bf16 MFU | 1624374 tok/s
step 10741/19560 | loss 3.408275 (+0.14z)| norm 0.2808 (+0.06z)| lr 2.70e-04 | 322.26 ms | 52.4% bf16 MFU | 1624500 tok/s
step 10742/19560 | loss 3.436077 (+0.71z)| norm 0.2718 (-0.48z)| lr 2.70e-04 | 322.48 ms | 52.3% bf16 MFU | 1624564 tok/s
step 10743/19560 | loss 3.466893 (+1.35z)| norm 0.2594 (-1.21z)| lr 2.69e-04 | 323.40 ms | 52.2% bf16 MFU | 1624395 tok/s
step 10744/19560 | loss 3.332922 (-1.45z)| norm 0.2825 (+0.14z)| lr 2.69e-04 | 322.46 ms | 52.3% bf16 MFU | 1624471 tok/s
step 10745/19560 | loss 3.430089 (+0.59z)| norm 0.2720 (-0.48z)| lr 2.69e-04 | 322.30 ms | 52.4% bf16 MFU | 1624582 tok/s
step 10746/19560 | loss 3.352609 (-1.02z)| norm 0.3014 (+1.25z)| lr 2.69e-04 | 322.96 ms | 52.3% bf16 MFU | 1624521 tok/s
step 10747/19560 | loss 3.384270 (-0.35z)| norm 0.2866 (+0.37z)| lr 2.69e-04 | 323.01 ms | 52.2% bf16 MFU | 1624451 tok/s
step 10748/19560 | loss 3.396918 (-0.09z)| norm 0.2656 (-0.87z)| lr 2.69e-04 | 322.84 ms | 52.3% bf16 MFU | 1624428 tok/s
step 10749/19560 | loss 3.348980 (-1.09z)| norm 0.2798 (-0.04z)| lr 2.69e-04 | 322.54 ms | 52.3% bf16 MFU | 1624481 tok/s
step 10750/19560 | loss 3.479374 (+1.61z)| norm 0.2975 (+1.00z)| lr 2.69e-04 | 322.91 ms | 52.3% bf16 MFU | 1624440 tok/s
val loss 3.378517
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2950/10042 = 0.293766
step 10751/19560 | loss 3.475396 (+1.50z)| norm 0.2900 (+0.55z)| lr 2.69e-04 | 322.82 ms | 52.3% bf16 MFU | 1624422 tok/s
step 10752/19560 | loss 3.434999 (+0.65z)| norm 0.2632 (-1.03z)| lr 2.69e-04 | 322.90 ms | 52.3% bf16 MFU | 1624385 tok/s
step 10753/19560 | loss 3.501738 (+1.99z)| norm 0.2915 (+0.63z)| lr 2.69e-04 | 323.01 ms | 52.2% bf16 MFU | 1624322 tok/s
step 10754/19560 | loss 3.390269 (-0.28z)| norm 0.2694 (-0.68z)| lr 2.69e-04 | 323.46 ms | 52.2% bf16 MFU | 1624151 tok/s
step 10755/19560 | loss 3.416140 (+0.24z)| norm 0.2568 (-1.40z)| lr 2.69e-04 | 323.46 ms | 52.2% bf16 MFU | 1623986 tok/s
step 10756/19560 | loss 3.391645 (-0.25z)| norm 0.2684 (-0.72z)| lr 2.69e-04 | 322.95 ms | 52.3% bf16 MFU | 1623958 tok/s
step 10757/19560 | loss 3.472360 (+1.39z)| norm 0.2710 (-0.56z)| lr 2.69e-04 | 322.69 ms | 52.3% bf16 MFU | 1623999 tok/s
step 10758/19560 | loss 3.436030 (+0.64z)| norm 0.2652 (-0.92z)| lr 2.69e-04 | 323.17 ms | 52.2% bf16 MFU | 1623914 tok/s
step 10759/19560 | loss 3.381620 (-0.47z)| norm 0.2978 (+1.01z)| lr 2.69e-04 | 323.47 ms | 52.2% bf16 MFU | 1623760 tok/s
step 10760/19560 | loss 3.389561 (-0.32z)| norm 0.3004 (+1.15z)| lr 2.69e-04 | 323.08 ms | 52.2% bf16 MFU | 1623711 tok/s
step 10761/19560 | loss 3.322088 (-1.72z)| norm 0.2851 (+0.25z)| lr 2.69e-04 | 322.91 ms | 52.3% bf16 MFU | 1623708 tok/s
step 10762/19560 | loss 3.366170 (-0.78z)| norm 0.2641 (-0.98z)| lr 2.69e-04 | 322.63 ms | 52.3% bf16 MFU | 1623774 tok/s
step 10763/19560 | loss 3.374590 (-0.59z)| norm 0.2728 (-0.45z)| lr 2.68e-04 | 323.09 ms | 52.2% bf16 MFU | 1623722 tok/s
step 10764/19560 | loss 3.451791 (+1.04z)| norm 0.2771 (-0.18z)| lr 2.68e-04 | 322.51 ms | 52.3% bf16 MFU | 1623818 tok/s
step 10765/19560 | loss 3.414336 (+0.26z)| norm 0.2660 (-0.84z)| lr 2.68e-04 | 323.11 ms | 52.2% bf16 MFU | 1623758 tok/s
step 10766/19560 | loss 3.408113 (+0.13z)| norm 0.2794 (-0.02z)| lr 2.68e-04 | 322.87 ms | 52.3% bf16 MFU | 1623762 tok/s
step 10767/19560 | loss 3.317378 (-1.77z)| norm 0.2639 (-0.96z)| lr 2.68e-04 | 322.73 ms | 52.3% bf16 MFU | 1623801 tok/s
step 10768/19560 | loss 3.369987 (-0.65z)| norm 0.2465 (-1.99z)| lr 2.68e-04 | 322.73 ms | 52.3% bf16 MFU | 1623837 tok/s
step 10769/19560 | loss 3.387951 (-0.28z)| norm 0.2579 (-1.27z)| lr 2.68e-04 | 323.06 ms | 52.2% bf16 MFU | 1623789 tok/s
step 10770/19560 | loss 3.386754 (-0.29z)| norm 0.2525 (-1.58z)| lr 2.68e-04 | 322.83 ms | 52.3% bf16 MFU | 1623803 tok/s
step 10771/19560 | loss 3.393100 (-0.13z)| norm 0.2564 (-1.32z)| lr 2.68e-04 | 323.00 ms | 52.3% bf16 MFU | 1623770 tok/s
step 10772/19560 | loss 3.400262 (+0.04z)| norm 0.2701 (-0.49z)| lr 2.68e-04 | 323.30 ms | 52.2% bf16 MFU | 1623665 tok/s
step 10773/19560 | loss 3.363621 (-0.82z)| norm 0.2529 (-1.49z)| lr 2.68e-04 | 323.32 ms | 52.2% bf16 MFU | 1623561 tok/s
step 10774/19560 | loss 3.336144 (-1.45z)| norm 0.2893 (+0.67z)| lr 2.68e-04 | 323.05 ms | 52.2% bf16 MFU | 1623529 tok/s
step 10775/19560 | loss 3.426497 (+0.65z)| norm 0.2712 (-0.40z)| lr 2.68e-04 | 323.22 ms | 52.2% bf16 MFU | 1623458 tok/s
step 10776/19560 | loss 3.345715 (-1.22z)| norm 0.2649 (-0.77z)| lr 2.68e-04 | 323.16 ms | 52.2% bf16 MFU | 1623404 tok/s
step 10777/19560 | loss 3.328927 (-1.62z)| norm 0.2980 (+1.20z)| lr 2.68e-04 | 322.66 ms | 52.3% bf16 MFU | 1623479 tok/s
step 10778/19560 | loss 3.389166 (-0.18z)| norm 0.2687 (-0.53z)| lr 2.68e-04 | 322.92 ms | 52.3% bf16 MFU | 1623484 tok/s
step 10779/19560 | loss 3.367229 (-0.71z)| norm 0.2895 (+0.71z)| lr 2.68e-04 | 322.89 ms | 52.3% bf16 MFU | 1623497 tok/s
step 10780/19560 | loss 3.308817 (-2.05z)| norm 0.2660 (-0.70z)| lr 2.68e-04 | 323.36 ms | 52.2% bf16 MFU | 1623391 tok/s
step 10781/19560 | loss 3.342303 (-1.26z)| norm 0.2827 (+0.31z)| lr 2.68e-04 | 322.82 ms | 52.3% bf16 MFU | 1623425 tok/s
step 10782/19560 | loss 3.368060 (-0.65z)| norm 0.2609 (-1.00z)| lr 2.68e-04 | 322.60 ms | 52.3% bf16 MFU | 1623514 tok/s
step 10783/19560 | loss 3.420115 (+0.56z)| norm 0.2749 (-0.14z)| lr 2.67e-04 | 323.30 ms | 52.2% bf16 MFU | 1623423 tok/s
step 10784/19560 | loss 3.383595 (-0.28z)| norm 0.2742 (-0.18z)| lr 2.67e-04 | 322.66 ms | 52.3% bf16 MFU | 1623495 tok/s
step 10785/19560 | loss 3.387244 (-0.20z)| norm 0.2965 (+1.20z)| lr 2.67e-04 | 322.87 ms | 52.3% bf16 MFU | 1623512 tok/s
step 10786/19560 | loss 3.414399 (+0.43z)| norm 0.2661 (-0.66z)| lr 2.67e-04 | 322.70 ms | 52.3% bf16 MFU | 1623571 tok/s
step 10787/19560 | loss 3.360445 (-0.82z)| norm 0.2842 (+0.46z)| lr 2.67e-04 | 323.25 ms | 52.2% bf16 MFU | 1623488 tok/s
step 10788/19560 | loss 3.388759 (-0.17z)| norm 0.2683 (-0.52z)| lr 2.67e-04 | 323.32 ms | 52.2% bf16 MFU | 1623393 tok/s
step 10789/19560 | loss 3.434206 (+0.89z)| norm 0.2766 (-0.01z)| lr 2.67e-04 | 322.63 ms | 52.3% bf16 MFU | 1623476 tok/s
step 10790/19560 | loss 3.527509 (+2.96z)| norm 0.3062 (+1.81z)| lr 2.67e-04 | 323.62 ms | 52.2% bf16 MFU | 1623305 tok/s
step 10791/19560 | loss 3.387893 (-0.20z)| norm 0.2820 (+0.33z)| lr 2.67e-04 | 322.77 ms | 52.3% bf16 MFU | 1623358 tok/s
step 10792/19560 | loss 3.387521 (-0.20z)| norm 0.2861 (+0.59z)| lr 2.67e-04 | 323.83 ms | 52.1% bf16 MFU | 1623142 tok/s
step 10793/19560 | loss 3.415329 (+0.43z)| norm 0.3120 (+2.14z)| lr 2.67e-04 | 323.32 ms | 52.2% bf16 MFU | 1623063 tok/s
step 10794/19560 | loss 3.407875 (+0.25z)| norm 0.2832 (+0.39z)| lr 2.67e-04 | 322.94 ms | 52.3% bf16 MFU | 1623084 tok/s
step 10795/19560 | loss 3.411982 (+0.35z)| norm 0.2961 (+1.17z)| lr 2.67e-04 | 323.79 ms | 52.1% bf16 MFU | 1622890 tok/s
step 10796/19560 | loss 3.388630 (-0.17z)| norm 0.2860 (+0.56z)| lr 2.67e-04 | 322.37 ms | 52.4% bf16 MFU | 1623063 tok/s
step 10797/19560 | loss 3.413072 (+0.38z)| norm 0.2875 (+0.64z)| lr 2.67e-04 | 323.99 ms | 52.1% bf16 MFU | 1622822 tok/s
step 10798/19560 | loss 3.301751 (-2.13z)| norm 0.2869 (+0.60z)| lr 2.67e-04 | 323.66 ms | 52.1% bf16 MFU | 1622675 tok/s
step 10799/19560 | loss 3.331262 (-1.45z)| norm 0.3050 (+1.68z)| lr 2.67e-04 | 323.16 ms | 52.2% bf16 MFU | 1622659 tok/s
step 10800/19560 | loss 3.385589 (-0.22z)| norm 0.2824 (+0.31z)| lr 2.67e-04 | 323.30 ms | 52.2% bf16 MFU | 1622611 tok/s
step 10801/19560 | loss 3.340526 (-1.23z)| norm 0.2814 (+0.24z)| lr 2.67e-04 | 322.82 ms | 52.3% bf16 MFU | 1622686 tok/s
step 10802/19560 | loss 3.370793 (-0.54z)| norm 0.2857 (+0.49z)| lr 2.67e-04 | 323.64 ms | 52.1% bf16 MFU | 1622550 tok/s
step 10803/19560 | loss 3.408600 (+0.31z)| norm 0.2819 (+0.26z)| lr 2.66e-04 | 322.85 ms | 52.3% bf16 MFU | 1622620 tok/s
step 10804/19560 | loss 3.444967 (+1.11z)| norm 0.2876 (+0.60z)| lr 2.66e-04 | 323.45 ms | 52.2% bf16 MFU | 1622536 tok/s
step 10805/19560 | loss 3.381202 (-0.31z)| norm 0.2782 (+0.02z)| lr 2.66e-04 | 324.24 ms | 52.1% bf16 MFU | 1622257 tok/s
step 10806/19560 | loss 3.382278 (-0.27z)| norm 0.2937 (+0.96z)| lr 2.66e-04 | 322.29 ms | 52.4% bf16 MFU | 1622482 tok/s
step 10807/19560 | loss 3.323643 (-1.57z)| norm 0.2713 (-0.41z)| lr 2.66e-04 | 323.11 ms | 52.2% bf16 MFU | 1622490 tok/s
step 10808/19560 | loss 3.412133 (+0.39z)| norm 0.3076 (+1.78z)| lr 2.66e-04 | 324.25 ms | 52.1% bf16 MFU | 1622212 tok/s
step 10809/19560 | loss 3.388397 (-0.14z)| norm 0.2887 (+0.61z)| lr 2.66e-04 | 322.87 ms | 52.3% bf16 MFU | 1622293 tok/s
step 10810/19560 | loss 3.419570 (+0.55z)| norm 0.2948 (+0.97z)| lr 2.66e-04 | 322.37 ms | 52.4% bf16 MFU | 1622497 tok/s
step 10811/19560 | loss 3.404265 (+0.21z)| norm 0.2721 (-0.42z)| lr 2.66e-04 | 323.51 ms | 52.2% bf16 MFU | 1622404 tok/s
step 10812/19560 | loss 3.373948 (-0.47z)| norm 0.3166 (+2.25z)| lr 2.66e-04 | 323.51 ms | 52.2% bf16 MFU | 1622315 tok/s
step 10813/19560 | loss 3.383276 (-0.27z)| norm 0.2867 (+0.44z)| lr 2.66e-04 | 322.84 ms | 52.3% bf16 MFU | 1622398 tok/s
step 10814/19560 | loss 3.356490 (-0.86z)| norm 0.2955 (+0.97z)| lr 2.66e-04 | 323.06 ms | 52.2% bf16 MFU | 1622422 tok/s
step 10815/19560 | loss 3.368933 (-0.58z)| norm 0.2694 (-0.60z)| lr 2.66e-04 | 322.77 ms | 52.3% bf16 MFU | 1622517 tok/s
step 10816/19560 | loss 3.314888 (-1.75z)| norm 0.3036 (+1.43z)| lr 2.66e-04 | 323.14 ms | 52.2% bf16 MFU | 1622515 tok/s
step 10817/19560 | loss 3.421405 (+0.60z)| norm 0.2882 (+0.49z)| lr 2.66e-04 | 323.03 ms | 52.2% bf16 MFU | 1622542 tok/s
step 10818/19560 | loss 3.373011 (-0.46z)| norm 0.3161 (+2.13z)| lr 2.66e-04 | 322.82 ms | 52.3% bf16 MFU | 1622620 tok/s
step 10819/19560 | loss 3.500968 (+2.31z)| norm 0.2716 (-0.54z)| lr 2.66e-04 | 322.81 ms | 52.3% bf16 MFU | 1622697 tok/s
step 10820/19560 | loss 3.401895 (+0.15z)| norm 0.3023 (+1.29z)| lr 2.66e-04 | 322.88 ms | 52.3% bf16 MFU | 1622751 tok/s
step 10821/19560 | loss 3.367666 (-0.59z)| norm 0.2821 (+0.07z)| lr 2.66e-04 | 323.38 ms | 52.2% bf16 MFU | 1622676 tok/s
step 10822/19560 | loss 3.420724 (+0.56z)| norm 0.2628 (-1.07z)| lr 2.66e-04 | 322.80 ms | 52.3% bf16 MFU | 1622753 tok/s
step 10823/19560 | loss 3.436229 (+0.89z)| norm 0.2805 (-0.02z)| lr 2.65e-04 | 322.56 ms | 52.3% bf16 MFU | 1622886 tok/s
step 10824/19560 | loss 3.360623 (-0.76z)| norm 0.2751 (-0.34z)| lr 2.65e-04 | 322.74 ms | 52.3% bf16 MFU | 1622966 tok/s
step 10825/19560 | loss 3.440873 (+0.99z)| norm 0.2743 (-0.38z)| lr 2.65e-04 | 323.43 ms | 52.2% bf16 MFU | 1622869 tok/s
step 10826/19560 | loss 3.368038 (-0.59z)| norm 0.2642 (-0.99z)| lr 2.65e-04 | 322.58 ms | 52.3% bf16 MFU | 1622989 tok/s
step 10827/19560 | loss 3.380231 (-0.33z)| norm 0.2796 (-0.07z)| lr 2.65e-04 | 323.18 ms | 52.2% bf16 MFU | 1622955 tok/s
step 10828/19560 | loss 3.358214 (-0.80z)| norm 0.2626 (-1.08z)| lr 2.65e-04 | 322.46 ms | 52.3% bf16 MFU | 1623102 tok/s
step 10829/19560 | loss 3.393135 (-0.03z)| norm 0.2735 (-0.41z)| lr 2.65e-04 | 322.74 ms | 52.3% bf16 MFU | 1623172 tok/s
step 10830/19560 | loss 3.377544 (-0.37z)| norm 0.2790 (-0.07z)| lr 2.65e-04 | 323.11 ms | 52.2% bf16 MFU | 1623145 tok/s
step 10831/19560 | loss 3.372752 (-0.47z)| norm 0.2699 (-0.62z)| lr 2.65e-04 | 322.78 ms | 52.3% bf16 MFU | 1623202 tok/s
step 10832/19560 | loss 3.371239 (-0.50z)| norm 0.2757 (-0.25z)| lr 2.65e-04 | 323.15 ms | 52.2% bf16 MFU | 1623163 tok/s
step 10833/19560 | loss 3.390530 (-0.07z)| norm 0.2659 (-0.85z)| lr 2.65e-04 | 322.54 ms | 52.3% bf16 MFU | 1623279 tok/s
step 10834/19560 | loss 3.357924 (-0.78z)| norm 0.2711 (-0.52z)| lr 2.65e-04 | 322.82 ms | 52.3% bf16 MFU | 1623320 tok/s
step 10835/19560 | loss 3.384309 (-0.20z)| norm 0.2866 (+0.45z)| lr 2.65e-04 | 322.12 ms | 52.4% bf16 MFU | 1623535 tok/s
step 10836/19560 | loss 3.387490 (-0.12z)| norm 0.2937 (+0.89z)| lr 2.65e-04 | 323.71 ms | 52.1% bf16 MFU | 1623339 tok/s
step 10837/19560 | loss 3.362585 (-0.66z)| norm 0.2567 (-1.40z)| lr 2.65e-04 | 323.01 ms | 52.2% bf16 MFU | 1623329 tok/s
step 10838/19560 | loss 3.540397 (+3.13z)| norm 0.2685 (-0.67z)| lr 2.65e-04 | 322.63 ms | 52.3% bf16 MFU | 1623414 tok/s
step 10839/19560 | loss 3.356410 (-0.78z)| norm 0.3137 (+2.16z)| lr 2.65e-04 | 322.85 ms | 52.3% bf16 MFU | 1623440 tok/s
step 10840/19560 | loss 3.401062 (+0.19z)| norm 0.3095 (+1.85z)| lr 2.65e-04 | 322.80 ms | 52.3% bf16 MFU | 1623478 tok/s
step 10841/19560 | loss 3.420581 (+0.61z)| norm 0.2957 (+0.99z)| lr 2.65e-04 | 322.92 ms | 52.3% bf16 MFU | 1623482 tok/s
step 10842/19560 | loss 3.429640 (+0.80z)| norm 0.2745 (-0.32z)| lr 2.65e-04 | 322.90 ms | 52.3% bf16 MFU | 1623492 tok/s
step 10843/19560 | loss 3.352988 (-0.87z)| norm 0.3672 (+4.84z)| lr 2.65e-04 | 322.55 ms | 52.3% bf16 MFU | 1623590 tok/s
step 10844/19560 | loss 3.367725 (-0.54z)| norm 0.3137 (+1.81z)| lr 2.64e-04 | 323.05 ms | 52.2% bf16 MFU | 1623557 tok/s
step 10845/19560 | loss 3.408656 (+0.34z)| norm 0.3281 (+2.52z)| lr 2.64e-04 | 323.02 ms | 52.2% bf16 MFU | 1623534 tok/s
step 10846/19560 | loss 3.378869 (-0.32z)| norm 0.2847 (+0.21z)| lr 2.64e-04 | 322.93 ms | 52.3% bf16 MFU | 1623535 tok/s
step 10847/19560 | loss 3.313420 (-1.74z)| norm 0.3043 (+1.28z)| lr 2.64e-04 | 322.84 ms | 52.3% bf16 MFU | 1623556 tok/s
step 10848/19560 | loss 3.333252 (-1.28z)| norm 0.2734 (-0.40z)| lr 2.64e-04 | 323.84 ms | 52.1% bf16 MFU | 1623326 tok/s
step 10849/19560 | loss 3.441890 (+1.11z)| norm 0.2928 (+0.67z)| lr 2.64e-04 | 322.36 ms | 52.4% bf16 MFU | 1623480 tok/s
step 10850/19560 | loss 3.365376 (-0.57z)| norm 0.3062 (+1.40z)| lr 2.64e-04 | 322.66 ms | 52.3% bf16 MFU | 1623551 tok/s
step 10851/19560 | loss 3.380898 (-0.22z)| norm 0.2649 (-0.87z)| lr 2.64e-04 | 322.68 ms | 52.3% bf16 MFU | 1623612 tok/s
step 10852/19560 | loss 3.472203 (+1.78z)| norm 0.3088 (+1.53z)| lr 2.64e-04 | 322.98 ms | 52.3% bf16 MFU | 1623594 tok/s
step 10853/19560 | loss 3.424512 (+0.71z)| norm 0.2679 (-0.73z)| lr 2.64e-04 | 322.67 ms | 52.3% bf16 MFU | 1623656 tok/s
step 10854/19560 | loss 3.597891 (+4.20z)| norm 0.4598 (+7.40z)| lr 2.64e-04 | 322.80 ms | 52.3% bf16 MFU | 1623684 tok/s
step 10855/19560 | loss 3.364655 (-0.61z)| norm 0.3146 (+1.31z)| lr 2.64e-04 | 322.83 ms | 52.3% bf16 MFU | 1623702 tok/s
step 10856/19560 | loss 3.420402 (+0.58z)| norm 0.3818 (+3.84z)| lr 2.64e-04 | 322.46 ms | 52.3% bf16 MFU | 1623813 tok/s
step 10857/19560 | loss 3.469375 (+1.59z)| norm 0.3224 (+1.49z)| lr 2.64e-04 | 322.75 ms | 52.3% bf16 MFU | 1623844 tok/s
step 10858/19560 | loss 3.417418 (+0.48z)| norm 0.3284 (+1.69z)| lr 2.64e-04 | 323.18 ms | 52.2% bf16 MFU | 1623766 tok/s
step 10859/19560 | loss 3.366975 (-0.58z)| norm 0.3074 (+0.88z)| lr 2.64e-04 | 322.79 ms | 52.3% bf16 MFU | 1623788 tok/s
step 10860/19560 | loss 3.396113 (+0.04z)| norm 0.2970 (+0.48z)| lr 2.64e-04 | 322.54 ms | 52.3% bf16 MFU | 1623874 tok/s
step 10861/19560 | loss 3.361443 (-0.69z)| norm 0.3032 (+0.71z)| lr 2.64e-04 | 322.95 ms | 52.3% bf16 MFU | 1623852 tok/s
step 10862/19560 | loss 3.315992 (-1.63z)| norm 0.2881 (+0.12z)| lr 2.64e-04 | 322.70 ms | 52.3% bf16 MFU | 1623893 tok/s
step 10863/19560 | loss 3.411374 (+0.36z)| norm 0.2888 (+0.14z)| lr 2.64e-04 | 323.24 ms | 52.2% bf16 MFU | 1623796 tok/s
step 10864/19560 | loss 3.430251 (+0.75z)| norm 0.2937 (+0.33z)| lr 2.63e-04 | 322.86 ms | 52.3% bf16 MFU | 1623801 tok/s
step 10865/19560 | loss 3.299731 (-1.96z)| norm 0.3047 (+0.74z)| lr 2.63e-04 | 323.24 ms | 52.2% bf16 MFU | 1623711 tok/s
step 10866/19560 | loss 3.424506 (+0.65z)| norm 0.2688 (-0.66z)| lr 2.63e-04 | 322.66 ms | 52.3% bf16 MFU | 1623769 tok/s
step 10867/19560 | loss 3.434247 (+0.84z)| norm 0.2887 (+0.11z)| lr 2.63e-04 | 323.02 ms | 52.2% bf16 MFU | 1623734 tok/s
step 10868/19560 | loss 3.312038 (-1.68z)| norm 0.2641 (-0.85z)| lr 2.63e-04 | 322.40 ms | 52.3% bf16 MFU | 1623857 tok/s
step 10869/19560 | loss 3.389585 (-0.08z)| norm 0.2655 (-0.79z)| lr 2.63e-04 | 322.77 ms | 52.3% bf16 MFU | 1623882 tok/s
step 10870/19560 | loss 3.486928 (+1.91z)| norm 0.2805 (-0.21z)| lr 2.63e-04 | 322.44 ms | 52.3% bf16 MFU | 1623986 tok/s
step 10871/19560 | loss 3.388424 (-0.09z)| norm 0.2570 (-1.12z)| lr 2.63e-04 | 322.90 ms | 52.3% bf16 MFU | 1623971 tok/s
step 10872/19560 | loss 3.377426 (-0.33z)| norm 0.2716 (-0.55z)| lr 2.63e-04 | 322.80 ms | 52.3% bf16 MFU | 1623982 tok/s
step 10873/19560 | loss 3.351853 (-0.85z)| norm 0.2681 (-0.68z)| lr 2.63e-04 | 322.96 ms | 52.3% bf16 MFU | 1623952 tok/s
step 10874/19560 | loss 3.406264 (+0.27z)| norm 0.2600 (-0.99z)| lr 2.63e-04 | 322.89 ms | 52.3% bf16 MFU | 1623942 tok/s
step 10875/19560 | loss 3.365117 (-0.58z)| norm 0.2633 (-0.85z)| lr 2.63e-04 | 322.56 ms | 52.3% bf16 MFU | 1624016 tok/s
step 10876/19560 | loss 3.429837 (+0.76z)| norm 0.2502 (-1.34z)| lr 2.63e-04 | 323.46 ms | 52.2% bf16 MFU | 1623858 tok/s
step 10877/19560 | loss 3.522551 (+2.59z)| norm 0.2677 (-0.67z)| lr 2.63e-04 | 322.83 ms | 52.3% bf16 MFU | 1623866 tok/s
step 10878/19560 | loss 3.470643 (+1.54z)| norm 0.2696 (-0.58z)| lr 2.63e-04 | 322.68 ms | 52.3% bf16 MFU | 1623913 tok/s
step 10879/19560 | loss 3.358827 (-0.71z)| norm 0.2703 (-0.55z)| lr 2.63e-04 | 322.96 ms | 52.3% bf16 MFU | 1623886 tok/s
step 10880/19560 | loss 3.432891 (+0.80z)| norm 0.2883 (+0.13z)| lr 2.63e-04 | 322.89 ms | 52.3% bf16 MFU | 1623878 tok/s
step 10881/19560 | loss 3.412041 (+0.40z)| norm 0.2770 (-0.29z)| lr 2.63e-04 | 322.50 ms | 52.3% bf16 MFU | 1623970 tok/s
step 10882/19560 | loss 3.372268 (-0.43z)| norm 0.2851 (+0.01z)| lr 2.63e-04 | 323.32 ms | 52.2% bf16 MFU | 1623850 tok/s
step 10883/19560 | loss 3.362860 (-0.62z)| norm 0.2879 (+0.11z)| lr 2.63e-04 | 323.06 ms | 52.2% bf16 MFU | 1623803 tok/s
step 10884/19560 | loss 3.368454 (-0.49z)| norm 0.2866 (+0.05z)| lr 2.62e-04 | 323.01 ms | 52.2% bf16 MFU | 1623769 tok/s
step 10885/19560 | loss 3.378842 (-0.27z)| norm 0.2707 (-0.56z)| lr 2.62e-04 | 322.69 ms | 52.3% bf16 MFU | 1623817 tok/s
step 10886/19560 | loss 3.564500 (+3.46z)| norm 0.3499 (+2.43z)| lr 2.62e-04 | 322.68 ms | 52.3% bf16 MFU | 1623864 tok/s
step 10887/19560 | loss 3.425547 (+0.66z)| norm 0.3145 (+1.08z)| lr 2.62e-04 | 323.10 ms | 52.2% bf16 MFU | 1623806 tok/s
step 10888/19560 | loss 3.379311 (-0.27z)| norm 0.2935 (+0.29z)| lr 2.62e-04 | 322.91 ms | 52.3% bf16 MFU | 1623797 tok/s
step 10889/19560 | loss 3.357249 (-0.72z)| norm 0.2869 (+0.04z)| lr 2.62e-04 | 322.48 ms | 52.3% bf16 MFU | 1623897 tok/s
step 10890/19560 | loss 3.366346 (-0.54z)| norm 0.2849 (-0.05z)| lr 2.62e-04 | 323.05 ms | 52.2% bf16 MFU | 1623848 tok/s
step 10891/19560 | loss 3.381555 (-0.23z)| norm 0.2738 (-0.47z)| lr 2.62e-04 | 322.95 ms | 52.3% bf16 MFU | 1623829 tok/s
step 10892/19560 | loss 3.360115 (-0.65z)| norm 0.2773 (-0.33z)| lr 2.62e-04 | 322.79 ms | 52.3% bf16 MFU | 1623849 tok/s
step 10893/19560 | loss 3.466302 (+1.48z)| norm 0.2779 (-0.32z)| lr 2.62e-04 | 322.50 ms | 52.3% bf16 MFU | 1623940 tok/s
step 10894/19560 | loss 3.417287 (+0.49z)| norm 0.2695 (-0.63z)| lr 2.62e-04 | 323.17 ms | 52.2% bf16 MFU | 1623860 tok/s
step 10895/19560 | loss 3.391749 (-0.03z)| norm 0.2980 (+0.44z)| lr 2.62e-04 | 322.86 ms | 52.3% bf16 MFU | 1623862 tok/s
step 10896/19560 | loss 3.356591 (-0.74z)| norm 0.2885 (+0.07z)| lr 2.62e-04 | 322.51 ms | 52.3% bf16 MFU | 1623951 tok/s
step 10897/19560 | loss 3.475636 (+1.63z)| norm 0.2815 (-0.21z)| lr 2.62e-04 | 323.52 ms | 52.2% bf16 MFU | 1623782 tok/s
step 10898/19560 | loss 3.505582 (+2.17z)| norm 0.2954 (+0.32z)| lr 2.62e-04 | 322.89 ms | 52.3% bf16 MFU | 1623780 tok/s
step 10899/19560 | loss 3.382524 (-0.24z)| norm 0.2766 (-0.42z)| lr 2.62e-04 | 322.69 ms | 52.3% bf16 MFU | 1623828 tok/s
step 10900/19560 | loss 3.377697 (-0.33z)| norm 0.2763 (-0.44z)| lr 2.62e-04 | 322.37 ms | 52.4% bf16 MFU | 1623954 tok/s
step 10901/19560 | loss 3.382364 (-0.24z)| norm 0.2596 (-1.09z)| lr 2.62e-04 | 323.49 ms | 52.2% bf16 MFU | 1623792 tok/s
step 10902/19560 | loss 3.463415 (+1.33z)| norm 0.2858 (-0.07z)| lr 2.62e-04 | 323.01 ms | 52.2% bf16 MFU | 1623758 tok/s
step 10903/19560 | loss 3.332949 (-1.21z)| norm 0.2698 (-0.69z)| lr 2.62e-04 | 323.30 ms | 52.2% bf16 MFU | 1623655 tok/s
step 10904/19560 | loss 3.445967 (+0.98z)| norm 0.2812 (-0.25z)| lr 2.61e-04 | 322.83 ms | 52.3% bf16 MFU | 1623674 tok/s
step 10905/19560 | loss 3.398821 (+0.05z)| norm 0.2595 (-1.09z)| lr 2.61e-04 | 322.51 ms | 52.3% bf16 MFU | 1623772 tok/s
step 10906/19560 | loss 3.357802 (-0.75z)| norm 0.2773 (-0.39z)| lr 2.61e-04 | 323.18 ms | 52.2% bf16 MFU | 1623698 tok/s
step 10907/19560 | loss 3.376558 (-0.39z)| norm 0.3491 (+2.35z)| lr 2.61e-04 | 322.83 ms | 52.3% bf16 MFU | 1623716 tok/s
step 10908/19560 | loss 3.378163 (-0.37z)| norm 0.2838 (-0.16z)| lr 2.61e-04 | 322.47 ms | 52.3% bf16 MFU | 1623822 tok/s
step 10909/19560 | loss 3.360914 (-0.72z)| norm 0.2690 (-0.72z)| lr 2.61e-04 | 323.41 ms | 52.2% bf16 MFU | 1623686 tok/s
step 10910/19560 | loss 3.378889 (-0.36z)| norm 0.2852 (-0.11z)| lr 2.61e-04 | 322.57 ms | 52.3% bf16 MFU | 1623769 tok/s
step 10911/19560 | loss 3.328840 (-1.34z)| norm 0.2694 (-0.72z)| lr 2.61e-04 | 322.73 ms | 52.3% bf16 MFU | 1623808 tok/s
step 10912/19560 | loss 3.463553 (+1.31z)| norm 0.3005 (+0.47z)| lr 2.61e-04 | 322.57 ms | 52.3% bf16 MFU | 1623884 tok/s
step 10913/19560 | loss 3.460267 (+1.23z)| norm 0.2626 (-0.97z)| lr 2.61e-04 | 323.02 ms | 52.2% bf16 MFU | 1623844 tok/s
step 10914/19560 | loss 3.312918 (-1.62z)| norm 0.2784 (-0.37z)| lr 2.61e-04 | 323.13 ms | 52.2% bf16 MFU | 1623778 tok/s
step 10915/19560 | loss 3.401324 (+0.08z)| norm 0.2645 (-0.90z)| lr 2.61e-04 | 322.81 ms | 52.3% bf16 MFU | 1623797 tok/s
step 10916/19560 | loss 3.405751 (+0.17z)| norm 0.2833 (-0.18z)| lr 2.61e-04 | 322.93 ms | 52.3% bf16 MFU | 1623785 tok/s
step 10917/19560 | loss 3.405307 (+0.16z)| norm 0.2748 (-0.51z)| lr 2.61e-04 | 323.01 ms | 52.3% bf16 MFU | 1623753 tok/s
step 10918/19560 | loss 3.336568 (-1.17z)| norm 0.2988 (+0.42z)| lr 2.61e-04 | 322.47 ms | 52.3% bf16 MFU | 1623856 tok/s
step 10919/19560 | loss 3.353712 (-0.82z)| norm 0.2748 (-0.50z)| lr 2.61e-04 | 323.11 ms | 52.2% bf16 MFU | 1623795 tok/s
step 10920/19560 | loss 3.397216 (+0.04z)| norm 0.2788 (-0.34z)| lr 2.61e-04 | 322.73 ms | 52.3% bf16 MFU | 1623832 tok/s
step 10921/19560 | loss 3.431175 (+0.71z)| norm 0.2805 (-0.27z)| lr 2.61e-04 | 323.18 ms | 52.2% bf16 MFU | 1623753 tok/s
step 10922/19560 | loss 3.342671 (-1.03z)| norm 0.2546 (-1.25z)| lr 2.61e-04 | 322.77 ms | 52.3% bf16 MFU | 1623782 tok/s
step 10923/19560 | loss 3.405630 (+0.21z)| norm 0.2785 (-0.33z)| lr 2.61e-04 | 322.22 ms | 52.4% bf16 MFU | 1623950 tok/s
step 10924/19560 | loss 3.409636 (+0.29z)| norm 0.2812 (-0.23z)| lr 2.60e-04 | 323.41 ms | 52.2% bf16 MFU | 1623809 tok/s
step 10925/19560 | loss 3.476847 (+1.59z)| norm 0.2607 (-1.00z)| lr 2.60e-04 | 322.38 ms | 52.4% bf16 MFU | 1623935 tok/s
step 10926/19560 | loss 3.418782 (+0.44z)| norm 0.2533 (-1.26z)| lr 2.60e-04 | 323.62 ms | 52.2% bf16 MFU | 1623741 tok/s
step 10927/19560 | loss 3.352304 (-0.88z)| norm 0.2652 (-0.80z)| lr 2.60e-04 | 322.50 ms | 52.3% bf16 MFU | 1623838 tok/s
step 10928/19560 | loss 3.406275 (+0.19z)| norm 0.2817 (-0.18z)| lr 2.60e-04 | 322.48 ms | 52.3% bf16 MFU | 1623936 tok/s
step 10929/19560 | loss 3.382541 (-0.29z)| norm 0.2920 (+0.21z)| lr 2.60e-04 | 323.49 ms | 52.2% bf16 MFU | 1623774 tok/s
step 10930/19560 | loss 3.409341 (+0.24z)| norm 0.2808 (-0.21z)| lr 2.60e-04 | 322.34 ms | 52.4% bf16 MFU | 1623910 tok/s
step 10931/19560 | loss 3.417996 (+0.41z)| norm 0.2534 (-1.24z)| lr 2.60e-04 | 322.53 ms | 52.3% bf16 MFU | 1623992 tok/s
step 10932/19560 | loss 3.398843 (+0.04z)| norm 0.2802 (-0.23z)| lr 2.60e-04 | 323.18 ms | 52.2% bf16 MFU | 1623905 tok/s
step 10933/19560 | loss 3.395838 (-0.03z)| norm 0.2569 (-1.09z)| lr 2.60e-04 | 322.86 ms | 52.3% bf16 MFU | 1623904 tok/s
step 10934/19560 | loss 3.416376 (+0.38z)| norm 0.2755 (-0.39z)| lr 2.60e-04 | 322.27 ms | 52.4% bf16 MFU | 1624051 tok/s
step 10935/19560 | loss 3.368089 (-0.60z)| norm 0.2512 (-1.28z)| lr 2.60e-04 | 322.80 ms | 52.3% bf16 MFU | 1624057 tok/s
step 10936/19560 | loss 3.385764 (-0.24z)| norm 0.2768 (-0.32z)| lr 2.60e-04 | 323.10 ms | 52.2% bf16 MFU | 1623987 tok/s
step 10937/19560 | loss 3.453818 (+1.12z)| norm 0.2637 (-0.80z)| lr 2.60e-04 | 323.30 ms | 52.2% bf16 MFU | 1623873 tok/s
step 10938/19560 | loss 3.382271 (-0.31z)| norm 0.2758 (-0.35z)| lr 2.60e-04 | 323.31 ms | 52.2% bf16 MFU | 1623761 tok/s
step 10939/19560 | loss 3.347185 (-1.00z)| norm 0.2688 (-0.61z)| lr 2.60e-04 | 322.46 ms | 52.3% bf16 MFU | 1623867 tok/s
step 10940/19560 | loss 3.382258 (-0.30z)| norm 0.2885 (+0.14z)| lr 2.60e-04 | 322.98 ms | 52.3% bf16 MFU | 1623839 tok/s
step 10941/19560 | loss 3.387803 (-0.19z)| norm 0.2778 (-0.26z)| lr 2.60e-04 | 323.06 ms | 52.2% bf16 MFU | 1623792 tok/s
step 10942/19560 | loss 3.391911 (-0.12z)| norm 0.2609 (-0.88z)| lr 2.60e-04 | 322.69 ms | 52.3% bf16 MFU | 1623840 tok/s
step 10943/19560 | loss 3.340930 (-1.13z)| norm 0.2848 (+0.00z)| lr 2.60e-04 | 323.62 ms | 52.2% bf16 MFU | 1623653 tok/s
step 10944/19560 | loss 3.367364 (-0.62z)| norm 0.2790 (-0.20z)| lr 2.59e-04 | 322.22 ms | 52.4% bf16 MFU | 1623826 tok/s
step 10945/19560 | loss 3.353403 (-0.88z)| norm 0.2698 (-0.54z)| lr 2.59e-04 | 322.76 ms | 52.3% bf16 MFU | 1623853 tok/s
step 10946/19560 | loss 3.560903 (+3.13z)| norm 0.2849 (+0.03z)| lr 2.59e-04 | 323.57 ms | 52.2% bf16 MFU | 1623676 tok/s
step 10947/19560 | loss 3.424632 (+0.52z)| norm 0.2851 (+0.04z)| lr 2.59e-04 | 322.86 ms | 52.3% bf16 MFU | 1623685 tok/s
step 10948/19560 | loss 3.372797 (-0.49z)| norm 0.2709 (-0.49z)| lr 2.59e-04 | 322.70 ms | 52.3% bf16 MFU | 1623736 tok/s
step 10949/19560 | loss 3.443588 (+0.88z)| norm 0.2957 (+0.44z)| lr 2.59e-04 | 322.39 ms | 52.4% bf16 MFU | 1623861 tok/s
step 10950/19560 | loss 3.379422 (-0.37z)| norm 0.2771 (-0.27z)| lr 2.59e-04 | 323.34 ms | 52.2% bf16 MFU | 1623743 tok/s
step 10951/19560 | loss 3.359477 (-0.75z)| norm 0.2785 (-0.21z)| lr 2.59e-04 | 322.62 ms | 52.3% bf16 MFU | 1623812 tok/s
step 10952/19560 | loss 3.426171 (+0.55z)| norm 0.2962 (+0.45z)| lr 2.59e-04 | 322.64 ms | 52.3% bf16 MFU | 1623870 tok/s
step 10953/19560 | loss 3.402125 (+0.08z)| norm 0.3020 (+0.66z)| lr 2.59e-04 | 322.90 ms | 52.3% bf16 MFU | 1623860 tok/s
step 10954/19560 | loss 3.398101 (-0.00z)| norm 0.2884 (+0.14z)| lr 2.59e-04 | 323.40 ms | 52.2% bf16 MFU | 1623725 tok/s
step 10955/19560 | loss 3.398775 (+0.01z)| norm 0.2885 (+0.14z)| lr 2.59e-04 | 323.05 ms | 52.2% bf16 MFU | 1623686 tok/s
step 10956/19560 | loss 3.406848 (+0.16z)| norm 0.2688 (-0.61z)| lr 2.59e-04 | 322.81 ms | 52.3% bf16 MFU | 1623708 tok/s
step 10957/19560 | loss 3.441510 (+0.84z)| norm 0.2826 (-0.09z)| lr 2.59e-04 | 322.56 ms | 52.3% bf16 MFU | 1623792 tok/s
step 10958/19560 | loss 3.359668 (-0.77z)| norm 0.2657 (-0.72z)| lr 2.59e-04 | 323.07 ms | 52.2% bf16 MFU | 1623744 tok/s
step 10959/19560 | loss 3.442680 (+0.85z)| norm 0.3063 (+0.80z)| lr 2.59e-04 | 323.29 ms | 52.2% bf16 MFU | 1623642 tok/s
step 10960/19560 | loss 3.431309 (+0.62z)| norm 0.3136 (+1.06z)| lr 2.59e-04 | 322.67 ms | 52.3% bf16 MFU | 1623703 tok/s
step 10961/19560 | loss 3.445880 (+0.89z)| norm 0.2882 (+0.10z)| lr 2.59e-04 | 322.79 ms | 52.3% bf16 MFU | 1623730 tok/s
step 10962/19560 | loss 3.415834 (+0.29z)| norm 0.2969 (+0.42z)| lr 2.59e-04 | 322.76 ms | 52.3% bf16 MFU | 1623763 tok/s
step 10963/19560 | loss 3.419170 (+0.35z)| norm 0.2846 (-0.04z)| lr 2.59e-04 | 323.02 ms | 52.2% bf16 MFU | 1623728 tok/s
step 10964/19560 | loss 3.483056 (+1.58z)| norm 0.2832 (-0.09z)| lr 2.59e-04 | 323.34 ms | 52.2% bf16 MFU | 1623615 tok/s
step 10965/19560 | loss 3.398981 (-0.06z)| norm 0.2817 (-0.16z)| lr 2.58e-04 | 322.37 ms | 52.4% bf16 MFU | 1623753 tok/s
step 10966/19560 | loss 3.483033 (+1.61z)| norm 0.3137 (+1.04z)| lr 2.58e-04 | 322.98 ms | 52.3% bf16 MFU | 1623729 tok/s
step 10967/19560 | loss 3.333787 (-1.34z)| norm 0.3070 (+0.79z)| lr 2.58e-04 | 323.30 ms | 52.2% bf16 MFU | 1623626 tok/s
step 10968/19560 | loss 3.375840 (-0.50z)| norm 0.3036 (+0.66z)| lr 2.58e-04 | 322.45 ms | 52.3% bf16 MFU | 1623742 tok/s
step 10969/19560 | loss 3.437755 (+0.72z)| norm 0.2905 (+0.17z)| lr 2.58e-04 | 323.16 ms | 52.2% bf16 MFU | 1623674 tok/s
step 10970/19560 | loss 3.451494 (+0.98z)| norm 0.3258 (+1.48z)| lr 2.58e-04 | 323.65 ms | 52.1% bf16 MFU | 1623487 tok/s
step 10971/19560 | loss 3.406440 (+0.09z)| norm 0.2931 (+0.28z)| lr 2.58e-04 | 322.73 ms | 52.3% bf16 MFU | 1623540 tok/s
step 10972/19560 | loss 3.423167 (+0.41z)| norm 0.2865 (+0.03z)| lr 2.58e-04 | 322.88 ms | 52.3% bf16 MFU | 1623553 tok/s
step 10973/19560 | loss 3.391899 (-0.21z)| norm 0.2913 (+0.24z)| lr 2.58e-04 | 322.92 ms | 52.3% bf16 MFU | 1623554 tok/s
step 10974/19560 | loss 3.405877 (+0.07z)| norm 0.2854 (+0.00z)| lr 2.58e-04 | 323.41 ms | 52.2% bf16 MFU | 1623433 tok/s
step 10975/19560 | loss 3.406671 (+0.07z)| norm 0.2662 (-0.75z)| lr 2.58e-04 | 322.70 ms | 52.3% bf16 MFU | 1623496 tok/s
step 10976/19560 | loss 3.377272 (-0.53z)| norm 0.2823 (-0.11z)| lr 2.58e-04 | 322.88 ms | 52.3% bf16 MFU | 1623512 tok/s
step 10977/19560 | loss 3.372987 (-0.61z)| norm 0.2699 (-0.60z)| lr 2.58e-04 | 323.02 ms | 52.2% bf16 MFU | 1623489 tok/s
step 10978/19560 | loss 3.394653 (-0.17z)| norm 0.2683 (-0.65z)| lr 2.58e-04 | 322.95 ms | 52.3% bf16 MFU | 1623487 tok/s
step 10979/19560 | loss 3.436208 (+0.66z)| norm 0.2747 (-0.40z)| lr 2.58e-04 | 323.45 ms | 52.2% bf16 MFU | 1623359 tok/s
step 10980/19560 | loss 3.345667 (-1.15z)| norm 0.2952 (+0.42z)| lr 2.58e-04 | 322.82 ms | 52.3% bf16 MFU | 1623395 tok/s
step 10981/19560 | loss 3.362231 (-0.81z)| norm 0.2807 (-0.16z)| lr 2.58e-04 | 322.71 ms | 52.3% bf16 MFU | 1623457 tok/s
step 10982/19560 | loss 3.366450 (-0.73z)| norm 0.2835 (+0.00z)| lr 2.58e-04 | 322.62 ms | 52.3% bf16 MFU | 1623538 tok/s
step 10983/19560 | loss 3.409961 (+0.20z)| norm 0.2762 (-0.35z)| lr 2.58e-04 | 323.00 ms | 52.3% bf16 MFU | 1623521 tok/s
step 10984/19560 | loss 3.373641 (-0.58z)| norm 0.2777 (-0.26z)| lr 2.58e-04 | 323.24 ms | 52.2% bf16 MFU | 1623443 tok/s
step 10985/19560 | loss 3.400505 (+0.01z)| norm 0.2612 (-1.20z)| lr 2.57e-04 | 322.91 ms | 52.3% bf16 MFU | 1623454 tok/s
step 10986/19560 | loss 3.399309 (-0.01z)| norm 0.2747 (-0.40z)| lr 2.57e-04 | 323.17 ms | 52.2% bf16 MFU | 1623398 tok/s
step 10987/19560 | loss 3.394558 (-0.12z)| norm 0.2555 (-1.53z)| lr 2.57e-04 | 323.04 ms | 52.2% bf16 MFU | 1623378 tok/s
step 10988/19560 | loss 3.320066 (-1.71z)| norm 0.3726 (+4.93z)| lr 2.57e-04 | 322.88 ms | 52.3% bf16 MFU | 1623397 tok/s
step 10989/19560 | loss 3.333389 (-1.41z)| norm 0.2922 (+0.58z)| lr 2.57e-04 | 322.80 ms | 52.3% bf16 MFU | 1623436 tok/s
step 10990/19560 | loss 3.371871 (-0.60z)| norm 0.2997 (+0.99z)| lr 2.57e-04 | 322.89 ms | 52.3% bf16 MFU | 1623451 tok/s
step 10991/19560 | loss 3.368840 (-0.66z)| norm 0.3094 (+1.49z)| lr 2.57e-04 | 323.25 ms | 52.2% bf16 MFU | 1623374 tok/s
step 10992/19560 | loss 3.395851 (-0.07z)| norm 0.2951 (+0.72z)| lr 2.57e-04 | 322.99 ms | 52.3% bf16 MFU | 1623367 tok/s
step 10993/19560 | loss 3.413866 (+0.31z)| norm 0.2829 (+0.07z)| lr 2.57e-04 | 322.62 ms | 52.3% bf16 MFU | 1623453 tok/s
step 10994/19560 | loss 3.298395 (-2.18z)| norm 0.2698 (-0.63z)| lr 2.57e-04 | 323.31 ms | 52.2% bf16 MFU | 1623362 tok/s
step 10995/19560 | loss 3.370652 (-0.60z)| norm 0.3012 (+1.05z)| lr 2.57e-04 | 323.07 ms | 52.2% bf16 MFU | 1623336 tok/s
step 10996/19560 | loss 3.427664 (+0.62z)| norm 0.2603 (-1.15z)| lr 2.57e-04 | 324.02 ms | 52.1% bf16 MFU | 1623074 tok/s
step 10997/19560 | loss 3.378148 (-0.46z)| norm 0.2710 (-0.57z)| lr 2.57e-04 | 323.45 ms | 52.2% bf16 MFU | 1622966 tok/s
step 10998/19560 | loss 3.423182 (+0.54z)| norm 0.2748 (-0.37z)| lr 2.57e-04 | 323.26 ms | 52.2% bf16 MFU | 1622911 tok/s
step 10999/19560 | loss 3.396835 (-0.04z)| norm 0.2591 (-1.22z)| lr 2.57e-04 | 323.29 ms | 52.2% bf16 MFU | 1622852 tok/s
step 11000/19560 | loss 3.354498 (-0.98z)| norm 0.2777 (-0.22z)| lr 2.57e-04 | 323.06 ms | 52.2% bf16 MFU | 1622852 tok/s
val loss 3.374903
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2939/10042 = 0.292671
step 11001/19560 | loss 3.407538 (+0.19z)| norm 0.2926 (+0.58z)| lr 2.57e-04 | 322.12 ms | 52.4% bf16 MFU | 1623090 tok/s
step 11002/19560 | loss 3.368099 (-0.68z)| norm 0.2902 (+0.44z)| lr 2.57e-04 | 323.44 ms | 52.2% bf16 MFU | 1622985 tok/s
step 11003/19560 | loss 3.314169 (-1.85z)| norm 0.3070 (+1.33z)| lr 2.57e-04 | 322.99 ms | 52.3% bf16 MFU | 1622997 tok/s
step 11004/19560 | loss 3.369700 (-0.62z)| norm 0.2693 (-0.73z)| lr 2.57e-04 | 323.10 ms | 52.2% bf16 MFU | 1622982 tok/s
step 11005/19560 | loss 3.359953 (-0.83z)| norm 0.2876 (+0.26z)| lr 2.56e-04 | 323.74 ms | 52.1% bf16 MFU | 1622806 tok/s
step 11006/19560 | loss 3.394910 (-0.03z)| norm 0.2651 (-0.96z)| lr 2.56e-04 | 322.67 ms | 52.3% bf16 MFU | 1622908 tok/s
step 11007/19560 | loss 3.366891 (-0.67z)| norm 0.2943 (+0.62z)| lr 2.56e-04 | 323.30 ms | 52.2% bf16 MFU | 1622848 tok/s
step 11008/19560 | loss 3.355477 (-0.92z)| norm 0.2773 (-0.30z)| lr 2.56e-04 | 323.30 ms | 52.2% bf16 MFU | 1622790 tok/s
step 11009/19560 | loss 3.414553 (+0.44z)| norm 0.2767 (-0.33z)| lr 2.56e-04 | 322.81 ms | 52.3% bf16 MFU | 1622858 tok/s
step 11010/19560 | loss 3.346579 (-1.11z)| norm 0.2814 (-0.08z)| lr 2.56e-04 | 322.99 ms | 52.3% bf16 MFU | 1622877 tok/s
step 11011/19560 | loss 3.372274 (-0.53z)| norm 0.2839 (+0.06z)| lr 2.56e-04 | 322.70 ms | 52.3% bf16 MFU | 1622968 tok/s
step 11012/19560 | loss 3.412208 (+0.38z)| norm 0.2671 (-0.85z)| lr 2.56e-04 | 322.97 ms | 52.3% bf16 MFU | 1622985 tok/s
step 11013/19560 | loss 3.319765 (-1.71z)| norm 0.2691 (-0.74z)| lr 2.56e-04 | 322.86 ms | 52.3% bf16 MFU | 1623031 tok/s
step 11014/19560 | loss 3.307441 (-2.03z)| norm 0.2754 (-0.38z)| lr 2.56e-04 | 322.88 ms | 52.3% bf16 MFU | 1623068 tok/s
step 11015/19560 | loss 3.401705 (+0.21z)| norm 0.2712 (-0.61z)| lr 2.56e-04 | 322.59 ms | 52.3% bf16 MFU | 1623176 tok/s
step 11016/19560 | loss 3.365945 (-0.64z)| norm 0.2704 (-0.65z)| lr 2.56e-04 | 323.11 ms | 52.2% bf16 MFU | 1623148 tok/s
step 11017/19560 | loss 3.432911 (+0.93z)| norm 0.3232 (+2.37z)| lr 2.56e-04 | 323.23 ms | 52.2% bf16 MFU | 1623092 tok/s
step 11018/19560 | loss 3.352254 (-0.97z)| norm 0.3093 (+1.55z)| lr 2.56e-04 | 322.78 ms | 52.3% bf16 MFU | 1623151 tok/s
step 11019/19560 | loss 3.401351 (+0.18z)| norm 0.3146 (+1.81z)| lr 2.56e-04 | 322.16 ms | 52.4% bf16 MFU | 1623363 tok/s
step 11020/19560 | loss 3.394053 (+0.01z)| norm 0.2652 (-0.95z)| lr 2.56e-04 | 323.07 ms | 52.2% bf16 MFU | 1623336 tok/s
step 11021/19560 | loss 3.344449 (-1.15z)| norm 0.2941 (+0.65z)| lr 2.56e-04 | 322.27 ms | 52.4% bf16 MFU | 1623512 tok/s
step 11022/19560 | loss 3.363213 (-0.70z)| norm 0.2575 (-1.37z)| lr 2.56e-04 | 322.62 ms | 52.3% bf16 MFU | 1623591 tok/s
step 11023/19560 | loss 3.312337 (-1.86z)| norm 0.3169 (+1.89z)| lr 2.56e-04 | 323.31 ms | 52.2% bf16 MFU | 1623493 tok/s
step 11024/19560 | loss 3.413115 (+0.49z)| norm 0.3202 (+2.02z)| lr 2.56e-04 | 322.48 ms | 52.3% bf16 MFU | 1623609 tok/s
step 11025/19560 | loss 3.365387 (-0.62z)| norm 0.2826 (+0.00z)| lr 2.55e-04 | 323.16 ms | 52.2% bf16 MFU | 1623546 tok/s
step 11026/19560 | loss 3.396526 (+0.15z)| norm 0.2867 (+0.22z)| lr 2.55e-04 | 322.25 ms | 52.4% bf16 MFU | 1623718 tok/s
step 11027/19560 | loss 3.482574 (+2.20z)| norm 0.2625 (-1.07z)| lr 2.55e-04 | 323.17 ms | 52.2% bf16 MFU | 1623650 tok/s
step 11028/19560 | loss 3.360198 (-0.75z)| norm 0.2946 (+0.65z)| lr 2.55e-04 | 322.88 ms | 52.3% bf16 MFU | 1623657 tok/s
step 11029/19560 | loss 3.371231 (-0.48z)| norm 0.2541 (-1.52z)| lr 2.55e-04 | 322.55 ms | 52.3% bf16 MFU | 1623746 tok/s
step 11030/19560 | loss 3.369794 (-0.50z)| norm 0.2945 (+0.63z)| lr 2.55e-04 | 322.82 ms | 52.3% bf16 MFU | 1623763 tok/s
step 11031/19560 | loss 3.381154 (-0.24z)| norm 0.2706 (-0.64z)| lr 2.55e-04 | 322.55 ms | 52.3% bf16 MFU | 1623847 tok/s
step 11032/19560 | loss 3.437511 (+1.15z)| norm 0.2905 (+0.42z)| lr 2.55e-04 | 322.70 ms | 52.3% bf16 MFU | 1623889 tok/s
step 11033/19560 | loss 3.365253 (-0.62z)| norm 0.2723 (-0.56z)| lr 2.55e-04 | 322.71 ms | 52.3% bf16 MFU | 1623926 tok/s
step 11034/19560 | loss 3.295172 (-2.28z)| norm 0.2765 (-0.34z)| lr 2.55e-04 | 323.12 ms | 52.2% bf16 MFU | 1623860 tok/s
step 11035/19560 | loss 3.377822 (-0.29z)| norm 0.2892 (+0.39z)| lr 2.55e-04 | 322.62 ms | 52.3% bf16 MFU | 1623921 tok/s
step 11036/19560 | loss 3.429087 (+0.93z)| norm 0.2706 (-0.65z)| lr 2.55e-04 | 322.91 ms | 52.3% bf16 MFU | 1623906 tok/s
step 11037/19560 | loss 3.333941 (-1.34z)| norm 0.2788 (-0.20z)| lr 2.55e-04 | 322.72 ms | 52.3% bf16 MFU | 1623940 tok/s
step 11038/19560 | loss 3.348154 (-0.99z)| norm 0.2949 (+0.71z)| lr 2.55e-04 | 322.75 ms | 52.3% bf16 MFU | 1623964 tok/s
step 11039/19560 | loss 3.389822 (-0.01z)| norm 0.2618 (-1.15z)| lr 2.55e-04 | 322.85 ms | 52.3% bf16 MFU | 1623962 tok/s
step 11040/19560 | loss 3.399871 (+0.24z)| norm 0.2643 (-1.00z)| lr 2.55e-04 | 322.65 ms | 52.3% bf16 MFU | 1624011 tok/s
step 11041/19560 | loss 3.372859 (-0.40z)| norm 0.2548 (-1.52z)| lr 2.55e-04 | 323.12 ms | 52.2% bf16 MFU | 1623939 tok/s
step 11042/19560 | loss 3.379656 (-0.25z)| norm 0.2716 (-0.58z)| lr 2.55e-04 | 323.04 ms | 52.2% bf16 MFU | 1623892 tok/s
step 11043/19560 | loss 3.399301 (+0.24z)| norm 0.2530 (-1.60z)| lr 2.55e-04 | 322.32 ms | 52.4% bf16 MFU | 1624027 tok/s
step 11044/19560 | loss 3.367078 (-0.55z)| norm 0.2713 (-0.58z)| lr 2.55e-04 | 322.45 ms | 52.3% bf16 MFU | 1624123 tok/s
step 11045/19560 | loss 3.413878 (+0.61z)| norm 0.2514 (-1.66z)| lr 2.55e-04 | 322.96 ms | 52.3% bf16 MFU | 1624086 tok/s
step 11046/19560 | loss 3.350343 (-0.98z)| norm 0.2630 (-1.00z)| lr 2.54e-04 | 322.59 ms | 52.3% bf16 MFU | 1624145 tok/s
step 11047/19560 | loss 3.388526 (-0.03z)| norm 0.2819 (+0.03z)| lr 2.54e-04 | 322.94 ms | 52.3% bf16 MFU | 1624111 tok/s
step 11048/19560 | loss 3.346682 (-1.06z)| norm 0.2761 (-0.29z)| lr 2.54e-04 | 322.48 ms | 52.3% bf16 MFU | 1624196 tok/s
step 11049/19560 | loss 3.408198 (+0.47z)| norm 0.2426 (-2.07z)| lr 2.54e-04 | 323.35 ms | 52.2% bf16 MFU | 1624058 tok/s
step 11050/19560 | loss 3.396147 (+0.16z)| norm 0.2572 (-1.29z)| lr 2.54e-04 | 322.51 ms | 52.3% bf16 MFU | 1624139 tok/s
step 11051/19560 | loss 3.381431 (-0.20z)| norm 0.2763 (-0.25z)| lr 2.54e-04 | 323.22 ms | 52.2% bf16 MFU | 1624037 tok/s
step 11052/19560 | loss 3.361299 (-0.70z)| norm 0.2493 (-1.68z)| lr 2.54e-04 | 322.74 ms | 52.3% bf16 MFU | 1624060 tok/s
step 11053/19560 | loss 3.387221 (-0.03z)| norm 0.2712 (-0.52z)| lr 2.54e-04 | 323.45 ms | 52.2% bf16 MFU | 1623904 tok/s
step 11054/19560 | loss 3.436425 (+1.22z)| norm 0.2834 (+0.13z)| lr 2.54e-04 | 322.88 ms | 52.3% bf16 MFU | 1623899 tok/s
step 11055/19560 | loss 3.335997 (-1.33z)| norm 0.2783 (-0.15z)| lr 2.54e-04 | 322.57 ms | 52.3% bf16 MFU | 1623973 tok/s
step 11056/19560 | loss 3.353018 (-0.89z)| norm 0.2852 (+0.22z)| lr 2.54e-04 | 322.67 ms | 52.3% bf16 MFU | 1624015 tok/s
step 11057/19560 | loss 3.385742 (-0.06z)| norm 0.2796 (-0.08z)| lr 2.54e-04 | 323.34 ms | 52.2% bf16 MFU | 1623889 tok/s
step 11058/19560 | loss 3.535572 (+3.53z)| norm 0.3307 (+2.60z)| lr 2.54e-04 | 323.04 ms | 52.2% bf16 MFU | 1623844 tok/s
step 11059/19560 | loss 3.336288 (-1.25z)| norm 0.3092 (+1.44z)| lr 2.54e-04 | 322.53 ms | 52.3% bf16 MFU | 1623929 tok/s
step 11060/19560 | loss 3.323988 (-1.52z)| norm 0.2971 (+0.79z)| lr 2.54e-04 | 322.88 ms | 52.3% bf16 MFU | 1623922 tok/s
step 11061/19560 | loss 3.395892 (+0.19z)| norm 0.2897 (+0.39z)| lr 2.54e-04 | 322.37 ms | 52.4% bf16 MFU | 1624044 tok/s
step 11062/19560 | loss 3.384055 (-0.08z)| norm 0.2774 (-0.26z)| lr 2.54e-04 | 323.00 ms | 52.3% bf16 MFU | 1624000 tok/s
step 11063/19560 | loss 3.332805 (-1.29z)| norm 0.2996 (+0.90z)| lr 2.54e-04 | 322.78 ms | 52.3% bf16 MFU | 1624014 tok/s
step 11064/19560 | loss 3.344814 (-0.99z)| norm 0.2837 (+0.05z)| lr 2.54e-04 | 323.05 ms | 52.2% bf16 MFU | 1623961 tok/s
step 11065/19560 | loss 3.394881 (+0.20z)| norm 0.2904 (+0.40z)| lr 2.54e-04 | 322.72 ms | 52.3% bf16 MFU | 1623993 tok/s
step 11066/19560 | loss 3.360167 (-0.62z)| norm 0.2766 (-0.34z)| lr 2.53e-04 | 323.15 ms | 52.2% bf16 MFU | 1623916 tok/s
step 11067/19560 | loss 3.378589 (-0.19z)| norm 0.2694 (-0.73z)| lr 2.53e-04 | 322.48 ms | 52.3% bf16 MFU | 1624010 tok/s
step 11068/19560 | loss 3.388614 (+0.05z)| norm 0.2903 (+0.39z)| lr 2.53e-04 | 322.51 ms | 52.3% bf16 MFU | 1624092 tok/s
step 11069/19560 | loss 3.320098 (-1.56z)| norm 0.2547 (-1.50z)| lr 2.53e-04 | 323.02 ms | 52.2% bf16 MFU | 1624043 tok/s
step 11070/19560 | loss 3.335566 (-1.18z)| norm 0.2825 (-0.03z)| lr 2.53e-04 | 322.58 ms | 52.3% bf16 MFU | 1624105 tok/s
step 11071/19560 | loss 3.395051 (+0.21z)| norm 0.2731 (-0.52z)| lr 2.53e-04 | 322.43 ms | 52.3% bf16 MFU | 1624201 tok/s
step 11072/19560 | loss 3.379403 (-0.16z)| norm 0.2842 (+0.07z)| lr 2.53e-04 | 322.85 ms | 52.3% bf16 MFU | 1624187 tok/s
step 11073/19560 | loss 3.377643 (-0.21z)| norm 0.2637 (-1.03z)| lr 2.53e-04 | 323.29 ms | 52.2% bf16 MFU | 1624064 tok/s
step 11074/19560 | loss 3.346910 (-0.96z)| norm 0.2833 (+0.02z)| lr 2.53e-04 | 323.08 ms | 52.2% bf16 MFU | 1624001 tok/s
step 11075/19560 | loss 3.447192 (+1.57z)| norm 0.2655 (-0.91z)| lr 2.53e-04 | 322.26 ms | 52.4% bf16 MFU | 1624146 tok/s
step 11076/19560 | loss 3.420214 (+0.88z)| norm 0.2909 (+0.43z)| lr 2.53e-04 | 323.47 ms | 52.2% bf16 MFU | 1623979 tok/s
step 11077/19560 | loss 3.310877 (-1.83z)| norm 0.2524 (-1.59z)| lr 2.53e-04 | 322.75 ms | 52.3% bf16 MFU | 1624002 tok/s
step 11078/19560 | loss 3.377047 (-0.18z)| norm 0.2640 (-0.97z)| lr 2.53e-04 | 323.36 ms | 52.2% bf16 MFU | 1623871 tok/s
step 11079/19560 | loss 3.447648 (+1.56z)| norm 0.2893 (+0.36z)| lr 2.53e-04 | 322.70 ms | 52.3% bf16 MFU | 1623912 tok/s
step 11080/19560 | loss 3.409815 (+0.62z)| norm 0.2626 (-1.03z)| lr 2.53e-04 | 323.22 ms | 52.2% bf16 MFU | 1623820 tok/s
step 11081/19560 | loss 3.335525 (-1.21z)| norm 0.2669 (-0.79z)| lr 2.53e-04 | 322.55 ms | 52.3% bf16 MFU | 1623902 tok/s
step 11082/19560 | loss 3.324372 (-1.46z)| norm 0.2674 (-0.75z)| lr 2.53e-04 | 322.65 ms | 52.3% bf16 MFU | 1623954 tok/s
step 11083/19560 | loss 3.427368 (+1.06z)| norm 0.2743 (-0.39z)| lr 2.53e-04 | 322.43 ms | 52.3% bf16 MFU | 1624059 tok/s
step 11084/19560 | loss 3.359340 (-0.59z)| norm 0.2671 (-0.77z)| lr 2.53e-04 | 323.00 ms | 52.3% bf16 MFU | 1624014 tok/s
step 11085/19560 | loss 3.441623 (+1.42z)| norm 0.2944 (+0.66z)| lr 2.53e-04 | 322.81 ms | 52.3% bf16 MFU | 1624021 tok/s
step 11086/19560 | loss 3.441297 (+1.39z)| norm 0.2801 (-0.09z)| lr 2.52e-04 | 323.31 ms | 52.2% bf16 MFU | 1623901 tok/s
step 11087/19560 | loss 3.389755 (+0.15z)| norm 0.2766 (-0.27z)| lr 2.52e-04 | 322.71 ms | 52.3% bf16 MFU | 1623938 tok/s
step 11088/19560 | loss 3.399422 (+0.39z)| norm 0.2610 (-1.08z)| lr 2.52e-04 | 322.84 ms | 52.3% bf16 MFU | 1623940 tok/s
step 11089/19560 | loss 3.344364 (-0.95z)| norm 0.2782 (-0.16z)| lr 2.52e-04 | 322.57 ms | 52.3% bf16 MFU | 1624010 tok/s
step 11090/19560 | loss 3.343909 (-0.94z)| norm 0.2678 (-0.70z)| lr 2.52e-04 | 323.13 ms | 52.2% bf16 MFU | 1623936 tok/s
step 11091/19560 | loss 3.319340 (-1.52z)| norm 0.3024 (+1.13z)| lr 2.52e-04 | 323.48 ms | 52.2% bf16 MFU | 1623779 tok/s
step 11092/19560 | loss 3.437980 (+1.42z)| norm 0.2769 (-0.22z)| lr 2.52e-04 | 322.90 ms | 52.3% bf16 MFU | 1623774 tok/s
step 11093/19560 | loss 3.431418 (+1.25z)| norm 0.2884 (+0.39z)| lr 2.52e-04 | 322.40 ms | 52.3% bf16 MFU | 1623896 tok/s
step 11094/19560 | loss 3.376319 (-0.10z)| norm 0.2747 (-0.33z)| lr 2.52e-04 | 323.02 ms | 52.2% bf16 MFU | 1623855 tok/s
step 11095/19560 | loss 3.324702 (-1.42z)| norm 0.2536 (-1.43z)| lr 2.52e-04 | 322.58 ms | 52.3% bf16 MFU | 1623926 tok/s
step 11096/19560 | loss 3.424867 (+1.12z)| norm 0.2915 (+0.60z)| lr 2.52e-04 | 323.49 ms | 52.2% bf16 MFU | 1623765 tok/s
step 11097/19560 | loss 3.344105 (-0.91z)| norm 0.2955 (+0.82z)| lr 2.52e-04 | 322.68 ms | 52.3% bf16 MFU | 1623815 tok/s
step 11098/19560 | loss 3.325717 (-1.37z)| norm 0.2671 (-0.70z)| lr 2.52e-04 | 323.04 ms | 52.2% bf16 MFU | 1623773 tok/s
step 11099/19560 | loss 3.378455 (-0.01z)| norm 0.2644 (-0.84z)| lr 2.52e-04 | 322.42 ms | 52.3% bf16 MFU | 1623890 tok/s
step 11100/19560 | loss 3.410278 (+0.81z)| norm 0.2636 (-0.87z)| lr 2.52e-04 | 323.52 ms | 52.2% bf16 MFU | 1623724 tok/s
step 11101/19560 | loss 3.344845 (-0.86z)| norm 0.2588 (-1.11z)| lr 2.52e-04 | 322.59 ms | 52.3% bf16 MFU | 1623800 tok/s
step 11102/19560 | loss 3.404319 (+0.67z)| norm 0.2632 (-0.86z)| lr 2.52e-04 | 322.26 ms | 52.4% bf16 MFU | 1623956 tok/s
step 11103/19560 | loss 3.409029 (+0.79z)| norm 0.2742 (-0.27z)| lr 2.52e-04 | 323.21 ms | 52.2% bf16 MFU | 1623864 tok/s
step 11104/19560 | loss 3.463022 (+2.12z)| norm 0.2916 (+0.68z)| lr 2.52e-04 | 323.06 ms | 52.2% bf16 MFU | 1623816 tok/s
step 11105/19560 | loss 3.384086 (+0.13z)| norm 0.3049 (+1.38z)| lr 2.52e-04 | 323.09 ms | 52.2% bf16 MFU | 1623762 tok/s
step 11106/19560 | loss 3.381534 (+0.06z)| norm 0.2804 (+0.05z)| lr 2.51e-04 | 322.40 ms | 52.3% bf16 MFU | 1623884 tok/s
step 11107/19560 | loss 3.430892 (+1.31z)| norm 0.3178 (+2.02z)| lr 2.51e-04 | 322.75 ms | 52.3% bf16 MFU | 1623912 tok/s
step 11108/19560 | loss 3.454001 (+1.85z)| norm 0.2885 (+0.47z)| lr 2.51e-04 | 322.70 ms | 52.3% bf16 MFU | 1623950 tok/s
step 11109/19560 | loss 3.405077 (+0.62z)| norm 0.2863 (+0.34z)| lr 2.51e-04 | 322.78 ms | 52.3% bf16 MFU | 1623967 tok/s
step 11110/19560 | loss 3.383351 (+0.08z)| norm 0.2774 (-0.13z)| lr 2.51e-04 | 322.72 ms | 52.3% bf16 MFU | 1623999 tok/s
step 11111/19560 | loss 3.392233 (+0.30z)| norm 0.2740 (-0.31z)| lr 2.51e-04 | 322.33 ms | 52.4% bf16 MFU | 1624127 tok/s
step 11112/19560 | loss 3.397425 (+0.43z)| norm 0.2896 (+0.52z)| lr 2.51e-04 | 322.64 ms | 52.3% bf16 MFU | 1624170 tok/s
step 11113/19560 | loss 3.432684 (+1.30z)| norm 0.2845 (+0.24z)| lr 2.51e-04 | 323.18 ms | 52.2% bf16 MFU | 1624077 tok/s
step 11114/19560 | loss 3.375187 (-0.13z)| norm 0.2749 (-0.28z)| lr 2.51e-04 | 323.11 ms | 52.2% bf16 MFU | 1624005 tok/s
step 11115/19560 | loss 3.403397 (+0.57z)| norm 0.2800 (-0.02z)| lr 2.51e-04 | 322.37 ms | 52.4% bf16 MFU | 1624124 tok/s
step 11116/19560 | loss 3.342804 (-0.95z)| norm 0.2626 (-1.00z)| lr 2.51e-04 | 322.31 ms | 52.4% bf16 MFU | 1624250 tok/s
step 11117/19560 | loss 3.402813 (+0.54z)| norm 0.2786 (-0.04z)| lr 2.51e-04 | 323.44 ms | 52.2% bf16 MFU | 1624086 tok/s
step 11118/19560 | loss 3.382585 (+0.03z)| norm 0.2640 (-0.91z)| lr 2.51e-04 | 322.86 ms | 52.3% bf16 MFU | 1624076 tok/s
step 11119/19560 | loss 3.418824 (+0.93z)| norm 0.2771 (-0.10z)| lr 2.51e-04 | 322.56 ms | 52.3% bf16 MFU | 1624143 tok/s
step 11120/19560 | loss 3.358804 (-0.56z)| norm 0.2794 (+0.05z)| lr 2.51e-04 | 322.67 ms | 52.3% bf16 MFU | 1624178 tok/s
step 11121/19560 | loss 3.400712 (+0.49z)| norm 0.2698 (-0.53z)| lr 2.51e-04 | 322.99 ms | 52.3% bf16 MFU | 1624131 tok/s
step 11122/19560 | loss 3.321480 (-1.51z)| norm 0.2800 (+0.08z)| lr 2.51e-04 | 322.87 ms | 52.3% bf16 MFU | 1624117 tok/s
step 11123/19560 | loss 3.377382 (-0.10z)| norm 0.2714 (-0.43z)| lr 2.51e-04 | 322.53 ms | 52.3% bf16 MFU | 1624189 tok/s
step 11124/19560 | loss 3.385196 (+0.10z)| norm 0.2949 (+1.00z)| lr 2.51e-04 | 322.79 ms | 52.3% bf16 MFU | 1624192 tok/s
step 11125/19560 | loss 3.293700 (-2.16z)| norm 0.2811 (+0.14z)| lr 2.51e-04 | 323.31 ms | 52.2% bf16 MFU | 1624063 tok/s
step 11126/19560 | loss 3.378428 (-0.04z)| norm 0.2863 (+0.46z)| lr 2.51e-04 | 323.26 ms | 52.2% bf16 MFU | 1623952 tok/s
step 11127/19560 | loss 3.359345 (-0.51z)| norm 0.2854 (+0.39z)| lr 2.50e-04 | 322.57 ms | 52.3% bf16 MFU | 1624023 tok/s
step 11128/19560 | loss 3.398314 (+0.45z)| norm 0.2830 (+0.25z)| lr 2.50e-04 | 323.04 ms | 52.2% bf16 MFU | 1623971 tok/s
step 11129/19560 | loss 3.395881 (+0.40z)| norm 0.2947 (+0.97z)| lr 2.50e-04 | 322.43 ms | 52.3% bf16 MFU | 1624074 tok/s
step 11130/19560 | loss 3.367864 (-0.31z)| norm 0.2678 (-0.69z)| lr 2.50e-04 | 323.27 ms | 52.2% bf16 MFU | 1623963 tok/s
step 11131/19560 | loss 3.392962 (+0.31z)| norm 0.2955 (+1.04z)| lr 2.50e-04 | 323.21 ms | 52.2% bf16 MFU | 1623872 tok/s
step 11132/19560 | loss 3.375574 (-0.13z)| norm 0.2673 (-0.72z)| lr 2.50e-04 | 322.76 ms | 52.3% bf16 MFU | 1623898 tok/s
step 11133/19560 | loss 3.375100 (-0.15z)| norm 0.2651 (-0.85z)| lr 2.50e-04 | 323.07 ms | 52.2% bf16 MFU | 1623844 tok/s
step 11134/19560 | loss 3.336477 (-1.11z)| norm 0.2638 (-0.93z)| lr 2.50e-04 | 323.04 ms | 52.2% bf16 MFU | 1623802 tok/s
step 11135/19560 | loss 3.368875 (-0.29z)| norm 0.2534 (-1.54z)| lr 2.50e-04 | 323.30 ms | 52.2% bf16 MFU | 1623697 tok/s
step 11136/19560 | loss 3.355823 (-0.62z)| norm 0.2593 (-1.16z)| lr 2.50e-04 | 323.09 ms | 52.2% bf16 MFU | 1623649 tok/s
step 11137/19560 | loss 3.352977 (-0.68z)| norm 0.2810 (+0.17z)| lr 2.50e-04 | 323.00 ms | 52.3% bf16 MFU | 1623626 tok/s
step 11138/19560 | loss 3.384356 (+0.10z)| norm 0.2635 (-0.89z)| lr 2.50e-04 | 322.66 ms | 52.3% bf16 MFU | 1623690 tok/s
step 11139/19560 | loss 3.435555 (+1.38z)| norm 0.2623 (-0.96z)| lr 2.50e-04 | 323.56 ms | 52.2% bf16 MFU | 1623523 tok/s
step 11140/19560 | loss 3.357614 (-0.57z)| norm 0.2544 (-1.42z)| lr 2.50e-04 | 323.37 ms | 52.2% bf16 MFU | 1623414 tok/s
step 11141/19560 | loss 3.331727 (-1.23z)| norm 0.2916 (+0.83z)| lr 2.50e-04 | 323.01 ms | 52.3% bf16 MFU | 1623401 tok/s
step 11142/19560 | loss 3.384552 (+0.09z)| norm 0.2670 (-0.66z)| lr 2.50e-04 | 323.11 ms | 52.2% bf16 MFU | 1623362 tok/s
step 11143/19560 | loss 3.375335 (-0.14z)| norm 0.2841 (+0.37z)| lr 2.50e-04 | 322.95 ms | 52.3% bf16 MFU | 1623365 tok/s
step 11144/19560 | loss 3.399145 (+0.46z)| norm 0.2811 (+0.18z)| lr 2.50e-04 | 322.68 ms | 52.3% bf16 MFU | 1623436 tok/s
step 11145/19560 | loss 3.361019 (-0.50z)| norm 0.2887 (+0.68z)| lr 2.50e-04 | 323.29 ms | 52.2% bf16 MFU | 1623349 tok/s
step 11146/19560 | loss 3.361370 (-0.50z)| norm 0.2633 (-0.89z)| lr 2.50e-04 | 322.70 ms | 52.3% bf16 MFU | 1623417 tok/s
step 11147/19560 | loss 3.329726 (-1.29z)| norm 0.2758 (-0.09z)| lr 2.49e-04 | 323.00 ms | 52.3% bf16 MFU | 1623405 tok/s
step 11148/19560 | loss 3.395396 (+0.40z)| norm 0.2535 (-1.51z)| lr 2.49e-04 | 323.23 ms | 52.2% bf16 MFU | 1623335 tok/s
step 11149/19560 | loss 3.360506 (-0.51z)| norm 0.2598 (-1.09z)| lr 2.49e-04 | 323.36 ms | 52.2% bf16 MFU | 1623236 tok/s
step 11150/19560 | loss 3.390432 (+0.26z)| norm 0.2666 (-0.66z)| lr 2.49e-04 | 323.13 ms | 52.2% bf16 MFU | 1623200 tok/s
step 11151/19560 | loss 3.338159 (-1.10z)| norm 0.2496 (-1.75z)| lr 2.49e-04 | 323.12 ms | 52.2% bf16 MFU | 1623170 tok/s
step 11152/19560 | loss 3.390570 (+0.27z)| norm 0.2458 (-2.00z)| lr 2.49e-04 | 322.85 ms | 52.3% bf16 MFU | 1623208 tok/s
step 11153/19560 | loss 3.359953 (-0.53z)| norm 0.2806 (+0.32z)| lr 2.49e-04 | 322.57 ms | 52.3% bf16 MFU | 1623315 tok/s
step 11154/19560 | loss 3.384495 (+0.11z)| norm 0.2492 (-1.73z)| lr 2.49e-04 | 322.85 ms | 52.3% bf16 MFU | 1623347 tok/s
step 11155/19560 | loss 3.355989 (-0.62z)| norm 0.2682 (-0.49z)| lr 2.49e-04 | 323.29 ms | 52.2% bf16 MFU | 1623266 tok/s
step 11156/19560 | loss 3.382443 (+0.08z)| norm 0.2753 (-0.00z)| lr 2.49e-04 | 322.70 ms | 52.3% bf16 MFU | 1623336 tok/s
step 11157/19560 | loss 3.362922 (-0.44z)| norm 0.2576 (-1.18z)| lr 2.49e-04 | 323.23 ms | 52.2% bf16 MFU | 1623269 tok/s
step 11158/19560 | loss 3.409071 (+0.79z)| norm 0.2598 (-1.02z)| lr 2.49e-04 | 323.28 ms | 52.2% bf16 MFU | 1623194 tok/s
step 11159/19560 | loss 3.370152 (-0.25z)| norm 0.2701 (-0.33z)| lr 2.49e-04 | 322.85 ms | 52.3% bf16 MFU | 1623231 tok/s
step 11160/19560 | loss 3.395981 (+0.45z)| norm 0.2670 (-0.53z)| lr 2.49e-04 | 322.65 ms | 52.3% bf16 MFU | 1623317 tok/s
step 11161/19560 | loss 3.349141 (-0.81z)| norm 0.2569 (-1.19z)| lr 2.49e-04 | 323.06 ms | 52.2% bf16 MFU | 1623295 tok/s
step 11162/19560 | loss 3.322082 (-1.56z)| norm 0.2644 (-0.69z)| lr 2.49e-04 | 323.15 ms | 52.2% bf16 MFU | 1623251 tok/s
step 11163/19560 | loss 3.385277 (+0.16z)| norm 0.2716 (-0.20z)| lr 2.49e-04 | 322.97 ms | 52.3% bf16 MFU | 1623255 tok/s
step 11164/19560 | loss 3.382079 (+0.08z)| norm 0.2709 (-0.25z)| lr 2.49e-04 | 322.76 ms | 52.3% bf16 MFU | 1623312 tok/s
step 11165/19560 | loss 3.398408 (+0.52z)| norm 0.3077 (+2.15z)| lr 2.49e-04 | 323.23 ms | 52.2% bf16 MFU | 1623248 tok/s
step 11166/19560 | loss 3.359005 (-0.57z)| norm 0.2823 (+0.50z)| lr 2.49e-04 | 322.81 ms | 52.3% bf16 MFU | 1623292 tok/s
step 11167/19560 | loss 3.301183 (-2.11z)| norm 0.2669 (-0.52z)| lr 2.48e-04 | 322.81 ms | 52.3% bf16 MFU | 1623334 tok/s
step 11168/19560 | loss 3.351744 (-0.73z)| norm 0.2682 (-0.44z)| lr 2.48e-04 | 323.47 ms | 52.2% bf16 MFU | 1623208 tok/s
step 11169/19560 | loss 3.508693 (+3.34z)| norm 0.2869 (+0.79z)| lr 2.48e-04 | 322.58 ms | 52.3% bf16 MFU | 1623312 tok/s
step 11170/19560 | loss 3.330128 (-1.26z)| norm 0.3372 (+3.85z)| lr 2.48e-04 | 322.88 ms | 52.3% bf16 MFU | 1623335 tok/s
step 11171/19560 | loss 3.341911 (-0.94z)| norm 0.2910 (+0.95z)| lr 2.48e-04 | 323.35 ms | 52.2% bf16 MFU | 1623241 tok/s
step 11172/19560 | loss 3.396732 (+0.45z)| norm 0.2910 (+0.94z)| lr 2.48e-04 | 322.62 ms | 52.3% bf16 MFU | 1623334 tok/s
step 11173/19560 | loss 3.414036 (+0.90z)| norm 0.3091 (+2.03z)| lr 2.48e-04 | 323.39 ms | 52.2% bf16 MFU | 1623229 tok/s
step 11174/19560 | loss 3.371596 (-0.19z)| norm 0.2866 (+0.62z)| lr 2.48e-04 | 323.16 ms | 52.2% bf16 MFU | 1623186 tok/s
step 11175/19560 | loss 3.385973 (+0.17z)| norm 0.3051 (+1.74z)| lr 2.48e-04 | 323.33 ms | 52.2% bf16 MFU | 1623102 tok/s
step 11176/19560 | loss 3.364126 (-0.39z)| norm 0.3068 (+1.80z)| lr 2.48e-04 | 323.30 ms | 52.2% bf16 MFU | 1623030 tok/s
step 11177/19560 | loss 3.362599 (-0.42z)| norm 0.2862 (+0.54z)| lr 2.48e-04 | 323.49 ms | 52.2% bf16 MFU | 1622915 tok/s
step 11178/19560 | loss 3.429489 (+1.29z)| norm 0.3155 (+2.28z)| lr 2.48e-04 | 323.16 ms | 52.2% bf16 MFU | 1622887 tok/s
step 11179/19560 | loss 3.354352 (-0.63z)| norm 0.2857 (+0.47z)| lr 2.48e-04 | 323.27 ms | 52.2% bf16 MFU | 1622834 tok/s
step 11180/19560 | loss 3.438532 (+1.49z)| norm 0.2790 (+0.05z)| lr 2.48e-04 | 322.38 ms | 52.4% bf16 MFU | 1623007 tok/s
step 11181/19560 | loss 3.442825 (+1.58z)| norm 0.2835 (+0.32z)| lr 2.48e-04 | 324.19 ms | 52.1% bf16 MFU | 1622718 tok/s
step 11182/19560 | loss 3.383060 (+0.09z)| norm 0.2790 (+0.05z)| lr 2.48e-04 | 322.62 ms | 52.3% bf16 MFU | 1622837 tok/s
step 11183/19560 | loss 3.420355 (+1.02z)| norm 0.2549 (-1.41z)| lr 2.48e-04 | 323.64 ms | 52.1% bf16 MFU | 1622695 tok/s
step 11184/19560 | loss 3.375225 (-0.13z)| norm 0.2686 (-0.57z)| lr 2.48e-04 | 322.75 ms | 52.3% bf16 MFU | 1622784 tok/s
step 11185/19560 | loss 3.386590 (+0.16z)| norm 0.2650 (-0.78z)| lr 2.48e-04 | 322.92 ms | 52.3% bf16 MFU | 1622824 tok/s
step 11186/19560 | loss 3.402696 (+0.63z)| norm 0.2913 (+0.88z)| lr 2.48e-04 | 323.67 ms | 52.1% bf16 MFU | 1622675 tok/s
step 11187/19560 | loss 3.347572 (-0.87z)| norm 0.2887 (+0.73z)| lr 2.48e-04 | 323.63 ms | 52.2% bf16 MFU | 1622544 tok/s
step 11188/19560 | loss 3.412267 (+0.87z)| norm 0.2904 (+0.85z)| lr 2.47e-04 | 323.05 ms | 52.2% bf16 MFU | 1622564 tok/s
step 11189/19560 | loss 3.463483 (+2.21z)| norm 0.2716 (-0.36z)| lr 2.47e-04 | 322.71 ms | 52.3% bf16 MFU | 1622668 tok/s
step 11190/19560 | loss 3.363306 (-0.46z)| norm 0.2845 (+0.47z)| lr 2.47e-04 | 323.17 ms | 52.2% bf16 MFU | 1622652 tok/s
step 11191/19560 | loss 3.324026 (-1.51z)| norm 0.2661 (-0.70z)| lr 2.47e-04 | 323.19 ms | 52.2% bf16 MFU | 1622630 tok/s
step 11192/19560 | loss 3.398089 (+0.46z)| norm 0.3120 (+2.22z)| lr 2.47e-04 | 323.12 ms | 52.2% bf16 MFU | 1622627 tok/s
step 11193/19560 | loss 3.384099 (+0.09z)| norm 0.2776 (+0.04z)| lr 2.47e-04 | 323.69 ms | 52.1% bf16 MFU | 1622482 tok/s
step 11194/19560 | loss 3.371670 (-0.25z)| norm 0.2580 (-1.20z)| lr 2.47e-04 | 322.75 ms | 52.3% bf16 MFU | 1622580 tok/s
step 11195/19560 | loss 3.377854 (-0.08z)| norm 0.2708 (-0.39z)| lr 2.47e-04 | 322.65 ms | 52.3% bf16 MFU | 1622698 tok/s
step 11196/19560 | loss 3.375607 (-0.14z)| norm 0.2656 (-0.71z)| lr 2.47e-04 | 322.98 ms | 52.3% bf16 MFU | 1622727 tok/s
step 11197/19560 | loss 3.388822 (+0.20z)| norm 0.2909 (+0.89z)| lr 2.47e-04 | 323.28 ms | 52.2% bf16 MFU | 1622679 tok/s
step 11198/19560 | loss 3.365231 (-0.44z)| norm 0.2672 (-0.61z)| lr 2.47e-04 | 322.98 ms | 52.3% bf16 MFU | 1622710 tok/s
step 11199/19560 | loss 3.398880 (+0.47z)| norm 0.3086 (+1.97z)| lr 2.47e-04 | 323.03 ms | 52.2% bf16 MFU | 1622727 tok/s
step 11200/19560 | loss 3.407948 (+0.71z)| norm 0.2862 (+0.57z)| lr 2.47e-04 | 322.30 ms | 52.4% bf16 MFU | 1622925 tok/s
step 11201/19560 | loss 3.382478 (+0.02z)| norm 0.2584 (-1.17z)| lr 2.47e-04 | 323.03 ms | 52.2% bf16 MFU | 1622931 tok/s
step 11202/19560 | loss 3.366732 (-0.42z)| norm 0.2829 (+0.36z)| lr 2.47e-04 | 322.92 ms | 52.3% bf16 MFU | 1622964 tok/s
step 11203/19560 | loss 3.334994 (-1.26z)| norm 0.2549 (-1.38z)| lr 2.47e-04 | 323.15 ms | 52.2% bf16 MFU | 1622936 tok/s
step 11204/19560 | loss 3.434685 (+1.46z)| norm 0.2847 (+0.48z)| lr 2.47e-04 | 323.06 ms | 52.2% bf16 MFU | 1622934 tok/s
step 11205/19560 | loss 3.371099 (-0.29z)| norm 0.2723 (-0.30z)| lr 2.47e-04 | 323.38 ms | 52.2% bf16 MFU | 1622853 tok/s
step 11206/19560 | loss 3.360428 (-0.59z)| norm 0.2714 (-0.36z)| lr 2.47e-04 | 322.83 ms | 52.3% bf16 MFU | 1622913 tok/s
step 11207/19560 | loss 3.408439 (+0.76z)| norm 0.2903 (+0.82z)| lr 2.47e-04 | 322.93 ms | 52.3% bf16 MFU | 1622944 tok/s
step 11208/19560 | loss 3.373779 (-0.20z)| norm 0.2671 (-0.64z)| lr 2.46e-04 | 323.50 ms | 52.2% bf16 MFU | 1622830 tok/s
step 11209/19560 | loss 3.496914 (+3.11z)| norm 0.3040 (+1.66z)| lr 2.46e-04 | 323.04 ms | 52.2% bf16 MFU | 1622838 tok/s
step 11210/19560 | loss 3.387493 (+0.13z)| norm 0.2755 (-0.13z)| lr 2.46e-04 | 322.66 ms | 52.3% bf16 MFU | 1622940 tok/s
step 11211/19560 | loss 3.416011 (+0.91z)| norm 0.3085 (+1.89z)| lr 2.46e-04 | 323.58 ms | 52.2% bf16 MFU | 1622807 tok/s
step 11212/19560 | loss 3.386681 (+0.10z)| norm 0.3153 (+2.25z)| lr 2.46e-04 | 323.22 ms | 52.2% bf16 MFU | 1622770 tok/s
step 11213/19560 | loss 3.452996 (+1.92z)| norm 0.2972 (+1.14z)| lr 2.46e-04 | 323.11 ms | 52.2% bf16 MFU | 1622764 tok/s
step 11214/19560 | loss 3.361632 (-0.57z)| norm 0.2917 (+0.81z)| lr 2.46e-04 | 322.67 ms | 52.3% bf16 MFU | 1622867 tok/s
step 11215/19560 | loss 3.395097 (+0.35z)| norm 0.2663 (-0.72z)| lr 2.46e-04 | 323.01 ms | 52.2% bf16 MFU | 1622879 tok/s
step 11216/19560 | loss 3.351115 (-0.85z)| norm 0.2848 (+0.38z)| lr 2.46e-04 | 322.58 ms | 52.3% bf16 MFU | 1623000 tok/s
step 11217/19560 | loss 3.435658 (+1.45z)| norm 0.2889 (+0.62z)| lr 2.46e-04 | 322.62 ms | 52.3% bf16 MFU | 1623105 tok/s
step 11218/19560 | loss 3.367878 (-0.41z)| norm 0.2736 (-0.30z)| lr 2.46e-04 | 323.22 ms | 52.2% bf16 MFU | 1623055 tok/s
step 11219/19560 | loss 3.396858 (+0.37z)| norm 0.2650 (-0.80z)| lr 2.46e-04 | 322.85 ms | 52.3% bf16 MFU | 1623099 tok/s
step 11220/19560 | loss 3.382779 (-0.01z)| norm 0.2780 (-0.02z)| lr 2.46e-04 | 322.93 ms | 52.3% bf16 MFU | 1623122 tok/s
step 11221/19560 | loss 3.429561 (+1.31z)| norm 0.2968 (+1.12z)| lr 2.46e-04 | 322.85 ms | 52.3% bf16 MFU | 1623162 tok/s
step 11222/19560 | loss 3.421540 (+1.07z)| norm 0.2714 (-0.42z)| lr 2.46e-04 | 323.20 ms | 52.2% bf16 MFU | 1623114 tok/s
step 11223/19560 | loss 3.378380 (-0.15z)| norm 0.2678 (-0.65z)| lr 2.46e-04 | 323.13 ms | 52.2% bf16 MFU | 1623084 tok/s
step 11224/19560 | loss 3.502557 (+3.22z)| norm 0.2646 (-0.83z)| lr 2.46e-04 | 322.65 ms | 52.3% bf16 MFU | 1623177 tok/s
step 11225/19560 | loss 3.428027 (+1.17z)| norm 0.2841 (+0.36z)| lr 2.46e-04 | 323.01 ms | 52.2% bf16 MFU | 1623175 tok/s
step 11226/19560 | loss 3.407452 (+0.60z)| norm 0.2815 (+0.20z)| lr 2.46e-04 | 322.61 ms | 52.3% bf16 MFU | 1623274 tok/s
step 11227/19560 | loss 3.394652 (+0.24z)| norm 0.2607 (-1.07z)| lr 2.46e-04 | 322.86 ms | 52.3% bf16 MFU | 1623304 tok/s
step 11228/19560 | loss 3.394917 (+0.25z)| norm 0.2649 (-0.82z)| lr 2.45e-04 | 322.64 ms | 52.3% bf16 MFU | 1623388 tok/s
step 11229/19560 | loss 3.364452 (-0.59z)| norm 0.2531 (-1.53z)| lr 2.45e-04 | 323.52 ms | 52.2% bf16 MFU | 1623248 tok/s
step 11230/19560 | loss 3.343951 (-1.14z)| norm 0.2615 (-1.02z)| lr 2.45e-04 | 322.68 ms | 52.3% bf16 MFU | 1623326 tok/s
step 11231/19560 | loss 3.407263 (+0.60z)| norm 0.2504 (-1.67z)| lr 2.45e-04 | 322.81 ms | 52.3% bf16 MFU | 1623366 tok/s
step 11232/19560 | loss 3.436740 (+1.43z)| norm 0.2585 (-1.16z)| lr 2.45e-04 | 323.49 ms | 52.2% bf16 MFU | 1623232 tok/s
step 11233/19560 | loss 3.416280 (+0.85z)| norm 0.2685 (-0.55z)| lr 2.45e-04 | 323.05 ms | 52.2% bf16 MFU | 1623217 tok/s
step 11234/19560 | loss 3.448468 (+1.71z)| norm 0.2755 (-0.12z)| lr 2.45e-04 | 322.25 ms | 52.4% bf16 MFU | 1623404 tok/s
step 11235/19560 | loss 3.467580 (+2.20z)| norm 0.2853 (+0.50z)| lr 2.45e-04 | 322.99 ms | 52.3% bf16 MFU | 1623395 tok/s
step 11236/19560 | loss 3.423439 (+1.02z)| norm 0.2760 (-0.07z)| lr 2.45e-04 | 322.00 ms | 52.4% bf16 MFU | 1623636 tok/s
step 11237/19560 | loss 3.367768 (-0.49z)| norm 0.2726 (-0.27z)| lr 2.45e-04 | 323.55 ms | 52.2% bf16 MFU | 1623476 tok/s
step 11238/19560 | loss 3.435862 (+1.35z)| norm 0.2914 (+0.89z)| lr 2.45e-04 | 323.14 ms | 52.2% bf16 MFU | 1623427 tok/s
step 11239/19560 | loss 3.448722 (+1.67z)| norm 0.2814 (+0.27z)| lr 2.45e-04 | 323.00 ms | 52.3% bf16 MFU | 1623415 tok/s
step 11240/19560 | loss 3.339283 (-1.25z)| norm 0.3023 (+1.55z)| lr 2.45e-04 | 323.05 ms | 52.2% bf16 MFU | 1623392 tok/s
step 11241/19560 | loss 3.375595 (-0.27z)| norm 0.3035 (+1.60z)| lr 2.45e-04 | 323.03 ms | 52.2% bf16 MFU | 1623372 tok/s
step 11242/19560 | loss 3.374951 (-0.29z)| norm 0.2808 (+0.21z)| lr 2.45e-04 | 322.88 ms | 52.3% bf16 MFU | 1623393 tok/s
step 11243/19560 | loss 3.369713 (-0.42z)| norm 0.3053 (+1.67z)| lr 2.45e-04 | 322.65 ms | 52.3% bf16 MFU | 1623471 tok/s
step 11244/19560 | loss 3.386519 (+0.02z)| norm 0.2783 (+0.03z)| lr 2.45e-04 | 322.29 ms | 52.4% bf16 MFU | 1623635 tok/s
step 11245/19560 | loss 3.365448 (-0.54z)| norm 0.2871 (+0.56z)| lr 2.45e-04 | 322.68 ms | 52.3% bf16 MFU | 1623692 tok/s
step 11246/19560 | loss 3.399605 (+0.38z)| norm 0.2832 (+0.32z)| lr 2.45e-04 | 322.94 ms | 52.3% bf16 MFU | 1623681 tok/s
step 11247/19560 | loss 3.384084 (-0.03z)| norm 0.2761 (-0.11z)| lr 2.45e-04 | 322.72 ms | 52.3% bf16 MFU | 1623727 tok/s
step 11248/19560 | loss 3.327023 (-1.56z)| norm 0.2746 (-0.20z)| lr 2.45e-04 | 322.61 ms | 52.3% bf16 MFU | 1623797 tok/s
step 11249/19560 | loss 3.438670 (+1.42z)| norm 0.2866 (+0.52z)| lr 2.44e-04 | 322.39 ms | 52.4% bf16 MFU | 1623920 tok/s
step 11250/19560 | loss 3.393407 (+0.20z)| norm 0.2638 (-0.85z)| lr 2.44e-04 | 323.46 ms | 52.2% bf16 MFU | 1623768 tok/s
val loss 3.367762
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2940/10042 = 0.292770
step 11251/19560 | loss 3.349939 (-0.96z)| norm 0.2933 (+0.92z)| lr 2.44e-04 | 322.25 ms | 52.4% bf16 MFU | 1623928 tok/s
step 11252/19560 | loss 3.400962 (+0.40z)| norm 0.2875 (+0.57z)| lr 2.44e-04 | 323.09 ms | 52.2% bf16 MFU | 1623869 tok/s
step 11253/19560 | loss 3.378553 (-0.22z)| norm 0.2725 (-0.33z)| lr 2.44e-04 | 324.25 ms | 52.0% bf16 MFU | 1623521 tok/s
step 11254/19560 | loss 3.410399 (+0.65z)| norm 0.2827 (+0.29z)| lr 2.44e-04 | 322.84 ms | 52.3% bf16 MFU | 1623544 tok/s
step 11255/19560 | loss 3.339971 (-1.27z)| norm 0.2612 (-1.00z)| lr 2.44e-04 | 323.42 ms | 52.2% bf16 MFU | 1623419 tok/s
step 11256/19560 | loss 3.400652 (+0.38z)| norm 0.2748 (-0.18z)| lr 2.44e-04 | 324.22 ms | 52.1% bf16 MFU | 1623102 tok/s
step 11257/19560 | loss 3.406282 (+0.53z)| norm 0.2663 (-0.68z)| lr 2.44e-04 | 323.50 ms | 52.2% bf16 MFU | 1622981 tok/s
step 11258/19560 | loss 3.392722 (+0.16z)| norm 0.2619 (-0.94z)| lr 2.44e-04 | 323.10 ms | 52.2% bf16 MFU | 1622965 tok/s
step 11259/19560 | loss 3.343314 (-1.17z)| norm 0.2780 (+0.05z)| lr 2.44e-04 | 323.50 ms | 52.2% bf16 MFU | 1622851 tok/s
step 11260/19560 | loss 3.374221 (-0.33z)| norm 0.2720 (-0.32z)| lr 2.44e-04 | 322.92 ms | 52.3% bf16 MFU | 1622887 tok/s
step 11261/19560 | loss 3.347782 (-1.04z)| norm 0.2719 (-0.33z)| lr 2.44e-04 | 323.25 ms | 52.2% bf16 MFU | 1622839 tok/s
step 11262/19560 | loss 3.391224 (+0.12z)| norm 0.2727 (-0.29z)| lr 2.44e-04 | 323.12 ms | 52.2% bf16 MFU | 1622827 tok/s
step 11263/19560 | loss 3.425869 (+1.05z)| norm 0.2714 (-0.38z)| lr 2.44e-04 | 323.09 ms | 52.2% bf16 MFU | 1622822 tok/s
step 11264/19560 | loss 3.356398 (-0.84z)| norm 0.3235 (+2.72z)| lr 2.44e-04 | 323.19 ms | 52.2% bf16 MFU | 1622792 tok/s
step 11265/19560 | loss 3.418895 (+0.85z)| norm 0.2597 (-1.09z)| lr 2.44e-04 | 323.03 ms | 52.2% bf16 MFU | 1622803 tok/s
step 11266/19560 | loss 3.390304 (+0.07z)| norm 0.3046 (+1.56z)| lr 2.44e-04 | 322.15 ms | 52.4% bf16 MFU | 1623036 tok/s
step 11267/19560 | loss 3.360223 (-0.74z)| norm 0.2939 (+0.91z)| lr 2.44e-04 | 323.05 ms | 52.2% bf16 MFU | 1623030 tok/s
step 11268/19560 | loss 3.367178 (-0.55z)| norm 0.2896 (+0.65z)| lr 2.44e-04 | 322.87 ms | 52.3% bf16 MFU | 1623071 tok/s
step 11269/19560 | loss 3.398496 (+0.30z)| norm 0.3030 (+1.43z)| lr 2.43e-04 | 322.71 ms | 52.3% bf16 MFU | 1623150 tok/s
step 11270/19560 | loss 3.372551 (-0.42z)| norm 0.2824 (+0.20z)| lr 2.43e-04 | 322.63 ms | 52.3% bf16 MFU | 1623245 tok/s
step 11271/19560 | loss 3.426566 (+1.06z)| norm 0.2808 (+0.11z)| lr 2.43e-04 | 322.27 ms | 52.4% bf16 MFU | 1623427 tok/s
step 11272/19560 | loss 3.370248 (-0.48z)| norm 0.3004 (+1.26z)| lr 2.43e-04 | 322.75 ms | 52.3% bf16 MFU | 1623478 tok/s
step 11273/19560 | loss 3.461196 (+1.97z)| norm 0.2795 (+0.03z)| lr 2.43e-04 | 323.61 ms | 52.2% bf16 MFU | 1623310 tok/s
step 11274/19560 | loss 3.338926 (-1.33z)| norm 0.2813 (+0.13z)| lr 2.43e-04 | 322.57 ms | 52.3% bf16 MFU | 1623412 tok/s
step 11275/19560 | loss 3.371026 (-0.48z)| norm 0.3451 (+3.67z)| lr 2.43e-04 | 322.87 ms | 52.3% bf16 MFU | 1623432 tok/s
step 11276/19560 | loss 3.364171 (-0.66z)| norm 0.2726 (-0.41z)| lr 2.43e-04 | 322.64 ms | 52.3% bf16 MFU | 1623511 tok/s
step 11277/19560 | loss 3.337303 (-1.38z)| norm 0.3002 (+1.14z)| lr 2.43e-04 | 322.46 ms | 52.3% bf16 MFU | 1623630 tok/s
step 11278/19560 | loss 3.353714 (-0.92z)| norm 0.2716 (-0.48z)| lr 2.43e-04 | 322.66 ms | 52.3% bf16 MFU | 1623693 tok/s
step 11279/19560 | loss 3.428047 (+1.06z)| norm 0.2816 (+0.07z)| lr 2.43e-04 | 322.96 ms | 52.3% bf16 MFU | 1623678 tok/s
step 11280/19560 | loss 3.398269 (+0.26z)| norm 0.2649 (-0.91z)| lr 2.43e-04 | 322.94 ms | 52.3% bf16 MFU | 1623669 tok/s
step 11281/19560 | loss 3.349710 (-1.05z)| norm 0.2827 (+0.12z)| lr 2.43e-04 | 322.68 ms | 52.3% bf16 MFU | 1623726 tok/s
step 11282/19560 | loss 3.369274 (-0.52z)| norm 0.2625 (-1.07z)| lr 2.43e-04 | 322.28 ms | 52.4% bf16 MFU | 1623880 tok/s
step 11283/19560 | loss 3.353822 (-0.94z)| norm 0.2777 (-0.18z)| lr 2.43e-04 | 323.21 ms | 52.2% bf16 MFU | 1623794 tok/s
step 11284/19560 | loss 3.350379 (-1.02z)| norm 0.2704 (-0.61z)| lr 2.43e-04 | 322.32 ms | 52.4% bf16 MFU | 1623935 tok/s
step 11285/19560 | loss 3.336398 (-1.38z)| norm 0.3257 (+2.56z)| lr 2.43e-04 | 322.44 ms | 52.3% bf16 MFU | 1624037 tok/s
step 11286/19560 | loss 3.386799 (-0.03z)| norm 0.2974 (+0.91z)| lr 2.43e-04 | 323.23 ms | 52.2% bf16 MFU | 1623937 tok/s
step 11287/19560 | loss 3.319681 (-1.79z)| norm 0.2618 (-1.13z)| lr 2.43e-04 | 322.60 ms | 52.3% bf16 MFU | 1624000 tok/s
step 11288/19560 | loss 3.431484 (+1.14z)| norm 0.3062 (+1.40z)| lr 2.43e-04 | 323.48 ms | 52.2% bf16 MFU | 1623839 tok/s
step 11289/19560 | loss 3.341716 (-1.21z)| norm 0.2549 (-1.54z)| lr 2.42e-04 | 322.70 ms | 52.3% bf16 MFU | 1623881 tok/s
step 11290/19560 | loss 3.366098 (-0.58z)| norm 0.2955 (+0.77z)| lr 2.42e-04 | 322.76 ms | 52.3% bf16 MFU | 1623905 tok/s
step 11291/19560 | loss 3.357759 (-0.80z)| norm 0.2834 (+0.07z)| lr 2.42e-04 | 322.93 ms | 52.3% bf16 MFU | 1623888 tok/s
step 11292/19560 | loss 3.422836 (+0.91z)| norm 0.2948 (+0.72z)| lr 2.42e-04 | 322.83 ms | 52.3% bf16 MFU | 1623895 tok/s
step 11293/19560 | loss 3.416729 (+0.74z)| norm 0.2902 (+0.46z)| lr 2.42e-04 | 323.03 ms | 52.2% bf16 MFU | 1623851 tok/s
step 11294/19560 | loss 3.393452 (+0.13z)| norm 0.2644 (-1.02z)| lr 2.42e-04 | 322.54 ms | 52.3% bf16 MFU | 1623935 tok/s
step 11295/19560 | loss 3.375315 (-0.37z)| norm 0.2824 (+0.01z)| lr 2.42e-04 | 323.19 ms | 52.2% bf16 MFU | 1623848 tok/s
step 11296/19560 | loss 3.428082 (+1.03z)| norm 0.2765 (-0.33z)| lr 2.42e-04 | 322.96 ms | 52.3% bf16 MFU | 1623824 tok/s
step 11297/19560 | loss 3.338974 (-1.38z)| norm 0.2770 (-0.30z)| lr 2.42e-04 | 322.75 ms | 52.3% bf16 MFU | 1623854 tok/s
step 11298/19560 | loss 3.351276 (-1.05z)| norm 0.2729 (-0.53z)| lr 2.42e-04 | 322.90 ms | 52.3% bf16 MFU | 1623846 tok/s
step 11299/19560 | loss 3.415585 (+0.74z)| norm 0.2721 (-0.57z)| lr 2.42e-04 | 323.27 ms | 52.2% bf16 MFU | 1623746 tok/s
step 11300/19560 | loss 3.383120 (-0.17z)| norm 0.3186 (+2.18z)| lr 2.42e-04 | 322.53 ms | 52.3% bf16 MFU | 1623836 tok/s
step 11301/19560 | loss 3.348356 (-1.13z)| norm 0.2945 (+0.77z)| lr 2.42e-04 | 323.07 ms | 52.2% bf16 MFU | 1623785 tok/s
step 11302/19560 | loss 3.420952 (+0.89z)| norm 0.3061 (+1.44z)| lr 2.42e-04 | 322.41 ms | 52.3% bf16 MFU | 1623903 tok/s
step 11303/19560 | loss 3.352058 (-1.02z)| norm 0.2882 (+0.39z)| lr 2.42e-04 | 323.18 ms | 52.2% bf16 MFU | 1623821 tok/s
step 11304/19560 | loss 3.377208 (-0.32z)| norm 0.2951 (+0.81z)| lr 2.42e-04 | 323.08 ms | 52.2% bf16 MFU | 1623770 tok/s
step 11305/19560 | loss 3.385046 (-0.11z)| norm 0.3095 (+1.65z)| lr 2.42e-04 | 322.67 ms | 52.3% bf16 MFU | 1623824 tok/s
step 11306/19560 | loss 3.399029 (+0.29z)| norm 0.3440 (+3.55z)| lr 2.42e-04 | 322.75 ms | 52.3% bf16 MFU | 1623855 tok/s
step 11307/19560 | loss 3.514551 (+3.35z)| norm 0.3470 (+3.51z)| lr 2.42e-04 | 323.04 ms | 52.2% bf16 MFU | 1623812 tok/s
step 11308/19560 | loss 3.351064 (-1.03z)| norm 0.2745 (-0.43z)| lr 2.42e-04 | 322.79 ms | 52.3% bf16 MFU | 1623834 tok/s
step 11309/19560 | loss 3.367267 (-0.58z)| norm 0.2863 (+0.21z)| lr 2.42e-04 | 322.83 ms | 52.3% bf16 MFU | 1623844 tok/s
step 11310/19560 | loss 3.443287 (+1.46z)| norm 0.2865 (+0.22z)| lr 2.41e-04 | 323.10 ms | 52.2% bf16 MFU | 1623786 tok/s
step 11311/19560 | loss 3.286265 (-2.67z)| norm 0.2874 (+0.26z)| lr 2.41e-04 | 322.98 ms | 52.3% bf16 MFU | 1623762 tok/s
step 11312/19560 | loss 3.346215 (-1.09z)| norm 0.2930 (+0.55z)| lr 2.41e-04 | 323.11 ms | 52.2% bf16 MFU | 1623705 tok/s
step 11313/19560 | loss 3.359225 (-0.74z)| norm 0.2623 (-1.13z)| lr 2.41e-04 | 322.85 ms | 52.3% bf16 MFU | 1623716 tok/s
step 11314/19560 | loss 3.477875 (+2.29z)| norm 0.3320 (+2.60z)| lr 2.41e-04 | 323.18 ms | 52.2% bf16 MFU | 1623645 tok/s
step 11315/19560 | loss 3.410455 (+0.55z)| norm 0.2864 (+0.17z)| lr 2.41e-04 | 322.61 ms | 52.3% bf16 MFU | 1623720 tok/s
step 11316/19560 | loss 3.390046 (+0.04z)| norm 0.3026 (+1.02z)| lr 2.41e-04 | 322.38 ms | 52.4% bf16 MFU | 1623849 tok/s
step 11317/19560 | loss 3.330055 (-1.49z)| norm 0.2659 (-0.92z)| lr 2.41e-04 | 323.22 ms | 52.2% bf16 MFU | 1623761 tok/s
step 11318/19560 | loss 3.409535 (+0.56z)| norm 0.2878 (+0.24z)| lr 2.41e-04 | 323.17 ms | 52.2% bf16 MFU | 1623690 tok/s
step 11319/19560 | loss 3.391452 (+0.08z)| norm 0.2651 (-0.96z)| lr 2.41e-04 | 322.88 ms | 52.3% bf16 MFU | 1623695 tok/s
step 11320/19560 | loss 3.435986 (+1.22z)| norm 0.2773 (-0.31z)| lr 2.41e-04 | 322.79 ms | 52.3% bf16 MFU | 1623721 tok/s
step 11321/19560 | loss 3.394653 (+0.15z)| norm 0.2725 (-0.56z)| lr 2.41e-04 | 322.63 ms | 52.3% bf16 MFU | 1623787 tok/s
step 11322/19560 | loss 3.394236 (+0.13z)| norm 0.2767 (-0.35z)| lr 2.41e-04 | 323.45 ms | 52.2% bf16 MFU | 1623644 tok/s
step 11323/19560 | loss 3.357311 (-0.82z)| norm 0.2754 (-0.41z)| lr 2.41e-04 | 322.69 ms | 52.3% bf16 MFU | 1623699 tok/s
step 11324/19560 | loss 3.359451 (-0.76z)| norm 0.2697 (-0.73z)| lr 2.41e-04 | 323.37 ms | 52.2% bf16 MFU | 1623580 tok/s
step 11325/19560 | loss 3.361821 (-0.69z)| norm 0.2720 (-0.59z)| lr 2.41e-04 | 322.81 ms | 52.3% bf16 MFU | 1623608 tok/s
step 11326/19560 | loss 3.356405 (-0.83z)| norm 0.2590 (-1.29z)| lr 2.41e-04 | 322.61 ms | 52.3% bf16 MFU | 1623685 tok/s
step 11327/19560 | loss 3.416913 (+0.73z)| norm 0.2962 (+0.72z)| lr 2.41e-04 | 323.69 ms | 52.1% bf16 MFU | 1623486 tok/s
step 11328/19560 | loss 3.407787 (+0.49z)| norm 0.2615 (-1.14z)| lr 2.41e-04 | 323.50 ms | 52.2% bf16 MFU | 1623346 tok/s
step 11329/19560 | loss 3.352269 (-0.93z)| norm 0.3045 (+1.16z)| lr 2.41e-04 | 322.75 ms | 52.3% bf16 MFU | 1623400 tok/s
step 11330/19560 | loss 3.390600 (+0.05z)| norm 0.2736 (-0.50z)| lr 2.40e-04 | 322.75 ms | 52.3% bf16 MFU | 1623451 tok/s
step 11331/19560 | loss 3.360388 (-0.73z)| norm 0.3115 (+1.51z)| lr 2.40e-04 | 322.68 ms | 52.3% bf16 MFU | 1623518 tok/s
step 11332/19560 | loss 3.372962 (-0.40z)| norm 0.2659 (-0.93z)| lr 2.40e-04 | 323.25 ms | 52.2% bf16 MFU | 1623440 tok/s
step 11333/19560 | loss 3.429404 (+1.05z)| norm 0.2787 (-0.25z)| lr 2.40e-04 | 322.86 ms | 52.3% bf16 MFU | 1623461 tok/s
step 11334/19560 | loss 3.358346 (-0.79z)| norm 0.2697 (-0.73z)| lr 2.40e-04 | 323.80 ms | 52.1% bf16 MFU | 1623247 tok/s
step 11335/19560 | loss 3.436059 (+1.21z)| norm 0.2886 (+0.29z)| lr 2.40e-04 | 322.56 ms | 52.3% bf16 MFU | 1623354 tok/s
step 11336/19560 | loss 3.353584 (-0.90z)| norm 0.2640 (-1.03z)| lr 2.40e-04 | 323.06 ms | 52.2% bf16 MFU | 1623331 tok/s
step 11337/19560 | loss 3.395318 (+0.19z)| norm 0.2874 (+0.23z)| lr 2.40e-04 | 322.97 ms | 52.3% bf16 MFU | 1623332 tok/s
step 11338/19560 | loss 3.351158 (-0.96z)| norm 0.2702 (-0.69z)| lr 2.40e-04 | 322.87 ms | 52.3% bf16 MFU | 1623356 tok/s
step 11339/19560 | loss 3.339150 (-1.26z)| norm 0.2786 (-0.23z)| lr 2.40e-04 | 322.82 ms | 52.3% bf16 MFU | 1623393 tok/s
step 11340/19560 | loss 3.424953 (+0.98z)| norm 0.2595 (-1.25z)| lr 2.40e-04 | 322.44 ms | 52.3% bf16 MFU | 1623524 tok/s
step 11341/19560 | loss 3.456024 (+1.79z)| norm 0.2866 (+0.23z)| lr 2.40e-04 | 323.05 ms | 52.2% bf16 MFU | 1623495 tok/s
step 11342/19560 | loss 3.412059 (+0.63z)| norm 0.2845 (+0.12z)| lr 2.40e-04 | 322.61 ms | 52.3% bf16 MFU | 1623577 tok/s
step 11343/19560 | loss 3.309792 (-1.99z)| norm 0.2619 (-1.11z)| lr 2.40e-04 | 323.16 ms | 52.2% bf16 MFU | 1623518 tok/s
step 11344/19560 | loss 3.433188 (+1.17z)| norm 0.2921 (+0.54z)| lr 2.40e-04 | 322.66 ms | 52.3% bf16 MFU | 1623586 tok/s
step 11345/19560 | loss 3.412740 (+0.65z)| norm 0.2807 (-0.08z)| lr 2.40e-04 | 322.72 ms | 52.3% bf16 MFU | 1623637 tok/s
step 11346/19560 | loss 3.537464 (+3.63z)| norm 0.2774 (-0.26z)| lr 2.40e-04 | 323.36 ms | 52.2% bf16 MFU | 1623524 tok/s
step 11347/19560 | loss 3.358624 (-0.73z)| norm 0.2908 (+0.46z)| lr 2.40e-04 | 322.45 ms | 52.3% bf16 MFU | 1623645 tok/s
step 11348/19560 | loss 3.386822 (-0.05z)| norm 0.2626 (-1.07z)| lr 2.40e-04 | 323.88 ms | 52.1% bf16 MFU | 1623402 tok/s
step 11349/19560 | loss 3.369936 (-0.45z)| norm 0.3025 (+1.09z)| lr 2.40e-04 | 323.59 ms | 52.2% bf16 MFU | 1623242 tok/s
step 11350/19560 | loss 3.341420 (-1.13z)| norm 0.2918 (+0.50z)| lr 2.40e-04 | 323.15 ms | 52.2% bf16 MFU | 1623200 tok/s
step 11351/19560 | loss 3.413208 (+0.62z)| norm 0.2783 (-0.23z)| lr 2.39e-04 | 323.74 ms | 52.1% bf16 MFU | 1623015 tok/s
step 11352/19560 | loss 3.382321 (-0.12z)| norm 0.2777 (-0.28z)| lr 2.39e-04 | 323.66 ms | 52.1% bf16 MFU | 1622858 tok/s
step 11353/19560 | loss 3.410635 (+0.60z)| norm 0.2888 (+0.33z)| lr 2.39e-04 | 323.25 ms | 52.2% bf16 MFU | 1622812 tok/s
step 11354/19560 | loss 3.442511 (+1.39z)| norm 0.2719 (-0.59z)| lr 2.39e-04 | 322.70 ms | 52.3% bf16 MFU | 1622905 tok/s
step 11355/19560 | loss 3.342445 (-1.10z)| norm 0.2625 (-1.10z)| lr 2.39e-04 | 323.29 ms | 52.2% bf16 MFU | 1622845 tok/s
step 11356/19560 | loss 3.411258 (+0.61z)| norm 0.2813 (-0.09z)| lr 2.39e-04 | 323.36 ms | 52.2% bf16 MFU | 1622772 tok/s
step 11357/19560 | loss 3.403519 (+0.41z)| norm 0.2556 (-1.49z)| lr 2.39e-04 | 322.83 ms | 52.3% bf16 MFU | 1622836 tok/s
step 11358/19560 | loss 3.429505 (+1.04z)| norm 0.2542 (-1.56z)| lr 2.39e-04 | 323.20 ms | 52.2% bf16 MFU | 1622803 tok/s
step 11359/19560 | loss 3.345632 (-1.04z)| norm 0.2552 (-1.52z)| lr 2.39e-04 | 323.61 ms | 52.2% bf16 MFU | 1622669 tok/s
step 11360/19560 | loss 3.361594 (-0.63z)| norm 0.2463 (-1.98z)| lr 2.39e-04 | 323.52 ms | 52.2% bf16 MFU | 1622565 tok/s
step 11361/19560 | loss 3.421134 (+0.86z)| norm 0.2747 (-0.44z)| lr 2.39e-04 | 322.98 ms | 52.3% bf16 MFU | 1622600 tok/s
step 11362/19560 | loss 3.390769 (+0.11z)| norm 0.2475 (-1.89z)| lr 2.39e-04 | 323.27 ms | 52.2% bf16 MFU | 1622560 tok/s
step 11363/19560 | loss 3.354399 (-0.79z)| norm 0.2818 (-0.04z)| lr 2.39e-04 | 322.77 ms | 52.3% bf16 MFU | 1622648 tok/s
step 11364/19560 | loss 3.337059 (-1.21z)| norm 0.2511 (-1.66z)| lr 2.39e-04 | 323.17 ms | 52.2% bf16 MFU | 1622632 tok/s
step 11365/19560 | loss 3.374363 (-0.27z)| norm 0.2636 (-0.99z)| lr 2.39e-04 | 323.95 ms | 52.1% bf16 MFU | 1622421 tok/s
step 11366/19560 | loss 3.440189 (+1.41z)| norm 0.3081 (+1.36z)| lr 2.39e-04 | 323.67 ms | 52.1% bf16 MFU | 1622291 tok/s
step 11367/19560 | loss 3.378811 (-0.14z)| norm 0.2656 (-0.87z)| lr 2.39e-04 | 323.82 ms | 52.1% bf16 MFU | 1622129 tok/s
step 11368/19560 | loss 3.377575 (-0.18z)| norm 0.3063 (+1.26z)| lr 2.39e-04 | 323.21 ms | 52.2% bf16 MFU | 1622128 tok/s
step 11369/19560 | loss 3.394562 (+0.26z)| norm 0.2662 (-0.83z)| lr 2.39e-04 | 322.80 ms | 52.3% bf16 MFU | 1622231 tok/s
step 11370/19560 | loss 3.434824 (+1.28z)| norm 0.2924 (+0.54z)| lr 2.39e-04 | 323.48 ms | 52.2% bf16 MFU | 1622159 tok/s
step 11371/19560 | loss 3.347746 (-0.96z)| norm 0.2709 (-0.58z)| lr 2.38e-04 | 323.45 ms | 52.2% bf16 MFU | 1622097 tok/s
step 11372/19560 | loss 3.593471 (+4.80z)| norm 0.2985 (+0.87z)| lr 2.38e-04 | 323.12 ms | 52.2% bf16 MFU | 1622121 tok/s
step 11373/19560 | loss 3.312921 (-1.68z)| norm 0.2712 (-0.56z)| lr 2.38e-04 | 323.85 ms | 52.1% bf16 MFU | 1621962 tok/s
step 11374/19560 | loss 3.331257 (-1.24z)| norm 0.2883 (+0.34z)| lr 2.38e-04 | 322.90 ms | 52.3% bf16 MFU | 1622048 tok/s
step 11375/19560 | loss 3.404339 (+0.42z)| norm 0.2776 (-0.23z)| lr 2.38e-04 | 322.90 ms | 52.3% bf16 MFU | 1622128 tok/s
step 11376/19560 | loss 3.380255 (-0.14z)| norm 0.2767 (-0.28z)| lr 2.38e-04 | 322.98 ms | 52.3% bf16 MFU | 1622185 tok/s
step 11377/19560 | loss 3.360752 (-0.57z)| norm 0.2875 (+0.29z)| lr 2.38e-04 | 323.24 ms | 52.2% bf16 MFU | 1622174 tok/s
step 11378/19560 | loss 3.345601 (-0.91z)| norm 0.2734 (-0.45z)| lr 2.38e-04 | 323.08 ms | 52.2% bf16 MFU | 1622205 tok/s
step 11379/19560 | loss 3.420688 (+0.80z)| norm 0.3040 (+1.15z)| lr 2.38e-04 | 322.61 ms | 52.3% bf16 MFU | 1622352 tok/s
step 11380/19560 | loss 3.405348 (+0.45z)| norm 0.3101 (+1.45z)| lr 2.38e-04 | 322.90 ms | 52.3% bf16 MFU | 1622420 tok/s
step 11381/19560 | loss 3.380025 (-0.13z)| norm 0.2847 (+0.12z)| lr 2.38e-04 | 322.85 ms | 52.3% bf16 MFU | 1622495 tok/s
step 11382/19560 | loss 3.402356 (+0.38z)| norm 0.2883 (+0.31z)| lr 2.38e-04 | 323.39 ms | 52.2% bf16 MFU | 1622432 tok/s
step 11383/19560 | loss 3.452492 (+1.51z)| norm 0.3087 (+1.35z)| lr 2.38e-04 | 323.25 ms | 52.2% bf16 MFU | 1622406 tok/s
step 11384/19560 | loss 3.444139 (+1.30z)| norm 0.2716 (-0.58z)| lr 2.38e-04 | 322.43 ms | 52.3% bf16 MFU | 1622588 tok/s
step 11385/19560 | loss 3.378070 (-0.20z)| norm 0.2626 (-1.04z)| lr 2.38e-04 | 322.36 ms | 52.4% bf16 MFU | 1622780 tok/s
step 11386/19560 | loss 3.420846 (+0.77z)| norm 0.2947 (+0.61z)| lr 2.38e-04 | 323.58 ms | 52.2% bf16 MFU | 1622655 tok/s
step 11387/19560 | loss 3.470077 (+1.85z)| norm 0.2703 (-0.66z)| lr 2.38e-04 | 322.89 ms | 52.3% bf16 MFU | 1622708 tok/s
step 11388/19560 | loss 3.426740 (+0.86z)| norm 0.2693 (-0.71z)| lr 2.38e-04 | 323.09 ms | 52.2% bf16 MFU | 1622709 tok/s
step 11389/19560 | loss 3.389289 (+0.01z)| norm 0.2807 (-0.12z)| lr 2.38e-04 | 323.03 ms | 52.2% bf16 MFU | 1622726 tok/s
step 11390/19560 | loss 3.316951 (-1.59z)| norm 0.2682 (-0.76z)| lr 2.38e-04 | 322.60 ms | 52.3% bf16 MFU | 1622848 tok/s
step 11391/19560 | loss 3.432075 (+0.98z)| norm 0.2902 (+0.37z)| lr 2.37e-04 | 322.53 ms | 52.3% bf16 MFU | 1622983 tok/s
step 11392/19560 | loss 3.433199 (+0.99z)| norm 0.2692 (-0.71z)| lr 2.37e-04 | 322.94 ms | 52.3% bf16 MFU | 1623010 tok/s
step 11393/19560 | loss 3.410547 (+0.49z)| norm 0.2847 (+0.10z)| lr 2.37e-04 | 322.85 ms | 52.3% bf16 MFU | 1623057 tok/s
step 11394/19560 | loss 3.423481 (+0.77z)| norm 0.3124 (+1.56z)| lr 2.37e-04 | 322.37 ms | 52.4% bf16 MFU | 1623222 tok/s
step 11395/19560 | loss 3.362748 (-0.58z)| norm 0.2981 (+0.80z)| lr 2.37e-04 | 322.78 ms | 52.3% bf16 MFU | 1623275 tok/s
step 11396/19560 | loss 3.373092 (-0.35z)| norm 0.2791 (-0.20z)| lr 2.37e-04 | 323.32 ms | 52.2% bf16 MFU | 1623191 tok/s
step 11397/19560 | loss 3.366408 (-0.50z)| norm 0.2815 (-0.07z)| lr 2.37e-04 | 322.36 ms | 52.4% bf16 MFU | 1623352 tok/s
step 11398/19560 | loss 3.349081 (-0.88z)| norm 0.2862 (+0.18z)| lr 2.37e-04 | 322.40 ms | 52.3% bf16 MFU | 1623493 tok/s
step 11399/19560 | loss 3.358907 (-0.65z)| norm 0.2639 (-0.99z)| lr 2.37e-04 | 323.26 ms | 52.2% bf16 MFU | 1623413 tok/s
step 11400/19560 | loss 3.386358 (-0.04z)| norm 0.2976 (+0.80z)| lr 2.37e-04 | 322.77 ms | 52.3% bf16 MFU | 1623460 tok/s
step 11401/19560 | loss 3.365861 (-0.48z)| norm 0.2780 (-0.24z)| lr 2.37e-04 | 322.92 ms | 52.3% bf16 MFU | 1623466 tok/s
step 11402/19560 | loss 3.335799 (-1.16z)| norm 0.2950 (+0.65z)| lr 2.37e-04 | 322.95 ms | 52.3% bf16 MFU | 1623464 tok/s
step 11403/19560 | loss 3.577778 (+3.96z)| norm 0.2767 (-0.30z)| lr 2.37e-04 | 323.36 ms | 52.2% bf16 MFU | 1623360 tok/s
step 11404/19560 | loss 3.413153 (+0.50z)| norm 0.3099 (+1.50z)| lr 2.37e-04 | 322.67 ms | 52.3% bf16 MFU | 1623434 tok/s
step 11405/19560 | loss 3.413753 (+0.50z)| norm 0.2783 (-0.22z)| lr 2.37e-04 | 322.38 ms | 52.4% bf16 MFU | 1623577 tok/s
step 11406/19560 | loss 3.363430 (-0.56z)| norm 0.2741 (-0.45z)| lr 2.37e-04 | 322.92 ms | 52.3% bf16 MFU | 1623576 tok/s
step 11407/19560 | loss 3.345068 (-0.94z)| norm 0.2990 (+0.91z)| lr 2.37e-04 | 322.84 ms | 52.3% bf16 MFU | 1623595 tok/s
step 11408/19560 | loss 3.314730 (-1.55z)| norm 0.2633 (-1.04z)| lr 2.37e-04 | 323.00 ms | 52.3% bf16 MFU | 1623574 tok/s
step 11409/19560 | loss 3.345699 (-0.90z)| norm 0.2959 (+0.73z)| lr 2.37e-04 | 323.23 ms | 52.2% bf16 MFU | 1623498 tok/s
step 11410/19560 | loss 3.385134 (-0.08z)| norm 0.2853 (+0.14z)| lr 2.37e-04 | 322.76 ms | 52.3% bf16 MFU | 1623542 tok/s
step 11411/19560 | loss 3.387252 (-0.04z)| norm 0.2855 (+0.15z)| lr 2.37e-04 | 322.93 ms | 52.3% bf16 MFU | 1623541 tok/s
step 11412/19560 | loss 3.383226 (-0.13z)| norm 0.2659 (-0.92z)| lr 2.36e-04 | 322.66 ms | 52.3% bf16 MFU | 1623608 tok/s
step 11413/19560 | loss 3.375375 (-0.30z)| norm 0.2774 (-0.28z)| lr 2.36e-04 | 322.46 ms | 52.3% bf16 MFU | 1623724 tok/s
step 11414/19560 | loss 3.400792 (+0.23z)| norm 0.2906 (+0.46z)| lr 2.36e-04 | 322.38 ms | 52.4% bf16 MFU | 1623853 tok/s
step 11415/19560 | loss 3.401385 (+0.23z)| norm 0.2803 (-0.12z)| lr 2.36e-04 | 323.17 ms | 52.2% bf16 MFU | 1623776 tok/s
step 11416/19560 | loss 3.389885 (-0.01z)| norm 0.3076 (+1.42z)| lr 2.36e-04 | 322.97 ms | 52.3% bf16 MFU | 1623754 tok/s
step 11417/19560 | loss 3.330446 (-1.27z)| norm 0.3183 (+1.98z)| lr 2.36e-04 | 322.36 ms | 52.4% bf16 MFU | 1623886 tok/s
step 11418/19560 | loss 3.398022 (+0.16z)| norm 0.2728 (-0.56z)| lr 2.36e-04 | 322.49 ms | 52.3% bf16 MFU | 1623980 tok/s
step 11419/19560 | loss 3.401576 (+0.23z)| norm 0.2899 (+0.40z)| lr 2.36e-04 | 322.80 ms | 52.3% bf16 MFU | 1623990 tok/s
step 11420/19560 | loss 3.373033 (-0.37z)| norm 0.2690 (-0.76z)| lr 2.36e-04 | 322.71 ms | 52.3% bf16 MFU | 1624022 tok/s
step 11421/19560 | loss 3.359185 (-0.65z)| norm 0.2806 (-0.11z)| lr 2.36e-04 | 323.06 ms | 52.2% bf16 MFU | 1623964 tok/s
step 11422/19560 | loss 3.404334 (+0.31z)| norm 0.2671 (-0.87z)| lr 2.36e-04 | 322.75 ms | 52.3% bf16 MFU | 1623987 tok/s
step 11423/19560 | loss 3.370276 (-0.42z)| norm 0.2617 (-1.16z)| lr 2.36e-04 | 322.25 ms | 52.4% bf16 MFU | 1624135 tok/s
step 11424/19560 | loss 3.410569 (+0.44z)| norm 0.2696 (-0.71z)| lr 2.36e-04 | 322.84 ms | 52.3% bf16 MFU | 1624129 tok/s
step 11425/19560 | loss 3.421516 (+0.67z)| norm 0.3639 (+4.19z)| lr 2.36e-04 | 322.70 ms | 52.3% bf16 MFU | 1624156 tok/s
step 11426/19560 | loss 3.350587 (-0.85z)| norm 0.2945 (+0.59z)| lr 2.36e-04 | 322.99 ms | 52.3% bf16 MFU | 1624109 tok/s
step 11427/19560 | loss 3.385116 (-0.11z)| norm 0.2750 (-0.43z)| lr 2.36e-04 | 322.58 ms | 52.3% bf16 MFU | 1624168 tok/s
step 11428/19560 | loss 3.403124 (+0.27z)| norm 0.2702 (-0.66z)| lr 2.36e-04 | 322.99 ms | 52.3% bf16 MFU | 1624121 tok/s
step 11429/19560 | loss 3.400949 (+0.22z)| norm 0.2575 (-1.30z)| lr 2.36e-04 | 322.94 ms | 52.3% bf16 MFU | 1624089 tok/s
step 11430/19560 | loss 3.354250 (-0.77z)| norm 0.2771 (-0.27z)| lr 2.36e-04 | 322.71 ms | 52.3% bf16 MFU | 1624117 tok/s
step 11431/19560 | loss 3.388403 (-0.04z)| norm 0.2696 (-0.66z)| lr 2.36e-04 | 322.61 ms | 52.3% bf16 MFU | 1624169 tok/s
step 11432/19560 | loss 3.368444 (-0.47z)| norm 0.2637 (-0.96z)| lr 2.35e-04 | 322.76 ms | 52.3% bf16 MFU | 1624181 tok/s
step 11433/19560 | loss 3.447565 (+1.21z)| norm 0.2987 (+0.88z)| lr 2.35e-04 | 323.06 ms | 52.2% bf16 MFU | 1624116 tok/s
step 11434/19560 | loss 3.370270 (-0.44z)| norm 0.2916 (+0.56z)| lr 2.35e-04 | 322.72 ms | 52.3% bf16 MFU | 1624141 tok/s
step 11435/19560 | loss 3.383688 (-0.13z)| norm 0.2879 (+0.40z)| lr 2.35e-04 | 323.16 ms | 52.2% bf16 MFU | 1624054 tok/s
step 11436/19560 | loss 3.516373 (+2.69z)| norm 0.2979 (+0.96z)| lr 2.35e-04 | 322.62 ms | 52.3% bf16 MFU | 1624106 tok/s
step 11437/19560 | loss 3.385255 (-0.12z)| norm 0.2921 (+0.63z)| lr 2.35e-04 | 322.71 ms | 52.3% bf16 MFU | 1624133 tok/s
step 11438/19560 | loss 3.359485 (-0.67z)| norm 0.2836 (+0.14z)| lr 2.35e-04 | 322.56 ms | 52.3% bf16 MFU | 1624197 tok/s
step 11439/19560 | loss 3.374127 (-0.37z)| norm 0.2927 (+0.66z)| lr 2.35e-04 | 322.74 ms | 52.3% bf16 MFU | 1624210 tok/s
step 11440/19560 | loss 3.414571 (+0.50z)| norm 0.2723 (-0.51z)| lr 2.35e-04 | 322.64 ms | 52.3% bf16 MFU | 1624250 tok/s
step 11441/19560 | loss 3.340610 (-1.12z)| norm 0.2730 (-0.47z)| lr 2.35e-04 | 322.28 ms | 52.4% bf16 MFU | 1624378 tok/s
step 11442/19560 | loss 3.285095 (-2.29z)| norm 0.2564 (-1.43z)| lr 2.35e-04 | 322.71 ms | 52.3% bf16 MFU | 1624391 tok/s
step 11443/19560 | loss 3.423633 (+0.73z)| norm 0.2699 (-0.63z)| lr 2.35e-04 | 323.06 ms | 52.2% bf16 MFU | 1624316 tok/s
step 11444/19560 | loss 3.403936 (+0.30z)| norm 0.2633 (-1.00z)| lr 2.35e-04 | 323.15 ms | 52.2% bf16 MFU | 1624222 tok/s
step 11445/19560 | loss 3.388769 (-0.04z)| norm 0.2685 (-0.69z)| lr 2.35e-04 | 322.68 ms | 52.3% bf16 MFU | 1624251 tok/s
step 11446/19560 | loss 3.354194 (-0.79z)| norm 0.2563 (-1.39z)| lr 2.35e-04 | 322.79 ms | 52.3% bf16 MFU | 1624252 tok/s
step 11447/19560 | loss 3.377363 (-0.28z)| norm 0.2640 (-0.94z)| lr 2.35e-04 | 323.07 ms | 52.2% bf16 MFU | 1624180 tok/s
step 11448/19560 | loss 3.408795 (+0.41z)| norm 0.2861 (+0.36z)| lr 2.35e-04 | 322.50 ms | 52.3% bf16 MFU | 1624256 tok/s
step 11449/19560 | loss 3.364482 (-0.55z)| norm 0.2549 (-1.46z)| lr 2.35e-04 | 322.95 ms | 52.3% bf16 MFU | 1624215 tok/s
step 11450/19560 | loss 3.345817 (-0.95z)| norm 0.2612 (-1.08z)| lr 2.35e-04 | 322.95 ms | 52.3% bf16 MFU | 1624176 tok/s
step 11451/19560 | loss 3.406130 (+0.36z)| norm 0.2798 (+0.01z)| lr 2.35e-04 | 322.77 ms | 52.3% bf16 MFU | 1624184 tok/s
step 11452/19560 | loss 3.385176 (-0.10z)| norm 0.2667 (-0.76z)| lr 2.35e-04 | 322.97 ms | 52.3% bf16 MFU | 1624141 tok/s
step 11453/19560 | loss 3.408204 (+0.39z)| norm 0.2666 (-0.76z)| lr 2.34e-04 | 323.29 ms | 52.2% bf16 MFU | 1624019 tok/s
step 11454/19560 | loss 3.414871 (+0.53z)| norm 0.2656 (-0.82z)| lr 2.34e-04 | 323.56 ms | 52.2% bf16 MFU | 1623837 tok/s
step 11455/19560 | loss 3.347389 (-0.94z)| norm 0.2642 (-0.89z)| lr 2.34e-04 | 322.99 ms | 52.3% bf16 MFU | 1623807 tok/s
step 11456/19560 | loss 3.350380 (-0.86z)| norm 0.2678 (-0.69z)| lr 2.34e-04 | 322.79 ms | 52.3% bf16 MFU | 1623829 tok/s
step 11457/19560 | loss 3.395406 (+0.12z)| norm 0.2597 (-1.15z)| lr 2.34e-04 | 323.12 ms | 52.2% bf16 MFU | 1623767 tok/s
step 11458/19560 | loss 3.344121 (-0.99z)| norm 0.2601 (-1.11z)| lr 2.34e-04 | 323.01 ms | 52.2% bf16 MFU | 1623734 tok/s
step 11459/19560 | loss 3.397514 (+0.16z)| norm 0.2644 (-0.84z)| lr 2.34e-04 | 323.18 ms | 52.2% bf16 MFU | 1623661 tok/s
step 11460/19560 | loss 3.385059 (-0.11z)| norm 0.2781 (-0.04z)| lr 2.34e-04 | 323.00 ms | 52.3% bf16 MFU | 1623638 tok/s
step 11461/19560 | loss 3.351810 (-0.82z)| norm 0.2666 (-0.72z)| lr 2.34e-04 | 322.88 ms | 52.3% bf16 MFU | 1623645 tok/s
step 11462/19560 | loss 3.438334 (+1.05z)| norm 0.2783 (-0.03z)| lr 2.34e-04 | 322.60 ms | 52.3% bf16 MFU | 1623722 tok/s
step 11463/19560 | loss 3.366551 (-0.50z)| norm 0.2564 (-1.30z)| lr 2.34e-04 | 323.03 ms | 52.2% bf16 MFU | 1623687 tok/s
step 11464/19560 | loss 3.368012 (-0.47z)| norm 0.2719 (-0.39z)| lr 2.34e-04 | 322.50 ms | 52.3% bf16 MFU | 1623787 tok/s
step 11465/19560 | loss 3.369509 (-0.44z)| norm 0.2544 (-1.40z)| lr 2.34e-04 | 322.95 ms | 52.3% bf16 MFU | 1623769 tok/s
step 11466/19560 | loss 3.338507 (-1.11z)| norm 0.2663 (-0.70z)| lr 2.34e-04 | 322.76 ms | 52.3% bf16 MFU | 1623800 tok/s
step 11467/19560 | loss 3.369225 (-0.45z)| norm 0.2632 (-0.87z)| lr 2.34e-04 | 322.60 ms | 52.3% bf16 MFU | 1623870 tok/s
step 11468/19560 | loss 3.368531 (-0.45z)| norm 0.2782 (-0.01z)| lr 2.34e-04 | 322.83 ms | 52.3% bf16 MFU | 1623879 tok/s
step 11469/19560 | loss 3.323531 (-1.42z)| norm 0.2681 (-0.59z)| lr 2.34e-04 | 323.00 ms | 52.3% bf16 MFU | 1623844 tok/s
step 11470/19560 | loss 3.300549 (-1.88z)| norm 0.2649 (-0.77z)| lr 2.34e-04 | 322.59 ms | 52.3% bf16 MFU | 1623914 tok/s
step 11471/19560 | loss 3.358073 (-0.65z)| norm 0.2688 (-0.54z)| lr 2.34e-04 | 322.88 ms | 52.3% bf16 MFU | 1623907 tok/s
step 11472/19560 | loss 3.457983 (+1.53z)| norm 0.2752 (-0.16z)| lr 2.34e-04 | 322.51 ms | 52.3% bf16 MFU | 1623993 tok/s
step 11473/19560 | loss 3.399292 (+0.25z)| norm 0.2633 (-0.85z)| lr 2.33e-04 | 323.06 ms | 52.2% bf16 MFU | 1623937 tok/s
step 11474/19560 | loss 3.381338 (-0.12z)| norm 0.2662 (-0.67z)| lr 2.33e-04 | 322.98 ms | 52.3% bf16 MFU | 1623905 tok/s
step 11475/19560 | loss 3.391780 (+0.11z)| norm 0.2702 (-0.43z)| lr 2.33e-04 | 322.63 ms | 52.3% bf16 MFU | 1623963 tok/s
step 11476/19560 | loss 3.340000 (-1.05z)| norm 0.2722 (-0.32z)| lr 2.33e-04 | 322.86 ms | 52.3% bf16 MFU | 1623959 tok/s
step 11477/19560 | loss 3.345318 (-0.93z)| norm 0.2715 (-0.35z)| lr 2.33e-04 | 322.63 ms | 52.3% bf16 MFU | 1624012 tok/s
step 11478/19560 | loss 3.411126 (+0.55z)| norm 0.2683 (-0.53z)| lr 2.33e-04 | 322.78 ms | 52.3% bf16 MFU | 1624025 tok/s
step 11479/19560 | loss 3.386445 (-0.00z)| norm 0.2699 (-0.43z)| lr 2.33e-04 | 322.34 ms | 52.4% bf16 MFU | 1624148 tok/s
step 11480/19560 | loss 3.385356 (-0.03z)| norm 0.2637 (-0.79z)| lr 2.33e-04 | 322.85 ms | 52.3% bf16 MFU | 1624138 tok/s
step 11481/19560 | loss 3.342505 (-0.99z)| norm 0.2705 (-0.38z)| lr 2.33e-04 | 323.20 ms | 52.2% bf16 MFU | 1624038 tok/s
step 11482/19560 | loss 3.354010 (-0.71z)| norm 0.2502 (-1.56z)| lr 2.33e-04 | 322.44 ms | 52.3% bf16 MFU | 1624136 tok/s
step 11483/19560 | loss 3.395000 (+0.21z)| norm 0.2703 (-0.38z)| lr 2.33e-04 | 322.67 ms | 52.3% bf16 MFU | 1624170 tok/s
step 11484/19560 | loss 3.331232 (-1.23z)| norm 0.2624 (-0.83z)| lr 2.33e-04 | 322.82 ms | 52.3% bf16 MFU | 1624167 tok/s
step 11485/19560 | loss 3.366970 (-0.41z)| norm 0.2642 (-0.73z)| lr 2.33e-04 | 322.42 ms | 52.3% bf16 MFU | 1624264 tok/s
step 11486/19560 | loss 3.437535 (+1.20z)| norm 0.2817 (+0.28z)| lr 2.33e-04 | 322.74 ms | 52.3% bf16 MFU | 1624275 tok/s
step 11487/19560 | loss 3.435642 (+1.14z)| norm 0.2720 (-0.30z)| lr 2.33e-04 | 323.00 ms | 52.3% bf16 MFU | 1624220 tok/s
step 11488/19560 | loss 3.387986 (+0.05z)| norm 0.2912 (+0.84z)| lr 2.33e-04 | 323.11 ms | 52.2% bf16 MFU | 1624140 tok/s
step 11489/19560 | loss 3.341389 (-0.99z)| norm 0.2819 (+0.27z)| lr 2.33e-04 | 322.99 ms | 52.3% bf16 MFU | 1624096 tok/s
step 11490/19560 | loss 3.320331 (-1.45z)| norm 0.3923 (+5.92z)| lr 2.33e-04 | 322.42 ms | 52.3% bf16 MFU | 1624195 tok/s
step 11491/19560 | loss 3.424619 (+0.89z)| norm 0.3212 (+2.16z)| lr 2.33e-04 | 322.82 ms | 52.3% bf16 MFU | 1624190 tok/s
step 11492/19560 | loss 3.332024 (-1.19z)| norm 0.2876 (+0.43z)| lr 2.33e-04 | 322.98 ms | 52.3% bf16 MFU | 1624145 tok/s
step 11493/19560 | loss 3.373626 (-0.26z)| norm 0.3286 (+2.46z)| lr 2.33e-04 | 322.10 ms | 52.4% bf16 MFU | 1624324 tok/s
step 11494/19560 | loss 3.402189 (+0.39z)| norm 0.3098 (+1.51z)| lr 2.32e-04 | 322.84 ms | 52.3% bf16 MFU | 1624306 tok/s
step 11495/19560 | loss 3.673583 (+5.61z)| norm 0.3205 (+2.00z)| lr 2.32e-04 | 322.60 ms | 52.3% bf16 MFU | 1624350 tok/s
step 11496/19560 | loss 3.432878 (+0.88z)| norm 0.2984 (+0.91z)| lr 2.32e-04 | 322.86 ms | 52.3% bf16 MFU | 1624327 tok/s
step 11497/19560 | loss 3.359678 (-0.54z)| norm 0.2902 (+0.50z)| lr 2.32e-04 | 322.72 ms | 52.3% bf16 MFU | 1624341 tok/s
step 11498/19560 | loss 3.380915 (-0.12z)| norm 0.2857 (+0.28z)| lr 2.32e-04 | 322.78 ms | 52.3% bf16 MFU | 1624338 tok/s
step 11499/19560 | loss 3.348270 (-0.76z)| norm 0.2818 (+0.07z)| lr 2.32e-04 | 323.04 ms | 52.2% bf16 MFU | 1624270 tok/s
step 11500/19560 | loss 3.421759 (+0.76z)| norm 0.2827 (+0.13z)| lr 2.32e-04 | 323.21 ms | 52.2% bf16 MFU | 1624162 tok/s
val loss 3.365113
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2983/10042 = 0.297052
step 11501/19560 | loss 3.412868 (+0.56z)| norm 0.3055 (+1.25z)| lr 2.32e-04 | 322.59 ms | 52.3% bf16 MFU | 1624216 tok/s
step 11502/19560 | loss 3.433424 (+0.98z)| norm 0.2948 (+0.71z)| lr 2.32e-04 | 322.49 ms | 52.3% bf16 MFU | 1624293 tok/s
step 11503/19560 | loss 3.390292 (+0.07z)| norm 0.2979 (+0.86z)| lr 2.32e-04 | 323.17 ms | 52.2% bf16 MFU | 1624196 tok/s
step 11504/19560 | loss 3.388114 (+0.02z)| norm 0.2800 (-0.03z)| lr 2.32e-04 | 324.16 ms | 52.1% bf16 MFU | 1623854 tok/s
step 11505/19560 | loss 3.345405 (-0.88z)| norm 0.2693 (-0.56z)| lr 2.32e-04 | 323.23 ms | 52.2% bf16 MFU | 1623762 tok/s
step 11506/19560 | loss 3.336669 (-1.06z)| norm 0.2964 (+0.78z)| lr 2.32e-04 | 323.99 ms | 52.1% bf16 MFU | 1623484 tok/s
step 11507/19560 | loss 3.375862 (-0.22z)| norm 0.2766 (-0.19z)| lr 2.32e-04 | 322.97 ms | 52.3% bf16 MFU | 1623476 tok/s
step 11508/19560 | loss 3.376040 (-0.22z)| norm 0.2975 (+0.86z)| lr 2.32e-04 | 323.30 ms | 52.2% bf16 MFU | 1623386 tok/s
step 11509/19560 | loss 3.342101 (-0.92z)| norm 0.2716 (-0.43z)| lr 2.32e-04 | 323.17 ms | 52.2% bf16 MFU | 1623334 tok/s
step 11510/19560 | loss 3.386523 (+0.01z)| norm 0.2658 (-0.71z)| lr 2.32e-04 | 323.57 ms | 52.2% bf16 MFU | 1623184 tok/s
step 11511/19560 | loss 3.391728 (+0.13z)| norm 0.2835 (+0.18z)| lr 2.32e-04 | 323.25 ms | 52.2% bf16 MFU | 1623120 tok/s
step 11512/19560 | loss 3.494066 (+2.26z)| norm 0.2837 (+0.19z)| lr 2.32e-04 | 322.99 ms | 52.3% bf16 MFU | 1623125 tok/s
step 11513/19560 | loss 3.318833 (-1.38z)| norm 0.2826 (+0.12z)| lr 2.32e-04 | 322.79 ms | 52.3% bf16 MFU | 1623181 tok/s
step 11514/19560 | loss 3.395224 (+0.21z)| norm 0.2759 (-0.21z)| lr 2.31e-04 | 322.91 ms | 52.3% bf16 MFU | 1623204 tok/s
step 11515/19560 | loss 3.308434 (-1.57z)| norm 0.2750 (-0.26z)| lr 2.31e-04 | 323.51 ms | 52.2% bf16 MFU | 1623075 tok/s
step 11516/19560 | loss 3.426590 (+0.89z)| norm 0.2669 (-0.66z)| lr 2.31e-04 | 323.02 ms | 52.2% bf16 MFU | 1623074 tok/s
step 11517/19560 | loss 3.392845 (+0.19z)| norm 0.2779 (-0.11z)| lr 2.31e-04 | 323.17 ms | 52.2% bf16 MFU | 1623036 tok/s
step 11518/19560 | loss 3.340656 (-0.91z)| norm 0.2646 (-0.78z)| lr 2.31e-04 | 322.98 ms | 52.3% bf16 MFU | 1623048 tok/s
step 11519/19560 | loss 3.376030 (-0.16z)| norm 0.2624 (-0.88z)| lr 2.31e-04 | 323.33 ms | 52.2% bf16 MFU | 1622973 tok/s
step 11520/19560 | loss 3.407479 (+0.51z)| norm 0.2664 (-0.67z)| lr 2.31e-04 | 323.07 ms | 52.2% bf16 MFU | 1622965 tok/s
step 11521/19560 | loss 3.414968 (+0.66z)| norm 0.3099 (+1.50z)| lr 2.31e-04 | 322.77 ms | 52.3% bf16 MFU | 1623035 tok/s
step 11522/19560 | loss 3.366289 (-0.35z)| norm 0.2746 (-0.25z)| lr 2.31e-04 | 324.07 ms | 52.1% bf16 MFU | 1622774 tok/s
step 11523/19560 | loss 3.428325 (+0.94z)| norm 0.2823 (+0.14z)| lr 2.31e-04 | 323.45 ms | 52.2% bf16 MFU | 1622680 tok/s
step 11524/19560 | loss 3.425727 (+0.88z)| norm 0.4955 (+7.82z)| lr 2.31e-04 | 322.70 ms | 52.3% bf16 MFU | 1622780 tok/s
step 11525/19560 | loss 3.356403 (-0.58z)| norm 0.2836 (+0.09z)| lr 2.31e-04 | 323.39 ms | 52.2% bf16 MFU | 1622703 tok/s
step 11526/19560 | loss 3.484123 (+2.05z)| norm 0.2843 (+0.11z)| lr 2.31e-04 | 323.58 ms | 52.2% bf16 MFU | 1622582 tok/s
step 11527/19560 | loss 3.342587 (-0.87z)| norm 0.2641 (-0.63z)| lr 2.31e-04 | 323.16 ms | 52.2% bf16 MFU | 1622571 tok/s
step 11528/19560 | loss 3.421784 (+0.76z)| norm 0.2822 (+0.04z)| lr 2.31e-04 | 323.22 ms | 52.2% bf16 MFU | 1622547 tok/s
step 11529/19560 | loss 3.387116 (+0.04z)| norm 0.2794 (-0.06z)| lr 2.31e-04 | 323.30 ms | 52.2% bf16 MFU | 1622504 tok/s
step 11530/19560 | loss 3.314882 (-1.44z)| norm 0.2702 (-0.39z)| lr 2.31e-04 | 324.23 ms | 52.1% bf16 MFU | 1622231 tok/s
step 11531/19560 | loss 3.364389 (-0.42z)| norm 0.2814 (+0.02z)| lr 2.31e-04 | 322.93 ms | 52.3% bf16 MFU | 1622297 tok/s
step 11532/19560 | loss 3.408499 (+0.55z)| norm 0.2763 (-0.16z)| lr 2.31e-04 | 322.27 ms | 52.4% bf16 MFU | 1622526 tok/s
step 11533/19560 | loss 3.339916 (-0.94z)| norm 0.2880 (+0.27z)| lr 2.31e-04 | 323.76 ms | 52.1% bf16 MFU | 1622368 tok/s
step 11534/19560 | loss 3.379692 (-0.07z)| norm 0.2637 (-0.62z)| lr 2.31e-04 | 323.72 ms | 52.1% bf16 MFU | 1622228 tok/s
step 11535/19560 | loss 3.391135 (+0.17z)| norm 0.2562 (-0.88z)| lr 2.30e-04 | 322.90 ms | 52.3% bf16 MFU | 1622301 tok/s
step 11536/19560 | loss 3.298742 (-1.84z)| norm 0.2517 (-1.04z)| lr 2.30e-04 | 322.72 ms | 52.3% bf16 MFU | 1622416 tok/s
step 11537/19560 | loss 3.309729 (-1.59z)| norm 0.2610 (-0.69z)| lr 2.30e-04 | 323.50 ms | 52.2% bf16 MFU | 1622330 tok/s
step 11538/19560 | loss 3.448892 (+1.41z)| norm 0.2569 (-0.83z)| lr 2.30e-04 | 323.05 ms | 52.2% bf16 MFU | 1622360 tok/s
step 11539/19560 | loss 3.367922 (-0.33z)| norm 0.2564 (-0.84z)| lr 2.30e-04 | 323.55 ms | 52.2% bf16 MFU | 1622263 tok/s
step 11540/19560 | loss 3.408462 (+0.54z)| norm 0.2542 (-0.91z)| lr 2.30e-04 | 323.47 ms | 52.2% bf16 MFU | 1622191 tok/s
step 11541/19560 | loss 3.369484 (-0.30z)| norm 0.2751 (-0.16z)| lr 2.30e-04 | 323.16 ms | 52.2% bf16 MFU | 1622200 tok/s
step 11542/19560 | loss 3.381276 (-0.04z)| norm 0.2774 (-0.07z)| lr 2.30e-04 | 322.94 ms | 52.3% bf16 MFU | 1622264 tok/s
step 11543/19560 | loss 3.385683 (+0.06z)| norm 0.2786 (-0.02z)| lr 2.30e-04 | 323.59 ms | 52.2% bf16 MFU | 1622163 tok/s
step 11544/19560 | loss 3.414531 (+0.67z)| norm 0.2703 (-0.32z)| lr 2.30e-04 | 323.70 ms | 52.1% bf16 MFU | 1622038 tok/s
step 11545/19560 | loss 3.325227 (-1.24z)| norm 0.2762 (-0.09z)| lr 2.30e-04 | 323.30 ms | 52.2% bf16 MFU | 1622020 tok/s
step 11546/19560 | loss 3.357004 (-0.56z)| norm 0.2875 (+0.32z)| lr 2.30e-04 | 323.03 ms | 52.2% bf16 MFU | 1622071 tok/s
step 11547/19560 | loss 3.354640 (-0.60z)| norm 0.2683 (-0.38z)| lr 2.30e-04 | 322.48 ms | 52.3% bf16 MFU | 1622259 tok/s
step 11548/19560 | loss 3.430720 (+1.02z)| norm 0.2734 (-0.19z)| lr 2.30e-04 | 323.17 ms | 52.2% bf16 MFU | 1622262 tok/s
step 11549/19560 | loss 3.383335 (+0.00z)| norm 0.2892 (+0.39z)| lr 2.30e-04 | 322.80 ms | 52.3% bf16 MFU | 1622360 tok/s
step 11550/19560 | loss 3.364897 (-0.38z)| norm 0.2762 (-0.10z)| lr 2.30e-04 | 323.37 ms | 52.2% bf16 MFU | 1622309 tok/s
step 11551/19560 | loss 3.377399 (-0.12z)| norm 0.3023 (+0.85z)| lr 2.30e-04 | 322.97 ms | 52.3% bf16 MFU | 1622360 tok/s
step 11552/19560 | loss 3.361221 (-0.46z)| norm 0.2709 (-0.30z)| lr 2.30e-04 | 322.56 ms | 52.3% bf16 MFU | 1622511 tok/s
step 11553/19560 | loss 3.426252 (+0.93z)| norm 0.3249 (+1.74z)| lr 2.30e-04 | 323.44 ms | 52.2% bf16 MFU | 1622435 tok/s
step 11554/19560 | loss 3.458990 (+1.60z)| norm 0.2869 (+0.31z)| lr 2.30e-04 | 323.33 ms | 52.2% bf16 MFU | 1622389 tok/s
step 11555/19560 | loss 3.381195 (-0.05z)| norm 0.2937 (+0.56z)| lr 2.30e-04 | 323.12 ms | 52.2% bf16 MFU | 1622398 tok/s
step 11556/19560 | loss 3.370533 (-0.27z)| norm 0.2818 (+0.10z)| lr 2.29e-04 | 322.54 ms | 52.3% bf16 MFU | 1622553 tok/s
step 11557/19560 | loss 3.341264 (-0.88z)| norm 0.2930 (+0.52z)| lr 2.29e-04 | 323.71 ms | 52.1% bf16 MFU | 1622406 tok/s
step 11558/19560 | loss 3.323035 (-1.25z)| norm 0.2574 (-0.82z)| lr 2.29e-04 | 322.52 ms | 52.3% bf16 MFU | 1622565 tok/s
step 11559/19560 | loss 3.366991 (-0.32z)| norm 0.2928 (+0.51z)| lr 2.29e-04 | 322.99 ms | 52.3% bf16 MFU | 1622598 tok/s
step 11560/19560 | loss 3.418665 (+0.76z)| norm 0.2782 (-0.05z)| lr 2.29e-04 | 322.87 ms | 52.3% bf16 MFU | 1622660 tok/s
step 11561/19560 | loss 3.351030 (-0.65z)| norm 0.2911 (+0.44z)| lr 2.29e-04 | 323.10 ms | 52.2% bf16 MFU | 1622661 tok/s
step 11562/19560 | loss 3.365094 (-0.35z)| norm 0.2879 (+0.32z)| lr 2.29e-04 | 322.78 ms | 52.3% bf16 MFU | 1622744 tok/s
step 11563/19560 | loss 3.358886 (-0.48z)| norm 0.3348 (+2.04z)| lr 2.29e-04 | 323.33 ms | 52.2% bf16 MFU | 1622684 tok/s
step 11564/19560 | loss 3.405023 (+0.53z)| norm 0.2996 (+0.74z)| lr 2.29e-04 | 322.65 ms | 52.3% bf16 MFU | 1622796 tok/s
step 11565/19560 | loss 3.428378 (+1.02z)| norm 0.2907 (+0.41z)| lr 2.29e-04 | 322.88 ms | 52.3% bf16 MFU | 1622846 tok/s
step 11566/19560 | loss 3.402225 (+0.45z)| norm 0.2793 (-0.01z)| lr 2.29e-04 | 322.51 ms | 52.3% bf16 MFU | 1622985 tok/s
step 11567/19560 | loss 3.279952 (-2.14z)| norm 0.2877 (+0.30z)| lr 2.29e-04 | 323.58 ms | 52.2% bf16 MFU | 1622849 tok/s
step 11568/19560 | loss 3.358862 (-0.46z)| norm 0.2724 (-0.27z)| lr 2.29e-04 | 323.21 ms | 52.2% bf16 MFU | 1622813 tok/s
step 11569/19560 | loss 3.370562 (-0.21z)| norm 0.2749 (-0.18z)| lr 2.29e-04 | 322.77 ms | 52.3% bf16 MFU | 1622889 tok/s
step 11570/19560 | loss 3.376958 (-0.09z)| norm 0.2627 (-0.63z)| lr 2.29e-04 | 322.25 ms | 52.4% bf16 MFU | 1623093 tok/s
step 11571/19560 | loss 3.343477 (-0.81z)| norm 0.2669 (-0.48z)| lr 2.29e-04 | 322.75 ms | 52.3% bf16 MFU | 1623160 tok/s
step 11572/19560 | loss 3.368424 (-0.26z)| norm 0.3030 (+0.86z)| lr 2.29e-04 | 322.92 ms | 52.3% bf16 MFU | 1623181 tok/s
step 11573/19560 | loss 3.386029 (+0.12z)| norm 0.2622 (-0.66z)| lr 2.29e-04 | 322.90 ms | 52.3% bf16 MFU | 1623205 tok/s
step 11574/19560 | loss 3.399543 (+0.41z)| norm 0.2978 (+0.65z)| lr 2.29e-04 | 322.45 ms | 52.3% bf16 MFU | 1623342 tok/s
step 11575/19560 | loss 3.326036 (-1.17z)| norm 0.2856 (+0.19z)| lr 2.29e-04 | 323.20 ms | 52.2% bf16 MFU | 1623284 tok/s
step 11576/19560 | loss 3.367195 (-0.28z)| norm 0.2810 (+0.02z)| lr 2.28e-04 | 322.87 ms | 52.3% bf16 MFU | 1623312 tok/s
step 11577/19560 | loss 3.357614 (-0.48z)| norm 0.2605 (-0.74z)| lr 2.28e-04 | 322.36 ms | 52.4% bf16 MFU | 1623467 tok/s
step 11578/19560 | loss 3.388114 (+0.17z)| norm 0.2663 (-0.53z)| lr 2.28e-04 | 322.98 ms | 52.3% bf16 MFU | 1623458 tok/s
step 11579/19560 | loss 3.339710 (-0.87z)| norm 0.2727 (-0.29z)| lr 2.28e-04 | 322.74 ms | 52.3% bf16 MFU | 1623510 tok/s
step 11580/19560 | loss 3.402504 (+0.49z)| norm 0.2690 (-0.43z)| lr 2.28e-04 | 322.76 ms | 52.3% bf16 MFU | 1623555 tok/s
step 11581/19560 | loss 3.374039 (-0.12z)| norm 0.2733 (-0.27z)| lr 2.28e-04 | 322.55 ms | 52.3% bf16 MFU | 1623650 tok/s
step 11582/19560 | loss 3.371597 (-0.17z)| norm 0.2742 (-0.24z)| lr 2.28e-04 | 322.54 ms | 52.3% bf16 MFU | 1623743 tok/s
step 11583/19560 | loss 3.385871 (+0.14z)| norm 0.2909 (+0.38z)| lr 2.28e-04 | 323.57 ms | 52.2% bf16 MFU | 1623573 tok/s
step 11584/19560 | loss 3.312613 (-1.44z)| norm 0.2802 (-0.02z)| lr 2.28e-04 | 322.23 ms | 52.4% bf16 MFU | 1623747 tok/s
step 11585/19560 | loss 3.358049 (-0.45z)| norm 0.2859 (+0.18z)| lr 2.28e-04 | 322.30 ms | 52.4% bf16 MFU | 1623894 tok/s
step 11586/19560 | loss 3.360893 (-0.39z)| norm 0.2662 (-0.56z)| lr 2.28e-04 | 322.82 ms | 52.3% bf16 MFU | 1623904 tok/s
step 11587/19560 | loss 3.424874 (+0.98z)| norm 0.2961 (+0.55z)| lr 2.28e-04 | 322.42 ms | 52.3% bf16 MFU | 1624013 tok/s
step 11588/19560 | loss 3.458968 (+1.69z)| norm 0.3009 (+0.73z)| lr 2.28e-04 | 322.36 ms | 52.4% bf16 MFU | 1624132 tok/s
step 11589/19560 | loss 3.319964 (-1.27z)| norm 0.2734 (-0.31z)| lr 2.28e-04 | 322.49 ms | 52.3% bf16 MFU | 1624211 tok/s
step 11590/19560 | loss 3.330614 (-1.03z)| norm 0.2986 (+0.63z)| lr 2.28e-04 | 323.29 ms | 52.2% bf16 MFU | 1624087 tok/s
step 11591/19560 | loss 3.361605 (-0.37z)| norm 0.2930 (+0.41z)| lr 2.28e-04 | 322.75 ms | 52.3% bf16 MFU | 1624106 tok/s
step 11592/19560 | loss 3.395790 (+0.36z)| norm 0.2714 (-0.40z)| lr 2.28e-04 | 322.40 ms | 52.3% bf16 MFU | 1624210 tok/s
step 11593/19560 | loss 3.338048 (-0.86z)| norm 0.2892 (+0.26z)| lr 2.28e-04 | 323.25 ms | 52.2% bf16 MFU | 1624095 tok/s
step 11594/19560 | loss 3.364635 (-0.30z)| norm 0.2725 (-0.37z)| lr 2.28e-04 | 322.70 ms | 52.3% bf16 MFU | 1624126 tok/s
step 11595/19560 | loss 3.370606 (-0.18z)| norm 0.2809 (-0.06z)| lr 2.28e-04 | 322.59 ms | 52.3% bf16 MFU | 1624182 tok/s
step 11596/19560 | loss 3.436621 (+1.21z)| norm 0.2898 (+0.27z)| lr 2.28e-04 | 322.07 ms | 52.4% bf16 MFU | 1624366 tok/s
step 11597/19560 | loss 3.354019 (-0.55z)| norm 0.2869 (+0.16z)| lr 2.27e-04 | 323.01 ms | 52.2% bf16 MFU | 1624303 tok/s
step 11598/19560 | loss 3.387758 (+0.16z)| norm 0.2903 (+0.28z)| lr 2.27e-04 | 322.78 ms | 52.3% bf16 MFU | 1624301 tok/s
step 11599/19560 | loss 3.347367 (-0.71z)| norm 0.2761 (-0.26z)| lr 2.27e-04 | 322.61 ms | 52.3% bf16 MFU | 1624344 tok/s
step 11600/19560 | loss 3.376473 (-0.07z)| norm 0.2691 (-0.52z)| lr 2.27e-04 | 322.66 ms | 52.3% bf16 MFU | 1624371 tok/s
step 11601/19560 | loss 3.368370 (-0.24z)| norm 0.2726 (-0.39z)| lr 2.27e-04 | 323.42 ms | 52.2% bf16 MFU | 1624206 tok/s
step 11602/19560 | loss 3.419567 (+0.86z)| norm 0.2830 (-0.00z)| lr 2.27e-04 | 322.29 ms | 52.4% bf16 MFU | 1624334 tok/s
step 11603/19560 | loss 3.373982 (-0.12z)| norm 0.2954 (+0.46z)| lr 2.27e-04 | 322.33 ms | 52.4% bf16 MFU | 1624444 tok/s
step 11604/19560 | loss 3.332423 (-1.02z)| norm 0.2926 (+0.35z)| lr 2.27e-04 | 323.25 ms | 52.2% bf16 MFU | 1624319 tok/s
step 11605/19560 | loss 3.408136 (+0.61z)| norm 0.2774 (-0.23z)| lr 2.27e-04 | 323.12 ms | 52.2% bf16 MFU | 1624232 tok/s
step 11606/19560 | loss 3.370544 (-0.20z)| norm 0.2700 (-0.52z)| lr 2.27e-04 | 322.32 ms | 52.4% bf16 MFU | 1624351 tok/s
step 11607/19560 | loss 3.304738 (-1.59z)| norm 0.2722 (-0.43z)| lr 2.27e-04 | 323.22 ms | 52.2% bf16 MFU | 1624238 tok/s
step 11608/19560 | loss 3.298992 (-1.68z)| norm 0.2782 (-0.21z)| lr 2.27e-04 | 322.93 ms | 52.3% bf16 MFU | 1624202 tok/s
step 11609/19560 | loss 3.308874 (-1.46z)| norm 0.2704 (-0.50z)| lr 2.27e-04 | 322.65 ms | 52.3% bf16 MFU | 1624239 tok/s
step 11610/19560 | loss 3.364081 (-0.30z)| norm 0.2829 (-0.04z)| lr 2.27e-04 | 322.96 ms | 52.3% bf16 MFU | 1624198 tok/s
step 11611/19560 | loss 3.378966 (+0.02z)| norm 0.2696 (-0.55z)| lr 2.27e-04 | 322.48 ms | 52.3% bf16 MFU | 1624278 tok/s
step 11612/19560 | loss 3.369446 (-0.19z)| norm 0.2769 (-0.27z)| lr 2.27e-04 | 322.49 ms | 52.3% bf16 MFU | 1624353 tok/s
step 11613/19560 | loss 3.371689 (-0.14z)| norm 0.2984 (+0.54z)| lr 2.27e-04 | 322.55 ms | 52.3% bf16 MFU | 1624406 tok/s
step 11614/19560 | loss 3.390197 (+0.26z)| norm 0.2764 (-0.30z)| lr 2.27e-04 | 323.34 ms | 52.2% bf16 MFU | 1624259 tok/s
step 11615/19560 | loss 3.420133 (+0.90z)| norm 0.2843 (-0.00z)| lr 2.27e-04 | 322.58 ms | 52.3% bf16 MFU | 1624311 tok/s
step 11616/19560 | loss 3.417359 (+0.84z)| norm 0.2930 (+0.33z)| lr 2.27e-04 | 322.96 ms | 52.3% bf16 MFU | 1624264 tok/s
step 11617/19560 | loss 3.368086 (-0.22z)| norm 0.2786 (-0.22z)| lr 2.26e-04 | 322.71 ms | 52.3% bf16 MFU | 1624282 tok/s
step 11618/19560 | loss 3.408612 (+0.63z)| norm 0.2797 (-0.15z)| lr 2.26e-04 | 322.82 ms | 52.3% bf16 MFU | 1624272 tok/s
step 11619/19560 | loss 3.383200 (+0.10z)| norm 0.3213 (+1.57z)| lr 2.26e-04 | 322.88 ms | 52.3% bf16 MFU | 1624248 tok/s
step 11620/19560 | loss 3.357155 (-0.47z)| norm 0.2622 (-0.87z)| lr 2.26e-04 | 323.01 ms | 52.2% bf16 MFU | 1624192 tok/s
step 11621/19560 | loss 3.337745 (-0.88z)| norm 0.3086 (+1.06z)| lr 2.26e-04 | 322.51 ms | 52.3% bf16 MFU | 1624265 tok/s
step 11622/19560 | loss 3.382522 (+0.09z)| norm 0.2752 (-0.32z)| lr 2.26e-04 | 322.35 ms | 52.4% bf16 MFU | 1624375 tok/s
step 11623/19560 | loss 3.320500 (-1.43z)| norm 0.2700 (-0.53z)| lr 2.26e-04 | 322.74 ms | 52.3% bf16 MFU | 1624380 tok/s
step 11624/19560 | loss 3.356355 (-0.49z)| norm 0.2785 (-0.16z)| lr 2.26e-04 | 322.82 ms | 52.3% bf16 MFU | 1624366 tok/s
step 11625/19560 | loss 3.358222 (-0.44z)| norm 0.2665 (-0.66z)| lr 2.26e-04 | 322.55 ms | 52.3% bf16 MFU | 1624421 tok/s
step 11626/19560 | loss 3.413247 (+0.98z)| norm 0.2790 (-0.13z)| lr 2.26e-04 | 323.47 ms | 52.2% bf16 MFU | 1624240 tok/s
step 11627/19560 | loss 3.374641 (-0.02z)| norm 0.2637 (-0.77z)| lr 2.26e-04 | 322.54 ms | 52.3% bf16 MFU | 1624303 tok/s
step 11628/19560 | loss 3.412019 (+0.95z)| norm 0.2934 (+0.48z)| lr 2.26e-04 | 322.44 ms | 52.3% bf16 MFU | 1624388 tok/s
step 11629/19560 | loss 3.376526 (+0.03z)| norm 0.2747 (-0.30z)| lr 2.26e-04 | 322.75 ms | 52.3% bf16 MFU | 1624391 tok/s
step 11630/19560 | loss 3.352063 (-0.60z)| norm 0.2904 (+0.37z)| lr 2.26e-04 | 322.60 ms | 52.3% bf16 MFU | 1624431 tok/s
step 11631/19560 | loss 3.361922 (-0.33z)| norm 0.2601 (-0.90z)| lr 2.26e-04 | 322.87 ms | 52.3% bf16 MFU | 1624402 tok/s
step 11632/19560 | loss 3.374759 (+0.01z)| norm 0.2782 (-0.14z)| lr 2.26e-04 | 323.39 ms | 52.2% bf16 MFU | 1624244 tok/s
step 11633/19560 | loss 3.380482 (+0.16z)| norm 0.2671 (-0.60z)| lr 2.26e-04 | 322.80 ms | 52.3% bf16 MFU | 1624241 tok/s
step 11634/19560 | loss 3.364611 (-0.27z)| norm 0.2694 (-0.50z)| lr 2.26e-04 | 322.14 ms | 52.4% bf16 MFU | 1624404 tok/s
step 11635/19560 | loss 3.355416 (-0.51z)| norm 0.2628 (-0.77z)| lr 2.26e-04 | 323.24 ms | 52.2% bf16 MFU | 1624281 tok/s
step 11636/19560 | loss 3.329989 (-1.17z)| norm 0.2812 (+0.01z)| lr 2.26e-04 | 322.58 ms | 52.3% bf16 MFU | 1624333 tok/s
step 11637/19560 | loss 3.373452 (-0.03z)| norm 0.2580 (-0.96z)| lr 2.26e-04 | 323.77 ms | 52.1% bf16 MFU | 1624083 tok/s
step 11638/19560 | loss 3.360100 (-0.38z)| norm 0.2915 (+0.44z)| lr 2.25e-04 | 322.84 ms | 52.3% bf16 MFU | 1624079 tok/s
step 11639/19560 | loss 3.344408 (-0.78z)| norm 0.2473 (-1.40z)| lr 2.25e-04 | 322.98 ms | 52.3% bf16 MFU | 1624040 tok/s
step 11640/19560 | loss 3.351846 (-0.58z)| norm 0.2948 (+0.59z)| lr 2.25e-04 | 322.29 ms | 52.4% bf16 MFU | 1624176 tok/s
step 11641/19560 | loss 3.355870 (-0.48z)| norm 0.2726 (-0.34z)| lr 2.25e-04 | 322.63 ms | 52.3% bf16 MFU | 1624218 tok/s
step 11642/19560 | loss 3.338950 (-0.93z)| norm 0.2860 (+0.21z)| lr 2.25e-04 | 322.50 ms | 52.3% bf16 MFU | 1624293 tok/s
step 11643/19560 | loss 3.343666 (-0.82z)| norm 0.2641 (-0.70z)| lr 2.25e-04 | 323.72 ms | 52.1% bf16 MFU | 1624056 tok/s
step 11644/19560 | loss 3.348054 (-0.68z)| norm 0.2795 (-0.06z)| lr 2.25e-04 | 322.51 ms | 52.3% bf16 MFU | 1624136 tok/s
step 11645/19560 | loss 3.309404 (-1.74z)| norm 0.2803 (-0.02z)| lr 2.25e-04 | 322.45 ms | 52.3% bf16 MFU | 1624226 tok/s
step 11646/19560 | loss 3.428029 (+1.54z)| norm 0.2643 (-0.69z)| lr 2.25e-04 | 323.24 ms | 52.2% bf16 MFU | 1624114 tok/s
step 11647/19560 | loss 3.334484 (-1.04z)| norm 0.3042 (+0.96z)| lr 2.25e-04 | 323.06 ms | 52.2% bf16 MFU | 1624053 tok/s
step 11648/19560 | loss 3.361902 (-0.27z)| norm 0.2540 (-1.13z)| lr 2.25e-04 | 322.64 ms | 52.3% bf16 MFU | 1624099 tok/s
step 11649/19560 | loss 3.377205 (+0.16z)| norm 0.2823 (+0.06z)| lr 2.25e-04 | 322.38 ms | 52.4% bf16 MFU | 1624208 tok/s
step 11650/19560 | loss 3.342810 (-0.79z)| norm 0.2699 (-0.46z)| lr 2.25e-04 | 322.97 ms | 52.3% bf16 MFU | 1624164 tok/s
step 11651/19560 | loss 3.341643 (-0.81z)| norm 0.2681 (-0.53z)| lr 2.25e-04 | 323.22 ms | 52.2% bf16 MFU | 1624061 tok/s
step 11652/19560 | loss 3.363844 (-0.17z)| norm 0.2808 (+0.12z)| lr 2.25e-04 | 322.65 ms | 52.3% bf16 MFU | 1624104 tok/s
step 11653/19560 | loss 3.392519 (+0.63z)| norm 0.2723 (-0.46z)| lr 2.25e-04 | 322.20 ms | 52.4% bf16 MFU | 1624261 tok/s
step 11654/19560 | loss 3.323015 (-1.35z)| norm 0.2772 (-0.12z)| lr 2.25e-04 | 323.20 ms | 52.2% bf16 MFU | 1624158 tok/s
step 11655/19560 | loss 3.358885 (-0.30z)| norm 0.2892 (+0.70z)| lr 2.25e-04 | 323.08 ms | 52.2% bf16 MFU | 1624088 tok/s
step 11656/19560 | loss 3.386463 (+0.52z)| norm 0.2777 (-0.10z)| lr 2.25e-04 | 322.20 ms | 52.4% bf16 MFU | 1624244 tok/s
step 11657/19560 | loss 3.428622 (+1.73z)| norm 0.2933 (+0.97z)| lr 2.25e-04 | 322.81 ms | 52.3% bf16 MFU | 1624238 tok/s
step 11658/19560 | loss 3.342825 (-0.78z)| norm 0.2731 (-0.42z)| lr 2.25e-04 | 322.69 ms | 52.3% bf16 MFU | 1624263 tok/s
step 11659/19560 | loss 3.371045 (+0.05z)| norm 0.2885 (+0.63z)| lr 2.24e-04 | 322.97 ms | 52.3% bf16 MFU | 1624216 tok/s
step 11660/19560 | loss 3.425349 (+1.63z)| norm 0.2647 (-0.99z)| lr 2.24e-04 | 322.78 ms | 52.3% bf16 MFU | 1624219 tok/s
step 11661/19560 | loss 3.344609 (-0.73z)| norm 0.2778 (-0.09z)| lr 2.24e-04 | 322.80 ms | 52.3% bf16 MFU | 1624218 tok/s
step 11662/19560 | loss 3.374864 (+0.15z)| norm 0.2651 (-0.96z)| lr 2.24e-04 | 323.20 ms | 52.2% bf16 MFU | 1624115 tok/s
step 11663/19560 | loss 3.428171 (+1.69z)| norm 0.2634 (-1.09z)| lr 2.24e-04 | 322.13 ms | 52.4% bf16 MFU | 1624288 tok/s
step 11664/19560 | loss 3.399472 (+0.85z)| norm 0.2683 (-0.77z)| lr 2.24e-04 | 322.65 ms | 52.3% bf16 MFU | 1624320 tok/s
step 11665/19560 | loss 3.375658 (+0.13z)| norm 0.2476 (-2.19z)| lr 2.24e-04 | 322.99 ms | 52.3% bf16 MFU | 1624265 tok/s
step 11666/19560 | loss 3.367616 (-0.09z)| norm 0.2893 (+0.68z)| lr 2.24e-04 | 323.21 ms | 52.2% bf16 MFU | 1624158 tok/s
step 11667/19560 | loss 3.373301 (+0.08z)| norm 0.2739 (-0.40z)| lr 2.24e-04 | 322.43 ms | 52.3% bf16 MFU | 1624252 tok/s
step 11668/19560 | loss 3.342618 (-0.84z)| norm 0.2707 (-0.64z)| lr 2.24e-04 | 322.53 ms | 52.3% bf16 MFU | 1624317 tok/s
step 11669/19560 | loss 3.390920 (+0.63z)| norm 0.2770 (-0.19z)| lr 2.24e-04 | 322.85 ms | 52.3% bf16 MFU | 1624299 tok/s
step 11670/19560 | loss 3.364587 (-0.17z)| norm 0.2616 (-1.27z)| lr 2.24e-04 | 322.47 ms | 52.3% bf16 MFU | 1624376 tok/s
step 11671/19560 | loss 3.339281 (-0.93z)| norm 0.2880 (+0.59z)| lr 2.24e-04 | 323.35 ms | 52.2% bf16 MFU | 1624228 tok/s
step 11672/19560 | loss 3.323861 (-1.38z)| norm 0.2607 (-1.33z)| lr 2.24e-04 | 323.10 ms | 52.2% bf16 MFU | 1624150 tok/s
step 11673/19560 | loss 3.372296 (+0.09z)| norm 0.2990 (+1.34z)| lr 2.24e-04 | 323.04 ms | 52.2% bf16 MFU | 1624092 tok/s
step 11674/19560 | loss 3.343718 (-0.78z)| norm 0.2692 (-0.72z)| lr 2.24e-04 | 322.60 ms | 52.3% bf16 MFU | 1624146 tok/s
step 11675/19560 | loss 3.348763 (-0.63z)| norm 0.2751 (-0.32z)| lr 2.24e-04 | 322.83 ms | 52.3% bf16 MFU | 1624141 tok/s
step 11676/19560 | loss 3.377898 (+0.28z)| norm 0.2739 (-0.41z)| lr 2.24e-04 | 323.28 ms | 52.2% bf16 MFU | 1624024 tok/s
step 11677/19560 | loss 3.291862 (-2.32z)| norm 0.2726 (-0.49z)| lr 2.24e-04 | 322.62 ms | 52.3% bf16 MFU | 1624077 tok/s
step 11678/19560 | loss 3.363690 (-0.14z)| norm 0.2944 (+1.02z)| lr 2.24e-04 | 322.67 ms | 52.3% bf16 MFU | 1624116 tok/s
step 11679/19560 | loss 3.369271 (+0.04z)| norm 0.2730 (-0.45z)| lr 2.23e-04 | 322.75 ms | 52.3% bf16 MFU | 1624133 tok/s
step 11680/19560 | loss 3.378152 (+0.30z)| norm 0.3336 (+3.57z)| lr 2.23e-04 | 323.20 ms | 52.2% bf16 MFU | 1624036 tok/s
step 11681/19560 | loss 3.378802 (+0.34z)| norm 0.3171 (+2.51z)| lr 2.23e-04 | 322.91 ms | 52.3% bf16 MFU | 1624017 tok/s
step 11682/19560 | loss 3.358146 (-0.28z)| norm 0.2895 (+0.65z)| lr 2.23e-04 | 323.14 ms | 52.2% bf16 MFU | 1623941 tok/s
step 11683/19560 | loss 3.450800 (+2.57z)| norm 0.2973 (+1.17z)| lr 2.23e-04 | 322.78 ms | 52.3% bf16 MFU | 1623959 tok/s
step 11684/19560 | loss 3.396230 (+0.87z)| norm 0.2940 (+0.93z)| lr 2.23e-04 | 323.55 ms | 52.2% bf16 MFU | 1623781 tok/s
step 11685/19560 | loss 3.350852 (-0.53z)| norm 0.2812 (+0.08z)| lr 2.23e-04 | 323.14 ms | 52.2% bf16 MFU | 1623717 tok/s
step 11686/19560 | loss 3.389237 (+0.65z)| norm 0.2854 (+0.35z)| lr 2.23e-04 | 322.72 ms | 52.3% bf16 MFU | 1623759 tok/s
step 11687/19560 | loss 3.373013 (+0.14z)| norm 0.2902 (+0.68z)| lr 2.23e-04 | 323.04 ms | 52.2% bf16 MFU | 1623722 tok/s
step 11688/19560 | loss 3.362770 (-0.16z)| norm 0.2723 (-0.53z)| lr 2.23e-04 | 323.77 ms | 52.1% bf16 MFU | 1623502 tok/s
step 11689/19560 | loss 3.313875 (-1.67z)| norm 0.2743 (-0.38z)| lr 2.23e-04 | 322.74 ms | 52.3% bf16 MFU | 1623551 tok/s
step 11690/19560 | loss 3.351736 (-0.49z)| norm 0.2995 (+1.31z)| lr 2.23e-04 | 322.73 ms | 52.3% bf16 MFU | 1623601 tok/s
step 11691/19560 | loss 3.339646 (-0.86z)| norm 0.2793 (-0.02z)| lr 2.23e-04 | 322.73 ms | 52.3% bf16 MFU | 1623647 tok/s
step 11692/19560 | loss 3.323178 (-1.35z)| norm 0.3377 (+3.90z)| lr 2.23e-04 | 323.19 ms | 52.2% bf16 MFU | 1623576 tok/s
step 11693/19560 | loss 3.382940 (+0.51z)| norm 0.2937 (+0.93z)| lr 2.23e-04 | 322.74 ms | 52.3% bf16 MFU | 1623623 tok/s
step 11694/19560 | loss 3.407115 (+1.27z)| norm 0.2729 (-0.47z)| lr 2.23e-04 | 322.92 ms | 52.3% bf16 MFU | 1623621 tok/s
step 11695/19560 | loss 3.406719 (+1.25z)| norm 0.2874 (+0.51z)| lr 2.23e-04 | 323.00 ms | 52.3% bf16 MFU | 1623599 tok/s
step 11696/19560 | loss 3.361948 (-0.18z)| norm 0.2693 (-0.71z)| lr 2.23e-04 | 323.45 ms | 52.2% bf16 MFU | 1623465 tok/s
step 11697/19560 | loss 3.381848 (+0.45z)| norm 0.2716 (-0.55z)| lr 2.23e-04 | 322.63 ms | 52.3% bf16 MFU | 1623542 tok/s
step 11698/19560 | loss 3.323764 (-1.38z)| norm 0.2822 (+0.15z)| lr 2.23e-04 | 323.50 ms | 52.2% bf16 MFU | 1623398 tok/s
step 11699/19560 | loss 3.353710 (-0.43z)| norm 0.2830 (+0.20z)| lr 2.23e-04 | 322.88 ms | 52.3% bf16 MFU | 1623417 tok/s
step 11700/19560 | loss 3.350926 (-0.51z)| norm 0.2852 (+0.35z)| lr 2.22e-04 | 322.52 ms | 52.3% bf16 MFU | 1623527 tok/s
step 11701/19560 | loss 3.382207 (+0.48z)| norm 0.2782 (-0.13z)| lr 2.22e-04 | 323.13 ms | 52.2% bf16 MFU | 1623477 tok/s
step 11702/19560 | loss 3.376737 (+0.31z)| norm 0.2917 (+0.80z)| lr 2.22e-04 | 322.99 ms | 52.3% bf16 MFU | 1623464 tok/s
step 11703/19560 | loss 3.371452 (+0.13z)| norm 0.2731 (-0.47z)| lr 2.22e-04 | 322.85 ms | 52.3% bf16 MFU | 1623487 tok/s
step 11704/19560 | loss 3.343738 (-0.75z)| norm 0.2864 (+0.44z)| lr 2.22e-04 | 322.96 ms | 52.3% bf16 MFU | 1623482 tok/s
step 11705/19560 | loss 3.404856 (+1.19z)| norm 0.2742 (-0.41z)| lr 2.22e-04 | 323.04 ms | 52.2% bf16 MFU | 1623456 tok/s
step 11706/19560 | loss 3.394147 (+0.85z)| norm 0.3112 (+2.11z)| lr 2.22e-04 | 323.24 ms | 52.2% bf16 MFU | 1623381 tok/s
step 11707/19560 | loss 3.407759 (+1.26z)| norm 0.2803 (-0.01z)| lr 2.22e-04 | 323.29 ms | 52.2% bf16 MFU | 1623298 tok/s
step 11708/19560 | loss 3.362218 (-0.18z)| norm 0.2885 (+0.53z)| lr 2.22e-04 | 322.65 ms | 52.3% bf16 MFU | 1623380 tok/s
step 11709/19560 | loss 3.341731 (-0.82z)| norm 0.2906 (+0.67z)| lr 2.22e-04 | 322.83 ms | 52.3% bf16 MFU | 1623413 tok/s
step 11710/19560 | loss 3.361692 (-0.18z)| norm 0.2702 (-0.73z)| lr 2.22e-04 | 323.35 ms | 52.2% bf16 MFU | 1623315 tok/s
step 11711/19560 | loss 3.345947 (-0.67z)| norm 0.2701 (-0.72z)| lr 2.22e-04 | 323.14 ms | 52.2% bf16 MFU | 1623273 tok/s
step 11712/19560 | loss 3.366821 (-0.02z)| norm 0.2699 (-0.73z)| lr 2.22e-04 | 322.96 ms | 52.3% bf16 MFU | 1623279 tok/s
step 11713/19560 | loss 3.362463 (-0.16z)| norm 0.2890 (+0.57z)| lr 2.22e-04 | 323.08 ms | 52.2% bf16 MFU | 1623253 tok/s
step 11714/19560 | loss 3.359731 (-0.25z)| norm 0.2616 (-1.29z)| lr 2.22e-04 | 322.81 ms | 52.3% bf16 MFU | 1623297 tok/s
step 11715/19560 | loss 3.382281 (+0.49z)| norm 0.3301 (+3.23z)| lr 2.22e-04 | 323.19 ms | 52.2% bf16 MFU | 1623243 tok/s
step 11716/19560 | loss 3.404343 (+1.26z)| norm 0.2648 (-1.03z)| lr 2.22e-04 | 322.65 ms | 52.3% bf16 MFU | 1623329 tok/s
step 11717/19560 | loss 3.335706 (-1.05z)| norm 0.3288 (+3.03z)| lr 2.22e-04 | 322.78 ms | 52.3% bf16 MFU | 1623377 tok/s
step 11718/19560 | loss 3.441865 (+2.45z)| norm 0.2709 (-0.62z)| lr 2.22e-04 | 322.59 ms | 52.3% bf16 MFU | 1623470 tok/s
step 11719/19560 | loss 3.341033 (-0.88z)| norm 0.2875 (+0.44z)| lr 2.22e-04 | 323.58 ms | 52.2% bf16 MFU | 1623310 tok/s
step 11720/19560 | loss 3.406782 (+1.29z)| norm 0.2741 (-0.42z)| lr 2.22e-04 | 322.75 ms | 52.3% bf16 MFU | 1623365 tok/s
step 11721/19560 | loss 3.301222 (-2.15z)| norm 0.2907 (+0.64z)| lr 2.21e-04 | 322.65 ms | 52.3% bf16 MFU | 1623443 tok/s
step 11722/19560 | loss 3.329763 (-1.21z)| norm 0.2676 (-0.83z)| lr 2.21e-04 | 322.95 ms | 52.3% bf16 MFU | 1623443 tok/s
step 11723/19560 | loss 3.406124 (+1.24z)| norm 0.2937 (+0.82z)| lr 2.21e-04 | 322.88 ms | 52.3% bf16 MFU | 1623460 tok/s
step 11724/19560 | loss 3.366101 (-0.03z)| norm 0.2550 (-1.60z)| lr 2.21e-04 | 322.93 ms | 52.3% bf16 MFU | 1623464 tok/s
step 11725/19560 | loss 3.341667 (-0.82z)| norm 0.2595 (-1.30z)| lr 2.21e-04 | 322.86 ms | 52.3% bf16 MFU | 1623486 tok/s
step 11726/19560 | loss 3.375121 (+0.28z)| norm 0.2592 (-1.30z)| lr 2.21e-04 | 323.61 ms | 52.2% bf16 MFU | 1623318 tok/s
step 11727/19560 | loss 3.324353 (-1.37z)| norm 0.3229 (+2.57z)| lr 2.21e-04 | 322.91 ms | 52.3% bf16 MFU | 1623334 tok/s
step 11728/19560 | loss 3.300823 (-2.08z)| norm 0.2755 (-0.30z)| lr 2.21e-04 | 322.46 ms | 52.3% bf16 MFU | 1623461 tok/s
step 11729/19560 | loss 3.361119 (-0.15z)| norm 0.2658 (-0.88z)| lr 2.21e-04 | 322.94 ms | 52.3% bf16 MFU | 1623462 tok/s
step 11730/19560 | loss 3.336909 (-0.91z)| norm 0.2657 (-0.88z)| lr 2.21e-04 | 323.56 ms | 52.2% bf16 MFU | 1623306 tok/s
step 11731/19560 | loss 3.345448 (-0.63z)| norm 0.2813 (+0.07z)| lr 2.21e-04 | 322.94 ms | 52.3% bf16 MFU | 1623316 tok/s
step 11732/19560 | loss 3.395150 (+0.95z)| norm 0.2690 (-0.66z)| lr 2.21e-04 | 323.53 ms | 52.2% bf16 MFU | 1623176 tok/s
step 11733/19560 | loss 3.364545 (-0.02z)| norm 0.2745 (-0.33z)| lr 2.21e-04 | 323.19 ms | 52.2% bf16 MFU | 1623129 tok/s
step 11734/19560 | loss 3.358648 (-0.21z)| norm 0.2715 (-0.51z)| lr 2.21e-04 | 323.20 ms | 52.2% bf16 MFU | 1623082 tok/s
step 11735/19560 | loss 3.374291 (+0.29z)| norm 0.2694 (-0.64z)| lr 2.21e-04 | 323.23 ms | 52.2% bf16 MFU | 1623029 tok/s
step 11736/19560 | loss 3.377133 (+0.37z)| norm 0.2733 (-0.40z)| lr 2.21e-04 | 322.64 ms | 52.3% bf16 MFU | 1623128 tok/s
step 11737/19560 | loss 3.366659 (+0.00z)| norm 0.2713 (-0.52z)| lr 2.21e-04 | 323.75 ms | 52.1% bf16 MFU | 1622943 tok/s
step 11738/19560 | loss 3.434196 (+2.23z)| norm 0.2482 (-1.87z)| lr 2.21e-04 | 323.96 ms | 52.1% bf16 MFU | 1622714 tok/s
step 11739/19560 | loss 3.439586 (+2.35z)| norm 0.2790 (-0.04z)| lr 2.21e-04 | 323.18 ms | 52.2% bf16 MFU | 1622692 tok/s
step 11740/19560 | loss 3.420244 (+1.68z)| norm 0.2854 (+0.33z)| lr 2.21e-04 | 323.44 ms | 52.2% bf16 MFU | 1622605 tok/s
step 11741/19560 | loss 3.373590 (+0.18z)| norm 0.2648 (-0.88z)| lr 2.21e-04 | 322.63 ms | 52.3% bf16 MFU | 1622728 tok/s
step 11742/19560 | loss 3.350705 (-0.55z)| norm 0.2854 (+0.35z)| lr 2.20e-04 | 323.29 ms | 52.2% bf16 MFU | 1622678 tok/s
step 11743/19560 | loss 3.357352 (-0.32z)| norm 0.2743 (-0.32z)| lr 2.20e-04 | 322.83 ms | 52.3% bf16 MFU | 1622748 tok/s
step 11744/19560 | loss 3.365309 (-0.05z)| norm 0.2899 (+0.62z)| lr 2.20e-04 | 322.89 ms | 52.3% bf16 MFU | 1622796 tok/s
step 11745/19560 | loss 3.351702 (-0.50z)| norm 0.2871 (+0.45z)| lr 2.20e-04 | 322.31 ms | 52.4% bf16 MFU | 1622990 tok/s
step 11746/19560 | loss 3.364617 (-0.06z)| norm 0.2761 (-0.21z)| lr 2.20e-04 | 323.45 ms | 52.2% bf16 MFU | 1622888 tok/s
step 11747/19560 | loss 3.275718 (-2.89z)| norm 0.3127 (+2.01z)| lr 2.20e-04 | 322.68 ms | 52.3% bf16 MFU | 1622984 tok/s
step 11748/19560 | loss 3.456981 (+2.82z)| norm 0.2839 (+0.26z)| lr 2.20e-04 | 322.57 ms | 52.3% bf16 MFU | 1623102 tok/s
step 11749/19560 | loss 3.467746 (+3.02z)| norm 0.3088 (+1.77z)| lr 2.20e-04 | 323.06 ms | 52.2% bf16 MFU | 1623090 tok/s
step 11750/19560 | loss 3.374060 (+0.20z)| norm 0.2805 (+0.05z)| lr 2.20e-04 | 322.49 ms | 52.3% bf16 MFU | 1623224 tok/s
val loss 3.358420
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2957/10042 = 0.294463
step 11751/19560 | loss 3.367248 (-0.01z)| norm 0.2943 (+0.87z)| lr 2.20e-04 | 322.85 ms | 52.3% bf16 MFU | 1623261 tok/s
step 11752/19560 | loss 3.378587 (+0.33z)| norm 0.2918 (+0.71z)| lr 2.20e-04 | 323.83 ms | 52.1% bf16 MFU | 1623049 tok/s
step 11753/19560 | loss 3.354220 (-0.41z)| norm 0.2717 (-0.51z)| lr 2.20e-04 | 322.85 ms | 52.3% bf16 MFU | 1623093 tok/s
step 11754/19560 | loss 3.357875 (-0.29z)| norm 0.2968 (+1.00z)| lr 2.20e-04 | 322.98 ms | 52.3% bf16 MFU | 1623102 tok/s
step 11755/19560 | loss 3.322591 (-1.35z)| norm 0.2778 (-0.15z)| lr 2.20e-04 | 323.38 ms | 52.2% bf16 MFU | 1623009 tok/s
step 11756/19560 | loss 3.358232 (-0.25z)| norm 0.2811 (+0.06z)| lr 2.20e-04 | 322.86 ms | 52.3% bf16 MFU | 1623052 tok/s
step 11757/19560 | loss 3.315955 (-1.52z)| norm 0.2531 (-1.61z)| lr 2.20e-04 | 322.61 ms | 52.3% bf16 MFU | 1623158 tok/s
step 11758/19560 | loss 3.317384 (-1.46z)| norm 0.2844 (+0.27z)| lr 2.20e-04 | 322.76 ms | 52.3% bf16 MFU | 1623218 tok/s
step 11759/19560 | loss 3.350128 (-0.47z)| norm 0.2818 (+0.10z)| lr 2.20e-04 | 323.32 ms | 52.2% bf16 MFU | 1623135 tok/s
step 11760/19560 | loss 3.412834 (+1.40z)| norm 0.2844 (+0.25z)| lr 2.20e-04 | 322.59 ms | 52.3% bf16 MFU | 1623240 tok/s
step 11761/19560 | loss 3.377955 (+0.36z)| norm 0.2807 (+0.02z)| lr 2.20e-04 | 322.65 ms | 52.3% bf16 MFU | 1623324 tok/s
step 11762/19560 | loss 3.425631 (+1.74z)| norm 0.2991 (+1.12z)| lr 2.19e-04 | 322.84 ms | 52.3% bf16 MFU | 1623358 tok/s
step 11763/19560 | loss 3.368107 (+0.05z)| norm 0.2722 (-0.51z)| lr 2.19e-04 | 322.29 ms | 52.4% bf16 MFU | 1623529 tok/s
step 11764/19560 | loss 3.302938 (-1.85z)| norm 0.2783 (-0.14z)| lr 2.19e-04 | 322.84 ms | 52.3% bf16 MFU | 1623552 tok/s
step 11765/19560 | loss 3.350144 (-0.47z)| norm 0.2557 (-1.50z)| lr 2.19e-04 | 322.77 ms | 52.3% bf16 MFU | 1623591 tok/s
step 11766/19560 | loss 3.383065 (+0.49z)| norm 0.2863 (+0.35z)| lr 2.19e-04 | 322.72 ms | 52.3% bf16 MFU | 1623641 tok/s
step 11767/19560 | loss 3.394155 (+0.80z)| norm 0.2591 (-1.31z)| lr 2.19e-04 | 323.04 ms | 52.2% bf16 MFU | 1623609 tok/s
step 11768/19560 | loss 3.390310 (+0.68z)| norm 0.2701 (-0.63z)| lr 2.19e-04 | 322.56 ms | 52.3% bf16 MFU | 1623698 tok/s
step 11769/19560 | loss 3.364845 (-0.07z)| norm 0.2854 (+0.30z)| lr 2.19e-04 | 322.66 ms | 52.3% bf16 MFU | 1623758 tok/s
step 11770/19560 | loss 3.369418 (+0.06z)| norm 0.2679 (-0.76z)| lr 2.19e-04 | 322.75 ms | 52.3% bf16 MFU | 1623792 tok/s
step 11771/19560 | loss 3.350046 (-0.51z)| norm 0.2745 (-0.37z)| lr 2.19e-04 | 322.54 ms | 52.3% bf16 MFU | 1623876 tok/s
step 11772/19560 | loss 3.471065 (+2.90z)| norm 0.2698 (-0.65z)| lr 2.19e-04 | 322.92 ms | 52.3% bf16 MFU | 1623860 tok/s
step 11773/19560 | loss 3.347878 (-0.59z)| norm 0.3031 (+1.37z)| lr 2.19e-04 | 322.86 ms | 52.3% bf16 MFU | 1623862 tok/s
step 11774/19560 | loss 3.410412 (+1.20z)| norm 0.2773 (-0.20z)| lr 2.19e-04 | 322.32 ms | 52.4% bf16 MFU | 1623998 tok/s
step 11775/19560 | loss 3.328821 (-1.14z)| norm 0.2834 (+0.18z)| lr 2.19e-04 | 322.71 ms | 52.3% bf16 MFU | 1624032 tok/s
step 11776/19560 | loss 3.393966 (+0.72z)| norm 0.2827 (+0.13z)| lr 2.19e-04 | 322.82 ms | 52.3% bf16 MFU | 1624033 tok/s
step 11777/19560 | loss 3.358152 (-0.30z)| norm 0.2852 (+0.27z)| lr 2.19e-04 | 323.07 ms | 52.2% bf16 MFU | 1623973 tok/s
step 11778/19560 | loss 3.397625 (+0.82z)| norm 0.2779 (-0.18z)| lr 2.19e-04 | 322.32 ms | 52.4% bf16 MFU | 1624106 tok/s
step 11779/19560 | loss 3.368808 (-0.01z)| norm 0.2709 (-0.62z)| lr 2.19e-04 | 323.04 ms | 52.2% bf16 MFU | 1624049 tok/s
step 11780/19560 | loss 3.354644 (-0.42z)| norm 0.2785 (-0.14z)| lr 2.19e-04 | 322.49 ms | 52.3% bf16 MFU | 1624134 tok/s
step 11781/19560 | loss 3.328889 (-1.14z)| norm 0.2620 (-1.16z)| lr 2.19e-04 | 322.34 ms | 52.4% bf16 MFU | 1624252 tok/s
step 11782/19560 | loss 3.335946 (-0.94z)| norm 0.2717 (-0.55z)| lr 2.19e-04 | 322.76 ms | 52.3% bf16 MFU | 1624259 tok/s
step 11783/19560 | loss 3.427028 (+1.64z)| norm 0.2624 (-1.12z)| lr 2.18e-04 | 322.88 ms | 52.3% bf16 MFU | 1624234 tok/s
step 11784/19560 | loss 3.417953 (+1.36z)| norm 0.2835 (+0.18z)| lr 2.18e-04 | 322.62 ms | 52.3% bf16 MFU | 1624276 tok/s
step 11785/19560 | loss 3.389315 (+0.57z)| norm 0.2655 (-0.91z)| lr 2.18e-04 | 322.81 ms | 52.3% bf16 MFU | 1624269 tok/s
step 11786/19560 | loss 3.399157 (+0.84z)| norm 0.2618 (-1.13z)| lr 2.18e-04 | 322.56 ms | 52.3% bf16 MFU | 1624324 tok/s
step 11787/19560 | loss 3.362298 (-0.21z)| norm 0.2791 (-0.06z)| lr 2.18e-04 | 323.41 ms | 52.2% bf16 MFU | 1624164 tok/s
step 11788/19560 | loss 3.323040 (-1.30z)| norm 0.2696 (-0.65z)| lr 2.18e-04 | 322.24 ms | 52.4% bf16 MFU | 1624307 tok/s
step 11789/19560 | loss 3.315237 (-1.51z)| norm 0.2849 (+0.29z)| lr 2.18e-04 | 323.20 ms | 52.2% bf16 MFU | 1624199 tok/s
step 11790/19560 | loss 3.331322 (-1.04z)| norm 0.2610 (-1.18z)| lr 2.18e-04 | 322.56 ms | 52.3% bf16 MFU | 1624259 tok/s
step 11791/19560 | loss 3.341327 (-0.75z)| norm 0.2742 (-0.37z)| lr 2.18e-04 | 322.55 ms | 52.3% bf16 MFU | 1624318 tok/s
step 11792/19560 | loss 3.339777 (-0.78z)| norm 0.2546 (-1.56z)| lr 2.18e-04 | 322.56 ms | 52.3% bf16 MFU | 1624371 tok/s
step 11793/19560 | loss 3.422205 (+1.55z)| norm 0.2936 (+0.81z)| lr 2.18e-04 | 323.01 ms | 52.3% bf16 MFU | 1624310 tok/s
step 11794/19560 | loss 3.413553 (+1.28z)| norm 0.3134 (+2.00z)| lr 2.18e-04 | 322.82 ms | 52.3% bf16 MFU | 1624300 tok/s
step 11795/19560 | loss 3.286255 (-2.22z)| norm 0.2752 (-0.34z)| lr 2.18e-04 | 322.81 ms | 52.3% bf16 MFU | 1624291 tok/s
step 11796/19560 | loss 3.327517 (-1.08z)| norm 0.2753 (-0.34z)| lr 2.18e-04 | 323.09 ms | 52.2% bf16 MFU | 1624213 tok/s
step 11797/19560 | loss 3.357020 (-0.27z)| norm 0.3193 (+2.29z)| lr 2.18e-04 | 322.68 ms | 52.3% bf16 MFU | 1624241 tok/s
step 11798/19560 | loss 3.367982 (+0.03z)| norm 0.3044 (+1.37z)| lr 2.18e-04 | 322.77 ms | 52.3% bf16 MFU | 1624247 tok/s
step 11799/19560 | loss 3.369147 (+0.06z)| norm 0.3014 (+1.18z)| lr 2.18e-04 | 322.19 ms | 52.4% bf16 MFU | 1624397 tok/s
step 11800/19560 | loss 3.305568 (-1.68z)| norm 0.3235 (+2.43z)| lr 2.18e-04 | 322.94 ms | 52.3% bf16 MFU | 1624353 tok/s
step 11801/19560 | loss 3.371876 (+0.14z)| norm 0.2972 (+0.89z)| lr 2.18e-04 | 322.97 ms | 52.3% bf16 MFU | 1624302 tok/s
step 11802/19560 | loss 3.390900 (+0.65z)| norm 0.3020 (+1.15z)| lr 2.18e-04 | 322.52 ms | 52.3% bf16 MFU | 1624367 tok/s
step 11803/19560 | loss 3.295591 (-1.92z)| norm 0.3002 (+1.03z)| lr 2.18e-04 | 322.76 ms | 52.3% bf16 MFU | 1624368 tok/s
step 11804/19560 | loss 3.376541 (+0.26z)| norm 0.2722 (-0.59z)| lr 2.17e-04 | 322.38 ms | 52.4% bf16 MFU | 1624464 tok/s
step 11805/19560 | loss 3.413548 (+1.25z)| norm 0.3018 (+1.11z)| lr 2.17e-04 | 323.03 ms | 52.2% bf16 MFU | 1624391 tok/s
step 11806/19560 | loss 3.370406 (+0.07z)| norm 0.3029 (+1.17z)| lr 2.17e-04 | 323.09 ms | 52.2% bf16 MFU | 1624308 tok/s
step 11807/19560 | loss 3.373189 (+0.15z)| norm 0.2813 (-0.09z)| lr 2.17e-04 | 322.57 ms | 52.3% bf16 MFU | 1624359 tok/s
step 11808/19560 | loss 3.338197 (-0.80z)| norm 0.2875 (+0.30z)| lr 2.17e-04 | 322.29 ms | 52.4% bf16 MFU | 1624480 tok/s
step 11809/19560 | loss 3.330487 (-0.99z)| norm 0.2738 (-0.50z)| lr 2.17e-04 | 322.67 ms | 52.3% bf16 MFU | 1624498 tok/s
step 11810/19560 | loss 3.457848 (+2.38z)| norm 0.2741 (-0.48z)| lr 2.17e-04 | 322.55 ms | 52.3% bf16 MFU | 1624546 tok/s
step 11811/19560 | loss 3.439367 (+1.90z)| norm 0.2913 (+0.57z)| lr 2.17e-04 | 323.10 ms | 52.2% bf16 MFU | 1624452 tok/s
step 11812/19560 | loss 3.369267 (+0.04z)| norm 0.2732 (-0.52z)| lr 2.17e-04 | 323.00 ms | 52.3% bf16 MFU | 1624390 tok/s
step 11813/19560 | loss 3.334694 (-0.87z)| norm 0.2948 (+0.78z)| lr 2.17e-04 | 322.35 ms | 52.4% bf16 MFU | 1624492 tok/s
step 11814/19560 | loss 3.300142 (-1.76z)| norm 0.2944 (+0.76z)| lr 2.17e-04 | 323.06 ms | 52.2% bf16 MFU | 1624413 tok/s
step 11815/19560 | loss 3.348671 (-0.47z)| norm 0.2735 (-0.50z)| lr 2.17e-04 | 322.44 ms | 52.3% bf16 MFU | 1624493 tok/s
step 11816/19560 | loss 3.358333 (-0.22z)| norm 0.2961 (+0.86z)| lr 2.17e-04 | 322.62 ms | 52.3% bf16 MFU | 1624524 tok/s
step 11817/19560 | loss 3.396930 (+0.79z)| norm 0.2791 (-0.18z)| lr 2.17e-04 | 322.21 ms | 52.4% bf16 MFU | 1624655 tok/s
step 11818/19560 | loss 3.395519 (+0.74z)| norm 0.2996 (+1.07z)| lr 2.17e-04 | 322.41 ms | 52.3% bf16 MFU | 1624730 tok/s
step 11819/19560 | loss 3.403993 (+0.95z)| norm 0.2827 (+0.04z)| lr 2.17e-04 | 322.61 ms | 52.3% bf16 MFU | 1624750 tok/s
step 11820/19560 | loss 3.397955 (+0.78z)| norm 0.2871 (+0.34z)| lr 2.17e-04 | 322.58 ms | 52.3% bf16 MFU | 1624778 tok/s
step 11821/19560 | loss 3.341526 (-0.71z)| norm 0.2769 (-0.30z)| lr 2.17e-04 | 322.51 ms | 52.3% bf16 MFU | 1624822 tok/s
step 11822/19560 | loss 3.352844 (-0.40z)| norm 0.2753 (-0.40z)| lr 2.17e-04 | 323.40 ms | 52.2% bf16 MFU | 1624639 tok/s
step 11823/19560 | loss 3.447942 (+2.09z)| norm 0.3373 (+3.37z)| lr 2.17e-04 | 322.66 ms | 52.3% bf16 MFU | 1624651 tok/s
step 11824/19560 | loss 3.337808 (-0.79z)| norm 0.3011 (+1.15z)| lr 2.17e-04 | 323.05 ms | 52.2% bf16 MFU | 1624564 tok/s
step 11825/19560 | loss 3.518690 (+3.69z)| norm 0.3342 (+3.01z)| lr 2.16e-04 | 322.72 ms | 52.3% bf16 MFU | 1624566 tok/s
step 11826/19560 | loss 3.338880 (-0.75z)| norm 0.2808 (-0.11z)| lr 2.16e-04 | 322.84 ms | 52.3% bf16 MFU | 1624538 tok/s
step 11827/19560 | loss 3.348250 (-0.52z)| norm 0.3511 (+3.75z)| lr 2.16e-04 | 322.95 ms | 52.3% bf16 MFU | 1624482 tok/s
step 11828/19560 | loss 3.408663 (+0.96z)| norm 0.3002 (+0.93z)| lr 2.16e-04 | 322.42 ms | 52.3% bf16 MFU | 1624564 tok/s
step 11829/19560 | loss 3.391565 (+0.54z)| norm 0.3075 (+1.31z)| lr 2.16e-04 | 323.05 ms | 52.2% bf16 MFU | 1624482 tok/s
step 11830/19560 | loss 3.322619 (-1.14z)| norm 0.2939 (+0.56z)| lr 2.16e-04 | 322.22 ms | 52.4% bf16 MFU | 1624612 tok/s
step 11831/19560 | loss 3.339476 (-0.72z)| norm 0.3083 (+1.33z)| lr 2.16e-04 | 322.49 ms | 52.3% bf16 MFU | 1624669 tok/s
step 11832/19560 | loss 3.439024 (+1.68z)| norm 0.2766 (-0.39z)| lr 2.16e-04 | 322.54 ms | 52.3% bf16 MFU | 1624711 tok/s
step 11833/19560 | loss 3.425245 (+1.33z)| norm 0.2945 (+0.58z)| lr 2.16e-04 | 322.84 ms | 52.3% bf16 MFU | 1624675 tok/s
step 11834/19560 | loss 3.346456 (-0.56z)| norm 0.2911 (+0.40z)| lr 2.16e-04 | 322.75 ms | 52.3% bf16 MFU | 1624663 tok/s
step 11835/19560 | loss 3.343071 (-0.63z)| norm 0.2925 (+0.47z)| lr 2.16e-04 | 322.88 ms | 52.3% bf16 MFU | 1624619 tok/s
step 11836/19560 | loss 3.391098 (+0.53z)| norm 0.2976 (+0.74z)| lr 2.16e-04 | 323.58 ms | 52.2% bf16 MFU | 1624400 tok/s
step 11837/19560 | loss 3.323399 (-1.10z)| norm 0.2907 (+0.37z)| lr 2.16e-04 | 322.50 ms | 52.3% bf16 MFU | 1624464 tok/s
step 11838/19560 | loss 3.438125 (+1.63z)| norm 0.2799 (-0.23z)| lr 2.16e-04 | 322.93 ms | 52.3% bf16 MFU | 1624418 tok/s
step 11839/19560 | loss 3.342568 (-0.65z)| norm 0.2708 (-0.73z)| lr 2.16e-04 | 323.09 ms | 52.2% bf16 MFU | 1624332 tok/s
step 11840/19560 | loss 3.392647 (+0.54z)| norm 0.2668 (-0.94z)| lr 2.16e-04 | 322.63 ms | 52.3% bf16 MFU | 1624368 tok/s
step 11841/19560 | loss 3.360598 (-0.22z)| norm 0.2685 (-0.83z)| lr 2.16e-04 | 322.91 ms | 52.3% bf16 MFU | 1624332 tok/s
step 11842/19560 | loss 3.337232 (-0.77z)| norm 0.2764 (-0.42z)| lr 2.16e-04 | 322.52 ms | 52.3% bf16 MFU | 1624395 tok/s
step 11843/19560 | loss 3.327680 (-0.98z)| norm 0.2496 (-1.87z)| lr 2.16e-04 | 322.48 ms | 52.3% bf16 MFU | 1624465 tok/s
step 11844/19560 | loss 3.381721 (+0.30z)| norm 0.2672 (-0.90z)| lr 2.16e-04 | 322.60 ms | 52.3% bf16 MFU | 1624501 tok/s
step 11845/19560 | loss 3.340085 (-0.69z)| norm 0.2684 (-0.82z)| lr 2.16e-04 | 323.30 ms | 52.2% bf16 MFU | 1624360 tok/s
step 11846/19560 | loss 3.378902 (+0.25z)| norm 0.2802 (-0.16z)| lr 2.15e-04 | 323.01 ms | 52.2% bf16 MFU | 1624298 tok/s
step 11847/19560 | loss 3.372317 (+0.08z)| norm 0.2529 (-1.67z)| lr 2.15e-04 | 322.45 ms | 52.3% bf16 MFU | 1624380 tok/s
step 11848/19560 | loss 3.341013 (-0.66z)| norm 0.2820 (-0.04z)| lr 2.15e-04 | 322.42 ms | 52.3% bf16 MFU | 1624466 tok/s
step 11849/19560 | loss 3.393254 (+0.59z)| norm 0.2545 (-1.56z)| lr 2.15e-04 | 322.61 ms | 52.3% bf16 MFU | 1624499 tok/s
step 11850/19560 | loss 3.397972 (+0.69z)| norm 0.2542 (-1.56z)| lr 2.15e-04 | 323.35 ms | 52.2% bf16 MFU | 1624344 tok/s
step 11851/19560 | loss 3.369483 (+0.00z)| norm 0.2530 (-1.60z)| lr 2.15e-04 | 322.91 ms | 52.3% bf16 MFU | 1624308 tok/s
step 11852/19560 | loss 3.368240 (-0.03z)| norm 0.2675 (-0.81z)| lr 2.15e-04 | 323.17 ms | 52.2% bf16 MFU | 1624210 tok/s
step 11853/19560 | loss 3.370648 (+0.03z)| norm 0.2612 (-1.16z)| lr 2.15e-04 | 321.94 ms | 52.4% bf16 MFU | 1624424 tok/s
step 11854/19560 | loss 3.387015 (+0.42z)| norm 0.2719 (-0.57z)| lr 2.15e-04 | 323.19 ms | 52.2% bf16 MFU | 1624315 tok/s
step 11855/19560 | loss 3.409971 (+0.97z)| norm 0.2655 (-0.92z)| lr 2.15e-04 | 324.12 ms | 52.1% bf16 MFU | 1623979 tok/s
step 11856/19560 | loss 3.390919 (+0.49z)| norm 0.2533 (-1.59z)| lr 2.15e-04 | 322.82 ms | 52.3% bf16 MFU | 1623984 tok/s
step 11857/19560 | loss 3.361041 (-0.25z)| norm 0.2685 (-0.74z)| lr 2.15e-04 | 322.52 ms | 52.3% bf16 MFU | 1624063 tok/s
step 11858/19560 | loss 3.316904 (-1.33z)| norm 0.2453 (-2.00z)| lr 2.15e-04 | 322.38 ms | 52.4% bf16 MFU | 1624176 tok/s
step 11859/19560 | loss 3.377518 (+0.16z)| norm 0.2736 (-0.43z)| lr 2.15e-04 | 322.93 ms | 52.3% bf16 MFU | 1624144 tok/s
step 11860/19560 | loss 3.392987 (+0.54z)| norm 0.2725 (-0.50z)| lr 2.15e-04 | 323.89 ms | 52.1% bf16 MFU | 1623873 tok/s
step 11861/19560 | loss 3.386509 (+0.37z)| norm 0.2833 (+0.10z)| lr 2.15e-04 | 322.83 ms | 52.3% bf16 MFU | 1623881 tok/s
step 11862/19560 | loss 3.302817 (-1.66z)| norm 0.2446 (-2.00z)| lr 2.15e-04 | 322.60 ms | 52.3% bf16 MFU | 1623946 tok/s
step 11863/19560 | loss 3.495317 (+2.91z)| norm 0.3174 (+1.93z)| lr 2.15e-04 | 322.64 ms | 52.3% bf16 MFU | 1623998 tok/s
step 11864/19560 | loss 3.368888 (-0.07z)| norm 0.2855 (+0.20z)| lr 2.15e-04 | 322.89 ms | 52.3% bf16 MFU | 1623984 tok/s
step 11865/19560 | loss 3.323391 (-1.13z)| norm 0.3217 (+2.10z)| lr 2.15e-04 | 323.55 ms | 52.2% bf16 MFU | 1623808 tok/s
step 11866/19560 | loss 3.422311 (+1.20z)| norm 0.2821 (-0.02z)| lr 2.14e-04 | 322.80 ms | 52.3% bf16 MFU | 1623826 tok/s
step 11867/19560 | loss 3.397010 (+0.62z)| norm 0.3013 (+1.00z)| lr 2.14e-04 | 322.74 ms | 52.3% bf16 MFU | 1623860 tok/s
step 11868/19560 | loss 3.376730 (+0.15z)| norm 0.3178 (+1.85z)| lr 2.14e-04 | 323.33 ms | 52.2% bf16 MFU | 1623743 tok/s
step 11869/19560 | loss 3.316895 (-1.26z)| norm 0.2828 (-0.01z)| lr 2.14e-04 | 323.20 ms | 52.2% bf16 MFU | 1623665 tok/s
step 11870/19560 | loss 3.362658 (-0.18z)| norm 0.3062 (+1.21z)| lr 2.14e-04 | 322.57 ms | 52.3% bf16 MFU | 1623749 tok/s
step 11871/19560 | loss 3.363848 (-0.15z)| norm 0.2708 (-0.65z)| lr 2.14e-04 | 323.04 ms | 52.2% bf16 MFU | 1623711 tok/s
step 11872/19560 | loss 3.354651 (-0.37z)| norm 0.2885 (+0.28z)| lr 2.14e-04 | 323.35 ms | 52.2% bf16 MFU | 1623597 tok/s
step 11873/19560 | loss 3.367292 (-0.07z)| norm 0.2814 (-0.09z)| lr 2.14e-04 | 322.77 ms | 52.3% bf16 MFU | 1623634 tok/s
step 11874/19560 | loss 3.357759 (-0.30z)| norm 0.2930 (+0.51z)| lr 2.14e-04 | 322.50 ms | 52.3% bf16 MFU | 1623736 tok/s
step 11875/19560 | loss 3.359250 (-0.28z)| norm 0.2695 (-0.71z)| lr 2.14e-04 | 323.35 ms | 52.2% bf16 MFU | 1623622 tok/s
step 11876/19560 | loss 3.349331 (-0.51z)| norm 0.2901 (+0.38z)| lr 2.14e-04 | 322.91 ms | 52.3% bf16 MFU | 1623623 tok/s
step 11877/19560 | loss 3.364008 (-0.13z)| norm 0.2695 (-0.70z)| lr 2.14e-04 | 322.83 ms | 52.3% bf16 MFU | 1623643 tok/s
step 11878/19560 | loss 3.398333 (+0.72z)| norm 0.2797 (-0.15z)| lr 2.14e-04 | 322.82 ms | 52.3% bf16 MFU | 1623664 tok/s
step 11879/19560 | loss 3.354474 (-0.38z)| norm 0.2878 (+0.28z)| lr 2.14e-04 | 323.10 ms | 52.2% bf16 MFU | 1623616 tok/s
step 11880/19560 | loss 3.368752 (-0.02z)| norm 0.2679 (-0.77z)| lr 2.14e-04 | 323.61 ms | 52.2% bf16 MFU | 1623442 tok/s
step 11881/19560 | loss 3.367196 (-0.06z)| norm 0.2744 (-0.43z)| lr 2.14e-04 | 322.65 ms | 52.3% bf16 MFU | 1623518 tok/s
step 11882/19560 | loss 3.352559 (-0.42z)| norm 0.2838 (+0.08z)| lr 2.14e-04 | 322.90 ms | 52.3% bf16 MFU | 1623526 tok/s
step 11883/19560 | loss 3.373750 (+0.10z)| norm 0.2616 (-1.10z)| lr 2.14e-04 | 323.02 ms | 52.2% bf16 MFU | 1623503 tok/s
step 11884/19560 | loss 3.346561 (-0.58z)| norm 0.2659 (-0.86z)| lr 2.14e-04 | 323.15 ms | 52.2% bf16 MFU | 1623449 tok/s
step 11885/19560 | loss 3.370491 (+0.01z)| norm 0.2765 (-0.31z)| lr 2.14e-04 | 322.44 ms | 52.3% bf16 MFU | 1623575 tok/s
step 11886/19560 | loss 3.374539 (+0.10z)| norm 0.2838 (+0.09z)| lr 2.14e-04 | 323.03 ms | 52.2% bf16 MFU | 1623548 tok/s
step 11887/19560 | loss 3.313077 (-1.46z)| norm 0.2758 (-0.34z)| lr 2.13e-04 | 323.17 ms | 52.2% bf16 MFU | 1623488 tok/s
step 11888/19560 | loss 3.357137 (-0.33z)| norm 0.2899 (+0.41z)| lr 2.13e-04 | 323.01 ms | 52.2% bf16 MFU | 1623470 tok/s
step 11889/19560 | loss 3.396751 (+0.68z)| norm 0.2684 (-0.73z)| lr 2.13e-04 | 323.21 ms | 52.2% bf16 MFU | 1623403 tok/s
step 11890/19560 | loss 3.316029 (-1.36z)| norm 0.2951 (+0.70z)| lr 2.13e-04 | 322.89 ms | 52.3% bf16 MFU | 1623420 tok/s
step 11891/19560 | loss 3.368402 (-0.02z)| norm 0.2972 (+0.80z)| lr 2.13e-04 | 323.40 ms | 52.2% bf16 MFU | 1623307 tok/s
step 11892/19560 | loss 3.350457 (-0.49z)| norm 0.2776 (-0.25z)| lr 2.13e-04 | 323.03 ms | 52.2% bf16 MFU | 1623294 tok/s
step 11893/19560 | loss 3.357803 (-0.30z)| norm 0.2881 (+0.30z)| lr 2.13e-04 | 322.95 ms | 52.3% bf16 MFU | 1623300 tok/s
step 11894/19560 | loss 3.397547 (+0.72z)| norm 0.2856 (+0.16z)| lr 2.13e-04 | 323.50 ms | 52.2% bf16 MFU | 1623169 tok/s
step 11895/19560 | loss 3.345141 (-0.62z)| norm 0.2974 (+0.79z)| lr 2.13e-04 | 322.93 ms | 52.3% bf16 MFU | 1623188 tok/s
step 11896/19560 | loss 3.318885 (-1.28z)| norm 0.2808 (-0.12z)| lr 2.13e-04 | 322.94 ms | 52.3% bf16 MFU | 1623202 tok/s
step 11897/19560 | loss 3.348319 (-0.52z)| norm 0.2848 (+0.10z)| lr 2.13e-04 | 322.89 ms | 52.3% bf16 MFU | 1623228 tok/s
step 11898/19560 | loss 3.416137 (+1.20z)| norm 0.2780 (-0.27z)| lr 2.13e-04 | 323.48 ms | 52.2% bf16 MFU | 1623105 tok/s
step 11899/19560 | loss 3.398105 (+0.73z)| norm 0.2999 (+0.90z)| lr 2.13e-04 | 323.13 ms | 52.2% bf16 MFU | 1623077 tok/s
step 11900/19560 | loss 3.341838 (-0.69z)| norm 0.3018 (+0.99z)| lr 2.13e-04 | 322.66 ms | 52.3% bf16 MFU | 1623166 tok/s
step 11901/19560 | loss 3.316873 (-1.33z)| norm 0.3033 (+1.07z)| lr 2.13e-04 | 322.67 ms | 52.3% bf16 MFU | 1623250 tok/s
step 11902/19560 | loss 3.375550 (+0.20z)| norm 0.2796 (-0.21z)| lr 2.13e-04 | 323.54 ms | 52.2% bf16 MFU | 1623112 tok/s
step 11903/19560 | loss 3.350419 (-0.46z)| norm 0.3457 (+3.20z)| lr 2.13e-04 | 323.49 ms | 52.2% bf16 MFU | 1622994 tok/s
step 11904/19560 | loss 3.339124 (-0.74z)| norm 0.3019 (+0.92z)| lr 2.13e-04 | 322.51 ms | 52.3% bf16 MFU | 1623126 tok/s
step 11905/19560 | loss 3.351222 (-0.43z)| norm 0.3038 (+1.01z)| lr 2.13e-04 | 323.90 ms | 52.1% bf16 MFU | 1622904 tok/s
step 11906/19560 | loss 3.378366 (+0.29z)| norm 0.2956 (+0.58z)| lr 2.13e-04 | 322.92 ms | 52.3% bf16 MFU | 1622936 tok/s
step 11907/19560 | loss 3.386327 (+0.49z)| norm 0.2755 (-0.46z)| lr 2.13e-04 | 322.34 ms | 52.4% bf16 MFU | 1623116 tok/s
step 11908/19560 | loss 3.336542 (-0.80z)| norm 0.2855 (+0.05z)| lr 2.12e-04 | 323.31 ms | 52.2% bf16 MFU | 1623040 tok/s
step 11909/19560 | loss 3.366901 (-0.02z)| norm 0.2738 (-0.55z)| lr 2.12e-04 | 323.40 ms | 52.2% bf16 MFU | 1622948 tok/s
step 11910/19560 | loss 3.448219 (+2.06z)| norm 0.2952 (+0.54z)| lr 2.12e-04 | 324.02 ms | 52.1% bf16 MFU | 1622705 tok/s
step 11911/19560 | loss 3.393095 (+0.64z)| norm 0.2631 (-1.12z)| lr 2.12e-04 | 322.81 ms | 52.3% bf16 MFU | 1622776 tok/s
step 11912/19560 | loss 3.444681 (+1.96z)| norm 0.3334 (+2.44z)| lr 2.12e-04 | 323.24 ms | 52.2% bf16 MFU | 1622737 tok/s
step 11913/19560 | loss 3.353172 (-0.39z)| norm 0.2649 (-1.02z)| lr 2.12e-04 | 322.94 ms | 52.3% bf16 MFU | 1622774 tok/s
step 11914/19560 | loss 3.335444 (-0.83z)| norm 0.2916 (+0.32z)| lr 2.12e-04 | 323.66 ms | 52.1% bf16 MFU | 1622630 tok/s
step 11915/19560 | loss 3.326068 (-1.06z)| norm 0.2750 (-0.53z)| lr 2.12e-04 | 323.11 ms | 52.2% bf16 MFU | 1622631 tok/s
step 11916/19560 | loss 3.332842 (-0.89z)| norm 0.2908 (+0.27z)| lr 2.12e-04 | 323.28 ms | 52.2% bf16 MFU | 1622587 tok/s
step 11917/19560 | loss 3.440438 (+1.84z)| norm 0.3335 (+2.36z)| lr 2.12e-04 | 323.55 ms | 52.2% bf16 MFU | 1622480 tok/s
step 11918/19560 | loss 3.327613 (-1.04z)| norm 0.3529 (+3.18z)| lr 2.12e-04 | 323.55 ms | 52.2% bf16 MFU | 1622377 tok/s
step 11919/19560 | loss 3.410243 (+1.05z)| norm 0.2825 (-0.20z)| lr 2.12e-04 | 322.45 ms | 52.3% bf16 MFU | 1622556 tok/s
step 11920/19560 | loss 3.385355 (+0.41z)| norm 0.3413 (+2.55z)| lr 2.12e-04 | 323.77 ms | 52.1% bf16 MFU | 1622395 tok/s
step 11921/19560 | loss 3.316012 (-1.34z)| norm 0.2797 (-0.35z)| lr 2.12e-04 | 323.63 ms | 52.1% bf16 MFU | 1622277 tok/s
step 11922/19560 | loss 3.277735 (-2.25z)| norm 0.3196 (+1.52z)| lr 2.12e-04 | 323.23 ms | 52.2% bf16 MFU | 1622264 tok/s
step 11923/19560 | loss 3.335763 (-0.82z)| norm 0.2936 (+0.29z)| lr 2.12e-04 | 323.83 ms | 52.1% bf16 MFU | 1622102 tok/s
step 11924/19560 | loss 3.334882 (-0.84z)| norm 0.3028 (+0.71z)| lr 2.12e-04 | 323.07 ms | 52.2% bf16 MFU | 1622138 tok/s
step 11925/19560 | loss 3.385900 (+0.45z)| norm 0.2729 (-0.68z)| lr 2.12e-04 | 322.81 ms | 52.3% bf16 MFU | 1622238 tok/s
step 11926/19560 | loss 3.339539 (-0.72z)| norm 0.3178 (+1.43z)| lr 2.12e-04 | 323.23 ms | 52.2% bf16 MFU | 1622226 tok/s
step 11927/19560 | loss 3.412942 (+1.13z)| norm 0.2751 (-0.57z)| lr 2.12e-04 | 323.28 ms | 52.2% bf16 MFU | 1622205 tok/s
step 11928/19560 | loss 3.407321 (+0.98z)| norm 0.2768 (-0.48z)| lr 2.12e-04 | 322.86 ms | 52.3% bf16 MFU | 1622290 tok/s
step 11929/19560 | loss 3.344667 (-0.61z)| norm 0.2646 (-1.04z)| lr 2.11e-04 | 322.83 ms | 52.3% bf16 MFU | 1622376 tok/s
step 11930/19560 | loss 3.352283 (-0.41z)| norm 0.2715 (-0.70z)| lr 2.11e-04 | 323.52 ms | 52.2% bf16 MFU | 1622286 tok/s
step 11931/19560 | loss 3.350886 (-0.47z)| norm 0.2730 (-0.62z)| lr 2.11e-04 | 322.87 ms | 52.3% bf16 MFU | 1622363 tok/s
step 11932/19560 | loss 3.388663 (+0.51z)| norm 0.2723 (-0.66z)| lr 2.11e-04 | 323.03 ms | 52.2% bf16 MFU | 1622396 tok/s
step 11933/19560 | loss 3.349439 (-0.49z)| norm 0.2556 (-1.42z)| lr 2.11e-04 | 323.25 ms | 52.2% bf16 MFU | 1622372 tok/s
step 11934/19560 | loss 3.307181 (-1.56z)| norm 0.2530 (-1.52z)| lr 2.11e-04 | 322.49 ms | 52.3% bf16 MFU | 1622542 tok/s
step 11935/19560 | loss 3.361942 (-0.15z)| norm 0.2633 (-1.02z)| lr 2.11e-04 | 323.70 ms | 52.1% bf16 MFU | 1622399 tok/s
step 11936/19560 | loss 3.390789 (+0.57z)| norm 0.2552 (-1.38z)| lr 2.11e-04 | 322.64 ms | 52.3% bf16 MFU | 1622529 tok/s
step 11937/19560 | loss 3.346406 (-0.57z)| norm 0.2593 (-1.18z)| lr 2.11e-04 | 322.79 ms | 52.3% bf16 MFU | 1622615 tok/s
step 11938/19560 | loss 3.306235 (-1.59z)| norm 0.2605 (-1.12z)| lr 2.11e-04 | 323.41 ms | 52.2% bf16 MFU | 1622542 tok/s
step 11939/19560 | loss 3.402494 (+0.93z)| norm 0.2532 (-1.43z)| lr 2.11e-04 | 322.91 ms | 52.3% bf16 MFU | 1622597 tok/s
step 11940/19560 | loss 3.351524 (-0.40z)| norm 0.2686 (-0.72z)| lr 2.11e-04 | 322.66 ms | 52.3% bf16 MFU | 1622712 tok/s
step 11941/19560 | loss 3.334248 (-0.86z)| norm 0.2677 (-0.75z)| lr 2.11e-04 | 323.02 ms | 52.2% bf16 MFU | 1622731 tok/s
step 11942/19560 | loss 3.326012 (-1.09z)| norm 0.2563 (-1.25z)| lr 2.11e-04 | 322.67 ms | 52.3% bf16 MFU | 1622836 tok/s
step 11943/19560 | loss 3.326190 (-1.07z)| norm 0.2677 (-0.73z)| lr 2.11e-04 | 322.67 ms | 52.3% bf16 MFU | 1622936 tok/s
step 11944/19560 | loss 3.346773 (-0.53z)| norm 0.2784 (-0.24z)| lr 2.11e-04 | 323.49 ms | 52.2% bf16 MFU | 1622825 tok/s
step 11945/19560 | loss 3.366046 (-0.01z)| norm 0.2746 (-0.41z)| lr 2.11e-04 | 323.03 ms | 52.2% bf16 MFU | 1622836 tok/s
step 11946/19560 | loss 3.333846 (-0.85z)| norm 0.2834 (-0.01z)| lr 2.11e-04 | 322.54 ms | 52.3% bf16 MFU | 1622968 tok/s
step 11947/19560 | loss 3.419566 (+1.40z)| norm 0.2936 (+0.45z)| lr 2.11e-04 | 322.97 ms | 52.3% bf16 MFU | 1622986 tok/s
step 11948/19560 | loss 3.382573 (+0.43z)| norm 0.3013 (+0.80z)| lr 2.11e-04 | 322.90 ms | 52.3% bf16 MFU | 1623020 tok/s
step 11949/19560 | loss 3.402342 (+0.94z)| norm 0.2718 (-0.54z)| lr 2.11e-04 | 322.44 ms | 52.3% bf16 MFU | 1623168 tok/s
step 11950/19560 | loss 3.330600 (-0.94z)| norm 0.2795 (-0.19z)| lr 2.10e-04 | 322.81 ms | 52.3% bf16 MFU | 1623216 tok/s
step 11951/19560 | loss 3.373002 (+0.19z)| norm 0.2733 (-0.46z)| lr 2.10e-04 | 323.22 ms | 52.2% bf16 MFU | 1623159 tok/s
step 11952/19560 | loss 3.432609 (+1.75z)| norm 0.2774 (-0.26z)| lr 2.10e-04 | 322.71 ms | 52.3% bf16 MFU | 1623232 tok/s
step 11953/19560 | loss 3.327451 (-1.06z)| norm 0.2849 (+0.11z)| lr 2.10e-04 | 322.88 ms | 52.3% bf16 MFU | 1623260 tok/s
step 11954/19560 | loss 3.338314 (-0.75z)| norm 0.2745 (-0.38z)| lr 2.10e-04 | 323.07 ms | 52.2% bf16 MFU | 1623239 tok/s
step 11955/19560 | loss 3.368973 (+0.11z)| norm 0.2665 (-0.76z)| lr 2.10e-04 | 322.95 ms | 52.3% bf16 MFU | 1623250 tok/s
step 11956/19560 | loss 3.413903 (+1.37z)| norm 0.2602 (-1.06z)| lr 2.10e-04 | 322.92 ms | 52.3% bf16 MFU | 1623267 tok/s
step 11957/19560 | loss 3.333749 (-0.87z)| norm 0.2765 (-0.25z)| lr 2.10e-04 | 322.59 ms | 52.3% bf16 MFU | 1623365 tok/s
step 11958/19560 | loss 3.360177 (-0.14z)| norm 0.2603 (-1.03z)| lr 2.10e-04 | 323.07 ms | 52.2% bf16 MFU | 1623337 tok/s
step 11959/19560 | loss 3.383301 (+0.51z)| norm 0.2596 (-1.05z)| lr 2.10e-04 | 322.85 ms | 52.3% bf16 MFU | 1623367 tok/s
step 11960/19560 | loss 3.364073 (-0.02z)| norm 0.2696 (-0.55z)| lr 2.10e-04 | 323.50 ms | 52.2% bf16 MFU | 1623231 tok/s
step 11961/19560 | loss 3.396752 (+0.93z)| norm 0.2619 (-0.92z)| lr 2.10e-04 | 322.83 ms | 52.3% bf16 MFU | 1623270 tok/s
step 11962/19560 | loss 3.355009 (-0.28z)| norm 0.2657 (-0.72z)| lr 2.10e-04 | 322.94 ms | 52.3% bf16 MFU | 1623281 tok/s
step 11963/19560 | loss 3.333709 (-0.90z)| norm 0.2511 (-1.42z)| lr 2.10e-04 | 322.67 ms | 52.3% bf16 MFU | 1623358 tok/s
step 11964/19560 | loss 3.350341 (-0.41z)| norm 0.2597 (-0.98z)| lr 2.10e-04 | 322.94 ms | 52.3% bf16 MFU | 1623364 tok/s
step 11965/19560 | loss 3.400911 (+1.05z)| norm 0.2479 (-1.53z)| lr 2.10e-04 | 322.51 ms | 52.3% bf16 MFU | 1623478 tok/s
step 11966/19560 | loss 3.345899 (-0.54z)| norm 0.2635 (-0.76z)| lr 2.10e-04 | 323.07 ms | 52.2% bf16 MFU | 1623446 tok/s
step 11967/19560 | loss 3.368697 (+0.13z)| norm 0.2576 (-1.04z)| lr 2.10e-04 | 322.39 ms | 52.4% bf16 MFU | 1623587 tok/s
step 11968/19560 | loss 3.359636 (-0.13z)| norm 0.2750 (-0.20z)| lr 2.10e-04 | 322.55 ms | 52.3% bf16 MFU | 1623680 tok/s
step 11969/19560 | loss 3.335336 (-0.85z)| norm 0.2705 (-0.42z)| lr 2.10e-04 | 323.01 ms | 52.2% bf16 MFU | 1623652 tok/s
step 11970/19560 | loss 3.304538 (-1.74z)| norm 0.2762 (-0.14z)| lr 2.10e-04 | 322.90 ms | 52.3% bf16 MFU | 1623653 tok/s
step 11971/19560 | loss 3.336266 (-0.81z)| norm 0.2694 (-0.48z)| lr 2.09e-04 | 322.43 ms | 52.3% bf16 MFU | 1623772 tok/s
step 11972/19560 | loss 3.304355 (-1.71z)| norm 0.2746 (-0.23z)| lr 2.09e-04 | 322.75 ms | 52.3% bf16 MFU | 1623804 tok/s
step 11973/19560 | loss 3.298469 (-1.85z)| norm 0.2682 (-0.54z)| lr 2.09e-04 | 323.51 ms | 52.2% bf16 MFU | 1623645 tok/s
step 11974/19560 | loss 3.306229 (-1.60z)| norm 0.2955 (+0.78z)| lr 2.09e-04 | 322.29 ms | 52.4% bf16 MFU | 1623801 tok/s
step 11975/19560 | loss 3.341227 (-0.59z)| norm 0.2666 (-0.64z)| lr 2.09e-04 | 322.34 ms | 52.4% bf16 MFU | 1623936 tok/s
step 11976/19560 | loss 3.388513 (+0.74z)| norm 0.2797 (+0.00z)| lr 2.09e-04 | 323.32 ms | 52.2% bf16 MFU | 1623817 tok/s
step 11977/19560 | loss 3.363074 (+0.03z)| norm 0.3266 (+2.24z)| lr 2.09e-04 | 322.51 ms | 52.3% bf16 MFU | 1623909 tok/s
step 11978/19560 | loss 3.351599 (-0.29z)| norm 0.2833 (+0.14z)| lr 2.09e-04 | 322.45 ms | 52.3% bf16 MFU | 1624012 tok/s
step 11979/19560 | loss 3.344551 (-0.49z)| norm 0.2918 (+0.54z)| lr 2.09e-04 | 323.44 ms | 52.2% bf16 MFU | 1623859 tok/s
step 11980/19560 | loss 3.343552 (-0.51z)| norm 0.2946 (+0.67z)| lr 2.09e-04 | 322.80 ms | 52.3% bf16 MFU | 1623875 tok/s
step 11981/19560 | loss 3.405247 (+1.24z)| norm 0.3100 (+1.40z)| lr 2.09e-04 | 322.78 ms | 52.3% bf16 MFU | 1623896 tok/s
step 11982/19560 | loss 3.359020 (-0.07z)| norm 0.2953 (+0.67z)| lr 2.09e-04 | 322.51 ms | 52.3% bf16 MFU | 1623983 tok/s
step 11983/19560 | loss 3.278760 (-2.30z)| norm 0.3961 (+4.97z)| lr 2.09e-04 | 322.82 ms | 52.3% bf16 MFU | 1623988 tok/s
step 11984/19560 | loss 3.362751 (+0.07z)| norm 0.2996 (+0.74z)| lr 2.09e-04 | 322.69 ms | 52.3% bf16 MFU | 1624026 tok/s
step 11985/19560 | loss 3.332687 (-0.77z)| norm 0.3121 (+1.26z)| lr 2.09e-04 | 322.23 ms | 52.4% bf16 MFU | 1624178 tok/s
step 11986/19560 | loss 3.362145 (+0.05z)| norm 0.3049 (+0.93z)| lr 2.09e-04 | 323.18 ms | 52.2% bf16 MFU | 1624083 tok/s
step 11987/19560 | loss 3.276647 (-2.30z)| norm 0.3104 (+1.16z)| lr 2.09e-04 | 322.66 ms | 52.3% bf16 MFU | 1624124 tok/s
step 11988/19560 | loss 3.333833 (-0.70z)| norm 0.2965 (+0.54z)| lr 2.09e-04 | 322.39 ms | 52.3% bf16 MFU | 1624230 tok/s
step 11989/19560 | loss 3.453955 (+2.56z)| norm 0.3196 (+1.53z)| lr 2.09e-04 | 323.00 ms | 52.3% bf16 MFU | 1624177 tok/s
step 11990/19560 | loss 3.330733 (-0.79z)| norm 0.3093 (+1.07z)| lr 2.09e-04 | 322.56 ms | 52.3% bf16 MFU | 1624238 tok/s
step 11991/19560 | loss 3.356019 (-0.08z)| norm 0.3144 (+1.29z)| lr 2.09e-04 | 323.18 ms | 52.2% bf16 MFU | 1624139 tok/s
step 11992/19560 | loss 3.344005 (-0.42z)| norm 0.3306 (+1.96z)| lr 2.08e-04 | 322.52 ms | 52.3% bf16 MFU | 1624212 tok/s
step 11993/19560 | loss 3.399895 (+1.17z)| norm 0.3108 (+1.11z)| lr 2.08e-04 | 322.54 ms | 52.3% bf16 MFU | 1624275 tok/s
step 11994/19560 | loss 3.375541 (+0.49z)| norm 0.3086 (+1.00z)| lr 2.08e-04 | 322.74 ms | 52.3% bf16 MFU | 1624287 tok/s
step 11995/19560 | loss 3.409511 (+1.47z)| norm 0.2889 (+0.16z)| lr 2.08e-04 | 322.81 ms | 52.3% bf16 MFU | 1624280 tok/s
step 11996/19560 | loss 3.397898 (+1.13z)| norm 0.2863 (+0.06z)| lr 2.08e-04 | 323.28 ms | 52.2% bf16 MFU | 1624155 tok/s
step 11997/19560 | loss 3.388883 (+0.85z)| norm 0.2786 (-0.28z)| lr 2.08e-04 | 322.43 ms | 52.3% bf16 MFU | 1624249 tok/s
step 11998/19560 | loss 3.382978 (+0.67z)| norm 0.2798 (-0.22z)| lr 2.08e-04 | 322.71 ms | 52.3% bf16 MFU | 1624268 tok/s
step 11999/19560 | loss 3.462801 (+2.87z)| norm 0.2803 (-0.20z)| lr 2.08e-04 | 322.31 ms | 52.4% bf16 MFU | 1624388 tok/s
step 12000/19560 | loss 3.385080 (+0.68z)| norm 0.2853 (+0.02z)| lr 2.08e-04 | 322.67 ms | 52.3% bf16 MFU | 1624410 tok/s
val loss 3.353923
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2974/10042 = 0.296156
step 12001/19560 | loss 3.334459 (-0.73z)| norm 0.2804 (-0.19z)| lr 2.08e-04 | 322.05 ms | 52.4% bf16 MFU | 1624590 tok/s
step 12002/19560 | loss 3.407122 (+1.28z)| norm 0.2702 (-0.63z)| lr 2.08e-04 | 321.82 ms | 52.4% bf16 MFU | 1624816 tok/s
step 12003/19560 | loss 3.417277 (+1.54z)| norm 0.3147 (+1.30z)| lr 2.08e-04 | 323.03 ms | 52.2% bf16 MFU | 1624727 tok/s
step 12004/19560 | loss 3.421587 (+1.62z)| norm 0.2789 (-0.26z)| lr 2.08e-04 | 322.45 ms | 52.3% bf16 MFU | 1624788 tok/s
step 12005/19560 | loss 3.313620 (-1.30z)| norm 0.2795 (-0.24z)| lr 2.08e-04 | 323.18 ms | 52.2% bf16 MFU | 1624663 tok/s
step 12006/19560 | loss 3.390612 (+0.79z)| norm 0.2813 (-0.16z)| lr 2.08e-04 | 322.48 ms | 52.3% bf16 MFU | 1624720 tok/s
step 12007/19560 | loss 3.358922 (-0.07z)| norm 0.2615 (-1.01z)| lr 2.08e-04 | 322.80 ms | 52.3% bf16 MFU | 1624694 tok/s
step 12008/19560 | loss 3.415715 (+1.45z)| norm 0.2724 (-0.54z)| lr 2.08e-04 | 322.51 ms | 52.3% bf16 MFU | 1624741 tok/s
step 12009/19560 | loss 3.344281 (-0.47z)| norm 0.2936 (+0.38z)| lr 2.08e-04 | 322.50 ms | 52.3% bf16 MFU | 1624790 tok/s
step 12010/19560 | loss 3.415092 (+1.41z)| norm 0.2696 (-0.66z)| lr 2.08e-04 | 322.42 ms | 52.3% bf16 MFU | 1624855 tok/s
step 12011/19560 | loss 3.317549 (-1.17z)| norm 0.2721 (-0.56z)| lr 2.08e-04 | 322.97 ms | 52.3% bf16 MFU | 1624779 tok/s
step 12012/19560 | loss 3.432914 (+1.85z)| norm 0.2696 (-0.67z)| lr 2.08e-04 | 322.65 ms | 52.3% bf16 MFU | 1624786 tok/s
step 12013/19560 | loss 3.313637 (-1.26z)| norm 0.2705 (-0.63z)| lr 2.07e-04 | 322.64 ms | 52.3% bf16 MFU | 1624796 tok/s
step 12014/19560 | loss 3.346080 (-0.41z)| norm 0.2896 (+0.20z)| lr 2.07e-04 | 323.07 ms | 52.2% bf16 MFU | 1624697 tok/s
step 12015/19560 | loss 3.355511 (-0.17z)| norm 0.2970 (+0.52z)| lr 2.07e-04 | 322.68 ms | 52.3% bf16 MFU | 1624700 tok/s
step 12016/19560 | loss 3.325326 (-0.95z)| norm 0.2907 (+0.24z)| lr 2.07e-04 | 322.97 ms | 52.3% bf16 MFU | 1624633 tok/s
step 12017/19560 | loss 3.392092 (+0.79z)| norm 0.2858 (+0.03z)| lr 2.07e-04 | 322.57 ms | 52.3% bf16 MFU | 1624669 tok/s
step 12018/19560 | loss 3.293463 (-1.77z)| norm 0.3177 (+1.40z)| lr 2.07e-04 | 322.95 ms | 52.3% bf16 MFU | 1624607 tok/s
step 12019/19560 | loss 3.330750 (-0.79z)| norm 0.3126 (+1.17z)| lr 2.07e-04 | 322.57 ms | 52.3% bf16 MFU | 1624644 tok/s
step 12020/19560 | loss 3.398470 (+0.95z)| norm 0.3065 (+0.89z)| lr 2.07e-04 | 322.31 ms | 52.4% bf16 MFU | 1624744 tok/s
step 12021/19560 | loss 3.317990 (-1.11z)| norm 0.3261 (+1.70z)| lr 2.07e-04 | 323.15 ms | 52.2% bf16 MFU | 1624628 tok/s
step 12022/19560 | loss 3.335978 (-0.64z)| norm 0.2822 (-0.16z)| lr 2.07e-04 | 322.15 ms | 52.4% bf16 MFU | 1624769 tok/s
step 12023/19560 | loss 3.518086 (+3.78z)| norm 0.3039 (+0.76z)| lr 2.07e-04 | 322.95 ms | 52.3% bf16 MFU | 1624703 tok/s
step 12024/19560 | loss 3.308812 (-1.29z)| norm 0.2707 (-0.65z)| lr 2.07e-04 | 323.11 ms | 52.2% bf16 MFU | 1624598 tok/s
step 12025/19560 | loss 3.517297 (+3.53z)| norm 0.3004 (+0.60z)| lr 2.07e-04 | 322.67 ms | 52.3% bf16 MFU | 1624610 tok/s
step 12026/19560 | loss 3.478903 (+2.58z)| norm 0.2877 (+0.06z)| lr 2.07e-04 | 323.27 ms | 52.2% bf16 MFU | 1624472 tok/s
step 12027/19560 | loss 3.370632 (+0.15z)| norm 0.3007 (+0.61z)| lr 2.07e-04 | 322.39 ms | 52.3% bf16 MFU | 1624560 tok/s
step 12028/19560 | loss 3.318407 (-1.02z)| norm 0.2970 (+0.46z)| lr 2.07e-04 | 323.32 ms | 52.2% bf16 MFU | 1624412 tok/s
step 12029/19560 | loss 3.293004 (-1.57z)| norm 0.2950 (+0.38z)| lr 2.07e-04 | 322.66 ms | 52.3% bf16 MFU | 1624436 tok/s
step 12030/19560 | loss 3.314255 (-1.08z)| norm 0.2748 (-0.48z)| lr 2.07e-04 | 322.41 ms | 52.3% bf16 MFU | 1624521 tok/s
step 12031/19560 | loss 3.379436 (+0.36z)| norm 0.2751 (-0.46z)| lr 2.07e-04 | 323.15 ms | 52.2% bf16 MFU | 1624417 tok/s
step 12032/19560 | loss 3.349136 (-0.31z)| norm 0.2693 (-0.69z)| lr 2.07e-04 | 322.64 ms | 52.3% bf16 MFU | 1624445 tok/s
step 12033/19560 | loss 3.397084 (+0.74z)| norm 0.2649 (-0.87z)| lr 2.07e-04 | 323.21 ms | 52.2% bf16 MFU | 1624330 tok/s
step 12034/19560 | loss 3.391475 (+0.62z)| norm 0.2709 (-0.60z)| lr 2.06e-04 | 323.29 ms | 52.2% bf16 MFU | 1624200 tok/s
step 12035/19560 | loss 3.416673 (+1.17z)| norm 0.2748 (-0.44z)| lr 2.06e-04 | 322.67 ms | 52.3% bf16 MFU | 1624233 tok/s
step 12036/19560 | loss 3.377261 (+0.29z)| norm 0.2683 (-0.71z)| lr 2.06e-04 | 322.85 ms | 52.3% bf16 MFU | 1624219 tok/s
step 12037/19560 | loss 3.351990 (-0.27z)| norm 0.2709 (-0.59z)| lr 2.06e-04 | 322.89 ms | 52.3% bf16 MFU | 1624196 tok/s
step 12038/19560 | loss 3.366657 (+0.07z)| norm 0.2527 (-1.36z)| lr 2.06e-04 | 322.94 ms | 52.3% bf16 MFU | 1624159 tok/s
step 12039/19560 | loss 3.359455 (-0.09z)| norm 0.2783 (-0.26z)| lr 2.06e-04 | 323.50 ms | 52.2% bf16 MFU | 1623986 tok/s
step 12040/19560 | loss 3.365938 (+0.08z)| norm 0.2456 (-1.66z)| lr 2.06e-04 | 322.65 ms | 52.3% bf16 MFU | 1624033 tok/s
step 12041/19560 | loss 3.441185 (+1.75z)| norm 0.2962 (+0.53z)| lr 2.06e-04 | 323.08 ms | 52.2% bf16 MFU | 1623970 tok/s
step 12042/19560 | loss 3.383689 (+0.45z)| norm 0.2595 (-1.05z)| lr 2.06e-04 | 323.10 ms | 52.2% bf16 MFU | 1623907 tok/s
step 12043/19560 | loss 3.374076 (+0.23z)| norm 0.2703 (-0.58z)| lr 2.06e-04 | 322.67 ms | 52.3% bf16 MFU | 1623954 tok/s
step 12044/19560 | loss 3.440803 (+1.70z)| norm 0.2589 (-1.06z)| lr 2.06e-04 | 322.83 ms | 52.3% bf16 MFU | 1623958 tok/s
step 12045/19560 | loss 3.450075 (+1.90z)| norm 0.2791 (-0.17z)| lr 2.06e-04 | 322.95 ms | 52.3% bf16 MFU | 1623931 tok/s
step 12046/19560 | loss 3.390596 (+0.56z)| norm 0.2821 (-0.02z)| lr 2.06e-04 | 322.82 ms | 52.3% bf16 MFU | 1623939 tok/s
step 12047/19560 | loss 3.388643 (+0.52z)| norm 0.2540 (-1.28z)| lr 2.06e-04 | 322.95 ms | 52.3% bf16 MFU | 1623914 tok/s
step 12048/19560 | loss 3.448864 (+1.84z)| norm 0.2655 (-0.75z)| lr 2.06e-04 | 322.93 ms | 52.3% bf16 MFU | 1623894 tok/s
step 12049/19560 | loss 3.388791 (+0.50z)| norm 0.2873 (+0.26z)| lr 2.06e-04 | 323.26 ms | 52.2% bf16 MFU | 1623792 tok/s
step 12050/19560 | loss 3.391762 (+0.55z)| norm 0.2425 (-1.80z)| lr 2.06e-04 | 323.00 ms | 52.3% bf16 MFU | 1623762 tok/s
step 12051/19560 | loss 3.370423 (+0.06z)| norm 0.2777 (-0.15z)| lr 2.06e-04 | 322.44 ms | 52.3% bf16 MFU | 1623873 tok/s
step 12052/19560 | loss 3.349772 (-0.40z)| norm 0.2770 (-0.18z)| lr 2.06e-04 | 323.15 ms | 52.2% bf16 MFU | 1623802 tok/s
step 12053/19560 | loss 3.319880 (-1.06z)| norm 0.2648 (-0.74z)| lr 2.06e-04 | 322.89 ms | 52.3% bf16 MFU | 1623798 tok/s
step 12054/19560 | loss 3.371293 (+0.09z)| norm 0.2746 (-0.27z)| lr 2.06e-04 | 322.54 ms | 52.3% bf16 MFU | 1623884 tok/s
step 12055/19560 | loss 3.323990 (-0.96z)| norm 0.2815 (+0.05z)| lr 2.05e-04 | 323.03 ms | 52.2% bf16 MFU | 1623841 tok/s
step 12056/19560 | loss 3.323969 (-0.95z)| norm 0.2857 (+0.25z)| lr 2.05e-04 | 322.83 ms | 52.3% bf16 MFU | 1623851 tok/s
step 12057/19560 | loss 3.399577 (+0.75z)| norm 0.2824 (+0.08z)| lr 2.05e-04 | 322.88 ms | 52.3% bf16 MFU | 1623849 tok/s
step 12058/19560 | loss 3.302571 (-1.42z)| norm 0.2898 (+0.43z)| lr 2.05e-04 | 322.82 ms | 52.3% bf16 MFU | 1623860 tok/s
step 12059/19560 | loss 3.355513 (-0.24z)| norm 0.2766 (-0.20z)| lr 2.05e-04 | 323.12 ms | 52.2% bf16 MFU | 1623797 tok/s
step 12060/19560 | loss 3.374137 (+0.18z)| norm 0.2777 (-0.15z)| lr 2.05e-04 | 322.64 ms | 52.3% bf16 MFU | 1623856 tok/s
step 12061/19560 | loss 3.373752 (+0.17z)| norm 0.2688 (-0.58z)| lr 2.05e-04 | 323.17 ms | 52.2% bf16 MFU | 1623779 tok/s
step 12062/19560 | loss 3.316663 (-1.11z)| norm 0.2905 (+0.45z)| lr 2.05e-04 | 323.05 ms | 52.2% bf16 MFU | 1623736 tok/s
step 12063/19560 | loss 3.373282 (+0.16z)| norm 0.2802 (-0.06z)| lr 2.05e-04 | 323.22 ms | 52.2% bf16 MFU | 1623652 tok/s
step 12064/19560 | loss 3.367082 (+0.02z)| norm 0.2826 (+0.05z)| lr 2.05e-04 | 323.67 ms | 52.1% bf16 MFU | 1623460 tok/s
step 12065/19560 | loss 3.418952 (+1.17z)| norm 0.2683 (-0.65z)| lr 2.05e-04 | 322.37 ms | 52.4% bf16 MFU | 1623605 tok/s
step 12066/19560 | loss 3.387585 (+0.45z)| norm 0.2850 (+0.15z)| lr 2.05e-04 | 322.43 ms | 52.3% bf16 MFU | 1623727 tok/s
step 12067/19560 | loss 3.336356 (-0.69z)| norm 0.2587 (-1.14z)| lr 2.05e-04 | 323.60 ms | 52.2% bf16 MFU | 1623550 tok/s
step 12068/19560 | loss 3.292544 (-1.64z)| norm 0.2918 (+0.47z)| lr 2.05e-04 | 322.78 ms | 52.3% bf16 MFU | 1623588 tok/s
step 12069/19560 | loss 3.343897 (-0.50z)| norm 0.2880 (+0.28z)| lr 2.05e-04 | 323.55 ms | 52.2% bf16 MFU | 1623429 tok/s
step 12070/19560 | loss 3.309953 (-1.25z)| norm 0.2586 (-1.16z)| lr 2.05e-04 | 322.61 ms | 52.3% bf16 MFU | 1623515 tok/s
step 12071/19560 | loss 3.375491 (+0.20z)| norm 0.2794 (-0.14z)| lr 2.05e-04 | 323.10 ms | 52.2% bf16 MFU | 1623472 tok/s
step 12072/19560 | loss 3.370610 (+0.08z)| norm 0.2858 (+0.17z)| lr 2.05e-04 | 322.87 ms | 52.3% bf16 MFU | 1623491 tok/s
step 12073/19560 | loss 3.415926 (+1.08z)| norm 0.2613 (-1.03z)| lr 2.05e-04 | 322.48 ms | 52.3% bf16 MFU | 1623607 tok/s
step 12074/19560 | loss 3.376726 (+0.20z)| norm 0.3048 (+1.09z)| lr 2.05e-04 | 323.41 ms | 52.2% bf16 MFU | 1623484 tok/s
step 12075/19560 | loss 3.418490 (+1.13z)| norm 0.2851 (+0.13z)| lr 2.05e-04 | 322.34 ms | 52.4% bf16 MFU | 1623634 tok/s
step 12076/19560 | loss 3.426930 (+1.30z)| norm 0.2746 (-0.37z)| lr 2.04e-04 | 322.55 ms | 52.3% bf16 MFU | 1623724 tok/s
step 12077/19560 | loss 3.396002 (+0.62z)| norm 0.2863 (+0.20z)| lr 2.04e-04 | 323.58 ms | 52.2% bf16 MFU | 1623552 tok/s
step 12078/19560 | loss 3.307772 (-1.32z)| norm 0.2720 (-0.50z)| lr 2.04e-04 | 322.67 ms | 52.3% bf16 MFU | 1623616 tok/s
step 12079/19560 | loss 3.381027 (+0.29z)| norm 0.2689 (-0.65z)| lr 2.04e-04 | 323.60 ms | 52.2% bf16 MFU | 1623444 tok/s
step 12080/19560 | loss 3.351159 (-0.36z)| norm 0.2862 (+0.19z)| lr 2.04e-04 | 323.15 ms | 52.2% bf16 MFU | 1623392 tok/s
step 12081/19560 | loss 3.371932 (+0.10z)| norm 0.2715 (-0.52z)| lr 2.04e-04 | 322.89 ms | 52.3% bf16 MFU | 1623409 tok/s
step 12082/19560 | loss 3.321880 (-1.01z)| norm 0.2757 (-0.32z)| lr 2.04e-04 | 322.87 ms | 52.3% bf16 MFU | 1623432 tok/s
step 12083/19560 | loss 3.300477 (-1.46z)| norm 0.2792 (-0.15z)| lr 2.04e-04 | 323.74 ms | 52.1% bf16 MFU | 1623233 tok/s
step 12084/19560 | loss 3.303330 (-1.38z)| norm 0.2738 (-0.42z)| lr 2.04e-04 | 322.97 ms | 52.3% bf16 MFU | 1623238 tok/s
step 12085/19560 | loss 3.373786 (+0.16z)| norm 0.2778 (-0.23z)| lr 2.04e-04 | 323.04 ms | 52.2% bf16 MFU | 1623224 tok/s
step 12086/19560 | loss 3.340338 (-0.57z)| norm 0.2615 (-1.03z)| lr 2.04e-04 | 323.30 ms | 52.2% bf16 MFU | 1623148 tok/s
step 12087/19560 | loss 3.339432 (-0.58z)| norm 0.2828 (+0.01z)| lr 2.04e-04 | 323.62 ms | 52.2% bf16 MFU | 1622993 tok/s
step 12088/19560 | loss 3.300784 (-1.41z)| norm 0.2612 (-1.05z)| lr 2.04e-04 | 322.03 ms | 52.4% bf16 MFU | 1623248 tok/s
step 12089/19560 | loss 3.373118 (+0.17z)| norm 0.2551 (-1.35z)| lr 2.04e-04 | 323.72 ms | 52.1% bf16 MFU | 1623064 tok/s
step 12090/19560 | loss 3.409425 (+0.95z)| norm 0.2783 (-0.21z)| lr 2.04e-04 | 323.91 ms | 52.1% bf16 MFU | 1622842 tok/s
step 12091/19560 | loss 3.348674 (-0.37z)| norm 0.2666 (-0.80z)| lr 2.04e-04 | 322.91 ms | 52.3% bf16 MFU | 1622881 tok/s
step 12092/19560 | loss 3.409345 (+0.94z)| norm 0.2922 (+0.46z)| lr 2.04e-04 | 323.89 ms | 52.1% bf16 MFU | 1622672 tok/s
step 12093/19560 | loss 3.355405 (-0.23z)| norm 0.2704 (-0.64z)| lr 2.04e-04 | 322.83 ms | 52.3% bf16 MFU | 1622741 tok/s
step 12094/19560 | loss 3.367908 (+0.04z)| norm 0.2676 (-0.78z)| lr 2.04e-04 | 322.22 ms | 52.4% bf16 MFU | 1622960 tok/s
step 12095/19560 | loss 3.323087 (-0.92z)| norm 0.2544 (-1.45z)| lr 2.04e-04 | 323.17 ms | 52.2% bf16 MFU | 1622928 tok/s
step 12096/19560 | loss 3.384596 (+0.41z)| norm 0.2702 (-0.65z)| lr 2.04e-04 | 322.44 ms | 52.3% bf16 MFU | 1623081 tok/s
step 12097/19560 | loss 3.341923 (-0.52z)| norm 0.2605 (-1.13z)| lr 2.04e-04 | 322.72 ms | 52.3% bf16 MFU | 1623156 tok/s
step 12098/19560 | loss 3.339465 (-0.58z)| norm 0.2894 (+0.31z)| lr 2.03e-04 | 322.82 ms | 52.3% bf16 MFU | 1623203 tok/s
step 12099/19560 | loss 3.328698 (-0.81z)| norm 0.2628 (-1.01z)| lr 2.03e-04 | 322.68 ms | 52.3% bf16 MFU | 1623281 tok/s
step 12100/19560 | loss 3.353439 (-0.29z)| norm 0.2841 (+0.05z)| lr 2.03e-04 | 322.13 ms | 52.4% bf16 MFU | 1623496 tok/s
step 12101/19560 | loss 3.410899 (+0.96z)| norm 0.2623 (-1.04z)| lr 2.03e-04 | 323.45 ms | 52.2% bf16 MFU | 1623368 tok/s
step 12102/19560 | loss 3.344084 (-0.52z)| norm 0.2693 (-0.68z)| lr 2.03e-04 | 323.42 ms | 52.2% bf16 MFU | 1623253 tok/s
step 12103/19560 | loss 3.353050 (-0.33z)| norm 0.2927 (+0.48z)| lr 2.03e-04 | 322.80 ms | 52.3% bf16 MFU | 1623300 tok/s
step 12104/19560 | loss 3.286738 (-1.76z)| norm 0.2889 (+0.29z)| lr 2.03e-04 | 322.97 ms | 52.3% bf16 MFU | 1623303 tok/s
step 12105/19560 | loss 3.360147 (-0.15z)| norm 0.3033 (+1.03z)| lr 2.03e-04 | 322.63 ms | 52.3% bf16 MFU | 1623390 tok/s
step 12106/19560 | loss 3.397216 (+0.66z)| norm 0.2894 (+0.32z)| lr 2.03e-04 | 322.90 ms | 52.3% bf16 MFU | 1623404 tok/s
step 12107/19560 | loss 3.333190 (-0.74z)| norm 0.2852 (+0.11z)| lr 2.03e-04 | 323.11 ms | 52.2% bf16 MFU | 1623364 tok/s
step 12108/19560 | loss 3.362122 (-0.11z)| norm 0.2981 (+0.77z)| lr 2.03e-04 | 322.81 ms | 52.3% bf16 MFU | 1623401 tok/s
step 12109/19560 | loss 3.382690 (+0.34z)| norm 0.3075 (+1.25z)| lr 2.03e-04 | 322.92 ms | 52.3% bf16 MFU | 1623411 tok/s
step 12110/19560 | loss 3.358845 (-0.18z)| norm 0.2867 (+0.19z)| lr 2.03e-04 | 323.17 ms | 52.2% bf16 MFU | 1623357 tok/s
step 12111/19560 | loss 3.386797 (+0.42z)| norm 0.2950 (+0.76z)| lr 2.03e-04 | 322.82 ms | 52.3% bf16 MFU | 1623394 tok/s
step 12112/19560 | loss 3.334975 (-0.73z)| norm 0.2948 (+0.75z)| lr 2.03e-04 | 322.67 ms | 52.3% bf16 MFU | 1623465 tok/s
step 12113/19560 | loss 3.357993 (-0.22z)| norm 0.3029 (+1.24z)| lr 2.03e-04 | 322.52 ms | 52.3% bf16 MFU | 1623572 tok/s
step 12114/19560 | loss 3.327696 (-0.89z)| norm 0.2754 (-0.38z)| lr 2.03e-04 | 323.43 ms | 52.2% bf16 MFU | 1623446 tok/s
step 12115/19560 | loss 3.401780 (+0.75z)| norm 0.2748 (-0.41z)| lr 2.03e-04 | 322.81 ms | 52.3% bf16 MFU | 1623482 tok/s
step 12116/19560 | loss 3.324247 (-1.00z)| norm 0.2824 (+0.06z)| lr 2.03e-04 | 323.05 ms | 52.2% bf16 MFU | 1623453 tok/s
step 12117/19560 | loss 3.364136 (-0.09z)| norm 0.2720 (-0.56z)| lr 2.03e-04 | 322.66 ms | 52.3% bf16 MFU | 1623525 tok/s
step 12118/19560 | loss 3.383195 (+0.34z)| norm 0.3362 (+3.30z)| lr 2.03e-04 | 322.84 ms | 52.3% bf16 MFU | 1623549 tok/s
step 12119/19560 | loss 3.377796 (+0.21z)| norm 0.3107 (+1.78z)| lr 2.02e-04 | 322.91 ms | 52.3% bf16 MFU | 1623553 tok/s
step 12120/19560 | loss 3.400772 (+0.73z)| norm 0.2956 (+0.92z)| lr 2.02e-04 | 322.61 ms | 52.3% bf16 MFU | 1623633 tok/s
step 12121/19560 | loss 3.393004 (+0.55z)| norm 0.3154 (+2.14z)| lr 2.02e-04 | 322.74 ms | 52.3% bf16 MFU | 1623677 tok/s
step 12122/19560 | loss 3.321167 (-1.08z)| norm 0.3085 (+1.71z)| lr 2.02e-04 | 323.09 ms | 52.2% bf16 MFU | 1623628 tok/s
step 12123/19560 | loss 3.391785 (+0.54z)| norm 0.3106 (+1.81z)| lr 2.02e-04 | 322.59 ms | 52.3% bf16 MFU | 1623710 tok/s
step 12124/19560 | loss 3.455712 (+1.96z)| norm 0.2927 (+0.70z)| lr 2.02e-04 | 322.66 ms | 52.3% bf16 MFU | 1623769 tok/s
step 12125/19560 | loss 3.358261 (-0.23z)| norm 0.2909 (+0.59z)| lr 2.02e-04 | 322.59 ms | 52.3% bf16 MFU | 1623842 tok/s
step 12126/19560 | loss 3.353777 (-0.33z)| norm 0.2876 (+0.38z)| lr 2.02e-04 | 323.09 ms | 52.2% bf16 MFU | 1623787 tok/s
step 12127/19560 | loss 3.317370 (-1.14z)| norm 0.2780 (-0.20z)| lr 2.02e-04 | 322.69 ms | 52.3% bf16 MFU | 1623835 tok/s
step 12128/19560 | loss 3.379774 (+0.29z)| norm 0.3034 (+1.33z)| lr 2.02e-04 | 322.19 ms | 52.4% bf16 MFU | 1624005 tok/s
step 12129/19560 | loss 3.374469 (+0.16z)| norm 0.2625 (-1.14z)| lr 2.02e-04 | 323.29 ms | 52.2% bf16 MFU | 1623890 tok/s
step 12130/19560 | loss 3.387407 (+0.46z)| norm 0.2961 (+0.88z)| lr 2.02e-04 | 322.55 ms | 52.3% bf16 MFU | 1623969 tok/s
step 12131/19560 | loss 3.353549 (-0.31z)| norm 0.2654 (-0.96z)| lr 2.02e-04 | 322.73 ms | 52.3% bf16 MFU | 1623998 tok/s
step 12132/19560 | loss 3.346644 (-0.45z)| norm 0.3035 (+1.35z)| lr 2.02e-04 | 322.49 ms | 52.3% bf16 MFU | 1624086 tok/s
step 12133/19560 | loss 3.318495 (-1.11z)| norm 0.2963 (+0.89z)| lr 2.02e-04 | 322.84 ms | 52.3% bf16 MFU | 1624081 tok/s
step 12134/19560 | loss 3.414367 (+1.11z)| norm 0.2792 (-0.14z)| lr 2.02e-04 | 322.49 ms | 52.3% bf16 MFU | 1624163 tok/s
step 12135/19560 | loss 3.369307 (+0.07z)| norm 0.2882 (+0.39z)| lr 2.02e-04 | 322.82 ms | 52.3% bf16 MFU | 1624160 tok/s
step 12136/19560 | loss 3.315089 (-1.17z)| norm 0.2866 (+0.29z)| lr 2.02e-04 | 322.77 ms | 52.3% bf16 MFU | 1624170 tok/s
step 12137/19560 | loss 3.378379 (+0.29z)| norm 0.3062 (+1.47z)| lr 2.02e-04 | 322.80 ms | 52.3% bf16 MFU | 1624171 tok/s
step 12138/19560 | loss 3.347023 (-0.43z)| norm 0.2797 (-0.13z)| lr 2.02e-04 | 322.44 ms | 52.3% bf16 MFU | 1624261 tok/s
step 12139/19560 | loss 3.374060 (+0.19z)| norm 0.2836 (+0.10z)| lr 2.02e-04 | 323.09 ms | 52.2% bf16 MFU | 1624185 tok/s
step 12140/19560 | loss 3.376065 (+0.25z)| norm 0.2672 (-0.90z)| lr 2.01e-04 | 322.90 ms | 52.3% bf16 MFU | 1624159 tok/s
step 12141/19560 | loss 3.394755 (+0.68z)| norm 0.2820 (-0.01z)| lr 2.01e-04 | 322.35 ms | 52.4% bf16 MFU | 1624273 tok/s
step 12142/19560 | loss 3.361763 (-0.10z)| norm 0.3068 (+1.48z)| lr 2.01e-04 | 322.45 ms | 52.3% bf16 MFU | 1624357 tok/s
step 12143/19560 | loss 3.308603 (-1.35z)| norm 0.2924 (+0.62z)| lr 2.01e-04 | 322.51 ms | 52.3% bf16 MFU | 1624422 tok/s
step 12144/19560 | loss 3.338401 (-0.65z)| norm 0.2901 (+0.48z)| lr 2.01e-04 | 322.58 ms | 52.3% bf16 MFU | 1624466 tok/s
step 12145/19560 | loss 3.355142 (-0.25z)| norm 0.2644 (-1.06z)| lr 2.01e-04 | 322.72 ms | 52.3% bf16 MFU | 1624473 tok/s
step 12146/19560 | loss 3.310272 (-1.32z)| norm 0.2774 (-0.26z)| lr 2.01e-04 | 322.44 ms | 52.3% bf16 MFU | 1624548 tok/s
step 12147/19560 | loss 3.389016 (+0.54z)| norm 0.2717 (-0.60z)| lr 2.01e-04 | 322.76 ms | 52.3% bf16 MFU | 1624539 tok/s
step 12148/19560 | loss 3.373145 (+0.17z)| norm 0.3027 (+1.32z)| lr 2.01e-04 | 322.84 ms | 52.3% bf16 MFU | 1624512 tok/s
step 12149/19560 | loss 3.311889 (-1.29z)| norm 0.2677 (-0.84z)| lr 2.01e-04 | 322.82 ms | 52.3% bf16 MFU | 1624490 tok/s
step 12150/19560 | loss 3.312374 (-1.27z)| norm 0.2611 (-1.25z)| lr 2.01e-04 | 322.65 ms | 52.3% bf16 MFU | 1624512 tok/s
step 12151/19560 | loss 3.284690 (-1.95z)| norm 0.2656 (-0.95z)| lr 2.01e-04 | 323.36 ms | 52.2% bf16 MFU | 1624357 tok/s
step 12152/19560 | loss 3.295732 (-1.67z)| norm 0.2658 (-0.93z)| lr 2.01e-04 | 322.83 ms | 52.3% bf16 MFU | 1624340 tok/s
step 12153/19560 | loss 3.357028 (-0.15z)| norm 0.2654 (-0.94z)| lr 2.01e-04 | 322.36 ms | 52.4% bf16 MFU | 1624442 tok/s
step 12154/19560 | loss 3.342314 (-0.52z)| norm 0.2837 (+0.23z)| lr 2.01e-04 | 322.98 ms | 52.3% bf16 MFU | 1624384 tok/s
step 12155/19560 | loss 3.377710 (+0.43z)| norm 0.2617 (-1.16z)| lr 2.01e-04 | 323.05 ms | 52.2% bf16 MFU | 1624310 tok/s
step 12156/19560 | loss 3.406937 (+1.20z)| norm 0.3075 (+1.76z)| lr 2.01e-04 | 322.46 ms | 52.3% bf16 MFU | 1624391 tok/s
step 12157/19560 | loss 3.358100 (-0.13z)| norm 0.2582 (-1.36z)| lr 2.01e-04 | 323.17 ms | 52.2% bf16 MFU | 1624286 tok/s
step 12158/19560 | loss 3.327735 (-0.97z)| norm 0.2884 (+0.55z)| lr 2.01e-04 | 322.72 ms | 52.3% bf16 MFU | 1624302 tok/s
step 12159/19560 | loss 3.319539 (-1.17z)| norm 0.2773 (-0.16z)| lr 2.01e-04 | 322.86 ms | 52.3% bf16 MFU | 1624281 tok/s
step 12160/19560 | loss 3.354390 (-0.22z)| norm 0.3046 (+1.55z)| lr 2.01e-04 | 322.59 ms | 52.3% bf16 MFU | 1624330 tok/s
step 12161/19560 | loss 3.343033 (-0.52z)| norm 0.2636 (-1.03z)| lr 2.00e-04 | 322.89 ms | 52.3% bf16 MFU | 1624299 tok/s
step 12162/19560 | loss 3.375848 (+0.38z)| norm 0.2927 (+0.79z)| lr 2.00e-04 | 322.95 ms | 52.3% bf16 MFU | 1624256 tok/s
step 12163/19560 | loss 3.290775 (-1.92z)| norm 0.2649 (-0.95z)| lr 2.00e-04 | 323.10 ms | 52.2% bf16 MFU | 1624178 tok/s
step 12164/19560 | loss 3.358599 (-0.06z)| norm 0.2686 (-0.72z)| lr 2.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1624212 tok/s
step 12165/19560 | loss 3.334640 (-0.71z)| norm 0.2688 (-0.70z)| lr 2.00e-04 | 322.61 ms | 52.3% bf16 MFU | 1624258 tok/s
step 12166/19560 | loss 3.383734 (+0.62z)| norm 0.2958 (+0.97z)| lr 2.00e-04 | 322.90 ms | 52.3% bf16 MFU | 1624228 tok/s
step 12167/19560 | loss 3.400257 (+1.06z)| norm 0.2545 (-1.60z)| lr 2.00e-04 | 322.74 ms | 52.3% bf16 MFU | 1624242 tok/s
step 12168/19560 | loss 3.367468 (+0.17z)| norm 0.2935 (+0.81z)| lr 2.00e-04 | 322.85 ms | 52.3% bf16 MFU | 1624226 tok/s
step 12169/19560 | loss 3.376684 (+0.44z)| norm 0.2647 (-0.99z)| lr 2.00e-04 | 323.19 ms | 52.2% bf16 MFU | 1624127 tok/s
step 12170/19560 | loss 3.406393 (+1.25z)| norm 0.2804 (-0.00z)| lr 2.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1624187 tok/s
step 12171/19560 | loss 3.383149 (+0.61z)| norm 0.3087 (+1.76z)| lr 2.00e-04 | 322.93 ms | 52.3% bf16 MFU | 1624154 tok/s
step 12172/19560 | loss 3.357607 (-0.07z)| norm 0.2735 (-0.47z)| lr 2.00e-04 | 322.17 ms | 52.4% bf16 MFU | 1624314 tok/s
step 12173/19560 | loss 3.374005 (+0.41z)| norm 0.3106 (+1.84z)| lr 2.00e-04 | 322.77 ms | 52.3% bf16 MFU | 1624316 tok/s
step 12174/19560 | loss 3.300607 (-1.66z)| norm 0.2789 (-0.14z)| lr 2.00e-04 | 322.24 ms | 52.4% bf16 MFU | 1624450 tok/s
step 12175/19560 | loss 3.306684 (-1.46z)| norm 0.2826 (+0.08z)| lr 2.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1624421 tok/s
step 12176/19560 | loss 3.416264 (+1.67z)| norm 0.2749 (-0.42z)| lr 2.00e-04 | 322.94 ms | 52.3% bf16 MFU | 1624374 tok/s
step 12177/19560 | loss 3.329770 (-0.80z)| norm 0.3017 (+1.27z)| lr 2.00e-04 | 322.53 ms | 52.3% bf16 MFU | 1624434 tok/s
step 12178/19560 | loss 3.307237 (-1.42z)| norm 0.2721 (-0.63z)| lr 2.00e-04 | 323.31 ms | 52.2% bf16 MFU | 1624295 tok/s
step 12179/19560 | loss 3.396601 (+1.13z)| norm 0.2778 (-0.26z)| lr 2.00e-04 | 322.70 ms | 52.3% bf16 MFU | 1624316 tok/s
step 12180/19560 | loss 3.317616 (-1.11z)| norm 0.2699 (-0.77z)| lr 2.00e-04 | 322.67 ms | 52.3% bf16 MFU | 1624342 tok/s
step 12181/19560 | loss 3.348468 (-0.25z)| norm 0.2916 (+0.62z)| lr 2.00e-04 | 322.83 ms | 52.3% bf16 MFU | 1624328 tok/s
step 12182/19560 | loss 3.450931 (+2.59z)| norm 0.2866 (+0.29z)| lr 1.99e-04 | 322.14 ms | 52.4% bf16 MFU | 1624487 tok/s
step 12183/19560 | loss 3.311615 (-1.27z)| norm 0.2936 (+0.74z)| lr 1.99e-04 | 322.68 ms | 52.3% bf16 MFU | 1624503 tok/s
step 12184/19560 | loss 3.388596 (+0.84z)| norm 0.2946 (+0.80z)| lr 1.99e-04 | 322.32 ms | 52.4% bf16 MFU | 1624608 tok/s
step 12185/19560 | loss 3.327501 (-0.83z)| norm 0.2689 (-0.85z)| lr 1.99e-04 | 322.70 ms | 52.3% bf16 MFU | 1624613 tok/s
step 12186/19560 | loss 3.348510 (-0.26z)| norm 0.2729 (-0.58z)| lr 1.99e-04 | 322.82 ms | 52.3% bf16 MFU | 1624586 tok/s
step 12187/19560 | loss 3.343785 (-0.39z)| norm 0.2723 (-0.62z)| lr 1.99e-04 | 322.25 ms | 52.4% bf16 MFU | 1624705 tok/s
step 12188/19560 | loss 3.335399 (-0.62z)| norm 0.2944 (+0.79z)| lr 1.99e-04 | 322.87 ms | 52.3% bf16 MFU | 1624662 tok/s
step 12189/19560 | loss 3.375156 (+0.49z)| norm 0.2852 (+0.19z)| lr 1.99e-04 | 322.55 ms | 52.3% bf16 MFU | 1624702 tok/s
step 12190/19560 | loss 3.311560 (-1.28z)| norm 0.2859 (+0.24z)| lr 1.99e-04 | 322.51 ms | 52.3% bf16 MFU | 1624749 tok/s
step 12191/19560 | loss 3.440938 (+2.27z)| norm 0.2696 (-0.80z)| lr 1.99e-04 | 322.56 ms | 52.3% bf16 MFU | 1624782 tok/s
step 12192/19560 | loss 3.378308 (+0.55z)| norm 0.2757 (-0.41z)| lr 1.99e-04 | 322.79 ms | 52.3% bf16 MFU | 1624755 tok/s
step 12193/19560 | loss 3.310572 (-1.28z)| norm 0.2869 (+0.31z)| lr 1.99e-04 | 322.67 ms | 52.3% bf16 MFU | 1624760 tok/s
step 12194/19560 | loss 3.364901 (+0.22z)| norm 0.2640 (-1.15z)| lr 1.99e-04 | 324.54 ms | 52.0% bf16 MFU | 1624295 tok/s
step 12195/19560 | loss 3.396215 (+1.06z)| norm 0.2979 (+1.00z)| lr 1.99e-04 | 321.90 ms | 52.4% bf16 MFU | 1624518 tok/s
step 12196/19560 | loss 3.324260 (-0.93z)| norm 0.2716 (-0.68z)| lr 1.99e-04 | 322.97 ms | 52.3% bf16 MFU | 1624459 tok/s
step 12197/19560 | loss 3.350285 (-0.21z)| norm 0.2784 (-0.23z)| lr 1.99e-04 | 322.46 ms | 52.3% bf16 MFU | 1624532 tok/s
step 12198/19560 | loss 3.358005 (-0.01z)| norm 0.2851 (+0.18z)| lr 1.99e-04 | 322.46 ms | 52.3% bf16 MFU | 1624600 tok/s
step 12199/19560 | loss 3.276259 (-2.22z)| norm 0.2724 (-0.64z)| lr 1.99e-04 | 322.82 ms | 52.3% bf16 MFU | 1624574 tok/s
step 12200/19560 | loss 3.397375 (+1.08z)| norm 0.2910 (+0.56z)| lr 1.99e-04 | 322.52 ms | 52.3% bf16 MFU | 1624625 tok/s
step 12201/19560 | loss 3.333959 (-0.63z)| norm 0.2703 (-0.79z)| lr 1.99e-04 | 322.87 ms | 52.3% bf16 MFU | 1624586 tok/s
step 12202/19560 | loss 3.360454 (+0.10z)| norm 0.2777 (-0.29z)| lr 1.99e-04 | 322.23 ms | 52.4% bf16 MFU | 1624711 tok/s
step 12203/19560 | loss 3.365143 (+0.24z)| norm 0.2619 (-1.30z)| lr 1.99e-04 | 322.22 ms | 52.4% bf16 MFU | 1624832 tok/s
step 12204/19560 | loss 3.306797 (-1.37z)| norm 0.2877 (+0.37z)| lr 1.98e-04 | 322.60 ms | 52.3% bf16 MFU | 1624849 tok/s
step 12205/19560 | loss 3.353886 (-0.04z)| norm 0.2743 (-0.50z)| lr 1.98e-04 | 322.55 ms | 52.3% bf16 MFU | 1624879 tok/s
step 12206/19560 | loss 3.402159 (+1.30z)| norm 0.2771 (-0.32z)| lr 1.98e-04 | 322.53 ms | 52.3% bf16 MFU | 1624913 tok/s
step 12207/19560 | loss 3.361086 (+0.15z)| norm 0.2724 (-0.63z)| lr 1.98e-04 | 322.78 ms | 52.3% bf16 MFU | 1624881 tok/s
step 12208/19560 | loss 3.308746 (-1.31z)| norm 0.2774 (-0.30z)| lr 1.98e-04 | 322.56 ms | 52.3% bf16 MFU | 1624908 tok/s
step 12209/19560 | loss 3.421848 (+1.83z)| norm 0.2751 (-0.45z)| lr 1.98e-04 | 322.32 ms | 52.4% bf16 MFU | 1624992 tok/s
step 12210/19560 | loss 3.318627 (-1.03z)| norm 0.2615 (-1.32z)| lr 1.98e-04 | 323.36 ms | 52.2% bf16 MFU | 1624812 tok/s
step 12211/19560 | loss 3.406935 (+1.40z)| norm 0.2700 (-0.77z)| lr 1.98e-04 | 323.27 ms | 52.2% bf16 MFU | 1624663 tok/s
step 12212/19560 | loss 3.423076 (+1.81z)| norm 0.2763 (-0.36z)| lr 1.98e-04 | 322.52 ms | 52.3% bf16 MFU | 1624709 tok/s
step 12213/19560 | loss 3.303588 (-1.47z)| norm 0.2693 (-0.81z)| lr 1.98e-04 | 322.15 ms | 52.4% bf16 MFU | 1624846 tok/s
step 12214/19560 | loss 3.425238 (+1.83z)| norm 0.2762 (-0.37z)| lr 1.98e-04 | 322.78 ms | 52.3% bf16 MFU | 1624817 tok/s
step 12215/19560 | loss 3.362451 (+0.12z)| norm 0.2638 (-1.16z)| lr 1.98e-04 | 323.43 ms | 52.2% bf16 MFU | 1624628 tok/s
step 12216/19560 | loss 3.417209 (+1.58z)| norm 0.2980 (+1.04z)| lr 1.98e-04 | 323.05 ms | 52.2% bf16 MFU | 1624543 tok/s
step 12217/19560 | loss 3.445335 (+2.28z)| norm 0.2739 (-0.54z)| lr 1.98e-04 | 322.13 ms | 52.4% bf16 MFU | 1624694 tok/s
step 12218/19560 | loss 3.369786 (+0.29z)| norm 0.2651 (-1.11z)| lr 1.98e-04 | 322.56 ms | 52.3% bf16 MFU | 1624728 tok/s
step 12219/19560 | loss 3.321808 (-0.99z)| norm 0.2678 (-0.93z)| lr 1.98e-04 | 322.73 ms | 52.3% bf16 MFU | 1624719 tok/s
step 12220/19560 | loss 3.361213 (+0.07z)| norm 0.2702 (-0.77z)| lr 1.98e-04 | 322.68 ms | 52.3% bf16 MFU | 1624722 tok/s
step 12221/19560 | loss 3.498924 (+3.55z)| norm 0.2792 (-0.18z)| lr 1.98e-04 | 323.03 ms | 52.2% bf16 MFU | 1624637 tok/s
step 12222/19560 | loss 3.330949 (-0.72z)| norm 0.2679 (-0.92z)| lr 1.98e-04 | 322.34 ms | 52.4% bf16 MFU | 1624730 tok/s
step 12223/19560 | loss 3.330184 (-0.74z)| norm 0.2599 (-1.45z)| lr 1.98e-04 | 322.59 ms | 52.3% bf16 MFU | 1624757 tok/s
step 12224/19560 | loss 3.327383 (-0.80z)| norm 0.2645 (-1.14z)| lr 1.98e-04 | 322.65 ms | 52.3% bf16 MFU | 1624765 tok/s
step 12225/19560 | loss 3.319908 (-0.98z)| norm 0.2547 (-1.78z)| lr 1.97e-04 | 323.01 ms | 52.2% bf16 MFU | 1624683 tok/s
step 12226/19560 | loss 3.316710 (-1.06z)| norm 0.2673 (-0.94z)| lr 1.97e-04 | 322.38 ms | 52.4% bf16 MFU | 1624765 tok/s
step 12227/19560 | loss 3.280988 (-1.93z)| norm 0.2921 (+0.67z)| lr 1.97e-04 | 322.87 ms | 52.3% bf16 MFU | 1624718 tok/s
step 12228/19560 | loss 3.351234 (-0.17z)| norm 0.2916 (+0.62z)| lr 1.97e-04 | 322.95 ms | 52.3% bf16 MFU | 1624654 tok/s
step 12229/19560 | loss 3.362517 (+0.12z)| norm 0.2791 (-0.20z)| lr 1.97e-04 | 322.84 ms | 52.3% bf16 MFU | 1624619 tok/s
step 12230/19560 | loss 3.377821 (+0.50z)| norm 0.2905 (+0.54z)| lr 1.97e-04 | 322.75 ms | 52.3% bf16 MFU | 1624611 tok/s
step 12231/19560 | loss 3.346873 (-0.28z)| norm 0.2515 (-1.98z)| lr 1.97e-04 | 323.30 ms | 52.2% bf16 MFU | 1624464 tok/s
step 12232/19560 | loss 3.347295 (-0.28z)| norm 0.3074 (+1.62z)| lr 1.97e-04 | 322.51 ms | 52.3% bf16 MFU | 1624524 tok/s
step 12233/19560 | loss 3.359926 (+0.04z)| norm 0.2623 (-1.26z)| lr 1.97e-04 | 323.03 ms | 52.2% bf16 MFU | 1624448 tok/s
step 12234/19560 | loss 3.362510 (+0.11z)| norm 0.2913 (+0.61z)| lr 1.97e-04 | 322.48 ms | 52.3% bf16 MFU | 1624515 tok/s
step 12235/19560 | loss 3.334467 (-0.61z)| norm 0.2656 (-1.03z)| lr 1.97e-04 | 322.68 ms | 52.3% bf16 MFU | 1624530 tok/s
step 12236/19560 | loss 3.393133 (+0.88z)| norm 0.2807 (-0.05z)| lr 1.97e-04 | 323.00 ms | 52.3% bf16 MFU | 1624462 tok/s
step 12237/19560 | loss 3.418959 (+1.52z)| norm 0.2720 (-0.61z)| lr 1.97e-04 | 323.25 ms | 52.2% bf16 MFU | 1624334 tok/s
step 12238/19560 | loss 3.428695 (+1.73z)| norm 0.2976 (+1.05z)| lr 1.97e-04 | 322.90 ms | 52.3% bf16 MFU | 1624302 tok/s
step 12239/19560 | loss 3.401848 (+1.06z)| norm 0.2971 (+1.02z)| lr 1.97e-04 | 323.19 ms | 52.2% bf16 MFU | 1624197 tok/s
step 12240/19560 | loss 3.271717 (-2.13z)| norm 0.2936 (+0.79z)| lr 1.97e-04 | 323.06 ms | 52.2% bf16 MFU | 1624132 tok/s
step 12241/19560 | loss 3.341808 (-0.41z)| norm 0.3004 (+1.23z)| lr 1.97e-04 | 323.60 ms | 52.2% bf16 MFU | 1623935 tok/s
step 12242/19560 | loss 3.323366 (-0.86z)| norm 0.3034 (+1.40z)| lr 1.97e-04 | 322.76 ms | 52.3% bf16 MFU | 1623958 tok/s
step 12243/19560 | loss 3.346624 (-0.29z)| norm 0.3006 (+1.20z)| lr 1.97e-04 | 323.03 ms | 52.2% bf16 MFU | 1623913 tok/s
step 12244/19560 | loss 3.352183 (-0.16z)| norm 0.3030 (+1.34z)| lr 1.97e-04 | 322.61 ms | 52.3% bf16 MFU | 1623974 tok/s
step 12245/19560 | loss 3.359501 (+0.02z)| norm 0.2891 (+0.45z)| lr 1.97e-04 | 323.16 ms | 52.2% bf16 MFU | 1623893 tok/s
step 12246/19560 | loss 3.415526 (+1.39z)| norm 0.2794 (-0.15z)| lr 1.96e-04 | 322.93 ms | 52.3% bf16 MFU | 1623875 tok/s
step 12247/19560 | loss 3.369259 (+0.26z)| norm 0.3024 (+1.41z)| lr 1.96e-04 | 322.79 ms | 52.3% bf16 MFU | 1623892 tok/s
step 12248/19560 | loss 3.324588 (-0.82z)| norm 0.2927 (+0.75z)| lr 1.96e-04 | 322.45 ms | 52.3% bf16 MFU | 1623995 tok/s
step 12249/19560 | loss 3.495600 (+3.22z)| norm 0.2880 (+0.46z)| lr 1.96e-04 | 323.25 ms | 52.2% bf16 MFU | 1623893 tok/s
step 12250/19560 | loss 3.358180 (-0.02z)| norm 0.2869 (+0.40z)| lr 1.96e-04 | 322.90 ms | 52.3% bf16 MFU | 1623883 tok/s
val loss 3.349317
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3002/10042 = 0.298944
step 12251/19560 | loss 3.376878 (+0.42z)| norm 0.2930 (+0.85z)| lr 1.96e-04 | 322.77 ms | 52.3% bf16 MFU | 1623905 tok/s
step 12252/19560 | loss 3.376822 (+0.44z)| norm 0.2791 (-0.13z)| lr 1.96e-04 | 323.97 ms | 52.1% bf16 MFU | 1623627 tok/s
step 12253/19560 | loss 3.373501 (+0.36z)| norm 0.3132 (+2.23z)| lr 1.96e-04 | 323.15 ms | 52.2% bf16 MFU | 1623566 tok/s
step 12254/19560 | loss 3.378597 (+0.48z)| norm 0.2655 (-1.07z)| lr 1.96e-04 | 323.88 ms | 52.1% bf16 MFU | 1623326 tok/s
step 12255/19560 | loss 3.357457 (-0.04z)| norm 0.3136 (+2.21z)| lr 1.96e-04 | 322.88 ms | 52.3% bf16 MFU | 1623349 tok/s
step 12256/19560 | loss 3.355645 (-0.08z)| norm 0.2656 (-1.04z)| lr 1.96e-04 | 323.60 ms | 52.2% bf16 MFU | 1623191 tok/s
step 12257/19560 | loss 3.346144 (-0.30z)| norm 0.2962 (+1.03z)| lr 1.96e-04 | 323.38 ms | 52.2% bf16 MFU | 1623096 tok/s
step 12258/19560 | loss 3.381057 (+0.54z)| norm 0.2813 (+0.02z)| lr 1.96e-04 | 322.19 ms | 52.4% bf16 MFU | 1623305 tok/s
step 12259/19560 | loss 3.363433 (+0.12z)| norm 0.2624 (-1.28z)| lr 1.96e-04 | 323.52 ms | 52.2% bf16 MFU | 1623168 tok/s
step 12260/19560 | loss 3.374812 (+0.39z)| norm 0.2783 (-0.18z)| lr 1.96e-04 | 323.64 ms | 52.1% bf16 MFU | 1623008 tok/s
step 12261/19560 | loss 3.310582 (-1.17z)| norm 0.2632 (-1.20z)| lr 1.96e-04 | 322.18 ms | 52.4% bf16 MFU | 1623224 tok/s
step 12262/19560 | loss 3.323811 (-0.83z)| norm 0.2708 (-0.67z)| lr 1.96e-04 | 323.86 ms | 52.1% bf16 MFU | 1623006 tok/s
step 12263/19560 | loss 3.414795 (+1.36z)| norm 0.2749 (-0.38z)| lr 1.96e-04 | 322.82 ms | 52.3% bf16 MFU | 1623059 tok/s
step 12264/19560 | loss 3.382386 (+0.57z)| norm 0.2554 (-1.69z)| lr 1.96e-04 | 322.66 ms | 52.3% bf16 MFU | 1623151 tok/s
step 12265/19560 | loss 3.308942 (-1.19z)| norm 0.3061 (+1.78z)| lr 1.96e-04 | 322.92 ms | 52.3% bf16 MFU | 1623173 tok/s
step 12266/19560 | loss 3.328909 (-0.71z)| norm 0.2578 (-1.51z)| lr 1.96e-04 | 322.25 ms | 52.4% bf16 MFU | 1623362 tok/s
step 12267/19560 | loss 3.356207 (-0.05z)| norm 0.2930 (+0.87z)| lr 1.95e-04 | 322.99 ms | 52.3% bf16 MFU | 1623355 tok/s
step 12268/19560 | loss 3.434761 (+1.81z)| norm 0.3058 (+1.70z)| lr 1.95e-04 | 322.86 ms | 52.3% bf16 MFU | 1623382 tok/s
step 12269/19560 | loss 3.352599 (-0.14z)| norm 0.3036 (+1.53z)| lr 1.95e-04 | 322.34 ms | 52.4% bf16 MFU | 1623537 tok/s
step 12270/19560 | loss 3.346701 (-0.27z)| norm 0.2791 (-0.08z)| lr 1.95e-04 | 322.93 ms | 52.3% bf16 MFU | 1623537 tok/s
step 12271/19560 | loss 3.365190 (+0.16z)| norm 0.3037 (+1.56z)| lr 1.95e-04 | 323.60 ms | 52.2% bf16 MFU | 1623369 tok/s
step 12272/19560 | loss 3.398155 (+0.93z)| norm 0.2809 (+0.04z)| lr 1.95e-04 | 323.37 ms | 52.2% bf16 MFU | 1623267 tok/s
step 12273/19560 | loss 3.343040 (-0.38z)| norm 0.3146 (+2.23z)| lr 1.95e-04 | 323.15 ms | 52.2% bf16 MFU | 1623224 tok/s
step 12274/19560 | loss 3.395762 (+0.86z)| norm 0.2968 (+1.05z)| lr 1.95e-04 | 322.61 ms | 52.3% bf16 MFU | 1623321 tok/s
step 12275/19560 | loss 3.356498 (-0.07z)| norm 0.2980 (+1.11z)| lr 1.95e-04 | 323.31 ms | 52.2% bf16 MFU | 1623237 tok/s
step 12276/19560 | loss 3.375750 (+0.39z)| norm 0.2857 (+0.32z)| lr 1.95e-04 | 323.37 ms | 52.2% bf16 MFU | 1623142 tok/s
step 12277/19560 | loss 3.355076 (-0.11z)| norm 0.3063 (+1.64z)| lr 1.95e-04 | 323.20 ms | 52.2% bf16 MFU | 1623095 tok/s
step 12278/19560 | loss 3.358598 (-0.04z)| norm 0.2952 (+0.90z)| lr 1.95e-04 | 322.79 ms | 52.3% bf16 MFU | 1623152 tok/s
step 12279/19560 | loss 3.393050 (+0.79z)| norm 0.2817 (+0.00z)| lr 1.95e-04 | 323.19 ms | 52.2% bf16 MFU | 1623105 tok/s
step 12280/19560 | loss 3.332858 (-0.70z)| norm 0.2868 (+0.33z)| lr 1.95e-04 | 323.07 ms | 52.2% bf16 MFU | 1623093 tok/s
step 12281/19560 | loss 3.341533 (-0.48z)| norm 0.2993 (+1.14z)| lr 1.95e-04 | 323.04 ms | 52.2% bf16 MFU | 1623087 tok/s
step 12282/19560 | loss 3.393568 (+0.79z)| norm 0.2833 (+0.08z)| lr 1.95e-04 | 322.88 ms | 52.3% bf16 MFU | 1623123 tok/s
step 12283/19560 | loss 3.342999 (-0.45z)| norm 0.2757 (-0.43z)| lr 1.95e-04 | 323.28 ms | 52.2% bf16 MFU | 1623054 tok/s
step 12284/19560 | loss 3.369969 (+0.22z)| norm 0.2803 (-0.11z)| lr 1.95e-04 | 322.83 ms | 52.3% bf16 MFU | 1623103 tok/s
step 12285/19560 | loss 3.366883 (+0.14z)| norm 0.2826 (+0.03z)| lr 1.95e-04 | 323.21 ms | 52.2% bf16 MFU | 1623054 tok/s
step 12286/19560 | loss 3.349201 (-0.30z)| norm 0.2892 (+0.48z)| lr 1.95e-04 | 323.49 ms | 52.2% bf16 MFU | 1622938 tok/s
step 12287/19560 | loss 3.415365 (+1.32z)| norm 0.2723 (-0.67z)| lr 1.95e-04 | 322.56 ms | 52.3% bf16 MFU | 1623060 tok/s
step 12288/19560 | loss 3.292051 (-1.70z)| norm 0.2945 (+0.85z)| lr 1.95e-04 | 322.80 ms | 52.3% bf16 MFU | 1623117 tok/s
step 12289/19560 | loss 3.338646 (-0.56z)| norm 0.2869 (+0.32z)| lr 1.94e-04 | 322.90 ms | 52.3% bf16 MFU | 1623146 tok/s
step 12290/19560 | loss 3.329845 (-0.76z)| norm 0.3000 (+1.21z)| lr 1.94e-04 | 323.45 ms | 52.2% bf16 MFU | 1623036 tok/s
step 12291/19560 | loss 3.478024 (+2.77z)| norm 0.2964 (+0.95z)| lr 1.94e-04 | 322.53 ms | 52.3% bf16 MFU | 1623162 tok/s
step 12292/19560 | loss 3.326125 (-0.86z)| norm 0.2670 (-1.06z)| lr 1.94e-04 | 323.09 ms | 52.2% bf16 MFU | 1623140 tok/s
step 12293/19560 | loss 3.373653 (+0.26z)| norm 0.2837 (+0.07z)| lr 1.94e-04 | 323.63 ms | 52.1% bf16 MFU | 1622985 tok/s
step 12294/19560 | loss 3.360372 (-0.05z)| norm 0.2739 (-0.59z)| lr 1.94e-04 | 323.24 ms | 52.2% bf16 MFU | 1622934 tok/s
step 12295/19560 | loss 3.357230 (-0.12z)| norm 0.2752 (-0.52z)| lr 1.94e-04 | 323.00 ms | 52.3% bf16 MFU | 1622947 tok/s
step 12296/19560 | loss 3.367839 (+0.14z)| norm 0.2824 (-0.01z)| lr 1.94e-04 | 322.72 ms | 52.3% bf16 MFU | 1623030 tok/s
step 12297/19560 | loss 3.345739 (-0.39z)| norm 0.2741 (-0.60z)| lr 1.94e-04 | 323.19 ms | 52.2% bf16 MFU | 1622990 tok/s
step 12298/19560 | loss 3.355520 (-0.14z)| norm 0.2789 (-0.26z)| lr 1.94e-04 | 322.67 ms | 52.3% bf16 MFU | 1623082 tok/s
step 12299/19560 | loss 3.374949 (+0.33z)| norm 0.2829 (+0.04z)| lr 1.94e-04 | 323.06 ms | 52.2% bf16 MFU | 1623071 tok/s
step 12300/19560 | loss 3.330519 (-0.74z)| norm 0.2601 (-1.57z)| lr 1.94e-04 | 322.86 ms | 52.3% bf16 MFU | 1623111 tok/s
step 12301/19560 | loss 3.385769 (+0.59z)| norm 0.2745 (-0.54z)| lr 1.94e-04 | 322.49 ms | 52.3% bf16 MFU | 1623242 tok/s
step 12302/19560 | loss 3.347240 (-0.35z)| norm 0.2798 (-0.16z)| lr 1.94e-04 | 322.72 ms | 52.3% bf16 MFU | 1623311 tok/s
step 12303/19560 | loss 3.392328 (+0.73z)| norm 0.2810 (-0.07z)| lr 1.94e-04 | 322.47 ms | 52.3% bf16 MFU | 1623439 tok/s
step 12304/19560 | loss 3.347253 (-0.36z)| norm 0.2723 (-0.69z)| lr 1.94e-04 | 322.49 ms | 52.3% bf16 MFU | 1623554 tok/s
step 12305/19560 | loss 3.417546 (+1.34z)| norm 0.2756 (-0.44z)| lr 1.94e-04 | 322.80 ms | 52.3% bf16 MFU | 1623584 tok/s
step 12306/19560 | loss 3.320048 (-1.04z)| norm 0.2806 (-0.09z)| lr 1.94e-04 | 322.86 ms | 52.3% bf16 MFU | 1623600 tok/s
step 12307/19560 | loss 3.360092 (-0.05z)| norm 0.2918 (+0.71z)| lr 1.94e-04 | 322.67 ms | 52.3% bf16 MFU | 1623662 tok/s
step 12308/19560 | loss 3.438507 (+1.83z)| norm 0.2791 (-0.21z)| lr 1.94e-04 | 322.50 ms | 52.3% bf16 MFU | 1623765 tok/s
step 12309/19560 | loss 3.407312 (+1.06z)| norm 0.2806 (-0.09z)| lr 1.94e-04 | 323.29 ms | 52.2% bf16 MFU | 1623663 tok/s
step 12310/19560 | loss 3.380962 (+0.44z)| norm 0.2682 (-0.98z)| lr 1.93e-04 | 322.51 ms | 52.3% bf16 MFU | 1623763 tok/s
step 12311/19560 | loss 3.380608 (+0.42z)| norm 0.2707 (-0.79z)| lr 1.93e-04 | 322.96 ms | 52.3% bf16 MFU | 1623744 tok/s
step 12312/19560 | loss 3.286120 (-1.88z)| norm 0.2777 (-0.28z)| lr 1.93e-04 | 322.49 ms | 52.3% bf16 MFU | 1623844 tok/s
step 12313/19560 | loss 3.335927 (-0.66z)| norm 0.2674 (-1.02z)| lr 1.93e-04 | 322.46 ms | 52.3% bf16 MFU | 1623947 tok/s
step 12314/19560 | loss 3.452900 (+2.15z)| norm 0.2816 (+0.00z)| lr 1.93e-04 | 322.63 ms | 52.3% bf16 MFU | 1624003 tok/s
step 12315/19560 | loss 3.389527 (+0.61z)| norm 0.2899 (+0.60z)| lr 1.93e-04 | 322.58 ms | 52.3% bf16 MFU | 1624067 tok/s
step 12316/19560 | loss 3.332509 (-0.76z)| norm 0.2829 (+0.09z)| lr 1.93e-04 | 323.13 ms | 52.2% bf16 MFU | 1623990 tok/s
step 12317/19560 | loss 3.328779 (-0.84z)| norm 0.2664 (-1.09z)| lr 1.93e-04 | 322.55 ms | 52.3% bf16 MFU | 1624062 tok/s
step 12318/19560 | loss 3.343055 (-0.51z)| norm 0.3106 (+2.07z)| lr 1.93e-04 | 322.62 ms | 52.3% bf16 MFU | 1624113 tok/s
step 12319/19560 | loss 3.385134 (+0.53z)| norm 0.2648 (-1.19z)| lr 1.93e-04 | 323.11 ms | 52.2% bf16 MFU | 1624039 tok/s
step 12320/19560 | loss 3.356452 (-0.17z)| norm 0.3013 (+1.38z)| lr 1.93e-04 | 322.75 ms | 52.3% bf16 MFU | 1624060 tok/s
step 12321/19560 | loss 3.344427 (-0.47z)| norm 0.2674 (-1.00z)| lr 1.93e-04 | 322.27 ms | 52.4% bf16 MFU | 1624199 tok/s
step 12322/19560 | loss 3.346762 (-0.41z)| norm 0.2914 (+0.68z)| lr 1.93e-04 | 322.97 ms | 52.3% bf16 MFU | 1624157 tok/s
step 12323/19560 | loss 3.383566 (+0.50z)| norm 0.2858 (+0.29z)| lr 1.93e-04 | 322.64 ms | 52.3% bf16 MFU | 1624199 tok/s
step 12324/19560 | loss 3.338164 (-0.63z)| norm 0.2802 (-0.12z)| lr 1.93e-04 | 322.73 ms | 52.3% bf16 MFU | 1624217 tok/s
step 12325/19560 | loss 3.341958 (-0.53z)| norm 0.2768 (-0.36z)| lr 1.93e-04 | 322.45 ms | 52.3% bf16 MFU | 1624304 tok/s
step 12326/19560 | loss 3.355189 (-0.20z)| norm 0.2738 (-0.56z)| lr 1.93e-04 | 322.52 ms | 52.3% bf16 MFU | 1624369 tok/s
step 12327/19560 | loss 3.329982 (-0.85z)| norm 0.2869 (+0.36z)| lr 1.93e-04 | 322.97 ms | 52.3% bf16 MFU | 1624316 tok/s
step 12328/19560 | loss 3.493544 (+3.11z)| norm 0.2874 (+0.40z)| lr 1.93e-04 | 323.26 ms | 52.2% bf16 MFU | 1624194 tok/s
step 12329/19560 | loss 3.368064 (+0.08z)| norm 0.3003 (+1.30z)| lr 1.93e-04 | 323.26 ms | 52.2% bf16 MFU | 1624078 tok/s
step 12330/19560 | loss 3.324189 (-0.97z)| norm 0.2905 (+0.59z)| lr 1.93e-04 | 322.70 ms | 52.3% bf16 MFU | 1624110 tok/s
step 12331/19560 | loss 3.351475 (-0.31z)| norm 0.2786 (-0.26z)| lr 1.93e-04 | 323.22 ms | 52.2% bf16 MFU | 1624007 tok/s
step 12332/19560 | loss 3.391479 (+0.64z)| norm 0.2984 (+1.14z)| lr 1.92e-04 | 323.02 ms | 52.2% bf16 MFU | 1623961 tok/s
step 12333/19560 | loss 3.428155 (+1.50z)| norm 0.2796 (-0.20z)| lr 1.92e-04 | 322.65 ms | 52.3% bf16 MFU | 1624009 tok/s
step 12334/19560 | loss 3.393874 (+0.68z)| norm 0.2751 (-0.52z)| lr 1.92e-04 | 322.95 ms | 52.3% bf16 MFU | 1623982 tok/s
step 12335/19560 | loss 3.312383 (-1.26z)| norm 0.2877 (+0.37z)| lr 1.92e-04 | 322.85 ms | 52.3% bf16 MFU | 1623978 tok/s
step 12336/19560 | loss 3.399471 (+0.80z)| norm 0.2677 (-1.05z)| lr 1.92e-04 | 322.84 ms | 52.3% bf16 MFU | 1623978 tok/s
step 12337/19560 | loss 3.369362 (+0.09z)| norm 0.2623 (-1.41z)| lr 1.92e-04 | 322.46 ms | 52.3% bf16 MFU | 1624074 tok/s
step 12338/19560 | loss 3.338154 (-0.67z)| norm 0.2708 (-0.82z)| lr 1.92e-04 | 322.47 ms | 52.3% bf16 MFU | 1624164 tok/s
step 12339/19560 | loss 3.336264 (-0.70z)| norm 0.2648 (-1.24z)| lr 1.92e-04 | 323.05 ms | 52.2% bf16 MFU | 1624102 tok/s
step 12340/19560 | loss 3.358108 (-0.16z)| norm 0.2682 (-0.99z)| lr 1.92e-04 | 323.04 ms | 52.2% bf16 MFU | 1624047 tok/s
step 12341/19560 | loss 3.341763 (-0.57z)| norm 0.2741 (-0.58z)| lr 1.92e-04 | 322.75 ms | 52.3% bf16 MFU | 1624066 tok/s
step 12342/19560 | loss 3.385547 (+0.52z)| norm 0.3009 (+1.29z)| lr 1.92e-04 | 322.93 ms | 52.3% bf16 MFU | 1624040 tok/s
step 12343/19560 | loss 3.375304 (+0.26z)| norm 0.2629 (-1.37z)| lr 1.92e-04 | 322.75 ms | 52.3% bf16 MFU | 1624058 tok/s
step 12344/19560 | loss 3.352536 (-0.29z)| norm 0.2606 (-1.51z)| lr 1.92e-04 | 323.23 ms | 52.2% bf16 MFU | 1623958 tok/s
step 12345/19560 | loss 3.395302 (+0.80z)| norm 0.2826 (+0.02z)| lr 1.92e-04 | 322.52 ms | 52.3% bf16 MFU | 1624040 tok/s
step 12346/19560 | loss 3.278639 (-2.11z)| norm 0.2614 (-1.45z)| lr 1.92e-04 | 322.76 ms | 52.3% bf16 MFU | 1624057 tok/s
step 12347/19560 | loss 3.373736 (+0.26z)| norm 0.3110 (+1.96z)| lr 1.92e-04 | 322.93 ms | 52.3% bf16 MFU | 1624029 tok/s
step 12348/19560 | loss 3.357435 (-0.15z)| norm 0.2910 (+0.57z)| lr 1.92e-04 | 323.22 ms | 52.2% bf16 MFU | 1623931 tok/s
step 12349/19560 | loss 3.392050 (+0.77z)| norm 0.2616 (-1.44z)| lr 1.92e-04 | 322.09 ms | 52.4% bf16 MFU | 1624122 tok/s
step 12350/19560 | loss 3.381136 (+0.47z)| norm 0.2929 (+0.69z)| lr 1.92e-04 | 322.96 ms | 52.3% bf16 MFU | 1624085 tok/s
step 12351/19560 | loss 3.341607 (-0.57z)| norm 0.2630 (-1.37z)| lr 1.92e-04 | 323.53 ms | 52.2% bf16 MFU | 1623906 tok/s
step 12352/19560 | loss 3.422936 (+1.54z)| norm 0.2904 (+0.51z)| lr 1.92e-04 | 322.35 ms | 52.4% bf16 MFU | 1624035 tok/s
step 12353/19560 | loss 3.327640 (-0.95z)| norm 0.2778 (-0.38z)| lr 1.91e-04 | 322.96 ms | 52.3% bf16 MFU | 1624003 tok/s
step 12354/19560 | loss 3.366210 (+0.05z)| norm 0.2850 (+0.12z)| lr 1.91e-04 | 322.75 ms | 52.3% bf16 MFU | 1624025 tok/s
step 12355/19560 | loss 3.325097 (-1.06z)| norm 0.2819 (-0.10z)| lr 1.91e-04 | 322.37 ms | 52.4% bf16 MFU | 1624142 tok/s
step 12356/19560 | loss 3.340722 (-0.64z)| norm 0.2552 (-1.94z)| lr 1.91e-04 | 323.12 ms | 52.2% bf16 MFU | 1624065 tok/s
step 12357/19560 | loss 3.383791 (+0.51z)| norm 0.2837 (+0.05z)| lr 1.91e-04 | 322.52 ms | 52.3% bf16 MFU | 1624142 tok/s
step 12358/19560 | loss 3.386497 (+0.58z)| norm 0.2913 (+0.57z)| lr 1.91e-04 | 322.55 ms | 52.3% bf16 MFU | 1624208 tok/s
step 12359/19560 | loss 3.379600 (+0.39z)| norm 0.2737 (-0.68z)| lr 1.91e-04 | 322.73 ms | 52.3% bf16 MFU | 1624224 tok/s
step 12360/19560 | loss 3.342657 (-0.60z)| norm 0.2720 (-0.78z)| lr 1.91e-04 | 323.11 ms | 52.2% bf16 MFU | 1624145 tok/s
step 12361/19560 | loss 3.305131 (-1.57z)| norm 0.2731 (-0.72z)| lr 1.91e-04 | 322.66 ms | 52.3% bf16 MFU | 1624182 tok/s
step 12362/19560 | loss 3.338954 (-0.67z)| norm 0.2749 (-0.57z)| lr 1.91e-04 | 322.37 ms | 52.4% bf16 MFU | 1624291 tok/s
step 12363/19560 | loss 3.356157 (-0.22z)| norm 0.2464 (-2.57z)| lr 1.91e-04 | 322.64 ms | 52.3% bf16 MFU | 1624325 tok/s
step 12364/19560 | loss 3.351835 (-0.33z)| norm 0.2721 (-0.74z)| lr 1.91e-04 | 322.42 ms | 52.3% bf16 MFU | 1624413 tok/s
step 12365/19560 | loss 3.406812 (+1.13z)| norm 0.2809 (-0.13z)| lr 1.91e-04 | 322.76 ms | 52.3% bf16 MFU | 1624411 tok/s
step 12366/19560 | loss 3.296863 (-1.76z)| norm 0.2608 (-1.52z)| lr 1.91e-04 | 323.00 ms | 52.3% bf16 MFU | 1624348 tok/s
step 12367/19560 | loss 3.397004 (+0.90z)| norm 0.2875 (+0.36z)| lr 1.91e-04 | 322.81 ms | 52.3% bf16 MFU | 1624338 tok/s
step 12368/19560 | loss 3.295751 (-1.81z)| norm 0.2494 (-2.27z)| lr 1.91e-04 | 323.25 ms | 52.2% bf16 MFU | 1624218 tok/s
step 12369/19560 | loss 3.388996 (+0.68z)| norm 0.2763 (-0.39z)| lr 1.91e-04 | 322.68 ms | 52.3% bf16 MFU | 1624246 tok/s
step 12370/19560 | loss 3.343711 (-0.54z)| norm 0.2839 (+0.15z)| lr 1.91e-04 | 323.18 ms | 52.2% bf16 MFU | 1624148 tok/s
step 12371/19560 | loss 3.386861 (+0.61z)| norm 0.2513 (-2.09z)| lr 1.91e-04 | 323.18 ms | 52.2% bf16 MFU | 1624056 tok/s
step 12372/19560 | loss 3.359577 (-0.13z)| norm 0.2809 (-0.02z)| lr 1.91e-04 | 322.27 ms | 52.4% bf16 MFU | 1624195 tok/s
step 12373/19560 | loss 3.325971 (-1.02z)| norm 0.2579 (-1.60z)| lr 1.91e-04 | 323.00 ms | 52.3% bf16 MFU | 1624145 tok/s
step 12374/19560 | loss 3.349406 (-0.38z)| norm 0.2806 (-0.02z)| lr 1.91e-04 | 323.05 ms | 52.2% bf16 MFU | 1624084 tok/s
step 12375/19560 | loss 3.414473 (+1.36z)| norm 0.2640 (-1.16z)| lr 1.90e-04 | 322.86 ms | 52.3% bf16 MFU | 1624075 tok/s
step 12376/19560 | loss 3.302724 (-1.62z)| norm 0.2856 (+0.36z)| lr 1.90e-04 | 323.04 ms | 52.2% bf16 MFU | 1624021 tok/s
step 12377/19560 | loss 3.305539 (-1.57z)| norm 0.2536 (-1.84z)| lr 1.90e-04 | 322.69 ms | 52.3% bf16 MFU | 1624057 tok/s
step 12378/19560 | loss 3.315079 (-1.29z)| norm 0.2751 (-0.35z)| lr 1.90e-04 | 322.50 ms | 52.3% bf16 MFU | 1624140 tok/s
step 12379/19560 | loss 3.493771 (+3.44z)| norm 0.2864 (+0.44z)| lr 1.90e-04 | 323.08 ms | 52.2% bf16 MFU | 1624071 tok/s
step 12380/19560 | loss 3.395673 (+0.86z)| norm 0.3300 (+3.28z)| lr 1.90e-04 | 323.13 ms | 52.2% bf16 MFU | 1623994 tok/s
step 12381/19560 | loss 3.402077 (+1.02z)| norm 0.2710 (-0.62z)| lr 1.90e-04 | 322.20 ms | 52.4% bf16 MFU | 1624155 tok/s
step 12382/19560 | loss 3.331713 (-0.81z)| norm 0.3041 (+1.58z)| lr 1.90e-04 | 323.06 ms | 52.2% bf16 MFU | 1624093 tok/s
step 12383/19560 | loss 3.367775 (+0.13z)| norm 0.2762 (-0.27z)| lr 1.90e-04 | 322.83 ms | 52.3% bf16 MFU | 1624090 tok/s
step 12384/19560 | loss 3.366108 (+0.08z)| norm 0.3088 (+1.91z)| lr 1.90e-04 | 322.70 ms | 52.3% bf16 MFU | 1624121 tok/s
step 12385/19560 | loss 3.296696 (-1.70z)| norm 0.2675 (-0.87z)| lr 1.90e-04 | 323.08 ms | 52.2% bf16 MFU | 1624053 tok/s
step 12386/19560 | loss 3.320260 (-1.07z)| norm 0.2896 (+0.62z)| lr 1.90e-04 | 322.89 ms | 52.3% bf16 MFU | 1624036 tok/s
step 12387/19560 | loss 3.363266 (+0.03z)| norm 0.2841 (+0.24z)| lr 1.90e-04 | 322.90 ms | 52.3% bf16 MFU | 1624018 tok/s
step 12388/19560 | loss 3.393480 (+0.80z)| norm 0.2874 (+0.46z)| lr 1.90e-04 | 323.13 ms | 52.2% bf16 MFU | 1623943 tok/s
step 12389/19560 | loss 3.325140 (-0.96z)| norm 0.2747 (-0.41z)| lr 1.90e-04 | 322.61 ms | 52.3% bf16 MFU | 1624002 tok/s
step 12390/19560 | loss 3.500414 (+3.37z)| norm 0.3151 (+2.28z)| lr 1.90e-04 | 322.82 ms | 52.3% bf16 MFU | 1624007 tok/s
step 12391/19560 | loss 3.380238 (+0.42z)| norm 0.3129 (+2.08z)| lr 1.90e-04 | 323.27 ms | 52.2% bf16 MFU | 1623899 tok/s
step 12392/19560 | loss 3.361805 (-0.04z)| norm 0.2905 (+0.59z)| lr 1.90e-04 | 322.85 ms | 52.3% bf16 MFU | 1623900 tok/s
step 12393/19560 | loss 3.404200 (+1.00z)| norm 0.3151 (+2.21z)| lr 1.90e-04 | 323.05 ms | 52.2% bf16 MFU | 1623852 tok/s
step 12394/19560 | loss 3.342469 (-0.54z)| norm 0.2817 (-0.01z)| lr 1.90e-04 | 322.80 ms | 52.3% bf16 MFU | 1623869 tok/s
step 12395/19560 | loss 3.370690 (+0.16z)| norm 0.2922 (+0.69z)| lr 1.90e-04 | 322.46 ms | 52.3% bf16 MFU | 1623971 tok/s
step 12396/19560 | loss 3.374794 (+0.28z)| norm 0.2864 (+0.31z)| lr 1.89e-04 | 323.04 ms | 52.2% bf16 MFU | 1623922 tok/s
step 12397/19560 | loss 3.390745 (+0.67z)| norm 0.3058 (+1.62z)| lr 1.89e-04 | 322.86 ms | 52.3% bf16 MFU | 1623919 tok/s
step 12398/19560 | loss 3.337223 (-0.67z)| norm 0.2671 (-0.97z)| lr 1.89e-04 | 322.96 ms | 52.3% bf16 MFU | 1623893 tok/s
step 12399/19560 | loss 3.331537 (-0.81z)| norm 0.2815 (+0.00z)| lr 1.89e-04 | 323.19 ms | 52.2% bf16 MFU | 1623810 tok/s
step 12400/19560 | loss 3.345057 (-0.46z)| norm 0.2612 (-1.35z)| lr 1.89e-04 | 323.12 ms | 52.2% bf16 MFU | 1623748 tok/s
step 12401/19560 | loss 3.289443 (-1.82z)| norm 0.2710 (-0.69z)| lr 1.89e-04 | 322.31 ms | 52.4% bf16 MFU | 1623893 tok/s
step 12402/19560 | loss 3.311672 (-1.25z)| norm 0.2790 (-0.13z)| lr 1.89e-04 | 322.49 ms | 52.3% bf16 MFU | 1623987 tok/s
step 12403/19560 | loss 3.377158 (+0.37z)| norm 0.2573 (-1.59z)| lr 1.89e-04 | 323.38 ms | 52.2% bf16 MFU | 1623852 tok/s
step 12404/19560 | loss 3.353102 (-0.22z)| norm 0.3033 (+1.53z)| lr 1.89e-04 | 322.88 ms | 52.3% bf16 MFU | 1623850 tok/s
step 12405/19560 | loss 3.367858 (+0.14z)| norm 0.2602 (-1.37z)| lr 1.89e-04 | 322.83 ms | 52.3% bf16 MFU | 1623859 tok/s
step 12406/19560 | loss 3.345612 (-0.41z)| norm 0.2965 (+1.10z)| lr 1.89e-04 | 322.70 ms | 52.3% bf16 MFU | 1623901 tok/s
step 12407/19560 | loss 3.324094 (-0.93z)| norm 0.2451 (-2.32z)| lr 1.89e-04 | 322.62 ms | 52.3% bf16 MFU | 1623960 tok/s
step 12408/19560 | loss 3.401475 (+0.97z)| norm 0.2726 (-0.49z)| lr 1.89e-04 | 322.24 ms | 52.4% bf16 MFU | 1624111 tok/s
step 12409/19560 | loss 3.300910 (-1.49z)| norm 0.2875 (+0.51z)| lr 1.89e-04 | 322.99 ms | 52.3% bf16 MFU | 1624067 tok/s
step 12410/19560 | loss 3.365242 (+0.09z)| norm 0.2633 (-1.09z)| lr 1.89e-04 | 322.87 ms | 52.3% bf16 MFU | 1624055 tok/s
step 12411/19560 | loss 3.285125 (-1.84z)| norm 0.2593 (-1.34z)| lr 1.89e-04 | 322.56 ms | 52.3% bf16 MFU | 1624123 tok/s
step 12412/19560 | loss 3.452148 (+2.15z)| norm 0.2743 (-0.35z)| lr 1.89e-04 | 322.75 ms | 52.3% bf16 MFU | 1624139 tok/s
step 12413/19560 | loss 3.392653 (+0.73z)| norm 0.2718 (-0.50z)| lr 1.89e-04 | 323.00 ms | 52.3% bf16 MFU | 1624091 tok/s
step 12414/19560 | loss 3.545488 (+4.03z)| norm 0.2810 (+0.11z)| lr 1.89e-04 | 323.35 ms | 52.2% bf16 MFU | 1623958 tok/s
step 12415/19560 | loss 3.336207 (-0.60z)| norm 0.2806 (+0.08z)| lr 1.89e-04 | 322.82 ms | 52.3% bf16 MFU | 1623964 tok/s
step 12416/19560 | loss 3.308980 (-1.21z)| norm 0.2807 (+0.10z)| lr 1.89e-04 | 322.70 ms | 52.3% bf16 MFU | 1624001 tok/s
step 12417/19560 | loss 3.319654 (-0.96z)| norm 0.2721 (-0.47z)| lr 1.89e-04 | 322.35 ms | 52.4% bf16 MFU | 1624124 tok/s
step 12418/19560 | loss 3.370776 (+0.17z)| norm 0.2880 (+0.59z)| lr 1.88e-04 | 323.31 ms | 52.2% bf16 MFU | 1623999 tok/s
step 12419/19560 | loss 3.306249 (-1.27z)| norm 0.2549 (-1.59z)| lr 1.88e-04 | 323.12 ms | 52.2% bf16 MFU | 1623927 tok/s
step 12420/19560 | loss 3.358249 (-0.09z)| norm 0.2891 (+0.68z)| lr 1.88e-04 | 323.00 ms | 52.3% bf16 MFU | 1623889 tok/s
step 12421/19560 | loss 3.351431 (-0.24z)| norm 0.2647 (-0.94z)| lr 1.88e-04 | 322.36 ms | 52.4% bf16 MFU | 1624015 tok/s
step 12422/19560 | loss 3.352004 (-0.23z)| norm 0.2818 (+0.20z)| lr 1.88e-04 | 323.06 ms | 52.2% bf16 MFU | 1623959 tok/s
step 12423/19560 | loss 3.377074 (+0.34z)| norm 0.2688 (-0.67z)| lr 1.88e-04 | 322.73 ms | 52.3% bf16 MFU | 1623988 tok/s
step 12424/19560 | loss 3.358829 (-0.07z)| norm 0.2804 (+0.11z)| lr 1.88e-04 | 323.24 ms | 52.2% bf16 MFU | 1623888 tok/s
step 12425/19560 | loss 3.368218 (+0.14z)| norm 0.2621 (-1.10z)| lr 1.88e-04 | 323.00 ms | 52.3% bf16 MFU | 1623853 tok/s
step 12426/19560 | loss 3.411455 (+1.11z)| norm 0.2736 (-0.33z)| lr 1.88e-04 | 322.83 ms | 52.3% bf16 MFU | 1623862 tok/s
step 12427/19560 | loss 3.397330 (+0.78z)| norm 0.2848 (+0.40z)| lr 1.88e-04 | 322.73 ms | 52.3% bf16 MFU | 1623896 tok/s
step 12428/19560 | loss 3.392767 (+0.67z)| norm 0.2722 (-0.43z)| lr 1.88e-04 | 322.95 ms | 52.3% bf16 MFU | 1623874 tok/s
step 12429/19560 | loss 3.330285 (-0.74z)| norm 0.2766 (-0.14z)| lr 1.88e-04 | 322.90 ms | 52.3% bf16 MFU | 1623865 tok/s
step 12430/19560 | loss 3.348773 (-0.32z)| norm 0.2785 (-0.02z)| lr 1.88e-04 | 323.09 ms | 52.2% bf16 MFU | 1623808 tok/s
step 12431/19560 | loss 3.331570 (-0.70z)| norm 0.2738 (-0.33z)| lr 1.88e-04 | 323.62 ms | 52.2% bf16 MFU | 1623621 tok/s
step 12432/19560 | loss 3.378697 (+0.36z)| norm 0.2623 (-1.08z)| lr 1.88e-04 | 323.00 ms | 52.3% bf16 MFU | 1623600 tok/s
step 12433/19560 | loss 3.360303 (-0.04z)| norm 0.2726 (-0.40z)| lr 1.88e-04 | 322.84 ms | 52.3% bf16 MFU | 1623619 tok/s
step 12434/19560 | loss 3.319079 (-0.98z)| norm 0.2589 (-1.28z)| lr 1.88e-04 | 323.59 ms | 52.2% bf16 MFU | 1623450 tok/s
step 12435/19560 | loss 3.362679 (+0.01z)| norm 0.2670 (-0.74z)| lr 1.88e-04 | 322.87 ms | 52.3% bf16 MFU | 1623469 tok/s
step 12436/19560 | loss 3.310524 (-1.16z)| norm 0.2908 (+0.82z)| lr 1.88e-04 | 323.30 ms | 52.2% bf16 MFU | 1623379 tok/s
step 12437/19560 | loss 3.250125 (-2.47z)| norm 0.2572 (-1.37z)| lr 1.88e-04 | 322.51 ms | 52.3% bf16 MFU | 1623494 tok/s
step 12438/19560 | loss 3.332463 (-0.61z)| norm 0.2681 (-0.65z)| lr 1.88e-04 | 322.79 ms | 52.3% bf16 MFU | 1623532 tok/s
step 12439/19560 | loss 3.429592 (+1.55z)| norm 0.2916 (+0.86z)| lr 1.87e-04 | 323.40 ms | 52.2% bf16 MFU | 1623414 tok/s
step 12440/19560 | loss 3.355060 (-0.12z)| norm 0.2897 (+0.73z)| lr 1.87e-04 | 323.24 ms | 52.2% bf16 MFU | 1623342 tok/s
step 12441/19560 | loss 3.345165 (-0.35z)| norm 0.2967 (+1.17z)| lr 1.87e-04 | 322.70 ms | 52.3% bf16 MFU | 1623411 tok/s
step 12442/19560 | loss 3.310522 (-1.12z)| norm 0.2715 (-0.45z)| lr 1.87e-04 | 323.06 ms | 52.2% bf16 MFU | 1623384 tok/s
step 12443/19560 | loss 3.368606 (+0.21z)| norm 0.2991 (+1.32z)| lr 1.87e-04 | 323.34 ms | 52.2% bf16 MFU | 1623289 tok/s
step 12444/19560 | loss 3.424917 (+1.47z)| norm 0.2691 (-0.60z)| lr 1.87e-04 | 323.18 ms | 52.2% bf16 MFU | 1623239 tok/s
step 12445/19560 | loss 3.368263 (+0.18z)| norm 0.2740 (-0.30z)| lr 1.87e-04 | 323.00 ms | 52.3% bf16 MFU | 1623236 tok/s
step 12446/19560 | loss 3.384497 (+0.54z)| norm 0.2677 (-0.69z)| lr 1.87e-04 | 322.41 ms | 52.3% bf16 MFU | 1623382 tok/s
step 12447/19560 | loss 3.318595 (-0.94z)| norm 0.2682 (-0.66z)| lr 1.87e-04 | 322.60 ms | 52.3% bf16 MFU | 1623474 tok/s
step 12448/19560 | loss 3.329208 (-0.70z)| norm 0.2831 (+0.33z)| lr 1.87e-04 | 323.48 ms | 52.2% bf16 MFU | 1623340 tok/s
step 12449/19560 | loss 3.306198 (-1.20z)| norm 0.2548 (-1.52z)| lr 1.87e-04 | 322.98 ms | 52.3% bf16 MFU | 1623337 tok/s
step 12450/19560 | loss 3.296593 (-1.40z)| norm 0.2776 (-0.02z)| lr 1.87e-04 | 322.55 ms | 52.3% bf16 MFU | 1623442 tok/s
step 12451/19560 | loss 3.366845 (+0.17z)| norm 0.2594 (-1.20z)| lr 1.87e-04 | 323.19 ms | 52.2% bf16 MFU | 1623382 tok/s
step 12452/19560 | loss 3.364623 (+0.12z)| norm 0.2766 (-0.07z)| lr 1.87e-04 | 323.90 ms | 52.1% bf16 MFU | 1623146 tok/s
step 12453/19560 | loss 3.363578 (+0.09z)| norm 0.2756 (-0.14z)| lr 1.87e-04 | 322.97 ms | 52.3% bf16 MFU | 1623154 tok/s
step 12454/19560 | loss 3.268311 (-2.00z)| norm 0.3746 (+5.49z)| lr 1.87e-04 | 323.37 ms | 52.2% bf16 MFU | 1623062 tok/s
step 12455/19560 | loss 3.367948 (+0.20z)| norm 0.3724 (+4.81z)| lr 1.87e-04 | 322.83 ms | 52.3% bf16 MFU | 1623112 tok/s
step 12456/19560 | loss 3.364747 (+0.15z)| norm 0.2901 (+0.57z)| lr 1.87e-04 | 322.87 ms | 52.3% bf16 MFU | 1623148 tok/s
step 12457/19560 | loss 3.429933 (+1.62z)| norm 0.3293 (+2.52z)| lr 1.87e-04 | 323.72 ms | 52.1% bf16 MFU | 1622970 tok/s
step 12458/19560 | loss 3.383776 (+0.56z)| norm 0.2871 (+0.39z)| lr 1.87e-04 | 322.57 ms | 52.3% bf16 MFU | 1623088 tok/s
step 12459/19560 | loss 3.322460 (-0.82z)| norm 0.2932 (+0.69z)| lr 1.87e-04 | 323.91 ms | 52.1% bf16 MFU | 1622864 tok/s
step 12460/19560 | loss 3.370080 (+0.26z)| norm 0.2975 (+0.91z)| lr 1.87e-04 | 322.58 ms | 52.3% bf16 MFU | 1622984 tok/s
step 12461/19560 | loss 3.347875 (-0.23z)| norm 0.2636 (-0.79z)| lr 1.86e-04 | 322.67 ms | 52.3% bf16 MFU | 1623078 tok/s
step 12462/19560 | loss 3.391257 (+0.76z)| norm 0.2857 (+0.32z)| lr 1.86e-04 | 323.74 ms | 52.1% bf16 MFU | 1622898 tok/s
step 12463/19560 | loss 3.351280 (-0.16z)| norm 0.2843 (+0.25z)| lr 1.86e-04 | 323.17 ms | 52.2% bf16 MFU | 1622869 tok/s
step 12464/19560 | loss 3.381248 (+0.53z)| norm 0.2602 (-0.96z)| lr 1.86e-04 | 322.80 ms | 52.3% bf16 MFU | 1622934 tok/s
step 12465/19560 | loss 3.427447 (+1.57z)| norm 0.2747 (-0.24z)| lr 1.86e-04 | 323.37 ms | 52.2% bf16 MFU | 1622855 tok/s
step 12466/19560 | loss 3.299635 (-1.33z)| norm 0.2740 (-0.28z)| lr 1.86e-04 | 323.29 ms | 52.2% bf16 MFU | 1622800 tok/s
step 12467/19560 | loss 3.388371 (+0.67z)| norm 0.2643 (-0.76z)| lr 1.86e-04 | 323.33 ms | 52.2% bf16 MFU | 1622736 tok/s
step 12468/19560 | loss 3.329186 (-0.66z)| norm 0.2719 (-0.38z)| lr 1.86e-04 | 323.27 ms | 52.2% bf16 MFU | 1622690 tok/s
step 12469/19560 | loss 3.397188 (+0.86z)| norm 0.2790 (-0.03z)| lr 1.86e-04 | 324.01 ms | 52.1% bf16 MFU | 1622463 tok/s
step 12470/19560 | loss 3.353432 (-0.12z)| norm 0.2828 (+0.17z)| lr 1.86e-04 | 322.83 ms | 52.3% bf16 MFU | 1622542 tok/s
step 12471/19560 | loss 3.350206 (-0.19z)| norm 0.3225 (+2.13z)| lr 1.86e-04 | 322.68 ms | 52.3% bf16 MFU | 1622656 tok/s
step 12472/19560 | loss 3.425222 (+1.48z)| norm 0.2718 (-0.41z)| lr 1.86e-04 | 324.50 ms | 52.0% bf16 MFU | 1622307 tok/s
step 12473/19560 | loss 3.364862 (+0.14z)| norm 0.2637 (-0.80z)| lr 1.86e-04 | 322.31 ms | 52.4% bf16 MFU | 1622524 tok/s
step 12474/19560 | loss 3.371495 (+0.27z)| norm 0.2711 (-0.44z)| lr 1.86e-04 | 323.00 ms | 52.3% bf16 MFU | 1622557 tok/s
step 12475/19560 | loss 3.312188 (-1.06z)| norm 0.2626 (-0.85z)| lr 1.86e-04 | 323.63 ms | 52.2% bf16 MFU | 1622431 tok/s
step 12476/19560 | loss 3.310949 (-1.08z)| norm 0.2590 (-1.02z)| lr 1.86e-04 | 322.24 ms | 52.4% bf16 MFU | 1622659 tok/s
step 12477/19560 | loss 3.416883 (+1.30z)| norm 0.2915 (+0.60z)| lr 1.86e-04 | 323.20 ms | 52.2% bf16 MFU | 1622634 tok/s
step 12478/19560 | loss 3.346442 (-0.27z)| norm 0.2619 (-0.87z)| lr 1.86e-04 | 323.00 ms | 52.3% bf16 MFU | 1622661 tok/s
step 12479/19560 | loss 3.379366 (+0.46z)| norm 0.3050 (+1.27z)| lr 1.86e-04 | 323.04 ms | 52.2% bf16 MFU | 1622678 tok/s
step 12480/19560 | loss 3.315825 (-0.95z)| norm 0.2943 (+0.73z)| lr 1.86e-04 | 322.47 ms | 52.3% bf16 MFU | 1622837 tok/s
step 12481/19560 | loss 3.365606 (+0.16z)| norm 0.2741 (-0.27z)| lr 1.86e-04 | 322.57 ms | 52.3% bf16 MFU | 1622963 tok/s
step 12482/19560 | loss 3.334651 (-0.53z)| norm 0.2843 (+0.24z)| lr 1.85e-04 | 322.59 ms | 52.3% bf16 MFU | 1623076 tok/s
step 12483/19560 | loss 3.340383 (-0.40z)| norm 0.2721 (-0.37z)| lr 1.85e-04 | 322.67 ms | 52.3% bf16 MFU | 1623165 tok/s
step 12484/19560 | loss 3.415587 (+1.28z)| norm 0.2782 (-0.07z)| lr 1.85e-04 | 323.84 ms | 52.1% bf16 MFU | 1622956 tok/s
step 12485/19560 | loss 3.464124 (+2.31z)| norm 0.2551 (-1.22z)| lr 1.85e-04 | 323.51 ms | 52.2% bf16 MFU | 1622839 tok/s
step 12486/19560 | loss 3.361283 (+0.04z)| norm 0.2963 (+0.84z)| lr 1.85e-04 | 323.28 ms | 52.2% bf16 MFU | 1622786 tok/s
step 12487/19560 | loss 3.430362 (+1.55z)| norm 0.2627 (-0.83z)| lr 1.85e-04 | 322.98 ms | 52.3% bf16 MFU | 1622811 tok/s
step 12488/19560 | loss 3.341773 (-0.39z)| norm 0.2733 (-0.30z)| lr 1.85e-04 | 322.55 ms | 52.3% bf16 MFU | 1622943 tok/s
step 12489/19560 | loss 3.353037 (-0.15z)| norm 0.2648 (-0.72z)| lr 1.85e-04 | 323.49 ms | 52.2% bf16 MFU | 1622833 tok/s
step 12490/19560 | loss 3.385444 (+0.55z)| norm 0.2846 (+0.26z)| lr 1.85e-04 | 323.11 ms | 52.2% bf16 MFU | 1622823 tok/s
step 12491/19560 | loss 3.582665 (+4.46z)| norm 0.2983 (+0.92z)| lr 1.85e-04 | 322.56 ms | 52.3% bf16 MFU | 1622950 tok/s
step 12492/19560 | loss 3.331209 (-0.62z)| norm 0.2824 (+0.12z)| lr 1.85e-04 | 323.09 ms | 52.2% bf16 MFU | 1622940 tok/s
step 12493/19560 | loss 3.250731 (-2.19z)| norm 0.3017 (+1.08z)| lr 1.85e-04 | 322.58 ms | 52.3% bf16 MFU | 1623057 tok/s
step 12494/19560 | loss 3.331849 (-0.58z)| norm 0.2709 (-0.46z)| lr 1.85e-04 | 322.69 ms | 52.3% bf16 MFU | 1623142 tok/s
step 12495/19560 | loss 3.456931 (+1.88z)| norm 0.2845 (+0.22z)| lr 1.85e-04 | 322.91 ms | 52.3% bf16 MFU | 1623166 tok/s
step 12496/19560 | loss 3.364840 (+0.06z)| norm 0.2722 (-0.41z)| lr 1.85e-04 | 323.19 ms | 52.2% bf16 MFU | 1623120 tok/s
step 12497/19560 | loss 3.453805 (+1.79z)| norm 0.3133 (+1.64z)| lr 1.85e-04 | 322.22 ms | 52.4% bf16 MFU | 1623318 tok/s
step 12498/19560 | loss 3.336666 (-0.51z)| norm 0.2794 (-0.06z)| lr 1.85e-04 | 323.05 ms | 52.2% bf16 MFU | 1623298 tok/s
step 12499/19560 | loss 3.353638 (-0.17z)| norm 0.2741 (-0.34z)| lr 1.85e-04 | 322.81 ms | 52.3% bf16 MFU | 1623340 tok/s
step 12500/19560 | loss 3.357971 (-0.08z)| norm 0.2813 (+0.03z)| lr 1.85e-04 | 323.10 ms | 52.2% bf16 MFU | 1623308 tok/s
val loss 3.344419
 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Helevaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2990/10042 = 0.297749
step 12501/19560 | loss 3.397574 (+0.68z)| norm 0.2679 (-0.65z)| lr 1.85e-04 | 321.91 ms | 52.4% bf16 MFU | 1623576 tok/s
step 12502/19560 | loss 3.367697 (+0.09z)| norm 0.2781 (-0.14z)| lr 1.85e-04 | 323.47 ms | 52.2% bf16 MFU | 1623439 tok/s
step 12503/19560 | loss 3.376879 (+0.28z)| norm 0.2644 (-0.83z)| lr 1.85e-04 | 322.88 ms | 52.3% bf16 MFU | 1623457 tok/s
step 12504/19560 | loss 3.289445 (-1.44z)| norm 0.2962 (+0.77z)| lr 1.84e-04 | 322.92 ms | 52.3% bf16 MFU | 1623465 tok/s
step 12505/19560 | loss 3.396997 (+0.67z)| norm 0.2762 (-0.25z)| lr 1.84e-04 | 322.88 ms | 52.3% bf16 MFU | 1623480 tok/s
step 12506/19560 | loss 3.408532 (+0.88z)| norm 0.2954 (+0.72z)| lr 1.84e-04 | 322.90 ms | 52.3% bf16 MFU | 1623490 tok/s
step 12507/19560 | loss 3.355777 (-0.14z)| norm 0.2665 (-0.74z)| lr 1.84e-04 | 322.91 ms | 52.3% bf16 MFU | 1623497 tok/s
step 12508/19560 | loss 3.351036 (-0.23z)| norm 0.2676 (-0.67z)| lr 1.84e-04 | 322.51 ms | 52.3% bf16 MFU | 1623606 tok/s
step 12509/19560 | loss 3.361803 (-0.01z)| norm 0.2649 (-0.81z)| lr 1.84e-04 | 323.05 ms | 52.2% bf16 MFU | 1623571 tok/s
step 12510/19560 | loss 3.328659 (-0.68z)| norm 0.2580 (-1.15z)| lr 1.84e-04 | 322.70 ms | 52.3% bf16 MFU | 1623628 tok/s
step 12511/19560 | loss 3.377151 (+0.30z)| norm 0.2639 (-0.83z)| lr 1.84e-04 | 322.31 ms | 52.4% bf16 MFU | 1623779 tok/s
step 12512/19560 | loss 3.349317 (-0.26z)| norm 0.2472 (-1.67z)| lr 1.84e-04 | 323.09 ms | 52.2% bf16 MFU | 1623727 tok/s
step 12513/19560 | loss 3.365549 (+0.06z)| norm 0.2535 (-1.33z)| lr 1.84e-04 | 322.53 ms | 52.3% bf16 MFU | 1623818 tok/s
step 12514/19560 | loss 3.303495 (-1.21z)| norm 0.2544 (-1.27z)| lr 1.84e-04 | 323.17 ms | 52.2% bf16 MFU | 1623744 tok/s
step 12515/19560 | loss 3.393181 (+0.62z)| norm 0.2630 (-0.82z)| lr 1.84e-04 | 322.40 ms | 52.3% bf16 MFU | 1623868 tok/s
step 12516/19560 | loss 3.386265 (+0.48z)| norm 0.2706 (-0.42z)| lr 1.84e-04 | 322.73 ms | 52.3% bf16 MFU | 1623902 tok/s
step 12517/19560 | loss 3.359986 (-0.06z)| norm 0.2800 (+0.05z)| lr 1.84e-04 | 323.23 ms | 52.2% bf16 MFU | 1623808 tok/s
step 12518/19560 | loss 3.373072 (+0.23z)| norm 0.2612 (-0.89z)| lr 1.84e-04 | 322.40 ms | 52.3% bf16 MFU | 1623926 tok/s
step 12519/19560 | loss 3.310583 (-1.07z)| norm 0.2576 (-1.06z)| lr 1.84e-04 | 322.82 ms | 52.3% bf16 MFU | 1623936 tok/s
step 12520/19560 | loss 3.263021 (-2.02z)| norm 0.2769 (-0.06z)| lr 1.84e-04 | 322.84 ms | 52.3% bf16 MFU | 1623940 tok/s
step 12521/19560 | loss 3.443741 (+1.70z)| norm 0.2592 (-0.97z)| lr 1.84e-04 | 323.08 ms | 52.2% bf16 MFU | 1623882 tok/s
step 12522/19560 | loss 3.401040 (+0.81z)| norm 0.2806 (+0.16z)| lr 1.84e-04 | 322.89 ms | 52.3% bf16 MFU | 1623875 tok/s
step 12523/19560 | loss 3.321222 (-0.81z)| norm 0.2912 (+0.72z)| lr 1.84e-04 | 322.15 ms | 52.4% bf16 MFU | 1624056 tok/s
step 12524/19560 | loss 3.344141 (-0.34z)| norm 0.2651 (-0.64z)| lr 1.84e-04 | 323.47 ms | 52.2% bf16 MFU | 1623894 tok/s
step 12525/19560 | loss 3.394375 (+0.68z)| norm 0.2827 (+0.29z)| lr 1.84e-04 | 322.26 ms | 52.4% bf16 MFU | 1624045 tok/s
step 12526/19560 | loss 3.321397 (-0.80z)| norm 0.2875 (+0.54z)| lr 1.83e-04 | 322.89 ms | 52.3% bf16 MFU | 1624030 tok/s
step 12527/19560 | loss 3.353487 (-0.15z)| norm 0.2751 (-0.12z)| lr 1.83e-04 | 323.01 ms | 52.2% bf16 MFU | 1623985 tok/s
step 12528/19560 | loss 3.332924 (-0.57z)| norm 0.2860 (+0.45z)| lr 1.83e-04 | 322.58 ms | 52.3% bf16 MFU | 1624050 tok/s
step 12529/19560 | loss 3.335288 (-0.53z)| norm 0.2684 (-0.48z)| lr 1.83e-04 | 323.16 ms | 52.2% bf16 MFU | 1623968 tok/s
step 12530/19560 | loss 3.393558 (+0.65z)| norm 0.2710 (-0.34z)| lr 1.83e-04 | 322.61 ms | 52.3% bf16 MFU | 1624026 tok/s
step 12531/19560 | loss 3.372673 (+0.22z)| norm 0.2922 (+0.77z)| lr 1.83e-04 | 322.76 ms | 52.3% bf16 MFU | 1624044 tok/s
step 12532/19560 | loss 3.314045 (-0.97z)| norm 0.2697 (-0.41z)| lr 1.83e-04 | 322.97 ms | 52.3% bf16 MFU | 1624007 tok/s
step 12533/19560 | loss 3.419054 (+1.17z)| norm 0.2862 (+0.46z)| lr 1.83e-04 | 322.53 ms | 52.3% bf16 MFU | 1624084 tok/s
step 12534/19560 | loss 3.405105 (+0.87z)| norm 0.2717 (-0.31z)| lr 1.83e-04 | 323.48 ms | 52.2% bf16 MFU | 1623919 tok/s
step 12535/19560 | loss 3.493047 (+2.57z)| norm 0.2885 (+0.59z)| lr 1.83e-04 | 322.20 ms | 52.4% bf16 MFU | 1624082 tok/s
step 12536/19560 | loss 3.348921 (-0.29z)| norm 0.2662 (-0.63z)| lr 1.83e-04 | 322.55 ms | 52.3% bf16 MFU | 1624150 tok/s
step 12537/19560 | loss 3.279914 (-1.65z)| norm 0.2851 (+0.40z)| lr 1.83e-04 | 323.07 ms | 52.2% bf16 MFU | 1624085 tok/s
step 12538/19560 | loss 3.319849 (-0.85z)| norm 0.2691 (-0.47z)| lr 1.83e-04 | 322.64 ms | 52.3% bf16 MFU | 1624132 tok/s
step 12539/19560 | loss 3.404449 (+0.81z)| norm 0.3019 (+1.29z)| lr 1.83e-04 | 323.39 ms | 52.2% bf16 MFU | 1623986 tok/s
step 12540/19560 | loss 3.336108 (-0.54z)| norm 0.2787 (+0.03z)| lr 1.83e-04 | 322.82 ms | 52.3% bf16 MFU | 1623991 tok/s
step 12541/19560 | loss 3.360634 (-0.04z)| norm 0.2960 (+0.96z)| lr 1.83e-04 | 322.95 ms | 52.3% bf16 MFU | 1623963 tok/s
step 12542/19560 | loss 3.407924 (+0.99z)| norm 0.2781 (-0.01z)| lr 1.83e-04 | 323.02 ms | 52.2% bf16 MFU | 1623920 tok/s
step 12543/19560 | loss 3.393648 (+0.67z)| norm 0.2919 (+0.73z)| lr 1.83e-04 | 322.47 ms | 52.3% bf16 MFU | 1624017 tok/s
step 12544/19560 | loss 3.381210 (+0.40z)| norm 0.2766 (-0.09z)| lr 1.83e-04 | 322.92 ms | 52.3% bf16 MFU | 1623996 tok/s
step 12545/19560 | loss 3.422563 (+1.26z)| norm 0.2862 (+0.42z)| lr 1.83e-04 | 322.57 ms | 52.3% bf16 MFU | 1624063 tok/s
step 12546/19560 | loss 3.353617 (-0.20z)| norm 0.2889 (+0.57z)| lr 1.83e-04 | 322.35 ms | 52.4% bf16 MFU | 1624182 tok/s
step 12547/19560 | loss 3.403317 (+0.84z)| norm 0.2952 (+0.89z)| lr 1.82e-04 | 323.00 ms | 52.3% bf16 MFU | 1624131 tok/s
step 12548/19560 | loss 3.284798 (-1.66z)| norm 0.2795 (+0.04z)| lr 1.82e-04 | 322.23 ms | 52.4% bf16 MFU | 1624277 tok/s
step 12549/19560 | loss 3.336424 (-0.56z)| norm 0.2931 (+0.77z)| lr 1.82e-04 | 322.86 ms | 52.3% bf16 MFU | 1624257 tok/s
step 12550/19560 | loss 3.373806 (+0.22z)| norm 0.2879 (+0.48z)| lr 1.82e-04 | 323.38 ms | 52.2% bf16 MFU | 1624108 tok/s
step 12551/19560 | loss 3.506761 (+2.90z)| norm 0.3236 (+2.34z)| lr 1.82e-04 | 322.56 ms | 52.3% bf16 MFU | 1624173 tok/s
step 12552/19560 | loss 3.355093 (-0.19z)| norm 0.2702 (-0.48z)| lr 1.82e-04 | 322.99 ms | 52.3% bf16 MFU | 1624126 tok/s
step 12553/19560 | loss 3.362253 (-0.04z)| norm 0.2978 (+0.96z)| lr 1.82e-04 | 322.43 ms | 52.3% bf16 MFU | 1624223 tok/s
step 12554/19560 | loss 3.366033 (+0.04z)| norm 0.2670 (-0.67z)| lr 1.82e-04 | 323.05 ms | 52.2% bf16 MFU | 1624158 tok/s
step 12555/19560 | loss 3.429612 (+1.33z)| norm 0.2865 (+0.37z)| lr 1.82e-04 | 322.45 ms | 52.3% bf16 MFU | 1624248 tok/s
step 12556/19560 | loss 3.377901 (+0.28z)| norm 0.2721 (-0.39z)| lr 1.82e-04 | 322.77 ms | 52.3% bf16 MFU | 1624253 tok/s
step 12557/19560 | loss 3.330944 (-0.67z)| norm 0.2682 (-0.60z)| lr 1.82e-04 | 323.48 ms | 52.2% bf16 MFU | 1624078 tok/s
step 12558/19560 | loss 3.425023 (+1.22z)| norm 0.3125 (+1.71z)| lr 1.82e-04 | 323.19 ms | 52.2% bf16 MFU | 1623987 tok/s
step 12559/19560 | loss 3.383039 (+0.36z)| norm 0.2872 (+0.38z)| lr 1.82e-04 | 322.31 ms | 52.4% bf16 MFU | 1624120 tok/s
step 12560/19560 | loss 3.309494 (-1.11z)| norm 0.2690 (-0.57z)| lr 1.82e-04 | 322.46 ms | 52.3% bf16 MFU | 1624210 tok/s
step 12561/19560 | loss 3.366351 (+0.04z)| norm 0.2783 (-0.09z)| lr 1.82e-04 | 323.00 ms | 52.3% bf16 MFU | 1624159 tok/s
step 12562/19560 | loss 3.365879 (+0.02z)| norm 0.2720 (-0.42z)| lr 1.82e-04 | 323.01 ms | 52.2% bf16 MFU | 1624106 tok/s
step 12563/19560 | loss 3.381361 (+0.33z)| norm 0.2680 (-0.63z)| lr 1.82e-04 | 322.63 ms | 52.3% bf16 MFU | 1624154 tok/s
step 12564/19560 | loss 3.368520 (+0.06z)| norm 0.2845 (+0.24z)| lr 1.82e-04 | 322.78 ms | 52.3% bf16 MFU | 1624161 tok/s
step 12565/19560 | loss 3.353103 (-0.28z)| norm 0.2893 (+0.48z)| lr 1.82e-04 | 322.65 ms | 52.3% bf16 MFU | 1624200 tok/s
step 12566/19560 | loss 3.367679 (+0.02z)| norm 0.2609 (-1.02z)| lr 1.82e-04 | 322.53 ms | 52.3% bf16 MFU | 1624267 tok/s
step 12567/19560 | loss 3.359642 (-0.14z)| norm 0.2921 (+0.63z)| lr 1.82e-04 | 322.76 ms | 52.3% bf16 MFU | 1624272 tok/s
step 12568/19560 | loss 3.335078 (-0.65z)| norm 0.2501 (-1.56z)| lr 1.82e-04 | 322.49 ms | 52.3% bf16 MFU | 1624345 tok/s
step 12569/19560 | loss 3.278957 (-1.79z)| norm 0.2726 (-0.37z)| lr 1.81e-04 | 322.71 ms | 52.3% bf16 MFU | 1624359 tok/s
step 12570/19560 | loss 3.357702 (-0.17z)| norm 0.2692 (-0.55z)| lr 1.81e-04 | 322.70 ms | 52.3% bf16 MFU | 1624375 tok/s
step 12571/19560 | loss 3.369958 (+0.09z)| norm 0.2604 (-1.00z)| lr 1.81e-04 | 322.56 ms | 52.3% bf16 MFU | 1624426 tok/s
step 12572/19560 | loss 3.446034 (+1.66z)| norm 0.2767 (-0.14z)| lr 1.81e-04 | 322.94 ms | 52.3% bf16 MFU | 1624379 tok/s
step 12573/19560 | loss 3.345256 (-0.42z)| norm 0.2592 (-1.05z)| lr 1.81e-04 | 322.71 ms | 52.3% bf16 MFU | 1624392 tok/s
step 12574/19560 | loss 3.401856 (+0.74z)| norm 0.2902 (+0.56z)| lr 1.81e-04 | 322.57 ms | 52.3% bf16 MFU | 1624441 tok/s
step 12575/19560 | loss 3.411320 (+0.92z)| norm 0.2898 (+0.53z)| lr 1.81e-04 | 322.89 ms | 52.3% bf16 MFU | 1624407 tok/s
step 12576/19560 | loss 3.522263 (+3.07z)| norm 0.2632 (-0.85z)| lr 1.81e-04 | 323.05 ms | 52.2% bf16 MFU | 1624333 tok/s
step 12577/19560 | loss 3.366346 (-0.05z)| norm 0.2901 (+0.54z)| lr 1.81e-04 | 322.55 ms | 52.3% bf16 MFU | 1624388 tok/s
step 12578/19560 | loss 3.319671 (-0.99z)| norm 0.3007 (+1.08z)| lr 1.81e-04 | 322.78 ms | 52.3% bf16 MFU | 1624384 tok/s
step 12579/19560 | loss 3.350756 (-0.36z)| norm 0.2881 (+0.41z)| lr 1.81e-04 | 323.00 ms | 52.3% bf16 MFU | 1624323 tok/s
step 12580/19560 | loss 3.326957 (-0.83z)| norm 0.2706 (-0.50z)| lr 1.81e-04 | 322.77 ms | 52.3% bf16 MFU | 1624323 tok/s
step 12581/19560 | loss 3.421091 (+1.04z)| norm 0.2871 (+0.36z)| lr 1.81e-04 | 323.05 ms | 52.2% bf16 MFU | 1624253 tok/s
step 12582/19560 | loss 3.374718 (+0.10z)| norm 0.2696 (-0.57z)| lr 1.81e-04 | 322.75 ms | 52.3% bf16 MFU | 1624263 tok/s
step 12583/19560 | loss 3.369173 (-0.01z)| norm 0.2655 (-0.87z)| lr 1.81e-04 | 323.02 ms | 52.2% bf16 MFU | 1624204 tok/s
step 12584/19560 | loss 3.366000 (-0.07z)| norm 0.2817 (+0.21z)| lr 1.81e-04 | 322.92 ms | 52.3% bf16 MFU | 1624173 tok/s
step 12585/19560 | loss 3.405756 (+0.74z)| norm 0.2578 (-1.39z)| lr 1.81e-04 | 322.63 ms | 52.3% bf16 MFU | 1624217 tok/s
step 12586/19560 | loss 3.402876 (+0.68z)| norm 0.2750 (-0.20z)| lr 1.81e-04 | 322.69 ms | 52.3% bf16 MFU | 1624243 tok/s
step 12587/19560 | loss 3.399749 (+0.60z)| norm 0.2698 (-0.55z)| lr 1.81e-04 | 323.25 ms | 52.2% bf16 MFU | 1624126 tok/s
step 12588/19560 | loss 3.407358 (+0.75z)| norm 0.2761 (-0.10z)| lr 1.81e-04 | 323.57 ms | 52.2% bf16 MFU | 1623936 tok/s
step 12589/19560 | loss 3.342702 (-0.56z)| norm 0.2701 (-0.52z)| lr 1.81e-04 | 323.10 ms | 52.2% bf16 MFU | 1623873 tok/s
step 12590/19560 | loss 3.368216 (-0.04z)| norm 0.2668 (-0.74z)| lr 1.81e-04 | 322.91 ms | 52.3% bf16 MFU | 1623862 tok/s
step 12591/19560 | loss 3.500997 (+2.57z)| norm 0.3306 (+3.50z)| lr 1.80e-04 | 322.99 ms | 52.3% bf16 MFU | 1623830 tok/s
step 12592/19560 | loss 3.354111 (-0.34z)| norm 0.3082 (+1.97z)| lr 1.80e-04 | 323.01 ms | 52.2% bf16 MFU | 1623795 tok/s
step 12593/19560 | loss 3.383279 (+0.25z)| norm 0.2835 (+0.35z)| lr 1.80e-04 | 323.05 ms | 52.2% bf16 MFU | 1623751 tok/s
step 12594/19560 | loss 3.370256 (-0.02z)| norm 0.2984 (+1.30z)| lr 1.80e-04 | 322.99 ms | 52.3% bf16 MFU | 1623726 tok/s
step 12595/19560 | loss 3.355904 (-0.31z)| norm 0.2966 (+1.17z)| lr 1.80e-04 | 322.15 ms | 52.4% bf16 MFU | 1623913 tok/s
step 12596/19560 | loss 3.349019 (-0.45z)| norm 0.2882 (+0.61z)| lr 1.80e-04 | 322.54 ms | 52.3% bf16 MFU | 1623992 tok/s
step 12597/19560 | loss 3.343328 (-0.56z)| norm 0.3014 (+1.44z)| lr 1.80e-04 | 323.49 ms | 52.2% bf16 MFU | 1623829 tok/s
step 12598/19560 | loss 3.362696 (-0.17z)| norm 0.2814 (+0.16z)| lr 1.80e-04 | 322.64 ms | 52.3% bf16 MFU | 1623887 tok/s
step 12599/19560 | loss 3.345431 (-0.51z)| norm 0.2878 (+0.60z)| lr 1.80e-04 | 323.40 ms | 52.2% bf16 MFU | 1623752 tok/s
step 12600/19560 | loss 3.428448 (+1.15z)| norm 0.3359 (+3.58z)| lr 1.80e-04 | 323.14 ms | 52.2% bf16 MFU | 1623688 tok/s
step 12601/19560 | loss 3.329965 (-0.82z)| norm 0.2829 (+0.23z)| lr 1.80e-04 | 322.34 ms | 52.4% bf16 MFU | 1623828 tok/s
step 12602/19560 | loss 3.416776 (+0.91z)| norm 0.3123 (+2.04z)| lr 1.80e-04 | 323.31 ms | 52.2% bf16 MFU | 1623717 tok/s
step 12603/19560 | loss 3.382224 (+0.21z)| norm 0.2980 (+1.12z)| lr 1.80e-04 | 323.62 ms | 52.2% bf16 MFU | 1623536 tok/s
step 12604/19560 | loss 3.358971 (-0.26z)| norm 0.3188 (+2.36z)| lr 1.80e-04 | 322.60 ms | 52.3% bf16 MFU | 1623618 tok/s
step 12605/19560 | loss 3.342363 (-0.59z)| norm 0.3105 (+1.82z)| lr 1.80e-04 | 322.75 ms | 52.3% bf16 MFU | 1623658 tok/s
step 12606/19560 | loss 3.396173 (+0.49z)| norm 0.2877 (+0.42z)| lr 1.80e-04 | 322.96 ms | 52.3% bf16 MFU | 1623643 tok/s
step 12607/19560 | loss 3.471767 (+1.98z)| norm 0.3111 (+1.84z)| lr 1.80e-04 | 323.05 ms | 52.2% bf16 MFU | 1623608 tok/s
step 12608/19560 | loss 3.427908 (+1.09z)| norm 0.3094 (+1.71z)| lr 1.80e-04 | 323.01 ms | 52.2% bf16 MFU | 1623583 tok/s
step 12609/19560 | loss 3.389919 (+0.32z)| norm 0.3161 (+2.06z)| lr 1.80e-04 | 323.49 ms | 52.2% bf16 MFU | 1623441 tok/s
step 12610/19560 | loss 3.375751 (+0.04z)| norm 0.2872 (+0.35z)| lr 1.80e-04 | 322.61 ms | 52.3% bf16 MFU | 1623526 tok/s
step 12611/19560 | loss 3.330059 (-0.87z)| norm 0.3018 (+1.19z)| lr 1.80e-04 | 322.71 ms | 52.3% bf16 MFU | 1623582 tok/s
step 12612/19560 | loss 3.313056 (-1.19z)| norm 0.2801 (-0.08z)| lr 1.80e-04 | 323.01 ms | 52.3% bf16 MFU | 1623561 tok/s
step 12613/19560 | loss 3.291218 (-1.61z)| norm 0.3004 (+1.09z)| lr 1.79e-04 | 322.83 ms | 52.3% bf16 MFU | 1623585 tok/s
step 12614/19560 | loss 3.400788 (+0.57z)| norm 0.3188 (+2.14z)| lr 1.79e-04 | 322.36 ms | 52.4% bf16 MFU | 1623725 tok/s
step 12615/19560 | loss 3.388808 (+0.34z)| norm 0.3280 (+2.59z)| lr 1.79e-04 | 323.49 ms | 52.2% bf16 MFU | 1623574 tok/s
step 12616/19560 | loss 3.364971 (-0.14z)| norm 0.2974 (+0.83z)| lr 1.79e-04 | 323.36 ms | 52.2% bf16 MFU | 1623465 tok/s
step 12617/19560 | loss 3.334997 (-0.74z)| norm 0.3325 (+2.73z)| lr 1.79e-04 | 322.60 ms | 52.3% bf16 MFU | 1623551 tok/s
step 12618/19560 | loss 3.411429 (+0.79z)| norm 0.3029 (+1.08z)| lr 1.79e-04 | 323.57 ms | 52.2% bf16 MFU | 1623390 tok/s
step 12619/19560 | loss 3.399843 (+0.63z)| norm 0.3212 (+2.04z)| lr 1.79e-04 | 323.43 ms | 52.2% bf16 MFU | 1623273 tok/s
step 12620/19560 | loss 3.355922 (-0.32z)| norm 0.2771 (-0.35z)| lr 1.79e-04 | 323.04 ms | 52.2% bf16 MFU | 1623258 tok/s
step 12621/19560 | loss 3.279839 (-1.98z)| norm 0.3054 (+1.19z)| lr 1.79e-04 | 322.78 ms | 52.3% bf16 MFU | 1623311 tok/s
step 12622/19560 | loss 3.402620 (+0.68z)| norm 0.2678 (-0.86z)| lr 1.79e-04 | 323.48 ms | 52.2% bf16 MFU | 1623184 tok/s
step 12623/19560 | loss 3.412094 (+0.90z)| norm 0.2993 (+0.84z)| lr 1.79e-04 | 322.58 ms | 52.3% bf16 MFU | 1623289 tok/s
step 12624/19560 | loss 3.416102 (+0.98z)| norm 0.2955 (+0.63z)| lr 1.79e-04 | 322.96 ms | 52.3% bf16 MFU | 1623293 tok/s
step 12625/19560 | loss 3.367846 (-0.07z)| norm 0.2949 (+0.61z)| lr 1.79e-04 | 322.67 ms | 52.3% bf16 MFU | 1623371 tok/s
step 12626/19560 | loss 3.333675 (-0.83z)| norm 0.2856 (+0.10z)| lr 1.79e-04 | 322.68 ms | 52.3% bf16 MFU | 1623443 tok/s
step 12627/19560 | loss 3.320131 (-1.12z)| norm 0.2826 (-0.07z)| lr 1.79e-04 | 323.56 ms | 52.2% bf16 MFU | 1623290 tok/s
step 12628/19560 | loss 3.378468 (+0.17z)| norm 0.2796 (-0.23z)| lr 1.79e-04 | 323.28 ms | 52.2% bf16 MFU | 1623215 tok/s
step 12629/19560 | loss 3.348752 (-0.48z)| norm 0.2663 (-0.95z)| lr 1.79e-04 | 322.89 ms | 52.3% bf16 MFU | 1623241 tok/s
step 12630/19560 | loss 3.390006 (+0.43z)| norm 0.2824 (-0.08z)| lr 1.79e-04 | 322.87 ms | 52.3% bf16 MFU | 1623270 tok/s
step 12631/19560 | loss 3.322721 (-1.04z)| norm 0.2698 (-0.77z)| lr 1.79e-04 | 323.05 ms | 52.2% bf16 MFU | 1623253 tok/s
step 12632/19560 | loss 3.320790 (-1.10z)| norm 0.2761 (-0.42z)| lr 1.79e-04 | 322.99 ms | 52.3% bf16 MFU | 1623252 tok/s
step 12633/19560 | loss 3.405154 (+0.77z)| norm 0.2541 (-1.60z)| lr 1.79e-04 | 322.92 ms | 52.3% bf16 MFU | 1623270 tok/s
step 12634/19560 | loss 3.307287 (-1.38z)| norm 0.2702 (-0.72z)| lr 1.79e-04 | 322.93 ms | 52.3% bf16 MFU | 1623282 tok/s
step 12635/19560 | loss 3.359483 (-0.22z)| norm 0.2654 (-0.98z)| lr 1.78e-04 | 322.84 ms | 52.3% bf16 MFU | 1623319 tok/s
step 12636/19560 | loss 3.353319 (-0.36z)| norm 0.2615 (-1.18z)| lr 1.78e-04 | 323.78 ms | 52.1% bf16 MFU | 1623118 tok/s
step 12637/19560 | loss 3.588689 (+4.42z)| norm 0.2853 (+0.10z)| lr 1.78e-04 | 322.72 ms | 52.3% bf16 MFU | 1623190 tok/s
step 12638/19560 | loss 3.445369 (+1.48z)| norm 0.2551 (-1.54z)| lr 1.78e-04 | 323.25 ms | 52.2% bf16 MFU | 1623128 tok/s
step 12639/19560 | loss 3.336322 (-0.72z)| norm 0.2808 (-0.15z)| lr 1.78e-04 | 322.85 ms | 52.3% bf16 MFU | 1623169 tok/s
step 12640/19560 | loss 3.349047 (-0.46z)| norm 0.2674 (-0.90z)| lr 1.78e-04 | 323.01 ms | 52.3% bf16 MFU | 1623168 tok/s
step 12641/19560 | loss 3.386092 (+0.28z)| norm 0.3002 (+0.90z)| lr 1.78e-04 | 323.64 ms | 52.1% bf16 MFU | 1623007 tok/s
step 12642/19560 | loss 3.388842 (+0.32z)| norm 0.2933 (+0.50z)| lr 1.78e-04 | 322.88 ms | 52.3% bf16 MFU | 1623047 tok/s
step 12643/19560 | loss 3.336862 (-0.72z)| norm 0.2713 (-0.74z)| lr 1.78e-04 | 323.86 ms | 52.1% bf16 MFU | 1622838 tok/s
step 12644/19560 | loss 3.364201 (-0.16z)| norm 0.2753 (-0.52z)| lr 1.78e-04 | 323.28 ms | 52.2% bf16 MFU | 1622783 tok/s
step 12645/19560 | loss 3.332624 (-0.80z)| norm 0.2887 (+0.23z)| lr 1.78e-04 | 322.95 ms | 52.3% bf16 MFU | 1622816 tok/s
step 12646/19560 | loss 3.345787 (-0.53z)| norm 0.2792 (-0.31z)| lr 1.78e-04 | 323.38 ms | 52.2% bf16 MFU | 1622740 tok/s
step 12647/19560 | loss 3.388291 (+0.32z)| norm 0.3195 (+1.94z)| lr 1.78e-04 | 323.71 ms | 52.1% bf16 MFU | 1622584 tok/s
step 12648/19560 | loss 3.415847 (+0.87z)| norm 0.2901 (+0.27z)| lr 1.78e-04 | 323.46 ms | 52.2% bf16 MFU | 1622498 tok/s
step 12649/19560 | loss 3.322598 (-1.04z)| norm 0.2988 (+0.75z)| lr 1.78e-04 | 322.55 ms | 52.3% bf16 MFU | 1622646 tok/s
step 12650/19560 | loss 3.373704 (+0.03z)| norm 0.2837 (-0.11z)| lr 1.78e-04 | 323.22 ms | 52.2% bf16 MFU | 1622617 tok/s
step 12651/19560 | loss 3.396755 (+0.49z)| norm 0.2898 (+0.24z)| lr 1.78e-04 | 323.93 ms | 52.1% bf16 MFU | 1622413 tok/s
step 12652/19560 | loss 3.362393 (-0.23z)| norm 0.2848 (-0.06z)| lr 1.78e-04 | 322.95 ms | 52.3% bf16 MFU | 1622465 tok/s
step 12653/19560 | loss 3.330345 (-0.88z)| norm 0.2693 (-0.93z)| lr 1.78e-04 | 323.58 ms | 52.2% bf16 MFU | 1622355 tok/s
step 12654/19560 | loss 3.374210 (+0.02z)| norm 0.2806 (-0.29z)| lr 1.78e-04 | 323.28 ms | 52.2% bf16 MFU | 1622327 tok/s
step 12655/19560 | loss 3.382211 (+0.19z)| norm 0.2713 (-0.81z)| lr 1.78e-04 | 322.68 ms | 52.3% bf16 MFU | 1622451 tok/s
step 12656/19560 | loss 3.371849 (-0.04z)| norm 0.2872 (+0.09z)| lr 1.78e-04 | 323.16 ms | 52.2% bf16 MFU | 1622447 tok/s
step 12657/19560 | loss 3.339768 (-0.71z)| norm 0.2783 (-0.42z)| lr 1.77e-04 | 323.72 ms | 52.1% bf16 MFU | 1622303 tok/s
step 12658/19560 | loss 3.381070 (+0.16z)| norm 0.2821 (-0.21z)| lr 1.77e-04 | 322.82 ms | 52.3% bf16 MFU | 1622392 tok/s
step 12659/19560 | loss 3.381829 (+0.17z)| norm 0.2524 (-1.87z)| lr 1.77e-04 | 323.29 ms | 52.2% bf16 MFU | 1622359 tok/s
step 12660/19560 | loss 3.364977 (-0.19z)| norm 0.2600 (-1.43z)| lr 1.77e-04 | 322.99 ms | 52.3% bf16 MFU | 1622402 tok/s
step 12661/19560 | loss 3.325653 (-1.01z)| norm 0.2843 (-0.06z)| lr 1.77e-04 | 323.22 ms | 52.2% bf16 MFU | 1622387 tok/s
step 12662/19560 | loss 3.351732 (-0.45z)| norm 0.2642 (-1.18z)| lr 1.77e-04 | 323.06 ms | 52.2% bf16 MFU | 1622413 tok/s
step 12663/19560 | loss 3.372595 (+0.01z)| norm 0.2808 (-0.25z)| lr 1.77e-04 | 322.70 ms | 52.3% bf16 MFU | 1622527 tok/s
step 12664/19560 | loss 3.338334 (-0.73z)| norm 0.2655 (-1.10z)| lr 1.77e-04 | 323.38 ms | 52.2% bf16 MFU | 1622464 tok/s
step 12665/19560 | loss 3.351244 (-0.47z)| norm 0.2910 (+0.32z)| lr 1.77e-04 | 322.89 ms | 52.3% bf16 MFU | 1622528 tok/s
step 12666/19560 | loss 3.411908 (+0.86z)| norm 0.2770 (-0.47z)| lr 1.77e-04 | 323.47 ms | 52.2% bf16 MFU | 1622444 tok/s
step 12667/19560 | loss 3.322962 (-1.09z)| norm 0.2731 (-0.68z)| lr 1.77e-04 | 323.30 ms | 52.2% bf16 MFU | 1622406 tok/s
step 12668/19560 | loss 3.337188 (-0.78z)| norm 0.2548 (-1.67z)| lr 1.77e-04 | 323.40 ms | 52.2% bf16 MFU | 1622344 tok/s
step 12669/19560 | loss 3.321909 (-1.10z)| norm 0.2645 (-1.12z)| lr 1.77e-04 | 323.09 ms | 52.2% bf16 MFU | 1622364 tok/s
step 12670/19560 | loss 3.382960 (+0.24z)| norm 0.2722 (-0.69z)| lr 1.77e-04 | 322.61 ms | 52.3% bf16 MFU | 1622503 tok/s
step 12671/19560 | loss 3.408882 (+0.81z)| norm 0.2690 (-0.85z)| lr 1.77e-04 | 323.53 ms | 52.2% bf16 MFU | 1622405 tok/s
step 12672/19560 | loss 3.373940 (+0.04z)| norm 0.2601 (-1.33z)| lr 1.77e-04 | 322.72 ms | 52.3% bf16 MFU | 1622515 tok/s
step 12673/19560 | loss 3.317054 (-1.19z)| norm 0.2813 (-0.17z)| lr 1.77e-04 | 323.03 ms | 52.2% bf16 MFU | 1622541 tok/s
step 12674/19560 | loss 3.493832 (+2.60z)| norm 0.2640 (-1.10z)| lr 1.77e-04 | 322.77 ms | 52.3% bf16 MFU | 1622632 tok/s
step 12675/19560 | loss 3.385758 (+0.29z)| norm 0.2819 (-0.12z)| lr 1.77e-04 | 322.98 ms | 52.3% bf16 MFU | 1622665 tok/s
step 12676/19560 | loss 3.337005 (-0.77z)| norm 0.2514 (-1.75z)| lr 1.77e-04 | 323.38 ms | 52.2% bf16 MFU | 1622597 tok/s
step 12677/19560 | loss 3.429588 (+1.21z)| norm 0.2730 (-0.57z)| lr 1.77e-04 | 323.00 ms | 52.3% bf16 MFU | 1622627 tok/s
step 12678/19560 | loss 3.352154 (-0.45z)| norm 0.2707 (-0.69z)| lr 1.77e-04 | 322.47 ms | 52.3% bf16 MFU | 1622789 tok/s
step 12679/19560 | loss 3.364305 (-0.17z)| norm 0.2563 (-1.45z)| lr 1.76e-04 | 322.52 ms | 52.3% bf16 MFU | 1622930 tok/s
step 12680/19560 | loss 3.337254 (-0.77z)| norm 0.2705 (-0.68z)| lr 1.76e-04 | 323.14 ms | 52.2% bf16 MFU | 1622908 tok/s
step 12681/19560 | loss 3.365562 (-0.14z)| norm 0.3114 (+1.53z)| lr 1.76e-04 | 323.04 ms | 52.2% bf16 MFU | 1622912 tok/s
step 12682/19560 | loss 3.323144 (-1.07z)| norm 0.3078 (+1.32z)| lr 1.76e-04 | 322.70 ms | 52.3% bf16 MFU | 1623000 tok/s
step 12683/19560 | loss 3.386126 (+0.33z)| norm 0.2749 (-0.45z)| lr 1.76e-04 | 322.55 ms | 52.3% bf16 MFU | 1623124 tok/s
step 12684/19560 | loss 3.320489 (-1.12z)| norm 0.3290 (+2.39z)| lr 1.76e-04 | 323.11 ms | 52.2% bf16 MFU | 1623099 tok/s
step 12685/19560 | loss 3.346438 (-0.54z)| norm 0.3022 (+0.96z)| lr 1.76e-04 | 322.80 ms | 52.3% bf16 MFU | 1623154 tok/s
step 12686/19560 | loss 3.368522 (-0.04z)| norm 0.3023 (+0.97z)| lr 1.76e-04 | 323.03 ms | 52.2% bf16 MFU | 1623147 tok/s
step 12687/19560 | loss 3.337900 (-0.72z)| norm 0.3035 (+1.02z)| lr 1.76e-04 | 322.95 ms | 52.3% bf16 MFU | 1623162 tok/s
step 12688/19560 | loss 3.371196 (+0.01z)| norm 0.2930 (+0.46z)| lr 1.76e-04 | 322.84 ms | 52.3% bf16 MFU | 1623204 tok/s
step 12689/19560 | loss 3.362803 (-0.18z)| norm 0.3279 (+2.24z)| lr 1.76e-04 | 322.76 ms | 52.3% bf16 MFU | 1623264 tok/s
step 12690/19560 | loss 3.320982 (-1.10z)| norm 0.2922 (+0.39z)| lr 1.76e-04 | 322.84 ms | 52.3% bf16 MFU | 1623299 tok/s
step 12691/19560 | loss 3.403399 (+0.74z)| norm 0.2906 (+0.29z)| lr 1.76e-04 | 322.70 ms | 52.3% bf16 MFU | 1623369 tok/s
step 12692/19560 | loss 3.347940 (-0.50z)| norm 0.3043 (+0.99z)| lr 1.76e-04 | 323.20 ms | 52.2% bf16 MFU | 1623308 tok/s
step 12693/19560 | loss 3.273614 (-2.11z)| norm 0.2730 (-0.62z)| lr 1.76e-04 | 322.91 ms | 52.3% bf16 MFU | 1623326 tok/s
step 12694/19560 | loss 3.398502 (+0.63z)| norm 0.3030 (+0.92z)| lr 1.76e-04 | 322.56 ms | 52.3% bf16 MFU | 1623428 tok/s
step 12695/19560 | loss 3.319339 (-1.09z)| norm 0.2818 (-0.18z)| lr 1.76e-04 | 322.92 ms | 52.3% bf16 MFU | 1623437 tok/s
step 12696/19560 | loss 3.336508 (-0.72z)| norm 0.2730 (-0.65z)| lr 1.76e-04 | 322.78 ms | 52.3% bf16 MFU | 1623479 tok/s
step 12697/19560 | loss 3.377958 (+0.17z)| norm 0.2746 (-0.57z)| lr 1.76e-04 | 322.48 ms | 52.3% bf16 MFU | 1623596 tok/s
step 12698/19560 | loss 3.339671 (-0.68z)| norm 0.2581 (-1.42z)| lr 1.76e-04 | 323.01 ms | 52.3% bf16 MFU | 1623574 tok/s
step 12699/19560 | loss 3.340447 (-0.65z)| norm 0.2782 (-0.38z)| lr 1.76e-04 | 322.82 ms | 52.3% bf16 MFU | 1623600 tok/s
step 12700/19560 | loss 3.298674 (-1.55z)| norm 0.2729 (-0.66z)| lr 1.76e-04 | 323.14 ms | 52.2% bf16 MFU | 1623544 tok/s
step 12701/19560 | loss 3.433329 (+1.40z)| norm 0.3332 (+2.44z)| lr 1.75e-04 | 322.71 ms | 52.3% bf16 MFU | 1623598 tok/s
step 12702/19560 | loss 3.419287 (+1.09z)| norm 0.3234 (+1.89z)| lr 1.75e-04 | 322.61 ms | 52.3% bf16 MFU | 1623675 tok/s
step 12703/19560 | loss 3.443274 (+1.60z)| norm 0.2979 (+0.59z)| lr 1.75e-04 | 322.62 ms | 52.3% bf16 MFU | 1623747 tok/s
step 12704/19560 | loss 3.448734 (+1.78z)| norm 0.2771 (-0.48z)| lr 1.75e-04 | 323.32 ms | 52.2% bf16 MFU | 1623640 tok/s
step 12705/19560 | loss 3.350915 (-0.41z)| norm 0.2920 (+0.28z)| lr 1.75e-04 | 322.98 ms | 52.3% bf16 MFU | 1623623 tok/s
step 12706/19560 | loss 3.332241 (-0.83z)| norm 0.2901 (+0.19z)| lr 1.75e-04 | 322.60 ms | 52.3% bf16 MFU | 1623702 tok/s
step 12707/19560 | loss 3.368881 (-0.01z)| norm 0.2698 (-0.84z)| lr 1.75e-04 | 322.88 ms | 52.3% bf16 MFU | 1623706 tok/s
step 12708/19560 | loss 3.274854 (-2.09z)| norm 0.2886 (+0.12z)| lr 1.75e-04 | 323.18 ms | 52.2% bf16 MFU | 1623636 tok/s
step 12709/19560 | loss 3.293517 (-1.65z)| norm 0.2830 (-0.17z)| lr 1.75e-04 | 322.78 ms | 52.3% bf16 MFU | 1623670 tok/s
step 12710/19560 | loss 3.343036 (-0.55z)| norm 0.2758 (-0.54z)| lr 1.75e-04 | 322.52 ms | 52.3% bf16 MFU | 1623766 tok/s
step 12711/19560 | loss 3.330394 (-0.82z)| norm 0.2888 (+0.12z)| lr 1.75e-04 | 322.98 ms | 52.3% bf16 MFU | 1623742 tok/s
step 12712/19560 | loss 3.342337 (-0.55z)| norm 0.2817 (-0.25z)| lr 1.75e-04 | 322.65 ms | 52.3% bf16 MFU | 1623801 tok/s
step 12713/19560 | loss 3.372405 (+0.12z)| norm 0.2719 (-0.76z)| lr 1.75e-04 | 322.87 ms | 52.3% bf16 MFU | 1623804 tok/s
step 12714/19560 | loss 3.364581 (-0.05z)| norm 0.3272 (+2.05z)| lr 1.75e-04 | 322.65 ms | 52.3% bf16 MFU | 1623861 tok/s
step 12715/19560 | loss 3.313624 (-1.16z)| norm 0.2895 (+0.11z)| lr 1.75e-04 | 322.93 ms | 52.3% bf16 MFU | 1623845 tok/s
step 12716/19560 | loss 3.367673 (+0.04z)| norm 0.3399 (+2.60z)| lr 1.75e-04 | 322.43 ms | 52.3% bf16 MFU | 1623954 tok/s
step 12717/19560 | loss 3.438736 (+1.58z)| norm 0.2759 (-0.59z)| lr 1.75e-04 | 322.57 ms | 52.3% bf16 MFU | 1624025 tok/s
step 12718/19560 | loss 3.383949 (+0.38z)| norm 0.2837 (-0.21z)| lr 1.75e-04 | 323.10 ms | 52.2% bf16 MFU | 1623958 tok/s
step 12719/19560 | loss 3.329483 (-0.81z)| norm 0.2742 (-0.68z)| lr 1.75e-04 | 322.85 ms | 52.3% bf16 MFU | 1623958 tok/s
step 12720/19560 | loss 3.400661 (+0.79z)| norm 0.2964 (+0.46z)| lr 1.75e-04 | 322.99 ms | 52.3% bf16 MFU | 1623922 tok/s
step 12721/19560 | loss 3.421280 (+1.24z)| norm 0.3000 (+0.64z)| lr 1.75e-04 | 322.82 ms | 52.3% bf16 MFU | 1623930 tok/s
step 12722/19560 | loss 3.393961 (+0.62z)| norm 0.3120 (+1.24z)| lr 1.75e-04 | 323.29 ms | 52.2% bf16 MFU | 1623819 tok/s
step 12723/19560 | loss 3.333270 (-0.73z)| norm 0.2920 (+0.22z)| lr 1.74e-04 | 322.08 ms | 52.4% bf16 MFU | 1624021 tok/s
step 12724/19560 | loss 3.389295 (+0.51z)| norm 0.3251 (+1.87z)| lr 1.74e-04 | 322.97 ms | 52.3% bf16 MFU | 1623986 tok/s
step 12725/19560 | loss 3.347388 (-0.42z)| norm 0.2934 (+0.28z)| lr 1.74e-04 | 322.72 ms | 52.3% bf16 MFU | 1624016 tok/s
step 12726/19560 | loss 3.311545 (-1.21z)| norm 0.2965 (+0.43z)| lr 1.74e-04 | 322.87 ms | 52.3% bf16 MFU | 1624005 tok/s
step 12727/19560 | loss 3.366612 (+0.01z)| norm 0.3063 (+0.91z)| lr 1.74e-04 | 322.89 ms | 52.3% bf16 MFU | 1623992 tok/s
step 12728/19560 | loss 3.414501 (+1.08z)| norm 0.2663 (-1.08z)| lr 1.74e-04 | 322.67 ms | 52.3% bf16 MFU | 1624034 tok/s
step 12729/19560 | loss 3.358295 (-0.18z)| norm 0.3506 (+3.07z)| lr 1.74e-04 | 323.30 ms | 52.2% bf16 MFU | 1623916 tok/s
step 12730/19560 | loss 3.341164 (-0.55z)| norm 0.3059 (+0.88z)| lr 1.74e-04 | 322.62 ms | 52.3% bf16 MFU | 1623975 tok/s
step 12731/19560 | loss 3.320783 (-0.99z)| norm 0.3059 (+0.88z)| lr 1.74e-04 | 323.12 ms | 52.2% bf16 MFU | 1623905 tok/s
step 12732/19560 | loss 3.342038 (-0.51z)| norm 0.3000 (+0.60z)| lr 1.74e-04 | 323.18 ms | 52.2% bf16 MFU | 1623823 tok/s
step 12733/19560 | loss 3.362096 (-0.07z)| norm 0.2808 (-0.34z)| lr 1.74e-04 | 322.62 ms | 52.3% bf16 MFU | 1623886 tok/s
step 12734/19560 | loss 3.425498 (+1.34z)| norm 0.2832 (-0.22z)| lr 1.74e-04 | 322.73 ms | 52.3% bf16 MFU | 1623919 tok/s
step 12735/19560 | loss 3.376431 (+0.27z)| norm 0.2701 (-0.86z)| lr 1.74e-04 | 322.95 ms | 52.3% bf16 MFU | 1623894 tok/s
step 12736/19560 | loss 3.351938 (-0.28z)| norm 0.2694 (-0.88z)| lr 1.74e-04 | 323.49 ms | 52.2% bf16 MFU | 1623737 tok/s
step 12737/19560 | loss 3.413030 (+1.12z)| norm 0.2639 (-1.14z)| lr 1.74e-04 | 323.16 ms | 52.2% bf16 MFU | 1623669 tok/s
step 12738/19560 | loss 3.358356 (-0.13z)| norm 0.2617 (-1.23z)| lr 1.74e-04 | 322.88 ms | 52.3% bf16 MFU | 1623675 tok/s
step 12739/19560 | loss 3.361107 (-0.07z)| norm 0.2548 (-1.54z)| lr 1.74e-04 | 322.60 ms | 52.3% bf16 MFU | 1623752 tok/s
step 12740/19560 | loss 3.397143 (+0.74z)| norm 0.2510 (-1.70z)| lr 1.74e-04 | 322.59 ms | 52.3% bf16 MFU | 1623827 tok/s
step 12741/19560 | loss 3.445978 (+1.83z)| norm 0.3056 (+0.96z)| lr 1.74e-04 | 323.27 ms | 52.2% bf16 MFU | 1623726 tok/s
step 12742/19560 | loss 3.352269 (-0.31z)| norm 0.2779 (-0.38z)| lr 1.74e-04 | 323.36 ms | 52.2% bf16 MFU | 1623609 tok/s
step 12743/19560 | loss 3.369574 (+0.09z)| norm 0.2545 (-1.52z)| lr 1.74e-04 | 323.04 ms | 52.2% bf16 MFU | 1623578 tok/s
step 12744/19560 | loss 3.419126 (+1.21z)| norm 0.2823 (-0.13z)| lr 1.74e-04 | 323.66 ms | 52.1% bf16 MFU | 1623393 tok/s
step 12745/19560 | loss 3.358156 (-0.19z)| norm 0.2678 (-0.84z)| lr 1.73e-04 | 322.74 ms | 52.3% bf16 MFU | 1623448 tok/s
step 12746/19560 | loss 3.405358 (+0.90z)| norm 0.2773 (-0.35z)| lr 1.73e-04 | 323.10 ms | 52.2% bf16 MFU | 1623409 tok/s
step 12747/19560 | loss 3.324502 (-0.94z)| norm 0.2632 (-1.05z)| lr 1.73e-04 | 322.89 ms | 52.3% bf16 MFU | 1623425 tok/s
step 12748/19560 | loss 3.352284 (-0.31z)| norm 0.2681 (-0.80z)| lr 1.73e-04 | 322.66 ms | 52.3% bf16 MFU | 1623498 tok/s
step 12749/19560 | loss 3.301863 (-1.48z)| norm 0.2602 (-1.18z)| lr 1.73e-04 | 322.79 ms | 52.3% bf16 MFU | 1623536 tok/s
step 12750/19560 | loss 3.362746 (-0.06z)| norm 0.2618 (-1.10z)| lr 1.73e-04 | 322.62 ms | 52.3% bf16 MFU | 1623613 tok/s
val loss 3.338635
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2992/10042 = 0.297949
step 12751/19560 | loss 3.321074 (-1.01z)| norm 0.2685 (-0.74z)| lr 1.73e-04 | 322.84 ms | 52.3% bf16 MFU | 1623631 tok/s
step 12752/19560 | loss 3.349110 (-0.35z)| norm 0.2570 (-1.31z)| lr 1.73e-04 | 323.15 ms | 52.2% bf16 MFU | 1623571 tok/s
step 12753/19560 | loss 3.305482 (-1.35z)| norm 0.2601 (-1.13z)| lr 1.73e-04 | 322.93 ms | 52.3% bf16 MFU | 1623569 tok/s
step 12754/19560 | loss 3.342967 (-0.48z)| norm 0.2657 (-0.84z)| lr 1.73e-04 | 323.26 ms | 52.2% bf16 MFU | 1623483 tok/s
step 12755/19560 | loss 3.402103 (+0.87z)| norm 0.2693 (-0.65z)| lr 1.73e-04 | 322.72 ms | 52.3% bf16 MFU | 1623538 tok/s
step 12756/19560 | loss 3.385600 (+0.49z)| norm 0.2655 (-0.83z)| lr 1.73e-04 | 323.57 ms | 52.2% bf16 MFU | 1623376 tok/s
step 12757/19560 | loss 3.336638 (-0.64z)| norm 0.2663 (-0.79z)| lr 1.73e-04 | 322.97 ms | 52.3% bf16 MFU | 1623375 tok/s
step 12758/19560 | loss 3.372143 (+0.18z)| norm 0.2732 (-0.44z)| lr 1.73e-04 | 323.35 ms | 52.2% bf16 MFU | 1623279 tok/s
step 12759/19560 | loss 3.337937 (-0.61z)| norm 0.2782 (-0.19z)| lr 1.73e-04 | 323.25 ms | 52.2% bf16 MFU | 1623212 tok/s
step 12760/19560 | loss 3.355800 (-0.21z)| norm 0.2639 (-0.91z)| lr 1.73e-04 | 322.93 ms | 52.3% bf16 MFU | 1623229 tok/s
step 12761/19560 | loss 3.392302 (+0.65z)| norm 0.2487 (-1.67z)| lr 1.73e-04 | 322.94 ms | 52.3% bf16 MFU | 1623243 tok/s
step 12762/19560 | loss 3.336947 (-0.65z)| norm 0.2694 (-0.62z)| lr 1.73e-04 | 322.99 ms | 52.3% bf16 MFU | 1623242 tok/s
step 12763/19560 | loss 3.326664 (-0.89z)| norm 0.2609 (-1.05z)| lr 1.73e-04 | 323.62 ms | 52.2% bf16 MFU | 1623085 tok/s
step 12764/19560 | loss 3.351354 (-0.31z)| norm 0.2611 (-1.04z)| lr 1.73e-04 | 323.00 ms | 52.3% bf16 MFU | 1623090 tok/s
step 12765/19560 | loss 3.336415 (-0.69z)| norm 0.2538 (-1.38z)| lr 1.73e-04 | 322.90 ms | 52.3% bf16 MFU | 1623120 tok/s
step 12766/19560 | loss 3.354888 (-0.19z)| norm 0.2543 (-1.36z)| lr 1.73e-04 | 322.97 ms | 52.3% bf16 MFU | 1623130 tok/s
step 12767/19560 | loss 3.320436 (-1.11z)| norm 0.2614 (-0.99z)| lr 1.72e-04 | 323.22 ms | 52.2% bf16 MFU | 1623077 tok/s
step 12768/19560 | loss 3.275859 (-2.24z)| norm 0.2596 (-1.07z)| lr 1.72e-04 | 322.96 ms | 52.3% bf16 MFU | 1623092 tok/s
step 12769/19560 | loss 3.319344 (-1.08z)| norm 0.2856 (+0.21z)| lr 1.72e-04 | 323.03 ms | 52.2% bf16 MFU | 1623089 tok/s
step 12770/19560 | loss 3.345730 (-0.38z)| norm 0.2838 (+0.13z)| lr 1.72e-04 | 322.72 ms | 52.3% bf16 MFU | 1623163 tok/s
step 12771/19560 | loss 3.345754 (-0.38z)| norm 0.2690 (-0.61z)| lr 1.72e-04 | 322.54 ms | 52.3% bf16 MFU | 1623280 tok/s
step 12772/19560 | loss 3.343257 (-0.45z)| norm 0.2751 (-0.30z)| lr 1.72e-04 | 322.92 ms | 52.3% bf16 MFU | 1623296 tok/s
step 12773/19560 | loss 3.359570 (-0.02z)| norm 0.2652 (-0.78z)| lr 1.72e-04 | 323.01 ms | 52.3% bf16 MFU | 1623288 tok/s
step 12774/19560 | loss 3.393395 (+0.86z)| norm 0.2689 (-0.59z)| lr 1.72e-04 | 322.59 ms | 52.3% bf16 MFU | 1623386 tok/s
step 12775/19560 | loss 3.344192 (-0.43z)| norm 0.2571 (-1.16z)| lr 1.72e-04 | 323.14 ms | 52.2% bf16 MFU | 1623342 tok/s
step 12776/19560 | loss 3.414279 (+1.42z)| norm 0.2765 (-0.19z)| lr 1.72e-04 | 323.22 ms | 52.2% bf16 MFU | 1623279 tok/s
step 12777/19560 | loss 3.396237 (+0.93z)| norm 0.2676 (-0.62z)| lr 1.72e-04 | 323.26 ms | 52.2% bf16 MFU | 1623208 tok/s
step 12778/19560 | loss 3.383690 (+0.59z)| norm 0.2562 (-1.18z)| lr 1.72e-04 | 322.37 ms | 52.4% bf16 MFU | 1623366 tok/s
step 12779/19560 | loss 3.359922 (-0.02z)| norm 0.2629 (-0.83z)| lr 1.72e-04 | 322.57 ms | 52.3% bf16 MFU | 1623466 tok/s
step 12780/19560 | loss 3.349768 (-0.29z)| norm 0.2642 (-0.76z)| lr 1.72e-04 | 323.31 ms | 52.2% bf16 MFU | 1623375 tok/s
step 12781/19560 | loss 3.354527 (-0.17z)| norm 0.2769 (-0.13z)| lr 1.72e-04 | 323.03 ms | 52.2% bf16 MFU | 1623358 tok/s
step 12782/19560 | loss 3.334883 (-0.68z)| norm 0.2647 (-0.73z)| lr 1.72e-04 | 322.56 ms | 52.3% bf16 MFU | 1623460 tok/s
step 12783/19560 | loss 3.369936 (+0.25z)| norm 0.3077 (+1.37z)| lr 1.72e-04 | 322.76 ms | 52.3% bf16 MFU | 1623506 tok/s
step 12784/19560 | loss 3.350720 (-0.25z)| norm 0.2604 (-0.93z)| lr 1.72e-04 | 322.56 ms | 52.3% bf16 MFU | 1623601 tok/s
step 12785/19560 | loss 3.309727 (-1.32z)| norm 0.3245 (+2.14z)| lr 1.72e-04 | 323.02 ms | 52.2% bf16 MFU | 1623574 tok/s
step 12786/19560 | loss 3.359282 (-0.02z)| norm 0.2722 (-0.36z)| lr 1.72e-04 | 323.15 ms | 52.2% bf16 MFU | 1623517 tok/s
step 12787/19560 | loss 3.352383 (-0.19z)| norm 0.3243 (+2.08z)| lr 1.72e-04 | 323.54 ms | 52.2% bf16 MFU | 1623366 tok/s
step 12788/19560 | loss 3.377707 (+0.47z)| norm 0.2956 (+0.71z)| lr 1.72e-04 | 322.68 ms | 52.3% bf16 MFU | 1623437 tok/s
step 12789/19560 | loss 3.348397 (-0.31z)| norm 0.2963 (+0.74z)| lr 1.71e-04 | 323.40 ms | 52.2% bf16 MFU | 1623325 tok/s
step 12790/19560 | loss 3.371620 (+0.30z)| norm 0.2804 (-0.02z)| lr 1.71e-04 | 322.85 ms | 52.3% bf16 MFU | 1623356 tok/s
step 12791/19560 | loss 3.350203 (-0.26z)| norm 0.2823 (+0.07z)| lr 1.71e-04 | 322.93 ms | 52.3% bf16 MFU | 1623365 tok/s
step 12792/19560 | loss 3.385792 (+0.67z)| norm 0.2916 (+0.50z)| lr 1.71e-04 | 323.55 ms | 52.2% bf16 MFU | 1623217 tok/s
step 12793/19560 | loss 3.309868 (-1.31z)| norm 0.2754 (-0.27z)| lr 1.71e-04 | 322.80 ms | 52.3% bf16 MFU | 1623264 tok/s
step 12794/19560 | loss 3.322360 (-0.97z)| norm 0.2904 (+0.45z)| lr 1.71e-04 | 323.45 ms | 52.2% bf16 MFU | 1623147 tok/s
step 12795/19560 | loss 3.291427 (-1.76z)| norm 0.2796 (-0.07z)| lr 1.71e-04 | 322.90 ms | 52.3% bf16 MFU | 1623174 tok/s
step 12796/19560 | loss 3.322289 (-0.95z)| norm 0.2867 (+0.26z)| lr 1.71e-04 | 323.63 ms | 52.1% bf16 MFU | 1623016 tok/s
step 12797/19560 | loss 3.363019 (+0.10z)| norm 0.2618 (-0.93z)| lr 1.71e-04 | 323.02 ms | 52.2% bf16 MFU | 1623019 tok/s
step 12798/19560 | loss 3.408757 (+1.28z)| norm 0.2756 (-0.28z)| lr 1.71e-04 | 322.49 ms | 52.3% bf16 MFU | 1623155 tok/s
step 12799/19560 | loss 3.404733 (+1.18z)| norm 0.3023 (+0.99z)| lr 1.71e-04 | 323.10 ms | 52.2% bf16 MFU | 1623132 tok/s
step 12800/19560 | loss 3.327591 (-0.82z)| norm 0.2678 (-0.67z)| lr 1.71e-04 | 323.93 ms | 52.1% bf16 MFU | 1622901 tok/s
step 12801/19560 | loss 3.315552 (-1.13z)| norm 0.2868 (+0.24z)| lr 1.71e-04 | 323.03 ms | 52.2% bf16 MFU | 1622908 tok/s
step 12802/19560 | loss 3.332793 (-0.68z)| norm 0.2674 (-0.69z)| lr 1.71e-04 | 323.72 ms | 52.1% bf16 MFU | 1622740 tok/s
step 12803/19560 | loss 3.353339 (-0.11z)| norm 0.3153 (+1.58z)| lr 1.71e-04 | 322.41 ms | 52.3% bf16 MFU | 1622911 tok/s
step 12804/19560 | loss 3.349325 (-0.23z)| norm 0.3005 (+0.86z)| lr 1.71e-04 | 323.45 ms | 52.2% bf16 MFU | 1622812 tok/s
step 12805/19560 | loss 3.344522 (-0.35z)| norm 0.2843 (+0.09z)| lr 1.71e-04 | 323.08 ms | 52.2% bf16 MFU | 1622811 tok/s
step 12806/19560 | loss 3.294454 (-1.70z)| norm 0.2862 (+0.17z)| lr 1.71e-04 | 322.82 ms | 52.3% bf16 MFU | 1622876 tok/s
step 12807/19560 | loss 3.386494 (+0.82z)| norm 0.2891 (+0.30z)| lr 1.71e-04 | 323.17 ms | 52.2% bf16 MFU | 1622848 tok/s
step 12808/19560 | loss 3.404604 (+1.29z)| norm 0.2965 (+0.64z)| lr 1.71e-04 | 322.97 ms | 52.3% bf16 MFU | 1622871 tok/s
step 12809/19560 | loss 3.372216 (+0.41z)| norm 0.2818 (-0.05z)| lr 1.71e-04 | 322.71 ms | 52.3% bf16 MFU | 1622961 tok/s
step 12810/19560 | loss 3.399821 (+1.14z)| norm 0.2841 (+0.07z)| lr 1.71e-04 | 323.32 ms | 52.2% bf16 MFU | 1622892 tok/s
step 12811/19560 | loss 3.378600 (+0.56z)| norm 0.2731 (-0.46z)| lr 1.70e-04 | 323.47 ms | 52.2% bf16 MFU | 1622790 tok/s
step 12812/19560 | loss 3.378435 (+0.55z)| norm 0.3026 (+1.00z)| lr 1.70e-04 | 322.73 ms | 52.3% bf16 MFU | 1622878 tok/s
step 12813/19560 | loss 3.395978 (+1.01z)| norm 0.2753 (-0.35z)| lr 1.70e-04 | 323.09 ms | 52.2% bf16 MFU | 1622872 tok/s
step 12814/19560 | loss 3.304538 (-1.44z)| norm 0.2766 (-0.27z)| lr 1.70e-04 | 323.42 ms | 52.2% bf16 MFU | 1622783 tok/s
step 12815/19560 | loss 3.390567 (+0.86z)| norm 0.3230 (+2.01z)| lr 1.70e-04 | 323.07 ms | 52.2% bf16 MFU | 1622785 tok/s
step 12816/19560 | loss 3.323518 (-0.93z)| norm 0.3027 (+1.01z)| lr 1.70e-04 | 323.32 ms | 52.2% bf16 MFU | 1622724 tok/s
step 12817/19560 | loss 3.309188 (-1.29z)| norm 0.3005 (+0.92z)| lr 1.70e-04 | 324.31 ms | 52.0% bf16 MFU | 1622418 tok/s
step 12818/19560 | loss 3.381931 (+0.63z)| norm 0.2995 (+0.87z)| lr 1.70e-04 | 322.76 ms | 52.3% bf16 MFU | 1622517 tok/s
step 12819/19560 | loss 3.309758 (-1.27z)| norm 0.3135 (+1.55z)| lr 1.70e-04 | 323.13 ms | 52.2% bf16 MFU | 1622519 tok/s
step 12820/19560 | loss 3.343984 (-0.36z)| norm 0.2976 (+0.76z)| lr 1.70e-04 | 323.40 ms | 52.2% bf16 MFU | 1622453 tok/s
step 12821/19560 | loss 3.332915 (-0.68z)| norm 0.2830 (+0.04z)| lr 1.70e-04 | 323.45 ms | 52.2% bf16 MFU | 1622376 tok/s
step 12822/19560 | loss 3.340649 (-0.46z)| norm 0.2844 (+0.11z)| lr 1.70e-04 | 323.13 ms | 52.2% bf16 MFU | 1622383 tok/s
step 12823/19560 | loss 3.321648 (-0.98z)| norm 0.2935 (+0.56z)| lr 1.70e-04 | 323.53 ms | 52.2% bf16 MFU | 1622290 tok/s
step 12824/19560 | loss 3.347668 (-0.27z)| norm 0.3026 (+1.00z)| lr 1.70e-04 | 322.80 ms | 52.3% bf16 MFU | 1622385 tok/s
step 12825/19560 | loss 3.365653 (+0.22z)| norm 0.2832 (+0.03z)| lr 1.70e-04 | 323.81 ms | 52.1% bf16 MFU | 1622222 tok/s
step 12826/19560 | loss 3.398651 (+1.11z)| norm 0.2896 (+0.34z)| lr 1.70e-04 | 322.90 ms | 52.3% bf16 MFU | 1622295 tok/s
step 12827/19560 | loss 3.299766 (-1.56z)| norm 0.2998 (+0.84z)| lr 1.70e-04 | 322.94 ms | 52.3% bf16 MFU | 1622354 tok/s
step 12828/19560 | loss 3.347747 (-0.28z)| norm 0.2754 (-0.38z)| lr 1.70e-04 | 323.84 ms | 52.1% bf16 MFU | 1622185 tok/s
step 12829/19560 | loss 3.320119 (-1.02z)| norm 0.3059 (+1.18z)| lr 1.70e-04 | 322.87 ms | 52.3% bf16 MFU | 1622267 tok/s
step 12830/19560 | loss 3.293153 (-1.74z)| norm 0.2761 (-0.33z)| lr 1.70e-04 | 323.23 ms | 52.2% bf16 MFU | 1622256 tok/s
step 12831/19560 | loss 3.332291 (-0.65z)| norm 0.2759 (-0.33z)| lr 1.70e-04 | 323.37 ms | 52.2% bf16 MFU | 1622208 tok/s
step 12832/19560 | loss 3.362396 (+0.23z)| norm 0.3136 (+1.60z)| lr 1.70e-04 | 323.02 ms | 52.2% bf16 MFU | 1622252 tok/s
step 12833/19560 | loss 3.335119 (-0.56z)| norm 0.2696 (-0.65z)| lr 1.69e-04 | 323.60 ms | 52.2% bf16 MFU | 1622149 tok/s
step 12834/19560 | loss 3.370581 (+0.46z)| norm 0.3272 (+2.24z)| lr 1.69e-04 | 323.24 ms | 52.2% bf16 MFU | 1622140 tok/s
step 12835/19560 | loss 3.311179 (-1.24z)| norm 0.2845 (+0.09z)| lr 1.69e-04 | 323.20 ms | 52.2% bf16 MFU | 1622142 tok/s
step 12836/19560 | loss 3.358840 (+0.11z)| norm 0.3352 (+2.55z)| lr 1.69e-04 | 323.34 ms | 52.2% bf16 MFU | 1622108 tok/s
step 12837/19560 | loss 3.334000 (-0.63z)| norm 0.2627 (-0.99z)| lr 1.69e-04 | 323.14 ms | 52.2% bf16 MFU | 1622126 tok/s
step 12838/19560 | loss 3.362798 (+0.22z)| norm 0.3181 (+1.68z)| lr 1.69e-04 | 323.75 ms | 52.1% bf16 MFU | 1621991 tok/s
step 12839/19560 | loss 3.359040 (+0.10z)| norm 0.2789 (-0.21z)| lr 1.69e-04 | 323.25 ms | 52.2% bf16 MFU | 1621988 tok/s
step 12840/19560 | loss 3.296512 (-1.74z)| norm 0.2741 (-0.44z)| lr 1.69e-04 | 323.16 ms | 52.2% bf16 MFU | 1622009 tok/s
step 12841/19560 | loss 3.317036 (-1.11z)| norm 0.2948 (+0.56z)| lr 1.69e-04 | 323.57 ms | 52.2% bf16 MFU | 1621924 tok/s
step 12842/19560 | loss 3.321392 (-0.97z)| norm 0.2608 (-1.08z)| lr 1.69e-04 | 323.18 ms | 52.2% bf16 MFU | 1621941 tok/s
step 12843/19560 | loss 3.364950 (+0.29z)| norm 0.2925 (+0.47z)| lr 1.69e-04 | 322.84 ms | 52.3% bf16 MFU | 1622043 tok/s
step 12844/19560 | loss 3.404977 (+1.45z)| norm 0.2830 (+0.03z)| lr 1.69e-04 | 322.81 ms | 52.3% bf16 MFU | 1622148 tok/s
step 12845/19560 | loss 3.344563 (-0.30z)| norm 0.2739 (-0.43z)| lr 1.69e-04 | 323.67 ms | 52.1% bf16 MFU | 1622033 tok/s
step 12846/19560 | loss 3.349336 (-0.15z)| norm 0.2694 (-0.65z)| lr 1.69e-04 | 323.20 ms | 52.2% bf16 MFU | 1622041 tok/s
step 12847/19560 | loss 3.346803 (-0.23z)| norm 0.2739 (-0.42z)| lr 1.69e-04 | 323.02 ms | 52.2% bf16 MFU | 1622094 tok/s
step 12848/19560 | loss 3.367169 (+0.40z)| norm 0.2622 (-1.00z)| lr 1.69e-04 | 322.78 ms | 52.3% bf16 MFU | 1622204 tok/s
step 12849/19560 | loss 3.293733 (-1.81z)| norm 0.2605 (-1.06z)| lr 1.69e-04 | 322.62 ms | 52.3% bf16 MFU | 1622349 tok/s
step 12850/19560 | loss 3.344975 (-0.24z)| norm 0.3026 (+1.06z)| lr 1.69e-04 | 323.50 ms | 52.2% bf16 MFU | 1622264 tok/s
step 12851/19560 | loss 3.349749 (-0.10z)| norm 0.2635 (-0.90z)| lr 1.69e-04 | 322.78 ms | 52.3% bf16 MFU | 1622366 tok/s
step 12852/19560 | loss 3.328543 (-0.73z)| norm 0.2795 (-0.08z)| lr 1.69e-04 | 323.76 ms | 52.1% bf16 MFU | 1622216 tok/s
step 12853/19560 | loss 3.366283 (+0.42z)| norm 0.2518 (-1.47z)| lr 1.69e-04 | 322.48 ms | 52.3% bf16 MFU | 1622396 tok/s
step 12854/19560 | loss 3.395098 (+1.29z)| norm 0.2720 (-0.43z)| lr 1.69e-04 | 323.30 ms | 52.2% bf16 MFU | 1622360 tok/s
step 12855/19560 | loss 3.360612 (+0.23z)| norm 0.2711 (-0.47z)| lr 1.68e-04 | 323.33 ms | 52.2% bf16 MFU | 1622318 tok/s
step 12856/19560 | loss 3.382817 (+0.93z)| norm 0.2677 (-0.64z)| lr 1.68e-04 | 322.66 ms | 52.3% bf16 MFU | 1622448 tok/s
step 12857/19560 | loss 3.351530 (-0.04z)| norm 0.2722 (-0.40z)| lr 1.68e-04 | 322.99 ms | 52.3% bf16 MFU | 1622486 tok/s
step 12858/19560 | loss 3.333888 (-0.59z)| norm 0.2888 (+0.51z)| lr 1.68e-04 | 323.23 ms | 52.2% bf16 MFU | 1622464 tok/s
step 12859/19560 | loss 3.367935 (+0.46z)| norm 0.2813 (+0.11z)| lr 1.68e-04 | 323.09 ms | 52.2% bf16 MFU | 1622476 tok/s
step 12860/19560 | loss 3.313126 (-1.23z)| norm 0.2967 (+0.96z)| lr 1.68e-04 | 322.67 ms | 52.3% bf16 MFU | 1622594 tok/s
step 12861/19560 | loss 3.295205 (-1.75z)| norm 0.2974 (+0.99z)| lr 1.68e-04 | 322.78 ms | 52.3% bf16 MFU | 1622678 tok/s
step 12862/19560 | loss 3.340927 (-0.34z)| norm 0.3135 (+1.84z)| lr 1.68e-04 | 323.69 ms | 52.1% bf16 MFU | 1622529 tok/s
step 12863/19560 | loss 3.302392 (-1.51z)| norm 0.2847 (+0.27z)| lr 1.68e-04 | 323.21 ms | 52.2% bf16 MFU | 1622508 tok/s
step 12864/19560 | loss 3.452501 (+3.00z)| norm 0.3205 (+2.15z)| lr 1.68e-04 | 322.75 ms | 52.3% bf16 MFU | 1622604 tok/s
step 12865/19560 | loss 3.337619 (-0.42z)| norm 0.3597 (+3.95z)| lr 1.68e-04 | 323.37 ms | 52.2% bf16 MFU | 1622540 tok/s
step 12866/19560 | loss 3.328727 (-0.68z)| norm 0.2922 (+0.55z)| lr 1.68e-04 | 322.94 ms | 52.3% bf16 MFU | 1622588 tok/s
step 12867/19560 | loss 3.292527 (-1.73z)| norm 0.3250 (+2.15z)| lr 1.68e-04 | 322.73 ms | 52.3% bf16 MFU | 1622685 tok/s
step 12868/19560 | loss 3.363705 (+0.40z)| norm 0.2874 (+0.27z)| lr 1.68e-04 | 323.20 ms | 52.2% bf16 MFU | 1622660 tok/s
step 12869/19560 | loss 3.409559 (+1.82z)| norm 0.3303 (+2.37z)| lr 1.68e-04 | 322.56 ms | 52.3% bf16 MFU | 1622796 tok/s
step 12870/19560 | loss 3.380419 (+0.92z)| norm 0.3114 (+1.42z)| lr 1.68e-04 | 323.66 ms | 52.1% bf16 MFU | 1622650 tok/s
step 12871/19560 | loss 3.315167 (-1.06z)| norm 0.2845 (+0.09z)| lr 1.68e-04 | 322.61 ms | 52.3% bf16 MFU | 1622774 tok/s
step 12872/19560 | loss 3.368020 (+0.57z)| norm 0.2923 (+0.47z)| lr 1.68e-04 | 322.33 ms | 52.4% bf16 MFU | 1622964 tok/s
step 12873/19560 | loss 3.385099 (+1.09z)| norm 0.3030 (+0.98z)| lr 1.68e-04 | 323.79 ms | 52.1% bf16 MFU | 1622778 tok/s
step 12874/19560 | loss 3.365867 (+0.51z)| norm 0.2949 (+0.57z)| lr 1.68e-04 | 322.77 ms | 52.3% bf16 MFU | 1622857 tok/s
step 12875/19560 | loss 3.289527 (-1.83z)| norm 0.2814 (-0.09z)| lr 1.68e-04 | 323.02 ms | 52.2% bf16 MFU | 1622868 tok/s
step 12876/19560 | loss 3.335312 (-0.42z)| norm 0.2791 (-0.21z)| lr 1.68e-04 | 322.75 ms | 52.3% bf16 MFU | 1622948 tok/s
step 12877/19560 | loss 3.378490 (+0.90z)| norm 0.2867 (+0.15z)| lr 1.68e-04 | 322.77 ms | 52.3% bf16 MFU | 1623017 tok/s
step 12878/19560 | loss 3.466913 (+3.44z)| norm 0.3787 (+4.33z)| lr 1.67e-04 | 323.10 ms | 52.2% bf16 MFU | 1623000 tok/s
step 12879/19560 | loss 3.327309 (-0.68z)| norm 0.3220 (+1.69z)| lr 1.67e-04 | 323.19 ms | 52.2% bf16 MFU | 1622960 tok/s
step 12880/19560 | loss 3.317770 (-0.95z)| norm 0.2861 (+0.04z)| lr 1.67e-04 | 322.68 ms | 52.3% bf16 MFU | 1623051 tok/s
step 12881/19560 | loss 3.314723 (-1.05z)| norm 0.2912 (+0.27z)| lr 1.67e-04 | 323.05 ms | 52.2% bf16 MFU | 1623046 tok/s
step 12882/19560 | loss 3.320127 (-0.88z)| norm 0.2923 (+0.31z)| lr 1.67e-04 | 323.10 ms | 52.2% bf16 MFU | 1623027 tok/s
step 12883/19560 | loss 3.434870 (+2.45z)| norm 0.2795 (-0.29z)| lr 1.67e-04 | 322.77 ms | 52.3% bf16 MFU | 1623092 tok/s
step 12884/19560 | loss 3.309380 (-1.17z)| norm 0.3129 (+1.24z)| lr 1.67e-04 | 322.34 ms | 52.4% bf16 MFU | 1623263 tok/s
step 12885/19560 | loss 3.366899 (+0.49z)| norm 0.2576 (-1.30z)| lr 1.67e-04 | 323.03 ms | 52.2% bf16 MFU | 1623252 tok/s
step 12886/19560 | loss 3.338614 (-0.32z)| norm 0.2702 (-0.73z)| lr 1.67e-04 | 322.73 ms | 52.3% bf16 MFU | 1623315 tok/s
step 12887/19560 | loss 3.341298 (-0.24z)| norm 0.2716 (-0.66z)| lr 1.67e-04 | 323.03 ms | 52.2% bf16 MFU | 1623301 tok/s
step 12888/19560 | loss 3.395963 (+1.32z)| norm 0.2639 (-1.01z)| lr 1.67e-04 | 322.81 ms | 52.3% bf16 MFU | 1623341 tok/s
step 12889/19560 | loss 3.347970 (-0.05z)| norm 0.2719 (-0.66z)| lr 1.67e-04 | 322.58 ms | 52.3% bf16 MFU | 1623438 tok/s
step 12890/19560 | loss 3.319777 (-0.86z)| norm 0.2621 (-1.11z)| lr 1.67e-04 | 323.01 ms | 52.2% bf16 MFU | 1623423 tok/s
step 12891/19560 | loss 3.405450 (+1.58z)| norm 0.2598 (-1.21z)| lr 1.67e-04 | 322.41 ms | 52.3% bf16 MFU | 1623560 tok/s
step 12892/19560 | loss 3.337666 (-0.35z)| norm 0.2613 (-1.14z)| lr 1.67e-04 | 323.08 ms | 52.2% bf16 MFU | 1623522 tok/s
step 12893/19560 | loss 3.429973 (+2.22z)| norm 0.2752 (-0.51z)| lr 1.67e-04 | 322.93 ms | 52.3% bf16 MFU | 1623522 tok/s
step 12894/19560 | loss 3.570937 (+5.39z)| norm 0.3430 (+2.57z)| lr 1.67e-04 | 322.84 ms | 52.3% bf16 MFU | 1623546 tok/s
step 12895/19560 | loss 3.289437 (-1.54z)| norm 0.2796 (-0.34z)| lr 1.67e-04 | 323.30 ms | 52.2% bf16 MFU | 1623453 tok/s
step 12896/19560 | loss 3.417961 (+1.59z)| norm 0.3207 (+1.52z)| lr 1.67e-04 | 322.86 ms | 52.3% bf16 MFU | 1623475 tok/s
step 12897/19560 | loss 3.323943 (-0.72z)| norm 0.2790 (-0.39z)| lr 1.67e-04 | 322.98 ms | 52.3% bf16 MFU | 1623465 tok/s
step 12898/19560 | loss 3.400726 (+1.15z)| norm 0.2896 (+0.09z)| lr 1.67e-04 | 322.65 ms | 52.3% bf16 MFU | 1623538 tok/s
step 12899/19560 | loss 3.356946 (+0.07z)| norm 0.2939 (+0.28z)| lr 1.67e-04 | 322.72 ms | 52.3% bf16 MFU | 1623592 tok/s
step 12900/19560 | loss 3.422724 (+1.65z)| norm 0.2925 (+0.22z)| lr 1.66e-04 | 323.06 ms | 52.2% bf16 MFU | 1623556 tok/s
step 12901/19560 | loss 3.356792 (+0.06z)| norm 0.2843 (-0.17z)| lr 1.66e-04 | 322.51 ms | 52.3% bf16 MFU | 1623659 tok/s
step 12902/19560 | loss 3.432036 (+1.85z)| norm 0.2946 (+0.30z)| lr 1.66e-04 | 322.88 ms | 52.3% bf16 MFU | 1623666 tok/s
step 12903/19560 | loss 3.358388 (+0.08z)| norm 0.2741 (-0.66z)| lr 1.66e-04 | 323.19 ms | 52.2% bf16 MFU | 1623595 tok/s
step 12904/19560 | loss 3.350992 (-0.08z)| norm 0.2800 (-0.39z)| lr 1.66e-04 | 323.16 ms | 52.2% bf16 MFU | 1623533 tok/s
step 12905/19560 | loss 3.389081 (+0.84z)| norm 0.2858 (-0.12z)| lr 1.66e-04 | 323.37 ms | 52.2% bf16 MFU | 1623424 tok/s
step 12906/19560 | loss 3.368573 (+0.35z)| norm 0.2935 (+0.22z)| lr 1.66e-04 | 323.28 ms | 52.2% bf16 MFU | 1623343 tok/s
step 12907/19560 | loss 3.388973 (+0.83z)| norm 0.3189 (+1.40z)| lr 1.66e-04 | 321.98 ms | 52.4% bf16 MFU | 1623593 tok/s
step 12908/19560 | loss 3.335765 (-0.45z)| norm 0.2825 (-0.32z)| lr 1.66e-04 | 322.85 ms | 52.3% bf16 MFU | 1623610 tok/s
step 12909/19560 | loss 3.341013 (-0.32z)| norm 0.2636 (-1.21z)| lr 1.66e-04 | 323.17 ms | 52.2% bf16 MFU | 1623545 tok/s
step 12910/19560 | loss 3.385536 (+0.74z)| norm 0.3019 (+0.59z)| lr 1.66e-04 | 322.57 ms | 52.3% bf16 MFU | 1623636 tok/s
step 12911/19560 | loss 3.371932 (+0.42z)| norm 0.2805 (-0.42z)| lr 1.66e-04 | 322.51 ms | 52.3% bf16 MFU | 1623738 tok/s
step 12912/19560 | loss 3.359172 (+0.11z)| norm 0.2764 (-0.63z)| lr 1.66e-04 | 323.37 ms | 52.2% bf16 MFU | 1623618 tok/s
step 12913/19560 | loss 3.351024 (-0.10z)| norm 0.2924 (+0.15z)| lr 1.66e-04 | 323.59 ms | 52.2% bf16 MFU | 1623450 tok/s
step 12914/19560 | loss 3.450909 (+2.26z)| norm 0.2953 (+0.29z)| lr 1.66e-04 | 322.71 ms | 52.3% bf16 MFU | 1623508 tok/s
step 12915/19560 | loss 3.377946 (+0.52z)| norm 0.2898 (+0.03z)| lr 1.66e-04 | 322.85 ms | 52.3% bf16 MFU | 1623531 tok/s
step 12916/19560 | loss 3.269551 (-2.00z)| norm 0.2881 (-0.05z)| lr 1.66e-04 | 322.48 ms | 52.3% bf16 MFU | 1623644 tok/s
step 12917/19560 | loss 3.355820 (+0.01z)| norm 0.2787 (-0.50z)| lr 1.66e-04 | 322.84 ms | 52.3% bf16 MFU | 1623662 tok/s
step 12918/19560 | loss 3.324581 (-0.71z)| norm 0.2848 (-0.21z)| lr 1.66e-04 | 322.97 ms | 52.3% bf16 MFU | 1623646 tok/s
step 12919/19560 | loss 3.326750 (-0.65z)| norm 0.2642 (-1.20z)| lr 1.66e-04 | 322.44 ms | 52.3% bf16 MFU | 1623764 tok/s
step 12920/19560 | loss 3.403472 (+1.13z)| norm 0.2779 (-0.53z)| lr 1.66e-04 | 322.98 ms | 52.3% bf16 MFU | 1623739 tok/s
step 12921/19560 | loss 3.382609 (+0.63z)| norm 0.2725 (-0.79z)| lr 1.66e-04 | 322.61 ms | 52.3% bf16 MFU | 1623810 tok/s
step 12922/19560 | loss 3.362519 (+0.16z)| norm 0.2961 (+0.36z)| lr 1.65e-04 | 323.47 ms | 52.2% bf16 MFU | 1623662 tok/s
step 12923/19560 | loss 3.395610 (+0.92z)| norm 0.2698 (-0.92z)| lr 1.65e-04 | 323.24 ms | 52.2% bf16 MFU | 1623578 tok/s
step 12924/19560 | loss 3.397078 (+0.94z)| norm 0.2760 (-0.61z)| lr 1.65e-04 | 322.15 ms | 52.4% bf16 MFU | 1623772 tok/s
step 12925/19560 | loss 3.371905 (+0.35z)| norm 0.2866 (-0.11z)| lr 1.65e-04 | 323.43 ms | 52.2% bf16 MFU | 1623634 tok/s
step 12926/19560 | loss 3.291753 (-1.51z)| norm 0.2732 (-0.76z)| lr 1.65e-04 | 322.65 ms | 52.3% bf16 MFU | 1623699 tok/s
step 12927/19560 | loss 3.350884 (-0.12z)| norm 0.2956 (+0.34z)| lr 1.65e-04 | 323.20 ms | 52.2% bf16 MFU | 1623623 tok/s
step 12928/19560 | loss 3.361259 (+0.12z)| norm 0.2603 (-1.38z)| lr 1.65e-04 | 323.30 ms | 52.2% bf16 MFU | 1623525 tok/s
step 12929/19560 | loss 3.372807 (+0.38z)| norm 0.2871 (-0.07z)| lr 1.65e-04 | 323.02 ms | 52.2% bf16 MFU | 1623504 tok/s
step 12930/19560 | loss 3.471953 (+2.63z)| norm 0.3018 (+0.63z)| lr 1.65e-04 | 322.81 ms | 52.3% bf16 MFU | 1623535 tok/s
step 12931/19560 | loss 3.389156 (+0.72z)| norm 0.2823 (-0.31z)| lr 1.65e-04 | 322.94 ms | 52.3% bf16 MFU | 1623531 tok/s
step 12932/19560 | loss 3.345228 (-0.29z)| norm 0.2911 (+0.12z)| lr 1.65e-04 | 322.80 ms | 52.3% bf16 MFU | 1623565 tok/s
step 12933/19560 | loss 3.428925 (+1.60z)| norm 0.3108 (+1.08z)| lr 1.65e-04 | 322.88 ms | 52.3% bf16 MFU | 1623575 tok/s
step 12934/19560 | loss 3.320220 (-0.88z)| norm 0.2808 (-0.39z)| lr 1.65e-04 | 322.91 ms | 52.3% bf16 MFU | 1623578 tok/s
step 12935/19560 | loss 3.433278 (+1.68z)| norm 0.2792 (-0.46z)| lr 1.65e-04 | 322.97 ms | 52.3% bf16 MFU | 1623565 tok/s
step 12936/19560 | loss 3.376552 (+0.40z)| norm 0.2712 (-0.84z)| lr 1.65e-04 | 323.47 ms | 52.2% bf16 MFU | 1623429 tok/s
step 12937/19560 | loss 3.340243 (-0.42z)| norm 0.2784 (-0.49z)| lr 1.65e-04 | 322.22 ms | 52.4% bf16 MFU | 1623613 tok/s
step 12938/19560 | loss 3.353826 (-0.10z)| norm 0.2712 (-0.83z)| lr 1.65e-04 | 323.01 ms | 52.2% bf16 MFU | 1623590 tok/s
step 12939/19560 | loss 3.380126 (+0.50z)| norm 0.2771 (-0.55z)| lr 1.65e-04 | 323.90 ms | 52.1% bf16 MFU | 1623344 tok/s
step 12940/19560 | loss 3.374002 (+0.36z)| norm 0.2835 (-0.23z)| lr 1.65e-04 | 322.62 ms | 52.3% bf16 MFU | 1623432 tok/s
step 12941/19560 | loss 3.410602 (+1.19z)| norm 0.2623 (-1.25z)| lr 1.65e-04 | 322.40 ms | 52.3% bf16 MFU | 1623571 tok/s
step 12942/19560 | loss 3.482011 (+2.71z)| norm 0.3257 (+1.78z)| lr 1.65e-04 | 322.74 ms | 52.3% bf16 MFU | 1623616 tok/s
step 12943/19560 | loss 3.361061 (+0.03z)| norm 0.2759 (-0.59z)| lr 1.65e-04 | 322.99 ms | 52.3% bf16 MFU | 1623596 tok/s
step 12944/19560 | loss 3.439446 (+1.74z)| norm 0.2565 (-1.50z)| lr 1.65e-04 | 322.63 ms | 52.3% bf16 MFU | 1623669 tok/s
step 12945/19560 | loss 3.306556 (-1.18z)| norm 0.3689 (+3.67z)| lr 1.64e-04 | 322.86 ms | 52.3% bf16 MFU | 1623680 tok/s
step 12946/19560 | loss 3.397948 (+0.82z)| norm 0.2869 (-0.06z)| lr 1.64e-04 | 323.14 ms | 52.2% bf16 MFU | 1623619 tok/s
step 12947/19560 | loss 3.330782 (-0.66z)| norm 0.2843 (-0.17z)| lr 1.64e-04 | 323.23 ms | 52.2% bf16 MFU | 1623540 tok/s
step 12948/19560 | loss 3.327525 (-0.73z)| norm 0.2981 (+0.46z)| lr 1.64e-04 | 323.02 ms | 52.2% bf16 MFU | 1623518 tok/s
step 12949/19560 | loss 3.329727 (-0.68z)| norm 0.3114 (+1.06z)| lr 1.64e-04 | 322.35 ms | 52.4% bf16 MFU | 1623664 tok/s
step 12950/19560 | loss 3.348587 (-0.26z)| norm 0.3082 (+0.90z)| lr 1.64e-04 | 323.24 ms | 52.2% bf16 MFU | 1623580 tok/s
step 12951/19560 | loss 3.347742 (-0.29z)| norm 0.2831 (-0.24z)| lr 1.64e-04 | 323.63 ms | 52.1% bf16 MFU | 1623402 tok/s
step 12952/19560 | loss 3.362114 (+0.03z)| norm 0.2735 (-0.66z)| lr 1.64e-04 | 322.15 ms | 52.4% bf16 MFU | 1623606 tok/s
step 12953/19560 | loss 3.387497 (+0.58z)| norm 0.2768 (-0.51z)| lr 1.64e-04 | 323.00 ms | 52.3% bf16 MFU | 1623584 tok/s
step 12954/19560 | loss 3.346343 (-0.32z)| norm 0.2691 (-0.85z)| lr 1.64e-04 | 322.46 ms | 52.3% bf16 MFU | 1623700 tok/s
step 12955/19560 | loss 3.364313 (+0.07z)| norm 0.2767 (-0.50z)| lr 1.64e-04 | 322.32 ms | 52.4% bf16 MFU | 1623845 tok/s
step 12956/19560 | loss 3.366176 (+0.11z)| norm 0.2717 (-0.72z)| lr 1.64e-04 | 323.93 ms | 52.1% bf16 MFU | 1623579 tok/s
step 12957/19560 | loss 3.320511 (-0.91z)| norm 0.2722 (-0.69z)| lr 1.64e-04 | 322.96 ms | 52.3% bf16 MFU | 1623568 tok/s
step 12958/19560 | loss 3.315964 (-1.02z)| norm 0.2794 (-0.36z)| lr 1.64e-04 | 323.97 ms | 52.1% bf16 MFU | 1623305 tok/s
step 12959/19560 | loss 3.315796 (-1.02z)| norm 0.2829 (-0.21z)| lr 1.64e-04 | 323.02 ms | 52.2% bf16 MFU | 1623294 tok/s
step 12960/19560 | loss 3.311631 (-1.09z)| norm 0.2981 (+0.49z)| lr 1.64e-04 | 322.90 ms | 52.3% bf16 MFU | 1623313 tok/s
step 12961/19560 | loss 3.345068 (-0.35z)| norm 0.2927 (+0.24z)| lr 1.64e-04 | 323.29 ms | 52.2% bf16 MFU | 1623233 tok/s
step 12962/19560 | loss 3.401308 (+0.89z)| norm 0.2923 (+0.23z)| lr 1.64e-04 | 322.41 ms | 52.3% bf16 MFU | 1623379 tok/s
step 12963/19560 | loss 3.377665 (+0.35z)| norm 0.2748 (-0.57z)| lr 1.64e-04 | 322.78 ms | 52.3% bf16 MFU | 1623425 tok/s
step 12964/19560 | loss 3.290928 (-1.55z)| norm 0.3003 (+0.63z)| lr 1.64e-04 | 322.90 ms | 52.3% bf16 MFU | 1623439 tok/s
step 12965/19560 | loss 3.404953 (+0.95z)| norm 0.2782 (-0.42z)| lr 1.64e-04 | 322.98 ms | 52.3% bf16 MFU | 1623430 tok/s
step 12966/19560 | loss 3.359761 (-0.04z)| norm 0.2914 (+0.22z)| lr 1.64e-04 | 322.77 ms | 52.3% bf16 MFU | 1623475 tok/s
step 12967/19560 | loss 3.326742 (-0.76z)| norm 0.2783 (-0.40z)| lr 1.63e-04 | 323.28 ms | 52.2% bf16 MFU | 1623390 tok/s
step 12968/19560 | loss 3.417374 (+1.21z)| norm 0.2874 (+0.02z)| lr 1.63e-04 | 323.17 ms | 52.2% bf16 MFU | 1623336 tok/s
step 12969/19560 | loss 3.317314 (-0.99z)| norm 0.2706 (-0.77z)| lr 1.63e-04 | 322.36 ms | 52.4% bf16 MFU | 1623490 tok/s
step 12970/19560 | loss 3.382093 (+0.42z)| norm 0.2727 (-0.68z)| lr 1.63e-04 | 323.47 ms | 52.2% bf16 MFU | 1623356 tok/s
step 12971/19560 | loss 3.342679 (-0.44z)| norm 0.2708 (-0.76z)| lr 1.63e-04 | 323.24 ms | 52.2% bf16 MFU | 1623286 tok/s
step 12972/19560 | loss 3.355575 (-0.15z)| norm 0.2582 (-1.34z)| lr 1.63e-04 | 323.18 ms | 52.2% bf16 MFU | 1623237 tok/s
step 12973/19560 | loss 3.303201 (-1.29z)| norm 0.2741 (-0.59z)| lr 1.63e-04 | 322.86 ms | 52.3% bf16 MFU | 1623270 tok/s
step 12974/19560 | loss 3.356440 (-0.12z)| norm 0.2897 (+0.14z)| lr 1.63e-04 | 322.91 ms | 52.3% bf16 MFU | 1623289 tok/s
step 12975/19560 | loss 3.355211 (-0.15z)| norm 0.2639 (-1.07z)| lr 1.63e-04 | 323.26 ms | 52.2% bf16 MFU | 1623217 tok/s
step 12976/19560 | loss 3.382821 (+0.45z)| norm 0.2771 (-0.45z)| lr 1.63e-04 | 322.58 ms | 52.3% bf16 MFU | 1623320 tok/s
step 12977/19560 | loss 3.391998 (+0.64z)| norm 0.2752 (-0.55z)| lr 1.63e-04 | 323.03 ms | 52.2% bf16 MFU | 1623306 tok/s
step 12978/19560 | loss 3.356313 (-0.15z)| norm 0.2557 (-1.46z)| lr 1.63e-04 | 323.30 ms | 52.2% bf16 MFU | 1623224 tok/s
step 12979/19560 | loss 3.486633 (+2.63z)| norm 0.2893 (+0.13z)| lr 1.63e-04 | 322.73 ms | 52.3% bf16 MFU | 1623289 tok/s
step 12980/19560 | loss 3.352143 (-0.26z)| norm 0.2669 (-0.93z)| lr 1.63e-04 | 322.68 ms | 52.3% bf16 MFU | 1623364 tok/s
step 12981/19560 | loss 3.290632 (-1.56z)| norm 0.2579 (-1.37z)| lr 1.63e-04 | 323.29 ms | 52.2% bf16 MFU | 1623282 tok/s
step 12982/19560 | loss 3.449127 (+1.79z)| norm 0.2722 (-0.68z)| lr 1.63e-04 | 322.92 ms | 52.3% bf16 MFU | 1623297 tok/s
step 12983/19560 | loss 3.506631 (+2.88z)| norm 0.2903 (+0.17z)| lr 1.63e-04 | 323.36 ms | 52.2% bf16 MFU | 1623202 tok/s
step 12984/19560 | loss 3.349512 (-0.32z)| norm 0.2873 (+0.02z)| lr 1.63e-04 | 322.12 ms | 52.4% bf16 MFU | 1623423 tok/s
step 12985/19560 | loss 3.324737 (-0.82z)| norm 0.2556 (-1.49z)| lr 1.63e-04 | 323.25 ms | 52.2% bf16 MFU | 1623349 tok/s
step 12986/19560 | loss 3.406124 (+0.83z)| norm 0.2816 (-0.24z)| lr 1.63e-04 | 323.12 ms | 52.2% bf16 MFU | 1623310 tok/s
step 12987/19560 | loss 3.362045 (-0.07z)| norm 0.2638 (-1.08z)| lr 1.63e-04 | 322.54 ms | 52.3% bf16 MFU | 1623419 tok/s
step 12988/19560 | loss 3.432011 (+1.33z)| norm 0.2801 (-0.30z)| lr 1.63e-04 | 323.26 ms | 52.2% bf16 MFU | 1623341 tok/s
step 12989/19560 | loss 3.314664 (-1.06z)| norm 0.2630 (-1.09z)| lr 1.63e-04 | 323.17 ms | 52.2% bf16 MFU | 1623291 tok/s
step 12990/19560 | loss 3.379909 (+0.27z)| norm 0.3016 (+0.74z)| lr 1.62e-04 | 323.32 ms | 52.2% bf16 MFU | 1623206 tok/s
step 12991/19560 | loss 3.288329 (-1.59z)| norm 0.2717 (-0.68z)| lr 1.62e-04 | 322.63 ms | 52.3% bf16 MFU | 1623298 tok/s
step 12992/19560 | loss 3.412222 (+0.94z)| norm 0.2911 (+0.26z)| lr 1.62e-04 | 323.01 ms | 52.3% bf16 MFU | 1623291 tok/s
step 12993/19560 | loss 3.305523 (-1.24z)| norm 0.2599 (-1.26z)| lr 1.62e-04 | 323.27 ms | 52.2% bf16 MFU | 1623219 tok/s
step 12994/19560 | loss 3.333998 (-0.66z)| norm 0.2850 (+0.01z)| lr 1.62e-04 | 322.84 ms | 52.3% bf16 MFU | 1623256 tok/s
step 12995/19560 | loss 3.296555 (-1.42z)| norm 0.2604 (-1.22z)| lr 1.62e-04 | 323.03 ms | 52.2% bf16 MFU | 1623244 tok/s
step 12996/19560 | loss 3.380033 (+0.28z)| norm 0.2795 (-0.25z)| lr 1.62e-04 | 322.77 ms | 52.3% bf16 MFU | 1623298 tok/s
step 12997/19560 | loss 3.355542 (-0.21z)| norm 0.2826 (-0.07z)| lr 1.62e-04 | 323.48 ms | 52.2% bf16 MFU | 1623171 tok/s
step 12998/19560 | loss 3.287052 (-1.59z)| norm 0.2605 (-1.20z)| lr 1.62e-04 | 323.09 ms | 52.2% bf16 MFU | 1623150 tok/s
step 12999/19560 | loss 3.361727 (-0.08z)| norm 0.2781 (-0.28z)| lr 1.62e-04 | 322.95 ms | 52.3% bf16 MFU | 1623164 tok/s
step 13000/19560 | loss 3.427708 (+1.25z)| norm 0.2819 (-0.08z)| lr 1.62e-04 | 322.97 ms | 52.3% bf16 MFU | 1623172 tok/s
val loss 3.333582
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3016/10042 = 0.300339
step 13001/19560 | loss 3.398769 (+0.66z)| norm 0.2868 (+0.19z)| lr 1.62e-04 | 322.14 ms | 52.4% bf16 MFU | 1623388 tok/s
step 13002/19560 | loss 3.416483 (+1.01z)| norm 0.2764 (-0.35z)| lr 1.62e-04 | 322.43 ms | 52.3% bf16 MFU | 1623521 tok/s
step 13003/19560 | loss 3.330786 (-0.73z)| norm 0.2825 (-0.03z)| lr 1.62e-04 | 323.30 ms | 52.2% bf16 MFU | 1623430 tok/s
step 13004/19560 | loss 3.342239 (-0.50z)| norm 0.2759 (-0.38z)| lr 1.62e-04 | 322.30 ms | 52.4% bf16 MFU | 1623592 tok/s
step 13005/19560 | loss 3.331124 (-0.72z)| norm 0.2636 (-1.01z)| lr 1.62e-04 | 322.81 ms | 52.3% bf16 MFU | 1623619 tok/s
step 13006/19560 | loss 3.408017 (+0.86z)| norm 0.2983 (+0.92z)| lr 1.62e-04 | 322.83 ms | 52.3% bf16 MFU | 1623638 tok/s
step 13007/19560 | loss 3.316964 (-1.01z)| norm 0.2888 (+0.40z)| lr 1.62e-04 | 322.56 ms | 52.3% bf16 MFU | 1623725 tok/s
step 13008/19560 | loss 3.353476 (-0.26z)| norm 0.2666 (-0.90z)| lr 1.62e-04 | 322.67 ms | 52.3% bf16 MFU | 1623781 tok/s
step 13009/19560 | loss 3.341581 (-0.51z)| norm 0.2758 (-0.35z)| lr 1.62e-04 | 323.18 ms | 52.2% bf16 MFU | 1623707 tok/s
step 13010/19560 | loss 3.393551 (+0.55z)| norm 0.2748 (-0.41z)| lr 1.62e-04 | 323.20 ms | 52.2% bf16 MFU | 1623630 tok/s
step 13011/19560 | loss 3.381678 (+0.31z)| norm 0.2947 (+0.76z)| lr 1.62e-04 | 323.07 ms | 52.2% bf16 MFU | 1623589 tok/s
step 13012/19560 | loss 3.382652 (+0.32z)| norm 0.2576 (-1.40z)| lr 1.61e-04 | 322.54 ms | 52.3% bf16 MFU | 1623684 tok/s
step 13013/19560 | loss 3.394907 (+0.58z)| norm 0.2921 (+0.62z)| lr 1.61e-04 | 322.15 ms | 52.4% bf16 MFU | 1623874 tok/s
step 13014/19560 | loss 3.365237 (-0.05z)| norm 0.2554 (-1.54z)| lr 1.61e-04 | 323.84 ms | 52.1% bf16 MFU | 1623628 tok/s
step 13015/19560 | loss 3.364198 (-0.08z)| norm 0.2858 (+0.25z)| lr 1.61e-04 | 323.64 ms | 52.1% bf16 MFU | 1623444 tok/s
step 13016/19560 | loss 3.310138 (-1.19z)| norm 0.2722 (-0.56z)| lr 1.61e-04 | 322.55 ms | 52.3% bf16 MFU | 1623546 tok/s
step 13017/19560 | loss 3.331293 (-0.75z)| norm 0.3014 (+1.16z)| lr 1.61e-04 | 323.12 ms | 52.2% bf16 MFU | 1623497 tok/s
step 13018/19560 | loss 3.444736 (+1.59z)| norm 0.2781 (-0.23z)| lr 1.61e-04 | 322.55 ms | 52.3% bf16 MFU | 1623593 tok/s
step 13019/19560 | loss 3.335629 (-0.66z)| norm 0.2932 (+0.65z)| lr 1.61e-04 | 322.73 ms | 52.3% bf16 MFU | 1623640 tok/s
step 13020/19560 | loss 3.360404 (-0.15z)| norm 0.3021 (+1.17z)| lr 1.61e-04 | 323.19 ms | 52.2% bf16 MFU | 1623570 tok/s
step 13021/19560 | loss 3.323184 (-0.91z)| norm 0.2699 (-0.76z)| lr 1.61e-04 | 322.97 ms | 52.3% bf16 MFU | 1623558 tok/s
step 13022/19560 | loss 3.381720 (+0.37z)| norm 0.3063 (+1.50z)| lr 1.61e-04 | 323.12 ms | 52.2% bf16 MFU | 1623510 tok/s
step 13023/19560 | loss 3.296122 (-1.56z)| norm 0.2875 (+0.33z)| lr 1.61e-04 | 323.22 ms | 52.2% bf16 MFU | 1623439 tok/s
step 13024/19560 | loss 3.353704 (-0.25z)| norm 0.2908 (+0.55z)| lr 1.61e-04 | 322.77 ms | 52.3% bf16 MFU | 1623483 tok/s
step 13025/19560 | loss 3.360581 (-0.10z)| norm 0.2982 (+1.01z)| lr 1.61e-04 | 322.98 ms | 52.3% bf16 MFU | 1623472 tok/s
step 13026/19560 | loss 3.360998 (-0.09z)| norm 0.2821 (-0.01z)| lr 1.61e-04 | 323.09 ms | 52.2% bf16 MFU | 1623435 tok/s
step 13027/19560 | loss 3.292133 (-1.63z)| norm 0.2823 (+0.01z)| lr 1.61e-04 | 322.88 ms | 52.3% bf16 MFU | 1623452 tok/s
step 13028/19560 | loss 3.381003 (+0.39z)| norm 0.2987 (+1.05z)| lr 1.61e-04 | 322.06 ms | 52.4% bf16 MFU | 1623675 tok/s
step 13029/19560 | loss 3.424357 (+1.35z)| norm 0.2639 (-1.14z)| lr 1.61e-04 | 323.11 ms | 52.2% bf16 MFU | 1623623 tok/s
step 13030/19560 | loss 3.330118 (-0.76z)| norm 0.2792 (-0.17z)| lr 1.61e-04 | 323.06 ms | 52.2% bf16 MFU | 1623586 tok/s
step 13031/19560 | loss 3.320408 (-0.98z)| norm 0.2681 (-0.87z)| lr 1.61e-04 | 323.08 ms | 52.2% bf16 MFU | 1623544 tok/s
step 13032/19560 | loss 3.299084 (-1.44z)| norm 0.2819 (+0.01z)| lr 1.61e-04 | 323.26 ms | 52.2% bf16 MFU | 1623462 tok/s
step 13033/19560 | loss 3.331971 (-0.69z)| norm 0.2678 (-0.88z)| lr 1.61e-04 | 322.53 ms | 52.3% bf16 MFU | 1623566 tok/s
step 13034/19560 | loss 3.340349 (-0.49z)| norm 0.2699 (-0.74z)| lr 1.61e-04 | 322.73 ms | 52.3% bf16 MFU | 1623614 tok/s
step 13035/19560 | loss 3.342603 (-0.44z)| norm 0.2758 (-0.35z)| lr 1.60e-04 | 323.21 ms | 52.2% bf16 MFU | 1623539 tok/s
step 13036/19560 | loss 3.356776 (-0.12z)| norm 0.2664 (-0.94z)| lr 1.60e-04 | 322.82 ms | 52.3% bf16 MFU | 1623566 tok/s
step 13037/19560 | loss 3.380744 (+0.41z)| norm 0.2542 (-1.71z)| lr 1.60e-04 | 322.80 ms | 52.3% bf16 MFU | 1623596 tok/s
step 13038/19560 | loss 3.391483 (+0.65z)| norm 0.2644 (-1.05z)| lr 1.60e-04 | 322.73 ms | 52.3% bf16 MFU | 1623644 tok/s
step 13039/19560 | loss 3.359743 (-0.06z)| norm 0.2504 (-1.90z)| lr 1.60e-04 | 323.16 ms | 52.2% bf16 MFU | 1623579 tok/s
step 13040/19560 | loss 3.325712 (-0.82z)| norm 0.2706 (-0.62z)| lr 1.60e-04 | 323.31 ms | 52.2% bf16 MFU | 1623481 tok/s
step 13041/19560 | loss 3.305321 (-1.26z)| norm 0.3136 (+2.06z)| lr 1.60e-04 | 322.86 ms | 52.3% bf16 MFU | 1623502 tok/s
step 13042/19560 | loss 3.327654 (-0.75z)| norm 0.2579 (-1.39z)| lr 1.60e-04 | 322.82 ms | 52.3% bf16 MFU | 1623532 tok/s
step 13043/19560 | loss 3.323494 (-0.83z)| norm 0.3105 (+1.84z)| lr 1.60e-04 | 322.59 ms | 52.3% bf16 MFU | 1623617 tok/s
step 13044/19560 | loss 3.356520 (-0.11z)| norm 0.2690 (-0.69z)| lr 1.60e-04 | 323.01 ms | 52.2% bf16 MFU | 1623592 tok/s
step 13045/19560 | loss 3.315086 (-1.04z)| norm 0.2673 (-0.78z)| lr 1.60e-04 | 322.79 ms | 52.3% bf16 MFU | 1623625 tok/s
step 13046/19560 | loss 3.371473 (+0.24z)| norm 0.2655 (-0.88z)| lr 1.60e-04 | 322.50 ms | 52.3% bf16 MFU | 1623729 tok/s
step 13047/19560 | loss 3.280761 (-1.81z)| norm 0.2595 (-1.24z)| lr 1.60e-04 | 322.77 ms | 52.3% bf16 MFU | 1623760 tok/s
step 13048/19560 | loss 3.385883 (+0.57z)| norm 0.2561 (-1.43z)| lr 1.60e-04 | 323.14 ms | 52.2% bf16 MFU | 1623696 tok/s
step 13049/19560 | loss 3.308207 (-1.17z)| norm 0.2716 (-0.49z)| lr 1.60e-04 | 323.66 ms | 52.1% bf16 MFU | 1623504 tok/s
step 13050/19560 | loss 3.378753 (+0.42z)| norm 0.2658 (-0.83z)| lr 1.60e-04 | 323.29 ms | 52.2% bf16 MFU | 1623416 tok/s
step 13051/19560 | loss 3.304804 (-1.23z)| norm 0.2835 (+0.23z)| lr 1.60e-04 | 322.42 ms | 52.3% bf16 MFU | 1623550 tok/s
step 13052/19560 | loss 3.372380 (+0.29z)| norm 0.2592 (-1.22z)| lr 1.60e-04 | 323.07 ms | 52.2% bf16 MFU | 1623513 tok/s
step 13053/19560 | loss 3.356151 (-0.07z)| norm 0.3445 (+3.66z)| lr 1.60e-04 | 323.05 ms | 52.2% bf16 MFU | 1623483 tok/s
step 13054/19560 | loss 3.267982 (-2.04z)| norm 0.2783 (-0.10z)| lr 1.60e-04 | 322.85 ms | 52.3% bf16 MFU | 1623506 tok/s
step 13055/19560 | loss 3.330063 (-0.64z)| norm 0.2739 (-0.34z)| lr 1.60e-04 | 322.86 ms | 52.3% bf16 MFU | 1623525 tok/s
step 13056/19560 | loss 3.377459 (+0.41z)| norm 0.2895 (+0.53z)| lr 1.60e-04 | 322.90 ms | 52.3% bf16 MFU | 1623532 tok/s
step 13057/19560 | loss 3.328577 (-0.67z)| norm 0.2593 (-1.17z)| lr 1.60e-04 | 322.43 ms | 52.3% bf16 MFU | 1623658 tok/s
step 13058/19560 | loss 3.379849 (+0.50z)| norm 0.2801 (+0.02z)| lr 1.59e-04 | 323.07 ms | 52.2% bf16 MFU | 1623617 tok/s
step 13059/19560 | loss 3.292924 (-1.46z)| norm 0.2672 (-0.71z)| lr 1.59e-04 | 322.77 ms | 52.3% bf16 MFU | 1623653 tok/s
step 13060/19560 | loss 3.386051 (+0.65z)| norm 0.2782 (-0.07z)| lr 1.59e-04 | 323.11 ms | 52.2% bf16 MFU | 1623602 tok/s
step 13061/19560 | loss 3.342391 (-0.33z)| norm 0.2851 (+0.33z)| lr 1.59e-04 | 323.13 ms | 52.2% bf16 MFU | 1623548 tok/s
step 13062/19560 | loss 3.397649 (+0.92z)| norm 0.2823 (+0.17z)| lr 1.59e-04 | 323.12 ms | 52.2% bf16 MFU | 1623500 tok/s
step 13063/19560 | loss 3.377798 (+0.48z)| norm 0.3015 (+1.26z)| lr 1.59e-04 | 322.68 ms | 52.3% bf16 MFU | 1623564 tok/s
step 13064/19560 | loss 3.391975 (+0.81z)| norm 0.2592 (-1.16z)| lr 1.59e-04 | 322.89 ms | 52.3% bf16 MFU | 1623574 tok/s
step 13065/19560 | loss 3.376673 (+0.45z)| norm 0.2633 (-0.91z)| lr 1.59e-04 | 322.62 ms | 52.3% bf16 MFU | 1623650 tok/s
step 13066/19560 | loss 3.418906 (+1.40z)| norm 0.2673 (-0.68z)| lr 1.59e-04 | 322.79 ms | 52.3% bf16 MFU | 1623680 tok/s
step 13067/19560 | loss 3.360286 (+0.06z)| norm 0.2666 (-0.72z)| lr 1.59e-04 | 322.91 ms | 52.3% bf16 MFU | 1623678 tok/s
step 13068/19560 | loss 3.330187 (-0.62z)| norm 0.2682 (-0.62z)| lr 1.59e-04 | 323.02 ms | 52.2% bf16 MFU | 1623649 tok/s
step 13069/19560 | loss 3.335124 (-0.50z)| norm 0.2585 (-1.16z)| lr 1.59e-04 | 322.66 ms | 52.3% bf16 MFU | 1623712 tok/s
step 13070/19560 | loss 3.392785 (+0.87z)| norm 0.2569 (-1.26z)| lr 1.59e-04 | 322.91 ms | 52.3% bf16 MFU | 1623707 tok/s
step 13071/19560 | loss 3.404177 (+1.13z)| norm 0.2737 (-0.28z)| lr 1.59e-04 | 323.10 ms | 52.2% bf16 MFU | 1623655 tok/s
step 13072/19560 | loss 3.380085 (+0.58z)| norm 0.2557 (-1.32z)| lr 1.59e-04 | 323.06 ms | 52.2% bf16 MFU | 1623617 tok/s
step 13073/19560 | loss 3.349551 (-0.16z)| norm 0.2678 (-0.65z)| lr 1.59e-04 | 322.83 ms | 52.3% bf16 MFU | 1623638 tok/s
step 13074/19560 | loss 3.324100 (-0.76z)| norm 0.2615 (-1.04z)| lr 1.59e-04 | 322.62 ms | 52.3% bf16 MFU | 1623709 tok/s
step 13075/19560 | loss 3.319758 (-0.87z)| norm 0.2650 (-0.80z)| lr 1.59e-04 | 323.09 ms | 52.2% bf16 MFU | 1623660 tok/s
step 13076/19560 | loss 3.321557 (-0.82z)| norm 0.2564 (-1.34z)| lr 1.59e-04 | 322.93 ms | 52.3% bf16 MFU | 1623653 tok/s
step 13077/19560 | loss 3.325208 (-0.73z)| norm 0.2680 (-0.57z)| lr 1.59e-04 | 323.10 ms | 52.2% bf16 MFU | 1623604 tok/s
step 13078/19560 | loss 3.349223 (-0.15z)| norm 0.3518 (+4.61z)| lr 1.59e-04 | 322.88 ms | 52.3% bf16 MFU | 1623612 tok/s
step 13079/19560 | loss 3.369303 (+0.33z)| norm 0.3045 (+1.67z)| lr 1.59e-04 | 322.68 ms | 52.3% bf16 MFU | 1623672 tok/s
step 13080/19560 | loss 3.390674 (+0.83z)| norm 0.2927 (+0.94z)| lr 1.58e-04 | 322.71 ms | 52.3% bf16 MFU | 1623721 tok/s
step 13081/19560 | loss 3.331457 (-0.58z)| norm 0.2956 (+1.10z)| lr 1.58e-04 | 322.72 ms | 52.3% bf16 MFU | 1623763 tok/s
step 13082/19560 | loss 3.302481 (-1.26z)| norm 0.3017 (+1.44z)| lr 1.58e-04 | 322.73 ms | 52.3% bf16 MFU | 1623803 tok/s
step 13083/19560 | loss 3.319521 (-0.84z)| norm 0.2816 (+0.23z)| lr 1.58e-04 | 322.86 ms | 52.3% bf16 MFU | 1623808 tok/s
step 13084/19560 | loss 3.331134 (-0.56z)| norm 0.3061 (+1.66z)| lr 1.58e-04 | 323.13 ms | 52.2% bf16 MFU | 1623743 tok/s
step 13085/19560 | loss 3.291545 (-1.49z)| norm 0.3076 (+1.72z)| lr 1.58e-04 | 323.29 ms | 52.2% bf16 MFU | 1623643 tok/s
step 13086/19560 | loss 3.293485 (-1.43z)| norm 0.2840 (+0.33z)| lr 1.58e-04 | 322.38 ms | 52.4% bf16 MFU | 1623776 tok/s
step 13087/19560 | loss 3.302801 (-1.20z)| norm 0.2949 (+0.96z)| lr 1.58e-04 | 323.44 ms | 52.2% bf16 MFU | 1623635 tok/s
step 13088/19560 | loss 3.340015 (-0.34z)| norm 0.2811 (+0.17z)| lr 1.58e-04 | 322.72 ms | 52.3% bf16 MFU | 1623683 tok/s
step 13089/19560 | loss 3.332996 (-0.50z)| norm 0.2940 (+0.92z)| lr 1.58e-04 | 322.90 ms | 52.3% bf16 MFU | 1623683 tok/s
step 13090/19560 | loss 3.323113 (-0.72z)| norm 0.2920 (+0.80z)| lr 1.58e-04 | 323.22 ms | 52.2% bf16 MFU | 1623603 tok/s
step 13091/19560 | loss 3.341410 (-0.28z)| norm 0.2667 (-0.68z)| lr 1.58e-04 | 322.85 ms | 52.3% bf16 MFU | 1623619 tok/s
step 13092/19560 | loss 3.325218 (-0.67z)| norm 0.2954 (+1.01z)| lr 1.58e-04 | 323.91 ms | 52.1% bf16 MFU | 1623368 tok/s
step 13093/19560 | loss 3.368968 (+0.38z)| norm 0.2754 (-0.17z)| lr 1.58e-04 | 322.74 ms | 52.3% bf16 MFU | 1623426 tok/s
step 13094/19560 | loss 3.346832 (-0.15z)| norm 0.2790 (+0.05z)| lr 1.58e-04 | 322.92 ms | 52.3% bf16 MFU | 1623433 tok/s
step 13095/19560 | loss 3.302007 (-1.22z)| norm 0.2660 (-0.71z)| lr 1.58e-04 | 322.86 ms | 52.3% bf16 MFU | 1623456 tok/s
step 13096/19560 | loss 3.377635 (+0.60z)| norm 0.2798 (+0.11z)| lr 1.58e-04 | 322.81 ms | 52.3% bf16 MFU | 1623490 tok/s
step 13097/19560 | loss 3.364291 (+0.27z)| norm 0.2665 (-0.67z)| lr 1.58e-04 | 323.89 ms | 52.1% bf16 MFU | 1623251 tok/s
step 13098/19560 | loss 3.287202 (-1.56z)| norm 0.2595 (-1.07z)| lr 1.58e-04 | 323.02 ms | 52.2% bf16 MFU | 1623243 tok/s
step 13099/19560 | loss 3.316705 (-0.84z)| norm 0.2644 (-0.78z)| lr 1.58e-04 | 322.80 ms | 52.3% bf16 MFU | 1623290 tok/s
step 13100/19560 | loss 3.318317 (-0.80z)| norm 0.2838 (+0.35z)| lr 1.58e-04 | 322.76 ms | 52.3% bf16 MFU | 1623345 tok/s
step 13101/19560 | loss 3.338438 (-0.33z)| norm 0.2859 (+0.46z)| lr 1.58e-04 | 322.79 ms | 52.3% bf16 MFU | 1623389 tok/s
step 13102/19560 | loss 3.397157 (+1.07z)| norm 0.2961 (+1.05z)| lr 1.58e-04 | 323.25 ms | 52.2% bf16 MFU | 1623316 tok/s
step 13103/19560 | loss 3.469940 (+2.70z)| norm 0.2931 (+0.87z)| lr 1.57e-04 | 323.11 ms | 52.2% bf16 MFU | 1623282 tok/s
step 13104/19560 | loss 3.433181 (+1.82z)| norm 0.2869 (+0.50z)| lr 1.57e-04 | 322.22 ms | 52.4% bf16 MFU | 1623473 tok/s
step 13105/19560 | loss 3.368700 (+0.35z)| norm 0.2816 (+0.18z)| lr 1.57e-04 | 323.00 ms | 52.3% bf16 MFU | 1623459 tok/s
step 13106/19560 | loss 3.346808 (-0.15z)| norm 0.2661 (-0.73z)| lr 1.57e-04 | 322.77 ms | 52.3% bf16 MFU | 1623503 tok/s
step 13107/19560 | loss 3.436244 (+1.95z)| norm 0.2757 (-0.16z)| lr 1.57e-04 | 323.28 ms | 52.2% bf16 MFU | 1623417 tok/s
step 13108/19560 | loss 3.493541 (+3.14z)| norm 0.2760 (-0.15z)| lr 1.57e-04 | 322.53 ms | 52.3% bf16 MFU | 1623523 tok/s
step 13109/19560 | loss 3.307009 (-1.07z)| norm 0.2850 (+0.37z)| lr 1.57e-04 | 322.77 ms | 52.3% bf16 MFU | 1623565 tok/s
step 13110/19560 | loss 3.390278 (+0.84z)| norm 0.2824 (+0.21z)| lr 1.57e-04 | 323.37 ms | 52.2% bf16 MFU | 1623452 tok/s
step 13111/19560 | loss 3.385828 (+0.79z)| norm 0.2768 (-0.11z)| lr 1.57e-04 | 323.02 ms | 52.2% bf16 MFU | 1623434 tok/s
step 13112/19560 | loss 3.354802 (+0.05z)| norm 0.2977 (+1.12z)| lr 1.57e-04 | 322.42 ms | 52.3% bf16 MFU | 1623566 tok/s
step 13113/19560 | loss 3.360506 (+0.18z)| norm 0.2891 (+0.60z)| lr 1.57e-04 | 322.99 ms | 52.3% bf16 MFU | 1623549 tok/s
step 13114/19560 | loss 3.341045 (-0.28z)| norm 0.2993 (+1.19z)| lr 1.57e-04 | 322.39 ms | 52.4% bf16 MFU | 1623685 tok/s
step 13115/19560 | loss 3.348724 (-0.09z)| norm 0.2988 (+1.15z)| lr 1.57e-04 | 322.53 ms | 52.3% bf16 MFU | 1623778 tok/s
step 13116/19560 | loss 3.330835 (-0.52z)| norm 0.2884 (+0.52z)| lr 1.57e-04 | 323.01 ms | 52.3% bf16 MFU | 1623746 tok/s
step 13117/19560 | loss 3.390482 (+0.94z)| norm 0.2986 (+1.11z)| lr 1.57e-04 | 323.22 ms | 52.2% bf16 MFU | 1623662 tok/s
step 13118/19560 | loss 3.311962 (-0.98z)| norm 0.2670 (-0.74z)| lr 1.57e-04 | 322.73 ms | 52.3% bf16 MFU | 1623705 tok/s
step 13119/19560 | loss 3.288753 (-1.55z)| norm 0.2775 (-0.12z)| lr 1.57e-04 | 322.33 ms | 52.4% bf16 MFU | 1623849 tok/s
step 13120/19560 | loss 3.368214 (+0.41z)| norm 0.2883 (+0.52z)| lr 1.57e-04 | 323.07 ms | 52.2% bf16 MFU | 1623798 tok/s
step 13121/19560 | loss 3.376071 (+0.60z)| norm 0.2753 (-0.26z)| lr 1.57e-04 | 322.92 ms | 52.3% bf16 MFU | 1623788 tok/s
step 13122/19560 | loss 3.343151 (-0.22z)| norm 0.3014 (+1.28z)| lr 1.57e-04 | 322.90 ms | 52.3% bf16 MFU | 1623783 tok/s
step 13123/19560 | loss 3.402382 (+1.23z)| norm 0.2754 (-0.27z)| lr 1.57e-04 | 322.34 ms | 52.4% bf16 MFU | 1623920 tok/s
step 13124/19560 | loss 3.320817 (-0.79z)| norm 0.2640 (-0.94z)| lr 1.57e-04 | 322.85 ms | 52.3% bf16 MFU | 1623921 tok/s
step 13125/19560 | loss 3.327661 (-0.61z)| norm 0.2854 (+0.33z)| lr 1.57e-04 | 323.35 ms | 52.2% bf16 MFU | 1623796 tok/s
step 13126/19560 | loss 3.298225 (-1.35z)| norm 0.2817 (+0.10z)| lr 1.56e-04 | 322.97 ms | 52.3% bf16 MFU | 1623774 tok/s
step 13127/19560 | loss 3.343908 (-0.21z)| norm 0.2779 (-0.12z)| lr 1.56e-04 | 322.47 ms | 52.3% bf16 MFU | 1623877 tok/s
step 13128/19560 | loss 3.341499 (-0.25z)| norm 0.2936 (+0.81z)| lr 1.56e-04 | 323.06 ms | 52.2% bf16 MFU | 1623828 tok/s
step 13129/19560 | loss 3.348983 (-0.06z)| norm 0.2804 (+0.03z)| lr 1.56e-04 | 322.72 ms | 52.3% bf16 MFU | 1623867 tok/s
step 13130/19560 | loss 3.369853 (+0.49z)| norm 0.2687 (-0.67z)| lr 1.56e-04 | 322.43 ms | 52.3% bf16 MFU | 1623977 tok/s
step 13131/19560 | loss 3.331285 (-0.50z)| norm 0.2970 (+1.01z)| lr 1.56e-04 | 322.62 ms | 52.3% bf16 MFU | 1624034 tok/s
step 13132/19560 | loss 3.318481 (-0.82z)| norm 0.2640 (-0.94z)| lr 1.56e-04 | 323.53 ms | 52.2% bf16 MFU | 1623858 tok/s
step 13133/19560 | loss 3.354921 (+0.11z)| norm 0.2884 (+0.49z)| lr 1.56e-04 | 322.12 ms | 52.4% bf16 MFU | 1624046 tok/s
step 13134/19560 | loss 3.310757 (-1.01z)| norm 0.2777 (-0.14z)| lr 1.56e-04 | 322.25 ms | 52.4% bf16 MFU | 1624193 tok/s
step 13135/19560 | loss 3.334502 (-0.40z)| norm 0.2677 (-0.72z)| lr 1.56e-04 | 323.50 ms | 52.2% bf16 MFU | 1624016 tok/s
step 13136/19560 | loss 3.331988 (-0.46z)| norm 0.2831 (+0.19z)| lr 1.56e-04 | 323.19 ms | 52.2% bf16 MFU | 1623927 tok/s
step 13137/19560 | loss 3.313402 (-0.94z)| norm 0.2908 (+0.64z)| lr 1.56e-04 | 323.32 ms | 52.2% bf16 MFU | 1623810 tok/s
step 13138/19560 | loss 3.340302 (-0.23z)| norm 0.2924 (+0.73z)| lr 1.56e-04 | 323.18 ms | 52.2% bf16 MFU | 1623733 tok/s
step 13139/19560 | loss 3.351012 (+0.05z)| norm 0.3172 (+2.15z)| lr 1.56e-04 | 322.90 ms | 52.3% bf16 MFU | 1623730 tok/s
step 13140/19560 | loss 3.342140 (-0.17z)| norm 0.2721 (-0.49z)| lr 1.56e-04 | 323.39 ms | 52.2% bf16 MFU | 1623606 tok/s
step 13141/19560 | loss 3.369702 (+0.55z)| norm 0.3186 (+2.19z)| lr 1.56e-04 | 322.22 ms | 52.4% bf16 MFU | 1623781 tok/s
step 13142/19560 | loss 3.347184 (-0.03z)| norm 0.2562 (-1.42z)| lr 1.56e-04 | 323.29 ms | 52.2% bf16 MFU | 1623678 tok/s
step 13143/19560 | loss 3.302464 (-1.18z)| norm 0.2807 (+0.00z)| lr 1.56e-04 | 323.04 ms | 52.2% bf16 MFU | 1623644 tok/s
step 13144/19560 | loss 3.339898 (-0.22z)| norm 0.2691 (-0.67z)| lr 1.56e-04 | 322.21 ms | 52.4% bf16 MFU | 1623821 tok/s
step 13145/19560 | loss 3.355040 (+0.17z)| norm 0.2925 (+0.69z)| lr 1.56e-04 | 322.70 ms | 52.3% bf16 MFU | 1623865 tok/s
step 13146/19560 | loss 3.284242 (-1.67z)| norm 0.3986 (+5.82z)| lr 1.56e-04 | 323.47 ms | 52.2% bf16 MFU | 1623713 tok/s
step 13147/19560 | loss 3.271193 (-1.97z)| norm 0.3113 (+1.46z)| lr 1.56e-04 | 322.38 ms | 52.4% bf16 MFU | 1623843 tok/s
step 13148/19560 | loss 3.341526 (-0.13z)| norm 0.2968 (+0.75z)| lr 1.56e-04 | 323.04 ms | 52.2% bf16 MFU | 1623800 tok/s
step 13149/19560 | loss 3.348227 (+0.04z)| norm 0.3340 (+2.51z)| lr 1.55e-04 | 323.01 ms | 52.2% bf16 MFU | 1623766 tok/s
step 13150/19560 | loss 3.410889 (+1.66z)| norm 0.3202 (+1.82z)| lr 1.55e-04 | 322.80 ms | 52.3% bf16 MFU | 1623788 tok/s
step 13151/19560 | loss 3.312001 (-0.91z)| norm 0.3283 (+2.15z)| lr 1.55e-04 | 322.95 ms | 52.3% bf16 MFU | 1623769 tok/s
step 13152/19560 | loss 3.342292 (-0.12z)| norm 0.2808 (-0.08z)| lr 1.55e-04 | 322.75 ms | 52.3% bf16 MFU | 1623801 tok/s
step 13153/19560 | loss 3.317799 (-0.75z)| norm 0.2913 (+0.42z)| lr 1.55e-04 | 323.02 ms | 52.2% bf16 MFU | 1623766 tok/s
step 13154/19560 | loss 3.333736 (-0.33z)| norm 0.3016 (+0.89z)| lr 1.55e-04 | 323.24 ms | 52.2% bf16 MFU | 1623676 tok/s
step 13155/19560 | loss 3.390222 (+1.12z)| norm 0.2674 (-0.70z)| lr 1.55e-04 | 322.79 ms | 52.3% bf16 MFU | 1623705 tok/s
step 13156/19560 | loss 3.330747 (-0.42z)| norm 0.2825 (+0.01z)| lr 1.55e-04 | 323.15 ms | 52.2% bf16 MFU | 1623640 tok/s
step 13157/19560 | loss 3.344424 (-0.05z)| norm 0.2996 (+0.80z)| lr 1.55e-04 | 322.77 ms | 52.3% bf16 MFU | 1623676 tok/s
step 13158/19560 | loss 3.283849 (-1.63z)| norm 0.2566 (-1.21z)| lr 1.55e-04 | 322.85 ms | 52.3% bf16 MFU | 1623688 tok/s
step 13159/19560 | loss 3.303838 (-1.10z)| norm 0.2922 (+0.45z)| lr 1.55e-04 | 323.06 ms | 52.2% bf16 MFU | 1623648 tok/s
step 13160/19560 | loss 3.326055 (-0.52z)| norm 0.3155 (+1.51z)| lr 1.55e-04 | 323.32 ms | 52.2% bf16 MFU | 1623544 tok/s
step 13161/19560 | loss 3.402936 (+1.48z)| norm 0.2870 (+0.18z)| lr 1.55e-04 | 322.83 ms | 52.3% bf16 MFU | 1623568 tok/s
step 13162/19560 | loss 3.337548 (-0.23z)| norm 0.2921 (+0.41z)| lr 1.55e-04 | 323.41 ms | 52.2% bf16 MFU | 1623446 tok/s
step 13163/19560 | loss 3.388220 (+1.08z)| norm 0.2640 (-0.88z)| lr 1.55e-04 | 322.98 ms | 52.3% bf16 MFU | 1623439 tok/s
step 13164/19560 | loss 3.379533 (+0.85z)| norm 0.2682 (-0.69z)| lr 1.55e-04 | 323.31 ms | 52.2% bf16 MFU | 1623348 tok/s
step 13165/19560 | loss 3.307349 (-1.01z)| norm 0.2775 (-0.27z)| lr 1.55e-04 | 322.96 ms | 52.3% bf16 MFU | 1623349 tok/s
step 13166/19560 | loss 3.300474 (-1.17z)| norm 0.2779 (-0.26z)| lr 1.55e-04 | 323.72 ms | 52.1% bf16 MFU | 1623159 tok/s
step 13167/19560 | loss 3.337467 (-0.21z)| norm 0.2628 (-0.97z)| lr 1.55e-04 | 322.58 ms | 52.3% bf16 MFU | 1623266 tok/s
step 13168/19560 | loss 3.371635 (+0.67z)| norm 0.2879 (+0.20z)| lr 1.55e-04 | 323.23 ms | 52.2% bf16 MFU | 1623204 tok/s
step 13169/19560 | loss 3.336385 (-0.25z)| norm 0.2612 (-1.04z)| lr 1.55e-04 | 323.14 ms | 52.2% bf16 MFU | 1623168 tok/s
step 13170/19560 | loss 3.338675 (-0.20z)| norm 0.2759 (-0.36z)| lr 1.55e-04 | 323.01 ms | 52.3% bf16 MFU | 1623167 tok/s
step 13171/19560 | loss 3.398988 (+1.35z)| norm 0.2675 (-0.74z)| lr 1.54e-04 | 322.78 ms | 52.3% bf16 MFU | 1623224 tok/s
step 13172/19560 | loss 3.326303 (-0.52z)| norm 0.2772 (-0.28z)| lr 1.54e-04 | 322.85 ms | 52.3% bf16 MFU | 1623258 tok/s
step 13173/19560 | loss 3.272974 (-1.87z)| norm 0.2541 (-1.37z)| lr 1.54e-04 | 322.76 ms | 52.3% bf16 MFU | 1623315 tok/s
step 13174/19560 | loss 3.343777 (-0.06z)| norm 0.3043 (+1.00z)| lr 1.54e-04 | 323.21 ms | 52.2% bf16 MFU | 1623255 tok/s
step 13175/19560 | loss 3.351463 (+0.13z)| norm 0.2630 (-0.97z)| lr 1.54e-04 | 322.91 ms | 52.3% bf16 MFU | 1623273 tok/s
step 13176/19560 | loss 3.406867 (+1.55z)| norm 0.2872 (+0.17z)| lr 1.54e-04 | 323.54 ms | 52.2% bf16 MFU | 1623134 tok/s
step 13177/19560 | loss 3.329792 (-0.44z)| norm 0.2930 (+0.44z)| lr 1.54e-04 | 322.46 ms | 52.3% bf16 MFU | 1623273 tok/s
step 13178/19560 | loss 3.331733 (-0.38z)| norm 0.2613 (-1.07z)| lr 1.54e-04 | 323.33 ms | 52.2% bf16 MFU | 1623186 tok/s
step 13179/19560 | loss 3.372852 (+0.67z)| norm 0.2831 (-0.03z)| lr 1.54e-04 | 323.56 ms | 52.2% bf16 MFU | 1623044 tok/s
step 13180/19560 | loss 3.368658 (+0.56z)| norm 0.2915 (+0.36z)| lr 1.54e-04 | 323.16 ms | 52.2% bf16 MFU | 1623012 tok/s
step 13181/19560 | loss 3.374020 (+0.70z)| norm 0.2688 (-0.73z)| lr 1.54e-04 | 322.61 ms | 52.3% bf16 MFU | 1623118 tok/s
step 13182/19560 | loss 3.343636 (-0.11z)| norm 0.2782 (-0.26z)| lr 1.54e-04 | 322.97 ms | 52.3% bf16 MFU | 1623128 tok/s
step 13183/19560 | loss 3.441850 (+2.41z)| norm 0.2710 (-0.61z)| lr 1.54e-04 | 322.90 ms | 52.3% bf16 MFU | 1623156 tok/s
step 13184/19560 | loss 3.328897 (-0.50z)| norm 0.2772 (-0.30z)| lr 1.54e-04 | 322.74 ms | 52.3% bf16 MFU | 1623224 tok/s
step 13185/19560 | loss 3.315398 (-0.84z)| norm 0.2540 (-1.45z)| lr 1.54e-04 | 322.89 ms | 52.3% bf16 MFU | 1623249 tok/s
step 13186/19560 | loss 3.335391 (-0.32z)| norm 0.2861 (+0.14z)| lr 1.54e-04 | 323.05 ms | 52.2% bf16 MFU | 1623232 tok/s
step 13187/19560 | loss 3.375480 (+0.70z)| norm 0.2813 (-0.10z)| lr 1.54e-04 | 323.01 ms | 52.2% bf16 MFU | 1623226 tok/s
step 13188/19560 | loss 3.360125 (+0.31z)| norm 0.2674 (-0.78z)| lr 1.54e-04 | 322.76 ms | 52.3% bf16 MFU | 1623284 tok/s
step 13189/19560 | loss 3.373705 (+0.66z)| norm 0.2796 (-0.18z)| lr 1.54e-04 | 322.94 ms | 52.3% bf16 MFU | 1623294 tok/s
step 13190/19560 | loss 3.310280 (-0.98z)| norm 0.2672 (-0.78z)| lr 1.54e-04 | 322.96 ms | 52.3% bf16 MFU | 1623299 tok/s
step 13191/19560 | loss 3.338978 (-0.22z)| norm 0.2789 (-0.20z)| lr 1.54e-04 | 323.45 ms | 52.2% bf16 MFU | 1623180 tok/s
step 13192/19560 | loss 3.341116 (-0.16z)| norm 0.2639 (-0.95z)| lr 1.54e-04 | 322.69 ms | 52.3% bf16 MFU | 1623258 tok/s
step 13193/19560 | loss 3.338231 (-0.23z)| norm 0.2770 (-0.30z)| lr 1.54e-04 | 323.20 ms | 52.2% bf16 MFU | 1623204 tok/s
step 13194/19560 | loss 3.310763 (-0.94z)| norm 0.2785 (-0.24z)| lr 1.53e-04 | 322.93 ms | 52.3% bf16 MFU | 1623220 tok/s
step 13195/19560 | loss 3.352880 (+0.19z)| norm 0.2711 (-0.61z)| lr 1.53e-04 | 323.32 ms | 52.2% bf16 MFU | 1623138 tok/s
step 13196/19560 | loss 3.352690 (+0.18z)| norm 0.2627 (-1.02z)| lr 1.53e-04 | 323.22 ms | 52.2% bf16 MFU | 1623086 tok/s
step 13197/19560 | loss 3.395186 (+1.29z)| norm 0.3178 (+1.70z)| lr 1.53e-04 | 323.01 ms | 52.2% bf16 MFU | 1623088 tok/s
step 13198/19560 | loss 3.411505 (+1.71z)| norm 0.2892 (+0.27z)| lr 1.53e-04 | 323.13 ms | 52.2% bf16 MFU | 1623060 tok/s
step 13199/19560 | loss 3.312102 (-0.90z)| norm 0.3176 (+1.65z)| lr 1.53e-04 | 323.48 ms | 52.2% bf16 MFU | 1622947 tok/s
step 13200/19560 | loss 3.268938 (-1.99z)| norm 0.2822 (-0.11z)| lr 1.53e-04 | 323.19 ms | 52.2% bf16 MFU | 1622912 tok/s
step 13201/19560 | loss 3.433969 (+2.26z)| norm 0.2949 (+0.51z)| lr 1.53e-04 | 323.29 ms | 52.2% bf16 MFU | 1622852 tok/s
step 13202/19560 | loss 3.311382 (-0.88z)| norm 0.2922 (+0.37z)| lr 1.53e-04 | 323.09 ms | 52.2% bf16 MFU | 1622846 tok/s
step 13203/19560 | loss 3.308841 (-0.94z)| norm 0.2665 (-0.92z)| lr 1.53e-04 | 323.54 ms | 52.2% bf16 MFU | 1622727 tok/s
step 13204/19560 | loss 3.374510 (+0.73z)| norm 0.2927 (+0.38z)| lr 1.53e-04 | 323.03 ms | 52.2% bf16 MFU | 1622742 tok/s
step 13205/19560 | loss 3.370029 (+0.61z)| norm 0.2924 (+0.36z)| lr 1.53e-04 | 323.12 ms | 52.2% bf16 MFU | 1622733 tok/s
step 13206/19560 | loss 3.313938 (-0.82z)| norm 0.2866 (+0.09z)| lr 1.53e-04 | 322.99 ms | 52.3% bf16 MFU | 1622758 tok/s
step 13207/19560 | loss 3.443592 (+2.42z)| norm 0.2844 (-0.02z)| lr 1.53e-04 | 322.95 ms | 52.3% bf16 MFU | 1622791 tok/s
step 13208/19560 | loss 3.356078 (+0.24z)| norm 0.2893 (+0.25z)| lr 1.53e-04 | 322.33 ms | 52.4% bf16 MFU | 1622979 tok/s
step 13209/19560 | loss 3.360491 (+0.35z)| norm 0.2730 (-0.62z)| lr 1.53e-04 | 322.35 ms | 52.4% bf16 MFU | 1623153 tok/s
step 13210/19560 | loss 3.335787 (-0.28z)| norm 0.2770 (-0.39z)| lr 1.53e-04 | 323.55 ms | 52.2% bf16 MFU | 1623016 tok/s
step 13211/19560 | loss 3.344670 (-0.06z)| norm 0.2603 (-1.27z)| lr 1.53e-04 | 322.51 ms | 52.3% bf16 MFU | 1623148 tok/s
step 13212/19560 | loss 3.355027 (+0.20z)| norm 0.2716 (-0.65z)| lr 1.53e-04 | 323.36 ms | 52.2% bf16 MFU | 1623058 tok/s
step 13213/19560 | loss 3.335066 (-0.32z)| norm 0.2861 (+0.13z)| lr 1.53e-04 | 323.16 ms | 52.2% bf16 MFU | 1623023 tok/s
step 13214/19560 | loss 3.391491 (+1.10z)| norm 0.2745 (-0.49z)| lr 1.53e-04 | 323.01 ms | 52.3% bf16 MFU | 1623030 tok/s
step 13215/19560 | loss 3.270062 (-1.97z)| norm 0.2684 (-0.81z)| lr 1.53e-04 | 323.10 ms | 52.2% bf16 MFU | 1623012 tok/s
step 13216/19560 | loss 3.355533 (+0.18z)| norm 0.2936 (+0.54z)| lr 1.53e-04 | 322.96 ms | 52.3% bf16 MFU | 1623031 tok/s
step 13217/19560 | loss 3.342635 (-0.14z)| norm 0.2656 (-0.95z)| lr 1.52e-04 | 323.55 ms | 52.2% bf16 MFU | 1622901 tok/s
step 13218/19560 | loss 3.332864 (-0.39z)| norm 0.2838 (+0.03z)| lr 1.52e-04 | 323.60 ms | 52.2% bf16 MFU | 1622764 tok/s
step 13219/19560 | loss 3.315835 (-0.81z)| norm 0.2749 (-0.45z)| lr 1.52e-04 | 322.84 ms | 52.3% bf16 MFU | 1622825 tok/s
step 13220/19560 | loss 3.350419 (+0.05z)| norm 0.2670 (-0.86z)| lr 1.52e-04 | 323.35 ms | 52.2% bf16 MFU | 1622754 tok/s
step 13221/19560 | loss 3.369273 (+0.53z)| norm 0.2792 (-0.21z)| lr 1.52e-04 | 322.28 ms | 52.4% bf16 MFU | 1622956 tok/s
step 13222/19560 | loss 3.357223 (+0.22z)| norm 0.2609 (-1.18z)| lr 1.52e-04 | 323.25 ms | 52.2% bf16 MFU | 1622905 tok/s
step 13223/19560 | loss 3.368250 (+0.49z)| norm 0.2764 (-0.35z)| lr 1.52e-04 | 323.16 ms | 52.2% bf16 MFU | 1622880 tok/s
step 13224/19560 | loss 3.366239 (+0.44z)| norm 0.2862 (+0.17z)| lr 1.52e-04 | 322.99 ms | 52.3% bf16 MFU | 1622898 tok/s
step 13225/19560 | loss 3.380651 (+0.80z)| norm 0.2616 (-1.14z)| lr 1.52e-04 | 322.81 ms | 52.3% bf16 MFU | 1622961 tok/s
step 13226/19560 | loss 3.447257 (+2.42z)| norm 0.2851 (+0.10z)| lr 1.52e-04 | 322.56 ms | 52.3% bf16 MFU | 1623084 tok/s
step 13227/19560 | loss 3.351877 (+0.03z)| norm 0.2645 (-1.01z)| lr 1.52e-04 | 323.24 ms | 52.2% bf16 MFU | 1623028 tok/s
step 13228/19560 | loss 3.346725 (-0.10z)| norm 0.2857 (+0.13z)| lr 1.52e-04 | 323.17 ms | 52.2% bf16 MFU | 1622992 tok/s
step 13229/19560 | loss 3.400778 (+1.24z)| norm 0.2636 (-1.04z)| lr 1.52e-04 | 323.02 ms | 52.2% bf16 MFU | 1622998 tok/s
step 13230/19560 | loss 3.398883 (+1.19z)| norm 0.2690 (-0.74z)| lr 1.52e-04 | 322.86 ms | 52.3% bf16 MFU | 1623041 tok/s
step 13231/19560 | loss 3.334824 (-0.40z)| norm 0.2772 (-0.30z)| lr 1.52e-04 | 322.45 ms | 52.3% bf16 MFU | 1623186 tok/s
step 13232/19560 | loss 3.331903 (-0.46z)| norm 0.2747 (-0.43z)| lr 1.52e-04 | 323.50 ms | 52.2% bf16 MFU | 1623059 tok/s
step 13233/19560 | loss 3.392551 (+1.13z)| norm 0.2907 (+0.42z)| lr 1.52e-04 | 322.58 ms | 52.3% bf16 MFU | 1623171 tok/s
step 13234/19560 | loss 3.247423 (-2.58z)| norm 0.2670 (-0.84z)| lr 1.52e-04 | 323.00 ms | 52.3% bf16 MFU | 1623170 tok/s
step 13235/19560 | loss 3.333443 (-0.38z)| norm 0.2811 (-0.09z)| lr 1.52e-04 | 322.99 ms | 52.3% bf16 MFU | 1623172 tok/s
step 13236/19560 | loss 3.336062 (-0.30z)| norm 0.2729 (-0.53z)| lr 1.52e-04 | 323.31 ms | 52.2% bf16 MFU | 1623095 tok/s
step 13237/19560 | loss 3.348660 (+0.04z)| norm 0.2614 (-1.13z)| lr 1.52e-04 | 323.12 ms | 52.2% bf16 MFU | 1623070 tok/s
step 13238/19560 | loss 3.318679 (-0.77z)| norm 0.2850 (+0.13z)| lr 1.52e-04 | 323.08 ms | 52.2% bf16 MFU | 1623055 tok/s
step 13239/19560 | loss 3.331538 (-0.41z)| norm 0.2587 (-1.25z)| lr 1.52e-04 | 322.55 ms | 52.3% bf16 MFU | 1623174 tok/s
step 13240/19560 | loss 3.334723 (-0.31z)| norm 0.2967 (+0.75z)| lr 1.51e-04 | 322.70 ms | 52.3% bf16 MFU | 1623250 tok/s
step 13241/19560 | loss 3.314700 (-0.86z)| norm 0.2655 (-0.88z)| lr 1.51e-04 | 323.21 ms | 52.2% bf16 MFU | 1623193 tok/s
step 13242/19560 | loss 3.356667 (+0.30z)| norm 0.2729 (-0.48z)| lr 1.51e-04 | 322.92 ms | 52.3% bf16 MFU | 1623212 tok/s
step 13243/19560 | loss 3.368518 (+0.63z)| norm 0.2637 (-0.96z)| lr 1.51e-04 | 323.61 ms | 52.2% bf16 MFU | 1623056 tok/s
step 13244/19560 | loss 3.409691 (+1.74z)| norm 0.2656 (-0.84z)| lr 1.51e-04 | 323.03 ms | 52.2% bf16 MFU | 1623055 tok/s
step 13245/19560 | loss 3.316786 (-0.80z)| norm 0.2765 (-0.26z)| lr 1.51e-04 | 322.96 ms | 52.3% bf16 MFU | 1623071 tok/s
step 13246/19560 | loss 3.328049 (-0.50z)| norm 0.2835 (+0.10z)| lr 1.51e-04 | 322.66 ms | 52.3% bf16 MFU | 1623162 tok/s
step 13247/19560 | loss 3.308448 (-1.05z)| norm 0.2645 (-0.90z)| lr 1.51e-04 | 322.78 ms | 52.3% bf16 MFU | 1623217 tok/s
step 13248/19560 | loss 3.327574 (-0.51z)| norm 0.2752 (-0.33z)| lr 1.51e-04 | 322.84 ms | 52.3% bf16 MFU | 1623256 tok/s
step 13249/19560 | loss 3.337945 (-0.21z)| norm 0.2619 (-1.02z)| lr 1.51e-04 | 322.91 ms | 52.3% bf16 MFU | 1623276 tok/s
step 13250/19560 | loss 3.293020 (-1.44z)| norm 0.2855 (+0.23z)| lr 1.51e-04 | 322.94 ms | 52.3% bf16 MFU | 1623287 tok/s
val loss 3.329666
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3003/10042 = 0.299044
step 13251/19560 | loss 3.339493 (-0.15z)| norm 0.2541 (-1.41z)| lr 1.51e-04 | 322.59 ms | 52.3% bf16 MFU | 1623384 tok/s
step 13252/19560 | loss 3.294064 (-1.40z)| norm 0.2796 (-0.08z)| lr 1.51e-04 | 323.35 ms | 52.2% bf16 MFU | 1623287 tok/s
step 13253/19560 | loss 3.282135 (-1.70z)| norm 0.2626 (-0.96z)| lr 1.51e-04 | 322.88 ms | 52.3% bf16 MFU | 1623311 tok/s
step 13254/19560 | loss 3.571083 (+5.44z)| norm 0.2791 (-0.09z)| lr 1.51e-04 | 322.60 ms | 52.3% bf16 MFU | 1623406 tok/s
step 13255/19560 | loss 3.285615 (-1.44z)| norm 0.2840 (+0.16z)| lr 1.51e-04 | 322.47 ms | 52.3% bf16 MFU | 1623527 tok/s
step 13256/19560 | loss 3.365157 (+0.46z)| norm 0.2731 (-0.40z)| lr 1.51e-04 | 322.80 ms | 52.3% bf16 MFU | 1623560 tok/s
step 13257/19560 | loss 3.294878 (-1.21z)| norm 0.2700 (-0.56z)| lr 1.51e-04 | 322.82 ms | 52.3% bf16 MFU | 1623587 tok/s
step 13258/19560 | loss 3.309510 (-0.85z)| norm 0.2885 (+0.40z)| lr 1.51e-04 | 323.24 ms | 52.2% bf16 MFU | 1623506 tok/s
step 13259/19560 | loss 3.339152 (-0.14z)| norm 0.2756 (-0.27z)| lr 1.51e-04 | 322.92 ms | 52.3% bf16 MFU | 1623511 tok/s
step 13260/19560 | loss 3.300207 (-1.06z)| norm 0.3026 (+1.14z)| lr 1.51e-04 | 322.58 ms | 52.3% bf16 MFU | 1623600 tok/s
step 13261/19560 | loss 3.376960 (+0.75z)| norm 0.2936 (+0.66z)| lr 1.51e-04 | 322.51 ms | 52.3% bf16 MFU | 1623703 tok/s
step 13262/19560 | loss 3.272938 (-1.69z)| norm 0.2817 (+0.04z)| lr 1.51e-04 | 323.11 ms | 52.2% bf16 MFU | 1623649 tok/s
step 13263/19560 | loss 3.331372 (-0.32z)| norm 0.2769 (-0.22z)| lr 1.50e-04 | 322.93 ms | 52.3% bf16 MFU | 1623645 tok/s
step 13264/19560 | loss 3.292324 (-1.22z)| norm 0.2953 (+0.74z)| lr 1.50e-04 | 322.80 ms | 52.3% bf16 MFU | 1623672 tok/s
step 13265/19560 | loss 3.375664 (+0.72z)| norm 0.2930 (+0.62z)| lr 1.50e-04 | 322.29 ms | 52.4% bf16 MFU | 1623827 tok/s
step 13266/19560 | loss 3.270020 (-1.72z)| norm 0.2796 (-0.08z)| lr 1.50e-04 | 322.89 ms | 52.3% bf16 MFU | 1623823 tok/s
step 13267/19560 | loss 3.399227 (+1.25z)| norm 0.2856 (+0.25z)| lr 1.50e-04 | 322.87 ms | 52.3% bf16 MFU | 1623825 tok/s
step 13268/19560 | loss 3.336763 (-0.18z)| norm 0.3924 (+5.22z)| lr 1.50e-04 | 322.18 ms | 52.4% bf16 MFU | 1623998 tok/s
step 13269/19560 | loss 3.359411 (+0.34z)| norm 0.2935 (+0.57z)| lr 1.50e-04 | 322.78 ms | 52.3% bf16 MFU | 1624014 tok/s
step 13270/19560 | loss 3.333028 (-0.27z)| norm 0.3286 (+2.19z)| lr 1.50e-04 | 323.27 ms | 52.2% bf16 MFU | 1623904 tok/s
step 13271/19560 | loss 3.315077 (-0.68z)| norm 0.3003 (+0.84z)| lr 1.50e-04 | 322.59 ms | 52.3% bf16 MFU | 1623972 tok/s
step 13272/19560 | loss 3.312236 (-0.74z)| norm 0.3105 (+1.30z)| lr 1.50e-04 | 322.63 ms | 52.3% bf16 MFU | 1624026 tok/s
step 13273/19560 | loss 3.352365 (+0.18z)| norm 0.2990 (+0.76z)| lr 1.50e-04 | 323.42 ms | 52.2% bf16 MFU | 1623878 tok/s
step 13274/19560 | loss 3.332426 (-0.29z)| norm 0.3086 (+1.40z)| lr 1.50e-04 | 322.69 ms | 52.3% bf16 MFU | 1623921 tok/s
step 13275/19560 | loss 3.327506 (-0.42z)| norm 0.3131 (+1.63z)| lr 1.50e-04 | 322.35 ms | 52.4% bf16 MFU | 1624047 tok/s
step 13276/19560 | loss 3.354770 (+0.22z)| norm 0.3089 (+1.40z)| lr 1.50e-04 | 322.92 ms | 52.3% bf16 MFU | 1624023 tok/s
step 13277/19560 | loss 3.386980 (+0.96z)| norm 0.2912 (+0.51z)| lr 1.50e-04 | 322.87 ms | 52.3% bf16 MFU | 1624013 tok/s
step 13278/19560 | loss 3.148583 (-4.25z)| norm 0.4444 (+6.97z)| lr 1.50e-04 | 322.66 ms | 52.3% bf16 MFU | 1624057 tok/s
step 13279/19560 | loss 3.344130 (+0.00z)| norm 0.2922 (+0.43z)| lr 1.50e-04 | 323.41 ms | 52.2% bf16 MFU | 1623910 tok/s
step 13280/19560 | loss 3.303004 (-0.88z)| norm 0.2987 (+0.70z)| lr 1.50e-04 | 322.47 ms | 52.3% bf16 MFU | 1624005 tok/s
step 13281/19560 | loss 3.288344 (-1.19z)| norm 0.2689 (-0.59z)| lr 1.50e-04 | 323.27 ms | 52.2% bf16 MFU | 1623898 tok/s
step 13282/19560 | loss 3.291433 (-1.11z)| norm 0.2806 (-0.08z)| lr 1.50e-04 | 322.77 ms | 52.3% bf16 MFU | 1623920 tok/s
step 13283/19560 | loss 3.385062 (+0.91z)| norm 0.2679 (-0.63z)| lr 1.50e-04 | 322.46 ms | 52.3% bf16 MFU | 1624019 tok/s
step 13284/19560 | loss 3.313063 (-0.64z)| norm 0.2818 (-0.02z)| lr 1.50e-04 | 322.80 ms | 52.3% bf16 MFU | 1624026 tok/s
step 13285/19560 | loss 3.361894 (+0.41z)| norm 0.2945 (+0.53z)| lr 1.50e-04 | 323.26 ms | 52.2% bf16 MFU | 1623918 tok/s
step 13286/19560 | loss 3.264485 (-1.68z)| norm 0.2696 (-0.56z)| lr 1.49e-04 | 322.67 ms | 52.3% bf16 MFU | 1623965 tok/s
step 13287/19560 | loss 3.460093 (+2.43z)| norm 0.3136 (+1.36z)| lr 1.49e-04 | 322.77 ms | 52.3% bf16 MFU | 1623982 tok/s
step 13288/19560 | loss 3.315486 (-0.60z)| norm 0.2954 (+0.57z)| lr 1.49e-04 | 322.44 ms | 52.3% bf16 MFU | 1624083 tok/s
step 13289/19560 | loss 3.357882 (+0.30z)| norm 0.2693 (-0.57z)| lr 1.49e-04 | 322.62 ms | 52.3% bf16 MFU | 1624134 tok/s
step 13290/19560 | loss 3.284454 (-1.23z)| norm 0.3034 (+0.93z)| lr 1.49e-04 | 323.24 ms | 52.2% bf16 MFU | 1624026 tok/s
step 13291/19560 | loss 3.357602 (+0.31z)| norm 0.3013 (+0.82z)| lr 1.49e-04 | 322.89 ms | 52.3% bf16 MFU | 1624011 tok/s
step 13292/19560 | loss 3.345352 (+0.06z)| norm 0.2552 (-1.19z)| lr 1.49e-04 | 323.37 ms | 52.2% bf16 MFU | 1623877 tok/s
step 13293/19560 | loss 3.302577 (-0.84z)| norm 0.2936 (+0.48z)| lr 1.49e-04 | 322.81 ms | 52.3% bf16 MFU | 1623890 tok/s
step 13294/19560 | loss 3.346548 (+0.07z)| norm 0.2761 (-0.28z)| lr 1.49e-04 | 323.24 ms | 52.2% bf16 MFU | 1623795 tok/s
step 13295/19560 | loss 3.362423 (+0.40z)| norm 0.2656 (-0.74z)| lr 1.49e-04 | 323.23 ms | 52.2% bf16 MFU | 1623706 tok/s
step 13296/19560 | loss 3.292336 (-1.06z)| norm 0.2618 (-0.90z)| lr 1.49e-04 | 322.99 ms | 52.3% bf16 MFU | 1623682 tok/s
step 13297/19560 | loss 3.346937 (+0.09z)| norm 0.2721 (-0.46z)| lr 1.49e-04 | 323.29 ms | 52.2% bf16 MFU | 1623585 tok/s
step 13298/19560 | loss 3.322954 (-0.41z)| norm 0.3750 (+3.77z)| lr 1.49e-04 | 322.49 ms | 52.3% bf16 MFU | 1623694 tok/s
step 13299/19560 | loss 3.391516 (+1.03z)| norm 0.2943 (+0.45z)| lr 1.49e-04 | 322.83 ms | 52.3% bf16 MFU | 1623710 tok/s
step 13300/19560 | loss 3.312261 (-0.63z)| norm 0.2620 (-0.88z)| lr 1.49e-04 | 323.12 ms | 52.2% bf16 MFU | 1623654 tok/s
step 13301/19560 | loss 3.448501 (+2.18z)| norm 0.2634 (-0.83z)| lr 1.49e-04 | 322.54 ms | 52.3% bf16 MFU | 1623746 tok/s
step 13302/19560 | loss 3.381381 (+0.77z)| norm 0.2827 (-0.02z)| lr 1.49e-04 | 322.40 ms | 52.3% bf16 MFU | 1623869 tok/s
step 13303/19560 | loss 3.365498 (+0.44z)| norm 0.2726 (-0.45z)| lr 1.49e-04 | 322.55 ms | 52.3% bf16 MFU | 1623947 tok/s
step 13304/19560 | loss 3.358951 (+0.32z)| norm 0.2812 (-0.09z)| lr 1.49e-04 | 322.87 ms | 52.3% bf16 MFU | 1623942 tok/s
step 13305/19560 | loss 3.381014 (+0.77z)| norm 0.2813 (-0.08z)| lr 1.49e-04 | 322.84 ms | 52.3% bf16 MFU | 1623945 tok/s
step 13306/19560 | loss 3.322793 (-0.44z)| norm 0.2794 (-0.17z)| lr 1.49e-04 | 322.42 ms | 52.3% bf16 MFU | 1624052 tok/s
step 13307/19560 | loss 3.371900 (+0.58z)| norm 0.2578 (-1.05z)| lr 1.49e-04 | 322.82 ms | 52.3% bf16 MFU | 1624053 tok/s
step 13308/19560 | loss 3.332172 (-0.24z)| norm 0.2956 (+0.51z)| lr 1.49e-04 | 323.54 ms | 52.2% bf16 MFU | 1623873 tok/s
step 13309/19560 | loss 3.318428 (-0.52z)| norm 0.2737 (-0.40z)| lr 1.49e-04 | 323.17 ms | 52.2% bf16 MFU | 1623795 tok/s
step 13310/19560 | loss 3.319021 (-0.50z)| norm 0.2902 (+0.28z)| lr 1.48e-04 | 322.64 ms | 52.3% bf16 MFU | 1623856 tok/s
step 13311/19560 | loss 3.379584 (+0.78z)| norm 0.2909 (+0.31z)| lr 1.48e-04 | 322.81 ms | 52.3% bf16 MFU | 1623869 tok/s
step 13312/19560 | loss 3.274375 (-1.43z)| norm 0.2766 (-0.28z)| lr 1.48e-04 | 322.39 ms | 52.4% bf16 MFU | 1623989 tok/s
step 13313/19560 | loss 3.313108 (-0.61z)| norm 0.2860 (+0.09z)| lr 1.48e-04 | 322.49 ms | 52.3% bf16 MFU | 1624076 tok/s
step 13314/19560 | loss 3.378565 (+0.75z)| norm 0.3712 (+3.44z)| lr 1.48e-04 | 322.87 ms | 52.3% bf16 MFU | 1624064 tok/s
step 13315/19560 | loss 3.296846 (-0.94z)| norm 0.3051 (+0.81z)| lr 1.48e-04 | 322.95 ms | 52.3% bf16 MFU | 1624033 tok/s
step 13316/19560 | loss 3.338048 (-0.08z)| norm 0.2844 (-0.01z)| lr 1.48e-04 | 323.48 ms | 52.2% bf16 MFU | 1623871 tok/s
step 13317/19560 | loss 3.332285 (-0.19z)| norm 0.2789 (-0.23z)| lr 1.48e-04 | 322.99 ms | 52.3% bf16 MFU | 1623838 tok/s
step 13318/19560 | loss 3.383408 (+0.86z)| norm 0.2621 (-0.89z)| lr 1.48e-04 | 323.01 ms | 52.3% bf16 MFU | 1623804 tok/s
step 13319/19560 | loss 3.276959 (-1.34z)| norm 0.2780 (-0.26z)| lr 1.48e-04 | 322.64 ms | 52.3% bf16 MFU | 1623863 tok/s
step 13320/19560 | loss 3.389135 (+0.98z)| norm 0.2795 (-0.21z)| lr 1.48e-04 | 323.34 ms | 52.2% bf16 MFU | 1623742 tok/s
step 13321/19560 | loss 3.284868 (-1.17z)| norm 0.2799 (-0.19z)| lr 1.48e-04 | 323.12 ms | 52.2% bf16 MFU | 1623683 tok/s
step 13322/19560 | loss 3.367884 (+0.53z)| norm 0.3019 (+0.67z)| lr 1.48e-04 | 323.25 ms | 52.2% bf16 MFU | 1623596 tok/s
step 13323/19560 | loss 3.437415 (+1.92z)| norm 0.2725 (-0.50z)| lr 1.48e-04 | 322.90 ms | 52.3% bf16 MFU | 1623602 tok/s
step 13324/19560 | loss 3.313076 (-0.59z)| norm 0.3002 (+0.59z)| lr 1.48e-04 | 323.10 ms | 52.2% bf16 MFU | 1623555 tok/s
step 13325/19560 | loss 3.368297 (+0.53z)| norm 0.2711 (-0.55z)| lr 1.48e-04 | 322.63 ms | 52.3% bf16 MFU | 1623631 tok/s
step 13326/19560 | loss 3.322444 (-0.39z)| norm 0.2727 (-0.48z)| lr 1.48e-04 | 323.34 ms | 52.2% bf16 MFU | 1623523 tok/s
step 13327/19560 | loss 3.464168 (+2.43z)| norm 0.2889 (+0.17z)| lr 1.48e-04 | 322.75 ms | 52.3% bf16 MFU | 1623570 tok/s
step 13328/19560 | loss 3.320343 (-0.46z)| norm 0.2859 (+0.05z)| lr 1.48e-04 | 322.61 ms | 52.3% bf16 MFU | 1623648 tok/s
step 13329/19560 | loss 3.354076 (+0.24z)| norm 0.2931 (+0.34z)| lr 1.48e-04 | 323.21 ms | 52.2% bf16 MFU | 1623572 tok/s
step 13330/19560 | loss 3.317674 (-0.51z)| norm 0.2963 (+0.47z)| lr 1.48e-04 | 323.55 ms | 52.2% bf16 MFU | 1623414 tok/s
step 13331/19560 | loss 3.358287 (+0.32z)| norm 0.2625 (-0.89z)| lr 1.48e-04 | 322.77 ms | 52.3% bf16 MFU | 1623460 tok/s
step 13332/19560 | loss 3.315583 (-0.55z)| norm 0.2761 (-0.34z)| lr 1.48e-04 | 323.12 ms | 52.2% bf16 MFU | 1623417 tok/s
step 13333/19560 | loss 3.355958 (+0.28z)| norm 0.2717 (-0.51z)| lr 1.47e-04 | 322.86 ms | 52.3% bf16 MFU | 1623442 tok/s
step 13334/19560 | loss 3.317071 (-0.52z)| norm 0.2860 (+0.07z)| lr 1.47e-04 | 322.85 ms | 52.3% bf16 MFU | 1623467 tok/s
step 13335/19560 | loss 3.397318 (+1.15z)| norm 0.2806 (-0.15z)| lr 1.47e-04 | 323.58 ms | 52.2% bf16 MFU | 1623307 tok/s
step 13336/19560 | loss 3.390915 (+1.01z)| norm 0.2923 (+0.32z)| lr 1.47e-04 | 322.90 ms | 52.3% bf16 MFU | 1623327 tok/s
step 13337/19560 | loss 3.294164 (-0.98z)| norm 0.3195 (+1.39z)| lr 1.47e-04 | 322.81 ms | 52.3% bf16 MFU | 1623368 tok/s
step 13338/19560 | loss 3.333239 (-0.17z)| norm 0.2875 (+0.11z)| lr 1.47e-04 | 322.94 ms | 52.3% bf16 MFU | 1623373 tok/s
step 13339/19560 | loss 3.329916 (-0.24z)| norm 0.2740 (-0.43z)| lr 1.47e-04 | 323.42 ms | 52.2% bf16 MFU | 1623258 tok/s
step 13340/19560 | loss 3.328771 (-0.26z)| norm 0.2762 (-0.35z)| lr 1.47e-04 | 322.99 ms | 52.3% bf16 MFU | 1623257 tok/s
step 13341/19560 | loss 3.322578 (-0.39z)| norm 0.2967 (+0.47z)| lr 1.47e-04 | 322.62 ms | 52.3% bf16 MFU | 1623350 tok/s
step 13342/19560 | loss 3.290073 (-1.04z)| norm 0.2792 (-0.23z)| lr 1.47e-04 | 322.87 ms | 52.3% bf16 MFU | 1623373 tok/s
step 13343/19560 | loss 3.304858 (-0.74z)| norm 0.2649 (-0.80z)| lr 1.47e-04 | 323.40 ms | 52.2% bf16 MFU | 1623263 tok/s
step 13344/19560 | loss 3.301628 (-0.80z)| norm 0.2717 (-0.52z)| lr 1.47e-04 | 322.58 ms | 52.3% bf16 MFU | 1623365 tok/s
step 13345/19560 | loss 3.388762 (+0.99z)| norm 0.2841 (-0.03z)| lr 1.47e-04 | 322.50 ms | 52.3% bf16 MFU | 1623480 tok/s
step 13346/19560 | loss 3.319866 (-0.43z)| norm 0.2592 (-1.02z)| lr 1.47e-04 | 323.39 ms | 52.2% bf16 MFU | 1623368 tok/s
step 13347/19560 | loss 3.288490 (-1.07z)| norm 0.2815 (-0.13z)| lr 1.47e-04 | 323.14 ms | 52.2% bf16 MFU | 1623324 tok/s
step 13348/19560 | loss 3.407331 (+1.36z)| norm 0.2600 (-0.98z)| lr 1.47e-04 | 322.91 ms | 52.3% bf16 MFU | 1623340 tok/s
step 13349/19560 | loss 3.387903 (+0.96z)| norm 0.2803 (-0.18z)| lr 1.47e-04 | 322.54 ms | 52.3% bf16 MFU | 1623448 tok/s
step 13350/19560 | loss 3.376728 (+0.72z)| norm 0.2636 (-0.84z)| lr 1.47e-04 | 323.74 ms | 52.1% bf16 MFU | 1623248 tok/s
step 13351/19560 | loss 3.457234 (+2.30z)| norm 0.2985 (+0.54z)| lr 1.47e-04 | 322.87 ms | 52.3% bf16 MFU | 1623278 tok/s
step 13352/19560 | loss 3.347347 (+0.11z)| norm 0.2721 (-0.51z)| lr 1.47e-04 | 322.51 ms | 52.3% bf16 MFU | 1623396 tok/s
step 13353/19560 | loss 3.284844 (-1.11z)| norm 0.2782 (-0.27z)| lr 1.47e-04 | 322.98 ms | 52.3% bf16 MFU | 1623389 tok/s
step 13354/19560 | loss 3.319557 (-0.41z)| norm 0.2593 (-1.01z)| lr 1.47e-04 | 322.58 ms | 52.3% bf16 MFU | 1623485 tok/s
step 13355/19560 | loss 3.354674 (+0.30z)| norm 0.2665 (-0.72z)| lr 1.47e-04 | 323.46 ms | 52.2% bf16 MFU | 1623355 tok/s
step 13356/19560 | loss 3.335806 (-0.08z)| norm 0.2800 (-0.19z)| lr 1.46e-04 | 323.09 ms | 52.2% bf16 MFU | 1623325 tok/s
step 13357/19560 | loss 3.344708 (+0.11z)| norm 0.2738 (-0.44z)| lr 1.46e-04 | 323.16 ms | 52.2% bf16 MFU | 1623277 tok/s
step 13358/19560 | loss 3.329097 (-0.20z)| norm 0.2674 (-0.69z)| lr 1.46e-04 | 322.96 ms | 52.3% bf16 MFU | 1623283 tok/s
step 13359/19560 | loss 3.313119 (-0.52z)| norm 0.2871 (+0.09z)| lr 1.46e-04 | 322.62 ms | 52.3% bf16 MFU | 1623374 tok/s
step 13360/19560 | loss 3.336162 (-0.05z)| norm 0.2704 (-0.57z)| lr 1.46e-04 | 323.23 ms | 52.2% bf16 MFU | 1623305 tok/s
step 13361/19560 | loss 3.427573 (+1.80z)| norm 0.2702 (-0.58z)| lr 1.46e-04 | 322.89 ms | 52.3% bf16 MFU | 1623327 tok/s
step 13362/19560 | loss 3.341713 (+0.04z)| norm 0.2690 (-0.62z)| lr 1.46e-04 | 323.51 ms | 52.2% bf16 MFU | 1623191 tok/s
step 13363/19560 | loss 3.341402 (+0.03z)| norm 0.2649 (-0.78z)| lr 1.46e-04 | 322.57 ms | 52.3% bf16 MFU | 1623300 tok/s
step 13364/19560 | loss 3.369656 (+0.61z)| norm 0.2854 (+0.03z)| lr 1.46e-04 | 322.88 ms | 52.3% bf16 MFU | 1623324 tok/s
step 13365/19560 | loss 3.336039 (-0.08z)| norm 0.2777 (-0.28z)| lr 1.46e-04 | 323.27 ms | 52.2% bf16 MFU | 1623250 tok/s
step 13366/19560 | loss 3.480013 (+2.76z)| norm 0.2699 (-0.59z)| lr 1.46e-04 | 323.14 ms | 52.2% bf16 MFU | 1623211 tok/s
step 13367/19560 | loss 3.299399 (-0.83z)| norm 0.3050 (+0.79z)| lr 1.46e-04 | 322.72 ms | 52.3% bf16 MFU | 1623281 tok/s
step 13368/19560 | loss 3.310830 (-0.59z)| norm 0.2628 (-0.87z)| lr 1.46e-04 | 322.82 ms | 52.3% bf16 MFU | 1623321 tok/s
step 13369/19560 | loss 3.387210 (+0.91z)| norm 0.2797 (-0.21z)| lr 1.46e-04 | 323.28 ms | 52.2% bf16 MFU | 1623243 tok/s
step 13370/19560 | loss 3.418399 (+1.50z)| norm 0.2835 (-0.06z)| lr 1.46e-04 | 323.25 ms | 52.2% bf16 MFU | 1623176 tok/s
step 13371/19560 | loss 3.268576 (-1.41z)| norm 0.2879 (+0.11z)| lr 1.46e-04 | 323.20 ms | 52.2% bf16 MFU | 1623126 tok/s
step 13372/19560 | loss 3.338026 (-0.05z)| norm 0.2784 (-0.27z)| lr 1.46e-04 | 323.13 ms | 52.2% bf16 MFU | 1623098 tok/s
step 13373/19560 | loss 3.334307 (-0.12z)| norm 0.2874 (+0.08z)| lr 1.46e-04 | 323.45 ms | 52.2% bf16 MFU | 1622989 tok/s
step 13374/19560 | loss 3.311552 (-0.57z)| norm 0.2660 (-0.77z)| lr 1.46e-04 | 323.11 ms | 52.2% bf16 MFU | 1622971 tok/s
step 13375/19560 | loss 3.281489 (-1.15z)| norm 0.2832 (-0.09z)| lr 1.46e-04 | 323.45 ms | 52.2% bf16 MFU | 1622870 tok/s
step 13376/19560 | loss 3.315327 (-0.49z)| norm 0.2649 (-0.82z)| lr 1.46e-04 | 323.47 ms | 52.2% bf16 MFU | 1622768 tok/s
step 13377/19560 | loss 3.597018 (+4.55z)| norm 0.3097 (+0.96z)| lr 1.46e-04 | 322.85 ms | 52.3% bf16 MFU | 1622826 tok/s
step 13378/19560 | loss 3.355805 (+0.23z)| norm 0.2732 (-0.49z)| lr 1.46e-04 | 323.16 ms | 52.2% bf16 MFU | 1622803 tok/s
step 13379/19560 | loss 3.345864 (+0.06z)| norm 0.2581 (-1.10z)| lr 1.45e-04 | 323.47 ms | 52.2% bf16 MFU | 1622705 tok/s
step 13380/19560 | loss 3.409395 (+1.18z)| norm 0.2811 (-0.18z)| lr 1.45e-04 | 323.63 ms | 52.1% bf16 MFU | 1622571 tok/s
step 13381/19560 | loss 3.412142 (+1.20z)| norm 0.2672 (-0.74z)| lr 1.45e-04 | 322.92 ms | 52.3% bf16 MFU | 1622620 tok/s
step 13382/19560 | loss 3.268701 (-1.40z)| norm 0.2747 (-0.43z)| lr 1.45e-04 | 323.10 ms | 52.2% bf16 MFU | 1622624 tok/s
step 13383/19560 | loss 3.320151 (-0.43z)| norm 0.2620 (-0.94z)| lr 1.45e-04 | 323.07 ms | 52.2% bf16 MFU | 1622634 tok/s
step 13384/19560 | loss 3.275295 (-1.26z)| norm 0.2759 (-0.38z)| lr 1.45e-04 | 323.18 ms | 52.2% bf16 MFU | 1622616 tok/s
step 13385/19560 | loss 3.368925 (+0.50z)| norm 0.2704 (-0.60z)| lr 1.45e-04 | 322.51 ms | 52.3% bf16 MFU | 1622768 tok/s
step 13386/19560 | loss 3.312900 (-0.56z)| norm 0.2730 (-0.49z)| lr 1.45e-04 | 322.61 ms | 52.3% bf16 MFU | 1622886 tok/s
step 13387/19560 | loss 3.372114 (+0.56z)| norm 0.2821 (-0.13z)| lr 1.45e-04 | 322.65 ms | 52.3% bf16 MFU | 1622989 tok/s
step 13388/19560 | loss 3.376225 (+0.62z)| norm 0.2632 (-0.87z)| lr 1.45e-04 | 322.82 ms | 52.3% bf16 MFU | 1623043 tok/s
step 13389/19560 | loss 3.336812 (-0.12z)| norm 0.2720 (-0.51z)| lr 1.45e-04 | 323.37 ms | 52.2% bf16 MFU | 1622957 tok/s
step 13390/19560 | loss 3.312468 (-0.59z)| norm 0.2637 (-0.83z)| lr 1.45e-04 | 322.70 ms | 52.3% bf16 MFU | 1623042 tok/s
step 13391/19560 | loss 3.345261 (+0.03z)| norm 0.2738 (-0.43z)| lr 1.45e-04 | 323.04 ms | 52.2% bf16 MFU | 1623038 tok/s
step 13392/19560 | loss 3.268651 (-1.42z)| norm 0.2650 (-0.77z)| lr 1.45e-04 | 323.31 ms | 52.2% bf16 MFU | 1622967 tok/s
step 13393/19560 | loss 3.405200 (+1.17z)| norm 0.2532 (-1.22z)| lr 1.45e-04 | 322.99 ms | 52.3% bf16 MFU | 1622981 tok/s
step 13394/19560 | loss 3.434842 (+1.70z)| norm 0.3085 (+0.95z)| lr 1.45e-04 | 323.57 ms | 52.2% bf16 MFU | 1622849 tok/s
step 13395/19560 | loss 3.291610 (-0.99z)| norm 0.2611 (-0.90z)| lr 1.45e-04 | 322.88 ms | 52.3% bf16 MFU | 1622896 tok/s
step 13396/19560 | loss 3.278905 (-1.21z)| norm 0.2757 (-0.32z)| lr 1.45e-04 | 322.91 ms | 52.3% bf16 MFU | 1622932 tok/s
step 13397/19560 | loss 3.354682 (+0.21z)| norm 0.2721 (-0.47z)| lr 1.45e-04 | 323.72 ms | 52.1% bf16 MFU | 1622765 tok/s
step 13398/19560 | loss 3.300188 (-0.81z)| norm 0.2533 (-1.24z)| lr 1.45e-04 | 323.18 ms | 52.2% bf16 MFU | 1622741 tok/s
step 13399/19560 | loss 3.318858 (-0.46z)| norm 0.2797 (-0.12z)| lr 1.45e-04 | 323.40 ms | 52.2% bf16 MFU | 1622663 tok/s
step 13400/19560 | loss 3.367361 (+0.45z)| norm 0.2736 (-0.36z)| lr 1.45e-04 | 322.75 ms | 52.3% bf16 MFU | 1622753 tok/s
step 13401/19560 | loss 3.325644 (-0.33z)| norm 0.3139 (+1.35z)| lr 1.45e-04 | 323.02 ms | 52.2% bf16 MFU | 1622769 tok/s
step 13402/19560 | loss 3.301346 (-0.78z)| norm 0.2669 (-0.64z)| lr 1.45e-04 | 322.90 ms | 52.3% bf16 MFU | 1622815 tok/s
step 13403/19560 | loss 3.403786 (+1.12z)| norm 0.2981 (+0.70z)| lr 1.44e-04 | 323.22 ms | 52.2% bf16 MFU | 1622780 tok/s
step 13404/19560 | loss 3.285129 (-1.08z)| norm 0.2866 (+0.21z)| lr 1.44e-04 | 323.34 ms | 52.2% bf16 MFU | 1622715 tok/s
step 13405/19560 | loss 3.353858 (+0.20z)| norm 0.2878 (+0.27z)| lr 1.44e-04 | 322.64 ms | 52.3% bf16 MFU | 1622830 tok/s
step 13406/19560 | loss 3.326581 (-0.35z)| norm 0.3140 (+1.83z)| lr 1.44e-04 | 323.32 ms | 52.2% bf16 MFU | 1622769 tok/s
step 13407/19560 | loss 3.351904 (+0.15z)| norm 0.2661 (-0.78z)| lr 1.44e-04 | 322.95 ms | 52.3% bf16 MFU | 1622802 tok/s
step 13408/19560 | loss 3.341806 (-0.06z)| norm 0.3016 (+1.16z)| lr 1.44e-04 | 323.39 ms | 52.2% bf16 MFU | 1622724 tok/s
step 13409/19560 | loss 3.321174 (-0.47z)| norm 0.2632 (-0.94z)| lr 1.44e-04 | 322.59 ms | 52.3% bf16 MFU | 1622851 tok/s
step 13410/19560 | loss 3.296313 (-0.96z)| norm 0.3149 (+1.84z)| lr 1.44e-04 | 322.67 ms | 52.3% bf16 MFU | 1622952 tok/s
step 13411/19560 | loss 3.363358 (+0.37z)| norm 0.2753 (-0.29z)| lr 1.44e-04 | 322.71 ms | 52.3% bf16 MFU | 1623037 tok/s
step 13412/19560 | loss 3.311138 (-0.67z)| norm 0.2916 (+0.59z)| lr 1.44e-04 | 322.87 ms | 52.3% bf16 MFU | 1623077 tok/s
step 13413/19560 | loss 3.380229 (+0.70z)| norm 0.2986 (+0.96z)| lr 1.44e-04 | 323.21 ms | 52.2% bf16 MFU | 1623028 tok/s
step 13414/19560 | loss 3.289514 (-1.11z)| norm 0.2654 (-0.82z)| lr 1.44e-04 | 322.32 ms | 52.4% bf16 MFU | 1623208 tok/s
step 13415/19560 | loss 3.394792 (+1.01z)| norm 0.3205 (+2.12z)| lr 1.44e-04 | 322.71 ms | 52.3% bf16 MFU | 1623278 tok/s
step 13416/19560 | loss 3.352185 (+0.15z)| norm 0.2847 (+0.21z)| lr 1.44e-04 | 323.30 ms | 52.2% bf16 MFU | 1623198 tok/s
step 13417/19560 | loss 3.327472 (-0.35z)| norm 0.2714 (-0.50z)| lr 1.44e-04 | 322.65 ms | 52.3% bf16 MFU | 1623287 tok/s
step 13418/19560 | loss 3.384723 (+0.80z)| norm 0.2853 (+0.25z)| lr 1.44e-04 | 322.97 ms | 52.3% bf16 MFU | 1623288 tok/s
step 13419/19560 | loss 3.374832 (+0.59z)| norm 0.2648 (-0.84z)| lr 1.44e-04 | 322.02 ms | 52.4% bf16 MFU | 1623529 tok/s
step 13420/19560 | loss 3.300743 (-0.90z)| norm 0.3078 (+1.46z)| lr 1.44e-04 | 323.27 ms | 52.2% bf16 MFU | 1623443 tok/s
step 13421/19560 | loss 3.321430 (-0.49z)| norm 0.2938 (+0.71z)| lr 1.44e-04 | 322.74 ms | 52.3% bf16 MFU | 1623496 tok/s
step 13422/19560 | loss 3.299649 (-0.92z)| norm 0.2573 (-1.24z)| lr 1.44e-04 | 322.23 ms | 52.4% bf16 MFU | 1623674 tok/s
step 13423/19560 | loss 3.375338 (+0.61z)| norm 0.3099 (+1.54z)| lr 1.44e-04 | 323.01 ms | 52.2% bf16 MFU | 1623646 tok/s
step 13424/19560 | loss 3.290782 (-1.10z)| norm 0.2774 (-0.19z)| lr 1.44e-04 | 322.46 ms | 52.3% bf16 MFU | 1623759 tok/s
step 13425/19560 | loss 3.379995 (+0.70z)| norm 0.2953 (+0.75z)| lr 1.44e-04 | 322.70 ms | 52.3% bf16 MFU | 1623804 tok/s
step 13426/19560 | loss 3.433867 (+1.74z)| norm 0.2837 (+0.19z)| lr 1.43e-04 | 322.44 ms | 52.3% bf16 MFU | 1623915 tok/s
step 13427/19560 | loss 3.367676 (+0.43z)| norm 0.2779 (-0.14z)| lr 1.43e-04 | 323.07 ms | 52.2% bf16 MFU | 1623860 tok/s
step 13428/19560 | loss 3.297295 (-0.97z)| norm 0.2773 (-0.19z)| lr 1.43e-04 | 323.05 ms | 52.2% bf16 MFU | 1623813 tok/s
step 13429/19560 | loss 3.308659 (-0.73z)| norm 0.2949 (+0.85z)| lr 1.43e-04 | 322.62 ms | 52.3% bf16 MFU | 1623877 tok/s
step 13430/19560 | loss 3.312784 (-0.64z)| norm 0.2920 (+0.67z)| lr 1.43e-04 | 322.42 ms | 52.3% bf16 MFU | 1623988 tok/s
step 13431/19560 | loss 3.279363 (-1.29z)| norm 0.2912 (+0.61z)| lr 1.43e-04 | 323.05 ms | 52.2% bf16 MFU | 1623935 tok/s
step 13432/19560 | loss 3.292781 (-1.01z)| norm 0.2815 (+0.03z)| lr 1.43e-04 | 323.05 ms | 52.2% bf16 MFU | 1623885 tok/s
step 13433/19560 | loss 3.326392 (-0.33z)| norm 0.2749 (-0.36z)| lr 1.43e-04 | 322.68 ms | 52.3% bf16 MFU | 1623930 tok/s
step 13434/19560 | loss 3.295665 (-0.94z)| norm 0.3484 (+3.77z)| lr 1.43e-04 | 322.42 ms | 52.3% bf16 MFU | 1624040 tok/s
step 13435/19560 | loss 3.322101 (-0.40z)| norm 0.3067 (+1.40z)| lr 1.43e-04 | 322.68 ms | 52.3% bf16 MFU | 1624078 tok/s
step 13436/19560 | loss 3.284727 (-1.14z)| norm 0.2740 (-0.43z)| lr 1.43e-04 | 323.08 ms | 52.2% bf16 MFU | 1624012 tok/s
step 13437/19560 | loss 3.372572 (+0.61z)| norm 0.2739 (-0.44z)| lr 1.43e-04 | 322.80 ms | 52.3% bf16 MFU | 1624021 tok/s
step 13438/19560 | loss 3.271555 (-1.39z)| norm 0.2767 (-0.27z)| lr 1.43e-04 | 322.54 ms | 52.3% bf16 MFU | 1624095 tok/s
step 13439/19560 | loss 3.296166 (-0.89z)| norm 0.2757 (-0.32z)| lr 1.43e-04 | 322.49 ms | 52.3% bf16 MFU | 1624179 tok/s
step 13440/19560 | loss 3.326634 (-0.30z)| norm 0.2481 (-1.84z)| lr 1.43e-04 | 323.15 ms | 52.2% bf16 MFU | 1624092 tok/s
step 13441/19560 | loss 3.324727 (-0.34z)| norm 0.2656 (-0.86z)| lr 1.43e-04 | 322.94 ms | 52.3% bf16 MFU | 1624061 tok/s
step 13442/19560 | loss 3.304435 (-0.73z)| norm 0.2732 (-0.44z)| lr 1.43e-04 | 322.99 ms | 52.3% bf16 MFU | 1624021 tok/s
step 13443/19560 | loss 3.353634 (+0.24z)| norm 0.2745 (-0.35z)| lr 1.43e-04 | 322.97 ms | 52.3% bf16 MFU | 1623987 tok/s
step 13444/19560 | loss 3.341381 (-0.00z)| norm 0.2616 (-1.13z)| lr 1.43e-04 | 322.43 ms | 52.3% bf16 MFU | 1624089 tok/s
step 13445/19560 | loss 3.338376 (-0.06z)| norm 0.2939 (+0.86z)| lr 1.43e-04 | 323.07 ms | 52.2% bf16 MFU | 1624027 tok/s
step 13446/19560 | loss 3.307636 (-0.67z)| norm 0.3032 (+1.41z)| lr 1.43e-04 | 322.69 ms | 52.3% bf16 MFU | 1624064 tok/s
step 13447/19560 | loss 3.364026 (+0.45z)| norm 0.2826 (+0.14z)| lr 1.43e-04 | 322.83 ms | 52.3% bf16 MFU | 1624063 tok/s
step 13448/19560 | loss 3.304246 (-0.74z)| norm 0.2950 (+0.90z)| lr 1.43e-04 | 322.91 ms | 52.3% bf16 MFU | 1624041 tok/s
step 13449/19560 | loss 3.360997 (+0.39z)| norm 0.2708 (-0.59z)| lr 1.43e-04 | 322.54 ms | 52.3% bf16 MFU | 1624112 tok/s
step 13450/19560 | loss 3.391961 (+1.01z)| norm 0.2714 (-0.54z)| lr 1.42e-04 | 322.98 ms | 52.3% bf16 MFU | 1624070 tok/s
step 13451/19560 | loss 3.299464 (-0.84z)| norm 0.2802 (-0.00z)| lr 1.42e-04 | 322.95 ms | 52.3% bf16 MFU | 1624038 tok/s
step 13452/19560 | loss 3.302936 (-0.77z)| norm 0.2707 (-0.58z)| lr 1.42e-04 | 322.95 ms | 52.3% bf16 MFU | 1624008 tok/s
step 13453/19560 | loss 3.331864 (-0.17z)| norm 0.2687 (-0.70z)| lr 1.42e-04 | 323.28 ms | 52.2% bf16 MFU | 1623896 tok/s
step 13454/19560 | loss 3.328482 (-0.24z)| norm 0.2805 (+0.03z)| lr 1.42e-04 | 322.12 ms | 52.4% bf16 MFU | 1624082 tok/s
step 13455/19560 | loss 3.250924 (-1.82z)| norm 0.2610 (-1.17z)| lr 1.42e-04 | 322.60 ms | 52.3% bf16 MFU | 1624138 tok/s
step 13456/19560 | loss 3.305270 (-0.69z)| norm 0.2904 (+0.65z)| lr 1.42e-04 | 322.93 ms | 52.3% bf16 MFU | 1624107 tok/s
step 13457/19560 | loss 3.372988 (+0.71z)| norm 0.2762 (-0.22z)| lr 1.42e-04 | 322.66 ms | 52.3% bf16 MFU | 1624147 tok/s
step 13458/19560 | loss 3.286178 (-1.08z)| norm 0.2692 (-0.64z)| lr 1.42e-04 | 322.46 ms | 52.3% bf16 MFU | 1624234 tok/s
step 13459/19560 | loss 3.319429 (-0.39z)| norm 0.2689 (-0.67z)| lr 1.42e-04 | 323.49 ms | 52.2% bf16 MFU | 1624058 tok/s
step 13460/19560 | loss 3.383361 (+0.91z)| norm 0.2791 (-0.03z)| lr 1.42e-04 | 322.18 ms | 52.4% bf16 MFU | 1624220 tok/s
step 13461/19560 | loss 3.323150 (-0.32z)| norm 0.2717 (-0.49z)| lr 1.42e-04 | 322.87 ms | 52.3% bf16 MFU | 1624200 tok/s
step 13462/19560 | loss 3.337984 (-0.01z)| norm 0.2797 (+0.01z)| lr 1.42e-04 | 322.79 ms | 52.3% bf16 MFU | 1624201 tok/s
step 13463/19560 | loss 3.351117 (+0.26z)| norm 0.2631 (-1.01z)| lr 1.42e-04 | 322.36 ms | 52.4% bf16 MFU | 1624310 tok/s
step 13464/19560 | loss 3.333121 (-0.10z)| norm 0.3174 (+2.31z)| lr 1.42e-04 | 322.66 ms | 52.3% bf16 MFU | 1624339 tok/s
step 13465/19560 | loss 3.300390 (-0.78z)| norm 0.2624 (-1.04z)| lr 1.42e-04 | 323.56 ms | 52.2% bf16 MFU | 1624141 tok/s
step 13466/19560 | loss 3.329214 (-0.18z)| norm 0.2836 (+0.28z)| lr 1.42e-04 | 322.38 ms | 52.4% bf16 MFU | 1624249 tok/s
step 13467/19560 | loss 3.259771 (-1.59z)| norm 0.2669 (-0.76z)| lr 1.42e-04 | 322.95 ms | 52.3% bf16 MFU | 1624209 tok/s
step 13468/19560 | loss 3.308601 (-0.59z)| norm 0.2724 (-0.41z)| lr 1.42e-04 | 322.81 ms | 52.3% bf16 MFU | 1624206 tok/s
step 13469/19560 | loss 3.331746 (-0.11z)| norm 0.2676 (-0.70z)| lr 1.42e-04 | 322.89 ms | 52.3% bf16 MFU | 1624182 tok/s
step 13470/19560 | loss 3.382826 (+0.92z)| norm 0.2894 (+0.65z)| lr 1.42e-04 | 322.82 ms | 52.3% bf16 MFU | 1624176 tok/s
step 13471/19560 | loss 3.318268 (-0.41z)| norm 0.3024 (+1.44z)| lr 1.42e-04 | 322.56 ms | 52.3% bf16 MFU | 1624236 tok/s
step 13472/19560 | loss 3.273623 (-1.31z)| norm 0.2634 (-0.97z)| lr 1.42e-04 | 323.18 ms | 52.2% bf16 MFU | 1624137 tok/s
step 13473/19560 | loss 3.341900 (+0.09z)| norm 0.3229 (+2.61z)| lr 1.41e-04 | 322.85 ms | 52.3% bf16 MFU | 1624126 tok/s
step 13474/19560 | loss 3.311180 (-0.54z)| norm 0.2855 (+0.36z)| lr 1.41e-04 | 322.44 ms | 52.3% bf16 MFU | 1624219 tok/s
step 13475/19560 | loss 3.337524 (-0.01z)| norm 0.2781 (-0.09z)| lr 1.41e-04 | 323.11 ms | 52.2% bf16 MFU | 1624141 tok/s
step 13476/19560 | loss 3.309378 (-0.57z)| norm 0.3156 (+2.12z)| lr 1.41e-04 | 322.53 ms | 52.3% bf16 MFU | 1624212 tok/s
step 13477/19560 | loss 3.351689 (+0.31z)| norm 0.2776 (-0.14z)| lr 1.41e-04 | 323.19 ms | 52.2% bf16 MFU | 1624112 tok/s
step 13478/19560 | loss 3.315226 (-0.44z)| norm 0.3475 (+3.77z)| lr 1.41e-04 | 322.87 ms | 52.3% bf16 MFU | 1624098 tok/s
step 13479/19560 | loss 3.350393 (+0.32z)| norm 0.3122 (+1.76z)| lr 1.41e-04 | 322.35 ms | 52.4% bf16 MFU | 1624215 tok/s
step 13480/19560 | loss 3.334234 (-0.02z)| norm 0.3139 (+1.81z)| lr 1.41e-04 | 322.71 ms | 52.3% bf16 MFU | 1624236 tok/s
step 13481/19560 | loss 3.405134 (+1.47z)| norm 0.3095 (+1.54z)| lr 1.41e-04 | 322.69 ms | 52.3% bf16 MFU | 1624261 tok/s
step 13482/19560 | loss 3.400102 (+1.34z)| norm 0.2835 (+0.11z)| lr 1.41e-04 | 322.99 ms | 52.3% bf16 MFU | 1624209 tok/s
step 13483/19560 | loss 3.270660 (-1.38z)| norm 0.2980 (+0.89z)| lr 1.41e-04 | 323.08 ms | 52.2% bf16 MFU | 1624138 tok/s
step 13484/19560 | loss 3.347564 (+0.24z)| norm 0.2726 (-0.50z)| lr 1.41e-04 | 322.56 ms | 52.3% bf16 MFU | 1624201 tok/s
step 13485/19560 | loss 3.337035 (+0.02z)| norm 0.2708 (-0.60z)| lr 1.41e-04 | 323.07 ms | 52.2% bf16 MFU | 1624131 tok/s
step 13486/19560 | loss 3.319358 (-0.35z)| norm 0.2792 (-0.14z)| lr 1.41e-04 | 323.03 ms | 52.2% bf16 MFU | 1624077 tok/s
step 13487/19560 | loss 3.315124 (-0.44z)| norm 0.2737 (-0.44z)| lr 1.41e-04 | 323.42 ms | 52.2% bf16 MFU | 1623928 tok/s
step 13488/19560 | loss 3.363985 (+0.58z)| norm 0.2533 (-1.55z)| lr 1.41e-04 | 322.11 ms | 52.4% bf16 MFU | 1624115 tok/s
step 13489/19560 | loss 3.487486 (+3.09z)| norm 0.2854 (+0.20z)| lr 1.41e-04 | 322.47 ms | 52.3% bf16 MFU | 1624202 tok/s
step 13490/19560 | loss 3.289455 (-0.96z)| norm 0.2715 (-0.56z)| lr 1.41e-04 | 323.05 ms | 52.2% bf16 MFU | 1624138 tok/s
step 13491/19560 | loss 3.294957 (-0.84z)| norm 0.2819 (+0.01z)| lr 1.41e-04 | 322.02 ms | 52.4% bf16 MFU | 1624337 tok/s
step 13492/19560 | loss 3.336154 (+0.01z)| norm 0.2700 (-0.64z)| lr 1.41e-04 | 323.52 ms | 52.2% bf16 MFU | 1624150 tok/s
step 13493/19560 | loss 3.306384 (-0.60z)| norm 0.2625 (-1.04z)| lr 1.41e-04 | 323.47 ms | 52.2% bf16 MFU | 1623983 tok/s
step 13494/19560 | loss 3.251548 (-1.72z)| norm 0.2747 (-0.38z)| lr 1.41e-04 | 322.42 ms | 52.3% bf16 MFU | 1624090 tok/s
step 13495/19560 | loss 3.335159 (+0.02z)| norm 0.2748 (-0.36z)| lr 1.41e-04 | 322.77 ms | 52.3% bf16 MFU | 1624103 tok/s
step 13496/19560 | loss 3.337528 (+0.07z)| norm 0.2701 (-0.62z)| lr 1.41e-04 | 322.43 ms | 52.3% bf16 MFU | 1624201 tok/s
step 13497/19560 | loss 3.329214 (-0.10z)| norm 0.2883 (+0.37z)| lr 1.40e-04 | 323.19 ms | 52.2% bf16 MFU | 1624102 tok/s
step 13498/19560 | loss 3.239049 (-1.96z)| norm 0.2696 (-0.65z)| lr 1.40e-04 | 323.05 ms | 52.2% bf16 MFU | 1624043 tok/s
step 13499/19560 | loss 3.314416 (-0.39z)| norm 0.2809 (-0.02z)| lr 1.40e-04 | 322.49 ms | 52.3% bf16 MFU | 1624127 tok/s
step 13500/19560 | loss 3.336457 (+0.08z)| norm 0.2852 (+0.21z)| lr 1.40e-04 | 322.83 ms | 52.3% bf16 MFU | 1624123 tok/s
val loss 3.325452
laSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluaevaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3011/10042 = 0.299841
step 13501/19560 | loss 3.327560 (-0.11z)| norm 0.2679 (-0.73z)| lr 1.40e-04 | 322.74 ms | 52.3% bf16 MFU | 1624141 tok/s
step 13502/19560 | loss 3.351025 (+0.38z)| norm 0.2592 (-1.20z)| lr 1.40e-04 | 322.67 ms | 52.3% bf16 MFU | 1624176 tok/s
step 13503/19560 | loss 3.439312 (+2.18z)| norm 0.3001 (+1.03z)| lr 1.40e-04 | 323.53 ms | 52.2% bf16 MFU | 1623992 tok/s
step 13504/19560 | loss 3.291939 (-0.88z)| norm 0.2737 (-0.42z)| lr 1.40e-04 | 322.45 ms | 52.3% bf16 MFU | 1624089 tok/s
step 13505/19560 | loss 3.306277 (-0.61z)| norm 0.2651 (-0.88z)| lr 1.40e-04 | 322.60 ms | 52.3% bf16 MFU | 1624143 tok/s
step 13506/19560 | loss 3.341771 (+0.24z)| norm 0.2581 (-1.25z)| lr 1.40e-04 | 322.92 ms | 52.3% bf16 MFU | 1624114 tok/s
step 13507/19560 | loss 3.332553 (+0.02z)| norm 0.2694 (-0.64z)| lr 1.40e-04 | 322.61 ms | 52.3% bf16 MFU | 1624167 tok/s
step 13508/19560 | loss 3.306077 (-0.60z)| norm 0.2795 (-0.08z)| lr 1.40e-04 | 322.82 ms | 52.3% bf16 MFU | 1624163 tok/s
step 13509/19560 | loss 3.299871 (-0.73z)| norm 0.2623 (-1.02z)| lr 1.40e-04 | 322.84 ms | 52.3% bf16 MFU | 1624154 tok/s
step 13510/19560 | loss 3.353617 (+0.56z)| norm 0.2619 (-1.04z)| lr 1.40e-04 | 322.95 ms | 52.3% bf16 MFU | 1624119 tok/s
step 13511/19560 | loss 3.337765 (+0.17z)| norm 0.2584 (-1.23z)| lr 1.40e-04 | 322.95 ms | 52.3% bf16 MFU | 1624086 tok/s
step 13512/19560 | loss 3.356378 (+0.62z)| norm 0.2907 (+0.53z)| lr 1.40e-04 | 323.21 ms | 52.2% bf16 MFU | 1623988 tok/s
step 13513/19560 | loss 3.300133 (-0.76z)| norm 0.3166 (+1.90z)| lr 1.40e-04 | 323.09 ms | 52.2% bf16 MFU | 1623925 tok/s
step 13514/19560 | loss 3.363967 (+0.81z)| norm 0.3074 (+1.38z)| lr 1.40e-04 | 322.75 ms | 52.3% bf16 MFU | 1623950 tok/s
step 13515/19560 | loss 3.325037 (-0.15z)| norm 0.3086 (+1.42z)| lr 1.40e-04 | 323.33 ms | 52.2% bf16 MFU | 1623829 tok/s
step 13516/19560 | loss 3.355296 (+0.61z)| norm 0.3063 (+1.28z)| lr 1.40e-04 | 322.67 ms | 52.3% bf16 MFU | 1623880 tok/s
step 13517/19560 | loss 3.278906 (-1.27z)| norm 0.3123 (+1.56z)| lr 1.40e-04 | 322.69 ms | 52.3% bf16 MFU | 1623923 tok/s
step 13518/19560 | loss 3.279961 (-1.23z)| norm 0.2878 (+0.27z)| lr 1.40e-04 | 322.92 ms | 52.3% bf16 MFU | 1623905 tok/s
step 13519/19560 | loss 3.303179 (-0.65z)| norm 0.2968 (+0.73z)| lr 1.40e-04 | 323.08 ms | 52.2% bf16 MFU | 1623849 tok/s
step 13520/19560 | loss 3.404385 (+1.80z)| norm 0.2945 (+0.60z)| lr 1.39e-04 | 323.30 ms | 52.2% bf16 MFU | 1623740 tok/s
step 13521/19560 | loss 3.269141 (-1.49z)| norm 0.2995 (+0.85z)| lr 1.39e-04 | 323.43 ms | 52.2% bf16 MFU | 1623606 tok/s
step 13522/19560 | loss 3.352109 (+0.58z)| norm 0.2937 (+0.56z)| lr 1.39e-04 | 323.32 ms | 52.2% bf16 MFU | 1623505 tok/s
step 13523/19560 | loss 3.361570 (+0.81z)| norm 0.2928 (+0.50z)| lr 1.39e-04 | 323.21 ms | 52.2% bf16 MFU | 1623436 tok/s
step 13524/19560 | loss 3.336541 (+0.16z)| norm 0.2942 (+0.57z)| lr 1.39e-04 | 323.01 ms | 52.3% bf16 MFU | 1623422 tok/s
step 13525/19560 | loss 3.358854 (+0.73z)| norm 0.2670 (-0.89z)| lr 1.39e-04 | 323.05 ms | 52.2% bf16 MFU | 1623399 tok/s
step 13526/19560 | loss 3.308054 (-0.56z)| norm 0.2963 (+0.66z)| lr 1.39e-04 | 323.36 ms | 52.2% bf16 MFU | 1623297 tok/s
step 13527/19560 | loss 3.342258 (+0.30z)| norm 0.2796 (-0.24z)| lr 1.39e-04 | 322.91 ms | 52.3% bf16 MFU | 1623314 tok/s
step 13528/19560 | loss 3.378618 (+1.22z)| norm 0.2963 (+0.66z)| lr 1.39e-04 | 323.09 ms | 52.2% bf16 MFU | 1623286 tok/s
step 13529/19560 | loss 3.414023 (+2.06z)| norm 0.3001 (+0.87z)| lr 1.39e-04 | 322.57 ms | 52.3% bf16 MFU | 1623389 tok/s
step 13530/19560 | loss 3.345017 (+0.34z)| norm 0.3126 (+1.53z)| lr 1.39e-04 | 322.42 ms | 52.3% bf16 MFU | 1623524 tok/s
step 13531/19560 | loss 3.325356 (-0.14z)| norm 0.2784 (-0.31z)| lr 1.39e-04 | 323.27 ms | 52.2% bf16 MFU | 1623439 tok/s
step 13532/19560 | loss 3.374982 (+1.10z)| norm 0.2995 (+0.82z)| lr 1.39e-04 | 323.18 ms | 52.2% bf16 MFU | 1623381 tok/s
step 13533/19560 | loss 3.284274 (-1.18z)| norm 0.3061 (+1.16z)| lr 1.39e-04 | 322.88 ms | 52.3% bf16 MFU | 1623402 tok/s
step 13534/19560 | loss 3.304409 (-0.66z)| norm 0.2959 (+0.63z)| lr 1.39e-04 | 322.61 ms | 52.3% bf16 MFU | 1623488 tok/s
step 13535/19560 | loss 3.290236 (-1.00z)| norm 0.3132 (+1.54z)| lr 1.39e-04 | 322.96 ms | 52.3% bf16 MFU | 1623484 tok/s
step 13536/19560 | loss 3.329749 (-0.01z)| norm 0.2971 (+0.67z)| lr 1.39e-04 | 323.16 ms | 52.2% bf16 MFU | 1623429 tok/s
step 13537/19560 | loss 3.300566 (-0.74z)| norm 0.2785 (-0.34z)| lr 1.39e-04 | 323.26 ms | 52.2% bf16 MFU | 1623352 tok/s
step 13538/19560 | loss 3.326464 (-0.10z)| norm 0.2755 (-0.49z)| lr 1.39e-04 | 323.07 ms | 52.2% bf16 MFU | 1623325 tok/s
step 13539/19560 | loss 3.267157 (-1.55z)| norm 0.2896 (+0.27z)| lr 1.39e-04 | 323.15 ms | 52.2% bf16 MFU | 1623281 tok/s
step 13540/19560 | loss 3.400898 (+1.74z)| norm 0.3291 (+2.37z)| lr 1.39e-04 | 322.65 ms | 52.3% bf16 MFU | 1623364 tok/s
step 13541/19560 | loss 3.327194 (-0.07z)| norm 0.2809 (-0.21z)| lr 1.39e-04 | 323.07 ms | 52.2% bf16 MFU | 1623338 tok/s
step 13542/19560 | loss 3.334133 (+0.10z)| norm 0.3112 (+1.39z)| lr 1.39e-04 | 323.51 ms | 52.2% bf16 MFU | 1623202 tok/s
step 13543/19560 | loss 3.282119 (-1.18z)| norm 0.2746 (-0.55z)| lr 1.39e-04 | 322.99 ms | 52.3% bf16 MFU | 1623204 tok/s
step 13544/19560 | loss 3.342654 (+0.33z)| norm 0.3119 (+1.45z)| lr 1.38e-04 | 322.62 ms | 52.3% bf16 MFU | 1623299 tok/s
step 13545/19560 | loss 3.295046 (-0.85z)| norm 0.2732 (-0.63z)| lr 1.38e-04 | 323.57 ms | 52.2% bf16 MFU | 1623150 tok/s
step 13546/19560 | loss 3.368838 (+1.00z)| norm 0.3042 (+1.02z)| lr 1.38e-04 | 322.66 ms | 52.3% bf16 MFU | 1623237 tok/s
step 13547/19560 | loss 3.309032 (-0.49z)| norm 0.3006 (+0.81z)| lr 1.38e-04 | 323.00 ms | 52.3% bf16 MFU | 1623235 tok/s
step 13548/19560 | loss 3.337713 (+0.23z)| norm 0.2807 (-0.24z)| lr 1.38e-04 | 323.74 ms | 52.1% bf16 MFU | 1623046 tok/s
step 13549/19560 | loss 3.384156 (+1.37z)| norm 0.2900 (+0.26z)| lr 1.38e-04 | 322.92 ms | 52.3% bf16 MFU | 1623072 tok/s
step 13550/19560 | loss 3.345390 (+0.40z)| norm 0.2948 (+0.51z)| lr 1.38e-04 | 323.34 ms | 52.2% bf16 MFU | 1622993 tok/s
step 13551/19560 | loss 3.387805 (+1.45z)| norm 0.2749 (-0.56z)| lr 1.38e-04 | 323.55 ms | 52.2% bf16 MFU | 1622864 tok/s
step 13552/19560 | loss 3.355412 (+0.63z)| norm 0.3087 (+1.27z)| lr 1.38e-04 | 323.35 ms | 52.2% bf16 MFU | 1622792 tok/s
step 13553/19560 | loss 3.320156 (-0.24z)| norm 0.2770 (-0.45z)| lr 1.38e-04 | 323.41 ms | 52.2% bf16 MFU | 1622709 tok/s
step 13554/19560 | loss 3.443106 (+2.84z)| norm 0.2697 (-0.84z)| lr 1.38e-04 | 324.26 ms | 52.0% bf16 MFU | 1622417 tok/s
step 13555/19560 | loss 3.334356 (+0.12z)| norm 0.2784 (-0.37z)| lr 1.38e-04 | 323.06 ms | 52.2% bf16 MFU | 1622440 tok/s
step 13556/19560 | loss 3.272827 (-1.41z)| norm 0.2760 (-0.50z)| lr 1.38e-04 | 323.46 ms | 52.2% bf16 MFU | 1622362 tok/s
step 13557/19560 | loss 3.299808 (-0.73z)| norm 0.2581 (-1.45z)| lr 1.38e-04 | 323.63 ms | 52.2% bf16 MFU | 1622246 tok/s
step 13558/19560 | loss 3.325429 (-0.10z)| norm 0.2604 (-1.30z)| lr 1.38e-04 | 323.48 ms | 52.2% bf16 MFU | 1622172 tok/s
step 13559/19560 | loss 3.303550 (-0.65z)| norm 0.2610 (-1.25z)| lr 1.38e-04 | 323.38 ms | 52.2% bf16 MFU | 1622126 tok/s
step 13560/19560 | loss 3.295396 (-0.86z)| norm 0.2602 (-1.27z)| lr 1.38e-04 | 323.25 ms | 52.2% bf16 MFU | 1622116 tok/s
step 13561/19560 | loss 3.388424 (+1.45z)| norm 0.2695 (-0.77z)| lr 1.38e-04 | 322.95 ms | 52.3% bf16 MFU | 1622183 tok/s
step 13562/19560 | loss 3.376046 (+1.13z)| norm 0.2688 (-0.82z)| lr 1.38e-04 | 323.51 ms | 52.2% bf16 MFU | 1622105 tok/s
step 13563/19560 | loss 3.341413 (+0.26z)| norm 0.2582 (-1.38z)| lr 1.38e-04 | 323.52 ms | 52.2% bf16 MFU | 1622028 tok/s
step 13564/19560 | loss 3.276009 (-1.35z)| norm 0.2732 (-0.55z)| lr 1.38e-04 | 323.13 ms | 52.2% bf16 MFU | 1622053 tok/s
step 13565/19560 | loss 3.269097 (-1.50z)| norm 0.2637 (-1.07z)| lr 1.38e-04 | 323.13 ms | 52.2% bf16 MFU | 1622076 tok/s
step 13566/19560 | loss 3.347245 (+0.42z)| norm 0.2793 (-0.21z)| lr 1.38e-04 | 323.20 ms | 52.2% bf16 MFU | 1622080 tok/s
step 13567/19560 | loss 3.388440 (+1.41z)| norm 0.2922 (+0.49z)| lr 1.38e-04 | 323.14 ms | 52.2% bf16 MFU | 1622100 tok/s
step 13568/19560 | loss 3.317989 (-0.32z)| norm 0.2844 (+0.05z)| lr 1.37e-04 | 323.96 ms | 52.1% bf16 MFU | 1621914 tok/s
step 13569/19560 | loss 3.294068 (-0.91z)| norm 0.2920 (+0.46z)| lr 1.37e-04 | 322.90 ms | 52.3% bf16 MFU | 1622002 tok/s
step 13570/19560 | loss 3.363969 (+0.80z)| norm 0.2668 (-0.94z)| lr 1.37e-04 | 322.87 ms | 52.3% bf16 MFU | 1622093 tok/s
step 13571/19560 | loss 3.330364 (-0.02z)| norm 0.2651 (-1.03z)| lr 1.37e-04 | 323.94 ms | 52.1% bf16 MFU | 1621911 tok/s
step 13572/19560 | loss 3.308720 (-0.55z)| norm 0.2777 (-0.34z)| lr 1.37e-04 | 323.22 ms | 52.2% bf16 MFU | 1621920 tok/s
step 13573/19560 | loss 3.325483 (-0.13z)| norm 0.2722 (-0.64z)| lr 1.37e-04 | 323.04 ms | 52.2% bf16 MFU | 1621974 tok/s
step 13574/19560 | loss 3.297391 (-0.82z)| norm 0.2657 (-0.99z)| lr 1.37e-04 | 323.62 ms | 52.2% bf16 MFU | 1621879 tok/s
step 13575/19560 | loss 3.343586 (+0.32z)| norm 0.2748 (-0.47z)| lr 1.37e-04 | 322.88 ms | 52.3% bf16 MFU | 1621974 tok/s
step 13576/19560 | loss 3.426519 (+2.29z)| norm 0.2909 (+0.43z)| lr 1.37e-04 | 323.40 ms | 52.2% bf16 MFU | 1621933 tok/s
step 13577/19560 | loss 3.285349 (-1.10z)| norm 0.2867 (+0.19z)| lr 1.37e-04 | 323.32 ms | 52.2% bf16 MFU | 1621916 tok/s
step 13578/19560 | loss 3.310770 (-0.47z)| norm 0.2944 (+0.61z)| lr 1.37e-04 | 322.45 ms | 52.3% bf16 MFU | 1622117 tok/s
step 13579/19560 | loss 3.346356 (+0.38z)| norm 0.2930 (+0.53z)| lr 1.37e-04 | 323.06 ms | 52.2% bf16 MFU | 1622155 tok/s
step 13580/19560 | loss 3.358038 (+0.66z)| norm 0.2998 (+0.89z)| lr 1.37e-04 | 322.77 ms | 52.3% bf16 MFU | 1622263 tok/s
step 13581/19560 | loss 3.310095 (-0.51z)| norm 0.2890 (+0.28z)| lr 1.37e-04 | 322.95 ms | 52.3% bf16 MFU | 1622321 tok/s
step 13582/19560 | loss 3.348957 (+0.43z)| norm 0.2809 (-0.17z)| lr 1.37e-04 | 323.45 ms | 52.2% bf16 MFU | 1622250 tok/s
step 13583/19560 | loss 3.349916 (+0.44z)| norm 0.2652 (-1.06z)| lr 1.37e-04 | 322.74 ms | 52.3% bf16 MFU | 1622364 tok/s
step 13584/19560 | loss 3.285688 (-1.13z)| norm 0.2844 (+0.02z)| lr 1.37e-04 | 322.69 ms | 52.3% bf16 MFU | 1622482 tok/s
step 13585/19560 | loss 3.432811 (+2.42z)| norm 0.2715 (-0.70z)| lr 1.37e-04 | 322.73 ms | 52.3% bf16 MFU | 1622586 tok/s
step 13586/19560 | loss 3.269520 (-1.50z)| norm 0.2679 (-0.90z)| lr 1.37e-04 | 323.19 ms | 52.2% bf16 MFU | 1622568 tok/s
step 13587/19560 | loss 3.385551 (+1.26z)| norm 0.2744 (-0.54z)| lr 1.37e-04 | 323.22 ms | 52.2% bf16 MFU | 1622543 tok/s
step 13588/19560 | loss 3.378325 (+1.09z)| norm 0.2882 (+0.23z)| lr 1.37e-04 | 322.61 ms | 52.3% bf16 MFU | 1622674 tok/s
step 13589/19560 | loss 3.312015 (-0.49z)| norm 0.2847 (+0.03z)| lr 1.37e-04 | 322.73 ms | 52.3% bf16 MFU | 1622766 tok/s
step 13590/19560 | loss 3.346784 (+0.34z)| norm 0.2675 (-0.93z)| lr 1.37e-04 | 323.13 ms | 52.2% bf16 MFU | 1622753 tok/s
step 13591/19560 | loss 3.328646 (-0.09z)| norm 0.2676 (-0.93z)| lr 1.37e-04 | 322.63 ms | 52.3% bf16 MFU | 1622868 tok/s
step 13592/19560 | loss 3.329097 (-0.08z)| norm 0.2757 (-0.46z)| lr 1.36e-04 | 323.26 ms | 52.2% bf16 MFU | 1622820 tok/s
step 13593/19560 | loss 3.330085 (-0.06z)| norm 0.2646 (-1.10z)| lr 1.36e-04 | 323.31 ms | 52.2% bf16 MFU | 1622760 tok/s
step 13594/19560 | loss 3.378780 (+1.09z)| norm 0.2586 (-1.42z)| lr 1.36e-04 | 322.36 ms | 52.4% bf16 MFU | 1622942 tok/s
step 13595/19560 | loss 3.250936 (-1.94z)| norm 0.2767 (-0.40z)| lr 1.36e-04 | 322.76 ms | 52.3% bf16 MFU | 1623013 tok/s
step 13596/19560 | loss 3.287750 (-1.06z)| norm 0.2570 (-1.50z)| lr 1.36e-04 | 323.91 ms | 52.1% bf16 MFU | 1622792 tok/s
step 13597/19560 | loss 3.330168 (-0.06z)| norm 0.2799 (-0.21z)| lr 1.36e-04 | 323.02 ms | 52.2% bf16 MFU | 1622808 tok/s
step 13598/19560 | loss 3.369594 (+0.88z)| norm 0.2850 (+0.08z)| lr 1.36e-04 | 322.37 ms | 52.4% bf16 MFU | 1622984 tok/s
step 13599/19560 | loss 3.361287 (+0.67z)| norm 0.2705 (-0.73z)| lr 1.36e-04 | 323.17 ms | 52.2% bf16 MFU | 1622953 tok/s
step 13600/19560 | loss 3.334436 (+0.03z)| norm 0.2746 (-0.51z)| lr 1.36e-04 | 323.27 ms | 52.2% bf16 MFU | 1622898 tok/s
step 13601/19560 | loss 3.365677 (+0.76z)| norm 0.2776 (-0.32z)| lr 1.36e-04 | 322.70 ms | 52.3% bf16 MFU | 1622986 tok/s
step 13602/19560 | loss 3.332706 (-0.02z)| norm 0.2722 (-0.63z)| lr 1.36e-04 | 323.31 ms | 52.2% bf16 MFU | 1622918 tok/s
step 13603/19560 | loss 3.251107 (-1.92z)| norm 0.2629 (-1.15z)| lr 1.36e-04 | 322.74 ms | 52.3% bf16 MFU | 1622996 tok/s
step 13604/19560 | loss 3.331508 (-0.04z)| norm 0.2821 (-0.03z)| lr 1.36e-04 | 323.05 ms | 52.2% bf16 MFU | 1622993 tok/s
step 13605/19560 | loss 3.335992 (+0.07z)| norm 0.2681 (-0.85z)| lr 1.36e-04 | 322.80 ms | 52.3% bf16 MFU | 1623053 tok/s
step 13606/19560 | loss 3.323848 (-0.22z)| norm 0.2835 (+0.09z)| lr 1.36e-04 | 322.65 ms | 52.3% bf16 MFU | 1623147 tok/s
step 13607/19560 | loss 3.327574 (-0.13z)| norm 0.2829 (+0.07z)| lr 1.36e-04 | 322.79 ms | 52.3% bf16 MFU | 1623201 tok/s
step 13608/19560 | loss 3.357059 (+0.56z)| norm 0.2724 (-0.58z)| lr 1.36e-04 | 323.13 ms | 52.2% bf16 MFU | 1623167 tok/s
step 13609/19560 | loss 3.305533 (-0.64z)| norm 0.2970 (+1.01z)| lr 1.36e-04 | 322.99 ms | 52.3% bf16 MFU | 1623171 tok/s
step 13610/19560 | loss 3.445098 (+2.62z)| norm 0.2755 (-0.38z)| lr 1.36e-04 | 322.86 ms | 52.3% bf16 MFU | 1623208 tok/s
step 13611/19560 | loss 3.397798 (+1.49z)| norm 0.2833 (+0.14z)| lr 1.36e-04 | 322.50 ms | 52.3% bf16 MFU | 1623332 tok/s
step 13612/19560 | loss 3.387586 (+1.24z)| norm 0.2717 (-0.62z)| lr 1.36e-04 | 322.89 ms | 52.3% bf16 MFU | 1623353 tok/s
step 13613/19560 | loss 3.389348 (+1.27z)| norm 0.2712 (-0.65z)| lr 1.36e-04 | 323.19 ms | 52.2% bf16 MFU | 1623296 tok/s
step 13614/19560 | loss 3.363147 (+0.65z)| norm 0.2788 (-0.16z)| lr 1.36e-04 | 323.00 ms | 52.3% bf16 MFU | 1623291 tok/s
step 13615/19560 | loss 3.395801 (+1.38z)| norm 0.2947 (+0.87z)| lr 1.36e-04 | 323.12 ms | 52.2% bf16 MFU | 1623256 tok/s
step 13616/19560 | loss 3.374963 (+0.90z)| norm 0.2942 (+0.82z)| lr 1.35e-04 | 322.84 ms | 52.3% bf16 MFU | 1623292 tok/s
step 13617/19560 | loss 3.348601 (+0.34z)| norm 0.3075 (+1.66z)| lr 1.35e-04 | 323.10 ms | 52.2% bf16 MFU | 1623262 tok/s
step 13618/19560 | loss 3.306411 (-0.68z)| norm 0.2740 (-0.51z)| lr 1.35e-04 | 322.41 ms | 52.3% bf16 MFU | 1623407 tok/s
step 13619/19560 | loss 3.298054 (-0.88z)| norm 0.3071 (+1.61z)| lr 1.35e-04 | 323.07 ms | 52.2% bf16 MFU | 1623378 tok/s
step 13620/19560 | loss 3.347902 (+0.32z)| norm 0.2777 (-0.28z)| lr 1.35e-04 | 322.70 ms | 52.3% bf16 MFU | 1623444 tok/s
step 13621/19560 | loss 3.351382 (+0.39z)| norm 0.2962 (+0.89z)| lr 1.35e-04 | 322.64 ms | 52.3% bf16 MFU | 1623522 tok/s
step 13622/19560 | loss 3.375439 (+0.96z)| norm 0.2816 (-0.05z)| lr 1.35e-04 | 322.69 ms | 52.3% bf16 MFU | 1623583 tok/s
step 13623/19560 | loss 3.353936 (+0.43z)| norm 0.3036 (+1.34z)| lr 1.35e-04 | 322.76 ms | 52.3% bf16 MFU | 1623623 tok/s
step 13624/19560 | loss 3.300499 (-0.86z)| norm 0.2847 (+0.12z)| lr 1.35e-04 | 322.92 ms | 52.3% bf16 MFU | 1623620 tok/s
step 13625/19560 | loss 3.355102 (+0.46z)| norm 0.2662 (-1.05z)| lr 1.35e-04 | 322.72 ms | 52.3% bf16 MFU | 1623670 tok/s
step 13626/19560 | loss 3.364964 (+0.69z)| norm 0.2945 (+0.75z)| lr 1.35e-04 | 323.28 ms | 52.2% bf16 MFU | 1623576 tok/s
step 13627/19560 | loss 3.257204 (-1.94z)| norm 0.2911 (+0.53z)| lr 1.35e-04 | 323.05 ms | 52.2% bf16 MFU | 1623542 tok/s
step 13628/19560 | loss 3.365954 (+0.71z)| norm 0.3060 (+1.46z)| lr 1.35e-04 | 322.81 ms | 52.3% bf16 MFU | 1623573 tok/s
step 13629/19560 | loss 3.298488 (-0.93z)| norm 0.2864 (+0.21z)| lr 1.35e-04 | 323.15 ms | 52.2% bf16 MFU | 1623514 tok/s
step 13630/19560 | loss 3.337186 (+0.02z)| norm 0.3067 (+1.48z)| lr 1.35e-04 | 322.93 ms | 52.3% bf16 MFU | 1623516 tok/s
step 13631/19560 | loss 3.316965 (-0.46z)| norm 0.2746 (-0.56z)| lr 1.35e-04 | 322.27 ms | 52.4% bf16 MFU | 1623684 tok/s
step 13632/19560 | loss 3.349117 (+0.33z)| norm 0.2860 (+0.16z)| lr 1.35e-04 | 323.02 ms | 52.2% bf16 MFU | 1623655 tok/s
step 13633/19560 | loss 3.560307 (+5.00z)| norm 0.2848 (+0.08z)| lr 1.35e-04 | 322.36 ms | 52.4% bf16 MFU | 1623792 tok/s
step 13634/19560 | loss 3.368132 (+0.67z)| norm 0.2871 (+0.21z)| lr 1.35e-04 | 323.27 ms | 52.2% bf16 MFU | 1623693 tok/s
step 13635/19560 | loss 3.330663 (-0.17z)| norm 0.2752 (-0.57z)| lr 1.35e-04 | 322.70 ms | 52.3% bf16 MFU | 1623743 tok/s
step 13636/19560 | loss 3.334734 (-0.08z)| norm 0.2775 (-0.41z)| lr 1.35e-04 | 322.96 ms | 52.3% bf16 MFU | 1623725 tok/s
step 13637/19560 | loss 3.289046 (-1.11z)| norm 0.2788 (-0.34z)| lr 1.35e-04 | 323.02 ms | 52.2% bf16 MFU | 1623692 tok/s
step 13638/19560 | loss 3.376869 (+0.86z)| norm 0.2774 (-0.44z)| lr 1.35e-04 | 322.47 ms | 52.3% bf16 MFU | 1623800 tok/s
step 13639/19560 | loss 3.346157 (+0.17z)| norm 0.3097 (+1.67z)| lr 1.35e-04 | 322.86 ms | 52.3% bf16 MFU | 1623804 tok/s
step 13640/19560 | loss 3.337537 (-0.02z)| norm 0.2758 (-0.57z)| lr 1.34e-04 | 323.03 ms | 52.2% bf16 MFU | 1623767 tok/s
step 13641/19560 | loss 3.289509 (-1.09z)| norm 0.2905 (+0.43z)| lr 1.34e-04 | 322.45 ms | 52.3% bf16 MFU | 1623876 tok/s
step 13642/19560 | loss 3.281205 (-1.26z)| norm 0.2686 (-1.03z)| lr 1.34e-04 | 323.00 ms | 52.3% bf16 MFU | 1623840 tok/s
step 13643/19560 | loss 3.355716 (+0.40z)| norm 0.3016 (+1.21z)| lr 1.34e-04 | 323.29 ms | 52.2% bf16 MFU | 1623733 tok/s
step 13644/19560 | loss 3.358130 (+0.45z)| norm 0.2863 (+0.18z)| lr 1.34e-04 | 322.44 ms | 52.3% bf16 MFU | 1623847 tok/s
step 13645/19560 | loss 3.326339 (-0.27z)| norm 0.2820 (-0.10z)| lr 1.34e-04 | 322.88 ms | 52.3% bf16 MFU | 1623845 tok/s
step 13646/19560 | loss 3.405047 (+1.47z)| norm 0.2849 (+0.10z)| lr 1.34e-04 | 323.23 ms | 52.2% bf16 MFU | 1623755 tok/s
step 13647/19560 | loss 3.337479 (-0.05z)| norm 0.2593 (-1.65z)| lr 1.34e-04 | 323.22 ms | 52.2% bf16 MFU | 1623671 tok/s
step 13648/19560 | loss 3.277039 (-1.38z)| norm 0.2716 (-0.79z)| lr 1.34e-04 | 322.44 ms | 52.3% bf16 MFU | 1623788 tok/s
step 13649/19560 | loss 3.318874 (-0.45z)| norm 0.2718 (-0.76z)| lr 1.34e-04 | 322.34 ms | 52.4% bf16 MFU | 1623923 tok/s
step 13650/19560 | loss 3.443286 (+2.30z)| norm 0.2810 (-0.11z)| lr 1.34e-04 | 322.74 ms | 52.3% bf16 MFU | 1623951 tok/s
step 13651/19560 | loss 3.330625 (-0.20z)| norm 0.2601 (-1.54z)| lr 1.34e-04 | 322.91 ms | 52.3% bf16 MFU | 1623935 tok/s
step 13652/19560 | loss 3.299245 (-0.88z)| norm 0.2877 (+0.37z)| lr 1.34e-04 | 323.06 ms | 52.2% bf16 MFU | 1623883 tok/s
step 13653/19560 | loss 3.291646 (-1.03z)| norm 0.2666 (-1.09z)| lr 1.34e-04 | 322.05 ms | 52.4% bf16 MFU | 1624088 tok/s
step 13654/19560 | loss 3.410331 (+1.55z)| norm 0.2901 (+0.54z)| lr 1.34e-04 | 323.17 ms | 52.2% bf16 MFU | 1624001 tok/s
step 13655/19560 | loss 3.347671 (+0.18z)| norm 0.2618 (-1.40z)| lr 1.34e-04 | 323.01 ms | 52.2% bf16 MFU | 1623958 tok/s
step 13656/19560 | loss 3.261919 (-1.66z)| norm 0.2846 (+0.18z)| lr 1.34e-04 | 322.52 ms | 52.3% bf16 MFU | 1624039 tok/s
step 13657/19560 | loss 3.310427 (-0.60z)| norm 0.2634 (-1.26z)| lr 1.34e-04 | 323.09 ms | 52.2% bf16 MFU | 1623973 tok/s
step 13658/19560 | loss 3.306569 (-0.67z)| norm 0.2546 (-1.86z)| lr 1.34e-04 | 323.19 ms | 52.2% bf16 MFU | 1623886 tok/s
step 13659/19560 | loss 3.427528 (+1.92z)| norm 0.2927 (+0.78z)| lr 1.34e-04 | 322.51 ms | 52.3% bf16 MFU | 1623975 tok/s
step 13660/19560 | loss 3.348820 (+0.23z)| norm 0.2626 (-1.28z)| lr 1.34e-04 | 323.03 ms | 52.2% bf16 MFU | 1623927 tok/s
step 13661/19560 | loss 3.386419 (+1.03z)| norm 0.2768 (-0.29z)| lr 1.34e-04 | 322.45 ms | 52.3% bf16 MFU | 1624028 tok/s
step 13662/19560 | loss 3.490816 (+3.13z)| norm 0.3065 (+1.77z)| lr 1.34e-04 | 323.22 ms | 52.2% bf16 MFU | 1623931 tok/s
step 13663/19560 | loss 3.437341 (+1.97z)| norm 0.2977 (+1.19z)| lr 1.34e-04 | 322.47 ms | 52.3% bf16 MFU | 1624027 tok/s
step 13664/19560 | loss 3.330607 (-0.22z)| norm 0.2858 (+0.36z)| lr 1.33e-04 | 323.24 ms | 52.2% bf16 MFU | 1623924 tok/s
step 13665/19560 | loss 3.290232 (-1.05z)| norm 0.2749 (-0.42z)| lr 1.33e-04 | 323.23 ms | 52.2% bf16 MFU | 1623830 tok/s
step 13666/19560 | loss 3.328588 (-0.26z)| norm 0.2641 (-1.16z)| lr 1.33e-04 | 322.83 ms | 52.3% bf16 MFU | 1623840 tok/s
step 13667/19560 | loss 3.378339 (+0.75z)| norm 0.2944 (+0.96z)| lr 1.33e-04 | 322.58 ms | 52.3% bf16 MFU | 1623912 tok/s
step 13668/19560 | loss 3.300769 (-0.84z)| norm 0.2981 (+1.29z)| lr 1.33e-04 | 323.19 ms | 52.2% bf16 MFU | 1623828 tok/s
step 13669/19560 | loss 3.272086 (-1.42z)| norm 0.2750 (-0.39z)| lr 1.33e-04 | 322.59 ms | 52.3% bf16 MFU | 1623900 tok/s
step 13670/19560 | loss 3.313466 (-0.56z)| norm 0.2781 (-0.15z)| lr 1.33e-04 | 322.93 ms | 52.3% bf16 MFU | 1623882 tok/s
step 13671/19560 | loss 3.362097 (+0.43z)| norm 0.2904 (+0.75z)| lr 1.33e-04 | 322.79 ms | 52.3% bf16 MFU | 1623899 tok/s
step 13672/19560 | loss 3.322010 (-0.40z)| norm 0.2753 (-0.35z)| lr 1.33e-04 | 322.86 ms | 52.3% bf16 MFU | 1623898 tok/s
step 13673/19560 | loss 3.330117 (-0.24z)| norm 0.2866 (+0.50z)| lr 1.33e-04 | 322.57 ms | 52.3% bf16 MFU | 1623969 tok/s
step 13674/19560 | loss 3.317471 (-0.49z)| norm 0.2692 (-0.82z)| lr 1.33e-04 | 322.87 ms | 52.3% bf16 MFU | 1623962 tok/s
step 13675/19560 | loss 3.268086 (-1.50z)| norm 0.2864 (+0.52z)| lr 1.33e-04 | 323.06 ms | 52.2% bf16 MFU | 1623908 tok/s
step 13676/19560 | loss 3.307734 (-0.68z)| norm 0.2828 (+0.24z)| lr 1.33e-04 | 323.04 ms | 52.2% bf16 MFU | 1623861 tok/s
step 13677/19560 | loss 3.291057 (-1.00z)| norm 0.3095 (+2.27z)| lr 1.33e-04 | 322.60 ms | 52.3% bf16 MFU | 1623928 tok/s
step 13678/19560 | loss 3.303586 (-0.74z)| norm 0.2980 (+1.38z)| lr 1.33e-04 | 322.77 ms | 52.3% bf16 MFU | 1623948 tok/s
step 13679/19560 | loss 3.361298 (+0.45z)| norm 0.3386 (+4.13z)| lr 1.33e-04 | 322.14 ms | 52.4% bf16 MFU | 1624127 tok/s
step 13680/19560 | loss 3.380270 (+0.83z)| norm 0.3375 (+3.85z)| lr 1.33e-04 | 323.26 ms | 52.2% bf16 MFU | 1624015 tok/s
step 13681/19560 | loss 3.303345 (-0.74z)| norm 0.2895 (+0.59z)| lr 1.33e-04 | 322.69 ms | 52.3% bf16 MFU | 1624052 tok/s
step 13682/19560 | loss 3.297071 (-0.86z)| norm 0.3330 (+3.35z)| lr 1.33e-04 | 322.03 ms | 52.4% bf16 MFU | 1624253 tok/s
step 13683/19560 | loss 3.318316 (-0.41z)| norm 0.3029 (+1.38z)| lr 1.33e-04 | 323.47 ms | 52.2% bf16 MFU | 1624083 tok/s
step 13684/19560 | loss 3.311013 (-0.57z)| norm 0.2804 (-0.07z)| lr 1.33e-04 | 322.89 ms | 52.3% bf16 MFU | 1624065 tok/s
step 13685/19560 | loss 3.372054 (+0.69z)| norm 0.3029 (+1.36z)| lr 1.33e-04 | 322.73 ms | 52.3% bf16 MFU | 1624088 tok/s
step 13686/19560 | loss 3.279313 (-1.23z)| norm 0.2897 (+0.50z)| lr 1.33e-04 | 322.20 ms | 52.4% bf16 MFU | 1624246 tok/s
step 13687/19560 | loss 3.311789 (-0.56z)| norm 0.2718 (-0.67z)| lr 1.33e-04 | 323.61 ms | 52.2% bf16 MFU | 1624039 tok/s
step 13688/19560 | loss 3.261241 (-1.60z)| norm 0.2945 (+0.80z)| lr 1.32e-04 | 323.33 ms | 52.2% bf16 MFU | 1623913 tok/s
step 13689/19560 | loss 3.331000 (-0.15z)| norm 0.2879 (+0.36z)| lr 1.32e-04 | 322.59 ms | 52.3% bf16 MFU | 1623979 tok/s
step 13690/19560 | loss 3.349953 (+0.25z)| norm 0.2845 (+0.13z)| lr 1.32e-04 | 323.28 ms | 52.2% bf16 MFU | 1623869 tok/s
step 13691/19560 | loss 3.364311 (+0.55z)| norm 0.2712 (-0.76z)| lr 1.32e-04 | 322.54 ms | 52.3% bf16 MFU | 1623952 tok/s
step 13692/19560 | loss 3.336226 (-0.05z)| norm 0.2837 (+0.06z)| lr 1.32e-04 | 323.04 ms | 52.2% bf16 MFU | 1623904 tok/s
step 13693/19560 | loss 3.275955 (-1.31z)| norm 0.2970 (+0.93z)| lr 1.32e-04 | 323.26 ms | 52.2% bf16 MFU | 1623802 tok/s
step 13694/19560 | loss 3.332779 (-0.12z)| norm 0.2790 (-0.27z)| lr 1.32e-04 | 322.84 ms | 52.3% bf16 MFU | 1623812 tok/s
step 13695/19560 | loss 3.326612 (-0.24z)| norm 0.2736 (-0.62z)| lr 1.32e-04 | 323.23 ms | 52.2% bf16 MFU | 1623723 tok/s
step 13696/19560 | loss 3.349712 (+0.24z)| norm 0.2616 (-1.40z)| lr 1.32e-04 | 322.93 ms | 52.3% bf16 MFU | 1623714 tok/s
step 13697/19560 | loss 3.278766 (-1.24z)| norm 0.2746 (-0.53z)| lr 1.32e-04 | 322.86 ms | 52.3% bf16 MFU | 1623723 tok/s
step 13698/19560 | loss 3.393667 (+1.16z)| norm 0.2715 (-0.74z)| lr 1.32e-04 | 322.63 ms | 52.3% bf16 MFU | 1623790 tok/s
step 13699/19560 | loss 3.273815 (-1.33z)| norm 0.2660 (-1.10z)| lr 1.32e-04 | 322.95 ms | 52.3% bf16 MFU | 1623773 tok/s
step 13700/19560 | loss 3.273969 (-1.31z)| norm 0.2788 (-0.26z)| lr 1.32e-04 | 323.29 ms | 52.2% bf16 MFU | 1623670 tok/s
step 13701/19560 | loss 3.365180 (+0.56z)| norm 0.2665 (-1.07z)| lr 1.32e-04 | 323.01 ms | 52.2% bf16 MFU | 1623642 tok/s
step 13702/19560 | loss 3.309470 (-0.59z)| norm 0.2820 (-0.05z)| lr 1.32e-04 | 323.19 ms | 52.2% bf16 MFU | 1623571 tok/s
step 13703/19560 | loss 3.349674 (+0.24z)| norm 0.2738 (-0.60z)| lr 1.32e-04 | 322.77 ms | 52.3% bf16 MFU | 1623609 tok/s
step 13704/19560 | loss 3.293581 (-0.91z)| norm 0.2723 (-0.69z)| lr 1.32e-04 | 322.35 ms | 52.4% bf16 MFU | 1623752 tok/s
step 13705/19560 | loss 3.292097 (-0.94z)| norm 0.2656 (-1.11z)| lr 1.32e-04 | 323.12 ms | 52.2% bf16 MFU | 1623694 tok/s
step 13706/19560 | loss 3.302092 (-0.73z)| norm 0.2667 (-1.03z)| lr 1.32e-04 | 323.14 ms | 52.2% bf16 MFU | 1623632 tok/s
step 13707/19560 | loss 3.318701 (-0.38z)| norm 0.2870 (+0.32z)| lr 1.32e-04 | 322.99 ms | 52.3% bf16 MFU | 1623612 tok/s
step 13708/19560 | loss 3.277295 (-1.22z)| norm 0.2759 (-0.41z)| lr 1.32e-04 | 322.84 ms | 52.3% bf16 MFU | 1623631 tok/s
step 13709/19560 | loss 3.290828 (-0.93z)| norm 0.2532 (-1.87z)| lr 1.32e-04 | 322.80 ms | 52.3% bf16 MFU | 1623658 tok/s
step 13710/19560 | loss 3.312067 (-0.49z)| norm 0.2727 (-0.58z)| lr 1.32e-04 | 323.06 ms | 52.2% bf16 MFU | 1623618 tok/s
step 13711/19560 | loss 3.304911 (-0.63z)| norm 0.2712 (-0.69z)| lr 1.32e-04 | 323.04 ms | 52.2% bf16 MFU | 1623586 tok/s
step 13712/19560 | loss 3.278140 (-1.18z)| norm 0.2513 (-1.95z)| lr 1.31e-04 | 322.87 ms | 52.3% bf16 MFU | 1623599 tok/s
step 13713/19560 | loss 3.355063 (+0.43z)| norm 0.2776 (-0.25z)| lr 1.31e-04 | 323.09 ms | 52.2% bf16 MFU | 1623556 tok/s
step 13714/19560 | loss 3.297175 (-0.79z)| norm 0.2615 (-1.29z)| lr 1.31e-04 | 322.83 ms | 52.3% bf16 MFU | 1623580 tok/s
step 13715/19560 | loss 3.409823 (+1.57z)| norm 0.2971 (+1.00z)| lr 1.31e-04 | 323.02 ms | 52.2% bf16 MFU | 1623556 tok/s
step 13716/19560 | loss 3.357793 (+0.48z)| norm 0.2905 (+0.57z)| lr 1.31e-04 | 323.49 ms | 52.2% bf16 MFU | 1623414 tok/s
step 13717/19560 | loss 3.311526 (-0.49z)| norm 0.2596 (-1.39z)| lr 1.31e-04 | 323.33 ms | 52.2% bf16 MFU | 1623321 tok/s
step 13718/19560 | loss 3.318915 (-0.33z)| norm 0.2750 (-0.42z)| lr 1.31e-04 | 322.82 ms | 52.3% bf16 MFU | 1623359 tok/s
step 13719/19560 | loss 3.320236 (-0.30z)| norm 0.2736 (-0.51z)| lr 1.31e-04 | 322.77 ms | 52.3% bf16 MFU | 1623408 tok/s
step 13720/19560 | loss 3.346876 (+0.25z)| norm 0.2650 (-1.05z)| lr 1.31e-04 | 323.15 ms | 52.2% bf16 MFU | 1623360 tok/s
step 13721/19560 | loss 3.335070 (+0.01z)| norm 0.2719 (-0.62z)| lr 1.31e-04 | 322.72 ms | 52.3% bf16 MFU | 1623420 tok/s
step 13722/19560 | loss 3.359338 (+0.52z)| norm 0.2738 (-0.51z)| lr 1.31e-04 | 322.95 ms | 52.3% bf16 MFU | 1623422 tok/s
step 13723/19560 | loss 3.288985 (-0.97z)| norm 0.2677 (-0.89z)| lr 1.31e-04 | 323.02 ms | 52.2% bf16 MFU | 1623405 tok/s
step 13724/19560 | loss 3.318907 (-0.35z)| norm 0.2814 (-0.02z)| lr 1.31e-04 | 322.93 ms | 52.3% bf16 MFU | 1623411 tok/s
step 13725/19560 | loss 3.344103 (+0.19z)| norm 0.2754 (-0.41z)| lr 1.31e-04 | 322.65 ms | 52.3% bf16 MFU | 1623487 tok/s
step 13726/19560 | loss 3.349695 (+0.31z)| norm 0.2677 (-0.90z)| lr 1.31e-04 | 323.17 ms | 52.2% bf16 MFU | 1623428 tok/s
step 13727/19560 | loss 3.290787 (-0.93z)| norm 0.2786 (-0.20z)| lr 1.31e-04 | 323.25 ms | 52.2% bf16 MFU | 1623352 tok/s
step 13728/19560 | loss 3.319041 (-0.33z)| norm 0.2668 (-0.96z)| lr 1.31e-04 | 323.30 ms | 52.2% bf16 MFU | 1623267 tok/s
step 13729/19560 | loss 3.418211 (+1.75z)| norm 0.3062 (+1.57z)| lr 1.31e-04 | 322.78 ms | 52.3% bf16 MFU | 1623318 tok/s
step 13730/19560 | loss 3.336362 (+0.03z)| norm 0.2943 (+0.79z)| lr 1.31e-04 | 322.94 ms | 52.3% bf16 MFU | 1623326 tok/s
step 13731/19560 | loss 3.315730 (-0.42z)| norm 0.2969 (+0.94z)| lr 1.31e-04 | 323.08 ms | 52.2% bf16 MFU | 1623300 tok/s
step 13732/19560 | loss 3.347969 (+0.26z)| norm 0.2707 (-0.73z)| lr 1.31e-04 | 322.59 ms | 52.3% bf16 MFU | 1623397 tok/s
step 13733/19560 | loss 3.351933 (+0.35z)| norm 0.2933 (+0.70z)| lr 1.31e-04 | 323.32 ms | 52.2% bf16 MFU | 1623305 tok/s
step 13734/19560 | loss 3.362563 (+0.56z)| norm 0.2891 (+0.43z)| lr 1.31e-04 | 322.59 ms | 52.3% bf16 MFU | 1623402 tok/s
step 13735/19560 | loss 3.315371 (-0.44z)| norm 0.2736 (-0.56z)| lr 1.31e-04 | 323.36 ms | 52.2% bf16 MFU | 1623301 tok/s
step 13736/19560 | loss 3.383746 (+1.01z)| norm 0.2888 (+0.41z)| lr 1.30e-04 | 323.12 ms | 52.2% bf16 MFU | 1623266 tok/s
step 13737/19560 | loss 3.304288 (-0.67z)| norm 0.2905 (+0.52z)| lr 1.30e-04 | 323.60 ms | 52.2% bf16 MFU | 1623112 tok/s
step 13738/19560 | loss 3.453538 (+2.48z)| norm 0.2845 (+0.13z)| lr 1.30e-04 | 323.06 ms | 52.2% bf16 MFU | 1623102 tok/s
step 13739/19560 | loss 3.286562 (-1.03z)| norm 0.2827 (+0.01z)| lr 1.30e-04 | 322.93 ms | 52.3% bf16 MFU | 1623123 tok/s
step 13740/19560 | loss 3.401368 (+1.39z)| norm 0.2690 (-0.87z)| lr 1.30e-04 | 323.03 ms | 52.2% bf16 MFU | 1623119 tok/s
step 13741/19560 | loss 3.325094 (-0.21z)| norm 0.2611 (-1.36z)| lr 1.30e-04 | 322.88 ms | 52.3% bf16 MFU | 1623152 tok/s
step 13742/19560 | loss 3.331391 (-0.07z)| norm 0.2559 (-1.66z)| lr 1.30e-04 | 322.99 ms | 52.3% bf16 MFU | 1623157 tok/s
step 13743/19560 | loss 3.328673 (-0.12z)| norm 0.2705 (-0.72z)| lr 1.30e-04 | 322.40 ms | 52.3% bf16 MFU | 1623310 tok/s
step 13744/19560 | loss 3.385743 (+1.10z)| norm 0.2741 (-0.49z)| lr 1.30e-04 | 323.35 ms | 52.2% bf16 MFU | 1623215 tok/s
step 13745/19560 | loss 3.328125 (-0.13z)| norm 0.2653 (-1.03z)| lr 1.30e-04 | 323.64 ms | 52.1% bf16 MFU | 1623053 tok/s
step 13746/19560 | loss 3.311029 (-0.49z)| norm 0.2565 (-1.57z)| lr 1.30e-04 | 323.18 ms | 52.2% bf16 MFU | 1623016 tok/s
step 13747/19560 | loss 3.355503 (+0.45z)| norm 0.2826 (+0.09z)| lr 1.30e-04 | 322.66 ms | 52.3% bf16 MFU | 1623110 tok/s
step 13748/19560 | loss 3.269360 (-1.37z)| norm 0.2806 (-0.04z)| lr 1.30e-04 | 322.87 ms | 52.3% bf16 MFU | 1623146 tok/s
step 13749/19560 | loss 3.300990 (-0.69z)| norm 0.2804 (-0.04z)| lr 1.30e-04 | 323.23 ms | 52.2% bf16 MFU | 1623090 tok/s
step 13750/19560 | loss 3.348850 (+0.33z)| norm 0.2663 (-0.94z)| lr 1.30e-04 | 322.44 ms | 52.3% bf16 MFU | 1623236 tok/s
val loss 3.321792
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3000/10042 = 0.298745
step 13751/19560 | loss 3.340814 (+0.16z)| norm 0.2730 (-0.50z)| lr 1.30e-04 | 323.11 ms | 52.2% bf16 MFU | 1623207 tok/s
step 13752/19560 | loss 3.348986 (+0.33z)| norm 0.2710 (-0.62z)| lr 1.30e-04 | 322.94 ms | 52.3% bf16 MFU | 1623221 tok/s
step 13753/19560 | loss 3.420696 (+1.82z)| norm 0.2853 (+0.29z)| lr 1.30e-04 | 322.32 ms | 52.4% bf16 MFU | 1623392 tok/s
step 13754/19560 | loss 3.286921 (-0.98z)| norm 0.2813 (+0.04z)| lr 1.30e-04 | 323.09 ms | 52.2% bf16 MFU | 1623359 tok/s
step 13755/19560 | loss 3.289020 (-0.94z)| norm 0.2844 (+0.25z)| lr 1.30e-04 | 322.92 ms | 52.3% bf16 MFU | 1623371 tok/s
step 13756/19560 | loss 3.342429 (+0.19z)| norm 0.2662 (-0.92z)| lr 1.30e-04 | 322.50 ms | 52.3% bf16 MFU | 1623489 tok/s
step 13757/19560 | loss 3.321210 (-0.27z)| norm 0.2727 (-0.49z)| lr 1.30e-04 | 322.94 ms | 52.3% bf16 MFU | 1623489 tok/s
step 13758/19560 | loss 3.324263 (-0.20z)| norm 0.2767 (-0.21z)| lr 1.30e-04 | 322.84 ms | 52.3% bf16 MFU | 1623514 tok/s
step 13759/19560 | loss 3.350331 (+0.35z)| norm 0.2593 (-1.34z)| lr 1.30e-04 | 323.19 ms | 52.2% bf16 MFU | 1623450 tok/s
step 13760/19560 | loss 3.332965 (-0.02z)| norm 0.2600 (-1.28z)| lr 1.29e-04 | 322.97 ms | 52.3% bf16 MFU | 1623444 tok/s
step 13761/19560 | loss 3.400998 (+1.59z)| norm 0.2656 (-0.90z)| lr 1.29e-04 | 322.76 ms | 52.3% bf16 MFU | 1623490 tok/s
step 13762/19560 | loss 3.350286 (+0.42z)| norm 0.2756 (-0.25z)| lr 1.29e-04 | 322.73 ms | 52.3% bf16 MFU | 1623544 tok/s
step 13763/19560 | loss 3.336277 (+0.09z)| norm 0.2635 (-1.02z)| lr 1.29e-04 | 322.80 ms | 52.3% bf16 MFU | 1623574 tok/s
step 13764/19560 | loss 3.351248 (+0.43z)| norm 0.2772 (-0.14z)| lr 1.29e-04 | 322.77 ms | 52.3% bf16 MFU | 1623612 tok/s
step 13765/19560 | loss 3.333608 (+0.02z)| norm 0.2683 (-0.71z)| lr 1.29e-04 | 323.10 ms | 52.2% bf16 MFU | 1623565 tok/s
step 13766/19560 | loss 3.330102 (-0.06z)| norm 0.2909 (+0.75z)| lr 1.29e-04 | 322.79 ms | 52.3% bf16 MFU | 1623597 tok/s
step 13767/19560 | loss 3.278894 (-1.24z)| norm 0.2609 (-1.18z)| lr 1.29e-04 | 322.94 ms | 52.3% bf16 MFU | 1623591 tok/s
step 13768/19560 | loss 3.325134 (-0.16z)| norm 0.2848 (+0.38z)| lr 1.29e-04 | 322.71 ms | 52.3% bf16 MFU | 1623643 tok/s
step 13769/19560 | loss 3.342790 (+0.24z)| norm 0.2773 (-0.11z)| lr 1.29e-04 | 322.69 ms | 52.3% bf16 MFU | 1623699 tok/s
step 13770/19560 | loss 3.338101 (+0.13z)| norm 0.2802 (+0.08z)| lr 1.29e-04 | 322.79 ms | 52.3% bf16 MFU | 1623726 tok/s
step 13771/19560 | loss 3.310549 (-0.51z)| norm 0.2878 (+0.59z)| lr 1.29e-04 | 322.63 ms | 52.3% bf16 MFU | 1623792 tok/s
step 13772/19560 | loss 3.473574 (+3.17z)| norm 0.3143 (+2.27z)| lr 1.29e-04 | 322.74 ms | 52.3% bf16 MFU | 1623828 tok/s
step 13773/19560 | loss 3.306646 (-0.60z)| norm 0.2946 (+0.99z)| lr 1.29e-04 | 322.69 ms | 52.3% bf16 MFU | 1623872 tok/s
step 13774/19560 | loss 3.327677 (-0.11z)| norm 0.2578 (-1.36z)| lr 1.29e-04 | 322.64 ms | 52.3% bf16 MFU | 1623928 tok/s
step 13775/19560 | loss 3.358135 (+0.58z)| norm 0.2722 (-0.44z)| lr 1.29e-04 | 323.21 ms | 52.2% bf16 MFU | 1623838 tok/s
step 13776/19560 | loss 3.369012 (+0.81z)| norm 0.2754 (-0.24z)| lr 1.29e-04 | 322.46 ms | 52.3% bf16 MFU | 1623942 tok/s
step 13777/19560 | loss 3.289974 (-0.99z)| norm 0.2724 (-0.43z)| lr 1.29e-04 | 323.00 ms | 52.3% bf16 MFU | 1623904 tok/s
step 13778/19560 | loss 3.338710 (+0.15z)| norm 0.2622 (-1.07z)| lr 1.29e-04 | 322.88 ms | 52.3% bf16 MFU | 1623898 tok/s
step 13779/19560 | loss 3.244661 (-2.00z)| norm 0.2738 (-0.34z)| lr 1.29e-04 | 322.85 ms | 52.3% bf16 MFU | 1623900 tok/s
step 13780/19560 | loss 3.270797 (-1.39z)| norm 0.2551 (-1.52z)| lr 1.29e-04 | 322.94 ms | 52.3% bf16 MFU | 1623880 tok/s
step 13781/19560 | loss 3.359772 (+0.64z)| norm 0.2819 (+0.19z)| lr 1.29e-04 | 322.66 ms | 52.3% bf16 MFU | 1623932 tok/s
step 13782/19560 | loss 3.320123 (-0.26z)| norm 0.2875 (+0.55z)| lr 1.29e-04 | 322.42 ms | 52.3% bf16 MFU | 1624040 tok/s
step 13783/19560 | loss 3.299752 (-0.72z)| norm 0.2744 (-0.30z)| lr 1.29e-04 | 322.91 ms | 52.3% bf16 MFU | 1624019 tok/s
step 13784/19560 | loss 3.258894 (-1.67z)| norm 0.2912 (+0.78z)| lr 1.29e-04 | 322.31 ms | 52.4% bf16 MFU | 1624150 tok/s
step 13785/19560 | loss 3.332629 (+0.04z)| norm 0.2818 (+0.17z)| lr 1.28e-04 | 322.85 ms | 52.3% bf16 MFU | 1624140 tok/s
step 13786/19560 | loss 3.278262 (-1.21z)| norm 0.2723 (-0.46z)| lr 1.28e-04 | 322.50 ms | 52.3% bf16 MFU | 1624219 tok/s
step 13787/19560 | loss 3.358421 (+0.66z)| norm 0.2815 (+0.15z)| lr 1.28e-04 | 322.64 ms | 52.3% bf16 MFU | 1624258 tok/s
step 13788/19560 | loss 3.297245 (-0.77z)| norm 0.2747 (-0.30z)| lr 1.28e-04 | 323.09 ms | 52.2% bf16 MFU | 1624180 tok/s
step 13789/19560 | loss 3.275877 (-1.25z)| norm 0.2892 (+0.64z)| lr 1.28e-04 | 322.65 ms | 52.3% bf16 MFU | 1624220 tok/s
step 13790/19560 | loss 3.350945 (+0.57z)| norm 0.2891 (+0.65z)| lr 1.28e-04 | 322.49 ms | 52.3% bf16 MFU | 1624296 tok/s
step 13791/19560 | loss 3.324158 (-0.08z)| norm 0.2740 (-0.34z)| lr 1.28e-04 | 322.79 ms | 52.3% bf16 MFU | 1624294 tok/s
step 13792/19560 | loss 3.401032 (+1.86z)| norm 0.2856 (+0.43z)| lr 1.28e-04 | 322.27 ms | 52.4% bf16 MFU | 1624423 tok/s
step 13793/19560 | loss 3.283458 (-1.12z)| norm 0.3064 (+1.77z)| lr 1.28e-04 | 322.98 ms | 52.3% bf16 MFU | 1624365 tok/s
step 13794/19560 | loss 3.299859 (-0.69z)| norm 0.2727 (-0.45z)| lr 1.28e-04 | 322.00 ms | 52.4% bf16 MFU | 1624557 tok/s
step 13795/19560 | loss 3.340737 (+0.35z)| norm 0.2834 (+0.27z)| lr 1.28e-04 | 322.99 ms | 52.3% bf16 MFU | 1624491 tok/s
step 13796/19560 | loss 3.295832 (-0.79z)| norm 0.2764 (-0.19z)| lr 1.28e-04 | 322.57 ms | 52.3% bf16 MFU | 1624533 tok/s
step 13797/19560 | loss 3.316079 (-0.29z)| norm 0.2721 (-0.47z)| lr 1.28e-04 | 322.90 ms | 52.3% bf16 MFU | 1624491 tok/s
step 13798/19560 | loss 3.358488 (+0.78z)| norm 0.2896 (+0.68z)| lr 1.28e-04 | 322.99 ms | 52.3% bf16 MFU | 1624427 tok/s
step 13799/19560 | loss 3.294733 (-0.83z)| norm 0.2818 (+0.17z)| lr 1.28e-04 | 322.40 ms | 52.3% bf16 MFU | 1624516 tok/s
step 13800/19560 | loss 3.286599 (-1.02z)| norm 0.2674 (-0.78z)| lr 1.28e-04 | 322.99 ms | 52.3% bf16 MFU | 1624452 tok/s
step 13801/19560 | loss 3.257959 (-1.72z)| norm 0.2592 (-1.31z)| lr 1.28e-04 | 322.50 ms | 52.3% bf16 MFU | 1624515 tok/s
step 13802/19560 | loss 3.302181 (-0.60z)| norm 0.2664 (-0.82z)| lr 1.28e-04 | 322.87 ms | 52.3% bf16 MFU | 1624481 tok/s
step 13803/19560 | loss 3.339918 (+0.33z)| norm 0.2898 (+0.72z)| lr 1.28e-04 | 323.05 ms | 52.2% bf16 MFU | 1624403 tok/s
step 13804/19560 | loss 3.321218 (-0.14z)| norm 0.2880 (+0.59z)| lr 1.28e-04 | 322.63 ms | 52.3% bf16 MFU | 1624435 tok/s
step 13805/19560 | loss 3.325907 (-0.03z)| norm 0.2913 (+0.83z)| lr 1.28e-04 | 323.07 ms | 52.2% bf16 MFU | 1624355 tok/s
step 13806/19560 | loss 3.394292 (+1.67z)| norm 0.3086 (+1.96z)| lr 1.28e-04 | 322.46 ms | 52.3% bf16 MFU | 1624433 tok/s
step 13807/19560 | loss 3.376171 (+1.21z)| norm 0.2754 (-0.21z)| lr 1.28e-04 | 322.63 ms | 52.3% bf16 MFU | 1624464 tok/s
step 13808/19560 | loss 3.306713 (-0.52z)| norm 0.2711 (-0.52z)| lr 1.28e-04 | 322.59 ms | 52.3% bf16 MFU | 1624504 tok/s
step 13809/19560 | loss 3.320732 (-0.17z)| norm 0.2806 (+0.21z)| lr 1.27e-04 | 323.02 ms | 52.2% bf16 MFU | 1624432 tok/s
step 13810/19560 | loss 3.278138 (-1.24z)| norm 0.2990 (+1.74z)| lr 1.27e-04 | 322.84 ms | 52.3% bf16 MFU | 1624409 tok/s
step 13811/19560 | loss 3.552944 (+5.03z)| norm 0.3405 (+4.70z)| lr 1.27e-04 | 323.16 ms | 52.2% bf16 MFU | 1624307 tok/s
step 13812/19560 | loss 3.353434 (+0.54z)| norm 0.2673 (-0.78z)| lr 1.27e-04 | 323.23 ms | 52.2% bf16 MFU | 1624193 tok/s
step 13813/19560 | loss 3.312381 (-0.38z)| norm 0.3088 (+2.30z)| lr 1.27e-04 | 322.56 ms | 52.3% bf16 MFU | 1624255 tok/s
step 13814/19560 | loss 3.292623 (-0.83z)| norm 0.2751 (-0.19z)| lr 1.27e-04 | 322.61 ms | 52.3% bf16 MFU | 1624299 tok/s
step 13815/19560 | loss 3.435244 (+2.33z)| norm 0.2859 (+0.60z)| lr 1.27e-04 | 322.86 ms | 52.3% bf16 MFU | 1624277 tok/s
step 13816/19560 | loss 3.371862 (+0.91z)| norm 0.3012 (+1.73z)| lr 1.27e-04 | 322.16 ms | 52.4% bf16 MFU | 1624435 tok/s
step 13817/19560 | loss 3.315904 (-0.33z)| norm 0.2656 (-0.89z)| lr 1.27e-04 | 322.83 ms | 52.3% bf16 MFU | 1624415 tok/s
step 13818/19560 | loss 3.251509 (-1.73z)| norm 0.2783 (+0.05z)| lr 1.27e-04 | 322.83 ms | 52.3% bf16 MFU | 1624396 tok/s
step 13819/19560 | loss 3.312081 (-0.39z)| norm 0.2970 (+1.41z)| lr 1.27e-04 | 322.95 ms | 52.3% bf16 MFU | 1624348 tok/s
step 13820/19560 | loss 3.398083 (+1.48z)| norm 0.2940 (+1.18z)| lr 1.27e-04 | 322.70 ms | 52.3% bf16 MFU | 1624365 tok/s
step 13821/19560 | loss 3.307393 (-0.51z)| norm 0.2880 (+0.75z)| lr 1.27e-04 | 322.86 ms | 52.3% bf16 MFU | 1624342 tok/s
step 13822/19560 | loss 3.361029 (+0.66z)| norm 0.2869 (+0.66z)| lr 1.27e-04 | 322.99 ms | 52.3% bf16 MFU | 1624287 tok/s
step 13823/19560 | loss 3.316962 (-0.30z)| norm 0.2848 (+0.50z)| lr 1.27e-04 | 323.15 ms | 52.2% bf16 MFU | 1624195 tok/s
step 13824/19560 | loss 3.358554 (+0.61z)| norm 0.2816 (+0.25z)| lr 1.27e-04 | 322.20 ms | 52.4% bf16 MFU | 1624344 tok/s
step 13825/19560 | loss 3.369469 (+0.84z)| norm 0.2994 (+1.54z)| lr 1.27e-04 | 323.10 ms | 52.2% bf16 MFU | 1624260 tok/s
step 13826/19560 | loss 3.314450 (-0.36z)| norm 0.2706 (-0.56z)| lr 1.27e-04 | 322.52 ms | 52.3% bf16 MFU | 1624328 tok/s
step 13827/19560 | loss 3.267669 (-1.39z)| norm 0.2880 (+0.69z)| lr 1.27e-04 | 323.02 ms | 52.2% bf16 MFU | 1624265 tok/s
step 13828/19560 | loss 3.339035 (+0.17z)| norm 0.2834 (+0.36z)| lr 1.27e-04 | 322.53 ms | 52.3% bf16 MFU | 1624330 tok/s
step 13829/19560 | loss 3.358451 (+0.61z)| norm 0.2725 (-0.44z)| lr 1.27e-04 | 322.74 ms | 52.3% bf16 MFU | 1624338 tok/s
step 13830/19560 | loss 3.289436 (-0.92z)| norm 0.2802 (+0.12z)| lr 1.27e-04 | 322.85 ms | 52.3% bf16 MFU | 1624318 tok/s
step 13831/19560 | loss 3.351670 (+0.46z)| norm 0.3267 (+3.34z)| lr 1.27e-04 | 323.13 ms | 52.2% bf16 MFU | 1624229 tok/s
step 13832/19560 | loss 3.296968 (-0.76z)| norm 0.3256 (+3.11z)| lr 1.27e-04 | 323.08 ms | 52.2% bf16 MFU | 1624157 tok/s
step 13833/19560 | loss 3.271167 (-1.32z)| norm 0.2670 (-0.83z)| lr 1.27e-04 | 322.67 ms | 52.3% bf16 MFU | 1624192 tok/s
step 13834/19560 | loss 3.338314 (+0.16z)| norm 0.3106 (+2.05z)| lr 1.26e-04 | 322.92 ms | 52.3% bf16 MFU | 1624162 tok/s
step 13835/19560 | loss 3.374503 (+0.94z)| norm 0.2851 (+0.36z)| lr 1.26e-04 | 323.21 ms | 52.2% bf16 MFU | 1624061 tok/s
step 13836/19560 | loss 3.330725 (-0.03z)| norm 0.2735 (-0.42z)| lr 1.26e-04 | 322.43 ms | 52.3% bf16 MFU | 1624161 tok/s
step 13837/19560 | loss 3.389804 (+1.26z)| norm 0.2873 (+0.49z)| lr 1.26e-04 | 322.80 ms | 52.3% bf16 MFU | 1624164 tok/s
step 13838/19560 | loss 3.437819 (+2.25z)| norm 0.2815 (+0.10z)| lr 1.26e-04 | 322.13 ms | 52.4% bf16 MFU | 1624334 tok/s
step 13839/19560 | loss 3.329142 (-0.11z)| norm 0.2899 (+0.65z)| lr 1.26e-04 | 322.93 ms | 52.3% bf16 MFU | 1624293 tok/s
step 13840/19560 | loss 3.313881 (-0.45z)| norm 0.2994 (+1.27z)| lr 1.26e-04 | 323.62 ms | 52.2% bf16 MFU | 1624083 tok/s
step 13841/19560 | loss 3.364501 (+0.66z)| norm 0.2908 (+0.69z)| lr 1.26e-04 | 322.67 ms | 52.3% bf16 MFU | 1624120 tok/s
step 13842/19560 | loss 3.298657 (-0.78z)| norm 0.2828 (+0.13z)| lr 1.26e-04 | 322.24 ms | 52.4% bf16 MFU | 1624266 tok/s
step 13843/19560 | loss 3.281227 (-1.14z)| norm 0.2963 (+1.05z)| lr 1.26e-04 | 323.24 ms | 52.2% bf16 MFU | 1624152 tok/s
step 13844/19560 | loss 3.232378 (-2.16z)| norm 0.2798 (-0.07z)| lr 1.26e-04 | 322.60 ms | 52.3% bf16 MFU | 1624203 tok/s
step 13845/19560 | loss 3.326520 (-0.13z)| norm 0.2811 (+0.01z)| lr 1.26e-04 | 322.62 ms | 52.3% bf16 MFU | 1624247 tok/s
step 13846/19560 | loss 3.346042 (+0.29z)| norm 0.2850 (+0.28z)| lr 1.26e-04 | 323.28 ms | 52.2% bf16 MFU | 1624123 tok/s
step 13847/19560 | loss 3.349241 (+0.35z)| norm 0.2670 (-0.96z)| lr 1.26e-04 | 322.31 ms | 52.4% bf16 MFU | 1624251 tok/s
step 13848/19560 | loss 3.362264 (+0.63z)| norm 0.3049 (+1.62z)| lr 1.26e-04 | 322.56 ms | 52.3% bf16 MFU | 1624309 tok/s
step 13849/19560 | loss 3.313780 (-0.41z)| norm 0.2808 (-0.04z)| lr 1.26e-04 | 323.13 ms | 52.2% bf16 MFU | 1624220 tok/s
step 13850/19560 | loss 3.340624 (+0.17z)| norm 0.2673 (-0.96z)| lr 1.26e-04 | 322.88 ms | 52.3% bf16 MFU | 1624199 tok/s
step 13851/19560 | loss 3.398698 (+1.40z)| norm 0.3078 (+1.77z)| lr 1.26e-04 | 322.72 ms | 52.3% bf16 MFU | 1624218 tok/s
step 13852/19560 | loss 3.353925 (+0.43z)| norm 0.2870 (+0.37z)| lr 1.26e-04 | 322.60 ms | 52.3% bf16 MFU | 1624267 tok/s
step 13853/19560 | loss 3.394666 (+1.29z)| norm 0.2728 (-0.59z)| lr 1.26e-04 | 322.71 ms | 52.3% bf16 MFU | 1624284 tok/s
step 13854/19560 | loss 3.260176 (-1.55z)| norm 0.2803 (-0.10z)| lr 1.26e-04 | 322.65 ms | 52.3% bf16 MFU | 1624318 tok/s
step 13855/19560 | loss 3.254383 (-1.65z)| norm 0.2711 (-0.72z)| lr 1.26e-04 | 322.84 ms | 52.3% bf16 MFU | 1624301 tok/s
step 13856/19560 | loss 3.345561 (+0.25z)| norm 0.2931 (+0.76z)| lr 1.26e-04 | 323.28 ms | 52.2% bf16 MFU | 1624174 tok/s
step 13857/19560 | loss 3.366923 (+0.72z)| norm 0.2603 (-1.44z)| lr 1.26e-04 | 322.83 ms | 52.3% bf16 MFU | 1624167 tok/s
step 13858/19560 | loss 3.311054 (-0.46z)| norm 0.2770 (-0.30z)| lr 1.25e-04 | 322.85 ms | 52.3% bf16 MFU | 1624156 tok/s
step 13859/19560 | loss 3.291815 (-0.86z)| norm 0.2707 (-0.71z)| lr 1.25e-04 | 322.31 ms | 52.4% bf16 MFU | 1624283 tok/s
step 13860/19560 | loss 3.341458 (+0.19z)| norm 0.2742 (-0.48z)| lr 1.25e-04 | 322.65 ms | 52.3% bf16 MFU | 1624315 tok/s
step 13861/19560 | loss 3.292765 (-0.83z)| norm 0.2734 (-0.53z)| lr 1.25e-04 | 322.93 ms | 52.3% bf16 MFU | 1624275 tok/s
step 13862/19560 | loss 3.301684 (-0.63z)| norm 0.2659 (-1.02z)| lr 1.25e-04 | 322.88 ms | 52.3% bf16 MFU | 1624251 tok/s
step 13863/19560 | loss 3.374392 (+0.89z)| norm 0.2651 (-1.07z)| lr 1.25e-04 | 322.93 ms | 52.3% bf16 MFU | 1624214 tok/s
step 13864/19560 | loss 3.308547 (-0.49z)| norm 0.2699 (-0.73z)| lr 1.25e-04 | 322.73 ms | 52.3% bf16 MFU | 1624232 tok/s
step 13865/19560 | loss 3.368207 (+0.76z)| norm 0.3026 (+1.47z)| lr 1.25e-04 | 323.17 ms | 52.2% bf16 MFU | 1624137 tok/s
step 13866/19560 | loss 3.314481 (-0.36z)| norm 0.2946 (+0.93z)| lr 1.25e-04 | 322.79 ms | 52.3% bf16 MFU | 1624141 tok/s
step 13867/19560 | loss 3.310019 (-0.46z)| norm 0.2757 (-0.34z)| lr 1.25e-04 | 322.85 ms | 52.3% bf16 MFU | 1624130 tok/s
step 13868/19560 | loss 3.282454 (-1.04z)| norm 0.2733 (-0.51z)| lr 1.25e-04 | 322.89 ms | 52.3% bf16 MFU | 1624112 tok/s
step 13869/19560 | loss 3.253078 (-1.65z)| norm 0.2570 (-1.60z)| lr 1.25e-04 | 322.30 ms | 52.4% bf16 MFU | 1624241 tok/s
step 13870/19560 | loss 3.376941 (+1.01z)| norm 0.2680 (-0.87z)| lr 1.25e-04 | 323.11 ms | 52.2% bf16 MFU | 1624160 tok/s
step 13871/19560 | loss 3.362615 (+0.69z)| norm 0.2884 (+0.50z)| lr 1.25e-04 | 323.00 ms | 52.3% bf16 MFU | 1624113 tok/s
step 13872/19560 | loss 3.340695 (+0.23z)| norm 0.2620 (-1.28z)| lr 1.25e-04 | 323.26 ms | 52.2% bf16 MFU | 1624001 tok/s
step 13873/19560 | loss 3.337907 (+0.17z)| norm 0.2624 (-1.24z)| lr 1.25e-04 | 322.62 ms | 52.3% bf16 MFU | 1624055 tok/s
step 13874/19560 | loss 3.226172 (-2.18z)| norm 0.2701 (-0.74z)| lr 1.25e-04 | 322.14 ms | 52.4% bf16 MFU | 1624227 tok/s
step 13875/19560 | loss 3.307448 (-0.46z)| norm 0.2718 (-0.61z)| lr 1.25e-04 | 323.05 ms | 52.2% bf16 MFU | 1624161 tok/s
step 13876/19560 | loss 3.306024 (-0.50z)| norm 0.2502 (-2.03z)| lr 1.25e-04 | 323.23 ms | 52.2% bf16 MFU | 1624054 tok/s
step 13877/19560 | loss 3.345786 (+0.34z)| norm 0.2835 (+0.19z)| lr 1.25e-04 | 323.07 ms | 52.2% bf16 MFU | 1623992 tok/s
step 13878/19560 | loss 3.321709 (-0.16z)| norm 0.2608 (-1.32z)| lr 1.25e-04 | 322.85 ms | 52.3% bf16 MFU | 1623988 tok/s
step 13879/19560 | loss 3.356484 (+0.57z)| norm 0.2667 (-0.92z)| lr 1.25e-04 | 322.45 ms | 52.3% bf16 MFU | 1624085 tok/s
step 13880/19560 | loss 3.307650 (-0.46z)| norm 0.2972 (+1.08z)| lr 1.25e-04 | 322.91 ms | 52.3% bf16 MFU | 1624062 tok/s
step 13881/19560 | loss 3.354403 (+0.55z)| norm 0.2568 (-1.56z)| lr 1.25e-04 | 323.27 ms | 52.2% bf16 MFU | 1623952 tok/s
step 13882/19560 | loss 3.330840 (+0.04z)| norm 0.2903 (+0.63z)| lr 1.25e-04 | 323.22 ms | 52.2% bf16 MFU | 1623859 tok/s
step 13883/19560 | loss 3.321471 (-0.17z)| norm 0.2778 (-0.18z)| lr 1.24e-04 | 322.63 ms | 52.3% bf16 MFU | 1623917 tok/s
step 13884/19560 | loss 3.359651 (+0.65z)| norm 0.2555 (-1.63z)| lr 1.24e-04 | 322.72 ms | 52.3% bf16 MFU | 1623950 tok/s
step 13885/19560 | loss 3.288198 (-0.89z)| norm 0.2697 (-0.70z)| lr 1.24e-04 | 323.42 ms | 52.2% bf16 MFU | 1623806 tok/s
step 13886/19560 | loss 3.343183 (+0.30z)| norm 0.2600 (-1.31z)| lr 1.24e-04 | 322.72 ms | 52.3% bf16 MFU | 1623846 tok/s
step 13887/19560 | loss 3.355037 (+0.55z)| norm 0.2565 (-1.53z)| lr 1.24e-04 | 323.69 ms | 52.1% bf16 MFU | 1623640 tok/s
step 13888/19560 | loss 3.283825 (-0.97z)| norm 0.2667 (-0.88z)| lr 1.24e-04 | 322.65 ms | 52.3% bf16 MFU | 1623706 tok/s
step 13889/19560 | loss 3.340995 (+0.27z)| norm 0.2852 (+0.30z)| lr 1.24e-04 | 322.62 ms | 52.3% bf16 MFU | 1623775 tok/s
step 13890/19560 | loss 3.353127 (+0.53z)| norm 0.2750 (-0.36z)| lr 1.24e-04 | 323.27 ms | 52.2% bf16 MFU | 1623676 tok/s
step 13891/19560 | loss 3.357613 (+0.63z)| norm 0.2785 (-0.14z)| lr 1.24e-04 | 323.19 ms | 52.2% bf16 MFU | 1623603 tok/s
step 13892/19560 | loss 3.366181 (+0.81z)| norm 0.2720 (-0.56z)| lr 1.24e-04 | 322.93 ms | 52.3% bf16 MFU | 1623600 tok/s
step 13893/19560 | loss 3.343920 (+0.32z)| norm 0.2650 (-1.01z)| lr 1.24e-04 | 322.49 ms | 52.3% bf16 MFU | 1623706 tok/s
step 13894/19560 | loss 3.365121 (+0.78z)| norm 0.2849 (+0.28z)| lr 1.24e-04 | 323.21 ms | 52.2% bf16 MFU | 1623627 tok/s
step 13895/19560 | loss 3.296726 (-0.71z)| norm 0.2529 (-1.78z)| lr 1.24e-04 | 322.34 ms | 52.4% bf16 MFU | 1623771 tok/s
step 13896/19560 | loss 3.316845 (-0.27z)| norm 0.2807 (+0.02z)| lr 1.24e-04 | 323.28 ms | 52.2% bf16 MFU | 1623672 tok/s
step 13897/19560 | loss 3.285208 (-0.94z)| norm 0.2804 (-0.01z)| lr 1.24e-04 | 322.85 ms | 52.3% bf16 MFU | 1623686 tok/s
step 13898/19560 | loss 3.458482 (+2.70z)| norm 0.2988 (+1.17z)| lr 1.24e-04 | 323.32 ms | 52.2% bf16 MFU | 1623580 tok/s
step 13899/19560 | loss 3.289827 (-0.83z)| norm 0.2977 (+1.09z)| lr 1.24e-04 | 322.85 ms | 52.3% bf16 MFU | 1623598 tok/s
step 13900/19560 | loss 3.299056 (-0.63z)| norm 0.2905 (+0.65z)| lr 1.24e-04 | 322.89 ms | 52.3% bf16 MFU | 1623606 tok/s
step 13901/19560 | loss 3.335663 (+0.16z)| norm 0.2716 (-0.57z)| lr 1.24e-04 | 322.37 ms | 52.4% bf16 MFU | 1623743 tok/s
step 13902/19560 | loss 3.424497 (+2.03z)| norm 0.2900 (+0.62z)| lr 1.24e-04 | 322.94 ms | 52.3% bf16 MFU | 1623730 tok/s
step 13903/19560 | loss 3.317561 (-0.24z)| norm 0.2929 (+0.80z)| lr 1.24e-04 | 323.34 ms | 52.2% bf16 MFU | 1623616 tok/s
step 13904/19560 | loss 3.344115 (+0.33z)| norm 0.2718 (-0.59z)| lr 1.24e-04 | 323.28 ms | 52.2% bf16 MFU | 1623524 tok/s
step 13905/19560 | loss 3.290251 (-0.82z)| norm 0.2690 (-0.77z)| lr 1.24e-04 | 323.07 ms | 52.2% bf16 MFU | 1623491 tok/s
step 13906/19560 | loss 3.301040 (-0.59z)| norm 0.2655 (-1.00z)| lr 1.24e-04 | 323.26 ms | 52.2% bf16 MFU | 1623409 tok/s
step 13907/19560 | loss 3.341361 (+0.26z)| norm 0.2588 (-1.42z)| lr 1.24e-04 | 322.78 ms | 52.3% bf16 MFU | 1623452 tok/s
step 13908/19560 | loss 3.371691 (+0.91z)| norm 0.2784 (-0.16z)| lr 1.23e-04 | 322.92 ms | 52.3% bf16 MFU | 1623458 tok/s
step 13909/19560 | loss 3.379506 (+1.07z)| norm 0.2631 (-1.15z)| lr 1.23e-04 | 323.00 ms | 52.3% bf16 MFU | 1623443 tok/s
step 13910/19560 | loss 3.322134 (-0.17z)| norm 0.3414 (+3.73z)| lr 1.23e-04 | 323.58 ms | 52.2% bf16 MFU | 1623284 tok/s
step 13911/19560 | loss 3.268174 (-1.33z)| norm 0.2675 (-0.83z)| lr 1.23e-04 | 322.88 ms | 52.3% bf16 MFU | 1623308 tok/s
step 13912/19560 | loss 3.310811 (-0.42z)| norm 0.2994 (+1.13z)| lr 1.23e-04 | 322.63 ms | 52.3% bf16 MFU | 1623396 tok/s
step 13913/19560 | loss 3.299602 (-0.66z)| norm 0.2788 (-0.14z)| lr 1.23e-04 | 323.28 ms | 52.2% bf16 MFU | 1623314 tok/s
step 13914/19560 | loss 3.329348 (-0.02z)| norm 0.2872 (+0.37z)| lr 1.23e-04 | 322.97 ms | 52.3% bf16 MFU | 1623316 tok/s
step 13915/19560 | loss 3.317441 (-0.28z)| norm 0.2925 (+0.69z)| lr 1.23e-04 | 322.53 ms | 52.3% bf16 MFU | 1623428 tok/s
step 13916/19560 | loss 3.237564 (-1.98z)| norm 0.2976 (+0.99z)| lr 1.23e-04 | 323.51 ms | 52.2% bf16 MFU | 1623287 tok/s
step 13917/19560 | loss 3.323709 (-0.14z)| norm 0.2697 (-0.71z)| lr 1.23e-04 | 322.97 ms | 52.3% bf16 MFU | 1623289 tok/s
step 13918/19560 | loss 3.343153 (+0.29z)| norm 0.2920 (+0.65z)| lr 1.23e-04 | 322.42 ms | 52.3% bf16 MFU | 1623429 tok/s
step 13919/19560 | loss 3.373352 (+0.93z)| norm 0.2887 (+0.45z)| lr 1.23e-04 | 323.18 ms | 52.2% bf16 MFU | 1623371 tok/s
step 13920/19560 | loss 3.316827 (-0.28z)| norm 0.2758 (-0.34z)| lr 1.23e-04 | 323.34 ms | 52.2% bf16 MFU | 1623275 tok/s
step 13921/19560 | loss 3.324671 (-0.12z)| norm 0.2915 (+0.63z)| lr 1.23e-04 | 323.09 ms | 52.2% bf16 MFU | 1623249 tok/s
step 13922/19560 | loss 3.331573 (+0.03z)| norm 0.2786 (-0.17z)| lr 1.23e-04 | 322.80 ms | 52.3% bf16 MFU | 1623296 tok/s
step 13923/19560 | loss 3.461599 (+2.77z)| norm 0.3040 (+1.38z)| lr 1.23e-04 | 322.99 ms | 52.3% bf16 MFU | 1623293 tok/s
step 13924/19560 | loss 3.369933 (+0.81z)| norm 0.2669 (-0.88z)| lr 1.23e-04 | 323.25 ms | 52.2% bf16 MFU | 1623225 tok/s
step 13925/19560 | loss 3.344810 (+0.27z)| norm 0.2894 (+0.48z)| lr 1.23e-04 | 322.65 ms | 52.3% bf16 MFU | 1623310 tok/s
step 13926/19560 | loss 3.353874 (+0.47z)| norm 0.2689 (-0.76z)| lr 1.23e-04 | 323.44 ms | 52.2% bf16 MFU | 1623193 tok/s
step 13927/19560 | loss 3.363746 (+0.66z)| norm 0.2622 (-1.15z)| lr 1.23e-04 | 322.88 ms | 52.3% bf16 MFU | 1623223 tok/s
step 13928/19560 | loss 3.300310 (-0.69z)| norm 0.2684 (-0.77z)| lr 1.23e-04 | 322.52 ms | 52.3% bf16 MFU | 1623343 tok/s
step 13929/19560 | loss 3.324683 (-0.18z)| norm 0.2700 (-0.69z)| lr 1.23e-04 | 323.60 ms | 52.2% bf16 MFU | 1623186 tok/s
step 13930/19560 | loss 3.295001 (-0.82z)| norm 0.2559 (-1.53z)| lr 1.23e-04 | 323.26 ms | 52.2% bf16 MFU | 1623119 tok/s
step 13931/19560 | loss 3.333643 (+0.01z)| norm 0.2650 (-0.97z)| lr 1.23e-04 | 323.02 ms | 52.2% bf16 MFU | 1623117 tok/s
step 13932/19560 | loss 3.298033 (-0.75z)| norm 0.2727 (-0.49z)| lr 1.22e-04 | 323.35 ms | 52.2% bf16 MFU | 1623032 tok/s
step 13933/19560 | loss 3.388669 (+1.18z)| norm 0.2796 (-0.07z)| lr 1.22e-04 | 323.18 ms | 52.2% bf16 MFU | 1622995 tok/s
step 13934/19560 | loss 3.516646 (+3.70z)| norm 0.3202 (+2.36z)| lr 1.22e-04 | 322.73 ms | 52.3% bf16 MFU | 1623071 tok/s
step 13935/19560 | loss 3.328699 (-0.11z)| norm 0.2814 (+0.03z)| lr 1.22e-04 | 323.18 ms | 52.2% bf16 MFU | 1623031 tok/s
step 13936/19560 | loss 3.262239 (-1.44z)| norm 0.3011 (+1.19z)| lr 1.22e-04 | 323.71 ms | 52.1% bf16 MFU | 1622861 tok/s
step 13937/19560 | loss 3.325179 (-0.17z)| norm 0.2812 (+0.01z)| lr 1.22e-04 | 324.05 ms | 52.1% bf16 MFU | 1622613 tok/s
step 13938/19560 | loss 3.307178 (-0.54z)| norm 0.2742 (-0.40z)| lr 1.22e-04 | 322.79 ms | 52.3% bf16 MFU | 1622695 tok/s
step 13939/19560 | loss 3.333094 (+0.02z)| norm 0.3332 (+3.17z)| lr 1.22e-04 | 323.29 ms | 52.2% bf16 MFU | 1622645 tok/s
step 13940/19560 | loss 3.363918 (+0.70z)| norm 0.2896 (+0.52z)| lr 1.22e-04 | 322.87 ms | 52.3% bf16 MFU | 1622705 tok/s
step 13941/19560 | loss 3.346755 (+0.32z)| norm 0.2807 (-0.01z)| lr 1.22e-04 | 323.05 ms | 52.2% bf16 MFU | 1622716 tok/s
step 13942/19560 | loss 3.548380 (+4.36z)| norm 0.3671 (+4.76z)| lr 1.22e-04 | 322.76 ms | 52.3% bf16 MFU | 1622798 tok/s
step 13943/19560 | loss 3.356596 (+0.47z)| norm 0.2852 (+0.20z)| lr 1.22e-04 | 322.69 ms | 52.3% bf16 MFU | 1622896 tok/s
step 13944/19560 | loss 3.364569 (+0.64z)| norm 0.3063 (+1.37z)| lr 1.22e-04 | 322.93 ms | 52.3% bf16 MFU | 1622929 tok/s
step 13945/19560 | loss 3.317692 (-0.33z)| norm 0.2978 (+0.88z)| lr 1.22e-04 | 322.94 ms | 52.3% bf16 MFU | 1622956 tok/s
step 13946/19560 | loss 3.310639 (-0.50z)| norm 0.3167 (+1.89z)| lr 1.22e-04 | 322.86 ms | 52.3% bf16 MFU | 1623002 tok/s
step 13947/19560 | loss 3.357650 (+0.48z)| norm 0.2848 (+0.15z)| lr 1.22e-04 | 322.91 ms | 52.3% bf16 MFU | 1623033 tok/s
step 13948/19560 | loss 3.279324 (-1.14z)| norm 0.3078 (+1.40z)| lr 1.22e-04 | 322.83 ms | 52.3% bf16 MFU | 1623083 tok/s
step 13949/19560 | loss 3.296560 (-0.78z)| norm 0.2748 (-0.40z)| lr 1.22e-04 | 323.21 ms | 52.2% bf16 MFU | 1623036 tok/s
step 13950/19560 | loss 3.294338 (-0.81z)| norm 0.2818 (-0.01z)| lr 1.22e-04 | 323.39 ms | 52.2% bf16 MFU | 1622945 tok/s
step 13951/19560 | loss 3.330242 (-0.06z)| norm 0.2814 (-0.03z)| lr 1.22e-04 | 322.70 ms | 52.3% bf16 MFU | 1623033 tok/s
step 13952/19560 | loss 3.330821 (-0.05z)| norm 0.2596 (-1.20z)| lr 1.22e-04 | 323.18 ms | 52.2% bf16 MFU | 1622996 tok/s
step 13953/19560 | loss 3.303900 (-0.60z)| norm 0.2789 (-0.15z)| lr 1.22e-04 | 323.18 ms | 52.2% bf16 MFU | 1622959 tok/s
step 13954/19560 | loss 3.323561 (-0.19z)| norm 0.2887 (+0.37z)| lr 1.22e-04 | 323.02 ms | 52.2% bf16 MFU | 1622964 tok/s
step 13955/19560 | loss 3.278767 (-1.13z)| norm 0.2836 (+0.10z)| lr 1.22e-04 | 322.64 ms | 52.3% bf16 MFU | 1623064 tok/s
step 13956/19560 | loss 3.388419 (+1.16z)| norm 0.2879 (+0.33z)| lr 1.22e-04 | 322.68 ms | 52.3% bf16 MFU | 1623152 tok/s
step 13957/19560 | loss 3.309320 (-0.49z)| norm 0.2800 (-0.10z)| lr 1.21e-04 | 322.94 ms | 52.3% bf16 MFU | 1623168 tok/s
step 13958/19560 | loss 3.367843 (+0.73z)| norm 0.2763 (-0.30z)| lr 1.21e-04 | 323.26 ms | 52.2% bf16 MFU | 1623103 tok/s
step 13959/19560 | loss 3.330907 (-0.05z)| norm 0.2746 (-0.38z)| lr 1.21e-04 | 322.79 ms | 52.3% bf16 MFU | 1623160 tok/s
step 13960/19560 | loss 3.352596 (+0.40z)| norm 0.2758 (-0.30z)| lr 1.21e-04 | 322.82 ms | 52.3% bf16 MFU | 1623207 tok/s
step 13961/19560 | loss 3.292963 (-0.86z)| norm 0.2630 (-1.03z)| lr 1.21e-04 | 323.01 ms | 52.3% bf16 MFU | 1623204 tok/s
step 13962/19560 | loss 3.275123 (-1.22z)| norm 0.2746 (-0.35z)| lr 1.21e-04 | 322.33 ms | 52.4% bf16 MFU | 1623372 tok/s
step 13963/19560 | loss 3.323547 (-0.19z)| norm 0.2731 (-0.43z)| lr 1.21e-04 | 323.30 ms | 52.2% bf16 MFU | 1623288 tok/s
step 13964/19560 | loss 3.298615 (-0.71z)| norm 0.2731 (-0.43z)| lr 1.21e-04 | 322.34 ms | 52.4% bf16 MFU | 1623448 tok/s
step 13965/19560 | loss 3.332891 (+0.02z)| norm 0.2796 (-0.05z)| lr 1.21e-04 | 323.24 ms | 52.2% bf16 MFU | 1623374 tok/s
step 13966/19560 | loss 3.295292 (-0.77z)| norm 0.2729 (-0.44z)| lr 1.21e-04 | 322.85 ms | 52.3% bf16 MFU | 1623402 tok/s
step 13967/19560 | loss 3.306411 (-0.52z)| norm 0.2842 (+0.22z)| lr 1.21e-04 | 323.05 ms | 52.2% bf16 MFU | 1623378 tok/s
step 13968/19560 | loss 3.334299 (+0.07z)| norm 0.3214 (+2.31z)| lr 1.21e-04 | 322.46 ms | 52.3% bf16 MFU | 1623503 tok/s
step 13969/19560 | loss 3.352386 (+0.46z)| norm 0.2919 (+0.64z)| lr 1.21e-04 | 322.72 ms | 52.3% bf16 MFU | 1623558 tok/s
step 13970/19560 | loss 3.311671 (-0.42z)| norm 0.2933 (+0.71z)| lr 1.21e-04 | 323.12 ms | 52.2% bf16 MFU | 1623510 tok/s
step 13971/19560 | loss 3.271853 (-1.27z)| norm 0.2553 (-1.41z)| lr 1.21e-04 | 322.16 ms | 52.4% bf16 MFU | 1623706 tok/s
step 13972/19560 | loss 3.289060 (-0.92z)| norm 0.2984 (+1.00z)| lr 1.21e-04 | 323.26 ms | 52.2% bf16 MFU | 1623615 tok/s
step 13973/19560 | loss 3.388655 (+1.23z)| norm 0.3041 (+1.30z)| lr 1.21e-04 | 322.69 ms | 52.3% bf16 MFU | 1623671 tok/s
step 13974/19560 | loss 3.344496 (+0.28z)| norm 0.2696 (-0.61z)| lr 1.21e-04 | 322.89 ms | 52.3% bf16 MFU | 1623674 tok/s
step 13975/19560 | loss 3.276182 (-1.19z)| norm 0.2798 (-0.05z)| lr 1.21e-04 | 322.53 ms | 52.3% bf16 MFU | 1623768 tok/s
step 13976/19560 | loss 3.315046 (-0.34z)| norm 0.2838 (+0.18z)| lr 1.21e-04 | 322.70 ms | 52.3% bf16 MFU | 1623815 tok/s
step 13977/19560 | loss 3.383871 (+1.13z)| norm 0.2768 (-0.21z)| lr 1.21e-04 | 322.68 ms | 52.3% bf16 MFU | 1623864 tok/s
step 13978/19560 | loss 3.431495 (+2.10z)| norm 0.3210 (+2.22z)| lr 1.21e-04 | 322.35 ms | 52.4% bf16 MFU | 1623994 tok/s
step 13979/19560 | loss 3.315174 (-0.35z)| norm 0.2947 (+0.77z)| lr 1.21e-04 | 323.23 ms | 52.2% bf16 MFU | 1623896 tok/s
step 13980/19560 | loss 3.359424 (+0.59z)| norm 0.2696 (-0.61z)| lr 1.21e-04 | 323.07 ms | 52.2% bf16 MFU | 1623844 tok/s
step 13981/19560 | loss 3.365727 (+0.74z)| norm 0.2889 (+0.45z)| lr 1.21e-04 | 323.04 ms | 52.2% bf16 MFU | 1623800 tok/s
step 13982/19560 | loss 3.334464 (+0.06z)| norm 0.2743 (-0.36z)| lr 1.20e-04 | 322.94 ms | 52.3% bf16 MFU | 1623783 tok/s
step 13983/19560 | loss 3.436969 (+2.22z)| norm 0.2947 (+0.76z)| lr 1.20e-04 | 322.85 ms | 52.3% bf16 MFU | 1623790 tok/s
step 13984/19560 | loss 3.345350 (+0.26z)| norm 0.2726 (-0.45z)| lr 1.20e-04 | 322.79 ms | 52.3% bf16 MFU | 1623812 tok/s
step 13985/19560 | loss 3.347952 (+0.32z)| norm 0.2739 (-0.39z)| lr 1.20e-04 | 323.17 ms | 52.2% bf16 MFU | 1623738 tok/s
step 13986/19560 | loss 3.276513 (-1.20z)| norm 0.2696 (-0.62z)| lr 1.20e-04 | 322.70 ms | 52.3% bf16 MFU | 1623787 tok/s
step 13987/19560 | loss 3.344913 (+0.25z)| norm 0.2838 (+0.16z)| lr 1.20e-04 | 322.37 ms | 52.4% bf16 MFU | 1623914 tok/s
step 13988/19560 | loss 3.318631 (-0.31z)| norm 0.2738 (-0.40z)| lr 1.20e-04 | 322.99 ms | 52.3% bf16 MFU | 1623880 tok/s
step 13989/19560 | loss 3.325671 (-0.16z)| norm 0.2882 (+0.40z)| lr 1.20e-04 | 322.79 ms | 52.3% bf16 MFU | 1623897 tok/s
step 13990/19560 | loss 3.370512 (+0.78z)| norm 0.2667 (-0.80z)| lr 1.20e-04 | 322.73 ms | 52.3% bf16 MFU | 1623930 tok/s
step 13991/19560 | loss 3.304899 (-0.61z)| norm 0.2673 (-0.77z)| lr 1.20e-04 | 322.66 ms | 52.3% bf16 MFU | 1623977 tok/s
step 13992/19560 | loss 3.355564 (+0.47z)| norm 0.2780 (-0.18z)| lr 1.20e-04 | 322.84 ms | 52.3% bf16 MFU | 1623978 tok/s
step 13993/19560 | loss 3.302797 (-0.65z)| norm 0.2540 (-1.49z)| lr 1.20e-04 | 322.13 ms | 52.4% bf16 MFU | 1624157 tok/s
step 13994/19560 | loss 3.309601 (-0.51z)| norm 0.2665 (-0.78z)| lr 1.20e-04 | 323.08 ms | 52.2% bf16 MFU | 1624088 tok/s
step 13995/19560 | loss 3.307176 (-0.56z)| norm 0.2621 (-1.02z)| lr 1.20e-04 | 322.96 ms | 52.3% bf16 MFU | 1624052 tok/s
step 13996/19560 | loss 3.256313 (-1.64z)| norm 0.2697 (-0.60z)| lr 1.20e-04 | 322.62 ms | 52.3% bf16 MFU | 1624104 tok/s
step 13997/19560 | loss 3.408674 (+1.59z)| norm 0.2658 (-0.82z)| lr 1.20e-04 | 322.98 ms | 52.3% bf16 MFU | 1624063 tok/s
step 13998/19560 | loss 3.317378 (-0.35z)| norm 0.2726 (-0.44z)| lr 1.20e-04 | 322.73 ms | 52.3% bf16 MFU | 1624087 tok/s
step 13999/19560 | loss 3.276218 (-1.21z)| norm 0.2595 (-1.15z)| lr 1.20e-04 | 322.54 ms | 52.3% bf16 MFU | 1624158 tok/s
step 14000/19560 | loss 3.323000 (-0.21z)| norm 0.2705 (-0.55z)| lr 1.20e-04 | 323.74 ms | 52.1% bf16 MFU | 1623923 tok/s
val loss 3.317825
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3012/10042 = 0.299940
step 14001/19560 | loss 3.298245 (-0.73z)| norm 0.2706 (-0.55z)| lr 1.20e-04 | 322.44 ms | 52.3% bf16 MFU | 1624027 tok/s
step 14002/19560 | loss 3.350191 (+0.36z)| norm 0.2787 (-0.10z)| lr 1.20e-04 | 322.60 ms | 52.3% bf16 MFU | 1624086 tok/s
step 14003/19560 | loss 3.315331 (-0.40z)| norm 0.2698 (-0.60z)| lr 1.20e-04 | 322.92 ms | 52.3% bf16 MFU | 1624060 tok/s
step 14004/19560 | loss 3.285808 (-1.03z)| norm 0.2742 (-0.37z)| lr 1.20e-04 | 322.42 ms | 52.3% bf16 MFU | 1624163 tok/s
step 14005/19560 | loss 3.316741 (-0.36z)| norm 0.2925 (+0.66z)| lr 1.20e-04 | 322.50 ms | 52.3% bf16 MFU | 1624241 tok/s
step 14006/19560 | loss 3.303274 (-0.65z)| norm 0.2641 (-0.94z)| lr 1.20e-04 | 323.04 ms | 52.2% bf16 MFU | 1624177 tok/s
step 14007/19560 | loss 3.310030 (-0.49z)| norm 0.2764 (-0.25z)| lr 1.19e-04 | 323.01 ms | 52.2% bf16 MFU | 1624125 tok/s
step 14008/19560 | loss 3.401561 (+1.46z)| norm 0.2847 (+0.22z)| lr 1.19e-04 | 322.37 ms | 52.4% bf16 MFU | 1624235 tok/s
step 14009/19560 | loss 3.375992 (+0.91z)| norm 0.2676 (-0.76z)| lr 1.19e-04 | 322.46 ms | 52.3% bf16 MFU | 1624318 tok/s
step 14010/19560 | loss 3.288927 (-0.95z)| norm 0.2705 (-0.58z)| lr 1.19e-04 | 323.10 ms | 52.2% bf16 MFU | 1624236 tok/s
step 14011/19560 | loss 3.328336 (-0.11z)| norm 0.2792 (-0.09z)| lr 1.19e-04 | 322.64 ms | 52.3% bf16 MFU | 1624275 tok/s
step 14012/19560 | loss 3.260869 (-1.52z)| norm 0.2740 (-0.39z)| lr 1.19e-04 | 322.75 ms | 52.3% bf16 MFU | 1624285 tok/s
step 14013/19560 | loss 3.280860 (-1.09z)| norm 0.2527 (-1.59z)| lr 1.19e-04 | 323.10 ms | 52.2% bf16 MFU | 1624205 tok/s
step 14014/19560 | loss 3.387874 (+1.16z)| norm 0.2783 (-0.14z)| lr 1.19e-04 | 322.48 ms | 52.3% bf16 MFU | 1624285 tok/s
step 14015/19560 | loss 3.343776 (+0.23z)| norm 0.2802 (-0.04z)| lr 1.19e-04 | 323.44 ms | 52.2% bf16 MFU | 1624120 tok/s
step 14016/19560 | loss 3.284104 (-1.03z)| norm 0.2528 (-1.61z)| lr 1.19e-04 | 322.47 ms | 52.3% bf16 MFU | 1624208 tok/s
step 14017/19560 | loss 3.339962 (+0.15z)| norm 0.2924 (+0.66z)| lr 1.19e-04 | 322.96 ms | 52.3% bf16 MFU | 1624167 tok/s
step 14018/19560 | loss 3.369094 (+0.76z)| norm 0.2699 (-0.63z)| lr 1.19e-04 | 322.80 ms | 52.3% bf16 MFU | 1624169 tok/s
step 14019/19560 | loss 3.340400 (+0.16z)| norm 0.3035 (+1.27z)| lr 1.19e-04 | 322.37 ms | 52.4% bf16 MFU | 1624279 tok/s
step 14020/19560 | loss 3.393204 (+1.26z)| norm 0.3181 (+2.04z)| lr 1.19e-04 | 323.34 ms | 52.2% bf16 MFU | 1624139 tok/s
step 14021/19560 | loss 3.330977 (-0.04z)| norm 0.2994 (+0.98z)| lr 1.19e-04 | 322.87 ms | 52.3% bf16 MFU | 1624123 tok/s
step 14022/19560 | loss 3.322808 (-0.20z)| norm 0.2923 (+0.59z)| lr 1.19e-04 | 322.59 ms | 52.3% bf16 MFU | 1624179 tok/s
step 14023/19560 | loss 3.374597 (+0.87z)| norm 0.3258 (+2.39z)| lr 1.19e-04 | 322.64 ms | 52.3% bf16 MFU | 1624221 tok/s
step 14024/19560 | loss 3.318940 (-0.30z)| norm 0.2802 (-0.12z)| lr 1.19e-04 | 322.56 ms | 52.3% bf16 MFU | 1624280 tok/s
step 14025/19560 | loss 3.297426 (-0.75z)| norm 0.2842 (+0.10z)| lr 1.19e-04 | 323.18 ms | 52.2% bf16 MFU | 1624180 tok/s
step 14026/19560 | loss 3.315622 (-0.36z)| norm 0.2876 (+0.29z)| lr 1.19e-04 | 322.28 ms | 52.4% bf16 MFU | 1624311 tok/s
step 14027/19560 | loss 3.364309 (+0.68z)| norm 0.3281 (+2.46z)| lr 1.19e-04 | 322.89 ms | 52.3% bf16 MFU | 1624282 tok/s
step 14028/19560 | loss 3.281745 (-1.10z)| norm 0.2643 (-0.97z)| lr 1.19e-04 | 323.25 ms | 52.2% bf16 MFU | 1624165 tok/s
step 14029/19560 | loss 3.326450 (-0.13z)| norm 0.2798 (-0.14z)| lr 1.19e-04 | 322.15 ms | 52.4% bf16 MFU | 1624331 tok/s
step 14030/19560 | loss 3.332259 (+0.01z)| norm 0.2817 (-0.03z)| lr 1.19e-04 | 323.39 ms | 52.2% bf16 MFU | 1624177 tok/s
step 14031/19560 | loss 3.387872 (+1.21z)| norm 0.2937 (+0.62z)| lr 1.19e-04 | 322.87 ms | 52.3% bf16 MFU | 1624160 tok/s
step 14032/19560 | loss 3.344112 (+0.26z)| norm 0.2854 (+0.16z)| lr 1.18e-04 | 323.33 ms | 52.2% bf16 MFU | 1624028 tok/s
step 14033/19560 | loss 3.352919 (+0.44z)| norm 0.2668 (-0.84z)| lr 1.18e-04 | 322.69 ms | 52.3% bf16 MFU | 1624064 tok/s
step 14034/19560 | loss 3.292271 (-0.88z)| norm 0.2695 (-0.70z)| lr 1.18e-04 | 322.70 ms | 52.3% bf16 MFU | 1624094 tok/s
step 14035/19560 | loss 3.348239 (+0.34z)| norm 0.2690 (-0.73z)| lr 1.18e-04 | 322.34 ms | 52.4% bf16 MFU | 1624215 tok/s
step 14036/19560 | loss 3.339957 (+0.16z)| norm 0.2685 (-0.76z)| lr 1.18e-04 | 322.82 ms | 52.3% bf16 MFU | 1624209 tok/s
step 14037/19560 | loss 3.289270 (-0.93z)| norm 0.2653 (-0.93z)| lr 1.18e-04 | 321.97 ms | 52.4% bf16 MFU | 1624417 tok/s
step 14038/19560 | loss 3.265213 (-1.44z)| norm 0.2893 (+0.41z)| lr 1.18e-04 | 323.31 ms | 52.2% bf16 MFU | 1624276 tok/s
step 14039/19560 | loss 3.335123 (+0.07z)| norm 0.2695 (-0.71z)| lr 1.18e-04 | 322.29 ms | 52.4% bf16 MFU | 1624401 tok/s
step 14040/19560 | loss 3.353635 (+0.47z)| norm 0.2668 (-0.85z)| lr 1.18e-04 | 322.54 ms | 52.3% bf16 MFU | 1624455 tok/s
step 14041/19560 | loss 3.395917 (+1.37z)| norm 0.2636 (-1.03z)| lr 1.18e-04 | 322.48 ms | 52.3% bf16 MFU | 1624522 tok/s
step 14042/19560 | loss 3.333601 (+0.01z)| norm 0.2630 (-1.04z)| lr 1.18e-04 | 323.04 ms | 52.2% bf16 MFU | 1624445 tok/s
step 14043/19560 | loss 3.346754 (+0.29z)| norm 0.2656 (-0.88z)| lr 1.18e-04 | 322.46 ms | 52.3% bf16 MFU | 1624518 tok/s
step 14044/19560 | loss 3.376853 (+0.94z)| norm 0.2604 (-1.15z)| lr 1.18e-04 | 322.47 ms | 52.3% bf16 MFU | 1624586 tok/s
step 14045/19560 | loss 3.311500 (-0.50z)| norm 0.2539 (-1.50z)| lr 1.18e-04 | 322.83 ms | 52.3% bf16 MFU | 1624558 tok/s
step 14046/19560 | loss 3.354945 (+0.45z)| norm 0.2653 (-0.86z)| lr 1.18e-04 | 322.98 ms | 52.3% bf16 MFU | 1624494 tok/s
step 14047/19560 | loss 3.290483 (-0.95z)| norm 0.2621 (-1.02z)| lr 1.18e-04 | 323.02 ms | 52.2% bf16 MFU | 1624424 tok/s
step 14048/19560 | loss 3.340961 (+0.16z)| norm 0.2700 (-0.58z)| lr 1.18e-04 | 322.69 ms | 52.3% bf16 MFU | 1624440 tok/s
step 14049/19560 | loss 3.342719 (+0.19z)| norm 0.2749 (-0.30z)| lr 1.18e-04 | 322.62 ms | 52.3% bf16 MFU | 1624472 tok/s
step 14050/19560 | loss 3.272576 (-1.34z)| norm 0.2599 (-1.12z)| lr 1.18e-04 | 323.12 ms | 52.2% bf16 MFU | 1624378 tok/s
step 14051/19560 | loss 3.330893 (-0.04z)| norm 0.2720 (-0.44z)| lr 1.18e-04 | 322.60 ms | 52.3% bf16 MFU | 1624420 tok/s
step 14052/19560 | loss 3.309248 (-0.52z)| norm 0.2725 (-0.41z)| lr 1.18e-04 | 323.18 ms | 52.2% bf16 MFU | 1624313 tok/s
step 14053/19560 | loss 3.363680 (+0.71z)| norm 0.2900 (+0.56z)| lr 1.18e-04 | 322.59 ms | 52.3% bf16 MFU | 1624359 tok/s
step 14054/19560 | loss 3.384230 (+1.16z)| norm 0.2812 (+0.06z)| lr 1.18e-04 | 323.13 ms | 52.2% bf16 MFU | 1624266 tok/s
step 14055/19560 | loss 3.290983 (-0.92z)| norm 0.2933 (+0.72z)| lr 1.18e-04 | 322.89 ms | 52.3% bf16 MFU | 1624241 tok/s
step 14056/19560 | loss 3.328060 (-0.09z)| norm 0.2795 (-0.05z)| lr 1.18e-04 | 323.76 ms | 52.1% bf16 MFU | 1623998 tok/s
step 14057/19560 | loss 3.180851 (-3.24z)| norm 0.2740 (-0.36z)| lr 1.17e-04 | 322.99 ms | 52.3% bf16 MFU | 1623958 tok/s
step 14058/19560 | loss 3.345722 (+0.31z)| norm 0.2991 (+1.03z)| lr 1.17e-04 | 322.81 ms | 52.3% bf16 MFU | 1623968 tok/s
step 14059/19560 | loss 3.352967 (+0.46z)| norm 0.2522 (-1.59z)| lr 1.17e-04 | 323.64 ms | 52.1% bf16 MFU | 1623767 tok/s
step 14060/19560 | loss 3.266270 (-1.40z)| norm 0.2956 (+0.82z)| lr 1.17e-04 | 323.06 ms | 52.2% bf16 MFU | 1623722 tok/s
step 14061/19560 | loss 3.320467 (-0.22z)| norm 0.2693 (-0.63z)| lr 1.17e-04 | 323.08 ms | 52.2% bf16 MFU | 1623675 tok/s
step 14062/19560 | loss 3.364221 (+0.80z)| norm 0.3507 (+3.72z)| lr 1.17e-04 | 322.60 ms | 52.3% bf16 MFU | 1623750 tok/s
step 14063/19560 | loss 3.322852 (-0.15z)| norm 0.2821 (+0.06z)| lr 1.17e-04 | 322.72 ms | 52.3% bf16 MFU | 1623791 tok/s
step 14064/19560 | loss 3.324468 (-0.13z)| norm 0.2852 (+0.23z)| lr 1.17e-04 | 323.52 ms | 52.2% bf16 MFU | 1623630 tok/s
step 14065/19560 | loss 3.301973 (-0.65z)| norm 0.3059 (+1.32z)| lr 1.17e-04 | 323.05 ms | 52.2% bf16 MFU | 1623596 tok/s
step 14066/19560 | loss 3.401860 (+1.64z)| norm 0.3021 (+1.10z)| lr 1.17e-04 | 322.86 ms | 52.3% bf16 MFU | 1623611 tok/s
step 14067/19560 | loss 3.320134 (-0.24z)| norm 0.2884 (+0.41z)| lr 1.17e-04 | 323.01 ms | 52.3% bf16 MFU | 1623588 tok/s
step 14068/19560 | loss 3.356383 (+0.60z)| norm 0.2714 (-0.51z)| lr 1.17e-04 | 322.82 ms | 52.3% bf16 MFU | 1623613 tok/s
step 14069/19560 | loss 3.287099 (-0.98z)| norm 0.3082 (+1.47z)| lr 1.17e-04 | 323.09 ms | 52.2% bf16 MFU | 1623570 tok/s
step 14070/19560 | loss 3.308634 (-0.50z)| norm 0.2809 (+0.03z)| lr 1.17e-04 | 323.57 ms | 52.2% bf16 MFU | 1623406 tok/s
step 14071/19560 | loss 3.298944 (-0.74z)| norm 0.2876 (+0.43z)| lr 1.17e-04 | 322.98 ms | 52.3% bf16 MFU | 1623400 tok/s
step 14072/19560 | loss 3.320139 (-0.18z)| norm 0.2871 (+0.41z)| lr 1.17e-04 | 322.93 ms | 52.3% bf16 MFU | 1623408 tok/s
step 14073/19560 | loss 3.273933 (-1.35z)| norm 0.2692 (-0.65z)| lr 1.17e-04 | 323.31 ms | 52.2% bf16 MFU | 1623318 tok/s
step 14074/19560 | loss 3.285946 (-1.04z)| norm 0.2973 (+1.07z)| lr 1.17e-04 | 323.67 ms | 52.1% bf16 MFU | 1623143 tok/s
step 14075/19560 | loss 3.344711 (+0.46z)| norm 0.2796 (-0.01z)| lr 1.17e-04 | 322.56 ms | 52.3% bf16 MFU | 1623257 tok/s
step 14076/19560 | loss 3.336578 (+0.24z)| norm 0.2898 (+0.63z)| lr 1.17e-04 | 323.17 ms | 52.2% bf16 MFU | 1623211 tok/s
step 14077/19560 | loss 3.356413 (+0.74z)| norm 0.2923 (+0.77z)| lr 1.17e-04 | 322.91 ms | 52.3% bf16 MFU | 1623232 tok/s
step 14078/19560 | loss 3.337855 (+0.26z)| norm 0.2847 (+0.30z)| lr 1.17e-04 | 322.99 ms | 52.3% bf16 MFU | 1623231 tok/s
step 14079/19560 | loss 3.340540 (+0.32z)| norm 0.2681 (-0.71z)| lr 1.17e-04 | 323.95 ms | 52.1% bf16 MFU | 1622991 tok/s
step 14080/19560 | loss 3.353834 (+0.66z)| norm 0.2956 (+0.96z)| lr 1.17e-04 | 322.98 ms | 52.3% bf16 MFU | 1623006 tok/s
step 14081/19560 | loss 3.342983 (+0.37z)| norm 0.3030 (+1.39z)| lr 1.17e-04 | 323.23 ms | 52.2% bf16 MFU | 1622957 tok/s
step 14082/19560 | loss 3.272157 (-1.42z)| norm 0.2873 (+0.43z)| lr 1.17e-04 | 322.87 ms | 52.3% bf16 MFU | 1623000 tok/s
step 14083/19560 | loss 3.310712 (-0.45z)| norm 0.2701 (-0.61z)| lr 1.16e-04 | 323.11 ms | 52.2% bf16 MFU | 1622982 tok/s
step 14084/19560 | loss 3.273499 (-1.38z)| norm 0.2825 (+0.15z)| lr 1.16e-04 | 323.12 ms | 52.2% bf16 MFU | 1622963 tok/s
step 14085/19560 | loss 3.337327 (+0.25z)| norm 0.2826 (+0.15z)| lr 1.16e-04 | 323.30 ms | 52.2% bf16 MFU | 1622898 tok/s
step 14086/19560 | loss 3.276414 (-1.29z)| norm 0.2902 (+0.61z)| lr 1.16e-04 | 322.88 ms | 52.3% bf16 MFU | 1622942 tok/s
step 14087/19560 | loss 3.294107 (-0.83z)| norm 0.2831 (+0.18z)| lr 1.16e-04 | 322.77 ms | 52.3% bf16 MFU | 1623012 tok/s
step 14088/19560 | loss 3.315148 (-0.29z)| norm 0.2769 (-0.20z)| lr 1.16e-04 | 323.22 ms | 52.2% bf16 MFU | 1622965 tok/s
step 14089/19560 | loss 3.313587 (-0.33z)| norm 0.2887 (+0.50z)| lr 1.16e-04 | 322.89 ms | 52.3% bf16 MFU | 1623004 tok/s
step 14090/19560 | loss 3.379635 (+1.34z)| norm 0.2900 (+0.58z)| lr 1.16e-04 | 323.32 ms | 52.2% bf16 MFU | 1622934 tok/s
step 14091/19560 | loss 3.268061 (-1.49z)| norm 0.2828 (+0.13z)| lr 1.16e-04 | 323.34 ms | 52.2% bf16 MFU | 1622860 tok/s
step 14092/19560 | loss 3.388343 (+1.53z)| norm 0.2612 (-1.17z)| lr 1.16e-04 | 323.40 ms | 52.2% bf16 MFU | 1622776 tok/s
step 14093/19560 | loss 3.372018 (+1.11z)| norm 0.3029 (+1.34z)| lr 1.16e-04 | 322.76 ms | 52.3% bf16 MFU | 1622858 tok/s
step 14094/19560 | loss 3.372842 (+1.11z)| norm 0.2758 (-0.30z)| lr 1.16e-04 | 323.10 ms | 52.2% bf16 MFU | 1622848 tok/s
step 14095/19560 | loss 3.319450 (-0.23z)| norm 0.2944 (+0.82z)| lr 1.16e-04 | 322.56 ms | 52.3% bf16 MFU | 1622977 tok/s
step 14096/19560 | loss 3.327371 (-0.03z)| norm 0.2786 (-0.12z)| lr 1.16e-04 | 322.57 ms | 52.3% bf16 MFU | 1623095 tok/s
step 14097/19560 | loss 3.316841 (-0.29z)| norm 0.2877 (+0.45z)| lr 1.16e-04 | 322.71 ms | 52.3% bf16 MFU | 1623171 tok/s
step 14098/19560 | loss 3.336947 (+0.21z)| norm 0.2937 (+0.82z)| lr 1.16e-04 | 323.14 ms | 52.2% bf16 MFU | 1623137 tok/s
step 14099/19560 | loss 3.304093 (-0.62z)| norm 0.2659 (-0.91z)| lr 1.16e-04 | 323.37 ms | 52.2% bf16 MFU | 1623047 tok/s
step 14100/19560 | loss 3.302311 (-0.67z)| norm 0.2794 (-0.06z)| lr 1.16e-04 | 322.75 ms | 52.3% bf16 MFU | 1623118 tok/s
step 14101/19560 | loss 3.344302 (+0.40z)| norm 0.2713 (-0.56z)| lr 1.16e-04 | 322.60 ms | 52.3% bf16 MFU | 1623221 tok/s
step 14102/19560 | loss 3.299888 (-0.72z)| norm 0.2997 (+1.22z)| lr 1.16e-04 | 323.11 ms | 52.2% bf16 MFU | 1623192 tok/s
step 14103/19560 | loss 3.344870 (+0.42z)| norm 0.2874 (+0.44z)| lr 1.16e-04 | 323.27 ms | 52.2% bf16 MFU | 1623124 tok/s
step 14104/19560 | loss 3.288192 (-1.03z)| norm 0.2902 (+0.61z)| lr 1.16e-04 | 323.16 ms | 52.2% bf16 MFU | 1623087 tok/s
step 14105/19560 | loss 3.332894 (+0.12z)| norm 0.2809 (+0.02z)| lr 1.16e-04 | 322.37 ms | 52.4% bf16 MFU | 1623249 tok/s
step 14106/19560 | loss 3.354282 (+0.71z)| norm 0.2874 (+0.46z)| lr 1.16e-04 | 323.87 ms | 52.1% bf16 MFU | 1623029 tok/s
step 14107/19560 | loss 3.350541 (+0.60z)| norm 0.3019 (+1.38z)| lr 1.16e-04 | 323.40 ms | 52.2% bf16 MFU | 1622936 tok/s
step 14108/19560 | loss 3.360310 (+0.86z)| norm 0.2694 (-0.69z)| lr 1.15e-04 | 322.73 ms | 52.3% bf16 MFU | 1623016 tok/s
step 14109/19560 | loss 3.336608 (+0.24z)| norm 0.2990 (+1.19z)| lr 1.15e-04 | 323.20 ms | 52.2% bf16 MFU | 1622974 tok/s
step 14110/19560 | loss 3.308435 (-0.50z)| norm 0.2766 (-0.24z)| lr 1.15e-04 | 323.02 ms | 52.2% bf16 MFU | 1622980 tok/s
step 14111/19560 | loss 3.305765 (-0.56z)| norm 0.2615 (-1.18z)| lr 1.15e-04 | 324.08 ms | 52.1% bf16 MFU | 1622720 tok/s
step 14112/19560 | loss 3.317224 (-0.24z)| norm 0.2804 (+0.01z)| lr 1.15e-04 | 323.20 ms | 52.2% bf16 MFU | 1622693 tok/s
step 14113/19560 | loss 3.386513 (+1.63z)| norm 0.2617 (-1.17z)| lr 1.15e-04 | 322.58 ms | 52.3% bf16 MFU | 1622824 tok/s
step 14114/19560 | loss 3.355646 (+0.78z)| norm 0.2636 (-1.04z)| lr 1.15e-04 | 323.42 ms | 52.2% bf16 MFU | 1622736 tok/s
step 14115/19560 | loss 3.303669 (-0.63z)| norm 0.2716 (-0.53z)| lr 1.15e-04 | 323.48 ms | 52.2% bf16 MFU | 1622638 tok/s
step 14116/19560 | loss 3.305959 (-0.56z)| norm 0.2527 (-1.69z)| lr 1.15e-04 | 322.78 ms | 52.3% bf16 MFU | 1622720 tok/s
step 14117/19560 | loss 3.314254 (-0.33z)| norm 0.3048 (+1.54z)| lr 1.15e-04 | 323.02 ms | 52.2% bf16 MFU | 1622737 tok/s
step 14118/19560 | loss 3.239240 (-2.31z)| norm 0.2580 (-1.35z)| lr 1.15e-04 | 322.56 ms | 52.3% bf16 MFU | 1622870 tok/s
step 14119/19560 | loss 3.281102 (-1.18z)| norm 0.3269 (+2.79z)| lr 1.15e-04 | 323.04 ms | 52.2% bf16 MFU | 1622876 tok/s
step 14120/19560 | loss 3.360022 (+0.93z)| norm 0.2632 (-1.02z)| lr 1.15e-04 | 322.71 ms | 52.3% bf16 MFU | 1622965 tok/s
step 14121/19560 | loss 3.307479 (-0.48z)| norm 0.2689 (-0.69z)| lr 1.15e-04 | 322.39 ms | 52.4% bf16 MFU | 1623129 tok/s
step 14122/19560 | loss 3.367025 (+1.10z)| norm 0.3433 (+3.56z)| lr 1.15e-04 | 322.77 ms | 52.3% bf16 MFU | 1623189 tok/s
step 14123/19560 | loss 3.363962 (+1.00z)| norm 0.2598 (-1.21z)| lr 1.15e-04 | 322.62 ms | 52.3% bf16 MFU | 1623286 tok/s
step 14124/19560 | loss 3.341010 (+0.38z)| norm 0.2870 (+0.34z)| lr 1.15e-04 | 323.03 ms | 52.2% bf16 MFU | 1623272 tok/s
step 14125/19560 | loss 3.308404 (-0.48z)| norm 0.2906 (+0.53z)| lr 1.15e-04 | 323.05 ms | 52.2% bf16 MFU | 1623256 tok/s
step 14126/19560 | loss 3.272192 (-1.45z)| norm 0.2814 (+0.00z)| lr 1.15e-04 | 322.91 ms | 52.3% bf16 MFU | 1623275 tok/s
step 14127/19560 | loss 3.341940 (+0.43z)| norm 0.2956 (+0.81z)| lr 1.15e-04 | 322.79 ms | 52.3% bf16 MFU | 1623324 tok/s
step 14128/19560 | loss 3.315019 (-0.31z)| norm 0.2807 (-0.06z)| lr 1.15e-04 | 323.32 ms | 52.2% bf16 MFU | 1623236 tok/s
step 14129/19560 | loss 3.313834 (-0.34z)| norm 0.2956 (+0.79z)| lr 1.15e-04 | 323.08 ms | 52.2% bf16 MFU | 1623212 tok/s
step 14130/19560 | loss 3.328103 (+0.05z)| norm 0.2693 (-0.72z)| lr 1.15e-04 | 322.60 ms | 52.3% bf16 MFU | 1623311 tok/s
step 14131/19560 | loss 3.385409 (+1.59z)| norm 0.3143 (+1.83z)| lr 1.15e-04 | 322.74 ms | 52.3% bf16 MFU | 1623370 tok/s
step 14132/19560 | loss 3.292066 (-0.94z)| norm 0.2850 (+0.16z)| lr 1.15e-04 | 323.28 ms | 52.2% bf16 MFU | 1623289 tok/s
step 14133/19560 | loss 3.367958 (+1.10z)| norm 0.2750 (-0.40z)| lr 1.14e-04 | 323.21 ms | 52.2% bf16 MFU | 1623231 tok/s
step 14134/19560 | loss 3.324466 (-0.08z)| norm 0.3052 (+1.29z)| lr 1.14e-04 | 322.87 ms | 52.3% bf16 MFU | 1623262 tok/s
step 14135/19560 | loss 3.334403 (+0.19z)| norm 0.2778 (-0.26z)| lr 1.14e-04 | 323.07 ms | 52.2% bf16 MFU | 1623239 tok/s
step 14136/19560 | loss 3.334037 (+0.19z)| norm 0.2934 (+0.62z)| lr 1.14e-04 | 323.59 ms | 52.2% bf16 MFU | 1623087 tok/s
step 14137/19560 | loss 3.295674 (-0.85z)| norm 0.2795 (-0.17z)| lr 1.14e-04 | 323.06 ms | 52.2% bf16 MFU | 1623077 tok/s
step 14138/19560 | loss 3.337204 (+0.29z)| norm 0.3014 (+1.05z)| lr 1.14e-04 | 322.86 ms | 52.3% bf16 MFU | 1623118 tok/s
step 14139/19560 | loss 3.355072 (+0.78z)| norm 0.2959 (+0.73z)| lr 1.14e-04 | 322.86 ms | 52.3% bf16 MFU | 1623156 tok/s
step 14140/19560 | loss 3.306514 (-0.58z)| norm 0.2684 (-0.82z)| lr 1.14e-04 | 322.84 ms | 52.3% bf16 MFU | 1623196 tok/s
step 14141/19560 | loss 3.382636 (+1.53z)| norm 0.2974 (+0.80z)| lr 1.14e-04 | 322.38 ms | 52.4% bf16 MFU | 1623352 tok/s
step 14142/19560 | loss 3.314351 (-0.37z)| norm 0.2726 (-0.61z)| lr 1.14e-04 | 323.04 ms | 52.2% bf16 MFU | 1623333 tok/s
step 14143/19560 | loss 3.327447 (+0.00z)| norm 0.3086 (+1.42z)| lr 1.14e-04 | 322.89 ms | 52.3% bf16 MFU | 1623353 tok/s
step 14144/19560 | loss 3.367082 (+1.11z)| norm 0.2754 (-0.47z)| lr 1.14e-04 | 322.78 ms | 52.3% bf16 MFU | 1623400 tok/s
step 14145/19560 | loss 3.384434 (+1.57z)| norm 0.3131 (+1.66z)| lr 1.14e-04 | 322.52 ms | 52.3% bf16 MFU | 1623510 tok/s
step 14146/19560 | loss 3.300690 (-0.77z)| norm 0.3031 (+1.08z)| lr 1.14e-04 | 322.87 ms | 52.3% bf16 MFU | 1623525 tok/s
step 14147/19560 | loss 3.311049 (-0.47z)| norm 0.3306 (+2.56z)| lr 1.14e-04 | 322.94 ms | 52.3% bf16 MFU | 1623523 tok/s
step 14148/19560 | loss 3.331418 (+0.12z)| norm 0.2873 (+0.19z)| lr 1.14e-04 | 322.58 ms | 52.3% bf16 MFU | 1623612 tok/s
step 14149/19560 | loss 3.377367 (+1.41z)| norm 0.3053 (+1.18z)| lr 1.14e-04 | 322.76 ms | 52.3% bf16 MFU | 1623652 tok/s
step 14150/19560 | loss 3.326991 (-0.02z)| norm 0.2769 (-0.39z)| lr 1.14e-04 | 323.26 ms | 52.2% bf16 MFU | 1623564 tok/s
step 14151/19560 | loss 3.345023 (+0.50z)| norm 0.3121 (+1.60z)| lr 1.14e-04 | 322.50 ms | 52.3% bf16 MFU | 1623671 tok/s
step 14152/19560 | loss 3.367961 (+1.14z)| norm 0.2935 (+0.54z)| lr 1.14e-04 | 322.60 ms | 52.3% bf16 MFU | 1623748 tok/s
step 14153/19560 | loss 3.337338 (+0.27z)| norm 0.2800 (-0.22z)| lr 1.14e-04 | 322.80 ms | 52.3% bf16 MFU | 1623769 tok/s
step 14154/19560 | loss 3.428993 (+2.76z)| norm 0.3157 (+1.76z)| lr 1.14e-04 | 322.98 ms | 52.3% bf16 MFU | 1623746 tok/s
step 14155/19560 | loss 3.349818 (+0.58z)| norm 0.2740 (-0.55z)| lr 1.14e-04 | 322.38 ms | 52.4% bf16 MFU | 1623874 tok/s
step 14156/19560 | loss 3.356455 (+0.75z)| norm 0.2965 (+0.72z)| lr 1.14e-04 | 322.92 ms | 52.3% bf16 MFU | 1623860 tok/s
step 14157/19560 | loss 3.307134 (-0.61z)| norm 0.2992 (+0.86z)| lr 1.14e-04 | 322.96 ms | 52.3% bf16 MFU | 1623835 tok/s
step 14158/19560 | loss 3.306260 (-0.63z)| norm 0.2655 (-1.05z)| lr 1.14e-04 | 322.51 ms | 52.3% bf16 MFU | 1623927 tok/s
step 14159/19560 | loss 3.323752 (-0.13z)| norm 0.2961 (+0.69z)| lr 1.13e-04 | 322.75 ms | 52.3% bf16 MFU | 1623954 tok/s
step 14160/19560 | loss 3.306221 (-0.61z)| norm 0.2666 (-0.97z)| lr 1.13e-04 | 323.34 ms | 52.2% bf16 MFU | 1623828 tok/s
step 14161/19560 | loss 3.292856 (-0.97z)| norm 0.2717 (-0.69z)| lr 1.13e-04 | 322.97 ms | 52.3% bf16 MFU | 1623803 tok/s
step 14162/19560 | loss 3.342580 (+0.40z)| norm 0.2720 (-0.68z)| lr 1.13e-04 | 322.71 ms | 52.3% bf16 MFU | 1623845 tok/s
step 14163/19560 | loss 3.395538 (+1.85z)| norm 0.2706 (-0.76z)| lr 1.13e-04 | 322.70 ms | 52.3% bf16 MFU | 1623888 tok/s
step 14164/19560 | loss 3.359900 (+0.86z)| norm 0.2776 (-0.36z)| lr 1.13e-04 | 323.09 ms | 52.2% bf16 MFU | 1623831 tok/s
step 14165/19560 | loss 3.342155 (+0.36z)| norm 0.2688 (-0.86z)| lr 1.13e-04 | 322.53 ms | 52.3% bf16 MFU | 1623918 tok/s
step 14166/19560 | loss 3.332550 (+0.08z)| norm 0.2720 (-0.68z)| lr 1.13e-04 | 322.81 ms | 52.3% bf16 MFU | 1623929 tok/s
step 14167/19560 | loss 3.402846 (+2.01z)| norm 0.2945 (+0.60z)| lr 1.13e-04 | 323.05 ms | 52.2% bf16 MFU | 1623878 tok/s
step 14168/19560 | loss 3.454607 (+3.27z)| norm 0.2582 (-1.46z)| lr 1.13e-04 | 322.52 ms | 52.3% bf16 MFU | 1623965 tok/s
step 14169/19560 | loss 3.304384 (-0.69z)| norm 0.2728 (-0.64z)| lr 1.13e-04 | 322.43 ms | 52.3% bf16 MFU | 1624069 tok/s
step 14170/19560 | loss 3.338022 (+0.21z)| norm 0.2716 (-0.72z)| lr 1.13e-04 | 322.99 ms | 52.3% bf16 MFU | 1624027 tok/s
step 14171/19560 | loss 3.315893 (-0.38z)| norm 0.2630 (-1.21z)| lr 1.13e-04 | 323.82 ms | 52.1% bf16 MFU | 1623779 tok/s
step 14172/19560 | loss 3.255866 (-1.94z)| norm 0.2688 (-0.88z)| lr 1.13e-04 | 322.47 ms | 52.3% bf16 MFU | 1623882 tok/s
step 14173/19560 | loss 3.299174 (-0.79z)| norm 0.2814 (-0.17z)| lr 1.13e-04 | 322.57 ms | 52.3% bf16 MFU | 1623954 tok/s
step 14174/19560 | loss 3.372974 (+1.16z)| norm 0.2711 (-0.77z)| lr 1.13e-04 | 322.67 ms | 52.3% bf16 MFU | 1623998 tok/s
step 14175/19560 | loss 3.326555 (-0.07z)| norm 0.2784 (-0.36z)| lr 1.13e-04 | 323.43 ms | 52.2% bf16 MFU | 1623849 tok/s
step 14176/19560 | loss 3.291039 (-1.00z)| norm 0.2792 (-0.32z)| lr 1.13e-04 | 322.46 ms | 52.3% bf16 MFU | 1623952 tok/s
step 14177/19560 | loss 3.328450 (-0.01z)| norm 0.2697 (-0.87z)| lr 1.13e-04 | 322.86 ms | 52.3% bf16 MFU | 1623948 tok/s
step 14178/19560 | loss 3.410426 (+2.10z)| norm 0.3448 (+3.36z)| lr 1.13e-04 | 322.93 ms | 52.3% bf16 MFU | 1623928 tok/s
step 14179/19560 | loss 3.349337 (+0.50z)| norm 0.2860 (+0.04z)| lr 1.13e-04 | 322.83 ms | 52.3% bf16 MFU | 1623934 tok/s
step 14180/19560 | loss 3.330376 (+0.00z)| norm 0.3199 (+1.90z)| lr 1.13e-04 | 322.52 ms | 52.3% bf16 MFU | 1624018 tok/s
step 14181/19560 | loss 3.442619 (+2.84z)| norm 0.2929 (+0.40z)| lr 1.13e-04 | 322.66 ms | 52.3% bf16 MFU | 1624062 tok/s
step 14182/19560 | loss 3.316651 (-0.35z)| norm 0.2845 (-0.07z)| lr 1.13e-04 | 323.05 ms | 52.2% bf16 MFU | 1624004 tok/s
step 14183/19560 | loss 3.290931 (-1.01z)| norm 0.2629 (-1.26z)| lr 1.13e-04 | 322.92 ms | 52.3% bf16 MFU | 1623984 tok/s
step 14184/19560 | loss 3.344337 (+0.35z)| norm 0.2782 (-0.41z)| lr 1.13e-04 | 322.68 ms | 52.3% bf16 MFU | 1624024 tok/s
step 14185/19560 | loss 3.289546 (-1.13z)| norm 0.2771 (-0.47z)| lr 1.12e-04 | 322.53 ms | 52.3% bf16 MFU | 1624101 tok/s
step 14186/19560 | loss 3.352615 (+0.57z)| norm 0.2964 (+0.61z)| lr 1.12e-04 | 323.27 ms | 52.2% bf16 MFU | 1623986 tok/s
step 14187/19560 | loss 3.326744 (-0.12z)| norm 0.2953 (+0.53z)| lr 1.12e-04 | 323.33 ms | 52.2% bf16 MFU | 1623864 tok/s
step 14188/19560 | loss 3.358019 (+0.71z)| norm 0.2925 (+0.37z)| lr 1.12e-04 | 323.37 ms | 52.2% bf16 MFU | 1623738 tok/s
step 14189/19560 | loss 3.350279 (+0.50z)| norm 0.2801 (-0.33z)| lr 1.12e-04 | 322.71 ms | 52.3% bf16 MFU | 1623783 tok/s
step 14190/19560 | loss 3.312021 (-0.54z)| norm 0.2900 (+0.27z)| lr 1.12e-04 | 322.95 ms | 52.3% bf16 MFU | 1623765 tok/s
step 14191/19560 | loss 3.293386 (-1.04z)| norm 0.2934 (+0.46z)| lr 1.12e-04 | 323.01 ms | 52.3% bf16 MFU | 1623735 tok/s
step 14192/19560 | loss 3.281198 (-1.35z)| norm 0.2894 (+0.23z)| lr 1.12e-04 | 322.85 ms | 52.3% bf16 MFU | 1623745 tok/s
step 14193/19560 | loss 3.336324 (+0.13z)| norm 0.2876 (+0.13z)| lr 1.12e-04 | 322.62 ms | 52.3% bf16 MFU | 1623814 tok/s
step 14194/19560 | loss 3.323690 (-0.20z)| norm 0.2919 (+0.39z)| lr 1.12e-04 | 322.41 ms | 52.3% bf16 MFU | 1623932 tok/s
step 14195/19560 | loss 3.381007 (+1.36z)| norm 0.2876 (+0.14z)| lr 1.12e-04 | 323.09 ms | 52.2% bf16 MFU | 1623872 tok/s
step 14196/19560 | loss 3.308372 (-0.62z)| norm 0.2983 (+0.76z)| lr 1.12e-04 | 322.62 ms | 52.3% bf16 MFU | 1623933 tok/s
step 14197/19560 | loss 3.296196 (-0.95z)| norm 0.2875 (+0.13z)| lr 1.12e-04 | 323.34 ms | 52.2% bf16 MFU | 1623812 tok/s
step 14198/19560 | loss 3.405409 (+1.99z)| norm 0.2726 (-0.77z)| lr 1.12e-04 | 323.50 ms | 52.2% bf16 MFU | 1623656 tok/s
step 14199/19560 | loss 3.342231 (+0.27z)| norm 0.2833 (-0.12z)| lr 1.12e-04 | 322.87 ms | 52.3% bf16 MFU | 1623664 tok/s
step 14200/19560 | loss 3.301174 (-0.83z)| norm 0.2803 (-0.30z)| lr 1.12e-04 | 322.44 ms | 52.3% bf16 MFU | 1623781 tok/s
step 14201/19560 | loss 3.256469 (-2.02z)| norm 0.2979 (+0.75z)| lr 1.12e-04 | 322.62 ms | 52.3% bf16 MFU | 1623847 tok/s
step 14202/19560 | loss 3.360441 (+0.76z)| norm 0.2770 (-0.50z)| lr 1.12e-04 | 322.71 ms | 52.3% bf16 MFU | 1623886 tok/s
step 14203/19560 | loss 3.444383 (+2.89z)| norm 0.2869 (+0.09z)| lr 1.12e-04 | 322.84 ms | 52.3% bf16 MFU | 1623892 tok/s
step 14204/19560 | loss 3.305210 (-0.72z)| norm 0.2677 (-1.06z)| lr 1.12e-04 | 323.28 ms | 52.2% bf16 MFU | 1623785 tok/s
step 14205/19560 | loss 3.310685 (-0.57z)| norm 0.2864 (+0.07z)| lr 1.12e-04 | 323.08 ms | 52.2% bf16 MFU | 1623735 tok/s
step 14206/19560 | loss 3.320024 (-0.32z)| norm 0.2826 (-0.15z)| lr 1.12e-04 | 322.25 ms | 52.4% bf16 MFU | 1623896 tok/s
step 14207/19560 | loss 3.382220 (+1.28z)| norm 0.2995 (+0.85z)| lr 1.12e-04 | 322.12 ms | 52.4% bf16 MFU | 1624081 tok/s
step 14208/19560 | loss 3.322162 (-0.27z)| norm 0.2836 (-0.11z)| lr 1.12e-04 | 323.22 ms | 52.2% bf16 MFU | 1623982 tok/s
step 14209/19560 | loss 3.303398 (-0.74z)| norm 0.2703 (-0.89z)| lr 1.12e-04 | 322.63 ms | 52.3% bf16 MFU | 1624034 tok/s
step 14210/19560 | loss 3.332067 (-0.01z)| norm 0.2808 (-0.25z)| lr 1.11e-04 | 322.22 ms | 52.4% bf16 MFU | 1624186 tok/s
step 14211/19560 | loss 3.258658 (-1.89z)| norm 0.2535 (-1.88z)| lr 1.11e-04 | 322.70 ms | 52.3% bf16 MFU | 1624210 tok/s
step 14212/19560 | loss 3.356395 (+0.61z)| norm 0.2726 (-0.73z)| lr 1.11e-04 | 322.94 ms | 52.3% bf16 MFU | 1624173 tok/s
step 14213/19560 | loss 3.344914 (+0.31z)| norm 0.2778 (-0.41z)| lr 1.11e-04 | 322.97 ms | 52.3% bf16 MFU | 1624132 tok/s
step 14214/19560 | loss 3.293135 (-1.04z)| norm 0.2797 (-0.30z)| lr 1.11e-04 | 322.33 ms | 52.4% bf16 MFU | 1624252 tok/s
step 14215/19560 | loss 3.286480 (-1.21z)| norm 0.2576 (-1.59z)| lr 1.11e-04 | 322.39 ms | 52.3% bf16 MFU | 1624351 tok/s
step 14216/19560 | loss 3.343443 (+0.26z)| norm 0.2933 (+0.52z)| lr 1.11e-04 | 322.54 ms | 52.3% bf16 MFU | 1624409 tok/s
step 14217/19560 | loss 3.329106 (-0.11z)| norm 0.2767 (-0.46z)| lr 1.11e-04 | 322.52 ms | 52.3% bf16 MFU | 1624470 tok/s
step 14218/19560 | loss 3.351773 (+0.49z)| norm 0.2827 (-0.11z)| lr 1.11e-04 | 322.91 ms | 52.3% bf16 MFU | 1624428 tok/s
step 14219/19560 | loss 3.277739 (-1.45z)| norm 0.2711 (-0.78z)| lr 1.11e-04 | 322.64 ms | 52.3% bf16 MFU | 1624455 tok/s
step 14220/19560 | loss 3.364832 (+0.84z)| norm 0.2947 (+0.60z)| lr 1.11e-04 | 322.21 ms | 52.4% bf16 MFU | 1624592 tok/s
step 14221/19560 | loss 3.342345 (+0.25z)| norm 0.3014 (+1.00z)| lr 1.11e-04 | 323.11 ms | 52.2% bf16 MFU | 1624494 tok/s
step 14222/19560 | loss 3.356436 (+0.63z)| norm 0.3234 (+2.24z)| lr 1.11e-04 | 322.96 ms | 52.3% bf16 MFU | 1624439 tok/s
step 14223/19560 | loss 3.346461 (+0.36z)| norm 0.2931 (+0.48z)| lr 1.11e-04 | 322.78 ms | 52.3% bf16 MFU | 1624430 tok/s
step 14224/19560 | loss 3.353383 (+0.54z)| norm 0.2805 (-0.26z)| lr 1.11e-04 | 322.78 ms | 52.3% bf16 MFU | 1624423 tok/s
step 14225/19560 | loss 3.301930 (-0.82z)| norm 0.2882 (+0.19z)| lr 1.11e-04 | 322.48 ms | 52.3% bf16 MFU | 1624492 tok/s
step 14226/19560 | loss 3.280506 (-1.36z)| norm 0.2762 (-0.51z)| lr 1.11e-04 | 322.65 ms | 52.3% bf16 MFU | 1624514 tok/s
step 14227/19560 | loss 3.357666 (+0.65z)| norm 0.3047 (+1.14z)| lr 1.11e-04 | 322.82 ms | 52.3% bf16 MFU | 1624492 tok/s
step 14228/19560 | loss 3.386168 (+1.37z)| norm 0.2708 (-0.83z)| lr 1.11e-04 | 322.66 ms | 52.3% bf16 MFU | 1624512 tok/s
step 14229/19560 | loss 3.393580 (+1.54z)| norm 0.2976 (+0.72z)| lr 1.11e-04 | 322.86 ms | 52.3% bf16 MFU | 1624482 tok/s
step 14230/19560 | loss 3.309910 (-0.63z)| norm 0.2938 (+0.50z)| lr 1.11e-04 | 322.74 ms | 52.3% bf16 MFU | 1624482 tok/s
step 14231/19560 | loss 3.319287 (-0.38z)| norm 0.2757 (-0.55z)| lr 1.11e-04 | 322.78 ms | 52.3% bf16 MFU | 1624473 tok/s
step 14232/19560 | loss 3.355098 (+0.54z)| norm 0.2846 (-0.03z)| lr 1.11e-04 | 322.55 ms | 52.3% bf16 MFU | 1624522 tok/s
step 14233/19560 | loss 3.255206 (-2.01z)| norm 0.2736 (-0.66z)| lr 1.11e-04 | 322.76 ms | 52.3% bf16 MFU | 1624516 tok/s
step 14234/19560 | loss 3.308106 (-0.65z)| norm 0.2719 (-0.76z)| lr 1.11e-04 | 322.51 ms | 52.3% bf16 MFU | 1624573 tok/s
step 14235/19560 | loss 3.240275 (-2.31z)| norm 0.2919 (+0.41z)| lr 1.11e-04 | 323.02 ms | 52.2% bf16 MFU | 1624499 tok/s
step 14236/19560 | loss 3.361682 (+0.73z)| norm 0.2706 (-0.83z)| lr 1.10e-04 | 323.31 ms | 52.2% bf16 MFU | 1624356 tok/s
step 14237/19560 | loss 3.271196 (-1.51z)| norm 0.2990 (+0.83z)| lr 1.10e-04 | 323.08 ms | 52.2% bf16 MFU | 1624278 tok/s
step 14238/19560 | loss 3.386790 (+1.34z)| norm 0.2871 (+0.12z)| lr 1.10e-04 | 322.25 ms | 52.4% bf16 MFU | 1624411 tok/s
step 14239/19560 | loss 3.305180 (-0.68z)| norm 0.3057 (+1.19z)| lr 1.10e-04 | 322.80 ms | 52.3% bf16 MFU | 1624399 tok/s
step 14240/19560 | loss 3.356334 (+0.58z)| norm 0.2750 (-0.60z)| lr 1.10e-04 | 322.82 ms | 52.3% bf16 MFU | 1624384 tok/s
step 14241/19560 | loss 3.388227 (+1.36z)| norm 0.3170 (+1.82z)| lr 1.10e-04 | 323.15 ms | 52.2% bf16 MFU | 1624287 tok/s
step 14242/19560 | loss 3.294613 (-0.93z)| norm 0.2901 (+0.25z)| lr 1.10e-04 | 322.63 ms | 52.3% bf16 MFU | 1624326 tok/s
step 14243/19560 | loss 3.344197 (+0.28z)| norm 0.3068 (+1.20z)| lr 1.10e-04 | 323.44 ms | 52.2% bf16 MFU | 1624159 tok/s
step 14244/19560 | loss 3.324678 (-0.20z)| norm 0.2831 (-0.20z)| lr 1.10e-04 | 323.11 ms | 52.2% bf16 MFU | 1624083 tok/s
step 14245/19560 | loss 3.372317 (+0.96z)| norm 0.3154 (+1.70z)| lr 1.10e-04 | 322.61 ms | 52.3% bf16 MFU | 1624136 tok/s
step 14246/19560 | loss 3.352541 (+0.46z)| norm 0.2887 (+0.12z)| lr 1.10e-04 | 322.61 ms | 52.3% bf16 MFU | 1624186 tok/s
step 14247/19560 | loss 3.285762 (-1.22z)| norm 0.2810 (-0.32z)| lr 1.10e-04 | 323.13 ms | 52.2% bf16 MFU | 1624103 tok/s
step 14248/19560 | loss 3.320269 (-0.35z)| norm 0.2871 (+0.03z)| lr 1.10e-04 | 322.61 ms | 52.3% bf16 MFU | 1624154 tok/s
step 14249/19560 | loss 3.404170 (+1.73z)| norm 0.3075 (+1.26z)| lr 1.10e-04 | 322.72 ms | 52.3% bf16 MFU | 1624177 tok/s
step 14250/19560 | loss 3.339527 (+0.12z)| norm 0.2842 (-0.14z)| lr 1.10e-04 | 322.70 ms | 52.3% bf16 MFU | 1624204 tok/s
val loss 3.314049
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3001/10042 = 0.298845
step 14251/19560 | loss 3.360406 (+0.65z)| norm 0.2800 (-0.42z)| lr 1.10e-04 | 322.19 ms | 52.4% bf16 MFU | 1624356 tok/s
step 14252/19560 | loss 3.287116 (-1.17z)| norm 0.2644 (-1.42z)| lr 1.10e-04 | 322.96 ms | 52.3% bf16 MFU | 1624307 tok/s
step 14253/19560 | loss 3.316465 (-0.44z)| norm 0.3114 (+1.59z)| lr 1.10e-04 | 324.24 ms | 52.1% bf16 MFU | 1623941 tok/s
step 14254/19560 | loss 3.340893 (+0.16z)| norm 0.2833 (-0.21z)| lr 1.10e-04 | 323.24 ms | 52.2% bf16 MFU | 1623843 tok/s
step 14255/19560 | loss 3.307702 (-0.67z)| norm 0.2750 (-0.73z)| lr 1.10e-04 | 323.50 ms | 52.2% bf16 MFU | 1623684 tok/s
step 14256/19560 | loss 3.320925 (-0.34z)| norm 0.3499 (+3.78z)| lr 1.10e-04 | 324.06 ms | 52.1% bf16 MFU | 1623393 tok/s
step 14257/19560 | loss 3.312570 (-0.55z)| norm 0.2878 (+0.06z)| lr 1.10e-04 | 323.03 ms | 52.2% bf16 MFU | 1623374 tok/s
step 14258/19560 | loss 3.290749 (-1.09z)| norm 0.3034 (+0.98z)| lr 1.10e-04 | 323.10 ms | 52.2% bf16 MFU | 1623338 tok/s
step 14259/19560 | loss 3.286013 (-1.19z)| norm 0.3210 (+2.02z)| lr 1.10e-04 | 323.35 ms | 52.2% bf16 MFU | 1623242 tok/s
step 14260/19560 | loss 3.392453 (+1.45z)| norm 0.3587 (+3.97z)| lr 1.10e-04 | 323.97 ms | 52.1% bf16 MFU | 1622996 tok/s
step 14261/19560 | loss 3.327363 (-0.16z)| norm 0.3015 (+0.76z)| lr 1.10e-04 | 323.00 ms | 52.3% bf16 MFU | 1623005 tok/s
step 14262/19560 | loss 3.337686 (+0.09z)| norm 0.3066 (+1.04z)| lr 1.09e-04 | 323.34 ms | 52.2% bf16 MFU | 1622930 tok/s
step 14263/19560 | loss 3.314694 (-0.48z)| norm 0.3043 (+0.90z)| lr 1.09e-04 | 322.92 ms | 52.3% bf16 MFU | 1622961 tok/s
step 14264/19560 | loss 3.286447 (-1.17z)| norm 0.2958 (+0.42z)| lr 1.09e-04 | 322.83 ms | 52.3% bf16 MFU | 1623014 tok/s
step 14265/19560 | loss 3.316322 (-0.43z)| norm 0.2918 (+0.19z)| lr 1.09e-04 | 322.96 ms | 52.3% bf16 MFU | 1623034 tok/s
step 14266/19560 | loss 3.290993 (-1.05z)| norm 0.3148 (+1.47z)| lr 1.09e-04 | 322.95 ms | 52.3% bf16 MFU | 1623054 tok/s
step 14267/19560 | loss 3.358653 (+0.63z)| norm 0.2730 (-0.85z)| lr 1.09e-04 | 322.85 ms | 52.3% bf16 MFU | 1623098 tok/s
step 14268/19560 | loss 3.280444 (-1.30z)| norm 0.2987 (+0.57z)| lr 1.09e-04 | 322.69 ms | 52.3% bf16 MFU | 1623181 tok/s
step 14269/19560 | loss 3.300551 (-0.79z)| norm 0.2901 (+0.09z)| lr 1.09e-04 | 323.73 ms | 52.1% bf16 MFU | 1622999 tok/s
step 14270/19560 | loss 3.357022 (+0.60z)| norm 0.3058 (+0.96z)| lr 1.09e-04 | 323.17 ms | 52.2% bf16 MFU | 1622965 tok/s
step 14271/19560 | loss 3.325712 (-0.17z)| norm 0.2845 (-0.22z)| lr 1.09e-04 | 322.67 ms | 52.3% bf16 MFU | 1623058 tok/s
step 14272/19560 | loss 3.300676 (-0.78z)| norm 0.3230 (+1.88z)| lr 1.09e-04 | 322.82 ms | 52.3% bf16 MFU | 1623110 tok/s
step 14273/19560 | loss 3.284376 (-1.17z)| norm 0.2724 (-0.89z)| lr 1.09e-04 | 323.06 ms | 52.2% bf16 MFU | 1623098 tok/s
step 14274/19560 | loss 3.307286 (-0.60z)| norm 0.2917 (+0.18z)| lr 1.09e-04 | 322.96 ms | 52.3% bf16 MFU | 1623111 tok/s
step 14275/19560 | loss 3.335915 (+0.10z)| norm 0.3066 (+1.04z)| lr 1.09e-04 | 323.13 ms | 52.2% bf16 MFU | 1623083 tok/s
step 14276/19560 | loss 3.261876 (-1.70z)| norm 0.2690 (-1.07z)| lr 1.09e-04 | 323.33 ms | 52.2% bf16 MFU | 1623006 tok/s
step 14277/19560 | loss 3.286573 (-1.08z)| norm 0.2960 (+0.45z)| lr 1.09e-04 | 323.10 ms | 52.2% bf16 MFU | 1622990 tok/s
step 14278/19560 | loss 3.376809 (+1.12z)| norm 0.2709 (-0.96z)| lr 1.09e-04 | 323.22 ms | 52.2% bf16 MFU | 1622946 tok/s
step 14279/19560 | loss 3.321535 (-0.22z)| norm 0.2794 (-0.47z)| lr 1.09e-04 | 323.78 ms | 52.1% bf16 MFU | 1622762 tok/s
step 14280/19560 | loss 3.224762 (-2.50z)| norm 0.2860 (-0.09z)| lr 1.09e-04 | 322.53 ms | 52.3% bf16 MFU | 1622900 tok/s
step 14281/19560 | loss 3.380488 (+1.20z)| norm 0.2657 (-1.23z)| lr 1.09e-04 | 323.57 ms | 52.2% bf16 MFU | 1622771 tok/s
step 14282/19560 | loss 3.302263 (-0.65z)| norm 0.2693 (-1.02z)| lr 1.09e-04 | 323.18 ms | 52.2% bf16 MFU | 1622747 tok/s
step 14283/19560 | loss 3.314754 (-0.34z)| norm 0.2863 (-0.06z)| lr 1.09e-04 | 323.24 ms | 52.2% bf16 MFU | 1622708 tok/s
step 14284/19560 | loss 3.350181 (+0.52z)| norm 0.2725 (-0.83z)| lr 1.09e-04 | 323.20 ms | 52.2% bf16 MFU | 1622682 tok/s
step 14285/19560 | loss 3.273557 (-1.32z)| norm 0.2811 (-0.34z)| lr 1.09e-04 | 323.12 ms | 52.2% bf16 MFU | 1622678 tok/s
step 14286/19560 | loss 3.358582 (+0.72z)| norm 0.2730 (-0.80z)| lr 1.09e-04 | 323.84 ms | 52.1% bf16 MFU | 1622493 tok/s
step 14287/19560 | loss 3.368066 (+0.94z)| norm 0.2790 (-0.45z)| lr 1.09e-04 | 323.00 ms | 52.3% bf16 MFU | 1622527 tok/s
step 14288/19560 | loss 3.355075 (+0.62z)| norm 0.3037 (+0.94z)| lr 1.08e-04 | 323.22 ms | 52.2% bf16 MFU | 1622505 tok/s
step 14289/19560 | loss 3.277064 (-1.25z)| norm 0.2784 (-0.50z)| lr 1.08e-04 | 324.63 ms | 52.0% bf16 MFU | 1622130 tok/s
step 14290/19560 | loss 3.352287 (+0.55z)| norm 0.2986 (+0.64z)| lr 1.08e-04 | 323.09 ms | 52.2% bf16 MFU | 1622161 tok/s
step 14291/19560 | loss 3.261124 (-1.61z)| norm 0.2750 (-0.72z)| lr 1.08e-04 | 322.96 ms | 52.3% bf16 MFU | 1622221 tok/s
step 14292/19560 | loss 3.347900 (+0.47z)| norm 0.2815 (-0.35z)| lr 1.08e-04 | 323.36 ms | 52.2% bf16 MFU | 1622178 tok/s
step 14293/19560 | loss 3.314220 (-0.33z)| norm 0.2938 (+0.35z)| lr 1.08e-04 | 322.99 ms | 52.3% bf16 MFU | 1622229 tok/s
step 14294/19560 | loss 3.267564 (-1.43z)| norm 0.3009 (+0.75z)| lr 1.08e-04 | 322.73 ms | 52.3% bf16 MFU | 1622346 tok/s
step 14295/19560 | loss 3.327742 (+0.02z)| norm 0.2809 (-0.40z)| lr 1.08e-04 | 322.66 ms | 52.3% bf16 MFU | 1622474 tok/s
step 14296/19560 | loss 3.247298 (-1.92z)| norm 0.2898 (+0.10z)| lr 1.08e-04 | 322.66 ms | 52.3% bf16 MFU | 1622594 tok/s
step 14297/19560 | loss 3.367953 (+1.04z)| norm 0.2967 (+0.50z)| lr 1.08e-04 | 322.94 ms | 52.3% bf16 MFU | 1622638 tok/s
step 14298/19560 | loss 3.374215 (+1.18z)| norm 0.2828 (-0.33z)| lr 1.08e-04 | 322.55 ms | 52.3% bf16 MFU | 1622778 tok/s
step 14299/19560 | loss 3.287570 (-0.93z)| norm 0.2930 (+0.27z)| lr 1.08e-04 | 323.22 ms | 52.2% bf16 MFU | 1622743 tok/s
step 14300/19560 | loss 3.392877 (+1.61z)| norm 0.2780 (-0.63z)| lr 1.08e-04 | 322.99 ms | 52.3% bf16 MFU | 1622767 tok/s
step 14301/19560 | loss 3.321683 (-0.13z)| norm 0.3131 (+1.43z)| lr 1.08e-04 | 323.47 ms | 52.2% bf16 MFU | 1622670 tok/s
step 14302/19560 | loss 3.288426 (-0.93z)| norm 0.2949 (+0.34z)| lr 1.08e-04 | 323.52 ms | 52.2% bf16 MFU | 1622565 tok/s
step 14303/19560 | loss 3.345513 (+0.46z)| norm 0.2796 (-0.56z)| lr 1.08e-04 | 322.94 ms | 52.3% bf16 MFU | 1622610 tok/s
step 14304/19560 | loss 3.322509 (-0.11z)| norm 0.3092 (+1.18z)| lr 1.08e-04 | 323.25 ms | 52.2% bf16 MFU | 1622576 tok/s
step 14305/19560 | loss 3.294992 (-0.77z)| norm 0.3235 (+1.98z)| lr 1.08e-04 | 323.15 ms | 52.2% bf16 MFU | 1622570 tok/s
step 14306/19560 | loss 3.348263 (+0.55z)| norm 0.2846 (-0.28z)| lr 1.08e-04 | 323.14 ms | 52.2% bf16 MFU | 1622566 tok/s
step 14307/19560 | loss 3.309325 (-0.41z)| norm 0.3003 (+0.67z)| lr 1.08e-04 | 322.55 ms | 52.3% bf16 MFU | 1622710 tok/s
step 14308/19560 | loss 3.327873 (+0.05z)| norm 0.3102 (+1.28z)| lr 1.08e-04 | 323.79 ms | 52.1% bf16 MFU | 1622535 tok/s
step 14309/19560 | loss 3.302672 (-0.57z)| norm 0.2883 (-0.06z)| lr 1.08e-04 | 322.99 ms | 52.3% bf16 MFU | 1622571 tok/s
step 14310/19560 | loss 3.304492 (-0.52z)| norm 0.3210 (+1.90z)| lr 1.08e-04 | 322.63 ms | 52.3% bf16 MFU | 1622696 tok/s
step 14311/19560 | loss 3.306229 (-0.48z)| norm 0.2954 (+0.34z)| lr 1.08e-04 | 322.82 ms | 52.3% bf16 MFU | 1622765 tok/s
step 14312/19560 | loss 3.320224 (-0.11z)| norm 0.3024 (+0.76z)| lr 1.08e-04 | 322.68 ms | 52.3% bf16 MFU | 1622866 tok/s
step 14313/19560 | loss 3.268016 (-1.45z)| norm 0.3121 (+1.33z)| lr 1.08e-04 | 323.41 ms | 52.2% bf16 MFU | 1622779 tok/s
step 14314/19560 | loss 3.356726 (+0.83z)| norm 0.3094 (+1.15z)| lr 1.07e-04 | 322.90 ms | 52.3% bf16 MFU | 1622824 tok/s
step 14315/19560 | loss 3.299897 (-0.62z)| norm 0.2965 (+0.37z)| lr 1.07e-04 | 323.00 ms | 52.3% bf16 MFU | 1622842 tok/s
step 14316/19560 | loss 3.316656 (-0.18z)| norm 0.2827 (-0.46z)| lr 1.07e-04 | 324.91 ms | 51.9% bf16 MFU | 1622382 tok/s
step 14317/19560 | loss 3.274950 (-1.24z)| norm 0.3096 (+1.15z)| lr 1.07e-04 | 322.62 ms | 52.3% bf16 MFU | 1622517 tok/s
step 14318/19560 | loss 3.345652 (+0.56z)| norm 0.2785 (-0.72z)| lr 1.07e-04 | 322.71 ms | 52.3% bf16 MFU | 1622624 tok/s
step 14319/19560 | loss 3.305883 (-0.45z)| norm 0.2922 (+0.10z)| lr 1.07e-04 | 322.52 ms | 52.3% bf16 MFU | 1622774 tok/s
step 14320/19560 | loss 3.296391 (-0.70z)| norm 0.2683 (-1.31z)| lr 1.07e-04 | 322.19 ms | 52.4% bf16 MFU | 1622998 tok/s
step 14321/19560 | loss 3.343911 (+0.52z)| norm 0.2725 (-1.05z)| lr 1.07e-04 | 322.95 ms | 52.3% bf16 MFU | 1623020 tok/s
step 14322/19560 | loss 3.355703 (+0.81z)| norm 0.2794 (-0.63z)| lr 1.07e-04 | 322.78 ms | 52.3% bf16 MFU | 1623084 tok/s
step 14323/19560 | loss 3.358456 (+0.89z)| norm 0.2749 (-0.89z)| lr 1.07e-04 | 322.44 ms | 52.3% bf16 MFU | 1623230 tok/s
step 14324/19560 | loss 3.316635 (-0.19z)| norm 0.2889 (-0.06z)| lr 1.07e-04 | 322.77 ms | 52.3% bf16 MFU | 1623286 tok/s
step 14325/19560 | loss 3.317874 (-0.16z)| norm 0.2632 (-1.56z)| lr 1.07e-04 | 322.94 ms | 52.3% bf16 MFU | 1623296 tok/s
step 14326/19560 | loss 3.331740 (+0.21z)| norm 0.2925 (+0.16z)| lr 1.07e-04 | 323.00 ms | 52.3% bf16 MFU | 1623290 tok/s
step 14327/19560 | loss 3.388166 (+1.67z)| norm 0.2745 (-0.90z)| lr 1.07e-04 | 322.84 ms | 52.3% bf16 MFU | 1623325 tok/s
step 14328/19560 | loss 3.266845 (-1.46z)| norm 0.3071 (+1.00z)| lr 1.07e-04 | 323.30 ms | 52.2% bf16 MFU | 1623243 tok/s
step 14329/19560 | loss 3.304924 (-0.50z)| norm 0.2883 (-0.10z)| lr 1.07e-04 | 323.37 ms | 52.2% bf16 MFU | 1623147 tok/s
step 14330/19560 | loss 3.340621 (+0.44z)| norm 0.2774 (-0.74z)| lr 1.07e-04 | 322.55 ms | 52.3% bf16 MFU | 1623263 tok/s
step 14331/19560 | loss 3.344281 (+0.58z)| norm 0.2818 (-0.47z)| lr 1.07e-04 | 322.77 ms | 52.3% bf16 MFU | 1623316 tok/s
step 14332/19560 | loss 3.356808 (+0.90z)| norm 0.2760 (-0.82z)| lr 1.07e-04 | 322.91 ms | 52.3% bf16 MFU | 1623331 tok/s
step 14333/19560 | loss 3.293360 (-0.81z)| norm 0.2544 (-2.05z)| lr 1.07e-04 | 323.02 ms | 52.2% bf16 MFU | 1623319 tok/s
step 14334/19560 | loss 3.370165 (+1.25z)| norm 0.2775 (-0.70z)| lr 1.07e-04 | 322.80 ms | 52.3% bf16 MFU | 1623364 tok/s
step 14335/19560 | loss 3.351778 (+0.77z)| norm 0.2848 (-0.28z)| lr 1.07e-04 | 322.74 ms | 52.3% bf16 MFU | 1623419 tok/s
step 14336/19560 | loss 3.430868 (+2.80z)| norm 0.2967 (+0.41z)| lr 1.07e-04 | 323.01 ms | 52.2% bf16 MFU | 1623405 tok/s
step 14337/19560 | loss 3.334655 (+0.26z)| norm 0.2843 (-0.32z)| lr 1.07e-04 | 322.63 ms | 52.3% bf16 MFU | 1623487 tok/s
step 14338/19560 | loss 3.362367 (+0.98z)| norm 0.2906 (+0.05z)| lr 1.07e-04 | 322.90 ms | 52.3% bf16 MFU | 1623496 tok/s
step 14339/19560 | loss 3.335188 (+0.26z)| norm 0.2918 (+0.10z)| lr 1.07e-04 | 322.58 ms | 52.3% bf16 MFU | 1623586 tok/s
step 14340/19560 | loss 3.320690 (-0.12z)| norm 0.2913 (+0.06z)| lr 1.06e-04 | 322.81 ms | 52.3% bf16 MFU | 1623613 tok/s
step 14341/19560 | loss 3.235624 (-2.31z)| norm 0.2902 (-0.01z)| lr 1.06e-04 | 322.89 ms | 52.3% bf16 MFU | 1623620 tok/s
step 14342/19560 | loss 3.264931 (-1.53z)| norm 0.2671 (-1.37z)| lr 1.06e-04 | 322.84 ms | 52.3% bf16 MFU | 1623639 tok/s
step 14343/19560 | loss 3.287728 (-0.94z)| norm 0.2873 (-0.19z)| lr 1.06e-04 | 322.73 ms | 52.3% bf16 MFU | 1623684 tok/s
step 14344/19560 | loss 3.351882 (+0.72z)| norm 0.2674 (-1.37z)| lr 1.06e-04 | 322.49 ms | 52.3% bf16 MFU | 1623788 tok/s
step 14345/19560 | loss 3.278441 (-1.16z)| norm 0.2625 (-1.64z)| lr 1.06e-04 | 322.78 ms | 52.3% bf16 MFU | 1623814 tok/s
step 14346/19560 | loss 3.266927 (-1.43z)| norm 0.2696 (-1.21z)| lr 1.06e-04 | 322.74 ms | 52.3% bf16 MFU | 1623846 tok/s
step 14347/19560 | loss 3.308524 (-0.38z)| norm 0.2765 (-0.81z)| lr 1.06e-04 | 322.80 ms | 52.3% bf16 MFU | 1623863 tok/s
step 14348/19560 | loss 3.319776 (-0.08z)| norm 0.2892 (-0.05z)| lr 1.06e-04 | 323.37 ms | 52.2% bf16 MFU | 1623735 tok/s
step 14349/19560 | loss 3.284618 (-0.97z)| norm 0.2761 (-0.81z)| lr 1.06e-04 | 322.47 ms | 52.3% bf16 MFU | 1623840 tok/s
step 14350/19560 | loss 3.289169 (-0.84z)| norm 0.2738 (-0.94z)| lr 1.06e-04 | 322.66 ms | 52.3% bf16 MFU | 1623894 tok/s
step 14351/19560 | loss 3.316802 (-0.13z)| norm 0.2973 (+0.46z)| lr 1.06e-04 | 322.96 ms | 52.3% bf16 MFU | 1623868 tok/s
step 14352/19560 | loss 3.344528 (+0.59z)| norm 0.2757 (-0.83z)| lr 1.06e-04 | 323.13 ms | 52.2% bf16 MFU | 1623800 tok/s
step 14353/19560 | loss 3.274940 (-1.19z)| norm 0.2733 (-0.96z)| lr 1.06e-04 | 323.27 ms | 52.2% bf16 MFU | 1623702 tok/s
step 14354/19560 | loss 3.318653 (-0.08z)| norm 0.2689 (-1.21z)| lr 1.06e-04 | 322.26 ms | 52.4% bf16 MFU | 1623862 tok/s
step 14355/19560 | loss 3.371246 (+1.27z)| norm 0.2638 (-1.48z)| lr 1.06e-04 | 322.57 ms | 52.3% bf16 MFU | 1623936 tok/s
step 14356/19560 | loss 3.353524 (+0.83z)| norm 0.2872 (-0.11z)| lr 1.06e-04 | 322.93 ms | 52.3% bf16 MFU | 1623916 tok/s
step 14357/19560 | loss 3.262573 (-1.51z)| norm 0.2502 (-2.24z)| lr 1.06e-04 | 323.22 ms | 52.2% bf16 MFU | 1623824 tok/s
step 14358/19560 | loss 3.292198 (-0.73z)| norm 0.2962 (+0.43z)| lr 1.06e-04 | 322.76 ms | 52.3% bf16 MFU | 1623852 tok/s
step 14359/19560 | loss 3.293933 (-0.68z)| norm 0.2625 (-1.51z)| lr 1.06e-04 | 322.42 ms | 52.3% bf16 MFU | 1623964 tok/s
step 14360/19560 | loss 3.330583 (+0.27z)| norm 0.3005 (+0.67z)| lr 1.06e-04 | 323.44 ms | 52.2% bf16 MFU | 1623815 tok/s
step 14361/19560 | loss 3.272102 (-1.26z)| norm 0.2843 (-0.26z)| lr 1.06e-04 | 322.81 ms | 52.3% bf16 MFU | 1623831 tok/s
step 14362/19560 | loss 3.292515 (-0.72z)| norm 0.2834 (-0.32z)| lr 1.06e-04 | 322.47 ms | 52.3% bf16 MFU | 1623931 tok/s
step 14363/19560 | loss 3.284548 (-0.95z)| norm 0.2799 (-0.52z)| lr 1.06e-04 | 322.99 ms | 52.3% bf16 MFU | 1623895 tok/s
step 14364/19560 | loss 3.356411 (+0.95z)| norm 0.2855 (-0.21z)| lr 1.06e-04 | 322.66 ms | 52.3% bf16 MFU | 1623946 tok/s
step 14365/19560 | loss 3.326279 (+0.15z)| norm 0.2979 (+0.52z)| lr 1.06e-04 | 322.93 ms | 52.3% bf16 MFU | 1623926 tok/s
step 14366/19560 | loss 3.346641 (+0.70z)| norm 0.2953 (+0.36z)| lr 1.05e-04 | 322.38 ms | 52.4% bf16 MFU | 1624045 tok/s
step 14367/19560 | loss 3.317018 (-0.10z)| norm 0.2749 (-0.81z)| lr 1.05e-04 | 322.64 ms | 52.3% bf16 MFU | 1624091 tok/s
step 14368/19560 | loss 3.349662 (+0.79z)| norm 0.2807 (-0.47z)| lr 1.05e-04 | 322.95 ms | 52.3% bf16 MFU | 1624059 tok/s
step 14369/19560 | loss 3.315687 (-0.12z)| norm 0.2849 (-0.22z)| lr 1.05e-04 | 322.72 ms | 52.3% bf16 MFU | 1624085 tok/s
step 14370/19560 | loss 3.260443 (-1.61z)| norm 0.2657 (-1.33z)| lr 1.05e-04 | 322.78 ms | 52.3% bf16 MFU | 1624096 tok/s
step 14371/19560 | loss 3.290702 (-0.78z)| norm 0.3065 (+1.06z)| lr 1.05e-04 | 322.77 ms | 52.3% bf16 MFU | 1624109 tok/s
step 14372/19560 | loss 3.359242 (+1.07z)| norm 0.2837 (-0.28z)| lr 1.05e-04 | 322.56 ms | 52.3% bf16 MFU | 1624173 tok/s
step 14373/19560 | loss 3.285109 (-0.92z)| norm 0.2867 (-0.09z)| lr 1.05e-04 | 322.88 ms | 52.3% bf16 MFU | 1624153 tok/s
step 14374/19560 | loss 3.372221 (+1.43z)| norm 0.2783 (-0.58z)| lr 1.05e-04 | 322.70 ms | 52.3% bf16 MFU | 1624181 tok/s
step 14375/19560 | loss 3.417495 (+2.57z)| norm 0.2923 (+0.24z)| lr 1.05e-04 | 322.52 ms | 52.3% bf16 MFU | 1624251 tok/s
step 14376/19560 | loss 3.267714 (-1.36z)| norm 0.3319 (+2.50z)| lr 1.05e-04 | 322.48 ms | 52.3% bf16 MFU | 1624328 tok/s
step 14377/19560 | loss 3.311211 (-0.21z)| norm 0.2680 (-1.16z)| lr 1.05e-04 | 323.15 ms | 52.2% bf16 MFU | 1624233 tok/s
step 14378/19560 | loss 3.361430 (+1.12z)| norm 0.3198 (+1.78z)| lr 1.05e-04 | 323.50 ms | 52.2% bf16 MFU | 1624056 tok/s
step 14379/19560 | loss 3.355523 (+0.97z)| norm 0.3079 (+1.09z)| lr 1.05e-04 | 322.75 ms | 52.3% bf16 MFU | 1624076 tok/s
step 14380/19560 | loss 3.333411 (+0.37z)| norm 0.2887 (-0.01z)| lr 1.05e-04 | 322.81 ms | 52.3% bf16 MFU | 1624079 tok/s
step 14381/19560 | loss 3.314545 (-0.13z)| norm 0.2963 (+0.43z)| lr 1.05e-04 | 322.57 ms | 52.3% bf16 MFU | 1624143 tok/s
step 14382/19560 | loss 3.343260 (+0.63z)| norm 0.3178 (+1.63z)| lr 1.05e-04 | 322.26 ms | 52.4% bf16 MFU | 1624281 tok/s
step 14383/19560 | loss 3.323649 (+0.11z)| norm 0.3071 (+1.01z)| lr 1.05e-04 | 322.73 ms | 52.3% bf16 MFU | 1624294 tok/s
step 14384/19560 | loss 3.320574 (+0.03z)| norm 0.2774 (-0.68z)| lr 1.05e-04 | 323.13 ms | 52.2% bf16 MFU | 1624205 tok/s
step 14385/19560 | loss 3.323982 (+0.12z)| norm 0.2988 (+0.59z)| lr 1.05e-04 | 322.88 ms | 52.3% bf16 MFU | 1624184 tok/s
step 14386/19560 | loss 3.317551 (-0.06z)| norm 0.2586 (-1.76z)| lr 1.05e-04 | 322.82 ms | 52.3% bf16 MFU | 1624179 tok/s
step 14387/19560 | loss 3.326583 (+0.17z)| norm 0.2867 (-0.09z)| lr 1.05e-04 | 323.19 ms | 52.2% bf16 MFU | 1624081 tok/s
step 14388/19560 | loss 3.311146 (-0.23z)| norm 0.2861 (-0.10z)| lr 1.05e-04 | 323.09 ms | 52.2% bf16 MFU | 1624014 tok/s
step 14389/19560 | loss 3.332850 (+0.36z)| norm 0.2784 (-0.59z)| lr 1.05e-04 | 321.95 ms | 52.4% bf16 MFU | 1624238 tok/s
step 14390/19560 | loss 3.309859 (-0.26z)| norm 0.2859 (-0.10z)| lr 1.05e-04 | 323.55 ms | 52.2% bf16 MFU | 1624047 tok/s
step 14391/19560 | loss 3.269767 (-1.33z)| norm 0.2743 (-0.83z)| lr 1.05e-04 | 323.08 ms | 52.2% bf16 MFU | 1623985 tok/s
step 14392/19560 | loss 3.323276 (+0.11z)| norm 0.2849 (-0.14z)| lr 1.05e-04 | 322.83 ms | 52.3% bf16 MFU | 1623989 tok/s
step 14393/19560 | loss 3.308395 (-0.29z)| norm 0.3038 (+1.08z)| lr 1.04e-04 | 322.72 ms | 52.3% bf16 MFU | 1624019 tok/s
step 14394/19560 | loss 3.281097 (-1.03z)| norm 0.2612 (-1.65z)| lr 1.04e-04 | 323.28 ms | 52.2% bf16 MFU | 1623907 tok/s
step 14395/19560 | loss 3.264449 (-1.45z)| norm 0.2800 (-0.44z)| lr 1.04e-04 | 322.89 ms | 52.3% bf16 MFU | 1623900 tok/s
step 14396/19560 | loss 3.373941 (+1.47z)| norm 0.2872 (+0.03z)| lr 1.04e-04 | 322.38 ms | 52.4% bf16 MFU | 1624020 tok/s
step 14397/19560 | loss 3.346158 (+0.71z)| norm 0.2882 (+0.10z)| lr 1.04e-04 | 322.53 ms | 52.3% bf16 MFU | 1624096 tok/s
step 14398/19560 | loss 3.339394 (+0.54z)| norm 0.2778 (-0.56z)| lr 1.04e-04 | 322.97 ms | 52.3% bf16 MFU | 1624058 tok/s
step 14399/19560 | loss 3.300462 (-0.50z)| norm 0.2672 (-1.24z)| lr 1.04e-04 | 322.24 ms | 52.4% bf16 MFU | 1624205 tok/s
step 14400/19560 | loss 3.356399 (+0.98z)| norm 0.2858 (-0.02z)| lr 1.04e-04 | 323.21 ms | 52.2% bf16 MFU | 1624101 tok/s
step 14401/19560 | loss 3.288258 (-0.84z)| norm 0.2757 (-0.69z)| lr 1.04e-04 | 322.48 ms | 52.3% bf16 MFU | 1624186 tok/s
step 14402/19560 | loss 3.356069 (+0.96z)| norm 0.2677 (-1.20z)| lr 1.04e-04 | 322.86 ms | 52.3% bf16 MFU | 1624171 tok/s
step 14403/19560 | loss 3.305538 (-0.38z)| norm 0.2788 (-0.45z)| lr 1.04e-04 | 323.14 ms | 52.2% bf16 MFU | 1624087 tok/s
step 14404/19560 | loss 3.337225 (+0.45z)| norm 0.2731 (-0.84z)| lr 1.04e-04 | 322.14 ms | 52.4% bf16 MFU | 1624259 tok/s
step 14405/19560 | loss 3.300649 (-0.54z)| norm 0.2834 (-0.14z)| lr 1.04e-04 | 323.29 ms | 52.2% bf16 MFU | 1624132 tok/s
step 14406/19560 | loss 3.331198 (+0.30z)| norm 0.3005 (+0.98z)| lr 1.04e-04 | 322.77 ms | 52.3% bf16 MFU | 1624142 tok/s
step 14407/19560 | loss 3.275123 (-1.21z)| norm 0.2851 (-0.05z)| lr 1.04e-04 | 322.52 ms | 52.3% bf16 MFU | 1624215 tok/s
step 14408/19560 | loss 3.325336 (+0.13z)| norm 0.2811 (-0.31z)| lr 1.04e-04 | 322.70 ms | 52.3% bf16 MFU | 1624239 tok/s
step 14409/19560 | loss 3.289037 (-0.86z)| norm 0.2718 (-0.94z)| lr 1.04e-04 | 323.19 ms | 52.2% bf16 MFU | 1624138 tok/s
step 14410/19560 | loss 3.333980 (+0.39z)| norm 0.2722 (-0.92z)| lr 1.04e-04 | 323.28 ms | 52.2% bf16 MFU | 1624019 tok/s
step 14411/19560 | loss 3.264474 (-1.53z)| norm 0.2788 (-0.47z)| lr 1.04e-04 | 322.80 ms | 52.3% bf16 MFU | 1624026 tok/s
step 14412/19560 | loss 3.307504 (-0.33z)| norm 0.2702 (-1.05z)| lr 1.04e-04 | 322.13 ms | 52.4% bf16 MFU | 1624204 tok/s
step 14413/19560 | loss 3.360250 (+1.12z)| norm 0.3041 (+1.21z)| lr 1.04e-04 | 322.89 ms | 52.3% bf16 MFU | 1624179 tok/s
step 14414/19560 | loss 3.348565 (+0.80z)| norm 0.2758 (-0.68z)| lr 1.04e-04 | 322.48 ms | 52.3% bf16 MFU | 1624260 tok/s
step 14415/19560 | loss 3.280521 (-1.09z)| norm 0.2794 (-0.44z)| lr 1.04e-04 | 322.76 ms | 52.3% bf16 MFU | 1624266 tok/s
step 14416/19560 | loss 3.339507 (+0.57z)| norm 0.2750 (-0.72z)| lr 1.04e-04 | 322.97 ms | 52.3% bf16 MFU | 1624219 tok/s
step 14417/19560 | loss 3.264318 (-1.53z)| norm 0.2805 (-0.35z)| lr 1.04e-04 | 322.03 ms | 52.4% bf16 MFU | 1624411 tok/s
step 14418/19560 | loss 3.277450 (-1.15z)| norm 0.2898 (+0.28z)| lr 1.04e-04 | 322.50 ms | 52.3% bf16 MFU | 1624475 tok/s
step 14419/19560 | loss 3.293281 (-0.72z)| norm 0.2756 (-0.68z)| lr 1.03e-04 | 322.89 ms | 52.3% bf16 MFU | 1624439 tok/s
step 14420/19560 | loss 3.399044 (+2.21z)| norm 0.3014 (+1.04z)| lr 1.03e-04 | 322.93 ms | 52.3% bf16 MFU | 1624393 tok/s
step 14421/19560 | loss 3.291202 (-0.77z)| norm 0.2619 (-1.58z)| lr 1.03e-04 | 322.37 ms | 52.4% bf16 MFU | 1624492 tok/s
step 14422/19560 | loss 3.360342 (+1.13z)| norm 0.2854 (-0.01z)| lr 1.03e-04 | 323.32 ms | 52.2% bf16 MFU | 1624346 tok/s
step 14423/19560 | loss 3.357445 (+1.04z)| norm 0.2670 (-1.22z)| lr 1.03e-04 | 323.31 ms | 52.2% bf16 MFU | 1624209 tok/s
step 14424/19560 | loss 3.353523 (+0.92z)| norm 0.2811 (-0.28z)| lr 1.03e-04 | 322.43 ms | 52.3% bf16 MFU | 1624302 tok/s
step 14425/19560 | loss 3.295098 (-0.70z)| norm 0.2720 (-0.87z)| lr 1.03e-04 | 322.72 ms | 52.3% bf16 MFU | 1624316 tok/s
step 14426/19560 | loss 3.345239 (+0.72z)| norm 0.2897 (+0.30z)| lr 1.03e-04 | 322.46 ms | 52.3% bf16 MFU | 1624395 tok/s
step 14427/19560 | loss 3.361671 (+1.16z)| norm 0.2739 (-0.74z)| lr 1.03e-04 | 323.22 ms | 52.2% bf16 MFU | 1624279 tok/s
step 14428/19560 | loss 3.303673 (-0.46z)| norm 0.2704 (-0.96z)| lr 1.03e-04 | 322.80 ms | 52.3% bf16 MFU | 1624274 tok/s
step 14429/19560 | loss 3.260574 (-1.67z)| norm 0.3068 (+1.45z)| lr 1.03e-04 | 322.32 ms | 52.4% bf16 MFU | 1624389 tok/s
step 14430/19560 | loss 3.314234 (-0.15z)| norm 0.2698 (-0.99z)| lr 1.03e-04 | 322.90 ms | 52.3% bf16 MFU | 1624354 tok/s
step 14431/19560 | loss 3.231983 (-2.42z)| norm 0.2706 (-0.93z)| lr 1.03e-04 | 322.70 ms | 52.3% bf16 MFU | 1624369 tok/s
step 14432/19560 | loss 3.342618 (+0.66z)| norm 0.2824 (-0.14z)| lr 1.03e-04 | 323.12 ms | 52.2% bf16 MFU | 1624280 tok/s
step 14433/19560 | loss 3.349674 (+0.85z)| norm 0.2886 (+0.30z)| lr 1.03e-04 | 323.31 ms | 52.2% bf16 MFU | 1624146 tok/s
step 14434/19560 | loss 3.297533 (-0.59z)| norm 0.3051 (+1.42z)| lr 1.03e-04 | 322.71 ms | 52.3% bf16 MFU | 1624170 tok/s
step 14435/19560 | loss 3.336462 (+0.48z)| norm 0.2659 (-1.24z)| lr 1.03e-04 | 323.08 ms | 52.2% bf16 MFU | 1624101 tok/s
step 14436/19560 | loss 3.380015 (+1.67z)| norm 0.2813 (-0.17z)| lr 1.03e-04 | 322.63 ms | 52.3% bf16 MFU | 1624147 tok/s
step 14437/19560 | loss 3.328799 (+0.25z)| norm 0.2884 (+0.31z)| lr 1.03e-04 | 323.58 ms | 52.2% bf16 MFU | 1623953 tok/s
step 14438/19560 | loss 3.303649 (-0.44z)| norm 0.2708 (-0.89z)| lr 1.03e-04 | 322.31 ms | 52.4% bf16 MFU | 1624089 tok/s
step 14439/19560 | loss 3.290867 (-0.79z)| norm 0.2880 (+0.33z)| lr 1.03e-04 | 323.05 ms | 52.2% bf16 MFU | 1624032 tok/s
step 14440/19560 | loss 3.329119 (+0.26z)| norm 0.3006 (+1.22z)| lr 1.03e-04 | 322.97 ms | 52.3% bf16 MFU | 1623997 tok/s
step 14441/19560 | loss 3.332431 (+0.34z)| norm 0.2915 (+0.59z)| lr 1.03e-04 | 323.02 ms | 52.2% bf16 MFU | 1623952 tok/s
step 14442/19560 | loss 3.304494 (-0.42z)| norm 0.2823 (-0.05z)| lr 1.03e-04 | 322.76 ms | 52.3% bf16 MFU | 1623974 tok/s
step 14443/19560 | loss 3.267978 (-1.42z)| norm 0.2702 (-0.92z)| lr 1.03e-04 | 323.21 ms | 52.2% bf16 MFU | 1623883 tok/s
step 14444/19560 | loss 3.289337 (-0.82z)| norm 0.2771 (-0.41z)| lr 1.03e-04 | 323.22 ms | 52.2% bf16 MFU | 1623792 tok/s
step 14445/19560 | loss 3.308694 (-0.30z)| norm 0.2518 (-2.21z)| lr 1.03e-04 | 323.06 ms | 52.2% bf16 MFU | 1623748 tok/s
step 14446/19560 | loss 3.284437 (-0.95z)| norm 0.2665 (-1.13z)| lr 1.02e-04 | 323.28 ms | 52.2% bf16 MFU | 1623651 tok/s
step 14447/19560 | loss 3.396113 (+2.07z)| norm 0.2729 (-0.66z)| lr 1.02e-04 | 322.82 ms | 52.3% bf16 MFU | 1623672 tok/s
step 14448/19560 | loss 3.344575 (+0.66z)| norm 0.2844 (+0.16z)| lr 1.02e-04 | 322.87 ms | 52.3% bf16 MFU | 1623682 tok/s
step 14449/19560 | loss 3.275890 (-1.18z)| norm 0.2770 (-0.38z)| lr 1.02e-04 | 322.95 ms | 52.3% bf16 MFU | 1623670 tok/s
step 14450/19560 | loss 3.222790 (-2.53z)| norm 0.2591 (-1.65z)| lr 1.02e-04 | 323.13 ms | 52.2% bf16 MFU | 1623613 tok/s
step 14451/19560 | loss 3.300548 (-0.47z)| norm 0.3154 (+2.32z)| lr 1.02e-04 | 322.80 ms | 52.3% bf16 MFU | 1623642 tok/s
step 14452/19560 | loss 3.410833 (+2.39z)| norm 0.2717 (-0.74z)| lr 1.02e-04 | 322.98 ms | 52.3% bf16 MFU | 1623626 tok/s
step 14453/19560 | loss 3.356586 (+0.97z)| norm 0.3192 (+2.52z)| lr 1.02e-04 | 323.32 ms | 52.2% bf16 MFU | 1623522 tok/s
step 14454/19560 | loss 3.340551 (+0.55z)| norm 0.2764 (-0.42z)| lr 1.02e-04 | 322.73 ms | 52.3% bf16 MFU | 1623574 tok/s
step 14455/19560 | loss 3.303500 (-0.39z)| norm 0.2920 (+0.64z)| lr 1.02e-04 | 323.54 ms | 52.2% bf16 MFU | 1623420 tok/s
step 14456/19560 | loss 3.356874 (+0.99z)| norm 0.2651 (-1.20z)| lr 1.02e-04 | 322.69 ms | 52.3% bf16 MFU | 1623486 tok/s
step 14457/19560 | loss 3.305791 (-0.35z)| norm 0.2749 (-0.51z)| lr 1.02e-04 | 322.74 ms | 52.3% bf16 MFU | 1623535 tok/s
step 14458/19560 | loss 3.307460 (-0.30z)| norm 0.2893 (+0.48z)| lr 1.02e-04 | 322.97 ms | 52.3% bf16 MFU | 1623524 tok/s
step 14459/19560 | loss 3.311300 (-0.20z)| norm 0.2686 (-0.94z)| lr 1.02e-04 | 322.88 ms | 52.3% bf16 MFU | 1623538 tok/s
step 14460/19560 | loss 3.298404 (-0.52z)| norm 0.2837 (+0.09z)| lr 1.02e-04 | 322.67 ms | 52.3% bf16 MFU | 1623603 tok/s
step 14461/19560 | loss 3.282277 (-0.95z)| norm 0.2810 (-0.11z)| lr 1.02e-04 | 322.88 ms | 52.3% bf16 MFU | 1623612 tok/s
step 14462/19560 | loss 3.316224 (-0.04z)| norm 0.2737 (-0.62z)| lr 1.02e-04 | 323.18 ms | 52.2% bf16 MFU | 1623546 tok/s
step 14463/19560 | loss 3.271884 (-1.20z)| norm 0.2697 (-0.89z)| lr 1.02e-04 | 323.25 ms | 52.2% bf16 MFU | 1623465 tok/s
step 14464/19560 | loss 3.286149 (-0.82z)| norm 0.2783 (-0.27z)| lr 1.02e-04 | 323.00 ms | 52.3% bf16 MFU | 1623451 tok/s
step 14465/19560 | loss 3.355691 (+1.08z)| norm 0.2663 (-1.10z)| lr 1.02e-04 | 323.12 ms | 52.2% bf16 MFU | 1623407 tok/s
step 14466/19560 | loss 3.341637 (+0.70z)| norm 0.2575 (-1.68z)| lr 1.02e-04 | 323.10 ms | 52.2% bf16 MFU | 1623372 tok/s
step 14467/19560 | loss 3.297103 (-0.51z)| norm 0.2770 (-0.32z)| lr 1.02e-04 | 323.05 ms | 52.2% bf16 MFU | 1623351 tok/s
step 14468/19560 | loss 3.308581 (-0.19z)| norm 0.2563 (-1.73z)| lr 1.02e-04 | 323.15 ms | 52.2% bf16 MFU | 1623304 tok/s
step 14469/19560 | loss 3.294961 (-0.59z)| norm 0.2647 (-1.13z)| lr 1.02e-04 | 322.98 ms | 52.3% bf16 MFU | 1623304 tok/s
step 14470/19560 | loss 3.405258 (+2.42z)| norm 0.3511 (+4.37z)| lr 1.02e-04 | 323.04 ms | 52.2% bf16 MFU | 1623287 tok/s
step 14471/19560 | loss 3.327796 (+0.28z)| norm 0.2750 (-0.43z)| lr 1.02e-04 | 322.72 ms | 52.3% bf16 MFU | 1623353 tok/s
step 14472/19560 | loss 3.372680 (+1.51z)| norm 0.2858 (+0.25z)| lr 1.01e-04 | 322.86 ms | 52.3% bf16 MFU | 1623380 tok/s
step 14473/19560 | loss 3.251233 (-1.80z)| norm 0.2907 (+0.54z)| lr 1.01e-04 | 323.10 ms | 52.2% bf16 MFU | 1623346 tok/s
step 14474/19560 | loss 3.339286 (+0.58z)| norm 0.2823 (+0.00z)| lr 1.01e-04 | 323.06 ms | 52.2% bf16 MFU | 1623323 tok/s
step 14475/19560 | loss 3.281777 (-0.98z)| norm 0.3221 (+2.46z)| lr 1.01e-04 | 322.81 ms | 52.3% bf16 MFU | 1623364 tok/s
step 14476/19560 | loss 3.265044 (-1.41z)| norm 0.2885 (+0.37z)| lr 1.01e-04 | 322.84 ms | 52.3% bf16 MFU | 1623395 tok/s
step 14477/19560 | loss 3.325948 (+0.22z)| norm 0.2859 (+0.20z)| lr 1.01e-04 | 322.94 ms | 52.3% bf16 MFU | 1623399 tok/s
step 14478/19560 | loss 3.291656 (-0.71z)| norm 0.2776 (-0.32z)| lr 1.01e-04 | 322.76 ms | 52.3% bf16 MFU | 1623449 tok/s
step 14479/19560 | loss 3.387683 (+1.86z)| norm 0.2870 (+0.27z)| lr 1.01e-04 | 323.06 ms | 52.2% bf16 MFU | 1623421 tok/s
step 14480/19560 | loss 3.322519 (+0.12z)| norm 0.2787 (-0.25z)| lr 1.01e-04 | 322.82 ms | 52.3% bf16 MFU | 1623454 tok/s
step 14481/19560 | loss 3.389024 (+1.86z)| norm 0.3029 (+1.25z)| lr 1.01e-04 | 323.01 ms | 52.2% bf16 MFU | 1623438 tok/s
step 14482/19560 | loss 3.309615 (-0.25z)| norm 0.2806 (-0.15z)| lr 1.01e-04 | 322.61 ms | 52.3% bf16 MFU | 1623523 tok/s
step 14483/19560 | loss 3.256525 (-1.63z)| norm 0.2898 (+0.41z)| lr 1.01e-04 | 322.98 ms | 52.3% bf16 MFU | 1623512 tok/s
step 14484/19560 | loss 3.337569 (+0.52z)| norm 0.3199 (+2.25z)| lr 1.01e-04 | 323.30 ms | 52.2% bf16 MFU | 1623420 tok/s
step 14485/19560 | loss 3.282506 (-0.95z)| norm 0.2894 (+0.36z)| lr 1.01e-04 | 323.06 ms | 52.2% bf16 MFU | 1623394 tok/s
step 14486/19560 | loss 3.252466 (-1.72z)| norm 0.3090 (+1.57z)| lr 1.01e-04 | 323.55 ms | 52.2% bf16 MFU | 1623247 tok/s
step 14487/19560 | loss 3.336040 (+0.47z)| norm 0.2902 (+0.39z)| lr 1.01e-04 | 323.09 ms | 52.2% bf16 MFU | 1623222 tok/s
step 14488/19560 | loss 3.421390 (+2.64z)| norm 0.2902 (+0.39z)| lr 1.01e-04 | 322.36 ms | 52.4% bf16 MFU | 1623382 tok/s
step 14489/19560 | loss 3.280060 (-1.00z)| norm 0.2963 (+0.77z)| lr 1.01e-04 | 323.07 ms | 52.2% bf16 MFU | 1623355 tok/s
step 14490/19560 | loss 3.379148 (+1.52z)| norm 0.3056 (+1.33z)| lr 1.01e-04 | 323.47 ms | 52.2% bf16 MFU | 1623228 tok/s
step 14491/19560 | loss 3.305438 (-0.36z)| norm 0.2985 (+0.87z)| lr 1.01e-04 | 323.14 ms | 52.2% bf16 MFU | 1623190 tok/s
step 14492/19560 | loss 3.310031 (-0.24z)| norm 0.2953 (+0.67z)| lr 1.01e-04 | 323.87 ms | 52.1% bf16 MFU | 1622971 tok/s
step 14493/19560 | loss 3.376719 (+1.45z)| norm 0.3055 (+1.29z)| lr 1.01e-04 | 322.91 ms | 52.3% bf16 MFU | 1623004 tok/s
step 14494/19560 | loss 3.336553 (+0.43z)| norm 0.3062 (+1.32z)| lr 1.01e-04 | 323.82 ms | 52.1% bf16 MFU | 1622807 tok/s
step 14495/19560 | loss 3.377620 (+1.46z)| norm 0.3014 (+1.02z)| lr 1.01e-04 | 322.63 ms | 52.3% bf16 MFU | 1622919 tok/s
step 14496/19560 | loss 3.347986 (+0.71z)| norm 0.2744 (-0.63z)| lr 1.01e-04 | 323.21 ms | 52.2% bf16 MFU | 1622880 tok/s
step 14497/19560 | loss 3.306366 (-0.35z)| norm 0.2874 (+0.16z)| lr 1.01e-04 | 323.88 ms | 52.1% bf16 MFU | 1622675 tok/s
step 14498/19560 | loss 3.273909 (-1.17z)| norm 0.2709 (-0.85z)| lr 1.01e-04 | 322.66 ms | 52.3% bf16 MFU | 1622786 tok/s
step 14499/19560 | loss 3.330202 (+0.25z)| norm 0.2922 (+0.46z)| lr 1.00e-04 | 323.16 ms | 52.2% bf16 MFU | 1622765 tok/s
step 14500/19560 | loss 3.368294 (+1.21z)| norm 0.2890 (+0.26z)| lr 1.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1622895 tok/s
val loss 3.310410
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2996/10042 = 0.298347
step 14501/19560 | loss 3.254345 (-1.66z)| norm 0.2705 (-0.87z)| lr 1.00e-04 | 323.01 ms | 52.3% bf16 MFU | 1622908 tok/s
step 14502/19560 | loss 3.303481 (-0.41z)| norm 0.2886 (+0.24z)| lr 1.00e-04 | 322.62 ms | 52.3% bf16 MFU | 1623017 tok/s
step 14503/19560 | loss 3.367205 (+1.24z)| norm 0.2987 (+0.86z)| lr 1.00e-04 | 323.01 ms | 52.2% bf16 MFU | 1623022 tok/s
step 14504/19560 | loss 3.330722 (+0.28z)| norm 0.2914 (+0.44z)| lr 1.00e-04 | 322.99 ms | 52.3% bf16 MFU | 1623033 tok/s
step 14505/19560 | loss 3.295710 (-0.62z)| norm 0.2587 (-1.62z)| lr 1.00e-04 | 323.16 ms | 52.2% bf16 MFU | 1623000 tok/s
step 14506/19560 | loss 3.311473 (-0.20z)| norm 0.3352 (+3.13z)| lr 1.00e-04 | 322.58 ms | 52.3% bf16 MFU | 1623115 tok/s
step 14507/19560 | loss 3.297852 (-0.55z)| norm 0.2623 (-1.35z)| lr 1.00e-04 | 322.81 ms | 52.3% bf16 MFU | 1623166 tok/s
step 14508/19560 | loss 3.315195 (-0.09z)| norm 0.2801 (-0.25z)| lr 1.00e-04 | 322.75 ms | 52.3% bf16 MFU | 1623229 tok/s
step 14509/19560 | loss 3.356063 (+0.96z)| norm 0.3256 (+2.49z)| lr 1.00e-04 | 322.86 ms | 52.3% bf16 MFU | 1623263 tok/s
step 14510/19560 | loss 3.332268 (+0.35z)| norm 0.2645 (-1.19z)| lr 1.00e-04 | 323.15 ms | 52.2% bf16 MFU | 1623222 tok/s
step 14511/19560 | loss 3.362339 (+1.12z)| norm 0.2870 (+0.20z)| lr 1.00e-04 | 322.57 ms | 52.3% bf16 MFU | 1623328 tok/s
step 14512/19560 | loss 3.279346 (-1.02z)| norm 0.2723 (-0.70z)| lr 1.00e-04 | 323.23 ms | 52.2% bf16 MFU | 1623264 tok/s
step 14513/19560 | loss 3.351907 (+0.84z)| norm 0.2717 (-0.72z)| lr 1.00e-04 | 322.76 ms | 52.3% bf16 MFU | 1623320 tok/s
step 14514/19560 | loss 3.309246 (-0.25z)| norm 0.2887 (+0.31z)| lr 9.99e-05 | 322.32 ms | 52.4% bf16 MFU | 1623483 tok/s
step 14515/19560 | loss 3.346628 (+0.70z)| norm 0.2644 (-1.18z)| lr 9.99e-05 | 323.01 ms | 52.2% bf16 MFU | 1623466 tok/s
step 14516/19560 | loss 3.284127 (-0.90z)| norm 0.2681 (-0.94z)| lr 9.98e-05 | 323.01 ms | 52.2% bf16 MFU | 1623449 tok/s
step 14517/19560 | loss 3.292823 (-0.66z)| norm 0.2763 (-0.43z)| lr 9.98e-05 | 322.74 ms | 52.3% bf16 MFU | 1623501 tok/s
step 14518/19560 | loss 3.335641 (+0.43z)| norm 0.2595 (-1.45z)| lr 9.98e-05 | 322.51 ms | 52.3% bf16 MFU | 1623608 tok/s
step 14519/19560 | loss 3.320058 (+0.02z)| norm 0.2809 (-0.14z)| lr 9.97e-05 | 322.89 ms | 52.3% bf16 MFU | 1623615 tok/s
step 14520/19560 | loss 3.347274 (+0.71z)| norm 0.2785 (-0.28z)| lr 9.97e-05 | 323.07 ms | 52.2% bf16 MFU | 1623577 tok/s
step 14521/19560 | loss 3.416597 (+2.42z)| norm 0.3540 (+4.04z)| lr 9.97e-05 | 322.58 ms | 52.3% bf16 MFU | 1623662 tok/s
step 14522/19560 | loss 3.351929 (+0.78z)| norm 0.2826 (-0.07z)| lr 9.96e-05 | 322.71 ms | 52.3% bf16 MFU | 1623712 tok/s
step 14523/19560 | loss 3.301274 (-0.50z)| norm 0.2634 (-1.16z)| lr 9.96e-05 | 322.72 ms | 52.3% bf16 MFU | 1623757 tok/s
step 14524/19560 | loss 3.336997 (+0.41z)| norm 0.2825 (-0.06z)| lr 9.95e-05 | 322.95 ms | 52.3% bf16 MFU | 1623740 tok/s
step 14525/19560 | loss 3.381130 (+1.52z)| norm 0.3093 (+1.46z)| lr 9.95e-05 | 323.05 ms | 52.2% bf16 MFU | 1623700 tok/s
step 14526/19560 | loss 3.347576 (+0.67z)| norm 0.2906 (+0.38z)| lr 9.95e-05 | 322.33 ms | 52.4% bf16 MFU | 1623843 tok/s
step 14527/19560 | loss 3.285291 (-0.90z)| norm 0.2720 (-0.68z)| lr 9.94e-05 | 323.19 ms | 52.2% bf16 MFU | 1623763 tok/s
step 14528/19560 | loss 3.284036 (-0.92z)| norm 0.2971 (+0.75z)| lr 9.94e-05 | 323.35 ms | 52.2% bf16 MFU | 1623648 tok/s
step 14529/19560 | loss 3.291179 (-0.74z)| norm 0.2823 (-0.10z)| lr 9.94e-05 | 322.43 ms | 52.3% bf16 MFU | 1623767 tok/s
step 14530/19560 | loss 3.423788 (+2.53z)| norm 0.2801 (-0.23z)| lr 9.93e-05 | 323.28 ms | 52.2% bf16 MFU | 1623668 tok/s
step 14531/19560 | loss 3.303305 (-0.44z)| norm 0.2548 (-1.65z)| lr 9.93e-05 | 322.56 ms | 52.3% bf16 MFU | 1623754 tok/s
step 14532/19560 | loss 3.346347 (+0.62z)| norm 0.2777 (-0.36z)| lr 9.92e-05 | 322.83 ms | 52.3% bf16 MFU | 1623768 tok/s
step 14533/19560 | loss 3.335580 (+0.35z)| norm 0.2861 (+0.12z)| lr 9.92e-05 | 323.17 ms | 52.2% bf16 MFU | 1623695 tok/s
step 14534/19560 | loss 3.311791 (-0.23z)| norm 0.2855 (+0.09z)| lr 9.92e-05 | 322.88 ms | 52.3% bf16 MFU | 1623700 tok/s
step 14535/19560 | loss 3.408856 (+2.11z)| norm 0.2901 (+0.35z)| lr 9.91e-05 | 323.18 ms | 52.2% bf16 MFU | 1623629 tok/s
step 14536/19560 | loss 3.336654 (+0.35z)| norm 0.2788 (-0.29z)| lr 9.91e-05 | 322.20 ms | 52.4% bf16 MFU | 1623807 tok/s
step 14537/19560 | loss 3.291463 (-0.75z)| norm 0.2591 (-1.39z)| lr 9.91e-05 | 322.89 ms | 52.3% bf16 MFU | 1623805 tok/s
step 14538/19560 | loss 3.421760 (+2.35z)| norm 0.3202 (+2.01z)| lr 9.90e-05 | 323.45 ms | 52.2% bf16 MFU | 1623659 tok/s
step 14539/19560 | loss 3.353514 (+0.71z)| norm 0.2774 (-0.38z)| lr 9.90e-05 | 322.39 ms | 52.4% bf16 MFU | 1623790 tok/s
step 14540/19560 | loss 3.325599 (+0.04z)| norm 0.2867 (+0.13z)| lr 9.90e-05 | 322.67 ms | 52.3% bf16 MFU | 1623844 tok/s
step 14541/19560 | loss 3.237325 (-2.03z)| norm 0.2684 (-0.87z)| lr 9.89e-05 | 322.28 ms | 52.4% bf16 MFU | 1623992 tok/s
step 14542/19560 | loss 3.280521 (-0.99z)| norm 0.2821 (-0.11z)| lr 9.89e-05 | 324.03 ms | 52.1% bf16 MFU | 1623692 tok/s
step 14543/19560 | loss 3.430794 (+2.48z)| norm 0.3022 (+1.00z)| lr 9.88e-05 | 322.91 ms | 52.3% bf16 MFU | 1623689 tok/s
step 14544/19560 | loss 3.314864 (-0.20z)| norm 0.2694 (-0.82z)| lr 9.88e-05 | 322.30 ms | 52.4% bf16 MFU | 1623839 tok/s
step 14545/19560 | loss 3.387948 (+1.47z)| norm 0.2684 (-0.87z)| lr 9.88e-05 | 322.80 ms | 52.3% bf16 MFU | 1623856 tok/s
step 14546/19560 | loss 3.349779 (+0.57z)| norm 0.2982 (+0.78z)| lr 9.87e-05 | 322.80 ms | 52.3% bf16 MFU | 1623872 tok/s
step 14547/19560 | loss 3.323548 (-0.04z)| norm 0.2856 (+0.08z)| lr 9.87e-05 | 322.72 ms | 52.3% bf16 MFU | 1623909 tok/s
step 14548/19560 | loss 3.297022 (-0.64z)| norm 0.3021 (+0.99z)| lr 9.87e-05 | 322.47 ms | 52.3% bf16 MFU | 1624005 tok/s
step 14549/19560 | loss 3.293712 (-0.72z)| norm 0.3024 (+0.99z)| lr 9.86e-05 | 323.09 ms | 52.2% bf16 MFU | 1623941 tok/s
step 14550/19560 | loss 3.363156 (+0.91z)| norm 0.2882 (+0.20z)| lr 9.86e-05 | 323.07 ms | 52.2% bf16 MFU | 1623886 tok/s
step 14551/19560 | loss 3.306400 (-0.41z)| norm 0.3139 (+1.60z)| lr 9.85e-05 | 322.87 ms | 52.3% bf16 MFU | 1623884 tok/s
step 14552/19560 | loss 3.336866 (+0.30z)| norm 0.2648 (-1.10z)| lr 9.85e-05 | 322.79 ms | 52.3% bf16 MFU | 1623902 tok/s
step 14553/19560 | loss 3.293725 (-0.71z)| norm 0.2828 (-0.11z)| lr 9.85e-05 | 322.85 ms | 52.3% bf16 MFU | 1623904 tok/s
step 14554/19560 | loss 3.282551 (-0.96z)| norm 0.2824 (-0.13z)| lr 9.84e-05 | 322.92 ms | 52.3% bf16 MFU | 1623887 tok/s
step 14555/19560 | loss 3.316817 (-0.15z)| norm 0.2637 (-1.15z)| lr 9.84e-05 | 323.10 ms | 52.2% bf16 MFU | 1623827 tok/s
step 14556/19560 | loss 3.300203 (-0.54z)| norm 0.2772 (-0.42z)| lr 9.84e-05 | 322.13 ms | 52.4% bf16 MFU | 1624014 tok/s
step 14557/19560 | loss 3.338936 (+0.36z)| norm 0.2718 (-0.70z)| lr 9.83e-05 | 323.42 ms | 52.2% bf16 MFU | 1623868 tok/s
step 14558/19560 | loss 3.318869 (-0.11z)| norm 0.2717 (-0.71z)| lr 9.83e-05 | 323.32 ms | 52.2% bf16 MFU | 1623752 tok/s
step 14559/19560 | loss 3.323725 (-0.02z)| norm 0.2898 (+0.28z)| lr 9.82e-05 | 322.36 ms | 52.4% bf16 MFU | 1623886 tok/s
step 14560/19560 | loss 3.324794 (+0.01z)| norm 0.2707 (-0.77z)| lr 9.82e-05 | 322.55 ms | 52.3% bf16 MFU | 1623964 tok/s
step 14561/19560 | loss 3.305248 (-0.45z)| norm 0.2648 (-1.08z)| lr 9.82e-05 | 322.80 ms | 52.3% bf16 MFU | 1623975 tok/s
step 14562/19560 | loss 3.389652 (+1.56z)| norm 0.2701 (-0.77z)| lr 9.81e-05 | 323.07 ms | 52.2% bf16 MFU | 1623918 tok/s
step 14563/19560 | loss 3.302207 (-0.53z)| norm 0.2725 (-0.64z)| lr 9.81e-05 | 322.70 ms | 52.3% bf16 MFU | 1623957 tok/s
step 14564/19560 | loss 3.324370 (+0.01z)| norm 0.2537 (-1.65z)| lr 9.81e-05 | 322.70 ms | 52.3% bf16 MFU | 1623993 tok/s
step 14565/19560 | loss 3.327768 (+0.09z)| norm 0.2705 (-0.73z)| lr 9.80e-05 | 322.63 ms | 52.3% bf16 MFU | 1624046 tok/s
step 14566/19560 | loss 3.240842 (-1.97z)| norm 0.2604 (-1.27z)| lr 9.80e-05 | 322.70 ms | 52.3% bf16 MFU | 1624077 tok/s
step 14567/19560 | loss 3.254075 (-1.63z)| norm 0.2683 (-0.83z)| lr 9.80e-05 | 323.09 ms | 52.2% bf16 MFU | 1624010 tok/s
step 14568/19560 | loss 3.278509 (-1.04z)| norm 0.2748 (-0.47z)| lr 9.79e-05 | 322.93 ms | 52.3% bf16 MFU | 1623987 tok/s
step 14569/19560 | loss 3.334594 (+0.28z)| norm 0.2639 (-1.04z)| lr 9.79e-05 | 322.55 ms | 52.3% bf16 MFU | 1624060 tok/s
step 14570/19560 | loss 3.356169 (+0.77z)| norm 0.2925 (+0.50z)| lr 9.78e-05 | 322.93 ms | 52.3% bf16 MFU | 1624035 tok/s
step 14571/19560 | loss 3.310243 (-0.31z)| norm 0.2757 (-0.41z)| lr 9.78e-05 | 322.63 ms | 52.3% bf16 MFU | 1624086 tok/s
step 14572/19560 | loss 3.343351 (+0.46z)| norm 0.2760 (-0.40z)| lr 9.78e-05 | 323.40 ms | 52.2% bf16 MFU | 1623942 tok/s
step 14573/19560 | loss 3.384600 (+1.41z)| norm 0.2644 (-1.03z)| lr 9.77e-05 | 322.92 ms | 52.3% bf16 MFU | 1623925 tok/s
step 14574/19560 | loss 3.278972 (-1.07z)| norm 0.2846 (+0.06z)| lr 9.77e-05 | 322.28 ms | 52.4% bf16 MFU | 1624070 tok/s
step 14575/19560 | loss 3.391264 (+1.57z)| norm 0.2963 (+0.68z)| lr 9.77e-05 | 322.70 ms | 52.3% bf16 MFU | 1624100 tok/s
step 14576/19560 | loss 3.276950 (-1.10z)| norm 0.2872 (+0.19z)| lr 9.76e-05 | 322.71 ms | 52.3% bf16 MFU | 1624126 tok/s
step 14577/19560 | loss 3.325688 (+0.03z)| norm 0.2929 (+0.49z)| lr 9.76e-05 | 323.20 ms | 52.2% bf16 MFU | 1624029 tok/s
step 14578/19560 | loss 3.282362 (-1.02z)| norm 0.2615 (-1.23z)| lr 9.75e-05 | 322.61 ms | 52.3% bf16 MFU | 1624086 tok/s
step 14579/19560 | loss 3.300758 (-0.58z)| norm 0.3108 (+1.48z)| lr 9.75e-05 | 322.55 ms | 52.3% bf16 MFU | 1624153 tok/s
step 14580/19560 | loss 3.309038 (-0.36z)| norm 0.2756 (-0.46z)| lr 9.75e-05 | 323.07 ms | 52.2% bf16 MFU | 1624087 tok/s
step 14581/19560 | loss 3.400812 (+1.84z)| norm 0.3030 (+1.07z)| lr 9.74e-05 | 322.74 ms | 52.3% bf16 MFU | 1624106 tok/s
step 14582/19560 | loss 3.308099 (-0.39z)| norm 0.2858 (+0.11z)| lr 9.74e-05 | 322.92 ms | 52.3% bf16 MFU | 1624079 tok/s
step 14583/19560 | loss 3.335306 (+0.27z)| norm 0.2745 (-0.51z)| lr 9.74e-05 | 322.55 ms | 52.3% bf16 MFU | 1624147 tok/s
step 14584/19560 | loss 3.362363 (+0.92z)| norm 0.2560 (-1.53z)| lr 9.73e-05 | 322.72 ms | 52.3% bf16 MFU | 1624169 tok/s
step 14585/19560 | loss 3.328340 (+0.09z)| norm 0.2602 (-1.28z)| lr 9.73e-05 | 322.63 ms | 52.3% bf16 MFU | 1624214 tok/s
step 14586/19560 | loss 3.359573 (+0.83z)| norm 0.2918 (+0.45z)| lr 9.73e-05 | 323.27 ms | 52.2% bf16 MFU | 1624095 tok/s
step 14587/19560 | loss 3.357725 (+0.78z)| norm 0.2645 (-1.04z)| lr 9.72e-05 | 322.95 ms | 52.3% bf16 MFU | 1624061 tok/s
step 14588/19560 | loss 3.326423 (+0.02z)| norm 0.2871 (+0.19z)| lr 9.72e-05 | 322.35 ms | 52.4% bf16 MFU | 1624181 tok/s
step 14589/19560 | loss 3.278386 (-1.13z)| norm 0.2873 (+0.20z)| lr 9.71e-05 | 322.71 ms | 52.3% bf16 MFU | 1624205 tok/s
step 14590/19560 | loss 3.344240 (+0.45z)| norm 0.3193 (+1.91z)| lr 9.71e-05 | 322.69 ms | 52.3% bf16 MFU | 1624231 tok/s
step 14591/19560 | loss 3.302583 (-0.56z)| norm 0.2987 (+0.79z)| lr 9.71e-05 | 323.14 ms | 52.2% bf16 MFU | 1624144 tok/s
step 14592/19560 | loss 3.330182 (+0.09z)| norm 0.2699 (-0.77z)| lr 9.70e-05 | 323.08 ms | 52.2% bf16 MFU | 1624075 tok/s
step 14593/19560 | loss 3.327936 (+0.04z)| norm 0.3039 (+1.05z)| lr 9.70e-05 | 322.43 ms | 52.3% bf16 MFU | 1624174 tok/s
step 14594/19560 | loss 3.340404 (+0.35z)| norm 0.2896 (+0.27z)| lr 9.70e-05 | 322.33 ms | 52.4% bf16 MFU | 1624292 tok/s
step 14595/19560 | loss 3.298463 (-0.67z)| norm 0.2933 (+0.46z)| lr 9.69e-05 | 323.27 ms | 52.2% bf16 MFU | 1624169 tok/s
step 14596/19560 | loss 3.340210 (+0.34z)| norm 0.2960 (+0.60z)| lr 9.69e-05 | 323.11 ms | 52.2% bf16 MFU | 1624091 tok/s
step 14597/19560 | loss 3.356422 (+0.72z)| norm 0.3077 (+1.22z)| lr 9.68e-05 | 322.76 ms | 52.3% bf16 MFU | 1624106 tok/s
step 14598/19560 | loss 3.327675 (+0.04z)| norm 0.2888 (+0.22z)| lr 9.68e-05 | 322.58 ms | 52.3% bf16 MFU | 1624166 tok/s
step 14599/19560 | loss 3.345803 (+0.48z)| norm 0.2740 (-0.63z)| lr 9.68e-05 | 322.66 ms | 52.3% bf16 MFU | 1624203 tok/s
step 14600/19560 | loss 3.309329 (-0.41z)| norm 0.2764 (-0.49z)| lr 9.67e-05 | 323.12 ms | 52.2% bf16 MFU | 1624123 tok/s
step 14601/19560 | loss 3.250314 (-1.87z)| norm 0.2716 (-0.76z)| lr 9.67e-05 | 322.66 ms | 52.3% bf16 MFU | 1624163 tok/s
step 14602/19560 | loss 3.309770 (-0.39z)| norm 0.2881 (+0.19z)| lr 9.67e-05 | 323.30 ms | 52.2% bf16 MFU | 1624040 tok/s
step 14603/19560 | loss 3.298617 (-0.67z)| norm 0.3456 (+3.39z)| lr 9.66e-05 | 322.71 ms | 52.3% bf16 MFU | 1624069 tok/s
step 14604/19560 | loss 3.328121 (+0.05z)| norm 0.2773 (-0.42z)| lr 9.66e-05 | 322.75 ms | 52.3% bf16 MFU | 1624087 tok/s
step 14605/19560 | loss 3.284479 (-1.03z)| norm 0.2701 (-0.82z)| lr 9.66e-05 | 323.15 ms | 52.2% bf16 MFU | 1624004 tok/s
step 14606/19560 | loss 3.279971 (-1.14z)| norm 0.3410 (+3.00z)| lr 9.65e-05 | 323.32 ms | 52.2% bf16 MFU | 1623883 tok/s
step 14607/19560 | loss 3.364467 (+0.97z)| norm 0.3621 (+3.85z)| lr 9.65e-05 | 322.83 ms | 52.3% bf16 MFU | 1623891 tok/s
step 14608/19560 | loss 3.339699 (+0.35z)| norm 0.2896 (+0.19z)| lr 9.64e-05 | 323.08 ms | 52.2% bf16 MFU | 1623835 tok/s
step 14609/19560 | loss 3.307239 (-0.45z)| norm 0.2969 (+0.56z)| lr 9.64e-05 | 322.52 ms | 52.3% bf16 MFU | 1623924 tok/s
step 14610/19560 | loss 3.372091 (+1.17z)| norm 0.3249 (+1.93z)| lr 9.64e-05 | 322.92 ms | 52.3% bf16 MFU | 1623908 tok/s
step 14611/19560 | loss 3.365892 (+1.00z)| norm 0.3117 (+1.25z)| lr 9.63e-05 | 323.09 ms | 52.2% bf16 MFU | 1623849 tok/s
step 14612/19560 | loss 3.362702 (+0.91z)| norm 0.2730 (-0.65z)| lr 9.63e-05 | 322.76 ms | 52.3% bf16 MFU | 1623877 tok/s
step 14613/19560 | loss 3.324939 (-0.05z)| norm 0.2909 (+0.24z)| lr 9.63e-05 | 323.03 ms | 52.2% bf16 MFU | 1623834 tok/s
step 14614/19560 | loss 3.320237 (-0.19z)| norm 0.2630 (-1.13z)| lr 9.62e-05 | 322.30 ms | 52.4% bf16 MFU | 1623977 tok/s
step 14615/19560 | loss 3.456885 (+3.17z)| norm 0.3158 (+1.49z)| lr 9.62e-05 | 323.04 ms | 52.2% bf16 MFU | 1623928 tok/s
step 14616/19560 | loss 3.290753 (-0.92z)| norm 0.2876 (+0.09z)| lr 9.61e-05 | 323.42 ms | 52.2% bf16 MFU | 1623785 tok/s
step 14617/19560 | loss 3.283289 (-1.11z)| norm 0.2877 (+0.10z)| lr 9.61e-05 | 322.46 ms | 52.3% bf16 MFU | 1623892 tok/s
step 14618/19560 | loss 3.302650 (-0.61z)| norm 0.2762 (-0.46z)| lr 9.61e-05 | 323.10 ms | 52.2% bf16 MFU | 1623832 tok/s
step 14619/19560 | loss 3.285702 (-1.04z)| norm 0.2897 (+0.21z)| lr 9.60e-05 | 322.95 ms | 52.3% bf16 MFU | 1623812 tok/s
step 14620/19560 | loss 3.287811 (-0.97z)| norm 0.2809 (-0.22z)| lr 9.60e-05 | 323.19 ms | 52.2% bf16 MFU | 1623733 tok/s
step 14621/19560 | loss 3.325985 (-0.00z)| norm 0.2598 (-1.25z)| lr 9.60e-05 | 323.03 ms | 52.2% bf16 MFU | 1623698 tok/s
step 14622/19560 | loss 3.339548 (+0.34z)| norm 0.2871 (+0.11z)| lr 9.59e-05 | 323.07 ms | 52.2% bf16 MFU | 1623655 tok/s
step 14623/19560 | loss 3.294035 (-0.80z)| norm 0.2797 (-0.25z)| lr 9.59e-05 | 322.77 ms | 52.3% bf16 MFU | 1623689 tok/s
step 14624/19560 | loss 3.402920 (+1.93z)| norm 0.2921 (+0.37z)| lr 9.59e-05 | 322.80 ms | 52.3% bf16 MFU | 1623713 tok/s
step 14625/19560 | loss 3.290045 (-0.90z)| norm 0.2831 (-0.08z)| lr 9.58e-05 | 322.81 ms | 52.3% bf16 MFU | 1623733 tok/s
step 14626/19560 | loss 3.335213 (+0.22z)| norm 0.2623 (-1.12z)| lr 9.58e-05 | 323.34 ms | 52.2% bf16 MFU | 1623619 tok/s
step 14627/19560 | loss 3.269559 (-1.41z)| norm 0.2850 (+0.02z)| lr 9.57e-05 | 322.78 ms | 52.3% bf16 MFU | 1623654 tok/s
step 14628/19560 | loss 3.272354 (-1.31z)| norm 0.2646 (-0.99z)| lr 9.57e-05 | 322.76 ms | 52.3% bf16 MFU | 1623691 tok/s
step 14629/19560 | loss 3.349859 (+0.61z)| norm 0.2525 (-1.57z)| lr 9.57e-05 | 322.96 ms | 52.3% bf16 MFU | 1623676 tok/s
step 14630/19560 | loss 3.246512 (-1.96z)| norm 0.2794 (-0.24z)| lr 9.56e-05 | 322.77 ms | 52.3% bf16 MFU | 1623708 tok/s
step 14631/19560 | loss 3.292756 (-0.80z)| norm 0.2989 (+0.73z)| lr 9.56e-05 | 323.39 ms | 52.2% bf16 MFU | 1623585 tok/s
step 14632/19560 | loss 3.376809 (+1.28z)| norm 0.2697 (-0.71z)| lr 9.56e-05 | 323.13 ms | 52.2% bf16 MFU | 1623532 tok/s
step 14633/19560 | loss 3.422136 (+2.33z)| norm 0.2699 (-0.71z)| lr 9.55e-05 | 323.75 ms | 52.1% bf16 MFU | 1623327 tok/s
step 14634/19560 | loss 3.376843 (+1.21z)| norm 0.2722 (-0.59z)| lr 9.55e-05 | 322.30 ms | 52.4% bf16 MFU | 1623497 tok/s
step 14635/19560 | loss 3.373631 (+1.11z)| norm 0.2763 (-0.38z)| lr 9.55e-05 | 323.00 ms | 52.3% bf16 MFU | 1623481 tok/s
step 14636/19560 | loss 3.423398 (+2.25z)| norm 0.2748 (-0.46z)| lr 9.54e-05 | 322.73 ms | 52.3% bf16 MFU | 1623534 tok/s
step 14637/19560 | loss 3.417484 (+2.06z)| norm 0.2751 (-0.43z)| lr 9.54e-05 | 322.60 ms | 52.3% bf16 MFU | 1623617 tok/s
step 14638/19560 | loss 3.290550 (-0.87z)| norm 0.2663 (-0.89z)| lr 9.53e-05 | 322.84 ms | 52.3% bf16 MFU | 1623636 tok/s
step 14639/19560 | loss 3.277784 (-1.15z)| norm 0.2699 (-0.69z)| lr 9.53e-05 | 323.09 ms | 52.2% bf16 MFU | 1623590 tok/s
step 14640/19560 | loss 3.329502 (+0.03z)| norm 0.2676 (-0.81z)| lr 9.53e-05 | 323.33 ms | 52.2% bf16 MFU | 1623488 tok/s
step 14641/19560 | loss 3.237856 (-2.04z)| norm 0.2708 (-0.64z)| lr 9.52e-05 | 323.34 ms | 52.2% bf16 MFU | 1623386 tok/s
step 14642/19560 | loss 3.285822 (-0.94z)| norm 0.2802 (-0.15z)| lr 9.52e-05 | 322.47 ms | 52.3% bf16 MFU | 1623509 tok/s
step 14643/19560 | loss 3.325940 (-0.02z)| norm 0.2873 (+0.21z)| lr 9.52e-05 | 322.67 ms | 52.3% bf16 MFU | 1623576 tok/s
step 14644/19560 | loss 3.291300 (-0.81z)| norm 0.2664 (-0.88z)| lr 9.51e-05 | 323.70 ms | 52.1% bf16 MFU | 1623380 tok/s
step 14645/19560 | loss 3.373269 (+1.04z)| norm 0.3090 (+1.31z)| lr 9.51e-05 | 323.68 ms | 52.1% bf16 MFU | 1623201 tok/s
step 14646/19560 | loss 3.418014 (+2.01z)| norm 0.2898 (+0.31z)| lr 9.51e-05 | 322.66 ms | 52.3% bf16 MFU | 1623285 tok/s
step 14647/19560 | loss 3.321961 (-0.14z)| norm 0.2913 (+0.39z)| lr 9.50e-05 | 323.08 ms | 52.2% bf16 MFU | 1623260 tok/s
step 14648/19560 | loss 3.282185 (-1.01z)| norm 0.2727 (-0.58z)| lr 9.50e-05 | 323.02 ms | 52.2% bf16 MFU | 1623252 tok/s
step 14649/19560 | loss 3.369435 (+0.95z)| norm 0.3054 (+1.20z)| lr 9.49e-05 | 323.45 ms | 52.2% bf16 MFU | 1623134 tok/s
step 14650/19560 | loss 3.447999 (+2.64z)| norm 0.2862 (+0.15z)| lr 9.49e-05 | 322.81 ms | 52.3% bf16 MFU | 1623184 tok/s
step 14651/19560 | loss 3.321913 (-0.14z)| norm 0.2973 (+0.74z)| lr 9.49e-05 | 323.02 ms | 52.2% bf16 MFU | 1623180 tok/s
step 14652/19560 | loss 3.350397 (+0.49z)| norm 0.2919 (+0.44z)| lr 9.48e-05 | 323.08 ms | 52.2% bf16 MFU | 1623160 tok/s
step 14653/19560 | loss 3.363472 (+0.78z)| norm 0.2771 (-0.35z)| lr 9.48e-05 | 322.48 ms | 52.3% bf16 MFU | 1623293 tok/s
step 14654/19560 | loss 3.315522 (-0.27z)| norm 0.3154 (+1.72z)| lr 9.48e-05 | 323.02 ms | 52.2% bf16 MFU | 1623282 tok/s
step 14655/19560 | loss 3.258824 (-1.51z)| norm 0.3260 (+2.24z)| lr 9.47e-05 | 323.38 ms | 52.2% bf16 MFU | 1623183 tok/s
step 14656/19560 | loss 3.477751 (+3.15z)| norm 0.3461 (+3.16z)| lr 9.47e-05 | 323.19 ms | 52.2% bf16 MFU | 1623136 tok/s
step 14657/19560 | loss 3.330485 (+0.02z)| norm 0.2966 (+0.61z)| lr 9.47e-05 | 322.86 ms | 52.3% bf16 MFU | 1623173 tok/s
step 14658/19560 | loss 3.318984 (-0.21z)| norm 0.2872 (+0.13z)| lr 9.46e-05 | 322.82 ms | 52.3% bf16 MFU | 1623220 tok/s
step 14659/19560 | loss 3.339489 (+0.23z)| norm 0.2949 (+0.51z)| lr 9.46e-05 | 323.45 ms | 52.2% bf16 MFU | 1623105 tok/s
step 14660/19560 | loss 3.351540 (+0.49z)| norm 0.2769 (-0.42z)| lr 9.45e-05 | 322.63 ms | 52.3% bf16 MFU | 1623202 tok/s
step 14661/19560 | loss 3.321476 (-0.16z)| norm 0.2710 (-0.72z)| lr 9.45e-05 | 323.27 ms | 52.2% bf16 MFU | 1623134 tok/s
step 14662/19560 | loss 3.325454 (-0.08z)| norm 0.2722 (-0.65z)| lr 9.45e-05 | 323.28 ms | 52.2% bf16 MFU | 1623067 tok/s
step 14663/19560 | loss 3.318755 (-0.21z)| norm 0.2830 (-0.09z)| lr 9.44e-05 | 323.15 ms | 52.2% bf16 MFU | 1623034 tok/s
step 14664/19560 | loss 3.309651 (-0.40z)| norm 0.2791 (-0.29z)| lr 9.44e-05 | 323.15 ms | 52.2% bf16 MFU | 1623004 tok/s
step 14665/19560 | loss 3.292417 (-0.78z)| norm 0.2611 (-1.22z)| lr 9.44e-05 | 323.24 ms | 52.2% bf16 MFU | 1622952 tok/s
step 14666/19560 | loss 3.384979 (+1.26z)| norm 0.3029 (+0.95z)| lr 9.43e-05 | 324.34 ms | 52.0% bf16 MFU | 1622629 tok/s
step 14667/19560 | loss 3.306721 (-0.46z)| norm 0.2843 (-0.02z)| lr 9.43e-05 | 323.20 ms | 52.2% bf16 MFU | 1622606 tok/s
step 14668/19560 | loss 3.268055 (-1.29z)| norm 0.2704 (-0.73z)| lr 9.43e-05 | 322.77 ms | 52.3% bf16 MFU | 1622692 tok/s
step 14669/19560 | loss 3.397589 (+1.53z)| norm 0.3009 (+0.84z)| lr 9.42e-05 | 323.85 ms | 52.1% bf16 MFU | 1622502 tok/s
step 14670/19560 | loss 3.334118 (+0.12z)| norm 0.2749 (-0.51z)| lr 9.42e-05 | 323.79 ms | 52.1% bf16 MFU | 1622338 tok/s
step 14671/19560 | loss 3.309590 (-0.41z)| norm 0.2774 (-0.37z)| lr 9.41e-05 | 323.07 ms | 52.2% bf16 MFU | 1622361 tok/s
step 14672/19560 | loss 3.295210 (-0.73z)| norm 0.2741 (-0.55z)| lr 9.41e-05 | 322.04 ms | 52.4% bf16 MFU | 1622644 tok/s
step 14673/19560 | loss 3.261425 (-1.47z)| norm 0.2666 (-0.94z)| lr 9.41e-05 | 323.08 ms | 52.2% bf16 MFU | 1622651 tok/s
step 14674/19560 | loss 3.296983 (-0.66z)| norm 0.2802 (-0.22z)| lr 9.40e-05 | 322.28 ms | 52.4% bf16 MFU | 1622858 tok/s
step 14675/19560 | loss 3.268457 (-1.28z)| norm 0.2908 (+0.33z)| lr 9.40e-05 | 323.88 ms | 52.1% bf16 MFU | 1622654 tok/s
step 14676/19560 | loss 3.319182 (-0.15z)| norm 0.2823 (-0.11z)| lr 9.40e-05 | 322.77 ms | 52.3% bf16 MFU | 1622738 tok/s
step 14677/19560 | loss 3.317216 (-0.20z)| norm 0.2758 (-0.43z)| lr 9.39e-05 | 322.52 ms | 52.3% bf16 MFU | 1622882 tok/s
step 14678/19560 | loss 3.353005 (+0.61z)| norm 0.2680 (-0.84z)| lr 9.39e-05 | 323.23 ms | 52.2% bf16 MFU | 1622838 tok/s
step 14679/19560 | loss 3.363497 (+0.83z)| norm 0.2601 (-1.23z)| lr 9.39e-05 | 322.79 ms | 52.3% bf16 MFU | 1622907 tok/s
step 14680/19560 | loss 3.318771 (-0.17z)| norm 0.2729 (-0.57z)| lr 9.38e-05 | 323.67 ms | 52.1% bf16 MFU | 1622754 tok/s
step 14681/19560 | loss 3.299824 (-0.60z)| norm 0.2605 (-1.20z)| lr 9.38e-05 | 323.07 ms | 52.2% bf16 MFU | 1622757 tok/s
step 14682/19560 | loss 3.260604 (-1.47z)| norm 0.2817 (-0.09z)| lr 9.37e-05 | 323.02 ms | 52.2% bf16 MFU | 1622773 tok/s
step 14683/19560 | loss 3.277892 (-1.07z)| norm 0.2740 (-0.50z)| lr 9.37e-05 | 322.97 ms | 52.3% bf16 MFU | 1622802 tok/s
step 14684/19560 | loss 3.329655 (+0.08z)| norm 0.2797 (-0.20z)| lr 9.37e-05 | 323.48 ms | 52.2% bf16 MFU | 1622701 tok/s
step 14685/19560 | loss 3.341185 (+0.34z)| norm 0.2943 (+0.56z)| lr 9.36e-05 | 322.98 ms | 52.3% bf16 MFU | 1622730 tok/s
step 14686/19560 | loss 3.218753 (-2.32z)| norm 0.2854 (+0.08z)| lr 9.36e-05 | 322.78 ms | 52.3% bf16 MFU | 1622809 tok/s
step 14687/19560 | loss 3.448954 (+2.60z)| norm 0.3147 (+1.60z)| lr 9.36e-05 | 322.80 ms | 52.3% bf16 MFU | 1622879 tok/s
step 14688/19560 | loss 3.354295 (+0.59z)| norm 0.2822 (-0.10z)| lr 9.35e-05 | 323.14 ms | 52.2% bf16 MFU | 1622859 tok/s
step 14689/19560 | loss 3.255864 (-1.48z)| norm 0.2903 (+0.32z)| lr 9.35e-05 | 323.15 ms | 52.2% bf16 MFU | 1622837 tok/s
step 14690/19560 | loss 3.331456 (+0.12z)| norm 0.2642 (-1.05z)| lr 9.35e-05 | 323.25 ms | 52.2% bf16 MFU | 1622793 tok/s
step 14691/19560 | loss 3.302756 (-0.49z)| norm 0.2754 (-0.47z)| lr 9.34e-05 | 322.49 ms | 52.3% bf16 MFU | 1622940 tok/s
step 14692/19560 | loss 3.316894 (-0.18z)| norm 0.2810 (-0.19z)| lr 9.34e-05 | 322.99 ms | 52.3% bf16 MFU | 1622956 tok/s
step 14693/19560 | loss 3.320228 (-0.11z)| norm 0.2633 (-1.12z)| lr 9.33e-05 | 323.16 ms | 52.2% bf16 MFU | 1622927 tok/s
step 14694/19560 | loss 3.330529 (+0.09z)| norm 0.2838 (-0.04z)| lr 9.33e-05 | 323.29 ms | 52.2% bf16 MFU | 1622866 tok/s
step 14695/19560 | loss 3.255223 (-1.52z)| norm 0.2667 (-0.95z)| lr 9.33e-05 | 322.77 ms | 52.3% bf16 MFU | 1622941 tok/s
step 14696/19560 | loss 3.299322 (-0.58z)| norm 0.2696 (-0.79z)| lr 9.32e-05 | 322.63 ms | 52.3% bf16 MFU | 1623045 tok/s
step 14697/19560 | loss 3.274226 (-1.11z)| norm 0.2803 (-0.23z)| lr 9.32e-05 | 322.83 ms | 52.3% bf16 MFU | 1623094 tok/s
step 14698/19560 | loss 3.358637 (+0.70z)| norm 0.2978 (+0.70z)| lr 9.32e-05 | 323.09 ms | 52.2% bf16 MFU | 1623075 tok/s
step 14699/19560 | loss 3.275182 (-1.08z)| norm 0.2547 (-1.58z)| lr 9.31e-05 | 323.23 ms | 52.2% bf16 MFU | 1623022 tok/s
step 14700/19560 | loss 3.314283 (-0.24z)| norm 0.3062 (+1.12z)| lr 9.31e-05 | 323.07 ms | 52.2% bf16 MFU | 1623012 tok/s
step 14701/19560 | loss 3.303664 (-0.45z)| norm 0.2961 (+0.58z)| lr 9.31e-05 | 322.13 ms | 52.4% bf16 MFU | 1623239 tok/s
step 14702/19560 | loss 3.315542 (-0.21z)| norm 0.2918 (+0.35z)| lr 9.30e-05 | 323.38 ms | 52.2% bf16 MFU | 1623141 tok/s
step 14703/19560 | loss 3.383028 (+1.25z)| norm 0.3296 (+2.28z)| lr 9.30e-05 | 323.22 ms | 52.2% bf16 MFU | 1623087 tok/s
step 14704/19560 | loss 3.268076 (-1.22z)| norm 0.2751 (-0.52z)| lr 9.29e-05 | 323.29 ms | 52.2% bf16 MFU | 1623020 tok/s
step 14705/19560 | loss 3.314040 (-0.23z)| norm 0.2893 (+0.21z)| lr 9.29e-05 | 323.37 ms | 52.2% bf16 MFU | 1622934 tok/s
step 14706/19560 | loss 3.316637 (-0.18z)| norm 0.2907 (+0.27z)| lr 9.29e-05 | 322.83 ms | 52.3% bf16 MFU | 1622990 tok/s
step 14707/19560 | loss 3.332049 (+0.14z)| norm 0.2998 (+0.75z)| lr 9.28e-05 | 322.96 ms | 52.3% bf16 MFU | 1623010 tok/s
step 14708/19560 | loss 3.326202 (+0.01z)| norm 0.3240 (+1.97z)| lr 9.28e-05 | 323.56 ms | 52.2% bf16 MFU | 1622878 tok/s
step 14709/19560 | loss 3.315637 (-0.20z)| norm 0.2740 (-0.59z)| lr 9.28e-05 | 322.97 ms | 52.3% bf16 MFU | 1622901 tok/s
step 14710/19560 | loss 3.315759 (-0.20z)| norm 0.2922 (+0.34z)| lr 9.27e-05 | 322.54 ms | 52.3% bf16 MFU | 1623030 tok/s
step 14711/19560 | loss 3.378990 (+1.17z)| norm 0.2849 (-0.04z)| lr 9.27e-05 | 322.82 ms | 52.3% bf16 MFU | 1623083 tok/s
step 14712/19560 | loss 3.270624 (-1.17z)| norm 0.2861 (+0.01z)| lr 9.27e-05 | 322.60 ms | 52.3% bf16 MFU | 1623187 tok/s
step 14713/19560 | loss 3.309302 (-0.33z)| norm 0.3101 (+1.24z)| lr 9.26e-05 | 322.83 ms | 52.3% bf16 MFU | 1623230 tok/s
step 14714/19560 | loss 3.243169 (-1.72z)| norm 0.2966 (+0.53z)| lr 9.26e-05 | 323.01 ms | 52.2% bf16 MFU | 1623225 tok/s
step 14715/19560 | loss 3.368210 (+0.96z)| norm 0.2899 (+0.18z)| lr 9.25e-05 | 322.51 ms | 52.3% bf16 MFU | 1623346 tok/s
step 14716/19560 | loss 3.332047 (+0.18z)| norm 0.3135 (+1.39z)| lr 9.25e-05 | 322.27 ms | 52.4% bf16 MFU | 1623522 tok/s
step 14717/19560 | loss 3.338883 (+0.32z)| norm 0.2730 (-0.70z)| lr 9.25e-05 | 322.89 ms | 52.3% bf16 MFU | 1623532 tok/s
step 14718/19560 | loss 3.312914 (-0.24z)| norm 0.2699 (-0.85z)| lr 9.24e-05 | 323.35 ms | 52.2% bf16 MFU | 1623426 tok/s
step 14719/19560 | loss 3.318918 (-0.11z)| norm 0.2842 (-0.10z)| lr 9.24e-05 | 322.36 ms | 52.4% bf16 MFU | 1623575 tok/s
step 14720/19560 | loss 3.341061 (+0.36z)| norm 0.2897 (+0.18z)| lr 9.24e-05 | 323.38 ms | 52.2% bf16 MFU | 1623460 tok/s
step 14721/19560 | loss 3.262737 (-1.30z)| norm 0.3106 (+1.27z)| lr 9.23e-05 | 322.83 ms | 52.3% bf16 MFU | 1623490 tok/s
step 14722/19560 | loss 3.301060 (-0.47z)| norm 0.3336 (+2.40z)| lr 9.23e-05 | 322.64 ms | 52.3% bf16 MFU | 1623566 tok/s
step 14723/19560 | loss 3.278948 (-0.94z)| norm 0.3056 (+0.96z)| lr 9.23e-05 | 323.21 ms | 52.2% bf16 MFU | 1623493 tok/s
step 14724/19560 | loss 3.297367 (-0.54z)| norm 0.3001 (+0.68z)| lr 9.22e-05 | 322.86 ms | 52.3% bf16 MFU | 1623514 tok/s
step 14725/19560 | loss 3.347916 (+0.54z)| norm 0.3110 (+1.23z)| lr 9.22e-05 | 323.15 ms | 52.2% bf16 MFU | 1623461 tok/s
step 14726/19560 | loss 3.291809 (-0.65z)| norm 0.2898 (+0.15z)| lr 9.22e-05 | 322.69 ms | 52.3% bf16 MFU | 1623524 tok/s
step 14727/19560 | loss 3.370411 (+1.01z)| norm 0.2889 (+0.10z)| lr 9.21e-05 | 322.64 ms | 52.3% bf16 MFU | 1623598 tok/s
step 14728/19560 | loss 3.278473 (-0.93z)| norm 0.3160 (+1.45z)| lr 9.21e-05 | 323.56 ms | 52.2% bf16 MFU | 1623437 tok/s
step 14729/19560 | loss 3.320635 (-0.05z)| norm 0.2870 (-0.02z)| lr 9.20e-05 | 322.69 ms | 52.3% bf16 MFU | 1623502 tok/s
step 14730/19560 | loss 3.347973 (+0.53z)| norm 0.2997 (+0.62z)| lr 9.20e-05 | 322.14 ms | 52.4% bf16 MFU | 1623704 tok/s
step 14731/19560 | loss 3.270536 (-1.11z)| norm 0.2975 (+0.54z)| lr 9.20e-05 | 322.99 ms | 52.3% bf16 MFU | 1623679 tok/s
step 14732/19560 | loss 3.265910 (-1.20z)| norm 0.3001 (+0.67z)| lr 9.19e-05 | 322.60 ms | 52.3% bf16 MFU | 1623754 tok/s
step 14733/19560 | loss 3.416976 (+1.95z)| norm 0.2851 (-0.12z)| lr 9.19e-05 | 323.18 ms | 52.2% bf16 MFU | 1623679 tok/s
step 14734/19560 | loss 3.371188 (+0.98z)| norm 0.2938 (+0.37z)| lr 9.19e-05 | 322.75 ms | 52.3% bf16 MFU | 1623717 tok/s
step 14735/19560 | loss 3.275900 (-0.99z)| norm 0.2823 (-0.24z)| lr 9.18e-05 | 323.12 ms | 52.2% bf16 MFU | 1623661 tok/s
step 14736/19560 | loss 3.295782 (-0.57z)| norm 0.2880 (+0.10z)| lr 9.18e-05 | 322.54 ms | 52.3% bf16 MFU | 1623751 tok/s
step 14737/19560 | loss 3.314758 (-0.18z)| norm 0.2751 (-0.64z)| lr 9.18e-05 | 322.43 ms | 52.3% bf16 MFU | 1623866 tok/s
step 14738/19560 | loss 3.301896 (-0.44z)| norm 0.2782 (-0.45z)| lr 9.17e-05 | 323.01 ms | 52.2% bf16 MFU | 1623829 tok/s
step 14739/19560 | loss 3.317125 (-0.11z)| norm 0.2636 (-1.29z)| lr 9.17e-05 | 323.16 ms | 52.2% bf16 MFU | 1623756 tok/s
step 14740/19560 | loss 3.366692 (+0.93z)| norm 0.2869 (+0.08z)| lr 9.16e-05 | 322.72 ms | 52.3% bf16 MFU | 1623797 tok/s
step 14741/19560 | loss 3.327028 (+0.10z)| norm 0.2796 (-0.35z)| lr 9.16e-05 | 323.31 ms | 52.2% bf16 MFU | 1623689 tok/s
step 14742/19560 | loss 3.351406 (+0.60z)| norm 0.2774 (-0.49z)| lr 9.16e-05 | 322.44 ms | 52.3% bf16 MFU | 1623805 tok/s
step 14743/19560 | loss 3.350744 (+0.62z)| norm 0.2627 (-1.35z)| lr 9.15e-05 | 322.72 ms | 52.3% bf16 MFU | 1623845 tok/s
step 14744/19560 | loss 3.355334 (+0.71z)| norm 0.2706 (-0.87z)| lr 9.15e-05 | 322.66 ms | 52.3% bf16 MFU | 1623897 tok/s
step 14745/19560 | loss 3.393557 (+1.51z)| norm 0.2704 (-0.87z)| lr 9.15e-05 | 323.25 ms | 52.2% bf16 MFU | 1623800 tok/s
step 14746/19560 | loss 3.330378 (+0.15z)| norm 0.2810 (-0.24z)| lr 9.14e-05 | 322.56 ms | 52.3% bf16 MFU | 1623878 tok/s
step 14747/19560 | loss 3.306025 (-0.38z)| norm 0.2596 (-1.49z)| lr 9.14e-05 | 322.95 ms | 52.3% bf16 MFU | 1623857 tok/s
step 14748/19560 | loss 3.297245 (-0.57z)| norm 0.2769 (-0.46z)| lr 9.14e-05 | 322.70 ms | 52.3% bf16 MFU | 1623897 tok/s
step 14749/19560 | loss 3.352581 (+0.62z)| norm 0.2870 (+0.12z)| lr 9.13e-05 | 322.76 ms | 52.3% bf16 MFU | 1623923 tok/s
step 14750/19560 | loss 3.274140 (-1.05z)| norm 0.2502 (-2.02z)| lr 9.13e-05 | 322.75 ms | 52.3% bf16 MFU | 1623948 tok/s
val loss 3.306326
ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 2968/10042 = 0.295559
step 14751/19560 | loss 3.315908 (-0.16z)| norm 0.2799 (-0.28z)| lr 9.13e-05 | 322.57 ms | 52.3% bf16 MFU | 1624019 tok/s
step 14752/19560 | loss 3.355447 (+0.70z)| norm 0.2820 (-0.15z)| lr 9.12e-05 | 323.24 ms | 52.2% bf16 MFU | 1623916 tok/s
step 14753/19560 | loss 3.268023 (-1.18z)| norm 0.2852 (+0.04z)| lr 9.12e-05 | 323.23 ms | 52.2% bf16 MFU | 1623821 tok/s
step 14754/19560 | loss 3.317389 (-0.12z)| norm 0.2656 (-1.12z)| lr 9.11e-05 | 322.91 ms | 52.3% bf16 MFU | 1623812 tok/s
step 14755/19560 | loss 3.273330 (-1.07z)| norm 0.2697 (-0.87z)| lr 9.11e-05 | 322.57 ms | 52.3% bf16 MFU | 1623889 tok/s
step 14756/19560 | loss 3.287201 (-0.77z)| norm 0.2724 (-0.71z)| lr 9.11e-05 | 322.72 ms | 52.3% bf16 MFU | 1623923 tok/s
step 14757/19560 | loss 3.265180 (-1.23z)| norm 0.2946 (+0.58z)| lr 9.10e-05 | 323.50 ms | 52.2% bf16 MFU | 1623762 tok/s
step 14758/19560 | loss 3.371528 (+1.04z)| norm 0.2878 (+0.17z)| lr 9.10e-05 | 322.73 ms | 52.3% bf16 MFU | 1623802 tok/s
step 14759/19560 | loss 3.352093 (+0.61z)| norm 0.2684 (-0.97z)| lr 9.10e-05 | 322.09 ms | 52.4% bf16 MFU | 1624001 tok/s
step 14760/19560 | loss 3.302168 (-0.46z)| norm 0.2828 (-0.12z)| lr 9.09e-05 | 323.01 ms | 52.2% bf16 MFU | 1623957 tok/s
step 14761/19560 | loss 3.322095 (-0.01z)| norm 0.2915 (+0.40z)| lr 9.09e-05 | 322.86 ms | 52.3% bf16 MFU | 1623953 tok/s
step 14762/19560 | loss 3.268325 (-1.18z)| norm 0.2710 (-0.83z)| lr 9.09e-05 | 323.06 ms | 52.2% bf16 MFU | 1623900 tok/s
step 14763/19560 | loss 3.307783 (-0.30z)| norm 0.2722 (-0.76z)| lr 9.08e-05 | 323.35 ms | 52.2% bf16 MFU | 1623777 tok/s
step 14764/19560 | loss 3.321353 (+0.03z)| norm 0.2862 (+0.07z)| lr 9.08e-05 | 322.60 ms | 52.3% bf16 MFU | 1623847 tok/s
step 14765/19560 | loss 3.322289 (+0.06z)| norm 0.2806 (-0.27z)| lr 9.07e-05 | 323.12 ms | 52.2% bf16 MFU | 1623784 tok/s
step 14766/19560 | loss 3.361911 (+0.97z)| norm 0.2929 (+0.46z)| lr 9.07e-05 | 323.02 ms | 52.2% bf16 MFU | 1623750 tok/s
step 14767/19560 | loss 3.368946 (+1.11z)| norm 0.2617 (-1.41z)| lr 9.07e-05 | 322.55 ms | 52.3% bf16 MFU | 1623835 tok/s
step 14768/19560 | loss 3.326156 (+0.13z)| norm 0.3015 (+0.97z)| lr 9.06e-05 | 323.09 ms | 52.2% bf16 MFU | 1623779 tok/s
step 14769/19560 | loss 3.339064 (+0.41z)| norm 0.2768 (-0.52z)| lr 9.06e-05 | 323.05 ms | 52.2% bf16 MFU | 1623738 tok/s
step 14770/19560 | loss 3.367064 (+1.05z)| norm 0.2710 (-0.87z)| lr 9.06e-05 | 323.23 ms | 52.2% bf16 MFU | 1623654 tok/s
step 14771/19560 | loss 3.356274 (+0.79z)| norm 0.3008 (+0.91z)| lr 9.05e-05 | 323.20 ms | 52.2% bf16 MFU | 1623578 tok/s
step 14772/19560 | loss 3.481865 (+3.51z)| norm 0.2801 (-0.33z)| lr 9.05e-05 | 322.76 ms | 52.3% bf16 MFU | 1623618 tok/s
step 14773/19560 | loss 3.438640 (+2.49z)| norm 0.2975 (+0.72z)| lr 9.05e-05 | 323.19 ms | 52.2% bf16 MFU | 1623549 tok/s
step 14774/19560 | loss 3.298642 (-0.55z)| norm 0.2951 (+0.57z)| lr 9.04e-05 | 322.83 ms | 52.3% bf16 MFU | 1623573 tok/s
step 14775/19560 | loss 3.299487 (-0.53z)| norm 0.2826 (-0.18z)| lr 9.04e-05 | 322.92 ms | 52.3% bf16 MFU | 1623573 tok/s
step 14776/19560 | loss 3.445737 (+2.61z)| norm 0.4006 (+5.89z)| lr 9.04e-05 | 322.39 ms | 52.4% bf16 MFU | 1623707 tok/s
step 14777/19560 | loss 3.349607 (+0.54z)| norm 0.2860 (-0.02z)| lr 9.03e-05 | 323.30 ms | 52.2% bf16 MFU | 1623606 tok/s
step 14778/19560 | loss 3.312284 (-0.25z)| norm 0.2911 (+0.24z)| lr 9.03e-05 | 322.60 ms | 52.3% bf16 MFU | 1623686 tok/s
step 14779/19560 | loss 3.324924 (+0.03z)| norm 0.2732 (-0.68z)| lr 9.02e-05 | 322.84 ms | 52.3% bf16 MFU | 1623701 tok/s
step 14780/19560 | loss 3.320654 (-0.06z)| norm 0.2928 (+0.34z)| lr 9.02e-05 | 323.19 ms | 52.2% bf16 MFU | 1623626 tok/s
step 14781/19560 | loss 3.306444 (-0.36z)| norm 0.2824 (-0.20z)| lr 9.02e-05 | 323.17 ms | 52.2% bf16 MFU | 1623562 tok/s
step 14782/19560 | loss 3.254657 (-1.50z)| norm 0.2593 (-1.38z)| lr 9.01e-05 | 323.11 ms | 52.2% bf16 MFU | 1623515 tok/s
step 14783/19560 | loss 3.322542 (-0.00z)| norm 0.2850 (-0.03z)| lr 9.01e-05 | 322.50 ms | 52.3% bf16 MFU | 1623625 tok/s
step 14784/19560 | loss 3.338822 (+0.40z)| norm 0.2694 (-0.85z)| lr 9.01e-05 | 322.90 ms | 52.3% bf16 MFU | 1623628 tok/s
step 14785/19560 | loss 3.331387 (+0.23z)| norm 0.3297 (+2.40z)| lr 9.00e-05 | 323.16 ms | 52.2% bf16 MFU | 1623566 tok/s
step 14786/19560 | loss 3.336391 (+0.34z)| norm 0.2732 (-0.64z)| lr 9.00e-05 | 322.31 ms | 52.4% bf16 MFU | 1623721 tok/s
step 14787/19560 | loss 3.357558 (+0.83z)| norm 0.2889 (+0.21z)| lr 9.00e-05 | 323.01 ms | 52.3% bf16 MFU | 1623693 tok/s
step 14788/19560 | loss 3.309794 (-0.28z)| norm 0.2844 (-0.03z)| lr 8.99e-05 | 323.32 ms | 52.2% bf16 MFU | 1623587 tok/s
step 14789/19560 | loss 3.297402 (-0.56z)| norm 0.2864 (+0.07z)| lr 8.99e-05 | 322.68 ms | 52.3% bf16 MFU | 1623646 tok/s
step 14790/19560 | loss 3.266138 (-1.27z)| norm 0.3149 (+1.57z)| lr 8.99e-05 | 322.95 ms | 52.3% bf16 MFU | 1623636 tok/s
step 14791/19560 | loss 3.399016 (+1.78z)| norm 0.3003 (+0.78z)| lr 8.98e-05 | 323.06 ms | 52.2% bf16 MFU | 1623598 tok/s
step 14792/19560 | loss 3.270177 (-1.17z)| norm 0.2974 (+0.62z)| lr 8.98e-05 | 322.63 ms | 52.3% bf16 MFU | 1623670 tok/s
step 14793/19560 | loss 3.369782 (+1.09z)| norm 0.2821 (-0.21z)| lr 8.97e-05 | 323.00 ms | 52.3% bf16 MFU | 1623646 tok/s
step 14794/19560 | loss 3.276211 (-1.02z)| norm 0.2892 (+0.18z)| lr 8.97e-05 | 323.00 ms | 52.3% bf16 MFU | 1623622 tok/s
step 14795/19560 | loss 3.366523 (+1.03z)| norm 0.3126 (+1.41z)| lr 8.97e-05 | 323.58 ms | 52.2% bf16 MFU | 1623456 tok/s
step 14796/19560 | loss 3.245254 (-1.72z)| norm 0.2686 (-0.93z)| lr 8.96e-05 | 322.65 ms | 52.3% bf16 MFU | 1623530 tok/s
step 14797/19560 | loss 3.322327 (+0.04z)| norm 0.2718 (-0.75z)| lr 8.96e-05 | 322.71 ms | 52.3% bf16 MFU | 1623587 tok/s
step 14798/19560 | loss 3.319678 (-0.02z)| norm 0.3340 (+2.49z)| lr 8.96e-05 | 322.94 ms | 52.3% bf16 MFU | 1623582 tok/s
step 14799/19560 | loss 3.366070 (+1.03z)| norm 0.3003 (+0.72z)| lr 8.95e-05 | 323.16 ms | 52.2% bf16 MFU | 1623522 tok/s
step 14800/19560 | loss 3.382704 (+1.38z)| norm 0.3406 (+2.71z)| lr 8.95e-05 | 323.47 ms | 52.2% bf16 MFU | 1623388 tok/s
step 14801/19560 | loss 3.371898 (+1.12z)| norm 0.3146 (+1.37z)| lr 8.95e-05 | 322.77 ms | 52.3% bf16 MFU | 1623435 tok/s
step 14802/19560 | loss 3.284118 (-0.87z)| norm 0.2716 (-0.79z)| lr 8.94e-05 | 322.97 ms | 52.3% bf16 MFU | 1623428 tok/s
step 14803/19560 | loss 3.270307 (-1.19z)| norm 0.3056 (+0.91z)| lr 8.94e-05 | 322.98 ms | 52.3% bf16 MFU | 1623422 tok/s
step 14804/19560 | loss 3.354450 (+0.72z)| norm 0.2796 (-0.39z)| lr 8.94e-05 | 323.05 ms | 52.2% bf16 MFU | 1623398 tok/s
step 14805/19560 | loss 3.260623 (-1.39z)| norm 0.2689 (-0.93z)| lr 8.93e-05 | 323.47 ms | 52.2% bf16 MFU | 1623269 tok/s
step 14806/19560 | loss 3.320391 (-0.04z)| norm 0.2893 (+0.09z)| lr 8.93e-05 | 322.65 ms | 52.3% bf16 MFU | 1623352 tok/s
step 14807/19560 | loss 3.292674 (-0.65z)| norm 0.2651 (-1.13z)| lr 8.93e-05 | 322.55 ms | 52.3% bf16 MFU | 1623457 tok/s
step 14808/19560 | loss 3.319310 (-0.05z)| norm 0.2637 (-1.20z)| lr 8.92e-05 | 323.14 ms | 52.2% bf16 MFU | 1623409 tok/s
step 14809/19560 | loss 3.238149 (-1.85z)| norm 0.2737 (-0.70z)| lr 8.92e-05 | 323.18 ms | 52.2% bf16 MFU | 1623352 tok/s
step 14810/19560 | loss 3.299793 (-0.48z)| norm 0.2712 (-0.82z)| lr 8.91e-05 | 323.44 ms | 52.2% bf16 MFU | 1623233 tok/s
step 14811/19560 | loss 3.225985 (-2.10z)| norm 0.2567 (-1.53z)| lr 8.91e-05 | 323.31 ms | 52.2% bf16 MFU | 1623151 tok/s
step 14812/19560 | loss 3.327724 (+0.15z)| norm 0.2783 (-0.45z)| lr 8.91e-05 | 322.70 ms | 52.3% bf16 MFU | 1623227 tok/s
step 14813/19560 | loss 3.351269 (+0.67z)| norm 0.2744 (-0.64z)| lr 8.90e-05 | 323.03 ms | 52.2% bf16 MFU | 1623218 tok/s
step 14814/19560 | loss 3.248946 (-1.61z)| norm 0.2522 (-1.72z)| lr 8.90e-05 | 322.69 ms | 52.3% bf16 MFU | 1623295 tok/s
step 14815/19560 | loss 3.259383 (-1.38z)| norm 0.2942 (+0.37z)| lr 8.90e-05 | 323.04 ms | 52.2% bf16 MFU | 1623279 tok/s
step 14816/19560 | loss 3.325563 (+0.14z)| norm 0.2557 (-1.52z)| lr 8.89e-05 | 323.54 ms | 52.2% bf16 MFU | 1623140 tok/s
step 14817/19560 | loss 3.322553 (+0.06z)| norm 0.2639 (-1.10z)| lr 8.89e-05 | 322.97 ms | 52.3% bf16 MFU | 1623150 tok/s
step 14818/19560 | loss 3.313095 (-0.16z)| norm 0.2867 (+0.01z)| lr 8.89e-05 | 323.01 ms | 52.3% bf16 MFU | 1623150 tok/s
step 14819/19560 | loss 3.318076 (-0.05z)| norm 0.2589 (-1.34z)| lr 8.88e-05 | 323.22 ms | 52.2% bf16 MFU | 1623096 tok/s
step 14820/19560 | loss 3.306306 (-0.32z)| norm 0.2785 (-0.39z)| lr 8.88e-05 | 323.07 ms | 52.2% bf16 MFU | 1623083 tok/s
step 14821/19560 | loss 3.373919 (+1.23z)| norm 0.2638 (-1.11z)| lr 8.88e-05 | 322.93 ms | 52.3% bf16 MFU | 1623105 tok/s
step 14822/19560 | loss 3.327048 (+0.15z)| norm 0.2639 (-1.09z)| lr 8.87e-05 | 322.86 ms | 52.3% bf16 MFU | 1623143 tok/s
step 14823/19560 | loss 3.305273 (-0.36z)| norm 0.2824 (-0.19z)| lr 8.87e-05 | 322.67 ms | 52.3% bf16 MFU | 1623229 tok/s
step 14824/19560 | loss 3.312473 (-0.19z)| norm 0.2720 (-0.70z)| lr 8.86e-05 | 323.14 ms | 52.2% bf16 MFU | 1623190 tok/s
step 14825/19560 | loss 3.363021 (+0.97z)| norm 0.2930 (+0.32z)| lr 8.86e-05 | 322.51 ms | 52.3% bf16 MFU | 1623314 tok/s
step 14826/19560 | loss 3.367828 (+1.07z)| norm 0.2953 (+0.43z)| lr 8.86e-05 | 323.24 ms | 52.2% bf16 MFU | 1623247 tok/s
step 14827/19560 | loss 3.343507 (+0.50z)| norm 0.2507 (-1.74z)| lr 8.85e-05 | 322.96 ms | 52.3% bf16 MFU | 1623253 tok/s
step 14828/19560 | loss 3.332286 (+0.23z)| norm 0.2687 (-0.85z)| lr 8.85e-05 | 323.11 ms | 52.2% bf16 MFU | 1623222 tok/s
step 14829/19560 | loss 3.303221 (-0.44z)| norm 0.2933 (+0.35z)| lr 8.85e-05 | 322.60 ms | 52.3% bf16 MFU | 1623321 tok/s
step 14830/19560 | loss 3.279265 (-0.99z)| norm 0.2706 (-0.75z)| lr 8.84e-05 | 323.16 ms | 52.2% bf16 MFU | 1623273 tok/s
step 14831/19560 | loss 3.283644 (-0.88z)| norm 0.2631 (-1.10z)| lr 8.84e-05 | 323.12 ms | 52.2% bf16 MFU | 1623238 tok/s
step 14832/19560 | loss 3.277079 (-1.03z)| norm 0.2732 (-0.60z)| lr 8.84e-05 | 322.61 ms | 52.3% bf16 MFU | 1623332 tok/s
step 14833/19560 | loss 3.336979 (+0.36z)| norm 0.2688 (-0.81z)| lr 8.83e-05 | 322.95 ms | 52.3% bf16 MFU | 1623338 tok/s
step 14834/19560 | loss 3.300973 (-0.48z)| norm 0.2760 (-0.45z)| lr 8.83e-05 | 322.60 ms | 52.3% bf16 MFU | 1623431 tok/s
step 14835/19560 | loss 3.293193 (-0.65z)| norm 0.2571 (-1.36z)| lr 8.83e-05 | 323.21 ms | 52.2% bf16 MFU | 1623366 tok/s
step 14836/19560 | loss 3.585188 (+5.37z)| norm 0.3555 (+3.35z)| lr 8.82e-05 | 323.03 ms | 52.2% bf16 MFU | 1623348 tok/s
step 14837/19560 | loss 3.306597 (-0.34z)| norm 0.2769 (-0.39z)| lr 8.82e-05 | 322.88 ms | 52.3% bf16 MFU | 1623369 tok/s
step 14838/19560 | loss 3.357816 (+0.71z)| norm 0.2799 (-0.24z)| lr 8.82e-05 | 322.71 ms | 52.3% bf16 MFU | 1623432 tok/s
step 14839/19560 | loss 3.307313 (-0.32z)| norm 0.2691 (-0.75z)| lr 8.81e-05 | 323.35 ms | 52.2% bf16 MFU | 1623331 tok/s
step 14840/19560 | loss 3.345808 (+0.46z)| norm 0.3080 (+1.09z)| lr 8.81e-05 | 323.04 ms | 52.2% bf16 MFU | 1623314 tok/s
step 14841/19560 | loss 3.293759 (-0.61z)| norm 0.2762 (-0.40z)| lr 8.80e-05 | 323.26 ms | 52.2% bf16 MFU | 1623242 tok/s
step 14842/19560 | loss 3.319727 (-0.09z)| norm 0.2969 (+0.57z)| lr 8.80e-05 | 323.68 ms | 52.1% bf16 MFU | 1623069 tok/s
step 14843/19560 | loss 3.328713 (+0.11z)| norm 0.2924 (+0.36z)| lr 8.80e-05 | 323.07 ms | 52.2% bf16 MFU | 1623058 tok/s
step 14844/19560 | loss 3.261623 (-1.27z)| norm 0.2664 (-0.86z)| lr 8.79e-05 | 323.40 ms | 52.2% bf16 MFU | 1622964 tok/s
step 14845/19560 | loss 3.279957 (-0.88z)| norm 0.2718 (-0.60z)| lr 8.79e-05 | 323.77 ms | 52.1% bf16 MFU | 1622782 tok/s
step 14846/19560 | loss 3.294071 (-0.59z)| norm 0.2717 (-0.61z)| lr 8.79e-05 | 323.09 ms | 52.2% bf16 MFU | 1622779 tok/s
step 14847/19560 | loss 3.327827 (+0.11z)| norm 0.2688 (-0.74z)| lr 8.78e-05 | 323.62 ms | 52.2% bf16 MFU | 1622643 tok/s
step 14848/19560 | loss 3.338100 (+0.32z)| norm 0.2682 (-0.76z)| lr 8.78e-05 | 323.48 ms | 52.2% bf16 MFU | 1622550 tok/s
step 14849/19560 | loss 3.367931 (+0.93z)| norm 0.3371 (+2.46z)| lr 8.78e-05 | 324.02 ms | 52.1% bf16 MFU | 1622327 tok/s
step 14850/19560 | loss 3.341817 (+0.38z)| norm 0.2834 (-0.03z)| lr 8.77e-05 | 323.41 ms | 52.2% bf16 MFU | 1622268 tok/s
step 14851/19560 | loss 3.319219 (-0.10z)| norm 0.2664 (-0.82z)| lr 8.77e-05 | 323.63 ms | 52.1% bf16 MFU | 1622155 tok/s
step 14852/19560 | loss 3.322579 (-0.03z)| norm 0.2671 (-0.78z)| lr 8.77e-05 | 322.88 ms | 52.3% bf16 MFU | 1622236 tok/s
step 14853/19560 | loss 3.225204 (-2.01z)| norm 0.2838 (+0.03z)| lr 8.76e-05 | 322.86 ms | 52.3% bf16 MFU | 1622319 tok/s
step 14854/19560 | loss 3.291073 (-0.66z)| norm 0.2632 (-0.95z)| lr 8.76e-05 | 323.38 ms | 52.2% bf16 MFU | 1622266 tok/s
step 14855/19560 | loss 3.327419 (+0.10z)| norm 0.2684 (-0.69z)| lr 8.76e-05 | 323.87 ms | 52.1% bf16 MFU | 1622095 tok/s
step 14856/19560 | loss 3.374954 (+1.06z)| norm 0.2565 (-1.24z)| lr 8.75e-05 | 323.95 ms | 52.1% bf16 MFU | 1621912 tok/s
step 14857/19560 | loss 3.288100 (-0.72z)| norm 0.2822 (-0.00z)| lr 8.75e-05 | 323.29 ms | 52.2% bf16 MFU | 1621902 tok/s
step 14858/19560 | loss 3.324928 (+0.04z)| norm 0.2573 (-1.18z)| lr 8.74e-05 | 323.13 ms | 52.2% bf16 MFU | 1621934 tok/s
step 14859/19560 | loss 3.326027 (+0.05z)| norm 0.2660 (-0.75z)| lr 8.74e-05 | 323.66 ms | 52.1% bf16 MFU | 1621831 tok/s
step 14860/19560 | loss 3.296196 (-0.57z)| norm 0.2490 (-1.54z)| lr 8.74e-05 | 323.60 ms | 52.2% bf16 MFU | 1621748 tok/s
step 14861/19560 | loss 3.421993 (+2.03z)| norm 0.2708 (-0.49z)| lr 8.73e-05 | 323.68 ms | 52.1% bf16 MFU | 1621649 tok/s
step 14862/19560 | loss 3.348094 (+0.51z)| norm 0.2840 (+0.14z)| lr 8.73e-05 | 323.47 ms | 52.2% bf16 MFU | 1621607 tok/s
step 14863/19560 | loss 3.369999 (+0.95z)| norm 0.2633 (-0.84z)| lr 8.73e-05 | 322.59 ms | 52.3% bf16 MFU | 1621788 tok/s
step 14864/19560 | loss 3.351243 (+0.55z)| norm 0.2940 (+0.61z)| lr 8.72e-05 | 323.58 ms | 52.2% bf16 MFU | 1621712 tok/s
step 14865/19560 | loss 3.275653 (-1.01z)| norm 0.2664 (-0.69z)| lr 8.72e-05 | 323.26 ms | 52.2% bf16 MFU | 1621720 tok/s
step 14866/19560 | loss 3.288883 (-0.73z)| norm 0.2707 (-0.48z)| lr 8.72e-05 | 323.11 ms | 52.2% bf16 MFU | 1621767 tok/s
step 14867/19560 | loss 3.345703 (+0.44z)| norm 0.2728 (-0.39z)| lr 8.71e-05 | 322.81 ms | 52.3% bf16 MFU | 1621884 tok/s
step 14868/19560 | loss 3.369045 (+0.92z)| norm 0.2468 (-1.59z)| lr 8.71e-05 | 323.19 ms | 52.2% bf16 MFU | 1621901 tok/s
step 14869/19560 | loss 3.257553 (-1.36z)| norm 0.2636 (-0.79z)| lr 8.71e-05 | 323.54 ms | 52.2% bf16 MFU | 1621830 tok/s
step 14870/19560 | loss 3.344409 (+0.42z)| norm 0.2687 (-0.55z)| lr 8.70e-05 | 323.11 ms | 52.2% bf16 MFU | 1621871 tok/s
step 14871/19560 | loss 3.356359 (+0.66z)| norm 0.2705 (-0.47z)| lr 8.70e-05 | 322.74 ms | 52.3% bf16 MFU | 1622002 tok/s
step 14872/19560 | loss 3.331661 (+0.16z)| norm 0.2818 (+0.05z)| lr 8.70e-05 | 323.29 ms | 52.2% bf16 MFU | 1621988 tok/s
step 14873/19560 | loss 3.336437 (+0.27z)| norm 0.2660 (-0.69z)| lr 8.69e-05 | 322.87 ms | 52.3% bf16 MFU | 1622080 tok/s
step 14874/19560 | loss 3.305993 (-0.36z)| norm 0.2787 (-0.09z)| lr 8.69e-05 | 323.32 ms | 52.2% bf16 MFU | 1622055 tok/s
step 14875/19560 | loss 3.285591 (-0.77z)| norm 0.2711 (-0.45z)| lr 8.68e-05 | 322.66 ms | 52.3% bf16 MFU | 1622197 tok/s
step 14876/19560 | loss 3.294886 (-0.58z)| norm 0.2869 (+0.29z)| lr 8.68e-05 | 323.76 ms | 52.1% bf16 MFU | 1622056 tok/s
step 14877/19560 | loss 3.326962 (+0.08z)| norm 0.2754 (-0.25z)| lr 8.68e-05 | 322.58 ms | 52.3% bf16 MFU | 1622217 tok/s
step 14878/19560 | loss 3.263231 (-1.23z)| norm 0.2675 (-0.63z)| lr 8.67e-05 | 322.71 ms | 52.3% bf16 MFU | 1622338 tok/s
step 14879/19560 | loss 3.356878 (+0.70z)| norm 0.3022 (+1.00z)| lr 8.67e-05 | 323.68 ms | 52.1% bf16 MFU | 1622211 tok/s
step 14880/19560 | loss 3.383675 (+1.24z)| norm 0.2595 (-1.00z)| lr 8.67e-05 | 322.74 ms | 52.3% bf16 MFU | 1622324 tok/s
step 14881/19560 | loss 3.321958 (-0.04z)| norm 0.2642 (-0.77z)| lr 8.66e-05 | 323.16 ms | 52.2% bf16 MFU | 1622327 tok/s
step 14882/19560 | loss 3.326441 (+0.05z)| norm 0.2809 (+0.01z)| lr 8.66e-05 | 322.72 ms | 52.3% bf16 MFU | 1622440 tok/s
step 14883/19560 | loss 3.295329 (-0.59z)| norm 0.2718 (-0.42z)| lr 8.66e-05 | 322.67 ms | 52.3% bf16 MFU | 1622560 tok/s
step 14884/19560 | loss 3.339441 (+0.31z)| norm 0.2687 (-0.56z)| lr 8.65e-05 | 322.90 ms | 52.3% bf16 MFU | 1622617 tok/s
step 14885/19560 | loss 3.297253 (-0.57z)| norm 0.2673 (-0.62z)| lr 8.65e-05 | 323.81 ms | 52.1% bf16 MFU | 1622442 tok/s
step 14886/19560 | loss 3.323385 (-0.02z)| norm 0.2705 (-0.46z)| lr 8.65e-05 | 322.59 ms | 52.3% bf16 MFU | 1622582 tok/s
step 14887/19560 | loss 3.315195 (-0.18z)| norm 0.2832 (+0.13z)| lr 8.64e-05 | 322.82 ms | 52.3% bf16 MFU | 1622656 tok/s
step 14888/19560 | loss 3.303366 (-0.43z)| norm 0.2682 (-0.57z)| lr 8.64e-05 | 322.97 ms | 52.3% bf16 MFU | 1622690 tok/s
step 14889/19560 | loss 3.335373 (+0.24z)| norm 0.2922 (+0.56z)| lr 8.64e-05 | 322.66 ms | 52.3% bf16 MFU | 1622799 tok/s
step 14890/19560 | loss 3.272952 (-1.07z)| norm 0.2843 (+0.18z)| lr 8.63e-05 | 322.58 ms | 52.3% bf16 MFU | 1622925 tok/s
step 14891/19560 | loss 3.367453 (+0.89z)| norm 0.2953 (+0.69z)| lr 8.63e-05 | 323.09 ms | 52.2% bf16 MFU | 1622916 tok/s
step 14892/19560 | loss 3.292826 (-0.66z)| norm 0.3198 (+1.80z)| lr 8.62e-05 | 323.10 ms | 52.2% bf16 MFU | 1622905 tok/s
step 14893/19560 | loss 3.283808 (-0.84z)| norm 0.2677 (-0.61z)| lr 8.62e-05 | 323.08 ms | 52.2% bf16 MFU | 1622898 tok/s
step 14894/19560 | loss 3.292761 (-0.64z)| norm 0.3370 (+2.52z)| lr 8.62e-05 | 322.60 ms | 52.3% bf16 MFU | 1623013 tok/s
step 14895/19560 | loss 3.294143 (-0.60z)| norm 0.2849 (+0.16z)| lr 8.61e-05 | 322.55 ms | 52.3% bf16 MFU | 1623134 tok/s
step 14896/19560 | loss 3.372518 (+1.02z)| norm 0.3321 (+2.24z)| lr 8.61e-05 | 323.39 ms | 52.2% bf16 MFU | 1623040 tok/s
step 14897/19560 | loss 3.320526 (-0.05z)| norm 0.3174 (+1.56z)| lr 8.61e-05 | 322.53 ms | 52.3% bf16 MFU | 1623166 tok/s
step 14898/19560 | loss 3.334100 (+0.23z)| norm 0.2833 (+0.06z)| lr 8.60e-05 | 322.82 ms | 52.3% bf16 MFU | 1623213 tok/s
step 14899/19560 | loss 3.320131 (-0.05z)| norm 0.3286 (+2.02z)| lr 8.60e-05 | 322.46 ms | 52.3% bf16 MFU | 1623347 tok/s
step 14900/19560 | loss 3.333313 (+0.26z)| norm 0.3109 (+1.23z)| lr 8.60e-05 | 323.12 ms | 52.2% bf16 MFU | 1623308 tok/s
step 14901/19560 | loss 3.279978 (-0.90z)| norm 0.2953 (+0.55z)| lr 8.59e-05 | 323.42 ms | 52.2% bf16 MFU | 1623196 tok/s
step 14902/19560 | loss 3.260215 (-1.33z)| norm 0.2966 (+0.61z)| lr 8.59e-05 | 322.70 ms | 52.3% bf16 MFU | 1623270 tok/s
step 14903/19560 | loss 3.270532 (-1.09z)| norm 0.2785 (-0.17z)| lr 8.59e-05 | 322.81 ms | 52.3% bf16 MFU | 1623314 tok/s
step 14904/19560 | loss 3.316763 (-0.04z)| norm 0.2845 (+0.14z)| lr 8.58e-05 | 323.03 ms | 52.2% bf16 MFU | 1623299 tok/s
step 14905/19560 | loss 3.258182 (-1.36z)| norm 0.2918 (+0.49z)| lr 8.58e-05 | 322.86 ms | 52.3% bf16 MFU | 1623329 tok/s
step 14906/19560 | loss 3.338679 (+0.47z)| norm 0.3006 (+0.92z)| lr 8.58e-05 | 322.76 ms | 52.3% bf16 MFU | 1623383 tok/s
step 14907/19560 | loss 3.335790 (+0.40z)| norm 0.2938 (+0.58z)| lr 8.57e-05 | 322.94 ms | 52.3% bf16 MFU | 1623388 tok/s
step 14908/19560 | loss 3.322053 (+0.09z)| norm 0.2624 (-0.92z)| lr 8.57e-05 | 322.73 ms | 52.3% bf16 MFU | 1623447 tok/s
step 14909/19560 | loss 3.341867 (+0.53z)| norm 0.3017 (+0.96z)| lr 8.57e-05 | 323.58 ms | 52.2% bf16 MFU | 1623288 tok/s
step 14910/19560 | loss 3.339721 (+0.47z)| norm 0.2928 (+0.52z)| lr 8.56e-05 | 322.39 ms | 52.3% bf16 MFU | 1623436 tok/s
step 14911/19560 | loss 3.280644 (-0.87z)| norm 0.2528 (-1.38z)| lr 8.56e-05 | 322.77 ms | 52.3% bf16 MFU | 1623481 tok/s
step 14912/19560 | loss 3.339982 (+0.48z)| norm 0.2903 (+0.40z)| lr 8.55e-05 | 322.91 ms | 52.3% bf16 MFU | 1623489 tok/s
step 14913/19560 | loss 3.244850 (-1.65z)| norm 0.2757 (-0.28z)| lr 8.55e-05 | 323.02 ms | 52.2% bf16 MFU | 1623468 tok/s
step 14914/19560 | loss 3.287596 (-0.68z)| norm 0.2871 (+0.27z)| lr 8.55e-05 | 322.75 ms | 52.3% bf16 MFU | 1623516 tok/s
step 14915/19560 | loss 3.378326 (+1.35z)| norm 0.2937 (+0.59z)| lr 8.54e-05 | 322.67 ms | 52.3% bf16 MFU | 1623582 tok/s
step 14916/19560 | loss 3.284620 (-0.74z)| norm 0.3028 (+1.02z)| lr 8.54e-05 | 322.76 ms | 52.3% bf16 MFU | 1623621 tok/s
step 14917/19560 | loss 3.379394 (+1.35z)| norm 0.2679 (-0.67z)| lr 8.54e-05 | 322.90 ms | 52.3% bf16 MFU | 1623626 tok/s
step 14918/19560 | loss 3.288945 (-0.66z)| norm 0.2888 (+0.36z)| lr 8.53e-05 | 322.68 ms | 52.3% bf16 MFU | 1623685 tok/s
step 14919/19560 | loss 3.264535 (-1.19z)| norm 0.2935 (+0.60z)| lr 8.53e-05 | 322.56 ms | 52.3% bf16 MFU | 1623772 tok/s
step 14920/19560 | loss 3.326157 (+0.18z)| norm 0.2874 (+0.30z)| lr 8.53e-05 | 323.26 ms | 52.2% bf16 MFU | 1623676 tok/s
step 14921/19560 | loss 3.344274 (+0.60z)| norm 0.2870 (+0.28z)| lr 8.52e-05 | 323.25 ms | 52.2% bf16 MFU | 1623588 tok/s
step 14922/19560 | loss 3.321726 (+0.08z)| norm 0.3028 (+1.05z)| lr 8.52e-05 | 323.24 ms | 52.2% bf16 MFU | 1623507 tok/s
step 14923/19560 | loss 3.345268 (+0.62z)| norm 0.2739 (-0.36z)| lr 8.52e-05 | 322.44 ms | 52.3% bf16 MFU | 1623632 tok/s
step 14924/19560 | loss 3.286825 (-0.72z)| norm 0.3153 (+1.66z)| lr 8.51e-05 | 322.91 ms | 52.3% bf16 MFU | 1623632 tok/s
step 14925/19560 | loss 3.335185 (+0.39z)| norm 0.2887 (+0.35z)| lr 8.51e-05 | 322.93 ms | 52.3% bf16 MFU | 1623627 tok/s
step 14926/19560 | loss 3.318074 (-0.01z)| norm 0.2750 (-0.31z)| lr 8.51e-05 | 322.85 ms | 52.3% bf16 MFU | 1623641 tok/s
step 14927/19560 | loss 3.344263 (+0.60z)| norm 0.3043 (+1.16z)| lr 8.50e-05 | 322.68 ms | 52.3% bf16 MFU | 1623699 tok/s
step 14928/19560 | loss 3.291999 (-0.59z)| norm 0.3138 (+1.69z)| lr 8.50e-05 | 322.22 ms | 52.4% bf16 MFU | 1623869 tok/s
step 14929/19560 | loss 3.243263 (-1.69z)| norm 0.2523 (-1.46z)| lr 8.50e-05 | 323.22 ms | 52.2% bf16 MFU | 1623780 tok/s
step 14930/19560 | loss 3.295797 (-0.48z)| norm 0.2810 (+0.02z)| lr 8.49e-05 | 323.19 ms | 52.2% bf16 MFU | 1623703 tok/s
step 14931/19560 | loss 3.322266 (+0.12z)| norm 0.2788 (-0.08z)| lr 8.49e-05 | 322.91 ms | 52.3% bf16 MFU | 1623699 tok/s
step 14932/19560 | loss 3.363373 (+1.07z)| norm 0.2657 (-0.76z)| lr 8.49e-05 | 322.83 ms | 52.3% bf16 MFU | 1623717 tok/s
step 14933/19560 | loss 3.335649 (+0.42z)| norm 0.2854 (+0.26z)| lr 8.48e-05 | 322.68 ms | 52.3% bf16 MFU | 1623772 tok/s
step 14934/19560 | loss 3.285916 (-0.73z)| norm 0.2799 (-0.02z)| lr 8.48e-05 | 322.99 ms | 52.3% bf16 MFU | 1623745 tok/s
step 14935/19560 | loss 3.279597 (-0.88z)| norm 0.2669 (-0.70z)| lr 8.47e-05 | 322.96 ms | 52.3% bf16 MFU | 1623727 tok/s
step 14936/19560 | loss 3.296430 (-0.48z)| norm 0.2808 (+0.02z)| lr 8.47e-05 | 322.86 ms | 52.3% bf16 MFU | 1623736 tok/s
step 14937/19560 | loss 3.265659 (-1.21z)| norm 0.2634 (-0.89z)| lr 8.47e-05 | 323.08 ms | 52.2% bf16 MFU | 1623687 tok/s
step 14938/19560 | loss 3.346330 (+0.67z)| norm 0.2649 (-0.80z)| lr 8.46e-05 | 322.28 ms | 52.4% bf16 MFU | 1623843 tok/s
step 14939/19560 | loss 3.303365 (-0.36z)| norm 0.2924 (+0.62z)| lr 8.46e-05 | 323.18 ms | 52.2% bf16 MFU | 1623764 tok/s
step 14940/19560 | loss 3.343558 (+0.60z)| norm 0.2726 (-0.41z)| lr 8.46e-05 | 322.69 ms | 52.3% bf16 MFU | 1623814 tok/s
step 14941/19560 | loss 3.342022 (+0.56z)| norm 0.3155 (+1.78z)| lr 8.45e-05 | 322.73 ms | 52.3% bf16 MFU | 1623851 tok/s
step 14942/19560 | loss 3.304346 (-0.35z)| norm 0.2922 (+0.57z)| lr 8.45e-05 | 322.50 ms | 52.3% bf16 MFU | 1623943 tok/s
step 14943/19560 | loss 3.370504 (+1.23z)| norm 0.2758 (-0.27z)| lr 8.45e-05 | 323.11 ms | 52.2% bf16 MFU | 1623877 tok/s
step 14944/19560 | loss 3.334793 (+0.36z)| norm 0.3170 (+1.83z)| lr 8.44e-05 | 323.19 ms | 52.2% bf16 MFU | 1623794 tok/s
step 14945/19560 | loss 3.312448 (-0.17z)| norm 0.2661 (-0.80z)| lr 8.44e-05 | 322.50 ms | 52.3% bf16 MFU | 1623889 tok/s
step 14946/19560 | loss 3.279186 (-0.97z)| norm 0.2885 (+0.36z)| lr 8.44e-05 | 323.32 ms | 52.2% bf16 MFU | 1623772 tok/s
step 14947/19560 | loss 3.309161 (-0.24z)| norm 0.2768 (-0.25z)| lr 8.43e-05 | 322.74 ms | 52.3% bf16 MFU | 1623808 tok/s
step 14948/19560 | loss 3.282634 (-0.87z)| norm 0.2665 (-0.78z)| lr 8.43e-05 | 322.27 ms | 52.4% bf16 MFU | 1623961 tok/s
step 14949/19560 | loss 3.289162 (-0.71z)| norm 0.3164 (+1.77z)| lr 8.43e-05 | 323.17 ms | 52.2% bf16 MFU | 1623880 tok/s
step 14950/19560 | loss 3.298436 (-0.48z)| norm 0.2638 (-0.93z)| lr 8.42e-05 | 322.36 ms | 52.4% bf16 MFU | 1624007 tok/s
step 14951/19560 | loss 3.272095 (-1.10z)| norm 0.2544 (-1.40z)| lr 8.42e-05 | 323.40 ms | 52.2% bf16 MFU | 1623866 tok/s
step 14952/19560 | loss 3.317352 (-0.02z)| norm 0.3030 (+1.07z)| lr 8.42e-05 | 322.96 ms | 52.3% bf16 MFU | 1623841 tok/s
step 14953/19560 | loss 3.258298 (-1.41z)| norm 0.2778 (-0.21z)| lr 8.41e-05 | 322.85 ms | 52.3% bf16 MFU | 1623846 tok/s
step 14954/19560 | loss 3.320545 (+0.09z)| norm 0.2692 (-0.63z)| lr 8.41e-05 | 322.89 ms | 52.3% bf16 MFU | 1623840 tok/s
step 14955/19560 | loss 3.413864 (+2.27z)| norm 0.3283 (+2.31z)| lr 8.41e-05 | 322.34 ms | 52.4% bf16 MFU | 1623974 tok/s
step 14956/19560 | loss 3.370889 (+1.25z)| norm 0.2716 (-0.54z)| lr 8.40e-05 | 322.84 ms | 52.3% bf16 MFU | 1623974 tok/s
step 14957/19560 | loss 3.317171 (-0.02z)| norm 0.2667 (-0.78z)| lr 8.40e-05 | 322.63 ms | 52.3% bf16 MFU | 1624028 tok/s
step 14958/19560 | loss 3.358171 (+0.93z)| norm 0.2607 (-1.07z)| lr 8.39e-05 | 322.99 ms | 52.3% bf16 MFU | 1623988 tok/s
step 14959/19560 | loss 3.300744 (-0.42z)| norm 0.2734 (-0.44z)| lr 8.39e-05 | 323.05 ms | 52.2% bf16 MFU | 1623934 tok/s
step 14960/19560 | loss 3.321808 (+0.07z)| norm 0.2569 (-1.25z)| lr 8.39e-05 | 322.40 ms | 52.3% bf16 MFU | 1624048 tok/s
step 14961/19560 | loss 3.343676 (+0.58z)| norm 0.2731 (-0.45z)| lr 8.38e-05 | 322.56 ms | 52.3% bf16 MFU | 1624115 tok/s
step 14962/19560 | loss 3.296909 (-0.52z)| norm 0.2639 (-0.90z)| lr 8.38e-05 | 323.07 ms | 52.2% bf16 MFU | 1624051 tok/s
step 14963/19560 | loss 3.240511 (-1.81z)| norm 0.2759 (-0.31z)| lr 8.38e-05 | 322.47 ms | 52.3% bf16 MFU | 1624142 tok/s
step 14964/19560 | loss 3.315485 (-0.03z)| norm 0.2685 (-0.68z)| lr 8.37e-05 | 322.62 ms | 52.3% bf16 MFU | 1624190 tok/s
step 14965/19560 | loss 3.380263 (+1.74z)| norm 0.2962 (+0.77z)| lr 8.37e-05 | 323.47 ms | 52.2% bf16 MFU | 1624020 tok/s
step 14966/19560 | loss 3.301943 (-0.40z)| norm 0.2761 (-0.29z)| lr 8.37e-05 | 322.96 ms | 52.3% bf16 MFU | 1623989 tok/s
step 14967/19560 | loss 3.272875 (-1.19z)| norm 0.2859 (+0.22z)| lr 8.36e-05 | 322.78 ms | 52.3% bf16 MFU | 1624004 tok/s
step 14968/19560 | loss 3.304837 (-0.31z)| norm 0.2569 (-1.29z)| lr 8.36e-05 | 322.35 ms | 52.4% bf16 MFU | 1624127 tok/s
step 14969/19560 | loss 3.360367 (+1.20z)| norm 0.2923 (+0.58z)| lr 8.36e-05 | 323.19 ms | 52.2% bf16 MFU | 1624032 tok/s
step 14970/19560 | loss 3.289155 (-0.74z)| norm 0.2863 (+0.26z)| lr 8.35e-05 | 322.52 ms | 52.3% bf16 MFU | 1624111 tok/s
step 14971/19560 | loss 3.443458 (+3.31z)| norm 0.2648 (-0.86z)| lr 8.35e-05 | 322.60 ms | 52.3% bf16 MFU | 1624164 tok/s
step 14972/19560 | loss 3.301742 (-0.41z)| norm 0.2740 (-0.38z)| lr 8.35e-05 | 322.79 ms | 52.3% bf16 MFU | 1624168 tok/s
step 14973/19560 | loss 3.282782 (-0.92z)| norm 0.2685 (-0.67z)| lr 8.34e-05 | 322.83 ms | 52.3% bf16 MFU | 1624162 tok/s
step 14974/19560 | loss 3.291063 (-0.70z)| norm 0.2661 (-0.79z)| lr 8.34e-05 | 322.74 ms | 52.3% bf16 MFU | 1624180 tok/s
step 14975/19560 | loss 3.282278 (-0.92z)| norm 0.2685 (-0.66z)| lr 8.34e-05 | 323.00 ms | 52.3% bf16 MFU | 1624129 tok/s
step 14976/19560 | loss 3.281315 (-0.93z)| norm 0.2761 (-0.27z)| lr 8.33e-05 | 322.56 ms | 52.3% bf16 MFU | 1624194 tok/s
step 14977/19560 | loss 3.271656 (-1.16z)| norm 0.2563 (-1.32z)| lr 8.33e-05 | 322.09 ms | 52.4% bf16 MFU | 1624372 tok/s
step 14978/19560 | loss 3.261695 (-1.40z)| norm 0.2714 (-0.49z)| lr 8.33e-05 | 322.83 ms | 52.3% bf16 MFU | 1624356 tok/s
step 14979/19560 | loss 3.298573 (-0.43z)| norm 0.2538 (-1.44z)| lr 8.32e-05 | 323.31 ms | 52.2% bf16 MFU | 1624218 tok/s
step 14980/19560 | loss 3.306345 (-0.23z)| norm 0.2551 (-1.35z)| lr 8.32e-05 | 323.04 ms | 52.2% bf16 MFU | 1624157 tok/s
step 14981/19560 | loss 3.246754 (-1.80z)| norm 0.2736 (-0.35z)| lr 8.32e-05 | 323.07 ms | 52.2% bf16 MFU | 1624092 tok/s
step 14982/19560 | loss 3.343922 (+0.75z)| norm 0.2813 (+0.05z)| lr 8.31e-05 | 322.23 ms | 52.4% bf16 MFU | 1624241 tok/s
step 14983/19560 | loss 3.305102 (-0.27z)| norm 0.2893 (+0.48z)| lr 8.31e-05 | 322.88 ms | 52.3% bf16 MFU | 1624218 tok/s
step 14984/19560 | loss 3.325479 (+0.28z)| norm 0.2687 (-0.64z)| lr 8.30e-05 | 322.86 ms | 52.3% bf16 MFU | 1624201 tok/s
step 14985/19560 | loss 3.291720 (-0.62z)| norm 0.2885 (+0.43z)| lr 8.30e-05 | 322.96 ms | 52.3% bf16 MFU | 1624160 tok/s
step 14986/19560 | loss 3.284084 (-0.81z)| norm 0.3259 (+2.39z)| lr 8.30e-05 | 323.06 ms | 52.2% bf16 MFU | 1624097 tok/s
step 14987/19560 | loss 3.345527 (+0.81z)| norm 0.2796 (-0.09z)| lr 8.29e-05 | 322.87 ms | 52.3% bf16 MFU | 1624084 tok/s
step 14988/19560 | loss 3.238836 (-1.97z)| norm 0.2977 (+0.87z)| lr 8.29e-05 | 322.58 ms | 52.3% bf16 MFU | 1624144 tok/s
step 14989/19560 | loss 3.279452 (-0.91z)| norm 0.3231 (+2.19z)| lr 8.29e-05 | 323.16 ms | 52.2% bf16 MFU | 1624055 tok/s
step 14990/19560 | loss 3.441341 (+3.29z)| norm 0.3263 (+2.29z)| lr 8.28e-05 | 322.63 ms | 52.3% bf16 MFU | 1624105 tok/s
step 14991/19560 | loss 3.259808 (-1.38z)| norm 0.3281 (+2.31z)| lr 8.28e-05 | 323.41 ms | 52.2% bf16 MFU | 1623956 tok/s
step 14992/19560 | loss 3.341467 (+0.73z)| norm 0.3471 (+3.14z)| lr 8.28e-05 | 322.65 ms | 52.3% bf16 MFU | 1624006 tok/s
step 14993/19560 | loss 3.319737 (+0.16z)| norm 0.2998 (+0.80z)| lr 8.27e-05 | 322.79 ms | 52.3% bf16 MFU | 1624019 tok/s
step 14994/19560 | loss 3.301687 (-0.31z)| norm 0.3418 (+2.76z)| lr 8.27e-05 | 322.95 ms | 52.3% bf16 MFU | 1623989 tok/s
step 14995/19560 | loss 3.359410 (+1.19z)| norm 0.3755 (+4.04z)| lr 8.27e-05 | 322.87 ms | 52.3% bf16 MFU | 1623980 tok/s
step 14996/19560 | loss 3.363850 (+1.30z)| norm 0.3058 (+0.92z)| lr 8.26e-05 | 323.33 ms | 52.2% bf16 MFU | 1623856 tok/s
step 14997/19560 | loss 3.345203 (+0.81z)| norm 0.3856 (+4.17z)| lr 8.26e-05 | 322.77 ms | 52.3% bf16 MFU | 1623880 tok/s
step 14998/19560 | loss 3.252320 (-1.59z)| norm 0.3904 (+4.05z)| lr 8.26e-05 | 323.20 ms | 52.2% bf16 MFU | 1623795 tok/s
step 14999/19560 | loss 3.263035 (-1.29z)| norm 0.3034 (+0.62z)| lr 8.25e-05 | 322.80 ms | 52.3% bf16 MFU | 1623813 tok/s
step 15000/19560 | loss 3.313325 (+0.02z)| norm 0.3154 (+1.08z)| lr 8.25e-05 | 323.24 ms | 52.2% bf16 MFU | 1623720 tok/s
val loss 3.303314

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00015000_00003.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00015000_00001.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00015000_00007.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00015000_00004.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00015000_00002.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00015000_00006.bin
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3010/10042 = 0.299741

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00015000_00005.bin
Writing checkpoint at step 15000
Writing model to log124M/model_00015000.bin
Writing state to log124M/state_00015000_00000.bin
step 15001/19560 | loss 3.339555 (+0.69z)| norm 0.3728 (+3.17z)| lr 8.25e-05 | 318.99 ms | 52.9% bf16 MFU | 1624714 tok/s
step 15002/19560 | loss 3.275087 (-0.96z)| norm 0.2884 (-0.01z)| lr 8.24e-05 | 321.47 ms | 52.5% bf16 MFU | 1625023 tok/s
step 15003/19560 | loss 3.335854 (+0.59z)| norm 0.2998 (+0.41z)| lr 8.24e-05 | 322.69 ms | 52.3% bf16 MFU | 1625009 tok/s
step 15004/19560 | loss 3.363179 (+1.28z)| norm 0.3135 (+0.91z)| lr 8.24e-05 | 322.29 ms | 52.4% bf16 MFU | 1625096 tok/s
step 15005/19560 | loss 3.270677 (-1.08z)| norm 0.2848 (-0.17z)| lr 8.23e-05 | 321.96 ms | 52.4% bf16 MFU | 1625263 tok/s
step 15006/19560 | loss 3.374072 (+1.53z)| norm 0.2822 (-0.27z)| lr 8.23e-05 | 321.82 ms | 52.4% bf16 MFU | 1625456 tok/s
step 15007/19560 | loss 3.346160 (+0.83z)| norm 0.3188 (+1.10z)| lr 8.23e-05 | 322.82 ms | 52.3% bf16 MFU | 1625388 tok/s
step 15008/19560 | loss 3.374310 (+1.55z)| norm 0.3255 (+1.33z)| lr 8.22e-05 | 321.41 ms | 52.5% bf16 MFU | 1625680 tok/s
step 15009/19560 | loss 3.324368 (+0.27z)| norm 0.3014 (+0.42z)| lr 8.22e-05 | 322.43 ms | 52.3% bf16 MFU | 1625698 tok/s
step 15010/19560 | loss 3.284075 (-0.75z)| norm 0.2799 (-0.39z)| lr 8.22e-05 | 323.29 ms | 52.2% bf16 MFU | 1625500 tok/s
step 15011/19560 | loss 3.292171 (-0.54z)| norm 0.3025 (+0.45z)| lr 8.21e-05 | 323.09 ms | 52.2% bf16 MFU | 1625361 tok/s
step 15012/19560 | loss 3.316826 (+0.09z)| norm 0.2956 (+0.18z)| lr 8.21e-05 | 321.72 ms | 52.5% bf16 MFU | 1625576 tok/s
step 15013/19560 | loss 3.361753 (+1.22z)| norm 0.2631 (-1.04z)| lr 8.20e-05 | 322.30 ms | 52.4% bf16 MFU | 1625633 tok/s
step 15014/19560 | loss 3.394211 (+2.01z)| norm 0.3094 (+0.70z)| lr 8.20e-05 | 322.38 ms | 52.4% bf16 MFU | 1625667 tok/s
step 15015/19560 | loss 3.276082 (-0.95z)| norm 0.3101 (+0.71z)| lr 8.20e-05 | 321.68 ms | 52.5% bf16 MFU | 1625877 tok/s
step 15016/19560 | loss 3.369907 (+1.38z)| norm 0.2880 (-0.12z)| lr 8.19e-05 | 322.18 ms | 52.4% bf16 MFU | 1625949 tok/s
step 15017/19560 | loss 3.337893 (+0.58z)| norm 0.3157 (+0.91z)| lr 8.19e-05 | 322.80 ms | 52.3% bf16 MFU | 1625862 tok/s
step 15018/19560 | loss 3.318864 (+0.10z)| norm 0.2901 (-0.05z)| lr 8.19e-05 | 321.95 ms | 52.4% bf16 MFU | 1625991 tok/s
step 15019/19560 | loss 3.292777 (-0.54z)| norm 0.2750 (-0.62z)| lr 8.18e-05 | 322.29 ms | 52.4% bf16 MFU | 1626029 tok/s
step 15020/19560 | loss 3.273915 (-1.00z)| norm 0.2854 (-0.22z)| lr 8.18e-05 | 322.61 ms | 52.3% bf16 MFU | 1625985 tok/s
step 15021/19560 | loss 3.306954 (-0.18z)| norm 0.2884 (-0.11z)| lr 8.18e-05 | 322.81 ms | 52.3% bf16 MFU | 1625892 tok/s
step 15022/19560 | loss 3.311782 (-0.07z)| norm 0.3026 (+0.44z)| lr 8.17e-05 | 322.56 ms | 52.3% bf16 MFU | 1625866 tok/s
step 15023/19560 | loss 3.259617 (-1.36z)| norm 0.2627 (-1.07z)| lr 8.17e-05 | 322.42 ms | 52.3% bf16 MFU | 1625877 tok/s
step 15024/19560 | loss 3.324609 (+0.27z)| norm 0.2992 (+0.33z)| lr 8.17e-05 | 323.01 ms | 52.3% bf16 MFU | 1625740 tok/s
step 15025/19560 | loss 3.282249 (-0.78z)| norm 0.3103 (+0.76z)| lr 8.16e-05 | 323.23 ms | 52.2% bf16 MFU | 1625554 tok/s
step 15026/19560 | loss 3.533534 (+4.93z)| norm 0.3289 (+1.45z)| lr 8.16e-05 | 323.54 ms | 52.2% bf16 MFU | 1625300 tok/s
step 15027/19560 | loss 3.379347 (+1.43z)| norm 0.3149 (+0.93z)| lr 8.16e-05 | 322.84 ms | 52.3% bf16 MFU | 1625234 tok/s
step 15028/19560 | loss 3.292508 (-0.51z)| norm 0.2899 (-0.03z)| lr 8.15e-05 | 323.25 ms | 52.2% bf16 MFU | 1625070 tok/s
step 15029/19560 | loss 3.301556 (-0.31z)| norm 0.2864 (-0.16z)| lr 8.15e-05 | 322.50 ms | 52.3% bf16 MFU | 1625100 tok/s
step 15030/19560 | loss 3.322221 (+0.14z)| norm 0.2932 (+0.10z)| lr 8.15e-05 | 323.41 ms | 52.2% bf16 MFU | 1624901 tok/s
step 15031/19560 | loss 3.261523 (-1.22z)| norm 0.2865 (-0.16z)| lr 8.14e-05 | 323.21 ms | 52.2% bf16 MFU | 1624763 tok/s
step 15032/19560 | loss 3.302095 (-0.30z)| norm 0.3030 (+0.47z)| lr 8.14e-05 | 322.52 ms | 52.3% bf16 MFU | 1624806 tok/s
step 15033/19560 | loss 3.340927 (+0.56z)| norm 0.3267 (+1.36z)| lr 8.14e-05 | 322.30 ms | 52.4% bf16 MFU | 1624900 tok/s
step 15034/19560 | loss 3.268974 (-1.05z)| norm 0.2877 (-0.12z)| lr 8.13e-05 | 323.63 ms | 52.1% bf16 MFU | 1624656 tok/s
step 15035/19560 | loss 3.268668 (-1.05z)| norm 0.2849 (-0.23z)| lr 8.13e-05 | 322.16 ms | 52.4% bf16 MFU | 1624795 tok/s
step 15036/19560 | loss 3.358355 (+0.96z)| norm 0.2864 (-0.18z)| lr 8.13e-05 | 322.80 ms | 52.3% bf16 MFU | 1624764 tok/s
step 15037/19560 | loss 3.360792 (+1.01z)| norm 0.2958 (+0.19z)| lr 8.12e-05 | 323.68 ms | 52.1% bf16 MFU | 1624514 tok/s
step 15038/19560 | loss 3.280667 (-0.77z)| norm 0.2809 (-0.38z)| lr 8.12e-05 | 322.71 ms | 52.3% bf16 MFU | 1624519 tok/s
step 15039/19560 | loss 3.344721 (+0.65z)| norm 0.3268 (+1.36z)| lr 8.12e-05 | 322.89 ms | 52.3% bf16 MFU | 1624480 tok/s
step 15040/19560 | loss 3.321173 (+0.13z)| norm 0.2930 (+0.06z)| lr 8.11e-05 | 322.68 ms | 52.3% bf16 MFU | 1624496 tok/s
step 15041/19560 | loss 3.324608 (+0.19z)| norm 0.2718 (-0.75z)| lr 8.11e-05 | 322.82 ms | 52.3% bf16 MFU | 1624477 tok/s
step 15042/19560 | loss 3.308671 (-0.17z)| norm 0.3156 (+0.91z)| lr 8.11e-05 | 321.87 ms | 52.4% bf16 MFU | 1624698 tok/s
step 15043/19560 | loss 3.301446 (-0.32z)| norm 0.2899 (-0.07z)| lr 8.10e-05 | 322.82 ms | 52.3% bf16 MFU | 1624667 tok/s
step 15044/19560 | loss 3.298476 (-0.40z)| norm 0.2910 (-0.02z)| lr 8.10e-05 | 322.86 ms | 52.3% bf16 MFU | 1624629 tok/s
step 15045/19560 | loss 3.241920 (-1.65z)| norm 0.2626 (-1.10z)| lr 8.10e-05 | 322.88 ms | 52.3% bf16 MFU | 1624587 tok/s
step 15046/19560 | loss 3.333664 (+0.42z)| norm 0.2812 (-0.39z)| lr 8.09e-05 | 322.08 ms | 52.4% bf16 MFU | 1624749 tok/s
step 15047/19560 | loss 3.336635 (+0.48z)| norm 0.2883 (-0.12z)| lr 8.09e-05 | 322.12 ms | 52.4% bf16 MFU | 1624892 tok/s
step 15048/19560 | loss 3.382780 (+1.51z)| norm 0.2753 (-0.61z)| lr 8.09e-05 | 323.03 ms | 52.2% bf16 MFU | 1624799 tok/s
step 15049/19560 | loss 3.361168 (+1.01z)| norm 0.2616 (-1.12z)| lr 8.08e-05 | 322.54 ms | 52.3% bf16 MFU | 1624832 tok/s
step 15050/19560 | loss 3.226642 (-1.98z)| norm 0.2859 (-0.19z)| lr 8.08e-05 | 322.14 ms | 52.4% bf16 MFU | 1624967 tok/s
step 15051/19560 | loss 3.279371 (-0.79z)| norm 0.2916 (+0.02z)| lr 8.07e-05 | 322.39 ms | 52.3% bf16 MFU | 1625030 tok/s
step 15052/19560 | loss 3.270129 (-0.99z)| norm 0.2762 (-0.56z)| lr 8.07e-05 | 323.71 ms | 52.1% bf16 MFU | 1624760 tok/s
step 15053/19560 | loss 3.350386 (+0.78z)| norm 0.2699 (-0.79z)| lr 8.07e-05 | 322.43 ms | 52.3% bf16 MFU | 1624825 tok/s
step 15054/19560 | loss 3.385578 (+1.54z)| norm 0.2972 (+0.24z)| lr 8.06e-05 | 322.73 ms | 52.3% bf16 MFU | 1624812 tok/s
step 15055/19560 | loss 3.329024 (+0.30z)| norm 0.2824 (-0.31z)| lr 8.06e-05 | 323.14 ms | 52.2% bf16 MFU | 1624696 tok/s
step 15056/19560 | loss 3.286777 (-0.63z)| norm 0.2632 (-1.03z)| lr 8.06e-05 | 322.84 ms | 52.3% bf16 MFU | 1624661 tok/s
step 15057/19560 | loss 3.261900 (-1.18z)| norm 0.2978 (+0.28z)| lr 8.05e-05 | 322.73 ms | 52.3% bf16 MFU | 1624654 tok/s
step 15058/19560 | loss 3.388624 (+1.58z)| norm 0.2682 (-0.85z)| lr 8.05e-05 | 322.61 ms | 52.3% bf16 MFU | 1624679 tok/s
step 15059/19560 | loss 3.313976 (-0.05z)| norm 0.2831 (-0.29z)| lr 8.05e-05 | 323.43 ms | 52.2% bf16 MFU | 1624496 tok/s
step 15060/19560 | loss 3.288452 (-0.59z)| norm 0.2840 (-0.26z)| lr 8.04e-05 | 322.48 ms | 52.3% bf16 MFU | 1624560 tok/s
step 15061/19560 | loss 3.322457 (+0.15z)| norm 0.2756 (-0.58z)| lr 8.04e-05 | 322.25 ms | 52.4% bf16 MFU | 1624682 tok/s
step 15062/19560 | loss 3.449821 (+2.83z)| norm 0.2822 (-0.33z)| lr 8.04e-05 | 323.44 ms | 52.2% bf16 MFU | 1624496 tok/s
step 15063/19560 | loss 3.351431 (+0.72z)| norm 0.2835 (-0.28z)| lr 8.03e-05 | 322.58 ms | 52.3% bf16 MFU | 1624536 tok/s
step 15064/19560 | loss 3.299224 (-0.38z)| norm 0.2753 (-0.59z)| lr 8.03e-05 | 323.06 ms | 52.2% bf16 MFU | 1624453 tok/s
step 15065/19560 | loss 3.341850 (+0.51z)| norm 0.2842 (-0.26z)| lr 8.03e-05 | 322.43 ms | 52.3% bf16 MFU | 1624532 tok/s
step 15066/19560 | loss 3.322635 (+0.10z)| norm 0.2700 (-0.81z)| lr 8.02e-05 | 322.57 ms | 52.3% bf16 MFU | 1624573 tok/s
step 15067/19560 | loss 3.272651 (-0.96z)| norm 0.2675 (-0.90z)| lr 8.02e-05 | 322.69 ms | 52.3% bf16 MFU | 1624582 tok/s
step 15068/19560 | loss 3.330768 (+0.28z)| norm 0.2653 (-0.98z)| lr 8.02e-05 | 322.88 ms | 52.3% bf16 MFU | 1624541 tok/s
step 15069/19560 | loss 3.378876 (+1.30z)| norm 0.2665 (-0.92z)| lr 8.01e-05 | 322.75 ms | 52.3% bf16 MFU | 1624535 tok/s
step 15070/19560 | loss 3.301400 (-0.34z)| norm 0.3083 (+0.69z)| lr 8.01e-05 | 323.30 ms | 52.2% bf16 MFU | 1624392 tok/s
step 15071/19560 | loss 3.320163 (+0.06z)| norm 0.2729 (-0.67z)| lr 8.01e-05 | 322.91 ms | 52.3% bf16 MFU | 1624355 tok/s
step 15072/19560 | loss 3.340259 (+0.49z)| norm 0.2922 (+0.08z)| lr 8.00e-05 | 322.24 ms | 52.4% bf16 MFU | 1624487 tok/s
step 15073/19560 | loss 3.372362 (+1.16z)| norm 0.3312 (+1.55z)| lr 8.00e-05 | 322.53 ms | 52.3% bf16 MFU | 1624540 tok/s
step 15074/19560 | loss 3.345758 (+0.58z)| norm 0.3003 (+0.36z)| lr 8.00e-05 | 322.60 ms | 52.3% bf16 MFU | 1624573 tok/s
step 15075/19560 | loss 3.245854 (-1.51z)| norm 0.3191 (+1.07z)| lr 7.99e-05 | 322.68 ms | 52.3% bf16 MFU | 1624584 tok/s
step 15076/19560 | loss 3.266935 (-1.06z)| norm 0.3075 (+0.61z)| lr 7.99e-05 | 322.73 ms | 52.3% bf16 MFU | 1624582 tok/s
step 15077/19560 | loss 3.259395 (-1.21z)| norm 0.2903 (-0.04z)| lr 7.99e-05 | 322.27 ms | 52.4% bf16 MFU | 1624695 tok/s
step 15078/19560 | loss 3.331757 (+0.29z)| norm 0.3206 (+1.11z)| lr 7.98e-05 | 322.57 ms | 52.3% bf16 MFU | 1624727 tok/s
step 15079/19560 | loss 3.319355 (+0.03z)| norm 0.3085 (+0.63z)| lr 7.98e-05 | 323.26 ms | 52.2% bf16 MFU | 1624584 tok/s
step 15080/19560 | loss 3.231566 (-1.78z)| norm 0.3125 (+0.78z)| lr 7.98e-05 | 322.26 ms | 52.4% bf16 MFU | 1624702 tok/s
step 15081/19560 | loss 3.248032 (-1.43z)| norm 0.2979 (+0.21z)| lr 7.97e-05 | 322.61 ms | 52.3% bf16 MFU | 1624724 tok/s
step 15082/19560 | loss 3.304242 (-0.27z)| norm 0.3032 (+0.41z)| lr 7.97e-05 | 324.12 ms | 52.1% bf16 MFU | 1624366 tok/s
step 15083/19560 | loss 3.336103 (+0.41z)| norm 0.2988 (+0.25z)| lr 7.97e-05 | 322.68 ms | 52.3% bf16 MFU | 1624388 tok/s
step 15084/19560 | loss 3.303766 (-0.26z)| norm 0.3332 (+1.56z)| lr 7.96e-05 | 322.41 ms | 52.3% bf16 MFU | 1624475 tok/s
step 15085/19560 | loss 3.278398 (-0.79z)| norm 0.2880 (-0.19z)| lr 7.96e-05 | 322.56 ms | 52.3% bf16 MFU | 1624520 tok/s
step 15086/19560 | loss 3.350621 (+0.73z)| norm 0.2821 (-0.44z)| lr 7.96e-05 | 322.26 ms | 52.4% bf16 MFU | 1624640 tok/s
step 15087/19560 | loss 3.297533 (-0.38z)| norm 0.2756 (-0.69z)| lr 7.95e-05 | 322.73 ms | 52.3% bf16 MFU | 1624634 tok/s
step 15088/19560 | loss 3.316504 (+0.02z)| norm 0.2665 (-1.05z)| lr 7.95e-05 | 322.28 ms | 52.4% bf16 MFU | 1624743 tok/s
step 15089/19560 | loss 3.285753 (-0.62z)| norm 0.2740 (-0.76z)| lr 7.95e-05 | 322.59 ms | 52.3% bf16 MFU | 1624768 tok/s
step 15090/19560 | loss 3.258875 (-1.17z)| norm 0.2812 (-0.48z)| lr 7.94e-05 | 322.84 ms | 52.3% bf16 MFU | 1624729 tok/s
step 15091/19560 | loss 3.305004 (-0.22z)| norm 0.2674 (-1.02z)| lr 7.94e-05 | 322.19 ms | 52.4% bf16 MFU | 1624855 tok/s
step 15092/19560 | loss 3.240846 (-1.55z)| norm 0.2758 (-0.70z)| lr 7.94e-05 | 322.33 ms | 52.4% bf16 MFU | 1624941 tok/s
step 15093/19560 | loss 3.319309 (+0.10z)| norm 0.3048 (+0.45z)| lr 7.93e-05 | 323.21 ms | 52.2% bf16 MFU | 1624800 tok/s
step 15094/19560 | loss 3.280749 (-0.70z)| norm 0.2671 (-1.04z)| lr 7.93e-05 | 322.45 ms | 52.3% bf16 MFU | 1624858 tok/s
step 15095/19560 | loss 3.402094 (+1.81z)| norm 0.3024 (+0.35z)| lr 7.93e-05 | 323.24 ms | 52.2% bf16 MFU | 1624713 tok/s
step 15096/19560 | loss 3.281249 (-0.70z)| norm 0.2624 (-1.23z)| lr 7.92e-05 | 323.24 ms | 52.2% bf16 MFU | 1624576 tok/s
step 15097/19560 | loss 3.286416 (-0.58z)| norm 0.2758 (-0.69z)| lr 7.92e-05 | 322.50 ms | 52.3% bf16 MFU | 1624631 tok/s
step 15098/19560 | loss 3.317927 (+0.07z)| norm 0.2725 (-0.82z)| lr 7.92e-05 | 322.54 ms | 52.3% bf16 MFU | 1624674 tok/s
step 15099/19560 | loss 3.268720 (-0.95z)| norm 0.2599 (-1.31z)| lr 7.91e-05 | 322.66 ms | 52.3% bf16 MFU | 1624685 tok/s
step 15100/19560 | loss 3.286063 (-0.58z)| norm 0.2867 (-0.26z)| lr 7.91e-05 | 322.92 ms | 52.3% bf16 MFU | 1624629 tok/s
step 15101/19560 | loss 3.328794 (+0.33z)| norm 0.2626 (-1.20z)| lr 7.91e-05 | 322.45 ms | 52.3% bf16 MFU | 1624694 tok/s
step 15102/19560 | loss 3.288008 (-0.55z)| norm 0.2845 (-0.35z)| lr 7.90e-05 | 322.86 ms | 52.3% bf16 MFU | 1624654 tok/s
step 15103/19560 | loss 3.315588 (+0.04z)| norm 0.2593 (-1.34z)| lr 7.90e-05 | 322.86 ms | 52.3% bf16 MFU | 1624614 tok/s
step 15104/19560 | loss 3.304869 (-0.19z)| norm 0.2936 (+0.00z)| lr 7.90e-05 | 322.78 ms | 52.3% bf16 MFU | 1624598 tok/s
step 15105/19560 | loss 3.335330 (+0.45z)| norm 0.2721 (-0.85z)| lr 7.89e-05 | 322.92 ms | 52.3% bf16 MFU | 1624547 tok/s
step 15106/19560 | loss 3.319139 (+0.09z)| norm 0.2747 (-0.75z)| lr 7.89e-05 | 322.47 ms | 52.3% bf16 MFU | 1624613 tok/s
step 15107/19560 | loss 3.297646 (-0.37z)| norm 0.2724 (-0.86z)| lr 7.88e-05 | 322.19 ms | 52.4% bf16 MFU | 1624746 tok/s
step 15108/19560 | loss 3.323203 (+0.18z)| norm 0.2678 (-1.05z)| lr 7.88e-05 | 322.81 ms | 52.3% bf16 MFU | 1624717 tok/s
step 15109/19560 | loss 3.313651 (-0.04z)| norm 0.2812 (-0.52z)| lr 7.88e-05 | 322.57 ms | 52.3% bf16 MFU | 1624748 tok/s
step 15110/19560 | loss 3.336625 (+0.46z)| norm 0.2952 (+0.04z)| lr 7.87e-05 | 322.99 ms | 52.3% bf16 MFU | 1624673 tok/s
step 15111/19560 | loss 3.329089 (+0.29z)| norm 0.2745 (-0.78z)| lr 7.87e-05 | 323.22 ms | 52.2% bf16 MFU | 1624542 tok/s
step 15112/19560 | loss 3.317773 (+0.05z)| norm 0.2906 (-0.14z)| lr 7.87e-05 | 322.56 ms | 52.3% bf16 MFU | 1624584 tok/s
step 15113/19560 | loss 3.306150 (-0.21z)| norm 0.2683 (-1.03z)| lr 7.86e-05 | 322.91 ms | 52.3% bf16 MFU | 1624536 tok/s
step 15114/19560 | loss 3.284643 (-0.68z)| norm 0.2550 (-1.53z)| lr 7.86e-05 | 322.38 ms | 52.4% bf16 MFU | 1624623 tok/s
step 15115/19560 | loss 3.279013 (-0.79z)| norm 0.2954 (+0.07z)| lr 7.86e-05 | 322.39 ms | 52.4% bf16 MFU | 1624704 tok/s
step 15116/19560 | loss 3.290862 (-0.55z)| norm 0.2780 (-0.62z)| lr 7.85e-05 | 323.88 ms | 52.1% bf16 MFU | 1624408 tok/s
step 15117/19560 | loss 3.345053 (+0.64z)| norm 0.3037 (+0.42z)| lr 7.85e-05 | 322.75 ms | 52.3% bf16 MFU | 1624411 tok/s
step 15118/19560 | loss 3.409974 (+2.11z)| norm 0.2910 (-0.08z)| lr 7.85e-05 | 322.44 ms | 52.3% bf16 MFU | 1624490 tok/s
step 15119/19560 | loss 3.348497 (+0.72z)| norm 0.2779 (-0.60z)| lr 7.84e-05 | 322.22 ms | 52.4% bf16 MFU | 1624621 tok/s
step 15120/19560 | loss 3.310935 (-0.12z)| norm 0.2739 (-0.75z)| lr 7.84e-05 | 322.90 ms | 52.3% bf16 MFU | 1624574 tok/s
step 15121/19560 | loss 3.308015 (-0.19z)| norm 0.3044 (+0.51z)| lr 7.84e-05 | 322.77 ms | 52.3% bf16 MFU | 1624562 tok/s
step 15122/19560 | loss 3.269627 (-1.04z)| norm 0.2890 (-0.11z)| lr 7.83e-05 | 322.85 ms | 52.3% bf16 MFU | 1624531 tok/s
step 15123/19560 | loss 3.319485 (+0.09z)| norm 0.2662 (-1.08z)| lr 7.83e-05 | 322.94 ms | 52.3% bf16 MFU | 1624480 tok/s
step 15124/19560 | loss 3.324730 (+0.21z)| norm 0.2822 (-0.37z)| lr 7.83e-05 | 323.17 ms | 52.2% bf16 MFU | 1624373 tok/s
step 15125/19560 | loss 3.407395 (+2.04z)| norm 0.2601 (-1.39z)| lr 7.82e-05 | 322.31 ms | 52.4% bf16 MFU | 1624486 tok/s
step 15126/19560 | loss 3.327479 (+0.25z)| norm 0.2723 (-0.85z)| lr 7.82e-05 | 322.57 ms | 52.3% bf16 MFU | 1624529 tok/s
step 15127/19560 | loss 3.301542 (-0.34z)| norm 0.2982 (+0.49z)| lr 7.82e-05 | 322.45 ms | 52.3% bf16 MFU | 1624599 tok/s
step 15128/19560 | loss 3.258536 (-1.29z)| norm 0.2541 (-1.76z)| lr 7.81e-05 | 322.64 ms | 52.3% bf16 MFU | 1624617 tok/s
step 15129/19560 | loss 3.258567 (-1.27z)| norm 0.2674 (-1.11z)| lr 7.81e-05 | 322.73 ms | 52.3% bf16 MFU | 1624613 tok/s
step 15130/19560 | loss 3.285120 (-0.68z)| norm 0.2701 (-0.95z)| lr 7.81e-05 | 323.17 ms | 52.2% bf16 MFU | 1624498 tok/s
step 15131/19560 | loss 3.310804 (-0.11z)| norm 0.2744 (-0.70z)| lr 7.80e-05 | 323.20 ms | 52.2% bf16 MFU | 1624382 tok/s
step 15132/19560 | loss 3.307254 (-0.18z)| norm 0.2523 (-1.89z)| lr 7.80e-05 | 322.53 ms | 52.3% bf16 MFU | 1624441 tok/s
step 15133/19560 | loss 3.285039 (-0.68z)| norm 0.2742 (-0.68z)| lr 7.80e-05 | 322.34 ms | 52.4% bf16 MFU | 1624545 tok/s
step 15134/19560 | loss 3.309337 (-0.12z)| norm 0.2578 (-1.56z)| lr 7.79e-05 | 323.13 ms | 52.2% bf16 MFU | 1624445 tok/s
step 15135/19560 | loss 3.335919 (+0.48z)| norm 0.2926 (+0.36z)| lr 7.79e-05 | 322.49 ms | 52.3% bf16 MFU | 1624510 tok/s
step 15136/19560 | loss 3.275663 (-0.87z)| norm 0.2717 (-0.79z)| lr 7.79e-05 | 322.67 ms | 52.3% bf16 MFU | 1624527 tok/s
step 15137/19560 | loss 3.288311 (-0.57z)| norm 0.2892 (+0.20z)| lr 7.78e-05 | 323.03 ms | 52.2% bf16 MFU | 1624452 tok/s
step 15138/19560 | loss 3.376027 (+1.39z)| norm 0.2801 (-0.31z)| lr 7.78e-05 | 322.61 ms | 52.3% bf16 MFU | 1624486 tok/s
step 15139/19560 | loss 3.254207 (-1.34z)| norm 0.2725 (-0.72z)| lr 7.78e-05 | 322.71 ms | 52.3% bf16 MFU | 1624495 tok/s
step 15140/19560 | loss 3.215520 (-2.15z)| norm 0.2867 (+0.08z)| lr 7.77e-05 | 323.50 ms | 52.2% bf16 MFU | 1624304 tok/s
step 15141/19560 | loss 3.345826 (+0.72z)| norm 0.2906 (+0.29z)| lr 7.77e-05 | 323.18 ms | 52.2% bf16 MFU | 1624203 tok/s
step 15142/19560 | loss 3.241550 (-1.56z)| norm 0.2944 (+0.52z)| lr 7.77e-05 | 322.64 ms | 52.3% bf16 MFU | 1624244 tok/s
step 15143/19560 | loss 3.332605 (+0.45z)| norm 0.2770 (-0.47z)| lr 7.76e-05 | 322.49 ms | 52.3% bf16 MFU | 1624319 tok/s
step 15144/19560 | loss 3.334171 (+0.49z)| norm 0.2895 (+0.25z)| lr 7.76e-05 | 322.68 ms | 52.3% bf16 MFU | 1624343 tok/s
step 15145/19560 | loss 3.268782 (-0.95z)| norm 0.2643 (-1.19z)| lr 7.76e-05 | 322.70 ms | 52.3% bf16 MFU | 1624361 tok/s
step 15146/19560 | loss 3.249202 (-1.37z)| norm 0.2637 (-1.20z)| lr 7.75e-05 | 323.01 ms | 52.2% bf16 MFU | 1624298 tok/s
step 15147/19560 | loss 3.307849 (-0.07z)| norm 0.3138 (+1.65z)| lr 7.75e-05 | 322.59 ms | 52.3% bf16 MFU | 1624345 tok/s
step 15148/19560 | loss 3.390685 (+1.72z)| norm 0.2651 (-1.11z)| lr 7.75e-05 | 322.57 ms | 52.3% bf16 MFU | 1624396 tok/s
step 15149/19560 | loss 3.314230 (+0.05z)| norm 0.2922 (+0.42z)| lr 7.74e-05 | 322.74 ms | 52.3% bf16 MFU | 1624400 tok/s
step 15150/19560 | loss 3.285735 (-0.57z)| norm 0.3072 (+1.27z)| lr 7.74e-05 | 323.02 ms | 52.2% bf16 MFU | 1624334 tok/s
step 15151/19560 | loss 3.295038 (-0.38z)| norm 0.2615 (-1.31z)| lr 7.74e-05 | 322.25 ms | 52.4% bf16 MFU | 1624464 tok/s
step 15152/19560 | loss 3.256570 (-1.20z)| norm 0.2955 (+0.61z)| lr 7.73e-05 | 322.34 ms | 52.4% bf16 MFU | 1624566 tok/s
step 15153/19560 | loss 3.265973 (-0.99z)| norm 0.2586 (-1.45z)| lr 7.73e-05 | 322.68 ms | 52.3% bf16 MFU | 1624577 tok/s
step 15154/19560 | loss 3.331362 (+0.52z)| norm 0.2819 (-0.12z)| lr 7.73e-05 | 322.77 ms | 52.3% bf16 MFU | 1624565 tok/s
step 15155/19560 | loss 3.439355 (+3.03z)| norm 0.2892 (+0.32z)| lr 7.72e-05 | 321.94 ms | 52.4% bf16 MFU | 1624762 tok/s
step 15156/19560 | loss 3.296359 (-0.33z)| norm 0.2724 (-0.66z)| lr 7.72e-05 | 322.45 ms | 52.3% bf16 MFU | 1624822 tok/s
step 15157/19560 | loss 3.298110 (-0.29z)| norm 0.3023 (+1.09z)| lr 7.72e-05 | 322.65 ms | 52.3% bf16 MFU | 1624828 tok/s
step 15158/19560 | loss 3.296868 (-0.31z)| norm 0.3122 (+1.64z)| lr 7.71e-05 | 322.45 ms | 52.3% bf16 MFU | 1624885 tok/s
step 15159/19560 | loss 3.372080 (+1.43z)| norm 0.2768 (-0.41z)| lr 7.71e-05 | 322.94 ms | 52.3% bf16 MFU | 1624814 tok/s
step 15160/19560 | loss 3.295197 (-0.37z)| norm 0.2992 (+0.90z)| lr 7.71e-05 | 322.41 ms | 52.3% bf16 MFU | 1624879 tok/s
step 15161/19560 | loss 3.328921 (+0.42z)| norm 0.2833 (-0.01z)| lr 7.70e-05 | 322.81 ms | 52.3% bf16 MFU | 1624843 tok/s
step 15162/19560 | loss 3.314336 (+0.07z)| norm 0.2802 (-0.19z)| lr 7.70e-05 | 323.10 ms | 52.2% bf16 MFU | 1624734 tok/s
step 15163/19560 | loss 3.274830 (-0.86z)| norm 0.2651 (-1.08z)| lr 7.70e-05 | 322.75 ms | 52.3% bf16 MFU | 1624720 tok/s
step 15164/19560 | loss 3.271286 (-0.93z)| norm 0.2760 (-0.42z)| lr 7.69e-05 | 322.75 ms | 52.3% bf16 MFU | 1624705 tok/s
step 15165/19560 | loss 3.281482 (-0.68z)| norm 0.2638 (-1.13z)| lr 7.69e-05 | 322.70 ms | 52.3% bf16 MFU | 1624705 tok/s
step 15166/19560 | loss 3.299334 (-0.26z)| norm 0.2646 (-1.07z)| lr 7.69e-05 | 323.38 ms | 52.2% bf16 MFU | 1624533 tok/s
step 15167/19560 | loss 3.338740 (+0.68z)| norm 0.2687 (-0.82z)| lr 7.68e-05 | 322.61 ms | 52.3% bf16 MFU | 1624564 tok/s
step 15168/19560 | loss 3.309043 (-0.02z)| norm 0.2811 (-0.07z)| lr 7.68e-05 | 323.19 ms | 52.2% bf16 MFU | 1624448 tok/s
step 15169/19560 | loss 3.332043 (+0.52z)| norm 0.2741 (-0.49z)| lr 7.68e-05 | 322.94 ms | 52.3% bf16 MFU | 1624400 tok/s
step 15170/19560 | loss 3.305979 (-0.10z)| norm 0.2561 (-1.56z)| lr 7.67e-05 | 322.86 ms | 52.3% bf16 MFU | 1624375 tok/s
step 15171/19560 | loss 3.266536 (-1.02z)| norm 0.2790 (-0.16z)| lr 7.67e-05 | 322.62 ms | 52.3% bf16 MFU | 1624410 tok/s
step 15172/19560 | loss 3.298154 (-0.28z)| norm 0.2872 (+0.34z)| lr 7.67e-05 | 323.15 ms | 52.2% bf16 MFU | 1624311 tok/s
step 15173/19560 | loss 3.310501 (+0.00z)| norm 0.2817 (-0.01z)| lr 7.66e-05 | 322.65 ms | 52.3% bf16 MFU | 1624344 tok/s
step 15174/19560 | loss 3.336463 (+0.62z)| norm 0.2728 (-0.54z)| lr 7.66e-05 | 322.97 ms | 52.3% bf16 MFU | 1624293 tok/s
step 15175/19560 | loss 3.275934 (-0.81z)| norm 0.3052 (+1.42z)| lr 7.66e-05 | 322.71 ms | 52.3% bf16 MFU | 1624311 tok/s
step 15176/19560 | loss 3.271324 (-0.91z)| norm 0.2714 (-0.63z)| lr 7.65e-05 | 322.75 ms | 52.3% bf16 MFU | 1624318 tok/s
step 15177/19560 | loss 3.297692 (-0.26z)| norm 0.2959 (+0.84z)| lr 7.65e-05 | 322.42 ms | 52.3% bf16 MFU | 1624408 tok/s
step 15178/19560 | loss 3.338856 (+0.72z)| norm 0.2698 (-0.74z)| lr 7.65e-05 | 323.10 ms | 52.2% bf16 MFU | 1624323 tok/s
step 15179/19560 | loss 3.337569 (+0.68z)| norm 0.2782 (-0.22z)| lr 7.64e-05 | 323.83 ms | 52.1% bf16 MFU | 1624059 tok/s
step 15180/19560 | loss 3.382039 (+1.74z)| norm 0.2937 (+0.71z)| lr 7.64e-05 | 322.95 ms | 52.3% bf16 MFU | 1624028 tok/s
step 15181/19560 | loss 3.321132 (+0.26z)| norm 0.2739 (-0.49z)| lr 7.64e-05 | 322.39 ms | 52.3% bf16 MFU | 1624138 tok/s
step 15182/19560 | loss 3.321879 (+0.30z)| norm 0.2569 (-1.50z)| lr 7.63e-05 | 323.52 ms | 52.2% bf16 MFU | 1623961 tok/s
step 15183/19560 | loss 3.374825 (+1.58z)| norm 0.2715 (-0.61z)| lr 7.63e-05 | 322.30 ms | 52.4% bf16 MFU | 1624098 tok/s
step 15184/19560 | loss 3.316201 (+0.14z)| norm 0.2678 (-0.84z)| lr 7.63e-05 | 323.36 ms | 52.2% bf16 MFU | 1623961 tok/s
step 15185/19560 | loss 3.306857 (-0.10z)| norm 0.2800 (-0.09z)| lr 7.62e-05 | 322.68 ms | 52.3% bf16 MFU | 1624002 tok/s
step 15186/19560 | loss 3.276575 (-0.84z)| norm 0.2561 (-1.53z)| lr 7.62e-05 | 322.68 ms | 52.3% bf16 MFU | 1624042 tok/s
step 15187/19560 | loss 3.307976 (-0.05z)| norm 0.2642 (-1.02z)| lr 7.62e-05 | 323.64 ms | 52.1% bf16 MFU | 1623839 tok/s
step 15188/19560 | loss 3.312163 (+0.05z)| norm 0.2710 (-0.61z)| lr 7.61e-05 | 322.57 ms | 52.3% bf16 MFU | 1623915 tok/s
step 15189/19560 | loss 3.284129 (-0.64z)| norm 0.2712 (-0.60z)| lr 7.61e-05 | 322.71 ms | 52.3% bf16 MFU | 1623951 tok/s
step 15190/19560 | loss 3.314688 (+0.15z)| norm 0.2687 (-0.74z)| lr 7.61e-05 | 322.83 ms | 52.3% bf16 MFU | 1623955 tok/s
step 15191/19560 | loss 3.322936 (+0.38z)| norm 0.2670 (-0.83z)| lr 7.60e-05 | 323.83 ms | 52.1% bf16 MFU | 1623709 tok/s
step 15192/19560 | loss 3.305765 (-0.08z)| norm 0.2611 (-1.17z)| lr 7.60e-05 | 323.20 ms | 52.2% bf16 MFU | 1623633 tok/s
step 15193/19560 | loss 3.292844 (-0.41z)| norm 0.2728 (-0.47z)| lr 7.60e-05 | 322.67 ms | 52.3% bf16 MFU | 1623693 tok/s
step 15194/19560 | loss 3.334220 (+0.68z)| norm 0.2558 (-1.46z)| lr 7.59e-05 | 322.66 ms | 52.3% bf16 MFU | 1623752 tok/s
step 15195/19560 | loss 3.356357 (+1.25z)| norm 0.2661 (-0.85z)| lr 7.59e-05 | 323.17 ms | 52.2% bf16 MFU | 1623682 tok/s
step 15196/19560 | loss 3.282118 (-0.70z)| norm 0.2701 (-0.62z)| lr 7.59e-05 | 322.68 ms | 52.3% bf16 MFU | 1623738 tok/s
step 15197/19560 | loss 3.323629 (+0.41z)| norm 0.2759 (-0.28z)| lr 7.58e-05 | 322.81 ms | 52.3% bf16 MFU | 1623759 tok/s
step 15198/19560 | loss 3.323187 (+0.40z)| norm 0.2844 (+0.23z)| lr 7.58e-05 | 322.90 ms | 52.3% bf16 MFU | 1623755 tok/s
step 15199/19560 | loss 3.350101 (+1.10z)| norm 0.2807 (+0.01z)| lr 7.58e-05 | 322.91 ms | 52.3% bf16 MFU | 1623750 tok/s
step 15200/19560 | loss 3.287691 (-0.55z)| norm 0.2657 (-0.87z)| lr 7.57e-05 | 323.15 ms | 52.2% bf16 MFU | 1623683 tok/s
step 15201/19560 | loss 3.373653 (+1.74z)| norm 0.2817 (+0.11z)| lr 7.57e-05 | 323.12 ms | 52.2% bf16 MFU | 1623628 tok/s
step 15202/19560 | loss 3.326251 (+0.49z)| norm 0.2805 (+0.04z)| lr 7.57e-05 | 322.23 ms | 52.4% bf16 MFU | 1623799 tok/s
step 15203/19560 | loss 3.363500 (+1.46z)| norm 0.2739 (-0.36z)| lr 7.56e-05 | 322.96 ms | 52.3% bf16 MFU | 1623779 tok/s
step 15204/19560 | loss 3.318198 (+0.24z)| norm 0.2762 (-0.19z)| lr 7.56e-05 | 323.21 ms | 52.2% bf16 MFU | 1623696 tok/s
step 15205/19560 | loss 3.279642 (-0.81z)| norm 0.2530 (-1.66z)| lr 7.56e-05 | 323.04 ms | 52.2% bf16 MFU | 1623662 tok/s
step 15206/19560 | loss 3.345854 (+0.98z)| norm 0.3018 (+1.50z)| lr 7.55e-05 | 323.27 ms | 52.2% bf16 MFU | 1623571 tok/s
step 15207/19560 | loss 3.311642 (+0.06z)| norm 0.2545 (-1.57z)| lr 7.55e-05 | 322.57 ms | 52.3% bf16 MFU | 1623659 tok/s
step 15208/19560 | loss 3.281972 (-0.77z)| norm 0.2690 (-0.60z)| lr 7.55e-05 | 323.12 ms | 52.2% bf16 MFU | 1623605 tok/s
step 15209/19560 | loss 3.331983 (+0.59z)| norm 0.3185 (+2.64z)| lr 7.54e-05 | 323.16 ms | 52.2% bf16 MFU | 1623542 tok/s
step 15210/19560 | loss 3.316501 (+0.16z)| norm 0.2677 (-0.68z)| lr 7.54e-05 | 322.84 ms | 52.3% bf16 MFU | 1623565 tok/s
step 15211/19560 | loss 3.391823 (+2.19z)| norm 0.3000 (+1.46z)| lr 7.54e-05 | 322.98 ms | 52.3% bf16 MFU | 1623550 tok/s
step 15212/19560 | loss 3.366264 (+1.47z)| norm 0.3220 (+2.97z)| lr 7.53e-05 | 322.39 ms | 52.4% bf16 MFU | 1623685 tok/s
step 15213/19560 | loss 3.368487 (+1.51z)| norm 0.2870 (+0.61z)| lr 7.53e-05 | 322.77 ms | 52.3% bf16 MFU | 1623717 tok/s
step 15214/19560 | loss 3.306843 (-0.14z)| norm 0.2981 (+1.35z)| lr 7.53e-05 | 322.89 ms | 52.3% bf16 MFU | 1623719 tok/s
step 15215/19560 | loss 3.344565 (+0.86z)| norm 0.2738 (-0.28z)| lr 7.52e-05 | 323.46 ms | 52.2% bf16 MFU | 1623577 tok/s
step 15216/19560 | loss 3.242756 (-1.83z)| norm 0.2882 (+0.68z)| lr 7.52e-05 | 322.96 ms | 52.3% bf16 MFU | 1623566 tok/s
step 15217/19560 | loss 3.330384 (+0.48z)| norm 0.2745 (-0.24z)| lr 7.52e-05 | 322.68 ms | 52.3% bf16 MFU | 1623627 tok/s
step 15218/19560 | loss 3.373233 (+1.59z)| norm 0.2972 (+1.26z)| lr 7.51e-05 | 322.93 ms | 52.3% bf16 MFU | 1623623 tok/s
step 15219/19560 | loss 3.363741 (+1.32z)| norm 0.3035 (+1.65z)| lr 7.51e-05 | 322.71 ms | 52.3% bf16 MFU | 1623673 tok/s
step 15220/19560 | loss 3.281473 (-0.86z)| norm 0.2885 (+0.65z)| lr 7.51e-05 | 323.46 ms | 52.2% bf16 MFU | 1623534 tok/s
step 15221/19560 | loss 3.270804 (-1.13z)| norm 0.2998 (+1.40z)| lr 7.50e-05 | 322.32 ms | 52.4% bf16 MFU | 1623687 tok/s
step 15222/19560 | loss 3.325529 (+0.31z)| norm 0.2769 (-0.12z)| lr 7.50e-05 | 322.89 ms | 52.3% bf16 MFU | 1623689 tok/s
step 15223/19560 | loss 3.326130 (+0.35z)| norm 0.2930 (+0.96z)| lr 7.50e-05 | 323.30 ms | 52.2% bf16 MFU | 1623588 tok/s
step 15224/19560 | loss 3.289474 (-0.65z)| norm 0.2677 (-0.73z)| lr 7.49e-05 | 322.65 ms | 52.3% bf16 MFU | 1623655 tok/s
step 15225/19560 | loss 3.339760 (+0.71z)| norm 0.2750 (-0.24z)| lr 7.49e-05 | 323.47 ms | 52.2% bf16 MFU | 1623514 tok/s
step 15226/19560 | loss 3.238889 (-1.98z)| norm 0.2885 (+0.65z)| lr 7.49e-05 | 323.12 ms | 52.2% bf16 MFU | 1623466 tok/s
step 15227/19560 | loss 3.363068 (+1.32z)| norm 0.2872 (+0.55z)| lr 7.48e-05 | 322.31 ms | 52.4% bf16 MFU | 1623625 tok/s
step 15228/19560 | loss 3.265866 (-1.27z)| norm 0.3028 (+1.58z)| lr 7.48e-05 | 322.71 ms | 52.3% bf16 MFU | 1623677 tok/s
step 15229/19560 | loss 3.310460 (-0.08z)| norm 0.2925 (+0.88z)| lr 7.48e-05 | 322.90 ms | 52.3% bf16 MFU | 1623679 tok/s
step 15230/19560 | loss 3.349427 (+0.94z)| norm 0.2794 (+0.01z)| lr 7.47e-05 | 323.11 ms | 52.2% bf16 MFU | 1623626 tok/s
step 15231/19560 | loss 3.311642 (-0.06z)| norm 0.2813 (+0.12z)| lr 7.47e-05 | 322.59 ms | 52.3% bf16 MFU | 1623706 tok/s
step 15232/19560 | loss 3.285033 (-0.76z)| norm 0.2822 (+0.19z)| lr 7.47e-05 | 322.91 ms | 52.3% bf16 MFU | 1623702 tok/s
step 15233/19560 | loss 3.294700 (-0.50z)| norm 0.2734 (-0.40z)| lr 7.46e-05 | 323.44 ms | 52.2% bf16 MFU | 1623565 tok/s
step 15234/19560 | loss 3.376242 (+1.64z)| norm 0.3065 (+1.79z)| lr 7.46e-05 | 322.66 ms | 52.3% bf16 MFU | 1623631 tok/s
step 15235/19560 | loss 3.347072 (+0.86z)| norm 0.2656 (-0.93z)| lr 7.46e-05 | 323.27 ms | 52.2% bf16 MFU | 1623540 tok/s
step 15236/19560 | loss 3.363432 (+1.27z)| norm 0.3049 (+1.65z)| lr 7.45e-05 | 323.02 ms | 52.2% bf16 MFU | 1623518 tok/s
step 15237/19560 | loss 3.333039 (+0.47z)| norm 0.2994 (+1.27z)| lr 7.45e-05 | 323.05 ms | 52.2% bf16 MFU | 1623488 tok/s
step 15238/19560 | loss 3.273262 (-1.06z)| norm 0.2866 (+0.43z)| lr 7.45e-05 | 323.15 ms | 52.2% bf16 MFU | 1623435 tok/s
step 15239/19560 | loss 3.308081 (-0.16z)| norm 0.2972 (+1.12z)| lr 7.44e-05 | 323.04 ms | 52.2% bf16 MFU | 1623412 tok/s
step 15240/19560 | loss 3.343467 (+0.75z)| norm 0.2633 (-1.08z)| lr 7.44e-05 | 323.16 ms | 52.2% bf16 MFU | 1623362 tok/s
step 15241/19560 | loss 3.290964 (-0.60z)| norm 0.2639 (-1.04z)| lr 7.44e-05 | 322.01 ms | 52.4% bf16 MFU | 1623602 tok/s
step 15242/19560 | loss 3.280447 (-0.87z)| norm 0.2737 (-0.41z)| lr 7.43e-05 | 322.59 ms | 52.3% bf16 MFU | 1623684 tok/s
step 15243/19560 | loss 3.273691 (-1.04z)| norm 0.2711 (-0.57z)| lr 7.43e-05 | 322.36 ms | 52.4% bf16 MFU | 1623819 tok/s
step 15244/19560 | loss 3.328596 (+0.37z)| norm 0.2820 (+0.14z)| lr 7.43e-05 | 322.40 ms | 52.3% bf16 MFU | 1623937 tok/s
step 15245/19560 | loss 3.302629 (-0.30z)| norm 0.2798 (+0.01z)| lr 7.42e-05 | 323.04 ms | 52.2% bf16 MFU | 1623890 tok/s
step 15246/19560 | loss 3.361895 (+1.27z)| norm 0.2875 (+0.53z)| lr 7.42e-05 | 322.43 ms | 52.3% bf16 MFU | 1623998 tok/s
step 15247/19560 | loss 3.343822 (+0.80z)| norm 0.2549 (-1.62z)| lr 7.42e-05 | 322.53 ms | 52.3% bf16 MFU | 1624076 tok/s
step 15248/19560 | loss 3.348194 (+0.90z)| norm 0.2708 (-0.57z)| lr 7.41e-05 | 322.52 ms | 52.3% bf16 MFU | 1624152 tok/s
step 15249/19560 | loss 3.325857 (+0.31z)| norm 0.2705 (-0.58z)| lr 7.41e-05 | 322.85 ms | 52.3% bf16 MFU | 1624141 tok/s
step 15250/19560 | loss 3.290964 (-0.61z)| norm 0.2503 (-1.87z)| lr 7.41e-05 | 322.77 ms | 52.3% bf16 MFU | 1624151 tok/s
val loss 3.299448
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3018/10042 = 0.300538
step 15251/19560 | loss 3.294781 (-0.51z)| norm 0.2598 (-1.24z)| lr 7.41e-05 | 321.72 ms | 52.5% bf16 MFU | 1624425 tok/s
step 15252/19560 | loss 3.298092 (-0.42z)| norm 0.2602 (-1.20z)| lr 7.40e-05 | 322.43 ms | 52.3% bf16 MFU | 1624507 tok/s
step 15253/19560 | loss 3.388398 (+1.98z)| norm 0.2867 (+0.51z)| lr 7.40e-05 | 323.02 ms | 52.2% bf16 MFU | 1624436 tok/s
step 15254/19560 | loss 3.274502 (-1.03z)| norm 0.2606 (-1.18z)| lr 7.40e-05 | 322.45 ms | 52.3% bf16 MFU | 1624512 tok/s
step 15255/19560 | loss 3.289401 (-0.63z)| norm 0.2650 (-0.88z)| lr 7.39e-05 | 322.81 ms | 52.3% bf16 MFU | 1624494 tok/s
step 15256/19560 | loss 3.330617 (+0.45z)| norm 0.2607 (-1.17z)| lr 7.39e-05 | 322.74 ms | 52.3% bf16 MFU | 1624495 tok/s
step 15257/19560 | loss 3.316682 (+0.07z)| norm 0.3052 (+1.71z)| lr 7.39e-05 | 322.75 ms | 52.3% bf16 MFU | 1624492 tok/s
step 15258/19560 | loss 3.295410 (-0.51z)| norm 0.2592 (-1.27z)| lr 7.38e-05 | 322.82 ms | 52.3% bf16 MFU | 1624473 tok/s
step 15259/19560 | loss 3.291161 (-0.62z)| norm 0.3071 (+1.79z)| lr 7.38e-05 | 323.02 ms | 52.2% bf16 MFU | 1624404 tok/s
step 15260/19560 | loss 3.282780 (-0.84z)| norm 0.2680 (-0.72z)| lr 7.38e-05 | 322.76 ms | 52.3% bf16 MFU | 1624404 tok/s
step 15261/19560 | loss 3.365700 (+1.36z)| norm 0.2804 (+0.08z)| lr 7.37e-05 | 322.37 ms | 52.4% bf16 MFU | 1624502 tok/s
step 15262/19560 | loss 3.262933 (-1.36z)| norm 0.2917 (+0.79z)| lr 7.37e-05 | 322.85 ms | 52.3% bf16 MFU | 1624474 tok/s
step 15263/19560 | loss 3.258400 (-1.45z)| norm 0.2947 (+0.98z)| lr 7.37e-05 | 322.13 ms | 52.4% bf16 MFU | 1624628 tok/s
step 15264/19560 | loss 3.269503 (-1.16z)| norm 0.2662 (-0.86z)| lr 7.36e-05 | 322.56 ms | 52.3% bf16 MFU | 1624668 tok/s
step 15265/19560 | loss 3.315083 (+0.03z)| norm 0.3206 (+2.58z)| lr 7.36e-05 | 323.47 ms | 52.2% bf16 MFU | 1624476 tok/s
step 15266/19560 | loss 3.276855 (-0.96z)| norm 0.3029 (+1.44z)| lr 7.36e-05 | 322.78 ms | 52.3% bf16 MFU | 1624465 tok/s
step 15267/19560 | loss 3.286198 (-0.72z)| norm 0.2918 (+0.73z)| lr 7.35e-05 | 322.20 ms | 52.4% bf16 MFU | 1624604 tok/s
step 15268/19560 | loss 3.320119 (+0.17z)| norm 0.3174 (+2.27z)| lr 7.35e-05 | 322.56 ms | 52.3% bf16 MFU | 1624644 tok/s
step 15269/19560 | loss 3.252386 (-1.66z)| norm 0.2658 (-0.87z)| lr 7.35e-05 | 322.75 ms | 52.3% bf16 MFU | 1624633 tok/s
step 15270/19560 | loss 3.324053 (+0.28z)| norm 0.2596 (-1.23z)| lr 7.34e-05 | 322.72 ms | 52.3% bf16 MFU | 1624630 tok/s
step 15271/19560 | loss 3.303825 (-0.27z)| norm 0.2754 (-0.27z)| lr 7.34e-05 | 322.17 ms | 52.4% bf16 MFU | 1624766 tok/s
step 15272/19560 | loss 3.320745 (+0.20z)| norm 0.2683 (-0.69z)| lr 7.34e-05 | 322.54 ms | 52.3% bf16 MFU | 1624802 tok/s
step 15273/19560 | loss 3.425535 (+2.97z)| norm 0.2515 (-1.69z)| lr 7.33e-05 | 322.49 ms | 52.3% bf16 MFU | 1624850 tok/s
step 15274/19560 | loss 3.321769 (+0.17z)| norm 0.2703 (-0.56z)| lr 7.33e-05 | 323.06 ms | 52.2% bf16 MFU | 1624752 tok/s
step 15275/19560 | loss 3.360687 (+1.21z)| norm 0.2780 (-0.08z)| lr 7.33e-05 | 322.93 ms | 52.3% bf16 MFU | 1624691 tok/s
step 15276/19560 | loss 3.280678 (-0.94z)| norm 0.2553 (-1.46z)| lr 7.32e-05 | 322.67 ms | 52.3% bf16 MFU | 1624698 tok/s
step 15277/19560 | loss 3.334030 (+0.52z)| norm 0.2730 (-0.37z)| lr 7.32e-05 | 322.45 ms | 52.3% bf16 MFU | 1624760 tok/s
step 15278/19560 | loss 3.303944 (-0.31z)| norm 0.2831 (+0.26z)| lr 7.32e-05 | 323.19 ms | 52.2% bf16 MFU | 1624633 tok/s
step 15279/19560 | loss 3.339145 (+0.64z)| norm 0.2825 (+0.22z)| lr 7.31e-05 | 322.56 ms | 52.3% bf16 MFU | 1624672 tok/s
step 15280/19560 | loss 3.425552 (+2.91z)| norm 0.3045 (+1.57z)| lr 7.31e-05 | 322.57 ms | 52.3% bf16 MFU | 1624706 tok/s
step 15281/19560 | loss 3.311158 (-0.17z)| norm 0.2836 (+0.27z)| lr 7.31e-05 | 322.84 ms | 52.3% bf16 MFU | 1624671 tok/s
step 15282/19560 | loss 3.278026 (-1.04z)| norm 0.2879 (+0.53z)| lr 7.30e-05 | 323.09 ms | 52.2% bf16 MFU | 1624574 tok/s
step 15283/19560 | loss 3.317092 (+0.03z)| norm 0.2786 (-0.04z)| lr 7.30e-05 | 323.09 ms | 52.2% bf16 MFU | 1624482 tok/s
step 15284/19560 | loss 3.284541 (-0.88z)| norm 0.2842 (+0.30z)| lr 7.30e-05 | 323.11 ms | 52.2% bf16 MFU | 1624388 tok/s
step 15285/19560 | loss 3.373597 (+1.59z)| norm 0.3130 (+2.07z)| lr 7.29e-05 | 322.44 ms | 52.3% bf16 MFU | 1624469 tok/s
step 15286/19560 | loss 3.345785 (+0.80z)| norm 0.2699 (-0.58z)| lr 7.29e-05 | 322.43 ms | 52.3% bf16 MFU | 1624548 tok/s
step 15287/19560 | loss 3.245441 (-1.94z)| norm 0.3109 (+1.95z)| lr 7.29e-05 | 322.83 ms | 52.3% bf16 MFU | 1624522 tok/s
step 15288/19560 | loss 3.302840 (-0.36z)| norm 0.2811 (+0.12z)| lr 7.28e-05 | 322.63 ms | 52.3% bf16 MFU | 1624548 tok/s
step 15289/19560 | loss 3.311500 (-0.12z)| norm 0.2810 (+0.11z)| lr 7.28e-05 | 322.92 ms | 52.3% bf16 MFU | 1624499 tok/s
step 15290/19560 | loss 3.318246 (+0.07z)| norm 0.3071 (+1.70z)| lr 7.28e-05 | 322.36 ms | 52.4% bf16 MFU | 1624594 tok/s
step 15291/19560 | loss 3.336018 (+0.55z)| norm 0.3029 (+1.42z)| lr 7.27e-05 | 322.07 ms | 52.4% bf16 MFU | 1624758 tok/s
step 15292/19560 | loss 3.322754 (+0.17z)| norm 0.3052 (+1.53z)| lr 7.27e-05 | 322.72 ms | 52.3% bf16 MFU | 1624751 tok/s
step 15293/19560 | loss 3.328971 (+0.33z)| norm 0.2986 (+1.12z)| lr 7.27e-05 | 322.87 ms | 52.3% bf16 MFU | 1624705 tok/s
step 15294/19560 | loss 3.324356 (+0.20z)| norm 0.2999 (+1.17z)| lr 7.26e-05 | 323.00 ms | 52.3% bf16 MFU | 1624628 tok/s
step 15295/19560 | loss 3.291273 (-0.72z)| norm 0.3199 (+2.31z)| lr 7.26e-05 | 322.45 ms | 52.3% bf16 MFU | 1624694 tok/s
step 15296/19560 | loss 3.295128 (-0.60z)| norm 0.2764 (-0.26z)| lr 7.26e-05 | 322.45 ms | 52.3% bf16 MFU | 1624758 tok/s
step 15297/19560 | loss 3.398558 (+2.23z)| norm 0.2782 (-0.16z)| lr 7.25e-05 | 322.30 ms | 52.4% bf16 MFU | 1624855 tok/s
step 15298/19560 | loss 3.262085 (-1.49z)| norm 0.2743 (-0.40z)| lr 7.25e-05 | 323.11 ms | 52.2% bf16 MFU | 1624744 tok/s
step 15299/19560 | loss 3.346103 (+0.78z)| norm 0.2873 (+0.37z)| lr 7.25e-05 | 322.57 ms | 52.3% bf16 MFU | 1624774 tok/s
step 15300/19560 | loss 3.348770 (+0.84z)| norm 0.2587 (-1.32z)| lr 7.24e-05 | 322.54 ms | 52.3% bf16 MFU | 1624810 tok/s
step 15301/19560 | loss 3.288762 (-0.79z)| norm 0.2543 (-1.55z)| lr 7.24e-05 | 322.48 ms | 52.3% bf16 MFU | 1624860 tok/s
step 15302/19560 | loss 3.335927 (+0.49z)| norm 0.2761 (-0.27z)| lr 7.24e-05 | 322.70 ms | 52.3% bf16 MFU | 1624852 tok/s
step 15303/19560 | loss 3.320034 (+0.05z)| norm 0.2904 (+0.58z)| lr 7.23e-05 | 322.77 ms | 52.3% bf16 MFU | 1624827 tok/s
step 15304/19560 | loss 3.321013 (+0.07z)| norm 0.2683 (-0.72z)| lr 7.23e-05 | 322.60 ms | 52.3% bf16 MFU | 1624845 tok/s
step 15305/19560 | loss 3.297588 (-0.58z)| norm 0.2767 (-0.22z)| lr 7.23e-05 | 323.22 ms | 52.2% bf16 MFU | 1624707 tok/s
step 15306/19560 | loss 3.366534 (+1.31z)| norm 0.2847 (+0.25z)| lr 7.23e-05 | 322.56 ms | 52.3% bf16 MFU | 1624741 tok/s
step 15307/19560 | loss 3.296649 (-0.60z)| norm 0.2874 (+0.40z)| lr 7.22e-05 | 322.68 ms | 52.3% bf16 MFU | 1624743 tok/s
step 15308/19560 | loss 3.321570 (+0.10z)| norm 0.2862 (+0.34z)| lr 7.22e-05 | 323.36 ms | 52.2% bf16 MFU | 1624575 tok/s
step 15309/19560 | loss 3.343668 (+0.71z)| norm 0.2467 (-1.97z)| lr 7.22e-05 | 323.36 ms | 52.2% bf16 MFU | 1624415 tok/s
step 15310/19560 | loss 3.302859 (-0.42z)| norm 0.2757 (-0.28z)| lr 7.21e-05 | 322.87 ms | 52.3% bf16 MFU | 1624385 tok/s
step 15311/19560 | loss 3.340312 (+0.63z)| norm 0.2853 (+0.28z)| lr 7.21e-05 | 322.33 ms | 52.4% bf16 MFU | 1624493 tok/s
step 15312/19560 | loss 3.349220 (+0.87z)| norm 0.2649 (-0.92z)| lr 7.21e-05 | 323.02 ms | 52.2% bf16 MFU | 1624422 tok/s
step 15313/19560 | loss 3.384783 (+1.82z)| norm 0.3088 (+1.64z)| lr 7.20e-05 | 322.78 ms | 52.3% bf16 MFU | 1624414 tok/s
step 15314/19560 | loss 3.355747 (+1.00z)| norm 0.2873 (+0.37z)| lr 7.20e-05 | 322.69 ms | 52.3% bf16 MFU | 1624431 tok/s
step 15315/19560 | loss 3.301489 (-0.48z)| norm 0.3028 (+1.26z)| lr 7.20e-05 | 322.89 ms | 52.3% bf16 MFU | 1624395 tok/s
step 15316/19560 | loss 3.293881 (-0.69z)| norm 0.2916 (+0.59z)| lr 7.19e-05 | 323.10 ms | 52.2% bf16 MFU | 1624311 tok/s
step 15317/19560 | loss 3.296680 (-0.62z)| norm 0.2642 (-1.01z)| lr 7.19e-05 | 322.25 ms | 52.4% bf16 MFU | 1624444 tok/s
step 15318/19560 | loss 3.281338 (-1.03z)| norm 0.2740 (-0.44z)| lr 7.19e-05 | 322.70 ms | 52.3% bf16 MFU | 1624457 tok/s
step 15319/19560 | loss 3.293621 (-0.68z)| norm 0.2867 (+0.30z)| lr 7.18e-05 | 322.35 ms | 52.4% bf16 MFU | 1624557 tok/s
step 15320/19560 | loss 3.292960 (-0.70z)| norm 0.3065 (+1.44z)| lr 7.18e-05 | 322.08 ms | 52.4% bf16 MFU | 1624720 tok/s
step 15321/19560 | loss 3.292481 (-0.71z)| norm 0.2912 (+0.54z)| lr 7.18e-05 | 322.66 ms | 52.3% bf16 MFU | 1624728 tok/s
step 15322/19560 | loss 3.307484 (-0.29z)| norm 0.3109 (+1.66z)| lr 7.17e-05 | 322.64 ms | 52.3% bf16 MFU | 1624741 tok/s
step 15323/19560 | loss 3.353870 (+0.97z)| norm 0.3106 (+1.61z)| lr 7.17e-05 | 322.63 ms | 52.3% bf16 MFU | 1624756 tok/s
step 15324/19560 | loss 3.346272 (+0.75z)| norm 0.2814 (-0.09z)| lr 7.17e-05 | 322.74 ms | 52.3% bf16 MFU | 1624742 tok/s
step 15325/19560 | loss 3.371138 (+1.41z)| norm 0.3419 (+3.26z)| lr 7.16e-05 | 322.90 ms | 52.3% bf16 MFU | 1624689 tok/s
step 15326/19560 | loss 3.316764 (-0.06z)| norm 0.2751 (-0.47z)| lr 7.16e-05 | 322.32 ms | 52.4% bf16 MFU | 1624786 tok/s
step 15327/19560 | loss 3.343944 (+0.68z)| norm 0.2934 (+0.55z)| lr 7.16e-05 | 322.34 ms | 52.4% bf16 MFU | 1624872 tok/s
step 15328/19560 | loss 3.284784 (-0.93z)| norm 0.2978 (+0.78z)| lr 7.15e-05 | 323.47 ms | 52.2% bf16 MFU | 1624670 tok/s
step 15329/19560 | loss 3.313370 (-0.14z)| norm 0.2714 (-0.68z)| lr 7.15e-05 | 322.77 ms | 52.3% bf16 MFU | 1624654 tok/s
step 15330/19560 | loss 3.311621 (-0.19z)| norm 0.2873 (+0.20z)| lr 7.15e-05 | 322.49 ms | 52.3% bf16 MFU | 1624709 tok/s
step 15331/19560 | loss 3.316537 (-0.04z)| norm 0.2944 (+0.59z)| lr 7.14e-05 | 322.85 ms | 52.3% bf16 MFU | 1624671 tok/s
step 15332/19560 | loss 3.316746 (-0.04z)| norm 0.2688 (-0.84z)| lr 7.14e-05 | 322.44 ms | 52.3% bf16 MFU | 1624738 tok/s
step 15333/19560 | loss 3.339746 (+0.59z)| norm 0.2972 (+0.73z)| lr 7.14e-05 | 322.86 ms | 52.3% bf16 MFU | 1624696 tok/s
step 15334/19560 | loss 3.315047 (-0.09z)| norm 0.2680 (-0.89z)| lr 7.13e-05 | 323.40 ms | 52.2% bf16 MFU | 1624519 tok/s
step 15335/19560 | loss 3.350718 (+0.89z)| norm 0.2904 (+0.35z)| lr 7.13e-05 | 322.78 ms | 52.3% bf16 MFU | 1624508 tok/s
step 15336/19560 | loss 3.306879 (-0.33z)| norm 0.2853 (+0.06z)| lr 7.13e-05 | 322.68 ms | 52.3% bf16 MFU | 1624522 tok/s
step 15337/19560 | loss 3.316825 (-0.05z)| norm 0.2778 (-0.36z)| lr 7.12e-05 | 322.43 ms | 52.3% bf16 MFU | 1624598 tok/s
step 15338/19560 | loss 3.236669 (-2.21z)| norm 0.2797 (-0.26z)| lr 7.12e-05 | 323.36 ms | 52.2% bf16 MFU | 1624438 tok/s
step 15339/19560 | loss 3.308762 (-0.24z)| norm 0.2875 (+0.20z)| lr 7.12e-05 | 322.48 ms | 52.3% bf16 MFU | 1624506 tok/s
step 15340/19560 | loss 3.358623 (+1.14z)| norm 0.2792 (-0.27z)| lr 7.11e-05 | 322.49 ms | 52.3% bf16 MFU | 1624569 tok/s
step 15341/19560 | loss 3.325711 (+0.24z)| norm 0.3491 (+3.64z)| lr 7.11e-05 | 322.49 ms | 52.3% bf16 MFU | 1624628 tok/s
step 15342/19560 | loss 3.270584 (-1.28z)| norm 0.2803 (-0.21z)| lr 7.11e-05 | 323.35 ms | 52.2% bf16 MFU | 1624467 tok/s
step 15343/19560 | loss 3.316060 (-0.01z)| norm 0.3113 (+1.50z)| lr 7.11e-05 | 322.71 ms | 52.3% bf16 MFU | 1624475 tok/s
step 15344/19560 | loss 3.293383 (-0.66z)| norm 0.3127 (+1.56z)| lr 7.10e-05 | 322.77 ms | 52.3% bf16 MFU | 1624468 tok/s
step 15345/19560 | loss 3.310471 (-0.18z)| norm 0.2802 (-0.24z)| lr 7.10e-05 | 322.69 ms | 52.3% bf16 MFU | 1624483 tok/s
step 15346/19560 | loss 3.345662 (+0.83z)| norm 0.3088 (+1.33z)| lr 7.10e-05 | 323.46 ms | 52.2% bf16 MFU | 1624303 tok/s
step 15347/19560 | loss 3.261568 (-1.53z)| norm 0.3049 (+1.11z)| lr 7.09e-05 | 322.51 ms | 52.3% bf16 MFU | 1624370 tok/s
step 15348/19560 | loss 3.322832 (+0.19z)| norm 0.2749 (-0.53z)| lr 7.09e-05 | 322.44 ms | 52.3% bf16 MFU | 1624452 tok/s
step 15349/19560 | loss 3.284119 (-0.91z)| norm 0.2869 (+0.14z)| lr 7.09e-05 | 322.94 ms | 52.3% bf16 MFU | 1624404 tok/s
step 15350/19560 | loss 3.337865 (+0.62z)| norm 0.2799 (-0.25z)| lr 7.08e-05 | 322.30 ms | 52.4% bf16 MFU | 1624520 tok/s
step 15351/19560 | loss 3.307393 (-0.25z)| norm 0.2911 (+0.36z)| lr 7.08e-05 | 323.24 ms | 52.2% bf16 MFU | 1624392 tok/s
step 15352/19560 | loss 3.295884 (-0.58z)| norm 0.3073 (+1.24z)| lr 7.08e-05 | 322.98 ms | 52.3% bf16 MFU | 1624337 tok/s
step 15353/19560 | loss 3.333461 (+0.50z)| norm 0.2660 (-1.03z)| lr 7.07e-05 | 322.83 ms | 52.3% bf16 MFU | 1624321 tok/s
step 15354/19560 | loss 3.263476 (-1.52z)| norm 0.2756 (-0.49z)| lr 7.07e-05 | 322.90 ms | 52.3% bf16 MFU | 1624289 tok/s
step 15355/19560 | loss 3.355654 (+1.14z)| norm 0.2661 (-1.00z)| lr 7.07e-05 | 322.64 ms | 52.3% bf16 MFU | 1624324 tok/s
step 15356/19560 | loss 3.323478 (+0.20z)| norm 0.2781 (-0.34z)| lr 7.06e-05 | 323.61 ms | 52.2% bf16 MFU | 1624115 tok/s
step 15357/19560 | loss 3.300827 (-0.46z)| norm 0.2795 (-0.25z)| lr 7.06e-05 | 322.50 ms | 52.3% bf16 MFU | 1624194 tok/s
step 15358/19560 | loss 3.343298 (+0.78z)| norm 0.2632 (-1.14z)| lr 7.06e-05 | 322.59 ms | 52.3% bf16 MFU | 1624245 tok/s
step 15359/19560 | loss 3.300252 (-0.47z)| norm 0.2594 (-1.32z)| lr 7.05e-05 | 322.84 ms | 52.3% bf16 MFU | 1624232 tok/s
step 15360/19560 | loss 3.371311 (+1.57z)| norm 0.2761 (-0.42z)| lr 7.05e-05 | 322.96 ms | 52.3% bf16 MFU | 1624189 tok/s
step 15361/19560 | loss 3.288242 (-0.83z)| norm 0.2747 (-0.50z)| lr 7.05e-05 | 322.84 ms | 52.3% bf16 MFU | 1624178 tok/s
step 15362/19560 | loss 3.357328 (+1.18z)| norm 0.3080 (+1.31z)| lr 7.04e-05 | 322.16 ms | 52.4% bf16 MFU | 1624340 tok/s
step 15363/19560 | loss 3.240865 (-2.16z)| norm 0.2983 (+0.77z)| lr 7.04e-05 | 322.56 ms | 52.3% bf16 MFU | 1624393 tok/s
step 15364/19560 | loss 3.280631 (-1.00z)| norm 0.2779 (-0.33z)| lr 7.04e-05 | 323.72 ms | 52.1% bf16 MFU | 1624153 tok/s
step 15365/19560 | loss 3.236044 (-2.22z)| norm 0.3185 (+1.86z)| lr 7.03e-05 | 322.19 ms | 52.4% bf16 MFU | 1624308 tok/s
step 15366/19560 | loss 3.300853 (-0.40z)| norm 0.2812 (-0.15z)| lr 7.03e-05 | 322.96 ms | 52.3% bf16 MFU | 1624263 tok/s
step 15367/19560 | loss 3.293277 (-0.61z)| norm 0.2996 (+0.84z)| lr 7.03e-05 | 323.21 ms | 52.2% bf16 MFU | 1624156 tok/s
step 15368/19560 | loss 3.333365 (+0.53z)| norm 0.3006 (+0.88z)| lr 7.02e-05 | 322.59 ms | 52.3% bf16 MFU | 1624210 tok/s
step 15369/19560 | loss 3.338366 (+0.66z)| norm 0.2991 (+0.78z)| lr 7.02e-05 | 323.00 ms | 52.3% bf16 MFU | 1624158 tok/s
step 15370/19560 | loss 3.266900 (-1.36z)| norm 0.2861 (+0.08z)| lr 7.02e-05 | 322.98 ms | 52.3% bf16 MFU | 1624114 tok/s
step 15371/19560 | loss 3.291215 (-0.68z)| norm 0.2785 (-0.34z)| lr 7.02e-05 | 323.45 ms | 52.2% bf16 MFU | 1623955 tok/s
step 15372/19560 | loss 3.265333 (-1.39z)| norm 0.3162 (+1.67z)| lr 7.01e-05 | 322.72 ms | 52.3% bf16 MFU | 1623988 tok/s
step 15373/19560 | loss 3.320567 (+0.16z)| norm 0.2836 (-0.08z)| lr 7.01e-05 | 322.51 ms | 52.3% bf16 MFU | 1624072 tok/s
step 15374/19560 | loss 3.319616 (+0.15z)| norm 0.2814 (-0.19z)| lr 7.01e-05 | 323.29 ms | 52.2% bf16 MFU | 1623954 tok/s
step 15375/19560 | loss 3.263851 (-1.41z)| norm 0.2927 (+0.40z)| lr 7.00e-05 | 322.77 ms | 52.3% bf16 MFU | 1623974 tok/s
step 15376/19560 | loss 3.290117 (-0.66z)| norm 0.3078 (+1.20z)| lr 7.00e-05 | 323.93 ms | 52.1% bf16 MFU | 1623702 tok/s
step 15377/19560 | loss 3.295142 (-0.51z)| norm 0.3046 (+1.01z)| lr 7.00e-05 | 322.37 ms | 52.4% bf16 MFU | 1623833 tok/s
step 15378/19560 | loss 3.318438 (+0.14z)| norm 0.3019 (+0.85z)| lr 6.99e-05 | 322.54 ms | 52.3% bf16 MFU | 1623918 tok/s
step 15379/19560 | loss 3.284743 (-0.81z)| norm 0.3651 (+4.01z)| lr 6.99e-05 | 323.17 ms | 52.2% bf16 MFU | 1623839 tok/s
step 15380/19560 | loss 3.333811 (+0.57z)| norm 0.3013 (+0.72z)| lr 6.99e-05 | 322.79 ms | 52.3% bf16 MFU | 1623859 tok/s
step 15381/19560 | loss 3.262278 (-1.43z)| norm 0.2950 (+0.39z)| lr 6.98e-05 | 323.16 ms | 52.2% bf16 MFU | 1623784 tok/s
step 15382/19560 | loss 3.297159 (-0.45z)| norm 0.3262 (+1.96z)| lr 6.98e-05 | 322.81 ms | 52.3% bf16 MFU | 1623802 tok/s
step 15383/19560 | loss 3.264198 (-1.38z)| norm 0.2752 (-0.66z)| lr 6.98e-05 | 322.24 ms | 52.4% bf16 MFU | 1623962 tok/s
step 15384/19560 | loss 3.269223 (-1.21z)| norm 0.3004 (+0.63z)| lr 6.97e-05 | 322.89 ms | 52.3% bf16 MFU | 1623951 tok/s
step 15385/19560 | loss 3.353480 (+1.16z)| norm 0.3007 (+0.64z)| lr 6.97e-05 | 322.70 ms | 52.3% bf16 MFU | 1623988 tok/s
step 15386/19560 | loss 3.320436 (+0.22z)| norm 0.2858 (-0.14z)| lr 6.97e-05 | 322.63 ms | 52.3% bf16 MFU | 1624040 tok/s
step 15387/19560 | loss 3.333998 (+0.60z)| norm 0.3136 (+1.31z)| lr 6.96e-05 | 322.30 ms | 52.4% bf16 MFU | 1624172 tok/s
step 15388/19560 | loss 3.277978 (-0.98z)| norm 0.2803 (-0.44z)| lr 6.96e-05 | 322.69 ms | 52.3% bf16 MFU | 1624201 tok/s
step 15389/19560 | loss 3.393161 (+2.23z)| norm 0.3195 (+1.58z)| lr 6.96e-05 | 322.87 ms | 52.3% bf16 MFU | 1624182 tok/s
step 15390/19560 | loss 3.400519 (+2.37z)| norm 0.2702 (-0.96z)| lr 6.95e-05 | 323.55 ms | 52.2% bf16 MFU | 1623995 tok/s
step 15391/19560 | loss 3.308096 (-0.18z)| norm 0.2932 (+0.23z)| lr 6.95e-05 | 322.83 ms | 52.3% bf16 MFU | 1623998 tok/s
step 15392/19560 | loss 3.394405 (+2.16z)| norm 0.3099 (+1.07z)| lr 6.95e-05 | 323.40 ms | 52.2% bf16 MFU | 1623858 tok/s
step 15393/19560 | loss 3.351272 (+0.97z)| norm 0.2606 (-1.45z)| lr 6.94e-05 | 322.96 ms | 52.3% bf16 MFU | 1623833 tok/s
step 15394/19560 | loss 3.287667 (-0.77z)| norm 0.2811 (-0.38z)| lr 6.94e-05 | 322.70 ms | 52.3% bf16 MFU | 1623877 tok/s
step 15395/19560 | loss 3.277529 (-1.04z)| norm 0.2845 (-0.20z)| lr 6.94e-05 | 323.42 ms | 52.2% bf16 MFU | 1623738 tok/s
step 15396/19560 | loss 3.348262 (+0.88z)| norm 0.2598 (-1.46z)| lr 6.94e-05 | 322.54 ms | 52.3% bf16 MFU | 1623826 tok/s
step 15397/19560 | loss 3.285059 (-0.86z)| norm 0.2775 (-0.55z)| lr 6.93e-05 | 322.79 ms | 52.3% bf16 MFU | 1623847 tok/s
step 15398/19560 | loss 3.335925 (+0.54z)| norm 0.2926 (+0.22z)| lr 6.93e-05 | 322.59 ms | 52.3% bf16 MFU | 1623917 tok/s
step 15399/19560 | loss 3.327876 (+0.31z)| norm 0.2710 (-0.90z)| lr 6.93e-05 | 322.91 ms | 52.3% bf16 MFU | 1623903 tok/s
step 15400/19560 | loss 3.276300 (-1.09z)| norm 0.2852 (-0.17z)| lr 6.92e-05 | 323.48 ms | 52.2% bf16 MFU | 1623746 tok/s
step 15401/19560 | loss 3.328180 (+0.36z)| norm 0.2970 (+0.44z)| lr 6.92e-05 | 323.12 ms | 52.2% bf16 MFU | 1623686 tok/s
step 15402/19560 | loss 3.276047 (-1.10z)| norm 0.2722 (-0.89z)| lr 6.92e-05 | 322.49 ms | 52.3% bf16 MFU | 1623789 tok/s
step 15403/19560 | loss 3.286435 (-0.79z)| norm 0.3133 (+1.29z)| lr 6.91e-05 | 322.90 ms | 52.3% bf16 MFU | 1623783 tok/s
step 15404/19560 | loss 3.358457 (+1.22z)| norm 0.2616 (-1.47z)| lr 6.91e-05 | 322.91 ms | 52.3% bf16 MFU | 1623776 tok/s
step 15405/19560 | loss 3.317216 (+0.06z)| norm 0.2738 (-0.82z)| lr 6.91e-05 | 322.55 ms | 52.3% bf16 MFU | 1623858 tok/s
step 15406/19560 | loss 3.289768 (-0.71z)| norm 0.3212 (+1.68z)| lr 6.90e-05 | 322.97 ms | 52.3% bf16 MFU | 1623831 tok/s
step 15407/19560 | loss 3.286584 (-0.79z)| norm 0.2683 (-1.11z)| lr 6.90e-05 | 322.54 ms | 52.3% bf16 MFU | 1623915 tok/s
step 15408/19560 | loss 3.314810 (+0.04z)| norm 0.2815 (-0.40z)| lr 6.90e-05 | 322.85 ms | 52.3% bf16 MFU | 1623916 tok/s
step 15409/19560 | loss 3.264786 (-1.41z)| norm 0.2649 (-1.26z)| lr 6.89e-05 | 322.53 ms | 52.3% bf16 MFU | 1623999 tok/s
step 15410/19560 | loss 3.318213 (+0.14z)| norm 0.2872 (-0.09z)| lr 6.89e-05 | 322.99 ms | 52.3% bf16 MFU | 1623961 tok/s
step 15411/19560 | loss 3.353457 (+1.15z)| norm 0.2685 (-1.06z)| lr 6.89e-05 | 323.00 ms | 52.3% bf16 MFU | 1623921 tok/s
step 15412/19560 | loss 3.287249 (-0.77z)| norm 0.2724 (-0.85z)| lr 6.88e-05 | 322.45 ms | 52.3% bf16 MFU | 1624022 tok/s
step 15413/19560 | loss 3.320418 (+0.21z)| norm 0.2732 (-0.80z)| lr 6.88e-05 | 323.03 ms | 52.2% bf16 MFU | 1623973 tok/s
step 15414/19560 | loss 3.237660 (-2.17z)| norm 0.2782 (-0.54z)| lr 6.88e-05 | 323.56 ms | 52.2% bf16 MFU | 1623792 tok/s
step 15415/19560 | loss 3.333554 (+0.60z)| norm 0.2740 (-0.75z)| lr 6.87e-05 | 323.00 ms | 52.3% bf16 MFU | 1623762 tok/s
step 15416/19560 | loss 3.394025 (+2.30z)| norm 0.3097 (+1.11z)| lr 6.87e-05 | 323.03 ms | 52.2% bf16 MFU | 1623725 tok/s
step 15417/19560 | loss 3.302297 (-0.33z)| norm 0.2870 (-0.08z)| lr 6.87e-05 | 322.72 ms | 52.3% bf16 MFU | 1623769 tok/s
step 15418/19560 | loss 3.300204 (-0.39z)| norm 0.2755 (-0.67z)| lr 6.86e-05 | 322.48 ms | 52.3% bf16 MFU | 1623871 tok/s
step 15419/19560 | loss 3.325223 (+0.33z)| norm 0.2717 (-0.86z)| lr 6.86e-05 | 322.81 ms | 52.3% bf16 MFU | 1623884 tok/s
step 15420/19560 | loss 3.384106 (+1.98z)| norm 0.2892 (+0.07z)| lr 6.86e-05 | 322.77 ms | 52.3% bf16 MFU | 1623908 tok/s
step 15421/19560 | loss 3.264338 (-1.39z)| norm 0.2741 (-0.71z)| lr 6.86e-05 | 322.91 ms | 52.3% bf16 MFU | 1623895 tok/s
step 15422/19560 | loss 3.290356 (-0.65z)| norm 0.2973 (+0.51z)| lr 6.85e-05 | 322.15 ms | 52.4% bf16 MFU | 1624073 tok/s
step 15423/19560 | loss 3.277060 (-1.02z)| norm 0.2867 (-0.04z)| lr 6.85e-05 | 322.86 ms | 52.3% bf16 MFU | 1624064 tok/s
step 15424/19560 | loss 3.359601 (+1.28z)| norm 0.2967 (+0.48z)| lr 6.85e-05 | 323.01 ms | 52.2% bf16 MFU | 1624017 tok/s
step 15425/19560 | loss 3.345845 (+0.92z)| norm 0.2837 (-0.21z)| lr 6.84e-05 | 321.90 ms | 52.4% bf16 MFU | 1624254 tok/s
step 15426/19560 | loss 3.281983 (-0.90z)| norm 0.2836 (-0.22z)| lr 6.84e-05 | 323.10 ms | 52.2% bf16 MFU | 1624176 tok/s
step 15427/19560 | loss 3.296087 (-0.49z)| norm 0.2815 (-0.33z)| lr 6.84e-05 | 322.44 ms | 52.3% bf16 MFU | 1624266 tok/s
step 15428/19560 | loss 3.308825 (-0.12z)| norm 0.2740 (-0.74z)| lr 6.83e-05 | 322.76 ms | 52.3% bf16 MFU | 1624273 tok/s
step 15429/19560 | loss 3.348823 (+1.02z)| norm 0.2681 (-1.07z)| lr 6.83e-05 | 322.89 ms | 52.3% bf16 MFU | 1624245 tok/s
step 15430/19560 | loss 3.370595 (+1.62z)| norm 0.2818 (-0.34z)| lr 6.83e-05 | 322.34 ms | 52.4% bf16 MFU | 1624358 tok/s
step 15431/19560 | loss 3.326978 (+0.38z)| norm 0.2661 (-1.17z)| lr 6.82e-05 | 322.42 ms | 52.3% bf16 MFU | 1624446 tok/s
step 15432/19560 | loss 3.289661 (-0.67z)| norm 0.2769 (-0.59z)| lr 6.82e-05 | 323.12 ms | 52.2% bf16 MFU | 1624353 tok/s
step 15433/19560 | loss 3.336066 (+0.63z)| norm 0.2829 (-0.27z)| lr 6.82e-05 | 322.72 ms | 52.3% bf16 MFU | 1624366 tok/s
step 15434/19560 | loss 3.328850 (+0.44z)| norm 0.2805 (-0.40z)| lr 6.81e-05 | 322.38 ms | 52.4% bf16 MFU | 1624462 tok/s
step 15435/19560 | loss 3.303176 (-0.29z)| norm 0.2966 (+0.47z)| lr 6.81e-05 | 322.33 ms | 52.4% bf16 MFU | 1624567 tok/s
step 15436/19560 | loss 3.297921 (-0.44z)| norm 0.2878 (-0.00z)| lr 6.81e-05 | 322.44 ms | 52.3% bf16 MFU | 1624639 tok/s
step 15437/19560 | loss 3.368805 (+1.57z)| norm 0.2961 (+0.43z)| lr 6.80e-05 | 322.91 ms | 52.3% bf16 MFU | 1624588 tok/s
step 15438/19560 | loss 3.309936 (-0.10z)| norm 0.2937 (+0.29z)| lr 6.80e-05 | 322.48 ms | 52.3% bf16 MFU | 1624648 tok/s
step 15439/19560 | loss 3.301069 (-0.35z)| norm 0.3303 (+2.25z)| lr 6.80e-05 | 322.51 ms | 52.3% bf16 MFU | 1624697 tok/s
step 15440/19560 | loss 3.352606 (+1.12z)| norm 0.3132 (+1.30z)| lr 6.80e-05 | 323.29 ms | 52.2% bf16 MFU | 1624549 tok/s
step 15441/19560 | loss 3.277440 (-1.01z)| norm 0.2681 (-1.13z)| lr 6.79e-05 | 322.25 ms | 52.4% bf16 MFU | 1624670 tok/s
step 15442/19560 | loss 3.356840 (+1.28z)| norm 0.5160 (+8.28z)| lr 6.79e-05 | 322.35 ms | 52.4% bf16 MFU | 1624758 tok/s
step 15443/19560 | loss 3.350005 (+1.07z)| norm 0.3085 (+0.65z)| lr 6.79e-05 | 322.42 ms | 52.3% bf16 MFU | 1624825 tok/s
step 15444/19560 | loss 3.343282 (+0.86z)| norm 0.3978 (+3.69z)| lr 6.78e-05 | 322.16 ms | 52.4% bf16 MFU | 1624953 tok/s
step 15445/19560 | loss 3.338972 (+0.73z)| norm 0.2758 (-0.55z)| lr 6.78e-05 | 323.44 ms | 52.2% bf16 MFU | 1624755 tok/s
step 15446/19560 | loss 3.322905 (+0.26z)| norm 0.2919 (+0.00z)| lr 6.78e-05 | 322.23 ms | 52.4% bf16 MFU | 1624870 tok/s
step 15447/19560 | loss 3.339322 (+0.72z)| norm 0.3115 (+0.68z)| lr 6.77e-05 | 322.92 ms | 52.3% bf16 MFU | 1624807 tok/s
step 15448/19560 | loss 3.342025 (+0.79z)| norm 0.2604 (-1.08z)| lr 6.77e-05 | 322.47 ms | 52.3% bf16 MFU | 1624860 tok/s
step 15449/19560 | loss 3.297905 (-0.48z)| norm 0.2879 (-0.13z)| lr 6.77e-05 | 322.61 ms | 52.3% bf16 MFU | 1624873 tok/s
step 15450/19560 | loss 3.302294 (-0.35z)| norm 0.2748 (-0.57z)| lr 6.76e-05 | 322.10 ms | 52.4% bf16 MFU | 1625015 tok/s
step 15451/19560 | loss 3.309471 (-0.14z)| norm 0.2779 (-0.46z)| lr 6.76e-05 | 322.94 ms | 52.3% bf16 MFU | 1624938 tok/s
step 15452/19560 | loss 3.345998 (+0.91z)| norm 0.2867 (-0.15z)| lr 6.76e-05 | 322.47 ms | 52.3% bf16 MFU | 1624983 tok/s
step 15453/19560 | loss 3.335218 (+0.62z)| norm 0.2661 (-0.85z)| lr 6.75e-05 | 322.57 ms | 52.3% bf16 MFU | 1625002 tok/s
step 15454/19560 | loss 3.363934 (+1.43z)| norm 0.3170 (+0.92z)| lr 6.75e-05 | 322.63 ms | 52.3% bf16 MFU | 1625004 tok/s
step 15455/19560 | loss 3.344882 (+0.88z)| norm 0.2801 (-0.37z)| lr 6.75e-05 | 322.75 ms | 52.3% bf16 MFU | 1624975 tok/s
step 15456/19560 | loss 3.360657 (+1.31z)| norm 0.2777 (-0.45z)| lr 6.74e-05 | 322.62 ms | 52.3% bf16 MFU | 1624981 tok/s
step 15457/19560 | loss 3.324382 (+0.27z)| norm 0.2745 (-0.56z)| lr 6.74e-05 | 322.49 ms | 52.3% bf16 MFU | 1625018 tok/s
step 15458/19560 | loss 3.389022 (+2.07z)| norm 0.2832 (-0.26z)| lr 6.74e-05 | 322.64 ms | 52.3% bf16 MFU | 1625017 tok/s
step 15459/19560 | loss 3.337989 (+0.63z)| norm 0.2889 (-0.06z)| lr 6.73e-05 | 323.29 ms | 52.2% bf16 MFU | 1624854 tok/s
step 15460/19560 | loss 3.280763 (-0.97z)| norm 0.2562 (-1.19z)| lr 6.73e-05 | 322.84 ms | 52.3% bf16 MFU | 1624809 tok/s
step 15461/19560 | loss 3.268225 (-1.30z)| norm 0.2742 (-0.56z)| lr 6.73e-05 | 322.23 ms | 52.4% bf16 MFU | 1624921 tok/s
step 15462/19560 | loss 3.291364 (-0.65z)| norm 0.2882 (-0.07z)| lr 6.73e-05 | 322.94 ms | 52.3% bf16 MFU | 1624850 tok/s
step 15463/19560 | loss 3.262861 (-1.42z)| norm 0.2593 (-1.07z)| lr 6.72e-05 | 322.79 ms | 52.3% bf16 MFU | 1624820 tok/s
step 15464/19560 | loss 3.279735 (-0.94z)| norm 0.2804 (-0.34z)| lr 6.72e-05 | 322.52 ms | 52.3% bf16 MFU | 1624859 tok/s
step 15465/19560 | loss 3.302840 (-0.30z)| norm 0.2952 (+0.17z)| lr 6.72e-05 | 322.49 ms | 52.3% bf16 MFU | 1624904 tok/s
step 15466/19560 | loss 3.320941 (+0.18z)| norm 0.2833 (-0.24z)| lr 6.71e-05 | 322.58 ms | 52.3% bf16 MFU | 1624923 tok/s
step 15467/19560 | loss 3.313227 (-0.03z)| norm 0.2825 (-0.27z)| lr 6.71e-05 | 323.02 ms | 52.2% bf16 MFU | 1624830 tok/s
step 15468/19560 | loss 3.246233 (-1.88z)| norm 0.2664 (-0.82z)| lr 6.71e-05 | 323.04 ms | 52.2% bf16 MFU | 1624739 tok/s
step 15469/19560 | loss 3.302305 (-0.31z)| norm 0.2685 (-0.74z)| lr 6.70e-05 | 322.91 ms | 52.3% bf16 MFU | 1624684 tok/s
step 15470/19560 | loss 3.383570 (+1.92z)| norm 0.2697 (-0.69z)| lr 6.70e-05 | 322.51 ms | 52.3% bf16 MFU | 1624733 tok/s
step 15471/19560 | loss 3.257597 (-1.54z)| norm 0.2690 (-0.70z)| lr 6.70e-05 | 322.43 ms | 52.3% bf16 MFU | 1624800 tok/s
step 15472/19560 | loss 3.257215 (-1.53z)| norm 0.2411 (-1.65z)| lr 6.69e-05 | 322.78 ms | 52.3% bf16 MFU | 1624775 tok/s
step 15473/19560 | loss 3.356747 (+1.16z)| norm 0.2597 (-0.99z)| lr 6.69e-05 | 322.55 ms | 52.3% bf16 MFU | 1624808 tok/s
step 15474/19560 | loss 3.311834 (-0.05z)| norm 0.2839 (-0.15z)| lr 6.69e-05 | 322.50 ms | 52.3% bf16 MFU | 1624851 tok/s
step 15475/19560 | loss 3.311355 (-0.07z)| norm 0.2823 (-0.20z)| lr 6.68e-05 | 322.80 ms | 52.3% bf16 MFU | 1624818 tok/s
step 15476/19560 | loss 3.304022 (-0.27z)| norm 0.2950 (+0.24z)| lr 6.68e-05 | 323.15 ms | 52.2% bf16 MFU | 1624698 tok/s
step 15477/19560 | loss 3.338295 (+0.66z)| norm 0.2556 (-1.12z)| lr 6.68e-05 | 322.76 ms | 52.3% bf16 MFU | 1624682 tok/s
step 15478/19560 | loss 3.265024 (-1.32z)| norm 0.3248 (+1.26z)| lr 6.68e-05 | 322.70 ms | 52.3% bf16 MFU | 1624683 tok/s
step 15479/19560 | loss 3.333060 (+0.52z)| norm 0.2738 (-0.49z)| lr 6.67e-05 | 322.50 ms | 52.3% bf16 MFU | 1624735 tok/s
step 15480/19560 | loss 3.343031 (+0.78z)| norm 0.2779 (-0.34z)| lr 6.67e-05 | 322.17 ms | 52.4% bf16 MFU | 1624866 tok/s
step 15481/19560 | loss 3.365903 (+1.39z)| norm 0.3489 (+2.05z)| lr 6.67e-05 | 322.70 ms | 52.3% bf16 MFU | 1624858 tok/s
step 15482/19560 | loss 3.342864 (+0.75z)| norm 0.2605 (-0.95z)| lr 6.66e-05 | 322.27 ms | 52.4% bf16 MFU | 1624958 tok/s
step 15483/19560 | loss 3.346300 (+0.85z)| norm 0.2989 (+0.35z)| lr 6.66e-05 | 322.34 ms | 52.4% bf16 MFU | 1625037 tok/s
step 15484/19560 | loss 3.277851 (-1.00z)| norm 0.3057 (+0.57z)| lr 6.66e-05 | 322.59 ms | 52.3% bf16 MFU | 1625047 tok/s
step 15485/19560 | loss 3.254693 (-1.60z)| norm 0.2796 (-0.31z)| lr 6.65e-05 | 322.50 ms | 52.3% bf16 MFU | 1625080 tok/s
step 15486/19560 | loss 3.305305 (-0.23z)| norm 0.2853 (-0.13z)| lr 6.65e-05 | 322.82 ms | 52.3% bf16 MFU | 1625030 tok/s
step 15487/19560 | loss 3.302796 (-0.30z)| norm 0.2995 (+0.34z)| lr 6.65e-05 | 322.48 ms | 52.3% bf16 MFU | 1625067 tok/s
step 15488/19560 | loss 3.345236 (+0.85z)| norm 0.2894 (-0.00z)| lr 6.64e-05 | 322.33 ms | 52.4% bf16 MFU | 1625142 tok/s
step 15489/19560 | loss 3.324184 (+0.27z)| norm 0.2678 (-0.74z)| lr 6.64e-05 | 322.62 ms | 52.3% bf16 MFU | 1625138 tok/s
step 15490/19560 | loss 3.300237 (-0.37z)| norm 0.2665 (-0.77z)| lr 6.64e-05 | 322.64 ms | 52.3% bf16 MFU | 1625131 tok/s
step 15491/19560 | loss 3.350171 (+0.98z)| norm 0.2916 (+0.09z)| lr 6.63e-05 | 322.75 ms | 52.3% bf16 MFU | 1625096 tok/s
step 15492/19560 | loss 3.275062 (-1.09z)| norm 0.2658 (-0.79z)| lr 6.63e-05 | 322.48 ms | 52.3% bf16 MFU | 1625132 tok/s
step 15493/19560 | loss 3.296583 (-0.52z)| norm 0.2882 (-0.02z)| lr 6.63e-05 | 322.76 ms | 52.3% bf16 MFU | 1625094 tok/s
step 15494/19560 | loss 3.306101 (-0.25z)| norm 0.3202 (+1.06z)| lr 6.62e-05 | 322.50 ms | 52.3% bf16 MFU | 1625123 tok/s
step 15495/19560 | loss 3.347191 (+0.89z)| norm 0.2768 (-0.41z)| lr 6.62e-05 | 322.94 ms | 52.3% bf16 MFU | 1625040 tok/s
step 15496/19560 | loss 3.237993 (-2.11z)| norm 0.3214 (+1.10z)| lr 6.62e-05 | 322.03 ms | 52.4% bf16 MFU | 1625190 tok/s
step 15497/19560 | loss 3.284107 (-0.83z)| norm 0.2888 (-0.00z)| lr 6.62e-05 | 322.59 ms | 52.3% bf16 MFU | 1625194 tok/s
step 15498/19560 | loss 3.318501 (+0.11z)| norm 0.2765 (-0.42z)| lr 6.61e-05 | 323.24 ms | 52.2% bf16 MFU | 1625032 tok/s
step 15499/19560 | loss 3.306947 (-0.22z)| norm 0.2956 (+0.22z)| lr 6.61e-05 | 322.60 ms | 52.3% bf16 MFU | 1625041 tok/s
step 15500/19560 | loss 3.315543 (+0.01z)| norm 0.2809 (-0.27z)| lr 6.61e-05 | 322.46 ms | 52.3% bf16 MFU | 1625085 tok/s
val loss 3.296848
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3033/10042 = 0.302031
step 15501/19560 | loss 3.282184 (-0.91z)| norm 0.2669 (-0.73z)| lr 6.60e-05 | 322.65 ms | 52.3% bf16 MFU | 1625078 tok/s
step 15502/19560 | loss 3.244999 (-1.90z)| norm 0.2988 (+0.34z)| lr 6.60e-05 | 322.71 ms | 52.3% bf16 MFU | 1625057 tok/s
step 15503/19560 | loss 3.313408 (-0.04z)| norm 0.2704 (-0.61z)| lr 6.60e-05 | 322.89 ms | 52.3% bf16 MFU | 1624992 tok/s
step 15504/19560 | loss 3.330112 (+0.42z)| norm 0.2759 (-0.42z)| lr 6.59e-05 | 322.94 ms | 52.3% bf16 MFU | 1624916 tok/s
step 15505/19560 | loss 3.231737 (-2.25z)| norm 0.3047 (+0.56z)| lr 6.59e-05 | 323.12 ms | 52.2% bf16 MFU | 1624799 tok/s
step 15506/19560 | loss 3.348282 (+0.91z)| norm 0.2727 (-0.52z)| lr 6.59e-05 | 322.95 ms | 52.3% bf16 MFU | 1624730 tok/s
step 15507/19560 | loss 3.338314 (+0.63z)| norm 0.3111 (+0.82z)| lr 6.58e-05 | 322.31 ms | 52.4% bf16 MFU | 1624826 tok/s
step 15508/19560 | loss 3.315588 (+0.02z)| norm 0.2747 (-0.44z)| lr 6.58e-05 | 323.17 ms | 52.2% bf16 MFU | 1624702 tok/s
step 15509/19560 | loss 3.335839 (+0.55z)| norm 0.2996 (+0.42z)| lr 6.58e-05 | 322.79 ms | 52.3% bf16 MFU | 1624679 tok/s
step 15510/19560 | loss 3.260414 (-1.49z)| norm 0.3032 (+0.56z)| lr 6.57e-05 | 322.21 ms | 52.4% bf16 MFU | 1624802 tok/s
step 15511/19560 | loss 3.381966 (+1.77z)| norm 0.2948 (+0.26z)| lr 6.57e-05 | 323.23 ms | 52.2% bf16 MFU | 1624662 tok/s
step 15512/19560 | loss 3.360322 (+1.17z)| norm 0.2846 (-0.09z)| lr 6.57e-05 | 322.89 ms | 52.3% bf16 MFU | 1624615 tok/s
step 15513/19560 | loss 3.315341 (-0.04z)| norm 0.2852 (-0.07z)| lr 6.57e-05 | 322.90 ms | 52.3% bf16 MFU | 1624567 tok/s
step 15514/19560 | loss 3.264465 (-1.39z)| norm 0.2674 (-0.69z)| lr 6.56e-05 | 322.20 ms | 52.4% bf16 MFU | 1624700 tok/s
step 15515/19560 | loss 3.262929 (-1.41z)| norm 0.2898 (+0.10z)| lr 6.56e-05 | 322.92 ms | 52.3% bf16 MFU | 1624644 tok/s
step 15516/19560 | loss 3.331639 (+0.42z)| norm 0.2581 (-1.00z)| lr 6.56e-05 | 322.83 ms | 52.3% bf16 MFU | 1624613 tok/s
step 15517/19560 | loss 3.347591 (+0.87z)| norm 0.2727 (-0.48z)| lr 6.55e-05 | 322.23 ms | 52.4% bf16 MFU | 1624737 tok/s
step 15518/19560 | loss 3.330818 (+0.43z)| norm 0.2995 (+0.46z)| lr 6.55e-05 | 322.86 ms | 52.3% bf16 MFU | 1624694 tok/s
step 15519/19560 | loss 3.366271 (+1.39z)| norm 0.2555 (-1.07z)| lr 6.55e-05 | 323.35 ms | 52.2% bf16 MFU | 1624531 tok/s
step 15520/19560 | loss 3.313786 (-0.03z)| norm 0.3003 (+0.49z)| lr 6.54e-05 | 322.76 ms | 52.3% bf16 MFU | 1624523 tok/s
step 15521/19560 | loss 3.273305 (-1.15z)| norm 0.2599 (-0.92z)| lr 6.54e-05 | 322.48 ms | 52.3% bf16 MFU | 1624587 tok/s
step 15522/19560 | loss 3.388946 (+2.04z)| norm 0.2747 (-0.40z)| lr 6.54e-05 | 322.46 ms | 52.3% bf16 MFU | 1624653 tok/s
step 15523/19560 | loss 3.327550 (+0.33z)| norm 0.2984 (+0.43z)| lr 6.53e-05 | 323.06 ms | 52.2% bf16 MFU | 1624565 tok/s
step 15524/19560 | loss 3.386227 (+1.93z)| norm 0.3130 (+0.92z)| lr 6.53e-05 | 322.53 ms | 52.3% bf16 MFU | 1624614 tok/s
step 15525/19560 | loss 3.340866 (+0.67z)| norm 0.2714 (-0.53z)| lr 6.53e-05 | 322.68 ms | 52.3% bf16 MFU | 1624624 tok/s
step 15526/19560 | loss 3.306020 (-0.28z)| norm 0.2704 (-0.56z)| lr 6.53e-05 | 322.63 ms | 52.3% bf16 MFU | 1624645 tok/s
step 15527/19560 | loss 3.359399 (+1.18z)| norm 0.3087 (+0.77z)| lr 6.52e-05 | 322.75 ms | 52.3% bf16 MFU | 1624634 tok/s
step 15528/19560 | loss 3.330968 (+0.39z)| norm 0.3038 (+0.59z)| lr 6.52e-05 | 322.94 ms | 52.3% bf16 MFU | 1624576 tok/s
step 15529/19560 | loss 3.357162 (+1.10z)| norm 0.3074 (+0.71z)| lr 6.52e-05 | 322.95 ms | 52.3% bf16 MFU | 1624519 tok/s
step 15530/19560 | loss 3.256123 (-1.65z)| norm 0.3358 (+1.66z)| lr 6.51e-05 | 322.56 ms | 52.3% bf16 MFU | 1624563 tok/s
step 15531/19560 | loss 3.299755 (-0.47z)| norm 0.2873 (+0.00z)| lr 6.51e-05 | 323.18 ms | 52.2% bf16 MFU | 1624449 tok/s
step 15532/19560 | loss 3.282850 (-0.91z)| norm 0.2988 (+0.39z)| lr 6.51e-05 | 322.69 ms | 52.3% bf16 MFU | 1624465 tok/s
step 15533/19560 | loss 3.303294 (-0.35z)| norm 0.2906 (+0.10z)| lr 6.50e-05 | 322.59 ms | 52.3% bf16 MFU | 1624504 tok/s
step 15534/19560 | loss 3.345183 (+0.78z)| norm 0.2651 (-0.77z)| lr 6.50e-05 | 322.65 ms | 52.3% bf16 MFU | 1624527 tok/s
step 15535/19560 | loss 3.337945 (+0.57z)| norm 0.3264 (+1.34z)| lr 6.50e-05 | 322.85 ms | 52.3% bf16 MFU | 1624496 tok/s
step 15536/19560 | loss 3.323241 (+0.17z)| norm 0.3281 (+1.38z)| lr 6.49e-05 | 322.52 ms | 52.3% bf16 MFU | 1624550 tok/s
step 15537/19560 | loss 3.294112 (-0.64z)| norm 0.2830 (-0.18z)| lr 6.49e-05 | 322.93 ms | 52.3% bf16 MFU | 1624498 tok/s
step 15538/19560 | loss 3.284894 (-0.88z)| norm 0.3884 (+3.27z)| lr 6.49e-05 | 322.97 ms | 52.3% bf16 MFU | 1624440 tok/s
step 15539/19560 | loss 3.327968 (+0.30z)| norm 0.3231 (+1.11z)| lr 6.48e-05 | 323.26 ms | 52.2% bf16 MFU | 1624312 tok/s
step 15540/19560 | loss 3.309868 (-0.20z)| norm 0.2921 (+0.08z)| lr 6.48e-05 | 322.91 ms | 52.3% bf16 MFU | 1624278 tok/s
step 15541/19560 | loss 3.277240 (-1.08z)| norm 0.3486 (+1.90z)| lr 6.48e-05 | 322.77 ms | 52.3% bf16 MFU | 1624281 tok/s
step 15542/19560 | loss 3.317564 (+0.01z)| norm 0.2881 (-0.07z)| lr 6.48e-05 | 322.87 ms | 52.3% bf16 MFU | 1624260 tok/s
step 15543/19560 | loss 3.313968 (-0.09z)| norm 0.2870 (-0.11z)| lr 6.47e-05 | 323.07 ms | 52.2% bf16 MFU | 1624188 tok/s
step 15544/19560 | loss 3.378313 (+1.72z)| norm 0.3662 (+2.40z)| lr 6.47e-05 | 323.43 ms | 52.2% bf16 MFU | 1624030 tok/s
step 15545/19560 | loss 3.346446 (+0.81z)| norm 0.2697 (-0.66z)| lr 6.47e-05 | 323.25 ms | 52.2% bf16 MFU | 1623926 tok/s
step 15546/19560 | loss 3.333279 (+0.44z)| norm 0.3013 (+0.33z)| lr 6.46e-05 | 322.49 ms | 52.3% bf16 MFU | 1624018 tok/s
step 15547/19560 | loss 3.291310 (-0.73z)| norm 0.2948 (+0.12z)| lr 6.46e-05 | 322.81 ms | 52.3% bf16 MFU | 1624024 tok/s
step 15548/19560 | loss 3.233962 (-2.29z)| norm 0.3058 (+0.46z)| lr 6.46e-05 | 322.93 ms | 52.3% bf16 MFU | 1624000 tok/s
step 15549/19560 | loss 3.330435 (+0.38z)| norm 0.2884 (-0.09z)| lr 6.45e-05 | 323.02 ms | 52.2% bf16 MFU | 1623953 tok/s
step 15550/19560 | loss 3.286788 (-0.84z)| norm 0.3091 (+0.56z)| lr 6.45e-05 | 323.58 ms | 52.2% bf16 MFU | 1623769 tok/s
step 15551/19560 | loss 3.297940 (-0.53z)| norm 0.3009 (+0.30z)| lr 6.45e-05 | 322.45 ms | 52.3% bf16 MFU | 1623878 tok/s
step 15552/19560 | loss 3.283747 (-0.92z)| norm 0.2816 (-0.31z)| lr 6.44e-05 | 322.80 ms | 52.3% bf16 MFU | 1623894 tok/s
step 15553/19560 | loss 3.258406 (-1.60z)| norm 0.3148 (+0.74z)| lr 6.44e-05 | 323.00 ms | 52.3% bf16 MFU | 1623858 tok/s
step 15554/19560 | loss 3.325418 (+0.27z)| norm 0.3005 (+0.28z)| lr 6.44e-05 | 322.93 ms | 52.3% bf16 MFU | 1623842 tok/s
step 15555/19560 | loss 3.187225 (-3.42z)| norm 0.2581 (-1.05z)| lr 6.44e-05 | 322.73 ms | 52.3% bf16 MFU | 1623878 tok/s
step 15556/19560 | loss 3.345971 (+0.82z)| norm 0.2852 (-0.20z)| lr 6.43e-05 | 322.91 ms | 52.3% bf16 MFU | 1623866 tok/s
step 15557/19560 | loss 3.270319 (-1.18z)| norm 0.2629 (-0.90z)| lr 6.43e-05 | 322.80 ms | 52.3% bf16 MFU | 1623881 tok/s
step 15558/19560 | loss 3.279578 (-0.92z)| norm 0.2942 (+0.08z)| lr 6.43e-05 | 323.05 ms | 52.2% bf16 MFU | 1623834 tok/s
step 15559/19560 | loss 3.299557 (-0.38z)| norm 0.2632 (-0.90z)| lr 6.42e-05 | 322.47 ms | 52.3% bf16 MFU | 1623936 tok/s
step 15560/19560 | loss 3.259266 (-1.45z)| norm 0.3017 (+0.31z)| lr 6.42e-05 | 322.94 ms | 52.3% bf16 MFU | 1623912 tok/s
step 15561/19560 | loss 3.333294 (+0.52z)| norm 0.2917 (-0.01z)| lr 6.42e-05 | 323.02 ms | 52.2% bf16 MFU | 1623871 tok/s
step 15562/19560 | loss 3.347846 (+0.90z)| norm 0.2990 (+0.22z)| lr 6.41e-05 | 322.80 ms | 52.3% bf16 MFU | 1623887 tok/s
step 15563/19560 | loss 3.275141 (-1.01z)| norm 0.3040 (+0.38z)| lr 6.41e-05 | 322.57 ms | 52.3% bf16 MFU | 1623960 tok/s
step 15564/19560 | loss 3.323999 (+0.27z)| norm 0.2607 (-0.98z)| lr 6.41e-05 | 322.96 ms | 52.3% bf16 MFU | 1623930 tok/s
step 15565/19560 | loss 3.345488 (+0.85z)| norm 0.3089 (+0.53z)| lr 6.40e-05 | 322.79 ms | 52.3% bf16 MFU | 1623947 tok/s
step 15566/19560 | loss 3.287883 (-0.68z)| norm 0.2725 (-0.61z)| lr 6.40e-05 | 323.02 ms | 52.2% bf16 MFU | 1623905 tok/s
step 15567/19560 | loss 3.258412 (-1.44z)| norm 0.2705 (-0.66z)| lr 6.40e-05 | 322.82 ms | 52.3% bf16 MFU | 1623913 tok/s
step 15568/19560 | loss 3.296114 (-0.44z)| norm 0.2857 (-0.17z)| lr 6.39e-05 | 322.84 ms | 52.3% bf16 MFU | 1623917 tok/s
step 15569/19560 | loss 3.441827 (+3.25z)| norm 0.3327 (+1.29z)| lr 6.39e-05 | 322.91 ms | 52.3% bf16 MFU | 1623903 tok/s
step 15570/19560 | loss 3.316841 (+0.08z)| norm 0.2790 (-0.43z)| lr 6.39e-05 | 322.91 ms | 52.3% bf16 MFU | 1623889 tok/s
step 15571/19560 | loss 3.299398 (-0.35z)| norm 0.3008 (+0.45z)| lr 6.39e-05 | 324.17 ms | 52.1% bf16 MFU | 1623559 tok/s
step 15572/19560 | loss 3.258006 (-1.39z)| norm 0.2759 (-0.56z)| lr 6.38e-05 | 322.63 ms | 52.3% bf16 MFU | 1623634 tok/s
step 15573/19560 | loss 3.301400 (-0.28z)| norm 0.2752 (-0.59z)| lr 6.38e-05 | 322.63 ms | 52.3% bf16 MFU | 1623705 tok/s
step 15574/19560 | loss 3.339476 (+0.69z)| norm 0.2937 (+0.21z)| lr 6.38e-05 | 322.82 ms | 52.3% bf16 MFU | 1623725 tok/s
step 15575/19560 | loss 3.270438 (-1.05z)| norm 0.2755 (-0.57z)| lr 6.37e-05 | 323.22 ms | 52.2% bf16 MFU | 1623644 tok/s
step 15576/19560 | loss 3.373793 (+1.56z)| norm 0.3357 (+2.02z)| lr 6.37e-05 | 323.03 ms | 52.2% bf16 MFU | 1623614 tok/s
step 15577/19560 | loss 3.278562 (-0.84z)| norm 0.2575 (-1.35z)| lr 6.37e-05 | 322.64 ms | 52.3% bf16 MFU | 1623683 tok/s
step 15578/19560 | loss 3.297365 (-0.36z)| norm 0.2953 (+0.27z)| lr 6.36e-05 | 322.75 ms | 52.3% bf16 MFU | 1623722 tok/s
step 15579/19560 | loss 3.308578 (-0.08z)| norm 0.2836 (-0.24z)| lr 6.36e-05 | 322.92 ms | 52.3% bf16 MFU | 1623714 tok/s
step 15580/19560 | loss 3.351128 (+0.99z)| norm 0.2953 (+0.26z)| lr 6.36e-05 | 323.16 ms | 52.2% bf16 MFU | 1623648 tok/s
step 15581/19560 | loss 3.264936 (-1.16z)| norm 0.2738 (-0.66z)| lr 6.35e-05 | 323.01 ms | 52.3% bf16 MFU | 1623623 tok/s
step 15582/19560 | loss 3.276300 (-0.86z)| norm 0.3006 (+0.50z)| lr 6.35e-05 | 322.64 ms | 52.3% bf16 MFU | 1623691 tok/s
step 15583/19560 | loss 3.317218 (+0.17z)| norm 0.3227 (+1.43z)| lr 6.35e-05 | 322.87 ms | 52.3% bf16 MFU | 1623698 tok/s
step 15584/19560 | loss 3.303051 (-0.18z)| norm 0.2728 (-0.71z)| lr 6.35e-05 | 322.36 ms | 52.4% bf16 MFU | 1623832 tok/s
step 15585/19560 | loss 3.326629 (+0.42z)| norm 0.2910 (+0.06z)| lr 6.34e-05 | 323.31 ms | 52.2% bf16 MFU | 1623722 tok/s
step 15586/19560 | loss 3.371260 (+1.57z)| norm 0.2887 (-0.04z)| lr 6.34e-05 | 322.83 ms | 52.3% bf16 MFU | 1623736 tok/s
step 15587/19560 | loss 3.268580 (-1.04z)| norm 0.3081 (+0.79z)| lr 6.34e-05 | 322.85 ms | 52.3% bf16 MFU | 1623746 tok/s
step 15588/19560 | loss 3.258643 (-1.28z)| norm 0.2707 (-0.82z)| lr 6.33e-05 | 322.88 ms | 52.3% bf16 MFU | 1623747 tok/s
step 15589/19560 | loss 3.366686 (+1.43z)| norm 0.3020 (+0.52z)| lr 6.33e-05 | 322.69 ms | 52.3% bf16 MFU | 1623796 tok/s
step 15590/19560 | loss 3.315500 (+0.14z)| norm 0.2767 (-0.57z)| lr 6.33e-05 | 323.49 ms | 52.2% bf16 MFU | 1623643 tok/s
step 15591/19560 | loss 3.298105 (-0.31z)| norm 0.2653 (-1.07z)| lr 6.32e-05 | 322.87 ms | 52.3% bf16 MFU | 1623652 tok/s
step 15592/19560 | loss 3.308944 (-0.04z)| norm 0.2799 (-0.44z)| lr 6.32e-05 | 323.03 ms | 52.2% bf16 MFU | 1623622 tok/s
step 15593/19560 | loss 3.314054 (+0.09z)| norm 0.3019 (+0.51z)| lr 6.32e-05 | 323.74 ms | 52.1% bf16 MFU | 1623414 tok/s
step 15594/19560 | loss 3.248414 (-1.56z)| norm 0.2841 (-0.26z)| lr 6.31e-05 | 322.77 ms | 52.3% bf16 MFU | 1623459 tok/s
step 15595/19560 | loss 3.279858 (-0.76z)| norm 0.2663 (-1.02z)| lr 6.31e-05 | 323.54 ms | 52.2% bf16 MFU | 1623309 tok/s
step 15596/19560 | loss 3.232467 (-1.94z)| norm 0.2658 (-1.04z)| lr 6.31e-05 | 322.85 ms | 52.3% bf16 MFU | 1623341 tok/s
step 15597/19560 | loss 3.273636 (-0.90z)| norm 0.2920 (+0.08z)| lr 6.31e-05 | 322.88 ms | 52.3% bf16 MFU | 1623362 tok/s
step 15598/19560 | loss 3.246844 (-1.55z)| norm 0.2917 (+0.06z)| lr 6.30e-05 | 322.69 ms | 52.3% bf16 MFU | 1623430 tok/s
step 15599/19560 | loss 3.306139 (-0.07z)| norm 0.2700 (-0.88z)| lr 6.30e-05 | 322.92 ms | 52.3% bf16 MFU | 1623438 tok/s
step 15600/19560 | loss 3.367086 (+1.44z)| norm 0.2872 (-0.15z)| lr 6.30e-05 | 322.78 ms | 52.3% bf16 MFU | 1623482 tok/s
step 15601/19560 | loss 3.287905 (-0.54z)| norm 0.2789 (-0.53z)| lr 6.29e-05 | 322.54 ms | 52.3% bf16 MFU | 1623583 tok/s
step 15602/19560 | loss 3.278730 (-0.76z)| norm 0.2653 (-1.12z)| lr 6.29e-05 | 323.53 ms | 52.2% bf16 MFU | 1623429 tok/s
step 15603/19560 | loss 3.363559 (+1.36z)| norm 0.2620 (-1.25z)| lr 6.29e-05 | 323.34 ms | 52.2% bf16 MFU | 1623331 tok/s
step 15604/19560 | loss 3.360320 (+1.26z)| norm 0.2883 (-0.09z)| lr 6.28e-05 | 322.33 ms | 52.4% bf16 MFU | 1623493 tok/s
step 15605/19560 | loss 3.246775 (-1.54z)| norm 0.2765 (-0.62z)| lr 6.28e-05 | 323.23 ms | 52.2% bf16 MFU | 1623420 tok/s
step 15606/19560 | loss 3.312950 (+0.09z)| norm 0.2591 (-1.37z)| lr 6.28e-05 | 322.88 ms | 52.3% bf16 MFU | 1623439 tok/s
step 15607/19560 | loss 3.302143 (-0.18z)| norm 0.2656 (-1.08z)| lr 6.28e-05 | 323.03 ms | 52.2% bf16 MFU | 1623419 tok/s
step 15608/19560 | loss 3.277745 (-0.77z)| norm 0.2592 (-1.35z)| lr 6.27e-05 | 322.39 ms | 52.3% bf16 MFU | 1623559 tok/s
step 15609/19560 | loss 3.266619 (-1.03z)| norm 0.2765 (-0.58z)| lr 6.27e-05 | 322.92 ms | 52.3% bf16 MFU | 1623561 tok/s
step 15610/19560 | loss 3.312412 (+0.12z)| norm 0.3163 (+1.20z)| lr 6.27e-05 | 323.32 ms | 52.2% bf16 MFU | 1623461 tok/s
step 15611/19560 | loss 3.331258 (+0.60z)| norm 0.2820 (-0.34z)| lr 6.26e-05 | 322.70 ms | 52.3% bf16 MFU | 1623522 tok/s
step 15612/19560 | loss 3.299199 (-0.21z)| norm 0.2749 (-0.65z)| lr 6.26e-05 | 322.84 ms | 52.3% bf16 MFU | 1623546 tok/s
step 15613/19560 | loss 3.348742 (+1.02z)| norm 0.3069 (+0.78z)| lr 6.26e-05 | 322.81 ms | 52.3% bf16 MFU | 1623575 tok/s
step 15614/19560 | loss 3.334333 (+0.65z)| norm 0.2860 (-0.16z)| lr 6.25e-05 | 322.88 ms | 52.3% bf16 MFU | 1623586 tok/s
step 15615/19560 | loss 3.394048 (+2.10z)| norm 0.3271 (+1.67z)| lr 6.25e-05 | 322.74 ms | 52.3% bf16 MFU | 1623631 tok/s
step 15616/19560 | loss 3.287886 (-0.52z)| norm 0.2938 (+0.18z)| lr 6.25e-05 | 322.57 ms | 52.3% bf16 MFU | 1623716 tok/s
step 15617/19560 | loss 3.414430 (+2.53z)| norm 0.2816 (-0.37z)| lr 6.24e-05 | 322.87 ms | 52.3% bf16 MFU | 1623723 tok/s
step 15618/19560 | loss 3.267411 (-1.01z)| norm 0.2996 (+0.42z)| lr 6.24e-05 | 322.64 ms | 52.3% bf16 MFU | 1623787 tok/s
step 15619/19560 | loss 3.337363 (+0.68z)| norm 0.3014 (+0.50z)| lr 6.24e-05 | 322.68 ms | 52.3% bf16 MFU | 1623838 tok/s
step 15620/19560 | loss 3.254951 (-1.30z)| norm 0.2641 (-1.18z)| lr 6.24e-05 | 322.64 ms | 52.3% bf16 MFU | 1623895 tok/s
step 15621/19560 | loss 3.308987 (-0.01z)| norm 0.2977 (+0.33z)| lr 6.23e-05 | 322.49 ms | 52.3% bf16 MFU | 1623987 tok/s
step 15622/19560 | loss 3.331757 (+0.54z)| norm 0.2764 (-0.61z)| lr 6.23e-05 | 322.80 ms | 52.3% bf16 MFU | 1623997 tok/s
step 15623/19560 | loss 3.319377 (+0.24z)| norm 0.2808 (-0.42z)| lr 6.23e-05 | 322.94 ms | 52.3% bf16 MFU | 1623971 tok/s
step 15624/19560 | loss 3.334444 (+0.60z)| norm 0.2629 (-1.21z)| lr 6.22e-05 | 323.08 ms | 52.2% bf16 MFU | 1623911 tok/s
step 15625/19560 | loss 3.269130 (-0.99z)| norm 0.2728 (-0.75z)| lr 6.22e-05 | 322.63 ms | 52.3% bf16 MFU | 1623968 tok/s
step 15626/19560 | loss 3.307322 (-0.06z)| norm 0.2780 (-0.52z)| lr 6.22e-05 | 322.39 ms | 52.3% bf16 MFU | 1624082 tok/s
step 15627/19560 | loss 3.275832 (-0.82z)| norm 0.2732 (-0.72z)| lr 6.21e-05 | 323.12 ms | 52.2% bf16 MFU | 1624006 tok/s
step 15628/19560 | loss 3.321856 (+0.30z)| norm 0.2846 (-0.21z)| lr 6.21e-05 | 322.80 ms | 52.3% bf16 MFU | 1624016 tok/s
step 15629/19560 | loss 3.413215 (+2.43z)| norm 0.3381 (+2.15z)| lr 6.21e-05 | 322.47 ms | 52.3% bf16 MFU | 1624108 tok/s
step 15630/19560 | loss 3.271474 (-0.94z)| norm 0.2668 (-1.01z)| lr 6.20e-05 | 322.91 ms | 52.3% bf16 MFU | 1624084 tok/s
step 15631/19560 | loss 3.296614 (-0.33z)| norm 0.3024 (+0.56z)| lr 6.20e-05 | 322.44 ms | 52.3% bf16 MFU | 1624179 tok/s
step 15632/19560 | loss 3.248388 (-1.46z)| norm 0.3102 (+0.89z)| lr 6.20e-05 | 322.48 ms | 52.3% bf16 MFU | 1624260 tok/s
step 15633/19560 | loss 3.371887 (+1.44z)| norm 0.3043 (+0.63z)| lr 6.20e-05 | 322.92 ms | 52.3% bf16 MFU | 1624227 tok/s
step 15634/19560 | loss 3.259087 (-1.22z)| norm 0.3074 (+0.75z)| lr 6.19e-05 | 322.46 ms | 52.3% bf16 MFU | 1624312 tok/s
step 15635/19560 | loss 3.333103 (+0.54z)| norm 0.2565 (-1.48z)| lr 6.19e-05 | 322.89 ms | 52.3% bf16 MFU | 1624284 tok/s
step 15636/19560 | loss 3.329679 (+0.45z)| norm 0.2943 (+0.18z)| lr 6.19e-05 | 322.22 ms | 52.4% bf16 MFU | 1624425 tok/s
step 15637/19560 | loss 3.240993 (-1.62z)| norm 0.2666 (-1.02z)| lr 6.18e-05 | 322.98 ms | 52.3% bf16 MFU | 1624368 tok/s
step 15638/19560 | loss 3.268625 (-0.97z)| norm 0.2766 (-0.58z)| lr 6.18e-05 | 322.71 ms | 52.3% bf16 MFU | 1624381 tok/s
step 15639/19560 | loss 3.288544 (-0.49z)| norm 0.3092 (+0.86z)| lr 6.18e-05 | 322.73 ms | 52.3% bf16 MFU | 1624390 tok/s
step 15640/19560 | loss 3.303088 (-0.13z)| norm 0.2822 (-0.33z)| lr 6.17e-05 | 322.73 ms | 52.3% bf16 MFU | 1624397 tok/s
step 15641/19560 | loss 3.537677 (+4.90z)| norm 0.3574 (+2.85z)| lr 6.17e-05 | 322.09 ms | 52.4% bf16 MFU | 1624566 tok/s
step 15642/19560 | loss 3.303550 (-0.15z)| norm 0.3292 (+1.62z)| lr 6.17e-05 | 322.79 ms | 52.3% bf16 MFU | 1624550 tok/s
step 15643/19560 | loss 3.293875 (-0.37z)| norm 0.2965 (+0.24z)| lr 6.17e-05 | 322.67 ms | 52.3% bf16 MFU | 1624564 tok/s
step 15644/19560 | loss 3.305145 (-0.12z)| norm 0.3186 (+1.15z)| lr 6.16e-05 | 322.74 ms | 52.3% bf16 MFU | 1624560 tok/s
step 15645/19560 | loss 3.385389 (+1.60z)| norm 0.3655 (+3.00z)| lr 6.16e-05 | 322.76 ms | 52.3% bf16 MFU | 1624552 tok/s
step 15646/19560 | loss 3.296909 (-0.30z)| norm 0.2734 (-0.75z)| lr 6.16e-05 | 322.78 ms | 52.3% bf16 MFU | 1624539 tok/s
step 15647/19560 | loss 3.296144 (-0.30z)| norm 0.2831 (-0.37z)| lr 6.15e-05 | 323.00 ms | 52.3% bf16 MFU | 1624471 tok/s
step 15648/19560 | loss 3.356756 (+1.00z)| norm 0.3276 (+1.44z)| lr 6.15e-05 | 322.62 ms | 52.3% bf16 MFU | 1624502 tok/s
step 15649/19560 | loss 3.304373 (-0.14z)| norm 0.2581 (-1.39z)| lr 6.15e-05 | 322.28 ms | 52.4% bf16 MFU | 1624618 tok/s
step 15650/19560 | loss 3.415120 (+2.23z)| norm 0.2938 (+0.06z)| lr 6.14e-05 | 322.70 ms | 52.3% bf16 MFU | 1624622 tok/s
step 15651/19560 | loss 3.354509 (+0.93z)| norm 0.2834 (-0.36z)| lr 6.14e-05 | 322.83 ms | 52.3% bf16 MFU | 1624592 tok/s
step 15652/19560 | loss 3.314305 (+0.08z)| norm 0.2681 (-0.97z)| lr 6.14e-05 | 322.68 ms | 52.3% bf16 MFU | 1624603 tok/s
step 15653/19560 | loss 3.344392 (+0.73z)| norm 0.2836 (-0.34z)| lr 6.14e-05 | 323.27 ms | 52.2% bf16 MFU | 1624464 tok/s
step 15654/19560 | loss 3.280114 (-0.65z)| norm 0.2713 (-0.85z)| lr 6.13e-05 | 322.38 ms | 52.4% bf16 MFU | 1624555 tok/s
step 15655/19560 | loss 3.322141 (+0.26z)| norm 0.2903 (-0.07z)| lr 6.13e-05 | 322.53 ms | 52.3% bf16 MFU | 1624603 tok/s
step 15656/19560 | loss 3.345502 (+0.76z)| norm 0.2801 (-0.48z)| lr 6.13e-05 | 322.09 ms | 52.4% bf16 MFU | 1624761 tok/s
step 15657/19560 | loss 3.252713 (-1.23z)| norm 0.2679 (-0.96z)| lr 6.12e-05 | 322.62 ms | 52.3% bf16 MFU | 1624777 tok/s
step 15658/19560 | loss 3.258253 (-1.11z)| norm 0.2699 (-0.87z)| lr 6.12e-05 | 323.10 ms | 52.2% bf16 MFU | 1624671 tok/s
step 15659/19560 | loss 3.305300 (-0.09z)| norm 0.2887 (-0.09z)| lr 6.12e-05 | 322.50 ms | 52.3% bf16 MFU | 1624723 tok/s
step 15660/19560 | loss 3.334657 (+0.53z)| norm 0.2983 (+0.31z)| lr 6.11e-05 | 322.69 ms | 52.3% bf16 MFU | 1624723 tok/s
step 15661/19560 | loss 3.305428 (-0.10z)| norm 0.2520 (-1.58z)| lr 6.11e-05 | 322.57 ms | 52.3% bf16 MFU | 1624753 tok/s
step 15662/19560 | loss 3.323383 (+0.29z)| norm 0.2812 (-0.39z)| lr 6.11e-05 | 322.74 ms | 52.3% bf16 MFU | 1624740 tok/s
step 15663/19560 | loss 3.216258 (-1.98z)| norm 0.2560 (-1.40z)| lr 6.10e-05 | 323.03 ms | 52.2% bf16 MFU | 1624655 tok/s
step 15664/19560 | loss 3.285901 (-0.48z)| norm 0.2756 (-0.59z)| lr 6.10e-05 | 322.16 ms | 52.4% bf16 MFU | 1624792 tok/s
step 15665/19560 | loss 3.271678 (-0.78z)| norm 0.2767 (-0.54z)| lr 6.10e-05 | 322.90 ms | 52.3% bf16 MFU | 1624738 tok/s
step 15666/19560 | loss 3.325727 (+0.36z)| norm 0.2712 (-0.78z)| lr 6.10e-05 | 322.48 ms | 52.3% bf16 MFU | 1624790 tok/s
step 15667/19560 | loss 3.278411 (-0.64z)| norm 0.2918 (+0.15z)| lr 6.09e-05 | 323.13 ms | 52.2% bf16 MFU | 1624677 tok/s
step 15668/19560 | loss 3.234324 (-1.55z)| norm 0.2747 (-0.61z)| lr 6.09e-05 | 322.78 ms | 52.3% bf16 MFU | 1624658 tok/s
step 15669/19560 | loss 3.308251 (+0.01z)| norm 0.2959 (+0.36z)| lr 6.09e-05 | 322.66 ms | 52.3% bf16 MFU | 1624670 tok/s
step 15670/19560 | loss 3.263224 (-0.93z)| norm 0.2653 (-1.03z)| lr 6.08e-05 | 322.74 ms | 52.3% bf16 MFU | 1624660 tok/s
step 15671/19560 | loss 3.315331 (+0.16z)| norm 0.3127 (+1.12z)| lr 6.08e-05 | 322.39 ms | 52.4% bf16 MFU | 1624741 tok/s
step 15672/19560 | loss 3.329542 (+0.47z)| norm 0.2887 (+0.06z)| lr 6.08e-05 | 322.44 ms | 52.3% bf16 MFU | 1624803 tok/s
step 15673/19560 | loss 3.378156 (+1.49z)| norm 0.2821 (-0.26z)| lr 6.07e-05 | 322.57 ms | 52.3% bf16 MFU | 1624830 tok/s
step 15674/19560 | loss 3.350182 (+0.90z)| norm 0.2755 (-0.57z)| lr 6.07e-05 | 322.68 ms | 52.3% bf16 MFU | 1624827 tok/s
step 15675/19560 | loss 3.244124 (-1.32z)| norm 0.2949 (+0.37z)| lr 6.07e-05 | 322.37 ms | 52.4% bf16 MFU | 1624904 tok/s
step 15676/19560 | loss 3.321280 (+0.28z)| norm 0.2764 (-0.52z)| lr 6.07e-05 | 322.39 ms | 52.4% bf16 MFU | 1624972 tok/s
step 15677/19560 | loss 3.299297 (-0.18z)| norm 0.2708 (-0.78z)| lr 6.06e-05 | 322.60 ms | 52.3% bf16 MFU | 1624983 tok/s
step 15678/19560 | loss 3.328806 (+0.44z)| norm 0.2808 (-0.29z)| lr 6.06e-05 | 322.91 ms | 52.3% bf16 MFU | 1624915 tok/s
step 15679/19560 | loss 3.300032 (-0.17z)| norm 0.3208 (+1.63z)| lr 6.06e-05 | 322.49 ms | 52.3% bf16 MFU | 1624956 tok/s
step 15680/19560 | loss 3.286966 (-0.44z)| norm 0.2710 (-0.76z)| lr 6.05e-05 | 322.88 ms | 52.3% bf16 MFU | 1624897 tok/s
step 15681/19560 | loss 3.370940 (+1.31z)| norm 0.3765 (+4.02z)| lr 6.05e-05 | 322.30 ms | 52.4% bf16 MFU | 1624987 tok/s
step 15682/19560 | loss 3.249509 (-1.23z)| norm 0.2863 (-0.04z)| lr 6.05e-05 | 322.45 ms | 52.3% bf16 MFU | 1625035 tok/s
step 15683/19560 | loss 3.321704 (+0.27z)| norm 0.2756 (-0.53z)| lr 6.04e-05 | 323.05 ms | 52.2% bf16 MFU | 1624930 tok/s
step 15684/19560 | loss 3.273089 (-0.77z)| norm 0.3341 (+2.07z)| lr 6.04e-05 | 322.77 ms | 52.3% bf16 MFU | 1624900 tok/s
step 15685/19560 | loss 3.314709 (+0.12z)| norm 0.2915 (+0.16z)| lr 6.04e-05 | 322.45 ms | 52.3% bf16 MFU | 1624953 tok/s
step 15686/19560 | loss 3.305824 (-0.07z)| norm 0.2778 (-0.45z)| lr 6.04e-05 | 322.39 ms | 52.4% bf16 MFU | 1625018 tok/s
step 15687/19560 | loss 3.282054 (-0.58z)| norm 0.3947 (+4.39z)| lr 6.03e-05 | 322.77 ms | 52.3% bf16 MFU | 1624983 tok/s
step 15688/19560 | loss 3.354115 (+0.95z)| norm 0.3181 (+1.20z)| lr 6.03e-05 | 322.67 ms | 52.3% bf16 MFU | 1624976 tok/s
step 15689/19560 | loss 3.271398 (-0.82z)| norm 0.3056 (+0.68z)| lr 6.03e-05 | 322.60 ms | 52.3% bf16 MFU | 1624987 tok/s
step 15690/19560 | loss 3.314296 (+0.11z)| norm 0.3490 (+2.40z)| lr 6.02e-05 | 323.17 ms | 52.2% bf16 MFU | 1624854 tok/s
step 15691/19560 | loss 3.277748 (-0.68z)| norm 0.2980 (+0.35z)| lr 6.02e-05 | 323.40 ms | 52.2% bf16 MFU | 1624668 tok/s
step 15692/19560 | loss 3.288063 (-0.45z)| norm 0.2950 (+0.22z)| lr 6.02e-05 | 322.84 ms | 52.3% bf16 MFU | 1624635 tok/s
step 15693/19560 | loss 3.312018 (+0.07z)| norm 0.3580 (+2.68z)| lr 6.01e-05 | 322.62 ms | 52.3% bf16 MFU | 1624658 tok/s
step 15694/19560 | loss 3.296770 (-0.26z)| norm 0.3383 (+1.86z)| lr 6.01e-05 | 322.92 ms | 52.3% bf16 MFU | 1624604 tok/s
step 15695/19560 | loss 3.285254 (-0.51z)| norm 0.2673 (-0.91z)| lr 6.01e-05 | 323.10 ms | 52.2% bf16 MFU | 1624509 tok/s
step 15696/19560 | loss 3.291440 (-0.38z)| norm 0.3454 (+2.08z)| lr 6.01e-05 | 322.32 ms | 52.4% bf16 MFU | 1624614 tok/s
step 15697/19560 | loss 3.274178 (-0.75z)| norm 0.3218 (+1.19z)| lr 6.00e-05 | 322.82 ms | 52.3% bf16 MFU | 1624589 tok/s
step 15698/19560 | loss 3.264045 (-0.96z)| norm 0.2632 (-1.06z)| lr 6.00e-05 | 322.22 ms | 52.4% bf16 MFU | 1624714 tok/s
step 15699/19560 | loss 3.317694 (+0.23z)| norm 0.3437 (+1.98z)| lr 6.00e-05 | 322.75 ms | 52.3% bf16 MFU | 1624701 tok/s
step 15700/19560 | loss 3.286458 (-0.47z)| norm 0.2721 (-0.72z)| lr 5.99e-05 | 322.61 ms | 52.3% bf16 MFU | 1624724 tok/s
step 15701/19560 | loss 3.285599 (-0.49z)| norm 0.2700 (-0.79z)| lr 5.99e-05 | 322.23 ms | 52.4% bf16 MFU | 1624840 tok/s
step 15702/19560 | loss 3.302894 (-0.09z)| norm 0.2991 (+0.30z)| lr 5.99e-05 | 322.68 ms | 52.3% bf16 MFU | 1624837 tok/s
step 15703/19560 | loss 3.324744 (+0.39z)| norm 0.2960 (+0.18z)| lr 5.98e-05 | 322.86 ms | 52.3% bf16 MFU | 1624790 tok/s
step 15704/19560 | loss 3.354576 (+1.07z)| norm 0.2621 (-1.08z)| lr 5.98e-05 | 323.44 ms | 52.2% bf16 MFU | 1624599 tok/s
step 15705/19560 | loss 3.400114 (+2.05z)| norm 0.2821 (-0.34z)| lr 5.98e-05 | 322.81 ms | 52.3% bf16 MFU | 1624576 tok/s
step 15706/19560 | loss 3.208009 (-2.18z)| norm 0.2660 (-0.94z)| lr 5.98e-05 | 322.88 ms | 52.3% bf16 MFU | 1624537 tok/s
step 15707/19560 | loss 3.240973 (-1.43z)| norm 0.2663 (-0.92z)| lr 5.97e-05 | 323.21 ms | 52.2% bf16 MFU | 1624416 tok/s
step 15708/19560 | loss 3.327707 (+0.45z)| norm 0.2862 (-0.16z)| lr 5.97e-05 | 322.62 ms | 52.3% bf16 MFU | 1624451 tok/s
step 15709/19560 | loss 3.318190 (+0.24z)| norm 0.2555 (-1.31z)| lr 5.97e-05 | 322.81 ms | 52.3% bf16 MFU | 1624436 tok/s
step 15710/19560 | loss 3.323902 (+0.35z)| norm 0.2770 (-0.50z)| lr 5.96e-05 | 322.33 ms | 52.4% bf16 MFU | 1624541 tok/s
step 15711/19560 | loss 3.239504 (-1.46z)| norm 0.2630 (-1.01z)| lr 5.96e-05 | 323.38 ms | 52.2% bf16 MFU | 1624378 tok/s
step 15712/19560 | loss 3.288790 (-0.39z)| norm 0.2646 (-0.94z)| lr 5.96e-05 | 323.15 ms | 52.2% bf16 MFU | 1624281 tok/s
step 15713/19560 | loss 3.334448 (+0.59z)| norm 0.2524 (-1.38z)| lr 5.95e-05 | 322.52 ms | 52.3% bf16 MFU | 1624346 tok/s
step 15714/19560 | loss 3.341696 (+0.76z)| norm 0.2663 (-0.85z)| lr 5.95e-05 | 323.04 ms | 52.2% bf16 MFU | 1624277 tok/s
step 15715/19560 | loss 3.300095 (-0.15z)| norm 0.2543 (-1.28z)| lr 5.95e-05 | 322.60 ms | 52.3% bf16 MFU | 1624323 tok/s
step 15716/19560 | loss 3.364867 (+1.24z)| norm 0.2756 (-0.49z)| lr 5.95e-05 | 322.28 ms | 52.4% bf16 MFU | 1624447 tok/s
step 15717/19560 | loss 3.341714 (+0.74z)| norm 0.2669 (-0.80z)| lr 5.94e-05 | 323.11 ms | 52.2% bf16 MFU | 1624356 tok/s
step 15718/19560 | loss 3.278214 (-0.64z)| norm 0.2763 (-0.45z)| lr 5.94e-05 | 322.68 ms | 52.3% bf16 MFU | 1624378 tok/s
step 15719/19560 | loss 3.272364 (-0.76z)| norm 0.2518 (-1.35z)| lr 5.94e-05 | 323.29 ms | 52.2% bf16 MFU | 1624244 tok/s
step 15720/19560 | loss 3.288136 (-0.41z)| norm 0.2601 (-1.03z)| lr 5.93e-05 | 322.39 ms | 52.4% bf16 MFU | 1624346 tok/s
step 15721/19560 | loss 3.391799 (+1.81z)| norm 0.2766 (-0.42z)| lr 5.93e-05 | 322.52 ms | 52.3% bf16 MFU | 1624407 tok/s
step 15722/19560 | loss 3.312927 (+0.10z)| norm 0.2658 (-0.81z)| lr 5.93e-05 | 323.43 ms | 52.2% bf16 MFU | 1624239 tok/s
step 15723/19560 | loss 3.286472 (-0.47z)| norm 0.2578 (-1.10z)| lr 5.92e-05 | 322.21 ms | 52.4% bf16 MFU | 1624386 tok/s
step 15724/19560 | loss 3.309852 (+0.02z)| norm 0.2932 (+0.19z)| lr 5.92e-05 | 323.31 ms | 52.2% bf16 MFU | 1624248 tok/s
step 15725/19560 | loss 3.325547 (+0.36z)| norm 0.2496 (-1.39z)| lr 5.92e-05 | 323.05 ms | 52.2% bf16 MFU | 1624182 tok/s
step 15726/19560 | loss 3.267305 (-0.93z)| norm 0.2703 (-0.62z)| lr 5.92e-05 | 322.39 ms | 52.3% bf16 MFU | 1624285 tok/s
step 15727/19560 | loss 3.269846 (-0.86z)| norm 0.2719 (-0.57z)| lr 5.91e-05 | 322.56 ms | 52.3% bf16 MFU | 1624341 tok/s
step 15728/19560 | loss 3.352263 (+0.95z)| norm 0.2623 (-0.91z)| lr 5.91e-05 | 322.49 ms | 52.3% bf16 MFU | 1624412 tok/s
step 15729/19560 | loss 3.370954 (+1.34z)| norm 0.2661 (-0.76z)| lr 5.91e-05 | 322.77 ms | 52.3% bf16 MFU | 1624408 tok/s
step 15730/19560 | loss 3.342689 (+0.71z)| norm 0.2756 (-0.42z)| lr 5.90e-05 | 323.02 ms | 52.2% bf16 MFU | 1624340 tok/s
step 15731/19560 | loss 3.387934 (+1.69z)| norm 0.2652 (-0.80z)| lr 5.90e-05 | 323.16 ms | 52.2% bf16 MFU | 1624242 tok/s
step 15732/19560 | loss 3.296551 (-0.29z)| norm 0.2655 (-0.78z)| lr 5.90e-05 | 322.56 ms | 52.3% bf16 MFU | 1624299 tok/s
step 15733/19560 | loss 3.289237 (-0.46z)| norm 0.2866 (-0.02z)| lr 5.90e-05 | 322.82 ms | 52.3% bf16 MFU | 1624288 tok/s
step 15734/19560 | loss 3.294855 (-0.33z)| norm 0.2787 (-0.32z)| lr 5.89e-05 | 323.16 ms | 52.2% bf16 MFU | 1624194 tok/s
step 15735/19560 | loss 3.313340 (+0.07z)| norm 0.3183 (+1.11z)| lr 5.89e-05 | 322.84 ms | 52.3% bf16 MFU | 1624182 tok/s
step 15736/19560 | loss 3.272177 (-0.83z)| norm 0.2627 (-0.91z)| lr 5.89e-05 | 323.19 ms | 52.2% bf16 MFU | 1624084 tok/s
step 15737/19560 | loss 3.325016 (+0.32z)| norm 0.3693 (+2.84z)| lr 5.88e-05 | 323.09 ms | 52.2% bf16 MFU | 1624016 tok/s
step 15738/19560 | loss 3.282737 (-0.61z)| norm 0.2841 (-0.15z)| lr 5.88e-05 | 322.97 ms | 52.3% bf16 MFU | 1623982 tok/s
step 15739/19560 | loss 3.339221 (+0.63z)| norm 0.3229 (+1.21z)| lr 5.88e-05 | 322.80 ms | 52.3% bf16 MFU | 1623993 tok/s
step 15740/19560 | loss 3.321546 (+0.24z)| norm 0.2781 (-0.37z)| lr 5.87e-05 | 322.36 ms | 52.4% bf16 MFU | 1624114 tok/s
step 15741/19560 | loss 3.274633 (-0.78z)| norm 0.2842 (-0.15z)| lr 5.87e-05 | 322.99 ms | 52.3% bf16 MFU | 1624069 tok/s
step 15742/19560 | loss 3.206883 (-2.20z)| norm 0.2685 (-0.70z)| lr 5.87e-05 | 323.36 ms | 52.2% bf16 MFU | 1623934 tok/s
step 15743/19560 | loss 3.322100 (+0.30z)| norm 0.2804 (-0.27z)| lr 5.87e-05 | 322.36 ms | 52.4% bf16 MFU | 1624056 tok/s
step 15744/19560 | loss 3.329569 (+0.46z)| norm 0.3137 (+0.90z)| lr 5.86e-05 | 322.90 ms | 52.3% bf16 MFU | 1624037 tok/s
step 15745/19560 | loss 3.311105 (+0.07z)| norm 0.2781 (-0.35z)| lr 5.86e-05 | 322.82 ms | 52.3% bf16 MFU | 1624040 tok/s
step 15746/19560 | loss 3.268824 (-0.87z)| norm 0.3037 (+0.55z)| lr 5.86e-05 | 322.62 ms | 52.3% bf16 MFU | 1624092 tok/s
step 15747/19560 | loss 3.303475 (-0.09z)| norm 0.2890 (+0.04z)| lr 5.85e-05 | 322.96 ms | 52.3% bf16 MFU | 1624056 tok/s
step 15748/19560 | loss 3.374389 (+1.47z)| norm 0.2962 (+0.28z)| lr 5.85e-05 | 323.04 ms | 52.2% bf16 MFU | 1624002 tok/s
step 15749/19560 | loss 3.373957 (+1.44z)| norm 0.4029 (+3.79z)| lr 5.85e-05 | 322.62 ms | 52.3% bf16 MFU | 1624056 tok/s
step 15750/19560 | loss 3.281834 (-0.59z)| norm 0.2780 (-0.37z)| lr 5.84e-05 | 322.82 ms | 52.3% bf16 MFU | 1624057 tok/s
val loss 3.294148
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3026/10042 = 0.301334
step 15751/19560 | loss 3.232567 (-1.65z)| norm 0.3183 (+0.96z)| lr 5.84e-05 | 321.99 ms | 52.4% bf16 MFU | 1624267 tok/s
step 15752/19560 | loss 3.250630 (-1.24z)| norm 0.2962 (+0.22z)| lr 5.84e-05 | 323.21 ms | 52.2% bf16 MFU | 1624159 tok/s
step 15753/19560 | loss 3.348643 (+0.89z)| norm 0.2829 (-0.23z)| lr 5.84e-05 | 322.86 ms | 52.3% bf16 MFU | 1624145 tok/s
step 15754/19560 | loss 3.313804 (+0.13z)| norm 0.3265 (+1.21z)| lr 5.83e-05 | 322.89 ms | 52.3% bf16 MFU | 1624124 tok/s
step 15755/19560 | loss 3.252082 (-1.21z)| norm 0.2523 (-1.24z)| lr 5.83e-05 | 323.27 ms | 52.2% bf16 MFU | 1624009 tok/s
step 15756/19560 | loss 3.258589 (-1.05z)| norm 0.2898 (-0.01z)| lr 5.83e-05 | 322.95 ms | 52.3% bf16 MFU | 1623980 tok/s
step 15757/19560 | loss 3.335629 (+0.64z)| norm 0.3082 (+0.61z)| lr 5.82e-05 | 322.93 ms | 52.3% bf16 MFU | 1623959 tok/s
step 15758/19560 | loss 3.297456 (-0.21z)| norm 0.2991 (+0.30z)| lr 5.82e-05 | 323.15 ms | 52.2% bf16 MFU | 1623882 tok/s
step 15759/19560 | loss 3.349145 (+0.92z)| norm 0.3048 (+0.49z)| lr 5.82e-05 | 322.93 ms | 52.3% bf16 MFU | 1623864 tok/s
step 15760/19560 | loss 3.310382 (+0.06z)| norm 0.3175 (+0.91z)| lr 5.81e-05 | 322.83 ms | 52.3% bf16 MFU | 1623874 tok/s
step 15761/19560 | loss 3.290837 (-0.37z)| norm 0.2819 (-0.27z)| lr 5.81e-05 | 322.59 ms | 52.3% bf16 MFU | 1623941 tok/s
step 15762/19560 | loss 3.285006 (-0.50z)| norm 0.2959 (+0.20z)| lr 5.81e-05 | 323.09 ms | 52.2% bf16 MFU | 1623881 tok/s
step 15763/19560 | loss 3.276655 (-0.68z)| norm 0.3163 (+0.87z)| lr 5.81e-05 | 322.76 ms | 52.3% bf16 MFU | 1623906 tok/s
step 15764/19560 | loss 3.244814 (-1.37z)| norm 0.3266 (+1.20z)| lr 5.80e-05 | 323.76 ms | 52.1% bf16 MFU | 1623679 tok/s
step 15765/19560 | loss 3.296903 (-0.22z)| norm 0.2938 (+0.10z)| lr 5.80e-05 | 322.98 ms | 52.3% bf16 MFU | 1623658 tok/s
step 15766/19560 | loss 3.350874 (+0.97z)| norm 0.3205 (+0.98z)| lr 5.80e-05 | 322.64 ms | 52.3% bf16 MFU | 1623726 tok/s
step 15767/19560 | loss 3.283719 (-0.53z)| norm 0.3203 (+0.97z)| lr 5.79e-05 | 323.08 ms | 52.2% bf16 MFU | 1623678 tok/s
step 15768/19560 | loss 3.317320 (+0.22z)| norm 0.3234 (+1.05z)| lr 5.79e-05 | 323.55 ms | 52.2% bf16 MFU | 1623515 tok/s
step 15769/19560 | loss 3.333263 (+0.69z)| norm 0.3186 (+0.92z)| lr 5.79e-05 | 323.54 ms | 52.2% bf16 MFU | 1623362 tok/s
step 15770/19560 | loss 3.315588 (+0.24z)| norm 0.2966 (+0.19z)| lr 5.79e-05 | 323.48 ms | 52.2% bf16 MFU | 1623232 tok/s
step 15771/19560 | loss 3.281963 (-0.60z)| norm 0.2804 (-0.35z)| lr 5.78e-05 | 323.15 ms | 52.2% bf16 MFU | 1623193 tok/s
step 15772/19560 | loss 3.295697 (-0.25z)| norm 0.3074 (+0.56z)| lr 5.78e-05 | 322.81 ms | 52.3% bf16 MFU | 1623239 tok/s
step 15773/19560 | loss 3.288959 (-0.41z)| norm 0.3039 (+0.47z)| lr 5.78e-05 | 324.01 ms | 52.1% bf16 MFU | 1622984 tok/s
step 15774/19560 | loss 3.270233 (-0.88z)| norm 0.2878 (-0.09z)| lr 5.77e-05 | 323.54 ms | 52.2% bf16 MFU | 1622858 tok/s
step 15775/19560 | loss 3.319970 (+0.38z)| norm 0.2661 (-0.84z)| lr 5.77e-05 | 323.23 ms | 52.2% bf16 MFU | 1622816 tok/s
step 15776/19560 | loss 3.315489 (+0.28z)| norm 0.2608 (-1.00z)| lr 5.77e-05 | 322.42 ms | 52.3% bf16 MFU | 1622980 tok/s
step 15777/19560 | loss 3.307907 (+0.08z)| norm 0.2856 (-0.15z)| lr 5.76e-05 | 323.33 ms | 52.2% bf16 MFU | 1622907 tok/s
step 15778/19560 | loss 3.291426 (-0.33z)| norm 0.2943 (+0.15z)| lr 5.76e-05 | 323.16 ms | 52.2% bf16 MFU | 1622880 tok/s
step 15779/19560 | loss 3.247213 (-1.47z)| norm 0.2765 (-0.47z)| lr 5.76e-05 | 322.98 ms | 52.3% bf16 MFU | 1622900 tok/s
step 15780/19560 | loss 3.283213 (-0.51z)| norm 0.2605 (-1.02z)| lr 5.76e-05 | 322.80 ms | 52.3% bf16 MFU | 1622964 tok/s
step 15781/19560 | loss 3.348511 (+1.21z)| norm 0.2936 (+0.13z)| lr 5.75e-05 | 322.90 ms | 52.3% bf16 MFU | 1623000 tok/s
step 15782/19560 | loss 3.337564 (+0.90z)| norm 0.2999 (+0.34z)| lr 5.75e-05 | 322.72 ms | 52.3% bf16 MFU | 1623081 tok/s
step 15783/19560 | loss 3.331201 (+0.73z)| norm 0.2732 (-0.58z)| lr 5.75e-05 | 322.54 ms | 52.3% bf16 MFU | 1623201 tok/s
step 15784/19560 | loss 3.308498 (+0.15z)| norm 0.2635 (-0.91z)| lr 5.74e-05 | 322.65 ms | 52.3% bf16 MFU | 1623289 tok/s
step 15785/19560 | loss 3.244336 (-1.54z)| norm 0.2853 (-0.16z)| lr 5.74e-05 | 322.89 ms | 52.3% bf16 MFU | 1623311 tok/s
step 15786/19560 | loss 3.375936 (+1.88z)| norm 0.2858 (-0.15z)| lr 5.74e-05 | 323.26 ms | 52.2% bf16 MFU | 1623240 tok/s
step 15787/19560 | loss 3.334582 (+0.80z)| norm 0.2594 (-1.06z)| lr 5.74e-05 | 322.21 ms | 52.4% bf16 MFU | 1623434 tok/s
step 15788/19560 | loss 3.288586 (-0.39z)| norm 0.2790 (-0.37z)| lr 5.73e-05 | 322.98 ms | 52.3% bf16 MFU | 1623426 tok/s
step 15789/19560 | loss 3.328195 (+0.63z)| norm 0.2794 (-0.37z)| lr 5.73e-05 | 322.66 ms | 52.3% bf16 MFU | 1623500 tok/s
step 15790/19560 | loss 3.281745 (-0.57z)| norm 0.3077 (+0.61z)| lr 5.73e-05 | 322.54 ms | 52.3% bf16 MFU | 1623600 tok/s
step 15791/19560 | loss 3.358877 (+1.43z)| norm 0.2925 (+0.07z)| lr 5.72e-05 | 323.46 ms | 52.2% bf16 MFU | 1623463 tok/s
step 15792/19560 | loss 3.324285 (+0.51z)| norm 0.2856 (-0.17z)| lr 5.72e-05 | 322.41 ms | 52.3% bf16 MFU | 1623596 tok/s
step 15793/19560 | loss 3.283091 (-0.58z)| norm 0.2994 (+0.31z)| lr 5.72e-05 | 322.88 ms | 52.3% bf16 MFU | 1623605 tok/s
step 15794/19560 | loss 3.269310 (-0.93z)| norm 0.2651 (-0.89z)| lr 5.71e-05 | 322.56 ms | 52.3% bf16 MFU | 1623694 tok/s
step 15795/19560 | loss 3.374408 (+1.80z)| norm 0.3102 (+0.68z)| lr 5.71e-05 | 322.46 ms | 52.3% bf16 MFU | 1623806 tok/s
step 15796/19560 | loss 3.306547 (+0.02z)| norm 0.2916 (+0.03z)| lr 5.71e-05 | 323.18 ms | 52.2% bf16 MFU | 1623730 tok/s
step 15797/19560 | loss 3.301663 (-0.11z)| norm 0.2827 (-0.28z)| lr 5.71e-05 | 322.46 ms | 52.3% bf16 MFU | 1623838 tok/s
step 15798/19560 | loss 3.338618 (+0.85z)| norm 0.2696 (-0.74z)| lr 5.70e-05 | 322.71 ms | 52.3% bf16 MFU | 1623878 tok/s
step 15799/19560 | loss 3.365335 (+1.53z)| norm 0.2698 (-0.73z)| lr 5.70e-05 | 322.74 ms | 52.3% bf16 MFU | 1623909 tok/s
step 15800/19560 | loss 3.333829 (+0.71z)| norm 0.2721 (-0.64z)| lr 5.70e-05 | 322.20 ms | 52.4% bf16 MFU | 1624075 tok/s
step 15801/19560 | loss 3.294943 (-0.30z)| norm 0.3353 (+1.54z)| lr 5.69e-05 | 322.61 ms | 52.3% bf16 MFU | 1624128 tok/s
step 15802/19560 | loss 3.381393 (+1.97z)| norm 0.2904 (-0.02z)| lr 5.69e-05 | 322.72 ms | 52.3% bf16 MFU | 1624150 tok/s
step 15803/19560 | loss 3.326046 (+0.50z)| norm 0.2796 (-0.39z)| lr 5.69e-05 | 322.44 ms | 52.3% bf16 MFU | 1624244 tok/s
step 15804/19560 | loss 3.248357 (-1.53z)| norm 0.3103 (+0.67z)| lr 5.69e-05 | 322.14 ms | 52.4% bf16 MFU | 1624408 tok/s
step 15805/19560 | loss 3.332364 (+0.67z)| norm 0.2796 (-0.40z)| lr 5.68e-05 | 322.24 ms | 52.4% bf16 MFU | 1624538 tok/s
step 15806/19560 | loss 3.315662 (+0.24z)| norm 0.3251 (+1.16z)| lr 5.68e-05 | 322.88 ms | 52.3% bf16 MFU | 1624500 tok/s
step 15807/19560 | loss 3.333548 (+0.70z)| norm 0.2769 (-0.49z)| lr 5.68e-05 | 322.50 ms | 52.3% bf16 MFU | 1624561 tok/s
step 15808/19560 | loss 3.265494 (-1.08z)| norm 0.2856 (-0.20z)| lr 5.67e-05 | 322.86 ms | 52.3% bf16 MFU | 1624527 tok/s
step 15809/19560 | loss 3.308744 (+0.07z)| norm 0.3441 (+1.89z)| lr 5.67e-05 | 322.60 ms | 52.3% bf16 MFU | 1624561 tok/s
step 15810/19560 | loss 3.295435 (-0.30z)| norm 0.2820 (-0.32z)| lr 5.67e-05 | 322.69 ms | 52.3% bf16 MFU | 1624569 tok/s
step 15811/19560 | loss 3.285979 (-0.54z)| norm 0.2811 (-0.35z)| lr 5.67e-05 | 322.50 ms | 52.3% bf16 MFU | 1624627 tok/s
step 15812/19560 | loss 3.337894 (+0.83z)| norm 0.2679 (-0.81z)| lr 5.66e-05 | 322.73 ms | 52.3% bf16 MFU | 1624622 tok/s
step 15813/19560 | loss 3.290182 (-0.44z)| norm 0.2863 (-0.15z)| lr 5.66e-05 | 322.42 ms | 52.3% bf16 MFU | 1624696 tok/s
step 15814/19560 | loss 3.350716 (+1.16z)| norm 0.3039 (+0.47z)| lr 5.66e-05 | 322.76 ms | 52.3% bf16 MFU | 1624682 tok/s
step 15815/19560 | loss 3.380135 (+1.90z)| norm 0.2643 (-0.95z)| lr 5.65e-05 | 322.60 ms | 52.3% bf16 MFU | 1624708 tok/s
step 15816/19560 | loss 3.359565 (+1.35z)| norm 0.2741 (-0.58z)| lr 5.65e-05 | 322.57 ms | 52.3% bf16 MFU | 1624740 tok/s
step 15817/19560 | loss 3.351880 (+1.13z)| norm 0.3057 (+0.62z)| lr 5.65e-05 | 322.98 ms | 52.3% bf16 MFU | 1624668 tok/s
step 15818/19560 | loss 3.346225 (+0.98z)| norm 0.2595 (-1.12z)| lr 5.64e-05 | 322.66 ms | 52.3% bf16 MFU | 1624678 tok/s
step 15819/19560 | loss 3.409121 (+2.53z)| norm 0.3157 (+1.03z)| lr 5.64e-05 | 322.62 ms | 52.3% bf16 MFU | 1624698 tok/s
step 15820/19560 | loss 3.276043 (-0.85z)| norm 0.2728 (-0.60z)| lr 5.64e-05 | 322.53 ms | 52.3% bf16 MFU | 1624742 tok/s
step 15821/19560 | loss 3.314806 (+0.13z)| norm 0.2666 (-0.84z)| lr 5.64e-05 | 322.65 ms | 52.3% bf16 MFU | 1624751 tok/s
step 15822/19560 | loss 3.299803 (-0.25z)| norm 0.2965 (+0.36z)| lr 5.63e-05 | 323.31 ms | 52.2% bf16 MFU | 1624594 tok/s
step 15823/19560 | loss 3.261367 (-1.22z)| norm 0.2687 (-0.75z)| lr 5.63e-05 | 321.98 ms | 52.4% bf16 MFU | 1624781 tok/s
step 15824/19560 | loss 3.302497 (-0.18z)| norm 0.2696 (-0.70z)| lr 5.63e-05 | 322.86 ms | 52.3% bf16 MFU | 1624736 tok/s
step 15825/19560 | loss 3.345087 (+0.88z)| norm 0.3022 (+0.63z)| lr 5.62e-05 | 323.13 ms | 52.2% bf16 MFU | 1624626 tok/s
step 15826/19560 | loss 3.279610 (-0.78z)| norm 0.2801 (-0.28z)| lr 5.62e-05 | 322.46 ms | 52.3% bf16 MFU | 1624689 tok/s
step 15827/19560 | loss 3.313918 (+0.09z)| norm 0.3701 (+3.33z)| lr 5.62e-05 | 322.40 ms | 52.3% bf16 MFU | 1624764 tok/s
step 15828/19560 | loss 3.552087 (+5.36z)| norm 0.2851 (-0.09z)| lr 5.62e-05 | 322.90 ms | 52.3% bf16 MFU | 1624710 tok/s
step 15829/19560 | loss 3.303560 (-0.20z)| norm 0.4378 (+5.30z)| lr 5.61e-05 | 322.64 ms | 52.3% bf16 MFU | 1624725 tok/s
step 15830/19560 | loss 3.277649 (-0.77z)| norm 0.2862 (-0.08z)| lr 5.61e-05 | 321.58 ms | 52.5% bf16 MFU | 1625006 tok/s
step 15831/19560 | loss 3.228375 (-1.83z)| norm 0.2979 (+0.34z)| lr 5.61e-05 | 322.73 ms | 52.3% bf16 MFU | 1624984 tok/s
step 15832/19560 | loss 3.313575 (+0.05z)| norm 0.3584 (+2.41z)| lr 5.60e-05 | 322.66 ms | 52.3% bf16 MFU | 1624978 tok/s
step 15833/19560 | loss 3.426693 (+2.52z)| norm 0.2975 (+0.29z)| lr 5.60e-05 | 322.45 ms | 52.3% bf16 MFU | 1625027 tok/s
step 15834/19560 | loss 3.351217 (+0.86z)| norm 0.2832 (-0.22z)| lr 5.60e-05 | 322.65 ms | 52.3% bf16 MFU | 1625022 tok/s
step 15835/19560 | loss 3.291666 (-0.48z)| norm 0.3687 (+2.67z)| lr 5.60e-05 | 322.64 ms | 52.3% bf16 MFU | 1625020 tok/s
step 15836/19560 | loss 3.298796 (-0.31z)| norm 0.3400 (+1.66z)| lr 5.59e-05 | 322.90 ms | 52.3% bf16 MFU | 1624953 tok/s
step 15837/19560 | loss 3.333651 (+0.47z)| norm 0.2889 (-0.07z)| lr 5.59e-05 | 322.72 ms | 52.3% bf16 MFU | 1624936 tok/s
step 15838/19560 | loss 3.385914 (+1.62z)| norm 0.3468 (+1.84z)| lr 5.59e-05 | 322.43 ms | 52.3% bf16 MFU | 1624992 tok/s
step 15839/19560 | loss 3.337990 (+0.54z)| norm 0.3303 (+1.27z)| lr 5.58e-05 | 322.70 ms | 52.3% bf16 MFU | 1624976 tok/s
step 15840/19560 | loss 3.266306 (-1.07z)| norm 0.3095 (+0.57z)| lr 5.58e-05 | 322.70 ms | 52.3% bf16 MFU | 1624962 tok/s
step 15841/19560 | loss 3.319892 (+0.14z)| norm 0.3482 (+1.83z)| lr 5.58e-05 | 322.42 ms | 52.3% bf16 MFU | 1625020 tok/s
step 15842/19560 | loss 3.265608 (-1.06z)| norm 0.3199 (+0.87z)| lr 5.57e-05 | 322.63 ms | 52.3% bf16 MFU | 1625021 tok/s
step 15843/19560 | loss 3.345695 (+0.72z)| norm 0.2754 (-0.61z)| lr 5.57e-05 | 322.87 ms | 52.3% bf16 MFU | 1624963 tok/s
step 15844/19560 | loss 3.285301 (-0.62z)| norm 0.3185 (+0.81z)| lr 5.57e-05 | 322.71 ms | 52.3% bf16 MFU | 1624946 tok/s
step 15845/19560 | loss 3.342798 (+0.67z)| norm 0.3287 (+1.14z)| lr 5.57e-05 | 322.76 ms | 52.3% bf16 MFU | 1624918 tok/s
step 15846/19560 | loss 3.305566 (-0.17z)| norm 0.2811 (-0.45z)| lr 5.56e-05 | 322.58 ms | 52.3% bf16 MFU | 1624937 tok/s
step 15847/19560 | loss 3.240244 (-1.62z)| norm 0.3024 (+0.25z)| lr 5.56e-05 | 322.32 ms | 52.4% bf16 MFU | 1625020 tok/s
step 15848/19560 | loss 3.319594 (+0.14z)| norm 0.2943 (-0.03z)| lr 5.56e-05 | 323.02 ms | 52.2% bf16 MFU | 1624923 tok/s
step 15849/19560 | loss 3.263166 (-1.10z)| norm 0.2825 (-0.43z)| lr 5.55e-05 | 322.72 ms | 52.3% bf16 MFU | 1624906 tok/s
step 15850/19560 | loss 3.263955 (-1.07z)| norm 0.2718 (-0.79z)| lr 5.55e-05 | 322.37 ms | 52.4% bf16 MFU | 1624978 tok/s
step 15851/19560 | loss 3.295385 (-0.37z)| norm 0.2806 (-0.51z)| lr 5.55e-05 | 322.68 ms | 52.3% bf16 MFU | 1624968 tok/s
step 15852/19560 | loss 3.490534 (+3.74z)| norm 0.2801 (-0.52z)| lr 5.55e-05 | 322.48 ms | 52.3% bf16 MFU | 1625009 tok/s
step 15853/19560 | loss 3.277009 (-0.75z)| norm 0.2905 (-0.18z)| lr 5.54e-05 | 322.68 ms | 52.3% bf16 MFU | 1624999 tok/s
step 15854/19560 | loss 3.225494 (-1.82z)| norm 0.2771 (-0.64z)| lr 5.54e-05 | 322.92 ms | 52.3% bf16 MFU | 1624927 tok/s
step 15855/19560 | loss 3.316146 (+0.07z)| norm 0.3215 (+0.87z)| lr 5.54e-05 | 322.34 ms | 52.4% bf16 MFU | 1625007 tok/s
step 15856/19560 | loss 3.244558 (-1.41z)| norm 0.2674 (-0.99z)| lr 5.53e-05 | 322.59 ms | 52.3% bf16 MFU | 1625018 tok/s
step 15857/19560 | loss 3.242324 (-1.43z)| norm 0.2744 (-0.75z)| lr 5.53e-05 | 322.72 ms | 52.3% bf16 MFU | 1624996 tok/s
step 15858/19560 | loss 3.276213 (-0.71z)| norm 0.2746 (-0.74z)| lr 5.53e-05 | 322.53 ms | 52.3% bf16 MFU | 1625023 tok/s
step 15859/19560 | loss 3.305795 (-0.09z)| norm 0.2637 (-1.12z)| lr 5.53e-05 | 322.78 ms | 52.3% bf16 MFU | 1624987 tok/s
step 15860/19560 | loss 3.277219 (-0.68z)| norm 0.2764 (-0.69z)| lr 5.52e-05 | 322.31 ms | 52.4% bf16 MFU | 1625070 tok/s
step 15861/19560 | loss 3.356165 (+0.96z)| norm 0.2985 (+0.07z)| lr 5.52e-05 | 322.41 ms | 52.3% bf16 MFU | 1625123 tok/s
step 15862/19560 | loss 3.245722 (-1.33z)| norm 0.2706 (-0.88z)| lr 5.52e-05 | 322.39 ms | 52.4% bf16 MFU | 1625180 tok/s
step 15863/19560 | loss 3.294869 (-0.31z)| norm 0.2587 (-1.27z)| lr 5.51e-05 | 322.79 ms | 52.3% bf16 MFU | 1625133 tok/s
step 15864/19560 | loss 3.360446 (+1.04z)| norm 0.3138 (+0.60z)| lr 5.51e-05 | 323.03 ms | 52.2% bf16 MFU | 1625028 tok/s
step 15865/19560 | loss 3.296899 (-0.28z)| norm 0.2948 (-0.03z)| lr 5.51e-05 | 322.67 ms | 52.3% bf16 MFU | 1625019 tok/s
step 15866/19560 | loss 3.314842 (+0.09z)| norm 0.2747 (-0.74z)| lr 5.51e-05 | 322.44 ms | 52.3% bf16 MFU | 1625068 tok/s
step 15867/19560 | loss 3.288740 (-0.44z)| norm 0.2745 (-0.73z)| lr 5.50e-05 | 322.67 ms | 52.3% bf16 MFU | 1625058 tok/s
step 15868/19560 | loss 3.319323 (+0.19z)| norm 0.2767 (-0.65z)| lr 5.50e-05 | 323.01 ms | 52.2% bf16 MFU | 1624960 tok/s
step 15869/19560 | loss 3.314276 (+0.08z)| norm 0.2766 (-0.65z)| lr 5.50e-05 | 322.57 ms | 52.3% bf16 MFU | 1624980 tok/s
step 15870/19560 | loss 3.269123 (-0.88z)| norm 0.2544 (-1.42z)| lr 5.49e-05 | 322.72 ms | 52.3% bf16 MFU | 1624960 tok/s
step 15871/19560 | loss 3.293546 (-0.36z)| norm 0.2632 (-1.11z)| lr 5.49e-05 | 322.60 ms | 52.3% bf16 MFU | 1624973 tok/s
step 15872/19560 | loss 3.261534 (-1.02z)| norm 0.2816 (-0.46z)| lr 5.49e-05 | 322.76 ms | 52.3% bf16 MFU | 1624943 tok/s
step 15873/19560 | loss 3.298521 (-0.24z)| norm 0.2605 (-1.18z)| lr 5.49e-05 | 322.83 ms | 52.3% bf16 MFU | 1624898 tok/s
step 15874/19560 | loss 3.277667 (-0.68z)| norm 0.2795 (-0.52z)| lr 5.48e-05 | 322.62 ms | 52.3% bf16 MFU | 1624908 tok/s
step 15875/19560 | loss 3.313507 (+0.07z)| norm 0.2630 (-1.08z)| lr 5.48e-05 | 322.80 ms | 52.3% bf16 MFU | 1624873 tok/s
step 15876/19560 | loss 3.256763 (-1.11z)| norm 0.2812 (-0.44z)| lr 5.48e-05 | 322.74 ms | 52.3% bf16 MFU | 1624854 tok/s
step 15877/19560 | loss 3.396717 (+1.83z)| norm 0.2788 (-0.52z)| lr 5.47e-05 | 322.47 ms | 52.3% bf16 MFU | 1624905 tok/s
step 15878/19560 | loss 3.348796 (+0.82z)| norm 0.2668 (-0.96z)| lr 5.47e-05 | 322.37 ms | 52.4% bf16 MFU | 1624977 tok/s
step 15879/19560 | loss 3.262887 (-1.00z)| norm 0.3188 (+0.94z)| lr 5.47e-05 | 322.71 ms | 52.3% bf16 MFU | 1624961 tok/s
step 15880/19560 | loss 3.322671 (+0.25z)| norm 0.2534 (-1.42z)| lr 5.46e-05 | 322.64 ms | 52.3% bf16 MFU | 1624962 tok/s
step 15881/19560 | loss 3.297833 (-0.27z)| norm 0.3028 (+0.36z)| lr 5.46e-05 | 322.51 ms | 52.3% bf16 MFU | 1624997 tok/s
step 15882/19560 | loss 3.404725 (+1.96z)| norm 0.3007 (+0.30z)| lr 5.46e-05 | 322.70 ms | 52.3% bf16 MFU | 1624982 tok/s
step 15883/19560 | loss 3.415526 (+2.14z)| norm 0.2685 (-0.89z)| lr 5.46e-05 | 322.60 ms | 52.3% bf16 MFU | 1624992 tok/s
step 15884/19560 | loss 3.297772 (-0.31z)| norm 0.2709 (-0.79z)| lr 5.45e-05 | 322.72 ms | 52.3% bf16 MFU | 1624972 tok/s
step 15885/19560 | loss 3.333205 (+0.43z)| norm 0.2640 (-1.03z)| lr 5.45e-05 | 322.60 ms | 52.3% bf16 MFU | 1624982 tok/s
step 15886/19560 | loss 3.270981 (-0.86z)| norm 0.2890 (-0.12z)| lr 5.45e-05 | 322.68 ms | 52.3% bf16 MFU | 1624973 tok/s
step 15887/19560 | loss 3.291874 (-0.42z)| norm 0.2691 (-0.83z)| lr 5.44e-05 | 322.77 ms | 52.3% bf16 MFU | 1624942 tok/s
step 15888/19560 | loss 3.360986 (+1.01z)| norm 0.3095 (+0.65z)| lr 5.44e-05 | 322.32 ms | 52.4% bf16 MFU | 1625026 tok/s
step 15889/19560 | loss 3.347969 (+0.73z)| norm 0.2712 (-0.75z)| lr 5.44e-05 | 322.92 ms | 52.3% bf16 MFU | 1624954 tok/s
step 15890/19560 | loss 3.283873 (-0.60z)| norm 0.3039 (+0.44z)| lr 5.44e-05 | 322.78 ms | 52.3% bf16 MFU | 1624921 tok/s
step 15891/19560 | loss 3.244272 (-1.40z)| norm 0.2868 (-0.17z)| lr 5.43e-05 | 322.45 ms | 52.3% bf16 MFU | 1624974 tok/s
step 15892/19560 | loss 3.289958 (-0.47z)| norm 0.3179 (+0.97z)| lr 5.43e-05 | 322.44 ms | 52.3% bf16 MFU | 1625026 tok/s
step 15893/19560 | loss 3.310553 (-0.05z)| norm 0.3010 (+0.35z)| lr 5.43e-05 | 322.59 ms | 52.3% bf16 MFU | 1625037 tok/s
step 15894/19560 | loss 3.322371 (+0.20z)| norm 0.3192 (+1.01z)| lr 5.42e-05 | 322.51 ms | 52.3% bf16 MFU | 1625069 tok/s
step 15895/19560 | loss 3.321095 (+0.17z)| norm 0.2858 (-0.20z)| lr 5.42e-05 | 322.67 ms | 52.3% bf16 MFU | 1625058 tok/s
step 15896/19560 | loss 3.353874 (+0.84z)| norm 0.3039 (+0.47z)| lr 5.42e-05 | 322.46 ms | 52.3% bf16 MFU | 1625101 tok/s
step 15897/19560 | loss 3.297665 (-0.32z)| norm 0.3947 (+3.62z)| lr 5.42e-05 | 322.80 ms | 52.3% bf16 MFU | 1625056 tok/s
step 15898/19560 | loss 3.226976 (-1.75z)| norm 0.2539 (-1.31z)| lr 5.41e-05 | 322.64 ms | 52.3% bf16 MFU | 1625052 tok/s
step 15899/19560 | loss 3.374509 (+1.25z)| norm 0.2759 (-0.54z)| lr 5.41e-05 | 322.48 ms | 52.3% bf16 MFU | 1625088 tok/s
step 15900/19560 | loss 3.375721 (+1.26z)| norm 0.3185 (+0.94z)| lr 5.41e-05 | 322.77 ms | 52.3% bf16 MFU | 1625051 tok/s
step 15901/19560 | loss 3.324793 (+0.22z)| norm 0.2656 (-0.88z)| lr 5.40e-05 | 322.54 ms | 52.3% bf16 MFU | 1625074 tok/s
step 15902/19560 | loss 3.327718 (+0.27z)| norm 0.2614 (-1.02z)| lr 5.40e-05 | 322.64 ms | 52.3% bf16 MFU | 1625070 tok/s
step 15903/19560 | loss 3.303598 (-0.22z)| norm 0.3152 (+0.83z)| lr 5.40e-05 | 322.57 ms | 52.3% bf16 MFU | 1625084 tok/s
step 15904/19560 | loss 3.279373 (-0.70z)| norm 0.2596 (-1.10z)| lr 5.40e-05 | 322.71 ms | 52.3% bf16 MFU | 1625061 tok/s
step 15905/19560 | loss 3.336605 (+0.45z)| norm 0.2788 (-0.43z)| lr 5.39e-05 | 322.71 ms | 52.3% bf16 MFU | 1625039 tok/s
step 15906/19560 | loss 3.276645 (-0.76z)| norm 0.2816 (-0.33z)| lr 5.39e-05 | 322.59 ms | 52.3% bf16 MFU | 1625049 tok/s
step 15907/19560 | loss 3.344423 (+0.60z)| norm 0.2847 (-0.22z)| lr 5.39e-05 | 322.87 ms | 52.3% bf16 MFU | 1624989 tok/s
step 15908/19560 | loss 3.337332 (+0.45z)| norm 0.2928 (+0.05z)| lr 5.38e-05 | 323.04 ms | 52.2% bf16 MFU | 1624888 tok/s
step 15909/19560 | loss 3.264651 (-1.02z)| norm 0.2961 (+0.16z)| lr 5.38e-05 | 322.29 ms | 52.4% bf16 MFU | 1624982 tok/s
step 15910/19560 | loss 3.231267 (-1.66z)| norm 0.2683 (-0.79z)| lr 5.38e-05 | 322.83 ms | 52.3% bf16 MFU | 1624935 tok/s
step 15911/19560 | loss 3.368084 (+1.08z)| norm 0.3098 (+0.63z)| lr 5.38e-05 | 323.15 ms | 52.2% bf16 MFU | 1624810 tok/s
step 15912/19560 | loss 3.294597 (-0.39z)| norm 0.2607 (-1.06z)| lr 5.37e-05 | 322.81 ms | 52.3% bf16 MFU | 1624776 tok/s
step 15913/19560 | loss 3.336380 (+0.44z)| norm 0.2748 (-0.57z)| lr 5.37e-05 | 323.33 ms | 52.2% bf16 MFU | 1624613 tok/s
step 15914/19560 | loss 3.319272 (+0.10z)| norm 0.2859 (-0.19z)| lr 5.37e-05 | 322.91 ms | 52.3% bf16 MFU | 1624565 tok/s
step 15915/19560 | loss 3.363168 (+0.98z)| norm 0.2839 (-0.27z)| lr 5.36e-05 | 322.52 ms | 52.3% bf16 MFU | 1624616 tok/s
step 15916/19560 | loss 3.296488 (-0.37z)| norm 0.2915 (-0.01z)| lr 5.36e-05 | 322.83 ms | 52.3% bf16 MFU | 1624588 tok/s
step 15917/19560 | loss 3.318724 (+0.09z)| norm 0.2844 (-0.26z)| lr 5.36e-05 | 322.94 ms | 52.3% bf16 MFU | 1624534 tok/s
step 15918/19560 | loss 3.309424 (-0.11z)| norm 0.3029 (+0.39z)| lr 5.36e-05 | 322.64 ms | 52.3% bf16 MFU | 1624558 tok/s
step 15919/19560 | loss 3.341392 (+0.55z)| norm 0.2736 (-0.62z)| lr 5.35e-05 | 323.14 ms | 52.2% bf16 MFU | 1624454 tok/s
step 15920/19560 | loss 3.245314 (-1.38z)| norm 0.2709 (-0.71z)| lr 5.35e-05 | 322.68 ms | 52.3% bf16 MFU | 1624470 tok/s
step 15921/19560 | loss 3.312248 (-0.04z)| norm 0.2822 (-0.32z)| lr 5.35e-05 | 323.05 ms | 52.2% bf16 MFU | 1624394 tok/s
step 15922/19560 | loss 3.274417 (-0.80z)| norm 0.2527 (-1.33z)| lr 5.34e-05 | 322.91 ms | 52.3% bf16 MFU | 1624355 tok/s
step 15923/19560 | loss 3.322131 (+0.17z)| norm 0.2682 (-0.78z)| lr 5.34e-05 | 322.58 ms | 52.3% bf16 MFU | 1624403 tok/s
step 15924/19560 | loss 3.292849 (-0.42z)| norm 0.2790 (-0.41z)| lr 5.34e-05 | 322.90 ms | 52.3% bf16 MFU | 1624366 tok/s
step 15925/19560 | loss 3.339962 (+0.53z)| norm 0.2636 (-0.93z)| lr 5.34e-05 | 323.00 ms | 52.3% bf16 MFU | 1624307 tok/s
step 15926/19560 | loss 3.303289 (-0.21z)| norm 0.2810 (-0.34z)| lr 5.33e-05 | 322.36 ms | 52.4% bf16 MFU | 1624412 tok/s
step 15927/19560 | loss 3.468261 (+3.02z)| norm 0.2671 (-0.81z)| lr 5.33e-05 | 323.43 ms | 52.2% bf16 MFU | 1624242 tok/s
step 15928/19560 | loss 3.259593 (-1.06z)| norm 0.2786 (-0.42z)| lr 5.33e-05 | 322.56 ms | 52.3% bf16 MFU | 1624300 tok/s
step 15929/19560 | loss 3.230689 (-1.60z)| norm 0.2678 (-0.78z)| lr 5.32e-05 | 322.73 ms | 52.3% bf16 MFU | 1624313 tok/s
step 15930/19560 | loss 3.324801 (+0.23z)| norm 0.2655 (-0.85z)| lr 5.32e-05 | 323.37 ms | 52.2% bf16 MFU | 1624163 tok/s
step 15931/19560 | loss 3.314368 (+0.03z)| norm 0.2808 (-0.32z)| lr 5.32e-05 | 322.63 ms | 52.3% bf16 MFU | 1624208 tok/s
step 15932/19560 | loss 3.296925 (-0.32z)| norm 0.2636 (-0.90z)| lr 5.32e-05 | 322.89 ms | 52.3% bf16 MFU | 1624186 tok/s
step 15933/19560 | loss 3.304350 (-0.17z)| norm 0.2811 (-0.30z)| lr 5.31e-05 | 323.32 ms | 52.2% bf16 MFU | 1624055 tok/s
step 15934/19560 | loss 3.310516 (-0.05z)| norm 0.2968 (+0.25z)| lr 5.31e-05 | 323.09 ms | 52.2% bf16 MFU | 1623990 tok/s
step 15935/19560 | loss 3.269867 (-0.84z)| norm 0.2693 (-0.70z)| lr 5.31e-05 | 322.76 ms | 52.3% bf16 MFU | 1624010 tok/s
step 15936/19560 | loss 3.291560 (-0.42z)| norm 0.2781 (-0.39z)| lr 5.31e-05 | 322.77 ms | 52.3% bf16 MFU | 1624025 tok/s
step 15937/19560 | loss 3.294729 (-0.35z)| norm 0.3037 (+0.52z)| lr 5.30e-05 | 322.94 ms | 52.3% bf16 MFU | 1623999 tok/s
step 15938/19560 | loss 3.286427 (-0.51z)| norm 0.2982 (+0.32z)| lr 5.30e-05 | 322.58 ms | 52.3% bf16 MFU | 1624063 tok/s
step 15939/19560 | loss 3.279420 (-0.65z)| norm 0.2714 (-0.62z)| lr 5.30e-05 | 323.24 ms | 52.2% bf16 MFU | 1623959 tok/s
step 15940/19560 | loss 3.316958 (+0.09z)| norm 0.3213 (+1.11z)| lr 5.29e-05 | 322.39 ms | 52.4% bf16 MFU | 1624074 tok/s
step 15941/19560 | loss 3.327137 (+0.29z)| norm 0.2911 (+0.05z)| lr 5.29e-05 | 323.12 ms | 52.2% bf16 MFU | 1624000 tok/s
step 15942/19560 | loss 3.371433 (+1.15z)| norm 0.2599 (-1.02z)| lr 5.29e-05 | 323.05 ms | 52.2% bf16 MFU | 1623945 tok/s
step 15943/19560 | loss 3.318101 (+0.11z)| norm 0.4217 (+4.26z)| lr 5.29e-05 | 322.30 ms | 52.4% bf16 MFU | 1624084 tok/s
step 15944/19560 | loss 3.288884 (-0.45z)| norm 0.3230 (+1.04z)| lr 5.28e-05 | 323.09 ms | 52.2% bf16 MFU | 1624015 tok/s
step 15945/19560 | loss 3.251914 (-1.17z)| norm 0.3100 (+0.62z)| lr 5.28e-05 | 323.03 ms | 52.2% bf16 MFU | 1623966 tok/s
step 15946/19560 | loss 3.295333 (-0.30z)| norm 0.3902 (+3.07z)| lr 5.28e-05 | 322.87 ms | 52.3% bf16 MFU | 1623960 tok/s
step 15947/19560 | loss 3.302118 (-0.15z)| norm 0.2894 (-0.07z)| lr 5.27e-05 | 322.73 ms | 52.3% bf16 MFU | 1623988 tok/s
step 15948/19560 | loss 3.282860 (-0.54z)| norm 0.2750 (-0.52z)| lr 5.27e-05 | 322.91 ms | 52.3% bf16 MFU | 1623970 tok/s
step 15949/19560 | loss 3.289931 (-0.39z)| norm 0.3564 (+1.98z)| lr 5.27e-05 | 322.85 ms | 52.3% bf16 MFU | 1623968 tok/s
step 15950/19560 | loss 3.276397 (-0.66z)| norm 0.2629 (-0.90z)| lr 5.27e-05 | 322.78 ms | 52.3% bf16 MFU | 1623984 tok/s
step 15951/19560 | loss 3.331661 (+0.44z)| norm 0.2845 (-0.24z)| lr 5.26e-05 | 323.14 ms | 52.2% bf16 MFU | 1623908 tok/s
step 15952/19560 | loss 3.310345 (+0.01z)| norm 0.3094 (+0.52z)| lr 5.26e-05 | 322.88 ms | 52.3% bf16 MFU | 1623902 tok/s
step 15953/19560 | loss 3.395862 (+1.70z)| norm 0.2836 (-0.27z)| lr 5.26e-05 | 322.76 ms | 52.3% bf16 MFU | 1623926 tok/s
step 15954/19560 | loss 3.301749 (-0.18z)| norm 0.2964 (+0.12z)| lr 5.25e-05 | 323.31 ms | 52.2% bf16 MFU | 1623811 tok/s
step 15955/19560 | loss 3.267351 (-0.85z)| norm 0.3423 (+1.57z)| lr 5.25e-05 | 322.78 ms | 52.3% bf16 MFU | 1623835 tok/s
step 15956/19560 | loss 3.328835 (+0.45z)| norm 0.2786 (-0.43z)| lr 5.25e-05 | 323.20 ms | 52.2% bf16 MFU | 1623752 tok/s
step 15957/19560 | loss 3.342355 (+0.74z)| norm 0.3409 (+1.67z)| lr 5.25e-05 | 322.63 ms | 52.3% bf16 MFU | 1623815 tok/s
step 15958/19560 | loss 3.281383 (-0.60z)| norm 0.3165 (+0.84z)| lr 5.24e-05 | 323.34 ms | 52.2% bf16 MFU | 1623699 tok/s
step 15959/19560 | loss 3.389221 (+1.74z)| norm 0.2844 (-0.24z)| lr 5.24e-05 | 323.23 ms | 52.2% bf16 MFU | 1623616 tok/s
step 15960/19560 | loss 3.279784 (-0.66z)| norm 0.3173 (+0.89z)| lr 5.24e-05 | 322.83 ms | 52.3% bf16 MFU | 1623636 tok/s
step 15961/19560 | loss 3.313101 (+0.09z)| norm 0.3346 (+1.46z)| lr 5.23e-05 | 323.19 ms | 52.2% bf16 MFU | 1623566 tok/s
step 15962/19560 | loss 3.275019 (-0.75z)| norm 0.2556 (-1.21z)| lr 5.23e-05 | 322.74 ms | 52.3% bf16 MFU | 1623612 tok/s
step 15963/19560 | loss 3.337929 (+0.66z)| norm 0.2674 (-0.81z)| lr 5.23e-05 | 323.10 ms | 52.2% bf16 MFU | 1623565 tok/s
step 15964/19560 | loss 3.327703 (+0.42z)| norm 0.2956 (+0.19z)| lr 5.23e-05 | 322.96 ms | 52.3% bf16 MFU | 1623557 tok/s
step 15965/19560 | loss 3.302185 (-0.15z)| norm 0.2789 (-0.39z)| lr 5.22e-05 | 322.45 ms | 52.3% bf16 MFU | 1623677 tok/s
step 15966/19560 | loss 3.306412 (-0.04z)| norm 0.2915 (+0.06z)| lr 5.22e-05 | 323.08 ms | 52.2% bf16 MFU | 1623632 tok/s
step 15967/19560 | loss 3.308930 (+0.03z)| norm 0.2954 (+0.21z)| lr 5.22e-05 | 322.78 ms | 52.3% bf16 MFU | 1623664 tok/s
step 15968/19560 | loss 3.211644 (-2.15z)| norm 0.2741 (-0.54z)| lr 5.21e-05 | 323.04 ms | 52.2% bf16 MFU | 1623631 tok/s
step 15969/19560 | loss 3.289784 (-0.39z)| norm 0.2902 (+0.05z)| lr 5.21e-05 | 323.11 ms | 52.2% bf16 MFU | 1623580 tok/s
step 15970/19560 | loss 3.252625 (-1.22z)| norm 0.2910 (+0.09z)| lr 5.21e-05 | 322.72 ms | 52.3% bf16 MFU | 1623630 tok/s
step 15971/19560 | loss 3.334761 (+0.62z)| norm 0.2887 (+0.00z)| lr 5.21e-05 | 322.78 ms | 52.3% bf16 MFU | 1623662 tok/s
step 15972/19560 | loss 3.296592 (-0.23z)| norm 0.3049 (+0.61z)| lr 5.20e-05 | 323.40 ms | 52.2% bf16 MFU | 1623538 tok/s
step 15973/19560 | loss 3.306613 (-0.00z)| norm 0.2989 (+0.40z)| lr 5.20e-05 | 322.62 ms | 52.3% bf16 MFU | 1623616 tok/s
step 15974/19560 | loss 3.279340 (-0.61z)| norm 0.2812 (-0.26z)| lr 5.20e-05 | 322.98 ms | 52.3% bf16 MFU | 1623599 tok/s
step 15975/19560 | loss 3.316048 (+0.20z)| norm 0.2922 (+0.15z)| lr 5.19e-05 | 323.01 ms | 52.3% bf16 MFU | 1623576 tok/s
step 15976/19560 | loss 3.374909 (+1.51z)| norm 0.3123 (+0.89z)| lr 5.19e-05 | 323.04 ms | 52.2% bf16 MFU | 1623548 tok/s
step 15977/19560 | loss 3.333577 (+0.57z)| norm 0.3067 (+0.68z)| lr 5.19e-05 | 322.49 ms | 52.3% bf16 MFU | 1623657 tok/s
step 15978/19560 | loss 3.331626 (+0.52z)| norm 0.2568 (-1.17z)| lr 5.19e-05 | 323.03 ms | 52.2% bf16 MFU | 1623627 tok/s
step 15979/19560 | loss 3.324714 (+0.36z)| norm 0.3094 (+0.77z)| lr 5.18e-05 | 322.60 ms | 52.3% bf16 MFU | 1623706 tok/s
step 15980/19560 | loss 3.284124 (-0.56z)| norm 0.2963 (+0.28z)| lr 5.18e-05 | 322.45 ms | 52.3% bf16 MFU | 1623817 tok/s
step 15981/19560 | loss 3.369151 (+1.47z)| norm 0.3291 (+1.47z)| lr 5.18e-05 | 322.95 ms | 52.3% bf16 MFU | 1623798 tok/s
step 15982/19560 | loss 3.272496 (-0.87z)| norm 0.2732 (-0.58z)| lr 5.18e-05 | 322.86 ms | 52.3% bf16 MFU | 1623804 tok/s
step 15983/19560 | loss 3.315534 (+0.17z)| norm 0.3174 (+1.04z)| lr 5.17e-05 | 322.47 ms | 52.3% bf16 MFU | 1623907 tok/s
step 15984/19560 | loss 3.333272 (+0.60z)| norm 0.3421 (+1.90z)| lr 5.17e-05 | 322.74 ms | 52.3% bf16 MFU | 1623936 tok/s
step 15985/19560 | loss 3.269085 (-0.99z)| norm 0.2778 (-0.43z)| lr 5.17e-05 | 323.36 ms | 52.2% bf16 MFU | 1623808 tok/s
step 15986/19560 | loss 3.362432 (+1.29z)| norm 0.3419 (+1.85z)| lr 5.16e-05 | 322.54 ms | 52.3% bf16 MFU | 1623892 tok/s
step 15987/19560 | loss 3.295207 (-0.36z)| norm 0.2781 (-0.43z)| lr 5.16e-05 | 322.34 ms | 52.4% bf16 MFU | 1624022 tok/s
step 15988/19560 | loss 3.378078 (+1.65z)| norm 0.3077 (+0.62z)| lr 5.16e-05 | 322.76 ms | 52.3% bf16 MFU | 1624040 tok/s
step 15989/19560 | loss 3.361683 (+1.24z)| norm 0.2969 (+0.23z)| lr 5.16e-05 | 323.52 ms | 52.2% bf16 MFU | 1623866 tok/s
step 15990/19560 | loss 3.295175 (-0.39z)| norm 0.2651 (-0.91z)| lr 5.15e-05 | 322.40 ms | 52.3% bf16 MFU | 1623982 tok/s
step 15991/19560 | loss 3.308168 (-0.07z)| norm 0.2608 (-1.06z)| lr 5.15e-05 | 322.62 ms | 52.3% bf16 MFU | 1624038 tok/s
step 15992/19560 | loss 3.289668 (-0.52z)| norm 0.2924 (+0.08z)| lr 5.15e-05 | 322.17 ms | 52.4% bf16 MFU | 1624205 tok/s
step 15993/19560 | loss 3.346703 (+0.88z)| norm 0.2739 (-0.58z)| lr 5.14e-05 | 322.52 ms | 52.3% bf16 MFU | 1624275 tok/s
step 15994/19560 | loss 3.348096 (+0.91z)| norm 0.2705 (-0.70z)| lr 5.14e-05 | 322.62 ms | 52.3% bf16 MFU | 1624316 tok/s
step 15995/19560 | loss 3.302676 (-0.21z)| norm 0.2554 (-1.23z)| lr 5.14e-05 | 322.67 ms | 52.3% bf16 MFU | 1624343 tok/s
step 15996/19560 | loss 3.286743 (-0.60z)| norm 0.2873 (-0.10z)| lr 5.14e-05 | 322.43 ms | 52.3% bf16 MFU | 1624428 tok/s
step 15997/19560 | loss 3.308393 (-0.07z)| norm 0.2833 (-0.24z)| lr 5.13e-05 | 322.76 ms | 52.3% bf16 MFU | 1624425 tok/s
step 15998/19560 | loss 3.356949 (+1.11z)| norm 0.2860 (-0.16z)| lr 5.13e-05 | 322.44 ms | 52.3% bf16 MFU | 1624503 tok/s
step 15999/19560 | loss 3.315861 (+0.10z)| norm 0.2945 (+0.14z)| lr 5.13e-05 | 322.69 ms | 52.3% bf16 MFU | 1624515 tok/s
step 16000/19560 | loss 3.338696 (+0.65z)| norm 0.2876 (-0.11z)| lr 5.12e-05 | 322.53 ms | 52.3% bf16 MFU | 1624568 tok/s
val loss 3.290730
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3030/10042 = 0.301733
step 16001/19560 | loss 3.344574 (+0.78z)| norm 0.3042 (+0.48z)| lr 5.12e-05 | 322.58 ms | 52.3% bf16 MFU | 1624603 tok/s
step 16002/19560 | loss 3.352335 (+0.96z)| norm 0.2575 (-1.20z)| lr 5.12e-05 | 323.16 ms | 52.2% bf16 MFU | 1624491 tok/s
step 16003/19560 | loss 3.291307 (-0.54z)| norm 0.3963 (+3.59z)| lr 5.12e-05 | 323.58 ms | 52.2% bf16 MFU | 1624280 tok/s
step 16004/19560 | loss 3.246445 (-1.64z)| norm 0.2459 (-1.55z)| lr 5.11e-05 | 323.04 ms | 52.2% bf16 MFU | 1624215 tok/s
step 16005/19560 | loss 3.306392 (-0.15z)| norm 0.2581 (-1.13z)| lr 5.11e-05 | 323.19 ms | 52.2% bf16 MFU | 1624116 tok/s
step 16006/19560 | loss 3.260842 (-1.27z)| norm 0.3310 (+1.32z)| lr 5.11e-05 | 322.78 ms | 52.3% bf16 MFU | 1624125 tok/s
step 16007/19560 | loss 3.293781 (-0.46z)| norm 0.2846 (-0.24z)| lr 5.11e-05 | 322.55 ms | 52.3% bf16 MFU | 1624190 tok/s
step 16008/19560 | loss 3.374294 (+1.53z)| norm 0.3138 (+0.74z)| lr 5.10e-05 | 323.07 ms | 52.2% bf16 MFU | 1624122 tok/s
step 16009/19560 | loss 3.377452 (+1.58z)| norm 0.3228 (+1.04z)| lr 5.10e-05 | 323.08 ms | 52.2% bf16 MFU | 1624055 tok/s
step 16010/19560 | loss 3.296765 (-0.39z)| norm 0.2769 (-0.51z)| lr 5.10e-05 | 322.67 ms | 52.3% bf16 MFU | 1624095 tok/s
step 16011/19560 | loss 3.377637 (+1.67z)| norm 0.2753 (-0.57z)| lr 5.09e-05 | 322.99 ms | 52.3% bf16 MFU | 1624053 tok/s
step 16012/19560 | loss 3.342715 (+0.77z)| norm 0.2786 (-0.46z)| lr 5.09e-05 | 322.87 ms | 52.3% bf16 MFU | 1624042 tok/s
step 16013/19560 | loss 3.303759 (-0.21z)| norm 0.2718 (-0.69z)| lr 5.09e-05 | 323.70 ms | 52.1% bf16 MFU | 1623824 tok/s
step 16014/19560 | loss 3.374368 (+1.56z)| norm 0.3265 (+1.15z)| lr 5.09e-05 | 322.85 ms | 52.3% bf16 MFU | 1623828 tok/s
step 16015/19560 | loss 3.332073 (+0.48z)| norm 0.3220 (+0.98z)| lr 5.08e-05 | 322.61 ms | 52.3% bf16 MFU | 1623894 tok/s
step 16016/19560 | loss 3.292134 (-0.52z)| norm 0.2781 (-0.49z)| lr 5.08e-05 | 323.07 ms | 52.2% bf16 MFU | 1623842 tok/s
step 16017/19560 | loss 3.355213 (+1.08z)| norm 0.3439 (+1.69z)| lr 5.08e-05 | 322.42 ms | 52.3% bf16 MFU | 1623956 tok/s
step 16018/19560 | loss 3.358333 (+1.14z)| norm 0.3030 (+0.33z)| lr 5.07e-05 | 322.84 ms | 52.3% bf16 MFU | 1623956 tok/s
step 16019/19560 | loss 3.310831 (-0.08z)| norm 0.2954 (+0.07z)| lr 5.07e-05 | 323.01 ms | 52.3% bf16 MFU | 1623916 tok/s
step 16020/19560 | loss 3.309415 (-0.12z)| norm 0.3126 (+0.65z)| lr 5.07e-05 | 322.57 ms | 52.3% bf16 MFU | 1623987 tok/s
step 16021/19560 | loss 3.296208 (-0.45z)| norm 0.2647 (-0.95z)| lr 5.07e-05 | 322.95 ms | 52.3% bf16 MFU | 1623959 tok/s
step 16022/19560 | loss 3.371191 (+1.45z)| norm 0.3177 (+0.83z)| lr 5.06e-05 | 322.89 ms | 52.3% bf16 MFU | 1623948 tok/s
step 16023/19560 | loss 3.308937 (-0.13z)| norm 0.2528 (-1.32z)| lr 5.06e-05 | 322.85 ms | 52.3% bf16 MFU | 1623948 tok/s
step 16024/19560 | loss 3.345737 (+0.81z)| norm 0.2904 (-0.08z)| lr 5.06e-05 | 323.09 ms | 52.2% bf16 MFU | 1623888 tok/s
step 16025/19560 | loss 3.355719 (+1.04z)| norm 0.3329 (+1.41z)| lr 5.06e-05 | 322.16 ms | 52.4% bf16 MFU | 1624064 tok/s
step 16026/19560 | loss 3.347530 (+0.83z)| norm 0.2649 (-0.95z)| lr 5.05e-05 | 322.84 ms | 52.3% bf16 MFU | 1624060 tok/s
step 16027/19560 | loss 3.271646 (-1.12z)| norm 0.2872 (-0.18z)| lr 5.05e-05 | 322.41 ms | 52.3% bf16 MFU | 1624166 tok/s
step 16028/19560 | loss 3.284704 (-0.77z)| norm 0.3387 (+1.59z)| lr 5.05e-05 | 322.58 ms | 52.3% bf16 MFU | 1624223 tok/s
step 16029/19560 | loss 3.379614 (+1.69z)| norm 0.2747 (-0.62z)| lr 5.04e-05 | 322.90 ms | 52.3% bf16 MFU | 1624195 tok/s
step 16030/19560 | loss 3.293390 (-0.54z)| norm 0.3272 (+1.18z)| lr 5.04e-05 | 322.52 ms | 52.3% bf16 MFU | 1624266 tok/s
step 16031/19560 | loss 3.217081 (-2.43z)| norm 0.3031 (+0.35z)| lr 5.04e-05 | 322.74 ms | 52.3% bf16 MFU | 1624278 tok/s
step 16032/19560 | loss 3.316861 (+0.08z)| norm 0.2598 (-1.15z)| lr 5.04e-05 | 322.95 ms | 52.3% bf16 MFU | 1624237 tok/s
step 16033/19560 | loss 3.235640 (-1.93z)| norm 0.3222 (+1.00z)| lr 5.03e-05 | 322.68 ms | 52.3% bf16 MFU | 1624264 tok/s
step 16034/19560 | loss 3.324781 (+0.29z)| norm 0.2792 (-0.48z)| lr 5.03e-05 | 322.83 ms | 52.3% bf16 MFU | 1624252 tok/s
step 16035/19560 | loss 3.288124 (-0.62z)| norm 0.2844 (-0.31z)| lr 5.03e-05 | 322.55 ms | 52.3% bf16 MFU | 1624312 tok/s
step 16036/19560 | loss 3.307736 (-0.12z)| norm 0.2772 (-0.55z)| lr 5.02e-05 | 322.93 ms | 52.3% bf16 MFU | 1624273 tok/s
step 16037/19560 | loss 3.320660 (+0.19z)| norm 0.2728 (-0.69z)| lr 5.02e-05 | 322.64 ms | 52.3% bf16 MFU | 1624310 tok/s
step 16038/19560 | loss 3.442259 (+3.14z)| norm 0.2849 (-0.28z)| lr 5.02e-05 | 322.86 ms | 52.3% bf16 MFU | 1624289 tok/s
step 16039/19560 | loss 3.306398 (-0.20z)| norm 0.2668 (-0.89z)| lr 5.02e-05 | 322.95 ms | 52.3% bf16 MFU | 1624246 tok/s
step 16040/19560 | loss 3.343848 (+0.72z)| norm 0.2660 (-0.92z)| lr 5.01e-05 | 322.24 ms | 52.4% bf16 MFU | 1624383 tok/s
step 16041/19560 | loss 3.286994 (-0.68z)| norm 0.2803 (-0.43z)| lr 5.01e-05 | 322.65 ms | 52.3% bf16 MFU | 1624412 tok/s
step 16042/19560 | loss 3.335349 (+0.52z)| norm 0.3042 (+0.39z)| lr 5.01e-05 | 322.71 ms | 52.3% bf16 MFU | 1624425 tok/s
step 16043/19560 | loss 3.315477 (+0.04z)| norm 0.2989 (+0.20z)| lr 5.01e-05 | 322.77 ms | 52.3% bf16 MFU | 1624420 tok/s
step 16044/19560 | loss 3.280830 (-0.82z)| norm 0.3033 (+0.35z)| lr 5.00e-05 | 322.80 ms | 52.3% bf16 MFU | 1624409 tok/s
step 16045/19560 | loss 3.332346 (+0.46z)| norm 0.3010 (+0.27z)| lr 5.00e-05 | 323.00 ms | 52.3% bf16 MFU | 1624348 tok/s
step 16046/19560 | loss 3.273233 (-1.00z)| norm 0.3242 (+1.05z)| lr 5.00e-05 | 322.76 ms | 52.3% bf16 MFU | 1624350 tok/s
step 16047/19560 | loss 3.280535 (-0.81z)| norm 0.2883 (-0.18z)| lr 4.99e-05 | 322.52 ms | 52.3% bf16 MFU | 1624413 tok/s
step 16048/19560 | loss 3.259958 (-1.33z)| norm 0.2766 (-0.59z)| lr 4.99e-05 | 323.00 ms | 52.3% bf16 MFU | 1624351 tok/s
step 16049/19560 | loss 3.322164 (+0.22z)| norm 0.3071 (+0.45z)| lr 4.99e-05 | 322.47 ms | 52.3% bf16 MFU | 1624427 tok/s
step 16050/19560 | loss 3.318680 (+0.12z)| norm 0.3127 (+0.64z)| lr 4.99e-05 | 322.17 ms | 52.4% bf16 MFU | 1624574 tok/s
step 16051/19560 | loss 3.266283 (-1.17z)| norm 0.2618 (-1.12z)| lr 4.98e-05 | 322.70 ms | 52.3% bf16 MFU | 1624580 tok/s
step 16052/19560 | loss 3.310936 (-0.06z)| norm 0.2747 (-0.67z)| lr 4.98e-05 | 322.57 ms | 52.3% bf16 MFU | 1624619 tok/s
step 16053/19560 | loss 3.309165 (-0.10z)| norm 0.3197 (+0.86z)| lr 4.98e-05 | 322.22 ms | 52.4% bf16 MFU | 1624743 tok/s
step 16054/19560 | loss 3.365729 (+1.28z)| norm 0.2970 (+0.08z)| lr 4.97e-05 | 322.78 ms | 52.3% bf16 MFU | 1624719 tok/s
step 16055/19560 | loss 3.303207 (-0.24z)| norm 0.2994 (+0.15z)| lr 4.97e-05 | 322.60 ms | 52.3% bf16 MFU | 1624743 tok/s
step 16056/19560 | loss 3.310751 (-0.06z)| norm 0.2976 (+0.08z)| lr 4.97e-05 | 322.92 ms | 52.3% bf16 MFU | 1624684 tok/s
step 16057/19560 | loss 3.332296 (+0.50z)| norm 0.3160 (+0.71z)| lr 4.97e-05 | 322.49 ms | 52.3% bf16 MFU | 1624736 tok/s
step 16058/19560 | loss 3.335915 (+0.60z)| norm 0.3916 (+3.18z)| lr 4.96e-05 | 322.72 ms | 52.3% bf16 MFU | 1624730 tok/s
step 16059/19560 | loss 3.307213 (-0.17z)| norm 0.2680 (-0.95z)| lr 4.96e-05 | 322.95 ms | 52.3% bf16 MFU | 1624666 tok/s
step 16060/19560 | loss 3.353127 (+1.04z)| norm 0.3870 (+2.91z)| lr 4.96e-05 | 322.46 ms | 52.3% bf16 MFU | 1624729 tok/s
step 16061/19560 | loss 3.310510 (-0.10z)| norm 0.3557 (+1.85z)| lr 4.96e-05 | 322.49 ms | 52.3% bf16 MFU | 1624780 tok/s
step 16062/19560 | loss 3.265234 (-1.29z)| norm 0.2666 (-1.00z)| lr 4.95e-05 | 322.70 ms | 52.3% bf16 MFU | 1624777 tok/s
step 16063/19560 | loss 3.335639 (+0.57z)| norm 0.3248 (+0.85z)| lr 4.95e-05 | 322.67 ms | 52.3% bf16 MFU | 1624780 tok/s
step 16064/19560 | loss 3.355290 (+1.08z)| norm 0.3081 (+0.31z)| lr 4.95e-05 | 322.59 ms | 52.3% bf16 MFU | 1624802 tok/s
step 16065/19560 | loss 3.293238 (-0.57z)| norm 0.2953 (-0.10z)| lr 4.94e-05 | 322.60 ms | 52.3% bf16 MFU | 1624822 tok/s
step 16066/19560 | loss 3.286665 (-0.75z)| norm 0.2783 (-0.64z)| lr 4.94e-05 | 323.09 ms | 52.2% bf16 MFU | 1624719 tok/s
step 16067/19560 | loss 3.307889 (-0.19z)| norm 0.3379 (+1.25z)| lr 4.94e-05 | 323.04 ms | 52.2% bf16 MFU | 1624632 tok/s
step 16068/19560 | loss 3.353800 (+1.02z)| norm 0.2745 (-0.76z)| lr 4.94e-05 | 322.50 ms | 52.3% bf16 MFU | 1624686 tok/s
step 16069/19560 | loss 3.298265 (-0.45z)| norm 0.2437 (-1.71z)| lr 4.93e-05 | 322.41 ms | 52.3% bf16 MFU | 1624758 tok/s
step 16070/19560 | loss 3.258660 (-1.48z)| norm 0.3878 (+2.73z)| lr 4.93e-05 | 322.37 ms | 52.4% bf16 MFU | 1624838 tok/s
step 16071/19560 | loss 3.361066 (+1.23z)| norm 0.2931 (-0.16z)| lr 4.93e-05 | 323.22 ms | 52.2% bf16 MFU | 1624701 tok/s
step 16072/19560 | loss 3.310133 (-0.12z)| norm 0.3038 (+0.19z)| lr 4.93e-05 | 322.78 ms | 52.3% bf16 MFU | 1624682 tok/s
step 16073/19560 | loss 3.347182 (+0.85z)| norm 0.3778 (+2.53z)| lr 4.92e-05 | 323.20 ms | 52.2% bf16 MFU | 1624557 tok/s
step 16074/19560 | loss 3.348867 (+0.88z)| norm 0.2823 (-0.50z)| lr 4.92e-05 | 322.82 ms | 52.3% bf16 MFU | 1624534 tok/s
step 16075/19560 | loss 3.321297 (+0.14z)| norm 0.3268 (+0.95z)| lr 4.92e-05 | 323.00 ms | 52.3% bf16 MFU | 1624466 tok/s
step 16076/19560 | loss 3.340135 (+0.63z)| norm 0.2976 (-0.01z)| lr 4.91e-05 | 322.38 ms | 52.4% bf16 MFU | 1624557 tok/s
step 16077/19560 | loss 3.198574 (-3.02z)| norm 0.2578 (-1.31z)| lr 4.91e-05 | 322.87 ms | 52.3% bf16 MFU | 1624522 tok/s
step 16078/19560 | loss 3.346881 (+0.79z)| norm 0.2869 (-0.35z)| lr 4.91e-05 | 322.60 ms | 52.3% bf16 MFU | 1624554 tok/s
step 16079/19560 | loss 3.265962 (-1.28z)| norm 0.2625 (-1.15z)| lr 4.91e-05 | 322.67 ms | 52.3% bf16 MFU | 1624568 tok/s
step 16080/19560 | loss 3.312431 (-0.09z)| norm 0.2647 (-1.07z)| lr 4.90e-05 | 323.35 ms | 52.2% bf16 MFU | 1624410 tok/s
step 16081/19560 | loss 3.225749 (-2.27z)| norm 0.2979 (+0.03z)| lr 4.90e-05 | 322.68 ms | 52.3% bf16 MFU | 1624428 tok/s
step 16082/19560 | loss 3.315031 (+0.01z)| norm 0.2856 (-0.37z)| lr 4.90e-05 | 322.96 ms | 52.3% bf16 MFU | 1624376 tok/s
step 16083/19560 | loss 3.361091 (+1.17z)| norm 0.3415 (+1.47z)| lr 4.90e-05 | 322.79 ms | 52.3% bf16 MFU | 1624368 tok/s
step 16084/19560 | loss 3.302012 (-0.34z)| norm 0.2782 (-0.62z)| lr 4.89e-05 | 323.14 ms | 52.2% bf16 MFU | 1624273 tok/s
step 16085/19560 | loss 3.287635 (-0.69z)| norm 0.2579 (-1.27z)| lr 4.89e-05 | 322.51 ms | 52.3% bf16 MFU | 1624341 tok/s
step 16086/19560 | loss 3.303934 (-0.28z)| norm 0.2688 (-0.90z)| lr 4.89e-05 | 322.65 ms | 52.3% bf16 MFU | 1624372 tok/s
step 16087/19560 | loss 3.276783 (-0.97z)| norm 0.2860 (-0.33z)| lr 4.88e-05 | 323.68 ms | 52.1% bf16 MFU | 1624142 tok/s
step 16088/19560 | loss 3.238891 (-1.92z)| norm 0.2839 (-0.39z)| lr 4.88e-05 | 322.68 ms | 52.3% bf16 MFU | 1624175 tok/s
step 16089/19560 | loss 3.344459 (+0.78z)| norm 0.2793 (-0.53z)| lr 4.88e-05 | 323.06 ms | 52.2% bf16 MFU | 1624111 tok/s
step 16090/19560 | loss 3.432699 (+2.92z)| norm 0.2833 (-0.41z)| lr 4.88e-05 | 322.97 ms | 52.3% bf16 MFU | 1624072 tok/s
step 16091/19560 | loss 3.263029 (-1.27z)| norm 0.2652 (-1.01z)| lr 4.87e-05 | 322.37 ms | 52.4% bf16 MFU | 1624186 tok/s
step 16092/19560 | loss 3.303225 (-0.28z)| norm 0.2644 (-1.03z)| lr 4.87e-05 | 323.35 ms | 52.2% bf16 MFU | 1624049 tok/s
step 16093/19560 | loss 3.251394 (-1.53z)| norm 0.2594 (-1.18z)| lr 4.87e-05 | 323.26 ms | 52.2% bf16 MFU | 1623941 tok/s
step 16094/19560 | loss 3.331973 (+0.44z)| norm 0.2646 (-1.00z)| lr 4.87e-05 | 322.70 ms | 52.3% bf16 MFU | 1623978 tok/s
step 16095/19560 | loss 3.223415 (-2.16z)| norm 0.4145 (+3.71z)| lr 4.86e-05 | 323.04 ms | 52.2% bf16 MFU | 1623929 tok/s
step 16096/19560 | loss 3.245883 (-1.65z)| norm 0.2777 (-0.56z)| lr 4.86e-05 | 322.84 ms | 52.3% bf16 MFU | 1623931 tok/s
step 16097/19560 | loss 3.346526 (+0.78z)| norm 0.2876 (-0.26z)| lr 4.86e-05 | 322.53 ms | 52.3% bf16 MFU | 1624011 tok/s
step 16098/19560 | loss 3.349602 (+0.85z)| norm 0.2918 (-0.12z)| lr 4.85e-05 | 322.91 ms | 52.3% bf16 MFU | 1623993 tok/s
step 16099/19560 | loss 3.354296 (+0.95z)| norm 0.2821 (-0.42z)| lr 4.85e-05 | 322.75 ms | 52.3% bf16 MFU | 1624016 tok/s
step 16100/19560 | loss 3.285868 (-0.71z)| norm 0.2769 (-0.58z)| lr 4.85e-05 | 323.16 ms | 52.2% bf16 MFU | 1623935 tok/s
step 16101/19560 | loss 3.235953 (-1.89z)| norm 0.2833 (-0.38z)| lr 4.85e-05 | 323.14 ms | 52.2% bf16 MFU | 1623863 tok/s
step 16102/19560 | loss 3.352267 (+0.89z)| norm 0.2870 (-0.26z)| lr 4.84e-05 | 323.06 ms | 52.2% bf16 MFU | 1623815 tok/s
step 16103/19560 | loss 3.317809 (+0.07z)| norm 0.2878 (-0.24z)| lr 4.84e-05 | 322.61 ms | 52.3% bf16 MFU | 1623881 tok/s
step 16104/19560 | loss 3.329689 (+0.36z)| norm 0.2937 (-0.05z)| lr 4.84e-05 | 322.80 ms | 52.3% bf16 MFU | 1623897 tok/s
step 16105/19560 | loss 3.311657 (-0.07z)| norm 0.2794 (-0.49z)| lr 4.84e-05 | 322.93 ms | 52.3% bf16 MFU | 1623879 tok/s
step 16106/19560 | loss 3.240816 (-1.75z)| norm 0.2864 (-0.28z)| lr 4.83e-05 | 322.65 ms | 52.3% bf16 MFU | 1623932 tok/s
step 16107/19560 | loss 3.326807 (+0.31z)| norm 0.2860 (-0.28z)| lr 4.83e-05 | 322.79 ms | 52.3% bf16 MFU | 1623948 tok/s
step 16108/19560 | loss 3.350672 (+0.87z)| norm 0.3158 (+0.64z)| lr 4.83e-05 | 323.41 ms | 52.2% bf16 MFU | 1623808 tok/s
step 16109/19560 | loss 3.337321 (+0.56z)| norm 0.2750 (-0.62z)| lr 4.82e-05 | 322.84 ms | 52.3% bf16 MFU | 1623818 tok/s
step 16110/19560 | loss 3.375608 (+1.45z)| norm 0.2699 (-0.78z)| lr 4.82e-05 | 323.00 ms | 52.3% bf16 MFU | 1623787 tok/s
step 16111/19560 | loss 3.251741 (-1.49z)| norm 0.2999 (+0.17z)| lr 4.82e-05 | 322.68 ms | 52.3% bf16 MFU | 1623837 tok/s
step 16112/19560 | loss 3.332036 (+0.42z)| norm 0.2855 (-0.28z)| lr 4.82e-05 | 322.90 ms | 52.3% bf16 MFU | 1623830 tok/s
step 16113/19560 | loss 3.268591 (-1.09z)| norm 0.2585 (-1.12z)| lr 4.81e-05 | 322.56 ms | 52.3% bf16 MFU | 1623908 tok/s
step 16114/19560 | loss 3.327411 (+0.32z)| norm 0.2618 (-1.01z)| lr 4.81e-05 | 322.55 ms | 52.3% bf16 MFU | 1623985 tok/s
step 16115/19560 | loss 3.376813 (+1.47z)| norm 0.3216 (+0.88z)| lr 4.81e-05 | 322.83 ms | 52.3% bf16 MFU | 1623988 tok/s
step 16116/19560 | loss 3.298031 (-0.38z)| norm 0.2852 (-0.26z)| lr 4.81e-05 | 323.43 ms | 52.2% bf16 MFU | 1623840 tok/s
step 16117/19560 | loss 3.269676 (-1.04z)| norm 0.3157 (+0.70z)| lr 4.80e-05 | 323.05 ms | 52.2% bf16 MFU | 1623796 tok/s
step 16118/19560 | loss 3.293563 (-0.47z)| norm 0.3479 (+1.68z)| lr 4.80e-05 | 322.95 ms | 52.3% bf16 MFU | 1623777 tok/s
step 16119/19560 | loss 3.340594 (+0.64z)| norm 0.2776 (-0.53z)| lr 4.80e-05 | 322.95 ms | 52.3% bf16 MFU | 1623759 tok/s
step 16120/19560 | loss 3.275745 (-0.90z)| norm 0.3161 (+0.67z)| lr 4.79e-05 | 323.08 ms | 52.2% bf16 MFU | 1623711 tok/s
step 16121/19560 | loss 3.334311 (+0.50z)| norm 0.3140 (+0.60z)| lr 4.79e-05 | 323.55 ms | 52.2% bf16 MFU | 1623546 tok/s
step 16122/19560 | loss 3.301463 (-0.28z)| norm 0.2661 (-0.91z)| lr 4.79e-05 | 322.76 ms | 52.3% bf16 MFU | 1623588 tok/s
step 16123/19560 | loss 3.323359 (+0.24z)| norm 0.3069 (+0.36z)| lr 4.79e-05 | 322.33 ms | 52.4% bf16 MFU | 1623736 tok/s
step 16124/19560 | loss 3.383227 (+1.64z)| norm 0.2701 (-0.79z)| lr 4.78e-05 | 323.79 ms | 52.1% bf16 MFU | 1623511 tok/s
step 16125/19560 | loss 3.282588 (-0.74z)| norm 0.2585 (-1.15z)| lr 4.78e-05 | 322.48 ms | 52.3% bf16 MFU | 1623626 tok/s
step 16126/19560 | loss 3.328521 (+0.35z)| norm 0.2720 (-0.72z)| lr 4.78e-05 | 323.06 ms | 52.2% bf16 MFU | 1623589 tok/s
step 16127/19560 | loss 3.308058 (-0.13z)| norm 0.2612 (-1.05z)| lr 4.78e-05 | 323.15 ms | 52.2% bf16 MFU | 1623530 tok/s
step 16128/19560 | loss 3.276701 (-0.86z)| norm 0.2922 (-0.08z)| lr 4.77e-05 | 322.46 ms | 52.3% bf16 MFU | 1623649 tok/s
step 16129/19560 | loss 3.378535 (+1.53z)| norm 0.2583 (-1.12z)| lr 4.77e-05 | 323.39 ms | 52.2% bf16 MFU | 1623528 tok/s
step 16130/19560 | loss 3.311102 (-0.04z)| norm 0.2723 (-0.69z)| lr 4.77e-05 | 322.90 ms | 52.3% bf16 MFU | 1623537 tok/s
step 16131/19560 | loss 3.326120 (+0.30z)| norm 0.2777 (-0.51z)| lr 4.76e-05 | 322.77 ms | 52.3% bf16 MFU | 1623576 tok/s
step 16132/19560 | loss 3.317485 (+0.09z)| norm 0.2693 (-0.80z)| lr 4.76e-05 | 322.31 ms | 52.4% bf16 MFU | 1623729 tok/s
step 16133/19560 | loss 3.311620 (-0.05z)| norm 0.2452 (-1.58z)| lr 4.76e-05 | 323.12 ms | 52.2% bf16 MFU | 1623673 tok/s
step 16134/19560 | loss 3.302126 (-0.29z)| norm 0.2799 (-0.44z)| lr 4.76e-05 | 323.60 ms | 52.2% bf16 MFU | 1623498 tok/s
step 16135/19560 | loss 3.317880 (+0.08z)| norm 0.3348 (+1.34z)| lr 4.75e-05 | 322.05 ms | 52.4% bf16 MFU | 1623721 tok/s
step 16136/19560 | loss 3.286719 (-0.65z)| norm 0.2819 (-0.37z)| lr 4.75e-05 | 323.35 ms | 52.2% bf16 MFU | 1623607 tok/s
step 16137/19560 | loss 3.279882 (-0.80z)| norm 0.3104 (+0.56z)| lr 4.75e-05 | 323.22 ms | 52.2% bf16 MFU | 1623532 tok/s
step 16138/19560 | loss 3.253936 (-1.42z)| norm 0.3381 (+1.44z)| lr 4.75e-05 | 323.02 ms | 52.2% bf16 MFU | 1623508 tok/s
step 16139/19560 | loss 3.296311 (-0.38z)| norm 0.3111 (+0.55z)| lr 4.74e-05 | 323.03 ms | 52.2% bf16 MFU | 1623485 tok/s
step 16140/19560 | loss 3.347900 (+0.87z)| norm 0.3012 (+0.23z)| lr 4.74e-05 | 322.79 ms | 52.3% bf16 MFU | 1623522 tok/s
step 16141/19560 | loss 3.324699 (+0.30z)| norm 0.3055 (+0.36z)| lr 4.74e-05 | 323.04 ms | 52.2% bf16 MFU | 1623495 tok/s
step 16142/19560 | loss 3.271284 (-0.98z)| norm 0.2802 (-0.45z)| lr 4.74e-05 | 322.63 ms | 52.3% bf16 MFU | 1623572 tok/s
step 16143/19560 | loss 3.284717 (-0.64z)| norm 0.2636 (-0.98z)| lr 4.73e-05 | 323.44 ms | 52.2% bf16 MFU | 1623443 tok/s
step 16144/19560 | loss 3.266397 (-1.08z)| norm 0.2736 (-0.65z)| lr 4.73e-05 | 322.74 ms | 52.3% bf16 MFU | 1623494 tok/s
step 16145/19560 | loss 3.250939 (-1.43z)| norm 0.2775 (-0.51z)| lr 4.73e-05 | 322.74 ms | 52.3% bf16 MFU | 1623543 tok/s
step 16146/19560 | loss 3.315921 (+0.15z)| norm 0.2741 (-0.62z)| lr 4.72e-05 | 322.86 ms | 52.3% bf16 MFU | 1623561 tok/s
step 16147/19560 | loss 3.391201 (+1.94z)| norm 0.2871 (-0.19z)| lr 4.72e-05 | 322.09 ms | 52.4% bf16 MFU | 1623772 tok/s
step 16148/19560 | loss 3.302616 (-0.18z)| norm 0.3029 (+0.33z)| lr 4.72e-05 | 322.93 ms | 52.3% bf16 MFU | 1623760 tok/s
step 16149/19560 | loss 3.332452 (+0.53z)| norm 0.2624 (-1.00z)| lr 4.72e-05 | 323.32 ms | 52.2% bf16 MFU | 1623651 tok/s
step 16150/19560 | loss 3.375373 (+1.55z)| norm 0.3486 (+1.81z)| lr 4.71e-05 | 323.10 ms | 52.2% bf16 MFU | 1623601 tok/s
step 16151/19560 | loss 3.320153 (+0.23z)| norm 0.2879 (-0.18z)| lr 4.71e-05 | 323.00 ms | 52.3% bf16 MFU | 1623580 tok/s
step 16152/19560 | loss 3.265921 (-1.05z)| norm 0.3293 (+1.16z)| lr 4.71e-05 | 322.65 ms | 52.3% bf16 MFU | 1623649 tok/s
step 16153/19560 | loss 3.322642 (+0.31z)| norm 0.2809 (-0.40z)| lr 4.71e-05 | 323.33 ms | 52.2% bf16 MFU | 1623543 tok/s
step 16154/19560 | loss 3.304676 (-0.11z)| norm 0.2814 (-0.39z)| lr 4.70e-05 | 323.35 ms | 52.2% bf16 MFU | 1623436 tok/s
step 16155/19560 | loss 3.279438 (-0.72z)| norm 0.2678 (-0.83z)| lr 4.70e-05 | 322.91 ms | 52.3% bf16 MFU | 1623445 tok/s
step 16156/19560 | loss 3.286697 (-0.55z)| norm 0.2710 (-0.71z)| lr 4.70e-05 | 323.22 ms | 52.2% bf16 MFU | 1623376 tok/s
step 16157/19560 | loss 3.309126 (+0.00z)| norm 0.2780 (-0.48z)| lr 4.69e-05 | 322.91 ms | 52.3% bf16 MFU | 1623390 tok/s
step 16158/19560 | loss 3.280148 (-0.70z)| norm 0.3190 (+0.87z)| lr 4.69e-05 | 322.81 ms | 52.3% bf16 MFU | 1623428 tok/s
step 16159/19560 | loss 3.245413 (-1.57z)| norm 0.2921 (-0.01z)| lr 4.69e-05 | 323.16 ms | 52.2% bf16 MFU | 1623377 tok/s
step 16160/19560 | loss 3.273786 (-0.86z)| norm 0.2746 (-0.60z)| lr 4.69e-05 | 322.76 ms | 52.3% bf16 MFU | 1623426 tok/s
step 16161/19560 | loss 3.295779 (-0.33z)| norm 0.3765 (+2.70z)| lr 4.68e-05 | 323.05 ms | 52.2% bf16 MFU | 1623400 tok/s
step 16162/19560 | loss 3.434667 (+2.99z)| norm 0.3347 (+1.33z)| lr 4.68e-05 | 322.67 ms | 52.3% bf16 MFU | 1623472 tok/s
step 16163/19560 | loss 3.309632 (-0.01z)| norm 0.3079 (+0.46z)| lr 4.68e-05 | 322.53 ms | 52.3% bf16 MFU | 1623576 tok/s
step 16164/19560 | loss 3.375283 (+1.54z)| norm 0.3732 (+2.47z)| lr 4.68e-05 | 322.85 ms | 52.3% bf16 MFU | 1623595 tok/s
step 16165/19560 | loss 3.277002 (-0.79z)| norm 0.3073 (+0.40z)| lr 4.67e-05 | 323.08 ms | 52.2% bf16 MFU | 1623555 tok/s
step 16166/19560 | loss 3.335339 (+0.64z)| norm 0.2605 (-1.06z)| lr 4.67e-05 | 322.69 ms | 52.3% bf16 MFU | 1623614 tok/s
step 16167/19560 | loss 3.258849 (-1.24z)| norm 0.2762 (-0.57z)| lr 4.67e-05 | 322.88 ms | 52.3% bf16 MFU | 1623623 tok/s
step 16168/19560 | loss 3.292316 (-0.41z)| norm 0.3426 (+1.48z)| lr 4.67e-05 | 322.77 ms | 52.3% bf16 MFU | 1623658 tok/s
step 16169/19560 | loss 3.329761 (+0.51z)| norm 0.2784 (-0.52z)| lr 4.66e-05 | 323.03 ms | 52.2% bf16 MFU | 1623626 tok/s
step 16170/19560 | loss 3.398138 (+2.14z)| norm 0.3193 (+0.75z)| lr 4.66e-05 | 322.61 ms | 52.3% bf16 MFU | 1623702 tok/s
step 16171/19560 | loss 3.320631 (+0.27z)| norm 0.3104 (+0.47z)| lr 4.66e-05 | 322.26 ms | 52.4% bf16 MFU | 1623861 tok/s
step 16172/19560 | loss 3.250484 (-1.42z)| norm 0.2820 (-0.41z)| lr 4.65e-05 | 322.88 ms | 52.3% bf16 MFU | 1623857 tok/s
step 16173/19560 | loss 3.290707 (-0.44z)| norm 0.3039 (+0.27z)| lr 4.65e-05 | 322.82 ms | 52.3% bf16 MFU | 1623869 tok/s
step 16174/19560 | loss 3.267334 (-1.00z)| norm 0.2712 (-0.73z)| lr 4.65e-05 | 322.81 ms | 52.3% bf16 MFU | 1623883 tok/s
step 16175/19560 | loss 3.290419 (-0.45z)| norm 0.3129 (+0.56z)| lr 4.65e-05 | 322.69 ms | 52.3% bf16 MFU | 1623926 tok/s
step 16176/19560 | loss 3.336328 (+0.64z)| norm 0.2866 (-0.26z)| lr 4.64e-05 | 322.47 ms | 52.3% bf16 MFU | 1624023 tok/s
step 16177/19560 | loss 3.245009 (-1.53z)| norm 0.2783 (-0.51z)| lr 4.64e-05 | 323.02 ms | 52.2% bf16 MFU | 1623977 tok/s
step 16178/19560 | loss 3.304717 (-0.10z)| norm 0.2648 (-0.92z)| lr 4.64e-05 | 323.02 ms | 52.2% bf16 MFU | 1623932 tok/s
step 16179/19560 | loss 3.291798 (-0.42z)| norm 0.3059 (+0.35z)| lr 4.64e-05 | 322.80 ms | 52.3% bf16 MFU | 1623945 tok/s
step 16180/19560 | loss 3.327510 (+0.44z)| norm 0.2970 (+0.06z)| lr 4.63e-05 | 322.76 ms | 52.3% bf16 MFU | 1623966 tok/s
step 16181/19560 | loss 3.352556 (+1.02z)| norm 0.2681 (-0.82z)| lr 4.63e-05 | 322.34 ms | 52.4% bf16 MFU | 1624093 tok/s
step 16182/19560 | loss 3.354105 (+1.07z)| norm 0.3086 (+0.44z)| lr 4.63e-05 | 322.46 ms | 52.3% bf16 MFU | 1624184 tok/s
step 16183/19560 | loss 3.306854 (-0.07z)| norm 0.3336 (+1.20z)| lr 4.63e-05 | 322.50 ms | 52.3% bf16 MFU | 1624260 tok/s
step 16184/19560 | loss 3.243268 (-1.56z)| norm 0.2843 (-0.32z)| lr 4.62e-05 | 322.87 ms | 52.3% bf16 MFU | 1624239 tok/s
step 16185/19560 | loss 3.259757 (-1.15z)| norm 0.3819 (+2.61z)| lr 4.62e-05 | 322.58 ms | 52.3% bf16 MFU | 1624291 tok/s
step 16186/19560 | loss 3.261718 (-1.09z)| norm 0.2875 (-0.22z)| lr 4.62e-05 | 322.86 ms | 52.3% bf16 MFU | 1624270 tok/s
step 16187/19560 | loss 3.332895 (+0.58z)| norm 0.2659 (-0.89z)| lr 4.61e-05 | 322.48 ms | 52.3% bf16 MFU | 1624348 tok/s
step 16188/19560 | loss 3.369525 (+1.44z)| norm 0.2793 (-0.46z)| lr 4.61e-05 | 322.52 ms | 52.3% bf16 MFU | 1624412 tok/s
step 16189/19560 | loss 3.329545 (+0.49z)| norm 0.3026 (+0.31z)| lr 4.61e-05 | 322.84 ms | 52.3% bf16 MFU | 1624389 tok/s
step 16190/19560 | loss 3.289575 (-0.45z)| norm 0.2790 (-0.47z)| lr 4.61e-05 | 322.24 ms | 52.4% bf16 MFU | 1624520 tok/s
step 16191/19560 | loss 3.220770 (-2.01z)| norm 0.2922 (-0.03z)| lr 4.60e-05 | 322.54 ms | 52.3% bf16 MFU | 1624570 tok/s
step 16192/19560 | loss 3.312626 (+0.12z)| norm 0.2714 (-0.70z)| lr 4.60e-05 | 322.65 ms | 52.3% bf16 MFU | 1624589 tok/s
step 16193/19560 | loss 3.205893 (-2.30z)| norm 0.3064 (+0.45z)| lr 4.60e-05 | 322.55 ms | 52.3% bf16 MFU | 1624633 tok/s
step 16194/19560 | loss 3.264461 (-0.96z)| norm 0.3022 (+0.30z)| lr 4.60e-05 | 322.87 ms | 52.3% bf16 MFU | 1624594 tok/s
step 16195/19560 | loss 3.320076 (+0.31z)| norm 0.2727 (-0.66z)| lr 4.59e-05 | 322.37 ms | 52.4% bf16 MFU | 1624683 tok/s
step 16196/19560 | loss 3.316575 (+0.23z)| norm 0.3126 (+0.66z)| lr 4.59e-05 | 322.84 ms | 52.3% bf16 MFU | 1624647 tok/s
step 16197/19560 | loss 3.307063 (+0.02z)| norm 0.2878 (-0.18z)| lr 4.59e-05 | 322.90 ms | 52.3% bf16 MFU | 1624599 tok/s
step 16198/19560 | loss 3.332960 (+0.59z)| norm 0.2915 (-0.03z)| lr 4.59e-05 | 322.87 ms | 52.3% bf16 MFU | 1624560 tok/s
step 16199/19560 | loss 3.299005 (-0.17z)| norm 0.2604 (-1.10z)| lr 4.58e-05 | 322.48 ms | 52.3% bf16 MFU | 1624622 tok/s
step 16200/19560 | loss 3.293189 (-0.30z)| norm 0.2931 (+0.04z)| lr 4.58e-05 | 322.65 ms | 52.3% bf16 MFU | 1624638 tok/s
step 16201/19560 | loss 3.293339 (-0.29z)| norm 0.2624 (-1.03z)| lr 4.58e-05 | 322.29 ms | 52.4% bf16 MFU | 1624743 tok/s
step 16202/19560 | loss 3.316230 (+0.24z)| norm 0.2626 (-1.02z)| lr 4.57e-05 | 322.71 ms | 52.3% bf16 MFU | 1624737 tok/s
step 16203/19560 | loss 3.429855 (+2.77z)| norm 0.3024 (+0.42z)| lr 4.57e-05 | 322.86 ms | 52.3% bf16 MFU | 1624695 tok/s
step 16204/19560 | loss 3.320213 (+0.31z)| norm 0.2818 (-0.32z)| lr 4.57e-05 | 322.76 ms | 52.3% bf16 MFU | 1624679 tok/s
step 16205/19560 | loss 3.277191 (-0.69z)| norm 0.2837 (-0.26z)| lr 4.57e-05 | 322.82 ms | 52.3% bf16 MFU | 1624648 tok/s
step 16206/19560 | loss 3.367312 (+1.38z)| norm 0.2805 (-0.38z)| lr 4.56e-05 | 322.22 ms | 52.4% bf16 MFU | 1624771 tok/s
step 16207/19560 | loss 3.317315 (+0.22z)| norm 0.3194 (+1.01z)| lr 4.56e-05 | 322.49 ms | 52.3% bf16 MFU | 1624819 tok/s
step 16208/19560 | loss 3.321697 (+0.32z)| norm 0.2788 (-0.46z)| lr 4.56e-05 | 322.56 ms | 52.3% bf16 MFU | 1624848 tok/s
step 16209/19560 | loss 3.344987 (+0.85z)| norm 0.2876 (-0.14z)| lr 4.56e-05 | 322.71 ms | 52.3% bf16 MFU | 1624838 tok/s
step 16210/19560 | loss 3.345674 (+0.86z)| norm 0.2808 (-0.38z)| lr 4.55e-05 | 322.77 ms | 52.3% bf16 MFU | 1624814 tok/s
step 16211/19560 | loss 3.321628 (+0.31z)| norm 0.2651 (-0.94z)| lr 4.55e-05 | 322.49 ms | 52.3% bf16 MFU | 1624862 tok/s
step 16212/19560 | loss 3.354471 (+1.06z)| norm 0.2546 (-1.31z)| lr 4.55e-05 | 322.63 ms | 52.3% bf16 MFU | 1624871 tok/s
step 16213/19560 | loss 3.323170 (+0.33z)| norm 0.2643 (-0.96z)| lr 4.55e-05 | 322.39 ms | 52.3% bf16 MFU | 1624939 tok/s
step 16214/19560 | loss 3.279946 (-0.67z)| norm 0.2978 (+0.26z)| lr 4.54e-05 | 322.39 ms | 52.4% bf16 MFU | 1625005 tok/s
step 16215/19560 | loss 3.302135 (-0.16z)| norm 0.2712 (-0.71z)| lr 4.54e-05 | 322.78 ms | 52.3% bf16 MFU | 1624970 tok/s
step 16216/19560 | loss 3.288766 (-0.49z)| norm 0.2754 (-0.55z)| lr 4.54e-05 | 322.84 ms | 52.3% bf16 MFU | 1624921 tok/s
step 16217/19560 | loss 3.293764 (-0.36z)| norm 0.2878 (-0.11z)| lr 4.54e-05 | 322.83 ms | 52.3% bf16 MFU | 1624878 tok/s
step 16218/19560 | loss 3.322199 (+0.34z)| norm 0.2825 (-0.30z)| lr 4.53e-05 | 322.68 ms | 52.3% bf16 MFU | 1624873 tok/s
step 16219/19560 | loss 3.251168 (-1.38z)| norm 0.2721 (-0.68z)| lr 4.53e-05 | 322.31 ms | 52.4% bf16 MFU | 1624961 tok/s
step 16220/19560 | loss 3.363352 (+1.32z)| norm 0.3006 (+0.35z)| lr 4.53e-05 | 323.01 ms | 52.2% bf16 MFU | 1624869 tok/s
step 16221/19560 | loss 3.270200 (-0.93z)| norm 0.2784 (-0.47z)| lr 4.52e-05 | 322.33 ms | 52.4% bf16 MFU | 1624954 tok/s
step 16222/19560 | loss 3.308388 (-0.01z)| norm 0.2691 (-0.81z)| lr 4.52e-05 | 322.87 ms | 52.3% bf16 MFU | 1624898 tok/s
step 16223/19560 | loss 3.284384 (-0.61z)| norm 0.2878 (-0.10z)| lr 4.52e-05 | 322.17 ms | 52.4% bf16 MFU | 1625021 tok/s
step 16224/19560 | loss 3.300249 (-0.23z)| norm 0.2583 (-1.27z)| lr 4.52e-05 | 322.65 ms | 52.3% bf16 MFU | 1625017 tok/s
step 16225/19560 | loss 3.286579 (-0.56z)| norm 0.3391 (+1.92z)| lr 4.51e-05 | 322.81 ms | 52.3% bf16 MFU | 1624972 tok/s
step 16226/19560 | loss 3.246285 (-1.53z)| norm 0.2529 (-1.46z)| lr 4.51e-05 | 322.78 ms | 52.3% bf16 MFU | 1624938 tok/s
step 16227/19560 | loss 3.379344 (+1.74z)| norm 0.3018 (+0.45z)| lr 4.51e-05 | 322.77 ms | 52.3% bf16 MFU | 1624908 tok/s
step 16228/19560 | loss 3.299297 (-0.23z)| norm 0.3464 (+2.13z)| lr 4.51e-05 | 322.52 ms | 52.3% bf16 MFU | 1624942 tok/s
step 16229/19560 | loss 3.295477 (-0.34z)| norm 0.2821 (-0.34z)| lr 4.50e-05 | 322.54 ms | 52.3% bf16 MFU | 1624970 tok/s
step 16230/19560 | loss 3.292104 (-0.41z)| norm 0.2638 (-1.03z)| lr 4.50e-05 | 322.71 ms | 52.3% bf16 MFU | 1624954 tok/s
step 16231/19560 | loss 3.310723 (+0.06z)| norm 0.2701 (-0.78z)| lr 4.50e-05 | 322.41 ms | 52.3% bf16 MFU | 1625013 tok/s
step 16232/19560 | loss 3.339365 (+0.77z)| norm 0.2802 (-0.39z)| lr 4.50e-05 | 322.65 ms | 52.3% bf16 MFU | 1625009 tok/s
step 16233/19560 | loss 3.376180 (+1.66z)| norm 0.3183 (+1.05z)| lr 4.49e-05 | 322.62 ms | 52.3% bf16 MFU | 1625014 tok/s
step 16234/19560 | loss 3.344656 (+0.87z)| norm 0.2620 (-1.08z)| lr 4.49e-05 | 322.48 ms | 52.3% bf16 MFU | 1625052 tok/s
step 16235/19560 | loss 3.301631 (-0.20z)| norm 0.2720 (-0.69z)| lr 4.49e-05 | 322.64 ms | 52.3% bf16 MFU | 1625050 tok/s
step 16236/19560 | loss 3.317511 (+0.20z)| norm 0.2760 (-0.53z)| lr 4.48e-05 | 322.70 ms | 52.3% bf16 MFU | 1625033 tok/s
step 16237/19560 | loss 3.227728 (-2.00z)| norm 0.3140 (+0.89z)| lr 4.48e-05 | 322.95 ms | 52.3% bf16 MFU | 1624953 tok/s
step 16238/19560 | loss 3.291444 (-0.41z)| norm 0.2900 (-0.02z)| lr 4.48e-05 | 322.38 ms | 52.4% bf16 MFU | 1625020 tok/s
step 16239/19560 | loss 3.336348 (+0.70z)| norm 0.2936 (+0.12z)| lr 4.48e-05 | 322.47 ms | 52.3% bf16 MFU | 1625062 tok/s
step 16240/19560 | loss 3.318377 (+0.25z)| norm 0.2880 (-0.10z)| lr 4.47e-05 | 322.69 ms | 52.3% bf16 MFU | 1625045 tok/s
step 16241/19560 | loss 3.245157 (-1.58z)| norm 0.2901 (-0.02z)| lr 4.47e-05 | 322.48 ms | 52.3% bf16 MFU | 1625083 tok/s
step 16242/19560 | loss 3.279016 (-0.72z)| norm 0.3156 (+0.93z)| lr 4.47e-05 | 322.59 ms | 52.3% bf16 MFU | 1625090 tok/s
step 16243/19560 | loss 3.337360 (+0.75z)| norm 0.2720 (-0.72z)| lr 4.47e-05 | 322.55 ms | 52.3% bf16 MFU | 1625108 tok/s
step 16244/19560 | loss 3.309698 (+0.05z)| norm 0.3174 (+1.00z)| lr 4.46e-05 | 322.69 ms | 52.3% bf16 MFU | 1625090 tok/s
step 16245/19560 | loss 3.329519 (+0.54z)| norm 0.2851 (-0.22z)| lr 4.46e-05 | 322.52 ms | 52.3% bf16 MFU | 1625115 tok/s
step 16246/19560 | loss 3.206055 (-2.50z)| norm 0.2798 (-0.41z)| lr 4.46e-05 | 322.91 ms | 52.3% bf16 MFU | 1625041 tok/s
step 16247/19560 | loss 3.290403 (-0.41z)| norm 0.2776 (-0.50z)| lr 4.46e-05 | 322.68 ms | 52.3% bf16 MFU | 1625029 tok/s
step 16248/19560 | loss 3.328204 (+0.51z)| norm 0.2813 (-0.34z)| lr 4.45e-05 | 322.49 ms | 52.3% bf16 MFU | 1625065 tok/s
step 16249/19560 | loss 3.241738 (-1.59z)| norm 0.2933 (+0.13z)| lr 4.45e-05 | 323.03 ms | 52.2% bf16 MFU | 1624964 tok/s
step 16250/19560 | loss 3.304708 (-0.05z)| norm 0.2661 (-0.93z)| lr 4.45e-05 | 322.34 ms | 52.4% bf16 MFU | 1625042 tok/s
val loss 3.288704
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag:evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3029/10042 = 0.301633
step 16251/19560 | loss 3.290507 (-0.39z)| norm 0.2924 (+0.10z)| lr 4.45e-05 | 322.58 ms | 52.3% bf16 MFU | 1625054 tok/s
step 16252/19560 | loss 3.350054 (+1.08z)| norm 0.2923 (+0.09z)| lr 4.44e-05 | 321.98 ms | 52.4% bf16 MFU | 1625218 tok/s
step 16253/19560 | loss 3.352694 (+1.13z)| norm 0.2944 (+0.17z)| lr 4.44e-05 | 322.54 ms | 52.3% bf16 MFU | 1625231 tok/s
step 16254/19560 | loss 3.262135 (-1.09z)| norm 0.3248 (+1.35z)| lr 4.44e-05 | 322.80 ms | 52.3% bf16 MFU | 1625179 tok/s
step 16255/19560 | loss 3.301381 (-0.12z)| norm 0.3266 (+1.40z)| lr 4.44e-05 | 323.15 ms | 52.2% bf16 MFU | 1625042 tok/s
step 16256/19560 | loss 3.263274 (-1.05z)| norm 0.2769 (-0.55z)| lr 4.43e-05 | 322.34 ms | 52.4% bf16 MFU | 1625116 tok/s
step 16257/19560 | loss 3.375211 (+1.69z)| norm 0.2609 (-1.18z)| lr 4.43e-05 | 323.02 ms | 52.2% bf16 MFU | 1625014 tok/s
step 16258/19560 | loss 3.300582 (-0.13z)| norm 0.2901 (-0.04z)| lr 4.43e-05 | 322.44 ms | 52.3% bf16 MFU | 1625062 tok/s
step 16259/19560 | loss 3.297259 (-0.21z)| norm 0.2828 (-0.33z)| lr 4.42e-05 | 322.52 ms | 52.3% bf16 MFU | 1625088 tok/s
step 16260/19560 | loss 3.325230 (+0.47z)| norm 0.3433 (+2.01z)| lr 4.42e-05 | 323.08 ms | 52.2% bf16 MFU | 1624972 tok/s
step 16261/19560 | loss 3.297262 (-0.21z)| norm 0.3001 (+0.31z)| lr 4.42e-05 | 322.94 ms | 52.3% bf16 MFU | 1624897 tok/s
step 16262/19560 | loss 3.332978 (+0.66z)| norm 0.3010 (+0.34z)| lr 4.42e-05 | 322.33 ms | 52.4% bf16 MFU | 1624979 tok/s
step 16263/19560 | loss 3.322524 (+0.40z)| norm 0.2693 (-0.90z)| lr 4.41e-05 | 322.47 ms | 52.3% bf16 MFU | 1625022 tok/s
step 16264/19560 | loss 3.257812 (-1.17z)| norm 0.2643 (-1.09z)| lr 4.41e-05 | 322.95 ms | 52.3% bf16 MFU | 1624944 tok/s
step 16265/19560 | loss 3.260560 (-1.10z)| norm 0.3082 (+0.65z)| lr 4.41e-05 | 322.74 ms | 52.3% bf16 MFU | 1624921 tok/s
step 16266/19560 | loss 3.340619 (+0.84z)| norm 0.2753 (-0.64z)| lr 4.41e-05 | 323.19 ms | 52.2% bf16 MFU | 1624787 tok/s
step 16267/19560 | loss 3.267001 (-0.95z)| norm 0.2956 (+0.18z)| lr 4.40e-05 | 322.87 ms | 52.3% bf16 MFU | 1624738 tok/s
step 16268/19560 | loss 3.362984 (+1.37z)| norm 0.2893 (-0.07z)| lr 4.40e-05 | 322.57 ms | 52.3% bf16 MFU | 1624768 tok/s
step 16269/19560 | loss 3.319015 (+0.31z)| norm 0.2902 (-0.03z)| lr 4.40e-05 | 322.91 ms | 52.3% bf16 MFU | 1624712 tok/s
step 16270/19560 | loss 3.268167 (-0.92z)| norm 0.2896 (-0.06z)| lr 4.40e-05 | 323.36 ms | 52.2% bf16 MFU | 1624546 tok/s
step 16271/19560 | loss 3.338973 (+0.78z)| norm 0.2484 (-1.70z)| lr 4.39e-05 | 322.53 ms | 52.3% bf16 MFU | 1624594 tok/s
step 16272/19560 | loss 3.320966 (+0.34z)| norm 0.3192 (+1.12z)| lr 4.39e-05 | 322.58 ms | 52.3% bf16 MFU | 1624630 tok/s
step 16273/19560 | loss 3.251131 (-1.36z)| norm 0.2613 (-1.18z)| lr 4.39e-05 | 322.68 ms | 52.3% bf16 MFU | 1624639 tok/s
step 16274/19560 | loss 3.264035 (-1.03z)| norm 0.2830 (-0.32z)| lr 4.39e-05 | 322.51 ms | 52.3% bf16 MFU | 1624689 tok/s
step 16275/19560 | loss 3.379508 (+1.77z)| norm 0.2655 (-1.01z)| lr 4.38e-05 | 323.00 ms | 52.3% bf16 MFU | 1624612 tok/s
step 16276/19560 | loss 3.287273 (-0.46z)| norm 0.2796 (-0.44z)| lr 4.38e-05 | 322.52 ms | 52.3% bf16 MFU | 1624663 tok/s
step 16277/19560 | loss 3.337662 (+0.76z)| norm 0.3058 (+0.58z)| lr 4.38e-05 | 322.71 ms | 52.3% bf16 MFU | 1624661 tok/s
step 16278/19560 | loss 3.416054 (+2.60z)| norm 0.3510 (+2.38z)| lr 4.38e-05 | 322.70 ms | 52.3% bf16 MFU | 1624663 tok/s
step 16279/19560 | loss 3.317865 (+0.26z)| norm 0.2913 (+0.01z)| lr 4.37e-05 | 323.03 ms | 52.2% bf16 MFU | 1624581 tok/s
step 16280/19560 | loss 3.380280 (+1.72z)| norm 0.2620 (-1.14z)| lr 4.37e-05 | 322.66 ms | 52.3% bf16 MFU | 1624598 tok/s
step 16281/19560 | loss 3.264099 (-1.02z)| norm 0.3191 (+1.12z)| lr 4.37e-05 | 322.79 ms | 52.3% bf16 MFU | 1624581 tok/s
step 16282/19560 | loss 3.469563 (+3.60z)| norm 0.3301 (+1.53z)| lr 4.36e-05 | 323.51 ms | 52.2% bf16 MFU | 1624384 tok/s
step 16283/19560 | loss 3.293607 (-0.34z)| norm 0.2845 (-0.27z)| lr 4.36e-05 | 322.61 ms | 52.3% bf16 MFU | 1624422 tok/s
step 16284/19560 | loss 3.270712 (-0.84z)| norm 0.2935 (+0.08z)| lr 4.36e-05 | 322.71 ms | 52.3% bf16 MFU | 1624432 tok/s
step 16285/19560 | loss 3.360177 (+1.14z)| norm 0.3273 (+1.39z)| lr 4.36e-05 | 323.22 ms | 52.2% bf16 MFU | 1624315 tok/s
step 16286/19560 | loss 3.300325 (-0.19z)| norm 0.3034 (+0.45z)| lr 4.35e-05 | 322.56 ms | 52.3% bf16 MFU | 1624368 tok/s
step 16287/19560 | loss 3.369973 (+1.33z)| norm 0.3317 (+1.54z)| lr 4.35e-05 | 322.99 ms | 52.3% bf16 MFU | 1624312 tok/s
step 16288/19560 | loss 3.306022 (-0.09z)| norm 0.3043 (+0.46z)| lr 4.35e-05 | 322.51 ms | 52.3% bf16 MFU | 1624378 tok/s
step 16289/19560 | loss 3.255936 (-1.20z)| norm 0.2765 (-0.62z)| lr 4.35e-05 | 323.16 ms | 52.2% bf16 MFU | 1624278 tok/s
step 16290/19560 | loss 3.255583 (-1.21z)| norm 0.3094 (+0.74z)| lr 4.34e-05 | 323.26 ms | 52.2% bf16 MFU | 1624159 tok/s
step 16291/19560 | loss 3.326446 (+0.40z)| norm 0.3880 (+3.74z)| lr 4.34e-05 | 322.47 ms | 52.3% bf16 MFU | 1624243 tok/s
step 16292/19560 | loss 3.345928 (+0.86z)| norm 0.2764 (-0.61z)| lr 4.34e-05 | 322.95 ms | 52.3% bf16 MFU | 1624202 tok/s
step 16293/19560 | loss 3.310531 (+0.04z)| norm 0.3395 (+1.92z)| lr 4.34e-05 | 322.49 ms | 52.3% bf16 MFU | 1624278 tok/s
step 16294/19560 | loss 3.249429 (-1.34z)| norm 0.3182 (+1.05z)| lr 4.33e-05 | 323.26 ms | 52.2% bf16 MFU | 1624157 tok/s
step 16295/19560 | loss 3.313797 (+0.12z)| norm 0.2747 (-0.69z)| lr 4.33e-05 | 323.01 ms | 52.2% bf16 MFU | 1624105 tok/s
step 16296/19560 | loss 3.324781 (+0.37z)| norm 0.3167 (+1.01z)| lr 4.33e-05 | 322.96 ms | 52.3% bf16 MFU | 1624070 tok/s
step 16297/19560 | loss 3.305362 (-0.07z)| norm 0.3556 (+2.50z)| lr 4.33e-05 | 323.07 ms | 52.2% bf16 MFU | 1624008 tok/s
step 16298/19560 | loss 3.325523 (+0.41z)| norm 0.2795 (-0.50z)| lr 4.32e-05 | 322.26 ms | 52.4% bf16 MFU | 1624153 tok/s
step 16299/19560 | loss 3.270936 (-0.85z)| norm 0.3326 (+1.59z)| lr 4.32e-05 | 323.02 ms | 52.2% bf16 MFU | 1624100 tok/s
step 16300/19560 | loss 3.254243 (-1.24z)| norm 0.2945 (+0.08z)| lr 4.32e-05 | 323.20 ms | 52.2% bf16 MFU | 1624003 tok/s
step 16301/19560 | loss 3.289152 (-0.43z)| norm 0.2500 (-1.64z)| lr 4.32e-05 | 322.74 ms | 52.3% bf16 MFU | 1624028 tok/s
step 16302/19560 | loss 3.304726 (-0.07z)| norm 0.2841 (-0.31z)| lr 4.31e-05 | 322.91 ms | 52.3% bf16 MFU | 1624008 tok/s
step 16303/19560 | loss 3.248821 (-1.36z)| norm 0.2895 (-0.09z)| lr 4.31e-05 | 324.03 ms | 52.1% bf16 MFU | 1623709 tok/s
step 16304/19560 | loss 3.316968 (+0.22z)| norm 0.2599 (-1.24z)| lr 4.31e-05 | 323.01 ms | 52.2% bf16 MFU | 1623680 tok/s
step 16305/19560 | loss 3.291976 (-0.37z)| norm 0.2562 (-1.37z)| lr 4.31e-05 | 322.61 ms | 52.3% bf16 MFU | 1623755 tok/s
step 16306/19560 | loss 3.182935 (-2.81z)| norm 0.2823 (-0.36z)| lr 4.30e-05 | 323.10 ms | 52.2% bf16 MFU | 1623701 tok/s
step 16307/19560 | loss 3.277169 (-0.67z)| norm 0.2526 (-1.49z)| lr 4.30e-05 | 322.78 ms | 52.3% bf16 MFU | 1623732 tok/s
step 16308/19560 | loss 3.322623 (+0.36z)| norm 0.2583 (-1.25z)| lr 4.30e-05 | 322.68 ms | 52.3% bf16 MFU | 1623784 tok/s
step 16309/19560 | loss 3.315260 (+0.20z)| norm 0.2688 (-0.85z)| lr 4.29e-05 | 323.83 ms | 52.1% bf16 MFU | 1623546 tok/s
step 16310/19560 | loss 3.301584 (-0.10z)| norm 0.2656 (-0.96z)| lr 4.29e-05 | 323.12 ms | 52.2% bf16 MFU | 1623498 tok/s
step 16311/19560 | loss 3.310711 (+0.11z)| norm 0.2745 (-0.61z)| lr 4.29e-05 | 322.89 ms | 52.3% bf16 MFU | 1623511 tok/s
step 16312/19560 | loss 3.370259 (+1.45z)| norm 0.2727 (-0.67z)| lr 4.29e-05 | 323.44 ms | 52.2% bf16 MFU | 1623385 tok/s
step 16313/19560 | loss 3.308918 (+0.04z)| norm 0.2705 (-0.76z)| lr 4.28e-05 | 323.16 ms | 52.2% bf16 MFU | 1623333 tok/s
step 16314/19560 | loss 3.290318 (-0.40z)| norm 0.2526 (-1.46z)| lr 4.28e-05 | 323.44 ms | 52.2% bf16 MFU | 1623216 tok/s
step 16315/19560 | loss 3.245969 (-1.40z)| norm 0.3010 (+0.48z)| lr 4.28e-05 | 323.25 ms | 52.2% bf16 MFU | 1623151 tok/s
step 16316/19560 | loss 3.306317 (-0.00z)| norm 0.2631 (-1.04z)| lr 4.28e-05 | 322.14 ms | 52.4% bf16 MFU | 1623369 tok/s
step 16317/19560 | loss 3.262173 (-1.01z)| norm 0.2820 (-0.28z)| lr 4.27e-05 | 322.84 ms | 52.3% bf16 MFU | 1623398 tok/s
step 16318/19560 | loss 3.307104 (+0.02z)| norm 0.2823 (-0.27z)| lr 4.27e-05 | 322.65 ms | 52.3% bf16 MFU | 1623474 tok/s
step 16319/19560 | loss 3.306061 (-0.01z)| norm 0.2638 (-1.00z)| lr 4.27e-05 | 323.17 ms | 52.2% bf16 MFU | 1623416 tok/s
step 16320/19560 | loss 3.343987 (+0.86z)| norm 0.2690 (-0.79z)| lr 4.27e-05 | 323.35 ms | 52.2% bf16 MFU | 1623318 tok/s
step 16321/19560 | loss 3.368940 (+1.44z)| norm 0.3230 (+1.36z)| lr 4.26e-05 | 322.30 ms | 52.4% bf16 MFU | 1623487 tok/s
step 16322/19560 | loss 3.300746 (-0.18z)| norm 0.2960 (+0.29z)| lr 4.26e-05 | 323.30 ms | 52.2% bf16 MFU | 1623395 tok/s
step 16323/19560 | loss 3.331840 (+0.55z)| norm 0.3280 (+1.54z)| lr 4.26e-05 | 322.97 ms | 52.3% bf16 MFU | 1623392 tok/s
step 16324/19560 | loss 3.335269 (+0.63z)| norm 0.2788 (-0.40z)| lr 4.26e-05 | 323.27 ms | 52.2% bf16 MFU | 1623313 tok/s
step 16325/19560 | loss 3.273798 (-0.82z)| norm 0.3588 (+2.68z)| lr 4.25e-05 | 323.09 ms | 52.2% bf16 MFU | 1623284 tok/s
step 16326/19560 | loss 3.308429 (+0.00z)| norm 0.3046 (+0.58z)| lr 4.25e-05 | 322.37 ms | 52.4% bf16 MFU | 1623438 tok/s
step 16327/19560 | loss 3.291100 (-0.41z)| norm 0.2530 (-1.40z)| lr 4.25e-05 | 323.19 ms | 52.2% bf16 MFU | 1623377 tok/s
step 16328/19560 | loss 3.309904 (+0.04z)| norm 0.3247 (+1.34z)| lr 4.25e-05 | 323.31 ms | 52.2% bf16 MFU | 1623288 tok/s
step 16329/19560 | loss 3.386476 (+1.81z)| norm 0.3745 (+3.09z)| lr 4.24e-05 | 323.19 ms | 52.2% bf16 MFU | 1623236 tok/s
step 16330/19560 | loss 3.288128 (-0.48z)| norm 0.2480 (-1.56z)| lr 4.24e-05 | 322.83 ms | 52.3% bf16 MFU | 1623276 tok/s
step 16331/19560 | loss 3.328356 (+0.49z)| norm 0.2653 (-0.91z)| lr 4.24e-05 | 322.52 ms | 52.3% bf16 MFU | 1623393 tok/s
step 16332/19560 | loss 3.431039 (+2.85z)| norm 0.3298 (+1.42z)| lr 4.24e-05 | 322.79 ms | 52.3% bf16 MFU | 1623436 tok/s
step 16333/19560 | loss 3.340601 (+0.73z)| norm 0.2943 (+0.13z)| lr 4.23e-05 | 322.77 ms | 52.3% bf16 MFU | 1623481 tok/s
step 16334/19560 | loss 3.276904 (-0.75z)| norm 0.3232 (+1.16z)| lr 4.23e-05 | 322.90 ms | 52.3% bf16 MFU | 1623492 tok/s
step 16335/19560 | loss 3.391956 (+1.91z)| norm 0.2813 (-0.34z)| lr 4.23e-05 | 322.60 ms | 52.3% bf16 MFU | 1623577 tok/s
step 16336/19560 | loss 3.318203 (+0.21z)| norm 0.2626 (-1.01z)| lr 4.23e-05 | 323.03 ms | 52.2% bf16 MFU | 1623549 tok/s
step 16337/19560 | loss 3.280746 (-0.65z)| norm 0.2717 (-0.68z)| lr 4.22e-05 | 323.14 ms | 52.2% bf16 MFU | 1623495 tok/s
step 16338/19560 | loss 3.339968 (+0.72z)| norm 0.2837 (-0.25z)| lr 4.22e-05 | 322.86 ms | 52.3% bf16 MFU | 1623514 tok/s
step 16339/19560 | loss 3.323088 (+0.33z)| norm 0.2706 (-0.72z)| lr 4.22e-05 | 323.55 ms | 52.2% bf16 MFU | 1623360 tok/s
step 16340/19560 | loss 3.290971 (-0.40z)| norm 0.2816 (-0.33z)| lr 4.22e-05 | 322.99 ms | 52.3% bf16 MFU | 1623353 tok/s
step 16341/19560 | loss 3.278877 (-0.67z)| norm 0.2779 (-0.47z)| lr 4.21e-05 | 322.85 ms | 52.3% bf16 MFU | 1623383 tok/s
step 16342/19560 | loss 3.349673 (+0.95z)| norm 0.2889 (-0.07z)| lr 4.21e-05 | 322.39 ms | 52.3% bf16 MFU | 1623526 tok/s
step 16343/19560 | loss 3.251643 (-1.30z)| norm 0.2761 (-0.54z)| lr 4.21e-05 | 322.90 ms | 52.3% bf16 MFU | 1623533 tok/s
step 16344/19560 | loss 3.198152 (-2.45z)| norm 0.2801 (-0.39z)| lr 4.21e-05 | 322.87 ms | 52.3% bf16 MFU | 1623548 tok/s
step 16345/19560 | loss 3.323901 (+0.37z)| norm 0.2665 (-0.88z)| lr 4.20e-05 | 322.65 ms | 52.3% bf16 MFU | 1623619 tok/s
step 16346/19560 | loss 3.254014 (-1.19z)| norm 0.2814 (-0.34z)| lr 4.20e-05 | 322.97 ms | 52.3% bf16 MFU | 1623604 tok/s
step 16347/19560 | loss 3.302856 (-0.10z)| norm 0.2953 (+0.16z)| lr 4.20e-05 | 322.50 ms | 52.3% bf16 MFU | 1623709 tok/s
step 16348/19560 | loss 3.280661 (-0.59z)| norm 0.2806 (-0.37z)| lr 4.20e-05 | 322.57 ms | 52.3% bf16 MFU | 1623790 tok/s
step 16349/19560 | loss 3.262933 (-0.99z)| norm 0.2651 (-0.93z)| lr 4.19e-05 | 323.03 ms | 52.2% bf16 MFU | 1623753 tok/s
step 16350/19560 | loss 3.252929 (-1.20z)| norm 0.2815 (-0.33z)| lr 4.19e-05 | 322.27 ms | 52.4% bf16 MFU | 1623908 tok/s
step 16351/19560 | loss 3.287290 (-0.43z)| norm 0.3449 (+1.93z)| lr 4.19e-05 | 322.42 ms | 52.3% bf16 MFU | 1624019 tok/s
step 16352/19560 | loss 3.317332 (+0.24z)| norm 0.2521 (-1.40z)| lr 4.18e-05 | 322.94 ms | 52.3% bf16 MFU | 1623991 tok/s
step 16353/19560 | loss 3.304743 (-0.04z)| norm 0.3268 (+1.29z)| lr 4.18e-05 | 323.24 ms | 52.2% bf16 MFU | 1623890 tok/s
step 16354/19560 | loss 3.316360 (+0.21z)| norm 0.3337 (+1.51z)| lr 4.18e-05 | 322.63 ms | 52.3% bf16 MFU | 1623947 tok/s
step 16355/19560 | loss 3.259843 (-1.06z)| norm 0.2839 (-0.27z)| lr 4.18e-05 | 322.65 ms | 52.3% bf16 MFU | 1623996 tok/s
step 16356/19560 | loss 3.270217 (-0.81z)| norm 0.2792 (-0.43z)| lr 4.17e-05 | 322.61 ms | 52.3% bf16 MFU | 1624054 tok/s
step 16357/19560 | loss 3.268197 (-0.85z)| norm 0.2897 (-0.05z)| lr 4.17e-05 | 322.52 ms | 52.3% bf16 MFU | 1624131 tok/s
step 16358/19560 | loss 3.306005 (+0.00z)| norm 0.2556 (-1.29z)| lr 4.17e-05 | 322.90 ms | 52.3% bf16 MFU | 1624109 tok/s
step 16359/19560 | loss 3.328443 (+0.51z)| norm 0.2989 (+0.28z)| lr 4.17e-05 | 322.64 ms | 52.3% bf16 MFU | 1624152 tok/s
step 16360/19560 | loss 3.305084 (-0.02z)| norm 0.2654 (-0.93z)| lr 4.16e-05 | 322.77 ms | 52.3% bf16 MFU | 1624160 tok/s
step 16361/19560 | loss 3.354999 (+1.13z)| norm 0.2925 (+0.06z)| lr 4.16e-05 | 322.67 ms | 52.3% bf16 MFU | 1624196 tok/s
step 16362/19560 | loss 3.353391 (+1.09z)| norm 0.3249 (+1.22z)| lr 4.16e-05 | 322.38 ms | 52.4% bf16 MFU | 1624302 tok/s
step 16363/19560 | loss 3.286504 (-0.43z)| norm 0.3292 (+1.36z)| lr 4.16e-05 | 322.46 ms | 52.3% bf16 MFU | 1624383 tok/s
step 16364/19560 | loss 3.310204 (+0.11z)| norm 0.2801 (-0.43z)| lr 4.15e-05 | 322.06 ms | 52.4% bf16 MFU | 1624560 tok/s
step 16365/19560 | loss 3.274872 (-0.71z)| norm 0.3092 (+0.63z)| lr 4.15e-05 | 322.81 ms | 52.3% bf16 MFU | 1624538 tok/s
step 16366/19560 | loss 3.269092 (-0.84z)| norm 0.2817 (-0.36z)| lr 4.15e-05 | 322.78 ms | 52.3% bf16 MFU | 1624525 tok/s
step 16367/19560 | loss 3.276804 (-0.65z)| norm 0.2732 (-0.67z)| lr 4.15e-05 | 322.35 ms | 52.4% bf16 MFU | 1624621 tok/s
step 16368/19560 | loss 3.366270 (+1.38z)| norm 0.2824 (-0.33z)| lr 4.14e-05 | 322.63 ms | 52.3% bf16 MFU | 1624643 tok/s
step 16369/19560 | loss 3.320399 (+0.33z)| norm 0.4041 (+3.81z)| lr 4.14e-05 | 322.87 ms | 52.3% bf16 MFU | 1624602 tok/s
step 16370/19560 | loss 3.304097 (-0.05z)| norm 0.2889 (-0.11z)| lr 4.14e-05 | 322.78 ms | 52.3% bf16 MFU | 1624587 tok/s
step 16371/19560 | loss 3.349198 (+0.98z)| norm 0.2839 (-0.29z)| lr 4.14e-05 | 322.50 ms | 52.3% bf16 MFU | 1624644 tok/s
step 16372/19560 | loss 3.332993 (+0.60z)| norm 0.3461 (+1.82z)| lr 4.13e-05 | 322.85 ms | 52.3% bf16 MFU | 1624608 tok/s
step 16373/19560 | loss 3.397851 (+2.05z)| norm 0.2780 (-0.49z)| lr 4.13e-05 | 322.49 ms | 52.3% bf16 MFU | 1624664 tok/s
step 16374/19560 | loss 3.316973 (+0.21z)| norm 0.2681 (-0.82z)| lr 4.13e-05 | 322.08 ms | 52.4% bf16 MFU | 1624822 tok/s
step 16375/19560 | loss 3.341741 (+0.77z)| norm 0.3144 (+0.73z)| lr 4.13e-05 | 322.36 ms | 52.4% bf16 MFU | 1624901 tok/s
step 16376/19560 | loss 3.287089 (-0.48z)| norm 0.2682 (-0.82z)| lr 4.12e-05 | 322.95 ms | 52.3% bf16 MFU | 1624829 tok/s
step 16377/19560 | loss 3.342067 (+0.77z)| norm 0.2838 (-0.29z)| lr 4.12e-05 | 322.84 ms | 52.3% bf16 MFU | 1624786 tok/s
step 16378/19560 | loss 3.403408 (+2.13z)| norm 0.3817 (+2.89z)| lr 4.12e-05 | 322.34 ms | 52.4% bf16 MFU | 1624872 tok/s
step 16379/19560 | loss 3.284225 (-0.58z)| norm 0.2622 (-1.01z)| lr 4.12e-05 | 322.49 ms | 52.3% bf16 MFU | 1624916 tok/s
step 16380/19560 | loss 3.259474 (-1.12z)| norm 0.2998 (+0.21z)| lr 4.11e-05 | 322.66 ms | 52.3% bf16 MFU | 1624915 tok/s
step 16381/19560 | loss 3.288079 (-0.46z)| norm 0.3322 (+1.25z)| lr 4.11e-05 | 322.16 ms | 52.4% bf16 MFU | 1625040 tok/s
step 16382/19560 | loss 3.340208 (+0.71z)| norm 0.3197 (+0.85z)| lr 4.11e-05 | 322.36 ms | 52.4% bf16 MFU | 1625109 tok/s
step 16383/19560 | loss 3.260351 (-1.10z)| norm 0.2702 (-0.74z)| lr 4.11e-05 | 323.23 ms | 52.2% bf16 MFU | 1624955 tok/s
step 16384/19560 | loss 3.313259 (+0.09z)| norm 0.2952 (+0.07z)| lr 4.10e-05 | 322.34 ms | 52.4% bf16 MFU | 1625031 tok/s
step 16385/19560 | loss 3.313835 (+0.12z)| norm 0.2823 (-0.36z)| lr 4.10e-05 | 322.51 ms | 52.3% bf16 MFU | 1625062 tok/s
step 16386/19560 | loss 3.320512 (+0.27z)| norm 0.2863 (-0.23z)| lr 4.10e-05 | 322.57 ms | 52.3% bf16 MFU | 1625075 tok/s
step 16387/19560 | loss 3.261185 (-1.08z)| norm 0.3691 (+2.39z)| lr 4.10e-05 | 322.06 ms | 52.4% bf16 MFU | 1625217 tok/s
step 16388/19560 | loss 3.287934 (-0.46z)| norm 0.2667 (-0.86z)| lr 4.09e-05 | 322.53 ms | 52.3% bf16 MFU | 1625235 tok/s
step 16389/19560 | loss 3.364692 (+1.27z)| norm 0.3583 (+2.03z)| lr 4.09e-05 | 323.28 ms | 52.2% bf16 MFU | 1625061 tok/s
step 16390/19560 | loss 3.225638 (-1.85z)| norm 0.2924 (-0.04z)| lr 4.09e-05 | 322.47 ms | 52.3% bf16 MFU | 1625100 tok/s
step 16391/19560 | loss 3.328859 (+0.47z)| norm 0.2596 (-1.08z)| lr 4.09e-05 | 323.16 ms | 52.2% bf16 MFU | 1624964 tok/s
step 16392/19560 | loss 3.284273 (-0.54z)| norm 0.3065 (+0.40z)| lr 4.08e-05 | 322.83 ms | 52.3% bf16 MFU | 1624916 tok/s
step 16393/19560 | loss 3.242080 (-1.48z)| norm 0.2937 (-0.01z)| lr 4.08e-05 | 322.25 ms | 52.4% bf16 MFU | 1625019 tok/s
step 16394/19560 | loss 3.219246 (-1.95z)| norm 0.2599 (-1.07z)| lr 4.08e-05 | 322.67 ms | 52.3% bf16 MFU | 1625010 tok/s
step 16395/19560 | loss 3.277968 (-0.65z)| norm 0.2795 (-0.45z)| lr 4.08e-05 | 322.44 ms | 52.3% bf16 MFU | 1625059 tok/s
step 16396/19560 | loss 3.275042 (-0.70z)| norm 0.3278 (+1.06z)| lr 4.07e-05 | 323.07 ms | 52.2% bf16 MFU | 1624947 tok/s
step 16397/19560 | loss 3.274691 (-0.70z)| norm 0.2783 (-0.49z)| lr 4.07e-05 | 322.62 ms | 52.3% bf16 MFU | 1624955 tok/s
step 16398/19560 | loss 3.295630 (-0.24z)| norm 0.2690 (-0.77z)| lr 4.07e-05 | 322.52 ms | 52.3% bf16 MFU | 1624988 tok/s
step 16399/19560 | loss 3.311182 (+0.11z)| norm 0.2688 (-0.79z)| lr 4.07e-05 | 322.94 ms | 52.3% bf16 MFU | 1624914 tok/s
step 16400/19560 | loss 3.242445 (-1.40z)| norm 0.2843 (-0.29z)| lr 4.06e-05 | 322.43 ms | 52.3% bf16 MFU | 1624970 tok/s
step 16401/19560 | loss 3.324846 (+0.42z)| norm 0.2767 (-0.54z)| lr 4.06e-05 | 322.55 ms | 52.3% bf16 MFU | 1624994 tok/s
step 16402/19560 | loss 3.316365 (+0.22z)| norm 0.3660 (+2.22z)| lr 4.06e-05 | 322.73 ms | 52.3% bf16 MFU | 1624973 tok/s
step 16403/19560 | loss 3.315795 (+0.22z)| norm 0.2794 (-0.47z)| lr 4.06e-05 | 322.89 ms | 52.3% bf16 MFU | 1624910 tok/s
step 16404/19560 | loss 3.326316 (+0.45z)| norm 0.2682 (-0.81z)| lr 4.05e-05 | 322.84 ms | 52.3% bf16 MFU | 1624863 tok/s
step 16405/19560 | loss 3.307618 (+0.04z)| norm 0.2824 (-0.37z)| lr 4.05e-05 | 322.83 ms | 52.3% bf16 MFU | 1624822 tok/s
step 16406/19560 | loss 3.392068 (+1.97z)| norm 0.2925 (-0.04z)| lr 4.05e-05 | 322.69 ms | 52.3% bf16 MFU | 1624818 tok/s
step 16407/19560 | loss 3.260496 (-1.02z)| norm 0.2858 (-0.25z)| lr 4.05e-05 | 322.63 ms | 52.3% bf16 MFU | 1624830 tok/s
step 16408/19560 | loss 3.364810 (+1.36z)| norm 0.3085 (+0.46z)| lr 4.04e-05 | 322.59 ms | 52.3% bf16 MFU | 1624851 tok/s
step 16409/19560 | loss 3.294367 (-0.26z)| norm 0.3013 (+0.23z)| lr 4.04e-05 | 322.60 ms | 52.3% bf16 MFU | 1624867 tok/s
step 16410/19560 | loss 3.308979 (+0.11z)| norm 0.2593 (-1.08z)| lr 4.04e-05 | 322.68 ms | 52.3% bf16 MFU | 1624864 tok/s
step 16411/19560 | loss 3.346279 (+1.01z)| norm 0.2735 (-0.62z)| lr 4.04e-05 | 322.62 ms | 52.3% bf16 MFU | 1624875 tok/s
step 16412/19560 | loss 3.390920 (+2.04z)| norm 0.3069 (+0.42z)| lr 4.03e-05 | 322.76 ms | 52.3% bf16 MFU | 1624852 tok/s
step 16413/19560 | loss 3.266790 (-0.92z)| norm 0.2601 (-1.03z)| lr 4.03e-05 | 322.66 ms | 52.3% bf16 MFU | 1624854 tok/s
step 16414/19560 | loss 3.322020 (+0.41z)| norm 0.2571 (-1.11z)| lr 4.03e-05 | 322.82 ms | 52.3% bf16 MFU | 1624814 tok/s
step 16415/19560 | loss 3.324240 (+0.47z)| norm 0.3413 (+1.52z)| lr 4.03e-05 | 322.77 ms | 52.3% bf16 MFU | 1624791 tok/s
step 16416/19560 | loss 3.316280 (+0.28z)| norm 0.3250 (+1.01z)| lr 4.02e-05 | 322.52 ms | 52.3% bf16 MFU | 1624832 tok/s
step 16417/19560 | loss 3.360333 (+1.32z)| norm 0.3058 (+0.40z)| lr 4.02e-05 | 322.41 ms | 52.3% bf16 MFU | 1624897 tok/s
step 16418/19560 | loss 3.267096 (-0.94z)| norm 0.3035 (+0.33z)| lr 4.02e-05 | 322.81 ms | 52.3% bf16 MFU | 1624860 tok/s
step 16419/19560 | loss 3.275811 (-0.71z)| norm 0.2799 (-0.39z)| lr 4.02e-05 | 322.53 ms | 52.3% bf16 MFU | 1624895 tok/s
step 16420/19560 | loss 3.222396 (-1.96z)| norm 0.2895 (-0.09z)| lr 4.01e-05 | 322.17 ms | 52.4% bf16 MFU | 1625018 tok/s
step 16421/19560 | loss 3.314935 (+0.25z)| norm 0.2784 (-0.43z)| lr 4.01e-05 | 322.84 ms | 52.3% bf16 MFU | 1624967 tok/s
step 16422/19560 | loss 3.271247 (-0.80z)| norm 0.2819 (-0.31z)| lr 4.01e-05 | 322.31 ms | 52.4% bf16 MFU | 1625051 tok/s
step 16423/19560 | loss 3.299649 (-0.12z)| norm 0.2766 (-0.49z)| lr 4.01e-05 | 322.73 ms | 52.3% bf16 MFU | 1625025 tok/s
step 16424/19560 | loss 3.306262 (+0.05z)| norm 0.2720 (-0.63z)| lr 4.00e-05 | 322.72 ms | 52.3% bf16 MFU | 1625002 tok/s
step 16425/19560 | loss 3.360282 (+1.33z)| norm 0.2755 (-0.50z)| lr 4.00e-05 | 322.45 ms | 52.3% bf16 MFU | 1625050 tok/s
step 16426/19560 | loss 3.317640 (+0.31z)| norm 0.2653 (-0.83z)| lr 4.00e-05 | 322.74 ms | 52.3% bf16 MFU | 1625020 tok/s
step 16427/19560 | loss 3.321185 (+0.39z)| norm 0.2766 (-0.45z)| lr 4.00e-05 | 322.84 ms | 52.3% bf16 MFU | 1624968 tok/s
step 16428/19560 | loss 3.282686 (-0.54z)| norm 0.2670 (-0.76z)| lr 3.99e-05 | 322.72 ms | 52.3% bf16 MFU | 1624950 tok/s
step 16429/19560 | loss 3.396554 (+2.14z)| norm 0.3194 (+0.98z)| lr 3.99e-05 | 322.36 ms | 52.4% bf16 MFU | 1625022 tok/s
step 16430/19560 | loss 3.196749 (-2.50z)| norm 0.2747 (-0.52z)| lr 3.99e-05 | 323.16 ms | 52.2% bf16 MFU | 1624890 tok/s
step 16431/19560 | loss 3.342948 (+0.85z)| norm 0.2716 (-0.62z)| lr 3.99e-05 | 322.84 ms | 52.3% bf16 MFU | 1624845 tok/s
step 16432/19560 | loss 3.281123 (-0.57z)| norm 0.2799 (-0.34z)| lr 3.98e-05 | 322.74 ms | 52.3% bf16 MFU | 1624827 tok/s
step 16433/19560 | loss 3.390692 (+1.92z)| norm 0.3191 (+0.95z)| lr 3.98e-05 | 322.90 ms | 52.3% bf16 MFU | 1624770 tok/s
step 16434/19560 | loss 3.327585 (+0.47z)| norm 0.2671 (-0.79z)| lr 3.98e-05 | 322.46 ms | 52.3% bf16 MFU | 1624827 tok/s
step 16435/19560 | loss 3.397954 (+2.07z)| norm 0.2768 (-0.47z)| lr 3.98e-05 | 322.35 ms | 52.4% bf16 MFU | 1624908 tok/s
step 16436/19560 | loss 3.275464 (-0.76z)| norm 0.3010 (+0.33z)| lr 3.97e-05 | 322.72 ms | 52.3% bf16 MFU | 1624892 tok/s
step 16437/19560 | loss 3.282324 (-0.59z)| norm 0.2755 (-0.53z)| lr 3.97e-05 | 322.64 ms | 52.3% bf16 MFU | 1624897 tok/s
step 16438/19560 | loss 3.280129 (-0.64z)| norm 0.2863 (-0.17z)| lr 3.97e-05 | 322.33 ms | 52.4% bf16 MFU | 1624979 tok/s
step 16439/19560 | loss 3.297043 (-0.24z)| norm 0.3089 (+0.59z)| lr 3.97e-05 | 323.02 ms | 52.2% bf16 MFU | 1624885 tok/s
step 16440/19560 | loss 3.293504 (-0.32z)| norm 0.2714 (-0.69z)| lr 3.96e-05 | 322.50 ms | 52.3% bf16 MFU | 1624925 tok/s
step 16441/19560 | loss 3.256910 (-1.15z)| norm 0.2669 (-0.84z)| lr 3.96e-05 | 322.96 ms | 52.3% bf16 MFU | 1624849 tok/s
step 16442/19560 | loss 3.332617 (+0.59z)| norm 0.2945 (+0.09z)| lr 3.96e-05 | 322.78 ms | 52.3% bf16 MFU | 1624821 tok/s
step 16443/19560 | loss 3.308899 (+0.03z)| norm 0.2820 (-0.33z)| lr 3.96e-05 | 322.53 ms | 52.3% bf16 MFU | 1624856 tok/s
step 16444/19560 | loss 3.268499 (-0.90z)| norm 0.2859 (-0.20z)| lr 3.95e-05 | 322.48 ms | 52.3% bf16 MFU | 1624902 tok/s
step 16445/19560 | loss 3.289650 (-0.41z)| norm 0.2723 (-0.67z)| lr 3.95e-05 | 322.35 ms | 52.4% bf16 MFU | 1624979 tok/s
step 16446/19560 | loss 3.325938 (+0.43z)| norm 0.2838 (-0.28z)| lr 3.95e-05 | 322.40 ms | 52.3% bf16 MFU | 1625040 tok/s
step 16447/19560 | loss 3.303209 (-0.10z)| norm 0.2664 (-0.87z)| lr 3.95e-05 | 322.97 ms | 52.3% bf16 MFU | 1624954 tok/s
step 16448/19560 | loss 3.264659 (-0.98z)| norm 0.3180 (+0.88z)| lr 3.94e-05 | 322.72 ms | 52.3% bf16 MFU | 1624936 tok/s
step 16449/19560 | loss 3.309604 (+0.07z)| norm 0.2963 (+0.15z)| lr 3.94e-05 | 322.94 ms | 52.3% bf16 MFU | 1624863 tok/s
step 16450/19560 | loss 3.321473 (+0.35z)| norm 0.2604 (-1.08z)| lr 3.94e-05 | 322.25 ms | 52.4% bf16 MFU | 1624967 tok/s
step 16451/19560 | loss 3.284626 (-0.51z)| norm 0.2522 (-1.33z)| lr 3.94e-05 | 322.56 ms | 52.3% bf16 MFU | 1624988 tok/s
step 16452/19560 | loss 3.270386 (-0.83z)| norm 0.2720 (-0.65z)| lr 3.93e-05 | 322.93 ms | 52.3% bf16 MFU | 1624916 tok/s
step 16453/19560 | loss 3.246535 (-1.37z)| norm 0.2769 (-0.47z)| lr 3.93e-05 | 322.97 ms | 52.3% bf16 MFU | 1624837 tok/s
step 16454/19560 | loss 3.299519 (-0.14z)| norm 0.2918 (+0.05z)| lr 3.93e-05 | 322.53 ms | 52.3% bf16 MFU | 1624872 tok/s
step 16455/19560 | loss 3.299080 (-0.15z)| norm 0.2639 (-0.93z)| lr 3.93e-05 | 322.82 ms | 52.3% bf16 MFU | 1624833 tok/s
step 16456/19560 | loss 3.330820 (+0.58z)| norm 0.2971 (+0.24z)| lr 3.92e-05 | 322.54 ms | 52.3% bf16 MFU | 1624868 tok/s
step 16457/19560 | loss 3.303781 (-0.03z)| norm 0.2571 (-1.17z)| lr 3.92e-05 | 323.33 ms | 52.2% bf16 MFU | 1624700 tok/s
step 16458/19560 | loss 3.274190 (-0.72z)| norm 0.2607 (-1.05z)| lr 3.92e-05 | 322.40 ms | 52.3% bf16 MFU | 1624776 tok/s
step 16459/19560 | loss 3.309096 (+0.10z)| norm 0.2691 (-0.75z)| lr 3.92e-05 | 322.69 ms | 52.3% bf16 MFU | 1624775 tok/s
step 16460/19560 | loss 3.342642 (+0.94z)| norm 0.2507 (-1.39z)| lr 3.91e-05 | 322.46 ms | 52.3% bf16 MFU | 1624832 tok/s
step 16461/19560 | loss 3.258626 (-1.09z)| norm 0.2508 (-1.37z)| lr 3.91e-05 | 322.71 ms | 52.3% bf16 MFU | 1624823 tok/s
step 16462/19560 | loss 3.292964 (-0.26z)| norm 0.2549 (-1.20z)| lr 3.91e-05 | 322.95 ms | 52.3% bf16 MFU | 1624754 tok/s
step 16463/19560 | loss 3.297112 (-0.14z)| norm 0.2590 (-1.04z)| lr 3.91e-05 | 322.61 ms | 52.3% bf16 MFU | 1624773 tok/s
step 16464/19560 | loss 3.370128 (+1.64z)| norm 0.2733 (-0.53z)| lr 3.90e-05 | 322.99 ms | 52.3% bf16 MFU | 1624697 tok/s
step 16465/19560 | loss 3.294873 (-0.21z)| norm 0.2661 (-0.79z)| lr 3.90e-05 | 322.70 ms | 52.3% bf16 MFU | 1624696 tok/s
step 16466/19560 | loss 3.332860 (+0.73z)| norm 0.2813 (-0.24z)| lr 3.90e-05 | 322.47 ms | 52.3% bf16 MFU | 1624754 tok/s
step 16467/19560 | loss 3.272244 (-0.75z)| norm 0.2691 (-0.68z)| lr 3.90e-05 | 322.89 ms | 52.3% bf16 MFU | 1624703 tok/s
step 16468/19560 | loss 3.311790 (+0.21z)| norm 0.2576 (-1.08z)| lr 3.89e-05 | 322.96 ms | 52.3% bf16 MFU | 1624638 tok/s
step 16469/19560 | loss 3.311468 (+0.20z)| norm 0.2560 (-1.13z)| lr 3.89e-05 | 322.56 ms | 52.3% bf16 MFU | 1624677 tok/s
step 16470/19560 | loss 3.329661 (+0.65z)| norm 0.3165 (+1.03z)| lr 3.89e-05 | 322.24 ms | 52.4% bf16 MFU | 1624794 tok/s
step 16471/19560 | loss 3.251581 (-1.27z)| norm 0.2852 (-0.09z)| lr 3.89e-05 | 323.20 ms | 52.2% bf16 MFU | 1624664 tok/s
step 16472/19560 | loss 3.340192 (+0.91z)| norm 0.2760 (-0.42z)| lr 3.88e-05 | 323.08 ms | 52.2% bf16 MFU | 1624568 tok/s
step 16473/19560 | loss 3.270460 (-0.84z)| norm 0.2541 (-1.19z)| lr 3.88e-05 | 322.74 ms | 52.3% bf16 MFU | 1624564 tok/s
step 16474/19560 | loss 3.318864 (+0.37z)| norm 0.2535 (-1.20z)| lr 3.88e-05 | 322.55 ms | 52.3% bf16 MFU | 1624608 tok/s
step 16475/19560 | loss 3.354285 (+1.25z)| norm 0.2496 (-1.32z)| lr 3.88e-05 | 322.41 ms | 52.3% bf16 MFU | 1624685 tok/s
step 16476/19560 | loss 3.325433 (+0.51z)| norm 0.2724 (-0.51z)| lr 3.87e-05 | 323.00 ms | 52.3% bf16 MFU | 1624609 tok/s
step 16477/19560 | loss 3.296438 (-0.23z)| norm 0.2575 (-1.03z)| lr 3.87e-05 | 322.82 ms | 52.3% bf16 MFU | 1624584 tok/s
step 16478/19560 | loss 3.331608 (+0.65z)| norm 0.2965 (+0.33z)| lr 3.87e-05 | 322.62 ms | 52.3% bf16 MFU | 1624609 tok/s
step 16479/19560 | loss 3.328317 (+0.56z)| norm 0.2697 (-0.60z)| lr 3.87e-05 | 322.64 ms | 52.3% bf16 MFU | 1624627 tok/s
step 16480/19560 | loss 3.288515 (-0.45z)| norm 0.3114 (+0.88z)| lr 3.86e-05 | 322.13 ms | 52.4% bf16 MFU | 1624775 tok/s
step 16481/19560 | loss 3.252731 (-1.34z)| norm 0.2700 (-0.59z)| lr 3.86e-05 | 322.91 ms | 52.3% bf16 MFU | 1624719 tok/s
step 16482/19560 | loss 3.286260 (-0.48z)| norm 0.2720 (-0.51z)| lr 3.86e-05 | 323.28 ms | 52.2% bf16 MFU | 1624571 tok/s
step 16483/19560 | loss 3.308676 (+0.07z)| norm 0.2932 (+0.26z)| lr 3.86e-05 | 323.02 ms | 52.2% bf16 MFU | 1624496 tok/s
step 16484/19560 | loss 3.271304 (-0.88z)| norm 0.3187 (+1.17z)| lr 3.86e-05 | 322.58 ms | 52.3% bf16 MFU | 1624535 tok/s
step 16485/19560 | loss 3.330052 (+0.61z)| norm 0.2632 (-0.83z)| lr 3.85e-05 | 322.63 ms | 52.3% bf16 MFU | 1624560 tok/s
step 16486/19560 | loss 3.228797 (-1.93z)| norm 0.2821 (-0.15z)| lr 3.85e-05 | 322.97 ms | 52.3% bf16 MFU | 1624498 tok/s
step 16487/19560 | loss 3.362646 (+1.41z)| norm 0.3129 (+0.96z)| lr 3.85e-05 | 322.72 ms | 52.3% bf16 MFU | 1624502 tok/s
step 16488/19560 | loss 3.294317 (-0.29z)| norm 0.2560 (-1.10z)| lr 3.85e-05 | 322.96 ms | 52.3% bf16 MFU | 1624446 tok/s
step 16489/19560 | loss 3.327762 (+0.55z)| norm 0.2909 (+0.16z)| lr 3.84e-05 | 322.76 ms | 52.3% bf16 MFU | 1624443 tok/s
step 16490/19560 | loss 3.266669 (-0.96z)| norm 0.2823 (-0.14z)| lr 3.84e-05 | 322.88 ms | 52.3% bf16 MFU | 1624409 tok/s
step 16491/19560 | loss 3.307388 (+0.06z)| norm 0.2510 (-1.26z)| lr 3.84e-05 | 323.04 ms | 52.2% bf16 MFU | 1624338 tok/s
step 16492/19560 | loss 3.280939 (-0.60z)| norm 0.2671 (-0.66z)| lr 3.84e-05 | 323.17 ms | 52.2% bf16 MFU | 1624239 tok/s
step 16493/19560 | loss 3.263504 (-1.03z)| norm 0.2531 (-1.15z)| lr 3.83e-05 | 323.35 ms | 52.2% bf16 MFU | 1624099 tok/s
step 16494/19560 | loss 3.311908 (+0.17z)| norm 0.2369 (-1.71z)| lr 3.83e-05 | 322.98 ms | 52.3% bf16 MFU | 1624059 tok/s
step 16495/19560 | loss 3.293109 (-0.31z)| norm 0.2797 (-0.18z)| lr 3.83e-05 | 322.82 ms | 52.3% bf16 MFU | 1624061 tok/s
step 16496/19560 | loss 3.256456 (-1.21z)| norm 0.2773 (-0.26z)| lr 3.83e-05 | 323.02 ms | 52.2% bf16 MFU | 1624012 tok/s
step 16497/19560 | loss 3.292933 (-0.29z)| norm 0.2609 (-0.87z)| lr 3.82e-05 | 322.78 ms | 52.3% bf16 MFU | 1624027 tok/s
step 16498/19560 | loss 3.364423 (+1.49z)| norm 0.2619 (-0.82z)| lr 3.82e-05 | 322.89 ms | 52.3% bf16 MFU | 1624011 tok/s
step 16499/19560 | loss 3.410125 (+2.57z)| norm 0.3765 (+3.41z)| lr 3.82e-05 | 322.88 ms | 52.3% bf16 MFU | 1624000 tok/s
step 16500/19560 | loss 3.267131 (-0.92z)| norm 0.3113 (+1.03z)| lr 3.82e-05 | 323.15 ms | 52.2% bf16 MFU | 1623921 tok/s
val loss 3.286502
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3027/10042 = 0.301434
step 16501/19560 | loss 3.353061 (+1.21z)| norm 0.2855 (+0.07z)| lr 3.81e-05 | 321.89 ms | 52.4% bf16 MFU | 1624163 tok/s
step 16502/19560 | loss 3.272374 (-0.78z)| norm 0.2941 (+0.38z)| lr 3.81e-05 | 322.46 ms | 52.3% bf16 MFU | 1624250 tok/s
step 16503/19560 | loss 3.303399 (-0.01z)| norm 0.3133 (+1.10z)| lr 3.81e-05 | 322.19 ms | 52.4% bf16 MFU | 1624400 tok/s
step 16504/19560 | loss 3.229595 (-1.81z)| norm 0.2539 (-1.12z)| lr 3.81e-05 | 323.01 ms | 52.3% bf16 MFU | 1624338 tok/s
step 16505/19560 | loss 3.315353 (+0.30z)| norm 0.3163 (+1.20z)| lr 3.80e-05 | 322.56 ms | 52.3% bf16 MFU | 1624391 tok/s
step 16506/19560 | loss 3.274426 (-0.69z)| norm 0.2924 (+0.35z)| lr 3.80e-05 | 322.45 ms | 52.3% bf16 MFU | 1624469 tok/s
step 16507/19560 | loss 3.313882 (+0.29z)| norm 0.3035 (+0.78z)| lr 3.80e-05 | 323.28 ms | 52.2% bf16 MFU | 1624336 tok/s
step 16508/19560 | loss 3.358336 (+1.39z)| norm 0.2958 (+0.48z)| lr 3.80e-05 | 322.77 ms | 52.3% bf16 MFU | 1624335 tok/s
step 16509/19560 | loss 3.305189 (+0.05z)| norm 0.2708 (-0.49z)| lr 3.79e-05 | 323.23 ms | 52.2% bf16 MFU | 1624219 tok/s
step 16510/19560 | loss 3.261764 (-1.02z)| norm 0.3169 (+1.34z)| lr 3.79e-05 | 322.51 ms | 52.3% bf16 MFU | 1624290 tok/s
step 16511/19560 | loss 3.218077 (-2.09z)| norm 0.2705 (-0.50z)| lr 3.79e-05 | 322.96 ms | 52.3% bf16 MFU | 1624245 tok/s
step 16512/19560 | loss 3.318540 (+0.40z)| norm 0.3011 (+0.71z)| lr 3.79e-05 | 323.02 ms | 52.2% bf16 MFU | 1624188 tok/s
step 16513/19560 | loss 3.266262 (-0.88z)| norm 0.2987 (+0.61z)| lr 3.78e-05 | 322.59 ms | 52.3% bf16 MFU | 1624241 tok/s
step 16514/19560 | loss 3.296078 (-0.14z)| norm 0.3008 (+0.69z)| lr 3.78e-05 | 323.06 ms | 52.2% bf16 MFU | 1624173 tok/s
step 16515/19560 | loss 3.296451 (-0.14z)| norm 0.3088 (+1.07z)| lr 3.78e-05 | 322.98 ms | 52.3% bf16 MFU | 1624128 tok/s
step 16516/19560 | loss 3.311029 (+0.22z)| norm 0.2730 (-0.42z)| lr 3.78e-05 | 322.49 ms | 52.3% bf16 MFU | 1624209 tok/s
step 16517/19560 | loss 3.256229 (-1.12z)| norm 0.2991 (+0.71z)| lr 3.77e-05 | 322.74 ms | 52.3% bf16 MFU | 1624222 tok/s
step 16518/19560 | loss 3.306861 (+0.13z)| norm 0.3240 (+1.75z)| lr 3.77e-05 | 322.79 ms | 52.3% bf16 MFU | 1624222 tok/s
step 16519/19560 | loss 3.294796 (-0.17z)| norm 0.2965 (+0.57z)| lr 3.77e-05 | 323.20 ms | 52.2% bf16 MFU | 1624120 tok/s
step 16520/19560 | loss 3.318474 (+0.42z)| norm 0.3265 (+1.83z)| lr 3.77e-05 | 322.63 ms | 52.3% bf16 MFU | 1624167 tok/s
step 16521/19560 | loss 3.280993 (-0.54z)| norm 0.2834 (+0.01z)| lr 3.76e-05 | 322.95 ms | 52.3% bf16 MFU | 1624131 tok/s
step 16522/19560 | loss 3.316628 (+0.36z)| norm 0.3271 (+1.82z)| lr 3.76e-05 | 322.72 ms | 52.3% bf16 MFU | 1624154 tok/s
step 16523/19560 | loss 3.309124 (+0.15z)| norm 0.3294 (+1.88z)| lr 3.76e-05 | 322.83 ms | 52.3% bf16 MFU | 1624150 tok/s
step 16524/19560 | loss 3.283027 (-0.53z)| norm 0.2637 (-0.83z)| lr 3.76e-05 | 323.23 ms | 52.2% bf16 MFU | 1624045 tok/s
step 16525/19560 | loss 3.326499 (+0.60z)| norm 0.2812 (-0.10z)| lr 3.76e-05 | 322.84 ms | 52.3% bf16 MFU | 1624041 tok/s
step 16526/19560 | loss 3.270221 (-0.87z)| norm 0.2696 (-0.58z)| lr 3.75e-05 | 322.83 ms | 52.3% bf16 MFU | 1624041 tok/s
step 16527/19560 | loss 3.256829 (-1.20z)| norm 0.3315 (+1.96z)| lr 3.75e-05 | 322.88 ms | 52.3% bf16 MFU | 1624029 tok/s
step 16528/19560 | loss 3.289734 (-0.36z)| norm 0.2644 (-0.81z)| lr 3.75e-05 | 324.50 ms | 52.0% bf16 MFU | 1623611 tok/s
step 16529/19560 | loss 3.301583 (-0.04z)| norm 0.2732 (-0.44z)| lr 3.75e-05 | 322.39 ms | 52.4% bf16 MFU | 1623743 tok/s
step 16530/19560 | loss 3.250686 (-1.35z)| norm 0.2420 (-1.74z)| lr 3.74e-05 | 322.76 ms | 52.3% bf16 MFU | 1623775 tok/s
step 16531/19560 | loss 3.220983 (-2.07z)| norm 0.2517 (-1.31z)| lr 3.74e-05 | 323.74 ms | 52.1% bf16 MFU | 1623560 tok/s
step 16532/19560 | loss 3.326568 (+0.63z)| norm 0.2878 (+0.21z)| lr 3.74e-05 | 322.75 ms | 52.3% bf16 MFU | 1623603 tok/s
step 16533/19560 | loss 3.250680 (-1.29z)| norm 0.2558 (-1.13z)| lr 3.74e-05 | 322.74 ms | 52.3% bf16 MFU | 1623647 tok/s
step 16534/19560 | loss 3.298177 (-0.07z)| norm 0.2713 (-0.47z)| lr 3.73e-05 | 322.93 ms | 52.3% bf16 MFU | 1623640 tok/s
step 16535/19560 | loss 3.303713 (+0.07z)| norm 0.2724 (-0.42z)| lr 3.73e-05 | 323.00 ms | 52.3% bf16 MFU | 1623618 tok/s
step 16536/19560 | loss 3.292826 (-0.20z)| norm 0.2630 (-0.80z)| lr 3.73e-05 | 322.70 ms | 52.3% bf16 MFU | 1623672 tok/s
step 16537/19560 | loss 3.314643 (+0.37z)| norm 0.3177 (+1.49z)| lr 3.73e-05 | 322.88 ms | 52.3% bf16 MFU | 1623676 tok/s
step 16538/19560 | loss 3.260071 (-1.06z)| norm 0.3013 (+0.79z)| lr 3.72e-05 | 322.97 ms | 52.3% bf16 MFU | 1623660 tok/s
step 16539/19560 | loss 3.246863 (-1.38z)| norm 0.2716 (-0.46z)| lr 3.72e-05 | 322.61 ms | 52.3% bf16 MFU | 1623734 tok/s
step 16540/19560 | loss 3.241768 (-1.50z)| norm 0.3266 (+1.82z)| lr 3.72e-05 | 322.72 ms | 52.3% bf16 MFU | 1623777 tok/s
step 16541/19560 | loss 3.353046 (+1.42z)| norm 0.2651 (-0.73z)| lr 3.72e-05 | 322.69 ms | 52.3% bf16 MFU | 1623825 tok/s
step 16542/19560 | loss 3.285610 (-0.35z)| norm 0.2566 (-1.09z)| lr 3.71e-05 | 322.85 ms | 52.3% bf16 MFU | 1623831 tok/s
step 16543/19560 | loss 3.373628 (+1.94z)| norm 0.3368 (+2.26z)| lr 3.71e-05 | 322.68 ms | 52.3% bf16 MFU | 1623879 tok/s
step 16544/19560 | loss 3.249356 (-1.28z)| norm 0.3138 (+1.31z)| lr 3.71e-05 | 322.59 ms | 52.3% bf16 MFU | 1623948 tok/s
step 16545/19560 | loss 3.315064 (+0.44z)| norm 0.2845 (+0.09z)| lr 3.71e-05 | 322.87 ms | 52.3% bf16 MFU | 1623942 tok/s
step 16546/19560 | loss 3.268363 (-0.78z)| norm 0.4091 (+4.81z)| lr 3.70e-05 | 322.74 ms | 52.3% bf16 MFU | 1623969 tok/s
step 16547/19560 | loss 3.269025 (-0.76z)| norm 0.2707 (-0.48z)| lr 3.70e-05 | 322.97 ms | 52.3% bf16 MFU | 1623937 tok/s
step 16548/19560 | loss 3.202594 (-2.48z)| norm 0.3339 (+1.90z)| lr 3.70e-05 | 322.68 ms | 52.3% bf16 MFU | 1623979 tok/s
step 16549/19560 | loss 3.365153 (+1.71z)| norm 0.3526 (+2.52z)| lr 3.70e-05 | 323.00 ms | 52.3% bf16 MFU | 1623940 tok/s
step 16550/19560 | loss 3.316312 (+0.45z)| norm 0.2731 (-0.40z)| lr 3.69e-05 | 322.96 ms | 52.3% bf16 MFU | 1623911 tok/s
step 16551/19560 | loss 3.291386 (-0.19z)| norm 0.3458 (+2.21z)| lr 3.69e-05 | 322.89 ms | 52.3% bf16 MFU | 1623902 tok/s
step 16552/19560 | loss 3.264408 (-0.87z)| norm 0.3101 (+0.91z)| lr 3.69e-05 | 323.16 ms | 52.2% bf16 MFU | 1623826 tok/s
step 16553/19560 | loss 3.277920 (-0.51z)| norm 0.2805 (-0.16z)| lr 3.69e-05 | 323.09 ms | 52.2% bf16 MFU | 1623771 tok/s
step 16554/19560 | loss 3.293404 (-0.11z)| norm 0.3375 (+1.85z)| lr 3.69e-05 | 322.37 ms | 52.4% bf16 MFU | 1623900 tok/s
step 16555/19560 | loss 3.311908 (+0.37z)| norm 0.2886 (+0.11z)| lr 3.68e-05 | 322.47 ms | 52.3% bf16 MFU | 1623999 tok/s
step 16556/19560 | loss 3.301979 (+0.11z)| norm 0.2620 (-0.83z)| lr 3.68e-05 | 322.95 ms | 52.3% bf16 MFU | 1623969 tok/s
step 16557/19560 | loss 3.316792 (+0.52z)| norm 0.2756 (-0.34z)| lr 3.68e-05 | 322.92 ms | 52.3% bf16 MFU | 1623949 tok/s
step 16558/19560 | loss 3.317091 (+0.52z)| norm 0.3376 (+1.83z)| lr 3.68e-05 | 322.67 ms | 52.3% bf16 MFU | 1623994 tok/s
step 16559/19560 | loss 3.246904 (-1.37z)| norm 0.2691 (-0.58z)| lr 3.67e-05 | 322.65 ms | 52.3% bf16 MFU | 1624041 tok/s
step 16560/19560 | loss 3.297428 (+0.00z)| norm 0.3467 (+2.09z)| lr 3.67e-05 | 322.81 ms | 52.3% bf16 MFU | 1624046 tok/s
step 16561/19560 | loss 3.256197 (-1.12z)| norm 0.2848 (-0.04z)| lr 3.67e-05 | 321.82 ms | 52.4% bf16 MFU | 1624301 tok/s
step 16562/19560 | loss 3.356767 (+1.66z)| norm 0.2702 (-0.55z)| lr 3.67e-05 | 322.72 ms | 52.3% bf16 MFU | 1624316 tok/s
step 16563/19560 | loss 3.273039 (-0.64z)| norm 0.2703 (-0.54z)| lr 3.66e-05 | 322.56 ms | 52.3% bf16 MFU | 1624370 tok/s
step 16564/19560 | loss 3.323797 (+0.79z)| norm 0.3235 (+1.30z)| lr 3.66e-05 | 322.62 ms | 52.3% bf16 MFU | 1624405 tok/s
step 16565/19560 | loss 3.220138 (-2.11z)| norm 0.2767 (-0.32z)| lr 3.66e-05 | 322.75 ms | 52.3% bf16 MFU | 1624406 tok/s
step 16566/19560 | loss 3.308892 (+0.37z)| norm 0.3075 (+0.73z)| lr 3.66e-05 | 322.58 ms | 52.3% bf16 MFU | 1624450 tok/s
step 16567/19560 | loss 3.319657 (+0.66z)| norm 0.3038 (+0.61z)| lr 3.65e-05 | 322.69 ms | 52.3% bf16 MFU | 1624465 tok/s
step 16568/19560 | loss 3.334323 (+1.06z)| norm 0.2562 (-1.03z)| lr 3.65e-05 | 322.27 ms | 52.4% bf16 MFU | 1624583 tok/s
step 16569/19560 | loss 3.259446 (-1.02z)| norm 0.2770 (-0.31z)| lr 3.65e-05 | 322.74 ms | 52.3% bf16 MFU | 1624578 tok/s
step 16570/19560 | loss 3.321398 (+0.70z)| norm 0.2696 (-0.56z)| lr 3.65e-05 | 322.79 ms | 52.3% bf16 MFU | 1624562 tok/s
step 16571/19560 | loss 3.309790 (+0.38z)| norm 0.2840 (-0.07z)| lr 3.64e-05 | 322.99 ms | 52.3% bf16 MFU | 1624496 tok/s
step 16572/19560 | loss 3.310837 (+0.40z)| norm 0.2499 (-1.23z)| lr 3.64e-05 | 322.77 ms | 52.3% bf16 MFU | 1624489 tok/s
step 16573/19560 | loss 3.346661 (+1.38z)| norm 0.2662 (-0.67z)| lr 3.64e-05 | 322.81 ms | 52.3% bf16 MFU | 1624470 tok/s
step 16574/19560 | loss 3.312372 (+0.43z)| norm 0.2635 (-0.75z)| lr 3.64e-05 | 322.69 ms | 52.3% bf16 MFU | 1624485 tok/s
step 16575/19560 | loss 3.343003 (+1.26z)| norm 0.2571 (-0.96z)| lr 3.64e-05 | 322.56 ms | 52.3% bf16 MFU | 1624531 tok/s
step 16576/19560 | loss 3.277262 (-0.55z)| norm 0.2550 (-1.02z)| lr 3.63e-05 | 322.69 ms | 52.3% bf16 MFU | 1624541 tok/s
step 16577/19560 | loss 3.306430 (+0.25z)| norm 0.2638 (-0.71z)| lr 3.63e-05 | 322.78 ms | 52.3% bf16 MFU | 1624528 tok/s
step 16578/19560 | loss 3.308053 (+0.30z)| norm 0.2751 (-0.33z)| lr 3.63e-05 | 323.05 ms | 52.2% bf16 MFU | 1624448 tok/s
step 16579/19560 | loss 3.292293 (-0.14z)| norm 0.2607 (-0.82z)| lr 3.63e-05 | 322.68 ms | 52.3% bf16 MFU | 1624464 tok/s
step 16580/19560 | loss 3.243430 (-1.47z)| norm 0.2729 (-0.41z)| lr 3.62e-05 | 322.51 ms | 52.3% bf16 MFU | 1624523 tok/s
step 16581/19560 | loss 3.287862 (-0.26z)| norm 0.2701 (-0.50z)| lr 3.62e-05 | 323.17 ms | 52.2% bf16 MFU | 1624412 tok/s
step 16582/19560 | loss 3.309161 (+0.33z)| norm 0.2702 (-0.49z)| lr 3.62e-05 | 322.67 ms | 52.3% bf16 MFU | 1624434 tok/s
step 16583/19560 | loss 3.361148 (+1.73z)| norm 0.2642 (-0.70z)| lr 3.62e-05 | 322.60 ms | 52.3% bf16 MFU | 1624471 tok/s
step 16584/19560 | loss 3.261962 (-0.97z)| norm 0.2864 (+0.06z)| lr 3.61e-05 | 322.89 ms | 52.3% bf16 MFU | 1624433 tok/s
step 16585/19560 | loss 3.280417 (-0.46z)| norm 0.2716 (-0.45z)| lr 3.61e-05 | 322.47 ms | 52.3% bf16 MFU | 1624504 tok/s
step 16586/19560 | loss 3.335760 (+1.04z)| norm 0.2596 (-0.86z)| lr 3.61e-05 | 322.59 ms | 52.3% bf16 MFU | 1624541 tok/s
step 16587/19560 | loss 3.282659 (-0.40z)| norm 0.2604 (-0.83z)| lr 3.61e-05 | 322.70 ms | 52.3% bf16 MFU | 1624549 tok/s
step 16588/19560 | loss 3.312972 (+0.43z)| norm 0.3080 (+0.79z)| lr 3.60e-05 | 322.67 ms | 52.3% bf16 MFU | 1624563 tok/s
step 16589/19560 | loss 3.285599 (-0.32z)| norm 0.2482 (-1.26z)| lr 3.60e-05 | 321.91 ms | 52.4% bf16 MFU | 1624769 tok/s
step 16590/19560 | loss 3.337682 (+1.09z)| norm 0.2938 (+0.29z)| lr 3.60e-05 | 322.18 ms | 52.4% bf16 MFU | 1624897 tok/s
step 16591/19560 | loss 3.251412 (-1.25z)| norm 0.3243 (+1.32z)| lr 3.60e-05 | 323.09 ms | 52.2% bf16 MFU | 1624788 tok/s
step 16592/19560 | loss 3.264926 (-0.87z)| norm 0.2617 (-0.82z)| lr 3.59e-05 | 322.57 ms | 52.3% bf16 MFU | 1624817 tok/s
step 16593/19560 | loss 3.276203 (-0.56z)| norm 0.3498 (+2.13z)| lr 3.59e-05 | 322.29 ms | 52.4% bf16 MFU | 1624914 tok/s
step 16594/19560 | loss 3.289661 (-0.18z)| norm 0.2831 (-0.11z)| lr 3.59e-05 | 322.79 ms | 52.3% bf16 MFU | 1624882 tok/s
step 16595/19560 | loss 3.354294 (+1.58z)| norm 0.2901 (+0.12z)| lr 3.59e-05 | 322.85 ms | 52.3% bf16 MFU | 1624835 tok/s
step 16596/19560 | loss 3.329592 (+0.89z)| norm 0.3869 (+3.22z)| lr 3.59e-05 | 323.13 ms | 52.2% bf16 MFU | 1624720 tok/s
step 16597/19560 | loss 3.259903 (-1.00z)| norm 0.2764 (-0.37z)| lr 3.58e-05 | 322.29 ms | 52.4% bf16 MFU | 1624821 tok/s
step 16598/19560 | loss 3.273523 (-0.61z)| norm 0.2758 (-0.38z)| lr 3.58e-05 | 322.82 ms | 52.3% bf16 MFU | 1624785 tok/s
step 16599/19560 | loss 3.338894 (+1.15z)| norm 0.3501 (+2.00z)| lr 3.58e-05 | 323.52 ms | 52.2% bf16 MFU | 1624573 tok/s
step 16600/19560 | loss 3.306998 (+0.29z)| norm 0.3154 (+0.87z)| lr 3.58e-05 | 322.45 ms | 52.3% bf16 MFU | 1624642 tok/s
step 16601/19560 | loss 3.319684 (+0.63z)| norm 0.2714 (-0.54z)| lr 3.57e-05 | 322.95 ms | 52.3% bf16 MFU | 1624582 tok/s
step 16602/19560 | loss 3.311470 (+0.40z)| norm 0.3461 (+1.82z)| lr 3.57e-05 | 322.95 ms | 52.3% bf16 MFU | 1624525 tok/s
step 16603/19560 | loss 3.306660 (+0.28z)| norm 0.2916 (+0.07z)| lr 3.57e-05 | 322.70 ms | 52.3% bf16 MFU | 1624534 tok/s
step 16604/19560 | loss 3.309899 (+0.38z)| norm 0.2882 (-0.04z)| lr 3.57e-05 | 322.87 ms | 52.3% bf16 MFU | 1624498 tok/s
step 16605/19560 | loss 3.256585 (-1.09z)| norm 0.3326 (+1.36z)| lr 3.56e-05 | 322.72 ms | 52.3% bf16 MFU | 1624502 tok/s
step 16606/19560 | loss 3.282332 (-0.37z)| norm 0.2964 (+0.20z)| lr 3.56e-05 | 322.51 ms | 52.3% bf16 MFU | 1624558 tok/s
step 16607/19560 | loss 3.348253 (+1.45z)| norm 0.3025 (+0.39z)| lr 3.56e-05 | 322.69 ms | 52.3% bf16 MFU | 1624567 tok/s
step 16608/19560 | loss 3.354650 (+1.60z)| norm 0.2966 (+0.20z)| lr 3.56e-05 | 322.65 ms | 52.3% bf16 MFU | 1624586 tok/s
step 16609/19560 | loss 3.288611 (-0.22z)| norm 0.2851 (-0.17z)| lr 3.55e-05 | 322.82 ms | 52.3% bf16 MFU | 1624562 tok/s
step 16610/19560 | loss 3.310118 (+0.37z)| norm 0.3178 (+0.87z)| lr 3.55e-05 | 322.75 ms | 52.3% bf16 MFU | 1624557 tok/s
step 16611/19560 | loss 3.306739 (+0.28z)| norm 0.2724 (-0.58z)| lr 3.55e-05 | 323.12 ms | 52.2% bf16 MFU | 1624457 tok/s
step 16612/19560 | loss 3.270181 (-0.73z)| norm 0.2642 (-0.83z)| lr 3.55e-05 | 322.63 ms | 52.3% bf16 MFU | 1624486 tok/s
step 16613/19560 | loss 3.300500 (+0.11z)| norm 0.2848 (-0.18z)| lr 3.55e-05 | 322.37 ms | 52.4% bf16 MFU | 1624579 tok/s
step 16614/19560 | loss 3.394133 (+2.62z)| norm 0.2734 (-0.54z)| lr 3.54e-05 | 322.96 ms | 52.3% bf16 MFU | 1624519 tok/s
step 16615/19560 | loss 3.294441 (-0.08z)| norm 0.2615 (-0.91z)| lr 3.54e-05 | 323.30 ms | 52.2% bf16 MFU | 1624378 tok/s
step 16616/19560 | loss 3.301679 (+0.12z)| norm 0.2564 (-1.07z)| lr 3.54e-05 | 322.31 ms | 52.4% bf16 MFU | 1624492 tok/s
step 16617/19560 | loss 3.272069 (-0.68z)| norm 0.2530 (-1.17z)| lr 3.54e-05 | 322.94 ms | 52.3% bf16 MFU | 1624441 tok/s
step 16618/19560 | loss 3.273871 (-0.64z)| norm 0.2750 (-0.46z)| lr 3.53e-05 | 322.53 ms | 52.3% bf16 MFU | 1624496 tok/s
step 16619/19560 | loss 3.259847 (-1.01z)| norm 0.2812 (-0.27z)| lr 3.53e-05 | 322.55 ms | 52.3% bf16 MFU | 1624544 tok/s
step 16620/19560 | loss 3.375793 (+2.12z)| norm 0.3157 (+0.82z)| lr 3.53e-05 | 323.09 ms | 52.2% bf16 MFU | 1624454 tok/s
step 16621/19560 | loss 3.315938 (+0.49z)| norm 0.2894 (-0.03z)| lr 3.53e-05 | 322.43 ms | 52.3% bf16 MFU | 1624535 tok/s
step 16622/19560 | loss 3.249550 (-1.28z)| norm 0.2507 (-1.29z)| lr 3.52e-05 | 323.09 ms | 52.2% bf16 MFU | 1624446 tok/s
step 16623/19560 | loss 3.267504 (-0.79z)| norm 0.2629 (-0.89z)| lr 3.52e-05 | 322.97 ms | 52.3% bf16 MFU | 1624390 tok/s
step 16624/19560 | loss 3.291410 (-0.16z)| norm 0.3185 (+0.90z)| lr 3.52e-05 | 322.54 ms | 52.3% bf16 MFU | 1624444 tok/s
step 16625/19560 | loss 3.306515 (+0.25z)| norm 0.2537 (-1.19z)| lr 3.52e-05 | 322.68 ms | 52.3% bf16 MFU | 1624462 tok/s
step 16626/19560 | loss 3.392040 (+2.52z)| norm 0.2798 (-0.35z)| lr 3.51e-05 | 322.74 ms | 52.3% bf16 MFU | 1624462 tok/s
step 16627/19560 | loss 3.302546 (+0.16z)| norm 0.2728 (-0.57z)| lr 3.51e-05 | 323.51 ms | 52.2% bf16 MFU | 1624269 tok/s
step 16628/19560 | loss 3.273297 (-0.65z)| norm 0.2840 (-0.19z)| lr 3.51e-05 | 322.48 ms | 52.3% bf16 MFU | 1624346 tok/s
step 16629/19560 | loss 3.355146 (+1.61z)| norm 0.2906 (+0.03z)| lr 3.51e-05 | 322.39 ms | 52.4% bf16 MFU | 1624441 tok/s
step 16630/19560 | loss 3.308299 (+0.31z)| norm 0.3383 (+1.59z)| lr 3.51e-05 | 323.10 ms | 52.2% bf16 MFU | 1624354 tok/s
step 16631/19560 | loss 3.354973 (+1.58z)| norm 0.3050 (+0.49z)| lr 3.50e-05 | 322.56 ms | 52.3% bf16 MFU | 1624406 tok/s
step 16632/19560 | loss 3.307720 (+0.27z)| norm 0.2543 (-1.18z)| lr 3.50e-05 | 322.63 ms | 52.3% bf16 MFU | 1624437 tok/s
step 16633/19560 | loss 3.251255 (-1.28z)| norm 0.2737 (-0.53z)| lr 3.50e-05 | 322.64 ms | 52.3% bf16 MFU | 1624465 tok/s
step 16634/19560 | loss 3.331100 (+0.91z)| norm 0.2937 (+0.13z)| lr 3.50e-05 | 323.76 ms | 52.1% bf16 MFU | 1624211 tok/s
step 16635/19560 | loss 3.308674 (+0.29z)| norm 0.2676 (-0.72z)| lr 3.49e-05 | 322.80 ms | 52.3% bf16 MFU | 1624211 tok/s
step 16636/19560 | loss 3.237132 (-1.65z)| norm 0.3027 (+0.44z)| lr 3.49e-05 | 323.03 ms | 52.2% bf16 MFU | 1624152 tok/s
step 16637/19560 | loss 3.247070 (-1.36z)| norm 0.2687 (-0.69z)| lr 3.49e-05 | 323.12 ms | 52.2% bf16 MFU | 1624072 tok/s
step 16638/19560 | loss 3.285245 (-0.32z)| norm 0.2570 (-1.06z)| lr 3.49e-05 | 322.73 ms | 52.3% bf16 MFU | 1624096 tok/s
step 16639/19560 | loss 3.268892 (-0.79z)| norm 0.2433 (-1.49z)| lr 3.48e-05 | 322.90 ms | 52.3% bf16 MFU | 1624074 tok/s
step 16640/19560 | loss 3.262554 (-0.95z)| norm 0.2899 (+0.04z)| lr 3.48e-05 | 323.09 ms | 52.2% bf16 MFU | 1624008 tok/s
step 16641/19560 | loss 3.295419 (-0.04z)| norm 0.2874 (-0.04z)| lr 3.48e-05 | 322.25 ms | 52.4% bf16 MFU | 1624157 tok/s
step 16642/19560 | loss 3.267302 (-0.82z)| norm 0.2637 (-0.81z)| lr 3.48e-05 | 322.65 ms | 52.3% bf16 MFU | 1624195 tok/s
step 16643/19560 | loss 3.307353 (+0.29z)| norm 0.2592 (-0.94z)| lr 3.47e-05 | 322.88 ms | 52.3% bf16 MFU | 1624174 tok/s
step 16644/19560 | loss 3.510494 (+5.24z)| norm 0.2808 (-0.24z)| lr 3.47e-05 | 322.78 ms | 52.3% bf16 MFU | 1624180 tok/s
step 16645/19560 | loss 3.295579 (-0.08z)| norm 0.2781 (-0.32z)| lr 3.47e-05 | 322.81 ms | 52.3% bf16 MFU | 1624178 tok/s
step 16646/19560 | loss 3.627929 (+6.58z)| norm 0.3651 (+2.47z)| lr 3.47e-05 | 322.93 ms | 52.3% bf16 MFU | 1624147 tok/s
step 16647/19560 | loss 3.264340 (-0.74z)| norm 0.3140 (+0.82z)| lr 3.47e-05 | 322.87 ms | 52.3% bf16 MFU | 1624131 tok/s
step 16648/19560 | loss 3.238213 (-1.24z)| norm 0.2874 (-0.02z)| lr 3.46e-05 | 323.68 ms | 52.1% bf16 MFU | 1623913 tok/s
step 16649/19560 | loss 3.325856 (+0.50z)| norm 0.2690 (-0.61z)| lr 3.46e-05 | 322.57 ms | 52.3% bf16 MFU | 1623984 tok/s
step 16650/19560 | loss 3.246291 (-1.07z)| norm 0.2541 (-1.07z)| lr 3.46e-05 | 322.90 ms | 52.3% bf16 MFU | 1623968 tok/s
step 16651/19560 | loss 3.291696 (-0.17z)| norm 0.2857 (-0.04z)| lr 3.46e-05 | 322.91 ms | 52.3% bf16 MFU | 1623952 tok/s
step 16652/19560 | loss 3.401072 (+1.96z)| norm 0.3258 (+1.24z)| lr 3.45e-05 | 322.97 ms | 52.3% bf16 MFU | 1623922 tok/s
step 16653/19560 | loss 3.348315 (+0.92z)| norm 0.2970 (+0.30z)| lr 3.45e-05 | 322.71 ms | 52.3% bf16 MFU | 1623957 tok/s
step 16654/19560 | loss 3.320825 (+0.38z)| norm 0.3258 (+1.21z)| lr 3.45e-05 | 322.92 ms | 52.3% bf16 MFU | 1623940 tok/s
step 16655/19560 | loss 3.309603 (+0.15z)| norm 0.2830 (-0.15z)| lr 3.45e-05 | 322.95 ms | 52.3% bf16 MFU | 1623915 tok/s
step 16656/19560 | loss 3.318171 (+0.31z)| norm 0.2672 (-0.66z)| lr 3.44e-05 | 322.95 ms | 52.3% bf16 MFU | 1623892 tok/s
step 16657/19560 | loss 3.276621 (-0.50z)| norm 0.2824 (-0.17z)| lr 3.44e-05 | 322.64 ms | 52.3% bf16 MFU | 1623947 tok/s
step 16658/19560 | loss 3.288654 (-0.27z)| norm 0.2589 (-0.94z)| lr 3.44e-05 | 323.11 ms | 52.2% bf16 MFU | 1623880 tok/s
step 16659/19560 | loss 3.250624 (-1.03z)| norm 0.2747 (-0.44z)| lr 3.44e-05 | 322.86 ms | 52.3% bf16 MFU | 1623881 tok/s
step 16660/19560 | loss 3.262854 (-0.77z)| norm 0.2720 (-0.52z)| lr 3.44e-05 | 323.22 ms | 52.2% bf16 MFU | 1623790 tok/s
step 16661/19560 | loss 3.273935 (-0.56z)| norm 0.2626 (-0.83z)| lr 3.43e-05 | 322.93 ms | 52.3% bf16 MFU | 1623777 tok/s
step 16662/19560 | loss 3.261455 (-0.80z)| norm 0.2837 (-0.15z)| lr 3.43e-05 | 322.60 ms | 52.3% bf16 MFU | 1623847 tok/s
step 16663/19560 | loss 3.265350 (-0.71z)| norm 0.2889 (+0.02z)| lr 3.43e-05 | 322.55 ms | 52.3% bf16 MFU | 1623928 tok/s
step 16664/19560 | loss 3.291650 (-0.20z)| norm 0.2774 (-0.36z)| lr 3.43e-05 | 323.26 ms | 52.2% bf16 MFU | 1623826 tok/s
step 16665/19560 | loss 3.275584 (-0.51z)| norm 0.2715 (-0.55z)| lr 3.42e-05 | 323.19 ms | 52.2% bf16 MFU | 1623745 tok/s
step 16666/19560 | loss 3.274842 (-0.52z)| norm 0.3530 (+2.10z)| lr 3.42e-05 | 322.98 ms | 52.3% bf16 MFU | 1623722 tok/s
step 16667/19560 | loss 3.239648 (-1.22z)| norm 0.3376 (+1.57z)| lr 3.42e-05 | 322.86 ms | 52.3% bf16 MFU | 1623729 tok/s
step 16668/19560 | loss 3.292221 (-0.19z)| norm 0.3015 (+0.41z)| lr 3.42e-05 | 322.67 ms | 52.3% bf16 MFU | 1623785 tok/s
step 16669/19560 | loss 3.330357 (+0.57z)| norm 0.3118 (+0.74z)| lr 3.41e-05 | 322.91 ms | 52.3% bf16 MFU | 1623778 tok/s
step 16670/19560 | loss 3.300941 (-0.01z)| norm 0.3413 (+1.66z)| lr 3.41e-05 | 322.72 ms | 52.3% bf16 MFU | 1623818 tok/s
step 16671/19560 | loss 3.367404 (+1.31z)| norm 0.2659 (-0.75z)| lr 3.41e-05 | 323.15 ms | 52.2% bf16 MFU | 1623749 tok/s
step 16672/19560 | loss 3.289483 (-0.25z)| norm 0.3614 (+2.29z)| lr 3.41e-05 | 322.43 ms | 52.3% bf16 MFU | 1623864 tok/s
step 16673/19560 | loss 3.302633 (+0.02z)| norm 0.3148 (+0.79z)| lr 3.40e-05 | 323.25 ms | 52.2% bf16 MFU | 1623767 tok/s
step 16674/19560 | loss 3.304492 (+0.05z)| norm 0.2588 (-1.00z)| lr 3.40e-05 | 323.34 ms | 52.2% bf16 MFU | 1623652 tok/s
step 16675/19560 | loss 3.315919 (+0.27z)| norm 0.3235 (+1.15z)| lr 3.40e-05 | 323.35 ms | 52.2% bf16 MFU | 1623540 tok/s
step 16676/19560 | loss 3.400908 (+1.95z)| norm 0.2914 (+0.09z)| lr 3.40e-05 | 322.10 ms | 52.4% bf16 MFU | 1623747 tok/s
step 16677/19560 | loss 3.250982 (-1.05z)| norm 0.2632 (-0.85z)| lr 3.40e-05 | 323.17 ms | 52.2% bf16 MFU | 1623676 tok/s
step 16678/19560 | loss 3.284535 (-0.37z)| norm 0.2642 (-0.81z)| lr 3.39e-05 | 322.90 ms | 52.3% bf16 MFU | 1623678 tok/s
step 16679/19560 | loss 3.260796 (-0.84z)| norm 0.3062 (+0.65z)| lr 3.39e-05 | 323.09 ms | 52.2% bf16 MFU | 1623631 tok/s
step 16680/19560 | loss 3.309076 (+0.12z)| norm 0.2653 (-0.76z)| lr 3.39e-05 | 322.81 ms | 52.3% bf16 MFU | 1623656 tok/s
step 16681/19560 | loss 3.267884 (-0.70z)| norm 0.3142 (+0.92z)| lr 3.39e-05 | 322.28 ms | 52.4% bf16 MFU | 1623812 tok/s
step 16682/19560 | loss 3.306868 (+0.08z)| norm 0.2807 (-0.22z)| lr 3.38e-05 | 322.65 ms | 52.3% bf16 MFU | 1623868 tok/s
step 16683/19560 | loss 3.298943 (-0.08z)| norm 0.2752 (-0.41z)| lr 3.38e-05 | 322.87 ms | 52.3% bf16 MFU | 1623866 tok/s
step 16684/19560 | loss 3.301015 (-0.04z)| norm 0.3067 (+0.68z)| lr 3.38e-05 | 323.01 ms | 52.2% bf16 MFU | 1623829 tok/s
step 16685/19560 | loss 3.285632 (-0.34z)| norm 0.2837 (-0.13z)| lr 3.38e-05 | 323.03 ms | 52.2% bf16 MFU | 1623790 tok/s
step 16686/19560 | loss 3.279464 (-0.46z)| norm 0.2905 (+0.12z)| lr 3.37e-05 | 322.80 ms | 52.3% bf16 MFU | 1623809 tok/s
step 16687/19560 | loss 3.285048 (-0.36z)| norm 0.2928 (+0.20z)| lr 3.37e-05 | 322.76 ms | 52.3% bf16 MFU | 1623839 tok/s
step 16688/19560 | loss 3.343126 (+0.81z)| norm 0.3173 (+1.09z)| lr 3.37e-05 | 322.91 ms | 52.3% bf16 MFU | 1623829 tok/s
step 16689/19560 | loss 3.247020 (-1.12z)| norm 0.3341 (+1.66z)| lr 3.37e-05 | 323.26 ms | 52.2% bf16 MFU | 1623732 tok/s
step 16690/19560 | loss 3.300536 (-0.04z)| norm 0.2570 (-1.07z)| lr 3.37e-05 | 323.10 ms | 52.2% bf16 MFU | 1623680 tok/s
step 16691/19560 | loss 3.450850 (+2.87z)| norm 0.3558 (+2.36z)| lr 3.36e-05 | 322.88 ms | 52.3% bf16 MFU | 1623684 tok/s
step 16692/19560 | loss 3.325626 (+0.42z)| norm 0.3349 (+1.62z)| lr 3.36e-05 | 322.52 ms | 52.3% bf16 MFU | 1623780 tok/s
step 16693/19560 | loss 3.376954 (+1.41z)| norm 0.2857 (-0.08z)| lr 3.36e-05 | 322.82 ms | 52.3% bf16 MFU | 1623795 tok/s
step 16694/19560 | loss 3.265018 (-0.78z)| norm 0.3093 (+0.73z)| lr 3.36e-05 | 323.21 ms | 52.2% bf16 MFU | 1623712 tok/s
step 16695/19560 | loss 3.242266 (-1.20z)| norm 0.2892 (+0.04z)| lr 3.35e-05 | 323.07 ms | 52.2% bf16 MFU | 1623669 tok/s
step 16696/19560 | loss 3.389545 (+1.64z)| norm 0.2748 (-0.46z)| lr 3.35e-05 | 322.83 ms | 52.3% bf16 MFU | 1623686 tok/s
step 16697/19560 | loss 3.244742 (-1.15z)| norm 0.2802 (-0.28z)| lr 3.35e-05 | 322.81 ms | 52.3% bf16 MFU | 1623709 tok/s
step 16698/19560 | loss 3.300840 (-0.07z)| norm 0.2520 (-1.25z)| lr 3.35e-05 | 322.89 ms | 52.3% bf16 MFU | 1623711 tok/s
step 16699/19560 | loss 3.394560 (+1.70z)| norm 0.2663 (-0.75z)| lr 3.35e-05 | 323.03 ms | 52.2% bf16 MFU | 1623676 tok/s
step 16700/19560 | loss 3.361934 (+1.07z)| norm 0.2943 (+0.21z)| lr 3.34e-05 | 322.65 ms | 52.3% bf16 MFU | 1623740 tok/s
step 16701/19560 | loss 3.280881 (-0.46z)| norm 0.2581 (-1.04z)| lr 3.34e-05 | 323.31 ms | 52.2% bf16 MFU | 1623635 tok/s
step 16702/19560 | loss 3.271270 (-0.63z)| norm 0.2919 (+0.12z)| lr 3.34e-05 | 322.84 ms | 52.3% bf16 MFU | 1623652 tok/s
step 16703/19560 | loss 3.314201 (+0.19z)| norm 0.3238 (+1.21z)| lr 3.34e-05 | 323.47 ms | 52.2% bf16 MFU | 1623511 tok/s
step 16704/19560 | loss 3.260691 (-0.82z)| norm 0.2536 (-1.22z)| lr 3.33e-05 | 323.12 ms | 52.2% bf16 MFU | 1623464 tok/s
step 16705/19560 | loss 3.260524 (-0.82z)| norm 0.2705 (-0.64z)| lr 3.33e-05 | 322.97 ms | 52.3% bf16 MFU | 1623459 tok/s
step 16706/19560 | loss 3.423868 (+2.21z)| norm 0.2796 (-0.33z)| lr 3.33e-05 | 323.32 ms | 52.2% bf16 MFU | 1623364 tok/s
step 16707/19560 | loss 3.305434 (+0.01z)| norm 0.2842 (-0.17z)| lr 3.33e-05 | 323.06 ms | 52.2% bf16 MFU | 1623341 tok/s
step 16708/19560 | loss 3.374289 (+1.27z)| norm 0.2774 (-0.41z)| lr 3.32e-05 | 323.23 ms | 52.2% bf16 MFU | 1623274 tok/s
step 16709/19560 | loss 3.288729 (-0.32z)| norm 0.2701 (-0.67z)| lr 3.32e-05 | 323.67 ms | 52.1% bf16 MFU | 1623101 tok/s
step 16710/19560 | loss 3.305774 (-0.00z)| norm 0.2840 (-0.18z)| lr 3.32e-05 | 322.71 ms | 52.3% bf16 MFU | 1623177 tok/s
step 16711/19560 | loss 3.265071 (-0.74z)| norm 0.2801 (-0.33z)| lr 3.32e-05 | 322.17 ms | 52.4% bf16 MFU | 1623387 tok/s
step 16712/19560 | loss 3.258179 (-0.87z)| norm 0.2717 (-0.62z)| lr 3.32e-05 | 323.30 ms | 52.2% bf16 MFU | 1623301 tok/s
step 16713/19560 | loss 3.311560 (+0.12z)| norm 0.2903 (+0.03z)| lr 3.31e-05 | 322.94 ms | 52.3% bf16 MFU | 1623311 tok/s
step 16714/19560 | loss 3.293574 (-0.21z)| norm 0.3030 (+0.46z)| lr 3.31e-05 | 323.03 ms | 52.2% bf16 MFU | 1623296 tok/s
step 16715/19560 | loss 3.262620 (-0.78z)| norm 0.2639 (-0.91z)| lr 3.31e-05 | 322.73 ms | 52.3% bf16 MFU | 1623358 tok/s
step 16716/19560 | loss 3.315212 (+0.19z)| norm 0.2771 (-0.44z)| lr 3.31e-05 | 322.22 ms | 52.4% bf16 MFU | 1623546 tok/s
step 16717/19560 | loss 3.303110 (-0.04z)| norm 0.2730 (-0.60z)| lr 3.30e-05 | 322.84 ms | 52.3% bf16 MFU | 1623568 tok/s
step 16718/19560 | loss 3.296162 (-0.16z)| norm 0.2820 (-0.27z)| lr 3.30e-05 | 322.61 ms | 52.3% bf16 MFU | 1623647 tok/s
step 16719/19560 | loss 3.343365 (+0.71z)| norm 0.2689 (-0.73z)| lr 3.30e-05 | 322.06 ms | 52.4% bf16 MFU | 1623862 tok/s
step 16720/19560 | loss 3.316480 (+0.20z)| norm 0.2833 (-0.22z)| lr 3.30e-05 | 322.27 ms | 52.4% bf16 MFU | 1624012 tok/s
step 16721/19560 | loss 3.319405 (+0.25z)| norm 0.2654 (-0.85z)| lr 3.29e-05 | 322.52 ms | 52.3% bf16 MFU | 1624090 tok/s
step 16722/19560 | loss 3.256915 (-0.91z)| norm 0.2923 (+0.13z)| lr 3.29e-05 | 321.79 ms | 52.4% bf16 MFU | 1624350 tok/s
step 16723/19560 | loss 3.292310 (-0.24z)| norm 0.2994 (+0.38z)| lr 3.29e-05 | 322.16 ms | 52.4% bf16 MFU | 1624503 tok/s
step 16724/19560 | loss 3.344808 (+0.73z)| norm 0.2721 (-0.61z)| lr 3.29e-05 | 322.85 ms | 52.3% bf16 MFU | 1624474 tok/s
step 16725/19560 | loss 3.296625 (-0.17z)| norm 0.2991 (+0.42z)| lr 3.29e-05 | 322.11 ms | 52.4% bf16 MFU | 1624632 tok/s
step 16726/19560 | loss 3.270389 (-0.66z)| norm 0.2551 (-1.26z)| lr 3.28e-05 | 322.84 ms | 52.3% bf16 MFU | 1624600 tok/s
step 16727/19560 | loss 3.329332 (+0.44z)| norm 0.3370 (+1.88z)| lr 3.28e-05 | 323.42 ms | 52.2% bf16 MFU | 1624423 tok/s
step 16728/19560 | loss 3.300608 (-0.09z)| norm 0.2761 (-0.44z)| lr 3.28e-05 | 322.49 ms | 52.3% bf16 MFU | 1624490 tok/s
step 16729/19560 | loss 3.415763 (+2.01z)| norm 0.2933 (+0.21z)| lr 3.28e-05 | 322.78 ms | 52.3% bf16 MFU | 1624481 tok/s
step 16730/19560 | loss 3.296687 (-0.18z)| norm 0.2648 (-0.88z)| lr 3.27e-05 | 322.52 ms | 52.3% bf16 MFU | 1624536 tok/s
step 16731/19560 | loss 3.236039 (-1.27z)| norm 0.3132 (+1.01z)| lr 3.27e-05 | 322.27 ms | 52.4% bf16 MFU | 1624652 tok/s
step 16732/19560 | loss 3.281040 (-0.45z)| norm 0.2472 (-1.54z)| lr 3.27e-05 | 322.84 ms | 52.3% bf16 MFU | 1624620 tok/s
step 16733/19560 | loss 3.392183 (+1.55z)| norm 0.8303 (+9.95z)| lr 3.27e-05 | 322.28 ms | 52.4% bf16 MFU | 1624728 tok/s
step 16734/19560 | loss 3.268112 (-0.70z)| norm 0.2857 (-0.09z)| lr 3.27e-05 | 322.85 ms | 52.3% bf16 MFU | 1624689 tok/s
step 16735/19560 | loss 3.302920 (-0.06z)| norm 0.2715 (-0.35z)| lr 3.26e-05 | 322.60 ms | 52.3% bf16 MFU | 1624715 tok/s
step 16736/19560 | loss 3.258206 (-0.86z)| norm 0.2723 (-0.33z)| lr 3.26e-05 | 322.78 ms | 52.3% bf16 MFU | 1624692 tok/s
step 16737/19560 | loss 3.318853 (+0.24z)| norm 0.2516 (-0.71z)| lr 3.26e-05 | 322.71 ms | 52.3% bf16 MFU | 1624690 tok/s
step 16738/19560 | loss 3.270596 (-0.63z)| norm 0.2588 (-0.57z)| lr 3.26e-05 | 322.59 ms | 52.3% bf16 MFU | 1624717 tok/s
step 16739/19560 | loss 3.302073 (-0.06z)| norm 0.2678 (-0.40z)| lr 3.25e-05 | 322.70 ms | 52.3% bf16 MFU | 1624715 tok/s
step 16740/19560 | loss 3.297340 (-0.15z)| norm 0.2975 (+0.14z)| lr 3.25e-05 | 322.82 ms | 52.3% bf16 MFU | 1624683 tok/s
step 16741/19560 | loss 3.252573 (-0.95z)| norm 0.2543 (-0.65z)| lr 3.25e-05 | 322.62 ms | 52.3% bf16 MFU | 1624703 tok/s
step 16742/19560 | loss 3.375947 (+1.29z)| norm 0.2634 (-0.48z)| lr 3.25e-05 | 323.10 ms | 52.2% bf16 MFU | 1624601 tok/s
step 16743/19560 | loss 3.270962 (-0.61z)| norm 0.3392 (+0.90z)| lr 3.24e-05 | 322.43 ms | 52.3% bf16 MFU | 1624673 tok/s
step 16744/19560 | loss 3.264617 (-0.72z)| norm 0.2506 (-0.72z)| lr 3.24e-05 | 322.35 ms | 52.4% bf16 MFU | 1624763 tok/s
step 16745/19560 | loss 3.409487 (+1.86z)| norm 0.2649 (-0.46z)| lr 3.24e-05 | 323.20 ms | 52.2% bf16 MFU | 1624633 tok/s
step 16746/19560 | loss 3.311878 (+0.11z)| norm 0.2804 (-0.18z)| lr 3.24e-05 | 322.84 ms | 52.3% bf16 MFU | 1624600 tok/s
step 16747/19560 | loss 3.325890 (+0.35z)| norm 0.3057 (+0.28z)| lr 3.24e-05 | 322.77 ms | 52.3% bf16 MFU | 1624587 tok/s
step 16748/19560 | loss 3.329287 (+0.42z)| norm 0.2969 (+0.12z)| lr 3.23e-05 | 322.33 ms | 52.4% bf16 MFU | 1624686 tok/s
step 16749/19560 | loss 3.253569 (-0.93z)| norm 0.2933 (+0.05z)| lr 3.23e-05 | 323.16 ms | 52.2% bf16 MFU | 1624570 tok/s
step 16750/19560 | loss 3.347923 (+0.75z)| norm 0.3237 (+0.60z)| lr 3.23e-05 | 323.14 ms | 52.2% bf16 MFU | 1624465 tok/s
val loss 3.284536
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3029/10042 = 0.301633
step 16751/19560 | loss 3.269932 (-0.66z)| norm 0.2479 (-0.79z)| lr 3.23e-05 | 321.42 ms | 52.5% bf16 MFU | 1624800 tok/s
step 16752/19560 | loss 3.199228 (-1.89z)| norm 0.2813 (-0.17z)| lr 3.22e-05 | 321.81 ms | 52.4% bf16 MFU | 1625018 tok/s
step 16753/19560 | loss 3.343802 (+0.67z)| norm 0.3221 (+0.57z)| lr 3.22e-05 | 322.66 ms | 52.3% bf16 MFU | 1625011 tok/s
step 16754/19560 | loss 3.253969 (-0.91z)| norm 0.2807 (-0.19z)| lr 3.22e-05 | 322.49 ms | 52.3% bf16 MFU | 1625049 tok/s
step 16755/19560 | loss 3.322571 (+0.31z)| norm 0.3099 (+0.34z)| lr 3.22e-05 | 322.26 ms | 52.4% bf16 MFU | 1625141 tok/s
step 16756/19560 | loss 3.311123 (+0.11z)| norm 0.2698 (-0.39z)| lr 3.22e-05 | 322.26 ms | 52.4% bf16 MFU | 1625230 tok/s
step 16757/19560 | loss 3.285699 (-0.34z)| norm 0.3353 (+0.80z)| lr 3.21e-05 | 322.49 ms | 52.3% bf16 MFU | 1625256 tok/s
step 16758/19560 | loss 3.245613 (-1.05z)| norm 0.2730 (-0.33z)| lr 3.21e-05 | 322.64 ms | 52.3% bf16 MFU | 1625242 tok/s
step 16759/19560 | loss 3.255566 (-0.85z)| norm 0.2914 (+0.01z)| lr 3.21e-05 | 322.28 ms | 52.4% bf16 MFU | 1625321 tok/s
step 16760/19560 | loss 3.260453 (-0.76z)| norm 0.2872 (-0.07z)| lr 3.21e-05 | 322.52 ms | 52.3% bf16 MFU | 1625334 tok/s
step 16761/19560 | loss 3.388437 (+1.49z)| norm 0.2841 (-0.13z)| lr 3.20e-05 | 322.68 ms | 52.3% bf16 MFU | 1625306 tok/s
step 16762/19560 | loss 3.291188 (-0.22z)| norm 0.2811 (-0.19z)| lr 3.20e-05 | 322.59 ms | 52.3% bf16 MFU | 1625304 tok/s
step 16763/19560 | loss 3.269654 (-0.60z)| norm 0.2798 (-0.21z)| lr 3.20e-05 | 323.35 ms | 52.2% bf16 MFU | 1625110 tok/s
step 16764/19560 | loss 3.257715 (-0.82z)| norm 0.2859 (-0.10z)| lr 3.20e-05 | 322.76 ms | 52.3% bf16 MFU | 1625074 tok/s
step 16765/19560 | loss 3.311861 (+0.14z)| norm 0.2763 (-0.27z)| lr 3.20e-05 | 322.69 ms | 52.3% bf16 MFU | 1625056 tok/s
step 16766/19560 | loss 3.235816 (-1.20z)| norm 0.3225 (+0.57z)| lr 3.19e-05 | 322.61 ms | 52.3% bf16 MFU | 1625062 tok/s
step 16767/19560 | loss 3.308638 (+0.08z)| norm 0.2815 (-0.19z)| lr 3.19e-05 | 323.02 ms | 52.2% bf16 MFU | 1624962 tok/s
step 16768/19560 | loss 3.332074 (+0.49z)| norm 0.2583 (-0.62z)| lr 3.19e-05 | 322.77 ms | 52.3% bf16 MFU | 1624932 tok/s
step 16769/19560 | loss 3.318044 (+0.24z)| norm 0.2781 (-0.25z)| lr 3.19e-05 | 322.33 ms | 52.4% bf16 MFU | 1625014 tok/s
step 16770/19560 | loss 3.269161 (-0.63z)| norm 0.2451 (-0.85z)| lr 3.18e-05 | 322.79 ms | 52.3% bf16 MFU | 1624975 tok/s
step 16771/19560 | loss 3.298409 (-0.11z)| norm 0.2643 (-0.50z)| lr 3.18e-05 | 322.57 ms | 52.3% bf16 MFU | 1624993 tok/s
step 16772/19560 | loss 3.306854 (+0.07z)| norm 0.2519 (-0.72z)| lr 3.18e-05 | 322.72 ms | 52.3% bf16 MFU | 1624973 tok/s
step 16773/19560 | loss 3.314536 (+0.21z)| norm 0.2851 (-0.12z)| lr 3.18e-05 | 322.81 ms | 52.3% bf16 MFU | 1624932 tok/s
step 16774/19560 | loss 3.298230 (-0.06z)| norm 0.2876 (-0.06z)| lr 3.18e-05 | 322.58 ms | 52.3% bf16 MFU | 1624950 tok/s
step 16775/19560 | loss 3.289515 (-0.26z)| norm 0.2866 (-0.07z)| lr 3.17e-05 | 322.41 ms | 52.3% bf16 MFU | 1625011 tok/s
step 16776/19560 | loss 3.276062 (-0.57z)| norm 0.2521 (-0.71z)| lr 3.17e-05 | 322.88 ms | 52.3% bf16 MFU | 1624950 tok/s
step 16777/19560 | loss 3.301143 (+0.00z)| norm 0.2442 (-0.85z)| lr 3.17e-05 | 322.11 ms | 52.4% bf16 MFU | 1625086 tok/s
step 16778/19560 | loss 3.274077 (-0.62z)| norm 0.2587 (-0.58z)| lr 3.17e-05 | 322.49 ms | 52.3% bf16 MFU | 1625119 tok/s
step 16779/19560 | loss 3.293022 (-0.19z)| norm 0.2774 (-0.23z)| lr 3.16e-05 | 322.59 ms | 52.3% bf16 MFU | 1625124 tok/s
step 16780/19560 | loss 3.288032 (-0.29z)| norm 0.2847 (-0.09z)| lr 3.16e-05 | 322.47 ms | 52.3% bf16 MFU | 1625161 tok/s
step 16781/19560 | loss 3.281570 (-0.42z)| norm 0.2391 (-0.92z)| lr 3.16e-05 | 322.93 ms | 52.3% bf16 MFU | 1625079 tok/s
step 16782/19560 | loss 3.213964 (-1.95z)| norm 0.2564 (-0.60z)| lr 3.16e-05 | 323.36 ms | 52.2% bf16 MFU | 1624892 tok/s
step 16783/19560 | loss 3.264927 (-0.77z)| norm 0.2400 (-0.89z)| lr 3.16e-05 | 322.79 ms | 52.3% bf16 MFU | 1624859 tok/s
step 16784/19560 | loss 3.384070 (+1.91z)| norm 0.2552 (-0.61z)| lr 3.15e-05 | 322.69 ms | 52.3% bf16 MFU | 1624854 tok/s
step 16785/19560 | loss 3.292271 (-0.16z)| norm 0.2889 (+0.01z)| lr 3.15e-05 | 323.03 ms | 52.2% bf16 MFU | 1624763 tok/s
step 16786/19560 | loss 3.307551 (+0.18z)| norm 0.3131 (+0.44z)| lr 3.15e-05 | 322.85 ms | 52.3% bf16 MFU | 1624720 tok/s
step 16787/19560 | loss 3.330037 (+0.68z)| norm 0.2595 (-0.54z)| lr 3.15e-05 | 322.51 ms | 52.3% bf16 MFU | 1624767 tok/s
step 16788/19560 | loss 3.312823 (+0.28z)| norm 0.2824 (-0.12z)| lr 3.14e-05 | 323.26 ms | 52.2% bf16 MFU | 1624621 tok/s
step 16789/19560 | loss 3.234319 (-1.48z)| norm 0.3226 (+0.61z)| lr 3.14e-05 | 322.60 ms | 52.3% bf16 MFU | 1624650 tok/s
step 16790/19560 | loss 3.286140 (-0.32z)| norm 0.2601 (-0.53z)| lr 3.14e-05 | 322.57 ms | 52.3% bf16 MFU | 1624686 tok/s
step 16791/19560 | loss 3.252568 (-1.07z)| norm 0.2605 (-0.52z)| lr 3.14e-05 | 322.74 ms | 52.3% bf16 MFU | 1624677 tok/s
step 16792/19560 | loss 3.235292 (-1.44z)| norm 0.2721 (-0.31z)| lr 3.14e-05 | 322.47 ms | 52.3% bf16 MFU | 1624736 tok/s
step 16793/19560 | loss 3.265966 (-0.75z)| norm 0.2871 (-0.03z)| lr 3.13e-05 | 323.09 ms | 52.2% bf16 MFU | 1624637 tok/s
step 16794/19560 | loss 3.342288 (+0.94z)| norm 0.2635 (-0.46z)| lr 3.13e-05 | 322.93 ms | 52.3% bf16 MFU | 1624582 tok/s
step 16795/19560 | loss 3.308746 (+0.18z)| norm 0.2765 (-0.21z)| lr 3.13e-05 | 322.65 ms | 52.3% bf16 MFU | 1624600 tok/s
step 16796/19560 | loss 3.222619 (-1.72z)| norm 0.2897 (+0.04z)| lr 3.13e-05 | 323.27 ms | 52.2% bf16 MFU | 1624460 tok/s
step 16797/19560 | loss 3.210274 (-1.95z)| norm 0.2645 (-0.42z)| lr 3.12e-05 | 322.86 ms | 52.3% bf16 MFU | 1624431 tok/s
step 16798/19560 | loss 3.330898 (+0.68z)| norm 0.2843 (-0.05z)| lr 3.12e-05 | 322.89 ms | 52.3% bf16 MFU | 1624397 tok/s
step 16799/19560 | loss 3.303941 (+0.11z)| norm 0.2576 (-0.54z)| lr 3.12e-05 | 323.05 ms | 52.2% bf16 MFU | 1624324 tok/s
step 16800/19560 | loss 3.293525 (-0.12z)| norm 0.3442 (+1.07z)| lr 3.12e-05 | 322.72 ms | 52.3% bf16 MFU | 1624338 tok/s
step 16801/19560 | loss 3.212166 (-1.87z)| norm 0.2752 (-0.21z)| lr 3.12e-05 | 322.48 ms | 52.3% bf16 MFU | 1624410 tok/s
step 16802/19560 | loss 3.291646 (-0.14z)| norm 0.2824 (-0.08z)| lr 3.11e-05 | 322.35 ms | 52.4% bf16 MFU | 1624511 tok/s
step 16803/19560 | loss 3.247314 (-1.09z)| norm 0.3119 (+0.47z)| lr 3.11e-05 | 322.77 ms | 52.3% bf16 MFU | 1624502 tok/s
step 16804/19560 | loss 3.403781 (+2.29z)| norm 0.2804 (-0.11z)| lr 3.11e-05 | 322.62 ms | 52.3% bf16 MFU | 1624530 tok/s
step 16805/19560 | loss 3.233455 (-1.38z)| norm 0.2731 (-0.25z)| lr 3.11e-05 | 322.59 ms | 52.3% bf16 MFU | 1624567 tok/s
step 16806/19560 | loss 3.256139 (-0.89z)| norm 0.3001 (+0.25z)| lr 3.10e-05 | 322.84 ms | 52.3% bf16 MFU | 1624539 tok/s
step 16807/19560 | loss 3.303163 (+0.12z)| norm 0.2689 (-0.33z)| lr 3.10e-05 | 322.86 ms | 52.3% bf16 MFU | 1624507 tok/s
step 16808/19560 | loss 3.279500 (-0.39z)| norm 0.3318 (+0.83z)| lr 3.10e-05 | 322.85 ms | 52.3% bf16 MFU | 1624477 tok/s
step 16809/19560 | loss 3.430587 (+2.75z)| norm 0.2769 (-0.18z)| lr 3.10e-05 | 322.46 ms | 52.3% bf16 MFU | 1624549 tok/s
step 16810/19560 | loss 3.313381 (+0.30z)| norm 0.2734 (-0.25z)| lr 3.10e-05 | 322.50 ms | 52.3% bf16 MFU | 1624606 tok/s
step 16811/19560 | loss 3.278500 (-0.42z)| norm 0.2838 (-0.05z)| lr 3.09e-05 | 323.55 ms | 52.2% bf16 MFU | 1624398 tok/s
step 16812/19560 | loss 3.323280 (+0.51z)| norm 0.3287 (+0.78z)| lr 3.09e-05 | 322.70 ms | 52.3% bf16 MFU | 1624411 tok/s
step 16813/19560 | loss 3.355698 (+1.17z)| norm 0.2783 (-0.16z)| lr 3.09e-05 | 322.35 ms | 52.4% bf16 MFU | 1624513 tok/s
step 16814/19560 | loss 3.277495 (-0.45z)| norm 0.3251 (+0.71z)| lr 3.09e-05 | 322.65 ms | 52.3% bf16 MFU | 1624534 tok/s
step 16815/19560 | loss 3.310220 (+0.22z)| norm 0.3000 (+0.24z)| lr 3.08e-05 | 322.86 ms | 52.3% bf16 MFU | 1624502 tok/s
step 16816/19560 | loss 3.298986 (-0.01z)| norm 0.2608 (-0.48z)| lr 3.08e-05 | 322.79 ms | 52.3% bf16 MFU | 1624489 tok/s
step 16817/19560 | loss 3.257123 (-0.88z)| norm 0.2568 (-0.54z)| lr 3.08e-05 | 323.42 ms | 52.2% bf16 MFU | 1624318 tok/s
step 16818/19560 | loss 3.268685 (-0.63z)| norm 0.2511 (-0.65z)| lr 3.08e-05 | 322.38 ms | 52.4% bf16 MFU | 1624416 tok/s
step 16819/19560 | loss 3.247325 (-1.08z)| norm 0.2842 (-0.02z)| lr 3.08e-05 | 322.74 ms | 52.3% bf16 MFU | 1624420 tok/s
step 16820/19560 | loss 3.414057 (+2.45z)| norm 0.2759 (-0.17z)| lr 3.07e-05 | 322.68 ms | 52.3% bf16 MFU | 1624438 tok/s
step 16821/19560 | loss 3.294058 (-0.07z)| norm 0.2675 (-0.33z)| lr 3.07e-05 | 323.24 ms | 52.2% bf16 MFU | 1624314 tok/s
step 16822/19560 | loss 3.306897 (+0.19z)| norm 0.2835 (-0.02z)| lr 3.07e-05 | 322.76 ms | 52.3% bf16 MFU | 1624317 tok/s
step 16823/19560 | loss 3.269547 (-0.61z)| norm 0.2928 (+0.15z)| lr 3.07e-05 | 323.01 ms | 52.3% bf16 MFU | 1624258 tok/s
step 16824/19560 | loss 3.340307 (+0.93z)| norm 0.2595 (-0.47z)| lr 3.06e-05 | 322.50 ms | 52.3% bf16 MFU | 1624330 tok/s
step 16825/19560 | loss 3.303787 (+0.12z)| norm 0.3263 (+0.77z)| lr 3.06e-05 | 323.00 ms | 52.3% bf16 MFU | 1624274 tok/s
step 16826/19560 | loss 3.305335 (+0.16z)| norm 0.2628 (-0.42z)| lr 3.06e-05 | 323.17 ms | 52.2% bf16 MFU | 1624177 tok/s
step 16827/19560 | loss 3.254234 (-0.95z)| norm 0.3080 (+0.42z)| lr 3.06e-05 | 322.43 ms | 52.3% bf16 MFU | 1624270 tok/s
step 16828/19560 | loss 3.259471 (-0.82z)| norm 0.2675 (-0.33z)| lr 3.06e-05 | 322.61 ms | 52.3% bf16 MFU | 1624313 tok/s
step 16829/19560 | loss 3.270185 (-0.58z)| norm 0.2595 (-0.48z)| lr 3.05e-05 | 322.99 ms | 52.3% bf16 MFU | 1624258 tok/s
step 16830/19560 | loss 3.275399 (-0.46z)| norm 0.3472 (+1.15z)| lr 3.05e-05 | 323.23 ms | 52.2% bf16 MFU | 1624146 tok/s
step 16831/19560 | loss 3.260386 (-0.79z)| norm 0.2524 (-0.61z)| lr 3.05e-05 | 322.98 ms | 52.3% bf16 MFU | 1624102 tok/s
step 16832/19560 | loss 3.293116 (-0.07z)| norm 0.2615 (-0.44z)| lr 3.05e-05 | 323.16 ms | 52.2% bf16 MFU | 1624017 tok/s
step 16833/19560 | loss 3.277722 (-0.41z)| norm 0.2672 (-0.33z)| lr 3.04e-05 | 322.74 ms | 52.3% bf16 MFU | 1624040 tok/s
step 16834/19560 | loss 3.382643 (+1.97z)| norm 0.3025 (+0.32z)| lr 3.04e-05 | 322.72 ms | 52.3% bf16 MFU | 1624066 tok/s
step 16835/19560 | loss 3.222112 (-1.64z)| norm 0.2597 (-0.47z)| lr 3.04e-05 | 322.81 ms | 52.3% bf16 MFU | 1624071 tok/s
step 16836/19560 | loss 3.271356 (-0.52z)| norm 0.2537 (-0.58z)| lr 3.04e-05 | 323.26 ms | 52.2% bf16 MFU | 1623961 tok/s
step 16837/19560 | loss 3.273221 (-0.48z)| norm 0.2621 (-0.42z)| lr 3.04e-05 | 322.81 ms | 52.3% bf16 MFU | 1623969 tok/s
step 16838/19560 | loss 3.325302 (+0.70z)| norm 0.2727 (-0.22z)| lr 3.03e-05 | 322.35 ms | 52.4% bf16 MFU | 1624093 tok/s
step 16839/19560 | loss 3.258019 (-0.82z)| norm 0.3197 (+0.64z)| lr 3.03e-05 | 322.90 ms | 52.3% bf16 MFU | 1624072 tok/s
step 16840/19560 | loss 3.267805 (-0.60z)| norm 0.2516 (-0.62z)| lr 3.03e-05 | 323.05 ms | 52.2% bf16 MFU | 1624015 tok/s
step 16841/19560 | loss 3.272656 (-0.49z)| norm 0.2461 (-0.71z)| lr 3.03e-05 | 322.47 ms | 52.3% bf16 MFU | 1624106 tok/s
step 16842/19560 | loss 3.294426 (+0.01z)| norm 0.2605 (-0.44z)| lr 3.02e-05 | 322.86 ms | 52.3% bf16 MFU | 1624094 tok/s
step 16843/19560 | loss 3.298155 (+0.08z)| norm 0.2480 (-0.67z)| lr 3.02e-05 | 323.11 ms | 52.2% bf16 MFU | 1624020 tok/s
step 16844/19560 | loss 3.337911 (+0.98z)| norm 0.2532 (-0.57z)| lr 3.02e-05 | 323.25 ms | 52.2% bf16 MFU | 1623916 tok/s
step 16845/19560 | loss 3.245756 (-1.09z)| norm 0.2688 (-0.28z)| lr 3.02e-05 | 322.62 ms | 52.3% bf16 MFU | 1623974 tok/s
step 16846/19560 | loss 3.186110 (-2.36z)| norm 0.2501 (-0.62z)| lr 3.02e-05 | 323.34 ms | 52.2% bf16 MFU | 1623848 tok/s
step 16847/19560 | loss 3.270825 (-0.49z)| norm 0.2401 (-0.79z)| lr 3.01e-05 | 322.37 ms | 52.4% bf16 MFU | 1623974 tok/s
step 16848/19560 | loss 3.252874 (-0.87z)| norm 0.2735 (-0.18z)| lr 3.01e-05 | 323.08 ms | 52.2% bf16 MFU | 1623914 tok/s
step 16849/19560 | loss 3.335304 (+0.95z)| norm 0.2501 (-0.61z)| lr 3.01e-05 | 323.18 ms | 52.2% bf16 MFU | 1623833 tok/s
step 16850/19560 | loss 3.327900 (+0.77z)| norm 0.2752 (-0.14z)| lr 3.01e-05 | 322.47 ms | 52.3% bf16 MFU | 1623935 tok/s
step 16851/19560 | loss 3.284239 (-0.19z)| norm 0.3743 (+1.65z)| lr 3.01e-05 | 322.52 ms | 52.3% bf16 MFU | 1624019 tok/s
step 16852/19560 | loss 3.420101 (+2.72z)| norm 0.2675 (-0.29z)| lr 3.00e-05 | 322.46 ms | 52.3% bf16 MFU | 1624114 tok/s
step 16853/19560 | loss 3.254483 (-0.83z)| norm 0.2645 (-0.34z)| lr 3.00e-05 | 323.25 ms | 52.2% bf16 MFU | 1624005 tok/s
step 16854/19560 | loss 3.306162 (+0.27z)| norm 0.2830 (-0.01z)| lr 3.00e-05 | 322.91 ms | 52.3% bf16 MFU | 1623986 tok/s
step 16855/19560 | loss 3.275909 (-0.37z)| norm 0.2692 (-0.25z)| lr 3.00e-05 | 323.12 ms | 52.2% bf16 MFU | 1623916 tok/s
step 16856/19560 | loss 3.308731 (+0.34z)| norm 0.2616 (-0.39z)| lr 2.99e-05 | 322.95 ms | 52.3% bf16 MFU | 1623892 tok/s
step 16857/19560 | loss 3.299145 (+0.15z)| norm 0.2892 (+0.12z)| lr 2.99e-05 | 322.56 ms | 52.3% bf16 MFU | 1623967 tok/s
step 16858/19560 | loss 3.358472 (+1.44z)| norm 0.2739 (-0.16z)| lr 2.99e-05 | 322.70 ms | 52.3% bf16 MFU | 1624002 tok/s
step 16859/19560 | loss 3.313369 (+0.44z)| norm 0.2838 (+0.02z)| lr 2.99e-05 | 322.83 ms | 52.3% bf16 MFU | 1624004 tok/s
step 16860/19560 | loss 3.302667 (+0.20z)| norm 0.2960 (+0.24z)| lr 2.99e-05 | 323.31 ms | 52.2% bf16 MFU | 1623885 tok/s
step 16861/19560 | loss 3.314776 (+0.49z)| norm 0.2518 (-1.06z)| lr 2.98e-05 | 322.67 ms | 52.3% bf16 MFU | 1623934 tok/s
step 16862/19560 | loss 3.305834 (+0.29z)| norm 0.3243 (+1.78z)| lr 2.98e-05 | 323.24 ms | 52.2% bf16 MFU | 1623837 tok/s
step 16863/19560 | loss 3.207859 (-1.87z)| norm 0.3347 (+2.12z)| lr 2.98e-05 | 323.79 ms | 52.1% bf16 MFU | 1623606 tok/s
step 16864/19560 | loss 3.244862 (-1.05z)| norm 0.2642 (-0.58z)| lr 2.98e-05 | 322.69 ms | 52.3% bf16 MFU | 1623664 tok/s
step 16865/19560 | loss 3.330002 (+0.83z)| norm 0.3689 (+3.27z)| lr 2.97e-05 | 323.21 ms | 52.2% bf16 MFU | 1623586 tok/s
step 16866/19560 | loss 3.296795 (+0.09z)| norm 0.3059 (+0.93z)| lr 2.97e-05 | 322.54 ms | 52.3% bf16 MFU | 1623683 tok/s
step 16867/19560 | loss 3.351979 (+1.30z)| norm 0.2601 (-0.75z)| lr 2.97e-05 | 323.52 ms | 52.2% bf16 MFU | 1623528 tok/s
step 16868/19560 | loss 3.250475 (-0.92z)| norm 0.3614 (+2.86z)| lr 2.97e-05 | 323.27 ms | 52.2% bf16 MFU | 1623442 tok/s
step 16869/19560 | loss 3.267936 (-0.54z)| norm 0.2810 (-0.01z)| lr 2.97e-05 | 322.83 ms | 52.3% bf16 MFU | 1623471 tok/s
step 16870/19560 | loss 3.300162 (+0.18z)| norm 0.2918 (+0.37z)| lr 2.96e-05 | 323.00 ms | 52.3% bf16 MFU | 1623457 tok/s
step 16871/19560 | loss 3.253981 (-0.84z)| norm 0.3200 (+1.40z)| lr 2.96e-05 | 322.49 ms | 52.3% bf16 MFU | 1623571 tok/s
step 16872/19560 | loss 3.285982 (-0.14z)| norm 0.2628 (-0.67z)| lr 2.96e-05 | 323.05 ms | 52.2% bf16 MFU | 1623539 tok/s
step 16873/19560 | loss 3.326150 (+0.79z)| norm 0.2961 (+0.52z)| lr 2.96e-05 | 323.03 ms | 52.2% bf16 MFU | 1623513 tok/s
step 16874/19560 | loss 3.296667 (+0.12z)| norm 0.3016 (+0.72z)| lr 2.96e-05 | 323.12 ms | 52.2% bf16 MFU | 1623466 tok/s
step 16875/19560 | loss 3.366267 (+1.68z)| norm 0.2627 (-0.68z)| lr 2.95e-05 | 323.96 ms | 52.1% bf16 MFU | 1623212 tok/s
step 16876/19560 | loss 3.305046 (+0.31z)| norm 0.2630 (-0.66z)| lr 2.95e-05 | 322.49 ms | 52.3% bf16 MFU | 1623337 tok/s
step 16877/19560 | loss 3.333586 (+0.94z)| norm 0.3220 (+1.46z)| lr 2.95e-05 | 323.04 ms | 52.2% bf16 MFU | 1623318 tok/s
step 16878/19560 | loss 3.337728 (+1.04z)| norm 0.2788 (-0.08z)| lr 2.95e-05 | 322.69 ms | 52.3% bf16 MFU | 1623390 tok/s
step 16879/19560 | loss 3.304240 (+0.27z)| norm 0.3078 (+0.95z)| lr 2.94e-05 | 322.43 ms | 52.3% bf16 MFU | 1623524 tok/s
step 16880/19560 | loss 3.270445 (-0.52z)| norm 0.3071 (+0.92z)| lr 2.94e-05 | 323.37 ms | 52.2% bf16 MFU | 1623416 tok/s
step 16881/19560 | loss 3.295683 (+0.07z)| norm 0.2600 (-0.77z)| lr 2.94e-05 | 322.89 ms | 52.3% bf16 MFU | 1623431 tok/s
step 16882/19560 | loss 3.240759 (-1.19z)| norm 0.2584 (-0.82z)| lr 2.94e-05 | 322.89 ms | 52.3% bf16 MFU | 1623445 tok/s
step 16883/19560 | loss 3.341768 (+1.14z)| norm 0.2862 (+0.19z)| lr 2.94e-05 | 323.44 ms | 52.2% bf16 MFU | 1623321 tok/s
step 16884/19560 | loss 3.253980 (-0.88z)| norm 0.2625 (-0.67z)| lr 2.93e-05 | 323.62 ms | 52.2% bf16 MFU | 1623160 tok/s
step 16885/19560 | loss 3.258138 (-0.77z)| norm 0.2869 (+0.24z)| lr 2.93e-05 | 322.87 ms | 52.3% bf16 MFU | 1623194 tok/s
step 16886/19560 | loss 3.271490 (-0.47z)| norm 0.2857 (+0.19z)| lr 2.93e-05 | 322.90 ms | 52.3% bf16 MFU | 1623219 tok/s
step 16887/19560 | loss 3.280939 (-0.26z)| norm 0.2773 (-0.12z)| lr 2.93e-05 | 323.10 ms | 52.2% bf16 MFU | 1623191 tok/s
step 16888/19560 | loss 3.306782 (+0.33z)| norm 0.2964 (+0.59z)| lr 2.92e-05 | 322.99 ms | 52.3% bf16 MFU | 1623192 tok/s
step 16889/19560 | loss 3.316284 (+0.57z)| norm 0.2965 (+0.59z)| lr 2.92e-05 | 322.96 ms | 52.3% bf16 MFU | 1623203 tok/s
step 16890/19560 | loss 3.342093 (+1.16z)| norm 0.2786 (-0.07z)| lr 2.92e-05 | 323.07 ms | 52.2% bf16 MFU | 1623185 tok/s
step 16891/19560 | loss 3.264574 (-0.65z)| norm 0.2680 (-0.46z)| lr 2.92e-05 | 322.73 ms | 52.3% bf16 MFU | 1623252 tok/s
step 16892/19560 | loss 3.245242 (-1.10z)| norm 0.2636 (-0.62z)| lr 2.92e-05 | 323.01 ms | 52.2% bf16 MFU | 1623245 tok/s
step 16893/19560 | loss 3.235922 (-1.30z)| norm 0.2709 (-0.35z)| lr 2.91e-05 | 322.80 ms | 52.3% bf16 MFU | 1623293 tok/s
step 16894/19560 | loss 3.274128 (-0.42z)| norm 0.2878 (+0.29z)| lr 2.91e-05 | 322.93 ms | 52.3% bf16 MFU | 1623305 tok/s
step 16895/19560 | loss 3.271613 (-0.47z)| norm 0.2604 (-0.72z)| lr 2.91e-05 | 323.12 ms | 52.2% bf16 MFU | 1623270 tok/s
step 16896/19560 | loss 3.342108 (+1.17z)| norm 0.2602 (-0.73z)| lr 2.91e-05 | 323.17 ms | 52.2% bf16 MFU | 1623222 tok/s
step 16897/19560 | loss 3.330101 (+0.89z)| norm 0.2770 (-0.11z)| lr 2.91e-05 | 322.58 ms | 52.3% bf16 MFU | 1623325 tok/s
step 16898/19560 | loss 3.278847 (-0.31z)| norm 0.2887 (+0.32z)| lr 2.90e-05 | 322.66 ms | 52.3% bf16 MFU | 1623403 tok/s
step 16899/19560 | loss 3.285707 (-0.14z)| norm 0.2614 (-0.70z)| lr 2.90e-05 | 322.83 ms | 52.3% bf16 MFU | 1623433 tok/s
step 16900/19560 | loss 3.340598 (+1.12z)| norm 0.2782 (-0.08z)| lr 2.90e-05 | 322.95 ms | 52.3% bf16 MFU | 1623434 tok/s
step 16901/19560 | loss 3.302666 (+0.25z)| norm 0.2889 (+0.32z)| lr 2.90e-05 | 322.32 ms | 52.4% bf16 MFU | 1623593 tok/s
step 16902/19560 | loss 3.271740 (-0.47z)| norm 0.2688 (-0.43z)| lr 2.89e-05 | 322.45 ms | 52.3% bf16 MFU | 1623710 tok/s
step 16903/19560 | loss 3.269018 (-0.53z)| norm 0.2708 (-0.35z)| lr 2.89e-05 | 323.15 ms | 52.2% bf16 MFU | 1623646 tok/s
step 16904/19560 | loss 3.286096 (-0.13z)| norm 0.3006 (+0.75z)| lr 2.89e-05 | 323.12 ms | 52.2% bf16 MFU | 1623594 tok/s
step 16905/19560 | loss 3.213816 (-1.77z)| norm 0.2676 (-0.50z)| lr 2.89e-05 | 322.62 ms | 52.3% bf16 MFU | 1623668 tok/s
step 16906/19560 | loss 3.309560 (+0.42z)| norm 0.2854 (+0.17z)| lr 2.89e-05 | 322.86 ms | 52.3% bf16 MFU | 1623680 tok/s
step 16907/19560 | loss 3.281371 (-0.23z)| norm 0.2538 (-1.01z)| lr 2.88e-05 | 322.64 ms | 52.3% bf16 MFU | 1623746 tok/s
step 16908/19560 | loss 3.283637 (-0.17z)| norm 0.2866 (+0.22z)| lr 2.88e-05 | 322.34 ms | 52.4% bf16 MFU | 1623884 tok/s
step 16909/19560 | loss 3.296359 (+0.11z)| norm 0.3022 (+0.80z)| lr 2.88e-05 | 322.89 ms | 52.3% bf16 MFU | 1623875 tok/s
step 16910/19560 | loss 3.244431 (-1.09z)| norm 0.2542 (-1.03z)| lr 2.88e-05 | 322.44 ms | 52.3% bf16 MFU | 1623982 tok/s
step 16911/19560 | loss 3.295084 (+0.08z)| norm 0.4193 (+4.75z)| lr 2.88e-05 | 322.67 ms | 52.3% bf16 MFU | 1624024 tok/s
step 16912/19560 | loss 3.262131 (-0.68z)| norm 0.2630 (-0.69z)| lr 2.87e-05 | 322.87 ms | 52.3% bf16 MFU | 1624015 tok/s
step 16913/19560 | loss 3.332422 (+0.97z)| norm 0.2975 (+0.51z)| lr 2.87e-05 | 322.63 ms | 52.3% bf16 MFU | 1624067 tok/s
step 16914/19560 | loss 3.265246 (-0.60z)| norm 0.2974 (+0.52z)| lr 2.87e-05 | 322.36 ms | 52.4% bf16 MFU | 1624183 tok/s
step 16915/19560 | loss 3.307707 (+0.40z)| norm 0.3030 (+0.70z)| lr 2.87e-05 | 322.27 ms | 52.4% bf16 MFU | 1624316 tok/s
step 16916/19560 | loss 3.229609 (-1.41z)| norm 0.2693 (-0.47z)| lr 2.86e-05 | 322.41 ms | 52.3% bf16 MFU | 1624408 tok/s
step 16917/19560 | loss 3.283763 (-0.16z)| norm 0.2924 (+0.34z)| lr 2.86e-05 | 322.63 ms | 52.3% bf16 MFU | 1624440 tok/s
step 16918/19560 | loss 3.282012 (-0.20z)| norm 0.2751 (-0.27z)| lr 2.86e-05 | 322.71 ms | 52.3% bf16 MFU | 1624451 tok/s
step 16919/19560 | loss 3.243258 (-1.10z)| norm 0.2431 (-1.39z)| lr 2.86e-05 | 322.73 ms | 52.3% bf16 MFU | 1624455 tok/s
step 16920/19560 | loss 3.288352 (-0.06z)| norm 0.2490 (-1.17z)| lr 2.86e-05 | 322.48 ms | 52.3% bf16 MFU | 1624522 tok/s
step 16921/19560 | loss 3.276731 (-0.33z)| norm 0.2477 (-1.19z)| lr 2.85e-05 | 322.82 ms | 52.3% bf16 MFU | 1624501 tok/s
step 16922/19560 | loss 3.317283 (+0.63z)| norm 0.2897 (+0.26z)| lr 2.85e-05 | 322.33 ms | 52.4% bf16 MFU | 1624603 tok/s
step 16923/19560 | loss 3.264728 (-0.60z)| norm 0.2511 (-1.07z)| lr 2.85e-05 | 322.85 ms | 52.3% bf16 MFU | 1624569 tok/s
step 16924/19560 | loss 3.281543 (-0.22z)| norm 0.3127 (+1.05z)| lr 2.85e-05 | 322.54 ms | 52.3% bf16 MFU | 1624616 tok/s
step 16925/19560 | loss 3.287445 (-0.09z)| norm 0.2546 (-0.95z)| lr 2.85e-05 | 322.70 ms | 52.3% bf16 MFU | 1624620 tok/s
step 16926/19560 | loss 3.201544 (-2.12z)| norm 0.2673 (-0.51z)| lr 2.84e-05 | 322.81 ms | 52.3% bf16 MFU | 1624597 tok/s
step 16927/19560 | loss 3.231972 (-1.37z)| norm 0.2561 (-0.89z)| lr 2.84e-05 | 322.56 ms | 52.3% bf16 MFU | 1624637 tok/s
step 16928/19560 | loss 3.326386 (+0.86z)| norm 0.3231 (+1.43z)| lr 2.84e-05 | 322.46 ms | 52.3% bf16 MFU | 1624699 tok/s
step 16929/19560 | loss 3.322318 (+0.75z)| norm 0.2659 (-0.55z)| lr 2.84e-05 | 322.87 ms | 52.3% bf16 MFU | 1624657 tok/s
step 16930/19560 | loss 3.273261 (-0.42z)| norm 0.2846 (+0.10z)| lr 2.84e-05 | 322.98 ms | 52.3% bf16 MFU | 1624589 tok/s
step 16931/19560 | loss 3.174431 (-2.70z)| norm 0.2881 (+0.22z)| lr 2.83e-05 | 322.74 ms | 52.3% bf16 MFU | 1624584 tok/s
step 16932/19560 | loss 3.280870 (-0.20z)| norm 0.2707 (-0.38z)| lr 2.83e-05 | 322.58 ms | 52.3% bf16 MFU | 1624619 tok/s
step 16933/19560 | loss 3.252079 (-0.90z)| norm 0.2591 (-0.78z)| lr 2.83e-05 | 323.05 ms | 52.2% bf16 MFU | 1624534 tok/s
step 16934/19560 | loss 3.228416 (-1.45z)| norm 0.3278 (+1.58z)| lr 2.83e-05 | 322.64 ms | 52.3% bf16 MFU | 1624556 tok/s
step 16935/19560 | loss 3.261342 (-0.66z)| norm 0.2750 (-0.23z)| lr 2.82e-05 | 322.97 ms | 52.3% bf16 MFU | 1624494 tok/s
step 16936/19560 | loss 3.268888 (-0.47z)| norm 0.2768 (-0.16z)| lr 2.82e-05 | 322.60 ms | 52.3% bf16 MFU | 1624530 tok/s
step 16937/19560 | loss 3.319880 (+0.80z)| norm 0.2912 (+0.34z)| lr 2.82e-05 | 322.69 ms | 52.3% bf16 MFU | 1624540 tok/s
step 16938/19560 | loss 3.348206 (+1.49z)| norm 0.2740 (-0.26z)| lr 2.82e-05 | 322.70 ms | 52.3% bf16 MFU | 1624549 tok/s
step 16939/19560 | loss 3.294881 (+0.16z)| norm 0.3045 (+0.79z)| lr 2.82e-05 | 322.92 ms | 52.3% bf16 MFU | 1624501 tok/s
step 16940/19560 | loss 3.349789 (+1.51z)| norm 0.2438 (-1.30z)| lr 2.81e-05 | 322.90 ms | 52.3% bf16 MFU | 1624460 tok/s
step 16941/19560 | loss 3.218041 (-1.71z)| norm 0.2854 (+0.15z)| lr 2.81e-05 | 322.53 ms | 52.3% bf16 MFU | 1624515 tok/s
step 16942/19560 | loss 3.312440 (+0.61z)| norm 0.2581 (-0.78z)| lr 2.81e-05 | 322.63 ms | 52.3% bf16 MFU | 1624541 tok/s
step 16943/19560 | loss 3.305122 (+0.43z)| norm 0.2514 (-1.00z)| lr 2.81e-05 | 323.04 ms | 52.2% bf16 MFU | 1624463 tok/s
step 16944/19560 | loss 3.223235 (-1.56z)| norm 0.3512 (+2.41z)| lr 2.81e-05 | 322.81 ms | 52.3% bf16 MFU | 1624446 tok/s
step 16945/19560 | loss 3.286501 (-0.02z)| norm 0.2537 (-0.93z)| lr 2.80e-05 | 322.42 ms | 52.3% bf16 MFU | 1624530 tok/s
step 16946/19560 | loss 3.260054 (-0.66z)| norm 0.2820 (+0.04z)| lr 2.80e-05 | 322.78 ms | 52.3% bf16 MFU | 1624519 tok/s
step 16947/19560 | loss 3.263941 (-0.57z)| norm 0.3135 (+1.10z)| lr 2.80e-05 | 322.65 ms | 52.3% bf16 MFU | 1624541 tok/s
step 16948/19560 | loss 3.286733 (+0.01z)| norm 0.2892 (+0.27z)| lr 2.80e-05 | 322.94 ms | 52.3% bf16 MFU | 1624487 tok/s
step 16949/19560 | loss 3.304066 (+0.45z)| norm 0.2898 (+0.28z)| lr 2.80e-05 | 322.62 ms | 52.3% bf16 MFU | 1624518 tok/s
step 16950/19560 | loss 3.265019 (-0.54z)| norm 0.2623 (-0.65z)| lr 2.79e-05 | 322.99 ms | 52.3% bf16 MFU | 1624454 tok/s
step 16951/19560 | loss 3.251135 (-0.88z)| norm 0.2719 (-0.32z)| lr 2.79e-05 | 322.31 ms | 52.4% bf16 MFU | 1624564 tok/s
step 16952/19560 | loss 3.294469 (+0.23z)| norm 0.2543 (-0.92z)| lr 2.79e-05 | 322.29 ms | 52.4% bf16 MFU | 1624674 tok/s
step 16953/19560 | loss 3.275650 (-0.25z)| norm 0.3845 (+3.38z)| lr 2.79e-05 | 322.66 ms | 52.3% bf16 MFU | 1624684 tok/s
step 16954/19560 | loss 3.327053 (+1.06z)| norm 0.2444 (-1.21z)| lr 2.78e-05 | 322.99 ms | 52.3% bf16 MFU | 1624611 tok/s
step 16955/19560 | loss 3.303129 (+0.44z)| norm 0.3479 (+2.13z)| lr 2.78e-05 | 322.27 ms | 52.4% bf16 MFU | 1624724 tok/s
step 16956/19560 | loss 3.305569 (+0.49z)| norm 0.2773 (-0.15z)| lr 2.78e-05 | 322.75 ms | 52.3% bf16 MFU | 1624709 tok/s
step 16957/19560 | loss 3.241556 (-1.13z)| norm 0.2864 (+0.14z)| lr 2.78e-05 | 322.59 ms | 52.3% bf16 MFU | 1624736 tok/s
step 16958/19560 | loss 3.356787 (+1.76z)| norm 0.3411 (+1.92z)| lr 2.78e-05 | 322.55 ms | 52.3% bf16 MFU | 1624772 tok/s
step 16959/19560 | loss 3.265430 (-0.54z)| norm 0.3107 (+0.91z)| lr 2.77e-05 | 322.76 ms | 52.3% bf16 MFU | 1624752 tok/s
step 16960/19560 | loss 3.265310 (-0.53z)| norm 0.2863 (+0.12z)| lr 2.77e-05 | 323.07 ms | 52.2% bf16 MFU | 1624656 tok/s
step 16961/19560 | loss 3.342945 (+1.40z)| norm 0.3127 (+0.96z)| lr 2.77e-05 | 322.68 ms | 52.3% bf16 MFU | 1624664 tok/s
step 16962/19560 | loss 3.284608 (-0.04z)| norm 0.2500 (-1.05z)| lr 2.77e-05 | 322.40 ms | 52.3% bf16 MFU | 1624740 tok/s
step 16963/19560 | loss 3.208146 (-1.98z)| norm 0.2494 (-1.07z)| lr 2.77e-05 | 322.67 ms | 52.3% bf16 MFU | 1624745 tok/s
step 16964/19560 | loss 3.275563 (-0.27z)| norm 0.2722 (-0.34z)| lr 2.76e-05 | 322.86 ms | 52.3% bf16 MFU | 1624701 tok/s
step 16965/19560 | loss 3.264453 (-0.55z)| norm 0.2545 (-0.91z)| lr 2.76e-05 | 322.97 ms | 52.3% bf16 MFU | 1624633 tok/s
step 16966/19560 | loss 3.293572 (+0.20z)| norm 0.2436 (-1.25z)| lr 2.76e-05 | 322.67 ms | 52.3% bf16 MFU | 1624643 tok/s
step 16967/19560 | loss 3.389595 (+2.56z)| norm 0.2678 (-0.46z)| lr 2.76e-05 | 322.54 ms | 52.3% bf16 MFU | 1624686 tok/s
step 16968/19560 | loss 3.298540 (+0.28z)| norm 0.2463 (-1.15z)| lr 2.76e-05 | 322.74 ms | 52.3% bf16 MFU | 1624676 tok/s
step 16969/19560 | loss 3.460010 (+3.99z)| norm 0.3144 (+1.03z)| lr 2.75e-05 | 322.59 ms | 52.3% bf16 MFU | 1624704 tok/s
step 16970/19560 | loss 3.344507 (+1.29z)| norm 0.2476 (-1.12z)| lr 2.75e-05 | 322.73 ms | 52.3% bf16 MFU | 1624696 tok/s
step 16971/19560 | loss 3.324063 (+0.81z)| norm 0.2506 (-1.02z)| lr 2.75e-05 | 322.58 ms | 52.3% bf16 MFU | 1624726 tok/s
step 16972/19560 | loss 3.320098 (+0.72z)| norm 0.2687 (-0.45z)| lr 2.75e-05 | 322.68 ms | 52.3% bf16 MFU | 1624731 tok/s
step 16973/19560 | loss 3.241669 (-1.10z)| norm 0.2413 (-1.32z)| lr 2.74e-05 | 322.69 ms | 52.3% bf16 MFU | 1624730 tok/s
step 16974/19560 | loss 3.245759 (-1.03z)| norm 0.2815 (-0.03z)| lr 2.74e-05 | 322.71 ms | 52.3% bf16 MFU | 1624726 tok/s
step 16975/19560 | loss 3.231895 (-1.34z)| norm 0.3164 (+1.07z)| lr 2.74e-05 | 322.39 ms | 52.3% bf16 MFU | 1624802 tok/s
step 16976/19560 | loss 3.285879 (-0.08z)| norm 0.2442 (-1.24z)| lr 2.74e-05 | 322.81 ms | 52.3% bf16 MFU | 1624768 tok/s
step 16977/19560 | loss 3.318507 (+0.69z)| norm 0.3158 (+1.04z)| lr 2.74e-05 | 322.63 ms | 52.3% bf16 MFU | 1624781 tok/s
step 16978/19560 | loss 3.321279 (+0.76z)| norm 0.2735 (-0.32z)| lr 2.73e-05 | 323.00 ms | 52.3% bf16 MFU | 1624701 tok/s
step 16979/19560 | loss 3.288300 (-0.02z)| norm 0.2480 (-1.14z)| lr 2.73e-05 | 322.98 ms | 52.3% bf16 MFU | 1624629 tok/s
step 16980/19560 | loss 3.249969 (-0.93z)| norm 0.3168 (+1.12z)| lr 2.73e-05 | 322.29 ms | 52.4% bf16 MFU | 1624735 tok/s
step 16981/19560 | loss 3.324527 (+0.88z)| norm 0.3181 (+1.14z)| lr 2.73e-05 | 322.61 ms | 52.3% bf16 MFU | 1624754 tok/s
step 16982/19560 | loss 3.258425 (-0.72z)| norm 0.2815 (-0.06z)| lr 2.73e-05 | 322.66 ms | 52.3% bf16 MFU | 1624762 tok/s
step 16983/19560 | loss 3.284637 (-0.09z)| norm 0.3744 (+2.87z)| lr 2.72e-05 | 322.25 ms | 52.4% bf16 MFU | 1624871 tok/s
step 16984/19560 | loss 3.384253 (+2.28z)| norm 0.2599 (-0.77z)| lr 2.72e-05 | 322.59 ms | 52.3% bf16 MFU | 1624891 tok/s
step 16985/19560 | loss 3.295795 (+0.17z)| norm 0.3049 (+0.66z)| lr 2.72e-05 | 322.71 ms | 52.3% bf16 MFU | 1624879 tok/s
step 16986/19560 | loss 3.351488 (+1.50z)| norm 0.2677 (-0.52z)| lr 2.72e-05 | 322.16 ms | 52.4% bf16 MFU | 1625007 tok/s
step 16987/19560 | loss 3.268316 (-0.48z)| norm 0.2686 (-0.49z)| lr 2.72e-05 | 322.80 ms | 52.3% bf16 MFU | 1624966 tok/s
step 16988/19560 | loss 3.336197 (+1.14z)| norm 0.2656 (-0.57z)| lr 2.71e-05 | 322.73 ms | 52.3% bf16 MFU | 1624944 tok/s
step 16989/19560 | loss 3.253224 (-0.83z)| norm 0.2570 (-0.85z)| lr 2.71e-05 | 322.73 ms | 52.3% bf16 MFU | 1624925 tok/s
step 16990/19560 | loss 3.252377 (-0.84z)| norm 0.2809 (-0.08z)| lr 2.71e-05 | 322.18 ms | 52.4% bf16 MFU | 1625043 tok/s
step 16991/19560 | loss 3.275214 (-0.31z)| norm 0.2831 (+0.00z)| lr 2.71e-05 | 322.68 ms | 52.3% bf16 MFU | 1625031 tok/s
step 16992/19560 | loss 3.241626 (-1.12z)| norm 0.2755 (-0.24z)| lr 2.71e-05 | 322.57 ms | 52.3% bf16 MFU | 1625048 tok/s
step 16993/19560 | loss 3.317876 (+0.72z)| norm 0.3022 (+0.65z)| lr 2.70e-05 | 323.11 ms | 52.2% bf16 MFU | 1624926 tok/s
step 16994/19560 | loss 3.337505 (+1.18z)| norm 0.3471 (+2.10z)| lr 2.70e-05 | 322.76 ms | 52.3% bf16 MFU | 1624899 tok/s
step 16995/19560 | loss 3.316359 (+0.68z)| norm 0.2777 (-0.18z)| lr 2.70e-05 | 322.76 ms | 52.3% bf16 MFU | 1624873 tok/s
step 16996/19560 | loss 3.274448 (-0.34z)| norm 0.3371 (+1.80z)| lr 2.70e-05 | 322.09 ms | 52.4% bf16 MFU | 1625019 tok/s
step 16997/19560 | loss 3.242675 (-1.10z)| norm 0.3492 (+2.15z)| lr 2.69e-05 | 322.31 ms | 52.4% bf16 MFU | 1625101 tok/s
step 16998/19560 | loss 3.354915 (+1.59z)| norm 0.2872 (+0.13z)| lr 2.69e-05 | 322.77 ms | 52.3% bf16 MFU | 1625064 tok/s
step 16999/19560 | loss 3.252008 (-0.88z)| norm 0.2812 (-0.06z)| lr 2.69e-05 | 322.77 ms | 52.3% bf16 MFU | 1625026 tok/s
step 17000/19560 | loss 3.348119 (+1.40z)| norm 0.3150 (+1.03z)| lr 2.69e-05 | 322.61 ms | 52.3% bf16 MFU | 1625032 tok/s
val loss 3.282649
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3036/10042 = 0.302330
step 17001/19560 | loss 3.314213 (+0.60z)| norm 0.2971 (+0.45z)| lr 2.69e-05 | 322.33 ms | 52.4% bf16 MFU | 1625107 tok/s
step 17002/19560 | loss 3.282849 (-0.14z)| norm 0.2668 (-0.54z)| lr 2.68e-05 | 322.47 ms | 52.3% bf16 MFU | 1625143 tok/s
step 17003/19560 | loss 3.441029 (+3.48z)| norm 0.2616 (-0.71z)| lr 2.68e-05 | 322.88 ms | 52.3% bf16 MFU | 1625074 tok/s
step 17004/19560 | loss 3.295593 (+0.14z)| norm 0.3483 (+2.08z)| lr 2.68e-05 | 322.86 ms | 52.3% bf16 MFU | 1625016 tok/s
step 17005/19560 | loss 3.328510 (+0.90z)| norm 0.3072 (+0.76z)| lr 2.68e-05 | 322.86 ms | 52.3% bf16 MFU | 1624960 tok/s
step 17006/19560 | loss 3.287755 (-0.03z)| norm 0.3012 (+0.56z)| lr 2.68e-05 | 322.95 ms | 52.3% bf16 MFU | 1624882 tok/s
step 17007/19560 | loss 3.262697 (-0.60z)| norm 0.2901 (+0.20z)| lr 2.67e-05 | 322.78 ms | 52.3% bf16 MFU | 1624852 tok/s
step 17008/19560 | loss 3.285554 (-0.07z)| norm 0.2923 (+0.28z)| lr 2.67e-05 | 322.73 ms | 52.3% bf16 MFU | 1624838 tok/s
step 17009/19560 | loss 3.238905 (-1.13z)| norm 0.3176 (+1.08z)| lr 2.67e-05 | 322.72 ms | 52.3% bf16 MFU | 1624825 tok/s
step 17010/19560 | loss 3.305457 (+0.38z)| norm 0.2791 (-0.17z)| lr 2.67e-05 | 322.92 ms | 52.3% bf16 MFU | 1624762 tok/s
step 17011/19560 | loss 3.254641 (-0.77z)| norm 0.2478 (-1.17z)| lr 2.67e-05 | 322.81 ms | 52.3% bf16 MFU | 1624730 tok/s
step 17012/19560 | loss 3.258979 (-0.68z)| norm 0.2754 (-0.28z)| lr 2.66e-05 | 322.84 ms | 52.3% bf16 MFU | 1624692 tok/s
step 17013/19560 | loss 3.299139 (+0.25z)| norm 0.2820 (-0.07z)| lr 2.66e-05 | 323.02 ms | 52.2% bf16 MFU | 1624610 tok/s
step 17014/19560 | loss 3.239089 (-1.13z)| norm 0.2526 (-1.01z)| lr 2.66e-05 | 322.81 ms | 52.3% bf16 MFU | 1624588 tok/s
step 17015/19560 | loss 3.276254 (-0.28z)| norm 0.2827 (-0.04z)| lr 2.66e-05 | 323.00 ms | 52.3% bf16 MFU | 1624517 tok/s
step 17016/19560 | loss 3.295315 (+0.17z)| norm 0.2763 (-0.24z)| lr 2.66e-05 | 323.22 ms | 52.2% bf16 MFU | 1624395 tok/s
step 17017/19560 | loss 3.305009 (+0.39z)| norm 0.3021 (+0.59z)| lr 2.65e-05 | 322.55 ms | 52.3% bf16 MFU | 1624448 tok/s
step 17018/19560 | loss 3.275978 (-0.27z)| norm 0.2801 (-0.12z)| lr 2.65e-05 | 322.70 ms | 52.3% bf16 MFU | 1624460 tok/s
step 17019/19560 | loss 3.267152 (-0.47z)| norm 0.3323 (+1.53z)| lr 2.65e-05 | 322.88 ms | 52.3% bf16 MFU | 1624425 tok/s
step 17020/19560 | loss 3.279873 (-0.18z)| norm 0.2850 (+0.02z)| lr 2.65e-05 | 322.77 ms | 52.3% bf16 MFU | 1624420 tok/s
step 17021/19560 | loss 3.334585 (+1.08z)| norm 0.2558 (-0.91z)| lr 2.65e-05 | 322.95 ms | 52.3% bf16 MFU | 1624371 tok/s
step 17022/19560 | loss 3.274843 (-0.32z)| norm 0.3572 (+2.26z)| lr 2.64e-05 | 322.48 ms | 52.3% bf16 MFU | 1624442 tok/s
step 17023/19560 | loss 3.372851 (+1.93z)| norm 0.3027 (+0.55z)| lr 2.64e-05 | 322.66 ms | 52.3% bf16 MFU | 1624464 tok/s
step 17024/19560 | loss 3.253300 (-0.82z)| norm 0.2562 (-0.91z)| lr 2.64e-05 | 322.98 ms | 52.3% bf16 MFU | 1624403 tok/s
step 17025/19560 | loss 3.289426 (+0.03z)| norm 0.3408 (+1.70z)| lr 2.64e-05 | 323.00 ms | 52.3% bf16 MFU | 1624342 tok/s
step 17026/19560 | loss 3.247413 (-0.94z)| norm 0.2811 (-0.14z)| lr 2.64e-05 | 323.10 ms | 52.2% bf16 MFU | 1624258 tok/s
step 17027/19560 | loss 3.341336 (+1.22z)| norm 0.2596 (-0.80z)| lr 2.63e-05 | 322.85 ms | 52.3% bf16 MFU | 1624242 tok/s
step 17028/19560 | loss 3.178657 (-2.45z)| norm 0.3291 (+1.33z)| lr 2.63e-05 | 322.36 ms | 52.4% bf16 MFU | 1624350 tok/s
step 17029/19560 | loss 3.324472 (+0.83z)| norm 0.3520 (+1.98z)| lr 2.63e-05 | 323.00 ms | 52.3% bf16 MFU | 1624292 tok/s
step 17030/19560 | loss 3.269155 (-0.41z)| norm 0.2855 (-0.03z)| lr 2.63e-05 | 323.09 ms | 52.2% bf16 MFU | 1624214 tok/s
step 17031/19560 | loss 3.402013 (+2.50z)| norm 0.3099 (+0.70z)| lr 2.62e-05 | 322.68 ms | 52.3% bf16 MFU | 1624242 tok/s
step 17032/19560 | loss 3.349601 (+1.32z)| norm 0.3325 (+1.36z)| lr 2.62e-05 | 322.34 ms | 52.4% bf16 MFU | 1624356 tok/s
step 17033/19560 | loss 3.307264 (+0.39z)| norm 0.2566 (-0.91z)| lr 2.62e-05 | 322.67 ms | 52.3% bf16 MFU | 1624379 tok/s
step 17034/19560 | loss 3.240538 (-1.07z)| norm 0.2583 (-0.85z)| lr 2.62e-05 | 322.88 ms | 52.3% bf16 MFU | 1624348 tok/s
step 17035/19560 | loss 3.296966 (+0.17z)| norm 0.2900 (+0.09z)| lr 2.62e-05 | 323.68 ms | 52.1% bf16 MFU | 1624119 tok/s
step 17036/19560 | loss 3.299164 (+0.22z)| norm 0.2809 (-0.18z)| lr 2.61e-05 | 322.73 ms | 52.3% bf16 MFU | 1624140 tok/s
step 17037/19560 | loss 3.269113 (-0.44z)| norm 0.2750 (-0.36z)| lr 2.61e-05 | 322.53 ms | 52.3% bf16 MFU | 1624209 tok/s
step 17038/19560 | loss 3.356388 (+1.45z)| norm 0.2453 (-1.24z)| lr 2.61e-05 | 323.29 ms | 52.2% bf16 MFU | 1624085 tok/s
step 17039/19560 | loss 3.272774 (-0.37z)| norm 0.2835 (-0.07z)| lr 2.61e-05 | 322.70 ms | 52.3% bf16 MFU | 1624115 tok/s
step 17040/19560 | loss 3.230104 (-1.29z)| norm 0.2623 (-0.75z)| lr 2.61e-05 | 323.44 ms | 52.2% bf16 MFU | 1623958 tok/s
step 17041/19560 | loss 3.316009 (+0.58z)| norm 0.2443 (-1.30z)| lr 2.60e-05 | 322.98 ms | 52.3% bf16 MFU | 1623925 tok/s
step 17042/19560 | loss 3.281206 (-0.18z)| norm 0.2799 (-0.17z)| lr 2.60e-05 | 322.87 ms | 52.3% bf16 MFU | 1623921 tok/s
step 17043/19560 | loss 3.297557 (+0.18z)| norm 0.2573 (-0.87z)| lr 2.60e-05 | 322.99 ms | 52.3% bf16 MFU | 1623887 tok/s
step 17044/19560 | loss 3.290103 (+0.00z)| norm 0.2457 (-1.23z)| lr 2.60e-05 | 322.94 ms | 52.3% bf16 MFU | 1623867 tok/s
step 17045/19560 | loss 3.315735 (+0.56z)| norm 0.2569 (-0.86z)| lr 2.60e-05 | 323.26 ms | 52.2% bf16 MFU | 1623768 tok/s
step 17046/19560 | loss 3.255708 (-0.75z)| norm 0.2573 (-0.85z)| lr 2.59e-05 | 322.26 ms | 52.4% bf16 MFU | 1623924 tok/s
step 17047/19560 | loss 3.266304 (-0.52z)| norm 0.2644 (-0.63z)| lr 2.59e-05 | 323.28 ms | 52.2% bf16 MFU | 1623817 tok/s
step 17048/19560 | loss 3.273053 (-0.37z)| norm 0.2509 (-1.06z)| lr 2.59e-05 | 322.88 ms | 52.3% bf16 MFU | 1623816 tok/s
step 17049/19560 | loss 3.253327 (-0.80z)| norm 0.2676 (-0.54z)| lr 2.59e-05 | 322.50 ms | 52.3% bf16 MFU | 1623910 tok/s
step 17050/19560 | loss 3.263214 (-0.58z)| norm 0.2722 (-0.39z)| lr 2.59e-05 | 323.29 ms | 52.2% bf16 MFU | 1623801 tok/s
step 17051/19560 | loss 3.272166 (-0.38z)| norm 0.2701 (-0.46z)| lr 2.58e-05 | 322.45 ms | 52.3% bf16 MFU | 1623908 tok/s
step 17052/19560 | loss 3.244655 (-0.97z)| norm 0.2544 (-0.94z)| lr 2.58e-05 | 322.73 ms | 52.3% bf16 MFU | 1623938 tok/s
step 17053/19560 | loss 3.293125 (+0.08z)| norm 0.2679 (-0.52z)| lr 2.58e-05 | 323.17 ms | 52.2% bf16 MFU | 1623857 tok/s
step 17054/19560 | loss 3.313490 (+0.52z)| norm 0.2606 (-0.75z)| lr 2.58e-05 | 322.81 ms | 52.3% bf16 MFU | 1623871 tok/s
step 17055/19560 | loss 3.264741 (-0.57z)| norm 0.2540 (-0.96z)| lr 2.58e-05 | 323.24 ms | 52.2% bf16 MFU | 1623777 tok/s
step 17056/19560 | loss 3.274300 (-0.35z)| norm 0.2493 (-1.09z)| lr 2.57e-05 | 322.68 ms | 52.3% bf16 MFU | 1623827 tok/s
step 17057/19560 | loss 3.283100 (-0.15z)| norm 0.2766 (-0.23z)| lr 2.57e-05 | 323.00 ms | 52.3% bf16 MFU | 1623795 tok/s
step 17058/19560 | loss 3.289454 (-0.01z)| norm 0.2515 (-1.01z)| lr 2.57e-05 | 323.55 ms | 52.2% bf16 MFU | 1623626 tok/s
step 17059/19560 | loss 3.343736 (+1.20z)| norm 0.2578 (-0.81z)| lr 2.57e-05 | 322.69 ms | 52.3% bf16 MFU | 1623682 tok/s
step 17060/19560 | loss 3.297201 (+0.13z)| norm 0.2797 (-0.11z)| lr 2.57e-05 | 322.95 ms | 52.3% bf16 MFU | 1623669 tok/s
step 17061/19560 | loss 3.303451 (+0.27z)| norm 0.2723 (-0.35z)| lr 2.56e-05 | 322.80 ms | 52.3% bf16 MFU | 1623694 tok/s
step 17062/19560 | loss 3.321055 (+0.66z)| norm 0.3425 (+1.87z)| lr 2.56e-05 | 323.22 ms | 52.2% bf16 MFU | 1623613 tok/s
step 17063/19560 | loss 3.274750 (-0.41z)| norm 0.2583 (-0.79z)| lr 2.56e-05 | 323.13 ms | 52.2% bf16 MFU | 1623559 tok/s
step 17064/19560 | loss 3.315272 (+0.52z)| norm 0.3117 (+0.88z)| lr 2.56e-05 | 323.43 ms | 52.2% bf16 MFU | 1623432 tok/s
step 17065/19560 | loss 3.403197 (+2.47z)| norm 0.2898 (+0.19z)| lr 2.56e-05 | 323.42 ms | 52.2% bf16 MFU | 1623313 tok/s
step 17066/19560 | loss 3.306365 (+0.30z)| norm 0.2789 (-0.15z)| lr 2.55e-05 | 322.75 ms | 52.3% bf16 MFU | 1623369 tok/s
step 17067/19560 | loss 3.301741 (+0.19z)| norm 0.2633 (-0.63z)| lr 2.55e-05 | 322.88 ms | 52.3% bf16 MFU | 1623390 tok/s
step 17068/19560 | loss 3.238950 (-1.21z)| norm 0.2598 (-0.74z)| lr 2.55e-05 | 323.23 ms | 52.2% bf16 MFU | 1623323 tok/s
step 17069/19560 | loss 3.255427 (-0.85z)| norm 0.2620 (-0.67z)| lr 2.55e-05 | 323.31 ms | 52.2% bf16 MFU | 1623238 tok/s
step 17070/19560 | loss 3.345162 (+1.19z)| norm 0.2532 (-0.94z)| lr 2.55e-05 | 323.23 ms | 52.2% bf16 MFU | 1623178 tok/s
step 17071/19560 | loss 3.293936 (+0.02z)| norm 0.2733 (-0.32z)| lr 2.54e-05 | 322.91 ms | 52.3% bf16 MFU | 1623201 tok/s
step 17072/19560 | loss 3.262367 (-0.71z)| norm 0.2595 (-0.74z)| lr 2.54e-05 | 323.54 ms | 52.2% bf16 MFU | 1623064 tok/s
step 17073/19560 | loss 3.239794 (-1.21z)| norm 0.2555 (-0.87z)| lr 2.54e-05 | 322.75 ms | 52.3% bf16 MFU | 1623132 tok/s
step 17074/19560 | loss 3.281126 (-0.27z)| norm 0.2644 (-0.58z)| lr 2.54e-05 | 323.87 ms | 52.1% bf16 MFU | 1622917 tok/s
step 17075/19560 | loss 3.241672 (-1.17z)| norm 0.3102 (+0.89z)| lr 2.54e-05 | 323.17 ms | 52.2% bf16 MFU | 1622888 tok/s
step 17076/19560 | loss 3.292370 (-0.01z)| norm 0.2458 (-1.16z)| lr 2.53e-05 | 322.60 ms | 52.3% bf16 MFU | 1623004 tok/s
step 17077/19560 | loss 3.242775 (-1.12z)| norm 0.2658 (-0.52z)| lr 2.53e-05 | 322.90 ms | 52.3% bf16 MFU | 1623038 tok/s
step 17078/19560 | loss 3.278851 (-0.31z)| norm 0.3225 (+1.27z)| lr 2.53e-05 | 323.20 ms | 52.2% bf16 MFU | 1622996 tok/s
step 17079/19560 | loss 3.262624 (-0.68z)| norm 0.2844 (+0.06z)| lr 2.53e-05 | 323.42 ms | 52.2% bf16 MFU | 1622899 tok/s
step 17080/19560 | loss 3.245156 (-1.06z)| norm 0.2570 (-0.81z)| lr 2.53e-05 | 322.75 ms | 52.3% bf16 MFU | 1622975 tok/s
step 17081/19560 | loss 3.318777 (+0.59z)| norm 0.3084 (+0.88z)| lr 2.52e-05 | 322.55 ms | 52.3% bf16 MFU | 1623098 tok/s
step 17082/19560 | loss 3.265798 (-0.59z)| norm 0.2843 (+0.07z)| lr 2.52e-05 | 322.65 ms | 52.3% bf16 MFU | 1623190 tok/s
step 17083/19560 | loss 3.269345 (-0.51z)| norm 0.2672 (-0.49z)| lr 2.52e-05 | 322.43 ms | 52.3% bf16 MFU | 1623334 tok/s
step 17084/19560 | loss 3.287759 (-0.09z)| norm 0.3545 (+2.39z)| lr 2.52e-05 | 322.73 ms | 52.3% bf16 MFU | 1623393 tok/s
step 17085/19560 | loss 3.320575 (+0.64z)| norm 0.2873 (+0.17z)| lr 2.52e-05 | 322.95 ms | 52.3% bf16 MFU | 1623394 tok/s
step 17086/19560 | loss 3.270534 (-0.48z)| norm 0.2976 (+0.53z)| lr 2.51e-05 | 322.87 ms | 52.3% bf16 MFU | 1623418 tok/s
step 17087/19560 | loss 3.260152 (-0.72z)| norm 0.2751 (-0.22z)| lr 2.51e-05 | 322.14 ms | 52.4% bf16 MFU | 1623622 tok/s
step 17088/19560 | loss 3.225769 (-1.49z)| norm 0.2623 (-0.65z)| lr 2.51e-05 | 322.82 ms | 52.3% bf16 MFU | 1623644 tok/s
step 17089/19560 | loss 3.302961 (+0.28z)| norm 0.2643 (-0.57z)| lr 2.51e-05 | 322.46 ms | 52.3% bf16 MFU | 1623758 tok/s
step 17090/19560 | loss 3.269382 (-0.49z)| norm 0.2591 (-0.75z)| lr 2.51e-05 | 322.47 ms | 52.3% bf16 MFU | 1623863 tok/s
step 17091/19560 | loss 3.300880 (+0.22z)| norm 0.2590 (-0.75z)| lr 2.50e-05 | 323.12 ms | 52.2% bf16 MFU | 1623799 tok/s
step 17092/19560 | loss 3.223337 (-1.55z)| norm 0.2805 (-0.03z)| lr 2.50e-05 | 322.46 ms | 52.3% bf16 MFU | 1623903 tok/s
step 17093/19560 | loss 3.267263 (-0.55z)| norm 0.2439 (-1.26z)| lr 2.50e-05 | 322.65 ms | 52.3% bf16 MFU | 1623956 tok/s
step 17094/19560 | loss 3.297924 (+0.16z)| norm 0.2647 (-0.57z)| lr 2.50e-05 | 322.53 ms | 52.3% bf16 MFU | 1624035 tok/s
step 17095/19560 | loss 3.282973 (-0.17z)| norm 0.2731 (-0.28z)| lr 2.50e-05 | 322.65 ms | 52.3% bf16 MFU | 1624079 tok/s
step 17096/19560 | loss 3.285148 (-0.12z)| norm 0.2602 (-0.73z)| lr 2.49e-05 | 322.61 ms | 52.3% bf16 MFU | 1624133 tok/s
step 17097/19560 | loss 3.292131 (+0.08z)| norm 0.3368 (+1.87z)| lr 2.49e-05 | 322.87 ms | 52.3% bf16 MFU | 1624119 tok/s
step 17098/19560 | loss 3.268462 (-0.50z)| norm 0.2667 (-0.51z)| lr 2.49e-05 | 322.60 ms | 52.3% bf16 MFU | 1624174 tok/s
step 17099/19560 | loss 3.290037 (+0.05z)| norm 0.3175 (+1.19z)| lr 2.49e-05 | 322.52 ms | 52.3% bf16 MFU | 1624246 tok/s
step 17100/19560 | loss 3.267101 (-0.52z)| norm 0.2852 (+0.09z)| lr 2.49e-05 | 322.40 ms | 52.3% bf16 MFU | 1624344 tok/s
step 17101/19560 | loss 3.284747 (-0.08z)| norm 0.2840 (+0.04z)| lr 2.48e-05 | 323.21 ms | 52.2% bf16 MFU | 1624233 tok/s
step 17102/19560 | loss 3.287952 (-0.01z)| norm 0.2695 (-0.45z)| lr 2.48e-05 | 322.43 ms | 52.3% bf16 MFU | 1624324 tok/s
step 17103/19560 | loss 3.281151 (-0.19z)| norm 0.2509 (-1.07z)| lr 2.48e-05 | 322.70 ms | 52.3% bf16 MFU | 1624342 tok/s
step 17104/19560 | loss 3.303829 (+0.39z)| norm 0.2954 (+0.44z)| lr 2.48e-05 | 322.55 ms | 52.3% bf16 MFU | 1624399 tok/s
step 17105/19560 | loss 3.254376 (-0.87z)| norm 0.2664 (-0.55z)| lr 2.48e-05 | 322.49 ms | 52.3% bf16 MFU | 1624466 tok/s
step 17106/19560 | loss 3.278880 (-0.24z)| norm 0.2884 (+0.21z)| lr 2.47e-05 | 322.47 ms | 52.3% bf16 MFU | 1624536 tok/s
step 17107/19560 | loss 3.264317 (-0.60z)| norm 0.2613 (-0.74z)| lr 2.47e-05 | 322.38 ms | 52.4% bf16 MFU | 1624624 tok/s
step 17108/19560 | loss 3.284691 (-0.09z)| norm 0.2471 (-1.21z)| lr 2.47e-05 | 322.73 ms | 52.3% bf16 MFU | 1624620 tok/s
step 17109/19560 | loss 3.283211 (-0.12z)| norm 0.2541 (-0.95z)| lr 2.47e-05 | 322.63 ms | 52.3% bf16 MFU | 1624640 tok/s
step 17110/19560 | loss 3.286036 (-0.05z)| norm 0.2513 (-1.03z)| lr 2.47e-05 | 322.98 ms | 52.3% bf16 MFU | 1624573 tok/s
step 17111/19560 | loss 3.280546 (-0.19z)| norm 0.2824 (+0.07z)| lr 2.46e-05 | 322.21 ms | 52.4% bf16 MFU | 1624702 tok/s
step 17112/19560 | loss 3.246867 (-1.06z)| norm 0.2513 (-1.05z)| lr 2.46e-05 | 322.72 ms | 52.3% bf16 MFU | 1624696 tok/s
step 17113/19560 | loss 3.285671 (-0.03z)| norm 0.2598 (-0.73z)| lr 2.46e-05 | 322.58 ms | 52.3% bf16 MFU | 1624728 tok/s
step 17114/19560 | loss 3.319499 (+0.88z)| norm 0.2954 (+0.55z)| lr 2.46e-05 | 323.37 ms | 52.2% bf16 MFU | 1624558 tok/s
step 17115/19560 | loss 3.272234 (-0.38z)| norm 0.2489 (-1.12z)| lr 2.46e-05 | 323.55 ms | 52.2% bf16 MFU | 1624352 tok/s
step 17116/19560 | loss 3.323723 (+1.00z)| norm 0.3200 (+1.41z)| lr 2.45e-05 | 322.62 ms | 52.3% bf16 MFU | 1624390 tok/s
step 17117/19560 | loss 3.319353 (+0.87z)| norm 0.2519 (-1.02z)| lr 2.45e-05 | 323.10 ms | 52.2% bf16 MFU | 1624304 tok/s
step 17118/19560 | loss 3.278506 (-0.23z)| norm 0.2650 (-0.55z)| lr 2.45e-05 | 322.96 ms | 52.3% bf16 MFU | 1624258 tok/s
step 17119/19560 | loss 3.297834 (+0.28z)| norm 0.2553 (-0.88z)| lr 2.45e-05 | 322.90 ms | 52.3% bf16 MFU | 1624230 tok/s
step 17120/19560 | loss 3.308108 (+0.55z)| norm 0.2901 (+0.35z)| lr 2.45e-05 | 322.28 ms | 52.4% bf16 MFU | 1624360 tok/s
step 17121/19560 | loss 3.247128 (-1.09z)| norm 0.2371 (-1.50z)| lr 2.44e-05 | 322.59 ms | 52.3% bf16 MFU | 1624404 tok/s
step 17122/19560 | loss 3.244802 (-1.13z)| norm 0.2723 (-0.25z)| lr 2.44e-05 | 323.05 ms | 52.2% bf16 MFU | 1624329 tok/s
step 17123/19560 | loss 3.282825 (-0.10z)| norm 0.2967 (+0.63z)| lr 2.44e-05 | 322.36 ms | 52.4% bf16 MFU | 1624432 tok/s
step 17124/19560 | loss 3.276876 (-0.26z)| norm 0.2758 (-0.11z)| lr 2.44e-05 | 322.93 ms | 52.3% bf16 MFU | 1624388 tok/s
step 17125/19560 | loss 3.390455 (+2.73z)| norm 0.3304 (+1.92z)| lr 2.44e-05 | 322.50 ms | 52.3% bf16 MFU | 1624453 tok/s
step 17126/19560 | loss 3.398291 (+2.87z)| norm 0.2694 (-0.34z)| lr 2.43e-05 | 322.24 ms | 52.4% bf16 MFU | 1624580 tok/s
step 17127/19560 | loss 3.286615 (-0.04z)| norm 0.2604 (-0.67z)| lr 2.43e-05 | 323.04 ms | 52.2% bf16 MFU | 1624501 tok/s
step 17128/19560 | loss 3.241783 (-1.19z)| norm 0.2604 (-0.65z)| lr 2.43e-05 | 322.61 ms | 52.3% bf16 MFU | 1624533 tok/s
step 17129/19560 | loss 3.270835 (-0.42z)| norm 0.3205 (+1.56z)| lr 2.43e-05 | 322.89 ms | 52.3% bf16 MFU | 1624492 tok/s
step 17130/19560 | loss 3.257074 (-0.78z)| norm 0.2573 (-0.77z)| lr 2.43e-05 | 322.52 ms | 52.3% bf16 MFU | 1624547 tok/s
step 17131/19560 | loss 3.288800 (+0.09z)| norm 0.3112 (+1.20z)| lr 2.42e-05 | 322.92 ms | 52.3% bf16 MFU | 1624500 tok/s
step 17132/19560 | loss 3.345719 (+1.65z)| norm 0.2569 (-0.79z)| lr 2.42e-05 | 322.93 ms | 52.3% bf16 MFU | 1624452 tok/s
step 17133/19560 | loss 3.263709 (-0.60z)| norm 0.2631 (-0.54z)| lr 2.42e-05 | 322.61 ms | 52.3% bf16 MFU | 1624486 tok/s
step 17134/19560 | loss 3.232534 (-1.45z)| norm 0.2748 (-0.09z)| lr 2.42e-05 | 322.64 ms | 52.3% bf16 MFU | 1624512 tok/s
step 17135/19560 | loss 3.305189 (+0.55z)| norm 0.2618 (-0.58z)| lr 2.42e-05 | 322.36 ms | 52.4% bf16 MFU | 1624608 tok/s
step 17136/19560 | loss 3.283975 (-0.04z)| norm 0.2969 (+0.75z)| lr 2.41e-05 | 322.76 ms | 52.3% bf16 MFU | 1624596 tok/s
step 17137/19560 | loss 3.283215 (-0.07z)| norm 0.2607 (-0.60z)| lr 2.41e-05 | 322.61 ms | 52.3% bf16 MFU | 1624624 tok/s
step 17138/19560 | loss 3.286959 (+0.04z)| norm 0.2588 (-0.67z)| lr 2.41e-05 | 322.42 ms | 52.3% bf16 MFU | 1624698 tok/s
step 17139/19560 | loss 3.266698 (-0.53z)| norm 0.2604 (-0.61z)| lr 2.41e-05 | 322.61 ms | 52.3% bf16 MFU | 1624720 tok/s
step 17140/19560 | loss 3.239151 (-1.28z)| norm 0.2806 (+0.15z)| lr 2.41e-05 | 323.09 ms | 52.2% bf16 MFU | 1624619 tok/s
step 17141/19560 | loss 3.223115 (-1.69z)| norm 0.2515 (-0.95z)| lr 2.40e-05 | 322.38 ms | 52.4% bf16 MFU | 1624704 tok/s
step 17142/19560 | loss 3.262325 (-0.63z)| norm 0.2707 (-0.22z)| lr 2.40e-05 | 323.02 ms | 52.2% bf16 MFU | 1624622 tok/s
step 17143/19560 | loss 3.351125 (+1.78z)| norm 0.3273 (+1.90z)| lr 2.40e-05 | 322.54 ms | 52.3% bf16 MFU | 1624667 tok/s
step 17144/19560 | loss 3.285269 (-0.01z)| norm 0.3432 (+2.42z)| lr 2.40e-05 | 322.55 ms | 52.3% bf16 MFU | 1624705 tok/s
step 17145/19560 | loss 3.277074 (-0.23z)| norm 0.2536 (-0.86z)| lr 2.40e-05 | 322.65 ms | 52.3% bf16 MFU | 1624718 tok/s
step 17146/19560 | loss 3.250323 (-0.95z)| norm 0.2911 (+0.52z)| lr 2.39e-05 | 323.29 ms | 52.2% bf16 MFU | 1624568 tok/s
step 17147/19560 | loss 3.237112 (-1.29z)| norm 0.2757 (-0.03z)| lr 2.39e-05 | 322.70 ms | 52.3% bf16 MFU | 1624574 tok/s
step 17148/19560 | loss 3.249430 (-0.95z)| norm 0.2467 (-1.10z)| lr 2.39e-05 | 322.81 ms | 52.3% bf16 MFU | 1624553 tok/s
step 17149/19560 | loss 3.291858 (+0.20z)| norm 0.2577 (-0.69z)| lr 2.39e-05 | 322.73 ms | 52.3% bf16 MFU | 1624553 tok/s
step 17150/19560 | loss 3.298610 (+0.38z)| norm 0.2854 (+0.37z)| lr 2.39e-05 | 322.97 ms | 52.3% bf16 MFU | 1624493 tok/s
step 17151/19560 | loss 3.260866 (-0.63z)| norm 0.2536 (-0.84z)| lr 2.39e-05 | 322.78 ms | 52.3% bf16 MFU | 1624483 tok/s
step 17152/19560 | loss 3.336012 (+1.42z)| norm 0.3049 (+1.13z)| lr 2.38e-05 | 322.38 ms | 52.4% bf16 MFU | 1624574 tok/s
step 17153/19560 | loss 3.308275 (+0.65z)| norm 0.2565 (-0.73z)| lr 2.38e-05 | 322.56 ms | 52.3% bf16 MFU | 1624614 tok/s
step 17154/19560 | loss 3.248439 (-0.99z)| norm 0.2484 (-1.04z)| lr 2.38e-05 | 322.97 ms | 52.3% bf16 MFU | 1624550 tok/s
step 17155/19560 | loss 3.273117 (-0.30z)| norm 0.3380 (+2.41z)| lr 2.38e-05 | 322.96 ms | 52.3% bf16 MFU | 1624492 tok/s
step 17156/19560 | loss 3.232679 (-1.47z)| norm 0.2427 (-1.25z)| lr 2.38e-05 | 322.51 ms | 52.3% bf16 MFU | 1624550 tok/s
step 17157/19560 | loss 3.281843 (-0.06z)| norm 0.2871 (+0.52z)| lr 2.37e-05 | 322.52 ms | 52.3% bf16 MFU | 1624604 tok/s
step 17158/19560 | loss 3.310403 (+0.74z)| norm 0.2820 (+0.31z)| lr 2.37e-05 | 322.57 ms | 52.3% bf16 MFU | 1624640 tok/s
step 17159/19560 | loss 3.307428 (+0.71z)| norm 0.2795 (+0.22z)| lr 2.37e-05 | 322.65 ms | 52.3% bf16 MFU | 1624655 tok/s
step 17160/19560 | loss 3.255727 (-0.82z)| norm 0.2819 (+0.34z)| lr 2.37e-05 | 323.27 ms | 52.2% bf16 MFU | 1624512 tok/s
step 17161/19560 | loss 3.291857 (+0.28z)| norm 0.3091 (+1.45z)| lr 2.37e-05 | 322.70 ms | 52.3% bf16 MFU | 1624522 tok/s
step 17162/19560 | loss 3.234534 (-1.46z)| norm 0.2705 (-0.15z)| lr 2.36e-05 | 322.70 ms | 52.3% bf16 MFU | 1624532 tok/s
step 17163/19560 | loss 3.316965 (+1.03z)| norm 0.2465 (-1.12z)| lr 2.36e-05 | 322.87 ms | 52.3% bf16 MFU | 1624496 tok/s
step 17164/19560 | loss 3.240929 (-1.25z)| norm 0.2515 (-0.90z)| lr 2.36e-05 | 323.17 ms | 52.2% bf16 MFU | 1624388 tok/s
step 17165/19560 | loss 3.305496 (+0.68z)| norm 0.2960 (+0.91z)| lr 2.36e-05 | 322.97 ms | 52.3% bf16 MFU | 1624335 tok/s
step 17166/19560 | loss 3.229936 (-1.57z)| norm 0.2637 (-0.42z)| lr 2.36e-05 | 322.71 ms | 52.3% bf16 MFU | 1624351 tok/s
step 17167/19560 | loss 3.297359 (+0.47z)| norm 0.2457 (-1.14z)| lr 2.35e-05 | 323.15 ms | 52.2% bf16 MFU | 1624256 tok/s
step 17168/19560 | loss 3.274774 (-0.23z)| norm 0.2965 (+0.93z)| lr 2.35e-05 | 323.00 ms | 52.3% bf16 MFU | 1624203 tok/s
step 17169/19560 | loss 3.318889 (+1.12z)| norm 0.2710 (-0.12z)| lr 2.35e-05 | 322.78 ms | 52.3% bf16 MFU | 1624208 tok/s
step 17170/19560 | loss 3.344119 (+1.85z)| norm 0.2673 (-0.27z)| lr 2.35e-05 | 322.30 ms | 52.4% bf16 MFU | 1624334 tok/s
step 17171/19560 | loss 3.261257 (-0.64z)| norm 0.3359 (+2.46z)| lr 2.35e-05 | 322.84 ms | 52.3% bf16 MFU | 1624317 tok/s
step 17172/19560 | loss 3.307450 (+0.75z)| norm 0.2458 (-1.15z)| lr 2.34e-05 | 323.17 ms | 52.2% bf16 MFU | 1624218 tok/s
step 17173/19560 | loss 3.381819 (+2.88z)| norm 0.3314 (+2.22z)| lr 2.34e-05 | 322.52 ms | 52.3% bf16 MFU | 1624288 tok/s
step 17174/19560 | loss 3.320676 (+1.08z)| norm 0.3017 (+1.03z)| lr 2.34e-05 | 322.81 ms | 52.3% bf16 MFU | 1624281 tok/s
step 17175/19560 | loss 3.281988 (-0.05z)| norm 0.2590 (-0.65z)| lr 2.34e-05 | 322.44 ms | 52.3% bf16 MFU | 1624367 tok/s
step 17176/19560 | loss 3.321718 (+1.09z)| norm 0.3007 (+0.98z)| lr 2.34e-05 | 322.47 ms | 52.3% bf16 MFU | 1624441 tok/s
step 17177/19560 | loss 3.294046 (+0.28z)| norm 0.3250 (+1.89z)| lr 2.33e-05 | 322.91 ms | 52.3% bf16 MFU | 1624400 tok/s
step 17178/19560 | loss 3.244299 (-1.16z)| norm 0.2886 (+0.47z)| lr 2.33e-05 | 322.90 ms | 52.3% bf16 MFU | 1624365 tok/s
step 17179/19560 | loss 3.244701 (-1.14z)| norm 0.2411 (-1.35z)| lr 2.33e-05 | 322.77 ms | 52.3% bf16 MFU | 1624365 tok/s
step 17180/19560 | loss 3.289114 (+0.14z)| norm 0.2735 (-0.11z)| lr 2.33e-05 | 322.66 ms | 52.3% bf16 MFU | 1624392 tok/s
step 17181/19560 | loss 3.290807 (+0.18z)| norm 0.3260 (+1.87z)| lr 2.33e-05 | 323.19 ms | 52.2% bf16 MFU | 1624283 tok/s
step 17182/19560 | loss 3.278129 (-0.17z)| norm 0.2548 (-0.83z)| lr 2.32e-05 | 322.63 ms | 52.3% bf16 MFU | 1624322 tok/s
step 17183/19560 | loss 3.285267 (+0.03z)| norm 0.2435 (-1.25z)| lr 2.32e-05 | 322.44 ms | 52.3% bf16 MFU | 1624405 tok/s
step 17184/19560 | loss 3.274307 (-0.29z)| norm 0.2645 (-0.47z)| lr 2.32e-05 | 323.03 ms | 52.2% bf16 MFU | 1624337 tok/s
step 17185/19560 | loss 3.303912 (+0.56z)| norm 0.2681 (-0.33z)| lr 2.32e-05 | 322.44 ms | 52.3% bf16 MFU | 1624419 tok/s
step 17186/19560 | loss 3.263403 (-0.61z)| norm 0.2492 (-1.04z)| lr 2.32e-05 | 323.26 ms | 52.2% bf16 MFU | 1624292 tok/s
step 17187/19560 | loss 3.250624 (-0.96z)| norm 0.2595 (-0.65z)| lr 2.32e-05 | 323.07 ms | 52.2% bf16 MFU | 1624218 tok/s
step 17188/19560 | loss 3.287797 (+0.13z)| norm 0.2587 (-0.68z)| lr 2.31e-05 | 323.43 ms | 52.2% bf16 MFU | 1624058 tok/s
step 17189/19560 | loss 3.291264 (+0.23z)| norm 0.2402 (-1.36z)| lr 2.31e-05 | 322.33 ms | 52.4% bf16 MFU | 1624184 tok/s
step 17190/19560 | loss 3.255054 (-0.82z)| norm 0.2726 (-0.12z)| lr 2.31e-05 | 322.23 ms | 52.4% bf16 MFU | 1624328 tok/s
step 17191/19560 | loss 3.305730 (+0.66z)| norm 0.2502 (-0.98z)| lr 2.31e-05 | 323.29 ms | 52.2% bf16 MFU | 1624196 tok/s
step 17192/19560 | loss 3.256000 (-0.78z)| norm 0.2434 (-1.22z)| lr 2.31e-05 | 322.78 ms | 52.3% bf16 MFU | 1624201 tok/s
step 17193/19560 | loss 3.320860 (+1.20z)| norm 0.3178 (+1.62z)| lr 2.30e-05 | 323.09 ms | 52.2% bf16 MFU | 1624127 tok/s
step 17194/19560 | loss 3.258804 (-0.70z)| norm 0.2526 (-0.86z)| lr 2.30e-05 | 323.02 ms | 52.2% bf16 MFU | 1624075 tok/s
step 17195/19560 | loss 3.334768 (+1.61z)| norm 0.2528 (-0.85z)| lr 2.30e-05 | 322.86 ms | 52.3% bf16 MFU | 1624064 tok/s
step 17196/19560 | loss 3.306091 (+0.73z)| norm 0.2815 (+0.24z)| lr 2.30e-05 | 322.58 ms | 52.3% bf16 MFU | 1624126 tok/s
step 17197/19560 | loss 3.314265 (+0.96z)| norm 0.2525 (-0.86z)| lr 2.30e-05 | 322.31 ms | 52.4% bf16 MFU | 1624251 tok/s
step 17198/19560 | loss 3.367238 (+2.55z)| norm 0.2806 (+0.20z)| lr 2.29e-05 | 322.97 ms | 52.3% bf16 MFU | 1624206 tok/s
step 17199/19560 | loss 3.331662 (+1.45z)| norm 0.2920 (+0.63z)| lr 2.29e-05 | 322.84 ms | 52.3% bf16 MFU | 1624194 tok/s
step 17200/19560 | loss 3.271398 (-0.36z)| norm 0.2402 (-1.33z)| lr 2.29e-05 | 322.86 ms | 52.3% bf16 MFU | 1624179 tok/s
step 17201/19560 | loss 3.280289 (-0.10z)| norm 0.2641 (-0.43z)| lr 2.29e-05 | 323.12 ms | 52.2% bf16 MFU | 1624099 tok/s
step 17202/19560 | loss 3.257174 (-0.80z)| norm 0.2807 (+0.19z)| lr 2.29e-05 | 322.57 ms | 52.3% bf16 MFU | 1624163 tok/s
step 17203/19560 | loss 3.329111 (+1.35z)| norm 0.3449 (+2.57z)| lr 2.28e-05 | 323.10 ms | 52.2% bf16 MFU | 1624088 tok/s
step 17204/19560 | loss 3.296315 (+0.36z)| norm 0.2517 (-0.90z)| lr 2.28e-05 | 322.32 ms | 52.4% bf16 MFU | 1624214 tok/s
step 17205/19560 | loss 3.234856 (-1.48z)| norm 0.2619 (-0.52z)| lr 2.28e-05 | 322.91 ms | 52.3% bf16 MFU | 1624186 tok/s
step 17206/19560 | loss 3.302272 (+0.54z)| norm 0.2790 (+0.13z)| lr 2.28e-05 | 323.28 ms | 52.2% bf16 MFU | 1624066 tok/s
step 17207/19560 | loss 3.316230 (+0.94z)| norm 0.2468 (-1.07z)| lr 2.28e-05 | 322.61 ms | 52.3% bf16 MFU | 1624121 tok/s
step 17208/19560 | loss 3.289887 (+0.14z)| norm 0.3026 (+1.01z)| lr 2.28e-05 | 322.99 ms | 52.3% bf16 MFU | 1624078 tok/s
step 17209/19560 | loss 3.284473 (-0.01z)| norm 0.2682 (-0.26z)| lr 2.27e-05 | 322.97 ms | 52.3% bf16 MFU | 1624041 tok/s
step 17210/19560 | loss 3.305777 (+0.62z)| norm 0.3178 (+1.58z)| lr 2.27e-05 | 322.91 ms | 52.3% bf16 MFU | 1624020 tok/s
step 17211/19560 | loss 3.258027 (-0.82z)| norm 0.3608 (+3.04z)| lr 2.27e-05 | 322.46 ms | 52.3% bf16 MFU | 1624114 tok/s
step 17212/19560 | loss 3.247927 (-1.11z)| norm 0.2597 (-0.58z)| lr 2.27e-05 | 322.78 ms | 52.3% bf16 MFU | 1624124 tok/s
step 17213/19560 | loss 3.293298 (+0.26z)| norm 0.2748 (-0.02z)| lr 2.27e-05 | 323.00 ms | 52.3% bf16 MFU | 1624078 tok/s
step 17214/19560 | loss 3.238417 (-1.38z)| norm 0.3166 (+1.51z)| lr 2.26e-05 | 322.87 ms | 52.3% bf16 MFU | 1624067 tok/s
step 17215/19560 | loss 3.283051 (-0.04z)| norm 0.2747 (-0.03z)| lr 2.26e-05 | 322.86 ms | 52.3% bf16 MFU | 1624057 tok/s
step 17216/19560 | loss 3.250876 (-1.02z)| norm 0.2770 (+0.05z)| lr 2.26e-05 | 322.90 ms | 52.3% bf16 MFU | 1624037 tok/s
step 17217/19560 | loss 3.280858 (-0.11z)| norm 0.3254 (+1.79z)| lr 2.26e-05 | 323.21 ms | 52.2% bf16 MFU | 1623941 tok/s
step 17218/19560 | loss 3.261027 (-0.71z)| norm 0.2665 (-0.35z)| lr 2.26e-05 | 323.41 ms | 52.2% bf16 MFU | 1623800 tok/s
step 17219/19560 | loss 3.327583 (+1.29z)| norm 0.2739 (-0.09z)| lr 2.25e-05 | 322.86 ms | 52.3% bf16 MFU | 1623804 tok/s
step 17220/19560 | loss 3.306469 (+0.65z)| norm 0.2662 (-0.37z)| lr 2.25e-05 | 322.90 ms | 52.3% bf16 MFU | 1623799 tok/s
step 17221/19560 | loss 3.304398 (+0.57z)| norm 0.2502 (-0.95z)| lr 2.25e-05 | 322.80 ms | 52.3% bf16 MFU | 1623819 tok/s
step 17222/19560 | loss 3.321238 (+1.08z)| norm 0.2858 (+0.34z)| lr 2.25e-05 | 323.39 ms | 52.2% bf16 MFU | 1623689 tok/s
step 17223/19560 | loss 3.272551 (-0.40z)| norm 0.2656 (-0.39z)| lr 2.25e-05 | 322.89 ms | 52.3% bf16 MFU | 1623690 tok/s
step 17224/19560 | loss 3.289547 (+0.12z)| norm 0.2638 (-0.46z)| lr 2.24e-05 | 322.74 ms | 52.3% bf16 MFU | 1623730 tok/s
step 17225/19560 | loss 3.346913 (+1.82z)| norm 0.2921 (+0.60z)| lr 2.24e-05 | 323.18 ms | 52.2% bf16 MFU | 1623658 tok/s
step 17226/19560 | loss 3.261263 (-0.74z)| norm 0.2531 (-0.85z)| lr 2.24e-05 | 322.78 ms | 52.3% bf16 MFU | 1623689 tok/s
step 17227/19560 | loss 3.314986 (+0.86z)| norm 0.2750 (-0.02z)| lr 2.24e-05 | 322.01 ms | 52.4% bf16 MFU | 1623914 tok/s
step 17228/19560 | loss 3.331784 (+1.33z)| norm 0.2599 (-0.58z)| lr 2.24e-05 | 322.54 ms | 52.3% bf16 MFU | 1623994 tok/s
step 17229/19560 | loss 3.334464 (+1.39z)| norm 0.2640 (-0.42z)| lr 2.24e-05 | 324.18 ms | 52.1% bf16 MFU | 1623657 tok/s
step 17230/19560 | loss 3.275765 (-0.33z)| norm 0.2835 (+0.31z)| lr 2.23e-05 | 322.87 ms | 52.3% bf16 MFU | 1623665 tok/s
step 17231/19560 | loss 3.244890 (-1.23z)| norm 0.2780 (+0.09z)| lr 2.23e-05 | 322.41 ms | 52.3% bf16 MFU | 1623791 tok/s
step 17232/19560 | loss 3.283972 (-0.08z)| norm 0.3079 (+1.21z)| lr 2.23e-05 | 322.77 ms | 52.3% bf16 MFU | 1623817 tok/s
step 17233/19560 | loss 3.224016 (-1.81z)| norm 0.2559 (-0.74z)| lr 2.23e-05 | 323.06 ms | 52.2% bf16 MFU | 1623769 tok/s
step 17234/19560 | loss 3.281747 (-0.14z)| norm 0.2489 (-0.98z)| lr 2.23e-05 | 322.99 ms | 52.3% bf16 MFU | 1623742 tok/s
step 17235/19560 | loss 3.288704 (+0.06z)| norm 0.2991 (+0.88z)| lr 2.22e-05 | 322.58 ms | 52.3% bf16 MFU | 1623820 tok/s
step 17236/19560 | loss 3.238750 (-1.37z)| norm 0.2592 (-0.62z)| lr 2.22e-05 | 322.75 ms | 52.3% bf16 MFU | 1623852 tok/s
step 17237/19560 | loss 3.351818 (+1.85z)| norm 0.2555 (-0.75z)| lr 2.22e-05 | 322.56 ms | 52.3% bf16 MFU | 1623929 tok/s
step 17238/19560 | loss 3.282562 (-0.12z)| norm 0.2823 (+0.24z)| lr 2.22e-05 | 322.99 ms | 52.3% bf16 MFU | 1623894 tok/s
step 17239/19560 | loss 3.263607 (-0.65z)| norm 0.2770 (+0.04z)| lr 2.22e-05 | 322.95 ms | 52.3% bf16 MFU | 1623870 tok/s
step 17240/19560 | loss 3.331879 (+1.26z)| norm 0.2566 (-0.73z)| lr 2.21e-05 | 322.99 ms | 52.3% bf16 MFU | 1623839 tok/s
step 17241/19560 | loss 3.325632 (+1.07z)| norm 0.2537 (-0.83z)| lr 2.21e-05 | 322.74 ms | 52.3% bf16 MFU | 1623871 tok/s
step 17242/19560 | loss 3.235048 (-1.46z)| norm 0.2579 (-0.66z)| lr 2.21e-05 | 322.78 ms | 52.3% bf16 MFU | 1623893 tok/s
step 17243/19560 | loss 3.301885 (+0.41z)| norm 0.2965 (+0.77z)| lr 2.21e-05 | 322.91 ms | 52.3% bf16 MFU | 1623879 tok/s
step 17244/19560 | loss 3.355959 (+1.90z)| norm 0.3084 (+1.23z)| lr 2.21e-05 | 323.93 ms | 52.1% bf16 MFU | 1623610 tok/s
step 17245/19560 | loss 3.299246 (+0.33z)| norm 0.2740 (-0.08z)| lr 2.20e-05 | 322.69 ms | 52.3% bf16 MFU | 1623666 tok/s
step 17246/19560 | loss 3.295013 (+0.21z)| norm 0.2582 (-0.67z)| lr 2.20e-05 | 322.98 ms | 52.3% bf16 MFU | 1623646 tok/s
step 17247/19560 | loss 3.233780 (-1.47z)| norm 0.2740 (-0.08z)| lr 2.20e-05 | 323.30 ms | 52.2% bf16 MFU | 1623548 tok/s
step 17248/19560 | loss 3.286995 (+0.01z)| norm 0.2616 (-0.54z)| lr 2.20e-05 | 322.74 ms | 52.3% bf16 MFU | 1623594 tok/s
step 17249/19560 | loss 3.231017 (-1.53z)| norm 0.2887 (+0.47z)| lr 2.20e-05 | 323.00 ms | 52.3% bf16 MFU | 1623572 tok/s
step 17250/19560 | loss 3.266960 (-0.55z)| norm 0.2545 (-0.83z)| lr 2.20e-05 | 322.36 ms | 52.4% bf16 MFU | 1623715 tok/s
val loss 3.280791
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3051/10042 = 0.303824
step 17251/19560 | loss 3.312043 (+0.69z)| norm 0.2523 (-0.90z)| lr 2.19e-05 | 322.23 ms | 52.4% bf16 MFU | 1623883 tok/s
step 17252/19560 | loss 3.269660 (-0.48z)| norm 0.2889 (+0.49z)| lr 2.19e-05 | 322.47 ms | 52.3% bf16 MFU | 1623981 tok/s
step 17253/19560 | loss 3.313327 (+0.77z)| norm 0.2804 (+0.19z)| lr 2.19e-05 | 322.74 ms | 52.3% bf16 MFU | 1624006 tok/s
step 17254/19560 | loss 3.234772 (-1.48z)| norm 0.2911 (+0.59z)| lr 2.19e-05 | 322.69 ms | 52.3% bf16 MFU | 1624043 tok/s
step 17255/19560 | loss 3.357088 (+2.07z)| norm 0.3004 (+0.94z)| lr 2.19e-05 | 322.44 ms | 52.3% bf16 MFU | 1624140 tok/s
step 17256/19560 | loss 3.201001 (-2.40z)| norm 0.3232 (+1.78z)| lr 2.18e-05 | 322.84 ms | 52.3% bf16 MFU | 1624134 tok/s
step 17257/19560 | loss 3.280097 (-0.15z)| norm 0.2738 (-0.09z)| lr 2.18e-05 | 322.97 ms | 52.3% bf16 MFU | 1624094 tok/s
step 17258/19560 | loss 3.304004 (+0.52z)| norm 0.3311 (+2.07z)| lr 2.18e-05 | 322.73 ms | 52.3% bf16 MFU | 1624116 tok/s
step 17259/19560 | loss 3.269796 (-0.45z)| norm 0.2330 (-1.63z)| lr 2.18e-05 | 322.81 ms | 52.3% bf16 MFU | 1624117 tok/s
step 17260/19560 | loss 3.332763 (+1.35z)| norm 0.2594 (-0.63z)| lr 2.18e-05 | 322.56 ms | 52.3% bf16 MFU | 1624180 tok/s
step 17261/19560 | loss 3.271514 (-0.40z)| norm 0.2873 (+0.42z)| lr 2.17e-05 | 322.48 ms | 52.3% bf16 MFU | 1624261 tok/s
step 17262/19560 | loss 3.307771 (+0.62z)| norm 0.2982 (+0.82z)| lr 2.17e-05 | 322.83 ms | 52.3% bf16 MFU | 1624251 tok/s
step 17263/19560 | loss 3.299664 (+0.39z)| norm 0.2695 (-0.27z)| lr 2.17e-05 | 322.65 ms | 52.3% bf16 MFU | 1624284 tok/s
step 17264/19560 | loss 3.254400 (-0.91z)| norm 0.2936 (+0.65z)| lr 2.17e-05 | 322.55 ms | 52.3% bf16 MFU | 1624341 tok/s
step 17265/19560 | loss 3.280265 (-0.16z)| norm 0.2885 (+0.44z)| lr 2.17e-05 | 322.26 ms | 52.4% bf16 MFU | 1624469 tok/s
step 17266/19560 | loss 3.264002 (-0.62z)| norm 0.2554 (-0.81z)| lr 2.17e-05 | 322.69 ms | 52.3% bf16 MFU | 1624483 tok/s
step 17267/19560 | loss 3.257506 (-0.81z)| norm 0.2951 (+0.68z)| lr 2.16e-05 | 322.75 ms | 52.3% bf16 MFU | 1624481 tok/s
step 17268/19560 | loss 3.291147 (+0.15z)| norm 0.2408 (-1.35z)| lr 2.16e-05 | 322.80 ms | 52.3% bf16 MFU | 1624467 tok/s
step 17269/19560 | loss 3.328950 (+1.22z)| norm 0.2522 (-0.92z)| lr 2.16e-05 | 324.01 ms | 52.1% bf16 MFU | 1624148 tok/s
step 17270/19560 | loss 3.261338 (-0.74z)| norm 0.2533 (-0.87z)| lr 2.16e-05 | 321.95 ms | 52.4% bf16 MFU | 1624366 tok/s
step 17271/19560 | loss 3.276001 (-0.30z)| norm 0.2722 (-0.15z)| lr 2.16e-05 | 322.73 ms | 52.3% bf16 MFU | 1624374 tok/s
step 17272/19560 | loss 3.329791 (+1.27z)| norm 0.2862 (+0.41z)| lr 2.15e-05 | 322.45 ms | 52.3% bf16 MFU | 1624452 tok/s
step 17273/19560 | loss 3.566962 (+6.60z)| norm 0.3155 (+1.52z)| lr 2.15e-05 | 322.83 ms | 52.3% bf16 MFU | 1624432 tok/s
step 17274/19560 | loss 3.268889 (-0.48z)| norm 0.2488 (-1.05z)| lr 2.15e-05 | 322.28 ms | 52.4% bf16 MFU | 1624550 tok/s
step 17275/19560 | loss 3.372537 (+1.95z)| norm 0.3206 (+1.70z)| lr 2.15e-05 | 323.02 ms | 52.2% bf16 MFU | 1624478 tok/s
step 17276/19560 | loss 3.283487 (-0.16z)| norm 0.2866 (+0.39z)| lr 2.15e-05 | 322.68 ms | 52.3% bf16 MFU | 1624495 tok/s
step 17277/19560 | loss 3.373507 (+1.93z)| norm 0.2544 (-0.84z)| lr 2.15e-05 | 322.37 ms | 52.4% bf16 MFU | 1624587 tok/s
step 17278/19560 | loss 3.241883 (-1.13z)| norm 0.2938 (+0.66z)| lr 2.14e-05 | 322.80 ms | 52.3% bf16 MFU | 1624568 tok/s
step 17279/19560 | loss 3.231263 (-1.36z)| norm 0.2732 (-0.13z)| lr 2.14e-05 | 322.88 ms | 52.3% bf16 MFU | 1624528 tok/s
step 17280/19560 | loss 3.232206 (-1.32z)| norm 0.2830 (+0.25z)| lr 2.14e-05 | 322.74 ms | 52.3% bf16 MFU | 1624526 tok/s
step 17281/19560 | loss 3.267974 (-0.49z)| norm 0.2711 (-0.21z)| lr 2.14e-05 | 322.45 ms | 52.3% bf16 MFU | 1624598 tok/s
step 17282/19560 | loss 3.275476 (-0.32z)| norm 0.2487 (-1.08z)| lr 2.14e-05 | 322.68 ms | 52.3% bf16 MFU | 1624608 tok/s
step 17283/19560 | loss 3.249242 (-0.92z)| norm 0.2764 (+0.01z)| lr 2.13e-05 | 323.21 ms | 52.2% bf16 MFU | 1624483 tok/s
step 17284/19560 | loss 3.265927 (-0.55z)| norm 0.2589 (-0.69z)| lr 2.13e-05 | 322.53 ms | 52.3% bf16 MFU | 1624538 tok/s
step 17285/19560 | loss 3.293086 (+0.08z)| norm 0.2545 (-0.85z)| lr 2.13e-05 | 322.39 ms | 52.3% bf16 MFU | 1624622 tok/s
step 17286/19560 | loss 3.583408 (+5.80z)| norm 0.4008 (+4.50z)| lr 2.13e-05 | 322.35 ms | 52.4% bf16 MFU | 1624715 tok/s
step 17287/19560 | loss 3.322418 (+0.61z)| norm 0.3046 (+0.99z)| lr 2.13e-05 | 323.07 ms | 52.2% bf16 MFU | 1624621 tok/s
step 17288/19560 | loss 3.263273 (-0.57z)| norm 0.2610 (-0.58z)| lr 2.12e-05 | 322.80 ms | 52.3% bf16 MFU | 1624600 tok/s
step 17289/19560 | loss 3.321045 (+0.57z)| norm 0.3286 (+1.85z)| lr 2.12e-05 | 322.74 ms | 52.3% bf16 MFU | 1624595 tok/s
step 17290/19560 | loss 3.375431 (+1.62z)| norm 0.2899 (+0.45z)| lr 2.12e-05 | 322.58 ms | 52.3% bf16 MFU | 1624629 tok/s
step 17291/19560 | loss 3.307062 (+0.28z)| norm 0.2624 (-0.54z)| lr 2.12e-05 | 322.78 ms | 52.3% bf16 MFU | 1624612 tok/s
step 17292/19560 | loss 3.339741 (+0.91z)| norm 0.2533 (-0.87z)| lr 2.12e-05 | 322.29 ms | 52.4% bf16 MFU | 1624720 tok/s
step 17293/19560 | loss 3.345605 (+1.01z)| norm 0.2837 (+0.23z)| lr 2.12e-05 | 322.48 ms | 52.3% bf16 MFU | 1624773 tok/s
step 17294/19560 | loss 3.306923 (+0.24z)| norm 0.2756 (-0.07z)| lr 2.11e-05 | 323.08 ms | 52.2% bf16 MFU | 1624674 tok/s
step 17295/19560 | loss 3.358721 (+1.25z)| norm 0.2658 (-0.43z)| lr 2.11e-05 | 323.16 ms | 52.2% bf16 MFU | 1624559 tok/s
step 17296/19560 | loss 3.298840 (+0.07z)| norm 0.2518 (-0.92z)| lr 2.11e-05 | 322.29 ms | 52.4% bf16 MFU | 1624668 tok/s
step 17297/19560 | loss 3.326853 (+0.62z)| norm 0.2484 (-1.03z)| lr 2.11e-05 | 322.35 ms | 52.4% bf16 MFU | 1624758 tok/s
step 17298/19560 | loss 3.179762 (-2.22z)| norm 0.2628 (-0.51z)| lr 2.11e-05 | 322.62 ms | 52.3% bf16 MFU | 1624774 tok/s
step 17299/19560 | loss 3.290376 (-0.08z)| norm 0.2714 (-0.19z)| lr 2.10e-05 | 322.86 ms | 52.3% bf16 MFU | 1624729 tok/s
step 17300/19560 | loss 3.299519 (+0.10z)| norm 0.2612 (-0.57z)| lr 2.10e-05 | 322.47 ms | 52.3% bf16 MFU | 1624784 tok/s
step 17301/19560 | loss 3.245174 (-0.94z)| norm 0.2739 (-0.09z)| lr 2.10e-05 | 322.83 ms | 52.3% bf16 MFU | 1624747 tok/s
step 17302/19560 | loss 3.313938 (+0.40z)| norm 0.2723 (-0.14z)| lr 2.10e-05 | 322.85 ms | 52.3% bf16 MFU | 1624705 tok/s
step 17303/19560 | loss 3.306156 (+0.25z)| norm 0.2612 (-0.55z)| lr 2.10e-05 | 322.47 ms | 52.3% bf16 MFU | 1624762 tok/s
step 17304/19560 | loss 3.279599 (-0.27z)| norm 0.3063 (+1.14z)| lr 2.10e-05 | 322.99 ms | 52.3% bf16 MFU | 1624685 tok/s
step 17305/19560 | loss 3.260363 (-0.64z)| norm 0.2591 (-0.62z)| lr 2.09e-05 | 322.28 ms | 52.4% bf16 MFU | 1624792 tok/s
step 17306/19560 | loss 3.490400 (+3.63z)| norm 0.2923 (+0.64z)| lr 2.09e-05 | 322.56 ms | 52.3% bf16 MFU | 1624822 tok/s
step 17307/19560 | loss 3.295466 (+0.00z)| norm 0.2737 (-0.08z)| lr 2.09e-05 | 322.78 ms | 52.3% bf16 MFU | 1624795 tok/s
step 17308/19560 | loss 3.270278 (-0.46z)| norm 0.2898 (+0.53z)| lr 2.09e-05 | 322.67 ms | 52.3% bf16 MFU | 1624797 tok/s
step 17309/19560 | loss 3.224243 (-1.30z)| norm 0.2636 (-0.46z)| lr 2.09e-05 | 322.80 ms | 52.3% bf16 MFU | 1624766 tok/s
step 17310/19560 | loss 3.320672 (+0.48z)| norm 0.2519 (-0.91z)| lr 2.08e-05 | 322.71 ms | 52.3% bf16 MFU | 1624759 tok/s
step 17311/19560 | loss 3.243780 (-0.94z)| norm 0.2676 (-0.31z)| lr 2.08e-05 | 323.09 ms | 52.2% bf16 MFU | 1624659 tok/s
step 17312/19560 | loss 3.356750 (+1.13z)| norm 0.3080 (+1.24z)| lr 2.08e-05 | 322.36 ms | 52.4% bf16 MFU | 1624747 tok/s
step 17313/19560 | loss 3.283858 (-0.21z)| norm 0.2521 (-0.91z)| lr 2.08e-05 | 322.60 ms | 52.3% bf16 MFU | 1624768 tok/s
step 17314/19560 | loss 3.305426 (+0.18z)| norm 0.2645 (-0.44z)| lr 2.08e-05 | 322.60 ms | 52.3% bf16 MFU | 1624791 tok/s
step 17315/19560 | loss 3.293847 (-0.03z)| norm 0.2782 (+0.08z)| lr 2.08e-05 | 322.46 ms | 52.3% bf16 MFU | 1624847 tok/s
step 17316/19560 | loss 3.314783 (+0.35z)| norm 0.2902 (+0.54z)| lr 2.07e-05 | 322.04 ms | 52.4% bf16 MFU | 1625005 tok/s
step 17317/19560 | loss 3.320719 (+0.45z)| norm 0.2970 (+0.79z)| lr 2.07e-05 | 322.63 ms | 52.3% bf16 MFU | 1625006 tok/s
step 17318/19560 | loss 3.260878 (-0.65z)| norm 0.2904 (+0.53z)| lr 2.07e-05 | 323.32 ms | 52.2% bf16 MFU | 1624834 tok/s
step 17319/19560 | loss 3.219458 (-1.39z)| norm 0.2732 (-0.15z)| lr 2.07e-05 | 322.30 ms | 52.4% bf16 MFU | 1624929 tok/s
step 17320/19560 | loss 3.290286 (-0.10z)| norm 0.2873 (+0.39z)| lr 2.07e-05 | 322.88 ms | 52.3% bf16 MFU | 1624872 tok/s
step 17321/19560 | loss 3.247434 (-0.87z)| norm 0.2829 (+0.23z)| lr 2.06e-05 | 323.11 ms | 52.2% bf16 MFU | 1624761 tok/s
step 17322/19560 | loss 3.246071 (-0.89z)| norm 0.2408 (-1.43z)| lr 2.06e-05 | 322.49 ms | 52.3% bf16 MFU | 1624811 tok/s
step 17323/19560 | loss 3.275111 (-0.36z)| norm 0.2621 (-0.60z)| lr 2.06e-05 | 322.94 ms | 52.3% bf16 MFU | 1624746 tok/s
step 17324/19560 | loss 3.339834 (+0.82z)| norm 0.2834 (+0.25z)| lr 2.06e-05 | 322.64 ms | 52.3% bf16 MFU | 1624759 tok/s
step 17325/19560 | loss 3.253412 (-0.75z)| norm 0.2417 (-1.39z)| lr 2.06e-05 | 322.13 ms | 52.4% bf16 MFU | 1624900 tok/s
step 17326/19560 | loss 3.344206 (+0.91z)| norm 0.2695 (-0.29z)| lr 2.06e-05 | 323.02 ms | 52.2% bf16 MFU | 1624808 tok/s
step 17327/19560 | loss 3.298612 (+0.08z)| norm 0.2705 (-0.25z)| lr 2.05e-05 | 322.26 ms | 52.4% bf16 MFU | 1624914 tok/s
step 17328/19560 | loss 3.215868 (-1.41z)| norm 0.2611 (-0.63z)| lr 2.05e-05 | 322.84 ms | 52.3% bf16 MFU | 1624867 tok/s
step 17329/19560 | loss 3.280323 (-0.24z)| norm 0.2605 (-0.65z)| lr 2.05e-05 | 322.62 ms | 52.3% bf16 MFU | 1624879 tok/s
step 17330/19560 | loss 3.316357 (+0.41z)| norm 0.3225 (+1.77z)| lr 2.05e-05 | 322.73 ms | 52.3% bf16 MFU | 1624862 tok/s
step 17331/19560 | loss 3.332293 (+0.70z)| norm 0.3110 (+1.36z)| lr 2.05e-05 | 322.56 ms | 52.3% bf16 MFU | 1624890 tok/s
step 17332/19560 | loss 3.339767 (+0.82z)| norm 0.2639 (-0.53z)| lr 2.04e-05 | 323.15 ms | 52.2% bf16 MFU | 1624768 tok/s
step 17333/19560 | loss 3.217152 (-1.39z)| norm 0.3095 (+1.28z)| lr 2.04e-05 | 323.05 ms | 52.2% bf16 MFU | 1624675 tok/s
step 17334/19560 | loss 3.298616 (+0.08z)| norm 0.3454 (+2.62z)| lr 2.04e-05 | 322.44 ms | 52.3% bf16 MFU | 1624741 tok/s
step 17335/19560 | loss 3.355794 (+1.10z)| norm 0.2650 (-0.51z)| lr 2.04e-05 | 322.85 ms | 52.3% bf16 MFU | 1624700 tok/s
step 17336/19560 | loss 3.302083 (+0.13z)| norm 0.3150 (+1.43z)| lr 2.04e-05 | 322.45 ms | 52.3% bf16 MFU | 1624764 tok/s
step 17337/19560 | loss 3.244126 (-0.90z)| norm 0.3420 (+2.40z)| lr 2.04e-05 | 322.44 ms | 52.3% bf16 MFU | 1624825 tok/s
step 17338/19560 | loss 3.305143 (+0.19z)| norm 0.2753 (-0.12z)| lr 2.03e-05 | 322.84 ms | 52.3% bf16 MFU | 1624783 tok/s
step 17339/19560 | loss 3.294733 (+0.00z)| norm 0.3146 (+1.45z)| lr 2.03e-05 | 323.27 ms | 52.2% bf16 MFU | 1624635 tok/s
step 17340/19560 | loss 3.290709 (-0.08z)| norm 0.2994 (+0.83z)| lr 2.03e-05 | 323.16 ms | 52.2% bf16 MFU | 1624522 tok/s
step 17341/19560 | loss 3.268086 (-0.48z)| norm 0.2646 (-0.54z)| lr 2.03e-05 | 322.60 ms | 52.3% bf16 MFU | 1624555 tok/s
step 17342/19560 | loss 3.330531 (+0.63z)| norm 0.2821 (+0.16z)| lr 2.03e-05 | 323.04 ms | 52.2% bf16 MFU | 1624476 tok/s
step 17343/19560 | loss 3.371231 (+1.35z)| norm 0.3004 (+0.88z)| lr 2.02e-05 | 323.06 ms | 52.2% bf16 MFU | 1624396 tok/s
step 17344/19560 | loss 3.257953 (-0.69z)| norm 0.2642 (-0.55z)| lr 2.02e-05 | 322.80 ms | 52.3% bf16 MFU | 1624387 tok/s
step 17345/19560 | loss 3.324034 (+0.49z)| norm 0.2491 (-1.14z)| lr 2.02e-05 | 322.79 ms | 52.3% bf16 MFU | 1624381 tok/s
step 17346/19560 | loss 3.283028 (-0.25z)| norm 0.3317 (+2.12z)| lr 2.02e-05 | 322.92 ms | 52.3% bf16 MFU | 1624340 tok/s
step 17347/19560 | loss 3.219585 (-1.36z)| norm 0.3092 (+1.21z)| lr 2.02e-05 | 322.33 ms | 52.4% bf16 MFU | 1624452 tok/s
step 17348/19560 | loss 3.264424 (-0.55z)| norm 0.2667 (-0.46z)| lr 2.02e-05 | 323.10 ms | 52.2% bf16 MFU | 1624365 tok/s
step 17349/19560 | loss 3.280387 (-0.27z)| norm 0.3257 (+1.82z)| lr 2.01e-05 | 322.64 ms | 52.3% bf16 MFU | 1624395 tok/s
step 17350/19560 | loss 3.340703 (+0.80z)| norm 0.3155 (+1.40z)| lr 2.01e-05 | 322.91 ms | 52.3% bf16 MFU | 1624358 tok/s
step 17351/19560 | loss 3.228322 (-1.18z)| norm 0.2635 (-0.60z)| lr 2.01e-05 | 323.20 ms | 52.2% bf16 MFU | 1624250 tok/s
step 17352/19560 | loss 3.284482 (-0.19z)| norm 0.2627 (-0.64z)| lr 2.01e-05 | 322.23 ms | 52.4% bf16 MFU | 1624391 tok/s
step 17353/19560 | loss 3.319625 (+0.44z)| norm 0.3076 (+1.09z)| lr 2.01e-05 | 322.49 ms | 52.3% bf16 MFU | 1624460 tok/s
step 17354/19560 | loss 3.339591 (+0.78z)| norm 0.2988 (+0.74z)| lr 2.00e-05 | 322.80 ms | 52.3% bf16 MFU | 1624446 tok/s
step 17355/19560 | loss 3.346192 (+0.89z)| norm 0.2901 (+0.40z)| lr 2.00e-05 | 322.09 ms | 52.4% bf16 MFU | 1624613 tok/s
step 17356/19560 | loss 3.228894 (-1.16z)| norm 0.2573 (-0.86z)| lr 2.00e-05 | 322.74 ms | 52.3% bf16 MFU | 1624608 tok/s
step 17357/19560 | loss 3.269889 (-0.43z)| norm 0.3847 (+3.78z)| lr 2.00e-05 | 322.71 ms | 52.3% bf16 MFU | 1624609 tok/s
step 17358/19560 | loss 3.302448 (+0.14z)| norm 0.2781 (-0.09z)| lr 2.00e-05 | 322.92 ms | 52.3% bf16 MFU | 1624558 tok/s
step 17359/19560 | loss 3.350692 (+0.97z)| norm 0.2948 (+0.51z)| lr 2.00e-05 | 322.75 ms | 52.3% bf16 MFU | 1624552 tok/s
step 17360/19560 | loss 3.277126 (-0.32z)| norm 0.3006 (+0.72z)| lr 1.99e-05 | 322.91 ms | 52.3% bf16 MFU | 1624508 tok/s
step 17361/19560 | loss 3.277946 (-0.32z)| norm 0.2601 (-0.75z)| lr 1.99e-05 | 323.27 ms | 52.2% bf16 MFU | 1624374 tok/s
step 17362/19560 | loss 3.341189 (+0.79z)| norm 0.2552 (-0.93z)| lr 1.99e-05 | 323.40 ms | 52.2% bf16 MFU | 1624213 tok/s
step 17363/19560 | loss 3.321651 (+0.44z)| norm 0.2893 (+0.31z)| lr 1.99e-05 | 322.08 ms | 52.4% bf16 MFU | 1624393 tok/s
step 17364/19560 | loss 3.370754 (+1.29z)| norm 0.2763 (-0.17z)| lr 1.99e-05 | 322.57 ms | 52.3% bf16 MFU | 1624442 tok/s
step 17365/19560 | loss 3.278710 (-0.32z)| norm 0.3062 (+0.91z)| lr 1.98e-05 | 322.90 ms | 52.3% bf16 MFU | 1624404 tok/s
step 17366/19560 | loss 3.216239 (-1.41z)| norm 0.2771 (-0.15z)| lr 1.98e-05 | 322.88 ms | 52.3% bf16 MFU | 1624374 tok/s
step 17367/19560 | loss 3.341904 (+0.79z)| norm 0.2703 (-0.40z)| lr 1.98e-05 | 322.21 ms | 52.4% bf16 MFU | 1624514 tok/s
step 17368/19560 | loss 3.308670 (+0.21z)| norm 0.2624 (-0.69z)| lr 1.98e-05 | 322.78 ms | 52.3% bf16 MFU | 1624503 tok/s
step 17369/19560 | loss 3.278512 (-0.32z)| norm 0.2581 (-0.85z)| lr 1.98e-05 | 323.52 ms | 52.2% bf16 MFU | 1624306 tok/s
step 17370/19560 | loss 3.264959 (-0.56z)| norm 0.2628 (-0.68z)| lr 1.98e-05 | 322.84 ms | 52.3% bf16 MFU | 1624290 tok/s
step 17371/19560 | loss 3.305696 (+0.16z)| norm 0.2687 (-0.45z)| lr 1.97e-05 | 322.57 ms | 52.3% bf16 MFU | 1624342 tok/s
step 17372/19560 | loss 3.485733 (+3.19z)| norm 0.3115 (+1.12z)| lr 1.97e-05 | 323.06 ms | 52.2% bf16 MFU | 1624269 tok/s
step 17373/19560 | loss 3.261024 (-0.62z)| norm 0.2699 (-0.41z)| lr 1.97e-05 | 322.41 ms | 52.3% bf16 MFU | 1624362 tok/s
step 17374/19560 | loss 3.275580 (-0.37z)| norm 0.2600 (-0.77z)| lr 1.97e-05 | 323.42 ms | 52.2% bf16 MFU | 1624198 tok/s
step 17375/19560 | loss 3.412144 (+1.90z)| norm 0.3230 (+1.51z)| lr 1.97e-05 | 323.11 ms | 52.2% bf16 MFU | 1624121 tok/s
step 17376/19560 | loss 3.260322 (-0.64z)| norm 0.2511 (-1.10z)| lr 1.97e-05 | 322.78 ms | 52.3% bf16 MFU | 1624128 tok/s
step 17377/19560 | loss 3.356745 (+0.96z)| norm 0.2495 (-1.14z)| lr 1.96e-05 | 322.40 ms | 52.3% bf16 MFU | 1624231 tok/s
step 17378/19560 | loss 3.237842 (-1.03z)| norm 0.3289 (+1.69z)| lr 1.96e-05 | 323.31 ms | 52.2% bf16 MFU | 1624100 tok/s
step 17379/19560 | loss 3.270169 (-0.48z)| norm 0.2520 (-1.06z)| lr 1.96e-05 | 322.80 ms | 52.3% bf16 MFU | 1624104 tok/s
step 17380/19560 | loss 3.327871 (+0.47z)| norm 0.2937 (+0.43z)| lr 1.96e-05 | 322.63 ms | 52.3% bf16 MFU | 1624152 tok/s
step 17381/19560 | loss 3.265551 (-0.56z)| norm 0.2541 (-0.98z)| lr 1.96e-05 | 322.66 ms | 52.3% bf16 MFU | 1624188 tok/s
step 17382/19560 | loss 3.303807 (+0.07z)| norm 0.2981 (+0.59z)| lr 1.95e-05 | 322.32 ms | 52.4% bf16 MFU | 1624310 tok/s
step 17383/19560 | loss 3.286039 (-0.22z)| norm 0.2790 (-0.09z)| lr 1.95e-05 | 322.94 ms | 52.3% bf16 MFU | 1624269 tok/s
step 17384/19560 | loss 3.346969 (+0.79z)| norm 0.2419 (-1.39z)| lr 1.95e-05 | 323.61 ms | 52.2% bf16 MFU | 1624061 tok/s
step 17385/19560 | loss 3.274573 (-0.43z)| norm 0.2706 (-0.36z)| lr 1.95e-05 | 322.87 ms | 52.3% bf16 MFU | 1624049 tok/s
step 17386/19560 | loss 3.338462 (+0.64z)| norm 0.2947 (+0.52z)| lr 1.95e-05 | 323.43 ms | 52.2% bf16 MFU | 1623898 tok/s
step 17387/19560 | loss 3.313376 (+0.21z)| norm 0.2478 (-1.19z)| lr 1.95e-05 | 322.08 ms | 52.4% bf16 MFU | 1624095 tok/s
step 17388/19560 | loss 3.415120 (+1.90z)| norm 0.2749 (-0.21z)| lr 1.94e-05 | 322.68 ms | 52.3% bf16 MFU | 1624129 tok/s
step 17389/19560 | loss 3.263247 (-0.64z)| norm 0.3324 (+1.85z)| lr 1.94e-05 | 323.13 ms | 52.2% bf16 MFU | 1624048 tok/s
step 17390/19560 | loss 3.251910 (-0.82z)| norm 0.3299 (+1.73z)| lr 1.94e-05 | 323.12 ms | 52.2% bf16 MFU | 1623973 tok/s
step 17391/19560 | loss 3.337412 (+0.60z)| norm 0.2609 (-0.72z)| lr 1.94e-05 | 322.42 ms | 52.3% bf16 MFU | 1624079 tok/s
step 17392/19560 | loss 3.313140 (+0.19z)| norm 0.3124 (+1.10z)| lr 1.94e-05 | 322.77 ms | 52.3% bf16 MFU | 1624093 tok/s
step 17393/19560 | loss 3.273806 (-0.46z)| norm 0.2933 (+0.42z)| lr 1.94e-05 | 322.87 ms | 52.3% bf16 MFU | 1624079 tok/s
step 17394/19560 | loss 3.272525 (-0.49z)| norm 0.2451 (-1.27z)| lr 1.93e-05 | 322.63 ms | 52.3% bf16 MFU | 1624126 tok/s
step 17395/19560 | loss 3.238048 (-1.06z)| norm 0.2677 (-0.47z)| lr 1.93e-05 | 322.73 ms | 52.3% bf16 MFU | 1624146 tok/s
step 17396/19560 | loss 3.302691 (+0.02z)| norm 0.2888 (+0.26z)| lr 1.93e-05 | 323.24 ms | 52.2% bf16 MFU | 1624038 tok/s
step 17397/19560 | loss 3.313158 (+0.19z)| norm 0.2812 (-0.02z)| lr 1.93e-05 | 322.55 ms | 52.3% bf16 MFU | 1624109 tok/s
step 17398/19560 | loss 3.294155 (-0.13z)| norm 0.2466 (-1.25z)| lr 1.93e-05 | 322.99 ms | 52.3% bf16 MFU | 1624064 tok/s
step 17399/19560 | loss 3.248007 (-0.89z)| norm 0.3003 (+0.66z)| lr 1.92e-05 | 322.65 ms | 52.3% bf16 MFU | 1624107 tok/s
step 17400/19560 | loss 3.279714 (-0.36z)| norm 0.2550 (-0.94z)| lr 1.92e-05 | 322.92 ms | 52.3% bf16 MFU | 1624081 tok/s
step 17401/19560 | loss 3.315958 (+0.30z)| norm 0.2903 (+0.32z)| lr 1.92e-05 | 323.33 ms | 52.2% bf16 MFU | 1623954 tok/s
step 17402/19560 | loss 3.304014 (+0.08z)| norm 0.2730 (-0.31z)| lr 1.92e-05 | 322.94 ms | 52.3% bf16 MFU | 1623932 tok/s
step 17403/19560 | loss 3.336856 (+0.69z)| norm 0.3388 (+2.03z)| lr 1.92e-05 | 322.81 ms | 52.3% bf16 MFU | 1623942 tok/s
step 17404/19560 | loss 3.271593 (-0.50z)| norm 0.2547 (-0.95z)| lr 1.92e-05 | 322.47 ms | 52.3% bf16 MFU | 1624036 tok/s
step 17405/19560 | loss 3.252200 (-0.84z)| norm 0.2572 (-0.86z)| lr 1.91e-05 | 323.30 ms | 52.2% bf16 MFU | 1623919 tok/s
step 17406/19560 | loss 3.298379 (-0.00z)| norm 0.2711 (-0.36z)| lr 1.91e-05 | 322.11 ms | 52.4% bf16 MFU | 1624108 tok/s
step 17407/19560 | loss 3.263458 (-0.65z)| norm 0.2897 (+0.29z)| lr 1.91e-05 | 323.35 ms | 52.2% bf16 MFU | 1623974 tok/s
step 17408/19560 | loss 3.268142 (-0.57z)| norm 0.2460 (-1.24z)| lr 1.91e-05 | 323.20 ms | 52.2% bf16 MFU | 1623884 tok/s
step 17409/19560 | loss 3.322747 (+0.43z)| norm 0.2574 (-0.83z)| lr 1.91e-05 | 322.55 ms | 52.3% bf16 MFU | 1623963 tok/s
step 17410/19560 | loss 3.250529 (-0.90z)| norm 0.3065 (+0.88z)| lr 1.91e-05 | 322.65 ms | 52.3% bf16 MFU | 1624010 tok/s
step 17411/19560 | loss 3.345227 (+0.84z)| norm 0.2836 (+0.07z)| lr 1.90e-05 | 322.51 ms | 52.3% bf16 MFU | 1624091 tok/s
step 17412/19560 | loss 3.248379 (-0.95z)| norm 0.2594 (-0.78z)| lr 1.90e-05 | 323.03 ms | 52.2% bf16 MFU | 1624039 tok/s
step 17413/19560 | loss 3.309551 (+0.17z)| norm 0.2773 (-0.16z)| lr 1.90e-05 | 322.60 ms | 52.3% bf16 MFU | 1624097 tok/s
step 17414/19560 | loss 3.261245 (-0.76z)| norm 0.2925 (+0.44z)| lr 1.90e-05 | 322.95 ms | 52.3% bf16 MFU | 1624064 tok/s
step 17415/19560 | loss 3.292654 (-0.10z)| norm 0.3025 (+0.82z)| lr 1.90e-05 | 323.68 ms | 52.1% bf16 MFU | 1623849 tok/s
step 17416/19560 | loss 3.322845 (+0.52z)| norm 0.2483 (-1.23z)| lr 1.89e-05 | 322.62 ms | 52.3% bf16 MFU | 1623910 tok/s
step 17417/19560 | loss 3.273798 (-0.49z)| norm 0.3661 (+3.14z)| lr 1.89e-05 | 322.44 ms | 52.3% bf16 MFU | 1624015 tok/s
step 17418/19560 | loss 3.324542 (+0.58z)| norm 0.2691 (-0.44z)| lr 1.89e-05 | 322.39 ms | 52.3% bf16 MFU | 1624127 tok/s
step 17419/19560 | loss 3.313137 (+0.34z)| norm 0.2648 (-0.60z)| lr 1.89e-05 | 323.09 ms | 52.2% bf16 MFU | 1624056 tok/s
step 17420/19560 | loss 3.271832 (-0.52z)| norm 0.2612 (-0.73z)| lr 1.89e-05 | 323.36 ms | 52.2% bf16 MFU | 1623923 tok/s
step 17421/19560 | loss 3.267069 (-0.61z)| norm 0.2744 (-0.24z)| lr 1.89e-05 | 322.54 ms | 52.3% bf16 MFU | 1624000 tok/s
step 17422/19560 | loss 3.329983 (+0.72z)| norm 0.2996 (+0.68z)| lr 1.88e-05 | 322.99 ms | 52.3% bf16 MFU | 1623962 tok/s
step 17423/19560 | loss 3.334794 (+0.82z)| norm 0.2507 (-1.11z)| lr 1.88e-05 | 322.89 ms | 52.3% bf16 MFU | 1623950 tok/s
step 17424/19560 | loss 3.294086 (-0.04z)| norm 0.2549 (-0.96z)| lr 1.88e-05 | 323.02 ms | 52.2% bf16 MFU | 1623906 tok/s
step 17425/19560 | loss 3.329564 (+0.71z)| norm 0.3201 (+1.41z)| lr 1.88e-05 | 323.30 ms | 52.2% bf16 MFU | 1623795 tok/s
step 17426/19560 | loss 3.254362 (-0.91z)| norm 0.2793 (-0.09z)| lr 1.88e-05 | 322.71 ms | 52.3% bf16 MFU | 1623838 tok/s
step 17427/19560 | loss 3.287480 (-0.20z)| norm 0.2806 (-0.04z)| lr 1.88e-05 | 323.71 ms | 52.1% bf16 MFU | 1623628 tok/s
step 17428/19560 | loss 3.309821 (+0.29z)| norm 0.2568 (-0.92z)| lr 1.87e-05 | 322.92 ms | 52.3% bf16 MFU | 1623626 tok/s
step 17429/19560 | loss 3.302241 (+0.11z)| norm 0.2615 (-0.74z)| lr 1.87e-05 | 322.83 ms | 52.3% bf16 MFU | 1623648 tok/s
step 17430/19560 | loss 3.235413 (-1.32z)| norm 0.3010 (+0.70z)| lr 1.87e-05 | 323.32 ms | 52.2% bf16 MFU | 1623544 tok/s
step 17431/19560 | loss 3.353914 (+1.23z)| norm 0.2593 (-0.83z)| lr 1.87e-05 | 322.96 ms | 52.3% bf16 MFU | 1623536 tok/s
step 17432/19560 | loss 3.271202 (-0.55z)| norm 0.2538 (-1.01z)| lr 1.87e-05 | 323.25 ms | 52.2% bf16 MFU | 1623455 tok/s
step 17433/19560 | loss 3.343831 (+1.00z)| norm 0.2664 (-0.55z)| lr 1.87e-05 | 323.28 ms | 52.2% bf16 MFU | 1623372 tok/s
step 17434/19560 | loss 3.340173 (+1.01z)| norm 0.2585 (-0.83z)| lr 1.86e-05 | 322.83 ms | 52.3% bf16 MFU | 1623405 tok/s
step 17435/19560 | loss 3.350822 (+1.24z)| norm 0.3120 (+1.11z)| lr 1.86e-05 | 322.96 ms | 52.3% bf16 MFU | 1623404 tok/s
step 17436/19560 | loss 3.339499 (+0.96z)| norm 0.2625 (-0.68z)| lr 1.86e-05 | 323.36 ms | 52.2% bf16 MFU | 1623304 tok/s
step 17437/19560 | loss 3.330556 (+0.75z)| norm 0.2676 (-0.50z)| lr 1.86e-05 | 322.85 ms | 52.3% bf16 MFU | 1623336 tok/s
step 17438/19560 | loss 3.326123 (+0.65z)| norm 0.2877 (+0.22z)| lr 1.86e-05 | 323.00 ms | 52.3% bf16 MFU | 1623327 tok/s
step 17439/19560 | loss 3.321723 (+0.53z)| norm 0.2561 (-0.93z)| lr 1.85e-05 | 323.20 ms | 52.2% bf16 MFU | 1623270 tok/s
step 17440/19560 | loss 3.247792 (-1.16z)| norm 0.2467 (-1.25z)| lr 1.85e-05 | 323.26 ms | 52.2% bf16 MFU | 1623200 tok/s
step 17441/19560 | loss 3.239688 (-1.33z)| norm 0.2412 (-1.44z)| lr 1.85e-05 | 323.17 ms | 52.2% bf16 MFU | 1623155 tok/s
step 17442/19560 | loss 3.363706 (+1.50z)| norm 0.3855 (+3.56z)| lr 1.85e-05 | 322.67 ms | 52.3% bf16 MFU | 1623241 tok/s
step 17443/19560 | loss 3.271386 (-0.60z)| norm 0.2660 (-0.54z)| lr 1.85e-05 | 323.66 ms | 52.1% bf16 MFU | 1623074 tok/s
step 17444/19560 | loss 3.322494 (+0.56z)| norm 0.2626 (-0.65z)| lr 1.85e-05 | 322.21 ms | 52.4% bf16 MFU | 1623278 tok/s
step 17445/19560 | loss 3.284584 (-0.29z)| norm 0.3268 (+1.53z)| lr 1.84e-05 | 322.53 ms | 52.3% bf16 MFU | 1623391 tok/s
step 17446/19560 | loss 3.250668 (-1.06z)| norm 0.2664 (-0.52z)| lr 1.84e-05 | 323.04 ms | 52.2% bf16 MFU | 1623371 tok/s
step 17447/19560 | loss 3.325049 (+0.62z)| norm 0.2631 (-0.63z)| lr 1.84e-05 | 322.43 ms | 52.3% bf16 MFU | 1623505 tok/s
step 17448/19560 | loss 3.360921 (+1.42z)| norm 0.3165 (+1.17z)| lr 1.84e-05 | 323.09 ms | 52.2% bf16 MFU | 1623466 tok/s
step 17449/19560 | loss 3.278292 (-0.47z)| norm 0.2437 (-1.27z)| lr 1.84e-05 | 323.16 ms | 52.2% bf16 MFU | 1623412 tok/s
step 17450/19560 | loss 3.274863 (-0.56z)| norm 0.2559 (-0.87z)| lr 1.84e-05 | 322.66 ms | 52.3% bf16 MFU | 1623486 tok/s
step 17451/19560 | loss 3.235728 (-1.44z)| norm 0.3757 (+3.03z)| lr 1.83e-05 | 323.27 ms | 52.2% bf16 MFU | 1623404 tok/s
step 17452/19560 | loss 3.257309 (-0.94z)| norm 0.2546 (-0.90z)| lr 1.83e-05 | 323.00 ms | 52.3% bf16 MFU | 1623392 tok/s
step 17453/19560 | loss 3.368227 (+1.57z)| norm 0.3404 (+1.85z)| lr 1.83e-05 | 322.51 ms | 52.3% bf16 MFU | 1623505 tok/s
step 17454/19560 | loss 3.284333 (-0.33z)| norm 0.3285 (+1.44z)| lr 1.83e-05 | 322.86 ms | 52.3% bf16 MFU | 1623524 tok/s
step 17455/19560 | loss 3.326195 (+0.62z)| norm 0.2603 (-0.74z)| lr 1.83e-05 | 322.73 ms | 52.3% bf16 MFU | 1623574 tok/s
step 17456/19560 | loss 3.312362 (+0.29z)| norm 0.2832 (-0.01z)| lr 1.83e-05 | 322.68 ms | 52.3% bf16 MFU | 1623636 tok/s
step 17457/19560 | loss 3.321769 (+0.50z)| norm 0.3068 (+0.73z)| lr 1.82e-05 | 323.23 ms | 52.2% bf16 MFU | 1623556 tok/s
step 17458/19560 | loss 3.222207 (-1.76z)| norm 0.2669 (-0.53z)| lr 1.82e-05 | 322.78 ms | 52.3% bf16 MFU | 1623593 tok/s
step 17459/19560 | loss 3.412818 (+2.51z)| norm 0.3552 (+2.26z)| lr 1.82e-05 | 322.73 ms | 52.3% bf16 MFU | 1623639 tok/s
step 17460/19560 | loss 3.296821 (-0.06z)| norm 0.2835 (-0.02z)| lr 1.82e-05 | 322.56 ms | 52.3% bf16 MFU | 1623726 tok/s
step 17461/19560 | loss 3.274013 (-0.59z)| norm 0.2889 (+0.16z)| lr 1.82e-05 | 322.99 ms | 52.3% bf16 MFU | 1623702 tok/s
step 17462/19560 | loss 3.270903 (-0.65z)| norm 0.3363 (+1.67z)| lr 1.82e-05 | 322.40 ms | 52.3% bf16 MFU | 1623827 tok/s
step 17463/19560 | loss 3.267529 (-0.72z)| norm 0.2629 (-0.67z)| lr 1.81e-05 | 322.96 ms | 52.3% bf16 MFU | 1623804 tok/s
step 17464/19560 | loss 3.428881 (+2.83z)| norm 0.2927 (+0.29z)| lr 1.81e-05 | 321.83 ms | 52.4% bf16 MFU | 1624068 tok/s
step 17465/19560 | loss 3.286092 (-0.32z)| norm 0.2919 (+0.28z)| lr 1.81e-05 | 322.68 ms | 52.3% bf16 MFU | 1624105 tok/s
step 17466/19560 | loss 3.309715 (+0.20z)| norm 0.2947 (+0.37z)| lr 1.81e-05 | 323.34 ms | 52.2% bf16 MFU | 1623974 tok/s
step 17467/19560 | loss 3.310405 (+0.22z)| norm 0.2886 (+0.18z)| lr 1.81e-05 | 322.53 ms | 52.3% bf16 MFU | 1624054 tok/s
step 17468/19560 | loss 3.274753 (-0.57z)| norm 0.3048 (+0.70z)| lr 1.80e-05 | 322.38 ms | 52.4% bf16 MFU | 1624165 tok/s
step 17469/19560 | loss 3.235530 (-1.42z)| norm 0.3322 (+1.56z)| lr 1.80e-05 | 322.45 ms | 52.3% bf16 MFU | 1624255 tok/s
step 17470/19560 | loss 3.269600 (-0.66z)| norm 0.2691 (-0.47z)| lr 1.80e-05 | 323.17 ms | 52.2% bf16 MFU | 1624159 tok/s
step 17471/19560 | loss 3.268202 (-0.68z)| norm 0.3156 (+1.02z)| lr 1.80e-05 | 322.61 ms | 52.3% bf16 MFU | 1624207 tok/s
step 17472/19560 | loss 3.236461 (-1.37z)| norm 0.2842 (+0.01z)| lr 1.80e-05 | 322.52 ms | 52.3% bf16 MFU | 1624278 tok/s
step 17473/19560 | loss 3.284055 (-0.32z)| norm 0.3013 (+0.55z)| lr 1.80e-05 | 322.79 ms | 52.3% bf16 MFU | 1624275 tok/s
step 17474/19560 | loss 3.263245 (-0.77z)| norm 0.2671 (-0.54z)| lr 1.79e-05 | 322.56 ms | 52.3% bf16 MFU | 1624330 tok/s
step 17475/19560 | loss 3.365033 (+1.45z)| norm 0.2750 (-0.28z)| lr 1.79e-05 | 322.48 ms | 52.3% bf16 MFU | 1624404 tok/s
step 17476/19560 | loss 3.305406 (+0.12z)| norm 0.2650 (-0.60z)| lr 1.79e-05 | 322.94 ms | 52.3% bf16 MFU | 1624357 tok/s
step 17477/19560 | loss 3.411692 (+2.40z)| norm 0.2773 (-0.19z)| lr 1.79e-05 | 322.52 ms | 52.3% bf16 MFU | 1624420 tok/s
step 17478/19560 | loss 3.261408 (-0.84z)| norm 0.2749 (-0.26z)| lr 1.79e-05 | 322.93 ms | 52.3% bf16 MFU | 1624377 tok/s
step 17479/19560 | loss 3.386892 (+1.84z)| norm 0.2793 (-0.12z)| lr 1.79e-05 | 322.61 ms | 52.3% bf16 MFU | 1624416 tok/s
step 17480/19560 | loss 3.239920 (-1.31z)| norm 0.2561 (-0.88z)| lr 1.78e-05 | 323.02 ms | 52.2% bf16 MFU | 1624348 tok/s
step 17481/19560 | loss 3.274385 (-0.56z)| norm 0.2371 (-1.49z)| lr 1.78e-05 | 322.45 ms | 52.3% bf16 MFU | 1624427 tok/s
step 17482/19560 | loss 3.265834 (-0.73z)| norm 0.3261 (+1.42z)| lr 1.78e-05 | 322.44 ms | 52.3% bf16 MFU | 1624507 tok/s
step 17483/19560 | loss 3.326741 (+0.57z)| norm 0.2807 (-0.06z)| lr 1.78e-05 | 323.19 ms | 52.2% bf16 MFU | 1624393 tok/s
step 17484/19560 | loss 3.270492 (-0.64z)| norm 0.2438 (-1.25z)| lr 1.78e-05 | 322.56 ms | 52.3% bf16 MFU | 1624443 tok/s
step 17485/19560 | loss 3.236870 (-1.36z)| norm 0.2763 (-0.18z)| lr 1.78e-05 | 322.32 ms | 52.4% bf16 MFU | 1624552 tok/s
step 17486/19560 | loss 3.304814 (+0.10z)| norm 0.2554 (-0.88z)| lr 1.77e-05 | 323.09 ms | 52.2% bf16 MFU | 1624461 tok/s
step 17487/19560 | loss 3.236747 (-1.34z)| norm 0.3000 (+0.63z)| lr 1.77e-05 | 322.92 ms | 52.3% bf16 MFU | 1624418 tok/s
step 17488/19560 | loss 3.231321 (-1.44z)| norm 0.3488 (+2.23z)| lr 1.77e-05 | 322.36 ms | 52.4% bf16 MFU | 1624518 tok/s
step 17489/19560 | loss 3.370585 (+1.50z)| norm 0.2782 (-0.13z)| lr 1.77e-05 | 322.55 ms | 52.3% bf16 MFU | 1624564 tok/s
step 17490/19560 | loss 3.309535 (+0.22z)| norm 0.2357 (-1.52z)| lr 1.77e-05 | 323.09 ms | 52.2% bf16 MFU | 1624473 tok/s
step 17491/19560 | loss 3.268968 (-0.63z)| norm 0.2622 (-0.64z)| lr 1.77e-05 | 322.23 ms | 52.4% bf16 MFU | 1624602 tok/s
step 17492/19560 | loss 3.261447 (-0.78z)| norm 0.2812 (-0.01z)| lr 1.76e-05 | 322.71 ms | 52.3% bf16 MFU | 1624605 tok/s
step 17493/19560 | loss 3.335372 (+0.78z)| norm 0.3003 (+0.62z)| lr 1.76e-05 | 322.90 ms | 52.3% bf16 MFU | 1624559 tok/s
step 17494/19560 | loss 3.324618 (+0.54z)| norm 0.2468 (-1.14z)| lr 1.76e-05 | 323.14 ms | 52.2% bf16 MFU | 1624456 tok/s
step 17495/19560 | loss 3.245024 (-1.15z)| norm 0.2557 (-0.84z)| lr 1.76e-05 | 322.58 ms | 52.3% bf16 MFU | 1624497 tok/s
step 17496/19560 | loss 3.253183 (-0.96z)| norm 0.2606 (-0.67z)| lr 1.76e-05 | 322.55 ms | 52.3% bf16 MFU | 1624546 tok/s
step 17497/19560 | loss 3.291315 (-0.15z)| norm 0.3389 (+1.85z)| lr 1.76e-05 | 323.08 ms | 52.2% bf16 MFU | 1624457 tok/s
step 17498/19560 | loss 3.276176 (-0.47z)| norm 0.2724 (-0.31z)| lr 1.75e-05 | 322.94 ms | 52.3% bf16 MFU | 1624407 tok/s
step 17499/19560 | loss 3.315861 (+0.37z)| norm 0.2460 (-1.16z)| lr 1.75e-05 | 322.88 ms | 52.3% bf16 MFU | 1624376 tok/s
step 17500/19560 | loss 3.281781 (-0.34z)| norm 0.2590 (-0.72z)| lr 1.75e-05 | 322.37 ms | 52.4% bf16 MFU | 1624474 tok/s
val loss 3.279209
 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Hel 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating Helevaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3056/10042 = 0.304322
step 17501/19560 | loss 3.276967 (-0.46z)| norm 0.2943 (+0.42z)| lr 1.75e-05 | 322.54 ms | 52.3% bf16 MFU | 1624526 tok/s
step 17502/19560 | loss 3.321721 (+0.56z)| norm 0.3638 (+2.57z)| lr 1.75e-05 | 322.88 ms | 52.3% bf16 MFU | 1624488 tok/s
step 17503/19560 | loss 3.277177 (-0.45z)| norm 0.2895 (+0.24z)| lr 1.75e-05 | 323.04 ms | 52.2% bf16 MFU | 1624412 tok/s
step 17504/19560 | loss 3.280571 (-0.37z)| norm 0.2905 (+0.26z)| lr 1.74e-05 | 322.39 ms | 52.4% bf16 MFU | 1624504 tok/s
step 17505/19560 | loss 3.229156 (-1.56z)| norm 0.2961 (+0.43z)| lr 1.74e-05 | 322.36 ms | 52.4% bf16 MFU | 1624600 tok/s
step 17506/19560 | loss 3.291233 (-0.11z)| norm 0.3246 (+1.35z)| lr 1.74e-05 | 322.69 ms | 52.3% bf16 MFU | 1624607 tok/s
step 17507/19560 | loss 3.284943 (-0.26z)| norm 0.2541 (-0.92z)| lr 1.74e-05 | 322.85 ms | 52.3% bf16 MFU | 1624574 tok/s
step 17508/19560 | loss 3.435487 (+3.17z)| norm 0.2983 (+0.50z)| lr 1.74e-05 | 322.97 ms | 52.3% bf16 MFU | 1624511 tok/s
step 17509/19560 | loss 3.364060 (+1.51z)| norm 0.3280 (+1.43z)| lr 1.74e-05 | 322.34 ms | 52.4% bf16 MFU | 1624612 tok/s
step 17510/19560 | loss 3.259096 (-0.86z)| norm 0.2900 (+0.22z)| lr 1.73e-05 | 322.96 ms | 52.3% bf16 MFU | 1624550 tok/s
step 17511/19560 | loss 3.288260 (-0.20z)| norm 0.2587 (-0.78z)| lr 1.73e-05 | 322.32 ms | 52.4% bf16 MFU | 1624653 tok/s
step 17512/19560 | loss 3.344691 (+1.08z)| norm 0.3797 (+2.96z)| lr 1.73e-05 | 322.70 ms | 52.3% bf16 MFU | 1624654 tok/s
step 17513/19560 | loss 3.235357 (-1.39z)| norm 0.2500 (-1.05z)| lr 1.73e-05 | 322.67 ms | 52.3% bf16 MFU | 1624663 tok/s
step 17514/19560 | loss 3.272874 (-0.53z)| norm 0.2429 (-1.25z)| lr 1.73e-05 | 322.13 ms | 52.4% bf16 MFU | 1624809 tok/s
step 17515/19560 | loss 3.227415 (-1.53z)| norm 0.2967 (+0.39z)| lr 1.73e-05 | 323.15 ms | 52.2% bf16 MFU | 1624689 tok/s
step 17516/19560 | loss 3.303832 (+0.21z)| norm 0.3678 (+2.50z)| lr 1.72e-05 | 322.72 ms | 52.3% bf16 MFU | 1624684 tok/s
step 17517/19560 | loss 3.306866 (+0.27z)| norm 0.2767 (-0.23z)| lr 1.72e-05 | 322.30 ms | 52.4% bf16 MFU | 1624786 tok/s
step 17518/19560 | loss 3.277580 (-0.41z)| norm 0.4015 (+3.39z)| lr 1.72e-05 | 323.07 ms | 52.2% bf16 MFU | 1624688 tok/s
step 17519/19560 | loss 3.273755 (-0.49z)| norm 0.3433 (+1.67z)| lr 1.72e-05 | 322.60 ms | 52.3% bf16 MFU | 1624713 tok/s
step 17520/19560 | loss 3.327467 (+0.75z)| norm 0.3005 (+0.44z)| lr 1.72e-05 | 322.33 ms | 52.4% bf16 MFU | 1624804 tok/s
step 17521/19560 | loss 3.288753 (-0.15z)| norm 0.2848 (-0.01z)| lr 1.72e-05 | 323.09 ms | 52.2% bf16 MFU | 1624700 tok/s
step 17522/19560 | loss 3.372236 (+1.75z)| norm 0.3184 (+0.94z)| lr 1.71e-05 | 322.68 ms | 52.3% bf16 MFU | 1624704 tok/s
step 17523/19560 | loss 3.271168 (-0.58z)| norm 0.2631 (-0.66z)| lr 1.71e-05 | 322.62 ms | 52.3% bf16 MFU | 1624724 tok/s
step 17524/19560 | loss 3.352932 (+1.29z)| norm 0.2902 (+0.13z)| lr 1.71e-05 | 322.27 ms | 52.4% bf16 MFU | 1624831 tok/s
step 17525/19560 | loss 3.234721 (-1.39z)| norm 0.2716 (-0.41z)| lr 1.71e-05 | 322.39 ms | 52.4% bf16 MFU | 1624902 tok/s
step 17526/19560 | loss 3.257600 (-0.86z)| norm 0.2764 (-0.28z)| lr 1.71e-05 | 322.61 ms | 52.3% bf16 MFU | 1624915 tok/s
step 17527/19560 | loss 3.281824 (-0.32z)| norm 0.2397 (-1.32z)| lr 1.71e-05 | 322.50 ms | 52.3% bf16 MFU | 1624953 tok/s
step 17528/19560 | loss 3.276271 (-0.45z)| norm 0.2529 (-0.94z)| lr 1.70e-05 | 322.46 ms | 52.3% bf16 MFU | 1625001 tok/s
step 17529/19560 | loss 3.353963 (+1.31z)| norm 0.2561 (-0.84z)| lr 1.70e-05 | 322.42 ms | 52.3% bf16 MFU | 1625056 tok/s
step 17530/19560 | loss 3.275576 (-0.46z)| norm 0.2706 (-0.42z)| lr 1.70e-05 | 322.88 ms | 52.3% bf16 MFU | 1624993 tok/s
step 17531/19560 | loss 3.328554 (+0.74z)| norm 0.2388 (-1.32z)| lr 1.70e-05 | 322.62 ms | 52.3% bf16 MFU | 1624997 tok/s
step 17532/19560 | loss 3.249938 (-1.03z)| norm 0.2544 (-0.87z)| lr 1.70e-05 | 322.71 ms | 52.3% bf16 MFU | 1624980 tok/s
step 17533/19560 | loss 3.357216 (+1.37z)| norm 0.2617 (-0.66z)| lr 1.70e-05 | 322.58 ms | 52.3% bf16 MFU | 1624994 tok/s
step 17534/19560 | loss 3.329868 (+0.74z)| norm 0.2462 (-1.10z)| lr 1.69e-05 | 321.89 ms | 52.4% bf16 MFU | 1625184 tok/s
step 17535/19560 | loss 3.249499 (-1.06z)| norm 0.2448 (-1.12z)| lr 1.69e-05 | 323.02 ms | 52.2% bf16 MFU | 1625080 tok/s
step 17536/19560 | loss 3.320623 (+0.53z)| norm 0.2563 (-0.80z)| lr 1.69e-05 | 322.71 ms | 52.3% bf16 MFU | 1625057 tok/s
step 17537/19560 | loss 3.343237 (+1.03z)| norm 0.2660 (-0.52z)| lr 1.69e-05 | 322.76 ms | 52.3% bf16 MFU | 1625024 tok/s
step 17538/19560 | loss 3.297152 (-0.01z)| norm 0.2534 (-0.87z)| lr 1.69e-05 | 322.69 ms | 52.3% bf16 MFU | 1625010 tok/s
step 17539/19560 | loss 3.255348 (-0.93z)| norm 0.2446 (-1.11z)| lr 1.69e-05 | 322.69 ms | 52.3% bf16 MFU | 1624996 tok/s
step 17540/19560 | loss 3.301646 (+0.10z)| norm 0.2869 (+0.09z)| lr 1.68e-05 | 322.43 ms | 52.3% bf16 MFU | 1625050 tok/s
step 17541/19560 | loss 3.189073 (-2.37z)| norm 0.2715 (-0.35z)| lr 1.68e-05 | 322.96 ms | 52.3% bf16 MFU | 1624967 tok/s
step 17542/19560 | loss 3.299287 (+0.06z)| norm 0.2670 (-0.47z)| lr 1.68e-05 | 322.14 ms | 52.4% bf16 MFU | 1625095 tok/s
step 17543/19560 | loss 3.283299 (-0.29z)| norm 0.2570 (-0.74z)| lr 1.68e-05 | 323.21 ms | 52.2% bf16 MFU | 1624946 tok/s
step 17544/19560 | loss 3.283462 (-0.28z)| norm 0.2671 (-0.46z)| lr 1.68e-05 | 322.11 ms | 52.4% bf16 MFU | 1625082 tok/s
step 17545/19560 | loss 3.307147 (+0.23z)| norm 0.2568 (-0.74z)| lr 1.68e-05 | 323.13 ms | 52.2% bf16 MFU | 1624954 tok/s
step 17546/19560 | loss 3.265800 (-0.67z)| norm 0.2873 (+0.14z)| lr 1.67e-05 | 322.61 ms | 52.3% bf16 MFU | 1624964 tok/s
step 17547/19560 | loss 3.284673 (-0.25z)| norm 0.2554 (-0.79z)| lr 1.67e-05 | 322.34 ms | 52.4% bf16 MFU | 1625042 tok/s
step 17548/19560 | loss 3.411373 (+2.48z)| norm 0.3293 (+1.34z)| lr 1.67e-05 | 322.48 ms | 52.3% bf16 MFU | 1625080 tok/s
step 17549/19560 | loss 3.294090 (-0.07z)| norm 0.2821 (-0.03z)| lr 1.67e-05 | 322.85 ms | 52.3% bf16 MFU | 1625022 tok/s
step 17550/19560 | loss 3.247851 (-1.05z)| norm 0.3421 (+1.69z)| lr 1.67e-05 | 322.25 ms | 52.4% bf16 MFU | 1625118 tok/s
step 17551/19560 | loss 3.336198 (+0.86z)| norm 0.2544 (-0.83z)| lr 1.67e-05 | 322.58 ms | 52.3% bf16 MFU | 1625128 tok/s
step 17552/19560 | loss 3.244819 (-1.10z)| norm 0.2493 (-0.98z)| lr 1.66e-05 | 322.51 ms | 52.3% bf16 MFU | 1625155 tok/s
step 17553/19560 | loss 3.247010 (-1.04z)| norm 0.3563 (+2.06z)| lr 1.66e-05 | 322.68 ms | 52.3% bf16 MFU | 1625136 tok/s
step 17554/19560 | loss 3.259838 (-0.77z)| norm 0.3286 (+1.26z)| lr 1.66e-05 | 322.54 ms | 52.3% bf16 MFU | 1625155 tok/s
step 17555/19560 | loss 3.293670 (-0.04z)| norm 0.2892 (+0.15z)| lr 1.66e-05 | 322.45 ms | 52.3% bf16 MFU | 1625195 tok/s
step 17556/19560 | loss 3.291685 (-0.08z)| norm 0.2956 (+0.32z)| lr 1.66e-05 | 322.52 ms | 52.3% bf16 MFU | 1625214 tok/s
step 17557/19560 | loss 3.243031 (-1.11z)| norm 0.2937 (+0.26z)| lr 1.66e-05 | 322.44 ms | 52.3% bf16 MFU | 1625254 tok/s
step 17558/19560 | loss 3.324006 (+0.61z)| norm 0.3252 (+1.14z)| lr 1.65e-05 | 323.22 ms | 52.2% bf16 MFU | 1625095 tok/s
step 17559/19560 | loss 3.271109 (-0.52z)| norm 0.2361 (-1.36z)| lr 1.65e-05 | 323.00 ms | 52.3% bf16 MFU | 1624999 tok/s
step 17560/19560 | loss 3.290243 (-0.11z)| norm 0.2640 (-0.58z)| lr 1.65e-05 | 322.77 ms | 52.3% bf16 MFU | 1624965 tok/s
step 17561/19560 | loss 3.292993 (-0.04z)| norm 0.3229 (+1.06z)| lr 1.65e-05 | 323.05 ms | 52.2% bf16 MFU | 1624863 tok/s
step 17562/19560 | loss 3.285933 (-0.18z)| norm 0.2571 (-0.78z)| lr 1.65e-05 | 322.94 ms | 52.3% bf16 MFU | 1624793 tok/s
step 17563/19560 | loss 3.273129 (-0.45z)| norm 0.3024 (+0.49z)| lr 1.65e-05 | 323.37 ms | 52.2% bf16 MFU | 1624619 tok/s
step 17564/19560 | loss 3.279817 (-0.30z)| norm 0.2699 (-0.42z)| lr 1.64e-05 | 322.54 ms | 52.3% bf16 MFU | 1624664 tok/s
step 17565/19560 | loss 3.335787 (+0.93z)| norm 0.2554 (-0.83z)| lr 1.64e-05 | 322.61 ms | 52.3% bf16 MFU | 1624689 tok/s
step 17566/19560 | loss 3.295144 (+0.05z)| norm 0.2613 (-0.66z)| lr 1.64e-05 | 322.06 ms | 52.4% bf16 MFU | 1624851 tok/s
step 17567/19560 | loss 3.308891 (+0.35z)| norm 0.2486 (-1.01z)| lr 1.64e-05 | 322.51 ms | 52.3% bf16 MFU | 1624891 tok/s
step 17568/19560 | loss 3.296027 (+0.06z)| norm 0.2976 (+0.35z)| lr 1.64e-05 | 323.57 ms | 52.2% bf16 MFU | 1624662 tok/s
step 17569/19560 | loss 3.301206 (+0.16z)| norm 0.2940 (+0.24z)| lr 1.64e-05 | 322.82 ms | 52.3% bf16 MFU | 1624634 tok/s
step 17570/19560 | loss 3.349372 (+1.24z)| norm 0.2900 (+0.15z)| lr 1.63e-05 | 322.80 ms | 52.3% bf16 MFU | 1624611 tok/s
step 17571/19560 | loss 3.265335 (-0.63z)| norm 0.2886 (+0.11z)| lr 1.63e-05 | 323.23 ms | 52.2% bf16 MFU | 1624482 tok/s
step 17572/19560 | loss 3.290456 (-0.07z)| norm 0.2432 (-1.20z)| lr 1.63e-05 | 323.49 ms | 52.2% bf16 MFU | 1624295 tok/s
step 17573/19560 | loss 3.250753 (-0.95z)| norm 0.3350 (+1.45z)| lr 1.63e-05 | 322.39 ms | 52.4% bf16 MFU | 1624394 tok/s
step 17574/19560 | loss 3.322382 (+0.64z)| norm 0.3243 (+1.12z)| lr 1.63e-05 | 323.26 ms | 52.2% bf16 MFU | 1624267 tok/s
step 17575/19560 | loss 3.241513 (-1.15z)| norm 0.2507 (-0.99z)| lr 1.63e-05 | 323.01 ms | 52.3% bf16 MFU | 1624211 tok/s
step 17576/19560 | loss 3.265840 (-0.59z)| norm 0.3065 (+0.61z)| lr 1.63e-05 | 322.82 ms | 52.3% bf16 MFU | 1624204 tok/s
step 17577/19560 | loss 3.363395 (+1.56z)| norm 0.2663 (-0.55z)| lr 1.62e-05 | 323.40 ms | 52.2% bf16 MFU | 1624052 tok/s
step 17578/19560 | loss 3.236898 (-1.23z)| norm 0.2929 (+0.21z)| lr 1.62e-05 | 323.13 ms | 52.2% bf16 MFU | 1623977 tok/s
step 17579/19560 | loss 3.223850 (-1.52z)| norm 0.3037 (+0.56z)| lr 1.62e-05 | 322.73 ms | 52.3% bf16 MFU | 1624004 tok/s
step 17580/19560 | loss 3.244440 (-1.06z)| norm 0.2430 (-1.24z)| lr 1.62e-05 | 323.04 ms | 52.2% bf16 MFU | 1623952 tok/s
step 17581/19560 | loss 3.274903 (-0.38z)| norm 0.2486 (-1.06z)| lr 1.62e-05 | 323.32 ms | 52.2% bf16 MFU | 1623833 tok/s
step 17582/19560 | loss 3.284743 (-0.16z)| norm 0.2818 (-0.06z)| lr 1.62e-05 | 322.80 ms | 52.3% bf16 MFU | 1623850 tok/s
step 17583/19560 | loss 3.288937 (-0.06z)| norm 0.2676 (-0.49z)| lr 1.61e-05 | 322.96 ms | 52.3% bf16 MFU | 1623826 tok/s
step 17584/19560 | loss 3.357127 (+1.44z)| norm 0.2580 (-0.77z)| lr 1.61e-05 | 322.70 ms | 52.3% bf16 MFU | 1623870 tok/s
step 17585/19560 | loss 3.319142 (+0.60z)| norm 0.2840 (+0.01z)| lr 1.61e-05 | 323.03 ms | 52.2% bf16 MFU | 1623827 tok/s
step 17586/19560 | loss 3.350982 (+1.29z)| norm 0.2812 (-0.07z)| lr 1.61e-05 | 322.82 ms | 52.3% bf16 MFU | 1623841 tok/s
step 17587/19560 | loss 3.298217 (+0.14z)| norm 0.2704 (-0.38z)| lr 1.61e-05 | 322.72 ms | 52.3% bf16 MFU | 1623879 tok/s
step 17588/19560 | loss 3.253621 (-0.86z)| norm 0.2518 (-0.94z)| lr 1.61e-05 | 322.58 ms | 52.3% bf16 MFU | 1623949 tok/s
step 17589/19560 | loss 3.248000 (-0.98z)| norm 0.3176 (+1.05z)| lr 1.60e-05 | 322.75 ms | 52.3% bf16 MFU | 1623974 tok/s
step 17590/19560 | loss 3.350189 (+1.31z)| norm 0.2929 (+0.32z)| lr 1.60e-05 | 323.55 ms | 52.2% bf16 MFU | 1623797 tok/s
step 17591/19560 | loss 3.232245 (-1.33z)| norm 0.3316 (+1.48z)| lr 1.60e-05 | 323.24 ms | 52.2% bf16 MFU | 1623707 tok/s
step 17592/19560 | loss 3.248604 (-0.97z)| norm 0.2553 (-0.84z)| lr 1.60e-05 | 323.17 ms | 52.2% bf16 MFU | 1623637 tok/s
step 17593/19560 | loss 3.334223 (+1.01z)| norm 0.2708 (-0.36z)| lr 1.60e-05 | 322.86 ms | 52.3% bf16 MFU | 1623650 tok/s
step 17594/19560 | loss 3.232740 (-1.32z)| norm 0.3616 (+2.33z)| lr 1.60e-05 | 322.93 ms | 52.3% bf16 MFU | 1623645 tok/s
step 17595/19560 | loss 3.309077 (+0.44z)| norm 0.2902 (+0.21z)| lr 1.59e-05 | 323.12 ms | 52.2% bf16 MFU | 1623592 tok/s
step 17596/19560 | loss 3.273287 (-0.39z)| norm 0.2425 (-1.19z)| lr 1.59e-05 | 323.06 ms | 52.2% bf16 MFU | 1623556 tok/s
step 17597/19560 | loss 3.252698 (-0.87z)| norm 0.2913 (+0.26z)| lr 1.59e-05 | 322.88 ms | 52.3% bf16 MFU | 1623567 tok/s
step 17598/19560 | loss 3.319443 (+0.66z)| norm 0.2624 (-0.60z)| lr 1.59e-05 | 322.54 ms | 52.3% bf16 MFU | 1623664 tok/s
step 17599/19560 | loss 3.290865 (+0.00z)| norm 0.2819 (-0.01z)| lr 1.59e-05 | 323.21 ms | 52.2% bf16 MFU | 1623587 tok/s
step 17600/19560 | loss 3.326237 (+0.80z)| norm 0.2408 (-1.22z)| lr 1.59e-05 | 323.89 ms | 52.1% bf16 MFU | 1623343 tok/s
step 17601/19560 | loss 3.293943 (+0.05z)| norm 0.2741 (-0.22z)| lr 1.58e-05 | 322.31 ms | 52.4% bf16 MFU | 1623509 tok/s
step 17602/19560 | loss 3.311321 (+0.45z)| norm 0.3074 (+0.76z)| lr 1.58e-05 | 322.49 ms | 52.3% bf16 MFU | 1623620 tok/s
step 17603/19560 | loss 3.303067 (+0.27z)| norm 0.2534 (-0.84z)| lr 1.58e-05 | 323.57 ms | 52.2% bf16 MFU | 1623456 tok/s
step 17604/19560 | loss 3.300170 (+0.21z)| norm 0.2813 (-0.02z)| lr 1.58e-05 | 323.08 ms | 52.2% bf16 MFU | 1623423 tok/s
step 17605/19560 | loss 3.267035 (-0.56z)| norm 0.2937 (+0.35z)| lr 1.58e-05 | 323.40 ms | 52.2% bf16 MFU | 1623310 tok/s
step 17606/19560 | loss 3.280930 (-0.23z)| norm 0.3359 (+1.57z)| lr 1.58e-05 | 322.70 ms | 52.3% bf16 MFU | 1623378 tok/s
step 17607/19560 | loss 3.299757 (+0.25z)| norm 0.3197 (+1.08z)| lr 1.58e-05 | 322.68 ms | 52.3% bf16 MFU | 1623449 tok/s
step 17608/19560 | loss 3.315613 (+0.63z)| norm 0.2930 (+0.29z)| lr 1.57e-05 | 323.69 ms | 52.1% bf16 MFU | 1623262 tok/s
step 17609/19560 | loss 3.310129 (+0.48z)| norm 0.2885 (+0.15z)| lr 1.57e-05 | 322.64 ms | 52.3% bf16 MFU | 1623350 tok/s
step 17610/19560 | loss 3.269887 (-0.52z)| norm 0.3061 (+0.68z)| lr 1.57e-05 | 323.07 ms | 52.2% bf16 MFU | 1623325 tok/s
step 17611/19560 | loss 3.328640 (+0.94z)| norm 0.3674 (+2.42z)| lr 1.57e-05 | 322.48 ms | 52.3% bf16 MFU | 1623449 tok/s
step 17612/19560 | loss 3.262671 (-0.69z)| norm 0.2430 (-1.19z)| lr 1.57e-05 | 323.01 ms | 52.3% bf16 MFU | 1623433 tok/s
step 17613/19560 | loss 3.275917 (-0.38z)| norm 0.2455 (-1.11z)| lr 1.57e-05 | 323.35 ms | 52.2% bf16 MFU | 1623334 tok/s
step 17614/19560 | loss 3.251896 (-0.96z)| norm 0.3103 (+0.75z)| lr 1.56e-05 | 322.72 ms | 52.3% bf16 MFU | 1623396 tok/s
step 17615/19560 | loss 3.276057 (-0.37z)| norm 0.2833 (-0.02z)| lr 1.56e-05 | 322.97 ms | 52.3% bf16 MFU | 1623392 tok/s
step 17616/19560 | loss 3.286730 (-0.11z)| norm 0.2457 (-1.10z)| lr 1.56e-05 | 322.48 ms | 52.3% bf16 MFU | 1623512 tok/s
step 17617/19560 | loss 3.216428 (-1.87z)| norm 0.2442 (-1.13z)| lr 1.56e-05 | 322.32 ms | 52.4% bf16 MFU | 1623666 tok/s
step 17618/19560 | loss 3.270838 (-0.48z)| norm 0.3343 (+1.47z)| lr 1.56e-05 | 323.01 ms | 52.2% bf16 MFU | 1623638 tok/s
step 17619/19560 | loss 3.275213 (-0.37z)| norm 0.2793 (-0.13z)| lr 1.56e-05 | 322.93 ms | 52.3% bf16 MFU | 1623632 tok/s
step 17620/19560 | loss 3.243997 (-1.15z)| norm 0.2454 (-1.10z)| lr 1.55e-05 | 322.32 ms | 52.4% bf16 MFU | 1623782 tok/s
step 17621/19560 | loss 3.358520 (+1.73z)| norm 0.2610 (-0.64z)| lr 1.55e-05 | 322.59 ms | 52.3% bf16 MFU | 1623854 tok/s
step 17622/19560 | loss 3.267085 (-0.56z)| norm 0.2699 (-0.39z)| lr 1.55e-05 | 322.73 ms | 52.3% bf16 MFU | 1623889 tok/s
step 17623/19560 | loss 3.301085 (+0.28z)| norm 0.2849 (+0.04z)| lr 1.55e-05 | 322.64 ms | 52.3% bf16 MFU | 1623945 tok/s
step 17624/19560 | loss 3.333068 (+1.08z)| norm 0.2572 (-0.77z)| lr 1.55e-05 | 323.04 ms | 52.2% bf16 MFU | 1623897 tok/s
step 17625/19560 | loss 3.283640 (-0.17z)| norm 0.2813 (-0.06z)| lr 1.55e-05 | 323.02 ms | 52.2% bf16 MFU | 1623856 tok/s
step 17626/19560 | loss 3.350541 (+1.49z)| norm 0.3194 (+1.05z)| lr 1.54e-05 | 322.99 ms | 52.3% bf16 MFU | 1623826 tok/s
step 17627/19560 | loss 3.361983 (+1.75z)| norm 0.2462 (-1.09z)| lr 1.54e-05 | 322.74 ms | 52.3% bf16 MFU | 1623858 tok/s
step 17628/19560 | loss 3.314964 (+0.58z)| norm 0.2835 (-0.01z)| lr 1.54e-05 | 322.69 ms | 52.3% bf16 MFU | 1623902 tok/s
step 17629/19560 | loss 3.271255 (-0.50z)| norm 0.2712 (-0.36z)| lr 1.54e-05 | 322.20 ms | 52.4% bf16 MFU | 1624068 tok/s
step 17630/19560 | loss 3.237472 (-1.32z)| norm 0.2327 (-1.48z)| lr 1.54e-05 | 323.25 ms | 52.2% bf16 MFU | 1623960 tok/s
step 17631/19560 | loss 3.433085 (+3.32z)| norm 0.3481 (+1.91z)| lr 1.54e-05 | 322.28 ms | 52.4% bf16 MFU | 1624102 tok/s
step 17632/19560 | loss 3.315484 (+0.54z)| norm 0.2580 (-0.72z)| lr 1.54e-05 | 323.01 ms | 52.3% bf16 MFU | 1624055 tok/s
step 17633/19560 | loss 3.287192 (-0.13z)| norm 0.2620 (-0.60z)| lr 1.53e-05 | 322.39 ms | 52.3% bf16 MFU | 1624164 tok/s
step 17634/19560 | loss 3.251552 (-0.97z)| norm 0.2585 (-0.69z)| lr 1.53e-05 | 322.22 ms | 52.4% bf16 MFU | 1624312 tok/s
step 17635/19560 | loss 3.276973 (-0.37z)| norm 0.2643 (-0.52z)| lr 1.53e-05 | 322.85 ms | 52.3% bf16 MFU | 1624294 tok/s
step 17636/19560 | loss 3.257225 (-0.84z)| norm 0.2649 (-0.50z)| lr 1.53e-05 | 322.49 ms | 52.3% bf16 MFU | 1624367 tok/s
step 17637/19560 | loss 3.245711 (-1.11z)| norm 0.2446 (-1.08z)| lr 1.53e-05 | 322.50 ms | 52.3% bf16 MFU | 1624434 tok/s
step 17638/19560 | loss 3.297360 (+0.17z)| norm 0.2705 (-0.31z)| lr 1.53e-05 | 322.63 ms | 52.3% bf16 MFU | 1624464 tok/s
step 17639/19560 | loss 3.336101 (+1.12z)| norm 0.2541 (-0.79z)| lr 1.52e-05 | 322.75 ms | 52.3% bf16 MFU | 1624463 tok/s
step 17640/19560 | loss 3.286094 (-0.11z)| norm 0.2548 (-0.77z)| lr 1.52e-05 | 322.60 ms | 52.3% bf16 MFU | 1624499 tok/s
step 17641/19560 | loss 3.236786 (-1.34z)| norm 0.2557 (-0.74z)| lr 1.52e-05 | 322.83 ms | 52.3% bf16 MFU | 1624476 tok/s
step 17642/19560 | loss 3.312523 (+0.55z)| norm 0.2422 (-1.15z)| lr 1.52e-05 | 322.63 ms | 52.3% bf16 MFU | 1624505 tok/s
step 17643/19560 | loss 3.291771 (+0.01z)| norm 0.2406 (-1.18z)| lr 1.52e-05 | 322.46 ms | 52.3% bf16 MFU | 1624575 tok/s
step 17644/19560 | loss 3.342881 (+1.29z)| norm 0.2462 (-1.01z)| lr 1.52e-05 | 322.56 ms | 52.3% bf16 MFU | 1624616 tok/s
step 17645/19560 | loss 3.307642 (+0.41z)| norm 0.2628 (-0.49z)| lr 1.51e-05 | 322.59 ms | 52.3% bf16 MFU | 1624647 tok/s
step 17646/19560 | loss 3.285892 (-0.14z)| norm 0.2854 (+0.25z)| lr 1.51e-05 | 322.51 ms | 52.3% bf16 MFU | 1624697 tok/s
step 17647/19560 | loss 3.286219 (-0.14z)| norm 0.2471 (-1.00z)| lr 1.51e-05 | 322.55 ms | 52.3% bf16 MFU | 1624734 tok/s
step 17648/19560 | loss 3.296699 (+0.13z)| norm 0.3315 (+1.80z)| lr 1.51e-05 | 322.81 ms | 52.3% bf16 MFU | 1624705 tok/s
step 17649/19560 | loss 3.277162 (-0.36z)| norm 0.2494 (-0.90z)| lr 1.51e-05 | 322.78 ms | 52.3% bf16 MFU | 1624685 tok/s
step 17650/19560 | loss 3.285707 (-0.13z)| norm 0.2341 (-1.39z)| lr 1.51e-05 | 322.59 ms | 52.3% bf16 MFU | 1624714 tok/s
step 17651/19560 | loss 3.268209 (-0.57z)| norm 0.2817 (+0.18z)| lr 1.51e-05 | 322.51 ms | 52.3% bf16 MFU | 1624760 tok/s
step 17652/19560 | loss 3.271281 (-0.48z)| norm 0.2471 (-0.95z)| lr 1.50e-05 | 322.57 ms | 52.3% bf16 MFU | 1624788 tok/s
step 17653/19560 | loss 3.263812 (-0.69z)| norm 0.2933 (+0.56z)| lr 1.50e-05 | 322.75 ms | 52.3% bf16 MFU | 1624771 tok/s
step 17654/19560 | loss 3.378882 (+2.24z)| norm 0.2996 (+0.76z)| lr 1.50e-05 | 322.66 ms | 52.3% bf16 MFU | 1624778 tok/s
step 17655/19560 | loss 3.257360 (-0.86z)| norm 0.2656 (-0.36z)| lr 1.50e-05 | 322.64 ms | 52.3% bf16 MFU | 1624788 tok/s
step 17656/19560 | loss 3.270919 (-0.51z)| norm 0.2571 (-0.64z)| lr 1.50e-05 | 322.73 ms | 52.3% bf16 MFU | 1624775 tok/s
step 17657/19560 | loss 3.292979 (+0.07z)| norm 0.2716 (-0.17z)| lr 1.50e-05 | 322.60 ms | 52.3% bf16 MFU | 1624796 tok/s
step 17658/19560 | loss 3.478061 (+4.41z)| norm 0.2730 (-0.12z)| lr 1.49e-05 | 322.74 ms | 52.3% bf16 MFU | 1624782 tok/s
step 17659/19560 | loss 3.256526 (-0.83z)| norm 0.2402 (-1.21z)| lr 1.49e-05 | 322.81 ms | 52.3% bf16 MFU | 1624750 tok/s
step 17660/19560 | loss 3.290643 (-0.03z)| norm 0.3515 (+2.40z)| lr 1.49e-05 | 322.18 ms | 52.4% bf16 MFU | 1624879 tok/s
step 17661/19560 | loss 3.339708 (+1.15z)| norm 0.2604 (-0.55z)| lr 1.49e-05 | 322.43 ms | 52.3% bf16 MFU | 1624937 tok/s
step 17662/19560 | loss 3.271096 (-0.48z)| norm 0.2551 (-0.73z)| lr 1.49e-05 | 322.50 ms | 52.3% bf16 MFU | 1624975 tok/s
step 17663/19560 | loss 3.265642 (-0.62z)| norm 0.2884 (+0.34z)| lr 1.49e-05 | 322.75 ms | 52.3% bf16 MFU | 1624948 tok/s
step 17664/19560 | loss 3.254461 (-0.87z)| norm 0.3163 (+1.23z)| lr 1.49e-05 | 322.78 ms | 52.3% bf16 MFU | 1624916 tok/s
step 17665/19560 | loss 3.245026 (-1.08z)| norm 0.2792 (+0.02z)| lr 1.48e-05 | 322.67 ms | 52.3% bf16 MFU | 1624912 tok/s
step 17666/19560 | loss 3.341995 (+1.23z)| norm 0.2389 (-1.28z)| lr 1.48e-05 | 322.92 ms | 52.3% bf16 MFU | 1624846 tok/s
step 17667/19560 | loss 3.282856 (-0.19z)| norm 0.3022 (+0.76z)| lr 1.48e-05 | 322.18 ms | 52.4% bf16 MFU | 1624970 tok/s
step 17668/19560 | loss 3.264770 (-0.61z)| norm 0.2300 (-1.55z)| lr 1.48e-05 | 322.90 ms | 52.3% bf16 MFU | 1624906 tok/s
step 17669/19560 | loss 3.247829 (-1.05z)| norm 0.2456 (-1.04z)| lr 1.48e-05 | 322.36 ms | 52.4% bf16 MFU | 1624981 tok/s
step 17670/19560 | loss 3.284600 (-0.15z)| norm 0.2499 (-0.90z)| lr 1.48e-05 | 322.52 ms | 52.3% bf16 MFU | 1625012 tok/s
step 17671/19560 | loss 3.240991 (-1.20z)| norm 0.2637 (-0.46z)| lr 1.47e-05 | 322.13 ms | 52.4% bf16 MFU | 1625138 tok/s
step 17672/19560 | loss 3.342175 (+1.24z)| norm 0.2560 (-0.70z)| lr 1.47e-05 | 322.63 ms | 52.3% bf16 MFU | 1625133 tok/s
step 17673/19560 | loss 3.249489 (-0.98z)| norm 0.2884 (+0.33z)| lr 1.47e-05 | 322.84 ms | 52.3% bf16 MFU | 1625075 tok/s
step 17674/19560 | loss 3.329531 (+0.93z)| norm 0.2608 (-0.55z)| lr 1.47e-05 | 322.75 ms | 52.3% bf16 MFU | 1625043 tok/s
step 17675/19560 | loss 3.250445 (-0.96z)| norm 0.2594 (-0.60z)| lr 1.47e-05 | 322.56 ms | 52.3% bf16 MFU | 1625060 tok/s
step 17676/19560 | loss 3.378227 (+2.14z)| norm 0.2883 (+0.34z)| lr 1.47e-05 | 322.48 ms | 52.3% bf16 MFU | 1625098 tok/s
step 17677/19560 | loss 3.382422 (+2.18z)| norm 0.2544 (-0.74z)| lr 1.47e-05 | 322.40 ms | 52.3% bf16 MFU | 1625152 tok/s
step 17678/19560 | loss 3.270936 (-0.48z)| norm 0.2450 (-1.04z)| lr 1.46e-05 | 322.48 ms | 52.3% bf16 MFU | 1625184 tok/s
step 17679/19560 | loss 3.337233 (+1.10z)| norm 0.2814 (+0.15z)| lr 1.46e-05 | 322.50 ms | 52.3% bf16 MFU | 1625211 tok/s
step 17680/19560 | loss 3.263586 (-0.67z)| norm 0.2731 (-0.13z)| lr 1.46e-05 | 323.28 ms | 52.2% bf16 MFU | 1625039 tok/s
step 17681/19560 | loss 3.337601 (+1.09z)| norm 0.2464 (-1.00z)| lr 1.46e-05 | 322.33 ms | 52.4% bf16 MFU | 1625114 tok/s
step 17682/19560 | loss 3.337651 (+1.08z)| norm 0.2498 (-0.87z)| lr 1.46e-05 | 322.77 ms | 52.3% bf16 MFU | 1625074 tok/s
step 17683/19560 | loss 3.276168 (-0.39z)| norm 0.2604 (-0.51z)| lr 1.46e-05 | 322.98 ms | 52.3% bf16 MFU | 1624983 tok/s
step 17684/19560 | loss 3.300870 (+0.20z)| norm 0.2549 (-0.69z)| lr 1.45e-05 | 323.44 ms | 52.2% bf16 MFU | 1624784 tok/s
step 17685/19560 | loss 3.271304 (-0.52z)| norm 0.3790 (+3.34z)| lr 1.45e-05 | 322.39 ms | 52.3% bf16 MFU | 1624856 tok/s
step 17686/19560 | loss 3.252033 (-0.96z)| norm 0.2535 (-0.71z)| lr 1.45e-05 | 322.51 ms | 52.3% bf16 MFU | 1624895 tok/s
step 17687/19560 | loss 3.327551 (+0.83z)| norm 0.2549 (-0.67z)| lr 1.45e-05 | 323.26 ms | 52.2% bf16 MFU | 1624743 tok/s
step 17688/19560 | loss 3.269554 (-0.55z)| norm 0.3072 (+1.03z)| lr 1.45e-05 | 322.68 ms | 52.3% bf16 MFU | 1624745 tok/s
step 17689/19560 | loss 3.236594 (-1.32z)| norm 0.3278 (+1.70z)| lr 1.45e-05 | 322.75 ms | 52.3% bf16 MFU | 1624731 tok/s
step 17690/19560 | loss 3.242898 (-1.15z)| norm 0.2813 (+0.17z)| lr 1.45e-05 | 322.91 ms | 52.3% bf16 MFU | 1624676 tok/s
step 17691/19560 | loss 3.323455 (+0.74z)| norm 0.2532 (-0.73z)| lr 1.44e-05 | 322.21 ms | 52.4% bf16 MFU | 1624800 tok/s
step 17692/19560 | loss 3.296385 (+0.10z)| norm 0.2473 (-0.91z)| lr 1.44e-05 | 322.98 ms | 52.3% bf16 MFU | 1624725 tok/s
step 17693/19560 | loss 3.276887 (-0.35z)| norm 0.2958 (+0.65z)| lr 1.44e-05 | 323.51 ms | 52.2% bf16 MFU | 1624518 tok/s
step 17694/19560 | loss 3.317758 (+0.61z)| norm 0.3087 (+1.06z)| lr 1.44e-05 | 322.51 ms | 52.3% bf16 MFU | 1624576 tok/s
step 17695/19560 | loss 3.253988 (-0.89z)| norm 0.2761 (-0.01z)| lr 1.44e-05 | 322.90 ms | 52.3% bf16 MFU | 1624530 tok/s
step 17696/19560 | loss 3.275993 (-0.36z)| norm 0.2634 (-0.41z)| lr 1.44e-05 | 322.58 ms | 52.3% bf16 MFU | 1624568 tok/s
step 17697/19560 | loss 3.308629 (+0.40z)| norm 0.3849 (+3.37z)| lr 1.43e-05 | 322.98 ms | 52.3% bf16 MFU | 1624505 tok/s
step 17698/19560 | loss 3.321463 (+0.72z)| norm 0.2451 (-0.97z)| lr 1.43e-05 | 322.68 ms | 52.3% bf16 MFU | 1624518 tok/s
step 17699/19560 | loss 3.282357 (-0.21z)| norm 0.2607 (-0.48z)| lr 1.43e-05 | 322.93 ms | 52.3% bf16 MFU | 1624470 tok/s
step 17700/19560 | loss 3.319180 (+0.65z)| norm 0.2523 (-0.75z)| lr 1.43e-05 | 322.73 ms | 52.3% bf16 MFU | 1624472 tok/s
step 17701/19560 | loss 3.293121 (+0.03z)| norm 0.2880 (+0.38z)| lr 1.43e-05 | 322.75 ms | 52.3% bf16 MFU | 1624470 tok/s
step 17702/19560 | loss 3.320235 (+0.67z)| norm 0.2567 (-0.59z)| lr 1.43e-05 | 322.69 ms | 52.3% bf16 MFU | 1624484 tok/s
step 17703/19560 | loss 3.300719 (+0.20z)| norm 0.2294 (-1.44z)| lr 1.43e-05 | 322.30 ms | 52.4% bf16 MFU | 1624595 tok/s
step 17704/19560 | loss 3.332115 (+0.94z)| norm 0.2943 (+0.61z)| lr 1.42e-05 | 322.91 ms | 52.3% bf16 MFU | 1624547 tok/s
step 17705/19560 | loss 3.266531 (-0.62z)| norm 0.2530 (-0.69z)| lr 1.42e-05 | 323.05 ms | 52.2% bf16 MFU | 1624466 tok/s
step 17706/19560 | loss 3.301092 (+0.21z)| norm 0.2586 (-0.51z)| lr 1.42e-05 | 322.94 ms | 52.3% bf16 MFU | 1624416 tok/s
step 17707/19560 | loss 3.265207 (-0.68z)| norm 0.3049 (+0.95z)| lr 1.42e-05 | 323.10 ms | 52.2% bf16 MFU | 1624329 tok/s
step 17708/19560 | loss 3.219154 (-1.79z)| norm 0.2578 (-0.54z)| lr 1.42e-05 | 322.10 ms | 52.4% bf16 MFU | 1624497 tok/s
step 17709/19560 | loss 3.255950 (-0.89z)| norm 0.2494 (-0.81z)| lr 1.42e-05 | 323.31 ms | 52.2% bf16 MFU | 1624353 tok/s
step 17710/19560 | loss 3.231789 (-1.45z)| norm 0.2938 (+0.60z)| lr 1.41e-05 | 322.77 ms | 52.3% bf16 MFU | 1624353 tok/s
step 17711/19560 | loss 3.237514 (-1.30z)| norm 0.2613 (-0.43z)| lr 1.41e-05 | 322.85 ms | 52.3% bf16 MFU | 1624333 tok/s
step 17712/19560 | loss 3.265784 (-0.61z)| norm 0.2451 (-0.94z)| lr 1.41e-05 | 322.49 ms | 52.3% bf16 MFU | 1624402 tok/s
step 17713/19560 | loss 3.171853 (-2.76z)| norm 0.2407 (-1.06z)| lr 1.41e-05 | 322.78 ms | 52.3% bf16 MFU | 1624398 tok/s
step 17714/19560 | loss 3.326299 (+0.86z)| norm 0.2697 (-0.15z)| lr 1.41e-05 | 323.27 ms | 52.2% bf16 MFU | 1624268 tok/s
step 17715/19560 | loss 3.295072 (+0.13z)| norm 0.2657 (-0.27z)| lr 1.41e-05 | 323.09 ms | 52.2% bf16 MFU | 1624192 tok/s
step 17716/19560 | loss 3.250805 (-0.91z)| norm 0.2491 (-0.79z)| lr 1.41e-05 | 322.65 ms | 52.3% bf16 MFU | 1624230 tok/s
step 17717/19560 | loss 3.207064 (-1.91z)| norm 0.2451 (-0.90z)| lr 1.40e-05 | 322.41 ms | 52.3% bf16 MFU | 1624327 tok/s
step 17718/19560 | loss 3.283294 (-0.13z)| norm 0.2422 (-0.98z)| lr 1.40e-05 | 322.51 ms | 52.3% bf16 MFU | 1624392 tok/s
step 17719/19560 | loss 3.246929 (-0.99z)| norm 0.2424 (-0.96z)| lr 1.40e-05 | 322.66 ms | 52.3% bf16 MFU | 1624417 tok/s
step 17720/19560 | loss 3.449125 (+3.55z)| norm 0.2595 (-0.42z)| lr 1.40e-05 | 322.56 ms | 52.3% bf16 MFU | 1624466 tok/s
step 17721/19560 | loss 3.270810 (-0.43z)| norm 0.2769 (+0.13z)| lr 1.40e-05 | 322.36 ms | 52.4% bf16 MFU | 1624563 tok/s
step 17722/19560 | loss 3.284606 (-0.13z)| norm 0.2929 (+0.68z)| lr 1.40e-05 | 322.51 ms | 52.3% bf16 MFU | 1624617 tok/s
step 17723/19560 | loss 3.256677 (-0.75z)| norm 0.2556 (-0.53z)| lr 1.40e-05 | 322.64 ms | 52.3% bf16 MFU | 1624637 tok/s
step 17724/19560 | loss 3.245801 (-0.99z)| norm 0.2718 (-0.01z)| lr 1.39e-05 | 322.84 ms | 52.3% bf16 MFU | 1624605 tok/s
step 17725/19560 | loss 3.284531 (-0.12z)| norm 0.2792 (+0.24z)| lr 1.39e-05 | 322.94 ms | 52.3% bf16 MFU | 1624548 tok/s
step 17726/19560 | loss 3.249904 (-0.89z)| norm 0.2562 (-0.52z)| lr 1.39e-05 | 322.30 ms | 52.4% bf16 MFU | 1624656 tok/s
step 17727/19560 | loss 3.310869 (+0.48z)| norm 0.2808 (+0.29z)| lr 1.39e-05 | 323.26 ms | 52.2% bf16 MFU | 1624518 tok/s
step 17728/19560 | loss 3.257968 (-0.70z)| norm 0.2922 (+0.65z)| lr 1.39e-05 | 322.44 ms | 52.3% bf16 MFU | 1624592 tok/s
step 17729/19560 | loss 3.294981 (+0.13z)| norm 0.2820 (+0.32z)| lr 1.39e-05 | 322.25 ms | 52.4% bf16 MFU | 1624710 tok/s
step 17730/19560 | loss 3.278792 (-0.23z)| norm 0.2771 (+0.16z)| lr 1.38e-05 | 322.69 ms | 52.3% bf16 MFU | 1624712 tok/s
step 17731/19560 | loss 3.201198 (-1.93z)| norm 0.2498 (-0.74z)| lr 1.38e-05 | 322.49 ms | 52.3% bf16 MFU | 1624764 tok/s
step 17732/19560 | loss 3.266198 (-0.48z)| norm 0.2478 (-0.80z)| lr 1.38e-05 | 322.83 ms | 52.3% bf16 MFU | 1624728 tok/s
step 17733/19560 | loss 3.278836 (-0.20z)| norm 0.3155 (+1.43z)| lr 1.38e-05 | 322.73 ms | 52.3% bf16 MFU | 1624718 tok/s
step 17734/19560 | loss 3.270021 (-0.40z)| norm 0.2846 (+0.43z)| lr 1.38e-05 | 322.51 ms | 52.3% bf16 MFU | 1624765 tok/s
step 17735/19560 | loss 3.451030 (+3.43z)| norm 0.4371 (+4.98z)| lr 1.38e-05 | 322.69 ms | 52.3% bf16 MFU | 1624763 tok/s
step 17736/19560 | loss 3.217350 (-1.49z)| norm 0.2449 (-0.83z)| lr 1.38e-05 | 322.36 ms | 52.4% bf16 MFU | 1624846 tok/s
step 17737/19560 | loss 3.307559 (+0.41z)| norm 0.2911 (+0.57z)| lr 1.37e-05 | 322.40 ms | 52.3% bf16 MFU | 1624913 tok/s
step 17738/19560 | loss 3.325979 (+0.78z)| norm 0.3001 (+0.85z)| lr 1.37e-05 | 322.86 ms | 52.3% bf16 MFU | 1624862 tok/s
step 17739/19560 | loss 3.329241 (+0.85z)| norm 0.2600 (-0.35z)| lr 1.37e-05 | 322.53 ms | 52.3% bf16 MFU | 1624896 tok/s
step 17740/19560 | loss 3.300809 (+0.25z)| norm 0.2447 (-0.84z)| lr 1.37e-05 | 322.50 ms | 52.3% bf16 MFU | 1624935 tok/s
step 17741/19560 | loss 3.310499 (+0.45z)| norm 0.4655 (+5.33z)| lr 1.37e-05 | 322.87 ms | 52.3% bf16 MFU | 1624880 tok/s
step 17742/19560 | loss 3.288117 (-0.03z)| norm 0.3248 (+1.42z)| lr 1.37e-05 | 322.40 ms | 52.3% bf16 MFU | 1624946 tok/s
step 17743/19560 | loss 3.355586 (+1.37z)| norm 0.2932 (+0.55z)| lr 1.37e-05 | 322.94 ms | 52.3% bf16 MFU | 1624873 tok/s
step 17744/19560 | loss 3.278975 (-0.23z)| norm 0.3282 (+1.49z)| lr 1.36e-05 | 322.86 ms | 52.3% bf16 MFU | 1624822 tok/s
step 17745/19560 | loss 3.280557 (-0.21z)| norm 0.3196 (+1.23z)| lr 1.36e-05 | 322.66 ms | 52.3% bf16 MFU | 1624827 tok/s
step 17746/19560 | loss 3.271191 (-0.41z)| norm 0.2651 (-0.24z)| lr 1.36e-05 | 322.67 ms | 52.3% bf16 MFU | 1624828 tok/s
step 17747/19560 | loss 3.292369 (+0.03z)| norm 0.2642 (-0.27z)| lr 1.36e-05 | 322.41 ms | 52.3% bf16 MFU | 1624895 tok/s
step 17748/19560 | loss 3.275579 (-0.33z)| norm 0.3446 (+1.90z)| lr 1.36e-05 | 323.04 ms | 52.2% bf16 MFU | 1624800 tok/s
step 17749/19560 | loss 3.239341 (-1.08z)| norm 0.2845 (+0.26z)| lr 1.36e-05 | 322.91 ms | 52.3% bf16 MFU | 1624741 tok/s
step 17750/19560 | loss 3.316895 (+0.56z)| norm 0.2708 (-0.11z)| lr 1.35e-05 | 322.99 ms | 52.3% bf16 MFU | 1624666 tok/s
val loss 3.278147
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3053/10042 = 0.304023
step 17751/19560 | loss 3.338180 (+1.00z)| norm 0.2670 (-0.21z)| lr 1.35e-05 | 321.99 ms | 52.4% bf16 MFU | 1624848 tok/s
step 17752/19560 | loss 3.301796 (+0.24z)| norm 0.2914 (+0.45z)| lr 1.35e-05 | 323.22 ms | 52.2% bf16 MFU | 1624709 tok/s
step 17753/19560 | loss 3.319060 (+0.60z)| norm 0.3699 (+2.50z)| lr 1.35e-05 | 322.41 ms | 52.3% bf16 MFU | 1624781 tok/s
step 17754/19560 | loss 3.273736 (-0.35z)| norm 0.3229 (+1.25z)| lr 1.35e-05 | 322.44 ms | 52.3% bf16 MFU | 1624842 tok/s
step 17755/19560 | loss 3.278165 (-0.24z)| norm 0.2673 (-0.23z)| lr 1.35e-05 | 322.66 ms | 52.3% bf16 MFU | 1624846 tok/s
step 17756/19560 | loss 3.331428 (+0.90z)| norm 0.2636 (-0.32z)| lr 1.35e-05 | 322.46 ms | 52.3% bf16 MFU | 1624899 tok/s
step 17757/19560 | loss 3.260735 (-0.62z)| norm 0.2393 (-0.96z)| lr 1.34e-05 | 323.32 ms | 52.2% bf16 MFU | 1624734 tok/s
step 17758/19560 | loss 3.366277 (+1.61z)| norm 0.2709 (-0.13z)| lr 1.34e-05 | 323.21 ms | 52.2% bf16 MFU | 1624603 tok/s
step 17759/19560 | loss 3.248975 (-0.89z)| norm 0.2829 (+0.21z)| lr 1.34e-05 | 322.51 ms | 52.3% bf16 MFU | 1624655 tok/s
step 17760/19560 | loss 3.205254 (-1.81z)| norm 0.2615 (-0.37z)| lr 1.34e-05 | 322.81 ms | 52.3% bf16 MFU | 1624629 tok/s
step 17761/19560 | loss 3.292259 (+0.09z)| norm 0.2916 (+0.43z)| lr 1.34e-05 | 322.43 ms | 52.3% bf16 MFU | 1624699 tok/s
step 17762/19560 | loss 3.342036 (+1.15z)| norm 0.2733 (-0.06z)| lr 1.34e-05 | 322.67 ms | 52.3% bf16 MFU | 1624706 tok/s
step 17763/19560 | loss 3.313774 (+0.53z)| norm 0.2410 (-0.93z)| lr 1.34e-05 | 322.14 ms | 52.4% bf16 MFU | 1624845 tok/s
step 17764/19560 | loss 3.311429 (+0.47z)| norm 0.2582 (-0.46z)| lr 1.33e-05 | 322.82 ms | 52.3% bf16 MFU | 1624806 tok/s
step 17765/19560 | loss 3.232869 (-1.23z)| norm 0.2754 (-0.01z)| lr 1.33e-05 | 322.43 ms | 52.3% bf16 MFU | 1624868 tok/s
step 17766/19560 | loss 3.283849 (-0.12z)| norm 0.2591 (-0.44z)| lr 1.33e-05 | 322.80 ms | 52.3% bf16 MFU | 1624833 tok/s
step 17767/19560 | loss 3.358805 (+1.49z)| norm 0.2660 (-0.26z)| lr 1.33e-05 | 323.30 ms | 52.2% bf16 MFU | 1624675 tok/s
step 17768/19560 | loss 3.324058 (+0.73z)| norm 0.2675 (-0.22z)| lr 1.33e-05 | 322.72 ms | 52.3% bf16 MFU | 1624671 tok/s
step 17769/19560 | loss 3.262434 (-0.60z)| norm 0.2887 (+0.35z)| lr 1.33e-05 | 323.18 ms | 52.2% bf16 MFU | 1624551 tok/s
step 17770/19560 | loss 3.267251 (-0.49z)| norm 0.2415 (-0.93z)| lr 1.33e-05 | 322.64 ms | 52.3% bf16 MFU | 1624574 tok/s
step 17771/19560 | loss 3.269582 (-0.43z)| norm 0.2985 (+0.60z)| lr 1.32e-05 | 322.43 ms | 52.3% bf16 MFU | 1624647 tok/s
step 17772/19560 | loss 3.263398 (-0.56z)| norm 0.2507 (-0.70z)| lr 1.32e-05 | 323.04 ms | 52.2% bf16 MFU | 1624563 tok/s
step 17773/19560 | loss 3.193478 (-2.02z)| norm 0.2487 (-0.75z)| lr 1.32e-05 | 322.48 ms | 52.3% bf16 MFU | 1624626 tok/s
step 17774/19560 | loss 3.322344 (+0.72z)| norm 0.2853 (+0.24z)| lr 1.32e-05 | 322.54 ms | 52.3% bf16 MFU | 1624669 tok/s
step 17775/19560 | loss 3.295298 (+0.14z)| norm 0.2508 (-0.69z)| lr 1.32e-05 | 323.24 ms | 52.2% bf16 MFU | 1624534 tok/s
step 17776/19560 | loss 3.313914 (+0.54z)| norm 0.3017 (+0.70z)| lr 1.32e-05 | 322.86 ms | 52.3% bf16 MFU | 1624500 tok/s
step 17777/19560 | loss 3.319911 (+0.66z)| norm 0.3115 (+0.95z)| lr 1.31e-05 | 323.43 ms | 52.2% bf16 MFU | 1624326 tok/s
step 17778/19560 | loss 3.270663 (-0.39z)| norm 0.2653 (-0.32z)| lr 1.31e-05 | 322.69 ms | 52.3% bf16 MFU | 1624348 tok/s
step 17779/19560 | loss 3.316943 (+0.59z)| norm 0.3692 (+2.45z)| lr 1.31e-05 | 322.63 ms | 52.3% bf16 MFU | 1624384 tok/s
step 17780/19560 | loss 3.288775 (-0.01z)| norm 0.2956 (+0.47z)| lr 1.31e-05 | 322.88 ms | 52.3% bf16 MFU | 1624354 tok/s
step 17781/19560 | loss 3.251430 (-0.80z)| norm 0.3140 (+0.96z)| lr 1.31e-05 | 322.73 ms | 52.3% bf16 MFU | 1624365 tok/s
step 17782/19560 | loss 3.290065 (+0.03z)| norm 0.2822 (+0.11z)| lr 1.31e-05 | 323.31 ms | 52.2% bf16 MFU | 1624228 tok/s
step 17783/19560 | loss 3.387444 (+2.07z)| norm 0.3212 (+1.14z)| lr 1.31e-05 | 322.52 ms | 52.3% bf16 MFU | 1624297 tok/s
step 17784/19560 | loss 3.235301 (-1.14z)| norm 0.2481 (-0.80z)| lr 1.30e-05 | 322.63 ms | 52.3% bf16 MFU | 1624335 tok/s
step 17785/19560 | loss 3.312220 (+0.48z)| norm 0.2672 (-0.30z)| lr 1.30e-05 | 322.32 ms | 52.4% bf16 MFU | 1624448 tok/s
step 17786/19560 | loss 3.318604 (+0.68z)| norm 0.2659 (-0.33z)| lr 1.30e-05 | 323.17 ms | 52.2% bf16 MFU | 1624341 tok/s
step 17787/19560 | loss 3.255044 (-0.75z)| norm 0.2514 (-0.72z)| lr 1.30e-05 | 322.91 ms | 52.3% bf16 MFU | 1624304 tok/s
step 17788/19560 | loss 3.294236 (+0.13z)| norm 0.2472 (-0.82z)| lr 1.30e-05 | 322.26 ms | 52.4% bf16 MFU | 1624435 tok/s
step 17789/19560 | loss 3.277074 (-0.24z)| norm 0.2733 (-0.12z)| lr 1.30e-05 | 323.31 ms | 52.2% bf16 MFU | 1624294 tok/s
step 17790/19560 | loss 3.320145 (+0.72z)| norm 0.2654 (-0.33z)| lr 1.30e-05 | 322.47 ms | 52.3% bf16 MFU | 1624372 tok/s
step 17791/19560 | loss 3.241089 (-1.05z)| norm 0.2330 (-1.19z)| lr 1.29e-05 | 322.69 ms | 52.3% bf16 MFU | 1624390 tok/s
step 17792/19560 | loss 3.303762 (+0.35z)| norm 0.2446 (-0.86z)| lr 1.29e-05 | 322.06 ms | 52.4% bf16 MFU | 1624566 tok/s
step 17793/19560 | loss 3.320782 (+0.72z)| norm 0.2497 (-0.72z)| lr 1.29e-05 | 322.62 ms | 52.3% bf16 MFU | 1624592 tok/s
step 17794/19560 | loss 3.310741 (+0.50z)| norm 0.2881 (+0.30z)| lr 1.29e-05 | 322.12 ms | 52.4% bf16 MFU | 1624742 tok/s
step 17795/19560 | loss 3.322381 (+0.76z)| norm 0.3053 (+0.77z)| lr 1.29e-05 | 322.78 ms | 52.3% bf16 MFU | 1624720 tok/s
step 17796/19560 | loss 3.306170 (+0.38z)| norm 0.2646 (-0.34z)| lr 1.29e-05 | 322.68 ms | 52.3% bf16 MFU | 1624723 tok/s
step 17797/19560 | loss 3.260706 (-0.65z)| norm 0.2658 (-0.31z)| lr 1.29e-05 | 322.64 ms | 52.3% bf16 MFU | 1624738 tok/s
step 17798/19560 | loss 3.383846 (+2.09z)| norm 0.2488 (-0.77z)| lr 1.28e-05 | 322.84 ms | 52.3% bf16 MFU | 1624699 tok/s
step 17799/19560 | loss 3.284462 (-0.14z)| norm 0.2920 (+0.39z)| lr 1.28e-05 | 322.64 ms | 52.3% bf16 MFU | 1624714 tok/s
step 17800/19560 | loss 3.229266 (-1.35z)| norm 0.2411 (-0.98z)| lr 1.28e-05 | 322.74 ms | 52.3% bf16 MFU | 1624703 tok/s
step 17801/19560 | loss 3.307191 (+0.38z)| norm 0.3079 (+0.82z)| lr 1.28e-05 | 322.33 ms | 52.4% bf16 MFU | 1624796 tok/s
step 17802/19560 | loss 3.351818 (+1.37z)| norm 0.2694 (-0.22z)| lr 1.28e-05 | 322.44 ms | 52.3% bf16 MFU | 1624855 tok/s
step 17803/19560 | loss 3.338340 (+1.06z)| norm 0.2572 (-0.55z)| lr 1.28e-05 | 322.36 ms | 52.4% bf16 MFU | 1624933 tok/s
step 17804/19560 | loss 3.336545 (+1.03z)| norm 0.2450 (-0.87z)| lr 1.28e-05 | 322.67 ms | 52.3% bf16 MFU | 1624929 tok/s
step 17805/19560 | loss 3.297991 (+0.18z)| norm 0.2617 (-0.42z)| lr 1.27e-05 | 322.88 ms | 52.3% bf16 MFU | 1624872 tok/s
step 17806/19560 | loss 3.221184 (-1.55z)| norm 0.2701 (-0.20z)| lr 1.27e-05 | 322.33 ms | 52.4% bf16 MFU | 1624956 tok/s
step 17807/19560 | loss 3.340801 (+1.16z)| norm 0.2675 (-0.27z)| lr 1.27e-05 | 322.35 ms | 52.4% bf16 MFU | 1625031 tok/s
step 17808/19560 | loss 3.244060 (-1.03z)| norm 0.2764 (-0.03z)| lr 1.27e-05 | 323.15 ms | 52.2% bf16 MFU | 1624901 tok/s
step 17809/19560 | loss 3.298744 (+0.22z)| norm 0.2399 (-1.01z)| lr 1.27e-05 | 322.57 ms | 52.3% bf16 MFU | 1624923 tok/s
step 17810/19560 | loss 3.286095 (-0.06z)| norm 0.2302 (-1.27z)| lr 1.27e-05 | 322.64 ms | 52.3% bf16 MFU | 1624925 tok/s
step 17811/19560 | loss 3.262868 (-0.59z)| norm 0.2802 (+0.08z)| lr 1.27e-05 | 322.57 ms | 52.3% bf16 MFU | 1624946 tok/s
step 17812/19560 | loss 3.285469 (-0.07z)| norm 0.3443 (+1.76z)| lr 1.26e-05 | 322.66 ms | 52.3% bf16 MFU | 1624944 tok/s
step 17813/19560 | loss 3.322846 (+0.77z)| norm 0.3356 (+1.57z)| lr 1.26e-05 | 322.67 ms | 52.3% bf16 MFU | 1624938 tok/s
step 17814/19560 | loss 3.347728 (+1.32z)| norm 0.2416 (-0.98z)| lr 1.26e-05 | 322.53 ms | 52.3% bf16 MFU | 1624967 tok/s
step 17815/19560 | loss 3.287562 (-0.04z)| norm 0.2662 (-0.31z)| lr 1.26e-05 | 322.59 ms | 52.3% bf16 MFU | 1624981 tok/s
step 17816/19560 | loss 3.326620 (+0.84z)| norm 0.3361 (+1.57z)| lr 1.26e-05 | 322.74 ms | 52.3% bf16 MFU | 1624955 tok/s
step 17817/19560 | loss 3.246335 (-0.99z)| norm 0.3061 (+0.77z)| lr 1.26e-05 | 322.77 ms | 52.3% bf16 MFU | 1624925 tok/s
step 17818/19560 | loss 3.255397 (-0.79z)| norm 0.2777 (-0.00z)| lr 1.26e-05 | 322.55 ms | 52.3% bf16 MFU | 1624952 tok/s
step 17819/19560 | loss 3.261069 (-0.65z)| norm 0.2561 (-0.59z)| lr 1.25e-05 | 322.44 ms | 52.3% bf16 MFU | 1625005 tok/s
step 17820/19560 | loss 3.270202 (-0.44z)| norm 0.2411 (-0.99z)| lr 1.25e-05 | 322.70 ms | 52.3% bf16 MFU | 1624988 tok/s
step 17821/19560 | loss 3.333224 (+0.99z)| norm 0.2480 (-0.79z)| lr 1.25e-05 | 322.54 ms | 52.3% bf16 MFU | 1625013 tok/s
step 17822/19560 | loss 3.263144 (-0.59z)| norm 0.2789 (+0.05z)| lr 1.25e-05 | 322.86 ms | 52.3% bf16 MFU | 1624957 tok/s
step 17823/19560 | loss 3.297463 (+0.18z)| norm 0.2430 (-0.91z)| lr 1.25e-05 | 323.05 ms | 52.2% bf16 MFU | 1624856 tok/s
step 17824/19560 | loss 3.471181 (+3.86z)| norm 0.2546 (-0.60z)| lr 1.25e-05 | 322.88 ms | 52.3% bf16 MFU | 1624802 tok/s
step 17825/19560 | loss 3.263049 (-0.59z)| norm 0.2522 (-0.66z)| lr 1.25e-05 | 322.49 ms | 52.3% bf16 MFU | 1624850 tok/s
step 17826/19560 | loss 3.289001 (-0.03z)| norm 0.2579 (-0.50z)| lr 1.24e-05 | 322.76 ms | 52.3% bf16 MFU | 1624827 tok/s
step 17827/19560 | loss 3.259204 (-0.67z)| norm 0.2583 (-0.49z)| lr 1.24e-05 | 322.76 ms | 52.3% bf16 MFU | 1624804 tok/s
step 17828/19560 | loss 3.261366 (-0.61z)| norm 0.2896 (+0.38z)| lr 1.24e-05 | 322.50 ms | 52.3% bf16 MFU | 1624849 tok/s
step 17829/19560 | loss 3.330281 (+0.86z)| norm 0.2630 (-0.36z)| lr 1.24e-05 | 322.49 ms | 52.3% bf16 MFU | 1624894 tok/s
step 17830/19560 | loss 3.317010 (+0.57z)| norm 0.2467 (-0.81z)| lr 1.24e-05 | 322.43 ms | 52.3% bf16 MFU | 1624952 tok/s
step 17831/19560 | loss 3.356102 (+1.39z)| norm 0.2809 (+0.13z)| lr 1.24e-05 | 322.81 ms | 52.3% bf16 MFU | 1624910 tok/s
step 17832/19560 | loss 3.321460 (+0.66z)| norm 0.3194 (+1.20z)| lr 1.24e-05 | 322.70 ms | 52.3% bf16 MFU | 1624900 tok/s
step 17833/19560 | loss 3.328651 (+0.80z)| norm 0.2632 (-0.37z)| lr 1.23e-05 | 322.35 ms | 52.4% bf16 MFU | 1624976 tok/s
step 17834/19560 | loss 3.348988 (+1.21z)| norm 0.2697 (-0.20z)| lr 1.23e-05 | 322.36 ms | 52.4% bf16 MFU | 1625048 tok/s
step 17835/19560 | loss 3.301910 (+0.22z)| norm 0.2392 (-1.03z)| lr 1.23e-05 | 322.85 ms | 52.3% bf16 MFU | 1624992 tok/s
step 17836/19560 | loss 3.342710 (+1.06z)| norm 0.3394 (+1.73z)| lr 1.23e-05 | 322.21 ms | 52.4% bf16 MFU | 1625102 tok/s
step 17837/19560 | loss 3.263385 (-0.62z)| norm 0.3127 (+0.98z)| lr 1.23e-05 | 322.66 ms | 52.3% bf16 MFU | 1625090 tok/s
step 17838/19560 | loss 3.302092 (+0.19z)| norm 0.2973 (+0.55z)| lr 1.23e-05 | 322.62 ms | 52.3% bf16 MFU | 1625092 tok/s
step 17839/19560 | loss 3.324536 (+0.66z)| norm 0.2649 (-0.34z)| lr 1.23e-05 | 322.93 ms | 52.3% bf16 MFU | 1625014 tok/s
step 17840/19560 | loss 3.323946 (+0.63z)| norm 0.3478 (+1.90z)| lr 1.22e-05 | 322.50 ms | 52.3% bf16 MFU | 1625048 tok/s
step 17841/19560 | loss 3.285742 (-0.21z)| norm 0.3203 (+1.14z)| lr 1.22e-05 | 322.47 ms | 52.3% bf16 MFU | 1625089 tok/s
step 17842/19560 | loss 3.271380 (-0.52z)| norm 0.2380 (-1.10z)| lr 1.22e-05 | 322.61 ms | 52.3% bf16 MFU | 1625092 tok/s
step 17843/19560 | loss 3.254422 (-0.88z)| norm 0.2350 (-1.17z)| lr 1.22e-05 | 322.71 ms | 52.3% bf16 MFU | 1625070 tok/s
step 17844/19560 | loss 3.266479 (-0.62z)| norm 0.2650 (-0.36z)| lr 1.22e-05 | 322.34 ms | 52.4% bf16 MFU | 1625141 tok/s
step 17845/19560 | loss 3.297091 (+0.04z)| norm 0.3118 (+0.89z)| lr 1.22e-05 | 322.95 ms | 52.3% bf16 MFU | 1625055 tok/s
step 17846/19560 | loss 3.319302 (+0.53z)| norm 0.2675 (-0.31z)| lr 1.22e-05 | 322.81 ms | 52.3% bf16 MFU | 1625008 tok/s
step 17847/19560 | loss 3.347338 (+1.14z)| norm 0.2401 (-1.06z)| lr 1.21e-05 | 322.46 ms | 52.3% bf16 MFU | 1625054 tok/s
step 17848/19560 | loss 3.349762 (+1.26z)| norm 0.3053 (+0.70z)| lr 1.21e-05 | 322.61 ms | 52.3% bf16 MFU | 1625058 tok/s
step 17849/19560 | loss 3.313068 (+0.40z)| norm 0.2430 (-0.98z)| lr 1.21e-05 | 323.16 ms | 52.2% bf16 MFU | 1624923 tok/s
step 17850/19560 | loss 3.204719 (-2.08z)| norm 0.2435 (-0.95z)| lr 1.21e-05 | 322.91 ms | 52.3% bf16 MFU | 1624860 tok/s
step 17851/19560 | loss 3.276665 (-0.43z)| norm 0.2525 (-0.71z)| lr 1.21e-05 | 322.57 ms | 52.3% bf16 MFU | 1624883 tok/s
step 17852/19560 | loss 3.270739 (-0.58z)| norm 0.2440 (-0.93z)| lr 1.21e-05 | 322.73 ms | 52.3% bf16 MFU | 1624866 tok/s
step 17853/19560 | loss 3.311086 (+0.35z)| norm 0.2547 (-0.63z)| lr 1.21e-05 | 322.80 ms | 52.3% bf16 MFU | 1624832 tok/s
step 17854/19560 | loss 3.255621 (-0.93z)| norm 0.2523 (-0.70z)| lr 1.20e-05 | 323.17 ms | 52.2% bf16 MFU | 1624706 tok/s
step 17855/19560 | loss 3.344196 (+1.10z)| norm 0.2670 (-0.30z)| lr 1.20e-05 | 322.63 ms | 52.3% bf16 MFU | 1624724 tok/s
step 17856/19560 | loss 3.347691 (+1.16z)| norm 0.2708 (-0.19z)| lr 1.20e-05 | 322.50 ms | 52.3% bf16 MFU | 1624773 tok/s
step 17857/19560 | loss 3.294809 (-0.05z)| norm 0.2357 (-1.12z)| lr 1.20e-05 | 322.63 ms | 52.3% bf16 MFU | 1624787 tok/s
step 17858/19560 | loss 3.265705 (-0.71z)| norm 0.2697 (-0.21z)| lr 1.20e-05 | 323.36 ms | 52.2% bf16 MFU | 1624618 tok/s
step 17859/19560 | loss 3.281894 (-0.36z)| norm 0.2770 (-0.02z)| lr 1.20e-05 | 322.62 ms | 52.3% bf16 MFU | 1624643 tok/s
step 17860/19560 | loss 3.315075 (+0.40z)| norm 0.2648 (-0.35z)| lr 1.20e-05 | 323.00 ms | 52.3% bf16 MFU | 1624569 tok/s
step 17861/19560 | loss 3.274794 (-0.54z)| norm 0.2645 (-0.35z)| lr 1.19e-05 | 322.38 ms | 52.4% bf16 MFU | 1624655 tok/s
step 17862/19560 | loss 3.340943 (+0.99z)| norm 0.3064 (+0.77z)| lr 1.19e-05 | 322.66 ms | 52.3% bf16 MFU | 1624668 tok/s
step 17863/19560 | loss 3.419839 (+2.89z)| norm 0.2783 (+0.05z)| lr 1.19e-05 | 323.13 ms | 52.2% bf16 MFU | 1624560 tok/s
step 17864/19560 | loss 3.302798 (+0.10z)| norm 0.3008 (+0.69z)| lr 1.19e-05 | 322.09 ms | 52.4% bf16 MFU | 1624720 tok/s
step 17865/19560 | loss 3.278864 (-0.47z)| norm 0.2908 (+0.40z)| lr 1.19e-05 | 322.89 ms | 52.3% bf16 MFU | 1624672 tok/s
step 17866/19560 | loss 3.295128 (-0.08z)| norm 0.2775 (+0.02z)| lr 1.19e-05 | 322.52 ms | 52.3% bf16 MFU | 1624718 tok/s
step 17867/19560 | loss 3.379851 (+1.93z)| norm 0.2575 (-0.56z)| lr 1.19e-05 | 322.78 ms | 52.3% bf16 MFU | 1624698 tok/s
step 17868/19560 | loss 3.302192 (+0.08z)| norm 0.2991 (+0.64z)| lr 1.19e-05 | 322.40 ms | 52.3% bf16 MFU | 1624772 tok/s
step 17869/19560 | loss 3.248747 (-1.17z)| norm 0.2709 (-0.16z)| lr 1.18e-05 | 322.63 ms | 52.3% bf16 MFU | 1624785 tok/s
step 17870/19560 | loss 3.299838 (+0.04z)| norm 0.2489 (-0.88z)| lr 1.18e-05 | 322.77 ms | 52.3% bf16 MFU | 1624764 tok/s
step 17871/19560 | loss 3.264743 (-0.78z)| norm 0.2621 (-0.43z)| lr 1.18e-05 | 322.79 ms | 52.3% bf16 MFU | 1624737 tok/s
step 17872/19560 | loss 3.197434 (-2.32z)| norm 0.2651 (-0.31z)| lr 1.18e-05 | 322.69 ms | 52.3% bf16 MFU | 1624736 tok/s
step 17873/19560 | loss 3.323654 (+0.61z)| norm 0.3075 (+1.14z)| lr 1.18e-05 | 322.52 ms | 52.3% bf16 MFU | 1624780 tok/s
step 17874/19560 | loss 3.358548 (+1.40z)| norm 0.2794 (+0.17z)| lr 1.18e-05 | 322.62 ms | 52.3% bf16 MFU | 1624794 tok/s
step 17875/19560 | loss 3.268689 (-0.67z)| norm 0.2389 (-1.20z)| lr 1.18e-05 | 322.75 ms | 52.3% bf16 MFU | 1624778 tok/s
step 17876/19560 | loss 3.324906 (+0.62z)| norm 0.2392 (-1.18z)| lr 1.17e-05 | 322.96 ms | 52.3% bf16 MFU | 1624707 tok/s
step 17877/19560 | loss 3.330526 (+0.73z)| norm 0.2533 (-0.68z)| lr 1.17e-05 | 322.72 ms | 52.3% bf16 MFU | 1624701 tok/s
step 17878/19560 | loss 3.255379 (-1.00z)| norm 0.2649 (-0.28z)| lr 1.17e-05 | 322.60 ms | 52.3% bf16 MFU | 1624727 tok/s
step 17879/19560 | loss 3.320633 (+0.52z)| norm 0.2490 (-0.82z)| lr 1.17e-05 | 322.71 ms | 52.3% bf16 MFU | 1624722 tok/s
step 17880/19560 | loss 3.288848 (-0.22z)| norm 0.2504 (-0.76z)| lr 1.17e-05 | 322.70 ms | 52.3% bf16 MFU | 1624719 tok/s
step 17881/19560 | loss 3.293031 (-0.11z)| norm 0.2331 (-1.37z)| lr 1.17e-05 | 322.89 ms | 52.3% bf16 MFU | 1624669 tok/s
step 17882/19560 | loss 3.366637 (+1.56z)| norm 0.2536 (-0.63z)| lr 1.17e-05 | 322.70 ms | 52.3% bf16 MFU | 1624669 tok/s
step 17883/19560 | loss 3.354517 (+1.26z)| norm 0.2898 (+0.68z)| lr 1.16e-05 | 323.22 ms | 52.2% bf16 MFU | 1624540 tok/s
step 17884/19560 | loss 3.307335 (+0.19z)| norm 0.2435 (-0.99z)| lr 1.16e-05 | 322.67 ms | 52.3% bf16 MFU | 1624555 tok/s
step 17885/19560 | loss 3.235213 (-1.45z)| norm 0.2728 (+0.06z)| lr 1.16e-05 | 322.78 ms | 52.3% bf16 MFU | 1624540 tok/s
step 17886/19560 | loss 3.350224 (+1.18z)| norm 0.2827 (+0.41z)| lr 1.16e-05 | 323.23 ms | 52.2% bf16 MFU | 1624414 tok/s
step 17887/19560 | loss 3.308678 (+0.22z)| norm 0.2683 (-0.11z)| lr 1.16e-05 | 322.51 ms | 52.3% bf16 MFU | 1624477 tok/s
step 17888/19560 | loss 3.344572 (+1.03z)| norm 0.2489 (-0.80z)| lr 1.16e-05 | 322.94 ms | 52.3% bf16 MFU | 1624426 tok/s
step 17889/19560 | loss 3.316770 (+0.38z)| norm 0.3678 (+3.33z)| lr 1.16e-05 | 322.79 ms | 52.3% bf16 MFU | 1624417 tok/s
step 17890/19560 | loss 3.445953 (+3.24z)| norm 0.2521 (-0.67z)| lr 1.15e-05 | 322.74 ms | 52.3% bf16 MFU | 1624421 tok/s
step 17891/19560 | loss 3.292171 (-0.20z)| norm 0.2406 (-1.07z)| lr 1.15e-05 | 322.53 ms | 52.3% bf16 MFU | 1624477 tok/s
step 17892/19560 | loss 3.326891 (+0.57z)| norm 0.2526 (-0.65z)| lr 1.15e-05 | 323.06 ms | 52.2% bf16 MFU | 1624397 tok/s
step 17893/19560 | loss 3.273166 (-0.64z)| norm 0.2809 (+0.33z)| lr 1.15e-05 | 322.53 ms | 52.3% bf16 MFU | 1624453 tok/s
step 17894/19560 | loss 3.359850 (+1.29z)| norm 0.2405 (-1.06z)| lr 1.15e-05 | 322.49 ms | 52.3% bf16 MFU | 1624519 tok/s
step 17895/19560 | loss 3.272506 (-0.65z)| norm 0.2364 (-1.19z)| lr 1.15e-05 | 323.04 ms | 52.2% bf16 MFU | 1624441 tok/s
step 17896/19560 | loss 3.330136 (+0.64z)| norm 0.2529 (-0.62z)| lr 1.15e-05 | 322.44 ms | 52.3% bf16 MFU | 1624518 tok/s
step 17897/19560 | loss 3.335424 (+0.75z)| norm 0.2444 (-0.90z)| lr 1.15e-05 | 323.13 ms | 52.2% bf16 MFU | 1624419 tok/s
step 17898/19560 | loss 3.283427 (-0.43z)| norm 0.2546 (-0.55z)| lr 1.14e-05 | 322.97 ms | 52.3% bf16 MFU | 1624364 tok/s
step 17899/19560 | loss 3.242138 (-1.35z)| norm 0.2596 (-0.37z)| lr 1.14e-05 | 322.75 ms | 52.3% bf16 MFU | 1624368 tok/s
step 17900/19560 | loss 3.357640 (+1.23z)| norm 0.2407 (-1.02z)| lr 1.14e-05 | 322.86 ms | 52.3% bf16 MFU | 1624343 tok/s
step 17901/19560 | loss 3.269984 (-0.77z)| norm 0.2321 (-1.30z)| lr 1.14e-05 | 322.64 ms | 52.3% bf16 MFU | 1624375 tok/s
step 17902/19560 | loss 3.358168 (+1.24z)| norm 0.2906 (+0.69z)| lr 1.14e-05 | 322.97 ms | 52.3% bf16 MFU | 1624322 tok/s
step 17903/19560 | loss 3.369128 (+1.47z)| norm 0.3796 (+3.51z)| lr 1.14e-05 | 323.09 ms | 52.2% bf16 MFU | 1624244 tok/s
step 17904/19560 | loss 3.338104 (+0.76z)| norm 0.2544 (-0.54z)| lr 1.14e-05 | 322.48 ms | 52.3% bf16 MFU | 1624322 tok/s
step 17905/19560 | loss 3.254267 (-1.11z)| norm 0.2942 (+0.76z)| lr 1.13e-05 | 323.13 ms | 52.2% bf16 MFU | 1624232 tok/s
step 17906/19560 | loss 3.285481 (-0.42z)| norm 0.2674 (-0.11z)| lr 1.13e-05 | 322.78 ms | 52.3% bf16 MFU | 1624236 tok/s
step 17907/19560 | loss 3.257020 (-1.04z)| norm 0.2573 (-0.43z)| lr 1.13e-05 | 323.16 ms | 52.2% bf16 MFU | 1624143 tok/s
step 17908/19560 | loss 3.220546 (-1.82z)| norm 0.2509 (-0.64z)| lr 1.13e-05 | 322.76 ms | 52.3% bf16 MFU | 1624156 tok/s
step 17909/19560 | loss 3.393065 (+1.95z)| norm 0.3043 (+1.19z)| lr 1.13e-05 | 322.41 ms | 52.3% bf16 MFU | 1624256 tok/s
step 17910/19560 | loss 3.336737 (+0.70z)| norm 0.2493 (-0.68z)| lr 1.13e-05 | 322.28 ms | 52.4% bf16 MFU | 1624383 tok/s
step 17911/19560 | loss 3.335347 (+0.69z)| norm 0.2368 (-1.10z)| lr 1.13e-05 | 322.34 ms | 52.4% bf16 MFU | 1624490 tok/s
step 17912/19560 | loss 3.282847 (-0.48z)| norm 0.2576 (-0.38z)| lr 1.12e-05 | 322.68 ms | 52.3% bf16 MFU | 1624504 tok/s
step 17913/19560 | loss 3.316552 (+0.27z)| norm 0.3068 (+1.29z)| lr 1.12e-05 | 322.28 ms | 52.4% bf16 MFU | 1624618 tok/s
step 17914/19560 | loss 3.245847 (-1.29z)| norm 0.2620 (-0.24z)| lr 1.12e-05 | 323.00 ms | 52.3% bf16 MFU | 1624546 tok/s
step 17915/19560 | loss 3.286084 (-0.40z)| norm 0.3002 (+1.05z)| lr 1.12e-05 | 323.04 ms | 52.2% bf16 MFU | 1624468 tok/s
step 17916/19560 | loss 3.372649 (+1.50z)| norm 0.3413 (+2.38z)| lr 1.12e-05 | 322.68 ms | 52.3% bf16 MFU | 1624483 tok/s
step 17917/19560 | loss 3.251466 (-1.17z)| norm 0.2440 (-0.86z)| lr 1.12e-05 | 322.23 ms | 52.4% bf16 MFU | 1624613 tok/s
step 17918/19560 | loss 3.276093 (-0.62z)| norm 0.2441 (-0.85z)| lr 1.12e-05 | 323.46 ms | 52.2% bf16 MFU | 1624427 tok/s
step 17919/19560 | loss 3.265024 (-0.87z)| norm 0.2453 (-0.82z)| lr 1.12e-05 | 322.20 ms | 52.4% bf16 MFU | 1624565 tok/s
step 17920/19560 | loss 3.304930 (+0.01z)| norm 0.2545 (-0.51z)| lr 1.11e-05 | 323.28 ms | 52.2% bf16 MFU | 1624425 tok/s
step 17921/19560 | loss 3.230608 (-1.60z)| norm 0.2455 (-0.81z)| lr 1.11e-05 | 322.18 ms | 52.4% bf16 MFU | 1624569 tok/s
step 17922/19560 | loss 3.449048 (+3.03z)| norm 0.2996 (+0.99z)| lr 1.11e-05 | 323.49 ms | 52.2% bf16 MFU | 1624378 tok/s
step 17923/19560 | loss 3.326334 (+0.45z)| norm 0.2458 (-0.79z)| lr 1.11e-05 | 323.01 ms | 52.2% bf16 MFU | 1624314 tok/s
step 17924/19560 | loss 3.230196 (-1.54z)| norm 0.2472 (-0.74z)| lr 1.11e-05 | 322.82 ms | 52.3% bf16 MFU | 1624302 tok/s
step 17925/19560 | loss 3.325364 (+0.43z)| norm 0.2603 (-0.30z)| lr 1.11e-05 | 322.99 ms | 52.3% bf16 MFU | 1624248 tok/s
step 17926/19560 | loss 3.281018 (-0.48z)| norm 0.2617 (-0.26z)| lr 1.11e-05 | 322.83 ms | 52.3% bf16 MFU | 1624238 tok/s
step 17927/19560 | loss 3.358731 (+1.14z)| norm 0.2421 (-0.90z)| lr 1.10e-05 | 322.81 ms | 52.3% bf16 MFU | 1624234 tok/s
step 17928/19560 | loss 3.325657 (+0.43z)| norm 0.2422 (-0.90z)| lr 1.10e-05 | 323.00 ms | 52.3% bf16 MFU | 1624181 tok/s
step 17929/19560 | loss 3.277217 (-0.59z)| norm 0.3385 (+2.28z)| lr 1.10e-05 | 322.78 ms | 52.3% bf16 MFU | 1624186 tok/s
step 17930/19560 | loss 3.394292 (+1.86z)| norm 0.2807 (+0.38z)| lr 1.10e-05 | 322.75 ms | 52.3% bf16 MFU | 1624198 tok/s
step 17931/19560 | loss 3.322435 (+0.36z)| norm 0.2470 (-0.73z)| lr 1.10e-05 | 322.59 ms | 52.3% bf16 MFU | 1624252 tok/s
step 17932/19560 | loss 3.310272 (+0.11z)| norm 0.2580 (-0.37z)| lr 1.10e-05 | 323.30 ms | 52.2% bf16 MFU | 1624122 tok/s
step 17933/19560 | loss 3.341747 (+0.76z)| norm 0.2478 (-0.70z)| lr 1.10e-05 | 322.89 ms | 52.3% bf16 MFU | 1624103 tok/s
step 17934/19560 | loss 3.304604 (-0.03z)| norm 0.2474 (-0.71z)| lr 1.10e-05 | 323.26 ms | 52.2% bf16 MFU | 1623991 tok/s
step 17935/19560 | loss 3.285777 (-0.42z)| norm 0.2840 (+0.49z)| lr 1.09e-05 | 323.01 ms | 52.2% bf16 MFU | 1623947 tok/s
step 17936/19560 | loss 3.350989 (+0.95z)| norm 0.2709 (+0.06z)| lr 1.09e-05 | 323.29 ms | 52.2% bf16 MFU | 1623835 tok/s
step 17937/19560 | loss 3.312602 (+0.13z)| norm 0.2629 (-0.21z)| lr 1.09e-05 | 322.90 ms | 52.3% bf16 MFU | 1623827 tok/s
step 17938/19560 | loss 3.293305 (-0.28z)| norm 0.3123 (+1.39z)| lr 1.09e-05 | 322.79 ms | 52.3% bf16 MFU | 1623846 tok/s
step 17939/19560 | loss 3.324832 (+0.38z)| norm 0.2828 (+0.42z)| lr 1.09e-05 | 322.75 ms | 52.3% bf16 MFU | 1623876 tok/s
step 17940/19560 | loss 3.298346 (-0.19z)| norm 0.2373 (-1.07z)| lr 1.09e-05 | 322.72 ms | 52.3% bf16 MFU | 1623911 tok/s
step 17941/19560 | loss 3.331439 (+0.52z)| norm 0.3429 (+2.45z)| lr 1.09e-05 | 323.34 ms | 52.2% bf16 MFU | 1623790 tok/s
step 17942/19560 | loss 3.327847 (+0.44z)| norm 0.3352 (+2.14z)| lr 1.08e-05 | 322.78 ms | 52.3% bf16 MFU | 1623815 tok/s
step 17943/19560 | loss 3.253392 (-1.14z)| norm 0.2513 (-0.61z)| lr 1.08e-05 | 322.76 ms | 52.3% bf16 MFU | 1623844 tok/s
step 17944/19560 | loss 3.302165 (-0.10z)| norm 0.2418 (-0.91z)| lr 1.08e-05 | 322.53 ms | 52.3% bf16 MFU | 1623929 tok/s
step 17945/19560 | loss 3.249317 (-1.23z)| norm 0.2384 (-1.01z)| lr 1.08e-05 | 323.13 ms | 52.2% bf16 MFU | 1623859 tok/s
step 17946/19560 | loss 3.286573 (-0.44z)| norm 0.3596 (+2.92z)| lr 1.08e-05 | 322.75 ms | 52.3% bf16 MFU | 1623887 tok/s
step 17947/19560 | loss 3.305269 (-0.04z)| norm 0.2726 (+0.11z)| lr 1.08e-05 | 323.25 ms | 52.2% bf16 MFU | 1623790 tok/s
step 17948/19560 | loss 3.267387 (-0.86z)| norm 0.2339 (-1.14z)| lr 1.08e-05 | 323.62 ms | 52.2% bf16 MFU | 1623603 tok/s
step 17949/19560 | loss 3.296976 (-0.22z)| norm 0.2463 (-0.74z)| lr 1.08e-05 | 322.36 ms | 52.4% bf16 MFU | 1623743 tok/s
step 17950/19560 | loss 3.317791 (+0.22z)| norm 0.2500 (-0.61z)| lr 1.07e-05 | 322.87 ms | 52.3% bf16 MFU | 1623748 tok/s
step 17951/19560 | loss 3.464935 (+3.24z)| norm 0.2636 (-0.18z)| lr 1.07e-05 | 323.26 ms | 52.2% bf16 MFU | 1623654 tok/s
step 17952/19560 | loss 3.282392 (-0.54z)| norm 0.2644 (-0.16z)| lr 1.07e-05 | 322.80 ms | 52.3% bf16 MFU | 1623681 tok/s
step 17953/19560 | loss 3.227130 (-1.72z)| norm 0.2485 (-0.67z)| lr 1.07e-05 | 323.00 ms | 52.3% bf16 MFU | 1623656 tok/s
step 17954/19560 | loss 3.333916 (+0.57z)| norm 0.2858 (+0.53z)| lr 1.07e-05 | 322.56 ms | 52.3% bf16 MFU | 1623743 tok/s
step 17955/19560 | loss 3.323606 (+0.34z)| norm 0.2478 (-0.69z)| lr 1.07e-05 | 323.30 ms | 52.2% bf16 MFU | 1623640 tok/s
step 17956/19560 | loss 3.286036 (-0.48z)| norm 0.2353 (-1.08z)| lr 1.07e-05 | 322.75 ms | 52.3% bf16 MFU | 1623680 tok/s
step 17957/19560 | loss 3.270811 (-0.79z)| norm 0.4177 (+4.37z)| lr 1.06e-05 | 322.78 ms | 52.3% bf16 MFU | 1623710 tok/s
step 17958/19560 | loss 3.247013 (-1.29z)| norm 0.3072 (+1.08z)| lr 1.06e-05 | 323.09 ms | 52.2% bf16 MFU | 1623662 tok/s
step 17959/19560 | loss 3.293774 (-0.27z)| norm 0.2847 (+0.41z)| lr 1.06e-05 | 322.78 ms | 52.3% bf16 MFU | 1623692 tok/s
step 17960/19560 | loss 3.250335 (-1.19z)| norm 0.2759 (+0.17z)| lr 1.06e-05 | 322.68 ms | 52.3% bf16 MFU | 1623748 tok/s
step 17961/19560 | loss 3.285566 (-0.43z)| norm 0.3187 (+1.41z)| lr 1.06e-05 | 322.88 ms | 52.3% bf16 MFU | 1623751 tok/s
step 17962/19560 | loss 3.214221 (-1.92z)| norm 0.3263 (+1.61z)| lr 1.06e-05 | 323.25 ms | 52.2% bf16 MFU | 1623658 tok/s
step 17963/19560 | loss 3.309553 (+0.10z)| norm 0.2681 (-0.10z)| lr 1.06e-05 | 322.46 ms | 52.3% bf16 MFU | 1623769 tok/s
step 17964/19560 | loss 3.237374 (-1.40z)| norm 0.2661 (-0.14z)| lr 1.06e-05 | 322.73 ms | 52.3% bf16 MFU | 1623808 tok/s
step 17965/19560 | loss 3.295509 (-0.18z)| norm 0.4014 (+3.68z)| lr 1.05e-05 | 323.26 ms | 52.2% bf16 MFU | 1623713 tok/s
step 17966/19560 | loss 3.298268 (-0.12z)| norm 0.2589 (-0.35z)| lr 1.05e-05 | 323.01 ms | 52.2% bf16 MFU | 1623684 tok/s
step 17967/19560 | loss 3.253103 (-1.06z)| norm 0.2550 (-0.46z)| lr 1.05e-05 | 322.90 ms | 52.3% bf16 MFU | 1623683 tok/s
step 17968/19560 | loss 3.260074 (-0.90z)| norm 0.2758 (+0.15z)| lr 1.05e-05 | 322.67 ms | 52.3% bf16 MFU | 1623741 tok/s
step 17969/19560 | loss 3.263219 (-0.83z)| norm 0.2743 (+0.12z)| lr 1.05e-05 | 323.34 ms | 52.2% bf16 MFU | 1623628 tok/s
step 17970/19560 | loss 3.259051 (-0.91z)| norm 0.2797 (+0.27z)| lr 1.05e-05 | 322.08 ms | 52.4% bf16 MFU | 1623837 tok/s
step 17971/19560 | loss 3.318632 (+0.32z)| norm 0.2364 (-1.00z)| lr 1.05e-05 | 323.12 ms | 52.2% bf16 MFU | 1623774 tok/s
step 17972/19560 | loss 3.275654 (-0.58z)| norm 0.2467 (-0.69z)| lr 1.04e-05 | 323.10 ms | 52.2% bf16 MFU | 1623719 tok/s
step 17973/19560 | loss 3.365904 (+1.30z)| norm 0.2579 (-0.36z)| lr 1.04e-05 | 322.94 ms | 52.3% bf16 MFU | 1623708 tok/s
step 17974/19560 | loss 3.276387 (-0.57z)| norm 0.2959 (+0.75z)| lr 1.04e-05 | 323.24 ms | 52.2% bf16 MFU | 1623621 tok/s
step 17975/19560 | loss 3.323956 (+0.43z)| norm 0.2469 (-0.68z)| lr 1.04e-05 | 322.92 ms | 52.3% bf16 MFU | 1623618 tok/s
step 17976/19560 | loss 3.274805 (-0.59z)| norm 0.2666 (-0.10z)| lr 1.04e-05 | 322.40 ms | 52.3% bf16 MFU | 1623747 tok/s
step 17977/19560 | loss 3.238397 (-1.33z)| norm 0.2563 (-0.41z)| lr 1.04e-05 | 323.52 ms | 52.2% bf16 MFU | 1623589 tok/s
step 17978/19560 | loss 3.301306 (-0.03z)| norm 0.2351 (-1.03z)| lr 1.04e-05 | 323.41 ms | 52.2% bf16 MFU | 1623465 tok/s
step 17979/19560 | loss 3.312302 (+0.19z)| norm 0.2423 (-0.81z)| lr 1.04e-05 | 322.96 ms | 52.3% bf16 MFU | 1623462 tok/s
step 17980/19560 | loss 3.303219 (-0.00z)| norm 0.2789 (+0.26z)| lr 1.03e-05 | 323.18 ms | 52.2% bf16 MFU | 1623403 tok/s
step 17981/19560 | loss 3.235128 (-1.43z)| norm 0.2381 (-0.94z)| lr 1.03e-05 | 322.65 ms | 52.3% bf16 MFU | 1623479 tok/s
step 17982/19560 | loss 3.276394 (-0.56z)| norm 0.2659 (-0.13z)| lr 1.03e-05 | 323.69 ms | 52.1% bf16 MFU | 1623290 tok/s
step 17983/19560 | loss 3.304863 (+0.05z)| norm 0.2956 (+0.74z)| lr 1.03e-05 | 323.19 ms | 52.2% bf16 MFU | 1623237 tok/s
step 17984/19560 | loss 3.303125 (+0.02z)| norm 0.2387 (-0.92z)| lr 1.03e-05 | 323.76 ms | 52.1% bf16 MFU | 1623043 tok/s
step 17985/19560 | loss 3.262596 (-0.84z)| norm 0.2391 (-0.91z)| lr 1.03e-05 | 323.09 ms | 52.2% bf16 MFU | 1623027 tok/s
step 17986/19560 | loss 3.252680 (-1.04z)| norm 0.2335 (-1.06z)| lr 1.03e-05 | 322.53 ms | 52.3% bf16 MFU | 1623154 tok/s
step 17987/19560 | loss 3.300690 (-0.03z)| norm 0.2401 (-0.86z)| lr 1.03e-05 | 323.49 ms | 52.2% bf16 MFU | 1623032 tok/s
step 17988/19560 | loss 3.278435 (-0.50z)| norm 0.2407 (-0.83z)| lr 1.02e-05 | 322.74 ms | 52.3% bf16 MFU | 1623105 tok/s
step 17989/19560 | loss 3.273777 (-0.59z)| norm 0.2416 (-0.80z)| lr 1.02e-05 | 322.37 ms | 52.4% bf16 MFU | 1623268 tok/s
step 17990/19560 | loss 3.333389 (+0.67z)| norm 0.2571 (-0.34z)| lr 1.02e-05 | 322.71 ms | 52.3% bf16 MFU | 1623336 tok/s
step 17991/19560 | loss 3.240731 (-1.29z)| norm 0.2380 (-0.88z)| lr 1.02e-05 | 322.58 ms | 52.3% bf16 MFU | 1623435 tok/s
step 17992/19560 | loss 3.320119 (+0.42z)| norm 0.3035 (+1.01z)| lr 1.02e-05 | 322.84 ms | 52.3% bf16 MFU | 1623463 tok/s
step 17993/19560 | loss 3.303632 (+0.06z)| norm 0.2688 (+0.01z)| lr 1.02e-05 | 323.31 ms | 52.2% bf16 MFU | 1623372 tok/s
step 17994/19560 | loss 3.283492 (-0.37z)| norm 0.2552 (-0.38z)| lr 1.02e-05 | 322.17 ms | 52.4% bf16 MFU | 1623572 tok/s
step 17995/19560 | loss 3.318911 (+0.41z)| norm 0.3328 (+1.83z)| lr 1.01e-05 | 322.56 ms | 52.3% bf16 MFU | 1623662 tok/s
step 17996/19560 | loss 3.327309 (+0.59z)| norm 0.2424 (-0.74z)| lr 1.01e-05 | 323.19 ms | 52.2% bf16 MFU | 1623590 tok/s
step 17997/19560 | loss 3.294260 (-0.14z)| norm 0.2474 (-0.59z)| lr 1.01e-05 | 323.10 ms | 52.2% bf16 MFU | 1623545 tok/s
step 17998/19560 | loss 3.239035 (-1.33z)| norm 0.2485 (-0.56z)| lr 1.01e-05 | 322.48 ms | 52.3% bf16 MFU | 1623658 tok/s
step 17999/19560 | loss 3.321126 (+0.45z)| norm 0.2726 (+0.12z)| lr 1.01e-05 | 322.95 ms | 52.3% bf16 MFU | 1623646 tok/s
step 18000/19560 | loss 3.277573 (-0.53z)| norm 0.2398 (-0.80z)| lr 1.01e-05 | 322.88 ms | 52.3% bf16 MFU | 1623654 tok/s
val loss 3.276819
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3058/10042 = 0.304521
step 18001/19560 | loss 3.250238 (-1.12z)| norm 0.3496 (+2.28z)| lr 1.01e-05 | 324.43 ms | 52.0% bf16 MFU | 1623271 tok/s
step 18002/19560 | loss 3.299793 (-0.01z)| norm 0.2352 (-0.92z)| lr 1.01e-05 | 322.62 ms | 52.3% bf16 MFU | 1623363 tok/s
step 18003/19560 | loss 3.260585 (-0.88z)| norm 0.2548 (-0.37z)| lr 1.00e-05 | 323.71 ms | 52.1% bf16 MFU | 1623176 tok/s
step 18004/19560 | loss 3.242704 (-1.26z)| norm 0.3200 (+1.42z)| lr 1.00e-05 | 323.29 ms | 52.2% bf16 MFU | 1623104 tok/s
step 18005/19560 | loss 3.291542 (-0.17z)| norm 0.3960 (+3.36z)| lr 1.00e-05 | 323.36 ms | 52.2% bf16 MFU | 1623017 tok/s
step 18006/19560 | loss 3.279990 (-0.43z)| norm 0.2831 (+0.35z)| lr 1.00e-05 | 323.01 ms | 52.2% bf16 MFU | 1623022 tok/s
step 18007/19560 | loss 3.265726 (-0.74z)| norm 0.2524 (-0.47z)| lr 1.00e-05 | 322.54 ms | 52.3% bf16 MFU | 1623144 tok/s
step 18008/19560 | loss 3.297340 (-0.04z)| norm 0.3140 (+1.15z)| lr 9.98e-06 | 323.91 ms | 52.1% bf16 MFU | 1622919 tok/s
step 18009/19560 | loss 3.274894 (-0.53z)| norm 0.4173 (+3.66z)| lr 9.97e-06 | 322.49 ms | 52.3% bf16 MFU | 1623060 tok/s
step 18010/19560 | loss 3.313731 (+0.34z)| norm 0.3100 (+0.94z)| lr 9.96e-06 | 322.83 ms | 52.3% bf16 MFU | 1623108 tok/s
step 18011/19560 | loss 3.260129 (-0.85z)| norm 0.2591 (-0.33z)| lr 9.94e-06 | 322.81 ms | 52.3% bf16 MFU | 1623160 tok/s
step 18012/19560 | loss 3.341240 (+0.97z)| norm 0.2905 (+0.45z)| lr 9.93e-06 | 322.56 ms | 52.3% bf16 MFU | 1623273 tok/s
step 18013/19560 | loss 3.270450 (-0.63z)| norm 0.2806 (+0.20z)| lr 9.92e-06 | 323.00 ms | 52.3% bf16 MFU | 1623269 tok/s
step 18014/19560 | loss 3.208427 (-1.98z)| norm 0.3435 (+1.75z)| lr 9.91e-06 | 322.68 ms | 52.3% bf16 MFU | 1623347 tok/s
step 18015/19560 | loss 3.315227 (+0.40z)| norm 0.2598 (-0.33z)| lr 9.89e-06 | 322.98 ms | 52.3% bf16 MFU | 1623342 tok/s
step 18016/19560 | loss 3.310914 (+0.31z)| norm 0.2588 (-0.36z)| lr 9.88e-06 | 322.50 ms | 52.3% bf16 MFU | 1623461 tok/s
step 18017/19560 | loss 3.295258 (-0.03z)| norm 0.2555 (-0.43z)| lr 9.87e-06 | 322.94 ms | 52.3% bf16 MFU | 1623462 tok/s
step 18018/19560 | loss 3.105268 (-4.12z)| norm 0.4967 (+5.06z)| lr 9.85e-06 | 322.48 ms | 52.3% bf16 MFU | 1623578 tok/s
step 18019/19560 | loss 3.351151 (+1.23z)| norm 0.2971 (+0.51z)| lr 9.84e-06 | 322.59 ms | 52.3% bf16 MFU | 1623662 tok/s
step 18020/19560 | loss 3.306831 (+0.27z)| norm 0.2558 (-0.43z)| lr 9.83e-06 | 322.87 ms | 52.3% bf16 MFU | 1623672 tok/s
step 18021/19560 | loss 3.292673 (-0.04z)| norm 0.2767 (+0.05z)| lr 9.82e-06 | 322.70 ms | 52.3% bf16 MFU | 1623724 tok/s
step 18022/19560 | loss 3.294436 (+0.01z)| norm 0.3113 (+0.82z)| lr 9.80e-06 | 323.03 ms | 52.2% bf16 MFU | 1623688 tok/s
step 18023/19560 | loss 3.302835 (+0.19z)| norm 0.3160 (+0.92z)| lr 9.79e-06 | 323.10 ms | 52.2% bf16 MFU | 1623638 tok/s
step 18024/19560 | loss 3.241401 (-1.14z)| norm 0.2564 (-0.44z)| lr 9.78e-06 | 322.30 ms | 52.4% bf16 MFU | 1623791 tok/s
step 18025/19560 | loss 3.338908 (+0.99z)| norm 0.2855 (+0.21z)| lr 9.77e-06 | 322.81 ms | 52.3% bf16 MFU | 1623809 tok/s
step 18026/19560 | loss 3.239223 (-1.18z)| norm 0.2471 (-0.66z)| lr 9.75e-06 | 322.95 ms | 52.3% bf16 MFU | 1623791 tok/s
step 18027/19560 | loss 3.291780 (-0.04z)| norm 0.3638 (+1.95z)| lr 9.74e-06 | 322.68 ms | 52.3% bf16 MFU | 1623841 tok/s
step 18028/19560 | loss 3.264989 (-0.62z)| norm 0.3240 (+1.04z)| lr 9.73e-06 | 322.63 ms | 52.3% bf16 MFU | 1623902 tok/s
step 18029/19560 | loss 3.258021 (-0.77z)| norm 0.2804 (+0.05z)| lr 9.72e-06 | 322.52 ms | 52.3% bf16 MFU | 1623986 tok/s
step 18030/19560 | loss 3.320369 (+0.61z)| norm 0.2467 (-0.69z)| lr 9.70e-06 | 322.85 ms | 52.3% bf16 MFU | 1623983 tok/s
step 18031/19560 | loss 3.279403 (-0.28z)| norm 0.2531 (-0.54z)| lr 9.69e-06 | 322.66 ms | 52.3% bf16 MFU | 1624027 tok/s
step 18032/19560 | loss 3.301585 (+0.22z)| norm 0.2632 (-0.31z)| lr 9.68e-06 | 322.38 ms | 52.4% bf16 MFU | 1624140 tok/s
step 18033/19560 | loss 3.268613 (-0.52z)| norm 0.2635 (-0.30z)| lr 9.67e-06 | 322.68 ms | 52.3% bf16 MFU | 1624172 tok/s
step 18034/19560 | loss 3.270647 (-0.47z)| norm 0.2607 (-0.36z)| lr 9.65e-06 | 322.77 ms | 52.3% bf16 MFU | 1624181 tok/s
step 18035/19560 | loss 3.270873 (-0.47z)| norm 0.2461 (-0.69z)| lr 9.64e-06 | 322.72 ms | 52.3% bf16 MFU | 1624200 tok/s
step 18036/19560 | loss 3.361184 (+1.54z)| norm 0.2598 (-0.38z)| lr 9.63e-06 | 322.87 ms | 52.3% bf16 MFU | 1624182 tok/s
step 18037/19560 | loss 3.335819 (+0.99z)| norm 0.2941 (+0.41z)| lr 9.61e-06 | 322.60 ms | 52.3% bf16 MFU | 1624233 tok/s
step 18038/19560 | loss 3.413571 (+2.69z)| norm 0.3139 (+0.85z)| lr 9.60e-06 | 322.70 ms | 52.3% bf16 MFU | 1624257 tok/s
step 18039/19560 | loss 3.334188 (+0.92z)| norm 0.2708 (-0.14z)| lr 9.59e-06 | 322.74 ms | 52.3% bf16 MFU | 1624268 tok/s
step 18040/19560 | loss 3.311039 (+0.40z)| norm 0.2884 (+0.26z)| lr 9.58e-06 | 322.61 ms | 52.3% bf16 MFU | 1624312 tok/s
step 18041/19560 | loss 3.260840 (-0.71z)| norm 0.2362 (-0.93z)| lr 9.56e-06 | 322.93 ms | 52.3% bf16 MFU | 1624272 tok/s
step 18042/19560 | loss 3.289304 (-0.08z)| norm 0.2472 (-0.67z)| lr 9.55e-06 | 322.61 ms | 52.3% bf16 MFU | 1624315 tok/s
step 18043/19560 | loss 3.330692 (+0.83z)| norm 0.2498 (-0.61z)| lr 9.54e-06 | 322.50 ms | 52.3% bf16 MFU | 1624385 tok/s
step 18044/19560 | loss 3.247438 (-1.01z)| norm 0.3035 (+0.64z)| lr 9.53e-06 | 322.44 ms | 52.3% bf16 MFU | 1624466 tok/s
step 18045/19560 | loss 3.354131 (+1.37z)| norm 0.2610 (-0.35z)| lr 9.51e-06 | 322.22 ms | 52.4% bf16 MFU | 1624599 tok/s
step 18046/19560 | loss 3.296557 (+0.07z)| norm 0.2421 (-0.79z)| lr 9.50e-06 | 322.54 ms | 52.3% bf16 MFU | 1624643 tok/s
step 18047/19560 | loss 3.315170 (+0.48z)| norm 0.2504 (-0.59z)| lr 9.49e-06 | 322.34 ms | 52.4% bf16 MFU | 1624736 tok/s
step 18048/19560 | loss 3.290925 (-0.06z)| norm 0.2662 (-0.23z)| lr 9.48e-06 | 322.91 ms | 52.3% bf16 MFU | 1624681 tok/s
step 18049/19560 | loss 3.273539 (-0.46z)| norm 0.2680 (-0.19z)| lr 9.46e-06 | 322.65 ms | 52.3% bf16 MFU | 1624696 tok/s
step 18050/19560 | loss 3.250565 (-0.99z)| norm 0.3198 (+1.00z)| lr 9.45e-06 | 322.56 ms | 52.3% bf16 MFU | 1624730 tok/s
step 18051/19560 | loss 3.255111 (-0.87z)| norm 0.2622 (-0.34z)| lr 9.44e-06 | 322.50 ms | 52.3% bf16 MFU | 1624778 tok/s
step 18052/19560 | loss 3.300788 (+0.20z)| norm 0.2411 (-0.82z)| lr 9.43e-06 | 322.66 ms | 52.3% bf16 MFU | 1624783 tok/s
step 18053/19560 | loss 3.330683 (+0.91z)| norm 0.2757 (-0.02z)| lr 9.42e-06 | 322.68 ms | 52.3% bf16 MFU | 1624783 tok/s
step 18054/19560 | loss 3.252488 (-0.95z)| norm 0.2354 (-0.95z)| lr 9.40e-06 | 322.49 ms | 52.3% bf16 MFU | 1624832 tok/s
step 18055/19560 | loss 3.257799 (-0.81z)| norm 0.2501 (-0.61z)| lr 9.39e-06 | 322.79 ms | 52.3% bf16 MFU | 1624804 tok/s
step 18056/19560 | loss 3.335398 (+1.05z)| norm 0.2499 (-0.62z)| lr 9.38e-06 | 323.61 ms | 52.2% bf16 MFU | 1624569 tok/s
step 18057/19560 | loss 3.274988 (-0.40z)| norm 0.2603 (-0.37z)| lr 9.37e-06 | 322.24 ms | 52.4% bf16 MFU | 1624690 tok/s
step 18058/19560 | loss 3.347419 (+1.37z)| norm 0.2620 (-0.32z)| lr 9.35e-06 | 322.59 ms | 52.3% bf16 MFU | 1624719 tok/s
step 18059/19560 | loss 3.458013 (+3.81z)| norm 0.3071 (+0.72z)| lr 9.34e-06 | 322.68 ms | 52.3% bf16 MFU | 1624724 tok/s
step 18060/19560 | loss 3.269989 (-0.50z)| norm 0.2983 (+0.50z)| lr 9.33e-06 | 322.70 ms | 52.3% bf16 MFU | 1624721 tok/s
step 18061/19560 | loss 3.268250 (-0.53z)| norm 0.2583 (-0.43z)| lr 9.32e-06 | 323.05 ms | 52.2% bf16 MFU | 1624632 tok/s
step 18062/19560 | loss 3.297654 (+0.15z)| norm 0.2701 (-0.16z)| lr 9.30e-06 | 322.40 ms | 52.3% bf16 MFU | 1624711 tok/s
step 18063/19560 | loss 3.328208 (+0.84z)| norm 0.3313 (+1.25z)| lr 9.29e-06 | 322.81 ms | 52.3% bf16 MFU | 1624682 tok/s
step 18064/19560 | loss 3.292306 (+0.03z)| norm 0.2320 (-1.04z)| lr 9.28e-06 | 322.71 ms | 52.3% bf16 MFU | 1624680 tok/s
step 18065/19560 | loss 3.284553 (-0.15z)| norm 0.2641 (-0.30z)| lr 9.27e-06 | 322.52 ms | 52.3% bf16 MFU | 1624727 tok/s
step 18066/19560 | loss 3.254292 (-0.84z)| norm 0.2612 (-0.36z)| lr 9.25e-06 | 323.07 ms | 52.2% bf16 MFU | 1624632 tok/s
step 18067/19560 | loss 3.332141 (+0.96z)| norm 0.2534 (-0.53z)| lr 9.24e-06 | 322.30 ms | 52.4% bf16 MFU | 1624735 tok/s
step 18068/19560 | loss 3.298731 (+0.18z)| norm 0.2403 (-0.84z)| lr 9.23e-06 | 322.58 ms | 52.3% bf16 MFU | 1624764 tok/s
step 18069/19560 | loss 3.286163 (-0.10z)| norm 0.2497 (-0.61z)| lr 9.22e-06 | 323.10 ms | 52.2% bf16 MFU | 1624660 tok/s
step 18070/19560 | loss 3.285571 (-0.10z)| norm 0.2584 (-0.39z)| lr 9.21e-06 | 322.42 ms | 52.3% bf16 MFU | 1624731 tok/s
step 18071/19560 | loss 3.302575 (+0.28z)| norm 0.2785 (+0.08z)| lr 9.19e-06 | 322.49 ms | 52.3% bf16 MFU | 1624780 tok/s
step 18072/19560 | loss 3.257066 (-0.77z)| norm 0.2538 (-0.51z)| lr 9.18e-06 | 322.96 ms | 52.3% bf16 MFU | 1624711 tok/s
step 18073/19560 | loss 3.265426 (-0.58z)| norm 0.2890 (+0.31z)| lr 9.17e-06 | 322.33 ms | 52.4% bf16 MFU | 1624804 tok/s
step 18074/19560 | loss 3.348826 (+1.35z)| norm 0.2542 (-0.50z)| lr 9.16e-06 | 322.80 ms | 52.3% bf16 MFU | 1624774 tok/s
step 18075/19560 | loss 3.241889 (-1.11z)| norm 0.2447 (-0.72z)| lr 9.14e-06 | 322.90 ms | 52.3% bf16 MFU | 1624719 tok/s
step 18076/19560 | loss 3.238966 (-1.17z)| norm 0.2372 (-0.90z)| lr 9.13e-06 | 323.01 ms | 52.2% bf16 MFU | 1624640 tok/s
step 18077/19560 | loss 3.288430 (-0.03z)| norm 0.2472 (-0.66z)| lr 9.12e-06 | 322.62 ms | 52.3% bf16 MFU | 1624661 tok/s
step 18078/19560 | loss 3.291446 (+0.04z)| norm 0.2427 (-0.76z)| lr 9.11e-06 | 322.87 ms | 52.3% bf16 MFU | 1624620 tok/s
step 18079/19560 | loss 3.324018 (+0.87z)| norm 0.2785 (+0.09z)| lr 9.09e-06 | 322.92 ms | 52.3% bf16 MFU | 1624569 tok/s
step 18080/19560 | loss 3.268722 (-0.48z)| norm 0.2569 (-0.43z)| lr 9.08e-06 | 322.83 ms | 52.3% bf16 MFU | 1624543 tok/s
step 18081/19560 | loss 3.280964 (-0.20z)| norm 0.2352 (-0.94z)| lr 9.07e-06 | 322.68 ms | 52.3% bf16 MFU | 1624555 tok/s
step 18082/19560 | loss 3.315452 (+0.66z)| norm 0.2543 (-0.48z)| lr 9.06e-06 | 322.54 ms | 52.3% bf16 MFU | 1624601 tok/s
step 18083/19560 | loss 3.291688 (+0.08z)| norm 0.2464 (-0.67z)| lr 9.05e-06 | 322.99 ms | 52.3% bf16 MFU | 1624534 tok/s
step 18084/19560 | loss 3.182864 (-2.54z)| norm 0.2337 (-0.97z)| lr 9.03e-06 | 322.46 ms | 52.3% bf16 MFU | 1624602 tok/s
step 18085/19560 | loss 3.339252 (+1.23z)| norm 0.2681 (-0.13z)| lr 9.02e-06 | 322.71 ms | 52.3% bf16 MFU | 1624604 tok/s
step 18086/19560 | loss 3.413209 (+2.89z)| norm 0.2550 (-0.45z)| lr 9.01e-06 | 322.23 ms | 52.4% bf16 MFU | 1624726 tok/s
step 18087/19560 | loss 3.285221 (-0.10z)| norm 0.2385 (-0.85z)| lr 9.00e-06 | 322.48 ms | 52.3% bf16 MFU | 1624779 tok/s
step 18088/19560 | loss 3.250527 (-0.91z)| norm 0.2316 (-1.01z)| lr 8.99e-06 | 322.54 ms | 52.3% bf16 MFU | 1624814 tok/s
step 18089/19560 | loss 3.331791 (+0.98z)| norm 0.2369 (-0.86z)| lr 8.97e-06 | 323.02 ms | 52.2% bf16 MFU | 1624727 tok/s
step 18090/19560 | loss 3.317214 (+0.63z)| norm 0.2561 (-0.37z)| lr 8.96e-06 | 322.53 ms | 52.3% bf16 MFU | 1624767 tok/s
step 18091/19560 | loss 3.276684 (-0.32z)| norm 0.2470 (-0.60z)| lr 8.95e-06 | 322.97 ms | 52.3% bf16 MFU | 1624695 tok/s
step 18092/19560 | loss 3.352931 (+1.45z)| norm 0.2328 (-0.94z)| lr 8.94e-06 | 322.41 ms | 52.3% bf16 MFU | 1624768 tok/s
step 18093/19560 | loss 3.305650 (+0.34z)| norm 0.2534 (-0.42z)| lr 8.92e-06 | 322.46 ms | 52.3% bf16 MFU | 1624824 tok/s
step 18094/19560 | loss 3.258686 (-0.76z)| norm 0.2340 (-0.91z)| lr 8.91e-06 | 322.65 ms | 52.3% bf16 MFU | 1624830 tok/s
step 18095/19560 | loss 3.254554 (-0.86z)| norm 0.2876 (+0.47z)| lr 8.90e-06 | 323.13 ms | 52.2% bf16 MFU | 1624716 tok/s
step 18096/19560 | loss 3.286077 (-0.12z)| norm 0.2289 (-1.04z)| lr 8.89e-06 | 323.11 ms | 52.2% bf16 MFU | 1624612 tok/s
step 18097/19560 | loss 3.279256 (-0.29z)| norm 0.2547 (-0.37z)| lr 8.88e-06 | 322.66 ms | 52.3% bf16 MFU | 1624625 tok/s
step 18098/19560 | loss 3.364171 (+1.68z)| norm 0.2594 (-0.24z)| lr 8.86e-06 | 322.77 ms | 52.3% bf16 MFU | 1624610 tok/s
step 18099/19560 | loss 3.236067 (-1.29z)| norm 0.2324 (-0.94z)| lr 8.85e-06 | 322.66 ms | 52.3% bf16 MFU | 1624625 tok/s
step 18100/19560 | loss 3.305411 (+0.32z)| norm 0.2368 (-0.82z)| lr 8.84e-06 | 322.41 ms | 52.3% bf16 MFU | 1624702 tok/s
step 18101/19560 | loss 3.275729 (-0.36z)| norm 0.2527 (-0.41z)| lr 8.83e-06 | 323.14 ms | 52.2% bf16 MFU | 1624590 tok/s
step 18102/19560 | loss 3.338038 (+1.09z)| norm 0.2432 (-0.65z)| lr 8.82e-06 | 323.31 ms | 52.2% bf16 MFU | 1624442 tok/s
step 18103/19560 | loss 3.273688 (-0.41z)| norm 0.2461 (-0.57z)| lr 8.80e-06 | 323.09 ms | 52.2% bf16 MFU | 1624357 tok/s
step 18104/19560 | loss 3.268604 (-0.53z)| norm 0.2445 (-0.61z)| lr 8.79e-06 | 323.27 ms | 52.2% bf16 MFU | 1624231 tok/s
step 18105/19560 | loss 3.305219 (+0.32z)| norm 0.2585 (-0.25z)| lr 8.78e-06 | 321.92 ms | 52.4% bf16 MFU | 1624450 tok/s
step 18106/19560 | loss 3.299850 (+0.19z)| norm 0.2347 (-0.86z)| lr 8.77e-06 | 323.34 ms | 52.2% bf16 MFU | 1624302 tok/s
step 18107/19560 | loss 3.320006 (+0.67z)| norm 0.3078 (+1.01z)| lr 8.76e-06 | 323.03 ms | 52.2% bf16 MFU | 1624237 tok/s
step 18108/19560 | loss 3.266597 (-0.58z)| norm 0.2385 (-0.76z)| lr 8.74e-06 | 322.33 ms | 52.4% bf16 MFU | 1624352 tok/s
step 18109/19560 | loss 3.285963 (-0.14z)| norm 0.2506 (-0.45z)| lr 8.73e-06 | 322.66 ms | 52.3% bf16 MFU | 1624378 tok/s
step 18110/19560 | loss 3.291953 (+0.00z)| norm 0.2469 (-0.55z)| lr 8.72e-06 | 322.46 ms | 52.3% bf16 MFU | 1624455 tok/s
step 18111/19560 | loss 3.274716 (-0.40z)| norm 0.2392 (-0.73z)| lr 8.71e-06 | 323.41 ms | 52.2% bf16 MFU | 1624288 tok/s
step 18112/19560 | loss 3.378399 (+2.01z)| norm 0.2590 (-0.23z)| lr 8.70e-06 | 322.53 ms | 52.3% bf16 MFU | 1624350 tok/s
step 18113/19560 | loss 3.254946 (-0.87z)| norm 0.2628 (-0.14z)| lr 8.68e-06 | 322.63 ms | 52.3% bf16 MFU | 1624386 tok/s
step 18114/19560 | loss 3.254602 (-0.88z)| norm 0.2411 (-0.70z)| lr 8.67e-06 | 322.95 ms | 52.3% bf16 MFU | 1624339 tok/s
step 18115/19560 | loss 3.321892 (+0.68z)| norm 0.2669 (-0.04z)| lr 8.66e-06 | 323.18 ms | 52.2% bf16 MFU | 1624237 tok/s
step 18116/19560 | loss 3.307480 (+0.34z)| norm 0.3122 (+1.12z)| lr 8.65e-06 | 323.43 ms | 52.2% bf16 MFU | 1624076 tok/s
step 18117/19560 | loss 3.218666 (-1.69z)| norm 0.2487 (-0.52z)| lr 8.64e-06 | 322.49 ms | 52.3% bf16 MFU | 1624161 tok/s
step 18118/19560 | loss 3.232972 (-1.34z)| norm 0.2463 (-0.58z)| lr 8.62e-06 | 322.91 ms | 52.3% bf16 MFU | 1624134 tok/s
step 18119/19560 | loss 3.274617 (-0.39z)| norm 0.2778 (+0.22z)| lr 8.61e-06 | 323.32 ms | 52.2% bf16 MFU | 1624007 tok/s
step 18120/19560 | loss 3.226882 (-1.47z)| norm 0.2379 (-0.80z)| lr 8.60e-06 | 323.14 ms | 52.2% bf16 MFU | 1623930 tok/s
step 18121/19560 | loss 3.265759 (-0.57z)| norm 0.2345 (-0.87z)| lr 8.59e-06 | 323.14 ms | 52.2% bf16 MFU | 1623856 tok/s
step 18122/19560 | loss 3.348753 (+1.31z)| norm 0.2404 (-0.72z)| lr 8.58e-06 | 322.80 ms | 52.3% bf16 MFU | 1623873 tok/s
step 18123/19560 | loss 3.255478 (-0.80z)| norm 0.2725 (+0.12z)| lr 8.57e-06 | 322.92 ms | 52.3% bf16 MFU | 1623860 tok/s
step 18124/19560 | loss 3.244022 (-1.04z)| norm 0.2820 (+0.36z)| lr 8.55e-06 | 322.85 ms | 52.3% bf16 MFU | 1623864 tok/s
step 18125/19560 | loss 3.263321 (-0.60z)| norm 0.2873 (+0.49z)| lr 8.54e-06 | 322.89 ms | 52.3% bf16 MFU | 1623858 tok/s
step 18126/19560 | loss 3.281090 (-0.21z)| norm 0.2368 (-0.82z)| lr 8.53e-06 | 323.21 ms | 52.2% bf16 MFU | 1623771 tok/s
step 18127/19560 | loss 3.270257 (-0.44z)| norm 0.3147 (+1.19z)| lr 8.52e-06 | 322.96 ms | 52.3% bf16 MFU | 1623752 tok/s
step 18128/19560 | loss 3.243855 (-1.03z)| norm 0.2296 (-1.01z)| lr 8.51e-06 | 322.70 ms | 52.3% bf16 MFU | 1623799 tok/s
step 18129/19560 | loss 3.277971 (-0.27z)| norm 0.2717 (+0.10z)| lr 8.49e-06 | 322.90 ms | 52.3% bf16 MFU | 1623792 tok/s
step 18130/19560 | loss 3.312814 (+0.52z)| norm 0.2567 (-0.30z)| lr 8.48e-06 | 322.75 ms | 52.3% bf16 MFU | 1623824 tok/s
step 18131/19560 | loss 3.304469 (+0.33z)| norm 0.2585 (-0.25z)| lr 8.47e-06 | 322.60 ms | 52.3% bf16 MFU | 1623892 tok/s
step 18132/19560 | loss 3.283493 (-0.16z)| norm 0.2375 (-0.80z)| lr 8.46e-06 | 323.06 ms | 52.2% bf16 MFU | 1623843 tok/s
step 18133/19560 | loss 3.293777 (+0.08z)| norm 0.2371 (-0.81z)| lr 8.45e-06 | 322.67 ms | 52.3% bf16 MFU | 1623893 tok/s
step 18134/19560 | loss 3.315964 (+0.58z)| norm 0.2619 (-0.12z)| lr 8.44e-06 | 323.04 ms | 52.2% bf16 MFU | 1623847 tok/s
step 18135/19560 | loss 3.248695 (-0.95z)| norm 0.2388 (-0.76z)| lr 8.42e-06 | 322.90 ms | 52.3% bf16 MFU | 1623839 tok/s
step 18136/19560 | loss 3.301317 (+0.24z)| norm 0.2418 (-0.66z)| lr 8.41e-06 | 322.98 ms | 52.3% bf16 MFU | 1623812 tok/s
step 18137/19560 | loss 3.234095 (-1.27z)| norm 0.2655 (+0.04z)| lr 8.40e-06 | 322.73 ms | 52.3% bf16 MFU | 1623849 tok/s
step 18138/19560 | loss 3.260365 (-0.67z)| norm 0.2649 (+0.03z)| lr 8.39e-06 | 322.81 ms | 52.3% bf16 MFU | 1623864 tok/s
step 18139/19560 | loss 3.297543 (+0.17z)| norm 0.2652 (+0.04z)| lr 8.38e-06 | 322.93 ms | 52.3% bf16 MFU | 1623848 tok/s
step 18140/19560 | loss 3.289175 (-0.01z)| norm 0.2757 (+0.36z)| lr 8.37e-06 | 322.72 ms | 52.3% bf16 MFU | 1623885 tok/s
step 18141/19560 | loss 3.305904 (+0.36z)| norm 0.2533 (-0.31z)| lr 8.35e-06 | 322.96 ms | 52.3% bf16 MFU | 1623861 tok/s
step 18142/19560 | loss 3.239938 (-1.16z)| norm 0.2471 (-0.49z)| lr 8.34e-06 | 322.76 ms | 52.3% bf16 MFU | 1623888 tok/s
step 18143/19560 | loss 3.268791 (-0.49z)| norm 0.2390 (-0.74z)| lr 8.33e-06 | 322.94 ms | 52.3% bf16 MFU | 1623867 tok/s
step 18144/19560 | loss 3.299971 (+0.23z)| norm 0.2448 (-0.55z)| lr 8.32e-06 | 322.53 ms | 52.3% bf16 MFU | 1623952 tok/s
step 18145/19560 | loss 3.274762 (-0.34z)| norm 0.2539 (-0.27z)| lr 8.31e-06 | 322.75 ms | 52.3% bf16 MFU | 1623977 tok/s
step 18146/19560 | loss 3.239751 (-1.26z)| norm 0.3118 (+2.03z)| lr 8.29e-06 | 322.79 ms | 52.3% bf16 MFU | 1623990 tok/s
step 18147/19560 | loss 3.289799 (-0.01z)| norm 0.2348 (-1.04z)| lr 8.28e-06 | 322.61 ms | 52.3% bf16 MFU | 1624048 tok/s
step 18148/19560 | loss 3.358533 (+1.67z)| norm 0.2614 (+0.03z)| lr 8.27e-06 | 323.58 ms | 52.2% bf16 MFU | 1623860 tok/s
step 18149/19560 | loss 3.340536 (+1.21z)| norm 0.2592 (-0.05z)| lr 8.26e-06 | 322.62 ms | 52.3% bf16 MFU | 1623922 tok/s
step 18150/19560 | loss 3.316361 (+0.61z)| norm 0.2416 (-0.75z)| lr 8.25e-06 | 322.76 ms | 52.3% bf16 MFU | 1623946 tok/s
step 18151/19560 | loss 3.354456 (+1.52z)| norm 0.2767 (+0.71z)| lr 8.24e-06 | 322.77 ms | 52.3% bf16 MFU | 1623966 tok/s
step 18152/19560 | loss 3.317811 (+0.62z)| norm 0.2572 (-0.10z)| lr 8.22e-06 | 322.51 ms | 52.3% bf16 MFU | 1624050 tok/s
step 18153/19560 | loss 3.248234 (-1.05z)| norm 0.2877 (+1.16z)| lr 8.21e-06 | 323.38 ms | 52.2% bf16 MFU | 1623912 tok/s
step 18154/19560 | loss 3.343321 (+1.24z)| norm 0.2535 (-0.26z)| lr 8.20e-06 | 322.92 ms | 52.3% bf16 MFU | 1623894 tok/s
step 18155/19560 | loss 3.277004 (-0.37z)| norm 0.2592 (+0.01z)| lr 8.19e-06 | 322.67 ms | 52.3% bf16 MFU | 1623943 tok/s
step 18156/19560 | loss 3.265049 (-0.66z)| norm 0.2455 (-0.60z)| lr 8.18e-06 | 323.14 ms | 52.2% bf16 MFU | 1623868 tok/s
step 18157/19560 | loss 3.337813 (+1.09z)| norm 0.2771 (+0.88z)| lr 8.17e-06 | 323.03 ms | 52.2% bf16 MFU | 1623826 tok/s
step 18158/19560 | loss 3.248721 (-1.05z)| norm 0.2644 (+0.28z)| lr 8.16e-06 | 323.07 ms | 52.2% bf16 MFU | 1623775 tok/s
step 18159/19560 | loss 3.309776 (+0.42z)| norm 0.2348 (-1.09z)| lr 8.14e-06 | 322.50 ms | 52.3% bf16 MFU | 1623873 tok/s
step 18160/19560 | loss 3.264848 (-0.66z)| norm 0.2561 (-0.10z)| lr 8.13e-06 | 323.00 ms | 52.3% bf16 MFU | 1623837 tok/s
step 18161/19560 | loss 3.266262 (-0.63z)| norm 0.2452 (-0.60z)| lr 8.12e-06 | 323.28 ms | 52.2% bf16 MFU | 1623735 tok/s
step 18162/19560 | loss 3.228844 (-1.51z)| norm 0.2480 (-0.46z)| lr 8.11e-06 | 323.12 ms | 52.2% bf16 MFU | 1623678 tok/s
step 18163/19560 | loss 3.294602 (+0.06z)| norm 0.2607 (+0.12z)| lr 8.10e-06 | 324.10 ms | 52.1% bf16 MFU | 1623378 tok/s
step 18164/19560 | loss 3.355886 (+1.53z)| norm 0.2455 (-0.58z)| lr 8.09e-06 | 322.22 ms | 52.4% bf16 MFU | 1623565 tok/s
step 18165/19560 | loss 3.271950 (-0.47z)| norm 0.2317 (-1.20z)| lr 8.07e-06 | 323.06 ms | 52.2% bf16 MFU | 1623532 tok/s
step 18166/19560 | loss 3.306728 (+0.40z)| norm 0.2359 (-1.00z)| lr 8.06e-06 | 323.25 ms | 52.2% bf16 MFU | 1623453 tok/s
step 18167/19560 | loss 3.345499 (+1.36z)| norm 0.2343 (-1.06z)| lr 8.05e-06 | 323.03 ms | 52.2% bf16 MFU | 1623431 tok/s
step 18168/19560 | loss 3.375797 (+2.07z)| norm 0.2664 (+0.48z)| lr 8.04e-06 | 323.87 ms | 52.1% bf16 MFU | 1623201 tok/s
step 18169/19560 | loss 3.318507 (+0.66z)| norm 0.2463 (-0.49z)| lr 8.03e-06 | 322.88 ms | 52.3% bf16 MFU | 1623230 tok/s
step 18170/19560 | loss 3.281429 (-0.25z)| norm 0.2457 (-0.52z)| lr 8.02e-06 | 322.69 ms | 52.3% bf16 MFU | 1623306 tok/s
step 18171/19560 | loss 3.280353 (-0.27z)| norm 0.2516 (-0.24z)| lr 8.01e-06 | 322.35 ms | 52.4% bf16 MFU | 1623463 tok/s
step 18172/19560 | loss 3.259138 (-0.79z)| norm 0.2742 (+0.87z)| lr 7.99e-06 | 323.27 ms | 52.2% bf16 MFU | 1623381 tok/s
step 18173/19560 | loss 3.333387 (+1.04z)| norm 0.2472 (-0.44z)| lr 7.98e-06 | 323.67 ms | 52.1% bf16 MFU | 1623204 tok/s
step 18174/19560 | loss 3.256294 (-0.85z)| norm 0.2394 (-0.82z)| lr 7.97e-06 | 322.96 ms | 52.3% bf16 MFU | 1623214 tok/s
step 18175/19560 | loss 3.219366 (-1.73z)| norm 0.2460 (-0.50z)| lr 7.96e-06 | 323.27 ms | 52.2% bf16 MFU | 1623145 tok/s
step 18176/19560 | loss 3.323337 (+0.80z)| norm 0.2574 (+0.06z)| lr 7.95e-06 | 322.65 ms | 52.3% bf16 MFU | 1623236 tok/s
step 18177/19560 | loss 3.259176 (-0.76z)| norm 0.2743 (+0.89z)| lr 7.94e-06 | 323.47 ms | 52.2% bf16 MFU | 1623114 tok/s
step 18178/19560 | loss 3.313140 (+0.55z)| norm 0.2736 (+0.90z)| lr 7.93e-06 | 322.74 ms | 52.3% bf16 MFU | 1623184 tok/s
step 18179/19560 | loss 3.283214 (-0.19z)| norm 0.2514 (-0.21z)| lr 7.91e-06 | 322.79 ms | 52.3% bf16 MFU | 1623237 tok/s
step 18180/19560 | loss 3.258051 (-0.80z)| norm 0.2384 (-0.87z)| lr 7.90e-06 | 322.70 ms | 52.3% bf16 MFU | 1623311 tok/s
step 18181/19560 | loss 3.287686 (-0.06z)| norm 0.2542 (-0.07z)| lr 7.89e-06 | 322.69 ms | 52.3% bf16 MFU | 1623382 tok/s
step 18182/19560 | loss 3.320388 (+0.72z)| norm 0.2656 (+0.50z)| lr 7.88e-06 | 322.76 ms | 52.3% bf16 MFU | 1623432 tok/s
step 18183/19560 | loss 3.299257 (+0.20z)| norm 0.3046 (+2.41z)| lr 7.87e-06 | 323.32 ms | 52.2% bf16 MFU | 1623338 tok/s
step 18184/19560 | loss 3.297302 (+0.16z)| norm 0.2328 (-1.15z)| lr 7.86e-06 | 323.09 ms | 52.2% bf16 MFU | 1623309 tok/s
step 18185/19560 | loss 3.294877 (+0.09z)| norm 0.2512 (-0.24z)| lr 7.85e-06 | 322.79 ms | 52.3% bf16 MFU | 1623356 tok/s
step 18186/19560 | loss 3.251232 (-0.97z)| norm 0.2655 (+0.47z)| lr 7.83e-06 | 323.01 ms | 52.2% bf16 MFU | 1623344 tok/s
step 18187/19560 | loss 3.324625 (+0.94z)| norm 0.3429 (+4.10z)| lr 7.82e-06 | 323.11 ms | 52.2% bf16 MFU | 1623310 tok/s
step 18188/19560 | loss 3.277841 (-0.31z)| norm 0.2679 (+0.57z)| lr 7.81e-06 | 323.37 ms | 52.2% bf16 MFU | 1623211 tok/s
step 18189/19560 | loss 3.342680 (+1.40z)| norm 0.2703 (+0.68z)| lr 7.80e-06 | 322.87 ms | 52.3% bf16 MFU | 1623242 tok/s
step 18190/19560 | loss 3.274953 (-0.39z)| norm 0.3188 (+2.89z)| lr 7.79e-06 | 323.06 ms | 52.2% bf16 MFU | 1623223 tok/s
step 18191/19560 | loss 3.304563 (+0.40z)| norm 0.3058 (+2.36z)| lr 7.78e-06 | 323.29 ms | 52.2% bf16 MFU | 1623148 tok/s
step 18192/19560 | loss 3.309649 (+0.53z)| norm 0.2489 (-0.36z)| lr 7.77e-06 | 323.35 ms | 52.2% bf16 MFU | 1623062 tok/s
step 18193/19560 | loss 3.325196 (+0.93z)| norm 0.2581 (+0.08z)| lr 7.76e-06 | 322.66 ms | 52.3% bf16 MFU | 1623154 tok/s
step 18194/19560 | loss 3.306631 (+0.43z)| norm 0.2359 (-0.97z)| lr 7.74e-06 | 323.18 ms | 52.2% bf16 MFU | 1623111 tok/s
step 18195/19560 | loss 3.273977 (-0.42z)| norm 0.2709 (+0.69z)| lr 7.73e-06 | 323.03 ms | 52.2% bf16 MFU | 1623106 tok/s
step 18196/19560 | loss 3.301892 (+0.32z)| norm 0.2522 (-0.20z)| lr 7.72e-06 | 322.92 ms | 52.3% bf16 MFU | 1623130 tok/s
step 18197/19560 | loss 3.245766 (-1.16z)| norm 0.2524 (-0.19z)| lr 7.71e-06 | 323.13 ms | 52.2% bf16 MFU | 1623101 tok/s
step 18198/19560 | loss 3.279444 (-0.27z)| norm 0.2462 (-0.48z)| lr 7.70e-06 | 322.30 ms | 52.4% bf16 MFU | 1623281 tok/s
step 18199/19560 | loss 3.261158 (-0.74z)| norm 0.2710 (+0.71z)| lr 7.69e-06 | 323.06 ms | 52.2% bf16 MFU | 1623262 tok/s
step 18200/19560 | loss 3.270471 (-0.50z)| norm 0.2581 (+0.09z)| lr 7.68e-06 | 322.76 ms | 52.3% bf16 MFU | 1623318 tok/s
step 18201/19560 | loss 3.262463 (-0.71z)| norm 0.2569 (+0.04z)| lr 7.67e-06 | 323.06 ms | 52.2% bf16 MFU | 1623297 tok/s
step 18202/19560 | loss 3.246073 (-1.13z)| norm 0.2598 (+0.18z)| lr 7.65e-06 | 322.85 ms | 52.3% bf16 MFU | 1623330 tok/s
step 18203/19560 | loss 3.291718 (+0.07z)| norm 0.2386 (-0.84z)| lr 7.64e-06 | 322.59 ms | 52.3% bf16 MFU | 1623426 tok/s
step 18204/19560 | loss 3.309621 (+0.54z)| norm 0.3607 (+4.59z)| lr 7.63e-06 | 323.20 ms | 52.2% bf16 MFU | 1623362 tok/s
step 18205/19560 | loss 3.276087 (-0.36z)| norm 0.2743 (+0.76z)| lr 7.62e-06 | 322.61 ms | 52.3% bf16 MFU | 1623452 tok/s
step 18206/19560 | loss 3.335985 (+1.24z)| norm 0.2904 (+1.44z)| lr 7.61e-06 | 322.54 ms | 52.3% bf16 MFU | 1623555 tok/s
step 18207/19560 | loss 3.278079 (-0.30z)| norm 0.2538 (-0.16z)| lr 7.60e-06 | 322.60 ms | 52.3% bf16 MFU | 1623636 tok/s
step 18208/19560 | loss 3.300336 (+0.29z)| norm 0.3091 (+2.22z)| lr 7.59e-06 | 322.26 ms | 52.4% bf16 MFU | 1623799 tok/s
step 18209/19560 | loss 3.248247 (-1.10z)| norm 0.3117 (+2.26z)| lr 7.58e-06 | 322.96 ms | 52.3% bf16 MFU | 1623778 tok/s
step 18210/19560 | loss 3.357856 (+1.80z)| norm 0.2508 (-0.32z)| lr 7.56e-06 | 322.67 ms | 52.3% bf16 MFU | 1623831 tok/s
step 18211/19560 | loss 3.300945 (+0.29z)| norm 0.2510 (-0.32z)| lr 7.55e-06 | 322.95 ms | 52.3% bf16 MFU | 1623812 tok/s
step 18212/19560 | loss 3.308278 (+0.48z)| norm 0.2785 (+0.84z)| lr 7.54e-06 | 322.79 ms | 52.3% bf16 MFU | 1623834 tok/s
step 18213/19560 | loss 3.302843 (+0.34z)| norm 0.2639 (+0.22z)| lr 7.53e-06 | 323.03 ms | 52.2% bf16 MFU | 1623794 tok/s
step 18214/19560 | loss 3.286234 (-0.09z)| norm 0.3088 (+2.08z)| lr 7.52e-06 | 322.83 ms | 52.3% bf16 MFU | 1623806 tok/s
step 18215/19560 | loss 3.470689 (+4.70z)| norm 0.3044 (+1.85z)| lr 7.51e-06 | 322.80 ms | 52.3% bf16 MFU | 1623826 tok/s
step 18216/19560 | loss 3.311576 (+0.53z)| norm 0.2512 (-0.36z)| lr 7.50e-06 | 323.34 ms | 52.2% bf16 MFU | 1623709 tok/s
step 18217/19560 | loss 3.260058 (-0.81z)| norm 0.2394 (-0.85z)| lr 7.49e-06 | 322.45 ms | 52.3% bf16 MFU | 1623821 tok/s
step 18218/19560 | loss 3.335699 (+1.17z)| norm 0.2490 (-0.45z)| lr 7.48e-06 | 322.75 ms | 52.3% bf16 MFU | 1623852 tok/s
step 18219/19560 | loss 3.278137 (-0.34z)| norm 0.2472 (-0.52z)| lr 7.46e-06 | 323.15 ms | 52.2% bf16 MFU | 1623782 tok/s
step 18220/19560 | loss 3.263429 (-0.71z)| norm 0.3226 (+2.53z)| lr 7.45e-06 | 322.84 ms | 52.3% bf16 MFU | 1623793 tok/s
step 18221/19560 | loss 3.283839 (-0.17z)| norm 0.2731 (+0.51z)| lr 7.44e-06 | 322.59 ms | 52.3% bf16 MFU | 1623866 tok/s
step 18222/19560 | loss 3.235214 (-1.44z)| norm 0.2247 (-1.46z)| lr 7.43e-06 | 322.99 ms | 52.3% bf16 MFU | 1623835 tok/s
step 18223/19560 | loss 3.280475 (-0.25z)| norm 0.2589 (-0.06z)| lr 7.42e-06 | 322.62 ms | 52.3% bf16 MFU | 1623898 tok/s
step 18224/19560 | loss 3.298440 (+0.22z)| norm 0.3006 (+1.61z)| lr 7.41e-06 | 322.91 ms | 52.3% bf16 MFU | 1623885 tok/s
step 18225/19560 | loss 3.316673 (+0.69z)| norm 0.2378 (-0.93z)| lr 7.40e-06 | 322.81 ms | 52.3% bf16 MFU | 1623898 tok/s
step 18226/19560 | loss 3.319988 (+0.79z)| norm 0.2567 (-0.16z)| lr 7.39e-06 | 322.77 ms | 52.3% bf16 MFU | 1623921 tok/s
step 18227/19560 | loss 3.253360 (-0.99z)| norm 0.2560 (-0.20z)| lr 7.38e-06 | 322.77 ms | 52.3% bf16 MFU | 1623943 tok/s
step 18228/19560 | loss 3.231151 (-1.55z)| norm 0.2617 (+0.02z)| lr 7.37e-06 | 322.22 ms | 52.4% bf16 MFU | 1624102 tok/s
step 18229/19560 | loss 3.296801 (+0.18z)| norm 0.2496 (-0.47z)| lr 7.35e-06 | 323.00 ms | 52.3% bf16 MFU | 1624056 tok/s
step 18230/19560 | loss 3.310152 (+0.55z)| norm 0.2409 (-0.82z)| lr 7.34e-06 | 322.37 ms | 52.4% bf16 MFU | 1624170 tok/s
step 18231/19560 | loss 3.303604 (+0.36z)| norm 0.2383 (-0.93z)| lr 7.33e-06 | 322.72 ms | 52.3% bf16 MFU | 1624192 tok/s
step 18232/19560 | loss 3.266025 (-0.64z)| norm 0.2386 (-0.91z)| lr 7.32e-06 | 323.13 ms | 52.2% bf16 MFU | 1624110 tok/s
step 18233/19560 | loss 3.206771 (-2.16z)| norm 0.2510 (-0.40z)| lr 7.31e-06 | 322.32 ms | 52.4% bf16 MFU | 1624233 tok/s
step 18234/19560 | loss 3.274297 (-0.38z)| norm 0.2305 (-1.23z)| lr 7.30e-06 | 322.31 ms | 52.4% bf16 MFU | 1624354 tok/s
step 18235/19560 | loss 3.254862 (-0.88z)| norm 0.2366 (-0.97z)| lr 7.29e-06 | 322.56 ms | 52.3% bf16 MFU | 1624404 tok/s
step 18236/19560 | loss 3.256876 (-0.82z)| norm 0.2465 (-0.57z)| lr 7.28e-06 | 322.93 ms | 52.3% bf16 MFU | 1624362 tok/s
step 18237/19560 | loss 3.298646 (+0.27z)| norm 0.2505 (-0.40z)| lr 7.27e-06 | 322.94 ms | 52.3% bf16 MFU | 1624318 tok/s
step 18238/19560 | loss 3.338927 (+1.30z)| norm 0.2798 (+0.78z)| lr 7.26e-06 | 322.46 ms | 52.3% bf16 MFU | 1624398 tok/s
step 18239/19560 | loss 3.275047 (-0.36z)| norm 0.2826 (+0.89z)| lr 7.24e-06 | 322.60 ms | 52.3% bf16 MFU | 1624436 tok/s
step 18240/19560 | loss 3.304962 (+0.44z)| norm 0.2420 (-0.77z)| lr 7.23e-06 | 322.72 ms | 52.3% bf16 MFU | 1624445 tok/s
step 18241/19560 | loss 3.280036 (-0.22z)| norm 0.2406 (-0.82z)| lr 7.22e-06 | 322.83 ms | 52.3% bf16 MFU | 1624424 tok/s
step 18242/19560 | loss 3.248935 (-1.05z)| norm 0.2474 (-0.55z)| lr 7.21e-06 | 322.54 ms | 52.3% bf16 MFU | 1624478 tok/s
step 18243/19560 | loss 3.265040 (-0.61z)| norm 0.2545 (-0.25z)| lr 7.20e-06 | 322.49 ms | 52.3% bf16 MFU | 1624541 tok/s
step 18244/19560 | loss 3.268386 (-0.51z)| norm 0.2444 (-0.65z)| lr 7.19e-06 | 322.75 ms | 52.3% bf16 MFU | 1624537 tok/s
step 18245/19560 | loss 3.311881 (+0.63z)| norm 0.2245 (-1.46z)| lr 7.18e-06 | 323.01 ms | 52.2% bf16 MFU | 1624467 tok/s
step 18246/19560 | loss 3.295741 (+0.19z)| norm 0.2402 (-0.81z)| lr 7.17e-06 | 323.52 ms | 52.2% bf16 MFU | 1624271 tok/s
step 18247/19560 | loss 3.343452 (+1.46z)| norm 0.2325 (-1.11z)| lr 7.16e-06 | 322.78 ms | 52.3% bf16 MFU | 1624273 tok/s
step 18248/19560 | loss 3.370337 (+2.13z)| norm 0.2440 (-0.64z)| lr 7.15e-06 | 322.81 ms | 52.3% bf16 MFU | 1624267 tok/s
step 18249/19560 | loss 3.249267 (-1.10z)| norm 0.2487 (-0.45z)| lr 7.14e-06 | 322.33 ms | 52.4% bf16 MFU | 1624380 tok/s
step 18250/19560 | loss 3.349565 (+1.58z)| norm 0.2546 (-0.21z)| lr 7.13e-06 | 322.68 ms | 52.3% bf16 MFU | 1624401 tok/s
val loss 3.275970
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3060/10042 = 0.304720
step 18251/19560 | loss 3.304297 (+0.36z)| norm 0.2367 (-0.94z)| lr 7.11e-06 | 322.25 ms | 52.4% bf16 MFU | 1624529 tok/s
step 18252/19560 | loss 3.262317 (-0.77z)| norm 0.2693 (+0.41z)| lr 7.10e-06 | 322.38 ms | 52.4% bf16 MFU | 1624618 tok/s
step 18253/19560 | loss 3.312063 (+0.56z)| norm 0.2401 (-0.78z)| lr 7.09e-06 | 322.85 ms | 52.3% bf16 MFU | 1624583 tok/s
step 18254/19560 | loss 3.262503 (-0.77z)| norm 0.2463 (-0.53z)| lr 7.08e-06 | 322.53 ms | 52.3% bf16 MFU | 1624632 tok/s
step 18255/19560 | loss 3.279706 (-0.31z)| norm 0.2416 (-0.72z)| lr 7.07e-06 | 323.14 ms | 52.2% bf16 MFU | 1624525 tok/s
step 18256/19560 | loss 3.253023 (-1.03z)| norm 0.2529 (-0.25z)| lr 7.06e-06 | 322.65 ms | 52.3% bf16 MFU | 1624546 tok/s
step 18257/19560 | loss 3.289636 (-0.05z)| norm 0.2374 (-0.90z)| lr 7.05e-06 | 322.82 ms | 52.3% bf16 MFU | 1624523 tok/s
step 18258/19560 | loss 3.409735 (+3.05z)| norm 0.2540 (-0.19z)| lr 7.04e-06 | 322.33 ms | 52.4% bf16 MFU | 1624625 tok/s
step 18259/19560 | loss 3.294162 (+0.05z)| norm 0.2342 (-1.02z)| lr 7.03e-06 | 322.85 ms | 52.3% bf16 MFU | 1624591 tok/s
step 18260/19560 | loss 3.264549 (-0.71z)| norm 0.2268 (-1.32z)| lr 7.02e-06 | 322.48 ms | 52.3% bf16 MFU | 1624652 tok/s
step 18261/19560 | loss 3.294351 (+0.06z)| norm 0.2423 (-0.67z)| lr 7.01e-06 | 322.63 ms | 52.3% bf16 MFU | 1624671 tok/s
step 18262/19560 | loss 3.235911 (-1.43z)| norm 0.2594 (+0.05z)| lr 7.00e-06 | 322.51 ms | 52.3% bf16 MFU | 1624719 tok/s
step 18263/19560 | loss 3.340103 (+1.24z)| norm 0.2967 (+1.60z)| lr 6.98e-06 | 322.96 ms | 52.3% bf16 MFU | 1624653 tok/s
step 18264/19560 | loss 3.260531 (-0.80z)| norm 0.2734 (+0.61z)| lr 6.97e-06 | 322.77 ms | 52.3% bf16 MFU | 1624636 tok/s
step 18265/19560 | loss 3.236544 (-1.42z)| norm 0.3096 (+2.08z)| lr 6.96e-06 | 322.69 ms | 52.3% bf16 MFU | 1624642 tok/s
step 18266/19560 | loss 3.242791 (-1.25z)| norm 0.2679 (+0.36z)| lr 6.95e-06 | 323.34 ms | 52.2% bf16 MFU | 1624482 tok/s
step 18267/19560 | loss 3.258122 (-0.85z)| norm 0.2376 (-0.88z)| lr 6.94e-06 | 322.50 ms | 52.3% bf16 MFU | 1624544 tok/s
step 18268/19560 | loss 3.309216 (+0.45z)| norm 0.2639 (+0.21z)| lr 6.93e-06 | 323.13 ms | 52.2% bf16 MFU | 1624444 tok/s
step 18269/19560 | loss 3.251060 (-1.01z)| norm 0.2509 (-0.33z)| lr 6.92e-06 | 322.68 ms | 52.3% bf16 MFU | 1624461 tok/s
step 18270/19560 | loss 3.328261 (+0.93z)| norm 0.2524 (-0.27z)| lr 6.91e-06 | 322.94 ms | 52.3% bf16 MFU | 1624412 tok/s
step 18271/19560 | loss 3.284261 (-0.19z)| norm 0.2436 (-0.63z)| lr 6.90e-06 | 322.77 ms | 52.3% bf16 MFU | 1624408 tok/s
step 18272/19560 | loss 3.221688 (-1.75z)| norm 0.2465 (-0.52z)| lr 6.89e-06 | 323.06 ms | 52.2% bf16 MFU | 1624332 tok/s
step 18273/19560 | loss 3.276875 (-0.36z)| norm 0.2532 (-0.24z)| lr 6.88e-06 | 322.07 ms | 52.4% bf16 MFU | 1624510 tok/s
step 18274/19560 | loss 3.367501 (+1.88z)| norm 0.2454 (-0.55z)| lr 6.87e-06 | 322.52 ms | 52.3% bf16 MFU | 1624564 tok/s
step 18275/19560 | loss 3.273562 (-0.46z)| norm 0.2630 (+0.18z)| lr 6.86e-06 | 322.99 ms | 52.3% bf16 MFU | 1624497 tok/s
step 18276/19560 | loss 3.299218 (+0.19z)| norm 0.2676 (+0.37z)| lr 6.85e-06 | 322.66 ms | 52.3% bf16 MFU | 1624517 tok/s
step 18277/19560 | loss 3.323662 (+0.81z)| norm 0.2609 (+0.09z)| lr 6.84e-06 | 322.60 ms | 52.3% bf16 MFU | 1624550 tok/s
step 18278/19560 | loss 3.354879 (+1.58z)| norm 0.2651 (+0.26z)| lr 6.83e-06 | 322.76 ms | 52.3% bf16 MFU | 1624541 tok/s
step 18279/19560 | loss 3.304588 (+0.33z)| norm 0.2568 (-0.09z)| lr 6.81e-06 | 322.98 ms | 52.3% bf16 MFU | 1624479 tok/s
step 18280/19560 | loss 3.301710 (+0.26z)| norm 0.2864 (+1.15z)| lr 6.80e-06 | 322.44 ms | 52.3% bf16 MFU | 1624555 tok/s
step 18281/19560 | loss 3.318192 (+0.67z)| norm 0.2675 (+0.37z)| lr 6.79e-06 | 322.85 ms | 52.3% bf16 MFU | 1624523 tok/s
step 18282/19560 | loss 3.269699 (-0.55z)| norm 0.2779 (+0.80z)| lr 6.78e-06 | 322.53 ms | 52.3% bf16 MFU | 1624574 tok/s
step 18283/19560 | loss 3.294365 (+0.08z)| norm 0.2525 (-0.28z)| lr 6.77e-06 | 323.23 ms | 52.2% bf16 MFU | 1624447 tok/s
step 18284/19560 | loss 3.297517 (+0.15z)| norm 0.2310 (-1.17z)| lr 6.76e-06 | 322.58 ms | 52.3% bf16 MFU | 1624491 tok/s
step 18285/19560 | loss 3.240027 (-1.30z)| norm 0.2763 (+0.73z)| lr 6.75e-06 | 322.73 ms | 52.3% bf16 MFU | 1624494 tok/s
step 18286/19560 | loss 3.381496 (+2.26z)| norm 0.2433 (-0.65z)| lr 6.74e-06 | 322.98 ms | 52.3% bf16 MFU | 1624433 tok/s
step 18287/19560 | loss 3.260010 (-0.79z)| norm 0.2439 (-0.63z)| lr 6.73e-06 | 322.81 ms | 52.3% bf16 MFU | 1624417 tok/s
step 18288/19560 | loss 3.285838 (-0.15z)| norm 0.2328 (-1.08z)| lr 6.72e-06 | 322.78 ms | 52.3% bf16 MFU | 1624411 tok/s
step 18289/19560 | loss 3.273460 (-0.46z)| norm 0.2884 (+1.23z)| lr 6.71e-06 | 322.78 ms | 52.3% bf16 MFU | 1624405 tok/s
step 18290/19560 | loss 3.317614 (+0.64z)| norm 0.2406 (-0.76z)| lr 6.70e-06 | 322.63 ms | 52.3% bf16 MFU | 1624437 tok/s
step 18291/19560 | loss 3.362110 (+1.74z)| norm 0.2587 (-0.01z)| lr 6.69e-06 | 322.72 ms | 52.3% bf16 MFU | 1624445 tok/s
step 18292/19560 | loss 3.277931 (-0.37z)| norm 0.2708 (+0.49z)| lr 6.68e-06 | 322.59 ms | 52.3% bf16 MFU | 1624486 tok/s
step 18293/19560 | loss 3.344641 (+1.31z)| norm 0.3226 (+2.56z)| lr 6.67e-06 | 323.00 ms | 52.3% bf16 MFU | 1624422 tok/s
step 18294/19560 | loss 3.356292 (+1.58z)| norm 0.2315 (-1.15z)| lr 6.66e-06 | 323.23 ms | 52.2% bf16 MFU | 1624301 tok/s
step 18295/19560 | loss 3.304217 (+0.28z)| norm 0.2722 (+0.49z)| lr 6.65e-06 | 322.50 ms | 52.3% bf16 MFU | 1624372 tok/s
step 18296/19560 | loss 3.308629 (+0.41z)| norm 0.2436 (-0.66z)| lr 6.64e-06 | 322.74 ms | 52.3% bf16 MFU | 1624377 tok/s
step 18297/19560 | loss 3.303639 (+0.29z)| norm 0.2372 (-0.92z)| lr 6.63e-06 | 322.81 ms | 52.3% bf16 MFU | 1624366 tok/s
step 18298/19560 | loss 3.290740 (-0.04z)| norm 0.2495 (-0.42z)| lr 6.61e-06 | 322.68 ms | 52.3% bf16 MFU | 1624386 tok/s
step 18299/19560 | loss 3.299921 (+0.19z)| norm 0.2658 (+0.24z)| lr 6.60e-06 | 322.42 ms | 52.3% bf16 MFU | 1624473 tok/s
step 18300/19560 | loss 3.269761 (-0.59z)| norm 0.2480 (-0.48z)| lr 6.59e-06 | 322.86 ms | 52.3% bf16 MFU | 1624443 tok/s
step 18301/19560 | loss 3.322317 (+0.77z)| norm 0.2325 (-1.10z)| lr 6.58e-06 | 322.72 ms | 52.3% bf16 MFU | 1624450 tok/s
step 18302/19560 | loss 3.266093 (-0.69z)| norm 0.2351 (-0.99z)| lr 6.57e-06 | 322.83 ms | 52.3% bf16 MFU | 1624429 tok/s
step 18303/19560 | loss 3.259406 (-0.88z)| norm 0.2318 (-1.12z)| lr 6.56e-06 | 323.07 ms | 52.2% bf16 MFU | 1624350 tok/s
step 18304/19560 | loss 3.297732 (+0.13z)| norm 0.2504 (-0.36z)| lr 6.55e-06 | 322.66 ms | 52.3% bf16 MFU | 1624378 tok/s
step 18305/19560 | loss 3.353915 (+1.57z)| norm 0.2623 (+0.12z)| lr 6.54e-06 | 322.47 ms | 52.3% bf16 MFU | 1624452 tok/s
step 18306/19560 | loss 3.320873 (+0.71z)| norm 0.2419 (-0.70z)| lr 6.53e-06 | 322.92 ms | 52.3% bf16 MFU | 1624409 tok/s
step 18307/19560 | loss 3.277966 (-0.40z)| norm 0.2285 (-1.22z)| lr 6.52e-06 | 322.91 ms | 52.3% bf16 MFU | 1624371 tok/s
step 18308/19560 | loss 3.253462 (-1.04z)| norm 0.2481 (-0.44z)| lr 6.51e-06 | 322.46 ms | 52.3% bf16 MFU | 1624447 tok/s
step 18309/19560 | loss 3.284571 (-0.23z)| norm 0.2272 (-1.26z)| lr 6.50e-06 | 322.70 ms | 52.3% bf16 MFU | 1624459 tok/s
step 18310/19560 | loss 3.244888 (-1.24z)| norm 0.3052 (+1.82z)| lr 6.49e-06 | 322.59 ms | 52.3% bf16 MFU | 1624498 tok/s
step 18311/19560 | loss 3.315924 (+0.59z)| norm 0.2449 (-0.55z)| lr 6.48e-06 | 322.11 ms | 52.4% bf16 MFU | 1624657 tok/s
step 18312/19560 | loss 3.274022 (-0.48z)| norm 0.2460 (-0.51z)| lr 6.47e-06 | 323.30 ms | 52.2% bf16 MFU | 1624507 tok/s
step 18313/19560 | loss 3.264357 (-0.73z)| norm 0.2555 (-0.13z)| lr 6.46e-06 | 322.87 ms | 52.3% bf16 MFU | 1624474 tok/s
step 18314/19560 | loss 3.268197 (-0.63z)| norm 0.2463 (-0.49z)| lr 6.45e-06 | 323.03 ms | 52.2% bf16 MFU | 1624400 tok/s
step 18315/19560 | loss 3.270846 (-0.55z)| norm 0.2930 (+1.45z)| lr 6.44e-06 | 322.72 ms | 52.3% bf16 MFU | 1624410 tok/s
step 18316/19560 | loss 3.256023 (-0.93z)| norm 0.2334 (-1.02z)| lr 6.43e-06 | 323.09 ms | 52.2% bf16 MFU | 1624325 tok/s
step 18317/19560 | loss 3.362989 (+1.81z)| norm 0.2955 (+1.54z)| lr 6.42e-06 | 322.65 ms | 52.3% bf16 MFU | 1624357 tok/s
step 18318/19560 | loss 3.343546 (+1.29z)| norm 0.2539 (-0.16z)| lr 6.41e-06 | 322.98 ms | 52.3% bf16 MFU | 1624303 tok/s
step 18319/19560 | loss 3.353436 (+1.52z)| norm 0.2804 (+0.98z)| lr 6.40e-06 | 322.57 ms | 52.3% bf16 MFU | 1624354 tok/s
step 18320/19560 | loss 3.361312 (+1.69z)| norm 0.2595 (+0.09z)| lr 6.39e-06 | 322.88 ms | 52.3% bf16 MFU | 1624325 tok/s
step 18321/19560 | loss 3.315340 (+0.54z)| norm 0.2259 (-1.33z)| lr 6.38e-06 | 322.40 ms | 52.3% bf16 MFU | 1624418 tok/s
step 18322/19560 | loss 3.249848 (-1.08z)| norm 0.2426 (-0.62z)| lr 6.37e-06 | 322.69 ms | 52.3% bf16 MFU | 1624434 tok/s
step 18323/19560 | loss 3.269919 (-0.58z)| norm 0.2879 (+1.29z)| lr 6.36e-06 | 323.00 ms | 52.3% bf16 MFU | 1624371 tok/s
step 18324/19560 | loss 3.218289 (-1.82z)| norm 0.2651 (+0.32z)| lr 6.35e-06 | 322.64 ms | 52.3% bf16 MFU | 1624403 tok/s
step 18325/19560 | loss 3.267599 (-0.62z)| norm 0.2447 (-0.54z)| lr 6.34e-06 | 322.96 ms | 52.3% bf16 MFU | 1624353 tok/s
step 18326/19560 | loss 3.268054 (-0.60z)| norm 0.2586 (+0.04z)| lr 6.33e-06 | 322.71 ms | 52.3% bf16 MFU | 1624367 tok/s
step 18327/19560 | loss 3.279229 (-0.33z)| norm 0.2398 (-0.74z)| lr 6.32e-06 | 323.03 ms | 52.2% bf16 MFU | 1624301 tok/s
step 18328/19560 | loss 3.277220 (-0.38z)| norm 0.2552 (-0.09z)| lr 6.31e-06 | 323.09 ms | 52.2% bf16 MFU | 1624222 tok/s
step 18329/19560 | loss 3.316353 (+0.57z)| norm 0.2904 (+1.38z)| lr 6.30e-06 | 322.71 ms | 52.3% bf16 MFU | 1624243 tok/s
step 18330/19560 | loss 3.249847 (-1.07z)| norm 0.2611 (+0.15z)| lr 6.28e-06 | 323.14 ms | 52.2% bf16 MFU | 1624156 tok/s
step 18331/19560 | loss 3.257603 (-0.87z)| norm 0.2592 (+0.06z)| lr 6.27e-06 | 322.65 ms | 52.3% bf16 MFU | 1624194 tok/s
step 18332/19560 | loss 3.408097 (+2.74z)| norm 0.3483 (+3.88z)| lr 6.26e-06 | 323.14 ms | 52.2% bf16 MFU | 1624110 tok/s
step 18333/19560 | loss 3.260533 (-0.79z)| norm 0.2319 (-1.09z)| lr 6.25e-06 | 323.13 ms | 52.2% bf16 MFU | 1624031 tok/s
step 18334/19560 | loss 3.251353 (-0.99z)| norm 0.2361 (-0.89z)| lr 6.24e-06 | 322.39 ms | 52.3% bf16 MFU | 1624142 tok/s
step 18335/19560 | loss 3.298623 (+0.13z)| norm 0.2322 (-1.05z)| lr 6.23e-06 | 322.98 ms | 52.3% bf16 MFU | 1624100 tok/s
step 18336/19560 | loss 3.256927 (-0.85z)| norm 0.2402 (-0.70z)| lr 6.22e-06 | 322.98 ms | 52.3% bf16 MFU | 1624059 tok/s
step 18337/19560 | loss 3.295744 (+0.06z)| norm 0.2651 (+0.41z)| lr 6.21e-06 | 323.45 ms | 52.2% bf16 MFU | 1623903 tok/s
step 18338/19560 | loss 3.254398 (-0.91z)| norm 0.2820 (+1.15z)| lr 6.20e-06 | 322.80 ms | 52.3% bf16 MFU | 1623917 tok/s
step 18339/19560 | loss 3.424999 (+3.05z)| norm 0.2435 (-0.55z)| lr 6.19e-06 | 322.52 ms | 52.3% bf16 MFU | 1624001 tok/s
step 18340/19560 | loss 3.325910 (+0.75z)| norm 0.2517 (-0.18z)| lr 6.18e-06 | 322.82 ms | 52.3% bf16 MFU | 1624005 tok/s
step 18341/19560 | loss 3.217882 (-1.71z)| norm 0.2359 (-0.87z)| lr 6.17e-06 | 323.25 ms | 52.2% bf16 MFU | 1623902 tok/s
step 18342/19560 | loss 3.332915 (+0.91z)| norm 0.2277 (-1.23z)| lr 6.16e-06 | 322.99 ms | 52.3% bf16 MFU | 1623868 tok/s
step 18343/19560 | loss 3.368410 (+1.84z)| norm 0.3411 (+3.72z)| lr 6.15e-06 | 323.24 ms | 52.2% bf16 MFU | 1623774 tok/s
step 18344/19560 | loss 3.290692 (-0.03z)| norm 0.2500 (-0.23z)| lr 6.14e-06 | 322.86 ms | 52.3% bf16 MFU | 1623781 tok/s
step 18345/19560 | loss 3.339978 (+1.14z)| norm 0.2449 (-0.45z)| lr 6.13e-06 | 323.26 ms | 52.2% bf16 MFU | 1623685 tok/s
step 18346/19560 | loss 3.300889 (+0.20z)| norm 0.2603 (+0.21z)| lr 6.12e-06 | 323.33 ms | 52.2% bf16 MFU | 1623577 tok/s
step 18347/19560 | loss 3.304136 (+0.28z)| norm 0.2499 (-0.24z)| lr 6.11e-06 | 323.15 ms | 52.2% bf16 MFU | 1623520 tok/s
step 18348/19560 | loss 3.267310 (-0.61z)| norm 0.2517 (-0.14z)| lr 6.10e-06 | 323.24 ms | 52.2% bf16 MFU | 1623442 tok/s
step 18349/19560 | loss 3.287941 (-0.12z)| norm 0.2700 (+0.68z)| lr 6.09e-06 | 322.90 ms | 52.3% bf16 MFU | 1623453 tok/s
step 18350/19560 | loss 3.284806 (-0.20z)| norm 0.2327 (-1.00z)| lr 6.08e-06 | 322.54 ms | 52.3% bf16 MFU | 1623555 tok/s
step 18351/19560 | loss 3.299520 (+0.15z)| norm 0.2437 (-0.50z)| lr 6.07e-06 | 322.73 ms | 52.3% bf16 MFU | 1623605 tok/s
step 18352/19560 | loss 3.271447 (-0.53z)| norm 0.2319 (-1.02z)| lr 6.06e-06 | 323.47 ms | 52.2% bf16 MFU | 1623466 tok/s
step 18353/19560 | loss 3.360278 (+1.61z)| norm 0.2448 (-0.43z)| lr 6.05e-06 | 323.14 ms | 52.2% bf16 MFU | 1623415 tok/s
step 18354/19560 | loss 3.275997 (-0.41z)| norm 0.2701 (+0.72z)| lr 6.04e-06 | 322.43 ms | 52.3% bf16 MFU | 1623548 tok/s
step 18355/19560 | loss 3.322173 (+0.69z)| norm 0.2482 (-0.28z)| lr 6.03e-06 | 323.65 ms | 52.1% bf16 MFU | 1623366 tok/s
step 18356/19560 | loss 3.301872 (+0.19z)| norm 0.2561 (+0.08z)| lr 6.02e-06 | 323.15 ms | 52.2% bf16 MFU | 1623320 tok/s
step 18357/19560 | loss 3.270076 (-0.58z)| norm 0.2546 (+0.02z)| lr 6.01e-06 | 323.17 ms | 52.2% bf16 MFU | 1623271 tok/s
step 18358/19560 | loss 3.322970 (+0.70z)| norm 0.3161 (+2.72z)| lr 6.00e-06 | 322.66 ms | 52.3% bf16 MFU | 1623352 tok/s
step 18359/19560 | loss 3.298923 (+0.12z)| norm 0.2493 (-0.25z)| lr 5.99e-06 | 322.70 ms | 52.3% bf16 MFU | 1623420 tok/s
step 18360/19560 | loss 3.288564 (-0.14z)| norm 0.2290 (-1.15z)| lr 5.98e-06 | 324.16 ms | 52.1% bf16 MFU | 1623118 tok/s
step 18361/19560 | loss 3.314997 (+0.50z)| norm 0.2674 (+0.55z)| lr 5.97e-06 | 323.07 ms | 52.2% bf16 MFU | 1623103 tok/s
step 18362/19560 | loss 3.355330 (+1.47z)| norm 0.2784 (+1.02z)| lr 5.96e-06 | 323.58 ms | 52.2% bf16 MFU | 1622961 tok/s
step 18363/19560 | loss 3.269561 (-0.65z)| norm 0.2396 (-0.70z)| lr 5.95e-06 | 323.01 ms | 52.2% bf16 MFU | 1622969 tok/s
step 18364/19560 | loss 3.360429 (+1.57z)| norm 0.2628 (+0.32z)| lr 5.94e-06 | 323.09 ms | 52.2% bf16 MFU | 1622956 tok/s
step 18365/19560 | loss 3.284696 (-0.29z)| norm 0.2349 (-0.91z)| lr 5.93e-06 | 322.74 ms | 52.3% bf16 MFU | 1623034 tok/s
step 18366/19560 | loss 3.312004 (+0.39z)| norm 0.3296 (+3.14z)| lr 5.92e-06 | 323.22 ms | 52.2% bf16 MFU | 1622986 tok/s
step 18367/19560 | loss 3.218547 (-1.88z)| norm 0.2498 (-0.25z)| lr 5.91e-06 | 323.09 ms | 52.2% bf16 MFU | 1622974 tok/s
step 18368/19560 | loss 3.259454 (-0.87z)| norm 0.2263 (-1.24z)| lr 5.90e-06 | 322.73 ms | 52.3% bf16 MFU | 1623053 tok/s
step 18369/19560 | loss 3.278793 (-0.40z)| norm 0.2312 (-1.03z)| lr 5.89e-06 | 323.03 ms | 52.2% bf16 MFU | 1623052 tok/s
step 18370/19560 | loss 3.245700 (-1.20z)| norm 0.2369 (-0.78z)| lr 5.88e-06 | 322.75 ms | 52.3% bf16 MFU | 1623122 tok/s
step 18371/19560 | loss 3.336880 (+0.99z)| norm 0.3105 (+2.28z)| lr 5.87e-06 | 322.98 ms | 52.3% bf16 MFU | 1623130 tok/s
step 18372/19560 | loss 3.283385 (-0.31z)| norm 0.2393 (-0.68z)| lr 5.86e-06 | 323.04 ms | 52.2% bf16 MFU | 1623122 tok/s
step 18373/19560 | loss 3.478818 (+4.09z)| norm 0.2874 (+1.30z)| lr 5.85e-06 | 322.91 ms | 52.3% bf16 MFU | 1623148 tok/s
step 18374/19560 | loss 3.292300 (-0.11z)| norm 0.2467 (-0.40z)| lr 5.85e-06 | 322.90 ms | 52.3% bf16 MFU | 1623174 tok/s
step 18375/19560 | loss 3.279347 (-0.40z)| norm 0.2587 (+0.09z)| lr 5.84e-06 | 322.78 ms | 52.3% bf16 MFU | 1623231 tok/s
step 18376/19560 | loss 3.212135 (-1.88z)| norm 0.2686 (+0.50z)| lr 5.83e-06 | 322.98 ms | 52.3% bf16 MFU | 1623234 tok/s
step 18377/19560 | loss 3.327749 (+0.71z)| norm 0.2677 (+0.46z)| lr 5.82e-06 | 322.76 ms | 52.3% bf16 MFU | 1623293 tok/s
step 18378/19560 | loss 3.264473 (-0.71z)| norm 0.2501 (-0.28z)| lr 5.81e-06 | 322.93 ms | 52.3% bf16 MFU | 1623305 tok/s
step 18379/19560 | loss 3.335409 (+0.90z)| norm 0.2574 (+0.02z)| lr 5.80e-06 | 322.86 ms | 52.3% bf16 MFU | 1623333 tok/s
step 18380/19560 | loss 3.282865 (-0.30z)| norm 0.3651 (+4.18z)| lr 5.79e-06 | 323.05 ms | 52.2% bf16 MFU | 1623314 tok/s
step 18381/19560 | loss 3.266394 (-0.66z)| norm 0.3081 (+1.92z)| lr 5.78e-06 | 322.95 ms | 52.3% bf16 MFU | 1623319 tok/s
step 18382/19560 | loss 3.303203 (+0.17z)| norm 0.2617 (+0.13z)| lr 5.77e-06 | 322.69 ms | 52.3% bf16 MFU | 1623389 tok/s
step 18383/19560 | loss 3.279065 (-0.38z)| norm 0.2522 (-0.24z)| lr 5.76e-06 | 322.80 ms | 52.3% bf16 MFU | 1623430 tok/s
step 18384/19560 | loss 3.306507 (+0.23z)| norm 0.2460 (-0.47z)| lr 5.75e-06 | 322.93 ms | 52.3% bf16 MFU | 1623435 tok/s
step 18385/19560 | loss 3.321968 (+0.58z)| norm 0.2424 (-0.62z)| lr 5.74e-06 | 322.80 ms | 52.3% bf16 MFU | 1623473 tok/s
step 18386/19560 | loss 3.314429 (+0.43z)| norm 0.2475 (-0.42z)| lr 5.73e-06 | 322.59 ms | 52.3% bf16 MFU | 1623561 tok/s
step 18387/19560 | loss 3.283377 (-0.29z)| norm 0.2958 (+1.42z)| lr 5.72e-06 | 322.78 ms | 52.3% bf16 MFU | 1623598 tok/s
step 18388/19560 | loss 3.271303 (-0.57z)| norm 0.2558 (-0.12z)| lr 5.71e-06 | 322.28 ms | 52.4% bf16 MFU | 1623759 tok/s
step 18389/19560 | loss 3.251791 (-1.02z)| norm 0.2691 (+0.38z)| lr 5.70e-06 | 323.13 ms | 52.2% bf16 MFU | 1623697 tok/s
step 18390/19560 | loss 3.274949 (-0.49z)| norm 0.2633 (+0.16z)| lr 5.69e-06 | 322.91 ms | 52.3% bf16 MFU | 1623694 tok/s
step 18391/19560 | loss 3.297174 (+0.04z)| norm 0.2538 (-0.20z)| lr 5.68e-06 | 322.87 ms | 52.3% bf16 MFU | 1623702 tok/s
step 18392/19560 | loss 3.247722 (-1.12z)| norm 0.3251 (+2.49z)| lr 5.67e-06 | 322.53 ms | 52.3% bf16 MFU | 1623793 tok/s
step 18393/19560 | loss 3.338940 (+1.01z)| norm 0.2685 (+0.36z)| lr 5.66e-06 | 322.65 ms | 52.3% bf16 MFU | 1623850 tok/s
step 18394/19560 | loss 3.285440 (-0.26z)| norm 0.2620 (+0.12z)| lr 5.65e-06 | 322.99 ms | 52.3% bf16 MFU | 1623818 tok/s
step 18395/19560 | loss 3.281372 (-0.36z)| norm 0.2307 (-1.09z)| lr 5.64e-06 | 322.81 ms | 52.3% bf16 MFU | 1623834 tok/s
step 18396/19560 | loss 3.363778 (+1.57z)| norm 0.2421 (-0.64z)| lr 5.63e-06 | 322.76 ms | 52.3% bf16 MFU | 1623862 tok/s
step 18397/19560 | loss 3.242887 (-1.27z)| norm 0.2867 (+1.06z)| lr 5.62e-06 | 322.84 ms | 52.3% bf16 MFU | 1623869 tok/s
step 18398/19560 | loss 3.249028 (-1.11z)| norm 0.3093 (+1.88z)| lr 5.61e-06 | 322.97 ms | 52.3% bf16 MFU | 1623844 tok/s
step 18399/19560 | loss 3.307799 (+0.26z)| norm 0.2645 (+0.19z)| lr 5.60e-06 | 322.58 ms | 52.3% bf16 MFU | 1623915 tok/s
step 18400/19560 | loss 3.277786 (-0.46z)| norm 0.2697 (+0.37z)| lr 5.59e-06 | 322.78 ms | 52.3% bf16 MFU | 1623933 tok/s
step 18401/19560 | loss 3.221102 (-1.77z)| norm 0.2281 (-1.18z)| lr 5.58e-06 | 322.70 ms | 52.3% bf16 MFU | 1623970 tok/s
step 18402/19560 | loss 3.251251 (-1.05z)| norm 0.2498 (-0.37z)| lr 5.57e-06 | 323.17 ms | 52.2% bf16 MFU | 1623889 tok/s
step 18403/19560 | loss 3.239295 (-1.32z)| norm 0.2930 (+1.24z)| lr 5.56e-06 | 322.54 ms | 52.3% bf16 MFU | 1623970 tok/s
step 18404/19560 | loss 3.323065 (+0.64z)| norm 0.2975 (+1.39z)| lr 5.55e-06 | 322.44 ms | 52.3% bf16 MFU | 1624072 tok/s
step 18405/19560 | loss 3.409681 (+2.59z)| norm 0.3177 (+2.08z)| lr 5.54e-06 | 323.00 ms | 52.3% bf16 MFU | 1624026 tok/s
step 18406/19560 | loss 3.295321 (-0.01z)| norm 0.2631 (+0.09z)| lr 5.54e-06 | 322.46 ms | 52.3% bf16 MFU | 1624121 tok/s
step 18407/19560 | loss 3.364992 (+1.56z)| norm 0.2476 (-0.47z)| lr 5.53e-06 | 322.95 ms | 52.3% bf16 MFU | 1624086 tok/s
step 18408/19560 | loss 3.284407 (-0.27z)| norm 0.2704 (+0.37z)| lr 5.52e-06 | 322.10 ms | 52.4% bf16 MFU | 1624267 tok/s
step 18409/19560 | loss 3.312747 (+0.38z)| norm 0.2639 (+0.13z)| lr 5.51e-06 | 322.67 ms | 52.3% bf16 MFU | 1624297 tok/s
step 18410/19560 | loss 3.260920 (-0.80z)| norm 0.2712 (+0.40z)| lr 5.50e-06 | 322.66 ms | 52.3% bf16 MFU | 1624327 tok/s
step 18411/19560 | loss 3.316608 (+0.46z)| norm 0.3339 (+2.60z)| lr 5.49e-06 | 322.92 ms | 52.3% bf16 MFU | 1624290 tok/s
step 18412/19560 | loss 3.276276 (-0.45z)| norm 0.2777 (+0.59z)| lr 5.48e-06 | 322.34 ms | 52.4% bf16 MFU | 1624400 tok/s
step 18413/19560 | loss 3.282557 (-0.32z)| norm 0.2707 (+0.34z)| lr 5.47e-06 | 323.62 ms | 52.2% bf16 MFU | 1624184 tok/s
step 18414/19560 | loss 3.276243 (-0.45z)| norm 0.2412 (-0.71z)| lr 5.46e-06 | 322.17 ms | 52.4% bf16 MFU | 1624344 tok/s
step 18415/19560 | loss 3.279338 (-0.38z)| norm 0.2685 (+0.25z)| lr 5.45e-06 | 322.79 ms | 52.3% bf16 MFU | 1624339 tok/s
step 18416/19560 | loss 3.269461 (-0.61z)| norm 0.3112 (+1.74z)| lr 5.44e-06 | 323.34 ms | 52.2% bf16 MFU | 1624197 tok/s
step 18417/19560 | loss 3.397789 (+2.30z)| norm 0.3295 (+2.34z)| lr 5.43e-06 | 322.60 ms | 52.3% bf16 MFU | 1624247 tok/s
step 18418/19560 | loss 3.290841 (-0.13z)| norm 0.2482 (-0.49z)| lr 5.42e-06 | 322.47 ms | 52.3% bf16 MFU | 1624326 tok/s
step 18419/19560 | loss 3.255232 (-0.92z)| norm 0.2361 (-0.91z)| lr 5.41e-06 | 322.74 ms | 52.3% bf16 MFU | 1624334 tok/s
step 18420/19560 | loss 3.275018 (-0.47z)| norm 0.2415 (-0.71z)| lr 5.40e-06 | 322.99 ms | 52.3% bf16 MFU | 1624278 tok/s
step 18421/19560 | loss 3.256689 (-0.87z)| norm 0.2360 (-0.89z)| lr 5.39e-06 | 323.03 ms | 52.2% bf16 MFU | 1624216 tok/s
step 18422/19560 | loss 3.327988 (+0.77z)| norm 0.3260 (+2.22z)| lr 5.38e-06 | 322.79 ms | 52.3% bf16 MFU | 1624216 tok/s
step 18423/19560 | loss 3.311656 (+0.39z)| norm 0.2715 (+0.33z)| lr 5.37e-06 | 322.47 ms | 52.3% bf16 MFU | 1624298 tok/s
step 18424/19560 | loss 3.264791 (-0.68z)| norm 0.2387 (-0.81z)| lr 5.36e-06 | 323.19 ms | 52.2% bf16 MFU | 1624196 tok/s
step 18425/19560 | loss 3.275331 (-0.43z)| norm 0.2908 (+0.98z)| lr 5.36e-06 | 323.03 ms | 52.2% bf16 MFU | 1624137 tok/s
step 18426/19560 | loss 3.314532 (+0.47z)| norm 0.2434 (-0.66z)| lr 5.35e-06 | 322.79 ms | 52.3% bf16 MFU | 1624142 tok/s
step 18427/19560 | loss 3.207500 (-1.95z)| norm 0.2669 (+0.16z)| lr 5.34e-06 | 322.68 ms | 52.3% bf16 MFU | 1624174 tok/s
step 18428/19560 | loss 3.289510 (-0.09z)| norm 0.2433 (-0.66z)| lr 5.33e-06 | 323.36 ms | 52.2% bf16 MFU | 1624034 tok/s
step 18429/19560 | loss 3.286589 (-0.15z)| norm 0.2959 (+1.14z)| lr 5.32e-06 | 322.47 ms | 52.3% bf16 MFU | 1624126 tok/s
step 18430/19560 | loss 3.305709 (+0.27z)| norm 0.2503 (-0.44z)| lr 5.31e-06 | 322.78 ms | 52.3% bf16 MFU | 1624133 tok/s
step 18431/19560 | loss 3.300078 (+0.14z)| norm 0.2239 (-1.35z)| lr 5.30e-06 | 323.34 ms | 52.2% bf16 MFU | 1624000 tok/s
step 18432/19560 | loss 3.303061 (+0.20z)| norm 0.2510 (-0.41z)| lr 5.29e-06 | 322.91 ms | 52.3% bf16 MFU | 1623981 tok/s
step 18433/19560 | loss 3.269316 (-0.55z)| norm 0.2427 (-0.69z)| lr 5.28e-06 | 322.91 ms | 52.3% bf16 MFU | 1623963 tok/s
step 18434/19560 | loss 3.290096 (-0.07z)| norm 0.2840 (+0.72z)| lr 5.27e-06 | 322.47 ms | 52.3% bf16 MFU | 1624058 tok/s
step 18435/19560 | loss 3.387055 (+2.10z)| norm 0.2488 (-0.50z)| lr 5.26e-06 | 322.82 ms | 52.3% bf16 MFU | 1624058 tok/s
step 18436/19560 | loss 3.312830 (+0.41z)| norm 0.2479 (-0.53z)| lr 5.25e-06 | 323.10 ms | 52.2% bf16 MFU | 1623989 tok/s
step 18437/19560 | loss 3.264040 (-0.68z)| norm 0.2665 (+0.10z)| lr 5.24e-06 | 322.68 ms | 52.3% bf16 MFU | 1624029 tok/s
step 18438/19560 | loss 3.220165 (-1.66z)| norm 0.2697 (+0.22z)| lr 5.23e-06 | 322.81 ms | 52.3% bf16 MFU | 1624033 tok/s
step 18439/19560 | loss 3.239778 (-1.20z)| norm 0.2395 (-0.83z)| lr 5.22e-06 | 322.64 ms | 52.3% bf16 MFU | 1624082 tok/s
step 18440/19560 | loss 3.237373 (-1.24z)| norm 0.2800 (+0.58z)| lr 5.22e-06 | 322.38 ms | 52.4% bf16 MFU | 1624192 tok/s
step 18441/19560 | loss 3.303757 (+0.23z)| norm 0.2809 (+0.60z)| lr 5.21e-06 | 322.66 ms | 52.3% bf16 MFU | 1624228 tok/s
step 18442/19560 | loss 3.330583 (+0.81z)| norm 0.2877 (+0.83z)| lr 5.20e-06 | 323.04 ms | 52.2% bf16 MFU | 1624166 tok/s
step 18443/19560 | loss 3.285138 (-0.20z)| norm 0.2482 (-0.54z)| lr 5.19e-06 | 322.74 ms | 52.3% bf16 MFU | 1624183 tok/s
step 18444/19560 | loss 3.246878 (-1.05z)| norm 0.2598 (-0.14z)| lr 5.18e-06 | 322.70 ms | 52.3% bf16 MFU | 1624209 tok/s
step 18445/19560 | loss 3.318507 (+0.55z)| norm 0.2691 (+0.19z)| lr 5.17e-06 | 322.57 ms | 52.3% bf16 MFU | 1624265 tok/s
step 18446/19560 | loss 3.268420 (-0.56z)| norm 0.3006 (+1.29z)| lr 5.16e-06 | 322.82 ms | 52.3% bf16 MFU | 1624257 tok/s
step 18447/19560 | loss 3.308509 (+0.36z)| norm 0.2432 (-0.72z)| lr 5.15e-06 | 322.87 ms | 52.3% bf16 MFU | 1624237 tok/s
step 18448/19560 | loss 3.252399 (-0.90z)| norm 0.3121 (+1.67z)| lr 5.14e-06 | 322.28 ms | 52.4% bf16 MFU | 1624366 tok/s
step 18449/19560 | loss 3.265692 (-0.59z)| norm 0.2405 (-0.83z)| lr 5.13e-06 | 322.57 ms | 52.3% bf16 MFU | 1624415 tok/s
step 18450/19560 | loss 3.347256 (+1.25z)| norm 0.3066 (+1.45z)| lr 5.12e-06 | 322.32 ms | 52.4% bf16 MFU | 1624525 tok/s
step 18451/19560 | loss 3.282042 (-0.24z)| norm 0.2342 (-1.05z)| lr 5.11e-06 | 322.94 ms | 52.3% bf16 MFU | 1624472 tok/s
step 18452/19560 | loss 3.317499 (+0.56z)| norm 0.2923 (+0.96z)| lr 5.10e-06 | 322.93 ms | 52.3% bf16 MFU | 1624425 tok/s
step 18453/19560 | loss 3.256329 (-0.84z)| norm 0.2788 (+0.48z)| lr 5.10e-06 | 322.73 ms | 52.3% bf16 MFU | 1624431 tok/s
step 18454/19560 | loss 3.290169 (-0.07z)| norm 0.2442 (-0.71z)| lr 5.09e-06 | 322.80 ms | 52.3% bf16 MFU | 1624420 tok/s
step 18455/19560 | loss 3.308051 (+0.33z)| norm 0.2361 (-0.99z)| lr 5.08e-06 | 322.67 ms | 52.3% bf16 MFU | 1624441 tok/s
step 18456/19560 | loss 3.299426 (+0.13z)| norm 0.2513 (-0.46z)| lr 5.07e-06 | 322.40 ms | 52.3% bf16 MFU | 1624529 tok/s
step 18457/19560 | loss 3.289407 (-0.09z)| norm 0.2534 (-0.38z)| lr 5.06e-06 | 322.82 ms | 52.3% bf16 MFU | 1624507 tok/s
step 18458/19560 | loss 3.295923 (+0.05z)| norm 0.2654 (+0.04z)| lr 5.05e-06 | 322.49 ms | 52.3% bf16 MFU | 1624569 tok/s
step 18459/19560 | loss 3.265481 (-0.66z)| norm 0.2667 (+0.08z)| lr 5.04e-06 | 322.40 ms | 52.3% bf16 MFU | 1624650 tok/s
step 18460/19560 | loss 3.285783 (-0.17z)| norm 0.2448 (-0.67z)| lr 5.03e-06 | 322.48 ms | 52.3% bf16 MFU | 1624708 tok/s
step 18461/19560 | loss 3.319207 (+0.61z)| norm 0.2369 (-0.96z)| lr 5.02e-06 | 323.16 ms | 52.2% bf16 MFU | 1624592 tok/s
step 18462/19560 | loss 3.283206 (-0.25z)| norm 0.2627 (-0.04z)| lr 5.01e-06 | 322.73 ms | 52.3% bf16 MFU | 1624590 tok/s
step 18463/19560 | loss 3.338814 (+1.07z)| norm 0.2610 (-0.11z)| lr 5.00e-06 | 322.83 ms | 52.3% bf16 MFU | 1624563 tok/s
step 18464/19560 | loss 3.262945 (-0.74z)| norm 0.2310 (-1.19z)| lr 4.99e-06 | 322.67 ms | 52.3% bf16 MFU | 1624576 tok/s
step 18465/19560 | loss 3.288380 (-0.13z)| norm 0.2805 (+0.59z)| lr 4.99e-06 | 322.70 ms | 52.3% bf16 MFU | 1624583 tok/s
step 18466/19560 | loss 3.304446 (+0.24z)| norm 0.2367 (-0.97z)| lr 4.98e-06 | 322.81 ms | 52.3% bf16 MFU | 1624561 tok/s
step 18467/19560 | loss 3.312589 (+0.47z)| norm 0.2334 (-1.08z)| lr 4.97e-06 | 322.54 ms | 52.3% bf16 MFU | 1624607 tok/s
step 18468/19560 | loss 3.341743 (+1.19z)| norm 0.2800 (+0.57z)| lr 4.96e-06 | 323.03 ms | 52.2% bf16 MFU | 1624527 tok/s
step 18469/19560 | loss 3.302727 (+0.21z)| norm 0.2374 (-0.95z)| lr 4.95e-06 | 322.54 ms | 52.3% bf16 MFU | 1624576 tok/s
step 18470/19560 | loss 3.302942 (+0.22z)| norm 0.2973 (+1.17z)| lr 4.94e-06 | 322.66 ms | 52.3% bf16 MFU | 1624591 tok/s
step 18471/19560 | loss 3.251067 (-1.07z)| norm 0.2769 (+0.48z)| lr 4.93e-06 | 322.73 ms | 52.3% bf16 MFU | 1624588 tok/s
step 18472/19560 | loss 3.357816 (+1.61z)| norm 0.2308 (-1.22z)| lr 4.92e-06 | 322.75 ms | 52.3% bf16 MFU | 1624580 tok/s
step 18473/19560 | loss 3.189945 (-2.52z)| norm 0.2561 (-0.29z)| lr 4.91e-06 | 322.54 ms | 52.3% bf16 MFU | 1624627 tok/s
step 18474/19560 | loss 3.265759 (-0.65z)| norm 0.2845 (+0.75z)| lr 4.90e-06 | 323.00 ms | 52.3% bf16 MFU | 1624554 tok/s
step 18475/19560 | loss 3.257347 (-0.85z)| norm 0.2741 (+0.36z)| lr 4.90e-06 | 322.71 ms | 52.3% bf16 MFU | 1624559 tok/s
step 18476/19560 | loss 3.274459 (-0.43z)| norm 0.2366 (-1.01z)| lr 4.89e-06 | 322.72 ms | 52.3% bf16 MFU | 1624562 tok/s
step 18477/19560 | loss 3.280415 (-0.28z)| norm 0.2331 (-1.12z)| lr 4.88e-06 | 322.72 ms | 52.3% bf16 MFU | 1624564 tok/s
step 18478/19560 | loss 3.305898 (+0.34z)| norm 0.2587 (-0.20z)| lr 4.87e-06 | 322.46 ms | 52.3% bf16 MFU | 1624630 tok/s
step 18479/19560 | loss 3.297201 (+0.13z)| norm 0.2814 (+0.62z)| lr 4.86e-06 | 323.09 ms | 52.2% bf16 MFU | 1624536 tok/s
step 18480/19560 | loss 3.299361 (+0.18z)| norm 0.2419 (-0.83z)| lr 4.85e-06 | 322.71 ms | 52.3% bf16 MFU | 1624542 tok/s
step 18481/19560 | loss 3.252047 (-0.97z)| norm 0.2458 (-0.69z)| lr 4.84e-06 | 322.69 ms | 52.3% bf16 MFU | 1624552 tok/s
step 18482/19560 | loss 3.295097 (+0.09z)| norm 0.3022 (+1.37z)| lr 4.83e-06 | 323.04 ms | 52.2% bf16 MFU | 1624472 tok/s
step 18483/19560 | loss 3.249091 (-1.03z)| norm 0.2380 (-0.97z)| lr 4.82e-06 | 322.80 ms | 52.3% bf16 MFU | 1624457 tok/s
step 18484/19560 | loss 3.297015 (+0.15z)| norm 0.2530 (-0.42z)| lr 4.81e-06 | 322.42 ms | 52.3% bf16 MFU | 1624538 tok/s
step 18485/19560 | loss 3.421580 (+3.08z)| norm 0.2653 (+0.02z)| lr 4.81e-06 | 322.87 ms | 52.3% bf16 MFU | 1624504 tok/s
step 18486/19560 | loss 3.254136 (-0.89z)| norm 0.2405 (-0.87z)| lr 4.80e-06 | 322.76 ms | 52.3% bf16 MFU | 1624498 tok/s
step 18487/19560 | loss 3.268581 (-0.54z)| norm 0.2486 (-0.57z)| lr 4.79e-06 | 322.69 ms | 52.3% bf16 MFU | 1624510 tok/s
step 18488/19560 | loss 3.292357 (+0.02z)| norm 0.2368 (-1.01z)| lr 4.78e-06 | 322.85 ms | 52.3% bf16 MFU | 1624482 tok/s
step 18489/19560 | loss 3.247189 (-1.03z)| norm 0.2606 (-0.13z)| lr 4.77e-06 | 323.17 ms | 52.2% bf16 MFU | 1624374 tok/s
step 18490/19560 | loss 3.295195 (+0.12z)| norm 0.2534 (-0.39z)| lr 4.76e-06 | 322.35 ms | 52.4% bf16 MFU | 1624478 tok/s
step 18491/19560 | loss 3.271513 (-0.45z)| norm 0.2442 (-0.73z)| lr 4.75e-06 | 322.67 ms | 52.3% bf16 MFU | 1624497 tok/s
step 18492/19560 | loss 3.262744 (-0.65z)| norm 0.2782 (+0.52z)| lr 4.74e-06 | 323.56 ms | 52.2% bf16 MFU | 1624292 tok/s
step 18493/19560 | loss 3.303837 (+0.34z)| norm 0.2401 (-0.89z)| lr 4.73e-06 | 323.21 ms | 52.2% bf16 MFU | 1624182 tok/s
step 18494/19560 | loss 3.247440 (-1.00z)| norm 0.2406 (-0.86z)| lr 4.73e-06 | 322.83 ms | 52.3% bf16 MFU | 1624176 tok/s
step 18495/19560 | loss 3.265857 (-0.58z)| norm 0.2652 (+0.06z)| lr 4.72e-06 | 323.01 ms | 52.3% bf16 MFU | 1624125 tok/s
step 18496/19560 | loss 3.305226 (+0.37z)| norm 0.2576 (-0.24z)| lr 4.71e-06 | 322.51 ms | 52.3% bf16 MFU | 1624201 tok/s
step 18497/19560 | loss 3.332416 (+1.02z)| norm 0.2351 (-1.10z)| lr 4.70e-06 | 323.02 ms | 52.2% bf16 MFU | 1624145 tok/s
step 18498/19560 | loss 3.311229 (+0.50z)| norm 0.3477 (+3.06z)| lr 4.69e-06 | 323.39 ms | 52.2% bf16 MFU | 1623998 tok/s
step 18499/19560 | loss 3.284923 (-0.14z)| norm 0.2346 (-1.09z)| lr 4.68e-06 | 322.73 ms | 52.3% bf16 MFU | 1624026 tok/s
step 18500/19560 | loss 3.218974 (-1.72z)| norm 0.2413 (-0.85z)| lr 4.67e-06 | 322.66 ms | 52.3% bf16 MFU | 1624069 tok/s
val loss 3.275321
laSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evalualaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluaevaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3046/10042 = 0.303326
step 18501/19560 | loss 3.295618 (+0.19z)| norm 0.2338 (-1.11z)| lr 4.66e-06 | 322.80 ms | 52.3% bf16 MFU | 1624074 tok/s
step 18502/19560 | loss 3.267593 (-0.55z)| norm 0.2478 (-0.59z)| lr 4.66e-06 | 322.37 ms | 52.4% bf16 MFU | 1624188 tok/s
step 18503/19560 | loss 3.271066 (-0.45z)| norm 0.2408 (-0.84z)| lr 4.65e-06 | 322.76 ms | 52.3% bf16 MFU | 1624198 tok/s
step 18504/19560 | loss 3.266036 (-0.61z)| norm 0.2585 (-0.18z)| lr 4.64e-06 | 322.91 ms | 52.3% bf16 MFU | 1624170 tok/s
step 18505/19560 | loss 3.250096 (-1.02z)| norm 0.2381 (-0.93z)| lr 4.63e-06 | 322.73 ms | 52.3% bf16 MFU | 1624188 tok/s
step 18506/19560 | loss 3.265188 (-0.61z)| norm 0.2646 (+0.04z)| lr 4.62e-06 | 323.36 ms | 52.2% bf16 MFU | 1624046 tok/s
step 18507/19560 | loss 3.306887 (+0.51z)| norm 0.2381 (-0.92z)| lr 4.61e-06 | 323.05 ms | 52.2% bf16 MFU | 1623990 tok/s
step 18508/19560 | loss 3.249743 (-1.02z)| norm 0.2368 (-0.99z)| lr 4.60e-06 | 322.85 ms | 52.3% bf16 MFU | 1623987 tok/s
step 18509/19560 | loss 3.255617 (-0.85z)| norm 0.2320 (-1.16z)| lr 4.59e-06 | 323.06 ms | 52.2% bf16 MFU | 1623932 tok/s
step 18510/19560 | loss 3.522037 (+5.46z)| norm 0.3349 (+2.75z)| lr 4.59e-06 | 322.09 ms | 52.4% bf16 MFU | 1624122 tok/s
step 18511/19560 | loss 3.233483 (-1.29z)| norm 0.2455 (-0.63z)| lr 4.58e-06 | 322.41 ms | 52.3% bf16 MFU | 1624223 tok/s
step 18512/19560 | loss 3.252504 (-0.84z)| norm 0.2724 (+0.38z)| lr 4.57e-06 | 322.22 ms | 52.4% bf16 MFU | 1624367 tok/s
step 18513/19560 | loss 3.307722 (+0.45z)| norm 0.2648 (+0.09z)| lr 4.56e-06 | 322.60 ms | 52.3% bf16 MFU | 1624409 tok/s
step 18514/19560 | loss 3.306224 (+0.42z)| norm 0.2332 (-1.11z)| lr 4.55e-06 | 322.78 ms | 52.3% bf16 MFU | 1624402 tok/s
step 18515/19560 | loss 3.319851 (+0.73z)| norm 0.2442 (-0.68z)| lr 4.54e-06 | 322.60 ms | 52.3% bf16 MFU | 1624440 tok/s
step 18516/19560 | loss 3.225246 (-1.45z)| norm 0.2308 (-1.17z)| lr 4.53e-06 | 322.37 ms | 52.4% bf16 MFU | 1624536 tok/s
step 18517/19560 | loss 3.317980 (+0.68z)| norm 0.2734 (+0.44z)| lr 4.52e-06 | 322.47 ms | 52.3% bf16 MFU | 1624601 tok/s
step 18518/19560 | loss 3.388556 (+2.24z)| norm 0.2610 (-0.03z)| lr 4.52e-06 | 323.32 ms | 52.2% bf16 MFU | 1624450 tok/s
step 18519/19560 | loss 3.278266 (-0.25z)| norm 0.2480 (-0.52z)| lr 4.51e-06 | 323.30 ms | 52.2% bf16 MFU | 1624311 tok/s
step 18520/19560 | loss 3.294351 (+0.10z)| norm 0.2405 (-0.79z)| lr 4.50e-06 | 322.93 ms | 52.3% bf16 MFU | 1624272 tok/s
step 18521/19560 | loss 3.269547 (-0.45z)| norm 0.2854 (+0.93z)| lr 4.49e-06 | 322.87 ms | 52.3% bf16 MFU | 1624250 tok/s
step 18522/19560 | loss 3.292658 (+0.08z)| norm 0.2900 (+1.09z)| lr 4.48e-06 | 322.55 ms | 52.3% bf16 MFU | 1624310 tok/s
step 18523/19560 | loss 3.277957 (-0.26z)| norm 0.2447 (-0.65z)| lr 4.47e-06 | 322.92 ms | 52.3% bf16 MFU | 1624274 tok/s
step 18524/19560 | loss 3.352420 (+1.45z)| norm 0.2459 (-0.61z)| lr 4.46e-06 | 323.26 ms | 52.2% bf16 MFU | 1624153 tok/s
step 18525/19560 | loss 3.246663 (-0.98z)| norm 0.2528 (-0.33z)| lr 4.46e-06 | 322.89 ms | 52.3% bf16 MFU | 1624132 tok/s
step 18526/19560 | loss 3.278426 (-0.25z)| norm 0.2406 (-0.79z)| lr 4.45e-06 | 323.47 ms | 52.2% bf16 MFU | 1623968 tok/s
step 18527/19560 | loss 3.303609 (+0.33z)| norm 0.2669 (+0.23z)| lr 4.44e-06 | 322.71 ms | 52.3% bf16 MFU | 1624002 tok/s
step 18528/19560 | loss 3.258716 (-0.70z)| norm 0.2350 (-0.99z)| lr 4.43e-06 | 323.10 ms | 52.2% bf16 MFU | 1623937 tok/s
step 18529/19560 | loss 3.286252 (-0.08z)| norm 0.2310 (-1.15z)| lr 4.42e-06 | 323.23 ms | 52.2% bf16 MFU | 1623841 tok/s
step 18530/19560 | loss 3.294682 (+0.11z)| norm 0.2361 (-0.95z)| lr 4.41e-06 | 323.01 ms | 52.2% bf16 MFU | 1623805 tok/s
step 18531/19560 | loss 3.306344 (+0.37z)| norm 0.2372 (-0.89z)| lr 4.40e-06 | 323.30 ms | 52.2% bf16 MFU | 1623699 tok/s
step 18532/19560 | loss 3.306628 (+0.38z)| norm 0.2652 (+0.21z)| lr 4.40e-06 | 322.61 ms | 52.3% bf16 MFU | 1623772 tok/s
step 18533/19560 | loss 3.229812 (-1.42z)| norm 0.2486 (-0.43z)| lr 4.39e-06 | 323.11 ms | 52.2% bf16 MFU | 1623716 tok/s
step 18534/19560 | loss 3.286232 (-0.07z)| norm 0.2901 (+1.22z)| lr 4.38e-06 | 323.16 ms | 52.2% bf16 MFU | 1623648 tok/s
step 18535/19560 | loss 3.241560 (-1.12z)| norm 0.2381 (-0.85z)| lr 4.37e-06 | 323.11 ms | 52.2% bf16 MFU | 1623597 tok/s
step 18536/19560 | loss 3.266796 (-0.51z)| norm 0.3342 (+2.86z)| lr 4.36e-06 | 323.24 ms | 52.2% bf16 MFU | 1623516 tok/s
step 18537/19560 | loss 3.287360 (-0.01z)| norm 0.3611 (+3.65z)| lr 4.35e-06 | 322.85 ms | 52.3% bf16 MFU | 1623536 tok/s
step 18538/19560 | loss 3.258431 (-0.71z)| norm 0.2379 (-0.81z)| lr 4.35e-06 | 322.90 ms | 52.3% bf16 MFU | 1623545 tok/s
step 18539/19560 | loss 3.377340 (+2.12z)| norm 0.2424 (-0.64z)| lr 4.34e-06 | 323.13 ms | 52.2% bf16 MFU | 1623495 tok/s
step 18540/19560 | loss 3.200082 (-2.05z)| norm 0.2366 (-0.85z)| lr 4.33e-06 | 322.87 ms | 52.3% bf16 MFU | 1623513 tok/s
step 18541/19560 | loss 3.270485 (-0.40z)| norm 0.2311 (-1.04z)| lr 4.32e-06 | 323.08 ms | 52.2% bf16 MFU | 1623476 tok/s
step 18542/19560 | loss 3.325905 (+0.88z)| norm 0.2523 (-0.25z)| lr 4.31e-06 | 322.68 ms | 52.3% bf16 MFU | 1623543 tok/s
step 18543/19560 | loss 3.264550 (-0.54z)| norm 0.2658 (+0.25z)| lr 4.30e-06 | 322.46 ms | 52.3% bf16 MFU | 1623661 tok/s
step 18544/19560 | loss 3.373167 (+1.94z)| norm 0.2919 (+1.23z)| lr 4.29e-06 | 322.78 ms | 52.3% bf16 MFU | 1623693 tok/s
step 18545/19560 | loss 3.317929 (+0.70z)| norm 0.2359 (-0.86z)| lr 4.29e-06 | 322.53 ms | 52.3% bf16 MFU | 1623786 tok/s
step 18546/19560 | loss 3.267930 (-0.47z)| norm 0.2299 (-1.08z)| lr 4.28e-06 | 322.60 ms | 52.3% bf16 MFU | 1623855 tok/s
step 18547/19560 | loss 3.269017 (-0.44z)| norm 0.2593 (+0.04z)| lr 4.27e-06 | 323.25 ms | 52.2% bf16 MFU | 1623758 tok/s
step 18548/19560 | loss 3.322968 (+0.81z)| norm 0.2483 (-0.39z)| lr 4.26e-06 | 322.71 ms | 52.3% bf16 MFU | 1623804 tok/s
step 18549/19560 | loss 3.318885 (+0.71z)| norm 0.2815 (+0.88z)| lr 4.25e-06 | 322.69 ms | 52.3% bf16 MFU | 1623852 tok/s
step 18550/19560 | loss 3.319858 (+0.73z)| norm 0.2261 (-1.25z)| lr 4.24e-06 | 322.55 ms | 52.3% bf16 MFU | 1623931 tok/s
step 18551/19560 | loss 3.323813 (+0.82z)| norm 0.2443 (-0.53z)| lr 4.24e-06 | 322.19 ms | 52.4% bf16 MFU | 1624098 tok/s
step 18552/19560 | loss 3.189939 (-2.26z)| norm 0.2359 (-0.86z)| lr 4.23e-06 | 323.04 ms | 52.2% bf16 MFU | 1624042 tok/s
step 18553/19560 | loss 3.288623 (+0.01z)| norm 0.2319 (-1.00z)| lr 4.22e-06 | 323.02 ms | 52.2% bf16 MFU | 1623994 tok/s
step 18554/19560 | loss 3.311421 (+0.53z)| norm 0.2469 (-0.41z)| lr 4.21e-06 | 323.03 ms | 52.2% bf16 MFU | 1623945 tok/s
step 18555/19560 | loss 3.286734 (-0.05z)| norm 0.2481 (-0.36z)| lr 4.20e-06 | 322.26 ms | 52.4% bf16 MFU | 1624092 tok/s
step 18556/19560 | loss 3.265873 (-0.53z)| norm 0.2467 (-0.41z)| lr 4.19e-06 | 323.08 ms | 52.2% bf16 MFU | 1624028 tok/s
step 18557/19560 | loss 3.288366 (-0.01z)| norm 0.2347 (-0.87z)| lr 4.19e-06 | 322.47 ms | 52.3% bf16 MFU | 1624120 tok/s
step 18558/19560 | loss 3.282193 (-0.15z)| norm 0.3208 (+2.46z)| lr 4.18e-06 | 322.31 ms | 52.4% bf16 MFU | 1624246 tok/s
step 18559/19560 | loss 3.316248 (+0.64z)| norm 0.2541 (-0.13z)| lr 4.17e-06 | 322.73 ms | 52.3% bf16 MFU | 1624261 tok/s
step 18560/19560 | loss 3.333498 (+1.03z)| norm 0.2601 (+0.10z)| lr 4.16e-06 | 322.41 ms | 52.3% bf16 MFU | 1624355 tok/s
step 18561/19560 | loss 3.268862 (-0.47z)| norm 0.2516 (-0.23z)| lr 4.15e-06 | 322.80 ms | 52.3% bf16 MFU | 1624347 tok/s
step 18562/19560 | loss 3.288085 (-0.02z)| norm 0.2778 (+0.79z)| lr 4.14e-06 | 323.20 ms | 52.2% bf16 MFU | 1624239 tok/s
step 18563/19560 | loss 3.323154 (+0.82z)| norm 0.2719 (+0.55z)| lr 4.14e-06 | 322.35 ms | 52.4% bf16 MFU | 1624351 tok/s
step 18564/19560 | loss 3.255582 (-0.76z)| norm 0.2241 (-1.30z)| lr 4.13e-06 | 322.68 ms | 52.3% bf16 MFU | 1624373 tok/s
step 18565/19560 | loss 3.291715 (+0.08z)| norm 0.2551 (-0.09z)| lr 4.12e-06 | 323.11 ms | 52.2% bf16 MFU | 1624286 tok/s
step 18566/19560 | loss 3.219470 (-1.62z)| norm 0.2367 (-0.79z)| lr 4.11e-06 | 322.66 ms | 52.3% bf16 MFU | 1624317 tok/s
step 18567/19560 | loss 3.299308 (+0.25z)| norm 0.2483 (-0.35z)| lr 4.10e-06 | 322.99 ms | 52.3% bf16 MFU | 1624263 tok/s
step 18568/19560 | loss 3.309853 (+0.49z)| norm 0.2296 (-1.05z)| lr 4.09e-06 | 322.93 ms | 52.3% bf16 MFU | 1624228 tok/s
step 18569/19560 | loss 3.257128 (-0.76z)| norm 0.2530 (-0.14z)| lr 4.09e-06 | 322.53 ms | 52.3% bf16 MFU | 1624294 tok/s
step 18570/19560 | loss 3.241935 (-1.10z)| norm 0.2287 (-1.07z)| lr 4.08e-06 | 322.63 ms | 52.3% bf16 MFU | 1624333 tok/s
step 18571/19560 | loss 3.228639 (-1.39z)| norm 0.2253 (-1.19z)| lr 4.07e-06 | 322.53 ms | 52.3% bf16 MFU | 1624393 tok/s
step 18572/19560 | loss 3.337428 (+1.16z)| norm 0.2525 (-0.13z)| lr 4.06e-06 | 322.81 ms | 52.3% bf16 MFU | 1624379 tok/s
step 18573/19560 | loss 3.310782 (+0.53z)| norm 0.2637 (+0.30z)| lr 4.05e-06 | 322.70 ms | 52.3% bf16 MFU | 1624396 tok/s
step 18574/19560 | loss 3.242663 (-1.07z)| norm 0.2232 (-1.25z)| lr 4.05e-06 | 322.99 ms | 52.3% bf16 MFU | 1624338 tok/s
step 18575/19560 | loss 3.230506 (-1.33z)| norm 0.2420 (-0.52z)| lr 4.04e-06 | 322.51 ms | 52.3% bf16 MFU | 1624403 tok/s
step 18576/19560 | loss 3.232754 (-1.27z)| norm 0.2379 (-0.66z)| lr 4.03e-06 | 322.55 ms | 52.3% bf16 MFU | 1624456 tok/s
step 18577/19560 | loss 3.301935 (+0.33z)| norm 0.2467 (-0.32z)| lr 4.02e-06 | 322.70 ms | 52.3% bf16 MFU | 1624468 tok/s
step 18578/19560 | loss 3.307203 (+0.47z)| norm 0.2488 (-0.22z)| lr 4.01e-06 | 322.77 ms | 52.3% bf16 MFU | 1624461 tok/s
step 18579/19560 | loss 3.280180 (-0.17z)| norm 0.2424 (-0.48z)| lr 4.00e-06 | 322.56 ms | 52.3% bf16 MFU | 1624508 tok/s
step 18580/19560 | loss 3.239376 (-1.11z)| norm 0.2687 (+0.59z)| lr 4.00e-06 | 322.49 ms | 52.3% bf16 MFU | 1624570 tok/s
step 18581/19560 | loss 3.262766 (-0.56z)| norm 0.2441 (-0.40z)| lr 3.99e-06 | 323.06 ms | 52.2% bf16 MFU | 1624485 tok/s
step 18582/19560 | loss 3.360257 (+1.69z)| norm 0.2683 (+0.58z)| lr 3.98e-06 | 322.65 ms | 52.3% bf16 MFU | 1624507 tok/s
step 18583/19560 | loss 3.272666 (-0.33z)| norm 0.2589 (+0.19z)| lr 3.97e-06 | 322.95 ms | 52.3% bf16 MFU | 1624454 tok/s
step 18584/19560 | loss 3.250831 (-0.83z)| norm 0.2405 (-0.56z)| lr 3.96e-06 | 322.80 ms | 52.3% bf16 MFU | 1624440 tok/s
step 18585/19560 | loss 3.381183 (+2.13z)| norm 0.2393 (-0.60z)| lr 3.96e-06 | 322.52 ms | 52.3% bf16 MFU | 1624498 tok/s
step 18586/19560 | loss 3.254117 (-0.75z)| norm 0.2332 (-0.84z)| lr 3.95e-06 | 322.67 ms | 52.3% bf16 MFU | 1624516 tok/s
step 18587/19560 | loss 3.278580 (-0.19z)| norm 0.2428 (-0.44z)| lr 3.94e-06 | 322.86 ms | 52.3% bf16 MFU | 1624484 tok/s
step 18588/19560 | loss 3.318782 (+0.71z)| norm 0.2709 (+0.70z)| lr 3.93e-06 | 322.98 ms | 52.3% bf16 MFU | 1624424 tok/s
step 18589/19560 | loss 3.278939 (-0.19z)| norm 0.2617 (+0.31z)| lr 3.92e-06 | 322.52 ms | 52.3% bf16 MFU | 1624483 tok/s
step 18590/19560 | loss 3.257201 (-0.67z)| norm 0.2326 (-0.86z)| lr 3.92e-06 | 323.12 ms | 52.2% bf16 MFU | 1624388 tok/s
step 18591/19560 | loss 3.302212 (+0.35z)| norm 0.2314 (-0.90z)| lr 3.91e-06 | 322.66 ms | 52.3% bf16 MFU | 1624414 tok/s
step 18592/19560 | loss 3.322012 (+0.79z)| norm 0.2492 (-0.18z)| lr 3.90e-06 | 322.66 ms | 52.3% bf16 MFU | 1624437 tok/s
step 18593/19560 | loss 3.350048 (+1.41z)| norm 0.2716 (+0.74z)| lr 3.89e-06 | 322.75 ms | 52.3% bf16 MFU | 1624437 tok/s
step 18594/19560 | loss 3.258156 (-0.65z)| norm 0.2362 (-0.71z)| lr 3.88e-06 | 322.83 ms | 52.3% bf16 MFU | 1624416 tok/s
step 18595/19560 | loss 3.271502 (-0.35z)| norm 0.2386 (-0.62z)| lr 3.88e-06 | 322.21 ms | 52.4% bf16 MFU | 1624554 tok/s
step 18596/19560 | loss 3.298515 (+0.27z)| norm 0.2288 (-1.00z)| lr 3.87e-06 | 322.96 ms | 52.3% bf16 MFU | 1624495 tok/s
step 18597/19560 | loss 3.266744 (-0.44z)| norm 0.2516 (-0.07z)| lr 3.86e-06 | 322.50 ms | 52.3% bf16 MFU | 1624554 tok/s
step 18598/19560 | loss 3.242206 (-0.98z)| norm 0.2347 (-0.75z)| lr 3.85e-06 | 322.54 ms | 52.3% bf16 MFU | 1624601 tok/s
step 18599/19560 | loss 3.331973 (+1.03z)| norm 0.2412 (-0.47z)| lr 3.84e-06 | 323.08 ms | 52.2% bf16 MFU | 1624509 tok/s
step 18600/19560 | loss 3.253447 (-0.73z)| norm 0.2386 (-0.59z)| lr 3.84e-06 | 322.56 ms | 52.3% bf16 MFU | 1624553 tok/s
step 18601/19560 | loss 3.267745 (-0.43z)| norm 0.2848 (+1.32z)| lr 3.83e-06 | 322.42 ms | 52.3% bf16 MFU | 1624631 tok/s
step 18602/19560 | loss 3.320376 (+0.78z)| norm 0.2293 (-0.96z)| lr 3.82e-06 | 322.96 ms | 52.3% bf16 MFU | 1624569 tok/s
step 18603/19560 | loss 3.222794 (-1.46z)| norm 0.2549 (+0.11z)| lr 3.81e-06 | 322.79 ms | 52.3% bf16 MFU | 1624554 tok/s
step 18604/19560 | loss 3.267420 (-0.43z)| norm 0.2375 (-0.62z)| lr 3.80e-06 | 322.76 ms | 52.3% bf16 MFU | 1624546 tok/s
step 18605/19560 | loss 3.293388 (+0.16z)| norm 0.2403 (-0.50z)| lr 3.80e-06 | 322.45 ms | 52.3% bf16 MFU | 1624618 tok/s
step 18606/19560 | loss 3.276943 (-0.21z)| norm 0.2782 (+1.07z)| lr 3.79e-06 | 322.70 ms | 52.3% bf16 MFU | 1624622 tok/s
step 18607/19560 | loss 3.306937 (+0.47z)| norm 0.2418 (-0.43z)| lr 3.78e-06 | 323.05 ms | 52.2% bf16 MFU | 1624536 tok/s
step 18608/19560 | loss 3.320903 (+0.79z)| norm 0.2431 (-0.38z)| lr 3.77e-06 | 322.57 ms | 52.3% bf16 MFU | 1624576 tok/s
step 18609/19560 | loss 3.267303 (-0.44z)| norm 0.2337 (-0.77z)| lr 3.76e-06 | 322.83 ms | 52.3% bf16 MFU | 1624550 tok/s
step 18610/19560 | loss 3.243651 (-0.97z)| norm 0.2486 (-0.13z)| lr 3.76e-06 | 322.65 ms | 52.3% bf16 MFU | 1624570 tok/s
step 18611/19560 | loss 3.232377 (-1.22z)| norm 0.2377 (-0.59z)| lr 3.75e-06 | 322.49 ms | 52.3% bf16 MFU | 1624629 tok/s
step 18612/19560 | loss 3.427091 (+3.07z)| norm 0.3696 (+4.54z)| lr 3.74e-06 | 322.77 ms | 52.3% bf16 MFU | 1624616 tok/s
step 18613/19560 | loss 3.308976 (+0.52z)| norm 0.2593 (+0.26z)| lr 3.73e-06 | 322.85 ms | 52.3% bf16 MFU | 1624581 tok/s
step 18614/19560 | loss 3.317636 (+0.70z)| norm 0.2445 (-0.32z)| lr 3.72e-06 | 322.78 ms | 52.3% bf16 MFU | 1624566 tok/s
step 18615/19560 | loss 3.218951 (-1.51z)| norm 0.2354 (-0.66z)| lr 3.72e-06 | 322.57 ms | 52.3% bf16 MFU | 1624606 tok/s
step 18616/19560 | loss 3.268718 (-0.39z)| norm 0.2363 (-0.63z)| lr 3.71e-06 | 322.73 ms | 52.3% bf16 MFU | 1624604 tok/s
step 18617/19560 | loss 3.341189 (+1.22z)| norm 0.2657 (+0.51z)| lr 3.70e-06 | 323.22 ms | 52.2% bf16 MFU | 1624478 tok/s
step 18618/19560 | loss 3.225703 (-1.35z)| norm 0.2484 (-0.16z)| lr 3.69e-06 | 322.93 ms | 52.3% bf16 MFU | 1624431 tok/s
step 18619/19560 | loss 3.531546 (+4.88z)| norm 0.2650 (+0.48z)| lr 3.69e-06 | 322.43 ms | 52.3% bf16 MFU | 1624512 tok/s
step 18620/19560 | loss 3.268405 (-0.40z)| norm 0.3516 (+3.62z)| lr 3.68e-06 | 322.34 ms | 52.4% bf16 MFU | 1624612 tok/s
step 18621/19560 | loss 3.277984 (-0.20z)| norm 0.2315 (-0.80z)| lr 3.67e-06 | 323.08 ms | 52.2% bf16 MFU | 1624521 tok/s
step 18622/19560 | loss 3.279598 (-0.18z)| norm 0.2493 (-0.15z)| lr 3.66e-06 | 322.93 ms | 52.3% bf16 MFU | 1624472 tok/s
step 18623/19560 | loss 3.285840 (-0.05z)| norm 0.2889 (+1.30z)| lr 3.65e-06 | 322.90 ms | 52.3% bf16 MFU | 1624433 tok/s
step 18624/19560 | loss 3.252923 (-0.71z)| norm 0.2414 (-0.43z)| lr 3.65e-06 | 322.46 ms | 52.3% bf16 MFU | 1624507 tok/s
step 18625/19560 | loss 3.331295 (+0.87z)| norm 0.2423 (-0.41z)| lr 3.64e-06 | 322.64 ms | 52.3% bf16 MFU | 1624532 tok/s
step 18626/19560 | loss 3.304424 (+0.33z)| norm 0.2624 (+0.37z)| lr 3.63e-06 | 322.41 ms | 52.3% bf16 MFU | 1624614 tok/s
step 18627/19560 | loss 3.305388 (+0.34z)| norm 0.2519 (-0.04z)| lr 3.62e-06 | 322.90 ms | 52.3% bf16 MFU | 1624566 tok/s
step 18628/19560 | loss 3.435011 (+2.84z)| norm 0.2845 (+1.20z)| lr 3.62e-06 | 322.77 ms | 52.3% bf16 MFU | 1624555 tok/s
step 18629/19560 | loss 3.221390 (-1.32z)| norm 0.2536 (+0.01z)| lr 3.61e-06 | 322.78 ms | 52.3% bf16 MFU | 1624542 tok/s
step 18630/19560 | loss 3.253282 (-0.70z)| norm 0.2439 (-0.36z)| lr 3.60e-06 | 322.60 ms | 52.3% bf16 MFU | 1624575 tok/s
step 18631/19560 | loss 3.324502 (+0.68z)| norm 0.2422 (-0.43z)| lr 3.59e-06 | 322.18 ms | 52.4% bf16 MFU | 1624712 tok/s
step 18632/19560 | loss 3.285215 (-0.09z)| norm 0.2360 (-0.65z)| lr 3.58e-06 | 322.91 ms | 52.3% bf16 MFU | 1624658 tok/s
step 18633/19560 | loss 3.293650 (+0.07z)| norm 0.2449 (-0.32z)| lr 3.58e-06 | 322.77 ms | 52.3% bf16 MFU | 1624642 tok/s
step 18634/19560 | loss 3.279930 (-0.20z)| norm 0.2428 (-0.39z)| lr 3.57e-06 | 323.42 ms | 52.2% bf16 MFU | 1624464 tok/s
step 18635/19560 | loss 3.205989 (-1.61z)| norm 0.2343 (-0.72z)| lr 3.56e-06 | 322.79 ms | 52.3% bf16 MFU | 1624454 tok/s
step 18636/19560 | loss 3.311542 (+0.42z)| norm 0.2336 (-0.74z)| lr 3.55e-06 | 322.48 ms | 52.3% bf16 MFU | 1624522 tok/s
step 18637/19560 | loss 3.286820 (-0.07z)| norm 0.2437 (-0.36z)| lr 3.55e-06 | 322.73 ms | 52.3% bf16 MFU | 1624524 tok/s
step 18638/19560 | loss 3.381895 (+1.93z)| norm 0.2688 (+0.65z)| lr 3.54e-06 | 322.76 ms | 52.3% bf16 MFU | 1624516 tok/s
step 18639/19560 | loss 3.299620 (+0.21z)| norm 0.2334 (-0.76z)| lr 3.53e-06 | 323.08 ms | 52.2% bf16 MFU | 1624430 tok/s
step 18640/19560 | loss 3.305620 (+0.33z)| norm 0.2499 (-0.09z)| lr 3.52e-06 | 322.38 ms | 52.4% bf16 MFU | 1624524 tok/s
step 18641/19560 | loss 3.227004 (-1.30z)| norm 0.2459 (-0.25z)| lr 3.52e-06 | 322.84 ms | 52.3% bf16 MFU | 1624497 tok/s
step 18642/19560 | loss 3.308572 (+0.40z)| norm 0.2608 (+0.34z)| lr 3.51e-06 | 323.02 ms | 52.2% bf16 MFU | 1624426 tok/s
step 18643/19560 | loss 3.240538 (-1.00z)| norm 0.2202 (-1.27z)| lr 3.50e-06 | 322.54 ms | 52.3% bf16 MFU | 1624479 tok/s
step 18644/19560 | loss 3.353965 (+1.33z)| norm 0.2360 (-0.64z)| lr 3.49e-06 | 322.97 ms | 52.3% bf16 MFU | 1624422 tok/s
step 18645/19560 | loss 3.291396 (+0.04z)| norm 0.2492 (-0.11z)| lr 3.49e-06 | 322.71 ms | 52.3% bf16 MFU | 1624433 tok/s
step 18646/19560 | loss 3.194317 (-1.95z)| norm 0.2306 (-0.84z)| lr 3.48e-06 | 322.85 ms | 52.3% bf16 MFU | 1624408 tok/s
step 18647/19560 | loss 3.316998 (+0.60z)| norm 0.2424 (-0.37z)| lr 3.47e-06 | 322.71 ms | 52.3% bf16 MFU | 1624418 tok/s
step 18648/19560 | loss 3.304633 (+0.34z)| norm 0.2295 (-0.88z)| lr 3.46e-06 | 322.66 ms | 52.3% bf16 MFU | 1624442 tok/s
step 18649/19560 | loss 3.227917 (-1.25z)| norm 0.2446 (-0.27z)| lr 3.46e-06 | 322.70 ms | 52.3% bf16 MFU | 1624455 tok/s
step 18650/19560 | loss 3.270401 (-0.36z)| norm 0.2493 (-0.07z)| lr 3.45e-06 | 323.03 ms | 52.2% bf16 MFU | 1624385 tok/s
step 18651/19560 | loss 3.297513 (+0.19z)| norm 0.2470 (-0.16z)| lr 3.44e-06 | 322.98 ms | 52.3% bf16 MFU | 1624330 tok/s
step 18652/19560 | loss 3.378664 (+1.86z)| norm 0.2881 (+1.47z)| lr 3.43e-06 | 322.63 ms | 52.3% bf16 MFU | 1624366 tok/s
step 18653/19560 | loss 3.305844 (+0.35z)| norm 0.2279 (-0.93z)| lr 3.42e-06 | 323.10 ms | 52.2% bf16 MFU | 1624281 tok/s
step 18654/19560 | loss 3.446186 (+3.09z)| norm 0.3001 (+1.91z)| lr 3.42e-06 | 322.37 ms | 52.4% bf16 MFU | 1624385 tok/s
step 18655/19560 | loss 3.215541 (-1.45z)| norm 0.2262 (-0.99z)| lr 3.41e-06 | 323.22 ms | 52.2% bf16 MFU | 1624270 tok/s
step 18656/19560 | loss 3.214519 (-1.45z)| norm 0.2528 (+0.06z)| lr 3.40e-06 | 323.00 ms | 52.3% bf16 MFU | 1624217 tok/s
step 18657/19560 | loss 3.248856 (-0.78z)| norm 0.3062 (+2.10z)| lr 3.39e-06 | 323.01 ms | 52.2% bf16 MFU | 1624161 tok/s
step 18658/19560 | loss 3.417179 (+2.42z)| norm 0.2426 (-0.37z)| lr 3.39e-06 | 322.84 ms | 52.3% bf16 MFU | 1624152 tok/s
step 18659/19560 | loss 3.234418 (-1.04z)| norm 0.2319 (-0.78z)| lr 3.38e-06 | 322.89 ms | 52.3% bf16 MFU | 1624131 tok/s
step 18660/19560 | loss 3.247744 (-0.77z)| norm 0.2453 (-0.26z)| lr 3.37e-06 | 322.70 ms | 52.3% bf16 MFU | 1624158 tok/s
step 18661/19560 | loss 3.255001 (-0.64z)| norm 0.2601 (+0.32z)| lr 3.36e-06 | 322.53 ms | 52.3% bf16 MFU | 1624229 tok/s
step 18662/19560 | loss 3.276276 (-0.24z)| norm 0.2591 (+0.29z)| lr 3.36e-06 | 323.40 ms | 52.2% bf16 MFU | 1624076 tok/s
step 18663/19560 | loss 3.288877 (-0.01z)| norm 0.2196 (-1.24z)| lr 3.35e-06 | 322.48 ms | 52.3% bf16 MFU | 1624161 tok/s
step 18664/19560 | loss 3.272985 (-0.31z)| norm 0.2582 (+0.29z)| lr 3.34e-06 | 322.91 ms | 52.3% bf16 MFU | 1624134 tok/s
step 18665/19560 | loss 3.227859 (-1.15z)| norm 0.2629 (+0.56z)| lr 3.34e-06 | 322.60 ms | 52.3% bf16 MFU | 1624187 tok/s
step 18666/19560 | loss 3.234786 (-1.02z)| norm 0.2344 (-0.69z)| lr 3.33e-06 | 322.57 ms | 52.3% bf16 MFU | 1624244 tok/s
step 18667/19560 | loss 3.266875 (-0.40z)| norm 0.2650 (+0.64z)| lr 3.32e-06 | 323.01 ms | 52.2% bf16 MFU | 1624189 tok/s
step 18668/19560 | loss 3.231194 (-1.09z)| norm 0.2849 (+1.49z)| lr 3.31e-06 | 322.93 ms | 52.3% bf16 MFU | 1624155 tok/s
step 18669/19560 | loss 3.250777 (-0.71z)| norm 0.2217 (-1.26z)| lr 3.31e-06 | 322.79 ms | 52.3% bf16 MFU | 1624159 tok/s
step 18670/19560 | loss 3.329657 (+0.80z)| norm 0.2253 (-1.09z)| lr 3.30e-06 | 323.04 ms | 52.2% bf16 MFU | 1624100 tok/s
step 18671/19560 | loss 3.364568 (+1.44z)| norm 0.2323 (-0.77z)| lr 3.29e-06 | 322.43 ms | 52.3% bf16 MFU | 1624197 tok/s
step 18672/19560 | loss 3.289660 (+0.03z)| norm 0.2470 (-0.12z)| lr 3.28e-06 | 322.79 ms | 52.3% bf16 MFU | 1624200 tok/s
step 18673/19560 | loss 3.310361 (+0.43z)| norm 0.2372 (-0.56z)| lr 3.28e-06 | 322.97 ms | 52.3% bf16 MFU | 1624157 tok/s
step 18674/19560 | loss 3.236815 (-0.97z)| norm 0.3203 (+2.96z)| lr 3.27e-06 | 323.09 ms | 52.2% bf16 MFU | 1624086 tok/s
step 18675/19560 | loss 3.289266 (+0.03z)| norm 0.2423 (-0.35z)| lr 3.26e-06 | 322.50 ms | 52.3% bf16 MFU | 1624167 tok/s
step 18676/19560 | loss 3.363450 (+1.43z)| norm 0.2668 (+0.69z)| lr 3.25e-06 | 322.99 ms | 52.3% bf16 MFU | 1624120 tok/s
step 18677/19560 | loss 3.230859 (-1.07z)| norm 0.2350 (-0.65z)| lr 3.25e-06 | 322.87 ms | 52.3% bf16 MFU | 1624107 tok/s
step 18678/19560 | loss 3.233205 (-1.01z)| norm 0.2401 (-0.44z)| lr 3.24e-06 | 322.77 ms | 52.3% bf16 MFU | 1624119 tok/s
step 18679/19560 | loss 3.281142 (-0.10z)| norm 0.2317 (-0.79z)| lr 3.23e-06 | 322.58 ms | 52.3% bf16 MFU | 1624178 tok/s
step 18680/19560 | loss 3.239777 (-0.90z)| norm 0.2743 (+1.01z)| lr 3.22e-06 | 322.91 ms | 52.3% bf16 MFU | 1624150 tok/s
step 18681/19560 | loss 3.260943 (-0.49z)| norm 0.2239 (-1.13z)| lr 3.22e-06 | 322.95 ms | 52.3% bf16 MFU | 1624114 tok/s
step 18682/19560 | loss 3.432793 (+2.70z)| norm 0.2642 (+0.58z)| lr 3.21e-06 | 322.91 ms | 52.3% bf16 MFU | 1624090 tok/s
step 18683/19560 | loss 3.375807 (+1.61z)| norm 0.2435 (-0.30z)| lr 3.20e-06 | 323.32 ms | 52.2% bf16 MFU | 1623965 tok/s
step 18684/19560 | loss 3.269985 (-0.34z)| norm 0.2363 (-0.60z)| lr 3.20e-06 | 323.00 ms | 52.3% bf16 MFU | 1623925 tok/s
step 18685/19560 | loss 3.260004 (-0.52z)| norm 0.2572 (+0.28z)| lr 3.19e-06 | 322.37 ms | 52.4% bf16 MFU | 1624046 tok/s
step 18686/19560 | loss 3.297478 (+0.17z)| norm 0.2490 (-0.05z)| lr 3.18e-06 | 323.02 ms | 52.2% bf16 MFU | 1623998 tok/s
step 18687/19560 | loss 3.333886 (+0.84z)| norm 0.3263 (+3.19z)| lr 3.17e-06 | 324.72 ms | 52.0% bf16 MFU | 1623528 tok/s
step 18688/19560 | loss 3.316261 (+0.52z)| norm 0.2322 (-0.77z)| lr 3.17e-06 | 322.88 ms | 52.3% bf16 MFU | 1623541 tok/s
step 18689/19560 | loss 3.261399 (-0.49z)| norm 0.2466 (-0.16z)| lr 3.16e-06 | 322.42 ms | 52.3% bf16 MFU | 1623669 tok/s
step 18690/19560 | loss 3.272787 (-0.28z)| norm 0.2540 (+0.16z)| lr 3.15e-06 | 322.64 ms | 52.3% bf16 MFU | 1623735 tok/s
step 18691/19560 | loss 3.327722 (+0.73z)| norm 0.2655 (+0.65z)| lr 3.14e-06 | 322.67 ms | 52.3% bf16 MFU | 1623791 tok/s
step 18692/19560 | loss 3.249255 (-0.71z)| norm 0.2207 (-1.24z)| lr 3.14e-06 | 322.75 ms | 52.3% bf16 MFU | 1623824 tok/s
step 18693/19560 | loss 3.306618 (+0.34z)| norm 0.2363 (-0.58z)| lr 3.13e-06 | 322.91 ms | 52.3% bf16 MFU | 1623814 tok/s
step 18694/19560 | loss 3.292875 (+0.08z)| norm 0.2261 (-1.00z)| lr 3.12e-06 | 322.58 ms | 52.3% bf16 MFU | 1623889 tok/s
step 18695/19560 | loss 3.276554 (-0.22z)| norm 0.2672 (+0.72z)| lr 3.12e-06 | 322.72 ms | 52.3% bf16 MFU | 1623924 tok/s
step 18696/19560 | loss 3.293103 (+0.09z)| norm 0.2451 (-0.21z)| lr 3.11e-06 | 323.07 ms | 52.2% bf16 MFU | 1623870 tok/s
step 18697/19560 | loss 3.295676 (+0.13z)| norm 0.2422 (-0.33z)| lr 3.10e-06 | 322.95 ms | 52.3% bf16 MFU | 1623850 tok/s
step 18698/19560 | loss 3.346864 (+1.06z)| norm 0.2394 (-0.46z)| lr 3.09e-06 | 322.69 ms | 52.3% bf16 MFU | 1623895 tok/s
step 18699/19560 | loss 3.339539 (+0.91z)| norm 0.2356 (-0.62z)| lr 3.09e-06 | 322.93 ms | 52.3% bf16 MFU | 1623878 tok/s
step 18700/19560 | loss 3.129602 (-2.86z)| norm 0.2629 (+0.53z)| lr 3.08e-06 | 322.51 ms | 52.3% bf16 MFU | 1623966 tok/s
step 18701/19560 | loss 3.267581 (-0.37z)| norm 0.2386 (-0.49z)| lr 3.07e-06 | 322.70 ms | 52.3% bf16 MFU | 1624002 tok/s
step 18702/19560 | loss 3.304049 (+0.27z)| norm 0.2269 (-0.99z)| lr 3.07e-06 | 322.86 ms | 52.3% bf16 MFU | 1623995 tok/s
step 18703/19560 | loss 3.298196 (+0.16z)| norm 0.2782 (+1.17z)| lr 3.06e-06 | 323.01 ms | 52.2% bf16 MFU | 1623952 tok/s
step 18704/19560 | loss 3.321163 (+0.56z)| norm 0.2855 (+1.45z)| lr 3.05e-06 | 323.43 ms | 52.2% bf16 MFU | 1623806 tok/s
step 18705/19560 | loss 3.321846 (+0.57z)| norm 0.2856 (+1.43z)| lr 3.04e-06 | 322.76 ms | 52.3% bf16 MFU | 1623836 tok/s
step 18706/19560 | loss 3.246193 (-0.79z)| norm 0.2724 (+0.87z)| lr 3.04e-06 | 322.88 ms | 52.3% bf16 MFU | 1623833 tok/s
step 18707/19560 | loss 3.352704 (+1.12z)| norm 0.2604 (+0.37z)| lr 3.03e-06 | 322.59 ms | 52.3% bf16 MFU | 1623902 tok/s
step 18708/19560 | loss 3.254686 (-0.65z)| norm 0.2500 (-0.05z)| lr 3.02e-06 | 322.86 ms | 52.3% bf16 MFU | 1623901 tok/s
step 18709/19560 | loss 3.263455 (-0.49z)| norm 0.2546 (+0.13z)| lr 3.02e-06 | 323.13 ms | 52.2% bf16 MFU | 1623834 tok/s
step 18710/19560 | loss 3.250041 (-0.72z)| norm 0.2609 (+0.40z)| lr 3.01e-06 | 322.62 ms | 52.3% bf16 MFU | 1623897 tok/s
step 18711/19560 | loss 3.302090 (+0.22z)| norm 0.2435 (-0.32z)| lr 3.00e-06 | 322.42 ms | 52.3% bf16 MFU | 1624006 tok/s
step 18712/19560 | loss 3.310364 (+0.36z)| norm 0.2668 (+0.64z)| lr 3.00e-06 | 323.11 ms | 52.2% bf16 MFU | 1623936 tok/s
step 18713/19560 | loss 3.296244 (+0.12z)| norm 0.2387 (-0.53z)| lr 2.99e-06 | 323.09 ms | 52.2% bf16 MFU | 1623875 tok/s
step 18714/19560 | loss 3.274742 (-0.28z)| norm 0.2573 (+0.24z)| lr 2.98e-06 | 322.94 ms | 52.3% bf16 MFU | 1623856 tok/s
step 18715/19560 | loss 3.323050 (+0.60z)| norm 0.2364 (-0.63z)| lr 2.97e-06 | 322.49 ms | 52.3% bf16 MFU | 1623952 tok/s
step 18716/19560 | loss 3.291443 (+0.03z)| norm 0.2554 (+0.17z)| lr 2.97e-06 | 322.82 ms | 52.3% bf16 MFU | 1623960 tok/s
step 18717/19560 | loss 3.254056 (-0.65z)| norm 0.2372 (-0.58z)| lr 2.96e-06 | 322.61 ms | 52.3% bf16 MFU | 1624020 tok/s
step 18718/19560 | loss 3.290553 (+0.01z)| norm 0.2351 (-0.67z)| lr 2.95e-06 | 323.33 ms | 52.2% bf16 MFU | 1623894 tok/s
step 18719/19560 | loss 3.269330 (-0.37z)| norm 0.2392 (-0.51z)| lr 2.95e-06 | 322.89 ms | 52.3% bf16 MFU | 1623885 tok/s
step 18720/19560 | loss 3.252903 (-0.67z)| norm 0.2507 (-0.02z)| lr 2.94e-06 | 322.63 ms | 52.3% bf16 MFU | 1623944 tok/s
step 18721/19560 | loss 3.286333 (-0.04z)| norm 0.2428 (-0.35z)| lr 2.93e-06 | 322.91 ms | 52.3% bf16 MFU | 1623928 tok/s
step 18722/19560 | loss 3.335832 (+0.85z)| norm 0.2368 (-0.60z)| lr 2.92e-06 | 323.12 ms | 52.2% bf16 MFU | 1623860 tok/s
step 18723/19560 | loss 3.241826 (-0.87z)| norm 0.2309 (-0.84z)| lr 2.92e-06 | 323.97 ms | 52.1% bf16 MFU | 1623582 tok/s
step 18724/19560 | loss 3.256489 (-0.59z)| norm 0.2798 (+1.18z)| lr 2.91e-06 | 323.12 ms | 52.2% bf16 MFU | 1623532 tok/s
step 18725/19560 | loss 3.323108 (+0.62z)| norm 0.2378 (-0.56z)| lr 2.90e-06 | 322.60 ms | 52.3% bf16 MFU | 1623615 tok/s
step 18726/19560 | loss 3.380808 (+1.64z)| norm 0.2520 (+0.02z)| lr 2.90e-06 | 322.60 ms | 52.3% bf16 MFU | 1623693 tok/s
step 18727/19560 | loss 3.302096 (+0.22z)| norm 0.2302 (-0.88z)| lr 2.89e-06 | 323.13 ms | 52.2% bf16 MFU | 1623636 tok/s
step 18728/19560 | loss 3.250896 (-0.71z)| norm 0.2400 (-0.47z)| lr 2.88e-06 | 322.91 ms | 52.3% bf16 MFU | 1623636 tok/s
step 18729/19560 | loss 3.333215 (+0.77z)| norm 0.2306 (-0.85z)| lr 2.88e-06 | 323.07 ms | 52.2% bf16 MFU | 1623595 tok/s
step 18730/19560 | loss 3.267298 (-0.42z)| norm 0.2287 (-0.93z)| lr 2.87e-06 | 322.88 ms | 52.3% bf16 MFU | 1623604 tok/s
step 18731/19560 | loss 3.234161 (-1.02z)| norm 0.2534 (+0.10z)| lr 2.86e-06 | 322.64 ms | 52.3% bf16 MFU | 1623673 tok/s
step 18732/19560 | loss 3.359853 (+1.25z)| norm 0.2493 (-0.07z)| lr 2.86e-06 | 323.08 ms | 52.2% bf16 MFU | 1623628 tok/s
step 18733/19560 | loss 3.334650 (+0.78z)| norm 0.2312 (-0.83z)| lr 2.85e-06 | 322.55 ms | 52.3% bf16 MFU | 1623720 tok/s
step 18734/19560 | loss 3.231149 (-1.07z)| norm 0.2453 (-0.23z)| lr 2.84e-06 | 322.89 ms | 52.3% bf16 MFU | 1623720 tok/s
step 18735/19560 | loss 3.311908 (+0.38z)| norm 0.2397 (-0.46z)| lr 2.84e-06 | 323.13 ms | 52.2% bf16 MFU | 1623660 tok/s
step 18736/19560 | loss 3.248470 (-0.75z)| norm 0.2304 (-0.85z)| lr 2.83e-06 | 323.27 ms | 52.2% bf16 MFU | 1623569 tok/s
step 18737/19560 | loss 3.251527 (-0.69z)| norm 0.2412 (-0.40z)| lr 2.82e-06 | 323.34 ms | 52.2% bf16 MFU | 1623466 tok/s
step 18738/19560 | loss 3.254503 (-0.64z)| norm 0.2431 (-0.32z)| lr 2.81e-06 | 323.42 ms | 52.2% bf16 MFU | 1623346 tok/s
step 18739/19560 | loss 3.288523 (-0.04z)| norm 0.2575 (+0.28z)| lr 2.81e-06 | 323.41 ms | 52.2% bf16 MFU | 1623236 tok/s
step 18740/19560 | loss 3.310740 (+0.39z)| norm 0.2661 (+0.76z)| lr 2.80e-06 | 323.09 ms | 52.2% bf16 MFU | 1623211 tok/s
step 18741/19560 | loss 3.276782 (-0.24z)| norm 0.2515 (+0.08z)| lr 2.79e-06 | 323.22 ms | 52.2% bf16 MFU | 1623154 tok/s
step 18742/19560 | loss 3.295381 (+0.11z)| norm 0.2528 (+0.13z)| lr 2.79e-06 | 322.98 ms | 52.3% bf16 MFU | 1623160 tok/s
step 18743/19560 | loss 3.334826 (+0.83z)| norm 0.2400 (-0.47z)| lr 2.78e-06 | 323.30 ms | 52.2% bf16 MFU | 1623086 tok/s
step 18744/19560 | loss 3.373391 (+1.51z)| norm 0.2406 (-0.44z)| lr 2.77e-06 | 322.70 ms | 52.3% bf16 MFU | 1623166 tok/s
step 18745/19560 | loss 3.294518 (+0.07z)| norm 0.2650 (+0.70z)| lr 2.77e-06 | 322.83 ms | 52.3% bf16 MFU | 1623210 tok/s
step 18746/19560 | loss 3.261997 (-0.54z)| norm 0.2714 (+0.99z)| lr 2.76e-06 | 322.78 ms | 52.3% bf16 MFU | 1623263 tok/s
step 18747/19560 | loss 3.218379 (-1.41z)| norm 0.2356 (-0.67z)| lr 2.75e-06 | 322.99 ms | 52.3% bf16 MFU | 1623261 tok/s
step 18748/19560 | loss 3.289619 (+0.02z)| norm 0.2584 (+0.47z)| lr 2.75e-06 | 323.12 ms | 52.2% bf16 MFU | 1623226 tok/s
step 18749/19560 | loss 3.256525 (-0.64z)| norm 0.2481 (-0.07z)| lr 2.74e-06 | 322.87 ms | 52.3% bf16 MFU | 1623256 tok/s
step 18750/19560 | loss 3.275263 (-0.27z)| norm 0.2332 (-0.83z)| lr 2.73e-06 | 322.40 ms | 52.3% bf16 MFU | 1623402 tok/s
val loss 3.274832
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3039/10042 = 0.302629
step 18751/19560 | loss 3.300226 (+0.23z)| norm 0.2537 (+0.25z)| lr 2.73e-06 | 324.21 ms | 52.1% bf16 MFU | 1623089 tok/s
step 18752/19560 | loss 3.293430 (+0.09z)| norm 0.2479 (-0.06z)| lr 2.72e-06 | 322.42 ms | 52.3% bf16 MFU | 1623240 tok/s
step 18753/19560 | loss 3.278530 (-0.20z)| norm 0.2242 (-1.29z)| lr 2.71e-06 | 323.99 ms | 52.1% bf16 MFU | 1622989 tok/s
step 18754/19560 | loss 3.352822 (+1.28z)| norm 0.2311 (-0.91z)| lr 2.71e-06 | 322.14 ms | 52.4% bf16 MFU | 1623214 tok/s
step 18755/19560 | loss 3.293490 (+0.09z)| norm 0.2326 (-0.82z)| lr 2.70e-06 | 322.36 ms | 52.4% bf16 MFU | 1623373 tok/s
step 18756/19560 | loss 3.323222 (+0.73z)| norm 0.2390 (-0.48z)| lr 2.69e-06 | 323.22 ms | 52.2% bf16 MFU | 1623308 tok/s
step 18757/19560 | loss 3.241441 (-0.97z)| norm 0.2466 (-0.08z)| lr 2.69e-06 | 322.32 ms | 52.4% bf16 MFU | 1623473 tok/s
step 18758/19560 | loss 3.276377 (-0.25z)| norm 0.2392 (-0.46z)| lr 2.68e-06 | 322.35 ms | 52.4% bf16 MFU | 1623622 tok/s
step 18759/19560 | loss 3.292942 (+0.10z)| norm 0.2225 (-1.32z)| lr 2.67e-06 | 322.84 ms | 52.3% bf16 MFU | 1623640 tok/s
step 18760/19560 | loss 3.276381 (-0.24z)| norm 0.2292 (-0.97z)| lr 2.67e-06 | 322.96 ms | 52.3% bf16 MFU | 1623628 tok/s
step 18761/19560 | loss 3.310692 (+0.47z)| norm 0.2361 (-0.61z)| lr 2.66e-06 | 322.67 ms | 52.3% bf16 MFU | 1623689 tok/s
step 18762/19560 | loss 3.305079 (+0.35z)| norm 0.2702 (+1.15z)| lr 2.65e-06 | 322.78 ms | 52.3% bf16 MFU | 1623718 tok/s
step 18763/19560 | loss 3.253973 (-0.73z)| norm 0.2875 (+1.99z)| lr 2.65e-06 | 322.93 ms | 52.3% bf16 MFU | 1623708 tok/s
step 18764/19560 | loss 3.243468 (-0.94z)| norm 0.2579 (+0.48z)| lr 2.64e-06 | 322.26 ms | 52.4% bf16 MFU | 1623869 tok/s
step 18765/19560 | loss 3.331790 (+0.90z)| norm 0.2508 (+0.11z)| lr 2.63e-06 | 323.15 ms | 52.2% bf16 MFU | 1623797 tok/s
step 18766/19560 | loss 3.295171 (+0.15z)| norm 0.2364 (-0.61z)| lr 2.63e-06 | 322.08 ms | 52.4% bf16 MFU | 1623999 tok/s
step 18767/19560 | loss 3.296134 (+0.17z)| norm 0.2379 (-0.54z)| lr 2.62e-06 | 322.06 ms | 52.4% bf16 MFU | 1624195 tok/s
step 18768/19560 | loss 3.355132 (+1.41z)| norm 0.2536 (+0.26z)| lr 2.61e-06 | 322.51 ms | 52.3% bf16 MFU | 1624268 tok/s
step 18769/19560 | loss 3.272763 (-0.34z)| norm 0.3089 (+2.97z)| lr 2.61e-06 | 322.45 ms | 52.3% bf16 MFU | 1624351 tok/s
step 18770/19560 | loss 3.312985 (+0.51z)| norm 0.2647 (+0.78z)| lr 2.60e-06 | 323.25 ms | 52.2% bf16 MFU | 1624229 tok/s
step 18771/19560 | loss 3.325457 (+0.77z)| norm 0.2431 (-0.30z)| lr 2.59e-06 | 323.29 ms | 52.2% bf16 MFU | 1624104 tok/s
step 18772/19560 | loss 3.304826 (+0.34z)| norm 0.2351 (-0.70z)| lr 2.59e-06 | 322.62 ms | 52.3% bf16 MFU | 1624153 tok/s
step 18773/19560 | loss 3.278815 (-0.21z)| norm 0.2340 (-0.75z)| lr 2.58e-06 | 321.84 ms | 52.4% bf16 MFU | 1624397 tok/s
step 18774/19560 | loss 3.330766 (+0.88z)| norm 0.2394 (-0.48z)| lr 2.57e-06 | 322.90 ms | 52.3% bf16 MFU | 1624361 tok/s
step 18775/19560 | loss 3.251463 (-0.82z)| norm 0.2287 (-1.01z)| lr 2.57e-06 | 322.51 ms | 52.3% bf16 MFU | 1624424 tok/s
step 18776/19560 | loss 3.233802 (-1.18z)| norm 0.2666 (+0.86z)| lr 2.56e-06 | 322.26 ms | 52.4% bf16 MFU | 1624549 tok/s
step 18777/19560 | loss 3.312859 (+0.50z)| norm 0.2318 (-0.86z)| lr 2.55e-06 | 322.84 ms | 52.3% bf16 MFU | 1624520 tok/s
step 18778/19560 | loss 3.324988 (+0.76z)| norm 0.2341 (-0.74z)| lr 2.55e-06 | 323.06 ms | 52.2% bf16 MFU | 1624438 tok/s
step 18779/19560 | loss 3.270796 (-0.41z)| norm 0.2451 (-0.20z)| lr 2.54e-06 | 322.53 ms | 52.3% bf16 MFU | 1624493 tok/s
step 18780/19560 | loss 3.327932 (+0.84z)| norm 0.2737 (+1.23z)| lr 2.54e-06 | 322.27 ms | 52.4% bf16 MFU | 1624610 tok/s
step 18781/19560 | loss 3.328646 (+0.85z)| norm 0.2659 (+0.83z)| lr 2.53e-06 | 322.39 ms | 52.4% bf16 MFU | 1624693 tok/s
step 18782/19560 | loss 3.266966 (-0.48z)| norm 0.2279 (-1.06z)| lr 2.52e-06 | 323.26 ms | 52.2% bf16 MFU | 1624553 tok/s
step 18783/19560 | loss 3.248856 (-0.91z)| norm 0.2327 (-0.82z)| lr 2.52e-06 | 322.74 ms | 52.3% bf16 MFU | 1624551 tok/s
step 18784/19560 | loss 3.326703 (+0.87z)| norm 0.2353 (-0.68z)| lr 2.51e-06 | 322.42 ms | 52.3% bf16 MFU | 1624629 tok/s
step 18785/19560 | loss 3.235108 (-1.25z)| norm 0.2244 (-1.24z)| lr 2.50e-06 | 322.57 ms | 52.3% bf16 MFU | 1624666 tok/s
step 18786/19560 | loss 3.287960 (-0.00z)| norm 0.2703 (+1.16z)| lr 2.50e-06 | 323.04 ms | 52.2% bf16 MFU | 1624582 tok/s
step 18787/19560 | loss 3.269600 (-0.45z)| norm 0.2367 (-0.60z)| lr 2.49e-06 | 322.53 ms | 52.3% bf16 MFU | 1624629 tok/s
step 18788/19560 | loss 3.341702 (+1.27z)| norm 0.2632 (+0.78z)| lr 2.48e-06 | 323.30 ms | 52.2% bf16 MFU | 1624481 tok/s
step 18789/19560 | loss 3.293993 (+0.11z)| norm 0.3066 (+2.93z)| lr 2.48e-06 | 322.55 ms | 52.3% bf16 MFU | 1624528 tok/s
step 18790/19560 | loss 3.295661 (+0.15z)| norm 0.3427 (+4.35z)| lr 2.47e-06 | 322.20 ms | 52.4% bf16 MFU | 1624663 tok/s
step 18791/19560 | loss 3.326539 (+0.88z)| norm 0.2398 (-0.46z)| lr 2.46e-06 | 322.71 ms | 52.3% bf16 MFU | 1624662 tok/s
step 18792/19560 | loss 3.339535 (+1.18z)| norm 0.2402 (-0.43z)| lr 2.46e-06 | 322.17 ms | 52.4% bf16 MFU | 1624797 tok/s
step 18793/19560 | loss 3.264181 (-0.64z)| norm 0.2324 (-0.79z)| lr 2.45e-06 | 322.79 ms | 52.3% bf16 MFU | 1624768 tok/s
step 18794/19560 | loss 3.256818 (-0.83z)| norm 0.2312 (-0.84z)| lr 2.45e-06 | 322.77 ms | 52.3% bf16 MFU | 1624746 tok/s
step 18795/19560 | loss 3.343364 (+1.25z)| norm 0.2871 (+1.76z)| lr 2.44e-06 | 322.19 ms | 52.4% bf16 MFU | 1624871 tok/s
step 18796/19560 | loss 3.298754 (+0.16z)| norm 0.2890 (+1.84z)| lr 2.43e-06 | 322.62 ms | 52.3% bf16 MFU | 1624883 tok/s
step 18797/19560 | loss 3.292513 (+0.00z)| norm 0.2379 (-0.54z)| lr 2.43e-06 | 322.45 ms | 52.3% bf16 MFU | 1624938 tok/s
step 18798/19560 | loss 3.299269 (+0.18z)| norm 0.2364 (-0.61z)| lr 2.42e-06 | 322.07 ms | 52.4% bf16 MFU | 1625085 tok/s
step 18799/19560 | loss 3.250381 (-1.01z)| norm 0.2478 (-0.09z)| lr 2.41e-06 | 322.82 ms | 52.3% bf16 MFU | 1625034 tok/s
step 18800/19560 | loss 3.489635 (+4.47z)| norm 0.2528 (+0.14z)| lr 2.41e-06 | 323.11 ms | 52.2% bf16 MFU | 1624914 tok/s
step 18801/19560 | loss 3.283621 (-0.20z)| norm 0.2675 (+0.82z)| lr 2.40e-06 | 322.49 ms | 52.3% bf16 MFU | 1624956 tok/s
step 18802/19560 | loss 3.286372 (-0.15z)| norm 0.2451 (-0.21z)| lr 2.39e-06 | 322.80 ms | 52.3% bf16 MFU | 1624917 tok/s
step 18803/19560 | loss 3.297237 (+0.10z)| norm 0.2436 (-0.28z)| lr 2.39e-06 | 322.56 ms | 52.3% bf16 MFU | 1624940 tok/s
step 18804/19560 | loss 3.268119 (-0.56z)| norm 0.2648 (+0.76z)| lr 2.38e-06 | 322.26 ms | 52.4% bf16 MFU | 1625039 tok/s
step 18805/19560 | loss 3.271845 (-0.48z)| norm 0.2598 (+0.50z)| lr 2.38e-06 | 322.56 ms | 52.3% bf16 MFU | 1625058 tok/s
step 18806/19560 | loss 3.304065 (+0.26z)| norm 0.2334 (-0.79z)| lr 2.37e-06 | 322.50 ms | 52.3% bf16 MFU | 1625091 tok/s
step 18807/19560 | loss 3.285176 (-0.19z)| norm 0.2319 (-0.86z)| lr 2.36e-06 | 323.18 ms | 52.2% bf16 MFU | 1624951 tok/s
step 18808/19560 | loss 3.341722 (+1.12z)| norm 0.2916 (+2.03z)| lr 2.36e-06 | 322.19 ms | 52.4% bf16 MFU | 1625066 tok/s
step 18809/19560 | loss 3.378070 (+1.92z)| norm 0.2369 (-0.62z)| lr 2.35e-06 | 322.79 ms | 52.3% bf16 MFU | 1625024 tok/s
step 18810/19560 | loss 3.238729 (-1.31z)| norm 0.2366 (-0.63z)| lr 2.34e-06 | 323.18 ms | 52.2% bf16 MFU | 1624887 tok/s
step 18811/19560 | loss 3.252499 (-0.97z)| norm 0.2515 (+0.09z)| lr 2.34e-06 | 322.10 ms | 52.4% bf16 MFU | 1625027 tok/s
step 18812/19560 | loss 3.394250 (+2.39z)| norm 0.2658 (+0.78z)| lr 2.33e-06 | 322.18 ms | 52.4% bf16 MFU | 1625142 tok/s
step 18813/19560 | loss 3.255625 (-0.89z)| norm 0.2304 (-0.93z)| lr 2.33e-06 | 322.52 ms | 52.3% bf16 MFU | 1625164 tok/s
step 18814/19560 | loss 3.283633 (-0.23z)| norm 0.2366 (-0.62z)| lr 2.32e-06 | 322.39 ms | 52.4% bf16 MFU | 1625220 tok/s
step 18815/19560 | loss 3.330753 (+0.89z)| norm 0.2983 (+2.45z)| lr 2.31e-06 | 322.68 ms | 52.3% bf16 MFU | 1625198 tok/s
step 18816/19560 | loss 3.214871 (-1.82z)| norm 0.2580 (+0.43z)| lr 2.31e-06 | 322.51 ms | 52.3% bf16 MFU | 1625221 tok/s
step 18817/19560 | loss 3.372562 (+1.84z)| norm 0.2296 (-0.98z)| lr 2.30e-06 | 322.59 ms | 52.3% bf16 MFU | 1625223 tok/s
step 18818/19560 | loss 3.318274 (+0.57z)| norm 0.2368 (-0.62z)| lr 2.29e-06 | 323.26 ms | 52.2% bf16 MFU | 1625055 tok/s
step 18819/19560 | loss 3.360260 (+1.53z)| norm 0.2257 (-1.16z)| lr 2.29e-06 | 322.32 ms | 52.4% bf16 MFU | 1625132 tok/s
step 18820/19560 | loss 3.259814 (-0.79z)| norm 0.2523 (+0.16z)| lr 2.28e-06 | 322.19 ms | 52.4% bf16 MFU | 1625239 tok/s
step 18821/19560 | loss 3.316622 (+0.52z)| norm 0.2394 (-0.49z)| lr 2.28e-06 | 322.51 ms | 52.3% bf16 MFU | 1625259 tok/s
step 18822/19560 | loss 3.300522 (+0.15z)| norm 0.2480 (-0.07z)| lr 2.27e-06 | 322.93 ms | 52.3% bf16 MFU | 1625173 tok/s
step 18823/19560 | loss 3.290457 (-0.09z)| norm 0.2338 (-0.77z)| lr 2.26e-06 | 322.42 ms | 52.3% bf16 MFU | 1625220 tok/s
step 18824/19560 | loss 3.262817 (-0.72z)| norm 0.2500 (+0.05z)| lr 2.26e-06 | 323.18 ms | 52.2% bf16 MFU | 1625074 tok/s
step 18825/19560 | loss 3.258433 (-0.81z)| norm 0.2284 (-1.04z)| lr 2.25e-06 | 322.67 ms | 52.3% bf16 MFU | 1625063 tok/s
step 18826/19560 | loss 3.303374 (+0.23z)| norm 0.2334 (-0.78z)| lr 2.25e-06 | 322.71 ms | 52.3% bf16 MFU | 1625043 tok/s
step 18827/19560 | loss 3.334145 (+0.94z)| norm 0.2983 (+2.40z)| lr 2.24e-06 | 322.85 ms | 52.3% bf16 MFU | 1624987 tok/s
step 18828/19560 | loss 3.340106 (+1.10z)| norm 0.2383 (-0.54z)| lr 2.23e-06 | 323.27 ms | 52.2% bf16 MFU | 1624829 tok/s
step 18829/19560 | loss 3.269754 (-0.61z)| norm 0.2320 (-0.85z)| lr 2.23e-06 | 322.40 ms | 52.3% bf16 MFU | 1624897 tok/s
step 18830/19560 | loss 3.248829 (-1.11z)| norm 0.2488 (-0.03z)| lr 2.22e-06 | 323.63 ms | 52.1% bf16 MFU | 1624653 tok/s
step 18831/19560 | loss 3.303454 (+0.22z)| norm 0.2307 (-0.91z)| lr 2.22e-06 | 322.81 ms | 52.3% bf16 MFU | 1624627 tok/s
step 18832/19560 | loss 3.309545 (+0.37z)| norm 0.2232 (-1.26z)| lr 2.21e-06 | 323.15 ms | 52.2% bf16 MFU | 1624518 tok/s
step 18833/19560 | loss 3.263886 (-0.73z)| norm 0.2274 (-1.04z)| lr 2.20e-06 | 322.18 ms | 52.4% bf16 MFU | 1624659 tok/s
step 18834/19560 | loss 3.341308 (+1.13z)| norm 0.2413 (-0.33z)| lr 2.20e-06 | 322.98 ms | 52.3% bf16 MFU | 1624589 tok/s
step 18835/19560 | loss 3.288013 (-0.15z)| norm 0.2355 (-0.62z)| lr 2.19e-06 | 322.47 ms | 52.3% bf16 MFU | 1624652 tok/s
step 18836/19560 | loss 3.358906 (+1.55z)| norm 0.2957 (+2.36z)| lr 2.19e-06 | 323.09 ms | 52.2% bf16 MFU | 1624557 tok/s
step 18837/19560 | loss 3.311873 (+0.40z)| norm 0.2535 (+0.27z)| lr 2.18e-06 | 323.63 ms | 52.1% bf16 MFU | 1624330 tok/s
step 18838/19560 | loss 3.335228 (+0.95z)| norm 0.2508 (+0.14z)| lr 2.17e-06 | 322.27 ms | 52.4% bf16 MFU | 1624456 tok/s
step 18839/19560 | loss 3.271599 (-0.59z)| norm 0.2279 (-0.98z)| lr 2.17e-06 | 322.78 ms | 52.3% bf16 MFU | 1624448 tok/s
step 18840/19560 | loss 3.262029 (-0.81z)| norm 0.2281 (-0.96z)| lr 2.16e-06 | 323.77 ms | 52.1% bf16 MFU | 1624191 tok/s
step 18841/19560 | loss 3.287608 (-0.19z)| norm 0.2345 (-0.64z)| lr 2.16e-06 | 322.84 ms | 52.3% bf16 MFU | 1624181 tok/s
step 18842/19560 | loss 3.308968 (+0.32z)| norm 0.2362 (-0.54z)| lr 2.15e-06 | 322.66 ms | 52.3% bf16 MFU | 1624218 tok/s
step 18843/19560 | loss 3.365084 (+1.66z)| norm 0.2429 (-0.22z)| lr 2.14e-06 | 323.08 ms | 52.2% bf16 MFU | 1624146 tok/s
step 18844/19560 | loss 3.295907 (-0.00z)| norm 0.2783 (+1.51z)| lr 2.14e-06 | 323.31 ms | 52.2% bf16 MFU | 1624019 tok/s
step 18845/19560 | loss 3.271472 (-0.60z)| norm 0.2424 (-0.25z)| lr 2.13e-06 | 322.74 ms | 52.3% bf16 MFU | 1624042 tok/s
step 18846/19560 | loss 3.289859 (-0.15z)| norm 0.2305 (-0.83z)| lr 2.13e-06 | 321.99 ms | 52.4% bf16 MFU | 1624254 tok/s
step 18847/19560 | loss 3.285199 (-0.27z)| norm 0.2536 (+0.29z)| lr 2.12e-06 | 322.64 ms | 52.3% bf16 MFU | 1624291 tok/s
step 18848/19560 | loss 3.274688 (-0.53z)| norm 0.2325 (-0.73z)| lr 2.11e-06 | 323.08 ms | 52.2% bf16 MFU | 1624214 tok/s
step 18849/19560 | loss 3.318478 (+0.53z)| norm 0.2362 (-0.55z)| lr 2.11e-06 | 322.70 ms | 52.3% bf16 MFU | 1624239 tok/s
step 18850/19560 | loss 3.310767 (+0.35z)| norm 0.2544 (+0.33z)| lr 2.10e-06 | 322.75 ms | 52.3% bf16 MFU | 1624249 tok/s
step 18851/19560 | loss 3.335711 (+0.94z)| norm 0.2324 (-0.74z)| lr 2.10e-06 | 322.61 ms | 52.3% bf16 MFU | 1624292 tok/s
step 18852/19560 | loss 3.335258 (+0.91z)| norm 0.2362 (-0.54z)| lr 2.09e-06 | 323.34 ms | 52.2% bf16 MFU | 1624152 tok/s
step 18853/19560 | loss 3.270185 (-0.67z)| norm 0.2319 (-0.75z)| lr 2.08e-06 | 322.78 ms | 52.3% bf16 MFU | 1624158 tok/s
step 18854/19560 | loss 3.357802 (+1.49z)| norm 0.2435 (-0.18z)| lr 2.08e-06 | 322.56 ms | 52.3% bf16 MFU | 1624220 tok/s
step 18855/19560 | loss 3.363469 (+1.60z)| norm 0.2420 (-0.26z)| lr 2.07e-06 | 323.31 ms | 52.2% bf16 MFU | 1624090 tok/s
step 18856/19560 | loss 3.315926 (+0.43z)| norm 0.2401 (-0.35z)| lr 2.07e-06 | 322.46 ms | 52.3% bf16 MFU | 1624182 tok/s
step 18857/19560 | loss 3.315904 (+0.44z)| norm 0.2616 (+0.70z)| lr 2.06e-06 | 322.74 ms | 52.3% bf16 MFU | 1624198 tok/s
step 18858/19560 | loss 3.265355 (-0.80z)| norm 0.2522 (+0.23z)| lr 2.05e-06 | 323.23 ms | 52.2% bf16 MFU | 1624090 tok/s
step 18859/19560 | loss 3.274149 (-0.60z)| norm 0.2777 (+1.47z)| lr 2.05e-06 | 322.83 ms | 52.3% bf16 MFU | 1624087 tok/s
step 18860/19560 | loss 3.281290 (-0.41z)| norm 0.2526 (+0.23z)| lr 2.04e-06 | 322.31 ms | 52.4% bf16 MFU | 1624216 tok/s
step 18861/19560 | loss 3.270825 (-0.66z)| norm 0.2360 (-0.58z)| lr 2.04e-06 | 323.13 ms | 52.2% bf16 MFU | 1624131 tok/s
step 18862/19560 | loss 3.326401 (+0.71z)| norm 0.2542 (+0.30z)| lr 2.03e-06 | 322.72 ms | 52.3% bf16 MFU | 1624155 tok/s
step 18863/19560 | loss 3.313870 (+0.40z)| norm 0.2847 (+1.77z)| lr 2.03e-06 | 323.09 ms | 52.2% bf16 MFU | 1624084 tok/s
step 18864/19560 | loss 3.280265 (-0.46z)| norm 0.2292 (-0.93z)| lr 2.02e-06 | 323.10 ms | 52.2% bf16 MFU | 1624014 tok/s
step 18865/19560 | loss 3.369154 (+1.76z)| norm 0.2313 (-0.82z)| lr 2.01e-06 | 323.30 ms | 52.2% bf16 MFU | 1623896 tok/s
step 18866/19560 | loss 3.295620 (-0.10z)| norm 0.2277 (-0.99z)| lr 2.01e-06 | 323.29 ms | 52.2% bf16 MFU | 1623788 tok/s
step 18867/19560 | loss 3.292583 (-0.18z)| norm 0.2295 (-0.89z)| lr 2.00e-06 | 322.69 ms | 52.3% bf16 MFU | 1623834 tok/s
step 18868/19560 | loss 3.367362 (+1.68z)| norm 0.2641 (+0.79z)| lr 2.00e-06 | 323.24 ms | 52.2% bf16 MFU | 1623741 tok/s
step 18869/19560 | loss 3.287007 (-0.33z)| norm 0.2732 (+1.21z)| lr 1.99e-06 | 322.54 ms | 52.3% bf16 MFU | 1623828 tok/s
step 18870/19560 | loss 3.292265 (-0.20z)| norm 0.2313 (-0.79z)| lr 1.99e-06 | 322.32 ms | 52.4% bf16 MFU | 1623967 tok/s
step 18871/19560 | loss 3.323164 (+0.58z)| norm 0.2647 (+0.80z)| lr 1.98e-06 | 322.62 ms | 52.3% bf16 MFU | 1624023 tok/s
step 18872/19560 | loss 3.309722 (+0.26z)| norm 0.2301 (-0.85z)| lr 1.97e-06 | 322.74 ms | 52.3% bf16 MFU | 1624046 tok/s
step 18873/19560 | loss 3.334164 (+0.87z)| norm 0.2502 (+0.11z)| lr 1.97e-06 | 322.64 ms | 52.3% bf16 MFU | 1624093 tok/s
step 18874/19560 | loss 3.280023 (-0.51z)| norm 0.2319 (-0.75z)| lr 1.96e-06 | 323.26 ms | 52.2% bf16 MFU | 1623982 tok/s
step 18875/19560 | loss 3.351855 (+1.30z)| norm 0.2278 (-0.94z)| lr 1.96e-06 | 323.12 ms | 52.2% bf16 MFU | 1623912 tok/s
step 18876/19560 | loss 3.272285 (-0.73z)| norm 0.2346 (-0.61z)| lr 1.95e-06 | 322.56 ms | 52.3% bf16 MFU | 1623987 tok/s
step 18877/19560 | loss 3.241423 (-1.51z)| norm 0.3087 (+2.82z)| lr 1.95e-06 | 323.07 ms | 52.2% bf16 MFU | 1623928 tok/s
step 18878/19560 | loss 3.358942 (+1.46z)| norm 0.2339 (-0.64z)| lr 1.94e-06 | 322.81 ms | 52.3% bf16 MFU | 1623938 tok/s
step 18879/19560 | loss 3.264940 (-0.91z)| norm 0.2455 (-0.10z)| lr 1.93e-06 | 323.43 ms | 52.2% bf16 MFU | 1623792 tok/s
step 18880/19560 | loss 3.402499 (+2.48z)| norm 0.2843 (+1.66z)| lr 1.93e-06 | 322.12 ms | 52.4% bf16 MFU | 1623982 tok/s
step 18881/19560 | loss 3.249750 (-1.28z)| norm 0.2391 (-0.42z)| lr 1.92e-06 | 322.96 ms | 52.3% bf16 MFU | 1623953 tok/s
step 18882/19560 | loss 3.301140 (-0.01z)| norm 0.2336 (-0.67z)| lr 1.92e-06 | 322.39 ms | 52.4% bf16 MFU | 1624068 tok/s
step 18883/19560 | loss 3.297466 (-0.10z)| norm 0.2965 (+2.16z)| lr 1.91e-06 | 322.54 ms | 52.3% bf16 MFU | 1624140 tok/s
step 18884/19560 | loss 3.270291 (-0.76z)| norm 0.2528 (+0.18z)| lr 1.91e-06 | 323.96 ms | 52.1% bf16 MFU | 1623852 tok/s
step 18885/19560 | loss 3.258807 (-1.05z)| norm 0.2529 (+0.18z)| lr 1.90e-06 | 323.02 ms | 52.2% bf16 MFU | 1623814 tok/s
step 18886/19560 | loss 3.332834 (+0.77z)| norm 0.2293 (-0.88z)| lr 1.89e-06 | 322.59 ms | 52.3% bf16 MFU | 1623886 tok/s
step 18887/19560 | loss 3.310992 (+0.23z)| norm 0.2674 (+0.83z)| lr 1.89e-06 | 322.51 ms | 52.3% bf16 MFU | 1623975 tok/s
step 18888/19560 | loss 3.290231 (-0.29z)| norm 0.2365 (-0.57z)| lr 1.88e-06 | 323.87 ms | 52.1% bf16 MFU | 1623719 tok/s
step 18889/19560 | loss 3.319905 (+0.45z)| norm 0.2322 (-0.77z)| lr 1.88e-06 | 322.83 ms | 52.3% bf16 MFU | 1623736 tok/s
step 18890/19560 | loss 3.282154 (-0.48z)| norm 0.2463 (-0.12z)| lr 1.87e-06 | 323.28 ms | 52.2% bf16 MFU | 1623637 tok/s
step 18891/19560 | loss 3.306343 (+0.11z)| norm 0.2267 (-1.00z)| lr 1.87e-06 | 322.79 ms | 52.3% bf16 MFU | 1623668 tok/s
step 18892/19560 | loss 3.265449 (-0.92z)| norm 0.2417 (-0.30z)| lr 1.86e-06 | 322.44 ms | 52.3% bf16 MFU | 1623786 tok/s
step 18893/19560 | loss 3.349765 (+1.18z)| norm 0.2443 (-0.18z)| lr 1.86e-06 | 322.96 ms | 52.3% bf16 MFU | 1623767 tok/s
step 18894/19560 | loss 3.269383 (-0.82z)| norm 0.2438 (-0.21z)| lr 1.85e-06 | 323.93 ms | 52.1% bf16 MFU | 1623504 tok/s
step 18895/19560 | loss 3.282837 (-0.48z)| norm 0.2696 (+0.96z)| lr 1.84e-06 | 323.34 ms | 52.2% bf16 MFU | 1623403 tok/s
step 18896/19560 | loss 3.334063 (+0.80z)| norm 0.2341 (-0.65z)| lr 1.84e-06 | 323.22 ms | 52.2% bf16 MFU | 1623336 tok/s
step 18897/19560 | loss 3.306421 (+0.10z)| norm 0.2583 (+0.48z)| lr 1.83e-06 | 322.78 ms | 52.3% bf16 MFU | 1623382 tok/s
step 18898/19560 | loss 3.304432 (+0.06z)| norm 0.2662 (+0.85z)| lr 1.83e-06 | 323.44 ms | 52.2% bf16 MFU | 1623262 tok/s
step 18899/19560 | loss 3.341844 (+0.99z)| norm 0.2435 (-0.21z)| lr 1.82e-06 | 323.99 ms | 52.1% bf16 MFU | 1623010 tok/s
step 18900/19560 | loss 3.335851 (+0.83z)| norm 0.2241 (-1.12z)| lr 1.82e-06 | 322.58 ms | 52.3% bf16 MFU | 1623125 tok/s
step 18901/19560 | loss 3.234061 (-1.68z)| norm 0.2287 (-0.90z)| lr 1.81e-06 | 323.59 ms | 52.2% bf16 MFU | 1622980 tok/s
step 18902/19560 | loss 3.360747 (+1.43z)| norm 0.2389 (-0.42z)| lr 1.81e-06 | 323.00 ms | 52.3% bf16 MFU | 1622989 tok/s
step 18903/19560 | loss 3.260835 (-1.02z)| norm 0.2244 (-1.10z)| lr 1.80e-06 | 323.33 ms | 52.2% bf16 MFU | 1622916 tok/s
step 18904/19560 | loss 3.324749 (+0.53z)| norm 0.2433 (-0.21z)| lr 1.79e-06 | 322.45 ms | 52.3% bf16 MFU | 1623068 tok/s
step 18905/19560 | loss 3.281020 (-0.54z)| norm 0.2442 (-0.17z)| lr 1.79e-06 | 323.29 ms | 52.2% bf16 MFU | 1623001 tok/s
step 18906/19560 | loss 3.295523 (-0.18z)| norm 0.2604 (+0.59z)| lr 1.78e-06 | 322.94 ms | 52.3% bf16 MFU | 1623027 tok/s
step 18907/19560 | loss 3.278681 (-0.60z)| norm 0.2904 (+1.95z)| lr 1.78e-06 | 323.02 ms | 52.2% bf16 MFU | 1623029 tok/s
step 18908/19560 | loss 3.272352 (-0.74z)| norm 0.2361 (-0.56z)| lr 1.77e-06 | 322.62 ms | 52.3% bf16 MFU | 1623132 tok/s
step 18909/19560 | loss 3.304499 (+0.06z)| norm 0.2496 (+0.08z)| lr 1.77e-06 | 322.87 ms | 52.3% bf16 MFU | 1623167 tok/s
step 18910/19560 | loss 3.298615 (-0.09z)| norm 0.2385 (-0.45z)| lr 1.76e-06 | 322.83 ms | 52.3% bf16 MFU | 1623211 tok/s
step 18911/19560 | loss 3.338891 (+0.89z)| norm 0.2524 (+0.20z)| lr 1.76e-06 | 323.03 ms | 52.2% bf16 MFU | 1623201 tok/s
step 18912/19560 | loss 3.266259 (-0.91z)| norm 0.2594 (+0.52z)| lr 1.75e-06 | 323.44 ms | 52.2% bf16 MFU | 1623090 tok/s
step 18913/19560 | loss 3.317482 (+0.36z)| norm 0.2275 (-0.98z)| lr 1.75e-06 | 322.75 ms | 52.3% bf16 MFU | 1623159 tok/s
step 18914/19560 | loss 3.312356 (+0.22z)| norm 0.2375 (-0.50z)| lr 1.74e-06 | 323.38 ms | 52.2% bf16 MFU | 1623064 tok/s
step 18915/19560 | loss 3.263609 (-1.00z)| norm 0.2987 (+2.31z)| lr 1.74e-06 | 322.77 ms | 52.3% bf16 MFU | 1623129 tok/s
step 18916/19560 | loss 3.246006 (-1.42z)| norm 0.2359 (-0.58z)| lr 1.73e-06 | 323.37 ms | 52.2% bf16 MFU | 1623039 tok/s
step 18917/19560 | loss 3.316644 (+0.35z)| norm 0.2536 (+0.26z)| lr 1.72e-06 | 322.54 ms | 52.3% bf16 MFU | 1623161 tok/s
step 18918/19560 | loss 3.413949 (+2.68z)| norm 0.2847 (+1.90z)| lr 1.72e-06 | 323.01 ms | 52.2% bf16 MFU | 1623158 tok/s
step 18919/19560 | loss 3.271730 (-0.77z)| norm 0.2376 (-0.50z)| lr 1.71e-06 | 323.00 ms | 52.3% bf16 MFU | 1623160 tok/s
step 18920/19560 | loss 3.318205 (+0.37z)| norm 0.2418 (-0.29z)| lr 1.71e-06 | 322.25 ms | 52.4% bf16 MFU | 1623350 tok/s
step 18921/19560 | loss 3.213272 (-2.15z)| norm 0.2357 (-0.61z)| lr 1.70e-06 | 323.05 ms | 52.2% bf16 MFU | 1623329 tok/s
step 18922/19560 | loss 3.275551 (-0.66z)| norm 0.2248 (-1.16z)| lr 1.70e-06 | 323.07 ms | 52.2% bf16 MFU | 1623303 tok/s
step 18923/19560 | loss 3.286259 (-0.39z)| norm 0.2383 (-0.46z)| lr 1.69e-06 | 323.01 ms | 52.2% bf16 MFU | 1623295 tok/s
step 18924/19560 | loss 3.277746 (-0.59z)| norm 0.2513 (+0.24z)| lr 1.69e-06 | 322.97 ms | 52.3% bf16 MFU | 1623297 tok/s
step 18925/19560 | loss 3.253678 (-1.16z)| norm 0.2718 (+1.30z)| lr 1.68e-06 | 322.27 ms | 52.4% bf16 MFU | 1623474 tok/s
step 18926/19560 | loss 3.296340 (-0.14z)| norm 0.2527 (+0.29z)| lr 1.68e-06 | 323.35 ms | 52.2% bf16 MFU | 1623371 tok/s
step 18927/19560 | loss 3.286576 (-0.38z)| norm 0.2521 (+0.26z)| lr 1.67e-06 | 322.60 ms | 52.3% bf16 MFU | 1623463 tok/s
step 18928/19560 | loss 3.247656 (-1.38z)| norm 0.2210 (-1.35z)| lr 1.67e-06 | 322.90 ms | 52.3% bf16 MFU | 1623474 tok/s
step 18929/19560 | loss 3.261998 (-1.00z)| norm 0.2300 (-0.87z)| lr 1.66e-06 | 323.45 ms | 52.2% bf16 MFU | 1623346 tok/s
step 18930/19560 | loss 3.207069 (-2.36z)| norm 0.2506 (+0.20z)| lr 1.66e-06 | 322.67 ms | 52.3% bf16 MFU | 1623420 tok/s
step 18931/19560 | loss 3.263300 (-0.92z)| norm 0.2294 (-0.90z)| lr 1.65e-06 | 323.17 ms | 52.2% bf16 MFU | 1623364 tok/s
step 18932/19560 | loss 3.267941 (-0.80z)| norm 0.2605 (+0.72z)| lr 1.65e-06 | 322.96 ms | 52.3% bf16 MFU | 1623366 tok/s
step 18933/19560 | loss 3.237557 (-1.55z)| norm 0.2544 (+0.41z)| lr 1.64e-06 | 323.03 ms | 52.2% bf16 MFU | 1623349 tok/s
step 18934/19560 | loss 3.350679 (+1.29z)| norm 0.2622 (+0.81z)| lr 1.63e-06 | 323.76 ms | 52.1% bf16 MFU | 1623150 tok/s
step 18935/19560 | loss 3.244780 (-1.35z)| norm 0.2306 (-0.84z)| lr 1.63e-06 | 323.00 ms | 52.3% bf16 MFU | 1623150 tok/s
step 18936/19560 | loss 3.267953 (-0.76z)| norm 0.2385 (-0.42z)| lr 1.62e-06 | 322.78 ms | 52.3% bf16 MFU | 1623208 tok/s
step 18937/19560 | loss 3.330532 (+0.82z)| norm 0.2819 (+1.85z)| lr 1.62e-06 | 322.91 ms | 52.3% bf16 MFU | 1623230 tok/s
step 18938/19560 | loss 3.330638 (+0.81z)| norm 0.2340 (-0.67z)| lr 1.61e-06 | 322.84 ms | 52.3% bf16 MFU | 1623266 tok/s
step 18939/19560 | loss 3.242438 (-1.43z)| norm 0.2235 (-1.20z)| lr 1.61e-06 | 323.11 ms | 52.2% bf16 MFU | 1623235 tok/s
step 18940/19560 | loss 3.238943 (-1.51z)| norm 0.2572 (+0.56z)| lr 1.60e-06 | 322.80 ms | 52.3% bf16 MFU | 1623282 tok/s
step 18941/19560 | loss 3.377970 (+2.02z)| norm 0.2841 (+1.93z)| lr 1.60e-06 | 322.72 ms | 52.3% bf16 MFU | 1623347 tok/s
step 18942/19560 | loss 3.281736 (-0.43z)| norm 0.2344 (-0.64z)| lr 1.59e-06 | 322.85 ms | 52.3% bf16 MFU | 1623376 tok/s
step 18943/19560 | loss 3.235565 (-1.57z)| norm 0.2318 (-0.77z)| lr 1.59e-06 | 322.33 ms | 52.4% bf16 MFU | 1623534 tok/s
step 18944/19560 | loss 3.309377 (+0.28z)| norm 0.2446 (-0.09z)| lr 1.58e-06 | 322.15 ms | 52.4% bf16 MFU | 1623732 tok/s
step 18945/19560 | loss 3.294449 (-0.09z)| norm 0.2232 (-1.22z)| lr 1.58e-06 | 323.26 ms | 52.2% bf16 MFU | 1623640 tok/s
step 18946/19560 | loss 3.293093 (-0.12z)| norm 0.2317 (-0.76z)| lr 1.57e-06 | 322.53 ms | 52.3% bf16 MFU | 1623734 tok/s
step 18947/19560 | loss 3.207849 (-2.29z)| norm 0.2463 (+0.00z)| lr 1.57e-06 | 322.59 ms | 52.3% bf16 MFU | 1623810 tok/s
step 18948/19560 | loss 3.264426 (-0.83z)| norm 0.2314 (-0.78z)| lr 1.56e-06 | 322.48 ms | 52.3% bf16 MFU | 1623909 tok/s
step 18949/19560 | loss 3.339788 (+1.11z)| norm 0.2994 (+2.73z)| lr 1.56e-06 | 322.88 ms | 52.3% bf16 MFU | 1623902 tok/s
step 18950/19560 | loss 3.216854 (-2.01z)| norm 0.2611 (+0.74z)| lr 1.55e-06 | 323.05 ms | 52.2% bf16 MFU | 1623854 tok/s
step 18951/19560 | loss 3.287913 (-0.21z)| norm 0.2222 (-1.25z)| lr 1.55e-06 | 322.61 ms | 52.3% bf16 MFU | 1623919 tok/s
step 18952/19560 | loss 3.280205 (-0.41z)| norm 0.2257 (-1.06z)| lr 1.54e-06 | 322.58 ms | 52.3% bf16 MFU | 1623989 tok/s
step 18953/19560 | loss 3.274002 (-0.57z)| norm 0.2609 (+0.73z)| lr 1.54e-06 | 323.27 ms | 52.2% bf16 MFU | 1623881 tok/s
step 18954/19560 | loss 3.319890 (+0.60z)| norm 0.2263 (-1.03z)| lr 1.53e-06 | 322.73 ms | 52.3% bf16 MFU | 1623915 tok/s
step 18955/19560 | loss 3.229926 (-1.66z)| norm 0.2918 (+2.32z)| lr 1.53e-06 | 322.62 ms | 52.3% bf16 MFU | 1623975 tok/s
step 18956/19560 | loss 3.230926 (-1.61z)| norm 0.2328 (-0.70z)| lr 1.52e-06 | 322.47 ms | 52.3% bf16 MFU | 1624069 tok/s
step 18957/19560 | loss 3.285321 (-0.24z)| norm 0.2435 (-0.16z)| lr 1.52e-06 | 323.03 ms | 52.2% bf16 MFU | 1624017 tok/s
step 18958/19560 | loss 3.345006 (+1.24z)| norm 0.2496 (+0.15z)| lr 1.51e-06 | 322.57 ms | 52.3% bf16 MFU | 1624083 tok/s
step 18959/19560 | loss 3.287722 (-0.20z)| norm 0.2516 (+0.25z)| lr 1.51e-06 | 322.62 ms | 52.3% bf16 MFU | 1624134 tok/s
step 18960/19560 | loss 3.302962 (+0.19z)| norm 0.3265 (+3.84z)| lr 1.50e-06 | 323.10 ms | 52.2% bf16 MFU | 1624060 tok/s
step 18961/19560 | loss 3.282878 (-0.32z)| norm 0.2523 (+0.22z)| lr 1.50e-06 | 322.18 ms | 52.4% bf16 MFU | 1624223 tok/s
step 18962/19560 | loss 3.287015 (-0.21z)| norm 0.2417 (-0.30z)| lr 1.49e-06 | 322.79 ms | 52.3% bf16 MFU | 1624224 tok/s
step 18963/19560 | loss 3.406940 (+2.72z)| norm 0.2429 (-0.24z)| lr 1.49e-06 | 322.46 ms | 52.3% bf16 MFU | 1624306 tok/s
step 18964/19560 | loss 3.344846 (+1.20z)| norm 0.2273 (-0.99z)| lr 1.48e-06 | 322.54 ms | 52.3% bf16 MFU | 1624367 tok/s
step 18965/19560 | loss 3.310990 (+0.37z)| norm 0.2583 (+0.54z)| lr 1.48e-06 | 322.90 ms | 52.3% bf16 MFU | 1624331 tok/s
step 18966/19560 | loss 3.295127 (-0.01z)| norm 0.2399 (-0.36z)| lr 1.47e-06 | 322.84 ms | 52.3% bf16 MFU | 1624314 tok/s
step 18967/19560 | loss 3.272367 (-0.58z)| norm 0.2234 (-1.18z)| lr 1.47e-06 | 322.24 ms | 52.4% bf16 MFU | 1624449 tok/s
step 18968/19560 | loss 3.251211 (-1.10z)| norm 0.2371 (-0.50z)| lr 1.46e-06 | 322.45 ms | 52.3% bf16 MFU | 1624523 tok/s
step 18969/19560 | loss 3.250452 (-1.10z)| norm 0.2449 (-0.12z)| lr 1.46e-06 | 322.59 ms | 52.3% bf16 MFU | 1624560 tok/s
step 18970/19560 | loss 3.309643 (+0.35z)| norm 0.2416 (-0.29z)| lr 1.45e-06 | 322.69 ms | 52.3% bf16 MFU | 1624569 tok/s
step 18971/19560 | loss 3.275833 (-0.47z)| norm 0.2403 (-0.35z)| lr 1.45e-06 | 322.26 ms | 52.4% bf16 MFU | 1624687 tok/s
step 18972/19560 | loss 3.427148 (+3.14z)| norm 0.3760 (+5.59z)| lr 1.44e-06 | 322.56 ms | 52.3% bf16 MFU | 1624724 tok/s
step 18973/19560 | loss 3.301376 (+0.13z)| norm 0.2569 (+0.38z)| lr 1.44e-06 | 322.62 ms | 52.3% bf16 MFU | 1624742 tok/s
step 18974/19560 | loss 3.272385 (-0.56z)| norm 0.2399 (-0.37z)| lr 1.43e-06 | 322.64 ms | 52.3% bf16 MFU | 1624756 tok/s
step 18975/19560 | loss 3.349358 (+1.26z)| norm 0.2619 (+0.59z)| lr 1.43e-06 | 322.07 ms | 52.4% bf16 MFU | 1624911 tok/s
step 18976/19560 | loss 3.259958 (-0.86z)| norm 0.2398 (-0.38z)| lr 1.42e-06 | 323.38 ms | 52.2% bf16 MFU | 1624729 tok/s
step 18977/19560 | loss 3.304344 (+0.20z)| norm 0.2406 (-0.34z)| lr 1.42e-06 | 322.16 ms | 52.4% bf16 MFU | 1624864 tok/s
step 18978/19560 | loss 3.279763 (-0.38z)| norm 0.2371 (-0.49z)| lr 1.41e-06 | 322.18 ms | 52.4% bf16 MFU | 1624986 tok/s
step 18979/19560 | loss 3.191910 (-2.39z)| norm 0.2628 (+0.62z)| lr 1.41e-06 | 323.30 ms | 52.2% bf16 MFU | 1624820 tok/s
step 18980/19560 | loss 3.308120 (+0.32z)| norm 0.2500 (+0.06z)| lr 1.40e-06 | 322.67 ms | 52.3% bf16 MFU | 1624821 tok/s
step 18981/19560 | loss 3.249804 (-1.04z)| norm 0.2974 (+2.08z)| lr 1.40e-06 | 322.67 ms | 52.3% bf16 MFU | 1624822 tok/s
step 18982/19560 | loss 3.222304 (-1.65z)| norm 0.2303 (-0.81z)| lr 1.39e-06 | 323.00 ms | 52.3% bf16 MFU | 1624740 tok/s
step 18983/19560 | loss 3.285583 (-0.17z)| norm 0.2282 (-0.89z)| lr 1.39e-06 | 323.18 ms | 52.2% bf16 MFU | 1624616 tok/s
step 18984/19560 | loss 3.239424 (-1.23z)| norm 0.2430 (-0.26z)| lr 1.38e-06 | 322.38 ms | 52.4% bf16 MFU | 1624701 tok/s
step 18985/19560 | loss 3.210497 (-1.86z)| norm 0.2315 (-0.74z)| lr 1.38e-06 | 322.25 ms | 52.4% bf16 MFU | 1624813 tok/s
step 18986/19560 | loss 3.322721 (+0.72z)| norm 0.2390 (-0.41z)| lr 1.38e-06 | 323.18 ms | 52.2% bf16 MFU | 1624686 tok/s
step 18987/19560 | loss 3.213575 (-1.77z)| norm 0.2236 (-1.06z)| lr 1.37e-06 | 322.68 ms | 52.3% bf16 MFU | 1624690 tok/s
step 18988/19560 | loss 3.274154 (-0.39z)| norm 0.2352 (-0.55z)| lr 1.37e-06 | 322.31 ms | 52.4% bf16 MFU | 1624790 tok/s
step 18989/19560 | loss 3.238815 (-1.18z)| norm 0.2348 (-0.57z)| lr 1.36e-06 | 322.57 ms | 52.3% bf16 MFU | 1624818 tok/s
step 18990/19560 | loss 3.228454 (-1.39z)| norm 0.2298 (-0.77z)| lr 1.36e-06 | 322.64 ms | 52.3% bf16 MFU | 1624826 tok/s
step 18991/19560 | loss 3.298103 (+0.18z)| norm 0.3034 (+2.34z)| lr 1.35e-06 | 322.90 ms | 52.3% bf16 MFU | 1624769 tok/s
step 18992/19560 | loss 3.287405 (-0.06z)| norm 0.2537 (+0.23z)| lr 1.35e-06 | 322.90 ms | 52.3% bf16 MFU | 1624714 tok/s
step 18993/19560 | loss 3.305640 (+0.37z)| norm 0.2880 (+1.65z)| lr 1.34e-06 | 322.49 ms | 52.3% bf16 MFU | 1624765 tok/s
step 18994/19560 | loss 3.280278 (-0.21z)| norm 0.2309 (-0.75z)| lr 1.34e-06 | 322.25 ms | 52.4% bf16 MFU | 1624876 tok/s
step 18995/19560 | loss 3.273921 (-0.35z)| norm 0.2379 (-0.46z)| lr 1.33e-06 | 322.65 ms | 52.3% bf16 MFU | 1624879 tok/s
step 18996/19560 | loss 3.325275 (+0.84z)| norm 0.2322 (-0.69z)| lr 1.33e-06 | 323.06 ms | 52.2% bf16 MFU | 1624778 tok/s
step 18997/19560 | loss 3.223436 (-1.49z)| norm 0.2383 (-0.42z)| lr 1.32e-06 | 322.42 ms | 52.3% bf16 MFU | 1624845 tok/s
step 18998/19560 | loss 3.229519 (-1.33z)| norm 0.2357 (-0.53z)| lr 1.32e-06 | 322.44 ms | 52.3% bf16 MFU | 1624904 tok/s
step 18999/19560 | loss 3.232114 (-1.25z)| norm 0.2327 (-0.65z)| lr 1.31e-06 | 322.93 ms | 52.3% bf16 MFU | 1624835 tok/s
step 19000/19560 | loss 3.246706 (-0.90z)| norm 0.2653 (+0.72z)| lr 1.31e-06 | 322.89 ms | 52.3% bf16 MFU | 1624779 tok/s
val loss 3.274507
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3052/10042 = 0.303924
step 19001/19560 | loss 3.315334 (+0.65z)| norm 0.2624 (+0.59z)| lr 1.30e-06 | 322.95 ms | 52.3% bf16 MFU | 1624713 tok/s
step 19002/19560 | loss 3.281774 (-0.11z)| norm 0.2259 (-0.95z)| lr 1.30e-06 | 322.70 ms | 52.3% bf16 MFU | 1624712 tok/s
step 19003/19560 | loss 3.287724 (+0.04z)| norm 0.2454 (-0.13z)| lr 1.29e-06 | 322.55 ms | 52.3% bf16 MFU | 1624748 tok/s
step 19004/19560 | loss 3.285656 (-0.01z)| norm 0.2290 (-0.82z)| lr 1.29e-06 | 322.81 ms | 52.3% bf16 MFU | 1624718 tok/s
step 19005/19560 | loss 3.332619 (+1.04z)| norm 0.2357 (-0.53z)| lr 1.29e-06 | 322.92 ms | 52.3% bf16 MFU | 1624662 tok/s
step 19006/19560 | loss 3.236593 (-1.13z)| norm 0.2459 (-0.09z)| lr 1.28e-06 | 322.78 ms | 52.3% bf16 MFU | 1624644 tok/s
step 19007/19560 | loss 3.289782 (+0.08z)| norm 0.2276 (-0.88z)| lr 1.28e-06 | 323.01 ms | 52.2% bf16 MFU | 1624568 tok/s
step 19008/19560 | loss 3.330597 (+1.06z)| norm 0.2599 (+0.53z)| lr 1.27e-06 | 322.59 ms | 52.3% bf16 MFU | 1624602 tok/s
step 19009/19560 | loss 3.303335 (+0.41z)| norm 0.2725 (+1.07z)| lr 1.27e-06 | 323.03 ms | 52.2% bf16 MFU | 1624524 tok/s
step 19010/19560 | loss 3.284701 (-0.03z)| norm 0.2597 (+0.51z)| lr 1.26e-06 | 322.85 ms | 52.3% bf16 MFU | 1624495 tok/s
step 19011/19560 | loss 3.360579 (+1.73z)| norm 0.2402 (-0.33z)| lr 1.26e-06 | 323.04 ms | 52.2% bf16 MFU | 1624420 tok/s
step 19012/19560 | loss 3.331506 (+1.04z)| norm 0.2374 (-0.45z)| lr 1.25e-06 | 322.53 ms | 52.3% bf16 MFU | 1624476 tok/s
step 19013/19560 | loss 3.255437 (-0.73z)| norm 0.2247 (-1.00z)| lr 1.25e-06 | 322.98 ms | 52.3% bf16 MFU | 1624417 tok/s
step 19014/19560 | loss 3.229300 (-1.32z)| norm 0.2260 (-0.94z)| lr 1.24e-06 | 322.96 ms | 52.3% bf16 MFU | 1624365 tok/s
step 19015/19560 | loss 3.449984 (+3.59z)| norm 0.2525 (+0.23z)| lr 1.24e-06 | 322.32 ms | 52.4% bf16 MFU | 1624477 tok/s
step 19016/19560 | loss 3.260325 (-0.58z)| norm 0.2284 (-0.83z)| lr 1.24e-06 | 322.96 ms | 52.3% bf16 MFU | 1624422 tok/s
step 19017/19560 | loss 3.242328 (-0.97z)| norm 0.2310 (-0.71z)| lr 1.23e-06 | 322.70 ms | 52.3% bf16 MFU | 1624436 tok/s
step 19018/19560 | loss 3.263934 (-0.49z)| norm 0.2326 (-0.63z)| lr 1.23e-06 | 322.72 ms | 52.3% bf16 MFU | 1624444 tok/s
step 19019/19560 | loss 3.318134 (+0.70z)| norm 0.2602 (+0.57z)| lr 1.22e-06 | 323.00 ms | 52.3% bf16 MFU | 1624382 tok/s
step 19020/19560 | loss 3.198064 (-1.90z)| norm 0.3425 (+3.91z)| lr 1.22e-06 | 322.21 ms | 52.4% bf16 MFU | 1624522 tok/s
step 19021/19560 | loss 3.202006 (-1.78z)| norm 0.2360 (-0.50z)| lr 1.21e-06 | 322.96 ms | 52.3% bf16 MFU | 1624464 tok/s
step 19022/19560 | loss 3.270724 (-0.30z)| norm 0.2572 (+0.38z)| lr 1.21e-06 | 322.89 ms | 52.3% bf16 MFU | 1624428 tok/s
step 19023/19560 | loss 3.287889 (+0.07z)| norm 0.2268 (-0.86z)| lr 1.20e-06 | 322.88 ms | 52.3% bf16 MFU | 1624396 tok/s
step 19024/19560 | loss 3.380030 (+2.02z)| norm 0.2297 (-0.74z)| lr 1.20e-06 | 322.34 ms | 52.4% bf16 MFU | 1624501 tok/s
step 19025/19560 | loss 3.400276 (+2.38z)| norm 0.2336 (-0.58z)| lr 1.19e-06 | 322.26 ms | 52.4% bf16 MFU | 1624621 tok/s
step 19026/19560 | loss 3.340494 (+1.13z)| norm 0.2309 (-0.68z)| lr 1.19e-06 | 322.36 ms | 52.4% bf16 MFU | 1624710 tok/s
step 19027/19560 | loss 3.336265 (+1.04z)| norm 0.2272 (-0.82z)| lr 1.19e-06 | 322.51 ms | 52.3% bf16 MFU | 1624757 tok/s
step 19028/19560 | loss 3.307832 (+0.46z)| norm 0.2792 (+1.30z)| lr 1.18e-06 | 322.92 ms | 52.3% bf16 MFU | 1624698 tok/s
step 19029/19560 | loss 3.271359 (-0.31z)| norm 0.2522 (+0.18z)| lr 1.18e-06 | 322.56 ms | 52.3% bf16 MFU | 1624734 tok/s
step 19030/19560 | loss 3.285008 (-0.01z)| norm 0.2307 (-0.70z)| lr 1.17e-06 | 322.60 ms | 52.3% bf16 MFU | 1624757 tok/s
step 19031/19560 | loss 3.196199 (-1.85z)| norm 0.2597 (+0.48z)| lr 1.17e-06 | 322.71 ms | 52.3% bf16 MFU | 1624752 tok/s
step 19032/19560 | loss 3.258995 (-0.53z)| norm 0.2380 (-0.41z)| lr 1.16e-06 | 323.27 ms | 52.2% bf16 MFU | 1624606 tok/s
step 19033/19560 | loss 3.301275 (+0.35z)| norm 0.2376 (-0.42z)| lr 1.16e-06 | 322.99 ms | 52.3% bf16 MFU | 1624538 tok/s
step 19034/19560 | loss 3.269713 (-0.31z)| norm 0.2416 (-0.25z)| lr 1.16e-06 | 322.99 ms | 52.3% bf16 MFU | 1624472 tok/s
step 19035/19560 | loss 3.338879 (+1.12z)| norm 0.2645 (+0.71z)| lr 1.15e-06 | 322.40 ms | 52.3% bf16 MFU | 1624560 tok/s
step 19036/19560 | loss 3.289355 (+0.09z)| norm 0.2330 (-0.60z)| lr 1.15e-06 | 322.63 ms | 52.3% bf16 MFU | 1624584 tok/s
step 19037/19560 | loss 3.239436 (-0.93z)| norm 0.2373 (-0.42z)| lr 1.14e-06 | 323.19 ms | 52.2% bf16 MFU | 1624466 tok/s
step 19038/19560 | loss 3.273980 (-0.21z)| norm 0.2336 (-0.57z)| lr 1.14e-06 | 323.27 ms | 52.2% bf16 MFU | 1624334 tok/s
step 19039/19560 | loss 3.253624 (-0.62z)| norm 0.2268 (-0.84z)| lr 1.13e-06 | 322.73 ms | 52.3% bf16 MFU | 1624345 tok/s
step 19040/19560 | loss 3.299861 (+0.33z)| norm 0.2613 (+0.59z)| lr 1.13e-06 | 322.72 ms | 52.3% bf16 MFU | 1624359 tok/s
step 19041/19560 | loss 3.298908 (+0.32z)| norm 0.2379 (-0.39z)| lr 1.12e-06 | 322.86 ms | 52.3% bf16 MFU | 1624335 tok/s
step 19042/19560 | loss 3.215261 (-1.40z)| norm 0.2414 (-0.24z)| lr 1.12e-06 | 322.99 ms | 52.3% bf16 MFU | 1624280 tok/s
step 19043/19560 | loss 3.241434 (-0.85z)| norm 0.2343 (-0.53z)| lr 1.12e-06 | 322.96 ms | 52.3% bf16 MFU | 1624236 tok/s
step 19044/19560 | loss 3.238705 (-0.91z)| norm 0.2308 (-0.67z)| lr 1.11e-06 | 322.82 ms | 52.3% bf16 MFU | 1624228 tok/s
step 19045/19560 | loss 3.346843 (+1.31z)| norm 0.2417 (-0.21z)| lr 1.11e-06 | 322.95 ms | 52.3% bf16 MFU | 1624188 tok/s
step 19046/19560 | loss 3.223727 (-1.21z)| norm 0.2281 (-0.77z)| lr 1.10e-06 | 322.76 ms | 52.3% bf16 MFU | 1624197 tok/s
step 19047/19560 | loss 3.296319 (+0.31z)| norm 0.2305 (-0.66z)| lr 1.10e-06 | 323.00 ms | 52.3% bf16 MFU | 1624145 tok/s
step 19048/19560 | loss 3.291643 (+0.21z)| norm 0.2514 (+0.22z)| lr 1.09e-06 | 322.79 ms | 52.3% bf16 MFU | 1624150 tok/s
step 19049/19560 | loss 3.342994 (+1.27z)| norm 0.2363 (-0.42z)| lr 1.09e-06 | 322.99 ms | 52.3% bf16 MFU | 1624104 tok/s
step 19050/19560 | loss 3.273520 (-0.19z)| norm 0.2393 (-0.30z)| lr 1.09e-06 | 323.12 ms | 52.2% bf16 MFU | 1624029 tok/s
step 19051/19560 | loss 3.309584 (+0.57z)| norm 0.2382 (-0.35z)| lr 1.08e-06 | 322.80 ms | 52.3% bf16 MFU | 1624036 tok/s
step 19052/19560 | loss 3.244061 (-0.80z)| norm 0.2195 (-1.13z)| lr 1.08e-06 | 322.74 ms | 52.3% bf16 MFU | 1624059 tok/s
step 19053/19560 | loss 3.229474 (-1.10z)| norm 0.2496 (+0.16z)| lr 1.07e-06 | 322.82 ms | 52.3% bf16 MFU | 1624061 tok/s
step 19054/19560 | loss 3.354494 (+1.49z)| norm 0.2309 (-0.63z)| lr 1.07e-06 | 323.12 ms | 52.2% bf16 MFU | 1623986 tok/s
step 19055/19560 | loss 3.281706 (-0.02z)| norm 0.2470 (+0.05z)| lr 1.06e-06 | 322.53 ms | 52.3% bf16 MFU | 1624064 tok/s
step 19056/19560 | loss 3.265389 (-0.36z)| norm 0.2360 (-0.42z)| lr 1.06e-06 | 322.49 ms | 52.3% bf16 MFU | 1624147 tok/s
step 19057/19560 | loss 3.297035 (+0.29z)| norm 0.2426 (-0.14z)| lr 1.06e-06 | 322.75 ms | 52.3% bf16 MFU | 1624162 tok/s
step 19058/19560 | loss 3.292149 (+0.18z)| norm 0.2249 (-0.89z)| lr 1.05e-06 | 322.77 ms | 52.3% bf16 MFU | 1624171 tok/s
step 19059/19560 | loss 3.302681 (+0.39z)| norm 0.2644 (+0.79z)| lr 1.05e-06 | 322.93 ms | 52.3% bf16 MFU | 1624138 tok/s
step 19060/19560 | loss 3.227076 (-1.18z)| norm 0.2267 (-0.81z)| lr 1.04e-06 | 322.64 ms | 52.3% bf16 MFU | 1624181 tok/s
step 19061/19560 | loss 3.256898 (-0.56z)| norm 0.2394 (-0.26z)| lr 1.04e-06 | 323.18 ms | 52.2% bf16 MFU | 1624085 tok/s
step 19062/19560 | loss 3.387888 (+2.15z)| norm 0.2397 (-0.25z)| lr 1.04e-06 | 323.37 ms | 52.2% bf16 MFU | 1623946 tok/s
step 19063/19560 | loss 3.252356 (-0.66z)| norm 0.2279 (-0.75z)| lr 1.03e-06 | 322.71 ms | 52.3% bf16 MFU | 1623982 tok/s
step 19064/19560 | loss 3.263747 (-0.42z)| norm 0.2841 (+1.62z)| lr 1.03e-06 | 323.01 ms | 52.2% bf16 MFU | 1623938 tok/s
step 19065/19560 | loss 3.277456 (-0.13z)| norm 0.2361 (-0.40z)| lr 1.02e-06 | 322.88 ms | 52.3% bf16 MFU | 1623931 tok/s
step 19066/19560 | loss 3.256107 (-0.56z)| norm 0.2652 (+0.83z)| lr 1.02e-06 | 322.86 ms | 52.3% bf16 MFU | 1623929 tok/s
step 19067/19560 | loss 3.287451 (+0.08z)| norm 0.2632 (+0.73z)| lr 1.02e-06 | 323.26 ms | 52.2% bf16 MFU | 1623827 tok/s
step 19068/19560 | loss 3.324170 (+0.84z)| norm 0.2412 (-0.20z)| lr 1.01e-06 | 323.36 ms | 52.2% bf16 MFU | 1623705 tok/s
step 19069/19560 | loss 3.357842 (+1.55z)| norm 0.2509 (+0.23z)| lr 1.01e-06 | 323.24 ms | 52.2% bf16 MFU | 1623618 tok/s
step 19070/19560 | loss 3.355707 (+1.48z)| norm 0.2751 (+1.25z)| lr 1.00e-06 | 322.96 ms | 52.3% bf16 MFU | 1623607 tok/s
step 19071/19560 | loss 3.264307 (-0.43z)| norm 0.2383 (-0.33z)| lr 9.99e-07 | 322.47 ms | 52.3% bf16 MFU | 1623718 tok/s
step 19072/19560 | loss 3.307116 (+0.47z)| norm 0.2285 (-0.74z)| lr 9.95e-07 | 323.41 ms | 52.2% bf16 MFU | 1623588 tok/s
step 19073/19560 | loss 3.239667 (-0.93z)| norm 0.2526 (+0.28z)| lr 9.91e-07 | 323.11 ms | 52.2% bf16 MFU | 1623539 tok/s
step 19074/19560 | loss 3.256722 (-0.57z)| norm 0.2410 (-0.22z)| lr 9.87e-07 | 322.67 ms | 52.3% bf16 MFU | 1623605 tok/s
step 19075/19560 | loss 3.283804 (-0.02z)| norm 0.2313 (-0.63z)| lr 9.83e-07 | 322.73 ms | 52.3% bf16 MFU | 1623651 tok/s
step 19076/19560 | loss 3.236456 (-1.01z)| norm 0.2390 (-0.31z)| lr 9.78e-07 | 323.40 ms | 52.2% bf16 MFU | 1623528 tok/s
step 19077/19560 | loss 3.203093 (-1.67z)| norm 0.2271 (-0.80z)| lr 9.74e-07 | 322.87 ms | 52.3% bf16 MFU | 1623542 tok/s
step 19078/19560 | loss 3.255949 (-0.58z)| norm 0.2250 (-0.88z)| lr 9.70e-07 | 323.30 ms | 52.2% bf16 MFU | 1623448 tok/s
step 19079/19560 | loss 3.276030 (-0.16z)| norm 0.2211 (-1.06z)| lr 9.66e-07 | 322.81 ms | 52.3% bf16 MFU | 1623484 tok/s
step 19080/19560 | loss 3.234917 (-1.01z)| norm 0.2319 (-0.59z)| lr 9.62e-07 | 322.22 ms | 52.4% bf16 MFU | 1623665 tok/s
step 19081/19560 | loss 3.147818 (-2.73z)| norm 0.2491 (+0.17z)| lr 9.58e-07 | 323.39 ms | 52.2% bf16 MFU | 1623544 tok/s
step 19082/19560 | loss 3.261972 (-0.40z)| norm 0.2360 (-0.41z)| lr 9.54e-07 | 323.04 ms | 52.2% bf16 MFU | 1623516 tok/s
step 19083/19560 | loss 3.252669 (-0.60z)| norm 0.2262 (-0.83z)| lr 9.50e-07 | 323.32 ms | 52.2% bf16 MFU | 1623418 tok/s
step 19084/19560 | loss 3.248785 (-0.68z)| norm 0.2258 (-0.84z)| lr 9.46e-07 | 322.89 ms | 52.3% bf16 MFU | 1623434 tok/s
step 19085/19560 | loss 3.256628 (-0.52z)| norm 0.2449 (+0.01z)| lr 9.43e-07 | 323.01 ms | 52.3% bf16 MFU | 1623420 tok/s
step 19086/19560 | loss 3.287144 (+0.12z)| norm 0.2253 (-0.85z)| lr 9.39e-07 | 322.92 ms | 52.3% bf16 MFU | 1623429 tok/s
step 19087/19560 | loss 3.262923 (-0.38z)| norm 0.2236 (-0.92z)| lr 9.35e-07 | 323.03 ms | 52.2% bf16 MFU | 1623408 tok/s
step 19088/19560 | loss 3.301624 (+0.42z)| norm 0.2228 (-0.96z)| lr 9.31e-07 | 323.15 ms | 52.2% bf16 MFU | 1623358 tok/s
step 19089/19560 | loss 3.273408 (-0.16z)| norm 0.2705 (+1.24z)| lr 9.27e-07 | 322.28 ms | 52.4% bf16 MFU | 1623531 tok/s
step 19090/19560 | loss 3.293086 (+0.24z)| norm 0.2333 (-0.47z)| lr 9.23e-07 | 322.99 ms | 52.3% bf16 MFU | 1623516 tok/s
step 19091/19560 | loss 3.269201 (-0.23z)| norm 0.2643 (+0.95z)| lr 9.19e-07 | 322.87 ms | 52.3% bf16 MFU | 1623532 tok/s
step 19092/19560 | loss 3.306929 (+0.57z)| norm 0.2611 (+0.79z)| lr 9.15e-07 | 323.22 ms | 52.2% bf16 MFU | 1623459 tok/s
step 19093/19560 | loss 3.248441 (-0.66z)| norm 0.2359 (-0.36z)| lr 9.11e-07 | 322.73 ms | 52.3% bf16 MFU | 1623512 tok/s
step 19094/19560 | loss 3.256440 (-0.48z)| norm 0.2389 (-0.23z)| lr 9.07e-07 | 322.94 ms | 52.3% bf16 MFU | 1623510 tok/s
step 19095/19560 | loss 3.210876 (-1.42z)| norm 0.2320 (-0.55z)| lr 9.03e-07 | 322.88 ms | 52.3% bf16 MFU | 1623523 tok/s
step 19096/19560 | loss 3.256052 (-0.47z)| norm 0.2542 (+0.47z)| lr 8.99e-07 | 322.83 ms | 52.3% bf16 MFU | 1623550 tok/s
step 19097/19560 | loss 3.276364 (-0.05z)| norm 0.2441 (+0.00z)| lr 8.96e-07 | 322.58 ms | 52.3% bf16 MFU | 1623636 tok/s
step 19098/19560 | loss 3.237965 (-0.85z)| norm 0.2252 (-0.86z)| lr 8.92e-07 | 322.55 ms | 52.3% bf16 MFU | 1623726 tok/s
step 19099/19560 | loss 3.246035 (-0.67z)| norm 0.2455 (+0.07z)| lr 8.88e-07 | 323.10 ms | 52.2% bf16 MFU | 1623675 tok/s
step 19100/19560 | loss 3.226894 (-1.08z)| norm 0.2473 (+0.24z)| lr 8.84e-07 | 323.14 ms | 52.2% bf16 MFU | 1623615 tok/s
step 19101/19560 | loss 3.245245 (-0.67z)| norm 0.2215 (-1.16z)| lr 8.80e-07 | 322.97 ms | 52.3% bf16 MFU | 1623602 tok/s
step 19102/19560 | loss 3.373526 (+2.07z)| norm 0.2831 (+2.15z)| lr 8.76e-07 | 322.42 ms | 52.3% bf16 MFU | 1623728 tok/s
step 19103/19560 | loss 3.280020 (+0.08z)| norm 0.2320 (-0.58z)| lr 8.73e-07 | 322.66 ms | 52.3% bf16 MFU | 1623785 tok/s
step 19104/19560 | loss 3.210567 (-1.40z)| norm 0.2765 (+1.78z)| lr 8.69e-07 | 322.53 ms | 52.3% bf16 MFU | 1623874 tok/s
step 19105/19560 | loss 3.335319 (+1.26z)| norm 0.2270 (-0.84z)| lr 8.65e-07 | 322.99 ms | 52.3% bf16 MFU | 1623842 tok/s
step 19106/19560 | loss 3.247234 (-0.61z)| norm 0.2335 (-0.50z)| lr 8.61e-07 | 322.48 ms | 52.3% bf16 MFU | 1623939 tok/s
step 19107/19560 | loss 3.327574 (+1.09z)| norm 0.2370 (-0.30z)| lr 8.57e-07 | 322.50 ms | 52.3% bf16 MFU | 1624028 tok/s
step 19108/19560 | loss 3.299099 (+0.48z)| norm 0.2460 (+0.18z)| lr 8.54e-07 | 322.70 ms | 52.3% bf16 MFU | 1624061 tok/s
step 19109/19560 | loss 3.248340 (-0.61z)| norm 0.2437 (+0.08z)| lr 8.50e-07 | 322.49 ms | 52.3% bf16 MFU | 1624146 tok/s
step 19110/19560 | loss 3.219116 (-1.24z)| norm 0.2270 (-0.83z)| lr 8.46e-07 | 323.01 ms | 52.2% bf16 MFU | 1624094 tok/s
step 19111/19560 | loss 3.352340 (+1.59z)| norm 0.2897 (+2.52z)| lr 8.42e-07 | 323.08 ms | 52.2% bf16 MFU | 1624028 tok/s
step 19112/19560 | loss 3.317619 (+0.84z)| norm 0.2530 (+0.55z)| lr 8.39e-07 | 322.80 ms | 52.3% bf16 MFU | 1624037 tok/s
step 19113/19560 | loss 3.264261 (-0.30z)| norm 0.2409 (-0.10z)| lr 8.35e-07 | 323.10 ms | 52.2% bf16 MFU | 1623969 tok/s
step 19114/19560 | loss 3.310109 (+0.68z)| norm 0.2384 (-0.24z)| lr 8.31e-07 | 322.68 ms | 52.3% bf16 MFU | 1624009 tok/s
step 19115/19560 | loss 3.297171 (+0.39z)| norm 0.2338 (-0.49z)| lr 8.28e-07 | 322.46 ms | 52.3% bf16 MFU | 1624104 tok/s
step 19116/19560 | loss 3.302969 (+0.51z)| norm 0.2276 (-0.82z)| lr 8.24e-07 | 323.12 ms | 52.2% bf16 MFU | 1624027 tok/s
step 19117/19560 | loss 3.321486 (+0.90z)| norm 0.2577 (+0.79z)| lr 8.20e-07 | 322.79 ms | 52.3% bf16 MFU | 1624038 tok/s
step 19118/19560 | loss 3.372553 (+1.96z)| norm 0.2708 (+1.46z)| lr 8.16e-07 | 322.50 ms | 52.3% bf16 MFU | 1624122 tok/s
step 19119/19560 | loss 3.334637 (+1.14z)| norm 0.2291 (-0.76z)| lr 8.13e-07 | 322.14 ms | 52.4% bf16 MFU | 1624292 tok/s
step 19120/19560 | loss 3.274260 (-0.15z)| norm 0.2488 (+0.34z)| lr 8.09e-07 | 323.44 ms | 52.2% bf16 MFU | 1624127 tok/s
step 19121/19560 | loss 3.255566 (-0.53z)| norm 0.2360 (-0.36z)| lr 8.05e-07 | 322.82 ms | 52.3% bf16 MFU | 1624126 tok/s
step 19122/19560 | loss 3.286314 (+0.12z)| norm 0.2562 (+0.78z)| lr 8.02e-07 | 322.69 ms | 52.3% bf16 MFU | 1624156 tok/s
step 19123/19560 | loss 3.324981 (+0.93z)| norm 0.2474 (+0.27z)| lr 7.98e-07 | 322.56 ms | 52.3% bf16 MFU | 1624218 tok/s
step 19124/19560 | loss 3.291421 (+0.22z)| norm 0.2306 (-0.68z)| lr 7.94e-07 | 323.07 ms | 52.2% bf16 MFU | 1624148 tok/s
step 19125/19560 | loss 3.356869 (+1.59z)| norm 0.2698 (+1.52z)| lr 7.91e-07 | 322.50 ms | 52.3% bf16 MFU | 1624225 tok/s
step 19126/19560 | loss 3.277878 (-0.09z)| norm 0.2333 (-0.54z)| lr 7.87e-07 | 322.09 ms | 52.4% bf16 MFU | 1624402 tok/s
step 19127/19560 | loss 3.333683 (+1.08z)| norm 0.2237 (-1.07z)| lr 7.84e-07 | 323.00 ms | 52.3% bf16 MFU | 1624341 tok/s
step 19128/19560 | loss 3.308987 (+0.54z)| norm 0.2303 (-0.68z)| lr 7.80e-07 | 322.20 ms | 52.4% bf16 MFU | 1624486 tok/s
step 19129/19560 | loss 3.272966 (-0.22z)| norm 0.2452 (+0.16z)| lr 7.76e-07 | 322.72 ms | 52.3% bf16 MFU | 1624490 tok/s
step 19130/19560 | loss 3.264721 (-0.39z)| norm 0.2347 (-0.44z)| lr 7.73e-07 | 323.35 ms | 52.2% bf16 MFU | 1624337 tok/s
step 19131/19560 | loss 3.221342 (-1.30z)| norm 0.3052 (+3.38z)| lr 7.69e-07 | 322.35 ms | 52.4% bf16 MFU | 1624442 tok/s
step 19132/19560 | loss 3.253765 (-0.61z)| norm 0.2369 (-0.33z)| lr 7.66e-07 | 322.39 ms | 52.4% bf16 MFU | 1624533 tok/s
step 19133/19560 | loss 3.300098 (+0.38z)| norm 0.2342 (-0.48z)| lr 7.62e-07 | 322.39 ms | 52.3% bf16 MFU | 1624618 tok/s
step 19134/19560 | loss 3.279559 (-0.06z)| norm 0.2213 (-1.16z)| lr 7.59e-07 | 322.72 ms | 52.3% bf16 MFU | 1624617 tok/s
step 19135/19560 | loss 3.303135 (+0.44z)| norm 0.2329 (-0.54z)| lr 7.55e-07 | 322.80 ms | 52.3% bf16 MFU | 1624596 tok/s
step 19136/19560 | loss 3.203981 (-1.64z)| norm 0.2300 (-0.68z)| lr 7.51e-07 | 323.18 ms | 52.2% bf16 MFU | 1624482 tok/s
step 19137/19560 | loss 3.297920 (+0.35z)| norm 0.2167 (-1.38z)| lr 7.48e-07 | 322.76 ms | 52.3% bf16 MFU | 1624476 tok/s
step 19138/19560 | loss 3.271682 (-0.21z)| norm 0.2474 (+0.29z)| lr 7.44e-07 | 322.66 ms | 52.3% bf16 MFU | 1624496 tok/s
step 19139/19560 | loss 3.281610 (+0.02z)| norm 0.2474 (+0.29z)| lr 7.41e-07 | 322.92 ms | 52.3% bf16 MFU | 1624452 tok/s
step 19140/19560 | loss 3.277777 (-0.06z)| norm 0.2298 (-0.66z)| lr 7.37e-07 | 322.77 ms | 52.3% bf16 MFU | 1624446 tok/s
step 19141/19560 | loss 3.246888 (-0.72z)| norm 0.2562 (+0.76z)| lr 7.34e-07 | 322.62 ms | 52.3% bf16 MFU | 1624477 tok/s
step 19142/19560 | loss 3.274911 (-0.12z)| norm 0.2279 (-0.78z)| lr 7.30e-07 | 322.55 ms | 52.3% bf16 MFU | 1624526 tok/s
step 19143/19560 | loss 3.330772 (+1.16z)| norm 0.2343 (-0.43z)| lr 7.27e-07 | 322.72 ms | 52.3% bf16 MFU | 1624529 tok/s
step 19144/19560 | loss 3.254549 (-0.57z)| norm 0.2332 (-0.49z)| lr 7.23e-07 | 322.34 ms | 52.4% bf16 MFU | 1624627 tok/s
step 19145/19560 | loss 3.244694 (-0.79z)| norm 0.2321 (-0.55z)| lr 7.20e-07 | 322.39 ms | 52.4% bf16 MFU | 1624708 tok/s
step 19146/19560 | loss 3.324116 (+0.99z)| norm 0.2470 (+0.26z)| lr 7.17e-07 | 322.82 ms | 52.3% bf16 MFU | 1624678 tok/s
step 19147/19560 | loss 3.304982 (+0.56z)| norm 0.2264 (-0.86z)| lr 7.13e-07 | 322.12 ms | 52.4% bf16 MFU | 1624825 tok/s
step 19148/19560 | loss 3.214737 (-1.49z)| norm 0.2222 (-1.18z)| lr 7.10e-07 | 322.41 ms | 52.3% bf16 MFU | 1624892 tok/s
step 19149/19560 | loss 3.297832 (+0.39z)| norm 0.2268 (-0.88z)| lr 7.06e-07 | 322.82 ms | 52.3% bf16 MFU | 1624852 tok/s
step 19150/19560 | loss 3.275377 (-0.13z)| norm 0.2700 (+1.78z)| lr 7.03e-07 | 322.62 ms | 52.3% bf16 MFU | 1624865 tok/s
step 19151/19560 | loss 3.348152 (+1.52z)| norm 0.2510 (+0.60z)| lr 6.99e-07 | 322.70 ms | 52.3% bf16 MFU | 1624856 tok/s
step 19152/19560 | loss 3.299596 (+0.44z)| norm 0.2263 (-0.92z)| lr 6.96e-07 | 322.57 ms | 52.3% bf16 MFU | 1624881 tok/s
step 19153/19560 | loss 3.259091 (-0.50z)| norm 0.2387 (-0.16z)| lr 6.93e-07 | 322.53 ms | 52.3% bf16 MFU | 1624914 tok/s
step 19154/19560 | loss 3.253928 (-0.61z)| norm 0.2931 (+3.05z)| lr 6.89e-07 | 322.06 ms | 52.4% bf16 MFU | 1625063 tok/s
step 19155/19560 | loss 3.280468 (+0.04z)| norm 0.2371 (-0.28z)| lr 6.86e-07 | 322.74 ms | 52.3% bf16 MFU | 1625036 tok/s
step 19156/19560 | loss 3.341593 (+1.52z)| norm 0.2406 (-0.06z)| lr 6.82e-07 | 322.56 ms | 52.3% bf16 MFU | 1625054 tok/s
step 19157/19560 | loss 3.291271 (+0.30z)| norm 0.2248 (-1.01z)| lr 6.79e-07 | 322.26 ms | 52.4% bf16 MFU | 1625146 tok/s
step 19158/19560 | loss 3.216537 (-1.49z)| norm 0.2275 (-0.84z)| lr 6.76e-07 | 322.47 ms | 52.3% bf16 MFU | 1625182 tok/s
step 19159/19560 | loss 3.289860 (+0.26z)| norm 0.2291 (-0.73z)| lr 6.72e-07 | 322.43 ms | 52.3% bf16 MFU | 1625226 tok/s
step 19160/19560 | loss 3.327106 (+1.15z)| norm 0.2231 (-1.08z)| lr 6.69e-07 | 322.59 ms | 52.3% bf16 MFU | 1625228 tok/s
step 19161/19560 | loss 3.354599 (+1.78z)| norm 0.2441 (+0.18z)| lr 6.66e-07 | 322.85 ms | 52.3% bf16 MFU | 1625162 tok/s
step 19162/19560 | loss 3.360396 (+1.88z)| norm 0.2468 (+0.35z)| lr 6.62e-07 | 322.55 ms | 52.3% bf16 MFU | 1625178 tok/s
step 19163/19560 | loss 3.262548 (-0.42z)| norm 0.2377 (-0.19z)| lr 6.59e-07 | 322.70 ms | 52.3% bf16 MFU | 1625155 tok/s
step 19164/19560 | loss 3.247655 (-0.77z)| norm 0.2385 (-0.14z)| lr 6.56e-07 | 322.49 ms | 52.3% bf16 MFU | 1625186 tok/s
step 19165/19560 | loss 3.405233 (+2.86z)| norm 0.2779 (+2.20z)| lr 6.52e-07 | 322.51 ms | 52.3% bf16 MFU | 1625208 tok/s
step 19166/19560 | loss 3.274490 (-0.16z)| norm 0.2300 (-0.67z)| lr 6.49e-07 | 322.70 ms | 52.3% bf16 MFU | 1625183 tok/s
step 19167/19560 | loss 3.334226 (+1.20z)| norm 0.2653 (+1.42z)| lr 6.46e-07 | 322.61 ms | 52.3% bf16 MFU | 1625182 tok/s
step 19168/19560 | loss 3.255370 (-0.60z)| norm 0.2320 (-0.55z)| lr 6.43e-07 | 322.55 ms | 52.3% bf16 MFU | 1625195 tok/s
step 19169/19560 | loss 3.269298 (-0.28z)| norm 0.2542 (+0.77z)| lr 6.39e-07 | 322.71 ms | 52.3% bf16 MFU | 1625166 tok/s
step 19170/19560 | loss 3.290468 (+0.20z)| norm 0.2546 (+0.78z)| lr 6.36e-07 | 322.45 ms | 52.3% bf16 MFU | 1625205 tok/s
step 19171/19560 | loss 3.266249 (-0.37z)| norm 0.2429 (+0.08z)| lr 6.33e-07 | 322.68 ms | 52.3% bf16 MFU | 1625185 tok/s
step 19172/19560 | loss 3.216163 (-1.52z)| norm 0.2308 (-0.64z)| lr 6.30e-07 | 323.20 ms | 52.2% bf16 MFU | 1625034 tok/s
step 19173/19560 | loss 3.295733 (+0.33z)| norm 0.2334 (-0.48z)| lr 6.26e-07 | 322.57 ms | 52.3% bf16 MFU | 1625050 tok/s
step 19174/19560 | loss 3.244622 (-0.87z)| norm 0.2289 (-0.75z)| lr 6.23e-07 | 322.74 ms | 52.3% bf16 MFU | 1625021 tok/s
step 19175/19560 | loss 3.337411 (+1.29z)| norm 0.2281 (-0.80z)| lr 6.20e-07 | 322.02 ms | 52.4% bf16 MFU | 1625175 tok/s
step 19176/19560 | loss 3.274814 (-0.16z)| norm 0.2456 (+0.25z)| lr 6.17e-07 | 322.48 ms | 52.3% bf16 MFU | 1625208 tok/s
step 19177/19560 | loss 3.261256 (-0.47z)| norm 0.2344 (-0.42z)| lr 6.14e-07 | 322.44 ms | 52.3% bf16 MFU | 1625247 tok/s
step 19178/19560 | loss 3.274510 (-0.16z)| norm 0.2392 (-0.13z)| lr 6.10e-07 | 323.04 ms | 52.2% bf16 MFU | 1625134 tok/s
step 19179/19560 | loss 3.322186 (+0.95z)| norm 0.2261 (-0.90z)| lr 6.07e-07 | 322.25 ms | 52.4% bf16 MFU | 1625224 tok/s
step 19180/19560 | loss 3.264317 (-0.40z)| norm 0.2674 (+1.52z)| lr 6.04e-07 | 322.52 ms | 52.3% bf16 MFU | 1625244 tok/s
step 19181/19560 | loss 3.330027 (+1.12z)| norm 0.2498 (+0.48z)| lr 6.01e-07 | 322.59 ms | 52.3% bf16 MFU | 1625243 tok/s
step 19182/19560 | loss 3.265554 (-0.38z)| norm 0.2452 (+0.20z)| lr 5.98e-07 | 322.44 ms | 52.3% bf16 MFU | 1625280 tok/s
step 19183/19560 | loss 3.243413 (-0.90z)| norm 0.2550 (+0.77z)| lr 5.94e-07 | 322.72 ms | 52.3% bf16 MFU | 1625244 tok/s
step 19184/19560 | loss 3.299675 (+0.43z)| norm 0.2371 (-0.29z)| lr 5.91e-07 | 322.28 ms | 52.4% bf16 MFU | 1625323 tok/s
step 19185/19560 | loss 3.327176 (+1.07z)| norm 0.2232 (-1.09z)| lr 5.88e-07 | 322.94 ms | 52.3% bf16 MFU | 1625231 tok/s
step 19186/19560 | loss 3.253609 (-0.66z)| norm 0.2249 (-0.99z)| lr 5.85e-07 | 322.67 ms | 52.3% bf16 MFU | 1625211 tok/s
step 19187/19560 | loss 3.275143 (-0.14z)| norm 0.2187 (-1.34z)| lr 5.82e-07 | 323.08 ms | 52.2% bf16 MFU | 1625089 tok/s
step 19188/19560 | loss 3.291121 (+0.22z)| norm 0.2264 (-0.88z)| lr 5.79e-07 | 322.83 ms | 52.3% bf16 MFU | 1625036 tok/s
step 19189/19560 | loss 3.280879 (-0.03z)| norm 0.2404 (-0.06z)| lr 5.76e-07 | 322.70 ms | 52.3% bf16 MFU | 1625018 tok/s
step 19190/19560 | loss 3.303783 (+0.55z)| norm 0.2298 (-0.67z)| lr 5.73e-07 | 322.29 ms | 52.4% bf16 MFU | 1625106 tok/s
step 19191/19560 | loss 3.368860 (+2.07z)| norm 0.2946 (+3.00z)| lr 5.70e-07 | 322.64 ms | 52.3% bf16 MFU | 1625101 tok/s
step 19192/19560 | loss 3.244087 (-0.91z)| norm 0.2329 (-0.49z)| lr 5.67e-07 | 322.26 ms | 52.4% bf16 MFU | 1625192 tok/s
step 19193/19560 | loss 3.330049 (+1.13z)| norm 0.2247 (-0.97z)| lr 5.63e-07 | 322.30 ms | 52.4% bf16 MFU | 1625267 tok/s
step 19194/19560 | loss 3.275425 (-0.17z)| norm 0.2342 (-0.40z)| lr 5.60e-07 | 322.60 ms | 52.3% bf16 MFU | 1625263 tok/s
step 19195/19560 | loss 3.306133 (+0.55z)| norm 0.2692 (+1.63z)| lr 5.57e-07 | 322.56 ms | 52.3% bf16 MFU | 1625271 tok/s
step 19196/19560 | loss 3.253018 (-0.70z)| norm 0.2355 (-0.32z)| lr 5.54e-07 | 322.67 ms | 52.3% bf16 MFU | 1625250 tok/s
step 19197/19560 | loss 3.270958 (-0.26z)| norm 0.2297 (-0.65z)| lr 5.51e-07 | 322.31 ms | 52.4% bf16 MFU | 1625321 tok/s
step 19198/19560 | loss 3.297131 (+0.39z)| norm 0.2318 (-0.52z)| lr 5.48e-07 | 323.03 ms | 52.2% bf16 MFU | 1625208 tok/s
step 19199/19560 | loss 3.326897 (+1.10z)| norm 0.2263 (-0.83z)| lr 5.45e-07 | 322.71 ms | 52.3% bf16 MFU | 1625180 tok/s
step 19200/19560 | loss 3.304204 (+0.55z)| norm 0.2456 (+0.29z)| lr 5.42e-07 | 322.86 ms | 52.3% bf16 MFU | 1625115 tok/s
step 19201/19560 | loss 3.208578 (-1.75z)| norm 0.2193 (-1.24z)| lr 5.39e-07 | 322.67 ms | 52.3% bf16 MFU | 1625101 tok/s
step 19202/19560 | loss 3.279503 (-0.05z)| norm 0.2262 (-0.82z)| lr 5.36e-07 | 322.12 ms | 52.4% bf16 MFU | 1625228 tok/s
step 19203/19560 | loss 3.355763 (+1.76z)| norm 0.2415 (+0.07z)| lr 5.33e-07 | 322.66 ms | 52.3% bf16 MFU | 1625211 tok/s
step 19204/19560 | loss 3.333178 (+1.20z)| norm 0.2430 (+0.15z)| lr 5.30e-07 | 323.23 ms | 52.2% bf16 MFU | 1625051 tok/s
step 19205/19560 | loss 3.288083 (+0.11z)| norm 0.2413 (+0.05z)| lr 5.27e-07 | 322.13 ms | 52.4% bf16 MFU | 1625178 tok/s
step 19206/19560 | loss 3.314533 (+0.74z)| norm 0.2333 (-0.43z)| lr 5.24e-07 | 322.59 ms | 52.3% bf16 MFU | 1625182 tok/s
step 19207/19560 | loss 3.298055 (+0.34z)| norm 0.2401 (-0.03z)| lr 5.21e-07 | 322.36 ms | 52.4% bf16 MFU | 1625244 tok/s
step 19208/19560 | loss 3.240735 (-1.05z)| norm 0.2281 (-0.74z)| lr 5.18e-07 | 322.62 ms | 52.3% bf16 MFU | 1625236 tok/s
step 19209/19560 | loss 3.311195 (+0.65z)| norm 0.2258 (-0.86z)| lr 5.16e-07 | 322.61 ms | 52.3% bf16 MFU | 1625233 tok/s
step 19210/19560 | loss 3.273878 (-0.29z)| norm 0.2243 (-0.95z)| lr 5.13e-07 | 322.84 ms | 52.3% bf16 MFU | 1625170 tok/s
step 19211/19560 | loss 3.268361 (-0.44z)| norm 0.2400 (-0.03z)| lr 5.10e-07 | 322.29 ms | 52.4% bf16 MFU | 1625249 tok/s
step 19212/19560 | loss 3.257576 (-0.71z)| norm 0.2498 (+0.54z)| lr 5.07e-07 | 322.42 ms | 52.3% bf16 MFU | 1625291 tok/s
step 19213/19560 | loss 3.342412 (+1.41z)| norm 0.2340 (-0.39z)| lr 5.04e-07 | 322.20 ms | 52.4% bf16 MFU | 1625387 tok/s
step 19214/19560 | loss 3.281402 (-0.12z)| norm 0.2383 (-0.14z)| lr 5.01e-07 | 322.50 ms | 52.3% bf16 MFU | 1625402 tok/s
step 19215/19560 | loss 3.257895 (-0.72z)| norm 0.2191 (-1.27z)| lr 4.98e-07 | 323.07 ms | 52.2% bf16 MFU | 1625274 tok/s
step 19216/19560 | loss 3.274011 (-0.30z)| norm 0.2462 (+0.32z)| lr 4.95e-07 | 322.51 ms | 52.3% bf16 MFU | 1625292 tok/s
step 19217/19560 | loss 3.286561 (+0.01z)| norm 0.2269 (-0.82z)| lr 4.92e-07 | 322.58 ms | 52.3% bf16 MFU | 1625294 tok/s
step 19218/19560 | loss 3.268508 (-0.44z)| norm 0.2337 (-0.41z)| lr 4.90e-07 | 322.95 ms | 52.3% bf16 MFU | 1625200 tok/s
step 19219/19560 | loss 3.325065 (+0.97z)| norm 0.2448 (+0.27z)| lr 4.87e-07 | 322.52 ms | 52.3% bf16 MFU | 1625219 tok/s
step 19220/19560 | loss 3.310286 (+0.60z)| norm 0.2297 (-0.63z)| lr 4.84e-07 | 323.06 ms | 52.2% bf16 MFU | 1625102 tok/s
step 19221/19560 | loss 3.296464 (+0.24z)| norm 0.2443 (+0.25z)| lr 4.81e-07 | 322.67 ms | 52.3% bf16 MFU | 1625090 tok/s
step 19222/19560 | loss 3.250954 (-0.90z)| norm 0.2605 (+1.21z)| lr 4.78e-07 | 322.82 ms | 52.3% bf16 MFU | 1625038 tok/s
step 19223/19560 | loss 3.345524 (+1.46z)| norm 0.2351 (-0.32z)| lr 4.75e-07 | 322.77 ms | 52.3% bf16 MFU | 1625002 tok/s
step 19224/19560 | loss 3.360904 (+1.81z)| norm 0.2342 (-0.36z)| lr 4.73e-07 | 322.45 ms | 52.3% bf16 MFU | 1625049 tok/s
step 19225/19560 | loss 3.303524 (+0.37z)| norm 0.2903 (+2.90z)| lr 4.70e-07 | 322.46 ms | 52.3% bf16 MFU | 1625093 tok/s
step 19226/19560 | loss 3.224030 (-1.61z)| norm 0.2472 (+0.38z)| lr 4.67e-07 | 323.33 ms | 52.2% bf16 MFU | 1624915 tok/s
step 19227/19560 | loss 3.222858 (-1.63z)| norm 0.2684 (+1.59z)| lr 4.64e-07 | 322.66 ms | 52.3% bf16 MFU | 1624913 tok/s
step 19228/19560 | loss 3.249607 (-0.98z)| norm 0.2366 (-0.24z)| lr 4.61e-07 | 322.26 ms | 52.4% bf16 MFU | 1625014 tok/s
step 19229/19560 | loss 3.277113 (-0.30z)| norm 0.2650 (+1.38z)| lr 4.59e-07 | 323.20 ms | 52.2% bf16 MFU | 1624872 tok/s
step 19230/19560 | loss 3.297790 (+0.24z)| norm 0.2445 (+0.22z)| lr 4.56e-07 | 322.78 ms | 52.3% bf16 MFU | 1624843 tok/s
step 19231/19560 | loss 3.295674 (+0.18z)| norm 0.2419 (+0.06z)| lr 4.53e-07 | 323.12 ms | 52.2% bf16 MFU | 1624730 tok/s
step 19232/19560 | loss 3.410510 (+3.00z)| norm 0.2904 (+2.88z)| lr 4.50e-07 | 322.75 ms | 52.3% bf16 MFU | 1624714 tok/s
step 19233/19560 | loss 3.471267 (+4.19z)| norm 0.3449 (+5.31z)| lr 4.48e-07 | 322.81 ms | 52.3% bf16 MFU | 1624686 tok/s
step 19234/19560 | loss 3.312278 (+0.48z)| norm 0.2460 (+0.20z)| lr 4.45e-07 | 322.96 ms | 52.3% bf16 MFU | 1624622 tok/s
step 19235/19560 | loss 3.384511 (+2.12z)| norm 0.2754 (+1.69z)| lr 4.42e-07 | 322.80 ms | 52.3% bf16 MFU | 1624600 tok/s
step 19236/19560 | loss 3.308131 (+0.37z)| norm 0.2793 (+1.85z)| lr 4.40e-07 | 322.93 ms | 52.3% bf16 MFU | 1624545 tok/s
step 19237/19560 | loss 3.378181 (+1.93z)| norm 0.2399 (-0.14z)| lr 4.37e-07 | 323.02 ms | 52.2% bf16 MFU | 1624472 tok/s
step 19238/19560 | loss 3.276423 (-0.40z)| norm 0.2304 (-0.62z)| lr 4.34e-07 | 322.73 ms | 52.3% bf16 MFU | 1624476 tok/s
step 19239/19560 | loss 3.301708 (+0.20z)| norm 0.2536 (+0.58z)| lr 4.31e-07 | 322.65 ms | 52.3% bf16 MFU | 1624500 tok/s
step 19240/19560 | loss 3.258559 (-0.79z)| norm 0.2454 (+0.16z)| lr 4.29e-07 | 322.77 ms | 52.3% bf16 MFU | 1624491 tok/s
step 19241/19560 | loss 3.291487 (-0.04z)| norm 0.2232 (-0.97z)| lr 4.26e-07 | 322.47 ms | 52.3% bf16 MFU | 1624560 tok/s
step 19242/19560 | loss 3.299929 (+0.16z)| norm 0.2347 (-0.38z)| lr 4.23e-07 | 322.98 ms | 52.3% bf16 MFU | 1624494 tok/s
step 19243/19560 | loss 3.288332 (-0.11z)| norm 0.2468 (+0.23z)| lr 4.21e-07 | 322.64 ms | 52.3% bf16 MFU | 1624518 tok/s
step 19244/19560 | loss 3.265860 (-0.62z)| norm 0.2290 (-0.68z)| lr 4.18e-07 | 323.50 ms | 52.2% bf16 MFU | 1624326 tok/s
step 19245/19560 | loss 3.303987 (+0.27z)| norm 0.2356 (-0.33z)| lr 4.16e-07 | 322.36 ms | 52.4% bf16 MFU | 1624429 tok/s
step 19246/19560 | loss 3.372912 (+1.86z)| norm 0.2394 (-0.13z)| lr 4.13e-07 | 323.08 ms | 52.2% bf16 MFU | 1624346 tok/s
step 19247/19560 | loss 3.288667 (-0.08z)| norm 0.2418 (-0.01z)| lr 4.10e-07 | 322.57 ms | 52.3% bf16 MFU | 1624397 tok/s
step 19248/19560 | loss 3.271493 (-0.48z)| norm 0.2298 (-0.62z)| lr 4.08e-07 | 323.06 ms | 52.2% bf16 MFU | 1624322 tok/s
step 19249/19560 | loss 3.308930 (+0.38z)| norm 0.2206 (-1.09z)| lr 4.05e-07 | 323.41 ms | 52.2% bf16 MFU | 1624161 tok/s
step 19250/19560 | loss 3.224555 (-1.56z)| norm 0.2409 (-0.03z)| lr 4.02e-07 | 323.06 ms | 52.2% bf16 MFU | 1624098 tok/s
val loss 3.274337
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3041/10042 = 0.302828
step 19251/19560 | loss 3.248930 (-0.98z)| norm 0.2272 (-0.73z)| lr 4.00e-07 | 323.01 ms | 52.3% bf16 MFU | 1624050 tok/s
step 19252/19560 | loss 3.302523 (+0.25z)| norm 0.2417 (+0.01z)| lr 3.97e-07 | 322.19 ms | 52.4% bf16 MFU | 1624211 tok/s
step 19253/19560 | loss 3.278228 (-0.29z)| norm 0.2425 (+0.07z)| lr 3.95e-07 | 322.48 ms | 52.3% bf16 MFU | 1624290 tok/s
step 19254/19560 | loss 3.285057 (-0.14z)| norm 0.2426 (+0.06z)| lr 3.92e-07 | 323.49 ms | 52.2% bf16 MFU | 1624112 tok/s
step 19255/19560 | loss 3.296559 (+0.14z)| norm 0.2344 (-0.37z)| lr 3.90e-07 | 323.10 ms | 52.2% bf16 MFU | 1624042 tok/s
step 19256/19560 | loss 3.267822 (-0.52z)| norm 0.2449 (+0.18z)| lr 3.87e-07 | 322.62 ms | 52.3% bf16 MFU | 1624095 tok/s
step 19257/19560 | loss 3.300872 (+0.24z)| norm 0.2243 (-0.89z)| lr 3.85e-07 | 322.57 ms | 52.3% bf16 MFU | 1624157 tok/s
step 19258/19560 | loss 3.283847 (-0.16z)| norm 0.2284 (-0.68z)| lr 3.82e-07 | 322.67 ms | 52.3% bf16 MFU | 1624192 tok/s
step 19259/19560 | loss 3.251215 (-0.93z)| norm 0.2361 (-0.26z)| lr 3.80e-07 | 323.31 ms | 52.2% bf16 MFU | 1624063 tok/s
step 19260/19560 | loss 3.272517 (-0.44z)| norm 0.2286 (-0.66z)| lr 3.77e-07 | 323.44 ms | 52.2% bf16 MFU | 1623909 tok/s
step 19261/19560 | loss 3.341210 (+1.17z)| norm 0.2687 (+1.50z)| lr 3.75e-07 | 322.33 ms | 52.4% bf16 MFU | 1624042 tok/s
step 19262/19560 | loss 3.254067 (-0.87z)| norm 0.2209 (-1.09z)| lr 3.72e-07 | 323.75 ms | 52.1% bf16 MFU | 1623812 tok/s
step 19263/19560 | loss 3.245092 (-1.06z)| norm 0.2428 (+0.10z)| lr 3.70e-07 | 322.74 ms | 52.3% bf16 MFU | 1623846 tok/s
step 19264/19560 | loss 3.288813 (-0.06z)| norm 0.2298 (-0.61z)| lr 3.67e-07 | 322.93 ms | 52.3% bf16 MFU | 1623830 tok/s
step 19265/19560 | loss 3.284988 (-0.15z)| norm 0.2554 (+0.77z)| lr 3.65e-07 | 323.12 ms | 52.2% bf16 MFU | 1623768 tok/s
step 19266/19560 | loss 3.356613 (+1.52z)| norm 0.2408 (-0.03z)| lr 3.62e-07 | 322.57 ms | 52.3% bf16 MFU | 1623847 tok/s
step 19267/19560 | loss 3.273652 (-0.43z)| norm 0.2254 (-0.86z)| lr 3.60e-07 | 322.51 ms | 52.3% bf16 MFU | 1623938 tok/s
step 19268/19560 | loss 3.262373 (-0.69z)| norm 0.2256 (-0.84z)| lr 3.57e-07 | 322.38 ms | 52.4% bf16 MFU | 1624057 tok/s
step 19269/19560 | loss 3.318710 (+0.62z)| norm 0.2349 (-0.33z)| lr 3.55e-07 | 323.08 ms | 52.2% bf16 MFU | 1623993 tok/s
step 19270/19560 | loss 3.288188 (-0.10z)| norm 0.2288 (-0.66z)| lr 3.52e-07 | 322.81 ms | 52.3% bf16 MFU | 1624000 tok/s
step 19271/19560 | loss 3.333214 (+0.96z)| norm 0.2757 (+1.85z)| lr 3.50e-07 | 322.40 ms | 52.3% bf16 MFU | 1624109 tok/s
step 19272/19560 | loss 3.296208 (+0.08z)| norm 0.2306 (-0.57z)| lr 3.48e-07 | 322.66 ms | 52.3% bf16 MFU | 1624148 tok/s
step 19273/19560 | loss 3.287485 (-0.13z)| norm 0.2399 (-0.07z)| lr 3.45e-07 | 323.23 ms | 52.2% bf16 MFU | 1624042 tok/s
step 19274/19560 | loss 3.295650 (+0.07z)| norm 0.2559 (+0.78z)| lr 3.43e-07 | 322.49 ms | 52.3% bf16 MFU | 1624128 tok/s
step 19275/19560 | loss 3.272704 (-0.47z)| norm 0.2420 (+0.03z)| lr 3.40e-07 | 322.76 ms | 52.3% bf16 MFU | 1624141 tok/s
step 19276/19560 | loss 3.218681 (-1.76z)| norm 0.2557 (+0.75z)| lr 3.38e-07 | 322.47 ms | 52.3% bf16 MFU | 1624226 tok/s
step 19277/19560 | loss 3.317580 (+0.59z)| norm 0.2312 (-0.57z)| lr 3.36e-07 | 323.25 ms | 52.2% bf16 MFU | 1624110 tok/s
step 19278/19560 | loss 3.263622 (-0.69z)| norm 0.2503 (+0.47z)| lr 3.33e-07 | 322.60 ms | 52.3% bf16 MFU | 1624164 tok/s
step 19279/19560 | loss 3.346094 (+1.27z)| norm 0.2398 (-0.10z)| lr 3.31e-07 | 322.97 ms | 52.3% bf16 MFU | 1624121 tok/s
step 19280/19560 | loss 3.272525 (-0.47z)| norm 0.2393 (-0.13z)| lr 3.29e-07 | 322.84 ms | 52.3% bf16 MFU | 1624115 tok/s
step 19281/19560 | loss 3.278217 (-0.34z)| norm 0.2254 (-0.88z)| lr 3.26e-07 | 323.09 ms | 52.2% bf16 MFU | 1624046 tok/s
step 19282/19560 | loss 3.299853 (+0.16z)| norm 0.2236 (-0.97z)| lr 3.24e-07 | 322.63 ms | 52.3% bf16 MFU | 1624094 tok/s
step 19283/19560 | loss 3.293855 (+0.02z)| norm 0.2750 (+1.86z)| lr 3.22e-07 | 323.17 ms | 52.2% bf16 MFU | 1624007 tok/s
step 19284/19560 | loss 3.314177 (+0.51z)| norm 0.2618 (+1.12z)| lr 3.19e-07 | 322.86 ms | 52.3% bf16 MFU | 1624002 tok/s
step 19285/19560 | loss 3.271951 (-0.50z)| norm 0.2440 (+0.13z)| lr 3.17e-07 | 322.61 ms | 52.3% bf16 MFU | 1624060 tok/s
step 19286/19560 | loss 3.359468 (+1.57z)| norm 0.2666 (+1.35z)| lr 3.15e-07 | 322.66 ms | 52.3% bf16 MFU | 1624102 tok/s
step 19287/19560 | loss 3.316138 (+0.53z)| norm 0.2409 (-0.06z)| lr 3.12e-07 | 323.15 ms | 52.2% bf16 MFU | 1624018 tok/s
step 19288/19560 | loss 3.354184 (+1.43z)| norm 0.2361 (-0.33z)| lr 3.10e-07 | 322.62 ms | 52.3% bf16 MFU | 1624072 tok/s
step 19289/19560 | loss 3.226495 (-1.59z)| norm 0.2192 (-1.25z)| lr 3.08e-07 | 322.75 ms | 52.3% bf16 MFU | 1624090 tok/s
step 19290/19560 | loss 3.255301 (-0.89z)| norm 0.2488 (+0.38z)| lr 3.06e-07 | 322.85 ms | 52.3% bf16 MFU | 1624082 tok/s
step 19291/19560 | loss 3.345230 (+1.24z)| norm 0.2347 (-0.40z)| lr 3.03e-07 | 322.20 ms | 52.4% bf16 MFU | 1624239 tok/s
step 19292/19560 | loss 3.337065 (+1.03z)| norm 0.2534 (+0.62z)| lr 3.01e-07 | 322.70 ms | 52.3% bf16 MFU | 1624262 tok/s
step 19293/19560 | loss 3.332549 (+0.96z)| norm 0.2357 (-0.33z)| lr 2.99e-07 | 322.67 ms | 52.3% bf16 MFU | 1624291 tok/s
step 19294/19560 | loss 3.320102 (+0.65z)| norm 0.2729 (+1.70z)| lr 2.97e-07 | 322.77 ms | 52.3% bf16 MFU | 1624294 tok/s
step 19295/19560 | loss 3.278474 (-0.36z)| norm 0.2221 (-1.08z)| lr 2.94e-07 | 322.81 ms | 52.3% bf16 MFU | 1624286 tok/s
step 19296/19560 | loss 3.336716 (+1.05z)| norm 0.2697 (+1.51z)| lr 2.92e-07 | 321.88 ms | 52.4% bf16 MFU | 1624512 tok/s
step 19297/19560 | loss 3.488705 (+4.37z)| norm 0.2432 (+0.07z)| lr 2.90e-07 | 322.90 ms | 52.3% bf16 MFU | 1624469 tok/s
step 19298/19560 | loss 3.233586 (-1.38z)| norm 0.2283 (-0.73z)| lr 2.88e-07 | 323.23 ms | 52.2% bf16 MFU | 1624346 tok/s
step 19299/19560 | loss 3.356093 (+1.35z)| norm 0.2506 (+0.48z)| lr 2.86e-07 | 322.42 ms | 52.3% bf16 MFU | 1624435 tok/s
step 19300/19560 | loss 3.303146 (+0.15z)| norm 0.2343 (-0.41z)| lr 2.83e-07 | 322.51 ms | 52.3% bf16 MFU | 1624497 tok/s
step 19301/19560 | loss 3.294331 (-0.05z)| norm 0.2260 (-0.86z)| lr 2.81e-07 | 322.99 ms | 52.3% bf16 MFU | 1624435 tok/s
step 19302/19560 | loss 3.329759 (+0.74z)| norm 0.2277 (-0.77z)| lr 2.79e-07 | 322.52 ms | 52.3% bf16 MFU | 1624493 tok/s
step 19303/19560 | loss 3.265265 (-0.71z)| norm 0.2366 (-0.28z)| lr 2.77e-07 | 322.60 ms | 52.3% bf16 MFU | 1624528 tok/s
step 19304/19560 | loss 3.349243 (+1.18z)| norm 0.2595 (+0.96z)| lr 2.75e-07 | 322.97 ms | 52.3% bf16 MFU | 1624469 tok/s
step 19305/19560 | loss 3.272334 (-0.56z)| norm 0.2215 (-1.10z)| lr 2.73e-07 | 321.88 ms | 52.4% bf16 MFU | 1624687 tok/s
step 19306/19560 | loss 3.339035 (+0.93z)| norm 0.2633 (+1.15z)| lr 2.71e-07 | 322.52 ms | 52.3% bf16 MFU | 1624732 tok/s
step 19307/19560 | loss 3.326389 (+0.65z)| norm 0.2483 (+0.33z)| lr 2.68e-07 | 322.85 ms | 52.3% bf16 MFU | 1624693 tok/s
step 19308/19560 | loss 3.239887 (-1.30z)| norm 0.2517 (+0.53z)| lr 2.66e-07 | 322.27 ms | 52.4% bf16 MFU | 1624801 tok/s
step 19309/19560 | loss 3.290012 (-0.16z)| norm 0.2586 (+0.90z)| lr 2.64e-07 | 322.73 ms | 52.3% bf16 MFU | 1624787 tok/s
step 19310/19560 | loss 3.229684 (-1.51z)| norm 0.2256 (-0.89z)| lr 2.62e-07 | 323.16 ms | 52.2% bf16 MFU | 1624666 tok/s
step 19311/19560 | loss 3.276692 (-0.46z)| norm 0.2206 (-1.14z)| lr 2.60e-07 | 322.74 ms | 52.3% bf16 MFU | 1624658 tok/s
step 19312/19560 | loss 3.251194 (-1.02z)| norm 0.2497 (+0.43z)| lr 2.58e-07 | 322.45 ms | 52.3% bf16 MFU | 1624722 tok/s
step 19313/19560 | loss 3.250871 (-1.01z)| norm 0.2559 (+0.75z)| lr 2.56e-07 | 322.19 ms | 52.4% bf16 MFU | 1624850 tok/s
step 19314/19560 | loss 3.337910 (+0.92z)| norm 0.2324 (-0.53z)| lr 2.54e-07 | 322.45 ms | 52.3% bf16 MFU | 1624906 tok/s
step 19315/19560 | loss 3.266329 (-0.68z)| norm 0.2269 (-0.83z)| lr 2.52e-07 | 322.13 ms | 52.4% bf16 MFU | 1625038 tok/s
step 19316/19560 | loss 3.290651 (-0.14z)| norm 0.2392 (-0.17z)| lr 2.50e-07 | 322.91 ms | 52.3% bf16 MFU | 1624968 tok/s
step 19317/19560 | loss 3.301366 (+0.10z)| norm 0.2353 (-0.38z)| lr 2.48e-07 | 322.93 ms | 52.3% bf16 MFU | 1624896 tok/s
step 19318/19560 | loss 3.312407 (+0.34z)| norm 0.2438 (+0.08z)| lr 2.46e-07 | 322.51 ms | 52.3% bf16 MFU | 1624932 tok/s
step 19319/19560 | loss 3.310884 (+0.32z)| norm 0.2264 (-0.87z)| lr 2.44e-07 | 322.89 ms | 52.3% bf16 MFU | 1624873 tok/s
step 19320/19560 | loss 3.288950 (-0.18z)| norm 0.2193 (-1.25z)| lr 2.42e-07 | 322.49 ms | 52.3% bf16 MFU | 1624916 tok/s
step 19321/19560 | loss 3.248623 (-1.08z)| norm 0.2287 (-0.73z)| lr 2.40e-07 | 322.46 ms | 52.3% bf16 MFU | 1624965 tok/s
step 19322/19560 | loss 3.337935 (+0.93z)| norm 0.2635 (+1.20z)| lr 2.38e-07 | 322.38 ms | 52.4% bf16 MFU | 1625033 tok/s
step 19323/19560 | loss 3.248996 (-1.06z)| norm 0.2209 (-1.16z)| lr 2.36e-07 | 322.97 ms | 52.3% bf16 MFU | 1624948 tok/s
step 19324/19560 | loss 3.275689 (-0.47z)| norm 0.2556 (+0.78z)| lr 2.34e-07 | 322.86 ms | 52.3% bf16 MFU | 1624896 tok/s
step 19325/19560 | loss 3.316733 (+0.45z)| norm 0.2467 (+0.27z)| lr 2.32e-07 | 322.53 ms | 52.3% bf16 MFU | 1624929 tok/s
step 19326/19560 | loss 3.275325 (-0.48z)| norm 0.2417 (-0.02z)| lr 2.30e-07 | 322.64 ms | 52.3% bf16 MFU | 1624931 tok/s
step 19327/19560 | loss 3.257483 (-0.87z)| norm 0.2353 (-0.37z)| lr 2.28e-07 | 322.56 ms | 52.3% bf16 MFU | 1624955 tok/s
step 19328/19560 | loss 3.358836 (+1.39z)| norm 0.2488 (+0.38z)| lr 2.26e-07 | 322.34 ms | 52.4% bf16 MFU | 1625033 tok/s
step 19329/19560 | loss 3.320922 (+0.53z)| norm 0.2408 (-0.08z)| lr 2.24e-07 | 322.82 ms | 52.3% bf16 MFU | 1624984 tok/s
step 19330/19560 | loss 3.324433 (+0.60z)| norm 0.2325 (-0.55z)| lr 2.22e-07 | 322.53 ms | 52.3% bf16 MFU | 1625013 tok/s
step 19331/19560 | loss 3.335994 (+0.87z)| norm 0.2665 (+1.35z)| lr 2.20e-07 | 322.76 ms | 52.3% bf16 MFU | 1624982 tok/s
step 19332/19560 | loss 3.241354 (-1.26z)| norm 0.2404 (-0.12z)| lr 2.18e-07 | 322.70 ms | 52.3% bf16 MFU | 1624966 tok/s
step 19333/19560 | loss 3.209420 (-1.94z)| norm 0.2798 (+2.05z)| lr 2.16e-07 | 322.53 ms | 52.3% bf16 MFU | 1624995 tok/s
step 19334/19560 | loss 3.302655 (+0.14z)| norm 0.2408 (-0.11z)| lr 2.14e-07 | 322.58 ms | 52.3% bf16 MFU | 1625011 tok/s
step 19335/19560 | loss 3.311715 (+0.34z)| norm 0.2304 (-0.68z)| lr 2.13e-07 | 322.52 ms | 52.3% bf16 MFU | 1625040 tok/s
step 19336/19560 | loss 3.239522 (-1.27z)| norm 0.2283 (-0.80z)| lr 2.11e-07 | 323.06 ms | 52.2% bf16 MFU | 1624932 tok/s
step 19337/19560 | loss 3.339724 (+0.96z)| norm 0.2421 (-0.04z)| lr 2.09e-07 | 322.69 ms | 52.3% bf16 MFU | 1624922 tok/s
step 19338/19560 | loss 3.266980 (-0.66z)| norm 0.2514 (+0.47z)| lr 2.07e-07 | 322.38 ms | 52.4% bf16 MFU | 1624991 tok/s
step 19339/19560 | loss 3.373092 (+1.67z)| norm 0.2317 (-0.63z)| lr 2.05e-07 | 322.46 ms | 52.3% bf16 MFU | 1625035 tok/s
step 19340/19560 | loss 3.279072 (-0.41z)| norm 0.2625 (+1.07z)| lr 2.03e-07 | 322.82 ms | 52.3% bf16 MFU | 1624989 tok/s
step 19341/19560 | loss 3.337915 (+0.89z)| norm 0.2397 (-0.19z)| lr 2.01e-07 | 322.60 ms | 52.3% bf16 MFU | 1624999 tok/s
step 19342/19560 | loss 3.308337 (+0.24z)| norm 0.2289 (-0.78z)| lr 2.00e-07 | 322.80 ms | 52.3% bf16 MFU | 1624959 tok/s
step 19343/19560 | loss 3.241470 (-1.23z)| norm 0.2268 (-0.91z)| lr 1.98e-07 | 323.03 ms | 52.2% bf16 MFU | 1624864 tok/s
step 19344/19560 | loss 3.332412 (+0.76z)| norm 0.2374 (-0.31z)| lr 1.96e-07 | 322.64 ms | 52.3% bf16 MFU | 1624870 tok/s
step 19345/19560 | loss 3.338848 (+0.89z)| norm 0.2296 (-0.75z)| lr 1.94e-07 | 323.14 ms | 52.2% bf16 MFU | 1624752 tok/s
step 19346/19560 | loss 3.345453 (+1.02z)| norm 0.2481 (+0.28z)| lr 1.92e-07 | 322.76 ms | 52.3% bf16 MFU | 1624733 tok/s
step 19347/19560 | loss 3.376701 (+1.67z)| norm 0.2333 (-0.54z)| lr 1.91e-07 | 322.40 ms | 52.3% bf16 MFU | 1624806 tok/s
step 19348/19560 | loss 3.280707 (-0.40z)| norm 0.2300 (-0.73z)| lr 1.89e-07 | 322.49 ms | 52.3% bf16 MFU | 1624854 tok/s
step 19349/19560 | loss 3.287130 (-0.26z)| norm 0.2276 (-0.85z)| lr 1.87e-07 | 323.11 ms | 52.2% bf16 MFU | 1624742 tok/s
step 19350/19560 | loss 3.286341 (-0.28z)| norm 0.2432 (+0.02z)| lr 1.85e-07 | 322.30 ms | 52.4% bf16 MFU | 1624840 tok/s
step 19351/19560 | loss 3.315761 (+0.36z)| norm 0.2318 (-0.61z)| lr 1.84e-07 | 322.13 ms | 52.4% bf16 MFU | 1624976 tok/s
step 19352/19560 | loss 3.292512 (-0.13z)| norm 0.2537 (+0.60z)| lr 1.82e-07 | 322.78 ms | 52.3% bf16 MFU | 1624941 tok/s
step 19353/19560 | loss 3.277290 (-0.46z)| norm 0.2371 (-0.31z)| lr 1.80e-07 | 322.81 ms | 52.3% bf16 MFU | 1624902 tok/s
step 19354/19560 | loss 3.293319 (-0.13z)| norm 0.2455 (+0.17z)| lr 1.78e-07 | 322.44 ms | 52.3% bf16 MFU | 1624956 tok/s
step 19355/19560 | loss 3.306525 (+0.15z)| norm 0.2412 (-0.06z)| lr 1.77e-07 | 322.64 ms | 52.3% bf16 MFU | 1624959 tok/s
step 19356/19560 | loss 3.321712 (+0.48z)| norm 0.2239 (-1.05z)| lr 1.75e-07 | 322.43 ms | 52.3% bf16 MFU | 1625014 tok/s
step 19357/19560 | loss 3.315708 (+0.34z)| norm 0.2241 (-1.02z)| lr 1.73e-07 | 322.76 ms | 52.3% bf16 MFU | 1624983 tok/s
step 19358/19560 | loss 3.307049 (+0.15z)| norm 0.2318 (-0.58z)| lr 1.72e-07 | 322.22 ms | 52.4% bf16 MFU | 1625089 tok/s
step 19359/19560 | loss 3.249009 (-1.15z)| norm 0.2290 (-0.73z)| lr 1.70e-07 | 322.42 ms | 52.3% bf16 MFU | 1625139 tok/s
step 19360/19560 | loss 3.294883 (-0.10z)| norm 0.2292 (-0.71z)| lr 1.68e-07 | 322.23 ms | 52.4% bf16 MFU | 1625236 tok/s
step 19361/19560 | loss 3.339671 (+1.01z)| norm 0.2376 (-0.20z)| lr 1.66e-07 | 322.69 ms | 52.3% bf16 MFU | 1625212 tok/s
step 19362/19560 | loss 3.251104 (-1.14z)| norm 0.2293 (-0.77z)| lr 1.65e-07 | 323.09 ms | 52.2% bf16 MFU | 1625088 tok/s
step 19363/19560 | loss 3.296405 (-0.02z)| norm 0.2249 (-1.07z)| lr 1.63e-07 | 322.97 ms | 52.3% bf16 MFU | 1624999 tok/s
step 19364/19560 | loss 3.298488 (+0.04z)| norm 0.2225 (-1.24z)| lr 1.62e-07 | 323.24 ms | 52.2% bf16 MFU | 1624847 tok/s
step 19365/19560 | loss 3.227999 (-1.69z)| norm 0.2282 (-0.81z)| lr 1.60e-07 | 322.85 ms | 52.3% bf16 MFU | 1624803 tok/s
step 19366/19560 | loss 3.272878 (-0.57z)| norm 0.2548 (+1.12z)| lr 1.58e-07 | 322.25 ms | 52.4% bf16 MFU | 1624910 tok/s
step 19367/19560 | loss 3.284221 (-0.29z)| norm 0.2252 (-1.03z)| lr 1.57e-07 | 322.64 ms | 52.3% bf16 MFU | 1624916 tok/s
step 19368/19560 | loss 3.262299 (-0.83z)| norm 0.2418 (+0.18z)| lr 1.55e-07 | 323.02 ms | 52.2% bf16 MFU | 1624823 tok/s
step 19369/19560 | loss 3.282937 (-0.32z)| norm 0.2259 (-0.98z)| lr 1.53e-07 | 322.86 ms | 52.3% bf16 MFU | 1624776 tok/s
step 19370/19560 | loss 3.249227 (-1.14z)| norm 0.2266 (-0.92z)| lr 1.52e-07 | 322.42 ms | 52.3% bf16 MFU | 1624843 tok/s
step 19371/19560 | loss 3.311473 (+0.40z)| norm 0.2802 (+2.88z)| lr 1.50e-07 | 322.92 ms | 52.3% bf16 MFU | 1624780 tok/s
step 19372/19560 | loss 3.344348 (+1.19z)| norm 0.2685 (+2.00z)| lr 1.49e-07 | 322.42 ms | 52.3% bf16 MFU | 1624845 tok/s
step 19373/19560 | loss 3.339947 (+1.07z)| norm 0.2435 (+0.25z)| lr 1.47e-07 | 322.61 ms | 52.3% bf16 MFU | 1624859 tok/s
step 19374/19560 | loss 3.272487 (-0.57z)| norm 0.2246 (-1.05z)| lr 1.46e-07 | 322.96 ms | 52.3% bf16 MFU | 1624784 tok/s
step 19375/19560 | loss 3.270739 (-0.61z)| norm 0.2337 (-0.41z)| lr 1.44e-07 | 322.61 ms | 52.3% bf16 MFU | 1624803 tok/s
step 19376/19560 | loss 3.291270 (-0.11z)| norm 0.2239 (-1.09z)| lr 1.42e-07 | 322.92 ms | 52.3% bf16 MFU | 1624742 tok/s
step 19377/19560 | loss 3.289497 (-0.15z)| norm 0.2286 (-0.77z)| lr 1.41e-07 | 322.37 ms | 52.4% bf16 MFU | 1624823 tok/s
step 19378/19560 | loss 3.301518 (+0.14z)| norm 0.2290 (-0.74z)| lr 1.39e-07 | 322.73 ms | 52.3% bf16 MFU | 1624809 tok/s
step 19379/19560 | loss 3.298518 (+0.05z)| norm 0.2243 (-1.06z)| lr 1.38e-07 | 322.77 ms | 52.3% bf16 MFU | 1624786 tok/s
step 19380/19560 | loss 3.302499 (+0.15z)| norm 0.2231 (-1.13z)| lr 1.36e-07 | 322.44 ms | 52.3% bf16 MFU | 1624847 tok/s
step 19381/19560 | loss 3.490825 (+4.48z)| norm 0.5067 (+9.59z)| lr 1.35e-07 | 322.81 ms | 52.3% bf16 MFU | 1624811 tok/s
step 19382/19560 | loss 3.228394 (-1.59z)| norm 0.2344 (-0.25z)| lr 1.33e-07 | 323.01 ms | 52.2% bf16 MFU | 1624727 tok/s
step 19383/19560 | loss 3.298752 (+0.03z)| norm 0.2616 (+0.72z)| lr 1.32e-07 | 323.01 ms | 52.2% bf16 MFU | 1624648 tok/s
step 19384/19560 | loss 3.332369 (+0.79z)| norm 0.2427 (+0.04z)| lr 1.30e-07 | 323.39 ms | 52.2% bf16 MFU | 1624477 tok/s
step 19385/19560 | loss 3.317844 (+0.45z)| norm 0.2266 (-0.54z)| lr 1.29e-07 | 322.51 ms | 52.3% bf16 MFU | 1624537 tok/s
step 19386/19560 | loss 3.371939 (+1.66z)| norm 0.2546 (+0.46z)| lr 1.27e-07 | 322.63 ms | 52.3% bf16 MFU | 1624563 tok/s
step 19387/19560 | loss 3.220060 (-1.78z)| norm 0.2706 (+1.03z)| lr 1.26e-07 | 322.23 ms | 52.4% bf16 MFU | 1624688 tok/s
step 19388/19560 | loss 3.258910 (-0.89z)| norm 0.2250 (-0.61z)| lr 1.25e-07 | 322.59 ms | 52.3% bf16 MFU | 1624716 tok/s
step 19389/19560 | loss 3.273052 (-0.57z)| norm 0.2253 (-0.59z)| lr 1.23e-07 | 322.82 ms | 52.3% bf16 MFU | 1624684 tok/s
step 19390/19560 | loss 3.297350 (-0.02z)| norm 0.2314 (-0.38z)| lr 1.22e-07 | 322.55 ms | 52.3% bf16 MFU | 1624722 tok/s
step 19391/19560 | loss 3.297080 (-0.04z)| norm 0.2389 (-0.11z)| lr 1.20e-07 | 322.79 ms | 52.3% bf16 MFU | 1624698 tok/s
step 19392/19560 | loss 3.268903 (-0.68z)| norm 0.2333 (-0.31z)| lr 1.19e-07 | 322.41 ms | 52.3% bf16 MFU | 1624772 tok/s
step 19393/19560 | loss 3.308504 (+0.22z)| norm 0.2738 (+1.14z)| lr 1.17e-07 | 322.52 ms | 52.3% bf16 MFU | 1624813 tok/s
step 19394/19560 | loss 3.263911 (-0.78z)| norm 0.2353 (-0.24z)| lr 1.16e-07 | 322.42 ms | 52.3% bf16 MFU | 1624878 tok/s
step 19395/19560 | loss 3.234714 (-1.43z)| norm 0.2164 (-0.91z)| lr 1.15e-07 | 322.98 ms | 52.3% bf16 MFU | 1624799 tok/s
step 19396/19560 | loss 3.258687 (-0.89z)| norm 0.2720 (+1.07z)| lr 1.13e-07 | 322.92 ms | 52.3% bf16 MFU | 1624738 tok/s
step 19397/19560 | loss 3.295298 (-0.05z)| norm 0.2248 (-0.62z)| lr 1.12e-07 | 322.25 ms | 52.4% bf16 MFU | 1624848 tok/s
step 19398/19560 | loss 3.296153 (-0.03z)| norm 0.2535 (+0.40z)| lr 1.11e-07 | 322.76 ms | 52.3% bf16 MFU | 1624826 tok/s
step 19399/19560 | loss 3.463237 (+3.56z)| norm 0.2638 (+0.77z)| lr 1.09e-07 | 323.30 ms | 52.2% bf16 MFU | 1624669 tok/s
step 19400/19560 | loss 3.270394 (-0.61z)| norm 0.2635 (+0.75z)| lr 1.08e-07 | 322.25 ms | 52.4% bf16 MFU | 1624784 tok/s
step 19401/19560 | loss 3.372547 (+1.57z)| norm 0.2283 (-0.50z)| lr 1.07e-07 | 322.44 ms | 52.3% bf16 MFU | 1624845 tok/s
step 19402/19560 | loss 3.291061 (-0.17z)| norm 0.2245 (-0.63z)| lr 1.05e-07 | 323.25 ms | 52.2% bf16 MFU | 1624698 tok/s
step 19403/19560 | loss 3.318369 (+0.40z)| norm 0.2268 (-0.54z)| lr 1.04e-07 | 322.47 ms | 52.3% bf16 MFU | 1624754 tok/s
step 19404/19560 | loss 3.255044 (-0.96z)| norm 0.2695 (+0.97z)| lr 1.03e-07 | 322.63 ms | 52.3% bf16 MFU | 1624769 tok/s
step 19405/19560 | loss 3.320327 (+0.44z)| norm 0.2267 (-0.55z)| lr 1.01e-07 | 322.76 ms | 52.3% bf16 MFU | 1624749 tok/s
step 19406/19560 | loss 3.312657 (+0.27z)| norm 0.2427 (+0.02z)| lr 1.00e-07 | 322.69 ms | 52.3% bf16 MFU | 1624749 tok/s
step 19407/19560 | loss 3.336913 (+0.80z)| norm 0.2309 (-0.39z)| lr 9.87e-08 | 322.75 ms | 52.3% bf16 MFU | 1624732 tok/s
step 19408/19560 | loss 3.321363 (+0.45z)| norm 0.2238 (-0.64z)| lr 9.74e-08 | 322.87 ms | 52.3% bf16 MFU | 1624687 tok/s
step 19409/19560 | loss 3.230602 (-1.49z)| norm 0.2266 (-0.54z)| lr 9.61e-08 | 323.33 ms | 52.2% bf16 MFU | 1624530 tok/s
step 19410/19560 | loss 3.319440 (+0.41z)| norm 0.2269 (-0.53z)| lr 9.49e-08 | 322.73 ms | 52.3% bf16 MFU | 1624531 tok/s
step 19411/19560 | loss 3.219660 (-1.70z)| norm 0.2465 (+0.17z)| lr 9.36e-08 | 322.81 ms | 52.3% bf16 MFU | 1624511 tok/s
step 19412/19560 | loss 3.301674 (+0.04z)| norm 0.2324 (-0.32z)| lr 9.24e-08 | 323.38 ms | 52.2% bf16 MFU | 1624350 tok/s
step 19413/19560 | loss 3.251629 (-1.01z)| norm 0.2321 (-0.33z)| lr 9.12e-08 | 322.88 ms | 52.3% bf16 MFU | 1624321 tok/s
step 19414/19560 | loss 3.277214 (-0.46z)| norm 0.2340 (-0.25z)| lr 8.99e-08 | 322.79 ms | 52.3% bf16 MFU | 1624316 tok/s
step 19415/19560 | loss 3.236461 (-1.31z)| norm 0.2348 (-0.22z)| lr 8.87e-08 | 322.63 ms | 52.3% bf16 MFU | 1624352 tok/s
step 19416/19560 | loss 3.300826 (+0.07z)| norm 0.2719 (+1.09z)| lr 8.75e-08 | 323.46 ms | 52.2% bf16 MFU | 1624178 tok/s
step 19417/19560 | loss 3.260288 (-0.81z)| norm 0.2478 (+0.22z)| lr 8.63e-08 | 322.55 ms | 52.3% bf16 MFU | 1624240 tok/s
step 19418/19560 | loss 3.282933 (-0.33z)| norm 0.2279 (-0.48z)| lr 8.51e-08 | 322.76 ms | 52.3% bf16 MFU | 1624247 tok/s
step 19419/19560 | loss 3.323429 (+0.55z)| norm 0.2308 (-0.38z)| lr 8.39e-08 | 323.03 ms | 52.2% bf16 MFU | 1624185 tok/s
step 19420/19560 | loss 3.239364 (-1.24z)| norm 0.2173 (-0.85z)| lr 8.27e-08 | 323.51 ms | 52.2% bf16 MFU | 1624006 tok/s
step 19421/19560 | loss 3.243805 (-1.13z)| norm 0.2396 (-0.06z)| lr 8.16e-08 | 322.85 ms | 52.3% bf16 MFU | 1624002 tok/s
step 19422/19560 | loss 3.284670 (-0.25z)| norm 0.2283 (-0.45z)| lr 8.04e-08 | 322.81 ms | 52.3% bf16 MFU | 1624009 tok/s
step 19423/19560 | loss 3.205769 (-1.90z)| norm 0.2414 (+0.02z)| lr 7.93e-08 | 322.84 ms | 52.3% bf16 MFU | 1624009 tok/s
step 19424/19560 | loss 3.260849 (-0.73z)| norm 0.2350 (-0.20z)| lr 7.81e-08 | 322.80 ms | 52.3% bf16 MFU | 1624019 tok/s
step 19425/19560 | loss 3.328660 (+0.79z)| norm 0.2454 (+0.17z)| lr 7.70e-08 | 322.89 ms | 52.3% bf16 MFU | 1624005 tok/s
step 19426/19560 | loss 3.262365 (-0.72z)| norm 0.2470 (+0.22z)| lr 7.59e-08 | 322.70 ms | 52.3% bf16 MFU | 1624039 tok/s
step 19427/19560 | loss 3.330067 (+0.83z)| norm 0.2322 (-0.31z)| lr 7.47e-08 | 322.78 ms | 52.3% bf16 MFU | 1624052 tok/s
step 19428/19560 | loss 3.247211 (-1.05z)| norm 0.2379 (-0.10z)| lr 7.36e-08 | 322.94 ms | 52.3% bf16 MFU | 1624024 tok/s
step 19429/19560 | loss 3.359724 (+1.48z)| norm 0.2613 (+0.73z)| lr 7.25e-08 | 322.41 ms | 52.3% bf16 MFU | 1624129 tok/s
step 19430/19560 | loss 3.282366 (-0.25z)| norm 0.2552 (+0.50z)| lr 7.14e-08 | 322.51 ms | 52.3% bf16 MFU | 1624205 tok/s
step 19431/19560 | loss 3.321820 (+0.63z)| norm 0.2441 (+0.10z)| lr 7.03e-08 | 322.71 ms | 52.3% bf16 MFU | 1624227 tok/s
step 19432/19560 | loss 3.302001 (+0.19z)| norm 0.2765 (+1.26z)| lr 6.93e-08 | 322.72 ms | 52.3% bf16 MFU | 1624246 tok/s
step 19433/19560 | loss 3.300609 (+0.15z)| norm 0.2248 (-0.59z)| lr 6.82e-08 | 322.56 ms | 52.3% bf16 MFU | 1624303 tok/s
step 19434/19560 | loss 3.267169 (-0.60z)| norm 0.2371 (-0.15z)| lr 6.71e-08 | 323.39 ms | 52.2% bf16 MFU | 1624149 tok/s
step 19435/19560 | loss 3.271062 (-0.50z)| norm 0.2227 (-0.66z)| lr 6.61e-08 | 322.73 ms | 52.3% bf16 MFU | 1624169 tok/s
step 19436/19560 | loss 3.304510 (+0.26z)| norm 0.2355 (-0.19z)| lr 6.50e-08 | 322.55 ms | 52.3% bf16 MFU | 1624234 tok/s
step 19437/19560 | loss 3.249028 (-1.01z)| norm 0.2412 (+0.02z)| lr 6.40e-08 | 322.44 ms | 52.3% bf16 MFU | 1624323 tok/s
step 19438/19560 | loss 3.312217 (+0.43z)| norm 0.2575 (+0.59z)| lr 6.30e-08 | 322.64 ms | 52.3% bf16 MFU | 1624355 tok/s
step 19439/19560 | loss 3.255569 (-0.88z)| norm 0.2282 (-0.46z)| lr 6.19e-08 | 323.43 ms | 52.2% bf16 MFU | 1624189 tok/s
step 19440/19560 | loss 3.271836 (-0.51z)| norm 0.2253 (-0.56z)| lr 6.09e-08 | 322.59 ms | 52.3% bf16 MFU | 1624241 tok/s
step 19441/19560 | loss 3.319942 (+0.59z)| norm 0.2317 (-0.32z)| lr 5.99e-08 | 322.48 ms | 52.3% bf16 MFU | 1624318 tok/s
step 19442/19560 | loss 3.312606 (+0.43z)| norm 0.2804 (+1.41z)| lr 5.89e-08 | 322.45 ms | 52.3% bf16 MFU | 1624400 tok/s
step 19443/19560 | loss 3.242254 (-1.20z)| norm 0.2350 (-0.22z)| lr 5.80e-08 | 322.71 ms | 52.3% bf16 MFU | 1624411 tok/s
step 19444/19560 | loss 3.297286 (+0.08z)| norm 0.2355 (-0.20z)| lr 5.70e-08 | 324.23 ms | 52.1% bf16 MFU | 1624042 tok/s
step 19445/19560 | loss 3.362718 (+1.57z)| norm 0.2337 (-0.26z)| lr 5.60e-08 | 322.94 ms | 52.3% bf16 MFU | 1624016 tok/s
step 19446/19560 | loss 3.370044 (+1.70z)| norm 0.2497 (+0.31z)| lr 5.50e-08 | 322.79 ms | 52.3% bf16 MFU | 1624028 tok/s
step 19447/19560 | loss 3.289576 (-0.12z)| norm 0.2620 (+0.74z)| lr 5.41e-08 | 322.84 ms | 52.3% bf16 MFU | 1624026 tok/s
step 19448/19560 | loss 3.445710 (+3.25z)| norm 0.3247 (+2.86z)| lr 5.31e-08 | 323.71 ms | 52.1% bf16 MFU | 1623806 tok/s
step 19449/19560 | loss 3.247884 (-1.04z)| norm 0.2235 (-0.65z)| lr 5.22e-08 | 323.52 ms | 52.2% bf16 MFU | 1623643 tok/s
step 19450/19560 | loss 3.319474 (+0.52z)| norm 0.2294 (-0.43z)| lr 5.13e-08 | 322.45 ms | 52.3% bf16 MFU | 1623758 tok/s
step 19451/19560 | loss 3.304584 (+0.18z)| norm 0.2261 (-0.55z)| lr 5.04e-08 | 322.76 ms | 52.3% bf16 MFU | 1623791 tok/s
step 19452/19560 | loss 3.339195 (+0.93z)| norm 0.2299 (-0.41z)| lr 4.94e-08 | 322.63 ms | 52.3% bf16 MFU | 1623855 tok/s
step 19453/19560 | loss 3.277215 (-0.42z)| norm 0.2302 (-0.40z)| lr 4.85e-08 | 322.71 ms | 52.3% bf16 MFU | 1623894 tok/s
step 19454/19560 | loss 3.354035 (+1.24z)| norm 0.2891 (+1.62z)| lr 4.77e-08 | 322.36 ms | 52.4% bf16 MFU | 1624018 tok/s
step 19455/19560 | loss 3.332899 (+0.77z)| norm 0.2319 (-0.34z)| lr 4.68e-08 | 323.18 ms | 52.2% bf16 MFU | 1623931 tok/s
step 19456/19560 | loss 3.296911 (-0.00z)| norm 0.2401 (-0.06z)| lr 4.59e-08 | 322.65 ms | 52.3% bf16 MFU | 1623982 tok/s
step 19457/19560 | loss 3.270682 (-0.57z)| norm 0.2279 (-0.48z)| lr 4.50e-08 | 322.67 ms | 52.3% bf16 MFU | 1624025 tok/s
step 19458/19560 | loss 3.249633 (-1.01z)| norm 0.2278 (-0.48z)| lr 4.41e-08 | 322.85 ms | 52.3% bf16 MFU | 1624021 tok/s
step 19459/19560 | loss 3.354621 (+1.27z)| norm 0.2241 (-0.59z)| lr 4.33e-08 | 322.84 ms | 52.3% bf16 MFU | 1624020 tok/s
step 19460/19560 | loss 3.311810 (+0.33z)| norm 0.2528 (+0.39z)| lr 4.25e-08 | 322.68 ms | 52.3% bf16 MFU | 1624058 tok/s
step 19461/19560 | loss 3.232205 (-1.42z)| norm 0.2358 (-0.18z)| lr 4.16e-08 | 322.39 ms | 52.4% bf16 MFU | 1624169 tok/s
step 19462/19560 | loss 3.263106 (-0.74z)| norm 0.2236 (-0.60z)| lr 4.08e-08 | 323.18 ms | 52.2% bf16 MFU | 1624074 tok/s
step 19463/19560 | loss 3.283748 (-0.28z)| norm 0.2328 (-0.28z)| lr 4.00e-08 | 322.44 ms | 52.3% bf16 MFU | 1624171 tok/s
step 19464/19560 | loss 3.277028 (-0.44z)| norm 0.2331 (-0.28z)| lr 3.92e-08 | 323.32 ms | 52.2% bf16 MFU | 1624042 tok/s
step 19465/19560 | loss 3.282331 (-0.31z)| norm 0.2381 (-0.10z)| lr 3.84e-08 | 323.22 ms | 52.2% bf16 MFU | 1623944 tok/s
step 19466/19560 | loss 3.234975 (-1.34z)| norm 0.2203 (-0.71z)| lr 3.76e-08 | 323.07 ms | 52.2% bf16 MFU | 1623888 tok/s
step 19467/19560 | loss 3.310495 (+0.33z)| norm 0.2458 (+0.17z)| lr 3.68e-08 | 322.92 ms | 52.3% bf16 MFU | 1623874 tok/s
step 19468/19560 | loss 3.225099 (-1.55z)| norm 0.2346 (-0.21z)| lr 3.60e-08 | 322.73 ms | 52.3% bf16 MFU | 1623907 tok/s
step 19469/19560 | loss 3.329664 (+0.76z)| norm 0.2698 (+0.99z)| lr 3.52e-08 | 323.14 ms | 52.2% bf16 MFU | 1623835 tok/s
step 19470/19560 | loss 3.255013 (-0.87z)| norm 0.2329 (-0.28z)| lr 3.45e-08 | 322.98 ms | 52.3% bf16 MFU | 1623807 tok/s
step 19471/19560 | loss 3.351332 (+1.23z)| norm 0.2359 (-0.18z)| lr 3.37e-08 | 323.00 ms | 52.3% bf16 MFU | 1623776 tok/s
step 19472/19560 | loss 3.314158 (+0.41z)| norm 0.2564 (+0.53z)| lr 3.30e-08 | 322.98 ms | 52.3% bf16 MFU | 1623750 tok/s
step 19473/19560 | loss 3.287467 (-0.17z)| norm 0.2201 (-0.72z)| lr 3.22e-08 | 322.71 ms | 52.3% bf16 MFU | 1623794 tok/s
step 19474/19560 | loss 3.283077 (-0.25z)| norm 0.2498 (+0.30z)| lr 3.15e-08 | 322.66 ms | 52.3% bf16 MFU | 1623848 tok/s
step 19475/19560 | loss 3.292237 (-0.04z)| norm 0.2308 (-0.35z)| lr 3.08e-08 | 323.35 ms | 52.2% bf16 MFU | 1623727 tok/s
step 19476/19560 | loss 3.288332 (-0.13z)| norm 0.2286 (-0.43z)| lr 3.01e-08 | 323.20 ms | 52.2% bf16 MFU | 1623648 tok/s
step 19477/19560 | loss 3.287972 (-0.13z)| norm 0.2317 (-0.33z)| lr 2.94e-08 | 322.87 ms | 52.3% bf16 MFU | 1623658 tok/s
step 19478/19560 | loss 3.254336 (-0.88z)| norm 0.2279 (-0.45z)| lr 2.87e-08 | 322.94 ms | 52.3% bf16 MFU | 1623649 tok/s
step 19479/19560 | loss 3.307001 (+0.30z)| norm 0.2320 (-0.31z)| lr 2.80e-08 | 322.35 ms | 52.4% bf16 MFU | 1623788 tok/s
step 19480/19560 | loss 3.290272 (-0.07z)| norm 0.2397 (-0.04z)| lr 2.73e-08 | 322.88 ms | 52.3% bf16 MFU | 1623788 tok/s
step 19481/19560 | loss 3.278827 (-0.33z)| norm 0.2268 (-0.48z)| lr 2.66e-08 | 323.38 ms | 52.2% bf16 MFU | 1623663 tok/s
step 19482/19560 | loss 3.279766 (-0.31z)| norm 0.2501 (+0.32z)| lr 2.60e-08 | 322.38 ms | 52.4% bf16 MFU | 1623796 tok/s
step 19483/19560 | loss 3.294460 (+0.02z)| norm 0.2183 (-0.77z)| lr 2.53e-08 | 322.08 ms | 52.4% bf16 MFU | 1623996 tok/s
step 19484/19560 | loss 3.294750 (+0.03z)| norm 0.2393 (-0.05z)| lr 2.47e-08 | 323.82 ms | 52.1% bf16 MFU | 1623750 tok/s
step 19485/19560 | loss 3.206136 (-1.91z)| norm 0.2298 (-0.38z)| lr 2.40e-08 | 322.44 ms | 52.3% bf16 MFU | 1623862 tok/s
step 19486/19560 | loss 3.305559 (+0.29z)| norm 0.2360 (-0.17z)| lr 2.34e-08 | 322.68 ms | 52.3% bf16 MFU | 1623910 tok/s
step 19487/19560 | loss 3.205659 (-1.90z)| norm 0.2344 (-0.22z)| lr 2.28e-08 | 322.67 ms | 52.3% bf16 MFU | 1623957 tok/s
step 19488/19560 | loss 3.295388 (+0.07z)| norm 0.2501 (+0.31z)| lr 2.22e-08 | 322.75 ms | 52.3% bf16 MFU | 1623981 tok/s
step 19489/19560 | loss 3.259730 (-0.70z)| norm 0.2356 (-0.19z)| lr 2.16e-08 | 322.52 ms | 52.3% bf16 MFU | 1624061 tok/s
step 19490/19560 | loss 3.376874 (+1.84z)| norm 0.2452 (+0.14z)| lr 2.10e-08 | 322.35 ms | 52.4% bf16 MFU | 1624182 tok/s
step 19491/19560 | loss 3.276298 (-0.35z)| norm 0.2307 (-0.36z)| lr 2.04e-08 | 323.50 ms | 52.2% bf16 MFU | 1624007 tok/s
step 19492/19560 | loss 3.333976 (+0.90z)| norm 0.2622 (+0.71z)| lr 1.98e-08 | 322.57 ms | 52.3% bf16 MFU | 1624075 tok/s
step 19493/19560 | loss 3.334810 (+0.91z)| norm 0.2779 (+1.23z)| lr 1.92e-08 | 322.77 ms | 52.3% bf16 MFU | 1624088 tok/s
step 19494/19560 | loss 3.282141 (-0.25z)| norm 0.2353 (-0.22z)| lr 1.87e-08 | 322.60 ms | 52.3% bf16 MFU | 1624145 tok/s
step 19495/19560 | loss 3.358419 (+1.40z)| norm 0.2803 (+1.30z)| lr 1.81e-08 | 322.53 ms | 52.3% bf16 MFU | 1624216 tok/s
step 19496/19560 | loss 3.172189 (-2.56z)| norm 0.2254 (-0.57z)| lr 1.76e-08 | 322.79 ms | 52.3% bf16 MFU | 1624218 tok/s
step 19497/19560 | loss 3.271140 (-0.47z)| norm 0.2321 (-0.34z)| lr 1.70e-08 | 322.97 ms | 52.3% bf16 MFU | 1624175 tok/s
step 19498/19560 | loss 3.234278 (-1.24z)| norm 0.2504 (+0.27z)| lr 1.65e-08 | 322.67 ms | 52.3% bf16 MFU | 1624208 tok/s
step 19499/19560 | loss 3.247674 (-0.94z)| norm 0.2773 (+1.19z)| lr 1.60e-08 | 322.28 ms | 52.4% bf16 MFU | 1624338 tok/s
step 19500/19560 | loss 3.259732 (-0.68z)| norm 0.2216 (-0.69z)| lr 1.55e-08 | 322.73 ms | 52.3% bf16 MFU | 1624348 tok/s
val loss 3.274286
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3053/10042 = 0.304023
step 19501/19560 | loss 3.259327 (-0.67z)| norm 0.2423 (+0.01z)| lr 1.50e-08 | 323.32 ms | 52.2% bf16 MFU | 1624210 tok/s
step 19502/19560 | loss 3.267449 (-0.50z)| norm 0.2386 (-0.12z)| lr 1.45e-08 | 323.02 ms | 52.2% bf16 MFU | 1624154 tok/s
step 19503/19560 | loss 3.249987 (-0.87z)| norm 0.2405 (-0.05z)| lr 1.40e-08 | 322.52 ms | 52.3% bf16 MFU | 1624226 tok/s
step 19504/19560 | loss 3.235215 (-1.16z)| norm 0.2419 (-0.01z)| lr 1.35e-08 | 322.47 ms | 52.3% bf16 MFU | 1624309 tok/s
step 19505/19560 | loss 3.286049 (-0.10z)| norm 0.2334 (-0.30z)| lr 1.31e-08 | 322.34 ms | 52.4% bf16 MFU | 1624418 tok/s
step 19506/19560 | loss 3.342950 (+1.09z)| norm 0.2419 (-0.01z)| lr 1.26e-08 | 322.50 ms | 52.3% bf16 MFU | 1624482 tok/s
step 19507/19560 | loss 3.176334 (-2.32z)| norm 0.2349 (-0.26z)| lr 1.21e-08 | 322.53 ms | 52.3% bf16 MFU | 1624536 tok/s
step 19508/19560 | loss 3.300934 (+0.22z)| norm 0.2415 (-0.04z)| lr 1.17e-08 | 322.45 ms | 52.3% bf16 MFU | 1624607 tok/s
step 19509/19560 | loss 3.215183 (-1.58z)| norm 0.2244 (-0.93z)| lr 1.12e-08 | 322.77 ms | 52.3% bf16 MFU | 1624592 tok/s
step 19510/19560 | loss 3.350995 (+1.35z)| norm 0.2495 (+0.52z)| lr 1.08e-08 | 322.13 ms | 52.4% bf16 MFU | 1624742 tok/s
step 19511/19560 | loss 3.223073 (-1.40z)| norm 0.2278 (-0.72z)| lr 1.04e-08 | 323.42 ms | 52.2% bf16 MFU | 1624559 tok/s
step 19512/19560 | loss 3.324532 (+0.78z)| norm 0.2270 (-0.76z)| lr 1.00e-08 | 322.18 ms | 52.4% bf16 MFU | 1624698 tok/s
step 19513/19560 | loss 3.299175 (+0.24z)| norm 0.2300 (-0.59z)| lr 9.58e-09 | 322.53 ms | 52.3% bf16 MFU | 1624741 tok/s
step 19514/19560 | loss 3.315346 (+0.61z)| norm 0.2231 (-0.97z)| lr 9.19e-09 | 322.23 ms | 52.4% bf16 MFU | 1624856 tok/s
step 19515/19560 | loss 3.339180 (+1.11z)| norm 0.2434 (+0.22z)| lr 8.82e-09 | 322.30 ms | 52.4% bf16 MFU | 1624949 tok/s
step 19516/19560 | loss 3.271267 (-0.38z)| norm 0.2632 (+1.36z)| lr 8.42e-09 | 323.23 ms | 52.2% bf16 MFU | 1624803 tok/s
step 19517/19560 | loss 3.279777 (-0.19z)| norm 0.2204 (-1.14z)| lr 8.06e-09 | 322.65 ms | 52.3% bf16 MFU | 1624809 tok/s
step 19518/19560 | loss 3.272051 (-0.36z)| norm 0.2416 (+0.09z)| lr 7.69e-09 | 322.47 ms | 52.3% bf16 MFU | 1624862 tok/s
step 19519/19560 | loss 3.292563 (+0.09z)| norm 0.2457 (+0.33z)| lr 7.35e-09 | 322.02 ms | 52.4% bf16 MFU | 1625027 tok/s
step 19520/19560 | loss 3.255743 (-0.71z)| norm 0.2337 (-0.37z)| lr 6.99e-09 | 322.48 ms | 52.3% bf16 MFU | 1625066 tok/s
step 19521/19560 | loss 3.306442 (+0.40z)| norm 0.4176 (+7.66z)| lr 6.65e-09 | 322.72 ms | 52.3% bf16 MFU | 1625043 tok/s
step 19522/19560 | loss 3.265384 (-0.50z)| norm 0.2388 (-0.11z)| lr 6.33e-09 | 322.31 ms | 52.4% bf16 MFU | 1625123 tok/s
step 19523/19560 | loss 3.287360 (-0.03z)| norm 0.2400 (-0.06z)| lr 6.01e-09 | 322.39 ms | 52.4% bf16 MFU | 1625180 tok/s
step 19524/19560 | loss 3.348942 (+1.31z)| norm 0.2311 (-0.44z)| lr 5.70e-09 | 322.59 ms | 52.3% bf16 MFU | 1625184 tok/s
step 19525/19560 | loss 3.288743 (-0.01z)| norm 0.2543 (+0.57z)| lr 5.40e-09 | 322.52 ms | 52.3% bf16 MFU | 1625204 tok/s
step 19526/19560 | loss 3.253451 (-0.78z)| norm 0.2482 (+0.30z)| lr 5.10e-09 | 322.17 ms | 52.4% bf16 MFU | 1625312 tok/s
step 19527/19560 | loss 3.297486 (+0.23z)| norm 0.2443 (+0.14z)| lr 4.81e-09 | 322.30 ms | 52.4% bf16 MFU | 1625381 tok/s
step 19528/19560 | loss 3.344249 (+1.29z)| norm 0.2426 (+0.07z)| lr 4.52e-09 | 322.26 ms | 52.4% bf16 MFU | 1625458 tok/s
step 19529/19560 | loss 3.320137 (+0.76z)| norm 0.2304 (-0.47z)| lr 4.26e-09 | 322.46 ms | 52.3% bf16 MFU | 1625480 tok/s
step 19530/19560 | loss 3.211420 (-1.75z)| norm 0.2271 (-0.62z)| lr 4.01e-09 | 322.17 ms | 52.4% bf16 MFU | 1625575 tok/s
step 19531/19560 | loss 3.282756 (-0.10z)| norm 0.2369 (-0.19z)| lr 3.74e-09 | 322.32 ms | 52.4% bf16 MFU | 1625627 tok/s
step 19532/19560 | loss 3.322522 (+0.81z)| norm 0.2297 (-0.50z)| lr 3.50e-09 | 322.49 ms | 52.3% bf16 MFU | 1625634 tok/s
step 19533/19560 | loss 3.183940 (-2.33z)| norm 0.2322 (-0.38z)| lr 3.25e-09 | 322.19 ms | 52.4% bf16 MFU | 1625715 tok/s
step 19534/19560 | loss 3.261054 (-0.57z)| norm 0.2298 (-0.49z)| lr 3.04e-09 | 322.31 ms | 52.4% bf16 MFU | 1625763 tok/s
step 19535/19560 | loss 3.285622 (+0.00z)| norm 0.2308 (-0.44z)| lr 2.81e-09 | 322.43 ms | 52.3% bf16 MFU | 1625777 tok/s
step 19536/19560 | loss 3.226166 (-1.33z)| norm 0.2262 (-0.65z)| lr 2.59e-09 | 322.62 ms | 52.3% bf16 MFU | 1625743 tok/s
step 19537/19560 | loss 3.296719 (+0.26z)| norm 0.2256 (-0.68z)| lr 2.40e-09 | 322.54 ms | 52.3% bf16 MFU | 1625730 tok/s
step 19538/19560 | loss 3.247455 (-0.85z)| norm 0.2376 (-0.14z)| lr 2.20e-09 | 322.36 ms | 52.4% bf16 MFU | 1625763 tok/s
step 19539/19560 | loss 3.270198 (-0.35z)| norm 0.2475 (+0.30z)| lr 2.02e-09 | 322.67 ms | 52.3% bf16 MFU | 1625716 tok/s
step 19540/19560 | loss 3.334180 (+1.12z)| norm 0.2611 (+0.89z)| lr 1.84e-09 | 322.56 ms | 52.3% bf16 MFU | 1625700 tok/s
step 19541/19560 | loss 3.291686 (+0.14z)| norm 0.2329 (-0.36z)| lr 1.66e-09 | 322.43 ms | 52.3% bf16 MFU | 1625718 tok/s
step 19542/19560 | loss 3.261523 (-0.55z)| norm 0.2602 (+0.84z)| lr 1.50e-09 | 322.77 ms | 52.3% bf16 MFU | 1625648 tok/s
step 19543/19560 | loss 3.297180 (+0.25z)| norm 0.2302 (-0.49z)| lr 1.34e-09 | 322.52 ms | 52.3% bf16 MFU | 1625645 tok/s
step 19544/19560 | loss 3.272327 (-0.31z)| norm 0.2410 (+0.00z)| lr 1.20e-09 | 322.53 ms | 52.3% bf16 MFU | 1625641 tok/s
step 19545/19560 | loss 3.272689 (-0.31z)| norm 0.2415 (+0.03z)| lr 1.07e-09 | 322.30 ms | 52.4% bf16 MFU | 1625694 tok/s
step 19546/19560 | loss 3.306659 (+0.47z)| norm 0.2433 (+0.10z)| lr 9.30e-10 | 322.30 ms | 52.4% bf16 MFU | 1625744 tok/s
step 19547/19560 | loss 3.195958 (-2.03z)| norm 0.2625 (+0.95z)| lr 8.23e-10 | 322.80 ms | 52.3% bf16 MFU | 1625667 tok/s
step 19548/19560 | loss 3.252087 (-0.76z)| norm 0.2371 (-0.20z)| lr 6.97e-10 | 322.88 ms | 52.3% bf16 MFU | 1625572 tok/s
step 19549/19560 | loss 3.253364 (-0.73z)| norm 0.2426 (+0.05z)| lr 6.08e-10 | 322.82 ms | 52.3% bf16 MFU | 1625498 tok/s
step 19550/19560 | loss 3.287394 (+0.05z)| norm 0.2722 (+1.36z)| lr 5.01e-10 | 322.11 ms | 52.4% bf16 MFU | 1625608 tok/s
step 19551/19560 | loss 3.300835 (+0.34z)| norm 0.2235 (-0.81z)| lr 4.11e-10 | 322.39 ms | 52.4% bf16 MFU | 1625641 tok/s
step 19552/19560 | loss 3.195441 (-2.06z)| norm 0.2398 (-0.08z)| lr 3.40e-10 | 322.58 ms | 52.3% bf16 MFU | 1625622 tok/s
step 19553/19560 | loss 3.188078 (-2.17z)| norm 0.2430 (+0.06z)| lr 2.68e-10 | 323.04 ms | 52.2% bf16 MFU | 1625492 tok/s
step 19554/19560 | loss 3.274673 (-0.22z)| norm 0.2263 (-0.68z)| lr 1.97e-10 | 322.78 ms | 52.3% bf16 MFU | 1625431 tok/s
step 19555/19560 | loss 3.320196 (+0.80z)| norm 0.2514 (+0.43z)| lr 1.43e-10 | 322.75 ms | 52.3% bf16 MFU | 1625381 tok/s
step 19556/19560 | loss 3.252189 (-0.73z)| norm 0.2931 (+2.23z)| lr 1.07e-10 | 322.91 ms | 52.3% bf16 MFU | 1625293 tok/s
step 19557/19560 | loss 3.257648 (-0.59z)| norm 0.2504 (+0.37z)| lr 7.15e-11 | 322.42 ms | 52.3% bf16 MFU | 1625334 tok/s
step 19558/19560 | loss 3.282717 (-0.02z)| norm 0.2317 (-0.44z)| lr 3.58e-11 | 322.74 ms | 52.3% bf16 MFU | 1625292 tok/s
step 19559/19560 | loss 3.315925 (+0.73z)| norm 0.2217 (-0.87z)| lr 1.79e-11 | 322.70 ms | 52.3% bf16 MFU | 1625262 tok/s
step 19560/19560 | loss 3.223261 (-1.35z)| norm 0.2269 (-0.63z)| lr 0.00e+00 | 322.63 ms | 52.3% bf16 MFU | 1625252 tok/s
val loss 3.274314
ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79ting HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79evaluating HellaSwag: 0/79
evaluating HellaSwag: 10/79
evaluating HellaSwag: 20/79
evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
HellaSwag: 3043/10042 = 0.303027

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00019560_00005.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00019560_00007.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00019560_00004.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00019560_00006.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00019560_00001.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00019560_00003.bin

evaluating HellaSwag: 30/79
evaluating HellaSwag: 40/79
evaluating HellaSwag: 50/79
evaluating HellaSwag: 60/79
evaluating HellaSwag: 70/79
Writing state to log124M/state_00019560_00002.bin
Error: Token out of vocabulary at train_gpt2.cu:675
Error details:
  File: train_gpt2.cu
  Line: 675
  Token: -1149026846
  Position: 0
  Vocab: 50257
generating:
---
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[20376,1],0]
  Exit code:    1
--------------------------------------------------------------------------