new data

logs/main_log.txt

@@ -36567,3 +36567,9 @@ steps: 98000 loss: 2.8400 iter time (s): 0.003 samples/sec: 173625.320
 -------------------------------------------------------------------------------------------------
  iteration    98200/  152972 | consumed samples:     45198784 | elapsed time per iteration (ms): 7696.6 | learning rate: 7.149E-05 | global batch size:   512 | lm loss: 2.799447E+00 | loss scale: 131072.0 | grad norm: 12783.497 | num zeros: 0.0 | number of skipped iterations:   0 | number of nan iterations:   0 |
 time (ms)
+ iteration    98400/  152972 | consumed samples:     45301184 | elapsed time per iteration (ms): 5960.1 | learning rate: 7.110E-05 | global batch size:   512 | lm loss: 2.800086E+00 | loss scale: 262144.0 | grad norm: 27676.668 | num zeros: 0.0 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration    98600/  152972 | consumed samples:     45403584 | elapsed time per iteration (ms): 5939.2 | learning rate: 7.071E-05 | global batch size:   512 | lm loss: 2.802239E+00 | loss scale: 262144.0 | grad norm: 26204.568 | num zeros: 0.0 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration    98800/  152972 | consumed samples:     45505984 | elapsed time per iteration (ms): 6202.3 | learning rate: 7.032E-05 | global batch size:   512 | lm loss: 2.799443E+00 | loss scale: 524288.0 | grad norm: 50434.969 | num zeros: 0.0 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)