new data

logs/main_log.txt

@@ -40628,3 +40628,10 @@ time (ms)
 steps: 110000 loss: 2.7673 iter time (s): 0.003 samples/sec: 172747.511
  iteration   110000/  152972 | consumed samples:     51240384 | elapsed time per iteration (ms): 5934.5 | learning rate: 4.982E-05 | global batch size:   512 | lm loss: 2.781886E+00 | loss scale: 524288.0 | grad norm: 51011.958 | num zeros: 0.0 | number of skipped iterations:   0 | number of nan iterations:   0 |
 time (ms)
+--------------------------------------------------------------------------------------------------
+ validation loss at iteration 110000 | lm loss value: 2.730394E+00 | lm loss PPL: 1.533892E+01 | 
+--------------------------------------------------------------------------------------------------
+ iteration   110200/  152972 | consumed samples:     51342784 | elapsed time per iteration (ms): 6804.5 | learning rate: 4.948E-05 | global batch size:   512 | lm loss: 2.781904E+00 | loss scale: 1048576.0 | grad norm: 123221.355 | num zeros: 0.0 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)
+ iteration   110400/  152972 | consumed samples:     51445184 | elapsed time per iteration (ms): 5943.6 | learning rate: 4.914E-05 | global batch size:   512 | lm loss: 2.781725E+00 | loss scale: 524288.0 | grad norm: 51531.358 | num zeros: 0.0 | number of skipped iterations:   0 | number of nan iterations:   0 |
+time (ms)