INFO:__main__:  Optimizer = adafactor
INFO:__main__:  Learning rate (peak) = 0.009
INFO:__main__:  Num examples = 94558172
INFO:__main__:  Num tokenized group examples 109037136
INFO:__main__:  Num Epochs = 1
INFO:__main__:  Instantaneous batch size per device = 4
INFO:__main__:  Total train batch size (w. parallel & grad accum) = 512
INFO:__main__:  Steps per epoch = 212963 (x grad accum (16) = 3407408)
INFO:__main__:  Total optimization steps = 212963 (x grad accum (16) = 3407408)