Model resetting every 600 iterations?
🐛 Bug
I'm getting a weird model param reset (maybe, not sure) every 600 iterations and haven't been able to pinpoint the reason... happens both on 2xV100 GPUs and v3-8 TPUs.

Does anyone happen to know if something mystic is taking place at iteration 600? I'm using fairly default args and trainer = Trainer.from_argparse_args(args).
Environment
Happens both on PTL 0.9.0 and 0.9.1rc1. Probably not an environment issue, since the CUDA and XLA environments are quite different... I'm hoping someone will recognize the magical number 600.