horovod mode increase lr
Not really a 🐛 Bug
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/distrib_parts.py#L299
Under horovod mode, the learning rate will automatically be increased by hvd.size().
This behavior is different from ddp, so it may confuse the users.