Scheduler should be initialized after Apex is configured
Description
Using jointly a scheduler and apex-amp throws a UserWarning:

Seems like optimizer.step() has been overridden after learning rate scheduler initialization. Please, make sure to call optimizer.step() before lr_scheduler.step().

As suggested here (https://discuss.pytorch.org/t/cyclic-learning-rate-how-to-use/53796), the scheduler should be initialized after Apex is configured.
To reproduce
Run the MNIST example with use_amp=True.
Expected behavior
No scheduler-related user warning.
Workaround
The following hack seems to work:
    def on_train_start(self):
        self.trainer.lr_schedulers = [optim.lr_scheduler.CosineAnnealingLR(self.trainer.optimizers[0], T_max=10)]