`use_amp` and multiple optimizers bug
🐛 Bug
Faced this issue when tried to use mixed precision with my two-head model which has two pairs of optimizer & scheduler. Without use_amp everything works fine.
With it enabled, I get:
TypeError: 'CosineAnnealingLR' object is not subscriptable

My investigation has ended up here:
    def reinit_scheduler_properties(self, optimizers: list, schedulers: list):
        # Reinitialize optimizer.step properties added by schedulers
        for scheduler in schedulers:
            for optimizer in optimizers:
                scheduler = scheduler['scheduler']  # <===== this place
                # ...
                if scheduler.optimizer == optimizer:
                    # ...
Obviously, next optimizer will get scheduler as an actual non-dict object as it was reassigned on the first iteration...
To Reproduce
Steps to reproduce the behavior:

Build any LightningModule, which configure_optimizers() method outputs lists of two optimizers and two schedulers. In my case it's something like:

def configure_optimizers(self):
    opt_1 = Adam(params=self.head_1.params(), ...)
    opt_2 = Adam(params=self.head_2.params(), ...)
    
    sch_1 = CosineAnnealingLR(optimizer=opt_1, ...)
    sch_2 = CosineAnnealingLR(optimizer=opt_2, ...)
    return [opt_1, opt_2], [sch_1, sch_2]

Build a Trainer object with  use_amp=True
Call trainer.fit(model)
See error

Environment
* CUDA:
        - GPU:
                - GeForce GTX 1080 Ti
        - available:         True
        - version:           10.2
* Packages:
        - numpy:             1.18.5
        - pyTorch_debug:     False
        - pyTorch_version:   1.5.0
        - pytorch-lightning: 0.8.1
        - tensorboard:       1.15.0
        - tqdm:              4.45.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                -
        - processor:         x86_64
        - python:            3.7.5
        - version:           #107-Ubuntu SMP Thu Jun 4 11:27:52 UTC 2020