Logging not working correctly with optimizer frequency.
🐛 Bug
Using multiple optimizers with different frequencies, the last (with the higher optimizer_idx) log dict that is returned by the training_step is not considered in tensorboard.
To Reproduce
With a configure_step like
    res_1 = {
        "optimizer": opt_1,
        "scheduler": scheduler_2,
        "frequency": 2
    }

    res_2 = {
        "optimizer": opt_2,
        "scheduler": scheduler_2,
        "frequency": 4
    }

    return res_1, res_2
and a training_step like:
def training_step(self, batch, batches_idx, optimizer_idx):
    """ Simple training step. """
    inputs, labels = batch

    if optimizer_idx == 0:
        loss_1 = self.forward(inputs, labels)
        if self.trainer.use_dp or self.trainer.use_ddp2:
           loss_1 = loss_1.unsqueeze(0)
          
        loss = loss_1
        tqdm_dict = {
            'loss_1': loss_1
        }

    if optimizer_idx == 1:
        loss_2 = self.forward(inputs, labels)
        if self.trainer.use_dp or self.trainer.use_ddp2:
            loss_2 = loss_2.unsqueeze(0)
        loss = loss_2
        tqdm_dict = {
            'loss_2': loss_2
        }

    return {
        'loss': loss,
        'progress_bar': tqdm_dict,
        'log': tqdm_dict
    }
loss_2 is not reported in tensorboard.
Expected behavior
loss_2 to be present in tensorboard.
Environment

CUDA:

GPU:

GeForce GTX 1080 Ti
GeForce GTX 1080 Ti


available:         True
version:           10.2


Packages:

numpy:             1.18.4
pyTorch_debug:     False
pyTorch_version:   1.5.0
pytorch-lightning: 0.7.6
tensorboard:       2.2.1
tqdm:              4.46.0


System:

OS:                Linux
architecture:

64bit



processor:         x86_64
python:            3.7.0
version:           #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020