Repository: pytorch
Issue: Learning rate scheduler list index out of range
Labels: module: optimizer,triaged,module: LrScheduler

## 🐛 Bug

Whenever I use the `ReduceLROnPlateau` scheduler alongside [BackboneFinetuning in PytorchLightning](https://pytorch-lightning.readthedocs.io/en/stable/_modules/pytorch_lightning/callbacks/finetuning.html#BackboneFinetuning) at some point I'll get the following error.

```
   File "xxx\anaconda3\envs\torch\lib\site-packages\torch\optim\lr_scheduler.py", line 651, in _reduce_lr
    new_lr = max(old_lr * self.factor, self.min_lrs[i])
IndexError: list index out of range
```

What seems to be happening is that at some point another parameter group is being created during training using `add_param_group` on the optimizer, but since min_lrs initialized at the start of training in the `__init__` function, it only has as many elements as the number of parameter groups in the original optimizer.

## To Reproduce

Steps to reproduce the behavior:

1. Start a torch lightning project with the `BackboneFinetuning` callback and the `ReduceLROnPlateau` scheduler
2. Train until the epoch that the backbone finetuning would trigger the model to be unfrozen (and thus another parameter group created)
3. Wait for the error (it doesn't always happen right away for whatever reason)

Full stacktrace:

```
Traceback (most recent call last):
  File "train_lightning.py", line 103, in train
    trainer.fit(net, datamodule=data_module)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 553, in fit
    self._run(model)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 918, in _run
    self._dispatch()
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 986, in _dispatch
    self.accelerator.start_training(self)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\accelerators\accelerator.py", line 92, in start_training
    self.training_type_plugin.start_training(trainer)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 161, in start_training
    self._results = trainer.run_stage()
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 996, in run_stage
    return self._run_train()
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1045, in _run_train
    self.fit_loop.run()
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\loops\base.py", line 111, in run
    self.advance(*args, **kwargs)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\loops\fit_loop.py", line 200, in advance
    epoch_output = self.epoch_loop.run(train_dataloader)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\loops\base.py", line 118, in run
    output = self.on_run_end()
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\loops\epoch\training_epoch_loop.py", line 240, in on_run_end
    self.update_lr_schedulers("epoch", update_plateau_schedulers=True)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\loops\epoch\training_epoch_loop.py", line 403, in update_lr_schedulers
    self.trainer.optimizer_connector.update_learning_rates(
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\pytorch_lightning\trainer\connectors\optimizer_connector.py", line 88, in update_learning_rates
    lr_scheduler["scheduler"].step(monitor_val)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\torch\optim\lr_scheduler.py", line 642, in step
    self._reduce_lr(epoch)
  File "c:\users\dlangerm\anaconda3\envs\torch\lib\site-packages\torch\optim\lr_scheduler.py", line 651, in _reduce_lr
    new_lr = max(old_lr * self.factor, self.min_lrs[i])
IndexError: list index out of range
```

Code that initializes the scheduler:
```
lr_scheduler.ReduceLROnPlateau(
    optimizer, 'min', min_lr=0, verbose=False, cooldown=2, patience=5
)
```

Code that initializes the optimizer (in the `configure_optimizers` function in lightning):

```
optimizer(
    filter(lambda p: p.requires_grad, self.parameters(recurse=True)),
    lr=self.hparams.lr
)
```

## Expected behavior

```
    def _reduce_lr(self, epoch):
        for i, param_group in enumerate(self.optimizer.param_groups):
            old_lr = float(param_group['lr'])
            new_lr = max(old_lr * self.factor, self.min_lrs[i]) # <- using i here assumes that the optimizer wasn't modified since __init__
            if old_lr - new_lr > self.eps:
                param_group['lr'] = new_lr
                if self.verbose:
                    print('Epoch {:5d}: reducing learning rate'
                          ' of group {} to {:.4e}.'.format(epoch, i, new_lr))
```

Since[ `Optimizer.add_param_group`](https://pytorch.org/docs/stable/generated/torch.optim.Optimizer.add_param_group.html) exists, the learning rate scheduler should not assume that the param_groups of the optimizer haven't been modified since __init__ was called. There needs to be a check for this and perhaps an initialization. I'm not familiar enough with the ecosystem to know how to do this in a non-breaking fashion though, but a PR should be opened.

## Environment
PyTorch version: 1.9.0
Is debug build: False
CUDA used to build PyTorch: 11.1
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro
GCC version: (MinGW.org GCC-8.2.0-5) 8.2.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.8.10 | packaged by conda-forge | (default, May 11 2021, 06:25:23) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19043-SP0
Is CUDA available: True
CUDA runtime version: 11.0.221
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 471.41
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.1
[pip3] pytorch-lightning==1.4.0
[pip3] torch==1.9.0
[pip3] torchaudio==0.9.0
[pip3] torchmetrics==0.4.1
[pip3] torchvision==0.10.0
[conda] blas                      2.110                       mkl    conda-forge
[conda] blas-devel                3.9.0                    10_mkl    conda-forge
[conda] cudatoolkit               11.1.1               heb2d755_7    conda-forge
[conda] libblas                   3.9.0                    10_mkl    conda-forge
[conda] libcblas                  3.9.0                    10_mkl    conda-forge
[conda] liblapack                 3.9.0                    10_mkl    conda-forge
[conda] liblapacke                3.9.0                    10_mkl    conda-forge
[conda] mkl                       2021.3.0           hb70f87d_564    conda-forge
[conda] mkl-devel                 2021.3.0           h57928b3_565    conda-forge
[conda] mkl-include               2021.3.0           hb70f87d_564    conda-forge
[conda] numpy                     1.21.1           py38h09042cb_0    conda-forge
[conda] pytorch                   1.9.0           py3.8_cuda11.1_cudnn8_0    pytorch
[conda] pytorch-lightning         1.4.0              pyhd8ed1ab_0    conda-forge
[conda] torchaudio                0.9.0                      py38    pytorch
[conda] torchmetrics              0.4.1              pyhd8ed1ab_0    conda-forge
[conda] torchvision               0.10.0               py38_cu111    pytorch
## Additional context

<!-- Add any other context about the problem here. -->


cc @vincentqb @jbschlosser @albanD @iramazanli