Repository: Lightning-AI/pytorch-lightning

Issue:
ReduceLROnPlateau scheduler type check
## 🐛 Bug

Incorrect type check for scheduler of class ReduceLROnPlateau.
https://github.com/PyTorchLightning/pytorch-lightning/blob/bc01b9ac1c0ddf503f83755c00d5541a2679e5b4/pytorch_lightning/trainer/trainer.py#L713

I believe, this check:
`isinstance(scheduler, optim.lr_scheduler.ReduceLROnPlateau)`

must look like this:
`isinstance(scheduler['scheduler'], optim.lr_scheduler.ReduceLROnPlateau)`

### To Reproduce

Steps to reproduce the behavior:

1. Create a scheduler of type `optim.lr_scheduler.ReduceLROnPlateau` in the `configure_optimizers` method of a `LightningModule` class.
2. Return an optimizer and scheduler from this method. Place them in lists: `return [optimizer], [scheduler]`.
2. Execute the `trainer.fit(module)`.
3. Put a break-point here: https://github.com/PyTorchLightning/pytorch-lightning/blob/bc01b9ac1c0ddf503f83755c00d5541a2679e5b4/pytorch_lightning/trainer/trainer.py#L712
4. Make sure that the condition is never True.


Discussion:
Hi! thanks for your contribution!, great first issue!
@SkafteNicki could you pls check it? as it comes from your PR :]
I agree with @alexeykarnachev that this is indeed a bug, that I introduced with the recent rework of the learning rate schedulers. I can send a new PR with the bug fix and include a test for `ReduceLROnPlateau` schedulers
Thx guys!
@alexeykarnachev minds send fix PR? 

Resolving patch:
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -707,8 +707,8 @@ def configure_schedulers(self, schedulers: list):
                 if 'scheduler' not in scheduler:
                     raise ValueError(f'Lr scheduler should have key `scheduler`',
                                      ' with item being a lr scheduler')
-                scheduler['reduce_on_plateau'] = \
-                    isinstance(scheduler, optim.lr_scheduler.ReduceLROnPlateau)
+                scheduler['reduce_on_plateau'] = isinstance(
+                    scheduler['scheduler'], optim.lr_scheduler.ReduceLROnPlateau)
 
                 lr_schedulers.append({**default_config, **scheduler})
 


Test patch:
diff --git a/tests/models/__init__.py b/tests/models/__init__.py
--- a/tests/models/__init__.py
+++ b/tests/models/__init__.py
@@ -24,7 +24,8 @@
     LightInfTestDataloader,
     LightTestOptimizerWithSchedulingMixin,
     LightTestMultipleOptimizersWithSchedulingMixin,
-    LightTestOptimizersWithMixedSchedulingMixin
+    LightTestOptimizersWithMixedSchedulingMixin,
+    LightTestReduceLROnPlateauMixin
 )
 
 
diff --git a/tests/models/mixins.py b/tests/models/mixins.py
--- a/tests/models/mixins.py
+++ b/tests/models/mixins.py
@@ -678,6 +678,16 @@ def configure_optimizers(self):
             [{'scheduler': lr_scheduler1, 'interval': 'step'}, lr_scheduler2]
 
 
+class LightTestReduceLROnPlateauMixin:
+    def configure_optimizers(self):
+        if self.hparams.optimizer_name == 'lbfgs':
+            optimizer = optim.LBFGS(self.parameters(), lr=self.hparams.learning_rate)
+        else:
+            optimizer = optim.Adam(self.parameters(), lr=self.hparams.learning_rate)
+        lr_scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer)
+        return [optimizer], [lr_scheduler]
+
+
 def _get_output_metric(output, name):
     if isinstance(output, dict):
         val = output[name]
diff --git a/tests/trainer/test_optimizers.py b/tests/trainer/test_optimizers.py
--- a/tests/trainer/test_optimizers.py
+++ b/tests/trainer/test_optimizers.py
@@ -10,9 +10,12 @@
 from tests.models import (
     TestModelBase,
     LightTrainDataloader,
+    LightValidationStepMixin,
+    LightValidationMixin,
     LightTestOptimizerWithSchedulingMixin,
     LightTestMultipleOptimizersWithSchedulingMixin,
-    LightTestOptimizersWithMixedSchedulingMixin
+    LightTestOptimizersWithMixedSchedulingMixin,
+    LightTestReduceLROnPlateauMixin
 )
 
 
@@ -144,3 +147,35 @@ class CurrentTestModel(
     # Called every 3 steps, meaning for 1 epoch of 11 batches, it is called 3 times
     assert init_lr * 0.1 == adjusted_lr2, \
         'lr for optimizer 2 not adjusted correctly'
+
+
+def test_reduce_lr_on_plateau_scheduling(tmpdir):
+    tutils.reset_seed()
+
+    class CurrentTestModel(
+            LightTestReduceLROnPlateauMixin,
+            LightTrainDataloader,
+            LightValidationMixin,
+            LightValidationStepMixin,
+            TestModelBase):
+        pass
+
+    hparams = tutils.get_hparams()
+    model = CurrentTestModel(hparams)
+
+    # logger file to get meta
+    trainer_options = dict(
+        default_save_path=tmpdir,
+        max_epochs=1,
+        val_percent_check=0.1,
+        train_percent_check=0.2
+    )
+
+    # fit model
+    trainer = Trainer(**trainer_options)
+    results = trainer.fit(model)
+
+    assert trainer.lr_schedulers[0] == \
+        dict(scheduler=trainer.lr_schedulers[0]['scheduler'], monitor='val_loss',
+             interval='epoch', frequency=1, reduce_on_plateau=True), \
+        'lr schduler was not correctly converted to dict'