Repository: Lightning-AI/pytorch-lightning

Issue:
How to use ReduceLROnPlateau methon in matster branch version?
#### What is your question?
I find the log design has changed a lot between version 0.8.5 and master branch [0c26468](https://github.com/PyTorchLightning/pytorch-lightning/tree/0c264689cb566582ac47333d8b7192d656e19440)
I got the error message when I follow the [docs logging-from-a-lightningmodule](https://pytorch-lightning.readthedocs.io/en/latest/loggers.html#logging-from-a-lightningmodule) to modify the log code.

error message:
```
MisconfigurationException: ReduceLROnPlateau conditioned on metric val_loss which is not available. Available metrics are: val_early_stop_on,val_checkpoint_on,epoch,checkpoint_on. Condition can be set using `monitor` key in lr scheduler dict
```


#### Code
```python
def configure_optimizers(self):
    optimizer = torch.optim.Adam(self.parameters(), lr = 0.01)
    scheduler = ReduceLROnPlateau(optimizer, patience=10)
    return [optimizer], [scheduler]

def validation_step(self, batch, batch_nb):
    x, y = batch
    
    y_hat = self(x)    
    loss = F.l1_loss(y, y_hat)
    result = pl.EvalResult()
    result.log('val_step_loss', loss)
    return result

def validation_epoch_end(self, outputs):
    avg_loss = outputs.val_step_loss.mean()
    result = pl.EvalResult()
    result.log('val_loss', avg_loss)
    return result
```
I wonder if I should defined `validation_epoch_end` like above, **if there are any example about how to use `ReduceLROnPlateau` in a right way?**

#### What's your environment?

 - OS: Ubuntu 18.04
 - Packaging pip
 - Version 0.9.0 rc 12 master branch [0c26468](https://github.com/PyTorchLightning/pytorch-lightning/tree/0c264689cb566582ac47333d8b7192d656e19440)



Discussion:
it's a bug I think since callback_metrics contains only `val_early_stop_on` and `val_checkpoint_on`.
https://github.com/PyTorchLightning/pytorch-lightning/blob/5bce06c05023b9798c42533bc1e7e5868930dcdb/pytorch_lightning/core/step_result.py#L727-L733 
https://github.com/PyTorchLightning/pytorch-lightning/blob/5bce06c05023b9798c42533bc1e7e5868930dcdb/pytorch_lightning/trainer/training_loop.py#L1254-L1263
yeah, that's a bug with the new version. adding a fix now. 

What should we condition it on? i guess the clear thing maybe is on the checkpoint key?

```python
result = EvalResult(checkpoint_on=the_thing_to_lr_reduce_on)
```

this does assume the scheduler currently adjusts using val loop and not train loop?
```python
result = EvalResult(checkpoint_on=the_thing_to_lr_reduce_on)
```
in that case, if some assigns checkpoint_on to be a metric `EvalResult(checkpoint_on=some_metric_tensor)` and ReduceLROnPlateau to monitor on `val_loss` then there might be a conflict here.
the point is that when using ReduceLROnPlateau there is no longer "val_loss"... it will monitor whatever the value of `checkpoint_on` is 
`val_loss` is just an example. My point here is if `checkpoint_on=metric1` and `monitor=metric2` for ReduceLROnPlateau, in such case it's a conflict. We should not force it to monitor `metric1` for ReduceLROnPlateau.
haha. i think you're still missing the point.

the keyword 'monitor' **does not have an effect** when using evalresults... instead, the ReduceLROnPlateau will look at whatever is on the `checkpoint_on`

You could set monitor='jiraffe' for ReduceLROnPlateau and it **won't matter**.
Lightning will use whatever is in checkpoint_on=X
yeah, maybe I am missing something here 😅 but not sure how this will work if we take `checkpoint_on` value for `ReduceLROnPlateau`:
```python
model_checkpoint = ModelCheckpoint(monitor='val_loss')

...

def validation_epoch_end(self, outputs):
    val_loss = ...
    val_recall = ...
    res = pl.EvalResult(checkpoint_on=val_loss) # not familier with EvalResult but I guess checkpoint_on will be used for modelcheckpoint
    res.log('val_recall', val_recall)
    return res

def configure_optimizers(self):
    optimizer = ...
    scheduler = {'scheduler': ReduceLROnPlateau(), 'interval': 'epoch', monitor: 'val_recall'}
    return [optimizer], [scheduler]
```
again... monitor has NO effect anywhere... with the results object. doesn’t matter what callback uses the word
Had the same question so arrived here.

@williamFalcon, I think what @rohitgr7  means, is that there might be cases where someone wish to use ReduceOnPlatue on metric1 and to save checkpoint on metric2.

i.e, I wish to use ReduceOnPlatue on train_loss (to allow the network to (over)fit in case the lr is not low enough) and use checkpoint_on='val_acc', to save the best model along the training routine. 
ok, then we should allow feeding the values logged as options to the callbacks?
Not sure myself (I'm not sure of the implementation details). 
Would love if you could maybe tag someone who could answer that better.

Ido

haha ok. yeah, i think that’s the sensible option since this allows any metric to be monitored by any callback in the future
yeah, will solve issues with `ModelCheckpoint` too. Also, why even add a `checkpoint_on/early_stop_on` as a parameter there? can't we just use log itself with `checkpoint=True/False` just like `on_epoch/on_step`, if we feed logs in the callbacks? Just a suggestion.
because we need a single, unique value to checkpoint/early stop on. with log we can’t enforce that. and also if you want to change what to ckpt on during training, this approach allows that. 

ie: in some tasks, i may use loss for a while, but switch to a metric after a certain number of epochs
then maybe take the first value in case if multiple checkpoint values are passed with log and raise a warning there?
```python
result.log('metric1', metric1_val, checkpoint=True)
result.log('metric2', metric2_val, checkpoint=True)
```
take metric1_val for checkpoint.
let's go with the current approach in the API and iterate on it if it causes issues.

Resolving patch:
diff --git a/pytorch_lightning/trainer/training_loop.py b/pytorch_lightning/trainer/training_loop.py
--- a/pytorch_lightning/trainer/training_loop.py
+++ b/pytorch_lightning/trainer/training_loop.py
@@ -167,6 +167,7 @@ def training_step(self, batch, batch_idx):
 import torch
 import torch.distributed as torch_distrib
 from torch.utils.data import DataLoader
+from copy import deepcopy
 
 from pytorch_lightning import _logger as log
 from pytorch_lightning.callbacks import ModelCheckpoint
@@ -504,9 +505,6 @@ def run_training_epoch(self):
                     opt_outputs = opt_outputs[0]
                 epoch_output[opt_idx].append(opt_outputs)
 
-            # update LR schedulers
-            self.update_train_loop_lr_schedulers()
-
             # when returning -1 from train_step, we end epoch early
             self.should_stop = batch_output.signal == -1
 
@@ -527,6 +525,11 @@ def run_training_epoch(self):
             # -----------------------------------------
             self.save_train_loop_metrics_to_loggers(batch_idx, batch_output)
 
+            # update LR schedulers
+            monitor_metrics = deepcopy(self.callback_metrics)
+            monitor_metrics.update(batch_output.batch_log_metrics)
+            self.update_train_loop_lr_schedulers(monitor_metrics=monitor_metrics)
+
             # progress global step according to grads progress
             self.increment_accumulated_grad_global_step()
 
@@ -589,11 +592,11 @@ def check_checkpoint_callback(self, should_check_val):
             checkpoint_callbacks = [c for c in self.callbacks if isinstance(c, ModelCheckpoint)]
             [c.on_validation_end(self, self.get_model()) for c in checkpoint_callbacks]
 
-    def update_train_loop_lr_schedulers(self):
+    def update_train_loop_lr_schedulers(self, monitor_metrics=None):
         if ((self.batch_idx + 1) % self.accumulate_grad_batches == 0
                 or (self.batch_idx + 1) == self.num_training_batches):
             # update lr
-            self.update_learning_rates(interval='step')
+            self.update_learning_rates(interval='step', monitor_metrics=monitor_metrics)
 
     def run_on_epoch_end_hook(self, model):
         with self.profiler.profile('on_epoch_end'):
@@ -1238,16 +1241,17 @@ def training_forward(self, batch, batch_idx, opt_idx, hiddens):
 
         return output
 
-    def update_learning_rates(self, interval: str):
+    def update_learning_rates(self, interval: str, monitor_metrics=None):
         """Update learning rates.
 
         Args:
             interval: either 'epoch' or 'step'.
+            monitor_metrics: dict of possible values to monitor
         """
         if not self.lr_schedulers:
             return
 
-        for lr_scheduler in self.lr_schedulers:
+        for scheduler_idx, lr_scheduler in enumerate(self.lr_schedulers):
             current_idx = self.batch_idx if interval == 'step' else self.current_epoch
             current_idx += 1  # account for both batch and epoch starts from 0
             # Take step if call to update_learning_rates matches the interval key and
@@ -1256,7 +1260,12 @@ def update_learning_rates(self, interval: str):
                 # If instance of ReduceLROnPlateau, we need to pass validation loss
                 if lr_scheduler['reduce_on_plateau']:
                     monitor_key = lr_scheduler['monitor']
-                    monitor_val = self.callback_metrics.get(monitor_key)
+
+                    if monitor_metrics is not None:
+                        monitor_val = monitor_metrics.get(monitor_key)
+                    else:
+                        monitor_val = self.callback_metrics.get(monitor_key)
+
                     if monitor_val is None:
                         avail_metrics = ','.join(list(self.callback_metrics.keys()))
                         raise MisconfigurationException(
@@ -1264,10 +1273,38 @@ def update_learning_rates(self, interval: str):
                             f' which is not available. Available metrics are: {avail_metrics}.'
                             ' Condition can be set using `monitor` key in lr scheduler dict'
                         )
+                    if self.dev_debugger.enabled:
+                        old_lr = lr_scheduler['scheduler'].optimizer.param_groups[0]['lr']
+
+                    # update LR
                     lr_scheduler['scheduler'].step(monitor_val)
+
+                    if self.dev_debugger.enabled:
+                        new_lr = lr_scheduler['scheduler'].optimizer.param_groups[0]['lr']
+                        self.dev_debugger.track_lr_schedulers_update(
+                            self.batch_idx,
+                            interval,
+                            scheduler_idx,
+                            old_lr,
+                            new_lr,
+                            monitor_key,
+                        )
                 else:
+                    if self.dev_debugger.enabled:
+                        old_lr = lr_scheduler['scheduler'].optimizer.param_groups[0]['lr']
+
+                    # update LR
                     lr_scheduler['scheduler'].step()
 
+                    if self.dev_debugger.enabled:
+                        new_lr = lr_scheduler['scheduler'].optimizer.param_groups[0]['lr']
+                        self.dev_debugger.track_lr_schedulers_update(
+                            self.batch_idx,
+                            interval,
+                            scheduler_idx,
+                            old_lr, new_lr
+                        )
+
 
 def _with_is_last(iterable):
     """Pass through values from the given iterable with an added boolean indicating if this is the last item.
diff --git a/pytorch_lightning/utilities/debugging.py b/pytorch_lightning/utilities/debugging.py
--- a/pytorch_lightning/utilities/debugging.py
+++ b/pytorch_lightning/utilities/debugging.py
@@ -35,6 +35,7 @@ def __init__(self, trainer):
         self.early_stopping_history = []
         self.checkpoint_callback_history = []
         self.events = []
+        self.saved_lr_scheduler_updates = []
 
     def track_event(
             self,
@@ -72,6 +73,19 @@ def track_train_loss_history(self, batch_idx, loss):
         loss_dict = {'batch_idx': batch_idx, 'epoch': self.trainer.current_epoch, 'loss': loss.detach()}
         self.saved_train_losses.append(loss_dict)
 
+    @enabled_only
+    def track_lr_schedulers_update(self, batch_idx, interval, scheduler_idx, old_lr, new_lr, monitor_key=None):
+        loss_dict = {
+            'batch_idx': batch_idx,
+            'interval': interval,
+            'scheduler_idx': scheduler_idx,
+            'epoch': self.trainer.current_epoch,
+            'monitor_key': monitor_key,
+            'old_lr': old_lr,
+            'new_lr': new_lr
+        }
+        self.saved_lr_scheduler_updates.append(loss_dict)
+
     @enabled_only
     def track_eval_loss_history(self, test_mode, batch_idx, dataloader_idx, output):
         loss_dict = {


Test patch:
diff --git a/tests/base/deterministic_model.py b/tests/base/deterministic_model.py
--- a/tests/base/deterministic_model.py
+++ b/tests/base/deterministic_model.py
@@ -456,6 +456,18 @@ def val_dataloader(self):
     def configure_optimizers(self):
         return torch.optim.Adam(self.parameters(), lr=0)
 
+    def configure_optimizers__lr_on_plateau_epoch(self):
+        optimizer = torch.optim.Adam(self.parameters(), lr=0)
+        lr_scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer)
+        scheduler = {'scheduler': lr_scheduler, 'interval': 'epoch', 'monitor': 'epoch_end_log_1'}
+        return [optimizer], [scheduler]
+
+    def configure_optimizers__lr_on_plateau_step(self):
+        optimizer = torch.optim.Adam(self.parameters(), lr=0)
+        lr_scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer)
+        scheduler = {'scheduler': lr_scheduler, 'interval': 'step', 'monitor': 'pbar_acc1'}
+        return [optimizer], [scheduler]
+
     def backward(self, trainer, loss, optimizer, optimizer_idx):
         if self.assert_backward:
             if self.trainer.precision == 16:
diff --git a/tests/trainer/test_trainer_steps_dict_return.py b/tests/trainer/test_trainer_steps_dict_return.py
--- a/tests/trainer/test_trainer_steps_dict_return.py
+++ b/tests/trainer/test_trainer_steps_dict_return.py
@@ -3,6 +3,7 @@
 """
 from pytorch_lightning import Trainer
 from tests.base.deterministic_model import DeterministicModel
+import os
 
 
 def test_training_step_dict(tmpdir):
@@ -130,6 +131,50 @@ def test_full_training_loop_dict(tmpdir):
     assert pbar_metrics['pbar_acc2'] == 21.0
 
 
+def test_result_obj_lr_scheduler_epoch(tmpdir):
+    """
+    test that the LR scheduler was called at the correct time with the correct metrics
+    """
+    os.environ['PL_DEV_DEBUG'] = '1'
+    model = DeterministicModel()
+    model.training_step = model.training_step_for_step_end_dict
+    model.training_step_end = model.training_step_end_dict
+    model.training_epoch_end = model.training_epoch_end_dict
+    model.val_dataloader = None
+    model.configure_optimizers = model.configure_optimizers__lr_on_plateau_epoch
+
+    trainer = Trainer(
+        default_root_dir=tmpdir,
+        max_epochs=3,
+        weights_summary=None,
+    )
+    trainer.fit(model)
+
+    assert len(trainer.dev_debugger.saved_lr_scheduler_updates) == 3
+
+
+def test_result_obj_lr_scheduler_step(tmpdir):
+    """
+    test that the LR scheduler was called at the correct time with the correct metrics
+    """
+    os.environ['PL_DEV_DEBUG'] = '1'
+    model = DeterministicModel()
+    model.training_step = model.training_step_for_step_end_dict
+    model.training_step_end = model.training_step_end_dict
+    model.training_epoch_end = model.training_epoch_end_dict
+    model.val_dataloader = None
+    model.configure_optimizers = model.configure_optimizers__lr_on_plateau_step
+
+    trainer = Trainer(
+        default_root_dir=tmpdir,
+        max_epochs=2,
+        weights_summary=None,
+    )
+    trainer.fit(model)
+
+    assert len(trainer.dev_debugger.saved_lr_scheduler_updates) == 8
+
+
 def test_train_step_epoch_end(tmpdir):
     """
     Checks train_step + training_epoch_end (NO training_step_end)