Repository: Lightning-AI/pytorch-lightning

Issue:
Add support for ReduceLROnPlateau
**Is your feature request related to a problem? Please describe.**
As of now it does not seem like it is possible to use ReduceLROnPlateau as a metric has to be passed to the step method of the lr_scheduler.

**Describe the solution you'd like**
A possibility to use ReduceLROnPlateau on some or any of the metrics calculated during training or validation.

**Describe alternatives you've considered**
In my use case I want to do the step based on a metric calculated on the validation set. As a workaround I define the lr_scheduler in the __init__ of the model and perform the step in the validation_end function 


Discussion:
@terkelbo  good suggestion. can you propose a clean way of doing this? maybe it can be merged with #29 
I think we can directly adjust lr in the optimizer as [keras](https://github.com/keras-team/keras/blob/master/keras/callbacks.py#L1393), which means we don't need ReduceLROnPlateau as a metric.

Specifically, perhaps we can consider adding a hook in `pytorch_lightning/root_module/root_module.py` like `optimizer_step`, so that this function can be exposed to callbacks?

@lr1d good point. you can just adjust the LR in the current callback also (optimizer_step). or in any of the other callbacks
closing because this should be implicitly supported since we can pass a [ReduceLROnPlateau](https://pytorch.org/docs/stable/optim.html) object... nothing we need to do, this is standard PyTorch functionality
When using the pytorch object, it requires to pass the metric as a param. for exmaple `lr_scheduler.step(val_loss)`. How do you overwrite the origianl schedulers in thie way?
![image](https://user-images.githubusercontent.com/10709657/63795534-b5619900-c936-11e9-8117-8bda6eb18fe6.png)

Please, I can't follow. Why the issue is closed? Do you suggest using some hooks like `optimizer_step` and implement reducing on plateau scheduling by hand?
I’m not sure either how to pass the ReduceLROnPlateau object as it needs the metric argument as pointed out by @Ir1d. @williamFalcon would it be possible that you give an example of how to use this scheduler with pytorch lightning?
It feels rather dirty to me, but you can save the loss in your `pl.LightningModule`'s `training_step()` method, and then use it in the `optimizer_step` method. I can't verify whether it works as expected right now though. 

```
def training_step(self, batch, batch_nb):
        # REQUIRED
        x, y = batch
        y_hat = self.forward(x)
        self.loss = F.mse_loss(y, y_hat)
        return {'loss': self.loss}

def configure_optimizers(self):
        # REQUIRED
        # can return multiple optimizers and learning_rate schedulers
        self.optimizer = torch.optim.Adam(self.parameters(), lr=self.lrinit)
        self.scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(self.optimizer, mode='min', 
                                                           factor=0.1, patience=10, 
                                                           verbose=False, threshold=0.0001, 
                                                           threshold_mode='rel', cooldown=0, 
                                                           min_lr=0, eps=1e-08)
        return self.optimizer

def optimizer_step(self, epoch_nb, batch_nb, optimizer, optimizer_i):
        """
        Do something instead of the standard optimizer behavior
        :param epoch_nb:
        :param batch_nb:
        :param optimizer:
        :param optimizer_i:
        :return:
        """
        # Sometimes (if closure is not None), this step method should return the loss. Now its None
        loss = optimizer.step()
        # So let's use self.loss, set in training_step, for LR scheduling
        self.scheduler.step(self.loss)
        # clear gradients
        optimizer.zero_grad()
```

Edit: This will not work as intended, since the optimizer step is called after every batch.
we could modify the scheduler step to take in the loss when it needs it? i have to look at this more carefully though
That would be the best solution indeed, but then you'd need a way to figure out which ones need the loss when calling step. I'm not sure how to do that at this moment.

For now, I solved it as follows, which may help people until there is a better solution:
- Use the `on_epoch_start` callback on your LightningModule to initialize an empty list
- In every `training_step` call, append the loss to the list
- In the `on_epoch_end` callback, process the list to get the averga loss, call `scheduler.step(mean_loss)` and clear the list
Also, at this moment https://github.com/williamFalcon/pytorch-lightning/blob/master/pytorch_lightning/trainer/trainer.py#L958 lr_scheduler.step() is called at the end of the epoch, while some scheduler (eg: https://pytorch.org/docs/master/optim.html#torch.optim.lr_scheduler.OneCycleLR) should be called at the end of the batch

Resolving patch:
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -966,7 +966,7 @@ def __train(self):
             # update LR schedulers
             if self.lr_schedulers is not None:
                 for lr_scheduler in self.lr_schedulers:
-                    lr_scheduler.step()
+                    lr_scheduler.step(self.current_epoch)
 
             # early stopping
             met_min_epochs = epoch_nb > self.min_nb_epochs


Test patch:
diff --git a/tests/test_models.py b/tests/test_models.py
--- a/tests/test_models.py
+++ b/tests/test_models.py
@@ -7,7 +7,6 @@
 import numpy as np
 import torch
 from test_tube import Experiment
-import time
 
 # sys.path += [os.path.abspath('..'), os.path.abspath('../..')]
 from pytorch_lightning import Trainer
@@ -40,53 +39,6 @@
 # ------------------------------------------------------------------------
 # TESTS
 # ------------------------------------------------------------------------
-def test_running_test_pretrained_model_ddp():
-    """Verify test() on pretrained model"""
-    if not can_run_gpu_test():
-        return
-
-    hparams = get_hparams()
-    model = LightningTestModel(hparams)
-
-    save_dir = init_save_dir()
-
-    # exp file to get meta
-    exp = get_exp(False)
-    exp.argparse(hparams)
-    exp.save()
-
-    # exp file to get weights
-    checkpoint = ModelCheckpoint(save_dir)
-
-    trainer_options = dict(
-        show_progress_bar=False,
-        max_nb_epochs=1,
-        train_percent_check=0.4,
-        val_percent_check=0.2,
-        checkpoint_callback=checkpoint,
-        experiment=exp,
-        gpus=2,
-        distributed_backend='ddp'
-    )
-
-    # fit model
-    trainer = Trainer(**trainer_options)
-    result = trainer.fit(model)
-
-    # correct result and ok accuracy
-    assert result == 1, 'training failed to complete'
-    pretrained_model = load_model(exp, save_dir, on_gpu=True, module_class=LightningTestModel)
-
-    # run test set
-    new_trainer = Trainer(**trainer_options)
-    new_trainer.test(pretrained_model)
-
-    run_prediction(model.test_dataloader, pretrained_model)
-
-    # test we have good test accuracy
-    clear_save_dir()
-
-
 def test_dp_resume():
     """
     Make sure DP continues training correctly
@@ -170,6 +122,53 @@ def assert_good_acc():
     clear_save_dir()
 
 
+def test_running_test_pretrained_model_ddp():
+    """Verify test() on pretrained model"""
+    if not can_run_gpu_test():
+        return
+
+    hparams = get_hparams()
+    model = LightningTestModel(hparams)
+
+    save_dir = init_save_dir()
+
+    # exp file to get meta
+    exp = get_exp(False)
+    exp.argparse(hparams)
+    exp.save()
+
+    # exp file to get weights
+    checkpoint = ModelCheckpoint(save_dir)
+
+    trainer_options = dict(
+        show_progress_bar=False,
+        max_nb_epochs=1,
+        train_percent_check=0.4,
+        val_percent_check=0.2,
+        checkpoint_callback=checkpoint,
+        experiment=exp,
+        gpus=[0, 1],
+        distributed_backend='ddp'
+    )
+
+    # fit model
+    trainer = Trainer(**trainer_options)
+    result = trainer.fit(model)
+
+    # correct result and ok accuracy
+    assert result == 1, 'training failed to complete'
+    pretrained_model = load_model(exp, save_dir, on_gpu=True, module_class=LightningTestModel)
+
+    # run test set
+    new_trainer = Trainer(**trainer_options)
+    new_trainer.test(pretrained_model)
+
+    run_prediction(model.test_dataloader, pretrained_model)
+
+    # test we have good test accuracy
+    clear_save_dir()
+
+
 def test_running_test_after_fitting():
     """Verify test() on fitted model"""
     hparams = get_hparams()
@@ -250,28 +249,6 @@ class CurrentTestModel(LightningTestMixin, LightningTestModelBase):
     clear_save_dir()
 
 
-def test_multi_gpu_model_ddp():
-    """
-    Make sure DDP works
-    :return:
-    """
-    if not can_run_gpu_test():
-        return
-
-    os.environ['MASTER_PORT'] = str(np.random.randint(12000, 19000, 1)[0])
-    model, hparams = get_model()
-    trainer_options = dict(
-        show_progress_bar=False,
-        max_nb_epochs=1,
-        train_percent_check=0.4,
-        val_percent_check=0.2,
-        gpus=2,
-        distributed_backend='ddp'
-    )
-
-    run_gpu_model_test(trainer_options, model, hparams)
-
-
 def test_running_test_pretrained_model():
     """Verify test() on pretrained model"""
     hparams = get_hparams()
@@ -339,7 +316,7 @@ def test_running_test_pretrained_model_dp():
         val_percent_check=0.2,
         checkpoint_callback=checkpoint,
         experiment=exp,
-        gpus=2,
+        gpus=[0, 1],
         distributed_backend='dp'
     )
 
@@ -431,6 +408,28 @@ def optimizer_step(self, epoch_nb, batch_nb, optimizer, optimizer_i):
     trainer.fit(model)
 
 
+def test_multi_gpu_model_ddp():
+    """
+    Make sure DDP works
+    :return:
+    """
+    if not can_run_gpu_test():
+        return
+
+    os.environ['MASTER_PORT'] = str(np.random.randint(12000, 19000, 1)[0])
+    model, hparams = get_model()
+    trainer_options = dict(
+        show_progress_bar=False,
+        max_nb_epochs=1,
+        train_percent_check=0.4,
+        val_percent_check=0.2,
+        gpus=[0, 1],
+        distributed_backend='ddp'
+    )
+
+    run_gpu_model_test(trainer_options, model, hparams)
+
+
 def test_optimizer_return_options():
 
     trainer = Trainer()
@@ -670,7 +669,7 @@ def test_amp_single_gpu():
         show_progress_bar=True,
         max_nb_epochs=1,
         gpus=1,
-        distributed_backend='dp',
+        distributed_backend='ddp',
         use_amp=True
     )
 
@@ -979,6 +978,79 @@ def test_model_freeze_unfreeze():
     model.unfreeze()
 
 
+def test_amp_gpu_ddp_slurm_managed():
+    """
+    Make sure DDP + AMP work
+    :return:
+    """
+    if not can_run_gpu_test():
+        return
+
+    # simulate setting slurm flags
+    os.environ['MASTER_PORT'] = str(np.random.randint(12000, 19000, 1)[0])
+    os.environ['SLURM_LOCALID'] = str(0)
+
+    hparams = get_hparams()
+    model = LightningTestModel(hparams)
+
+    trainer_options = dict(
+        show_progress_bar=True,
+        max_nb_epochs=1,
+        gpus=[0],
+        distributed_backend='ddp',
+        use_amp=True
+    )
+
+    save_dir = init_save_dir()
+
+    # exp file to get meta
+    exp = get_exp(False)
+    exp.argparse(hparams)
+    exp.save()
+
+    # exp file to get weights
+    checkpoint = ModelCheckpoint(save_dir)
+
+    # add these to the trainer options
+    trainer_options['checkpoint_callback'] = checkpoint
+    trainer_options['experiment'] = exp
+
+    # fit model
+    trainer = Trainer(**trainer_options)
+    trainer.is_slurm_managing_tasks = True
+    result = trainer.fit(model)
+
+    # correct result and ok accuracy
+    assert result == 1, 'amp + ddp model failed to complete'
+
+    # test root model address
+    assert trainer.resolve_root_node_address('abc') == 'abc'
+    assert trainer.resolve_root_node_address('abc[23]') == 'abc23'
+    assert trainer.resolve_root_node_address('abc[23-24]') == 'abc23'
+    assert trainer.resolve_root_node_address('abc[23-24, 45-40, 40]') == 'abc23'
+
+    # test model loading with a map_location
+    pretrained_model = load_model(exp, save_dir, True)
+
+    # test model preds
+    run_prediction(model.test_dataloader, pretrained_model)
+
+    if trainer.use_ddp:
+        # on hpc this would work fine... but need to hack it for the purpose of the test
+        trainer.model = pretrained_model
+        trainer.optimizers, trainer.lr_schedulers = pretrained_model.configure_optimizers()
+
+    # test HPC loading / saving
+    trainer.hpc_save(save_dir, exp)
+    trainer.hpc_load(save_dir, on_gpu=True)
+
+    # test freeze on gpu
+    model.freeze()
+    model.unfreeze()
+
+    clear_save_dir()
+
+
 def test_cpu_model_with_amp():
     """
     Make sure model trains on CPU
@@ -1149,7 +1221,7 @@ def test_ddp_sampler_error():
         experiment=exp,
         show_progress_bar=False,
         max_nb_epochs=1,
-        gpus=2,
+        gpus=[0, 1],
         distributed_backend='ddp',
         use_amp=True
     )
@@ -1228,79 +1300,6 @@ class CurrentTestModel(
     trainer.test()
 
 
-def test_amp_gpu_ddp_slurm_managed():
-    """
-    Make sure DDP + AMP work
-    :return:
-    """
-    if not can_run_gpu_test():
-        return
-
-    # simulate setting slurm flags
-    os.environ['MASTER_PORT'] = str(np.random.randint(12000, 19000, 1)[0])
-    os.environ['SLURM_LOCALID'] = str(0)
-    os.environ['FAKE_SLURM_MANAGING_TASKS'] = str(1)
-
-    hparams = get_hparams()
-    model = LightningTestModel(hparams)
-
-    trainer_options = dict(
-        show_progress_bar=True,
-        max_nb_epochs=1,
-        gpus=2,
-        distributed_backend='ddp',
-        use_amp=True
-    )
-
-    save_dir = init_save_dir()
-
-    # exp file to get meta
-    exp = get_exp(False)
-    exp.argparse(hparams)
-    exp.save()
-
-    # exp file to get weights
-    checkpoint = ModelCheckpoint(save_dir)
-
-    # add these to the trainer options
-    trainer_options['checkpoint_callback'] = checkpoint
-    trainer_options['experiment'] = exp
-
-    # fit model
-    trainer = Trainer(**trainer_options)
-    result = trainer.fit(model)
-
-    # correct result and ok accuracy
-    assert result == 1, 'amp + ddp model failed to complete'
-
-    # test root model address
-    assert trainer.resolve_root_node_address('abc') == 'abc'
-    assert trainer.resolve_root_node_address('abc[23]') == 'abc23'
-    assert trainer.resolve_root_node_address('abc[23-24]') == 'abc23'
-    assert trainer.resolve_root_node_address('abc[23-24, 45-40, 40]') == 'abc23'
-
-    # test model loading with a map_location
-    pretrained_model = load_model(exp, save_dir, True)
-
-    # test model preds
-    run_prediction(model.test_dataloader, pretrained_model)
-
-    if trainer.use_ddp:
-        # on hpc this would work fine... but need to hack it for the purpose of the test
-        trainer.model = pretrained_model
-        trainer.optimizers, trainer.lr_schedulers = pretrained_model.configure_optimizers()
-
-    # test HPC loading / saving
-    trainer.hpc_save(save_dir, exp)
-    trainer.hpc_load(save_dir, on_gpu=True)
-
-    # test freeze on gpu
-    model.freeze()
-    model.unfreeze()
-
-    clear_save_dir()
-
-
 # ------------------------------------------------------------------------
 # UTILS
 # ------------------------------------------------------------------------
@@ -1463,14 +1462,13 @@ def assert_ok_test_acc(trainer):
 
 def can_run_gpu_test():
     if not torch.cuda.is_available():
-        warnings.warn('GPU test cannot run.'
+        warnings.warn('test_multi_gpu_model_ddp cannot run.'
                       ' Rerun on a GPU node to run this test')
         return False
     if not torch.cuda.device_count() > 1:
-        warnings.warn('GPU test cannot run.'
+        warnings.warn('test_multi_gpu_model_ddp cannot run.'
                       ' Rerun on a node with 2+ GPUs to run this test')
         return False
-
     return True