Repository: Lightning-AI/pytorch-lightning

Issue:
Trainer.scale_batch_size requires model.batch_size instead of model.hparams.batch_size
<!-- 
### Common bugs:
1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    
2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    
-->

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

`Trainer.scale_batch_size` only works if a model has the `batch_size` property and does not work with `model.hparams.batch_size` even though all documentation points to the reverse.

### To Reproduce

All of my hyperparameters are available as model.hparams like suggested in the documentation: ([hyperparameters, option 3](https://pytorch-lightning.readthedocs.io/en/0.8.4/hyperparameters.html#lightningmodule-hyperparameters).
This means that my `batch_size` is available as `model.hparams.batch_size`.


This should be fully compatible with the [documented example code](https://pytorch-lightning.readthedocs.io/en/0.8.4/training_tricks.html#auto-scaling-of-batch-size) of `Trainer.scale_batch_size()` since that code also uses `model.hparams.batch_size` instead of `model.batch_size`.

However, when I put my model in `Trainer.scale_batch_size`, I get the following error:

    pytorch_lightning.utilities.exceptions.MisconfigurationException: Field batch_size not found in `model.hparams`


### Example code

```
class LitModel(pl.LightningModule):
    def __init__(self, hparams):
        super().__init__()
        self.hparams = args

model = LitModel(args)
trainer = Trainer()
trainer.scale_batch_size(model)
```

### Expected behavior

Either `Trainer.scale_batch_size` should work with `model.hparams` or the error message, linked documentation examples and docstrings should all change (i.e. [here](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/training_tricks.py#L139), [here](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/training_tricks.py#L228) and [here](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/training_tricks.py#L233)).

(I would prefer the second option. I think that it should work with both `model.batch_size` and `model.hparams.batch_size`.)

### Environment

 - pytorch-lightning         0.8.4



Discussion:
Hi! thanks for your contribution!, great first issue!
it seems like a nice first issue, @wietsedv mind send a PR? :rabbit: 
Appears to be the same with the learning rate parameter.
A clean fix on the user side while waiting for the PR is to actually use `self.hparams.batch_size` and define `self.batch_size` as a property of your module:
```python
@property
def batch_size(self):
    return self.hparams.batch_size

@batch_size.setter
def batch_size(self, batch_size):
    self.hparams.batch_size = batch_size
```
That way you keep your hyper parameters together in case you want to dump them somewhere without having to add specific code.
From #1896 it seems that the problem is rather on the docs side than `scale_batch_size()`'s implementation.
`self.batch_size` is the correct location to look for the parameter, not `self.hparams.batch_size`.
My above fix is thus also obsolete.
> From #1896 it seems that the problem is rather on the docs side than scale_batch_size()'s implementation.
My above fix is thus also obsolete.

I just tried your fix and it seemed to work :)
Yes it does work, but from what they said in the PR I linked, `hparams` was just there as a temporary solution, and all hyper parameters are intended to be set as direct instance attributes in `__init__`.
My fix is obsolete regarding the intended usage of PL.

Resolving patch:
diff --git a/pytorch_lightning/trainer/training_tricks.py b/pytorch_lightning/trainer/training_tricks.py
--- a/pytorch_lightning/trainer/training_tricks.py
+++ b/pytorch_lightning/trainer/training_tricks.py
@@ -24,9 +24,10 @@
 from pytorch_lightning.callbacks import GradientAccumulationScheduler
 from pytorch_lightning.core.lightning import LightningModule
 from pytorch_lightning.loggers.base import DummyLogger
-from pytorch_lightning.utilities import AMPType
+from pytorch_lightning.utilities import AMPType, rank_zero_warn
 from pytorch_lightning.utilities.exceptions import MisconfigurationException
 from pytorch_lightning.utilities.memory import is_oom_error, garbage_collection_cuda
+from pytorch_lightning.utilities.parsing import lightning_hasattr, lightning_getattr, lightning_setattr
 
 try:
     from apex import amp
@@ -158,11 +159,15 @@ def scale_batch_size(self,
                algorithm is terminated
 
         """
-        if not hasattr(model, batch_arg_name):
-            if not hasattr(model.hparams, batch_arg_name):
-                raise MisconfigurationException(
-                    'Neither of `model.batch_size` and `model.hparams.batch_size` found.'
-                )
+        if not lightning_hasattr(model, batch_arg_name):
+            raise MisconfigurationException(
+                f'Field {batch_arg_name} not found in both `model` and `model.hparams`')
+        if hasattr(model, batch_arg_name) and hasattr(model, "hparams") and batch_arg_name in model.hparams:
+            rank_zero_warn(
+                f'Field `model.{batch_arg_name}` and `model.hparams.{batch_arg_name}` are mutually exclusive!'
+                f' `model.{batch_arg_name}` will be used as the initial batch size for scaling.'
+                f' If this is not the intended behavior, please remove either one.'
+            )
 
         if hasattr(model.train_dataloader, 'patch_loader_code'):
             raise MisconfigurationException('The batch scaling feature cannot be used with dataloaders'
@@ -268,15 +273,9 @@ def _adjust_batch_size(trainer,
 
     """
     model = trainer.get_model()
-    if hasattr(model, batch_arg_name):
-        batch_size = getattr(model, batch_arg_name)
-    else:
-        batch_size = getattr(model.hparams, batch_arg_name)
+    batch_size = lightning_getattr(model, batch_arg_name)
     if value:
-        if hasattr(model, batch_arg_name):
-            setattr(model, batch_arg_name, value)
-        else:
-            setattr(model.hparams, batch_arg_name, value)
+        lightning_setattr(model, batch_arg_name, value)
         new_size = value
         if desc:
             log.info(f'Batch size {batch_size} {desc}, trying batch size {new_size}')
@@ -284,7 +283,7 @@ def _adjust_batch_size(trainer,
         new_size = int(batch_size * factor)
         if desc:
             log.info(f'Batch size {batch_size} {desc}, trying batch size {new_size}')
-        setattr(model.hparams, batch_arg_name, new_size)
+        lightning_setattr(model, batch_arg_name, new_size)
     return new_size
 
 


Test patch:
diff --git a/tests/trainer/test_trainer_tricks.py b/tests/trainer/test_trainer_tricks.py
--- a/tests/trainer/test_trainer_tricks.py
+++ b/tests/trainer/test_trainer_tricks.py
@@ -196,28 +196,59 @@ def test_trainer_reset_correctly(tmpdir):
             f'Attribute {key} was not reset correctly after learning rate finder'
 
 
-@pytest.mark.parametrize('scale_arg', ['power', 'binsearch'])
-def test_trainer_arg(tmpdir, scale_arg):
-    """ Check that trainer arg works with bool input. """
+@pytest.mark.parametrize('scale_arg', ['power', 'binsearch', True])
+def test_auto_scale_batch_size_trainer_arg(tmpdir, scale_arg):
+    """ Test possible values for 'batch size auto scaling' Trainer argument. """
     tutils.reset_seed()
-
     hparams = EvalModelTemplate.get_default_hparams()
     model = EvalModelTemplate(**hparams)
-
     before_batch_size = hparams.get('batch_size')
-    # logger file to get meta
-    trainer = Trainer(
-        default_root_dir=tmpdir,
-        max_epochs=1,
-        auto_scale_batch_size=scale_arg,
-    )
-
+    trainer = Trainer(default_root_dir=tmpdir, max_epochs=1, auto_scale_batch_size=scale_arg)
     trainer.fit(model)
     after_batch_size = model.batch_size
     assert before_batch_size != after_batch_size, \
         'Batch size was not altered after running auto scaling of batch size'
 
 
+@pytest.mark.parametrize('use_hparams', [True, False])
+def test_auto_scale_batch_size_set_model_attribute(tmpdir, use_hparams):
+    """ Test that new batch size gets written to the correct hyperparameter attribute. """
+    tutils.reset_seed()
+
+    hparams = EvalModelTemplate.get_default_hparams()
+    before_batch_size = hparams.get('batch_size')
+
+    class HparamsEvalModelTemplate(EvalModelTemplate):
+
+        def dataloader(self, *args, **kwargs):
+            # artificially set batch_size so we can get a dataloader
+            # remove it immediately after, because we want only self.hparams.batch_size
+            setattr(self, "batch_size", before_batch_size)
+            dataloader = super().dataloader(*args, **kwargs)
+            del self.batch_size
+            return dataloader
+
+    model_class = HparamsEvalModelTemplate if use_hparams else EvalModelTemplate
+    model = model_class(**hparams)
+
+    trainer = Trainer(default_root_dir=tmpdir, max_epochs=1, auto_scale_batch_size=True)
+    trainer.fit(model)
+    after_batch_size = model.hparams.batch_size if use_hparams else model.batch_size
+    assert before_batch_size != after_batch_size
+
+
+def test_auto_scale_batch_size_duplicate_attribute_warning(tmpdir):
+    """ Test for a warning when model.batch_size and model.hparams.batch_size both present. """
+    hparams = EvalModelTemplate.get_default_hparams()
+    model = EvalModelTemplate(**hparams)
+    model.hparams = hparams
+    # now we have model.batch_size and model.hparams.batch_size
+    trainer = Trainer(default_root_dir=tmpdir, max_steps=1, auto_scale_batch_size=True)
+    expected_message = "Field `model.batch_size` and `model.hparams.batch_size` are mutually exclusive!"
+    with pytest.warns(UserWarning, match=expected_message):
+        trainer.fit(model)
+
+
 @pytest.mark.parametrize('scale_method', ['power', 'binsearch'])
 def test_call_to_trainer_method(tmpdir, scale_method):
     """ Test that calling the trainer method itself works. """