Repository: Lightning-AI/pytorch-lightning

Issue:
Mixing hparams and arguments in LightningModule.__init__() crashes load_from_checkpoint()
<!-- 
### Common bugs:
1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    
2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    
-->

## 🐛 Bug

Right now, if you initialize a Lightning Module with a mixture of a Namespace (hparams) as well as additional arguments (say to a Dataset), load_from_checkpoint can't recover.

### To Reproduce

Create a LightningModule as follows:

```python
class Model(pl.LightningModule):
      def __init__(self, hparams, train_dataset, val_dataset):
              self.hparams = hparams
              self.train_dset, self.val_dset = train_dataset, val_dataset
              ...
```

Run training, then try to restore from checkpoint, via:

```python
nn = Model.restore_from_checkpoint(<PATH>, train_dataset=None, val_dataset=None)
```

### Expected behavior

Ideally, you'd just be able to pass in the additional arguments (as above) and everything would work.

Mixing hparams and arguments in LightningModule.__init__() crashes load_from_checkpoint()
<!-- 
### Common bugs:
1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    
2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    
-->

## 🐛 Bug

Right now, if you initialize a Lightning Module with a mixture of a Namespace (hparams) as well as additional arguments (say to a Dataset), load_from_checkpoint can't recover.

### To Reproduce

Create a LightningModule as follows:

```python
class Model(pl.LightningModule):
      def __init__(self, hparams, train_dataset, val_dataset):
              self.hparams = hparams
              self.train_dset, self.val_dset = train_dataset, val_dataset
              ...
```

Run training, then try to restore from checkpoint, via:

```python
nn = Model.restore_from_checkpoint(<PATH>, train_dataset=None, val_dataset=None)
```

### Expected behavior

Ideally, you'd just be able to pass in the additional arguments (as above) and everything would work.



Discussion:
Hi! thanks for your contribution!, great first issue!
Very interesting issue. I think the whole restore_from_checkpoint could use a bit better documentation. I think you are trying to use checkpoints for something that was not the intention. 
I think what you want is run the model without training (this is why the train and val datasets are empty). In this case I would try falling back to plain torch solution, such as saving and reusing the state_dict:
```
torch.save(nn.state_dict(), SAVE_PATH)
loaded_model = nn(hparams)
loaded_model.load_state_dict(torch.load(SAVE_PATH))
loaded_model.eval()
print("Model's state_dict:")
for param_tensor in loaded_model.state_dict():
  print(param_tensor, "\t", loaded_model.state_dict()[param_tensor].size())
```
That makes sense for this usecase --> but say I want to pause/resume training later on (the intended usecase of restore from checkpoint). Here, I really want to be able to leverage the fact that Lightning remembers my hyperparameters, and I want to be able to just pass in the additional arguments (like the datasets I've constructed).
This is the behavior in 0.7.3

https://pytorch-lightning.readthedocs.io/en/0.7.3/lightning-module.html#lightningmodule-class

```python
# or load passing whatever args the model takes to load
MyLightningModule.load_from_checkpoint(
    'path/to/checkpoint.ckpt',
    learning_rate=0.1,
    layers=2,
    pretrained_model=some_model
)
```
This doesn't seem to be true if I explicitly pass a `hparams` argument... only if I break out each of the arguments and pass them to the __init__() method.
share a colab? this should be true on 0.7.3. 

For now you can do Trainer(PATH, **hparams)
https://colab.research.google.com/drive/1WmuZfOQxyi4nF_YvjFcJrcpVhNvuCXST
Hi! thanks for your contribution!, great first issue!
Very interesting issue. I think the whole restore_from_checkpoint could use a bit better documentation. I think you are trying to use checkpoints for something that was not the intention. 
I think what you want is run the model without training (this is why the train and val datasets are empty). In this case I would try falling back to plain torch solution, such as saving and reusing the state_dict:
```
torch.save(nn.state_dict(), SAVE_PATH)
loaded_model = nn(hparams)
loaded_model.load_state_dict(torch.load(SAVE_PATH))
loaded_model.eval()
print("Model's state_dict:")
for param_tensor in loaded_model.state_dict():
  print(param_tensor, "\t", loaded_model.state_dict()[param_tensor].size())
```
That makes sense for this usecase --> but say I want to pause/resume training later on (the intended usecase of restore from checkpoint). Here, I really want to be able to leverage the fact that Lightning remembers my hyperparameters, and I want to be able to just pass in the additional arguments (like the datasets I've constructed).
This is the behavior in 0.7.3

https://pytorch-lightning.readthedocs.io/en/0.7.3/lightning-module.html#lightningmodule-class

```python
# or load passing whatever args the model takes to load
MyLightningModule.load_from_checkpoint(
    'path/to/checkpoint.ckpt',
    learning_rate=0.1,
    layers=2,
    pretrained_model=some_model
)
```
This doesn't seem to be true if I explicitly pass a `hparams` argument... only if I break out each of the arguments and pass them to the __init__() method.
share a colab? this should be true on 0.7.3. 

For now you can do Trainer(PATH, **hparams)
https://colab.research.google.com/drive/1WmuZfOQxyi4nF_YvjFcJrcpVhNvuCXST

Resolving patch:
diff --git a/pytorch_lightning/core/lightning.py b/pytorch_lightning/core/lightning.py
--- a/pytorch_lightning/core/lightning.py
+++ b/pytorch_lightning/core/lightning.py
@@ -1434,6 +1434,7 @@ def load_from_checkpoint(
         it stores the hyperparameters in the checkpoint if you initialized your :class:`LightningModule`
         with an argument called ``hparams`` which is a :class:`~argparse.Namespace`
         (output of :meth:`~argparse.ArgumentParser.parse_args` when parsing command line arguments).
+        Any other arguments specified through \*args and \*\*kwargs will be passed to the model.
 
         Example:
             .. code-block:: python
@@ -1493,7 +1494,7 @@ def __init__(self, hparams):
                 # or load passing whatever args the model takes to load
                 MyLightningModule.load_from_checkpoint(
                     'path/to/checkpoint.ckpt',
-                    learning_rate=0.1,
+                    learning_rate=0.1, # These arguments will be passed to the model using **kwargs
                     layers=2,
                     pretrained_model=some_model
                 )
@@ -1544,10 +1545,7 @@ def _load_model_state(cls, checkpoint: Dict[str, Any], *args, **kwargs) -> 'Ligh
 
         # load the state_dict on the model automatically
         model_args = [hparams] if hparams else []
-        if len(model_args) > 0:
-            model = cls(*model_args)
-        else:
-            model = cls(*args, **kwargs)
+        model = cls(*model_args, *args, **kwargs)
         model.load_state_dict(checkpoint['state_dict'])
 
         # give model a chance to load something


Test patch: