Repository: Lightning-AI/pytorch-lightning

Issue:
Trainer.from_argparse_args fails on unknown kwargs
<!-- 
### Common bugs:
1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    
2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    
-->

## 🐛 Bug

<!-- A clear and concise description of what the bug is. -->

Since `**kwargs` was removed from Trainer's init in #1820, initializing `Trainer` objects fails if you have any non `Trainer` specific arguments in your parser.

If this is the expected behavior, [the docs](https://pytorch-lightning.readthedocs.io/en/latest/hyperparameters.html#multiple-lightning-modules) should be updated to reflect the workaround I mention below, as a few of them would currently fail.

### To Reproduce

#### This works
```python
parser = ArgumentParser()
parser = Trainer.add_argparse_args(parser)
args = parser.parse_args("")
trainer = Trainer.from_argparse_args(args)
```

#### Trainer init fails on unexpected kwarg 'script_specific_arg'
```python
parser = ArgumentParser()
parser = Trainer.add_argparse_args(parser)

parser.add_argument('--script_specific_arg', type=str, default='hope this works')

args = parser.parse_args('')
trainer = Trainer.from_argparse_args(args)
```

#### Trainer init fails on unexpected kwarg 'some_argument'
```python
class SomeModel(LightningModule):

    def __init__(self, hparams):
        super().__init__()
        self.hparams = hparams

    @staticmethod
    def add_model_specific_args(parent_parser):
        parser = ArgumentParser(parents=[parent_parser], add_help=False)
        parser.add_argument('--some_argument', type=int, default=128)
        return parser


parser = ArgumentParser()
parser = Trainer.add_argparse_args(parser)

parser = SomeModel.add_model_specific_args(parser)

args = parser.parse_args("")
trainer = Trainer.from_argparse_args(args)
```

#### You can get around it if you init Trainer on temp args

```python
parser = ArgumentParser()
parser = Trainer.add_argparse_args(parser)

# Grab only the trainer args and init it right away
temp_args, _ = parser.parse_known_args('')
trainer = Trainer.from_argparse_args(temp_args)

parser.add_argument('--script_specific_arg', type=str, default='hope this works')
args = parser.parse_args("")
```

### Expected behavior

`Trainer.from_argparse_args` should ignore unknown kwargs.

### Environment

- Google Colab
- Current master branch (`0.7.7.dev0`)

### Additional context

- No error if using stable PyPi version (`0.7.6`)



Discussion:
@awaelchli let’s add back kwargs? why did we remove them?
We removed it because
`Trainer(checkpont_calbuck=False)`
would be accepted even though it's misspelled. This is bad, so we have to remove the kwargs from Trainer.

The issue described here is not with Trainer. It is with `from_argparse_args`, it should pick only the valid args and ignore the others.

For example, in `from_argparse_args` we could inspect the Trainer and get a list of accepted args. If that's possible, this would solve the problem.
Here's how we could do it:
```python
from pytorch_lightning import Trainer
variables = Trainer.__init__.__code__.co_varnames
print(variables)
```
We get a list of accepted args

Resolving patch:
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -730,20 +730,32 @@ def parse_argparser(arg_parser: Union[ArgumentParser, Namespace]) -> Namespace:
 
     @classmethod
     def from_argparse_args(cls, args: Union[Namespace, ArgumentParser], **kwargs) -> 'Trainer':
-        """create an instance from CLI arguments
+        """
+        Create an instance from CLI arguments.
+
+        Args:
+            args: The parser or namespace to take arguments from. Only known arguments will be
+                parsed and passed to the :class:`Trainer`.
+            **kwargs: Additional keyword arguments that may override ones in the parser or namespace.
+                These must be valid Trainer arguments.
 
         Example:
             >>> parser = ArgumentParser(add_help=False)
             >>> parser = Trainer.add_argparse_args(parser)
+            >>> parser.add_argument('--my_custom_arg', default='something')  # doctest: +SKIP
             >>> args = Trainer.parse_argparser(parser.parse_args(""))
-            >>> trainer = Trainer.from_argparse_args(args)
+            >>> trainer = Trainer.from_argparse_args(args, logger=False)
         """
         if isinstance(args, ArgumentParser):
-            args = Trainer.parse_argparser(args)
+            args = cls.parse_argparser(args)
         params = vars(args)
-        params.update(**kwargs)
 
-        return cls(**params)
+        # we only want to pass in valid Trainer args, the rest may be user specific
+        valid_kwargs = inspect.signature(cls.__init__).parameters
+        trainer_kwargs = dict((name, params[name]) for name in valid_kwargs if name in params)
+        trainer_kwargs.update(**kwargs)
+
+        return cls(**trainer_kwargs)
 
     @property
     def num_gpus(self) -> int:


Test patch:
diff --git a/tests/trainer/test_trainer_cli.py b/tests/trainer/test_trainer_cli.py
--- a/tests/trainer/test_trainer_cli.py
+++ b/tests/trainer/test_trainer_cli.py
@@ -1,5 +1,6 @@
 import inspect
 import pickle
+import sys
 from argparse import ArgumentParser, Namespace
 from unittest import mock
 
@@ -110,3 +111,28 @@ def test_argparse_args_parsing(cli_args, expected):
     for k, v in expected.items():
         assert getattr(args, k) == v
     assert Trainer.from_argparse_args(args)
+
+
+@pytest.mark.skipif(
+    sys.version_info < (3, 7),
+    reason="signature inspection while mocking is not working in Python < 3.7 despite autospec"
+)
+@pytest.mark.parametrize(['cli_args', 'extra_args'], [
+    pytest.param({}, {}),
+    pytest.param({'logger': False}, {}),
+    pytest.param({'logger': False}, {'logger': True}),
+    pytest.param({'logger': False}, {'checkpoint_callback': True}),
+])
+def test_init_from_argparse_args(cli_args, extra_args):
+    unknown_args = dict(unknown_arg=0)
+
+    # unkown args in the argparser/namespace should be ignored
+    with mock.patch('pytorch_lightning.Trainer.__init__', autospec=True, return_value=None) as init:
+        trainer = Trainer.from_argparse_args(Namespace(**cli_args, **unknown_args), **extra_args)
+        expected = dict(cli_args)
+        expected.update(extra_args)  # extra args should override any cli arg
+        init.assert_called_with(trainer, **expected)
+
+    # passing in unknown manual args should throw an error
+    with pytest.raises(TypeError, match=r"__init__\(\) got an unexpected keyword argument 'unknown_arg'"):
+        Trainer.from_argparse_args(Namespace(**cli_args), **extra_args, **unknown_args)