Repository: Lightning-AI/pytorch-lightning

Issue:
Incompatible Trainer.test()
## 🐛 Bug

Trainer.test() stops on missing definitions of train & validation dataloaders methods.

### To Reproduce

Steps to reproduce the behavior:

- Define LightningModule without train & validation dataloaders
- Train such model with data loaders provided to `Trainer.fit(...)` method
- Load model from checkpoint and test it with `Trainer.test(...)` method

test failes on :
~~~python
'No `train_dataloader()` method defined. Lightning `Trainer` expects as minimum a'
pytorch_lightning.utilities.exceptions.MisconfigurationException: No `train_dataloader()` method defined. Lightning `Trainer` expects as minimum a `training_step()`, `training_dataloader()` and `configure_optimizers()` to be defined.
~~~

#### Code sample
~~~python
test_data = data_loader(...)
estimator = MyEstimator.load_from_checkpoint(checkpoint_path=CKPT_PATH)
trainer = pl.Trainer(...)
trainer.test(model=estimator, test_dataloaders=test_data)
~~~

### Expected behaviour

Run the test with provided test data loader, and do not stop on condition related to missing train & validation dataloaders which actually are not mandatory.

### Environment

~~~
* Packages:
        - numpy:             1.18.4
        - pyTorch_debug:     False
        - pyTorch_version:   1.5.0
        - pytorch-lightning: 0.7.5
        - tensorboard:       2.2.1
        - tqdm:              4.46.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                - ELF
        - processor:         
        - python:            3.8.2
        - version:           #1 SMP Wed, 29 Apr 2020 16:23:03 +0000
~~~



Discussion:
Hi! thanks for your contribution!, great first issue!
This issue relates to #1195, `fit` and `test` should be decoupled. Is there any consensus or proposition of further steps?
Can't we just add 
```python
if not self.testing:
    self.check_model_configuration(model)
```
[here](https://github.com/PyTorchLightning/pytorch-lightning/blob/2a2f303ae91a4a17b1cd8127e5b811e38cb2d978/pytorch_lightning/trainer/trainer.py#L732) because during testing the testing configuraton is already checked [here](https://github.com/PyTorchLightning/pytorch-lightning/blob/2a2f303ae91a4a17b1cd8127e5b811e38cb2d978/pytorch_lightning/trainer/trainer.py#L970).
Also, using `trainer.test(model)` on loaded checkpoint doesn't work without explicitly passing `test_dataloader` parameter even if all `*_dataloader` methods are defined. [Example](https://colab.research.google.com/drive/1qtbS48QrNqGW4qXvfEdMdwIcURjBc8ne?usp=sharing).
This use case isn't documented in the 0.7.5 documentation but evidently was relevant at some point: https://pytorch-lightning.readthedocs.io/en/latest/test_set.html#test-pre-trained-model.
I suggest to either:
* fix Test set documentation
* (preferred) allow to call `trainer.test(model)` without specifying `test_dataloader` parameter. After all, what's the point of defining `Model.test_dataloader` method?
@iakremnev can you fix the link? It's asking for request access.
@rohitgr7 oops, fixed it.
Error coming from [this line](https://github.com/PyTorchLightning/pytorch-lightning/blob/3a642601e84c3abf1f1b438f9acc932a1f150f7f/pytorch_lightning/trainer/trainer.py#L1045). Why is it checking whether test_dataloader() returns something or not. If we just check whether it is overridden or not wouldn't that be enough just like it is done in case of `.fit()` [here](https://github.com/PyTorchLightning/pytorch-lightning/blob/3a642601e84c3abf1f1b438f9acc932a1f150f7f/pytorch_lightning/trainer/trainer.py#L988).
I would suggest
```python
gave_test_loader = isinstance(model.test_dataloader, _PatchDataLoader) or self.is_overridden('test_dataloader', model)
```
[here](https://github.com/PyTorchLightning/pytorch-lightning/blob/3a642601e84c3abf1f1b438f9acc932a1f150f7f/pytorch_lightning/trainer/trainer.py#L1045), and
```python
if not self.testing:
    self.check_model_configuration(model)
``` 
[here](https://github.com/PyTorchLightning/pytorch-lightning/blob/2a2f303ae91a4a17b1cd8127e5b811e38cb2d978/pytorch_lightning/trainer/trainer.py#L732)
it is the same as #1754

Resolving patch:
diff --git a/pytorch_lightning/trainer/evaluation_loop.py b/pytorch_lightning/trainer/evaluation_loop.py
--- a/pytorch_lightning/trainer/evaluation_loop.py
+++ b/pytorch_lightning/trainer/evaluation_loop.py
@@ -334,20 +334,13 @@ def _evaluate(self, model: LightningModule, dataloaders, max_batches: int, test_
         return eval_results
 
     def run_evaluation(self, test_mode: bool = False):
-        # when testing make sure user defined a test step
-        if test_mode and not self.is_overridden('test_step'):
-            raise MisconfigurationException(
-                "You called `.test()` without defining model's `.test_step()`."
-                " Please define and try again")
-
         # hook
         model = self.get_model()
         model.on_pre_performance_check()
 
         # select dataloaders
         if test_mode:
-            if self.test_dataloaders is None:
-                self.reset_test_dataloader(model)
+            self.reset_test_dataloader(model)
 
             dataloaders = self.test_dataloaders
             max_batches = self.num_test_batches
diff --git a/pytorch_lightning/trainer/trainer.py b/pytorch_lightning/trainer/trainer.py
--- a/pytorch_lightning/trainer/trainer.py
+++ b/pytorch_lightning/trainer/trainer.py
@@ -1021,9 +1021,6 @@ def test(
             else:
                 self.__attach_dataloaders(self.model, test_dataloaders=test_dataloaders)
 
-        # give proper warnings if user only passed in loader without hooks
-        self.check_testing_model_configuration(model if model else self.model)
-
         if model is not None:
             self.model = model
             self.fit(model)
@@ -1042,44 +1039,45 @@ def test(
 
     def check_model_configuration(self, model: LightningModule):
         r"""
-        Checks that the model is configured correctly before training is started.
+        Checks that the model is configured correctly before training or testing is started.
 
         Args:
-            model: The model to test.
+            model: The model to check the configuration.
 
         """
         # Check training_step, train_dataloader, configure_optimizer methods
-        if not self.is_overridden('training_step', model):
-            raise MisconfigurationException(
-                'No `training_step()` method defined. Lightning `Trainer` expects as minimum a'
-                ' `training_step()`, `train_dataloader()` and `configure_optimizers()` to be defined.')
-
-        if not self.is_overridden('train_dataloader', model):
-            raise MisconfigurationException(
-                'No `train_dataloader()` method defined. Lightning `Trainer` expects as minimum a'
-                ' `training_step()`, `train_dataloader()` and `configure_optimizers()` to be defined.')
-
-        if not self.is_overridden('configure_optimizers', model):
-            raise MisconfigurationException(
-                'No `configure_optimizers()` method defined. Lightning `Trainer` expects as minimum a'
-                ' `training_step()`, `train_dataloader()` and `configure_optimizers()` to be defined.')
-
-        # Check val_dataloader, validation_step and validation_epoch_end
-        if self.is_overridden('val_dataloader', model):
-            if not self.is_overridden('validation_step', model):
-                raise MisconfigurationException('You have passed in a `val_dataloader()`'
-                                                ' but have not defined `validation_step()`.')
+        if not self.testing:
+            if not self.is_overridden('training_step', model):
+                raise MisconfigurationException(
+                    'No `training_step()` method defined. Lightning `Trainer` expects as minimum a'
+                    ' `training_step()`, `training_dataloader()` and `configure_optimizers()` to be defined.')
+
+            if not self.is_overridden('train_dataloader', model):
+                raise MisconfigurationException(
+                    'No `train_dataloader()` method defined. Lightning `Trainer` expects as minimum a'
+                    ' `training_step()`, `training_dataloader()` and `configure_optimizers()` to be defined.')
+
+            if not self.is_overridden('configure_optimizers', model):
+                raise MisconfigurationException(
+                    'No `configure_optimizers()` method defined. Lightning `Trainer` expects as minimum a'
+                    ' `training_step()`, `training_dataloader()` and `configure_optimizers()` to be defined.')
+
+            # Check val_dataloader, validation_step and validation_epoch_end
+            if self.is_overridden('val_dataloader', model):
+                if not self.is_overridden('validation_step', model):
+                    raise MisconfigurationException('You have passed in a `val_dataloader()`'
+                                                    ' but have not defined `validation_step()`.')
+                else:
+                    if not self.is_overridden('validation_epoch_end', model):
+                        rank_zero_warn(
+                            'You have defined a `val_dataloader()` and have defined a `validation_step()`,'
+                            ' you may also want to define `validation_epoch_end()` for accumulating stats.',
+                            RuntimeWarning
+                        )
             else:
-                if not self.is_overridden('validation_epoch_end', model):
-                    rank_zero_warn(
-                        'You have defined a `val_dataloader()` and have defined a `validation_step()`,'
-                        ' you may also want to define `validation_epoch_end()` for accumulating stats.',
-                        RuntimeWarning
-                    )
-        else:
-            if self.is_overridden('validation_step', model):
-                raise MisconfigurationException('You have defined `validation_step()`,'
-                                                ' but have not passed in a val_dataloader().')
+                if self.is_overridden('validation_step', model):
+                    raise MisconfigurationException('You have defined `validation_step()`,'
+                                                    ' but have not passed in a `val_dataloader()`.')
 
         # Check test_dataloader, test_step and test_epoch_end
         if self.is_overridden('test_dataloader', model):
@@ -1092,25 +1090,10 @@ def check_model_configuration(self, model: LightningModule):
                         'You have defined a `test_dataloader()` and have defined a `test_step()`, you may also want to'
                         ' define `test_epoch_end()` for accumulating stats.', RuntimeWarning
                     )
-
-    def check_testing_model_configuration(self, model: LightningModule):
-
-        has_test_step = self.is_overridden('test_step', model)
-        has_test_epoch_end = self.is_overridden('test_epoch_end', model)
-        gave_test_loader = self.is_overridden('test_dataloader', model)
-
-        if gave_test_loader and not has_test_step:
-            raise MisconfigurationException('You passed in a `test_dataloader` but did not implement `test_step()`')
-
-        if has_test_step and not gave_test_loader:
-            raise MisconfigurationException('You defined `test_step()` but did not implement'
-                                            ' `test_dataloader` nor passed in `.fit(test_dataloaders`.')
-
-        if has_test_step and gave_test_loader and not has_test_epoch_end:
-            rank_zero_warn(
-                'You passed  in a `test_dataloader` and have defined a `test_step()`, you may also want to'
-                ' define `test_epoch_end()` for accumulating stats.', RuntimeWarning
-            )
+        else:
+            if self.testing and self.is_overridden('test_step', model):
+                raise MisconfigurationException('You have defined `test_step()` but did not'
+                                                ' implement `test_dataloader` nor passed in `.test(test_dataloader)`.')
 
 
 class _PatchDataLoader(object):


Test patch: