Repository: Lightning-AI/pytorch-lightning
Pull request: Error on zero length dataloaders
# Before submitting

- [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
- [x] Did you read the [contributor guideline](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/.github/CONTRIBUTING.md), Pull Request section?
- [x] Did you make sure to update the docs?   
- [ ] Did you write any new necessary tests?  
- [x] If you made a notable change (that affects users), did you update the [CHANGELOG](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/.github/CHANGELOG.md)?

## What does this PR do?
Fixes https://github.com/PyTorchLightning/pytorch-lightning/issues/1131.
Return a `ValueError` if user specified dataloaders has zero length (no batches). Current error returned are non-informative.

## PR review    
Anyone in the community is free to review the PR once the tests have passed.     
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

## Did you have fun?
Make sure you had fun coding 🙃


Problem statement:
Better message when DataLoader is wrong
On the verge between bug and improvement.

There was a bug in my Validation DataLoader and was returning irrelevant staff. Accidentally the length was 0.  Probably an edge case combination. The error I was getting during the validation sanity check was quite cryptic:

```
Traceback (most recent call last):
  File "UNet_WaveProp.py", line 174, in <module>
    trainer.fit(model)
  File "/mnt/RDS/home/code/pytorch-lightning/pytorch_lightning/trainer/trainer.py", line 629, in fit
    self.run_pretrain_routine(model)
  File "/mnt/RDS/home/code/pytorch-lightning/pytorch_lightning/trainer/trainer.py", line 809, in run_pretrain_routine
    False)
  File "/mnt/RDS/home/code/pytorch-lightning/pytorch_lightning/trainer/evaluation_loop.py", line 300, in evaluate
    eval_results = model.validation_epoch_end(outputs)
  File "UNet_WaveProp.py", line 138, in validation_epoch_end
    avg_loss = torch.stack([x['val_loss'] for x in outputs]).mean()
RuntimeError: stack expects a non-empty TensorList
```

I had to go through the code of pytorch-lightning for few hours to understand what was happening. 
Maybe a more informative message would make more sense?

One thing would be to check if the DataLoader's size is 0.
What do you think? I could take a stab at a PR.

Hints from issue discussion:
Hi! thanks for your contribution!, great first issue!
yeah, better error messing always helps, thx and PR is welcome! :]

Commit message: rebase

Patch to review:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 90c9f49566bae..3dfc810b8b911 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,6 +17,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 - Added a check that stops the training when loss or weights contain `NaN` or `inf` values. ([#1097](https://github.com/PyTorchLightning/pytorch-lightning/pull/1097))
 - Updated references to self.forward() to instead use the `__call__` interface. ([#1211](https://github.com/PyTorchLightning/pytorch-lightning/pull/1211))
 - Added support for `IterableDataset` when `val_check_interval=1.0` (default), this will trigger validation at the end of each epoch. ([#1283](https://github.com/PyTorchLightning/pytorch-lightning/pull/1283))
+- Added informative errors if user defined dataloader has zero length ([#1280](https://github.com/PyTorchLightning/pytorch-lightning/pull/1280))
 
 ### Changed
 
diff --git a/pytorch_lightning/trainer/data_loading.py b/pytorch_lightning/trainer/data_loading.py
index dced28144fbe8..f950ac49d8aa7 100644
--- a/pytorch_lightning/trainer/data_loading.py
+++ b/pytorch_lightning/trainer/data_loading.py
@@ -26,9 +26,14 @@
 
 
 def _has_len(dataloader: DataLoader) -> bool:
+    """ Checks if a given Dataloader has __len__ method implemented i.e. if
+    it is a finite dataloader or infinite dataloader """
     try:
         # try getting the length
-        _ = len(dataloader)
+        val = len(dataloader)
+        if val == 0:
+            raise ValueError('Dataloader returned 0 length. Please make sure'
+                             ' that your Dataloader atleast returns 1 batch')
         return True
     except TypeError:
         return False
diff --git a/tests/base/__init__.py b/tests/base/__init__.py
index 1e68469871d25..3c174ece038b3 100644
--- a/tests/base/__init__.py
+++ b/tests/base/__init__.py
@@ -25,7 +25,8 @@
     LightTestOptimizerWithSchedulingMixin,
     LightTestMultipleOptimizersWithSchedulingMixin,
     LightTestOptimizersWithMixedSchedulingMixin,
-    LightTestReduceLROnPlateauMixin
+    LightTestReduceLROnPlateauMixin,
+    LightZeroLenDataloader
 )
 
 
diff --git a/tests/base/mixins.py b/tests/base/mixins.py
index 1a05049f44f5f..4c249b1b71b18 100644
--- a/tests/base/mixins.py
+++ b/tests/base/mixins.py
@@ -255,6 +255,16 @@ def test_dataloader(self):
         return CustomInfDataloader(self._dataloader(train=False))
 
 
+class LightZeroLenDataloader:
+    """ Simple dataloader that has zero length. """
+
+    def train_dataloader(self):
+        dataloader = self._dataloader(train=True)
+        dataloader.dataset.data = dataloader.dataset.data[:0]
+        dataloader.dataset.targets = dataloader.dataset.targets[:0]
+        return dataloader
+
+
 class LightEmptyTestStep:
     """Empty test step."""
 
diff --git a/tests/trainer/test_dataloaders.py b/tests/trainer/test_dataloaders.py
index 194d67fe07858..6d2332cdf538b 100644
--- a/tests/trainer/test_dataloaders.py
+++ b/tests/trainer/test_dataloaders.py
@@ -16,7 +16,8 @@
     LightTrainDataloader,
     LightInfTrainDataloader,
     LightInfValDataloader,
-    LightInfTestDataloader
+    LightInfTestDataloader,
+    LightZeroLenDataloader
 )
 
 
@@ -458,3 +459,26 @@ class CurrentTestModel(
 
     # verify training completed
     assert result == 1
+
+
+def test_error_on_zero_len_dataloader(tmpdir):
+    """ Test that error is raised if a zero-length dataloader is defined """
+    tutils.reset_seed()
+
+    class CurrentTestModel(
+        LightZeroLenDataloader,
+        LightningTestModel
+    ):
+        pass
+
+    hparams = tutils.get_default_hparams()
+    model = CurrentTestModel(hparams)
+
+    # fit model
+    with pytest.raises(ValueError):
+        trainer = Trainer(
+            default_save_path=tmpdir,
+            max_epochs=1,
+            test_percent_check=0.5
+        )
+        trainer.fit(model)


Code review comments:
diff_hunk: @@ -26,9 +26,14 @@
 
 
 def _has_len(dataloader: DataLoader) -> bool:
+    """ Checks if a given Dataloader has __len__ method implemented i.e. if
+    it is a finite dataloader or infinite dataloader """
     try:
         # try getting the length
-        _ = len(dataloader)
+        val = len(dataloader)
+        if val == 0:
path: pytorch_lightning/trainer/data_loading.py
text: @user1:
```suggestion
        if len(dataloader) == 0:
```
what about generator like dataloader with fix size but not accessible because of generator type