Repository: Lightning-AI/pytorch-lightning

Issue:
Trainer.on_gpu incorrectly set to False when specifying `gpus=0`
<!-- 
### Common bugs:
1. Tensorboard not showing in Jupyter-notebook see [issue 79](https://github.com/PyTorchLightning/pytorch-lightning/issues/79).    
2. PyTorch 1.1.0 vs 1.2.0 support [see FAQ](https://github.com/PyTorchLightning/pytorch-lightning#faq)    
-->

## 🐛 Bug

When creating a trainer with the arg `gpus=0`, the field `on_gpu` is always set `False`, even on machines with CUDA available.

The existing logic for `on_gpu` is:

```
self.on_gpu = True if (gpus and torch.cuda.is_available()) else False
```

is buggy because `0` is "falsy". It should probably be:

```
self.on_gpu = gpus is not None and torch.cuda.is_available()
```

### To Reproduce

```
trainer = trainer.Trainer(gpus=0, ...)
```



Discussion:
Hi! thanks for your contribution!, great first issue!
int value specifies the number of GPUs to train on. You can set gpus='0'
@rohitgr7 Sure! While I don't mean to turn this into an overly-academic conversation, but the type signature for the `gpus` arg is `Optional[Union[List[int], str, int]]`, which is somewhat misleading given that `gpus=0` doesn't work as expected.
yeah, I guess docs and docstrings can be improved here.
https://pytorch-lightning.readthedocs.io/en/latest/trainer.html#gpus
We also have a complete table of all possible configurations and their meaning here:
https://pytorch-lightning.readthedocs.io/en/latest/multi_gpu.html#select-gpu-devices
@rohitgr7 @pgeez  how about this clarification in the docs in the section that @rohitgr7 linked:

- Number of GPUs to train on (int)
- or Which GPUs to train on (list)
- can handle strings

```
# default used by the Trainer (ie: train on CPU)
trainer = Trainer(gpus=None)
# equivalent
trainer = Trainer(gpus=0)
```

Would that avoid the misunderstanding you had?
I don't think this is a documentation error, I think this is a bug. In other words, specifying `gpus=0` should be perfectly valid and supported because device `0` is a valid device.
No it's not a bug, this is expected behavior and documented that way. `gpus=int` should mean the number of devices, not the index. It is like this by design. 
Here #563 also an old discussions on these choices and why they are the way they are (see in particular @williamFalcon's comments)
> We also have a complete table of all possible configurations and their meaning here:
> https://pytorch-lightning.readthedocs.io/en/latest/multi_gpu.html#select-gpu-devices

In that table, would it be more user readable if we sort by `Meaning`?
Now they are sorted by `Type`
Ahh, ok! Thanks for the clarification all.

I'm not questioning the implementation, but was confused because in `trainer.py`, the doc for the `gpus` arg is 

```gpus: Which GPUs to train on.```

https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/trainer.py#L231

@awaelchli To your point about above documenting the equivalence of `gpus=None` and `gpus=0`, I like that idea.

@ydcjeff I think keeping them sorted by `Type` is reasonable.



Resolving patch:
diff --git a/pytorch_lightning/trainer/__init__.py b/pytorch_lightning/trainer/__init__.py
--- a/pytorch_lightning/trainer/__init__.py
+++ b/pytorch_lightning/trainer/__init__.py
@@ -414,8 +414,8 @@ def on_train_end(self, trainer, pl_module):
 gpus
 ^^^^
 
-- Number of GPUs to train on
-- or Which GPUs to train on
+- Number of GPUs to train on (int)
+- or which GPUs to train on (list)
 - can handle strings
 
 .. testcode::
@@ -423,6 +423,9 @@ def on_train_end(self, trainer, pl_module):
     # default used by the Trainer (ie: train on CPU)
     trainer = Trainer(gpus=None)
 
+    # equivalent
+    trainer = Trainer(gpus=0)
+
 Example::
 
     # int: train on 2 gpus
@@ -440,6 +443,9 @@ def on_train_end(self, trainer, pl_module):
     # uses 8 gpus in total
     trainer = Trainer(gpus=2, num_nodes=4)
 
+    # train only on GPUs 1 and 4 across nodes
+    trainer = Trainer(gpus=[1, 4], num_nodes=4)
+
 See Also:
     - `Multi-GPU training guide <multi_gpu.rst>`_
 


Test patch: