Repository: Lightning-AI/pytorch-lightning
Pull request: Handle edge case for `find_usable_cuda_devices(0)`
## What does this PR do?

Fixes #18720

Handles an edge case for input validation in the `find_usable_cuda_devices` function.


<!-- readthedocs-preview pytorch-lightning start -->
----
:books: Documentation preview :books:: https://pytorch-lightning--18722.org.readthedocs.build/en/18722/

<!-- readthedocs-preview pytorch-lightning end -->

cc @borda @carmocca @justusschock @awaelchli

Problem statement:
find_usable_cuda_devices always returns GPU 0
### Bug description

The function `find_usable_cuda_device` always returns the first GPU (index `0`), even though if it's being currently utilized and there exists another GPU (index `1`) with 0% utilization.

Reproducible example:
```python
>>>from pytorch_lightning.accelerators import find_usable_cuda_devices
>>>find_usable_cuda_devices(0)
[0]
```

Screen capture of `nvidia-smi`:
```
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.161.03   Driver Version: 470.161.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-PCIE...  Off  | 00000000:AF:00.0 Off |                    0 |
| N/A   45C    P0    52W / 250W |  14642MiB / 16160MiB |     16%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-PCIE...  Off  | 00000000:D8:00.0 Off |                    0 |
| N/A   39C    P0    39W / 250W |    719MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
```

As you can see, the GPU 1 is currently free, but `find_usable_cuda_devices` returns `[0]` nonetheless.

### What version are you seeing the problem on?

v2.0

### How to reproduce the bug

```python
>>>from pytorch_lightning.accelerators import find_usable_cuda_devices
>>>find_usable_cuda_devices(0)
[0]
```


### Error messages and logs

```
# Error messages and logs here please
```


### Environment

<details>
  <summary>Current environment</summary>

```
#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):
```

</details>


### More info

I am running this code inside a cluster with many GPUs, although the container my code is running in only has access to two GPUs. These two GPUs are mine alone (not shared with other users).

cc @justusschock @awaelchli

Hints from issue discussion:
Hi @TheAeryan 

The docstring says at the very beginning:

>     A GPU is considered usable if we can successfully move a tensor to the device, and this is what this function
>     tests for each GPU on the system until the target number of usable devices is found.
> 
>     A subset of GPUs on the system might be used by other processes, and if the GPU is configured to operate in
>     'exclusive' mode (configurable by the admin), then only one process is allowed to occupy it.

Both of your GPUs are usable, and so if you do

```py
find_usable_cuda_devices(2)
```

the expectation is that it will return [0, 1]. 

`find_usable_cuda_devices(0)` shouldn't be a valid input, you can't request to find 0 usable GPU devices. This is an edge case we would want to fix.

Commit message: chlog

Patch to review:
diff --git a/src/lightning/fabric/CHANGELOG.md b/src/lightning/fabric/CHANGELOG.md
index bdd59c40e5380..d47a281668a9b 100644
--- a/src/lightning/fabric/CHANGELOG.md
+++ b/src/lightning/fabric/CHANGELOG.md
@@ -221,6 +221,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 - Fixed redundant input-type casting in FSDP precision ([#18630](https://github.com/Lightning-AI/lightning/pull/18630))
 
 
+- Fixed an issue with `find_usable_cuda_devices(0)` incorrectly returning a list of devices ([#18722](https://github.com/Lightning-AI/lightning/pull/18722))
+
+
 ## [2.0.9] - 2023-09-14
 
 ### Fixed
diff --git a/src/lightning/fabric/accelerators/cuda.py b/src/lightning/fabric/accelerators/cuda.py
index 68d2237d93abd..d03e8a2e2baa6 100644
--- a/src/lightning/fabric/accelerators/cuda.py
+++ b/src/lightning/fabric/accelerators/cuda.py
@@ -90,6 +90,8 @@ def find_usable_cuda_devices(num_devices: int = -1) -> List[int]:
         both processes determine that the device is unoccupied, leading into one of them crashing later on.
 
     """
+    if num_devices == 0:
+        raise ValueError("`num_devices=0` is not a valid number of devices to look for. Pass a positive number.")
     visible_devices = _get_all_visible_cuda_devices()
     if not visible_devices:
         raise ValueError(
diff --git a/tests/tests_fabric/accelerators/test_cuda.py b/tests/tests_fabric/accelerators/test_cuda.py
index ae1f146bf6fad..d4e514b9f34a0 100644
--- a/tests/tests_fabric/accelerators/test_cuda.py
+++ b/tests/tests_fabric/accelerators/test_cuda.py
@@ -157,3 +157,7 @@ def test_find_usable_cuda_devices_error_handling():
         "lightning.fabric.accelerators.cuda.torch.tensor"
     ):
         assert find_usable_cuda_devices(-1) == [0, 1, 2, 3, 4]
+
+    # Asking to find 0 usable GPUs is something only a philosopher would be interested in
+    with pytest.raises(ValueError, match="`num_devices=0` is not a valid number"):
+        find_usable_cuda_devices(0)


Code review comments:
diff_hunk: @@ -90,6 +90,8 @@ def find_usable_cuda_devices(num_devices: int = -1) -> List[int]:
         both processes determine that the device is unoccupied, leading into one of them crashing later on.
 
     """
+    if num_devices == 0:
+        raise ValueError("`num_devices=0` is not a valid number of devices to look for. Pass a positive number.")
path: src/lightning/fabric/accelerators/cuda.py
text: @user1:
One could argue that this should return an empty list instead. If there are valid reasons to call `find_usable_cuda_devices(0)`, then returning an empty list would make the user's code nicer than having to catch this ValueError