Repository: vllm-project/vllm
Pull request: Fix error message on `TORCH_CUDA_ARCH_LIST`
Fixes #1225 

This PR fixes the error message when the `TORCH_CUDA_ARCH_LIST` includes an unsupported CUDA architecture.

Problem statement:
Installation Error
When I install vLLM, I have pull the docker as the tutorial and install from source. But there are an error encountered, how can I fix this:
`ValueError: Unsupported CUDA arch (5.2). Valid CUDA arch strings are: ['7.0', '7.5', '8.0', '8.6', '8.9', '9.0', '7.0+PTX', '7.5+PTX', '8.0+PTX', '8.6+PTX', '8.9+PTX', '9.0+PTX'].`

Hints from issue discussion:
I managed to get it working by setting `ENV TORCH_CUDA_ARCH_LIST="8.0;8.6;8.9"` (depends on your GPU) but I think an `ENV TORCH_CUDA_ARCH_LIST=""` could also work.

I believe the check [here](https://github.com/vllm-project/vllm/blob/v0.2.0/setup.py#L59-L63) should be a
> if none of the architectures in `arch_list` are in `valid_arch_strs`, then raise an error

instead of

>if **one** of the architectures in `arch_list` is not in `valid_arch_strs`, then raise an error

Commit message: Fix error msg

Patch to review:
diff --git a/setup.py b/setup.py
index 8b2ad97dd540..815d7834cf93 100644
--- a/setup.py
+++ b/setup.py
@@ -50,16 +50,17 @@ def get_torch_arch_list() -> Set[str]:
     # not give the best performance on the newer architectures, it provides
     # forward compatibility.
     valid_arch_strs = SUPPORTED_ARCHS + [s + "+PTX" for s in SUPPORTED_ARCHS]
-    arch_list = os.environ.get("TORCH_CUDA_ARCH_LIST", None)
-    if arch_list is None:
+    env_arch_list = os.environ.get("TORCH_CUDA_ARCH_LIST", None)
+    if env_arch_list is None:
         return set()
 
     # List are separated by ; or space.
-    arch_list = arch_list.replace(" ", ";").split(";")
+    arch_list = env_arch_list.replace(" ", ";").split(";")
     for arch in arch_list:
         if arch not in valid_arch_strs:
             raise ValueError(
-                f"Unsupported CUDA arch ({arch}). "
+                f"Unsupported CUDA arch ({arch}) is included in the "
+                f"`TORCH_CUDA_ARCH_LIST` env variable ({env_arch_list}). "
                 f"Valid CUDA arch strings are: {valid_arch_strs}.")
     return set(arch_list)
 


Code review comments:
diff_hunk: @@ -50,16 +50,17 @@ def get_torch_arch_list() -> Set[str]:
     # not give the best performance on the newer architectures, it provides
     # forward compatibility.
     valid_arch_strs = SUPPORTED_ARCHS + [s + "+PTX" for s in SUPPORTED_ARCHS]
-    arch_list = os.environ.get("TORCH_CUDA_ARCH_LIST", None)
-    if arch_list is None:
+    env_arch_list = os.environ.get("TORCH_CUDA_ARCH_LIST", None)
+    if env_arch_list is None:
         return set()
 
     # List are separated by ; or space.
-    arch_list = arch_list.replace(" ", ";").split(";")
+    arch_list = env_arch_list.replace(" ", ";").split(";")
     for arch in arch_list:
         if arch not in valid_arch_strs:
             raise ValueError(
-                f"Unsupported CUDA arch ({arch}). "
+                f"Unsupported CUDA arch ({arch}) is included in the "
+                f"`TORCH_CUDA_ARCH_LIST` env variable ({env_arch_list}). "
                 f"Valid CUDA arch strings are: {valid_arch_strs}.")
     return set(arch_list)
path: setup.py
text: @user1:
This doesn't quite fix the issue at hand which is due to flawed logic in the check. Reason being, the check should be

>  if none of the architectures in arch_list are in valid_arch_strs, then raise an error

instead of

>  if one of the architectures in arch_list is not in valid_arch_strs, then raise an error

I think will have to update the logic to following to fix this

```python
    arch_list = [arch for arch in arch_list if arch in valid_arch_strs]
    if not len(arch_list):
        raise ValueError(
            f"Unsupported CUDA arch ({arch}) is included in the "
            f"`TORCH_CUDA_ARCH_LIST` env variable ({env_arch_list}). "
            f"Valid CUDA arch strings are: {valid_arch_strs}.")
    return set(arch_list)
```




@author:
Thanks for the proposal. I fixed the PR and merged it.