Repository: vllm-project/vllm

Issue:
Slow installation:  install all versions of one package.
I follow the document and it will install all versions of one package.
```
  Downloading fonttools-4.24.0-py3-none-any.whl (853 kB)
     |████████████████████████████████| 853 kB 22.8 MB/s
  Downloading fonttools-4.23.1-py3-none-any.whl (853 kB)
     |████████████████████████████████| 853 kB 24.7 MB/s
  Downloading fonttools-4.23.0-py3-none-any.whl (852 kB)
     |████████████████████████████████| 852 kB 23.7 MB/s
  Downloading fonttools-4.22.1-py3-none-any.whl (850 kB)
     |████████████████████████████████| 850 kB 70.3 MB/s
  Downloading fonttools-4.22.0-py3-none-any.whl (850 kB)
     |████████████████████████████████| 850 kB 45.1 MB/s
INFO: pip is looking at multiple versions of cycler to determine which version is compatible with other requirements. This could take a while.
Collecting cycler>=0.10
```


Discussion:
Same here. 

I am running `pip install vllm` from the docker container created with `docker run --gpus all -it --rm --shm-size=8g nvcr.io/nvidia/pytorch:22.12-py3` (and after running `pip uninstall torch`)

```
INFO: pip is looking at multiple versions of contourpy to determine which version is compatible with other requirements. This could take a while.
Collecting contourpy>=1.0.1
  Downloading contourpy-1.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (300 kB)
     |████████████████████████████████| 300 kB 46.6 MB/s
  Downloading contourpy-1.0.7-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (300 kB)
     |████████████████████████████████| 300 kB 37.3 MB/s
INFO: pip is looking at multiple versions of cycler to determine which version is compatible with other requirements. This could take a while.
  Downloading contourpy-1.0.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (295 kB)
     |████████████████████████████████| 295 kB 58.0 MB/s
  Downloading contourpy-1.0.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (295 kB)
     |████████████████████████████████| 295 kB 47.0 MB/s
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Downloading contourpy-1.0.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (285 kB)
     |████████████████████████████████| 285 kB 64.5 MB/s
  Downloading contourpy-1.0.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (272 kB)
     |████████████████████████████████| 272 kB 46.6 MB/s
  Downloading contourpy-1.0.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (270 kB)
     |████████████████████████████████| 270 kB 44.7 MB/s
INFO: pip is looking at multiple versions of contourpy to determine which version is compatible with other requirements. This could take a while.
  Downloading contourpy-1.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (269 kB)
     |████████████████████████████████| 269 kB 45.0 MB/s
INFO: pip is looking at multiple versions of matplotlib to determine which version is compatible with other requirements. This could take a while.
Collecting matplotlib~=3.0
  Downloading matplotlib-3.7.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (9.2 MB)
     |████████████████████████████████| 9.2 MB 49.6 MB/s
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Downloading matplotlib-3.7.1-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (9.2 MB)
     |████████████████████████████████| 9.2 MB 29.1 MB/s


```
Upgrading pip might help

or ~adding this pip parameter: `--use-deprecated=legacy-resolver`~ (got an error with legacy resolver `ERROR: pip's legacy dependency resolver does not consider dependency conflicts when selecting packages. This behaviour is the source of the following dependency conflicts.`)


https://stackoverflow.com/questions/65122957/resolving-new-pip-backtracking-runtime-issue


> 

Thank you . 
I upgrade to pip23.2, but there is still some problems.
```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency
conflicts.
torch-tensorrt 1.3.0a0 requires torch<1.15.0,>=1.13.0dev, but you have torch 2.0.1 which is incompatible.
torchtext 0.13.0a0+fae8e8c requires torch==1.14.0a0+410ce96, but you have torch 2.0.1 which is incompatible.
torchvision 0.15.0a0 requires torch==1.14.0a0+410ce96, but you have torch 2.0.1 which is incompatible.
```
I was able to install (under 10 minutes) and succesfully run vllm following this: 
1. Run from inside a docker container: https://vllm.readthedocs.io/en/latest/getting_started/installation.html
2. pip uninstall torch
3. install miniconda, create a conda environment as shown in the page above
4. Inside conda environment: pip install vllm
> I was able to install (under 10 minutes) and succesfully run vllm following this:
> 
> 1. Run from inside a docker container: https://vllm.readthedocs.io/en/latest/getting_started/installation.html
> 2. pip uninstall torch
> 3. install miniconda, create a conda environment as shown in the page above
> 4. Inside conda environment: pip install vllm

Thank you for your reply. In my practice, I uninstall three packages like torchtext. And it seems to work.

Resolving patch:
diff --git a/examples/openai_chatcompletion_client.py b/examples/openai_chatcompletion_client.py
new file mode 100644
--- /dev/null
+++ b/examples/openai_chatcompletion_client.py
@@ -0,0 +1,33 @@
+import openai
+
+# Modify OpenAI's API key and API base to use vLLM's API server.
+openai.api_key = "EMPTY"
+openai.api_base = "http://localhost:8000/v1"
+
+# List models API
+models = openai.Model.list()
+print("Models:", models)
+
+model = models["data"][0]["id"]
+
+# Chat completion API
+chat_completion = openai.ChatCompletion.create(
+    model=model,
+    messages=[{
+        "role": "system",
+        "content": "You are a helpful assistant."
+    }, {
+        "role": "user",
+        "content": "Who won the world series in 2020?"
+    }, {
+        "role":
+        "assistant",
+        "content":
+        "The Los Angeles Dodgers won the World Series in 2020."
+    }, {
+        "role": "user",
+        "content": "Where was it played?"
+    }])
+
+print("Chat completion results:")
+print(chat_completion)
diff --git a/examples/openai_client.py b/examples/openai_completion_client.py
similarity index 71%
rename from examples/openai_client.py
rename to examples/openai_completion_client.py
--- a/examples/openai_client.py
+++ b/examples/openai_completion_client.py
@@ -3,26 +3,26 @@
 # Modify OpenAI's API key and API base to use vLLM's API server.
 openai.api_key = "EMPTY"
 openai.api_base = "http://localhost:8000/v1"
-model = "facebook/opt-125m"
 
-# Test list models API
+# List models API
 models = openai.Model.list()
 print("Models:", models)
 
-# Test completion API
-stream = True
+model = models["data"][0]["id"]
+
+# Completion API
+stream = False
 completion = openai.Completion.create(
     model=model,
     prompt="A robot may not injure a human being",
     echo=False,
     n=2,
-    best_of=3,
     stream=stream,
     logprobs=3)
 
-# print the completion
+print("Completion results:")
 if stream:
     for c in completion:
         print(c)
 else:
-    print("Completion result:", completion)
+    print(completion)
diff --git a/vllm/entrypoints/openai/api_server.py b/vllm/entrypoints/openai/api_server.py
--- a/vllm/entrypoints/openai/api_server.py
+++ b/vllm/entrypoints/openai/api_server.py
@@ -13,9 +13,6 @@
 from fastapi.exceptions import RequestValidationError
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.responses import JSONResponse, StreamingResponse
-from fastchat.conversation import Conversation, SeparatorStyle
-from fastchat.model.model_adapter import get_conversation_template
-
 import uvicorn
 
 from vllm.engine.arg_utils import AsyncEngineArgs
@@ -33,6 +30,13 @@
 from vllm.transformers_utils.tokenizer import get_tokenizer
 from vllm.utils import random_uuid
 
+try:
+    from fastchat.conversation import Conversation, SeparatorStyle
+    from fastchat.model.model_adapter import get_conversation_template
+    _fastchat_available = True
+except ImportError:
+    _fastchat_available = False
+
 TIMEOUT_KEEP_ALIVE = 5  # seconds
 
 logger = init_logger(__name__)
@@ -63,6 +67,11 @@ async def check_model(request) -> Optional[JSONResponse]:
 
 
 async def get_gen_prompt(request) -> str:
+    if not _fastchat_available:
+        raise ModuleNotFoundError(
+            "fastchat is not installed. Please install fastchat to use "
+            "the chat completion and conversation APIs: `$ pip install fschat`"
+        )
     conv = get_conversation_template(request.model)
     conv = Conversation(
         name=conv.name,


Test patch: