| Collecting vllm | |
| Using cached vllm-0.5.3.post1-cp38-abi3-manylinux1_x86_64.whl.metadata (1.8 kB) | |
| Requirement already satisfied: cmake>=3.21 in /opt/conda/lib/python3.10/site-packages (from vllm) (3.30.1) | |
| Requirement already satisfied: ninja in /opt/conda/lib/python3.10/site-packages (from vllm) (1.11.1.1) | |
| Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from vllm) (5.9.0) | |
| Requirement already satisfied: sentencepiece in /opt/conda/lib/python3.10/site-packages (from vllm) (0.1.99) | |
| Requirement already satisfied: numpy<2.0.0 in /opt/conda/lib/python3.10/site-packages (from vllm) (1.26.3) | |
| Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from vllm) (2.32.3) | |
| Requirement already satisfied: tqdm in /opt/conda/lib/python3.10/site-packages (from vllm) (4.66.4) | |
| Requirement already satisfied: py-cpuinfo in /opt/conda/lib/python3.10/site-packages (from vllm) (9.0.0) | |
| Requirement already satisfied: transformers>=4.42.4 in /opt/conda/lib/python3.10/site-packages (from vllm) (4.43.2) | |
| Requirement already satisfied: tokenizers>=0.19.1 in /opt/conda/lib/python3.10/site-packages (from vllm) (0.19.1) | |
| Collecting fastapi (from vllm) | |
| Using cached fastapi-0.111.1-py3-none-any.whl.metadata (26 kB) | |
| Requirement already satisfied: aiohttp in /opt/conda/lib/python3.10/site-packages (from vllm) (3.9.5) | |
| Collecting openai (from vllm) | |
| Using cached openai-1.37.1-py3-none-any.whl.metadata (22 kB) | |
| Requirement already satisfied: uvicorn[standard] in /opt/conda/lib/python3.10/site-packages (from vllm) (0.30.3) | |
| Collecting pydantic>=2.0 (from vllm) | |
| Using cached pydantic-2.8.2-py3-none-any.whl.metadata (125 kB) | |
| Requirement already satisfied: pillow in /opt/conda/lib/python3.10/site-packages (from vllm) (10.2.0) | |
| Requirement already satisfied: prometheus-client>=0.18.0 in /opt/conda/lib/python3.10/site-packages (from vllm) (0.20.0) | |
| Collecting prometheus-fastapi-instrumentator>=7.0.0 (from vllm) | |
| Using cached prometheus_fastapi_instrumentator-7.0.0-py3-none-any.whl.metadata (13 kB) | |
| Collecting tiktoken>=0.6.0 (from vllm) | |
| Using cached tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB) | |
| Collecting lm-format-enforcer==0.10.3 (from vllm) | |
| Using cached lm_format_enforcer-0.10.3-py3-none-any.whl.metadata (16 kB) | |
| Collecting outlines<0.1,>=0.0.43 (from vllm) | |
| Using cached outlines-0.0.46-py3-none-any.whl.metadata (15 kB) | |
| Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from vllm) (4.9.0) | |
| Requirement already satisfied: filelock>=3.10.4 in /opt/conda/lib/python3.10/site-packages (from vllm) (3.13.1) | |
| Requirement already satisfied: pyzmq in /opt/conda/lib/python3.10/site-packages (from vllm) (26.0.3) | |
| Collecting ray>=2.9 (from vllm) | |
| Using cached ray-2.33.0-cp310-cp310-manylinux2014_x86_64.whl.metadata (13 kB) | |
| Requirement already satisfied: nvidia-ml-py in /opt/conda/lib/python3.10/site-packages (from vllm) (12.555.43) | |
| Collecting torch==2.3.1 (from vllm) | |
| Using cached torch-2.3.1-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB) | |
| Collecting torchvision==0.18.1 (from vllm) | |
| Using cached torchvision-0.18.1-cp310-cp310-manylinux1_x86_64.whl.metadata (6.6 kB) | |
| Collecting xformers==0.0.27 (from vllm) | |
| Using cached xformers-0.0.27-cp310-cp310-manylinux2014_x86_64.whl.metadata (1.0 kB) | |
| Collecting vllm-flash-attn==2.5.9.post1 (from vllm) | |
| Using cached vllm_flash_attn-2.5.9.post1-cp310-cp310-manylinux1_x86_64.whl.metadata (482 bytes) | |
| Requirement already satisfied: interegular>=0.3.2 in /opt/conda/lib/python3.10/site-packages (from lm-format-enforcer==0.10.3->vllm) (0.3.3) | |
| Requirement already satisfied: packaging in /opt/conda/lib/python3.10/site-packages (from lm-format-enforcer==0.10.3->vllm) (23.1) | |
| Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from lm-format-enforcer==0.10.3->vllm) (6.0.1) | |
| Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (1.12) | |
| Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (3.1) | |
| Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (3.1.3) | |
| Requirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (2023.10.0) | |
| Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.105) | |
| Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.105) | |
| Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.105) | |
| Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.3.1->vllm) | |
| Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) | |
| Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.3.1) | |
| Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (11.0.2.54) | |
| Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (10.3.2.106) | |
| Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.3.1->vllm) | |
| Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) | |
| Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.3.1->vllm) | |
| Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB) | |
| Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (2.20.5) | |
| Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.105) | |
| Requirement already satisfied: triton==2.3.1 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (2.3.1) | |
| Requirement already satisfied: nvidia-nvjitlink-cu12 in /opt/conda/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch==2.3.1->vllm) (12.5.82) | |
| Requirement already satisfied: lark in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (1.1.9) | |
| Requirement already satisfied: nest-asyncio in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (1.6.0) | |
| Collecting cloudpickle (from outlines<0.1,>=0.0.43->vllm) | |
| Using cached cloudpickle-3.0.0-py3-none-any.whl.metadata (7.0 kB) | |
| Requirement already satisfied: diskcache in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (5.6.3) | |
| Requirement already satisfied: numba in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (0.60.0) | |
| Requirement already satisfied: referencing in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (0.30.2) | |
| Requirement already satisfied: jsonschema in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (4.19.2) | |
| Requirement already satisfied: datasets in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (2.20.0) | |
| Requirement already satisfied: pycountry in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (24.6.1) | |
| Requirement already satisfied: pyairports in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (2.1.1) | |
| Collecting starlette<1.0.0,>=0.30.0 (from prometheus-fastapi-instrumentator>=7.0.0->vllm) | |
| Using cached starlette-0.38.1-py3-none-any.whl.metadata (5.9 kB) | |
| Collecting annotated-types>=0.4.0 (from pydantic>=2.0->vllm) | |
| Using cached annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB) | |
| Requirement already satisfied: pydantic-core==2.20.1 in /opt/conda/lib/python3.10/site-packages (from pydantic>=2.0->vllm) (2.20.1) | |
| Requirement already satisfied: click>=7.0 in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (8.1.7) | |
| Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (1.0.8) | |
| Requirement already satisfied: protobuf!=3.19.5,>=3.15.3 in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (5.27.2) | |
| Requirement already satisfied: aiosignal in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (1.3.1) | |
| Requirement already satisfied: frozenlist in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (1.4.1) | |
| Requirement already satisfied: regex>=2022.1.18 in /opt/conda/lib/python3.10/site-packages (from tiktoken>=0.6.0->vllm) (2024.7.24) | |
| Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->vllm) (2.0.4) | |
| Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->vllm) (3.4) | |
| Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->vllm) (2.1.0) | |
| Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->vllm) (2024.2.2) | |
| Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /opt/conda/lib/python3.10/site-packages (from tokenizers>=0.19.1->vllm) (0.24.2) | |
| Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.42.4->vllm) (0.4.3) | |
| Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->vllm) (23.1.0) | |
| Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp->vllm) (6.0.5) | |
| Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->vllm) (1.9.4) | |
| Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->vllm) (4.0.3) | |
| Collecting starlette<1.0.0,>=0.30.0 (from prometheus-fastapi-instrumentator>=7.0.0->vllm) | |
| Using cached starlette-0.37.2-py3-none-any.whl.metadata (5.9 kB) | |
| Collecting fastapi-cli>=0.0.2 (from fastapi->vllm) | |
| Using cached fastapi_cli-0.0.4-py3-none-any.whl.metadata (7.0 kB) | |
| Requirement already satisfied: httpx>=0.23.0 in /opt/conda/lib/python3.10/site-packages (from fastapi->vllm) (0.27.0) | |
| Requirement already satisfied: python-multipart>=0.0.7 in /opt/conda/lib/python3.10/site-packages (from fastapi->vllm) (0.0.9) | |
| Requirement already satisfied: email_validator>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from fastapi->vllm) (2.2.0) | |
| Requirement already satisfied: h11>=0.8 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (0.14.0) | |
| Requirement already satisfied: httptools>=0.5.0 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (0.6.1) | |
| Requirement already satisfied: python-dotenv>=0.13 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (1.0.1) | |
| Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (0.19.0) | |
| Collecting watchfiles>=0.13 (from uvicorn[standard]->vllm) | |
| Using cached watchfiles-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.9 kB) | |
| Requirement already satisfied: websockets>=10.4 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (12.0) | |
| Requirement already satisfied: anyio<5,>=3.5.0 in /opt/conda/lib/python3.10/site-packages (from openai->vllm) (4.4.0) | |
| Requirement already satisfied: distro<2,>=1.7.0 in /opt/conda/lib/python3.10/site-packages (from openai->vllm) (1.8.0) | |
| Requirement already satisfied: sniffio in /opt/conda/lib/python3.10/site-packages (from openai->vllm) (1.3.1) | |
| Requirement already satisfied: exceptiongroup>=1.0.2 in /opt/conda/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai->vllm) (1.2.0) | |
| Requirement already satisfied: dnspython>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from email_validator>=2.0.0->fastapi->vllm) (2.6.1) | |
| Collecting typer>=0.12.3 (from fastapi-cli>=0.0.2->fastapi->vllm) | |
| Using cached typer-0.12.3-py3-none-any.whl.metadata (15 kB) | |
| Requirement already satisfied: httpcore==1.* in /opt/conda/lib/python3.10/site-packages (from httpx>=0.23.0->fastapi->vllm) (1.0.5) | |
| Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch==2.3.1->vllm) (2.1.3) | |
| Requirement already satisfied: pyarrow>=15.0.0 in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (17.0.0) | |
| Requirement already satisfied: pyarrow-hotfix in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (0.6) | |
| Requirement already satisfied: dill<0.3.9,>=0.3.0 in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (0.3.7) | |
| Requirement already satisfied: pandas in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (2.2.2) | |
| Requirement already satisfied: xxhash in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (3.4.1) | |
| Requirement already satisfied: multiprocess in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (0.70.15) | |
| Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/conda/lib/python3.10/site-packages (from jsonschema->outlines<0.1,>=0.0.43->vllm) (2023.7.1) | |
| Requirement already satisfied: rpds-py>=0.7.1 in /opt/conda/lib/python3.10/site-packages (from jsonschema->outlines<0.1,>=0.0.43->vllm) (0.10.6) | |
| Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /opt/conda/lib/python3.10/site-packages (from numba->outlines<0.1,>=0.0.43->vllm) (0.43.0) | |
| Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch==2.3.1->vllm) (1.3.0) | |
| Requirement already satisfied: shellingham>=1.3.0 in /opt/conda/lib/python3.10/site-packages (from typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm) (1.5.4) | |
| Collecting rich>=10.11.0 (from typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm) | |
| Using cached rich-13.7.1-py3-none-any.whl.metadata (18 kB) | |
| Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.10/site-packages (from pandas->datasets->outlines<0.1,>=0.0.43->vllm) (2.9.0.post0) | |
| Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.10/site-packages (from pandas->datasets->outlines<0.1,>=0.0.43->vllm) (2023.3.post1) | |
| Requirement already satisfied: tzdata>=2022.7 in /opt/conda/lib/python3.10/site-packages (from pandas->datasets->outlines<0.1,>=0.0.43->vllm) (2024.1) | |
| Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->datasets->outlines<0.1,>=0.0.43->vllm) (1.16.0) | |
| Collecting markdown-it-py>=2.2.0 (from rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm) | |
| Using cached markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB) | |
| Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/conda/lib/python3.10/site-packages (from rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm) (2.15.1) | |
| Requirement already satisfied: mdurl~=0.1 in /opt/conda/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm) (0.1.2) | |
| Using cached vllm-0.5.3.post1-cp38-abi3-manylinux1_x86_64.whl (158.3 MB) | |
| Using cached lm_format_enforcer-0.10.3-py3-none-any.whl (43 kB) | |
| Using cached torch-2.3.1-cp310-cp310-manylinux1_x86_64.whl (779.1 MB) | |
| Using cached torchvision-0.18.1-cp310-cp310-manylinux1_x86_64.whl (7.0 MB) | |
| Using cached vllm_flash_attn-2.5.9.post1-cp310-cp310-manylinux1_x86_64.whl (37.1 MB) | |
| Using cached xformers-0.0.27-cp310-cp310-manylinux2014_x86_64.whl (164.1 MB) | |
| Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB) | |
| Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB) | |
| Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB) | |
| Using cached outlines-0.0.46-py3-none-any.whl (101 kB) | |
| Using cached prometheus_fastapi_instrumentator-7.0.0-py3-none-any.whl (19 kB) | |
| Using cached pydantic-2.8.2-py3-none-any.whl (423 kB) | |
| Using cached ray-2.33.0-cp310-cp310-manylinux2014_x86_64.whl (64.8 MB) | |
| Using cached tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB) | |
| Using cached fastapi-0.111.1-py3-none-any.whl (92 kB) | |
| Using cached openai-1.37.1-py3-none-any.whl (337 kB) | |
| Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB) | |
| Using cached fastapi_cli-0.0.4-py3-none-any.whl (9.5 kB) | |
| Using cached starlette-0.37.2-py3-none-any.whl (71 kB) | |
| Using cached watchfiles-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB) | |
| Using cached cloudpickle-3.0.0-py3-none-any.whl (20 kB) | |
| Using cached typer-0.12.3-py3-none-any.whl (47 kB) | |
| Using cached rich-13.7.1-py3-none-any.whl (240 kB) | |
| Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB) | |
| Installing collected packages: nvidia-cusparse-cu12, nvidia-cudnn-cu12, markdown-it-py, cloudpickle, annotated-types, watchfiles, tiktoken, starlette, rich, pydantic, nvidia-cusolver-cu12, typer, torch, prometheus-fastapi-instrumentator, openai, lm-format-enforcer, xformers, vllm-flash-attn, torchvision, ray, fastapi-cli, outlines, fastapi, vllm | |
| Attempting uninstall: torch | |
| Found existing installation: torch 2.2.1 | |
| Can't uninstall 'torch'. No files were found to uninstall. | |
| Attempting uninstall: torchvision | |
| Found existing installation: torchvision 0.17.1 | |
| Can't uninstall 'torchvision'. No files were found to uninstall. | |
| Successfully installed annotated-types-0.7.0 cloudpickle-3.0.0 fastapi-0.111.1 fastapi-cli-0.0.4 lm-format-enforcer-0.10.3 markdown-it-py-3.0.0 nvidia-cudnn-cu12-8.9.2.26 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 openai-1.37.1 outlines-0.0.46 prometheus-fastapi-instrumentator-7.0.0 pydantic-2.8.2 ray-2.33.0 rich-13.7.1 starlette-0.37.2 tiktoken-0.7.0 torch-2.3.1 torchvision-0.18.1 typer-0.12.3 vllm-0.5.3.post1 vllm-flash-attn-2.5.9.post1 watchfiles-0.22.0 xformers-0.0.27 | |