catid
/

cat-llama-3-8b-awq-q128-w4-gemm

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

catid commited on Apr 19

Commit

7a52f00

•

1 Parent(s): 1039b19

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -10,7 +10,7 @@ git clone https://huggingface.co/catid/cat-llama-3-8b-awq-q128-w4-gemm
 conda create -n vllm8 python=3.10 -y && conda activate vllm8
-pip install git+https://github.com/vllm-project/vllm.git
 python -m vllm.entrypoints.openai.api_server --model cat-llama-3-8b-awq-q128-w4-gemm
 ```

 conda create -n vllm8 python=3.10 -y && conda activate vllm8
+pip install -U git+https://github.com/vllm-project/vllm.git
 python -m vllm.entrypoints.openai.api_server --model cat-llama-3-8b-awq-q128-w4-gemm
 ```