Moses25
/

Mistral-7B-chat-32k

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Moses25 commited on Jun 3

Commit

710fd5c

•

1 Parent(s): 6d0e6b1

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -75,7 +75,7 @@ However, I should note that the actual fruit distribution in your basket may var
 ## vLLM server
 ```shell
 #llama2-chat-template.jinja file is chat-template above 'mistral-template'
-model_path=Meta-LlaMA-3-8B-Instruct-16k
 python  -m vllm.entrypoints.openai.api_server --model=$model_path \
         --trust-remote-code --host 0.0.0.0  --port 7777 \
         --gpu-memory-utilization 0.8 \

 ## vLLM server
 ```shell
 #llama2-chat-template.jinja file is chat-template above 'mistral-template'
+model_path="Moses25/Mistral-7B-chat-32k"
 python  -m vllm.entrypoints.openai.api_server --model=$model_path \
         --trust-remote-code --host 0.0.0.0  --port 7777 \
         --gpu-memory-utilization 0.8 \