zephyr-7b-beta with VLLM
#49
by
D3v
- opened
HI there ,i find out that Mistral based model are supported by VLLM, As zephyr also a finetuned version of Mistral can it support VLLM ?
Hi,
Since Mistral-7B is supported by vLLM, this one will be too, since it uses the same architecture, only the weights are updated. Hence the following will work:
python -m vllm.entrypoints.api_server \
--model HuggingFaceH4/zephyr-7b-beta