Hugging Face's logo

cloudyu
/
Yi-34Bx2-MoE-60B

Text Generation
Transformers
Safetensors
mixtral
yi
Mixture of Experts
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
17

Can VLLM be used for inference acceleration?

#2
by obtion - opened
Discussion
obtion

Can VLLM be used for inference acceleration?

cloudyu
 Owner

"architectures": [
"MixtralForCausalLM"
],
you need to check whether vllm support "MixtralForCausalLM"

ehartford

yeah VLLM supports that

Sign up or log in to comment