Could this model run with vllm?
#1
by
edwarddukewu
- opened
just want to use vllm to run model, vllm seems more efficient than gguf with GPU
Sorry, I didn't notice, you can run on vllm/sglang and I made for it.
vllm serve neody/r1-14b-awq
googlefan
changed discussion status to
closed