Sglang?
#2
by jpsequeira - opened
Would you consider supporting it? In my experience NVFP4 and blackwell support is just better with it
Our models are in the compressed-tensors format which has much better support in vLLM which is our primary target for inference serving.
If you're seeing any particular issues with vLLM and NVFP4, feel free to share in the vLLM slack: https://communityinviter.com/apps/vllm-dev/join-vllm-developers-slack!
dsikka changed discussion status to closed