这个一般推荐什么推理框架（来做stream）？
#2
by
hanswang73
- opened
这个一般推荐什么推理框架（来做stream）？
vllm works well with awq quantization method
vllm works well with awq quantization method
多谢！我用 huggingface 的 TextIteratorStreamer 也能搞定了
这个一般推荐什么推理框架（来做stream）？
vllm works well with awq quantization method
vllm works well with awq quantization method
多谢！我用 huggingface 的 TextIteratorStreamer 也能搞定了