vllm-project/vllm: A high-throughput and memory-efficient inference and serving engine for LLMs
Category: llm
Language: Python
Topics: gpt, inference, llama, llm, llm-serving, llmops, mlops, model-serving, pytorch, transformer
Homepage: https://docs.vllm.ai