GitHub repository: slinusc/fast_llm_inference
Primary language: Jupyter Notebook
Created: 2025-02-23T07:26:39Z
Description: Bench360 is a modular benchmarking suite for local LLM inference. It offers a full-stack, extensible pipeline to evaluate the latency, throughput, quality, and cost of LLM inference on consumer and enterprise GPUs. Bench360 supports flexible backends, tasks and scenarios, enabling fair and reproducible comparisons for researchers and practitioners.