GitHub repository: usyd-fsalab/fp6_llm
Primary language: Cuda
Description: An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).