GitHub repository: FMInference/H2O
Primary language: Python
Description: [NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.