GitHub repository: microsoft/RetrievalAttention
Primary language: Python
Description: Scalable long-context LLM decoding that leverages sparsity—by treating the KV cache as a vector storage system.