Title: LlavaCode: Compressed Code Representations for Retrieval-Augmented Code Generation
Primary area: generative models
Keywords: rag, ast, dct, llm, code

Abstract: Retrieval-augmented generation has emerged as one of the most effective approaches for code completion, particularly when context from a surrounding repository is essential. However, incorporating context significantly extends sequence length, leading to slower inference—a critical limitation for interactive settings such as IDEs.

In this work, we introduce LlavaCode, a framework that compresses code into compact, semantically rich representations interpretable by code LLM, enhancing generation quality while reducing the retrieved context to only a few compressed single-token vectors. Using a small projector module we can significantly increase the EM and ES metrics of coding model with negligible latency increase.
Our experiments demonstrate that compressed context enables 20-38% reduction in Time-to-First-Token (TTFT) on line completion tasks compared to full-RAG pipelines.

TL;DR: We propose LlavaCode, a method to compress repository context for retrieval-augmented code generation. By using compact representations, we cut sequence length and speed up line completion by up to 19% without hurting prediction quality.