SEK : Self-Explained Keywords Empower Large Language Models for Code Generation (2025)

Abstract: Large language models (LLMs) have achieved impressive performance in code generation. Despite the remarkable success, we observed that LLMs often misunderstand or overlook some problem-specific undertrained keywords during code generation, compromising the accuracy of the generated code. After explicitly explaining these undertrained keywords using well-trained terms in the prompt, LLMs are more likely to generate correct code implementation. Inspired by this observation, we propose a novel technique named SEK(Self-Explained Keywords), which empowers an LLM for better code generation by extracting and explaining the key terms in the problem description with the LLM itself. Comprehensive experiments across four benchmarks, i.e., HumanEval(+), MBPP(+), APPS and BigCodeBench, with five representative LLMs, show that SEK can significantly improve LLMs in code generation, yielding substantial and consistent gains. For instance, SEK improves the Pass@1 of DeepSeek-Coder-V2-Instruct from 85.4% to 93.3% on the Humaneval benchmark. Further analysis confirms that SEK enables the LLMs to shift their attention from low-frequency keywords to their corresponding explanations.

## introduction code generation aims to generate a code snippet that meets the intent described in natural language. this process can potentially reduce the costs of software development @xcite @xcite . recently, the notable success of llms such as chatgpt @xcite and llama-3 (ai@meta, 2024) has substantially enhanced the state-of-the-art in code generation. these llms demonstrate remarkable proficiency in comprehending natural language descriptions and translating them into code snippets. ## methodology code generation aims to generate a solution program based on a problem description. typically, a problem description includes implementation requirements, and several test cases to help further understand the problem. figure 3 illustrates the overview of sek. sek is designed to address the issue of llms' misunderstanding of undertrained terms in the program description due to the long-tail distribution in their training data. to address it, one key is to leverage the llm's capabilities to identify and explain potentially misunderstood keywords within the problem description. we employ a carefully crafted prompt with a few-shot learning method to achieve this. after obtaining the keywords and their explanations, we introduce a frequency-based ranking algorithm that prioritizes less frequent tokens, which are more likely to be overlooked by the llm. these ranked keywords are then appended to the original problem description, serving to guide the llm towards generating an accurate solution. the process comprises three main steps: keyextract & explain (section 2.1): based on the problem description, sek constructs a prompt to guide the llm to identify and explain keywords within the problem description. keyrank (section 2.2): sek employs a frequency-based ranking algorithm to prioritize the extracted keywords. promptenrich (section 2.3), sek concatenates the ranked keywords and their explanations with the original problem description to create an enriched problem description. this comprehensive formulation serves as the final input for the llm to generate code solutions. ## conclusion in this work, we propose sek, a simple yet effective method to enhance the code generation ca-pabilities of llms. sek leverages the llm to extract and explain keywords from the problem description, followed by ranking them based on their frequency. through extensive experiments, we demonstrate that sek facilitates llms in capturing and clarifying key concepts within problems, thereby generating more accurate code solutions.