C o RAC : Integrating Selective API Document Retrieval with Question Semantic Intent for Code Question Answering (2025)

Abstract: Automatic code question answering aims to generate precise answers to questions about code by analyzing code snippets. To provide an appropriate answer, it is necessary to accurately understand the relevant part of the code and correctly interpret the intent of the question. However, in real-world scenarios, the questioner often provides only a portion of the code along with the question, making it challenging to find an answer. The responder should be capable of providing a suitable answer using such limited information. We propose a knowledge-based framework, CoRAC, an automatic code question responder that enhances understanding through selective API document retrieval and question semantic intent clustering. We evaluate our method on three real-world benchmark datasets and demonstrate its effectiveness through various experiments. We also show that our method can generate high-quality answers compared to large language models, such as ChatGPT.

## introduction online communities such as stack overflow, reddit, and github have become essential resources for developers seeking solutions to coding challenges. developers frequently rely on these platforms to overcome obstacles, deepen their understanding of programming concepts, and enhance their productivity. these communities serve as knowledge hubs for a wide audience, ranging from professional software developers to students studying programming, and they have accumulated vast amounts of information. the growing volume of question-and-answer interactions on these platforms has led to a need for automated methods to efficiently provide accurate and relevant answers to code-related queries. recently, many pre-trained language models (plms) for programming and natural language have been developed, such as codebert @xcite , plbart @xcite , @xcite , codet5 @xcite , and codegen @xcite . these models have achieved remarkable success across various code understanding tasks, including defect detection @xcite , clone detection @xcite , code translation @xcite , code summarization @xcite , and code generation @xcite . this success is due to the utilization of large-scale corpora of code and text. these plms learn not only from code (uni-modal data) but also codecomment pairs (bi-modal data) in the pre-training stage. this allows the models to learn both the semantic information of code and the relationships between code and natural language. these models have shown significant performance on various tasks after fine-tuning. @xcite also showed the possibility of plms for code question answering. to address these challenging aspects of code question answering, we utilize specific domain knowledge used within the code and a precise understanding of the question' intent to generate answers. for example, as shown in figure 1 , responders should have knowledge of the api functions used in the code, such as loads(), get(), and keys(), to understand the code and infer the answer. if they lack knowledge of it, api documentation can provide information on how to use functions and their parameters. the key point is to identify the api functions relevant to the question being asked. question #1 involves understanding api function calls related to json, while question #2 specifically requires knowledge of the keys() function. therefore, it is important to identify which api function documentation is helpful in answering the questions. furthermore, to provide more accurate and relevant answers, responders should understand the intent of the questions. as shown in figure 1 , question #1 asks about the process of retrieving json data from a url using the given code, so the answer should provide information about usage of apis. in contrast, question #2 inquires about the purpose and functionality of the given code, so the answer should provide purpose or functionality information. by understanding the underlying purpose of the question, responders can generate a more tailored and relevant answer that directly addresses the asker's concerns. however, in real-world scenarios, questioners often provide only code and questions without explicitly indicating their question intent. misinterpreting the intent can result in answers that fail to meet the questioner's expectations. for example, if a question seeks the "purpose" of a code block but the response focuses on its "usage", the answer may mislead the questioner even when utilizing specific domain knowledge such as api documentation. thus, accurately understanding the intent of questions is crucial in generating accurate and relevant answers. in this paper, we propose a knowledge-based framework, corac, an automatic code question responder by utilizing api documentation as external domain knowledge and clustered question intent instruction. we propose selective api document retrieval that extracts helpful api function descriptions highly relevant to the question through an external plm. also, we design question intent instruction that represents the semantic intent of the question about the code. by training question semantic intent instruction with the cluster prompt template, we can indirectly inform the type of question and guide the model to improve the quality of answers to code-related questions. to demonstrate the effectiveness of our model, we conduct experiments on three real-world datasets: two github datasets (java and python) and one introductory programming course dataset (python). we show that our model can generate high-quality answers compared to recent large language models (llms). ## related work automatic question answering has been consistently studied in the field of natural language processing @xcite @xcite . recently, attention has shifted toward automatic code question answering in program language processing. @xcite introduced a system that generates answers to technical questions from stackoverflow. similarly, @xcite designed an answering system for questions about subroutines using an rnn encoder-decoder network @xcite . however, questions in these datasets tend to be simple, where answers can be easily obtained without requiring complex inference or reasoning about the code. recently, the rise of llms in programming and natural language processing, such as llama3.1 @xcite , code llama (roziere et al., 2023), qwen2.5-coder @xcite has led to notable improvements in various code intelligence tasks. however, when it comes to code question answering, these llms often produce responses that are overly detailed and focused on the code itself, which makes it difficult to align with the user's specific intent or request. @xcite also shows impressive abilities across various domains due to its extensive training with vast data, resources, and instructions. yet, when given only fragments of code and questions, gpt-4 also struggles to provide accurate answers and it also requires well-structured prompts for proper responses. ## proposed method in this section, we introduce a knowledge-based framework, corac, to leverage helpful api documentation related to the question and to utilize the semantic intent of the question as shown in figure 2 . ## conclusion we proposed a knowledge-based framework that leveraged helpful api function documents (selective api document retrieval) and instruction of question semantic intent (question semantic intent clustering) to generate more accurate and relevant answers to code questions. we demonstrated the effectiveness of our methods and highlighted the need for more efficient ways to obtain accurate and relevant answers to code-related inquiries. in this paper, we focus on generating natural language answers that include concise function calls rather than full function-level code. as future work, we could explore generating answers that integrate textual explanations with full function-level code snippets.