Improving Repository-level Code Search with Text Conversion (2024)

Abstract: The ability to generate code using large language models (LLMs) has been increasing year by year. However, studies on code generation at the repository level are not very active. In repository-level code generation, it is necessary to refer to related code snippets among multiple files. By taking the similarity between code snippets, related files are searched and input into an LLM, and generation is performed. This paper proposes a method to search for related files (code search) by taking similarities not between code snippets but between the texts converted from the code snippets by the LLM. We confirmed that converting to text improves the accuracy of code search.

## introduction currently, the code generation capability of large language models (llms) has significantly improved. the accuracy of understanding and generating individual pieces of code has become high. however, there is little research at the repository level, which is closer to actual software development, and the ability of llms to generate code at the repository level is very low. llms' best debugging accuracy at the repository level is only 1.96% on the debugging benchmark swe-bench @xcite . code-related tasks at the repository level require referring to many files. however, most llms are based on transformer @xcite , which has a limitation on input length, preventing the input of many files. therefore, methods have been proposed to search for relevant code snippets based on similarity and input only these into llms @xcite . the accuracy of code search is low on swe-bench and repobench @xcite , repository-level code completion and search benchmark. this paper focuses on improving the code search method for code completion tasks. the code completion task is a task that predicts the next line of unfinished code. for predicting the next line, multiple related files are provided based on the dependencies between files obtained by parsing the unfinished code. in this paper, the unfinished code is referred to as the target code, and the multiple related files are referred to as code candidates. the task of selecting the relevant code from the code candidates based on the information of the target code is referred to as code search. existing studies obtain features, such as embeddings from language models, of both the target code and code candidates to calculate similarity for code search @xcite . repocoder @xcite searches for code using such methods, generates code once, and then re-searches and generates the output code using the generated code. in this study, instead of directly taking the similarity between code snippets, similarity is calculated after transforming the code with an llm. code candidates are converted to text, and the target code is converted to text or to the prediction of the next line and its explanation, which is a combination of repocoder's method and text conversion. figure 1 shows the flow of text conversion. we confirmed an improvement in the accuracy of code search in code completion experiments using our proposed method. we also examined the prompts used for text conversion with llms. ## conclusion in this study, we proposed a method for code search in code completion tasks, which involves converting code into text to obtain similarity. additionally, we proposed an automatic prompting method that creates prompts for llms. while an improvement in accuracy was confirmed for text conversion, no improvement in accuracy was observed for automatic prompting. we hope that this study will contribute to the development of code generation tasks at the repository level. in the future, we aim to apply text conversion not only to tasks other than code completion tasks, such as debugging, but also beyond repository-level tasks.