Syntax-Aware Retrieval Augmented Code Generation (2023)

Abstract: Neural code generation models are nowadays widely adopted to generate code from natural language descriptions automatically. Recently, pre-trained neural models equipped with token-level retrieval capabilities have exhibited great potentials in neural machine translation. However, applying them directly to code generation experience challenges: the use of the retrieval-based mechanism inevitably introduces extraneous noise to the generation process, resulting in even syntactically incorrect code. Computationally, such models necessitate frequent searches of the cached datastore, which turns out to be time-consuming. To address these issues, we propose k NN-TRANX, a token-level retrieval augmented code generation method. k NN-TRANX allows for searches in smaller datastores tailored for the code generation task. It leverages syntax constraints for the retrieval of datastores, which reduces the impact of retrieve noise. We evaluate k NN-TRANX on two public datasets and the experimental results confirm the effectiveness of our approach.

## introduction neural code generation aims to map the input natural language (nl) to code snippets using deep learning. due to its great potential to streamline software development, it has garnered significant attentions from both natural language processing and software engineering communities. various methods have been explored to facilitate code generation @xcite @xcite . recent progress in neural machine translation (nmt) shows that the non-parametric knearest-neighbour machine translation (knn-mt) approach may significantly boost the performance of standard nmt models @xcite and other text generation models @xcite by equipping ## background in this section, we provide an overview of the knn-mt paradigm and the seq2tree model. ## knn-tranx the workflow of knn-tranx is depicted in figure 2. it consists of two main components: datastore building and model inferring. ## experiments in this section, we first introduce the datasets and evaluation metrics. then, we conduct a comprehensive study and analysis on code generation and model canonical incremental adaptation. ## conclusion in this paper, we propose knn-tranx. by providing syntactic and semantic datastores for seq2tree model, we are able to outperform the baselines. in addition, we provide more knowledge for the model by switch the datastores without fine-tuning the neural network. experimental results show that knn-tranx exhibits competitive performance against learning-based methods through incremental learning. in the future, we plan to construct a smaller and more fine-grained syntactic datas-