# C2B: A Semantic Source Code Retrieval Model Using CodeT5 and Bi-LSTM
## 2 Literature Review
### Deep Learning Approaches  
There has been a notable increase in interest in implementing deep learning approaches in software engineering applications. Our research utilizes a deep learning algorithm for the task of code search [23; 24]. Recent studies [25; 26; 27; 28; 29; 30; 31; 32] explored the potential uses of deep learning techniques in source code with applications including code generation, code completion, and code clone detection. For instance, [PERSON] et al. [33] introduced an RNN encoder-decoder model to generate code from natural language user intentions, demonstrating its effectiveness in a dataset of simple programming assignments. [PERSON] et al. [34] applied deep learning for application programming interface (API) learning, generating API usage sequences based on natural language queries. Additionally, [PERSON] et al. [35] used an RNN language model to predict software tokens in source code files for the task of code completion. Our research focuses on exploring the application of deep learning specifically to code search. Prior research in NLP [36; 37; 38] creates embedding only for limited bilingual data. A particular challenge within the context of code search revolves around the alignment of embedding, emphasizing that the alignment should not solely focus on the word (i.e., token) level. Instead, the embedding should effectively aggregate and represent entire code snippets and queries comprehensively. The application of NLP research techniques is highly beneficial in tackling this particular issue, providing valuable insights into potential solutions and approaches. For instance, [PERSON] et al. [39] presented a probabilistic model that can synthesize a program snippet based on a natural language query. Meanwhile,a Bayesian statistical approach--BAYOU [40]--leverages deep neural networks (DNNs) to generate code programs from a combination of API calls and natural language input. The CODE-NN [41] approach uses long short-term memory (LSTM) networks to generate natural language descriptions from code snippets, enhancing software development by linking code and natural language. Our research assesses the performance of neural network designs in code search tasks, concentrating on diverse source code snippets and utilizing natural language queries as input. Traditional approaches [42; 43] heavily relied on information retrieval (IR) algorithms, which include rule-based strategies [44; 45; 46], statistical language models [47], and statistical machine translation [48]. These methods often faced challenges in effectively capturing the complex and diverse nature of source codes, especially in terms of structural and semantic representations. Deep learning approaches powered by neural networks offer the ability to automatically learn intricate patterns and representations from data that enable more accurate and flexible modeling of source codes. This shift is driven by the desire to enhance the representation and understanding of source codes, addressing the shortcomings of manual feature extraction and rule-based techniques prevalent in earlier approaches. For instance, [PERSON] et al. [41] used an RNN with attention to produce summaries straight from token embeddings. To generate function-name-like summaries, [PERSON] et al. [25] presented a convolutional attention network for the purpose of extracting features from token sequences. Additionally, structural information from abstract syntax trees (AST) [49] is taken into account. [PERSON] et al. [50] serialized ASTs using a traversal technique called structure-based traversal (SBT) and then summarized the results using a conventional sequence-to-sequence (Seq2 Seq) model. To identify pertinent pathways during decoding, [PERSON] et al. [51] expressed code as a set of AST paths and encoded it using an RNN. [PERSON] et al. [52] provided a different strategy [52] that uses the most comparable code snippets that were recovered from the training set to improve the neural model in a retrieval-based neural code summarization technique. [PERSON] et al.