# C2B: A Semantic Source Code Retrieval Model Using CodeT5 and Bi-LSTM
## 5 Experimental Results
### Results and Discussions
#### 5.4.2 Experiment 2  
In this experiment, we are interested in finding out the total time required by both the original CodeT5 and our proposed C2B retrieval model to answer n queries. Further, we compare through visualization how accurate both systems are at answering the queries. * Code retrieval process: The retrieval process with the Bi-LSTM model using cosine similarity begins with the precomputation of code embeddings, where vector representations capturing semantic information are generated and stored for all codes within the dataset. In the CodeSearchNet testing dataset, code pairs are provided, with the first code designated as the query code for retrieval purposes. The fine-tuned Bi-LSTM architecture is then employed to process this query code, extracting code embeddings that encapsulate its semantic content. This approach employs cosine similarity as the primary comparison metric. Cosine similarity computes similarity scores by measuring the cosine of the angle between two vectors, providing a numerical indication of their similarity. The computed cosine similarity scores are used to assess the likeness between the query code's embeddings and the precomputed embeddings of all codes present in the dataset. Following the computation of cosine similarity scores, the retrieval process identifies the top \"k\" source codes that exhibit the highest cosine similarity scores with the query. The proposed approach leverages cosine similarity to retrieve the 20 code snippets most closely resembling the semantic content of the query, providing an efficient and effective method for code retrieval based on semantic similarity. * Benchmark queries: The query dataset used in this study was sourced from the CodeSearchNet challenge [[https://github.com/github/CodeSearchNet](https://github.com/github/CodeSearchNet)]([https://github.com/github/CodeSearchNet](https://github.com/github/CodeSearchNet)) (accessed on 15 April 2024), which was developed by [PERSON] et al. (2015). This dataset consists of 99 queries [[https://github.com/github/CodeSearchNet/blob/master/resources/](https://github.com/github/CodeSearchNet/blob/master/resources/)]([https://github.com/github/CodeSearchNet/blob/master/resources/](https://github.com/github/CodeSearchNet/blob/master/resources/)) \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline Model & Ruby & Javascript & Go & Python & Java & Php & Overall \\ \hline NBow & 0.162 & 0.157 & 0.330 & 0.161 & 0.171 & 0.152 & 0.189 \\ CNN & 0.276 & 0.224 & 0.680 & 0.242 & 0.260 & 0.260 & 0.324 \\ BiRNN & 0.213 & 0.193 & 0.688 & 0.290 & 0.338 & 0.338 & 0.338 \\ selfAtt & 0.275 & 0.287 & 0.723 & 0.398 & 0.404 & 0.426 & 0.419 \\ RoBERTa & 0.587 & 0.517 & 0.850 & 0.587 & 0.599 & 0.560 & 0.617 \\ CodeBERT & 0.679 & 0.620 & 0.882 & 0.672 & 0.676 & 0.628 & 0.693 \\ FA-AST & 0.628 & 0.562 & 0.859 & 0.610 & 0.620 & 0.579 & 0.643 \\ GraphCodeBERT & 0.703 & 0.644 & 0.897 & 0.692 & 0.691 & 0.649 & 0.713 \\ CodeT5 & **0.872** & **0.731** & **0.921** & **0.712** & **0.723** & **0.710** & **0.801** \\ Proposed C2B hybrid & 0.863 & 0.730 & 0.901 & 0.703 & 0.711 & 0.731 & 0.799 \\ model & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results on code search. Figures in **bold** refer to the best values, whereas the underlined figures refer to the second-best values for the MRR. queries_csv (accessed on 15 April 2024), representing commonly searched queries on Bing that are characterized by their high clickthrough rates to code snippets. These queries typically contain technical keywords [95]. * Execution time: Figure 7 illustrates the time-based comparison of the original CodeT5 architecture [18] and our proposed C2B model. We can see that the original architecture takes a great deal of time as the number of queries increases, in the order of 102 min. However, the proposed C2B model takes around 1 min to address 50 queries. Thus, we can conclude that the proposed C2B model is scalable.