# C2B: A Semantic Source Code Retrieval Model Using CodeT5 and Bi-LSTM
## 5 Experimental Results
### Results and Discussions
#### 5.4.1 Experiment 1
Thus, we can conclude that our best model achieves a considerable performance on the clone detection task. When comparing the results of our proposed C2B hybrid model with other models in Table 5 for code search tasks, we can observe that our proposed C2B model delivers competitive performance, particularly in specific programming languages. In the Ruby language, the C2B model achieves an MRR of 0.863, which is close to the MRR of 0.872 achieved by the top-performing CodeT5. This suggests that the C2B model exhibits strong effectiveness in retrieving Ruby code, closely approaching the performance level of CodeT5. The C2B model performs impressively in JavaScript queries with an MRR of 0.730, closely trailing behind CodeT5, which has an MRR of 0.731. This demonstrates that our proposed C2B hybrid model is nearly as effective as CodeT5 in this language. The C2B model achieves an MRR of 0.901 for Go, whereas CodeT5 outperforms it slightly with an MRR of 0.921. However, the C2B model remains a strong contender and is very close in performance. For Python, the C2B model delivers an MRR of 0.703 for queries, which is slightly below the MRR of 0.712 achieved by CodeT5. Nevertheless, it demonstrates competitive performance. For Java language, the C2B model achieves an MRR of 0.711, while CodeT5 leads with an MRR of 0.723. The C2B model maintains a strong presence in this language. In the case of PHP, the C2B model performs exceptionally well in PHP queries, with an MRR of 0.731, slightly surpassing CodeT5, which has an MRR of 0.710. This highlights the C2B model's strength in retrieving PHP code. Across all programming languages, the C2B model maintains a competitive position with an MRR of 0.799, closely following \"CodeT5,\" which leads with an MRR of 0.801. The C2B model demonstrates remarkable effectiveness in code search tasks with particularly strong performance. It consistently ranks as the second-best or closely behind CodeT5 in various languages, showcasing its versatility and competitive \begin{table}
\begin{tabular}{l c c c} \hline
**Architecture** & **Precision** & **Recall** & **F1 Score** \\ \hline NBoW [91] & 0.90 & 0.01 & 0.01 \\ CNN [92] & 0.93 & 0.02 & 0.03 \\ BiRNN [93] & 0.92 & 0.74 & 0.82 \\ selfAtt [94] & 0.92 & 0.94 & 0.93 \\ ROBERTa [56] & 0.949 & 0.922 & 0.935 \\ CodeBERT [55] & 0.947 & 0.934 & 0.941 \\ FA-AST [53] & **0.96** & 0.94 & **0.95** \\ GraphCodeBERT [54] & 0.948 & **0.952** & **0.95** \\ CodeT5 [18] & 0.952 & 0.947 & **0.95** \\ Proposed C2B hybrid model & 0.897 & 0.907 & 0.902 \\ \hline \end{tabular}
\end{table} 
Table 4: Results on code search. Figures in **bold** refer to the best values, whereas the underlined figures refer to be the second-best values for the evaluation metrics. capabilities in retrieving relevant code snippets. These results suggest that the C2B model is a valuable contender for code search applications. C2B hybrid model performance in comparison to other models: The C2B hybrid model is a novel approach to code search tasks, combining natural language understanding with contextual code comprehension. It consistently ranks among the top performers across multiple programming languages with an overall MRR of 0.799 compared to the highest-performing model, CodeT5. The model's architecture includes linear and nonlinear transformations and a softmax pooling layer that ensures the effective ranking and retrieval of relevant code snippets. This hybrid approach outperforms traditional models like NBow, CNN, and BiRNN and more advanced models like RoBERTa and GraphCodeBERT. The C2B hybrid model offers a robust and versatile solution for code search tasks. This model significantly enhances developer productivity and code quality in practical applications.