# C2B: A Semantic Source Code Retrieval Model Using CodeT5 and Bi-LSTM
## 4 Proposed Architecture for C2B Hybrid Model
### The Main Building Blocks of the Proposed C2B Hybrid Model
#### 4.1.3 C2B Layer  
This layer performs the following operations:  
* Concatenation: To combine the query and code embedding vectors (_Vec_) from CodeT5 and Bi-LSTM using concatenation, we concatenate the vectors along a specific axis. Let us assume the query embedding vectors obtained from CodeT5 and Bi-LSTM are represented as \"\(Qemb_{CodeT5}\)\" and \"\(Qemb_{BiLSTM}\)\", respectively. Similarly, the code embedding vectors from CodeT5 and Bi-LSTM are represented as \"\(Cemb_{CodeT5}\)\" and \"\(Cemb_{BiLSTM}\)\", respectively. The concatenation \(Concat\) operation can be represented using Equations (11a) and (11b). \[Concat_{(Q\_Vec)}=Concat[Qemb_{CodeT5},Qemb_{BiLSTM}]\] (11a) \[Concat_{(c\_Vec)}=Concat[Cemb_{CodeT5},Cemb_{BiLSTM}]\] (11b) Here, we concatenate the query embedding vectors from CodeT5 and Bi-LSTM to create a new joint representation \"\(Concat_{(Q\_Vec)}\)\" Similarly, we concatenate the code embedding vectors from CodeT5 and Bi-LSTM to create a new joint representation \"\(Concat_{(C\_Vec)}\)\". The resulting shapes of \"\(Concat_{(Q\_Vec)}\)\" and \"\(Concat_{(C\_Vec)}\)\" will be \((d_{q}+d_{b})\), where \"\(d_{q}\)\" represents the dimensionality of the query embeddings from CodeT5, \"\(d_{b}\)\" represents the dimensionality of the query embeddings from Bi-LSTM, \"\(d_{c}\)\" represents the dimensionality of the code embeddings from CodeT5, and \"\(d_{l}\)\" represents the dimensionality of the code embeddings from Bi-LSTM. These concatenated representations can then be used for downstream tasks that require joint representations of the query and code information from both CodeT5 and Bi-LSTM. * Attention: The first step is to generate query and code embeddings. This typically involves mapping each token in the query and code to a continuous vector representation using techniques like word embeddings or pretrained language models. The attention mechanism calculates the relevance or similarity between the query and code tokens. One common approach is to use the dot product or cosine similarity to measure the similarity between the embeddings of each query token and code token. Cosine similarity attention is used to calculate the attention weights based on the cosine similarity between the query embedding and code embedding vectors. This measures the cosine of the angle between the two vectors, capturing their directional similarity. The attention weights represent the importance or similarity of each code token with respect to the query. They can be normalized using softmax to ensure a valid probability distribution over the code tokens. After calculating the attention weights, they are used to weight the code embeddings. Each code embedding is multiplied by its corresponding attention weight. The resulting weighted code embeddings are then summed or averaged to obtain an aggregated code representation. The aggregation step combines the relevant parts of the code based on the attention weights, focusing the model's attention on the code tokens that are most similar to the query. Let us consider a query token embedding \(Q_{i}\) and a code token embedding \(C_{i}\). The cosine similarity attention is calculated using the cosine similarity between the query and code embeddings. -  
* Cosine similarity calculation: Cosine similarity calculates the cosine of the angle between two vectors, representing their directional similarity as shown in Equation (12). Here is the formula for calculating the cosine similarity between the query and code embeddings: \[Cosine\ Similarity(Q_{i},C_{i})=\frac{\big{(}Q_{i}\cdot C_{j}\big{)}}{(\|Q_{i} \|)\cdot\big{\|}C_{j}\big{\|}}\] (12) In Equation (12), \((Q_{i},C_{i})\) denotes the dot product between the query embedding \(Q_{i}\) and the code embedding \(C_{i}\). \(\|Q_{i}\|\) represents the Euclidean norm (magnitude) of the query embedding \(Q_{i}\). \(\|C_{j}\|\) represents the Euclidean norm (magnitude) of the code embedding \(C_{j}\).