# Incorporating Code Structure and Quality in Deep Code Search
## 4 Model
### Architecture
#### 4.1.2 Method Feature Embedding Module  
Method feature embedding module (MF-Module) embeds code representations into code vectors. A code snippet can be represented into a code representation \(C=[M,S,K]\) using the code representation method described in Section 3, where \(M=m_{1},m_{2},\dots,m_{N_{M}}\) is the sequence of \(N_{M}\) tokens and represents the method name of the code snippet; \(S=s_{1},s_{2},\dots,s_{N_{S}}\) is the program slice with \(N_{S}\) consecutive tokens; \(K=\{k_{1},k_{2},\dots,k_{N_{K}}\}\) is the collection of tokens that appeared in the code snippet. Each part of the code snippet is embedded into partial embedding vectors. In order to highlight the high-quality parts of the code snippet, these partial embedding vectors are concentrated into a represented code vector with attention mechanism. MF-Module processes the code representation \(C\) according to the following steps:  
Firstly, MF-Module embeds the method name \(M\) into a sequence of separated tokens using BLSTM:  
\[h_{t}=tanh(W^{M}[h_{t-1};\mathbf{m}_{t}]+b^{M}),\forall t=1,2,\dots,N_{M} \tag{15}\]  
where \(h_{t},t=1,2,\dots,N_{M}\), are the hidden states of BLSTM. \(m_{t}\in R^{d}\)represents the embedded representation of tokens. \(W^{M}\) and \(b^{M}\) are the weight matrix and bias matrix of BLSTM.  
Secondly, MF-Module embeds program slice \(S\) into \(N_{S}\)\(d\)-demensional hidden vectors \(h_{t}\):  
\[h_{t}=tanh(W^{S}[h_{t-1};\mathbf{s}_{t}]+b^{S}),\forall t=1,2,\ldots,N_{S} \tag{16}\]  
where \(h_{t},t=1,2,\ldots,N_{S}\), are the hidden states of the BLSTM, \(s_{t}\in R^{d}\) represents the embedded representation of the tokens in program slice \(s_{t}\), \(W^{S}\) and \(b^{S}\) are the weight matrix and bias matrix of BLSTM.  
Finally, as the tokens \(K\) are not strictly ordered, MF-Module embeds the tokens \(K\) by a fully connected layer:  
\[h_{t}=tanh(W^{K}k_{t}),\forall t=1,2,\ldots,N_{K} \tag{17}\]  
where \(h_{t},t=1,2,\ldots,N_{K}\), are the embedding vectors of tokens. \(k_{t}\in R^{d}\) represents the embedded representation of tokens. \(W^{K}\) is the weight matrix of the fully connected layer.  
Since different features of code snippets have different qualities, it is necessary to give higher weight to the high-quality features. After embedding the three components (method name, program slice, and tokens) of a code snippet, MF-Module emphasizes the high-quality parts of the code snippet with attention mechanism:  
\[h_{t}=[\mathbf{m};\mathbf{s};\mathbf{k}] \tag{18}\]  
\[\alpha_{i}=\frac{exp(\widetilde{h_{i}}^{T}\cdot\alpha)}{\sum_{j=1}^{n}exp( \widetilde{h_{j}}^{T}\cdot\alpha)} \tag{19}\]  
\[c=\sum_{i=1}^{n}\alpha_{i}\cdot\widetilde{h_{i}} \tag{20}\]  
where \([x;y;z]\) is a concentrate operate to integrate \(x\), \(y\), and \(z\). \(\alpha\in R^{d}\) is initialized in random, \(h_{t},t=1,2,\ldots,N_{H}\), are \(H\)\(d\)-demensional vectors. \(\alpha_{i}\) is the attention weight of each \(\widetilde{h_{i}}\). In this way, the code vector \(c\) can be viewed as the final representation of the code snippet.  
In summary, a code snippet is first processed into three features: the method name, the program slice, and the tokens. In the training phase, each feature of \(<\)method name, program slice, tokens\(>\) is embedded into a feature vector by BLSTM, BLSTM, and MLP, accordingly. These feature vectors are then concatenated into a code vector through an attention layer.