# Incorporating Code Structure and Quality in Deep Code Search
## 2 Related Work  
### Recurrent Neural Network  
In code search, code snippets and natural language queries are required to be embedded into vectors so that their semantic similarity can be measured. Variable-length  
Figure 1: Example code snippets.  
sequential data, such as code snippet and natural language query, are often precessed by recurrent neural network (RNN). RNN is composed of multiple neural network units and takes sequential data as input [15; 16; 17]. RNN has the ability to map sequential input into sequential hidden state. Compared with the ordinary fully connected network, the current time step output of the neurons in RNN hidden layers depends not only on the input of the current time step but also on the output of the previous time step. This feature of RNN is particularly suitable for processing code [18].  
With the length of input sequence increasing, RNN will face the problems of long-term dependencies [19; 20]. To alleviate this problem, [PERSON] introduced bi-directional long-short term memory network (BLSTM) [21]. BLSTM combined memory cells and RNN to preserve memory information. BLSTM controls selective memory and forgetting information through three gating units: forget gate, input gate, and output gate. The forget gate determines how much information in the previous time step can be preserved to the current time step. The input gate determines how many input signals will be fused, and the output gate controls how much memory is finally output. The specific calculation method is defined as follows:  
\[i_{t}=sigmoid(W_{i}X_{t}+V_{i}H_{t-1}+b_{i}) \tag{1}\]  
\[f_{t}=sigmoid(W_{f}X_{t}+V_{f}H_{t-1}+b_{f}) \tag{2}\]  
\[o_{t}=sigmoid(W_{o}X_{t}+V_{o}H_{t-1}+b_{o}) \tag{3}\]  
\[g_{t}=tanh(W_{g}X_{t}+V_{g}H_{t-1}+b_{g}) \tag{4}\]  
where \(i_{t}\),\(f_{t}\),\(o_{t}\),\(g_{t}\) represent the input state, the forget state, the output state, and the unit state of the current time step. \(X_{t}\) is the input signal of the current time step. \(H_{t-1}\) represents the output signal of the previous time step. \(W\),\(V\),\(b\) are the coefficient matrix of BLSTM to be trained. By training these weights, BLSTM can selectively ignore or strengthen the current memory \(c\) or input signals according to the current input signals and memory information. In this way, BLSTM better learns the semantic information of long sentences. \(c\) and \(h\) are determined by:  
\[c_{t}=f_{t}\cdot c_{t-1}+i_{t}\cdot g_{t} \tag{5}\]  
\[h_{t}=o_{t}\cdot tanh(c_{t}) \tag{6}\]  
where \(c_{t}\) represents the memory signal of current time step \(t\). \(h_{t}\) is the hidden state of \(t\). The above process can be simplified as:  
\[h_{t}=tanh(W_{D}[h_{t-1};w_{t}]+b_{D}),\forall t=1,2,\ldots,N_{D} \tag{7}\]  
where \(h_{t},t=1,2,\ldots,N_{D}\) are the hidden states of BLSTM. \([x;y]\in R^{2d}\) is a concentrate operate to integrate \(x\) and \(y\). \(tanh\) is a commonly used activation function. \(w_{t}\in R^{d}\) represents the embedded representation of the natural language word \(w_{t}\). \(W^{D}\) and \(b^{D}\) are the weight matrix and bias matrix of BLSTM. Experimental results show that BLSTM can overcome RNN when processing long sequential data in code search [11].