## 3 Re-Ranking Model Based on Artificial Neural Network
The query and passage encoders convert the query vector \(Q=(q_{1},\ q_{2},\ \dots,\ q_{n})\) with \(n\) word vectors and the passage vector \(P=(p_{1},\ p_{2},\ \dots,\ p_{m})\) with \(m\) word vectors into the encoded vectors, \(\overset{\leftrightarrow}{Q}\) and \(\overset{\leftrightarrow}{P}\), respectively, which embed contextual information using bidirectional gated recurrent units (biGRUs) [16], as represented by Equation (1):  
\[\begin{split}\overset{\rightarrow}{h}_{i}^{Q}&= \text{GRU}\bigg{(}E(q_{i}),\ \overset{\rightarrow}{h}_{i-1}\bigg{)},\ \overset{\leftarrow}{h}_{i}^{Q}=\text{GRU}\bigg{(}E(q_{i}),\ \overset{ \leftarrow}{h}_{i+1}\bigg{)},\ \overset{\leftarrow}{h}_{i}^  
Using the CNNs depicted in Figure 5, the encoded vectors \(\overleftrightarrow{Q}\) and \(\overleftrightarrow{P}\) are represented as three types of phrase vectors: unigram phrase vectors \(\overleftrightarrow{Q_{1}}\) and \(\overleftrightarrow{P_{1}}\), bigram phrase vectors \(\overleftrightarrow{Q_{2}}\) and \(\overleftrightarrow{P_{2}}\), and trigram phrase vectors \(\overleftrightarrow{Q_{3}}\) and \(\overleftrightarrow{P_{3}}\). To simplify the equations, we rewrite the \(n\)-gram phrase vectors as \(\overleftrightarrow{Q_{n}}\) and \(\overleftrightarrow{P_{n}}\). Thereafter, the \(n\)-gram phrase vectors are input to the attention layer.  
The attention layer consists of two sublayers: a passage-query (P-Q) attention layer and an attention pooling layer. In the P-Q attention layer, the proposed model calculates the degrees of associations between the \(n\)-gram phrases in a query and those in a passage. The P-Q attention vector of an \(n\)-gram phrase vector, \(P_{\text{att}}^{n}\), is calculated using the scaled dot product [18] expressed in Equation (2):  
\[P_{\text{att}}^{n}=\text{softmax}\left(\frac{\overleftrightarrow{P_{n}} \widetilde{Q_{n}}^{T}}{\sqrt{d_{\widetilde{Q}_{n}}}}\right)\overleftrightarrow{P _{n}} \tag{2}\]  
In Equation (2), \(\overleftrightarrow{P_{n}}\) denotes an \(n\)-gram phrase vector of a passage, and \(\widetilde{Q}_{n}\) denotes an \(n\)-gram phrase vector of a query mapped onto the \(d_{\widetilde{Q}_{n}}\) dimension using a conventional pooling mechanism [19]. That is, the proposed model converts \(\overleftrightarrow{Q_{n}}\) and \(P_{\text{att}}^{n}\) into fixed-length vectors in the attentive pooling layer, as expressed in Equation (3):  
\[ \begin{split} g_{n}^{q}&=\text{softmax}\left(w_{n}^{ q}\Big{(}\overleftrightarrow{Q_{n}}\Big{)}+b_{q}\right)\\ \widetilde{Q}_{n}&=\left(\sum g_{n}^{q}\times \overleftrightarrow{Q_{n}}\right)\\ g_{n}^{p}&=\text{softmax}\left(w_{n}^{p}\Big{(}P_{ \text{att}}^{n}\Big{)}+b_{p}\right)\\ \widetilde{P}_{n}&=\left(\sum g_{n}^{p}\times P_{ \text{att}}^{n}\right)\end{split}  \tag{3}\]  
In Equation (3), \(w_{n}\) and \(b_{n}\) denote a weight matrix and bias vector, respectively. Therefore, \(g_{n}^{q}\) and \(g_{n}^{p}\) are generated by feed-forward neural networks (FNNs). Then, \(\times\) denotes a cross product between two vectors. The normalized attention vectors \(\widetilde{Q}_{n}\) and \(\widetilde{P}_{n}\) for the \(n\)-gram phrase vectors are input to the similarity network. Figure 5: Generation of phrase-level features using convolutional neural network (CNN)s.  
To calculate the similarity between the normalized attention vector \(\overline{P}_{n}\) and the normalized query vector \(\overline{Q}_{n}\), we adopt the similarity vector representation proposed in a report on sentence embedding by [PERSON] et al. [20], as expressed in Equation (4):  
\[\text{sim}\big{(}\overline{Q}_{n},\ \overline{P}_{n}\big{)}=\bigg{[}\overline{Q}_ {n},\ \overline{P}_{n},\ \Big{|}\overline{Q}_{n}-\overline{P}_{n}\Big{|}, \overline{Q}_{n}\times\overline{P}_{n}\bigg{]} \tag{4}\]  
In Equation (4), \(\Big{[}\overline{Q},\ \overline{P},\ \ldots\Big{]}\) and \(\times\) denote the concatenation of vectors and a cross product, respectively.