## 4 Methodology
### Bidirectional LSTM  
We now describe the Bidirectional Long Short-Term Memory network for this paper. The model receives a sequence of tokens \((w_{1},w_{2},w_{3},\ \dots,w_{n})\) as input and predicts a label corresponding to each of the input tokens.  
First, a sequence of tokens \((w_{1},w_{2},w_{3},\ \dots,w_{n})\) is obtained. The tokens are mapped to a distributed vector space, which we have explained, resulting in a sequence of word embeddings \((i_{1},i_{2},i_{3},\ \dots,i_{n})\).  
Next, the embeddings are given as input to two LSTM components moving in opposite directions through the text, creating context-specific representations. The respective forward- and backward-conditioned representations are concatenated for each word position, resulting in representations that are conditioned on the whole sequence:  
\[\overrightarrow{h_{i}^{\overrightarrow{i}}}=LSTM(c_{i},\overrightarrow{h_{ i-1}^{\overrightarrow{i}}});\overleftarrow{h_{i}^{\overrightarrow{i}}}=LSTM(c_{i}, \overleftarrow{h_{i+1}^{\overrightarrow{i}}}) \tag{5}\]  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Word** & **Type of Feature** & **Feature Vector Value** \\ \hline \multirow{4}{*}{(learn)} & POS & 0000000000010000000000000 \\  & DP & 00000000001000 \\  & DIS & 1000000 \\ \hline \multirow{4}{*}{(car)} & POS & 0000000000000000010000000000 \\  & DP & 000000000001000 \\ \cline{1-1}  & DIS & 000100 \\ \hline \multirow{4}{*}{(Shanghai)} & POS & 00000000000000000000000000 \\ \cline{1-1}  & DP & 0000001000000000 \\ \cline{1-1}  & DIS & 0010000 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Examples of feature representation.  
\[h^{*}=[\overrightarrow{h_{N_{f}}^{*}},\overleftarrow{h_{1}^{*}}] \tag{6}\]  
Then, the results are mapped to the m dimension, resulting in a sequence of probability \((p_{1},p_{2},p_{3},\ \ldots,p_{n})\). \(M\) is the number of entity types, and \(p_{ij}\) describes how confident the network is that the label on the \(w_{i}\) word is \(j\).  
Finally, a CRF is used, which conditions each prediction on the previously predicted label. In this architecture, the last hidden layer is used to predict the confidence scores for the word with each of the possible labels. A separate weight matrix is used to learn the transition probabilities between different labels, and the Viterbi algorithm is used to find an optimal sequence of weights. Given that \(y\) is a sequence of labels \([y_{1},y_{2},y_{3},\ \ldots,y_{n}]\), then the CRF score for this sequence can be calculated as  
\[score(x,y)=\sum_{i=1}^{n}P_{i,y_{i}}+\sum_{i=1}^{n+1}A_{y_{i-1},y_{i}}. \tag{7}\]  
\(P_{i,y_{i}}\) shows how confident the network is that the label on the \(w_{i}\) word is \(y_{i}\). \(A_{y_{i-1},y_{i}}\) shows the likelihood of transitioning from label \(y_{i-1}\) to label \(y_{i}\), and these values are optimized during training. The output from the model is the sequence of labels with the largest \(score(x,y)\), which can be found efficiently using the Viterbi algorithm. In order to optimize the CRF model, the loss function maximizes the score for the correct label sequence, while minimizing the scores for all other sequences:  
\[E=-score(x,y)+log\sum_{y\in Y}e^{s(y)}. \tag{8}\]  
\(Y\) is the set of all possible label sequences.