## 3 Proposed Methods
### Recurrent Networks Classifiers
#### 3.2.1 Recurrent Neural Networks  
A recurrent neural network (RNN) [50] is a type of artificial neural network that can store information about preceding inputs in a sequence in its memory. Figure 5 depicts the structure of basic RNNs. In RNNs, each word of the input sequence \(\mathbf{x}_{1},\ \mathbf{x}_{2},\ \ldots,\ \mathbf{x}_{n}\) turns into vector form \(\mathbf{\hat{y}}_{t}\) by using the following equations:  
\[\mathbf{h}_{t}\ =\ \mathbf{H}(\mathbf{W}_{xh}\mathbf{x}_{t}\ +\ \mathbf{W}_{hh}\mathbf{h}_{t-1}\ +\ \mathbf{b}_{h}) \tag{1}\]  
\[\mathbf{\hat{y}}_{t}\ =\ \mathbf{W}_{hy}\mathbf{h}_{t}\ +\ \mathbf{b}_{y} \tag{2}\]  
where \(\mathbf{W}_{xh}\), \(\mathbf{W}_{hh}\), and \(\mathbf{W}_{hy}\) represent the weight matrices, \(\mathbf{h}_{t}\) is the vector of hidden states that capture the information at the time slice \(t(t=1,2,\ \ldots,\ T)\), \(\mathbf{b}\) represent biases, and \(\mathbf{H}\) is the collection of activation functions for the hidden layer. The basic RNN with unidirectional flow passes the data from left to right. Shared parameters are used for each time step. In Figure 4, \(\mathbf{W}_{xh}\), \(\mathbf{W}_{hh}\), and \(\mathbf{W}_{hy}\) are the same for each time step. When generating the prediction at time \(t\), it uses not only the current input \(\mathbf{x}_{t}\) at time \(t\) but also information from prior input at time \(t-1\) through the activation parameter \(\mathbf{H}\) and weights \(\mathbf{W}\), which passes from the previously hidden layer to the current hidden layer. One problem with this RNN is that it can only make predictions based on the information that comes before it. In that case, RNN uses information from earlier in the sequence to make a prediction at a certain time, but not information from later in the sequence. To train RNN, back-propagation through time (BPTT) is often used [51]. However, due to the gradient-vanishing and exploding problem, it is not viable to train standard RNNs with BPTT [52]. It is difficult to propagate errors from later time steps back to previous time steps enough to adjust network settings correctly. Thus, in the subsequent development of the RNNs, the gated recurrent unit (GRU) and the long short-term memory (LSTM) network have also been employed to resolve this concern [53; 54; 55].  
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Tag** & **Definition** \\ \hline B (Begin) & Prefix before a tag indicates that the tag is the beginning of a chunk. \\ I (Inside) & Prefix before a tag indicates that the tag is inside a chunk. \\ O (Outside) & Tag indicates that a token belongs to a non-chunk (outside). \\ \hline \hline \end{tabular}
\end{table}
Table 4: IOB tagging scheme.  
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Named-Entity Type** & **Count** & **Ratio (\%)** \\ \hline B-INGREDIENT & 210,082 & 15.03 \\ B-PRODUCT & 17,325 & 1.24 \\ B-QUANTITY & 209,867 & 15.01 \\ B-STATE & 135,315 & 9.68 \\ B-UNIT & 174,993 & 12.52 \\ I-INGREDIENT & 240,436 & 17.20 \\ I-PRODUCT & 55,212 & 3.95 \\ I-QUANTITY & 1919 & 0.14 \\ I-STATE & 130,158 & 9.31 \\ I-UNIT & 2353 & 0.17 \\ O (outside or non-entity chunk) & 220,300 & 15.76 \\ \hline Total & 1,397,960 & 100 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Distribution of the dataset for named entities.