# Multi-Feature Fusion Method for Chinese Pesticide Named Entity Recognition
## 4 Method
### BiLSTM Layer  
In BiLSTM, the input is the embedding vector of each word, which contains character embedding and word embedding, and the output is the score of each word prediction label, which will be the input of the IDCNN layer. Its structure is shown in Figure 2. BiLSTM learns the sequence data from left to right as well as from right to left through two layers of LSTM neurons, and the feature output is the future feature of the forward word and the historical feature of the backward word.  
Given input sequence \(X=[x_{1},x_{2},\cdots,x_{t},\cdots,x_{n}]\), of which \(x_{t}\) represents the input variable corresponding to time \(t\), and \(n\) is the length of the sequence. \(h_{t}\) is the output of BiLSTM, \(h_{t}^{f}\) and \(h_{t}^{b}\) represents the output vectors of the forward and backward LSTM, respectively, then:  
\[h_{t}=h_{t}^{f}\oplus h_{t}^{b} \tag{1}\]  
where \(\oplus\) represents the vector splicing operation; that is, two feature vectors are spliced according to a certain feature dimension. For a given input \(X=[x_{1},x_{2},\cdots,x_{t},\cdots,x_{n}]\), the output from the BiLSTM model \(H=[h_{1},h_{2},\cdots,h_{t},\cdots,h_{n}]\).  
Forward module output \(h_{t}^{f}\) and backward module output \(h_{t}^{b}\) can be implemented by basic LSTM modular units. The internal structure of the LSTM [9] model is shown in Figure 3:  
The LSTM model consists of three gate units: forget gate, input gate, and output gate, which determine the memory and forgetting of information at each moment.  
The forget gate controls whether the information will be forgotten at each moment.\(f_{t}\) is the output of the forget gate, which represents the feature information retained by the model that is important to the current sequence. According to the input of the current time step \(x_{t}\) and the hidden state of the previous time step \(h_{t-1}\), the forget gate determines to forget the past information carried by the cell state of the previous layer; we have:  
\[f_{t}=\sigma(W_{fx}x_{t}+W_{fh}h_{t-1}+W_{fc}c_{t-1}+b_{f}) \tag{2}\]  
Figure 1: Framework of Chinese pesticide named-entity recognition system. Notes: The English translation of the Chinese character is “This product is pyrazone”.  
Figure 2: BiLSTM model diagram.  
The input gate determines how much new information is added to the model. As the input of input gate, \(i_{t}\) represents how much of the input information needs to be filtered; \(\widetilde{c}_{t}\) represents the current cell state, then:  
\[i_{t}=\sigma(W_{ix}x_{t}+W_{ih}h_{t-1}+W_{ic}c_{t-1}+b_{i}) \tag{3}\]  
\[\widetilde{c}_{t}=\tanh(W_{cx}x_{t}+W_{ch}h_{t-1}+b_{c}) \tag{4}\]  
The updated \(c_{t}\) as part of the input for the next time step, we have:  
\[c_{t}=f_{t}c_{t-1}+i_{t}\widetilde{c}_{t} \tag{5}\]  
The output gate determines whether there is information output at each moment. As the output result of the output gate, \(o_{t}\) indicates the selected output characteristic according to the correlation in the modeling. \(h_{t}\) represents the hidden state generated by the gate value of the output gate, acting on the updated cell state \(c_{t}\), and \(\tanh\) is used as an activation function, and as part of the input for the next time step. The whole process of the output gate is to generate the hidden state \(h_{t}\), then there are:  
\[o_{t}=\sigma(W_{ox}x_{t}+W_{oh}h_{t-1}+W_{oc}c_{t}+b_{o}) \tag{6}\]  
\[h_{t}=o_{t}\text{tanh}(c_{t}) \tag{7}\]  
where \(h_{t-1}\) denotes the hidden state at time \(t-1\); \(c_{t-1}\) represents the state of the cell at time \(t-1\); \(W_{*}\) is the corresponding trainable parameter obtained by random initialization at the beginning of training and optimized continuously; \(b_{*}\) is the corresponding bias parameter that does not participate in the training and is obtained by random initialization.