# Research on Medical Text Parsing Method Based on BiGRU-BiLSTM Multi-Task Learning
## 3 Materials and Methods
### Semantic Feature Extraction  
#### 3.2.1 Bidirectional Long Short-Term Memory  
Long short-term memory (LSTM) [27] is a variant of recurrent neural networks (RNNs) specifically designed to solve long-term dependency problems. By introducing gating mechanisms to control the input, output, and forgetting of information, it can effectively capture and retain in long-term memory the dependencies in the sequence.  
The LSTM unit consists of three gates (forget gate, input gate, and output gate) and a memory unit, which work together to process and remember long-term dependencies in the sequence. Their structure is shown in Figure 5.  
The forget gate determines how many memories of the past to retain, and decides whether to forget the past information based on the current input \(x_{t}\) and the hidden state of the previous time step \(h_{t-1}\). When \(f_{t}\) approaches 0, it indicates that the model will forget the memory information of the previous time step. As \(f_{t}\) approaches 1, the model retains more historical information. The input gate determines what information from the current input \(x_{t}\) needs to be added to the memory unit, and updates the current memory with the candidate memory \(\widetilde{C}_{t}\). The input gate determines how to update the memory unit by  
Figure 4: Embedding schematic diagram of BERT.  
\(i_{t}\), and \(\widetilde{C}_{t}\) generates a new candidate memory that is added to the memory state under the input gate's control. The closer \(i_{t}\) is to 1, the more information currently entered is retained and added to the memory unit. The output gate determines the current hidden state \(h_{t}\) and the final output. When the output gate \(o_{t}\) approaches 1, it indicates that the current memory state \(C_{t}\) will greatly affect the output of the current time step. The specific calculation formula is:  
\[f_{t}=\sigma\left(W_{f}\cdot[h_{t-1},x_{t}]+b_{f}\right) \tag{4}\]  
\[\widetilde{C}_{t}=\tanh(W_{C}\cdot[h_{t-1},x_{t}]+b_{C}) \tag{5}\]  
\[i_{t}=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i}) \tag{6}\]  
\[C_{t}=f_{t}*C_{t-1}+i_{t}*\widetilde{C}_{t} \tag{7}\]  
\[o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o}) \tag{8}\]  
\[h_{t}=o_{t}*\tanh(C_{t}) \tag{9}\]  
where \(\sigma\) is the sigmoid function, \(W\) and \(b\) are the weights and biases of the forgetting gate, input gate and input gate, \(\tanh\) is the hyperbolic tangent function, and \(*\) represents element-by-element multiplication.  
Bidirectional long short-term memory (BiLSTM) is a variant of a recurrent neural network that combines the advantages of standard LSTM and bidirectional propagation, and is particularly suitable for sequential data modeling, such as text processing tasks in natural language processing. BiLSTM improves on traditional one-way LSTM by considering both past and future contextual information at each time step. The output of BiLSTM is a concatenation of forward and backward hidden states:  
\[\text{Forward LSTM: }\overrightarrow{h_{t}}=\text{LSTM}\left( \overrightarrow{h_{t-1}},x_{t}\right) \tag{10}\]  
\[\text{Backward LSTM: }\overleftarrow{h_{t}}=\text{LSTM}\left( \overleftarrow{h_{t+1}},x_{t}\right) \tag{11}\]  
\[h_{t}=\left[\overrightarrow{h_{t}};\overleftarrow{h_{t}}\right] \tag{12}\]  
where \([\ ;\ ]\) indicates the connection operation.