# Feature Fusion Text Classification Model Combining CNN and BiGRU with Multi-Attention Mechanism
## 3 MATT-CNN+BiGRU Feature Fusion Model
### The Brief Introduction of MATT-CNN+BiGRU
#### 3.1.2 Basic Structure of GRU and BiGRU  
Along with the rapid development of LSTM in the field of natural language processing, especially text categorization, and the increase in the number of samples, the training time is long, the parameters are many, and the internal computational complexity is high. Based on this, [PERSON] et al. proposed a simpler GRU model in 2014 [30]. The GRU model maintains the original LSTM original effect, with a simpler structure, fewer parameters, a better convergence model, plus a GRU model. It consists of two doors, an update door and a reset door. The update gate determines the extent to which the previous output hidden layer affects the current layer. The larger the value is, the stronger the influence is. The reset gate determines the extent to which the previous hidden layer information is ignored. The smaller the reset gate value is, the more ignored the information is. The specific structure of the GRU is shown in Figure 1:The GRU model is updated in the following ways:  
\[r_{t}=\sigma(W_{r}*[h_{t-1},x_{t}]) \tag{5}\]  
\[z_{t}=\sigma(W_{z}*[h_{t-1},x_{t}]) \tag{6}\]  
\[\widetilde{h}_{t}=\tanh\Bigl{(}W_{\widetilde{h}}*[r_{t}*h_{t-1},x_{t}]\Bigr{)} \tag{7}\]  
\[h_{t}=(1-z_{t})*h_{t-1}+z_{t}*\widetilde{h}_{t} \tag{8}\]  
where \(r_{t}\) represents the reset gate at time \(t\), \(Z_{t}\) represents the update gate at time \(\widetilde{h}_{t}\), \(h_{t}\) represents the candidate activation state at time \(t\), \(h_{t-1}\) represents the active state at time \(t\), and \(h_{t-1}\) represents the hidden layer state at time \((t-1)\). The update gate \(z\) is determined by the history information that the current state needs to be forgotten and the new information that is accepted; the reset gate \(r\) is determined by the information obtained from the history information of the candidate state.  
GRU is a kind of one-way neural network structure, and a one-way neural network is always output from the back. However, in the text sentiment classification, the output of the current moment often has a relationship with the moments before and after. BiGRU is a one-way, opposite direction, and the output is determined by the GRU common state of the two GRUs to construct a neural network model. Occurring at each moment, the input provides two GRUs in opposite directions, and the output is determined by the two unidirectional GRUs. The specific structure of BiGRU is shown in Figure 2.  
\[h_{t}=w_{t}\overset{\rightarrow}{h}_{t}+v_{t}\overset{\leftarrow}{h}_{t}+b_{t} \tag{11}\]  
The \(GRU()\) function represents a nonlinear change to the input word vector, and the word vector is encoded into a corresponding GRU hidden layer state. \(w_{t}\) and \(v_{t}\) represent the weights corresponding to the forward hidden layer state \(\overset{\rightarrow}{h}_{t}\) and the reverse hidden state, \(\overset{\leftarrow}{h}_{t}\), respectively, corresponding to the bidirectional GRU at the time \(t\); \(b_{t}\) represents the offset value corresponding to the hidden layer state at the time \(t\).