# The Study on the Text Classification Based on Graph Convolutional Network and BiLSTM +
## 3 Proposed Method
### Feature Fusion and Classification  
Since words are represented as nodes, the neighborhood information of nodes is aggregated through an adjacency matrix, ignoring the order structure of a text, resulting in the loss of contextual semantic information of a text. Therefore, we combine the global vocabulary information obtained by GCN with the context information obtained by the pre-trained BERT. While using the global vocabulary information, we can also take into account the order of words to capture contextual information. In order to dig deeper into the semantics, the sequence model LSTM can be used to fuse the two parts of information, but it can only capture features in a single direction, from front to back or from back to front. The mode of human reading text is to use the foregoing and the following to jointly infer the meaning of a word when encountering a word. Therefore, BiLSTM models that can capture features in both front-to-back and back-to-front directions are more in line with this pattern. Considering that text has a hierarchical structure, the text is composed of sentences, and sentences are composed of words. Therefore, we use a two-layer BiLSTM to learn word features and sentence features, respectively. Since the feature words have different importance to the text, and the output vector of BiLSTM does not distinguish the importance of words, we add an attention mechanism to its hidden output. We changed the initialization method of the context vector in the attention mechanism and combined the bidirectional word hidden features as the initial value of the context vector to guide the learning of the attention mechanism. At the same time, in order to prevent the deepening of the model layers from affecting the classification results, we introduced the residual operation. The specific structure of the model feature fusion module is shown in Figure 4.  
The module consists of a semantic feature splicing layer, word-level feature extraction layer, contextual attention layer, and a sentence-level feature extraction layer. First, the GCN and the pre-trained BERT model convert the text word sequence into two different word vector representations. Secondly, the corresponding fusion of the obtained two different word vector representations is input into the first-layer BiLSTM model to extract the semantic dependency information between words within a single sentence, and a layer of attention mechanism is added to the first-layer BiLSTM, constructs an overall representation of the current sentence. Here, the bidirectional word hidden features are combined as the context vector of the attention mechanism. Then the obtained sentence feature representation is input to the second layer BiLSTM to extract semantic dependency information between sentences within the text. The learned sentence feature representation and the sentence feature representation obtained by the context attention layer are subjected to residual operation to form an overall text representation. Finally, the overall text representation is input to the Softmax classification layer to obtain the final category of the text.