# Knowledge-Graph- and GCN-Based Domain Chinese Long Text Classification Method
## 3 Domain Chinese Long Text Classification Model
### Entity Relations and Dependencies-Based Graph Convolutional Neural Network  
In the classical GCN, the connection relationship between words is not distinguished, and if there is a connection between two nodes, then that element in the adjacency matrix is 1; otherwise, it is 0. Therefore, the GCN model cannot distinguish the importance of different connections, nor can it reflect the difference between entity relationships and dependency relationships among nodes. To fully learn and utilize the knowledge graph information in Section 3.1, we use two GCN modules--one for entity-relationship information and the other for dependency information--drawing on the idea of the A-GCN model proposed by [PERSON] et al. [26]. We also improve the original simple connection matrix of GCN by using an attention mechanism and adding node and edge-type features. Specifically, set the initial feature matrix of nodes, \(X=I_{n_{doc}+n_{entity}}\), where \(n_{doc}\) is the number of text nodes and \(n_{entity}\) is the number of extracted entities, and use BERT to obtain the node embedding. For an utterance of length n, a connection matrix, \(A=\big{(}a_{i,j}\big{)}_{n\times n}\), is constructed, and when there is a syntactic dependency or entity relationship between words \(x_{i}\) and \(x_{j}\), then \(a_{i,j}=1\), and vice versa, when no relationship exists, then \(a_{i,j}=0\). If there exists any word \(x_{i}\), then the output of the lth level of GCN is represented as \(h_{i}^{(l)}\), the expression of which is shown in Equation (2):  
\[h_{i}^{(l)}=\sigma\Bigg{(}\sum_{j=1}^{n}a_{i,j}\Big{(}W^{(l)}.h_{j}^{(l-1)}+b^ {(l)}\Big{)}\Bigg{)} \tag{2}\]where \(a_{i,j}\in A\), \(h_{j}^{(l-1)}\) is the output of word \(x_{j}\) at GCN layer \(l-1\), \(W^{(l)}\) is the trainable matrix, \(b^{(l)}\) is the bias of GCN at layer \(l\), and \(\sigma\) is the activation function _ReLU_. On this basis, two graphs are constructed based on syntactic dependency types and entity-relationship types. Denote the matrix of entity-relationship types by \(B=\big{(}r_{i,j}\big{)}_{n\times n}\), where \(r_{i,j}\) is the entity-relationship type with \(x_{i}\) and \(x_{j}\), and map each type, \(r_{i,j}\), to its embedding, \(e_{i,j}^{r}\). Calculate the weight of the connection between nodes \(I\) and \(j\) at layer \(l\) in GCN based on the attention mechanism as \({p^{(l)}}_{i,j}\), the expression of which is shown in Equation (3):  
\[{p^{(l)}}_{i,j}=\frac{a_{i,j}\exp\Big{(}c_{i}^{(l)}c_{j}^{(l)}\Big{)}}{\sum_{j= 1}^{n}a_{i,j}\exp\Big{(}c_{i}^{(l)}c_{j}^{(l)}\Big{)}} \tag{3}\]  
where \(c_{i}^{(l)}\) and \(c_{j}^{(l)}\) are the intermediate vectors of \(x_{i}\) and \(x_{j}\), respectively, the expressions of which are shown in Equations (4) and (5):  
\[c_{i}^{(I)}=h_{i}^{(l-1)}\oplus e_{i,j}^{r} \tag{4}\]  
\[c_{j}^{(I)}=h_{j}^{(l-1)}\oplus e_{i,j}^{r} \tag{5}\]  
where \(h_{i}^{(l-1)}\) and \(h_{j}^{(l-1)}\) are the outputs of nodes \(i\) and \(j\) at the \(l-\)1 st layer of GCN, respectively, with \(\oplus\) denoting the splicing. The expression of the final entity-relationship GCN output vector, \(h_{rel,gen,i}^{(l)}\), is shown in Equation (6):  
\[h_{rel,gen,i}^{(l)}=\sigma\Big{(}\sum_{j=1}^{n}p_{i,j}^{(l)}\Big{(}W^{(l)}. \widetilde{h}_{rel,gen,j}^{(l-1)}+b^{(l)}\Big{)}\Big{)} \tag{6}\]  
\(\widetilde{h}_{rel,gen,j}^{(l-1)}\) is the vector with the added edge-type information, the expression of which is shown in Equation (7):  
\[\widetilde{h}_{rel,gen,j}^{(l-1)}=h_{j}^{(l-1)}+W_{T}^{(l)}.e_{i,j}^{r} \tag{7}\]  
where \(W_{T}^{(l)}\) maps the dependency-type embedding, \(e_{i,j}^{r}\), to the same dimension as \(h_{j}^{(l-1)}\), and \(h_{j}^{(l-1)}\) is the output of word \(x_{j}\) at GCN layer \(l-1\). To obtain the syntactic dependencies, the text is experimented with Stanza, an open-source NLP tool from Stanford University, to process it and obtain the syntactic dependency tree.