# Relational Graph Convolutional Network for Text-Mining-Based Accident Causal Classification
## 2 Methodology
### R-Gcn  
A graph convolutional network (GCN) [37; 38] is a topological network model based on graph theory, which was originally proposed to deal with non-Euclidean data. On the basis of a graph neural network (GNN), the convolution operation in GCNs is performed to realize the differentiable information transfer process of adjacent graph nodes. The transmitted information is usually the hidden state of the node itself, which is essentially high-dimensional feature vectors. GCNs naturally have the advantage of processing text  
Figure 2: Architecture of BERT in [30]. data [39]. Every word, symbol and datum in the text can be regarded as a node of the network. Based on the word co-occurrence relationship and the relationship between document words, a text graph can be built for a specific corpus and then a text graph convolutional network (text-GCN) model can be built. Let us suppose that a directed graph \(\mathcal{G}=(\mathcal{V},\mathcal{E})\) has nodes \(v_{i}\in\mathcal{V}\) and edges \((v_{i},v_{j})\in\mathcal{E}\). According to the definition by [PERSON] et al. [37], each node \(v_{i}\) contains a self-loop edge, namely, \((v_{i},v_{i})\in\mathcal{E}\). Let \(X\in R^{n\times m}\) be a matrix containing the eigenvectors of \(n\) nodes, where \(m\) is the dimension of the eigenvectors and each row of \(x_{v}\in R^{m}\) is the eigenvector of node \(v\). Let \(A\) be the adjacency matrix of graph \(\mathcal{G}\) and \(D\) be the degree matrix of \(\mathcal{G}\), where \(D_{ii}=\sum_{j}A_{ij}\). The diagonal element of \(A\) is 1 due to the presence of self-loops. One convolutional layer of the GCN can only capture near-domain information. When multiple GCN layers are stacked, larger domain information is aggregated. For a single-layer GCN, the \(k\)-dimensional node feature matrix \(L^{(1)}\in R^{n\times k}\) is calculated as follows:  
\[L^{(1)}=\rho\big{(}\tilde{A}XW_{0}\big{)}\,, \tag{1}\]  
where \(\tilde{A}=D^{-\frac{1}{2}}AD^{-\frac{1}{2}}\) is the normalized symmetric adjacency matrix and \(W_{0}\in R^{m\times k}\) is the weight matrix. As mentioned earlier, higher-order neighborhood information can be incorporated by stacking multiple GCN layers. \[L^{j+1}=\rho\big{(}\tilde{A}L^{j}W_{j}\big{)}\,, \tag{2}\]  
where \(j\) represents the number of layers and \(L^{0}=X\). Therefore, the forward propagation process in the R-GCN can be defined as  
\[h_{i}^{(l+1)}=\text{ReLU}\left(\sum_{u\in\mathcal{N}(v_{i})}\frac{1}{c_{i}}W^{ (l)}h_{u}^{(l)}\right), \tag{3}\]  
where \(\mathcal{N}_{r}(v_{i})\) represents the set of neighbor nodes whose relationship is \(r\) for node \(i\), \(l\) denotes the layer number and \(c_{i}\) is a normalization constant. It should also be noted that the bias term is ignored in the formula and the bias is added to the calculation to promote the convergence of the model during training. By constructing a large heterogeneous text graph containing word nodes and document nodes, global word co-occurrences can be explicitly modeled and graph convolutions can be easily applied. The number of text graph nodes \(|v|\) is equal to the number of documents (corpus size) plus the number of distinct words in the corpus (vocabulary size). The text-GCN simply lets the feature matrix \(X=I\) be the identity matrix, meaning that each word or document is represented as a one-hot vector as input to the text-GCN. Edges are established between nodes based on word occurrences in the document (document node-word node edges) and word co-occurrences in the entire corpus (word node-word node edges). The weight of an edge between a document node and a word node is the term frequency-inverse document frequency (TF-IDF) of that word in the document. However, due to the prior information of the syntactic structure between sentences, the traditional GCN can only represent text as an isomorphic graph and the relationship between different words may be different, which also means that the topological structure of the text is essentially heterogeneous graph [40]. [PERSON] et al.