# A Survey of Information Extraction Based on Deep Learning
## 2 Entity Relationship Extraction Based on Deep Learning
### Supervised Learning--Pipeline Extraction
#### 2.1.3 GNNs  
RNN models can only deal with directed positional acyclic graphs (DPAGs), and are generally used to deal with graph-focused problems. Graph neural networks (GNNs) process input data encoded as generic labeled graphs. Graph neural networks can model node-centric and graph-centric functions. GNN can be used for sentence extraction, document clustering, and classification [27]. CNN can only process regular Euclidean data, such as images (two-dimensional grid) and text (one-dimensional sequence), for non-Euclidean data, the processing results are not satisfactory. The disadvantage of the graph embedding model is that the parameters are not shared among the nodes in the encoder and the number of parameters grows linearly with the number of nodes, resulting in low computational efficiency. Second, the direct embedding method lacks the ability to generalize to dynamic graphs and cannot be generalized to new graphs. Based on CNN and graph embeddings, a graph neural network (GNN) is proposed to collectively aggregate the information in the graph structure. Models of GNN can be divided into graph convolutional networks (GCN), graph attention networks (GAT), graph spatial-temporal networks (GAT), etc. [28]. Currently, researchers more often use the GCN model and GAT model for relationship extraction. The GCN model combines the features of nodes themselves and their neighbors to analyze nodes, while the disadvantage of the GCN model is that the same weight is assigned to each neighboring node, but the strength of association between different neighboring nodes is usually different. Compared with the GCN model, the GAT model allows different weights to be assigned to the same neighboring nodes, but most of the current GAT studies only have information about first-order neighbors and have not explored information about higher orders. Graph neural networks (GNNs), which can represent an entire document and consider implicit correlations between different entities, show great potential for document-level relationship extraction. [PERSON] et al. proposed a new edge-oriented graph neural network based on document structure and external knowledge for document-level medical relationship extraction, called SKEoG, which can make full use of document structure and external knowledge [9]. In terms of GCN, [PERSON] et al. constructed the multi-head attention graph convolutional network (multi-GCN) to improve the performance of relationship extraction [29]. [PERSON] et al. proposed an attention-based graph convolutional network (AGCN) to perform DDI (drug-drug interaction) extraction. Instead of the previous rule-based pruning, this model used a new attention-based pruning strategy [30]. A feature combination-based  
Figure 4: Global-level relationship extractor model GREG, which contained two modules, including a context-aware LSTM relationship extraction module and a knowledge graph constructor module for generating a knowledge graph from a given document [26]. graph convolutional neural (FC-GCN) model was proposed by [PERSON] et al. [31]. Although deep models are more effective, deeper models face the vanishing gradient problem. [PERSON] et al. [32] proposed densely connected networks (DenseNets) to solve these problems. DenseNets provide direct connections between entire layers, improving the flow of gradients and information throughout the entire layer. Additionally, they can reduce overfitting problems. The DenseNets have shown good performance in image classification tasks [33]. [PERSON] et al. proposed a densely connected graph attention network (IPR-DCGAT) based on iterative path inference, which uses a densely connected graph attention network to update the representation of nodes and a two-step iterative algorithm to update the representation of edges. Using the densely connected graph attention network to model local and global information between documents, the model achieves an F1 score of 84% on CDR, which is approximately 16.3%-22.5% higher than other models with insignificant margins.