# Multigranularity Syntax Guidance with Graph Structure for Machine Reading Comprehension
## 3 Methodology
### Graph with Dependence Module
#### 3.2.2 Graph Attention Network  
We apply graph attention networks to model the information flow between nodes, which can further improve the representations of nodes through the attention mechanism over features from its neighbors. It can be seen from Figure 4 that the traditional one-dimensional convolution usually processes a few words near the central word, whereas the graph convolution considers these dependencies as edges. The central word and all other nodes connected by the edges are processed during the convolution. For example, the size of the convolution kernel depicted in Figure 4a is 3, and the process of convolution involves _related_, _fields_, and _are_. However, the graph convolution depicted in Figure 4b involves only _related_ and _fields_. In particular, the graph of GD module can be represented as \(G(N,S)\), where \(N\) is the set of nodes of the graph, \(k\) is the number of nodes, \(S\) is the set of edges of the graph, and an edge exists between two nodes if they are related. Figure 4 indicates that an undirected  
Figure 3: This is the overall structure of the feature encoder part. On the left side of the figure, PrLM applies special delimiters ((CLS) and (SEP)) to text sequences. On the right side of the figure, the GD module extracts the dependencies and builds the graph network. graph can be constructed for all the tokens using the tokens as the graph's nodes and the edges of the graph as the association relations between the tokens. In this study, We know that the graph node needs to use the word vector as a substitute for the word. GloVe [8] is used as the embedding representation for each token as a feature of the nodes; sentence-based dependencies are used as association relations. If the current token is not in GloVe's vocabulary, we will skip the current token and not act as a node. First, in the graph with dependency module, our input will generate multiple sets of dependencies after dependency analysis, and each set has multiple words with hierarchical relationships. Furthermore, the input is passed into the dependency parser, and if there are dependencies among the tokens, an edge weight initialized to 1 is added to both the nodes and 1 is added to the edge weight if the association is repeated subsequently. We also set a threshold to limit our filter on the level of dependency syntactic relations. If the number of dependencies set does not exceed the threshold, the information will not be incorporated into the graph structure. The edges of the graph are created as follows:  
\[B_{dp}=\sum_{i=0}^{z}B_{dp}^{(i)} \tag{3}\]  
where \(B_{dp}\) represent the edges of the dependence relation, \(z\) is the number of layers of dependencies. For example, extending the child node only depends on the parent node to the child node also depends on the parent node of the parent node, that is, two layers of dependencies. Instead of other graph-based models, graph attention networks are used to maintain consistency with the multihead attention module of the BERT model. In the following, we will describe a single layer of the proposed graph attention network. For consistency with the multiheaded attention module in the BERT model, the graph attention network used in the text uses a multiheaded attention mechanism. We describe one of the \(m\) attention heads here and \(h_{i}\) is the representation of each node. All the parameters for each attention head and layer are unique. If an edge exists from node \(j\) to node \(i\), the attention factor \(e_{ij}\) is calculated as follows:  
\[e_{ij}=\frac{\left(\mathbf{h}_{i}\mathbf{W}^{\text{Q}}\right)\left(\mathbf{h}_{j} \mathbf{W}^{\text{K}}\right)^{\text{T}}}{\sqrt{d_{z}}} \tag{4}\]  
In the preceding equations, \(\mathbf{W}^{\text{Q}}\), \(\mathbf{W}^{\text{K}}\), and \(\mathbf{W}^{\text{V}}\) are parameter matrices. We use the softmax function across all the neighbor nodes \(j\in\mathcal{N}_{i}\) to normalize the  
Figure 4: Difference between traditional convolution and graph convolution. (**a**) is the traditional convolution.