# Construction of Cultural Heritage Knowledge Graph Based on Graph Attention Neural Network
## 4 Data Model Construction
### Entity-Relationship Extraction Model Construction
#### 4.1.2 Entity Recognition Layer  
In the entity recognition layer, the text is divided into segments of different lengths. A classifier is used to classify these segments to determine whether each segment belongs to a predefined entity category or is a nonentity. The entity categories are defined before model training. Let the entity categories be \(O=\{o_{1},o_{2},\cdots,o_{n}\}\), where \(n\) is the number of categories. The input text \(U\) is segmented to obtain segments \(P=\{u_{i},u_{i+1},\cdots,u_{i+k}\}\), where the length of each segment is \(k+1\). To reduce misclassification during entity recognition and accurately identify entities and nonentities, a nonentity set \(\{none\}\) is defined. Each segment \(p\) is mapped to \(O\cup\{none\}\) and the input vector for the segment classifier is constructed by concatenating the segment's semantic feature vector, length feature vector, and global feature vector. (1) Segment semantic feature vector: This is obtained by performing global average pooling on the word embedding vectors of each word within the segment, representing the semantic content of the entire segment. The word vector layer corresponding to segment \(a\) is \(e\), calculated as shown in Equation (4) [29]  
\[span=Avgpooling(e_{i},e_{i+1},\cdots,e_{i+k}) \tag{4}\]  
(2) Segment semantic feature vector: A predefined matrix \(W\) contains feature vectors \(w_{k+1}\) of different lengths within the text, encompassing the vectors corresponding to each segment. The embedding matrix can be learned through backpropagation and its contents are updatable, thereby enhancing the model's capability to process text. (3) Segment global feature vector: The global feature vector of a text segment is extracted through a pooling attention mechanism that captures the contextual information of the text segment. The attention mechanism calculates the relevance weights between the text segment and other parts of the text. Then, average pooling is applied to aggregate the weighted features, resulting in a compact representation of the global context, as shown in Figure 3.  
The calculation process of the pooling attention mechanism is as follows:  
Step 1: Construct the query vector, key vector, and value vector. Convert the segment semantic feature vector \(e(b)\) into a query vector \(Q_{b}\) and convert the sentence semantic feature vector \(E\) into a key vector \(K_{b}\) and a value vector \(V_{b}\). These converted vectors are used to calculate the attention weights, thereby extracting the most relevant contextual information for the specific text segment. The calculation formulas are shown in Equations (5)-(7) [30]  
\[Q_{p}=e(p)W_{p}^{Q} \tag{5}\]  
\[K_{p}=EW_{p}^{K} \tag{6}\]  
\[V_{p}=EW_{p}^{V} \tag{7}\]  
In this context, the dimensions of the vectors are as follows: the query vector \(Q_{p}\) has a dimension of \((k+1)\times d_{q}\), the key vector \(K_{b}\) has a dimension of \(n\times d_{k}\), and the value vector \(V_{b}\) has a dimension of \(n\times d_{p}\). Step 2: In the pooling attention mechanism, calculate the attention coefficient \(att\) based on the between the query vector \(Q_{p}\) and the key vector \(K_{p}\). This coefficient is used to assess the relevance of each word in the sentence to the specific text segment, such as entity pairs. The dimension is \(n\) and \(T\) denotes the transpose operation, as shown in Equation (8) [31; 32]  
\[att=\text{softmax}\left(\text{maxpooling}\left(\frac{Q_{p}K_{p}^{T}}{\sqrt{d_{ k}}}\right)\right) \tag{8}\]  
Step 3: Use the attention coefficient \(att\) to generate the final weighted representation. Multiply the attention coefficient \(att\) by the value vector \(V_{p}\) to obtain the segment global vector \(c_{p}\), as shown in Equation (9) [31; 32]  
\[c_{p}=att\cdot V_{p} \tag{9}\]  
Concatenate the three types of feature vectors to obtain the comprehensive input text feature vector \(x_{p}\), as shown in Equation (10) [31; 32].