# Chinese Named Entity Recognition Based on Multi-Level Representation Learning
## 3 WP-XAG Model
### Decoding Layer  
GlobalPointer [23] is a span-based decoding method. Compared to sequence labeling methods such as CRF, GlobalPointer treats the start and end of an entity as a unified whole, allowing it to more accurately identify the complete span of an entity.  
For an input sequence of length u, the encoded vector sequence is \(\mathbf{H}=[\mathbf{h}_{1},\,\mathbf{h}_{2},\,\ldots,\,\mathbf{h}_{u}]\). Let i and j denote the start and end of an entity in the sequence, respectively. The coordinates (i, j) are marked as 1, while all other positions are marked as 0, indicating the position information of entities and non-entities. This process is illustrated in Figure 4 and can be represented by Equations (21)-(23).  
\[\mathbf{q}_{i,\alpha}=\mathbf{w}_{\alpha\alpha}\mathbf{h}_{i}+b_{\theta,\alpha} \tag{21}\]  
\[\mathbf{k}_{j,\alpha}=\mathbf{w}_{k\alpha}\mathbf{h}_{j}+b_{k,\alpha} \tag{22}\]  
\[s_{\alpha}(i,j)=\mathbf{q}_{i,\alpha}^{T},\mathbf{k}_{j,\alpha} \tag{23}\]  
where \(\mathbf{q}_{i,\alpha}\) and \(\mathbf{k}_{j,\alpha}\) represent the vector sequences, while \(\mathbf{h}_{i}\) and \(\mathbf{h}_{j}\) denote the character vectors. \(\mathbf{w}_{\alpha,\alpha},\mathbf{w}_{k,\alpha}\), \(b_{\theta,\alpha}\), and \(b_{k,\alpha}\) are the matrix parameters and bias parameters. The function \(s_{\alpha}\) scores the segment [i, j] as entity type \(\alpha\) by taking the inner product of \(\mathbf{q}_{i,\alpha}\) and \(\mathbf{k}_{j,\alpha}\).  
In the absence of relative position information for entities, GlobalPointer often struggles with accurately recognizing entities due to its insensitivity to entity length and span.  
Figure 4: GlobalPointer for entity tagging.  
Figure 3: Adaptive multi-head attention structure diagram.  
However, Rotary Position Embedding (RoPE) [29] enhances GlobalPointer's ability to distinguish and capture entity information, thereby improving recognition performance. This is illustrated in Equation (24), where \(\mathbf{R}_{i}\) and \(\mathbf{R}_{j}\) represent orthogonal matrices, and \(\mathbf{R}_{i-j}\) denotes the relative position vector.  
\[s^{\prime}_{\alpha}(i,j)=\left(\mathbf{R}_{i}\mathbf{q}_{i,\alpha}\right)^{T}\left(\mathbf{ R}_{j}\mathbf{k}_{j,\alpha}\right)=\mathbf{q}_{i,\alpha}^{T}\mathbf{R}_{i}^{T}\mathbf{R}_{j}\mathbf{k}_{j, \alpha}=\mathbf{q}_{i,\alpha}^{T}\mathbf{R}_{j}-i\mathbf{k}_{j,\alpha} \tag{24}\]  
The loss function designed for GlobalPointer is suitable for multi-label classification tasks such as NER. It is an extension of the single-target, multi-label classification cross-entropy, as shown in Equation (25).  
\[Loss=\ln(1+\sum_{(i,j)\in P_{\alpha}}e^{-s_{\alpha}(i,j)})+\ln(1+\sum_{(i,j) \in Q_{\alpha}}e^{s_{\alpha}(i,j)}) \tag{25}\]  
where \(P_{\alpha}\) denotes the set of all start and end positions of entities with type \(\alpha\) in the sample, while \(Q_{\alpha}\) represents the set of start and end positions for all non-entity types or entities that are not of type \(\alpha\) in the sample.  
The complementary strengths of the various modules in WP-XAG significantly enhance the model's overall performance. The inclusion of PGD effectively improves the model's robustness. WoBERT alleviates issues with polysemy and ambiguity through word-level pre-training, while XLSTM processes these disambiguated semantic representations, ensuring the capture of long-range dependencies. ADMHA further optimizes information capture through adaptive adjustments, making the handling of polysemous words more precise and enhancing the model's ability to capture long-range dependencies. Finally, GlobalPointer enables the model to identify and classify entities more accurately, further boosting the model's performance.