# Chinese Named Entity Recognition Based on Multi-Level Representation Learning
## 3 WP-XAG Model
### Feature Fusion Layer
#### 3.2.1 Xlstm  
LSTMs often struggle to effectively manage memory when processing long dialogues or complex contexts due to their limited storage capacity, which compresses information into a scalar cell state. This limitation hinders their ability to handle rare tokens or long-range dependencies, resulting in poor performance on long texts or complex tasks.  
While WoBERT provides accurate word-level representations, XLSTM [26] features an enhanced memory structure that better accommodates longer contexts and complex dependencies. XLSTM integrates scalar storage and updates from sLSTM with matrix storage and covariance updates from mLSTM, all enhanced by an exponential gating mechanism for improved performance. In XLSTM, mLSTM and sLSTM are combined into \"residual blocks\", which are stacked to create the overall architecture. This gating mechanism allows more flexible control of information flow, enabling better adjustments to new contexts.  
Furthermore, XLSTM's matrix memory structure increases storage capacity, enhancing its efficiency in managing rare tokens and long-range dependencies. The residual network blocks help mitigate the vanishing gradient problem through skip connections, allowing XLSTM to effectively process complex sequence data. Due to hardware constraints and limited parallelization of sLSTM, the model primarily utilizes XLSTM [1:0] (indicating a 1:0 ratio of mLSTM-based to sLSTM-based blocks) to capture semantic representations from WoBERT. The block structure of mLSTM is illustrated in Figure 2, with forward propagation represented by Equations (4)-(14).  
\[\sigma(x)=\frac{1}{1+e^{-x}} \tag{4}\]  
\[\tanh(x)=\frac{\sinh(x)}{\cosh(x)}=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}} \tag{5}\]  
\[C_{t}=f_{t}\odot C_{t-1}+i_{t}\odot v_{t}k_{t}^{T} \tag{6}\]\[n_{t}=f_{t}\odot n_{t-1}+i_{t}\odot k_{t} \tag{7}\]  
\[h_{t}=o_{t}\odot C_{t}q_{t}/\text{max}\Big{\{}n_{t}^{T}q_{t},1\Big{\}} \tag{8}\]  
\[q_{t}=W_{q}\cdot x_{t}+b_{q} \tag{9}\]  
\[k_{t}=\frac{1}{\sqrt{d}}W_{k}\cdot x_{t}+b_{k} \tag{10}\]  
\[v_{t}=W_{v}\cdot x_{t}+b_{v} \tag{11}\]  
\[i_{t}=e^{W_{t}^{T}\cdot x_{t}+b_{t}} \tag{12}\]  
\[f_{t}=\sigma(W_{f}^{T}\cdot x_{t}+b_{f})\text{ OR }e^{W_{f}^{T}\cdot x_{t}+b_{f}} \tag{13}\]  
\[o_{t}=\sigma(W_{o}\cdot x_{t}+b_{o}) \tag{14}\]  
where \(\sigma\) denotes the sigmoid activation function, and tanh represents the hyperbolic tangent activation function. \(C_{t}\) and \(C_{t-1}\) represent the cell states at the current and previous time steps, respectively. \(n_{t}\) and \(n_{t-1}\) represent the normalized states at the current and previous time steps, respectively. \(h_{t}\) is the hidden state, \(x_{t}\) represents the input vector, and \(q_{t}\), \(k_{t}\), and \(v_{t}\) are the query, key, and value inputs, respectively. \(W_{q}\), \(W_{k}\), \(W_{v}\), and \(W_{i}\) denote matrix parameters, while \(b_{q}\), \(b_{k}\), \(b_{v}\), and \(b_{i}\) represent bias parameters. \(i_{t}\), \(f_{t}\), and \(o_{t}\) denote the input gate, forget gate, and output gate, respectively.  
Figure 2: mLSTM block structure diagram.