# SLNER: Chinese Few-Shot Named Entity Recognition with Enhanced Span and Label Semantics
## 3 Method
### Overall Structure  
The overall architecture of the SLNER model is illustrated in Figure 5. For span representation, given a sentence (\(T=\{t_{1},\cdots,t_{l}\}\)) of length \(l\), we use BERT as our encoder, which encodes the context of the \(i\)-th token in the sentence as follows:  
\[h_{i}=BERT(t_{i})\ \ \ \ \ \forall t_{i},t_{i}\in T \tag{1}\]  
where \(d_{h}\) is the hidden dimension of the encoder, and the output dimension after passing the original sentence through the encoder is \(\mathbb{R}^{l\times d_{h}}\).  
To further enhance the modelling of the sequential order of the text, the embedding representation obtained from BERT is then passed through a bidirectional LSTM layer. The forward LSTM network captures the hidden forward states (historical features), while the backward LSTM network captures the hidden backward states (future features), resulting in a context-aware encoding representation:  
\[x_{i}=\bigg{[}\ \overrightarrow{LSTM}\left(h_{i},x_{i-1}\right);\ \ \ \ \ \overleftarrow{LSTM}\left(h_{i},x_{i+1}\right)\bigg{]} \tag{2}\]  
At this stage, the output dimension of the original sentence through the bidirectional LSTM layer is \(\mathbb{R}^{l\times d_{x}}\).  
We predefine an n-gram value (\(w\)), which represents the maximum length of spans that can be formed in a text. The number of possible spans that can be formed in a sentence of length \(l\) is given by:  
\[N_{s}=\left\{\begin{array}{ll} w[l-(w-1)]+\sum_{i=1}^{w-1}i&l\geq w\\ \sum_{i=1}^{w-1}i&l<w\end{array}\right. \tag{3}\]  
Next, the token-embedding sequence obtained from the LSTM layer is used to construct the span-level feature vector representation (\(h_{span}\)) through a span extractor (details in Section 3.3.1). The dimension of \(h_{span}\) is finally expanded to \(\mathbb{R}^{N_{s}\times d_{h}}\) through a feed-forward layer.  
For label representation (details in Section 3.3.2), we manually define the appropriate full label name for each label. Similarly, we use BERT as the encoder and directly encode the label using Equation \((1)\) to obtain the global semantic feature (\(h_{[CLS]}\)). The difference from span encoding is that we further pass \(h_{[CLS]}\) through a pooler layer to obtain the semantic feature vector representation (\(h_{label}\)), which serves as the final representation of the label:  
\[h_{label}=Tanh\Big{(}W_{(p)}\times h_{[CLS]}+b_{(p)}\Big{)} \tag{4}\]  
where \(W_{(p)}\) represents the weight parameters in the pooler layer, \(b_{(p)}\) represents the bias parameters, and \(Tanh\) is the activation function. The dimension of \(h_{label}\) needs to be expanded to \(\mathbb{R}^{d_{h}\times N_{c}}\), where \(N_{c}\) is the number of entity categories.  
According to our approach, there is a correlation between labels and spans appearing in the text. Therefore, we capture this correlation through the dot product:  
\[\mathrm{H}=h_{LST}\times h_{span} \tag{5}\]  
where the similarity matrix (H) has dimensions of \(\mathbb{R}^{N_{c}\times N_{c}}\). We use a standard linear classifier with a softmax function to predict the entity type for each span, resulting in the final predicted output (\(\hat{g}_{span}\)):  
Figure 5: The overall structure of SLNER. The grey module on the left learns span representations, while the grey module on the right learns label representations. The model’s final predictions are calculated through distance matching. \(N_{s}\) represents the number of spans, and \(N_{c}\) represents the number of entity categories.  
\[\hat{y}_{span}=Softmax\Big{(}W_{(c)}\times h_{span}+b_{(c)}\Big{)} \tag{6}\]  
where \(W_{(c)}\) is the trainable parameter of the classifier, and \(b_{(c)}\) is the bias. Finally, we use the cross-entropy loss function to compute the loss, which measures the difference between the predicted results and the ground truth labels:  
\[\mathcal{L}_{PSNNER}=-\sum\mathcal{Y}_{span}\log(\hat{y}_{span}) \tag{7}\]