# SLNER: Chinese Few-Shot Named Entity Recognition with Enhanced Span and Label Semantics
## 3 Method
### Specific Structure  
#### 3.3.1 Enhanced Span Representation  
In previous span-based NER models [35], it was common practice to concatenate the embedding information of the start-position token and the end-position token of the entity (referred to as the \"outer span\") to represent the span of that entity, which is then used for the final classification decision:  
\[h_{span}=[x_{start};x_{end}] \tag{8}\]  
This approach lacks interaction between the start and end tokens and fails to fully utilize the informative content within the span. Moreover, this span representation is coarse-grained. To address these limitations, [PERSON] et al. [26] proposed a biaffine decoder that utilizes two fully connected layers to enable interaction between the start and end tokens while simultaneously predicting the span type. However, in this biaffine method, the information within the span is still ignored. To fully utilize the informative content within the span, we employ enhanced span to generate the final span representation (as shown in Figure 6). Specifically, we pass the token-embedding information through the outer and inner span modules. The outer span module, similar to the biaffine decoder method, utilizes the biaffine attention mechanism to obtain the outer span representation:  
\[h_{outer\ span}=h_{start}^{T}Uh_{end}+W(h_{start}\oplus h_{end})+b \tag{9}\]  
where \(h_{start}\) and \(h_{end}\) represent the start and end token embedding of spans in a text, respectively; \(U\) and \(W\) are learnable parameters; and \(b\) is the bias. The dimension of \(h_{outer\ span}\) is expanded to \(\mathbb{R}^{N_{b}\times x_{s}}\). We designed the inner span module to capture the token-level information within the span. For this purpose, we use linear attention to generate information interaction for each token. Specifically, this module uses span and the start- and end-position indices of the span as input. It starts by applying a feed-forward neural network (FFNN) to  
Figure 6: Different ways of span representation. (**a**) Simple start- and end-token concatenation as a span representation. (**b**) Span representation using the biaffine decoder method, which allows for information interaction between the start and end tokens (indicated by blue arrows). (**c**) Span representation method used in our model, which incorporates the interaction between the tokens within the span (indicated by yellow arrows). non-linearly transform the input representations, obtaining context-aware representations. Then, it computes normalized scores for each position. Finally, these representations are fed into a self-attention layer, which combines the representations of each position with those of other positions, weighted by the attention scores. This allows the model to capture potential relationships between the tokens within the span. The result is the inner span representation:  
\[a_{i}=W_{(f)}\times x_{i}+b_{(f)} \tag{10}\]  
\[s_{i}=\frac{exp(a_{i})}{\sum_{k=n}^{m}exp(a_{k})} \tag{11}\]  
\[h_{inner\ span}=\underset{i=n}{\overset{m}{\sum}}s_{i}\times x_{i} \tag{12}\]  
where \(x_{i}\) represents the hidden representation from the bidirectional LSTM, and \(W_{(f)}\) and \(b_{(f)}\) are the learnable weights and biases of the feed-forward neural network, respectively. The indices \(i\in\{n,n+1,\cdots,m\}\) correspond to the token indices within the span, where \(n\) and \(m\) represent the start and end indices of the span, respectively. When \(n=m\) (indicating a span of length 1), we do not extract additional features and simply use the hidden representation (\(x_{i}\)). To predict the entity type, we integrate the outer span representation and inner span representation in a gate network to obtain the weight coefficient (\(\gamma\)) (as shown in Figure 7):  
\[\gamma=\sigma\Big{(}U_{(g)}[h_{outer\ span};h_{inner\ span}]+b_{(g)}\Big{)} \tag{13}\]  
where \(U_{(g)}\) and \(b_{(g)}\) are trainable parameters of the gate network, and \(\sigma\) represents the sigmoid function.