# Label-Guided Data Augmentation for Chinese Named Entity Recognition
## 4 Approach
### Loss Function Optimization  
In low-resource scenarios, the issue of imbalanced sample categories is prominent, with the \"O type\" entities dominating the sample sequence labels. Predicting the labels for these entity characters is relatively simple compared with other entity types with richer semantics. However, traditional cross-entropy loss functions do not distinguish between the difficulty levels of category discrimination, which can cause the model to focus on the easily distinguishable \"O type\" throughout the training process, failing to fully learn other entity types in the samples.  
Therefore, we introduce a modulation factor \(\gamma\) to enable the model to focus more on the difficult-to-distinguish entity types, achieving the effect of adaptive weight tuning. At the same time, we also adopt an uncertainty weighting strategy, adding a regularization term to further balance the loss weights of easy-to-distinguish and difficult-to-distinguish categories. The training loss process of the model is shown in Formulas (8) and (9):  
\[\hat{p}=\text{soft}\max(WC+b), \tag{8}\]  
\[\mathcal{L}=\sum_{j}^{t}-\alpha_{j}(1-\hat{p}^{j})^{\gamma}\log(\hat{p}^{j}) \hat{\
u}(j==y). \tag{9}\]  
Firstly, Formula (8) applies a linear layer to the sample representation \(C\), where \(W\) and \(b\) are learnable parameters. Formula (9) represents the final loss function, where \(\alpha_{j}\) is a hyperparameter representing the loss weight for different label categories, \(\hat{\
u}(\cdot)\) is the indicator function, \(j\) denotes the \(j\)-th category, and \(y\) represents the label category. When \(j=y\), the value of \(\hat{\
u}(j==y)\) is 1; otherwise, it is 0. \(\gamma\) is a hyperparameter with a value range of \([0,5]\). When its value is zero, the above formula becomes the common cross-entropy loss function (we found that \(\gamma=2\) achieves the best result in our experiments; Section 6.5). \((1-\hat{p}^{j})^{\gamma}\) can reduce the loss contribution of easy-to-distinguish categories. Specifically, if \(\hat{p}^{j}\) approaches 1, meaning that \(j\) is an easy-to-distinguish category, then \((1-\hat{p}^{j})^{\gamma}\) approaches 0, and the contribution of this category to the loss is small; if \((1-\hat{p}^{j})^{\gamma}\) approaches 1, it indicates a difficult-to-distinguish category, and its loss weight increases. By introducing this modulation mechanism, the performance of the model can be significantly enhanced, and we will present detailed experimental results in Section 6.3.  
For the entire training process of the LGDA model, we provide an explanation in the pseudocode shown in Algorithm 1. In Algorithm 1, given the training dataset \(\mathbb{D}_{train}\), model \(f\), and encoder network \(PLM\), the goal is to find the model parameters \(\theta\) that optimize performance. Initially, in the data augmentation module, a mini-batch of original samples \((X,Y)\) is extracted from the training data and linearized to obtain the sample \(X^{\prime}\); then, entities in \(X^{\prime}\) are randomly masked to obtain \(X_{mask}\), and the encoder network is fine-tuned. Subsequently, the masked entity generation language model is used to predict the masked characters, thereby generating augmented samples \(X_{aug}\). In the label semantic fusion module, the labels \(Y\) are first converted into label annotations \(L\); then, twin encoders are used to obtain the hidden layer features of \(L\) and the sample \(X\), and finally, the cross-attention mechanism is utilized to integrate label semantic knowledge into the sample representation.