# Distantly Supervised Named Entity Recognition with Self-Adaptive Label Correction
## 4 Experiment
### Analysis  
Finally, the reason that SALO achieves performance improvement was analyzed. To this end, experiments were conducted on the CoNLL03 dataset and revealed that NER performance is highly correlated with the noise rate of training labels. As shown in Table 7, when using the baseline BiLSTM-CRF model on the clean (noise rate 0%) and noisy (noise rate 67.21%) labels, we obtain \(F_{1}\) scores of 91.21% and 59.50%, respectively. Using implicit denoising models to repredict labels slightly alleviates label noise, with a noise rate of 63.33%. With the identical BiLSTM-CRT, the \(F_{1}\) score improves to 66.67%. The proposed SALO obtains state-of-the-art performance, with 80.08% in terms of \(F_{1}\) score. After the label correction, the noise rate is further reduced to 48.43%. This indicates that distinguishing from correct and wrong labels is crucial for noisy label learning and the built model manager to alleviate the fluctuations of noisy labels.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & \multicolumn{3}{c}{**RoBERTa (Baseline)**} & \multicolumn{3}{c}{**RoBERTa (SALO)**} \\ \hline
**Dataset** & \(F_{1}\)** **Score** & **Precision** & **Recall** & \(F_{1}\)** **Score** & **Precision** & **Recall** \\ \hline CoNLL03 & 75.93 & 82.29 & 70.47 & 84.90 & 86.20 & 83.64 \\ CoNLL03-e & 76.39 & 86.82 & 68.20 & 84.51 & 87.81 & 81.45 \\ \hline Twitter & 46.45 & 50.97 & 42.66 & 52.50 & 68.48 & 42.57 \\ Twitter-e & 48.56 & 53.51 & 44.45 & 53.79 & 46.14 & 64.47 \\ \hline Webpage & 60.98 & 59.24 & 62.84 & 69.66 & 78.15 & 62.84 \\ Webpage-e & 61.33 & 58.31 & 64.69 & 70.01 & 71.17 & 69.05 \\ \hline Wikigold & 52.57 & 47.67 & 58.59 & 65.72 & 63.31 & 68.33 \\ Wikigold-e & 53.79 & 46.14 & 64.47 & 66.98 & 65.11 & 68.97 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Comparison of RoBERTa baseline and SOLO on CoNLL03, Tweet, Webpage and Wikigold datasets, in terms of \(F_{1}\) score, Precision and Recall (in %). Results using original noisy labels and extended labels (denoted with -e) are both presented.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & \multicolumn{3}{c}{**BiLSTM-CRF (Baseline)**} & \multicolumn{3}{c}{**BiLSTM-CRF (SALO)**} \\ \hline
**Dataset** & \(F_{1}\)** **Score** & **Precision** & **Recall** & \(F_{1}\)** **Score** & **Precision** & **Recall** \\ \hline CoNLL03 & 59.50 & 75.50 & 49.10 & 80.08 & 85.59 & 75.24 \\ CoNLL03-e & 61.22 & 85.67 & 47.63 & 81.95 & 88.83 & 77.41 \\ \hline Twitter & 21.77 & 46.91 & 14.18 & 44.96 & 54.02 & 38.50 \\ Twitter-e & 22.90 & 55.16 & 14.45 & 45.11 & 60.92 & 35.81 \\ \hline Webpage & 43.34 & 58.05 & 34.59 & 54.90 & 74.07 & 40.54 \\ Webpage-e & 45.97 & 57.00 & 38.51 & 56.07 & 73.63 & 45.27 \\ \hline Wikigold & 42.92 & 47.55 & 39.11 & 55.71 & 53.31 & 58.33 \\ Wikigold-e & 44.54 & 51.84 & 39.04 & 56.13 & 67.96 & 47.97 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Comparison of BiLSTM-CRF baseline and SOLO on CoNLL03, Tweet, Webpage and Wikigold datasets, in terms of \(F_{1}\) score, Precision and Recall (in %). Results using original noisy labels and extended labels (denoted with -e) are both presented.