# Iterative Named Entity Recognition with Conditional Random Fields
## 4 Discussion
### Influence of Self-Training on Model Performance  
The lack of improvement in model performance during the experiment with only self-training as well as by the comparison of model performance with and without ST indicates that self-training, as performed in our experimental setting, had no positive effect on model performance. This result contrasts with the studies of [PERSON] et al. [17] and [PERSON] et al. [30], which indicated that self-training and automatic data annotation can work in principle. We attribute our divergent results to the large number of highly diverse ENAMEX entities in our dataset, which are more difficult for NER models to recognise. Furthermore, the sentences analysed were collected from different sources, including texts from newspapers and from technical reports. The resulting dataset is likely to have a larger diversity in linguistic style analysed relative to previous works--for example, from analysing Twitter posts--and this may also affect model performance [7]. However, more investigations are required to verify these hypotheses, and the effects of NLP models to recognise entities of different classes deserves more research. A possible research path involves the combination of CRFs with BiLSTMs, which can achieve state-of-the-art performance for specific language resources [10] and be potentially useful in the \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Experiment** & **\#Tokens** & \(\mathbf{t_{am}}\) **(h)** & **Mean Number of Iterations** & \(\mathbf{t_{calc}}\) **(h)** & \(\mathbf{t_{train}}\) **(h)** \\ \hline
20-QQ; no ST & 54,157 & 9.2 & 38.1 & 2.2 & 11.4 \\
20-QQ, 20-0.2Q, no ST & 66,945 & 11.4 & 29.1 & 1.9 & 13.3 \\
20-QQ, 20-0.2Q; ST & 75,231 & 12.8 & 18.0 & 1.4 & 14.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of the overall time (\(t_{train}\)) required to train the model to an F1-score of 0.885, for three different training experiments. This overall time was estimated as the sum of the annotation time (\(t_{am}\)) plus the calculation time (\(t_{calc}\)). identification of entities of the ENAMEX group, e.g., [7; 31]. In addition, methodological approaches involving latent factor analysis may also contribute to improve computational efficiency and prediction accuracy of trained models for unseen data [32; 33; 34; 35; 36]. In contrast to the lack of model improvement via (only) self-training, the marginal increase in model performance in the experiment without self-training and with a small pool size indicates that the model may learn better with the same number of annotated tokens. This is potentially associated with the higher number of iterations in these experiments, enabling more 'unsure' data points to be selected per iteration, driving better model performance. Optimising Selection of Manually Annotated Sentences to Enhance Model Performance and Procedural Efficiency  
The better performance of the models fitted with manually annotated data selected via active learning underscores our above results, indicating that annotation of 'unsure' sentences provides the most valuable information to the model. Although there was no statistical difference between the models based on manual annotation of sentences from the 0.2Q upwards or random sampling, the higher variance in model performance of the latter indicates that models based on random sampling of sentences are less reliable in their results than models created using uncertainty-based sampling in an active learning procedure. Compared with annotation of random sentences or sentences with a greater certainty estimated by the model, our results confirm that iterative manual annotation of highly uncertain sentences enhances final model performance and reliability as found in other studies [14]. The better performance of the model built with the manual annotation of a smaller number of highly uncertain data points per iteration can be attributed to an increase in the number of iterations performed when training the model.