# BERT-Based Transfer-Learning Approach for Nested Named-Entity Recognition Using Joint Labeling
## 8 Conclusions  
In this research work, the transfer-learning approach was used to solve the nested named-entity-recognition problem. The presented transfer-learning approach fine-tuned the pretrained BERT language models for the NER task. The experiments were conducted with different variants of the pretrained BERT-based language models belonging to three popular categories based on the domain. The performance comparison has been done with the existing approaches for each of the datasets. In addition, the experiments were conducted using the conditional random field (CRF) and the Bi-LSTM-CRF models for performance comparison. Manual feature extraction and word embeddings were required for the CRF and Bi-LSTM-CRF models. However, there were no such requirements for the presented transfer-learning approach. The performance was evaluated using two biomedical datasets and a German language NER dataset, out of which one biomedical dataset (i.e., the GENIA dataset) and the German language dataset (i.e., GermEval 2014 dataset) contained nested annotations. The different levels of annotation were joined together for the nested datasets so that the nested named-entity-recognition problem could be treated as a flat named-entity-recognition problem. It was found that the performance of the presented transfer-learning approach was much better than that of the other two models and many of the existing approaches. The presented transfer-learning approach achieved better results than many of the existing research works for the nested and non-nested NER datasets. This research work presented a performance comparison between the pretrained BERT models based on domain, size of models, and cased and uncased versions. It was found that the performance of the presented BERT-based language model depended on the domain and the language of the downstream task. In addition, the presented transfer-learning-based approach had more generalization capability and was much simpler than any of the existing approaches. The presented transfer-learning approach can be used for any of the similar downstream natural language processing tasks. In the future, we will conduct a similar study of several different natural language processing tasks other than named-entity recognition to further test the performance and generalization capabilities of the presented transfer-learning approach.  
Conceptualization, A.A., S.T. and M.V.; methodology, A.A.; software, A.A.; validation, A.A.; investigation, A.A.; resources, A.A., S.T. and M.V.; data curation, A.A.; writing--original draft preparation, A.A.; writing--review and editing, S.T., M.V., V.S., G.C. and N.D.; visualization, A.A.; supervision, S.T. and M.V.; project administration, V.S., G.C. and N.D.; funding acquisition, G.C. and N.D. All authors have read and agreed to the published version of the manuscript.  
This work was partially supported by the Danish Industry Foundation through \"CIDI-Cybersecure IoT in Danish Industry\" under Project 2018-0197.  
Not applicable.  
The data that are used in this research work are openly available at the following links: 1. GermEval 2014 dataset: [[https://sites.google.com/site/germeval2014](https://sites.google.com/site/germeval2014) ner/data]([https://sites.google.com/site/germeval2014](https://sites.google.com/site/germeval2014) ner/data) (accessed on: 4 March 2021); 2. GENIA dataset: [[http://www.geniaproject.org/genia-corpus](http://www.geniaproject.org/genia-corpus)]([http://www.geniaproject.org/genia-corpus](http://www.geniaproject.org/genia-corpus)) (accessed on: 5 March 2021); 3. JNLPBA dataset: [[http://www.geniaproject.org/shared-tasks/bionlp-jnlpba-shared-task-2004](http://www.geniaproject.org/shared-tasks/bionlp-jnlpba-shared-task-2004)]([http://www.geniaproject.org/shared-tasks/bionlp-jnlpba-shared-task-2004](http://www.geniaproject.org/shared-tasks/bionlp-jnlpba-shared-task-2004)) (accessed on: 5 March 2021). All the above datasets can also be found at the GitHub repositories of [13; 57].  
The authors declare no conflict of interest.