# Ontology Attention Layer for Medical Named Entity Recognition
## 6 Conclusions  
In this study, we proposed a novel attention layer called the \"ontology attention layer\" to improve NER performance for medical texts by incorporating ontology. It enhances the NER performance of a BERT-based language model for clinical text by leveraging an ontology to improve the recognition of domain-specific entities. In the experiments, the proposed method enhanced three different BERT-based language models' ability to accurately identify and classify entities, resulting in an improved F1 score for the medical NER task when utilizing a tailored ontology. We further studied the effects to the performance by the design details of the ontology, which were also studied in several experiments. The results suggest that to maximize the performance of the proposed model, the granularity of the ontology should be tailored, and the name of the properties of different classes should be as distinct as possible. Our research findings highlight that integrating an ontology can significantly improve the NER performance of a language model when implemented through a well-designed attention layer. This suggests that introducing the ontology reasoning model in entity recognition represents a practical approach to enhance the performance of BERT-based NER models. In the practical construction of a knowledge base, an ontology is typically crafted prior to processing unstructured corpora. In such scenarios, our proposed method would be valuable during the NER phase of corpus processing, requiring little additional manual effort. Given that the design of the ontology significantly influences the effectiveness of the proposed method, our primary focus in future work is to explore an automated method for tailoring an ontology to suit the proposed approach. An ontology contains connections between entity names and their alternative representations may also be helpful but it is out of the scope of this work. Additionally, we aim to investigate similar approaches on a larger and more complex ontology. The combination of text-based embedding and graph-based embedding would prove beneficial in such scenarios, with the challenge lying in developing an effective method to integrate them seamlessly. Furthermore, the proposed \begin{table}
\begin{tabular}{c c c} \hline \hline
**Settings** & **Medium-Grained** & **Fine-Grained** \\ \hline w/ BERT and class name & **56.41**\% & 56.21\% \\ w/ BERT, w/o class name & 56.14\% & **56.57**\% \\ \hline w/ BERT-WWM and class name & **58.91**\% & 58.60\% \\ w/ BERT-WWM, w/o class name & 58.53\% & **58.92**\% \\ \hline w/ RoBERTa and class name & 59.23\% & 59.00\% \\ w/ RoBERTa, w/o class name & **59.38**\% & **59.19**\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Effects of the usage of the ontology class name on the proposed model under different base language models and different granularity settings. Micro F1 scores are reported. method may find utility in other domain-specific NLP tasks, such as relation extraction or question answering. Our plan is to delve into applications for these tasks in the future. Conceptualization, Y.K. ; Methodology, Y.Z., X.H. and Y.K. ; Software, Y.Z. and Y.K. ; Validation, Y.K. ; Formal analysis, Y.Z. ; Investigation, Y.Z. and X.H. ; Resources, C.X. ; Data curation, Y.K. ; Writing--original draft preparation, Y.Z. ; Writing--review and editing, Y.K. and C.X. ; Visualization, Y.Z. and Y.K. ; Supervision, C.X. and Y.K. ; Project administration, C.X. ; Funding acquisition, X.H. and C.X. All authors have read and agreed to the published version of the manuscript. This work was funded by National College Students Innovation and Entrepreneurship Training Program, China (grant number: 202210500030). Not applicable. Code is available at [[https://github.com/yuanzhiKe/NER_KG_Att_2023](https://github.com/yuanzhiKe/NER_KG_Att_2023)]([https://github.com/yuanzhiKe/NER_KG_Att_2023](https://github.com/yuanzhiKe/NER_KG_Att_2023)), accessed on 25 December 2023.