Leveraging Probabilistic Graph Models in Nested Named Entity Recognition for P olish (2023)

Abstract: This paper presents ongoing work on leveraging probabilistic graph models, specifically conditional random fields and hidden Markov models, in nested named entity recognition for the Polish language. NER is a crucial task in natural language processing that involves identifying and classifying named entities in text documents. Nested NER deals with recognizing hierarchical structures of entities that overlap with one another, presenting additional challenges. The paper discusses the methodologies and approaches used in nested NER, focusing on CRF and HMM. Related works and their contributions are reviewed, and experiments using the KPWr dataset are conducted, particularly with the BiLSTM-CRF model and Word2Vec and HerBERT embeddings. The results show promise in addressing nested NER for Polish, but further research is needed to develop robust and accurate models for this complex task.

## introduction named entity recognition (ner) is a fundamental task in natural language processing (nlp) that involves identifying and classifying named entities, such as names of people, organizations, locations, and more, within text documents. the ability to accurately extract and categorize these entities plays a crucial role in various nlp applications, including information retrieval, question answering, text summarization, and machine translation. ner serves as the foundation for understanding the semantics and context of textual data, enabling more advanced language understanding systems. in the realm of ner, there exists a more intricate and challenging variant known as nested ner. while traditional ner focuses on identifying individual named entities within a sentence, nested ner deals with the recognition of nested or hierarchical structures of entities that overlap with one another. this added complexity arises when named entities, such as organizations or locations, encompass other entities within them, such as person names or specific addresses. nested entities refer to entities that are embedded within other entities. for instance, consider the entity "john smith" here entity can include two additional entities inside of it -"john" and "smith" which can be labeled as first and last names. the goal of nested ner is to accurately extract these overlapping entities while preserving their hierarchical relationships, allowing for a more nuanced understanding of the information contained within the text. i will examine the methodologies and approaches employed in this task, focusing on probabilistic graph models, such as conditional random fields @xcite and hidden markov models @xcite , which have proven effective in capturing the contextual dependencies and relationships between entities. these methods are further described in the sections 2 and 3. ## conditional random fields conditional random fields (crf) are probabilistic models used for structured prediction tasks, particularly in the field of natural language processing. they are often employed in tasks such as sequence labeling, named entity recognition, part-of-speech tagging, and speech recognition. in the context of nested entity recognition, crf play a significant role in identifying and labeling hierarchical or nested entities within a text. nested entity recognition involves identifying and labeling both the outer and inner entities correctly. this task is challenging because the boundaries of nested entities can overlap, making it difficult to determine the correct labeling. crf address this challenge by considering the dependencies and correlations among neighboring words and labels in a sequence. ## hidden markov models hidden markov models (hmm) are statistical models widely used in various fields, including natural language processing. they are particularly useful in sequence labeling tasks, such as nested entity recognition, where the goal is to identify and classify named entities within a text. in the context of ner, hmm are often employed for the task of nested named entity recognition, which involves identifying named entities that are hierarchically structured and nested within each other. the basic idea behind hmm is to model the underlying structure of a sequence of observations and their corresponding labels. in the case of ner, the observations are the words or tokens in a text, and the labels represent different named entity categories. hmm assume that the underlying labels (states) generating the observations (emissions) form a markov chain, where the current state depends only on the previous state. one common approach is to use a layered hmm, where each layer corresponds to a level of nesting. the innermost layer represents the most specific entities, and as we move outward, the layers represent progressively broader entities. during the training phase, the model learns the transition probabilities between different states (labels) based on the training data. it also learns the emission probabilities, which represent the likelihood of observing a particular word given a certain state. these probabilities are estimated using techniques such as the maximum likelihood estimation or the viterbi algorithm. the viterbi algorithm is often employed to efficiently compute the most probable label sequence. for polish, there is a system for the ner task called poldeepner2 @xcite . it is based on a pre-trained language model of the transformer type. it has the ability to detect nested ner by extending the set of possible label classes to include classes representing overlapping annotation types. the solution was trained and tested on a dataset available as part of the poleval 2018 @xcite competition. a noticeable problem of such a solution is that with numerous label classes (and this is the collection we are dealing with in this work), the number of class combinations that can overlap grows very quickly. for example, in the case of the set we are analyzing, assuming that we are only analyzing one degree nesting we will get 13,285 classes, and assuming that we are analyzing possible double nesting it will already be 1,062,721. ## experiments the purpose of the experiments is to identify the best model in terms of prediction accuracy for the polish language, which is challenging due to its rich inflectional system, compound words, ambiguity, and context sensitivity. the methods will be tested on a larger set for the nested ner task, which is several times bigger than the current best-known corpora in terms of class size. the solution presented in (shibuya and hovy, 2020) scored the highest f1-score in nested ner task benchmark corpora such as ace2005 @xcite and genia @xcite . therefore, it will be adopted as the first to the polish corpus. ## conclusions in this paper, i have focused on exploring the task of nested named entity recognition (ner) for the polish language and investigated the use of probabilistic graph models, specifically conditional random fields (crf) and hidden markov models (hmm), to address this challenging problem. i reviewed a few related works that have employed crf and hmm for nested ner. these studies proposed various models and techniques, such as incorporating semantic features, cascading crf layers, and leveraging neural models, to improve nested ner performance. to evaluate the effectiveness of these approaches for the polish language, i conducted experiments using the kpwr dataset focusing on the bilstm-crf model and comparing the performance of word2vec and herbert embeddings. overall, leveraging probabilistic graph models, such as crf and hmm, shows promise for addressing nested ner in the polish language. further research and experimentation are needed to develop robust and accurate models for this complex task, which has important implications for various nlp applications.