CNER : Concept and Named Entity Recognition (2024)

Abstract: Named entities – typically expressed via proper nouns – play a key role in Natural Language Processing, as their identification and comprehension are crucial in tasks such as Relation Extraction, Coreference Resolution and Question Answering, among others. Tasks like these also often entail dealing with concepts – typically represented by common nouns – which, however, have not received as much attention. Indeed, the potential of their identification and understanding remains underexplored, as does the benefit of a synergistic formulation with named entities. To fill this gap, we introduce Concept and Named Entity Recognition (CNER), a new unified task that handles concepts and entities mentioned in unstructured texts seamlessly. We put forward a comprehensive set of categories that can be used to model concepts and named entities jointly, and propose new approaches for the creation of CNER datasets. We evaluate the benefits of performing CNER as a unified task extensively, showing that a CNER model gains up to +5.4 and +8 macro F1 points when compared to specialized named entity and concept recognition systems, respectively. Finally, to encourage the development of CNER systems, we release our datasets and models at https://github.com/Babelscape/cner.

## introduction in the age of big data, the extraction of valuable knowledge from unstructured text is crucial for a wide range of applications, from information retrieval to text mining @xcite . within this context, "nouns and noun phrases have a special status in describing the concepts that people are interested in searching for" @xcite . consequently, to work at their best, natural language processing (nlp) systems should not only leverage the information about named entities, but also that concerning nominal concepts, as the nature of the two is strongly intertwined. a prototypical task in which such interaction is intrinsic to its very nature is relation extraction @xcite , where the triples extracted from text often connect named entities with concepts, e.g. (carlsen, plays, chess) or (chatgpt, is a, chatbot). as a second example, consider coreference resolution @xcite , where the named entities mentioned in a text (e.g. joe biden) are paired with subsequent expressions referring to them, including common nouns such as president or state leader. indeed, many tasks have been shown to benefit from the recognition of named entities @xcite @xcite @xcite , but little attention has been devoted to the identification of concepts. in this paper, we fill this gap by, first, introducing concept recognition (cr), namely, the task of identifying and classifying concepts into predefined semantic types, and, second, harmonizing cr with the well-established task of named entity recognition (ner). as shown in figure 1 , our new joint formulation enables a denser, richer, and more cohesive semantic annotation. specifically, we put forward the following innovative contributions: finally, to encourage the development of cner systems, we release our datasets and models to the research community at https://github.com/ babelscape/cner . ## related work we now review established approaches that deal with the identification and classification of concepts and named entities. specifically, we observe that the vast majority of works in the literature focus exclusively on either concepts or named entities. this is particularly relevant because, as we will see, the currently available categorizations are not suitable for a unified task that integrates both concept and entity recognition. ## cner task we now formalize the concept and named entity recognition (cner) task and then introduce our process to obtain cner categories. ## resources in the previous section, we described the procedure we used to obtain a new comprehensive categorization specifically designed to jointly capture con-cepts and named entities. we now provide datasets to enable the training and evaluation of cner models. crucially, we note that none of the existing datasets provides full coverage of both concepts and entities, usually expressed by common and proper nouns, respectively. to fill this gap, we present a methodology for automatically annotating a large corpus of sentences with cner categories (section 4.1), which we refer to as cner silver . then, we describe the annotation procedure we adopted to produce cner gold , a manually-curated dataset for model evaluation (section 4.2). finally, in section 4.3 we present a detailed analysis of our two resources compared to the existing ones. ## experimental setup we now describe the setup, including datasets (section 5.1) and model architecture (section 5.2), used to answer the following research questions: ## qualitative analysis to better understand the behaviour of cner, ner and cr models, we conduct a qualitative error analysis on cner test gold and report some representative examples in in the first example, we can immediately appreciate the higher annotation density of both concepts and named entities produced by the cner model compared to the specialized ner and cr alternatives. we also report, in the second example, an instance in which the cner model correctly type prediction example 1 gold the [first] measure [town] loc to fall into the [lombards] group ' [hands] part was [forum iulii] loc ([cividale del friuli] loc ), the [seat] loc of the local [magister militum] per . ner the first town to fall into the [lombards] group ' hands was [forum iulii] struct ([cividale del friuli] loc ), the seat of the local magister militum. cr the [first] measure [town] loc to fall into the lombards' [hands] part was forum iulii (cividale del friuli), the [seat] loc of the local [magister] per militum cner the [first] measure [town] loc to fall into the [lombards] group ' [hands] part was [forum iulii] struct ([cividale del friuli] loc ), the [seat] loc of the local [magister] per militum example 2 gold [tom mccarthy] per highlighted the prominent [role] property of [tobacco] plant in the [story] media , drawing on the [ideas] psych of [philosopher jacques derrida] per to suggest the potential [symbolism] media of this. ner [tom mccarthy] per highlighted the prominent role of tobacco in the story, drawing on the ideas of [philosopher jacques derrida] per to suggest the potential symbolism of this. cr tom mccarthy highlighted the prominent [role] event of [tobacco] substance in the [story] media , drawing on the [ideas] psych of jacques derrida to suggest the potential [symbolism] psych of this. cner [tom mccarthy] per highlighted the prominent [role] property of [tobacco] food in the [story] media , drawing on the [ideas] psych of [philosopher jacques derrida] per to suggest the potential [symbolism] psych of this. example 3 gold [john] super has new [weapons] artifact , including [holy water] substance , [bait] biology for the [undead] super , and a [blunderbuss] artifact that uses [zombie] super [parts] artifact as [ammunition] artifact . ner [john] per has new weapons, including holy water, bait for the undead, and a blunderbuss that uses zombie parts as ammunition. cr john has new [weapons] artifact , including [holy water] substance , [bait] substance for the [undead] biology , and a blunderbuss that uses [zombie] biology [parts] artifact as [ammunition] artifact . cner [john] per has new [weapons] artifact , including [holy water] substance , [bait] substance for the undead, and a blunderbuss that uses [zombie] super [parts] artifact as [ammunition] artifact . finally, in the third example, we present a sentence in which our system shows better generalization capabilities: while the named entity john is misclassified as per by both the ner and cner models, the cner system correctly assigns the label super to the concept zombie, which is misclassified by the cr model with the tag biology. ## conclusions in this paper, we presented the novel task of concept and named entity recognition (cner) and introduced a comprehensive set of categories specifically tailored to encompassing the annotation of both proper and common nouns in a unified frame-work. to fulfill the need for specific resources in our newly introduced task, we proposed an automatic procedure that enabled the creation of the first large-scale training corpus for the cner task. our dataset consists of more than 300k annotated sentences, with dense coverage of both proper and common nouns obtained by devising several heuristics for solving sparsity. moreover, we carried out a manual annotation of 2,000 sentences and used the resulting data for validation and testing. our experiments showed that a competitive pretrained language model was able to successfully learn the cner task, achieving 87.20 micro, 59.38 macro and 66.72 span f1 score points. additionally, we also compared the performance of cner as a joint task rather than separately identifying and classifying nominal concepts and named entities, reporting a remarkable increase in performance on both tasks. finally, to encourage the use and development of cner systems, we publicly release our data and models to the research community and leave to future work the experiments on possible downstream task applications.