GL i NER : Generalist Model for Named Entity Recognition using Bidirectional Transformer (2024)

Abstract: Named Entity Recognition (NER) is essential in various Natural Language Processing (NLP) applications. Traditional NER models are effective but limited to a set of predefined entity types. In contrast, Large Language Models (LLMs) can extract arbitrary entities through natural language instructions, offering greater flexibility. However, their size and cost, particularly for those accessed via APIs like ChatGPT, make them impractical in resource-limited scenarios. In this paper, we introduce a compact NER model trained to identify any type of entity. Leveraging a bidirectional transformer encoder, our model, GLiNER, facilitates parallel entity extraction, an advantage over the slow sequential token generation of LLMs. Through comprehensive testing, GLiNER demonstrate strong performance, outperforming both ChatGPT and fine-tuned LLMs in zero-shot evaluations on various NER benchmarks.

## introduction named entity recognition plays a crucial role in various real-world applications, such as constructing knowledge graphs @xcite . traditional ner models are limited to a predefined set of entity types.expanding the number of entity types can be beneficial for many applications, but it may require labeling data and retraining a model, which can be costly and time-consuming. the emergence of large language models (llms), like @xcite , has introduced a new era for open ner by enabling the extraction of any entity type using only natural language instructions. however, powerful llms typically comprise billions of parameters and thus require substantial computing resources. although it is possible to access some llms via apis @xcite , using them at scale can incur high costs. in spite of cost considerations, researchers have recently explored the fine-tuning of open-source gliner is capable of identifying any entity type using a bidirectional transformer encoder (bert-like). it provides a practical alternative to traditional ner models, which are limited to predefined entities, and large language models (llms) that, despite their flexibility, are costly and large for resource-constrained scenarios. gliner can be easily installed via pip and its pretrained models are hosted on huggingface, ensuring easy accessibility. moreover, it can run efficiently on cpu, making gliner especially suitable for environments with limited computing power. ## method this section presents our model, gliner, which is trained to extract any type of entity using a bidirectional language models. our model has three main components: i) a pre-trained textual encoder (a bilm such as bert), ii) a span representation module which computes span embeddings from token embeddings, iii) an entity representation module which computes entity embeddings that the model seeks to extract. the goal is to have entity and span embeddings in the same latent space to assess their compatibility (degree of matching). the overall architecture of our model is depicted in figure 2 . ## further analysis and ablations in this section, we conduct different set of experiments to better investigate our model. ## related works named entity recognition ner is a wellestablished task in the field of nlp, with numerous applications. initially, ner models relied on rulebased system @xcite that were built using handcrafted algorithms and gazetteers @xcite @xcite . however, these models had limitations in terms of scalability and adaptability to new domains or languages. to overcome these issues, machine learning approaches have been proposed @xcite . in the early stages, ner tasks were designed as sequence labeling @xcite @xcite where the objective was to predict tagged sequences (e.g., bilou tags @xcite ). since then, several paradigm shifts have occurred: span-based approaches treating ner as span classification @xcite @xcite @xcite ; ner being treated as a question answering problem @xcite ; and even as a generation task @xcite . zero-shot learning for ner the advent of large-scale autoregressive models has recently transformed many paradigms in nlp through natural language prompting @xcite @xcite . this is also the case for ner @xcite @xcite . others have fine-tuned these models for tasks to better align their capabilities with the requirements of entity recognition @xcite or information extraction in general @xcite @xcite @xcite . this is sometime done through-instruction tuning @xcite @xcite . ## conclusion in this paper, we introduced gliner, a new method for identifying various types of entities in text using bidirectional language models. our model not only outperforms state-of-the-art large language models like chatgpt in zero-shot scenarios but also offers a more resource-efficient alternative, crucial for environments with limited computing power. gliner is versatile, performing well in multiple languages, including those it wasn't trained on. in future work, we aim to further improve gliner's design for enhanced performance and to better adapt it for low-resource languages.