Exploring Nested Named Entity Recognition with Large Language Models: Methods, Challenges, and Insights (2024)

Abstract: Nested Named Entity Recognition (NER) poses a significant challenge in Natural Language Processing (NLP), demanding sophisticated techniques to identify entities within entities. This research investigates the application of Large Language Models (LLMs) to nested NER, exploring methodologies from prior work and introducing specific reasoning techniques and instructions to improve LLM efficacy. Through experiments conducted on the ACE 2004, ACE 2005, and GENIA datasets, we evaluate the impact of these approaches on nested NER performance. Results indicate that output format critically influences nested NER performance, methodologies from previous works are less effective, and our nested NER-tailored instructions significantly enhance performance. Additionally, we find that label information and descriptions of nested cases are crucial in eliciting the capabilities of LLMs for nested NER, especially in specific domains (i.e., the GENIA dataset). However, these methods still do not outperform BERT-based models, highlighting the ongoing need for innovative approaches in nested NER with LLMs.

## introduction named entity recognition (ner) is a prominent task in information extraction (ie) that entails detecting entities within the text and classifying them into predefined categories, such as person, location, and organization. a notable challenge within ner is nested ner, where named entities are embedded within other entities. this complexity introduces an additional dimension of difficulty to the task, making it a fertile area for research. recent advancements in large language models (llms) have demonstrated their exceptional capabilities across a variety of natural language processing (nlp) tasks, showcasing notable zero-shot and few-shot generalization abilities @xcite . ## method initially, we examine the impact of various output formats on the task's performance. after selecting the optimal output format, we independently apply concepts derived from prior nested ner studies and reasoning techniques previously utilized with llms on ner. subsequently, we develop reasoning techniques specifically tailored for nested ner to enhance the llms' capability to understand complex entity hierarchies. ## conclusions we explored the nested ner using various concepts, methodologies, and techniques from the previous nested ner works. we also proposed nested ner-tailored reasoning techniques and instructions, which demonstrated significant effectiveness on nested ner. we observe the output format that llms would generate is significantly important in the ner task. we also confirmed the previous reasoning technique, especially decomposed-qa, which is highly effective on flat ner, is not robust on the nested ner. our proposed instruction that incorporating the ne label information and nested case description using llms is significantly effective on nested ner. unfortunately, the performance of all of our methods yet outperformed the previous bert-based fine-tuned models. it reveals nested ner still needs to be further explored.