# Advancing Few-Shot Named Entity Recognition with Large Language Model
## 3 Methodology
### Prompt Design for LLM  
Large language models acquire knowledge from prompts by combining task descriptions with demonstrations, enabling them to identify and perform new tasks without explicit gradient updates. As illustrated in Figure 3, we design a structured prompt for the classification of candidate word. This prompt consists of eleven components: _Context_, _Objective_, _Type Definition_, _Few-shot Examples_, _Sentence_, _Types_, _Candidate_, _Question_, _Prompt Engineering Guide_, _Answer Audience_, and _Answer Format_. In this section, we focus on three key components: Type Definition, Few-shot Examples, and Prompt Engineering Guide. 1. Type Definition Across different datasets, entity types may share the same label but have distinct definitions. Relying solely on the LLM's inherent knowledge may fail to capture these nuances. To mitigate this issue, we create dataset-specific entity-type definitions. Specifically, we compile official annotation guidelines, relevant examples, and entity-type details from the dataset, then utilize GPT-4o to generate descriptive definitions based on this information. 2. Few-shot Examples First, we generate sentence embeddings for each sentence in support set using the sentence embedding model. Next, we rank the sentences based on their semantic similarity to the test sample and select the top four most relevant sentences as few-shot examples. Moreover, LLM performance in few-shot learning is sensitive to positional bias, where the order of examples affects the results [28]. To mitigate this effect, examples with higher similarity to the test sample are positioned closer to it in the prompt sequence. 3. Prompt Engineering Guide To harness the reasoning capabilities of the LLM, we introduce three zero-shot prompt engineering techniques: COT prompting, metacognitive prompting, and self-verification prompting. COT prompting operates by decomposing complex tasks into a sequence of subproblems, reformulating the conventional \"input-output\" structure into an \"input-reasoning chain-output\" framework. We implement the zero-shot COT strategy by incorporating the instruction \"Answer the question step by step\" into the prompt, incrementally guiding the model's reasoning process. Figure 2: The overview of two ways for limiting the scope of labels to be predicted. While COT prompting is highly effective for arithmetic and commonsense reasoning, capturing task semantics and implicit context is more critical for NLU tasks. To address this, we introduce metacognitive prompting, which emulates human metacognition to enhance the model's understanding of task requirements. The metacognitive prompt is shown in Table 1. The metacognitive prompting consists of five key steps:  
1. Text Interpretation: Guiding the model to understand the sentence and candidate word, including their positions and roles, similar to human understanding. 2. Preliminary Judgment: Conducting an initial classification of the candidate's type based on type definitions and constraints, reflecting human early judgment. 3. Critical Evaluation: Reviewing the preliminary classification. If uncertainty remains, cross-checking type definitions and few-shot examples to refine the classification, akin to human self-reflection. 4. Decision: Confirming the final entity type and offering a concise rationale, paralleling human decision-making. 5. Confidence Assessment: Evaluating confidence in the final output, simulating the human process of judging reliability. Figure 3: The template of candidate word classification prompt. Moreover, large language models can exhibit hallucination or over-prediction, particularly in NER tasks, where they often label non-entities as valid entities [8; 22]. To address this, we introduce self-verification prompting, which enables the model to validate its own predictions.