VERIFINER: Verification-augmented NER via Knowledge-grounded Reasoning with Large Language Models (2024)

Abstract: Recent approaches in domain-specific named entity recognition (NER), such as biomedical NER, have shown remarkable advances. However, they still lack of faithfulness, producing erroneous predictions. We assume that knowledge of entities can be useful in verifying the correctness of the predictions. Despite the usefulness of knowledge, resolving such errors with knowledge is nontrivial, since the knowledge itself does not directly indicate the groundtruth label. To this end, we propose VERIFINER, a post-hoc verification framework that identifies errors from existing NER methods using knowledge and revises them into more faithful predictions. Our framework leverages the reasoning abilities of large language models to adequately ground on knowledge and the contextual information in the verification process. We validate effectiveness of VERIFINER through extensive experiments on biomedical datasets. The results suggest that VERIFINER can successfully verify errors from existing models as a model-agnostic approach. Further analyses on out-of-domain and low-resource settings show the usefulness of VERIFINER on real-world applications.1

heading: 1 Introduction
text: Named entity recognition (NER) is a fundamental task in natural language processing that aims to identify entity mentions in input text and assign them to specific types (Mikheev et al., 1999; Lample et al., 2016). Previous works solve NER tasks by training models on human-annotated datasets (Kim et al., 2003; Li et al., 2016; Zhang et al., 2021), improving neural model architectures (Jeong and Kang, 2023), or leveraging external knowledge (Liu et al., 2019; Mengge et al.,
∗Equal contribution †Corresponding author
1Our code and data are available at https://github. com/emseoyk/VerifiNER.
2020; Wang et al., 2021). More recently, simply prompting large language models (LLMs) without training can also perform NER tasks (Wang et al., 2023a; Ashok and Lipton, 2023).
Despite the promising results of these approaches, they still produce plausible but imprecise outputs. The risk to produce such errors is especially salient in domains that require expert-level knowledge such as the biomedical domain. Taking Figure 1 for example, “NF-kappa B” is incorrectly labeled as “RNA” type and “endothelia cells” is identified with wrong span. Erroneous prediction is a significant threat to their application in domains where high precision is required (Dai, 2021; Karim et al., 2023).
In this paper, we seek to minimize these errors by incorporating knowledge into the inference process. Figure 1 suggests that knowledge can serve as a useful evidence for humans to verify that the type of “NF-kappa B” is “protein”. However, avoiding the errors during inference time is challenging for neural models, because they lack in
2441
factual evidence that can assist the models to judge the correctness of predictions. Furthermore, even if the models are equipped with knowledge, there exists a mismatch between knowledge and entity prediction that discourages models from properly detecting and correcting the errors. For example, the definition of “NF-kappa B” as “an ubiquitous, inducible, nuclear transcriptional activator” does not explicitly indicate that the type is “protein”.
To resolve the aforementioned challenge, we aim to propose a post-hoc verification framework that identifies errors from existing NER methods and revises them into more faithful predictions. We employ knowledge base (KB) to provide factual evidence to existing NER systems, and leverage reasoning and in-context learning ability of LLMs to verify entities via knowledge-grounded reasoning. By verifying errors of the previous models in a post-hoc manner, our work exhibits remarkable performance without re-training models.
To this end, we introduce VERIFINER, a novel Verification-augmented NER via Knowledgegrounded Reasoning. With the notion that the errors can be alleviated with factual knowledge and contextual cue, our framework verifies entities in terms of factuality and contextual relevance. For factuality verification, VERIFINER formulates queries from each predicted entity to retrieve knowledge from KB, then reassigns its span and type based on the knowledge. The contextual relevance verification module employs the reasoning ability of LLMs to consider input context when selecting correct entity from candidates that are passed from previous step.
For effective demonstration of our framework, we conduct experiments on biomedical NER which requires domain-expert knowledge to solve the task. While our approach is model-agnostic, we validate that VERIFINER can be applied to both fine-tuned models (Lee et al., 2019; Jeong and Kang, 2023) and a LLM-based NER method (Wang
et al., 2023a) with significant improvement. In addition, we evaluate our framework under out-ofdistribution and low-resource settings, showing its advantages on real-world scenarios.
To the best of our knowledge, this is the first work that solves NER with a verification module that exploits reasoning ability of LLMs. The main contributions of this work are as follows: • We propose a novel framework for identifying
and resolving errors via knowledge-grounded reasoning by utilizing knowledge and LLMs.
• We present VERIFINER, a post-hoc verification module that corrects entity prediction with respect to factuality and contextual relevance.
• We demonstrate the effectiveness and generalization ability of VERIFINER through extensive experiments and analyses.

heading: 2 Preliminary Analysis
text: In this section, we first define the common error types of NER we aim to resolve. Then, we provide further analysis on error cases in existing models to gain insight on how to rectify the errors.
Error Types in NER. For evaluating NER model predictions against the gold annotations, error cases can be categorized as either false positives (FP) or false negatives (FN) based on whether the model incorrectly identifies entities or fails to recognize the gold entities that is present. Within the FP, errors can occur from mismatches of type or span.
Depending on the combinations of these mismatches, errors are classified into more finegrained error types, including Type, Span, Type&Span and Spurious errors. We provide definitions and examples of each error type in Table 1.
Error Analysis on NER Models. We conduct an analysis on existing errors using two fine-tuned models, i.e., ConNER (Jeong and Kang, 2023),
BioBERT (Lee et al., 2019), and one promptingbased LLM, i.e., GPT-NER (Wang et al., 2023a). Figure 2 shows the ratio of different error types for each model. In case of the fine-tuned models, false positive errors take the majority of the total error types. For prompting-based LLM, FP cases take more than one third. As low precision in domainspecific NER is a crucial problem (Dai, 2021; Karim et al., 2023) and FP constitute a significant proportion across all three models, we focus on correcting the FP cases from the initial predictions. By doing so, we can boost the NER performance regardless of the model that comes beforehand.
To gain clues on how to resolve the errors, we examine where existing models fall short to make precise predictions. Among the FP cases, we take a closer look on errors (i.e., Type, Span, and T&S) where the models’ predictions partially overlap with the gold annotation, but either the type or span is incorrect. In Figure 2, almost 60% of FP cases are partially overlapping predictions for all three models. For span error, in Figure 3, more than 80% of predicted entities show difference in length with gold annotation within a margin of two tokens. This implies that NER models resort to plausible yet incorrect predictions with a small margin. Thus, instead of completely discarding the wrong entity predictions, we use them as queries to initialize our verification process in a post-hoc manner.
3 VerifiNER
In this section, we propose VERIFINER, a framework that verifies errors from NER models with external knowledge and the reasoning ability of LLMs. In this paper, we define the term verification as a process that also includes revision. Motivated by the observations in Section 2, we design our framework to particularly focus on enhancing precision; that is, we first identify errors in predicted
entities and then correct them into accurate outputs. The overall framework is illustrated in Figure 4.
Formally, given an input sequence X = {x1, x2, · · · , xn} of n tokens and a predefined type set T , our goal is to produce a revised entity prediction ē by verifying an entity e = (s, t) that is originally predicted by an off-the-shelf NER model, where s = [xbeg : xend] and t (∈ T ) represent its span and type, respectively. In this sense, the verification process targets both the span and type of the entity. Note that span verification must precede type verification, because predicting the type t of an entity e depends on the semantics of e from its span s. For example, the type of “PEBP2” is “protein”, but a longer span, “PEBP2 site”, is classified as “DNA”; thus, we first identify spans and then proceed to determine types.
Specifically, we first verify the factuality of the span and type (Sections 3.1 and 3.2), then verify whether the factually approved entity is relevant to the input context (Section 3.3) for precise revision.

heading: 3.1 Span Factuality Verification
text: In this step, we rectify span error through collecting candidate spans from the predicted entity and verifying them by leveraging a KB. Based on the observation in Section 2, we assume that the gold entity is likely to be adjacent to the predicted entity. Therefore, we expand the range of candidate spans around the predicted entity span s in both directions to increase the likelihood of the gold entity within our candidate set.
We collect a set of candidate spans S̃ by extending the left and right offsets of a span s with hyperparameter α, then enumerate sub-sequences within the offsets [xbeg−α : xend+α], so that each candidate partially or completely overlaps with the span s of the predicted entity e. As shown in Figure 4 (a), {“human mononuclear”, “human”,
“from human”, ..., “mononuclear”, “mononuclear leukocytes”, “human mononuclear leukocytes”} can be extracted from the predicted entity “human mononuclear”.
Then, we search the KB to prune only factually valid candidates.2 Using each candidate span s̃ (∈ S̃) as a query, we check whether knowledge for each candidate exists in the KB. If a candidate is found in the search, we consider its factuality is verified and collect its associated knowledge k. On the other hand, if the candidate is not defined in the KB, we assume it is a noisy candidate and do not consider it further in the remaining process. For example, in Figure 4 (a), the search result for “human mononuclear” is not found, and this candidate is excluded from the next step. In consequence, we obtain a pruned set of candidate spans S̃.

heading: 3.2 Type Factuality Verification
text: As we collect the set of candidate spans s̃ and their associated knowledge k, we proceed to re-assign types to s̃ grounded on the retrieved knowledge.
2We assume a domain-specific NER scenario where all gold entities exist entirely within the KB. More explanation is in Appendix A.
While knowledge serves as a reliable source for verifying candidates, directly applying it for verification is challenging. It is due to the fact that knowledge often lacks explicit indications regarding whether a candidate is correctly labeled. In Figure 5, the knowledge of “mononuclear leukocytes”, i.e., “a white blood cell ...” and “Quantitative Concept” do not exactly match with “cell_type”.
To this end, we leverage the reasoning ability of LLMs to project knowledge into predefined types. We accomplish this by generating evidence k′ to
assist in assigning types grounded on knowledge. Specifically, we rationalize the source knowledge k into verbalized form to generate k′. Then, we provide (s̃, k′) with the predefined label set T and prompt the LLM to re-assign type t̃ based on the evidence k′. An example of knowledge-grounded evidence is provided in Figure 5. Based on provided definition and semantic type of the candidate “mononuclear leukocytes”, LLM generates a knowledge-grounded evidence and assigns type to the candidate.
If the knowledge is irrelevant to the domain, the LLM will assign t̃ as NONE. Consequently, each entity will have a set of factuality-verified candidates ẽ ∈ Ẽ , where each candidate is ẽ = (s̃, t̃).

heading: 3.3 Contextual Relevance Verification
text: Lastly, we select a final candidate entity from Ẽ based on contextual relevance for revising the prediction e to ē. A candidate ẽ can be a valid entity ē if (1) it is semantically relevant to the input context X , and (2) its knowledge-grounded evidence k′ is aligned well with the context compared to other candidates. To determine the final candidate grounded on knowledge and context, we employ in-context learning ability of LLMs.
When verifying contextual relevance of the can-
didate, both local and global contextual information should be considered. However, reasoning only once may lead to insufficient attention on limited contexts. Therefore, we sample multiple reasoning paths to gather answers that reflect various aspects of the context. This process resembles the human annotation process of gathering various opinions from multiple annotators and converging them into a single consistent answer through discussion. To facilitate this process, we employ self-consistency (Wang et al., 2023b) and use consistency voting to select the candidate that is most suitable for the context. As demonstrated in Figure 6, LLMs generate reasoning paths that are properly grounded on knowledge.
Given the input context X , candidates Ẽ , and evidence for each candidate k′, we prompt the LLMs to sample N reasoning paths where each path selects a single candidate that has the most faithful evidence and is relevant to X . We conduct majority voting over the collected N answers and select the answer that receives the most votes. Finally, the prediction is revised as ē = (s̄, t̄). The prompts used in our framework are provided in Appendix D.

heading: 4 Experiments
text: In this section, we design our experiments to answer the following research questions: • RQ1: Can VERIFINER faithfully identify and
revise errors? • RQ2: Can VERIFINER induce generalizability
to fine-tuned models in various test distributions? • RQ3: Can VERIFINER effectively mitigate the
low-resource challenge in domain-specific NER?

heading: 4.1 Experimental Settings
text: We evaluate our framework on two biomedical datasets, each with different set of predefined types: BC5CDR (Li et al., 2016) is annotated with Chemical and Disease, while GENIA (Kim et al., 2003) includes five types, including cell_line, cell_type, DNA, RNA, and protein. We use the Unified Medical Language System (UMLS) (Bodenreider, 2004) as a KB, which is a database containing over two millions of biomedical terminologies annotated with their definitions, semantic types, and lexical relationships. We employ ChatGPT as an LLM to implement our framework (OpenAI, 2023).3 Following Jeong and
3We use gpt-3.5-turbo-1106 model among the various versions of ChatGPT.
Kang (2023), we measure the performance utilizing entity-level Precision (P), Recall (R), and F1.

heading: 4.2 NER Models
text: To validate the effectiveness of VERIFINER as a model-agnostic approach, we employ two groups of NER models as a test-bed for our evaluation: For fine-tuned models, we use ConNER (Jeong and Kang, 2023) and BioBERT (Lee et al., 2019), trained for the NER task on each dataset. For prompting-based LLMs, we consider GPTNER (Wang et al., 2023a), which predicts entity spans of interest and their types in a generative manner via few-shot prompting.

heading: 4.3 Baseline Methods
text: We compare VERIFINER with other methods that revise the initial prediction in a post-hoc manner. (1) Manual Mapping: Naive use of external knowledge to verify predictions. We reassign entity types by manually mapping semantic types found in the KB to the predefined labels. Assuming that entities with the same semantic type will have the same label accordingly, the purpose of Manual Mapping is to solely rely on the KB and not the reasoning function of LLM. To construct a semantic type-label map, we use entities from the train set to retrieve the semantic types from KB and manually align them with the label. For example, if the KB retrieves [“Organic Chemical”, “Pharmacologic Substance”] as semantic types of “cyproterone acetate” and its label is “Chemical”, then we align the semantic
type “Pharmacologic Substance” to the label “Chemical”. The statistics of semantic types assigned to each label are in Appendix A. (2) LLMrevision: Simple revision using an LLM, where the model re-examines the input context and generates revised context based on marked predicted entities. (3) LLM-revision w/ CoT: We incorporate zero-shot CoT (Kojima et al., 2022) in addition to LLM-revision. We provide prompts for LLMrevision and w/CoT in Appendix D.

heading: 5 Results

heading: 5.1 Effectiveness of VERIFINER (RQ1)
text: Comparison with Baselines. From the results in Table 2, we have the following observations: (1) For all NER models, VERIFINER consistently achieves significant improvements over initial predictions on both datasets. This demonstrates the effectiveness of our model-agnostic verification method. (2)VERIFINER also outperforms other revision baselines by notable margins. When comparing LLM-revision and w/ CoT to ours, we find that relying solely on the internal knowledge of LLMs degrades their performance, necessitating a reliable external knowledge source to faithfully verify entity predictions. (3) While the incorporation of reliable knowledge is essential for verification, the performance drop in Manual Mapping additionally highlights the need for an intermediate reasoning process to bridge the gap between the retrieved knowledge and model predictions. (4) As VERIFINER aims to precisely correct entity predic-
tions, an inevitable decrease in recall is expected. However we notice that VERIFINER achieves considerable increase in precision across all NER models without a severe drop in recall. Across all datasets and models, the average increase in precision is 20.03%, while recall decreases by only 1.09%. This indicates that our approach can faithfully correct errors without imposing a significant trade-off between precision and recall. We provide further case study in Appendix C.
Ablation Study. To validate the effectiveness of each component in our framework, we conduct ablation studies as follows: excluding consistency voting (w/o CV), omitting knowledge-grounded evidence generation and consistency voting (w/o evidence, CV), generating evidence based on internal knowledge of LLMs (w/o KB). In Table 3, we have the following observations for each ablated approach: (1) Verifying contextual relevance with multiple reasoning paths is helpful for enhanced accuracy, suggesting that diverse aspects within the context should be considered to select the consistent answer. (2) Knowledge-grounded evidence largely affects the final NER accuracy, and this implies that it is important to bridge the gap between the collected knowledge and the target application (or task). (3) Parametric knowledge from the LLM still lacks in domain expertise, supporting the necessity of employing external knowledge in our framework.
Analysis on Error Correction. To have a better understanding of how VERIFINER works, we provide an in-depth analysis on error correction rate compared to other revision baselines. Figure 7 shows that VERIFINER rectifies errors more faithfully than the baselines across all error types. Notably, 52% and 78% of errors are corrected in total for GENIA and BC5CDR, respectively. Upon examining the results for each error type,
VERIFINER successfully corrects the majority of type errors, demonstrating its effective execution of knowledge-grounded re-typing. For span error, other revision baselines show significantly lower correction rates, while VERIFINER successfully rectifies it twice as much. When comparing the results across datasets, VERIFINER tends to struggle more on GENIA compared to BC5CDR, due to more fine-grained type set of GENIA. However, even on GENIA, VERIFINER manages to correct over half of the errors in most cases while other revision baselines tend to remain at less than onethird. Additionally, in the case of BC5CDR, it achieves correction rates close to 90% for spurious errors, highlighting its capability to properly filter out invalid entities.

heading: 5.2 Generalizability of VERIFINER (RQ2)
text: Although fine-tuned NER models show promising accuracy in in-distribution settings, i.e., the train and test data are sampled from the same dataset (or distribution), they can hardly be applied to unseen labels or shifted distribution, lacking robustness in out-of-distribution (OOD) settings. Thus, we investigate if our VERIFINER framework can also prove effectiveness in OOD scenarios, where finetuned models are trained on the datasets that are distinct from the test distribution.
Unseen Distribution. We apply VERIFINER to the NER models fine-tuned on a source dataset, and evaluate it on a target dataset whose labels are not seen during the training (i.e., train on GENIA and infer on BC5CDR, and vice versa). Intuitively, the
fine-tuned models themselves cannot be evaluated in the cross-dataset setting due to the different entity type sets between the source and target datasets. Thus, we denote them as not applicable (N/A) in Table 4. On the contrary, the fine-tuned models equipped with VERIFINER perform much better than the prompting-based LLM (GPT-NER), and even achieve comparable performance to baselines trained on the target dataset. This suggests that VERIFINER can heighten the performance of finetuned NER models on unseen datasets, where the training dataset is not accessible.
Shifted Distribution. Additionally, we evaluate the performance of VERIFINER in scenarios where training and test datasets have different distributions but their entity type sets are identical. Specifically, we split the original train data train into subsets train′1, train ′ 2 ⊂ train, making their shifted type distributions distinct from the distribution of target, i.e., Dtrain′1 ,Dtrain′2 ̸= Dtest(≈ Dtrain), but their type sets are the same, i.e., Ttrain′1 , Ttrain′2 = Ttest. Then, using each subset as training data, we fine-tune NER models (i.e., ConNER and BioBERT) to perform inference on test. Please refer to Appendix A for experimental details and results of other subsets. In Table 5, VERIFINER brings a significant improvement on both models. For both BioBERT and ConNER, our approach is even comparable to the in-distribution setting that is trained on the target distribution. In particular, VERIFINER even outperforms BioBERT trained on the target dataset of GENIA by 14.59% on F1 score. This highlights VERIFINER’s ability to generalize without overfitting to the source dis-
tribution, as it verifies predictions based on reliable external knowledge sources.

heading: 5.3 Robustness of VERIFINER in Low-resource Scenarios (RQ3)
text: To assess how our framework can elevate robustness of NER models in low-resource scenarios, we investigate the performance changes of two fine-tuned models and those combined with VERIFINER, while varying the scale of training data for fine-tuning. For this, we generate training datasets of varying sizes, by randomly sampling examples from the original dataset across a range from 5% to 100%. Figure 8 illustrates that post-hoc verification of VERIFINER leads to notable enhancements in the performance of both BioBERT and ConNER across all low-resource scenarios examined in both datasets. Specifically, the gap between the finetuned models and those with VERIFINER becomes large as the number of training examples decreases. Furthermore, VERIFINER consistently achieves high precision irrespective of the number of training examples. In conclusion, VERIFINER is robust in diverse low-resource scenarios within the biomedical domain, where the scarcity of available datasets has remained as a long-standing challenge.

heading: 6 Related Work
text: Named Entity Recognition. A conventional approach for solving NER task is to predict probability distribution of entity types and sequentially label each token (Jeong and Kang, 2023; Lee et al., 2019). In domain-specific NER, data scarcity challenge has been a consistent challenge. Recent works attempt to solve the issue with data augmentation (Chen et al., 2021; Zhou et al., 2022). Other works mitigate the challenge with domain adaptation methods (Zhang et al., 2021; Yang et al., 2022; Xu et al., 2023). In most recent years, with the advent of LLMs, a line of work reformulate task definition of NER in a generative manner to leverage LLMs’ capabilities in NER tasks. Ashok and Lipton (2023) prompts an LLM to produce a list of potential entities with explanations that justify their compatibility with the provided entity type definitions. Wang et al. (2023a) instructs an LLM to recognize entities by using special tokens to surround them.
Knowledge-augmented LMs. For knowledge augmentation, previous works (Guu et al., 2020; Yamada et al., 2020; Borgeaud et al., 2022)
show performance improvements on knowledgeintensive tasks. Zhang et al. (2019) utilize both large-scale corpora and knowledge graphs to train a language model and take full advantage of various knowledge simultaneously. Qin et al. (2021) proposes a novel contrastive learning framework to obtain a deep understanding of the entities and their relations in text. Similarly, later works (Lewis et al., 2020; Kang et al., 2022; Li et al., 2022; Izacard et al., 2022; Chae et al., 2023) augment LMs with external knowledge during fine-tuning. However, they lack in generalization ability across different domains, and require changing model architectures or additional training steps on each task and dataset.
Considering these challenges, recent methods (Lazaridou et al., 2022; Trivedi et al., 2022; Peng et al., 2023) use generalization capabilities of large or instruction-tuned LMs to incorporate external knowledge. Baek et al. (2023) first retrieve relevant facts from a knowledge graph and prepend the retrieved facts to the input question, which is then forwarded to LLMs to generate the answer. Zhao et al. (2023) post-edits reasoning chains according to external knowledge to increase prediction factuality.
NER with External Knowledge. There are a number of works that augment external knowledge on NER task. Nie et al. (2021) proposes a KB-aware NER framework which incorporates KB knowledge to via type projection to alleviate heterogeneity between KB knowledge and NER type scheme. Fu et al. (2023) proposes a dictionarybased method that recognizes biomedical entities from input text via synonym generalization. However, such dictionary-based methods yet fail to generalize over multiple domains.

heading: 7 Conclusion
text: In this paper, we present VERIFINER, a verification framework that identifies errors from existing NER methods and revises them into more faithful predictions via knowledge-grounded reasoning. It is the first attempt to solve NER with verification. Specifically, we introduce VERIFINER, a plug-and-play verification module which verifies the factuality and contextual relevance of entity with knowledge-grounded reasoning. Through extensive experiments, we demonstrate effectiveness of our framework over baselines. It is worth noting that VERIFINER is also robust in out-of-domain settings and low-resource scenarios with remarkable performance. Considering the feasibility of VERIFINER as a model-agnostic approach and its demonstrated generalization capabilities, we expect our work will enhance reliability in NER and be applicable to other domains in future research.
Limitation
Despite the remarkable performance of VERIFINER, there is still room for improvements on our framework. First of all, we conduct experiments using gpt-3.5-turbo-1106 model from OpenAI for LLM. Considering the significance of LLM as a major component in the framework, it is yet to be discovered whether the effectiveness of VERIFINER is valid with other LLMs. Thus we leave application of our framework on different open and closed LLMs for future work.
In this work, we confine the scope of our work to biomedical domain. However, as a plug-andplay framework that can be applied to any model, VERIFINER has a potential on other knowledge-
intensive domains such as legal or scientific domains. In the future, we plan to investigate its application on other domains.
Lastly, our framework necessitates multiple inferences using LLMs, which can be computationally expensive. This aspect becomes particularly significant when integrating our framework into applications that require real-time inference. We believe this limitation can be resolved by using smaller models which are distilled from LLMs.
Ethical Consideration
The main aspect of our work with the potential for ethical pitfalls is the use of LLMs for our framework. Recent work has highlighted the risks of LLMs in hallucination (Zhang et al., 2023). This problem might be more critical in biomedical NER, where prediction can be used for clinical decision support, drug discovery, and personalized medicine. Inaccurate or hallucinated information could lead to misdiagnoses, inappropriate treatment recommendations, or erroneous scientific research directions, ultimately posing a significant risk to patient safety and public health.
We argue here that this risk is largely mitigated in our work, mainly due to our verification process that incorporates knowledge from KBs. The goal of our work is reducing such errors with verification, specifically by integrating structured data and validated information from KBs. This integration allows for cross-referencing and validation of the outputs generated by the LLM, ensuring that any identified entities and associated information align with reliable biomedical knowledge.

heading: Acknowledgements
text: This work was supported by the IITP grant funded by the Korea government (MSIT) (No. RS-2020II201361) and the NRF grant funded by the Korea government (MSIT) (No. RS-2023-00244689).

heading: B Experimental Settings
text: Unseen Distribution Settings. We fine-tune CONNER and BIOBERT using the training sets of the GENIA and BC5CDR datasets, respectively. Then using the checkpoints from each trained
model, we cross-infer on the test set of remaining dataset, which is not used during train phase.
When applying VERIFINER to the inferred predictions, we adjust the initial predictions by a threshold to the logits of the model output. This step is implemented to mitigate the occurrence of an excessive number of "O" tags in the generated output, which could otherwise lead to a scarcity of entity input to the verification. Specifically, we compute the output logit percentile for the "O" tags across all instances. Then, for the top 10% of "O" tokens, we retain the original logits, while for the rest, we multiply the logits by -9999. This adjustment ensure that only predictions confidently classified as "O" tags remained, while the rest are assigned as entities for the input of verification.
Shifted Distribution Settings. To assume the scenarios where the target distributions differ from the source distributions, we manually split both datasets into two distinct subsets. We then finetune CONNER and BIOBERT using each subset as a source and inferred on each test set. The distribution of each subset and the experimental results for subset 2 are presented in Figure 9 and Table 10, respectively.
Computational Resources and API Cost. We run ConNER and BioBERT on eight NVIDIA RTX A5000 GPUs. For ChatGPT API usage, we use $ 470 in total.

heading: C Case Study
text: We select representative examples of verification result by VERIFINER from span error case in GENIA and t&s error case in BC5CDR.
Table 11 presents a span error verification from GENIA. The two candidates are very similar with each other, yet VERIFINER manages to distinguish the answer between the two. Specifically, VERIFINER selects the gold annotation “human lymphocytes” as the final answer, based on the evidence that “human lymphocytes” narrows down the scope to human cells specifically.
Table 12 shows a type & span error verification result from BC5CDR. It is evident that the intitial prediction, “bupivacaine arrhythmogenicity” is not in the candidate list. This indicates that the NER model has predicted an invalid entity that is not registered in the KB. Luckily, VERIFINER is able to filter out such invalid entities by searching the candidates in KB and verify their correctness with using knowledge as evidence.

heading: D Prompts
text: We provide type factuality verification prompts in Table 13, 14, and contextual relevance verification prompts in Table 15, 16. Note that we use individual prompt for GENIA and BC5CDR respectively.
Type Factuality Verification Prompt (GENIA)
[Task Description] You are solving named entity recognition task for biomedical articles. There is an entity to which you need to assign either one of the six classes : protein, DNA, RNA, cell_line, cell_type or None. Class None is assigned when an entity does not belong to any of the five classes: protein, DNA, RNA, cell_line, and cell_type. I will give you an entity and reliable knowledge about the entity for reference. There are two types of knowledge: definition and semantic type of entities you need to predict. Identify the class of the entity and explain why. When generating explanation, you need to quote the knowledge. If both definition and semantic type are not provided, use context information of entity in Paragraph or your own knowledge to explain. Remember, you should assign at least one class for each entity from above six (protein, DNA, RNA, cell_line, cell_type or None). Using Example 1 to 6 as reference, answer Example 7.
[Example 1] Entity: GRalpha Knowledge about this entity: The definition of GRalpha is not provided. The semantic type of GRalpha is [‘Amino Acid, Peptide, or Protein’, ‘Receptor’]. Explanation: Based on the given knowledge, the semantic type of GRalpha is ‘Amino Acid, Peptide, or Protein’ and ‘Receptor’. Proteins are large molecules that perform various functions in the body, including acting as receptors for hormones and other signaling molecules. In this case, GRalpha is described as a hormone-activated transcription factor, which aligns with its role as a protein receptor. Therefore, the class of the entity GRalpha is protein.
[Example 2] Entity: neutrophils Knowledge about this entity: The definition of neutrophils is [‘Granular leukocytes having a nucleus with three to five lobes connected by slender threads of chromatin, and cytoplasm containing fine inconspicuous granules and stainable by neutral dyes.’]. The semantic type of neutrophils is [‘Cell’]. Explanation: Based on the given knowledge, the definition of neutrophils is ‘Granular leukocytes having a nucleus with three to five lobes connected by slender threads of chromatin, and cytoplasm containing fine inconspicuous granules and stainable by neutral dyes.’ This definition indicates that neutrophils are a type of cell, specifically a type of granular leukocyte. Therefore, the class of the entity neutrophils is cell_type.
[Example 3] ...
Table 13: The prompt for type factuality verification on GENIA (Examples 3 to 6 are omitted in this table).
Type Factuality Verification Prompt (BC5CDR)
[Task Description] You are solving named entity recognition task for biomedical articles. There is an entity to which you need to assign either one of the three classes: Chemical or Disease or None. You should only use one of the three classes. No other class exists. I will give you an entity and reliable knowledge about the entity. There are two types of knowledge: definition and semantic type. Identify the class of the entity and explain why. When generating explanation, you need to use the given knowledge. If you are not certain with your decision, you may explain why you are not certain. Even if it does not fit to any of the classes, try to assign the most related one.
[Example 1] Paragraph: Entity: chest pain Knowledge about this entity: The definition of chest pain is [’Pressure, burning, or numbness in the chest.’]. The semantic type of chest pain is [’Sign or Symptom’]. Explanation: Chest pain is a symptom characterized by pressure, burning, or numbness in the chest. Based on the provided knowledge, chest pain falls under the semantic type of ’Sign or Symptom’. Therefore, the class of the entity chest pain is Disease.
[Example 2] Entity: amoxicillin Knowledge about this entity: The definition of amoxicillin is [’A broad-spectrum semisynthetic antibiotic similar to AMPICILLIN except that its resistance to gastric acid permits higher serum levels with oral administration.’]. The semantic type of amoxicillin is [’Organic Chemical’, ’Antibiotic’]. Explanation: amoxicillin is a broad-spectrum semisynthetic antibiotic that is similar to AMPICILLIN. It has the ability to resist gastric acid, allowing for higher serum levels with oral administration. Additionally, it is classified as an organic chemical and an antibiotic. Based on this information, it can be concluded that amoxicillin belongs to the class of Chemical. Therefore, the class of the entity amoxicillin is Chemical.
[Example 3] ...
Table 14: The prompt for type factuality verification on BC5CDR (Examples 3 to 6 are omitted in this table).
Contextual Relevance Verification Prompt (GENIA)
[Task Description] You are solving named entity recognition task for biomedical articles. You will be provided pairs of candidate entities and explanations of why each entity is labeled such, along with a context in which the candidate entities appear. The entities are very similar, and your job is to choose a pair that has the most reliable explanation, and is most likely to be a biomedical entity considering the context. Think Step-by-step and explain why you think that entity is the answer and compare your answer entity to other candidate entities. Using Example 1 to 8 as reference, answer Example 9.
[Example 1] Pair of candidate entity and explanation: [‘proto-oncogene c-rel’, ‘Since the definition and semantic type of proto-oncogene c-rel are not provided, we can use our own knowledge to determine the class of the entity. Proto-oncogenes are genes that have the potential to become oncogenes, which are genes that can cause cancer when mutated or overexpressed. In this case, proto-oncogene c-rel is mentioned in the context of a DNA probe that spans a conserved domain among other genes, including the p50 DNA binding subunit of NF-kappa B. NF-kappa B is a transcription factor that plays a key role in regulating genes involved in immune responses and cell survival. The fact that proto-oncogene c-rel is mentioned in relation to DNA binding suggests that it may also be involved in gene regulation. Therefore, based on our knowledge, the class of the entity proto-oncogene c-rel is protein.’] [‘c-rel’, ‘In the given paragraph, c-rel is mentioned as a proto-oncogene that is conserved among other genes, including the p50 DNA binding subunit of NF-kappa B. Proto-oncogenes are genes that have the potential to cause cancer when mutated or overexpressed. In this case, c-rel is described as a proto-oncogene, indicating its role in cancer development. Therefore, the class of the entity c-rel is protein.’] Your decision: Based on the provided explanation, ‘proto-oncogene c-rel’ links it to a conserved domain among other genes, including the p50 DNA binding subunit of NF-kappa B. The explanation goes on to connect proto-oncogene c-rel to the role of proto-oncogenes in cancer development, suggesting its involvement in gene regulation. This detailed explanation provides a broader understanding of the entity in the given biomedical context. In comparison, the other candidate entity ‘c-rel’ is less specific in its explanation, merely stating that c-rel is a proto-oncogene without providing the additional context about its conserved domain and its association with the p50 DNA binding subunit of NF-kappa B. The chosen entity, ‘proto-oncogene c-rel,’ is more comprehensive and aligns better with the information provided in the paragraph. Therefore, the final answer is (‘proto-oncogene c-rel’ , protein).
[Example 2] Pair of candidate entity and explanation: [‘CBF’, ‘Based on the given knowledge, the semantic type of CBF is protein. Proteins are large molecules that perform various functions in the body. In the given paragraph, CBF is described as consisting of two subunits, CBF alpha and CBF beta, with CBF alpha being a DNA binding subunit. This aligns with the role of proteins as molecules that can bind to DNA. Therefore, based on the semantic type information and the context provided, the class of the entity CBF is protein.’] [‘CBF beta’, ‘Based on the given knowledge, the semantic type of CBF beta is protein. In the given paragraph, CBF beta is described as a subunit that stimulates the DNA binding activity of CBF alpha. Therefore, the class of the entity CBF beta is protein.’] [‘beta’, ’Based on the given knowledge, the semantic type of beta is ’protein’. Proteins are large molecules that perform various functions in the body. In this case, ’beta’ is mentioned as a subunit that stimulates the DNA binding activity of CBF alpha, indicating its role as a protein. Therefore, the class of the entity ‘beta’ is protein.’] Your decision: Based on the provided explanations, candidate ‘CBF beta’ is the subunit of CBF, but ‘CBF beta’ is specifically highlighted as a subunit that stimulates the DNA binding activity of CBF. The explanation for ‘CBF beta’ explicitly assigns it the semantic type of protein, aligning with the general role of proteins in the body. It is a specific subunit with a clear role in stimulating DNA binding activity, and its semantic type aligns with the expected classification of proteins. Therefore, the final answer is (‘CBF beta’, protein).
[Example 3] ...
Table 15: The prompt for contextual relevance verification on GENIA (Examples 3 to 8 are omitted in this table).
Contextual Relevance Verification Prompt (BC5CDR)
[Task Description] You are solving named entity recognition task for biomedical articles. You will be provided pairs of candidate entities and explanations of why each entity is labeled such, along with a context in which the candidate entities appear. The entities are very similar, and your job is to choose a pair that has the most reliable explanation, and is most likely to be a biomedical entity considering the context. Explain why you think that entity is the answer and compare your answer entity to other candidate entities. Use Example 1 to 20 as reference.
[Example 1] Pair of candidate entity and explanation: [‘hepatitis’, ‘hepatitis is defined as inflammation of the liver. Based on this definition, it can be classified as a disease. Additionally, the semantic type of hepatitis is also ‘Disease or Syndrome’. Therefore, the entity hepatitis should be classified as a Disease.’] [‘fulminant hepatitis’, ‘Since there is no definition available for fulminant hepatitis, I will rely solely on the semantic type. The semantic type of fulminant hepatitis is ’Disease or Syndrome’, indicating that it is a disease. Therefore, I would classify fulminant hepatitis as a Disease.’] Your decision: Based on the given knowledge, hepatitis is defined as inflammation of the liver, which falls under the category of a disease. Additionally, the semantic type of hepatitis is also ‘Disease or Syndrome’. Therefore, the entity hepatitis should be classified as a Disease. On the other hand, there is no specific definition available for fulminant hepatitis. However, the semantic type of fulminant hepatitis is ‘Disease or Syndrome’, indicating that it is a disease. Therefore, both entities can be classified as Disease. However, ‘fulminant hepatitis’ provides additional information about the severity of the hepatitis, indicating a rapid and severe progression. This makes it a more specific mention compared to ‘hepatitis’. Therefore, the final answer is (‘fulminant hepatitis’, Disease).
[Example 2] Pair of candidate entity and explanation: [‘haloperidol’, ‘The definition of haloperidol is ‘A phenyl-piperidinyl-butyrophenone that is used primarily to treat SCHIZOPHRENIA and other PSYCHOSES. It is also used in schizoaffective disorder, DELUSIONAL DISORDERS, ballism, and TOURETTE SYNDROME (a drug of choice) and occasionally as adjunctive therapy in INTELLECTUAL DISABILITY and the chorea of HUNTINGTON DISEASE. It is a potent antiemetic and is used in the treatment of intractable HICCUPS. (From AMA Drug Evaluations Annual, 1994, p279) The semantic type of haloperidol is ‘Organic Chemical’, ‘Pharmacologic Substance’.] Your decision: haloperidol is an organic chemical and pharmacologic substance that is primarily used to treat schizophrenia and other psychoses. It is also used in schizoaffective disorder, delusional disorders, ballism, Tourette syndrome, and occasionally as adjunctive therapy in intellectual disability and the chorea of Huntington disease. It is a potent antiemetic and is used in the treatment of intractable hiccups. Based on the provided knowledge, haloperidol is classified as a pharmacologic substance. Therefore, the final answer is (‘haloperidol’, Chemical).
[Example 3] ...
Table 16: The prompt for contextual relevance verification on BC5CDR (Examples 3 to 20 are omitted in this table).
LLM-revision Prompt (GENIA)
[Task Description] Within the GENIA dataset consisting of PubMed articles, the sentences are tagged with labels. The task is to verify whether the word is properly tagged with the correct label from the given sentence and revise it. If the entity is annotated incorrectly, you may revise tags. Here’s what each label means: <P> is ‘protein’, <D> is ‘DNA’, <R> is ‘RNA’, <CL> is ‘cell_line’, <CT> is ‘cell_type’. If the entity corresponds to Protein, <P> should be added before the entity and </P> after it. Follow the format of Example 1 to answer Example 2.
[Example 1] The given sentence: Whereas activation of the <D>HIV-1 enhancer</D> following T-cell stimulation is mediated largely through binding of the transcription factor <P>NF-kappa B</P> to two adjacent <D>kappa B sites</D> in the <R>HIV1</R> long terminal repeat , activation of the <D>HIV-2 enhancer</D> in <CT>monocytes</CT> and <CT>T cells</CT> is dependent on four <D>cis-acting elements</D> : a single kappa B site , two purine-rich binding sites , <D>PuB1</D> and <D>PuB2</D> , and a pets site . The labeled sentence: Whereas activation of the <D>HIV-1 enhancer</D> following T-cell stimulation is mediated largely through binding of the <P>transcription factor</P> <P>NF-kappa B</P> to two adjacent <D>kappa B sites</D> in the <D>HIV-1 long terminal repeat</D> , activation of the <D>HIV-2 enhancer</D> in <CT>monocytes</> and </CT>T cells</> is dependent on four <D>cis-acting elements</D> : a single <D>kappa B site</D> , two purine-rich binding sites , <D>PuB1</D> and <D>PuB2</D> , and a <D>pets site</D> .
[Example 2] The given sentence: {context} The labeled sentence:
Table 17: The prompt for LLM-revision on GENIA.
LLM-revision Prompt (BC5CDR)
[Task Description] Within the BC5CDR dataset consisting of PubMed articles, the sentences are tagged with labels. The task is to verify whether the word is properly tagged with the correct label from the given sentence and revise it. If the entity is annotated incorrectly, you may revise tags. Here’s what each label means: <C> is for ‘Chemical’, <D> is for ‘Disease’. If the entity corresponds to Disease, <D> should be added before the entity and </D> after it. Follow the format of Example 1 to answer Example 2.
[Example 1] The given sentence: Recurarization in the recovery room . A case of recurarization in the recovery room is reported . Accumulation of <C>atracurium</C> in the intravenous line led to recurarization after flushing the line in the recovery room . A <D>respiratory arrest</D> with severe desaturation and bradycardia occurred . Circumstances leading to this event and the mechanisms enabling a neuromuscular blockade to occur , following the administration of a small dose of relaxant , are discussed . The labeled sentence: Recurarization in the recovery room . A case of recurarization in the recovery room is reported . Accumulation of atracurium in the intravenous line led to recurarization after flushing the line in the recovery room . A respiratory arrest with severe desaturation and bradycardia occurred . Circumstances leading to this event and the mechanisms enabling a <D>neuromuscular blockade</D> to occur , following the administration of a small dose of relaxant , are discussed .
[Example 2] The given sentence: {context} The labeled sentence:
Table 18: The prompt for LLM-revision on BC5CDR.

heading: Future Work
text: Considering the feasibility of VERIFINER as a model-agnostic approach and its demonstrated generalization capabilities, we expect our work will enhance reliability in NER and be applicable to other domains in future research. Limitation
Despite the remarkable performance of VERIFINER, there is still room for improvements on our framework. First of all, we conduct experiments using gpt-3. 5-turbo-1106 model from OpenAI for LLM. Considering the significance of LLM as a major component in the framework, it is yet to be discovered whether the effectiveness of VERIFINER is valid with other LLMs. Thus we leave application of our framework on different open and closed LLMs for future work. In this work, we confine the scope of our work to biomedical domain. However, as a plug-andplay framework that can be applied to any model, VERIFINER has a potential on other knowledge-
intensive domains such as legal or scientific domains. In the future, we plan to investigate its application on other domains. Lastly, our framework necessitates multiple inferences using LLMs, which can be computationally expensive. This aspect becomes particularly significant when integrating our framework into applications that require real-time inference. We believe this limitation can be resolved by using smaller models which are distilled from LLMs. Ethical Consideration
The main aspect of our work with the potential for ethical pitfalls is the use of LLMs for our framework. Recent work has highlighted the risks of LLMs in hallucination (Zhang et al. , 2023). This problem might be more critical in biomedical NER, where prediction can be used for clinical decision support, drug discovery, and personalized medicine. Inaccurate or hallucinated information could lead to misdiagnoses, inappropriate treatment recommendations, or erroneous scientific research directions, ultimately posing a significant risk to patient safety and public health. We argue here that this risk is largely mitigated in our work, mainly due to our verification process that incorporates knowledge from KBs. The goal of our work is reducing such errors with verification, specifically by integrating structured data and validated information from KBs. This integration allows for cross-referencing and validation of the outputs generated by the LLM, ensuring that any identified entities and associated information align with reliable biomedical knowledge