From RAG to Reality: Coarse-Grained Hallucination Detection via NLI Fine-Tuning (2025)

Abstract: We present our submission to SciHal Subtask 1: coarse-grained hallucination detection for scientific question answering. We frame hallucination detection as an NLI-style three-way classification (entailment, contradiction, unverifiable) and show that simple fine-tuning of NLI-adapted encoder models on task data outperforms more elaborate feature-based pipelines and large language model prompting. In particular, DeBERTa-V3-large, a model pretrained on five diverse NLI corpora, achieves the highest weighted F1 on the public leaderboard. We additionally explore a pipeline combining joint claim–reference embeddings and NLI softmax probabilities fed into a classifier, but find its performance consistently below direct encoder fine-tuning. Our findings demonstrate that, for reference-grounded hallucination detection, targeted encoder fine-tuning remains the most accurate and efficient approach.

## introduction generative ai assistants are increasingly utilized to produce reference-based answers in scientific and research contexts, particularly via retrievalaugmented generation (rag) systems that combine large language models with external knowledge sources. while rag can greatly improve factual coverage, it also introduces a critical problem: hallucinations, wherein the model generates claims that are unsupported or directly contradicted by the cited references. detecting such hallucinations is essential for trustworthy scientific communication, yet remains a major challenge for evaluation pipelines. the scihal shared task on hallucination detection for scientific content @xcite , organized at workshop on scholarly document processing at acl 2025, formalizes this problem as a multi-label classification task. given a research question, a model-generated summary, and an extracted claim, participants must determine whether each claim is entailment, contradiction, or unverifiable with respect to the provided reference abstracts (subtask 1: coarse-grained detection). in this paper, we report our approach that was ranked fourth and share our experiments for sci-hal subtask 1. we explore three families of approaches: our analysis shows that among the methods listed above, straightforward fine-tuning of an nliadapted encoder yields the best performance on the task test dataset. we conclude that, for coarsegrained hallucination detection, simpler encoderonly architectures might be an efficient choice. ## related work automatic verification of factual consistency has attracted intense attention in recent years, reflecting a surge of methods and datasets devoted to achieving this goal @xcite . within the scientific domain, this research tradition is grounded in claimverification datasets such as scifact @xcite and follow-up shared tasks like sciver @xcite . the naacl sciver scaled verification to a 5 m-abstract corpus, systems had to retrieve evidence and assign support or refute labels. all top-3 teams combined sparse retrieval with domain specific bert-family encoders like scibert @xcite , biobert @xcite and roberta @xcite . recent developments (mor-lan and levi, 2024) @xcite show that nli crossencoders demonstrate the ability to discern factual claims from non-factual ones given evidence. nli is the task of determining whether a "hypothesis" text can be inferred (entailment), contradicted (contradiction), or is undetermined (neutral) from a "premise" text. nli cross-encoders (mor-lan and levi, 2024) are a natural fit for scientific claim verification because their label space: entailment, contradiction, and neutral is isomorphic to the support, refute, unverifiable taxonomy used in scifact, sciver and scihal subtask 1, so no label remapping is required. ## data the dataset is a claim-level annotated benchmark designed to measure factual correctness and hallucination detection quality in scientific rag systems. data was originally sourced from logs of a scientific research assistant tool. 50,000 samples were collected over one week. organizers used an llm to classify user questions by domain, ensuring completeness and correctness, and filtered out non-english texts. given the feasibility of employing subject matter experts (smes) for annotation, the dataset authors included texts from multiple domains: engineering, environmental science, medicine, agricultural and biological sciences, and computer science. all questions were rewritten using an llm, after which human annotators removed confidential or commercial information. this process yielded a refined dataset of 500 samples. organizers then used a rag scientific research assistant to retrieve 20 relevant article abstracts for each question. finally, they generated answers and extracted claims with corresponding references. to balance class distribution, the authors prompted an llm to synthetically falsify claims. they modified 75% of samples by corrupting claims according to predefined fallacy types (for sub-task 2), while maintaining class balance. the resulting dataset was manually verified and anno-tated by smes. this methodology ensured only 25% of samples were marked as entailment, with other classes each representing less than 10% of samples. synthetic corruption also reduced manual annotation costs. in the first annotation round, one sme validated samples within their domain, establishing baseline human annotations. samples where both the llm and sme agreed formed batches 1 and 2. the remaining samples were annotated by a second sme. to achieve consensus, a third sme made final label decisions after reviewing justifications from prior annotators. this process improved the quality and challenge level of batch 3 and the test set. the resulting batch 2 (that also includes batch 1 data), batch 3 and test set contain 2092, 1500 and 1000 samples accordingly. each record contains: the original question, the ai-generated answer, one or more claims (extracted from the answer), one or more references (article abstracts from the rag tool), a label (for training sets only; three-class scheme: entail, unver, contra) and justification (sme reasoning for the label; training sets only). ## experiments we approach this task as a pure classification problem, namely we view it as a nli task. all three nli labels are identical in their sense to the labels provided in the training data of this task. given limited time and resources, we train three groups of models. the choice of bert-like models fine-tuned for the nli task is intuitive, since the labels we need to predict are directly used in nli. to test a more complicated pipeline, we train a classification model on features extracted from the data: cosine similarity scores for the embedded claims and references and probabilities for nli classes. the third group is llm-based: we prompt qwen3-8b foot_0 to predict the labels with two examples for each class taken from the training set. the resulting scores can be found in ## error analysis we conduct a qualitative error analysis on a development set comprising 360 samples (10% of the training data). we also evaluated the catboost-based pipeline with similarity and nli features, which demonstrated notably lower performance (weighted f1: 0.57 on the development set), primarily due to increased confusion across classes, particularly between contradiction and entailment. additionally, we assessed the qwen model, which exhibited a significantly higher error rate (43.4%) on a similar development set. the qwen model predominantly confused contradiction with entailment and vice versa, highlighting fundamental issues in distinguishing these classes effectively. this suggests that the qwen model requires further adaptation or training enhancements to reliably detect textual support and contradiction in scientific contexts. some classification examples can be found in ## results the resulting weighted f1 scores on the public leaderboard are shown in by contrast, more elaborate pipelines that extract features via embeddings and nli probability outputs incur substantial computational overhead and still underperform compared to direct crossencoder fine-tuning. similarly, in-context learning with llm that we evaluated, while a popular choice, is markedly more expensive to run and achieves lower f1 scores than the smaller, specialized encoder models. these results resurface the importance of taskspecific transfer learning and highlight the role of training data quality over multi-domain generalization abilities of a model. you are a scientific claim validator for the scihal task. given (1) question, (2) claim, (3) references (abstracts) decide which label applies following the decision tree: ## conclusion we have addressed the challenging problem of coarse-grained hallucination detection in scientific question answering by framing it as a three-way nli task (entailment, contradiction, unverifiable) on retrieved reference abstracts. we evaluated a range of methods-from simple encoder finetuning (scibert, modernbert, and deberta-v3) to more elaborate feature-based pipelines combining joint claim-reference embeddings and nli softmax scores, as well as few-shot prompting of large language models. our experiments demonstrate that direct fine-tuning of an nli-adapted cross-encoder, and in particular deberta-v3large, offers competitive accuracy in solving the hallucination detection task, achieving fourth place on the public leaderboard for scihal subtask 1. the modest performance of unsupervised embedding clustering and the underwhelming results of more complex pipelines underscore the inherent difficulty of reliably detecting scientific hallucinations without explicit supervision. our findings reaffirm that, despite the task's complexity, targeted cross-encoder fine-tuning remains an effective strategy for reference-grounded hallucination detection.