Natural Language Inference Fine-tuning for Scientific Hallucination Detection (2025)

Abstract: Modern generative Large Language Models (LLMs) are capable of generating text that sounds coherent and convincing, but are also prone to producing hallucinations , facts that contradict the world knowledge. Even in the case of Retrieval-Augmented Generation (RAG) systems, where relevant context is first retrieved and passed in the input, the generated facts can contradict or not be verifiable by the provided references. This has motivated SciHal 2025, a shared task that focuses on the detection of hallucinations for scientific content. The two subtasks focused on: (1) predicting whether a claim from a generated LLM answer is entailed, contradicted, or unverifiable by the used references; (2) predicting a fine-grained category of erroneous claims. Our best performing approach used an ensemble of fine-tuned encoder-only ModernBERT and DeBERTa-v3 models for classification. Out of nine competing teams, our approach achieved the first place in sub-task 1 and the second place in sub-task 2.

## introduction the increasing availability of academic research assistants based on large language models (llms) have revolutionized the way research is conducted, enabling users to pose research-related questions in natural language and receive structured and concise summaries supported by relevant references @xcite . these systems have the potential to greatly accelerate the research process, facilitating the discovery of new knowledge and insights @xcite . however, the tendency of llms to introduce hallucinations -claims that are not supported or grounded in relevant evidence or established world knowledge -poses a significant challenge to the reliability of these automatically generated scientific answers @xcite . hallucinations can lead to the dissemination of misinforma-tion, undermining the validity of research findings and the trustworthiness of ai-powered research tools @xcite . to address this issue, the scihal shared task was established, focusing on the detection of hallucinated claims in answers generated by ai-powered research assistants. the task provides a dataset of research-oriented questions, the corresponding answers and references, annotated with labels indicating the presence and type of hallucinations. by developing systems that can accurately detect hallucinations, researchers can take a crucial step towards ensuring the reliability and trustworthiness of ai-enhanced research assistants. in response to this challenge, we developed an approach using an ensemble of fine-tuned encoderonly models deberta-v3 and modernbert. this approach achieved the first place on sub-task 1. this paper describes our model architecture, training procedure, and results on the shared task. the performance of our approach on the task demonstrates the potential of machine learning models to identify hallucinations and improve the accuracy of generated answers. we outline our findings, challenges, and directions for future improvements. ## related work hallucinations in llms refer to the generation of fluent but factually incorrect or inconsistent claims @xcite @xcite @xcite . factual hallucinations are outputs that deviate from real-world facts and can be addressed through factchecking, which verifies the accuracy of claims @xcite . manual fact checking is labor intensive and time consuming @xcite , prompting research into automated approaches. these approaches typically involve broad classifications (e.g., supported, refuted, not enough infor-mation), limiting their applicability in real-world scenarios @xcite . to improve utility, finer-grained classification schemes have been proposed, reflecting degrees of truthfulness @xcite . some methods retain original fact-checking labels @xcite , while others consolidate categories for simplicity @xcite @xcite . typically, scientific text classification is conducted in a supervised manner @xcite @xcite , while some approaches support scenarios where labeled training data is scarce @xcite @xcite . final claim veracity prediction is often modeled as a natural language inference (nli) task, where a relation between a premise and a hypothesis (entailment, contradiction, neutral) must be predicted @xcite . this paper investigates two sub-tasks: one using coarse-grained labels and another with finer-grained classifications to assess whether an llm generated claim is a hallucination, given reference evidence. ## task description the scihal 2025 shared task addresses the critical challenge of factual inconsistency in responses generated by generative ai-powered academic research assistants. scihal formulates this problem as a classification task, focused on evaluating the factual alignment between individual claims and their supporting evidence. given a researchfocused question, an llm generated response from a retrieval-augmented generation (rag) system, an extracted claim from the response, and a reference retrieved from a large corpus of scientific literature that is used to ground the generated response, the objective is to classify the claim based on its factual consistency with the provided reference. scihal 2025 is structured into two sub-tasks: sub-task 1 involves coarse-grained classification of each claim into one of three categories: entailment, unverifiable, or contradiction. sub-task 2 extends this formulation by employing a fine-grained label set. each claim must be categorized as one of the following: entailment, unrelated and unverifiable, related but unverifiable, misrepresentation, missing information, numeric error, entity error, or opposite meaning. ## dataset the scihal dataset comprises labeled claims designed to evaluate hallucination detection in scientific assistant outputs. the data creation process involves both real and synthetic components, ensuring a diverse and balanced distribution of hallucination types. data collection over 50,000 real-user queries were collected from a live academic assistant system over a week. these questions focused on the five scientific fields engineering, environmental science, medicine, agricultural and biological sciences, and computer science. after deidentification and refinement, 500 questions were retained. for each question, a rag system indexed over a million scientific abstracts to retrieve the top 20 most relevant documents. the system then generated an answer, from which individual claims were extracted. each claim was paired with the retrieved references used to justify the answer. ## approaches to identify hallucinated claims, we explore a range of approaches spanning zero-shot prompting and supervised fine-tuning, leveraging both encoderonly and decoder-only models. deepseek-r1 zero-shot we use the deepseek-r1 model @xcite in a zero-shot setting to classify claims into predefined categories using the associated reference as supporting evidence. the prompt includes a task definition and detailed descriptions of each classification label. the full prompt is provided in figure 1 . deepseek-r1 zero-shot with claim decomposition building on the basic zero-shot setup, we extend the prompting strategy by explicitly instructing deepseek-r1 to first decompose the claim into its constituent subclaims. the model then classifies each subclaim individually and aggregates the results into a final prediction for the full claim. this decomposition aims to enhance reasoning granularity. the corresponding prompt is in figure 2 . ## evaluation the primary evaluation metric for the shared task is the weighted f 1 score. it is computed by calculating the f 1 score independently for each class and then taking the average, weighted by the number of true instances (support) for each class. each sub-task's test set comprises 1,000 examples, with 50% designated for official evaluation and leaderboard ranking during the challenge. the remaining 50% is withheld and only evaluated after the competition concludes. accordingly, all reported results in this paper are based on the publicly accessible 50% split of the respective test sets. as shown in ## discussion our findings show that the dataset poses a considerable challenge and that fine-tuned models clearly outperform prompting-based approaches. notably, the smaller encoder-only deberta-v3 and mod-ernbert models achieve better results than much larger decoder-only llms. despite their scale, llms such as deepseek-r1 and gpt-4o struggle in prompting setups compared to fine-tuned mod-ernbert and deberta-v3 variants. interestingly, advanced prompting techniques, such as claim decomposition, do not improve classification performance. in fact, they often underperform compared to simpler zero-shot prompting. to understand this behavior, we perform a detailed analysis of both the dataset and the prediction behaviors of the model. we observe that the test sets are inherently difficult due to the way they were constructed: they include only those instances where initial predictions by smes and llms diverged. these disagreements were later resolved by a third sme. however, the data annotations remain often ambiguous, inconsistent, and challenging. during our manual inspection, we identified multiple very similar instances with different labels. inconsistent labels were particularly common in examples annotated as unverifiable (unver) or contradiction (contra). for instance, claims that involved information not present in the reference were sometimes labeled 'contra' and other times 'unver', even when the annotation justification was nearly identical. prompt-based approaches are particularly affected by this inconsistency. given that prompts contain fixed class definitions, the models tend to adhere to those instructions. for instance, when claim content is missing from the reference, llms frequently predict 'unver', aligning with the prompt's class description, although the example is labeled as 'contra'. we also identified inconsistencies in the annotation of entailment (entail) cases. some instances were labeled as 'entail' only when the claim's content was explicitly stated in the reference, while others were labeled 'entail' even when the reference only implicitly supported the claim through inference. however, the instructions provided in the prompt resulted in the llm to rely strictly on explicit information and often misclassified such implicit entailment examples as 'unver'. internal validation supports these observations: all prompting-based approaches demonstrated particularly low precision for the 'unver' class. contrary to our expectations, decomposing claims into subclaims did not improve performance. in fact, this led to overly conservative predictions. for example, the model would identify one unsupported detail within a claim and classify the entire example accordingly, even when the overall meaning was supported. the annotators, by contrast, appeared to take a more holistic view, labeling a claim as entailment based on general alignment, even when minor details were not mentioned. overall, these findings suggest that promptingbased methods lack the flexibility required to handle the annotation noise and implicit reasoning present in the dataset. in contrast, fine-tuned models can better adapt to such irregularities, likely because they learn implicit patterns and labeling conventions from the training data. finally, the strong performance of smaller encoder-only models highlights the importance of task-specific training. the modernbert and deberta-v3 models were already trained on a diverse set of nli datasets, whereas the llama3.1-8b-instruct model was not. this likely gave the smaller models a major advantage, suggesting that task-specific training on relevant datasets can outweigh model scale for downstream performance. ## future work in future work, we aim to further improve the finetuning process of decoder-only language models considering their vast world knowledge and reasoning capabilities. given that we achieved the best result using an ensemble, we additionally aim to experiment more with advanced ensembles and committee voting techniques, including the introduction of weighting mechanisms. finally, we plan to incorporate hierarchical classification in the form of multi-step predictions for sub-task 2 involving fine-grained labels. ## conclusion we presented fine-tuning approaches based on modernbert and deberta-v3 that consistently outperformed baseline methods and other submitted solutions. this success is largely attributable to prior training on extensive nli datasets, which closely align with the nature of the target tasks. notably, the same approach demonstrates strong performance on both sub-task 1 and sub-task 2, underscoring its generalizability across related tasks. our findings further suggest that in scenarios where the data is inherently challenging-due to ambiguity or inconsistent labeling, fine-tuning offers a clear advantage over prompt-based llm approaches. while prompting yields consistent predictions based on static label definitions, it lacks the flexibility to adapt to subtle patterns and inconsistencies in the data. in contrast, fine-tuned models are better able to internalize such nuances. moreover, our results highlight the importance of training on data that closely resembles the target task. models exposed to large volumes of relevant data prior to task-specific fine-tuning consistently achieve superior downstream performance. notably, this results in smaller models using this strategy outperforming larger models that lack similar task-aligned training.