Detecting Hallucinations in Scientific Claims by Combining Prompting Strategies and Internal State Classification (2025)

Abstract: Large Language Model (LLM)–based research assistant tools demonstrate impressive capabilities, yet their outputs may contain hallucinations that compromise reliability. Therefore, detecting hallucinations in automatically generated scientific content is essential. SciHal2025: Hallucination Detection for Scientific Content challenge @ ACL 2025 provides a valuable platform for advancing this goal. This paper presents our solution to the SciHal2025 challenge. Our method combines several prompting strategies with the fine-tuned base LLMs. We first benchmark multiple LLMs on the SciHal dataset. Next, we developed a detection pipeline that integrates few-shot and chain-of-thought prompting. Hidden representations extracted from the LLMs serve as features for an auxiliary classifier, further improving accuracy. Finally, we fine-tuned the selected base LLMs to enhance end-to-end performance. In this paper, we present comprehensive experimental results and discuss the implications of our findings for future hallucination detection research for scientific content.

## introduction large language models (llms) have demonstrated remarkable capabilities in generating scientific content across various domains @xcite @xcite . llm-powered research assistant tools further streamline scholarly workflows by answering research-related questions and output structured, concise responses. however, hallucinations may be introduced by llms pose a significant challenge to fully trusting these automatically generated scientific outputs @xcite . consequently, detecting hallucination content from the llm-powered system is essential for their safe deployment. hallucination detection for scientific content challenge (scihal 2025) @ acl 2025 provides a rigorous test platform for this problem @xcite . the dataset contains real-user questions, retrieved scientific abstracts, llm-generated responses, and extracted claims from responses with human annotation. the goal is to classify each claim based on the provided reference abstracts into different hallucination types. this paper describes our technical solution for the scihal challenge. our solution integrates prompting techniques with llms and leverages the models' internal representations for classification. we begin by benchmarking multiple llms on the scihal dataset under zero-shot prompting to gauge their out-of-thebox performance. subsequently, we develop a detection pipeline by combining domain-specific few-shot examples with chain-of-thought (cot) prompting @xcite . specifically, we first classify each data point into its respective domain, then pair it with corresponding domain-aware fewshot examples to construct refined cot prompts. then, the hidden states produced by the llm serve as features for training a classifier, enhancing predictive accuracy. on the evaluation set, our approach achieves f1 scores of 0.59 on subtask 1 and 0.51 on subtask 2, as reported on the leaderboard. a detailed performance analysis is provided in section 4. ## methodology in this section, we outline the proposed pipeline for hallucination detection in scientific claims (see in figure 1 ). we first assign claims to their domain and select a few corresponding examples. by leveraging the in-context learning (icl) capacity of llms @xcite @xcite dong et al., 2024), we then construct the detection pipeline that utilizes llm's hidden states as features to train the classifier. ## experiments and results in this section, we present a detailed analysis of our experimental results and discussion. specific details regarding experiment setup and configurations are provided in the appendix b. ## conclusion in this paper, we present our solution to the sci-hal 2025 challenge. by integrating domain-aware few-shot and cot prompt, and the model's hidden state as the feature, our method achieved promising results. due to time constraints, additional experiments are ongoing and will be reported later.