Lightweight Query Checkpoint: Classifying Faulty User Queries to Mitigate Hallucinations in Large Language Model Question Answering (2025)

Abstract: Question Answering (QA) with large language models has shown impressive performance, yet hallucinations still persist, particularly when user queries carry incorrect premises, insufficient context, or linguistic ambiguity. To address this issue, we propose Lightweight Query Checkpoint (LQC), a small classification model that detects verification-required queries before the LLM generates a potentially faulty answer. LQC leverages hidden states extracted from intermediate layers of a smaller-scale, non-instruct-tuned LLM to effectively distinguish queries requiring verification from clear queries. We first systematically define categories of queries that need verification, construct a dataset comprising both defective and clear queries, and train a binary contrastive learning model. Through extensive experiments on various QA datasets, we demonstrate that incorporating LQC into QA pipelines reduces hallucinations while preserving strong answer quality.

## introduction recently, as the performance of large language models (llm) has steadily improved, user experience in qa systems has also been greatly enhanced @xcite @xcite . through simple interfaces such as web browsers or chatbots, users can easily input queries and receive immediate answers, providing a more intuitive user experience. however, in real-world usage, questions containing incorrect premises, insufficient context, or linguistic ambiguities are often entered @xcite @xcite , which still poses a high risk of the model producing incorrect answers-so-called "hallucinations." for instance, as shown in figure 1 , a question asking about the legal age to buy alcohol, such as ## methodology the primary goal of this study is to enable an opendomain, llm-based question-answering system to recognize the "verification needed" when a user query contains flaws, explaining this need to the user, while still delivering high-quality answers for queries that do not contain any flaws. to achieve this, we propose an llm-based classifier called lightweight query checkpoint (lqc), which determines in advance whether the user query requires verification before the llm generates an answer. the overall pipeline of the llm-based qa system incorporating lqc is shown in figure in which different answer templates are generated according to lqc's classification results (whether verification is needed). in this chapter, we examine each of these steps in detail. ## experiments in this chapter, we evaluate the proposed lqc framework, which classifies queries and then generates responses, using various datasets and baseline methods. ## analysis in this section, we conduct an in-depth analysis of lqc, a classifier that learns from the hidden states of an llm in order to determine whether a user's query requires verification. unless otherwise specified, the settings are the same as in the main exper-model w/ cl w/o cl acc. f1 acc. f1 llama-3.2-1b 91.3 91.3 89.1 89.7 qwen2.5-0.5b 88.7 89.1 85.5 87.2 ## conclusion in this study, we propose a query-verification model called lqc to prevent hallucination issues in llms that can arise from user's incorrect queries, and we present a complete pipeline that integrates lqc into a qa system. lqc extracts the hidden states from intermediate transformer layers using a newly proposed "hybrid pooling" technique, then explicitly notifies users of queries that require verification-demonstrating superior performance compared to other baseline. furthermore, the model can be easily trained by extracting hidden states from a lightweight llm and employing a simple mlp-based binary classification approach. above all, our results show that it is possible to clearly identify and address defects in user queries without modifying the llm's weights or input prompts in any way. this lightweight approach functions as an initial safeguard that complements the input and output stages of llms, helping prevent the generation of incorrect information in qa systems and ultimately enhancing user satisfaction.