Title: When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: LLM Abstention, Temporal and non-temporal reasoning, Question answering

Abstract: Large language models (LLMs) rarely admit uncertainty, often producing fluent but misleading answers. This weakness is even evident in temporal question answering (QA), where models frequently ignore time-sensitive evidence and conflate facts across different periods. In this paper, we present the first empirical study of reinforcement learning (RL) at the intersection of abstention and reasoning ability in LLMs for temporal QA. Existing approaches such as calibration might be unreliable in capturing uncertainty in complex reasoning. We instead frame abstention as a teachable skill and introduce a pipeline that couples Chain-of-Thought supervision with RL guided by abstention-aware rewards. Our novel method yields strong empirical gains on reasoning: a model initialized by Qwen2.5-1.5B-Instruct surpasses GPT-4o by 3.46\% and 5.80\% in Exact Match on TimeQA-Easy and Hard, respectively. Moreover, it improves the True Positive rate on unanswerable questions by 20\% over a pure supervised fine-tuned (SFT) variant. In addition, further analysis shows that SFT induces overconfidence and harms reliability, while RL improves overall accuracy but carries similar risks.  Our study provides new insights into how abstention and reasoning can be jointly optimized, offering a foundation for building more dependable language models.

TL;DR: We introduce a new model based on Qwen2.5-1.5B-Instruct that outperforms GPT-4o on selective abstention and reasoning on temporal questions.