SIFT : Grounding LLM Reasoning in Contexts via Stickers (2025)

Abstract: This paper identifies that misinterpreting the context can be a significant issue during the reasoning process of large language models, spanning from smaller models like Llama3.2-3B-Instruct to cutting-edge ones like DeepSeek-R1. We introduce a novel, post-training approach called **Stick to the Facts (SIFT)** to tackle this. SIFT leverages increasing inference-time compute to ground LLM reasoning in contexts. At the core of SIFT lies the Sticker, which is generated by the model itself to explicitly emphasize the key information within the context. Given the Sticker, SIFT generates two predictions—one from the Sticker alone and one from the query augmented with the Sticker. If they differ, the Sticker is sequentially refined via forward optimization (to better align the extracted facts with the query) and inverse generation (to conform with the model’s inherent tendencies) for more faithful reasoning outcomes. Studies across diverse models (from 3B to 100B+) and benchmarks (e.g., MATH, AIME) reveal consistent performance improvements. Notably, SIFT improves the pass@1 accuracy of DeepSeek-R1 on AIME2024 from 78.33% to **85.67%** and that on AIME2025 from 69.8% to **77.33%**. Code will be public after acceptance.

## introduction recent advancements in large language models (llms) @xcite @xcite have significantly advanced the field of natural language processing. techniques including chain-of-thought (cot) prompting @xcite and self-consistency @xcite , as well as reasoning-enhanced models, e.g., openai-o1 @xcite , @xcite , and kimi-k1.5 @xcite , have all query josh decides to try flipping a house. he buys a house for @xmath050,000 in repairs. this increased the value of the house by 150%. how much profit did he make? sticker conditions: 1. josh buys a house for @xmath150,000 on repairs. ## related work reasoning has long been a significant challenge for llms. several approaches aim to improve the reasoning capabilities of llms. these methods can be broadly categorized into training-based alignment, search and planning enhancement, and inference-time augmentation. some approaches focus on aligning the reasoning path of llms through supervised fine-tuning (sft) or reinforcement learning (rl). star @xcite enables the model to use reject sampling and learn from its mistakes by rationalizing its outputs, progressively enhancing its reasoning capabilities. quiet-star @xcite generates multiple rationales in parallel before each output token, thereby improving the model's ability to predict subsequent tokens. v-star @xcite employs a dual-system framework where the generator creates preference pairs to train the verifier, which then scores the candidate solutions. additionally, a significant body of work aims to enhance model reasoning abilities through search and planning. q* @xcite formalizes multi-step reasoning as a markov decision process (mdp) and uses the a* algorithm to guide the model in selecting the optimal next step. rstar @xcite employs monte carlo tree search (mcts) to enhance the model's reasoning exploration and uses mutual verification to evaluate the reasoning paths. sr-mcts @xcite combines self-refinement and mcts to iteratively improve and optimize newly discovered reasoning paths. mcts-dpo @xcite leverages mcts to collect step-level preference data and uses decision-policy optimization (dpo) to refine the model's policy through multiple iterations. rest-mcts* @xcite takes a broader approach in evaluating reasoning paths, considering not only the correctness of the results but also the quality of the reasoning process, such as the shortest path and error-free intermediate steps. core @xcite constructs a dual-system approach with system 1 for generation and system 2 for verification, training, and reasoning simultaneously to simulate human-like reasoning processes. alphamath @xcite treats the output of the llm as an action and integrates a value model and a policy model, iteratively training the model to enhance its reasoning capabilities. there are also methods that focus on enhancing reasoning abilities during inference. innovations in prompt engineering have contributed to advancements in reasoning capabilities. chain-of-thought (cot) prompting @xcite guides models in stepwise reasoning, such as by manually annotating natural language rationales or appending "let's think step by step" after questions. auto-cot @xcite clusters questions and uses zero-shot chain-of-thought to generate reasoning chains, which are then used as prompts to guide the model's answers. tot @xcite removes the constraints of chain structures by incorporating tree structures and search algorithms, allowing models to explore widely during reasoning. the seminal self-consistency method @xcite aggregates answers through majority voting over multiple reasoning paths, while @xcite introduces iterative self-correction via feedback loops. however, these methods focus on refining how models reason rather than ensuring they address the correct problem. our approach differs by prioritizing factual comprehension to ensure proper problem understanding before answer generation. ## method we first presents the factual drift issue during llm reasoning and then elaborates on the proposed stick to the facts (sift) approach. find more discussion on the definition of sticker in appendix a. ## experiments this section first validates the effectiveness and generalization of sift (section 4.1). next, we explore several variants (section 4.2 & 4.3). finally, we include ablation studies to gain further insights into our approach (section 4.4 and appendix d). ## conclusion this study presents stick to the facts (sift), a training-free framework that grounds llm reasoning in contextual facts through iterative selfrefinement. our approach enhances reasoning reliability without requiring extra data or training.