ALW : Adaptive Layer-Wise contrastive decoding enhancing reasoning ability in Large Language Models (2025)

Abstract: Large language models (LLMs) have achieved remarkable performance across various reasoning tasks. However, many LLMs still encounter challenges in reasoning, especially for LLMs with fewer parameters or insufficient pre-training data. Through our experiments, we identify that noise accumulation across layers often leads to unstable token predictions during reasoning. We find that contrasting the probability distributions across layers effectively mitigates this interference. Building on this insight, we propose Adaptive Layer-Wise contrastive decoding (ALW), a novel framework that enhances reasoning ability by dynamically disentangling noise in shallow layers from critical signals in deep layers. Extensive experiments on several reasoning benchmarks demonstrate that ALW consistently improves answer accuracy across multiple LLMs while maintaining inference efficiency. For example, we achieve a 48% improvement on the Gsm8k using the LLaMA-7B model and an absolute accuracy increase of 5.2 points on the BBH evaluation benchmark with the LLaMA-65B model.

## introduction recent years have witnessed significant advancements in reasoning tasks @xcite due to the development and deployment of large language models (llms) @xcite @xcite . however, not all language models (lms) possess strong reasoning abilities. they encounter a variety of reasoning issues, such as arithmetic mistakes and semantic errors @xcite . existing research enhances reasoning capabilities by linking to external knowledge and retrieving reasoning evidence @xcite . recent works find that contrast decoding of two different models can improve text quality in open-ended text generation @xcite and have great potential in reasoning tasks (o @xcite . @xcite further achieve performance improvement on open-ended generation tasks through self-contrast decoding but with limited gains on reasoning tasks. in view of this, we start from recent observations on noise signals within @xcite , in which llms assign disproportionately high scores to noncritical tokens, and directly subtracting the attention distributions of different attention heads can effectively diminish the influence of noisy irrelevant tokens to the generation results. this insight leads us to question whether the noise from overlyattended irrelevant tokens accumulates across different layers and results in diverse reasoning errors observed from the final layer, e.g., repetitive generation. figure 1 preliminarily confirms that subtracting the prediction distribution of the shallow layer from that of the final layer can weaken noisy context and produce the correct answer "no". therefore, we mitigate noise at the layer level and propose adaptive layer-wise contrastive decoding, a framework designed to enhance reasoning capabilities by explicitly disentangling shallow noise from deep reasoning. to achieve this, we develop a quantification strategy to determine the optimal early exit layer for noise suppression. furthermore, we introduce a lightweight lm that learns which layer of the llm requires noise suppression in an end-to-end manner and dynamically adjusts the contrastive strategy during the inference stage. additionally, we extend alw by developing a tuningfree method based on semantic similarity, reducing resource consumption. specifically, we first identify the optimal noise layer that requires the most suppression for different inputs and tasks. then, a predictor, either lmbased or similarity-based, is employed to capture the mapping between the prompt and the optimal contrastive layer. during inference, the prompt is processed by both the llm and the predictor. the llm contrasts the final layer with the early exit layer selected by the predictor, effectively filtering noise from shallow layers while preserving key signals in the final layer. extensive experiments demonstrate that alw achieves stable performance improvements and significantly enhances the reasoning abilities of llms. our contributions are as follows: ## conclusion in this paper, we start from the assumptions of attention noise and explore the noise across different layers of llms. we observe that noise accumula-tion across layers often leads to unstable token predictions during reasoning. to address this issue, we propose adaptive layer-wise contrastive decoding, a search-based decoding approach that enhances reasoning ability by dynamically disentangling noise in shallow layers from critical signals in deep layers. we design a quantification strategy and optimal layer selection methods based on lms or similarity. the experimental results demonstrate that alw significantly enhances the reasoning capabilities of llms, and further analysis reveals that alw improves reasoning by focusing on the correct reasoning path through noise suppression.