Efficient Multimodal Spatial Reasoning via Dynamic and Asymmetric Routing
Authors: Yixian Shen
Qi Bi
Zihan Wang
Zhiheng Yang
Changshuo Wang
Zhi Zhang
Prayag Tiwari
Andy D. Pimentel
Anuj Pathania
Venue: ICLR 2026 Poster (2026)
Keywords: Autoregressive multimodal reasoning
Layer- and hop-aware retention
Green AI
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce DARE, a novel framework for efficient multimodal reasoning that leverages visual-to-text information flow and asymmetric routing, cutting FLOPs and KV-cache by over 40% without loss of accuracy.

Abstract: Recently, visualization-of-thought (VoT) has unlocked new opportunities for complex spatial reasoning in multimodal large language models (MLLMs) by complementing verbal reasoning with visual thinking.
However, the autoregressive accumulation of lengthy and redundant tokens substantially increases computation and memory costs.
In this paper, we present a new efficient framework for multimodal spatial reasoning, named *DARE*, designed to adaptively prune multimodal tokens across different network depths, reasoning hops, and modalities. 
First, *DARE* devises an intra- and inter-hop-aware differentiable retention mechanism to dynamically estimate token importance both within each reasoning step and across successive hops. 
Recognizing that deeper network layers encode visual cues into verbal streams, *DARE* introduces an asymmetric compression strategy that prunes tokens according to modality-specific redundancy and semantic importance.
Furthermore, *DARE* incorporates a progressive KV-cache retention policy aligned with cross-modal fusion dynamics, further reducing memory overhead during autoregressive reasoning.
Our method delivers substantial reductions in computation and memory footprint, averaging a 40.37\% reduction in FLOPs and 46.07\% reduction in KV caches usage, 
while consistently preserving or even improving reasoning performance across seven multimodal spatial reasoning benchmarks, and further generalizing to broader multimodal reasoning tasks, 
establishing a scalable and robust recipe for efficient multimodal reasoning.

Published as a conference paper at ICLR 2026
EFFICIENT MULTIMODAL SPATIAL REASONING VIA DY- NAMIC AND ASYMMETRIC ROUTING
Yixian Shen' Qi Bi Zihan Wang Zhiheng Yang' Prayag Tiwaris Andy D. Pimentel
Changshuo Wang? Zhi Zhang'
Anuj Pathania
'University of Amsterdam
(y.shen, q.bi, 2. yang, zhang, a.d. a.d.pimentel a. a.pahaniaa auva.n!
sihanwang.sddbe smaugmaaaaaa com
University College London
Halmstad University
wangchangshuolegman com
prayag. tiwrriieeerriir.rgggg
ABSTRACT
Assulization-of-thooee (VoT) has unlocked new capabilities for multimodal large language models (MLLMs) in complex spatial reasoning by integrating visual thinking into verbal reasoning. However, autoregressive reasoning over lengthy multimodal tokens introduces significant computation and memory overhead. We propose DARE, an efficient framework that adaptively prunes tokens across network depths, reasoning hops, and modalities. DARE employs a differentiable intra- and inter-hop retention mechanism to estimate token importance throughout reasoning, alongside asymmetric modality-awan compression strategy that accounts for redundancy in visual-verbal representations. Moreover, a progressive K KV-cache retention policy aligned with cross-modal fusion further reduces memory usage during autoregressive inference. DARE reduces FLOPs by 40.37% and KV-cache usage by 46.07% on average, while consistently maintaining or improving perfor- mance across seven multimodal spatial reasoning benchmarks and generalizing to broader multimodal reasoning tasks.
INTRODUCTION
Multimodal spatial reasoning involves understanding object layouts, movements, and interactions by jointly leveraging visual and linguistic cues. Such tasks often require multi-hop reasoning, where intermediate visual and textual "thoughts" are appended to the input and reprocessed in subsequent iterations. Although approaches such as VoT and MVoT are effective (Li et al., 2025a; Wu al., 2024b), their design introduces severe scalability issues. For example, MVoT interleaves visual and verbal tokens for each thought (e.g., 32 x 32 tokens per image), allowing only the most recent three multimodal thoughts within 4K context window. More broadly, the autoregressive accumulation of intermediate tokens rapidly increases sequence lengths, causing quadratic growth in attention cost and memory usage, which fundamentally limits current MILMSs in multi-hop spatial reasoning.
While numerous studies have explored efficient reasoning for large language models (Aytes et al., 2025; Liu et al., 2024a; Kang et al., 2025; Tan et al., 2025; Hao et al., 2024; Su al., 2025; Cheng and Van Durme, 2024; Zhang et al., 2025a; Chen et al., 2024c; Shen et al., 2025; Wang et al., 2025; Lyu al., 2025), these methods encounter two key challenges when applying multi-hop multimodal spatial reasoning:
Challenge Multi-modal token importance shifts within- and across-hops. Prior frameworks typically compress tokens within single reasoming hop and rely on fixed or heuristic retention ratios across layers (e.g. (Chen et al., 2024b)). However, token importance varies considerably both within a hop (across network depths) and across successive hops. Specifically, as shown in Fig. 1, both visual and textual tokens follow diverse importance trajectories across layers and hops, reflecting the different semantic roles captured at each stage (e.g., objects, relations, background context). As result, existing models either over-retain redundant tokens or prematurely discard cues that becomes
"Corresponding Author

---

Published as a conference paper at ICLR 2026
Ingut Image
Routcr's importance score for tokens
Routers s importance score for text tokens
fs
410
415
lan
225
La0
ls
L10
Lns
Igp
somsone is washing dishes at the sink, which direction would they turn access the oven, and what spatial evidence supports your conclusion?
h4
hus
Answer: The person would (nft from the sink reach the oven, the positioned along the adlgccent wall the Ieft. with open space allowing direct access.
Figure I: DARE's router predictions on spatial reasoning task. The left shows the image and question. The right figures visualize token importance scores for visual tokens (green, left grid) and text tokens (red, right grid) across layers (11.4.-110-111420 125) and 5 reasoning hops (h1 to hş).
critical in later reasoning steps. This gap necessitates an intra- and inter-hop adaptive retention mechanism that can recurrently trace token utility throughout the reasoning process.
Challenge 2: Divergent redundancy patterns across modalities. Existing approaches mainly target single modality (e.g., text or images) or apply heuristic and uniform pruning across modalities. However, visual tokens exhibit distinct redundancy patterns compared to text tokens (Tan et al., 2025; Zhang et al., 2025c). As illustrated in Fig. 1, the semantic importance of visual tokens drops sharply after layer 11s, reflecting the flow of visual information into textual streams, while textual tokens remain semantically active in deeper layers. Lack of identifying and exploiting this visual-to- verbal transition leads to redundant visual retention and unnecessary computation. This gap necessitates an asymmetric compression strategy that aligns pruning with modality-specine redundancy patterns.
To tackle the challenges identified above, we propose a new framework, Dynamic and Asymmetric Routing for Efficient multimodal spatial reasoning, named DARE. To address Challenge 1, we design intra- and inter-hop-awar differentiable retention mechanism that adaptively estimates token utility at each network depth and reasoning step, capturing both intra-hop and inter-hop critical multimodal thoughts. To address Challenge 2, we introduce asymmetric cross-modal compression strategy that independently prunes visual and textual tokens based on their distinct semantic roles and redundancy dynamics. Furthermore, motivated by the observation (see Fig. 1) that the visual token importance decays significantly after the early layers across all reasoning hops, DARE employs progressive KV-cache retention strategy during inference, retaining a broader set of tokens in early layers where modal fusion is most active and pruning redundant key-value entries in later stages where high-level semantic abstraction predominates.
Notably, our proposed DARE'S is fully differentiable and integrated end-to-end, allowing the model to learn optimal token retention policies during fine-tuning. Extensive experiments and ablation studies demonstrate that DARE substantially reduces token redundancy, computation, and memory overhead, while preserving or even enhancing reasoning performance. These results establish DARE scalable and robust recipe for efficient multi-hop multimodal reasoning. Our contributions are summarized as follows:
We propose an intra- and inter-hop -awere differentiable retention mechanism that is fully end- to-end trainable and architecture- aggnosii, enabling MILMs to adaptively estimate token utility across depths and hops for fine-gre fine-grained control.
We develop an asymmetric cross-modal compression strategy that leverages the visual-to- -istal-to-text information flow: it prunes visual tokens more aggressively while preserving semantically critical textual tokens, aligning pruning with modality-specifit redundancy dynamics.
We introduce a progressive KV-cache retention policy that aligns with cross-modes fusion dynam- ics, retaining richer token sets in early layers and pruning aggressively in later stages to reduce computation and memory overhead.
Notably. DARE delivers significant efficiency gains, reducing FLOPs by 40.37% and KV-cache usage by 46.07% across seven multimodal spatial reasoning benchmarks, while consistently
2

---

Published as a conference paper at ICLR 2026
preserving or improving accuracy. Moreover, it generalizes robustly to broader reasoning tasks such as general reasoning, hallucination detection, and dialog-based VQA.
2 RELATED WORK
Multimodal Spatial Reasoning (MSR). Directly applying Chain-of-Thought (CoT) (Wei et al., 2022) to the multimodal setting poses substantial challenges in representation and efficiency (Li et al., 2024a; Wu et al., 2024a; Hurst et al., 2024). Existing efforts to improve the spatial reasoning capabilities of MLLMs generally follow three major directions: (1) Two-stage abstraction methods first convert visual content into intermediate symbolic representations (e.g.. text (Zhang et al., 2024), graph (Mitra et al., 2024; Mondal et al., 2024), bounding boxes (Lei et al., 2024)), and are then used for downstream reasoning. (2) Tool-augmented pipelines integrate external components to perform reasoning over complex visual observations (Yao et al., 2023; Yang et al., 2023; Hu et al., 2024; Zhou et al., 2024; Li et al., 2024c; Gao et al., 2024). (3) Unified sequence models (Li et al., 2025b;a) directly interleave visual and textual tokens within the same reasoning stream, enabling fully end-to-end multimodal reasoning over multiple hops. However, such unified models produce visual and textual intermediate "thoughts" that lead to the rapidly growing sequence lengths and quadratic attention costs, limiting the reasoning depth and memory efficiency of MLLMs.
Textual Token Compression. Existing approaches can be broadly categorized into two types. (1) discrete token reduction methods aim to reduce the number tokens via prompt engineering (Han et al., 2024; Nayab et al., 2024; Aytes et al., 2025), instruction fine-tuning (Liu et al., 2024a; Kang et al., 2025; Zhang et al., 2025a), or reinforcement learning (Arora and Zanette, 2025; Luo et al., 2025; Yang et al., 2025; Xu et al., 2025a; Mu et al., 2023). (2) continuous latent reasoning methods (Hao et al., 2024; Cheng and Van Durme, 2024; Deng et al., 2024; Xu et al., 2025b; Shen et al., 2025; Su al., 2025) project intermediate reasoning steps into continuous latent spaces, which are often more efficient but suffer from less interpretability and degraded performance.
Visual Token Compression. As visual tokens often constitute the majority of the multimodal sequence (Tan et al., 2025) and exhibit structural redundancy, existing methods can be divided into two major paradigms. (1) training- -based methods (Li al., 2024b; Zhang et al., 2025b; Tong et al., 2024; Raposo et al., 2024) integrate token compression into the model architecture but overlook the visual-to- verbal information flow. (2) training-free methods (Chen al., 2024c; Zhang et al., 2025c; Tan et al., 2025) compress visual tokens at inference time without modifying the model. While lightweight, these methods are limited to single-pass inference and lack hop-wise, recurrent adaptation, limiting their effectiveness and applicability for autoregressive accumulation in MSR.
METHODOLOGY
3.1 PRELIMINARIES
Multimodal spatial reasoning jointly interprets and reasons multiple modalities for complex decision-making tasks. Let Pa denote a pre-trained MILM parameterized by 8, and let the input consist of textual observations (+()) and visual observations x=()) The model aims to capture both symbolic and spatial dynamics throughout the reasoning trajectory before producing the final answer. Formally, the model autoregressively generates a sequence of textual thoughts (4,..., (n) and visual thoughts (wi,-.., ) such that:
(1) where l and é; denote the previously generated verbal and visual thoughts, respectively. The process alternates between generating new visual thought 6: and a new textual thought 42p1, conditioned on the full input and the history of previous thoughts.
0+f1 Po(u+41 (. &1, .4, 0:), f Po(ti+1 x0 (x))
(0) 81 R. S...,........
3.2 MODALITY-AWARY TOKEN ROUTING
In multi-hop multimodal spatial reasoning, the model produces long streams of visual and textual tokens, yet large portion contributes minimally to the final prediction. Visual tokens commonly
3

---

(e) are aggressively pruned in deeper layers, guided by the learned retention ratio Ph.1* while textual Published as a conference paper at ICLR 2026
Hop
Layer
Layer
Attn.
Attn.
Input tokens
FFN
FFN
Intermediate
Router
Router
Hop
HF
ML
Hb
Attn.
H
Attn.
Inpul Intermediate tokens
Router
FFN
Router
FFN
Answer
Figure 2: An overview of DARE. DARE introduces dynamic and asymmetric token routing strategy that compresses visual /h, and textual tokens Th1 in an intra- and inter-hop manner. Visual tokens
tokens are retained according to the learnable ratio Ph.1
encode redundant background content, while textual tokens often serve only shallow syntactic purposes. These inefficiencies motivate a selective routing strategy that prioritizes semantically meaningful tokens while pruning redundant and low-utility ones.
To this end, DARE introduces a lightweight and modality-specife routing mechanism, as exhibited in Fig. 2, where each transformer layer integrates modality-speciie gating head to score token importance. Dedicated linear routers are designed for textual and visual embeddings, enabling fine- grained, token-level retention decisions. By pruning low-utility tokens across depths and hops, DARE concentrates computation on semantically and spatially critical content, significantly improving efficiency without compromising reasoning quality.
Token Importance Prediction. At each layer I, DARE introduces two lightweight, modality-specifit
routers: for text tokens and for visual tokens. Given the i-th token 2 (f.m) of modality m E (t. v), the router produces scalar importance value through a linear projection followed by sigmoid, which bounds it in [0, 1] and makes it directly interpretable retention ratio.
(6.m) o (w/ 27
(fim) + b(()))
where wr(m) and ((m) are learnable router parameters, and (-)) denotes the sigmoid function. During fine-tuning, tokens retained under the target ratio Peange have their activations scaled by the predicted importance scores. This design allows gradient signals to flow not only through the backbone layers but also into the router, enabling its parameters be optimized end-to-end. Here, f(m) is predefined global sparsity budget that specifies the desired retention level (see Fig. 5 for an ablation study). Formally, the processed output y/
(i.m) [ayer(z: (f.m)) is updated as:
gf
fi.m) af (f.m) (ff..m) (f:m))) + (1 - af.mm)))) -
(3)
where (((.m)) is the routed output the next layer. The binary mask (i.m) {0, 1) denotes whether (i.m) 1 if retained and otherwise.
(m) token i falls within the top Pearget fraction of scores, with aj
3.3 INTRA- AND INTER-HOP AWARE RETENTION AND ASYMMETRIC COMPRESSION
Reasoning hops and network depth emphasize different semantic and thus exhibit distinct token redundancy. To address this, we propose an intra- and inter-hop differentiable retention mechanism
(t) that learns modality-specin per-layer ratios Ph.t and ((.)) indicating the fraction of textual and
visual tokens to retain. These ratios are initialized to zero and jointly optimized with model training,
allowing retention to adapt dynamically across hops and depths. global budget Paag provides an
(mn) overall constraint, while the learned ratios Pi. are softly regularized toward it, enabling fine-grained
control within and across hops to balance retention and preserve performance (App. D.6).
4

---

Published as a conference paper at ICLR 2026
We incorporate Gumbel-Softmax relaxation (Jang al., 2017) to approximate discrete token selection in a differentiable manner. Given token-level importance scores $1.1 with Gumbel noise and apply softmax, yielding a continuous approximation of the selection mask.
(f.m) we perturb them
exp (5 g.)/T)
(5.m) 9h.1
(4)
Σ, exp (3 + 93)/T)
(..m)
where 9; Gumbel(0,) 1) and 7 > 0 is temperature hyperparameter. As - 0, sharpens into one-hot vector, collapsing hard token selection. During training. the forward pass computes the (m)
average retention ratio a((m) from and measures its gap to Purget while straight-through estimator treats y as a surrogate in backpropagation, preserving gradient flow to the routing scores under the
(m)
ratio constraint. At inference, switch to deterministic pruning by keeping the top Ph. fraction of tokens based the raw scores $1.1
(f.m) An ablation analysis of is provided in App. F.2.
Another key challenge in multimodal spatial reasoning lies in the structural and semantic heterogeneity between textual and visual tokens. Uniform pruning overlooks these differences, often leading to suboptimal compression. To address this, DARE adopts dynamic and asymmetric routing that learns separate pruning policies for each modality. This design enables adaptive token flow control, effectively reducing redundant intermediate representations while preserving reasoning fidelity.
Text Token Retention. We introduce an auxiliary MSE loss over hops and layers to encourage the model to maintain a desired sparsity level for text tokens. Let P%, denote the average retention ratio
of text tokens at hop / and layer I, and let Ptarget be the predefined target ratio. The loss is defined as:
(i)
H
L
i0 Ph.t
ratio
HL
Punget
(5)
R=1
Two-Phase Visual Token Retention.
Visual tokens exhibit high spatial re- dundancy and contribute less rea- soning in deeper layers. To exploit this property, we introduce two- phase retention strategy. In the soft retention phase (layers / < (e), visual tokens are softly gated using impor-
Um
(i.v) [0, 1] predicted tance scores hhd
JN Rocere Bate
by the router. We design auxiliary
Figure Token retention ratios across depths and reason- ing hops in DARE. (a) Visual token retention drops sharply in deeper layers, as spatial details lose utility once fused into language. (b) Textual token retention remains stable or rises, underscoring its increasing role in semantic reasoning. These contrasting trends motivate DARE's E two-phase strat- egy: pruning redundant visual tokens in later layers while retaining text for semantic reasoning, thus improving effi- ciency without loss of accuracy.
MSE loss between the empirical re- tention ratio ph1 and fixed target
Pranget encouraging the model to fol- low predefined visual sparsity sched- ule in early layers. This loss de- noted as ciyh and follows the same formulation as cit) for text ones.
In the hard pruning phase (layers > lc), the cutoff layer 1 is identified via simple yet highly effective ccorethrresooding algorithm (e.g., the first layer where the mean visual importance falls below threshold for two consecutive layers; see App. D.7). Once visual cues have been fused into language representations, we suppress residual activations with a penalty loss.
hard Σ
P max(0, sh -c))
(6)
where y is penalty weight and is small threshold (e.g.. 0.01). An ablation study the effect of e is provided in App. F.1. This two-phase design retains visual tokens only when they are beneficial and prunes them aggressively in later layers, thereby significantly reducing computational overhead.
5

---

Published as a conference paper at ICLR 2026
The overall training objective becomes:
L Lrask + Fraio + L + L
where Cask is the primary multimodal reasoning loss.
3.4 KV-CACHI RETENTION STRATEGY
DARE prunes tokens dynamically across depths and hops, so many tokens never produce key/value entries at layer Since pruned tokens produce no KV entries, we enrich the standard causal mask with an execution mask to block queries from attending to them. Formally, let ((1m)) E {0, 1} indicate whether token j of modality m E {t. v) is executed, i.e., retained, at hop h and layer I. In addition, we reserve a small prefix of tokens per hop, validated by sensitivity analysis in Tab. 5, (i) preserve system-level tokens such as BOS/CLS and (ii) maintain early cross-modal alignment. Because tokens from multiple hops are interleaved in the sequence, indices are defined hop-locally to avoid ambiguity. The execution mask is then defined as:
is x or 6,.1 = 1 or 0h,1 0, 00, otherwise,
(6i)) 1,
5 =
(8)
and apply the composite attention mask
Ma.1 Memua En.d,
(9)
so pruned tokens receive oo logits and are never queried. During inference, we cache K, V only when f().m) = 1; skipped tokens incur zero storage. Let mem KV-cache memory that would be required at hop h, layer if all textual (resp. visual) tokens were cached with no pruning. The expected VV-cache memory therefore becomes
ful-t full-t (resp. memfil denote the h.l
(v) [[mem..,] = Ph. memp,.
fulloy
fullar P%) mem.,.
(10)
yielding substantial memory savings without degrading reasoning quality (see Tab. 4).
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
We evaluate DARE on two interleaved reasoning architectures that jointly process visual and textual tokens. (I) We integrate DARE into VolCano(Li et al., 2025b), which interleaves text with RefBind- based visual tokens, and fine-tune it the VoCoT instruction-tuning dataset using the AdamW optimizer(Loshchllor and Hutter, 2017) with a learning rate of 10 and weight decay of 3 x 10 10-? (2) We incorporate DARE into Anole-7B(Chern et al., 2024), which interleaves text with generated mental images, and fine-tune it for 60 epochs on three multi-hop spatial reasoning benchmarks: al., 2023), and FROZENLAKE(Wu, et al., 2024a), following the original training hyperparameters. All experiments are run on A100 GPUs. Additional training and evaluation details are provided in the App. C. Tables highlight the best results as best and second-best as second
MAZE(Ivanisskiy al., 2023), MINIBEHAVIOR(ii
Baselines. We compare DARE against diverse set of baselines, spanning prompt-based heuristic- based, and latent-space token compression methods. SoT (Aytes et al., 2025) is a static prompt-base baseline that enforces fixed token budget through task-specific instructions. LightFast combines LighTThinker (Zhang et al., 2025a) for textual token pruning with FastV (Chen et al., 2024c) for visual token pruning, simulating independent, modality-specine heuristies. Heima (Shen et al., 2025) is adapted to the multimodal setting as a latent-space reasoning baseline. SparseVLM (Zhang al., 2025c) iteratively sparsifies visual tokens, retaining only the most informative ones reduce computation. Finally, we introduce UniPrune, a symmetric pruning baseline that applies differentiable token scoring but omits the hard pruning loss L
Evaluation Benchmarks. We evaluate DARE across a wide range of multimodal reasoning tasks to assess both effectiveness and generality. (1) Compositional and multi-step reasoning, including

---

Published as a conference paper at ICLR 2026
Table 1: Comparison on multimodal spatial reasoning. 1 indicates higher is better; indicates lower better. DARE-LH achieves competitive accuracy while significantly reducing FLOPs, latency, and memory. "-L" and "-LH" denote layer-wa wise and layer/hop-wiss variants of DARE, respectively.
Compositional tasks, egg, spatial reasoning and visual search
Methods
Punga Paezzt
VSR
V-Star
Acc.(%))
FLOPs(G)** (s) Mcm.(GB)
Acc.
FLOPs((G)* Lat.(s)* Mcm.
VolCano 100% 100%
67.18
19842.37
0.63
8.91
58.40
21785.41
0.69
9.37
SoT
53.22
17240.58
0.58
8.32
37.12
19820.16
0.64
8.94
LightFastV
57.19
15880.31
0.54
7.85
45,68
18340.33
0.59
8.43
SparseVLM
62.31
14827 25
0.49
7.23
55.95
17365.19
0.53
8.17
Heima
50.42
10138.96
0.39
6.14
40.67
13963.21
0.41
6.42
Unipru
40%
70%
63.71
14252.74
0.49
7.11
53.22
16620.27
0.53
7.69
DARE-L
40%
70%
67.13
12825.18 18
0.45
6.62
57.39
15128.02
0.48
7.14
DARE-LH
40%
70%
68.09
11310.63
0.41
6.13
60.07
13584.72
0.43
6.62
EmbSpatial
Winoground
Methods
Punst Puize Acc.(%))
(v)
(1)
FLOPP(())+
(st. Mem, Acc.((%) FLOPs(G)* Lat(s)* Mem.(GiB)*
VolCano
100% 100%
58.29
26542.87
0.72
9.73
68.37
27411.36
0.78
9.84
SoT
54.76
24175.63
0.66
9.18
62.90
25984.27
0.72
9.51
LightFastV
56.89
22904.75
0.62
8.71
64.21
24632.88
0.67
8.97
SparseVLM
62.32
20275.21
0.72
8.21
64.39
23155.26
0.77
8.357
Heima
53.32
17174.83
0.41
6,92
64.03
17811.47
0.49
6.81
Unipru
40%
70%
60.75
21283.59
0.58
8.13
65.83
23124.63
0.63
8.43
DARE-L
40%
70%
64.37
19627 41
0.53
7.59
67.22
21543.90
0.59
7,94
DARE-LH
40%
70%
68.09
17964.82
0.45
7.09
68.31
19873.15
0.54
7.42
Table 2: Comparison on mental image generation tasks using Anole-7B across three dynamic reasoning benchmarks. We report accuracy (Acc.), compute cost (FLOPs), and inference latency (Lat.). DARE achieves the best trade-off between accuracy and efficiency.
MAZE
MINIBERAVIOR
FROZENLAKE
Model
Methods
Acc.
HLDPs((F)
Lanis)*
Acc.(98)
FLOPSEG)+
[s)
Acc.(%:)
PLOPsiG)
Lar.(s)
VoT
86.56
25640.14
0.79
64.40
24815.83
0.75
80.21
23792.67
0.74
MYaT
92.95
22130.56
0.71
95.14
21247.29
0.68
85,60
20591.34
0.67
Heima
80.37
13243.92
0.42
68.49
10628.11
0.47
79.32
10870.96
0.38
Amole-7B Sparse
88.01
17223.23
Unipr
84.22
16962.41
0.59
90.34
16792.31
0.53
84.11
16225. 72
0.57
DARE-L
89.78
15962.41
0.54
91.25
14891.20
0.52
84.11
14072.38
0.50
DARE-LII
93.32
12739.67
0.43
95.47
11856.21
0.41
86.11
11032.87
0.39
spatial reasoning (VSR (Liu et al., 2023), EmbSpatial (Du et al., 2024)), visual search (V-Star (Wu and Xie, 2024)), and Winoground (Thrush et al., 2022)). (2) Dynamic spatial reasoning, using MAZE(vanniskkiy et al., 2023), MINIBEHAVIOR (Jin et al., 2023) and FROZENLAKE (Wu et al., 2024a). (3) General VQA, including GQA (Hudson and Manning, 2019) and MMBench (Liu et al., 2024b). (4) Hallucination detection, evaluated on POPE (Li et al., 2023) and AMBER (Wang et al., 2023), with CHAIR (Peng et al., 2023) used the metric for AMBER and accuracy for the others. Additional benchmark details are provided in the App. B.
4.2 MAIN RESULTS
DARE Excels in Compositional Multi-hop Reasoning with Superior Accuracy-Effiiieng Trade- off. Tab. presents the results of the VolCano model (Li et al., 2025b) across four compositional reasoning benchmarks: VSR, V-Star, EmbSpatial, and Winoground. DARE-LH consistently achieves the highest accuracy across all tasks while significantly reducing FLOPs, latency, and memory. On VSR and V-Star, it improves accuracy over VolCano by 0.91% and 1.67%, while reducing FLOPs by 43.0% and 37.6%, respectively. Similar efficiency gains are observed on EmbSpatial and Winoground.
Notably, while Heima employs latent-space reasoning for high efficiency, its accuracy lags behind due to the loss of fine- -rei-gd spatial details and explicit multimodal thought. In contrast, DARE-LH's hop-aware routing selectively preserves critical spatial information, balancing accuracy and efficiency.
DARE Generalizes to Dynamic Spatial Reasoning with Lighter Computation. We deploy DARE in Anole-BB and evaluate it on three dynamic visual reasoning benchmarks (MAZE, MINIBEHAV.
7

---

Published as a conference paper at ICLR 2026
Table 3: Comparison on general QA benchmarks and hallucination benchmarks. 4 indicates lower better, indicates higher is better. DARE-LH maintains competitive accuracy while significantly reducing FLOPs, latency, and memory usage.
General VQA Tasks
Methods
Puna Plezet
GQA
MMBench
Acc.(%)) FLOPs(())*
(s) Mem.(GB)
Acc.(%)] FLOPP((G) Lat.(s) Mcm.(GB)*
VolCano SoT
100% 100%
64.40
6052.37
0.30
6.13
68.11
7285.41
0.35
6.52
60.23
5240.58
0.28
5.76
63.27
6720.16
0.32
6.08
LightFastV SparseVLM
61.23
4980.31
0.26
5.28
63.84
6240.33
0.29
5,59
61.09
5021.36
0.29
5,91
63.77
6492.73
0.31
Heima
63.42
3120.36
0.20
3,51
66.45
4375.63
0.22
4.12
Unipru
40%
70%
62.58
4352.74
0.24
4.83
65.12
5835,27
0.27
4.98
DARE-L
40%
70%
64.71
3510.63
0.19
4.21
68.27
4684.72
0.22
4.28
Hallucination detection tasks
Methods
Punget
Pagget
POPE
AMBER
Acc FLOPs(G)*
(s) (GB)
AMB
FLOPs(G) Lat(s) MGB)
VolCano SoT
100% 100%
86.50
13285.70
0.43
6.21
4.60
13285.70
0.43
6.21
74.32
11598.12
0.39
5.82
6.13
11598.12
0.39
5.82
LightFastV SporseVI.M
77.89
10492.24
0.36
5.38
5.42
10492.24
0.36
5,38
78.32
11247.22
0.39
6.21
5.41
11764.31
0.37
Heima
71.38
6920.18
0.25
3,92
5.01
6920.18
0.24
Unipru
40%
70%
80.05
9272.41
0.33
4.79
5.12
9272.41
0.33
DARE-L
40%
70%
87.27
7390.41
0.27
4.27
4.67
7390.41
0.25
4.27
Table 4: KV-cache memory usage across spatial reasoning benchmarks. '-:; Not reported.
Method
EmbSpatial
Winoground
MINIBEHAVIOR PROZENLAKE
VoCoT
5.92 GB
GB
MVTT
4.61 GB
4.52 GR
4.46 GB
DARE-L
3.28 GB
3.52 GB
3.76 GB
3.83 GB
2.49 GB
2.40 GB
2.34 GB
DARE-LH
297GB
3.28GB
3.52GB
3.60GB
2.28GB
2.2IGB
2.13GB
IOR, and FROZENLAKE) to assess its architectural generalization and adaptability to dynamic tasks. As shown in Tab. 2. DARE-LII achieves the highest accuracy across all benchmarks while reducing FLOPs by 40-50% and lowering latency compared to MVoT and Vo7. Notably, it cuts compute on MAZE from 25.6K GFLOPs (VoT) to 12.7K, while improving accuracy by 6.8%.
DARE Preserves Accuracy with Lighter Computation General VQA Benchmarks. We evaluate DARE-L on GQA and MMBench and observe consistent gains in both accuracy and efficiency. DARE-L achieves the highest accuracy, matching surpassing the full-token VolCano baseline while reducing FLOPs by 42%, latency by 30%, and memory by over 1.8 GB. Compared to LightFastV, which is efficient but 3.5% less accurate GQA, DARE preserves performance while maintaining low compute cost, highlighting significant token redundancy even in standard benchmarks.
DARE Mitigates Hallucination in VLMs. We evaluate DARE on POPE and AMBER assess its ability to mitigate hallucinations in vision-language models. Tab. 3 shows DARE-L L achieves the highest accuracy on POPE (87.27%) and the second-best hallucination rate on AMBER (AMB- 4.67). While VolCano maintains competitive accuracy, it incurs substantially higher FLOPs and memory costs. Compared to Heima, which relies on latent reuse, DARE delivers a +11.9% gain on POPE and reduces hallucinations by 0.25 on AMBER, while maintaining comparable superior efficiency. These results demonstrate that DARE not only compresses effectively but also preserves factual consistency by adaptively retaining cross-modal cues essential for grounded reasoning.
KV Cache Efficiency of DARE. We analyze KV-cache usage, a dominant contributor to GPU memory during autoregressive multi-hop spatial reasoning. Tab. 4 shows that VoCoT and MVoT store all intermediate tokens within and across hops, resulting in substantial KV-cache overhead. In contrast, DARE-LH employs intra- and inter-hop adaptive routing, consistently achieving the lowest KV memory across all seven spatial benchmarks and reducing cache usage by over 40% on average.
Adaptive Token Accumulation in DARE Mitigates Context Saturation. Fig. 4 visualizes the temporal token usage across four representative tasks, revealing stark differences in intermediate token accumulation behavior across models. VolCano immediately saturates the available context
8

---

Published as a conference paper at ICLR 2026
VolCano
DARE-L
DARB-LH Winngroand
MAST
VSR
V-STAR
MAZE
4000
Joken Couat
6000
4000
0.0
01.2
11.4
07.6
00.0
0.2
10.4
00.5
00.0
0.4
0.6
0.88
0000
0.2
0.4
0.6
0.5
Time (4)
Time (5)
Time (5)
Time (5)
Figure 4: Token accumulation dynamics across four multimodal reasoning tasks. DARE-LH grows more gradually, keeping both fluctuations and token volume better controlled across hops, thereby facilitating the retention of critical information under constrained context length.
with maximal tokens (8192 or 4096), sustaining a full context limitation throughout execution. In contrast, both DARE variants show gradual and adaptive accumulation. DARE-L and DARE-LH exhibit slower growth pattern, with both fluctuation and token volume regulated more conservatively across hops, saving 60% tokens compared with VolCano and MVoT.
4.3 ABLATION STUDIES
Impact of c(T)
hard Loss. The ( objective promotes aggressive pruning of low-utility visual tokens in deeper layers without sacrificing performance. Tab. shows that this strategy consistently reduces latency, GPU memory usage, and overall computational cost. Statistical analysis using a Student's I-test reveals that DARE-LH and DARE-L significantly outperform UniPru in efficiency, with p-values of 0.0003% and 0.0007%, respectively. These results provide strong evidence that ( enhances token efficiency while maintaining improving accuracy.
hard
Token Retention Trade-Off. We as- sess the impact of varying modality- specific token retention ratios on per- formance. Fig. 5 shows that accu-
EmbrSqrist Acceracy
MAZE Acmmayy
20% sp%. 40% 50% eiks
Virual Token Matio pl
93
10% MPN S0%6
93.35
Acunay (N)
Acumny
racy peaks when retaining 70% of textual tokens and 40% of visual to- kens. Visual tokens are more toler- ant to aggressive pruning, particularly in deeper layers where their semantic contributions diminish due to cross- modal fusion. In contrast, textual tokens more sensitive to compression, they likely form the core of the reasoning process and are essential for maintaining both semantic and syntactic integrity.
91.10
02:60
90%
75%
70%
65%
60%
80%
75%
70%
65%
60%
phi
Figure 5: Ablation on modality-peccifc token retention.
Impact of the prefix size K. DARE reserves fixed prefix of R tokens per hop to ensure (i) system-level special tokens (c.g., BOS/CLS) are retained and (ii) early modsl alignment cues remain accessible under aggressive pruning. We ablate 0, 2, 2, 8, 16, 32 VSR and MAZE. Tab. 5 shows that when = 0 or 1, accuracy drops by 1.45-4.02%, and BOS token masking occasionally causes decoding failures. Increasing beyond? 2 yields negligible gains (<0.1% 1%) while increasing KV memory by up to 70%. We find ş it fully recovers performance, preserves key initialization tokens, and achieves KV cache efficiency.
Table 5: Impact of prefix size on accuracy (%) and KV cache (GB).
VSR
MAZE
Acc.t
KV
Acc."
KV
0
64.12
2.26
89,30
1.38
65.55
2.60
91.88
1.84
68.09
2.97
93.32
2.28
68.09
3.40
93.32
2.62
be the optimal choice:
8
68.07
3.85
93.32
3.05
16
68.05
4.35
93.31
3.55
32
68.06
4.85
93.33
4.05
5 CONCLUSION
This paper introduced DARE, dynamic and asymmetric token routing framework for efficient multimodal spatial reasoning. DARE learns modality-. intra- and mher-hop-aware retention strategies to adaptively prune visual and textual tokens based on their evolving importance across the reasoning process. This approach enables scalable, interpretable, and resource-efficien multi-hop reasoning. Extensive experiments across diverse spatial reasoning benchmarks show that DARE significantly

---

Published as a conference paper at ICLR 2026
reduces computational and memory costs, cutting FLOPs and KV-cache usage by over 40% while maintaining or even improving task performance. Beyond efficiency, its progressive KV-cache management alleviates context saturation, enabling longer-horizon multimodal reasoning. These results establish DARE as a scalable and robust framework for efficient multimodal reasoning.
ACKNOWLEDGMENT
This work was funded under the COIN-3D project, which has received funding from the European Union's Horizon Europe research and innovation program under grant agreement No. 101159667.
REFERENCES
Daman Arora and Andrea Zanette. Training language models to reason efficiently. arXiv preprint arXiv:2502.04450 2025.
Simon A Aytes, Jinheon Baek, and Sung Ju Hwang. Sketch-of-thought: Efficient IIm reasoning with adaptive rognitive-inspired sketching. arXiv preprint arXiv:2503.05179, 2025.
Heinz H Bauschke, Jérôme Bolte, and Mare Teboulle. A descent lemma beyond lipschitz gradient continuity: first-order methods revisited and applications. Mathematics of Operations Research, 42(2):330-348, 2017.
Guiming Hardy Chen, Shunian Chen, Ruifei Zhang. Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. Allava: Harnessing gpt4v-synthesized data for lite vision-language models. arXiv preprint arXiv:2402.1 1684, 2024a.
Jieneng Chen, Luoxin Ye, Ju He, Zhao-Yang Wang, Daniel Khashabi, and Alan Yuille. Efficient large multi-modal models via visual context compression. Advances in Neural Information Processing Systems, 77.73986-74001 2024b.
Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer Plug-and-play inference acceleration for large vision- language models. In European Conference on Computer Vision, pages 19-35. Springer, 2024c.
Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412. 13171, 2024.
Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu. Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation. arXiv preprint arXiv:2407.06133, 2024.
Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit cot to implicit cot: Learning to internalize cot step by step. arXiv preprint arXiv:240S. /4838. 2024.
Mengfei Du, Binhao Wu, Zejun Li, Xuanjing Huang, and Zhongyu Wei. Embspatial-benchh Bench- marking spatial understanding for embodied tasks with large vision-language models. arXiv preprint XXXx:22406.057666 2024.
Timin Gao, Peixian Chen, Mengdan Zhang, Chaoyou Fu, Yunhang Shen, Yan Zhang, Shengchuan Zhang, Xiawu Zheng, Xing Sun, Liujuan Cao, et al. Cantor: Inspiring multimodal chain-of- thought of milm. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 9096 9105, 2024.
Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token- budget-aware Ilm reasoning. arXiv preprint arXiv:24/2. 18547, 2024.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in continuous latent space. arXiv preprint
arXiv:2412.06769 2024.
Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. The 38th Annual Conference Neural Information Processing Systems, 2024.
10

---

Published as a conference paper at ICLR 2026
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 5700-6709, 2019.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Michael Igorevich Ivanitskiy, Rusheb Shah, Alex Spies. Tilman Räuker, Dan Valentine, Can Rager, Lucia Quirke, Chris Mathwin, Guillaume Corlouer, Cecilia Diniz Behn, et al. A configurable library for generating and manipulating maze datasets. arXiv preprint arXiv:2309. 10498, 2023.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gunbel-softmax. International Conference Learning Representations 2017, 2017.
Emily Jin, Jiaheng Hu, Zhuoyi Huang, Ruohan Zhang, Jiajun Wu, Li Fei-Fei, and Roberto Martín- Martín. Mini- Wini-behavior: procedurally generated benchmark for long-horizon decision-making in embodied ai. arXiv preprint arXiv:2310.01824 2023.
Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. C3ot: Generating shorter thain-of-thoughh with- out compromising effecttivnness. In Proceedings of the AAAI Conference on Artificial Intelligence, number 23 in 39, pages 24312-24320, 2025.
Satwik Kottur, José MF Moura, Devi Parikh, Dhruv Batra, and Marcus Rohrbach. Clevr-dialog: A diagnostic dataset for multi-round reasoning in visual dialog. arXiv preprint arXiv:1903.033: 03666, 2019.
Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. What is more likely happen next? aileo-ad-llnguuaee future event prediction. arXiv preprint arXiv:2010.07999, 2020.
Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, and Yang Liu. Scaffolding coordinates to promote models. Proceedings of the 31st International
vision-language coordination in large Conference on Computational Linguisties, 2024.
Chengzu Li, Caiqi Zhang, Han Zhou, Nigel Collier, Anna Korhonen, and Ivan Vulić. Topviewrs: languag models as top-view spatial reasoners. In The 2024 Conference on Empirical Methods in Natural Language Processing, 2024a.
Chengzu Li, Wenshan Wu, Huanyu Zhang. Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulié, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-oftthouggg Ferty-Second International Conference on Machine Learning, 2025a.
Kevin Y Li, Sachin Goyal, Joao D Semedo, and Zico Kolter. Inference optimal vlms need only one visual token but larger models. arXiv preprini arXiv:2411.03312 2024b.
Yifan Li, Yifan Du Kun Zhou, Jinpeng Wang. Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination large vision-language models. arXiv preprint arXiv:2305. 10355, 2023.
Zejun Li, Ruipu Luo, Jiwen Zhang, Minghui Qiu, Xuanjing Huang, and Zhongyu Wei. VoCoT: Unleashing visually grounded multi-step reasoning in large multi-modal models. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2025b.
Zhiyuan Li, Dongnan Liu, Chaoyi Zhang, Heng Wang, Tengfei Xue, and Weidong Cai. Enhancing advanced visual reasoning ability of large language models. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024c.
Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 11:355-65 651, 2023.
Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. Can language models learn to skip steps? arXiv preprint 1.X0v22411.01855, 2024a.
11

---

Published as a conference paper at ICLR 2026
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216-233, Springer, 2024b.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17Xiv:1711.05101, 2017.
Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length- hargmoniinnn fine-tuning for ol-like reasoning pruning. arXiv preprint arXiv:2501.12570 2025.
Yougang Lyu, Lingyong Yan, Zihan Wang, Dawei Yin, Pengjie Ren, Maarten de Rijke, and Zhaochun Ren. MACPO: weak-to-strong alignment via multi-agent contrastive preference optimization. In The Thirteenth International Conference on Learming Representations, 2025.
Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of-thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14420- 14420-14431, 2024.
Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, Rituraj Singh, and Godawari Sudhakar Rao. Kam-cot: Knowledge augmented multimodal chain-of -fthouhh-s reasoning. In Proceedings of the AAAI conference on artificial intelligence. number 17 38, pages 18798-18806, 2024.
Jesse Mu, Xiang Li, and Noah Goodman. Learning to compress prompts with gist tokens. Advances in Neural Information Processing Systems, 36: 16:19927-1935, 2023.
Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. Concise thoughts: Impact of output length on Ilm reasoning and cost. arXiv preprint (xXiv:2407.19823 2024.
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models the world. arXiv preprint arXiv:2306.14824 2023.
Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision, pages 2641-2649, 2015.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. PmLR, 2021.
David Raposo, Sam Ritter, Blake Richards, Timothy Lillicrap, Peter Conway Humphreys, and Adam Santoro. Mixture-of-depths. Dynamically allocating compute in transformer-base language models. arXiv preprint arXiv:2404.02223 2024.
Sashank Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of adam and beyond. arXiv preprint arXiv: 09237, 2019.
Xuan Shen, Yizhou Wang, Xiangxi Shi, Yanzhi Wang, Pu Zhao, and Jiuxiang Gu. Efficient reasoning with hidden thinking. arXiv preprint arXiv:2501. 19201, 2025.
DiJia Su, Hanlin Zhu, Yingchen Xu, Jiantao Jiao, Yuandong Tian, and Qinqing Zheng. Token assorted: Mixing latent and text tokens for improved language model reasoning, arXiv preprint arXiv:2502.03275. 2025.
Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:181 1811,0091, 2018.
12

---

Published as a conference paper at ICLR 2026
Xudong Tan, Peng Ye, Chongjun Tu, Jianjian Cao, Yaoxin Yang, Lin Zhang, Dongzhan Zhou, and Tao Chen. Tokencarve: Information peeenattioneeeevvingg visual token compression in multimodal large language models. arXiv preprint arXhv:2503.10501, 2025.
Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Can- dace Ross. Winoground: Probing vision and language models for visio-linguistic compositionality. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1888-524,409 2022.
Bo Tong, Bokai Lai, Yiyi Zhou, Gen Luo, Yunhang Shen, Ke Li, Xiaoshuai Sun, and Rongrong Ji. Flashsloth: Lightning multimodal large language models via embedded visual compression. arXiv preprint arXiv:2412.04317, 2024.
Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jía, Ming Yan, Ji Zhang, and Jitao Sang. An lim-free cilti-dimensional benchmark for milms hallucination evaluation. CoRR, 2023.
Zihan Wang, Ziqi Zhao, Yougang Lyu, Zhumin Chen, Maarten de Rijke, and Zhaochun Ren. A cooperative multi-agent framework for zero-shot named entity recognition. In Proceedings of the ACM Web Conference 2025, pages 4183-4195, 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Penghao Wu and Saining Xie. v?: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 13084-13094, 2024.
Qiucheng Wu, Handong Zhao, Michael Saxon, Trung Bui, William Yang Wang, Yang Zhang, and Shiyu Chang. Vsp: Assessing the dual challenges of perception and reasoning in spatial planning tasks for vlms. arXiv preprint arXiv:2447.0188 00600163,, 2024a.
Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, and Furu Wei. Mind's eye of Ilms: Visualization-of-thon Thirty-eighth Arnual Conference on Neural Information Processing Systems, 2024b.
elicits spatial reasoning in large language models. In The
Silci Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint ar0iv:2502. 186000 2025a.
Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. Softcot: Soft chain-of-thought for efficient reasoning with llms. arXiv preprint arXiv:2502. 1214, 2025b.
Junjie Yang, Ke Lin, and Xing Yu. Think when you need: Self-adaptive chain-of-thouge learning. 2025.
arXiv preprint arXiv:2504.0322
Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303. //811, 2023.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao.
React: Synergizing reasoning and acting in language models. In International Conference Learning Representations (ICLR), 2023.
Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. Lightthinker: Thinking step-by-step compression. arXiv preprint arXiv:2502.15589, 2025a.
Shaolei Zhang, Qingkai Fang. Zhe Yang, and Yang Feng. Llava-mini: Efficient image and video large multimodal models with one vision token. arXiv preprint arXiv:2501.00 03895, 2025b.
Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gu- dovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al. Sparsevim: Visual token sparsifi- cation for efficient vision-language model inference. Forty-Secona International Conference on Machine Learning, 2025c.
13

---

Published as a conference paper at ICLR 2026
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, et al. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems, 36:34661-34710, 2023.
Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal Chain-of-thoupe reasoning language models. Transactions on Machine Learning Research, 2024.
Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu, Siyang Gao, and Yue Zhang. Image-of-thouue prompting for visual reasoning refinement multimodal large language models. arXiv preprint 20X.::2405..3872. 2024.
Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, and Yejin Choi. Multimodal c4: An open, billion-scale corpus of images interleaved with text. Advances in Neural Information Processing Systems, 36: 9958-8974, 2023.
14

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
2 Related work
3
Methodology
3.1
Preliminaries
3.2 Modality-Awar Token Routing
3.3 Intra- and Inter-Hop Aware Retention and Asymmetric Compression
3.4
KV-Cache Retention Strategy
Experiments
4.1
Experimental Setup
4.2 Main Results
4.3 Ablation Studies
5 Conclusion
Statements
17
A.1 LLM Usage Statement
17
A.2 Ethics Statement
17
A.3 Reproducibility Statement
17
B Dataset Details
17
B. Spatial Reasoning Benchmarks.
17
B.2
Dynamic Spatial Reasoning Tasks.
18
B.3 General VQA Benchmarks.
19
B.4
Hallucination Benchmarks.
19
B.5
Vasion-Language Reasoning Datasets
20
B.6
Dialog VQA
20
C Implementation and Hyperparameter Settings
20
C.I Fine-tuning DARE in the VolCano Model.
20
C.2 Fine-tuning DARE on Anole-7B Model
21
C.3 Implementation of Baselines
D Mechanisms and Analysis of Learnable Token Retention
23
D.1 Correlation Between Learned Retention Ratios and Router Predictions
23
D.2 Effectiveness of MSE Loss for Retention Prediction
23
D.3 Why Gumbel-Softmax Instead of Softmax?
24
D.4 Comparison with Heuristic Token Retention and Routing Methods
25
15

---

Published as a conference paper at ICLR 2026
D.5 Stability of Learned Retention Masks Across Seeds
26
D.6 Details of Retention Ratios Ph.l' Targets Punget and Their Rationale
(m)
(m)
27
D.7 Data-Driven Determination and Robustness of the Visual Pruning Phase 4e
28
E
Efficiency Analysis
29
E.1 Parameter Overhead..
29
E.2 Training Stability and Convergence Speed
29
E.3
Engineering Effort
30
F
Hyperparameter Sensitivity and Ablation
30
E.1 Ablation on pruning threshold
30
F22 Impact of Temperature Scaling
30
Theoretical Analysis
30
G.1 Notation and Setup
31
G.2 Convergence of Retention Ratio Optimization
G.3 Differentiable Soft-to-Hard Token Selection
32
H Algorithmic Specification of DARE
33
16

---

Published as a conference paper at ICLR 2026
A STATEMENTS
A.1 LLM USAGE STATEMENT
LLMs were used for language polishing and formatting. Specifically: (i) to shorten sentences, refine grammar, and improve readability (e.g., compressing section summaries, rewriting figure captions, and smoothing phrasing in the scalability and generalization sections); (ii) to provide guidance on LaTeX formatting adjustments in Overleaf, such as tuning prapfigute spacing with \vspace and line height options; and (iii) to brainstorm alternative titles for DARE. The authors take full responsibility for all ideas, methods, and claims presented in this paper.
A.2 ETHICS STATEMENT
This work does not involve human subjects, personally identifiable information, or practices that could raise ethical concerns. There are potential conflicts of interest, and no sensitive or harmful methodologies were employed. We adhered to the ICLR Code of Ethics.
A.3 REPRODUCIBILITY STATEMENT
We have taken measures to ensure reproducibility. Key code components are provided, and dataset descriptions, hyperparameter settings, and training procedures are detailed in the appendix and supplementary material.
B DATASET DETAILS
B.1 SPATIAL REASONING BENCHMARKS.
We evaluate DARE across four spatial reasoning benchmarks that assess the model's ability to understand object configurations, spatial relationships, and fine-grained grounding across vision and language.
VSR (Liu et al., 2023) (Visual Spatial Reasoning) consists of image-text pairs where each sample includes factual claim about the image. The model must decide whether the claim is supported by the visual evidence, making it a binary classification task. We use the unseen test split to measure generalization in a zero-shot setting. Following prior work (Li et al., 2025b), format the input using the prompt: "Is there an event [description) in the image?" to contextualize the claim in question-like form that encourages explicit grounding.
EmbSpatial (Du et al., 2024) focuses on embodied spatial understanding and includes visually complex layouts described with spatial expressions. Each question involves reasoning over fine- grained spatial relations between entities (e.g., "the red box to the left of the blue ball"). We use the official test split and preserve the original visual-textual inputs ensure faithful evaluation of grounding precision.
Winoground (Thrush al., 2022) tests multimodal compositionality via challenging image-text alignment tasks. Each sample includes two images and two captions that differ subtly in word order or semantics (e.g., "a person holding a ball" vs. "a ball holding person"). The model must match each image to the correct caption. We cast this as caption selection task using the prompt: "Please describe the image.", which encourages the model to choose the caption that most accurately describes each visual scene.
V-Star (Wu and Xie, 2024) a benchmark for visual search cluttered scenes, requiring models to locate specific visual concepts given descriptive queries. It evaluates fine-grainee recognition, disambiguation, and pross-instance grounding in complex environments. We follow the standard protocol and use the benchmark as-is without additional prompting.
In short, these datasets offer a comprehensive evaluation of spatial reasoning under diverse settings, from binary judgment and relational grounding compositional alignment and object-level retrieval. They serve as the primary testbed for assessing the effectiveness of DARE's intra- and inter-hop- token pruning in structured multimodal reasoning.
17

---

Published as a conference paper at ICLR 2026
Table 6: Overview of spatial reasoning benchmarks.
Category
VSR
EmbSpat.
V-Star
Wino
Split
test unseen
test
test
Size
1222
3625
238
800
B.2 DYNAMIC SPATIAL REASONING TASKS.
We evaluate DARE three dynamic spatial reasoning benchmarks, namely, MAZE (Ivanitskiy et al., 2023), MINIBEHAVIOR (Jin et al., 2023), and FROZENLAKE (Wu et al., 2024a). Each dataset requires multi-step visual reasoning simulated environments. These datasets test the model's ability to understand evolving spatial configurations, action trajectories, and implicit goals in low-level visual domains.
MAZE is constructed using the Maze-Dataset framework (Ivanitskiy et al., 2023), which generates 2D grid mazes via an iterative depth- trpth-firstt search algorithm. Mazes of size 3 to 6 are generated using multiple random seeds to diversify the layout complexity. For each instance, navigation path is constructed, and redundant or repeated paths are filtered out to minimize knowledge leakage between training and test splits. At test time, each input consists of maze configuratio and three destination candidates (i.e., coordinate points), among which the model must select the correct goal cell. This setup emphasizes long-horizon spatial reasoning over visual layouts with minimal linguistic input.
MINIBEHAVIOR (Jin et al., 2023) is derived from the INSTALLINGAPRINTER simulation suite, where reinforeement learning (RL) agents trained to complete procedural tasks in 7x7 to 10x10 grid environments using the Sable-Baselines3 library. The dataset contains diverse agent trajectories, and only successful action sequences (i.e., those completing the simulated printer installation task) are retained. The dataset applies controlled environment perturbations to prevent memorization, Specifically, for repeated action paths or previously encountered environments, there is 40% probability of perturbing either the printer or table coordinates, and a 20% probability of removing one of these objects. After perturbation, the agent's action sequence is replayed in the modified environment to confirm its validity. This design encourages the model to generalize reasoning across near-duplicate yet semantically different scenes,
FROZENLAKE (Wu et al., 2024a) is adapted from OpenAl Gym's 's FrozenLake environment. It consists of grid-based navigation tasks where an agent must reach goal while avoiding holes, using Q-table- babble policies to guide action selection. Trajectories are generated from agents acting greedily with respect Q-values. Successful action sequences are included only if they haven't 't been seen before in the same environment. For unsuccessful attempts (e.g., falling into hole), the trajectory is included with 50% probability either in its original form or with appended random actions. In ambiguous cases (c.g., the agent neither fails nor succeeds), the trajectory is retained to increase coverage. Additionally, Q-tables are re-learned with randomly perturbed reward paths to introduce variance and avoid overfitting. The resulting benchmark challenges models reason uncertain, sparse-reward settings where the structure of the environment must be inferred through action sequences.
We follow the same experimental conditions as MVoT (Li et al., 2025a). Table 7 summarizes the core characteristics of the three dynamic spatial reasoning benchmarks used to evaluate DARE: MAZE, MINIBEHAVIOO, and FRÖZENLAKE, These benchmarks differ in grid sizes, entity types, and action sequence complexity, capturing a range of ppatial-tempora reasoning challenges. MAZE features procedurally generated layouts with deterministic navigation paths and moderate action diversity. MINIBEHAVIOR introduces greater behavioral complexity, offering a larger action space and controlled environment perturbations that test generalization over procedural tasks. FROZEN- LAKE, by contrast, defined by its stochastic transitions, variable action lengths, and explicit reasoning over patterned environments (e.g., slippery tiles and traps), making it the most structurally demanding. Notably, only FROZENLKKE includes explicit pattern modeling, as indicated in the "Pattern Details" row. Reported action lengths and entity counts averaged across samples to reflect typical task complexity. All three datasets provide comparably sized training and testing splits, supporting controlled evaluations of DARE'S spatial-temporal token retention strategies. Overall,
18

---

Published as a conference paper at ICLR 2026
Table 7: Characteristics of dynamic spatial reasoning tasks, highlighting varying complexities in action dynamics and structural patterns.
Task
MAZE
MINIBEHAVIOR
FROZENLAKE
Grid Sizes
6
5-8
3-6
Entity Types
Entities Numbers
5
3
7.16
Action Length
9.11
7.83
6.56
Action Types
4
7
4
Pattern Details
Train Size
5007
6400
6846
Test Set Size
1255
1604
1664
Table 8: Overview of General VQA Benchmarks and Hallucination Benchmarks
Category
General VQA
Hallucination
Datasct
GQA testdev_balen __alanccdddd
MMBench
POPE adversarial
AMBER
Split
DEV 4329
generative
Size
12578
3000
1004
these benchmarks form diverse testbed that complements static spatial reasoning tasks, emphasizing trajectory grounding, goal-directed prediction, and dynamic decision-path comprehension.
B.3 GENERAL VQA BENCHMARKS.
We evaluate DARE on widely used languale benchmarks assess its generalization beyond spatial reasoning: GQA (Hudson and Manning, 2019) and MMBench (Liu et al., 2024b).
For GQA, we follow prior work (Liu et al., 2023) and use the "testdev _bllancedd split, which provides balanced distribution over question types and supports rigorous evaluation of compositional reasoning and object-centric grounding. We prepend the instruction prompt: "Please visualize the answer you not sure about the details." in order to promote concise and confident outputs. This encourages the model to rely more explicitly on visual content when answering.
For MMBench, we adopt the official "DEV" split for efficient evaluation. MMBench is a manually curated multi-dimensional benchmark that tests model capabilities across 12 skill categories, includ- ing object recognition, attribute understanding, spatial relations, and commonsense reasoning. It emphasizes fine-gran grained vision-language alignment and has been used as a standard diagnostic tool for evaluating VLM performance under diverse visual and linguistic challenges.
Overall, these two benchmarks offer complementary perspectives on general vision- languuee under- standing and help validate the effectiveness of DARE's compression mechanism beyond its primary spatial reasoning setup.
B.4 HALLUCINATION BENCHMARKS.
We assess the factual reliability and grounding capability of DARE by benehmarking on two recent hallucination detection datasets: POPE (Li et al., 2023) and AMBER (Wang et al., 2023). These benchmarks evaluate whether a model generates content that not supported entailed by the visual input.
POPE Perception-Oriente Oriented Probe for Evaluation) (Li et al., 2023) introduces suite of contrastive visual examples designed to probe factual grounding and resistance to object hallucination. We follow prior work (Li et al., 2025b) and evaluate on the adversarial subset, which contains the most challenging examples where visual distractors introduced to provoke hallucinations. Each sample presents a claim about the image (c.g., "There a cat on the table"), and the model is asked to verify its correctness. We adopt the binary yes-or-no prompting protocol from the original dataset,
19

---

Published as a conference paper at ICLR 2026
ensuring consistency with the official evaluation setup. This setting stresses the model's ability to avoid overconfident assertions unsupported by visual evidence.
AMBER (Wang et al., 2023) (A Benchmark for Evaluating Realistic Hallucinations in Multimodal Models) tests the model's ability to generate grounded, factual descriptions of images in free- form language. It consists of diverse images with multiple levels of hallucination risk and uses CHAIR (Peng et al., 2023) as the underlying hallucination scoring metric. We evaluate DARE on the generative task split, where the model must produce description for each image. The original prompts from the dataset are preserved to maintain fair comparison. Generated responses are assessed for consistency, relevance, and hallucination rate using the CHAIR metric, which quantifies object-level mismatches between the output and the ground truth.
These two benchmarks offer complementary views of hallucination: POPE provides focused and controllable binary setting, while AMBER offers open-ended generation under real-world uncertainty. In a nutshell, they allow us to assess whether DARE'S s sparsity-aware routing improves not only efficiency but also the factual alignment of multimodal outputs.
B.5 VISION-LANGUAGE REASONING DATASETS
NLVR2 (Suhr et al., 2018) is benchmark dataset designed evaluate models' ability to perform compositional reasoning over images paired with text. Each example presents natural photograph and a pair of human-written captions that are closely related but differ in subtle semantic aspects, requiring fine-grained visual and linguistic understanding to determine which caption is true of the image. Unlike simple recognition tasks, this dataset emphasizes relational and contextual reasoning, such spatial relations, object attributes, and logical consistency between image and language.
VLEP (Video-and-Lae Langgugg Event Prediction) (Lei et al., 2020) is a large-scale dataset introduced to study future event prediction in multimodal settings. Each example consists of short video clip paired with a natural language description of the observed context, followed by two candidate textual hypotheses about what is more likely to happen next. Models must select the correct continua- tion, requiring them to integrate temporal visual cues with linguistic semantics and commonsense knowledge. Covering diverse everyday scenarios, VLEP emphasizes anticipatory reasoning beyond recognition, making it a challenging benchmark for evaluating laneo-la models' ability to understand events and predict plausible outcomes.
B.6 DIALOG VQA
CLEVR-Dialog (Kottur et al., 2019) is a synthetic diagnostic dataset designed to evaluate multi- round reasoning in visual dialog systems. Built the CLEVR framework. it pairs images of 3D-rendered objects with automatically generated multi-turn question- answerr dialogues that probe complex reasoning skills such as counting, attribute comparison, spatial relations, and coreference resolution. Each dialogue requires the model to maintain contextual memory across rounds while grounding linguistic references in the visual scene. By controlling the visual environment and dialogue generation process, CLEVR-Dialog provides a clean and interpretable benchmark for studying compositional, multi-step reasoning vision-languee language interaction.
C IMPLEMENTATION AND HYPERPARAMETER SETTINGS
C.I FINE-TUNING DARE IN THE VOLCANO MODEL.
We embed DARE into the 7B parameterr VolCano model (Li et al., 2025b), which interleaves textual tokens with RefBind- based visual tokens for fine-grained grounding. Instruction tuning performed on the VoCoT corpus under the configuration summarized in Table 9. Briefly, we keep the CLIP VIT-L/14 visual encoder frozen (input resolution 336 336) and initialize the language backbone with multi-domain caption checkpoints VALLaVA-Caption GRIT, fiikkr3kk-Entities MMC4). Training uses the Adam W optimiser (Loshchilov and Hutter, 2017) with 81 = 0.9, 82 = 0.95, <= 10-4 a base learning rate of 1x100 for visual-router parameters (and a peak LLM LR of 1x10 5), weight decay 3 10 no warm-up. and a cosine schedule. A global batch of 128 sequences (cach < 3072 tokens) is optimised for one epoch, using 1 precision on NVIDIA A 100 GPUs; gradients are clipped
20

---

Published as a conference paper at ICLR 2026
Table 9: DARE's training configuration for the instruction tuning stage.
Configuration
Instruction Tuning
Visual Encoder
OpenAT-CLIP ViT-L/14 (Radford et al., 2021)
Backbone Init
ALLaVA-Caption (Chen et al., 2024a), GRIT (Peng et al., 2023)
Flick330k-Entities (Plummer et al., 2015), MMC4 (Zhu et al., 2023)
Optimizer
AdamW
Optimizer Hyperparameters
B1 0.9, 8/ 0.95, le-4
Global batch size
64
Peak learning rate of L.LM
le-5
Learning rate schedule
Cosine
Training Epochs
Warm-up ratio
Weight decay
10
Gradient clipping
1.0
Input image resolution
336 x 336
Input sequence to LLM
3072
Numerical precision
bfloat16
GPU Usage
4 x NVIDIA A100s
Training Time
115h
1.0, and total wall-clock time is ~00 Under this regime, DARE jointly minimizes task loss plus sparsity regularizers, converging to retention ratios of roughly 40% for visual tokens and 70% for text tokens, while preserving VolCano's task accuracy and reducing both FLOPs and KV cache by 2 40%.
After completing instruction tuning, we evaluate the fine-tuned DARE-VolCano model on suite of downstream benchmarks covering static spatial reasoning, dynamic spatial reasoning, general visual question answering, and hallucination detection. For spatial reasoning, we assess perfor- mance VSR, EmbSpatial, V-Star, and Winoground. For general VQA, we use GQA with the split to measure compositional question answering performance. Finally, we assess factual grounding and hallucination resistance using POPE (binary classification) and AMBER (generative description), which stress the model's 's ability to align outputs with visual evidence.
C.2 FINE-TUNING DARE ON ANOLE-7B MODEL
We integrate DARE into the Anole-7B model (Chern et al., 2024), a multimodal architecture that combines textual inputs with self-generated mental imagery to support abstract spatial reasoning. We fine-tune this model for 60 epochs on three challenging multi-hop. dynamic spatial reasoning tasks: MAZE (Ivanitskiy al., 2023), MINIBEHAVIOR (Jin al., 2023), and FROZENLAKE (Wu et al., 2024a). These tasks require the model to reason over evolving spatial environments and multi-step action trajectories. Fine-tuning follows the MVoT configuration shown in Table 10, using AdamW W with =0-9.9, -0.95, and c-2e-4, weight decay 3x10-2 and a batch size of 8 with gradient accumulation of Training is distributed across NVIDIA GPUs.
This setup enables DARE dynamically prune uninformative tokens while preserving crucial trajectory and spatial cues. By integrating DARE's retention-aware routing into A we demonstrate that method generalizes beyond static instruction-tume models, enabling scalable and efficient reasoning in long-horie long-horizon, procedurally generated environments.
C.3 IMPLEMENTATION OF BASELINES
We adopt three recent efficiency-oricceee oriented reasoning baselines, SoT, LightFast, and Heima, and align their settings with DARE evaluation pipeline.
21

---

Published as a conference paper at ICLR 2026
SoT (Sketch-ofThhoughh) SoT is static prompt framework that condenses reasoning into cog- nitively inspired "sketch" phrases, reducing token usage by a predefined retention ratio p without model fine-tuning (Aytes et al., 2025). For each task, we prepend the task task-speccfic SoT instruction ("<sket.c>>)) to VolCano's 's original prompt, explicitly injecting the same retention ratio used in DARE ensure consistent sparsity constraints. We maintain the same generation parameters (temperature 0.7). Because SoT is purely prompt-based, no additional training or model modification required.
LightFast (LightThinker (Zhang et al., 2025a) + Fast (Chen al., 2024c). LightFast combines LighTThinker, text-side token compression module that prunes low-utility reasoning tokens, with FastV, a visual pruning component that discards redundant visual tokens after the second transformer layer. We implement LightThinker as lightweight controller atop VolCano's ' language blocks, using the original hyperparameters (retention ratio matched to DARE, Acntropy = 5e ve FastV is integrated into the frozen CLIP encoder with its default layer-2 cutoff strategy. Both modules are fine-tuned jointly for one epoch on the VoCoT corpus to ensure a fair comparison with DARE under the same training budget. This setup reflects strong modular baseline that simulates independent, modality-specific compression without unified routing.
Heima. Heima shifts chain-of-thought reasoning into latent "hidden thinking" space by compress- ing intermediate reasoning into single thinking token, which is decoded only at the final step (Shen et al., 2025). We condition the Heima Encoder on interleaved image and text inputs, allowing the latent reasoning vector to capture fused cross-modal semantics to adapt it for multimodal reasoning. Each intermediate reasoning step is encoded as a 128- 228-dimensional vector, while frozen Heima Decoder reconstructs the final output needed. We follow the official implementation, fine-tuning only the encoder using AdamW (3 009, 32=0.95, LR 1 x 10-4) for 3 epochs on the VoCoT corpus. This setup offers strong latent-space baseline that climinates intermediate tokens while leveraging multimodal context.
SparseVLM SparseVLM introduces visual token sparsification framework that prunes redundant image tokens before they are fed into the language backbone (Zhang et al., 2025c). We integrate Sparse into the VolCano pipeline by applying its token selection module CLIP-encoded visual features, using the same retention ratio as in DARE to ensure fairness. This module ranks tokens by learned importance scores and discards background critical object-centric cues. Following the official setup, the pruning module is jointly fine-tuned with VolCano for one epoch on the VoCoT corpus under identical optimization settings (AdamW, =1=9.9, =0995, LR x 10-4). This configuration reflects strong visual sparsification baseline that reduces computation and KV-cache usage without altering the textual pathway.
low-saliency patches while preserving
Table 10: Hyperparameters used for fine-tuning Anole 7B.
Configurations
DARE
Random Seed
42
Epochs
60
Optimizer
Adam W
Optimizer Hyperparameters
81 = 0.9, 81 0.95, = 28-4
Weight decay
3 x 10
Train Batch Size
8
Val Batch Size
Grad Accumulation
2
22

---

Published as a conference paper at ICLR 2026
Visual Tokens
Text Tokens
r= 0.85
0.89
1.0
Retention Ralio
0.6
0.4
0.0 0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Token Score
Token Score
Figure 6: Scatter-density plots router-predicted token scores vs. learned retention ratios for visual and fext tokens. Pearson correlations (r 0.85 visual; r 0.89 text) indicate that DARE aligns token importance with retention, despite different sparsity budgets (40 % visual, 75 % text).
D MECHANISMS AND ANALYSIS OF LEARNABLE TOKEN RETENTION
D.1 CORRELATION BETWEEN LEARNED RETENTION RATIOS AND ROUTER PREDICTIONS
Figure 6 illustrates the relationship between token importance scores predicted by the DARE router and the corresponding retention ratios after differentiable token compression. Each point represents a token, with its router score the x-axis and its final retention ratio the y-axis. Pearson correlation coefficients are computed over ~25,000 5,000 tokens spanning hops and 32 layers on the validation set, while 200 tokens in each subfigure are uniformly sampled for visualization the figure. We observe strong positive correlations (r = 0.85 for visual tokens, r = 0.89 for text tokens), despite differing average retention budgets (40% for visual tokens and 75% for text tokens). This trend confirms that higher router scores consistently translate into higher retention probabilities. Importantly, this alignment emerges without any explicit supervision, validating that DARE's joint continncoomressionn framework effectively couples token scoring with sparsity-awarr retention across heterogeneous modalities.
D.2 EFFECTIVENESS OF MSE LOSS FOR RETENTION PREDICTION
DARE relies on a lightweight mean-squared-error (MSE) objective to steer its router toward desired sparsity profile. For each modality m E (t, v). hop h, and layer the auxiliary loss c(m) = (Ph.T)
m) Planget) penalizes deviations between the empirical retention ratio Ph,1 (m)
(mm) and target
ratio Purgee Taking the gradient with respect to the learnable ratio parameter yields
i (()) ratio
2
(mm)
(m) Plarget
(11)
(m)
Pph. h.
which is an unbiased estimator of the enror magnitude. Hence the update magnitude shrinks linearly as the observed ratio approaches the target, guaranteeing first-order stationary point at ý Ptarget-
Because cim) is convex in p. gradient descent with diminishing step size converges to this optimum under the same Lipschitz conditions stated in Proposition I (Sec. G). In practice observe rapid stabilization of the per-layer retention ratios (<5 training epochs), providing reliable sparsity signal to the router while adding negligible computational overhead.
23

---

Published as a conference paper at ICLR 2026
Table 11: Ablation on modality-speeifii MSE losses. Accuracy (%, 1) after (i) removing the text-ratio loss, (ii) removing the visual soft + hard losses, and (iii) removing all MSE regularisers.
Benchmark
All MSE
- ratio
- + - T
Both
EmbSpatial
68.09
67.12
64.42
60.01
Winoground
68.31
67.01
61.78
57.34
V-Star
60.07
56.21
52,52
49.21
VSR
68.09
66.62
62.31
59.23
MAZE
93.32
90.90
82.73
76.42
MINIBEHAAVIOR
95,47
92.11
85.02
70.71
FROZENLAKE
86.11
83.13
77.61
69.21
Avg. A
-234
-7.58
-13.90
Empirical Evidence. Table 11 presents ablation study on the role of modality-specifie MSE regularizers in DARE. Removing the text-token ratio loss c(t) results in 2.34% average accuracy drop, with notable degradation on V-Star and Winoground. Excluding the visual retention losses c/id C causes larger 7.58% drop, underscoring their role in preserving spatial precision. Removing all MSE terms leads to 13.90% average decline, confirming that task supervision alone insufficient for effective sparsity-aware routing.
These results show that the MSE regularizers play a critical role in guiding DARE"s router toward consistent and effective token selection. Without these losses, the model tends misalign token importance scores with retention behavior, leading to degraded accuracy across both visual and textual modalities. As illustrated in Figure 6, the inclusion of MSE supervision enhances the correlation between router scores and final retention decisions. This complements our theoretical analysis, where the convex form of the MSE objective ensures stable optimization toward the target sparsity. Overall, the empirical and theoretical evidence supports the inclusion of modality-seccifie MSE losses as essential components for reliable and efficient routing.
D.3 WHY GUMBEL-SOFTMAX INSTRAD OF SOFTMAX?
Why Gumbel-Softmax? We employ Gumbel-Softmax (Jang et al., 2017) to enable differentiable yet sparse token selection, which standard softmax cannot provide. Unlike softmax, which always outputs dense, positive weights that retain all tokens and thus prevents any FLOP or KV-cache reduction (Table 12), Gumbel-Softne offers three advantages: (1) inference-tine sparsity, where setting 7 = yields deterministic top-k token selection and allows token skipping in attention/MLP layers, (2) stable gradients, the added i.i.d. Gumbel(001) noise and temperature 7 produce smooth, differentiable relaxation unlike non-differentiable softmax-thresholding and (3) improved exploration early training, since the injected noise prevents premature collapse on arbitrarily ranked tokens.
Table 12: Comparison between Softmax and GumbellSofttax,
Operation
Forward-pass output
Gradients w.rtt. logits
Softmax
Dense, all all-poostevee weights that 1.
Non-zero, but every token is still processed; no FLOP KV- cache savings.
Gumbel-Softmax (add Gumbel noise, divide by 7, softmax)
For 7 s 1, behaves like noisy softmax (encouraging ploration); as 5 + 0, converges one-hot vectors, mimicking top-k:
Non-zero until 7 + 0, enabling end-to-en training of the router and retention ratios p.
24

---

Published as a conference paper at ICLR 2026
Empirical evidence. Our ablation study (Table 13) highlights the necessity of Clumbel-Softman Replacing it with softmax-threesholding reduces average accuracy by over 6% (77.01% - 70.92%) without any gain in KV-cache savings, while fixing drops and weaker memory reduction (-34% vs. -46%). These findings demonstrate that Gumbel- Softmax is crucial for maintaining both stable training dynamics and efficient inference, enabling deterministic top-k token selection in DARE.
throughout leads to even larger performance
Table 13: Ablation on Gumbel-Softmant Accuracy is averaged across tasks.
Variant
Avg. Accuracy (9 tasks)
KV-cache A
Full DARE (Gumbel-Softman
77.01%
-46 %
Replace with softmax+threshold
70.92%
-37% %
No perturbation, 7 throughout
70.73%
-34%
D.4 COMPARISON WITH HEURISTIC TOKEN RETENTION AND ROUTING METHODS
We benchmarked DARE against four budget-matched heuristic token retention strategies evaluate the effectiveness of learned routing: (1) Gist token (Mu et al., 2023). which compresses input sequences into a summary token, (2) average attention scores, which retain tokens with the highest mean attention across heads, (3) hidden-state norm pruning. which removes tokens with low activation magnitudes, and (4) heavy-hitter routing (Zhang et al., 2023), which prioritizes tokens frequently activated across layers.
Table 14: Comparison of heuristic routers and DARE on VSR and MAZE benchmarks.
Router Type
VSR Ace.
MAZE Acc.
Avg. GFLOPs x10
Gist token
62.8%
84.5%
14.8
Hidden-stan state norm
57.2%
83.6%
14.3
Mean attention score
60.7%
81.8%
15.5
Heavy-hitter DARE (learned)
63.0%
88.7%
13.2
68.1 %
93.3%
10.1
As shown in Table 14, heuristic methods underperform DARE by 4.6-11.5% accuracy, while also incurring higher computational cost per token retained. The primary limitation of these heuristics is that they operate in a layer-wise manner, without adapting to evolving cross-hop or cross-modee dependencies, and are unable to dynamically detect task-dependent information cliffs. In contrast, DARE's s learned routing mechanism provides consistent gains in both efficiency and accuracy across benchmarks.
We further compared DARE with the heuristic MoD (Raposo et al., 2024) routing mechanism. MoD prunes tokens using fixed, hard-coded top-k ratios each layer, designed primarily for unimodal, single-pass inference. This approach lacks the ability to adapt retention across layers, reasoning hops, or modalities, and does not address memory growth from recurrent reasoning.
By contrast, DARE represents a principled departure from heuristic-based MoD and introduces four key innovations. First, implements modality-aware routing through asymmetric routing heads for vision and text. This enables pruning decisions to adapt to each modality's 's changing informativeness after fusion, whereas MoD restricted to unimodal routing and cannot distinguish modality-specific contributions. As result, DARE-L.H outperforms the unimodal UniPru baseline by up to +9.1 % accuracy while simultaneously reducing FLOPs by as much as 32% (Table 15).
Second, DARE employs dynamic, intra- and inter-hop-aware retention ratio. Its retention ratios are learnable and differentiated by both hop and layer, optimized end-to-end via Gumbel-Softmax, In contrast, MoD applies fixed, hard-coded top-k ratios per layer, preventing adaptation to evolving reasoning dynamics. In matched- matched-budget comparisons, DARE improves accuracy by up to +8.7% over MoD (EmbSpatial), shown in Table 16. Further evidence in Table 17 shows that hop-awareness alone provides an additional +4.2% accuracy and -21.6% FLOPs savings compared to DARE-L, underscoring the importance of intra- and inter-aware retention rates.
25

---

Published as a conference paper at ICLR 2026
Third, DARE explicitly aligns routing with the information fusion process
modalities. Multi- hop spatial reasoning requires joint processing of vision and text, where visual cues are progressively absorbed into the textual stream. DARE captures this transition: early layers retain more visual tokens for grounding and spatial alignment, while deeper layers prioritize text tokens for abstract reasoning.
Finally, DARE addresses the often-overlooked challenge of KV-cache efficiency in recurrent reasoning. In multi-hop settings, recurrent computation causes rapid growth KV-cache size, which in practice, rather than FLOPs, often the dominant bottleneck for scalability. MoD entirely overlooks this issue. DARE mitigates it with an execution mask (Eq. 9) that prunes dropped tokens from the cache at every hop, reducing memory requirements by 35-48% (Table 18). This mechanism substantially improves scalability and makes DARE well-suited for long-horizon reasoning tasks where cache size, not compute, the limiting factor.
Table 15: Comparison of DARE-LH with unimodal UniPru baseline.
Task
A Acc. (DARE-LH UniPru)
DARE-LH FLOPs Saving
VSR
+4.38
-20.6%
V-Star
+6.85
-18.3%
EmbSpatial
+7.34
-15.7%
Winoground
+2.48
-14.1%
MAZE
+9.10
-24.9%
MiniBehavior
+5.13
-24.4%
LrkzenLake
+2.00
-32.0%
Table 16: Comparison between MoD and DARE under matched budgets.
Task
MoD
DARE
A Acc. PARE-MooD
VSR
62.11
68.09
+5.98
V-Star
50.72
60.07
49.35
EmbSpatial
59.37
68.09
+8.72
Winoground
62.65
68.31
+5.66
Table 17: Accuracy and FLOP savings of DARE-LH relative DARE-L.
Task
A Acc. (DARE-LH DARE-L) DARE-LH FLOPs Saving
VSR
+0.96
-11.8
V-Star
+2,68
-10.2%
EmbSpatial
+3.72
-8.5%
Winoground
+1.09
-7.8%
MAZE
+3.54
-20.2%
MiniBehavior
+4.22
-20.4%
FrozenLake
+2.00
-21.6%
These results confirm that DARE takes a principled departure from heuristic-based MoD. Its learnable and modality-aware design, aligned with cross-modal fusion and adaptive across hops, not only delivers substantial accuracy gains but also addresses KV-cache efficiency, a limitation unaddressed by MoD. This distinction crucial for efficient multi-hop multimodal reasoning and explains DARE's consistent superiority across benchmarks, as heuristic approaches tend to accumulate and propagate errors while lacking the fine-grained signals required to calibrate compression the level of each layer, hop, and modality.
D.5 STABILITY OF LEARNED RETENTION MASKS ACROSS SEEDS
We trained the model five times with independent random seeds (11111, 22222, 33333, 44444, 55555) evaluate the robustness of DARE to initialization.
26

---

Published as a conference paper at ICLR 2026
Table 18: KV-cache memory usage (GB) across hops in VSR and related tasks.
Thought 1
Thought 3
Thought 5
Task
MoD
DARE
MoD
DARE
MoD
DARE
VSR
1.92
1.21
3.28
2.09
5.69
2.97
V-Star
1.96
1.35
3.34
2.37
5.52
3,28
EmbSpatial
2.01
1.51
3.41
2.65
5.37
3.52
Winoground
2.05
1.62
3.48
2.86
5.72
3.60
Accuracy stability. As shown Table 19, performance remains highly consistent across all seven spatial reasoning tasks. For instance, VSR achieves 68.3 + 0.24, and all other tasks exhibit standard deviations below 0.32 points, which is well within expected random variation. These results confirm that DARE maintains stable accuracy regardless of initialization.
Retention mask consistency. In addition to accuracy, the learned retention masks are also repro- ducible across seeds. Specifically, the average Jaccard overlap of retained tokens across seed pairs is 0.91 + 0.02, while the Spearman correlation between router scores exceeds 0.94 across all layers and modalities. This demonstrates that DARE consistently learns near-identical retention patterns across runs, further reinforcing its stability and reliability.
Table 19: Accuracy stability of DARE across random seeds. Reported mean + standard deviation.
Metric (across seeds)
VSR
V-Star
EmbSpatial
Wineground
Mean to
68.3 + 0.24
60.4 + 0.06
68.1 + 0.07
68.4 + 0.17
Metric (across seeds)
MAZE
MiniBchavior
FrozenLake
Mcan to
93.5 = 0.09
95.8 + 0.11
86.4 + 0.32
D.6 DETAILS OF RETENTION RATIOS (m)) TARGETS PTARGET AND THEIR RATIONALE
We provide additional clarification on the retention ratios used DARE. The framework distinguishes (m) between three quantities: (i) learnable per-hop, per-layer, per-modality ratios Ph. (ii) the corre-
sponding empirical ratios p/m) realized during execution, and (iii) modality-specifie global targets (m) Puuget that act sparsity anchors.
Learnable empirical ratios. Each ratio (m) is learnable scalar parameter, defined sepa- rately for each modality m [text, vision). hop and layer These parameters are initialized to zero and updated jointly with the model via backpropagation. During the forward pass, the router
produces token importance scores hh,
(Eq. 2). A differentiable Gumbel-Soffe Softmax mask (Eq. 4) selects the top fraction specified by Puaget producing routed activations Vh,1 retention ratio is then
(f.m)
(m)
_(f.m) (Eq. 3). The empirical
N (m)
Ph
(8m)) (r(m)
(12)
(m) {0, 1) the selection mask. Thus, Piarget encodes the intended budget, while Ph. reflects the actual fraction realized in the forward pass.
where (v((m)) is the token count and a
(i.m)
(m)
Consistency with global targets Ptarget We introduce modality-specifie global anchors Purrget that
(m)
(m)
regulate average retention across hops and layers. Without these anchors, the learned ratios Ph.1 drift
upward during fine-tuning, leading to excessive memory use (up to 2.7x KV-cache growth on VSR and nearly 2x average across seven multimodal spatial reasoning tasks).
27

---

Published as a conference paper at ICLR 2026
A quadratic auxiliary loss
H
(m)) budget
[m) Phut
(m) PParget
(13)
HL
softly penalizes deviations from the global budget while preserving local flexibility. These anchors stabilize FL.OP and memory usage, prevent error accumulation in autoregressive reasoning, and keep (t)
KV-cache growth under control. With Purrgtt 0.7 and Panget 0.40, DARE achieves large efficiency
gains (2 40% FLOP reduction and 46% KV-cache savings) without sacrificing accuracy.
Sensitivity analysis. We validated robustness by sweeping text targets Purget from 60-80% and visual targets from 20-60% (Figure 5). Accuracy remained within 1% of baseline across most settings, degrading only when the text ratio fell below 20%. The 70%/40%% configuration provides the best trade-off between accuracy and efficiency. Under these targets, DARE consistently outperforms or matches baselines on nine benchmarks while maintaining over 40% FLOP reduction and 46% KV-cache savings.
D.7 DATA-DRIVEN DETERMINATION AND ROBUSTNESS OF THE VISUAL. PRUNING PHASE le
DARE's transition from soft to hard visual token pruning is not heuristic, but automated, data- driven procedure that generalizes across models and tasks. The method is described in Alg. 4.
Automated detection. The pruning phase boundary 4, is determined at the start of training by analyzing the router's layer-wise mean visual token importance. Specifically, the importance curve is first smoothed with a short moving average. A greedy scan is then applied to identify the first layer where importance consistently falls below a fixed threshold c, after which hard pruning applied. This process is transparent, deterministic, and requires no manual tuning. As a result, le naturally adapts across architectures and scales (e.g.. Le 14 for VSR-7B, 4 18 for Chameleon-344
Robustness of l.- The boundary highly robust to hyperparameters and offsets. Shifting Le by +3 layers alters accuracy by at most 0.3% and FLOP savings by at most 1%, confirming that le serves as stable diagnostic rather than sensitive knob (Table 20). Across ten diverse benchmarks, the detected thresholds are consistently below 0.01, and le typically lies near the midpoint of the network (Table 21), reflecting consistent dynamics visual-token utility.
Comparison with learned gates. We also experimented with learning le using a differentiable gating mechanism. While the gate converged to a similar location (within +-2 layers of the automati- cally detected (c), introduced gradient noise and increased variance (+.0.3% accuracy fluctuations) without providing any performance gains. This demonstrates that the automated procedure is not only simpler, but also more reliable.
Interpretation. The automatically detected split aligns with a genuine semantic transition in multimodal reasoning. As shown in Figure 3, visual token importance drops sharply after 4,, exactly when text importance rises, marking the point where visual features have been absorbed and linguistic reasoning becomes dominant. Thus, DARE's pruning boundary reflects the natural dynamics of information flow, ensuring that pruning decisions remain both effective and semantically grounded.
Table 20: Robustness of DARE to shifts in the pruning boundary la. Accuracy (%) and FLOP savings reported.
Offset from 1
VSR Acc.
FLOPs.
MAZE Acc.
FLOPs!
-3
68.0
40.8%
93.2
-1.5%
(auto)
68.1
41.6%
93.3
-4.1%
+3
67.9
40.9%
93.1
-41.3%
28

---

Published as a conference paper at ICLR 2026
Table 21: Post-jump importance values and corresponding le values across benchmarks.
Benchmark
Post-jump importance values
le (layer index)
VSR
0.003
15
V-Star
0.007
14
EmbSpatial
0.002
15
Winoground
0.003
15
MAZE
0.004
14
MiniBehavior
0.003
13
Frke
0.007
13
NLVR2
0.010
15
GQA
0.009
15
POPE
0.002
14
E EFFICIENCY ANALYSIS
E.1 PARAMETER OVERHEAD
DARE introduces only two categories of additional parameters: (i) lightweight per-layer routing heads, and (ii) learnable retention ratios p that determine the fraction of tokens preserved per modality, hop, and layer. For a 7B- 1B-parameter VolCano backbone with L = 32 layers, hidden size d = 4096, H reasoning hops, and M modalities, the total number of added parameters
PDARE L x M (d -
L M 262,528.
This accounts for less than 0.004% of the base model size, over three orders of magnitude smaller than typical LoRA adapters. Such overhead is negligible, confirming that DARE's efficiency gains are achieved without materially increasing model size or training complexity.
E.2 TRAINING STABILITY AND CONVERGENCE SPEED
DARE maintains stable optimization while accelerating training efficiency. As shown in Table 22, training across five independent random seeds exhibited no instability events, with accuracy variance below 0.70%. Moreover, DARE reduced wall-clock time to the same validation loss by 32%. primarily due to early token pruming. which decreases per-layer computation by 25-40% from the first mini-batch onward. Importantly, convergence speed in terms of optimization steps remains nearly unchanged (17.5k vs. 17.2k), indicating that faster training arises from lower per-step cost. rather than shallower optimization.
Table 22: Training metrics for DARE vs. baseline Anole-7B.
Aspect
Baseline Anole-7B DARE-Anole-7B
Extra parameters
+0.26M
Per-step FI.OPs (train)
100%
57-69 %
Wall-clock time same val. loss
28h
19h (-32%)
Convergence steps (to 99% final acc.)
17.2k
17.5k
Instability events (loss spikes >5x median, 5 seeds)
0/5
0/5
Sources of stability. The robustness of DARE is attributable to three simple design factors. First, the auxiliary retention-target losses are deliberately weightee (0.1x) and removed after the early epochs; excluding them changes final accuracy by less than 0.3%, confirming their role as gentle regularizers. Second, the Gumbel-Softmax relaxation with moderate temperature (T = 0.7) provides smooth gradients and stable training; performance remains consistent across [0.5, 0.9). while hard masking is applied only at inference. Finally, DARE reuses the same optimizer (Adam and schedule the baseline model. requiring additional warm-up phases learning rate groups. These choices ensure that DARE integrates seamlessly into standard training pipelines, achieving significant efficiency gains without sacrificing stability or convergence quality.
29

---

Published as a conference paper at ICLR 2026
E.3 ENGINEERING EFFORT
DARE integrates with existing MLLM frameworks with minimal overhead. The implementation introduces lightweight router module responsible for three tasks: (i) token importance scoring, (ii) differentiable soft masking during training, and (iii) KV-cache pruning at inference. All components are implemented at the Python layer, requiring no modifications to attention kemels, checkpoint formats, or training loops. Consequently, DARE can be incorporated as plug-and-p and-play option with only few hundred lines of additional code, leaving the backbone architecture and training pipeline entirely unchanged.
F HYPERPARAMETER SENSITIVITY AND ABLATION
F.1 ABLATION PRUNING C
We further evaluated the sensitivity of DARE to the pruning threshold e on both a static benchmark (VSR) and a dynamic benchmark (MAZE). As shown in Table 23, varying over a wide range produces negligible changes in performance: accuracy shifts by less than 0.2% and FLOP savings by less than 1%. This demonstrates that DARE's efficiency and accuracy are highly robust the choice of e, making the method stable across different settings without the need for hyperparameter tuning.
Table 23: Sensitivity of DARE on VSR and MAZE, Accuracy (%) and FLOP savings are reported.
VSR Acc.
FLOPs]
MAZE Ace.
FLOPs]
(no penalty)
67.9
-0.2
-0..1
93.1
-0.1
-1.2%
0.005
68.1
0
41.6%
93.1
-0.2
-42.0%
0.010 (def.)
68.1
0
41.6%
93.3
0
-2.11% 1%
0.020
68.0
-0.1
41.3%
93.2
-0.1
-11.9%
0.050
67.7
0.4
40.8%
93.1
-0.2
41.0%
F.2 IMPACT OF TEMPERATURE 7 SCALING
Temperature 5 DARE controls the sharpness of the token importance distribution produced by the Gumbel-Softmall routing mechanism. Lower temperatures lead to harder, more discrete selections, while higher temperatures introduce softer token scores. Table 24 presents an ablation over E (0.3, 0.5, 0.7, 0.9, 1.1, 1.3) across all seven benchmarks. We observe that both excessively low and high temperatures degrade performance slightly, likely due to increased noise or over-smoothing in the token selection process.
A moderate temperature of T = 0.7 consistently yields the best accuracy across most datasets, balancing sharpness and stability in routing decisions. Notably, this value performs well across both static (e.g., EmbSpatial, VSR) and dynamic spatial reasoning tasks (e.g., MAZE, FROZENLAKE), indicating that the routing mechanism generalizes effectively under shared hyperparameter settings. These results validate our choice of a fixed temperature during training and confirm that DARE"s differentiable routing robust to moderate variations in r.
G THEORETICAL ANALYSIS
We provide a formal analysis of the retention mechanism in DARE, showing (i) that the (m) mod-layy ratios Pb. optimized with the AdamW algorithm converge, asymptotically driving the expected gradient norm | to and (ii) that the Gumbel-Softmee relaxation employed during training approximates the hard top-k masking used at inference within an O(r) gap. where is the final temperature.
30

---

Published as a conference paper at ICLR 2026
Table 24:
Impact of routing temperature on accuracy. Lower higher values lead slightly noisier routing decisions, while T = 0.7 provides the most balanced and robust results across tasks.
Temperature
EmbSpatial
Winoground'
V-Star
VSR
MAZE'
MINIBEHAVIOR
FROZENLAKE'
0.3
67.81
67.42
59.33
67.58
92.97
94.95
85.62
0.5
68.02
67.91
59.75
67.92
93.21
95.12
85.79
0.7
68.09
68.31
60.07
68.07
93,32
95.47
86,11
0.9
67.95
68.14
59.82
67.89
93.17
95.47
85.97
1.1
67.63
67.55
59.01
67.41
92.84
94.87
85.44
1.3
67.29
67.10
58.76
66.98
92.53
94.52
85.03
G.1
NOTATION AND SETUP
Let θ denote the model parameters excluding the retention variables p. The full loss is decomposed as:
L(O, p) Lassk +
ratio
soft
hard"
(14)
Each auxiliary term regularizes the learned retention schedule to promote sparsity in a structured and adaptive way. For hop h, layer 1, and modality {t, v), let (t.m) be the router score for 8A,1 (i.m) (0, 1) the soft score during (2,m) E [0, the top-k binary mask inference, and "i, token i, 0h training.
Assumptions:
A1 (Lipschitz Continuity). The gradient V VpC L-Lipschitz continuous.
A2 (Bounded Variance). There exists G such that B[IlV, Gº.
G.2 CONVERGENCE OF RETENTION RATIO OPTIMIZATION
We now analyze the update rule for when optimized with the AdamW algorithm' Let m, and Va be the first- and second-moment moment estimates maintained by AdamW, with paper-pprameters B1, ,2 (0. 1) and learning-rate schedule ?)- = 7 Vt. For each step
g: vpL(Or, Pt);
(15)
m, fim--1 (1-Bi)ge.
(16)
V. 32, (1- (22)B²² 32
Bi
(17)
Vi = vi/(1 (1- Bb), Pt1 P: 1): mnt
m, m//(1 -B1),
(18)
1). Am Pt:
(19)
where 0 is small stability constant and \ is the decoupled weight-decay coefficient applied to P.
Lemma 1 (First-order Stationarity under Adam). Assume AI-A2, Et di-
xc, and Ei= "i co.
Let be generated by AdamW with 31 < 1, 82 < 1, and > 0. Then the retention schedule satisfies
lim T-000
min IS1T
[(,,,,
(20)
Sketch. Following Reddi et al. (2019) for Adam-type methods, bound the bias-corrected moments under Lipschitz gradients (Bauschke et al., 2017), then show that the aggregated expected decrease in C is lower-bounded by Σ, M/ || ppl1 - n?. Because Σ, n < while Σ, 7e oc, the gradient norm must decay to 0. Even with AdamW's adaptive moments and decoupled weight decay, the learned retention policy converges
first-order stationary point, ensuring stable sparsity pattern that jointly respects task loss and regularization.
We use the decoupled formulation of Loshchilov and Hutter (2017).
31

---

Published as a conference paper at ICLR 2026
G.3 DIFFERENTIABLE SOFT-TO-HARI TOKEN SELECTION
Although retention is specified as ratio, each (ht, m)) instance with NE tokens naturally induces an equivalent top-k formulation by setting ki/,1) Lo,, (m)) NYT and retaining the top-k tokens.
(mm)
During training, relaxed weights ② are obtained via Gumbel-Sofiiee -ofttmx,, and the top-ki,, (m) entries are selected in the forward pass, while straight- straight-through through estimator treats g as the surrogate for gradient ((.m)) be the token importance scores, ordered $(1) $(2) 2 2 ((N),
flow. Formally, let [ li=1
and denote the top-runner-up margin by A = 8(1) $(2) 0. This discrete top-k selection is relaxation in a differentiable manner.
approximated via
exp((sh,) (i.m) + 9:)/T)
(..m) 1
9. ~ Gumbel(O, 1), r > 0,
(21)
Σx= exp( (( (..) 9/)/T)
which provides smooth gradients; at inference we take the hard top-k mask directly from the raw scores
Top-1 selection under Gumbel-Max. Consider the hard Gumbel-Max sample
=
arg max; (s//t gj}. The classic identity gives
<1//
Ply = (1)]
(22)
N Σラ= eu//T 1+ Lj= (81)
Using 1/(1+z)) 2 1 x and 8(1)
866) 2 A for j 2 2. we obtain the margin- based lower bound
N
Ply = (1)) 2 Σ
e
(1))
2 (N 1)e
(23)
j=2
Thus, as the margin grows or the temperature decreases, the probability that the hard sample agrees with the true top index rapidly approaches 1, with the exponential lower bound in equation 23,
Sketch. This follows from the Gumbel-Max trick (see, e.g., Jang et al. (2017)): adding i.i.d. Gumbel noise to (scaled) logits and taking the arg max produces categorical draw with probabilities proportional et//r, The lower bound uses one-step relaxation of the softmax denominator and union bound over competitors to the top logit.
Top-k agreement. Let Sy (1),..... ... (A)) be the set of true top-k indices of s, and let S& be the top-k indices after Gumbel perturbation of S/T. Denote the k-th margin by A := S(k) s(k+1) > Then simple union bound yields
3 = St] 2 -- Σ P(as/T 2 (wo/++9()) 2 1- Σ j=b+1
e (x00 2 1 1- Au/T
5=
(24)
Hence, larger k-margins Ag or lower temperatures 7 make top-k agreement exponentially likely.
Remark the relaxed vector. The relaxed output q in equation 21 continuous probability vector (it is almost never exactly one-hot for 7 > 0). Events like {Ū(1) 1} have probability zero unless - 0. In practice we train with ğ (to pass gradients) and switch the deterministic hard top-k mask at inference, recovering computational savings while benefiting from the high agreement guarantees in equation 23 and equation 24.
Empirical observation. With annealed temperature schedule (r |0.7, 0.1]), the Hamming disagreement between the relaxed top-k mask (thresholded from g) and the deterministic hard top-k mask remains below 1% under typical router margins (Ax 2 0.7). In practice, the soft training procedure closely matches inference-tie time behavior, and the learned retention policy converges reliably.
32

---

Published as a conference paper at ICLR 2026
H ALGORITHMIC SPECIFICATION OF DARE
During training, DARE employs learnable token router (Alg. 1) that assigns modality- and hop- specific importance scores. Differentiable retention achieved via a Gumbel-Softtax relaxation, further regularized by ratio-matching losses. To capture tross-modal fusion dynamics, visual pruning follows a two-phase strategy (Alg. 2): early layers apply a soft sparsity regularizer, while deeper layers impose hard penalty suppress redundant tokens. At inference time, pruning is implemented through an execution mask with selective VV-cache retention (Alg. 3), which blocks attention to discarded tokens and thereby reduces both memory footprint and FLOPs. The transition between the soft and hard phases is determined automatically by greedy threshold-based search (Alg. 4), ensuring efficiency gains without compromising accuracy.
Algorithm 1 Modality-Aware Routing with Intra/Inter-Hop Differentiable Retention (Training) (m)
Require: Tokens {#h.1 (1)))
} for hops h-1..H, layers 1-1..,, modalities me v3; target ratios Phargeti
temperature T
1: for h to H/ do
2: for t to L do
3:
for E (t, do
4:
// modallty-specific, learnable router
5:
11.1 +- (W( Thi
(8.m)
(f.m) ((m²)
6:
I| differentiable retention via Sumfel-Softmax, aligned with Puargetin STE in backprop
(5.m) (xp((=(.")) 9i Gumbel(0, 1)
7:
9h.t
8:
track retention ratio across layers and hops Ph.1
9:
+ No Huht
(6.m) in Top-Puagel (m)
10:
I| residual bypass stabilizes training
(t.m)
(f.m) (t.m) (f.m))
and )o (t..))
(b.m) ((
(5.m)
Viat I| hard mask forward pass; gradients via 9 (6.m) Layer(r (8.m), (3.m)
Vhl t- Onp!!
H Poop-e (m)
13:
yh.1
14: end for
end for
16: end for
// explicit control over sparsity
18: L (m)
HI En.(Ph (m) Pugga)? m E {t,0}
19: 11 combined objective with text ratio + two-phase visual retention
20: L Luusk + ( + ((v) + L
Algorithm 2 TWo-Phase Visual Retention (Soft - Hard)
Require: Visual scores $h,1 layer cutoff lc. targets Piagets weights μ. threshold
(i.z)
1: // soft phase: enforce visual sparsity by matching retention ratio Ptarget
2: (w)
HI ΣH Els (ph.)
(e)
Piarget
(v)
(v) Σ, (retained under Purgerl
3: Ph.l No.
4: // hard phase: suppress residual visual activations beyond cutoff l
.o(e) 5: (v) = - Et.,-1 Einr μ max(0, (3,e)
-e)
8h,1
6: total loss: combine task loss, text ratio, and two-phase visual regularizers
7: L = Lusk + (t) + (c) + L
33

---

Published as a conference paper at ICLR 2026
Algorithm 3 Execution Mask and KV-Cache Policy (Inference)
(2,m² Require: Deterministic scores 1/,1 learned retention ratios Ph. prefix size k.
(m)
for
H do
2: for = 1 to L do
(m) 3: // ratio-based deterministic retention per modality (top Ph. by score)
4: for m E {t, 0) do
(1.m) in [MT) I/ b {0, 1] marks retained tokens
5:
6:
end for
7:
I/ stability; reserve a small prefix regardless of scores
ogi <- 1 SK
8:
9:
// execution mask blocks attention to pruned tokens; preserves causal structure
En,(i,,) + 0, J or b b()+==1 or 6 h.l
10:
-00, otherwise
11:
I/ combine with causal mask for attention
12: Mh.1 < Mrausal + Ei.!
13: I/V caching only for executed (retained) tokens reduces memory/latency
cache (K(im)) Vil v-lim)) iff ((.m)
14:
h,l
h,1
// skip KV for pruned tokens
end for
16: end for
Algorithm Threshold- Based Selection of Visual Cutoff Layer l. (as described Sec. D.7)
Require: Visual scores $h,l" for hops he layers lE[Z]]; smoothing window w; fixed threshold
(f.r)
persistence (consecutive layers); minimum layer liii
1: I/ aggregate mean visual importance per layer across tokens and hops
2: 14 LA=1 N
M(e)
(t.v) Ah
VIE (1,.....1) L.}
3: // smooth with a short moving average to reduce noise
4: 1u 2, Z [w/2] Pj (clip indices to [1, L)
[+1//21
5: // greedy scan: find first layer after lmin where importance stays below for consecutive layers
6: le +- L. // default: no early cutoff found
7: I- L-r
8: if mmxx P142, ,.trt S then
9: 1o+1; break
10: end if
end for
12: I/ sanity check: avoid cutting before the smoothed median depth
13: if Le < median([,..... L} 1 then le +- me.ian L}
14: return le
34


Reviews:
[{"review_id": "WqedtGbXEX", "reviewer": "Reviewer_UNBZ", "summary": "This paper introduces DARE, a novel and well-motivated framework for efficient multimodal spatial reasoning. By dynamically and asymmetrically pruning tokens across layers and reasoning hops, the method achieves reductions in computation and memory while often improving task performance. The paper is supported with comprehensive analysis and evaluation with comparisons over different system variants.", "questions": "See as above in weaknesses. I'm happy to adjust the scores if the author can address the concerns in the weaknesses.\n\nTypo: L037 MoT should be MVoT", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-motivated. The paper excels at identifying and motivating two nuanced challenges in multi-hop multimodal reasoning: the dynamic, shifting importance of tokens across both network depth and reasoning steps, and the asymmetric redundancy patterns between visual and textual data. \n2. Interesting technical design. I really like the use of a differentiable, end-to-end learnable routing mechanism compared to fixed heuristics. Furthermore, the asymmetric compression strategy and the explicit focus on reducing KV-cache usage are critical for autoregressive models.\n3. Comprehensive empirical evaluation. The authors validate DARE across an extensive and diverse set of benchmarks, comparing it against diverse baselines. The results are good, showing that DARE not only meets its efficiency goals in terms of FLOPs and KV cache reduction but also even improves accuracy, demonstrating the method's effectiveness.", "weaknesses": "1. Reproducibility and reliability of the results: I'm concerned with the experimental results in Table 2, especially with MVoT and VoT. To be more specific, I would appreciate it if the authors can clarify:\n* Computational resources: can Anole 7B be trained with 40 GB GPUs for how long? Could the authors provide training logs for clarification purposes?\n* Experiment results: given that there is no public available datasets and model checkpoints MVoT uses in the original paper, and the reported numbers in Table 2 strictly aligns with the results in MVoT paper, plus the experimental settings (number of epochs, types of GPUs, numbers of GPUs) are not same as in the original paper, I would appreciate the authors provide more details regarding this. (I tried to look into the supplementary materials in the zip file, but didn't find the training script)\n2. Methodological complexity and hyperparameters. While effective, DARE is a complex system with numerous interacting components and hyperparameters (target retention ratios, pruning thresholds, Gumbel temperature, etc.). The paper could benefit from a clearer discussion of the tuning effort required and the generalizability of the chosen hyperparameter values to different model architectures or domains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HkEs2NjAu4", "reviewer": "Reviewer_8DKA", "summary": "This paper introduces DARE, a novel framework designed to improve the efficiency of multi-hop multimodal spatial reasoning by addressing the high computational costs of existing methods. It achieves this through several key contributions: a modality-aware token routing mechanism with dedicated routers at each transformer layer to score token importance; an intra- and inter-hop aware retention strategy that uses Gumbel-Softmax to learn dynamic pruning ratios across both network layers and reasoning steps; and a KV-cache policy that reduces memory overhead by aligning with token pruning decisions. The authors report significant efficiency gains while maintaining or improving performance.", "questions": "1. How do the router's importance scores change when the model is given a different task? For instance, does a visual search task yield a different tendency compared to a dynamic navigation task (like MAZE)?\n\n2. The term \"hop\" is central to the paper but is not explicitly defined. Is a \"hop\" defined as a single autoregressive generation step for an intermediate thought (which could be a mix of visual and textual tokens), as suggested by Figure 2? A precise definition would be helpful.\n\n3. In Table 1, DARE-LH shows a particularly large accuracy improvement over the VolCano baseline on EmbSpatial compared to other compositional tasks like VSR. Is there an intuition for why the proposed method is especially effective for this specific benchmark?\n\n4. Table 3 reports results for DARE-L but not DARE-LH on the General VQA and Hallucination benchmarks. Is this because these tasks are typically solved in a single reasoning step (i.e., one hop), making the inter-hop (-LH) mechanism not applicable?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. DARE shows compelling performance, consistently achieving comparable or superior accuracy to its baselines while being significantly more efficient.\n2. The framework demonstrates impressive generalization by showing effectiveness on two different interleaved reasoning architectures: one based on visual token referencing (VolCano) and another on mental image generation (Anole).\n3. The paper is supported by a comprehensive set of experiments and ablation studies that strongly validate the proposed design choices. The authors' effort in providing this level of detail, including a well-organized appendix, is very impressive.", "weaknesses": "1. **Impact of New Hyperparameters on Applicability:** The proposed method introduces several new hyperparameters (e.g., target retention ratios $p_{target}$, hard pruning threshold $\\epsilon$, prefix size $\\kappa$). While the paper includes helpful ablation studies, finding the optimal set of these hyperparameters for a new model or task could be a non-trivial and expensive process, potentially limiting the method's plug-and-play applicability.\n\n2. **Fixed Number of Reasoning Hops**: The framework appears to operate with a pre-defined maximum number of reasoning hops ($H$). Real-world problems may require a variable number of reasoning steps. It is unclear how DARE would adapt to tasks that require more or fewer hops than what it was trained for, which may limit its flexibility in more open-ended scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kipq0gdHAK", "reviewer": "Reviewer_wQMw", "summary": "This paper proposes a new framework that enhances the efficiency of MLLMs during spatial-reasoning tasks by adaptively pruning multimodal tokens across network depths, reasoning hops, and modalities. DARE integrates a layer- and hop-aware differentiable retention mechanism with an asymmetric compression strategy to selectively preserve essential tokens. Experiments on multiple multimodal spatial reasoning benchmarks show that DARE significantly reduces FLOPs and memory overhead while maintaining performance.\n\nI have reviewed this paper for NeurIPS 2025. After careful check, I noticed that the authors have incorporated the reviewers' suggestions and addressed their concerns in this version. For example, the comparison between DARE and MoD (Appendix D.4: Comparison with Heuristic Token Retention and Routing Methods) and the experiments across models of different sizes (Scalability to Larger and Smaller Models, Section G.1). I think this version is clear and well-written, meeting the standard for acceptance.", "questions": "After careful review, the authors have addressed all the questions I raised in my previous review, and I have no further questions.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. The motivation is solid, and the method is well-motivated. The authors tackle a key efficiency bottleneck in multi-hop multimodal spatial reasoning. The approach is clearly formulated and effectively presented.\n\n2. The main experiments are comprehensive, covering seven spatial-reasoning benchmarks, general VQA, hallucination detection, and detailed ablation studies.\n\n3. The authors provide thorough discussion and analysis, including comparisons between DARE and MoD, interpretation of retention rates, the necessity of learning-based routing, and how DARE scales as model size increases. Overall, the paper offers a complete and clear discussion of the DARE method and its position within the broader research landscape.", "weaknesses": "No obvious weaknesses.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pqWFlKYAq5", "reviewer": "Reviewer_ddYw", "summary": "This paper introduces DARE (Dynamic and Asymmetric Routing), a new framework designed to address the significant computational and memory costs associated with multi-hop reasoning in Multimodal Large Language Models (MLLMs). It proposes a dynamic token pruning mechanism. Through experiments on seven spatial reasoning benchmarks, the authors demonstrate that DARE achieves substantial reductions in FLOPs and KV-cache usage.", "questions": "Please see the weaknesses above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The paper provides a clear motivation by outlining the computational efficiency challenges in multi-hop multimodal reasoning. Furthermore, the work is presented with a coherent structure, making the methodology and experimental sections straightforward to follow.", "weaknesses": "Regarding the experiments on dynamic spatial reasoning, I am genuinely interested in understanding how the results in Table 2 were obtained. As someone familiar with unified MLLMs and interleaved multimodal reasoning, I noticed that the datasets mentioned—MAZE, MINIBEHAVIOR, and FROZENLAKE—do not appear to be publicly available, at least to the best of my knowledge. This raises some concerns about the reproducibility and authenticity of the reported results, as independent verification is currently not feasible. I believe that providing access to the source code and implementation details of this part would help clarify how the experiments were conducted. To adhere to the double-blind review policy, an anonymous link to a code repository would be very helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)