BOLT: Decision‑Aligned Distillation and Budget-Aware Routing for Constrained Multimodal QA on Robots
Authors: Tengjun Ni
Xin Yuan
Shenghong Li
Kai Wu
Ren Ping Liu
Wei Ni
Wenjie Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: multimodal question answering
vision-language models
robotics
knowledge distillation
resource-constrained AI
Primary area: applications to robotics, autonomy, planning

Abstract: Robotic systems can require multimodal reasoning under stringent constraints of latency, memory, and energy. Standard instruction tuning and token-level distillation fail to deliver decision quality, reliability, and interpretability under these constraints. We introduce BOLT, a decision-aligned distillation and budget-aware routing framework that treats multi-choice prediction as a decision surface to be aligned during training and selectively refined at inference. During training, BOLT introduces Option-level Decision Distillation to align student models directly on the decision surface of multi-choice answers, thereby eliminating prompt artifacts, improving calibration, and optimizing the exact output space. At inference, BOLT activates Budget-aware Test-time Augmentation, a calibrated router that uses low-cost signals such as confidence, margin, entropy, retrieval affinity, and agreement across short question decompositions to trigger high-resolution reevaluation, type-matched retrieval exemplars, or question decomposition only when their expected benefit outweighs cost. On Robo2VLM-1, a 2B BOLT student distilled from LLaVA-1.5-13B improves accuracy from 28.66 in zero-shot to 42.89 with decision distillation and to 50.50 with budgeted routing, surpassing the 13B teacher at 36.74. It lowers expected calibration error, strengthens the risk-coverage frontier, and slashes GPU memory from 26,878 MB for the teacher to 3,035 MB for the distilled student, and 3,817 MB with all augmentations enabled. By constraining outputs to valid options while exposing retrieved evidence and decomposition traces, BOLT reduces hallucination and provides transparent decision-making, enabling large-model quality on edge robots.

Published as a conference paper at ICLR 2026
BOLT: DECISION-ALIGNER DISTILLATION AND BUDGET-AWARE ROUTING FOR CONSTRAINED MUL- TIMODAL QA ON ROBOTS
Tengjun Ni', Xin Yuan Shenghong Li Kai Wu', Ren Ping Liu', Wei Ni², Wenjie Zhang² University of Technology Sydney, Sydney, Australia Data61,, Commonwealth Scientific and Industrial Research Organization (CSIRO), Sydney, Australia University of New South Wales, Sydney, Australia "Edith Cowan University, Perth, Australia Trengjun.Mo Nun,Ni, Kai Kui RenPing. Liue@uts, letuuts.eduuer edu.au (xin.yuan, yuan, wei wei.nin rii..ii@eeee org shenghong. lis wenjie. zhang@unsw.edu.ea
ABSTRACT
Robotic systems can require multimodal reasoning under stringent constraints of latency, memory, and energy. Standard instruction tuning and token-level distil- lation fail to deliver decision quality, reliability, and interpretability under these constraints. We introduce BOLT, a peecision-alinned distillation and budget-aware routing framework that treats multi-choice prediction as a decision surface be aligned during training and selectively refined at inference. During training, BOLT introduces Option-level Decision Distillation to align student models directly on the decision surface of multi-choice answers, thereby eliminating prompt artifacts, improving calibration, and optimizing the exact output space. At inference, BOLT activates Budget-aware Test-time Augmentation, a calibrated router that uses low- cost signals such as confidence, margin, entropy, retrieval affinity, and agreement across short question decompositions trigger resouutin reevaluation, type- matched retrieval exemplars, or question decomposition only when their expected benefit outweighs cost. On Robo2VLM-1,,
2B BOLT student distilled from LLa ALaVA-1.5--13B improves accuracy from 28.66 in zero-shot to 42.89 with deci- sion distillation and 50.50 with budgeted routing, surpassing the 13B teacher 36.74. It lowers expected calibration error, strengthens the risk-coverage frontier, and slashes GPU memory from 26,878 MB for the teacher 3,035 MB for the distilled student, and 3,817 MB with all augmentations enabled. By constrain- ing outputs to valid options while exposing retrieved evidence and decomposition traces, BOLT reduces hallucination and provides transparent decision decision-making, abling large-model quality on edge robots.
INTRODUCTION
Multimodal foundation models have progressed rapidly from contrastive vision-language pretrain- ing to instruction followwng lanion-language models capable of grounded reasoning and multi-step perception (Radford et al. 202T] Jia et al. 202T). Systems, such LLaVA (Liu et al. 2023) and Qwen2-VI. LI. (Wang et al. 2024). demonstrate strong zero-/few-shot performance across diverse vi- sual question answering (VQA) tasks. There is growing interest in pushing these capabilities onto robots and embedded platforms. In parallel, many robotics benchmarks adopt constrained-output formulations (e.g., colors, arrow directions, options A-E, yes/no), which enable deterministic inter- faces and safety checks and are well suited to on-device control and real-time loops (Brohan et al. 2022 Chen et al. 2025 Gordon et al., 2018: Teney et al. 2018).
A practical challenge arises: achieving the decision quality of large vision- vision-languuan language models (VLMs) for constrained, multi-choice decision making while respecting strict latency, memory, and energy budgets edge hardware. Prior work takes several approaches to this goal. Token-level knowledge

---

Published as a conference paper at ICLR 2026
distillation (KD) inherited from text-only language models (LMs) seeks transfer teacher behavior at the sequence level (Hinton et al. 2015; Kim & Rush| 2016). Compact VLMs are directly fine-
tuned on instruction foolluctioon-oolooinngg data to better conform to task prompts. Always-on test-time enhance- ments such as higher-resolution re-evaluation and retrieval- -aageetel- prompting with same-domain exemplars aim to boost accuracy (Lewis et al. 2020 Rubin et al. (02T).
Selective prediction with budgeted or dynamic inference, covering confidence- -based abstention (Geifiman & El-ianiv 7017) as well as early-exit and adaptive-compute methods (Teerapittayanon
et 2016 Figurnov et al. 2017). trades coverage for risk under resource constraints. Parameter- QLoRA further reduce adaptation cost and memory (Hu et al. 2022] Dettmers et al. 2023). Beyond raw accuracy, many strategies pursue bet- ter interpretability and reduced hallucination, including retrieved-evidence provenance, decomposi- tion traces, and calibration, but most such methods incur substantially higher compute, memory, or latency when applied uniformly. However, optimizing explicitly for multi-choice decision surfaces remains limited, and few studies simultancously improve decision accuracy, increase interpretability, and mitigate hallucination under tight on-device resource constraints.
efficient adapters like LoRA and quantization-awee
These gaps manifest in constrained, multi-choice robotic perception as several persistent pain points. Token-level distillation aligns surface form under a particular prompt template, not the decision surface over the option set used in constrained decoding, which can make the student brittle and misaligned with evaluation. Always-on enhancements improve accuracy but increase latency and energy consumption, violating tight budgets; naive decomposition procedures may introduce spu- rious intermediate steps that diverge from visual evidence (Kim et al. (2020). Compact VLMs are commonly under-calibrated (Guo al. 2017). undermining selective computation and abstention.
Hallucination remains nontrivial in small models and is compounded by limited interpretability: it is often unclear why a decision was taken or which evidence supported it (Li et al. 2023b). Existing selective or budgeted inference seldom couples uncertainty with retrieval affimity or with agreement across decompositions, and evaluations rarely report risk-coverage or accuracy-budger frontiers for constrained multimodal QA. Panel-based layouts and tiny colored markers exacerbate small-model failures, while real-time control imposes per-frame budgets and VRAM ceilings. Taken together, these limitations leave optimization for the multi-choice decision surface underdeveloped; under tight on-device budgets, few methods simultaneously improve decision accuracy, increase interpretability, and mitigate hallucination.
Contributions We address the above-mentioned limitations with a decision-centric strategy that aligns the student with the teacher at the level of answer options and allocates additional test-time compute only when inexpensive signals indicate positive expected gain under a target budget. The training component performs decision- aediinnn- distillation at the option level so that the student learns the teacher's preference over candidate answers under constrained decoding. The inference component uses budgeted, risk-calibrated routing to decide whether to re-evaluate higher reso- lution, augment with retrieved same-domain exemplars, or invoke Question Decomposition (QD). This design not only improves ccuracy-budget tradeoffs and calibration, but also mitigates halluci- nation by constraining outputs to valid options and grounding with retrieved context, and enhances interpretability by exposing decomposition traces and the retrieved exemplars that inform decisions.
ceciisoon-centri framework for con- strained, multiple-choice VQA on robots L BOLT treats multiple-choice prediction as a decision surface be aligned and then selectively refined: training performs option-level distillation to match teacher-student preferences over answers, while inference uses a budgeted, risk-calibrated router that spends extra compute only when inexpensive signals suggest positive expected gain (e.g., high-resolution re-evaluation, type-matched retrieval, short QD). By unifying decision align- ment with selective computation, BOLT achieves large-model decision quality under tight on-device latency/iemmory/eeeryy budgets, delivering better socuracy-budger and risk-coverage trade-offs, sharper calibration, fewer hallucinations through constrained outputs and grounding, and clearer interpretability via visible evidence traces.
We introduce Budgeted Option-Leve Transfer (BOLT),
The key contributions of this paper are summarized as follows.
https://github..... wttmm//AAeeeea///eeeaaa
2

---

Published as a conference paper at ICLR 2026
Budgeted Option-Level Transfer (BOLT): A decision-centric framework for constrained, multi-choice VQA on robots that unifies option-level decision distillation (ODD) with bud- geted test-time augmentation (bTTA). By aligning training and inference around the multi- choice decision surface and spending compute only when inexpensive signals predict bene- fit, BOLT attains large-VILM VI.M decision quality under tight on-device latency/nemorryeennry budgets, improving accuracy-budget and risk-coverage frontiers and calibration while re- ducing hallucination and increasing interpretability.
Option-level Decision Distillation: A decision-aligned objective that matches teacher- student option distributions derived from answer-segment likelihoods, improving Exact- Match and calibration over token-level distillation on constrained multimodal QA
Budgeted Test-time Augmentation: A hisk-calibrated router that adapts inference compute (Hi-Res, retrieval augmentation, QD) per instance using uncertainty and retrieval- -ffiniit features; under mild monotonicity assumptions, this induces a thar-lh policy that optimizes accuracy subject to compute budget and empirically improves risk-coverage and accuracy-budget frontiers.
Mitigating hallucination and improving interpretability: By constraining outputs to valid option sets, grounding predictions with retrieved exemplars, and exposing QD traces, the framework reduces contradiciion-to-imae errors and invalid-option responses while pro- viding human-inspectable evidence chains; quantitative analysis appears in Section 4.6
Although our empirical study is conducted on a single panel-style robotic VQA benchmark, the set- ting it represents is not merely a toy benchmark. In many deployed robotic systems, perception and planning stacks reduce the world to a small set of discrete options. such as candidate grasps, poses, waypoints, high-level skills, or yes/no checks, and controllers are required to choose exactly one option under tight safety and latency constraints. Large-scale public datasets with this constrained, multimodal decision interface are still relatively scarce, especially in robotics, but the underlying pattern of option- based decisions is pervasive in industrial and household robots. Our focus on con- strained multi-choice QA should therefore be understood as modeling a realistic decision interface rather than overfitting to a single benchmark.
2
RELATED WORK
2.1
VISION-LANGUAGE MODELS FOR ROBOTICS AND CONSTRAINED-OUTFUS QA
Early vision- Lssion-anguaae pretraining focused on contrastive objectives that align images and texts in a shared embedding space (Radford et al. 202 I Jia et al. 2021). Subsequent instruction-followw
VLMs integrate autoregressive decoding and multimodll instruction tuning (c.g.. LLaVA (Liu et al,, 2023). Qwen2-VL. VL (Wang et al. 2024), BLIP-2 (Li al. (023a)), enabling strong shro- generalization on open-ended and daiken QA. In robotics, there is an increasing emphasis on models that interface with perception and control stacks under strict latency and energy constraints (e.g., RT- (Brohan et al.) 2022 Zitkovich et al. 2023). Many robotic and diagnostic VQA set- tings adopt constrained
to facilitate deterministic interfaces, safety checks, and reliable evaluation (Chen et al.] 2025; Gordon et al. 2018 Teney et al.) 2018). However, this regime exposes a specific gap: most training and adaptation practices remain token-level and open-ended, which misaligns with the option-based decision surface used at evaluation and tends to increase compute when applied uniformly on devices. We therefore study an option-aligned alternative and show that aligning option distributions better preserves decision quality for constrained decoding under on-device budgets.
formulati
2.2 KNOWLEDGE DISTILLATION FOR MULTIMODAL MODELS AND CONSTRAINED DECISIONS
Knowledge distillation (KD) was introduced as logit-matching with temperature scaling (Hinton et al. 2015) and later extended to sequence-level distillation for text generation (Kim & Rush 2016 In vision and NLP, distillation spans token/logit-base bassed matching, response-level training, and feature/attention transfer, often improving latency and memory without fully retaining calibra- tion or decision boundaries (Sanh al. 2019 Touvron al. 2021). Multimodal KD for VQA
3

---

Published as a conference paper at ICLR 2026
typically mirrors token-level or cross-entropy supervision, which entangle prompt-template idiosyncrasies with answer learning and may misalign with the constrained option space used at evaluation. Parameter-effiiiet tuning reduces adaptation cost and memory footprint (Hu et al. 2022 Dettmers et al., 2023), but does not by itself address decision alignment or budgeted test-time compute. Progress in distillation and parameter-efficient tuning notwithstanding, faithfully trans- ferring a teacher's option-level decision quality and calibration to compact VLMs for constrained multiple-choice decoding without overrunning on-device memory budgets remains elusive.
2.3
RETRIEVAL--AUGMENTIIN INFERENCE AND BUDGRRTED/DYYAMMN COMPUTE
Retrieval-augmented methods provide external evidence or exemplars to improve factuality and domain transfer, including RAG-style retrieval+generation, Fusion-in-Decoder, and memory- augmented language models (Lewis et al. 2020 Izacard & Grave, 2020 Borgeaud et al., 2022 Khandelwal et al. 2019 Rubin et al. 2021 exemplars for in-context guidance, but naive always-on use increases latency and energy and can degrade reliability when irrelevant evidence is injected. Separately, budgeted or dynamic inference studies how to adapt computation to instance difficulty: early-exit and conditional computation in routing, and selective prediction with abstention trade
vision-language QA, retrieval can supply task-type
ENNs/Transformers adaptive-depth off coverage for risk under resource constraints (Teerapittayanon
al. 2016 Figurnov al. 2017 Geifman & EI- Yaniv| 2017). What remains absent is cost-aware controller that, under an explicit compute budget, relies trustworthy instance-level signals to trigger high-cost augmentations only when they deliver positive net benefit.
2.4 CALIBRATION, HALLUCINATION, AND STRUCTURED DECOMPOSITION
Modern neural networks are often miscalibrated: post-hoc temperature scaling partially remedies this, but can be unstable across domains (Guo et al. 2017 Minderer et al. (021). }). In VLMs, multi- modal hallucination persists (e.g.. contradictions the image or invalid option outputs), and mea- suring/mitigating it remains an active area (Li et al. 2023b Rohrbach et al.. 2018). Interpretability tools range from retrieved-evidence provenance structured reasoning traces. Modular/structurred approaches, including neural module networks, program-like reasoning. and decomposition-styy prompting, seek to expose intermediate structure and reduce spurious correlations (Yi et al.) 2018) Hudson & Manning, 2018 Zhou et al., (0022). Despite techniques for post-hoc calibration, hallu- cination mitigation, and structured decomposition. it remains difficult to simultancously improve accuracy, calibration, and hallucination while preserving tight on-device budgets and avoiding uni- form overhead.
METHODOLOGY
3.1 PROBLEM SETUP AND NOTATION
We study constrained-outpur VQA for robotics. Each example is denoted as (z. O v), where is an image or panel layout, 9 natural-language question, O (oi,..., . ox is a finite option set, and y E fI...., K) is the ground-truth index set. We use large teacher VLM T and compact student VLM S, with parameters 0. Both are evaluated under constrained decoding: the model must output exactly one option text from O.
We a chat template that places (x,, q) in a user turn and the answer in an assistant turn. For option Ok, let the tokenized answer be a (a) (k) (k)
,...,,,,!!! and let the full sequence be (8) (Zi:to (kk) where indices ((k) (Lo + +1. I,..., Lo + LR} correspond to the answer segment. For model M, denote its next-token distribution by pas( (- z.:).
Answer-segment likelihood.
We deliberately score only the assistant answer segment:
sM
x q)
Σ (EA(()
1. L
(k)
zzt
(k)
This removes prompt-template wording from supervision and focuses exactly on the part that is evaluated under constrained decoding.
4

---

Published as a conference paper at ICLR 2026
Figure 1: Pipeline overview. Left: Distill phase (ODD). The teacher VLM provides option-level supervision via answer-segment likelihoods; the student learns to match the teacher's ' decision dis- tribution and is then calibrated. Right: Inference phase (bTTA). fast constrained pass produces confidence features; router selectively triggers HR re-evaluation, tmRAG, and QD and combines their outputs to form the final prediction under a compute budget.
Curataand
Opsons
Swedd Studien
H--M
Rentwaal
Decompnuee
Quastion
TescherVLM
Fax:
in deckire
Soudint Oppliie pictet
Los caker colerd enow
desciani vilii hau??
Hige
Hip
That Vilkor Maar
Direet Rosut
Data
Saved Student
Disiit Phose (OD0)
Innnnce (TTA))
Roadmap to test time. Having defined how decisions are formed under constrained decoding, first train the student so that its option-level decisions match the teacher, then design a budgeted router that decides when to pay for costlier augmentations at inference.
3.2 SYSTEM OVERVIEW AND DESIGN
As shown in Fig we design BOLT, a decision-centric framework for constrained multimodal QA on robots that delivers large-model decision quality small-model cost.
During training, BOLT utilizes Option-level Decision Distillation (ODD) to match the teacher's answer options, which computed from answer-segment likelihoods. A LoRA/QLoRA student is optimized with a small cross-entropy anchor, aligning the constrained- decoding decision surface, improving calibration, and mitigating prompt and length artifacts.
cumperature-softmat
During inference, a budgeted router reads lightweight signals from the student distribution aug- mented by type-matched retrieval affinity and agreement across short QDs. It triggers only the helpful augmentations under budget, namely HR high-resolution re-evaluation, tmRAG type- matched retrieval exemplars, and QD. This design concentrates computation where pays off, yielding stronger accuracy-budget and risk-coverage tradeoffs than always-on enhancements. reduces invalid-option and conaeeccntraiictioon errors by grounding predictions, and improves inter- pretability via retrieved evidence and decomposition traces. Temperature scaling further calibrates probabilities used for routing.
3.2.1 OPTION-LEVEL DECISION DISTILLATION (ODD)
To transfer the decision quality of large teacher to a compact student under constrained decoding, we propose ODD, decision- allinned objective that supervises the model the level of answer op- tions rather than tokens. Token-level KD from text LMs mixes prompt and answer tokens, penalizes template wording differences that are irrelevant evaluation, and encourages surface-font forrfoomm imita- tion; in constrained QA, the evaluation hinges on the teacher's preference over the option set. ODD therefore scores, for each option, only the assistant's answer segment, sums its token log-likelihoods to form per-option scores, converts them with a temperature-softmax into teacher option distribu- tion, and trains the student to match this distribution with a KL term plus a small cross-entropy to the ground-true -ruth option. This directly targets the decision surface realized by constrained decoding, improves calibration, and avoids the prompt-answer tug-of-war inherent to token-level distillation; when option strings differ markedly in length, light length-bias correction be applied.
5

---

Published as a conference paper at ICLR 2026
Option distributions and decision-aligned loss. We first turn the teacher's answer-segment scores into a calibrated preference over options by applying a temperature-softme which smooths overconfident peaks and exposes relative utilities across O:
exp(sT(k)/Tad
(r 2.9) Σラ (xp(s1 (j)/Taa
Tkd 0.
We cache {pr(k)} offline for all training items so that student training compares against a fixed teacher distribution without repeatedly querying the teacher. This cache is used only during offline training; at inference time the deployed system never queries the teacher and relies solely on the dis- tilled student. To make the teacher and student directly comparable in the same probability simplex, we build the student's s option distribution by normalizing its own answer-segment scores using the same softmax functional form (but without temperature scaling):
exp(s>(1))
(kkk x. x.q:0)
so(k) ssp (k: x.
(3)
Σ₁= exp(so(5))
This places both models' decisions common, prompt-invariant option space that mirrors strained decoding at evaluation.
With pr and ps defined, optimize a decision-aligned objective that pulls the entire student dis- tribution toward the teacher while retaining minimal anchor to ground truth:
(0) AKL (L
|| ps) + ACE CE(6, ps), AKL, ACE 0.
(4)
Intuitively, the KT term shapes the student's decision surface by matching the teacher's option pref- erences, and the small CE term provides a ground-truth anchor that stabilizes learning, preserves rare-option recall, and corrects teacher bias in ambiguous cases.
Invariances, length bias, and gradient shape. ODD operates on sums of answer-token pogpprbbabilities, invariant to adding any constant all so skk).. and is robust benign tokeniza- tion changes for fixed option strings. This robustness comes from the fact that splitting or merging tokens within an option only changes how the sequence log-likelihood is decomposed over time;
the total sum Aee log p(a; a remains a consistent measure of that option's probability under given model, and our loss always compares options within the same model and tokenization. When option strings differ substantially in length, we correct the scores by
(k) (kk y log Lg or so(k) Lk,
Y E [0, 1].
(5)
For intuition about decision alignment, let so (se())....,s((())) respect to Sy are given by
so(K]] The gradients of Eq. (4) with
V (L Ps Pr.
Va, CE(, Ps Ky.
so the total signal pushes the student option distribution toward the teacher and the ground-truth anchor. When backpropagated token logits, this supervision is applied only to segment tokens, avoiding the wrompt-answed tug-of-war wa- that plagues token-level distillation.
Parameter-efficient training. We train LoRA adapters in attention and MLP projections (option- ally the multimodal projector) while keeping the base quantized with QLoRA. ODD gradients flow only through adapter paths and the projector, enabling single-GPU training.
3.2.2 BUDGETED TEST-TIME AUGMENTATION (BTTA)
To maximize accuracy under an explicit compute budget while preserving low latency on edge hardware, we propose bTTA, an adaptive inference-time framework that allocates computation per instance. The distilled student first performs a fast constrained pass, from which we derive a compact routing feature vector based on the option distribution and auxiliary cues. A learned policy decides whether to execute high-resolution (HR) re-evaluation, type-mate type-matched retrieval exemplars (tmRAG), Question Decomposition (QD). Each augmentation is modeled and a learned success probability; bTTA triggers actions only when the predicted gain exceeds the cost within the budget, yielding calibrated final decisions and improved accuracy-budee -uuget tradeoffs.
action with measurable cost

---

Published as a conference paper at ICLR 2026
Routing features and policy. margin, and entropy,
From the pass-1 option distribution ps compute confidence,
Piax max (k),,
A P()) P(2)
H Eps(k) log Ps(k).
We augment them with a retrieval affinity
Σ
cos( 4), o(z), 9))).
P
Kr jETop-K,
where the memory stores only same-type items to avoid cross-type interference and, with an agree- ment score across short QD, runs
Σ kek
JS(P' (2)
1 ))
K Ka(Ka -
which increases when independent decompositions concur. The feature vector f [maax A, H, p, n] drives the router. Let actions A {HR, RAG, QD} collect HR, tmRAG, and QD. Each action has a normalized marginal cost C. and a binary improvement label AAce, rela- tive to the current prediction. A gain model gw a) xs Pr(AAcc, logs, and the per-instance decision solves
f] is learned on validation
Σ
aa 9n (f,
Eaa C. B,
On (0,1)
aEA
aEA
with small empirical weights Wa and budget B. Its Lagrangian relaxation yields simple near- threshold rule
trigger a = (f, Wa 2 Ca: with cumulative cost S B,
which activates augmentation when predicted gain exceeds cost by threshold r tuned on valida- tion to satisfy the average budget and maximize accuracy. With diminishing returns across actions, greedy selection by this net value is strong approximation.
Actions and calibration. HR replaces the image with a larger short-edge for second constrained pass recover fine detail. mRRG retrieves Top-K, same-type exemplars (desc, q, a) by cosine similarity in an encoder space and appends them to the prompt; the student re-answers to obtain ps RAG. QD elicits Ka short decompositions (two-three checks) with diversity via seeds or few-shot permutations, producing distributions {ps Jkmi and aggregated vote
(k) K2
ps(-)
1
Ps kK)
(+).
arg max (o)
Kd
while the agreement feeds back into routing suppress unhelpful decompositions, Because rout- ing relies on probabilities, we apply temperature scaling on a validation split,
exp (kk)/T))
ps (k)
Exep
expl
and reuse calibrated Prax. margin and entropy in Appendix
EXPERIMENTS
4.1 EXPERIMENT SETUP
We conduct the main study Robo2VLM-1 (Chen et al.) 2025).
panel-style robotic perception
QA benchmark with constrained option sets. Following our constrained decoding interface (Sec. 33) evaluate by Accuracy (Acc) over options. For Robo2VLM-1. splits by unique image identifiers: train-kd (used only to build the teacher aption-distributin cache), val (router calibration and temperature scaling), and test (final reporting). Retrieval
form three non-overlapping
7

---

Published as a conference paper at ICLR 2026
memories and decomposition exemplars constructed exclusively from train-kd to avoid leak- age into val/test. Unless otherwise stated, all results are single-image, batch size 1.
We distill from three teachers spanning families and sizes, Qwen2.5-VL-7B. LLaVA-1.5-7B. and LLaVA-I.5-13B, using their per-option answer-segment likelihoods to form temperature-sote softmmx teacher distributions pr for ODD. The student is Qwen2- -L--VB-----B------ LoRA/QLORA, To assess cross-architecture generality, we also report results for a PaliGemma-2:: student distilled from LLaVA-1.5-13B, All models share the same constrained-decoding interface and bTTA configuration for fair comparison. Unless otherwise specified, we report each teacher- student setting with Tkd and the CE weight tuned on val.
trained with ODD via
To isolate the effect of decision-alignes distillation, we also implement a strong token-level KD baseline. Specifically, the teacher's next-token logits over the full answer sequence (including both prompt and answer tokens) supervise the student via a temperature-scaled KL term plus a standard cross-entropy on ground-truth tokens. In all result tables, rows marked as "Token- KD" correspond to this baseline; rows marked as "ODD" replace token-level supervision with our option-level decision distillation while keeping other settings identical.
4.2
MAIN RESULTS
Table 2 summarizes accuracy under a unified constrained-decodine protocol. 2B student trained with option-level decision distillation attains 42.89% when distilled from LLaVA-115--33 exceed- ing the 13B teacher at 36.74% and the 2B zero-shot baseline at 28.66% by sizable margins. Adding the budgeted test-time augmentation policy further increases accuracy to 50.50, while maintaining the same decoding interface. Replacing ODD with token-level KD yields lower accuracy across teachers: 33.91 9133.221%/333 5/37.588% when distilled from 2.2V-777BQ0w222.55LL--B111L. points, respectively. With bTTA, token-level KD improves to 39.92/44.42/44 44/444.244702 02 but still lags behind ODD+bTTA at 45.43% wtt.1666500000 The pattern is con- sistent across other teachers: distillation from LLaVA- yields 38.53% and rises to 45.43% with the policy, and distillation from Qwen2.5-VL-77B yields 40.52% and rises to 47.16%. These re- sults indicate that aligning decisions at the option level closes most of the capacity gap, and instance- adaptive compute converts residual uncertainty into additional accuracy without changing the model architecture or the evaluation protocol. For completeness, NLL, Brier, ECE, and AURC, together with the full ablation breakdown, are reported in Appendix Table 9
trailing ODD by 4.62/4.31/5.3
4.3 ABLATIONS
Table T examines the contribution of each inference-time component to the student distilled from LLaVA-I.5-13B. High-resolution re-evaluation primarily fixes resolution- errors and moves accuracy from 42.89% to 46.64%. Type-matched retrieval adds fomain-appropee moves accuracy to 44.31 % when used alone, and 48.25% when combined with high resolu- tion. QD reduces reasoning variance on difficult cases and moves accuracy to 45.47% alone, and to 48.92% together with high resolution. Enabling all three components achieves 50.50%. The gains are monotone and nearly additive, supporting the design of a router that triggers only the actions whose predicted gain exceeds their cost.
exemplars and
4.4 BUDGETED ROUTING: ACCURACY- ECCURACY-BUDGET BEHAVIOR AND BASELINES
We study how the budgeted router allocates test-time compute and how this impacts accuracy un- der an explicit average budget B for +DD+ Each optional augmentation is treated as a bi- nary action with fixed per-trigger costs (CIR: CIBRAG, Cop) = (0.50, 0.30, 0.35), on top of a base constrained pass costing 1.00. Sweeping calibrated threshold yields discrete eccuracy-budger frontier (Appx. Fig. 10 Accuracy improves monotonically from the single-pass ODD baseline at B-1.00 full- full-budget setting around B==200,, where gains saturate.
Trigger compositions increase smoothly with B (Appx. Table 4). Feature- centur--mmportncce diagnostics on the learned gain model show that HR is primarily gated by entropy H, tmRAG by same-type retrieval affinity p, and QD by agreement x across short decompositions, consistent with the router design in Sec. 3.2.2
8

---

Published as a conference paper at ICLR 2026
Table Module ablations on Robo2VLM-1 (Acc %). Columns indicate whether HR (High- Resolution), tmRAG (type-matched retrieval ex- emplars), and QD (Question Decomposition) are enabled.
Table 2: Comparison on Robo2VLM-1 (Acc %). All methods share the same constrained decoding and prompt template.
Mudel
Parame
LLaVA izero
LLeVA 1.5-138 Reacher, sere-hoot)
13
36.74
Vastant
ImiKA
QD
(9)
Qwen2 VL 2B (2ero shot)
28.66
Qwen2 VL-2B (peroshot)
N
N
N
28.66
Toacker: LLAVA-1.5-77 Student: Qwen2 VL-2B
Qwen2 VL-2B dismiled LLaVA-13B (ODD)
N
N
N
42.89
dusille by LLaVA-78
+QD
N
N
Y
45.47
Qwen2 V1-28 duailled by 1LVV--76 (Ours, ODD)
38.55
HR
Y
N
46.64
BTTA
45.43
Teacher: Qwee2.5 VL Studeat: Qwes2 VL
IIR QQ
Y
N
Y
48.92
Owen2 distlled Qwez2.5 (Teken
HR tmRAG QD
Y
Y
Y
$0.50
+ STTA
40.42
Qwen2 VL 28 distlled by Qwes2.5 VL-7B (Oxrs, ODD) bTTA
40.52
47.06
L-VVA-1---188 Studen: PaliCcmmaa--38
PaliGemms2 dimilled LLaVA-13B (Token-KD)
+ -TTTA
40.21
PaiCianmm2-3B dinilled LLaVA-138 (Ours, ODD)
39.64
BTTA
LLAVA-1.5-120
VL-2B
Owen2 VL-2B distileed by LLAVA 13B (Token KD)
37.55
+ BTTA
47.002
Qwen2 VL-2B disnlled LLaVA-13B (Oun, ODD)
42.89
-TTTA
Under matched budgets, compare to two single-signal dynamic-compute baselines (HR-Thress TuR-Thheshold and one-branch Early-Exit). Our router yields +~1.4-222 points accuracy and lower ECE/AURC by deciding both whether to escalate and which augmentation to trigger per instance (Appx. Table more details are provided in Appx.
4.5 SYSTEM FOOTPRINT AND DEPLOYABILITY
Appx. Table Ia reports GPU memory with batch size one. The distilled 2B student occupies 3,035 MB. which reduces memory by 88.7% relative the 13B teacher at 26,878 MB and by 41.0% relative to the 2B zero-shot baseline at 5, 5,144 MB. Activating the full set of augmentations raises the footprint to 3,817 MB, adding only 782 MB and remaining about seven times lighter than the 13B teacher. In conjunction with the accuracy improvements in Table 2 this footprint makes the pipeline practical for edge deployment, since the model fits within commodity GPU memory while delivering higher accuracy than much larger teacher.
For the LLLVA-1155-33B to Qwen2-VL. VL-2BB setting, the end-to-end mean latency per question is 8.97 Of this, the bTTA pipeline (routing + selectively triggered HR/mRAG//DD contributes only 1.52 s on average, leaving 7.45 s for the first constrained pass (ODD student). Thus, bTTA accounts for 16.9% of total latency, while the pass-1 accounts for 83.1%. The effective throughput is 6.69 items/min (0. 1 QPS); see Table [1b]
For reference, the 13B LLaVA teacher itself achieves 36.74% accuracy Robo2VLM-1 with an average per-question latency of 22.30 s and an energy cost of 8,697 on the same hardware. Our full BOLT pipeline (ODD student bTTA) reaches 50.50% accuracy with 8.97 latency and 1,522J per query, corresponding to roughly a 2.5x speedup and an 82.5% reduction in energy while im- proving accuracy by 13.8 points. This operating point is suitable for high-level robotic perception and planning loops, whereas strict real-time low-level control would require further engineering and additional optimization, which we leave for future work.
4.6 HALLUCINATION ANALYSIS
We assess hallucination under constrained decoding using six proxies (IOR, NOA, Flip. HO [00_menn_wron OCW and augmenttaiion-conditonnee contradiction rates RCR/QDC). The constrained interface removes string-form hallucinations outright (IOR= 0), while decision-level misuse of the "None of the above" sentinel is substantially reduced from 1.08% (zero-shot) to 0.37% (ODD pass-l) and 0.22% (ODD+bTTA). The router actively corrects uncertain cases, 26.71% % of labels flip relative to pass-1, yielding fewer over-confident mistakes (OCW@0.7

---

Published as a conference paper at ICLR 2026
4. 4.18%+4.27%-000 despite slight rise in average confidence remaining errors (HO_mean_ww) meaa_wrong (0778~0.2294), augmentation-condiite retrieved exemplars can conflict with image-consistent answers (RCR 21.73%), while gated QD is largely but not perfectly self-consistent (QDC 1.74%). BOLT eliminates string-level hallucinations by design, curbs NOA misuse, and suppresses the high-confidence error tail; remaining errors primarily driven by retrieval quality and decomposition policy. More details are in Appendix H
diagnostics localize residual risks:
LIMITATIONS AND FUTURE WORK
Our evaluation focuses on a single panel-style robotic VQA benchmark (Robo2VL.MII) reflecting the broader limitation that existing robotic multi-choice VQA datasets are scarce. Although our ex- periments focus on robotic perception benchmark, the ODD objective and the budgeted router are modaliity-gnnosiie and can be applied to other multimodal multi-choice QA settings where the an- swer space is relatively uniform and constrained, as long as these settings also care about latency or model size. The tmRAG component induces retrieval-driven hallucinations (RCR 21.73%), which a common limitation of RAG pipelines when exemplars imperfectly match the query. In the fu- ture, we will (i) expand to multiple public datasets and report per-type risk-coverage and calibration, (ii) replace or complement tmRAG with conflict-aware exemplar filtering, visual-entailment style option verification, and retrieval-free self-consistency voting, (iii) optimize the router and student under compute-awan objectives (latency/enerry) to improve calibration and robustness under dis- tribution shift, and (iv) move beyond purely offline supervised distillation by exploring on on-policy decision distillation for constrained VLMs using logged robot interactions and simulated rollouts.
CONCLUSION
This work reframes constrained multimodal question answering for robotics as the joint problem of option-level decision alignment and on-demand inference, and demonstrates that aligning the stu- dent to the teacher in the option space used at evaluation, then spending computation only when inexpensive signals predict benefit, yields compact system that is both accurate and reliable under tight resource budgets. BOLT transfers the teacher decision surface through Option-level Deci- sion Distillation and concentrates computation via calibrated, budgeted router, leading substan- tial gains in accuracy, calibration, and interpretability without changing the interface or inflating memory, with a 2B student exceeding 13B teacher on Robo2VLM-1 and showing stronger risk- coverage and accuracy-budee trade-offs along with lower invalid-option and image-contradiction errors. Future extensions include generalizing option-level alignment to broader structured outputs and multi-step control, strengthening retrieval filtering and consistency modeling while optimizing the router and the student jointly, and making energy and latency explicit objectives in end-to-end training followed by comprehensive evaluation across diverse robotic tasks and public benchmarks to promote reproducible assessment and responsible deployment.
REFERENCES
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, -ean-Baptiste Lespiau. Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. International conference on machine learning, pp. 2246-2240. PMLR, 2022.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, al. Rt-1: Robotics transformer for wall-wrrld control at scale. arXiv preprint arXiv:22 2022.
Kaiyuan Chen, Shuangyu Xie, Zehan Ma, Pannag R Sanketi, and Ken Goldberg. Robo2vlm: Vi- sual question answering from large-scale in-the-wild robot manipulation datasets. arXiv preprint 0rXiv:2505.15517, 2025.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36: 1110008800011 15, 2023.
10

---

Published as a conference paper at ICLR 2026
Michael Figurnov, Maxwell D Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, and Ruslan Salakhutdinov. Spatially adaptive computation time for residual networks. Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1039-1048, 2017.
Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks, Advances in neural information processing systems, 30, 2017.
Daniel Gordon, Aniruddha Kembhavi, Mohammad Rastegari, Joseph Redmon, Dieter Fox, and Ali Farhadi. Iqa: Visual question answering interactive environments. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4089 4098, 2018.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint 1.03iv:5503.00 025311, 2015.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3 2022.
Drew A Hudson and Christopher D Manning. Compositional attention networks for machine rea- soning. arXiv preprint 1803.00377 2018.
Gautier Izacard and Edouard Grave, Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arxiv:2007.01222. 2020.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference machine learning, pp. 4904-4916. PMLR, 2021.
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. arXiv preprint 2019.
Ildoo Kim, Younghoon Kim, and Sungwoong Kim. Learning loss for test-time augmentation. Ad- vances neural information processing systems, 33:4163-41744 2020.
Yoon Kim and Alexander Rush. Sequence- knowledge distillation. In Proceedings of the 2016 conference on empirical methods in natural language processing, pp. 1317-1327, 2016.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented gener- ation for wewwwledg-inntnssine nlp tasks. Advances in neural information processing systems, 33: 2020.
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference machine learning, pp. 19730-19742. PMLR, 2023a.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large laniun-language models. arXiv preprint arXiv:2305. 10355, 2023b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916. 2023.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. Advances in neural information processing systems, 34:1 5682-15694, 2021.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PmLR, 2021.
11

---

Published as a conference paper at ICLR 2026
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination image captioning. arXiv preprint arXiv:1809.02156, 2018.
Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.0883: 2021.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv: 108, 2019.
Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung. Branchynet: Fast inference via early exiting from deep neural networks. In 2016 23rd international conference on pattern recognition (ICPR), pp. 2464-2469. IEEE, 2016.
Damien Teney, Peter Anderson, Xiaodong He. and Anton Van Den Hengel. Tips and tricks for visual question answering: Learnings from the 2017 challenge. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2223-4232, 2018
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pp. 10347-10357. PMLR, 2021.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai. Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, et al. Qwen2-vl: Enhancing rision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. /2/9/, 2024.
Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Josh Tenenbaum. Neural- symbolic vqa: Disentangling reasoning from vision and language understanding. Advances in neural information processing systems, 31, 2018.
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuur- mans, Claire Cui, Olivier Bousquet, Quoc Le, et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205. 10625, 2022.
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision language-guutttenn models transfer web knowledge to robotic control. In Conference on Robot Learning, pp. 2165-2183, PMLR, 2023.
A ADDITIONAL DERIVATIONS AND DETAILS
A.I MASKED LIKELIHOOD ON ANSWER SEGMENT
Let (k) = (z), (z1,...,,y 21. and mask my = T[t E ((). The masked NLL is
C ALL (0) Σ my log (= 2x0))
Σ
log pa zer)).
(k) By definition (k) (( NNL (0). In practice, set labels outside the answer segment -100 to exclude them from loss.
A.2 GRADIENTS FOR ODD AND TOKEN-LEVEL BACKPROP
Let S9 = (1o( [K)) Ps = softmax(so). For fixed pr, we have
2 OKL(PTIPs) -p(()) - Pr()),
à @CE(,, ps(j) - -[(j-n) aso(j)
(10)
Osa
Thus
aLODD AkL(ps(i) pr()) + AcE(ps(s) 1lj=vl). Oso(j)
(1I)
12

---

Published as a conference paper at ICLR 2026
Each (J) is sum of answer-token log-probs. Let e be pre-softmax logits and y be the gold token at time t. For & A Osp 0e. Ey, (4ftmaxx(4))
Oso(i) O for / A0) ae.
(12)
Hence, ODD produces an option-weighted learning signal focused on answer tokens, unlike loken-level KD, which distributes weight over prompt tokens as well.
A.3 TEMPERATURE, LABEL SMOOTHING, AND LENGTH CORRECTION
Teacher temperature Tha controls the softness of pr: we select Tua E [1.5,3) We optionally use label smoothing the CE term: (1((1=)),,+=//b ps). stabilizing optimization when pr sharp. Length correction in equation mitigates rare cases where L₁ differs across options.
A.4 ROUTER LEARNING AS CONTEXTUAL BANDITS
We cast routing as a contextual bandit with context f and actions a E A producing binary reward AAcc, and cost Ca. We collect offline log held-out split by executing pass-1, then all aug- mentations (for supervision only), recording (f, a, AAcc. Ca). We train a probabilistic classifier (f, a) Pr[AAcc, 1/ʃ] with class-imbalance weights and temperature calibration. At test time, we apply the near-threshold rule in equation 7 When logs are collected under a different pol- icy, an unbiased estimator of the expected gain can be formed by inverse propensity scores (IPS): E[A Acc [f] A|cou]AAAco, where ı the logging propensity; we find that direct supervised gw is (foof)
sufficiently stable in our setting.
A.5 LAGRANGIAN VIEW, THRESHOLD POLICY, AND SEQUENTIAL AUGMENTATIONS
The constrained problem, i.e., ( has Lagrangian L(a, = E. aa (E|AAAc,, -C,) + rB. For fixed т, the maximizer Ca 1 iff the net value is nonnegative, giving the near-threshold rule in (7). When applying multiple augmentations sequentially, if the expected improvement exhibits diminishing returns (submodularity) over the action set, the greedy selection that iteratively picks the largest positive net value achieves a (1 1/e) appaaopprxiiiitiion to the optimal set (Nemhauser bound). Empirically, can cap the number of augmentation rounds (e.g., one or two) to keep latency predictable.
A.6 AGREEMENT METRICS AND ALTERNATIVES
Beyond 6 via average pairwise Jensen-Shannon similarity, we consider: (i) majority-vote rate among (arg maxpss
(k) ]: (ii) Kendall's W option rankings; (iii) variance of (p& (k) along the top
eigenvector. We use & for differentiability ablations and majority-vote for reporting.
A.7 SELECTIVE RISK, RISK-COVERAN AND CALIBRATION
Given confidence score e (e.g., Pmax) and threshold т., the selective classifier accepts items with 2 To. Coverage is IP(c Ta) and selective risk is the error rate accepted items. The under the risk-coverage curve (AURC) summarizes the trade-off. We estimate ECE on the accepted set using fixed bins and report Brier score calibrated pal, post-hoc Teal is fitted by minimizing NLL on a validation split not used for routing.
A.8 COMPLEXITY AND MEMORY
Training. ODD needs forward passes per sample; with batching across options the per-step complexity is O(K- Lans d) where Luns is answer length (small) and d the model width. Only LoRA parameters and (optionally) the projector are trainable; memory scales as is LoRA rank. Inference. Pass- dominates little; Hi-Res adds factor the vision backbone; retrieval cost is OKK, ds) per query (pre-encoded memory); QD repeats decoding K₂₃ times with short outputs. We cap (Kr, Ka) (e.g., and 3).
where
13

---

Published as a conference paper at ICLR 2026
A.9 IMPLEMENTATION NOTES
(i) Template alignment. Use the same template for teacher and student; verify Lo by tokenizing prompt-only vs. prompttanswer. (ii) Quantization and merge. If merging LoRA into full model for deployment, reload the base in fp16/bf16 (not 4-bit) before merge. (iii) Calibration reuse. Fit Teal once on validation and reuse across ablations to avoid information leakage. (iv) Numerical subtract max; sr before softmax.
stability. When computing equation 2
B ALGORITHMS (PSEUDO-CODE)
KD cache construction (teacher).
Require: dataset Drain teacher T,, temperature 7ld
1: for each (z, 9, O. E Durin do
2: for each oę E O do
3: Build (²) compute sT()) by equation]
4: end for
5: (T()) & exp(ss (k))
6: Write JSONL: (image, question, options, gt-idx, teacher.prooss]
end for
Student training (ODD + LORA/QLORA).
Require: KD cache, student Sa. λKI., ACE
1: while not converged do
2: Sample minibatch B
3: for each (x, q, O, M, pr) in do
4: for each oş E O do
5: compute 80 (k) by equation]
6: end for
7: form ps by equation 3 compute LODD by equation
8: end for
9: Update LoRA (and projector) parameters via AdamW
end while
Inference with budgeted routing (bTTA).
Require: instance (I, g, O), student Sp, router Iws costs C., budget B, threshold
1: Pass-l: get ps and features [pmaxs A H, K,....]
2: R + 0; Brem B
3: for at most two rounds do A with C, Brems compute score u. Iw (f, 5: if all ua < then
4: For each
6: break
7: end if
8:
+ arg max ug; apply at: + R {a*}; Brem Brem Ca. Update ps. f (and memory if needed)
9:
end for
11: Return final prediction by majority over R U {pass-1} or max calibrated Pmax
C DATASET ANALYSIS
Overview. Robo2VLM-1 is panel-style robotic perception QA corpus with constrained option sets. The original release contains approximately 678k training items and 6.68k test items.
Our splits. We form three non-overlapping splits by unigue image identifier to avoid leakage across partitions: a train-kd split used exclusively to build the teacher option-distributin cache and to train the ODD student, val split for router calibration and temperature scaling, and the official test split for final reporting. Concretely, we subsample 6. 78k items from the original 678k training pool as va and use the remainder for train-ke
14

---

Published as a conference paper at ICLR 2026
Figure 2: Letters
Q: The robot's task is to the blue disc in the white cup and orange discs in the clear cup. Which configuration shows the goal state that the robot should achieve?
C: ['Configuration A; 'Configuration B, 'Configuration C; 'Configuration D; 'Configuration E]]
A: Configuration A
Figure 3: Colours
Q: In the image from ext2, which colored point FARTHEST from the camera?
C: ['None of the above; 'Purple' 'Red', 'Blue, 'Green')
A: None of the above
Category. Robo2VLM-1 contains five constrained families: Fig. 2 Letttrs (choose A-E that matches the goal panel), Fig. Colours (select the coloured point by distance rule), Fig. 4 Ar- rows (choose the colored arrow that satisfies the described relation). Fig. Instructions (pick the option consistent with short instruction and the scene), and Fig. Yes/No (queries about robot or scene state).
D CASE STUDY
We illustrate how ODD + bTTA behaves on three representative scenarios under constrained de- coding. For each case, we show the input image, the router's chosen augmentation(s), and short rationale.
Case A Letters. The first pass is indecisive because several candidate panels share almost iden- tical geometry and object placements. The router predicts that both higher spatial detail and struc- tured checks could break ties, and triggers Hi-Res together with tmRAG (same-type exemplars) and QD. tmRAG nudges the model toward discriminative cues (e.g., specific contour junetions and rel- ative offsets), while QD prunes inconsistent options via few short checks. The subsequent Hi-Res re-evaluation clarifies fine edges and small gaps, aligning all signals single panel and fixing the initial mistake.
Case B Arrows. At the default resolution, multiple elongated structures resemble arrow shafts heads. The router detects that uncertainty mainly stems from local visual detail rather than miss- ing priors, so it triggers only Hi-Res. Higher short-edge resolution makes the arrowhead orientation and shaft continuity unambiguous; the decision flips the correct option without requiring retrieval or decomposition.
Case - Colours. The first pass confuses nearby colored markers due to tiny blobs, slanted edges, and mild occlusion. The router fires Hi-Res to sharpen boundaries and tmRAG to anchor color naming with in-domain exemplars. Although edges become cleaner, the retrieved exemplars do not fully match the viewpoint and lighting, and residual aliasing between adjacent markers remains. The final choice stays incorrect, exposing a failure mode where HR+tmAAG insufficient without additional structural check (e.g., verifying spatial constraints) stronger color normalization.
15

---

Published as a conference paper at ICLR 2026
Figure 4: Arrows
Q: The robot task take the blue cup from the left and move to right across the book. Which colored arrow correctly shows the direction the robot will move next?
C: ['Purple; 'None of the above; 'Yellow, 'Blue', 'Green')
A: None of the above
Figure 5: Instructions
image
Q: The robot is tasked to close bottom drawer. The robot is interacting with the drawer. Which phase of the grasp action is shown in the image?
C: ['Firmly grasping the drawer, 'Releasing the drawer by opening gripper', 'Approaching the drawer with open gripper', 'Closing gripper to grasp the drawer,, 'Moving away with open gripper after releasing the drawer')
A: Closing gripper to grasp the drawer
E RETRIEVE-AUGMENTE INFERENCE ANALYSIS
We report the fraction of evaluation items for which the tmRAG was actually executed ("hit rate"). The per-type usage on Robo2VLM-1 is: colors 18.57%, arrows 22.18%, letters 15.28%, yes/no 19.51%, and instructions 4.48%, the overall tmRAG usage equals their sum, 80.02% of all items (Table 3.
tmRAG yields the least utility instruction-type questions, evidenced by the router rarely trig- gering it (only 4.48%). This aligns with the nature of such items: the decision hinges more on following short procedural rule or template than on recalling domain exemplars. Two factors sup- press tmRAG's gain here: (i) instruction phrasings vary widely across panels, lowering retrieval affinity and increasing the risk that injected exemplars are off-pattern; (ii) when exemplars do not structurally match the required steps, they can introduce irrelevant cues and dilute the option dis- tribution. In contrast, arrows (22.18% (2%) and, to a lesser extent, yes/no and colors benefit more from tmRAG because type-consistent exemplars provide stable visual-verbal anchors (e.g.. orientation disambiguation color naming) that the student can reliably reuse under constrained decoding. Prac- tically, we gate tmRAG for instruction-type items more conservatively and prefer QD or HR when the router predicts higher net gain.
F ROUTING DIAGNOSTICS AND BUDGET SENSITIVITY
We analyze how the router allocates test-time compute under an explicit average budget B for ODD bTTA and how this translates into accuracy. Actions binary with fixed per-trigger costs (CHR, CIRRAG, CQD) = (0.50, 0.30, 0.35), and the base constrained pass costs 1.00. Sweeping the calibrated threshold changes the trigger composition, yielding discrete accuracy-budget frontier. The average budget
B 1.0 + 0.50 THig 0.30 Trig(tmRAG) + 0.35 Trig(QD),
where Trig(a) is the fraction of items for which action a fires. We report six operating points B E {1.00, 1.17, 1.29, 1.53, 1.95, 2.00), chosen to align with the empirical quantiles of HR usage (approximately 0/25/40/60/90%) and round budget cap 2.00.
Budget-accuracy frontier. Figure plots accuracy versus B. bTTA concentrates compute on in- stances predicted to benefit, raising accuracy from 42.9% B=1.00 (ODD only) to 46. 147.3/49..99
16

---

Published as a conference paper at ICLR 2026
Figure 6: Yes/No
image
Q: robot's gripper open?
C: ['Partially open' 'Yes', 'No; 'Cannot be determined]]
A: Yes
Q: The robot's task is remove the black pen from the white mug. Which configuration shows the goal state that the robot should achieve?
C: ['Configuration D; 'Configuration B' 'Configuration E;
'None of the above', 'Configuration C*]
GT: None of the above
ODD result: Configuration D Wrong
bTTA choice route: RAG+QD+Hi-Res
Full result: None of the above v Correct
Question Decomposition:
1) Are the target objects correctly identified (a black pen and white mug)?
Ans: Yes
2) the black pen completely outside the mug (nothing crossing the plane)?
3) Is the mug empty and upright, with the pen no longer contacting the mug's inner cavity?
Ans:
Figure 7:
Letters. Select the panel (A-E) whose layout best matches the query view; several candi- dates appear near-duplicate at default resolution.
at B=1.17/1.29) 19.5/ and further to 50.4% at B-1.95 and 50.5% % at B-2.00. Gains saturate as easy wins exhausted.
Trigger compositions. Table shows the trigger rates that realize each budget; all increase mono- tonically. tmRAG fires when same-type affinity is high, HR when uncertainty is high, and QD when short decompositions agree, matching Sec. 3.2.2
Feature importance. A calibrated logistic gain model confirms that HR is most sensitive to en- tropy H. tmRAG same-type affinity p. and QD agreement K; consistent with the design (details in Appendix).
G ROUTING BASELINES: CONFIDENCE THRESHOLD AND EARLY EXIT
We next compare our budgeted router against two classic tynamic-compute policies under the same average budget (base pass costs 1.00; actions: HR=0.50). The HR- UR--hreshold baseline fires Hi- Res if Pmax Tp: the Early-Exil (one-branch) baseline accepts the pass-1 decision if the margin A exceeds TA and otherwise runs Hi-Res. Table reports accuracy and calibration at three budgets aligned with sweep in $F Our router consistently outperforms single-signal gating by +1.4-2.2 Ace at matched B, while also lowering ECE and AURC. The gains stem from exploiting multiple inexpensive signals (entropy, margin), retrieval affinity, and agreement across short decompositions to decide when and what to spend.
At equal average budgets, single-trigger policies that only escalate Hi-Res leave accuracy the table and remain less calibrated. Our multi milti-signal router provides consistent gains by tailoring both depth (whether to escalate) and type (HR vs. retrieval vs. QD) of extra compute per instance.
17

---

Published as a conference paper at ICLR 2026
task remove black from grey pot on the stove and place inside the sink. Which colored arrow correctly shows the direction the robot will next?
C: ["Ylloww, 'Purple,, 'Red', 'Blue,, 'Green']
GT: Red
ODD result:Blue Wrong btTA choice route: Only Hi-Res Full result: Red Correct
Figure 8: Arrows Decide which colored arrow satisfies the described direction/relation in a clut- tered kitchen scene.
Q: In the image from ext2, which colored point is FARTHEST from the camera?
C: ["Purple', 'Green', 'Yellow; 'Blue; 'Red']
GT: Green
ODD result:Purple- X Wrong btTA choice route: RAG+Hi-Res Full result: Yellow X Wrong
Figure 9: Colours. Identify the correct colored target in tilted view where small markers and partial occlusions make boundaries ambiguous.
H HALLUCINATION ANALYSIS
We quantify different facets of hallucination under constrained decoding using six proxies: (i) Invalid-Option Rate (IOR): fraction outputs not in the allowed option set; (ii) None-of-the-Abow misuse (NOA): fraction of cases where "None of the above" is predicted but contradicted by the image/goal; (iii) Flip: fraction of examples whose final label differs from the first-pass label (a measure of how often the router forces change); (iv) Mean Pmax wrong (HO mean wrong: average confidence on incorrect predictions; (v) Over-Confiden Wrong @ 0.7 (OCW@D @0.7): share of wrong predictions with Pmax 0.7: and (vi) contradiction rates tied to optional augmentations: RCR (retrieval-comtradictto) rate, share of routed-to-RAG cases where retrieved exemplars point to label inconsistent with the image-consistent answer) and QDC (QD-contradiction. share of QD runs whose intermediate checks conflict with the final option).
Table 6 shows that the constrained interface eliminates string-form hallucinations (IOR across all settings), while decision-level misuse of the "None of the above" sentinel is sub- stantially reduced but not fully removed, dropping from 1.08% (zero-shot) to 0.37% (ODD pass-l) and 0.22% % (ODD+bTTA). The router then actively revises uncertain cases: the fi- nal stage flips 26.71% % of labels relative to pass-1, converting many first-pass errors into cor- rect answers, the cost of slight increase in the average confidence on the remaining mis- takes (HO LO0_MEAN_WROR
0.2678- +029466) the tail of over-confident errors shrinks markedly (OCW@0.7 4.18% 00.77% 27% 0..1%).. Augmentationn-condiionn diagnosties localize residual risks: retrieval can inject conflicting cues (RCR 21.73%), whereas QD is largely but not per- fectly self-consistent once gated (QDC= 1.74%).
18

---

Published as a conference paper at ICLR 2026
Table 3: tmRAG routing frequency by constrained type on Robo2VLM-1 (share of all evaluated items).
Type
Routed (%)
Colors
18.57
Arrows
22.18
Letters
15.28
Yes/No
19.51
Instructions
4.48
Total (any tmRAG)
80.02
0DD + bTTA
50
49
48
Accuracy
47
46
45
44
43
1.0
1.2
1.4
1.6
1.8
2.0
Average normalized budget
Figure 10: Accuracy-budget frontier on Robo2VLM-1 with ODD + bTTA. Points are B E
(1.00, 1.17, 1.29, 1.53, 2.00).
I POWER AND ENERGY.
We log board power at ms gramularity on the same single-GPU host and integrate over action windows. Using the per-trigger durations and power (HR: 1.05 s/175 W, tmRAG: 0.40 s/1 11 W, QD: 0.65 s/160 W) and a base constrained pass of 7.45 s/165 W (1,229 J), Table 7 converts the unified trigger rates from Appendix. into per-query energy. At the mid-budget operating point B-1.53 (HR/tmRAG/QD trigger rates 60%/40%/31 (444%, the added energy over the base is 160.7J, for total of 1,389.7 J and a +6,4 Ace gain 442.9%~44..34. i... 25.1 per additional accuracy point. At the budget cap B=2.00 990%/80%%/8..6% the total reaches 1.521.9J for 50.5% accuracy, i.e., 38.6 per additional point versus the ODD baseline. Energy scales nearly linearly with B and dominated by the base pass; among actions, HR contributes the most, tmRAG is the lightest, and QD lies between. Absolute wattage can vary across devices; relative trends are robust under the same pipeline.
summary, the LLaVA-1.5-13R1 teacher requires on average 22.30 and 8,697J per Robo2VLM-1 query to reach 74.74% accuracy on the same hardware; our full BOLT configuration thus uses about 5.7x less energy and runs about 2.5x faster while achieving 13.8 points higher accuracy.
TEACHER- TEACHER-STUDENT DIRECTION ABLATION
On Robo2VLM-1, we observe the following phenomenon: the compact Qwen2 VL-2B student is already stronger than LLaVA-1.5-7B in zero-shot accuracy under constrained decoding interface (28.66% vs. 21.58%) yet when we use the weaker LLaVA-115-1 1-5--B as a teacher and apply ODD, the 2B student still improves substantially. Option-level decision distillation leverages the teacher's soft
19

---

Published as a conference paper at ICLR 2026
Table Threshold sweep for ODD bTTA: trigger rates that realize each budget (base 1.00; action costs 0.50/0.3000.55).
B
HR trig.
tmRAG trig.
QD trig.
Acc (%)
1.00
0%
0%
0%%
42.9
1.17
25%
10%
4.3%
46.1
1.29
40%
20%
8.6%
47.3
1.53
60%
40%
31.4%
49.3
1.95
90%
80%
74.3%
50.4
2.00
90%
80%
88.6%
50.5
Table 5: Routing/dynamic-cooppuu baselines on Robo2VLM-1 (teacher LLaVA-13B + student
ODD; constrained decoding). Budgets B {1.29, 1.53, 1.95) match the sweep in SF
Method @ Budget B
HR trig.
tmRAG trig.
QD trig.
Acc (%)
ECE
AURC
HR-Threshold @ 1.29
40%
0%
0%
45.7
0.233
0.3209
Early-Exit @ 1.29
35%
0%
0%
45.1
0.238
0.3216
bTTA (ours) 1.29
40%
20%
8.6%
47.3
0.219
0.3176
HR-Threshold @ 1.53
60%
0%
0%
47.6
0,226
0.3198
Exrly-Exit @ 1.53
55%
0%
0%
46.9
0.229
0.3204
bTTA (ours) @ 1.53
60%
40%
31.4%
49.3
0.197
0.3160
HR- HR-Threshold @ 1.95
90%
0%
0%
49.1
0.221
0.3189
Early-Exit @ 1.95
85%
0%
0%
48.7
0.223
0.3193
bTTA (ours) @ 1.95
90%
80%
74.3%
50.4
0.172
0.3115
option distributions and inter-option structure, rather than relying solely on the teacher's top-1 label. To validate this explanation, perform additional ablation that swaps the teacher and student roles between Qwen2 VL-2B and LLaVA-1.5 -1.5A-1.--7BB as shown in Table [ When LLaVA-1.5-7B is the teacher and Qwen2 VL-2B is the student, ODD raises the student from 28.66% (zero-shot) 38.53%, and to 45.43% with bTTA. When we swap the roles by taking Qwen2 VL-2B as the teacher for L.L ODD again improves the 7B student from 21.58% 30.83% (and 36.81 1% with bTTA), but the resulting model remains less accurate and less parameteer-fffcieet than the distilled 2B student. As revealed in these results, ODD can transfer useful decision structure even from nominally weaker teacher, and the best overall trade-off is obtained when a larger model is used as an offline teacher and compact model is deployed the student.
K ADDITIONAL TABLES
L DECLARATION OF LLM USAGE
During the preparation of this manuscript, large language models were used only to improve the clarity of writing.
20

---

Published as a conference paper at ICLR 2026
Table 6: Hallucination proxies Robo2VLM-1 under constrained decoding for all settings. Zero- shot uses the undistilled Qwen2-VL-2B (single constrained pass). Pass-1 uses the ODD student (single pass). Final uses ODD + bTTA (HR ImRAG QD when routed). Lower is better for all metrics except Flip.
Metric
Zero-shot (no ODD)
Pass-1 (ODD)
Final (ODD bTTA)
Invalid-Option Rate (IOR)
0.00%
0.00%
0.00%
None-of None-of-the-Abos the-Above misuse (NOA) +
1.08%
0.37%
0.22%
Flip (label changed)
0.00%
0.00%
26.71%
Mcan Pnax wrong (HO mean wrong) +
0.3312
0.2678
0.2946
OCW @0.7 (over-confident wrong)
4.18%
0.27%
0.19%
RCR (retrieval contradiction).
21.73%
QDC (QD contradiction)
1.74%
Table 7: Power/energy on Robo2VLM-1 (batch=1, single GPU). "EEnegy/Trrigger" Power x Duration. Trigger rates at B=1.53 are 60% for HR/mmRAG/QD; at B-2.00 are 90%/880/88.665 Totals are base energy (1,229 J) plus per-action contributions.
Action Stage
Avg Power (W)
Dur//Tigger (s)
Emnegy/Trrggeer (J)
Contribê 1,53 (J)
Contrib@2.000 (J)
Puss-1 (ODD)
165
7.45
1,229
(base)
184
110.4
165.6
tmRAG
110
0.40
17.6
35.2
QD
160
0.65
104
32.7
92.1
Total energy query (J)
1,389.7
1,521.9
Table 8: Teacher-student role swap on Robo2VLM-1 under the same constrained decoding interface.
Teacher
Student
Method
Acc (%)
LLaVA-1.5-7B
zero-shot
21.58
Qwen2 VL-2B
zero-shot
28.66
LLaVA-1.5-7B
Qwen2 VL.2BB
Ours (ODD)
38.53
LLaVA-1.5-7B
Qwen2 VL-2B
Ours (ODD) bTTA
45.43
Qwen2 VL-2B
LLaVA-L.5-7B
Ours (ODD)
30.83
Qwen2 VI.2BB
IJLVA-1.5-7B
Ours (ODD) + bTTA
36.81
21

---

Published as a conference paper at ICLR 2026
Table 9: Calibration and selective-computation metrics Robo2VLM-1. We report negative log- likelihood (NLL), Brier score, Expected Calibration Error (ECE), and area under the risk-coverage curve (AURC). All methods share the same constrained decoding and prompt template; temperature calibration is fitted on validation only.
Variant
NLL
Brier
ECE 4
AURC
Teacher: LLaVA-1.5-13B
Qwen2 VL-2B distilled by LLaVA-13B
1.4717
0.6843
0.2440
0.3198
tmRAG
1.4578
0.6791
0.2295
0.3181
+ QD
1.4464
0.6748
0.2175
0.3166
tmRAG + QD
1.4367
0.6715
0.2080
0.3157
+ HR
1,4354
0.6709
0.2063
0,3155
HR tmRAG
1.4196
0.6650
0.1899
0.3135
HR + QD
1.4136
0,6630
0.1842
0,3131
+HR+ (mRAG QD
1.3984
0.6574
0.1685
0.3113
Teacher: LLaVA-1.5-7B
Qwen2 VL-2B distilled by LLaVA-7B
1.5200
0.7005
0.2700
0.3245
tmRAG
1.5113
0.6972
0.2608
0,3234
QD
1.5033
0.6942
0.2524
0.3223
tmRAG + QD
1.4942
0.6910
0.2434
0.3215
HR
1.4866
0.6882
0.2354
0.3205
+ HR tmRAG
1,4803
0.6857
0.2287
0.3196
HRR +QD
1.4752
0.6841
0.2239
0.3193
+ HR + imRAG + QD
1.4586
0.6779
0.2066
0.3173
Teacher: Qwen2.5-VI--78
Qwen2 VL-2B distilled by Qwen2.5-VL-7B
1.4950
0.6920
0,2560
0.3218
tmRAG
1.4827
0.6874
0.2431
0,3202
QD
1.4750
0.6845
0.2350
0.3192
tmRAG + QD
1.4639
0.6806
0.2240
0.3182
+ HR
1,4631
0.6798
0.2230
0.3174
HR tmRAG
1.4481
0.6746
0.2073
0.3161
+ HR QD
1.4431
0,6729
0.2025
0.3158
HR imRAG + QD
1.4321
0.6688
0.1910
0.3144
22

---

Published as a conference paper at ICLR 2026
Table 10: Comparison on Robo2VLM-1 (Ace %). All methods share the same constrained decoding and prompt template.
Model
Params (B)
Acc (%)
LLaVA 1.5-7B (zero-shot)
21.58
LLaVA 1.5-13B (teacher, zero-shot)
13
36.74
Qwen2 VL-2B (zero-shot)
28.66
Qwen2 VI.-2B distilled by LLaVA-7B
38.53
tmRAG
39.49
QD
40.36
tmRAG + QD
41.44
HR
42.27
HR tmRAG
42.96
HR + QD
43.58
HR tmRAG QD
45.43
Qwen2 VI.-2B distilled by Qwen2.5-VL-77
40.52
tmRAG
41.80
QD
42.59
tmRAG + QD
43,81
HR
43.18
HR tmRAG
45.44
HR + QD
46.02
HR tmRAG QD
47.16
Qwen2 VL-2B distilled by LLaVA-13B
42.89
tmRAG
44.31
QD
45.47
tmRAG + QD
46.52
HR
46.64
HR tmRAG
2
48.25
HR + QD
48.92
HR tmRAG + QD
50.50
(a) GPU memory usage (MB). Distillation cuts mem- ory by 88.7% 7% vs the 13B teacher; enabling all augmen- tations adds only 782 MB.
(b) Latency on Robo2VLM-1 for the LLaVA-13B + Qwen2-VI--2B setting (batch=1, single GPU). BTTA includes HR/mRAG//D only when routed.
Momory
Pass-1 only bTTA overhead End-io-end
bege
per questittoo latency
compact
Share of toral Throughput (tems/minn)
83.1%
16.9%
100%
Table 11: (a) Memory and (b) latency
23


Reviews:
[{"review_id": "3JMIkvVLAV", "reviewer": "Reviewer_2aU9", "summary": "The paper considers the task of robotic VQA, which is characterized by strict latency, memory, and energy constraints and therefore makes direct deployment of state-of-the-art VLMs infeasible. The paper therefore proposes a two-pronged approach: option-level decision distillation (ODD) and budget-aware test-time augmentation (bTTA). Unlike token knowledge distillation, ODD operates on the sums of answer-token log-probabilities. bTTA performs test-time routing based on cheap statistics derived from the student and optionally performs retrieval, question decomposition, and high-resolution reevaluation to augment the student’s response. The paper empirically shows that the distilled student outperforms the teacher’s zero-shot accuracy by almost 15% while having a 7x smaller memory footprint.", "questions": "* What is the teacher’s latency?\n* L267 states that the student’s option distribution is normalized “in the same way” as the teacher's, but Equation 3 does not have the temperature parameter that is present in Equation 2.\n* Why is the sum of answer-token log-probabilities robust to benign tokenization changes for fixed option strings (L262)?\n* When would option strings differ in length between students and teachers (L263)?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed approach is original, simple (in a good way), and sensible, and the claims are validated by empirical evidence. The results are convincing, as the student outperforms the teacher by a large margin at a fraction of the memory footprint. The paper conducts a series of ablations, showing that each component of the proposed method is relevant. The paper is well-written and easy to follow.\n\nI am not familiar with the literature on robotics VQA at all, so I cannot speak to the significance of this paper’s contribution.", "weaknesses": "It is not quite clear to me how relevant the task of VQA for robotics is. From my limited understanding, it does not appear to be very relevant for real-world applications (i.e., how realistic is it to have a large-scale dataset of options?), as evidenced by the lack of datasets for this task (see Limitations).\n\nIt is also not quite clear to me how realistic it is to be able to cache the teacher distributions for all training items offline in a real-world setting (L264).\n\nOn L61, “optimization” should be replaced by “optimizing”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RAKlVOGUPr", "reviewer": "Reviewer_Gf6B", "summary": "This paper introduces BOLT, a decision-aligned training and inference framework that enables small multimodal models to achieve large-model reasoning performance under tight computational budgets.\n(1) They proposed Option-Level Decision Distillation (ODD), a new distillation objective that aligns the student model with the teacher’s option-level decision boundaries, rather than token-level or sequence-level supervision. This directly optimizes the prediction space that determines final correctness and avoids prompt and chain-of-thought artifacts common in conventional distillation.\n(2) They developed Budget-Aware Test-Time Augmentation (bTTA), a lightweight routing mechanism that dynamically allocates additional compute only when necessary, selectively invoking higher-resolution perception, retrieval exemplars, or question decomposition based on low-cost uncertainty and affinity signals.\n(3) The results show that BOLT enables a 2B-parameter model to match or surpass a 13B-parameter teacher on multimodal multiple-choice reasoning benchmarks while operating within strict compute constraints relevant to robotic platforms. Together, these results demonstrate that decision-aligned supervision and adaptive inference provide a scalable path to high-accuracy multimodal reasoning in resource-limited settings.", "questions": "Are we training student model using on-policy distillation? Have we compared the effectiveness between supervised distillation and on-policy distillation?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper introduces novel distillation objective. The proposed Option-Level Decision Distillation (ODD) directly aligns the student model with the teacher’s decision boundaries rather than token-level generation trajectories. This is a conceptually novel and clean setup, since multimodal multiple-choice QA depends only on final option-level classification, not on the internal chain-of-thought sequence.\n\nThe budget-aware test-time augmentation (bTTA) module provides a principled mechanism for dynamic compute allocation, selectively activating high-cost inference steps only when necessary. This is both practically beneficial and generally applicable to real-world deployment where compute and power constraints vary.\n\nThe paper shows strong experimental results. A 2B-parameter student model distilled with ODD and equipped with bTTA matches or outperforms a 13B teacher on multimodal reasoning tasks under constrained compute. These improvements are consistent across multiple datasets and inference conditions, giving credibility to the claims.", "weaknesses": "The paper provides some empirical success, but has limited novelty in the general domain of knowledge distillation. For example, the option-level decision distillation (ODD) only applies to some domain-specific tasks. Besides the proposed distillation objective, the innovation is rather limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jQlyEMFh7s", "reviewer": "Reviewer_23s7", "summary": "The paper introduces two novel methods to improve performance on multi-choice Q&A applied in the robotics domain, where latency, memory and energy efficiency are critical. The first method, Option-level Decision Distillation, provides a method to align a student model with a teacher model based on a normalized the score of the full answer segment rather than just the answer token, and budget test time augmentation decides on a test time budget for a sample based on entropy like measures of the option distribution. The paper shows the effectiveness of the proposed methods on the Robo2VLM-1 benchmark, where a 2b model with both ODD and bTTA surpass a 13b teacher model performance.", "questions": "If the answer options are A,B,C,D, and E, why do we need this score formulation (Eq 1), as the answer would only be one token. If this represent a reasoning trace, do we then for each question need to do rollouts with the teacher model until we get coverage on all answer options? \n\nIs the method robotics specific or even multi-modal specific or can the same method also be used for standard mutli-choice Q&A tasks where latency/model size is important?\n\nIn Table 2, Llava 1.5-7b has worse performance than Qwen 2 VL-2B on the zero shot eval. Why is then the Llava model used as a teacher model when it's worse than the student model, and why does this improve the performance?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The idea of the bTTA is interesting, and it makes intuitive sense that the test time compute should depend on the initial properties of the answer distribution (such as max probability and entropy), in a resource limited setting. \nI am also not aware of other methods looking at the combined score of the assistant answer rather than just the answer token when doing distillation. It's also impressive that this approach improves on the performance of using token level distillation.", "weaknesses": "The abstract, and more broadly the paper in general, reads in a quite convoluted way, with some unclear terms such as \"multi-choice prediction as a decision surface to be aligned\", two main ideas that are essentially independent (bTTA and ODD), and a lot of abbreviations: QD, ODD, BOLT, HR, bTTA, KD, tmRAG, VQA. \n\nA major weakness is that while the proposed method significantly improves performance on the Robo2VLM-1 benchmark, the paper does not compare the method with other distillation methods, hence it's not clear that this approach improves upon prior ways of doing distillation. \n\nAnother weakness is that the method is only applied to one dataset, which limits the understanding of the general usefulness of the method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9FuHTmvQyx", "reviewer": "Reviewer_BJBk", "summary": "The paper proposes a framework (BOLT) for resource-constrained multimodal question answering in robotics settings with multiple-choice outputs. The framework combines a knowledge distillation component (ODD) that trains a student VLM to match a teacher's distribution over answer spans rather than individual tokens, and a routing component (bTTA) that selectively triggers compute-expensive modules like high-resolution re-evaluation, retrieval-augmented generation, or question decomposition based on heuristic signals like confidence, entropy, and retrieval affinity.\n\nExperiments on Robo2VLM-1 show a 2B student distilled from LLaVA-1.5-13B achieves 42.89% accuracy (vs 36.74% teacher, 28.66% zero-shot) with just distillation, and 50.50% when the router is enabled. GPU memory drops from 26.9GB to 3.8GB. The paper also reports calibration improvements (ECE, AURC) and reduced invalid outputs.", "questions": "1. **Multi-dataset evaluation**: Can you evaluate on additional robotic VQA benchmarks beyond Robo2VLM-1? This is critical for validating that your approach generalizes and isn't overfit to this specific panel-based dataset format.\n\n2. **Router feature ablations**: Which of the 5 routing features actually matter? Can you provide ablations showing the contribution of each feature individually (confidence, margin, entropy, retrieval affinity, decomposition agreement)? Are all of them necessary or would a simpler router work?\n\n3. **Applicability beyond this benchmark**: How would your method work on robotic tasks where type-matched exemplars are not available or questions don't naturally decompose? What components of bTTA are truly general vs specific to this evaluation setup?\n\n4. **Retrieval contradiction rate**: 21.73% RCR is concerning and suggests retrieval often hurts rather than helps. Can you characterize when retrieval contradicts the correct answer and propose mitigations?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 2, "contribution": 2, "strengths": "- **Well-motivated problem**: The mismatch between token-level distillation and constrained decoding is real, and the paper makes a reasonable case for option-level alignment.\n\n- **Practical system**: The combination of distillation and budgeted routing addresses a real deployment scenario. The router framework is sensible and the budgeted formulation is clear.\n\n- **Solid empirical gains**: The 2B student outperforms the 13B teacher on Robo2VLM-1. The memory reduction (88.7%) makes a good case for edge deployment.\n\n- **Comprehensive evaluation**: The paper reports accuracy, calibration metrics (ECE, AURC, Brier), memory, latency, energy, and hallucination proxies. The ablations for the distillation method are thorough (though this is not the case for the routing method).\n\n- **Interpretability angle**: The authors make a good case that the components of their system (constraining outputs to valid options, exposing retrieved exemplars, and showing decomposition traces) can help with transparency.", "weaknesses": "- **Single-dataset evaluation**: All experiments are on Robo2VLM-1. Since the experiments are on static datasets rather than real robots, it should be straightforward to evaluate on multiple robotic VQA benchmarks to demonstrate generalization. The dataset is quite specific (panel layouts, constrained options), and it's unclear how well the approach transfers. Other robotic VQA datasets exist that could validate the claims.\n\n- **Router has too many moving parts without sufficient analysis**: The router uses 5 features (confidence, margin, entropy, retrieval affinity, decomposition agreement) and triggers 3 augmentations (HR, tmRAG, QD), but there's no ablation isolating the contribution of individual features. It's unclear which signals actually matter. Some components like type-matched retrieval and question decomposition seem engineered specifically for this benchmark (panel-based tasks with available exemplars and decomposable questions) and may not apply to other robotics tasks where exemplars are unavailable or questions don't naturally decompose. This makes the method's practical applicability questionable.\n\n- **Limited teacher/student diversity**: The student is always Qwen2-VL-2B. Would the gains hold with different student architectures? The teachers are all instruction-tuned VLMs, but no ablation on teacher quality or family is shown.\n\n- **Option-level distillation not deeply novel**: Matching distributions over discrete outputs is fairly standard. The contribution here is applying it to VLMs in a specific constrained setting, but the technical novelty is incremental. The length-bias correction (Eq. 5) is mentioned but not thoroughly analyzed.\n\n- **Hallucination analysis is limited**: The proxies (IOR, NOA, Flip, etc.) are somewhat ad-hoc. The retrieval contradiction rate (RCR) is 21.73%, which is high and not addressed beyond acknowledging it. The paper claims constrained decoding eliminates hallucinations, but the NOA misuse and RCR suggest decision-level hallucinations remain.\n\n- **Energy/latency analysis is incomplete**: Table 7 gives energy per query, but no comparison to the teacher's energy or analysis of the energy-accuracy tradeoff. The latency breakdown (Table 10b) shows bTTA adds 1.52s on top of 7.45s, but no comparison to teacher latency or discussion of real-time requirements for robotics.\n\n- **Writing and presentation**: The paper is dense and tries to cover a lot. Some sections (e.g., Section 3.2.2 on bTTA) pack too much into a small space. The notation is heavy and could be simplified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)