Efficient Test-Time Scaling for Small Vision-Language Models
Authors: Mehmet Onurcan Kaya
Desmond Elliott
Dim Papadopoulos
Venue: ICLR 2026 Poster (2026)
Keywords: VLMs
test-time scaling
test-time augmentation
test-time adaptation
test-time compute
multimodal learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose two efficient and effective methods improving multimodal small language models at test-time: TTAug (input augmentation + token-level aggregation) and TTAdapt (parameter adaptation via pseudolabels).

Abstract: Small Vision-Language Models (VLMs) provide a computationally efficient alternative to larger models, at the cost of weaker generalization abilities and downstream task performance. These shortcomings could be addressed by test-time scaling techniques, but existing methods are typically computationally demanding, contradicting the resource-efficient design goals of small models. To address these limitations, we propose two novel and efficient test-time scaling strategies that leverage the model-internal features rather than external supervision: (i) Test-Time Augmentation (TTAug), which generates multiple augmented inputs and aggregates outputs at the token level without parameter updates, and (ii) Test-Time Adaptation (TTAdapt), which adapts model parameters during inference using consensus-based pseudolabels from TTAug. Through extensive experiments across nine benchmarks, we demonstrate consistent performance improvements while maintaining computational efficiency suitable for resource-constrained environments. The generality of our approach is demonstrated both within models at different scales and across different VLMs without additional tuning.

Published as a conference paper at ICLR 2026
EFFICIENT TEST-TIME SCALING FOR SMALL VISION-LANGUAGE MODELS
Mehmet Onurcan Kayal.2 Desmond Elliott Dim P. Papadopoulos"
3 University of Copenhagen
Technical University of Denmark 2 Pioneer Center for Al de@di mkn
ku. dk, dimp@dtu.co
Project Page: http::/wwwooannnngii.iiiiiiiiiie
ABSTRACT
Small Vision-Language Models (VLMs) provide computationally efficient alterna- tive to larger models, at the cost weaker generalization abilities and downstream task performance. These shortcomings could be addressed by test-time scaling techniques, but existing methods are typically computationally demanding, con- tradicting the resource-efficient design goals of small models. To address these limitations, we propose two novel and efficient test-time scaling strategies that leverage the model-internal features rather than external supervision: (i) Test-Time Augmentation (TTAug), which generates multiple augmented inputs and aggre- gates outputs at the token level without parameter updates, and (ii) Test-Time Adaptation (TTAdapt), which adapts model parameters during inference using consensus-basee based pseudolabels from TTAug. Through extensive experiments across nine benchmarks, we demonstrate consistent performance improvements while maintaining computational efficiency suitable for csouurceconstraiied environ- ments. The generality of our approach is demonstrated both within models at different scales and across different VLMs without additional tuning.
INTRODUCTION
Small Vision-Language Models (VLMs) offer computational efficiency and accessibility, yet their performance frequently degrades under domain shift due to inherent biases and limited generalization capabilities (Marafioti et al., 2025 Lu et al.) 2025). While test-time scaling methods can, in principle, improve their performance, there are several critical limitations that undermine their practicality for small models in resource-constraanee settings.
First, many test-time scaling methods rely on external verification models or computationally intensive reranking strategies, making them unsuitable for deployment resource- consurcceccnttrreeeed consumer GPUs (Zhang et al., 20244 Singh et al.) 2025). This contradicts the resource-efficiens design goals of small VLMs. Second, existing approaches that avoid external verifiers, such sampling multiple candidate responses and aggregating them into a final prediction using the model's internal signals (Wang al. 2023b; Adiwardana et al. 22200) Chen al., 2024a), remain unsatisfactory because they typically operate only at the answer level, ignoring local signals for aggregation. Global measures like average confidence obscure token-level fluctuations that signal response quality, and averaging across entire sequences masks reasoning breakdowns at intermediate steps. Moreover, these methods require complete response generation before evaluation, preventing early termination and wasting computation. Finally, many existing methods are restricted to tasks with extractable final answers (e.g., multiple-choice or numerical reasoning), limiting their applicability to open-ended tasks such as visual question answering and captioning (Zhang et al. 2025a; Chen et al. 2024a).
In this paper, we leverage model-internal representations to overcome these limitations. Our goal is to improve the robustness and accuracy of small VLMs at inference time through efficient, lightweight, and practical test-time scaling strategies that require no external models or additional training data. We introduce two methods in unified framework: Test- Time Augmentation and Test-Tine Time Adaptation. Test-Time Augmentation generates multiple responses by applying input-level augmentations to both images and text. Crucially, it aggregates outputs at the token-level rather than the answer-level,

---

Published as a conference paper at ICLR 2026
Test-Time Augmentation (TTAug Sec. 3.1)
MIE- RealoRd
AMBER
Image Transforms
ADDD
COCO Captons
What total number of towels present
Indicate number of towels shown
16
How many towels are in the image?
Text Transforms
in the image?
the image.
Original Input
N A'Agmeeted Inputs
Charcaa
Next Token Probabilities
Aggregated Probabilities
75
Shared Generated Context
Next
G0A
VLM
Aggregate
Sample
Token
OCRBench
OCRVQA
Append
SmolVLMM-22.28
Supervised Fine-Tuning
Generated Response
Test-Time Augmentin Augmentation (Sec. 3.1)
(Pseudolabel)
Test-Time Adaptation (Sac. 3.2)
Test-Time Adaptation (TTAdapt Sec. 3.2)
(a) Our efficient test-time scaling framework
(b) Performance comparison
Our framework consists of two main pipelines: (1) Dest-Time Augmentation: Given an input image and text prompt, we apply various transformations to create multiple augmented versions. VLM processes each augmented input to produce next token probability distributions, which are then aggregated at the token level generate the final response. (2) Test-Time Adaptation: We create pscudolabels through test-time augmentation and fine-tune the VLM parameters, then repeat the process. Both methods demonstrate effectiveness across nine diverse benchmarks as shown in (b).
Figure 1:
which allows the model to quickly detect low-quality responses, and allows for more fine- ane-grained exploitation of the mooell-nteemal signals. This method requires no parameter updates, making it both simple and efficient. Our second method, Test-Time Adaptation, extends this idea by adapting model parameters during inference. It leverages consensus signals from TTAug as pseudolabels, which guide lightweight fine-tuning on test samples without any labeled data. This enables the model to dynamically adjust to domain-specific characteristics while retaining computational efficiency.
Our approach consistently outperforms existing test-time scaling methods, such self- consistency [Wang et al., 2023b). sample-and-ram (Adiwardana et al. 20200) self-selector (Chen al.
2024at Parmar et al. (205), and self-synthesizer (Li al.) 2025d]] Jiang al.. [2023] Wang
et al. 2025a Li al. 2025b). Furthermore, these improvements do not come with a heavy computa- tional cost, allowing our approach to be used in resource-constrained settings. Beyond performance gains, our study reveals two important general insights for test-time scaling: (1) generating multiple candidate answers through input augmentations with greedy decoding more effective than the commonly-used temperature sampling strategy, and (2) token-level aggregation provides stronger signals than aggregating only at the final-answer level. These findings highlight practical principles for scaling VLMs efficiently inference. Our experiments across nine diverse benchmarks and multiple VLM architectures confirm that these insights translate into consistent improvements and broad generalization, underscoring the effectiveness and generality of our framework.
Our contributions are threefold: (1) We present two efficient test-time scaling methods for small VLMs deployable on consumer GPUs. (2) We provide the first comprehensive analysis of Test- Time Augmentation for VLMs, investigating augmentation strategies, aggregation methods, and optimal aggregation layers. Despite being simple and easily integrable technique, its application to multimodal settings remains surprisingly underexplored.
(3) We introduce the first Time Adaptation method for multimodal language models, whereas prior work on VLM test-time adaptation has focused primarily on CL.P--based models (Liang et al. 2025 Dong et al. 2025 Ji et al. 2025
2 RELATED WORK
Test-time scaling is paradigm in which current large language models increasingly achieve superior performance by allocating substantial computational resources during inference (Zhang et al.] 2025a] Ji et al. 2025). A popular test-time scaling strategy is parallel sampling, which generates multiple
2

---

Published as a conference paper at ICLR 2026
outputs simultaneously and aggregates them. However, existing parallel sampling methods face several critical limitations that make them impractical for assource-constrained deployments. Most approaches rely on external verifier models or compute-heavy strategies. making them incompatible with the small model paradigm (Zhang et al., 2024, Singh et al., 2025). We address these limitations by proposing two lightweight but effective methods via test-time augmentation.
Test-time augmentation (TTAug) improves model robustness and generalization by averaging
predictions across augmented views (Shorten & Khoshgoftaar 2019). Recent work extends TTAug
through learnable policies (Lyzhov et al. 2020, Kim et al., 2020; Shanmugam et al. 2021) by
optimizing augmentation selection and weighting. However, these active methods typically require labeled datasets to learn optimal policies. limiting their practical applicability. Prior TTAug research for (multimodal) language models (Mashrur et al.) [0222] Kamoda et al., (023) mainly addresses hallucination detection and robustness, not accuracy improvement, and does not treat TTAug as a systematic test-time scaling method. Our work closes this gap by extending both non-learnable and learnable TTAug strategies to Vision-Language Models (VLMs), systematically evaluating how augmentation design, aggregation, and scaling affect performance across tasks, and leveraging sulf-supervised objectives from eest-time adaptation literature avoid reliance on labeled data.
Test-time adaptation (TTAdapt) is an emerging paradigm for adapting pretrained models to new data batches during inference by updating model weights or inputs to maximize prediction accuracy without ground-truth labels (Xiao & Snoek 2024). The choice of optimization target and objective is crucial for adaptation effectiveness. In multimodal learning, most prior TTAdapt work focuses CLIP-based VLMs (Liang et al. [2025] Dong et al. 22025) Ji et al. (025)), with entropy minimization as the optimization strategy (Shu al.] 2022) and widespread use of self-training with pseudolabels. In
language models, TTAdapt is less explored (Dong el al. 2025 Ji et al. (222).. Hübotter el al. (2025)
require training datasets and is not source- free, while Huang al. (2025) extend an existing test-time scaling method called self-consistency (Wang et al. (223b) for better confidence calibration but suffers from the same limitations of applicability and generalization, Akyürek et al. (2025) explore test-time training with methods similar to ours (i.e., aggregated predictions via hierarchical voting and per-instance adaptation); however, their method is specifically designed for the ARC benchmark and lacks broader applicability. Our universal and fource-free TTAdapt method overcomes these limitations by leveraging consensus- bassed pseudolabeling from our TTAug method.
METHODS
We propose a comprehensive framework for test-time scaling of small Vision-Language Models (VLMs) through two complementary approaches: test-time augmentation (TTAug) and test-time adaptation (TTAdapt). Fig. Ia illustrates our framework, which addresses the fundamental chal- lenge of improving model performance and robustness without requiring additional training data or substantial computational overhead.
3.1 TEST-TIME AUGMENTATION (TTAUG)
Our approach leverages input diversity to improve model robustness through systematic aggregation of predictions from semantically equivalent inputs. Given an input consisting an image and text prompt we generate set of N augmented versions [[I, [:)] through penearic-preserving trans- formations (Sec. 4.4).. Each transformation preserves the semantic content essential for multimodal understanding while introducing controlled textual and visual diversity (Sec. 4.5 and 4.6).
Our token generation process follows an autoregressive approach where aggregation occurs at each step during generation. Starting with an empty sequence y = (). we iteratively generate tokens. At generation step j, for each augmented input (I, t,), the VLM computes the probability distribution
over the vocabulary V conditioned on the current shared context:
(t) = p(e|I, L, yz)) softmax (fo t,, (,, Y<j))
(1)
where fo represents the VLM with parameters 6, and y<j (vi...., 1j-1} denotes the shared sequence previously generated tokens. We then aggregate the probability distributions across all augmented inputs through token-level averaging:
Py(u) = N (t)
(2)
3

---

Published as a conference paper at ICLR 2026
The next token is selected greedily from this aggregated distribution:
Vj arg max (1)
VEV
This selected token yú is then appended to the shared context y = U {ız}. and the process repeats for the next step. This autoregressive aggregation ensures that each token decision leverages the collective confidence from all augmented views while maintaining a single coherent output sequence.
This token-level aggregation strategy enables the model to leverage local confidence signals from multiple augmented views at each generation step, combining the strengths of different input repre- sentations (Sec. 4.3).. Moreover, semantic- pressrinng input perturbations with greedy decoding yield superior diversity than temperature sampling used in prior test-time scaling methods (Sec. 4.2)
3.2 TEST-TIME ADAPTATION (TTADAPT)
We also introduce learnable variant that adapts model parameters during inference through iterative pseudolabel generation and fine-tuning Our TTAdapt method operates without requiring labeled data by leveraging the consensus from TTAug as a supervision signal.
The TTAdapt process optimizes the entire VLM parameter set & through consensus-driven supervision in an iterative three-stage loop: (1) generate high-confidence pseudolabels using the current model state with TTAug consensus, (2) fine-tune model parameters using these pseudolabels supervision through efficient training with gradient checkpointing or parameter-efficien methods, and (3) reset to initial weights before processing each new question prevent catastrophic forgetting. This iterative process allows the model to progressively adapt to the test distribution while maintaining stability through consensus based pseudolabeling. See Appendix 1.7 for implementation details.
Formally, given an input image-text pair (I. t), initial model parameters fg and number of adaptation iterations K, the TTAdapt process proceeds as outlined in Algorithm
Algorithm Test- Time Adaptation (TTAdapt)
Require: Input image text prompt t, initial parameters G. iterations K
1: <- 0g
> Initialize with original weights
K do 3: y (k) < II, 0)
2: for k
> Generate pseudolabel via TTAug
4: f + aaag ming logg(( I, 0))
Update parameters
5: end for
6: y + TTAug 0)
> Generate final adapted response > Reset to initial weights for the next question
7: 8 8 P
8: return y
The TTAug method generates multiple predictions for each test input and aggregates them using token-level averaging create high-confidence pseudolabels. These pseudolabels represent the collective wisdom of the augmented predictions and serve as training targets for model adaptation. By iteratively refining predictions through TTAug consensus and parameter updates, enable the model to adapt to test-time distribution shifts while preserving its core capabilities. Through this iterative process, we adapt the model parameters achieve locally-optimal performance for the specific question type encountered during inference (Sec. 4.7)
Our unified framework provides flexibility for different deployment scenarios: TTAug offers immedi- ate improvements without parameter updates, while TTAdapt enables more substantial gains when brief optimization is feasible. We systematically evaluate both approaches across diverse benchmarks and models to understand their effectiveness and computational trade-offs (Sec. 4.8)
4 EXPERIMENTS
We conduct comprehensive experiments to validate the test-time scaling framework presented in the previous section. Each major design decision explored here, e.g. how can we generate high- quality diverse answers, or should we perform aggregation the level of the final answer at
4

---

Published as a conference paper at ICLR 2026
the token-level, using the SmolVLMM-2.22B (Marafioti al. 2025 model the baseline. Our
experiments encompass 9 benchmarks covering various task types: visual question answering (VQA)
including ChartQA (Masry et 2022), OCRBench (Liu et al. 2024), OCRVQA (Mishra et al. 2019).
GQA (Hudson & Manning] 2019), and TextVQA (Singh et al. 2019); multiple-choiee questions
(MCQ) with AT2D (Kembhavi al., 2016) and
(Zhngg et al.] 2025b); yes/no
questions using AMBER (Wang et al. 2023a); and image captioning with COCO Captions (Lin
al. 2014). We utilize the evaluation protocols provided by VLMEvalKit (Duan et al., 2024) to
ensure standardized and reproducible results. For computational efficiency and fair comparison across all methods, we sample 1000 samples from each benchmark using uniform intervals maintain representative coverage of the original data distribution while enabling extensive ablation studies. The evaluation metric is accuracy for most benchmarks, with ROUGE-L. used specifically for COCO Captions. For a comprehensive description of the evaluation metrics, refer to Appendix Standard errors for all tables in this section are provided Appendix K
4.1 COMPARISON WITH OTHER TEST-TIME SCALING METHODS
We compare our TTAug approach against four representative test-time scaling methods from the existing literature that can potentially operate without external model dependencies.
1 Self-Consistency aggregates candidate answers via majority voting across multiple sampled outputs (Wang et al. 2023b). While effective for tasks where final answers can be parsed, it struggles in creative or open-ended settings where the final answer not easy to parse.
2 Self- uses the VLM itself as verifier to select response among the candidates (Chen et al. 2024a Parmar et al. 2025). This approach extends applicability beyond tasks suited to majority voting. See Appendix [1 for implementation details.
3 Sample-and-Raal Self-Consistency ignores the model's internal signals for selection; majority voting treats all reasoning traces equally, ignoring quality variations (Wang et al.) 2025b). Sample- and-Rank (Adiwardana et al.] 2020). leverages next-token distribution statistics to assess response quality by selecting the response with the highest log probability, arg max log p(y).
Self-Synthhsizer. The selection of only one answer, as in previous strategies, ignores information from other responses. To combine potentially correct parts from different responses, VLM to aggregate responses into one coherent final answer (Li et al., [2025d] Jiang et al. 2023 Wang
the tested
et al.
2025a Li et al. 20256). See Appendix for implementation details.
5 TTAug (Ours). Our Token-level aggregation with simple averaging approach aggregates the predictions at each step using a token-level aggregation of the final logits, as defined in Eq 2
In this experiment, for our TTAug method, we aug- ment the inputs = 8 times. For all other com- pared methods, similarly generate 8 candidate answers before aggregation.
Table 1: Comparison of our TTAug method
against existing test-time scaling methods. Our method outperforms all others across accuracy and efficiency metrics.
Tab. T demonstrates the superiority of our TTAug method over the existing methods. Interestingly, most existing methods fail to consistently outper- form the baseline model across all benchmarks. In contrast, our TTAug method achieves +4.1% abso- lute improvement over the baseline model. Also, our method is more efficient in terms of both runtime and number of output tokens generated. This consis- tent advantage can be attributed to two key factors. First, by leveraging input perturbations with greedy decoding for diversity inducement, our method gen- erates higher-quality candidate responses than tem- perature sampling, which is what all other methods rely on. Second, token-level aggregation preserves local confidence signals during generation, enabling more nuanced error correction compared to global
Baseline
Others
Ours
2
3
ChartQA
74.2
74.4
73.4
72.5
71.7
75.6
OCRBench
72.9
72.6
71.9
70.2
71.9
73.4
OCRVQA
0.0
0.0
0.0
0.0
0.2
11.8
Accuracy
GQA
0.0
0.0
0.0
0.0
0.0
5.8
TextVQA
73.2
72.6
71.6
69.5
72.0
72.8
A12D
68.5
3.1
69.2
69.1
67.4
68.8
MME-RW
27.8
26.2
26.4
27.6
27.6
31.1
AMBER
68.7
70.4
64.5
53.5
67.8
75.4
COCO
9.1
8.2
8.4
6.2
16.7
15.9
Mean
43.8 36.4
42.8
41.0
43.9
47.9
Runtime (s) 1.43 3.73
4.18
3.74
4.46
2.99
Tokens
8.7
74.5
77.4
74.5
82.3
70.3
5

---

Published as a conference paper at ICLR 2026
answer-level methods that discard such information. In the following two sections, separately validate these two critical components of our method.
Takeaway: Our TTAug method consistently outperforms existing test-time scaling methods while being significantly more efficient.
4.2 DIVERSITY-INDUCEEENN METHODS
Generating diverse, high-quality candidate answers is critical for test-time scaling. We compare two approaches for inducing diversity: Temperature Sampling, and Input Perturbations combined with greedy decoding. Temperature Sampling introduces randomness into the process by sampling from a softened probability distribution, while Input Perturbations applies classic semantic-preserving augmentations to inputs (Sec. 4.5 and 4.6) and then decodes greedily.
Tab. shows that Input Perturbations with Greedy Decoding outperform Temperature Sampling for generating high-quality candidate responses under both the ① Seelf-Consistenge and (2 Self-Selector strategies. This approach achieves the largest gains on OCRVQA and GQA, where temperature sam- pling fails. The theoretical analysis in Appendix A shows that greedy decoding with input perturbations maintains higher correlation and better alignment with the model training objective, making it more effective for test-time scaling.
Table 2: Comparison of diversity-inducemen methods compared to the Baseline. Input Per- turbation outperforms Temperature Sampling.
Baseline
Temperature Sampling
Input Perturbation
①
2
I
ChartQA
74.2
74.4
73.4
74.8
70,9
OCRBench
72.9
72.6
71.9
72.7
73.1
OCRVQA
0.0
0.0
0.0
12.0
4.5
GQA
0.0
0.0
0.0
7.6
3.7
Takeaway: Input Perturbations with greedy decod- ing ultimately performs better than the Baseline or Temperature Sampling. This fundamental insight forms the basis of our method throughout the re-
TextVQA
73.2
72.6
71.6
72.3
72.9
A12D
68.5
3.1
69.2
3.6
66.6
MME-RW
27.8
26.2
26.4
30.8
29.6
AMBER
68.7
70.4
64.5
72.7
67.0
COCO
9.1
8.2
8.4
21.2
13.0
mainder of the paper.
Mean
43.8
36.4
42.8
40.9
44.6
4.3 AGGREGATION LEVELS
We now compare different aggregation levels for test-time scaling: Answer-Level versus Token-Level aggregation. Existing test-time scaling methods predominantly employ answer-level aggregation with temperature sampling for diversity inducement (Zhang al., 2025a). However, given that input perturbations with greedy decoding provide superior diversity inducement, we evaluate answer-level versus token-level aggregation using this improved diversity-inducemme method for comparison.
Nevertheless, all of the answer-level aggregation methods have critical limitations. First, global mea- sures like confidence obscure confidence fluctua- tions at local reasoning steps, which can provide valuable signals for estimating response quality. Av- eraging across entire sequences masks critical rea- soning breakdowns that occur at intermediate steps. Additionally, global measures require generating complete responses before calculation, preventing early stopping of low-quality generations and re- sulting computational inefficiency. They gener- ate constant number of responses per question rather than adaptively distributing computational budget based on response agreement. Moreover, small VLMs also often lack sufficient synthesis ca- pabilities for reliable response combination.
Table 3: Comparison of Answer-level versus Token-level aggregation methods. Token-level aggregation outperforms all other approaches.
Baseline
Answer-Level
Token
2
CharQA
74.2
74.8
70.9
61.1
72.8
75.6
OCRBench
72.9
72.7
73.1
60.9
71.1
73.4
OCRVQA
0.0
12.0
4.5
0.2
3.3
11.8
GQA
0.0
7.6
3.7
0.0
0.0
5.8
TexiVQA
73.2
72.3
72.9
61.6
71.6
72.8
A12D
68.5
3.6
66.6
69.9
68.0
68.8
MME-RW
27.8
30.8
29.6
29.0
29.2
31.1
AMBER
68.7
72.7
67.0
58.9
75.8
75.4
COCO
9.1
21.2
13.0
8.6
29.5
15.9
Tab. 3 demonstrates the effectiveness Token-level aggregation compared to the Answer-level methods.
Mean
43.8
40.9
44.6
38.9
46.8
47.9
This consistent advantage validates our hypothesis that token-level aggregation preserves valuable local confidence information that global answer-level methods discard. Particularly notable the

---

Published as a conference paper at ICLR 2026
improvements OCRVQA, GQA, and COCO, where the baseline model struggles, indicating that token-level aggregation effectively leverages augmentation diversity to recover from initial prediction failures. The method's ability to outperform answer-level approaches, such as Self- Seff-Synthesizer, despite their access to the full model's s reasoning capabilities, underscores the fundamental advantage of preserving local confidence signals during generation rather than attempting post-hoc response combination. Appendix B provides a mathematical analysis of this phenomenon. Appendix [ presents experiments using different Token-level aggregation methods, including entropy-weighted averaging, majority voting, and most confident token. Finally, Appendix D shows that aggregation of earlier layer outputs can produce better results for some tasks.
Takeaway: Token-level aggregation consistently outperforms Answer-level aggregation. This validates our test-time augmentation method as more practical alternative to existing test-time scaling approaches that rely on Answer-level approaches based on Selection or Synthesis. We use Token-level aggregation with simple averaging the final logits for all subsequent experiments.
4.4 NUMBER OF AUGMENTATIONS
We study how performance scales with the num- ber of augmented inputs to understand the optimal balance between computational cost and accuracy. Augmentation counts range from (baseline) to 64 with simple averaging aggregation. This analysis clarifies diminishing returns in test- time scaling and
16
Improvement over Baseline
12-
@ ChartGA .C ARD
OCBBncc MME
AMBER
TextVO
provides practical guidance for deployment scenar- ios with varying computational budgets.
Fig. 2 reveals diverse scaling behaviors across benchmarks, reflecting task-specific characteristics. Benchmarks showing monotonic improvement with saturation (OCRVQA, AMBER, MME-Real Real-dd) follow established test-time scaling patterns (Snell
Number Augmentations
Figure 2: Performance scaling as function of the number of augmentations. Performance gains generally plateau after 16 augmentations.
ct al. 2025 Brown et al., 2025) Wu et al.] 2024). with performance increasing steadily before plateau- ing. In contrast, several benchmarks exhibit non-monotonic curves (ChartQA, COCO, GQA) where performance peaks intermediate augmentation counts before declining due to the consistency- diversity tradeoff (Geiping et al.) [2023). This decline probably occurs because excessive augmentation introduces outlier predictions and semantic drift that degrade aggregated signal quality, simple teken-level averaging assumes equal validity across augmented predictions. Mixed behaviors (OCR- Bench, Vex.VQA, A12D) show irregular patterns with spec-fp characteristics.
The average performance curve (dashed line) indicates peak performance at 16 augmen- tations, which we adopt for subsequent experiments. This translates to a peak GPU memory usage of 8.75 GB (1.9x increase from 4.60 GB baseline) and an inference time of 4.77 per query x3.33 increase from 1.43 s baseline), when using parallel batch inference on an NVIDIA A A100 GPU. For detailed computational overhead analysis across different augmentation counts, refer to Appendix
Takeaway:
E
4.5 TEXT AUGMENTATION METHODS
We now compare different text augmentation strategies to understand the trade-offs between quality, practicality, and computational overhead in our resource-connstrineed setting.
Image- Emage-Only uses classical image augmentations (Sec. (46) without text augmentation, serving as apply the same image augmentations along with their respective text strategies.
a control.
1 APgGPT uses ChatGPT (Achiam al. 2023) to generate high-quality paraphrases, evaluate the ability of high-capacity finetuned paraphraser models distilled for our scenario (Dai et al., (2025). This provides high-quality paraphrasing augmentation using state-of-the-art external models, but is not practical as it requires external models in resource-constrained deployment scenarios.
Palf-Paraphrasing uses the LLM of the VLM to paraphrase the input prompt. Since small MS cannot reliably do this in shot, we split the prompt into sentences and paraphrase each
7

---

Published as a conference paper at ICLR 2026
with structured generation to obtain fixed number of variants. The final paraphrased prompt is the concatenation of these outputs. This approach maintains consistency with the target model's 's internal linguistic patterns while remaining self-contained. See Appendix 1.3 for implementation details.
3 Classical Augmentation uses simple and fast sementic-preserving augmentations sequentially and randomly with minimal cost (Mal 2019 Aepli Sennrich, 2022). Keyboard errors simulate common typing mistakes by replacing characters with nearby keys leveraging one-key distance to generate realistic character substitutions. Word splitting introduces spacing variations within compound words. Word deletion removes individual words. Sentence reordering swaps adjacent sentences.
Consistency Enforcement is applied by appending the original prompt after each augmented version, structured as "In other words," followed by the original prompt, mirroring the alpha blending technique in AugMix (Hendrycks al. 2020). We report ablation study results without consistency enforcement using the classical augmentation method in the ( column of Tab. all other columns
① 2 3 employ consistency enforcement technique.
Tab. 4 shows that self paraphraapphsnngggg achieves su- perior performance by leveraging model-aligned augmentations, as the model's own weights influ- ence how prompts generated, resulting in aug- mentations that exhibit superior alignment with the model's internal representations. This approach creates linguistic patterns within the training man- ifold, leading better-calibrated confidence timates during token-level aggregation. Consis- tency enforcement proves critical for semantic co- herence, with large drops observed in the ablation study, though notable exceptions occur in GQA and MME-Real where diversity outweighs con- sistency. Classical augmentations remain compet- itive with minimal computational overhead, mak-
Table 4: Comparison of text augmentation strategies. Pelf-Parrphrrsing 2 and Classical Augmentations 3 consistently perform best.
Baseline
1
ChartQA
74.2
74.7
76.9
76.6
76.1
71.4
OCRBench
72.9
73.3
73.5
72.8
73.7
70.6
OCRVQA
0.0
2.6
12.6
GQA
0.0
0.0
0.0
0.0
5.5
31.2
TextVQA
73.2
74.2
73.5
74.0
72.4
63.9
A12D
68.5
69.8
69.9
68.4
69.6
63.9
MME-RW
27.8
26.6
30.0
25.9
31.9
32.1
AMBER
68.7
64.7
68.8
72.9
75.9
60.0
ing them the most practical choice for resource-
COCO
9.1
8.4
20.6
46.2
16.9
13.2
constrained deployment. Their similar performance to self- self-paraphrasing suggests simple perturbations provide sufficient diversity for purposes.
Mean
43.8
43.5
46.2
48.5
48.3
45.1
Takeaway: Self self-paraphrasing > Classical > AugGPT. Consistency enforcement is critical for
reliable performance. For the remaining experiments, classical augmentation with consistency enforcement to balance accuracy and efficiency.
4.6 IMAGE AUGMENTATION METHODS
We evaluate three different image augmentation strategies to understand their effectiveness for multi- modal test-time scaling: classical transformations, established methods, and generative approaches.
Text-Only uses classical text augmentation (Sec. 4.5) without image augmentation, serving as control. 1 apply the same text augmentation along with their respective image strategies.
1 Classical Augmentations apply traditional computer vision transformations including bright- ness/contrast adjustments, rotation, blurring, noise injection, and geometric distortions, shown useful in other vision- vision-language tasks (Vendrov et al. 2016). We test three augmentation intensity lev- els: L Low (conservative), (M) Medium (moderate), and (H) High (aggressive) to explore the diversity-coosiisennyy trade-off. See Appendix [.4 for detailed implementation specifications.
Aug Henddrycks 2020 employs a mixing strategy that combines multiple augmentation chains with convex combinations, originally designed for robustness in image classification tasks.
3 Generative Augmentations use FLUX. 1-dev Labs et al. 2025 to create semantically similar but visually distinct image variants. However, this approach excludes text-containing images to
8

---

Published as a conference paper at ICLR 2026
prevent OCR corruption. Also, it requires external diffusion models, making it impractical for resource-constrainee deployments. See Appendix I.3 for detailed implementation specifications.
reveals several key insights about image augmentation strategies. Classical augmentations with high and low strengths marginally outperform medium strength augmentations. This non-monotonic relationship reflects the fundamental
trade-off: low strength preserves semantic coherence but provides limited diversity; high strength introduces beneficial variance without exces- sive semantic drift (Geiping al. 2023); while medium strength falls into suboptimal region where augmentations disrupt model confidence without compensating diversity benefits.
AugMix performs competitively but falls short of classical methods. suggesting that the principled mixing strategy designed for unimodal classifica- tion may not align with the token-level aggregation in VLMs. Generative augmentations underperform despite their semantic richness, primarily because text-containing images must excluded, reducing the effective augmentation coverage.
Table 5: Comparison across different image augmentation strategies. Classical Augmenta- tions ( (I) perform the best.
Baseline
M
H
3
ChartQA
74.2
75.8
77.0
76.4
76.1
74,1
75.7
For modality-wise decomposition of TTAug per- formance gains, see Appendix [F) for representative samples of augmented inputs with classical methods and corresponding outputs, Appendix L
OCRBench
72.9
73.1
73.7
73.
73.7
72.4
65.3
OCRVQA
0.0
13.5
12.1
10.6
12.6
12.9
12.0
GQA
0.0
2.0
4.1
3.7
5.5
2.5
TexiVQa
73.2
73.0
72.6
73.3
72.4
72.4
71.6
A12D
68.5
68.1
69.1
69.0
69.6
68.9
67.0
Takeaway: Classical high/low strength augmen- tations outperform AugMix and generative ap- proaches; with medium strength falling into a subop- timal iiversity-consistenne trade-off. Thus, high-strength classical augmentations for all subse- quent experiments.
MME-RW
27.8
31.6
31.8
32.5
31.9
32.1
31.1
AMBER
68.7
77.3
77.0
75.9
75.9
77.3
76.2
COCO
9.1 19.0
17.8
17.1
16.9
17.8
18.0
Mean
43.8 48.2
48.3
48.0
48.3
47.9
46.6
4.7 TEST-TIME ADAPTATION METHODS
While ITAug provides improvements, test-time adaptation (TTAdapt) extends this framework by op- timizing learnable components during inference. Unlike conventional test-time scaling that generates and selects among multiple candidate responses, approach directly optimizes model behavior using self- or semi-supervised objectives. We investigate two different adaptation strategies targeting distinct components of the aggregation pipeline, each with unique optimization objectives.
! Aggregation Weights Optimization learns adaptive token-wise weights W.j to replace the uniform averaging scheme in Eq. At each generation step j. we initialize learnable parameters as w, E N and optimize them through gradient descent to minimize the marginal entropy H (()))
EvEY (v) logT;(t) ()) of the weighted aggregated distribution by performing multiple micro-steps per token to achieve convergence. This approach requires minimal computational overhead with a compact computational graph, making it suitable for real-time deployment. Marginal entropy minimization represents the dominant optimization paradigm in test-time adaptation for CLiP-based models (Shu et al. 2022 Liang et al., 2025). We include this method as an ablation study and as computationally efficient alternative to our main adaptation approach. See Appendix [.6] details.
2 Model Parameter Adaptation implements the iterative pseudolabel generation and fine-tuning framework detailed Sec.
Tab. 0 shows clear performance differences among adaptation methods with distinct efficiency- performance trade-offs. Aggregation weights optimization performs on par with TTAug, mainly improving benchmarks that require precise confidence calibration (e.g.. AMBER, TextVQA), where adaptive weighting highlights high-qualin predictions. This supports findings that TTAdapt via marginal entropy minimization not more effective than TTAug for based VLMs (Farinal et al. 2024). Its average performance matches TTAug, but it notably fixes simple averaging's s underperformance on TextVQA, outperforming the baseline model on all benchmarks. Model parameter adaptation delivers the strongest overall gains, particularly excelling on COCO. Given its superior performance, we refer to model parameter adaptation as TTAdapt throughout this paper.

---

Published as a conference paper at ICLR 2026
However, performance occasionally degrades spe- cialized benchmarks with strong baseline capabil- ities, indicating that aggressive parameter adapta- tion can disrupt carefully calibrated domain-specifiic knowledge. This pattern suggests that adaptation intensity should be task-dependent, conservative for well-calibrated domains where the base model al- ready performs well, and more aggressive for chal- lenging distributions where consensus-baue based super- vision provides reliable guidance.
Table Performance comparison of test-time adaptation strategies. Model parameter adapta- tion 2 yields the best performance.
Baseline
TTAug
2
ChartQA
74.2
76.1
76.1
76.7
OCRBench
72.9
73.7
73.0
70,5
OCRVQA
0.0
12.6
11.9
13.8
GQA
0.0
5.5
5.2
13.5
TextVQA
73.2
72.4
74.2
70.5
Takeaway: Model adaptation achieves superior gains through consensus-based learning. Aggrega- tion weight optimization provides an efficient mid- dle ground with minimal computational overhead.
A12D
68.5
69.6
69.7
67.4
MME-RW
27.8
31.9
30.9
31.4
AMBER
68.7
75.9
76.9
72.8
COCO
9.1
16.9
16.4
35.9
Mean
43.8
48.3
48.3
50.3
4.8 CROSS-MODEL GENERALIZATION
Finally, we test our method's generalization to other VLMs by applying the SmolVLMM-2.22 config- uration (greedy decoding. 16 classical augmentations, token-level averaging) to diverse architectures and parameter scales. See Appendix H for more details.
Fig. shows performance gains across model fam- ilies and parameter scales. The best performance gains are found for SmolVLM2-2222
Ovis2
but we find consistent improvements across different architec- tures and scales. TTAug prevents error propagation through token-level aggregation, which provides fun- damental advantages regardless of model specifics. Key findings include: (1) Although optimal hyper- parameters vary across models due to differences in training data, architecture, and training augmenta- tions, our framework generalizes well and provides improvements. However, universal of hyper- parameters exists that optimally serves all models. Hyperparameter transferability inherently limited due to model-specific characteristics, including train- ing data biases, architectural differences, and train- ing augmentation strategies. (2) TTAug effectiveness does not simply correlate with model size but rather with model family and architectural similarity. This challenges our initial expectation that TTAug would primarily benefit smaller models by mitigating biases (with larger models being more robust). Instead, improvements appear more dependent on similarity to our hyperparameter optimization target. Hyperparameter transferability is stronger within model families sharing similar architectures and training procedures, as optimal hyperparameters depend on dataset biases, training augmentations, and architectural inductive biases. Also, models with similar parameter counts exhibit better hyperparameter transferability, suggesting that model capacity also influences optimal augmen- tation strategies. Even with suboptimal hyperparameters, our methods yield robust improvements, though dedicated tuning is recommended for best results. See Appendix G for more detailed results.
65
Ovis2
Ovis2
Ovis2
Accuracy
55-
InternVL2
SmalVLM2
Baseline
Smo/VLM22
TTAug
SmolvLM2
TTAdapt
0.3
0.5
Model Parameter Size (B, log-scaled)
Figure 3: Improvements across different mod-
els, demonstrating cross-model generalization.
Takeaway: Despite hyperparameters being optimized for SmolVLM2-22225 our methods provide consistent improvements across diverse models, though transferability varies by family and size.
5 CONCLUSION
We propose two efficient test-time scaling methods, Test-Time Augmentation and Test-Time Adap- tation. Our comprehensive experiments demonstrate that both methods consistently improve per- formance by outperforming existing test-time scaling approaches with minimal overhead, making them suitable for essource-constraigg environments. Our work provides a systematic way to tune hyperparameters for a given model, though optimal strategies remain task- and model-dependent.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
Dim P. Papadopoulos was supported by the DFF Sapere Aude Starting Grant "ACHILLES". Desmond Elliott was supported by the European Union's Horizon 2020 research and innovation program under grant agreement No. 101 101635671 (TrustLLM) and a research grant (VIL53122) from Villum Fonden. This work was supported by the Pioneer Centre for AI, DNRF grant number PI. We would like to thank Marco Schouten, Thanos Delatolas and Stella Frank for proofreading and insightful discussions.
REPRODUCIBILITY STATEMENT
To ensure the replicability of our findings, we release our code, allowing the research community to reproduce our results and build upon our contributions. Our experimental setup exclusively employs publicly accessible models, ensuring that all resources are readily obtainable by other researchers. We provide comprchensive details regarding all prompts and hyperparameters utilized across experiments in Appendix [ Additionally, Appendix[connainnn thorough descriptions of the benchmarks and evaluation metrics employed in our study. All evaluation benchmarks utilized in this work are established and widely-used standards within the field. We include references to these resources to facilitate easy access for interested researchers. Our commitment to transparency extends beyond code release, enable faithful reproduction of our work.
meticulously detail every aspect of experimental methodology to
AUTHOR STATEMENT ON THE USE OF LARGE LANGUAGE MODELS
During the preparation of this paper, large language models were used solely for minor grammar and language polishing. They were used for research ideation, experiment design, analysis, or writing of scientific content. All research contributions are the sole responsibility of the authors.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv:2303.08774, 2023.
Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. Towards a human-like open- domain chatbot. arXiv:2001.09)) 2020.
Noëmi Aepli and Rico Sennrich. Improving zero-shot cross-lingual transfer between closely related languages by injecting character-level noise. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguistics; ACL 2022, pp. 4074-4083, Dublin, Ireland, May 2022. Association for Computational Linguistics.
Unsloth AI, Daniel Han-Chen, and Michael Han-Chen. Unsloth. uns Amslothai/unssot 2025.
https://github.
Ekin Akyürek, Mehul Damani, Adam Zweiger, Linlu Qiu, Han Guo, Jyothish Pari, Yoon Kim, and Jacob Andreas. The surprising effectiveness of test-time training for few-shot learning. In Forty-second International Conference on Machine Learning, 2025.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv:2309. 16609, 2023.
Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Ce- line Lin, and Pavlo Molchanov. Small language models are the future agentic ai, 2025.
Gabriela Ben Melech Stan, Estelle Aflalo, Raanan Yehezkel Rohekar, Anahita Bhiwandiwalla, Shao- Yen Tseng, Matthew Lyle Olson, Yaniv Gurwicz, Chenfei Wu, Nan Duan, and Vasudev Lal. Lvlm-intrepre:: An interpretability tool for large vision-language models. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 8182-8188 2024.
11

---

Published as a conference paper at ICLR 2026
Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Gryener, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong. Julian Fisenschlos, Rishabh Kabra, Matthias Bauer, Matko Bošnjak, Xi Chen, Matthias Minderer, Paul Voigtlaender, Ioana Bica, Ivana Balazevic, Joan Puigcerver. Pinelopi Papalampidi, Olivier Henaff, Xi Xiong. Radu Soricut, Jeremiah Harmsen, and Xiaohua Zhai. Paligemma: A versatile 3b vim for transfer, 2024.
Zenab Bosheah and Vilmos Bilicki. Challenges in generating accurate text in images: A benchmark for text-to-image models on specialized content. Applied Sciences, 15(5), 2025.
Bradley Brown, Jordan Juravsky, Ryan Saul Ehrlich, Ronald Clark, Quoe V Le. Christopher Re, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling, 2025.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Alexander Buslaev, Vladimir I. Iglovikov, Eugene Khvedchenya, Alex Parinov, Mikhail Druzhinin, and Alexandr A. Kalinin. Albumentations: Fast and flexible image augmentations, Information, 11 (2), 2020.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv:2501. 2025.
Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal self-consistency for large language models. In ICML 2024 Workshop on In-Context Learning, 2024a.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generie visual- linguisic tasks. In Proceedings the IEEE/CVF conference on computer vision and pallern recognition, pp. 24185-24198, 2024b.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. In The Twelfth International Conference on Learning Representations, 2024.
Sewhan Chun, Jae Young Lee, and Junmo Kim. Cyclic test time augmentation with entropy weight method. In Uncertainty in Artificial Intelligence, pp. 433-442. PMLR, 2022.
Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Fang Zeng, Wei Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, Hongmin Cai, Lichao Sun, Quanzheng Li, Dinggang Shen, Tianming Liu, and Xiang Li. AugGPT: Leveraging ChatGPT for Text Data Augmentation. IEEE Transactions on Big Data, 1 11(03):007-918 June 2025.
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang. Jae Sung Park, Moham- madreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-ar rision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 91-104, 2025.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Hao Dong, Lijun Sheng, Jian Liang, Ran He, Eleni Chatzi, and Olga Fink. Adapting vision- lisionelanguage models without labels: A comprehensive survey. 055554 2025.
12

---

Published as a conference paper at ICLR 2026
Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong. Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 11198-11 1001, 2024.
Matteo Farina, Gianni Franchi, Giovanni Iacca, Massimiliano Mancini, and Elisa Ricci. Frustratingly easy test-time adaptation of vision-language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Jonas Geiping, Micah Goldblum, Gowthami Somepalli, Ravid Shwartz-Zii Tom Goldstein, and Andrew Gordon Wilson. How much data are augmentations worth? an investigation into scaling laws, invariance, and implicit regularization. In The Eleventh International Conference Learning Representations, 2023.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and diit----distiiuution examples in neural networks. In International Conference on Learning Representations, 2017.
Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. AugMix: A simple data processing method to improve robustness and uncertainty. Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Matthew Honnibal, Ines Montani, Sofie Van Landeghem, Adriane Boyd, et al. spacy: Industrial- strength natural language processing in python, 2020.
Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via self-calibration. arXiv:2503.0003 00033, 2025.
Jonas Hübotter, Sascha Bongni, Ido Hakimi, and Andreas Krause. Efficiently learning at test- time: Active fine-tuning of LLMs. In The Thirteenth International Conference on Learning Reprexentations, 2025.
Drew Hudson and Christopher Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 6700-6709, 2019.
Yixin Ji, Juntao Li, Yang Xiang, Hai Ye, Kaixin Wu, Kai Yao, Jia Xu, Linjian Mo, and Min Zhang. A survey of test-time compute: From intuitive inference deliberate reasoning, 2025.
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM-blender: Ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Pupers). pp. 14165-14178, Toronto, Canada, July 2023. Association for Computational Linguisties.
Go Kamoda, Benjamin Heinzerling, Keisuke Sakaguchi, and Kentaro Inui. Test-time augmentation for factual probing. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 3650-3661, Singapore, December 2023. Association for Computational Linguistics.
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi. and Ali Farhadi. A diagram is worth a dozen images. In European conference on computer vision, pp. 235-251. Springer, 2016.
I1doo Kim, Younghoon Kim, and Sungwoong Kim. Learning loss for test-time augmentation. Advances in neural information processing systems, 01:4163-4174 2020.
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux. kontext: Flow matching for in-context image generation and editing in latent space, 2025.
13

---

Published as a conference paper at ICLR 2026
Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? Advances in Neural Information Processing Systems, 37:87874- 2024.
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. LLaVA-onevision: Easy visual task transfer. Transactions on Machine Learning Research, 2025a.
Cheryl Li, Tianyuan Xu, and Steven Y. Guo. maasoning-as-looic-niii Scaling test-time reasoning in large language models through logic unit alignment. In Forty-second International Conference on Machine Learning, 2025b.
Zhuowei Li, Haizhou Shi, Yunhe Gao, Di Liu, Zhenting Wang, Yuxiao Chen, Ting Liu, Long Zhao, Hao Wang, and Dimitris N. Metaxas. The hidden life of tokens: Reducing hallucination of large vision-language models via visual information steering. In Forty-second International Conference on Machine Learning, 2025c.
Zichong Li, Xinyu Feng, Yuheng Cai, Zixuan Zhang, Tianyi Liu, Chen Liang, Weizhu Chen, Haoyu Wang, and Tuo Zhao. Lims can generate better answer by aggregating their own responses. 0rX1v22503.04104, 2025d.
Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribu- tion shifts. International Journal of Computer Vision, 133(1):31-64, 2025.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe, Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS. 2023.
Yuliang Liu, Zhang Li. Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences, 67(12), December 2024.
Shiyin Lu, Yang LI, Yu Xia, Yuwei Hu, Shanshan Zhao, Yanqing Ma, Zhichao Wei, Yinglun Li, Lunhao Duan, Jianshan Zhao, Yuxuan Han, Haijun Li, Wanying Chen, Junke Tang, Chengkun Hou, Zhixing Du, Tianli Zhou, Wenjie Zhang, Huping Ding, Jiahe Li, Wen Li, Gui Hu, Yiliang Gu, Siran Yang, Jiamang Wang, Hailong Sun, al. Ovis2.5 technical report. arXiv:2508. 1737, 2025.
Alexander Lyzhov, Yuliya Molchanova, Arsenii Ashukha, Dmitry Molchanov, and Dmitry Vetrov. Greedy policy search: A simple baseline for learnable test-time augmentation. In Conference uncertainty in artificial intelligence, pp. 1308-1317. PMLR, 2020.
Edward Ma. NIp augmentation. https///gttbbbccommmmcccwwwwwnnt 2019.
Andrés Marafioti, Orr Zohar, Miquel Farré, Merve noyan, Elie Bakouch, Pedro Manuel Cuenca Jiménez, Cyril Zakka, Loubna Ben allal, Anton Lozhkov, Nouamane Tazi, Vaibhav Srivastav, Joshua Lochner, Hugo Larcher, Mathieu Morlon, Lewis Tunstall, Leandro Von Werra, and Thomas Wolf. SmolVLM: Redefining small and efficient multimodal models. In Second Conference Language Modeling, 2025.
Akib Mashrur, Wei Luo, Nayyar A. Zaidi, and Antonio Kelles-Kelly, Semantic multi-modal reprojection for robust visual question answering. In 2022 International Conference Digital Image Computing: Techniques and Applications (DICTA), pp. 1-6, 2022.
Ahmed Masry, Do Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A benchmark for
question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, pp. (669, Dublin, Ireland, May 2022. Association for Computational Linguisties.
14

---

Published as a conference paper at ICLR 2026
Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In ICDAR, 2019.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International conference on machine learning, pp. 16888-16905. PMLR, 2022.
Nostalgebraist.
Interpreting gpt: the logit lens.
LessWrong, August 31, 2020,
2020. interpreting gpt-the-tt-t-- the teetteeeooo wow.-ooggiit- -og
https: www lesswrong. com www..cooopppo.cc
Mihir Parmar, Xin Liu, Palash Goyal, Yanfei Chen, Long Le, Swaroop Mishra, Hossein Mobahi, Jindong Gu, Zifeng Wang, Hootan Nakhost, et al. Plangen: A multi-agent framework for generating planning and reasoning trajectories for complex problem solving. 1rX1v 2025.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Divya Shanmugam, Davis Blalock, Guha Balakrishnan, and John Guttag. Better aggregation in test-time augmentation. In Proceedings of the IEEE/CVF international conference computer vision, pp. 1214-1223, 2021.
Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48) 2019.
Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. Advances in Neural Information Processing Systems, 35: 35:14274-1425 2022.
Amanpreet Singh, Vivek Natarjan, Meet Shah, Yu Jiang, Xinlei Chen, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings the Vision and Pattern Recognition, pp. 8317-8326, 2019.
the IEEE Conference on Computer
Joykirat Singh, Tanmoy Chakraborty, and Akshay Nambi. Self-evolved preference optimization for enhancing mathematical reasoning in small language models. 1.Xiv:2503.04813 2025.
Ray Smith. An overview of the tesseract ocr engine. In ICDAR '07: Proceedings of the Ninth International Conference on Document Analysis and Recognition, pp. 629-633, Washington, DC, USA, 2007. IEEE Computer Society.
Charlie Victor Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time com- pute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, 2025.
Gemma Team. Gemma: Open models based on gemini research and technology. 25X 2024.
InternLM Team. Internim: multilingual language model with progressively enhanced capabilities, 2023.
Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Asxociation for Computational Linguistics, pp. 4593 4601, Florence, Italy, July 2019. Association for Computational Linguistics.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie- Annie-AAneea Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302. 13971, 2023.
15

---

Published as a conference paper at ICLR 2026
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Ivan Vendrov, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. Order-embeddings of images and language. ICLR, 2016.
Junlin Wang, Jue WANG, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agents enhances large language model capabilities. In The Thirteenth International Conference on Learning Representations, 2025a.
Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. An lim-free cilti-dimensional benchmark for milms hallucination evaluation. ar arXiv:231..07397 2023a.
Qin Wang. Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7201-7211, 2022.
Weigin Wang, Yile Wang, and Hui Huang. Ranked voting based self-consistency large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025, pp. 14410-14426, Vienna, Austria, July 2025b. Association for Computational Linguisties.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. selfsconnistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023b.
Brandon T Willard and Rémi Louf. Efficient guided generation for large language models. D.Xiv:3307.0970. 2023.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick VOn Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38 45, Online, October 2020. Association for Computational Linguistics.
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Scaling inference computa- tion: Compute-optimal inference for problem-solving with language models. In The 4th Workshop on Mathematical Reasoning and Al NeurlPS'24 24, 2024.
Zehao Xiao and Cees GM Snoek. Beyond model adaptation test time: A survey. arX1v:2411.03687. 2024.
Yuxi Xie. Kenji Kawaguchi, Yiran Zhao, Xu Zhao, Min- Yen Kan, Junxian He, and Qizhe Xie. Self- evaluation guided beam search for reasoning. In seirth- Conference Neural Information Processing Systems, 2023.
Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael S. Ryoo, Shrikant Kendre, Jicyu Zhang, Can Qin, Shu Zhang, Chia- Chih Chen, Ning Yu, Juntao Tan, Tulika Manoj Awalgaonkar, Shelby Heinecke, Huan Wang, Yejin Choi, Ludwig Schmidt, Zeyuan Chen, Silvio Savarese, Juan Carlos Niebles, Caiming Xiong, and Ran Xu. xgen-mm (blip-3): A family of open large multimodal models. CoRR, 088724 2024.
Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff. et al. A survey on test-time scaling in large language models: What, how, where, and how well? arXiv:2503.244. 242355, 2025a.
16

---

Published as a conference paper at ICLR 2026
YiFan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, Liang Wang, and Rong Jin. MME-realworld: Could your multimodal LLM challenge high- high-resolution real-world scenarios that difficult for humans? In The Thirteenth International Conference on Learning Representations, 2025b.
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. Small language models need strong verifiers self-correct reason- ing. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 15637-15653, Bangkok, Thailand, August 2024, Association for Computational Linguisties.
17

---

Published as a conference paper at ICLR 2026
A THEORETICAL ANALYSIS OF DIVERSITY-INDUEMENN METHODS
Formally, let each candidate response y have a latent quality Q The model also assigns an internal signal, such that the confidence score, S(y), which is used for candidate selection. In practice, since the true quality (v) is unknown at test time, the practical selector chooses the candidate
arg max Sty). yEy
We can approximate the joint distribution of (Q. S) as a bivariate distribution. This distribution has means bg and uas, variances of and o and correlation p Corr(Q, S). The expected quality of the selected candidate can then be expressed as:
[[Q(x")) R μQ + pookN.
where AN = - (z)(2) ((2) dz is the expected maximum of N standard normal variables. Here, (22) is the standard normal probability density function, and ((2) is the standard normal cumulative distribution function. Notably, kN grows slowly as the candidate pool size increases.
Temperature sampling generates candidates with high variance in quality, Øq. However, these samples are often drawn from low-likelihood regions, where the model's internal confidence 5(y) is poorly aligned with the true quality (v)) As a result, the correlation p between Q and S is small, which leads to weak scaling as more candidates are added.
In contrast, input perturbations combined with greedy decoding produce candidates with lower variance but higher mean quality μlQ. More importantly, the correlation p is stronger, because these responses remain on the likelihood manifold where the model was trained to assign high confidence. This difference arises from the training objective of language models: next-token prediction under maximum likelihood estimation. During training, the model optimized for greedy decoding, and temperature sampling is not simulated (e.g., there is no Gumbel-softmax trick in training). making temperature sampling less natural for the model.
Furthermore, language models are often miscalibrated, especially after post-training (Achiam et al.) 2023). This miscalibration further reduces the correlation p for candidates from temperature sampling.
Under seffidenee-based selection, the product poi is provably larger for greedy decoding with input perturbations than for temperature sampling. This establishes greedy decoding with augmented inputs as superior mechanism for generating diverse candidates in test-time scaling.
B THEORETICAL ANALYSIS OF TOKEN-LEVEL AGGREGATION VS. ANSWER-LEVEL AGGREGATION
Consider generating a response of length T tokens. Let Pt denote the probability of the base model generating the correct token at step t given the correct prefix, with 0 < Pmin S Pmax < 1.
Token-level selection. At each step t, 2 2 candidate tokens are generated. A selector with accuracy (probability of selecting the correct token if available) yields correctness probability l: 8t 10--4 M..) Thus, the overall correctness probability
T
Proken
II t=1
Answer-level selection. N independent responses generated. A selector with accuracy (probability of selecting the fully correct response if available) yields a correctness probability given by
N
T
1
P Panswer = 1-
Ps
Theorem. Assume there exists 8 > 0 such that q: 2 (1 5)p, for all t. Then for sufficiently large T, token-level selection achieves higher expected correctness probability, Praken > P. Panswer-
18

---

Published as a conference paper at ICLR 2026
Proof. From the assumption qi (1 8)pp:
P Pokknn 2 (1+6)² 8)" II Pr +
where Poorret [T Pt. For answer-level selection:
Panswer S. N- -N-Poone Poonrct
since 1 (1-11) x3) Nx for x E [0, 1]. Comparing the two:
P Punken
(1 + P
correct
(1 (1+
P Panswer
sNN
sN
Since 8 > 0, (1 + 5) grows exponentially with T, while sN is constant. Therefore, for sufficiently large T:
(1 ++) > Proken P Panswer-
sN
Feasibility of q: 2 + 8)pt. The condition holds if:
Pt
(1+)) 1- (1-1pp)) Pc)
Since 1 - (1 -1- Pt for N > and pt 1, the right-hand side Thus, there exists < satisfying the inequality. For typical Pt E (0.5, 0.99) and N 2 2, reasonable 8t ( 0.7 - 0.95) suffice.
Conclusion. Token-level selection achieves superior performance because it corrects errors imme- diately each generation step, preventing error propagation through the sequence. The per-token improvement factor (1 + 5) compounds multiplicatively across steps. In contrast, answer-level selec- tion suffers from exponential decay correctness probability (II P.) and provides only constant-factor improvement (sN) through response selection.
This exponential scaling with sequence length means that token-level aggregation provides a rapidly growing advantage responses become longer, making it especially effective for reasoning tasks such as chain-of-thought and thinking models. In these settings, each token represents step reasoning process, so the ability to correct errors every step prevents error accumulation and leads to much higher overall correctness compared to answer-level selection, whose benefits do not scale with sequence length.
Also, the superiority of increased granularity aligns with empirical observations that process reward models outperform outcome reward models (Lightman et al. 2023). and reasoning step- wise ap- proaches like step-level self-evaluation (Xie et al.] (2023) and REBASE (Wu et al. 2024) surpass answer-level methods. However, these reasoning step-wit wise strategies remain limited to problems where reasoning steps can be clearly defined and still fall short of token-level granularity. But, they exemplify general trend: increased granularity yields better performance in test-time scaling.
For autoregressive generation with imperfect selectors, token-level selection achieves higher expected correctness than answer-level selection when the token selectors provide consistent multiplicative improvement over base probabilities and the response length is sufficiently large. The critical advantage comes from per-step error correction that mitigates compounding errors.
C AGGREGATION METHODS
compare different token-level aggregation methods for test-time augmentation.
Simple averaging uniformly weights all augmented predictions by computing the arithmetic mean of probability distributions across all augmented inputs, in Eq (v) + MN (v)
weightd averaging assigns higher weights to more confident predictions by computing the entropy Hijj - "vev Pi,j (v) for each augmented input at step j. deriving weights
19

---

Published as a conference paper at ICLR 2026
wij c Huj/
Σ
Hess through softmax over negative entropy. and aggregating P5(c) 2022p.
k=
E Wt Wi.Pic (v) Chun et al.
Majority voting aggregates predictions by selecting the token that receives the most votes across augmented inputs. For each vocabulary token at step j. we compute the vote count 6,(w)
Σi Marg w Pt.jeV = 2/, where T[-] is the indicator function. The final token is selected as yjj arg m Oj (,(,), choosing the vocabulary token with the highest vote count across augmented predictions (Farina et al. 2024).
Most confident token method selects the token with the highest predicted probability across all augmented inputs, Uj = arg Pt. confidence as shown by Guo et al. (2017), this approach effectively chooses the most confident token across all augmentations (Hendrycks & Gimpel
(v). Since the predicted probability offers a noisy proxy for
2017).
The experimental results in Tab. reveal that averaging- based methods consistently outper- form discrete voting approaches, challenging the widespread adoption of majority voting in established test-time scaling methods like self-consistency (Wang et al., 2023b). This performance hierarchy reflects fundamental differences in handling prediction uncertainty and model calibration: averaging- based ap- proaches leverage continuous probability dis-
Table 7: Comparison of token-level aggregation methods for test-time augmentation.
No TTA
Most Conf.
Maj. Vote
EW
Simple
Av.
Av.
ChartQA
74.2
73.6
74.8
76.6
75.6
OCRBench
72.9
72.0
72.2
73.4
73.4
OCRVQA
0.0
3.5
9.0
11.4
11.8
GQA
0.0
3.4
4.3
5.8
TextVQa
73.2
70.5
71.5
73.3
72.8
tributions from all augmented inputs, preserv- ing valuable confidence information that dis- crete methods discard, while the majority vot- ing and the most confident selection rely on dis-
AI2D
68.5
68.7
68.7
68.8
68.8
MME-RW
27.8
29.5
30.4
31.0
31.1
AMBER
68.7
72.3
71.4
74.6
75.4
COCO
9.1
14.2
18.4
14.6
15.9
crete decisions from poorly calibrated predic- tions (Achiam et al. 2023). Simple averaging demonstrates superior robustness compared
Mean
43.8
45.6
46.6
47.6
47.9
entropy-weighte averaging, suggesting that equal weighting provides better stability than confidence- based weighting given the miscalibration issues in language models. But, confidence-based weighting can be beneficial when the model's 's internal confidence aligns well with true prediction quality.
aggregation outperforms discrete selection methods, with simple averaging achieving the best overall performance. Continuous probability aggregation preserves valuable uncertainty information that discrete voting methods discard.
Takeaway: Averaging-
D AGGREGATION IN EARLY LAYERS
To understand the optimal point for feature aggregation within the model architecture, we systemati- cally evaluate aggregation at different transformer layers rather than exclusively at the final output logits. Instead of averaging probability distributions from the final layer, we aggregate hidden repre- sentations from intermediate layers and continue forward propagation through the remaining layers using the aggregated features.
Formally, for aggregation at layer f, we compute the averaged hidden states v h.... across all N augmented inputs generation step j. then feed this aggregated representation through layers l + 1 to L to produce the final token probabilities. This approach investigates whether early semantic representations late linguistic features provide superior aggregation targets for multimodal understanding.
20

---

Published as a conference paper at ICLR 2026
ChartQA
OCRBench
OCRVQA
GQA
TextvQA
Accuracy (%)
75
Accuracy (%)
Accuracy (%)
50
Accuracy
60
10
Accuracy
5.
25
25
20
10
20
Layer Index
Layer Index
Layer Index
Layer Index
Layer Index
A12D
MME-RealWorld
AMBER
COCO Captions
Average
70
80
Accuracy (%)
Accuracy (%)
Accuracy (%)
Accuracy (%)
Accuracy (%)
40
60
20
50
10
40
10
20
10
20
10
20
10
20
10
20
10
20
Layer Index
Layer Index
Layer Index
Layer Index
Layer Index
Figure Performance across aggregation layers. Each subplot shows accuracy function of the transformer layer where feature aggregation occurs. Different benchmarks exhibit distinct optimal aggregation points: later layers favor language-heavy tasks (ChartQA, TextVQA), while earlier layers benefit visual reasoning tasks (OCRVQA, GQA).
The experimental results in Fig. reveal task-dependent variations in optimal aggregation layers, exposing fundamental differences in how VLMs process multimodal information across different reasoning types. Three distinct patterns emerge that reflect the hierarchical nature of multimodal understanding in transformer architectures.
Late-layer preference for linguistic reasoning. Language-hear benchmarks, including ChartQA, OCRBench, and TextVQA, consistently achieve optimal performance when aggregating later layers (layers 18-24), with monotonic improvement as aggregation approaches the final output. This pattern aligns with established findings from logit lens analysis (Nostalgebraist 2020), where later layers increasingly specialize in linguistic refinement and task-specific formatting. Recent work by Chuang al. (2024) demonstrates that factual knowledge progressively accumulates in higher transformer layers, with later layers exhibiting stronger factual representations than earlier ones when contrasted through layer-wise decoding strategies. This hierarchical knowledge encoding suggests that deeper layers contain more refined and task-specific information essential for accurate linguistic reasoning. For tasks requiring precise text extraction and numerical reasoning, the specialized linguistic representations in deeper layers provide more reliable aggregation targets than earlier semantic features.
Early-layer advantage for visual reasoning. Conversely, Aisuall-inttensie benchmarks like OCRVQA and GQA demonstrate superior performance when aggregating at earlier layers (layers 6-12), with performance degrading as aggregation moves toward final layers. This counterintuitive finding reflects the model's 's information processing hierarchy: early layers capture rich multimodal semantic representations before aggressive compression into linguistic tokens. Recent work visual information steering by Li et al. (2025c) reveals that visual information gradually attenuates through transformer layers, with genuine visual tokens losing prominence as language priors dominate in deeper layers. This gradual visual information loss phenomenon explains why early aggregation preserves critical visual details that become diluted in later layers optimized for autoregressive text generation. The early excitation pattern observed in multimodal models (Li et al. (025c) further supports this finding, showing that semantically meaningful visual tokens achieve peak activation in penultimate carlier layers rather than the final output layer. For tasks requiring complex visual understanding and spatial reasoning, these early semantic representations retain critical visual details that are progressively lost in later transformer layers.
Task-specific optimal points. Benchmarks like AI2D, AMBER, and COCO Captions exhibit intermediate optimal points around layers 10-16. suggesting these tasks benefit from balanced deatimodal-linguuiiee representations. This intermediate optimum reflects the complex interplay between visual understanding and linguistic expression required for these tasks. The non-monotonic
21

---

Published as a conference paper at ICLR 2026
patterns observed in several benchmarks indicate that aggregation timing must carefully balance semantic richness against linguistic specificity. This finding resonates with the token ranking dy- namics identified by Li et al. (2025c), who demonstrate that different token types (genuine visual vs. hallucinated linguistic) achieve peak confidence at different layer depths, suggesting that optimal aggregation strategies should account for the hierarchical emergence of multimodal information processing patterns.
The observed layer preferences can be attributed to fundamental architectural properties of VLMs and align with recent discoveries about information flow in transformer-basee multimodal models. Early layers primarily encode multimodal semantic relationships and spatial structures, while later layers increasingly focus on autoregressive text generation and fask-specifie output formatting (Tenney et al. 2019p. This hierarchical specialization creates a trade-off: early aggregation preserves rich semantic diversity, but may introduce inconsistencies in linguistic expression, while late aggregation ensures coherent text generation, but may lose crucial semantic nuances. The dynamic contrastive decoding work of Chuang et al. (024) provides additional theoretical support for our findings, demonstrating that factual knowledge evolves systematically across transformer layers, with different types of information reaching peak reliability at distinct layer depths. Our layer-depnndent aggregation results extend these insights to the multimodal domain, revealing that visual and linguistic information follow distinct developmental trajectories through the network architecture.
From a theoretical perspective, these findings suggest that optimal aggregation requires matching the representational granularity the task demands. Visual reasoning tasks benefit from the semantic spaces of early layers, where diverse augmented views can provide complementary visual interpreta- tions. Conversely, linguistic tasks require the refined representations of later layers, where augmented inputs converge toward consistent textual expressions.
The practical implications are significant for deployment optimization. Rather than universally aggre- gating at final layers, practitioners can achieve substantial improvements by selecting approprpte aggregation points. This layer-aware aggregation strategy could be implemented adaptively, with the aggregation layer selected based on task classification or learned through validation performance. However, the computational overhead of this approach remains modest, as early aggregation actually reduces computation by bypassing later layers for individual augmented inputs.
Notably, the average performance trend shows late-layer aggregation as generally superior, but this global pattern obscures important task-specific exceptions where early aggregation provides substantial benefits. This finding challenges the common assumption that final-layer representations universally optimal for test-time scaling and suggests that hicrarchical aggregation strategies could unlock further improvements in multimodal understanding.
Takeaway: Optimal aggregation layers depend critically task type: sogguage-heave tasks benefit from late-layer aggregation that preserves linguistic refinement, while visual reasoning tasks achieve superior performance through early-layer aggregation that retains semantic richness. adap-tde layer selection can provide substantial improvements over universal late-layer aggregation.
E COMPUTATIONAL OVERHEAD OF TTAUG
A critical consideration for deploying TTAug on resource-constrainee devices is the computational overhead introduced by processing multiple augmented inputs. We analyze two implementation strate- gies that offer different trade-offs between memory usage and inference latency, enabling practitioners to select the most suitable approach based on their hardware constraints and requirements.
Parallel implementation. In the parallel strategy, all N augmented inputs are processed simul- taneously within a single forward pass by concatenating them into a larger batch. This approach maximizes GPU utilization and minimizes wall-clock time by leveraging parallel computation capa- bilities. The memory overhead scales linearly with the number of augmentations, as the model must store activations for all inputs concurrently. Peak memory consumption increases substantially due to the need to maintain intermediate representations for the entire augmented batch during forward propagation.
Sequential implementation. The sequential approach processes each augmented input indepen- dently in separate forward passes, accumulating token-level probability distributions for subsequent
22

---

Published as a conference paper at ICLR 2026
aggregation. While this strategy significantly reduces peak memory requirements by processing only one augmentation at a time, incurs higher latency due to the sequential nature of computation. The modest memory increase observed in sequential processing primarily stems from the accumulation of key-value cache states across multiple forward passes, which must be retained for faster generation. Note that without such key-value caching mechanism, the sequential implementation any platform capable of supporting the baseline small model, ensuring broad accessibility.
Peak k Memory (GB)
Runtime (sec)
Parallel Implementation
21.52
Parallel Implementation
88.87
Sequential Implementation
Sequential Implementation
12.89
8.75
40-
6.75
4.604.60 4.9% 5.43
A.76 4.57
5.68
22.60
11.24
1.431433
2.142.84
2.45
5.66
2.99
8.59
N6
Number of Augmentations
Number of Augmentations
Figure 5: Overhead in peak GPU memory usage and runtime for different numbers of augmentations, comparing parallel and sequential implementation strategies.
The experimental results in Fig. demonstrate distinet scaling behaviors for the two strategies, measured on an NVIDIA A100 GPU. Parallel implementation exhibits substantial memory overhead that grows approximately linearly with the number of augmentations, while sequential implementation maintains relatively constant memory usage with only minor increases due to key-value cache accumulation. Conversely, runtime overhead follows the opposite pattern: parallel processing achieves near-constant inference time regardless of augmentation count, while sequential processing incurs linear time penalties proportional to the number of augmentations.
These complementary trade-offs enable flexible deployment across diverse hardware configurations. For applications with abundant GPU memory but strict latency constraints, parallel implementation provides optimal performance. Conversely, constry-conttraine environments benefit from sequential processing, which maintains feasible memory footprints at the cost of increased inference time. Practitioners can select the appropriate strategy based on their specific resource limitations and performance requirements, with both approaches representing practical extremes of the memory- latency trade-off spectrum.
While our computational overhead analysis was conducted exclusively on NVIDIA A100 GPUs, the observed patterns are highly transferable across different hardware platforms. Peak memory requirements remain platform-agnosticc determined by model architecture and batch size rather than specific hardware. Similarly, the scaling behaviors and relative trade-offs between parallel and sequential strategies exhibit consistent patterns across diverse configurations, confirming that the provided analysis is sufficient for practitioners' reference when deploying on different platforms.
Takeaway: Parallel implementation minimizes latency but requires substantial memory. while sequential implementation conserves memory the cost of increased runtime. The choice between strategies depends on hardware constraints and application priorities.
F MULTIMODAL AUGMENTATION DECOMPOSITION
To understand the individual contributions of different modality-specific augmentations to our TTAug framework, we conduct ablation study that isolates the effects of text-only, image-only, and combined multimodal augmentations. This analysis addresses a fundamental question in multimodal test-time scaling: whether the benefits of joint augmentation components from individual modalities, or whether multimodal synergies introduce non non-linen ainear interactions that exceed the sum of single-modal improvements.
be decomposed into additive
23

---

Published as a conference paper at ICLR 2026
We design three experimental conditions to systematically evaluate modality-speciin contributions. In the text-only condition, we apply classical textual augmentations while keeping the input image identical across all augmented samples. Conversely, the image-only condition applies classical visual transformations while maintaining identical text prompts. The both condition applies augmentations to both modalities simultaneously, representing enables us to quantify the relative importance of each modality and assess whether multimodal interactions produce emergent benefits beyond simple additive effects.
full TTAug framework. This decomposition
No TTA
Accuracy (96)
Image Only
Text Only
482442
Both
22.2277 2143
nenz
GQA
OCRVOA
ChartQA
OCRBench
TextvQA
AI2D MME-Re WME-ReelWorld AMBER Captions Average
Figure 6: Performance comparison across different augmentation strategies showing the relative contributions of text-only, image-only, and combined multimodal augmentations. Fach benchmark demonstrates different sensitivity patterns modality-speciffe augmentations, with text augmenta- tions consistently providing larger improvements than image augmentations across most tasks.
The experimental results in Fig. reveal several critical insights about multimodal augmentation dynamics. First, combined multimodal augmentation consistently outperforms both single-modality approaches across all benchmarks, demonstrating the value of joint augmentation strategies. However, the magnitude of improvement varies substantially across different task types, suggesting that multimodal synergies are task-dependent rather than universally additive.
Second, text-only augmentation emerges as the dominant contributor to performance gains, sub- stantially outperforming image-only augmentation across most benchmarks. This asymmetry is particularly pronounced on language-heavy benchmarks, where textual diversity appears more critical for robust understanding than visual transformations.
Third, our analysis reveals that the combined effect exhibits non-linear characteristics that cannot be predicted by simply summing the individual contributions of text-only and image-only augmentations. On several benchmarks, the joint augmentation achieves improvements that exceed the arithmetic sum of linglemmodling gains, indicating positive synergistic interactions between visual and textual diversity. This non-li ain-inearyty suggests that multimodal augmentation creates richer semantic spaces that enhance the model's ability to extract consistent signals across diverse input representations.
The observed modality asymmetry can be attributed to several fundamental architectural and rep- resentational factors inherent to multimodal language models. First, multimodal language models typically employ heavily compressed visual representations to maintain computational efficiency, often reducing high-resolution images to low-dimensional feature vectors through aggressive pooling or patch-based tokenization (Marafioti et al., 2025). These compression operations inherently filter out fine-gre fine-grained visual details that our image augmentations target, rendering subtle transformations like brightness adjustments or minor rotations largely imperceptible to the model's internal represen- tations. Consequently, visual augmentations operate in a severely constrained semantic space where meaningful diversity is difficult to achieve.
Second, our findings align with recent interpretability research demonstrating that when one modality dominates the reasoning process, variations in the subordinate modality become largely irrelevant to model outputs (Ben Melech Stan et al. 2024). In many of our benchmarks, the textual component carries the primary semantic load, specifying the question type, reasoning requirements, and output format, while the visual component provides supplementary information. This inherent task structure naturally amplifies the impact of textual diversity while diminishing the influence visual variations.
Third, the token-level architecture of multimodal language models creates additional bias toward textual processing. Since both visual and textual inputs are eventually projected into a shared token space for text generation. the model's training predominantly optimizes for linguistic coherence and next-token prediction accuracy. This architectural choice inherently favors modalities that directly influence the language generation process, explaining why textual augmentations, which directly
24

---

Published as a conference paper at ICLR 2026
modify the prompt structure and linguistic context, yield more substantial improvements than visual transformations that must traverse multiple encoding layers before affecting token-level decisions.
The observed modality asymmetry has important implications for practical deployment. Since text augmentation provides disproportionate benefits while requiring minimal computational overhead compared image processing,
applications might prioritize textual diversity generation over complex visual transformations. However, the non-additive nature of multimodal interactions suggests that completely eliminating visual augmentation would sacrifice valuable synergistic effects, supporting our unified approach that leverages both modalities while emphasizing textual diversity. Future work might explore augmentation strategies that operate directly in the compressed visual feature space or develop modality-awaz weighting schemes that account for task-specific dominance patterns.
Takeaway: Combined multimodal augmentation outperforms single-modality approaches through non-linear synergistic effects. Text augmentations contribute more substantially than image augmen- tations, but their combination produces emergent benefits that exceed simple additive predictions.
G DETAILED RESULTS FOR DIFFERENT MODELS
Table 8: Performance comparison across SmoIVLM2 family models (256M, 500M, 2.2B parameters) with no TTA, TTAug, and TTAdapt approaches.
SmolVLMM22250X
SmolVLMM-5000.
SmolVLLM222.2B
No
TT
TT
No TTA
TT
TT Adapt
No TTA
TT
TT
TTA
Aug
Adapt
Aug
Aug
Adapt
ChartQA
65.1
59.4
55.1
64.1
64.8
65.5
74.2
76.1
76.7
OCRBench
56.7
53.3
50.3
61.0
60.0
57.6
72.9
73.7
70.5
OCRVQA
0.2
0.4
0.3
0.0
4.6
5.2
0.0
12.6
13.8
GQA
0.1
5,8
18.4
0.0
0.0
0.9
0.0
5.5
13.5
TexIVQA
47.8
45.1
40.1
59.9
58.0
57.7
73.2
72.4
70.5
A12D
37.0
35.4
34.0
56.6
55.3
52.1
68.5
69.6
67.4
MME-RW
21.0
20.7
27.6
27.6
27.2
31.9
31.4
AMBER
29.5
53.3
43.0
55.3
56.1
52.8
68.7
75.9
72.8
COCO
29.0
40.6
38.5
6.2
9.2
3166
9.1
16.9
35.9
Mean
31.8
35.0
33.4
36.7
37.3
38.9
43.8
48.3
50.3
Table 9; TTAug performance across Ovis2 model family (IB, 2B, 4B, 9B) and Intern VL2-
Ovis2-1B
Ovis2-2B
Ovis2-4B
Ovis2-9B
Intern VL2-1B
No
TT
No TTA
TT
No TTA
TT
No TTA
TT
No TTA
TT
TTA
Aug
Aug
Aug
Aug
Aug
ChartQA
80.4
81.6
86.6
85.9
87.6
87.8
87.4
87.9
72.1
72.1
OCRBench
88.8
84.9
87.3
86.0
91.2
89.2
89.2
87.2
75.7
75.1
OCRVQA
74.3
70.5
76.7
73.1
80.2
76.9
79.3
78.7
43.3
42.0
GQA
30.0
34.5
58.7
40.5
55.7
$2.0
51.3
TexIVQA
79.2
77.2
78.8
79.5
83.5
83.9
83.1
84.0
69.6
67.6
AI2D
76.5
73.3
81.9
82.2
84.9
84.5
87.1
87.2
52.8
52.6
MME-RW
35.5
35.6
38.6
40.5
45.7
44.1
45.7
46.5
13.5
13.3
AMBER
73.8
84,9
85.9
87.4
87.4
87.3
72.6
75.7
COCO
22.7
13.7
17.3
13.1
14.0
12.5
13.8
13.3
17.2
24.6
62.6
62.8
65.2
67.2
68.3
69.1
70.3
71.0
52.7
Note that TTAdapt method is not implemented for Ovis2 and Intern VL model families due to practical constraints; the Unsloth library does not currently support those model families yet.
25

---

Published as a conference paper at ICLR 2026
Table 10: Evaluation on diverse baseline models for ref- erence. These models are evaluated without our meth- ods just to establish performance baselines across differ- ent architectures. Baseline results are shown in Fig.
Baseline
TTAug
Owis2
65
TTAdapt
Ovisz
Molmo-D
60
Ovis2
XGen-MM
Accuracy (9)
LLaVA-OV
65
PaliGemma
Pali
xGen -MM
LLaVA Molmo Idefics
Janus -Pro
50
InternVL2
Gemma
-OV
-D
ChartQA
40.7
65.0
72.3
85.8
31.6
31.0
40-
SmotvLM2
Idefcs
OCRBench
61.4
55.5
61.2
66.3
63.4
58.9
35
Janus-Pro
OCRVQA
61.2
70.7
69.5
44.9
0.0
2.5
SmotvLA2
GQA
61.5
60.2
62.5
55.1
0.0
13.7
30
0.5
TextVQA
70.7
72.8
60.8
81.5
72.6
55.0
Model Parameter Size (B)
A12D
67.9
73.5
78.2
80.7
72.2
67.5
MME-RW
25.4
35.1
31.1
36.8
34.3
23.4
Figure 7: Performance improvements
AMBER
84.9
82.1
84.4
85.0
85.4
74.8
across different models. Each point repre-
COCO
45.9
15.7
13.9
12.1
24.4
18.0
sents different model-strategy pair; x-axis shows model parameter size (B) using asinh scaling, and y-axis shows accuracy (%).
Mean
57.7
59.0
59.3
60.9
42.7
38.3
H SMALL VISION-LANGUAGE MODELS
The Transformer architecture (Vaswani et al. 2017) revolutionized language modeling, enabling models like BERT (Devlin et al. 2019) through bidirectional pretraining and GPT (Radford et al.
2019 Brown et al.) 2020) via autoregressive generation. These foundational advances led to large- scale models such GPT-3 (Brown et al.) 2020) with human-like text generation abilities. More recent developments have emphasized efficiency, with LLaMA (Touvron et al. 2023) demonstrating that smaller, well-trained models can outperform earlier, larger counterparts. Open-source families including Qwen (Bai et al.] (023), InternIM (Team] (023), and Gemma (Team] 2024) further
expanded access capable language models. In the multimodal domain, CLIP (Radford al. 2021)
introduced contrastive vision-language pretraining, facilitating strong zero-shot visual understanding. This inspired the integration of vision encoders with LLMs to produce multimodal large language models, such as GPT-4V (Achiam et al., (2023). LLaVA (Liu et al. (202)). Qwen- VL (Bai et al., (2023).
and Intern VI. (Chen et al.) 2024b). Notably, Molmo (Deitke et al.52025, provides transparency by releasing full training data and evaluation protocols. Recently, the emergence of small vision-language models or multimodal small language models, models under 10B parameters, has shifted attention toward efficient, accessible architectures suitable for edge deployment. Examples include Ovis2 (Lu 2025), InternVL2 (Chen et al.] 2024b). Janus-Pro (Chen et al. (025). Idefics2 (Laurençon 2024). LLaVA-One Vision (Li et al. 2025a), Molmo (Deitke et al.] 2025), XGen-MM (Xue
ct al.
et al.
et al.
2024), PaliGemma (Beyer et al. 2024), and the SmolVLM family (Marafioti et al.) 2025). with
models as small 256M parameters. These compact models achieve competitive performance on vision-language benchmarks while significantly reducing computational cost, making them attractive for real-world, resource-cone constrrinnd applications. They offer compelling advantages for practical deployment: they enable inference on consumer GPUs and edge devices, support local processing, and demonstrate superior post-performance ratios for specialized tasks (Belcak et al. 2025). But, their limited parameter capacity makes them particularly vulnerable to domain shifts, various biases, and distribution mismatches at inference time.
I IMPLEMENTATION DETAILS
I.I SELF-SELECTOR
Self-selector uses the tested VLM itself as a verifier to select one response among the candidates (Chen et al.) 2024a Parmar et al. 2025). We enforce the VLM to choose between available indices ranging from 0 to the number of augmentations. Since small VLMs are not capable of reliably following this constrained output behavior through prompt engineering alone, employ structured generation techniques to guarantee valid responses. We use the Outlines library (Willard & Louf 2023) for structured generation. We the prompt given below:
26

---

Published as a conference paper at ICLR 2026
Prompt
"rrpput_questinn
Different people answered this question in different ways. Select the best response from these candidate answers:
(responses]
Just return the index of the best response. Return an integer between O and (n_aug).
1.2 SHLF-SYNTHESIZER
Self-synthesizer method uses the tested VLM to aggregate responses into coherent final answer
(Li et al. 2025d Jiang et al., 2023 Wang et al. 2025a Li et al. 2025b). We use the prompt given
below:
Prompt
"[input_question"
Different people answered this question in different ways. Combine these responses into a single, coherent and accurate answer:
(responses)
Just return the final answer.
1.3 SELF-PARAPHIRASING
Self-paran paraphrasing uses the text backbone of the tested VLM to paraphrase the input prompt. Since the model is not good enough to do this one shot, we split the prompt into sentences and feed each sentence to the model to paraphrase using structured generation to get fixed number of paraphrases. After that, we concatenate all paraphrased sentences to get the final paraphrased prompt. This approach maintains consistency with the target model's internal linguistic patterns while remaining self-contained. We use the prompt given below:
Prompt
You are an expert paraphraser.
Your task is to paraphrase input text without changing its meaning. Keep the details and core content. Generate [n aug) paraphrased versions. Return your output JSON object with the key "paraphrases", mapped to list of (n Jug] unique paraphrased versions.
Now, paraphrase the following text:
Since small VLMs are not capable of paraphrasing complex long prompts reliably in one shot, we first split the input text into individual sentences using spaCy (Honnibal et al. 2020) for sentence splitting. We then paraphrase each sentence independently. Also, since small VLMs are not capable of reliably following a constrained output behavior, outputting the exact number of paraphrases, through prompt engineering alone, employ structured generation techniques to guarantee valid responses. We use a JSON schema that enforces an output with exactly desired number paraphrases.
After obtaining paraphrases for each sentence independently, compute the Cartesian product across all sentence-level paraphrase sets to generate diverse combinations of the complete prompt. This approach produces final paraphrased prompts by systematically combining different paraphrased versions of each sentence, ensuring both local sentence-level diversity and global prompt-level variation while maintaining semantic consistency.
27

---

Published as a conference paper at ICLR 2026
1.4 CLASSICAL IMAGE AUGMENTATIONS
We implement classical image augmentations using the Albumentations library (Buslaev al. 2020). For each input image, we randomly select three transformations from our predefined set and apply them sequentially through composed transformation pipeline. This random selection approach ensures diverse augmentation combinations while maintaining computational efficiency. The predefined sets for different augmentation strengths are given below.
High
A. h.TanddonariighhreesCotttat (p=0.6),
A. B.SdfefRttate p=0.6, fi11=144),
A. G.sassiianBBurra (blur A.CIAAH
7), p=0.6),
A. (p=000)
A. Htesstturatioovaaueepppp..
A. RanndosSSsieerssalll__iitttbt 1im p=066),
A. (p
[eeediai@Buurriaaee_iiittt,,,,, p=0.6),
A. imngccooppreaiiioo..uaaaaaaaaaaa (85, 95), p-0.45),
A. Sharrpe
A. :t...cckiannittterr).....
A. Randon/(oeeCuree()
A. Emboss
A. (.GridDsttootiinn()
A. Perspectiveescaaeee 05, fit_output-True),
A. G.GridDropout (ratio=0.25 zandom_ offedmmefffe #il1=144, p=0.66),
A. ..Corsserrooout (fi1l=144, p-0.7),
Medium
A. Handomttriggttreestootttaaass (wwighttssssssssiiiiitii.
cuntrast_liitt-0
A. [.iii wwwder__ooeeeeeeeeeeeeeeeeeeeeee mode=ev2. BORDDER oo0 fi11=144),
A. G.russsiannLuurrbbbbe..aaiiiii, 7), p=0.5),
A. CLASE (La 120), 120), p-0.5),
A. RandonGGamm_ccaammaaaiililleee,
Htt.ttuuetoooolllloocoeaaaaaaaaa sat_shifttllmit-115
+ vaLehhifc_Jlnntc-11pppp.....
A. -.aaddam_ccieeeaaaaae...i....... p-0.5),
p
A. [etaghhhttiiaahhfiiiiiiii.a.aaaa
A. Mediaaniarrrbll__..in-333......3 3.gn b_shift_limit=11,
A. Ing.gooaapreeiioooooaaaaaaaaaege 95), p=0.35),
A. 0haaarpeeeeaappa111222.....,,,,, 0.9), (ights 1.0), p=0.5),
Lightness=
A. 1.2_Leccciaac21teee1100......
A. Handdmr/oggallpaa_ccefr....,,, ))
A. 2andem/oonturree/scaeee11222,,,, p-0.5),
o.t (0.5, 0.7), p-0.5),
A. siddiistorrioo.nnmn_stttesssss distort 1imit=0.2, p=0.5),
A. T.er:peeciiveeecaalee0..0011 fit cut_uuttutrreee p=0.5),
A. Gri.Drrpoout (1at:o000,,0. randon_ offdomooffse--rree fil1=144, p),
A. CoarscDropout (f p=0.5),
28

---

Published as a conference paper at ICLR 2026
Low
A. A.BarddoBBFiiitttrssootaaaa (atp://onttttttti..iiaaaaeeeaaaa (brightness_ Contrast_limi==0-1
+ p-0.3),
A. (1 1immmmmm0)) p=0.3, border_mode BOKKR HORDERR
fi11-144),
A. (beuussianBBuurruuuuuiilii
5), p=C.3),
A. 0, p=0.)) p=0.3),
A. wtw.adddeeemmaagggaaaaaaaaaaddea 110), p=C.3),
A. httpsstwwwttooalllloo....oa.aaae
eat_shift_limit-e
val_shifft_limtt-11,pp0000
A. 1..eadorsccllieecall__ittto.... p-0.3),
A. ( p=0.3),
A. Neddiiin8llrrrbblor__iiii---.-.. p=0.3)
A. wtt.gaacwaaaea.a......aaaaaaaaaa (85, 95), p=0.25),
A. ww_.aarreeeaaapaaaa00000........ 0.3),
4tg.app......................... Lightness=(0.1) 1.0), p=0.3),
A. [.22aackklaccitee:((p0......
A. Hend@prroog(e1pppa_ccefdd001 p-0.2),
landom/lonstarreefcccalee.......
p-..3))
(mbossa 0.3), 8trength- (0.3, 0.5), p-0.3),
A. GridDisstorriionnnnnnnteeeeeeeee distort limit=0.1, p),.3),
A. ieeeepppctivveeccaaeeeeeee
fit fit_output- p=0.3),
1.5 GENERATIVE IMAGE AUGMENTATIONS
Generative augmentations use FLUX. 1-dev (Labs al. 2025) to create semantically similar but visually distinct image variants. We employ an image-to-iman pipeline that, unlike traditional flow matching which starts from random noise, begins denoising from a fixed intermediate timestep with a noisy version of the input image. This approach preserves semantic similarity the original while introducing visual diversity through the prompt "realistic image."
However, recent generative models struggle with creating images containing textual ele- ments (Bosheah & Bilicki 2025). Therefore, our approach excludes text-containing images to prevent OCR corruption, using Tesseract for text detection (Smith 2007). Two key hyperparameters control the generation process: strength (chosen as 0.25) determines the initial denoising timestep, lower values preserve more of the original image structure, and guidance scale (chosen as 3.0) controls the classifier- free guidance parameter.
While this method produces diverse and consistent image variations, it requires external diffusion models and significant computation budget. It is not practical for resource-constraiigg deployments.
1.6 AGGREGATION WEIGHTS OPTIMIZATION
Aggregation weights optimization learns adaptive token-wise weights way to replace the uni- form averaging scheme in Eq. 2 At each generation step j. we initialize learnable parameters as w E R and optimize them through gradient descent to minimize the marginal entropy H e Py (e) log (v) of the weighted aggregated distribution. The optimization em- ploys Adam W with adaptive learning rates and gradient clipping for stability, performing multiple micro-steps per token to achieve convergence. This approach requires minimal computational overhead with compact computational graph, making it suitable for real-time deployment.
Optimization Parameters. We use the AdamW optimizer with an initial learning rate of x 10-2 and weight decay of 1 10-4, The aggregation weights w initialized uniformly W 1/N where N is the number of augmentations. We perform 20 optimization micro-steps per token generation step to ensure convergence of the entropy minimization objective. We reinitialize the aggregation weights before processing each new question to ensure independent optimization across different inputs.
Gradient Clipping, To maintain training stability, we apply gradient clipping with a maximum norm of 1.0. This prevents gradient explosion during the iterative optimization process.
29

---

Published as a conference paper at ICLR 2026
Numerical Stability. We add small epsilon value of 1 x 10- to the logarithm computation in the entropy calculation to prevent numerical instabilities when probabilities approach zero. The softmax temperature is kept at the default value of 1.0. At each optimization step, apply softmax
cxp(O..))
normalization to the raw learnable parameters to ensure the weights sum to I: Wijj Σ (xp()
where 8, the raw learnable parameters.
Computational Efficiency. The optimization process uses detached probability distributions from the forward pass to prevent gradients from flowing back through the entire model, maintaining the compact computational graph.
1.7 MODEL PARAMETER ADAPTATION
Model parameter adaptation (TTAdapt) performs iterative fnne-tunin during inference using pseu- dolabels generated from TTAug consensus. The method employs full parameter fine-tuning with gradient checkpointing for memory efficiency and implements three-stage iterative loop: pseudola- bel generation, parameter updates, and weight reset between questions.
Training Configuration. We use the AdamW optimizer the Unsloth (AI et al. 2025) library. The learning rate is set to 2 x 10-6 with cosine learning rate scheduler and 5 warmup steps. We apply weight decay 0.01 for regularization and perform training steps per pseudolabel iteration with a batch size of 64 and gradient accumulation steps of 2.
Iterative Adaptation Process. We perform 3 pscudolabel iterations per question. Each iteration generates pseudolabels using the current model state with TTAug consensus (average aggregation), then fine-tunes the model parameters using these pseudolabels as supervision. The final iteration generates the output without additional training prevent overfitting.
Resetting Weights. A fundamental challenge continual test-time adaptation is catastrophic forgetting (Niu al., 2022 Wang al. (022), where models suffer severe performance degradation on original training samples after adaptation. During cample-by-aapple adaptation to test streams, models can lose important information through unsupervised learning. causing rare domains to disappear while abundant ones dominate. One solution involves episodic adaptation, which means restarting from the original model for each sample rather than continual learning. Thus, our method, model parameters are reset to their initial state before processing each new question to prevent catastrophic forgetting.
1.8 IMPLEMENTATION DETAILS FOR TTAUG
TTAug implementation requires careful integration with the model's generation pipeline enable efficient token-level aggregation while preserving KV caching and other optimization features. We achieve this through dynamic method patching that intercepts the sampling process without disrupting the underlying generation mechanics.
Monkey patching critical for KV cache compatibility. We override the model's _samp____leeee_ method to inject our aggregation logic while maintaining compatibility with existing optimizations. The patched method preserves the original sampling interface but intercepts logits before token selection to perform aggregation across augmented inputs:
The modified sampling method extracts logits from multiple augmented forward passes, applies the specified aggregation strategy (uniform averaging, learned weights, or entropy optimization), and returns aggregated token selections. This approach enables seamless integration with existing generation pipelines, including beam search, nucleus sampling, and temperature scaling.
Our implementation leverages KV caching by processing augmented inputs in batches and sharing cached valuv pairs across the prefix tokens. The aggregation computation adds minimal overhead as it operates only on the final logits rather than intermediate representations, maintaining the model's inference speed while enabling test-time adaptation.
The patched method maintains full compatibility with the Transformer (Woff et al. 2020 library's generation utilities, preserving advanced sampling techniques such top-k, top-p. and temperature scaling. The aggregation occurs at the logit level before these sampling strategics are applied, ensuring that the enhanced diversity from TTAug benefits from sophisticated decoding procedures.
30

---

Published as a conference paper at ICLR 2026
J EVALUATION METRICS DETAILS
We provide detailed mathematical formulations for the evaluation metrics used across all benchmarks in our study. We carefully selected nine benchmarks from VLMEvalKit (Duan et al. (022) to ensure representative, reliable, and reproducible evaluation while maintaining computational feasibility for extensive ablation studies. Our selection prioritizes benchmarks with objective evaluation metries (visual question answering. multiple-choice questions, yes/no questions, and captioning tasks) over LLM-as-a-judge approaches, which suffer from model bias and lack reproducibility. We exclude text-dominant benchmarks as well as specialized benchmarks focused on specific domains. The selected benchmarks represent diverse visual reasoning capabilities.
J.1 EXACT STRING MATCHING (OCRVQA, GQA)
For datasets requiring exact string correspondence, we define the accuracy metric as:
Accuracy
Σ
1b. = =yl
(4)
N t=1
where ú: is the predicted answer and is the ground truth answer.
J.2 VQA SCORE WITH INTER-ANNOTATON AGREEMENT (TEXTVQA)
Following the standard VQA evaluation protocol that accounts for multiple valid answers and inter-annotator variability:
N
A.
VQA Score
min
1, Σ j=1
Toe = vns1
(5)
N
3
j#k
where A₁ is the of ground truth answers for question i, Mt.y represents the j-th ground truth answer, and & indexes through each ground truth answer to simulate the leave-one-out evaluation process. For each answer Yi.k, count how many of the remaining annotators (j # k) would agree with prediction matching that answer. The factor 1/ reflects the standard VQA scoring that considers an answer correct if at least 3 out of |A, annotators agree. In order to be consistent with "human accuracies", machine accuracies averaged over all (p.. with leave-one-on evaluation process.
[A:l sets of human annotators
J.3 RELAXED STRING MATCHING (CHARTQA)
For numerical and chart-based questions requiring approximate matching:
Relaxed Accuracy
max R(g., 3i.j) JEA,
(6)
where A, is the set of acceptable answers for question i, and R(g. y) is defined as:
I [e vol [
0.05 if both numeric
R(j.v)
otherwise
where 1y and uy represent the numerical values extracted from the predicted and ground truth answers,
respectively.
J.4 SUBSTRING CONTAINMENT MATCHING (OCRBENCH)
OCRBench evaluates text recognition performance using substring containment matching:
Accuracy
max 1(yi, S út] JEA;
where A; represents the set of acceptable answers for question i, and C denotes substring containment.
31

---

Published as a conference paper at ICLR 2026
J.5 MULTIPLE-CHOICE AND YES/NO EXTRACTION MIME-REALWOO AI2D, AMBER)
For multiple-choice and yes/no questions, we extract the choice label from predictions and perform exact matching:
Accuracy 1 Σ c4
(9)
where & E (,,.). {yes, not is the correct choice label for question i, and 1 is the extracted choice label from the predicted answer 9s-
J.6 ROUGE-L EVALUATION (COCO CAPTIONS)
evaluate captioning quality using ROUGE-L, which measures the longest common subsequence between predicted and reference captions:
ROUGE-L Pies RLes
(10)
Pias + RI.s
where the precision and recall defined
ILCS v)1
Ptcs
(11)
j
LCS v)1
Ra.cs
(12)
ly]
and (CS( v) computes the longest common subsequence between the predicted caption ý and reference caption y, with | denoting sequence length.
For multiple reference captions, we compute ROUGE-L against each reference and take the maximum score:
ROUGE-Lmuti max ROUGE-L(Ú, v4.)
(13)
JER,
where R₂ is the set of reference captions for image
J.7 IMPLEMENTATION NOTES
All text preprocessing follows consistent normalization procedures: (1) converting to lowercase, (2) stripping leading and trailing whitespace, (3) replacing multiple consecutive spaces with single spaces, and (4) removing newline characters where appropriate. For mathematical expressions OCRBench, additional preprocessing removes all whitespace to handle formatting variations. For ChartQA relaxed matching, numerical values are extracted by handling percentage symbols (converting "X%" to X/100) and parsing floating-poit numbers. For multiple-choice extraction in and A12D, choice labels are identified using regular expressions that match single uppercase letters (A-Z) appearing in isolation with minimal surrounding punctuation. For detailed implementation specifics and evaluation protocols, refer to VI.MEval (Duan et al. 2024).
K STANDARD ERROR VALUES
This appendix presents the standard error values corresponding to all experimental results reported in the main text tables. We calculate all reported standard errors using the empirical variance of the observed per-sample scores 81,..., Sn. Let ş = 1 Ee 8; denote the sample mean. This average accuracy is the value reported in the main text tables. Following the Central Limit Theorem, the corresponding standard error is estimated as
1 Σ (s: 8)2 /n.
SECLT. VVar(s)/n=
1
/
32

---

Published as a conference paper at ICLR 2026
Table 11: Comparison of our TTAug method against existing test-time scaling methods. Our method
outperforms all others across accuracy and efficiency metrics.
(a) Mean accuracy values
(b) Standard error values
Baseline
Others
Ours
Baseline
Others
Ours
CharQA
74.2
74.4
73.4
72.5
71.7
75.6
ChartQA
1.4
1.4
1.4
1.4
1.4
1.4
OCRBench
72.9
72.6
71.9
70.2
71.9
73.4
OCRBench
1.4
1.4
1.4
1.4
OCRVQA
0.0
0.0
0.0
0.0
0.2
11.8
OCRVQA
0.0
0.0
0.0
0.0
0.1
1.0
GQA
0.0
0.0
0.0
0.0
0.0
5.8
GQA
0.0
0.0
0.0
0.0
0.0
0.7
TextVQA
73.2
72.6
71.6
69.5
72.0
72.8
TextVQA
1.3
1.3
1.4
1.4
1.3
1.3
A12D
68.5
3.1
69.2
69.1
67.4
68.8
A12D
1.5
0.5
1.5
1.5
1.5
1.5
MME-RW
27.6
27.6
31.1
MME-RW
1.4
1.4
1.4
1.4
AMBER
68.7
70.4
64.5
53.5
67.8
75.4
AMBER
1.5
1.4
1.5
1.6
1.5
1.4
COCO
9.1
8.2
8.4
6.2
16.7
15.9
COCO
0.1
0.1
0.1
0.1
0.4
0.3
Mean
43.8
36.4
42.8
41.0
43.9
47.9
Mean
0.4
0.3
0.4
0.4
0.4
0.4
Table 12: Comparison of diversity-indueemenn methods compared to the Baseline. Input Perturbation outperforms Temperature Sampling.
Mean accuracy values
(b) Standard error values
Baseline
Temperature Sampling
Input Perturbation
Baseline
Temperature
Input Perturbation
Sampling
ChartQA
74.2
74.4
73.4
74.8
70.9
ChartQA
1.4
1.4
1.4
1.4
1.4
OCRBench
72.9
72.6
71.9
72.7
73.1
OCRBench
1.4
1.4
1.4
OCRVQA
0.0
0.0
0.0
12.0
4.5
OCRVQA
0.0
0.0
0.0
1.0
0.7
GQA
0.0
0.0
0.0
7.6
3.7
GQA
0.0
0.0
0.0
0.8
0.6
TextVQA
73.2
72.6
71.6
72.3
72.9
TextVQA
1.3
1.3
1.4
1.3
1.3
A12D
68.5
3.1
69.2
66.6
A12D
1.5
0.5
1.5
0.6
MME-RW
27.8
26.2
26.4
30.8
29.6
MME- RW
1.4
1.4
1.4
1.5
14
AMBER
68.7
70.4
64.5
72.7
67.0
AMBER
1.5
1.4
1.5
1.4
1.5
COCO
9.1
8.2
8.4
21.2
13.0
COCO
0.1
0.1
0.1
0.4
0.3
Mean
43.8
36.4
42.8
40.9
44.6
Mean
0.4
0.3
0.4
0.4
0.4
Table 13: Comparison of Answer-level versus Token-level aggregation methods. Token-level aggre- gation outperforms other approaches.
(a) Mean accuracy values
(b) Standard error values
Baseline
Answer-level
Token
Baseline
Answer-level
Token
ChartQA
74.2
74.8
70.9
61.1
72.8
75.6
ChartQA
1.4
1.4
1.4
1.5
1.4
1.4
OCRBench
72.9
72.7
73.1
60.9
71.1
73.4
OCRBench
1.4
1.4
1.4
1.5
1.4
1.4
OCRVQA
0.0
12.0
4.5
0.2
3.3
11.8
OCRVQA
0.0
1.0
0.7
0.1
0.6
1.0
GQA
7.6
3.7
0.0
0.0
5.8
GQA
0.0
0.8
0.6
0.0
0.0
0.7
TextVQA
73.2
72.3
72.9
61.6
71.6
72.8
TextVQA
1.3
1.3
1.3
1.5
1.4
1.3
A12D
68.5
3.6
66.6
69.9
68.0
68.8
A12D
1.5
0.6
1.5
1.5
1.5
1.5
MME-RW
27.8
30.8
29.6
29.0
29.2
31.1
MME-RW
1.4
1.5
1.4
1.4
1.4
1.5
AMBER
68.7
72.7
67.0
58.9
75.8
75.4
AMBER
1.5
1.4
1.5
1.6
1.4
1.4
COCO
21.2
13.0
8.6
29.5
15.9
COCO
0.4
0.3
0.1
0.3
Mean
43.8
44.6
38.9
46.8
47.9
Mean
0.4
0.4
0.4
0.4
0.4
0.4
33

---

Published as a conference paper at ICLR 2026
Table 14: Comparison of text augmentation strategies. Self-Paraphrasing 2 and Classical Augmen-
tations 3 consistently perform best.
(a) Mean accuracy values
(b) Standard error values
Baseline
Baseline
ChartQA
74.2
74.7
76.9
76.6
76.1
71.4
ChartQA
1.4
1.4
1.3
1.3
1.3
1.4
OCRBench
72.9
73.3
73.5
72.8
73.7
70.6
OCRBench
1.4
1.4
1.4
1.4
1.4
1.4
OCRVQA
0.0
0.0
2.6
0.0
12.6
0.0
OCRVQA
0.0
0.0
0.5
0.0
1.0
0.0
GQA
0.0
0.0
0.0
0.0
5.5
31.2
GQA
0.0
0.0
0.0
0.0
0.7
1.5
TextVQA
73.2
74.2
73.5
74.0
72.4
63.9
TextVQA
1.3
1.3
1.3
1.3
1.3
1.4
A12D
68.5
69.8
69.9
68.4
69.6
63.9
A12D
1.5
1.5
1.5
1.5
1.5
MME-RW
27.8
26.6
30.0
25.9
31.9
32.1
MME-RW RW
1.4
1.4
144
1.4
1,5
1.5
AMBER
68.7
64.7
68.8
72.9
75,9
60.0
AMBER
1.5
1.5
1.5
1.4
1.4
1.5
COCO
9.1
8.4
20.6
46.2
16.9
13.2
COCO
0.1
0.1
0.4
0.5
0.3
0.2
Mean
43.8
43.5
46.2
48.5
48.3
45.1
Mean
0.4
0.4
0.4
0.4
0.4
0.4
Table 15: Comparison across different image augmentation strategies. Classical Augmentations L H perform the best.
(a) Mean accuracy values
(b) Standard error values
Baseline
Baseline
H
M
CharQA
74.2
75.8
77.0
76.4
76.1
74.1
75.7
ChartQA
1.4
1.4
1.3
1.3
1.3
1.4
1.4
OCRBench
72.9
73.1
73.7
73.3
72.4
65.3
OCRBench
1.4
1.4
1.4
1.4
1.4
1.4
1.5
OCRVQA
0.0
13.5
12.1
10.6
12.6
12.9
12.0
OCRVQA
0.0
1.1
1.0
1.0
1.0
1.1
1.0
GQA
0.0
2.0
4.1
3.7
5.5
3.1
2.5
GQA
0.0
0.4
0.6
0.6
0.7
0.5
0.5
TexiVQA
73.2
73.0
72.6
73.3
72.4
72.4
71.6
TexiVQA
1.3
1.3
1.3
1.3
1.3
1.3
1.4
A12D
68.5
68.1
69.1
69.0
69.6
68.9
67.0
A12D
1.5
1.5
1.5
1.5
1.5
1.5
1.5
MME-RW
27.8
31.6
31.8
32.5
MME-RW
1.5
1.5
1.5
1.5
1.5
1.5
AMBER
68.7
77.3
77.0
75.9
75.9
77.3
76.2
AMBER
1.5
1.3
1.3
1.4
1.4
1.3
1.3
COCO
9.1
19.0
17.8
17.1
16.9
17.8
18.0
COCO
0.1
0.3
0.3
0.3
0.3
0.3
0.3
Mean
43.8
48.2
48.3
48.0
48.3
47.9
46.6
Mean
0.4
0.4
0.4
0.4
0.4
0.4
0.4
Table 16: Performance comparison of test-time adaptation strategies. Model parameter adaptation yields the best performance.
(a) Mean accuracy values
(b) Standard error values
Baseline
TTAug
Baseline
TTAug
ChartQA
74.2
76.1
76.1
76.7
ChartQA
1.4
1.3
1.3
1.3
OCRBench
72.9
73.7
73.0
70.5
OCRBench
1.4
1.4
1.4
OCRVQA
0.0
12.6
11.9
13.8
OCRVQA
0.0
1.0
1.0
1.1
GQA
0.0
5.5
5.2
13.5
GQA
0.0
0.7
0.7
1.1
TextVQA
73.2
72.4
74.2
70.5
TextVQA
1.3
1.3
1.3
1.4
A12D
69.7
67.4
AI2D
1.5
1.5
1.5
MME-RW
27.8
31.9
30,9
31.4
MME RW
1.4
1.5
1.5
1.5
AMBER
68.7
75,9
76,9
72.8
AMBER
1.5
1.4
1.3
1.4
COCO
9.1
16.9
16.4
35.9
COCO
0.1
0.3
0.3
0.4
Mean
43.8
48.3
48.3
50.3
Mean
0.4
0.4
0.4
0.4
34

---

Published as a conference paper at ICLR 2026
L QUALITATIVE RESULTS
Both classical text augmentations (Sec. 4.5) and classical image augmentations with high strength (Sec. (466 are applied, with 16 augmentations per sample. Thus, the shown cases correspond to samples underlying the quantitative results in Sec. 4.7
ChartQA
Original Inputs
Augmented Prompts
Froep: Sigarar other saycifoo the peett fos selrey fotlow the following
TTh sheurd coskati sorts poosible
-Acaee question wiEl
<Mem asked glee romericst we They preetiee natter The resteed of "en. poocids
-Dbo asted to give catto, give ouc thk deciasl value like 2:25 inateed
Whhn axted glee percentagyy gliv ast the whole valce Like 11 incoead of
:Donts leclide arite arawer,
-Bo out praciate any atopa axd axsaes.
possttons Hhiah cossecp had the no an vi aitocs to tasy in 2 1183 ana wer the beas visttora
please.
20183
Kuomn cusaiiin saara a aungle mard phasse
Doastion: shich coulity had ean vin stoce ieady 0503 Roaw ebite plocint tn Millin sae Itady
Ecoeps Ieee teancated, aura Fooags
Lestoce any in 23 201 aos
quest
Ecopp: Jore seascated, aaea Frongs aod
cesgics sesno skig wurd she ais, other veeds, mike cocnedy had
Froep:
aaea Frong a.d ozice segiet
hease. in otine vecoa, Maice country aas
nest vistbora Italy 2035 Teas terecated, sene Dranpt
Froep: Tore
waen Fromes
Which country had the most visitors to Italy in 2018? Augmented Input Images
geet bear visteore Eracy in 2018 fer seascated, zare Fcoeps
auegle uned you otier
Frump Toue sessestedt war ceuutiion
to beat visttoca to Eracy in 20132 for. seascated, zara Scoaps
Ies. teascaced, sare is kooags a.. romce the countey the ocsz varibore staly 20157 (o.. sesecaced, sate incrpe
Ecoepe 1o
Ecoeps as Ie.. teascaced, saca is kroags a.d bheael
ocher
han
Froepe in Ier. teascated, AME Frongs and the most
staly 20157 (o.. tesecated, sase incape aod
Staly beee tesecated, Pearge
Froepe sesscated, Frongs codl
rary the visitorn tealy
the near winibore to Iealy 20107 Ier. sececaced, sate Deange ..al
Prumpt andd
posations Which codse by had the west visiteen it 0201 aver the seat vindbore to Iealy 2536 lor. sececaced, sate Searge
From conuttdd
poastions seich curney had the visltoca taly in 2018) aoer the pcss viadbore to staly in atids ler. sececaced, sate as searge
pansel lase on eneet my had the Froege seas vasstoaa teady in 20183 lene seascated, aaea Fcoaps and
scoeps 12 sesscated,
Doest lonn cnrst by had the the woss vasabock stady deaky sesecated, sase ieaage
The 2014t fender the
posattons thich coustry had the nose vistsses to toaty in 20139 Cor. cruncated.
Baseline Output
TTAug Output
TTAdapt Output
Answer: France
Answer: Germany
Answer: Germany
Accuracy: 0.0%
Accuracy: 100.0%
Accuracy: 100.0%
35

---

Published as a conference paper at ICLR 2026
OCRBench
Original Inputs
Augmented Prompts
Promett an
wite, start (itber cimsstiater the cocore take recelets tkip queatiic the cact the laage diceeday. Es atter wards, stat the total dicectig.
very brtef arawer, www._o...ttteeeneee.. stast:
the us Loage slcectyy to osass sont,, axst sutas decast ct tai Reas
tata cosatien extng the the inace dieectiy.
Fromut 21 steeaaaa (ther Conpeeaetee. is lopar coaadly iais gpe das uning lauge Streetly. in other wards, shat the tocal stoere slow racy beied acaveey sadd peteraace
COS
201
Froaps stars (ntieeo ciaapeooo tetak amsest of ceckips? keaser ths pecsigst acaree s318 quarties caing the cast the inage dieectiy. Toul
5.00
(ST Ino
71:30
the cast in lmage, 26 other veeda, whas is the cotal sscuns of tose peckigt) brind wtw.ee/oooo_....eeeee....
directly.
Acalutasts
Frougt 5i ciiacatace pitter conagesetac La the total staeee sncsigts gusstar lmage sifeeslyy
what
diceetlly.
Give very beied arsver, send ceterancee
Froupt stare intleer songpeetar total Asaust this seceipe) koaver this question ustng the tac is the lmage diceetly. kaaletase:
what the total amount this receipt? Answer this question using the text in the image directly.
quarcics caing cast in the zs other soeds, shat La the tocal sscers of cola foot stants
beist ondd peteeracee
Augmented Input Images
Froaps E: x stace (ntpee) conagaoda the tctal anoust of this cavaagez saoust this oft___f_
tes
<iia_ (otterr cinaperetar total absurt of cais untng the the lease dyceeity In other steth what is the tocal iouue of very betef analutant:
siceetiy.
Frompt LEc <ils start siseeetss ancett of chist kraver to coeatas recstas? kraver cots qusrtion caing the Biw aeny brist
iaace dieettl..
thix untrg Llage. other seeds, whst the raous this coseupt
usssiti saing the tn the ssgg diaerlyy.
Assletatt:
Froept <lla seace cisrceiaaaaei satal saoun of this Jasess this maan the coa the other winthy shot is Ihe total swears of tois betef dwwt.r comeeer
dieectiy.
Froeet <its anare othern serett awors recetre moet ton sat cvts estin the there dirertly.
vasy boint oaod jetcerasees
the in disectlyy other words, what total this reonigt? cass cusicia ssong the test tae stage dicestiyy asasatasts
Froept ki seact otbser ehseeeneee sotal sscuns of this recetge? fesaer Loss sseaa sain Lae Loat unage disestiy..
Baseline Output
TTAug Output
TTAdapt Output
Answer: 100.00
Answer: 71.10
Answer: 71.10
Accuracy: 0.0%
Accuracy: 100.0%
Accuracy: 100.0%
36

---

Published as a conference paper at ICLR 2026
OCRVQA
Original Inputs
Augmented Prompts
Promet Lecattrtt jothert cineeniteeer the opent entrn stretle ward or this book?
Acame quertiss sxing stegle ward plzass.
Roal stalistatt:
Fromst xite, start (nbber conpriempeer the qpantier eaing strale word prease. cas suttor stax boxt be other warda, Roo tie acchor at aeaser the quesiies caing steple word plosss.
Analetast:
Smuck Goart
pouts the rultinn ot thi texe? otter was Liee mnder
2O15 CAL
acame quaries stegle word
TWLL Or TTK LOTUS
Froupt sise cinpprncoee the qussice sxint stcple or acthor bookz other
authir
FLO' roun 291 FLAZIIT MANTIM
siegle
mery betef arowen dosd orutttemmee
Froupt 4: <taa_ oileer cinageniosse the qustice cadig liegoe Nord de Acams queciis sking stegle word plzass. Acalatase:
Fromp 3: <ite, statt (oiber) congeriooeer the quertier eaing strile word pexsae
suttor sqas boxt otter warka, Hoo tie acchor
Chem de quillty
Frame queciios
siegle wond plrsss
that makes more tht fed with oundee..
eery belef araver sedd ptceeacces
plucer rullon ot Lai tox?? otter wirtyy Liee wnden
bosk2
Gize racy beisd azavec, sadd petearaccc
Froaps stae (ntleez coappeecee the quassics "adng stcgle Need os
tida boces
Who is the author of this book?
mery betef arowen west. ofterteeeee
strete
Acaistatt:
Augmented Input Images
Frogpt E: ciia_ (nttar) conageslowee the qussiics sking slcgge Nord de rettol this boot? other worth, the atcher
queccias sking stegle word plzass.
Masisastt
<1ie, (otta conspetoourr gusctiss ptzase. sho la the sothor of this book? ie other worth, the la the accher of
frams speeciio seang siegle son placss. very belet araver ptceeaacn
THA To
aucher this lost) in other
the
Cais 2008s
5ize rasy boiet azavec, sadd peceraaces
Proect Qiteaarart Stherr wnriitee the specatior collng sieete and of Cais books
beint
Toal stants
Froggt 121 dita_asace stber Soaar cosageotoone the quatize caing sitgle wers polaes the achher this In other vertb the exthre cooks
Seawer the qpeectos entne steste word phoess. vesy
GOR
Atalatast:
otterr or parase. tho ia the aucfor of this bosk) in other vtrtt, Who ls the exthee
quercies stegle word
Biw ansy brist assuer dww.aat uitimasee?
the Sive resy bosed azaunc, oadd 24 peceeaaaacc
auchor this boskl in other
ol Lsss
Froect viiaa Stheer sucelattr in welher of this bosks
mery betef dww. srutttemeer
Baseline Output
TTAug Output
TTAdapt Output
Answer: Brushy.
Answer: Brush Dance.
Answer: Brush Dance.
Accuracy: 0.0%
Accuracy: 100.0%
Accuracy: 100.0%
37

---

Published as a conference paper at ICLR 2026
GQA
Original Inputs
Augmented Prompts
Froeet 3n vite,atart iither consesie het foost or the tow? Inserr tse que stcan asag ogge so od plozee. in otter morda, skadta teaser the queectos enting steste word on phoesn, serc ueceerreee
Frougs io ciancatact prttaur conageoees in at the usssoet Auades fromt the stadowe
sxegle
kealitast:
the sparati uniing valay in acamez
ucker modda what
stegle
Rout stants
Prompi sie iecution shont (otter cimcanioe of in for at the rouet Miture the quastion
nole vnca oz pir axe, in astac sodds, statts
feaser spany
steate word or plooee snsrunaae remerree
Froups <iaa titeer
dt ast the us rded) Asshac
Cronc of the sindowt
the qu eation bag sin gie seed Lacase, tn asses sarda, weat in Ieaner quertice Aeasstasts
stegle word plosse, wet__o..eceeeeree
Froigt bn <lin, sla (rtteer consueoee has an too at the asssedy Aesess the quea
avng or boaze, in otter words, shaels
Reaens spatociio say siegle snnd on plosee desngat insarend
Froups <ide, sture (n Peapee/oea in the vi stov? frond of the sindow:
What's in front of the window?
plesss, wetdeeedddddddeeeeeeee
boat alati
Augmented Input Images
the Nartta dn ust single us cd scoee, in cttae seeda, itacla Teaeer Matietatt
steate word plnose dnei tottrrreee
Frogpt 3: ciia_ (ottaer conageott as in deaas ef the si scovy to ause ale whrdd ase,
the
worda,
Naae quectios sking stegle word as pleass, owsiued
Froept <its start Hiberr stoepeitt In feast of the window?
frrst the
aeatalasks
passt the que atson ust allg be voed or tease, in cttee sorda, ttaela in acamer AaxlssetE:
stegle please, oedt oecacacces
Froapt 12: <i saaaee sileer cinapeot has In frest of the us rdoe) Ase frons the sindow? Seseer saae steate wond ploose ennuunaae conereed
parofee morty
aod tease.
stacta
from: the
stegle
sttaeess conependias
192r the leen eten ceceo teste word on eho ase. In other werth Whats Inawer ital
strgle
Froept Lt: K1D, stact Hiborr whweeeeeeteeeeeeeeee is Sn. from of the sindewt
asegle plocoe desigetg
Analatatt:
Baseline Output Answer: Blinds.
TTAug Output
TTAdapt Output
Answer: Desk.
Answer: Desk.
Accuracy: 0.0%
Accuracy: 100.0%
Accuracy: 100.0%
38

---

Published as a conference paper at ICLR 2026
TeX
Original Inputs
Augmented Prompts
Promet Te start jither comsenieeeer the not loxt mane shout tee ndlways meter
with No.
-abas saked shac zite eaply wath the time assa the irage.
<he pat quecactie mitks acteed the exsuer.
<bo reply comer senos knowleteee srawer cas foad the Image. Dussciias atuck san ooss aukgoa22 uue acies ng asague surd os aeaser the querties exing stegle word ct plesas, sert.nd ucceearces
Frouut
lor ceassatest Proept aod
bo uther whacts Laeee teans cesuuily adlepeed
Acamez quarties cxing stegle word please, orceeaccee. Roat slant
Fromct 21 Toan co conaaee Priept 5 aorl
Doartic:: base carsly apeed by sutdlkk2 quastion using sing eer
of books
pleass, wet.eeeeee.sdeeeesseeeee
NICHAAN
Dusstica: od thess vas reoss asapeed owtell same creaci as zs ing voed
MORGAN
NICHARO
IN GORGAT
M MORGHI
FICKART
HENLEW
Teaser
AcalucasEs
ALTERED
B BRORED ANGELS
WOKEN FURIES
Frougt 41 Tere ocascated, sace as Froeg S a..)
Dieecizen: sorda, was 2p1 the meeetos est strste books
rcapted recfite?
CARBON
Relees quecciis cring ssegle wont plessn ensiga aecoeaae
AFFTIEY
Town
Anne Cr
HOTTE MICHATREY
For Vatillro
wurd pa to wurtly Garee tooko was cesuuity atypted by Teumm
Froept Texe teascated, sane Froept a.dl
siegle aon plasse dreiga inrareed
PERT
Analstase:
PERN
Texe cesscared, sats as Proept E serl and
Froapt E:
pacdlike
Ansiutase:
which of these books recently adapted by net-
Fromet Tive crperated, Proeet E sid
flix?
Dosstiias Ldh. books cecartly by need Like And was using sard
sacille?
Augmented Input Images
mpeeccee tro strste phoose, deviiatateeeeree
Dusicizai vii ot the books vis adupted by asvec the asag hrase,
these books was repestly odected by
Asalutase:
Frongt Tooe cxaecated, sase as Proept 5 aed
cowenice: dke was repertly by unsee at agin or plca acaver the quartice cadngg steple word dn pleass, oedd_od ueseearces
Froaps Iocs searcated, aata Foorge aod
plzsas. 2n othee veeda. shics of these beoks vax cecuccly adapped by foot 20
stegle please, veceacccee
Froept Lis Jo.. seaccated, aaca Incaps 8 and Drend otber sakca thais
siegle
Acaiutats:
Prospt seaceated,
Duessioe: v.t. hese beoks cxceraly adapeed n2 Ansver que attor coino cocoetay
re:file?
knawer quertics exing strgle word cr pleass, wedd uecseerree
Donstics: Ire.
speed netfllal never caing veed
rean, stter stise these ane coosaily edugies by sarl
Anxlutate:
Froept I... teancated, Foorss
Disu wbice tanns selil fesmer woul stople sactilee
cecensly
stegle plosse
Asaletast:
Drey shewe horts receritl odggeed by net mi strate phoose, sew.
Baseline Output
TTAug Output
TTAdapt Output
Answer: broken angels
Answer: altered earbon
Answer: altered carbon
Accuracy: 0.0%
Accuracy: 100.0%
Accuracy: 100.0%
39

---

Published as a conference paper at ICLR 2026
A12D
Original Inputs
Augmented Prompts
Caroas
Froep (elier) conaperions ttons that world the casket poputa tis aetrooodt de otoer wassky Gesstioes Wai woxed hupgess A. lleerta sould deccease
z0om vould incenaze
AasletntE: Acame:
sid happez if chst plae tre aneedr oater worth Fore crsseated naer Proeet
<it (otlee conapendon
other sececated,
Froest 3: star (otber comseiieestiion that and seper the ort
opacat los deccnased) other wadda, Ior sestcated, sara Exarge
cisa
popclation deceessed? other words, traccated, ood
Frongt
Frompt 3: iiee stare (ntteer coapeeooeentee thos was 15 happes LE the oe
What would happen if the cricket population de- creased? choices are listed below:
chet aaluiala uecevseest atter soaultell Praps
(A) lizards would decrease
Froeci strection piberre conieeig Bhs weate tups
(B) eagle would increase
(C) king brown snake would increast
the
(D) salt bush would decrease
ker pore denese wen saker worth, Fore cnsscateds saer Proeet
Augmented Input Images
Froups stses intlae doapeecooss ccll
spoke happas the
Froest <l start itherrs renerriee stoet 1d
crloess popuiation secomased) ether vocoe, Ioes sesecated, trarge
stapeiient tenn Mhe cld citfee populaytes ressed? other
Froeps and
Froegt 12s qiaagacast Stharr supeiuuraa cies Mai sa snd hugues is the ericoss pop cractos decres and! other veoday for troncaced, 1956
sideel bo diserssedt otter wartn Iose insredted, 0.oc
Prong
Frogps 14: 4il soact silaes sileers ooaeeooiitttse-eeooio in has cell happ the
Fromct
crices popoiacion decomanddospcion
nat's would Ixcetave
king sc0m ssace sould lacennat
Kcams vitt the lattec tt st_tteeaaoss iestitants trawr:
Baseline Output
TTAug Output
TTAdapt Output
Answer: C
Answer: A Accuracy: 100.0%
Answer: A Accuracy: 100.0%
Accuracy: 0.0%
40

---

Published as a conference paper at ICLR 2026
MME-RealWorlke
Original Inputs
Augmented Prompts
caliee stau la palieci suldict The loles Thas Racannage snnripee
monugle siteeee chalee qeveliss seeed on the unepr The woenery uid maltyy This leige thewe
loush pais muttial salay sortl (or
the
Freer: riteunant poower the Hope tne stee car the mene the scare with vay
millige
Saceed siatisn aue botiol incer ITI
caee Dhis shaye slars the
san, wikee boo
Eommy Cita sust clnsye dist tie lousse at Maad pour prliitaae
enterr meticpla deaiee tuanttio the leager was aneerr fesprnal
acciss the iiree and toot
This image shows the front view of the ego car. What is the future state of the black pants pedestrian in the middle? The choices are listed below:
Froes cosppand teat sneme lai The codtes are ciiced tatier Tuus the laaey lee sagee Resaes with only the taskew She
the
(A) Turn left.
mous view
(B) Stationary.
on aeralien on teess dess the aesice sits
(C) Keep going straight.
(D) Turn right.
This bett. at muster that th butur stace pesstcter
(E) The image does not feature the object.
koagee sas bnduae the up mades vnyy
Augmented Input Images
lerm Joic Inage shous
unaye dus Paponca calias ih, by hinde
pllay the ryllone
in
pelectiian citee mnet cheee extlert (at vers laft ned bonarr san Zase aloae Uie creik
scles wetes
indess teepertti foom view als Laver
leers dee addra accasiele 10) utale Bitecd
with enly the lemee ueica
the proes the stactar moltes coe caft In seep esine
muccaate cacice beisd the seaee on other soidl, thaestie site co Trmer
tie klack akdleed The dloscas iisted di Bane Bust bay
Latter is haree lik leage. meem aneeer wnter metty
inlees the baes
the Issare moaee
OJ elina sapppe the se anewer maliiple dhoie qasster base fecmiie indern
the
use
scart parse polieriiaa in mittle the atstiess are Liees belwe
molludtnn aiee cise lunn tie lenve Aouu
Baseline Output
TTAug Output
TTAdapt Output
Answer: E
Answer: B
Answer: B
Accuracy: 0.0%
Accuracy: 100.0%
Accuracy: 100.0%
41

---

Published as a conference paper at ICLR 2026
AMBER
Original Inputs
Augmented Prompts
other vocdt, the Reas stasts
Froept
I: rit, state intpect conapeno oes the plaeos ate nd co ca Inage? te www.eeigaaaliiioo
Ataletase:
other voeda, Doss the págeos utard in sosa snagervass oew._oe accooeeae
Proect siuu (etsere chneerstue eiter on Toal stants
tmage?
(otlae conapeoboos other woreh, boos the plases stard in cats Inogehornt infurreteaaeee
Froupt <iaa inttaey cinagend oss the page stard iaago? stats
Fromt (obber somapeite plawos ucoes vacak, Doss the pagess stare sass sugeroaaillllttaaaaaa
Image?
Does the pigeon stand in this image?
other the
Teas stanks
Augmented Input Images
Frompi siinarlas (thtrr Conperitters Use alanal kis
other vocda, the plgees stard t358 inageloest saage/oni__ff.cccrraee pcoerazze
sflece earehy the pianss stand in psis Cntt Aaalutast:
Proect vitaaarart other seeeh oes the frams this leapet Boat atanes
In ocaee socas, doss the pigace atand in tase of of soceeareo
Froupt
ottaare claapebs as the pia was stand lmage?
Maaistass
Fromp klls sttwers rtoeceioo
stand Image?
ucass vonil does the pigass stard than tnoperceenaat stteaaree
Froups
ciseeeeeeeee the at image?
Boot sbanks
Fromci Lie si suga Stherr seppoittrr Lie pluats atanal in Jais Snap42rerd_ed ucsecanze:
Baseline Output
TTAug Output
TTAdapt Output
Answer: No, the pigeon is perched top of the coin return machine.
Answer: Yes
Answer: Yes
Accuracy: 100.0%
Accuracy: 100.0%
Accuracy: 0.0%
42

---

Published as a conference paper at ICLR 2026
COCO Captions
Original Inputs
Augmented Prompts
Pecepe: 3: c Dugltaas boeen icinagoFienn desceabe S0 La cas de lenal, 31 recely paeciae the tesertecion, Lactede pre fin Tie mest ceplecs other teplecat, sertlet onttraaree
deseribe postia "OLL
Fiseetlly
description,
Fezepe Duutart thageeeeeee as describe chts oe or remp, Streetly 5 theie the sane as Peonge o v.xl
Froepe duutart simeerolneeeiite destcibe the perenal, Streesly pestide the das Le sepplim, cossase pontin dhe sage depusto atter misn For sesacated, wase
Bieeslly trspelon, inclate pondix ba it dapucts stear ward,, seascated, in Fromet Firr
Tromp ingaini curititttee potee reteppot trose cetter sis iaar grvent, to not pres lnk the ierertatio,
prstis
tway a qpteri other worth Fore
revertee lor, de centere ofis the a to taay repters "n deher worthy traeectedy ease recope s.el
Trowe iuuacemt iieeer eeterrsttaat desce the csts tne or is sree net, 3ieee chy preatie the proces
Please describe this image in gen- eral. Directly provide the descrip- tion, do not include prefix like "This image depicts".
deecrigsios,
lmage dep
vieae, Ior.
Pronet
deccriecion, do nos Laslide pler to 15ke 1. tis mage copl In other veediy Ii.. tecnosced,
Pecape sagataass boeee estnagare sease desceate caLe sage La gue etat, diseetas pazzuae the sake Pronge t aorl
Augmented Input Images
Froepe Eauatar -iweer--nngeeeeeeeeee des coibe his (naps geracal. Oice peers the the sxnduske in wile Like This and or septorn To steen worth Iere Lesesoiint
sccobe th imsgs pare zal, ceesly peo vide Fczape <ida sreggr casa sepieta descripsion,
tegae on (otuerrcienn an cven decen the tover rend "ireet plee ie the descripsion, nos Lack ode pr wdix he 1854 las qe dep less ocher vooda, (or toonoaced,
cisaLasace ease iecage ...l
the hite depietio 1. other wertty seerested,
Pecape 13) ciaL,acazziooee oo aegnnggiiennn desccabe e bis in age is geceral, ais ect.y prees de the image
voeoe, Iore
secori stine, de nos clude peecia aike ct in las iepista t. other voeda, Iorn tescoatid, zame Pronge
Pecage <lancataa Eew.u agatt cease desceate case cauge in gereeat, disestly prozide the
Baseline Output
TTAug Output
TTAdapt Output
Answer: The image features a white bird with a long, curved beak and type
Answer: A white bird with a long curved beak standing on green bench. The bird is looking to the left.
Answer: A white bird with long beak black legs standing on a green bench.
black legs standing on green bench. bird appears of ibis, characterized by its long, slender neck and beak. The bench is made of wood and has slatted design. with the bird perched on the top stat. The buckground includes paved area with few scatlered leaves and some greenery, suggesting an ouldoar setting, possibly a park or zoo. The bird facing to the right, and its body oriented towards the left side of the image.
Accuracy (%):
Accuracy (%),
ROUGE-L:
ROUGE-L: 45.02.
BLEU-I: 62.50.
BLEU-1: 52.38.
BLEU-2: 40.82,
BLEU-2: 36,
BLEU-3: 32.93,
Accuracy (%) ROUGE-L: 15.59, BLEU-I: 12.24, BLEU-2: 7.94, BLEU-3: 5.08, BLEU-4: 3.43, CIDER 0.00
BLEUU::: 27.45,
BI.EU]44 27.23,
BLEU-4: 21.89.
CIDEr: 60.98
CIDEr: 12.62
43


Reviews:
[{"review_id": "TqN1ylMdI8", "reviewer": "Reviewer_HfFB", "summary": "The article studies how to perform test-time scaling for small vision-language models. In particular, it investigates different design choices in terms of test-time augmentation (TTAug) and test-time adaptation (TTAdapt), analyzing how performance changes in 9 benchmarks w.r.t. diversity induced methods, aggregation levels, number of augmentations, text/image augmentation methods, and adaptation strategies. Each tested design choice comes with recipes and findings that, when put together, lead to consistent improvement across various VLMs.", "questions": "Related (point-wise) to the weaknesses above:\n\n1. Given the small gaps, how general might the result hold for other architectures?\n2. Are there insights on why TTAdapt and TTAug could show inconsistent behaviours? \n3. Could you elaborate on the choice of benchmarks and how to select the best strategy (given the variance of results across benchmarks)?\n4. How does the final model compare with existing alternatives?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This is an extensive study on how a large variety of design choices impacts the performance of test-time scaling. The study is well conducted, exploring a large variety of alternatives with regard to both test-time augmentation and adaptation. The results can serve as a blueprint for developers of test-time scaling techniques and a reference for practitioners and researchers in this field.\n\n2. The choices have been primarily analyzed for SmolVLM2-2.2B, a small-scale model, but they also partly generalize to larger ones of different families, as shown with the experiments of Fig. 3.\n\n3. The appendix is remarkable, providing extensive details on the implementation as well as theoretical analyses. This level of detail is important not only to ground the design choices (with the theoretical part) but also to allow for easy reproducibility.", "weaknesses": "1. The gaps across alternative strategies are sometimes small and might be hard to draw general conclusions from. For instance, 5 methods in Tab. 4 have only a 0.4 gap in performance, with the caption claiming that classical augmentations (L-M-H)  are the best, while the text-only strategy (0) performs comparably or even better. I am aware that experiments are costly, but reporting error bars or analyses on the significance of the gap could help strengthen the main takeaways.\n\n2. Related to the general nature of the results, after each analysis, the best approach is selected for subsequent experiments/as a reference. Given the small gaps, optimal choices for one architecture might not be transferable to other ones. This behaviour can be seen from Fig. 3, as i) only the SmolVLM2 family shows consistent improvements (e.g., compared to Ovis2 and InternVL2), but only for SmolVLM2 2B, those are large. Moreover, even within that family, test-time augmentation and test-time scaling may show different behaviours (e.g., SmolVLM2 0.3B has TTAug helping more than TTAdapt, while SmolVLM2 0.5B shows the opposite trend). Performing the same analyses but considering more architectures/families would strengthen the applicability of the core takeaways.\n\n3. The choice of benchmarks from VLMEvalKit (Duan et al. 2024) needs some clarification. There are 20 benchmarks in the VLMEvalKit (Duan et al. 2024), but 9 are considered: why is this subset considered the most representative? Note also that the AI2D benchmark on diagram understanding polarizes some of the results: e.g., self-consistency in Tab. 1 outperforms or is comparable to self-selection alternatives everywhere but for AI2D, where there is a huge gap (>60% points), and the same happens in Tab. 2 (1 vs 2 and 4). As the choice of benchmarks may have a high impact on the results and takeaways, the choice of benchmarks may need some clarification, as well as the use of the average to assess the best-performing method.\n\n4. (relatively minor) From the tables, various alternatives are presented, but an overall comparison with the state-of-the-art is unclear (yet performed when testing various alternatives, e.g., Tab. 1 (Chen et al. 2024a; Pruner et al. 2025, Wang et al. 2023b). Given that one of the aims of the article is to outperform existing test-time scaling approaches (e.g., lines 81-85), it would be helpful to report a conclusive summary table where the final strategy is explicitly compared with existing works. \n\n**Minor points:**\n\n5. While the article focuses on small models, to my knowledge, the tested strategies are not tailored to small models in particular. While I understand that test-time scaling is more helpful in the context of lower-performing VLMs, showing how the final findings generalize to large models could give further insights into the main outcomes. I deem this as a minor point, as the analysis is still helpful and several models have already been tested.\n\n6. Section 3.1 provides a clear formalization of TTAug. However, 3.2 describes TTAdapt without any formalization. While excessive notations or unnecessary formulas may hinder the clarity, providing a general definition of TTAdapt with a mathematical formalization would be helpful to avoid potential confusion in the reader and make the manuscript (3.1 and 3.2) more consistent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NLAOh9VIWi", "reviewer": "Reviewer_bAuU", "summary": "This paper proposes two efficient test-time scaling methods for small vision-language models (VLMs): Test-Time Augmentation (TTAug), which aggregates token-level predictions from semantically preserved input augmentations without updating model parameters, and Test-Time Adaptation (TTAdapt), which further refines model parameters during inference using pseudolabels derived from TTAug consensus. Evaluated across nine benchmarks, both methods consistently improve performance while maintaining computational efficiency suitable for resource-constrained environments, outperforming existing answer-level test-time scaling approaches. The study also demonstrates that input perturbations with greedy decoding and token-level aggregation are more effective than temperature sampling and answer-level aggregation, respectively.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Proposes token-level aggregation, a novel and empirically superior alternative to answer-level methods\n- Demonstrates consistent gains across nine diverse benchmarks, including challenging open-ended tasks like captioning and VQA, where prior methods fail due to reliance on extractable final answers.\n- Provides a comprehensive ablation study on augmentation strategies, aggregation layers, and modality-specific contributions, revealing task-dependent optima (e.g., early layer aggregation for visual reasoning).", "weaknesses": "- The paper's claims of efficiency are undermined by significant computational overhead: TTAug increases inference time by 3.3× and GPU memory usage by 1.9× at 16 augmentations, which contradicts the goal of resource-constrained deployment.\n- TTAdapt requires iterative fine-tuning during inference and parameter resets per sample, introducing non-trivial latency and complexity that are not adequately justified by marginal gains on most benchmarks.\n- The experimental evaluation lacks ablation on real-world hardware constraints, such as latency-sensitive edge devices or energy budgets, making the practical viability of the methods questionable.\n- Text augmentation dominates performance gains, yet the simplest classical perturbations (e.g., keyboard typos) often match or outperform more sophisticated self-paraphrasing, casting doubt on the necessity of complex augmentation design.\n- The model-agnostic claims are weakened by hyperparameters (e.g., 16 augmentations, classical high-strength augmentations) being optimized solely on SmolVLM2-2.2B and not validated for transfer across all model families.\n- The generative image augmentation failure is attributed to text exclusion, but the paper ignores alternative approaches like synthetic text insertion or text-aware augmentations that could preserve OCR-relevant content.\n- Qualitative results show that TTAug frequently fixes errors by “stitching” correct tokens from noisy augmented inputs, suggesting the model’s base competence is fragile and the method acts more as a heuristic ensemble than a principled improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HyjplBmiJk", "reviewer": "Reviewer_6M4C", "summary": "This paper addresses the performance limitations of small Vision-Language Models (VLMs) under domain shift by proposing two efficient test-time scaling strategies: Test-Time Augmentation (TTAug) and Test-Time Adaptation (TTAdapt). TTAug generates multiple semantically equivalent inputs through augmentations and aggregates predictions at the token level without parameter updates. TTAdapt extends this by fine-tuning model parameters during inference using pseudolabels derived from TTAug consensus. The methods are evaluated across nine diverse benchmarks, demonstrating consistent improvements while maintaining computational efficiency suitable for resource-constrained environments. The approach generalizes across model architectures and scales without additional tuning.", "questions": "- The results in Figure 4 show that optimal aggregation layers vary by task. Could the authors explain how these layer choices were determined for each benchmark? Is there a principled criterion behind the selection, or was it based on empirical validation?\n- The paper claims the methods are suitable for resource-constrained environments but only provides results for an A100 GPU. Could the authors clarify if any tests were conducted on consumer-grade hardware?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The paper introduces a novel framework for test-time scaling in small VLMs, different from existing methods that rely on external models or answer-level aggregation. The token-level aggregation strategy is novel, addressing limitations of global confidence measures by leveraging fine-grained signals during generation. \n+ The work is technically sound, with extensive experiments covering nine benchmarks and multiple model families. Ablation studies evaluate different design choices. Theoretical analysis in the appendix justifies some key decisions. \n+ The paper is well-structured, with good motivations, method descriptions, and visual demonstrations.\n+ The work addresses the gap in deploying small VLMs under resource constraints. The proposed method can achieve practical improvements without heavy computational costs, making them potentially accessible for real-world applications.", "weaknesses": "- Limited comparison to baselines:​​ While the paper compares against methods like self-consistency and self-selector, it does not include more test-time adaptation techniques for VLMs. The comparison with these approaches would strengthen the claim of superiority.\n- Computational Efficiency Claims:​​ The analysis in Section 4.3 and Appendix E reports memory and runtime overhead but lacks a breakdown of latency across different hardware. More explanation on this concerns are necessary.\n- The benchmarks focus on question-answering and captioning, but those tasks requiring complex reasoning are not tested. The reviewer cannot confirm if this method will bring benefits for these tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r42fEcbEUb", "reviewer": "Reviewer_9zi6", "summary": "This paper proposes two efficient test-time scaling strategies, Test-Time Augmentation and Test-Time Adaptation, for improving the generalization performance of existing methods. Test-Time Augmentation aggregates the outputs of multiple augmented inputs at the token level without parameter updates. Test-Time Adaptation further improves the performance of Test-Time Adaptation by using it’s outputs as pseudolabels and updating model parameters. The effectiveness of the two methods is verified by extensive experiments.", "questions": "* In Table2, if we exclude the dataset AI2D, it seems that answer-level aggregation with Self-Consistency is not worse than token-level aggregation with simple averaging. Besides, for the same dataset, answer-level aggregation with Sample-and-Rank is also better than token-level aggregation with simple averaging. We can observe the same results on the GQA datasets and answer-level aggregation with Self-Synthesizer. Therefore, it seems that token-level aggregation does not consistently outperforms answer-level aggregation. Could the authors explain those experimental results?\n\n* How can the proposed methods implement early-stopping for low-quality generations?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This paper proposes a new approach for aggregating multiple outputs at the token level.\n\n* The experimental results are sufficient.", "weaknesses": "* The explanations on some of experimental results seems insufficient. For instances, in Table 1-5, the value of performance evaluation is zero on some datasets. It is better to explain such results. \n\n* There is a lack of comparison on the computational cost across different methods. Test-Time Adaptation must update model parameters and thus requires more computational resources than Test-Time Adaptation and similar methods. The authors can provide some theoretical analyses or empirical results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)