Zoomer: Adaptive Image Focus Optimization for Black-box MLLM
Authors: Jiaxu Qian
Chendong Wang
Yifan Yang
Chaoyun Zhang
Huiqiang Jiang
Xufang Luo
Yu Kang
Qingwei Lin
Anlan Zhang
Shiqi Jiang
Ting Cao
Tianjun Mao
Suman Banerjee
Guyue Liu
Saravan Rajmohan
Dongmei Zhang
Yuqing Yang
Qi Zhang
Lili Qiu
Venue: Accepted by TMLR ()

Abstract: Multimodal large language models (MLLMs) such as GPT-4o, Gemini Pro, and Claude 3.5 have enabled unified reasoning over text and visual inputs, yet they often hallucinate in real-world scenarios—especially when small objects or fine spatial context are involved. We pinpoint two core causes of this failure: the absence of region-adaptive attention and inflexible token budgets that force uniform downsampling, leading to critical information loss. To overcome these limitations, we introduce Zoomer a visual prompting framework that delivers token-efficient, detail-preserving image representations for black-box MLLMs. Zoomer integrates (1) a prompt-aware emphasis module to highlight semantically relevant regions, (2) a spatial-preserving orchestration schema to maintain object relationships, and (3) a budget-aware strategy to optimally allocate tokens between global context and local details. Extensive experiments on nine benchmarks and three commercial MLLMs demonstrate that Zoomer boosts accuracy by up to 27% while cutting image token usage by up to 67\%. Our approach establishes a principled methodology for robust, resource-aware multimodal understanding in settings where model internals are inaccessible.

Under review as submission to TMLR
Zoomer: Adaptive Visual Token Allocation for Black-box MLLM
Anonymous authors Paper under double-blind review
Abstract
Multimodal large language models (MLLMs) such as GPT-4o, Gemini Pro, and Clande 3.5 have enabled unified reasoning over text and visual inputs, yet they often hallucinate in real-world scenarios especially when small objects or fine spatial context are involved. We pinpoint two core causes of this failure: the absence of region-adaptive attention and inflexible token budgets that force uniform downsampling, leading to critical information loss. To overcome these limitations, we introduce Zoomer, visual prompting framework that delivers token-efficient, detail-preserving image representations for black-box MLLMs. Zoomer integrates (1) a prompt-aware emphasis module to highlight semantically relevant regions, (2) a spatial-preserving orchestration schema to maintain object relationships, and (3) a budget-aware strategy to adaptively allocate tokens between global context and local details. Exlensive experiments on nine benchmarks and three commercial MLLMs demon- strate that Zoomer boosts accuracy by up 27% while cutting image token usage by up to 67%. Our approach establishes principled methodology for robust, resourco-aware multi- modal understanding in settings where model internals are inaccessible.
1 Introduction
Vision-language understanding has emerged as a central challenge in multimodal artificial intelligence, with broad applications ranging from robotics and autonomous driving scientific diagram analysis and human- computer interaction. Recent advances in multimodal large language models (MLLMs). such as GPT-4o, Gemini Pro, and Claude 3.5, have significantly pushed the boundaries of this field by enabling unified reasoning over lext and visual inputs Li el al. (2024): Gu et al. (2024). These models have demonstrated impressive performance or even PhD-level capabilitics in controlled environments. However, despite those promising rosults, our large-scale empirical study uncovers a critical limitation that remains underexplored: MLLMs exhibit a pronounced hallucination tendency real-world visual reasoning tasks, largely due to their inherent difficulty in perceiving small objects and limited field-of-view awareness. This "small object blindness" can lead to severe and often undetected reasoning failures, raising concerns about the deployment of MLLMs in real- world visual reasoning scenarios.
a range of benchmarks and are increasingly perceived as possessing near-human
To investigate the root causes of hallucination in MLLMs, we conduct a systematic analysis and uncover two fundamental limitations in the visual processing pipeline of current black-box MLLMs. First, these models lack region-adaptive attention mechanisms, resulting uniform spatial processing that disregards task- relevant visual saliency. Despite their architectural complexity, black-box MLLMs fail to emulate the human visual system's 's ability focus selectively while preserving contextual structure. As illustrated in Figure [ this limitation leads to persistent failure even in elementary tasks such as object counting. Notably, the issue is consistent across different model families and prompting strategies, indicating fundamental limitation rooted in the design of their visual tokenization and attention mechanisms, rather than implementation- specific flaws.
Sccond, we identify a critical tension between token budget constraints and visual fidelity. Black-box MLLMs are constrained by fixed token limits for both textual and visual inputs an engineering choice originally motivated by inference efficiency and fairness considerations (Chen et al. 2024h. However, our analysis reveals

---

Under review as submission to TMLR
MLLM Answer:
Question: How many cactus image?
Raw Prompt
There are two small cactivile
visible in the imagr-
Original Image Prompt:
MLLM Answer:
Original Image Prompt= Text Prompt:
Q
From the image. there small cacti placed the lefi side of shelves.
XAPXO
first locate cactus, And then enswer the Question
Image Read
MLLM Answer:
Processed Image Prompt: Processed Image Prompt+Ted Text Prompt:
The fell shows small white pots, whille the eight shows single enerns in white pot Therefore, total number afcatii in images
Show you zoomed-in images of cactis And then answer the Question
Figure 1: Illustration of a black-box MLLM's approach to counting cacti in an image. The model identifies two small cacti on the left side and overlooks the single cactus on the right side of the image, arriving at total of three cacti. The processed prompt highlights specífic regions of interest facilitate the correct object count.
Text Prompt: What is title book?
Processed Image Prompt:
MLLM
Original Image Prompt:
Answer:
Cropping
Information Loss
:Websersl
T
146x248
MLLM
Answer:
Uniformed Scaling
Image Resolution: 4240x2832
:Sorry, cannot read the textll.rly,
1536x1 0226
Figure Illstrationn of information loss during image processing in black-box MLLMs. The original high- resolution image (4240x2832) is downscaled to meet. token limits (1536x1026) leading the loss of critical details. Cropping to focus region of interest (146x246) allows the model to correctly identify the book title as "Webster's II'.
that token budgeting in vision tasks is core computational bottleneck that directly impacts perception granularity. As shown in Figure 2 the commonly adopted strategy of uniform image downsampling leads to substantial loss of local detail, especially for small or spatially distributed objects. Crucially, we demonstrate that neither prompt engineering nor improved resizing algorithms can compensate for this representational loss.
These observations molivate us to formally define new problem in visual prompting for MLLMs: How can region-aware, spatially coherent visual inputs be constructed under strict token constraints in black-box inference settings? We argue that solving this problem requires a paradigm shift from heuristic prompt design to principled visual token orchestration. We decompose this problem into three interdependent sub-tasks: (1) Region Selection identifying and prioritizing task-relevant regions without model internals; (2) Spatial Preservation retaining spatial relations and object continuity across selected regions; and (3) Aware Prompting balancing the allocation of tokens between global context and fino-grained details.
To address these challenges, present Zoomer, visual prompting framework designed for token-efficient, region-aware visual representation. Zoomer comprises three integrated components:
A prompt-aware visual emphasis module, which selectively highlights salient regions by ma- nipulating image input structure, enabling focused attention without altering the model; A spatial-preserving orchestration scheme, which ensures that contextual coherence and rel- ative object positioning are preserved across split patched inputs;
2

---

demonstrating that Under review as submission to TMLR
A budget-aware region selection strategy, which dynamically allocates token capacity based on content density and task relevance.
We evaluate Zoomer on a suite of diverse benchmarks, including Vstar & Xie, 2023), CVBench Tong et al, 2024a), and RealWorldQA [xAI 2024), which collectively capture controlled, open-domain, and in- the-wild reasoning scenarios. Zoomer consistently outperforms competitive baselines in both accuracy and efficiency. On the Vstar dataset, the Zoomer-Patches variant achieves 26.9% absolute accuracy gain, while on RealWorldQA, Zoomer-Adaptive exceeds baseline performance by 12. %. Moreover, in the TerraIncognia. setting, our method reduces token usage by 67% while improving accuracy by 6.4%. These gains hold across multiple API providers, including GPT-46 Gemini-1.5Pr@ and Claude-3.5-Sonme Zoomer generalizes well across model archilectures and tokenization schemes.
In summary, this work makes the following contributions: (1) We provide the first in-depth empirical analysis of how token allocation and spatial attention impact visual reasoning performance in black-box MLLMs. (2) We formulate the contt-constrained, region-aware visual prompting problem and identify its core design dimensions. (3) We propose Zoomer, a general-purpose framework for token-efficient, spatially coherent visual prompting. (4) We demonstrate that Zoomer improves both accuracy and efficiency across models, datasets, and domains, highlighting a new path toward robust, aesorrce-wwar multinodal understanding.
2 Pilot Experiments
A fundamental challenge in black-box multimodal large language models (MLLMs) is their inability to process high-resolution visual imputs efficiently under strict token budget constraints. In platforms such as GPT-4o, visual inputs are internally segmented into fixed-size image patches (typically 512x512 pixels), each corresponding a predefined token cost (e.g., 170 tokens per patch [ This patch- tokenization strategy, designed for balancing inference efficiency and fairness, introduces critical trade-off: higher image resolution yields more detailed information but leads to rapid token consumption, limiting the number of patches model can process. Conversely, reducing token usage through downsampling or cropping risks omitting essential fine-grained visual cues.
To quantify the impact of this trade-off, we conducted a series of pilot experiments using GPT-40-0513 on the Vstar-Bench dataset, which requires precise visual grounding of small or occluded objects in complex scenes. We compare three visual prompting strategies: (I) Unaltered Input, where the original image is submitted directly without preprocessing: (2) Image Crop, Unless otherwise stated, all regions of interest (ROIs) used in this paper are obtained automatically at inference time using the multi-scale, prompt-aware open-vocabulary detector pipclinc described in Scction ??. We do not use any ground-truth bounding boxes from the datasets when constructing ROIs, including the introductory examples in this section; and (3) Zoomed Crop, which magnifies the cropped region to maximize visual feature visibility while preserving the 512x512 input constraint. Each cropped ROI normalized to a consistent visual scale before composition. Concretely, we resize each ROI such that its longer side equals 512 pixels while preserving aspect ratio. ROIs that are originally smaller than this scale are not up-scaled beyond their native resolution, in order to avoid excessive blurring or hallucinations, For large input images, we first crop ROIs using the detector and then apply this max-side-512 rule per ROI. This normalization ensures a consistent scale before spatial composition in Section 4.2
The results in Table D reveal several critical insights. While Unaltered Input provides full-scene context, it incurs excessive token cost (955 tokens), with limited performance Saans-suggestingg that high token usage alone does not guarantee better model understanding, In contrast, the Image Crop method reduces token usage substantially but fails to improve accuracy, likely due to the lack of visual emphasis and limited discriminative detail in the cropped region.
2 htttts:/pplatoormoooeaaai htttp:://gemini.gooll.coom
Shttps//aamthropic.cmm/
4 https///oppeaacco/p//////rreeiie
3

---

Under review as submission to TMLR
Prompt Tokens
Method
Accuracy
Unaltered Input
57%
955
Image Crop
58%
270
Zoomed Crop
64%
270
Table 1: Performance of different methods on Image from Vstar.
The Zoomed Crop strategy significantly outperforms both baselines, achieving 64% accuracy with only 270 tokens. This improvement highlights the importance of visual emphasis: by magnifying relevant re- gions, Zoomed Crop restores critical visual gramularity lost in standard downsampling, while maintaining input compactness. Importantly, this approach respects the model's fixed patch budget without requiring architectural changes or access internal weights.
These findings underscore fundamental limitation in black-box MLLMs: existing visual prompting tech- niques fail to manage the resolution-tokee trade-off effectively. Naive approaches such as downscaling region cropping without adaptive enhancement do not sufficiently preserve semantic structural informa- tion. Our pilot experiments suggest that vision enhancement strategies particularly those that incorporate selective magnification are essential for preserving task-relevant detail in constrained inference settings.
In subsequent sections, we build upon this insight to design Zoomer, visual prompting framework that generalizes the principles of adaplive emphasis, spatial preservation, and budget-aware token allocation across diverse visual reasoning tasks.
3
Related Work
3.1
Multimodal LLMs: Open-Source and Black-Box Models
The integration of visual and textual modalities in large language models (LLMs) has led to significant advancements in multimodal models (MLLMs) like GPT-4o, Gemini Pro and Claude3-Sonnet These models rely on effective visual encoding strategies to bridge the gap between language and vision. Approaches such as CLIP (Yang et al.) align visual and language embeddings through contrastive learning, while models like Flamingo (Alayrac al.p and BLIP-2 (Dai et al.) usee attentioon mechanisms or pretraining modules link vision encoders with LLMs. However, these methods often rely on fixed low-resolution inputs (e.g.. 224x224), limiting their ability to process high-resolution images non-standard aspect ratios (Lin et al. which hampers performance on fine-grained tasks such as OCR and small object detection
In contrast, open-source multimodal models (Li et al. Xu et al. Zhang et al. a Li et al. Zhao et al.) allow for architectural modifications and fine-tuning to accommodate any-rcsolution inputs. However, black- box MLLMs such as GPT-4o and Gemini Pro, which impose strict token limits for computational efficiency, require alternative solutions. The need downsample or crop images to meet these constraints often results in the loss of crucial visual details, particularly in tasks requiring detailed visual understanding. While position embedding interpolation
[Bai al. Wang et al. Luo et al. Hong et al.) Chen et al.) and patch- based cropping (Xu et al. Li et. al.] widely adpoted in open-soure models offer promising directions for any aspect ratio and res-rusolution image processing, they are not applicable to black-box models, where architectural changes and extra soariingfinottnnnng are not permitted.
3.2 Object Detection
Traditional object detection models, such as Faster R-CNN (Ren et al. and YOLO (Remoon et al.), effectively identify and localize objects within predefined categories. However, they st struggle with open-set scenarios, where novel objects not seen during training need to be detected.
Recent advances address this limitation through open-set detection models that leverage natural language processing. For instance, OV-DETR (Zan et. al.) integrates CLIP with object detection to generate
4

---

Under review as submission to TMLR
Text Prompt
Image Prompt
Image Prompt
Text Prompt:
ROI
From the information on that alvertising hourd, what type this shop?
scall Emphasizer
process
Off-the-sher object detection model
Text of Interest advertising: board: shop
Zoomer
Budget
Emphasized ROIs
Black-box LLMs
Spatial Preserving Orchestration Schema
Gemini
Tookn E Elliciims Prompt
GPT4:: (coss 955 tokens)
Zoomer-GPT4o:
Budget aware Image Prompt Construction
Token Budget
(cost 277 tokena)
The aehetiocnentt lrcend in the imgge not enongh tn read.
but
Based the advetiieemnt board shop a yoga CCeePower Yoga".
512x512 512x512
Zoomer
Text & Processed Image Prompt
Figure 3: The Zoomer framework. Left: Raw Input image (Q) and text prompt are processed by Zoomer and then fed into a black-box LLM (e.g,, GPT-40) for analysis, resulting in more accurale and detailed responses compared standard input methods with even token saving, Right: Zoomer processes the text extract key terms and uses multi-scale emphasizer(( ($4.1) with an off-tho-shelf object detection model to identify regions of interest (ROIs). The idintified ROIs 2 are then processed through spatial preserving orchestration schema ($4.2) for filtered emphasized patch (@) and a budget-aware image prompt construction module ($4.3) to create a token-efficient prompt within the specified budget. A scaled global view (3) is also generated for potential prompting.
category-specifie bounding boxes from textual prompts, enabling detection in open-world settings. Simi- larly, GLIP (Li et al. b) reframes detection as grounding problem, improving alignment between visual regions and Lextual descriptions. DetCLIP (Yao et al.) exlends this further using pseudo labels from large- scale captioning datasets, enhancing generalization. Grounding DINO (Liu et al. b), built on the DETR framework (Carion et al.),, also advances open-set detection through natural language integration.
In addition, SAM (Kirillov al. 2023) and SAM-2 (Ravi et al. 2024 offer zero-prompt minimal-prompt segmentation for arbitrary objects but lack robust text-prompt handling. EVF-SAM (Zhang et al. overcomes this by extending SAM's capabilities to better manage complex text-based object segmentation.
By incorporating these models, Zoomer enhances its ability to dynamically detect and emphasize regions of interest (RoIs), cnabling black-box MLLMs to focus on the most relovant visual content without losing critical details, which is essential for maintaining high performance across varied resolutions.
4 Method Overview
Building upon the findings of our pilot experiments, we introduce Zoomer, unified visual prompting frame- work that enables detail- token-efficient encoding of high-resolution visual inputs for black-bor box multimodal LLMs. Existing models such as GPT-4o and Gemini 1.5 often rely on uniform downsampling or tiling to fit image inputs into fixed-size patches, which leads to significant information loss and degraded performance on fine grained vision tasks.
Zoomer addresses this challenge through three tightly coupled modules, as shown in Figure 3
Prompt-Aware Visual Emphasizer: extracts task-relevant image regions based semantic cues
derived from natural language prompts.
Spatial-Preserving Orchestration Schema: reconstructs extracted regions into spatially faithful
layouts that preserve object relationships and scene structure.
Budget-Aware Prompting Strategy: manages token allocation under user-specified constraints by adapting the number and organization of image regions.
5

---

Under review as submission to TMLR
4.1 Prompt-aware Visual Emphasizer
The prompt-aware visual emphasizer utilizes multi-scale emphasizing strategy to prioritize image slices that are most relevant to the input prompts. By analyzing the semantic content of the prompts, this component dynamically selects and enhances specific regions of the image at varying resolutions. This approach not only enriches the contextual information available to the model but also mitigates the adverse effects of losing critical details during the resizing process.
Prompt Tokenization Prompt tokenization critical first step in which input prompts are parsed into meaningful tokens. This process segments the prompt into components that can be easily analyzed for semantic relevance. Specifically, the prompt is divided into structural components, and our focus is on processing the relevant sections that contribute directly to visual emphasis.
To enhance the extraction of semantically relevant tokens, we apply advanced natural language processing (NIPP) techniques. First, we use the NLTK library to remove stopwords, reducing noise and ensuring that the model's attention remains on the most critical visual elements. By eliminating these non-essential words, we concentrate key terms that directly influence the visual emphasis.
In addition to basic stopword removal, we utilize dependency parsing (Sarthi et al. 2024 De Marneffe
& Manning, 2008) to analyze the syntactic structure of the prompt. We use lightweight dependency parser to identify noun phrases and their syntactic roles (e.g.. question objects versus modifiers). Rather than constructing a full scene graph, we use these dependencies as an importance-rankinn higher-priority noun phrascs are passed to the apen-vocabulary detector first, and (2) when multiple entities compete for limited detection budget, the ranking is used to break ties. In other words, dependency parsing is used for importance weighting and ordering prior to detection, not for explicit relation modeling. This deeper analysis identifies core entities and relationships, such as subject-object pairs and action verbs, which are crucial for interpreting the user's 's intent. By focusing on these core semantic elements, we ensure that the visual emphasis aligns precisely with the underlying meaning of the prompt.
mechanism: (1)
Finally, we strip away any irrelevant formatting or non-content-relateed details, allowing the visual emphatizer to focus solely on the essential information. This multi layer- tokenization approach ensures an optimal match between the tokenized prompt and the image features selected for emphasis.
Multi-Scale Emphasizing Algorithm: Given a key object term extracted from the text prompt, the Multi-Scale Emphasizing Algorithm T utilizes state-of-the-art object detection model to localize the corre- sponding object in the image prompt. In our experiments, we primarily employ GroundingDINO (Liu et al. b) as our localization model.
The encoder such models typically downsamples the input image to a resolution of 224 x 224 or 336 x 336, potentially resulting in information loss when localizing the target object at a coarse granularity. To address this limitation, we propose a Multi-Scale Emphasizing Algorithm that processes the original image at multiple resolutions. The algorithm divides the input image into patches at various granularities, e.g., 2x 3 3x 3, and beyond. For cach generated patch, we apply the object detection model to localize the target object. The algorithm retains bounding boxes returned by the model that exceed a predefined confidence threshold. These high- conficonfieeece bounding boxes collectively form the output of our algorithm, providing a comprehensive multi-scale representation of the target object's location.
4.2
Spatial-preserving Orchestration Schema
Building upon the Multi-Scale Emphasizing Algorithm, we introduce Spatial-preserving Orchestration Schema to maintain the structural integrity of the image during the encoding process. This schema filters the bounding boxes obtained from the Multi-Scale Emphasizing Algorithm and ensures that the relative positions of the selected image slices are preserved, facilitating a more faithful representation of the original image layout and enabling coherent reconstruetion when processed by the multimodal LLM. To refine the selection of bounding boxes, implement Non-Maximum Suppression (NMS) based slice filtering method. NMS is employed to climinate redundant and overlapping slices, retaining only the most salient features that align with the prompt. The process works as described in Algorithm
"https//wwww.llk..rrgg
6

---

Under review as submission to TMLR
Algorithm Multi-Scale Emphasizing Algorithm
Require: imput image, k: key object term, M: object detection model, T:: confidence threshold
Ensure: B: set of bounding boxcs
1: B+ 0
5 <
Smax]
b. Set of scaling factors
3: for each 8 S do
4: Ps + --DiiiddintoPPacche((..ss
for each patch p E Ps do
b, k))
Get bounding box and confidence
if T then B B end if
10: end for
11: end for
12: return B
By setting appropriate threshold T' for the Intersection of Union (IoU) of bounding boxes around the selected regions, we ensure that only the highest-quality slices retained for the encoding process. This filtering step enhances computational efficiency by reducing the number of slices to be processed and improves the clarity and relevance of the visual information provided to subsequent stages of the model.
The resulting set of filtered slices are then orchestrated to preserve their original relative positions within the image. This orchestration process involves the following steps: Slice Extraction: For each bounding box b, in the filtered set F, we extract the corresponding image slice from the original image. Blank Image Creation: We create new blank image with the same dimensions as the original image. Slice Placement: We place each extracted slice onto the blank image at its original position, leaving the rest of the image blank. Image Shrinking: The resulting image, containing only the selected slices in their original positions with the rest left blank, then shrunk to a predetermined size while maintaining its aspect ratio.
In all configurations, Zoomer composes the selected ROTs and optional global view into single image canvas, which is then passed the black-box MLLM. In particular, in Zoomer-Adaptive and Zoomer-Glood ROIs are placed according to their original relative coordinates using the spatial-preserving layout, and, when additional strip the same canvas. Thus, the
enabled, down-scaled global overview is appended MLLM always receives exactly one reconstrueted image rather than multiple separate images.
Algorithm 2 NMS-based Slice Filtering
Require: B: set of bounding boxes, T: IoU threshold
Ensure: F: set of filtered bounding boxes
1: F +--0 0
Sort B in descending order of confidence scores
3: while B + do
4:
bmax + arg max,e score(b)
+ Fubmax
B\ B+B'
6max
for each b B do
if IbU b) 2 T then
B t- B B
10: end if
11: end for
12: end while
13: return F

---

Under review as submission to TMLR
4.3 Budget-aware Prompting Strategy:
Our approach incorporates a sophisticated budget-aware prompting strategy that optimizes the allocation of token budget for image processing. This strategy begins with a user-specified total token budget Btotal, allowing for customization based on specific task requirements or computational constraints. We propose four varieties Zoomer to accommodate different. budget scenarios and task requirements:
Zoomer-Local (+): This variant utilizes only the meatial-preserving schema to consolidate all focused image slices into single image patch(@ in Figure 3 Tt is optimal for scenarios with very limited token budgets, prioritizing the most relevant visual information.
Zoomer-Adaptive (+ + @ ): This approach dynamically includes a global view of the original image if the cropped portion falls below certain threshold Ta. This allows the MLLM to better understand the overall scene context when the budget permits, while still focusing on key areas of
interest.
Zoomer-Global (4 + 3): This variant assigns global view to all images, regardless of the specific regions of interest. Tt is suitable for tasks that require consistent. overall context and when the token budget is sufficient to include both global and local information. Zoomer-Patchest
+ @): This is the most token-intensive approach, assigning each image slice its own patch without spatial preservation, along with a global view. It provides the most detailed information but requires the largest token budget.
Adaptive global view decision.
total area all cropped ROls
TA = area. of the original image
(1)
Let EQ [ denote the fraction of the original image covered by the selected ROIs Zoomer-Adaptive decides whether to append global overview based on Ta: if Ta < 0.35, append down-scaled global view to preserve overall scene context; if Ta > 0.35, omit the global view to save visual tokens, since the ROIs already cover sufficiently large portion of the scene. The threshold 0.35 is chosen empirically on the VSTAR validation set to balance accuracy and token efficiency.
The selection among these variotics depends on the uscr-specified budget and the nature of the task. For cach variant, the number of higl-resolution slices or patches N calculated based on the available budgel and the token cost per slice or patch. These slices are selected from the output of our Multi-Scale Emphasizing Algorithm, prioritizing based on their relevance to the key term of text prompts. To present the methods more clearly and vividly, refer to Figure which outlines the methodology, and Figure which showcases specific case study.
Question: How many cactus in the image?
Text of Interest: cactus Image
Image
Zoomer
Region of Interest
OXOK
Figure 4: The example of applying Zoomer
5 Experiments
We evaluate the performance of Zoomer on a diverse set of visual reasoning tasks using multiple black- box MLLMs. Our experiments are designed to assess both quantitative gains in accuracy and qualitative improvements in visual fidelity under constrained token budgets.
Specifically, our evaluation is guided by the following research questions: RQ1: Accuracy. Does Zoomer
improve task accuracy across different. black-box MLLMs on image-grounded reasoning tasks, compared
8

---

Under review as submission to TMLR
Zoomer Local
Zoomer Adaptive
Global
Zoomer Patches
VAXKO
Figure The example of different settings of Zoomer
Acc./Tooken
Bench
Vatar
CVBench-2D CVBendh-3D RealworliQA
SQA-1
MMVP
MMMU
HR-4K
BR-8K
Mothod
Raw
56.5% /955
68.5%,/428
78.2%/
67.6%,/995
87.3%,/353
83.3% /200
68.4%,/008
50.6%/ 1105
46.8%
Resize
41.9%,
66.3%,
75.2%,
61.19,/270
86.8%,/270
83.3%//270
62.9%%
36.8%,/270
33.4%%
Loomer-Lonil
67.1% /200
72.4%,/270
86.2%,220
72.4%,/27
88.3%
87.1%/270
59.6%/270
58.896/270
57.7%
Noomer-Adaplive
67.3%, 119
72.9%/3744
87.9% 808
74.7%/336
91.1%
88.7%
61.69/3312
60.85//331
59 59.3%
Zoomer-Glen Globel
/5440
73.19//540
88.3% 25.3%%/54 /540
75.3%%/540
92.372/540
88.9%/5400
67.3%
61.3%
59.8%/644
Goomer-Patclan
71.7%//1029
7.16%6/709
85.8% /1113
75.8%/997
92.8%
88.156/228
68.9%/811
60.156/718
986,
Table 2: Performance of GPT-4o across different datasets using various image prompt processing meth- ods, focusing on accuracy and token consumption. Among these approaches: Local: Only the extracted Rols are used. Adaptive: Selectively provides the MLLM with a global view of the image based on the prompt strategy. Global: Every request includes the global view of the image. Patches: Does not use the Spatial-Preserve Orchestration Schema; instead, each possible Rol is independently provided to the MILM, including the global view.
standard prompting basclines? RQ2: Token Efficiency. How does Zoomer balance the trade-off between accuracy and token usage, and how does it perform relative to methods that either over-utilize under-utilize the token budget? RQ3: Component Effectiveness. What is the contribution of each major component of Zoomer specifically, the multi-scale visual emphasis module and the budget-aware prompting strategy- to the overall performance?
Backbone MLLMs. We evaluate Zoomer on both commercial APls (e.g., GPT-4o, Gemini, Claude) and open-source MILMMs. For the latter, we use Qwen2-VI VL-7BB and InternVL2..5SR as representative models this work, primarily due to time and resource constraints. As shown in Table & applying Zoomer on top of both open-source backbones consistently improves performance across four benchmarks, indicating that

---

Under review as submission to TMLR
Accuracy
Tokens
Lateney
Money Cost ($10-c3)
Method
Zero-Shot
15-Shot
Zero-Shot
15-Shot
Zero-Shot
15-Shot
Zero-Shot
15-Shot
Raw
78%
84%
963
13488
4.8s
18.7s
4.815
67.
Resize
61%
74%
255
4080
2.9s
7.58
1.275
20.4
Low-Detail
60%
70%
85
1360
2.1s
6.5s
0.425
6.8
Zoomer-Adaptive
83%
88%
315
5112
3.1s
9.8s
1.575
25.56
Table 3: Performance in terms accuracy, latency, and image token cost on Terralncoguita under ICL conditions specifically with 15 examples per question and under zero-shot conditions.
API
Method
Vstar
CVBench-2D
RealworldQA
MMVP
Raw Zoomer
56.5%
68.5%
67.6%
83.3%
GPT-4o
71.7%
74.6%
75.8%
88.9%
Gemini-
Raw 53.1%
65.4%
64.0%
79.8%
1.5Pro
Zoomer
70.4%
73.2%
73.9%
87.8%
Claude-
Raw 51.8%
66.7%
61.0%
80.2%
3.5-Sonnet
Zoomer
69.7%
72.8%
74.1%
87.2%
Table 4: Accuracy of Different Black-box MLLM APIs. For Vstar, CVBench-2D. and RealworldQA, we used the Patches version SysName. For MMVP, inspired by Table 2 we employed the Global version.
the proposed adaptive visual token allocation is tied proprietary APls. Extending the evaluation to additional high-resolution open-source models such Qwen3-VI, Qwen2.5-VL, LLaVA-UHD, and MG- LLaVA is an important direction for future work.
5.1 Setup
Assessment and Datasets We evaluated our system on a series of challenging multimodal tasks, using commercial black-box MLLMs for applications ranging from visual-language reasoning to image understand- ing and question answering. The experiments were conducted on a variety of different public datasets, including:
1) Vstar (wu & Xie 2023 A benchmark datasct focused image classification, used to evaluate fine- grained visual recognition capabilities in object detection and classification tasks.
2) CVBench (Tong et al. 2024a): Contains 2 sub-category, CV Benchap and CV Benchap, respectively, representing two-dimensional and throe-dimensional visual image, respectively, to evaluate the performance of the model when processing images different dimensions, especially the understanding ability in complex scenes.
3) RealworldQ4 XAI 2024 Used to test the multimodal question answering performance of the model in real-world scenarios, involving cross-language and cross-image information processing.
4) MMVP (Tong et al.
2024b A validation set for multimodal visual processing, designed to evaluate the comprehensive understanding of models for complex visual scenes.
5) ScienceQA (Tu et al. questions across a diverse range of science topics.
2022):
: multimodal scientific question-answering dataset featuring multiple-choice
6) MMMU Yue et al. performance of multimodal models multidisciplinary tasks that require university-Leve. subject knowledge and deliberate reasoning.
20224
The validation part of new benchmark, which designed to evaluate the
7) HR Wang et al. (204)): high-resolution multimodal benchmark consisting of 4K and 8K images and corresponding questions.
Models We employed three black-box MLLMs GPT-40-0513,
and Gemini-
Pro accessed via their respective APIs (OpenAT, Claude, Google). Across all experiments, we set the
10

---

Under review as submission to TMLR
Method
Model
Prompt Strategy
Vatar
CVBench-9D CVBench-3D ReatworldQA SQA-1
MMVP
MMMU
Local
57.1%
67.6%
79.9%
68.5%
87.8%
84.0%
53.3%
Adaptive
57.5%
71.3%
82.5%
72.1%
88.3%
85.3%
55.9%
EVF-SAM
Global
57.8%
72.1%
83.0%
72.4%
88.8%
87.3%
56.8%
Patches
57.1%
72.7%
83.8%
72.1%
86.8%
84.7%
56.1%
Defanlt
Local
58.1%
70.6%
82.5%
71.5%
86.3%
84.3%
55.6%
Adaptive
58.3%
71.5%
83.9%
73.1%
90.1%
85.6%
56.3%
Ground Dimo
Global
58.3%
71.8%
84.8%
73.4%
90.3%
87.8%
57.0%
Patches
58.8%
70.6%
83.1%
72.6%
90.9%
87.7%
56.6%
Loral
58.4%
69.2%
84.0%
T2.5%
88.3%
84.7%
57. 37%1%
Adaptive
58.5%
71.3%
86.3%
73.1%
91.1%
86.8%
58.7%
EVE-SAM
Global
58.4%
72.1%
85.8%
73.4%
91.8%
87 8%%
59.6%
Multi- Resolution
Patches
60.2%
71.3%
85.7%
73.1%
91.3%
86.8%
59.2%
Local
63.6%
71.8%
82.6%
70.2%
88.8%
85.1%
56.8%
Adaptive
64.4%
71.8%
84.3%
70.6%
90.3%
86.5%
58.5%
Cround Dino
Global
66.4%
72.2%
84.7%
71.4%
91.8%
86.9%
59.3%
Patches
66.2%
72.6%
83.6%
70.2%
92.1%
86.9%
58.8%
Local
63.7%
72.1%
85.2%
70.4%
85.3%
85.7%
57.5%
Adaptive
64.3%
72.4%
86.1%
70.6%
90.1%
87.6%
58.5%
EVF-SAM
Global
64.3%
72.8%
87.9%
71.7%
90.3%
88.8%
59.9%
Patches
67.2%
73.7%
87.1%
73.0%
90.9%
88 0%
59.7%
Multi-Sealc
Local
67.1%
72.4%
86.2%
72.4%
88.3%
87.1%
57.7%
Adaptive
67.5%
72.9%
87.9%
74.7%
91.1%
88.7%
59.3%
Ground Dino
Global
67.6%
73.1%
88.3%
75.3%
92.3%
88.9%
59.8%
Patches
71.7%
74.6%
85.8%
75.8%
92.8%
88.4%
58.9%
Table 5: Performance of Zoomer Across Dalasets for Different Emphasis Methods, Models, and Prompt Strategies.
temperature to 0 and used greedy decoding for consistency, optimizing the stability of outputs. NMS applied with a confidence score threshold of 0.8 to filter irrelevant regions from high-resolution images.
Metrics We used classification accuracy across all examples as the primary evaluation metric. Additionally, we compared token usage for each model configuration to evaluate the efficiency improvements offered by Zoomer.
Token efficiency metrics. Different providers adopt different image tokenization schemes. Whenever an API exposes image-token usage, directly report the provider's visual token count. When this information is not exposed, instead count the number of 512 512 patches that are actually transmitted after reconstruction proxy for visual tokens.
To ensure fair cross-model comparison, we always measure token efficiency as relative reduction with respect to each model's own Raw bascline. For a model m and method I, we compute
tookens (m, x) tokens (m) tokens,
tokensRaw (m)
and pair this with the corresponding accuracy change. This normalization prevents differences in provider- specific tokenization rules from biasing our conclusions about token savings.
Baselines We compare Zoomer against the following baseline methods:
1) Row:
This baseline feeds MILM the unmodified prompt, with no adjustments made to the image.
Here, images larger than 512x512 pixels are resized to fit within the GPT-4o's patch limit, while smaller images remain unchanged.
2) Resize:
5.2 Main results
RQI: Accuracy Under Token Constraints
We first evaluate the accuracy Zoomer across multiple visual reasoning benchmarks using GPT-4o. Table compares its performance with bascline prompting strategies, including Raw (unaltered image input), Crop, and heuristic patching. Across all datasets, Zoomer demonstrates consistent and significant gains.
11

---

Under review as submission to TMLR
Sywe22VL-77D
InternVL2.5-8B
Qwen2-VI.. VL-7B
InternVL2.5-88
Zoomer
+ Zoomer
CVBench-2D
0.6448
0.656
0,683
(+0.035)
0.697 (+0.041)
CVBench-3D
0.632
0.629
0.675
(+0.043)
0.669 (+0.040)
RealWorldQA
0.698
0.698
0.732 (+0.034)
0.741 (+0.043)
VSTAR
0.584
0.373
0.648 (+0.064)
0.635 (+0.082)
Table 6: Performance of Qwen2- VL-7B and InternVL2.5-8B with and without Zoomer. Zoomer consistently improves accuracy across all four benchmarks.
Method
VSTAR
CVBench-2D
CVBench-3D
RealWorldQA
Zoomer
0.717(+0.013)
0.746(+0.045) 0.701
0.883(+00.000)
0.758(+00.00)
V
0.704
0.793
0.698
Table 7: Comparison between Zoomer and stronger high-resolution baseline V V already uses high- resolution inputs without our cropping and composition.
Notably, Zoocer-Patches achieves 71.7% accuracy on the Vstar dataset, outperforming the Raw bascline by 26.9% (from 0.565 to 0.717). On the more challenging RealWorldQ4 dataset, which requires complex visual- linguistic reasoning, Zoomer-Adaptive yields 0.758 accuracy- 12.1% higher than the Raw input (0.676). These results indicate that Zoomer enhances the model's ability to ground object references and interpret fino-grained visual cues by preserving sigh-fidelity regions while respecting token limits.
RQ2: Cross-Moder Generalizability
To assess generalizability, apply Zoomer to other closed-source MLLMs, including Claude-3.5-Sonnee and Gemini-1.5Pro. As shown in Table g the performance trends hold across architectures. For instance, on Vstar, Zoomer improves accuracy from 0.531 to 0.704 on Gemini-1.5Pro -a 32.6% gain. On RealWorldQA. 3.5-one achieves a 34.5% improvement using Zoomer (from 0.610 to 0.741). These results suggest that Zoomer is model-agnostic and portable across different commercial MLLMs without requiring model access or retraining.
As suggested by reviewers, also evaluate Zoomer open-source MLLMs. In particular, use Qwen2- VL-7B and InternVL2..-8B as representative backbones. As shown in Table 5 applying Zoomer top of both models yields consistent gains across all four benchmarks (e.g., +0.034-00 0.043 on RealWorldQA and +0.064-0.082 on VSTAR), confirming that our adaptive image focus mechanism not tied to proprictary APIs.
RQ3: Token Efficiency and Latency
One of the central goals of Zoomer is to maximize information utility under token constraints. Table reports token usage and performance on the TerraTncognita dataset under the ManyTCT, (Jiang et al. 2024 setting. Zoomer achieves 0.83 accuracy with 315 tokens, while the Raw method reaches only 0.78 with 963 tokens yielding a 6.4% accuracy improvement with 67% fewer tokens. This substantial reduction in visual token cost translates directly into lower API usage and faster inference.
Morcover, rcal-time scttings such as autonomous navigation, latency bocomes critical. In a zero-shot Terralncognita setting, Zoomer reduces response time from 4.8 seconds to 3.1 seconds (a 35.4% reduction) without sacrificing accuracy. These results underscore the practical benefits of Zoomer for latency-sensitive and resource-constrained applications such as real-time surveillance, embodied agents, and mobile vision systems.
We further compare Zoomer against a stronger high-resolution baseline V*, which already uses high-resolution visual inputs without our prompt-aware cropping and composition. As shown in Table 7 Zoomer still provides clear gains, especially on CVBench-3D and RealWorldQA. This indicates that adaptive image focus and spatial-preserving orchestration remain beneficial when the baseline already operates in high- resolutio regime, and strengthens connection to high-resolution prompting methods such LLa VA-UD and MG-LLaVA.
12

---

Under review as submission to TMLR
DOG
Raw Image
Zoom-Local
Zoom-Adaptin
Figure Failure case due missed small object. Left: raw image (dog highlighted) and the ques- tion: "Is the dog on the left right side of the golden tower?" (answer: left). Middle: Zoomer-Local reconstruction, where the detector fails to include the dog in any ROI, leading to incorrect prediction. Right: Hoomer-Adaptin reconstruction; appending global view does not fully resolve the issue when the dog is indistinguishable at the down-scaled resolution.
5.3 Findings
While Zoomer-Patches generally performs well under generous token budgets, observe that in certain datasets it underperforms relative to other variants. For instance, the CVBenchap dataset, Zoomer- Patches yields an accuracy that is 2.5 percentage points lower than Zoomer-Global and 4.0 points lower than Zoomer-Local. Similarly, on the MMVP benchmark, the Patches variant trails Zoomer-Global by 0.5 percentage points and Zoomer-Adaptive by 0.3 points. These differences persist across multiple trials and are consistent beyond minor fluctuations attributable model non-determinism.
We hypothesize that this degradation stems from the way Zoomer-Patches encodes visual inputs. Specifically, independent image patch and passed separately to the black-box MLLM. This
each Rol is treated approach, while maximizing local detail preservation, inherently disrupts the global spatial context and inhibits inter-region integration. When the number of Rols increases, the model is forced to reason over fragmented visual inputs without spatial continuity, which may lead to failures in tasks requiring holistic scene understanding or relational reasoning.
Failure case: missed small object in ROI extraction. Despite the overall gains, Zoomer may fail when the ROT extractor misses a small but question-critical object. Figure shows example where the question asks: "Is the dog on the left or right side of the golden tower?" The dog is tiny object in the scene. If the detector fails to propose an ROI covering the dog, Zoomer-Loca will not include the dog in the reconstructed input, causing the MLLM to answer incorrectly. this issue by appending a global view, but still cannot recover the missing evidence when the dog becomes indistinguishable in the down-scaled overview. This highlights a key limitation of detector-guided visual focus and motivates future detector-fre -free visual focusing strategies.
may partially mitigate
These findings suggest that preserving global structure-oven at the cost of slightly reduced local detail-is beneficial in tasks that involve complex spatial semantic dependencies. They also underscore the need for future designs to better balance granularity with contextual coherence in visual prompting pipelines.
5.4 Ablation Study
To assess the contribution of individual components Zoomer, we conducted a comprchensive ablation study along two primary dimensions: (1) the role of multi-scale visual emphasis strategies, and (2) the impact of different Rol localization models. The results are summarized in Table
13

---

Under review as submission to TMLR
Method
CVBench-2D
CVBench-3D
VSTAR
Zoomer-P Patchess
0.746
0.858
0.717
Sandomm-Patcher
0.571(-0. 1-5)
0.613(-0.245)
0.597(-0.120)
Uniform- Patches
(.637(-00109)
0.731(-0.127)
0.664(-0.011)
Table 8: Comparison of patch-selection strategies under the same patch budget. All methods use the same number of patches; Zoomer-Patches gains come from semantic selection and spatial composition.
Patch selection under fixed budget. As stronger baseline than Raw and Resize, we compare Zoomer-Patches with Random and Uniform patch selection under the same patch budget (Table S Since all three strategies transmit the same number of patches, the improvements of come from semantically guided region selection and spatial-preserving composition, rather than from simply increasing visual resolution.
can only
Visual Emphasis Strategies. We compare three emphasis methods: (i) baseline strategy using the full image without emphasis ("Default"), (ii) a multi-resolution method that applies image resizing at various scales without cropping, and (iii) our proposed multi-scale strategy, which extracts overlapping Rol patches across hierarchical gramularities.
Across all tested datascts, the multi-scale approach consistently outperforms the other two. For instance, the Vstar and CVBench datasets, the multi-scale strategy yields relative improvements of 8.2% and 6.4% over the multi-resolution method, respectively. This suggests that the hierarchical region recall provided by the multi-scale cropping mechanism compensates for potential object boundary fragmentation. In contrast, multi-rcsolution wothods-though preserving global structure- structure-nngativelyy impact performance, likely due to the model's reliance on fixed-size input distributiins during pretraining. Adjusting resolution without architectural adaptation may lead to degraded feature extraction in the frozen backbone of black- box MLLMs.
Backbone Variants for Region Extraction. We also evaluate the effect of using different visual back- bone models for region proposal and saliency detection. Specifically, we compare GroudingDINO (Liu al., b) and EVF-SAM as Rol extractors in the multi-scale visual emphasis pipeline. Both variants yield signif- icant improvements over the baseline; however, GroundingDINO exhibits slightly higher accuracy across datasets. This result suggests that while both models effectively localize lask-relevanl regions, the localiza- tion precision and language grounding of GroundingDINO may offer a marginal advantage in aligning visual slices with text prompts.
Area Ratio Threshold TA We vary the threshold in Eq. equation Jmong {0.25, 0.35, 0.45) and report results on the VSTAR validation set.
Ta
Accuracy
Relative token usage
0.25
0.645
0.82
0.35
0.652
0.79
0.45
0.647
0.76
Table 9: Ablation on the area ratio threshold TA for Zoomer- Adaptive on VSTAR (val).
We observe that Ta = 0.35 achieves the best trade-off between accuracy and token efficiency: smaller thresholds append the global view too often, increasing token usage with marginal accuracy gains, while larger thresholds skip the global view in cases that benefit from additional context.
Combining the multi-scale emphasis with the Patches variant Zoomer yields the strongest overall performance, validating the importance of both region-level granularity and architectural alignment. These findings highlight that visual prompting should be co-designed with content-aware pre- processing and prompt-space organization, especially under token-constrained inference regimes.
Synergistic Effects.
14

---

Under review as submission to TMLR
6 Conclusion
This paper presents Zoomer, a novel visual prompting framework that addresses the challenge of preserving fino-grained visual detail under token constraints in black-box multimodal language models (MLLMs). By decomposing the prompting task into compt-aware emphasis, spatially structured orchestration, and budget- aware input construction, Zoomer enables effective visual grounding without access to model internals or architectural modifications.
Comprehensive evaluations across datasets such as Vstar, RealWorldQA, and TerraIncognita demonstrate that Zoomer consistently improves task accuracy while reducing token consumption. For example, Zoomer- Patches achieves a 26.9% gain over baseline accuracy on Vstar, while Zoomer-Adaptive improves Real- WorldQA A accuracy by 12.1%. On Termahcoogila, Zoomer achieves higher accuracy with 67% fewer tokens, highlighting its practical utility in resource-constrainee settings.
While the present work focuses on token-level efficiency, an important emerging challenge is the communica- tion cost associated with transferring high-resolution images from edge devices to cloud-based MLLMs. This concern is particularly relevant in scenarios such as wearable computing or mobile robotics. A promising direction for future rescarch is to extend Zoomer toward edge-aware visual prompting, enabling lightweight pre-processing directly on-device to reduce upstream bandwidth and latency while maintaining task perfor- mance.
In summary, Zoomer contributes a modular and model-agnostic framework for enhancing visual processing in black-box MLLMs. Its integration of spatial structure, semantic relevance, and token budget constraints offers a principled foundation for both immediate deployment and future extensions in bandwidtt--constrineed real-time vision-language applications.
From heuristics to explicit optimization. The current Zoomer pipeline heuristic in nature: it relies prompt-aware detection, spatial- composition, and simple token budgeting rules. A natural next step is to formalize adaplive visual token allocation as constrained optimization problem, e.g., maximizing task accuracy or expected utility under fixed visual-token budget and latency constraint, and to develop trainable surrogates or reinforcement learning policies that optimize this objective end-to-end.
Beyond external detectors. Zoomer currently relies on open-vocabulary detectors (e.g., Ground- ingDINO, EVF-SAM) applied in a multi-scale fashion, which can become a bottleneck when detector perfor- mance degrades. Our experiments partially mitigate this via Zoomer-Adaptive and Zoomer-Globa which automatically include a global view when the cropped area ratio small. Nonetheless, exploring detector- light or detector-free mechanisms such as leveraging internal attention maps or saliency predictions from open-source MLLMs-is an exciting dircction for future work.
References
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Jain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooci, Marianne Montciro, Jacob Menick, Sebastian Borgcaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, An- drew Zisserman, and Karen Simonyan. Flamingo: A Visual Language Model for Few-Shot Learning, URL /ttp://arxiv http:://xxxvvoorg///// wwg/ /bw.a////.....222
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen- VL: Versatile VasguanLanguage Model for Understanding, Localization, Text Reading, and Beyond. URL
12966
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-End Object Detection with Transformers. htRp: http://axxwv..org//////////// 12872.
15

---

Under review as submission to TMLR
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spa- tialylm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 14455 14465, 2024.
Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoor- thi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny, MiniGPT-v2: Large language model as a unified interface for vision-language multi-task learning. URL http:://axii..ooo http://aaxxii...g/bbbbba///// wwtg/b/////aaeee httpaab//22100947..
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. InstructBLIP: Towards General-purpose Vision- Lisiun-Language Models with Instruction Tuning. URL httt:://aaxxxi// org/
De Marneffe and Christopher D Manning. The stanford typed dependencies representation. In Coling 2008: proceedings of the workshop on cross-frameuork and cross-domain parser evaluation, pp. 1 8, 2008.
Zhaopeng Gu, Bingke Zhu, Guibo Zhu, Yingying Chen, Ming Tang, and Jinqiao Wang. Anomalygpi: Detecting industrial anomalies using large rision-langue models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 1932-1940, 2024.
Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. CogAgent: A Visual Language Model for GUT Agents. URL http: httt:://arxivv http://aawivv..g.bbb////aaaa
08914
Yixing Jiang, Jeremy Irvin, Ji Hun Wang, Muhammad Ahmed Chaudhry, Jonathan H. Chen, and Andrew Y. Ng. Many-shot in-context. learning in multimodal foundation models, 2024. URL https://arxiv.orr org/
abs/2405.09799 09798
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tote Xiao, Spencer Whitehead, Alexander C. Berg, Wan- Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything, 2023. URL
02643
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. LLaVA- NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models, a. URL http:
//arxiv. .wwg/rrggge.o. htee/bb//200 ht////bb////0/0// 2707.07895
Jiaxuan Li, Duc Minh Vo, Akihiro Sugimoto, and Hideki Nakayama. Evcap: Retrieval-angmented im- age captioning with external visual-name memory for open-world comprehension. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 13733 13742, 2024.
Liunian Harold Li, Pengehuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Ta Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded Language- Image Pre-training, b. URL /tttpi//arxiv.. www.aawwww...gaab.aaaaaae org/abbs/212..00 03857
Yanwei Li, Chengyao Wang, and Jiaya Jia. LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models.
Haotian Liu, Chumyuan Li, Qingyang Wu, and Yong Jae Lee. Visual Instruction Tuning, a. URL http:
ohaxxiv.org/are wtgp.//////a.......///// 08488
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding DINO: Marrying DINO with Grounded Pre- Training for Open-Set Object Detection, b. URI. http:://aatiii http://axxxvv.or//aa/// wtt.:orgg/ab...ooooooo 05499
Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In The 36th Conference on Neural Information Processing Systems (NeurIPS), 2022.
16

---

Under review as submission to TMLR
Gen Luo, Yiyi Zhou, Yuxin Zhang, Xiawu Zheng, Xiaoshuai Sun, and Rongrong Ji. Feast Your Eyes: Mfxture-of- Adaptation for Multimodal Large Language Models. URL http://arxiv.orgg Abs/2403.03003
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Roughang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloc Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam 2: Segment anything in images and videos. are preprint ar (rXii:2448.0077 2024. URL https: /tapxiv//axxv....// org/abe/
00714
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You Only Look Once: Unified, Real-Time Object Detection. URL
1101//1166.00 02640
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. URL http:://arrdl..orrabb////55 000000009100019977 01497
Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning, Raptor: Recursive abstractive processing for tree-organized retrieval. In International Conference on Learning Representations (ICLR), 2024. URL htttp:://arxtii www.aawwww....iaaoaaaaaa 2001
Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-l: A fully open, vision-centric exploration of multimodal Ilms, 2024a.
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms, 2024b.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2- VL; Enhancing Vision-Language Model's Perception of the World at Any Resolution. URL httl:ttpp/// htttp://arxiv.ort www.oorgg//bbbbbbeee.. htbbs//449.1200
Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models, 2024. URL https://arxiv.orge org/ www..rrg////////e///.....eeeeeee 040s///48.... 15556
Penghao Wu and Saining Xic. V*: Guided visual search as mechanism in multimodal 1lms. arXiv preprinl ar Xrv: 14135, 2023.
XAI. Grok. 2024.
Ruyi Xu, Yuan Yao, Zonghao Guo, Junbo Cui, Zanlin Ni, Chunjiang Ge, Tat-Seng Chua, Zhiyuan Liu, Maosong Sun, and Gao Huang. LLaVA-UHD: An LMM Perceiving Any Aspect Ratio and High-Resoluse Images. URL http:///axii http:://arxiv...m/ org/ abs/ faabs/2405
Yifan Yang, Weiquan Huang, Yixuan Wei, Houwen Peng, Xinyang Jiang, Huiqiang Jiang, Fangyun Wei, Yin Wang, Han Hu, Lili Qiu, and Yuging Yang. Attentive Mask CLIP. URL http:///axii org/ abs/2212 08653
Lewei Yao, Jianhua Han, Youpeng Wen, Xiaodan Liang, Dan Xu, Wei Zhang, Zhenguo Li, Chunjing Xu, and Hang Xu. DetCLIP: Dintionary-Enriched Visual-Concopt Paralleled Pre-training for Open-world Delection. URL http://arxiv. org/abs//////oooooooooooooo.eerr 220b9//220..... .94007
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Slevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi- discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of CVPR, 2024.
17

---

Under review as submission to TMLR
Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Open-Vocabulary DETR with Conditional Matching. volume 13669, pp. 102-122. doi: 10.1007/978-33431--007777 7. URL http:
wwwpivv.rrggbbb///0/ 2201/ 11876
Yi-Fan Zhang, Qingsong Wen, Chaoyon Fu, Xue Wang, Zhang Zhang, Liang Wang, and Rong Jin. Beyond LLaVA-HD: Diving into High-Rcsolution Large Multimodal Modcls, URL http://arxiv.org/ org/ abs/ 2406.08483
Yuxuan Zhang, Tianheng Cheng, Rui Hu, Lei Liu, Heng Liu, Longjin Ran, and Xiaoxin Chen. EVF-SAM: Early Vision-Tanguage Fusion for Text-Prompted Segment Anything Model. 14(8), b.
Xiangyu Zhao, Xianglai Li, Haodong Duan, Haian Huang, Yining Li, Kai Chen, and Hua Yang. MG-LLa Towards Multi- Multi-Gramin Granularity Visual Instruction Tuning. URL http: //arx//rivvv orgg/bs/////aaaeeeo 17770
18


Reviews:
[{"review_id": "8AhhZOYT32", "reviewer": "Reviewer_mgTW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Zoomer, a visual prompting method that improves visual reasoning in black-box multimodal LLMs by performing prompt-guided localization before the image is sent to the model. Given a text query, Zoomer extracts key nouns from the prompt (e.g., “cactus,” “book title”) and uses an external open-vocabulary detector (GroundingDINO or EVF-SAM) to search for these objects across multiple scales of the original image. It divides the image into patches and runs detection on each patch to avoid missing small objects, collects all bounding boxes with confidence above a threshold, and applies Non-Maximum Suppression (NMS) to remove redundant or overlapping regions. The remaining regions are then cropped and reconstructed into a new image while preserving their original spatial layout, effectively highlighting relevant areas without losing context. Finally, Zoomer performs token-budget allocation, choosing among several variants — Local (only ROI patches), Adaptive (ROI + global view if tokens allow), Global (always retain global layout), or Patches (each ROI sent separately) — to ensure that visual detail is preserved under the model’s fixed image token limit.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper supports its claims with detailed experiment results.", "audience_yn": "Yes", "audience_explanation": "The paper findings is about adaptive fine grained reasoning which is interesting to the community.", "requested_changes": "The word “optimization’’ is used throughout the title and paper, but there is no explicit optimization objective, no loss function, and no algorithmic optimization process described. The method consists entirely of heuristic region selection (via external detectors), cropping, and token budgeting; consequently, the use of the term optimization may misrepresent the actual technical contribution. In fact, it will be interesting if the author can formulate this token budget saving as a constrained optimization problem for adaptive allocation.\n\n \nI would be particularly interested in a strategy that does not rely on an external object detector, since detector performance becomes the bottleneck of the system. Nevertheless, the proposed approach represents a reasonable first step toward prompt-driven, detector-free visual focus for black-box MLLMs.\n\nThe paper would benefit from a sensitivity analysis or visualization of failure cases in the baseline (e.g., uniform downsampling or direct input) that Zoomer successfully resolves. Showing side-by-side examples where the baseline fails to perceive small objects or loses spatial context, but Zoomer succeeds, would make the contribution more concrete and highlight the value of the proposed approach.\n\n\nFinally, it would be interesting to compare this approach with sota methods such as Llava-uhd.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OJCg0X8uQQ", "reviewer": "Reviewer_BSyD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses the challenge that many multimodal large language models (MLLMs) (e.g., GPT‑4o, Gemini Pro) struggle with fine-grained vision tasks because strict token budgets force them to uniformly down-sample high-resolution images, causing loss of critical detail. The authors identify two core problems: (1) absence of region-adaptive attention (i.e., the models treat all image regions equally) and (2) inflexible token budgets that don’t allow balancing global context vs local detail. To overcome this, they propose Zoomer, a visual-prompting framework that treats the model as a black box (no internal modifications). Zoomer consists of (a) a prompt-aware emphasis module to focus on semantically relevant image regions, (b) a spatial-preserving orchestration schema to maintain object relationships in the cropped/viewed patches, and (c) a budget-aware strategy allocating tokens between global and local image content. Experiments on nine benchmarks and three commercial MLLMs show accuracy improvements up to ~27% while reducing image token usage by up to ~67%. The work suggests a principled way to improve vision-language performance in resource-constrained, black-box settings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors present extensive experiments across nine benchmarks and three commercial black-box MLLMs (GPT-4o, Gemini, Claude), showing consistent performance gains of up to 27% while reducing token usage by about 67%, which strongly supports their claims of improved efficiency and accuracy. The ablation studies further validate the contribution of each module—prompt-aware emphasis, token-budget allocation, and spatial orchestration—demonstrating that each component is necessary for the overall improvement.", "audience_yn": "Yes", "audience_explanation": "The paper is clear and has comprehensive experiments on whether adaptive image focus can help visual understanding.", "requested_changes": "1. I would recommend that the authors include an additional experiment using an open-sourced MLLM such as Qwen3-VL or Qwen2.5-VL. This would likely attract greater interest from the open-source community, as the models evaluated in the paper are relatively costly for academic researchers. If similar conclusions can be drawn for open-source models, practitioners may be more inclined to adopt the proposed method. To be specific, adding two more rows for Qwen3-VL in Table 4 would be sufficient.\n\n2. I appreciate Table 3 and would suggest adding more rows showing additional variants of Zoomer, such as patches and global, to better illustrate the trade-off between speed, cost, and accuracy.\n\n3. How do you select the threshold $T_A$ for Zoomer Adaptive? It would be nice if you could include some ablation details.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sydpFlIGms", "reviewer": "Reviewer_Tnwh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces Zoomer, a visual prompting framework that enhances multimodal large language models (MLLMs)—specifically black-box ones such as GPT-4o, Gemini Pro, and Claude 3.5—by improving their ability to process high-resolution images under token constraints. The core idea is to adaptively select and emphasize relevant image regions while respecting strict token budgets, thereby preserving fine-grained details without modifying model internals. \n3 main modules:\nVisual Emphasizer which is tasj aware - extracts and magnifies regions of interest based on semantic cues from text prompts.\nSpatial-preserving Orchestration Schema – reconstructs selected regions to retain spatial coherence.\nBudget-aware Prompting Strategy – dynamically allocates tokens between local detail and global context.\n\nThe proposed approach is validated on 9 benchmarks showing accuracy accuracy gains up to 27% while reducing image token usage by up to 67%.", "claims_and_evidence_yn": "Yes", "claims_explanation": "“Large accuracy gains under token budgets.” The paper backs this with per-dataset tables and an explicit example: on Vstar, Zoomer-Patches 71.7% vs Raw 56.5% (Δ=+26.9 points), and on RealWorldQA Zoomer-Adaptive 75.8% vs Raw 67.6% (Δ=+12.1), directly supporting the headline claim of big gains under constraints.\n\nUp to ~67% fewer image tokens at similar or better accuracy. The TerraIncognita result shows 0.83 accuracy with 315 tokens vs 0.78 with 963 tokens for raw ~67% fewer tokens while improving accuracy, matching the abstract’s efficiency claim.", "audience_yn": "Yes", "audience_explanation": "Definitely applicable for the vision community", "requested_changes": "* Current baselines (Raw, Resize) are relatively weak. There are no comparisons against open-source high-resolution visual prompting methods (e.g., LLaVA-UHD, Qwen-VL, or MG-LLaVA).\n\n* “Token savings” are central to the paper’s claims, but methods for counting image tokens differ across APIs (e.g., OpenAI vs Gemini vs Claude). The paper doesn’t specify how token equivalence was maintained.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hKYIch2GbW", "reviewer": "Reviewer_zvUt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes Zoomer, a visual prompting method to improve the effectiveness of visual inputs to proprietary MLLMs while reducing the number of visual tokens. Zoomer first extracts visual regions that are semantically relevant to the text prompts and then arrange these crops in a spatial-preserving and budget aware way in visual prompts. The proposed method, Zoomer, significantly improves the performance of proprietary MLLMs (up to 27%) while reducing the visual tokens (up to 67%). More importantly, it is shown to be effective across three models and seven benchmarks. The paper conducted comprehensive experiments to ablate the effectiveness of each module and try out four budget-aware visual prompting strategies.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper conducts comprehensive experiments to verify the effectiveness of the proposed method. It is evaluated across three models and seven benchmarks. The paper also compares four different budget-aware prompting strategies, validating the effectiveness of the design choice.", "audience_yn": "Yes", "audience_explanation": "The proposed method, Zoomer, improves the performance of proprietary MLLMs while reducing the visual tokens. This is valuable for the wide applications of proprietary MLLMs.", "requested_changes": "1. Some of the methodology parts, including Section 2 and Section 4, are not clear. The paper should clarify the following questions.\n+ In step 2 (Image Crop) of the second paragraph of Section 2, how is the region of interest and target object extracted? Are they annotated in the dataset or if they are extracted in the same way as described by Section 4.1?\n+ In step 3 (Zoomed Crop) of the second paragraph of Section 2, what is the $512\\times512$ constraint? Does it mean that the cropped region is resized so that the longer side has 512 pixels? What if the raw image is larger than $512\\times512$?\n+ In Prompt Tokenization in Section 4.1, dependency parsing is used to identify the relationships between entities. However, the relationships are not utilized in the subsequent process of Zoomer. Why is dependency parsing used in the method?\n+ In Zoomer-Adaptive and Zoomer-Global, is the image containing region of interest concatenated with the global view? Or are they fed into the MLLM as two separate images?\n\n2. In Section 5.2 RQ3 and Table 3, the models are evaluated in both zero-shot setting and in-context learning setting. However, RQ3 only discusses the performance in the zero-shot setting. The purpose and conclusion of the in-context learning setting are unclear.\n\n3. It would be appreciated if the paper could include qualitative results. Qualitative results would help readers to understand a) how effective is the visual emphasizer to extract region of interest, b) what do the visual prompts look like, and c) what are the model outputs under different prompting strategies. Moreover, providing a few failure cases would be helpful to understand the limitation of the proposed method.\n\n4. Minor writing issues:\n+ The titles of Section 5.2, 5.3 and 6 are not capitalized.\n+ The first paragraph of Section 1 mentions that \"These models ... are increasingly perceived as possessing near-human or even PhD-level capabilities in controlled environments\". It would be better to cite references that support this point.", "broader_impact_concerns": "No broader impact concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)