SigLIP-HD by Fine-to-Coarse Supervision
Authors: Lihe Yang
Zhen Zhao
Hengshuang Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: vision-language model
multi-modality LLM
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: High-quality visual representation is a long-standing pursuit in computer vision. In the context of multimodal LLMs (MLLMs), feeding higher-resolution images can produce more fine-grained visual tokens. However, it introduces additional computational and design complexity, due to multiple forward passes and post-processing of increased tokens. Before simply adopting a higher resolution, have we truly unlocked the model's full perception capability at a standard resolution? Therefore, we study an interesting problem: how to achieve fine visual perception under lower cost without larger images. We present SigLIP-HD in this work. The core is a highly simple fine-to-coarse supervision design. We enforce the coarse feature of a mid-resolution image to mimic the fine-grained feature of its high-resolution version. We build this framework on the advanced SigLIP 2 model. Our final model produces better visual tokens at exactly the same inference budget. It is validated on extensive MLLM benchmarks and consistently delivers stronger results than our baseline model, especially on OCR-related tasks.

Published as a conference paper at ICLR 2026
SIGLIP-HD BY FINE-TO-COARE SUPERVISION
Lihe Yang'
Zhen Zhao
Hengshuang Zhao'
'The University of Hong Kong
Shanghai AI Laboratory
https: //ggthbbb.tb////g/g//gght.ttbbtt com Sonm///n/e/en///oooooo hatpeeunnnngggggiiiieeegg SigLIF-HD
ABSTRACT
High-quality visual representation is long-standing pursuit in computer vision. In the context of multimodal LLMs (MLLMs), feeding higher-resolution images can produce more fine-grand fine-grained visual tokens. However, it introduces additional computational and design complexity, due to multiple forward passes and post- processing of increased tokens. Before simply adopting higher resolution, have we truly unlocked the model's full perception capability at a standard resolution? Therefore, we study an interesting problem: how to achieve fine visual perception under lower cost without larger images. We present $ SigLIP-HD in this work. The core is highly simple fine-to-coarse supervision design. We enforce the coarse feature high-resolution version. We build this framework on the advanced SigLIP 2 model. Our final model produces better visual tokens at exactly the same inference budget. It is validated on extensive MLLM benchmarks and consistently delivers stronger results than our baseline model, especially on OCR-related tasks.
mid-resolution image to mimic the fine- -raiind feature its
INTRODUCTION
From supervised pre-training (Deng al., 2009), vision-centric self-supervised learning (Wu et al., 2018; He et al., 2022), to vision-language contrastive learning (Radford et al., 2021), further to hybrid training paradigms (Maninis al., 2025), the computer vision community keeps pursuing more transferable visual representations. High-quality image embeddings (Tschannen et al., 2025; Siméoni et al., 2025) have fundamentally advanced the development of wide range of perception and generation tasks (Lin et al., 2014; Yu et al., 2025). In recent years, witnessing the power of LLMs (Achiam et al., 2023), finding better visual tokens for multimodal LLMs (MLLMs) (Liu et al., 2023) is receiving growing attention (Tong et al., 2024b;a). The quality of these tokens is critical for MLLMs to accurately perceive and reason over visual signals.
There are three mainstream approaches improving the visual representations in MLLMs. The first is to directly pre-train a better vision encoder from scratch with better algorithms (Oquab et al., 2024; Tschannen al., 2025), more data (Fan et al., 2025; Bolya al., 2025), or larger models (Siméoni et al., 2025). This line of work requires tremendous resources (e.g., million GPU hours, billion data), which are unaffordable for most researchers. The second approach is leveraging multiple encoders (Tong et al., 2024b;a; Shi et al., 2025b). Different encoders have their own strengths. CLIPs (Radford al., 2021) are good modeling vision-language correspondence, while vision-only models (Oquab et al., 2024) excel at capturing detailed visual correlations. Incorporating them may amplify their distinct advantages while suppressing their drawbacks. Despite being intuitively promising, the actual gain is limited or even negative (Tong et al., 2024a).
The last approach simply to forward higher-resolutin input (Liu et al., 2024a;b; Li et al., 2024). Higher-resolution images can yield more fine- mme-grained visual tokens, making MLLMs see more clearly. This principle gradually strengthened by state-of-the-art MILMS (Liu et al., 2025; Wu et al., 2024b; Deitke et al., 2025), from resizing to fixed larger size (Liu et al., 2024a). to fully preserving the native resolution (Li et al., 2025a; Bai et al., 2025), steadily contributing to better OCR capability. Our work is inspired by this observation, but with fundamentally different roadmaps. We highlight that, although increased image size improves visual perception, it brings significant extra complexity in both compute and design. The image has be sliced into small tiles (Liu et al., 2024b;
Corresponding author

---

Published as a conference paper at ICLR 2026
KOBE BRYANT
Have we fully unlocked the perception capability at medium resolution?
LLaVA-OV, Qwen2.5-VL, etc
Cost
LLaVA, LLoVA-155
KOBE BRYANT
easy
challenging
33. B43
impossible
Became oldest pleyer in NBA history to score 60 more paints game
Became the aidest player NBA history score 60 more points in game
Image Resolution
Low
Medium
Native
Figure I: Early MILMS (Liu et al., 2023; 2024a) resize images to a fixed low resolution (e-g-, px)' while recent works (Li et al., 2025a; Bai al., 2025; Liu al., 2025) operate on native resolution with huge costs. But indeed, at a medium resolution (e.g., 512px), humans can already understand the content. How make Al systems achieve this?
Li et al., 2025a) to match the pre-trained vision resolution. This not only requires time-consuming multiple forward passes, but also incurs more visual tokens. To alleviate the LLM's burden, further post-processing, e.g., resampler (Alayrac et al., 2022) and pixel unshuffle (Chen et al., 2024b), is necessary for token compression, making the framework even more complicated.
We reflect on the scaling trend. Before increasing the input resolution, have we fully unlocked the model's 's perception capability at standard (medium) resolution (e.g.. 512?px)? Can we achieve fine-graine visual perception under lower cost without larger images?
At least our human visual system can. As shown in Figure 1, when downsampling the image from its native resolution (1722px in height) to a medium size (512px), although the characters are blurred, we can still accurately understand the content. Therefore, if an Al visual system developed capable enough to match human performance, it should be able to perceive at such medium resolution, without requiring a larger image. It will avoid unnecessary computation at high resolutions, reducing token redundancy and improving system efficiency.
So the remaining problem is, how to make vision models acquire such promising perception capability. Our solution is highly simple yet effective. We design fine-to- supervision mechanism to transfer high-quality visual representation at a high resolution to coarse visual tokens at a standard resolution. Concretely, given pre-trained vision encoder (Tschannen et al., 2025), further fine- tune it at a standard resolution by enforcing its output features to mimic the fine the-grained features of the corresponding high-resolution images. Through this mechanism, the encoder gradually bootstraps its representations from coarse level to fine level. Our method does not rely on any human or synthetic labels for aine-grained perception (Chen et al., 2025; Choi et al., 2025; Shi et al., 2025a), purely exploring cheap raw images for self improvement. Without using any auxiliary upsamplers (Fu et al., 2024; Huang et al., 2025), our final model inherits the same structure and IO as our pre-trained model, making it very easy and efficient to deploy in complex systems. Users only need to modify the checkpoint path to ours, and then enjoy better perception capability.
Our main contributions are summarized below:
We reflect on the scaling trend of image resolution in MLLMs. Beyond using high-resolution images for high-quality visual tokens, we study an intriguing problem: how achieve fine- grained perception for MLLMs under lower cost without larger images.
We present a highly simple fine-to-coarse supervision mechanism to strengthen perception capability standard resolution by imitating "teacher" tokens of high-resolution images.
We build our method on the competitive SigLIP 2 encoder (Tschannen et al., 2025). With the same inference cost, our SigLIP-HD delivers better results across diverse MLLM bench- marks under various protocols, especially for scenarios that favor fine-grained perception.
Following the philosophy of our fine-to-coarse supervision, in Section 2, first investigate what good fine -aaigre representation to learn. Then in Section 3, we present our framework on how to learn from the fine-grained representation.
2

---

Published as a conference paper at ICLR 2026
Table 1: Comparison between CLIP (Radford et al., 2021) and SigLIP 2 (Tschannen et al., 2025) on MLLM benchmarks following the training protocol of LLa ALLVA-1.5--B (Liu et al., 2024a). We evaluate their pe-ppefooreed versions. "We do not include OCR tokens into prompts, following (Zhang et al., 2025). We divide the MMEP score by 20 before averaging.
OCR & Chart
General
Knowledge
Doc
ChanQA
Text
InfoVQA
TexpCps
HRBench
MMBench
Vision encoder
RWQA
GQA
MME
POPE
SQA
A12D
Avg
CLIP-L/14-336p
22.4
17.5
46.8
20.8
69.0
36,9
56.0
63.0
1520.9
86.9
66.2
70.2
55.9
52.9
SigLIP 2-L/16-384pp
25.1
16.4
55.1
21.5
70.6
39.3
56.0
63.9
1538.7
87.2
67.5
69.1
58.7
54.4
SigLIP 2-400m/144384px
29.5
19.0
59.7
22.9
71.5
42.2
57.4
64.8
1538.4
87.8
67.8
70.8
58.2
56.0
SigLIP 2-400m/11-5512pm
32.2
19.3
61.0
22.1
71.1
41.3
57.5
62.8
1529.2
87.8
67.7
69.9
56.0
55.8
SigLIP 2-G/16-3444
26.7
18.5
58.2
21.4
71.3
42.7
59.9
64.6
1559.4
87.0
68.6
71.1
59.9
56.0
Table 2: Comparison among different image scales to obtain final features based on SigLIP 2-512px. Here, we interpolate high-resolution features to base scale and average multi-scale features (if any) to ensure the same number of visual tokens across all settings.
OCR & Chart
General
Knowledge
DocVQA
ChanQA
TeXVVQ
InfaVQA
TextCaps
HRBench
MMBench
Image scale
RWQA
GQA
MME
POPE
SQA'
A12D
Avg
5122
32.2
19.3
61.0
22.1
71.1
41.3
57.5
62.8
1529.2
87.8
67.7
69.9
56.0
55.8
5122+1024
37.0
21.2
64.2
23.8
71.0
47.9
59.6
64.9
1560.1
88.0
67.4
71.0
56.2
57.7
512++1224++1536
36.1
19.2
64.4
23.1
71.0
48.4
59.4
64.8
1565.1
88.7
67.2
71.4
56.2
57.6
1123+ +2048
35.5
20.9
64.3
23.4
71.6
46.8
59.5
65.5
1560.0
88.0
67.0
69.8
56.1
57.4
10242
35.4
19.4
60.5
23.6
70.4
46.0
57.7
64.4
1500.8
88.4
66.7
69.3
55.7
56.3
1536
34.4
17.7
59.1
23.4
69.5
49.3
59.1
64.6
1502.7
88.3
65.8
69.0
$4.9
56.2
2
WHAT IS GOOD REPRESENTATION TO LEARN?
In this work, build our fine-to-coars. supervision methodology the SigLIP 2 encoder (Tschan- nen et al., 2025), or more specifically, its So400m/16-512px version. This model of 429M parameters takes 51222px px image as its input, and produces 322 visual tokens under patch size 16. As compared in Table 1, its So400m/ 16-5mmmm6111pxx version much better than other counterparts in important OCR scenarios (Mathew al., 2021; Singh et al., 2019), due its larger resolution. So mainly use this version for validation. Furthermore, to be more convincing, we also provide our results built on the legacy model OpenAI-CLIP (Radford al., 2021) (Table 11).
Prior to fine-to-coarse supervision, a primary step is to find out what optimal fine-grained features to learn. Therefore, this section presents necessary pilot investigations.
Which image scales should be used to obtain fine-grained features? Common practices (Liu et al., 2024b; Chen et al., 2024b) suggest that it beneficial to incorporate fine-grained but local features from high-resolution images with coarse but global features from base-scale images (i.e., thumbnails). We feed pre-trained SigLIP 2-5 2-512px with single-scale image or multi-scale images: 5122 scale, 2) 5122 1 scales, 3) 5122+ 0222+0042-44 10242+1536' scales, 4) 5122+1024" 15362 scales, 5) 10242 scale, and 6) 1536 scale. By default, high-resolution images, we split them into non-overlapping base-scale images for inference. Then, interpolate the re-assembled high-resolution features to the same shape as base-scale features (322 tokens). Finally, we average multi-scale features (if any) to ensure the same number of visual tokens across all settings.
As shown in Table 2, even with the same number of tokens, introducing high-resolution images (e.g., 512 + 512"+10244 facilitates most benchmarks (10 out of 12), especially for OCR and chart tasks. But such improvement saturates or disappears when further adding the 4x (2048²) scale, showcasing is suboptimal to blindly scale up the resolution. The single high-resolution view is generally inferior
3

---

Published as a conference paper at ICLR 2026
Table 3: Comparison among different inference strategies for high-resolution images (1024)). PE
interpolation: interpolate the pre-trained positional embeddings to the targeted resolution.
OCR & Chart
General
Knowledge
MMBench
DoeVQA
CharQQ
TesTQA
InfoVQA
TexCCaps
HRBench
Infer high-res image
RWQA
GQA
MME
POPE
SQA'
A12D
Avg
Sliding (wo overlap)
37.0
21.2
64.2
23.8
71.0
47.9
59.6
64.9
1560.1
88.0
67.4
71.0
56.2
57.7
Sliding (w/ half' overlap)
34.8
21.4
63.2
23.3
71.3
43.9
56.2
65.0
1535.3
88.2
68.4
70.3
56.4
56.9
Entire (PE interpolation)
33.5
21.2
63.2
22.7
71.8
41.5
58.6
64.6
1548.2
87.9
67.2
69.8
57.3
56.7
Table 4: Comparison among different fusion strategies for multi-scale features from 5122+1024 scales. We first employ interpolation or pixel unshuffling (Shi et al., 2016) to downsample high- resolution features, and then average or concatenate (channel-wise) them with base-scale features.
OCR & Chart
General
Knowledge
DoeVQA
CharqQ
TexIQQ
InfoVQA
Tex:Caps
HRBench
MMBench
Fuse multi-scale features
RWQA
GQA
MMEE
POPE
SQA'
A12D
Avg
Interpolate + average
37.0
21.2
64.2
23,8
71.0
47.9
59.6
64,9
1560.1
88.0
67.4
71.0
56.2
57.7
Interpolate concat
34.8
21.2
63.2
22.9
71.3
44.3
57.1
64.5
1555.5
88.2
68.3
70.9
$7.3
57.1
Pixel unshuffle + concat
35.9
20.4
63.8
22.9
70.9
42.1
59.5
64.5
1542.4
88.1
66.5
69.9
56.4
56.8
to multi-scale views, highlighting the necessity of maintaining global view. Lastly, we can observe some benchmarks, e-g., MMBench (Liu et al., 2024c), even do not prefer fine-grained features.
How to infer model on high-resolution images? To apply model trained on base scale (e.g., 512px) to higher-resolution input, existing works (Liu et al., 2024b; Chen et al., 2024b) typically slide a base-scale window across the entire image with non-overlapping stride. However, alternative strategies exist. For example, following practices in dense prediction tasks (Cordts et al., 2016), using overlapping sliding windows can enhance local feature coherence and reduce boundary artifacts. Besides, we can directly interpolate the pre-traine rreitraieed positional embeddings to match the high-resolution input (Bai et al., 2023), though this may introduce distortion in fine-gras fineggrained spatial relationships.
We compare these inference approaches in Table 3. As expected, interpolating positional embeddings underperforms the basic sliding window strategy. Surprisingly, overlapping windows also degrade performance, contrary to lessons from dense prediction tasks. While the exact cause remains unclear, we hypothesize two potential factors: 1) inconsistent token distributions: overlapping regions receive ensemble treatment while others do not, or 2) positional embedding conflicts for tokens appearing in multiple windows. Finally, the best practice still using non-overlapping sliding window.
How to fuse features from multiple resolutions? For multi-scale features, some works (Shi et al., 2024: Liu et al., 2025) first interpolate high-resolution features the same scale base-resolution features and then concatenate them along the channel dimension. Other than concatenation, we can simply average them. Besides, pixel unshuffle (Shi al., 2016) another choice (Chen et al., 2024a) to downsample features.
As compared in Table 4, the simplest "interpolate + average" practice stands out as the best. Beyond delivering better performance, this strategy produces ensembled features with the same dimension as base-scale features, eliminating the need for a projection head (Ranzinger al., 2024) later learning. Indeed, there are more sophisticated fusion options, e.g., using weighted average instead of a simple mean. We leave these investigations to our final main experiments (Table 10).
Summary. Our pilot studies demonstrate that the optimal multi-scale configuration is two three scales (5122+1024 or 1022++0244+) For nigh-resolution inference, non non-overlapping sliding window yields the best results. Finally, high-resolution features should be interpolated and then averaged with the base-scale features to produce the final features. Based on these findings, we will next present our fine-to-coarse supervision mechanism.
4

---

Published as a conference paper at ICLR 2026
1024px
Re-assemble
SigLIP 2
Local
Fine
Interpolate
initialize
Global Info
Average
Globol
Coarse Detail
512px
Frozen
SigLIP-HD
Supervision
Trainable
Figure 2: Overview of our fine-to-coarse supervision framework for training our SigLIP-HD, The frozen pre-trained SigLIP encoder is inferred on multi-scale images to produce high-quality fine- grained features. Our SigLIP-HD is trained to mimic the features at a standard resolution (512²).
SIGLIP-HD
Our core methodology is fine-to-coarse supervision. As illustrated in Figure 2, our framework is very simple. It comprises two branches, one inference branch to produce high-quality visual features, and one trainable branch to learn towards the better features.
Inference branch. As studied in our pilot experiments (Table 2), multi-scale input is better than single-scale input. can harvest both advantages of high-resolution images (fine-grained details) and base-resolution images (global information), while suppressing their weaknesses (local information and coarse details). Therefore, two scales: base scale (512²) and 2x scale (1024²). We do not introduce more scales for inference, because we empirically find that more scales fail consistently yield stronger results (see Table 9). In addition, too many scales significantly increase the computational burden, e.g., requiring nine more feedforward passes adding 3x scale.
feed our pre-trained SigL 2-So400m/ 1044000mm165112/0 encoder with
The pre-trained SigLIP 2 model is frozen and it is inferred on the multi-scale input images by non- overlapping sliding window if needed (supported by Table 3). We obtain a base-resolution feature F E RRCH xW and high-resolution feature ph E RC22H x2W from the two scales, respectively. To fuse the feature maps of different scales, we first downsample 4 to the same size as Eo via bilinear interpolation. We then produce the final high-qu -uialiuuuuyyyy feature F* by directly averaging Eob and the downsampled fob, This simple strategy is more effective than other counterparts, such as pixel unshuffle and concatenation, validated in Table 4.
Trainable branch. Our SigLIP-HD in the trainable branch is initialized with the pre-traines SigLIP parameters. It shares exactly the same architecture and input/output as the pre- pre-trained model, without adding any projection modules (Ranzinger et al., 2024). takes a base-scale image 5122 pixels input and produces 322 tokens. We denote its feature map as Fs. The optimization target of this branch is enforcing 1 to align with the better feature 4r at the patch level.
There are various options in the loss function for feature alignment, such as cosine similarity loss (Fang et al., 2023), or a combination of smooth LI and cosine similarity loss (Ranzinger et al., 2024). In practice, we observe the strictest and simplest L1 loss performs the best, compared in Table
Application scope. Although our method can polish the visual representation at standard resolution, we do not anticipate will entirely bypass the native-resolution practice in MLLMs. Some visual details may be completely lost after down-sampling. Fortunately, our method is fully compatible with existing practices, regardless of using down-sampled resolution (Table 5) or operating on native resolution (Table 6), as they both rely on pre-trained vision encoder.
4 EXPERIMENT
Implementation details. We train fine-to-coarse supervision framework the 4.5M raw images from the Cambrian-1 collected data (Tong et al., 2024a), since they cover diverse scenarios, including natural images, scene text images, documents, We train our SigLIP-HD with AdamW
5

---

Published as a conference paper at ICLR 2026
Table 5: Comparison between our SigLIP-HD and the most capable SigI.IP 2-So400m/16-512pp encoder (Tschannen et al., 2025). We follow the two-stage training pipeline of LLaVA (Liu et al., 2024a) with Vicuna-1.5-7B (Zheng et al., 2023). We try to freeze (V) or unfieeze (X) the vision encoder to provide better insights into our advantages.
OCR & Chart
General
Knowledge
Doc
CharQA
TexVQA
InfoVQA
Tex:Caps
HRBench
MMBench
SFT data
Freeze Encoder
RWQA
GQA
MME?
POPE
SQA'
A12D
Avg
2igLIP
32.2
19.3
61.0 22.1
71.1
57.5
62.8
1529.2
87.8
67.7
69.9
56.0
55.8
LLaVA-1.5
SigLIP-IID
34.7
20.2 63.1
23.0
71.6 46.2
59.5
64.3
1554.0
88.1
67.7
71.5
58.0
57.4
SigL.P
32.8
20.8
62.3
22.1 71.9
42.9
56.5
63.4
1479.0
88.3
67.6
70.6
56.3
56.1
LLaVA-1.5
x
SiDLIP-HD
35.2
21.8
63.9
23.2
72.0
43,9
58.8
64.1
1550.1
88.8
67.0
70.4
57.3
57.2
SigLIP
54.2
58.2
64.7
23.5
67.2
(43.3
58.0
63.4
1490.8
87.8
68.2
70.2
68.3
61.7
LLaVA-NeXT
SiDLIP-HD
55.2
60.9
64,9 25.4 67.6 46.2
59.8 64.4
1536.7
88.1
68.6
72.6
68.7
63.0
SigLIP 2 56.0
61.6
65.8
23.2
67.5
43.5 60.8
63.8
1532.
88.1
68.4
71.8
69.2
62.8
LLaVA-NeXT X
SD 59.6 65.2 65.7 25.5 68.1 48.3 59.2 64.4
1558.3
88.6
70.3
74.4
70.5
64.4
Table 6: Evaluation under the AnyRes (Li et al., 2025a) strategy (i.e., operating on native resolution). Here we use VLaVA- data for visual instruction tuning and unfieeze the vision encoder.
Encoder
DocVQA
CharQA
TextVQA
InfoVQA
TextCaps
GQA
MME"
POPE
SQA!
A12D
Avg
SigLIP 2
67.6
63.9
66.9
27.2
65.6
61.1
1431.6
87.4
70.7
65.8
64.88
SigLIP-HD
69.7
67.4
68.4
27.7
65.8
61.9
1452.6
87.9
72.3
69.3
66.3
optimizer (Loshchilov & Hutter, 2019), with an initial learning rate of 5e-5 and weight decay of 0.04.
The model is trained for 90K iterations with a total batch size of 512. We use the cosine learning rate scheduler with warm-up period of 4K iterations. We exactly follow the image pre-processing pipeline of SigLIP 2 (Tschannen et al., 2025), except when producing the high-resolution image (size adopt the strict LI loss to optimize our features. It takes only 34 hours on 32 A A100 GPUs with BFloat 6 training.
changed from 512 to 1024). Finally, aforementioned,
We evaluate on diverse MILM datasets (Zhang al., 2024): including Doc (Mathew al., 2021), ChartQA (Masry et al., 2022), TextVQA (Singh et al., 2019), InfoVQA (Mathew et al., 2022), TexICaps (Sidorov al., 2020), HRBench (Wang et al., 2025), RealWoridQA (x.ai, 2024), GQA (Hudson & Manning, 2019), MME Perception (Fu et al., 2025), POPE (Li et al., 2023), MMBench (Liu al., 2024c), ScienceQA-IMe (Lu et al., 2022), and A12D (Kembhavi et al., 2016).
4.1 COMPARISON WITH SIGLIP 2
Our framework is built the highly capable SigLIP 2 model. Through fine-to-coarse supervision, we aim to further enhance its perception capability in MLLMs. Therefore, we systematically compare our SigLIP-HD encoder with the original SigLIP 2-S04000/116-1 12px encoder. By default, we adopt the two-stage training pipeline of LLaVA (Liu et al., 2024a). The input image is resized to 512" pixels and encoded into 32 visual tokens to be sent into the LLM.
As compared in Table 5, with Vicuma-1.5 1.5-7B (Zheng et al., 2023) as the LLM, we try different supervised fine-tuning (SFT) data, including 1L. (Liu al., 2024a) and LLaVA-NeXT (Liu et al., 2024b) that contains more OCR-related data. We also attempt different training configurations, including freezing or unfreezing the vision encoder during the fine-tuning stage. Across all settings, SigLIP-HH HD consistently outperforms our SigLIP 2 bascline. Notably, on OCR-related benchmarks, such as DQA and CharlQA, we improve Sig 2 from 56.0 + 59.6 (+3.6) and from 61.6 -> 65.2 (+3.6), respectively. On general VQA benchmarks that require fine-grained perception, e-g., HRBench, our Li outperforms its baseline by +4.8 (43.5 - 48.3). Qualitative comparisons provided in Figure 3. Our SigLIP-HD exhibits better capability in perceiving fine-grained content.
Using Any Res. Although we primarily focus on unleashing the perception capability at a standard (medium) resolution, our encoder is indeed compatible with the AnyRes (Li et al., 2025a) strategy

---

Published as a conference paper at ICLR 2026
Table 7: Comparison between our SigLIP-HD and SigI.IP 2 under more LLMss Here we use the LLa VA-NeTT data for visual instruction tuning and freeze the vision encoder.
LLM
Encoder
DocVQA
ChariQA
TexiVQA
InfoVQA
TexiCaps
GQA
POPE
SQA'
Avg
SigLIP
47.3
45.2
59.2
22.2
66.1
59.2
88.3
70.4
$7.2
Llama-3.2-3B
SigLIP-HD
49,9
49.8
60.8
23,9
60.2
88.5
71.4
58.9
SigLIP
62.5
66.1
66.1
30.6
68.4
64.0
88.3
79.6
65.7
Qwen2.5-7B
SigL.I-HDD
64.2
66.8
66.3
30.8
68.7
64.2
88.6
79.9
66.2
Table 8: Ablation study on the type of feature alignment loss. We adopt L.1 loss for its better results.
Feature alignment loss
Doc
ChartQA
TexiVQA
InfoVQA
TextCaps
GQA
POPE
SQA'
Avg
Cosine similarity (Fang al,, 2023)
33.8
20.2
62.5
23.8
71.3
64.4
88.0
70.1
54.3
Cosine sim smooth LI (Ranzinger et al., 2024)
34.1
20.9
62,9
23.2
70.6
63.8
87.8
70.5
54.2
34.7
20.2
63.1
71.6
88.1
71.5
54.6
Table 9: Ablation study on the image scale used to supervise our SigLIP-HD encoder. This is different from the experiment in Table 2, as we here study the scales directly under our training framework.
Image scale
DocVQA
ChartQA
TexVQQ
InfoVQA
TextCaps
GQA
POPE
SQA'
Avg
1 scale (1024)) scales (512'+10249)
27.8
18.6
51.7
21.4
68.9
62.2
88.0
67.3
50.7
34.7
20.2
63.1
23.0
71.6
64.3
88.1
71.5
54.6
3 scales (512++104"+15336
33.4
19.4
62.7
22.9
70.8
64.0
88.2
70.6
54.0
(i.e., operating on native resolution). We report the results in Table 6. We choose the closest resolution from 512 x {[1 x ..,..,, (3 x 3] The max tokens are set as 322 x4. While the baseline method has been significantly enhanced by such an AnyRes strategy, our SigLIP-HD can further outperform it by aN even larger margin on OCR tasks, egg., improving ChartQA from 63.9 to 67.4 (43.5).
More LLMs. In addition to the widely used Vicuna-1.5 (Zheng et al., 2023), which is fine-tuned from Llama 2 (Touvron et al., 2023), we further extend our vision encoder to other LLMs. As shown in Table 7, with smaller -lama-3.2-3B LLM (Grattafiori al., 2024), our SigLIP-HD still showcases clear advantages. It surpasses SigLIP 2 by +4.6 (45.2 - 49.8) on ChariQA, +2.6 (47.3 49.9) on Doc VQA, +1.6 (59.2 + 60.8) on TextVQA. Beyond the Llama series, with the latest Qwen2.5-7B (Yang et al., 2024) the LLM, our SigLIP-HD is also consistently superior to our baseline encoder, e.g., boosting from 62.5 + 64.2 (+1.7) on DOA
4.2 ABLATION STUDY
We the LLa VA-1.5 data (Liu al., 2024a) for instruction tuning and freeze the vision encoder.
Feature alignment loss. Existing works have provided several candidate choices on the loss function used for feature alignment, such the cosine similarity loss adopted by EVA (Fang et al., 2023) and the cosine similarity + smooth L1 loss adopted by AM-RADIO (Ranzinger et al., 2024). In Table 8, we compare these losses, well the plain LI loss. These three losses deliver very close results on average, all surpassing our SigLIP 2 baseline. Nevertheless, the simplest and strictest LI loss is slightly better than the other two losses. So we choose it as our final feature alignment loss.
Image scale. From our pilot studies (Table 2), we can conclude that feeding multi-scale images of two scales or three scales the MILM performs the best. Here, we further examine the final performance of our SigLIP-HD encoder when trained under different configurations of multiple scales. As shown in Table 9, similar to our observations in pilot studies, the two-scale configuration (one base- scale 5 5122px image one high-resolution 10242px px image) achieves the best results, providing the most suitable features for our SigLIP-HD to learn. We believe there may be more sophisticated and better configurations, e.g., searching for the optimal fusion weight for the three-scale features, but they are out of the scope of this paper. We aim to provide a neat, universal, and efficient training framework without too many hand-crafter hyper-pane parameters.
four-scale
Multi-scale fusion weight. We find the simplest "interpolate + average" practice delivers better results than pixel unshuffle and channel-wise concatenation in pilot studies (Table 4). Here
7

---

Published as a conference paper at ICLR 2026
Erien Ohakon
Jenvng
User:
Who is et the center of of this?
SigLIP
SigLIP-DD: Bryan Owens
HOKA TIME TO FLY
User:
nome the fession wearfclothing advertie?? SigLIP Wills
User: What
bonner?
Dealer?
SigLIP 2: HOOKA TIME TO FLY
SigLIP 2: C. Mook
SigLIP-DD: HOKA TIME FLY
Monk
SigLIP-HD: lifestyle
Figure 3: Qualitative comparison between our SigLIP-HD and SigLIP when applied in MLLMs.
Table 10: Ablation study the fusion weight for averaging base-scale and high-resolution features.
Base:high
Voc
ChariQA
TextVQA
InfoVQA
TextCaps
RWQA
GQA
MME"
POPE
SQA'
Avg
1:1
34.7
20.2
63.1
23.0
71.6
59,5
64.3
1554.0
88.1
71.5
57.4
1:2
32.6
19.8
61.5
22.0
71.3
57.1
62.7
1513.4
88.0
68.8
55.9
2:1
33.6
19.5
61.3
23.5
70.9
57.5
63.9
1544.0
87.9
69.2
56.5
Table 11: Shifting our baseline model from SigLIP (Tschannen et al., 2025) to OpenAI-CLIP (Rad- ford et al., 2021). For fair comparison, we feed CLIP with multi-scale inputs and keep the same number of visual tokens our CLIP-HD.
Encoder
Resolution Tokens
Doc ChartQA
TextVQA InfoVQA TextCaps
GQA
MME
POPE
SQA'
Avg
CLIP 336²
24
22.4
17.5
46.8
20.8
69.0
63.0
1520.9
86.9
70.2
52.5
CLIP
336-+6723
37
31.1
18.1
52.0
22.3
69.5
63.0
1510.7
87.3
70.0
54.3
CLIP-HD 518"
37
33.2
20.4
53.0
21.5
70.2
63.3
1519.6
87.8
70.8
55.1
further validate whether mean average is better than weighted average. As compared in Table 10, we attempt to increase the weight of high-resolution features the weight of base-scale features. As result, neither of them outperforms the default mean average strategy. Slightly to our surprise, even on benchmarks that require fine-grained details, e.g., DocVQA, increasing the weight of high-resolution features still deteriorates the final result. This further proves the indispensable role of base-scale images capturing the global content.
4.3 COMPARISON WITH LEGACY MODEL OPENAI-CLIP
Until now, have provided comprehensive results to demonstrate the superiority of our SigLIP-HH over SigLIP 2. To be more convincing, we here further validate our fine-to-coarse supervision (Radford et al., 2021). However, as illustrated in Figure 1, it impossible even for humans to recognize detailed image content under the 336px low resolution. Therefore, our model is trained 5182 pixels (372 visual tokens) by interpolating the pre- treitra positional embeddings. The high-quality features used for learning are produced under three image scales +662++677+1008 We interpolate the three-scale features the same shape (372) and then average them to supervise our CLIP-HD.
methodology on the legacy model OpenAL-CLLPPLL1443331
To ensure a fair comparison due our larger resolution and more visual tokens, when using the pre-trained CLIP in MILMMs, we feed a multi-scale input (336 +3762) following LLa (Liu et al., 2024b). The obtained two-scale features are both interpolated to the same size as ours and then averaged to be sent into LLM. As compared in Table 11, although multi-scale inputs and increased visual tokens have significantly boosted the results on OCR-related benchmarks, our CLIP-HD can further enhance the performance at the same cost, e.g., 22.4 + 31.1 + 33.2 DocVQA. Our superior results based on the legacy model CLIP and the latest SOTA model SigLIP 2 demonstrate the universality of our proposed fine-to-coarse supervision mechanism.
8

---

Published as a conference paper at ICLR 2026
5 RELATED WORK
Visual representation learning. Learning robust visual representation has been a fundamental goal in computer vision for decades. The rise of deep learning revolutionized the field (Lowe, 2004), starting with supervised learning (Krizhevsky et al., 2012). But human labels are inherently biased (Yun et al., 2021), not enough to produce transferable features. Therefore, CLIPs (Radford et al., 2021) leverage web captions to learn more informative representations. However, such alt-text data is often noisy, prompting continued interest in vision-centric self-super learning (Caron al., 2021). This line of work, popularized by contrastive learning (Chen et al., 2020) and masked image modeling (He et al., 2022), remains highly active. Thanks to large-scale curated data (Xu et al., 2024) and hybrid supervision signals (Oquab et al., 2024), latest works (Tschannen et al., 2025; Bolya et al., 2025) demonstrate the potential of a universal representation for diverse downstream tasks.
This work does not aim to propose new pre-training strategy, but to further enhance the capability of a pre-trained encoder with a simple and efficient framework.
Multi-modality large language model (MLLM). There are two mainstream roadmaps to deal with visual inputs: encoder- based and encoder-free free. Encoder-basse MLLMs (Liu et al., 2024a; Chen et al., 2024b; Li et al., 2025b) use a pre-trained vision encoder (Radford et al., 2021; Zhai et al., 2023) to extract visual tokens for LLMs. They can leverage rich pre-trained visual knowledge. But they are not unified, and the visual inputs are constrained by the pre-trained resolution. To bypass these limitations, encoder-free MLLMs (Bavishi et al., 2023; Diao et al., 2024; 2025; Lei et al., 2025) are attracting growing research interest. Unfortunately, until now, they require more training budget and still lag behind encoder-based models. Therefore, our work on seeking better visual representations for MLLMs is still of high value.
Visual representation in MLLMs. There are broadly three approaches to enhancing visual repre- sentation in MILMs. The first involves directly pre-training more powerful vision encoders (Fan et al., 2025; Bolya et al., 2025). Despite the success, their high training costs remain prohibitive for most researchers. The second approach combines multiple pre-trained encoders (Tong et al., 2024b), aiming for mutual gains. In practice, unfortunately, they rarely yield substantial gains over a single well-optimized encoder (Shi et al., 2025b). The third and most prevalent strategy is scaling up image resolution (Chen et al., 2024a; Bai et al., 2025). Early works resize images fixed larger size (Liu et al., 2024a), but recent techniques preserve native resolutions (Bai et al., 2025). Beyond global resizing, local zoom-in methods (Zhang et al., 2025; Qian et al., 2025) can also amplify details.
In contrast to this scaling trend, our work takes a step back, demonstrating that even at a standard resolution, fin-grained perception can be achieved effectively.
Knowledge distillation. Our work shares core spirit with knowledge distillation (Hinton et al., 2015), in that both learn from better "teacher" tokens. However, we do not rely on any external knowledge of an auxiliary model. Instead, we unleash the inherent potential of the current model. We eliminate the need to align multiple models into a shared space (Ranzinger al., 2024; Heinrich et al., 2025). CLIPSelf (Wu et al., 2024a) similarly refines CLIP features. But it uses region-level coarse supervision and targets the open- ppen-vocabllary dense prediction, while we adopt patch- pate fine-grained supervision and address the key challenge of MLLMs. Lastly, our work is slightly related to a recent work (He et al., 2025). But we address fundamentally different problems (MLLM fine-grained perception vs. depth estimation) and operate on different spaces (feature vs. label).
While knowledge distillation or teacher-student training well-established techniques, what to distill is critical for effective learning. Our key insight lies in presenting multi-resolution to standard-resolute
distillation framework that enhances representation quality without increasing inference cost. This is orthogonal to existing multi-teacher distillation works (Ranzinger et al., 2024; Heinrich et al., 2025). We demonstrate that multi-resolution distillation can serve as an effective, lightweight post-training stage for enhancing fineggrained understanding without architectural changes or inference overhead. It can be a practical solution for resource-constrained deployment scenarios.
6 CONCLUSION
In this work, we reflect on the scaling trend of image resolution in MLLMs. Motivated by the amazing capability of the human visual system, investigate how to enhance the perception capability of

---

Published as a conference paper at ICLR 2026
AI systems standard (medium) resolution, without using larger images. We present highly simple yet effective fine-to-coarse supervision mechanism to address this. Features the base-scale image are enforced to mimic the high-quality ensembled features of multi-scale images. Built on the latest Sig encoder, our fine-tuned SigLIP-HD encoder delivers stronger results across extensive MLLMM benchmarks under various training protocols, especially for OCR-related tasks.
Acknowledgment. This work is supported by the National Natural Science Foundation of China (No. 62422606, 62441615) and Hong Kong Research Grant Council General Research Fund (No. 17213925).
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv:2303.08774, 2023. 1
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. In NeurIPS, 2022. 2
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv:2308. /2966, 2023. 4
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv:2502.13393, 2025, 1, 2.
Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, and Sagnak Taşırlar. Introducing our multimodal models, 2023. URL httpp::////.wttt wttp://wwwwwpp..././/////////t.t ai/
blog/ Euyu-8b. 8b. 9
Daniel Bolya, Po- Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, Junke Wang, Marco Monteiro, Hu Xu, Shiyu Dong, Nikhila Ravi, Daniel Li, Piotr Dollár, and Christoph Feichtenhofer. Perception encoder: The best visual embeddings are not at the output of the network. In NeurlPS, 2025. 1,9
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. ICCV, 2021. 9
Hong-Yo You Chen, Zhengfeng Lai, Haotian Zhang, Xinze Wang, Marcin Eichner, Keen You, Meng Cao, Bowen Zhang, Yinfei Yang, and Zhe Gan. Contrastive localized language-image pre-training. In ICML, 2025.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. ICML, 2020. 9
Zhe Chen, Weiyun Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao. Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv:2412.05227, 2024a. 4,
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing. Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In CVPR, 2024b. 2, 3, 4,9
Hyungyu Choi, Young Kyun Jang, and Chanho Eom. Goal: Global-local object alignment learning. In CVPR, 2025. 2
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016. 4
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Moham- madreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models. In CVPR, 2025.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Haiwen Diao, Yufeng Cui, Xiaotong Li, Yueze Wang, Huchuan Lu, and Xinlong Wang. Unveiling encoder-free vasion-language models. In NeurIPS, 2024. 9
Haiwen Diao, Xiaotong Li, Yufeng Cui, Yueze Wang. Haoge Deng, Ting Pan, Wenxuan Wang, Huchuan Lu, and Xinlong Wang. Evev2: Improved baselines for encoder-free esion-language models. In ICCV, 2025.
11

---

Published as a conference paper at ICLR 2026
David Fan, Shengbang Tong, Jiachen Zhu, Koustuv Sinha, Zhuang Liu, Xinlei Chen, Michael Rabbat, Nicolas Ballas, Yann LeCun, Amir Bar, and Saining Xie. Scaling language-fre free visual representation learning. In ICCV, 2025. 1,9
Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang. and Yue Cao. Eva: Exploring the limits of masked visual representation learning at scale. In CVPR, 2023. 7,
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang. Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models, 2024. In NeurlPS, 2025. 6
Stephanie Fu, Mark Hamilton, Laura Brandt, Axel Feldman, Zhoutong Zhang, and William T Freeman. Featup: A model-agnostic framework for features at any resolution. In ICLR, 2024. 2
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama 3 herd of models. arXiv:2407.2178. 2024. 7
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. autoencoders are scalable vision learners. In CVPR, 2022. 1,9
Masked
Xiankang He, Dongyan Guo, Hongji Li, Ruibo Li, Ying Cui, and Chi Zhang. Distill any depth: /9204, Distillation creates a stronger monocular depth estimator. arXiv:2502. 29ii:99442.9244, 2025. 9
Greg Heinrich, Mike Ranzinger, Yao Lu, Jan Kautz, Andrew Tao, Bryan Catanzaro, Pavlo Molchanov, et al. Radio amplified: Improved baselines for agglomerative vision foundation models. In CVPR, 2025. 9
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. 150300350,.,233,,, 2015. 9
Haiwen Huang. Anpei Chen, Volodymyr Havrylov, Andreas Geiger, and Dan Zhang. Loftup: Learning a coordinate- basddd feature upsampler for vision foundation models. In ICCV, 2025. 2
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR, 2019.
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi.
A diagram is worth a dozen images. In ECCV, 2016.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. In NeurlPS, 2012. 9
Weixian Lei, Jiacong Wang, Haochen Wang, Xiangtai Li, Jun Hao Liew, Jiashi Feng, and Zilong Huang. The scalability of simplicity: Empirical analysis of vision-lane asionnlanguage learning with a single transformer. In ICCV, 2025. 9
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. alava-onevisson: Easy visual task transfer. TMLR, 2025a. 1, 2,
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao and Ji- Ji-Rong Wen. Evaluating object hallucination in large languaee models. In EMNLP, 2023. 6
Zhang Li, Biao Yang. Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xiang Bai. Monkey: Image resolution and text label are important things for large multi-modal models. In CVPR, 2024.
Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, al. Eagle 2: Building post-training data strategies from scratch for frontier -aiguag models. arXiv:2501. 14818, 2025b. 9
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 1
12

---

Published as a conference paper at ICLR 2026
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurlPS, 2023. 1,2
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, 2024a. 1, 2, 3, 6, 7.9
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024b. URL https: //llava-vl gi github........ io ho/bblog//224-- 10022--01-3-11aaa 1ea---lava-eeetttt next/ 1, 4, 6, 8
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In ECCV, 2024c. 4, 6
Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang. Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. Nvila: Efficient frontier visual language models. In
CVPR, 2025. 2,
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 6
David G Lowe. Distinctive image features from scale-invariant keypoints. 1/CV, 2004.
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Chgu-Chn Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan, Learn explain: Multimodal reasoning via thought chains for science question answering. In NeurlPS, 2022. 6
Kevis- Kokis-K Maninis, Kaifeng Chen, Soham Ghosh, Arjun Karpur, Koert Chen, Ye Xia, Bingyi Cao, Daniel Salz, Guangxing Han, Jan Dlabal, et al. Tips: Text-image pretraining with spatial awareness, In ICLR, 2025.
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: benchmark for question answering about charts with visual and logical reasoning. In ACL, 2022. 6
Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: dataset for vqa on document images. In WACV, 2021. 3,6
Minesh Mathew, Viraj Bagal, Ruben Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar.
Infographievga. In WACV, 2022. 6
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. TMLR, 2024. 1, 9.
Jiaxu Qian, Chendong Wang, Yifan Yang, Chaoyun Zhang, Huiqiang Jiang, Xufang Luo, Yu Kang, Qingwei Lin, Anlan Zhang, Shiqi Jiang, et al. Zoomer: Enhancing milm performance with adaptive image focus optimization. MXXv:2505.00042. 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021. 8,9,9
Mike Ranzinger, Greg Heinrich, Jan Kautz, and Pavlo Molchanov. Am-radio: Agglomerative vision foundation model reduce all domains into one. In CVPR, 2024. 7,
Baifeng Shi, Ziyang Wu, Maolin Mao, Xin Wang, and Trevor Darrell. When do we not need larger vision models? In ECCV, 2024. 4
Baifeng Shi, Boyi Li, Han Cai, Yao Lu, Sifei Liu, Marco Pavone, Jan Kautz, Song Han, Trevor Darrell, Pavlo Molchanov, et al. Scaling vision pre-training to 4k resolution. In CVPR, 2025a. 2
Min Shi, Fuxiao Liu, Shihao Wang, Shijia Liao, Subhashree Radhakrishnan, De-An Huang, Hongxu Yin. Karan Sapra, Yaser Yacoob, Humphrey Shi, et al. Eagle: Exploring the design space for multimodal umms with mixture of encoders. In ICLR, 2025b. 1,9
13

---

Published as a conference paper at ICLR 2026
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR, 2016. 4
Oleksii Sidorov, Ronghang Hu, Marcus Rohrbach, and Amanpreet Singh. Textcaps: a dataset for image captioning with reading comprehension. In ECCV, 2020. 6
Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, and Maxime Oquab. Cijo jose, vasil khalidov, mare szafraniec, seungeun yi, michaël ramamonjisoa, al. dinov3. arXiv:2508. 10104, 2025. 1
Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In CVPR, 2019. 3, 6
Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. In NeurlPS, 2024a. 1,5
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal 1lms. In CVPR, 2024b. 1,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.099 09288, 2023.
Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdul- mohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv:2502. 2025. 1, 3, 8,
Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. Divide, conquer and combine: A training- free framework for high-resolution image perception in multimodal large language models. In AAAI, 2025.
Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Xiangtai Li, Wentao Liu, and Chen Change Loy. Clipself: Vision transformer distills itself for open-vocabulary dense prediction. In ICLR, 2024a.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non-parametric instance discrimination. In CVPR, 2018.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, al. Deepseek-v12: -ofeerr-oo-experts vision-language models for advanced multimodal understanding. arXiv:2412.10302, 2024b. 1
x.ai. Grok-1.5 vision preview, April 2024. URL http:pp////..a////////oooo---
http//.ooooooooo 1tt 6
Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Yao Huang, Russell Howes, Vasu Sharma, Shang- Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Demystifying clip data. In ICLR, 2024.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. 15Xiv:24/2.15i15, 2024.
Sihyun Yu. Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang. Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is casier than you think. ICLR, 2025.
Sangdoo Yun, Seong Joon Oh, Byeongho Heo, Dongyoon Han, Junsuk Choe, and Sanghyuk Chun. Re-labeling imagenet: from single to multi-labels, from global to localized labels. In CVPR, 2021.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, 2023. 9
14

---

Published as a conference paper at ICLR 2026
Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. Milms know where to look: Training- free perception of small visual details with multimodal llms. In ICLR, 2025. 3, 9
Kaichen Zhang, Bo Li. Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang. Chunyuan Li, al. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv:2407.1277, 2024.
Lianmin Zheng, Wii Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging Ilm-as-aa as-jugee with mt-bench and chatbot arena. In NeurIPS, 2023. 6, 7
15


Reviews:
[{"review_id": "ujVvwkeviy", "reviewer": "Reviewer_2LRA", "summary": "This paper proposes to distill high-resolution features of SigLIP2 to its low-resolution encoding results to improve the performance under a limited encoding budget. The idea is intuitive and sound, and the experiments show improvements under reasonable training computation. The major concern is that the findings from Section 2 are relatively trivial, making the paper lack enough substance.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Distilling high-resolution features to low-resolution ViTs is intuitive and sound. The method is simple and easy to follow.\n2. The experiments show improvements over SigLIP after reasonable computation of training.\n3. The paper is well written.", "weaknesses": "1. It seems Table 1 is only used to explain why the authors chose So400m/16-512px as the default backbone choice of ViT. This is not necessary, considering Table 1 takes much of the space in the main paper. Likewise, it would be better if the pilot experiments in section 2 could present more interesting and insightful results. Also, the multi-scale integration experiments have little to do with the final method.\n2. There are previous works investigating the problem of dealing with high-resolution images using overlapped sliding windows[1]. The authors can refer to this paper for a deeper understanding of why overlapping slices may lead to inferior results.\n\n[1] LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Images. ECCV 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aywIUQvFM8", "reviewer": "Reviewer_vqYK", "summary": "This paper presents SigLIP-HD, a method to enhance the fine-grained perception of Multimodal Large Language Models (MLLMs) without incurring the high computational cost of processing high-resolution images at inference time. The authors challenge the trend of scaling MLLMs by increasing input resolution, arguing that the perception capability of encoders at standard resolutions has not been fully unlocked. The core of the work is a simple \"fine-to-coarse supervision\" framework, a self-distillation mechanism where a \"student\" encoder (SigLIP-HD) is trained at a standard resolution to mimic the richer, \"teacher\" features produced by a frozen version of the same encoder fed with multi-scale inputs. The resulting SigLIP-HD encoder serves as a drop-in replacement for the original, offering significantly improved performance on fine-grained tasks (especially OCR) at the exact same inference budget and architecture.", "questions": "1. The method is a form of feature-based distillation. Why was a simple L1 loss (which performs best in Table 8) superior to a logit-based (e.g., contrastive) distillation, which is standard for training models like CLIP and SigLIP?\n\n2. How critical is the Cambrian-1 dataset to this method's success? Have you experimented with performing this fine-to-coarse supervision using more standard, large-scale web datasets or common VQA datasets?\n\n3. Given the modest gains on general benchmarks, does the fine-tuning cause any significant regressions on specific general vision categories (e.g., object counting, spatial relations) that the original SigLIP 2 was strong in?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper addresses a highly relevant problem in MLLMs. The trend of scaling MLLMs to native, high-resolution inputs creates significant computational and design burdens (e.g., image slicing, multiple forward passes, token compression) . The goal of improving fine-grained perception while maintaining a low, standard-resolution input is practical and valuable.\n\n2. The proposed fine-to-coarse supervision is efficient and effective. It acts as a form of self-distillation that \"distills\" multi-scale knowledge into a standard-resolution encoder. Crucially, it adds no modules or latency at inference time, as it is simply a new set of weights for the original encoder.\n\n3. The authors provide a strong empirical case for their method, which validate the effectiveness and generalizability.", "weaknesses": "1. The core idea is an effective application of knowledge distillation, specifically self-distillation from a multi-scale ensemble \"teacher\" to a single-scale \"student.\" While the application to MLLM encoder resolution is valuable, the underlying mechanism adapts established distillation concepts .\n\n2. The method's improvements are overwhelmingly concentrated on OCR and fine-grained text-heavy benchmarks. The performance on general vision benchmarks is often marginal or flat. This suggests a specialized, rather than a universal, representational improvement.\n\n3. While the method shows strong gains with Vicuna-7B and Llama-3.2-3B, the improvements on other powerful LLMs like Qwen2.5-7B appear marginal. This might suggest the benefits of the distilled representation are less pronounced when combined with certain advanced LLM architectures.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eQGCNDmfSK", "reviewer": "Reviewer_gGd3", "summary": "This paper aims to enhance the perceptual capability of the visual encoder in multimodal large language models (MLLMs). The core idea is to optimize the encoder’s small-scale representations using guidance from its larger-scale representations. During training, an L1 loss is applied to directly align the representations between different scales. Experimental results demonstrate that SigLIP-HD produces more effective visual tokens than its base model under the same input scale.", "questions": "1. In Table 2, would using image scales such as 512² + 1536² or 512² + 2048² yield better representations compared to combining multiple higher scales? Since S² demonstrates that scaling scales can lead to improved results, could a similar setting be applied here for distillation?\n2. The name SigLIP-HD may be somewhat confusing, as the work does not actually increase the input resolution but rather aims to obtain better visual representations at low resolution.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper investigates how different image scales and multi-scale visual feature fusion strategies affect the performance of multimodal large language models (MLLMs).\n2. It introduces a simple method that employs an L1 loss to align single-scale visual features with their multi-scale counterparts, thereby enabling the visual encoder to generate higher-quality single-scale features during inference.\n3. Experimental results demonstrate that the optimized single-scale features significantly improve the performance of both standard-resolution and AnyRes MLLMs across diverse downstream benchmarks.", "weaknesses": "1. The paper requires more experimental evidence to validate its effectiveness. It lacks a comparison with the approach that directly merges multi-scale visual features during inference (i.e., using the teacher-generated features). Such an experiment would help quantify the performance gap between the trained single-scale features and the multi-scale features. In addition, the impact of both approaches on inference time for different MLLM sizes should be reported. Given that the visual encoder is relatively lightweight and multi-scale features can be obtained in parallel within a single forward pass, the additional inference cost may be negligible. This suggests that directly using teacher features at inference time could be a more practical choice.\n2. The novelty of the work is limited. The idea of leveraging multi-scale visual feature fusion to enhance visual representations for MLLMs has been previously explored, for example in S$^2$ [1]. Although the author revisits the impact of image scale, it does not provide new insights. Moreover, a direct experimental comparison with S$^2$ [1] is missing and should be included.\n3. The training cost of retraining the visual encoder in this work is higher than that of existing models such as LLaVA 1.5 and LLaVA-NeXT. Considering that the visual encoder is non-autoregressive and relatively small in size, performing multi-scale fusion directly during inference might be a more efficient and reasonable alternative.\n\n[1] When Do We Not Need Larger Vision Models? ECCV 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JMjoTbDGTd", "reviewer": "Reviewer_rVfE", "summary": "This paper focuses on the high-quality visual feature engineering for MLLMs, and propose a strong vision backbone SigLIP-HD via a fine-to-coarse supervision training regime on SigLIP2. The experiments show that this new backbone can improve MLLMs' performance on most benchmarks without increasing input image resolutions and using more tokens.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The motivation of this paper is reasonable, and the design and training of SigLIP-HD are also supported by sufficient empirical studies. \n\n2. The proposed SigLIP-HD performs well on most benchmarks for two MLLMs, showing its stronger visual representations without increasing the image resolution.", "weaknesses": "1. The method part needs more detailed and clearer descriptions. The description of the method is too brief and colloquial, making it difficult for readers to understand the specific details and procedures. For instance, the distillation between the multi-patch feature maps and the SigLIP-HD. \n\n2. More in-depth insights. The importance of visual features in the field has already reached a consensus. The authors demonstrate that distillation learning with high and low resolutions can improve the quality of visual features, which is reasonable and expected. In addition, the authors should preferably provide more insights and discoveries to strengthen the contribution of the paper. \n\nMinors:\n\n1. What about the results of the distillation with dynamic res. settings? Some recent works show that different MLLMs favors different resolutions of input images. In this case, dynamic resolution may be a better solution for fixed HD images. \n\n2. Missing some relevant works about efficient HD feature engineering, e.g., LLaVA-HD [a]. LLaVA-HD uses two backbones to process low and high resolutions of the input images, which also keeps a small number of visual tokens. \n\n3. The authors all uses tables to report the experimental results, while some of them may be replaced by plot charts to save the paper length. Thus, the authors can give more descriptions to their methods. \n\n[a] Luo, Gen, et al. \"Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models.\" The Thirteenth International Conference on Learning Representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)