InfoScan: Information-Efficient Visual Scanning via Resource-Adaptive Walks
Authors: Yifeng Wu
Huimin Huang
Shangjie Zhou
Yawen Huang
Hao Zheng
Yun Chen
Xian Wu
Ruize Han
Guanhua Chen
Venue: ICLR 2026 Poster (2026)
Keywords: Vision Model; Scan Strategy; Markov Decision Processes; Information Scoring
Primary area: learning theory

Abstract: High-resolution visual representation learning remains challenging due to the quadratic complexity of Vision Transformers and the limitations of existing efficient approaches, where fixed scanning patterns in recent Mamba-based models hinder content-adaptive perception.
To address these limitations, a novel Information-aware Scanning mechanism (InfoScan) tailored for state-space visual backbones is proposed, which dynamically allocates computational resources to the most salient regions of an image. 
Specifically, InfoScan rigorously assesses the informativeness of image patches by integrating entropy with local structural analyses, formulates a joint optimization objective balancing fine-grained detail preservation and broader contextual coherence, and learns an adaptive scanning policy via reinforcement learning. 
Built upon the innovative Visual Information State Space (VISS) block, InfoScan establishes a new family of models that achieve superior efficiency-accuracy trade-offs across diverse tasks. 
Extensive empirical evaluation in different downstream vision tasks demonstrates that our information-driven dynamic scanning paradigm offers a robust and principled alternative to fixed or global-first traversal methods. 
Collectively, our work positions adaptive, content-aware processing as a promising and effective new paradigm for efficient high-resolution visual representation.

Published as a conference paper at ICLR 2026
INFOSCAN: IITORMATIIO-EFFICIIN VISUAL SCAN- NING VIA RESOURCE- ADAPTIVE WALKS
Yifeng Wu Huimin Huang Shangjie Zhou Yawen Huang", Hao Zheng*,
Yun Chen Xian Wu², Ruize Han' Guanhua Chenat
Shenzhen University of Advanced Technology. 2Southern University of Science and Technology Teencet Jarvis Lab, 'Xi'an Jiaotong University, Shanghai University of Finance and Economics
ABSTRACT
High-resolution visual representation learning remains challenging due to the quadratic complexity of Vision Transformers and the limitations of existing ef- ficient approaches, where fixed scanning patterns in recent Mamba-based mod- els hinder content-adaptive perception. To address these limitations, novel Information-aware Scanning mechanism (InfoScan) tailored for state-space visual backbones is proposed, which dynamically allocates computational resources the most salient regions of an image. Specifically, InfoScan rigorously assesses the informativeness of image patches by integrating entropy with local struc- tural analyses, formulates a joint optimization objective balancing fine-grained detail preservation and broader contextual coherence, and learns an adaptive sean- ning policy via reinforcement learning. Built upon the innovative Visual Infor- mation State Space (VISS) block, InfoScan establishes a new family of models that achieve superior efficiency- sive empirical evaluation in different downstream vision tasks demonstrates that information-drivee dynamic scanning paradigm offers robust and princi- pled alternative to fixed or global-first traversal methods. Collectively, our work positions adaptive, content-aware processing a promising and effective new paradigm for efficient high-resolution visual representation. Our code and mod- els are publicly available at https: /gi thub. com/ SIAT- CV- wuy feng/ Croscan-ICLRR00
trade-offs across diverse tasks. Exten-
INTRODUCTION
Visual representation learning, cornerstone of computer vision, aims to extract complex patterns from visual data. Vision Transformers (ViTs) (Dosovitskiy et al., 2021; Vaswani et al., 2017) have become dominant backbone for visual representation learning, achieving widespread success across diverse downstream tasks such as classification, segmentation, and detection. By incorporat- ing self-attention mechanisms, VITs demonstrate superior learning capacity on large-scale datasets. However, their computational cost scales quadratically with the number of input tokens, making them prohibitively expensive when processing high-resolution images-where the number of tokens grows significantly.
To mitigate this issue, extensive research has focused on reducing computational complexity while preserving performance (Dong et al., 2022; Liu et al., 2021). These approaches typically oper- ate through either token sparsification or hierarchical downsampling. Nevertheless, they still face fundamental trade-off: methods that restrict token interaction often limit the effective receptive field, while aggressive downsampling leads to non-negligible performance degradation across di- verse tasks. Consequently, achieving both efficiency and strong representational power remains an open challenge.
Recently, Mamba-based models such VMMmbaa (Liu et al., 2024), RainMamba (Wu et al., 2024), and others (Mchta et al., 2023; Zubić et al., 2024; Zhu et al., 2024b; Gu & Dao, 2024) have
"Equal Contribution.
1 Corresponning authors.

---

Published as a conference paper at ICLR 2026
Clssification
Seg
90
70
FPS thigher is bettar)
90
CPU GMMMMrrrry (GB)
80
90
InfoScan
mod
Vin--1
80
DeiT-Ti
30
39-
1.6
DefFT1
20
10
Infoscan
1.2
512
Resolution
Resolution
Figure 1: Performance comparison across resolutions for classification accuracy, segmentation mloU, FPS, and GPU memory usage of DeiT-Ti, Vim-Ti, and InfoScan.
emerged for efficient visual representation learning, leveraging structured scanning patterns within state-space architectures to compress hidden states and capture long-range dependencies. These approaches, built upon predefined traversal orders, have alleviated computational bottlenecks to some extent. However, they perform uniform scanning-such as raster or Hilbert curves-treating all patches identically regardless of their informational content. This rigid scanning paradigm im- plicitly assumes uniform information distribution across the image, thereby overlooking the varying significance of local regions and limiting adaptive feature aggregation.
In this paper, we propose InfoScan, an peermatiion-gii-driveeee novel Vision model that adaptively allocates computation based on feature significance. Our framework built upon the VISS (Vi- sion Information State Space) block and consists of three key components: an information scoring module estimate the informativeness of each patch, patch selection mechanism to prioritize content-rich regions, and a sequential scanning policy learned via reinforcement to dynamically ad- just the processing order. By focusing computation on high-value regions early in the forward pass, InfoScan achieves strong performance with significantly reduced computational overhead, enabling efficient and adaptive vision modeling.
Compared with benchmark vision models based on CNNs (ConvNeXt (Liu et al., 2022a)), VITs (Swin (Liu et al., 2021), DeiT (Touvron et al., 2021b)), state-space models (Vim (Zhu et al., 2024a)), and SS2D architectures (VMamba (Liu et al., 2024)), InfoScan achieves consistent improvements of +0.8% to +1.9% in mloU and +0.6% to +5.8% in Top-1 accuracy across image classification, segmentation, and object detection tasks, while reducing model parameters by 18% 32%. No- tably, on image classification, InfoScan shows particularly strong gains, outperforming all baselines by over +1.5% Top-1 on ImageNet-Iee IK. Under the Mask R-CNN framework (Han et al., 2021), outperforms Swin-B and ConvNeXt-B on MSCOC02017 (Lin et al., 2014) with 30M fewer param- eters. These gains are consistent across model scales and domains, including natural and medical imaging.
Our contributions are summarized follows: (1) We introduce an information-awen awaree scanning mechanism that quantifies the significance of each image patch through weighted combination of Shannon entropy and local variance, enabling the model to adaptively prioritize high-information regions. (2) We propose principled mathematical framework to jointly optimize patch information content, information loss, and scanning step size, yielding a more efficient and effective traversal strategy beyond fixed or heuristic scanning paths. (3) We design reward-driven dynamic scanning policy based on a Markov decision process, allowing the model to learn where to attend next accord- ing to contextual information density, thereby enhancing both local detail preservation and global context integration.
2 RELATED WORK
2.1 EFFICIENT AND ADAPTIVE COMPUTATION IN VISTON
Modern vision models face growing computational demands, especially when processing high- resolution inputs. A dominant paradigm for efficiency is sparse computation, which reduces FLOPss
2

---

Published as a conference paper at ICLR 2026
Archibecter
#XWx3
(R)4 X
8/0 x MXX
8/16 W
8/32 X W222
Parch Parttton
Stape
Stage
Stage
Stape
Vies Black
Dosir Sampline
Vis tlock
VidO Black
Desurn Samppling
VISS Block
Block
Xamba Block
(b) VSS Blods
( VISS Block
Lineur
Unvor
PPN
FPN
Faiea
IN
Selection
S
SUUI
EWDDurFdd
SHIP
SELU
SaPSI
Liter
Lingar
Blos
BMWGonsid
Linexr
RHDNern
Liner
Infommation
SECERIE
Architecture of InfoScan and its key components. (a) Overall network with patch embed- ding and hierarchical VISS blocks. Mamba block with RMSNorm, DWConv Id, and linear layers. (b) VSS block integrating SS2D and FFN. (c) VISS block with SPS2D for adaptive scanning.
Figure 2:
by selectively activating model components or processing only subset of visual tokens. Sparse attention mechanisms (Child et al., 2019) restrict contextual interactions to local neighborhoods top-k salient regions, while dynamic token pruning (Kim et al., 2024) removes low-importance patches during inference. Concurrently, conditional computation approaches (Bengio et al., 2013; Wang et al., 2022) adapt model capacity based on input complexity, such as allocating more re- sources to semantically rich regions.
However, most existing methods operate under static or reactive paradigm: they either apply fixed sparsity patterns or reweight tokens after full forward pass over all patches. In contrast, our work proposes a proactive efficiency strategy-by learning to sean patches in an order that prioritizes high- information regions early, we reduce redundant computation at the input level, before feature aggre- gation begins. This shifts the efficiency bottleneck from post-hoc pruning to front-loaded perceptual prioritization, aligning with cognitive principles of selective attention (Itti et al., 2001) while main- taining end-to-cn and-to-end trainability.
2.2 SCANNING STRAGE
The design of visual scanning policies has long been foundational consideration in both classical and modern vision architectures. Early approaches, such as raster (Gu & Dao, 2024) and zigzag scanning (Ma al.. 2019), enforce fixed, connent-agnostic orders based solely on spatial coordi- nates. These deterministic strategies are computationally efficient and easy to implement, making them prevalent standard State Space Models. Alternatively, space-filling curves like Hilbert (Ka- mata et al., 1999) and Z-order curves aim to enhance spatial coherence by minimizing the Euclidean distance between consecutive patches, improving locality in sequential processing. Despite their geometric elegance, all such methods assume uniform information density across the image-a strong prior that rarely holds in real-world data. This leads to inefficient computation, as high- entropy, semantically rich regions (e.g., object boundaries) neous backgrounds. More recent works explore dynamic attention mechanisms to reweight patch importance (Dai al., 2021), but they still rely globally scanning all patches first.
processed earlier than homoge-
3

---

The numbers in Published as a conference paper at ICLR 2026
Raster Scon
Zipaog Scan
Hlbert Scan
INFOSCAN
Figure 3: A comparison of InfoScan scanning paths with other scanning patterns, the figure indicate the number of InfoScan scans.
3 METHODOL.OGY
Visual scanning defines a policy a that maps a 2D grid of image patches P = {pr.} to sequence S (s1...., 8N), where such as state-space models (SSMs) and causal Transformers, where the processing order directly influences contextual integration and computational efficiency. The problem is thus to find an optimal policy x" E IT that maximizes a task-specific objective F(S,). ). We argue that the optimal scanning policy must be content-aware. To formalize this principle, decompose the problem into two coupled subproblems. (1) Patch Importance Quantification: Define an importance function finto P + E that assigns a scalar value In (pnto(ps) to each patch based on its content, measuring its informativeness. (2) Policy Optimization: Given the importance scores {Za}, find the policy that maximizes the cumulative discounted information gain a more informed objective:
P. This sequencing is critical for directional models
π
arg max sell
I
where s, denotes the patch selected step & under policy and (0, 1] is discount factor that prioritizes early acquisition of high-information regions.
To achieve this information maximizing scanning policy, we propose information cettratiionnniin---e gain-driven novel Vision model: InfoScan, which is built upon Visual Information State Space (VISS) blocks illustrated in Figure 2. Each VISS block consists of single network branch and residual modules. In contrast to the standard VSS block, we replace the SS2D component with a Patch Selection Block (PSB), an Information Scoring Module (ISM), and Path Planning Module (PPM) (see Figure 2); further details are provided in the next section. If not specificed, all results reported in this paper are obtained using InfoScan models instantiated with this architecture.
3.1 INFORMATION SCORING MODULE
The Information Scoring Module provides content-awand prior for adaptive scanning by quanti- fying the information content image patches. We propose a composite score I(S) that jointly models global color diversity and local texture complexity:
I(S) A W2 where w1,42 2 wi W2
Here, f and V denote the zero-mean respectively. The weights w/ and w2 are determined via grid search on the ImageNet- validation set to maximize classification accuracy under a fixed scanning budget, and are then fixed across all downstream experiments. We find the optimal setting be both global and local cues contribute meaningfully, with slight bias toward color diversity (The feasibility of w/ 0.6, W2 0.4 other vision tasks is validated in the Appendix C.I).
normalized Shannon entropy and local variance,
0.6, wa 0.4, indicating that
Shannon Entropy (Global Diversity). We compute the entropy of quantized color distributions to measure global color variety (Bromiley et al., 2004). Each RGB channel is uniformly quantized into C' bins, resulting K = 8 discrete levels. Let Pk: denote the empirical frequency of bin k. The entropy is H E
1= Pie log Pk (with 0 Olog 0 & 0), which is then standardized across the dataset to obtain f.. Higher values indicate greater chromatic variation.
4

---

Published as a conference paper at ICLR 2026
Local Variance (Texture Complexity). To capture fine- tine-nnained structure, compute local in- tensity variance within x 3 neighborhoods. For a pixel (r.z) y) with neighborhood . (2, y), the mean color is IN
INT (u.)))N [(u, v), and the local variance is V(z, [N] ((.,)) [2, u) - INl where || denotes the squared Euclidean norm. The patch- level variance = n (aai)es (z, y) is standardized to yield V.
Boundary Information. We further model inter-patch transitions by defining boundary salience. For a boundary e between patches S and S2. we define Iς(e) I(S)) I(S2), encouraging scanning paths to traverse between high-information regions and enhance contextual coherence in sequential processing.
3.2 PATCH SELECTION MODEL
Let Np denote the side length of square image patches (i.e., patch size is Np Np). When dividing an image into such patches, there a fundamental trade-off: smaller Np disrupts spatial context and incurs high computational overhead due to the generation of numerous patches; larger Np- the other hand, risks losing fine-grained details and increases processing latency per patch. To balance efficiency and information fidelity, select the optimal patch size Nr by minimizing the total cost function Crotal (N)):
(NV)) (NN,) + (1 A)Cinto A [0,
(3)
where / WH is the total number of pixels in the image, = I/N2 the number of patches, (NNp) measures efficiency-relattee
costs, (n quantifies information loss, (Np) = N T paach (Np) represents the total time required to scan the entire image, and ppachh(Ni is the time needed to process one N, x N, patch.
We model the delay per patch using power-law model fitted calibration dataset Dealih. Specifically, D Dealih consists of 50K images from ImageNet- Val set, covering mul- tiple classes ensure diversity. For each image H, E Dealih.
Efficiency Term.
measure the execution times y: under different Np settings on the target hardware. This leads to the model: Tmmch((,)
kp No and (Np) = N Taacch (Np) = k No-2 where kı == kp. Here, reflects the effective time complexity of patch processing. When > 2, the efficiency cost CNN) increases with Np: it remains constant when a 2: and decreases when a <
Information Term. We model information loss as a U-shaped function, capturing the dual risk of insufficient global context at small N, and lost local details at large N,:
102
C
+ K3N p
12 Kzg > 0, 3,7 > 0.
(4)
No
The first term decays with increasing Np (indicating more complete global context), while the sec- ond term grows with N, (indicating worse local resolution).
Optimization and Solution Strategy.
The optimal patch size is given by:
hy
N arg min N₂
Aky AkyINa A +(1 (1-11) x)
+ k
(5)
No
We solve this numerically: first, identify an interval containing the minimum over the feasible set S (determined by image dimensions, stride constraints, and memory limits); then apply golden-section search in the continuous relaxation space; finally, round the result to the nearest valid N, E S. The trade-off parameter X is calibrated once to meet preset latency memory budgets and remains fixed in subsequent experiments. Complete algorithmic details are provided in Appendix A.2. Notably, the determination of parameters k1, f2, and K3 is detailed in Appendix A.4.
3.3 PATH PLANNING MODULE
We reframe image scanning as an adaptive sequential decision process, moving beyond fixed, content-agnostic paths (e.g., raster, zigzag, space-filling curves). Our core idea is to model the
5

---

Published as a conference paper at ICLR 2026
Spaced Repetition Mechanism
(b) Adaptive Rehearsal Pattern
Cumulative Performance
1.
1.0
8.69
inforaate Low interationn
Adeative reviits
Normal laec Information Gain Gr
6.6
Amai
0.43
100%
0.4
044
10 20
22
30
Un/tum
Cecmratiic
Adapèiva
Since
Time Stops
Figure 4: (a) Spatial repetition enables priority decay with reward attenuation for high-information regions, (b) dynamic rehearsal adjusts revisit intervals based on content importance, and (c) demon- strates that the proposed method achieves a significant 98% improvement in cumulative information gain compared traditional scanning strategies.
scanner agent traversing the image plane, where the scanning path is dynamically shaped by the underlying content.
Formally, we partition the image into an n. x n grid of patches, with each patch indexed by its coordinates (i.j), where j E {1, ....., n). The state space is defined as:
S ((,,)) i,j 2 ,.........],1 nn]]
The scanning process is modeled as trajectory = (sp, $1 ....... ST), where 8, E S denotes the agent's location time t, starting from an initial patch 8g.
Unlike traditional random walks with uniform transition probabilities, we formulate scanning guided random walk (Zhu & Ghahramani, 2002), which can be formalized as a Markov Decision Process (MDP) M (S, r))
(State:)s; (1c. optionally augmented with a visitation map V22 E {0, (0,1]" where Vi(i,))
1 if patch (i,j) has been visited before t.
(Action:) A = (1.4. - (4-connectivity), so at E A(t)) moves the agent to a neighboring patch.
UTrase a policy aa(a, $t), the next state deterministic: 8141 = f(r,, az). (Discount:) 7 E 1] weights immediate rewards more heavily.
The goal is to learn policy, the details of the policy model are provided in Appendix A.5 mn that maximizes the expected cumulative return:
max 0
[G]
LESTO
Σ
rre oes (4.)))))) (ii1)
Markov Decision Process unifies two key objectives: (1) exploration of unvisited regions, and (2) exploitation of semantically rich areas.
3.4 REWARD-DRIVER SCANNING
Adaptively discovering the optimal scanning path requires a reward mechanism that dynamically prioritizes information-rioe regions while ensuring broad coverage. Our design is inspired by cog- nitive principles in human learning- specifficlll Levin (1986), spaced repetition and focused re- where important stimuli are revisited over time to strengthen perception and memory. To emulate this behavior, we introduce a adaptivv reward function that slows the decay revisit incentives for semantically salient patches, enabling periodic re-scanning while avoiding redundant fixations.
At each step, the model receives reward that balances revisiting informative regions and exploring unvisited areas. Let I(s) E R++ denote the information content of patch s. Let k($++1) be the number of times patch 8141 has been visited within the past t steps (with k 0 if never visited). The decay factor a e (0, 1) is content-acde content-adaptive: we set a = Chigh if [(s) > 8, and a = atow < Chigh otherwise. This ensures that high-salience regions are "remembered" longer, promoting sustained

---

Published as a conference paper at ICLR 2026
yet sparse revisits. The visitation indicator Vi(s) E {0, } if patch has been observed before time t, and 0 otherwise. The weight À > 0 controls the exploration bonus.
The reward function is then defined as:
r rSst des S1+1) adaptive revisitation insentive
(s$+11) a M(())))
A 11 (1(--(())+1))
+
B. (isitted
exploration bonus
neighborhood information gain
The adaptive revisitation incentive aligns with human-like visual rehavior-repeatted attending to meaningful content- while the exploration bonus ensures systematic scanning across the entire image. As shown in the Figure 4, Our reward design captures 98% of the achievable information gain, demonstrating high efficiency in perceptual resource allocation.
4 EXPERIMENTS
In this section, conduct series of experiments to evaluate the performance of InfoScan across various vision tasks and compare with mainstream baseline models. The results are presented in the accompanying Figure 1. We further validate the effectiveness of each component in the proposed scanning strategy. All experiments are conducted on a server equipped with 16 V100 GPUs.
4.1 IMAGE CLASSIFICATION
Settings: We evaluate on ImageNet-IK (1.28M train, 50K val) (Deng et al., 2009) using 224 x 224 resolution. We primarily adopt the hyperparameter settings and experimental configurations from VMamba. During training, we use AdamW optimizer, cosine decay (initial LR x 10 10-3), and standard augmentations (e.g.. random crop, flip, label smoothing).
Results: As shown in Table la, InfoScan outperforms CNN (Liu et al., 2022b; Koonce, 2021) Transformer, and SSM baselines in accuracy and efficiency. InfoScan-S achieves 84.64% top-1 accuracy, surpassing DeïT-S (Touvron et al., 2021a) (+4.0%) with fewer parameters. Compared to VMamba-S (83.2 (83.24%), InfoScan-S gains +1.4% accuracy with 26M fewer parameters. These results demonstrate that information-aware dynamic scanning enables stronger and more efficient visual representation learning.
4.2 OBJECT DETECTION
Setting: We conduct experiments for object detection on the MSCOC02017 (Lin et al., 2014)
dataset, and provide detailed experimental settings in the Appendix B.2.
Results: Table 1b demonstrates that InfoScan achieves competitive segmentation performance while maintaining good parameter efficiency. It's worthy note that InfoScan-B achieves the highest APhox (49.8 %) and Apmaak (44.7 %) scores among all methods tested, while using significantly fewer parameters (78M) compared to the other high fighhperforming backbones like Swin-B (107M) and Ct-v (108M). The results verify InfoScan's effectiveness for detection tasks, which also implies that the information-awarc scanning mechanism can be transfered well to detection tasks, not only classification and segmentation.
4.3 SEMANTIC SEGMENTATION
Settings: We evaluate semantic segmentation on ADE 20K (Zhou et al., 2017) and BraTS- 2021 (Baid al., 2021) using UperNet at 512 x
Table 2: Semantic segmentation performance ADE- 20K and BraTS-2021 with UPerNet 512 x 512. P.Params; F:FLops.
512. The training process follows standard pro- tocols, and full details can be found Ap- pendix B.I. Notably, additionally con- ducted generalization experiments on BraTS- 2021 and ADE-20K. The experimental results and detailed analysis can be found in the Ap- pendix C.2.
ADE-20K
BraTS-203
Backbone
mloU
P(M)
mIoU
P(M)
ResNet-50
41.9
67
17.4
67
ResNet- 101
45.2
86
20.7
86
D DeiT-S
38.3
11
14.3
11
42.4
43
21.3
43
Vim-T
40.3
13
18.7
13
7 Vim-S
45.3
46
20.4
46
InfoScan-T
40.9
10
19.3
InfoScan-S
45.8
38
22.3
38

---

Published as a conference paper at ICLR 2026
Table I: (a) ImageNet-IK classification 224 input (DeiT-B" at 3842). Throughput: per-GPU
img/s. (b) Mask R-CNN on MSCOC02017 val (512x2048). AP'/P"m box/mask AP; Sch.: sched- ule; MS: multi-scale; P: Params (M); F: FLOPs (G).
(b) Object Detection
Backbone
Sch.
AP
Apm
P(M)
(a) Image Classification
Swin-T
1x
42.9
38.1
48
Method
P(M)
F(G)
Thr//rain
Acc(%)
Swin-S
1x
44.5
39.6
DeïT-S
22
4.6
96.4/137.3
74.70
Swin-B
1x
45.6
41.5
107
DeïT-B
85
17.4
24.8/55.9
80.11
Swin-T
3xMS
46.5
41.5
48
DeiT-B*
86
55.3
18.7/20.1
83.23
ConvNeXt-T
1x
44.3
39.6
48
Swin-T
28
4.6
68.6/59.8
81.60
Nen-NeXI-S
1x
45.5
40.9
70
Swin-S
50
8.7
40.4/35.8
83,23
ConvNeXI-B
1x
47.2
42.4
108
Swin-B
88
15.4
20.6/18.7
83.91
ConvNeXi-T 3xMS
46.1
41.3
48
VMamba-T
31
4.9
77.2/24.8
82.47
VMamba-T
1x
46.8
42.9
VMamba-S
50
8.7
47.1/17.0
83.24
VMamba-S
1x
48.4
43.3
70
VMamba-B
89
15.4
29.4/12.2
84.32
VMamba-B B
1x
49.1
44.3
108
VMamba-T
3XMS
48.3
43.6
50
InfoScan-T
10
2.5
91.7/46.5
83.43
InfoScan-S
24
4.8
63.8/33.6
84.64
InfoScan-T
1x
47.1
41.2
42
InfoScan-B 38
8.4
54.0/26.2
85.19
InfoScan-S
1x
48.8
43.4
59
InfoScan-B
1x
49.8
44.7
InfoScan-T
3xMS
48.6
43.2
42
Results: As shown in Table 2, INFOSCAN achieves superior accuracy-efficiency trade- offs. On ADE 20K, INFOSCAN-TI obtains 40.9% mloU (vs. DeiT-Ti: 38.3%) with fewer parameters, and INFOSCAN-S reaches 45.8% (+0.5% over Vim-S) with 38M params. On BraTS-2021. achieves 19.3% (T1) and 22.3% (S), outperforming DeiT' and Vim. Notably, InfoScan-S matches RpeNNet-ResNet-Il 101 (45.2% on both datasets with 56% fewer parameters. These results demonstrate strong generalization across natural and medical images.
4.4 ABLATION STUDY
We first conduct ablation experiments the path scanning module and the patch planning module, using Vision Mamba as our baseline model for comparison. We fix the input image size to 512 x 512 and evaluate three datasets. As shown in Table 3, ImageNet- 1K, the path scanning module improves InfoScan's accuracy from the baseline 82.5% to 83.4%, and the patch selection module further increases 85.9%.
Table 3: Ablation core modules. El (Patch Selection), E2 (Path Planning).
ImageNet-IK 1K
ADE-20K mloU (%)
BraTS-2021 mloU (%)
E2
Top-1 Acc (%)
x
82.5
45.3
18.7
x
83.4
45.7
18.9
85.9
45.9
19.3
We conduct ablation experiments the three components of the reward function, using Vision Mamba as the baseline model. We fix the input image resolution to 512x512 and evaluate on three datasets. As shown in the Table 4, ImageNet-IK, IK, the combined effect of the adaptive revisitation incentive, exploration bonus, and neighborhood information gain boosts InfoScan's 's accuracy from 84.2% to 85.9%.
8

---

Published as a conference paper at ICLR 2026
Figure 5: Visualization of the content-adaptive scanning path InfoScan.
Notably, the ablation studies Shannon Entropy and Local Variance in the Information Scoring Module are provided the Appendix A.3.
Table 4: Ablation on reward components. M1 (revisitation), M2 (exploration), M3 (neighborhood gain).
ImageNet-I 1K
ADE-20K mIoU (%)
BraTS-2022 mloU (%)
M1
M2
M3
Top-I Acc (%)
X
X
80.4
42.3
16.7
/
X
X
81.1
42.7
17.8
/
V
X
84.2
43.6
18.3
/
V
85.9
45.9
19.1
4.5 SCANNING METHOD VISUAL COMPARISON
Figure iillsstrates the visualization of the learned adaptive scanning path in InfoScan. From left to right: (1) the input image. (2) the information heatmap computed by combining Shannon entropy and local variance, where warmer colors (red/orange) indicate regions with higher complexity and richer content, and (3) the overlaid scanning trajectory on the heatmap. The red lines represent the adaptive scan path, which dynamically adjusts its density based content importance: sparse scanning is applied low-information areas (e.g., uniform walls), while dense, multi-pass traver- sal is adopted in complex regions (e.g., the suitcase and surrounding clutter). This demonstrates that InfoScan effectively prioritizes informative regions through a content-ave aware scanning strategy, achieving both efficiency and accuracy.
5 ANALYSES
key question is whether performance gains stem from sean repetition or adaptive routing. To disentangle these factors, we conduct ablation studies with fixed scanning patterns under different configurations, as summarized Table 5.
Repetition yields diminishing returns. Comparing single vs. triple passes of fixed patterns (ID vs. ID 2, ID vs. ID 5), we find that repeating scans brings marginal or even negative gains. Specifically, Triple Raster Scan (ID 2) achieves no improvement in Top-1 accuracy and degrades ADE-20K mloU to 41.8%, likely due to redundant computation on low-salience regions. Triple Hilbert Scan (ID 5) improves ImageNet Top-1 by only +0.7% and ADE-20K mloU by +0.6%, but reduces BraTS-202 mloU by -0.6% indicating poor adaptation to structural heterogeneity in medical images.

---

Published as a conference paper at ICLR 2026
Stochastic initialization enhances coverage. Randomizing the origin (ID 4) improves Single Hilbert Scan from 83.6% to 84.5% Top-1 and 42.9% to 43.5% mloU on ADE-20K, confirming that random starts enhance spatial diversity. However, even with randomization, fixed fixed-patter methods plateau ge. ID 5), suggesting inherent limitations of static routing.
Adaptive routing outperforms repetition and randomness. InfoScan with random initialization (ID 7) achieves 85.9% Top-1 and 45.9% mloU, surpassing all static baselines. Crucially, even with fixed initialization (ID 6), InfoScan (84.2%, 44.6%) outperforms all ablated variants-including Triple Hilbert (ID 5) and the randomized Single Hilbert (ID 4)-demonstrating that learned adaptive routing the primary driver of gains, not mechanical repetition or stochastic exploration.
Learned policy enables intelligent revisitation. These results confirm that performance improve- ments arise not from redundancy, but from when and how regions are revisited-uuided by a reward- driven policy that balances exploration, uncertainty-base based refinement, and local coherence. Intelli- gent policy design, rather than scan repetition, enables robust generalization across domains.
Table Ablation study on scanning policy and starting point. "Fixed" denotes a predefined starting position; "Random" denotes a stochastically chosen start. InfoScan uses a reward- adaptive scanning policy with random initialization.
ImageNet-IK
ADE-20K mIoU (%)
BraTS-2021
ID
Method
Start
Top-1 (%)
mloU (%)
1
Single Raster Scan
Fixed
83.5
42.9
15.3
2
Triple Raster Scan
Fixed
83.6
41.8
16.5
3
Single Hilbert Scan
Fixed
83.6
42.9
17.1
4
Single Hilbert Scan
Random
84.5
43.5
18.3
5
Triple Hilbert Scan
Fixed
84.8
44.2
16.8
6
InfoScan
Fixed
84.2
44.6
18.0
7
InfoScan
Random
85.9
45.9
19.3
CONCLUSION
This paper presents InfoScan, novel visual backbone that introduces information-aware dynamic scanning for efficient high-resolution representation learning. By casting 2D spatial traversal as sequential decision process guided by learned salience metric, InfoScan departs from fixed or heuristic scanning patterns and adaptively allocates computation to informative regions. The core mechanism is integrated into a state space framework through the Visual Information State Space block, which supports flexible, content-dependent paths while maintaining near- near-linear complexity. Compared to prior state space and hierarchical vision models, InfoScan achieves improved efficiency and robust generalization across diverse vision tasks, including classification, dense prediction, and medical imaging. Notably, the learned scanning policy exhibits strong interpretability. aligning with human visual attention and enabling diagnostic analysis of model behavior. The design principle- scanming less but smarter-opent a new direction for efficient visual architectures beyond static token processing. In future work, will adapt InfoScan more vision tasks (Feng al., 2025; Qian et al., 2025), and explore extending it to video modeling and adapting to vision-language mod- els (Liu al., 2023; Bai al., 2023).
ACKNOWLEDGGEMENT
This project supported by National Natural Science Foundation of China (No. 62306132, 62402490, 62576225), Guangdong Basic and Applied Basic Research Foundation (No. 2025A151501 1564), Natural Science Foundation of Shanghai (No. 25ZR1402136), We thank the anonymous reviewers for their insightful feedback on this work.
ETHICS STATEMENT
This work focuses on advancing the efficiency and adaptability of visual representation learning through algorithmic innovation. All experiments are conducted on publicly available datasets, in-
10

---

Published as a conference paper at ICLR 2026
cluding ImageNet-1K, ADE-20K, MSCOC02017, and BraTS--2211 which have been widely used in prior research under established ethical guidelines. We do not collect or use any private or sensi- tive data. The proposed method does not involve human subjects, personal information, or biometric identification. While the framework is general-purpose potential misuse (e.g., in surveillance or deepfakes) is not specifie our approach and remains broader concern for the machine learning community.
REPRODUCIBILITY STATEMENT
We are committed to full reproducibility. All experimental details necessary reproduce our results are provided in the main paper and the appendix, including model architectures, hyperparameters. training schedules, and optimization settings. We standard benchmarks and publicly available datasets. The codebase, including training and evaluation scripts, will be released. Pre-trained models and detailed inference instructions will also be made publicly available. All experiments conducted on standard hardware (16x NVIDIA V100 GPUs).
REFERENCES
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng. Yang Fan, Wenbin Ge, Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Uday Baid, Satyananda Ghodasara, Suyash Mohan, et al. The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314, 2021.7 7
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: 2013.
PA Bromiley, NA Thacker, and E Souhova-Thackers Shannon entropy, renyi entropy, and informa- tion. Statistics and Inf. Series (2004-004), 9(2004):2 8, 2004.
Rewon Child, Scott Gray, Alec Radford, Jeff Wu, and Ilya Sutskever. Generating long sequences with sparse transformers. arXiv preprint 1904::9904.10555 2019.
Xingyi Dai, Yutong Chen, Jian Yang, Peihua Lin, Yuchen Yuan, and Qi Chu. Dynamic DETR: End- to-end object detection with dynamic attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), 2021. 3
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2009.
Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Lu Yuan, Dong Chen, and Baining Guo. Cswin transformer: A general vision transformer backbone with cross-shaped windows. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), 2022. 1
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x words: Transformers for image recognition at scale. In Proceedings of the International Conference on Learning Representations (ICLR). 2021.
Wei Feng, Feifan Wang. Ruize Han, Yiyang Gan, Zekun Qian, Junhui Hou, and Song Wang. Un- veiling the power of self-supervision for multi-view multi-human association and tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(1):351-368, 2025. 10
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference Language Modeling, 2024. 1,3

---

Published as a conference paper at ICLR 2026
Qi Han, Zejia Fan, Qi Dai, Lei Sun, Ming-Ming Cheng, Jiaying Liu, and Jingdong Wang. On the connection between local attention and dynamic depth-wise wise convolution. In Proceedings of the International Conference Learning Representations (ICLR). 2021. 2
Laurent Itti, Christof Koch, and Ernst Niebur. A model of caliency-baseld visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11))1254 1268, 2001.
Sei-Ichiro Kamata, Robin O. Eason, and Yutaka Bandou. A new algorithm for N-dimensional hilbert scanning. IEEE Transactions on Image Processing. 8(7): 1999. 3
Minsu Kim, Shang Gao, Yi-Chun Hsu, Jiajun Wang, Jiajun Shen, Chen Lee, and C. Lee Giles.
Token fusion: Bridging the gap between token pruning and token merging. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision (WACV). 2024.
B. Koonce. ResNet 50. In Convolutional Neural Networks with Swift for TensorFlow: Image Recognition and Dataset Categorization, pp. 63-72. 2021.
Joel R. Levin. Four cognitive principles of learning-strategy instruction. Educational Psychologist, 21(1-2):3-17, 1986.
Tsung-Yi Yi Lin, Michael Maire, Serge J. Belongie, et al. Microsoft COCO: Common objects in context. In Proceedings of the Furopean Conference on Computer Vision (ECCV). 2014. 2,
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. The Annual Conference Neural Information Processing Systems (NeurlPS), 2023. 10
Yuzheng Liu, Yan Tian, Yunchang Zhao, Bin Ni, Xinghao Li, Jie Tang, Yu Qiao, Dahua Lin, Jiefeng Qiu, and Yiheng He. VMamba: Visual state space model. In Advances in Neural Information Processing Systems (NeurIPS). 2024. 1,2
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hicrarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV). 2021. 1,2
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition (CVPR). 2022a. 2
Zhuang Liu, Hanzi Mao, Chao-Yus Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A ConvNet for the 2020s. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022b. 7
H. Ma, A. Sang, C. Zhou, and et al. A zigzag scanning ordering of four-dimensional Walsh basis for single-pixel imaging. Optics Communications, 443:69-75, 2019. 3
Harsh Mehta, Ankit Gupta, Ashok Cutkosky, and Behnam Neyshabur. Long range language mod- eling via gated state spaces. In Proceedings of the International Conference on Learning Repre- sentations (ICLR). 2023.
Zekun Qian, Ruize Han. Zhixiang Wang, Junhui Hou, and Wei Feng. DOVTrack: Data-efficient open-vocabulary tracking. In The Annual Conference on Neural Information Processing Systems (NeurIPS), 2025. 10
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning (ICML), 2021a.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In Proceedings the International Conference Machine Learning (ICML). 2021b. 2
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In The Anmual Conference on Neural Information Processing Systems (NeurIPS), 2017.
12

---

Published as a conference paper at ICLR 2026
Wenhai Wang. Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Zheng Zhang. PVT v2: Improved baselines with pyramid vision transformer. Computational Visual Media, 8(3):415-435 2022.
Hao Wu, Yao Yang, Hui Xu, Yulun Zhang, Jian Wang. Xian Liu, Yunlong Zhang, Zhi Liu, and Yanning Zhang. RainMamba: Enhanced locality learning with state space models for video deraining. In Proceedings of the ACM International Conference on Multimedia (ACM MM). 2024.
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ADE20K dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.
Li Zhu, Baoqun Liao, Qingyu Zhang, Xinghao Li, Jie Tang, Yu Qiao, Dahua Lin, Jiefeng Qiu, and Yiheng He. Vision Mamba: Efficient visual representation learning with bidirectional state space model. In arXiv preprint vxXi:2401.09411. 2024a. 2
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vi- sion mamba: Efficient visual representation learning with bidirectional state space model. In International Conference on Machine Learning (ICML). 2024b.
Xiaojin Zhu and Zoubin Ghahramani. Learning from labeled and unlabeled data with label propa- gation. Technical Report, School of Computer Science, Carnegie Mellon University, 2002. 6
Nikola Zubié, Mathias Gehrig, and Davide Scaramuzza. State space models for event cameras. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2024.
A MATHEMATICAL DERIVATIONS
A.1 PATCH SIZE OPTIMIZATION DERIVATION
Here we provide the complete derivation of the optimal patch size formula from Section 3.1. Starting from the total cost function:
hy No
Crotal (Np) = Adi No
Na-2 (1 (1-1 A)
+
hig
Ny
(9)
Taking the derivative with respect to and setting it to zero:
d
dNp Cueaa Ali [( - No Na-3 (1 A)k23 N 8-1 (1 A)k37 Ng-
(10)
Multiplying through by p8+1 to eliminate negative exponents:
lk - N) (1 - 2)423 + (1 - N3
(1I)
Let m = n, = 7 + + and
A Ikyl(a 2), (1-,=,1- (1-1--==1-----0 -)kay7
we arrive at
ANT + CNT B.
A.2 INFORMATION THEORY FOUNDATIONS
Normalization of Information Scores:
Given a dataset of patches {Si, .N), the normal-
ization of Shannon entropy H and local variance V proceeds as follows:
13

---

Published as a conference paper at ICLR 2026
(a) Efficiency Cost
(b) Information Cost
[c] Total Cost
decrestig)
0000
2-03 wntterr
Intrrmaton Cost Crue7
Pltce
Patch Stre
Figure 6: Patch size P affects cost components differently; (a) efficiency cost increases with P. governed by exponent a; (b) information cost drops sharply at small p. reflecting local redundancy; (c) total cost balances both, revealing optimal p" z 1.0 and sensitivity to trade-off weight X.
For Shannon entropy:
1
H(:))
(12)
N
N Σ (H().) pH)
T = N -
(13)
i=1
HSS)) = H(S)) PH
(14)
OH
Similarly for local variance V. This ensures = 0 and Var[ = VarlV] = 1.
Boundary Information Justification: The boundary information Ie(e) I(Si) x I(S2) is mo- tivated by the principle that transitions between high-information regions carry multiplicative im- portance. This captures the intuition that moving from one informative patch to another informative patch provides compound value for the scanning process.
A.3 ANALYSES INFORMATION METRIC VALIDITY
To validate the necessity and effectiveness of our information metric, conduct ablation study on the components used to compute the information map: Shannon entropy (QI) and local pixel variance (Q2). Results in Table 6 show that Shannon entropy component alone improves perfor- mance over the baseline without any information scoring (79.7% - 81.6% Top-1 on ImageNet- confirming their individual utility in guiding adaptive scanning.
However, combining both QI and Q2 yields the best performance across all datasets-85..0 85.9% Top-1 accuracy ImageNet- IK, +4.3% over Q1-only and +6.2% over booring-demonstrattiinn that tropy and variance capture complementary aspects of visual information: global semantic diversity and local texture richness, respectively.
These results support the design of our composite information metric as not only empirically effec- tive but also functionally justified. The sigmificant gains in downstream tasks (c.g., +8.7% mloU on BraTS-2021 further confirm that accurate information estimation is critical for efficient and adaptive vision modeling.
Table Ablation on Information Scoring module. QI (Shannon Entropy), Q2 (Local Variance).
Q2
ImageNet-IK 1K
ADE-20K mloU (%)
BraTS-202
Top-1 Acc (%)
mloU (%)
X
79.7
37.2
12.1
X
81.6
43.2
17.2
/
85,9
45.9
19.3
14

---

Published as a conference paper at ICLR 2026
A.4 PARAMETER ESTIMATION FOR THE PATCH SIZE OPTIMIZATION FRAMEWORK
We detail the procedure for estimating the model parameters ki, k2, and K3 used in the total cost function:
Gonat (N)) (Np) + (1 - X)Cinta (Np), kg
where CxpNN) k2 Np No-2 and Cmno(Np)) Np
Ny.
kj Efficiency scaling coefficient. The parameter b (equivalent to kp the patch timing model) captures the hardware- and model-specific constant in the power-law relationship (pach((Np)) kp No. It is estimated via linear regression on log-tranfformed timing measurements from the calibration dataset Dealib- Specifically, we collect execution times {yi } for varying Np on 50K im- ages from ImageNet-1K Val, and fit:
log Tunch (NV) a log N₂ + log
Using ordinary least squares, we obtain estimates for a and log ky from which kj = kp is derived. This ensures (IN) accurately reflects empirical computational latency.
22, k3t Information loss coefficients. To estimate k2 and k3, Let Φ(x; Np) denote the deep fea- tures extracted from an image I using patch size Np. We measure information loss as the deviation from a reference representation o(x; N, where Appt = chosen as a high-resolution baseline:
Linto (2, (z, NNp) = [|r(;; N2) o(æ; (
We compute Linto (I, Np) across Dealb for multiple Np values and average obtain empirical infor- mation loss P(Np).
We then fit the parametric model Cunto (Np) k2>p² N C(Np) using non-linear least squares, the resulting k2 and ky ensure that the information cost term reflects the U-shaped trade-off between global context and local detail preservation.
A.S POLICY NETWORK ARCHITECTURE
The policy network in InfoScan is responsible for learning an adaptive scanning policy x (a, (sc; that selects next patch to process based on the current state 8t- The network is trained via reinforcement learning to maximize the cumulative discounted information gain.
State Representation The input state at step is 4-channel tensor composed of:
Current position (fe,:)): encoded as two scalar maps where each spatial location (i,j) is assigned the normalized coordinates (A. : 4)
Information Map InfoMap,: the spatial map of information scores / (px.) computed by the Information Scoring Module (ISM), normalized to [0, 1].
Visitation Map Vy: binary map indicating which patches have been visited if visited, otherwise).
These four channels are concatenated to form the input tensor of shape n x n x 4.
Network Structure The policy network is lightweight convolutional neural network (CNN) with the following layers:
1. Input: n x n X 4 state tensor.
2. Convolutional Layer 1: 3 kernel, 64 filters, ReLU activation, stride padding 1.
3. Convolutional Layer
3 kernel, 64 filters, ReLU activation, stride padding 1.
4. Global Average Pooling: reduces spatial dimensions 1 64.
5. Fully Connected Layer: 64 units, ReLU activation.
15

---

Published as a conference paper at ICLR 2026
6. Output Layer: 4 units (corresponding to actions ,,,4,,,,1) followed by softmax to produce action probabilities.
This design ensures near-constant computational cost regardless of image resolution, as the final layers operate on a fixed-size vector.
Action Masking To prevent out-of-bound moves, we apply action masking during inference: the agent is at the image boundary (e.g., d 1), the "up" action masked (set zero probability) before softmax.
Training Procedure We train the scanning policy using Proximal Policy Optimization (PPO). The agent interacts with the image grid environment over episodes of fixed length T 10 steps, corresponding to a sparse sean path across the image.
At each step the policy network takes input the state tensor 8, R and outputs prob- ability distribution over the four movement actions. An action is sampled from this distribution during training for exploration. After executing dg, the agent transitions patch 9t41 and receives a reward r2 r(st, az 8t41), composed of adaptive revisitation incentive, exploration bonus, and neighborhood gain.
The total objective maximizes the discounted cumulative reward:
J(0) ET
Σ ===
with discount factor y 0.99. To stabilize training:
Reward normalization: We maintain moving average of recent rewards and normalize
each r online using zero-mean, unit-variance scaling.
Entropy regularization:
We include an entropy bonus Lentopy to encourage exploration in
early stages.
Gradient clipping: Global norm clipped at 0.5.
We use the Adam optimizer with learning rate 3 x 10 batch size 64 (aggregated over 8 parallel environments), and update the policy every = 10 steps using 3 PPO epochs per update. The value head (an additional output branch from the penultimate FC layer) is trained jointly to estimate state value (sx), with coefficient A 0.5 balancing the value loss. Training runs for 200K iterations on ImageNet-IK training set images resized to 512 x 512.
B EXPERIMENTAL SETUP DETAILS
B.1 SETTINGS FOR SEMANTIC SEGMENTATION.
We conduct semantic segmentation experiments on the ADE-20K and BraTS-2021 datasets, ADE- 20K contains 150 fine-grained semantic categories, with 20K images for training, 2K for validation, and 3K for testing. BraTTS22021 includes three semantic classes (tumor sub-regions), and we use T1-weighted MRI scans as input. The training, validation, and test sets contain 21 21K, 3K, and 6K slices, respectively. We adopt UperNet the base framework. During training, we use the AdamW optimizer with a weight decay 0.01 and a total batch size of 24. The learning rate initialized to 8 x 10-5, decayed linearly, and warmed up over the first 2,000 iterations. Total training runs for 180K iterations. Standard data augmentations are applied: random horizontal flipping, random rescaling within the range (0.5, 2.0), and random photometric distortion. At evaluation, input images are resized such that the shorter side is 512 pixels.
B.2 SETTINGS FOR OBJECT DETECTION.
We conduct object detection experiments on the MS-COCO 2017 dataset. The dataset contains 1 118K training images, 5K validation images, and 20K test images. We adopt the standard Cascade
16

---

Published as a conference paper at ICLR 2026
Mask R-CNN the base framework. For ViT-based backbones, we follow ViTDet and apply addi- tional designs such as interleaved window and global attention to handle high-resolution inputs. the original architecture without modifications. All other
For SSM-based Vim backbones, training and evaluation settings remain consistent across variants. We optimize using Adam W with weight decay of 0.1 and total batch size of 64. The learning rate is initialized to 1 x 10-4 and decayed linearly over 380K iterations.
C GENERALIZATION EXPERIMENTS
C.I FEASIBILITY OF THE INFORMATION SCORING WEIGHT IN OTHER TASKS
The weights wi and w2 in the information scoring module are determined via grid search on the ImageNet-IK IK validation set to maximize classification accuracy under a fixed computational bud- get. To validate the generalization of this optimal setting to other vision tasks, we evaluate the per- formance of InfoScan on two distinct downstream tasks semantic segmentation on ADE-20K and medical image segmentation on BraTS-2021 using the same fixed weights, without task-specific re-tuning.
Results in Table 7 demonstrate that the weight configuration optimized on ImageNet-IK (wi 0.6, w2 0.4) consistently yields the best performance across both datasets. This indicates strong transferability of the information scoring mechanism, suggesting that the relative importance of global color diversity (w1) and local texture complexity (wz) learned from natural images generalizes well to both complex scene parsing and fine aeee-graied medical analysis.
Table Ablation on information scoring weights wi (entropy) and we (variance) evaluated on ADE- 20K and BraTS-2021 The optimal weights (w) 0.6, wy = 0.4) selected on ImageNet- 1K achieve the highest mloU on both tasks, confirming their cross-task effectiveness.
W1
W2
ADE-20K mloU (%)
BraTS-2021 mloU (%)
0.5
0.5
36.7
18.7
0.4
0.6
42.7
20.4
0.6
0.4
45.8
22.3
C.2 GENERALIZATION EXPERIMENTS ON SEGMENTATION TASKS
We conduct cross-task and cross-domain generalization experiments in semantic segmentation. Specifically, we test whether models trained on one segmentation dataset can generalize to sig- nificantly different one without fine-tuning, simulating real-world deployment where target-domain labels unavailable.
We compare InfoScan-T with standard CNN (ResNet-50) and recent vision architectures (DeiT-T, Vim-T) under zero-shot domain transfer setting. All models use the same segmentation head UPerNet, are trained on one dataset, and directly evaluated on the other. Input resolution is fixed at 512 512. The two datasets represent highly distinct domains: Results are reported in Table 8, measured by mean Intersection-over-Uninn (mloU).
D LIMITATIONS
(i) First, the current implementation assumes a uniform patch size and grid structure, which may not optimally capture multi-scale semantics in high-resolution images. Future work could explore adaptive patching or hicrarchical scanning strategies.
(econn,, the information scoring module, though lightweight, introduces additional latency during inference. end-to-end deployment in real-time systems requires further optimization of the scoring and policy inference pipeline.
(ii))Thrr, the reward function contains hyperparameters (c.g., Ohigh A) that currently require mild tnning for extreme domain shifts (e.g., natural to medical). Although we observe consistent rank-
17

---

Published as a conference paper at ICLR 2026
Table Zero-shot cross-dataset generalization performance on semantic segmentation tasks. Mod- els are trained on one dataset and evaluated on the other without fine-tuning. InfoScan-T shows superior transferability, achieving higher mloU in both directions, indicating stronger generalization to unseen domains and modalities.
Backbone
Train Dataset
Test Dataset
mloU (%)
ResNet-50
ADB-20K
BraTS-2021
4.9
ResNet-50
BraTS-2021
ADE-20K
5.3
DeïT-T
ADE-20K
BraTS-2021
8.7
DeïT-T
BraTS-2021
ADE-20K
8.2
Vim-T
ADE-20K
BraTS-2021
7.8
Vim-T
BraTS-2021
ADE-20K
7.9
InfoScan-T
ADE-20K
BraTS-2021
11.4
InfoScan-T
BraTSS2002
ADE-20K
11.2
ing across settings, fully automatic adaptation without any validation feedback remains open challenge.
(iv) Finalln all experiments focus 2D images; extension 3D Videos would require re-designing the action space and scanning policy, which we leave for future work.
E
LLM USAGE
The manuscript was polished using large language model (LLM). After revision, the methodolog- ical and experimental details were verified and confirmed by the authors.
18


Reviews:
[{"review_id": "JDZdfjzvaH", "reviewer": "Reviewer_apcD", "summary": "This work aims to solve a challenging task of high-resolution visual representation learning which is constrained by the quadratic complexity of popular Transformer-based networks and fixed scanning patterns of Mamba. The authors improve the Mamba scanning mechanism by designing a reinforcement learning based methods learning the adaptive scanning policy. Instead of treating each image patch equally, this work combines local structural information and entropy to evaluate the information values of image patches, allocating more computational resources to the salient ones. Extensive experiments on various computer vision tasks demonstrate the favorable accuracy-efficiency balance of this work.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "a.\tThis work is well organized and easy to follow. The authors effectively identify the weakness of existing Mamba-based methods which overlook the content-adaptive perception, especially from the aspect of scanning strategy.\n\nb.\tThis work builds a series of vision backbones that achieve superior performance than existing methods with similar numbers of parameters.\n\nc.\tThe authors design a dynamic scanning strategy by building this task as a Markov decision process and solving it by the reinforcement learning algorithm.", "weaknesses": "i.\tThe authors only evaluate their methods on pure vision based tasks. Recently, the vision transformers also show high performance on the vision-language large models (VLLMs), such as LLava [1]. Could the proposed vision backbones be applied on these applications?\n\nii.\tIs the training process of the scanning policy stable? Please compare the computational costs in the training phase between your method and existing Mamba-based methods.\n\niii.\tFigure 1 is not mentioned and introduced in this manuscript.\n\niv.\tDoes the scanning policy bring any randomness in the testing phase? Is it sensitive to the image resolution? Please analyze the behaviors of the scanning policy if the training images and testing images have different resolutions.\n\nv.\tDoes the policy network have to repeatedly run on each network block for every image?\n\nvi.\tThe proposed method has too many new hyperparameters, such k2, k3, beta, than original Mamba.\n\n[1] Liu H, Li C, Wu Q, et al. Visual instruction tuning[J]. Advances in neural information processing systems, 2023, 36: 34892-34916.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "09IxVeTMJr", "reviewer": "Reviewer_UDg4", "summary": "This paper introduces InfoScan，a visual scanning method for a mamba-based vision backbone. The core innovation is an information-aware scanning mechanism that dynamically allocates computational resources to the most salient image patches. This is achieved through three key components: an Information Scoring Module that quantifies patch informativeness using Shannon entropy and local variance, a Patch Selection Model that optimizes patch size for efficiency and fidelity, and a Path Planning Module that uses Reinforcement Learning to learn an adaptive scanning policy. Extensive experiments on image classification, object detection, and semantic segmentation demonstrate state-of-the-art or highly competitive performance.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The motivation and method are novel.  The idea of replacing fixed, content-agnostic scanning patterns with a dynamic, information-driven policy is reasonable. The motivation is clearly grounded.\n\n2. The Methodology is good.  It rigorously formalizes the problem with a joint optimization objective and a reinforcement learning setup. The combination of information theory, variance, and RL is good, which may impact other researchers.\n\n3.  The paper provides extensive evaluations across multiple core vision tasks and domains. The results are compelling, consistently showing performance improvements over strong baselines (CNNs, ViTs, SSMs). The gains in zero-shot cross-dataset generalization are good.", "weaknesses": "1. While the overall model is parameter-efficient, the paper does not provide a detailed analysis of the latency overhead introduced by the Information Scoring Module and the iterative, sequential decision-making of the Path Planning Module during inference. \n\n2. The hyperparameters of the model architecture are not presented. More technical details should be well presented.\n\n3. The information importance evaluation method should be explained more solidly or provide more ablation studies.\n\n4. More previous Mamba-based vision backbones should be included.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5Ee2Z2wjls", "reviewer": "Reviewer_tKuV", "summary": "This paper introduces InfoScan, an information-aware dynamic scanning mechanism for high-resolution visual representation learning. Unlike fixed or global scanning methods in prior Mamba-based models, InfoScan allocates computation adaptively to the most informative image regions by combining entropy-based and structural analysis. It optimizes for both fine detail and contextual coherence through a reinforcement-learned adaptive scanning policy. Built on the Visual Information State Space (VISS) block, InfoScan achieves a superior efficiency–accuracy trade-off across various vision tasks, highlighting content-aware processing as a promising direction for efficient high-resolution vision models.", "questions": "1. What is the overall objective of InfoScan? How the policy model for path planning and downstream tasks are jointly optimized?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. InfoScan demonstrates strong performance and efficiency, significantly outperforming VMamba.\n\n2. The ablation studies on reward components, patch selection/path planning, and scanning methods are comprehensive, validating the effectiveness of the proposed approach.\n\n3. The design logic of InfoScan is sound—it allocates more computation to complex regions, effectively balancing performance and computational cost.", "weaknesses": "1. More visualizations of the learned policy model for path planning could be provided to illustrate how simple and complex regions correspond to different scanning patterns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NPj4I5PXh6", "reviewer": "Reviewer_7J1d", "summary": "This paper proposes InfoScan, an SSM-based visual foundation model developed to improve the scanning strategy from deterministic, which has been adopted in most existing related studies, to adaptive / content-aware. The authors propose an approach to estimate the importance of a certain image patch, and more inspiringly, a reinforcement learning-based formulation of the patch selection algorithm. Experimental results demonstrate the clear performance gain brought by the InfoScan method and the contribution of each newly proposed modules.", "questions": "Please refer to the 'Weaknesses' section for the details of my concerns. I am willing to improve my ratings if the first three concerns are addressed by concrete experimental results and illustrations.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "Follow the instruction, the strengths of this manuscript are summarized in the following four aspects:\n\n1. **Originality**: The motivation of this study is intuitive and reasonable. Although it may be straightforward or even easy to think of the idea, this work is one of the first studies that works it out (to the best of the reviewers' knowledge).\n\n2. **Quality**: This manuscript is generally of good quality, including the contant organization of language usage.\n\n3. **Clarity**: The clarity of the manuscript is satisfactory. The central idea can be received despite the appendix is frequently mentioned to help make clear of some technical details.\n\n4. **Significance**: This work may be inspiring to researchers in fields like (Visual) SSMs and representation learning.", "weaknesses": "Although this work is somehow inspiring to me, there is much room for further improvement and more information is required to develop a more comprehensive evaluation.\n\n**1. The Computation Overhead:** The content-agnostic scanning patterns are there for a reason. It is crucial to quantify the extra burden imposed by the new content-aware modules. Figure 1 reports FPS and GPU memory versus Vim-Ti and DeiT-Ti, yet omits VMamba, which is the principal baseline throughout the paper. To be concrete, a detailed comparison on the computational complexity in terms of throughput (train and test), GPU memory, FLOPs are needed to gain a comprehensive evaluation of the efficiency of the proposed method. Without these metrics, it is difficult to judge whether InfoScan’s accuracy gains justify its added complexity.\n\n**2. Qualitative Analysis**: InfoScan is motivated by its ability to adapt the scan path to image content, but no qualitative evidence is shown. Visualizations of the learned scanning routes overlaid on sample images (or attention heatmaps) would help readers verify that the model behaves as intended. Quantitative scores alone cannot reveal whether improvements stem from genuine content awareness or simply from a larger, more intricate architecture.\n\n**3. Insufficient Benchmark Methods:** In the main quantitative experiments, the authors mainly compare InfoScan to the benchmarking CNN and ViT models, as well as VMamba and Vim, which are too restricted. More CNN/ViT/SSM-based models recently proposed in top venues should be included for comparison. For instance, Deformable Mamba, another Visual Mamba-related work with deformable scanning route, appearing in CVPR 2025, should at least be included for comparison.\n\n**4. Heavy Manual Design:** InfoScan relies on numerous hand-crafted mechanisms, hyper-parameters, and loss terms, raising questions about its generalizability and robustness. Please discuss the sensitivity of performance to these design choices and, if possible, include ablation studies or automatic tuning results to demonstrate stability across datasets and tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)