Multi-Kernel Correlation-Attention Vision Transformer for Enhanced Contextual Understanding and Multi-Scale Integration
Authors: Hongkang Zhang
Shao-Lun Huang
Ercan Engin KURUOGLU
Yanlong Wang
Venue: NeurIPS 2025 poster ()
Keywords: Multi-Kernel Vision Transformer
HGR Correlation Learning
Dynamic Feature Fusion
Efficient Attention Mechanism
Multi-Scale Representation Learning
Primary area: deep_learning
TL;DR: We propose MK-CAViT, a multi-kernel Vision Transformer with HGR-based correlation attention, achieving efficient multi-scale feature learning.

Abstract: Significant progress has been achieved using Vision Transformers (ViTs) in computer vision. However, challenges persist in modeling multi-scale spatial relationships, hindering effective integration of fine-grained local details and long-range global dependencies. To address this limitation, a Multi-Kernel Correlation-Attention Vision Transformer (MK-CAViT) grounded in the Hirschfeld-Gebelein-Rényi (HGR) theory was proposed, introducing three key innovations. A parallel multi-kernel architecture was utilized to extract multi-scale features through small, medium, and large kernels, overcoming the single-scale constraints of conventional ViTs. The cross-scale interactions were enhanced through the Fast-HGR attention mechanism, which models nonlinear dependencies and applies adaptive scaling to weigh connections and refine contextual reasoning. Additionally, a stable multi-scale fusion strategy was adopted, integrating dynamic normalization and staged learning to mitigate gradient variance, progressively fusing local and global contexts, and improving training stability. The experimental results on ImageNet, COCO, and ADE20K validated the superiority of MK-CAViT in classification, detection, and segmentation, surpassing state-of-the-art baselines in capturing complex spatial relationships while maintaining efficiency. These contributions can establish a theoretically grounded framework for visual representation learning and address the longstanding limitations of ViTs.

Multi-Kernel Correlation-Attention Vision Transformer for Enhanced Contextual Understanding and Multi-Scale Integration
Hongkang Zhang, Shao-Lun Huang: Ercan Engin Kuruoglu, Yanlong Wang Tsinghua Shenzhen International Graduate School, Tsinghua University 2hanghk21@mails. @mails. tsinghua, edu. cn, shaolun huang@sz. tsinghua. edu. cn kuruogludsz. tsinghua.enn cattua.eddu.com edu.en, en, Langyan121@mails tsinghua. edu. cn
Abstract
Significant progress has been achieved using Vision Transformers (VīTs) com- puter vision. However, challenges persist in modeling multi-scale spatial rela- tionships, hindering effective integration of fine-grained local details and long- range global dependencies. To address this limitation, Multi-Kernel Correlation- Attention Vision Transformer (MK-CAViT) grounded in the Hirschfeld-Gebeleinn- Rényi (HGR) theory was proposed, introducing three key innovations. A parallel multi-kernel architecture was utilized to extract multi-scale features through small, medium, and large kernels, overcoming the single-scale constraints of conven- tional ViTs. The cross-scale interactions were enhanced through the Fast-HGR attention mechanism, which models nonlinear dependencies and applies adaptive scaling to weigh connections and refine contextual reasoning. Additionally, a stable multi-scale fusion strategy adopted, integrating dynamic normalization and staged learning to mitigate gradient variance, progressively fusing local and global contexts, and improving training stability. The experimental results on ImageNet, COCO, and ADE20K validated the superiority of MK-CAVIT in classification, detection, and segmentation, surpassing slate-of-the-arr baselines in capturing com- plex spatial relationships while maintaining efficiency. These contributions can establish theoretically grounded framework for visual representation learning and address the longstanding limitations of ViTs.
Introduction
In recent years, deep learning, particularly Transformer-based models, has demonstrated remarkable effectiveness in handling large-scale, high-dimensional, and multi-modal data ( Initially developed for natural language processing (NLP) tasks, Transformers [2] have been recognized for their capacity to capture long-range dependencies and complex relationships within sequential data, thereby becoming foundational to sequence modeling. With the emergence of Vision Transformers (ViTs) (3), this architecture has been successfully extended to computer vision tasks, including image classification 4 5 6) object detection 7 g and multi-modal learning g
10 11 These models centered on the self-atrention mechanism, which enhances token representations by encoding statistical correlations among sequence elements. Through global self-attention enabling each token attend all others, ViTs effectively model detailed patterns across entire images [12), surpassing convolutional neural networks (CNNs) in capturing long-range interactions beyond localized receptive fields [3 [4]. Despite their demonstrated effectiveness, ViTs remain largely empirically driven and lack rigorous mathematical foundation. The outputs of self self-attention layers are often interpreted
"Corresponinng author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

heuristically with ambiguous statistical and geometric relationships between data distributions and the resulting representations [6]. Moreover, vision tasks involve -igeniimensional inputs with complex spatial dependencies, rendering the global self-attention computationally burdensome. The Softmax-based attention mechanism [17] introduces quadratic complexity with respect to the number of tokens, thereby presenting scalability challenges for high-resolution visual data.
To address these limitations, multiple approaches have been developed reduce the computational demands of self-attention while preserving the capacity to model long-range dependencies. The Swin Transformer [8] and Pyramid Vision Transformer (PVT) 19 decrease computational costs by restricting attention to local windows or employing sparse attention mechanisms, respectively. Recent state-of the-ttt-rt methods, including Vis [20] and Agent-Swin 2N, further enhance efficiency through hierarchical feature merging and reinforcement learning- basen attention selection. Despite their effectiveness in reducing computational complexity, these approaches have two significant limi- tations: first, they frequently depend on heuristic design choices lacking solid theoretical foundation for modeling mcale feature dependencies; second, their emphasis on local or sparse interac- tions inherently compromises the capture of global dependencies, critical capability of full global self-attention that remains essential for tasks demanding holistic scene understanding. Additionally, the application of pre-trained Transformer models to new tasks architectures often encounters compatibility challenges, necessitating substantial modifications to the training process. Typically, ViTs rely fixed image patch structure and experience difficulty in transferring parameters across tasks. In contrast to CNNs, which can extract local features through sliding windows, ViTs utilize self-attention to model dependencies between image patches across the entire image. Although this method effectively captures the global context, it introduces inefficiencies, particularly in capturing fine-grained details, such as textures and edges.
Recent studies have proposed certain hybrid architectures that integrate CNNs with Transformers 22 23 24, leveraging the strengths of CNNs for local feature extraction and Transformers for global context modeling. However, the incorporation of CNNs into ViTs increases the computational complexity primarily because of the global self-attention mechanism, which amplifies the compu- tational demands and reduces training and inference efficiency. Although lightweight CNNs and optimized VIT architectures have been explored [25] 26 27 28 29), achieving a balance between computational cost and model performance remains a significant challenge. Against this backdrop, a novel framework was proposed to enhance ViTs with the following contributions:
Multi-Kernel
inttllationAAtentttin Vision Transformer (MK-CAViT): MK-CAVIT was intro- duced as an advanced Vision Transformer that integrated the correlation attention with multi-kernel architecture. Parallel kernels of varying sizes were employed to extract features at multiple resolutions. This unified design addressed the single patch size limitation of Transformers, enabling the modeling of both local and global dependencies. Through the fusion of multi-scale semantic information, MK-CAW IAACTVIT enhanced the feature representations and established robust foundation for visual tasks.
Hierarchical Multi-Scale Feature Correlation Strategy: A two-stage hierarchical fusion strategy was proposed to facilitate effective cross-scale integration. Learnable gating dynamically combines small-kernel local details with medium-kernel features, preserving spatial precision before incorpo- rating large-kernel global context. This dynamic weighting aligns local and global cues, enhancing model robustness and generalization for multi-scale object recognition in complex scenes.
Efficient HGR-Based Correlation Attention Mechanism: A Fast-HGR correlation mechanism was developed to efficiently model nonlinear feature dependencies, grounded in Srrschfeld-Gebelee Rényi (HGR) maximal correlation theory. This mechanism utilized cosine similarity for local feature alignment and incorporated trace regularization to enforce global consistency. These design choices preserved the theoretical advantages of HGR while significantly reducing computational complexity. focused attention on relevant regions, reduced noise, and modeled cross-scale feature interactions, thereby enhancing feature discriminability across various visual environments.
Unified Multi-Scale Attention Framework: MK-CAVIT established unified framework merging correlation attention with multi-kernel pathways to model local and global dependencies. The multi-resolution features were extracted through parallel kernels and calibrated using the Fast-HGR mechanism to ensure semantic consistency. A dynamic attention strategy was applied to adaptively weigh the features across granularities. enhancing the representations for challenging targets, such as small objects and ambiguous boundaries. This architecture can provide an efficient solution for both image-level and pixel-level tasks by effectively integrating fine details with the global context.
2

---

2 Related Works
2.1 Vision Transformers
ViT's have transformed computer vision by adapting self-attention mechanisms to capture long-range dependencies [30]. Recent developments have focused on two primary challenges: the efficient learning of multi-scale features and robust modeling of local-global interactions [2]. The Focal Transformer [33 introduces a hierarchical self-attention mechanism that combines fine-grained local attention for nearby tokens with coarse-grained global attention through pooled summaries for distant tokens. This design reduces computational complexity while effectively capturing multi-scale dependencies, achieving state-of- tt-the-art performance in dense prediction tasks. In parallel, MPV [34] utilizes multi-scale patch embedding with parallel transformer paths, extracting diverse features from overlapping convolutional patches (c.g., 3x3, 5x5, and 7x7) and aggregating them to enhance multi-scale representation. These approaches have underscored the significance hierarchical and parallel processing in capturing spatial details and global context. Existing multi-scale modeling approaches can be classified into three main paradigms: (1) structural pyramid designs, such as PVT [19 351 and MVIT (36), which apply spatial reduction operations across stages; (2) window-based hybrids, including Swin [8] and CSwin [37], which balance the local attention within windows using shifted window strategies; and (3) dynamic atlention mechanisms such DAT (38 and BiFormer
39. which adaptively adjust receptive fields through deformable or routing operations.
Although these methods have advanced in the field, critical limitations persist. The hierarchical attention of the Focal Transformer introduces complex window- -oaaullggaaul interactions that increase memory consumption, while MPVIT's multi-path structure encounters difficulties in cross-scale feature fusion. Moreover, most existing approaches rely on simple dot-pre dot-product attention to measure feature correlations, potentially neglecting complex nonlinear dependencies among multi-scale features. To address these challenges, a novel multi-kernel correlation attention mechanism was proposed that integrates the HGR maximal correlation with parallel pathway fusion enable more effective cross-scale dependency modeling.
2.2 Maximal Correlation in Deep Learning
The Hürschffld-GGbbllein-Reee (HGR) maximal correlation [40 41 42 provides theoretically grounded framework for measuring nonlinear dependencies, and offers clear advantages over con- ventional linear correlation metrics [43] Recent developments have adapted HGR principles for deep learning via Soft-HGR proximations. This adaptation enables practical deployment in neural networks while preserving the HGR's s capacity to capture the maximal information between feature representations. based methods have evolved in three main directions: (1) enhancement of computational efficiency through covariance trace optimization 45 (2) improvement of multimodal fusion via joint covariance-trace constraints, and (3) stability optimization through eigenvalue normalization techniques.
which substitutes strict whitening constraints with low-rank ap-
However, current implementations focus more on feature embedding alignment than on designing attention mechanisms. This paper introduces three key innovations for integrating HGR into VITs:(1) a dynamic covariance projection that adapts to varying feature scales across transformer layers, (2) multi-kernel trace constraints that stabilize the correlation computation across parallel pathways, and (3) gradient-awar whitening that facilitates end-to-end learning without explicit matrix inversion. These advancements enable the effective integration of HGR principles into attention mechanisms while preserving compatibility with standard transformer optimization processes.
The improved Soft-HGR formulation in (q) addressed two primary limitations of prior imple- mentations: (1) the variance instability in high-dimensional features was mitigated through trace regularization, and (2) cross-kerner compatibility was achieved via dimension- aware covariance projection, enabling effective correlation measurement within the proposed multi-kernel framework:
L1-SoftHRR
=
Σ
E( (X)
v. v.1. ((Eooo(T))= (Cteei)))GGoooooooooooo (_ooo(oov((()))- (cou (c)o(((()))ov(((()) ))) B(covig)) (0))))))((()
fr
..E.. (...tt))..ccvie.. K(g) Rig)=o,cavigia
(1)
where f(X) and gYY) denote feature mappings or transformations of inputs X and Y, respectively.
3

---

2.3 Key Differentiation
Inpur Image
The proposed MK-CAVIT integrated principles from vision transformer ar- chitectures and the correlation measure- ment theory. In contrast to MPViT's separate pathway processing 34 ex-
Frature Extraction
Multi-calle Tokenization
Large Kerzel 15x15 Com
Mid Komel 7x7 Conv
Small Kerne Conr
plicit correlation channels were estah- lished between the multi-kernel fea- tures through HGR-based attention. Unlike the granularity-level attention employed by the Focal Transformer [33]. simple concatenation was re- placed with learned correlation weight- ing enabling dynamic importance al- location across scales. While prior studies have focused either on architec- tural multi-scale designs (e.g., MPViT) or attention mechanism enhancements (e.g., Focal Transformer), MK-CAVIT was the first to unify both under an tnformation-theoretiic framework. By explicitly optimizing cross-scale fea- ture correlations through HGR maxi- mal correlation, we established novel paradigm for scale-aware visual rep- resentation learning, addressing both computational efficiency and theoreti- cal robustness.
Project @ A3, 12
Project (?), Ri, ja
Project (*. K3,
Module
Fas-HRiR
1-00 A)
Sodlaast F-HORID.))
Altentiong: K.1)
Multi-Scale Fusion
Lacal-Miil Fusim
Aitei-Atttionn Pap ae
Global Conrest Integration 7-Attention Aa AngPool
Arau
Adaaptive Multi-Head Artention Abuata (agi-Attsniinna
Final Outpet
3 Methodology
Z- WF--Concan, Racks "
Les "plaan
Figure I: Overall Framework of MK-CAVIT The architec- ture processes input images through three parallel pathways with different kernel sizes to capture multi-scale features. It employs Fast-HGR correlation attention for modeling nonlinear dependencies, followed by adaptive multi-scale fusion that dynamically balances contributions from differ- ent scales.
This section presents MK-CAVIT, an enhanced ViT model that integrates correlation attention with a three-path multi-kernel structure. Unlike stan- dard ViTs employing a single patch size, the proposed model utilized small, medium, and Barge-kerne pathways to capture fine-grained local details alongside global contextual dependencies.
3.1 Fast-HGR Correlation
To address the computational inefficiency of exact HGR computations while preserving their ability to model nonlinear feature dependencies, theoretically grounded approximation was developed. This approach utilized cosine similarity and trace regularization, inspired by Soft-HGR and improved Soft-HGR, while removing expensive whitening.
3.1.1 HGR Maximal Correlation Revisited
Let X Rd and Y e d be random vectors representing features from different network branches. The k-dimensional HGR maximal correlation o (kk) (X, Y) is defined as:
(k) (X,)) sup [[x(x)T g(Y)) s.t. [[ E[g] 0, cov(f) cov(g) I.
where / and y are measurable functions, and (-) denotes covariance. This measures the strongest statistical dependence between X and Y, with p = 0 indicating independence and p 1 indicating deterministic dependence. For linear f(X) W WX, gYY) = VY, the optimum reduces to cosine similarity, motivating our efficient approximation.

---

3.1.2 Fast-HGR Approximation Derivation
The HGR objective was approximated by replacing the computationally expensive covariance whiten- ing with batch-witd normalization and combining the cosine similarity for local token alignment with trace term to enforce global distributional consistency. Soft-HGR substituted the strict whitening constraints with a soft regularizer, preserving the feature geometry of the original HGR formulation. To address the Soft-HGR's sensitivity to signal variance, the improved Soft-HGR introduced addi- tional variance constraints. In contrast, the proposed Fast-HGR (F-HGR) approximation removed orthonormality requirements while retaining directional alignment.
f(x,) go)s))
F-HGR(f(X). - [f((z)) Ilo + X to((coonn (x)
(3)
-
Cosine Similarity for Local Dependence. For feature vectors f(zz)) E Rd in batch of size N, the cosine similarity between each pair is:
cos(f(z;), = f(x,) g(y)
(T(xx)) (
This efficiently captures pairwise local dependencies with lower computational cost than full covari- ance matrix operations.
Trace Regularization for Global Structure. To retain second-order statistical information, trace term over centered feature covariances is introduced:
tr(cov(f(X)) cov(g(Y))) = (J(X) (J(X) (o()) ()) Ø)
(5)
N
where ff and are the batch means. This term measures the alignment of the feature distributions across the entire batch, ensuring global consistency without explicit whitening.
Combined Formulation. The Fast-HGR correlation score balances local and global terms:
1 1 F-HGR(((X): N
cos(f(r;). A tr (cov(f(X)) cov(g(Y))) (6) N j=1
Σ
where À is hyperparameter. This formulation inherits the theoretical guarantee that it upper-bounds the true HGR correlation under linear transformations, with error bound O(1v/N) due to batch-wise approximation.
The cosine similarity normalized the feature vectors to the unit hypersphere, constraining the F-HGR values within [-1, I] and stabilizing the gradients without the layer normalization. This is aligned with the Transformers without the Normalization [46], where bounded activations (e.g.. tanh) prevent gradient explosion or vanishing. The theoretical analysis demonstrated the Lipschitz continuity of the F-HGR operator, thereby ensuring stable training in deep architectures.
3.2 Multi-Kernel Correlation Attention Vision Transformer
MK-CAViT integrates multi-scale tokenization, F-HGR attention, and hierarchical fusion to model cross-scale dependencies. The overall architecture of MK-CAViT is illustrated in Figure [
3.2.1
Multi-Scale Tokenization
Given input image X E RBxC*H*W three convolutional tokenizers with different kernel sizes were applied to capture the hierarchical features:
Xprtch Conv2D(X, Kg. S;); Xpmich Conv2D KT" T SSp"): X pucch = Conv2D(X, Ky St)
(7)
where, K2. KT and K3 are the small, medium, and large kernels, Sy Sp and St. are their respective strides, with X
patch E B x xH,WW, for E m.
After tokenization, the feature maps were linearly projected into query, key, and value embeddings: Q' = Wi Flatten(X Kpach): K' W Flatten X ppach)); Vi W Flatten X (pass))
(8)
where W 4: W W E RdxC are the learnable projection matrices, and Flatien represents the flattening of the spatial dimensions into token sequences.
5

---

3.2.2 Fast HGR Correlation Attention Mechanism
Traditional self-attention mechanisms rely on Softmax -base similarity computation, which may suppress subtle feature dependencies. In contrast, F-HGR correlation attention was employed to capture the relationships between the query and key pairs more effectively:
= F-HGR(Q' K E {s, m,
(9)
where 11 directly represents the pairwise dependencies between tokens.
From Eq. (3), correlation is computed as:
(2)) K3 @3
H = N
+ A (ovov((""c)(("))
(10)
The first term measures local token dependencies via cosine similarity; the second term encodes global structural dependencies via trace operation. This formulation preserves both local token interactions and global feature consistency. The correlation attention mechanism is defined as:
Attention(() (Q', K" V*) = Softmax F-HGR(Q' K') V
(11)
Vd
This approach employs HGR to compute correlations between query and key matrices, replacing conventional dot-product similarity. Consequently, the model captures intricate feature interactions while avoiding computational inefficiencies of traditional HGR methods.
3.2.3 Multi-Scale Fusion
The multi-scale fusion mechanism combined features through gated Fast-HGR correlations with theoretical guarantees of stability. Let Q", K* BxxNxx and ,T E BBBxNmxe denote the
Km
queries/keys from the small- and mid-kernel pathways respectively, with V RBxNixd as the large-kernel values.
Local-Mid Fusion (Small + Mid-Kernel): Small-kemell (s) and mid-kernel (m) features are fused using gated F-HGR attention:
Aem = (a)) Attention(Q",, K", V") + oBB) Attention(Q",, V*)
(12)
where 3 B E Rd are learned gating vectors, a--) denotes sigmoid activation, and VMm V are mid/small-kernel value matrices. This balances contributions from cross-kernel value interactions.
Global Context Integration (Local + Large-Kernel) The fused features Asm then interact with global context through adaptive mixing:
Annal 7 Attention(Aem Aem. V²) (1 7) AvgPool(Aani
(13)
where the mixing coefficient is dynamically computed from global context, enabling task-aware scaling where small-kernel features emphasize fine details and large-kernel features prioritize global patterns.
3.2.4 Adaptive Multi-Head Attention
Traditional multi-head attention mechanisms assign fixed contributions to each head during training and inference. However, input complexity varies, and not all attention heads contribute equally. In simpler image regions, certain heads may capture redundant information, increasing computational cost. To address this issue, adaptive mulli-head attention mechanism was introduced within the MK-CAVIT. This mechanism adjusted each head's 's contribution based on the input complexity using a lightweight gating module. Each attention head f was associated with learnable gating parameter Gh. which was either trained through a simple network or learned as a standalone variable. The output of the h-th attention head, Aheadh, computed
A (g)) Attention, (Qr,, Kn, V%)
(14)
where (-) maps the learnable gating parameter Gh. to (0, 1) to scale the output, and Attention, denotes the attention computation for head h.

---

When the imput is simple, the model can reduce the gating parameter JA toward zero for less relevant attention heads, thereby lowering computational cost. For more complex imputs, 9h increases toward one for critical heads. For instance, attention heads may be deactivated in background regions and fully utilized in target-relevant areas.
In MK-CAViT, an adaptive attention mechanism is integrated with a multi-kernel architecture and F-HGR computation. The multi-kera structure extracts multi-scale features, F-HGR captures cross- scale dependencies, and the gating mechanism optimizes processing efficiency. Features are projected into queries, keys, and values. F-HGR computes correlation scores, and the gating module adjusts attention outputs. The final representation is derived from gated, correlated feature interactions.
Z = W3- Concatt (Auead, Ancad, H
(15)
where Wa is learnable matrix and H is the number of heads. This integration optimizes performance varying input complexities.
3.3 Loss Function
The proposed loss function combines cross-entropy loss LCE and F-HGR loss LI-OO- The cross- entropy loss ensures prediction accuracy by minimizing the discrepancy between predicted and true labels, whereas the F-HGR loss promotes feature correlation, thereby enhancing the model's capacity to capture data dependencies. The overall loss function is defined as:
L LCe JLL-HOR
(16)
where i controls the F-HGR term's influence, and LF-HG computed using the F-HGR scalar defined in Equation (10). This combination enables the model to achieve high predictive accuracy while maintaining robust feature correlation, improving suitability for complex tasks.
Table 1: Classification comparison on ImageNet- 1K dataset.
Experimental Results and Analysis
MK-CAViT was extensively evaluated across im- age classification, object detection, and semantic segmentation tasks. The component contributions were assessed through ablation studies, and the efficiency and scalability were analyzed. All ex- periments adhered to standardized protocols to ensure fair and consistent comparisons.
Model
#Params(M) FLOPs(G) Top-I(Si
ResNet-50
25.0
76.2
Deil-Small/16
22.1
Swin-Tiny
28.3
81.3
ConvNeXi-T
28.6
4.5
82.0
Agent-Swin-T
29.0
4.5
82.6
FasterViT-O
31.4
3.3
82.1
Focal-Tiny Tiny
29.1
4.9
82.2
MPVIT-S
22.8
83.0
MK-CAVIT-Tiny
22.7
4.6
83.5
4.1 Dataset and Baseline Selection
ResNet-101
45.0
77.4
Swin-Small
49.6
8.7
83.1
Three benchmark datasets used evalu- ate the model performance: ImageNet-1K [47), COCO (48), and ADE20K [49]. MK-CAVIT was compared against state-of-the-arr ViTs and CNNs, which
ConvNeXt-S
50.2
8.7
83.1
Agent-Swin-S
50.0
8.7
83.7
Focal-Small
51.1
SA-CCAlTT-mall
49.7
categorized by model scale (Tiny, Small, Base) to ensure the fair compari- son. The baseline models included: ResNet [50],
ResNet-152
60.0
78.3
VIT-Basc/16
86.6
17.6
77.9
DeiT-Base/16
86.6
17.5
81.8
Swin-Base
87.8
15.4
83.4
ResNeXt [51). ViT 3. DeiT [4) Swin [18. Con-
ConvNeXt-B
88.6
15.4
83.8
vNeXt (52). Focal Transformer 33, MPVIT
Agent-Swin- Base
88.0
84.0
34
Agent-Swin 21), and Faster VIT
20
FasterViT-BI
87.6
FasterViT-3
159.5
18.2
84.9
Focal-Base
89.8
16.0
83.8
4.2 Image Classification on ImageNet-1K
MPVIT-B
74.8
16.4
84.3
MK-CAVTT-Base
88.0
15.6
85.6
Table demonstrates MK-CAViT's superior ac- curacy across model scales. The Base variant achieves 85.6% Top-1 accuracy, surpassing Agent- Swin-Base (84.0%) and FasterViT-B1 (84.8%) while maintaining comparable computational cost.
FasterViT-4
424.6
36.6
85.4
Agent-Swin-Laan
197.0
11.8
85.2
ConvNeett-Lage
198.0
MIK-CAVTT-Large
186.0
28.9
86.1
The consistent improvements across Tiny (83.5%), Small (84.3%), and Base scales validate the

---

effectiveness of HGR-cone Core-corrl attention in capturing nonlinear feature dependencies that heuristic- based multi-scale methods miss. For large-scale variants, MK-CAViT-LLIE achieves 86.1% 1% Top-1 accuracy, outperforming FasterViT-4 (85.4%) with 54% fewer parameters and 21 % lower FL.OPs, demonstrating superior scalability the correlation attention framework.
Table 2: COCO object detection and instance seg- mentation with RetinaNet and Mask R-CNN (1x schedule).
Table 3: Semantic segmentation on ADE20K using UperNet.
Model
aPanan[MM) FLOPP(() mlol((%) mAcc|
Model
APrrms(M) FLOPs(G) RetimeNet
Mask-R R-NN
Swin: Tiny
60
945
44.5
55.6
AP
AP
AP
APs
ConvNeXi-T
59
939
46.7
58.2
ResNet-50
Agenr-Swin-T
61
58.5
Swin-Tiny
Focal-Tiny
MPVIT-S
47.8
48.8
291
278
43,7
42.0)
43.7
44.8
41.0
39.8
26.8
25.3
MPVIT-S
Focal-Tiny
FasterViT-2
62
76
998
974
45.8
47.2
57.2
59.7
58.8
MK CAVIT-Ting
41.3
236
46.7
48.0
43.6
29.5
MI-CAViT-Tiny
58
940
49.5
60.2
ResNet-101
63.2
336
38.5
40.4
36.4
23.9
Swin-Small
81
1038
47.6
58.4
PVTM
302
41.9
39.0
ConvNeXt-S
59.5
Swin-Small
69.1
45.0
46.5
42.1
28.9
Agent-Swin-S
81
1043
48.1
59.8
Focal-Small
71.2
401
45.6
47.4
42.8
29.3
FisterViT-3
98
1076
48.7
59.6
MK-CAVIT-
65.3
315
47.5
49.1
44.3
30.7
58.5
Small
MK-CAVIT-
80
1055
50.2
60.9
Small
ResNeXt101-
102.8 0
493
41.0
42,8
38
25.2
64x4d
Swin Base
121
1188
48.1
59.1
Bwin-BBse
ConvNeXt-B
120
1170
48.9
59.8
Focal Bocal-Buse
110.0
533
46.3
47.8
43.2
30.1
Ageur-Swin-B
60.0
MPVTT-B
95.0
503
47.2
48.5
13.8
30.5
VisterVIT-4
136
1290
49.1
60.3
Agens-wwn.BB Swin B
112.3
501
47.9
49.0
44.0
30.5
Tocal Base
126
1354
49.0
59.6
VIT-R1
111.8
44
MPVIT-B
MK-CAVVT-Bave
93.2
481
48.7
50.3
45.1
31.9
MI-CCAVI-Barrs
113
1182
50.8
61.7
4.3 Object Detection and Semantic Segmentation
COCO Object Detection: The integrating of MK-CAViT as a backbone in RetinaNet [53 and Mask R-CNN [54) (Table 2) demonstrated significant performance improvements, particularly for small objects. MK- -AViT-TTTyn achieved 48.0 0 AP and 43.6 APm outperforming Swin-Tiny (43.7/39.8) .8), Focal-Tiny (44.8/41.0), and MPVIT-S (46.5/42.9). MK-CAViT-Bass achieved 50.3 AP and 31.9 AP$ (bbecl-object AP), outperforming Agent- -went-Swin--oe Swin-B(49.0 APh, 30.5 AP*) and FasterViT-B (49.1 AP 30.8 ).S). The 3x3 kernel in the multi-scale design preserves fine-grained details critical for small-object localization, while the 15x15 kernel provides global context to reduce false positives.
ADE20K Semantic Segmentation: Using UPerNet 55 as the decoder (Table 3). MK-CAViT-Tiny achieved 49.5% mloU, surpassing Swin-Tiny (44.5%), ConvNeXt-T (46.7%), and MPViT-S (48.3%) by notable margins, indicating the strong fine-grained feature extraction capability. MK-CAViT-Base reached 50.8% mloU, outperforming MPViT-B (50.3%) and FasterViT-4 (49.1%) 16). The multi-scale feature fusion mechanism ensures accurate boundary localization and effective context aggregation, both which are critical for pixel-level prediction.
4.4 Necessity of Multi-Scale Design
Table 4: Performance comparison of single-scale versus multi-scale kernel configurations across vision tasks.
To validate the necessity of multi-scale processing, Table 4 compares single- kernel configurations (3x3, 5x5, 7x7, 9x9, 11x11, 1, 15x15) against the multi- kernel (3/7/15) design across three benchmarks. The multi-kernel model
Configuration
ImugeNet Top-1 (%)
COCO AP (%)
ADE20K
AP (%)
mloU (%)
3x3 only
43.8
5x5 only
83.0
43.8
26.9
44.1
7x7 only
83,2
43.6
26.1
44,3
only
lixll only
82,6
42.5
24.8
43.4
outperforms all single-scale variants by meaningful margins: 2.4-3.0% in Ima- geNet Top-1 accuracy, 6.5% in COCO APh, and 6.5-7.7% in ADE20K mloU. This consistent performance gap confirms that no single kernel size captures the full spectrum of visual features needed for diverse vision tasks. Specifically, single- scale designs exhibit inherent limitations: Small kernels (3x3, 5x5) achieve competitive small-object detection (APS 26.9-27.3 (..-2%7.%%) but lack global context, hindering performance on context-dependent tasks. Large kernels (1xx11, 15x15) over-smooth fine-grained features, resulting in the lowest ADE20K mloU (43.1%) and COCO AP$ (24.7%) among all single-scale variants. Mid-sized kernels (7x7) underperform the multi-kernn model by 2.4% (ImageNet) and 6.2% (COCO AP*), they
15x15 only
Multi-Kerma (3/7/15)
85.6
50.3
31.9
50.8

---

cannot integrate fine details and global context. Notably, the multi-kerne' model's superiority stems from complementary synergy: 3x3 kernels preserve texture/edge details, 15x15 kernels capture global scene structure, and 7x7 kernels mediate cross-scale interactions. These strengths combine to produce gains that exceed the sum of individual single-kernel performance coiimannce-oofimiinng multi-scale design is essential for comprehensive visual understanding.
4.5
Ablation Studies
Comprehensive ablations validate MK-CAViT's design choices, with results quantified in Table
Fast-HGR Module: Removing this feature alignment component results in consistent performance degradation: ImageNet Top-1 accuracy decreases by 0.9%, COCO APb by 1.5%, and ADE20K mloU by 0.8%. Small-object detection is particularly affected, with COCO AP$ dropping by 2. %. Training convergence slows by 25%, highlighting the module's critical role in enhancing gradient quality through maximizing feature-target correlations.
Hierarchical Gating Fusion: Replacing the two-stage gating mechanism with naive concatenation or element-we wise addition degrades performance: ImageNet Top-I decreases by 0.8%, COCO Aph by 1.1%, and ADE20K mloU by 0.9%, while increasing FLOPs by 14%. This confirms the gating mechanism's 's efficiency in mediating cross-scale information interaction.
The hybrid multi-token attention mechanism achieves an optimal balance between efficiency and accuracy, Dense global attention provides marginal 0.2% improvement in ImageNet Top-1 but increases FLOPs by 50%, rendering it computationally impractical. In contrast, sparse local attention reduces FLOPs by 20% but causes a 2.2% drop in COCO APb, validating the hybrid design's superiority for multi-task performance.
Attention Mechanism:
Dynamic Normalization:
Replacing dynamic normalization with static LayerNorm reduces Ima- geNet Top-1 1 by 0.5%, while BatchNorm induces more severe declines: 1.3% in ImageNet Top-1, 1.9% in COCO APb, and 1.7% ADE20K mloU. Dynamic normalization also enhances robustness, achieving 1.7% lower mean corruption error (mCE) on ImageNet-C compared to LayerNorm.
Kernel Configuration: The 3x3/7x7/15x15 kernel combination is confirmed as optimal through comprehensive kernel configuration analysis. Smaller kernel sets (3x3/5x5/7x7) result in a 1.9% loss in ADE20K mloU due insufficient global context capture. The 5x5/9x9/13x13 configuration achieves competitive ImageNet accuracy (85.1%) but underperforms on small-object detection (30.7% APS) and segmentation (49.6% mloU), indicating the critical importance of the 3x3 kernel for fine-grained feature preservation. Similarly, the 3x3/9x9/15xxl configuration shows improved small-object detection (30.2% AP$) and segmentation (50.2% mloU) over the 5x5/9x9/13x113 variant, but still underperforms the optimal 3x3/7x7/15xll combination. This highlights the 7x7 kernel's role as an essential bridge between fine and coarse scales. Larger sets (7x7/1 Ix 1/15xx115 suffer 2.2% mloU drop due to over-smoothing of fine-grained features.
The importance of core components persists across model scales. When Fast- HGR is removed, the Tiny variant exhibits a 33% larger relative accuracy drop than the Base model. Ablating multi-scale pathways (Base model with single-scale 7x7 kernel) causes 2.5% decline in ImageNet T and a 4.3% drop in COCO APh, underscoring multi-path fusion as a foundational design element.
Model Scalability:
Task Adaptability: Removing task-specific heads (FPN for detection, decoder for segmentation) results in minimal performance loss: COCO AP" decreases by 0.3% to 50.0, and ADE20K mloU decreases by 1.5% to 49.3. This indicates the backbone's 's inherent strength in learning discriminative multi-scale features.
Discussion
Model Enhancement and Feature Understanding. The integration of Fast-HGR correlation atten- tion enhances the capacity of MK-CAViT to model complex feature dependencies, outperforming traditional dot-product attention. By combining local token similarity with global distributional consistency, the model effectively captures fine-grained spatial details and long-range context, both essential for tasks such as small-object detection and semantic segmentation. The theoretical founda-

---

Table 5: Comprehensive ablation study results.
Component Variant
#Params(M)
FLOPs(G)
ImageNet Top-1(%)
COCO
ADE20K
(P))))
(P)
mloU(S)
MK-CAVIT-Baal Base (Full)
88.0
15.6
85.6
50.3
31.9
50.8
A. Core Architecture Components
w/o Fast- Fast-HGR Module
(-1.8)
(-0.4)
84.7 (-0.9)
48.8 (-1.5)
(-2.)
50.0 (-0.8)
w/o Hierarchical Gating (Concat/Add)
$8.0
17.8 (+14%)
84.8 (40.8
49.2 (-1.1)
(-1.5)
49,9 (-0.9)
Multi-Token Attention (Dense)
132.0
23.4 (+50%)
85.8 (+0.2)
49.8 (-0.5)
(1.6(-0.3)
50.5 (-0.3)
Multi-Token Attention (Spurse)
70.4 (-17.6)
12.5 (-20%)
(-0.8)
48.1 (-2.2)
(-25))
(-1.2
Dynamic Norm (LayerNorm)
$8.0
15.6
85.1 (-0.5)
49.9 (-0.4)
31.7
50.3 (-0.5)
Dynamic Norm (BatchNorn)
88.0
15.6
48.4 (3.4))).3)
(-.0)
(9.7)
B. Kernel Configuration
3/5/7 Kernels
15.1
83,9 (-1.7)
48.7 (-1.6)
28.7
48.9 (-19)
5/9/13 Kernels
87.5
15.7
85.1 (-0.5)
49.5 (-0.8)
(-1.()
49.6 (-12)
Kernels
15.8
85.0 (-0.6)
49.8 (-0.5)
30.2
(.0.6)
1/15 Kemels
89.3
15,9
83,5 (-2.1)
48.3 (-2.0)
(-3.4)
49.1 (-1.7)
C. Model Scalability
Tiny w/o Fast HGR Base (Single-Scale
82.3 (-1.2)
46.2 (-1.8)
27.7 (-1.8
48.3
79.8
13.1
83.1 (-2.5)
46.0 (-4.3)
27.2 (7.77
(-19)
D. Tash. Adaptability COCO w/o FPN
88.0
15.6
50.0 (-0.3)
(-0.))
ADE20K w/o Decoder
88.0
15.6
49.3 (-1.5)
tion in HGR maximal correlation provides a rigorous framework for capturing nonlinear dependencies that conventional attention mechanisms often miss.
Multi-Kernel Architecture Advantages. The multi-kernel design enabled hierarchical feature extraction, with small kernels capturing local details, large kernels modeling global context, and mid- sized kernels bridging spatial scales. This synergy enhanced representation robustness, as evidenced by consistent performance gains in image classification, object detection, and semantic segmentation.
Comparison with State-of-the-Art Methods. MK-CAVIT outperforms both CNNs and vision transformers in uecuracy-efficinncy trade-offs. Compared with Swin and ConvNeXt, it achieves higher accuracy at comparable computational costs, enabled by efficient attention mechanisms and lightweight normalization strategies. The model demonstrates particular advantages over State-of- the-Art methods including VisterViT and Agent-Swin, achieving 1.6% higher ImageNet accuracy and 1.3% higher COCO AP while maintaining similar parameter counts, validating the effectiveness of theoretically grounded correlation modeling.
I Limitations and Future Directions. Although MK-CAVIT demonstrates strong performance across diverse vision tasks, several limitations warrant future investigation. The multi-kernel design intro- duces computational overhead that may challenge deployment in resource-constraingg environments. Performance degradation is observed on low-resolution images where large kernels cover most pixels, and correlation attention may amplify noise in highly corrupted inputs. Additionally, potential bias toward majority classes emerges in extremely imbalanced datasets. Future work will explore adaptive kernel selection, noise-robust attention mechanisms, class-awar HGR loss weighting, and extension 3D vision tasks and video analysis for comprehensive cross- modal feature alignment. Hardware- aware optimization represents another promising direction to enhance computational efficiency while maintaining performance advantages.
6 Conclusion
In this study, MK-CAVIT was proposed as an enhanced Vision Transformer that integrates multi-scale kernel pathways with a correlation attention mechanism. The framework strengthened the capacity of the model to capture complex contextual relationships by leveraging the HGR maximal correlation to represent both fine-grained local details and long- long-range global context. The multi-kernel multi-scale feature correlation strategy effectively balanced the local and global information, improving the robustness and generalization across tasks such as image classification, object detection, and semantic segmentation. The Fast-HGR mechanism further optimized the efficiency, interpretability, and consis- tency of correlation attention, enabling MK-CAViT to capture complex feature interactions while maintaining computational efficiency. This approach enhances feature extraction and interpretability without compromising performance, achieving an effective balance between precision and efficiency.
10

---

Acknowledgements
This work was supported in part by the National Key R&D Program of China under Grant 2021 YFA0715202, Shenzhen Key Laboratory of Ubiquitous Data Enabling (Grant No. 2DSY$2020052111406601 and the Shenzhen Science and Technology Program under Grant KOTD201708101508211 146 and Grant JCYI00220533113000005.
References
[1] Tianyang Lin, Yuxin Wang, Xiangyang Liu, and Xipeng Qiu. A survey of transformers. Al open, 3:111 132, 2022.
[2] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017.
[3] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition scale. In International Conference Learning Representations, 2021.
[4] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data efffcient image transformers and distillation through attention. ICML. PMLR, 2021.
Xuran Pan, Chunjiang Ge, Rui Lu, Shiji Song, Guanfu Chen, Zeyi Huang, and Gao Huang. On the integration of self attention and convolution. Conference on Computer Vision and Pattern Recognition, 2022.
[6] Wenhai Wang, Enze Xie, Xiang Li, Deng Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In International Conference Computer Vision, 2021.
[7] Nicolas Carion, Francisco Massa, Gabriel Synnacve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End - to - end object detection with transformers. In European Conference on Computer Vision, 2020.
[8] Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In European Conference Computer Vision, 2022.
[9] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning, 2021.
[10] Xuran Pan, Tianzhu Ye, Dongchen Han, Shiji Song, and Gao Huang. Contrastive language image pre- training with knowledge graphs. In Advances in Neural Information Processing Systems, 2022.
[11] Z Xia. D Han, Y Han, X Pan, S Song, and G Huang. Gsva: Generalized segmentation via multimodal large language models. In CVPR, 2024.
[12] Derya Soydaner. Attention mechanism neural networks: where it comes and where it goes. Neural Computing and Applicationn, 34(16):133) - 13385, 2022.
[13] Kai Han, Yunhe Wang, Hanting Chen, Xinghao Chen, Jianyuan Guo, Zhenhua Liu, Yehui Tang, et al. survey on vision transformer. IEEE transactions on pattern analysis and machine intelligence, 45(1):87- 110, 2022.
[14] Francesco Pinto, Philip HS Torr, and Punect K Dokania. An impartial take to the enn vs transformer robustness contest. In European Conference on Computer Vision, Cham: Springer Nature Switzerland, 2022.
[15] Hongkang Li, Meng Wang, Sijia Liu, and Pin Yu Chen. A thcoretical understanding of shallow vision transformers: Learning, generalization. and sample complexity. arXiv preprint arXiv:2302.06015, 2023.
[16] Yaodong Yu, Sam Buchanan, Druv Pai, Tianzhe Chu, Ziyang Wu, Shengbang Tong, Benjamin Haeffele, and Yi Ma. White box transformers via sparse rate reduction. In Advances in Neural Information Processing Systems, 2023.

---

[17] Y Han, Z Liu. Z Yuan, Y Pu, C Wang. S Song, and G Huang. Latency aware uniffed dynamic networks for efffcient image recognition. TPAMI, 2024.
[18] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In International Conference on Computer Vision 2021.
[19] W Wang, E Xie, X Li, D.P Fan, K Song, D Liang, T Lu, P Luo, and I. Shao. Pyramid vision transformer:
A versatile backbone for dense prediction without convolutions. In ICCV, 2021.
[20] A Hatamizadeh, G Heinrich, H. Yin, al. Fastervit: Fast vision transformers with hicrarchical attention. In International Conference on Learning Representations. ICLR, 2024.
[21] D Han, T Ye, Y Han, et Agent attention: On the integration softmax and linear attention. arXiv preprint 2311:2312208874, 2023.
[22] Jianyuan Guo, Kai Han, Han Wu, Yehui Tang, Xinghao Chen, Yunhe Wang, and Chang Xu. Cmt: Convolutional neural networks meet vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 12175 12185, 2022.
[23] Xuran Pan, Chunjiang Ge. Rui Lu, Shiji Song. Guanfu Chen, Zeyi Huang, and Gao Huang. On integration self attention and convolution. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 815 825, 2022.
[24] Omid Nejati Manzari, Hamid Ahmadabadi, Hossein Kashiani, Shahriar B Shokouhi, and Ahmad Ayatollahi. Medvit: robust vision transformer for generalized medical image classification. Computers in Biology and Medicine, 157:10677 2023.
[25] Yinpeng Chen, Xiyang Dai. Dongdong Chen, Mengchen Liu, Xiaoyi Dong. Lu Yuan, and Zicheng Liu. Mobileformer: Bridging mobilenet and transformer. In CVPR, 2022.
[26] Muhammad Maaz, Abdelrahman Shaker, Hisham Cholakkal, Salman Khan, Syed Waqas Zamir, Rao Muhammad Anwer, and Fahad Shahbaz. Khan. Edgenext: efffciently amalgamated can trans- former architecture for mobile vision applications. In ECCVW,001 2022.
[27] Chenglin Yang, Yilin Wang. Jianming Zhang, He Zhang. Zijun Wei, Zhe Lin, and Alan Yuille. Lite vision transformer with enhanced self attention. CVPR, pages 11998 - 12008, 2022.
[28] Yulin Wang, Yang Yuc, Rui Lu, Tiajjiao Liu, Zhao Zhong, Shiji Song, and Gao Huang. Efffcienttrain: Exploring generalized curriculum learning for training visual backbones. In International Conference on Computer Vision, 2023.
[29] Sachin Mehta and Mohammad Rastegari. Mobilevit: lightweight, general purpose, and mobile- friendly vision transforerr. In International Conference on Learning Representations, 2022.
[30] Jacob Devlin, Ming Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre training deep bidirectional transformers for language understanding. In Anmual Conference of the North American Chapter of the Association for Computational Linguistics, 2019.
[31] Y Li, Y Fan, X Xiang, D Demandolx, R Ranjan, R Timofte, and L Van Gool. Efffcient and explicit modelling of image hierarchies for image restoration. In CVPR, 2023.
[32] Y Li, G Yuan, Y Wen, al. Efficientformer: Vision transformers mobilenet speed. Advances in Neural Information Processing Systems, 2022.
[33] Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, and Jianfeng Gao. Focal self-attention for local-gla gooball interactions in vision transformers. In NeurlPS, 2021.
[34] Youngwan Lee, Jonghee Kim, Jeff Willette, and Sung Ju Hwang. Mpvit: Multi-path vision transformer for dense prediction. In CVPR, 2022.
[35] Wenhai Wang, Enze Xie, Xiang Li, Deng Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pvt v2: Improved baselines with pyramid vision transformer. Computational Visual Media, 2022.
[36] Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichtenhofer. Multiscale vision transformers. In International Conference Computer Vision, 2021.
12

---

[37] Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Lu Yuan. Dong Chen, and Baining Guo. Cswin transformer: A general vision transformer backbone with cross shaped windows, In Conference on Computer Vision and Pattern Recognition, 2022.
[38] Z Xia, Pan, S Song, L.E Li, and G Huang. Vision transformer with deformable attention. CVPR. 2022.
[39] Zhu, X Wang, Z Ke, Zhang, and R.W Lau. Biformer: Vision transformer with level routing attention. In CVPR, 2023.
[40] H. O. Hirschfeld. A connection between correlation and contingency. Mathematical Proceedings of the Cambridge Philosophical Society, 31(4):520 - 524, 1935.
[41] Herbert Gebelein. Das statistische problem der korrelation als variations - und cigenwertproblem und sein zusammenhang mit der ausgleichsrechnung. ZAMM - Journal of Applied Mathematics and Mechan- ics/Zetschrift fu Angewandte Mathematik und Mechanik, 21(6):364 -779, 1941.
[42] Alfre' Re' 'yii On measures of dependence. Acta Mathematica Hungarica, 10(3 4):441 451, 1959.
[43] S. L. Huang, A. Makur, L. Zheng, and G. W. Wornell. An information -hhoorrticttcotttiteee approach to universal feature selection high dimensional inference. In International Symposiun on Information Theory (ISIT), pages 1336-1340, 2017.
[44] Li Wang, Jie Wu, Shang Lung Huang. Li Zheng, Xiao Xu, Li Zhang, and Jie Huang. An efffcient approach to informative feature extraction from multimodal data. In Proceedings of the AAAI Conference on Artiffcial Intelligence, volume 33, pages 5281- - 5288, 2019.
[45] Hongkang Zhang, Shao Lun Huang, and Ercan Engin Kuruoglu. Mhfnet: An improved bgr multimodal network for informative correlation fusion in remote sensing image classiffcation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17: 150666, 2024.
[46] Jiachen Zhu, Xinlei Chen, Kaiming He, Yann LeCun, and Zhuang Liu. Transformers without normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2025.
[47] Deng, Wei Dong. Richard Socher, Li Li Jia, Kai Li, and Li Fei Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. IEFE, 2009.
[48] Tsung Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014.
[49] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler. Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference computer vision and pattern recognition. pages 633-641, 2017.
[50] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[51] Saining Xie. Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transforma- tions for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492-1500, 2017.
[52] Z Liu, H Mao, CY Y Wu, et al. A convnet for the 2020s. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2022.
[53] Tsung- Yi Lin, Priya Goyal, Ross Girshick. Kaiming He. and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE International Conference Computer Vision (ICCV), pages 2980-2988, 2017.
[54] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
[55] Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. Unified perceptual parsing for scene understanding. In Proceedings of the European Conference on Computer Vision (ECCV), pages 418-434, 2018.
[56] Mingxing Tan and Quoc V Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference Machine Learning, pages 661566114 PMLR, 2019.
13

---

[57] Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Scott Kim, Jeannette N Chang. Sungbok Lee, and Shrikanth S Narayanan. lemocap: Interactive emotional dyadic motion capture database. Language Resources and Evaluation, 42(4):335-359, 2008.
[58] Noel C. F. Codella, David Gutman, M. Emre Celebi, Brian Helba, Michael A. Marchetti, Stephen W. Dusza, Aadi Kalloo, Konstantinos Liopyris, Nabin Mishra, Harald Kittler, and Allan Halpern. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 20/8), pages 168-172. IEEE, 2018.
[59] Jing Lin, Fei Gao, Xiao Shi, and Jin Dong. Ss-mae: Spatial spectral masked autoencoder for multisource remote sensing image classitication. IEEE Transactions Geoscience and Remote Sensing, 61:1-14, 2023.
[60] Joseph Sherrah. Fully convolutional networks for dense semantic labelling of high-resolution acrial imagery. IEEE Geoscience and Remote Sensing Letters, pages 1-5, 2016.
[61] X. Ma, X. Zhang, M.-O. Pun, and B. Huang. A unified framework with multimodal fine-tuning for remote sensing semantic segmentation. IEEE Transactions Geoscience and Remote Sensing, 63:1-1, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of research papers typically outline the core contributions, scope, and key findings, ensuring alignment with the paper's content.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: This paper includes a "Limitations" section to discuss constraints, such as model weaknesses or scope limitations.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [Yes]
Justification: The paper provides complete theoretical results with numbered theorems, formulas, and proofs, which are cross-referenced throughout the main text and supplemental material, adhering to the guidelines for formal documentation.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: This paper provides detailed information about the dataset, hyperparameters, and methods in the experimental section to enable reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
The paper uses publicly available datasets and provides code the supplemen- tal material, with plans for public release upon publication, ensuring sufficient instructions for reproducing experimental results.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https
www.eeeeeeoeeemmmeeeeeee colmmmsssion aassonoooicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Key details like dataset splits, hyperparameters, and optimization strategies are reported in the experimental sections and supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The paper does not report error bars or statistical significance tests. Most experiments were conducted 3 to 5 times and averaged. According to the statistical results, the errors were less than +0.3%. However, to maintain clarity, the authors omitted error bars and detailed statistical tests from the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eπror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided sufficient information the computer resources required for the experiment in the attached materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper complies with NeurlPS ethical standards in all aspects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
18

---

Justification:
The paper focuses on technical contributions without explicit discussion of societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No high-risk assets (e.g., large pretrained models) are introduced, so safeguards are not applicable.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Existing datasets (c.g., ImageNet- IK, ADE20K,COCO) are properly cited with DOIs, respecting their licenses and terms of use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
19

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: This paper introduces an improved VIT model. The code is provided as supplementary material and will be made public after paper officially published.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: This study does not involve human subjects or crowdsourcing, and focuses on ViT models and computational experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects are involved, making IRB approval irrelevant.
20

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The methodology does not rely large language models (LLMs) key component.
Guidelines:
The answer NA means that the core method development in this research does involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy http::://eerrpps..cco/oeeeeeeeee for what should or should not be described.
21

---

Appendix
A Theoretical Analysis of Fast-HGR Approximation
Theorem (Statistical Consistency of Fast-HGR)
Let X e Nxdx and Y E NRdx be feature matrices with i.id. samples, where each sample Xj X and yj Y has bounded norms. Define linear transformations f(X) XW and g(Y) - YV with W E xk and V E Edyxk, xk, The Fast-HGR correlation score is:
1 N
f(x,)T((y,)) g(y,) + + (cooo((())))evv(((()) (17) [fxx)]]l>>>>>|]
f(x,)
E-HGR(f(X),
This formulation approximates the true k-dimensional HGR maximal correlation ((() (X, Y) with high-probability estimation error bounded by O(1/VN).
Proof
For linear 9. the HGR objective measures the strongest statistical dependence as (l)) (X, Y)
supy.g (X) g(Y) under centering and covariance constraints. Fast-HGR approximates this by:
1. Replacing the population expectation with batch-averaged cosine similarity term, which captures local pairwise dependencies; 2. Retaining global structural information via the trace of covariance products.
For the trace term:
tr (X) (U(() ((' - (x(Y) - g) YY
(18)
--1
N
where f and ğ batch means of f(X) and g (Y) respectively.
Under iid. sampling, empirical covariances Σn cov(f(X) and Σ, CY)(g(YY) converge to their population counterparts y, and Mg in Frobenius norm. By McDiarmid's s inequality, the deviation between empirical and population estimates decays exponentially with N, leading to overall error bound of O(1/VN).
Theorem (Lipschitz Continuity of Fast-HGR)
The Fast-HGR operator is Lipschitz continuous with respect to feature perturbations. For any feature matrices X1, X2 and Y1, Y2,
(- PTHCH//)YYY))))-HH//)/)))))) <(()((X,,) ([(ll((x))-(()))))) f(x))ll++ (lg(Y)) g( ( (l)),
(19)
where the Lipschitz constant is
N - + + +*(1SEfle= |(gll=))
(20)
with E,, E.g denoting empirical covariances of f(X) and gYY), respectively.
Proof
The cosine similarity term is Lipschitz continuous due to unit normalization of fxx)) and g(y,). For pairwise terms, cos(a], , b1) cas(a2, b2) < 2[[[a] azll + ||b b2ll) under unit norms, leading collective bound of 2/(N 1) for the summed term.
For the trace term, using the inequality for matrix traces:
tt tr(A'B') < | A' Blee || B'll,
(21)
the perturbation of the trace term is bounded by X(]| || + |(اg|(])) times the feature perturbations. Combining both terms yields the Lipschitz constant L.
22

---

B Derivation of Fast-HGR from Soft-HGR Variants
Fast-HGR is derived by simplifying and adapting the improved Soft-HGR (I-SoftHGR) objective, which retains the core of HGR while relaxing strict whitening constraints. The I-SoftHGR objective is:
SoRHHRR ( o g(Y)
(Ilcov(((X)) IIβ ||cov(g(Y))
Il(
(22)
2
where the penalty term enforces covariances close to the identity matrix. Fast-HGR modifies this via two key steps:
Local Dependence: Replace Expectation with Cosine Similarity The population expecta- tion (X) g(Y) dependencies:
is approximated using batch-wise cosine similarity to capture local pairwise
(1x))T(y))) [[xx)]]l
(I((X) g(Y) N
(23)
2. Global Structure: Replace Whitening Penalty with Covariance Alignment I-SoftHGR's s penalty term ||cov(f) - I variance. Fast-HGR removes this constraint, instead capturing global distributional alignment via the trace of covariance products:
((oo(g) I enforces soft whitening but introduces sensitivity to
Expanding the I-SoftHGR penalty term:
|]cov([) I|βP = tr(cov 2tr(cov([)) + k
(24)
where & is the dimension transformed features. Fast-HGR replaces these with cross-term that measures alignment between cov(f) and cov(g) without enforcing unit covariance:
À tr (cov
(25)
Combining these steps yields the Fast-HGR formulation:
N
f(x,) g(x;) + tr (coo(((())cov(((())
F-HGR(((X), N
(26)
Ilxx))]]
This derivation preserves the core objective of maximizing feature dependence while replacing computationally expensive whitening constraints with efficient trace-based regularization.
C Implementation Details
Multi-Scale Tokenization Table presents the optimized kernel configurations designed for efficient hierarchical feature extraction.
Table Optimized Kernel Configurations for Feature Extraction
Kernel Type Size/Stride/Paddiin
Output Size
Channels
Receptive Field Impact
Small
3x3/1/1
H W
64
Fine-grained details
Medium
7x7/2/3
H/2 x W/2
128
Mid-level semantics
Large
15x15/1/7
W
256
Global context
The medium kernel (7x7, stride=2) uses padding=3 to achieve an output size of H/22 calculated as padding kernel size-1
3.
2
The large kernel (15x15, stride=1) employs padding=7 to maintain the input resolution (H x W), consistent with "same" padding semantics.
The combination of 3x3, 7x7, and 15x15 kernels effectively balances the capture of fine details, mid-level objects, and global scene context.
23

---

Training Protocol
Optimizer: AdamW is utilized with weight decay of 0.05, 88 0.9, and /2 0.999.
Warmup: warmup phase of 20k steps (equivalent 10% of the total 200 epochs) implemented to stabilize the initial training dynamics.
Learning Rate: The learning rate follows a cosine decay schedule starting from 5e-5 for the Base model, incorporating mixed precision (FP16) to accelerate convergence by 2x and reduce memory usage.
Regularization: DropPath Rate: This rate is linearly increased from 0 to 0.1 across layers to enhance feature robustness, applying stronger regularization to deeper layers that handle global features. Position Bias: 2D learnable relative position embeddings (size: H/2 x W/2) are employed to encode spatial dependencies in the features derived from medium and large kernels.
D Visualization of HGR-Correlatto Attention Maps
This section presents qualitative analyses of attention maps across representative backbones using two ImageNer samples (a flower and a dog). The objective is examine how correlation-aware multi-kernel modeling influences the spatial distribution of attention, particularly in terms of semantic alignment, background suppression, and integration of discriminative features at varying scales. All attention maps are generated using the same protocol, normalized per image using min-max scaling, and visualized with an identical color scale to ensure comparability.
(b)
(c)
(d)
(g)
(h)
ii)
(k)
(I)
(m)
(n)
(0)
(p)
Figure 2: Qualitative comparison of attention maps across backbones. Top flower; bottom row: dog. From left to right: (a.i) original images; (b.j) ViT-B; (c.k) DeiT-B; (d.I) Swin-B; (c.m)
Agent-Swin; (f.n) FasterViT; (g.o) MPViT; (h.p) MK-CAViT. All maps use the same visualization pipeline and color scale for consistency.
D.1 Patterns in Attention Distribution
Flower sample (Fig. h)) ViT-B and DeiT-B exhibit two common limitations: either attention leaks into the background or concentrates excessively
single peripheral region (e.g., edge petal), failing to capture the hierarchical structure of disc and petals (b,c). Swin-B and Agent-Swin produce fragmented attention with isolated local peaks. lacking coherence across the floral structure (d,c). Faster ViT shows sparse, discontinuous activations with noticeable "holes" the attention coverage (f). MPVIT improves petal boundary delineation but weakens response the central disc, critical semantic feature (g). In contrast, MK-CAVIT forms compact primary peak on the floral disc, with coherent secondary along the petal edges, while effectively suppressing background noise (h). This "primary-auxiliarr"" structure aligns with the flower's intrinsic semantic hierarchy.
Dog sample (Fig, -pp) VIT-B allocates significant attention to background regions and non- discriminative parts (e.g., ears), diluting focus on key facial features (j). DeiT-B and Swin-B fixate on single dominant region (e.g., nose or jaw) but underweight other critical components like eyes (k.l). Agent-Swin and Faster ViT generate scattered hotspots across the face without clear prioritization of discriminative features (m,n). MPVIT covers broader facial area but disperses attention energy, reducing contrast between key and secondary features (o). MK-CAVIT, however, concentrates primary attention the nose (the most discriminative facial feature) while maintaining distinct secondary

---

peaks on both eyes, with minimal activation in background regions (p). This pattern reflects robust alignment with the semantic importance of facial components.
D.2 Implications for Multi-Scale Modeling
The visualized attention maps represent the final fused output of each architecture; per-scale kernel attention is not explicitly shown here. Nevertheless, MK-CAViT's consistent ability integrate fine-grained details (e-g., floral disc texture, eye contours) with extended structures (e.g., petal rings, facial contours) suggests effective aggregation of information across spatial scales. This aligns with the design intent of cor--crrelatioo attention: to model nonlinear dependencies between features at different scales, rather than treating them as independent streams.
D.3
Connection to Quantitative Performance
The qualitative improvements observed in MK-CAViT- semantic alignment, reduced back- ground interference, and coherent integration of discriminative features- correspood with quanti- tative gains across tasks (ImageNet classification, COCO detection, ADE20K segmentation). These visual patterns provide intuitive support for the claim that correlation-aware multi-scale fusion en- hances the model's ability to prioritize semantically relevant features, a mechanism underlying its superior performance.
E
Cross-Domain Generalization
To rigorously evaluate the generalization capability of MK-CAVIT beyond standard vision bench- marks, extensive experiments were conducted across three distinct application domains: multimodal emotion recognition, medical imaging, and remote sensing. Domain generalization (DG) aims to learn models from multiple source domains that perform well on unseen target domains, which a challenging and practical scenario since models are often deployed in environments different from where they were trained [39. [6). As summarized in Table 7 the multi-scale design of MK- CAViT demonstrates consistent performance advantages over specialized basclines across all domains, highlighting its robustness to domain shift.
Table 7: Comprehensive cross-domain generalization performance comparison.
Domain-Tast
Dataset
Metric
Model
MK-CAVIT-Ba
Swin-Ras Wain-Base
ViT-Base
ConyNeXI-Base
Emotion Recignitooon
JEMOCAP
ACC
73.5%
70.1%
68.5% 68.7%
69.8%
W-FI
73.6%
69.8%
70.1%
MK-CAViT-Base
Swin-UNet
TransFuse
EfficientNet-B4
Mocdical-Segmentttii
ISIC2018
mIoU Dice
83.43%
82.78%
80.63%
81.21%
89.96%
89.78%
88.21%
88.77%
MK-CAVIT-Base
FasterViT
3D-CNN
ViT-Base
Remote Sensing- Classification
Houston 2018
93.68%
92.13% 95.22%
89.59% 93.77%
91.87%
95.82%
94.93%
MK- -A-V---BBBBaee
Swin-UNet
U-Net
DeepLabV3+
Remote Sensing- Segmentation
Vaihingen
92.61%
91.56%
89.93%
88 98.92%
mloU
84.43%
82.62%
80.15%
81.56%
E.1 Multimodal Emotion Recognition
In multimodal emotion recognition on the IEMOCAP dataset, MK-CAViT-Bass achieved weighted accuracy of 73.5% % and an F1-score of 73.6%. outperforming Swin-Base (70.1% % accuracy, 69.8% Fl-score), ViT-Base (68.5% accuracy, 68.7% Fl-score), and ConvNe (69.8% accuracy, 70. % Fl-score). This task utilized only visual frames to focus on spatial feature learning across four emotion categories (happy, sad, angry, neutral), which inherently involves dealing with domain shifts such variations in lighting, head pose, and individual expressions [57].
25

---

The performance advantage stems from the multi-scale architecture's ability to capture complemen- tary emotional cues. The 3x3 kernel identifies fine-grained facial microexpressions (c.g., smile creases, brow furrows), while the 15x15 kernel models global facial dynamics and head orientation patterns. The Fast-HGR attention mechanism integrates these scale-specific features by modeling their nonlinear correlations, enabling robust distinction of subtle emotion cues that require simul- tancous local detail analysis and global context understanding. This approach effectively learns domain-invariant representations that are crucial for handling variations across different speakers and recording sessions.
E.2
Medical Imaging
For skin lesion segmentation on the ISIC2018 dataset, which poses the critical challenge of dis- tinguishing melanoma from nevus through subtle boundary variations, MK-CAViT-Base achieved 83.43% mloU and 89.96% Dice coefficient. This surpassed specialized medical imaging baselines including Swin-UNet 78% mloU, 89.78% Dice), TransFuse (80.63% mloU, 88.21 % Dice), and EfficientNet-B4 (81.21% % mloU, 88.77 77% Dice). The medical imaging domain frequently encoun- ters domain shift problems due to variations in imaging devices, lighting conditions, and patient populations (58).
The multi-scale design addresses essential requirements in medical diagnostics through comple- mentary feature extraction. The 3x3 kernel detects subtle lesion boundaries and texture variations crucial for early melanoma identification, while the 15x15 kernel captures global lesion structure including asymmetric shapes and spatial distribution patterns. HGR-correlation attention effectively models the complex spatial relationships between lesions and surrounding healthy tissue, significantly reducing false positives caused by spurious correlations. The consistent performance gains across all medical metrics validate the architecture's capability for precise medical image analysis without domain-specific architectural modifications.
E.3
Remote Sensing Applications
Remote sensing evaluation encompasses two distinct tasks with complementary spatial requirements. For land-cover classification on the Houston 2018 dataset[59). MK-CAVIT achieved 93.68% overall accuracy and 95.82% % average accuracy. outperforming Faster Vist-rVVT-Small (92. 13% OA, 95.22% AA), 3D-CNN 989.59% OA, 93.77 73.77% AA), and ViT-Base (91.87% OA, 94.939. 44.93% AA). In urban segmentation on the Vaihingen dataset 60) 61, the model achieved 84.43 % mloU and 92.61 % overall accuracy, surpassing -Net (82.62% 62% mIoU, 91.56% OA), U-Net (80.15% mloU, 89.93% OA), and DeepLabV3+ (81.56% mloU, 88.92% OA). Remote sensing applications inherently face domain shifts due to seasonal variations, geographical differences, and sensor specifications.
The multi-scale architecture demonstrates natural alignment with remote sensing imagery charac- teristics. The 15x15 kernel captures large-scale geographical patterns and land-cover distributions essential for regional classification, while the 3x3 kernel identifies small structural elements such as road markers and individual vegetation features. For urban segmentation tasks, the 7x7 and 15x15 kernels collaboratively model building and road contexts at appropriate scales, while the 3x3 kernel precisely segments small urban objects including street furniture and vehicle clusters. This scale-aware processing enables comprehensive scene understanding across varying spatial resolutions inherent to remote sensing data, effectively addressing the domain shift challenge through multi-scale invariant feature learning.
E.4 Generalization Analysis
The consistent performance advantages across emotionally nuanced, medically critical, and geograph- ically complex domains demonstrate the robustness of MK-CAViT's multi-scale design principle against various types of domain shifts. Several interconnected factors contribute to this generalization capability:
Scale adaptability enables automatic adjustment to domain-specitn feature hierarchies without architectural modifications. The parallel kernel pathways capture information across spatial scales that align naturally with different application requirements, from microscopic medical features to macroscopic geographical patterns. This adaptability allows the model to maintain performance when facing domain shifts characterized by scale variations in target features.
26

---

Feature complementarity ensures preservation and integration of both local details and global context. This proves particularly valuable domains where both micro-level patterns and macro- level structures carry diagnostic information, such as facial microexpressions in emotion recognition or lesion boundaries in medical imaging- By capturing features multiple scales, the model reduces dependence on domain-specific superficial patterns, thus enhancing generalization [18]
Correlation-basee feature integration through Fast-HGR attention provides theoretically grounded mechanism for modeling nonlinear dependencies across scales. This approach effectively suppresses spurious correlations that vary across domains while enhancing true causal features that remain invariant, aligning with the principles of stable learning for out- out-of--distrri of-diiiiibution generalization.
The cross- domain validation establishes that the multi-scale correlation attention mechanism provides fundamental advantages for visual understanding tasks requiring simultaneous processing of fine details and global context. The consistent outperformance of specialized baselines across diverse applications positions MK-CAVIT as a versatile architecture with strong generalization potential for real-world deployment where domain shift is a common challenge.
F Parameter Sensitivity Analysis
The parameter X, which balances local cosine similarity and global trace regularization, was tested on the ImageNet validation set. The results are summarized in Table
Table 8: Parameter Sensitivity of À on ImageNet-1
Top-1 Acc (%)
0.01
83.2
0.05
83.9
0.1
84.3
0.2
83.7
0.5
82.9
A Á value of 0. optimally balances local token alignment (cosine term) and global feature distribution consistency (trace term). Smaller values (e.g., 0.01) under-regularize, leading to unstable feature distributions, while larger values (e.g., 0.5) over-constrain the model, suppressing fine-grained dependencies.
This aligns with theoretical predictions that À controls the trade-off between capturing pairwise correlations (local) and second-order statistical alignment (global), as demonstrated in Theorems and 2.
G Relationship to Prior Work
Nonlinear Dependency Modeling: Fast-HGR explicitly maximizes correlation coefficients to capture nonlinear dependencies, such as quadratic interactions. In contrast, dot-product attention relies on implicit nonlinearity through softmax mechanisms and often struggles with high-order statistical modeling.
Asymmetric Fusion: MK-CAViT employs an asymmetric fusion strategy, utilizing small/mid- kernels for queries and keys while leveraging large-kernels for values. This design contrasts with symmetric fusion approaches, such as those used in Focal Transformer and MpVIT, which process all scales uniformly. By adopting this asymmetric method, MK-CAVI achieves more efficient cross- scale information flow, significantly reducing computational expenses while enhancing interaction efficiency across different scales.
27


Reviews:
[{"review_id": "AvMunGnble", "reviewer": "Reviewer_PqDQ", "summary": "This paper introduces the Multi-Kernel Correlation-Attention Vision Transformer (MK-CAViT) to model multi-scale spatial relationships and integrating fine-grained local details with long-range global dependencies. Grounded in the Hirschfeld-Gebelein-Rényi (HGR) theory, MK-CAVIT proposes a parallel multi-kernel architecture for extracting multi-scale features, an HGR-correlation attention mechanism to enhance cross-scale interactions by modeling nonlinear dependencies, and a stable multi-scale fusion strategy for improved training stability. The   Fast-HGR  approximation to address computational inefficiencies of exact HGR computations by using cosine similarity for local dependence and trace regularization for global consistency. Experimental results on ImageNet, COCO, and ADE20K datasets demonstrate that MK-CAVIT surpasses other methods in classification, detection, and segmentation tasks.", "questions": "See the weaknesses part.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Without Bells and Whistles. The paper is straightforward and easy to read, avoiding unnecessary complexity or \"flamboyant\" presentations. This simple yet effective approach is highly valued in the computer vision field.\n2. Novelty and Theoretical Grounding.The paper introduces a novel approach by integrating HGR maximal correlation theory into Vision Transformers, providing a rigorous mathematical foundation that is often lacking in empirically driven ViTs. This is a significant contribution. \n3. Enhanced Cross-Scale Interactions. The HGR-correlation attention mechanism effectively models nonlinear dependencies and applies adaptive scaling to weigh connections, refining contextual reasoning. The Fast-HGR approximation maintains the theoretical advantages of HGR while significantly reducing computational complexity. \n4. Strong Experimental Results. MK-CAVIT demonstrates superior performance across various computer vision tasks (image classification, object detection, and semantic segmentation) on benchmark datasets\n5. Computational Efficiency. The Fast-HGR approximation and the adaptive multi-head attention mechanism contribute to maintaining efficiency while achieving strong performance. \n6. The ablation study is thorough.\n\nWeaknesses:  \n1. The multi-scale motivation is general.\n2. Limited Justification of Multi-Scale Necessity. While the paper proposes a multi-kernel architecture, the necessity of multi-scale processing could be more robustly justified. It would be beneficial to see a direct comparison of the model's performance using only single-scale kernels (e.g., separate experiments with only 3x3, 7x7, and 15x15 equivalent kernels) to clearly demonstrate the performance gains achieved by integrating multiple scales. This would empirically confirm the added value of the multi-scale approach.\n3. Visualization of Attention Maps. While the paper discusses how HGR models complex feature interactions, visualizing the attention maps generated by the HGR-correlation attention mechanism could provide more intuitive insights into how it captures multi-scale dependencies compared to traditional self-attention.", "was_revised": true, "final_justification": "The author's positive response has addressed all my concerns.", "rebuttal": "# Response to Reviewer PqDQ\n\nWe sincerely appreciate Reviewer PqDQ for the insightful feedback, which helps strengthen our manuscript. We address the key concerns regarding the necessity of multi-scale design and attention map visualizations using our new single-scale kernel experiments and planned visualizations, ensuring consistency with the paper's findings.\n\n\n## 1. Necessity of Multi-Scale Design: Evidence from Single-Scale Kernel Experiments\n\nWe fully agree that robust justification of multi-scale processing requires direct comparisons with single-scale kernel configurations. To address this, we conducted comprehensive experiments evaluating models using only single-scale kernels (3×3, 5×5, 7×7, 9×9, 11×11, 15×15) across three benchmark datasets, with results summarized in Table 1.\n\n| Configuration    | ImageNet Top-1 (%) | COCO AP (%) | COCO AP_S (%) | ADE20K mIoU (%) |\n|------------------|--------------------|-------------|---------------|-----------------|\n| 3×3 only         | 82.7               | 43.1        | 27.3          | 43.8            |\n| 5×5 only         | 83.0               | 43.8        | 26.9          | 44.1            |\n| 7×7 only         | 83.2               | 43.6        | 26.1          | 44.3            |\n| 9×9 only         | 82.9               | 43.0        | 25.3          | 43.9            |\n| 11×11 only       | 82.6               | 42.5        | 24.8          | 43.4            |\n| 15×15 only       | 82.9               | 42.8        | 24.7          | 43.1            |\n| **Multi-Kernel (3/7/15)** | **85.6**       | **50.3**    | **31.9**      | **50.8**        |\n\n\n### 1.1 Consistent Performance Gaps Across Tasks\n\nThe results reveal a clear performance ceiling for single-scale designs:\n- **ImageNet Classification**: All single-kernel models achieve 82.6–83.2% Top-1 accuracy, while the multi-kernel model outperforms them by 2.4–3.0% (85.6%). This gap persists regardless of kernel size, indicating that no single scale can capture the full spectrum of visual features required for classification.\n- **COCO Object Detection**: The multi-kernel model achieves 50.3% AP, surpassing the best single-kernel configuration (5×5, 43.8%) by 6.5%. This improvement is particularly pronounced for small objects (AP_S = 31.9% vs. 27.3% for 3×3), demonstrating that multi-scale fusion resolves the inability of single kernels to balance local detail and global context.\n- **ADE20K Semantic Segmentation**: The multi-kernel model's mIoU (50.8%) is 6.5–7.7% higher than all single-kernel variants, confirming that single scales cannot simultaneously preserve fine boundaries and model large-scale scene structure.\n\n### 1.2 Scale-Specific Limitations of Single-Kernel Designs\n\nEach kernel size exhibits task-specific weaknesses that multi-scale fusion overcomes:\n- **Small kernels (3×3, 5×5)**: While achieving the highest single-kernel performance on small-object detection (AP_S = 26.9–27.3%), they fail to capture global context, resulting in lower overall COCO AP and ADE20K mIoU compared to larger single kernels. For example, 3×3 kernels struggle with classifying large, context-dependent categories on ImageNet (e.g., \"mountain\" vs. \"hill\").\n- **Large kernels (11×11, 15×15)**: Perform poorly on detail-sensitive tasks, with 15×15 achieving the lowest ADE20K mIoU (43.1%) and small-object AP_S (24.7%). This confirms that large single scales over-smooth fine-grained features critical for distinguishing textures and small objects.\n- **Mid-sized kernels (7×7)**: Often considered \"balanced,\" they still underperform the multi-kernel model by 2.4% on ImageNet and 6.2% on COCO AP, as they cannot integrate the fine details captured by small kernels with the global context of large kernels.\n\n### 1.3 Synergy of Multi-Scale Fusion\n\nThe multi-kernel model's superior performance stems from its ability to combine complementary strengths:\n- Small kernels (3×3) contribute fine-grained texture and edge information, enhancing small-object detection and boundary preservation.\n- Large kernels (15×15) provide global scene context, improving classification of large objects and large-scale regions.\n- The mid-sized kernel (7×7) mediates cross-scale interactions, ensuring features from different scales are coherently integrated rather than processed independently.\n\nThis synergy is not merely additive: the multi-kernel model's gains (e.g., 6.5% COCO AP over the best single kernel) exceed the sum of individual single-kernel strengths, confirming the necessity of intentional multi-scale integration.\n\n## 2. Visualization of HGR-Correlation Attention Maps\n\nWe concur that visualizing attention maps will provide intuitive support for our multi-scale dependency modeling claims. In the revised manuscript, we will add a dedicated section (Section 3.4) with key visualizations:\n- **Attention heatmaps** for 3×3, 7×7, and 15×15 kernels on representative images from COCO and ADE20K, showing scale-specific focus (fine details for 3×3, global context for 15×15).\n- **Comparisons with single-scale models** to highlight how HGR-correlation attention in the multi-kernel model reduces noise in fine-scale maps and preserves details in coarse-scale maps.\n- **Quantitative metrics** (attention-semantic overlap IoU and cross-scale consistency scores) to validate that multi-scale attention aligns better with semantic structures than single-scale attention.\n\nThese visualizations will complement our quantitative results, demonstrating how HGR-correlation attention enables effective cross-scale interaction.\n\n## 3. Summary of Revisions\n\nTo address the reviewer's concerns, we will:\n1. Integrate Table 1 and its analysis into Section 4.3, explicitly demonstrating multi-scale necessity through direct single-vs.-multi-kernel comparisons across ImageNet, COCO, and ADE20K.\n2. Add attention map visualizations (Section 3.4) to intuitively demonstrate HGR-correlation attention's multi-scale dependency modeling, with supporting quantitative metrics.\n\nThese revisions directly address the request for robust justification of multi-scale design, leveraging both new experimental evidence and intuitive visualizations. We thank the reviewer for their valuable input, which will enhance the manuscript's rigor and clarity."}, {"review_id": "5P6hpEwOjm", "reviewer": "Reviewer_jAS7", "summary": "This paper proposes a Multi-Kernel Correlation Attention Vision Transformer (MK-CAViT). By leveraging multi-scale features and a Hirschfeld-Gebelein Rényi (HGR)-based interaction strategy, the method enables effective information exchange across scales. The proposed MK-CAViT outperforms compared methods across multiple tasks and datasets.", "questions": "1. The paper should clarify the differences and advantages of the proposed method over existing multi-scale designs.\n2. More recent comparison methods should be included to demonstrate the effectiveness of MK-CAViT better.\n3. An ablation study should be added to validate the necessity of using the three different kernel sizes.", "limitations": "The authors have discussed the limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n1. The idea of using Multi-Kernel design is reasonable. The proposed hierarchical multi-scale feature strategy balances efficiency and effectiveness.\n2. The method is evaluated on various tasks, including classification, segmentation, and detection. Experimental results show that the proposed approach is effective to a certain extent.\n\n\n\n### Weaknesses\n\n1. The method lacks novelty, as multi-scale attention has been extensively studied in previous works, e.g., [1, 2]. The main contribution is Fast-HGR Correlation, which has a limited impact on the performance.\n2. The method is not clearly presented. The authors only provide a simple flowchart (Fig. 1), with insufficient illustration. Moreover, the textual description in the method section does not correspond to the proposed architecture.\n3. The comparison is not up to date. The latest method compared is MPViT from 2022, which does not reflect the current state-of-the-art.\n4. The authors do not provide ablation studies to justify the necessity of using three kernels (large, mid, and small).\n\n\n\n[1] Multi-scale Vision Transformers, ICCV 2021.\n\n[2] MPViT: Multi-Path Vision Transformer for Dense Prediction, CVPR 2022.", "was_revised": true, "final_justification": "While the method demonstrates solid performance, its novelty is limited (primary concern, also noted by other reviewers). Therefore, I think a score of 4.", "rebuttal": "# Response to Reviewer jAS7\n﻿\nWe sincerely appreciate Reviewer jAS7 for the constructive feedback, which helps strengthen our manuscript. We address each concern—novelty, architectural clarity, up-to-date comparisons, and kernel ablation validation—with direct evidence from our experimental results and methodological details, as follows.\n﻿\n\n## 1. Novelty: Distinct Contributions Beyond Prior Multi-Scale Works\n﻿\nWe acknowledge the need to clarify MK-CAViT's innovations relative to existing multi-scale vision transformers, including recent advances like FasterViT (ICLR 2024)  and Agent-Swin (ECCV 2024) . Our core novelty lies in the *theoretical integration of multi-scale feature extraction with information-theoretic correlation modeling*, a framework absent in these works and validated by our experimental results.\n﻿\n\n### 1.1 Key Methodological Differences\n﻿\nAs shown in our results, MK-CAViT differs fundamentally from representative multi-scale methods, including 2024 state-of-the-art:\n﻿\n| Method          | Year | Multi-Scale Strategy       | Attention Mechanism       | Theoretical Basis       |\n|-----------------|------|----------------------------|----------------------------|-------------------------|\n| MViT         | 2021 | Cascaded expansion         | Multi Head Pooling Attention (MHPA) | Multiscale feature hierarchy |\n| MPViT        | 2022 | Parallel multi-scale patch embedding | Factorized multi-head self-attention | Multi-path feature fusion |\n| Agent-Swin   | 2024 | Agent-based information aggregation | Hybrid of Softmax and linear attention (via agent tokens) | Integration of global context and local details via agent tokens |\n| FasterViT   | 2024 | Hierarchical merging       | Hierarchical Attention (HAT) with carrier tokens | Efficient cross-window communication via carrier tokens |\n| **MK-CAViT (Ours)** | 2025 | Parallel multi-kernel + adaptive fusion | HGR-correlation attention | HGR maximal correlation |\n﻿\n\n### 1.2 Experimental Validation of Novelty\n﻿\nOur Fast-HGR module—central to this innovation—demonstrates significant, non-incremental impact compared to 2024 baselines:\n- Outperforms Agent-Swin-Base by 1.6% ImageNet Top-1 (85.6 vs. 84.0) and 1.3% COCO AP$^\\text{b}$ (50.3 vs. 49.0) through more effective cross-scale correlation modeling.\n- Surpasses FasterViT-B1 by 0.8% ImageNet accuracy and 1.0% ADE20K mIoU (50.8 vs. 49.8) while maintaining comparable computational cost (Table 3 in main text).\n- Ablations confirm removing Fast-HGR reduces performance by 0.9–1.5% across tasks (Table 4), validating its critical role absent in 2024 methods.\n﻿\nThis contrasts with Agent-Swin's reinforcement learning-based selection and FasterViT's heuristic token merging, neither of which employs a theoretically grounded mechanism for modeling nonlinear cross-scale dependencies.\n﻿\n\n## 2. Architectural Clarity: Detailed Explanations Grounded in Experiments\n﻿\nWe agree that architectural clarity can be enhanced. We will revise the description using concrete experimental anchors from the paper:\n﻿\n\n### 2.1 Module-by-Module Breakdown\n﻿\n- **Multi-Kernel Tokenization**: Parallel 3×3, 7×7, 15×15 convolutions generate scale-specific features. Table 6 confirms this configuration outperforms alternatives (e.g., 3×3+5×5+7×7) by 1.9% ADE20K mIoU, as it spans fine textures (3×3), mid-range context (7×7), and global structure (15×15).\n  \n- **Dynamic Normalization**: Unlike static LayerNorm/BatchNorm, this scale-adaptive mechanism improves ImageNet-C mean corruption error (mCE) by 1.7% and achieves 0.5–1.3% higher accuracy across tasks (Table 4). It adapts $\\gamma$ and $\\beta$ per kernel: $\\gamma_k = \\text{sigmoid}(\\mathbf{W}_k \\cdot \\text{avg}(\\mathbf{F}_k))$, where $\\mathbf{F}_k$ is the feature map for kernel size $k$.\n﻿\n- **Fast-HGR Attention**: Combines cosine similarity (local dependence) and trace regularization (global consistency) to model cross-scale correlations. Ablations show this reduces complexity to $O(N^2)$ while preserving gains—critical for efficiency (Section 3.2).\n﻿\n- **Adaptive Fusion**: Gated by HGR scores, this merges scales task-awarely. For example, small-object detection relies more on 3×3 features, while large-object detection emphasizes 15×15.\n﻿\n\n### 2.2 Revised Visualization and Terminology\n﻿\n- **Figure 1** will be redrawn to include:\n  - Clear labels for all modules (e.g., \"Fast-HGR Module\" with annotation of its 0.9% accuracy impact).\n  - Data flow arrows labeled with kernel sizes and fusion weights ($\\alpha_3, \\alpha_7, \\alpha_{15}$).\n  - Color-coded scale branches aligned with Table 6's kernel analysis.\n﻿\n- **Glossary Addition**: Key terms will be defined in Section 3.2 using experimental context:\n  - \"Multi-Token Attention (Dense/Sparse)\": Dense uses a single attention matrix (50% higher FLOPs, +0.2% ImageNet), while sparse limits cross-scale interactions (20% lower FLOPs, -2.2% COCO AP) (Table 4).\n  - \"HGR-correlation Attention\": Mechanism increasing small-object AP$_\\text{S}$ by 2.1% vs. standard attention (Table 4).\n﻿\n\n## 3. Up-to-Date Comparisons with 2024 State-of-the-Art\n﻿\nOur results include comprehensive comparisons with 2024 methods, including FasterViT (ICLR 2024) and Agent-Swin (ECCV 2024), demonstrating MK-CAViT's superiority:\n﻿\n| Method                  | #Params (M) | FLOPs (G) | ImageNet Top-1 (%) | COCO AP (%) | ADE20K mIoU (%) |\n|-------------------------|-------------|-----------|--------------------|-------------|-----------------|\n| Agent-Swin-Base [2]     | 88.0        | 15.4      | 84.0               | 49.0        | 49.5            |\n| FasterViT-B1 [1]        | 87.6        | 14.9      | 84.8               | 49.1        | 49.8            |\n| MK-CAViT-Base (Ours)    | 88.0        | 15.6      | **85.6**           | **50.3**    | **50.8**        |\n| **Performance Gain**    | (±0.0)      | (+0.7)    | (+0.8 to +1.6)     | (+1.2 to +1.3) | (+1.0 to +1.3) |\n﻿\n\nKey advantages over 2024 baselines:\n- **Small-object detection**: MK-CAViT's 31.9% AP$_\\text{S}$ exceeds FasterViT's 30.8% (+1.1%) and Agent-Swin's 30.5% (+1.4%), due to 3×3 kernel preservation of fine details (Table 6).\n- **Boundary precision**: 1.8% higher boundary IoU (bIoU = 68.3%) on ADE20K compared to FasterViT (66.5%), as HGR attention aligns local edges with global regions.\n- **Efficiency-accuracy tradeoff**: Despite 0.7G higher FLOPs than FasterViT, 1.0%+ gains across tasks justify the investment in HGR correlation, which 2024 methods lack.\n﻿\nThese results are presented in Section 4.2 of the main text, with explicit discussion of how 2024 methods' focus on speed (FasterViT) or agent-based selection (Agent-Swin) limits their ability to model complex cross-scale dependencies.\n﻿\n\n## 4. Validation of Three Kernel Sizes (3×3+7×7+15×15)\n﻿\nOur existing ablations (Table 4-B and Table 6) directly validate the necessity of three kernels, avoiding redundant experiments:\n﻿\n### 4.1 Kernel Configuration Ablations\n﻿\nTable 4-B shows that deviating from 3×3+7×7+15×15 degrades performance:\n- Smaller kernels (3/5/7): ImageNet Top-1 drops by 1.7% (85.6 → 83.9), COCO AP$^\\text{b}$ by 1.6% (50.3 → 48.7), ADE20K mIoU by 1.9% (50.8 → 48.9) due to insufficient global context.\n- Larger kernels (7/11/15): Further declines (ImageNet -2.1%, COCO -2.0%, ADE20K -2.2%) as fine textures are lost (consistent with Table 6's 7×7+11×11+15×15 result of 49.1% mIoU).\n﻿\n### 4.2 Multi-Scale Synergy in Table 6\n﻿\nThe 3×3+7×7+15×15 combination outperforms all alternatives:\n- Surpasses 3×3+5×5+7×7 by 1.9% mIoU (50.8 vs. 48.9), validating 15×15's role in capturing global context (e.g., room layouts).\n- Outperforms 5×5+11×11+15×15 by 1.2% mIoU (50.8 vs. 49.6), confirming 3×3's necessity for fine textures (e.g., \"pipe\" edges).\n- Exceeds 3×3+9×9+15×15 by 0.6% mIoU (50.8 vs. 50.2), highlighting 7×7 as a critical mid-scale bridge (e.g., linking \"window\" to \"wall\").\n﻿\nEach kernel addresses specific limitations: 3×3 captures details, 7×7 mediates scales, 15×15 provides context. Removing any breaks this hierarchy, as shown by consistent performance drops.\n﻿\n\n## 5. Summary of Revisions\n﻿\nTo address Reviewer jAS7's concerns, we will:\n1. Strengthen novelty discussion by emphasizing comparisons with 2024 methods (FasterViT, Agent-Swin) and linking HGR-theoretic innovations to experimental gains.\n2. Enhance architectural clarity with a revised Figure 1 and glossary, grounding descriptions in experimental results (e.g., Dynamic Norm's 1.7% mCE improvement).\n3. Explicitly highlight 2024 baseline comparisons in Section 4.2, emphasizing MK-CAViT's gains in small-object detection and boundary precision.\n4. Reference Table 4-B and Table 6  to validate three kernel sizes, with task-specific analysis of each kernel's role.\n﻿\nThese revisions directly address each concern, leveraging existing experimental data to ensure rigor. We thank the reviewer for their valuable input, which will enhance the manuscript's clarity and impact."}, {"review_id": "pq8o5rBDy5", "reviewer": "Reviewer_bLHh", "summary": "This paper proposes the Multi-Kernel Correlation-Attention Vision Transformer (MK-CAViT), a unified framework designed to enhance contextual understanding and multi-scale integration in vision tasks. Building on the Hirschfeld-Gebelein-Rényi (HGR) theory, MK-CAViT introduces multi-kernel correlation attention modules, combining global and local spatial representations through innovative multi-scale tokenization and fusion mechanisms.", "questions": "Can the authors provide more quantitative comparisons with state-of-the-art large-scale models?\n\nAre there potential limitations in terms of generalization to other domains or tasks?", "limitations": "The authors include a separate section on limitations, but additional discussion of scenarios where the method may underperform would be beneficial.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nThe correlation-based attention mechanism is novel and demonstrates strong potential for improving multi-scale feature integration.\n\nThe paper provides a theoretical justification and empirical results that support the method's effectiveness on standard benchmarks.\n\nThe discussion of computational efficiency is valuable for practical deployment.\n\nWeaknesses\n\nWhile efficiency is discussed, the evaluation could benefit from comparisons with larger-scale or more recent models to better position the proposed method in terms of real-world applicability and scalability.\n\nCertain methodological aspects—such as implementation details or the choice of kernels—could be clarified further to enhance reproducibility.\n\nThe ablation studies are promising, but more detailed exploration of individual module contributions would strengthen the work.", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer bLHh\n﻿\nWe appreciate the constructive feedback from Reviewer bLHh, which helps strengthen our work. Below, we address each concern with detailed analyses, supplementary experiments, and clarifications aligned with our paper's methodology, ensuring all claims are grounded in our original findings.\n﻿\n﻿\n### 1. Quantitative Comparisons with State-of-the-Art Large-Scale Models\n﻿\nThe reviewer rightly emphasized the need for broader comparisons with large-scale and recent models. We expanded our evaluations to include  state-of-the-art methods and larger model variants to contextualize MK-CAViT’s scalability, drawing on our original experimental setup.\n﻿\n#### 1.1 Extended Comparisons with 2024 Models and Large-Scale Variants  \n﻿\nOur original results showed MK-CAViT-Base (88M parameters) outperforming established models, and we further validate the scalability of correlation attention by incorporating comparisons with strong multiscale ViTs and correlation-based architectures:  \n﻿\n| Model              | Year | Params (M) | ImageNet Top-1 (%) | COCO AP$^\\text{b}$ | FLOPs (G) |\n| ------------------ | ---- | ---------- | ------------------ | ------------------ | --------- |\n| CSWin-Base         | 2021 | 78    | 84.2   | 48.7    | 15.0      |\n| MViTv2-B           | 2021 | 52  | 84.4    | 51.0   | 10.2      |\n| FasterViT-4        | 2024 | 424.6      | 85.4       | 52.8    | 36.6      |\n| Agent-Swin-Large   | 2024 | 197    | 85.2    | 53.5       | 11.8      |\n| ConvNeXt-Large     | 2022 | 198    | 84.3      | 53.1     | 34.4      |\n| **MK-CAViT-Large** | -    | 186    | **86.1**   | **54.2**      | **28.9**  |  \n| **MK-CAViT-Base**  | -    | 88    | 85.6   | 53.3      | 15.6      |  \n﻿\n\nMK-CAViT-Base outperforms CSWin-Base by 1.4% on ImageNet with comparable parameters, leveraging correlation attention to surpass purely convolutional designs. Against MViTv2-B, it achieves 1.2% higher Top-1 accuracy with 2% fewer parameters, as our multi-kernel structure outperforms MViT’s linear attention in capturing fine-grained dependencies.  \n﻿\nFor large-scale variants, MK-CAViT-Large outperforms FasterViT-4 by 0.7% on ImageNet with 54% fewer parameters and 21% lower FLOPs, addressing FasterViT’s inefficiency in large configurations. Against Agent-Swin-Large, it delivers 0.9% higher Top-1 accuracy with 1% fewer parameters, leveraging multi-kernel design to outperform Agent Attention’s linear complexity. Compared to ConvNeXt-Large, our model achieves 1.8% higher ImageNet accuracy with comparable parameters, validating correlation attention’s superiority over pure convolutional designs in scaling scenarios.  \n﻿\nAdditionally, integrating our multi-kernel correlation attention with the earlier multi-scale analysis, MK-CAViT-Large surpasses CSWin-Base and MViTv2-B across resolutions—achieving 85.6% (384×384) vs. 85.0% (MViTv2-B) and 84.2% (CSWin-Base)—confirming that correlation-based multi-kernel designs outperform both convolutional and linear attention paradigms in scaling scenarios.\n﻿\n﻿\n### 2. Clarifying Methodological Details and Kernel Selection\n﻿\nThe reviewer noted the need for clearer implementation details and kernel rationale. We expand on these using our paper’s ablations  and design principles.\n﻿\n#### 2.1 Kernel Configuration Rationale\n﻿\nAs validated in our kernel sensitivity analysis, our multi-kernel design (3×3, 7×7, 15×15) is optimized for hierarchical feature extraction:\n﻿\n| Kernel Type | Size/Stride/Padding | Output Size      | Channels | Receptive Field Impact |\n| ----------- | ------------------- | ---------------- | -------- | ---------------------- |\n| Small       | 3x3/1/1      | $H \\times W$     | 64       | Fine-grained details (e.g., textures) |\n| Medium      | 7x7/2/3     | $H/2 \\times W/2$ | 128      | Mid-level semantics (e.g., object parts) |\n| Large       | 15x15/1/7    | $H \\times W$     | 256      | Global context (e.g., scene structures) |\n﻿\n\nThe medium kernel uses stride=2 with padding=3 to halve spatial dimensions; the large kernel uses stride=1 with padding=7 to maintain resolution (\"same\" padding). Additional ablations (consistent with Table 4) validate this configuration:\n﻿\n| Kernel Configurations              | ImageNet Top-1 (%) | COCO AP$^\\text{S}$ (small) | COCO AP$^\\text{L}$ (large) | FLOPs (G) |\n| ---------------------------------- | ------------------ | -------------------------- | -------------------------- | --------- |\n| Single kernel (7×7)      | 83.2    | 31.2    | 62.5    | 78      |\n| Dual kernels (3×3 + 7×7)      | 84.5      | 33.8       | 63.1      | 82   |\n| **Triple kernels (3×3+7×7+15×15)** | **85.6**     | **35.9**    | **64.8**    | **89**    |\n| Triple kernels (5×5+9×9+13×13)     | 85.1   | 34.7    | 64.2     | 94    |\n﻿\n\nThe 3×3 kernel captures fine textures (+4.7 AP$^\\text{S}$ over single kernel), 7×7 bridges scales, and 15×15 models global context (+2.3 AP$^\\text{L}$ over single kernel), as supported by our multi-scale fusion mechanism (Section 3.2.3).\n﻿\n#### 2.2 Implementation Details for Reproducibility\n﻿\nConsistent with our training protocols:\n- **Multi-Scale Tokenization**: As in Eq. (7), each kernel is implemented via Conv2D with specified strides/padding.\n- **Optimizer**: AdamW (weight decay=0.05, β₁=0.9, β₂=0.999), with a cosine-decayed learning rate starting at 5e-5 (Base model).\n- **Training**: 200 epochs with 20k warmup steps, DropPath (0→0.1), and 2D learnable relative position embeddings.\n- **Hardware**: 8×A100 GPUs (40GB) with FP16 mixed precision (accelerating convergence by 2×).\n﻿\nThese details ensure reproducibility of our original results.\n﻿\n﻿\n### 3. Detailed Module-Wise Contribution Analysis\n﻿\nWe expand on Table 4 to quantify each module’s impact:\n﻿\n#### 3.1 Fast-HGR Correlation Attention\n﻿\nReplacing Fast-HGR with dot-product attention (Table 4 ablation) degrades performance:\n- ImageNet Top-1: 85.6 → 84.7 (-0.9%)\n- COCO AP$^\\text{b}$: 50.3 → 48.8 (-1.5)\n- ADE20K mIoU: 50.8 → 50.0 (-0.8)\n- Small-object AP$^\\text{S}$: 35.9 → 33.8 (-2.1)\n﻿\nFast-HGR is critical for aligning multi-scale features, particularly benefiting small objects by preserving local dependencies via cosine similarity (Eq. 3) and global consistency via trace regularization (Eq. 4). Training slows by 25% without its optimized gradients.\n﻿\n#### 3.2 Multi-Scale Kernel Fusion\n﻿\nAblating multi-kernel pathways (single 7×7 kernel) reduces performance:\n- ImageNet Top-1: 85.6 → 83.2 (-2.4%)\n- COCO AP$^\\text{S}$: 35.9 → 31.2 (-4.7)\n- COCO AP$^\\text{L}$: 64.8 → 62.5 (-2.3)\n﻿\nParallel pathways model scale-specific patterns: 3×3 drives small-object gains; 15×15 enhances large-object performance, validated by our multi-scale fusion results.\n﻿\n#### 3.3 Dynamic Normalization\n﻿\nOur dynamic normalization outperforms alternatives:\n- LayerNorm replacement: ImageNet Top-1 85.6 → 85.1 (-0.5%)\n- BatchNorm replacement: COCO AP$^\\text{b}$ 50.3 → 48.4 (-1.9); ADE20K mIoU 50.8 → 49.1 (-1.7)\n﻿\nBatchNorm fails due to scale-specific batch sensitivity; dynamic normalization stabilizes training across kernels.\n﻿\n#### 3.4 Hierarchical Gating Fusion\n﻿\nOur two-stage fusion (small+medium → +large) balances efficiency and accuracy:\n- Dense fusion: +0.2% ImageNet Top-1 but +50% FLOPs and -0.5 COCO AP$^\\text{b}$.\n- Sparse fusion: -0.8% Top-1, -2.2 COCO AP$^\\text{b}$, but -20% FLOPs.\n- Our gating: Matches dense accuracy with 30% fewer FLOPs (Eq. 11-12).\n﻿\n﻿\n### 4. Generalization to Other Domains and Tasks\n﻿\nTo address generalization concerns, we tested MK-CAViT on diverse datasets, leveraging its multi-scale design:\n﻿\n#### 4.1 Multimodal Emotion Recognition\n﻿\nOn IEMOCAP (audio-visual emotion recognition), MK-CAViT-Base achieved 73.5% weighted accuracy, outperforming ViT-Base (68.5%), Swin-Base (70.1%), and ConvNeXt-Base (69.8%). The 3×3 kernel captures facial microexpressions; 15×15 models global dynamics. Fast-HGR integrates audio-visual features, with the medium kernel aligning temporal segments.\n﻿\n#### 4.2 Medical Imaging \n﻿\nFor ISIC2018 skin lesion segmentation, MK-CAViT-Base achieved 83.43% mIoU, exceeding TransFuse (80.63%), EfficientNet-B4 (81.21%), and Swin-UNet (82.78%). The 3×3 kernel detects subtle lesion boundaries; 15×15 captures global structure, with HGR modeling spatial relationships between lesions and surrounding skin.\n﻿\n#### 4.3 Remote Sensing \n﻿\n- Houston 2018 (land-cover classification): 93.68% overall accuracy, outperforming 3D-CNN (89.59%), ViT-Base (91.87%), and FasterViT-Small (92.13%). The 15×15 kernel captures large-scale geographical patterns; 3×3 identifies small structures.\n- Vaihingen (urban segmentation): 84.43% mIoU, surpassing U-Net (80.15%), DeepLabV3+ (81.56%), and Swin-Unet (82.62%). The 7×7/15×15 kernels model buildings/roads; 3×3 segments small objects (e.g., street furniture).\n﻿\nThese results demonstrate strong cross-domain generalization, with the multi-scale design adapting to diverse data.\n﻿\n﻿\n### 5. Limitations and Underperformance Scenarios\n﻿\nWe expand the \"Limitations\" section to discuss scenarios where MK-CAViT may underperform:\n﻿\n- **Extremely low-resolution images** (<32×32): The 15×15 kernel covers most pixels, causing 2-3% accuracy drops on ImageNet-32. Future work will implement adaptive kernel sizing (e.g., 1×1 for low-res inputs).\n- **Highly noisy data**: Correlation-based attention amplifies noise. Adding non-local means filtering could mitigate this (preliminary tests: +1.2% mIoU on noisy ADE20K).\n- **Edge deployment with strict latency** (<10ms): MK-CAViT-Base operates within 15ms on edge GPUs, but the Large variant may be too slow. A \"Tiny\" variant (12M parameters) prioritizing 3×3 kernels reduces latency to 7ms (81.2% ImageNet Top-1).\n- **Extremely imbalanced datasets**: Correlation attention may bias toward majority classes. Class-aware weighting in HGR loss could improve performance (planned).\n﻿\n﻿\nThese additions strengthen rigor while aligning with our original methodology. We thank the reviewer for insights that improved our work."}, {"review_id": "ViQfb1RvdV", "reviewer": "Reviewer_txY3", "summary": "This submission explores the integration of the HGR-correlation attention mechanism into transformer architecture, paired with a parallel multi-kernel Vision Transformer (ViT) design. The approach aims to improve ViT performance. The use of HGR-correlation attention appears to be novel, and the experimental results across several benchmarks suggest it may offer a viable alternative to existing ViT models.", "questions": "See my comments above.", "limitations": "As noted above, the main concerns lie in the unclear motivation for developing FastHGR over existing I-Soft-HGR, and in the correctness of its derivation and implementation. I encourage the authors to provide a thorough clarification in the rebuttal.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\nThe main strength of the work lies in the novel application of HGR-correlation in attention computation and the multi-scale fusion strategy based on HGR, which shows promise for enhancing content learning across different scales. The ablation studies support some of the design choices, though not all aspects are thoroughly validated.\n\nWeaknesses.\n--Justification for Replacing Standard Attention. The paper proposes replacing standard attention with Fast-HGR, but it does not convincingly address whether multi-layer standard attention is insufficient for modeling complex nonlinear dependencies. Moreover, it seems intuitive to apply a Soft-HGR loss on Q and K to encourage their correlation, without explicitly using HGR as attention weights. A deeper discussion, supported by attention map visualizations, would clarify the role Fast-HGR plays. The current benchmark table alone does not provide sufficient insight. \n-- I-Soft-HGR and Fast-HGR.  (1The first term in Equation (1) of I-Soft-HGR is missing a minus sign, which should be corrected.  (2)  The motivation behind introducing Fast-HGR is not clearly articulated. Why can't I-Soft-HGR be used directly? Is its computational cost prohibitive for training and inference? In fact, when comparing Algorithm 1 and Equation (6) of [43] with Equation (3) in Section 3.1.2, the only notable difference appears to be vector normalization. Therefore, the claim that Fast-HGR significantly reduces computational complexity compared to full covariance matrix operations seems unsubstantiated. A more detailed discussion and empirical validation (using I-Soft-HGR as a baseline) are necessary to support this claim. (3) The coefficient λ appears without explanation. In the original Soft-HGR and I-Soft-HGR formulations, λ is derived to be 1/2. \n-- Fast-HGR.  (1) The first term in Equation (6) represents an empirical expectation, but it is averaged over N−1 instead of N. (2)  Using Equation (6) directly as a loss term is problematic. The objective should be to maximize F-HGR to enhance nonlinear dependence, which implies that “−F-HGR” should be minimized. This is consistent with Reference [43], which minimizes “−I-Soft-HGR”. However, upon reviewing the supplementary code, it appears that the implementation minimizes “+F-HGR”, which contradicts the intended objective. This is not a typographical error but a fundamental issue in the implementation. \n-- Proofs in the supplementary.  (1) The proof of Theorem 1 is overly brief. A more rigorous and comprehensive derivation is necessary; otherwise, the argument is difficult to follow and verify.  (2) for theorem 2, It is unclear why L is referred to as the Lipschitz constant: it appears to involve the covariance whose upper bound is not estimated.   (3 The derivation of Fast-HGR lacks a clear logical progression. Terms are substituted without justification or rigorous error analysis, making the reasoning difficult to follow.\nUnclear Ablation Study. The ablation study does not clearly explain how the Fast-HGR module is removed. Is it replaced with a standard attention module? Simply removing it reduces the number of parameters (as evidenced by Table 4), making it difficult to determine whether performance drops are due to the module removal or the reduced model capacity. \n-Module validation\n1. Unsubstantiated Claims. The introduction outlines several claimed contributions related to module design—such as learnable gating, dynamic weighting, and dynamic attention. However, these claims are not substantiated by experimental validation. If these designs do not contribute significantly, the paper should avoid overstating them; otherwise providing the corresponding ablation studies.\n2. Section 4.4 is too brief. (1) It is unclear what “dynamic normalization” refers to. Presumably, it relates to the use of feature normalization described in Section 3.1. This should be explicitly clarified. (2) The terms “dense attention” and “sparse attention” are introduced without definition. Their meaning and implementation should be clearly explained. (3) The claim that the proposed approach enables task adaptation does not contribute to the work. Similar behavior could likely be achieved by other existing methods, and the paper does not provide evidence that this capability is unique to the proposed design.\n3. Unclear Competitor Selection. The criteria for selecting baseline models in the comparative study are not discussed. In particular, for multiscale ViTs, it is unclear why stronger models such as CSwin, SwinV2 or MViTv2 were not included. \n\nOther issues.\nFigure 1 is misleading. The Large/Mid/Small-Kernel attention blocks shown do not actually exist in the implementation; rather, they appear to describe the subsequent operations, not represent actual network blocks.", "was_revised": true, "final_justification": "The rebuttal has adequately addressed my main concerns. Although the novelty of this work is not especially large in my opinion, I think the paper can nevertheless be accepted.", "rebuttal": "# Response to Reviewer txY3\n\nWe thank Reviewer txY3 for the rigorous and insightful feedback, which has helped us refine the theoretical rigor and clarity of our work. Below, we address each concern with detailed corrections, supplementary analyses, and direct references to our original methodology, ensuring all responses are grounded in our experimental results.\n\n## 1. Justification for Replacing Standard Attention with Fast-HGR\n\nThe reviewer rightly questions why Fast-HGR is preferable to standard attention and how it differs from applying Soft-HGR as a standalone loss.\n\n### 1.1 Limitations of Standard Attention\n\nStandard dot-product attention with Softmax primarily captures linear correlations and can suppress subtle nonlinear dependencies (e.g., between low-contrast textures or small objects). In contrast, HGR maximal correlation (Eq. 1) is theoretically designed to model *nonlinear statistical dependencies*, which we approximate efficiently via Fast-HGR. Our ablation study (Table 4) confirms this:\n\n- Replacing Fast-HGR with standard attention reduces ImageNet Top-1 accuracy by 0.9%.\n- COCO AP$^\\text{b}$ drops by 1.5%.\n- ADE20K mIoU decreases by 0.8%.\n- Small-object detection (AP$^\\text{S}$) falls by 2.1%.\n\nThese results are consistent with the inability of standard attention to model fine-grained nonlinear relationships.\n\n### 1.2 Attention Map Visualizations\n\nTo clarify, we add supplementary Figure S2 (in revision) showing:\n\n- Fast-HGR attention maps preserve fine details (e.g., leaf veins, fabric textures) that standard attention blurs.\n- For small objects (e.g., \"bird\" in ImageNet), Fast-HGR attention focuses more accurately on the target, whereas standard attention spreads to background regions.\n\n### 1.3 Fast-HGR in Attention vs. F-HGR as a Loss\n\nThe reviewer suggests applying F-HGR as a loss on Q and K, but our experiments show integrating Fast-HGR into attention weights is more effective. This is because attention weights directly modulate feature aggregation during forward propagation, whereas a standalone loss acts indirectly. Supplementary ablations (Table S1) validate:\n\n- Fast-HGR attention + F-HGR loss: 85.8% Top-1\n- Fast-HGR attention alone: 85.6% Top-1\n- Standard attention + F-HGR loss: 84.7% Top-1\n\n## 2. Clarifications on I-Soft-HGR and Fast-HGR\n\n### 2.1 Equation Corrections and λ Motivation\n\n#### I-Soft-HGR Formulation and Sign Clarification\n\nI-Soft-HGR in our derivation (Appendix B) includes a negative sign for the primary correlation term to align with its role as a loss function (minimizing the negative correlation to maximize dependence).\n\nThe negative sign on the first term (expectation of $ \\mathbf{f}^\\top \\mathbf{g} $) confirms that I-Soft-HGR minimizes the negative of the HGR correlation, aligning with its goal to maximize statistical dependence. This resolves the sign ambiguity and will be clarified in the revised manuscript.\n\n#### λ in Fast-HGR vs. I-Soft-HGR\n\nI-Soft-HGR enforces strict constraints on feature and covariance distributions (e.g., $ \\text{cov}(\\mathbf{f}) = \\mathbf{I} $) with a fixed coefficient of $ -\\frac{1}{2} $ for the regularization term, ensuring rigid normalization. In contrast, Fast-HGR relaxes these constraints and introduces $ \\lambda $ to balance:\n\n- **Local dependence**: Cosine similarity (Eq. 3), which inherently normalizes features to the unit hypersphere (bounded in [-1,1]) without strict orthonormality.\n- **Global structure**: Trace of covariance products (Eq. 4), simplified from I-Soft-HGR’s Frobenius norm penalties on covariance deviations.\n\nWe selected $ \\lambda = 0.1 $ via cross-validation on ImageNet-10% (supplementary Figure S3), with stability in [0.05, 0.2]. This flexibility avoids I-Soft-HGR’s rigid constraints, maintaining efficiency while preserving correlation modeling.\n\n### 2.2 Why Fast-HGR Over I-Soft-HGR?\n\nTo validate the practical advantage of Fast-HGR over I-Soft-HGR, we benchmarked their correlation calculation performance using randomly generated tensors. For each configuration, paired tensors $ f, g \\in \\mathbb{R}^{\\text{bz} \\times \\text{dim}} $ were generated, and the average execution time was measured over 10,000 trials (batch size = 128, dimensions = [10, 50, 100, 150, 200, 300, 400, 500, 800, 1000]).\n\nThe results (supplementary Table S2) show that Fast-HGR consistently outperforms I-Soft-HGR in speed across all dimensions:\n\n- For small dimensions (e.g., dim = 10), Fast-HGR is ~2.4× faster (0.00043s vs. 0.00104s).\n- For large dimensions (e.g., dim = 1000), Fast-HGR remains ~2.5× faster (0.00065s vs. 0.00164s).\n\nThis speed advantage arises from Fast-HGR’s streamlined design:\n\n- It replaces I-Soft-HGR’s multi-step feature normalization (mean subtraction + variance scaling) with cosine similarity, which inherently combines normalization and correlation measurement.\n- It simplifies I-Soft-HGR’s nested covariance constraints (e.g., penalizing deviations from $ \\mathbf{I} $) with a direct trace of covariance products, eliminating redundant statistical regularizations.\n\nBeyond efficiency, Fast-HGR outperforms I-Soft-HGR by 0.4% on ImageNet Top-1 accuracy (Table 1). This is because its relaxed constraints better preserve feature geometry, critical for capturing fine-grained dependencies in vision tasks (e.g., small-object textures and global scene contexts).\n\n## 3. Fast-HGR Formulation and Implementation\n\n### 3.1 Equation (6) and Loss Objective\n\n- **N-1 Averaging**: The use of $ N-1 $ (instead of $ N $) in Eq. (6) follows unbiased covariance estimation (consistent with Eq. 4), ensuring the trace term aligns with statistical practice for finite batches—this matches the formulation in our ablation studies (Table 4).\n\n- **Loss Sign Correction**: The reviewer identified a critical issue. The incorrect implementation of minimizing $ +L_{\\text{F-HGR}} $ in the supplementary code was a leftover from ablation experiments where we tested reversed objectives. Our intended implementation, consistent with the goal of maximizing feature dependence, minimizes $ -L_{\\text{F-HGR}} $. The code will be updated to reflect this correction.\n\n### 3.2 Theoretical Rigor in Supplementary Proofs\n\n- **Theorem 1 (Upper Bound)**: We expand the proof to explicitly show:\n\n  \n  $$\\rho^{(k)}(X,Y) \\leq \\text{F-HGR}(f(X),g(Y)) + O(1/\\sqrt{N})$$\n  \n\n  using McDiarmid’s inequality to bound the error from replacing population expectations with empirical batch averages (Appendix A). Supplementary Figure S4 validates convergence as $ N $ increases, supporting the error bound.\n\n- **Theorem 2 (Lipschitz Continuity)**: We clarify that $ L $ is bounded by $ \\max(1, \\lambda \\cdot \\sigma_f \\cdot \\sigma_g) $, where $ \\sigma_f, \\sigma_g \\leq C $ (constants) due to batch normalization of features. This ensures:\n\n  \n  $$| \\text{F-HGR}(X) - \\text{F-HGR}(X') | \\leq L \\|X - X'\\|$$\n \n\n  preventing unstable gradients in deep networks, consistent with our training stability results.\n\n## 4. Ablation Study Clarity\n\n- **Fast-HGR Removal**: When ablating Fast-HGR, we replace it with standard attention while keeping $ W_q, W_k, W_v $ dimensions identical to preserve model capacity. The parameter reduction (-1.8M, Table 4) stems solely from removing trace regularization layers, not from altering feature projection dimensions. Performance drops thus reflect Fast-HGR’s unique contribution to modeling correlations.\n\n- **Dense/Sparse Attention Definitions**:\n\n  - *Dense attention*: All tokens (from small/medium/large kernels) attend globally in a single matrix, with complexity $ O((n_s+n_m+n_l)^2 d) $.\n  - *Sparse attention*: Tokens attend only within their kernel pathway, with complexity $ O(n_s^2 d + n_m^2 d + n_l^2 d) $.\n\n  Our gated fusion (Eqs. 11-12) achieves $ O((n_s+n_m)n_l d) $ complexity, balancing efficiency and cross-scale interaction—outperforming dense (50% higher FLOPs) and sparse (2.2% lower COCO AP$^\\text{b}$) alternatives (Table 4).\n\n## 5. Validating Modules and Competitor Selection\n\n- **Dynamic Normalization**: Refers to scale-adaptive normalization where small kernels use LayerNorm (preserving fine details) and large kernels use BatchNorm (stabilizing global statistics), with learnable gates $ \\gamma_i $ to balance their contributions. Ablations (Table 4) show this outperforms fixed LayerNorm (-0.5% Top-1) or BatchNorm (-1.3% Top-1), with 1.7% lower mean corruption error (mCE) on ImageNet-C than LayerNorm.\n\n- **Learnable Gating**: Gating vectors $ \\alpha, \\beta $ (Eq. 11) learn task-specific weights: e.g., $ \\alpha \\approx 0.6 $ for segmentation (prioritizing small kernels) vs. $ \\beta \\approx 0.7 $ for classification (prioritizing mid-scale features). Removing gating reduces COCO AP$^\\text{b}$ by 1.1% , validating its role.\n\n- **Additional Competitors**: To comprehensively validate our model's performance across scales and architectures, we incorporate CSWin, MViTv2, and other robust multiscale models into our comparative analysis.  MK-CAViT-Base outperforms CSWin-Base by 1.4% on ImageNet (85.6% vs. 84.2%) and MViTv2-B by 1.2% (85.6% vs. 84.4%). MK-CAViT-Large (186M) further excels, surpassing FasterViT-4 (424.6M) by 0.7% on ImageNet. These results highlight the effectiveness of correlation attention in outperforming convolutional and linear attention designs in medium-scale scenarios.\n\n\n## 6. Figure 1 Correction\n\nThe reviewer is correct: Figure 1 depicts conceptual pathways, not literal \"attention blocks\". We revise the caption to: \"Schematic of multi-kernel pathways with shared Fast-HGR attention computation\" and adjust the diagram to show unified attention layers across paths, clarifying that attention is computed once and shared across scales.\n\n\n\nWe address all concerns by correcting equations, expanding theoretical proofs, clarifying ablations, and validating modules with experimental results. These revisions strengthen rigor while preserving our core contributions. We thank the reviewer for their invaluable feedback, which has significantly improved our work."}]

Author rebuttal:

Meta-review: This paper introduces MK-CAViT, a multi-kernel correlation-attention Vision Transformer grounded in HGR theory, with a Fast-HGR approximation for efficiency. The motivation is clear, the methodology is technically sound, and the experiments across ImageNet, COCO, and ADE20K are comprehensive.

Reviewers initially raised concerns about the necessity of multi-scale design, novelty, clarity of architecture, and comparisons with recent methods. The rebuttal provided new single-scale kernel experiments, visualizations, updated baselines, detailed ablations, and clarified theoretical and implementation issues. These additions effectively addressed the main concerns, and reviewers increased their ratings.

Overall, the paper is well-motivated, clearly written, and demonstrates solid empirical and theoretical contributions. I recommend acceptance.
Decision: Accept (poster)