IFORMER: INTEGRATING CONVNET AND TRANSFORMER FOR MOBILE APPLICATION
Authors: Chuanyang Zheng
Venue: ICLR 2025 Poster (2025)
Keywords: Lightweight Networks
Efficient Networks
Vision Transformers
Classification
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: We present a new family of mobile hybrid vision networks, called iFormer, with a focus on optimizing latency and accuracy on mobile applications. iFormer effectively integrates the fast local representation capacity of convolution with the efficient global modeling ability of self-attention. The local interactions are derived from transforming a standard convolutional network, \textit{i.e.}, ConvNeXt, to design a more lightweight mobile network. Our newly introduced mobile modulation attention removes memory-intensive operations in MHA and employs an efficient modulation mechanism to boost dynamic global representational capacity. We conduct comprehensive experiments demonstrating that iFormer outperforms existing lightweight networks across various tasks. Notably, iFormer achieves an impressive Top-1 accuracy of 80.4% on ImageNet-1k with a latency of only 1.10 ms on an iPhone 13, surpassing the recently proposed MobileNetV4 under similar latency constraints. Additionally, our method shows significant improvements in downstream tasks, including COCO object detection, instance segmentation, and ADE20k semantic segmentation, while still maintaining low latency on mobile devices for high-resolution inputs in these scenarios. Code and models are available at: https://github.com/ChuanyangZheng/iFormer.

Published as a conference paper at ICLR 2025
IFORMER: INTEGRATING CONVNET AND TRANS- FORMER FOR MOBILE APPLICATION
Chuanyang Zheng
Independent Researcher
chuanyang zhengês jtu edu.ed..oo cn
ABSTRACT
We present a new family of mobile hybrid vision networks, called iFormer, with a focus optimizing latency and accuracy on mobile applications. iFormer effec- tively integrates the fast local representation capacity of convolution with the effi- cient global modeling ability of self-attention. The local interactions derived from transforming standard convolutional network, i.e., ConvNeXt, to design a more lightweight mobile network. Our newly introduced mobile modulation at- tention removes memory-intensive operations in MHA and employs an efficient modulation mechanism to boost dynamic global representational capacity. We conduct comprehensive experiments demonstrating that iFormer outperforms ex- isting lightweight networks across various tasks. Notably, iFormer achieves an impressive Top-1 accuracy of 80.4% on ImageNet-Ik 1k with latency of only 1.10 ms on an iPhone 13, surpassing the recently proposed MobileNetV4 under simi- lar latency constraints. Additionally, our method shows significant improvements in downstream tasks, including COCO object detection, instance segmentation, and ADE20K semantic segmentation, while still maintaining low latency on mo- bile devices for high-resolution inputs in these scenarios. Code and models are available at: http:/giihwbboooocaaaaaanaannnee
INTRODUCTION
Building lightweight neural networks facilitates real-time analysis of images and videos cap- tured by mobile applications such as smartphones. This not only enhances privacy protection and security by processing data locally on the device but also improves overall user experience.
Through the decades, convolutional neural net- works (CNNs) (Krizhevsky et al., 2012; Szegedy et al., 2015; He et al., 2016) have emerged as the primary choice for balancing latency and per- formance on resource-constraiied mobile devices. However, significant limitation of CNNs is their reliance local sliding window mechanism, which imposes crucial inductive biases that may hinder modeling flexibility. Recently, the soaring development of vision transformers (ViTs) (Doso- vitskiy al., 2020) has begun to dominate vari- ous computer vision tasks, including image classi- fication (Zhai al., 2022), object detection (Liu et al., 2021a). and semantic segmentation (Xie al., 2021). The core mechanism underlying ViTs is self-attention, which dynamically learns interac- tions between all image patches. This enables the model to focus important regions adaptively and capture more global features. Nevertheless, de- ploying ViTs mobile devices with limited sources poses significant challenges. On the one hand, the quadratic computational complexity of attention renders them unsuitable for large feature maps, which common in the early stages of vi-
Enkieesttee
Entaenthrmmena
Top-1 (c
78
Mizi cole
Paodel
76
Ghidiwevy
Pamer
70
0.75
1.00
1.25
1.50
1.75 2.00 2.25 2.50
Latency (ms)
Figure 1: Comparison of latency and accu- racy between our iFormer and other exist- ing methods on ImageNet-Ik. The latency is measured on an iPhone 13, Our iFormer is Pareto-optimal.

---

Published as a conference paper at ICLR 2025
sion networks. On the other hand, the multi-head mechanism requires reshaping operations, leading to increased memory usage.
Many research efforts are devoted combining the advantages of both CNNs and ViTs in de- signing lightweight networks while mitigating inefficient operations in mobile applications. Some studies (Zhang et al., 2023; Wang al., 2024; Ma et al., 2024) revisit the architectural designs of lightweight CNNs from a ViT perspective and incorporate key components that contribute to the performance of ViTs into CNNs. Although these pure lightweight CNNs show improved perfor- mance compared to previous mobile networks (Howard et al., 2017; Zhang et al., 2018; Sandler al., 2018), they still lag behind the powerful self-attention in ViTs. Another line of works (Mehta & Rastegari, 2021; Chen et al., 2022b; Li et al., 2023; Cai et al., 2023; Shaker et al., 2023; Vasu et al., 2023a; Qin et al., 2024) proposes innovative attention mechanisms to address the limitation of standard attention ( (Vaswani, 2017) and blend convolutions to achieve a better balance between latency and performance. These attention mechanisms either reduce the number of queries and keys (Shaker et al., 2023; Qin et al., 2024), limit the attention span (Wan et al., 2023), or adopt linear attention (Cai et al., 2023), which may compromise performance to some extent.
In this work, we present the iFormer, a herd of lightweight models that integrates the strengths of both CNNs and ViTs, achieving a state-of-thee ofrthe--rtt--ttt- balance between latency and accuracy. Specifi- cally, we employ a hierarchical architecture consisting of four stages. In the earlier, high-resolution stages, we utilize fast convolution to extract local representations. To construct the convolutional block, we start with "modern" ConvNeXt (Liu et al., 2022), which incorporates series of design decisions inspired by ViTs. Then we progressively "lighten" the ConvNeXt to create a streamlined lightweight network, optimizing it for real-time mobile latency on an iPhone 13, in contrast to the FLOPs and parameters used in prior works (Mehta & Rastegari, 2021; Chen et al., 2022b). This results in fast convolutional architecture with strong performance. To further enhance the dynamic properties and its ability to model long-range contexts, we incorporate self-attention in the later low-resolution stages. However, direct implementation of standard multi-head self-attention (MHA) brings notable memory overheads and slows down inference speed on mobile devices. We identify that the increased latency stems primarily from the reshaping operations in MHA. More analyses reveal that multiple attention heads behave similarly. Therefore, we propose simple yet effective single-head modulation self-attention (SHMA), which significantly minimizes memory costs while preserving strong performance. Fig. 4 provides illustration of SHMA. In detail, SHMA lears spatial context interactions through optimized self-attention. Concurrently, a parallel feature extrac- tion branch is employed to capture informative features. Finally, we fuse the outputs of these two branches to facilitate a more flexible and dynamic exchange of information, compensating for the slight performance degradation of the single-head attention when compared to MHA.
Benefiting from the fast local representation capacity of convolution and the efficient global model- ing proficiency of the proposed SHMA, iFormer outperforms existing pure lightweight CNNs and hybrid networks across multiple visual recognition tasks, including image classification, object de- tection, instance segmentation, and semantic segmentation. For instance, in the context of image classification as shown in Fig. 1, iFormer-M achieves a Top-1 accuracy of 80.4% with only 1.10 ms on an iPhone 13 without advanced training strategies such as knowledge distillation (Touvron et al., 2021a) or reparameterization (Ding et al., 2021). Notably, our model obtains a 0.5% improve- ment in Top-1 accuracy compared to the recent MNV4-Conv-M (Qin ct al., 2024), while being 1.4x faster than mistViT-SA12 (Vasu et al., 2023a) with similar accuracy. These results demonstrate the effectiveness of the proposed network in capturing both local and global feature representations.
2
RELATED WORK
2.1 EFFICIENT CONVOLUTIONAL NETWORKS
In the past 2010s, computer vision was dominated by CNNs, and so were efficient networks. The first remarkable breakthrough in mobile CNNs is MobileNets (Howard et al., 2017), which hatches the concept of decomposing standard convolution into depthwise and pointwise counterparts. Sub- sequently, MobileNetV2 (Sandler et al., 2018) introduces an inverted residual bottleneck block to push the state-of-th the-the-rtttt art for mobile models. Numerous studies have aimed to accelerate CNNs via various approaches, such as channel shuffle in ShuffleNet (Zhang et al., 2018: Ma et al., 2018) and cheap linear transformations in GhostNet (Han al., 2020). Meanwhile, Neural architecture search
2

---

Published as a conference paper at ICLR 2025
(NAS) has emerged as method for automating the design of neural networks, optimizing for per- formance under resource constraints. EfficientNet (Tan & Le, 2019), MobileNetV3 (Howard et al,, 2019), and FBNet (Wu et al., 2019) all achieve rather good performance. Besides, MobileOne (Vasu et al., 2023b) proposes to train a model using reparameterizable branches, which are merged during inference. Recently, following the revolution of ViT', several methods reexamine the design spaces and training strategies (Liu et al., 2024) for mobile CNNs. For instance, Rep (Wang et al., 2024) integrates efficient architectural designs from ViTs into MobileNetV3, outperforming exist- ing lightweight CNNs. Other approaches, such as FocalNet (Yang et al., 2022a), Conv2Former (Hou et al., 2024), and EfficientMod (Ma et al., 2024). fuse features from context modeling and feature projection branches, also known as modulation mechanism, to enhance the model with dynamic properties analogous to attention. However, pure CNNS remain inherently spatially localized and their reliance on stationary weights restricts their flexibility. Although modulation can partially mit- igate this limitation by enhancing dynamic capacity, they still exhibit deficiencies in building global interactions.
2.2 EFFICIENT VISION TRANSFORMERS
The success of Vision Transformer (Dosovitskiy et al., 2020) offers compelling demonstration of the potential to apply transformer to computer vision tasks. Following this, ViT and its numer- ous variants (Liu et al., 2021a; a; Dong et al., 2022; Li et al., 2022a) sweep across various scenarios. However, the quadratic complexity of self-attention behind ViTs poses significant challenges for efficiency. The following researches seek to boost ViT efficiency through efficient attention mecha- nisms (Wang et al., 2021; Zhu et al., 2023; Hatamizadeh et al., 2023), model compression (Liu et al., 202 Zheng et al., 2022), knowledge distillation (Hao et al., 2021), and token reduction (Rao et al., 2021: Bolya et al., 2022). Recent studies further introduce ViTs into mobile applications. One main- stream of work combines efficient convolution and ViT to create lightweight hybrid networks (Mehta & Rastegari, 2022; Vasu et al., 2023a). Mobile VIT (Mehta & Rastegari, 2021) directly integrates MobileNetv2 blocks and ViT blocks, while Mobile-Former (Chen et al., 2022b) features a paral- lel design of MobileNet and ViT with two-way bridge connecting the two. To further accelerate inference, some approaches replace the standard attention ( (Vaswani, 2017) with efficient variants within the hybrid networks. These include reducing the number of delegate tokens for computing attention (Pan et al., 2022), employing channel attention (Maaz et al., 2022). substituting projection in attention with efficient ghost modules (Ma et al., 2022), and utilizing linear attention mecha- nisms (Zhao et al., 2022). Besides manual designs, EfficientFormer (Li et al., 2022b; 2023) and MobileNetV4 (Qin al., 2024) search for efficient architectures in unified space encompassing both convolution operators and transformer operators. Another stream of work focuses efficient attention mechanisms and directly employs them throughout the entire network (Shaker et al., 2023; Cai et al., 2023). For example, CMT (Guo et al., 2022) takes advantage of depth-wise convolution to downsample key and value to reduce computation. GhostNetV2 (Tang et al., 2022) applies two fully connected layers along the horizontal and vertical directions to compute attention, decoupled ver- sion of MI Mixerr (Tolstikhin et al., 2021). Recently, SHViT observes computational redundancy in the multi- attention module and proposes to apply sing-head attention. In contrast these existing approaches, we introduce a novel efficient attention module without sacrificing informative interactions, thereby maintaining strong representational capacity. Regarding attention design, ours is bit similar to SHViT but is considerably superior as shown in Table 18 in the supplementary material. The key difference lies in the novel modulation attention. In addition, we explore efficient attention mechanisms in an on-device environment while SHViT focuses on general- purpose GPUs, fundamentally different hardware.
general-purpose
3
METHOD
We present the overall architecture of our iFormer in Fig. 4, which offers a Pareto-optimal accuracy- latency trade-off on mobile applications. Our exploration towards a streamlined lightweight network unfolds as follows: 1) establishing the baseline and measure metric in Sec. 3.1. 2) exploring accel- eration techniques consisting of macro and micro designs in Sec. 3.2. 3) injecting global attention in Sec. 3.3. Finally, we create
family of efficient hybrid vision transformers tailored for mobile applications in Sec. 3.3. A detailed trajectory illustrating the evolution from a general hierarchical CNN to a fast hybrid vision transformer is depicted in Fig. 2.
3

---

Published as a conference paper at ICLR 2025
Latencylms)
75
0.5
1,0
Average Cosine Simi antty
70
Baseline
74.90
1.07
65
conv stem
60
77:10
55
Lightening-
deeper
78.80
1:05
-00
stage ratio
79.10
0.99
kemel SZ.
79.40
1.00
Layer
SHA
79.80
1.12
Figure 3: The distribution of aver- age cosine similarity among multiple heads within the MHA mechanism. As the layer depth increases, the simi- larity goes higher.
1.15
Attention
rd. width
80.20
1.10
PE
80.40
1.10
iFormer
75
Table I: Latency comparison be- tween multi-head and single-head baseline.
Top-1 Acc.(%)
Figure 2: Illustration of the evolution from the Con- vNeXt baseline towards the lightweight iFormer. The orange bars are model accuracies and the light blue bars model latencies. We also include red latency outline for better visualization.
Models
Latency (ms) Top-1 1 Acc. (%)
MHA Baseline
1.40
79.9
SHA Baseline
1.12 (1.25x)
79.8
3.1 PREPARING CONVNEXT
Our goal is to create efficient multiscale network, where spatial dimensions of intermediate rep- resentations shrink as inference proceeds. In this hierarchical architecture, early network layers have larger spatial dimensions and fewer channels (e.g. 56x56x48), which renders them memory- bound. Highly optimized convolution is more appropriate for these layers. Guided by this principle, we choose pure convolutional network as our base architecture, specifically ConvNeXt (Liu al., 2022) which absorbed several key components from ViT' and competes favorably against ViTs. We gradually "lighten" the network achieve a more favorable balance between latency and aceuracy. For speed metric, we utilize on-device latency, measured on actual iPhone 13 and compiled by Core ML Tools (CoreML), rather than FLOPs and parameter counts in previous methods (Mchta & Rastegari, 2021; Chen et al., 2022b; Zhang et al., 2022), which are not well correlated with latency. Regarding performance, we follow the training recipe in ConvNeXt while removing the layer scale to align prior methods (Li et al., 2022b; Wang al., 2024) for fair comparison. Please refer to Sec. B in the supplementary material for more details. To initiate our study, we systematically scale down the ConvNeXt by reducing the number of blocks and the width. This results in a lightweight model with latency of 1.07 ms and a Top-1 accuracy of 74.9%, serving as our initial baseline.
3.2 LIGHTENING BASELINE
Seeing Better with Early Convolutions Following ViTs, ConvNeXt adopts an aggressive "patchify" strategy as the stem cell, specifically by splitting the input image into series of non- overlapping patches via a 4x4 non-overlapping convolutional layer. However, some studies (Xiao al., 2021; Chen al., 2022a) indicate that an early convolutional stem can increase optimiza- tion stability and facilitate faster model convergence. Moreover, compared to general models, lightweight models typically have fewer parameters and a reduced capacity. An aggressive non- overlapping layer may lead to the premature loss of rich information. Consequently, replace the overolellapinng "patchify" stem with stack overlapping convolutional layers, as shown in Fig. 4. This modification elevates the top-1 accuracy 76.7% 7% with neglectable increase in latency of 0.1 ms.
opt to
Normalization An obvious difference between ConvNeXt and previous CNNs the normaliza- tion layer. ConvNeXt utilizes Layer Normalization (LN) (Ba et al., 2016), commonly used in Natural Language Processing (NLP), whereas the latter uses Batch Normalization (BN) (loffe, 2015). Al- beit its superior performance, LN requires on-the-fly statisties calculation in inference along with
4

---

Published as a conference paper at ICLR 2025
Siage
Stage Cotteee
Siage
Stage Maz
Stage
Inpur
Stem
Block
Conv
Block
Bicck
Bluck
Arth
Hoad
BXG g B
BxCxX 16
16
3xd
Cony (33, s2 ?
ExHxxW
Corv (Kd, so:
CPE
Poo
Redluga
Transpooe
Lx
Mi.
Trampood
Roshato
Aut Adtivation
Reo Win
Corv (K3, s2 26,)
Project 4C
SHMA
Frasz
ACKNH*
Project
Project
Project
CKL
#
CXNWW
C.HSP
Conv Stem
Conv Block
Attn
SHIMA
Figure 4: Overview of iFormer architecture, detailed convolutional stem, block design, and SHMA. The hatched area in SHMA indicates extra memory-intensiee reshaping operations that are eliminated by SHMA. S()) denotes the softmax function. R is the ratio for reducing channels of query and key. It is set to 2 in iFormer. We omit BN following project or convolution for simplicity.
division and square root operations, leading inefficiency mobile hardware (Yang et al., 2022b). On the contrary, BN operates with fixed statistics during inference as an offline method and can be seamlessly fused with other linear operations, providing "free lunch". This significantly reduces computational demands and memory overheads on mobile devices. Therefore, we substitute LN with BN throughout the network and merge during inference. Additionally, we also substitute non-overlapping downsample layers with overlapping counterparts. These adjustments result in a reduction of overall latency to 0.77 ms while enhancing the Top-1 accuracy slightly to 77.10%.
Going Deeper There is considerable evidence indicating that increasing the depth of model can enhance its capacity and yield performance benefits (Touvron al., 2021b; Yang al., 2022a). Most lightweight models typically stack more blocks to boost performance within constrained resources, exemplified by the MobileNet series (Howard al., 2019; Qin et al., 2024). In this study, explore the potential of deepening ConvNeXt by increasing the number of blocks in each stage from (2,2,6,2) (3,3,9.3). This increase in depth leads substantial improvement, raising the accuracy from 77.1% to 78.8%, although causing a temporary increase latency to 1.05 ms.
Stage Ratio The stage ratio in ConNeXt is not optimized for lightweight models. A substan- tial number of depthwise convolutions in the early stages incurs significant memory transfer costs. Meanwhile, the presence of many blocks with channel expansion ratio of 4 in the Network (FFN) in the last stage, which already has a high channel dimension, imposes substantial computational demands. These factors lead sub-optimal allocation of computational resources. To address these issues, we propose reallocating more computational resources to the third stage while reducing memory access costs in the early stage. Specifically, the blocks in each stage ad- justed from (3,3,933 to (2,2,18,2). As expected, this achieves a better balance between latency and performance, with Top-1 accuracy increasing to 79.15 % while enjoying lower latency of 1.01ms.
Kernel Size Here we examine the effects of different ker- nel sizes in mobile settings and observe that utilizing larger kernel size introduces nearly no latency burden, as shown in Table 2. So we maintain the convolutional kernel size at 7x7 in each basic block, consistent with NeXv Furthermore, previous approaches use kernel size of 3x3 in the convolu- tional stem. This small receptive field may hinder feature rep- resentation during the early downsampling process. As pre- viously noted, the early layers are memory-bound, allowing for opportunities to employ compute- intensive operations (i.e,, dense convolution). Therefore, enlarge the kernel size of the dense
Table 2: Latency under different convolutional kernel sizes.
Kernel Size
Latency (ms)
3x3
1.00
7x7
1.01
5

---

Published as a conference paper at ICLR 2025
convolutional layer in the stem cell to 5x5. As illustrated in Fig. 2, this change has no impact on inference latency while enhancing Top-1 1 accuracy by 0.3%.
3.3 SINGLE-HEAD MODULATION ATTENTION
Single-Head vs. Multi-Head ViTs typically apply MHA, which projects the queries, keys, and values multiple times with different learnable linear projections and performs multiple attention functions simultaneously. In practice, the multi-head mechanism requires the reshaping of feature maps first, causing large memory access and transfer costs. This can seriously impact inference latency, especially on resource-constrainee mobile devices. To investigate this issue, we substitute the last half of the convolutional blocks in the third stage and all blocks in the last stage with standard VIT blocks, as depicted in Fig. 4. We refer to this hybrid network the MHA baseline. Next, we build another network by substituting the MHA with Single-Head self-Attention (SHA), referring to it as the SHA baseline. The comparison is shown in Table 1. The SHA baseline shows 1.25x acceleration over its MHA counterpart on the iPhone 13. This verifies that additional reshaping operations in MHA incur significant memory access costs, leading considerable decline in inference speed.
This naturally calls for optimizing MHA. Recent methods (Pan et al., 2022; Qin et al., 2024) primar- ily focus on downsampling the query or the key. which may hurt global attention capacity. Instead, aim to reduce the redundant reshaping of MHA while preserving all token-to-token interactions. Previous works (Michel et al., 2019; Yun & Ro, 2024) indicate that a single attention head can approach the performance of multiple heads in general plain transformer models, such as DeiT. To investigate this on the mobile application, we analyze the average cosine similarity of multiple heads within the same layer of the aforementioned MHA baseline, which is hierarchical lightweight net- work, and present our findings in Fig. 3. We clearly see that the average cosine similarity reaches 50% and even 75% in the final layer. Furthermore, the SHA baseline, as shown in Table 1, exhibits only negligible accuracy drop of 0.1 %. These suggest that SHA achieves more favorable balance between accuracy and latency, obtaining an accuracy of 79.8% with a latency of 1.12 ms.
Modulation Attention We further introduce a novel modulation attention to boost performance and strengthen flexibility in modeling, illustrated in Fig. 4. Formally, we start from the abstracted modulation mechanism (Ma et al., 2024), similar to the gate mechanism Shazeer (2020). Assume are given an input feature map RC RC where C, H, and W denote the channels, height, and width of the feature map. The modulated output can be written as follows:
X f(x @ ctx(x))
(1)
where f(.) denotes the feature mapping branch and ctx(-) is the context modeling branch. The output X. is the fused features from both branches via efficient element- wise multiplication. The key idea of our approach is to modulate the feature using SHA instead of relying on convolutional layers, as seen in previous works ( (Yang et al., 2022a; Ma al., 2024). Since SHA captures global interactions through self-attention, it excels in extracting rich contextual information and better controlling the flow of information. This process can be expressed as follows:
ctx(x) = X W VY)),
where Wº, WK WV are the project weights for query, key, and value, respectively. For simplicity, we omit the bias term. To minimize inference costs, we utilize a single projection layer in the feature mapping branch. To enhance expressivity and improve optimization stability, apply individual nonlinear activation functions to both branches, as follows:
X = o (WWMM x) @ a(cxx(x)),
where o is the sigmoid function and W denotes the feature projection weight. We also experiment with various activation functions for modulation in Sec. 5 and observe that the sigmoid works rather well. Finally, the output from the modulation attention is projected in a manner as standard attention.
Equipped with Single-Head Head Modulation Attention (SHMA), our model improves the accuracy to 80.4% with an intermediate latency of 1.15 ms. This performance notably surpasses that of the recent MobileNetV4, which achieves accuracy of 79.9%.

---

Published as a conference paper at ICLR 2025
Reducing Width Until now, have developed a lightweight network that performs pretty well, but at bit slow speed. To push the trade-off toward the ofte-of--hhearr we revise the width con- figuration in the SHMA. The modulation mechanism enriches the output by enabling more dynamic modeling in both spatial and channel dimensions, making it possible to use a weaker SHA and FFN. In light of this, we reduce the head dimension in the SHMA (i.e., WQ, W to small factor of the feature dimension, further details can be found in Table 15 in the supplementary material. Simulta- neously, we shrink the expansion ratio in FFN following SHMA from 3. This process obtains lower latency of 1.10 ms, although slight drop of 0.2% in accuracy.
Positional Embedding Last but not least, positional information plays crucial role in self- attention as it regards input as a set of tokens. Adding positional embedding will help the attention learn permutation-variat 2021) that are dynamically generated and conditioned on the local neighborhood of the input to- kens, as illustrated in Fig. 4. The integration of CPE further enhances our model's performance, achieving Top-1 accuracy of 80.4% with only 1.10 ms, establishing a state-of-the-aan trade-off.
features. We apply conditional positional encodings (CPE) (Chu et al.,
The result of these modifications an extremely fast and efficient hybrid network. which we denote iFormer. The overall architecture is depicted inFig. 4. It integrates fast local convolu- tional layers in the early stages that operate on higher resolution and global SHMA later stages which processes lower resolution. Besides, we create series of iFormer models tailored to various hardware resource constraints. For detailed architectural hyperparameters of these model variants, please refer to Table 15 in the supplementary material.
iFormer
4 EXPERIMENTS
4.1 IMAGE CLASSIFICATION
Settings. We first evaluate our models on classification on ImageNet-IK (Deng al., 2009). To ensure fair comparison with prior studies, we follow the previous training recipe (Touvron et al., 2021a; Liu al., 2022) and train all models for 300 epochs with standard image size of 224x224. Please refer to Sec. in the supplementary material for details. Besides Top-1 validation accu- racy, we also report the latency measured on iPhone 13 with models compiled by Core ML Tools (CoreML) under a batch size of I, as done in (Li et al., 2023; Wang et al., 2024; Vasu et al., 2023b). It's worth highlighting that we do not apply any advanced strategies such distillation (Li et al., 2023) and reparameterization (Ding et al., 2021).
Table 3 summarizes a com- parison between our iFormer and state-of-the-arr lightweight models, organized by latency. iFormer demonstrates Pareto- optimal trade-off between accu- racy and latency. For exam- ple, iFormer-M obtains 80.4% top-1 accuracy with a latency of only 1.1 ms, surpassing re- cent MobileNeerV4CCon-M and VepViT-MI by 0.5% and 1.0%, respectively. This is noteworthy considering that MobileNetV4 requires longer training sched- ule (500 vs. 300) and takes a larger input resolution (256 vs. 224). When compared to other recent models using reparameterization, including FastViT-T12, G3..tNe and MobileOne-S3. iFormer-M achieves superior accuracy while maintaining lower latency. Moreover, iFormer outperforms various hybrid networks. Thanks to the efficient SHMA, iFormer-L achieves more outstanding performance than other attention variants, such as multi-query attention in MNV4-Hybrid-M, additive attention in SwiftFormer-L. and linear atten- tion in
Table 4: Results with distillation on ImageNet-IK. "iniccttss the model is trained with strong training strategy (i.e., repa- rameterization).
Model
Latency (ms)
Reso.
Epochs
Top: I (%)
EffhiienFoomerV22. 2-S1 (2023)
1.02
224
300
79.0
EffiiienFormee22-511 (2023)
1.02
224
450
79.7
MoblleVidv2-$((2244)
1.24
224
300
79.8
FasIViT-T12* (2023a)
1.12
256
300
80.3
RepViT-M1.1 (2024)
1.04
224
300
80.7
iFormer-M
1.10
224
300
$1.1
(2024)
1.48
224
300
80.2
EfiivienFormee222-22
1.60
81.6
MohileViGr2-MM2224
1.70
224
300
81.7
FA1 (2023a)
81.9
EfficennFormerV2-55/ (2023)
1.60
224
450
82.0
RepViT M1.55 (2024)
1.54
224
300
82.3
iFormer-I.
1.60
224
300
82.7
7

---

Published as a conference paper at ICLR 2025
Table 3: Classification results on ImageNet-1K. indicates models that are trained with a variety of advanced training strategies including complex reparameterization, distillation, optimizer, and so on. We provide a more comprehensive comparison in Sec. in the supplementary material.
Lateney (ms)
Model
Params (M)
GMACs
Reso.
Epochs
Top-1 (pp-(gp (%)
MobileNerV2 1.0x (2018)
3.4
0.30
0.73
500
72.0
MobileNev3--Lrge 0.75x (2019)
4.0
0.16
0.67
224
600
73.3
MNV4-Comv-S (2024)
3.8
0.20
0.60
224
500
73.8
iFormer-T
2.9
0.53
0.60
224
300
74.1
MobileNerV 1.4x (2018)
6.9
0.59
1.02
500
74.7
MobileNetV3-Large 1.0x (2019)
5,4
0.22
0.76
224
600
75.2
SwiftFormer-Ke (2023)
3.5
0.60
0.95
224
300
75.7
SBCFormerXXS (2024)
5.6
0.70
0.79
224
300
75.8
GhostNetV3 1.0x (2024)
6.1
0.17
0.99
224
600
77.1
MobileOme-S2 (2023h)
7.8
0.92
224
300
77.4
RepViT-MM1.0 (2024)
6.8
1.10
0.85
224
300
78.6
FFormerSS
6.5
1.09
0.85
224
300
78.8
EfficientMod--ee (2024)
4.7
0.60
1.29
224
300
76.0
SBCFormer-S (2024)
8.5
224
300
MobilcOne-S3 (2023h)
10.1
1.16
224
300
78.1
SwiftFormmr-S (2023)
6.1
1,00
1.12
224
300
78.5
GhostNerV3 1.3xl (2024)
8.9
0.27
1.24
224
600
FastViT-T12 (2023a)
6.8
1.40
1.12
256
300
79.1
RepViT-M1.1 (2024)
8.2
1.30
1.04
224
300
79.4
MNV4-Conv-M (2024)
9.2
1,00
1.08
256
500
79.9
iFormer-M
8.9
1.64
1.10
224
300
80.4
Mobile-Former-294M (2022h)
0.29
224
450
MobileViT-S (2021)
5.6
2.00
3.55
256
300
78.4
MobileOne-S4 (2023h)
14.8
2.98
1.74
224
300
79.4
SBCFommer-B (2024)
13.8
1.60
1.44
224
300
80.0
GhostNerV3 1.6x (2024)
12.3
0.40
1,49
224
600
80.4
EMcciem\ViT-B112288 (2023)
9.1
0.86
3.87
288
450
80.4
FasIViT-SAI2 (2023a)
10.9
1.90
1.50
256
300
80.6
MNV4-Hybrid-M (2024)
10.5
1.20
1.75
256
500
80.7
SwiftFormer-LI (2023)
12.1
1.60
1.60
224
300
80,9
EfficientMod-s (2024)
12.9
1.40
2.57
224
300
81.0
RepViT-MI.5 (2024)
2.30
1.54
300
81.2
FFomeer-
14.7
2.63
1.60
224
300
conducted rigorously fair training as the previ- methods above. Recently, some works report enhanced performance leveraging more advanced training strategies. We investigate whether these training recipes can also improve iFormer. Fol- lowing previous works (Li et al., 2023; Wang al., 2024), employ the RegNetY-16GF (Ra- dosavovic et al., 2020) model with a top-1 accuracy of 82.9% as the teacher model for distilla- tion. Our findings reveal that iFormer improves obviously over its counterpart without distillation. For example, iFormer-L shows 1.0% increase under the same latency. iFormer also outperforms EllicientFormerVes V2-S2, despite the latter being trained with 1.5 longer schedule.
Results with distillation on ImageNet-IK.
4.2 OBJECT DETECTION AND INSTANCE SEGMENTATION
To validate the effectiveness of iFormer on downstream tasks, we train Mask R-CNN (He et al., 2017) with iFormer as the backbone for 12 epochs (1x), using the MMDetection toolkit (Chen et al., 2019). We also report backbone latency measured at a resolution of 512x512 on an iPhone The results are presented in Table 5. comparison lightweight models, iFormer-M sur- passes FastViT-SA 12 by +1 1.99%/2.00% in Apbox /Apmask while running 1.32x faster. iFormer-L also obtains +0. +0.1% in Apbox /Apmask than EfficientMod-S, which utilizes a convolutional mod- ulation mechanism learn dynamics similar to self-attention. Notably, EfficientMod-S operates 3.7x slower when processing mechanism is more suitable for mobile networks. Meanwhile, when compared to general networks that are not optimized for mobile applications, iFormer demonstrates significant advantages. For in- stance, iFormer-L exceeds the performance of Xt-- with improvements of +1.2 +1.2% 1./ 1.4% in
input, underscoring that the proposed novel attention
8

---

Published as a conference paper at ICLR 2025
Table 5: Object detection & instance segmentation results on MS COCO 2017 using Mask R- CNN. Semantic segmentation results ADE20K using the Semantic FPN framework. We mea- sure all backbone latencies with image crops of 512x512 on iPhone 13 by Core ML Tools. Failed indicated that the model runs too long report latency by the Core MI..
Puram (M)
Latency (ms)
Object Detection
Instance Segmentation
Semantic
Backbone
Ap
AP
AP
Ap
AF
AP1S
mloU
Etficien-Net-OO (2019)
4.55
31.9
34,5
29.4
47.9
ResNet18 (2016)
11.7
2.85
34.0
54.0
36.7
31.2
$1.0
32.7
32.9
PoolFormer-S11 (2022)
11.9
5.70
37.3
59.0
40.1
34.6
55.8
36.9
37.2
EfficienFForerr.. (2022h)
37.9
57.3
FastViT-SA (2025a)
10.9
5.27
38.9
60.5
42.2
35.9
57.6
38.1
38.0
Rep (2024)
8.2
3.18
39.8
43.5
37.2
58.8
40.1
40.6
iFormer-M
4,00
40.8
62.5
44.8
37.9
59.7
40.7
42.4
ResNet50 (2016)
25.5
7.20
38.0
58.6
41.4
34,4
55.1
36.7
36.7
PoolFormer-S22 (2022)
21.4
10.0
40.1
62.2
43.4
37.0
59.1
39.6
40.3
Tex-NeXt-T 2022)
EMiiienFormer.// (2022h)
31.3
8.40
41.4
63.9
44.7
38.1
61.0
40.4
43.5
RepViT-M1.5 (2024)
5,00
41.6
45.3
38.6
60.5
41.5
43.6
PVTv2-BI (2022)
14.0
27.00
41.8
64.3
45.9
38.8
61.2
41.6
42.5
FastVTT-SA24 (2025a)
20.6
8.97
42.0
63.5
45.8
38.0
60.5
40.5
41.0
EllicientMod-S (2024)
Swin-T (2021a)
28.3
Failed
42.2
64.4
46.2
39.1
61.6
42.0
41.5
iFormer-L I.
14.7
6.60
42.2
64.2
46.0
39.1
61.4
41.9
44.5
A pbox / /A IApaask while requiring fewer parameters and only 50% mobile latency, suggesting iFormer's efficient design in feature extraction and strong potential for mobile applications.
4.3 SEMANTIC SEGMENTATION
We conduct experiments on ADE20K (Zhou et al., 2017) using the Semantic FPN (Kirillov al., 2019), based on the MMSegmentation toolkit (Contributors, 2020). Thanks to its efficient attention design, iFormer outperforms all competing methods in mloU with similar and much lower latency. For example, iFormer-L surpasses FastViT-SA24 by +3.5% in mloU with 1.36x faster inference speed. addition, iFormer-M demonstrates superior mloU compared to general networks, which typically exhibit substantially greater latency when processing higher-resolution inputs devices. Although PVTv2-B utilizes downsampled attention, it still requires 27 ms for latency. Similarly, Swin-T involves intensive operations in window partitioning, making it less suitable for mobile applications. Running at 6.6 ms, iFormer-L achieves +2.0% better mloU than PVTv2-B1 and +3.0 +3.0% better than Swin-T. These results suggest that the proposed attention mechanism offers significant benefits for tasks requiring the perception of fine-grained details.
mobile
5 ABLATION STUDIES
Activation Function Here we explore whether activation func- tion without an upper bound can enhance the SHMA by allow- ing neurons to express arbitrarily large values. We compare the widely used Sigmoid Linear Unit (SiLU) (Shazeer, 2020) with the sigmoid function and present the results in Table 6. Directly re- placing the activation function in SHMA with SiLU will encounter diverging loss during training. The underlying cause is primar- ily attributed to the element-wing multiplication of the unbounded context branch. To address this, we replace Post-BN in SHMA with Pre-LN, as LN adaptively normalizes each token feature. The modified model experiences slight decrease in accuracy but incurs additional 0.07 ms latency, primarily brought by LN. The results suggest that the sigmoid function not only mitigates training instability but also facilitates better convergence.
Table 6: Activation function comparison in SHMA. Post- BN indicates that BN is applied after projection. Pre-LN means that LN is implemented before the projection, as in standard MHA (Vaswani, 2017).
SHIMA Setting
Params (M) GMACs Latency (ms)
Top-1 Acc. (%)
SiLU Post-BN
8.9
1.60
1.10ms
Diverged
SiLU Pre-LN
1.17ms
80.3
Sigmoid Post-BN
8.9
1.60
1.10ms
80.4

---

Published as a conference paper at ICLR 2025
Choice of Conv v.s. VIT Blcoks In Section 3.3, replace the convolutional blocks in Stages and with the proposed SHMA block. We provide further ablation studies on the choice of ratio for the ViT blocks. Specifically, We choose the model after enlarging the kernel size a starting point, then we progressively replace the convolutional blocks in Stages 3 and 4. We do not modify Stages 1 and 2 as their larger spatial dimensions would considerably increase the memory requirements for the self-attention mechanism.
Table 7: Different ratio of ViT Block.
Ratio Setting
Params (M) GMACss Lateney (ms) Top-1 Acc. (%)
Baseline
9.4M
1760M
1.0ms
79.4
Replacing 22% Conv Blocks Stage as SHA
9.1M
1724M
1.02ms
79.5
Replacing 22% Conv Blocks in Stage as SHMA
9.2M
1739M
1.04ms
79.6
Replacing 50% Conv Blocks in Stage as SHA
8.8M
1689M
1.04ms
79.5
Roplacing 50% Conv Blocks in Stage 3 as SHMA
8.9M
1712M
1.07ms
79.8
Replacing 78% Conv Blocks in Stage as SHA
8.3M
1635M
1.12ms
79.3
Replacing 78% Conv Blocks Stage as SHMA
8.5M
1685M
1.17ms
79.6
Replacing 100% Conv Blacks Stuge 3 SHA
7.9M
1599M
1.17ms
78.1
Replacing 100% Conv Blocks in Stage 3 as SHMA
8.3M
1665M
1,25ms
79.0
Replacing 100% Conv Blocks Stage 3 as SHMA and 100% in Stage 4
10.0M
1792M
1.15ms
80.4
We present our findings in Table 7. Given that Stage 4 contains only two blocks, we do not conduct further splitting for the ratio. As illustrated Table 7, although the ViT block has lower FLOPs, it still incurs increased runtime. Substituting all the convolutional blocks in Stage 3 results in the worst performance and the highest latency. Instead, by replacing half of the convolutional blocks in the third stage and all blocks in the final stage, achieving a favorable trade-off between accuracy and latency.
better integrate these two operators, thus
Scaling to Larger Model Although iFormer is designed for mobile-device applications, the com- bination of fast local representation capacity of convolution and the efficient global modeling profi- ciency of the proposed SHMA enables its scalability for broader range of applications. To demon- strate the scalability of iFormer, we developed a larger model named iFormer-H with 99M parame- ters and trained it for 300 epochs following the same strategy outlined Section B. It is important to note that we add drop path and layer scale, which are commonly used in the training of larger models (Liu et al., 2022; Tu al., 2022; Shi, 2024).
We summarize the results in Table 8. A highlight from the results is that iFormer is not specifically designed or trained for this scale. De- spite this, iFormer-H outper- forms ConvNeXt, achieving a 1.0% increase in accuracy while maintaining similar number of FLOPs. Additionally, it demonstrates comparable performance to TransNeXt-Base, despite utilizing fewer FLOPS. These findings indicate the potential for broader applications of iFormer. We plan to explore larger models suitable for mobile devices in future work. Further ablation studies can be found in Sec. the supplementary material.
Table 8: Scaling to the larger model with 99M parameters.
Model
Params (M) GMACs (G) Top-1 Acc. (%)
ConyNeXt-Bass (2022)
89
15.4
83.8
TransNeXt- -Bae (2024)
90
18.4
84.8
iFormer-H
99
15.5
84.8
MaxViT-Base (2022)
120
24.0
84.9
6 CONCLUSION
This work proposes iFormer, which integrates highly optimized convolutional operations for the early layers alongside a novel and efficient single-head modulation attention for the later layers. iFormer achieves SOTA Pareto-front in terms of Top-1 accuracy and mobile latency. We also validate the effectiveness of iFormer on downstream dense prediction tasks, including COCO object detec- tion, instance segmentation, and ADE20K semantic segmentation. These inspiring results highlight the potential for mobile applications. We hope iFormer can facilitate the application of artificial intelligence more mobile devices. In future work, we will seek to alleviate inference bottlenecks sociated with high-resolution images. Meanwhile, we plan to optimize iFormer for more hardware platforms, such Android devices and NVIDIA Jetson Nano.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
William Avery, Mustafa Munir, and Radu Marculescu. Scaling graph convolutions for mobile vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 5857-5865, 2024.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv: 1xXiv:607.000 (600.066450 2016.
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. arXiv preprint arXiv:2210.09461, 2022.
Han Cai, Junyan Li, Muyan Hu, Chuang Gan, and Song Han. Efficientvit: Lightweight multi-scale attention for high-resolution dense prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17302-17313, 2023. 2023.
Kai Chen, Jiaqi Wang. Jiangmiao Pang. Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv: 00005, 2019.
Qiang Chen, Qiman Wu, Jian Wang, Qinghao Hu, Tao Hu, Errui Ding, Jian Cheng, and Jingdong Wang. Mixformer: Mixing features across windows and dimensions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5249-5259, 2022a.
Yinpeng Chen, Xiyang Dai, Dongdong Chen, Mengchen Liu, Xiaoyi Dong, Lu Yuan, and Zicheng
Liu. Mobile-former: Bridging mobilenet and transformer. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 5270-5279, 2022b.
Xiangxiang Chu, Zhi Tian, Bo Zhang, Xinlong Wang, and Chunhua Shen. Conditional positional
encodings for vision transformers. arXiv preprint arXiv:2102, 10882, 2021.
MMSegmentation Contributors. MMSegmentation: Openmmlab semantic segmentation toolbox and benehmark. https: / com/ op///mpennmmmmmmm al/ www.aaaaaemmeeetttaiiioooo 2020.
CoreML. In https://gituub.comaapppe//oooe/e
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Xiaohan Ding. Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun. Repvgg: Making vgg-style convnets great again. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp. 13733-13742, 2021.
Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Lu Yuan, Dong Chen, and Baining Guo. Cswin transformer: A general vision transformer backbone with cross-shaped windows. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- fion, pp. 12124-12134, 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 1929, 2020.
Jianyuan Guo, Kai Han, Han Wu, Yehui Tang, Xinghao Chen, Yunhe Wang, and Chang Xu. Cmt: Convolutional neural networks meet vision transformers. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 12175-12185, 2022.
Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In Proceedings of the IEFF/CVE conference on computer vision and pattern recognition, pp. 1580-1589, 2020.
Zhiwei Hao, Jianyuan Guo, Ding Jia, Kai Han, Yehui Tang, Chao Zhang, Han Hu, and Yunhe Wang. Learning efficient vision transformers via fine- iin-igrined manifold distillation. arXiv preprint 17.1v:2107.01378. 2021.
11

---

Published as a conference paper at ICLR 2025
Ali Hatamizadeh, Greg Heinrich, Hongxu Yin, Andrew Tao, Jose M Alvarez, Jan Kautz, and Pavlo Molchanov. Fastervit: Fast vision transformers with hierarchical attention. arXiv preprint arXiv:2306.06 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and paitern recognition, pp- 770-778, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-enn. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969, 2017.
Qibin Hou, Cheng-Ze Lu, Ming-Ming Cheng, and Jiashi Feng. Conv2former: A simple transformer- style convnet for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 2024.
Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al. Searching for mobilenetv3. In Pro- ceedings of the IEEE/CVF international conference computer vision, pp. 1314-1324, 2019.
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.0466,, 2017.
Sergey loffe. Batch normalization: Accelerating deep network training by reducing internal covari- ate shift. arXiv preprini arXiv: 2015.
Alexander Kirillov, Ross Girshick, Kaiming He, and Piotr Dollár. Panoptic feature pyramid net- works. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6399-6408, 2019.
Alex Krizhevsky, Ily Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Advances neural information processing systems, 25, 2012.
Yanghao Li, Chao- Wu, Haoqi Fan, Karttikeya Mangalam, Bo Xiong. Jitendra Malik, and Christoph Feichtenhofer. Mvitv2: Improved multiscale vision transformers for classification and detection. In Proceedings of the IEEE/CVE conference on computer vision and pattern recogni- tion, pp. 4804-4814, 2022a.
Yanyu Li, Geng Yuan, Yang Wen, Ju Hu, Georgios Evangelidis, Sergey Tulyakov, Yanzhi Wang, and Jian Ren. Efficientformer: Vision transformers at mobilenet speed. Advances in Neural Information Processing Systems, 15::129341123 12949, 2022b.
Yanyu Li, Ju Hu, Yang Wen, Georgios Evangelidis, Kamyar Salahi, Yanzhi Wang, Sergey Tulyakov, and Jian Ren. Rethinking vision transformers for mobilenet size and speed. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 16889-16900, 2023.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021a.
Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. Post-training quanti- zation for vision transformer. Advances in Neural Information Processing Systems, 34:28092-
28103, 2021b.
Zhenhua Liu, Zhiwei Hao, Kai Han, Yehui Tang, and Yunhe Wang. Ghostnetv3: Exploring the training strategies for compact models. arXiv preprint arXiv:2404.11202. 2024.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11976-11986 2022.
Xiangyong Lu, Masanori Suganuma, and Takayuki Okatani. Sbeformer: Lightweight network pable of full-size imagenet classification at fps on single board computers. In Proceedings of the IEFE/CVF Winter Conference on Applications of Computer Vision, pp. 1123-1133, 2024.
12

---

Published as a conference paper at ICLR 2025
Hailong Ma, Xin Xia, Xing Wang, Xuefeng Xiao, Jiashi Li, and Min Zheng. Mocovit: Mobile convolutional vision transformer. arXiv preprint arXiv:2205. 12635, 2022.
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelines for efficient enn architecture design. In Proceedings of the European conference on computer vision (ECCV), pp. 116-131, 2018.
Xu Ma, Xiyang Dai, Jianwei Yang, Bin Xiao, Yinpeng Chen, Yun Fu, and Lu Yuan. Efficient modulation for vision networks. arXiv preprint arXiv:2403. 19969333..9963, 2024.
Muhammad Maaz, Abdelrahman Shaker, Hisham Cholakkal, Salman Khan, Syed Waqas Zamir, Rao Muhammad Anwer, and Fahad Shahbaz Khan. Edgenext: efficiently amalgamated cnn- transformer architecture for mobile vision applications. In European conference on computer vision, pp. 3-20. Springer, 2022.
Sachin Mehta and Mohammad Rastegari. Mobilevit: light-weight, general-purpose, and mobile- friendly vision transformer. arXiv preprint arXiv:21 2021.
Sachin Mehta and Mohammad Rastegari. Separable self-attention for mobile vision transformers. arXiv preprint arXiv:2206 220.00280,0 2022.
Paul Michel, Omer Levy, and Graham Neubig. Are sixteen heads really better than one? Advances in nearal information processing systems, 32, 2019.
Moritz Nottebaum, Matteo Dunnhofer, and Christian Micheloni. Lowformer: Hardware efficient design for convolutional transformer backbones. arXiv preprint arXiv:2409.03660, 2024.
Junting Pan, Adrian Bulat, Fuwen Tan, Xiatian Zhu, Lukasz Dudziak, Hongsheng Li, Georgios Tzimiropoulos, and Brais Martinez. Edgevits: Competing light-weight enns mobile devices with vision transformers. In European Conference on Computer Vision, pp. 294-311. Springer, 2022.
Danfeng Qin, Chas Leichner, Manolis Delakis, Marco Fornoni, Shixin Luo, Fan Yang. Weijun Wang, Colby Banbury, Chengxi Ye, Berkin Akin, et al. Mobilenetv4-universsa models for the mobile ecosystem. arXiv preprint arXiv:2404. 10518, 2024.
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. Designing network design spaces. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 10428-10436, 2020.
Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dynamicvit: Efficient vision transformers with dynamic token sparsification. Advances in neural information processing systems, 34: 13997-13949, 2021.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Chinh Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018
Abdelrahman Shaker, Muhammad Maaz, Hanoona Rasheed, Salman Khan, Ming Hiung Yang, and Fahad Shahbaz Khan. Swiftformer: Efficient additive attention for transformer baseed real real-time mobile vision applications. In Proceedings of the IEEE/CVF International Conference Com- puter Vision, pp. 17425-17436, 2023.
Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202. 2020.
Dai Shi. Transnext: Robust foveal visual perception for vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 17773-17783 2024.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural net- works. In International conference on machine learning, pp. 6105-6114 14. PMLR, 2019.
13

---

Published as a conference paper at ICLR 2025
Yehui Tang, Kai Han, Jianyuan Guo, Chang Xu, Chao Xu, and Yunhe Wang. Ghostnetv2: Enhance cheap operation with long-range attention. Advances in Neural Information Processing Systems, 15:9969-9982, 2022.
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mip-mixer: An all-mip architecture for vision. Advances in neural information processing systems, 34:24261- 24272, 2021.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pp. 10347-10357. PMLR, 2021a.
Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve. and Hervé Jégou. Going deeper with image transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 32 42, 2021b.
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. Maxvit: Multi-axis vision transformer. In European conference computer vision, pp. 459- 479. Springer, 2022.
Pavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan. Fastvit: A fast hybrid vision transformer using structural reparameterization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5785-5795, 2023a.
Pavan Kumar Anasosalu Vasu, James Gabriel, Jeff Zhu, Oncel Tuzel, and Anurag Ranjan. Mo- bileone: An improved millisecond mobile backbone. In Proceedings of the IEEE/CVF ference on computer vision and pattern recognition, pp. 7907-7917, 2023b.
Ashish Vaswani. Attention is all you need. arXiv preprint arXiv: arXiv: 1706.0366... 2017.
Qiang Wan, Zilong Huang, Jiachen Lu, Gang Yu, and Li Zhang. Seaformer: Squeeze-enhanced axial transformer for mobile semantic segmentation. arXiv preprint arX1v:2301.13156, 2023.
Ao Wang, Hui Chen, Zijia Lin, Jungong Han, and Guiguang Ding. Repvit: Revisiting mobile cnn from vit perspective. In Proceedingy of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15909-15922 2024.
Wenhai Wang. Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference computer vision, pp- 568-578. 2021.
Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song. Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyt v2: Improved baselines with pyramid vision transformer. Computational Visual Media, 8(3):415 424, 2022.
Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE/CVF conference on computer vision and pallern recognition, pp. 10734-10742, 2019.
Tete Xiao, Mannat Singh, Eric Mintun, Trevor Darrell, Piotr Dollár, and Ross Girshick. Early convolutions help transformers see better. Advances in neural information processing systems, 34:30392-30400, 2021.
Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Seg- former: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34:12077-12090, 2021.
Jianwei Yang, Chunyuan Li, Xiyang Dai, and Jianfeng Gao. Focal modulation networks. Advances in Neural Information Processing Systems, 35:4203-4217, 2022a.
14

---

Published as a conference paper at ICLR 2025
Qiming Yang, Kai Zhang, Chaoxiang Lan, Zhi Yang, Zheyang Li, Wenming Tan, Jun Xiao, and Shiliang Pu. Unified normalization for accelerating and stabilizing transformers. In Proceedings of the 30th ACM International Conference Multimedia, pp. 4445-4455, 2022b.
Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, and Shuicheng Yan. Metaformer is actually what you need for vision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10819-10829, 2022.
Seokju Yun and Youngmin Ro. Shvit: Single-head vision transformer with memory efficient maero design. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recogni- tion, pp. 9756-5767, 2024.
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp- 12104-12113, 2022.
Haokui Zhang, Wenze Hu, and Xiaoyu Wang. Edgeformer: Improving light-weight convnets by learning from vision transformers. arXiv preprint arXiv:2203.0399 039552, 2022.
Jiangning Zhang, Xiangtai Li, Jian Li, Liang Liu, Zhucun Xue, Boshen Zhang, Zhengkai Jiang, Tianxin Huang, Yabiao Wang, and Chengjie Wang. Rethinking mobile block for efficient attention-based models. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1389-1400 IEEE Computer Society, 2023.
Tianfang Zhang, Lei Li, Yang Zhou, Wentao Liu, Chen Qian, and Xiangyang Ji. Cas-vit: Convolu- tional additive self-attention vision transformers for efficient mobile applications. arXiv preprint arXiv:2408. 03703, 2024.
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6848-6856, 2018.
Mingshu Zhao, Yi Luo, and Yong Ouyang. Repnext: A fast multi- eon using structural repa- rameterization. arXiv preprint arXiv:2406. 16004, 2024.
Youpeng Zhao, Huadong Tang, Yingying Jiang, Qiang Wu, al. Lightweight vision transformer with cross feature attention. arXiv preprint arXiv:2207.07228. 2022.
Chuanyang Zheng, Kai Zhang, Zhi Yang, Wenming Tan, Jun Xiao, Ye Ren, Shiliang Pu, et al. Savit: Structure-aware vision transformer pruning via collaborative optimization. Advances in Neural Information Processing Systems, 35:9010-9023, 2022.
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 633-641, 2017.
Lei Zhu, Xinjiang Wang, Zhanghan Ke, Wayne Zhang, and Rynson WH Lau. Biformer: Vision transformer with bi-le routing attention. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pp.
2023.
15


Reviews:
[{"review_id": "3oG64OOpSU", "reviewer": "Reviewer_wKqn", "summary": "This paper designs iFormer, a new family of efficient mobile vision networks combining ConvNet and Transformers. The iFormer evolves from ConvNeXt with a series of efficiency designs. \n\nSingle-Head Modulated Attention(SHMA) is proposed as substitutional Transformer blocks to replace part of the Conv blocks in later stages of the enhanced ConvNeXt. SHMA replaces multi-head attention with single-head attention to improve efficiency and introduces a modulation mechanism to boost performance. \n\nThe resulting iFormer series achieves the best performance compared with state-of-the-art mobile-level models on different downstream tasks with lower latency.", "questions": "1. What is the motivation and justification for the necessity of the design that replaces half of the third stage and full last stage conv blocks with transformer blocks? Please refer to Weaknesses 1. \n\n2. I wonder what is the performance and latency of MHA as the missing step between 'kernel sz.' and 'SHA' in Figure 2.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is well-organized and easy to follow. Detailed design specifications and comprehensive experiments enhanced the integrity of the article and demonstrated its contributions.\n\nThe main contribution, SHMA, provides a new approach to designing efficient attention and Transformer blocks. The resulting iFormer series outperforms sota baseline mobile networks with stronger performance and lower latency.", "weaknesses": "W1: \n\nThe motivation and necessity of substituting half of the conv blocks at the third stage and all blocks at the last stage into Transformer blocks in ConvNeXt are still not very clear. From Figure 2, changing the conv blocks into SHA blocks gains a 0.4% improvement in performance but is also 0.12 ms (about 10%) slower. I'd like to know further explanation for this design and ablation studies on the choice of stages or different ratios of Conv versus Transformer blocks if possible. \n\n\nW2:\n\nAccording to the citation of SHViT in this paper, I suppose the SHA refers to the Single-head self-Attention in SHViT design. But in Figure 4, full channels of input (CxHxW) are projected to Q/K/V (CxL) which does not align with the design of SHA in SHViT but looks like the traditional definition of Single-head attention that performs a self-attention on all channels of input using a single head. \n\nConsidering there are limited words about the details of SHA in this paper, I would expect further specification of which SHA is used in iFormer and comply with the pipeline figure accordingly.  \n\nW3:\n\nIn this paper, the additional reshaping operations in MHA are considered as the reason for the slower inference speed compared with SHA. But multiple factors have an impact on the runtime speed difference and there's no evidence to support the extra runtime only or mainly comes from extra reshapings. \n\nFirst, depending on the code implementation, replacing MHA with Single-head self-Attention may remove the reshaping operation in self-attention, but also introduce additional split and concat operations. And generally, split and concat operations cost more memory and are slower than reshape. \n\nSecondly, SHA applies self-attention on fewer channels, which largely reduces the computational cost and speeds up runtime. \n\nTherefore I suggest the authors conduct an ablation study or provide empirical evidence to isolate the impact of reshaping operations versus other factors like split/concat operation and reduced self-attention channels on the inference speed. This would help clarify the main factors contributing to SHA's efficiency and provide a more comprehensive understanding of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7EnCO1h5f4", "reviewer": "Reviewer_XLkf", "summary": "This paper introduces a new family of mobile hybrid vision networks. By integrating the rapid local representation capability of convolution with the efficient global modeling ability of self-attention, the proposed architecture, iFormer, achieves significant performance in classification and several downstream tasks, while maintaining low latency on mobile devices for high-resolution inputs.", "questions": "please refer to weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The study of model architecture could inspire further exploration in designing more efficient architectures.\n2. The paper is well-organized and easy to follow.", "weaknesses": "1. In Table 1, iFormer-S achieves the same latency as RepViT-M1.0 with slightly fewer parameters, yet in larger variants, iFormer achieves lower latency with substantially more parameters compared to RepViT. What is the reason for this difference?\n2. Some studies are not included in the comparison or the related wotk section, such as [1, 2].\n\n[1] Cmt: Convolutional neural networks meet vision transformers.\n\n[2] Learning efficient vision transformers via fine-grained manifold distillation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CjlXIoXEWX", "reviewer": "Reviewer_waKF", "summary": "This paper presents a mobile hybrid vision network, iFormer. The paper goes from ConvNeXt to a lightweight mobile network. iFormer removes memory-intensive operations in MHA and employs an efficient modulation mechanism. The author conduct standard benchmark experiments on ImageNet, COCO and ADE20K.", "questions": "The largest model shown by iFormer, iFormer-L, is only about 15M, which isn’t considered large, even for edge devices, especially since recent edge LLMs can reach 1B parameters. I wonder how well a larger iFormer (around 100M) would perform.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I think the logic of exploration in this article, starting with ConvNeXt, first “lightening” the ConvNeXt to create a streamlined\nlightweight network, then exploring the attention module, is reasonable. \n\nI think the analysis about “cosine similarity between multiples” proves that using a single attention is good and worth supporting. \n\nI think the experiment reported in this paper is comprehensive (imagenet, coco, ade-20k). The paper also reports some knowledge distillation results, which is suitable in mobile network papers.", "weaknesses": "1:  \nSingle head self-attention has been conducted in \"Shvit: Single-head vision transformer with memory efficient macro design\" .\n\nAlternative to standard self-attention has been conducted in GhostNetV2.\n\nModulation in the token mixer module has been conducted in Conv2Former. \n\nThis paper references many related methods, and while that is one approach, I don't think it stands out. Although such research is a decent format, I believe it impacts the novelty of this paper. \n\n2: \nThe process of evolving from the ConvNeXt baseline to the lightweight iFormer may not apply to slightly larger models, and some steps show very minimal improvements, making them hard to justify.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fV716kKFMA", "reviewer": "Reviewer_fhTG", "summary": "This paper presents a new family of mobile hybrid vision networks, called iFormer,  by integrating the fast local representation capacity of convolution with the efficient global modeling ability of self-attention.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper is easy to follow, with clear writing and presentation.\n2. Evaluation results are comprehensive.", "weaknesses": "1. How does this method compare with neural architecture search (NAS) methods?\n\n2. How does the designed model perform on other mobile devices, such as NVIDIA Jetson Nano or Raspberry Pi?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nfK6rbOgir", "reviewer": "Reviewer_aPLf", "summary": "Summary\n\nThis paper proposes a mobile friendly vision network that improves the latency and accuracy by combining the strengths of both CNNs and ViTs. The novel aspect of this work is the single head modulation self-attention (SHMA). This SHMA learns spatial context through optimized self-attention. It takes ConvNext as the base model and improves it further with various techniques. The authors streamline the ConvNeXt architecture, making it suitable for real-time use on mobile devices, such as the iPhone 13, focusing on reducing latency rather than FLOPs or parameter count. The combined techniques lead to more than 80% top-1 accuracy with 1.1ms latency on iphone 13. Overall a great contribution to the research community.", "questions": "See weakness", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. fast local representation capacity of convolution and the efficient global modeling proficiency of the proposed SHMA\n2. A series of novel techniques such as stack of overlapping convolution instead of aggressive non-overlapping patch in the early layers\n3. The model is structured in four stages. The early stages use fast convolution to capture local features efficiently, using a modified and lightweight version of ConvNeXt optimized for mobile latency.\n4. In the lower-resolution stages, self-attention is used to model long-range dependencies. To address the challenges of traditional multi-head self-attention (MHA), the authors propose SHMA, which uses a single-head attention mechanism to minimize memory costs while retaining high performance. SHMA reduces latency by optimizing reshaping operations and leveraging spatial context interactions. SHMA is combined with a parallel feature extraction branch to enhance feature representation. The outputs from both branches are fused to enable dynamic information exchange, mitigating any performance drop caused by simplifying MHA", "weaknesses": "1. What is the runtime complexity of iFormer network?\n2. When running on iPhone (mobile device), what is the peak memory consumption? \n3. How long the iPhone charge will last if an iFormer based app is run on certain fps?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)