[1] Moab Arar, Ariel Shamir, Amit H. Bermano, et al. Learned queries for efficient local attention. In CVPR, 2022.
[2] Chun-Fu (Richard) Chen, Rameswar Panda, and Quanfu Fan. RegionViT: Regional-to-Local Attention for Vision Transformers. In ICLR, 2022.
[3] Kai Chen, Jiaqi Wang, Jiangmiao Pang, et al. MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155, 2019.
[4] Yinpeng Chen, Xiyang Dai, Dongdong Chen, et al. Mobile-former: Bridging mobilenet and transformer. In CVPR, 2022.
[5] Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, and Chunhua Shen. Twins: Revisiting the design of spatial attention in vision transformers. In NeurIPS, 2021.
[6] Xiangxiang Chu, Zhi Tian, Bo Zhang, Xinlong Wang, and Chunhua Shen. Conditional positional encodings for vision transformers. In ICLR, 2023.
[7] MMSegmentation Contributors. Mmsegmentation, an open source semantic segmentation toolbox, 2020.
[8] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, et al. Randaugment: Practical automated data augmentation with a reduced search space. In CVPRW, 2020.
[9] Jia Deng, Wei Dong, Richard Socher, et al. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
[10] Mingyu Ding, Bin Xiao, Noel Codella, et al. Davit: Dual attention vision transformers. In ECCV, 2022.
[11] Xiaohan Ding, Xiangyu Zhang, Yizhuang Zhou, Jungong Han, Guiguang Ding, and Jian Sun. Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. 2022.
[12] Xiaoyi Dong, Jianmin Bao, Dongdong Chen, et al. Cswin transformer: A general vision transformer backbone with cross-shaped windows. In CVPR, 2022.
[13] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
[14] Alaaeldin El-Nouby, Hugo Touvron, Mathilde Caron, et al. Xcit: Cross-covariance image transformers. In NeurIPS, 2021.
[15] Li Gao, Dong Nie, Bo Li, and Xiaofeng Ren. Doubly-fused vit: Fuse information from vision transformer doubly with local representation. In ECCV, 2022.
[16] Jianyuan Guo, Kai Han, Han Wu, Chang Xu, Yehui Tang, Chunjing Xu, and Yunhe Wang. Cmt: Convolutional neural networks meet vision transformers. CVPR, 2022.
[17] Meng-Hao Guo, Cheng-Ze Lu, Zheng-Ning Liu, Ming-Ming Cheng, and Shi-Min Hu. Visual attention network. arXiv preprint arXiv:2202.09741, 2022.
[18] Kai Han, An Xiao, Enhua Wu, et al. Transformer in transformer. In NeurIPS, 2021.
[19] Kaiming He, Georgia Gkioxari, Piotr Dólar, and Ross B. Girshick. Mask r-cnn. In ICCV, 2017.
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Sun Jian. Deep residual learning for image recognition. In CVPR, 2016.
[21] Byeongho Heo, Sangwoo Yun, Dongyoon Han, et al. Rethinking spatial dimensions of vision transformers. In ICCV, 2021.
[22] Qibin Hou, Cheng-Ze Lu, Ming-Ming Cheng, and Jiashi Feng. Conv2former: A simple transformer-style convnet for visual recognition. arXiv preprint arXiv:2211.11943, 2022.
[23] Andrew Howard, Mark Sandler, Grace Chu, et al. Searching for mobilenetv3. In ICCV, 2019.
[24] Gao Huang, Yu Sun, and Zhuang Liu. Deep networks with stochastic depth. In ECCV, 2016.
[25] Huaibo Huang, Xiaogiang Zhou, and Ran He. Orthogonal transformer: An efficient vision transformer backbone with token orthogonalization. In NeurIPS, 2022.
[26] Tao Huang, Lang Huang, Shan You, Fei Wang, Chen Qian, and Chang Xu. Lightvit: Towards light-weight convolution-free vision transformers. arXiv preprint arXiv:2207.05557, 2022.
[27] Alexander Kirillov, Ross Girshick, Kaiming He, and Piotr Dólár. Panoptic feature pyramid networks. In CVPR, 2019.
[28] Youngwan Lee, Jonghee Kim, Jeffrey Willette, and Sung Ju Hwang. Mpvit: Multi-path vision transformer for dense prediction. In CVPR, 2022.
[29] Kunchang Li, Yali Wang, Peng Gao, et al. Uniformer: Unified transformer for efficient spatial-temporal representation learning. In ICLR, 2022.
[30] Yanyu Li, Geng Yuan, Yang Wen, et al. Efficientformer: Vision transformers at mobilenet speed. In NeurIPS, 2022.
[31] Yawei Li, Kai Zhang, Jiezhang Cao, Radu Timofte, and Luc Van Gool. Localvit: Bringing locality to vision transformers. arXiv preprint arXiv:2104.05707, 2021.
[32] Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, and Kaiming He and Piotr Dólár. Focal loss for dense object detection. In ICCV, 2017.
[33] Tsung-Yi Lin, Michael Maire, Serge Belongie, et al. Microsoft coco: Common objects in context. In ECCV, 2014.
[34] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
[35] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, et al. A convnet for the 2020s. In CVPR, 2022.
[36] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In  $ICLR$ , 2019.
[37] Muhammad Maaz, Abdelrahman Shaker, Hisham Cholakkal, Salman Khan, Syed Waqas Zamir, Rao Muhammad Anwer, and Fahad Shahbaz Khan. Edgenext: Efficiently amalgamated cnn-transformer architecture for mobile vision applications. In CADL, 2022.