Original: HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions
Revised: HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

Original: Abstract
Revised: Abstract

Original: Recent progress in vision Transformers exhibits great success in various tasks driven by the new spatial modeling mechanism based on dot-product self-attention.
Revised: Recent progress in vision Transformers exhibits great success in various tasks driven by the new spatial modeling mechanism based on dot-product self-attention.

Original: In this paper, we show that the key ingredients behind the vision Transformers, namely input-adaptive, long-range and high-order spatial interactions, can also be efficiently implemented with a convolution-based framework.
Revised: In this paper, we show that the key ingredients behind the vision Transformers, namely input-adaptive, long-range and high-order spatial interactions, can also be efficiently implemented with a convolution-based framework.

Original: We present the
Revised: We present the

Original: Recursive Gated Convolution ( g n Conv) that performs high-order spatial interactions with gated convolutions and recursive designs.
Revised: Recursive Gated Convolution ( g n Conv) that performs high-order spatial interactions with gated convolutions and recursive designs.

Original: The new operation is highly flexible and customizable, which is compatible with various variants of convolution and extends the two-order interactions in self-attention to arbitrary orders without introducing significant extra computation.
Revised: The new operation is highly flexible and customizable, which is compatible with various variants of convolution and extends the two-order interactions in self-attention to arbitrary orders without introducing significant extra computation.

Original: g n Conv can serve as a plug-and-play module to improve various vision Transformers and convolution-based models.
Revised: g n Conv can serve as a plug-and-play module to improve various vision Transformers and convolution-based models.

Original: Based on the operation, we construct a new family of generic vision backbones named HorNet.
Revised: Based on the operation, we construct a new family of generic vision backbones named HorNet.

Original: Extensive experiments on ImageNet classification, COCO object detection and ADE20K semantic segmentation show
Revised: Extensive experiments on ImageNet classification, COCO object detection and ADE20K semantic segmentation show

Original: HorNet outperform
Revised: HorNet outperform

Original: Swin
Revised: Swin

Original: Transformers and ConvNeXt by a significant margin with similar overall architecture and training configurations.
Revised: Transformers and ConvNeXt by a significant margin with similar overall architecture and training configurations.

Original: HorNet also shows favorable scalability to more training data anda larger model size.
Revised: HorNet also shows favorable scalability to more training data and larger model sizes.

Original: Apart from the effectiveness in visual encoders, we also show g n Conv can be applied to task-specific decoders and consistently improve dense prediction performance with less computation.
Revised: Apart from the effectiveness in visual encoders, we also show g n Conv can be applied to task-specific decoders and consistently improve dense prediction performance with less computation.

Original: Our results demonstrate that g n Conv can be a new basic module for visual modeling that effectively combines the merits of both vision Transformers and CNNs.
Revised: Our results demonstrate that g n Conv can be a new basic module for visual modeling that effectively combines the merits of both vision Transformers and CNNs.

Original: Code is available at https://github.com/raoyongming/HorNet .
Revised: Code is available at https://github.com/raoyongming/HorNet .

Original:  Introduction
Revised:  Introduction

Original: Convolutional neural networks (CNN) have driven remarkable progress in deep learning and computation vision since the introduction of AlexNet [31] in the last decade.
Revised: Convolutional neural networks (CNN) have driven remarkable progress in deep learning and computation vision since the introduction of AlexNet [31] in the last decade.

Original: There are quite a few nice properties of CNNs making them naturally suitable for a wide range of vision applications.
Revised: There are quite a few nice properties of CNNs making them naturally suitable for a wide range of vision applications.

Original: Translation equivariance introduces useful inductive biases to major vision tasks and enables transferability across different input resolutions.
Revised: Translation equivariance introduces useful inductive biases to major vision tasks and enables transferability across different input resolutions.

Original: The highly optimized implementation makes it efficient on both high-performance GPUs and edge devices.
Revised: The highly optimized implementation makes it efficient on both high-performance GPUs and edge devices.

Original: The evolution of architectures [32, 31, 49, 50, 22, 24, 51] further increases its popularity on various vision tasks.
Revised: The evolution of architectures [32, 31, 49, 50, 22, 24, 51] further increases its popularity on various vision tasks.

Original: The emergence of Transformer-based architectures [16, 52, 42] greatly challenges the dominance of CNNs.
Revised: The emergence of Transformer-based architectures [16, 52, 42] greatly challenges the dominance of CNNs.

Original: By combining some successful designs in CNN architectures and the new self-attention mechanism, vision Transformers have shown leading performance on various vision tasks such as image classification [12, 42, 48], object detection [70, 41], semantic segmentation [6, 8] and video understanding [64, 18].
Revised: By combining some successful designs in CNN architectures and the new self-attention mechanism, vision Transformers have shown leading performance on various vision tasks such as image classification [12, 42, 48], object detection [70, 41], semantic segmentation [6, 8] and video understanding [64, 18].

Original: What makes vision Transformers more powerful than CNNs?
Revised: What makes vision Transformers more powerful than CNNs?

Original: Some efforts have been made to improve the CNN architectures by learning from the new designs in vision
Revised: Some efforts have been made to improve the CNN architectures by learning from the new designs in vision

Original: [Figure]
Revised: [Figure]

Original: Transformers.
Revised: Transformers.

Original: [43] presents a thorough study to adopt the meta architecture of vision Transformer to improve CNNs and proposes to use a large 7 × 7 kernel to construct a modern CNN.
Revised: [43] presents a thorough study to adopt the meta architecture of vision Transformer to improve CNNs and proposes to use a large 7 × 7 kernel to construct a modern CNN.

Original: [46] and [14] propose to use even larger kernels to learn long-range relations with global filters and up to 31 × 31 convolutions, respectively.
Revised: [46] and [14] propose to use even larger kernels to learn long-range relations with global filters and up to 31 × 31 convolutions, respectively.

Original: [20] shows that the input-adaptive weights play a key role in vision Transformers and achieve similar performance with Swin Transformers with dynamic convolutions [4, 28].
Revised: [20] shows that the input-adaptive weights play a key role in vision Transformers and achieve similar performance with Swin Transformers with dynamic convolutions [4, 28].

Original: However, the effectiveness of dot-product self-attention in vision tasks has not been analyzed from the prospective of high-order spatial interactions .
Revised: However, the effectiveness of dot-product self-attention in vision tasks has not been analyzed from the prospective of high-order spatial interactions .

Original: While there exists complex and often high-order interactions between two spatial locations in a deep model due to the non-linearity, the success of self-attention and other dynamic networks suggests that the explicit and high-order spatial interactions introduced by the architectural designs are beneficial to improving the modeling power of vision models.
Revised: While there exists complex and often high-order interactions between two spatial locations in a deep model due to the non-linearity, the success of self-attention and other dynamic networks suggests that the explicit and high-order spatial interactions introduced by the architectural designs are beneficial to improving the modeling power of vision models.

Original: As illustrated in Figure 1, the plain convolution operation does not explicitly consider the spatial interactions between a spatial location ( i.e ., the red feature) and its neighboring region ( i.e ., the light gray region).
Revised: As illustrated in Figure 1, the plain convolution operation does not explicitly consider the spatial interactions between a spatial location ( i.e ., the red feature) and its neighboring region ( i.e ., the light gray region).

Original: Enhanced convolution operations like dynamic convolution [4, 28, 20] introduce explicit spatial interaction by generating dynamic weights.
Revised: Enhanced convolution operations like dynamic convolution [4, 28, 20] introduce explicit spatial interaction by generating dynamic weights.

Original: The dot-product self-attention operation in Transformers [56] consists of two successive spatial interactions by performing matrix multiplication among queries, keys and values.
Revised: The dot-product self-attention operation in Transformers [56] consists of two successive spatial interactions by performing matrix multiplication among queries, keys and values.

Original: The trend of the basic operations for visual modeling indicates that the network capacity can be improved by increasing the order of spatial interactions.
Revised: The trend of the basic operations for visual modeling indicates that the network capacity can be improved by increasing the order of spatial interactions.

Original: In this paper, we summarize that the key ingredient behind the success of vision Transformers is the new way of spatial modeling with input-adaptive , long-range and high-order spatial interactions performed by the self-attention operation.
Revised: In this paper, we summarize that the key ingredient behind the success of vision Transformers is the new way of spatial modeling with input-adaptive , long-range and high-order spatial interactions performed by the self-attention operation.

Original: While previous work has successfully migrated the meta architecture [43, 20, 46, 14], input-adaptive weight generation strategy [20] and large-range modeling ability [46, 14] of vision Transformers to CNN models, a higher-order spatial interaction mechanism has not been studied.
Revised: While previous work has successfully migrated the meta architecture [43, 20, 46, 14], input-adaptive weight generation strategy [20] and large-range modeling ability [46, 14] of vision Transformers to CNN models, a higher-order spatial interaction mechanism has not been studied.

Original: We show that all the three key ingredients can be efficiently implemented using a convolution-based framework.
Revised: We show that all the three key ingredients can be efficiently implemented using a convolution-based framework.

Original: We propose the Recursive Gated Convolution ( g n Conv) that performs high-order spatial interactions with gated convolutions and recursive deigns.
Revised: We propose the Recursive Gated Convolution ( g n Conv) that performs high-order spatial interactions with gated convolutions and recursive deigns.

Original: Instead of simply imitating the successful designs in self-attention, g n Conv has several extra favorable properties: 1) Efficient .
Revised: Instead of simply imitating the successful designs in self-attention, g n Conv has several extra favorable properties: 1) Efficient .

Original: The convolution-based implementation avoids the quadratic complexity of self-attention.
Revised: The convolution-based implementation avoids the quadratic complexity of self-attention.

Original: The design that progressively increases the channel width during performing spatial interactions also enables us to achieve higher-order interactions with bounded complexity; 2) Extendable .
Revised: The design that progressively increases the channel width during performing spatial interactions also enables us to achieve higher-order interactions with bounded complexity; 2) Extendable .

Original: We extend the two-order interaction in self-attention to arbitrary orders to further improve the modeling power.
Revised: We extend the two-order interaction in self-attention to arbitrary orders to further improve the modeling power.

Original: Since we do not make assumptions on the type of spatial convolution, g n Conv is compatible with various kernel size and spatial mixing strategies like [46, 14]; 3) Translation-equivariant .
Revised: Since we do not make assumptions on the type of spatial convolution, g n Conv is compatible with various kernel size and spatial mixing strategies like [46, 14]; 3) Translation-equivariant .

Original: g n Conv fully inherits the translation equivariance of the standard convolution, which introduces beneficial inductive biases to major vision tasks and avoids the asymmetry brought by local attention [42, 34].
Revised: g n Conv fully inherits the translation equivariance of the standard convolution, which introduces beneficial inductive biases to major vision tasks and avoids the asymmetry brought by local attention [42, 34].

Original: Based on g n Conv, we construct a new family of generic vision backbones named HorNet.
Revised: Based on g n Conv, we construct a new family of generic vision backbones named HorNet.

Original: We conduct extensive experiments on ImageNet classification [13], COCO object detection [38] and
Revised: We conduct extensive experiments on ImageNet classification [13], COCO object detection [38] and

Original: ADE20K semantic segmentation [71] to verify the effectiveness of our models.
Revised: ADE20K semantic segmentation [71] to verify the effectiveness of our models.

Original: With the same 7 ×kernel/window and similar overall architecture and training configurations, HorNet outperforms Swin and ConvNeXt by a large margin on all tasks at different levels of complexity.
Revised: With the same 7 ×kernel/window and similar overall architecture and training configurations, HorNet outperforms Swin and ConvNeXt by a large margin on all tasks at different levels of complexity.

Original: The gap can be further enlarged by using a global kernel size [46].
Revised: The gap can be further enlarged by using a global kernel size [46].

Original: HorNet also shows favorable scalability to more training data and larger model size, attaining 87.7% top-1 accuracy on ImageNet, 57.9% mIoU on ADE20K val and 59.2% bounding box AP on COCO val with ImageNet-22K pre-training.
Revised: HorNet also shows favorable scalability to more training data and larger model size, attaining 87.7% top-1 accuracy on ImageNet, 57.9% mIoU on ADE20K val and 59.2% bounding box AP on COCO val with ImageNet-22K pre-training.

Original: Apart from applying g n Conv in visual encoders, we further test the generality of our designs on task-specific decoders.
Revised: Apart from applying g n Conv in visual encoders, we further test the generality of our designs on task-specific decoders.

Original: By adding g Conv to the widely used feature fusion model FPN [36], we develop HorFPN to model the high-order spatial relationships of features from different hierarchical levels.
Revised: By adding g Conv to the widely used feature fusion model FPN [36], we develop HorFPN to model the high-order spatial relationships of features from different hierarchical levels.

Original: We observe that
Revised: We observe that

Original: HorFPN can also consistently improve various dense prediction models with lower computational costs.
Revised: HorFPN can also consistently improve various dense prediction models with lower computational costs.

Original: Our results demonstrate that g n Conv can be a promising alternative to self-attention for visual modeling and effectively combine the merits of both vision Transformers and CNNs.
Revised: Our results demonstrate that g n Conv can be a promising alternative to self-attention for visual modeling and effectively combine the merits of both vision Transformers and CNNs.

Original:  Related Work
Revised:  Related Work

Original: Vision Transformers.
Revised: Vision Transformers.

Original: The Transformer architecture [56] is originally designed for the natural language processing tasks.
Revised: The Transformer architecture [56] is originally designed for the natural language processing tasks.

Original: Since Dosovitskiy et al .
Revised: Since Dosovitskiy et al .

Original: [16] show that vision models constructed only by the Transformer blocks and a patch embedding layer can also achieve competitive performance to CNNs, many new models have been proposed to modify the Transformer-based architecture and make it more suitable for various vision tasks [42, 58, 60, 9, 66, 55].
Revised: [16] show that vision models constructed only by the Transformer blocks and a patch embedding layer can also achieve competitive performance to CNNs, many new models have been proposed to modify the Transformer-based architecture and make it more suitable for various vision tasks [42, 58, 60, 9, 66, 55].

Original: Different from the original designs in [16], state-of-the-art vision Transformers usually utilize a CNN-like hierarchical architecture and change the global self-attention among all patches to local self-attention to avoid the quadratic complexity.
Revised: Different from the original designs in [16], state-of-the-art vision Transformers usually utilize a CNN-like hierarchical architecture and change the global self-attention among all patches to local self-attention to avoid the quadratic complexity.

Original: In this paper, we follow the overall architecture of the previous hierarchical vision Transformers [42] and replace the self-attention sub-layer with our proposed g n Conv to fairly compare with the previous Transformer-based models.
Revised: In this paper, we follow the overall architecture of the previous hierarchical vision Transformers [42] and replace the self-attention sub-layer with our proposed g n Conv to fairly compare with the previous Transformer-based models.

Original: Convolution-based models.
Revised: Convolution-based models.

Original: Inspired by the recent success of vision Transformers, several papers propose to adopt the Transformer-style architecture and spatial convolutions with a large kernel size to improve the performance of CNNs.
Revised: Inspired by the recent success of vision Transformers, several papers propose to adopt the Transformer-style architecture and spatial convolutions with a large kernel size to improve the performance of CNNs.

Original: Han et al .
Revised: Han et al .

Original: [20] replace the window self-attention in Swin
Revised: [20] replace the window self-attention in Swin

Original: Transformers with large-kernel dynamic convolutions and achieve better performance.
Revised: Transformers with large-kernel dynamic convolutions and achieve better performance.

Original: GFNet [46] proposes to perform the global spatial interactions like vision Transformers with global filters in the frequency domain, which are equivalent to depth-wise convolutions with a global kernel size and circular padding.
Revised: GFNet [46] proposes to perform the global spatial interactions like vision Transformers with global filters in the frequency domain, which are equivalent to depth-wise convolutions with a global kernel size and circular padding.

Original: ConvNeXt [43] thoroughly analyzes the designs in recent vision Transformers and presents a strong convolutional model with 7 × 7 depth-wise convolutions.
Revised: ConvNeXt [43] thoroughly analyzes the designs in recent vision Transformers and presents a strong convolutional model with 7 × 7 depth-wise convolutions.

Original: RepLKNet
Revised: RepLKNet

Original: [14] explores
Revised: [14] explores

Original: CNN models with very large kernels (up to 31 × 31), showing good scalability as vision Transformers.
Revised: CNN models with very large kernels (up to 31 × 31), showing good scalability as vision Transformers.

Original: VAN [19] and FocalNet [65] use gated convolutions to perform input-adaptive attention and adopts large-kernel dilated convolutions and multiple successive 3 × 3 convolutions respectively to produce the weights.
Revised: VAN [19] and FocalNet [65] use gated convolutions to perform input-adaptive attention and adopts large-kernel dilated convolutions and multiple successive 3 × 3 convolutions respectively to produce the weights.

Original: Previous work focuses on the meta architecture [67], large-kernel designs and inputadaptive weights to improve CNNs by learning from vision Transformers.
Revised: Previous work focuses on the meta architecture [67], large-kernel designs and inputadaptive weights to improve CNNs by learning from vision Transformers.

Original: In this paper, we offer a new perspective of high-order spatial attention to analyze the merits of vision Transformers.
Revised: In this paper, we offer a new perspective of high-order spatial attention to analyze the merits of vision Transformers.

Original: We show that the proposed HorNet that combines the advantages of both CNNs and vision Transformers is a better architecture for various vision tasks.
Revised: We show that the proposed HorNet that combines the advantages of both CNNs and vision Transformers is a better architecture for various vision tasks.

Original: Hybrid models.
Revised: Hybrid models.

Original: Combining vision Transformers and CNNs to develop hybrid architectures is a new direction in various visual recognition problems.
Revised: Combining vision Transformers and CNNs to develop hybrid architectures is a new direction in various visual recognition problems.

Original: Recently, several efforts have been made to integrate the two types of blocks into a unified model with a sequential [12, 29, 68, 63] or parallel [45, 11] design.
Revised: Recently, several efforts have been made to integrate the two types of blocks into a unified model with a sequential [12, 29, 68, 63] or parallel [45, 11] design.

Original: Many enhanced vision Transformers also use lightweight convolutions in the basic building block to efficiently capture neighboring patterns [15, 60, 17] or relax the quadratic complexity of self-attention [9, 58, 18].
Revised: Many enhanced vision Transformers also use lightweight convolutions in the basic building block to efficiently capture neighboring patterns [15, 60, 17] or relax the quadratic complexity of self-attention [9, 58, 18].

Original: Different from these hybrid models, we aim to develop a self-attention free model while combining the favorable properties of both vision Transformers and CNNs.
Revised: Different from these hybrid models, we aim to develop a self-attention free model while combining the favorable properties of both vision Transformers and CNNs.

Original:  Method 3.1 g n Conv: Recursive Gated Convolutions
Revised:  Method 3.1 g n Conv: Recursive Gated Convolutions

Original: In this section, we will present g n Conv, an efficient operation to achieve long-term and high-order spatial interactions.
Revised: In this section, we will present g n Conv, an efficient operation to achieve long-term and high-order spatial interactions.

Original: The g n Conv is built with standard convolutions, linear projections and elementwise multiplications, but has a similar function of input-adaptive spatial mixing to self-attention.
Revised: The g n Conv is built with standard convolutions, linear projections and elementwise multiplications, but has a similar function of input-adaptive spatial mixing to self-attention.

Original: Input-adaptive interactions with gated convolution.
Revised: Input-adaptive interactions with gated convolution.

Original: Recent success in vision Transformers mainly depends on the proper modeling of the spatial interactions in visual data.
Revised: Recent success in vision Transformers mainly depends on the proper modeling of the spatial interactions in visual data.

Original: Unlike CNNs that simply use the static convolution kernel to aggregate neighboring features, vision Transformers apply multi-head self-attention to dynamically generate the weights to mix spatial tokens.
Revised: Unlike CNNs that simply use the static convolution kernel to aggregate neighboring features, vision Transformers apply multi-head self-attention to dynamically generate the weights to mix spatial tokens.

Original: However, the quadratic complexity w.r.t.
Revised: However, the quadratic complexity w.r.t.

Original: the input size of the self-attention largely hinders the application of vision Transformers, especially on downstream tasks including segmentation and detection where higher-resolution feature maps are required.
Revised: the input size of the self-attention largely hinders the application of vision Transformers, especially on downstream tasks including segmentation and detection where higher-resolution feature maps are required.

Original: In this work, instead of reducing the complexity of self-attention like previous methods [42, 9, 57], we seek a more efficient and effective way to perform spatial interactions with simple operations like convolution and fully-connected layers.
Revised: In this work, instead of reducing the complexity of self-attention like previous methods [42, 9, 57], we seek a more efficient and effective way to perform spatial interactions with simple operations like convolution and fully-connected layers.

Original: The basic operation of our method is the gated convolution ( g Conv).
Revised: The basic operation of our method is the gated convolution ( g Conv).

Original: Let x ∈ R
Revised: Let x ∈ R

Original: HW × C be the input feature, the output of the gated convolution y =
Revised: HW × C be the input feature, the output of the gated convolution y =

Original: g Conv ( x ) can be written as:
Revised: g Conv ( x ) can be written as:

Original: [Equation]
Revised: [Equation]

Original: where ϕ in , ϕ out are linear projection layers to perform channel mixing and f is a depth-wise convolution.
Revised: where ϕ in , ϕ out are linear projection layers to perform channel mixing and f is a depth-wise convolution.

Original: Note that p ( i,c )
Revised: Note that p ( i,c )

Original: [Equation]
Revised: [Equation]

Original: , where Ω i is the local window centered at i and w represents the convolution weight of f .
Revised: , where Ω i is the local window centered at i and w represents the convolution weight of f .

Original: Therefore, the above formulation explicitly introduce interactions among the neighboring features p ( i ) 0 and q ( j ) 0 through the element-wise multiplication.
Revised: Therefore, the above formulation explicitly introduce interactions among the neighboring features p ( i ) 0 and q ( j ) 0 through the element-wise multiplication.

Original: We consider the interaction in g Conv as 1-order interaction as each p ( i ) 0 has interacted with its neighbor feature q ( j ) 0 only once.
Revised: We consider the interaction in g Conv as 1-order interaction as each p ( i ) 0 has interacted with its neighbor feature q ( j ) 0 only once.

Original: High-order interactions with recursive gating.
Revised: High-order interactions with recursive gating.

Original: After achieving an efficient 1-order spatial interactions with the g Conv, we then design the g n Conv, a recursive gated convolution to further enhance the model capacity by introducing higher-order interactions.
Revised: After achieving an efficient 1-order spatial interactions with the g Conv, we then design the g n Conv, a recursive gated convolution to further enhance the model capacity by introducing higher-order interactions.

Original: Formally, we first use ϕ in to obtain a set of projected features p 0 and
Revised: Formally, we first use ϕ in to obtain a set of projected features p 0 and

Original: [Equation]
Revised: [Equation]

Original: We then perform the gated convolution recursively by
Revised: We then perform the gated convolution recursively by

Original: [Equation]
Revised: [Equation]

Original: where we scale the output by 1 /α to stabilize the training.
Revised: where we scale the output by 1 /α to stabilize the training.

Original: { f k } are a set of depth-wise convolution layers and { g k } are used to match the dimension in different orders:
Revised: { f k } are a set of depth-wise convolution layers and { g k } are used to match the dimension in different orders:

Original: [Equation]
Revised: [Equation]

Original: Finally, we feed the output of the last recursion step q n to the projection layer ϕ out to obtain the result of the g n Conv.
Revised: Finally, we feed the output of the last recursion step q n to the projection layer ϕ out to obtain the result of the g n Conv.

Original: From the recursive formula Equation (3.3), it is easy to show that the interaction-order of p k will be increased by 1 after each step.
Revised: From the recursive formula Equation (3.3), it is easy to show that the interaction-order of p k will be increased by 1 after each step.

Original: As a result, we can see that the g n Conv achieves n -order spatial interactions.
Revised: As a result, we can see that the g n Conv achieves n -order spatial interactions.

Original: It is also worth noting that we need only a single f to perform depthwise convolution to the concatenation of the features { q k } n − 1 k =0 together instead of computing the convolution in each recursive step as in Equation (3.3), which can further simplify the implementation and improve the efficiency on GPUs.
Revised: It is also worth noting that we need only a single f to perform depthwise convolution to the concatenation of the features { q k } n − 1 k =0 together instead of computing the convolution in each recursive step as in Equation (3.3), which can further simplify the implementation and improve the efficiency on GPUs.

Original: To ensure that the high-order interactions do not introduce too much computational overhead, we set the channel dimension in each order as:
Revised: To ensure that the high-order interactions do not introduce too much computational overhead, we set the channel dimension in each order as:

Original: [Equation]
Revised: [Equation]

Original: This design indicates that we perform the interactions in a coarse-to-fine manner, where lower orders are computed with fewer channels.
Revised: This design indicates that we perform the interactions in a coarse-to-fine manner, where lower orders are computed with fewer channels.

Original: Besides, the channel dimension of ϕ in ( x ) is exactly 2 C and the total FLOPs can be strictly bounded even with n increasing.
Revised: Besides, the channel dimension of ϕ in ( x ) is exactly 2 C and the total FLOPs can be strictly bounded even with n increasing.

Original: It can be proved that (see Appendix A):
Revised: It can be proved that (see Appendix A):

Original: [Equation]
Revised: [Equation]

Original: where K is the kernel size of the depth-wise convolution.
Revised: where K is the kernel size of the depth-wise convolution.

Original: Therefore, our g n Conv achieves high-order interactions with a similar computational cost to a convolutional layer.
Revised: Therefore, our g n Conv achieves high-order interactions with a similar computational cost to a convolutional layer.

Original: Long-term interactions with large kernel convolutions.
Revised: Long-term interactions with large kernel convolutions.

Original: Another difference between vision
Revised: Another difference between vision

Original: Transformers and conventional CNNs is the receptive field.
Revised: Transformers and conventional CNNs is the receptive field.

Original: Conventional CNNs [49, 22] often use 3 × 3 convolution through the whole network, while vision Transformers calculate self-attention on the whole feature maps [16, 52] or inside a relatively large local window ( e.g ., 7 × 7).
Revised: Conventional CNNs [49, 22] often use 3 × 3 convolution through the whole network, while vision Transformers calculate self-attention on the whole feature maps [16, 52] or inside a relatively large local window ( e.g ., 7 × 7).

Original: The large
Revised: The large

Original: [Figure]
Revised: [Figure]

Original: HorBlock
Revised: HorBlock

Original: [Figure]
Revised: [Figure]

Original: receptive field in vision
Revised: receptive field in vision

Original: Transformers makes it easier to capture long-term dependencies, which is also recognized as one of the key advantages of vision Transformers.
Revised: Transformers makes it easier to capture long-term dependencies, which is also recognized as one of the key advantages of vision Transformers.

Original: Inspired by this design, there are some efforts to introduce large kernel convolutions to CNNs recently [14, 43, 46].
Revised: Inspired by this design, there are some efforts to introduce large kernel convolutions to CNNs recently [14, 43, 46].

Original: To make our g n Conv capable of capturing long-term interactions, we adopt two implementations for the depth-wise convolution f :
Revised: To make our g n Conv capable of capturing long-term interactions, we adopt two implementations for the depth-wise convolution f :

Original: • 7 × 7 Convolution .
Revised: • 7 × 7 Convolution .

Original: 7 × 7 is the default window/kernel size of Swin Transformers [42] and ConvNext [43].
Revised: 7 × 7 is the default window/kernel size of Swin Transformers [42] and ConvNext [43].

Original: Studies in [43] show that the kernel size produces good performance on ImageNet classification and various downstream tasks.
Revised: Studies in [43] show that the kernel size produces good performance on ImageNet classification and various downstream tasks.

Original: We follow this configuration to fairly compare with representative work of vision Transformers and modern CNNs.
Revised: We follow this configuration to fairly compare with representative work of vision Transformers and modern CNNs.

Original: • Global Filter (GF) .
Revised: • Global Filter (GF) .

Original: The GF layer [46] multiplies the frequency domain features with learnable global filters, which is equivalent to a convolution in the spatial domain with a global kernel size and circular padding.
Revised: The GF layer [46] multiplies the frequency domain features with learnable global filters, which is equivalent to a convolution in the spatial domain with a global kernel size and circular padding.

Original: We use a modified version of the GF layer by processing half of the channels with the global filter and the other half with 3 × 3 depth-wise convolutions and only use GF layers in late stages to preserve more local details.
Revised: We use a modified version of the GF layer by processing half of the channels with the global filter and the other half with 3 × 3 depth-wise convolutions and only use GF layers in late stages to preserve more local details.

Original: Spatial interactions in vision models.
Revised: Spatial interactions in vision models.

Original: We review some representative vision model designs from the perspective of spatial interactions, as shown in Figure 1.
Revised: We review some representative vision model designs from the perspective of spatial interactions, as shown in Figure 1.

Original: Specifically, we are interested in the interactions between a feature x i and its neighboring feature x j , j ∈ Ω i .
Revised: Specifically, we are interested in the interactions between a feature x i and its neighboring feature x j , j ∈ Ω i .

Original: By using the tool designed for explaining the interaction effect (IE) in [33, 1], we provide an intuitive analysis of the order of explicit spatial interactions in Appendix B.
Revised: By using the tool designed for explaining the interaction effect (IE) in [33, 1], we provide an intuitive analysis of the order of explicit spatial interactions in Appendix B.

Original: Our analysis reveals a key difference between vision
Revised: Our analysis reveals a key difference between vision

Original: Transformers and previous architectures from a new view, i.e ., vision Transformers have higher-order spatial interactions in each basic block.
Revised: Transformers and previous architectures from a new view, i.e ., vision Transformers have higher-order spatial interactions in each basic block.

Original: The result inspires us to explore an architecture that can realize more efficient and effective spatial interactions with more than two orders.
Revised: The result inspires us to explore an architecture that can realize more efficient and effective spatial interactions with more than two orders.

Original: As discussed above, our proposed g n Conv can achieve arbitrary-order interactions with bounded complexity.
Revised: As discussed above, our proposed g n Conv can achieve arbitrary-order interactions with bounded complexity.

Original: It is also worth noting that similar to other scaling factors in deep models like width [69] and depth [22], simply increasing the order of spatial interactions without considering the overall model capacity will not lead to a good trade-off [51].
Revised: It is also worth noting that similar to other scaling factors in deep models like width [69] and depth [22], simply increasing the order of spatial interactions without considering the overall model capacity will not lead to a good trade-off [51].

Original: In this paper, we focus on developing a stronger visual modeling architecture based on the analysis of the spatial interaction orders of well-designed models.
Revised: In this paper, we focus on developing a stronger visual modeling architecture based on the analysis of the spatial interaction orders of well-designed models.

Original: We believe a more thorough and formal discussion on the high-order spatial interactions can be an important future direction.
Revised: We believe a more thorough and formal discussion on the high-order spatial interactions can be an important future direction.

Original: Relation to dot-product self-attention.
Revised: Relation to dot-product self-attention.

Original: Although the computation of our g n Conv largely differs from dot-product self-attention, we will show that g n Conv also accomplishes the goal of inputadaptive spatial mixing.
Revised: Although the computation of our g n Conv largely differs from dot-product self-attention, we will show that g n Conv also accomplishes the goal of inputadaptive spatial mixing.

Original: Let M be the attention matrix obtained by multi-head self-attention (MHSA), we write M as ( m c ij ) since the mixing weight may vary across the channels.
Revised: Let M be the attention matrix obtained by multi-head self-attention (MHSA), we write M as ( m c ij ) since the mixing weight may vary across the channels.

Original: The spatial mixing result (before the final channel mixing projection) of the c -th channel at location i is
Revised: The spatial mixing result (before the final channel mixing projection) of the c -th channel at location i is

Original: [Equation]
Revised: [Equation]

Original: where w V is the weight of the V-projection layer.
Revised: where w V is the weight of the V-projection layer.

Original: Note that m ij obtained by the dot-product operation contains 1-order interaction.
Revised: Note that m ij obtained by the dot-product operation contains 1-order interaction.

Original: On the other hand, the output of our g n Conv (before the ϕ out ) can be written as
Revised: On the other hand, the output of our g n Conv (before the ϕ out ) can be written as

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where w n − 1 is the convolutional weight for f n − 1 , w ϕ in is the linear weight of ϕ in , and g n − 1 =
Revised: where w n − 1 is the convolutional weight for f n − 1 , w ϕ in is the linear weight of ϕ in , and g n − 1 =

Original: g n − 1 ( p n − 1 ) is a projection of p n − 1 .
Revised: g n − 1 ( p n − 1 ) is a projection of p n − 1 .

Original: From the formulation in Equation (3.8) we find our g n Conv also achieves input-adaptive spatial mixing with { h cij } as the weights.
Revised: From the formulation in Equation (3.8) we find our g n Conv also achieves input-adaptive spatial mixing with { h cij } as the weights.

Original: Observing that h ij is computed from p n − 1 which contains n − 1 order interactions, we can regard our g n Conv as an extension of the self-attention in terms of the order of the spatial mixing weight.
Revised: Observing that h ij is computed from p n − 1 which contains n − 1 order interactions, we can regard our g n Conv as an extension of the self-attention in terms of the order of the spatial mixing weight.

Original: Therefore, our g n Conv can better model more complex spatial interactions.
Revised: Therefore, our g n Conv can better model more complex spatial interactions.

Original: The details of g n Conv and our implementation are summarized in Figure 2.
Revised: The details of g n Conv and our implementation are summarized in Figure 2.

Original: 3.2 Model Architectures
Revised: 3.2 Model Architectures

Original: HorNet.
Revised: HorNet.

Original: The g n Conv can be a drop-in replacement of the spatial mixing layer in vision Transformers [52, 42] or modern CNNs [43].
Revised: The g n Conv can be a drop-in replacement of the spatial mixing layer in vision Transformers [52, 42] or modern CNNs [43].

Original: We follow the same meta-architecture as [56, 42] to construct
Revised: We follow the same meta-architecture as [56, 42] to construct

Original: HorNet, where the basic block contains a spatial mixing layer and a feed-forward network (FFN).
Revised: HorNet, where the basic block contains a spatial mixing layer and a feed-forward network (FFN).

Original: Depending on the model size and the implementation of the depth-wise convolution f k in our g n Conv, we have two series of model variants named HorNet-T/S/B/L 7 × 7 and HorNet-T/S/B/L GF .
Revised: Depending on the model size and the implementation of the depth-wise convolution f k in our g n Conv, we have two series of model variants named HorNet-T/S/B/L 7 × 7 and HorNet-T/S/B/L GF .

Original: We consider the popular Swin Transformer [42] and ConvNeXt [43] as the vision Transformer and CNN baselines since our models are implemented based on a convolution-based framework while having high-order interactions like vision Transformers.
Revised: We consider the popular Swin Transformer [42] and ConvNeXt [43] as the vision Transformer and CNN baselines since our models are implemented based on a convolution-based framework while having high-order interactions like vision Transformers.

Original: To fairly compare with the baselines, we directly follow the number of blocks of Swin Transformers-S/B/L [42] but insert an extra block to the stageto make the overall complexity close, resulting in [2 , 3 , 18 , 2] blocks in each stage in all of the model variants.
Revised: To fairly compare with the baselines, we directly follow the number of blocks of Swin Transformers-S/B/L [42] but insert an extra block to the stageto make the overall complexity close, resulting in [2 , 3 , 18 , 2] blocks in each stage in all of the model variants.

Original: We simply adjust the base number of channels C to construct models with different sizes and set the number of channels in 4 stages as [ C, 2 C, 4 C, 8 C ] following common practice.
Revised: We simply adjust the base number of channels C to construct models with different sizes and set the number of channels in 4 stages as [ C, 2 C, 4 C, 8 C ] following common practice.

Original: We use
Revised: We use

Original: C = 64 , 96 , 128 , 192 for HorNet-T/S/B/L, respectively.
Revised: C = 64 , 96 , 128 , 192 for HorNet-T/S/B/L, respectively.

Original: We set the interaction orders ( i.e ., the n in g n Conv) for each stage as 2,3,4,5 by default, such that the channels of the coarsest order C 0 is the same across different stages.
Revised: We set the interaction orders ( i.e ., the n in g n Conv) for each stage as 2,3,4,5 by default, such that the channels of the coarsest order C 0 is the same across different stages.

Original: HorFPN.
Revised: HorFPN.

Original: Apart from using g n Conv in visual encoders, we find our g n Conv can be an enhanced alternative for standard convolution that considers higher-order spatial interactions in a wide range of convolution-based models.
Revised: Apart from using g n Conv in visual encoders, we find our g n Conv can be an enhanced alternative for standard convolution that considers higher-order spatial interactions in a wide range of convolution-based models.

Original: Thus, we replace spatial convolutions for feature fusion in the FPN [37] with our g n Conv to improve spatial interactions for downstream tasks.
Revised: Thus, we replace spatial convolutions for feature fusion in the FPN [37] with our g n Conv to improve spatial interactions for downstream tasks.

Original: Specifically, we add our g n Conv after the fusion of features from different pyramid levels.
Revised: Specifically, we add our g n Conv after the fusion of features from different pyramid levels.

Original: For object detection, we replace the 3 × 3 convolution after the top-down pathway with the g n Conv in each level.
Revised: For object detection, we replace the 3 × 3 convolution after the top-down pathway with the g n Conv in each level.

Original: For semantic segmentation, we simply replace the 3 × 3 convolution after the concatenation of the multi-level feature maps with g n Conv since the final results are directly predicted from this concatenated feature.
Revised: For semantic segmentation, we simply replace the 3 × 3 convolution after the concatenation of the multi-level feature maps with g n Conv since the final results are directly predicted from this concatenated feature.

Original: We also have two implementations called HorFPN 7 × 7 and HorFPN GF decided by the choice of f k .
Revised: We also have two implementations called HorFPN 7 × 7 and HorFPN GF decided by the choice of f k .

Original:  Experiments
Revised:  Experiments

Original: We conduct extensive experiments to verify the effectiveness of our method.
Revised: We conduct extensive experiments to verify the effectiveness of our method.

Original: We present the main results on ImageNet [13] and compare them with various architectures.
Revised: We present the main results on ImageNet [13] and compare them with various architectures.

Original: We also test our models on the downstream dense prediction tasks on commonly used semantic segmentation benchmark
Revised: We also test our models on the downstream dense prediction tasks on commonly used semantic segmentation benchmark

Original: ADE20K [71] and object detection dataset COCO [38].
Revised: ADE20K [71] and object detection dataset COCO [38].

Original: Lastly, we provide ablation studies of our designs and analyze the effectiveness of g n Conv on a wide range of models.
Revised: Lastly, we provide ablation studies of our designs and analyze the effectiveness of g n Conv on a wide range of models.

Original: 4.1 ImageNet Classification
Revised: 4.1 ImageNet Classification

Original: Setups.
Revised: Setups.

Original: We conduct image classification experiments on the widely used ImageNet [13] dataset.
Revised: We conduct image classification experiments on the widely used ImageNet [13] dataset.

Original: We train our HorNet-T/S/B models using the standard ImageNet-1K dataset following common practice.
Revised: We train our HorNet-T/S/B models using the standard ImageNet-1K dataset following common practice.

Original: To fairly compare with previous work, we directly use the training configurations of [43, 42, 52] to train our models.
Revised: To fairly compare with previous work, we directly use the training configurations of [43, 42, 52] to train our models.

Original: We train the models for 300 epochs with 224 × 224 input.
Revised: We train the models for 300 epochs with 224 × 224 input.

Original: To evaluate the scaling ability of our designs, we further train the HorNet-L models on the ImageNet-22K dataset that contains over 10 × images and more categories.
Revised: To evaluate the scaling ability of our designs, we further train the HorNet-L models on the ImageNet-22K dataset that contains over 10 × images and more categories.

Original: We follow previous practice [42, 43] to train our models for 90 epochs and use a similar data augmentation strategy as ImageNet-1K experiments.
Revised: We follow previous practice [42, 43] to train our models for 90 epochs and use a similar data augmentation strategy as ImageNet-1K experiments.

Original: [Table]
Revised: [Table]

Original: We fine-tune the models pre-trained on ImageNet-22K or at the 224×224 resolution to ImageNet-1K or/and 384×384 resolution for 30 epochs following [43].
Revised: We fine-tune the models pre-trained on ImageNet-22K or at the 224×224 resolution to ImageNet-1K or/and 384×384 resolution for 30 epochs following [43].

Original: When adapting the ImageNet-22K models to ImageNet-1K, we initialize the classifier with the pre-trained class centers to stabilize the training process.
Revised: When adapting the ImageNet-22K models to ImageNet-1K, we initialize the classifier with the pre-trained class centers to stabilize the training process.

Original: More details can be found in Appendix C.
Revised: More details can be found in Appendix C.

Original: Results.
Revised: Results.

Original: The results of our ImageNet classification experiments are summarized in Table 1.
Revised: The results of our ImageNet classification experiments are summarized in Table 1.

Original: We see that our models achieve very competitive performance with state-of-the-art vision Transformers and
Revised: We see that our models achieve very competitive performance with state-of-the-art vision Transformers and

Original: CNNs.
Revised: CNNs.

Original: Notably, HorNet surpasses Swin Transformers and ConvNeXt which have similar overall architectures and training configurations by a healthy margin on various model sizes and settings.
Revised: Notably, HorNet surpasses Swin Transformers and ConvNeXt which have similar overall architectures and training configurations by a healthy margin on various model sizes and settings.

Original: Our models also generalize well to a larger image resolution, larger model sizes and more training data.
Revised: Our models also generalize well to a larger image resolution, larger model sizes and more training data.

Original: These results clearly demonstrate the effectiveness and generality of our designs.
Revised: These results clearly demonstrate the effectiveness and generality of our designs.

Original: 4.2 Dense Prediction Tasks
Revised: 4.2 Dense Prediction Tasks

Original: HorNet for semantic segmentation.
Revised: HorNet for semantic segmentation.

Original: We evaluate our HorNet for semantic segmentation task on
Revised: We evaluate our HorNet for semantic segmentation task on

Original: ADE20K [71] dataset using the commonly used UperNet [62] framework.
Revised: ADE20K [71] dataset using the commonly used UperNet [62] framework.

Original: All the models are trained for 160k iterations using AdamW [44] optimizer with a global batch size of 16.
Revised: All the models are trained for 160k iterations using AdamW [44] optimizer with a global batch size of 16.

Original: The image size during training is 512 × 512 for ImagNet-1k (HorNet-T/S/B) pre-trained models and 640 × 640 for the
Revised: The image size during training is 512 × 512 for ImagNet-1k (HorNet-T/S/B) pre-trained models and 640 × 640 for the

Original: ImageNet-22K pre-trained models (HorNet-L).
Revised: ImageNet-22K pre-trained models (HorNet-L).

Original: The results are summarized in the left part of Table 2, where we report both the single-scale (SS) and multi-scale (MS) mIoU on the validation set.
Revised: The results are summarized in the left part of Table 2, where we report both the single-scale (SS) and multi-scale (MS) mIoU on the validation set.

Original: Both our
Revised: Both our

Original: HorNet 7 × 7 and HorNet GF models outperform Swin [42] and ConvNeXt [43] models with similar model sizes and FLOPs.
Revised: HorNet 7 × 7 and HorNet GF models outperform Swin [42] and ConvNeXt [43] models with similar model sizes and FLOPs.

Original: Specifically, HorNet GF models achieve better results than HorNet 7 × 7 and
Revised: Specifically, HorNet GF models achieve better results than HorNet 7 × 7 and

Original: ConvNeXt series by large margins in single-scale mIoU, indicating the global interactions captured by the global filter are helpful for semantic segmentation.
Revised: ConvNeXt series by large margins in single-scale mIoU, indicating the global interactions captured by the global filter are helpful for semantic segmentation.

Original: Notably, we find both our HorNet-L 7 ×and HorNet-L GF even outperform ConvNeXt-XL with ∼ 25% fewer FLOPs.
Revised: Notably, we find both our HorNet-L 7 ×and HorNet-L GF even outperform ConvNeXt-XL with ∼ 25% fewer FLOPs.

Original: These results clearly demonstrate the effectiveness and scalability of our HorNet on semantic segmentation.
Revised: These results clearly demonstrate the effectiveness and scalability of our HorNet on semantic segmentation.

Original: HorNet for object detection.
Revised: HorNet for object detection.

Original: We also evaluate our models on the COCO [38] dataset.
Revised: We also evaluate our models on the COCO [38] dataset.

Original: We adopt the cascade Mask R-CNN framework [21, 2] to perform object detection and instance segmentation using HorNet-T/S/B/L backbones.
Revised: We adopt the cascade Mask R-CNN framework [21, 2] to perform object detection and instance segmentation using HorNet-T/S/B/L backbones.

Original: Following Swin [42] and ConvNeXt [43], we use 3 × schedule with multi-scale training.
Revised: Following Swin [42] and ConvNeXt [43], we use 3 × schedule with multi-scale training.

Original: The right part of Table 2 compares the box AP and mask AP of our HorNet models and Swin/ConvNeXt models.
Revised: The right part of Table 2 compares the box AP and mask AP of our HorNet models and Swin/ConvNeXt models.

Original: Similarly, we show our HorNet models achieve consistently and significantly better performance than the Swin/ConvNeXt counterparts, in both box AP and mask
Revised: Similarly, we show our HorNet models achieve consistently and significantly better performance than the Swin/ConvNeXt counterparts, in both box AP and mask

Original: AP.
Revised: AP.

Original: The HorNet GF series obtain +1.2 ∼ 2.0 box AP and +1.0 ∼ 1.9 mask AP compared with ConvNeXt.
Revised: The HorNet GF series obtain +1.2 ∼ 2.0 box AP and +1.0 ∼ 1.9 mask AP compared with ConvNeXt.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Again
Revised: Again

Original: [Table]
Revised: [Table]

Original: our large model HorNet-L 7 × 7 and HorNet GF can outperform ConvNeXt-XL
Revised: our large model HorNet-L 7 × 7 and HorNet GF can outperform ConvNeXt-XL

Original: [Table]
Revised: [Table]

Original: which further validates the favorable transferability with a larger model size and
Revised: which further validates the favorable transferability with a larger model size and

Original: larger pre-trained dataset
Revised: larger pre-trained dataset

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: HorFPN for dense prediction.
Revised: HorFPN for dense prediction.

Original: We now show another application of the proposed g n Conv, i.e ., to serve as a better fusion module that can better capture the higher-order interactions among different levels of features in dense prediction tasks.
Revised: We now show another application of the proposed g n Conv, i.e ., to serve as a better fusion module that can better capture the higher-order interactions among different levels of features in dense prediction tasks.

Original: Specifically, we directly modify the FPN [37] as described in Section 3.2 in UperNet [62] and Mask R-CNN [21] for semantic segmentation and object detection, respectively.
Revised: Specifically, we directly modify the FPN [37] as described in Section 3.2 in UperNet [62] and Mask R-CNN [21] for semantic segmentation and object detection, respectively.

Original: We show the results in Table 3, where we compare the performance of our HorFPN and standard
Revised: We show the results in Table 3, where we compare the performance of our HorFPN and standard

Original: FPN on different backbones including ResNet-50/101 [22], Swin-S [42] and HorNet-S 7 × 7 .
Revised: FPN on different backbones including ResNet-50/101 [22], Swin-S [42] and HorNet-S 7 × 7 .

Original: For semantic segmentation, we find our HorFPN can significantly reduce the FLOPs ( ∼ 50%) while achieving better validation mIoU. For object detection,
Revised: For semantic segmentation, we find our HorFPN can significantly reduce the FLOPs ( ∼ 50%) while achieving better validation mIoU. For object detection,

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: our HorFPN can also outperform standard FPN in terms of both box AP and mask AP on different backbones with about 30G fewer FLOPs.
Revised: our HorFPN can also outperform standard FPN in terms of both box AP and mask AP on different backbones with about 30G fewer FLOPs.

Original: Besides, we observe that the HorFPN GF is consistently better than HorFPN 7 × 7 , indicating that global interactions are also important when fusing hierarchical features.
Revised: Besides, we observe that the HorFPN GF is consistently better than HorFPN 7 × 7 , indicating that global interactions are also important when fusing hierarchical features.

Original: Results with state-of-the-art frameworks.
Revised: Results with state-of-the-art frameworks.

Original: To further show the effectiveness our backbone, we conduct experiments to combine our large HorNet model with recent state-of-the-art dense prediction frameworks including HTC++ [3], DINO [70] and Mask2Former [7].
Revised: To further show the effectiveness our backbone, we conduct experiments to combine our large HorNet model with recent state-of-the-art dense prediction frameworks including HTC++ [3], DINO [70] and Mask2Former [7].

Original: For HTC++ and DINO, we train our models on COCO for 36 epochs (3 × schedule) and does not introduce extra pre-training data like Object365 in [70].
Revised: For HTC++ and DINO, we train our models on COCO for 36 epochs (3 × schedule) and does not introduce extra pre-training data like Object365 in [70].

Original: We report the single-scale performance on the validation set and compared with several state-of-the-art methods in Table 4.
Revised: We report the single-scale performance on the validation set and compared with several state-of-the-art methods in Table 4.

Original: For Mask2Former, we train our models on ADE20K with 640 × 640 .
Revised: For Mask2Former, we train our models on ADE20K with 640 × 640 .

Original: We report the mIoU of both single-scale and multi-scale testing on the validation set in Table 5.
Revised: We report the mIoU of both single-scale and multi-scale testing on the validation set in Table 5.

Original: 4.3 Analysis
Revised: 4.3 Analysis

Original: Ablation study.
Revised: Ablation study.

Original: We provide detailed ablation studies of the g n Conv and our HorNet in Table 6.
Revised: We provide detailed ablation studies of the g n Conv and our HorNet in Table 6.

Original: We first study the model designs of our HorNet in Table 6a.
Revised: We first study the model designs of our HorNet in Table 6a.

Original: Our baseline ([*]) is obtained by simply replacing the self-attention with 7 × 7 depth-wise convolution in Swin-T [42].
Revised: Our baseline ([*]) is obtained by simply replacing the self-attention with 7 × 7 depth-wise convolution in Swin-T [42].

Original: We first show that both SE [25] and our g n Conv with n = 1 ( g {1,1,1,1} Conv) can improve over the baseline model [*], and g {1,1,1,1} Conv is slightly better.
Revised: We first show that both SE [25] and our g n Conv with n = 1 ( g {1,1,1,1} Conv) can improve over the baseline model [*], and g {1,1,1,1} Conv is slightly better.

Original: We then perform ablations on the interaction order n for each stage and find: (1) if n is shared across the 4 stages, the accuracy will increase with larger n but saturate at 82.5 when n = 4 ; (2) progressively increased order ( g {2,3,4,5} Conv) can further improve the accuracy.
Revised: We then perform ablations on the interaction order n for each stage and find: (1) if n is shared across the 4 stages, the accuracy will increase with larger n but saturate at 82.5 when n = 4 ; (2) progressively increased order ( g {2,3,4,5} Conv) can further improve the accuracy.

Original: Our final models are built on g {2,3,4,5} Conv by adjusting the depth and width of the networks (HorNet-T 7 × 7 ) and applying Global Filter [46] for the depth-wise convolution (HorNet-T GF ).
Revised: Our final models are built on g {2,3,4,5} Conv by adjusting the depth and width of the networks (HorNet-T 7 × 7 ) and applying Global Filter [46] for the depth-wise convolution (HorNet-T GF ).

Original: These results clearly show that our g n Conv is an efficient and extendable operation that can better capture high-order spatial interactions than both self-attention and depth-wise convolution.
Revised: These results clearly show that our g n Conv is an efficient and extendable operation that can better capture high-order spatial interactions than both self-attention and depth-wise convolution.

Original: g n Conv for isotropic models.
Revised: g n Conv for isotropic models.

Original: We also evaluate g n Conv on isotropic architectures (with constant spatial resolutions).
Revised: We also evaluate g n Conv on isotropic architectures (with constant spatial resolutions).

Original: We replace the self-attention in DeiT-S [52] with our g n Conv and adjust the number of blocks to 13 to obtain the isotropic HorNet-S 7 × 7 and HorNet-S GF .
Revised: We replace the self-attention in DeiT-S [52] with our g n Conv and adjust the number of blocks to 13 to obtain the isotropic HorNet-S 7 × 7 and HorNet-S GF .

Original: We compare DeiT-S, isotropic ConvNeXt-S and isotropic HorNet-S in Table 6b.
Revised: We compare DeiT-S, isotropic ConvNeXt-S and isotropic HorNet-S in Table 6b.

Original: While isotropic ConvNeXt-S cannot improve DeiT-S, our isotropic HorNet surpasses DeiT-S by a large margin.
Revised: While isotropic ConvNeXt-S cannot improve DeiT-S, our isotropic HorNet surpasses DeiT-S by a large margin.

Original: These results indicate that our g n Conv can better realize the functions of self-attention compared to plain convolutions and have better ability to model the complex spatial interactions.
Revised: These results indicate that our g n Conv can better realize the functions of self-attention compared to plain convolutions and have better ability to model the complex spatial interactions.

Original: g n Conv for other operations.
Revised: g n Conv for other operations.

Original: To further demonstrate the universality of g n Conv, we use 3 ×depth-wise convolution and 3 × 3 pooling [67] as the basic operation in the g n Conv.
Revised: To further demonstrate the universality of g n Conv, we use 3 ×depth-wise convolution and 3 × 3 pooling [67] as the basic operation in the g n Conv.

Original: The results in
Revised: The results in

Original: Table 6c show that g n Conv can also improve these two operations by large margins, indicating our g n Conv is potentially more powerful when equipped with some better basic operations.
Revised: Table 6c show that g n Conv can also improve these two operations by large margins, indicating our g n Conv is potentially more powerful when equipped with some better basic operations.

Original: Accuracy-complexity trade-offs.
Revised: Accuracy-complexity trade-offs.

Original: We visualize accuracy-complexity trade-offs of Swin, ConvNeXt and HorNet series in Figure 3.
Revised: We visualize accuracy-complexity trade-offs of Swin, ConvNeXt and HorNet series in Figure 3.

Original: For fair comparisons, we fix the input image size to 224 × 224 and use
Revised: For fair comparisons, we fix the input image size to 224 × 224 and use

Original: HorNet 7 × 7 such that all the compared models are based on 7 × 7 local window.
Revised: HorNet 7 × 7 such that all the compared models are based on 7 × 7 local window.

Original: We see HorNet can achieve better trade-offs than the representative vision Transformers and modern CNNs with regards to model size, FLOPs and GPU latency.
Revised: We see HorNet can achieve better trade-offs than the representative vision Transformers and modern CNNs with regards to model size, FLOPs and GPU latency.

Original: Visualization.
Revised: Visualization.

Original: We provide some visualizations of the adaptive weights learned by g n Conv in
Revised: We provide some visualizations of the adaptive weights learned by g n Conv in

Original: Figure 4.
Revised: Figure 4.

Original: For each sample, we show the value of 1 C (cid:80)
Revised: For each sample, we show the value of 1 C (cid:80)

Original: Cc =1 h cij (see Equation (3.8) or the definition of h cij ) for two random spatial locations i from layer {1, 3, 5, 7, 8, 12} of the isotropic HorNet-S model.
Revised: Cc =1 h cij (see Equation (3.8) or the definition of h cij ) for two random spatial locations i from layer {1, 3, 5, 7, 8, 12} of the isotropic HorNet-S model.

Original: Figure 4 demonstrates that the spatial mixing weights of our g n Conv are adaptive both to input samples and spatial locations, which further indicates that g n Conv shares these two desirable characteristics with the self-attention operation.
Revised: Figure 4 demonstrates that the spatial mixing weights of our g n Conv are adaptive both to input samples and spatial locations, which further indicates that g n Conv shares these two desirable characteristics with the self-attention operation.

Original: Limitations.
Revised: Limitations.

Original: While HorNet shows better overall latency-accuracy trade-offs, we notice that HorNet is slower than ConvNeXt with similar FLOPs on GPU, which may be caused by the more complex designs to perform the high-order interactions.
Revised: While HorNet shows better overall latency-accuracy trade-offs, we notice that HorNet is slower than ConvNeXt with similar FLOPs on GPU, which may be caused by the more complex designs to perform the high-order interactions.

Original: We think that developing a more hardware-friendly operation for high-order spatial interactions is an interesting future direction to improve our work.
Revised: We think that developing a more hardware-friendly operation for high-order spatial interactions is an interesting future direction to improve our work.

Original:  Conclusion
Revised:  Conclusion

Original: We have presented the Recursive Gated Convolution ( g n Conv) that performs efficient, extendable, and translation-equivariant high-order spatial interactions with gated convolutions and recursive deigns.
Revised: We have presented the Recursive Gated Convolution ( g n Conv) that performs efficient, extendable, and translation-equivariant high-order spatial interactions with gated convolutions and recursive deigns.

Original: g n Conv can serve as a drop-in replace of the spatial mixing layer in various vision Transformers and convolution-based models.
Revised: g n Conv can serve as a drop-in replace of the spatial mixing layer in various vision Transformers and convolution-based models.

Original: Based on the operation, we have constructed a new family of generic vision backbones HorNet.
Revised: Based on the operation, we have constructed a new family of generic vision backbones HorNet.

Original: Extensive experiments demonstrate the effectiveness of g n Conv and
Revised: Extensive experiments demonstrate the effectiveness of g n Conv and

Original: HorNet on commonly used visual recognition benchmarks.
Revised: HorNet on commonly used visual recognition benchmarks.

Original: We hope our attempt can inspire future work to further explore the high-order spatial interactions in vision models.
Revised: We hope our attempt can inspire future work to further explore the high-order spatial interactions in vision models.

Original: Acknowledgments
Revised: Acknowledgments

Original: Jiwen Lu was supported in part by the National Key Research and Development Program of China under Grant 2017YFA0700802, the National Natural Science Foundation of China under Grant 62125603 and Grant U1813218, and a grant from the Beijing Academy of Artificial Intelligence (BAAI).
Revised: Jiwen Lu was supported in part by the National Key Research and Development Program of China under Grant 2017YFA0700802, the National Natural Science Foundation of China under Grant 62125603 and Grant U1813218, and a grant from the Beijing Academy of Artificial Intelligence (BAAI).

Original: A FLOPs of g n Conv
Revised: A FLOPs of g n Conv

Original: We will divide the computation of our g n Conv into 3 parts, and calculate the FLOPs for each part.
Revised: We will divide the computation of our g n Conv into 3 parts, and calculate the FLOPs for each part.

Original: • Projection layers.
Revised: • Projection layers.

Original: The FLOPs of two projection layers ϕ in and ϕ out can be
Revised: The FLOPs of two projection layers ϕ in and ϕ out can be

Original: easily
Revised: easily

Original: derived
Revised: derived

Original: as:
Revised: as:

Original: [Equation]
Revised: [Equation]

Original: • Depth-wise convolution.
Revised: • Depth-wise convolution.

Original: We first consider the standard depth-wise convolution (DWConv) with kernel size K .
Revised: We first consider the standard depth-wise convolution (DWConv) with kernel size K .

Original: The DWConv is performed for all { q k } n − 1 k
Revised: The DWConv is performed for all { q k } n − 1 k

Original: =
Revised: =

Original: where q k ∈ R HW × C k and
Revised: where q k ∈ R HW × C k and

Original: C k
Revised: C k

Original: [Equation]
Revised: [Equation]

Original: n − k − 1 .
Revised: n − k − 1 .

Original: Therefore, the FLOPs for DWConv are
Revised: Therefore, the FLOPs for DWConv are

Original: [Equation]
Revised: [Equation]

Original: • Recursive Gating.
Revised: • Recursive Gating.

Original: We consider both the flops of the projection layer g k and the elementwise multiplication.
Revised: We consider both the flops of the projection layer g k and the elementwise multiplication.

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: the total FLOPs are:
Revised: the total FLOPs are:

Original: [Equation]
Revised: [Equation]

Original: B Spatial Interactions in Vision Models.
Revised: B Spatial Interactions in Vision Models.

Original: We review some representative vision model designs from the perspective of spatial interactions, as shown in Figure 1.
Revised: We review some representative vision model designs from the perspective of spatial interactions, as shown in Figure 1.

Original: Specifically, we are interested in the interactions between a feature x i and its neighbor feature x j , j ∈ Ω i .
Revised: Specifically, we are interested in the interactions between a feature x i and its neighbor feature x j , j ∈ Ω i .

Original: Inspired by the interaction effect (IE) [33, 1], we consider that a binary function F ( x i , x j ) which directly operates on x i , x j introduces an effective interaction between x i x j , if
Revised: Inspired by the interaction effect (IE) [33, 1], we consider that a binary function F ( x i , x j ) which directly operates on x i , x j introduces an effective interaction between x i x j , if

Original: [Equation]
Revised: [Equation]

Original: We now analyze the cases in Figure 1 of our main paper using the above rule.
Revised: We now analyze the cases in Figure 1 of our main paper using the above rule.

Original: (a): Convolution.
Revised: (a): Convolution.

Original: The output F i =
Revised: The output F i =

Original: (cid:80)
Revised: (cid:80)

Original: j ∈ Ω w i → j x j , which leads to IE( F ) = 0 .
Revised: j ∈ Ω w i → j x j , which leads to IE( F ) = 0 .

Original: Therefore, standard convolution introduce no interaction between x i and x j and we call it a 0-order interaction .
Revised: Therefore, standard convolution introduce no interaction between x i and x j and we call it a 0-order interaction .

Original: (b): SE Block/Gated
Revised: (b): SE Block/Gated

Original: Convolution.
Revised: Convolution.

Original: In this case, we have F i
Revised: In this case, we have F i

Original: =
Revised: =

Original: (cid:80)
Revised: (cid:80)

Original: j ∈ Ω w i → j x j s i ( x ) , where s i ( x ) = 

Revised: j ∈ Ω w i → j x j s i ( x ) , where s i ( x ) =

Original: HW (cid:80) HWl =1 x l for the
Revised: HW (cid:80) HWl =1 x l for the

Original: SE block and s i ( x )
Revised: SE block and s i ( x )

Original: =
Revised: =

Original: x i for the gated convolution.
Revised: x i for the gated convolution.

Original: It is easy to show IE( F ) ̸ = 0 because ∂s i ∂ x i ̸ = 0 .
Revised: It is easy to show IE( F ) ̸ = 0 because ∂s i ∂ x i ̸ = 0 .

Original: Hence, these two operations both introduce 1-order interaction .
Revised: Hence, these two operations both introduce 1-order interaction .

Original: (c): Self-attention (SA).
Revised: (c): Self-attention (SA).

Original: We first denote the projected query/key/value features as q , k , v .
Revised: We first denote the projected query/key/value features as q , k , v .

Original: The SA first perform an 1-order interaction by computing the attention with dot-product: a i =
Revised: The SA first perform an 1-order interaction by computing the attention with dot-product: a i =

Original: q ⊤ i [ k 1 , . .
Revised: q ⊤ i [ k 1 , . .

Original: , k HW ] / √ C .
Revised: , k HW ] / √ C .

Original: We then view a i as the feature at location i in the following computation.
Revised: We then view a i as the feature at location i in the following computation.

Original: The normalized ˆ a i is then obtained by Softmax, which do not contribute to the order since it can be viewed as an implicit interaction that does not explicitly introduce x j to the computation.
Revised: The normalized ˆ a i is then obtained by Softmax, which do not contribute to the order since it can be viewed as an implicit interaction that does not explicitly introduce x j to the computation.

Original: The second interaction is performed by x i = (cid:80)
Revised: The second interaction is performed by x i = (cid:80)

Original: j ∈ Ω ˆ a
Revised: j ∈ Ω ˆ a

Original: i v j .
Revised: i v j .

Original: To sum up, the SA is a 2-order interaction .
Revised: To sum up, the SA is a 2-order interaction .

Original: (d) : g n Conv.
Revised: (d) : g n Conv.

Original: According to Section 3.1, we have already known that g n Conv can achieve n -order interaction with bounded computational cost.
Revised: According to Section 3.1, we have already known that g n Conv can achieve n -order interaction with bounded computational cost.

Original: From the above discussion, we reveal a key difference between ViTs and previous architectures from a new view, i.e ., ViTs have higher-order spatial interactions in each basic block.
Revised: From the above discussion, we reveal a key difference between ViTs and previous architectures from a new view, i.e ., ViTs have higher-order spatial interactions in each basic block.

Original: Then it begs the question that whether we can achieve better accuracy-complexity trade-offs viz interactions with more than 2 orders.
Revised: Then it begs the question that whether we can achieve better accuracy-complexity trade-offs viz interactions with more than 2 orders.

Original: Our proposed g n Conv exactly targets this question for the first time.
Revised: Our proposed g n Conv exactly targets this question for the first time.

Original: First, we can realize arbitrary n -order interaction as long as 1 ≤ n ≤ 1 + log 2 C easily.
Revised: First, we can realize arbitrary n -order interaction as long as 1 ≤ n ≤ 1 + log 2 C easily.

Original: Second, unlike the
Revised: Second, unlike the

Original: [Table]
Revised: [Table]

Original: quadratic complexity of self-attention, the computational cost of g n Conv has an upper bound w.r.t.
Revised: quadratic complexity of self-attention, the computational cost of g n Conv has an upper bound w.r.t.

Original: the order n .
Revised: the order n .

Original: In our implementation of g n Conv, the higher-order spatial interactions are based on the gating mechanism, which has also been investigated in LSTM [23] and some vision modules [59, 54, 39].
Revised: In our implementation of g n Conv, the higher-order spatial interactions are based on the gating mechanism, which has also been investigated in LSTM [23] and some vision modules [59, 54, 39].

Original: However, these previous methods can only achieve up to 2-order interactions, and did not fully reveal the potential of higher-order interactions.
Revised: However, these previous methods can only achieve up to 2-order interactions, and did not fully reveal the potential of higher-order interactions.

Original: On the contrary, our g n Conv is more extendable to achieve arbitrary higher-order spatial interactions under a controllable computational budget.
Revised: On the contrary, our g n Conv is more extendable to achieve arbitrary higher-order spatial interactions under a controllable computational budget.

Original: C Implementation Details C.1 Architecture Details.
Revised: C Implementation Details C.1 Architecture Details.

Original: To better verify the effectiveness of our new designs, we introduce minimal changes in the overall architecture of Swin Transformers [42].
Revised: To better verify the effectiveness of our new designs, we introduce minimal changes in the overall architecture of Swin Transformers [42].

Original: Specifically, we make two changes to the overall architecture of Swin Transformers [42]: 1) We add one block in stage 2 to make the overall computation and parameters close to previous models; 2)
Revised: Specifically, we make two changes to the overall architecture of Swin Transformers [42]: 1) We add one block in stage 2 to make the overall computation and parameters close to previous models; 2)

Original: We use the LayerScale [53] techniques to make our models more stable during training following the practice of ConvNeXt [43].
Revised: We use the LayerScale [53] techniques to make our models more stable during training following the practice of ConvNeXt [43].

Original: Note that the two changes have been applied to the baseline model considered in our ablation study to clearly show the effects of our designs.
Revised: Note that the two changes have been applied to the baseline model considered in our ablation study to clearly show the effects of our designs.

Original: The detailed architectures of ConvNeXt [43], Swin Transformers [42] and HorNet are summarized in Table 7.
Revised: The detailed architectures of ConvNeXt [43], Swin Transformers [42] and HorNet are summarized in Table 7.

Original: C.2 Experimental Settings for Image Classification.
Revised: C.2 Experimental Settings for Image Classification.

Original: ImageNet-1K training.
Revised: ImageNet-1K training.

Original: ImageNet-1K [13] is a widely used large-scale benchmark for image classification, which contains around 1.2 million images from 1,000 categories.
Revised: ImageNet-1K [13] is a widely used large-scale benchmark for image classification, which contains around 1.2 million images from 1,000 categories.

Original: Following common practice [22, 43], we train our models on the training set of ImageNet and report the single-crop top-1 accuracy on 50,000 validation images.
Revised: Following common practice [22, 43], we train our models on the training set of ImageNet and report the single-crop top-1 accuracy on 50,000 validation images.

Original: To fairly compare with our baseline methods ( i.e .,
Revised: To fairly compare with our baseline methods ( i.e .,

Original: Swin Transformers [42] and ConvNeXt [43]), we follow the most training details of ConvNeXt and make several small modifications to make the training configurations suitable for our models.
Revised: Swin Transformers [42] and ConvNeXt [43]), we follow the most training details of ConvNeXt and make several small modifications to make the training configurations suitable for our models.

Original: For
Revised: For

Original: HorNet with 7 × 7 convolutions, we find that applying gradient clipping with a maximal norm ofwill significantly stabilize the training process, which may be due to the large gradients brought by the high-order structures in our models.
Revised: HorNet with 7 × 7 convolutions, we find that applying gradient clipping with a maximal norm ofwill significantly stabilize the training process, which may be due to the large gradients brought by the high-order structures in our models.

Original: For HorNet with global filters, we use stronger regularization strategies since we find that larger kernels will improve the model capacity but may also cause more severe overfitting.
Revised: For HorNet with global filters, we use stronger regularization strategies since we find that larger kernels will improve the model capacity but may also cause more severe overfitting.

Original: Specifically, we set the gradient norm to 1 and use more aggressive RandAug [10] data augmentation strategies ( i.e ., we adjust the magnitudes for tiny, small and base models to 9,and 15, respectively).
Revised: Specifically, we set the gradient norm to 1 and use more aggressive RandAug [10] data augmentation strategies ( i.e ., we adjust the magnitudes for tiny, small and base models to 9,and 15, respectively).

Original: We set the stochastic depth coefficient of HorNet-T/S/B models to 0.2, 0.4 and 0.5.
Revised: We set the stochastic depth coefficient of HorNet-T/S/B models to 0.2, 0.4 and 0.5.

Original: The other details are identical to ConvNeXt [43].
Revised: The other details are identical to ConvNeXt [43].

Original: Our models are trained using 32 NVIDIA
Revised: Our models are trained using 32 NVIDIA

Original: A100 GPUs with a global batch size of 4096.
Revised: A100 GPUs with a global batch size of 4096.

Original: [Table]
Revised: [Table]

Original: MViTv
Revised: MViTv

Original: [Table]
Revised: [Table]

Original: ImageNet-22K training.
Revised: ImageNet-22K training.

Original: ImageNet-22K [13] is a larger dataset that contains > 21k classes and around 14M images.
Revised: ImageNet-22K [13] is a larger dataset that contains > 21k classes and around 14M images.

Original: We use the subset suggested by [47] since the new winter 2021 release is the accessible version now.
Revised: We use the subset suggested by [47] since the new winter 2021 release is the accessible version now.

Original: We also follow the [47] to remove categories with few images, resulting in roughly half fewer categories and only 13% fewer images compared to the original dataset.
Revised: We also follow the [47] to remove categories with few images, resulting in roughly half fewer categories and only 13% fewer images compared to the original dataset.

Original: We follow previous practice [42, 43] to train our models for 90 epochs and use a similar data augmentation strategy as ImageNet-1K experiments.
Revised: We follow previous practice [42, 43] to train our models for 90 epochs and use a similar data augmentation strategy as ImageNet-1K experiments.

Original: We set the stochastic depth coefficient [26] to 0.2.
Revised: We set the stochastic depth coefficient [26] to 0.2.

Original: We also set the maximal gradient norm to 5 and 1 for our large models with standard 7 × 7 convolutions and global filters respectively.
Revised: We also set the maximal gradient norm to 5 and 1 for our large models with standard 7 × 7 convolutions and global filters respectively.

Original: We also adjust the weight decay to 0.1.
Revised: We also adjust the weight decay to 0.1.

Original: The other details are identical to
Revised: The other details are identical to

Original: ConvNeXt [43].
Revised: ConvNeXt [43].

Original: We also fine-tune our best model HorNet-L GF on 384 × 384 images on ImageNet22K for 10 epochs compete with state-of-the-art models on downstream tasks.
Revised: We also fine-tune our best model HorNet-L GF on 384 × 384 images on ImageNet22K for 10 epochs compete with state-of-the-art models on downstream tasks.

Original: The model is only used in the experiments in Appendix 4.2.
Revised: The model is only used in the experiments in Appendix 4.2.

Original: ImageNet-1K fine-tuning.
Revised: ImageNet-1K fine-tuning.

Original: We fine-tune the models pre-trained on ImageNet-22K or at the 224 ×resolution to ImageNet-1K or/and 384 × 384 resolution for 30 epochs with a batch size of 512 and a cosine learning rate schedule with an initial learning rate of 5 e − 5 .
Revised: We fine-tune the models pre-trained on ImageNet-22K or at the 224 ×resolution to ImageNet-1K or/and 384 × 384 resolution for 30 epochs with a batch size of 512 and a cosine learning rate schedule with an initial learning rate of 5 e − 5 .

Original: We set the weight decay to 1 e −and disable MixUp and CutMix following [43].
Revised: We set the weight decay to 1 e −and disable MixUp and CutMix following [43].

Original: We initialize the ImageNet-1K classifier with the corresponding classifier weights for ImageNet-22K classes to further stabilize the training process.
Revised: We initialize the ImageNet-1K classifier with the corresponding classifier weights for ImageNet-22K classes to further stabilize the training process.

Original: C.3 Experimental
Revised: C.3 Experimental

Original: Settings for Downstream
Revised: Settings for Downstream

Original: Tasks.
Revised: Tasks.

Original: Object detection and instance segmentation on COCO.
Revised: Object detection and instance segmentation on COCO.

Original: We adopt the widely used Cascade Mask
Revised: We adopt the widely used Cascade Mask

Original: R-CNN [2] framework to perform object detection and instance segmentation on COCO, following
Revised: R-CNN [2] framework to perform object detection and instance segmentation on COCO, following

Original: Swin [42] and ConvNeXt [43].
Revised: Swin [42] and ConvNeXt [43].

Original: Our backbones are pre-trained on ImageNet-1K for the HorNet-T/S/B and ImageNet-22K for the HorNet-L.
Revised: Our backbones are pre-trained on ImageNet-1K for the HorNet-T/S/B and ImageNet-22K for the HorNet-L.

Original: We use the 3 × schedule where we train all of our model for 36 epochs with AdamW [44] optimizer and a global batch size of 16.
Revised: We use the 3 × schedule where we train all of our model for 36 epochs with AdamW [44] optimizer and a global batch size of 16.

Original: We set the learning rate of as {2e-4, 2e-4, 2e-4, 1e-4} and the stochastic depth rate as {0.4, 0.6, 0.7, 0.7}for HorNet-T/S/B/L.
Revised: We set the learning rate of as {2e-4, 2e-4, 2e-4, 1e-4} and the stochastic depth rate as {0.4, 0.6, 0.7, 0.7}for HorNet-T/S/B/L.

Original: We set the weight decay as 0.05 for all the models.
Revised: We set the weight decay as 0.05 for all the models.

Original: Semantic Segmentation on ADE20K.
Revised: Semantic Segmentation on ADE20K.

Original: We use the UperNet 160K [62] framework for semantic segmentation on ADE20K.
Revised: We use the UperNet 160K [62] framework for semantic segmentation on ADE20K.

Original: We use a global batch size of 16 and train all the models for 160 iterations with the AdamW [44] optimizer.
Revised: We use a global batch size of 16 and train all the models for 160 iterations with the AdamW [44] optimizer.

Original: We use 512 × 512 image for ImageNet-1K pre-trained HorNet-T/S/B and 640 × 640 image for ImagNet-22K pre-trained HorNet-L.
Revised: We use 512 × 512 image for ImageNet-1K pre-trained HorNet-T/S/B and 640 × 640 image for ImagNet-22K pre-trained HorNet-L.

Original: We set the learning rate as 1e-4 and the weight decay as 0.05 for all the models.
Revised: We set the learning rate as 1e-4 and the weight decay as 0.05 for all the models.

Original: We report the mIoU of both single-scale and multi-scale testing on the validation set.
Revised: We report the mIoU of both single-scale and multi-scale testing on the validation set.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: D More Analysis and Visualization
Revised: D More Analysis and Visualization

Original: Comparisons with state-of-the-art methods on ImageNet.
Revised: Comparisons with state-of-the-art methods on ImageNet.

Original: Our experiments are designed to clearly verify the superior of our design over previous basic operations like plain convolution and self-attention.
Revised: Our experiments are designed to clearly verify the superior of our design over previous basic operations like plain convolution and self-attention.

Original: Therefore, we choose to follow the basic architecture and the training configuration of widely used architectures Swin Transformers and ConvNeXt
Revised: Therefore, we choose to follow the basic architecture and the training configuration of widely used architectures Swin Transformers and ConvNeXt

Original: Therefore, there is still substantial room to further improve the performance on ImageNet-1K.
Revised: Therefore, there is still substantial room to further improve the performance on ImageNet-1K.

Original: We notice that some recent work like
Revised: We notice that some recent work like

Original: Pale Transformers [61] and Dynamic Group Transformer [40] with hybrid architectures or more careful designs achieve better performance than HorNet on ImageNet-1K.
Revised: Pale Transformers [61] and Dynamic Group Transformer [40] with hybrid architectures or more careful designs achieve better performance than HorNet on ImageNet-1K.

Original: We think many techniques that have been used in previous work can be useful to further imporve our models, including further optimized overall architectures ( e.g ., optimized depth/width for each stage), better patch embedding strategies ( e.g ., overlapping convolutional layers for input embedding and downsampling), more efficient ways to compute adaptive weights ( e.g ., using downsmapled features to produce attention weights like [35]), and more advanced training methods and hybrid architectures ( e.g ., combining g n Conv with self-attention and plain convolutions).
Revised: We think many techniques that have been used in previous work can be useful to further imporve our models, including further optimized overall architectures ( e.g ., optimized depth/width for each stage), better patch embedding strategies ( e.g ., overlapping convolutional layers for input embedding and downsampling), more efficient ways to compute adaptive weights ( e.g ., using downsmapled features to produce attention weights like [35]), and more advanced training methods and hybrid architectures ( e.g ., combining g n Conv with self-attention and plain convolutions).

Original: Throughput analysis.
Revised: Throughput analysis.

Original: We provide the detailed throughput statistics of our models and several baseline methods in Table 8.
Revised: We provide the detailed throughput statistics of our models and several baseline methods in Table 8.

Original: Apart from ConvNeXt and Swin Transformers, we also compare our model with recent MViTv2 [35] models.
Revised: Apart from ConvNeXt and Swin Transformers, we also compare our model with recent MViTv2 [35] models.

Original: The multiple small matrix multiplications introduced by g n Conv will affect the speed of our models on GPU.
Revised: The multiple small matrix multiplications introduced by g n Conv will affect the speed of our models on GPU.

Original: We observed that our method is slower than
Revised: We observed that our method is slower than

Original: ConvNeXt by 7% 15% with similar FLOPs.
Revised: ConvNeXt by 7% 15% with similar FLOPs.

Original: Meanwhile, thanks to the highly efficient depth-wise convolutions implementation of CuDNN, we also see that our models achieve similar or slightly faster speed than typical vision Transformers with similar FLOPs.
Revised: Meanwhile, thanks to the highly efficient depth-wise convolutions implementation of CuDNN, we also see that our models achieve similar or slightly faster speed than typical vision Transformers with similar FLOPs.

Original: Notably, as shown in Figure 3(c), the higher classification accuracy helps our models achieve better speed-accuracy trade-offs than
Revised: Notably, as shown in Figure 3(c), the higher classification accuracy helps our models achieve better speed-accuracy trade-offs than

Original: ConvNeXt and Swin Transformers.
Revised: ConvNeXt and Swin Transformers.

Original: Therefore, we believe the speed of our method is still competitive with these recent models.
Revised: Therefore, we believe the speed of our method is still competitive with these recent models.

Original: Effects of α .
Revised: Effects of α .

Original: We find that re-scaling the output of gated convolution will avoid the large values produced by the recursive process and stabilize the training process.
Revised: We find that re-scaling the output of gated convolution will avoid the large values produced by the recursive process and stabilize the training process.

Original: We analyze the the effects of α on our ImageNet experiments based on HorNet-B 7 × 7 .
Revised: We analyze the the effects of α on our ImageNet experiments based on HorNet-B 7 × 7 .

Original: The results are summarized in Table 9.
Revised: The results are summarized in Table 9.

Original: We see α = 3 leads the best performance.
Revised: We see α = 3 leads the best performance.

Original: Therefore, we set α to 3 in all our models.
Revised: Therefore, we set α to 3 in all our models.

Original: Effects of activation functions in gated convolutions.
Revised: Effects of activation functions in gated convolutions.

Original: The gated convolutions used in our models can be viewed as a type of channel attention that uses different attention weights in different locations and generates weights based on spatial interactions.
Revised: The gated convolutions used in our models can be viewed as a type of channel attention that uses different attention weights in different locations and generates weights based on spatial interactions.

Original: Previous channel attention methods like SENet [25] usually add a sigmoid function to the attention weights to generate bounded attention.
Revised: Previous channel attention methods like SENet [25] usually add a sigmoid function to the attention weights to generate bounded attention.

Original: Therefore, we investigate several possible activation functions in our models.
Revised: Therefore, we investigate several possible activation functions in our models.

Original: The results are presented in Table 10.
Revised: The results are presented in Table 10.

Original: We see the version described in our paper ( i.e ., no activation function) achieves the best performance.
Revised: We see the version described in our paper ( i.e ., no activation function) achieves the best performance.

Original: The result also suggests that g n Conv exhibits a different behavior from conventional channel attention methods.
Revised: The result also suggests that g n Conv exhibits a different behavior from conventional channel attention methods.

Original: Since the gating weights are critical components in our models, activation functions that can cause significant information losses like GELU and sigmoid will severely hurt the performance.
Revised: Since the gating weights are critical components in our models, activation functions that can cause significant information losses like GELU and sigmoid will severely hurt the performance.