Original: SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation
Revised: SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation

Original: Abstract
Revised: Abstract

Original: We present SegNeXt, a simple convolutional network architecture for semantic segmentation.
Revised: We present SegNeXt, a simple convolutional network architecture for semantic segmentation.

Original: Recent transformer-based models have dominated the field of semantic segmentation due to the efficiency of self-attention in encoding spatial information.
Revised: Recent transformer-based models have dominated the field of semantic segmentation due to the efficiency of self-attention in encoding spatial information.

Original: In this paper, we show that convolutional attention is a more efficient and effective way to encode contextual information than the self-attention mechanism in transformers.
Revised: In this paper, we show that convolutional attention is a more efficient and effective way to encode contextual information than the self-attention mechanism in transformers.

Original: By re-examining the characteristics owned by successful segmentation models, we discover several key components leading to the performance improvement of segmentation models.
Revised: By re-examining the characteristics owned by successful segmentation models, we discover several key components leading to the performance improvement of segmentation models.

Original: This motivates us to design a novel convolutional attention network that uses cheap convolutional operations.
Revised: This motivates us to design a novel convolutional attention network that uses cheap convolutional operations.

Original: Without bells and whistles, our SegNeXt significantly improves the performance of previous state-of-the-art methods on popular benchmarks, including ADE20K, Cityscapes,
Revised: Without bells and whistles, our SegNeXt significantly improves the performance of previous state-of-the-art methods on popular benchmarks, including ADE20K, Cityscapes,

Original: COCO-Stuff, Pascal VOC, Pascal Context, and iSAID.
Revised: COCO-Stuff, Pascal VOC, Pascal Context, and iSAID.

Original: Notably, SegNeXt outperforms EfficientNet-L2 w/ NAS-FPN and achieves 90 .
Revised: Notably, SegNeXt outperforms EfficientNet-L2 w/ NAS-FPN and achieves 90 .

Original: 6% mIoU on the Pascal
Revised: 6% mIoU on the Pascal

Original: VOC 2012 test leaderboard using only 1 / 10 parameters of it.
Revised: VOC 2012 test leaderboard using only 1 / 10 parameters of it.

Original: On average, SegNeXt achieves about 2 . 0% mIoU improvements compared to the state-of-the-art methods on the ADE20K datasets with the same or fewer computations.
Revised: On average, SegNeXt achieves about 2 . 0% mIoU improvements compared to the state-of-the-art methods on the ADE20K datasets with the same or fewer computations.

Original:  Introduction
Revised:  Introduction

Original: As one of the most fundamental research topics in computer vision, semantic segmentation, which aims at assigning each pixel a semantic category, has attracted great attention over the past decade.
Revised: As one of the most fundamental research topics in computer vision, semantic segmentation, which aims at assigning each pixel a semantic category, has attracted great attention over the past decade.

Original: From early CNN-based models, typified by FCN [52] and DeepLab series [4, 5, 7], to recent transformer-based methods, represented by SETR [96] and SegFormer [79], semantic segmentation models have experienced significant revolution in terms of network architectures.
Revised: From early CNN-based models, typified by FCN [60] and DeepLab series [4, 5, 7], to recent transformer-based methods, represented by SETR [108] and SegFormer [90] semantic segmentation models have experienced significant revolution in terms of network architectures.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: The size of the circle indicates the number of parameters.
Revised: The size of the circle indicates the number of parameters.

Original: Larger circles mean more parameters.
Revised: Larger circles mean more parameters.

Original: We can see that our SegNeXt achieves the best trade-off between segmentation performance and computational complexity.
Revised: We can see that our SegNeXt achieves the best trade-off between segmentation performance and computational complexity.

Original: By revisiting previous successful semantic segmentation works, we summarize several key properties different models possess as shown in Tab. 1.
Revised: By revisiting previous successful semantic segmentation works, we summarize several key properties different models possess as shown in Tab. 1.

Original: Based on the above observation, we argue a successful semantic segmentation model should have the following characteristics:
Revised: Based on the above observation, we argue a successful semantic segmentation model should have the following characteristics:

Original: (i)
Revised: (i)

Original: A strong backbone network as encoder.
Revised: A strong backbone network as encoder.

Original: Compared to previous CNN-based models, the performance improvement of transformer-based models is mostly from a stronger backbone network.
Revised: Compared to previous CNN-based models, the performance improvement of transformer-based models is mostly from a stronger backbone network.

Original: (ii) Multi-scale information interaction.
Revised: (ii) Multi-scale information interaction.

Original: Different from the image classification task that mostly identifies a single object, semantic segmentation is a dense prediction task and hence needs to process objects of varying sizes in a single image.
Revised: Different from the image classification task that mostly identifies a single object, semantic segmentation is a dense prediction task and hence needs to process objects of varying sizes in a single image.

Original: (iii) Spatial attention.
Revised: (iii) Spatial attention.

Original: Spatial attention allows models to perform segmentation through prioritization of areas within the semantic regions.
Revised: Spatial attention allows models to perform segmentation through prioritization of areas within the semantic regions.

Original: (iv) Low computational complexity.
Revised: (iv) Low computational complexity.

Original: This is especially crucial when dealing with high-resolution images from remote sensing and urban scenes.
Revised: This is especially crucial when dealing with high-resolution images from remote sensing and urban scenes.

Original: Taking the aforementioned analysis into account, in this paper, we rethink the design of convolutional attention and propose an efficient yet effective architecture for semantic segmentation.
Revised: Taking the aforementioned analysis into account, in this paper, we rethink the design of convolutional attention and propose an efficient yet effective architecture for semantic segmentation.

Original: Unlike previous transformer-based models that use convolutions in decoders as feature refiners, our method inverts the transformer-convolution encoder-decoder architecture.
Revised: Unlike previous transformer-based models that use convolutions in decoders as feature refiners, our method inverts the transformer-convolution encoder-decoder architecture.

Original: Specifically, for each block in our encoder, we renovate the design of conventional convolutional blocks and utilize multi-scale convolutional features to evoke spatial attention via a simple element-wise multiplication following [23].
Revised: Specifically, for each block in our encoder, we renovate the design of conventional convolutional blocks and utilize multi-scale convolutional features to evoke spatial attention via a simple element-wise multiplication following [25].

Original: We found such a simple way to build spatial attention is more efficient than both the standard convolutions and self-attention in spatial information encoding.
Revised: We found such a simple way to build spatial attention is more efficient than both the standard convolutions and self-attention in spatial information encoding.

Original: For decoder, we collect multi-level features from different stages and use Hamburger [20] to further extract global context.
Revised: For decoder, we collect multi-level features from different stages and use Hamburger [22] to further extract global context.

Original: Under this setting, our method can obtain multi-scale context from local to global, achieve adaptability in spatial and channel dimensions, and aggregate information from low to high levels.
Revised: Under this setting, our method can obtain multi-scale context from local to global, achieve adaptability in spatial and channel dimensions, and aggregate information from low to high levels.

Original: Our network, termed SegNeXt, is mostly composed of convolutional operations except the decoder part, which contains a decomposition-based Hamburger module [20] (Ham) for global information extraction.
Revised: Our network, termed SegNeXt, is mostly composed of convolutional operations except the decoder part, which contains a decomposition-based Hamburger module [22] (Ham) for global information extraction.

Original: This makes our SegNeXt much more efficient than previous segmentation methods that heavily rely on transformers.
Revised: This makes our SegNeXt much more efficient than previous segmentation methods that heavily rely on transformers.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, SegNeXt outperforms recent transformer-based methods significantly.
Revised: 1, SegNeXt outperforms recent transformer-based methods significantly.

Original: In particular, our SegNeXt-S outperforms SegFormer-B2 (81.3% vs. 81.0%) using only about 1 / 6 (124.6G vs. 717.1G) computational cost and 1 / 2 parameters (13.9M vs. 27.6M) when dealing with high-resolution urban scenes from the Cityscapes dataset.
Revised: In particular, our SegNeXt-S outperforms SegFormer-B2 (81.3% vs. 81.0%) using only about 1 / 6 (124.6G vs. 717.1G) computational cost and 1 / 2 parameters (13.9M vs. 27.6M) when dealing with high-resolution urban scenes from the Cityscapes dataset.

Original: Our contributions can be summarized as follows:
Revised: Our contributions can be summarized as follows:

Original: • We identify the characteristics that a good semantic segmentation model should own and present a novel tailored network architecture, termed SegNeXt, that evokes spatial attention via multi-scale convolutional features.
Revised: • We identify the characteristics that a good semantic segmentation model should own and present a novel tailored network architecture, termed SegNeXt, that evokes spatial attention via multi-scale convolutional features.

Original: • We show that an encoder with simple and cheap convolutions can still perform better than vision transformers, especially when processing object details, while it requires much less computational cost.
Revised: • We show that an encoder with simple and cheap convolutions can still perform better than vision transformers, especially when processing object details, while it requires much less computational cost.

Original: • Our method improves the performance of state-of-the-art semantic segmentation methods by a large margin on various segmentation benchmarks, including ADE20K, Cityscapes, COCO-Stuff, Pascal VOC, Pascal Context, and iSAID.
Revised: • Our method improves the performance of state-of-the-art semantic segmentation methods by a large margin on various segmentation benchmarks, including ADE20K, Cityscapes, COCO-Stuff, Pascal VOC, Pascal Context, and iSAID.

Original:  Related Work 2.1 Semantic Segmentation
Revised:  Related Work 2.1 Semantic Segmentation

Original: Semantic segmentation is a fundamental computer vision task.
Revised: Semantic segmentation is a fundamental computer vision task.

Original: Since FCN [52] was proposed, convolutional neural networks (CNNs) [1, 63, 86, 94, 18, 87, 70, 19, 44] have achieved great success and become a popular architecture for semantic segmentation.
Revised: Since FCN [60] was proposed, convolutional neural networks (CNNs) [1, 71, 98, 106, 20, 99, 79, 21, 51] have achieved great success and become a popular architecture for semantic segmentation.

Original: Recently, transformer-based methods [96, 79, 88, 64, 62, 43, 10, 9] have shown great potentials and outperform CNN-based methods.
Revised: Recently, transformer-based methods [108, 90, 100, 73, 70, 50, 10, 9] have shown great potentials and outperform CNN-based methods.

Original: In the era of deep learning, the architecture of segmentation models can be roughly divided into two parts: encoder and decoder.
Revised: In the era of deep learning, the architecture of segmentation models can be roughly divided into two parts: encoder and decoder.

Original: For the encoder, researchers usually adopt popular classification networks ( e.g., ResNet [26], ResNeXt [80] and DenseNet [31]) instead of tailored architecture.
Revised: For the encoder, researchers usually adopt popular classification networks ( e.g., ResNet [28], ResNeXt [91] and DenseNet [33]) instead of tailored architecture.

Original: However, semantic segmentation is a kind of dense prediction task, which is different from image classification.
Revised: However, semantic segmentation is a kind of dense prediction task, which is different from image classification.

Original: The improvement in classification may not appear in the challenging segmentation task [27].
Revised: The improvement in classification may not appear in the challenging segmentation task [29].

Original: Thus, some tailored encoders appear, including Res2Net [19], HRNet [70], SETR [96],
Revised: Thus, some tailored encoders appear, including Res2Net [21], HRNet [79], SETR [108],

Original: SegFormer [79], HRFormer [88], MPViT [37], DPT [62], etc .
Revised: SegFormer [90], HRFormer [100], MPViT [44], DPT [70], etc .

Original: For the decoder, it is often used in cooperating with encoders to achieve better results.
Revised: For the decoder, it is often used in cooperating with encoders to achieve better results.

Original: There are different types of decoders for different goals, including achieving multi-scale receptive fields [94, 6, 77], collecting multi-scale semantics [63, 79, 7], enlarging receptive field [4, 4, 61], strengthening edge features [95, 2, 15, 41, 90], and capturing global context [18, 33, 89, 39, 22, 25, 91].
Revised: There are different types of decoders for different goals, including achieving multi-scale receptive fields [106, 6, 88], collecting multi-scale semantics [71, 90, 7], enlarging receptive field [4, 4, 69], strengthening edge features [107, 2, 15, 48, 102], and capturing global context [20, 35, 101, 46, 24, 27, 103].

Original: In this paper, we summarize the characteristics of those successful models designed for semantic segmentation and present a CNN-based model, named SegNeXt.
Revised: In this paper, we summarize the characteristics of those successful models designed for semantic segmentation and present a CNN-based model, named SegNeXt.

Original: The most related work to our paper, is [61], which decomposes a k × k convolution into a pair of k × 1 and 1 × k convolutions.
Revised: The most related work to our paper, is [69], which decomposes a k × k convolution into a pair of k × 1 and 1 × k convolutions.

Original: Though this work has shown large convolutional kernels matter in semantic segmentation, it ignores the importance of multi-scale receptive field and does not consider how to leverage these multi-scale features extracted by large kernels for segmentation in the form of attention.
Revised: Though this work has shown large convolutional kernels matter in semantic segmentation, it ignores the importance of multi-scale receptive field and does not consider how to leverage these multi-scale features extracted by large kernels for segmentation in the form of attention.

Original: 2.2 Multi-Scale Networks
Revised: 2.2 Multi-Scale Networks

Original: Designing multi-scale network is one of the popular directions in computer vision.
Revised: Designing multi-scale network is one of the popular directions in computer vision.

Original: For segmentation models, multi-scale blocks appear in both the encoder [70, 19, 66] and the decoder [94, 86, 5] parts.
Revised: For segmentation models, multi-scale blocks appear in both the encoder [79, 21, 75] and the decoder [106, 98, parts.

Original: GoogleNet [66] is one of the most related multi-scale architectures to our method, which uses a multi-branch structure to achieve multi-scale feature extraction.
Revised: GoogleNet [75] is one of the most related multi-scale architectures to our method, which uses a multi-branch structure to achieve multi-scale feature extraction.

Original: Another work that is related to our method is HRNet [70].
Revised: Another work that is related to our method is HRNet [79].

Original: In the deeper stages, HRNet also keeps high-resolution features, which are aggregated with low-resolution features, to enable multi-scale feature extraction.
Revised: In the deeper stages, HRNet also keeps high-resolution features, which are aggregated with low-resolution features, to enable multi-scale feature extraction.

Original: Different from previous methods, SegNeXt, besides capturing multi-scale features in encoder, introduces an efficient attention mechanism and employs cheaper and larger kernel convolutions.
Revised: Different from previous methods, SegNeXt, besides capturing multi-scale features in encoder, introduces an efficient attention mechanism and employs cheaper and larger kernel convolutions.

Original: These enable our model to achieve higher performance than the aforementioned segmentation methods.
Revised: These enable our model to achieve higher performance than the aforementioned segmentation methods.

Original: 2.3 Attention Mechanisms
Revised: 2.3 Attention Mechanisms

Original: Attention mechanism is a kind of adaptive selection process, which aims to make the network focus on the important part.
Revised: Attention mechanism is a kind of adaptive selection process, which aims to make the network focus on the important part.

Original: Generally speaking, it can be divided into two categories in semantic segmentation [24], including channel attention and spatial attention.
Revised: Generally speaking, it can be divided into two categories in semantic segmentation [26], including channel attention and spatial attention.

Original: Different types of attentions play different roles.
Revised: Different types of attentions play different roles.

Original: For instance, spatial attentions mainly care about the important spatial regions [16, 13, 56, 50, 21].
Revised: For instance, spatial attentions mainly care about the important spatial regions [16, 13, 64, 58, 23].

Original: Differently, the goal of using channel attention is to make the network selectively attend to those important objects, which has been demonstrated important in previous works [29, 8, 71].
Revised: Differently, the goal of using channel attention is to make the network selectively attend to those important objects, which has been demonstrated important in previous works [31, 8, 80].

Original: Speaking of the recent popular vision transformers [16, 50, 82, 72, 73, 49, 79, 32, 48, 88, 81], they usually ignore adaptability in channel dimension.
Revised: Speaking of the recent popular vision transformers [16, 58, 94, 81, 82, 57, 90, 34, 56, 100, 93], they usually ignore adaptability in channel dimension.

Original: Visual attention network (VAN) [23] is the most related work to SegNeXt, which also proposes to leverage the large-kernel attention (LKA) mechanism to build both channel and spatial attention.
Revised: Visual attention network (VAN) [25] is the most related work to SegNeXt, which also proposes to leverage the large-kernel attention (LKA) mechanism to build both channel and spatial attention.

Original: Though VAN has achieved great performance in image classification, it neglects the role of multi-scale feature aggregation during the network design, which is crucial for segmentation-like tasks.
Revised: Though VAN has achieved great performance in image classification, it neglects the role of multi-scale feature aggregation during the network design, which is crucial for segmentation-like tasks.

Original: d, 5 xd, 1 xd, 11 x 1 d, 7 xd, 1 xd, 1 xd, 21 x1 x
Revised: [Figure] 1 ×1 ×

Original: BN
Revised: MSCA

Original: Attention BN FFN ×L 1 x
Revised: GELU 1 ×1 × Dconv 3×

Original: GELU
Revised: GELU

Original: MSCA 1 x1 xd, 3 x GELU 1 x
Revised: Dconv 5× d, 1× d, 7× d, 1× d, 11× d, 1× d, 21× 1 × Multi-scale Features

Original: Channel Mixing
Revised: Channel Mixing

Original: Multi-scale Feature

Original: Convolutional Attention
Revised: Convolutional Attention

Original: [Figure]
Revised: [Figure]

Original:  Method
Revised:  Method

Original: In this section, we describe the architecture of the proposed SegNeXt in detail.
Revised: In this section, we describe the architecture of the proposed SegNeXt in detail.

Original: Basically, we adopt an encoder-decoder architecture following most previous works, which is simple and easy to follow.
Revised: Basically, we adopt an encoder-decoder architecture following most previous works, which is simple and easy to follow.

Original: 3.1 Convolutional Encoder
Revised: 3.1 Convolutional Encoder

Original: We adopt the pyramid structure for our encoder following most previous work [79, 4, 18].
Revised: We adopt the pyramid structure for our encoder following most previous work [90, 4, 20].

Original: For the building block in our encoder, we adopt a similar structure to that of ViT [16, 79] but what is different is that we do not use the self-attention mechanism but design a novel multi-scale convolutional attention (MSCA) module.
Revised: For the building block in our encoder, we adopt a similar structure to that of ViT [16, 90] but what is different is that we do not use the self-attention mechanism but design a novel multi-scale convolutional attention (MSCA) module.

Original: As depicted in Fig.
Revised: As depicted in Fig.

Original: 2 (a), MSCA contains three parts: a depth-wise convolution to aggregate local information, multi-branch depth-wise strip convolutions to capture multi-scale context, and an 1 × 1 convolution to model relationship between different channels.
Revised: 2 (a), MSCA contains three parts: a depth-wise convolution to aggregate local information, multi-branch depth-wise strip convolutions to capture multi-scale context, and an 1 × 1 convolution to model relationship between different channels.

Original: The output of the 1 × 1 convolution is used as attention weights directly to reweigh the input of MSCA.
Revised: The output of the 1 × 1 convolution is used as attention weights directly to reweigh the input of MSCA.

Original: Mathematically, our MSCA can be written as:
Revised: Mathematically, our MSCA can be written as:

Original: [Equation]
Revised: [Equation]

Original: where F represents the input feature.
Revised: where F represents the input feature.

Original: Att and Out are the attention map and output, respectively.
Revised: Att and Out are the attention map and output, respectively.

Original: ⊗ is the element-wise matrix multiplication operation.
Revised: ⊗ is the element-wise matrix multiplication operation.

Original: DW-Conv denotes depth-wise convolution and Scale i , i ∈ { 0 , 1 , 2 , 3 } , denotes the i th branch in Fig. 2(b).
Revised: DW-Conv denotes depth-wise convolution and Scale i , i ∈ { 0 , 1 , 2 , 3 } , denotes the i th branch in Fig. 2(b).

Original: Scale 0 is the identity connection.
Revised: Scale 0 is the identity connection.

Original: Following [61], in each branch, we use two depth-wise strip convolutions to approximate standard depth-wise convolutions with large kernels.
Revised: Following [69], in each branch, we use two depth-wise strip convolutions to approximate standard depth-wise convolutions with large kernels.

Original: Here, the kernel size for each branch is set to 7, 11, and 21, respectively.
Revised: Here, the kernel size for each branch is set to 7, 11, and 21, respectively.

Original: The reasons why we choose depth-wise strip convolutions are two-fold.
Revised: The reasons why we choose depth-wise strip convolutions are two-fold.

Original: On one hand, strip convolution is lightweight.
Revised: On one hand, strip convolution is lightweight.

Original: To mimic a standard 2D convolution with kernel size 7 × 7 , we only need a pair of 7 × 1 and 1 × 7 convolutions.
Revised: To mimic a standard 2D convolution with kernel size 7 × 7 , we only need a pair of 7 × 1 and 1 × 7 convolutions.

Original: On the other hand, there are some strip-like objects, such as human and telephone pole in the segmentation scenes.
Revised: On the other hand, there are some strip-like objects, such as human and telephone pole in the segmentation scenes.

Original: Thus, strip convolution can be a complement of grid convolutions and helps extract strip-like features [61, 28].
Revised: Thus, strip convolution can be a complement of grid convolutions and helps extract strip-like features [69, 30].

Original: Stacking a sequence of building blocks yields the proposed convolutional encoder, named MSCAN.
Revised: Stacking a sequence of building blocks yields the proposed convolutional encoder, named MSCAN.

Original: For MSCAN, we adopt a common hierarchical structure, which contains four stages with decreasing spatial resolutions H 4 × W 4 , H 8 × W 8 , H 16 × W 16 and H 32 × W 32 .
Revised: For MSCAN, we adopt a common hierarchical structure, which contains four stages with decreasing spatial resolutions H 4 × W 4 , H 8 × W 8 , H 16 × W 16 and H 32 × W 32 .

Original: Here, H and W are height and width of the input image, respectively.
Revised: Here, H and W are height and width of the input image, respectively.

Original: Each stage contains a down-sampling block and a stack of building blocks as described above.
Revised: Each stage contains a down-sampling block and a stack of building blocks as described above.

Original: The down-sampling block has a convolution with stride 2 and kernel size 3 × 3 , followed by a batch normalization layer [34].
Revised: The down-sampling block has a convolution with stride 2 and kernel size 3 × 3 , followed by a batch normalization layer [36].

Original: Note that, in each building block of MSCAN, we use batch normalization instead of layer normalization as we found batch normalization gains more for the segmentation performance.
Revised: Note that, in each building block of MSCAN, we use batch normalization instead of layer normalization as we found batch normalization gains more for the segmentation performance.

Original: [Table]
Revised: [Table]

Original: We desgin four encoder models with different sizes, named MSCAN-T, MSCAN-S, MSCAN-B
Revised: We desgin four encoder models with different sizes, named MSCAN-T, MSCAN-S, MSCAN-B

Original: [Table]
Revised: [Table]

Original: and
Revised: and

Original: MSCAN-L, respectively.
Revised: MSCAN-L, respectively.

Original: The corresponding overall segmentation models are termed SegNeXt-T,
Revised: The corresponding overall segmentation models are termed SegNeXt-T,

Original: SegNeXt-S, SegNeXt-B, SegNeXt-L, respectively.
Revised: SegNeXt-S, SegNeXt-B, SegNeXt-L, respectively.

Original: Detailed network settings are displayed in Tab. 2.
Revised: Detailed network settings are displayed in Tab. 2.

Original: 3.2 Decoder
Revised: 3.2 Decoder

Original: In segmentation models [79, 96, 4]
Revised: In segmentation models [90, 108, 4]

Original: [Table]
Revised: [Table]

Original: the encoders are mostly pretrained on the ImageNet dataset.
Revised: the encoders are mostly pretrained on the ImageNet dataset.

Original: To capture high-level semantics, a decoder is usually necessary, which is applied upon the encoder.
Revised: To capture high-level semantics, a decoder is usually necessary, which is applied upon the encoder.

Original: In this work, we investigate three simple decoder structures, which have been shown in Fig.
Revised: In this work, we investigate three simple decoder structures, which have been shown in Fig.

Original: The first one, adopted in SegFormer [79], is a purely MLP-based structure.
Revised: The first one, adopted in SegFormer [90], is a purely MLP-based structure.

Original: The second one is mostly adopted
Revised: The second one is mostly adopted

Original: CNN-based models.
Revised: CNN-based models.

Original: In this kind of structure, the output of the encoder is directly used as the input to a heavy decoder head, like ASPP [4], PSP [94], and DANet [18].
Revised: In this kind of structure, the output of the encoder is directly used as the input to a heavy decoder head, like ASPP [4], PSP [106], and DANet [20].

Original: The last one is the structure adopted in our SegNeXt.
Revised: The last one is the structure adopted in our SegNeXt.

Original: We aggregate features from the last three stages and use a lightweight
Revised: We aggregate features from the last three stages and use a lightweight

Original: Hamburger [20] to further model the global context.
Revised: Hamburger [22] to further model the global context.

Original: Combined with our powerful convolutional encoder, we found that using a lightweight decoder improves performance-computation efficiency.
Revised: Combined with our powerful convolutional encoder, we found that using a lightweight decoder improves performance-computation efficiency.

Original: S t a g e
Revised: S t a g e

Original: S t a g e S t a g e 
Revised: S t a g e S t a g e S t a g e

Original: S t a g e S t a g eS
Revised: S t a g e S t a g e

Original: [Figure] a g e
Revised: S t a g e

Original: S
Revised: S

Original: [Figure]
Revised: [Figure]

Original: a g e
Revised: a g e

Original: [Figure]

Original: S
Revised: S

Original: [Figure]

Original: a g e

Original: [Figure]
Revised: [Figure]

Original: It is worth nothing that unlike SegFormer whose decoder aggregates the features from Stage 1 to
Revised: It is worth nothing that unlike SegFormer whose decoder aggregates the features from Stage 1 to

Original: Stage 4, our decoder only receives features from the last three stages.
Revised: Stage 4, our decoder only receives features from the last three stages.

Original: This is because our SegNeXt is based on convolutions.
Revised: This is because our SegNeXt is based on convolutions.

Original: The features from Stage 1 contain too much low-level information and hurts the performance.
Revised: The features from Stage 1 contain too much low-level information and hurts the performance.

Original: Besides, operations on Stage 1 bring heavy computational overhead.
Revised: Besides, operations on Stage 1 bring heavy computational overhead.

Original: In our experiment section, we will show that our convolutional SegNeXt performs much better than the recent state-of-the-art transformer-based SegFormer [79] and HRFormer [88].
Revised: In our experiment section, we will show that our convolutional SegNeXt performs much better than the recent state-of-the-art transformer-based SegFormer [90] and HRFormer [100].

Original:  Experiments
Revised:  Experiments

Original: Dataset.
Revised: Dataset.

Original: We evaluate our methods on seven popular datasets, including ImageNet-1K [14],
Revised: We evaluate our methods on seven popular datasets, including ImageNet-1K [14],

Original: ADE20K [98], Cityscapes [12], Pascal VOC [17], Pascal Context [57], COCO-Stuff [3], and iSAID [75].
Revised: ADE20K [111], Cityscapes [12], Pascal VOC [17], Pascal Context [65], COCO-Stuff [3], and iSAID [84].

Original: ImageNet [14] is the best-known dataset for image classification, which contains
Revised: ImageNet [14] is the best-known dataset for image classification, which contains

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 1,000 categories.
Revised: 1,000 categories.

Original: Similar to most segmentation methods, we use it to pretrain our MSCAN encoder.
Revised: Similar to most segmentation methods, we use it to pretrain our MSCAN encoder.

Original: ADE20K [98] is a challenging dataset which contains 150 semantic classes.
Revised: ADE20K [111] is a challenging dataset which contains 150 semantic classes.

Original: It consists of 20,210/2,000/3,352 images in the training, validation and test sets.
Revised: It consists of 20,210/2,000/3,352 images in the training, validation and test sets.

Original: Cityscapes [12] mainly focuses on urban scenes and contains 5.000 high-resolution images with 19 categories.
Revised: Cityscapes [12] mainly focuses on urban scenes and contains 5.000 high-resolution images with 19 categories.

Original: There are 2,975/500/1,525 images for training, validation and testing, respectively.
Revised: There are 2,975/500/1,525 images for training, validation and testing, respectively.

Original: Pascal VOC [17] involves 20 foreground classes and a background class.
Revised: Pascal VOC [17] involves 20 foreground classes and a background class.

Original: After augmentation, it has 10, 582/1, 449/1,images for training, validation and testing, respectively.
Revised: After augmentation, it has 10, 582/1, 449/1,images for training, validation and testing, respectively.

Original: Pascal Context [57] contains 59 foreground classes and a background class.
Revised: Pascal Context [65] contains 59 foreground classes and a background class.

Original: The training set and validation set contain 4,996 and 5,104 images, respectively.
Revised: The training set and validation set contain 4,996 and 5,104 images, respectively.

Original: COCO-Stuff
Revised: COCO-Stuff

Original: [3] is also a challenging benchmark, which contains 172 semantic categories and 164k images in total.
Revised: [3] is also a challenging benchmark, which contains 172 semantic categories and 164k images in total.

Original: iSAID [75] is a large-scale aerial image segmentation benchmark, which includes 15 foreground classes and a background class.
Revised: iSAID [84] is a large-scale aerial image segmentation benchmark, which includes 15 foreground classes and a background class.

Original: Its training, validation and test sets separately involve 1,411/458/937 images.
Revised: Its training, validation and test sets separately involve 1,411/458/937 images.

Original: Implementation details.
Revised: Implementation details.

Original: We conduct experiments by using Jittor [30] and Pytorch [60].
Revised: We conduct experiments by using Jittor [32] and Pytorch [68].

Original: Our implementation is based on timm (Apache-2.0) [76] and mmsegmentation (Apache-2.0) [11] libraries for classification and segmentation, respectively.
Revised: Our implementation is based on timm (Apache-2.0) [85] and mmsegmentation (Apache-2.0) [11] libraries for classification and segmentation, respectively.

Original: All encoders of our segmentation models are pretrained on the ImageNet-1K dataset [14].
Revised: All encoders of our segmentation models are pretrained on the ImageNet-1K dataset [14].

Original: We adopt Top-1 accuracy and mean Intersection over
Revised: We adopt Top-1 accuracy and mean Intersection over

Original: Union (mIoU) as our evaluation metrics for classification and segmentation, respectively.
Revised: Union (mIoU) as our evaluation metrics for classification and segmentation, respectively.

Original: All models are trained on a node with 8 RTX 3090 GPUs.
Revised: All models are trained on a node with 8 RTX 3090 GPUs.

Original: For ImageNet pretraining, our data augmentation method and training settings are the same as
Revised: For ImageNet pretraining, our data augmentation method and training settings are the same as

Original: DeiT [69].
Revised: DeiT [78].

Original: For segmentation experiments, we adopt some common data augmentation including random horizontal flipping, random scaling (from 0.5 to 2) and random cropping.
Revised: For segmentation experiments, we adopt some common data augmentation including random horizontal flipping, random scaling (from 0.5 to 2) and random cropping.

Original: The batch size is set to 8 for the Cityscapes dataset and 16 for all the other datasets.
Revised: The batch size is set to 8 for the Cityscapes dataset and 16 for all the other datasets.

Original: AdamW [53] is applied to train our models.
Revised: AdamW [61] is applied to train our models.

Original: We set the initial learning rate as 0.00006 and employ the poly-learning rate decay policy.
Revised: We set the initial learning rate as 0.00006 and employ the poly-learning rate decay policy.

Original: We train our model 160K iterations for ADE20K, Cityscapes and iSAID datasets and 80K iterations for COCO-Stuff, Pascal VOC and Pascal Context datasets.
Revised: We train our model 160K iterations for ADE20K, Cityscapes and iSAID datasets and 80K iterations for COCO-Stuff, Pascal VOC and Pascal Context datasets.

Original: During testing, we use both the single-scale (SS) and multi-scale (MS) flip test strategies for a fair comparison.
Revised: During testing, we use both the single-scale (SS) and multi-scale (MS) flip test strategies for a fair comparison.

Original: More details can be found in our supplementary materials.
Revised: More details can be found in our supplementary materials.

Original: 4.1 Encoder Performance on ImageNet
Revised: 4.1 Encoder Performance on ImageNet

Original: ImageNet pretraining is a common strategy for training segmentation models [94, 5, 79, 88, 4].
Revised: ImageNet pretraining is a common strategy for training segmentation models [106, 5, 90, 100, 4].

Original: Here, we compare the performance of our MSCAN with several recent popular CNN-based and transformerbased classification models.
Revised: Here, we compare the performance of our MSCAN with several recent popular CNN-based and transformer-based classification models.

Original: As shown in Tab. 3, our MSCAN achieves better results than the recent state-of-the-art CNN-based method, ConvNeXt [51] and outperforms popular transformer-based methods, like Swin Transformer [50] and MiT, the encoder of SegFormer [79].
Revised: As shown in Tab. 3, our MSCAN achieves better results than the recent state-of-the-art CNN-based method, ConvNeXt [59] and outperforms popular transformerbased methods, like Swin Transformer [58] and MiT, the encoder of SegFormer [90].

Original: [Table]
Revised: [Table]

Original: 4.2 Ablation study
Revised: 4.2 Ablation study

Original: Ablation on MSCA design.
Revised: Ablation on MSCA design.

Original: We conduct ablation study on MSCA design on both ImageNet and
Revised: We conduct ablation study on MSCA design on both ImageNet and

Original: ADE20K dataset.
Revised: ADE20K dataset.

Original: K × K branch contains a depth-wise 1 × K convolution and a K × 1 depth-wise convolution.
Revised: K × K branch contains a depth-wise 1 × K convolution and a K × 1 depth-wise convolution.

Original: 1 × 1 conv means the channel mixing operation.
Revised: 1 × 1 conv means the channel mixing operation.

Original: Attention means the element-wise product, which makes the network obtain adaptive ability.
Revised: Attention means the element-wise product, which makes the network obtain adaptive ability.

Original: Results are shown in Tab. 6.
Revised: Results are shown in Tab. 6.

Original: We can find that each part contributes to the final performance.
Revised: We can find that each part contributes to the final performance.

Original: [Table]
Revised: [Table]

Original: Global Context for Decoder.
Revised: Global Context for Decoder.

Original: Decoder plays an important role in integrating global context from multi-scale features for segmentation models.
Revised: Decoder plays an important role in integrating global context from multi-scale features for segmentation models.

Original: Here, we investigate the influence of different global context modules on decoder.
Revised: Here, we investigate the influence of different global context modules on decoder.

Original: As shown in most previous works [74, 18], attention-based decoders achieves better performance for CNNs than pyramid structures [94, 4], we thus only show the results using attention-based decoders.
Revised: As shown in most previous works [83, 20], attention-based decoders achieves better performance for CNNs than pyramid structures [106, 4], we thus only show the results using attention-based decoders.

Original: Specifically, we show results with 4 different types of attentionbased decoders, including non-local (NL) attention [74] with O ( n 2 ) complexity and CCNet [33],
Revised: Specifically, we show results with 4 different types of attentionbased decoders, including non-local (NL) attention [83] with O ( n 2 ) complexity and CCNet [35],

Original: EMANet [39], and HamNet [20] with O ( n ) complexity.
Revised: EMANet [46], and HamNet [22] with O ( n ) complexity.

Original: As shown in Tab. 5, Ham achieves the best trade-off between complexity and performance.
Revised: As shown in Tab. 5, Ham achieves the best trade-off between complexity and performance.

Original: Therefore, we use Hamburger [20] in our decoder.
Revised: Therefore, we use Hamburger [22] in our decoder.

Original: [Table]
Revised: [Table]

Original: Decoder Structure.
Revised: Decoder Structure.

Original: Unlike image classification, segmentation models need high-resolution outputs.
Revised: Unlike image classification, segmentation models need high-resolution outputs.

Original: We ablate three different decoder designs for segmentation, all of which have been shown in Fig.
Revised: We ablate three different decoder designs for segmentation, all of which have been shown in Fig.

Original: The corresponding results are listed in Tab. 7.
Revised: The corresponding results are listed in Tab. 7.

Original: We can see that SegNeXt (c) achieves the best performance and the computational cost is also low.
Revised: We can see that SegNeXt (c) achieves the best performance and the computational cost is also low.

Original: Importance of Our MSCA.
Revised: Importance of Our MSCA.

Original: Here, we conduct experiments to demonstrate the importance of
Revised: Here, we conduct experiments to demonstrate the importance of

Original: MSCA for segmentation.
Revised: MSCA for segmentation.

Original: As a comparison, we follow VAN [23] and replace the multiple branches in our MSCA with a single convolution with a large kernel.
Revised: As a comparison, we follow VAN [25] and replace the multiple branches in our MSCA with a single convolution with a large kernel.

Original: As shown in Tab. 8 and Tab. 3, we can observe that though the performance of the two encoders is close in ImageNet classification, SegNeXt[Table] SegNeXt
Revised: As shown in Tab. 8 and Tab. 3, we can observe that though the performance of the two encoders is close in ImageNet classification, SegNeXt

Original: [Table]
Revised: [Table]

Original: SegFormer-B
Revised: SegFormer-B

Original: SegNeXt-B
Revised: SegNeXt-B

Original: SegFormer-B
Revised: SegFormer-B

Original: SegNeXt-B w/ MSCA yields much better results than the setting w/o MSCA.
Revised: SegNeXt-B w/ MSCA yields much better results than the setting w/o MSCA.

Original: This indicates that aggregating multi-scale features is crucial in encoder for semantic segmentation.
Revised: This indicates that aggregating multi-scale features is crucial in encoder for semantic segmentation.

Original: 4.3 Comparison with state-of-the-art methods
Revised: 4.3 Comparison with state-of-the-art methods

Original: In this subsection, we compare our method with state-of-the-art CNN-based methods, such as
Revised: In this subsection, we compare our method with state-of-the-art CNN-based methods, such as

Original: HRNet [70], ResNeSt [92], and EfficientNet [68], and transformer-based methods, like Swin Transformer [50], SegFormer [79], HRFormer [88], MaskFormer [10], and Mask2Former [9].
Revised: HRNet [79], ResNeSt [104], and EfficientNet [77], and transformer-based methods, like Swin Transformer [58], SegFormer [90], HRFormer [100], MaskFormer [10], and Mask2Former [9].

Original: Performance-computation trade-off.
Revised: Performance-computation trade-off.

Original: ADE20K and Cityscapes are two widely used benchmarks in semantic segmentation.
Revised: ADE20K and Cityscapes are two widely used benchmarks in semantic segmentation.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, we plot the performance-computation curves of different methods on the Cityscape and ADE20K validation set.
Revised: 1, we plot the performance-computation curves of different methods on the Cityscape and ADE20K validation set.

Original: Clearly, our method achieves the best trade-off between performance and computations compared to other state-of-the-art methods, like SegFormer [79], HRFormer [88], and MaskFormer [10].
Revised: Clearly, our method achieves the best trade-off between performance and computations compared to other state-of-the-art methods, like SegFormer [90], HRFormer [100], and MaskFormer [10].

Original: Comparison with state-of-the-art transformers.
Revised: Comparison with state-of-the-art transformers.

Original: We compare SegNeXt with state-of-the-art transformer models on the ADE20K, Cityscapes, COCO-Stuff and Pascal Context benchmarks.
Revised: We compare SegNeXt with state-of-the-art transformer models on the ADE20K, Cityscapes, COCO-Stuff and Pascal Context benchmarks.

Original: As shown in Tab. 9, SegNeXt-L surpasses Mask2Former with Swin-T backbone by 3.3 mIoU (51.0 v.s. 47.7) with similar parameters and computational cost on he ADE20K dataset.
Revised: As shown in Tab. 9, SegNeXt-L surpasses Mask2Former with Swin-T backbone by 3.3 mIoU (51.0 v.s. 47.7) with similar parameters and computational cost on he ADE20K dataset.

Original: Moreover, SegNeXt-B yields 2.0 mIoU improvement (48.5 v.s. 46.5) compared to SegFormer-B2 using only 56% computations on the ADE20K dataset.
Revised: Moreover, SegNeXt-B yields 2.0 mIoU improvement (48.5 v.s. 46.5) compared to SegFormer-B2 using only 56% computations on the ADE20K dataset.

Original: In particular, since the self-attention in SegFormer [79] is of quadratic complexity w.r.t.
Revised: In particular, since the self-attention in SegFormer [90] is of quadratic complexity w.r.t.

Original: , the input size while our method uses convolutions, this makes our method perform greatly well when dealing with high-resolution images from the Cityscapes dataset.
Revised: , the input size while our method uses convolutions, this makes our method perform greatly well when dealing with high-resolution images from the Cityscapes dataset.

Original: For instance,
Revised: For instance,

Original: SegNeXt-B gains 1.6 mIoU (81.0 v.s. 82.6) over SegFormer-B2 but uses 40% less computations.
Revised: SegNeXt-B gains 1.6 mIoU (81.0 v.s. 82.6) over SegFormer-B2 but uses 40% less computations.

Original: In Fig.
Revised: In Fig.

Original: 4, we also show a qualitative comparison with SegFormer.
Revised: 4, we also show a qualitative comparison with SegFormer.

Original: We can see that thanks to the proposed MSCA, our method recognizes well when processing object details.
Revised: We can see that thanks to the proposed MSCA, our method recognizes well when processing object details.

Original: Comparison with state-of-the-art CNNs.
Revised: Comparison with state-of-the-art CNNs.

Original: As shown in Tab. 4, Tab. 10, and Tab. 12, we compare our SegNeXt with state-of-the-art CNNs such as ResNeSt-269 [92], EfficientNet-L2 [99], and HRNetW48 [70] on the Pascal VOC 2012, Pascal Context, and iSAID datasets.
Revised: As shown in Tab. 4, Tab. 10, and Tab. 12, we compare our SegNeXt with state-of-the-art CNNs such as ResNeSt-269 [104], EfficientNet-L2 [112], and HRNetW48 [79] on the Pascal VOC 2012, Pascal Context, and iSAID datasets.

Original: SegNeXt-L outperforms the popular HRNet (OCR) [70, 87] model (60.3 v.s. 56.3) using even less parameters and computations, which is elaborately designed for the segmentation task.
Revised: SegNeXt-L outperforms the popular HRNet (OCR) [79, 99] model (60.3 v.s. 56.3) using even less parameters and computations, which is elaborately designed for the segmentation task.

Original: Moreover, SegNeXt-L performs even better than EfficientNet-L2 (NAS-FPN), which is pretrained on additional 300 million unavailable images, on the Pascal VOC 2012 test leaderboard.
Revised: Moreover, SegNeXt-L performs even better than EfficientNet-L2 (NAS-FPN), which is pretrained on additional 300 million unavailable images, on the Pascal VOC 2012 test leaderboard.

Original: It is worth noting that EfficientNet-L2 (NAS-FPN) has 485M parameters, while SegNeXt-L has only 48.7M parameters.
Revised: It is worth noting that EfficientNet-L2 (NAS-FPN) has 485M parameters, while SegNeXt-L has only 48.7M parameters.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Comparison with real-time methods.
Revised: Comparison with real-time methods.

Original: In addition to the state-of-the-art performance, our method is also suitable for real-time deployments.
Revised: In addition to the state-of-the-art performance, our method is also suitable for real-time deployments.

Original: Even without any specific software or hardware acceleration,
Revised: Even without any specific software or hardware acceleration,

Original: SegNeXt-T realizes 25 frames per second (FPS) using a single 3090 RTX GPU when dealing with an image of size 768 × 1,536.
Revised: SegNeXt-T realizes 25 frames per second (FPS) using a single 3090 RTX GPU when dealing with an image of size 768 × 1,536.

Original: As shown in Tab. 11, our method sets new state-of-the-art results for real-time segmentation on the Cityscapes test set.
Revised: As shown in Tab. 11, our method sets new state-of-the-art results for real-time segmentation on the Cityscapes test set.

Revised: 4.4 Weakly-Supervised Semantic Segmentation

Revised: In this subsection, we apply the proposed network to the weakly-supervised semantic segmentation task.

Revised: In this task, a pseudo segmentation map is often generated by a classification model using

Original: Conclusions and Discussion
Revised: CAM [110]. Previous works mostly utilize VGGNet [72] or ResNets [28, 87] as the CAM generator. Here, we test the performance of the CAMs produced by our MSCAN. We use the EPS [43] architecture and follow the training strategies and recipes.

Revised: The numerical results are shown in Tab. 13.

Revised: We can see that simply replacing the ResNet38 backbone with our MSCAN can clearly improve the performance compared to the EPS baseline.

Revised: When using our SegNeXt as the segmentation network, the performance gain increases further.

Revised: [Table]

Revised: [Table]

Revised:  Conclusions and Discussion

Original: In this paper
Revised: In this paper

Original: [Table]
Revised: [Table]

Original: we analyze previous successful segmentation models and find the good characteristics owned by them.
Revised: we analyze previous successful segmentation models and find the good characteristics owned by them.

Original: Based on the findings
Revised: Based on the findings

Original: [Table]
Revised: [Table]

Original: we present a tailored convolutional attention module MSCA and a CNN-style network SegNeXt.
Revised: we present a tailored convolutional attention module MSCA and a CNN-style network SegNeXt.

Original: Experimental results demonstrate that SegNeXt surpasses current state-of-the-art transformer-based methods by a considerable margin.
Revised: Experimental results demonstrate that SegNeXt surpasses current state-of-the-art transformer-based methods by a considerable margin.

Original: Recently, transformer-based models have dominated various segmentation leaderboards.
Revised: Recently [Table] transformer-based models have dominated various segmentation leaderboards.

Original: Instead, this paper shows that CNN-based methods can still perform better than transformer-based methods when using a proper design.
Revised: Instead, this paper shows that CNN-based methods can still perform better than transformer-based methods when using a proper design.

Original: We hope this paper could encourage researchers to further investigate the potential of CNNs.[Table]
Revised: We hope this paper could encourage researchers to further investigate the potential of CNNs.

Original: Our model also has its limitations, for example, extending this method to large-scale models with 100M+ parameters and the performance on other vision or NLP tasks.
Revised: Our model also has its limitations, for example, extending this method to large-scale models with 100M+ parameters and the performance on other vision or NLP tasks.

Original: These will be addressed in our future works
Revised: These will be addressed in our future works.

Original: [Table]

Original: Acknowledgment
Revised: Acknowledgment

Original: This work was supported by the National Key R&D Program of China (No. 2021ZD0112902) and the Natural Science Foundation of China (No. 62220106003, 62176130).
Revised: This work was supported by the National Key R&D Program of China (NO. 2018AAA0100400) and the Natural Science Foundation of China (No. 62220106003, No. 62176130, and No. 62276145).

Original: We would like to thank Yi Zhang and Zhengyang Geng for their kind help in experiments
Revised: We would like to thank Yi Zhang and Zhengyang Geng for their kind help in experiments.