Original: SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation
Revised: SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: We present SegNeXt, a simple convolutional network architecture for semanticsegmentation.
Revised: We present SegNeXt, a simple convolutional network architecture for semantic segmentation.

Original: Recent transformer-based models have dominated the field of semantic segmentation due to the efficiency of self-attention in encoding spatialinformation.
Revised: Recent transformer-based models have dominated the field of semantic segmentation due to the efficiency of self-attention in encoding spatial information.

Original: In this paper, we show that convolutional attention is a more efficientand effective way to encode contextual information than the self-attention mechanism in transformers.
Revised: In this paper, we show that convolutional attention is a more efficient and effective way to encode contextual information than the self-attention mechanism in transformers.

Original: By re-examining the characteristics owned by successfulsegmentation models, we discover several key components leading to the performance improvement of segmentation models.
Revised: By re-examining the characteristics owned by successful segmentation models, we discover several key components leading to the performance improvement of segmentation models.

Original: This motivates us to design a novelconvolutional attention network that uses cheap convolutional operations.
Revised: This motivates us to design a novel convolutional attention network that uses cheap convolutional operations.

Original: Withoutbells and whistles, our SegNeXt significantly improves the performance of previousstate-of-the-art methods on popular benchmarks, including ADE20K, Cityscapes,
Revised: Without bells and whistles, our SegNeXt significantly improves the performance of previous state-of-the-art methods on popular benchmarks, including ADE20K, Cityscapes,

Original: COCO-Stuff, Pascal VOC, Pascal Context, and iSAID.
Revised: COCO-Stuff, Pascal VOC, Pascal Context, and iSAID.

Original: Notably, SegNeXt outperforms EfficientNet-L2 w/ NAS-FPN and achieves 90 .
Revised: Notably, SegNeXt outperforms EfficientNet-L2 w/ NAS-FPN and achieves 90 .

Original: 6% mIoU on the Pascal
Revised: 6% mIoU on the Pascal

Original: VOC 2012 test leaderboard using only 1 / 10 parameters of it.
Revised: VOC 2012 test leaderboard using only 1 / 10 parameters of it.

Original: On average, SegNeXtachieves about 2 . 0% mIoU improvements compared to the state-of-the-art methodson the ADE20K datasets with the same or fewer computations.
Revised: On average, SegNeXt achieves about 2 . 0% mIoU improvements compared to the state-of-the-art methods on the ADE20K datasets with the same or fewer computations.

Original: Code will be madepublicly available.

Original:  Introduction
Revised:  Introduction

Original: As one of the most fundamental research topics in computer vision, semantic segmentation, whichaims at assigning each pixel a semantic category, has attracted great attention over the past decade.
Revised: As one of the most fundamental research topics in computer vision, semantic segmentation, which aims at assigning each pixel a semantic category, has attracted great attention over the past decade.

Original: From early CNN-based models, typified by FCN [48] and DeepLab series [4, 6, 8], to recenttransformer-based methods, represented by SETR [90] and SegFormer [74], semantic segmentationmodels have experienced significant revolution in terms of network architectures.
Revised: From early CNN-based models, typified by FCN [52] and DeepLab series [4, 5, 7], to recent transformer-based methods, represented by SETR [96] and SegFormer [79], semantic segmentation models have experienced significant revolution in terms of network architectures.

Original: Here, n refers to the number of pixels or tokens.

Original: Strong encoder denotes strong backbones, like ViT [17], and adopts the advanced training strategy.

Original: [Table]
Revised: [Table]

Original: DeepLabV3+

Original: HRNet

Original: SETR

Original: SegFormer

Original: SegNeXt

Original: [Table]
Revised: [Figure]

Original: ✗ ✗ ✓ ✓ ✓ [Table] ✓ ✓ ✗ ✗ ✓ [Table]
Revised: The size of the circle indicates the number of parameters.

Original: ✗ ✗ ✓ ✓ ✓ [Table] complexity O ( n )
Revised: Larger circles mean more parameters.

Original: O ( n ) O ( n 2 ) O ( n 2 ) O ( n )
Revised: We can see that our SegNeXt achieves the best trade-off between segmentation performance and computational complexity.

Original: By revisiting previous successful semantic segmentation works, we summarize several key propertiesdifferent models possess as shown in Tab. 1.
Revised: By revisiting previous successful semantic segmentation works, we summarize several key properties different models possess as shown in Tab. 1.

Original: Based on the above observation, we argue a successful[Figure] semantic segmentation model should have the following characteristics:
Revised: Based on the above observation, we argue a successful semantic segmentation model should have the following characteristics:

Original: (i)
Revised: (i)

Original: A strong backbonenetwork as encoder.
Revised: A strong backbone network as encoder.

Original: Compared to previous CNN-based models, the performance improvement oftransformer-based models is mostly from a stronger backbone network.
Revised: Compared to previous CNN-based models, the performance improvement of transformer-based models is mostly from a stronger backbone network.

Original: (ii) Multi-scale informationinteraction.
Revised: (ii) Multi-scale information interaction.

Original: Different from the image classification task that mostly identifies a single object, semanticsegmentation is a dense prediction task and hence needs to process objects of varying sizes in asingle image.
Revised: Different from the image classification task that mostly identifies a single object, semantic segmentation is a dense prediction task and hence needs to process objects of varying sizes in a single image.

Original: (iii) Spatial attention.
Revised: (iii) Spatial attention.

Original: Spatial attention allows models to perform segmentation throughprioritization of areas within the semantic regions.
Revised: Spatial attention allows models to perform segmentation through prioritization of areas within the semantic regions.

Original: (iv) Low computational complexity.
Revised: (iv) Low computational complexity.

Original: This isespecially crucial when dealing with high-resolution images from remote sensing and urban scenes.
Revised: This is especially crucial when dealing with high-resolution images from remote sensing and urban scenes.

Original: Taking the aforementioned analysis into account, in this paper, we rethink the design of convolutional attention and propose an efficient yet effectiveencoder-decoder architecture for semanticsegmentation.
Revised: Taking the aforementioned analysis into account, in this paper, we rethink the design of convolutional attention and propose an efficient yet effective architecture for semantic segmentation.

Original: Unlike previous transformer-based models that use convolutions in decoders as featurerefiners, our method inverts the transformer-convolution encoder-decoder architecture.
Revised: Unlike previous transformer-based models that use convolutions in decoders as feature refiners, our method inverts the transformer-convolution encoder-decoder architecture.

Original: Specifically,for each block in our encoder, we renovate the design of conventional convolutional blocks and utilizemulti-scale convolutional features to evoke spatial attention via a simple element-wise multiplicationfollowing [24].
Revised: Specifically, for each block in our encoder, we renovate the design of conventional convolutional blocks and utilize multi-scale convolutional features to evoke spatial attention via a simple element-wise multiplication following [23].

Original: We found such a simple way to build spatial attention is more efficient than both thestandard convolutions and self-attention in spatial information encoding.
Revised: We found such a simple way to build spatial attention is more efficient than both the standard convolutions and self-attention in spatial information encoding.

Original: For decoder, we collectmulti-level features from different stages and use Hamburger [21] to further extract global context.
Revised: For decoder, we collect multi-level features from different stages and use Hamburger [20] to further extract global context.

Original: Under this setting, our method can obtain multi-scale context from local to global, achieve adaptabilityin spatial and channel dimensions, and aggregate information from low to high levels.
Revised: Under this setting, our method can obtain multi-scale context from local to global, achieve adaptability in spatial and channel dimensions, and aggregate information from low to high levels.

Original: Our network, termed SegNeXt, is mostly composed of convolutional operations except the decoderpart, which contains a decomposition-based Hamburger module [21] (Ham) for global informationextraction.
Revised: Our network, termed SegNeXt, is mostly composed of convolutional operations except the decoder part, which contains a decomposition-based Hamburger module [20] (Ham) for global information extraction.

Original: This makes our SegNeXt much more efficient than previous segmentation methods thatheavily rely on transformers.
Revised: This makes our SegNeXt much more efficient than previous segmentation methods that heavily rely on transformers.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, SegNeXt outperforms recent transformer-basedmethods significantly.
Revised: 1, SegNeXt outperforms recent transformer-based methods significantly.

Original: In particular, our SegNeXt-S outperforms SegFormer-B2 (81.3% vs. 81.0%)using only about 1 / 6 (124.6G vs. 717.1G) computational cost and 1 / 2 parameters (13.9M vs. 27.6M)when dealing with high-resolution urban scenes from the Cityscapes dataset.
Revised: In particular, our SegNeXt-S outperforms SegFormer-B2 (81.3% vs. 81.0%) using only about 1 / 6 (124.6G vs. 717.1G) computational cost and 1 / 2 parameters (13.9M vs. 27.6M) when dealing with high-resolution urban scenes from the Cityscapes dataset.

Original: Our contributions can be summarized as follows:
Revised: Our contributions can be summarized as follows:

Original: • We identify the characteristics that a good semantic segmentation model should own and present a novel tailored network architecture, termed SegNeXt, that evokes spatial attention via multi-scale convolutional features.
Revised: • We identify the characteristics that a good semantic segmentation model should own and present a novel tailored network architecture, termed SegNeXt, that evokes spatial attention via multi-scale convolutional features.

Original: • We show that an encoder with simple and cheap convolutions can still perform better than vision transformers, especially when processing object details, while it requires much less computational cost.
Revised: • We show that an encoder with simple and cheap convolutions can still perform better than vision transformers, especially when processing object details, while it requires much less computational cost.

Original: • Our method improves the performance of state-of-the-art semantic segmentation methods by a large margin on various segmentation benchmarks, including ADE20K, Cityscapes, COCO-Stuff, Pascal VOC, Pascal Context, and iSAID.
Revised: • Our method improves the performance of state-of-the-art semantic segmentation methods by a large margin on various segmentation benchmarks, including ADE20K, Cityscapes, COCO-Stuff, Pascal VOC, Pascal Context, and iSAID.

Original:  Related Work2.1 Semantic Segmentation
Revised:  Related Work 2.1 Semantic Segmentation

Original: Semantic segmentation is a fundamental computer vision task.
Revised: Semantic segmentation is a fundamental computer vision task.

Original: Since FCN [48] was proposed,convolutional neural networks (CNNs) [1, 58, 80, 88, 19, 81, 65, 20] have achieved great success andbecome a popular architecture for semantic segmentation.
Revised: Since FCN [52] was proposed, convolutional neural networks (CNNs) [1, 63, 86, 94, 18, 87, 70, 19, 44] have achieved great success and become a popular architecture for semantic segmentation.

Original: Recently, transformer-based methods [90,74, 82, 59, 57, 11, 10] have shown great potentials and outperform CNN-based methods.
Revised: Recently, transformer-based methods [96, 79, 88, 64, 62, 43, 10, 9] have shown great potentials and outperform CNN-based methods.

Original: In the era of deep learning, the architecture of segmentation models can be roughly divided into twoparts: encoder and decoder.
Revised: In the era of deep learning, the architecture of segmentation models can be roughly divided into two parts: encoder and decoder.

Original: For the encoder, researchers usually adopt popular classification networks( e.g., ResNet [27], ResNeXt [75] and DenseNet [31]) instead of tailored architecture.
Revised: For the encoder, researchers usually adopt popular classification networks ( e.g., ResNet [26], ResNeXt [80] and DenseNet [31]) instead of tailored architecture.

Original: However,semantic segmentation is a kind of dense prediction task, which is different from image classification.
Revised: However, semantic segmentation is a kind of dense prediction task, which is different from image classification.

Original: The improvement in classification may not appear in the challenging segmentation task [28].
Revised: The improvement in classification may not appear in the challenging segmentation task [27].

Original: Thus,some tailored encoders appear, including Res2Net [20], HRNet [65], SETR [90], SegFormer [74],
Revised: Thus, some tailored encoders appear, including Res2Net [19], HRNet [70], SETR [96],

Original:  HRFormer [82], MPViT [37], DPT [57], etc .
Revised: SegFormer [79], HRFormer [88], MPViT [37], DPT [62], etc .

Original: For the decoder, it is often used in cooperatingwith encoders to achieve better results.
Revised: For the decoder, it is often used in cooperating with encoders to achieve better results.

Original: There are different types of decoders for different goals,including achieving multi-scale receptive fields [88, 7, 72], collecting multi-scale semantics [58, 74, 8],enlarging receptive field [5, 4, 56], strengthening edge features [89, 2, 16, 41, 84], and capturingglobal context [19, 33, 83, 39, 23, 26, 85].
Revised: There are different types of decoders for different goals, including achieving multi-scale receptive fields [94, 6, 77], collecting multi-scale semantics [63, 79, 7], enlarging receptive field [4, 4, 61], strengthening edge features [95, 2, 15, 41, 90], and capturing global context [18, 33, 89, 39, 22, 25, 91].

Original: In this paper, we summarize the characteristics of those successful models designed for semanticsegmentation and present a CNN-based model, named SegNeXt.
Revised: In this paper, we summarize the characteristics of those successful models designed for semantic segmentation and present a CNN-based model, named SegNeXt.

Original: The most related work to our paper,is [56], which decomposes a k × k convolution into a pair of k × 1 and 1 × k convolutions.
Revised: The most related work to our paper, is [61], which decomposes a k × k convolution into a pair of k × 1 and 1 × k convolutions.

Original: Thoughthis work has shown large convolutional kernels matter in semantic segmentation, it ignores theimportance of multi-scale receptive field and does not consider how to leverage these multi-scalefeatures extracted by large kernels for segmentation in the form of attention.
Revised: Though this work has shown large convolutional kernels matter in semantic segmentation, it ignores the importance of multi-scale receptive field and does not consider how to leverage these multi-scale features extracted by large kernels for segmentation in the form of attention.

Original: 2.2 Multi-Scale Networks
Revised: 2.2 Multi-Scale Networks

Original: Designing multi-scale network is one of the popular directions in computer vision.
Revised: Designing multi-scale network is one of the popular directions in computer vision.

Original: For segmentationmodels, multi-scale blocks appear in both the encoder [65, 20, 61] and the decoder [88, 80, 6] parts.
Revised: For segmentation models, multi-scale blocks appear in both the encoder [70, 19, 66] and the decoder [94, 86, 5] parts.

Original: GoogleNet [61] is one of the most related multi-scale architectures to our method, which uses amulti-branch structure to achieve multi-scale feature extraction.
Revised: GoogleNet [66] is one of the most related multi-scale architectures to our method, which uses a multi-branch structure to achieve multi-scale feature extraction.

Original: Another work that is related to ourmethod is HRNet [65].
Revised: Another work that is related to our method is HRNet [70].

Original: In the deeper stages, HRNet also keeps high-resolution features, which areaggregated with low-resolution features, to enable multi-scale feature extraction.
Revised: In the deeper stages, HRNet also keeps high-resolution features, which are aggregated with low-resolution features, to enable multi-scale feature extraction.

Original: Different from previous methods, SegNeXt, besides capturing multi-scale features in encoder, introduces an efficient attention mechanism and employs cheaper and larger kernel convolutions.
Revised: Different from previous methods, SegNeXt, besides capturing multi-scale features in encoder, introduces an efficient attention mechanism and employs cheaper and larger kernel convolutions.

Original: Theseenable our model to achieve higher performance than the aforementioned segmentation methods.
Revised: These enable our model to achieve higher performance than the aforementioned segmentation methods.

Original: 2.3 Attention Mechanisms
Revised: 2.3 Attention Mechanisms

Original: Attention mechanism is a kind of adaptive selection process, which aims to make the networkfocus on the important part.
Revised: Attention mechanism is a kind of adaptive selection process, which aims to make the network focus on the important part.

Original: Generally speaking, it can be divided into two categories in semanticsegmentation [25], including channel attention and spatial attention.
Revised: Generally speaking, it can be divided into two categories in semantic segmentation [24], including channel attention and spatial attention.

Original: Different types of attentions playdifferent roles.
Revised: Different types of attentions play different roles.

Original: For instance, spatial attentions mainly care about the important spatial regions [17, 14,52, 46, 22].
Revised: For instance, spatial attentions mainly care about the important spatial regions [16, 13, 56, 50, 21].

Original: Differently, the goal of using channel attention is to make the network selectively attendto those important objects, which has been demonstrated important in previous works [30, 9, 66].
Revised: Differently, the goal of using channel attention is to make the network selectively attend to those important objects, which has been demonstrated important in previous works [29, 8, 71].

Original: Speaking of the recent popular vision transformers [17, 46, 76, 68, 67, 74, 32, 82], they usuallyignore adaptability in channel dimension.
Revised: Speaking of the recent popular vision transformers [16, 50, 82, 72, 73, 49, 79, 32, 48, 88, 81], they usually ignore adaptability in channel dimension.

Original: Visual attention network (VAN) [24] is the most related work to SegNeXt, which also proposes toleverage the large-kernel attention (LKA) mechanism to build both channel and spatial attention.
Revised: Visual attention network (VAN) [23] is the most related work to SegNeXt, which also proposes to leverage the large-kernel attention (LKA) mechanism to build both channel and spatial attention.

Original: Though VAN has achieved great performance in image classification, it neglects the role of multi-scalefeature aggregation during the network design, which is crucial for segmentation-like tasks.Method
Revised: Though VAN has achieved great performance in image classification, it neglects the role of multi-scale feature aggregation during the network design, which is crucial for segmentation-like tasks.

Original: In this section, we describe the architecture of the proposed SegNeXt in detail.

Original: Basically, we adoptan encoder-decoder architecture following most previous works, which is simple and easy to follow.

Original: d, 5 xd, 1 xd, 11 x 1 d, 7 xd, 1 xd, 1 xd, 21 x1 x
Revised: d, 5 xd, 1 xd, 11 x 1 d, 7 xd, 1 xd, 1 xd, 21 x1 x

Original: BN
Revised: BN

Original: Attention
Revised: Attention

Original: BN
Revised: BN

Original: FFN ×L 1 x
Revised: FFN ×L 1 x

Original: GELU
Revised: GELU

Original: MSCA 1 x1 xd, 3 x
Revised: MSCA 1 x1 xd, 3 x

Original: GELU 1 x
Revised: GELU 1 x

Original: Channel Mixing
Revised: Channel Mixing

Original: Multi-scale Feature
Revised: Multi-scale Feature

Original: Convolutional Attention
Revised: Convolutional Attention

Original: [Figure]
Revised: [Figure]

Revised:  Method

Revised: In this section, we describe the architecture of the proposed SegNeXt in detail.

Revised: Basically, we adopt an encoder-decoder architecture following most previous works, which is simple and easy to follow.

Original: 3.1 Convolutional Encoder
Revised: 3.1 Convolutional Encoder

Original: We adopt the pyramid structure for our encoder following most previous work [74, 5, 19].
Revised: We adopt the pyramid structure for our encoder following most previous work [79, 4, 18].

Original: For thebuilding block in our encoder, we adopt a similar structure to that of ViT [17, 74] but what is differentis that we do not use the self-attention mechanism but design a novel multi-scale convolutionalattention (MSCA) module.
Revised: For the building block in our encoder, we adopt a similar structure to that of ViT [16, 79] but what is different is that we do not use the self-attention mechanism but design a novel multi-scale convolutional attention (MSCA) module.

Original: As depicted in Fig.
Revised: As depicted in Fig.

Original: 2 (a), MSCA contains three parts: a depth-wiseconvolution to aggregate local information, multi-branch depth-wise strip convolutions to capturemulti-scale context, and an 1 × 1 convolution to model relationship between different channels.
Revised: 2 (a), MSCA contains three parts: a depth-wise convolution to aggregate local information, multi-branch depth-wise strip convolutions to capture multi-scale context, and an 1 × 1 convolution to model relationship between different channels.

Original: Theoutput of the 1 × 1 convolution is used as attention weights directly to reweigh the input of MSCA.
Revised: The output of the 1 × 1 convolution is used as attention weights directly to reweigh the input of MSCA.

Original: Mathematically, our MSCA can be written as:
Revised: Mathematically, our MSCA can be written as:

Original: [Equation]
Revised: [Equation]

Original: where F represents the input feature.
Revised: where F represents the input feature.

Original: Att and Out are the attention map and output, respectively.⊗ is the element-wise matrix multiplication operation.
Revised: Att and Out are the attention map and output, respectively. ⊗ is the element-wise matrix multiplication operation.

Original: DW-Conv denotes depth-wise convolutionand Scale i , i ∈ { 0 , 1 , 2 , 3 } , denotes the i th branch in Fig. 2(b).
Revised: DW-Conv denotes depth-wise convolution and Scale i , i ∈ { 0 , 1 , 2 , 3 } , denotes the i th branch in Fig. 2(b).

Original: Scale 0 is the identity connection.
Revised: Scale 0 is the identity connection.

Original: Following [56], in each branch, we use two depth-wise strip convolutions to approximate standarddepth-wise convolutions with large kernels.
Revised: Following [61], in each branch, we use two depth-wise strip convolutions to approximate standard depth-wise convolutions with large kernels.

Original: Here, the kernel size for each branch is set to 7, 11, and21, respectively.
Revised: Here, the kernel size for each branch is set to 7, 11, and 21, respectively.

Original: The reasons why we choose depth-wise strip convolutions are two-fold.
Revised: The reasons why we choose depth-wise strip convolutions are two-fold.

Original: On onehand, strip convolution is lightweight.
Revised: On one hand, strip convolution is lightweight.

Original: To mimic a standard 2D convolution with kernel size 7 × 7 ,we only need a pair of 7 × 1 and 1 × 7 convolutions.
Revised: To mimic a standard 2D convolution with kernel size 7 × 7 , we only need a pair of 7 × 1 and 1 × 7 convolutions.

Original: On the other hand, there are some strip-likeobjects, such as human and telephone pole in the segmentation scenes.
Revised: On the other hand, there are some strip-like objects, such as human and telephone pole in the segmentation scenes.

Original: Thus, strip convolution can bea complement of grid convolutions and helps extract strip-like features [56, 29].
Revised: Thus, strip convolution can be a complement of grid convolutions and helps extract strip-like features [61, 28].

Original: Stacking a sequence of building blocks yields the proposed convolutional encoder, named MSCAN.
Revised: Stacking a sequence of building blocks yields the proposed convolutional encoder, named MSCAN.

Original: For MSCAN, we adopt a common hierarchical structure, which contains four stages with decreasingspatial resolutions H 4 × W 4 , H 8 × W 8 , H 16 × W 16 and H 32 × W 32 .
Revised: For MSCAN, we adopt a common hierarchical structure, which contains four stages with decreasing spatial resolutions H 4 × W 4 , H 8 × W 8 , H 16 × W 16 and H 32 × W 32 .

Original: Here, H and W are height and width ofthe input image, respectively.
Revised: Here, H and W are height and width of the input image, respectively.

Original: Each stage contains a down-sampling block and a stack of buildingblocks as described above.
Revised: Each stage contains a down-sampling block and a stack of building blocks as described above.

Original: The down-sampling block has a convolution with stride 2 and kernel size3 × 3 , followed by a batch normalization layer [34].
Revised: The down-sampling block has a convolution with stride 2 and kernel size 3 × 3 , followed by a batch normalization layer [34].

Original: Note that, in each building block of MSCAN,we use batch normalization instead of layer normalization as we found batch normalization gainsmore for the segmentation performance. 
Revised: Note that, in each building block of MSCAN, we use batch normalization instead of layer normalization as we found batch normalization gains more for the segmentation performance. [Table]

Original: We desgin four encoder models with different sizes, named MSCAN-T, MSCAN-S, MSCAN-B, and
Revised: We desgin four encoder models with different sizes, named MSCAN-T, MSCAN-S, MSCAN-B [Table] and

Original: MSCAN-L, respectively.
Revised: MSCAN-L, respectively.

Original: The corresponding overall segmentation models are termed SegNeXt-T,
Revised: The corresponding overall segmentation models are termed SegNeXt-T,

Original: SegNeXt-S, SegNeXt-B, SegNeXt-L, respectively.
Revised: SegNeXt-S, SegNeXt-B, SegNeXt-L, respectively.

Original: Detailed network settings are displayed in Tab. 2.
Revised: Detailed network settings are displayed in Tab. 2.

Original: [Table]
Revised: 3.2 Decoder

Original: In segmentation models [74, 90, 5]
Revised: In segmentation models [79, 96, 4]

Original: [Table]
Revised: [Table]

Original: the encoders are mostly pretrained on the ImageNet dataset.
Revised: the encoders are mostly pretrained on the ImageNet dataset.

Original: Tocapture high-level semantics, a decoder is usually necessary, which is applied upon the encoder.
Revised: To capture high-level semantics, a decoder is usually necessary, which is applied upon the encoder.

Original: Inthis work [Table] we investigate three simple decoder structures, which have been shown in Fig.
Revised: In this work, we investigate three simple decoder structures, which have been shown in Fig.

Original: The firstone, adopted in SegFormer [74], is a purely MLP-based structure.
Revised: The first one, adopted in SegFormer [79], is a purely MLP-based structure.

Original: The second one is mostly adopted
Revised: The second one is mostly adopted

Original: CNN-based models.
Revised: CNN-based models.

Original: In this kind of structure, the output of the encoder is directly used as the inputto a heavy decoder head, like ASPP [5], PSP [88], and DANet [19].
Revised: In this kind of structure, the output of the encoder is directly used as the input to a heavy decoder head, like ASPP [4], PSP [94], and DANet [18].

Original: The last one is the structureadopted in our SegNeXt.
Revised: The last one is the structure adopted in our SegNeXt.

Original: We aggregate features from the last three stages and use a lightweight
Revised: We aggregate features from the last three stages and use a lightweight

Original: Hamburger [21] to further model the global context.
Revised: Hamburger [20] to further model the global context.

Original: Combined with our powerful convolutionalencoder, we found that using a lightweight decoder improves performance-computation efficiency.
Revised: Combined with our powerful convolutional encoder, we found that using a lightweight decoder improves performance-computation efficiency.

Original: S t a g e
Revised: S t a g e

Original: S t a g e S t a g eS t a g e
Revised: S t a g e S t a g e

Original: S t a g e S t a g e 
Revised: S t a g e S t a g e S

Revised: [Figure]

Original: S t a g e 
Revised: a g e S

Original: S t a g e S t a g e S t a g e S t a g e
Revised: [Figure]

Original: S t a g e
Revised: a g e

Original: [Figure]
Revised: [Figure]

Original: It is worth nothing that unlike SegFormer whose decoder aggregates the features from Stage 1 to
Revised: S [Figure] a g e [Figure]

Revised: It is worth nothing that unlike SegFormer whose decoder aggregates the features from Stage 1 to

Original: Stage 4, our decoder only receives features from the last three stages.
Revised: Stage 4, our decoder only receives features from the last three stages.

Original: This is because our SegNeXtis based on convolutions.
Revised: This is because our SegNeXt is based on convolutions.

Original: The features from Stage 1 contain too much low-level information andhurts the performance.
Revised: The features from Stage 1 contain too much low-level information and hurts the performance.

Original: Besides, operations on Stage 1 bring heavy computational overhead.
Revised: Besides, operations on Stage 1 bring heavy computational overhead.

Original: In ourexperiment section, we will show that our convolutional SegNeXt performs much better than therecent state-of-the-art transformer-based SegFormer [74] and HRFormer [82].
Revised: In our experiment section, we will show that our convolutional SegNeXt performs much better than the recent state-of-the-art transformer-based SegFormer [79] and HRFormer [88].

Original:  Experiments
Revised:  Experiments

Original: Dataset.
Revised: Dataset.

Original: We evaluate our methods on seven popular datasets, including ImageNet-1K [15],
Revised: We evaluate our methods on seven popular datasets, including ImageNet-1K [14],

Original: ADE20K [92], Cityscapes [13], Pascal VOC [18], Pascal Context [53], COCO-Stuff [3], andiSAID [70].
Revised: ADE20K [98], Cityscapes [12], Pascal VOC [17], Pascal Context [57], COCO-Stuff [3], and iSAID [75].

Original: ImageNet [15] is the best-known dataset for image classification, which contains1,000 categories.
Revised: ImageNet [14] is the best-known dataset for image classification, which contains [Table] [Table] 1,000 categories.

Original: Similar to most segmentation methods, we use it to pretrain our MSCAN encoder.
Revised: Similar to most segmentation methods, we use it to pretrain our MSCAN encoder.

Original: ADE20K [92] is a challenging dataset which contains 150 semantic classes.
Revised: ADE20K [98] is a challenging dataset which contains 150 semantic classes.

Original: It consistsof 20,210/2,000/3,352 images in the training, validation and test sets.
Revised: It consists of 20,210/2,000/3,352 images in the training, validation and test sets.

Original: Cityscapes [13] mainly focuses on urban scenes and contains 5.000 high-resolution images with 19 categories.
Revised: Cityscapes [12] mainly focuses on urban scenes and contains 5.000 high-resolution images with 19 categories.

Original: There are2,975/500/1,525 images for training, validation and testing, respectively.
Revised: There are 2,975/500/1,525 images for training, validation and testing, respectively.

Original: Pascal VOC [18] involves[Table] [Table] 20 foreground classes and a background class.
Revised: Pascal VOC [17] involves 20 foreground classes and a background class.

Original: After augmentation, it has 10, 582/1, 449/1, 456images for training, validation and testing, respectively.
Revised: After augmentation, it has 10, 582/1, 449/1,images for training, validation and testing, respectively.

Original: Pascal Context [53] contains 59 foregroundclasses and a background class.
Revised: Pascal Context [57] contains 59 foreground classes and a background class.

Original: The training set and validation set contain 4,996 and 5,104 images,respectively.
Revised: The training set and validation set contain 4,996 and 5,104 images, respectively.

Original: COCO-Stuff
Revised: COCO-Stuff

Original: [3] is also a challenging benchmark, which contains 172 semantic categories and 164k images in total.
Revised: [3] is also a challenging benchmark, which contains 172 semantic categories and 164k images in total.

Original: iSAID [70] is a large-scale aerial image segmentation benchmark,which includes 15 foreground classes and a background class.
Revised: iSAID [75] is a large-scale aerial image segmentation benchmark, which includes 15 foreground classes and a background class.

Original: Its training, validation and test setsseparately involve 1,411/458/937 images.
Revised: Its training, validation and test sets separately involve 1,411/458/937 images.

Original: Implementation details.
Revised: Implementation details.

Revised: We conduct experiments by using Jittor [30] and Pytorch [60].

Original: Our implementation is based onthe timm (Apache-2.0) [71] and mmsegmentation (Apache-2.0) [12] libraries for classification and segmentation, respectively.
Revised: Our implementation is based on timm (Apache-2.0) [76] and mmsegmentation (Apache-2.0) [11] libraries for classification and segmentation, respectively.

Original: All encodersof our segmentation models are pretrained on the ImageNet-1K dataset [15].
Revised: All encoders of our segmentation models are pretrained on the ImageNet-1K dataset [14].

Original: We adopt Top-1 accuracy and mean Intersection over Union (mIoU) as our evaluation metrics for classification andsegmentation, respectively.
Revised: We adopt Top-1 accuracy and mean Intersection over Union (mIoU) as our evaluation metrics for classification and segmentation, respectively.

Original: All models are trained on a node with 8 RTX 3090 GPUs.
Revised: All models are trained on a node with 8 RTX 3090 GPUs.

Original: For ImageNet pretraining, our data augmentation method and training settings are the same as
Revised: For ImageNet pretraining, our data augmentation method and training settings are the same as

Original: DeiT [64].
Revised: DeiT [69].

Original: For segmentation experiments, we adopt some common data augmentation includingrandom horizontal flipping, random scaling (from 0.5 to 2) and random cropping.
Revised: For segmentation experiments, we adopt some common data augmentation including random horizontal flipping, random scaling (from 0.5 to 2) and random cropping.

Original: The batch sizeis set to 8 for the Cityscapes dataset and 16 for all the other datasets.
Revised: The batch size is set to 8 for the Cityscapes dataset and 16 for all the other datasets.

Original: AdamW [49] is applied totrain our models.
Revised: AdamW [53] is applied to train our models.

Original: We set the initial learning rate as 0.00006 and employ the poly-learning rate decaypolicy.
Revised: We set the initial learning rate as 0.00006 and employ the poly-learning rate decay policy.

Original: We train our model 160K iterations for ADE20K, Cityscapes and iSAID datasets and 80Kiterations for COCO-Stuff, Pascal VOC and Pascal Context datasets.
Revised: We train our model 160K iterations for ADE20K, Cityscapes and iSAID datasets and 80K iterations for COCO-Stuff, Pascal VOC and Pascal Context datasets.

Original: During testing, we use both thesingle-scale (SS) and multi-scale (MS) flip test strategies for a fair comparison.
Revised: During testing, we use both the single-scale (SS) and multi-scale (MS) flip test strategies for a fair comparison.

Original: More details can befound in our supplementary materials.
Revised: More details can be found in our supplementary materials.

Original: 4.1 Encoder Performance on ImageNet
Revised: 4.1 Encoder Performance on ImageNet

Original: ImageNet pretraining is a common strategy for training segmentation models [88, 6, 74, 82, 5].
Revised: ImageNet pretraining is a common strategy for training segmentation models [94, 5, 79, 88, 4].

Original: Here,we compare the performance of our MSCAN with several recent popular CNN-based and transformerbased classification models.
Revised: Here, we compare the performance of our MSCAN with several recent popular CNN-based and transformerbased classification models.

Original: As shown in Tab. 3, our MSCAN achieves better results than the recentstate-of-the-art CNN-based method, ConvNeXt [47] and outperforms popular transformer-basedmethods, like Swin Transformer [46] and MiT, the encoder of SegFormer [74].
Revised: As shown in Tab. 3, our MSCAN achieves better results than the recent state-of-the-art CNN-based method, ConvNeXt [51] and outperforms popular transformer-based methods, like Swin Transformer [50] and MiT, the encoder of SegFormer [79]. [Table]

Original: 4.2 Ablation study
Revised: 4.2 Ablation study

Original: Global Context for Decoder.
Revised: Ablation on MSCA design.

Original: Decoder plays an important role in integrating global context frommulti-scale features for segmentation models.
Revised: We conduct ablation study on MSCA design on both ImageNet and ADE20K dataset. K × K branch contains a depth-wise 1 × K convolution and a K × 1 depth-wise convolution. 1 × 1 conv means the channel mixing operation.

Original: Here, we investigate the influence of different globalcontext modules on decoder. As shown in most previous works [69, 19], attention-based decodersachieves better performance for CNNs than pyramid structures [88, 5], we thus only show the results
Revised: Attention means the element-wise product, which makes the network obtain adaptive ability. Results are shown in Tab. 6. We can find that each part contributes to the final performance.

Original: [Table]
Revised: [Table]

Original:  using attention-based decoders.
Revised: Global Context for Decoder. Decoder plays an important role in integrating global context from multi-scale features for segmentation models. Here, we investigate the influence of different global context modules on decoder. As shown in most previous works [74, 18], attention-based decoders achieves better performance for CNNs than pyramid structures [94, 4], we thus only show the results using attention-based decoders.

Original: Specifically, we show results with 4 different types of attentionbased decoders, including non-local (NL) attention [69] with O ( n 2 ) complexity and CCNet 
Revised: Specifically, we show results with 4 different types of attentionbased decoders, including non-local (NL) attention [74] with O ( n 2 ) complexity and CCNet [33],

Original: EMANet [39], and HamNet [21] with O ( n ) complexity.
Revised: EMANet [39], and HamNet [20] with O ( n ) complexity.

Original: As shown in Tab. 5, Ham achieves the besttrade-off between complexity and performance.
Revised: As shown in Tab. 5, Ham achieves the best trade-off between complexity and performance.

Original: Therefore, we use Hamburger [21] in our decoder.
Revised: Therefore, we use Hamburger [20] in our decoder.

Original: [Table]
Revised: [Table]

Original: Decoder Structure.
Revised: Decoder Structure.

Original: Unlike image classification, segmentation models need high-resolution outputs.
Revised: Unlike image classification, segmentation models need high-resolution outputs.

Original: We ablate three different decoder designs for segmentation, all of which have been shown in Fig.
Revised: We ablate three different decoder designs for segmentation, all of which have been shown in Fig.

Original: The corresponding results are listed in Tab. 6.
Revised: The corresponding results are listed in Tab. 7.

Original: We can see that SegNeXt (c) achieves the bestperformance and the computational cost is also low.[Table]
Revised: We can see that SegNeXt (c) achieves the best performance and the computational cost is also low.

Original: Importance of Our MSCA.
Revised: Importance of Our MSCA.

Original: Here, we conduct experiments to demonstrate the importance of
Revised: Here, we conduct experiments to demonstrate the importance of

Original: MSCA for segmentation.
Revised: MSCA for segmentation.

Original: As a comparison, we follow VAN [24] and replace the multiple branches inour MSCA with a single convolution with a large kernel.
Revised: As a comparison, we follow VAN [23] and replace the multiple branches in our MSCA with a single convolution with a large kernel.

Original: As shown in Tab. 7 and Tab. 3, we canobserve that though the performance of the two encoders is close in ImageNet classification, SegNeXtw/ MSCA yields much better results than the setting w/o MSCA.
Revised: As shown in Tab. 8 and Tab. 3, we can observe that though the performance of the two encoders is close in ImageNet classification, SegNeXt [Table] SegNeXt [Table]

Revised: SegFormer-B

Original:  This indicates that aggregatingmulti-scale features is crucial in encoder for semantic segmentation.
Revised: SegNeXt-B SegFormer-B SegNeXt-B w/ MSCA yields much better results than the setting w/o MSCA. This indicates that aggregating multi-scale features is crucial in encoder for semantic segmentation.

Original: 4.3 Comparison with state-of-the-art methods
Revised: 4.3 Comparison with state-of-the-art methods

Original: In this subsection, we compare our method with state-of-the-art CNN-based methods, such as
Revised: In this subsection, we compare our method with state-of-the-art CNN-based methods, such as

Original: HRNet [65], ResNeSt [86], and EfficientNet [63], and transformer-based methods, like Swin Transformer [46], SegFormer [74], HRFormer [82], MaskFormer [11], and Mask2Former [10].
Revised: HRNet [70], ResNeSt [92], and EfficientNet [68], and transformer-based methods, like Swin Transformer [50], SegFormer [79], HRFormer [88], MaskFormer [10], and Mask2Former [9].

Original: Performance-computation trade-off.
Revised: Performance-computation trade-off.

Original: ADE20K and Cityscapes are two widely used benchmarksin semantic segmentation.
Revised: ADE20K and Cityscapes are two widely used benchmarks in semantic segmentation.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, we plot the performance-computation curves ofdifferent methods on the Cityscape and ADE20K validation set.
Revised: 1, we plot the performance-computation curves of different methods on the Cityscape and ADE20K validation set.

Original: Clearly, our method achieves thebest trade-off between performance and computations compared to other state-of-the-art methods,like SegFormer [74], HRFormer [82], and MaskFormer [11]. [Figure] [Table] [Table]
Revised: Clearly, our method achieves the best trade-off between performance and computations compared to other state-of-the-art methods, like SegFormer [79], HRFormer [88], and MaskFormer [10].

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: Comparison with state-of-the-art transformers.
Revised: Comparison with state-of-the-art transformers.

Original: We compare SegNeXt with state-of-the-art transformer models on the ADE20K, Cityscapes, COCO-Stuff and Pascal Context benchmarks.
Revised: We compare SegNeXt with state-of-the-art transformer models on the ADE20K, Cityscapes, COCO-Stuff and Pascal Context benchmarks.

Original: As shownin Tab. 8, SegNeXt-L surpasses Mask2Former with Swin-T backbone by 3.3 mIoU (51.0 v.s. 47.7)with similar parameters and computational cost on he ADE20K dataset.
Revised: As shown in Tab. 9, SegNeXt-L surpasses Mask2Former with Swin-T backbone by 3.3 mIoU (51.0 v.s. 47.7) with similar parameters and computational cost on he ADE20K dataset.

Original: Moreover, SegNeXt-B yields2.0 mIoU improvement (48.5 v.s. 46.5) compared to SegFormer-B2 using only 56% computationson the ADE20K dataset.
Revised: Moreover, SegNeXt-B yields 2.0 mIoU improvement (48.5 v.s. 46.5) compared to SegFormer-B2 using only 56% computations on the ADE20K dataset.

Original: In particular, since the self-attention in SegFormer [74] is of quadraticcomplexity w.r.t.
Revised: In particular, since the self-attention in SegFormer [79] is of quadratic complexity w.r.t.

Original: , the input size while our method uses convolutions, this makes our method performgreatly well when dealing with high-resolution images from the Cityscapes dataset.
Revised: , the input size while our method uses convolutions, this makes our method perform greatly well when dealing with high-resolution images from the Cityscapes dataset.

Original: For instance,
Revised: For instance,

Original: SegNeXt-B gains 1.6 mIoU (81.0 v.s. 82.6) over SegFormer-B2 but uses 40% less computations.
Revised: SegNeXt-B gains 1.6 mIoU (81.0 v.s. 82.6) over SegFormer-B2 but uses 40% less computations.

Original: In Fig.
Revised: In Fig.

Original: 4, we also show a qualitative comparison with SegFormer.
Revised: 4, we also show a qualitative comparison with SegFormer.

Original: We can see that thanks to theproposed MSCA, our method recognizes well when processing object details.
Revised: We can see that thanks to the proposed MSCA, our method recognizes well when processing object details.

Original: Comparison with state-of-the-art CNNs.
Revised: Comparison with state-of-the-art CNNs.

Original: As shown in Tab. 4, Tab. 9, and Tab. 11, we compare our SegNeXt with state-of-the-art CNNs such as ResNeSt-269 [86], EfficientNet-L2 [93], and HRNetW48 [65] on the Pascal VOC 2012, Pascal Context, and iSAID datasets.
Revised: As shown in Tab. 4, Tab. 10, and Tab. 12, we compare our SegNeXt with state-of-the-art CNNs such as ResNeSt-269 [92], EfficientNet-L2 [99], and HRNetW48 [70] on the Pascal VOC 2012, Pascal Context, and iSAID datasets.

Original: SegNeXt-L outperforms thepopular HRNet (OCR) [65, 81] model (60.3 v.s. 56.3) using even less parameters and computations,which is elaborately designed for the segmentation task.
Revised: SegNeXt-L outperforms the popular HRNet (OCR) [70, 87] model (60.3 v.s. 56.3) using even less parameters and computations, which is elaborately designed for the segmentation task.

Original: Moreover, SegNeXt-L performs even betterthan EfficientNet-L2 (NAS-FPN), which is pretrained on additional 300 million unavailable images,on the Pascal VOC 2012 test leaderboard.
Revised: Moreover, SegNeXt-L performs even better than EfficientNet-L2 (NAS-FPN), which is pretrained on additional 300 million unavailable images, on the Pascal VOC 2012 test leaderboard.

Original: It is worth noting that EfficientNet-L2 (NAS-FPN) has485M parameters, while SegNeXt-L has only 48.7M parameters. 
Revised: It is worth noting that EfficientNet-L2 (NAS-FPN) has 485M parameters, while SegNeXt-L has only 48.7M parameters. [Table] [Table]

Revised: [Table]

Original: Comparison with real-time methods.
Revised: Comparison with real-time methods.

Original: In addition to the state-of-the-art performance, our method isalso suitable for real-time deployments.
Revised: In addition to the state-of-the-art performance, our method is also suitable for real-time deployments.

Original: Even without any specific software or hardware acceleration,
Revised: Even without any specific software or hardware acceleration,

Original: SegNeXt-T realizes 25 frames per second (FPS) using a single 3090 RTX GPU when dealing with [Table] [Table] [Table]
Revised: SegNeXt-T realizes 25 frames per second (FPS) using a single 3090 RTX GPU when dealing with an image of size 768 × 1,536.

Original: an image of size 768 × 1,536.

Original: As shown in Tab. 10, our method sets new state-of-the-art results forreal-time segmentation on the Cityscapes test set.
Revised: As shown in Tab. 11, our method sets new state-of-the-art results for real-time segmentation on the Cityscapes test set.

Original:  Conclusions and Discussion
Revised:  Conclusions and Discussion

Original: In this paper, we analyze previous successful segmentation models and find the good characteristicsowned by them.
Revised: In this paper [Table] we analyze previous successful segmentation models and find the good characteristics owned by them.

Original: Based on the findings, we present a tailored convolutional attention module MSCAand a CNN-style network SegNeXt.
Revised: Based on the findings [Table] we present a tailored convolutional attention module MSCA and a CNN-style network SegNeXt.

Original: Experimental results demonstrate that SegNeXt surpasses currentstate-of-the-art transformer-based methods by a considerable margin.
Revised: Experimental results demonstrate that SegNeXt surpasses current state-of-the-art transformer-based methods by a considerable margin.

Original: Recently, transformer-based models have dominated various segmentation leaderboards.
Revised: Recently, transformer-based models have dominated various segmentation leaderboards.

Original: Instead, thispaper shows that CNN-based methods can still perform better than transformer-based methods usinga proper design.
Revised: Instead, this paper shows that CNN-based methods can still perform better than transformer-based methods when using a proper design.

Original: We hope this paper could encourage researchers to further investigate the potentialof CNNs. 
Revised: We hope this paper could encourage researchers to further investigate the potential of CNNs. [Table]

Original: Our model also has its limitations, for example, extending this method to large-scale models with100M+ parameters and the performance on other vision or NLP tasks.
Revised: Our model also has its limitations, for example, extending this method to large-scale models with 100M+ parameters and the performance on other vision or NLP tasks.

Original: These will be addressed in ourfuture works.
Revised: These will be addressed in our future works

Revised: [Table]

Revised:  Acknowledgment

Revised: This work was supported by the National Key R&D Program of China (No. 2021ZD0112902) and the Natural Science Foundation of China (No. 62220106003, 62176130).

Revised: We would like to thank Yi

Revised: Zhang and Zhengyang Geng for their kind help in experiments