Semantic Diffusion Network for Semantic Segmentation
Abstract
Precise and accurate predictions over boundary areas are essential for semantic segmentation.
However, the commonly used convolutional operators tend to smooth and blur local detail cues, making it difﬁcult for deep models to generate accurate boundary predictions.
In this paper, we introduce an operator-level approach to enhance semantic boundary awareness, so as to improve the prediction of the deep semantic segmentation model.
Speciﬁcally, we formulate the boundary feature enhancement process as an anisotropic diffusion process.
We propose a novel learnable approach called semantic diffusion network (SDN) for approximating the diffusion process, which contains a parameterized semantic difference convolution operator followed by a feature fusion module and constructs a differentiable mapping from original backbone features to advanced boundary-aware features.
The proposed SDN is an efﬁcient and ﬂexible module that can be plugged into existing encoder-decoder segmentation models.
Extensive experiments show that our approach can achieve consistent improvements over several typical state-of-the-art segmentation baseline models on challenging public benchmarks.
Introduction
Semantic segmentation aims at assigning pixel-wise semantic labels to an image.
It plays an important role for various applications such as autonomous driving [9, 12], medical diagnostics [32] and virtual reality [4].
Although great progress has been made in this ﬁeld since the rise of deep learning, there are still many challenges to be solved, one of the most typical of which is the fuzzy boundary prediction problem [21, 44, 60] (as shown in Figure 1).
People have been working on improving the boundary quality of semantic segmentation in three different ways: (i) adopting a post-processing module to reﬁne the boundary [26, 23, 45, 40, 60], (ii) jointly learned the edge detection and semantic segmentation task for boundary-awareness [62, 7, 31], and (iii) making training more focused on boundary by designing a boundary-aware loss with higher sensitivity to the boundary changes [3, 20, 47].
All these methods explored the boundary information via either post-processing modules or additional supervisions.
A better solution, however, is to directly address the intrinsic reason of fuzzy boundary and detail degradation, i.e. , vanilla convolution [24] naturally tends to smooth the feature [42, 55–58], making it difﬁcult to capture the boundary cues (see Figure 2b).
Although traditional gradient operators [19, 22] and difference convolutions [55–58, 42] have the ability to perceive boundary, they are sensitive to all the boundaries including intra-class texture edges, which makes it not suitable for semantic segmentation.
Our method.
This paper proposes an operator-level approach to enhance semantic boundary awareness, so as to improve the prediction of the deep semantic segmentation model.
Speciﬁcally, we formulate the boundary feature enhancement process as a partial differential model of the anisotropic diffusion process.
The classical solution of such PDEs is the ﬁnite difference method with multiple iterations, which is often unstable and rarely has satisfactory efﬁciency.
We propose a novel learnable approach called semantic diffusion network (SDN) for approximating the diffusion process, which contains a parameterized semantic difference convolution operator followed by a feature fusion module and constructs a differentiable mapping from original backbone features to advanced boundary-aware features.
The proposed SDN is an efﬁcient and ﬂexible module that can be plugged into existing encoder-decoder segmentation models.
Beneﬁting from the superior boundary awareness, our method signiﬁcantly improves the boundary quality of the segmentation and shows consistent improvements over several typical segmentation models.
In particular, it can bring +4.3% 1-px 3 boundary F-score improvements on Cityscapes-val [9] and +2.6% mIoU gains on ADE20K- val [64] for the OCRNet [59].
Related Work
Semantic Segmentation.
In 2015, FCN [18] ﬁrst proposed to use a fully convolutional network to generate dense predictions, which is regarded as a milestone for semantic segmentation.
Since then, lots of works have been exploring larger receptive ﬁelds for better scene understanding based on CNN architecture.
For example, PSPNet [15] proposed a general pyramid pooling module to fuse features at different scales.
DeepLab family [26, 27, 25, 28] used atrous convolution to enlarge the receptive ﬁelds while keeping the resolution.
Recent works focused on how to capture more precise context [59, 17, 61, 29, 54] and better boundary [60, 3].
Considering that the attention operation [48] has a large receptive ﬁelds and can adaptively aggregate features, [11, 16, 65, 2] employed the attention-based modules on the top of CNN segmentation networks to adaptively combine local features with their global dependencies.
Inspired by the success of Transformer [46] in NLP tasks, SETR [63] and DPT [36] attempted to use Vision Transformer [10] as the backbone.
They demonstrated effectiveness of global receptive ﬁelds for semantic segmentation task.
Inpirsed by [5], Tran2Seg [53] explored a hybrid encoder and prototype-based transformer decoder.
FTN [51] and Segmenter [41] further designed fully transformer networks.
SegFormer [52] proved that a lightweight MLP-decoder can also achieve satisfactory results under a strong transformer encoder.
Recently, MaskFormer [8] proposed a simple but effective mask classiﬁcation scheme to predict a set of binary masks, which uniﬁed the semantic, instance and panoptic segmentation.
Boundary-aware Segmentation.
Methods for promoting the boundary quality in semantic segmentation can be divided into three categories: (1) Adopting the post-processing module.
Most methods applied a post-processing module on the top of coarse segmentation results to reﬁne the boundary and recover the details [26, 23, 45, 40, 60].
DenseCRF model [23] is the most well-known post processing method, which achieves ﬁne boundary results by encouraging assigning neighboring pixels with similar colors to the same semantic label.
SegFix [60] proposed a model-agnostic boundary reﬁnement model, which aims to replace the ambiguous predictions at boundaries with predictions at the intra-class areas.
(2) Auxiliary branch for boundary-awareness.
[62, 7, 31] utilized the multi-task learning strategy by jointly learning boundary detection and semantic segmentation together.
For example, RPCNet [62] introduced the auxiliary branch for boundary enhancement to the semantic segmentation model.
(3) Boundary-aware loss.
These works guide the model to focus more on boundary accuracy by designing the boundary-aware loss function with high sensitivity to the boundary changes [3, 20, 47].
Inverseform [3] introduced a boundary distance-based measure into the popular segmentation loss functions.
Kervadec et.al.
[20] proposed a boundary loss taking the form of a distance metric on the space of contours for highly unbalanced situations.
ABL [47] designed an active boundary loss function for progressively encouraging the predicted boundaries matching the ground-truth boundaries.
Instead of imposing various boundary-related constraints, we consider the intrinsic reason of fuzzy boundary from the operator level, and design a boundary-sensitive operator to make the network naturally capable of modeling and perceiving boundaries.
Related Operators.
The convolution operator, who calculates the kernel’s response to each local patch of the input feature maps, is not good at modeling boundary information.
To solve this, people try to combine the traditional image gradient operator with convolution operators.
Local binary convolution (LBC) [19] used a series of pre-deﬁned binary ﬁlters to replace learnable kernels in convolution.
Subsequently, a series of difference convolution operators represented by central difference convolution (CDC) [55–58] used learnable kernels to capture useful boundary information on the central difference map.
PDC [42] further extended the ﬂexibly of CDC and achieved excellent results in boundary detection tasks.
In addition, local self-attention (LSA) [35] operator replaces the ﬁxed kernel with the content-adaptive attentive weights to achieve local feature fusion, making it good at capturing contextual dependence.
But it is still not good at perceiving the boundary.
Different from existing methods, we are committed to designing a novel and efﬁcient operator-level solution with a high sensitivity to the inter-class boundaries rather than all the edges.
Preliminaries 3.1 Diffusion Process
Diffusion is a physical model aimed at minimizing the spatial concentration difference [38], which is widely used in image procesing [34, 6, 50, 49].
Given an image tensor U to be smoothed, this process needs to solve the well-known second-order partial differential equation:
[Equation]
where ∂U∂t is the time derivative of the solution tensor, Div is the divergence operator, D is the diffusivity tensor, which describes the speed of the diffusion process, ∇ is the gradient operator.
With increasing time t , the solution U t of this process will correspond to increasingly smoothed versions of the original tensor.
The concrete form of D determines the properties of the PDE.
For D = 1 , it is called homogeneous linear diffusion, where the diffusion velocity of spatial concentration is exactly the same in all directions.
For a spatial-dependent D = D ( x ) ( e.g. learnable kernel), the process is linear inhomogeneous.
However, the two kinds of simple linear diffusion processes not only smooth the noise but also blur the edges.
An ideal diffusivity function allows more smoothing parallel to image edges and less smoothing perpendicular to these edges, which is actually anisotropic and nonlinear.
For this reason, some classic milestones [34, 6, 50, 49] increase the nonlinear modeling ability by designing complex diffusivity functions D
= D ( U ) depending on the input U .
3.2 Rethinking Operator-Level Boundary Awareness
Here we discuss the operator-level causes of potential CNN failures in boundary predictions.
Given the input feature map x , the vanilla convolution calculates the output tensor y through the inner product between the kernel w and each local feature patch.
Without loss of generality, we assume that the channel number of both input and output is 1 .
The vanilla convolution with kernel w ∈ R h × w over a input feature x ∈ R H
× W could be written as: y p
= (cid:104) w , x p (cid:105) , where x p is the local patch centering at the position p .
Intuitively, the vanilla convolution is just a local weighted average operation, which tends to smooth the local detailed information [57].
Especially when entries within [Figure] the kernel are non-negative, the smoothness will be particularly signiﬁcant as it forms a low-pass ﬁlter.
This results in a poor capacity for modeling the semantic boundary information, see Figure 2b.
Recently, people try to combine the traditional gradient operator (edge-detector) with convolution operators to enhance its edge awareness [19, 55–58, 42].
Local binary convolution (LBC) [19] used a series of pre-deﬁned binary ﬁlters to replace learnable kernels in convolution.
Subsequently, a series of difference convolution operators represented by central difference convolution (CDC) [55–58] used learnable kernels to capture useful boundary information on the central difference map, that is, y p
= (cid:104) w , ( x p − x center ) (cid:105) , where x center indicates the center entry of the current patch, and the rest symbols are deﬁned in the same way as that of the vanilla convolution.
It aggregates the center-oriented gradient within the local region instead of the feature values.
Even though this kind of operator has better edge awareness, however, because the gradient operator cannot distinguish between real semantic boundary and pseudo boundary caused by textures or noise (see Figure 2c), it is difﬁcult for us to directly apply them to tasks such as semantic segmentation.
Methodology
This section introduces the semantic diffusion network (SDN) for semantic segmentation tasks to compensate for the detail blur caused by the vanilla convolution operator.
It is combined with
CNN backbone when used, so as to strengthen semantic boundary information and ﬁlter out pseudo boundary information caused by texture or noise.
4.1 Formulation
Considering a CNN backbone model with L stacked neural stages mapping an image I to pixel-wise feature maps X .
Let X (cid:96) denote the feature map from the (cid:96) -th stage.
We deﬁne the semantic diffusion process over the feature map X (cid:96) as:
[Equation]
where ∂U∂t is the time derivative of the solution tensor, Div is the divergence operator, ∇ is the gradient operator.
V is the guidance feature map with the same resolution of U .
The diffusivity term is a monotonically decreasing function of the square of the gradient |∇ V | 2 , for example, g ( |∇ V | 2 )
= 1 / (cid:112) 1 + |∇ V | 2 /λ 2 .
Ideally, the diffusion close to the semantic boundary areas will be suppressed, while the diffusion far away from the semantic boundary will be accelerated.
Therefore,
V should contain sufﬁcient semantic information, so the gradient norm |∇ V | 2 can be used as a reliable semantic boundary indicator to construct the diffusivity.
The PDE of the non-linear diffusion process deﬁned by Eq.(2) could be approximately solved by performing the ﬁnite difference update rule for T steps:
[Equation]
where the step index t ∈ { 0 , ..., T − 1 } , the tuple ( i, j ) enumerates the index of every entry within the local neighbor region N ( m,n ) of the position ( m, n ) , and the initial state is U
= X (cid:96) .
In fact, it is obvious that the diffusivity g ( | V ( i,j ) − V
( h,w )
| 2 ) calculates the similarity between semantic guidance features at ( i, j ) and ( m, n ) .
The solver builds a differentiable map from the input feature map X (cid:96) to the boundary-enhanced feature.
Remark:
The ﬁnite difference based solver can hardly be directly integrated with deep segmentation models for several reasons: (1).
the stability of the ﬁnite difference method depends on the setting of boundary conditions and the selection of parameters ( α, β ) ; (2).
the solver requires multiple iterations, which will consume high computational complexity.
integrating the iterative solver with the deep networks would result in numerical instability to the training process, such as gradient explosion [40].
4.2 Semantic Diffusion Network
This subsection describes a novel learnable approach called semantic diffusion network (SDN) for approximating the diffusion process, which contains a parameterized semantic difference convolution operator followed by a feature fusion module and constructs a differentiable mapping from original backbone features to advanced boundary-aware features.
As a learnable PDE approximate solver, the well-trained SDN could approximate the diffusion process in only one forward computation, reducing the computational complexity and improving the stability.
[Figure]
𝒅 𝟎, ⊗ 𝒙⊗ 𝒅 𝟏,𝟒 𝒅 𝟐, 𝒅 𝟑,𝟒 𝒅 𝟒,𝟒 𝒅 𝟓, 𝒅 𝟔,𝟒 𝒅 𝟕,𝟒 𝒅 𝟖, 𝒙 𝟏 𝒙𝒙 𝟑 𝒙 𝟒 𝒙𝒙 𝟔 𝒙 𝟕 𝒙𝒘 𝟎 𝒘 𝟏 𝒘𝒘 𝟑 𝒘 𝟒 𝒘𝒘 𝟔 𝒘 𝟕 𝒘𝒘 𝟎 𝒘 𝟏 𝒘𝒘 𝟑 𝒘 𝟒 𝒘𝒘 𝟔 𝒘 𝟕 𝒘(b) Central Difference Convolution 𝒅 𝟎, ⊗ 𝒅 𝟏,𝟒 𝒅 𝟐, 𝒅 𝟑,𝟒 𝒅 𝟒,𝟒 𝒅 𝟓, 𝒅 𝟔,𝟒 𝒅 𝟕,𝟒 𝒅 𝟖, 𝒘 𝟎 𝒘 𝟏 𝒘𝒘 𝟑 𝒘 𝟒 𝒘𝒘 𝟔 𝒘 𝟕 𝒘(c) Semantic Difference Convolution ⊗ 𝒔 𝟎,𝟒 𝒔 𝟏,𝟒 𝒔 𝟐, 𝒔 𝟑,𝟒 𝒔 𝟒,𝟒 𝒔 𝟓, 𝒔 𝟔,𝟒 𝒔 𝟕,𝟒 𝒔 𝟖, : pixel feature 𝒘 𝒊 : learnable weight 𝒙 𝒊 : pixel-wise difference between 𝑥 2 and 𝑥 3 𝒅 𝒊,𝒋 : semantic similaritybetween 𝑥 2 and 𝑥 3 𝒔 𝒊,𝒋
Semantic Difference Convolution.
Inspired on Eq.(3), we develop the semantic difference convolution (SDC) operator.
We set the N m,n as a h × w local region centered at position ( m, n ) .
SDC takes the feature map U and the semantic guidance V as inputs and calculates the output feature map
Y .
Speciﬁcally, each component of y indexed by ( m, n ) is calculated via
[Equation]
where the tuple ( i, j ) enumerates the index of every entry within the local neighbor region N m,n .
In the right hand of Eq.(5), the ﬁrst term is the learnable convolutional kernel W , of which (hight, width) =
( h, w ) .
The kernel introduces a more powerful modeling capability than the naive non-linear diffusion process in Eq.(2).
The second term S (cid:0) V ( i,j ) , V ( m,n ) (cid:1) , called the semantic similarity term, is a scalar measuring the similarity between semantic guidance features at ( i, j ) and ( m, n ) .
This term corresponds to the diffusivity g ( | V ( i,j ) − V
( h,w )
| 2 ) in Eq.(3).
The third term ( U ( i,j ) − U ( m,n ) ) , called the pixel difference term, calculates the feature difference between features at the position ( i, j ) and ( m, n ) .
Note that Eq.(5) can be implemented efﬁciently using CUDA.
Figure 3 provides the comparison between SDC with other well-known convolution operators.
As deﬁned in Eq.(5), the proposed SDC operator contains two sequential steps: (i) calculating the semantic similarity, and (ii) calculating the output feature.
The ﬁrst step consumes a complexity of
O ( HWhwC f ) , and the second step consumes a complexity of O ( HWhwC i C o ) .
Thus, the overall complexity is Complexity
= O
( HWhw ( C i C o + C f )) , where H, W are the height and width of the input feature, and h, w are the height and width of the kernel tensor.
C i , C o , and C f are the channel number of input, output, and semantic features, respectively. [Equation] [Figure] [Equation]
[Figure]
Feature Fusion.
Instead of using the simple weighted addition operation for feature fusing in Eq.(4), we develop a learnable feature fusion module for fusing the input feature map U and the SDC output Y ,
[Equation]
Note that both U and Y should have the same shape.
Otherwise, we need to perform the bilinear interpolation upsampling operation on the smaller one to align the shapes.
First, the SDC’s feature map Y would be fed into a ReLU [13] activation function and the subsequent batch normalization (BN) [39] layer to produce the processed feature map (cid:101) Y via Eq.(6).
Finally, Eq.(7) concatenates the original input feature U and the processed feature (cid:101) Y and uses another 1 × 1 convolution to achieve channel-wise feature fusion, resulting in the boundaryenhanced feature F .
Overall Pipeline.
The pipeline of the proposed SDN is illustrated in Fig. Here, we introduce how to make our SDN applicable to all the existing encoder-decoder segmentation models without breaking their original design.
Considering an encoderdecoder segmentation model, the input image I ∈ R 3 × H × W is ﬁrst passed through the encoder part with totally L scales (generally L = 4 ), resulting in the output features { X (cid:96) } L(cid:96)
=1 .
We incorporate the SDN after each backbone stage to enhance the boundary details for all the decoder input features.
For the
SDC operator in each SDN module, we set the kernel size as 3 × 3 , the dilation rate as 1, the stride size as 1, and the padding size as 1.
Finally, the boundary-enhanced features are fed into the decoder to generate the pixel-wise probability map.
During the training stage, we apply the cross entropy loss between the predicted and target probability map.
Experiments
In this section, we ﬁrst introduce the datasets and implementation details in Sec. 5.1.
Then, we evaluate our approach over several baseline segmentation models on these public benchmarks in Sec.
Finally, in Sec. 5.3, we conduct ablation studies to analyze the effect of key designs and the compatibility of our approach.
5.1 Experimental Setup
Datasets.
Experiments are conducted on two widely-used public benchmarks: Cityscapes [9] is a real-world scene parsing benchmark, which contains over 5000 urban scene images with 19 classes.
The number of images for training, validation, and testing are 2975, 500, and 1525, respectively.
ADE20K [64] is a very challenging benchmark including 150 categories, which is split intoand 2000 images for training and validation.
Baseline models.
In order to evaluate the effectiveness and generalization of our approach, we apply our approach to two typical CNN-based segmentation models (DeepLabV3 [25] and OCRNet [59]) [Table] and the state-of-the-art transformer-based method (MaskFormer [8]).
We test these baseline methods with different backbone choices: DeepLabV3 with ResNet-101 [14]; OCRNet with ResNet-101 and
HRNet-48 [43] [Table] and MaskFormer with ResNet-101 and Swin Transformer [30].
Training.
We follow the previous works [25, 59, 8] to set up the training strategies for fair comparisons.
All the experiments are implemented with PyTorch [1] and conducted on 4 NVIDIA V
GPUs.
Synchronized BN [33] is used to synchronize the mean and standard-deviation of BN [39] across multiple GPUs.
The data augmentation consists of three steps: (i) random horizontal ﬂip, (ii) random scale with a factor ranged from [0 . 5 , 2 . 0] , (iii) random crop.
We initialize the encoder by the
ImageNet-1K [37] pre-trained parameters, and other parts randomly.
Evaluation.
We adopt the widely-used mean intersection of union (mIoU) to measure the overall segmentation performance.
In order to measure the boundary quality of the prediction, we compute
F-score only on the 1px and 3px boundary region 3 , which denotes the 1 pixel and 3 pixels width area along the ground-truth boundary.
For the multi-scale inference, we follow the previous works [59, 8] to perform horizontal ﬂip and average the predictions at multiple scales [0.5, 0.75, 1.0, 1.25, 1.5, 1.75].
Unless otherwise stated, the reported results of our methods are the average score ofindependent runs.
5.2 Main Results
To verify the effectiveness and generalization of our approach, we apply the proposed SDN to several baseline segmentation models on various datasets.
Table 1 reports the quantitative comparisons.
[Table]
Cityscapes.
Table 1(a) shows that our approach brings nontrivial mIoU improvements for all the baselines, i.e. , improves MaskFormer by +0.9% and OCRNet by +1.0% on
Cityscapes- val , We also compare the performance of only boundary region in Table 2.
Our method brings signiﬁcant improvement to the boundary prediction accuracy, i.e. , +4.3 and +1.5 for
OCRNet in terms of the 1px and 2px F-score.
Fig.
5 provides some typical visualizations to show the boundary improvement effect of our method.
Moreover, our approach (84.7% mIoU) surpasses the original OCRNet (83.0% mIoU) by 1.7% and outperforms the OCRNet+SegFix (84.5% mIoU) by +0.2% on Cityscapes- test , whiere the detailed results can be found in the web-link.
ADE20K.
As shown in Table 1(b), the improvements seem more signiﬁcant on the challenging
ADE20K dataset, which could be attributed to the fact that explicit boundary information is particularly useful for ADE20K images with complex scenarios.
With the HRNet-48, ResNetand Swin-B backbone, we contributes +2.6%, +1.7% and +1.1% mIoU gain over OCRNet and
MaskFormers, respectively.
Fig.
6 illustrates some typical visualization results on ADE20K- val , which proves the good boundary improvement ability of our method.
5.3 Further Study
Compare SDC with other operators.
For direct comparisons with other related operators, we replace the semantic difference convolution operator (SDC) in each SDN module with the vanilla convolution [24] and CDC [57], and compare their performance on ADE20K- val .
We choose the
OCRNet with HRNet-48 backbone as the base encoder-decoder model.
Experimental results are reported in Table 3(a).
We can conclude that the SDC in the SDN module cannot be replaced with other classic convolution operators.
Complexity.
Table 3(b) compares the efﬁciency of the entire network before and after applying our approach.
We measure the number of parameters, GPU memory, computation complexity(in FLOPs), and inference time.
The numbers are obtained on a single NVIDIA V100 GPU with CUDA 10.1.
Obviously, the extra complexity of our approach only takes a small proportion of the whole network.
For example, it only brings 8.7ms time cost, 0.1G FLOPs, +3.1M parameters, and +12.4M memory over the baseline model. [Table] [Table]
Compatibility with other boundary promoting methods.
Our approach can be incorporated with other boundary improvement methods, for example, using our SDN and a post-processing boundary reﬁnement module simultaneously in a network.
Table 3(c) presents the good compatibility of our approach with three well-known methods on Cityscapes- val .
Speciﬁcally, the additional
DenseCRF [23], SegFix [60] and InverseForm [3] further bring +0 . 2 , +0 . 6 and +1 .
0 boundary
F-score gains, respectively.
It is worth noting that, our method can directly achieve +4 .
4 boundary
F-score improvement over the baseline without any other methods.
Beneﬁting from the intrinsic boundary exploration, segmentation networks with our SDN can relieve the dependence on other boundary improvement techniques.
Effect of SDN in each stage.
By default, we add SDN to the output of all the encoder stages.
Fig.visualizes the feature before/after the SDN module at each stage.
Obviously, comparing the output features of the encoder, the SDN feature has a clear response at the boundary.
To study the impact of our SDN on each stage, we successively deploy SDN from stage-4 to stage-1 of OCRNet and conduct experiments on ADE20K- val .
As shown in Table 4a, the stage-2 has the greatest demand for SDN.
Effect of kernel settings of SDC.
We conduct experiments under different choices of the kernel size and dilation rate of SDC in Table 4b. The experiments is also conducted on ADE20K with OCRNet as the base model. It can be found that: (1). Increasing the size of the convolution kernel in SDC does not bring signiﬁcant changes (48.3 → 48.3).
Excessive dilation rate leads to negative effects (48.3 → 48.1 → 47.4). In fact, large kernels and dilated convolution are mainly used for enlarging the receptive ﬁelds. While, the goal of our SDC is to infer the inter-class boundary cue in a local region, which may not require more contexts brought by large kernels. Similarly, a large dilation rate may lead to the lack of local information, where, however, the local information is critical for local boundary exploration.
Conclusion
This paper proposes the semantic diffusion network (SDN), a novel operator-level approach for capturing boundary-aware features.
The proposed SDN is an efﬁcient, stable, and ﬂexible module that can be plugged into any existing encoder-decoder segmentation model.
Extensive experiments show that our approach can achieve consistent improvement over various baseline segmentation models and signiﬁcantly improve the boundary quality of segmentation results.
However, this paper only veriﬁes the effect of SDN in distinguishing semantic category boundaries at the feature level on the semantic segmentation task.
In the recent future, we will explore the application of SDN in other tasks, such as instance segmentation, image de-noising, image super-resolution, and so on.
Social Impact.
SDN has potential value in important applications such as autonomous driving and medical image analysis.
But it can also deploy inhumane surveillance.
The potential negative effects can be avoided by implementing strict and secure regulations. Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[Yes] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[Yes] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[Yes] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[Yes] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[Yes]