Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper introduces one light-weight feature decoder (NRD) for semantic segmentation. Compared with the dilated encoder, the 
proposed NRD is effective for semantic segmentation due to it can be directly applied on the 
standard feature encoding network. In the experimental validation, this work shows satisfied performance with much less computational cost. 
main_review: Strengths: 
The core idea of NRD firstly exploits one dynamic filter networks to generate the data-driven parameters of  the dynamic neural representational decoder. Then, the generated dynamic neural representational decoder both considers the low-level features and the coordinates to perform the feature upsampling. Then, the proposed NRD takes the advantages of the coordinates, the low-level features and the high-level features to make the dense prediction for semantic segmentation. 

Weaknesses:
1). Lack of speed analysis, the experiments have compared GFLOPs of different segmentation networks. However, there is no comparisons of inference speed between the proposed network and prior work. The improvement on inference speed will be more interesting than reducing FLOPs.
2). For the detail of the proposed NRD, it is reasonable that the guidance maps are generated from the low-level feature maps. And the guidance maps can be predicted from the the first-stage feature maps or the second-stage feature maps. It is better to provide one ablation study about the effect for them. 
3). Important references are missing. The GFF[1] and EfficientFCN[2] both aims to implement the fast semantic segmentation method in the encode-decoder architecture. I encourage the authors to have a comprehensive comparison with these work.

[1]. Gated Fully Fusion for Semantic Segmentation, AAAI'20.
[2]. EfficientFCN: Holistically-guided Decoding for Semantic Segmentation, ECCV'20.
limitations_and_societal_impact: See above. The societal impact is shown one the last page of the manuscript.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the authors design a simple and effective decoder named NRD for the Semantic Segmentation task. They represent the local patch of the label map with a neural network, which makes it possible to restore details in the prediction with a low computational cost. Experiments on three benchmarks show the effectiveness of the proposed approach.
main_review: In this paper, the authors propose a computationally friendly decoder NRD. From the tables in the paper, it seems that NRD could achieve a somehow good result with a lower cost. However, the positioning of this paper is strange. On the one hand, in neither of the 3 datasets (ADE, Context, and City) does NRD achieve state-of-the-art performance. If the performance is SOTA and the computational cost is lower, it will convince me. On the other hand, the proposed method seems to be far from real-time ones and no speed metric is reported in the paper.

Besides, I have some questions about the paper.
1.	Missing comparisons. When compared with state-of-the-art, some methods are ignored. For example, SFNet [32] is listed in Table 5 and 7, but it does not show up in Table 6. According to their paper, SFNet achieves 53.8 (50.7) with ResNet-101 (ResNet-50) on Pascal Context (60 classes), which is much higher than the proposed method. For the 59-class setting, some methods even achieve more than 56 mIoU, such as OCRNet.
2.	Ablation study. In Section 3.1, when compared with DeepLabv3+, the authors do not use dilation convolutions in the backbone, resulting in output stride = 32. How about using output stride = 16 or 8? I think the ASPP module in DeepLabv3+ needs a relatively larger feature map. If the input image is 512x512, then the output feature map is 16x16 for output stride = 32.
3.	How the proposed NRD works. In table 4, only when the guidance map and the coordinate map both be used do you obtain the best performance, e.g., 79.8 mIoU. Could the authors explain how these two items improve the performance in a more detailed manner? For example, how the x-y coordinate map helps the guidance map?  
4.	Implementation details. In Section 2.2, could explain how the 1/4 resolution feature map be converted to the guidance map m that has a shape of (s*s*C_m)?
5.	Line 289, the saying “better speed-accuracy tradeoff” is wrong because GFlops does not reflect the speed. You should test the speed and report the FPS number.

limitations_and_societal_impact: The conclusion part in the introduction section is too redundant. The authors may consider simplifying it.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the authors propose a novel decoder that is effective and compact for semantic segmentation, to recover the spatial resolutions. For the first time, we represent the local label patches using neural networks and make use of dynamic convolutions to parametrize these neural networks. The experiments show on a number of semantic segmentation benchmarks that the method is highly efficient. 
main_review: This paper propose a dynamic decoder to predict the mask for a given image. Unlike previous decoder, the proposed represents each local label patch by a compact neural network g_{\theta}with a few convolution layers interleaved with non-linearities. I think the idea is good and novel. But the paper writing and description are not clear and detailed. Many details are missing or ambiguous.

My concerns and suggestions about this paper are as follows:
   1. In line 155-160, the P is one patch from groundtruth Y'. but in line 162, a compact network $g_{\theta}$ is designed to represent the local mask patch P. So the P is the input of the network $g_{\theta}$? But the input in Fig3, the input is the guidance map and coordinate map, which is ambiguous.
   2. In line 166, at first it says the network $g_{\theta}$ is applied to coordinate map Q, but in line 170, it says the guidance map m is taken as additional input. I think it's weird to describe it like this. 
   3. In line 190-191, it says "The number of output channels of the convolution is equal to the number of parameters in  $\theta$", the number of channels is C, right?    "generated parameters are then split and 191 reshaped into the weights and biases in g", what's the generates parameters' shape, how to split and reshape to weights and bias, what's the shape of weights and bias? it need to be described in detail. otherwise, it's hard to understand.
  4. in line 179, "we do not adopt this because using bilinear is sufficient when the upsampling factor is small (e.g., being 4 here).", my understanding is that the network $g_{\theta}$ is applied to coordinate map and guidance map, the coordinate map is with shape sxsx2, so how about the guidance map? please give a detailed description.
 In addition, there is no experiments for the upsampling applied or not, please add it.
   5. In table 5, 6,7, it shows DRN can achieve comparable performance with less time and computation. but in table 7, there is no more detailed comparison like DANet, HRNet, deeplabv3+. please add.





limitations_and_societal_impact: the limitation is presented in Main Review.
review_title: null

Meta-review: This paper introduces a new type of feature decoder for semantic segmentation, which is more efficient than previous work. Experimental work is ok, but could have been more convincing if architectures were achieving state-of-the-art performance.

There were a number of improvements suggested by the reviewers, in particular i) improving the clarity of the paper ii) speed comparison with previous work which should be included in the final version of the paper.
Decision: Accept (Poster)