  The capability of image semantic segmentation may be deteriorated due to
noisy input image, where image denoising prior to segmentation helps. Both
image denoising and semantic segmentation have been developed significantly
with the advance of deep learning. Thus, we are interested in the synergy
between them by using a holistic deep model. We observe that not only denoising
helps combat the drop of segmentation accuracy due to noise, but also
pixel-wise semantic information boosts the capability of denoising. We then
propose a boosting network to perform denoising and segmentation alternately.
The proposed network is composed of multiple segmentation and denoising blocks
(SDBs), each of which estimates semantic map then uses the map to regularize
denoising. Experimental results show that the denoised image quality is
improved substantially and the segmentation accuracy is improved to close to
that of clean images. Our code and models will be made publicly available.
