  The current neural networks for semantic segmentation usually predict the
pixel-wise semantics on the down-sampled grid of images to alleviate the
computational cost for dense maps. However, the accuracy of resultant
segmentation maps may also be down graded particularly in the regions near
object boundaries. In this paper, we advance to have a deeper investigation on
the sampling efficiency of the down-sampled grid. By applying the spectral
analysis that analyze on the network back propagation process in frequency
domain, we discover that cross-entropy is mainly contributed by the
low-frequency components of segmentation maps, as well as that of the feature
in CNNs. The network performance maintains as long as the resolution of the
down sampled grid meets the cut-off frequency. Such finding leads us to propose
a simple yet effective feature truncation method that limits the feature size
in CNNs and removes the associated high-frequency components. This method can
not only reduce the computational cost but also maintain the performance of
semantic segmentation networks. Moreover, one can seamlessly integrate this
method with the typical network pruning approaches for further model reduction.
On the other hand, we propose to employee a block-wise weak annotation for
semantic segmentation that captures the low-frequency information of the
segmentation map and is easy to collect. Using the proposed analysis scheme,
one can easily estimate the efficacy of the block-wise annotation and the
feature truncation method.
