Paper:
DifNet: Semantic Segmentation by Diffusion Networks
Deep Neural Networks (DNNs) have recently shown state of the art performance on semantic segmentation tasks, however, they still suffer from problems of poor boundary localization and spatial fragmented predictions. The difficulties lie in the requirement of making dense predictions from a long path model all at once, since details are hard to keep when data goes through deeper layers. Instead, in this work, we decompose this difficult task into two relative simple sub-tasks: seed detection which is required to predict initial predictions without the need of wholeness and preciseness, and similarity estimation which measures the possibility of any two nodes belong to the same class without the need of knowing which class they are. We use one branch network for one sub-task each, and apply a cascade of random walks base on hierarchical semantics to approximate a complex diffusion process which propagates seed information to the whole image according to the estimated similarities. The proposed DifNet consistently produces improvements over the baseline models with the same depth and with the equivalent number of parameters, and also achieves promising performance on Pascal VOC and Pascal Context dataset. Our DifNet is trained end-to-end without complex loss functions.

Review:
This paper proposes a method for semantic image segmentation in which a single network integrates initial per-pixel class predictions from a DeepLab V2[7]-based network with pairwise affinities derived from learned per-pixel embeddings.  The final predictions of the network are obtained by updating the initial predictions using a series of matrix multiplies by transition matrices computed efficiently from the pairwise affinities.   The main contribution of the work seems to be the use of multiple embeddings, rather than a single one, and the use of a more computationally-efficient inference procedure than some, but not all, prior works.  The authors evaluate the proposed method on the Pascal VOC and Pascal Context datasets, and compare to the Deeplab V2 [7] approach on which the proposed method is based.  Strengths:  - The entire network is trained end-to-end, directly with a cross-entropy loss on the final predictions; in contrast, prior work has relied on more complicated, ad-hoc training strategies, although in many cases still end-to-end.  - DeepLab V2 is a reasonable baseline for comparison.  - The Pascal VOC 2012 and Pascal Context datasets are reasonable choices for evaluation.  - Proposed method is much more computationally efficient than DeepLab V2 with the CRF step included, and is also more computationally efficient than a similar prior work [3].  Weaknesses:  - While the proposed method outperforms certain restricted variants of DeepLab V2 [7], it performs significantly worse than the full DeepLab V2 on both of the two datasets used for comparisons.  - The experimental results show that the proposed method outperforms a restricted variant of DeepLab V2 that does not include the MSC, COCO, and CRF improvements (that are also not used with the proposed method).  However, the proposed method has worse accuracy than the full DeepLab V2 approach, with a particularly large accuracy gap on the Pascal VOC 2012 dataset.  The authors should give results for the proposed method in combination with the MSC, COCO, and CRF improvements.  - The proposed method is very similar to two prior works cited by the authors [3, 5].  The paper should more clearly compare the proposed method to these prior works, and include them for comparison in the tables of experimental results.  - The proposed method seems to have lower accuracy than the two very similar prior works [3] and [5], and also seems to offer no advantage in terms of computational efficiency over [5].  - The paper would benefit from proofreading of the grammar.  Response to author rebuttal: I thank the authors for clarifying the computational complexity of the diffusion step, specifically the fact that it is done with the features downsampled to 64x64; this addresses my confusion regarding the reported computation times.  I appreciate the difficulty of reproducing and comparing against [3] and [5] without source code, and that the that they may also be more difficult to train than the proposed method.  While a more robust method is a valuable contribution even if it does not provider better accuracy or speed, I think a more thorough investigation than is described in the paper would be needed to show that.  Regarding the lack of evaluation with the DeepLab MSC, COCO and CRF steps, while the authors make the valid point that the MSC step and especially the CRF step increase the computational complexity, and that the COCO pretraining adds training data and therefore it may not be fair to compare COCO-pretrained models to those without it, evaluations including these steps are nonetheless important for properly comparing this method to other methods, given the existing similar work.