Deep and Wide Multiscale Recursive Networks for Robust Image Labeling
Authors: Gary B. Huang, Viren Jain
ICLR 2014

Abstract: Feedforward multilayer networks trained by supervised learning have recently demonstrated state of the art performance on image labeling problems such as boundary prediction and scene parsing. As even very low error rates can limit practical usage of such systems, methods that perform closer to human accuracy remain desirable. In this work, we propose a new type of network with the following properties that address what we hypothesize to be limiting aspects of existing methods: (1) a `wide' structure with thousands of features, (2) a large field of view, (3) recursive iterations that exploit statistical dependencies in label space, and (4) a parallelizable architecture that can be trained in a fraction of the time compared to benchmark multilayer convolutional networks. For the specific image labeling problem of boundary prediction, we also introduce a novel example weighting algorithm that improves segmentation accuracy. Experiments in the challenging domain of connectomic reconstruction of neural circuity from 3d electron microscopy data show that these 'Deep And Wide Multiscale Recursive' (DAWMR) networks lead to new levels of image labeling performance. The highest performing architecture has twelve layers, interwoven supervised and unsupervised stages, and uses an input field of view of 157,464 voxels ($54^3$) to make a prediction at each image location. We present an associated open source software package that enables the simple and flexible creation of DAWMR networks.


Peer review:
[Review]:
This paper presents the application of a composite classification system to the classification of 3D microscopy scans. 
The problem is formulated as predicting an affinity graph (binary classification 'pixels belong to same foreground object' vs other cases).
The classifier is a composite construction from supervised and unsupervised methods: vector quantization, pooling, subsampling, whitening and a 1-hidden-layer neural network. 

Alternative classifier architectures are explored (using multiscale or single scale VQ, 

These classifiers are stacked ('recursively') with the resulting classifier performing better, though it is not clear where the improvement comes: increased depth or increased field of view.

How does this work if you make the classification MLP deeper? 

I would like to see comparisons of the different techniques in terms of number of parameters. 

Describes an interesting composite architecture on a specialist problem, with 3D data. It is interesting to see the comparison of this composite classifier approach and the effectiveness of stacking. 

Pros: 
Interesting comparison of a more hand-built approach to a CNN.
Interesting to see application to 3D data
Interesting to see the effectiveness of stacking. 

Cons: 
A somewhat specialist problem (and proprietary data) limit the audience and
applicability of the results.