Despite the success of convolutional neural networks, selecting the optimal architecture for a given task remains an open problem. Instead of aiming to select a single optimal architecture, we propose a "fabric" that embeds an exponentially large number of CNN architectures. The fabric consists of a 3D trellis that connects response maps at different layers, scales, and channels with a sparse homogeneous local connectivity pattern. The only hyper-parameters of the model (nr. of channels and layers) are not critical for performance. While individual CNN architectures can be recovered as paths in the trellis, the trellis can in addition ensemble all embedded architectures together, sharing their weights where their paths overlap. The trellis parameters can be learned using standard methods based on back-propagation, at a cost that scales linearly in the fabric size. We present benchmark results competitive with the state of the art for image classification on MNIST and CIFAR10, and for semantic segmentation on the Part Labels dataset. Scales Output Figure 1 : Trellis embedding of two seven-layer CNNs (red, green) and a ten-layer deconvolutional network (blue). Feature map size of the CNN layers are given by height. Trellis nodes receiving the input and producing output are encircled. All edges are oriented to the right, down in the first layer, and towards the output in the last layer. The channel dimension of the 3D trellis is omitted for clarity. All chain-structured (de)convolutional architectures embedded in the fabric can be recovered appropriately setting certain weights to zero, so that only a single activation path is non-zero along the scale and layer axes. General non-path weight settings correspond to ensembling many architectures together, which share parameters where edges overlap. The acyclic trellis structure allows for learning using standard error back-propagation methods. Learning can thus efficiently configure the fabric to implement each one of exponentially many architectures and, more generally, mixtures of all of them. Experimental results competitive with the state of the art validate the effectiveness of our approach. Our contributions are: (1) Our fabric allows to by and large sidestep the CNN model architecture selection problem. Avoiding explicitly training and evaluating individual architectures using, e.g., greedy local-search strategies [3] . (2) While scaling linearly in terms of computation and memory requirements, our approach leverages exponentially many chain-structured architectures in parallel by massively sharing weights between them. (3) Since our fabric is multi-scale by construction, it can naturally generate output at multiple resolutions, e.g. for image classification and semantic segmentation, by connecting a prediction layer to nodes at different scales in the last layer. Below, we first discuss related work, and then present our trellis model in Section 3. We present a selection of our experimental results in Section 4 (more in Appendix A), and conclude in Section 5. The chain-structured CNN architecture used by Krizhevsky et al . [14] for ImageNet classification is widely used for many other vision tasks. Other widely adopted architectures are the VGG-16 and VGG-19 networks of Simonyan and Zisserman [27] . Although effective for many tasks, it is by no means clear that these architectures are among the best ones given their computational and memory requirements. Their widespread adoption is at least in part due to the lack of more effective techniques to find good architectures than extremely costly brute-force exhaustive or local search [3] . Many of the (de)convolutional neural networks used for semantic segmentation, as well as other structured prediction tasks such as pose estimation [25] , are often based on the CNN architectures developed for image classification of [14, 27] , see e.g. [2, 8, 18, 21, 24, 30, 32]. In addition to convolution and pooling operators, deconvolutional networks also involve up-sampling operators to increase the resolution of the predictions in the output layer [8, 24] . Ronneberger et al . [26] present a deconvolutional network with additional links that couple distant convolutional and deconvolutional layers of the same resolution. Multi-scale architectures are commonly used for semantic segmentation [2, 4, 18] . They re-sample the input image to several resolutions, process them independently for several layers, and then fuse all streams by up-sampling to the maximum resolution. The streams are then fused, and passed through several more layers to produce the final output. Such architectures can be thought of as a tree that has leafs in different re-scaled input images, and the root at the output. In the next section we show that nearly all of the (de)convolutional networks discussed above are embedded in our fabric, either as paths or other simple sub-graphs.
The Neural Fabrics REF embeds an exponentially large number of architectures with 3 × 3 filters.
Convolutional Neural Fabrics
ArXiv
ArXiv
Computer Science