Geometry of Deep Convolutional Networks
Authors: Stefan Carlsson
Keywords: convolutional networks, geometry
TL;DR: Analysis of deep convolutional networks in terms of associated arrangement of hyperplanes
Abstract:  We give a formal procedure for computing preimages of convolutional
  network outputs using the dual basis defined from the set of
  hyperplanes associated with the layers of the network. We point out
  the special symmetry associated with arrangements of hyperplanes of
  convolutional networks that take the form of regular
  multidimensional polyhedral cones. We discuss  the efficiency of of
  large number of layers of nested cones that result from incremental
  small size convolutions in order to give a good compromise between
  efficient contraction of data to low dimensions and shaping of
  preimage manifolds. We demonstrate how a specific network flattens a
  non linear input manifold to an affine output manifold and discuss
  it's relevance to understanding classification properties of deep
  networks.