BVLC/caffe — pull request #1637 by longjon (2014-12-28 03:33:57+00:00)
Title: Augment layers with their induced coordinate maps

After #1615, not because that PR is necessary for this one, but so that `DeconvolutionLayer` is already supported.

All layers map values from some array to another, often of a different size. For most basic convnet layers, this forward mapping naturally induces a mapping on the _coordinates_ that index the values of the input and output arrays. Computing this map is often useful for constructing and analyzing nets. Doing so by hand is laborious and error-prone; this PR provides an automated mechanism.

There are a few basic ways this could be done:

1. Provide an external utility (perhaps implemented in Python).
2. Provide a mechanism within Caffe proper, but keep all information about the coordinate maps separate from the layer implementations.
3. Augment each layer with its induced coordinate map.

Options 1 and 2 avoid touching the layer class definition, but require separated pieces of code to be kept in sync. Option 1 makes it impossible to use these coordinate maps within Caffe proper, e.g., by a future layer. Option 3 is the one implemented here.

This PR augments layers with a member function, `coord_map`, that keeps track of the coordinate mapping. A new type, `DiagonalAffineMap`, is introduced to store these mappings. (Note that they all that they all take the form `Ax + b`, where `A` is diagonal. This could be generalized to a class hierarchy in the future.) Note that the coordinate maps depend only on the layer (specification) parameters, not on bottom data (though the latter is not strictly prohibited).

For all existing layers, the induced coordinate map is one of the following:
* not meaningful
* the identity
* the mapping induced by deconvolution, implemented by `FilterMap`
* the mapping induced by convolution, i.e. the inverse of the above, implemented via `FilterMap(...).inv()`.

Certain conventions need to be chosen to define the induced coordinate map. The conventions used here:
* The coordinate map does not make sense for all dimensions; for convolution, for example, while the `num` dimension induces the identity map, the `channel` dimension does not induce a meaningful coordinate transformation. The mapping returned by `coord_map` therefore applies only to the coordinates for which such a mapping makes sense and is useful (i.e., the spatial dimensions). Thus all implemented maps are 2-dimensional (but the code should be easily extendable to #1486).
* The induced mapping goes from bottom coordinates to top coordinates, in the direction of `Forward`. (Often one wants the reverse, easily accessible with `DiagonalAffineMap::inv`.)
* When an output value depends on input values within a rectangular region (as in convolution or pooling), the corresponding input coordinates are given by the center of that rectangle. (By symmetry, this is the only sensible choice.)

Questions
* Neither existing code nor the Google style guide give firm rules about which methods should be CamelCase and which should be underscore_case. I made some fairly arbitrary choices here; do you like them?
* I implemented `DiagonalAffineMap::identity` as a static method, but `FilterMap` as a separate function. Should they both be separate functions?