BVLC/caffe — pull request #1663 by longjon (2014-12-31 07:11:34+00:00)
Title: Decouple the computational batch size and minibatch size by accumulating gradients

After #1615, so that this code already supports deconv layer. (The actual diff is just +37/-40 lines.)

This PRs the gradient accumulation branch living at https://github.com/shelhamer/caffe/tree/accum-grad. I took a lighter approach here than the one there: parameter gradients are always accumulated, there is no other option. The gradient checker is made correct by zero-initing parameter diffs.

Issues:
* This changes the behavior of `Backward`. External code that used `Backward` is likely to break, if there is any.
* I think this breaks solvers other than `SGDSolver`, but haven't thought carefully about that yet.