BVLC/caffe — pull request #497 by jeffdonahue (2014-06-13 00:33:20+00:00)
Title: Improve Backward method interface

This PR is built on top of the refactoring in #495 -- all actually relevant changes are in the final commit of the PR.

This improves the Backward method's interface: previously, the `Backward{,_cpu,_gpu}` methods had an interface `Backward(const vector*>& top, const bool propagate_down, vector*>* bottom)`. There were a couple problems with this: (1) The `propagate_down` variable was never actually set to anything but true. (2) The scalar `bool propagate_down` input did not quite make sense for some layers, namely those with multiple bottom blobs (`bottom->size() > 1`) -- it's possible some of these bottom blobs need backward computation, but others don't. The best example is loss layers, which typically take a prediction and a target/label. In any useful network, the prediction input will need backpropagation, but the target/label input probably won't. I changed the interface to `Backward(const vector*>& top, vector propagate_down, vector*>* bottom)`, so these can now get a `propagate_down` input of `[true, false]` which tells them to propagate down to the prediction but not to the target (and crash if there are weights below the label input, as Caffe doesn't know how to handle this case). One exception may be a Euclidean loss layer, which in `dev` relies on the assumption that its second input is a target which doesn't need backprop. However, it's possible you might want to train a network with a Euclidean loss where both inputs have weights below them and therefore both need backward. The new `vector propagate_down` input allows us to handle this case correctly, as `Net::Init` will infer for each bottom blob (rather than just for each layer) whether it needs backprop.