BVLC/caffe — pull request #1083 by longjon (2014-09-15 21:35:25+00:00)
Title: Fix solver GPU initialization order (e.g., training with cuDNN on non-default device)

To address #925, allowing the --gpu flag to override the `device_id` setting in the solver prototxt, patch #961 reads the --gpu flag _after_ constructing the solver.

Unfortunately, this means that nets used by the solver are constructed using the default GPU, which is unexpected and breaks cuDNN, which constructs handles in `LayerSetUp` calls. To fix this, we take care of setting mode and device (with --gpu flag overriding `device_id` param) before the solver is constructed.

In accordance with the feeling at #925, setting mode and device is completely removed from the solver code. N.B., this does change the behavior when invoking the solver _without_ using `caffe train`. But I don't really expect this to affect anybody.