Multi fixes (pieces of the multi-gpu code)

To help review, I've split off a couples of the changes I made for the multi-gpu PR.  There will be a huge chunk to review eventually, but I can still split off a bunch of trailing parts like these.

This will conflict with #2182, but I'll fix the problems after all the small pieces are in master if we go this way.