Add roles

@feanil I'm hopeful this will fix the Mongo issues that @wedaly was seeing, see hipchat.  Will try to test from here and see how it goes.
We were doing this this way so that we didn't lose any data or mess up anyone's clusters by doing an unannounced upgrade.  Should we put this behind a flag?