First step along the way to queue restarts. Create a "prequeue" module whose job it is to decide how to proceed, and then have the supervisor start that. At the moment we only handle rabbit_amqqueue_process startup through this, but that will change. Also move the tx which decides whether we successfully declared the queue or not into the first part of the queue startup (before we call into the queue) since in future it will be needed for other things.
Various spec-ish fixes.
Roll slave startup into the new mechanism.
Another copy of the master_in_recovery check. Not sure that's very elegant, but I don't think we have much choice.
Remove a pointless check here - we will check from within an Mnesia tx in rabbit_mirror_queue_slave:init_it/4 anyway, which makes much more sense (and it will make little sense to check at slave start if we can restart a crashed slave).
Allow crashing queues to recover themselves.
Log that we recovered. Also store updated pid in Mnesia!
Add an intermediate supervisor so that if a queue exceeds its restart limit we don't bring the whole server down!
We handled that case in 359da524140b
Allow transient queues to recover.
This is perhaps more honest; if the queue was transient we will not recover anything and if the queue is mirrored we will restart a slave and then also not recover anything.
Allow crashing queue processes to come back as slaves if that's what we need.
Switch to making our restart decisions based on explicitly determining whether we are starting for the first time or not. This is not very OTPish but it turns out to be necessary: there's no way to distinguish between losing a race to declare, starting a new slave, and restarting to a new slave otherwise. As an upside this code is shorter and more obviously correct (to me at least).
Make this test make sense for the new regime. There will of course need to be lots of new tests but at least this passes now.
Cosmetic
Merge in default
Remove silly race.
Don't leak supervisors.
Unmislead comment.
Merge in default.