JENKINS-21363. Don't provision slaves while we're in quietdown or shutting-down