provision/juju: preempt the queue in the end of test suites

This helps working around suite instability, but I've filled #305, so we
can definitely fix the issue.