Revert "Revert "Prevent things from running during provisioning.""

This reverts commit 9186d04e8dee0130756be35838e4ee5694ae007a.