Revert "Always run Puppet provisioner"

Doesn't work in <1.6

This reverts commit b5f7e0962d9acbe899e673ae56e7deabe159de3e.

Change-Id: I8a8a773ec13ea9cab8a4dd43c463312ee211cd4c