Remove previous minion configs before provisioning