Remove old provisioning files.