Ensure provisioning tools doesn't blow up when removing a CNAME entry if it doesn't already exist