Also check append_domain when removing the key