Reconfigure allowed hostnames when deleting entries