Running this in a loop makes removing labels not an atomic operation. Need to handle the case of failing partway through the list.