Don't remove users on background thread