Remove quitting users from their channels.