Only remove remote users' nicks if they had nicks in the first place