Use remote IP to distribute connections over selectors