add_to_set(): fix around non-negative SOCKET on Windows