Use ConcurrentSkipListSet for lower contention for wakeup selector.