Using a CopyOnWriteArraySet instead of an AtomicReference<Set> that retries while copying.