The set may be added to from any thread, though.