Synchronize HashSet which is vulnerable to corrupting its backing HashMap under load