Update workers to track a set of visited URIs as well as hashes. Use that set to reduce what is added to crawl queue.
Bump version and update changes.