You can get nearly all the speed benefit in most cases by changing the 'min' to 'max'

But the following remain issues:

any unhandled exception in a thread doesn't raise any error in the main process, when this happens, the page-getting step may be incomplete.

While the for loop of thread.join allows the first link to add a bunch of rel-links to the queue, and this is the most common pattern, the current thread design still allows all but the last thread to return, while that last thread could still dump a bunch of new rel-links in the queue.  Most of the time, the bulk of the rel-links are added from the first location, and added by the first thread - so in practice, probably not a huge issue, certainly no worse than the current state of things.