Improved delay handling. The original version had a politeness delay per
CrawlController / PageFetcher instance which means that any two requests
are at least 200 ms apart. This only makes sense for the same host. The
new approach stores the last fetch time for each hostname and maintains
a list of recently visited hosts, making sure that at least 200 ms are
between requests to the same host while allowing multiple requests
within the same ms over different hosts. Now, to optimally utilize this,
the distribution of getNextURLs from the frontier should be such that
the pages are distributed over the various hosts. This is still a todo.