code is stable, tests regarding performance given the number of worker threads is done (and disabled on the crawler)