placed an upper ceiling to the number of messages in the queue before the spider stops extracting links anymore