Fix an issue where a huge content may cause the crawler to drain when it still have more to crawl