Crawler checks if limits reached, does not re-crawl same user