I mean to extend it a bit.. should've left a comment.
1) I don't want to burn an it_flag on the crawler if possible, since those're precious and I have some other ideas which will require flags.
2) I want to extend new things on the bottom for counters/statistics/crawler state. Like "remaining" for how many items left to crawl before stopping.