Fixed a couple of bugs in the crawler.