crawlbot fixes.