crawl additions and fixes