more crawlbot bug fixes