crawlbot api fixes.