crawlbot api fixes