Added crawlbot support and support for V3 APIs

Also learning what a pull request is...