Crawling is how the entire web is found and indexed. One of the most important ways that Google finds backlinks is by use of its search crawlers. These are essentially just bots (web-spiders) that visit websites and look for new links through other backlinks. This creates a tiered structure in the way the search crawlers find one URL and then continue to another URL on that page and so on.