Web crawling is when bots scan a website for its keywords, images, and links. This process helps search engines like Google, Bing, and Yahoo! index websites quickly and accurately. Bots regularly “crawl” the internet for new content.