A web crawler, also known as a web spider or web bot, is an automated program or script used by search engines to scan and index web pages on the internet. It navigates through websites, follows links, and collects information about web content, which is then used by search engines to provide relevant search results to users.