These programs traverse the world wide web gathering the contents of web sites and storing that information for later processing. There are two basic ways that spiders can find your web site. You can tell the search engine about your web site, or let it find your site on its own. Typically search engines will have a place on their web site that allows you to suggest a site to them. After a site has been suggested, the search engines spider will visits that web site to collect information about it.
Spiders also follow the links on each web site to find linked sites to visit.
This is how a spider will find your site by itself. Usually search engine spiders will revisit your site when you submit your URL again. When the spider finds a link to your site, or after a specified amount of time has passed since its last visit. Depending on the number of web sites that the spider needs to visit and the resources that the spider has at its disposal, it can take days or months for a spider to visit or revisit your web site. Displaying information search engines take a search request from a user and displays a list of web pages that relate to that topic.