They are also looking to see if the site is used. Well-used sites will be indexed or included in the database long before a page like this one. The data captured from scouring the web by these computer programs is then added to the search engine's database. These are gigantic databases with loads of information. Here relevant information is stored, updated, and indexed.
Many search engines allow people to submit their pages directly to the database.
This way, if the search robot only passes through your server every month so or so the page designer doesn't have to wait before the page is entered in the database. If the item you want is not in the database, you will not be able to retrieve it. When you enter a query into a search engine, the words in your query are checked against what's in the database or its index. Many search engines rank search results based on a specific calculation or algorithim. Even if the item you want is not in the database it will return information that it thinks is close.