Many also index the headers of each section. Others include the first few lines of text or the most frequently mentioned words. A few include the full text of every WWW site included. Many sites are behind firewalls or require a login. Robots will not find these, therefore these sites will not be added to a database.
Robots also place a heavy bandwidth burden on networks and servers.
Many servers will request that a robot not roam their site or prohibit robot access. Be aware that even the largest database available to a search engine will not be comprehensive. And, in part due to variations in indexing, you will get different results from different search engines. Always try your search in several search engines. Search engines also vary in their search features and in the organization of search results.