But at Northern Light Boolean searches account for 15 percent of the queries, suggesting a more sophisticated clientele. Searches on the Web also aren't as instant as they seem. Search engines build their page indexes by using electronic spiders, which take days to do a complete run of the Internet. Lycos reports about 56 million indexed Web pages. But before consumers see their search results, Lycos already has filtered out duplicates, spam, pornography and other low quality stuff.
Some search engines are more picky about what they deem is relevant.
Newcomb and Doughty said Northern Light doesn't whittle down a lot of its initial data before returning search results. They said if you want to find your mother's doctoral thesis on a really obscure computer in Denmark, then Northern Light and Fast are the search engines for you. HotBot, the search engine owned by Lycos, consistently ranks among the top as the most relevant indexing site, but only reports about 40 million Web pages. Lycos invests in technology that will identify search trends and predict user behavior. Tracking the popularity of sites and how long people stay have become important information in the building of search engine databases.