The spiders catalog and index information on an intranet, so users can search and access it via standard Web browsers. They continuously crawl the intranet, report new information and changes, and automatically update the catalog. The spider supplies this information to the catalog for indexing, automatically building a catalog of information that users can then query. This process repeats indefinitely, without any human intervention required.
Users then can query the index of information, which allows results to be returned faster.
As intranets grow and change, corporations find themselves wasting precious time trying to find what they need. It would be much easier to keep track of and find information on your intranet if all the information was in a single format in a central location. But the very nature of intranets makes this an unlikely scenario. In reality, most intranets consist of a jumbled collection of document formats, from word processing files to spreadsheets to HTML documents. And while the information may be organized to some extent by departments, only the department itself knows how and where to locate its own information.