In large scale intranets it may be desirable to index documents in each local subnetwork separately. It makes the breadth of the search scalable from a local subnetwork to the intranet and to the Internet. There amount of Web sites is growing, the number and size of stored documents grows even faster, and site contents get updated more and more often. Centralised systems can not grow that fast and cover ever decreasing segment of the Net. There are two major roles, a collection and a broker .
Collections store parts of document index.
The broker selects the collections for query propagation relying on the collection descriptions stored in the LDAP directory service. It is the responsibility of collections to create and update their descriptions in the specified format. Topical collections Distributed search is efficient only when each query is propagated to a relatively small number of nodes. When the total number of collections is large, it becomes possible only when collections are different . When the collections consist of documents belonging to a reasonably well defined topic area, the broker can make a reasonable query propagation decision.