Perhaps the most surprising finding is the size of each region. Intuitively, one would expect the core to be the largest component of the Web. It is, but it makes up only about one-third of the total. Origination and termination pages both make up about a quarter of the Web, and disconnected pages about one-fifth. Broder believes the core is growing faster than the other regions because Web authors are increasingly aware of the importance search engines place on link analysis in calculating relevance.
People try to get links from the core.
Commercial sites grow very fast, but don t grow so fast in terms of pages we want to crawl, said Broder. Many are catalog pages and of little value in a general purpose search engine, he said. If a direct hypertext path does exist between randomly chosen pages, its average length is 16 links. This finding is less than the 19 degrees of separation postulated in a previous study, but it also excludes the 76 percent of pages lacking direct paths. More than 90 percent of all pages on the Web are reachable from one another by following either forward or backward links.