Previous theories suggested that the Web was highly connected, with no more than 19 degrees of separation from any one site to another. By contrast, the new map reveals a subtler structure that may lead to more efficient search engine crawling techniques and a greater understanding of the sociology of content creation, and that may help predict the emergence of new phenomena on the Web such as Web rings and spam clusters. We realized that something is going on that the model where everyone is linked to everyone else really doesn t quite work, said Andrei Broder, AltaVista s vice president of research and lead author of the study. The old picture where no matter where you start, very soon you ll reach the entire Web is not quite right. The study analyzed the connectivity of more than 1.5 billion links on over 200 million Web pages.
Links are the glue that connect pages on the Web, and they re vital for navigation.
People rely on them to browse the Web, and search engines rely on them to locate pages to add to their indexes. By creating a map showing all links pointing to and from this large sample of pages, the researchers aimed to create a comprehensive topography of the Web. They found some surprising statistics, according to Broder. At the center of the bow tie is the knot, which the study calls the strongly connected core. This core is the heart of the Web.