Understanding benefits of different vantage points in today's Internet

Measuring the Internet is indispensable to a better understanding of its current state and trends, but obtaining measurements is difficult both qualitatively and quantitatively. The challenges of “Internet Measurement” are manifold due to the nature of the beast, namely its complexity, distribution, and constant change. The Internet grows continuously and since it consists of many interdependent autonomous systems, there is no ground truth regarding how it looks – not even retrospectively. Nonetheless, we rely on a fundamental understanding of its state and dynamics to approach a solution. Since it is impractical to understand such complex systems at once – research on complex systems is older than the Internet itself – in this study we focus on a better understanding of the key players on the Internet by measuring Internet service providers (ISPs), Internet exchange points (IXPs), and systems running the Internet, such as routing, packet exchange, and the Domain Name System (DNS). We describe our methodology of passively measuring large amounts of network traffic data at different vantage points, and discuss the challenges, solutions, and best practices that we experienced in the course of our work. Our measurements require an understanding of the vantage points in terms of their characteristics, the systems we measure, and the data we obtain. In the course of the work, we do not exclusively rely on passive data collection and its analysis. Instead, combining our active and passive measurements helps us to improve the understanding of the data in the domain of Internet network operation. Our primary findings regard the role of IXPs in the current Internet ecosystem. We find that IXPs are understudied compared to their importance as hubs for exchanging Internet traffic, some of them handling traffic volumes comparable to major ISPs. We identify and describe different models of IXPs’ operation specific to marketplaces, namely Europe and North America. We make use of different kinds of publicly available data and proprietary data collection of Internet traffic to which we have been granted access. Our measurement results show that the Internet peering complexity is higher than anticipated in previous publications, and that IXPs are the key to this unexpected complexity. This highlights the importance of IXPs and the role they play in today’s Internet ecosystem. To further improve our understanding of global players’ operation in the Internet, we use a DNS protocol extension (EDNS0) to reveal the mapping of users to servers for one of the early adopters of this extension. The elegance of this particular measurement is in its ability to run a global crawl from a single vantage point without the need to access proprietary data or a significant amount of infrastructure. We find it useful to examine both dominant and emerging Internet components to gain a better understanding of how the Internet changes and how it is used. It is critical to measure the Internet’s driving forces, but this is a difficult task and comes with technical and legal restrictions. In order to make the best use of the data we have, it is possible and practical to combine measurement methods. As the Internet evolves constantly and rapidly, the quest to understand becomes more