Visual analysis of network traffic: interactive monitoring, detection, and interpretation of security threats

The Internet has become a dangerous place: malicious code gets spread on personal computers across the world, creating botnets ready to attack the network infrastructure at any time. Monitoring network traffic and keeping track of the vast number of security incidents or other anomalies in the network are challenging tasks. While monitoring and intrusion detection systems are widely used to collect operational data in real-time, attempts to manually analyze their output at a fine-granular level are often tedious, require exhaustive human resources, or completely fail to provide the necessary insight due to the complexity and the volume of the underlying data. This dissertation represents an effort to complement automatic monitoring and intrusion detection systems with visual exploration interfaces that empower human analysts to gain deeper insight into large, complex, and dynamically changing data sets. In this context, one key aspect of visual analysis is the refinement of existing visualization methods to improve their scalability with respect to a) data volume, b) visual limitations of computer screens, and c) human perception capacities. In addition to that, developmet of innovative visualization metaphors for viewing network data is a further key aspect of this thesis. In particular, this dissertation deals with scalable visualization techniques for detailed analysis of large network time series. By grouping time series according to their logical intervals in pixel visualizations and by coloring them for better discrimination, our methods enable accurate comparisons of temporal aspects in network security data sets. In order to reveal the peculiarities of network traffic and distributed attacks with regard to the distribution of the participating hosts, a hierarchical map of the IP address space, which takes both geographical and topological aspects of the Internet into account, is proposed. Since visual clutter becomes an issue when naively connecting the major communication partners on top of this map, hierarchical edge bundles are used for grouping traffic links based on the map’s hierarchy, thereby facilitating a more scalable analysis of communication partners. Furthermore, the map is complemented by multivariate analysis techniques for visually studying the multidimensional nature of network traffic and security event data. Especially the interaction of the implemented prototypes reveals the ability of the proposed visualization methods to provide an overview, to relate communication partners, to zoom into regions of interest, and to retrieve detailed information. For an even more detailed analysis of hosts in the network, we introduce a graph-based approach to tracking behavioral changes of hosts and higher-level network entities. This information is particularly useful for detecting misbehaving computers within the local network infrastructure, which can otherwise substantially compromise the security of the network. To complete the comprehensive view on network traffic, a Self-Organizing Map was used to demonstrate the usefulness of visualization methods for analyzing not only structured network protocol data, but also unstructured information, e.g., textual context of email messages. By