The study of complex activities such as scientific production and software development often require modeling connections among heterogeneous entities including people, institutions and artifacts. Despite advances in algorithms and visualization techniques for understanding such social networks, the process of constructing network models and performing exploratory analysis remains difficult and time-consuming. In this paper we present Orion, a system for interactive modeling, transformation and visualization of network data. Orion's interface enables the rapid manipulation of large graphsincluding the specification of complex linking relationships -using simple drag-and-drop operations with desired node types. Orion maps these user interactions to statements in a declarative workflow language that incorporates both relational operators (e.g., selection, aggregation and joins) and network analytics (e.g., centrality measures). We demonstrate how these features enable analysts to flexibly construct and compare networks in domains such as online health communities, electronic medical records, academic collaboration and distributed software development. Keywords: social network analysis, data management, data transformation, graphs, visualization, enduser programming. As social network analysis has gained popularity, researchers have developed novel statistical techniques, visualization designs, and user interfaces to make sense of large networks. However, many of these advances take the process of assembling a network model for granted. Much data that is collected for analysis, whether scraped from online data sources or tabulated using traditional surveys, is not inherently in 1 Online Submission ID: 0 the form of a network but instead a raw list of data points and their corresponding attributes. This requires analysts to extract their own model of a network from the raw data. For many data sets, networks can be modeled in as many different ways as analysts have hypotheses. For instance, after collecting a database of online community data, analysts may wish to examine the relationships between community members to measure collaborative support, or the relationships between thread posts to measure the dissemination of information, or the relationships between communities as a whole to measure comparative community success. To analyze each of these scenarios, completely different network models need to be extracted from the original data. Typically, refactoring network data into such various models requires custom code that can take analysts days or even weeks to write. While it is also possible to express most of the necessary operations as database queries, this requires defining a schema, loading the data, and then forming the correct SQL queries -including complex queries such as multi-table joins. Repeating this level of effort as new questions emerge may undermine the exploratory process and even dissuade some analysts from testing all hypotheses. To address these issues, we introduce Orion, a system for interactive modeling, transformation and visualization of network data. While many visualization and data mining techniques have been proposed for social network analysis, Orion focuses on the often-overlooked early stages of data transformation and assessment when forming network models from source data. Orion enables iterative, exploratory analysis by reducing hours of programming and transformation to a few minutes of interactive, graphical specification. We make the following contributions: A unified model and workflow language for network data. We use relational data tables as our fundamental model and represent networks as edge tables over a domain of integer node indices. We chose this model to correspond to those used by analytic databases and scalable network analysis packages. Our workflow language provides both relational operators and graph analysis routines and enables the generation of reusable analysis scripts. The language supports a range of analysis tasks including network definition, filtering, aggregation and statistics computation. A graphical user interface for iterative network manipulation and visualization. Orion translates user interface actions, such as drag-and-drop and menu commands, into operations in the underlying work-2 Online Submission ID: 0 flow language. Orion also supports the specification of complex linking relationships. The system first constructs a graph model of links among table columns; a traversal algorithm then identifies all feasible linking paths defining networks for a set of user-selected node types. This approach simplifies the otherwise difficult process of specifying a series of relational join and aggregation operations. Once a network has been defined, Orion enables visual exploration using tabular, matrix and node-link views. Networks can also be exported for use in other analysis tools. The rest of the paper is structured as follows. After reviewing prior work, we describe our data model and present the Orion interface. Next, we discuss our enabling algorithms for network extraction and describe our workflow language. As a preliminary evaluation of Orion, we demonstrate its use in case studies of online health communities, electronic medical records, academic collaboration and distributed software development. We then discuss future work and conclude. This article is an extended version of an earlier publication [14] and features more network operators, further details on our workflow language, and an additional case study. This version also includes an appendix with implementation and evaluation details for our foreign key identification algorithm. Orion draws on related work in graph visualization, analysis tools, and data management. We discuss selected relevant projects below. Researchers have devised a variety of visualization techniques for networks [18] . Two common representations used in social network analysis are node-link diagrams (typically using force-directed placement) and adjacency matrix views [9] . Hybrids of the two have also been proposed [16, 17] . These approaches organize elements according to the linkage structure of the graph. An alternative approach is to plot network data according to the attributes of the nodes (e.g., [32, 38] ), as in a scatter plot or so-called "semantic substrates" [32] . Network links can then be drawn between nodes. This approach is well-suited for assessing potential correlations between node attributes and network structure. In a related vein, PaperLens [25] uses multiple coordinated views of network attributes to explore 3 Online Submission ID: 0 publication databases. The NetLens system [24] generalizes this approach to support networks that fit a "content-actor" data model, i.e., bipartite networks such as publications and authors. In contrast, Orion supports an arbitrary number of linking relationships both within and between data tables. Others have researched means of dealing with large graphs in excess of tens of thousands nodes. Orion draws on this prior work: it provides both node-link and matrix visualizations and supports network aggregation based on node attributes. However, with the sole exception of NetLens [24] , each of the above systems assume a well-defined network is given as input to the tool. None of these tools help the user define and assess a variety of network models derived from arbitrary data tables. Recent years have seen a proliferation of network analysis tools. Many of these tools combine visualization and statistics within an interactive environment [1, 8, 28, 30, 33] . Others are programming libraries [20, 21, 26] or menu-driven interfaces [3, 35] that provide access to analysis algorithms. While these tools support data import from common file formats (e.g., GraphML) and external data services (e.g., Twitter or Facebook), they do little to facilitate the flexible construction of network models from arbitrary data tables. Orion is not intended as a replacement for these systems; rather, it is designed to assist the unsupported early stages of network analysis. In the process, it enables the use of these downstream tools. Another domain of related work is graph data management. Database researchers [2, 11] have developed storage strategies and query languages [11] for large networks. Similarly, a number of commercial systems -including neo4j, InfiniteGraph, AllegroGraph, DEX, OrientDB, and sones/GraphDB -are now 4
Heer and Perer REF presented a system for modelling and visualization of multidimensional heterogeneous networks.
Orion: A system for modeling, transformation and visualization of multidimensional heterogeneous networks
IEEE VAST
Computer Science