A perennially interesting research topic in the field of visual analytics is how to effectively develop systems that support organizational users' decision-making and reasoning processes. The problem is, however, most domain analytical practices generally vary from organization to organization. This leads to diverse designs of visual analytics systems in incorporating domain analytical processes, making it difficult to generalize the success from one domain to another. Exacerbating this problem is the dearth of general models of analytical workflows available to enable such timely and effective designs. To alleviate these problems, we present a two-stage framework for informing the design of a visual analytics system. This design framework builds upon and extends current practices pertaining to analytical workflow and focuses, in particular, on incorporating both general domain analysis processes as well as individual's analytical activities. We illustrate both stages and their design components through examples, and hope this framework will be useful for designing future visual analytics systems. We validate the soundness of our framework with two visual analytics systems, namely Entity Workspace [8] [14, 41, 46] suggest that the establishment of a general design framework is significant. The objective for such a VA design framework is threefold: firstly, the framework must inform designers to systematically incorporate the support for domain analytical processes. Secondly, the framework should provide a basis for designers to evaluate their system and further help them identify a cohesive technology transition progress, from system design and implementation to its release and deployment [49] . Finally, the framework must serve an educational purpose, contributing to the identification of potential course materials that are necessary to educate others regarding the field of VA [1] . However, constructing a convincing and appropriate design framework is challenging. The framework must be validated against existing systems and more importantly, it must give researchers and designers new ideas regarding how to evaluate and improve their own work. Given the need to incorporate successes from diverse VA systems, it is difficult to generate a framework that can summarize and instruct all the design requirements from a top-down perspective. High-level VA design frameworks like [14, 41, 44, 55] are certainly of great value. Nonetheless, little specific guidance or recommendation is currently available to articulate the boundaries within which particular design assumptions apply, leaving the system design to be solely based on designers' prior experience. For example, how does a designer know which analysis method is suitable to characterize an organization? Are there components that a designer should follow to systematically incorporate a domain analytical process? Further, what recommendations exist that specify the appropriate methods for supporting both general and individual analytical workflows? Encouraged by the discussions in the VAC consortium 2010 [1], the authors of this paper resolved to document their experiences with VA framework design. In the pursuit of a general VA design framework, we grounded the conceptualization of such a framework in our collaborations with three organizations [58, 59, 64] , and materialized the framework by integrating guidelines from existing literature [41] , [61] with our experience. We started by designing the framework to fit the emerging characteristics of the field of visual analytics, and further enriched its components based on our design practices and the novel ideas presented by others. To preserve the generalizability of the framework, we studied a broad selection of literature on modeling organizational analytical processes and designing information systems from research fields such as Organizational Learning (OL), Knowledge Management (KM), Machine Learning (ML), Human Computer Interaction (HCI), and Information Visualization (InfoVis). Although much of our work can be applied to these domains, in this paper, we focus on presenting and validating a two-stage framework that is primarily specific to the VA field. We hope this framework will be useful in designing and evaluating VA systems. We illustrate both stages and their design components through examples. We further use external evidence [40] to evaluate the usefulness and novelty of the framework. In some cases, we compare components of our framework with existing work. In other cases, we present recommendations that are largely unique to the design of VA framework. Due to space constraints, we cannot review all visual analytics systems nor exemplify all the components of such systems. In the following section, we discuss the two design fundamentals of a VA framework that designers must consider. Existing models and frameworks related to these fundamentals are further described and discussed. In the subsequent sections, we present the details of our two-stage design framework and its related design components. We organize these components into two design stages. First, we discuss the Observation and Email to: xwang25@uncc.edu 251 October 23 -28, Providence, RI, USA 978-1-4673-0014-8/11/$26.00 ©2011 IEEE Designing stage (Section 4.2), which designers should follow to characterize the targeted domain and support its general analytical processes. While the first stage is valid in presenting the synthesis of the majority of domain analysis activities, individual differences aren't captured in this stage. Consequently, we describe the need for a "feedback" process to integrate the individual's analytical practices into VA systems. In particular, we present the second, User-centric Refinement stage (Section 4.3), in which designers find recommendations to customize the visual analytics system to support individual ways in performing analysis. We design both stages to be complementary and hope they can collectively provide recommendations that are informative for developing an organizational VA system. Similar to the emphasis in the information system design [19] , many researchers have recognized the importance for VA designs to conflate task activities within both the organizational level (high-level holistic task workflow) and the individual level (specific tasks or operations) [44, 46] . The pioneers of visual analytics---researchers from data analysis, InfoVis, and HCI---contributed invaluable application designs, algorithmic thinking, and engineering traditions. Quite early on, the intelligence community stressed the application of analysis models and theories of sense-making processes when designing VA systems [11] . As shown in the sense-making loop, Card and Pirolli [45] presented a theoretical basis for understanding and portraying the analytical discourse that an analyst performs. Later on, cognitive scientists showed interests in this emerging field and introduced a scientific design approach, focusing on the value of empirical observations of analysis processes and users' performances [27] . Recently, influences from ML, HCI and KM contributed to the establishment of the methodological grounds for VA research. These influences have broadened the scope for VA designs from focusing on creating visual interfaces, to accommodating broader and more complex analytical needs between individuals, teams and organizations. The contemporary visual analytics field is multidisciplinary and is rapidly maturing thanks to the collective contributions from these diversified research areas. The Fundamentals of a VA Design Framework Like many empirical sciences (e.g. HCI and InfoVis), the field of visual analytics does not solely research on existing technologies, styles of interaction, or interface solutions. The design of organizational VA systems extends beyond merely the final visual interfaces and representations. While these interface features are undeniably essential considerations in visual analytics system development, they are not the major concern of this work. As shown in Figure 1 , the core foundation of a VA framework emphasizes the integration of general domain analytics processes, the visual facilitation of domain analytical tasks and finally, the customization for individual's analytical workflows. Essentially, the design of a VA system is analysis-centric, in that it needs to encapsulate the organizational analysis discourse [41] and support its related reasoning tasks and user behaviors [44] . In particular, such design needs to consider and alleviate potential incompatibilities and challenges that could affect users' acceptances of, and reactions to, an interactive VA system. This design also needs to incorporate supports for various domain analysis tasks, properly match the nature of the task and the support from the system, provide logical organization of targeted data, utilize accurate statistics to meaningfully transform data, and most importantly, guarantee consistency between analytical workflows and VA system operations. Human-centered design is another significant factor in VA design. Recent research and practices in HCI have shown that user' affective reactions and their holistic experiences with technology are becoming increasingly important [2, 61] . As suggested by many empirical studies, a better understanding of various human cognitive, affective, and behavioral factors in the context of problem solving processes, are essential in designing a more successful analytical system [61] . The central tenet of human-centered design is to fully engage domain users during the modelling and design process of a VA system, ensuring their requirements and demands are clearly understood and conveyed to designers. Such design is a bidirectional process: elicitation of system requirement demands VA designers to communicate fluently in the same "analysis language" of the targeted domain; it also requires designers to introduce the merits of visual analytics to influence the organization and help improving its existing analysis processes. MOTIVATION
proposed a two-stage framework for designing visual analytics systems REF .
A two-stage framework for designing visual analytics system in organizational environments
2011 IEEE Conference on Visual Analytics Science and Technology (VAST)
2011 IEEE Conference on Visual Analytics Science and Technology (VAST)
Computer Science