# Immersive Analytics: An Introduction

Tim Dwyer, Kim Marriott, Tobias Isenberg, Karsten Klein, Nathalie Riche, Falk Schreiber, Wolfgang Stuerzlinger, Bruce Thomas

<figure><img src="image_1.png" /></figure>

## To cite this version:

Tim Dwyer, Kim Marriott, Tobias Isenberg, Karsten Klein, Nathalie Riche, et al., Immersive Analytics: An Introduction. Immersive Analytics, Springer, pp.1 - 23, 2018, 10.1007/978-3-030-01388-2_1.hal-01907533

HAL Id: hal-01907533

https://inria.hal.science/hal-01907533

Submitted on 29 Oct 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# 1. Immersive Analytics: An Introduction

Tim Dwyer<fnref n="1" />, Kim Marriott<fnref n="1" />, Tobias Isenberg<fnref n="2" />, Karsten Klein<fnref n="3" />, Nathalie Riche<fnref n="4" />, Falk Schreiber<fnref n="1" />,<fnref n="3" />, Wolfgang Stuerzlinger<fnref n="5" />, and Bruce H. Thomas<fnref n="6" />

**Abstract.** Immersive Analytics is a new research initiative that aims to remove barriers between people, their data and the tools they use for analysis and decision making. Here we clarify the aims of immersive analytics research, its opportunities and historical context, as well as providing a broad research agenda for the field. In addition, we review how the term immersion has been used to refer to both technological and psychological immersion, both of which are central to immersive analytics research.

**Keywords:** immersive analytics, multi-sensory, 2D and 3D, data analytics, decision making

## 1.1. What is Immersive Analytics?

*Immersive Analytics is the use of engaging, embodied analysis tools to support data understanding and decision making.* Immersive analytics builds upon the fields of data visualisation, visual analytics, virtual reality, computer graphics, and human-computer interaction. Its goal is to remove barriers between people, their data, and the tools they use for analysis. It aims to support data understanding and decision making everywhere and by everyone, both working individually and collaboratively. While this may be achieved through the use of immersive virtual environment technologies, multisensory presentation, data physicalisation, natural interfaces, or responsive analytics, the field of immersive analytics is not tied to the use of specific techniques.

We live in an age where the amount and complexity of data available to us far surpass our ability to understand or to utilise in decision making. This is not only

<footnote n="1">¹ Monash University, Australia
[Tim.Dwyer, Kim.Marriott]@monash.edu</footnote>

<footnote n="2">² Inria and University Paris-Saclay
tobias.isenberg@inria.fr</footnote>

<footnote n="3">³ ? University of Konstanz, Germany
[Karsten.Klein, Falk Schreiber]@uni-konstanz.de</footnote>

<footnote n="4">⁴ Microsoft, USA
Nathalie.Henry@microsoft.com</footnote>

<footnote n="5">⁵ Simon Fraser University, Canada
w.s@sfu.ca</footnote>

<footnote n="6">⁶ University of South Australia
bruce.thomas@unisa.edu.au</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 1: The research field of immersive analytics is exploring techniques that will enable seamless local and remote collaborative work informed by data in immersive environments. While the ideal technologies are not yet quite available (left), researchers can already explore the design space and develop the necessary display and interaction techniques using existing technologies (right). (Figure © 2015 IEEE. Reprinted, with permission, from Chandler et al. [16]).</figcaption></figure>

true for business analysts, scientists, and policymakers, but also for members of the general public who have increasing access to, for instance, personalised health data, IoT and other sensor data, as well as social media. In recent years we have seen rapid progress in the development and availability of immersive technologies such as head-mounted Virtual and Augmented Reality (VR and AR), large wall-mounted, hand-held, or wearable displays. Similarly, progress in sensor technology and the application of machine learning technologies to interpret user gestures and utterances have fuelled the development of natural interfaces making use of speech, gesture and touch. The combination of these new kinds of display and interaction technologies is building towards a(nother) revolution in how people use computers and offers a new approach to data analytics and decision making; one that liberates these activities from the office desktop and supports both collocated and remote collaboration (see Figure 1).

Immersive visual analytics has the potential to dramatically improve all areas of our lives. One example application is healthcare, see Chapter 10 (Immersive Analytics Applications in Life and Health Sciences). For medical specialists treating a patient with a complex multi-faceted medical condition, a mix of AR headsets and tiled wall displays potentially support collocated and remote collaboration, see Chapter 8 (Immersive Collaborative Analytics). This collaboration may be synchronous, such as a group of nurses and doctors conducting surgery, or asynchronous, where the collaborators participate at different times, perhaps throughout the entire course of the medical treatment. Current technologies allow specialists to create and remotely share immersive data visualisations. In the near future they will be able to see, hear, touch and perhaps even smell an anatomically correct model of the patient. They will be able to walk through the model, overlaying it with, for example, 2D and 3D scans, or population-based tumour occurrence models. By using AR to overlay the patient's body with
visualisations of their internal organs, they will be able to show the patient what they believe is happening and the impact of possible treatment options.

Another example is urban planning and disaster management, as detailed in Chapter 11 (*Exploring Immersive Analytics for Built Environments*). Urban planners, engineers, and local authorities will soon be able to collaboratively explore different flood, wildfire or other disaster mitigation strategies using a 3D-printed topographical model of the affected area, overlaying it with personal and shared holographic displays and showing the results of computer simulations. These technologies allow the local community to more deeply engage in decision making by using immersive VR or AR to viscerally understand the complex impact that these choices will have on their local neighbourhood.

Even more excitingly, these new technologies potentially democratise the use of sophisticated data analysis and decision support tools, taking them outside the workplace and into everyday life, see Chapter 7 (*Situated Analytics*). Already, GPS-based navigation tools—computing optimal routes to the desired destination using current location and live traffic data—have changed the way in which most people navigate while driving. Ubiquitous AR promises a future in which we live in a mixed-reality environment, where physical objects are overlaid with virtual data and everyday analysis and decisions are computer-mediated. For instance, when walking through a supermarket, products could be overlaid with personalised dietary, ethical and pecuniary information to inform and guide purchase decisions.

However, while many of the display and interaction hardware technologies required for these scenarios already exist, the fundamental knowledge of how to design appropriate human-computer interfaces and data visualisations is missing. This is the subject of Immersive Analytics (IA). As the brief examples above illustrate, IA offers several opportunities beyond more traditional visual analytics:

- The first opportunity is *situated analytics* in which user-controlled data analytics information is linked with objects in the physical world. This might be products in a supermarket, attendees at a conference, machinery in a workshop, instruments in a lab, or objects at a building site. This has many applications in the workplace and even more in other aspects of everyday life through ubiquitous personalised analytics [37] (see Chapter 7).

- The second opportunity is *embodied data exploration*. By moving away from the mouse and keyboard, it is hoped that touch, gesture, voice and tangible interaction allow more intuitive and engaging data exploration in which the computer becomes invisible, but continues to facilitate analytics behind the scene. These topics are covered in Chapter 4 (*Interaction for Immersive Analytics*) and Chapter 5 (*Immersive Human-Centered Computational Analytics*).

- The third opportunity is *collaboration*, where the collaboration might be collocated or remote as well as synchronous or asynchronous. IA potentially supports deeper, more equitable and socially engaging collaboration (see Chapter 8).
- The fourth opportunity is *spatial immersion*. Moving away from the desktop allows users to use the space around them as a three-dimensional workspace in which they place coordinated 2D, 2½D, and 3D visualisations, see Chapter 2 (*Immersive Analytics: Time to Reconsider the Value of 3D for Information Visualisation*).

- The fifth opportunity is *multi-sensory presentation*. Traditional visual analytics has focussed on visualisation but audio and the other senses can also be used to provide additional information or as an alternative to vision where this is unavailable or not useful. This topic is discussed in detail in Chapter 3 (*Multisensory Immersive Analytics*).

- The final opportunity is more informed and increased *engagement* in data-informed decision making by the general public and other stakeholders. Immersive interactive narrative visualisations can be used, for instance, to engage the local community in climate change mitigation or to see the potential impact of their local carbon footprint. Such immersive visual data stories are discussed in Chapter 6 (*Immersive Visual Data Stories*).

Immersive analytics brings together researchers and data practitioners from data visualisation and visual analytics, virtual and mixed reality, human-computer-interaction and human-in-the-loop algorithmics. This book provides the first comprehensive introduction to this emerging interdisciplinary research field. We survey academic research and provide the necessary background material for new PhD students or for more experienced researchers who wish to work in this field. In addition, the book gives a roadmap of open questions to guide further research. It is also designed to be accessible to a broader audience who are interested in how emerging technologies may be used in data analysis applications in science, technology, business, health, or other aspects of modern life.

## 1.2. Historical Context

Today we interact with computers through phones, laptops and other devices so regularly and ubiquitously that many people forget or are unaware that such activities were once almost unimaginable. Before the 1980s, however, computers were intimidating and expensive tools. Computing pioneer Howard Aiken once reminisc ed of the early days of computing: "there was no thought in mind that computing machines should be used for anything except out-and-out mathemat- ics" [18]. It took considerable time and research effort [44, 87] for the human imagination to move beyond the notion that computers were centralised resources to be used only by skilled technicians for very "serious" applications.

### 1.2.1. Human-Computer Interaction

Researchers in computing labs in the 1960s foresaw that this rapidly evolving technology could play a more transformative role. For example, Engelbart and English saw their computing lab at Stanford as "a research center for augmenting
human intellect” [28]. Their work led to the first demonstrations of a windowed
UI, the mouse, hyperlinked documents, video conferencing, and visual word
processing. Around the same time, Ivan Sutherland at Harvard demonstrated
direct interaction with a graphical display using a light pen [92] and then the
first mixed reality head-mounted display system [93].

An early journal, *Man-Machine Studies*, began to link technologies for computer interaction with more human-focussed studies in 1969 [56]. The increased use of computers in real-time applications such as avionics meant that the initial focus was on making interaction as simple and efficient as possible. Then, as computers became cheaper and smaller, they moved into mainstream use. The focus changed to computer use by non-specialists, and the field of Human-Computer Interaction (HCI) emerged at the intersection of the previously disparate disciplines of computer science, cognitive and perceptual psychology, and ergonomics. The creation of the ACM SIGCHI organisation and its first conference in 1982, established HCI as an important new field of computer science.

The history of HCI is intricately bound to developments in interaction tech-
nologies. Sometimes research in HCI drives innovation in hardware and devices,
for example the light pen or the mouse. Sometimes hardware innovation drives
HCI research by allowing new ways to integrate computers into our work and
our lives. For example, motion tracking—conceived as a means to create lifelike
animated movie characters—is now supported by consumer devices<fnref n="1" /> and has
become a viable input for interaction, as processor speeds and algorithms for
feature detection have improved allowing real-time gesture recognition. Motion
tracking is only one example of an emerging technology that allows more nat-
ural, fluid, and embodied<fnref n="2" /> interaction with computers. Such natural and fluid
interaction is a potential key to engaging users in their data, a primary goal of
immersive analytics (as further discussed in Section 1.3.).

1.2.2. Graphics and Visualisation

In addition to input devices, the other critical technological advance that has
opened up new ways to interact with computers has been display technologies.
This advance began in the 1950s with computer-driven oscilloscopes that were
able to trace lines to form what we now call *vector graphics*. Once again, research
that began in labs made its way into the mainstream over the course of several
decades. With the introduction of the ACM SIGGRAPH conference series in 1974,
graphics was recognised as a distinct sub-field of computer science. Computer
graphics research rapidly evolved, and many sought to achieve photo-realistic
renderings of virtual scenes. By the 1980s, the opportunities for visualising

<footnote n="1">¹ Examples of low-cost motion tracking devices being Microsoft Kinect [60]—launched in 2010—and Leap Motion [49]—launched in 2013.</footnote>

<footnote n="2">² Embodied interaction refers to the notion that the affordances of a user interface or interaction device somehow naturally embody the purpose or nature of the intended action [24]. See Chapter 4 for more background on this and other aspects of immersive interaction.</footnote>
data were evident, especially to the scientists already using powerful computers with sophisticated graphics capabilities. A report based on the NSF sponsored workshop on "Visualization in Scientific Computing" in 1987 [58] gave rise to the IEEE Visualization conference series in 1990.

As mentioned earlier, virtual reality (VR) and augmented reality (AR) prototypes were demonstrated by Ivan Sutherland in the late 1960s [93]. However, it was not until the 1990s that advances in dedicated graphics hardware and real-time rendering techniques led to the first practical interactive virtual reality systems. An early example of such a VR system was the CAVE [19]. Research interest in the area in industry and university labs was such that a dedicated conference series, IEEE Virtual Reality (IEEE VR), was established in 1993.

After successful workshops in the two preceding years, the first Symposium on 3D User Interfaces (3DUI) was held in 2006. This symposium series focussed on the interaction aspects of VR and AR systems. Reflecting a growing interest in interaction with virtual content, the ACM Symposium on Spatial interaction (SUI) was established in 2013. Then, as a result of a convergence of research within virtual reality, IEEE VR and 3DUI merged into the IEEE Conference on Virtual Reality and 3D User Interfaces in 2018.

As of early 2018, there are several well-established commercial virtual reality equipment providers. Some notable examples of VR equipment include: HTC Vive,<fnref n="3" /> Oculus Rift,<fnref n="4" /> Microsoft Mixed Reality,<fnref n="5" /> and Google Daydream.<fnref n="6" /> What is important about this latest generation of VR equipment is that they are modestly priced, compatible with mid-range graphics PCs, and simple to set up as they are a fully integrated system. In contrast, only a few years ago, head-mounted displays and tracking subsystems were sold separately, and the input devices were, by and large, custom-made by the end user. Furthermore, a major barrier to the widespread use of earlier VR equipment was that integration of the subsystems was left to the end user, and included such fundamental operations as mounting tracking sensors onto the head mounted display.

In the past, while there have been commercial software development environments for VR, these were not widely used. WorldViz<fnref n="7" /> is an early example of an easy to use VR development and runtime software environment, but it was not commonly employed. This has changed, and emerging software standards are allowing the rapid growth of VR applications and software ecosystems. Games engines such as Unity 3D<fnref n="8" /> and UnReal Engine<fnref n="9" /> provide support for the development and deployment of VR applications while software tools such as SteamVR<fnref n="10" /> provide cross-platform support.

<footnote n="3">³ https://www.vive.com/</footnote>

<footnote n="4">⁴ https://www.oculus.com/</footnote>

<footnote n="5">⁵ https://www.microsoft.com/en-us/store/collections/vrandmixedrealityheadsets</footnote>

<footnote n="6">⁶ https://vr.google.com/daydream/</footnote>

<footnote n="7">⁷ http://www.worldviz.com/</footnote>

<footnote n="8">⁸ https://unity3d.com/</footnote>

<footnote n="9">⁹ https://www.unrealengine.com/</footnote>

<footnote n="10">¹⁰ http://store.steampowered.com/steamvr</footnote>
The visualisation field's early emphasis on scientific applications concentrated on data that had a natural spatial embedding in two or three dimensions. This included geographic data such as ocean currents or architectural, physical or biological data such as fluid flows in wind-tunnels or density in medical scans. Such data was naturally visualised using immersive 3D display environments. Early pioneers (e. g., Card *et al.* [15]) enthusiastically explored immersive visualisation of data that was more abstract, i. e., purely numerical or relational and, therefore, with more freedom in how it could be represented and spatially embedded. Exploration of this design space of spatial mappings for abstract data became a key theme of a new sub-field of visualisation called Information Visualisation (InfoVis). Information visualisation built upon the work of graphic designers such as Bertin [3], statisticians such as Tukey [99], Tufte [98], and Cleveland [17], as well as HCI researchers such as Card, McKinley, and Shneiderman [14], and many others, to create a new interdisciplinary field that explored how to effectively utilise computer graphics for abstract data visualisation and its interactive exploration. In 1995, the new IEEE Symposium on Information Visualization (IEEE InfoVis) was established to focus on this new theme, becoming a conference in 2007.<fnref n="11" />

Despite an early enthusiasm for the kind of 3D representations of abstract data made possible by first-generation graphics hardware (e. g., Figure 2), in the 2000s information visualisation researchers became more conservative. As information visualisation established itself as a new research discipline, researchers concentrated on techniques suited to standard desktop and WIMP interfaces, rather than to immersive technologies which, at that time, were only available in labs. This conservatism was supported by studies finding that some 3D representations of abstract data on standard 2D monitors were not particularly useful. It remains an open question whether 3D representations of abstract data can be more effective in genuinely immersive environments, for some settings or application scenarios. More generally, the question of how we should be using the space around users for representing data in immersive environments (whether VR or AR) is a much more complicated issue than simple 2D versus 3D effectiveness. Both questions are discussed in detail in Chapter 2.

### 1.2.3. Visual Analytics

In the early 2000s, national security concerns and exponential growth in data and communication on the internet created an urgent need for more effective techniques to analyse large, complex data. By this time information visualisation research had shown some promise and developed novel techniques along with guidelines for the design of useful visual data representations. However, this new urgency suggested a need for a more holistic but also pragmatic research agenda to tackle the grand challenge of creating tools that genuinely met the requirements of big data analytics. In their agenda for the new National Visual-ization and Analytics Center, Thomas and Cook called for a new science called

<footnote n="11">¹¹ For a brief history of the IEEE VIS conferences see Section 2.2 in [40].</footnote>
<figure><img src="image_4.png" /><figcaption>Fig. 2: Early enthusiastic exploration of immersive visualisation of abstract data. (a) Cone Trees and the horizontal variant Cam Trees were introduced by Robertson et al. in 1991 as a way to navigate complex tree structures [80]. The internal nodes of the tree could be rotated to bring a different children to the fore, exploiting perspective distortion as a “focus and context” technique. (b) In 1997 Munzner introduced a 3D hyperbolic network browser that exploited an even more exotic space for focus and context [63]. (Figure (a) courtesy of S. Card, J. Mackinlay and G. Robertson and used with permission from Xerox, Figure (b) © 1998 IEEE. Reprinted, with permission, from [63])</figcaption></figure>

Visual Analytics concerned with “analytical reasoning facilitated by interactive visual interfaces” [96]. They made 19 high-level recommendations for the field of visual analytics and, in particular, in contrast to earlier work on information visualisation, they called for research that would enable visualisation techniques to:

- scale to huge data [96, pg. 7] and

- enable a higher-level discourse between analyst and information [96, pg. 77] rather than what they saw as a focus by information visualisation on straightforward and low-level analysis tasks.

Thomas and Cook's definition of visual analytics was agnostic of the actual interface and display devices employed by visual analysis systems. Nevertheless, most visual analytics researchers inherited a certain scepticism of immersive environments from the InfoVis community and continued to focus on developing techniques for traditional desktop environments. However, as discussed in regard to human-computer interaction, the affordances of the display and input devices strongly affect the experience of users and so their degree of engagement and productivity. While it may be that work in visual analytics has tried to have a high-level focus on human cognition and workflows for “sense making” rather than on the details of devices and computing environments, we feel that the embodiment
of data afforded by the new generation of immersive environment technologies
fundamentally impacts the analyst's experience and warrants consideration.

1.2.4. The Birth of Immersive Analytics

We contend that emerging display and interaction technologies open up new
possibilities for achieving Thomas and Cook's goals for visual analytics described
above. But, in addition, as discussed in Section 1.1. these technologies can bring
data analytics:

- to a wider audience through tools and technologies that more fully engage the senses,

- to a new generation whose first "language" of computer interaction is not the mouse and keyboard,

- to new situations where desktop computing is impossible, and

- to teams of people where all participants are equally empowered.

This potential has already excited a large community of researchers from
the various fields we have discussed above. The idea that information should
be immersive is at least as old as the early work of Robertson, Card, and
McKinley [15]. However, the belief that visualisation and visual analytics needed
to embrace the explosion of new interaction paradigms took off in 2014, with
a workshop entitled "Death of the Desktop: Envisioning Visualization without
Desktop Computing" at IEEE VIS 2014.<fnref n="12" />

Shortly after this, the name Immersive Analytics was coined by researchers
exploring possibilities for data visualisation in virtual and mixed reality [16]
and introduced at IEEE BDVA 2015. Two coordinated workshops under the
banner of Immersive Analytics were held in 2016. The first at Shonan, Japan,
with 25 invited international experts on visualisation, visual analytics, human-
computer interaction and virtual and augmented reality [41] and the second was
held soon after at Dagstuhl, Germany with an additional 38 experts spanning
similar disciplines [27]. It was at these workshops that a broader understanding of
immersive analytics and the chapter topics for this book emerged. Most authors
of chapters in this book attended one of the two workshops, but some additional
experts were invited as required.

Since then there has been a flood of workshops on immersive analytics and
the community has grown rapidly. Immersive analytics workshops were held at:

- IEEE BDVA in 2016 and 2017, Australia

- IEEE VR 2016, USA

- Interactive Surfaces and Spaces 2016, Canada

- IEEE VIS 2017, USA

And, in 2018, the first symposium with an explicit call for Immersive Analytics
submissions will be held in Germany at *IEEE Big Data Visual and Immersive
Analytics*.<fnref n="13" />

<footnote n="12">¹² http://dataphys.org/workshops/vis14/tag/visualization/</footnote>

<footnote n="13">¹³ http://bdva.net/2018/</footnote>
## 1.3. Engagement and Immersion

Engagement and immersion are at the heart of immersive analytics. We broadly distinguish between technological immersion—which relates to the technologies used in VR to immerse the user in a virtual world—and psychological immersion—which relates to the cognitive state experienced by a user when they are absorbed by some task. The latter could also be called engagement. As both technical and psychological immersion are relevant to immersive analytics research, we provide a brief review here. While clearly different they are connected: technological immersion is, for example, widely employed in computer gaming with the belief that it will increase psychological immersion.

### 1.3.1. Technological Immersion/Presence

A primary focus of VR research has been to develop technologies that immerse the user in a virtual world by providing a sensory experience that is so similar to that offered by the real world that the user feels and behaves like they are indeed present in the virtual world, temporarily unaware that they are inhabiting a computer-mediated environment. This is the intended meaning of “immersive” in the terms “immersive environments” or “immersive technologies”. A significant focus of immersive analytics is the use of immersive technologies to support collaboration and embodied data exploration and decision making. Therefore, a common, shared understanding of technological immersion is necessary for immersive analytics researchers.

Following Slater [91], we distinguish between *presence* (the subjective psychological experience of being in a virtual or remote space) and *immersion* (the objective characteristics of the technology used to present the virtual space). Since the term telepresence was introduced to describe the experience of being virtually transported via telecommunication systems to another physical location [62], there has been considerable research into different aspects of technological immersion and presence by IT researchers and psychologists, e. g., [25, 26, 34, 53, 54, 77, 82–85, 88, 90, 91, 101].

Immersive technologies can produce three main kinds of presence:

- *Spatial presence* is the psychological state in which the environment and objects in the virtual world are experienced as actual physical objects [50]. According to Slater [88] it is (a) the sense of “being there” in the world depicted by the virtual environment (VE), (b) the extent to which the VE becomes the dominant one—i. e., the tendency to respond to events in the VE rather than in the “real world”, and (c) the extent to which participants remember the VE experience as having visited a “place” rather than just having seen images generated by a computer [88].

- *Social presence* is the “sense of being with another” [6] or, more broadly, the psychological state in which virtual social actors (both human and artificial) are experienced as actual social actors [50]. Many different aspects have been identified [6, 50, 53, 67]. One is *co-presence* which requires the actors to be
in the same virtual space and measures the degree of mutual awareness and psychological involvement between actors [6]. Another aspect (confusingly also called social presence) is the degree to which the actors believe the virtual world supports interpersonal communication. This was originally introduced to measure the effectiveness of telecommunication technologies. Related to social presence is the notion of cultural presence [57].

- *Self presence* is the psychological state in which virtual selves are experienced as the actual self [50] and measures the “effect of the VE on the perception of one’s body (i. e., body schema or body image), psychological states, emotional states, perceived traits and identity” [5] or “the extent to which some aspect of a person’s proto (body-schema) self, core (emotion-driven) self, and/or extended (identity-relevant) self is relevant during media use” [76].

Researchers have found that different kinds of presence are correlated (e. g., [67]) and that many features of the VE have been identified as affecting the subjective feeling of presence. These include:

- *Inclusiveness*: the degree to which the virtual world blocks out the real world [91].

- *Extensiveness*: the range of sensory channels (visual, audio, haptic, olfactory etc.) provided in the environment [91]. While vision remains the primary channel in VE, studies show that the use of spatialised audio and the provision of other sensory channels increase the sense of spatial presence [23, 35, 36, 75, 82].

- *Vividness*: the degree of visual realism or fidelity of the virtual environment [91]. Studies have found that higher frame rate, stereopsis and large field of view can increase spatial presence [82]. Surprisingly, visual realism does not seem to have a significant impact [82].

- *Proprioceptive matching*: the degree to which the perceptual effects of movement by the user in the virtual world mirror their effects in the real world [91]. For instance, head tracking and the ability to walk through the virtual world increase presence [82].

- *Autonomous action*: the extent to which the participant can act autonomously and interact with objects in the VE and the affordances provided [30, 86, 102]. As Schultze writes: "The sense of 'being there' is grounded in the ability to 'do there.'" [79, 84].

- *Representation of virtual self*: the degree of realism of the representation of the participant in the VE. This affects self-presence and behaviour [43, 68].

- *Plausability*: the extent to which objects and actors in the virtual world exhibit real-world behaviour [89].

- *Representation of others*: the degree of realism of the representation of other actors in the VE [48].

- *Communication channels*: the range of ways, e. g. verbal and nonverbal, in which communication is made available to individuals or groups in the environment. Communication greatly benefits from awareness features in VE [66].
1.3.2. Psychological Immersion/Engagement

Even more central to immersive analytics research is understanding how to cognitively immerse the user in the analytics or decision-making task. Such task immersion has been studied by psychologists, computer game, user interface (UI) and other researchers.

Csikszentmihalyi et al. [20, 21] extensively studied intrinsically motivated activity in both work and play settings. They introduced the notion of *flow* to describe the enjoyable subjective state that people experience when they are completely involved in something to the point of forgetting time, fatigue and everything but the activity itself. This might come from reading an engrossing novel, playing chess or when engaging in sport. Flow is an example of psychological absorption, the term describing total engagement in the present experience. Unlike some other kinds of psychological absorption [39], flow is always enjoyable.

Flow is characterised by: intense and focussed concentration on what one is doing in the current moment; merging of action and awareness; loss of reflective self-consciousness; a sense of control; temporal disassociation (typically time passes faster than usual); and enjoyment of the activity itself rather than only the outcome [64]. Two necessary preconditions for flow are that the perceived challenges or opportunities stretch existing skills but are not over-demanding, and that there are clear sub-goals with immediate feedback on performance.

Flow has been applied to software use [1, 97, 100]. Building upon the notion of flow, Agawal and Karahanna [1] define cognitive absorption as “a state of deep involvement with software”. It is characterised by: focussed immersion and total engagement in using the software; temporal disassociation; enjoyment; a sense of control and curiosity. Like flow, cognitive absorption requires an extreme state of engagement.

It seems unrealistic to expect that immersive analytics systems will regularly engender the degree of immersion required for flow or cognitive absorption. A more general notion of software engagement has been studied by HCI researchers wishing to move beyond usability and to characterise and design more engaging interfaces [2, 69, 70]. They have found that an engaging experience is encouraged by the aesthetic and sensory appeal of the software and that the task should be regarded as interesting or novel, be sufficiently challenging and provide regular feedback. Engaged users are emotionally involved, motivated and perceive themselves to be in control. However, while the user must be focussed on the activity, engagement, unlike flow, does not require them to be so focussed that they lose all awareness of the surrounding world.

Software engagement and immersion has been most extensively studied for computer games [7, 11, 13]. Immersion is commonly used in the games industry to refer to the feeling of “being in the game”. Unlike flow, but like software engagement, games immersion is not an all or nothing experience [9]. The first level of immersion is called *engagement*, simply investing enough time and energy to play the game, the next level is *engrossment* at which point the players devote considerable effort and have an emotional attachment. The final level is *total immersion* in which the player is completely involved in the game and nothing
else matters. This is similar to flow, though more transient. It is important to note that being “in the game” does not refer to immersion in a virtual world but to a cognitive state and can occur in both digital and non-digital games.

Jennett *et al.* [42] identified five constituent factors to immersion: emotional involvement, cognitive involvement, real-world disassociation, challenge, and control, while Ermi and Mäyrä [29] identified three kinds of immersive experience: sensory, challenge-based, and imaginary. These are related: challenge-based corresponds to cognitive involvement, challenge, and control, while imaginary relates to emotional involvement and sensory to real-world disassociation through the use of immersive technologies to create a sense of presence [11]. More generally, Poels *et al.* [73] identified the following dimensions to descriptions of the gaming experience by digital game players: enjoyment, flow, imaginative immersion, sensory immersion, suspense, competence, control, social presence, as well as negative affect.

It is implicit in many discussions of immersion in computer gaming that the use of immersive technologies will increase player engagement and sense of “being there” in the game. However, study results are somewhat mixed [11, 33]. Social presence is now recognised as an important aspect of many digital games [22] and high social presence, i. e., the sense of being socially connected to others, leads to greater enjoyment [31] and to increased immersion [12]. There is some evidence that flow in immersive environments leads to better performance, though this may be confounded by non-congruent interfaces [4].

Narrative has been identified as another factor contributing to immersion [74] and increased engagement [33], though there is some disagreement as to whether digital games make use of narrative. If we mean the use of storytelling techniques to emotionally engage the player in the characters and setting of the game, then it is clear that some games utilise narrative while others, such as *Tetris*, do not [11].

Narrative as a device for immersion has been previously studied in literature and film-making [32, 65, 81]. Like engagement in games, immersion in a book or film is graduated. Nell [65] writes of *absorption* sometimes deepening to become *entrancement*, while Ryan [81] identifies four successively deeper levels of immersion: *concentration*, in which the reader is focusing on reading or understanding the text itself and is hardly immersed; *imaginative involvement* in which the users engages emotionally and imaginatively with the text but is also aware of the text and in the case of non-fiction will critically analyse the arguments; *entrancement* in which the reader is wholly engaged in the book and cannot easily put it down but is still aware that the textual world is not real; and the pathological state of *addiction* in which the reader cannot distinguish between the fictional and real world or compulsively reads book after book without finding pleasure. Immersion in a text or film relies “on being transported” to the textual world [32]. It may involve spatial or sensorial immersion, social connection with the characters, or game-like engagement in the plot.

The line between literature and film and new digital media is blurring. Increasingly, journalists are utilising computer games, online communities and immersive technologies to present news stories interactively. An extreme form of
such immersive journalism uses VR to recreate the news event and allow the user to experience it as one of the actors [71]. Undoubtedly, immersive analytics can learn from the techniques and tricks used by UI and game designers and journalists to engage their audience using immersive technologies, social connection, narrative as well as responsive interfaces that allow the user to feel in control through frequent feedback.

**1.4. Overview of this Book**

The chapters of this book study different aspects of immersive analytics, focusing on potential opportunities and applications. The chapters are self-contained and are intended to provide background for researchers and interested practitioners outside the field as well as a theoretical framework for future research. All chapters provide a list of research issues and questions.

In Chapter 2, Marriott *et al.* explore spatial immersion and investigate whether immersive analytic applications should continue to use traditional 2D information visualisations or whether there are situations when the use of 3D depth cues may offer benefits. They also discuss the opportunity that spatial immersion provides to arrange multiple views in the 3D space around the user.

In Chapter 3, McCormack *et al.* look at another important aspect of technological immersion. They review how non-visual sensory channels—audio, haptic, smell and taste—can be used to present data, both as an adjunct to vision and as a replacement when vision is unavailable. They summarise the physiological characteristics of each channel, current presentation technologies including data physicalisation, and provide a preliminary investigation of the resulting design space for multisensory data exploration.

As mentioned in Sections 1.2. and 1.3. of this introductory chapter, effective interaction is key to giving people a fluid and, hence, immersive experience while exploring their data. The state of the art in immersive interaction technologies and the ramifications of these is discussed by Büschel *et al.* in Chapter 4.

Chapter 5 discusses the broader question of how to immersively integrate the “human in the loop” of automated machine processes for processing data. In this chapter, Stuerzlinger *et al.* reinterpret the sense-making loop of visual analytics as it applies to user engagement during data exploration in immersive environments.

We have identified the potential of immersive analytics for data-driven storytelling (also known as narrative visualisation). This is explored by Isenberg *et al.* in Chapter 6. They give examples of immersive storytelling and review the theoretical frameworks for understanding data-driven narratives. In particular, they provide a detailed analysis of how immersion has been understood in related fields such as gaming and relates this back to data-driven storytelling.

Thomas *et al.*, in Chapter 7, explore another potential benefit of immersive analytics. They provide a theoretical framework for understanding situated analytics—the use of “data representations organised in relation to relevant objects, places, and persons in the physical world for the purpose of understanding,
sense-making, and decision-making”. They characterise its properties, give a number of examples, and clarify the relationship to related fields and concepts.

The chapter by Bezerianos et al. (Chapter 8) explores the shared use of immersive technologies to support collaborative analytical reasoning and decision making. They provide a taxonomy of collaborative systems based on space and time, give examples of immersive applications, and analyse the different roles people may play in these systems.

In Chapter 9, Marriott *et al.* present an initial design framework for immersive analytics that extends Brehmer and Munzner’s “What, Why and How” [8] data visualisation framework by considering display and interaction capabilities, collaboration and all sensory channels.

The two final chapters in the book look in more detail at application domains of immersive analytics. The chapter by Czauderna *et al.* (Chapter 10) investigates typical use-cases of immersive analytics in life and health sciences ranging from structural biology to neurosciences.

Chandler *et al.*, in Chapter 11, present immersive analytics applications to simulations of built environments. They investigate virtual prototyping tools, 3D simulation of urban areas, and the reconstruction and crowd simulation of the medieval Cambodian temple complex of Angkor Wat.

*Chapter Review Process* The chapter groups were formed across the Shonan and Dagstuhl immersive analytics workshops in 2016. Authors submitted manuscripts for their chapters in mid-2017. These initial versions were reviewed at first “internally” by one of the book editors. After an initial round of revisions based on the recommendations of these internal reviews, updated manuscripts were sent to expert reviewers invited from the community. We are grateful to these people for their detailed and insightful reviews.

**1.5. Research Agenda**

While the chapters provide a detailed introduction to the field of immersive analytics, they also make it clear that we have only begun to scratch the surface of this new field and that many questions remain. While the individual chapters detail research questions for a particular topic, we can group these under the following seven headings.

**1.5.1. Underpinning Theory**

We have seen how technological and psychological immersion has been studied for virtual environments [10], telepresence [94], telecommunications, and gaming [95]. A fundamental question for immersive analytics is how to modify these theories to identify and formalise the various kinds of immersion and engagement that might occur during data analytics and decision making. Such a theory would need to extend the current understanding of physical or spatial presence from representation of virtual worlds in virtual reality to representation of abstract
data in mixed reality [61] and even data physicalisation [10]. It will need to clarify
the links between technological and psychological immersion, the connection
between these and the emotional state of the viewer, as well as the relationship
with user performance.

Immersive analytics also requires the development of models for the processes
and workflows employed by users. Variants of the sense-making loop underpin
visual analytics [45, 72] and recently the problem-solving loop was introduced for
interactive optimisation [52]. Is it possible to develop a more general model of
human-in-the-loop decision making? Furthermore, current models do not capture
all of the activities of users in immersive analytics. They focus on the analyst, not
on the stakeholders trying to understand and evaluate the result of the analysis.
They do not adequately consider collaboration and the different roles that people
may play in this process (see Chapter 8).

1.5.2. Effectiveness

A necessary question is to identify when immersive analytics techniques are
useful. When (if ever) does the use of immersive technologies offer benefits over
current desktop visual analytics tools? Traditionally data visualisation researchers
have focussed on task performance, typically accuracy and speed. As Chapter 2
identifies, the use of depth cues may improve task effectiveness for some kinds
of visualisation tasks. However, task performance is not the only measure of
effectiveness. Emotional engagement and recall may also be important, especially
when communicating findings to stakeholders. There is some evidence that both
presence and emotional engagement improves recall [55]. In general, presence is
bi-directionally linked to the emotional state of the viewer: the emotional state
is influenced by the degree of presence and the feeling of presence is increased
in “emotional” environments [78]. More research is required to understand this
in the context of immersive data-driven narratives (see Chapter 6) and other
aspects of immersive analytics.

Such an understanding crucially relies on methodologies for measuring the
various aspects of user engagement and immersion [47]. These are most com-
monly evaluated using self-reported measure and a number of surveys have
been developed to measure spatial and social presence [5, 50, 66, 101] in virtual
worlds and immersion in games [22, 42, 74]. Less subjective measures include
behavioural measures such as the responses to virtual and external stimuli and
psychophysiological measures such as skin conductance, heart rate, or brain
activity [38, 46, 59].

1.5.3. Responsive Analytics

Few machine learning, data mining, optimisation, or simulation algorithms are
designed to be used in responsive, interactive analytics applications. Human-in-
the-loop algorithmics is still an emerging research field (see Chapter 5). More
research is required to understand the trade-offs between responsiveness, pre-
dictability and the quality of answer, as well as how to design systems that
leverage from the real-world knowledge of the human analyst.

**1.5.4. Work Flows and Work Spaces**

Immersive analytics, in particular situated analytics, allows very different work
flows in which analytics can be employed in virtually any process or situation.
What are general guidelines for doing this? Collaboration raises many additional
questions, e. g., division and allocation of work, group dynamics and consensus.
A related question is how to organise both shared and individual workspaces.
Some further background on these questions is provided in Chapters 5 and 8.

1.5.5. Design Space

Many of the chapters have identified the need to explore and understand the
design space for immersive analytics applications. Chapter 9 presents a prelimi-
nary framework but decades of research will be required to refine and validate
this framework. Particular areas that require attention are interaction, mixed
reality, collaboration and the use of audio and haptic feedback. The difficulty is
compounded by rapidly changing technology, which means that previous findings
and guidelines may no longer apply.

1.5.6. Applications

In this book, we have presented two possible application domains: life and health
sciences, and the built environment, but there are many more. Examples include
disaster management (where participants must quickly plan and direct reactions
to threats), healthcare, including personalised health, manufacturing applications,
and so on.

1.5.7. Societal Impact

The democratisation of data analysis and decision making has potentially far-reaching consequences for our society. There are also potential dangers: Will the use of emotionally engaging immersive data-driven narratives lead to less objective and possibly worse decisions by stakeholders (Chapter 6)? What will be the health effects and impact on social cohesion of increased use of virtual environments at work as well as at home?

Acknowledgements

Early efforts that prompted the conversations and workshops that have eventually
led to this book were strongly supported by the Monash University Faculty of
Information Technology. Chapters in this book were written by working groups
formed at workshops on Immersive Analytics at Shonan, Meeting 2016-2 [41], and Dagstuhl, Seminar 16231 [27]. Continuing work by the authors of this chapter was funded by the Australian Research Council Discovery Scheme, project DP180100755 and the German Research Foundation (DFG), project D4 of SFB/Transregio 161.

## References

1. Agarwal, R., Karahanna, E.: Time flies when you're having fun: cognitive absorption and beliefs about information technology usage. MIS Quarterly pp. 665-694 (2000)

2. Attfield, S., Kazai, G., Lalmas, M., Piwowarski, B.: Towards a science of user engagement (position paper). In: WSDM Workshop on User Modelling for Web Applications. pp. 9-12 (2011)

3. Bertin, J.: Semiology of graphics: diagrams, networks, maps. University of Wisconsin press (1983)

4. Bian, Y., Yang, C., Zhou, C., Liu, J., Gai, W., Meng, X., Tian, F., Shen, C.: Exploring the weak association between flow experience and performance in virtual environments. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI). pp. 401:1-401:12. ACM (2018)

5. Biocca, F.: The cyborg's dilemma: progressive embodiment in virtual environments. Journal of Computer-Mediated Communication 3(2), 12-26 (1997)

6. Biocca, F., Harms, C., Burgoon, J.K.: Towards a more robust theory and measure of social presence: Review and suggested criteria. Presence: Teleoperators & Virtual Environments 12(5), 456-480 (2003)

7. Boyle, E.A., Connolly, T.M., Hainey, T., Boyle, J.M.: Engagement in digital entertainment games: a systematic review. Computers in Human Behavior 28(3), 771-780 (2012)

8. Brehmer, M., Munzner, T.: A multi-level typology of abstract visualization tasks. IEEE Transactions on Visualization and Computer Graphics 19(12), 2376-2385 (2013)

9. Brown, E., Cairns, P.: A grounded investigation of game immersion. In: CHI'04 extended abstracts on Human Factors in Computing Systems. pp. 1297-1300. ACM (2004)

10. Burdea, G.C., Coiffet, P.: Virtual Reality Technology. John Wiley & Sons, Inc., New York, NY, USA, 2 edn. (2003)

11. Cairns, P., Cox, A., Nordin, A.I.: Immersion in digital games: review of gaming experience research. Handbook of Digital Games 339, 337-361 (2014)

12. Cairns, P., Cox, A.L., Day, M., Martin, H., Perryman, T.: Who but not where: the effect of social play on immersion in digital games. International Journal of Human-Computer Studies 71(11), 1069-1077 (2013)

13. Calleja, G.: In-game: From immersion to incorporation. MIT Press (2011)

14. Card, S.K., Mackinlay, J.D., Shneiderman, B.: Readings in information visualization: using vision to think. Morgan Kaufmann (1999)

15. Card, S.K., Robertson, G.G., Mackinlay, J.D.: The information visualizer, an information workspace. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 181-186. ACM (1991)

16. Chandler, T., Cordeil, M., Czauderna, T., Dwyer, T., Glowacki, J., Goncu, C., Klapperstueck, M., Klein, K., Marriott, K., Schreiber, F., Wilson, E.: Immersive analytics. In: IEEE 2015 Big Data Visual Analytics (BDVA). pp. 1-8 (2015)
17. Cleveland, W.S., McGill, R.: Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American Statistical Association 79(387), 531–554 (1984)

18. Cohen, I.B.: Howard Aiken on the number of computers needed for the nation. *IEEE Annals of the History of Computing* 20(3), 27–32 (1998)

19. Cruz-Neira, C., Sandin, D.J., DeFanti, T.A., Kenyon, R.V., Hart, J.C.: The CAVE: audio visual experience automatic virtual environment. Commun. ACM 35(6), 64–72 (1992)

20. Csikszentmihalyi, M.: Beyond boredom and anxiety. Jossey-Bass Publishers (1975)

21. Csikszentmihalyi, M., Abuhamdeh, S., Nakamura, J.: Flow. In: Flow and the Foundations of Positive Psychology, pp. 227–238. Springer (2014)

22. De Kort, Y.A., IJsselsteijn, W.A., Poels, K.: Digital games as social presence technology: development of the social presence in gaming questionnaire (SPGQ). *Proceedings of PRESENCE 195203* (2007)

23. Dinh, H.Q., Walker, N., Hodges, L.F., Song, C., Kobayashi, A.: Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. In: *Proceedings IEEE Virtual Reality*. pp. 222–228 (1999)

24. Dourish, P.: Where the action is: the foundations of embodied interaction. MIT Press (2004)

25. Draper, J.V., Kaber, D.B., Usher, J.M.: Telepresence. Human Factors 40(3), 354–375 (1998)

26. Durlach, N., Slater, M.: Presence in shared virtual environments and virtual togetherness. Presence: Teleoperators & Virtual Environments 9(2), 214–217 (2000)

27. Dwyer, T., Riche, N.H., Klein, K., Stuerzlinger, W., Thomas, B.: Immersive analytics (Dagstuhl seminar 16231). *Dagstuhl Reports* 6(6), 1–9 (2016)

28. Engelbart, D.C., English, W.K.: A research center for augmenting human intellect. In: *Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I*. pp. 395–410. AFIPS '68 (Fall, part I), ACM (1968)

29. Ermi, L., Mäyrä, F.: Fundamental components of the gameplay experience: analysing immersion. *Worlds in Play: International Perspectives on Digital Games Research* 37(2), 37–53 (2005)

30. Flach, J.M., Holden, J.G.: The reality of experience: Gibson's way. Presence: Teleoperators & Virtual Environments 7(1), 90–95 (1998)

31. Gajadhar, B.J., De Kort, Y.A., Ijsselsteijn, W.A.: Shared fun is doubled fun: player enjoyment as a function of social setting. In: *Fun and Games, pp. 106–117*. Springer (2008)

32. Gerrig, R.J.: Experiencing narrative worlds: on the psychological activities of reading. Yale University Press (1993)

33. Gorini, A., Capideville, C.S., De Leo, G., Mantovani, F., Riva, G.: The role of immersion and narrative in mediated presence: the virtual hospital experience. Cyberpsychology, Behavior, and Social Networking 14(3), 99–105 (2011)

34. Hartmann, T., Wirth, W., Vorderer, P., Klimmt, C., Schramm, H., Böcking, S.: Spatial presence theory: state of the art and challenges ahead. In: *Immersed in Media, pp. 115–135*. Springer (2015)

35. Hoffman, H.G.: Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments. In: *Proceedings IEEE 1998 Virtual Reality Annual International Symposium*. pp. 59–63 (1998)

36. Hoffman, H.G., Hollander, A., Schroder, K., Rousseau, S., Furness, T.: Physically touching and tasting virtual objects enhances the realism of virtual experiences. *Virtual Reality* 3(4), 226–234 (1998)
37. Huang, D., Tory, M., Aseniero, B.A., Bartram, L., Bateman, S., Carpendale, S., Tang, A., Woodbury, R.: Personal visualization and personal visual analytics. IEEE Transactions on Visualization and Computer Graphics 21(3), 420–433 (2015)

38. IJsselsteijn, W.A., de Ridder, H., Freeman, J., Avons, S.E.: Presence: Concept, determinants and measurement. In: Human Vision and Electronic Imaging. vol. 3959, pp. 520–529 (2000)

39. Irwin, H.J.: Pathological and nonpathological dissociation: the relevance of child-hood trauma. The Journal of Psychology 133(2), 157–164 (1999)

40. Isenberg, P., Heimerl, F., Koch, S., Isenberg, T., Xu, P., Stolper, C.D., Sedlmair, M., Chen, J., Möller, T., Stasko, J.: vispubdata.org: A metadata collection about IEEE visualization (VIS) publications. IEEE Transactions on Visualization and Computer Graphics 23(9), 2199–2206 (Sep 2017)

41. Itoh, T., Marriott, K., Schreiber, F., Wössner, U.: Immersive analytics: a new multidisciplinary initiative to explore future interaction technologies for data analytics. Shonan Reports (2016)

42. Jennett, C., Cox, A.L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., Walton, A.: Measuring and defining the experience of immersion in games. International Journal of Human-Computer Studies 66(9), 641–661 (2008)

43. Jin, S.A.A.: Avatars mirroring the actual self versus projecting the ideal self: The effects of self-priming on interactivity and immersion in an exergame, wii fit. CyberPsychology & Behavior 12(6), 761–765 (2009)

44. Kay, A., Goldberg, A.: Personal dynamic media. Computer 10(3), 31–41 (1977)

45. Kerren, A., Schreiber, F.: Toward the role of interaction in visual analytics. In: Rose, O., Uhrmacher, A.M. (eds.) Proceedings Winter Simulation Conference. p. 420 (2012)

46. Kober, S.E., Kurzmann, J., Neuper, C.: Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: an EEG study. International Journal of Psychophysiology 83(3), 365–374 (2012)

47. Laarni, J., Ravaja, N., Saari, T., Böcking, S., Hartmann, T., Schramm, H.: Ways to measure spatial presence: review and future directions. In: Immersed in Media, pp. 139–185. Springer (2015)

48. Latoschik, M.E., Roth, D., Gall, D., Achenbach, J., Waltemate, T., Botsch, M.: The effect of avatar realism in immersive social virtual realities. In: 23rd ACM Symposium on Virtual Reality Software and Technology (VRST). pp. 39:1–39:10 (2017)

49. Leap Motion. http://www.leapmotion.com, accessed: January 2018

50. Lee, K.M.: Presence, explicated. Communication Theory 14(1), 27–50 (2004)

51. Lessiter, J., Freeman, J., Keogh, E., Davidoff, J.: A cross-media presence questionnaire: the itc-sense of presence inventory. Presence: Teleoperators & Virtual Environments 10(3), 282–297 (2001)

52. Liu, J., Dwyer, T., Marriott, K., Millar, J., Haworth, A.: Understanding the relationship between interactive optimisation and visual analytics in the context of prostate brachytherapy. IEEE Transactions on Visualization and Computer Graphics 24(1), 319–329 (2018)

53. Lombard, M., Ditton, T.: At the heart of it all: the concept of presence. Journal of Computer-Mediated Communication 3(2) (1997)

54. Lombard, M., Jones, M.T.: Defining presence. In: Immersed in Media, pp. 13–34. Springer (2015)

55. Makowski, D., Sperduti, M., Nicolas, S., Piolino, P.: Being there and remembering it: presence improves memory encoding. Consciousness and Cognition 53, 194–202 (2017)
56. International journal of man-machine studies (From 1969 to 1993)

57. Mantovani, G., Riva, G.: ?Real? presence: how different ontologies generate different criteria for presence, telepresence, and virtual presence. Presence: Teleoperators & Virtual Environments 8(5), 540–550 (1999)

58. McCormick, B.H.: Visualization in scientific computing. Computer Graphics 21(6), 1–14 (1987)

59. Meehan, M., Insko, B., Whitton, M., Brooks Jr, F.P.: Physiological measures of presence in stressful virtual environments. ACM Transactions on Graphics (TOG) 21(3), 645–652 (2002)

60. Microsoft Kinect. https://en.wikipedia.org/wiki/Kinect, accessed: January 2018

61. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems 77(12), 1321–1329 (1994)

62. Minsky, M.: Telepresence. Omni (1980)

63. Munzner, T.: Exploring large graphs in 3D hyperbolic space. IEEE Computer Graphics and Applications 18(4), 18–23 (1998)

64. Nakamura, J., Csikszentmihalyi, M.: The concept of flow. In: Flow and the Foundations of Positive Psychology, pp. 239–263. Springer (2014)

65. Nell, V.: Lost in a book: the psychology of reading for pleasure. Yale University Press (1988)

66. Nguyen, T.T.H., Duva, T.: A survey of communication and awareness in collaborative virtual environments. In: Proceedings of IEEE International Workshop on Collaborative Virtual Environments (3DCVE) (2014)

67. Nowak, K.: Defining and differentiating copresence, social presence and presence as transportation. In: Presence 2001 Conference, Philadelphia, PA. pp. 1–23 (2001)

68. Nowak, K.L., Biocca, F.: The effect of the agency and anthropomorphism on users’ sense of telepresence, copresence, and social presence in virtual environments. Presence: Teleoperators & Virtual Environments 12(5), 481–494 (2003)

69. O’Brien, H.L., Toms, E.G.: What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the Association for Information Science and Technology 59(6), 938–955 (2008)

70. O’Brien, H.L., Toms, E.G.: The development and evaluation of a survey to measure user engagement. Journal of the Association for Information Science and Technology 61(1), 50–69 (2010)

71. De la Peña, N., Weil, P., Llobera, J., Giannopoulos, E., Pomés, A., Spanlang, B., Friedman, D., Sanchez-Vives, M.V., Slater, M.: Immersive journalism: immersive virtual reality for the first-person experience of news. Presence: Teleoperators & Virtual Environments 19(4), 291–301 (2010)

72. Pirolli, P., Card, S.: The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proceedings of International Conference on Intelligence Analysis. vol. 5, pp. 2–4 (2005)

73. Poels, K., De Kort, Y., Ijsselsteijn, W.: It is always a lot of fun!: exploring dimensions of digital game experience using focus group methodology. In: Proceedings of the 2007 Conference on Future Play. pp. 83–89. ACM (2007)

74. Qin, H., Patrick Rau, P.L., Salvendy, G.: Measuring player immersion in the computer game narrative. International Journal of Human-Computer Interaction 25(2), 107–133 (2009)

75. Ranasinghe, N., Jain, P., Thi Ngoc Tram, N., Koh, K.C.R., Tolley, D., Karwita, S., Lien-Ya, L., Liangkun, Y., Shamaiah, K., Eason Wai Tung, C., Yen, C.C., Do, E.Y.L.: Season traveller: Multisensory narration for enhancing the virtual reality experience. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI). pp. 577:1–577:13. ACM (2018)
76. Ratan, R.: Self-presence, explicated: body, emotion, and identity. Handbook of Research on Technoself: Identity in a Technological Society (2012)

77. Riva, G., Davide, F., Ijsselsteijn, W.: Being there: the experience of presence in mediated environments. Being there: Concepts, Effects and Measurement of User Presence in Synthetic Environments 5 (2003)

78. Riva, G., Mantovani, F., Capideville, C.S., Preziosa, A., Morganti, F., Villani, D., Gaggioli, A., Botella, C., Alcañiz, M.: Affective interactions using virtual reality: the link between presence and emotions. CyberPsychology & Behavior 10(1), 45–56 (2007)

79. Riva, G., Waterworth, J.A., Waterworth, E.L., Mantovani, F.: From intention to action: The role of presence. New Ideas in Psychology 29(1), 24–37 (2011)

80. Robertson, G.G., Mackinlay, J.D., Card, S.K.: Cone trees: animated 3D visualizations of hierarchical information. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 189–194. ACM (1991)

81. Ryan, M.L.: Narrative as virtual reality: immersion and interactivity in literature and electronic media. Johns Hopkins University Press (2001)

82. Sanchez-Vives, M.V., Slater, M.: From presence to consciousness through virtual reality. Nature Reviews Neuroscience 6(4), 332–339 (2005)

83. Schuemie, M.J., Van Der Straaten, P., Krijn, M., Van Der Mast, C.A.: Research on presence in virtual reality: a survey. CyberPsychology & Behavior 4(2), 183–201 (2001)

84. Schultze, U.: Embodiment and presence in virtual worlds: a review. Journal of Information Technology 25(4), 434–449 (2010)

85. Sheridan, T.B.: Musings on telepresence and virtual presence. Presence: Teleoperators & Virtual Environments 1(1), 120–126 (1992)

86. Sheridan, T.B.: Further musings on the psychophysics of presence. Presence: Teleoperators & Virtual Environments 5(2), 241–246 (1996)

87. Shneiderman, B.: Direct manipulation: A step beyond programming languages. In: ACM SIGSOC Bulletin. vol. 13 (2-3), p. 143. ACM (1982)

88. Slater, M.: Measuring presence: A response to the Witmer and Singer presence questionnaire. Presence: Teleoperators & Virtual Environments 8(5), 560–565 (1999)

89. Slater, M.: Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society of London B: Biological Sciences 364(1535), 3549–3557 (2009)

90. Slater, M., Lotto, B., Arnold, M.M., Sanchez-Vives, M.V.: How we experience immersive virtual environments: the concept of presence and its measurement. Anuario de psicología/The UB Journal of psychology 40(2), 193–210 (2009)

91. Slater, M., Wilbur, S.: A framework for immersive virtual environments five: speculations on the role of presence in virtual environments. Presence: Teleoperators & Virtual Environments 6(6), 603–616 (1997)

92. Sutherland, I.E.: Sketchpad a man-machine graphical communication system. Transactions of the Society for Computer Simulation 2(5), R-3 (1964)

93. Sutherland, I.E.: A head-mounted three dimensional display. In: Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I. pp. 757–764. AFIPS '68 (Fall, part I), ACM (1968)

94. Tachi, S.: Telexistence. In: Virtual Realities, pp. 229–259. Springer (2015)

95. Thomas, B.H.: A survey of visual, mixed, and augmented reality gaming. Computers in Entertainment (CIE) 10(1), 3 (2012)

96. Thomas, J.J., Cook, K.A.: Illuminating the path: the research and development agenda for visual analytics. National Visualization and Analytics Ctr (2005)
97. Trevino, L.K., Webster, J.: Flow in computer-mediated communication: electronic mail and voice mail evaluation and impacts. Communication Research 19(5), 539–573 (1992)

98. Tufte, E.R.: Envisioning information. Graphics press (1990)

99. Tukey, J.W.: Exploratory data analysis. Reading, Mass. (1977)

100. Webster, J., Trevino, L.K., Ryan, L.: The dimensionality and correlates of flow in human-computer interactions. Computers in Human Behavior 9(4), 411–426 (1993)

101. Witmer, B.G., Singer, M.J.: Measuring presence in virtual environments: a presence questionnaire. Presence: Teleoperators & Virtual Environments 7(3), 225–240 (1998)

102. Zeltzer, D.: Autonomy, interaction, and presence. Presence: Teleoperators & Virtual Environments 1(1), 127–132 (1992)