# V-Mail: 3D-Enabled Correspondence about Spatial Data on (Almost) All Your Devices

Jung Who Nam, Tobias Isenberg, Daniel Keefe

► To cite this version:

Jung Who Nam, Tobias Isenberg, Daniel Keefe. V-Mail: 3D-Enabled Correspondence about Spatial Data on (Almost) All Your Devices. IEEE Transactions on Visualization and Computer Graphics, 2024, 30 (4), pp.1853-1867. 10.1109/TVCG.2022.3229017. hal-03924707

HAL Id: hal-03924707

https://inria.hal.science/hal-03924707

Submitted on 5 Jan 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# V-Mail: 3D-Enabled Correspondence about Spatial Data on (Almost) All Your Devices

Jung Who Nam<sup>D</sup>, Tobias Isenberg<sup>D</sup>, and Daniel F. Keefe<sup>D</sup>, Senior Member, IEEE

**Abstract**—We present V-Mail, a framework of cross-platform applications, interactive techniques, and communication protocols for improved multi-person correspondence about spatial 3D datasets. Inspired by the daily use of e-mail, V-Mail seeks to enable a similar style of rapid, multi-person communication accessible on any device; however, it aims to do this in the new context of spatial 3D communication, where limited access to 3D graphics hardware typically prevents such communication. The approach integrates visual data storytelling with data exploration, spatial annotations, and animated transitions. V-Mail “data stories” are exported in a standard video file format to establish a common baseline level of access on (almost) any device. The V-Mail framework also includes a series of complementary client applications and plugins that enable different degrees of story co-authoring and data exploration, adjusted automatically to match the capabilities of various devices. A lightweight, phone-based V-Mail app makes it possible to annotate data by adding captions to the video. These spatial annotations are then immediately accessible to team members running high-end 3D graphics visualization systems that also include a V-Mail client, implemented as a plugin. Results and evaluation from applying V-Mail to assist communication within an interdisciplinary science team studying Antarctic ice sheets confirm the utility of the asynchronous, cross-platform collaborative framework while also highlighting some current limitations and opportunities for future work.

**Index Terms**—Human-computer interaction, visualization of scientific 3D data, communication, storytelling, immersive analytics.

<figure><img src="image_2.png" /></figure>

## 1 INTRODUCTION

SPATIAL data visualization has long been essential for breakthroughs in medicine, science, engineering, the humanities, and more (e. g., [1], [6], [19], [42]). Sharing these visualizations with others, however, remains a major challenge for many 3D, scientific, and/or big data visualization applications because they typically require high-end devices and data processing—the type of computing infrastructure that is not portable and not accessible to most non-experts. One strategy for addressing this problem is to port the exploratory visualization tools scientists use on their current high-end visualization systems (e. g., advanced volume rendering) to the commodity devices that people use regularly, such as through remote rendering [27] or new web technologies [30], [39]. A fundamental limitation, however, is that scientific visualization applications are often complex, requiring substantial expertise on the data and on how to use software features. Thus, even when it is possible for applications to run on commodity devices, there are still serious challenges to overcome in order to engage new audiences (e. g., people who may have little incentive to explore the data) and, often, even our team-science collaborators, who take on advisory roles in developing visualization techniques.

In light of this situation, we take a different, complementary approach. We argue that bringing full-fetched data exploration capabilities to every device may not be the most important requirement of high-end collaborative

visualization for spatial 3D data. Instead, what is fundamentally needed is to improve *human-to-human communication* about the data, and the way to do that is to make this *communication* work on everyone's existing devices. This approach conceptually builds closely on prior work in storytelling with data visualizations [8], [20], [32], which has been explored extensively for 2D abstract data but less so for spatial 3D data. By adopting a keyframe-based storytelling metaphor [3], [41], we have developed a system that creates animated data stories from data views captured directly from exploratory visualizations along with a mechanism of using an URL of a standard video file as the base 'token' for distributing a data story. By packing the communication about the data into a standard video format and sending this critical human-to-human communication rather than the raw data and visualization application, our system leverages existing e-mail and social media applications as tools for distributing stories.

Many forms of storytelling are, however, collaborative activities by nature [21], [24], [43], and people often take on different roles in this process [7], [12], [24]. We, thus, also support tasks that data story audiences may carry out after an initial viewing of a story such as providing feedback to the original author, verifying claims made in the story, or even adding new content to the story.

To turn data storytelling into a collaborative and iterative process, we extend our system and introduce the V(isualization)-Mail framework. The key approach is flexible export of data story where a single V-Mail story token can trigger experiences with different storytelling fidelities, based on the absence or presence of dedicated software (Fig. 1). By having options from simply viewing the story on a video player to opening it within specialized visualization software to explore and annotate the 3D data, it is possible to cover

* Jung Who Nam and Daniel F. Keefe are with the University of Minnesota, USA. E-mail: { namxx054 | dfk } @umn.edu.

* Tobias Isenberg is with Université Paris-Saclay, CNRS, Inria, LISN, France. E-mail: tobias.isenberg@inria.fr.
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 1. Three working platforms that demonstrate different storytelling fidelities of V-Mail for analyzing Antarctic ice-sheet data: (right) anyone with a video player can at least passively view the story, including annotations made by others; (left) in the highest-fidelity case, the story unlocks data on a V-Mail server than can be loaded via a plugin for desktop-based visualization applications, where users can explore and annotate the 3D data more deeply; (middle) the mobile client works as a custom video player with mechanisms for interactively viewing the story and adding annotations.</figcaption></figure>

a wide range of use-cases for collaborative discussion and 3D data analysis, including making it possible for users with different tasks and levels of familiarity with visualization applications to work in unison.

We also present several extensions to this core idea to further enhance 3D-specific data storytelling. For example, since understanding the spatial context for specific data views is often critically important, the views are represented as 3D frames floating in the same space as the data visualization, enabling the audience to see them from various vintage points. In addition, to increase clarity, we employ a staged animation when the transition between data views would require changes to several visualization parameters, for example, when new data filters or color maps have been applied.

In summary, the main contributions of this article are:

* the design of a novel visualization framework to facilitate 3D-enabled multi-person correspondence about spatial datasets for team science,

* an extension of keyframe-based storytelling metaphors to support co-authorship, treating the created stories as collaborative documents authored asynchronously,

* the design of a data story server and communication protocol for using V-Mail across an ecosystem of hardware platforms that seamlessly adapt the fidelity of the experience to the hardware, and

* user feedback and lessons learned from a first evaluation of using V-Mail in a real team-science setting where team members utilized all V-Mail client platforms we created (desktop app, mobile app, and native video players on multiple mobile and web-based devices).

## 2 RELATED WORK

V-Mail relates to research on interactive tools, techniques, and formats for storytelling with data visualization and builds upon multiple topics (e. g., animation and annotation, interactivity, co-authorship and collaboration) that have become consistent themes in this research. We note that, although storytelling techniques have been used for presenting both spatial [28], [29], [37], [41] and non-spatial [4], [11], [23], [24], [33], [35] data, recent work has focused more consistently on non-spatial data. V-Mail, therefore, builds upon a trend of customizing storytelling support tools for specific types of data (e. g., [3], [26], [36], [41]), but does so for the specific case of spatial 3D data, as often encountered in domains such as physics, astronomy, biology, and medicine.

### 2.1 Co-Authoring and Interacting with Stories

Many forms of storytelling are, by nature, collaborative activities [21], [24], [43] and people often take on different roles in this process [7], [12], [24]. In a recent survey, Tong et al. [37] describe authoring tools for parallel storytelling, where multiple authors work simultaneously to create the final visualization content. Like V-Mail, this type of storytelling blurs the lines between storyteller and audience; the audience partakes in the storytelling and the initial storyteller is also an audience member. In sense.us [14], for example, users communicate collaboratively using annotations and comments posted online on web-based data representations. The audience also partakes in the storytelling process in GeoTime [9]. In an accompanying text editor, authors write text-based data stories using bookmarked data views as references to claims. By scrolling the editor filled with the links, the audience has the ability not only to see the message with text descriptions but also to flip through saved visualization states for further exploration. Further, by modifying text in the editor, the audience can add new content or even new data views found in their own data analysis sessions.

V-Mail is similar to GeoTime in that multiple users collaborate on visualization presentations. Unlike GeoTime, however, V-Mail employs several simple yet effective techniques to convey different users' contributions. For example, as in the work by Heer, Viegas, et al. [14], [40], each comment shows the author of the edit. Further, V-Mail incorporates lower-fidelity options for viewing and collaborating with others on low-cost devices such as mobile phones. By sharing a visualization state via a URL [14], [40], V-Mails can be referenced in other communication media such as personal blogs. Similarly, by using video files as the means of distribution [15], V-Mails can be widely distributed, including through social media. V-Mails provide users with different options for consuming data stories, from passive viewing to interacting with them, as there is no one correct level of interactivity on the spectrum for constructing a narrative visualization [12], [29], [35]. While interaction can distract viewers from an initial story arc [21], the ability to interactively view data stories is typically seen as giving the audience a feeling of involvement and lends credibility [22], [28], [29], [41], [43].

Existing web-based systems that rely on remote rendering [27] or new web technologies [30], [39] can also allow people to carry out visualization tasks on the commodity devices they use regularly. These systems, however, often target data exploration tasks, whereas we focus on data communication with the support for collaborative annotation and storytelling.
Also, our first prototype provides storytelling techniques on top of visual outputs from exploratory visualization systems, so the V-Mail concept could be applied to visualization software built as standalone applications.

## 2.2 Annotating Spatial 3D Data

When compared to creating data stories for 2D or non-spatial data, authoring stories about 3D spatial data includes another special consideration in that it requires connecting text, speech, drawings, or other input from authors to specific spatial views or data elements. These software features are often described as 3D annotations, which can be tied directly to data objects [38], [41]. Alternatively, graphical annotations can be drawn in 'screen-space' [3], [14], [26], [38]. Without the need to explicitly reference data elements, this 'screen-space' drawing with 'free-form' annotations enables users to be more expressive [14] and even to draw in 'empty space,' which can be useful in sparse 3D scenes [38]. However, free-form 'screen-space' annotations are typically not useful when viewed from other vantage points, as the spatial context is only valid from the exact viewpoint where the annotations were made. Some form of navigation between annotations is needed along with an ability to see what viewpoints include annotations. V-Mail thus implements screen-space annotations in the style of Boom Chameleon [38]. After creating an annotation from one viewpoint, we add a snapshot of the annotated view to the scene as a 3D widget (view frustum with a textured film plane) that can be seen from other vantage points and returned to later.

## 2.3 Arranging and Transitioning Between Multiple Views

Many data storytelling systems provide multiple complementary data views, and they are considered critical for 3D data storytelling [3], [37], [41]. One reason is that understanding the complex spatial relationships in 3D data often requires multiple and/or animated perspectives [21], [25], [29], [35]. By smoothly transitioning between these perspectives, well-designed animations can allow the audience to build a congruent mental model of the spatial data [11]. Different mechanisms exist for authoring animations for data storytelling. These can be as simple as recording a sequence of interactions performed by the storyteller [41] or providing animation templates to assist users in creating a final animation [3]. Authoring a meaningful animation, however, can be complicated. Automated techniques are thus also useful, especially in time-critical scenarios such as urgent care in a hospital [15] and even for science dissemination [22]. V-Mail, adopts a semi-automatic approach, similar to that of Lidal et al. [26]; the animations are created by interpolating the values the storyteller has captured/created in discrete states. We extend this concept by utilizing a staged animation [3], [13] to increase clarity when the transition would require changes to several parameters at once. Additionally, V-Mail includes a linked 2D storyboard view to define the temporal context for each key state and to re-sequence the story if desired. This feature is motivated by prior work that created guided 3D visualizations by capturing and ordering multiple views and found that breaking the presentation into steps can improve comprehensibility [41].

# 3 DESIGN CRITERIA AND CORE DESIGN CONCEPTS

Based on this discussion of existing means for communicating about or using spatial 3D data and the challenges we outlined in the previous sections, we now translate these points into specific design criteria and introduce the design concepts of V-Mail that guided us toward our vision.

## 3.1 Design Criteria

Our overall approach is driven by today's huge spectrum of data communication possibilities, yet with no solution that matches our vision. On one end of the spectrum, people use e-mails enriched with screenshots from general-purpose visualization tools such as ParaView [2]. This has the advantage that one does not have to worry about the device of the addressee, yet messages are limited to texts with few images, with little expressiveness. Video exports can also be used to showcase visual representations with higher fidelity and in a continuous fashion, yet this only supports one-way communication. Moreover, communication features such as drawing and commenting are typically missing in e-mail based communication. The other end of the spectrum is marked by storytelling tools, such as that by Wohlfart and Hauser [41], in which recipients can load a shared visualization not only to view but also to explore. Unfortunately, such tools require collaborators to have a similar hardware and software setup and often hours of training to use the visualization application. As argued previously, this often makes them accessible only to experts. Our key goal is thus to facilitate communication about spatial 3D data without having to worry about recipients' devices and skill levels, yet with an ease of communication, responding, and commenting that approaches that of e-mail. To achieve this vision, we formulated the following design criteria, which are closely informed by ongoing data visualization research on storytelling, communication, and collaboration:
As audiences have different goals and needs,

C1 we need to provide them with different options for consuming the presented material, e. g., with different levels of interaction and detail [12], [24], [43] and

C2 we want to support a range of devices to view messages and provide feedback so users can pick a visualization suitable for their tasks and current situations [16].

As there can be multiple ways of interpreting results,

C3 messages should capture the thought process that lead to a conclusion so others can make informed decisions [26], [31], [41]

C4 and, when going through the steps of presenting spatial data, it is important to a method should be provided for the audience to maintain spatial correspondence as the presentation proceeds and to understand where, spatially within the data, each step takes place [22], [29], [37], [38],

C5 presenters should thus have various options to express their ideas and thoughts and to draw the audience's attention to particular viewpoints and data features [26], and

C6 the audience should have the ability to add to and edit the content so that groups can exchange ideas back and forth iteratively as they work collaboratively to understand and interpret the impact of the data [24].
## 3.2 V-Mail Core Metaphor and Design Concepts

The core metaphor behind V-Mail is that of a book—specifically, we recall children’s picture books that include text and imagery on each page and are often organized so that each page functions as a mini “episode” of the story. A child looks at a page in a book and reads the text at their own pace to understand the setting and characters pictured. Then, to advance, the child *flips to the next page* to continue to the next episode. Books for adult readers follow a similar progression of steps, but this is often arranged at the level of chapters. The relationship of one chapter to the next may not be immediately apparent, but the perspective provided by each chapter is, ultimately, critical to the interpretation of the whole book. Books of all forms naturally lend themselves to a variety of annotations, from hand-drawn illustrations to remarks scribbled in the margins.

In V-Mail's approach to storytelling with spatial 3D data, a 'page' is a **view of the 3D dataset** that shows a particular situation. Annotations can be provided by **sketching** onto the 2D projection of the 3D view or by attaching text messages to it. 'Flipping to the next page' **animates from one 3D view to the next**, allowing viewers to follow along while maintaining spatial context. Several page flips can be connected into **'chapters'**, but the transition from one chapter's end to the start of the next can be more abrupt (i. e., without an animated transition). As in the children's picture book, it is possible to animate **forward and backward** through the data story, with **different playback modes** to control the time with precision or to quickly skip to a new chapter. With this approach, we break complex messages down into more digestible, visual, and expressive forms (C3–C5).

The V-Mail frameworks requires just three additional concepts to be added to this core "book metaphor" functionality:

**Floating 3D pages**—When presenting visualization results with a set of captured data views, we often think of a panel listing the views; e. g., thumbnails on the viewer in AniViz [3]. V-Mail includes such a panel, but in addition to this panel, the data views are also included directly within the spatial visualization of the underlying 3D data. Together with the smooth transitions between the data views achieved with animations, these floating pages assist users in understand the spatial correspondences between views as they progress through a data story (C4).

**Living documents**—To remove the one-way character of existing 3D data correspondence and turn data communication into a collaborative and iterative process accessible to non-experts, V-Mail treats data stories as living documents (C1, C6). The framework makes it possible for users to explore a V-Mail, provide annotations, and extend stories in a way that scales with the capabilities of their used device.

Communication via a **lightweight token**—To realize this device flexibility (C2) and thus be able to engage a broad audience (C1), V-Mail does not rely on dedicated rendering support. Instead, stories are shared as a video URL—online token that can be interpreted based on the used device and the absence or presence of dedicated software. The video-based, low-fidelity version of the data story makes it possible for users to open it on a mobile phone with a video player, for instance while traveling. However, this lowest level of fidelity supports only simple viewing options (e.g.,

<figure><img src="image_6.png" /><figcaption>Fig. 2. The V-Mail desktop client allows users to create a story directly from a visualization application: (a) the top portion is the visualization application; (b) the *storyboard panel* is placed on the bottom; (c) captured *data views* in a story; (d) the * timeline interface* provides playback options and indicates the current time in a story; (e) the *drop-down menu* provides options for opening, uploading, and sharing a story; (f) the *comment panel* shows comments on the current data view.</figcaption></figure>

the web client in Table 1). A range of higher-fidelity versions that require dedicated software offer additional exploration and annotation capabilities. On a high-end desktop, users can freely explore the data depicted in the story and add additional pages or chapters (e.g., the desktop client in Table 1). A middle-fidelity version only supports exploration at certain view locations but with fairly advanced annotation capabilities and story navigation features (e.g., the mobile client in Table 1). To make it easy to manage such device flexibility, V-Mail associates all story data with the video, and shares it as a URL token that is opened appropriately by the different devices.

# 4 DETAILED DESIGN AND IMPLEMENTATION

We now describe the visual components, interaction techniques, and distribution mechanisms required to realize this vision.

## 4.1 Visual Components

A V-Mail story is composed of data views with drawings and comments from different users. We add animations in-between the data views to convey their spatial correspondences in a continuous fashion. Our V-Mail desktop client is one example of a V-Mail client (Section 4.3.3) and is a plugin to an exploratory visualization application [18]. The plugin is comprised of three visual components: (1) the storyboard, (2) floating 3D data views, and (3) animated-transitions.

### 4.1.1 The Storyboard with the Comment Panel

This first visual component shows an overview of a data story and the input from contributors (Fig. 2(b)). *Thumbnails* of data views provide a quick glance at important points in a story and its presentation order (Fig. 2(c)). We also overlay *drawings* on the thumbnails to indicate visual highlights from users. Sometimes it may be more appropriate to textually describe what is shown in a shared data view; e. g., for writing a case number in a data ensemble or the name of a rendering algorithm. The *comment panel* provides an interface for leaving such notes and viewing comments left by others (Fig. 2(f)); each comment includes the author's name.
<figure><img src="image_7.png" /><figcaption>Fig. 3. Example usages of 3D data views in data communication: (a) the user talks about data features focused in the views; (b) a bird's eye view shows the locations being discussed in the data.</figcaption></figure>

### 4.1.2 Floating 3D Data Views

This second visual component shows the user how the story is spatially located in the 3D data space. As shown in Fig. 3 and inspired by Tsang et al. [38], captured data views are rendered in the same space as the underlying data visualization so that they are visible from other vantage points. If a data view is captured close to a feature of interest, the user can zoom out and see a representation of that viewpoint in the context of a wider angle view of the data. By making this spatial information visually explicit, users can also visually reference other data views from the current view: a shot can show adjoining data views to convey their spatial relationship, before transitioning to each view (Fig. 3(a)). A spatial overview of all of the activities in a data analysis session can also be easily created by showing the data views scattered around visualization, from a bird's-eye perspective (Fig. 3(b)).

### 4.1.3 Animated Transitions

Animated transitions help to convey the spatial and contextual correspondences between data views in a story. Instead of jumping from one data view to the next, we use animated transitions that the audience can gradually adapt to the visual changes in different data views. The animated transitions are created automatically whenever a new data view is added

to minimize the time and cognitive effort needed to create a V-Mail story.

Four types of animated transitions (Fig. 4) can be used, depending on the differences between the viewpoints and visualization states in the adjoining data views. If only the camera location changes, the animation (1) translates the camera from a starting location to the next location.¹ When only the visualization state has changed, the animation (2) gradually changes the values of visualization parameters from the previous state to the next by interpolating the values of just the parameters that have changed. Users may stop the transition to explore the visualization in an intermediate frame, but this exploration is only possible when the set of visualization parameters of two states are the same and the parameters can be changed continuously (e. g., changing a slicing plane in volume rendering). In cases where there are no obvious intermediate frames (e. g., changing a visualization type or dataset) we (3) use an image-based blending approach. This blends two images where the screenshot from the previous data view fades-out and gradually fades-in into that of the next data view (see the second row of Fig. 4). When both camera location and state change, we (4) break the transition into two stages. First the camera moves to the next location. Then, the visualization state gradually changes to that of the new data view (see the last row of Fig. 4).

## 4.2 Basic Story Interaction

We introduce several interaction techniques to create, view, and explore V-Mails.

1. In our prototype, a linear interpolation is used; however, other motions that optimize camera motion to support specific goals could be useful. We thus implemented the animated transitions as abstract methods to enable future developers to override default animations with alternatives.

<figure><img src="image_8.png" /><figcaption>Fig. 4. Different animations to transition from one data view to another. Each type shows the changes in-between two adjacent data views.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Fig. 5. The user sketches and leaves comments on the current view in the annotation mode. A drawing pallet is for specifying the color and the size of a brush. Clicking the toggle on the top of the storyboard switches the current mode to the exploration mode.</figcaption></figure>

### 4.2.1 Creating a story and annotating data views

We begin creating a story with an empty storyboard. Users can navigate in the data visualization application (Fig. 2(a)) using typical 3D view manipulation, and then add a data view to the storyboard via a dedicated button ("+" in Fig. 2(b)). We then capture the view's thumbnail and insert it as a reference into the storyboard, also extending the timeline (Fig. 2(d)) appropriately (for ≥ 2 thumbnails) to account for the needed transition animations. Users can also re-arrange the data view sequence using simple drag-and-drop interaction, and they can remove data views as desired.

To be able to tell a story effectively or to respond to one, it is also often useful to sketch directly on a view to direct someone's attention to particular features, which we support on captured data views (Fig. 5). To avoid conflicts between sketching and view manipulation, we differentiate explicitly between these two modes and users can switch between them using two buttons at the bottom of the data view. In addition to the sketches, longer textual comments can be typed and associated with the data views (Fig. 2(f)).

### 4.2.2 Uploading a story

When ready to share the V-Mail, users upload the story to a server. Instead of uploading a story to the server whenever there is a change, we initiate this upload process explicitly (via an entry in the drop-down menu). We chose this design to enable the user to ideate and draft a story before sharing it to others, similar to drafting an e-mail before sending it.

### 4.2.3 Viewing an animated data story

As a sequence of data views with animated transitions, a V-Mail story is akin to a video with chapters. We thus borrow different playback modes from traditional video players and adjust them to our application (Fig. 6).

As in normal video players, playback continues until the end of the story (top row in Fig. 6). However, different from normal videos, the V-Mail player pauses briefly at each 'chapter mark' to devote additional time to and indicate the significance of these key perspectives. To further emphasize the data views, the player also includes mode where playback stops completely at each data view so users may read annotations in detail (second row in Fig. 6) and respond to them—similar to the playback style used by Wohlfahrt and Hauser [41]. Finally, for faster navigation, the player also

<figure><img src="image_10.png" /><figcaption>Fig. 6. Playback options & corresponding buttons. Top: plays the animation from start to finish, or in reverse; middle: plays it until it reaches the next/previous data view; bottom: jumps to the next/previous data view.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 7. The data visualization application and the storyboard are visually and interactively linked. (a) Hovering over a 3D data view frame in the 3D view (the center one here) highlights the frame and its corresponding thumbnail on the storyboard; (b) clicking on a data view frame in the 3D view makes the storyboard jump to this data view.</figcaption></figure>

supports jumping between data views without animations (bottom row in Fig. 6).<fnref n="2" /> Another aspect of data stories that is different from normal videos is that it can be useful to view data stories, especially the transitions between views, in reverse. Thus, all three of the play modes include support for reverse playback.

To allow users to explore V-Mails in a more fine-grained manner, similar to video players we also provide a timeline slider (Fig. 2(d)). To emphasize the aspect of chapter transitions we constrain the timeline dragging to between two neighboring data views, so that it is easy for a user to focus on the visualization view while controlling the timeline with a mouse. This constraint is reset on each mouse down (for exploring a new transition) and can be removed with a modifier key.

### 4.2.4 Linking free exploration and storyboard timeline

Again different from traditional video players, the interaction in a data story is not restricted to just playing back a pre-created video stream. Instead, a user may exploring the 3D data visualization freely, potentially to capture slightly different shots or to better understand a feature in the data. After such a free exploration (and if no new shot was added to a timeline), the viewer may want to transition back to the timeline. We support both of these types of transitions by keeping the 3D visualization view and the timeline exploration linked and use animation transitions from the set described in Section 4.1.3. Each time a user switches from timeline to 3D view, we store the current position in the timeline to be able to transition back. In the opposite direction, we create a transition from a current state in the 3D viewer to the last active one. We also allow users to jump to stored data views, not only by interacting with the

<footnote n="2">2. The desktop client does not have these skip buttons because the skipping navigation can be done by clicking thumbnails (Fig. 2(c)). In the mobile client, however, we place the buttons in the bottom corners of the display to provide easier access (Fig. 9).</footnote>
<figure><img src="image_12.png" /><figcaption>Fig. 8. A story URL works as a lightweight token for sharing and opening the story. (a) our V-Mail clients show several stories with their URL; (b) after a story's upload, the client shows the URL; (c) opening the URL takes the user to a webpage where the story video can be played and the story can be opened in a dedicated client via a button (bottom).</figcaption></figure>

storyboard view, but also by clicking on the respective data view frames in the 3D view (Fig. 7).

## 4.3 Cross-platform Collaboration Mechanisms

To truly support our cross-device collaboration vision (C2) we cannot rely on all users having access to a high-end PC, which may be required for the interaction style described this far. We thus now describe how users respond to V-Mails on different devices and introduce the platform-specific prototypes (i. e., V-Mail clients) we have developed so far.

### 4.3.1 Flexible export of data story

As we want to support a wide ecology of devices, we have to ensure that they can show a V-Mail without advanced rendering hardware and even potentially without a dedicated client. As a most basic representation we thus rely on a traditional video that we generate from the data story, just as if one would have used a normal playback to watch it. Any device that supports video file playback (we use MP4 files) can thus show this most basic V-Mail format—including virtually all mobile phones, tablets, and laptops. Devices or players that also support video subtitles embedded in video files can also show the textual comments. Similarly, we also embed chapter marks for those video players that support it.

If a person receives V-Mails on a more regular basis, they can decide to install a dedicated tool that can enable them to further engage with V-Mail. To support additional features, we export a collection of files for each story in addition to the video file (Table 2 in Section 5). In this case we provide access to the dedicated play modes we described in Section 4.2.3, and for this purpose we also include a video file exported in the reverse direction (*video_backward.mp4*). To allow users to adjust or add to the graphical and textual comments, we save both types as separate files (*ann_i.png*, *msg.json*), that can be updated by the dedicated player after a change. In addition, we provide some simple '3D exploration' on low-power devices. As we cannot rely on rendering hardware, we use 360° images that we create during story export for each of the data views (*img360_i.png*). We also store all data views, camera locations, and visualization states in the *msg.json* file for users with sufficient high-end hardware and the full visualization software. Only with this data is it possible to adjust the data views, add new ones, rearrange their sequence, and re-export an updated story.

<figure><img src="image_13.png" /><figcaption>Fig. 9. We created the V-Mail mobile client (a) as a custom video player with visual components and mechanisms for interactively viewing and editing a story; clicking on the top-left share button in the viewer copies the story's URL and shows messaging apps to share the URL with (b); when clicking an e-mail app, we create a message with the name of the story as a title and the story's URL pasted into the main body (c).</figcaption></figure>

### 4.3.2 URL as a lightweight token for story sharing

While we export a collection of files for each story (Table 2), we do not want to bother users with having to manage and send these files. We thus manage the file collection on a dedicated server (that only has to provide file management and remote access) and instead share a story in form of a URL that points to the repository. An essential benefit of using a URL is that this allows users to not only link to a story in an e-mail but also to share a story on social media. We create this URL after each export (Fig. 8(b)). When opened in a web browser, the URL simply shows a video player (Fig. 8(c)). On devices with access to our dedicated clients the website, the URL provides a means to open the story in the dedicated software. Alternatively, the clients also provide a list of stories stored on the server (Fig. 8(a)). In both cases, the clients then access the files needed for their specific capabilities and provide the respective interaction tools, as we detail next.

### 4.3.3 A device ecology for collaborating on data stories

We implemented three specific clients as a proof of concept and to mirror the data levels we described in Section 4.3.1. In Fig. 1 we show photos of these solutions and in Table 1 we compare them by the features they support.

As just described, our most basic **web client** relies on the video player available on virtually all connected devices (Fig. 8(c)) to facilitate passive storytelling [41]. Depending on the functionality of this player, one may only perceive the story as a simple playback with graphical annotations for the data views. If the player supports it, however, one may also jump between chapters and/or see the textual comments that are available as video subtitles.

The web client can be accessed on virtually any connected device and uses the device's built-in video playback. With it we can thus reach completely new audiences with data stories. Even if someone never used a data story before and thus does not have or is not willing to install dedicated software, they can still view them. Depending on the media channel used to share the URL, they can also respond to a story such as by commenting in a separate e-mail. Moreover, this form of consuming also allows users to explore the story on their own first [24], before, potentially, adding to it and working collaboratively with other authors.

To explore the situation where users are willing to install a dedicated tool such as a **dedicated mobile app** on their phone or tablet, we developed an Android V-Mail client (Fig. 9) to facilitate a higher interaction fidelity for both phones and
<table><caption>TABLE 1
Availability and fidelity of operations that facilitate collaboration around shared stories and their prerequisites, by client/platform.</caption><thead><tr><th></th><th>web client: traditional video player</th><th>mobile client: custom video player</th><th>desktop client: plugin to vis. tool</th></tr></thead><tbody><tr><td></td><td colspan="3" style="text-align: center;">— interactive storytelling —</td></tr><tr><td>story navigation</td><td>playback and timeline controls without pausing points</td><td>playback and timeline controls with pausing points, storytelling affordance that distinguishes story nodes and transitions</td><td></td></tr><tr><td>exploration</td><td>no support (i.e., passive storytelling [41])</td><td>partially (360° views; i. e., storytelling with interactive approval [41])</td><td>full support (all four modes of author-and reader-driven storytelling [41])</td></tr><tr><td></td><td colspan="3" style="text-align: center;">— collaboration around shared stories —</td></tr><tr><td>view comments</td><td>partially (if player shows subtitles)</td><td>yes</td><td>yes</td></tr><tr><td>view annotations</td><td>yes</td><td>yes</td><td>yes</td></tr><tr><td>leave comments</td><td>no support</td><td>yes</td><td>yes</td></tr><tr><td>add annotations</td><td>no support</td><td>yes</td><td>yes</td></tr><tr><td>extend story</td><td>no support</td><td>no support</td><td>yes (add, remove, rearrange story)</td></tr><tr><td></td><td colspan="3" style="text-align: center;">— software and hardware prerequisites —</td></tr><tr><td>software needs</td><td>none</td><td>dedicated app</td><td>special visualization software</td></tr><tr><td>hardware needs</td><td>virtually none</td><td>virtually none</td><td>high-end GPU & CPU processing</td></tr></tbody></table>

tablets. On top of playing back the video of the data stories, this app also allows users to use our advanced playback modes, add new annotations (both graphical and textual), and have limited 3D exploration via the pre-computed 360° images for each data view. As it is likely used on small-screen devices, the app does not support changing the chapter order. And instead of using thumbnails for the data views in the storyboard, the app places crumbs on the timeline to provide visual and navigational support to users.

The mobile app facilitates a much richer form of collaboration by means of data stories than the web client, including being able to directly comment within the viewing environment and some 3D exploration. However, since this functionality relies only on the built-in video playback available in virtually all devices, the processing power required is essentially the same as the web client. Also, because it runs on mobile devices, it inherently allows people to engage with data stories like they do with other online media, for example, during a break, at home, or while traveling. In fact, the URL format makes it possible for people to share V-Mails on a variety of communication platforms, from e-mail to messaging apps, already installed on their phones.

For full-fidelity interaction, however, we created a desktop V-Mail plugin to a data visualization application [18] for both Windows and MacOS. This setup provides the functionality described in Section 4.2. It requires a high-performance computer, but it provides full flexibility in data exploration, story creation, and editing. At least one desktop client like this is needed to create an initial sharable data story, and this high-end client also makes sure that any updates from mobile app users get re-embedded into the video exports. This client also supports creating story branches with new sharable URLs.

# 5 SYSTEM AND IMPLEMENTATION NOTES

Our current V-Mail implementation comprises three clients (Fig. 10). We built the desktop plugin on top of an exploratory visualization software; it provides all the storytelling capabilities. When the user of the plugin uploads a story, files shown in Table 2 are generated and uploaded to the V-Mail server. One of these files is a video, and the mobile and web

<figure><img src="image_14.png" /><figcaption>Fig. 10. System architecture. V-Mail is composed of a server and three visualization clients. The desktop plugin is connected to an exploratory visualization software. The mobile and web clients support a portion of data storytelling activities around visual outputs stored in the server.</figcaption></figure>

clients work with this video file. The mobile client is a special video player for data videos with the dedicated play modes we described in Section 4.2.3 and annotation features. The web client relies on a video player within a browser.

## 5.1 V-Mail server and general operation

The V-Mail server comprises a database and a file storage. The purpose of integrating the database is to check available stories without opening files in the storage. It has a single table; each row represents a V-Mail story, and the columns are: an unique ID of the story, a name given by the user who created it, and last times modified by each client. The file storage contains additional files for V-Mail stories (Table 2). Files for each story are saved in separate folders, and names of the folders are the unique IDs listed in the database.

When a user opens the desktop or mobile client, the client accesses available stories from the database and shows these as a list (Fig. 8(a)). When the user clicks a button to open a specific story, files used by the current client (“Use by” column in Table 2) are downloaded and loaded. When the
<table><caption>TABLE 2
Files in a V-Mail story folder</caption><thead><tr><th>File</th><th>Use by</th><th>Description</th></tr></thead><tbody><tr><td>msg.json</td><td>Desktop, Mobile</td><td>Contains information about camera views, visualization states, and comments on data views.</td></tr><tr><td>video.mp4</td><td>Mobile, Web</td><td>Shows the animation as a video file.</td></tr><tr><td>video_backward.mp4</td><td>Mobile</td><td>Shows the animation in a reverse direction, and is used to provide a smooth backward play.</td></tr><tr><td colspan="3">— per data view —</td></tr><tr><td>img_i.png</td><td>Desktop</td><td>A snapshot of the current view, that is shown as a thumbnail for the data view</td></tr><tr><td>nextImg_i.png</td><td>Desktop</td><td>A snapshot captured from a camera location of the next data view, that is used to support the fourth type of animated transition</td></tr><tr><td>annImg_i.png</td><td>Desktop, Mobile</td><td>An image of drawings</td></tr><tr><td>img360_i.png</td><td>Mobile</td><td>A 360° image of the view, that is used to provide limited 3D exploration</td></tr><tr><td colspan="3">— for the web-client —</td></tr><tr><td>index.html</td><td>Web</td><td>The default page</td></tr><tr><td> subtitle.vtt</td><td>Web</td><td>Shows comments as a subtitle for the video.</td></tr><tr><td> playlist.m3u</td><td>Web</td><td>Provides data view portions as chapters in the video.</td></tr></tbody></table>

<figure><img src="image_15.png" /><figcaption>Fig. 11. Three components of viewing/editing a story in `StoryManager`</figcaption></figure>

user uploads a story from the desktop plugin, all the files are generated and uploaded to the server. The mobile client only modifies `msg.json` and `annImg_i.png` files.

When the user adds annotations on the mobile client, we need to re-create `video.mp4` and `video_backward.mp4` to reflect the changes. For this purpose we integrated another server-based application into the V-Mail infrastructure. This application shares much functionality with the desktop client and checks the database every minute for the changes. If it finds any, it recreates the video files and uploads these to the server. For this check we compare the last modified time by a mobile client with that of a desktop in the database.

The mobile client plays `video.mp4` or `video_backward.mp4` depending on the current play direction. When the user clicks a backward playback button (Fig. 6) while watching a video in a forward direction, the client skips time in `video_backward.mp4` to match that in `video.mp4` and plays `video_backward.mp4`. The same happens for `video.mp4` when the user clicks a forward playback button.

We built the desktop and mobile clients with Unity3D, and the web client is a web page with a HTML video element. We realized the database with MySQL, and wrote the server-side scripts for accessing the database and downloading/uploading files in PHP. We used an FFmpeg executable file for creating a video file from a series of images captured from an exploratory visualization application [18].

## 5.2 Desktop and mobile clients details

We built the desktop and mobile clients using C# in Unity3D. We handle storytelling and collaborative activities in V-Mail with a `StoryManager` (Fig. 11). In addition to providing features for loading and uploading a story to the server, the `StoryManager` has three components for viewing and editing a story. A `Message` manages content in the current story and has an ordered list of `Page(s)`. Each `Page` contains view and visualization state information captured from an underlying visualization application. It also contains comments and sketches drawn by users. The `MessagePlayer` updates the story time depending on the current playback option (Fig. 6). It also provides methods for accessing the current `Page` and `Transition` in a story. A `Transition` is composed of two `Pages` and a transitioning amount. `Viewers` are visual components that show the current `Message` and provide a means for editing it. `CommentViewer` updates the comment panel (Fig. 2), and `AnnotationViewer` shows drawings on the render view. The `VisualizationViewer` updates an underlying visualization. If the current time is on a page, the viewer updates the visualization from the current `Page` accessed from the `MessagePlayer`. If on a transition portion, the viewer interpolates between two `Pages` from the `Transition`. As the mobile client plays video files, the `StoryManager` in the client is different from that of the desktop plugin. The `VisualizationViewer` is removed as the client does not rely on an underlying visualization, and we add a `SphericalImageViewer` to provide limited 3D exploration. The `MessagePlayer` updates the timeline slider based on the current time in the video, and skips time upon clicks on the corresponding buttons or a specific point on the slider.

# 6 APPLICATION CASE AND EVALUATION

To evaluate the effectiveness of our concept for team collaboration, we recruited members of a leading, interdisciplinary **climate science research team** to integrate V-Mail into their normal workflow over a two-week period. This section reports on training, observations, results, user feedback, and other lessons learned from this real-world application. We also showed the tool to members of a **neuroscience research team**, and report lessons and user feedback from their initial encounter with our tool in Section 6.3. Unlike the climate science team, they decided not to be part of a longitudinal
study. In the section, we also report limitations and areas
could be addressed in future work.

## 6.1 Methodology

The interdisciplinary climate science research team uses a
combination of tools, including some of the world's most
advanced high-performance computing (i.e., supercomputer)
simulations, to study complex interactions of Antarctic water
masses that may explain the global rise in sea temperature.
The team includes oceanographers, geologists, computer
scientists, a visual artist, and a science writer. Like many
research teams today, they place an increasing emphasis
on improved communication with the public. The artist and
writer have played leading roles in these efforts, and the team
has produced five short videos describing the science in the
past seven years, all of which were national competition
finalists (two won awards as best submissions). One of their
recent videos was created with the Unity-based Artifact-
Based Rendering (ABR) engine [18]—the same engine used
for our first implementation of a V-Mail plugin.

Since one of our goals was to understand how V-Mail
might integrate with whatever data communication work
is at hand in a real, research-active science team, we asked
the team to use V-Mail for whatever tasks they thought
made sense during the two-week trial we had agreed upon.
The team decided that the most interesting communication
task at hand was to develop one or more complementary
videos to go deeper into certain aspects of the science and
provide useful material for the scientists to use in their
presentations. Thus, exploring how V-Mail might be utilized
to support this work became the primary task of the V-
Mail trial, and the two primary users became the visual
artist, a professional sculptor and painter for more than 30
years whose collaborations with scientists led her to data
visualization, and the science writer, a Ph.D. student in
cultural geography. During the first few days of the two-
week period, we met individually with the artist and science
writer to provide training for the V-Mail clients they chose to
use and to assist them in their tasks. We recorded qualitative
feedback by taking notes from interviews we conducted after
the training sessions as well as via questionnaires before and
after the two-week period. In addition, we saved each V-Mail
that they created and configured the V-Mail clients to log
all edits. After the two weeks, the artist shared the resulting
V-Mail in a regularly scheduled, "live" video-conference
with the scientists, gathering feedback on whether the story
covers important data features and whether it employs
appropriate storytelling techniques.

## 6.2 Observations and Results

We organize this section by the type of data we collected.

### 6.2.1 Pre-study Questionnaire

In the questionnaire we asked for the teams' pre-existing
communications and related needs, and the two primary
participants filled out the questionnaire. Their responses
showed that they generally take different approaches for data
communication. For urgent matters, participants typically
communicate in-person or send e-mail-like messages. When
answering a question that could be answered in few words,

participants use an e-mail-like tool. However, when it takes
multiple steps to explain a concept, they typically use syn-
chronous communication, such as a live video conferencing
tool. The choice was the same for describing spatial features
in their data and for public-facing communication. When
asked about their use of visuals in e-mails, participants
responded differently. One does not use e-mails for sharing
3D visualization results. The other said that about 60% of her
e-mails contains images or videos, and 5% of these contain
annotations. With regards to device usage, both participants
typically have some access (2–8 hours) to high-end or
specialized devices. However, they typically use personal
PCs or mobile devices for science-related communications.
One participant indicated that her responses to questions
may vary depending on goals, as she often uses a high-end
PC when working on a video or visualization project.

### 6.2.2 Produced V-Mails

In total, the final V-Mail contains ten data views (Fig. 12). The first three (#1–#3) provide an overview of data visualization, highlighting all visible variables. The last shot (#10) is another overview shot. Data views in the middle (#4–#9) describe two different concepts about interactions of water masses and currents on the ice shelf. Each concept was communicated with three shots. The first (#4, #7) gives a spatial context, the second (#5, #8) zooms in further to focus on visual decisions, and the last (#6, #9) zooms in further to relate the visual decisions to specific data features. Each shot is discussed in detail with textual comments contributed by both artists.

Interested in sharing their work to collaborating scien-
tists and possibly using it as video footage for their next
video projects, the artist requested another version of the
V-Mail that shows the fly-through animation at a higher
resolution (1920 × 1080) and without the floating 3D pages.
Treating it like a final draft of their work, the “final” V-Mail
shows only the voice-over script, without all the low-level
communications such as questions in a form of comments.

### 6.2.3 Software Logs

Our log data and users' comments made in the story indicate
that there were five back-and-forth communications over the
two weeks (Table 3). At each turn, after making edits, the
participants used their normal e-mail to pass the V-Mail link
to each other and request comments. Both URL and name
were used to reference updated V-Mails.

The textual comments were added to data views and
were used mainly to provide guidelines and script a voice-
over. Questions were also asked in a form of comments. For
example, confused with the scripting guidelines, the science
writer left questions on corresponding data views. We also
found that the comments contained imperfections, like typos
and this accidental use of caps lock, "hERE WE CAN SPEAK
about." Capitalization was also used for emphasis, as in
"IGNORE the above." The artist and science writer did not
use the sketching feature to annotate data views.

The visual artist used the desktop client and the science
writer used the mobile client. While working on the script, the
science writer realized that a few data views were not needed.
While working on the mobile client, she left comments on
these extra data views to request a removal. Because the artist
using the desktop client forgot how to remove data views and
<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /><figcaption>Fig. 12. Visualizations of data views' placements in the V-Mail participants created. Each circle represents a data view with a number indicating the view's order and with color indicating its thematic section.</figcaption></figure>

<table><caption>TABLE 3
Five back-and-forth communications between participants.</caption><thead><tr><th>#</th><th>User</th><th>Client</th><th>Goals</th><th># of data views</th><th># of edited data views</th><th># of views w/ new comments</th><th># of views w/ new sketches</th></tr></thead><tbody><tr><td>1</td><td>artist</td><td>desktop</td><td>capture data views, provide scripting guidelines</td><td>15</td><td>15 added</td><td>13</td><td>0</td></tr><tr><td>2</td><td>writer</td><td>mobile</td><td>draft a voice-over script, ask questions</td><td>15</td><td>none</td><td>9</td><td>0</td></tr><tr><td>3</td><td>artist</td><td>desktop</td><td>answer questions, make requests</td><td>15</td><td>none</td><td>3</td><td>0</td></tr><tr><td>4</td><td>writer</td><td>mobile</td><td>make requests</td><td>15</td><td>none</td><td>5</td><td>0</td></tr><tr><td>5</td><td>author</td><td>desktop</td><td>address the requests</td><td>10</td><td>5 removed,<br>1 rearranged</td><td>0</td><td>0</td></tr></tbody></table>

we (the authors) were also monitoring the communication, we were able to quickly help out and resolve the request in the desktop client.

### 6.2.4 Observations during introductory sessions

At the end of each training session, we asked users to now begin to use the tool while we observed their actions and answered any questions that came up.

In the first V-Mail, **the artist** captured data views with scripting guidelines written as comments (Table 3). The installation steps and training time for this stage took about 20 minutes for the desktop-client. In the next hour session, she created three V-Mail drafts from scratch (from an empty storyboard) and did not report any major difficulties. She discarded two of them, and proceeded to refine to third by adding a few more data views, rearranging their placement, and adding notes and guidelines to send to her colleague. The artist created V-Mails with multiple sections (i. e., data views) to cover different data features (Fig. 12) and used **different approaches to extend and strengthen a story line**. We noticed that the most recently added data view was often used as a spatial reference frame for the next data view. For example, after adding view #6, view #7 was described using the language, "now looking down and reveal the currents." The participant also added data views "out of sequence" and rearranged previously added views to strengthen the story.

In the second V-Mail, **the science writer** opened the message from the artist, then proceeded to work on drafting a voice-over script for each data view using the guidelines contained in the V-Mail (Table 3). The science writer used the mobile-client because it provided all features needed for her tasks. The installation and training took her about 5 minutes. She flipped through pre-existing data views and added the script as comments, finishing this task in next 30 minutes. She did not report any major difficulties while using the tool.

### 6.2.5 Feedback from User Interviews

After they created their first V-Mails, we interviewed the participants to learn about their views of both tool and concept.

The artist described gaps in their current tools and **relevant cases where V-Mail could be most useful**. The artist described her typical approach of creating a fly-through animation as, first, establishing the concepts to convey and, second, capturing the shots needed for the concepts. She described the process as "melding story into visuals." For all but simple topics, in-person communication or video conferencing are preferred for describing such visual and spatial concepts, but in practice, they often resort to sending several fly-through animations to the scientists to confirm the concepts. In their traditional workflow, this process takes a long time as they often require technical assistance from additional team members to create their animations. V-Mails, in contrast, enable artists, science writers, or other team members to control this process on their own. They can focus on identifying the right views to convey the concepts, and the animation is automatically created. The ability to quickly mock-up animations from data views expedites the creation process, and the ability to view the results on mobile devices and web browsers simplifies the distribution.

Imagining how their experiences might translate to a **future trial with the larger group**, the artist pointed out that the mobile client will provide a means for the scientists to provide feedback without the need to go through complex installation steps and a long training. She also pointed out the importance of the web-client as it enables them to reach people outside of the group, e. g., the public and scientists from other groups. The artist did not use **the sketching feature to annotate data views**, even though, in several data views, she described features at specific locations of the views (e. g., eddies formed in a current shown in the bottom). When asked about it during the interview, the artist explained that she thought this would be useful in some situations, but in this case she knew the science writer was
already familiar with data features and visual mappings
from their close recent work together.

The participants made several **feature requests**. She re-
quested to implement additional V-Mail clients for ParaView
[2], a visualization tool the team frequently uses. The science
writer suggested supporting audio recording to annotate
existing data views as it is often easier than writing thoughts
down. The writer told us that opening a V-Mail takes a long
time on her device. She suggested another data presentation
mode that shows a V-Mail without the animated transitions,
as this can reduce the download time and the animated
transitions might not be needed for some users who are
already familiar with data presented in the shared views.
Also, in our current implementation, the user first needs to
add a data view before annotating it. She suggested another
approach where a data view is automatically added when
the user starts drawing on the current camera view.

6.2.6 Feedback from scientists

After the two weeks, the artist shared the resulting V-Mail in a regularly scheduled, “live” video-conference with the larger group, and led a discussion about the work using the web-client. The scientists were most interested in seeing details of water-mass distributions, current trails, and their impacts on melting glaciers. The created V-Mail narrative covers these regions of interests and employs proper storytelling techniques—e. g., a series of first overview, then zoomed-in data views [34] connected with transitioning animations—to show not only details of these regions but also their contextual information. They were also interested in seeing changes of specific regions over time. Unfortunately, time-series data is not yet supported in the underlying visualization software. The scientists requested this as a future work and brainstormed ideas for storytelling techniques to convey these in future iterations. They envisioned to station the camera to focus on a specific region and then show changes over time. They would do this for multiple data regions and employ V-Mail’s interactive storytelling techniques to return to previous camera locations to facilitate comparative analysis of the sites.

6.2.7 Post-study Questionnaire

One user from the climate science team participated in the
questionnaire. Her responses reaffirm what she described
about our tool in the interview session (Section 6.2.5), giving
positive responses on features that enable her to make movies
or tell data stories. However, she noted several additional
points. The participant noted that connecting views with
animations is a crucial component in data communication.
The ability to pause in-between and play backwards as well
as the ability to see the 3D frames, however, received neutral
responses. Nonetheless, she was concerned about using small
mobile displays as they may not be able to convey the visual
detail of the visualizations.

6.3 Initial Feedback from a Brain Science Team

We integrated V-Mail into a different system that visualizes
crossing pathways in white matter areas in a human brain
(see Appendix B). We showed the V-Mail during a live video
conference to members of a neuroscience research team,

who was already familiar with the visualization system.
Their initial reaction was somewhat mixed; they told us
the tool itself is novel and could be useful in presenting
findings to a broader audience, but there is no urgent need
for such tool for their team-science collaboration. The team
already meets regularly, almost weekly, using a live video
conference or in a meeting room, and the team is led by
one senior member to whom his students report. They use
remote asynchronous communication tools such as e-mails
to coordinate the meetings or share datasets.

Participants described situations where cross-platform
collaboration tools like V-Mail may not work and could be
useful. In clinics, radiologists use specialized monitors for
viewing lossless DICOM images to see subtle differences in
tissue contrasts. Also a participant pointed out that there is a
limit to what information they can share with the public as
certain information is sensitive and private. In cases where
sharing information does not have such constraints, V-Mails
could be used to enhance the credibility of presented results.
They envision that stories created in V-Mail are submitted
as supplementary material for research papers. Linked from
figures or text, V-Mails could allow reviewers or readers to
validate the authors' claims and further explore the data.

6.4 Discussion, Limitations, and Future Work

The success of adding a cross-device, asynchronous communication tool, like V-Mail, into the existing science team's workflow depends greatly on how the team already works. The brain science team decided not to partake in a two-week trial of V-Mail, as this team included just one senior member and several grad students who already meet regularly to discuss their work. They also raised concerns about security with respect to patient data that should not be widely shared (Section 6.3). The climate science team also expressed a preference for synchronous communication. However, schedules are more complex within this larger team, so the ability to communicate asynchronously is more valuable.

In addition to using the comment feature for drafting a
voice-over script, participants used it to ask questions and
delegate tasks, and these comments often contain typos and
accidental use of caps lock. While these low-level communi-
cations were removed from the "final" V-Mail, they can help
people to clarify tasks and enable other team members to
provide immediate help. All of these reinforce the chat-like
immediacy and draft-in-progress qualities of V-Mail. These
qualities are unusual in science communication, especially
high-end data visualization, but we believe they may, in fact,
help to put a human face on science communication, helping
scientists and stakeholders more freely discuss scientific data.

The science writer pointed out that communicating data
in a form of stories might introduce unnecessary overhead for
audiences who are already familiar with data (Section 6.2.5).
For these users, the ability to disable certain storytelling
features and share only portions of a data story could be
useful. By adapting the feature that encodes all viewing
properties in a shared link [30], we could associate an URL
parameter to each data view in a story, so that a specific
view could be referenced in e-mails or comments. The
feature would also be required if V-Mail is used with other
storytelling media, e. g., for providing interactive figures in a
paper submission as discussed by the brain science team. In the same spirit, other storytelling paradigms that rely less on animations could also be integrated in V-Mail. For instance, scrolly-telling [35] can be used to rapidly control the speed and direction of viewing. We could also spread out data views in a layout, e. g., as done in data comics [5], which could be more useful for audiences who are already familiar with spatial locations of data features.

A major limitation of the current V-Mail implementation is that it can only be integrated into applications built using Unity3D, and a separate instance of the V-Mail plug-in is needed for each system. To ease the integration process, we are considering a more loosely coupled implementation where a separate storyboard application communicates over a network with visualization software. Since the webpage and the mobile app do not rely on other data visualization applications (these clients play a video), we can continue using these clients for showing and interacting with stories created from new connected software.

We observed that participants used lightweight clients if the features required for their tasks were available in those clients. We wonder, if the clients supported some lightweight data exploration features, participants might have explored data even though their tasks did not require this. Indeed, the brain science team viewed facilitating interactive exploration on the web-client as an important direction for future work. Thus, upgrading the web-client with remote rendering [27] or new web technologies [39] could be a useful next direction. In addition, as mentioned by the artist and the brain scientists, seeing visualization on small screens, like a phone, is just limiting. Further studies are needed to better characterize exactly how much value the capability provides to science teams. In addition, further studies with more and other domain experts should be carried out to confirm our findings from our qualitative evaluation.

## 7 CONCLUSION

On a conceptual level, our work extends the state of art on storytelling in visualization and provides a rich tool set for asynchronous remote collaboration. We build upon existing spatial 3D data storytelling techniques that tightly couple storytelling with data exploration and employ semi-automatic procedures to expedite the story creation process. What is unique in V-Mail is the ability to collectively work on data stories, which is often seen with abstract data visualization, e. g., via web-based visualizations, but less so with visualization of spatial 3D data. Essentially this multi-author editing in V-Mail is a type of parallel storytelling, with the difference that users not only view or edit shared data stories but also do this across different devices. V-Mail is thus an example of using multiple display devices in collaborative data visualization [16], [17]. V-Mail focuses, however, on asynchronous remote communication and provides lightweight mobile and web clients for partaking in collaboration and simplifies the distribution process using URLs to reference data stories.

The desktop-client is an example of asynchronous collaboration frameworks that support collaborative activities around exploratory visualizations. The mobile-client, however, supports these activities around video-based visualiza-

tions created from the desktop-client. The ability to provide visual feedback directly onto shared visualization results is similar to the ability to annotate on shared data views and stories in the web-based visualization. However, there are two major differences in our approach: (1) communication mechanisms are tailored for spatial 3D data visualization, e. g., with spatial annotations and animated transitions, and (2) the app is designed to solely work with shared visualization results. The latter point may be of interest to visualization practitioners as this approach enables them to bypass installation steps and computation requirements often imposed by using exploratory visualization software such as ParaView [2]. Also this separation suggests an interesting future direction of turning V-Mail into a collaborative framework that unifies well-known software in the field for creating data stories and conducting discussion.

In our first V-Mail deployment and evaluation, we took the approach of asking a real science team to integrate V-Mail in whatever ways they found most useful for the tasks at hand during a two-week period. The results were generally positive, re-confirming the urgent need for improved tools for spatial communication in this context, while also suggesting areas of required improvements to advance V-Mail from the prototype stage to a production-level tool. In this first application, only the artist and science writer actively used the tool to draft a video-based visualization, and the scientists were involved at the last stage to provide feedback on the created visualization. The brain science team provided initial feedback on the tool but did not partake in a longitude study. Notably, if we are able to secure funding to support additional work on V-Mail, we plan follow-on studies to understand how V-Mail might be used and/or tailored to best support specific scientist-to-scientist, public-to-science-team, and lay-person-to-specialist communications.

## ACKNOWLEDGMENTS

The authors thank Seth Johnson and Bridger Herman for assistance with integrating the ABR engine. This research was supported in part by the National Science Foundation (IIS-1704604, IIS-1704904).

## IMAGES/FIGURES LICENSE AND COPYRIGHT

We as authors state that all of our figures in this article are and remain under our own personal copyright, with the permission to be used here. We also make them available under the Creative Commons Attribution 4.0 International (CC BY 4.0) license and share them at osf.io/qehvs.

## REFERENCES

[1] D. Acevedo, E. Vote, D. H. Laidlaw, and M. S. Joukowsky, "Archaeological data visualization in VR: Analysis of lamp finds at the Great Temple of Petra, a case study," in Proc. Visualization. Los Alamitos: IEEE CS, 2001, pp. 493–496. doi: 10.1109/VISUAL.2001.964560

[2] J. Ahrens, B. Geveci, and C. Law, "Paraview: An end-user tool for large data visualization," in The Visualization Handbook, C. D. Hansen and C. R. Johnson, Eds. Oxford: Elsevier, 2005, ch. 36, pp. 717–731. doi: 10.1016/B978-012387582-2/50038-1

[3] H. Akiba, C. Wang, and K.-L. Ma, "AniViz: A template-based animation tool for volume visualization," IEEE Comput Graph Appl, vol. 30, no. 5, pp. 61–71, 2010. doi: 10.1109/MCG.2009.107
[4] B. Bach, N. Kerracher, K. W. Hall, S. Carpendale, J. Kennedy, and N. Henry Riche, "Telling stories about dynamic networks with graph comics," in *Proc. CHI*. New York: ACM, 2016, pp. 3670-3682. doi: 10.1145/2858036.2858387

[5] B. Bach, Z. Wang, M. Farinella, D. Murray-Rust, and N. Henry Riche, "Design patterns for data comics," in *Proc. CHI*. New York: ACM, 2018, pp. 38:1-38:12. doi: 10.1145/3173574.3173612

[6] K. Bladin, E. Axelsson, E. Broberg, C. Emmart, P. Ljung, A. Bock, and A. Ynnerman, "Globe browsing: Contextualized spatio-temporal planetary surface visualization," *IEEE Trans Vis Comput Graph*, vol. 24, no. 1, pp. 802-811, 2018. doi: 10.1109/TVCG.2017.2743958

[7] P. de Lange, P. Nicolaescu, J. Benscheid, and R. Klamma, "Collaborative non-linear storytelling around 3D objects," in *Proc. ICWL*. Cham: Springer, 2018, pp. 88-98. doi: 10.1007/978-3-319-96565-9_9

[8] N. Duarte, *Data Story: Explain Data and Inspire Action Through Story*. Ideapress Publishing, 2019.

[9] R. Eccles, T. Kapler, R. Harper, and W. Wright, "Stories in geotime," *Inf Vis*, vol. 7, no. 1, pp. 3-17, 2008. doi: 10.1057/palgrave.ivs.9500173

[10] H. Farooq, J. Xu, J. W. Nam, D. F. Keefe, E. Yacoub, T. Georgiou, and C. Lenglet, "Microstructure imaging of crossing (MIX) white matter fibers from diffusion MRI," *Sci Rep*, vol. 6, pp. 38 927:1-38 927:9, 2016. doi: 10.1038/srep38927

[11] N. Gershon and W. Page, "What storytelling can do for information visualization," *Commun ACM*, vol. 44, no. 8, pp. 31-37, 2001. doi: 10.1145/381641.381653

[12] J. Heer and M. Agrawala, "Design considerations for collaborative visual analytics," *Inf Vis*, vol. 7, no. 1, pp. 49-62, 2008. doi: 10.1057/palgrave.ivs.9500167

[13] J. Heer and G. Robertson, "Animated transitions in statistical data graphics," *IEEE Trans Vis Comput Graph*, vol. 13, no. 6, pp. 1240-1247, 2007. doi: 10.1109/TVCG.2007.70539

[14] J. Heer, F. B. Viégas, and M. Wattenberg, "Voyagers and voyeurs: supporting asynchronous collaborative information visualization," in *Proc. CHI*. New York: ACM, 2007, pp. 1029-1038. doi: 10.1145/1240624.1240781

[15] S. Iserhardt-Bauer, P. Hastreiter, T. Ertl, K. Eberhardt, and B. Tomandl, "Case study: Medical Web service for the automatic 3D documentation for neuroradiological diagnosis," in *Proc. Visualization*. Los Alamitos: IEEE CS, 2001, pp. 425-581. doi: 10.1109/VISUAL.2001.964542

[16] D. H. Jeong, S.-Y. Ji, E. A. Suma, B. Yu, and R. Chang, "Designing a collaborative visual analytics system to support users' continuous analytical processes," *Hum-Centric Comput Inf Sci*, vol. 5, no. 1, pp. 5:1-5:20, 2015. doi: 10.1186/s13673-015-0023-4

[17] D. H. Jeong, E. Suma, T. Butkiewicz, W. Ribarsky, and R. Chang, "A continuous analysis process between desktop and collaborative visual analytics environments," in *Proc. VAST*. Los Alamitos: IEEE CS, 2010, pp. 231-232. doi: 10.1109/VAST.2010.562598

[18] S. Johnson, F. Samsel, G. Abram, D. Olson, A. J. Solis, B. Herman, P. J. Wolfram, C. Lenglet, and D. F. Keefe, "Artifact-based rendering: Harnessing natural and traditional visual media for more expressive and engaging 3D visualizations," *IEEE Trans Vis Comput Graph*, vol. 26, no. 1, pp. 492-502, 2019. doi: 10.1109/TVCG.2019.2934260

[19] T. Klein, L. Autin, B. Kozlíková, D. S. Goodsell, A. Olson, M. E. Gröller, and I. Viola, "Instant construction and visualization of crowded biological environments," *IEEE Trans Vis Comput Graph*, vol. 24, no. 1, pp. 862-872, 2017. doi: 10.1109/TVCG.2017.2744258

[20] C. N. Knaflic, *Storytelling with Data: A Data Visualization Guide for business Professionals*. Hoboken: Wiley, 2015.

[21] R. Kosara and J. Mackinlay, "Storytelling: The next step for visualization," *Comput.*, vol. 46, no. 5, pp. 44-50, 2013. doi: 10.1109/MC.2013.36

[22] D. Kouřil, O. Strnad, P. Mindek, S. Halladjian, T. Isenberg, M. E. Gröller, and I. Viola, "Molecular documentary: Adaptable narrated documents using molecular visualization," *IEEE Trans Vis Comput Graph*, 2023, to appear. doi: 10.1109/TVCG.2021.3130670

[23] B. Lee, R. H. Kazi, and G. Smith, "SketchStory: Telling more engaging stories with data through freeform sketching," *IEEE Trans Vis Comput Graph*, vol. 19, no. 12, pp. 2416-2425, 2013. doi: 10.1109/TVCG.2013.191

[24] B. Lee, N. H. Riche, P. Isenberg, and S. Carpendale, "More than telling a story: Transforming data into visually shared stories," *IEEE Comput Graph Appl.*, vol. 35, no. 5, pp. 84-90, 2015. doi: 10.1109/MCG.2015.99

[25] I. Liao, W.-H. Hsu, and K.-L. Ma, "Storytelling via navigation: A novel approach to animation for scientific visualization," in *Proc.*

*Smart Graphics*. Cham: Springer, 2014, pp. 1-14. doi: 10.1007/978-3-319-11650-1_1

[26] E. M. Lidal, M. Natali, D. Patel, H. Hauser, and I. Viola, "Geological storytelling," *Comput Graph*, vol. 37, no. 5, pp. 445-459, 2013. doi: 10.1016/j.cag.2013.01.010

[27] E. J. Luke and C. D. Hansen, "Semotus Visum: A flexible remote visualization framework," in *Proc. Visualization*. Los Alamitos: IEEE CS, 2002, pp. 61-68. doi: 10.1109/VISUAL.2002.1183758

[28] P. Lundblad and M. Jern, "Geovisual analytics and storytelling using HTML5," in *Proc. IV*. Los Alamitos: IEEE CS, 2013, pp. 263-271. doi: 10.1109/IV.2013.35

[29] K.-L. Ma, I. Liao, J. Frazier, H. Hauser, and H.-N. Kostis, "Scientific storytelling using visualization," *IEEE Comput Graph Appl*, vol. 32, no. 1, pp. 12-19, 2012. doi: 10.1109/MCG.2012.24

[30] J. Maitin-Shepard, A. Baden, W. Silversmith, E. Perlman, F. Collman, T. Blakely, J. Funke, C. Jordan, B. Falk, N. Kemnitz, tingzhao, C. Roat, M. Castro, S. Jagannathan, moenigin, J. Clements, A. Hoag, B. Katz, D. Parsons, J. Wu, L. Kamentsky, P. Chervakov, P. Hubbard, S. Berg, J. Hoffer, A. Halageri, C. Machacek, K. Mader, L. Roeder, and P. H. Li, "google/neuroglancer;" Software: WebGL-based viewer for volumetric data, 2021. doi: 10.5281/zenodo.5573294

[31] M. Meuschke, L. Garrison, N. Smit, S. Bruckner, K. Lawonn, and B. Preim, "Towards narrative medical visualization," arXiv preprint 2108.05462, 2021.

[32] N. H. Riche, C. Hurter, N. Diakopoulos, and S. Carpendale, *Data-Driven Storytelling*. Boca Raton: CRC Press, 2018. doi: 10.1201/9781315281575

[33] E. Segel and J. Heer, "Narrative visualization: Telling stories with data," *IEEE Trans Vis Comput Graph*, vol. 16, no. 6, pp. 1139-1148, 2010. doi: 10.1109/TVCG.2010.179

[34] B. Shneiderman, "The eyes have it: A task by data type taxonomy for information visualizations," in *Proc. VL*. Los Alamitos: IEEE CS, 1996, pp. 336-343. doi: 10.1109/VL.1996.545307

[35] C. D. Stolper, B. Lee, N. H. Riche, and J. Stasko, "Emerging and recurring data-driven storytelling techniques: Analysis of a curated collection of recent stories," Microsoft Research, Washington, Tech. Rep. MSR-TR-2016-14, 2016.

[36] M. Thöny, R. Schnürer, R. Sieber, L. Hurni, and R. Pajarola, "Storytelling in interactive 3D geographic visualization systems," *ISPRS Int J Geo-Inf*, vol. 7, no. 3, pp. 123:1-123:14, 2018. doi: 10.3390/ijgi7030123

[37] C. Tong, R. Roberts, R. Borgo, S. Walton, R. Laramee, K. Wegba, A. Lu, Y. Wang, H. Qu, Q. Luo et al., "Storytelling and visualization: An extended survey," *Inf*, vol. 9, no. 3, pp. 65:1-65:42, 2018. doi: 10.3390/info09030065

[38] M. Tsang, G. W. Fitzmaurice, G. Kurtenbach, A. Khan, and B. Buxton, "Boom Chameleon: Simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display," in *Proc. UIST*. New York: ACM, 2002, pp. 111-120. doi: 10.1145/571985.572001

[39] W. Usher and V. Pascucci, "Interactive visualization of terascale data in the browser: Fact or fiction?" in *Proc. LDAV*. Los Alamitos: IEEE CS, 2020, pp. 27-36.

[40] F. B. Viegas, M. Wattenberg, F. Van Ham, J. Kriss, and M. McKeon, "ManyEyes: A site for visualization at internet scale," *IEEE Trans Vis Comput Graph*, vol. 13, no. 6, pp. 1121-1128, 2007. doi: 10.1109/TVCG.2007.70577

[41] M. Wohlfart and H. Hauser, "Story telling for presentation in volume visualization," in *Proc. EuroVis*. Goslar: Eurographics Assoc, 2007, pp. 91-98. doi: 10.2312/VisSym/EuroVis07/091-098

[42] K. Wongsuphasawat, D. Smilkov, J. Wexler, J. Wilson, D. Mane, D. Fritz, D. Krishnan, F. B. Viégas, and M. Wattenberg, "Visualizing dataflow graphs of deep learning models in tensorflow," *IEEE Trans Vis Comput Graph*, vol. 24, no. 1, pp. 1-12, 2017. doi: 10.1109/TVCG.2017.2744878

[43] A. Ynnerman, J. Löwgren, and L. Tibell, "Exploranation: A new science communication paradigm," *IEEE Comput Graph Appl*, vol. 38, no. 3, pp. 13-20, 2018. doi: 10.1109/MCG.2018.032421649
<figure><img src="image_19.png" /></figure>

**Jung Who Nam** is a doctoral student at the University of Minnesota, Twin Cities. His research interests include scientific visualization, immersive analytics, and data storytelling. He is particularly interested in novel visualization and interaction methods that support exploration and collaboration around spatial 3D data visualization.

<figure><img src="image_20.png" /></figure>

**Tobias Isenberg** is a senior research scientist at Inria, France. Previously he held positions as post-doctoral fellow at the University of Calgary, Canada, and as assistant professor at the University of Groningen, the Netherlands. His research interests include scientific visualization, illustrative and non-photorealistic rendering, and interactive visualization techniques. He is particularly interested in interactive visualization environments for 3D spatial data.

<figure><img src="image_21.png" /></figure>

**Daniel F. Keefe** is a Distinguished University Teaching Professor and Professor of Computer Science and Engineering at the University of Minnesota. His research centers on scientific data visualization and interactive computer graphics. In addition to his work in computer science, Keefe is also an accomplished artist and has published and exhibited work in top international venues for digital art. Before joining the University of Minnesota, Keefe did post-doctoral work at Brown University jointly with the departments of Computer Science and Ecology and Evolutionary Biology and with the Rhode Island School of Design. He received the Ph.D. in 2007 from Brown University's Department of Computer Science and the B.S. in Computer Engineering summa cum laude from Tufts University in 1999.
# V-Mail: 3D-Enabled Correspondence about Spatial Data on (Almost) All Your Devices

## APPENDIX

In this appendix we provide additional content that we could include in the main paper due to space limitations.

<figure><img src="image_22.png" /><figcaption>Fig. 13. We integrated the V-Mail desktop plugin into a system that visualizes fiber orientations in brain white matter [10].</figcaption></figure>

### A. DETAILS FOR THE MAIN SUPPLEMENTARY VIDEO

In the main complementary video (00:53–4:23), we demonstrate our tool with a use-case based on the Antarctic ice sheet dataset. Here, we describe this use-case and provide references to sections in this paper. In **Act #1**, which starts at 0:58, a student receives a V-Mail URL token in a message (Section 4.3.2). The link takes him to our web client, which shows the story as a video file in a web page (web client in Section 4.3.3). The student clicks a button on the web page to open the story in the our custom desktop client (Fig. 8(c)). In **Act #2**, which starts at 1:23, the student extends the V-Mail to ask a question to his supervisor. He captures additional data views and adds annotations (Section 4.2.1). Animated transitions are automatically constructed by the system (Section 4.1.3). To review the story, the student uses different playback modes (Section 4.2.3). He uploads the changes (Section 4.2.2) and sends the URL token to his supervisor, which was created by the system after it finished the uploading process. The changes are uploaded to V-Mail server so that others can view these using one of our clients (Section 5). In **Act #3**, which starts at 3:14, the supervisor opens the V-Mail URL token sent by his student. Since he is using a phone, clicking the Open button opens the story in the mobile client (mobile client in Section 4.3.3). After done making the edits, he shares the updated story by sending the URL of the V-Mail Section 4.3.2. After noticing a change made by the mobile client, the system updates the video file in the server so other users can see the latest results (Section 5).

### B. ADDITIONAL USE-CASE

The V-Mail concept can easily be implemented for other exploratory visualizations, and in this section we describe another use-case of V-Mail and the corresponding integration process. Integrated into the system that visualizes fiber orientations in brain white matter [10], V-Mail enables users

to create data stories that highlight fiber configurations in different regions of a human brain (Fig. 13).

The first author created a V-Mail based on explanations of observed data patterns (Zeppelin-Cylinder-Cylinder-Dot (ZCCD) section in [10]). The story starts by showing a coronal slice and zooms into the upper-part and shows data glyphs. In the next two data views, the author discusses about fiber configurations in corpus callosum (CC) and centrum semi-ovale (CSO) regions; the camera zooms into the corresponding region, and implication of glyph alignments are written as comments. In the last data view, the camera zooms out to provide a closing shot. We share the video of this V-Mail also in the supplementary materials.

The underlying brain visualization application was also built using Unity3D. We thus imported the desktop plugin used for the climate data visualization into the brain visualization and adjusted the necessary parts. To control the current view, the *StoryManager* references a camera object in the scene. We implemented a get method to return values of the current visualization state to save the state in a new *Page*, and a set method to update the underlying visualization from the saved state. We re-implemented the method that handles the transition. If opacity values of the glyphs change between *Pages*, we simply interpolate the values. If there is no valid intermediate state (e. g., changing a slice), we resorted to using the image-blending option. We did not modify the mobile and web clients as these rely on video outputs.

### IMAGES/FIGURES LICENSE AND COPYRIGHT

We as authors state that all of our figures in this appendix are and remain under our own personal copyright, with the permission to be used here. We also make them available under the Creative Commons Attribution 4.0 International (CC BY 4.0) license and share them at osf.io/qehvs.