# Improving Spatial Awareness for Human Trajectory Visualization in Space-Time Cubes

Tiago Gonçalves, Ana Paula Afonso, Bruno Martins

► To cite this version:

Tiago Gonçalves, Ana Paula Afonso, Bruno Martins. Improving Spatial Awareness for Human Trajectory Visualization in Space-Time Cubes. 15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. pp.327-334, 10.1007/978-3-319-22723-8_26 . hal-01610845

HAL Id: hal-01610845

https://inria.hal.science/hal-01610845

Submitted on 5 Oct 2017

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Improving Spatial Awareness for Human Trajectory Visualization in Space-Time Cubes

Tiago Gonçalves, Ana Paula Afonso, and Bruno Martins

Universidade de Lisboa, Lisbon, Portugal
tgoncalves@lasige.di.fc.ul.pt, apa@di.fc.ul.pt,
bruno.g.martins@ist.utl.pt

**Abstract.** With the increasing evolution of computer graphics, 3D visualizations have become more common and are nowadays seen as a promising way to represent complex types of information. In particular, space-time cubes (STC) have been proposed as an alternative to 2D maps for the visualization of spatio-temporal data, and they have become increasingly used to explore the dynamics and patterns of human movement. However, previous research has pointed out perceptual limitations that can condition the use of 3D views for decoding locations and spatial properties. We aim to address those issues by presenting a comparative study between three variants of the STC technique, with different methods to improve spatial awareness. Our results support that the use of a movable plane or an additional 2D map view improve users' accuracy when performing common tasks, and are preferred over simpler, yet less cluttered approaches. Additionally, it also supports the possible advantages of combining 2D and 3D views for human trajectory visualization.

**Keywords:** Spatio-temporal data, trajectories, information visualization, visual analytics, space-time cube, usability

## 1 Introduction

Throughout the years, researchers have tried to understand the dynamics associated with human movement and possible mobility patterns, e.g., in the context of urbanism studies and to improve the lives of citizens [3]. Nowadays, with the increasing popularity and accuracy of mobile computing technologies and navigational systems, large volumes of spatio-temporal data, representing human trajectories, have become available [3]. A trajectory can be defined as the evolution of an object's position through time, and represented as a time-stamped sequence of location points that may contain other types of *thematic* attributes, derived from the spatio-temporal locations or associated from other datasets.

Due to the critical role that spatial, temporal, and thematic attributes play in understanding trajectory data [13, 2] several challenges remain unsolved, in particular in areas related with visualization and human-computer interaction for the exploration of these data [2]. Considering the spatial properties of trajectories, maps are often seen as important tools for their visualization [12]. In particular, 2D maps are among the
most used techniques to represent georeferenced information. These take advantage of
the manipulation of several visual variables (e.g., colour or size) from different graph-
ical elements, such as points, lines, or areas, to display various types of information,
as present in trajectories, over a geographical plane [6]. However, although excelling
in the representation of the spatial component of trajectories, 2D maps tend to under-
mine the representation of the data's temporal component, thus often requiring the
combination of additional visualizations (e.g., time graphs) [3].

With the increasing advancements of computer graphics, 3D maps, and in particu-
lar space-time cubes (STCs) [8], have been proposed as viable options for the visuali-
zation of trajectories [12, 10]. STCs represent both spatial and temporal information
within a cube, where the x-y axes usually represent spatial information (e.g., lati-
tude/longitude), while the z-axis represents time [8]. Typically, time increases along
the z-axis, implying that the higher the information is within the cube, the most recent
it is [10, 1]. Similarly to 2D maps, trajectories can be displayed as a sequence of sym-
bols, graphically encoded to represent variations in the thematic attributes. However,
since time is represented as a spatial position, other visual variables are available to
represent the thematic attributes, when compared with traditional 2D maps [10]. STCs
also allow the representation of various layers of information, each one defined as a
plane in the z-axis [15], representing the state of an object in different moments in
time.

Some studies have been conducted in order to show that 3D visualizations, and STCs by extension, can be more effective than 2D visualizations in helping users understanding shapes and finding patterns/relations in the displayed data [9, 7]. However, due to their 3D characteristics, the interaction with STCs can be affected by human perceptual limitations. Previous studies have shown that 3D views are not as effective as 2D alternatives in location/positioning based tasks [9, 7]. To minimize these problems, previous studies suggested the use of interactive features, such as changing the point of view within the cube [10] or moving the plane representing spatial information up/downwards to facilitate locating objects in space and time [11]. As a result, while STCs can be considered as relevant tools for trajectory visualization, it is important to: (i) understand how to improve these techniques for spatially-related tasks; and (ii) empirically validate the proposed features to help the interaction with STCs, taking into account the types of tasks a user might perform to achieve a given goal [14].

In this paper, we aim to address those issues. We present a comparative user study
between three variants of the STC technique, aiming to understand: (i) methods that
non-expert users apply to acquire spatial information with STCs, as these increasingly
have to deal with spatio-temporal analysis issues [2]; (ii) features that may improve
spatial-information awareness in STCs; and, (iii) if there exist significant differences,
in terms of performance and preferences, between those features. The remainder of
this paper is organized as follows: the next section presents the three variants of the
STC technique. Then, the paper describes the user study and results obtained with
prototypes. The paper concludes with a discussion on the results, and with ideas for
future work.
## 2 Compared Visualizations

We developed three prototypes integrating variants of the STC technique (Figure 1), which allowed the visualization of pickup/dropoff locations of taxis, based on data provided by the Taxi and Limousine Commission of New York City.

The prototypes involved five main components. The first is the control panel (Figure 1a), which allows the selection of which data to visualize according to various filters, including the dates of movement and several thematic attributes, like number of passengers or payment type. The second component (Figure 1b) allows selecting the information to be represented by the visual variables of colour, shape, and size, supporting several combinations between the data attributes. Each colour and shape can be used to represent location types (pickup/dropoff), payment types and the periods of the day. Colour can also be used to represent the identification of the moving objects. Size can be used to display quantitative information, including the number of passengers or trip fares (larger icons representing larger values). The third component (Figure 1c) displays all data associated with any point highlighted in the STC. The fourth component (Figure 1d) describes the meaning of each visual variable illustrated in the STC, and allows un/viewing sub-sets of the data, based on those variables.

The last component consists of a STC visualization, as depicted in Figures 1e, 1f, and 1g, respectively depending on the prototype. In all three prototypes, the STC is composed by a 2D map plane, at the bottom of the cube, providing spatial information, and by several labels along the cube's height, providing temporal information. Trajectories are depicted as a sequence of points connected with lines, and coloured/sized according to the attributes selected in the representation panel. All prototypes support common interactive features such as panning and zooming, for the entire 3D view or just the 2D map displayed at the bottom. Each visualization also allows the user to rotate the cube along any of its axes and to rescale its height enabling analysts to, respectively, change the point of view and manipulate the temporal granularity of the data.

As emphasized in Figure 1, the differences between the three prototypes focus on this component, namely in how the spatial-component of the data is represented. In the first prototype (Figure 1f - *P1*), representing the most simple STC variant, selecting an object will display the time moment in which the object was detected, with a thick line pointing at the object's location in the map plane. In the second prototype (Figure 1g - *P2*), based on the most common (yet not validated) proposal to improve spatial awareness on STCs [11], in addition to the previous methods we have that a copy of the map plane is displayed at the same height as the selected object. Although this approach should provide, more easily, spatial context regarding the selected object, adding a plane to a certain height will, necessarily, occlude all data located below the selected object. Finally, the third prototype (Figure 1e - *P3*) aims to combine the advantages of 2D and 3D views by displaying, at the right-bottom of the visualization, a 2D map overview. Although this alternative may lead users to divide their attention between views, also providing information with less detail [5], it will continuously provide some spatial context to the user.
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Components of the prototypes used in the experiment: (a) Control Panel; (b) Representation Panel; (c) Information Panel; (d) Legend; (e) P3 STC with overview; (f) P1 basic STC; (g) P2 STC with moveable spatial plane</figcaption></figure>

# 3 User Study

This section describes the comparative user study conducted with the described prototypes. In this study we aimed at: (i) identifying strategies that analysts may adopt to obtain spatial information when using a STC; (ii) identifying possible interactive techniques to help obtaining spatial information; and (iii) assessing the users' experience with those methods, and empirically compare them.

Based on the features of each prototype, our hypotheses were the following: (H1) participants will prefer the interfaces with complementary spatial information (P2 and P3), due to the additional maps displayed; (H2) participants will have a better performance and a lower number of interactive actions with P2 and P3, due to spatial-context aids that are provided; and (H3) in elementary tasks, i.e. tasks that focus in just one object or time moment, users will have a better performance, due to the smaller number of points/lines displayed.

A total of 20 participants volunteered to the study, aged between 19 and 34 (Av: 24.6, SD: 4.1). Although, all participants were knowledgeable with computer applications and geographic information systems for the search of directions towards specific
points of interest (e.g., Google Maps), none of them were familiar with trajectory data analysis, nor familiarized with New York's geography.

## 3.1 Experimental Design

To test our hypotheses, the participants performed two tasks, affected by the spatial components of trajectory data. These tasks are based on the most common types of cartographic visualization objectives described in the literature [14]. The first, identify, required users to locate taxis according to some spatio-temporal and thematic constraints (e.g., which taxis dropped passengers at the Laguardia Airport? where was taxi n°1 by 3pm?). The second, compare, required the analysis of similarity/difference relations between data elements in the STCs (e.g., which driver served, more frequently, over a larger geographical area? in which periods of the day did driver 1 serve over a larger area?).

We considered two independent variables: Visualization technique (*Vt*), with three levels, corresponding to the three prototypes, *P1*, *P2*, and *P3*; and Query category (*Qc*), with four levels adapted from the spatio-temporal data queries identified by [4]: (i) *elementary what+where*, *elementary when (EE)*; (ii) *elementary what+where, general when (EG)*; (iii) *general what+where*, *elementary when (GE)*; and (iv) *general what+where, general when (GG)*. These categories determine whether the focus of the task is in just one (*elementary*) moving object (*what+where?*) and/or in a specific time moment (*when?*), or if the focus of the task is instead in several (*general*) objects and/or across a time period.

The experiment followed a within subjects design and all participants carried out each task individually, in a controlled environment. At the beginning of the study, subjects were briefed about the objectives of the experiment, and they viewed a demonstration of the prototypes. Before carrying out the tasks, they were asked to interact with the applications, and were encouraged to clarify any doubts. After the training phase, the participants performed the two tasks, taking into account the different visualizations and query categories. To mitigate sequence effects, the order in which the independent variables were presented was counterbalanced using a *latin-square* design. However, since comparison tasks require necessarily more than one data item (either from the same, or from different trajectories), the level *elementary what+where* elementary when (*Qc(EE)*) was not considered for these tasks. Consequently, each participant performed a total of 21 trials: (3 *Vt*) x (4 identify + 3 compare).

To assess our hypotheses, we considered the following dependent variables: (i) subjective preferences (i.e., participants were asked to rate the ease of use of each prototype on 10-Likert scale); (ii) task accuracy (i.e., each task was rated between 0 and 10, depending on the detail given by the participants - e.g. saying that a taxi dropped a passenger in *New York* is less detailed than *Manhattan*, which is less detailed than *Central Park*); (iii) task completion time; and (iv) number of actions performed, including panning and zooming operations, and rescaling the STC's height. During the study, participants were encouraged to *think aloud*, and share their opinions about the techniques.
## 3.2 Results

This section overviews the results obtained in the study, with a focus on the most statistically significant ones.

We applied Friedman's test, followed by a Wilcoxon Signed Rank test with a Bonferroni correction for pairwise comparisons, to compare the differences between the **participants' opinions** after each set of tasks. In the **identify** task, participants have shown a higher preference ($X^2(2)=14.8, p=0.001$) for $Vt(P3)$ (7.3/10) and $Vt(P2)$ (6.5/10) over $Vt(P1)$ (5.5/10) ($Z=-3.42, p=0.001$ and $Z=-2.98, p=0.001$, respectively). In the **compare** task, participants have also shown a significant higher preference ($X^2(2)=11.7, p = 0.003$) for $Vt(P3)$ (7.2/10) over $Vt(P1)$ (6/10) and $Vt(P2)$ (6.5/10) ($Z=-2.96, p=0.003$ and $Z=-2.56, p=0.01$, respectively). These results, in turn, support our first hypothesis (*H1: participants would prefer P2 and P3 over P1*).

A similar procedure was used to compare the **participants' accuracy**, with the visualizations and in the different categories. Table 1a) shows the average scores obtained in the various tasks. The tests revealed significant differences in $Vt(X^2(2)=9.796, p=0.007)$ and $Qc(X^2(3)=26.154, p < 0.001)$, in the **identify task**. Pairwise comparison tests revealed significantly less accurate results with $Vt(P1)$ comparatively to the other two ($Z = -2.72, p=0.006$ and $Z=-2.54, p=0.011$, for $Vt(P2)$ and $Vt(P3)$ respectively). Participants were also significantly less accurate in $Qc(GE)$ tasks, comparatively to all others ($Z = -3.57, p < 0.001$; $Z = -4.81, p < 0.001$; and $Z=-2.787, p=0.005$ for $Qc(EE)$, $Qc(EG)$, and $Qc(GG)$ respectively). Similarly, in **comparison** tasks, participants were significant less accurate in $Qc(GE)$ tasks ($X^2(2)=8.54, p=0.014$) comparatively to $Qc(EG)$ tasks ($Z = -2.83, p=0.005$). These results were somewhat expected, as in this category of tasks (*GE*) users need to focus in several objects in one specific time moment. This implies focusing user attention on a single temporal plane, having to ignore/filter others, thus resulting in a higher visual noise and/or cognitive workload.

On the other hand, we applied a repeated measures ANOVA, followed by Bonferroni tests for pairwise comparisons, for the comparative analysis of the participants' **task completion times** and the **number of interactive actions**. Table 1 (b and c) shows the participants' mean results for all tasks, with all combinations of the two independent variables. Regarding **task completion times**, the tests revealed a significant effect from $Qc(F(2.5)=6.671, p=0.001)$ in the **identify** task. The pairwise comparison tests revealed significant lower times in the less demanding type of task ($Qc(EE)$), comparatively to the remaining three (with $p \le 0.05$ in all cases). Regarding **the number of actions**, significant effects were detected from $Qc$ and $Vt$ in the identify task ($F(2.57)=10.464, p < 0.001$ and $F(1.68)=3.260, p=0.05$, respectively). Pairwise comparison tests revealed also a significantly lower number of actions in $Qc(EE)$ tasks ($p \le 0.013$, comparing to all cases), and a generally significantly lower number of actions of $Vt(P2)$ comparatively to $Vt(P1)$ ($p=0.046$). As such, these results go in agreement with our second (*H2: better performance with P2 and P3*) and third hypothesis (*H3: better performance in 'simpler' tasks*).
<table><caption>**Table 1.** Mean results to the different participants' in terms of a) task accuracy (acc), b) task completion time, and c) number of interactive actions</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="4">Identify</th><th colspan="3">Compare</th></tr><tr><th>EE</th><th>GE</th><th>EG</th><th>GG</th><th>GE</th><th>EG</th><th>GG</th></tr></thead><tbody><tr><td rowspan="3">a) Acc [0-10]</td><td>P1</td><td>8.05</td><td>8.30</td><td>6.40</td><td>7.50</td><td>7.45</td><td>8.05</td><td>8.13</td></tr><tr><td>P2</td><td>8.40</td><td><strong>9.20</strong></td><td><strong>7.55</strong></td><td>8.35</td><td>7.75</td><td>8.83</td><td>8.13</td></tr><tr><td>P3</td><td><strong>8.65</strong></td><td>8.70</td><td>7.45</td><td><strong>8.63</strong></td><td><strong>8.25</strong></td><td><strong>9.50</strong></td><td><strong>8.85</strong></td></tr><tr><td rowspan="3">b) Time (sec)</td><td>P1</td><td><strong>122.90</strong></td><td>166.55</td><td>203.71</td><td>176.33</td><td>141.40</td><td>116.26</td><td>122.38</td></tr><tr><td>P2</td><td><strong>126.24</strong></td><td>168.55</td><td>193.24</td><td>174.36</td><td>152.48</td><td>123.10</td><td>146.52</td></tr><tr><td>P3</td><td><strong>127.93</strong></td><td>180.05</td><td>178.81</td><td>142.64</td><td>156.17</td><td>121.57</td><td>107.07</td></tr><tr><td rowspan="3">c) N° Actions</td><td>P1</td><td>26.65</td><td>31.95</td><td>41.80</td><td>51.25</td><td>35.20</td><td>38.80</td><td>46.65</td></tr><tr><td>P2</td><td><strong>22.85</strong></td><td><strong>28.10</strong></td><td><strong>35.90</strong></td><td><strong>40.15</strong></td><td><strong>34.55</strong></td><td><strong>32.95</strong></td><td>46.55</td></tr><tr><td>P3</td><td><strong>22.55</strong></td><td>45.65</td><td>41.45</td><td>44.55</td><td><strong>34.80</strong></td><td>35.35</td><td><strong>38.00</strong></td></tr></tbody></table>

Overall, participants commented that using a moveable plane, or a 2D map overview on the STC, indeed helped them to acquire spatially-related information more easily. They said that the 2D map overview helped them to have a better overall perception of the geographical space, and that the map allowed them to see spatial and temporal information simultaneously. The moveable plane, on the other hand, was considered more helpful to find the locations of specific points, or to analyse the evolution of locations over time from a given taxi. Some participants, however, commented that, sometimes, the moveable plane would occlude information, conditioning the interaction. Some users also expressed an interest in having more control over the 2D map overview on P3, possibly due to their familiarity with interactive 2D maps.

# 4 Conclusions and Future Work

This paper presented a comparative study between three variants of space-time cubes for the visualization of human trajectories, combining general types of visualization tasks with the main categories of spatio-temporal queries. The results point out that the use of a moveable plane with spatial information and/or the use of a 2D map overview significantly improves users' performance in tasks for the identification of locations and objects, even though sometimes these techniques can occlude information. Moreover, users have shown a significantly higher preference towards a variant that combines 2D and 3D views. This further supports the importance of studying the advantages of combining both types of techniques [1, 7], which may, in turn, be useful for developers and analysts needing to decide which features to use on a given technique for trajectory visualization.

Nevertheless, further studies should still be conducted. As future work, we propose to continue studying these issues, in particular, the combination of 2D maps and 3D STCs within the same visualization, and assessing their advantages and disadvantages comparatively to the individual use of 2D maps and 3D STCs.
**Acknowledgements** This work was funded by FCT (Portuguese Science and Technology Foundation) through the scholarship with reference SFRH/BD/78837/2011, the LaSIGE Strategic project with reference PEst-OE/EEI/U10408/2014, the INESC-ID multi-annual funding project, with reference PEst-OE/EEI/LA0021/2013, and through the KD-LBSN project, with reference EXPL/EEI-ESS/0427/2013.

**References**

1. Amini, F., Rufiange, S., Hossain, Z., Ventura, Q., Irani, P., McGuffin, M.: The impact of interactivity on comprehending 2d and 3d visualizations of movement data. IEEE Transactions on Visualization and Computer Graphics 21(1), 122–135 (2015)

2. Andrienko, G., Andrienko, N., Demsar, U., Dransch, D., Dykes, J., Fabrikant, S.I., Jern, M., Kraak, M.J., Schumann, H., Tominski, C.: Space, time and visual analytics. International Journal of Geographical Information Science 24(10), 1577–1600 (2010)

3. Andrienko, N., Andrienko, G., Bak, P., Keim, D., Stefan, W.: Visual Analytics of Movement. Springer, 1st edn. (2013)

4. Andrienko, N., Andrienko, G., Gatalsky, P.: Exploratory spatio-temporal visualization: an analytical review. Journal of Visual Languages & Computing 14(6), 503–541 (2003)

5. Baudisch, P., Good, N., Bellotti, V., Schraedley, P.: Keeping things in context: A comparative evaluation of focus plus context screens, overviews, and zooming. In: Proceedings of CHI 2002. pp. 259–266 (2002)

6. Bertin, J.: Semiology of Graphics: Diagrams, Networks, Maps. University of Wisconsin Press (1967)

7. Gonçalves, T., Afonso, A.P., Martins, B.: Visualizing human trajectories: Comparing space-time cubes and static maps. In: Proceedings of HCI 2014. pp. 207–212 (2014)

8. Hägerstrand, T.: What about people in regional science? Papers of the Regional Science Association 24(1), 6–21 (1970)

9. John, M.S., Cowen, M.B., Smallman, H.S., Oonk, H.M.: The use of 2D and 3D displays for shape-understanding versus relative-position tasks. Human Factors: The Journal of the Human Factors and Ergonomics Society 43(1), 79–98 (2001)

10. Kjellin, A., Pettersson, L.W., Seipel, S., Lind, M.: Evaluating 2D and 3D visualizations of spatiotemporal information. ACM Transactions on Applied Perception 7(3), 19:1–19:23 (2010)

11. Kraak, M.J.: The space-time cube revisited from a geovisualization perspective. In: Proceedings of ICC 2003. pp. 1988–1995 (2003)

12. Kraak, M.J.: Geovisualization and time new opportunities for the space-time cube. In: Dodge, M., McDerby, M., Turner, M. (eds.) Geographic Visualization Concepts, Tools and Applications, chap. 15, pp. 294–306. Wiley (2008)

13. Peuquet, D.J.: It's about time: A conceptual framework for the representation of temporal dynamics in geographic information systems. Association of American Geographers 84(3), 441–461 (1994)

14. Roth, R.E.: Cartographic interaction: What we know and what we need to know. Journal of Spatial Information Science (6), 59–115 (2013)

15. Thakur, S., Hanson, A.: A 3D visualization of multiple time series on maps. In: Proceedings of IV 2010. pp. 336–343 (2010)