<figure><img src="image_1.png" /></figure>

HAL
open science

# Designing Interactive Spatiotemporal Visualizations to Enhance Movie Browsing

Ana Jorge, Nuno Correia, Teresa Chambel

<figure><img src="image_2.png" /></figure>

## To cite this version:

Ana Jorge, Nuno Correia, Teresa Chambel. Designing Interactive Spatiotemporal Visualizations to Enhance Movie Browsing. 16th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2017, Bombay, India. pp.352-355, 10.1007/978-3-319-68059-0_28. hal-01679778

HAL Id: hal-01679778

https://inria.hal.science/hal-01679778

Submitted on 10 Jan 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Designing Interactive Spatiotemporal Visualizations
to Enhance Movie Browsing

Ana Jorge<fnref n="1" />,<fnref n="2" />, Nuno Correia<fnref n="1" />, Teresa Chambel<fnref n="2" />

**Abstract.** This paper presents a case study on the design of spatiotemporal in-
teractive visualizations of movies, both collections and contents, to provide en-
hanced support for conveying meaning and for browsing, targeting casual and
professional users, with encouraging results for future research and adoption.

**Keywords:** Design. Interactive. Visualization. Movie. Time. Space. Evaluation

# 1 Introduction

Movies, with their rich contents conveyed in images, text, music and narration, along
time, tell us stories of different places and have great emotional impact on us. Moreo-
ver, technological advances are making available huge amounts of movies over the
years, on the Internet and interactive TV. However, the richness that makes these mov-
ies so interesting and accessible, comes with a challenging complexity, highlighting the
need for new and powerful ways to access, browse, and view them, and interactive visu-
alization can play an important role. In spite of the several contributions to this field,
reviewed in previous publications [1,2], most approaches and services for movies and
video access do not fully support this richness, and the approaches to visualization in
time and space [3] do not usually address video and movies. In this paper, we present a
case study on the spatiotemporal interactive visualizations that we designed to help ful-
fill this need, both for movie collections and contents, as an extension to our previous
work on: MovieClouds [4], an interactive web application that allows to browse movies
based on their content, mainly audio and subtitles, with an emotional perspective; and
Sight Surfers [5], an interactive web application for sharing and browsing georeferenced
360° interactive videos, in the context of the VIRUS and ImTV projects. The proposed
visualizations contribute with visual, temporal and spatial dimensions, allowing to ac-
cess movies released over time, and enriching the representations and access based on
different aspects of their contents, along time and space. This approach [1,2] has been
refined with the ultimate goal of providing effective overviews and browsing mecha-
nisms that may provide insights in analytical or more ludic uses, and its recent evalua-
tion had very encouraging results towards their development and inclusion in real uses.

<footnote n="1">¹,²NOVA LINCS, Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa, Portugal
ananunesjorge@gmail.com; nmc@fct.unl.pt</footnote>

<footnote n="2">²LASIGE, Faculdade de Ciências, Universidade de Lisboa, Portugal
tc@di.fc.ul.pt</footnote>
## 2 Goals

In the design of the interactive visualizations we aimed at their effectiveness in showing meaning; their ability in providing good usability and user experience; and their usefulness and adequacy to be adopted by casual users and professionals in real scenarios. The evaluation assessed this, following the order of these research questions:

RQ1. Representations: are they intuitive and clear in reflecting meaning?

RQ2. Visualizations from collection to contents: are they able to effectively show the first level of information in a glance while guiding the user in the interaction towards a goal (learnability)?

RQ3. Navigation: are visualizations an added value to explore and access movies?

RQ4. Target: which target users take the most of visualizations?

## 3 Spatiotemporal Visualizations: Design and Evaluation

This section presents an overview of the main visualizations designed (Fig.1) and some highlights about their recent evaluation, in order to provide insights and lessons to inform future research and developments. It was a Task-oriented evaluation focusing on the Observation of users performing the tasks, their hesitations and errors, and semi-structured Interviews focusing on perceived usefulness, ease of use and satisfaction (USE) with 1-5 scales, and ergonomic, hedonic and appeal Quality aspects [6], while providing comments as qualitative feedback, in sessions lasting on average one hour per participant. 20 participants aged 22-64, both from general public and professionals (e.g. movie directors and editors, film critics, and sound technicians).

Visualizations go from movie collections (Fig.1 top) overviews (b-f, 2abc) or trajectories (3abc), to individual movie contents details (bottom) (4 and 5):

**Time:** the user can observe quantities of movies released over a year by genre (1a-b), and ask for details through titles by List (1c), where the most rated movies are highlighted; or by Spots (1d), each spot corresponding to one genre of the selected movie (half dimension, e.g., corresponds to a movie with two genres). These visualizations proved effective with almost all the participants accomplishing tasks such as locating the month with more movies released. The lower frequencies are the most difficult to perceive, but this issue is easily solved by zooming in. In (1e) users can compare movies in order to know about their images, speech, audio amplitude and genre, represented in circles that reflect the duration, being possible to disclose the title, plot, images of a requested moment of the movie (by hover), or the scene playing (by click). Many people would like to have this visualization available (M:3.8; Std:1.2) and most of them considered it more suited for exploration. A movie director called it an "Evocation of the film" due to its ability to reflect emotions. In general, the participants considered the visualization a good way to find a movie to watch, and used Supporting, Interesting, Pleasant and Comprehensible [7] to characterize it. When compared with Netflix, this visualization was considered more emotional and ludic although the traditional cover image was referred to complement information. Visualization in figure (1f) seemed clear since almost all the participants could identify the
most dynamic movie. It was considered less innovative than the previous visualiza-
tion for its resemblance to a bar chart and more suited for movie professionals due to
its perceived accuracy, reflecting the “mood” of the movie and its structure.

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Interactive Spatiotemporal Visualizations of Movies. All visualizations allow to access more information; and to navigate, as shown by the arrows.</figcaption></figure>

**Space:** the amounts of genres released through location are represented (2a), being exemplified the detail of the romantic movies (2b) and choosing one to watch (2b-c). These visualizations were highly rated since genre and maps are commonly used.

**Spatiotemporal:** after being aware of the country with more movies shot (3a), one area can be zoomed in towards the more shot (3b), and detailed to the more dated (brownish) and slower (higher) (3c) trajectories. The latter visualization caused some strangeness due to its novelty, but the results were still encouraging (M: 3.8).

**Movie contents:** show the properties of a whole movie (4) through circular elements that represent properties over time/duration (motion, colors, scene length, audio, speech, expressed and felt emotions) with the selected property highlighted as a tag
cloud in the center. The user drags the circular elements that rotate in synchrony with
the timeline. On the right, current scene is playing, synchronized with corresponding
trajectory in a map, that can be zoomed. This visualization revealed very good results
with professional users more determined to use it. The possibilities both to hide and
turn visible the tracks of information, and to change properties were strongly appreci-
ated. Visualization (5) represents one trajectory through image, audio amplitude,
speech and other videos shot in that area allowing to play video at the clicked position
in the trajectory. In the e.g. Before Sunrise movie in Vienna. It was very appreciated
and understood by the participants. The preferred visualizations were 1e, followed by
4, and 5, reflecting interest in visualizing and exploring content in richer ways.

4 Lessons Learned

The results were very encouraging since all visualizations were appreciated and most
proposed tasks were readily accomplished. Regarding the 4 research questions: RQ1.
Representations were easy to understand, e.g. users are able to relate and compare
properties among scenes. RQ2. Visualizations show in a glance what they represent; and
allow RQ3. Navigation to access rich information to be explored at different levels of
detail, in time and space, e.g. enter one scene and explore the location where the movie
happens. They were considered an added value by all the participants, being highlighted
the aesthetics and ludic way of interacting. RQ4. Casual uses were the most referred,
although professionals have mentioned their interest in this kind of visualizations.

5 Acknowledgments

This work is supported by an FCT fellowship under the UT Austin|Portugal Program (SFRH/-BD/51798/2011); LASIGE, Faculdade de Ciências (UL); and NOVA LINCS, Faculdade de Ciências e Tecnologia (UNL).

6 References

1. Jorge, A., Chambel, T.: "Exploring Movies through Interactive Visualizations," in *Proc. of British HCI*, 6 pgs (2013)

2. Jorge, A., Serra, S., Chambel, T.: "Interactive Visualizations of Video Tours in Space and Time," in *Proc. of British HCI*, 329-334 (2014)

3. Andrienko, G., Andrienko, N., Demsar, U., Dransch, D., Dykes, J., Fabricant, S., Jern, M., Kraak, M.-J., Schumann, H., Tominski, C.: "Space, time and visual analytics", Taylor & Francis, 24(10), Oct. (2010)

4. Martins, P., Langlois, T., Chambel, T.: "MovieClouds: Content-Based Overviews and Exploratory Browsing of Movies". In *Proc. of Academic MindTrek*, ACM, 133-140 (2011)

5. Noronha, G., Álvares, C., and Chambel, T. (2012a) Sight Surfers: 360º Videos and Maps Navigation. In Proc. of the ACM Multimedia Workshop GeoMM’12, 19-22, ACM (2012).

6. Hassenzahl, M., Platz, A., Burmester, M., Lehner, K., “Hedonic and ergonomic quality aspects determine a software’s appeal,” in *Proc. of CHI*, 201–208 (2000).