# Situated Visualization in Motion for Swimming

Lijie Yao, Anastasia Bezerianos, Romain Vuillemot, Petra Isenberg

► To cite this version:

Lijie Yao, Anastasia Bezerianos, Romain Vuillemot, Petra Isenberg. Situated Visualization in Motion for Swimming. Journée Visu 2022, Jun 2022, Bordeaux, France. hal-03700406

HAL Id: hal-03700406

https://inria.hal.science/hal-03700406

Submitted on 21 Jun 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Visualisation Localisée en Mouvement pour la Natation

## Situated Visualization in Motion for Swimming

Lijie Yao, Anastasia Bezerianos, Romain Vuillemot, and Petra Isenberg

**English Abstract**—Competitive sports coverage increasingly includes information on athlete or team statistics and records. Sports video coverage has traditionally embedded representations of this data in fixed locations on the screen; but more recently also attached representations to athletes or other targets in motion. These publicly used representations so far have been rather simple and systematic investigations of the research space of embedded visualizations in motion is still missing. Here we report on our preliminary research in the domain of professional and amateur swimming. We analyzed how visualizations are currently added to the coverage of Olympics swimming competitions and then plan to derive a design space for embedded data representations for swimming competitions. We are currently conducting a crowdsourced survey to explore which kind of swimming-related data general audiences are interested in, in order to identify opportunities for additional visualizations to be added to swimming competition coverage.

## 1 INTRODUCTION

We define visualizations in motion as visual data representations that are used in contexts that exhibit relative motion between a viewer and an entire visualization [19]. There are already many scenarios that involve visualizations in motion such as sports analytics, video games, wearable devices, and data physicalizations. However, the relative movement between viewers and visualizations in scenarios above has not received much attention in the visualization research community and we know very little about how effectively people can read these visualizations.

To explore this research space we first focused on sports analytics as an application scenario in which embedded moving visualizations are particularly promising. In many sports athletes are in motion and stationary viewers are interested in seeing a data such as statistics, past records, trajectories, etc. We began our work by focusing on one specific sport, swimming, for the following reasons: Like other sports it has rich dynamic data and we have observed an increase of real-time moving graphics during competitions (e.g., record lines), sometimes attached to athletes (e.g., country flags or speed labels). This indicates that audiences may be open to the addition of more complex moving data representations. Moreover, it is a sport where the movement of athletes is constrained in lanes, making it an appealing starting

point for our investigation into how viewers perceive and track this data.

To better understand the visualization opportunities in this context, we first analyzed current visualizations embedded in swimming competitions. We next plan to understand what other data we could visualize. Currently, we are conducting a survey to explore the information interests of general audiences when watching a swimming competition. Our next phase is to develop a prototype with embedded moving visualizations, to show the data of interest to general audiences, and to evaluate the efficiency of *visualization in motion* in an actual concrete application context.

## 2 RELATED WORK

Our research is closely related to the topic of situated and embedded visualization [17] that represent information close to a data referent. Specifically, we focus on embedded visualizations that are in motion. Although there is little discussion in the visualization community about *relative* motion between the entire visualization and the viewers, visualization research involving movement inside the visualization is extensive. For example, animation is frequently used for smooth transition between different data points [1], [6], or to morph between different representations [3], [7], [13]. In contrast, in our work we focus on the potential impact and application scenarios when the entire visualization is moving rather than its individual components.

A large number of visual analytics tools have been developed for sports visualization. Examples include tools for table tennis [8], [16], [18], soccer [4], [10], [14], basketball [2], [15], badminton [11], [20], and tennis [12]. In contrast to entire moving visualizations,

* Lijie Yao: Université Paris-Saclay, CNRS, Inria, LISN, Orsay, France; E-mail: lijie.yao@inria.fr.

* Anastasia Bezerianos: Université Paris-Saclay, CNRS, Inria, LISN, Orsay, France; E-mail: anastasia.bezerianos@universite-paris-saclay.fr.

* Romain Vuillemot: Université de Lyon, École Centrale de Lyon, CNRS, UMR5205, LIRIS, France; E-mail: romain.vuillemot@ec-lyon.fr.

* Petra Isenberg: Université Paris-Saclay, CNRS, Inria, LISN, Orsay, France; E-mail: petra.isenberg@inria.fr.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Examples of embedded visualizations for swimming. Left: Stationary swimmers' names in each lane. Right: Swimmers' current position circles and record line moving with the swimmers' movement. Both images have a stationary timer in the top left corner.</figcaption></figure>

Image credits: We analyzed the videos from the 2020 Olympics. Since we did not obtain permission to use screenshots of these videos, we use our own recordings of other competitions.

these tools consistently placed visualizations at specific fixed locations: for example, in VisCommentator [5], visualizations are shown when video playback is paused, rather than embedded in the video stream. Besides, the target users for most previous visualization tools were experts, such as coaches and/or athletes; whereas, we target general audiences.

## 3 SWIMMING VISUALIZATION ANALYSIS

As our research focuses on *visualization in motion* in swimming competitions, we investigated a) which data is displayed in current competitions, b) using which representations, c) where on the screen, and d) under which camera positions and perspectives. We analyzed all 16 available swimming videos of the 2020 Tokyo Olympics from the official Olympic Aquatics YouTube channel [9]. We picked all 16 available videos so that they would cover all 4 strokes and all 5 race lengths (50m – 400m, including 4 x 100m medlay). Since we were interested in scenarios where the entire visualizations moved, we only focused on the swimming process. That is, we do not discuss any visualizations shown outside of the swimming activity itself such as when athletes stand before the race or during the awards ceremony. We found:

**Data items:** There were three categories of data shown in the broadcast: swimmers': nationalities, names, lane number, current speed, and distance swam; temporal information, and record-related information. Temporal information included the time taken from the start of the race to the current time and the elapsed time difference between the current swimmer and a specific record. The records shown were the world record and the Olympic record.

**Representations:** We saw three types of representations: symbols, numbers, text, and marker lines. Symbols represented nationality using a flag; text represented lane numbers, names, times, speeds, and distances; and line marks represented where a swimmer should be to match a record. Only two visualizations moved in the videos we surveyed: text moved behind the swimmer to show their current speed and the record marker line moved to show an interpolated position of a record (see Figure 1: Right).

**Display positions:** Apart from the two moving visualizations mentioned above, the rest of the visualizations remained in a fixed location. They were attached to the corners of the screen or embedded in the swimming pool (see Figure 1: Left). The top left, bottom left and bottom right corners were the three most frequently used display areas for real-time changing data. When a swimmer turned or a handoff occurred, the current standings and elapsed time appeared in the upper left corner (see Figure 1: Left), the record and the gap to the record appeared in the lower left corner, and the current timing and meta information about the competition appeared in the lower right corner. Once the first three swimmers arrived their names, nationalities, and lanes were also embedded in the pool.

**Camera positions and perspectives:** There were two main camera positions: in the air and under water. The camera perspectives were more varied. We saw bird's-eye views (top), side views, and diagonal views. In the diagonal view, swimmers swam along the diagonal of the screen – from the bottom right to the top left or from the bottom left to the top right corner. According to our statistics on the display time of each view, the diagonal view was visible the longest. When the camera moved or the camera perspective changed, the moving and embedded visualizations were deformed to match the current view.

## 4 WORK IN PROGRESS: SURVEY

After analyzing Olympic swimming coverage, we found that the displayed data items had not changed for over 20 years, and the visualizations used were consistently simple and basic. To understand if people's preferences and interest in information on swimming has changed we are now conducting a crowdsourcing survey.

## 5 DISCUSSION AND FUTURE WORK

While our survey is running we are preparing a prototype for embedding different data visualizations in swimming videos. We will apply our survey findings to our prototype: we will embed the most interesting data next to the swimmers. In the future, we want to evaluate the impact of contextual factors, such as the background or camera perspectives on *visualizations in motion*. We also plan to examine how efficient visualizations in motion are and how much information viewers can perceive in a real application scenario.

## 6 ACKNOWLEDGEMENT

This work was partly supported by the Agence Nationale de la Recherche (ANR), grant number ANR-19-CE33-0012. The authors thank all volunteers who participated in our survey.
REFERENCES

[1] B. Bach, E. Pietriga, and J. Fekete. GraphDiaries: Ani-
mated transitions and temporal navigation for dynamic net-
works. *IEEE Transactions on Visualization and Computer Graph-
ics*, 20(5):740–754, 05 2014.

[2] L. Bai, C. Efstratiou, and C. S. Ang. wesport: Utilising wrist-band sensing to detect player activities in basketball games. In *Proc. of 2016 IEEE International Conference on Pervasive Computing and Communication Workshops*, pages 1–6, 2016.

[3] T. Bladh, D. Carr, and M. Kljun. The effect of animated transitions on user navigation in 3D tree-maps. In *Proc. of the International Conference on Information Visualisation*, pages 297–305, 09 2005.

[4] S. Chen, B. Hu, Y. Gao, Z. Liao, J. Li, and A. Hao. Lower limb balance rehabilitation of post-stroke patients using an evaluating and training combined augmented reality system. In *Proc. of 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct*, pages 217–218, 2020.

[5] Z. Chen, S. Ye, X. Chu, H. Xia, H. Zhang, H. Qu, and Y. Wu. Augmenting sports videos with viscommentator. *IEEE Transactions on Visualization and Computer Graphics*, pages 1–13, 2021.

[6] F. Chevalier, P. Dragicevic, and S. Franconeri. The not-so-staggering effect of staggered animated transitions on visual tracking. *IEEE Transactions on Visualization and Computer Graphics*, 20(12):2241–2250, 12 2014.

[7] J. Heer and G. Robertson. Animated transitions in statistical data graphics. *IEEE Transactions on Visualization and Computer Graphics*, 13(6):1240–1247, 11-12 2007.

[8] H. Oagaz, B. Schoun, and M. Choi. Performance improvement and skill transfer in table tennis through training in virtual reality. *IEEE Transactions on Visualization and Computer Graphics*, 2021.

[9] Olympics Aquatics. Welcome to the official home of aquatics at the Olympics. https://www.youtube.com/channel/UCyu7apf_VsBIOOcu53hhgcA Accessed 03 2022.

[10] C. Perin, R. Vuillemot, and J. D. Feket. Real-tima crowdsourcing of detailed soccer data. In *Proc. of the 1st VIS’13*

[19] L. Yao, A. Bezerianos, and P. Isenberg. Situated visualization in motion. In *Posters of the IEEE Visualization Conference*, 10 2020. https://hal.inria.fr/hal-02946587.

Workshop on Sports Data Visualization, 10 2013. https://hal.inria.fr/hal-00868775.

[11] G. Pingali, A. Opalach, Y. Jean, and I. Carlbom. Visualization of sports using motion trajectories: providing insights into performance, style, and strategy. In *Proc. of 2001 IEEE Visualization Conference*, pages 75–544, 2001.

[12] T. Polk, J. Yang, Y. Hu, and Y. Zhao. TenniVis: Visualization for tennis match analysis. *IEEE Transactions on Visualization and Computer Graphics*, 20(12):2339–2348, 12 2014.

[13] G. G. Robertson, J. D. Mackinlay, and S. K. Card. Cone Trees: Animated 3d visualizations of hierarchical information. In *Proc. of the Conference on Human Factors in Computing Systems*, pages 189–194, 04 1991.

[14] M. Stein, H. Janetzko, A. Lamprecht, T. Breitkreutz, P. Zimmermann, B. Goldlücke, T. Schreck, G. Andrienko, M. Grossniklaus, and D. A. Keim. Bring it to the pitch: Combining video and movement data to enhance team sport analysis. *IEEE Transactions on Visualization and Computer Graphics*, 24(1):13–22, 01 2018.

[15] L. Tica, S. Rishi, Y. Yalong, N. Carolina, B. Johanna, A. S. Maurice, and P. Hanspeter. Towards an understanding of situated ar visualization for basketball free-throw training. In *Proc. of the Conference on Human Factors in Computing Systems*, number 461 in CHI’21, pages 1–13, 05 2021.

[16] J. Wang, J. Wu, A. Cao, Z. Zhou, H. Zhang, and Y. Wu. Tac-miner: Visual tactic mining for multiple table tennis matches. *IEEE Transactions on Visualization and Computer Graphics*, 27(6):2770–2782, 2021.

[17] W. Willett, Y. Jansen, and P. Dragicevic. Embedded data representations. *IEEE Transactions on Visualization and Computer Graphics*, 23(1):461–470, 01 2016.

[18] E. Wu, M. Piekenbrock, T. Nakumura, and H. Koike. Spinpong - virtual reality table tennis skill acquisition using visual, haptic and temporal cues. *IEEE Transactions on Visualization and Computer Graphics*, 27(5):2566–2576, 2021.

[20] S. Ye, Z. Chen, X. Chu, Y. Wang, S. Fu, L. Shen, K. Zhou, and Y. Wu. ShuttleSpace: Exploring and analyzing movement trajectory in immersive visualization. *IEEE Transactions on Visualization and Computer Graphics*, 27(2):860–869, 02 2021.