# HomeOlympics: Turning Everyday Outdoor Physical Activities into Olympic-Style Experience

Petra Isenberg, Lionel Reveret, Romain Vuillemot

► To cite this version:

Petra Isenberg, Lionel Reveret, Romain Vuillemot. HomeOlympics: Turning Everyday Outdoor Physical Activities into Olympic-Style Experience. IEEE VIS Workshop on First-Person Visualizations for Outdoor Physical Activities: Challenges and Opportunities, Oct 2024, Virtual, Unknown Region. hal-04782928

HAL Id: hal-04782928

https://inria.hal.science/hal-04782928v1

Submitted on 14 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# HomeOlympics: Turning Everyday Outdoor Physical Activities into Olympic-Style Experience

Petra Isenberg*

Inria Saclay

Lionel Reveret†

Inria Grenoble

Romain Vuillemot‡

Ecole Centrale de Lyon
LIRIS UMR 5205

<figure><img src="image_2.png" /><figcaption>Figure 1: Examples of first-person views that can benefit from embedded visualization designs to help the athlete improve their performance or learn the sport.</figcaption></figure>

## 1 CONTEXT

With the 2024 Olympics in Paris, a lot of attention has been put on elite sports. Elite sports concern a limited set of top athletes from around the globe who have been training for many years, from nutrition to mental preparation, to achieve the best possible performance during the competition. Many researchers —including the co-authors of this paper — have been involved in helping athletes to reach their best level of performance by carrying out research in close collaboration with Olympic federations, analysts, and coaches. This led to a deeper understanding of needs, applying or developing new data capturing, analysis and presentation methods. However, this work remained only in the perimeter of high-performance sports while the benefits of the work can reach beyond elite sports to casual activities, health, or recovery.

We now want to explore possible applications that transfer from excellence to casual physical activities. We are motivated by the possible benefits of: 1) being more inclusive in terms of participants, 2) reaching a wider set of physical activities in the realm of personal fitness rather than elite-level sports, and 3) applications outside of official sports environments (e. g. a stadium), for example in people's homes or public places such as forests, public roads, etc. What we have learned about elite sports, however, will only

transfer to casual physical activities to some extent — especially when we consider that these visualizations are meant to be seen live, during an activity, and from a first-person perspective. We argue that casual sports activities could be supported with efficient visual mapping of personal data using similar techniques as seen in TV broadcasts, as well as a novel generation of visualizations that people can relate to, giving them a personalized sense of performance. Techniques can, for example, use historical sports visualizations [5] to provide context on the current pace of a race or the quality of play during table tennis tournaments.

In this workshop, we would like to specifically discuss challenges related to *where to map* visualizations and the impact of motion on this process. Of course, other challenges related to *what* to map, *when* to show visualization, and *how* to design them are equally important. They will touch on our challenge but are not our focus here. In the professional settings that we have so far focused on, third-person, spectators, or bird's eye perspectives are common because they can be used for TV broadcasts or analysis after an activity by coaches and athletes. When we switch to a first-person perspective, we will have a limited *personal perspective* (see teaser image) of a much larger environment, with perspective distortion and relative motion with both the athlete's body parts and the environment (Figure 3). Making visualizations available in these first-person views presents numerous technical challenges, which we discuss next.

## 2 MAPPING THE FIRST-PERSON VIEW AND ENVIRONMENT

When mapping visualizations into a first-person perspective our work so far has considered three mapping environments:

<footnote>*e-mail: petra.isenberg@inria.fr</footnote>

<footnote>†e-mail: lionel.reveret@inria.fr</footnote>

<footnote>‡e-mail: romain.vuillemot@ec-lyon.fr</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 2: Types of points of view (from [1]): A) overhead/bird's eye, B) first-person, C) third-person-high, D) third-person-low</figcaption></figure>

**Surrounding the physical activity:** The surroundings of a physical activity can be both static or dynamic. The surrounding environment is most often static and includes entities such as roads, a court (tennis/badminton/beach volleyball/swimming pool, ...), or fixed game equipment such as a table tennis table. However, environments may also dynamically change (Figure 3, left) and the change may be essential to the activity. For example imagine that we may want to add visualizations to waves for an amateur surfer or on the sand of a beach volleyball court. In this case, visualizations become more difficult to map.

**Parts of the activity:** We may also want to map visualizations to entities that are essential aspects of the sports activity. These entities include the athletes themselves and their bodies, various props (e. g., a racket, a goal, or a punching bag), or sports accessories (e. g., a ball). Visualizations that are mapped closely to entities that are part of the activity usually have a particularly tight embedding between the data and the referent [6].

**Overlays from the participant's point of view:** Overlays are somewhere between visualizations surrounding the activity and those that are mapped onto parts of the activity. Overlays are typically displayed in a somewhat disconnected manner, like in a heads-up-display, but still connect closely to the activity itself.

The first question we would like to discuss is how to find the right balance between these mapping spaces to prevent visual overload while providing relevant situated visualizations. For example, a ball's speed trajectory might be relevant to its location but not from a first-person point of view. What is the impact of distortion due to the point of view? For instance, compared to the third person view (Figure 2, d) which provides only a few meters ahead, an overhead view (Figure 2, a) enables more space with fewer distortions, giving up one spatial dimension (the height).

## 3 DEALING WITH MOTION

Our past work has explored how people are affected when reading visualizations under relative motion between the viewer and a visualization [2, 9, 7, 8]. Only in one project have we explored how viewers are affected by reading a visualization while walking and running [2]. We saw that people became more inaccurate reading visualizations from a smartwatch the faster they moved. Yet, we only tested speeds of up to 6 km/h, which is far below the speed of many, even casual, sports activities that require faster bursts of movement (ball games in particular). In other past work, we ran perception studies of reading visualization moving on a screen at different speeds [8]. We again found that with increasing speed, people were less accurate in reading the visualizations. Again, the first-person perspectives we discuss here, however, only par-

<figure><img src="image_4.png" /><figcaption>Figure 3: Difference between a first-person view of a pool (mockup) and our tool SwimFlow [9] meant for broadcast of swimming races.</figcaption></figure>

tially relate to this past work as the environments we target for casual sports are much more complex, seen in 3D, and with changing backgrounds and lighting conditions. When visualizations are embedded static in the environment, the movement of the visualizations will depend on the athletes themselves and their changing viewpoints. When visualizations are dynamic, for example, when attached to a ball or another athlete, there will be two types of motion: the autonomous, semi-predictable motion of the ball, and the controlled motion of the athletes themselves.

The second question we would like to discuss is related to motions from a first-person perspective. How will it impact people's ability to read visualizations? Should the kind of mapping of visualizations in the 3D space be based on the motion characteristics during the activities? For instance, running at high speeds is known to cause a loss of peripheral vision, which reduces focus and attention to the surrounding environment.

## 4 TECHNICAL CHALLENGES OF MOTION TRACKING AND EMBEDDING VISUALIZATIONS ON A MOVER

Generally, there are three technical solutions to displaying visualizations for casual sports activities: using displays, projecting visualizations in the environment, or wearing AR headsets that give an entirely personalized and private view.

It is much simpler, although not necessarily easy, to display visualizations statically in a fixed environment surrounding a sports activity — similar to how in soccer the sides of the field are already used to display consumer information. For example, fixed displays could be placed in the environment and fed with personal information related to an athlete passing by; or the displays could be placed in the typical view direction of an athlete, for example somewhere on the opposite side of a tennis field. When projectors or AR equipment are to be used, we need to have a physical model of the environment to correctly place visualizations in them. This could be achieved by fitting well-known 3D shapes or using recent advances in computer vision and sensor technologies.

When visualizations can be physically attached to sports equipment, they can relatively easily support showing data related to parts of an environment. For example, imagine a ball that changes color based on how hard it has been hit. Physical displays are already often worn by athletes in the form of smartwatches although physical displays can also be embedded in clothing, for example. However, displays attached to people or equipment might not always be visible, and for an athlete, it might be dangerous to look away to see them.

For rendering visualization in a reconstructed 3D environment (via projectors or an AR headset), it is difficult to equip participants with embedded cameras. In elite sports, we can sometimes see first-person views as 3D reconstructions of activities recorded from a third-person perspective; sometimes achieved with spidercam cameras that descend close to the player (e. g., before shooting a penalty). Depending on the activity, multiple cameras may be required, and these are typically reserved for major events (e. g.,
finals of highly popular sports). At home or in other smaller environments, it might eventually be possible to find low-cost alternatives to those expensive setups, using a set of cameras as a complement to the first person to provide an overview or just a context of the current position and orientation. In other environments, drones could be used to follow runners for instance. But static panoramic cameras would also be a good complement, in particular, to collect tracking data e. g., using marker-less pose estimation [4].

Rendering, is, however, only one part of mapping visualizations to an environment. We also need data and capturing data may be challenging. While some wearable capturing devices exist [3], like smartwatches, sometimes even small sensors of a couple of mg are discarded by athletes due to the imbalance it creates. Capturing accurate data in open environments is also challenging due to the weather and light variations, for example, when races can happen during night time.

The third question we would like to discuss is: what are the first-person tracking devices suitable for casual physical activities? In particular, what are the trade-offs between wearable devices and those installed in the environment? Our take is that video recording could play an important role in capturing real-time motion and enabling replay for self-reflection or communication. However, providing real-time feedback remains challenging, especially when the devices are embedded in props or body parts that move quickly.

## ACKNOWLEDGMENTS

This work is part of the SPORTSVIZ grant from the French ANR research agency, grant number: ANR-24-CE33-4945.

## REFERENCES

[1] S. Bateman, A. Doucette, R. Xiao, C. Gutwin, R. L. Mandryk, and A. Cockburn. Effects of view, input device, and track width on video game driving. In *Proceedings of Graphics Interface (GI)*, vol. 2011, pp. 207–214. Canadian Human-Computer Communications Society, 2011. doi: 10.5555/1992917.1992952 2

[2] F. Grioui, T. Blascheck, L. Yao, and P. Isenberg. Micro Visualizations on a Smartwatch: Assessing Reading Performance While Walking. In *Short Papers of the IEEE Conference on Visualization and Visual Analytics (VIS)*, 2024. To appear. 2

[3] A. Islam, L. Yao, A. Bezerianos, T. Blascheck, T. He, B. Lee, R. Vuillemot, and P. Isenberg. Reflections on Visualization in Motion for Fitness Trackers. In *MobileHCI Workshop on New Trends in HCI and Sports*. Vancouver, Canada, Sept. 2022. 3

[4] D. Pagnon, M. Domalain, and L. Reveret. Pose2Sim: An open-source Python package for multiview markerless kinematics. *Journal of Open Source Software*, 7(77):4362, 2022. Publisher: The Open Journal. doi: 10.21105/joss.04362 3

[5] C. Perin, R. Vuillemot, C. D. Stolper, J. T. Stasko, J. Wood, and S. Carpendale. State of the Art of Sports Data Visualization. *Computer Graphics Forum (EuroVis'18)*, 37(3):663–686, 2018. doi: 10.1111/cgf.13447 1

[6] W. Willett, Y. Jansen, and P. Dragicevic. Embedded data representations. *IEEE Transactions on Visualization and Computer Graphics*, 23(1):461–470, 01 2016. doi: 10.1109/TVCG.2016.2598608 2

[7] L. Yao, A. Bezerianos, R. Vuillemot, and P. Isenberg. Situated Visualization in Motion for Swimming. In *Journée Visu 2022*. Bordeaux, France, June 2022. 2

[8] L. Yao, A. Bezerianos, R. Vuillemot, and P. Isenberg. Visualization in Motion: A Research Agenda and Two Evaluations. *IEEE Transactions on Visualization and Computer Graphics*, 28(10):3546–3562, Oct. 2022. doi: 10.1109/TVCG.2022.3184993 2

[9] L. Yao, R. Vuillemot, A. Bezerianos, and P. Isenberg. Designing for Visualization in Motion: Embedding Visualizations in Swimming Videos. *IEEE Transactions on Visualization and Computer Graphics*, 30(3):1821–1836, Mar. 2024. doi: 10.1109/tvcg.2023.3341990 2