# Field of View Limitation-Driven Design of a Mixed Reality Game for Heritage Sites

Giuseppe Caggianese, Luigi Gallo

► To cite this version:

Giuseppe Caggianese, Luigi Gallo. Field of View Limitation-Driven Design of a Mixed Reality Game for Heritage Sites. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.426-430, 10.1007/978-3-030-85607-6_51. hal-04291254

HAL Id: hal-04291254

https://inria.hal.science/hal-04291254v1

Submitted on 17 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Field of view limitation-driven Design of a
# Mixed Reality Game for Heritage Sites *

Giuseppe Caggianese¹[0000-0001-6607-6591] and
Luigi Gallo¹[0000-0002-1281-404X]

Institute for High Performance Computing and Networking,
National Research Council of Italy, Naples, Italy
{giuseppe.caggianese, luigi.gallo}@icar.cnr.it

**Abstract.** In this work, we describe the design of a customized user interface (UI) for a mixed reality application in a heritage site. The visuals, widgets and spatial interaction techniques have been customized to improve the user experience without diverting the user's attention by minimizing the effect of the limited field of view (FOV) of the see-through head-mounted display (HMD) used, Microsoft HoloLens v1. The approach consists in using diegetic elements that are coherent with the narrative of the heritage site, and widgets and augmented layers always entirely included in the FOV of the see-through display.

**Keywords:** mixed reality · user interface design · limited FOV.

## 1 Introduction and Background

Emerging Mixed Reality (MR) technologies, which aim at a symbiotic blending of the real world with the virtual world [7], present many exciting opportunities for the cultural heritage (CH) sector. Such a merging of the real and the virtual, combined with the opportunity to realize immersive and interactive experiences, allows a creative enhancement, extension and dissemination of cultural content. In this context, curators often want to provide visitors to the physical exhibition spaces with a new way to engage with their collection. MR technologies, based on innovative interaction paradigms, could enrich these spaces as shown by the studies of Chrysanthi et al. [3], Okura et al. [8] and Brancati et al. [2] In addition, serious games could improve the knowledge transfer in the CH, since they offer the possibility of realizing cultural content personalized by age group and of enhancing motivation towards acquiring new knowledge about the heritage site [5, 6].

This paper describes some of the compelling opportunities and challenges involved in realizing a MR game to improve knowledge transfer in the CH. In particular, certain strategies aimed at mitigating the adverse effects associated with the narrow FOV of most commonly used see-through HMDs are proposed

<footnote>* Supported by the project Parco Archeologico Urbano di Napoli (RIPA-PAUN), POR Campania FESR 2014/2020.</footnote>
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** (left to right, top to bottom) Used HMD's narrow FOV able to show only a portion of the scene which for this reason was hidden under the physical floor; the sphere widget, fully visible in the FOV of the device, that collides the floor opening a breach over the archaeological excavations; manipulation of the collected finding, the text shows the finding name *Puleggia di bozzello* and the question *Do you think it could belong to the ship?*; interactive panels for the presentation of cultural information related to the artifacts collected.</figcaption></figure>

and discussed. We have chosen to use Microsoft HoloLens v1 which, according to Hammady et al. [4] exhibits a FOV of 34 degrees that must be considered in the design of a spatial UI. This work aims at leveraging custom design strategies to improve the user's sense of presence, reducing the effect of the FOV limitation, regardless of the HMD used.

## 2 MR Game Overview

The MR game recreates the setting of the archaeological excavations of the ancient Roman port of Naples, where archaeologists have found the artifacts displayed in the exhibition. The exhibition includes objects of daily life and equipment of the ship to maneuver the sails. 3D digitized scans of some of these artifacts have been used in the game together with reconstruction data from one of the Roman ships found nearby [1].

The game is a location-aware application in which the synthetic scene has been incorporated into the surrounding environment and the cultural information proposed has been organized in diegetic elements adaptive to the real environment, the user's position, and the achievement of the game objectives.
The game's mission is to spatially explore the augmented archaeological site to identify and group together all the digital reproductions of the artifacts displayed, so as to create a personal collection of the individual findings to enhance an understanding of their functions and uses in the past time period. The virtual environment designed to augment the physical one has been realized with a double functionality, aimed at mitigating the limited FOV of the visor and at introducing engaging game strategies. The digital reconstruction of the archaeological excavation (see figure 1 on top left) is hidden from the user because it is situated under the line of the floor. The user can explore the digital environment, one section at a time, by manipulating a widget, a digital sphere that, when placed in contact with the physical floor, opens a breach through which it is possible to gain a glimpse of the reconstruction of the underlying archaeological excavation (see figure 1 on top right). Through the sphere widget, the visitor is also able to locate and group together reconstructions of the findings. Each time one of the objects is found by the user, it can be acquired and placed in the collection. However, before being placed in the collection, the artifact needs to be catalogued as belonging, or not, to the shipwreck that can be glimpsed at the base of the excavation reconstruction (see figure 1 on bottom left).

Finally, for any of the collected artifacts, it is possible to request additional information at a virtual desk. The desk also shows the search progress (e.g., how many objects have been found), the cataloguing work already carried out correctly during the game and, among those already found, the objects that belong to the ship (see figure 1 on bottom right).

## 3 Interaction Techniques Design andWidgets

The user can use the gaze to point to an element in the scene and the air tap gesture to perform the click and to grab and move the interactive elements. The most compelling aspect of the design phase involved identifying an interaction mechanism that could combine gaming mechanisms with the need to mitigate the limited FOV offered by the device employed. As mentioned, the game settings are initially hidden under the physical floor, and the user has to interact with a sphere widget provided to explore them. The designed interaction technique binds the vision to the virtual object's position. The sphere becomes a manipulable tool used to access cultural information. The advantage is that, when the user manipulates this tool, she/he naturally places it at the center of his FOV, without perceiving its limitations in space. The user can move the sphere in the physical environment until it collides with the part of the floor where she/he wants to open a breach. At that point, along the section of the sphere that intersects with the floor, the scene below will become visible and the user can start exploring it. To interact with the sphere and move it along the physical environment, the user should point at it with her/his gaze and maintain the air tap gesture to simulate a grab of the object. Once grabbed, the sphere will follow the user's hand movements. The sphere pointed at with the gaze can be enlarged or reduced by increasing or decreasing, respectively, the distance
between the two hands used to maintain the air tap gesture. The maximum size
of the sphere is continuously monitored to ensure that it is always completely
visible within the FOV of the MR HMD. In a preliminary evaluation of the sys-
tem, we observed that the use of the sphere widget leads the user to interact
with it also in an indirect way. Users alternate the movement of the sphere with
the variation of her/his point of view: the former action in order to start the
exploration of an area; the latter to refine the search for findings by looking in
the surrounding area.

Finally, the information submitted to the user is organized in interactive
information panels linked to the user position or anchored into the physical space.
Using panels is coherent with the methods of presenting information usually
adopted by museums. Moreover, the panels have been chosen because, thanks
to the contrast between text and background, they are easily readable, even in
the presence of cluttered backgrounds.

4 Conclusions

In this paper we have introduced a novel MR serious game for the exploration of
a heritage site. The focus has been on the design of a custom UI, based on the
HoloLens device, aimed at minimizing the limitations of the FOV of the device
itself. Empirical studies on the perceived effectiveness of the FOV-aware widgets
are currently ongoing.

References

1. Boetto, G., Poveda, P.: La restitution de napoli c. un navire romain à tableau. Dossiers d'Archéologie (364), 64-65 (2014)

2. Brancati, N., Caggianese, G., Frucci, M., Gallo, L., Neroni, P.: Experiencing touch-less interaction with augmented content on wearable head-mounted displays in cultural heritage applications. Personal and Ubiquitous Computing **21**(2), 203–217 (2017)

3. Chrysanthi, A., Papadopoulos, C., Frankland, T., Earl, G.: ‘tangible pasts’: user-centred design of a mixed reality application for cultural heritage. In: Archaeology in the Digital Era, pp. 31–39. Amsterdam University Press (2014)

4. Hammady, R., Ma, M., Strathearn, C.: User experience design for mixed reality: a case study of hololens in museum. International Journal of Technology Marketing **13**(3-4), 354–375 (2019)

5. Liarokapis, F., Petridis, P., Andrews, D., de Freitas, S.: Multimodal serious games technologies for cultural heritage. In: Mixed reality and gamification for cultural heritage, pp. 371–392. Springer (2017)

6. Malegiannaki, I.A., Daradoumis, T., Retalis, S.: Teaching cultural heritage through a narrative-based game. Journal on Computing and Cultural Heritage (JOCCH) **13**(4), 1–28 (2020)

7. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems **77**(12), 1321–1329 (1994)

8. Okura, F., Kanbara, M., Yokoya, N.: Mixed-reality world exploration using image-based rendering. Journal on Computing and Cultural Heritage (JOCCH) **8**(2), 1–26 (2015)