# Virtualization of Digital Location-Based Experiences

Jannicke Baalsrud Hauge, Heinrich Söbke, Ioana A. Stefan

► To cite this version:

Jannicke Baalsrud Hauge, Heinrich Söbke, Ioana A. Stefan. Virtualization of Digital Location-Based Experiences. 20th International Conference on Entertainment Computing (ICEC), Nov 2021, Coimbra, Portugal. pp.467-474, 10.1007/978-3-030-89394-1_40. hal-04144385

HAL Id: hal-04144385

https://inria.hal.science/hal-04144385

Submitted on 28 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Virtualization of Digital Location-based Experiences

Jannicke Baalsrud Hauge¹, ² [0000-0002-3747-0845], Heinrich Söbke³[0000-0002-0105-3126], and
Ioana A. Stefan⁴[0000-0003-3371-9313]

¹ BIBA – Bremer Institut für Produktion und Logistik GmbH,
Hochschulring 20, 28359 Bremen, Germany
baa@biba.uni-bremen.de

² Royal Institute of Technology, Kvarnbergagatan 12, Södertälje, Sweden
jmbh@kth.se

³ Bauhaus-Universität Weimar, Bauhaus-Institute for Infrastructure Solutions (b.is),
Goetheplatz 7/8, 99423 Weimar, Germany
heinrich.soebke@uni-weimar.de

⁴ Advanced Technology Systems, Str. Tineretului Nr 1., 130029 Targoviste, Romania
ioana.stefan@ats.com.ro

**Abstract.** Digital location-based experiences, such as app-guided digital scavengers, depend on users being on-site. The requirement to be on-site is a virtue, but it can also be a shortcoming of location-based experiences, such as in the COVID 19 pandemic. This workshop addresses the possibilities of eliminating the on-site requirement, thus extending the potential application of location-based experiences. First, virtualization approaches will be presented, which allow eliminating the need to be on-site, such as the usage of 360° technologies or the employment of videoconferencing software. Further, various challenges, such as the application of existing design principles for location-based experiences and instructional design for virtualized location-based experiences, are discussed and options to overcome the limitations imposed by the virtualization. This paper provides insights into theoretical foundations relevant to solving the aspects of virtualizing location-based experiences. The workshop results are intended to provide lasting advancements to eliminating the on-site requirements for location-based experiences.

**Keywords:** location-based; serious games; 360° technology; virtual field trips; smart city

## 1 Introduction

Digital location-based experiences (LBEs), in the scope of this paper, are the place-based extension of reality with the help of digital technologies. The extension can refer to the digital generation of optical sensory stimuli, for example, and the provision of interaction options depending on the respective position of the user contributes to digital LBEs. A prominent group of digital LBEs are location-based augmented reality (AR) games, which have received increasing attention in the recent decade. Examples of these AR games include INGRESS [1], Pokémon GO [2], Minecraft Earth [3], and
Harry Potter: Wizards Unite [4]. Yet apps that support geo-caching [5] and scavenger hunt activities [6] also enable digital LBEs. In this paper, LBEs are considered in a broader sense, and not limited to gaming LBEs.

There is a multitude of positive implications of digital LBEs. Across location-based games, Laato et al. [7] analyze corresponding game mechanics and potential effects. In another work [8], Laato et al. observe an association between playing a location-based game and players' psychological well-being. Also, location-based games are used as a tool in Citizen Science initiatives [9, 10]. Furthermore, LBEs are part of a smart city, increasing quality of life and more efficient use of resources [11]. Important potential use of LBEs is also evident in learning contexts [12–15], not only because a wide variety of learning approaches are supported, such as inquiry-based learning, project-based learning, or situational learning [16].

Despite all benefits, however, the characteristic of being tied to a specific location also has a limiting effect. For example, Minecraft Earth has been discontinued at the end of June 2021 [17], among other reasons due to the COVID 19 pandemia caused the number of users to fall short of expectations [18]. Especially in learning contexts, it may be too costly and time-consuming to be on-site physically, so the LBE might not be conducted at all [19]. On these grounds, it may be beneficial to de-couple LBEs from the need to be on-site. This is the point this workshop addresses. The workshop aims to present approaches to virtualizing LBEs, e.g., using 360° technologies or video conferencing systems. Virtual field trips (VFTs) [20] might be considered as a subgenre of virtualized LBEs. Overall, the workshop intends to contribute to making the tool LBE more versatile by lowering the on-site requirement. The workshop can be seen as a continuation of previous workshops at IFIP ICEC in Vienna, Austria in 2016 about pervasive gamified learning [21], in Arequipa, Peru in 2019 about designing serious location-based games [22], and in Xi'an, China in 2020 about facilitating serious location-based games [23]. The research questions that will guide the workshop include the following:

RQ 1: What are the technical options for freeing LBEs from the need to be on-site, i.e., for digitizing LBEs?

RQ 2: In how far may existing frameworks for designing LBEs also be applied to virtualized location-based experiences?

RQ 3: How may the limitations caused by the virtualization of LBEs be compensated?

In the following Section, two, foundations for addressing these research questions are presented. After that, in Section three, limitations are discussed and conclusions drawn in Section four.

# 2 Location-based experiences in the light of virtualization

## 2.1 Technical and organizational options for virtualization of LBE

Virtualization of LBEs can be achieved through various methods, such that a reality-virtuality continuum [24] might also apply to LBEs. Methods that lead to differentially
latitudinal positioning of an LBE on the reality-virtuality continuum towards the
virtuality pole include:

* **Modification of interaction mechanics.** Interaction mechanics in LBEs that require on-site presence might be redesigned to eliminate the need for on-site presence, e.g., remote trading in Pokémon GO [25]. While this does not eliminate the need for on-site presence for users, it may significantly reduce the intensity of on-site presence.

* **Location spoofing.** GPS spoofing allows location-based apps to simulate being at a specific location [26]. While this technique is perceived as cheating in various games [27], it may help in virtualizing LBEs: Users remain in a fixed location yet interact with apps as if they were on-site. This method might need to complement other methods if the LBE requires sensations that can only be obtained on-site, such as the current temperature on-site.

* **Replicating real worlds.** The creation of virtual worlds used for LBEs is one of the methods that may lead to completely virtualized LBEs. There are various technical measures for creating virtual worlds. 360° video technology allows the creation of worlds close to reality but may not be completely navigable, with comparatively little effort. Geometric modelling of virtual worlds is more complex but will enable users more positioning freedom. As alternative options adding spatial information to 360° models, the use of laser scanners and the creation of geometric models by photogrammetry may be considered. Additional information, which would otherwise be given via location-based apps, must be integrated into the worlds created. It is also conceivable to implement further interactions, such as talking to people simulated by chatbots.

Interaction mechanics in LBEs may be differentiated according to the different
interaction target groups: the users of LBEs may interact with (a) the location-based
app, (b) additional co-users and (c) the real world. For each of these interaction target
groups, there is a demand for virtualization: the location-based app (a) may be directly
integrated into the virtual world; for example, by replicating the app controls, co-users
(b) may be supported either in virtual worlds providing multi-user features or manually
via screen sharing in a video conferencing software. Interactions must replace
interactions with the real world (c) with the corresponding virtual world.

## 2.2 Design principles for LBE virtualization

There are several design frameworks for LBEs whose applicability for virtualized LBEs still needs to be reviewed [28–30]. As an example, the aspects of the Pervasive Game Design Framework (PGDF) [21] will be discussed in the following regarding possible interference with virtualization:

* **Pervasive Context.** A pervasive game should reflect a pervasive context. In terms of LBEs, this seems to be a challenging requirement: the location-based context has to be represented virtually, at least in the LBE-relevant aspects.
* **Pedagogical Objective.** The learning goals of an educational LBE may be impacted by virtualization. It has to be investigated which learning goals are no longer achieved or by what adaptations of the LBE the learning goals become achievable.

* **Assessment Metrics.** Virtualising LBEs might also eliminate assessment metrics (e.g., distance walked) and might need to be replaced by other metrics (e.g., virtual distance walked). It may also be necessary to adjust the assessment metrics, e.g., the time users spend in an LBE could be reduced by the walking distance saved that could be completed faster virtually.

* **Difficulty Level.** The difficulty level could be affected by virtualization since not all information might be collectable in a virtualized LBE in the same way as in reality. On the other hand, information can also be made more accessible through virtualization.

* **User Skills.** The skills of the users of the LBE are not dependent on whether LBEs are on-site or virtual. However, the virtualization of LBEs could provide better accessibility for impaired users.

* **Social Interaction.** Ensuring social interactions in virtual LBES appears to be a significant challenge as many virtual environments are not designed for multi-user operation. There is also a lack of uninvolved externals, which are crucial for some LBEs, e.g., as described in [31].

* **Motivation.** Although there is evidence that LBEs cause high motivation in their users [19], it remains likely that there are different preferences for virtualized or real LBEs depending on the user profile. If appropriate, virtualized LBEs must be explicitly designed for optimizing motivation.

This non-exhaustive discussion suggests that the virtualization of LBEs is not an automated process but that further design decisions deserve careful consideration. Suppose LBEs are designed for learning, for example. In that case, the cognitive theories of multimedia learning [32], the Cognitive Load Theory [33], and the Cognitive Affective Model of Immersive Learning [34], which builds on them, are essential theoretical frameworks that are to be accounted for in the design of virtualized LBEs. For example, Petersen et al. [35] present a VFT and its instructional design-oriented integration into an educational virtual LBE. Yepes-Serna et al. [36] discuss principles relevant for virtualizing learning scenarios in general.

## 2.3 Overcoming Limitations of Virtualized LBEs

Shifting elements of the LBE from reality to virtuality may change or limit the LBE. In such cases, it becomes necessary to investigate to what extent organizational or technical changes may compensate for the limitations. In the following, some examples are presented:

* Due to the COVID pandemic, the game mechanics of Pokémon GO were changed, among others, to limit player contacts [37]. This led to considerations of how to change the game mechanics in a manner that would affect the original game experience as little as possible [25]
* In virtualized LBEs, recognizing the characteristics of reality that are important to the LBE may become more complicated, among other things. For example, in a learning scenario for planning sewage infrastructure, it is essential to recognize the slope of the terrain. However, this is more challenging in 360° models, to begin with [19]. Coloured markings in the 360° models may provide a remedy for these difficulties.

* Slingerland et al. [31] present an LBE in which interactions are an essential part of the game, both with other users and with strangers on the street. Special arrangements need to be made for the potential virtualization of this LBE, such as how strangers on the street can continue to be part of the game. Chatbots would undoubtedly be an approach to be validated.

## 3 Limitations

The LBEs and their virtualizations considered include neighbouring manifestations that will not be considered as much in this workshop. These manifestations include pervasive games [38], where absolute positioning in the real world might be not an integral part of the game but rather a relative positioning to other players. Similarly, there are intersections with mixed reality sports platforms [39], though here, too, absolute positioning in the real world is less important. Likewise, not covered are the usages of LBEs to serve human health, especially in times of the COVID pandemic [40]. Not included in this workshop are the affordances of using LBEs to manage the player population in pandemics [41].

## 4 Conclusions

Location-based experiences (LBEs) have characteristics that turn them into successful learning tools, such as supporting exploratory learning. Virtualizing LBEs, i.e., eliminating the need to be on-site, increases the variability of the possible uses of LBEs. The COVID pandemic or travel constraints due to cost and time eliminate the need to be on-site. This article, which is the basis of a workshop on virtualizing LBEs, technical options for virtualization, the applicability of design frameworks, and mitigating limitations of virtualized LBEs compared to LBEs, were presented as relevant fundamentals of virtualizing LBEs. It is to be assumed that the virtualization of LBEs requires a non-negligible effort. The workshop intends to contribute to a topic that has so far been insufficiently covered in the literature.

## 5 Acknowledgement

This work has been partly funded by the German Federal Ministry of Education and Research (BMBF) through the projects DigiLab4U (No. 16DHB2113) and AuCity2 (No. 16DHB2131)
6 References

1. Niantic Labs: Ingress, http://www.ingress.com/.

2. Niantic Inc.: Pokémon GO, http://www.pokemongo.com/.

3. Mojang Studios: Minecraft Earth, https://www.minecraf t.net/en-us/about-earth/.

4. Warner Bros. Entertainment Inc.: Harry Potter: Wizards Unite, https://www.harrypotterwizardsunite.com.

5. Fornasini, S., Dianti, M., Bacchiega, A., Forti, S., Conforti, D.: Using geocaching to promote active aging: Qualitative study. J. Med. Internet Res. 22, (2020). https://doi.org/10.2196/15339.

6. Hutzler, A., Wagner, R., Pirker, J., Gütl, C.: Mythhunter: Gamification in an educational location-based scavenger hunt. Commun. Comput. Inf. Sci. 725, 155-169 (2017). https://doi.org/10.1007/978-3-319-60633-0_13.

7. Laato, S., Pietarinen, T., Rauti, S., Sutinen, E.: Potential Benefits of Playing Location-Based Games: An Analysis of Game Mechanics. In: Lane, H.C., Zvacek, S., and Uhomoibhi, J. (eds.) Computer Supported Education. CSEDU 2019. pp. 557-581. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-58459-7.

8. Laato, S., Islam, A.K.M.N., Laine, T.H.: Playing location-based games is associated with psychological well-being: an empirical study of Pokémon GO players. Behav. Inf. Technol. 0, 1-17 (2021). https://doi.org/10.1080/0144929X.2021.1905878.

9. Celino, I.: Location-Based Games for Citizen Computation. In: Michelucci, P. (ed.) Handbook of Human Computation. pp. 297-316. Springer New York, New York, NY (2013). https://doi.org/10.1007/978-1-4614-8806-4_25.

10. Loureiro, P., Prandi, C., Nunes, N., Nisi, V.: Citizen Science and Game with a Purpose to Foster Biodiversity Awareness and Bioacoustic Data Validation. In: Brooks, A.L., Brooks, E., and Sylla, C. (eds.) Interactivity, Game Creation, Design, Learning, and Innovation. pp. 245-255. Springer International Publishing, Cham (2019).

11. Nijholt, A.: Towards Playful and Playable Cities. In: Nijholt, A. (ed.) Playable Cities: The City as a Digital Playground. pp. 1-20. Springer Singapore, Singapore (2017). https://doi.org/10.1007/978-981-10-1962-3_1.

12. Schneider, J., Schaal, S., Schlieder, C.: Integrating simulation tasks into an outdoor location-based game flow. Multimed. Tools Appl. (2019). https://doi.org/10.1007/s11042-019-07931-4.

13. Schaal, S.: Location-based games for geography and environmental education. In: Walshe, N. and Healy, G. (eds.) Geography Education in the Digital World: Linking Theory and Practice. p. 54. Routledge (2020).

14. Wolf, M., Wehking, F., Söbke, H., Londong, J.: Location-based Apps in Environmental Engineering Higher Education: A Case Study in Technical Infrastructure Planning. In: Söbke, H., Baalsrud Hauge, J., Wolf, M., and Wehking, F. (eds.) Proceedings of DELbA 2020 - Workshop on Designing and Facilitating Educational Location-based Applications co-located with the
Fifteenth European Conference on Technology Enhanced Learning (EC-TEL 2020) Heidelberg, Germany, Online, September 15, 2020. CEUR Workshop Proceedings (2020).

15. Avouris, N.M., Yiannoutsou, N.: A review of mobile location-based games for learning across physical and virtual spaces. J. UCS. 18, 2120–2142 (2012).

16. Zydney, J.M., Warner, Z.: Mobile apps for science learning: Review of research. Comput. Educ. 94, 1–17 (2016).
https://doi.org/10.1016/j.compedu.2015.11.001.

17. Mojang Studios: Minecraft Earth coming to an end, https://www.minecraft.net/en-us/article/minecraft-earth-coming-end.

18. Kumbarak, G.: Minecraft Earth will shut down in June, https://techcrunch.com/2021/01/05/minecraft-earth-will-shut-down-in-jun.

19. Springer, C., Wehking, F., Wolf, M., Söbke, H.: Virtualization of Virtual Field Trips: A Case Study from Higher Education in Environmental Engineering. In: Söbke, H., Baalsrud Hauge, J., Wolf, M., and Wehking, F. (eds.) Proceedings of DELbA 2020 - Workshop on Designing and Facilitating Educational Location-based Applications co-located with the Fifteenth European Conference on Technology Enhanced Learning (EC-TEL 2020) Heidelberg, Germany, Online, September 15, 2020. CEUR Workshop Proceedings (2020).

20. Tuthill, G., Klemm, E.B.: Virtual field trips: Alternatives to actual field trips. Int. J. Instr. Media. 29, 453 (2002).

21. Baalsrud Hauge, J., Stanescu, I.A., Stefan, A.: Constructing and Experimenting Pervasive, Gamified Learning. In: Entertainment Computing – ICEC 2016 15th IFIP TC 14 International Conference Vienna, Austria, September 28–30, 2016 (2016).

22. Baalsrud Hauge, J., Söbke, H., Stefan, I.A., Stefan, A.: Designing Serious Mobile Location-Based Games. In: van der Spek, E., Göbel, S., Do, E.Y.-L., Clua, E., and Baalsrud Hauge, J. (eds.) Entertainment Computing and Serious Games - ICEC -JCSG 2019 LNCS 11863. pp. 479–484. Springer International Publishing, Cham (2019).

23. Baalsrud Hauge, J., Söbke, H., Stefan, I.A., Stefan, A.: Applying and Facilitating Serious Location-based Games. In: Nunes, N.J., Ma, L., Wang, M., Correia, N., and Pan, Z. (eds.) Entertainment Computing – ICEC 2020 19th IFIP TC 14 International Conference, ICEC 2020, Xi’an, China, November 10–13, 2020, Proceedings. Springer International Publishing (2020).
https://doi.org/10.1007/978-3-030-65736-9.

24. Milgram, P., Takemura, H., Utsumi, A., Kishino, F., Fumio, K.: Augmented reality: a class of displays on the reality-virtuality continuum. In: SPIE Proceedings Volume 2351: Telemanipulator and Telepresence Technologies. pp. 282–292 (1994). https://doi.org/10.1.1.83.6861.

25. GamePress: It’s Time for Niantic to Allow Remote Trading in Pokemon GO.

26. Larcom, J.A., Liu, H.: Modeling and characterization of GPS spoofing. In: 2013 IEEE International Conference on Technologies for Homeland Security (HST). pp. 729–734 (2013). https://doi.org/10.1109/THS.2013.6699094.

27. Paay, J., Kjeldskov, J., Internicola, D., Thomasen, M.: Motivations and
practices for cheating in Pokémon Go. MobileHCI 2018 - Beyond Mob. Next 20 Years - 20th Int. Conf. Human-Computer Interact. with Mob. Devices Serv. Conf. Proc. 1-13 (2018). https://doi.org/10.1145/3229434.3229466.

28. Rauschnabel, P.A., Rossmann, A., tom Dieck, M.C.: An adoption framework for mobile augmented reality games: The case of Pokémon Go. Comput. Human Behav. 76, 276-286 (2017). https://doi.org/10.1016/j.chb.2017.07.030.

29. Hamari, J., Malik, A., Koski, J., Johri, A.: Uses and Gratifications of Pokémon Go: Why do People Play Mobile Location-Based Augmented Reality Games? Int. J. Hum. Comput. Interact. 00, 1-16 (2018). https://doi.org/10.1080/10447318.2018.1497115.

30. Söbke, H., Baalsrud Hauge, J., Stefan, I.A.: Prime Example Ingress: Reframing the Pervasive Game Design Framework (PGDF). Int. J. Serious Games. 4, 39-58 (2017). https://doi.org/10.17083/ijsg.v4i2.182.

31. Slingerland, G., Fonseca, X., Lukosch, S., Brazier, F.: Location-based challenges for playful neighbourhood exploration. Behav. Inf. Technol. 0, 1-19 (2020). https://doi.org/10.1080/0144929X.2020.1829707.

32. Mayer, R.E.: Cognitive Theory of Multimedia Learning. In: Cambridge Handbook of Multimedia Learning. pp. 31-48 (2005).

33. Sweller, J., Ayres, P., Kalyuga, S.: Cognitive Load Theory. Springer, New York, NY, USA (2011). https://doi.org/10.1007/978-1-4419-8126-4.

34. Makransky, G., Petersen, G.B.: The Cognitive Affective Model of Immersive Learning (CAMIL): a Theoretical Research-Based Model of Learning in Immersive Virtual Reality. Educ. Psychol. Rev. (2021). https://doi.org/10.1007/s10648-020-09586-2.

35. Petersen, G.B., Klingenberg, S., Mayer, R.E., Makransky, G.: The virtual field trip: Investigating how to optimize immersive virtual learning in climate change education. Br. J. Educ. Technol. 51, 2098-2114 (2020). https://doi.org/10.1111/bjet.12991.

36. Yepes-Serna, V., Wolf, M., Söbke, H., Zander, S.: Design Principles for Educational Mixed Reality?: Adaptions of the Design Recommendations of Multimedia Learning. In: Akcayir, G. and Demmans Epp, C. (eds.) Designing, Deploying, and Evaluating Virtual and Augmented Reality in Education. pp. 76-99. IGI Global (2020).

37. Laato, S., Laine, T.H., Najmul Islam, A.K.M.: Location-based games and the COVID-19 pandemic: An analysis of responses from game developers and players. Multimodal Technol. Interact. 4, 1-25 (2020). https://doi.org/10.3390/mti4020029.

38. Oppermann, L., Slussareff, M.: Pervasive Games. In: Dörner, R., Göbel, S., Rust-Kickmeier, M.D., Masuch, M., and Zweig, K. (eds.) Entertainment Computing and Serious Games. pp. 475-520. Springer International Publishing (2016). https://doi.org/10.1007/978-3-319-46152-6_18.

39. Westmattelmann, D., Grotenhermen, J.-G., Sprenger, M., Rand, W., Schewe, G.: Apart we ride together: The motivations behind users of mixed-reality sports. J. Bus. Res. 134, 316-328 (2021). https://doi.org/10.1016/j.jbusres.2021.05.044.
40. Ellis, L.A., Lee, M.D., Ijaz, K., Smith, J., Braithwaite, J., Yin, K.: COVID-19 as ‘game changer’ for the physical activity and mental well-being of augmented reality game players during the pandemic: Mixed methods survey study. J. Med. Internet Res. 22, (2020). https://doi.org/10.2196/25117.

41. Laato, S., Islam, A.K.M.N., Laine, T.H.: Did location-based games motivate players to socialize during COVID-19? Telemat. Informatics. 54, 101458 (2020). https://doi.org/10.1016/j.tele.2020.101458.