# Diegetic and Non-diegetic Health Interfaces in VR Shooter Games

Kay Köhle, Matthias Hoppe, Albrecht Schmidt, Ville Mäkelä

► To cite this version:

Kay Köhle, Matthias Hoppe, Albrecht Schmidt, Ville Mäkelä. Diegetic and Non-diegetic Health Interfaces in VR Shooter Games. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.3-11, 10.1007/978-3-030-85613-7_1. hal-04292375

HAL Id: hal-04292375

https://inria.hal.science/hal-04292375v1

Submitted on 17 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Diegetic and Non-Diegetic Health Interfaces in VR Shooter Games

Kay Köhle<fnref n="1" />, Matthias Hoppe<fnref n="1" />, Albrecht Schmidt<fnref n="1" />, and Ville Mäkelä<fnref n="1" />,<fnref n="2" />,<fnref n="3" />

**Abstract.** The player's *health* is one of the most pervasive components in computer games. However, in virtual reality games, it is unclear how different representations of player health function compared to traditional flat-screen games. Because the viewpoint changes based on the player's head movement, non-diegetic UI elements may not be ideal. Also, the sense of embodiment in VR provides opportunities to experiment with diegetic ways of communicating the player's health. To investigate different implementations of player health in VR games, we developed three health interfaces and evaluated them in a shooter game. The health interfaces included: 1) A non-diegetic health bar, visible on the screen at all times, 2) A diegetic health value on a virtual wristwatch, and 3) A diegetic physical interface, where lost health results in trembling and slower movement. 37 participants played the game using all three health interfaces and provided feedback. We found that all three interfaces had their own strengths. The non-diegetic health bar was seen as suitable for multi-player games, while the wristwatch was seen as suitable for single-player, story-driven games. The physical interface was liked for its impact on gameplay, and was also seen as suitable for story-driven games.

**Keywords:** Virtual Reality · Games · Diegetic Interfaces · First-Person Shooters · Game Design.

## 1 Introduction

In many computer games, players face a variety of challenges like hordes of enemies that attempt to harm the player. The player's *health* is at the center of this type of gameplay. Health is reduced, for example, when enemies hit the player, or if the player stumbles into a trap. When health is reduced to zero, the game typically ends, or other repercussions are faced.

Traditionally, player health is represented on the screen as a *non-diegetic* user interface element, like a progress bar or a numerical value. Non-diegetic elements exist outside of the story and space of the game [1, 2]. In contrast, *diegetic* elements exist in the game world (e.g., they might also be perceived by other characters). Diegetic representations of player health also exist; for

<footnote n="1">¹ LMU Munich, Munich, Germany kay.koehle@campus.lmu.de, matthias.hoppe@ifi.lmu.de, albrecht.schmidt@ifi.lmu.de, ville.maekelae@ifi.lmu.de</footnote>

<footnote n="2">² Bundeswehr University Munich, Munich, Germany</footnote>

<footnote n="3">³ University of Waterloo, Waterloo, Canada</footnote>
example, the player character may start limping when they are close to death, or they may have visible wounds. Audio cues, like shortness of breath and an elevated heartbeat, are common additional diegetic representations.

In virtual reality (VR) games, however, it is unclear how different health interfaces function. In particular, due to the sense of embodiment in VR, we can experiment with novel diegetic health interfaces that directly affect the player. To investigate different ways of communicating the player's health in VR games, we developed three health interfaces and evaluated them in a shooter game. 37 participants played the game using all three health interfaces and provided feedback. The health interfaces included: **Overlay:** A non-diegetic health bar, visible on the screen at all times, **Wristwatch:** A diegetic health value on a virtual watch, attached to the user's wrist, and **Physical:** A diegetic, movement-based health interface, where low health causes trembling and slower movement.

We found that all three interfaces have their unique strengths. Participants appreciated that the **overlay** was quickly available, stating that it is suitable for multi-player games. The **wristwatch** was appreciated for its balance of unobtrusiveness, immersion, and accuracy, and it was seen as suitable for single-player and story-driven games. The **physical** interface was appreciated for its direct impact on gameplay, and it was seen as suitable for story-driven games. The interfaces were rated equally for presence. Our work is useful for designing interfaces in VR, and for directing future work in physical interfaces in VR.

## 2 Background

Diegetic and non-diegetic interfaces have been studied in games and beyond, and both in traditional computer games as well as in VR games. Iacovides et al. [3] found that a diegetic interface was more immersive than a non-diegetic interface in a 2D first-person shooter. Raffaele et al. [6] found that players consistently rated diegetic UIs more immersive in VR games. Similar results were obtained by Salomoni et al. [9]. Diegetic cues have also been studied in other areas of VR, like guidance [8] and cultural heritage [1].

We are not aware of existing research that investigates diegetic cues in VR beyond their effect on immersion. Also, despite some research on health interfaces in flat-screen computer games [5], we are not aware of prior work that investigates health interfaces in VR. Especially, in VR we can use not only "traditional" diegetic UIs (e.g., health displayed on a virtual screen), but we can also make players *experience* the diegetic UI, as we can manipulate the way players move and interact in VR. We hypothesize that this could add to the player experience, creating, e.g., a stronger sense of danger, but also feelings of victory and triumph. With this work, we aim to close these gaps.

## 3 Study

We conducted a study where participants played a VR shooter game with three different health interfaces (Figure 1).
<figure><img src="image_3.png" /><figcaption>Fig. 1: The VR shooter game and the tested health interfaces.</figcaption></figure>

## 3.1 VR Shooter Game

We implemented a VR shooter game using Unity and the SteamVR, containing the typical gameplay elements for shooters. The game is divided into three scenes.

**Start.** In the start scene, a menu and instructions for the game and the study are displayed (Figure 1a). Two guns are hovering in front of the user, roughly at eye height, which the player can grab by touching them with their virtual hands and pressing the trigger buttons on the controllers.

The guns can then be used to shoot at one of the three buttons (by pointing and pressing the trigger) to activate one of the health interfaces. The order of the health interface buttons is randomized, but the player can choose any one of them. Upon selection, a brief explanation of the selected health interface is displayed, below which a button appears for starting the shooting round.
**Shooting Round.** The shooting round contains the main gameplay. The round lasts for four minutes, during which drones appear around the scene. The drones shoot at the player, whose task is to avoid getting hit and destroy as many drones as they can by shooting at them. All shots leave a red trail so that the player can follow where they are going. The drones die in one hit. The drones spawn in a half-circle in front of the player.

The player starts with 100 health points. Each hit damages the player by 10 points. Players recover health over time, and the recovery rate depends on the remaining health — the less health the player has, the quicker they recover. Such mechanisms are common in games, so that players can experience the thrill of barely surviving. Moreover, we did not want players to die too early, so they could properly experience each health interface.

After the four minutes are up, or the player's health has been reduced to zero, the round ends. Interactions and events in the round are supported by audio cues. This includes the round starting, the round ending, shooting, drones appearing, drones getting hit and destroyed, and the player getting hit.

**Endgame.** The endgame scene displays statistics about the shooting round, and instructions for the next steps in the study (Figure 1c). At this point in the study, the user is asked to take off the headset and fill in forms on their computer before returning to the game. The user has options to quit the game, or return to the main menu to select another health interface and play another round.

## 3.2 Health Interfaces

Below, the three health interfaces are described. All three interfaces share an identical visual and auditory cue. The edges of the screen turn red as the player gets hurt, and the strength of the effect was determined by how much health was left (seen in Figures 1d and 1f). Similarly, a heartbeat audio cue is played when the player gets hurt, which gets more intense as health decreases.

These two shared effects were added because they are commonly used in games as additional cues. Moreover, while our investigation was not focused on auditory cues, it did not make sense to leave out auditory cues entirely, as they are part of a complete gaming experience.

**Overlay.** The Overlay condition displays a traditional, two-dimensional health bar (Figure 1d). It stays visible on the screen, following the player's head movements. This health interface was included because it is a typical non-diegetic representation of player health, used in games for several decades.

**Wristwatch.** The Wristwatch condition displays health as a numerical value on the player's wrist (Figure 1e). This health interface was included because it has been used in VR shooting games (e.g., Half-Life: Alyx [11]). This health interface is diegetic, i.e., it exists in the game world, and players need to lift their arm and turn the watch towards them to see it.
**Physical.** The Physical condition is an experimental health interface where, when player's lose health, their movements become slower and twitchy, to represent the "reality" of being hurt. The player's real pointing location is displayed as a transparent gun (Figure 1f), as opposed to the current pointing as a result of slower movement. This interface was included because we wanted to experiment not only with typical diegetic and non-diegetic interfaces, but also physical interfaces that VR makes possible.

## 3.3 Study Design

The study was of within-subjects design, where participants played three rounds of the shooter game, once with each health interface. Between the rounds, participants filled in a questionnaire that enquired about their experience with each interface. The study was designed as a remote study, so that people could participate from their own homes using their own VR equipment. In addition, we set up a physical testing space, where participants could attend the study with no human contact. The study procedure was exactly the same regardless of how participants attended the study. For an in-depth description of our remote study design and procedure and a discussion on lessons learned, see Rivu et al. [7].

## 3.4 Recruitment and Participants

We advertised the study through social media and targeted forums like VR-related subreddits in Reddit, where we expected to get in contact with people who own a head-mounted display. In our advertisement mail, we provided a registration link. Registered participants then received instructions by email and a link to download the VR shooter game. In addition, we recruited local participants via university mailing lists.

A total of 37 participants completed the study (24 male, 11 female, 2 undisclosed). Their average age was 23 (SD = 6.6). Several additional participants attended but did not fully complete the study, and so their data was excluded from the analysis. 13 participants attended the local study and used the HTC Vive Pro. 24 participants attended remotely using different HMDs, the most popular being the HTC Vive Pro and the Valve Index (7 participants each), and the HTC Vive (6 participants). Most participants (27) reported playing 0–5 hours of VR games weekly; the remaining ten participants played more. Participants played other digital games more actively: 0–5 hours per week (10 participants), 5–10 hours (14), 10–15 hours (4), 15–20 hours (2), and 20+ hours (7).

## 3.5 Procedure

The VR game placed participants immediately in the Start scene, where they could review the instructions to the game and the study, and play the first round with one of the health interfaces. Once the round was over, the endgame scene was displayed, and a feedback questionnaire was automatically opened on the
default browser of their computer. At this point, the game asked participants to take off their HMD, fill in the questionnaire, and then return to the game. Participants repeated this procedure three times, which took around 40 minutes.

We chose to have participants fill in the questionnaire outside of VR because the questionnaire was long and we included open-ended questions that could not be reasonably answered in VR. The questionnaire contained the Slater-Usoh-Steed presence questionnaire [10], the Game Experience Questionnaire (GEQ) [4], and custom statements and open-ended questions. Four custom statements asked about the interface's suitability for different kinds of games (Figure 2). One additional statement asked for an **overall rating** for the interface on a 10-point scale (1 - "very bad", 10 - "very good"). The open-ended questions asked what the participant liked and disliked about the interface.

<figure><img src="image_4.png" /><figcaption>Fig. 2: Participants rated the suitability of the three health interfaces on four dimensions: single-player, multi-player, action-focused, and story-driven games. The ratings were on a 7-point scale (1 = not suitable at all, 7 = very suitable).</figcaption></figure>

# 4 Results and Discussion

In this section, we present the results and discuss implications for design for each health interface. We conducted thematic analysis on the participants qualitative feedback, and statistical analyses on the participants' subjective ratings using Friedman's test and the Wilcoxon signed-rank test. The preferences are shown in Figure 2.

There were no significant differences between the interfaces' **overall scores** ($X^2_{F}(2) = 2.17, p = .339$). The Overlay and Wristwatch received a median rating of 7/10, and the Physical received a median rating of 6/10.

With the GEQ, we tested against the seven experience components [4] separately. Only one of them, *Sensory and Imaginative Immersion*, had significant differences between the interfaces ($X^2_{F}(2) = 13.52, p = .001$): the Wristwatch was rated significantly higher than the Overlay (T = 192.50, z = -3.31, p = .001). Hence, overall the differences between the GEQ scores were minimal.

There were no significant differences between the health interfaces' *presence* scores in any of the six tested dimensions [10]. This is an interesting finding. Previous research suggests that diegetic interfaces result in a higher sense of presence [3, 6, 9], but this was not the case in our study.
## 4.1 Overlay

The most common positive traits for the non-diegetic Overlay interface were that it offers a clear way to judge the player's health status (12 participants), that it is always visible and accessible anytime (11), and that it is immediately understandable and familiar (5). At the same time, it was commonly perceived to be intrusive, i.e., in the way of the player's view (14). Moreover, participants criticized that it was not always clear when health changed as their was no clear association with gameplay (5), and that it hurt the game's immersion (5).

The Overlay was seen as significantly more suitable for multi-player games (MD = 5, Friedman: $X^2_F(2) = 13.61, p = .001$) than the Wristwatch (MD = 4, Wilcoxon: $T = 324.50, z = -2.79, p = .005$) and Physical interfaces (MD = 3, Wilcoxon: $T = 430.50, z = -4.09, p = .000$). This is likely due to the competitive nature of multi-player games, where accuracy and efficiency (i.e., that health information is available with a glance) may be favored over immersive, less efficient interfaces.

While the individual positive and negative traits of the overlay are not surprising, it is somewhat surprising that overlay-like UI elements still seem to have a useful function in VR games, as they do bring the benefits of clarity, efficiency, and familiarity. Some of the negative traits of overlays could be alleviated with further design choices. In some games, the health bar could be part of a diegetic interface, like an augmented helmet that the player's avatar is wearing. At the same time, the health bar could be made transparent, becoming opaque only when health actually changes – this would make it less intrusive and also better highlight the exact moments when the player is hit.

## 4.2 Wristwatch

The most common positive traits for the Wristwatch interface were that it offers a clear and accurate way to assess health status (10 participants), it is immersive (9), unobtrusive (8), and that it blends well into the VR action (7). For negative traits, participants commonly stated that checking health was impractical during intense action (8), and that participants had to actively choose to check their health status in order to remain aware of it (8).

The Wristwatch was seen as significantly more suitable for single-player games (MD = 5, Friedman: $X^2_F(2) = 6.54, p = .038$) than the Physical interface (MD = 5, Wilcoxon: $T = 333.00, z = -2.55, p = .011$) and the Overlay (MD = 4, Wilcoxon: $T = 328.50, z = -1.99, p = .047$). Similarly, the Wristwatch was seen as significantly more suitable for story-driven games (MD = 6, Friedman: $X^2_F(2) = 33.18, p = .000$) than the Physical interface (MD = 5, Wilcoxon: $T = 375.00, z = -3.44, p = .001$) and the Overlay (MD = 2, Wilcoxon: $T = 502.50, z = -4.48, p = .000$).

This above preferences seem logical considering the identified positive and negative traits; diegetic interfaces are perceived as more immersive [6, 9], which is likely rated as a more important trait in story-driven games than, e.g., competitive multi-player games.
Based on the participants' comments and rating, the wristwatch seemed to be the most well-rounded and the most liked health interface in our study. In the future, haptic cues could be added to the controllers whenever the player gets hurt, to make those exact moments more clear. Clearer audio cues about getting hit could also be used.

## 4.3 Physical

The most common positive traits for the Physical interface were that it impacts gameplay, i.e., there are consequences for getting hurt (7), and that it is unobtrusive (5), and immersive (4). However, most players complained that it was hard to tell how much health they actually had (23). Moreover, some participants stated that the slow movement was frustrating (4), and that it was annoying that the game got harder when they were already hurt (3).

The Physical interface was seen as significantly more suitable for story-driven games (MD = 5) than the Overlay (T = 364.00, z = -2.72, p = .006), but the Wristwatch was the most preferred. The Physical was not seen as suitable for multi-player games (MD = 3). This seems logical, as the intensity of the interface might enhance a story-driven game's story and setting, but might be not work in a competitive environment.

The clearest downside of this interface was that it does not communicate the exact health status. To overcome this, the Physical interface could be easily combined with another health interface, like the Wristwatch.

Despite the criticism towards the Physical interface, we believe we have only begun to uncover the potential of physical interfaces in VR. We already observed positive effects, e.g., its direct impact on gameplay. Investigating physical interfaces further, not only for player health but for other gameplay functions, like altered states (e.g., boosts and debuffs), is a clear direction for future work.

# 5 Conclusion and Future Work

Our work suggests that both diegetic and non-diegetic interfaces might have their place in VR games. Our three tested health interfaces were equal in terms of presence and received comparable overall and game experience ratings. Still, we uncovered unique positive and negative traits, and we also found clear differences in terms of what types of games each health interface might fit in. Non-diegetic interfaces might be useful in competitive multi-player games, where clarity and quick access to information are important factors. In turn, diegetic interfaces might fit better in single-player and story-driven games, where immersive experiences and consistency may be highly valued factors. Our work here provided initial insight into the design of health interfaces for VR; future studies should investigate diegetic and non-diegetic health interfaces more systematically. We also believe that future VR research should dive deeper into physical interfaces. Their direct and observable impact on gameplay and interaction can be utilized for many novel possibilities in VR games and beyond.
References

1. Caggianese, G., Gallo, L., Neroni, P.: Exploring the feasibility of diegetic user interfaces in immersive virtual exhibitions within the cultural heritage. In: 2018 14th International Conference on Signal-Image Technology Internet-Based Systems (SITIS). pp. 625–631 (2018). https://doi.org/10.1109/SITIS.2018.00101

2. Cecchi, A.: Diegetic versus nondiegetic: A reconsideration of the conceptual opposition as a contribution to the theory of audiovision. *Worlds of Audiovision* pp. 1–10 (2010)

3. Iacovides, I., Cox, A., Kennedy, R., Cairns, P., Jennett, C.: Removing the hud: The impact of non-diegetic game elements and expertise on player involvement. In: Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play. p. 13–22. CHI PLAY ’15, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2793107.2793120. https://doi.org/10.1145/2793107.2793120

4. IJsselsteijn, W.A., de Kort, Y.A., Poels, K.: The game experience questionnaire. Eindhoven: Technische Universiteit Eindhoven pp. 3–9 (2013)

5. Peacocke, M., Teather, R.J., Carette, J., MacKenzie, I.S., McArthur, V.: An empirical comparison of first-person shooter information displays: Huds, diegetic displays, and spatial representations. Entertainment Computing **26**, 41–58 (2018). https://doi.org/10.1016/j.entcom.2018.01.003; https://www.sciencedirect.com/science/article/pii/S1875952117300435

6. Raffaele, R., Carvalho, B., Silva, F.: Evaluation of immersive user interfaces in virtual reality first person games. 24º Encontro Português de Computação Gráfica e Interação pp. 123–126 (2017)

7. Rivu, R., Mäkelä, V., Prange, S., Rodriguez, S.D., Piening, R., Zhou, Y., Köhle, K., Pfeuffer, K., Abdelrahman, Y., Hoppe, M., Schmidt, A., Alt, F.: Remote VR studies - A framework for running virtual reality studies remotely via participant-owned hmds. CoRR abs/2102.11207 (2021), https://arxiv.org/abs/2102.11207

8. Rothe, S., Hußmann, H., Allary, M.: Diegetic cues for guiding the viewer in cinematic virtual reality. In: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. VRST ’17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3139131.3143421; https://doi.org/10.1145/3139131.3143421

9. Salomoni, P., Prandi, C., Roccetti, M., Casanova, L., Marchetti, L., Marfia, G.: Diegetic user interfaces for virtual environments with hmds: a user experience study with oculus rift. Journal on Multimodal User Interfaces **11**(2), 173–184 (2017)

10. Slater, M., McCarthy, J., Maringelli, F.: The influence of body movement on subjective presence in virtual environments. Human Factors **40**(3), 469–477 (1998). https://doi.org/10.1518/001872098779591368; https://doi.org/10.1518/001872098779591368, pMID: 9849105

11. Valve: Half-life: Alyx. Game [Windows] (2020)