<figure><img src="image_1.png" /></figure>

HAL
open science

# That Sound's Juicy! Exploring Juicy Audio Effects in Video Games

Jolie Smets, Erik Spek

► To cite this version:

Jolie Smets, Erik Spek. That Sound's Juicy! Exploring Juicy Audio Effects in Video Games. 20th International Conference on Entertainment Computing (ICEC), Nov 2021, Coimbra, Portugal. pp.319-335, 10.1007/978-3-030-89394-1_24. hal-04144377

HAL Id: hal-04144377

https://inria.hal.science/hal-04144377

Submitted on 28 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# That sound's Juicy! Exploring juicy audio effects in video games

Jolie H.K. Smets¹ and Erik D. van der Spek¹[0000-0001-8885-0373]

¹ Department of Industrial Design, Eindhoven University of Technology
Den Dolech 2, 5612AZ Eindhoven, The Netherlands
j.h.k.smets@student.tue.nl, e.d.vanderspek@tue.nl

**Abstract.** Juiciness describes exaggerated redundant audio/visual feedback in games, creating a better player experience. As computer games are principally a visual medium, sound is an underused potential for creating juiciness. This study aims to explore juicy audio. A mixed-methods approach is used to investigate the influence of juicy audio on the experience of presence in the player, and how players affectively experience and evaluate the juicy audio. Two versions of a game were created. One containing juicy audio effects, and the other without juicy audio effects. Results show a significant effect of juicy audio on presence as expressed in immersion and sensory fidelity, where participants experienced more presence in the juicy audio condition. Regarding the affective evaluation of juicy audio, three themes are identified; association & expectation, pragmatic quality, and describing sounds. The latter is an interesting direction for future research, as we appear to lack a shared, intuitive vocabulary for game sounds.

**Keywords:** Juiciness, game feel, game design, sound design, game audio, audio effects.

## 1 Introduction

Video games as an entertainment medium have become so popular that, at the time of writing, its total industry revenue dwarves the combined global revenue of the film and music industries [1, 2]. How to design games to actually be entertaining is still somewhat of a mystery, with researchers likening game design to 'something of a black art' (Bogost, in [3]). However, far from the generally dubious merits of black arts, games that lead to more positive play experiences are correlated to higher review scores [4], which in turn could predict higher sales [5]. If game companies can get a more predictable return of investment from ballooning entertainment game production budgets, because they can better predict whether a game is engaging, this could lead to a healthier, more egalitarian, industry, possibly with less need for predatory monetization practices. However, a paucity in scientific knowledge on what makes games motivating (as far as that can ever really be known) increasingly has ramifications outside of entertainment as well, as the motivating qualities of games are used for non-entertainment purposes, such as gamification and serious games [6].

Nowadays, the use of serious games and gamification designed for the user to develop several different skills in a fun way [7], needs little introduction. However,
while meta-analyses have proven the merits of serious games from a learning perspec-
tive, there is considerably less evidence for their motivating qualities [8]. There are
many reasons why serious games may end up boring [9], but one particular reason is
that the serious content matter that needs to be learned does not translate well to the
fantastical worlds players come to expect from playing entertainment games [10]. If
the content cannot be changed, perhaps the interaction can be designed to be more
enjoyable and feel more like a game.

Related to this is the concept of juicy game design. Juiciness is a term to describe exaggerated redundant audio/visual feedback on the players' actions [11], making the game 'feel good' to create a better player experience [12, 13]. As this term is rather vague still, it is often described using examples from games. An example is the coins in Super Mario Bros. These shiny pieces of gold float and rotate in the sky grabbing your attention, and make a satisfying 'ping' sound combined with a small particle effect when picked up by the famous plumber. Collecting hundred coins will usually grant you an extra life, but picking up difficult to reach coins will frequently make you lose lives as well, making their net benefit dubious. That players continue to try to collect the coins is partially a factor of their juicy game feel. Other more visual elements to be considered juicy are screen shake, particle effects, and exaggerated animations.

An important aspect of juiciness is feedback. Schell [14] proposes a lens of feed-
back, where he nudges game designers to think about what players need to know and
need to feel while playing the game. He introduces a lens of juiciness, focusing on
providing continuous and redundant rewards originating from players' actions. Hicks
et al. [15] interviewed game designers to create a framework to analyze juicy games.
In this research, juiciness is explained based on three components mostly related to
giving the right feedback at the right times. These components are related to coher-
ence of game characteristics, how the game state is communicated, and if the feed-
back is confirmatory and unambiguous. Deterding [16] highlights the importance of
the senses (audio, visual, and tactile) and its' promise to promote competence and
curiosity.

In his study, Atanasov [17] defined the quality of juiciness as an emotional re-
sponse, a feeling of reward and satisfaction, and enjoyment of being in the game
world. Swink [18] captures these experiences in the term game feel. In this theory,
juiciness is part of the game feel in the form of the building block of polish. Swink
[18] describes polish as "any effect that enhances the interaction between objects in
the game world, giving clues about the physical properties of objects". These effects
could be screen shake hinting the impact of a collision to the player, particles of dust
indicating the mass of an enemy or a high pitched "ping" sound as a sign of picking
up something of value like a coin.

What these studies have in common is a desire for positive experiences related to
satisfaction in the game and feedback as a means to achieve this feeling. Juiciness is a
balance between feedback and emotional experiences, used to convey the game world
through audio, visual, or haptic feedback responding to players' actions. To feel these
positive experiences, the player needs to be engaged with the game first. As Brown
and Cairns [19] state, the first stage of immersion is engagement and must occur be-
fore any other level. Immersion is the quality of a game that makes a player shuts out
their physical reality and is pulled into the game, creating a sense of presence. Pres-
ence is the feeling of being in the game world based on the user's psychology [20].
Brown and Cairns [19] see presence as total immersion, creating an atmosphere. Here
the graphics, plot, and sound combine this feature. Immersion and presence can influ-
ence the attention of the player, and the more attention invested by the player, the
more immersed a player feels.

As research has shown, visual juiciness can contribute to both these factors [21]
But compared to the extensive use of visual information, sound remains an underused
potential [15, 21]. Moreover, sound is often seen as decoration and not relevant for
playing. Audio has the potential to contribute to the juiciness of a game as it can pro-
vide feedback and has an effect on the emotions and fantasy of the player. Audio can
support gameplay [22], and ease the use of a game by providing information about the
states of the system [23]. Additionally, sound is a valuable component of overall
game aesthetics and affective perception [24]. Furthermore, it may be used to create
and enhance emotional impact [25] and contribute to immersion [26, 24]). As it seems
that computer games are thought of as a principally visual medium, sound could
prove to create more possibilities for creating juiciness. Therefore, this study aims to
explore juicy audio and its relationship with presence and immersion.

First, this paper discusses related work to explore the term juiciness. The second part consists of theoretical background on sound, how this may relate to juicy sound effects, and can be operationalized in our experiment. We perform a mixed-method experiment to a) determine whether and how juicy audio effects can improve the feeling of presence and immersion in a 2D entertainment game, and b) how sound is experienced by players of the game. The main contributions of this paper are therefore providing more evidence for the importance of juiciness in games, especially in terms of audio effects, as well as reflections on audio design and directions for future research when it comes to audio in games.

## 2 Related Works

The visual appeal of juicy games is one of the contributors to an improved player experience. Van den Abeele et al [27] created two versions of a game to measure the boundary of which tones children could hear; one made by the researchers and one by professional game designers. One could say the game designed by the game designers contains more juice, as it contained more interesting visuals and juicy feedback. Van den Abeele et al. [27] concluded that this game provided a better player experience compared to the game made by researchers. The children had a preference for the professionally designed game and proved to be more effective in-game as well. An early study of Juul & Begy [28] compared a juicy and non-juicy version of a tile-matching game and received a negative correlation between juiciness and player experience. This could be related to the fact that it is not clear which amount of juiciness is needed to increase the player experience. Kao [29] tested four different conditions of juiciness, minimal to extreme, of the same RPG game to test at which level juicy
effects hinder players. He uses a more extensive method of creating this juiciness and uses an iterative method to see if the visual and sound effects fit the game, hinting towards the contextual nature of juiciness. The results proved that both no application of juiciness, as well as extreme applications of juiciness, can hinder the player experience.

Furthermore, Hicks et al. [21] researched the visual appeal of two games using juiciness and related this to the players' experience and performance. They recreated a simplistic version of Frogger called Cuber, and an RPG-like game called Dungeon Descent and added juicy game elements to both of the games. In both of these juicy games, player experience was increased especially regarding presence and immersion.

Another way to provide feedback to the player is through the use of sound. Kao [30] used juicy audio effects to stimulate players in opening more loot boxes, which was efficient. Audio can be used as sensory gratification [31] and engage the player because it is sensorially pleasing. As it can also provide feedback about the actions of the player in the game world and make the game feel real [24], juicy audio may contribute to a better player experience and game feel. Sound is also a way to engage players by creating immersion and presence in games [25, 21]. Ermi and Mäyrä [32] describe immersion using three dimensions; sensory-, challenge-based- and imaginative immersion. Juicy sound can contribute to all three of these elements. In the case of sensory immersion, audio can make the game world feel real. This contributes to the imagination as well, as footsteps can make players identify with the characters and create a sense of realism [24]. Physical components of audio, like surround sound or headphones, can also contribute to sensory immersion [33]. Challenge-based immersion requires motor and mental skills to progress in the game. Juicy feedback is important in this case, and audio can contribute by using navigational listening modes to indicate for example where a threat, like a growl, is coming from [24]. However, Huiberts [31] identified that out-of-context sound and music effects, as well as non-responsive audio feedback and repetition of sound can decrease immersion. Importantly, the addition of sound effects hence does not improve immersion regardless of the type of sound.

## 3 Theoretical Background

Sound has original properties as it is omnidirectional and uninterruptible [33]. This opens up opportunities to use sound as (redundant) confirmatory feedback, but also shift the focus of the player during the gameplay [15]. Another property is that sound is ambiguous, meaning that one specific event is connected to one specific sound effect [15, 33]. To understand this property, it is important to understand how we perceive sounds both in a fictive world and in the real world.

According to Tuuri et al. [34], there are four types of listening modes: pre-conscious, source-orientated, context-orientated, and quality-orientated. Juicy audio is context-oriented, as it can convey the purpose of sound, what it means, and its' suitability. It is also pre-conscious, as it can invoke associations to properties of an event (big, strong, power). Related to these listening modes is Gavers' theory of Everyday
Listening [35]. Using this approach, sounds are described based on their source rather than using technical terms like pitch or compression (quality-orientated listening mode). As Gaver [35] describes: "Everyday listening is the experience of hearing events in the world rather than sounds per se, resulting in information about the interaction of materials at a location in an environment." For instance: a collision of two wooden objects in a closed space, next to your ears results in a loud and sharp bang. This is related to Swinks' [17] idea about polish, meaning that juiciness can give clues about the physical properties of an object. Therefore, it is important to choose sounds and context with high sonic potential when designing juicy game audio [36].

Juicy audio is also source-oriented [34] as it can convey information about the source of the sound, specifically its' cause and emotion. According to the appraisal theory, there are different types of emotions for fictive events [25]. Related to juiciness are the artifact emotions (A-emotions), which are linked to sensory pleasure. These A-emotions make the player enjoy the aesthetics of the fictive world including beautifully crafted sound. According to Perron [37], there is also another emotion specifically related to games; gameplay emotions (G-emotions). These G-emotions arise because the player cares about the progress of the game, meaning feedback is again important. As sound can influence a players' emotional reaction [38], it is important to select game elements that can convey emotions or an emotional context through their sound [36] For example, a flock of birds that are relaxed or flee give a completely different feel for the emotional context.

In fictive worlds, sounds are categorized in diegetic and non-diegetic sounds [39]. Diegetic sounds are sound real in the virtual world and have a source in this world. For instance footsteps, or gunfire. Non-diegetic sounds are from outside of the game world. Non-diegetic sounds do not have a source in the virtual world and have been added in. For example, music or sound effects added on the menu screen. Emotional responses to fictive events are possible because the diegetic effect of these sounds makes this reality perceivable. [25] However, to create immersion a reduced realism is enough to create a perceived reality [24]. This implies that not all sounds have to emerge from reality to be perceived as existing/realistic in the fictive environment.

# 4 Method

## 4.1 Game

A game was made using Unity and exported as WebGL to make it work in any browser on a desktop computer. This game, called Space Adventure<fnref n="1" />, is a space-themed arcade game where one has to get as many points as possible by rescuing lost astronauts and shooting down enemy ships while avoiding meteorites. The game is based on an endless runner, providing objects to avoid or pick up which are spawned

<footnote n="1">¹ A video of both versions of the game can be found in the supplemental material. Playable versions are hosted externally, and are playable for as long as the external host allows here: Juicy version: https://i.simmer.io/@keijioch/~09826b4b-43e3-7e09-6513-799f4b654a4a Control version: https://simmer.io/@keijioch/~36d78477-536d-d273-5b07-ef4b06dd51dd</footnote>
in randomized patterns. These patterns spawn more quickly based on how much time
has passed. The player (light grey spaceship) can move up and down using the arrow
keys to avoid enemy spaceships (black spaceship), their bullets and meteorites. By
pressing the space bar, the player can shoot and destroy the enemy spaceship after
three hits (image 1). Shooting down an enemy spaceship will gain the player one
point. Floating astronauts can be picked up and give five points. The player can take
damage by crashing into meteorites or enemy spaceships, or by getting shot by the
enemy spaceship. The player has three lives and the game is over when the player has
taken three hits.

This arcade format made it possible to use many sound effects, ranging from feed-back to contextual sounds. In order to research the effect of juicy audio effects on player presence, this game was developed into two different versions. Version A containing juicy sound and version B (control) without juicy sound. The visual design of the game, as well as juicy visual effects (explosions, screen shake), are the same in both versions. By comparing these versions, the following hypothesis can be tested:

*The version containing juicy audio effects creates a greater feeling of presence compared with the version without juicy audio effects.*

<figure><img src="image_4.png" /><figcaption>Fig. 1. screenshots of the game Space Adventure</figcaption></figure>

**4.2 Audio Design**

For each version, 10 different sounds were created by the first author. As there are no guidelines for creating audio, the theory of Everyday Listening by Gaver [35] is used as a starting point. This means the sound is based on an event and material. Impact sounds like collisions are made using an impact and a material related to the collision. For instance, the crumpling of aluminum foil is used to create the crumpling sound of colliding with an enemy ship. These unedited sounds are used in the control version of the game. For the juicy version, these sounds were edited using Adobe Audition. The variation between the juicy and non-juicy sounds is the amount of 'polish' that is used while editing, and using excessive amounts of effects to shape the sounds. Creating the sounds was an iterative process. They were pilot tested with four students of Eindhoven University of Technology using a video showing multiple sound options for an event. A professional game designer also reviewed the game and sounds, to independently approve whether they fit the context of the game. Two sounds were left
out in both versions, namely when the player shoots at an astronaut or a meteorite. The sounds are described in table 1.

<table><caption>**Table 1. Description of sounds**</caption><thead><tr><td>Action</td><td>Description of sound</td></tr></thead><tbody><tr><td>Jet Engine</td><td>This sound needs to resemble working engines. It is based on white noise, edited with flanger and distortion to resemble Star Wars engines in the juicy version.</td></tr><tr><td>Moving</td><td>This sound needs to resemble air. A male voice saying 'whoosh' was used to produce this airy sound, which was edited with a higher pitch and more verb in the juicy version.</td></tr><tr><td>Shooting</td><td>This sound needs to resemble a laser, and its' movement through air. A female voice saying 'pew' was used to produce this airy sound. This was edited for the juicy version into a laser sound by pitching it higher, using verb and distortion. For the enemy shooting, the sound was pitched down in order to create difference.</td></tr><tr><td>Hit & dead Enemy</td><td>Getting hit is an impact sound, so these sounds are created using the impact of a drum set in GarageBand. Shooting down an enemy has no different sound in the control version of the game. In the juicy version this resembled a small explosion. These explosions were made by layering multiple drum impacts, distorting them and adding verb. When an enemy was killed, the explosion contained more verb in order to prolong the sound.</td></tr><tr><td>Hit & dead Player (Game Over)</td><td>As the sounds for hitting an enemy, the sounds are also based on impact of a drum set in GarageBand. In the control version these events produce no different sounds. In the juice version, the sound needed to sound 'negative'. For getting hit by the enemy, this resulted in a sound resembling engine failure made by editing the impact sound by pitching down the sound gradually, and adding chorus and distortion to the sound to make it a 'unknown' engine. The explosion was made using the same method as the enemy explosion. To add more negativity, the sound was heavily stretched while pitching the sound down gradually.</td></tr><tr><td>Meteor crash</td><td>This sound needed to resemble a crumbling impact sound. This was done by recording a piece of paper being crumpled. In the juice version, this was edited with distortion and delay.</td></tr><tr><td>Astronaut entering</td><td>This is a feedforward sound, a notification. This sound was made with the xylophone in GarageBand. In the juicy version this sound was edited with verb and by pitching it higher.</td></tr><tr><td>Astronaut Pick-up</td><td>As the astronaut is a human, a female 'thanks' is heard when picked up. In the juice version, it has an extra sound indicating scored points. This sound is a variation of the astronaut entering sound with more delay and by pitching it higher. This same sound is also present when shooting down an enemy.</td></tr></tbody></table>
## 4.3 Participants

A total of N=61 participants participated in the experiment, the majority of which were between 18 and 25 years of age. 30 participants, of whom 20 self-identified as male and 10 as female, got randomly assigned to the juicy condition. The remaining 31 participants, of whom 24 self-identified as male and 7 as female, got randomly assigned to the control condition. Due to Covid-19 restrictions, the experiment was moved fully online and the participants were solicited through social media (Facebook and Instagram). As that only garnered 27 participants, a crowdsourcing platform (Prolific) was used to get more participants and improve the power of the study. This did mean that one half of the participants did not get a reward for participating and the other half did get a (minor, around 0.90 GBP) for participating in the short experiment. Therefore, the type of participant solicitation was recorded so that it could be used as a covariate. In the Juicy version, 15 participants were solicited through social media and 15 participants were crowdsourced through Prolific. In the Control version, 12 participants were solicited through social media and 19 participants were crowdsourced through Prolific.

## 4.4 Measurements

This research uses a mixed-methods approach, using both quantitative data to evaluate the feeling of presence in the two versions of the game and a qualitative evaluation of the sounds. A smaller set of participants followed up with a semi-structured interview to get additional qualitative data. To research the influence of juicy audio effects on the experienced presence, a quantitative evaluation was done using the Presence Questionnaire by Witmer and Singer [40]. For this study, the categories Involvement and Interface Quality were left out as they did not particularly match well with the simple mechanics and 2D nature of the game (Involvement) or were more about the input modality and lag of the game (Interface Quality), which we did not expect to be influenced by the intervention, but was also something we could not control well over the internet. All questions of the PQ were presented using a 7-point Likert scale. In addition, an open-ended evaluation of the juicy audio used in both versions of the game was employed. Participants were asked questions like which audio effects they did and did not appreciate, found pretty or ugly, and why, and what their association was with specific sounds. To not make the questionnaire too exhausting, four sounds were chosen at random to be evaluated using the questionnaire (shooting, player hit & dead, enemy hit & dead, astronaut).

11 Participants signed up for the follow up interview (5 male, 6 female), where they played both versions and then gave their thoughts. This interview was conducted in a semi-structured fashion starting from similar questions to the online questionnaire, to tease out more in-depth statements regarding the sound quality and sound experience after having played both conditions, as well as elaborate on the sounds' affective associations.
**4.5 Procedure**

For the quantitative evaluation of the games, a web page containing a WebGL version of the game was created. By clicking a hyperlink the participants were guided to one of the test pages containing one of the two games (A: juicy & B: control) including a step-by-step explanation to support participants in going through the study smoothly. After agreeing with the study conditions, a participant played the game until they were satisfied. When a participant wanted to completely stop the game, the 'quit game' button needed to be pressed to see their total playing time. After the participant quit the game, the questionnaire containing the PQ items and open-ended questions was filled out

Those who were interested and left their email while filling in the questionnaire, participated in an interview afterwards to get more in-depth insight on all the sounds and the players' experience. Here the participants first played both versions of the game, and then participated in the semi-structured interview, guided by a set of questions similar to the open-ended questions in the questionnaire. All interviews except one were conducted in Dutch. The interviews were recorded and transcribed. The Dutch quotes are translated in English when used in this paper.

**5 Results**

**5.1 Quantitative research**

In order to analyze the effect of juiciness on presence, a MANCOVA was performed in IBM SPSS v25, with condition as fixed factor and the Sensory Fidelity and Adaption/Immersion dimensions of the Presence Questionnaire as dependent variables. Since there were two ways of soliciting participants, either through social media or a crowdsourcing platform (Prolific), and this could influence the way people engage with the game, which in turn could influence their experienced presence, the way participants were solicited was entered as a binary covariate. The MANCOVA showed a significant effect of condition on the combined dependent variables, while controlling for participant solicitation procedure: $F(2, 57) = 4.678, Wilks' \Lambda = .859, p = .013, partial \eta^2 = .141$.

Individual ANCOVAs showed that this effect was significant for both sensory fidelity [$F(1, 58) = 5.435, p = .013, partial \eta^2 = .102$] and adaption/immersion [$F(1, 58) = 3.652, p = .019, partial \eta^2 = .091$] dimensions. In both cases, the juicy version led to significantly higher sensory fidelity (control version M = 4.17, SD = 1.08 vs. juicy version M = 4.69, SD = 0.86) and a significantly higher adaption/immersion (control version M = 5.41, SD = 0.85 vs. juicy version M = 5.87, SD = 0.74).

The covariate was confirmed to have a significant influence on the sensory fidelity dependent variable [$F(1, 58) = 8.241, p = .003$] but not the adaption/immersion variable [$F(1,58) < 1$]. There was no effect of condition on total playtime $F(1,58) < 1$.
## 5.2 Qualitative research

For the qualitative part of the questionnaire and the interview, a thematic analysis [41] was carried out. There are two types of quotes: quotes derived from the questionnaire following the experiment are indicated with the version and participant number (A.. (juicy) & B.. (control)); quotes derived from the follow up interview with the 11 interested participants who played both versions of the game, are indicated with a P followed by the interviewee number. This analysis resulted in three main themes and their respective sub-themes related to the appreciation of the juicy sound; the way they are perceived and described, associations and expectations and the practical quality of sound. The themes and sub-themes are described below.

### Theme 1: Describing sounds

When the participant was asked to describe a sound or what they appreciate about it, they described the sound using various methods. Some participants with musical knowledge used sound-related terms: "There is more bass in this sound" (P9), "one of them is clearly higher pitched." (P11), and "[...] It sounds like regular distortion" (B4). Most participants described sounds using other ways, like mimicking the sounds or using associations. They described situations where they heard similar sounds: "It sounds like a space battle [...], like in Star Wars." (P2) The shooting sound in the control version was often described as "a child making his own sound effects" (P7).

#### Materials or events

Another way the sounds were described was to use the perceived material or event. For instance, in regards to material, participants mentioned "I know the sounds are telling me about the material of the spaceship. I would expect metal against metal" (P11), or "I miss the feeling of material and space. It is a very general sound not specific to the game." (A3). Events were also used to describe a sound. For instance, "It sounds like that air is escaping from somewhere" (P2), and "It sounds like a bag of chips being crumpled." (P5)

#### Affective terms

Besides this, participants also used affective ways of describing why they liked a certain sound, such as "It sounds soothing" (P8), and "The background is really exciting" (P1). This affective quality of sound is also reflected in the difficulty of describing sounds, sometimes resulting in insecurity ("I have no idea how I can explain this." (P2), "This doesn't help you at all, does it?" (P3), or "Nothing" (common response)), or confusion for participants ("We are not talking about the same thing at all!" (P7)).

### Theme 2: Expectations & associations

Players have certain expectations and associations with a sound, using this to describe sounds from the game. "It sounds like lasers. That sounds futuristic." (P9), "When such a big spaceship shoots at me, and then I hear small pew-sounds." (P7). Some participants had conflicting preferences. An example is the use of a human voice say-
ing "thanks" for rescuing an astronaut. "I think *the thanks when you pick up an astronaut is subtle, and nice to listen to*" (P3), but also "*The thanks were quite grating after multiple times*" (A14). An important factor to determine whether a sound matches a players' expectation, is the visual that is paired to it. The sound and visual have to match for a players' association to be correct. For instance, one participant mentioned; "I think it is satisfying when you hit and explode an enemy ship. That this happens in phases. [...] It is probably a combination of the visuals and sound, which really suit each other here." (P7). Another factor is timing. If a sound is played with too much delay compared to the visual, the sound and event are not linked. "I have not heard the "ping" [score sound] before, and I don't know what it does. You already search for next objects, and then the sound is a bit too late." (P4)

## Theme 3: Pragmatic quality

### Feedforward and feedback

As juiciness is highly related to giving feedback, sound was often used as a way to indicate if an action was done correctly. "I think most sounds are confirming what action you took." (P6), and "It is in both cases positive feedback that your actions have the desired impact." (P1). Sound was also used to gather information as feed-forward. "It helps me to understand the game and focus my attention on something. For instance, sometimes I am not worrying about enemies shooting until I hear it." (P3), or "It gave me the idea if I needed to do something. Watch out, you need to take an action." (P10). It also differed between participants. For example, one participant used feedforward by sound to improve his performance: "I recognized the pattern in which I shoot, and when something interrupts. I press the space key as often as possible, and if I hear another sound I know it is not mine but the enemies." (P1).

### Hierarchy

One important aspect participants often mentioned was related to the hierarchy of the sound. One participant mentioned: "*The background is not overwhelming, it did not drown out what was important in the game. The hierarchy of information is important, and this sound should be on the bottom.*" (P10). Often mentioned was that the sound of the players' shooting drowns out the other sounds, giving less room for feedback and feedforward. "*It drowns out other sounds. You do not hear that the other ship is shooting at you.*" (P8). Or when the astronauts spawned quicker further in the game. "*[...] a lot of astronauts with a lot of sounds. I was not sure what to do with that.*" (P10). The sounds with a higher volume were perceived more often than those with a lower volume, enabling the game designer to influence this hierarchy. "*[this sound] seems to be louder than the other one, making me believe it responds to what I am doing.*" (P7).

### Redundancy

Overlapping or similar sounds are often seen as something positive by players. "I liked it that the sounds overlap. If you shoot multiple times, you hear the sound multiple times as well." (P1). For similar sounds, the difference in shooting between the player and enemy can be taken as an example. In the juice version of the game, one
was pitched lower than the other. While in the control version it was the same sound. "I liked it that there were different sounds for when I shoot and when the enemy shoots." (P3), and "I miss in this version [control] the difference between shooting from me and the enemy" (P2). But there seems to be a fine line in differences in sounds that players can perceive. As participant 1 mentioned "I like it that they sound alike, so you know these are linked. But maybe they sound a bit too much alike. You have to listen closely to hear the difference. But this happens automatically with the visuals" (P1). Adding an extra sound to indicate when points are scored after picking up an astronaut, was perceived differently by participants. The score point sound was perceived as pretty most of the time: "[..] the sound of coins and "thanks" when picking up a human" (A8). During the interviews, it became clear that this sound was most of the time not noticed. "Is there an extra sound? You mean the bell! I think it is unnecessary." (P8).

# 6 Discussion

A MANCOVA shows a significant main effect of juicy sound on presence. This significant effect holds for both tested scales of Sensory Fidelity and Adaptation/Immersion, where in both cases the juicy audio version led to more presence than a base version with less juicy audio feedback. These results suggest that juicy sound, operationalized in this study as more polish to the sound, can contribute to a greater feeling of presence in the player. This can also be seen in the thematic analysis of the qualitative data, which also describe what aspects are considered by players regarding juicy sound.

The themes of expectations & associations and pragmatic qualities are two main themes describing what players look for in juicy audio. When looking into the theme of expectations & associations, the findings indicate that players do not only listen to the quality of sound, but they connect it to different associations and memories. One participant makes a clear distinction between an association with a sound, and its' aesthetic qualities: "I find the "pling" sound after you catch an astronaut the prettiest. It gives me a nice feeling after the accomplishment. But if by prettiest you mean the best sound, I have to say the game over sound. It is so good in the sense that it makes me sad that I died, but not too mad to not play the game again." (A13). When a player makes associations, two outcomes can occur. When the players' association with a certain sound matches their expectations, it seems to increase their immersion in the game. More reactions were elicited while playing the juicy version of the game. Positive emotional reactions like "oof" when the player hit an obstacle, or short "yes!" when they shot down an enemy was more common compared to the control version of the game. Claims connected to satisfaction were also heard. "I like it when it blows up. It makes you think 'Haha!'" (P10). When the sound matches the players' expectations with the perceived context, it can also deepen their emotional connection to it. "There is a sort of dark undertone, the buzz. It makes it feel like there is something exciting bound to happen." (P10). Some participants mentioned terms related to immersion when talking about what the sound does for them. "If you don't hear anything
it will become boring, which makes you quit the game." (P4), "The sounds [...] were in my experience mostly to keep you in the bubble." (P1), and "In the [control] version, I thought 'okay sure'. But in the [juicy] version, you saw from my face that I got really into it." (P9)

When the association does not match the expectation of the player, negative emotional responses like confusion or irritation occur. The sounds that were mentioned the most were the notification sounds when an astronaut appears, the point collecting sound, and the difference between shooting sounds emanating from the player and enemy. Some participants mentioned: "It is almost annoying that I do not know what sound it is and where it is coming from." (P9), and "I thought the ding in the [non-juicy] game was very confusing. I could not figure out where that's coming from" (P11). In the theoretical background section, we characterized juicy feedback as being unambiguous [15] and redundant [11]. While ambiguity was more often reported in the control condition, not all sounds may therefore have qualified as juicy in the juicy condition. Vice versa, less ambiguity in the juicy condition may also mean that the polish or embellishments were not fully redundant. This may furthermore beg the question whether juicy feedback could be considered redundant in the first place and not a reinforcement, both in feedback as in the understanding of the audiovisual effect itself. This requires more disentanglement in future research.

Sometimes, ambiguous associations can be learned over time while playing the game as mentioned by P6: "I think that you learn to associate different sounds with scoring points, so you do not need to use the same one everywhere." One exception was the shooting sound in the control version of the game. This sound was perceived as comical, but not necessarily bad when applied to the right target group and goal. One participant mentioned "This is not necessarily something negative, as it is really funny. It does lead to a different intensity, which is maybe more suited in games for children." (P1). This may mean that juicy sound does not need to fit a certain association to create an engaging experience but can contribute to a different goal and context of the game.

Regarding the pragmatic quality of juicy sound, factors like feedback and feedforward are still important. During the interviews, it became clear that sounds and their expectations are often linked to the information the sounds needed to convey. The same procedure occurs here described for the associations & expectations theme; if the information matches the players' expectations, it may enhance immersion. Otherwise, it will result in confusion or irritate the player. The pragmatic quality can be influenced by the game designer by playing in into the hierarchy of the sounds. By using volume to put certain sound on the foreground or background and timing to create links between audio and visual information, certain information can be prioritized. It is therefore important to keep the goal of the game in mind, to establish a hierarchy in sounds to help the participant understand the game. The use of redundant sounds, for instance, two sounds to indicate the same event, is perceived differently by the player. Some found the extra sound unnecessary and did not perceive it immediately. This can mean that juiciness happens subconsciously and is not supposed to be remembered. "I may have noticed that happened, but I have no memory of it anymore. It is hard to remember these things, especially since they're supposed to be
*subtle or subconscious.*" (P11). This can also be related to the amount of experience a player has with games. Participants with more gaming experience, tend to focus more on the pragmatic qualities of sound than casual players to improve their performance.

# 7 Limitations & Future Work

This experiment was conducted online due to Covid-19, as such it is not possible to say whether all the participants had similar sound reproduction quality, or technically even turned on their sound when prompted. However, randomization and the significant effect in the expected direction, as well as detailed interview feedback, would suggest that there was a notable difference. Using multiple ways to gather participants, using a crowdsourcing platform and social media, may also have influenced the results of this study. The significance of adaption/immersion improves after leaving the variation of Prolific out of the results, indicating there is a difference in results between participants gathered on Prolific and social media. Participants may have different motivations for participating in the study. Monetary rewards could lead to participants being more motivated to put in the effort, but this can also go the other way of clearing the study as fast as possible. But if the latter was the case, one would expect the results to be less significant. In this case, significant effects hold with or without adding the solicitation procedure as a covariate.

Another limitation is the use of the PQ and adapting it to fit the game. In the original PQ questionnaire, presence is the mean of all the questions in the questionnaire. As some sub-categories (Interface quality & Involvement) were left out, the scale does not provide a total measure of presence. Therefore, care should be exercised in comparing these results with those in other research papers that do use the full questionnaire. It's also theoretically possible that interface quality and involvement are negatively impacted by juicy audio somehow.

As there are no guidelines for creating juiciness, let alone juicy sound, this research used combined theories of Game Feel (polish) and everyday listening [35] to create sounds. More research can be done on what juiciness is when it comes to sound, to provide more exact guidelines that can be implemented in game design. The themes provided in this research could be a base for creating these guidelines for sound.

Lastly, sound is part of juiciness but difficult to communicate. It is a personal interpretation, and every player has a different background. For instance, a player proficient in playing music would use more technical terms than a player with no musical knowledge. During the interview sessions, we frequently encountered difficulties by the participants to verbalize their experiences, and in fact game research does not provide many useful pointers to describe the myriad of sound effects related to game interactions either. It would be beneficial for game designers and game experience researchers to develop a common vocabulary that can be used to describe sound qualities or find a method to evaluate comments from players with a different musical background. Associations may be a good way to contribute to this vocabulary, as well as Gavers' theory of everyday listening [35].
# 8 Conclusion

In games, it is hypothesized that juiciness (exaggerated redundant audio/visual feedback on the players' actions) [11] can improve the player experience in a game. As the term juiciness is still rather vague, research on this topic is new. Some research has been done based on the visual aspect, but sound is often neglected. This research can be seen as a method to explore and evaluate juicy audio in video games. This was done by looking into how it affects the feeling of presence and how players experience and evaluate the sounds affectively during gameplay. By comparing two versions of a game, one with juicy sound and one without juicy sound, there is a significant effect with a large effect size (partial η² = .141) of juicy sound effects for combined immersion and sensory fidelity, as factors of Witmer and Singer's Presence Questionnaire [40], in the game. This holds for the separate subscales with a medium to large effect size. This result was in line with the hypothesis: the version containing juicy audio effects, which in this study means more polish to the sound, creates a greater sense of presence as expressed in Immersion and Sensory Fidelity, than the version without juicy audio effects. While this experiment was performed with a simple 2D space shooter game, we think that creating juicy audio could also be a viable way to improve the immersion in serious games (and possibly engagement as a result), because juiciness is relatively independent of the content matter and game mechanics.

Regarding the affective experience and evaluation of juicy audio, a thematic analysis reveals three main themes. First, players have a certain association and expectation from a sound, based on their memories and the image that they see. If this expectation is met, players have a positive experience resulting in more immersion. If this expectation is not met, players experience negative feelings like confusion or irritation. Secondly, sound has a pragmatic quality, where players use sound as feedback and feedforward to understand and improve in the game. Game designers can influence the pragmatic quality by thinking about the hierarchy of sounds. It is important to understand the goal of the game, to provide important information first and establish a hierarchy in sounds. The third theme revolves around the way players describe sounds. The way a player listens to sound, comprehends it and communicates their qualities differs for each person. For instance, someone with more musical knowledge uses more technical terms to communicate a sound. Interesting for future research is to create a common vocabulary to describe sound qualities for game designers and game experience designers.

# References

1. IFPI. 2020. IFPI issues annual Global Music Report. (May 2020).
https://www.ifpi.org/ifpi-issues-annual-global-music-report/. Retrieved February 2021.

2. Witkowski, W.: Videogames are a bigger industry than movies and North American sports combined, thanks to the pandemic. (January 2021). 2021.
https://www.marketwatch.com/story/videogames-are-a-bigger-industry-than-sports-and-movies-combined-thanks-to-the-pandemic-11608654990. Retrieved February 2021.
3. Fullerton, T., Swain, C.: Game Design Workshop: A Playcentric Approach to Creating Innovative Games (2nd. ed.). Gama Network Series. 2008.

4. Johnson, D., Watling, C., Gardner, J., Nacke, L. E.: The edge of glory: the relationship between metacritic scores and player experience. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play, pp. 141-150. 2014.

5. Orland, K.: Steam Gauge: Do strong reviews lead to stronger sales on Steam? https://arstechnica.com/gaming/2014/04/steam-gauge-do-strong-reviews-lead-to-stronger-sales-on-steam/ (April 2014). Retrieved February 2021.

6. Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to gamefulness: defining "gamification". In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, pp. 9-15. 2011.

7. Susi, T., Johannesson, M., Backlund, P.: Serious games: An overview. IKI Technical Reports. Skövde: Institutionen för kommunikation och information. 2007.

8. Wouters, P., Van Nimwegen, C., Van Oostendorp, H., Van der Spek, E.D.: A meta-analysis of the cognitive and motivational effects of serious games. Journal of educational psychology 105, no. 2: 249. 2013.

9. Buday, R., Baranowski, T., Thompson, D.: *Fun and Games and Boredom*. Games for Health Journal 1, 4: 257–261. 2012. https://doi.org/10.1089/g4h.2012.0026

10. Van der Spek, E. D., Sidorenkova, T., Porskamp, P., Rauterberg, M.: The effect of familiar and fantasy aesthetics on learning and experience of serious games. In International Conference on Entertainment Computing, pp. 133-138. 2014.

11. Gray, K., Gabler, K., Shodhan, S., Kucic, M.: How to Prototype a Game in Under 7 Days. 2005. Retrieved 7 April 2020, from https://www.gamasutra.com/view/feature130848%20how_to_prototype_aZYXWVUTSRQPONLMN7_.php

12. Jonasson, M., Petri, P.: Juice it or lose it. Video. (2012, May 24). 2012. Retrieved January 13, 2020 from https://www.youtube.com/watch?time continue=3&v=Fy0aCDmgnxg

13. Pears, M.: Level Design Lobby - Ep 32: Juice. Podcast. (2018, December 11). Retrieved December 9 2019 from https://www.spotify.com/

14. Schell, J.: The Art of Game Design: A book of lenses. CRC press. 2008.

15. Hicks, K., Dickinson, P., Holopainen, J., Gerling, K.: *Good Game Feel: An Empirically Grounded Framework for Juicy Design*. Digital Games Research Association Conference 2018.

16. Deterding, S.: The Lens of Intrinsic Skill Atoms: A Method for Gameful Design. Human-Computer Interaction 30, 3-4: 294-335. 2015. https://doi.org/10.1080/07370024.2014.993471

17. Atanasov, S.: Juiciness: Exploring and designing around experience of feedback in video games. Faculty of Culture and Society (KS), Malmö, 2013.

18. Swink, S.: Game feel: a game designer's guide to virtual sensation. CRC Press. 2008.

19. Brown, E., Cairns, P.: *A grounded investigation of game immersion*. In Extended abstracts of the 2004 conference on Human factors and computing systems - CHI '04. 2004. https://doi.org/10.1145/985921.986048

20. Cummings, J. J., Bailenson, J. N.: *How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence*. Media Psychology 19, 2: 272–309. 2015. https://doi.org/10.1080/15213269.2015.1015740

21. Hicks, K., Gerling, K., Dickinson, P., Vanden Abeele. V.: *Juicy Game Design: Understanding the Impact of Visual Embellishment on Player Experience*. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play. 2019. https://doi.org/10.1145/3311350.3347171
22. Jørgensen, K.: Audio and Gameplay: An Analysis of PvP Battlegrounds in World of Warcraft. Gamestudies, 8(2). 2008.

23. Jørgensen, K.: On the functional aspects of computer game audio. In: Proceedings of the 3rd Conference on Interaction with Sound, Audio Mostly 2006 (pp. 48-52). 2006.

24. Grimshaw, M.: Sound and immersion in the first-person shooter. International Journal of Intelligent Games and Simulation, 5(1), 119-124. 2008.

25. Ekman, I.: Psychologically motivated techniques for emotional sound in computer games. Proceedings of AudioMostly, 20-26. 2008.

26. Collins, K.: Game sound: an introduction to the history, theory, and practice of video game music and sound design. 2008. MIT Press.

27. Van den Abeele, V., Wouters, J., Ghesquière, P., Goeleven, A., Geurts. L.: *Game-based Assessment of Psycho-acoustic Thresholds*. In: Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play. 2015. https://doi.org/10.1145/2793107.2793132

28. Juul, J., Begy, J. S.: *Good Feedback for bad Players? A preliminary Study of 'juicy' Interface feedback*. In: Proceedings of first joint FDG/DIGRA Conference, Dundee. 2016.

29. Kao, D.: The effects of juiciness in an action RPG. Entertainment Computing 34: 100359. 2020. https://doi.org/10.1016/j.entcom.2020.100359

30. Kao, D.: Infinite Loot Box: A Platform for Simulating Video Game Loot Boxes. IEEE Transactions on Games 12, 2: 219–224. 2020. https://doi.org/10.1109/tg.2019.2913320

31. Huiberts, S.: *Captivating Sound the Role of Audio for Immersion in Computer Games*. Ph. D. Dissertation. HKU University of the Arts, Utrecht. 2010.

32. Ermi, L., Mäyrä, F.: *Fundamental components of the gameplay experience: Analysing immersion*. Worlds in play: International perspectives on digital games research, 37(2), 37-53. 2005.

33. Liljedahl, M.: Sound for Fantasy and Freedom. In: Game Sound Technology and Player Interaction. IGI Global, 22–43. 2011. https://doi.org/10.4018/978-1-61692-828-5.ch002

34. Tuuri, K., Mustonen, M-S., Pirhonen, A.: Same sound–different meanings: A novel scheme for modes of listening. Proceedings of Audio Mostly, 13-18. 2007.

35. Gaver. W.W.: *What in the World Do We Hear?*: An Ecological Approach to Auditory Event Perception. Ecological Psychology 5, 1: 1–29. 1993. https://doi.org/10.1207/s15326969eco0501_1

36. Alves, V., Roque, L.: *Guidelines for Sound Design in Computer Games*. In: Game Sound Technology and Player Interaction. IGI Global, 362–383. 2011. https://doi.org/10.4018/978-1-61692-828-5.ch017

37. Perron, B.: A cognitive psychological approach to gameplay emotions. In: DiGRA 2005: Changing Views: Worlds in Play, 2005 International Conference. 2005.

38. Nacke, L. E., Grimshaw, M.: Player-Game Interaction Through Affective Sound. In Game Sound Technology and Player Interaction. IGI Global, 264–285. 2011. https://doi.org/10.4018/978-1-61692-828-5.ch013

39. Ekman, I.: Meaningful noise: Understanding sound effects in computer games. Proc. Digital Arts and Cultures, 17. 2005.

40. Witmer, B. G., Jerome, C. J., Singer, M. J.: The Factor Structure of the Presence Questionnaire. Presence: Teleoperators and Virtual Environments 14, 3: 298–312. 2005. https://doi.org/10.1162/105474605323384654

41. Braun, V., Clarke, V.: *Thematic analysis*. In APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. American Psychological Association, 57–71. 2012. https://doi.org/10.1037/13620-004