# A Conceptual Framework for the Analysis of First-Person Shooter Audio and its Potential Use for Game Engines

Authors: Mark Grimshaw; Gareth Schott
Journal: International Journal of Computer Games Technology
 (2008)

---

## Abstract

We introduce and describe a new conceptual framework for the design and analysis of audio for immersive first-person shooter games, and discuss its potential implications for the development of the audio component of game engines. The framework was created in order to illustrate and acknowledge the direct role of in-game audio in shaping player-player interactions and in creating a sense of immersion in the game world.
Furthermore, it is argued that the relationship between player and sound is best conceptualized theoretically as an acoustic ecology.
Current game engines are capable of game world spatiality through acoustic shading, but the ideas presented here provide a framework to explore other immersive possibilities for game audio through real-time synthesis.


---

## Body

## 1. Introduction

Members   of your platoon cluster around you and over the radio comes the message “follow me” with others
responding “affirmative.” You follow the sound of organ music discovering
that it emanates from a Gothic church with buttressed superstructure. In the distance, the sharp crack of gunfire
and the dull thud of explosions catch your attention. Eager to join the fray, the metronomic rhythm
of your running boots on the hard surface of the path is soon matched by the
sound of your panting breath; this quickly overtakes the pace of your slowing
footsteps so you slow to a walk. Soon
the organ music is left behind and the cacophony of battle intensifies. Suddenly, a siren indicates that a platoon
member has managed to steal the enemy's flag and, amidst the sharp staccato of
machine gun fire, you see and hear him weaving and running towards you, flag in
hand, and hotly pursued by a posse of enemy soldiers.An account of player experiences when playing first-person shooter (FPS)
games is useful in several respects when establishing the groundwork and rationale
for the development of the conceptual framework presented in this paper. To
begin with, it may intimate a level of engagement with games signaling, among
other things, the pleasures and satisfaction at being involved in an individual
or team situation. Such pleasures result from (amongst other reasons) a
demonstration of individual skill and mastery (in a multiplayer
situation, apublic demonstration of
such skill) combined with the pursuit of collaborative
objectives in an environment that, in many respects, simulates real-world
problem-solving scenarios. The account may also intimate that the perspective
presented and the nature of the engagement with the game are both subjective
and individually constructed. Indeed, the plot of the very story may be very
different when narrated by the other players involved in the same game
scenario. Importantly, it outlines how FPS games typically place the player in
a hostile environment (the hunter and the hunted) that demands attentiveness to
all available cues (especially, and crucially for the aims of this paper, sound
cues) for team success, character survival and individual glory. What is
described is not fiction or imagination, but a lived account of experiences
within the immersive spaces of a particular genre of computer games.Many of the points raised above may be applied to many computer games (not
to mention other forms of gaming) with varying
degrees of success and prioritization. Some games provide different cues for
the solution of a puzzle,  some have less of a team aspect or none at all,
others are less combative while others offer different perspectives. (For
a more complex and extensive attempt at taxonomizing computer game types, seeA Multi-Dimensional Typology of Games [1].) Broadly speaking, it is the types of cues offered, the
hostile environment, the mix of team and individual skills, the immersive,
first-person perspective, and, of course, the combat that signals the FPS genre. It is our contention that sound
cues in FPS games afford more possibilities than in other genres to live the
type of game experience signaled in the account. A first-person perspective
game uses sound to immerse the player within the game environment in a way that
a 2-dimensional platform game such as  Donkey
Kong [2] or a variety of role playing games (RPGs) do not typically
attempt. This paper presents a conceptual framework for FPS game audio and a
model of game worlds as an acoustic ecology in order to increase the ability of
both game scholars and developers to analyze both the relevance of current
applications of audio and its effect upon player immersion and player-player
interaction.The conceptual framework (Figure1) and examples given here account for
multiplayer run and gun FPS games, that is, networked games in which there is
more than one human player. Bots (computer-generated
characters) do not (yet) respond to sound but, in tracking down
or evading player-controlled characters, make use of game code variables that
change according to that character's position, actions and status. The assumption is made that the conceptual
framework for single-player games constitutes a subset of those found in multiplayer
games, hence it is the latter that is investigated here.Figure 1
A conceptual framework of the FPS game acoustic ecology in multiplayer mode.Of the terminology that resides in the conceptual framework constructed
and outlined here, a large proportion is derived from a broad range of
disparate work on the nature and function of sound spread across a variety of
media. Such areas utilized in the framework include kinaesthetics, affordance
theories, modes of listening, auditory icons, diegetic sound, sonification,
causality, indexicality, soundscapes and immersion theories to name but a few. In doing so, it was necessary for
all these treatments of sound to be adapted to account for the medium through
which the FPS genre is experienced, either by pointing out the significant
differences (between the medium to which the terminology was originally applied
and that of the FPS genre) and adjusting accordingly or by extending the
theoretical model to include new terminology where existing terminology proved
insufficient. In order to further
illustrate the framework, the FPS gameUrban
Terror [3] is used (Figure 2).Figure 2
A screenshot fromUrban Terror.This paper extends a paper [4] presented by the authors at the Cybergames
2007 conference by exploring the implications of the conceptual framework
described there for the future development of the audio component of game
engines.


## 2. The Conceptual Framework

Understanding of sound in the FPS game world is a matter of
experience. This experience, and the
resultant comprehension, is the result of the training and conditioning which
occurs either external to the FPS game (for example, through exposure to
popular commercial cinema sound conventions) or which takes place during
initial exposure to the sonic conventions of the FPS genre as a whole or to the
specific FPS game being played. These
conditions apply equally to both the sound designer and the player who,
ideally, should have broadly similar socio-cultural experiences and
understandings of sound. FPS game sounds
may therefore be described as a set of sonic signs or auditory icons which may
be analyzed through semiotic terminology, such as indexicality, iconicism, symbolism or metaphor, in an attempt to explain how the intended meaning is
(ideally correctly) translated to the received meaning. Thus, the FPS game
engine may be understood as a sonification system in which sounds are
(re)encodings of non-audio data.  This
game world data may derive either directly from the game engine, as in the case
of game status sounds for instance, or, in the majority of cases, is an
expression of player activity, such as the sounds of footsteps or the firing of
weapons.


### 2.1. Audio Sample Categorization

Our first approach in constructing a taxonomy of FPS game sounds was one
that perused the classification of audio samples as found on either the
distribution medium or on the installation drive of the product itself. This
initial approach provided useful insights into the sound designer's
classification system which itself may be extrapolated to the meaning that is
intended for particular sounds. While this approach has already been used to
account for game sound within games studies community through reference to
character, interactable or environment sounds, for example, [5–7], none of
the literature explicitly examines the distribution or installation media for
further clues as to the sound designers’ intentions. At the very least, our
approach revealed a division between diegetic and nondiegetic sounds as there
is typically a separate directory for music or menu interface audio samples as
opposed to other audio samples which,  themselves, may be subclassified into
character, interactable, environment or feedback audio samples.
Thus, of the 607 base audio samples in  Urban Terror (game-specific audio samples as opposed to
level-specific audio samples), fully 601 are available to be used during
gameplay with the remaining six being the menu music (one) and menu interface
sounds (five). The 601 audio samples
are, therefore, diegetic whilst the remaining six are nondiegetic.The game designer-constructed organization of audio samples inUrban Terror is illuminating in several
respects. Firstly, it is an indication
of how the game code deals with sound and its relationship to a variety of
characters, objects, and locations within the game. Sounds that players’ characters create as they
move, fire, or taunt are separated from environment sounds which are part of a
location; sounds of interactable objects are separated from the sounds non-interactable objects make, and diegetic sounds are separated from nondiegetic sounds. Secondly, the sheer number of sounds is an
indication of the importance of sound to the game experience. Thirdly, this
organization of sound indicates some of the technical limitations of the game,
namely in the areas of media storage and computer memory. As an example, some audio samples of
footsteps are shared between the characters and this decreases the number of
sounds which must be stored on the game  distribution medium (a compact disc or
Internet download in this case) and which must be loaded in the computer memory
while playing.Alone, this mode of categorizing sound offers little insight into the
function and meaning of sound, how sound is used in the game by the player or
how sound functions to form an acoustic ecology. In order to explore such
issues, it was necessary to employ and devise other taxonomic approaches. However, before proceeding to these other
possible forms of classification, consideration of the means of sound creation
and production at the game design stage is useful as it sheds some light on the
degree of interaction made possible in the FPS game which directly relates to
the player immersion within and participation in a game-related acoustic
ecology. In any computer game, sounds
heard during gameplay and from within the game environment are synthesized or
digitally recorded and stored as discrete audio samples. In all modern run and
gun FPS games, most, if not all such sounds consist of audio samples and this
is certainly the case forUrban Terror. Most
of these audio samples are sounded in response to player input, game status
(which, in most cases, is an indication of player activity) or bot activity in
games where bots are employed. A smaller
number of environment audio samples are under the control of the game engine
although their audification may be responsive to player location (by pan and
intensity) or where the game engine is capable of acoustic shading.Such audio samples (as described above) are labeled nomic auditory icons
by Gaver [8]. However, in the context of
games,  nomic is an ill-advised term to use, and so we prefer to call
them causal auditory icons. They bear a
strong degree of causality and indexicality to the actions they represent
because they are usually recordings of real-world analogues represented in the
game, hence the virtual causality of the sounds. Conversely, a symbolic auditory icon has a more arbitrary
mapping between the sound and the event it represents and within games aspiring
to a degree of realism (such as  Urban
Terror [9]) there are few such auditory icons.The abundance of recorded audio samples (as opposed to synthesized audio
samples), which may be described as causal auditory icons, combined with their
appropriate in-game use (in other words, they are causal sounds with a high
degree of virtual indexicality, for example, a recording of a shot-gun
is sounded each time a shot-gun is fired), is a good
indication of the level of realism the FPS game aspires to.Urban
Terror, which is usually described as a realism mod, is a prime example; of
the 601 diegetic audio samples available, the only synthetic audio samples (the
only symbolic auditory icons) are those related to game status events, such as
when a flag has been captured. This may be
compared to Quake III Arena [10] or Quake 4 [11] which, set as they are in a
more fantastical gamescape, have a greater proportion of symbolic auditory
icons representing not just game status events but also various audio samples
sounded by player input (such as those to do with power-ups and teleporters).


### 2.2. Diegetic Audio

It is possible to classify all audio samples as either diegetic or nondiegetic following film sound theory.  However, differences in the creation and resultant nature of the FPS game soundscape compared to the film soundscape require refinements of the term diegetic. As already noted, sounds in an FPS game consist of discrete audio samples and, unlike film, there is no complete game soundtrack that is stored on the distribution medium and played during gameplay. The FPS game soundscape, which forms a part of the acoustic ecology, is created in real-time through the agency of game engine actions (the sounding of game status feedback or ambient audio samples, for example) or through the agency of player input acting upon the discrete audio samples which form the soundscape's palette. Furthermore, with any playing of the game (even the same level), the resultant soundscape will be substantially different for the one player and, in a multiplayer game, the soundscape experienced by one player will also be substantially different to that experienced simultaneously by other players. It is for this latter reason that we define the terms ideodiegetic (those sounds that any one player hears) and telediegetic (those heard and responded to by a player—they are ideodiegetic for that player—but which have consequence for another player; they are telediegetic for the second player).  Furthermore, ideodiegetic sounds may be classified as kinediegetic (sounds initiated directly by that player's actions) and exodiegetic (all other ideodiegetic sounds).Of the class of diegetic audio samples, and in the context of a
multiplayer game, all global feedback sounds (such as game status messages) may
be classed as exodiegetic sounds. They
are ideodiegetic in that they are heard by all players (simultaneously) but initiated
by the game engine in response to significant events. All other audio samples may be ideodiegetic
or telediegetic depending upon context. These include environment sounds (which
are usually level-specific audio samples rather than game-specific audio
samples). Ideodiegetic sounds may be
classed as either kinediegetic or exodiegetic.
For the player who triggers them, the sounds are kinediegetic; they are
exodiegetic for other players. If such
sounds have consequences for other players who do not hear them (for example,
the blast of the shotgun which kills an enemy may draw others of her teammates
to that location which itself may provide opportunities for the opposing team),
they may also be classed as telediegetic for these other players.As has been noted by several writers [6, 7], sound in an FPS game may be
attended to in one of three modes: reduced listening, semantic listening, and
causal listening. Reduced listening, as noted by Stockburger [7], is little
used by experienced players. What these
writers do not suggest is that the mode of listening may change depending upon
context and experience. Furthermore, we identify a fourth mode, navigational
listening. This is required because of
the unique (compared to electro-acoustic music and film sound theory where the
original three modes were first described) abilities of the FPS player to move
her character around the 3-dimensional game world. In this mode, certain sounds may be used as
audio beacons helping to guide players, especially those new to the particular
game level, around the game world structures.


### 2.3. The FPS Game Soundscape

Schafer's [12] keynote sounds are, in this context, audio samples which form part of the sonic ambience and may not be directly triggered by the
player being, instead, sounded by the game engine. (They may be
triggered by other players but are judged by the one player to be distant and
of little interest and so form part of the general ambience of battle.) An example in Urban Terror is
the Bach organ fugue in the Abbey level or, in the same level but in a different location, the twittering of
birds. There is some ambiguity here that
is not captured by the brief descriptions of such environment or ambient sounds
in existing games studies literature. For example, the player does typically
have some kinaesthetic control over the sounding of these sounds; by simply moving away from a location, the sound may be attenuated to silence (and vice versa). Furthermore, if a keynote sound is a sound
which is not intended to be consciously listened to, merely forming the
background for more perceptually important sounds, the decision to consciously
attend to a sound or not is often a matter of player choice, indeed, musicians may respond to the organ fugue differently to non-musicians.A signal sound is a foregrounded sound which is designed to be
consciously attended to because it potentially contains important information
encoded within it. Most of the
game-specific audio samples inUrban
Terror may be classified as signal sounds when they are sounded in a
context which foregrounds them. Thus,
the loud, and therefore proximate, sounding of gunshot samples is worthwhile
paying attention to (particularly in the individual deathmatch game mode). However, if the sounds of battle are distant,
they may be classed as keynote sounds particularly if they are relatively
constant and the player's attention is directed elsewhere. All game status indicators and team radio
messages are signal sounds because, although they are as pervasive as keynote
sounds, they are usually louder and therefore more proximate and, in the case
of radio messages, have no reverberation, thereby foregrounding them through
the lack of depth cues.Soundmarks are identifying aural features of the acoustic ecology and may
be either signal sounds or keynote sounds which are consciously attended
to. Symbolic auditory icons, such as
flag status signals, are more likely to be uniquely identifying of an FPS game
than causal auditory icons because the latter are derived from recordings of
existing real-world, external sounds whereas the former reference the internal
game world.


### 2.4. Immersion through Sound

FPS game sounds may be categorized according to a variety of immersive
principles. Following Ermi and Mäyrä's
ideas [13], all FPS sounds can contribute to sensory immersion where the sounds
of the game world override those in the player environment. It should be noted, though, that the degree
of sensory immersion is dependent upon a range of factors beyond the control of
the game designers including the relative loudness of the two sets of sound and
the audio hardware used; one of the factors influencing the decision of most
FPS players to use headphones [14] is likely to be a greater sensory immersion.
Many sounds offer challenge-based immersion by requiring a response which
includes the use of both mental and kinetic skills. It is typically the case that these sounds
are ones which are produced by other players and they usually relate to actions
involving weapons. However, whilst most
level-specific environment sounds in Urban
Terror, for example, generally do not offer challenge-based immersion
possibilities, audio beacons require the navigational listening mode and,
therefore, mental skills.Sounds offering imaginative immersion possibilities are those which help
the player identify with her character and the game environment and
action. In the first case, FPS games
offer a range of character sounds, some of which may be classed as
proprioceptive sounds (such as the character breathing whose rate may vary
according to the exertions of the character) and which, with a high level of
immersion, may be seen as aural prostheses similar to the prosthetic limbs seen
receding into the screen. Exteroceptive
sounds affording imaginative immersion through identification include a range
of sounds which aid in contextualizing the player character within the
environment.McMahan categorises computer game elements as perceptual sureties,
surprises, or shocks [15]. The latter
militate against immersion in the game world by being external stimuli (or
errors in the game) that remind the player that this is just a game taking
place within the player’s real-world environment.
Sureties are mundane cues, expected details providing an experience
which is consistent with the rules and conventions of the game world. The creaking of a door as it opens and closes
or the footsteps of a player moving around are aural examples of this. Surprises, according to McMahan, consist of three types: attractors (inviting the player to do something); connectors (helping player orientation) and retainers (causing the player to linger in game world locations).A variety of sounds in FPS games fulfil these requirements. Indeed, any sound inviting an active response
may be said to be an attractor. Thus,
the sound of gunfire in the distance may tempt the player to investigate and team radio messages detailing enemy actions invite a response on the part of
the team player. Many sounds,
particularly environment sounds, function as connectors and they are often
attended to in the navigational listening mode.
Locational and depth properties are important parameters of sounds
functioning as connectors. Although, at
first playing of the game, the player may derive enjoyment out of certain
sounds and so may linger in a particular location in order to hear more, the
nature of the FPS game is such that more-or-less continual movement is usually
required of the player to seek out or avoid the enemy or to attack the enemy base and so, for the experienced player, no sounds in the FPS game may be said
to be retainers.We propose four terms to describe the spatializing and temporal
affordances offered by FPS game sounds.
The perception of a variety of spaces is one of the main contributing
factors of FPS sounds to the perception of, and immersion within, the game world. In terms of our phrase resonating space,
there is a real resonating space, which is the acoustic volume enveloping and
morphing around the player, and a virtual resonating space, matching, through a
process of synchresis, the illusory visual space depicted on the screen (other
virtual spaces may be identified as separate volumes within the game world), the perception of which is created by parameters of sound such as
localization, depth cues and reverberation.
Such cues may be processed in real time (acoustic shading) with more
sophisticated game audio engines or they may be encoded into the audio samples
on the distribution medium. Sounds
providing this affordance are choraplasts.
Sounds may also function as topoplasts where they create the perception
of paraspaces such as locations in the game.
Additionally, sound may provide the affordance of the perception of time
passing or of a particular temporal period in the past, present or future. The former are chronoplasts and,
because sound is vectored through time, that is, it takes time to hear a sound,
all sounds have a basic chronoplastic function (in addition to any explicit
function they may have in this area).
The latter are aionoplasts and weapon sounds typically have this
function setting the game, for example, in the modern era rather than the
mediæval age.


## 2.1. Audio Sample Categorization

Our first approach in constructing a taxonomy of FPS game sounds was one
that perused the classification of audio samples as found on either the
distribution medium or on the installation drive of the product itself. This
initial approach provided useful insights into the sound designer's
classification system which itself may be extrapolated to the meaning that is
intended for particular sounds. While this approach has already been used to
account for game sound within games studies community through reference to
character, interactable or environment sounds, for example, [5–7], none of
the literature explicitly examines the distribution or installation media for
further clues as to the sound designers’ intentions. At the very least, our
approach revealed a division between diegetic and nondiegetic sounds as there
is typically a separate directory for music or menu interface audio samples as
opposed to other audio samples which,  themselves, may be subclassified into
character, interactable, environment or feedback audio samples.
Thus, of the 607 base audio samples in  Urban Terror (game-specific audio samples as opposed to
level-specific audio samples), fully 601 are available to be used during
gameplay with the remaining six being the menu music (one) and menu interface
sounds (five). The 601 audio samples
are, therefore, diegetic whilst the remaining six are nondiegetic.The game designer-constructed organization of audio samples inUrban Terror is illuminating in several
respects. Firstly, it is an indication
of how the game code deals with sound and its relationship to a variety of
characters, objects, and locations within the game. Sounds that players’ characters create as they
move, fire, or taunt are separated from environment sounds which are part of a
location; sounds of interactable objects are separated from the sounds non-interactable objects make, and diegetic sounds are separated from nondiegetic sounds. Secondly, the sheer number of sounds is an
indication of the importance of sound to the game experience. Thirdly, this
organization of sound indicates some of the technical limitations of the game,
namely in the areas of media storage and computer memory. As an example, some audio samples of
footsteps are shared between the characters and this decreases the number of
sounds which must be stored on the game  distribution medium (a compact disc or
Internet download in this case) and which must be loaded in the computer memory
while playing.Alone, this mode of categorizing sound offers little insight into the
function and meaning of sound, how sound is used in the game by the player or
how sound functions to form an acoustic ecology. In order to explore such
issues, it was necessary to employ and devise other taxonomic approaches. However, before proceeding to these other
possible forms of classification, consideration of the means of sound creation
and production at the game design stage is useful as it sheds some light on the
degree of interaction made possible in the FPS game which directly relates to
the player immersion within and participation in a game-related acoustic
ecology. In any computer game, sounds
heard during gameplay and from within the game environment are synthesized or
digitally recorded and stored as discrete audio samples. In all modern run and
gun FPS games, most, if not all such sounds consist of audio samples and this
is certainly the case forUrban Terror. Most
of these audio samples are sounded in response to player input, game status
(which, in most cases, is an indication of player activity) or bot activity in
games where bots are employed. A smaller
number of environment audio samples are under the control of the game engine
although their audification may be responsive to player location (by pan and
intensity) or where the game engine is capable of acoustic shading.Such audio samples (as described above) are labeled nomic auditory icons
by Gaver [8]. However, in the context of
games,  nomic is an ill-advised term to use, and so we prefer to call
them causal auditory icons. They bear a
strong degree of causality and indexicality to the actions they represent
because they are usually recordings of real-world analogues represented in the
game, hence the virtual causality of the sounds. Conversely, a symbolic auditory icon has a more arbitrary
mapping between the sound and the event it represents and within games aspiring
to a degree of realism (such as  Urban
Terror [9]) there are few such auditory icons.The abundance of recorded audio samples (as opposed to synthesized audio
samples), which may be described as causal auditory icons, combined with their
appropriate in-game use (in other words, they are causal sounds with a high
degree of virtual indexicality, for example, a recording of a shot-gun
is sounded each time a shot-gun is fired), is a good
indication of the level of realism the FPS game aspires to.Urban
Terror, which is usually described as a realism mod, is a prime example; of
the 601 diegetic audio samples available, the only synthetic audio samples (the
only symbolic auditory icons) are those related to game status events, such as
when a flag has been captured. This may be
compared to Quake III Arena [10] or Quake 4 [11] which, set as they are in a
more fantastical gamescape, have a greater proportion of symbolic auditory
icons representing not just game status events but also various audio samples
sounded by player input (such as those to do with power-ups and teleporters).


## 2.2. Diegetic Audio

It is possible to classify all audio samples as either diegetic or nondiegetic following film sound theory.  However, differences in the creation and resultant nature of the FPS game soundscape compared to the film soundscape require refinements of the term diegetic. As already noted, sounds in an FPS game consist of discrete audio samples and, unlike film, there is no complete game soundtrack that is stored on the distribution medium and played during gameplay. The FPS game soundscape, which forms a part of the acoustic ecology, is created in real-time through the agency of game engine actions (the sounding of game status feedback or ambient audio samples, for example) or through the agency of player input acting upon the discrete audio samples which form the soundscape's palette. Furthermore, with any playing of the game (even the same level), the resultant soundscape will be substantially different for the one player and, in a multiplayer game, the soundscape experienced by one player will also be substantially different to that experienced simultaneously by other players. It is for this latter reason that we define the terms ideodiegetic (those sounds that any one player hears) and telediegetic (those heard and responded to by a player—they are ideodiegetic for that player—but which have consequence for another player; they are telediegetic for the second player).  Furthermore, ideodiegetic sounds may be classified as kinediegetic (sounds initiated directly by that player's actions) and exodiegetic (all other ideodiegetic sounds).Of the class of diegetic audio samples, and in the context of a
multiplayer game, all global feedback sounds (such as game status messages) may
be classed as exodiegetic sounds. They
are ideodiegetic in that they are heard by all players (simultaneously) but initiated
by the game engine in response to significant events. All other audio samples may be ideodiegetic
or telediegetic depending upon context. These include environment sounds (which
are usually level-specific audio samples rather than game-specific audio
samples). Ideodiegetic sounds may be
classed as either kinediegetic or exodiegetic.
For the player who triggers them, the sounds are kinediegetic; they are
exodiegetic for other players. If such
sounds have consequences for other players who do not hear them (for example,
the blast of the shotgun which kills an enemy may draw others of her teammates
to that location which itself may provide opportunities for the opposing team),
they may also be classed as telediegetic for these other players.As has been noted by several writers [6, 7], sound in an FPS game may be
attended to in one of three modes: reduced listening, semantic listening, and
causal listening. Reduced listening, as noted by Stockburger [7], is little
used by experienced players. What these
writers do not suggest is that the mode of listening may change depending upon
context and experience. Furthermore, we identify a fourth mode, navigational
listening. This is required because of
the unique (compared to electro-acoustic music and film sound theory where the
original three modes were first described) abilities of the FPS player to move
her character around the 3-dimensional game world. In this mode, certain sounds may be used as
audio beacons helping to guide players, especially those new to the particular
game level, around the game world structures.


## 2.3. The FPS Game Soundscape

Schafer's [12] keynote sounds are, in this context, audio samples which form part of the sonic ambience and may not be directly triggered by the
player being, instead, sounded by the game engine. (They may be
triggered by other players but are judged by the one player to be distant and
of little interest and so form part of the general ambience of battle.) An example in Urban Terror is
the Bach organ fugue in the Abbey level or, in the same level but in a different location, the twittering of
birds. There is some ambiguity here that
is not captured by the brief descriptions of such environment or ambient sounds
in existing games studies literature. For example, the player does typically
have some kinaesthetic control over the sounding of these sounds; by simply moving away from a location, the sound may be attenuated to silence (and vice versa). Furthermore, if a keynote sound is a sound
which is not intended to be consciously listened to, merely forming the
background for more perceptually important sounds, the decision to consciously
attend to a sound or not is often a matter of player choice, indeed, musicians may respond to the organ fugue differently to non-musicians.A signal sound is a foregrounded sound which is designed to be
consciously attended to because it potentially contains important information
encoded within it. Most of the
game-specific audio samples inUrban
Terror may be classified as signal sounds when they are sounded in a
context which foregrounds them. Thus,
the loud, and therefore proximate, sounding of gunshot samples is worthwhile
paying attention to (particularly in the individual deathmatch game mode). However, if the sounds of battle are distant,
they may be classed as keynote sounds particularly if they are relatively
constant and the player's attention is directed elsewhere. All game status indicators and team radio
messages are signal sounds because, although they are as pervasive as keynote
sounds, they are usually louder and therefore more proximate and, in the case
of radio messages, have no reverberation, thereby foregrounding them through
the lack of depth cues.Soundmarks are identifying aural features of the acoustic ecology and may
be either signal sounds or keynote sounds which are consciously attended
to. Symbolic auditory icons, such as
flag status signals, are more likely to be uniquely identifying of an FPS game
than causal auditory icons because the latter are derived from recordings of
existing real-world, external sounds whereas the former reference the internal
game world.


## 2.4. Immersion through Sound

FPS game sounds may be categorized according to a variety of immersive
principles. Following Ermi and Mäyrä's
ideas [13], all FPS sounds can contribute to sensory immersion where the sounds
of the game world override those in the player environment. It should be noted, though, that the degree
of sensory immersion is dependent upon a range of factors beyond the control of
the game designers including the relative loudness of the two sets of sound and
the audio hardware used; one of the factors influencing the decision of most
FPS players to use headphones [14] is likely to be a greater sensory immersion.
Many sounds offer challenge-based immersion by requiring a response which
includes the use of both mental and kinetic skills. It is typically the case that these sounds
are ones which are produced by other players and they usually relate to actions
involving weapons. However, whilst most
level-specific environment sounds in Urban
Terror, for example, generally do not offer challenge-based immersion
possibilities, audio beacons require the navigational listening mode and,
therefore, mental skills.Sounds offering imaginative immersion possibilities are those which help
the player identify with her character and the game environment and
action. In the first case, FPS games
offer a range of character sounds, some of which may be classed as
proprioceptive sounds (such as the character breathing whose rate may vary
according to the exertions of the character) and which, with a high level of
immersion, may be seen as aural prostheses similar to the prosthetic limbs seen
receding into the screen. Exteroceptive
sounds affording imaginative immersion through identification include a range
of sounds which aid in contextualizing the player character within the
environment.McMahan categorises computer game elements as perceptual sureties,
surprises, or shocks [15]. The latter
militate against immersion in the game world by being external stimuli (or
errors in the game) that remind the player that this is just a game taking
place within the player’s real-world environment.
Sureties are mundane cues, expected details providing an experience
which is consistent with the rules and conventions of the game world. The creaking of a door as it opens and closes
or the footsteps of a player moving around are aural examples of this. Surprises, according to McMahan, consist of three types: attractors (inviting the player to do something); connectors (helping player orientation) and retainers (causing the player to linger in game world locations).A variety of sounds in FPS games fulfil these requirements. Indeed, any sound inviting an active response
may be said to be an attractor. Thus,
the sound of gunfire in the distance may tempt the player to investigate and team radio messages detailing enemy actions invite a response on the part of
the team player. Many sounds,
particularly environment sounds, function as connectors and they are often
attended to in the navigational listening mode.
Locational and depth properties are important parameters of sounds
functioning as connectors. Although, at
first playing of the game, the player may derive enjoyment out of certain
sounds and so may linger in a particular location in order to hear more, the
nature of the FPS game is such that more-or-less continual movement is usually
required of the player to seek out or avoid the enemy or to attack the enemy base and so, for the experienced player, no sounds in the FPS game may be said
to be retainers.We propose four terms to describe the spatializing and temporal
affordances offered by FPS game sounds.
The perception of a variety of spaces is one of the main contributing
factors of FPS sounds to the perception of, and immersion within, the game world. In terms of our phrase resonating space,
there is a real resonating space, which is the acoustic volume enveloping and
morphing around the player, and a virtual resonating space, matching, through a
process of synchresis, the illusory visual space depicted on the screen (other
virtual spaces may be identified as separate volumes within the game world), the perception of which is created by parameters of sound such as
localization, depth cues and reverberation.
Such cues may be processed in real time (acoustic shading) with more
sophisticated game audio engines or they may be encoded into the audio samples
on the distribution medium. Sounds
providing this affordance are choraplasts.
Sounds may also function as topoplasts where they create the perception
of paraspaces such as locations in the game.
Additionally, sound may provide the affordance of the perception of time
passing or of a particular temporal period in the past, present or future. The former are chronoplasts and,
because sound is vectored through time, that is, it takes time to hear a sound,
all sounds have a basic chronoplastic function (in addition to any explicit
function they may have in this area).
The latter are aionoplasts and weapon sounds typically have this
function setting the game, for example, in the modern era rather than the
mediæval age.


## 3. The Implications for Game Engine Design

As previously stated, modern FPS games make use of audio samples that may
be treated as causal auditory icons (the most common form and typically
recordings of real-world events) or symbolic auditory icons (more common in
games with a less realist scenario).
Whilst the use of causal auditory icons provides quite accurate sonic representations of real-world artifacts in the game world, that use comes at a
price that is calculated in memory, both distribution medium storage and game
system random access memory (RAM).
Assuming a 16 bit, monophonic, 44.1 kHz game audio system, 100 one-second
audio samples require a total of almost 9 MB of storage.This may not seem expensive with the technology available in 2007. However, there are several factors conspiring
to push this cost up. Firstly, many game
audio samples are longer than one second particularly if they are vocal audio
samples. Secondly, and perhaps more
importantly, the game designers’ desire to provide the player with an
increasingly rich sonic environment requires, as an initial solution, the
provision of yet more samples. Unlike
sound design for a film, game sound designers work to a non-linear script and “it is not possible to make every gunshot sound unique if you do not know how many gunshot sounds are needed!” [16]. Games
such as Urban Terror (based on the Quake III Arena game engine) must
therefore strike a balance between a wish to provide an audio sample for every
sonic possibility (an infinite number) and paying regard to storage and memory
requirements (Urban Terror is
typically provided as an Internet download).Later game engines, such as that used byHalf-Life 2 [17], use acoustic shading techniques. This provides a part solution to the audio
sample memory problem by real-time processing of audio samples with
reverberation that approximates the virtual acoustic properties of the character’s environment. This is also a step towards solving the non-linearity enigma as expressed by Boyd because any one audio sample does multiple duty by having different reverberant characteristics in different locations of the game.
However, acoustic shading of audio samples still requires a high use of
memory and it provides solely a reverberation solution without taking account
of other encodings possible in sound (its emotive aspects or the direct sound
source, for example).Gaver notes that, using what he callseveryday listening, sounds
are usually described by one or two of their salient characteristics (object and action); a metallic clang, a wooden thump, a glass-like shattering, for example [18]. Indeed, he provides
algorithms to show it is possible to conceptualize and synthesize sound
according to these characteristics rather than directly by the use of
properties such as frequency and intensity; a top-down model as opposed to a
bottom-up model. The resultant
caricature sounds should then provide the minimum information required to
enable at least an approximate identification of both source object and action.Populating an FPS game with such caricature, synthesized sounds would seem
to militate against realism and the requirements of a rich, immersive player
experience. However, there is evidence
to suggest that, where sound is concerned, a reduced realism may be all that is
required to achieve the desired immersive effect [19–21]; a simulation, rather than emulation, that is based upon convention, expectation, and
caricature. Certainly, in film, this is
how many sound FX work; recordings are made (and enhanced in post-production)
of sources and actions that are not necessarily the same as those depicted on
the screen. However, by matching the
main characteristics of the recorded sound to the expected sound of the screen
depiction (such expectation often being the result of cinematic convention) and
synchronizing sound and visual action, the audience is persuaded that that
screen event really has produced that sound.The suggestion, then, is that real-time synthesis of sound in digital
games may prove to be of benefit in dealing with the twin challenges of memory and non-linear practice without imperiling the immersive experience and, indeed,
perhaps enriching it. The ideal scenario
might be to have a combination of synthesis and audio samples because (despite
advancements in synthesis) some sounds (such as the human voice) are still best
represented by audio samples. Synthesis
may be of use for the more symbolic auditory icons, fast-repeating sounds (such
as gunfire), and background, keynote sounds and may be enhanced through any
acoustic shading the game engine offers.


## 4. Conclusion

All sounds, or the use of some sounds, in the FPS game contribute in some way to player immersion in the acoustic ecology and it is this immersion within (and the player's creative participation in the game's acoustic ecology) that, in large part, affords the perception of immersion in the FPS game world. Thus, the player is physically immersed in
the real resonating space and, through kinaesthetic techniques and the ability
to trigger a range of sounds through various input methods, is drawn into the
virtual resonating space that is then synchretically mapped to the visual game
world and activity that are represented either on- or off-screen.The model in Figure1 exhibits all the elements of the conceptual
framework described thus far. Because it
is a model of an acoustic ecology, it
importantly shows relationships between the player (the listener) and
soundscape. As it is a model of the acoustic ecology of the FPS run and gun game, though, it includes a variety of components and relationships which are unique to digital games (some of which may be unique to the subgenre) such as the game engine, image, a range of spatial and immersive elements and perceptual factors. Furthermore, because this is a model of a multiplayer game acoustic ecology, it also includes the game server and other players and their soundscapes. (For clarity, only one other player and soundscape are shown here.)Although the
conceptual framework and the model are focused on FPS games as their paradigm,
it may well be the case that they (or aspects of them) may also be used to
analyze the wider area of digital game sound in the future with the caveat that
much further research and testing (using different digital game genres) is
required. Furthermore, it is suggested
that the conceptual framework and model may prove to be of use in the design of
the audio component of game engines by supporting the notion that real-time
synthesis of sound in the game is a valid means of providing an immersive
acoustic ecology.



---

*Source: 720280-2008-01-22.xml*