THE ROLE OF MUSIC COMMUNICATION IN CINEMA
(2004)

[Authors’ note: This paper is an abbreviated version of a chapter included in a forthcoming book entitled Music Communication (D. Miell, R. MacDonald, & D. Hargreaves, Eds.), to be published by Oxford University Press.] Past research leaves no doubt about the efficacy of music as a means of communication. In the following pages, after presenting a general model of music communication, the authors will introduce models – both empirical and theoretical – of film music perception and the role of music in film, referencing some of the most significant research investigating the relationship between sound and image in the cinematic context. We shall then enumerate the many ways in which the motion picture soundtrack can supplement, enhance, and expand upon the meaning of a film’s narrative. The relationship between the auditory and visual components in cinema is both active and dynamic, affording a multiplicity of possible relations than can evolve – sometimes dramatically – as the narrative unfolds. This paper will take a cognitive approach to the study of musical communication in cinema. As a result, much credence will be given to the results of empirical research investigating human cognitive processing in response to the motion picture experience. In conclusion, the present authors will argue for a more inclusive definition of the term “film music” than that utilized or implied in previous publications. In our view, film music is one component of a sonic fabric that includes the musical score, ambient sound, dialogue, sound effects, and silence. The functions of these constituent elements often overlap or interact with one another, creating a harmonious counterpoint to the visual image. 1. A MODEL OF MUSIC COMMUNICATION Many studies have investigated various aspects of musical communication as a form of expression (Bengtsson & Gabrielsson, 1983; Clarke, 1988; Clynes, 1983, Gabrielsson, 1988, Seashore, 1967/1938; Senju & Ohgushi, 1987; Sundberg, 1988; Sundberg et al., 1983). A tripartite communication model was proposed by Campbell and Heller (1980), consisting simply of a composer, performer, and listener. Using this previous model as a foundation, Kendall and Carterette (1990) elegantly expanded upon this model of music communication, detailing a process involving multiple states of coding, decoding, and recoding. Kendall and Carterette suggest that this process involves the “grouping and parsing of elementary thought units” (p. 132), these “thought units” (metasymbols) are mental representations involved in the process of creating, performing, and listening to musical sound. 2. MODELS OF FILM MUSIC COMMUNICATION 2.1 Empirical Evidence Several researchers have proposed models specific to the perception and cognition of music within the cinematic context. Initiating this systematic effort, Marshall and Cohen’s (1988) bipartite “congruence-associationist” model suggests that the meaning of a film is altered by the music as the result of two complex cognitive processes. Based upon subject responses, the researchers determined that musical sound directly effects subject ratings on the Potency (strong-weak) and Activity (active-passive) dimensions, while the Evaluative dimension (good-bad) relies on the degree of congruence between the audio and visual components on all three dimensions, as determined by a “comparator” component. The second part of the model describes how musical meaning is ascribed to the film. Marshall and Cohen claim that attention is directed to the overlapping congruent meaning of the music and the film. Referential meanings associated with the music are ascribed to the overlapped (congruent) audio-visual components upon which attention is focused. As a result, “the music alters meaning of a particular aspect of the film” (1988, p. 109). Marshall and Cohen also acknowledge the important role played by temporal characteristics of the sound and image, stating that “the assignment of accent to events will affect retention, processing, and interpretation” (1988, p. 108). Incorporation of this important component of the developing model was provided by Lipscomb and Kendall’s (1994) Film Music Paradigm, in which two implicit processes are considered as the basis for whether attentional focus is shifted to the musical component or whether it is likely to remain at the subconscious – cognitively “inaudible” – level. The authors suggested that these two implicit processes include an association judgment (similar to Marshall and Cohen’s assessment of “congruence”) and an evaluation of the accent structure relationship between the auditory and visual components. Based on the results of a series of three experiments utilizing stimuli ranging from extremely simple, single-object animations to actual movie excerpts, Lipscomb (1995) determined that the role of the two implicit judgments appears to be dynamic such that, with simple stimuli (such as that used in Lipscomb, 1995, Experiment 1 and Marshall & Cohen, 1988), accent structure alignment plays a dominant role. As the stimuli become more complex (e.g., multi-object animations and actual movie excerpts) the primary determinant of meaning in the auditory domain appears to shift to the associational judgment, with the accent structure alignment aspect receding to a supporting role, i.e., focusing audience attention on certain aspects of the visual image (Boltz, 2001). The most complex and fully developed model of film music perception proposed to date is Cohen’s (2001) “congruence-associationist framework for understanding film-music communication” (p. 259; see Figure 1). This multi-stage model attempts to account for meaning derived from the spoken narrative, visual images, and musical sound. Level A represents bottom-up processing based on physical features derived from input to each perceptual modality. Level B represents the determination of cross-modal congruence, based on both semantic (associational) and syntactic (temporal) grouping features. Level D represents top-down processing, determined by an individual’s past experience and the retention of that experience in long term memory. According to this model, the input from levels B (bottom-up) and D (top-down) meet in the observer’s conscious mind (level C), where information is prepared for transfer to short term memory. In its details, clearly documented in Cohen (2001), this model is based on an assumption of visual primacy, citing several studies that have suggested a subservient role for the auditory component (Bolivar et al., 1994; Driver, 1997; Thompson et al., 1994). Though a common assumption throughout the literature, the present authors would like to express reservation about this assumption and suggest that additional research is required before such a claim can be supported definitively. Figure 1. Cohen’s “congruence-associationist framework. 2.2 Theoretical Evidence Richard Wagner, creator of the idealized Gesamtkunstwerk in the form of the 19 century music drama, claimed that “as pure organ of the feeling, [music] speaks out the very thing which word speech in itself can not speak out ... that which, looked at from the standpoint of our human intellect, is the unspeakable” (Wagner 1849/1964, p. 217). According to Suzanne K. Langer, “music has all the earmarks of a true symbolism, except one: the existence of an assigned connotation” and, though music is clearly a symbolic form, it remains an “unconsummated symbol” (1942, p. 240). In order for a film to make the greatest possible impact, there must be an interaction between the verbal dialogue (consummated symbol), the cinematic images (also, typically, a consummated symbol), and the musical score (unconsummated symbol). To answer the question “How does music in film narration create a point of experience for the spectator?,” Gorbman (1987) suggests three methods by which music can “signify” in the context of a narrative film. Purely musical signification results from the highly coded syntactical relationships inherent in the association of one musical tone with another. Patterns of tension and release provide a sense of organization and meaning to the musical sound, apart from any extramusical association that might exist. Cultural musical codes are exemplified by music that has come to be associated with a certain mood or state of mind. These associations have been further canonized by the Hollywood film industry into certain conventional expectations – implicitly anticipated by enculturated audience members – determined by the narrative content of a given scene. Finally, cinematic codes influence musical meaning merely due to the placement of musical sound within the filmic context. Opening credit and end title music illustrate this type of signification, as well as recurring musical themes that come to represent characters or situations within the film. There is a commonly held belief that film music is not to be heard (Burt, 1994; Gorbman, 1987). Instead, it is believed to fulfill its role in communicating the underlying psychological drama of the narrative at a subconscious level (Lipscomb, 1989). There is, however, certain music that is intended to be heard by the audience as part of the cinematic diegesis, i.e., “the narratively implied spatiotemporal world of the actions and characters” (Gorbman 1987, p. 21). This “world” includes, naturally, a sonic component. Therefore, all sounds that are understood to be heard by characters in the narrative – including music – are referred to as diegetic, while those that are not part of the diegesis (e.g., the orchestral score) are referred to as nondiegetic. This would suggest that diegetic music is more likely to be processed at the conscious level while nondiegetic music might remain at the subconscious level, though research is needed to determine whether this is true, in fact. It is worth noting also, that the source of diegetic sound can be either seen or unseen. Michel Chion (199