Supramodal Representations in Melodic Perception

Supramodal Representations in Melodic Perception Ahnate Lim (ahnate@hawaii.edu) Department of Psychology, University of Hawaii at Manoa 2530 Dole Street, Honolulu, HI 96822 USA Leonidas A. A. Doumas (alex.doumas@ed.ac.uk) Department of Psychology, University of Edinburgh 7 George Square, Edinburgh, Midlothian EH8 9JZ, UK Scott Sinnett (ssinnett@hawaii.edu) Department of Psychology, University of Hawaii at Manoa 2530 Dole Street, Honolulu, HI 96822 USA Abstract explicit learning of grammatical structures (Rohrmeier & Rebuschat, 2012), and complex event sequencing (Tillmann, 2012), to name but a few. Other approaches have sought to uncover links between musical training and performance in other domains such as mathematics, language, spatial- temporal abilities, and verbal memory (for a summary, see Rauscher, 2003). Thus it is clear that gaining a better understanding of the underlying processes and building blocks in music perception can aid towards understanding the underlying mechanisms and resources used in other cognition domains as well. Music is highly relational and in this manner shares much in common with other characteristically human behavior. While this may suggest that the processes used in music perception could be domain general, the nature and flexibility of these representations remain less understood. If the underlying representations and manipulations required for perceiving music overlap with those in other cognitive domains, it should be fairly easy to map such representations across domains. This hypothesis was tested and supported using a novel experiment with melodic stimuli in the auditory modality and analogous visual sequential stimuli (Gabor sequences) in the visual modality. Testing for transfer across the two modalities and for the two types of representations (contour and intervallic) was done through four counterbalanced conditions. Cross-modal mapping was successful in three out of the four conditions, implying general flexibility of representational transfer. Implications for representational flexibility, sequential learning and future studies are discussed. The Melody Keywords: Melodic perception; relation learning; cross- modal mapping; representations; categorization. Introduction As research techniques for studying human behavior and the brain have evolved, the underlying processes of music perception have continued to fascinate cognitive scientists and spur much research. Recent advances in neuroscience for example, have shown that the simple act of perceiving music involves distributed activity throughout the brain, including diverse regions such as Broca’s area (Fadiga, Craighero, & D’Ausilio, 2009), the pre-frontal cortex (Bengtsson, Csikszentmihalyi, & Ullen, 2007), as well as the amygdala (Limb, 2006). Indeed, many if not all of these regions are used in other tasks such as speech processing, for example (Koelsch et al., 2004). Given the integrated and distributed neurological underpinnings of music, it should come as no surprise that a host of relationships has been made between music and other areas of cognition. Aside from the common comparisons made between music and language, many have suggested that music may also help to aide us in understanding other behavior such as domain- general aesthetic preferences (Marcus, 2012), implicit and One of the most fundamental and salient aspects of music is the melody. Simple melodies consist of discrete units or notes, where each note is characterized by a pitch, or fundamental frequency (e.g., Hertz value). The core strategy humans use to process and store familiar melodies is through a relative pitch code (Attneave & Olson, 1971; Page, 1994). This relative pitch code stores the pitch sequence of a melody in terms of the relations or intervals (specific frequency differences) between each note. For example, the song Happy Birthday is immediately recognizable due to the unique intervals between each of the notes. That is, one immediately recognizes this song regardless of whether it starts on a low or high note due to the unique intervallic pattern between all subsequent notes. There is considerable evidence on the use of relative pitch information in adults through both behavioral (Dowling, 1978, 1988) as well as neuroimaging studies (Fujioka, Trainor, Ross, Kakigi, & Pantev, 2004; Trainor, McDonald, & Alain, 2002). In addition to relative pitch, the contour (general shape, or sequence of up and down movements in frequencies from note to note) is another characteristic upon which melodies can be categorized. Given the existence of these characteristics, the question remains as to how they contribute to a listener’s mental representation of a melody. Note that while a melody with the same contour as Happy Birthday but with a different intervallic sequence would be perceived as a completely different song, it would still have the same general “shape”, or up and down pattern. Although