title: Children's Matching of Melodies and Their Visual Representations
Creators: name: Nisbet, Robert Stevenson
Publisher: name: Griffith University
Year: 1998
Subjects: subject: Melody
subject: Auditory perception
subject: Visual perception
subject: Visual-auditory

description: The matching of melodies with their visual representations is predicated on the ability to relate changes of melodic pitch with changes of spatial position, usually in a vertical direÃ³tion. Previous studies have investigated the matching process in terms of factors such as melodic tonality, contour complexity, presentation rate (in notes per second), modality (visual &amp; auditory) and musical training. This investigation sought to answer a number of questions which arose from reflections on the results of such prior work - questions about musical training and the related notion of musical ability; questions about the role, if any, of mathematical ability, given the graphical nature of the visual materials; questions about the role of the type of visual materials; and questions about the strategies used in the matching process. The investigation was carried out with school children aged 10 and 11 years. The first three experiments in this investigation investigated the effects associated with the nature of the visual materials, along with the effects of ability factors (musical ability, mathematical ability, and simultaneous and successive cognitive processing ability). The last two experiments examined the strategies that children used in the auditory/visual matching process, and whether analytical or holistic processing took precedence during the matching process. Experiment 1 investigated cross-modal (visual-auditory, auditory-visual) and intramodal (visual-visual, auditory-auditory) matching of short melodies and line graphs, and showed that the matching process was influenced by visual/graphical factors as well as auditory/melodic factors in that matching with conventional format graphs (time on the horizontal axis) was superior to matching with non-conventional graphs (time on the vertical axis). It was also found that intramodal tasks were superior to cross-modal tasks, and within these categories, visual-first tasks were superior to auditory-first tasks. This result was at variance with the claim in the literature (the contour abstraction hypothesis) that visual-first matching tasks were superior to auditory-first matching tasks across intramodal and cross-modal categories. Limited positive effects of musical ability and musical training were observed but a close relationship between the two factors was noted. A positive effect of mathematical ability was revealed also, and evidence relating to type of visual format pointed towards the effect being attributable to mathematics experience, rather than just mathematical ability. The effect of visual factors on the matching process was further investigated in Experiment 2 with the use of music notation. Again, it was demonstrated that the process was influenced by visual as well as auditory factors. Also, the modality effects of the first experiment were observed although performance levels with tasks involving visual materials indicated that matching melodies with music notation was more difficult than graphic notation for the children. Results for the visual-to-melody condition confirmed previous claims that the process of reading music is more complex than the cross-modal transfer of auditory and visual information. Musical ability and music experience were positive factors in the melody-to-visual condition. However, overall, the effects of ability factors were overshadowed by the effects of modality condition and complexity. The fact that musically experienced children did not outperform their inexperienced counterparts suggests that, generally speaking, children who learn music find the task of reading music notation a difficult exercise. The third experiment examined the matching of melodies and their visual representations with respect to abilities in simultaneous and successive cognitive processing. Simultaneous cognitive processing was a significant positive factor in the performance of tasks in the two visual-first modality conditions (visual-to-visual and visual-to-melody), whereas successive cognitive processing was a significant positive factor in all four modality conditions. The results indicate that simultaneous processing was involved in the ability to inter-relate features not only of the visual materials, but also of the short melodies in the case of low complexity examples. The effect of successive cognitive processing ability was attributed firstly to the processing of the notes of a melody as elements of a chain-like progression, and secondly to the consecutive presentation of the two stimuli to the children. Experiment 3 also confirmed the assertion made in consideration of the results of Experiments 1 and 2, that music notation is more complex visually than line graphs, and thus requires a higher level of simultaneous processing to abstract the significant perceptual and symbolic features. The features of the melodic and visual materials and their associated processing strategies were the major issues investigated in Experiments 4 and 5. Children's recognition of differences in the materials at the local and global levels was examined with respect to analytical and global processing, and presentation rate. It was found that global processing took precedence in this context, confirming Navon's (1977, 1981) global precedence hypothesis. Global information with respect to overall contour was accessed more easily and more quickly than local information in the form of interval sizes. Attention to these local and global properties was able to be manipulated by mode of instruction (as predicted from the results of Palmer, 1990), such that detection of local differences was reinforced by instructions to act analytically and hindered by instructions to act globally. Similarly, detection of global differences was reinforced by instructions to act holistically and hindered by instructions to act analytically, notably at the faster presentation rate. Decreasing the presentation rate led to a reduction of cohesion of local and global melodic information in terms of the children's perception of relative interval sizes. Although the children recognised global-change items reasonably well, they incorrectly reported more differences for global-change items compared to local-change items. The results from this investigation indicate that the form of the visual representation of musical melodies has a significant influence on matching-task performance levels, even for musically trained children. It appears that the more perceptual and the less abstract and symbolic a visual representation system is, the more easily children will be able to perform the melody/visual matching. It is clear that, in general, children find the task of reading music notation difficult and even those who have had two or three years of formal musical experience would not be able to rely on it in their music lessons, rehearsals or performances to any great extent. Reading music notation requires an ability to process symbolic as well as perceptual information, which, in turn, requires a high level of simultaneous cognitive processing ability. Reading notation also requires an ability to judge the size of musical intervals and to match their notated form with the aural interval. The research conducted in my project complements the work of Morrongiello and Roes (1990) in identifying the influence of visual factors (such as graphical format and system of music notation) as well as auditory factors in auditory/visual matching task performance. Although the results confirmed the existence of presentation-rate and modality effects established by Balch &amp; Muscatelli (1986), the patterns of results from my project showed that the contour abstraction hypothesis does not necessarily hold in other auditory/visual contexts. It appears that orders of performance of the various modality conditions depend also on the nature and complexities of the auditory and visual materials as well as modality condition. Although positive effects of musical ability and mathematics ability on auditory/visual matching were demonstrated, it was clear that these effects could be attributed to the closely-associated notions of music experience and mathematics experience respectively. Nevertheless, the demonstrated positive effects of simultaneous and successive cognitive processing abilities in the various conditions of auditory/visual matching suggest that the ability factors are more generic than the disciplines, and lend support for the Luria model of cognitive processing (Das, Kirby &amp; Jarman, 1979; Naglieri &amp; Das, 1990). The investigation of further aspects of processing showed that (i) global processing took precedence over analytical processing, thus confirming Navon's (1977, 1981) global precedence hypothesis in the context of auditory/visual matching, mid (ii) that attention to the local and global properties could be manipulated by instructions, as predicted from the work of Palmer (1990). A number of areas for further research arise from the results of this investigation. One is the role of instructions (global and local) in the matching of short melodies with music notation, focussing on the conditions which facilitate the recognition of local features such as inter-note intervals. It has been shown firstly that music notation is more complex visually than line graphs, secondly that children have more difficulty processing local information such as interval details comÃ˜red to global features such as overall shape, and thirdly that local instructions reinforce the processing of local information. One would expect that levels of performance at melody/notation matching tasks involving local differences would be lower than those reported for melody/graph matching tasks in Experiment 5, but would be more dependent on type of instruction. An extension of that research would be an investigation of the extent to which dimensions such as tempo, rhythm and timbre could be considered to be locaL/analytic or globallholistic. Following on from this, an investigation could be carried out on the role and possible benefits of instruction and extended practice in the matching of musical intervals (aural) and their notated forms. If found to be beneficial to music students in terms of reading music notation, such instruction and practice may exemplify those activities required in addressing the call in the music education literature (Walker, 1992) to develop the ability to integrate information gained from auditory and visual perception.