Analysis of Musical Content in Digital Audio

Automatic analysis of digital audio with musical content is a difficult — but important — task for various applications in computer music, audio compression, and music information retrieval. This chapter contains a brief review of audio analysis as it relates to music, followed by three case studies of recently developed systems which analyse specific aspects of music. The first system is BeatRoot, a beat tracking system that finds the temporal location of musical beats in an audio recording, analogous to the way that people tap their feet in time to music. The second system is JTranscriber, an interactive automatic transcription system, which recognises musical notes and converts them into MIDI format allowing interactive monitoring and correction of the extracted MIDI data via a multimedia interface. The third system is the Performance Worm, a real time system for visualisation of musical expression, which presents in real time a two-dimensional animation of variations in tempo and loudness. This chapter appears in the book, Computer Graphics and Multimedia: Applications, Problems and Solutions, edited by John DiMarco. Copyright © 2004, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. 701 E. Chocolate Avenue, Suite 200, Hershey PA 17033-1240, USA Tel: 717/533-8845; Fax 717/533-8661; URL-http://www.idea-group.com IDEA GROUP PUBLISHING Analysis of Musical Content in Digital Audio 215 Copyright © 2004, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. INTRODUCTION The history of audio analysis reveals an intensely difficult, laborious, and error-prone task, where analysis tools have proved helpful but final measurements have been based mostly upon human judgement. Only since the 1980s has it become feasible to process audio data automatically with computers; and as it developed, audio content analysis took an important place in the emerging fields of computer music, audio compression, and music information retrieval. That the field is reaching maturity is evident from the recent international standard for multimedia content description (MPEG7), one main part of which relates to audio (ISO, 2001). Audio content analysis finds applications in automatic indexing, classification, and content-based retrieval of audio data, such as in multimedia databases and libraries. It is also necessary for tasks such as the automatic transcription of music and for the study of expressive interpretation of music. A further application is the automatic synchronisation of devices such as lights, electronic musical instruments, recording equipment, computer animation, and video with musical data. Such synchronisation might be necessary for multimedia or interactive performances or for studio post-production work. In this chapter, we restrict ourselves to a brief review of audio analysis as it relates to music, followed by three case studies of recently developed systems which analyse specific aspects of music. The first system is BeatRoot (Dixon, 2001a, c), a beat tracking system that finds the temporal location of musical beats in an audio recording, analogous to the way that people tap their feet in time to music. The second system is JTranscriber, an interactive automatic transcription system (Dixon, 2000a, b), which recognises musical notes and converts them into MIDI format, displaying the audio data as a spectrogram with the MIDI data overlaid in piano-roll notation. It allows interactive monitoring and correction of the extracted MIDI data. The third system is the Performance Worm (Dixon, Goebl & Widmer, 2002), a real time system for visualisation of musical expression. It presents in real time a two dimensional animation of variations in tempo and in loudness (Langner & Goebl, 2002). BACKGROUND Sound analysis research has a long history, which is reviewed quite thoroughly by Roads (1996). The problems that have received the most attention are pitch detection, spectral analysis, and rhythm recognition — areas 20 more pages are available in the full version of this document, which may be purchased using the "Add to Cart" button on the publisher's webpage: www.igi-global.com/chapter/analysis-musical-content-digitalaudio/6856