Automatic Music Transcription based on Prior Knowledge from Musical Acoustics. Application to the repertoires of the Marovany zither of Madagascar

Ethnomusicology is the study of musics around the world that emphasize their cultural, social, material, cognitive and/or biological. This PhD sub- ject, initiated by Pr. Marc CHEMILLIER, ethnomusicolog at the laboratory CAMS-EHESS, deals with the development of an automatic transcription system dedicated to the repertoires of the traditional marovany zither from Madagascar. These repertoires are orally transmitted, resulting from a pro- cess of memorization/transformation of original base musical motives. These motives represent an important culture patrimony, and are evolving contin- ually under the inuences of other musical practices and genres mainly due to globalization. Current ethnomusicological studies aim at understanding the evolution of the traditional repertoire through the transformation of its original base motives, and preserving this patrimony. Our objectives serve this cause by providing computational tools of musical analysis to organize and structure audio recordings of this instrument. Automatic Music Transcription (AMT) consists in automatically estimating the notes in a recording, through three attributes: onset time, duration and pitch. On the long range, AMT systems, with the purpose of retrieving meaningful information from complex audio, could be used in a variety of user scenarios such as searching and organizing music collections with barely any human labor. One common denominator of our diferent approaches to the task of AMT lays in the use of explicit music-related prior knowledge in our computational systems. A step of this PhD thesis was then to develop tools to generate automatically this information. We chose not to restrict ourselves to a speciprior knowledge class, and rather explore the multi-modal characteristics of musical signals, including both timbre (i.e. modeling of the generic \morphological" features of the sound related to the physics of an instrument, e.g. intermodulation, sympathetic resonances, inharmonicity) and musicological (e.g. harmonic transition, playing dynamics, tempo and rhythm) classes. This prior knowledge can then be used in com- putational systems of transcriptions. The research work on AMT performed in this PhD can be divided into a more \applied research" (axis 1), with the development of ready-to-use operational transcription tools meeting the cur- rent needs of ethnomusicologs to get reliable automatic transcriptions, and a more \basic research" (axis 2), providing deeper insight into the functioning of these tools. Our axis of research requires a transcription accuracy high enough 1 (i.e. average F-measure superior to 95 % with standard error tolerances) to provide analytical supports for musicological studies. Despite a large enthusiasm for AMT challenges, and several audio-to-MIDI converters available commercially, perfect polyphonic AMT systems are out of reach of today's al- gorithms. In this PhD, we explore the use of multichannel capturing sensory systems for AMT of several acoustic plucked string instruments, including the following traditional African zithers: the marovany (Madagascar), the Mvet (Cameroun), the N'Goni (Mali). These systems use multiple string- dependent sensors to retrieve discriminatingly some physical features of their vibrations. For the AMT task, such a system has an obvious advantage in this application, as it allows breaking down a polyphonic musical signal into the sum of monophonic signals respective to each string.