Package: ctc-chroma
Using Weakly Aligned Score–Audio Pairs to Train Deep Chroma Models for Cross-Modal Music RetrievalBackgroundThis repository contains accompanying code for the following papers. If you use code from this repository, please consider citing them.[1]: Frank Zalkow and Meinard Müller: Using Weakly Aligned Score–Audio Pairs to Train Deep Chroma Models for Cross-Modal Music Retrieval. In Proceedings of the International Society for Music Information Retrieval Conference, Montréal, Canada, 2020.

[2]: Frank Zalkow and Meinard Müller: CTC-Based Learning of Deep Chroma Features for Cross-Modal Music Retrieval. Currently under review.There is an accompanying website for the paper.https://www.audiolabs-erlangen.de/resources/MIR/2020-ISMIR-ctc-chromaUsageYou can install the code in this repository with pip:pip install ctc_chromaThere are two ways to use the models of this repository. The first way is to use aJupyter notebook. This notebook applies the model and visualizes its output. The second way is to use a script to batch process audio files in a folder. This script can be executed like this:python apply_model.py -m MODEL_ID -i INPUT -o OUTPUTHere,INPUTis a directory with audio files,OUTPUTis a directory for the output files, andMODEL_IDspecifies the model variant. There are several model variants contained in the repository. These variants are due to different versions of the training data (used in the papers [1] and [2], respectively), due to different training and validation splits, and due to different training procedures. The following table specifies all model identifiers of the repository.Model IdentifierUsed in PaperTraining Procedurev1_ctc_train1234valid5[1]CTCv1_ctc_train123valid4[1]CTCv1_ctc_train2345valid1[1]CTCv1_ctc_train234valid5[1]CTCv1_ctc_train3451valid2[1]CTCv1_ctc_train345valid1[1]CTCv1_ctc_train4512valid3[1]CTCv1_ctc_train451valid2[1]CTCv1_ctc_train5123valid4[1]CTCv1_ctc_train512valid3[1]CTCv2_ctc_train123valid4[2]CTCv2_ctc_train234valid5[2]CTCv2_ctc_train345valid1[2]CTCv2_ctc_train451valid2[2]CTCv2_ctc_train512valid3[2]CTCv2_linear_train123valid4[2]Crossentropy, linear alignmentv2_linear_train234valid5[2]Crossentropy, linear alignmentv2_linear_train345valid1[2]Crossentropy, linear alignmentv2_linear_train451valid2[2]Crossentropy, linear alignmentv2_linear_train512valid3[2]Crossentropy, linear alignmentv2_strong_train123valid4[2]Crossentropy, strong alignmentv2_strong_train234valid5[2]Crossentropy, strong alignmentv2_strong_train345valid1[2]Crossentropy, strong alignmentv2_strong_train451valid2[2]Crossentropy, strong alignmentv2_strong_train512valid3[2]Crossentropy, strong alignmentRecordingsFor making it easy to directly try out the code of this repository, we included two excerpts from public domain recordings, which we downloaded fromMusopen. The excerpts correspond to the musical sections that are used for the figures in the paper (Figure 3 and 4). However, different performances (not public domain) have been used to generate the figures in the paper. Below you find a small table with details for the excerpts.FilenameComposerWorkPerformerDescriptionBeethoven_Op067-01_DavidHighSchool.wavBeethovenSymphony no. 5, op. 67Davis High School Symphony OrchestraFirst movement, first themeBeethoven_Op002-2-01_Pitman.wavBeethovenPiano Sonata no. 2, op. 2 no. 2Paul PitmanFirst movement, second themeAcknowledgementsFrank Zalkow and Meinard Müller are supported by the German Research Foundation (DFG-MU 2686/11-1, MU 2686/12-1). We thank Daniel Stoller for fruitful discussions on the CTC loss, and Michael Krause for proof-reading the manuscript. We also thank Stefan Balke and Vlora Arifi-Müller as well as all students involved in the annotation work, especially Lena Krauß and Quirin Seilbeck. The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) and Fraunhofer Institute for Integrated Circuits IIS. The authors gratefully acknowledge the compute resources and support provided by the Erlangen Regional Computing Center (RRZE).