DOI: 10.5281/zenodo.13845439
title: The Distant Listening Corpus

description: This is a README file for a data repository originating from the DCML corpus initiative and serves as welcome page for both



the GitHub repo https://github.com/DCMLab/distant_listening_corpus and the corresponding

documentation page https://dcmlab.github.io/distant_listening_corpus


For information on how to obtain and use the dataset, please refer to this documentation page.



The Distant Listening Corpus



Cite as

Version history

Getting the data



With full version history

Without full version history



Data Formats



Opening Scores

Opening TSV files in a spreadsheet

Loading TSV files in Python



How to read metadata.tsv



File information

Composition information

Score information

Identifiers



Generating all TSV files from the scores

Questions, Suggestions, Corrections, Bug Reports

License



Overview



ABC

bach_en_fr_suites

bach_solo

bartok_bagatelles

beethoven_piano_sonatas

c_schumann_lieder

chopin_mazurkas

corelli

couperin_clavecin

couperin_concerts

cpe_bach_keyboard

debussy_suite_bergamasque

dvorak_silhouettes

frescobaldi_fiori_musicali

grieg_lyric_pieces

handel_keyboard

jc_bach_sonatas

kleine_geistliche_konzerte

kozeluh_sonatas

liszt_pelerinage

mahler_kindertotenlieder

medtner_tales

mendelssohn_quartets

mozart_piano_sonatas

pergolesi_stabat_mater

peri_euridice

pleyel_quartets

poulenc_mouvements_perpetuels

rachmaninoff_piano

ravel_piano

scarlatti_sonatas

schubert_winterreise

schulhoff_suite_dansante_en_jazz

schumann_kinderszenen

schumann_liederkreis

sweelinck_keyboard

tchaikovsky_seasons

wagner_overtures

wf_bach_sonatas






The Distant Listening Corpus
This corpus has been created within the DCML corpus initiative and employs the DCML harmony annotation standard.

The publication covers the following public corpora (the DOI links always point at the latest version respectively):



Ludwig van Beethoven - String Quartets

Ludwig van Beethoven - Piano Sonatas

Frédéric Chopin - Mazurkas

Arcangelo Corelli – Trio Sonatas

Claude Debussy - Suite Bergamasque

Antonín Dvořák - Silhouettes

Edvard Grieg - Lyric Pieces

Franz Liszt - Années de Pèlerinage

Nikolai Medtner - Tales

Wolfgang Amadeus Mozart - Piano Sonatas

Robert Schumann - Kinderszenen

Pyotr Tchaikovsky - The Seasons


Moreover, it covers the following corpora that will become public with publication of the DLC data report:



J.S. Bach – English and French Suites

J.S. Bach – Solo Pieces (A corpus of annotated scores)

Béla Bartók – 14 Bagatelles, Op. 6

François Couperin – L'art de toucher le clavecin

François Couperin – Concerts Royaux

Clara Schumann – Lieder

Carl Philipp Emanuel Bach – Works for Keyboard

Girolamo Frescobaldi (1583-1643) – Fiori Musicali, op. 12 (1635)

Georg Friedrich Händel – Grobschmied Variations (The Harmonious Blacksmith), HWV 430

J.C. Bach – Keyboard Sonatas

Heinrich Schütz – Kleine Geistliche Konzerte

Leopold Koželuch – Piano Sonatas

Gustav Mahler – Kindertotenlieder

Felix Mendelssohn – String Quartets

Giovanni Battista Pergolesi – Stabat Mater (1736)

Jacopo Peri – Euridice (1600)

Ignaz Pleyel – String Quartets

Francis Poulenc – Mouvements Perpetuels

Sergei Rachmaninoff – Piano Pieces

Maurice Ravel – Piano Pieces

Domenico Scarlatti – Keyboard Sonatas

Franz Schubert – Winterreise

Erwin Schulhoff – Suite dansante en jazz

Robert Schumann – Liederkreis

Jan Sweelinck – Organ Pieces

Richard Wagner – OverturesWilhelm Friedemann Bach – Piano Sonatas




Cite as




Version history
See the GitHub releases.



Getting the data

With full version history
The dataset is version-controlled via git. In order to download the files with all revisions they have gone through, git needs to be installed on your machine. Then you can clone this repository using the command



git clone --recurse-submodules -j8 https://github.com/DCMLab/distant_listening_corpus.git

 


-j8 is available from git >= 2.8 and fetches up to 8 submodules at a time in parallel



Without full version history


If you are only interested in the current version of the corpus, you can download the ZIP file of each submodule:



ABC

bach_en_fr_suites

bach_solo

bartok_bagatelles

beethoven_piano_sonatas

c_schumann_lieder

chopin_mazurkas

corelli

couperin_clavecin

couperin_concerts

cpe_bach_keyboard

debussy_suite_bergamasque

dvorak_silhouettes

frescobaldi_fiori_musicali

grieg_lyric_pieces

handel_keyboard

jc_bach_sonatas

kleine_geistliche_konzerte

kozeluh_sonatas

liszt_pelerinage

mahler_kindertotenlieder

medtner_tales

mendelssohn_quartets

mozart_piano_sonatas

pergolesi_stabat_mater

peri_euridice

pleyel_quartets

poulenc_mouvements_perpetuels

rachmaninoff_piano

ravel_piano

scarlatti_sonatas

schubert_winterreise

schulhoff_suite_dansante_en_jazz

schumann_kinderszenen

schumann_liederkreis

sweelinck_keyboard

tchaikovsky_seasons

wagner_overtures

wf_bach_sonatas




Data Formats
Each piece in this corpus is represented by four files with identical names, each in its own folder. For example, the Prélude of J.S. Bach's first English Suite, BWV 806, has the following files:



bach_en_fr_suites/MS3/BWV806_01_Prelude.mscx: Uncompressed MuseScore file including the music and annotation labels.

bach_en_fr_suites/notes/BWV806_01_Prelude.tsv: A table of all note heads contained in the score and their relevant features (not each of them represents an onset, some are tied together)

bach_en_fr_suites/measures/BWV806_01_Prelude.tsv: A table with relevant information about the measures in the score.

bach_en_fr_suites/harmonies/BWV806_01_Prelude.tsv: A list of the included harmony labels (including cadences and phrases) with their positions in the score.




Opening Scores
After navigating to your local copy, you can open the scores in the folder MS3 with the free and open source score editor MuseScore. Please note that the scores have been edited, annotated and tested with MuseScore 3.6.2. MuseScore 4 has since been released and preliminary tests suggest that it renders them correctly.



Opening TSV files in a spreadsheet
Tab-separated value (TSV) files are like Comma-separated value (CSV) files and can be opened with most modern text editors. However, for correctly displaying the columns, you might want to use a spreadsheet or an addon for your favourite text editor. When you use a spreadsheet such as Excel, it might annoy you by interpreting fractions as dates. This can be circumvented by using Data --> From Text/CSV or the free alternative LibreOffice Calc. Other than that, TSV data can be loaded with every modern programming language.



Loading TSV files in Python
Since the TSV files contain null values, lists, fractions, and numbers that are to be treated as strings, you may want to use this code to load any TSV files related to this repository (provided you're doing it in Python). After a quick pip install -U ms3 (requires Python 3.10) you'll be able to load any TSV like this:



import ms3

labels = ms3.load_tsv('bach_en_fr_suites/harmonies/BWV806_01_Prelude.tsv')
notes = ms3.load_tsv('bach_en_fr_suites/notes/BWV806_01_Prelude.tsv')