DOI: 10.17035/d.2020.0098047203
title: Data from an Investigation of Music Analysis by the Application of Grammar-based Compressor

description: This dataset is composed of output and result data from various experiments performed on a substantial collection of digital musical scores. It does not contain these scores, but at the time of writing they are publicly available from the following resources: 1. The Acadia Early Music Archive (http://www.acadiau.ca/~gcallon/www/archive/) 2. The Choral Public Domain Library (http://www2.cpdl.org/) 3. Musopen (https://musopen.org/) 4. Music21 (https://web.mit.edu/music21/) 5. KernScores (http://kern.ccarh.org/) 6. The 1850 edition of O‘Neill’s Music Of Ireland (http://trillian.mit.edu/~jc/music/book/oneills/1850) 7. The Meertens Tune Collections (http://www.liederenbank.nl/mtc/) 8. The Johannes Kepler University Patterns Development Database (http://tomcollinsresearch.net/research/data/mirex/JKUPDD-Aug2013.zip) Digital scores were transformed into specific representations, and compressive models built using various compressors (ZZ, IRR, LZW, BWT, GZIP and COSIATEC). Performance on various tasks was evaluated from the model attributes (primarily model size, measured in symbols). Where possible, model metrics and computed performance figures are included in this dataset. 3.1.1.1. An exhaustive test of sensitivity to an increasing number of errors (for Bach’s Fugue No. 10 from Das Wohltemperierte Clavier Book I) ---------------------------------------------------------------------------------------------------------------------------------------------- Rows: 2-46 Compressor used: ZZ. Representations used: chromatic and diatonic pitch, chromatic and diatonic intervals, chromatic and diatonic contour, chromatic and diatonic pitch modulo 12, note duration. Attributes: Common to all experiments: representation Type of data taken from each score as input to the compressor. Compressor output: initial_model_size The size of the unaltered, compressed model. model_size_change Increase in size from initial_model_size as an error is introduced to each position in sequence. Experiment result: average_change The average increase in model size over all positions. standard_deviation Standard deviation within model_size_change. 3.1.1. Sensitivity to point errors ---------------------------------- Rows: 47-260460 Compressors used: ZZ, IRR, LZW, BWT, GZIP, COSIATEC. Representation used: diatonic intervals. Attributes: Common to all experiments: change_made The changes made to a given position. The experiment is repeated in its entirety for each value. positions_tested A list of indices into the input data; an alteration (change_value), representing an error, is made at each index. Compressor output: initial_model_size The size of the unaltered, compressed model. model_size_change Change in size from initial_model_size when an error is present at exactly one location (from an index in positions_tested). Only one error is present within the piece at each iteration. One set of size changes exists for each value in change_made. Experiment result: average_change The average increase in model size over all positions, for each change made. standard_deviation Standard deviation within each set of values in model_size_change. elapsed_time Time taken to perform all compression and measurement operations, in seconds. Times are system-dependent and circumstantial. 3.1.2. Sensitivity to increasing number of errors ------------------------------------------------- Rows: 260461-501987 Compressors used: ZZ, IRR, LZW, BWT, GZIP, COSIATEC. Representation used: diatonic intervals. Attributes: Common to all experiments: change_made The changes made to a given position. The experiment is repeated once for each value, but each change is chosen randomly from this list of values. positions_tested A list of indices into the input data; an alteration (change_value), representing an error, is made at each index. Compressor output: initial_model_size The size of the unaltered, compressed model. model_size_change Change in size from initial_model_size when an error is added to a new location (from the indices in positions_tested). Number of errors within the piece increases at each iteration. One set of size changes exists for each value in change_made, but the actual change at each position is a random selection from change_made. Experiment result: average_change The average increase in model size over all positions, for each additional change made. standard_deviation Standard deviation within each set of values in model_size_change. elapsed_time Time taken to perform all compression and measurement operations, in seconds. Times are system-dependent and circumstantial. 3.1.3. Automatic selection of candidate Transcription Error Positions --------------------------------------------------------------------- Rows: 501988-3079962 Compressors used: ZZ, IRR, LZW, BWT, GZIP, COSIATEC. Representation used: diatonic intervals. Attributes: Common to all experiments: change_made The changes made to a given position. The experiment is repeated in its entirety for each value. error_positions A list of indices into the input data; an alteration (change_value), representing an error, is made at each index. positions_tested A list of indices into the input data; an alteration (change_value), representing a possible correction, is made at each index, upon a model containing exactly one alteration (error) at one of each position from the error_positions list. Compressor output: initial_model_size The size of a model containing exactly one alteration (representing an error). One model exists for each position in error_positions. model_size_change Change in size from initial_model_size when a potential correction is added to a new location (from the indices in positions_tested). For each error, one attempt to correct at all positions_tested is made. One output set exists for each value in change_made. num_all_positives Number of true and false positives, chosen by selecting all indices where the "corrected" model is smaller that the model containing only the error. num_true_positives Number of true positives; when 1, this means an attempt to correct the value at the index of the error resulted in a smaller model size. rank_of_true_positive Of all sorted unique model sizes occurring at the index of the error, the correct change belongs to the nth group. When -1, no attempt to correct the error resulted in a smaller model. Experiment result: average_f_measure Average F-measure from all attempts to select the correct position of the error. average_precision Average Precision from all attempts to select the correct position of the error. average_recall Average Recall from all attempts to select the correct position of the error. average_rank Average rank from all instances where the correct position of the error was found. elapsed_time Time taken to perform all compression and measurement operations, in seconds. Times are system-dependent and circumstantial. 3.2.1. Classification of the Meertens Tune Collections by Family ---------------------------------------------------------------- Rows: 3079963-3136586 Compressors used: ZZ, IRR, LZW. Representation used: chromatic and diatonic pitch, chromatic and diatonic intervals, chromatic and diatonic contour, chromatic and diatonic pitch modulo 12, note duration. Attributes: Results (for the given representation): model_size (piece_a) The size of the unaltered, compressed model for the named piece. piece_b The names of all pieces to whom distance is calculated. model_size ( ab ) The size of the model resulting from the concatenation [ piece_a piece_b ]. model_size ( ba ) The size of the model resulting from the concatenation [ piece_b piece_a ]. ( ab - min( a b ) ) / max( a b ) Normalised Compression Formula by the specified formula between piece_a, piece_b. max( ab ba ) / max( a b ) Normalised Compression Formula by the specified formula between piece_a, piece_b. ( ( ab - b ) + ( ba - a ) ) / ( ab + ba ) Normalised Compression Formula by the specified formula between piece_a, piece_b. a + b Simple distance calculated as the sum of sizes of models for piece_a and piece_b. 3.3.1. MIREX 2016 Discovery of Repeated Themes & Sections task -------------------------------------------------------------- Rows: 3136587-3136671 Compressor used: ZZ. Representation used: diatonic intervals. Attributes, as defined in the MIREX 2016 Discovery of Repeated Themes and Sections task: Experiment result: n_P Number of patterns sought, from ground truth. n_Q Number of patterns identified the by algorithm. P_est Establishment precision. R_est Establishment recall. F1_est Establishment F-measure. P_occ(c=.75) Occurrence precision, with detection threshold of 0.75. R_occ(c=.75) Occurrence recall, with detection threshold of 0.75. F_1occ(c=.75) Occurrence F-measure, with detection threshold of 0.75. P_3 Three-layer precision. R_3 Three-layer recall. TLF_1 Three-layer F-measure. P_occ(c=.5) Occurrence precision, with detection threshold of 0.5. R_occ(c=.5) Occurrence recall, with detection threshold of 0.5. F_1occ(c=.5) Occurrence F-measure, with detection threshold of 0.5. P Standard precision. R Standard recall. F_1 Standard F-measure. 3.3.2. Structural Analysis of Bach's Well-Tempered Clavier ---------------------------------------------------------- Rows: 3136672-3136719 Compressor used: ZZ Representation used: diatonic intervals. Attributes: Result of matching model segmentation to that specified by S. Bruhn: section Semantic name of defined segment. rule_number Number of the closet-matching rule. jaccard_index Jaccard Index (intersection over union) for the match. in_voices Voice numbers containing instances of the chosen rule. first_instance_start_symbol Position within the first voice where the first instance of the chosen rule occurs. first_instance_end_symbol Position within the first voice where the first instance of the chosen rule ends.