DOI: 10.5281/zenodo.7773823
title: Data from: Trade-offs in Coordination Strategies for Duet Jazz Performances Subject to Network Delay and Jitter

description: This dataset is associated with the paper “Trade-offs in Coordination Strategies for Duet Jazz Performances Subject to Network Delay and Jitter” and includes recordings of improvisations by jazz duos over a network. The related paper is published in Music Perception and is accessible at doi:10.1525/mp.2024.42.1.48 

Introduction:

This dataset includes data from approximately four hours of live, improvised musical duo performances over a simulated network environment collected in Cambridge, United Kingdom between April-July 2022 as part of a doctoral research project. Data includes audio and video recordings of 130 individual performances, biometric data, and subjective evaluations and comments from the musicians. The primary aim of the project was to collect data via a novel performance capture and manipulation system for use in the empirical modelling of ensemble coordination strategies during networked music-making. This analysis is reported in Cheston, Cross, and Harrison (2023), "Trade-offs in Coordination Strategies for Networked Jazz Performances". Please refer to this publication for full details on the data collection procedure. Our codebook is hosted on GitHub and, in conjunction with this dataset, can be used to reproduce the analysis contained in the article.

The ten musicians shown in these recordings were recruited for their expertise in jazz improvisation. They were grouped into five duos consisting each of one pianist and drummer, with no musician performing in more than one duo. Participants were instructed to improvise together over a standard twelve-bar blues musical structure, but following a formula which required them to provide a clear and unambiguous pulse of continuous quarter notes. Varying amounts of network latency and jitter were simulated for each performance, consisting respectively of the minimum amount of delay applied to the live feedback a musician heard from their partner and the degree that this delay varied. The amount of latency and jitter applied to the performance is summarised in the file or directory name for each performance and is described in detail in the above publication. Note that latency and jitter conditions were presented in a random order for each duo.

Data collected includes:



audio recordings for each performance, with and without delay, collected via direct line-in (MIDI, WAV).

video recordings, collected via high-quality webcams (MKV, AVI).

streams of the quarter note pulse provided by each musician in a performance (MIDI).

muxed audio-visual recordings of both participants in each performance (MP4)

accelerometer and photoplethysmography streams, collected from arm-worn devices (TXT, duos 3-5 only)

questionnaire responses from performers, evaluating each condition (XLSX)

ratings of performance quality from an unbiased sample of listeners, collected during an online perceptual study (CSV)


Repository structure:

NB: please see this section of the code documentation website for a full description of how to recreate the analyses and models created in the paper.

The files data.zip and data.z0* contain all data collected from the study, APART from the perceptual study stimuli & results. To open these files, download the data.zip file and all the corresponding volumes ending in .z0 and open the data.zip file using a tool for opening multi-part zip files, such as WinRAR. Do not try to open the files ending in .z0, otherwise you may get a message about the data being corrupted. Inside data.zip, you'll see the following folders and files:



avmanip_output: the raw MIDI, audio, and video output from each performance



the subfolders are organised with a single folder per participant duo, experimental block, and condition.

avmanip_output\trial_1\Block 1\Condition 1 - 23 05 relates to the performance of the first duo of participants in the first session of the experiment, in the first condition they encountered, with 23ms of latency and 0.5x jitter.



midi_bpm_cleaning: the cleaned MIDI files (quarter note onset positions)



the subfolders are organised similarly to the avmanip_output folder, using the same conventions.



muxed_performances: the combined audio-video .mp4 files from each performance



these files are labelled in the format: duo_session_latency_jitter_keysfmt_drumsfmt.

muxed_performances\kdelay_ddelay\d1_s1_l23_j00_kdelay_ddelay.mp4 relates to the performance of the first duo of participants in the first session of the experiment, in the first condition they encountered, with 23ms of latency and 0.5x jitter, and with latency and jitter applied to both keys and drummer.

for more information on recreating these videos, see the linked section of the code documentation website.



questionnaire_anonymized: the anonymized questionnaire responses given by participants, also contained in the supplementary material of the associated paper (see preprint).


Alongside data.zip and the data.z0* archives, there are two further loose files, Database View Participant - Dashboard.csv, Database View SuccessTrial - Dashboard.csv, which are the anonymized demographic and response data from the perceptual experiment, and one loose archive folder perceptual_study_videos.rar, which contains the stimuli used in the perceptual experiment.

To reproduce the analysis from the paper, all files should be unzipped into the \data\raw directory of the code repository created after cloning this from GitHub. For more detail and instructions on installation, see the section of the code documentation website linked here.

Usage:

These recordings of live, improvised duo performances are unattributed and anonymised as agreed with participants at the point of data collection. The musicians involved received a one-off, fixed payment for their time and had their travel expenses reimbursed, with funding provided by Cambridge Digital Humanities (project page). All participants consented to the use of their recordings for projects by the current authors and for these recordings to be shared with interested members of the music psychology community, with the intention of furthering academic research. The musicians did not intend that the recordings be used for commercial, artistic, or entertainment purposes, and such use is not permitted.

Citation:

If you use this dataset in your research, please cite the paper it relates to:

@article{10.1525/mp.2024.42.1.48,
    author = {Cheston, Huw and Cross, Ian and Harrison, Peter M. C.},
    title = "{Trade-offs in Coordination Strategies for Duet Jazz Performances Subject to Network Delay and Jitter}",
    journal = {Music Perception},
    volume = {42},
    number = {1},
    pages = {48-72},
    year = {2024},
    month = {09},
    issn = {0730-7829},
    doi = {10.1525/mp.2024.42.1.48},
    url = {https://doi.org/10.1525/mp.2024.42.1.48},
    eprint = {https://online.ucpress.edu/mp/article-pdf/42/1/48/833292/mp.2024.42.1.48.pdf},
}

Contact:

Huw Cheston - @huwcheston - hwc31@cam.ac.uk