DOI: 10.5281/zenodo.10396364
title: Multimodal datataset of EEG, ECG and audio recordings of individual musicians playing emotional music

description: This dataset contains EEG, ECG and audio recordings of 11 individual musicians playing emotional music on their instrument. The dataset consists of two parts:





Experiment: Musicians’ self-reported ratings, audio recordings, and physiological recordings where the 11 expert musicians were asked to play at least 4~2-minute unfamiliar (non-popularly known) musical pieces. Participants were asked to play at least once one of the following  emotions:  happiness, sadness, relaxation, and anger. For the  physiological recordings EEG, ECG, and GSR signals were recorded. Each musican’s data is denoted by MS_ followed by the order in which they were recorded.




Self-report questionnaire: A self-assessment questionnaire and their answers where 11 expert musicians were asked to rate musical pieces recorded based on:





Objective valence & arousal they felt the piece had;




Felt valence & arousal during playing.





 

For a more detailed explanation of the dataset, its recording procedure, and its contents, see 

L. Turchet, B. O'Sullivan, R. Ortner & C. Gugher (2024). Emotion Recognition of Playing Musicians from EEG, ECG, and Acoustic Signals. IEEE Transactions on Human-Machine Systems.

 

File Listing

The following files are available (each explained in more detail below):








Name



Format



Contents





EEG_ECG_data_for_each_musician



mat



This folder contains the raw EEG, ECG, & GSR data for all 11 musicians for each piece they played as well as a resting state recording, which was recorded while a neutral audio stimulus was played.





audio_data_for_each_musician



wav, JSON, csv



This folder contains 3 subfolders: 1) wav_audio_files_original: This is the raw audio data recorded for each musician and includes all 56 pieces included in the paper reported above, please see below regarding rejected trials.

2) wav_audio_files (original split_into_3_parts): Here, the 56 pieces are appropriately split into 3 separate parts.3) analysis_audio_files: This folder contains the acoustic features extracted, 1714 acoustic features were extracted from each split trial. Each result is stored in .JSON, however, the collated results can be seen in all_results.csv.





self_report_questionnaire



pdf, xls



Two files exist in this folder:1. The self-reported questionnaire given to the musicians of the questionnaire during the experiment.2. 








File Details

 

EEG_ECG_data_for_each_musician

These are the original raw data recordings. EEG data were recorded using a g.GAMMAcap2 by g.tec Medical Engineering, a 64-channel cap with g.SCARABEO active electrodes, with two g.GAMMAsys reference active ear clip electrodes. Two g.GAMMAbox electrode connector boxes were used to connect the active electrodes to two g.USBamp biosignal amplifiers with a sampling frequency of 256 Hz.The following 31 EEG channels were used: Fp1, Fp2, AFz, AF3, AF4, AF7, AF8, Fz, F3, F4, F7, F8, Cz, C3, C4, CP3, CP4, CP5, CP6, P1, P2, P3, P4, P5, P6, P7, P8, PO7, PO8, O1, and O2. AFz was used as a ground electrode and Cz was used as a re-reference electrode. The right-side ear clip electrode was used as a reference electrode. 

ECG data were recorded using a single g.GAMMAclip active electrode clip connected directly to the g.GAMMAbox, sharing the same ground electrode with the EEG cap and placed on position V4 of the subjects.GSR data was recorded using the g.GSRsensor² box which contains two small dry electrodes placed underneath the participant’s toes (due to the amount of hand movement required for playing an instrument). The g.GSRsensor² was connected directly to a g.GAMMAbox, using jumper cables connected to different ports in the g.USBAMPs to share the same reference and ground electrodes as the EEG cap. 

 

The locations of the channels and their corresponding number in the raw data is as follows:








Channel Number



Channel Name





1



Time series





2



AF3





3



AF4





4



AF7





5



AF8





6



CP3





7



CP4





8



CP5





9



CP6





10



P1





11



P2





12



P5





13



P6





14



P7





15



P8





16



O1





17



O2





18



Cz





19



Fp1





20



Fp2





21



F3





22



F4





23



F7





24



F8





25



C3





26



C4





27



P3





28



P4





29



PO7





30



PO8





31



AFz





32



GSR





33



ECG








audio_data_for_each_musician/wav_audio_files_original

This folder contains all of the raw audio files recorded during the experiment. All pieces were recorded using the software Audacity and exported as WAV files encoded with a bit depth of 32-bits and a sampling rate of 44.1 kHz. 56 of the included trials are present in this folder.

 

audio_data_for_each_musician/wav_audio_files (original split_into_3_parts)

This folder contains the above mentioned 56 raw audio pieces separated into 3 appropriately sized recordings.

 

audio_data_for_each_musician/analysis_audio_filesThis folder contains the 1714 acoustic features from each of the 3 separated trials from 56 accepted pieces, these are denoted by MS_ followed by the order in which the musicians were recorded and the order in which the trails were split, the individual features are in JSON format. Within this folder, we have collated all results in a csv file (all_results.csv) where the columns show the intended emotion, trial name and number, followed by the names of the acoustic features. 

 

self_report_questionnaire

This pdf is the questionnaire which each musician was given following each trial relating to the emotions communicated and felt during each trial. Most* questions in the questionnaire were multiple-choice and speak pretty much for themselves. The answers for which were collated and are described below.

 

self_report_questionnaire_anwsers

This .xls file contains the results of all the musicians self-reported ratings from the above described questionnaire. 








Column name



Description





Subject



The subject code of the musician, denoted by MS_





recording_filename



The name of the trial denoted by MS_01_ followed by the trial number.





intended_emotion



The intended emotion the musician was instructed to communicate. The emotions are as follows:





Angry




Sad




Relaxed




Happy



They are intended to be reported by using the valence-arousal space.





valence_communicated



The valence rating (integer between 1 and 5), that participants were asked to objectively rate how they thought the music played would be perceived. 





arousal_communicated



As above, but relating to arousal rather than valence.





valence_felt



The valence rating (integer between 1 and 5), that participants were asked to rate how they felt while playing the piece.






 








arousal_felt



As above, but relating to arousal rather than valence.





instrument_played



The instrument played for each trial.