<figure><img src="image_1.png" /></figure>

HAL
open science

# GUITAR IMPROVISATIONS WITH HEXAPHONIC MULTIEFFECT (GIHME) DATASET AND PRACTICE ANALYSIS

Loïc Reboursiere, Thierry Dutoit, Vincent Tiffon

► To cite this version:

Loïc Reboursiere, Thierry Dutoit, Vincent Tiffon. GUITAR IMPROVISATIONS WITH HEXAPHONIC MULTIEFFECT (GIHME) DATASET AND PRACTICE ANALYSIS. Sound and Music Computing Conference 2022, Jun 2022, Saint-Etienne, France. hal-03962876v2

HAL Id: hal-03962876

https://hal.science/hal-03962876v2

Submitted on 6 Apr 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

HAL
open science

GUITAR IMPROVISATIONS WITH HEXAPHONIC MULTIEFFECT (GIHME) DATASET AND PRACTICE ANALYSIS

Loïc Reboursiere, Thierry Dutoit, Vincent Tiffon

► To cite this version:

Loïc Reboursiere, Thierry Dutoit, Vincent Tiffon. GUITAR IMPROVISATIONS WITH HEXAPHONIC MULTIEFFECT (GIHME) DATASET AND PRACTICE ANALYSIS. Sound and Music Computing Conference 2022, Jun 2022, Saint-Etienne, France. hal-03962876

HAL Id: hal-03962876

https://hal.science/hal-03962876

Submitted on 30 Jan 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
GUITAR IMPROVISATIONS WITH HEXAPHONIC MULTIEFFECT (GIHME) DATASET AND PRACTICE ANALYSIS

Loïc Reboursiere, Thierry Dutoit, Vincent Tiffon

► To cite this version:

Loïc Reboursiere, Thierry Dutoit, Vincent Tiffon. GUITAR IMPROVISATIONS WITH HEXAPHONIC MULTIEFFECT (GIHME) DATASET AND PRACTICE ANALYSIS. Sound and Music Computing Conference 2022, Jun 2022, Saint-Etienne, France. hal-03962876

HAL Id: hal-03962876

https://hal.science/hal-03962876

Submitted on 30 Jan 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# GUITAR IMPROVISATIONS WITH HEXAPHONIC MULTIEFFECT (GIHME) DATASET AND PRACTICE ANALYSIS

**Loïc Reboursière**

ISIA Lab
UMONS

loic.reboursiere@umons.ac.be

**Thierry Dutoit**

ISIA Lab
UMONS

thierry.dutoit@umons.ac.be

**Vincent Tiffon**

PRISM, CNRS

Aix Marseille Université

tiffon@prism.cnrs.fr

## ABSTRACT

This paper presents a new guitar dataset made out of richly annotated guitarist improvisations. The annotations include notes, playing techniques, instrument tuning, audio effects configurations as well as transcriptions of post improvisation interviews. The dataset gathers ten hours of improvisations and around five hours of interviews. These accompanying data make this dataset suitable for a variety of research domains: from MIR to musical improvisation analysis and musicology. The recordings that have yielded to this dataset were done in the context of an hexaphonic multieffect practice study. This hexaphonic multieffect is meant to work with an hexaphonic guitar (one pickup per string guitar) and grants the player with independent audio effects configurations for each string. This paper presents the dataset, and the experiment it has been gathered from. It also details, based on the transcriptions of the interviews, a first analysis of the specificities of an hexaphonic multieffect practice.

## 1. INTRODUCTION

An hexaphonic guitar is an electric (or acoustic) guitar equipped with an hexaphonic pickup. This device gathers six individual pickups, one per string. As a result, six audio signals are available for further processing<fnref n="1" />. With such a system, different audio effects or analysis tools can be applied to each string independently. This type of pickup appeared in the late 1970s with the guitar synthesizer which as its name implies corresponds to a guitar that can control a synthesizer. On this type of instrument, hexaphonic pickups are of great help in converting notes to control signals, because they narrow down the complexity of pitch detection from polyphonic to six monophonic algorithms running in parallel. Apart from that main commercially developed use case, hexaphonic pickups can be used for independent-string audio processing. The first physical units integrating independent-string audio processing (either in

synthesizers or in guitar pedal effects) appeared at the end of the 1970s but, despite the large amount of creative potential it seems to have, hexaphonic audio processing has never reached a larger audience and remained a niche. Interests in such an approach have resurfaced in recent years with companies like Cycfi<fnref n="2" /> or Synquanon<fnref n="3" />. The former manufactures the Nu Series pickups<fnref n="4" />. The Nu is a one-string pickup active pickup which can be gathered in a series of any number (i.e. it can be adapted to the bass guitar, double bass, 6, 7 or 8-string guitar, etc.). The latter develops hexaphonic audio effects in the form of Eurorack modules.

The series of experiments and the resulting dataset<fnref n="5" /> presented here were made to try to understand how individual string processing can modify (or not) the practice of guitarists. Five guitarists used the hexaphonic multieffect for three days to record improvisations based on predefined scenarios. The resulting dataset is made out of objective data (notes, playing techniques, effect configurations, etc.) which were collected automatically, semi-automatically or by hand, and subjective data (transcriptions of guitarist interviews). These different types of data make the dataset suitable for different research fields ranging from Music Information Retrieval (MIR) to musicology as well as musical improvisation analysis. Ten hours of musical improvisations are being annotated and four and a half hour of interviews have been transcribed. As the amount of objective data is quite important (despite the limited number of guitarists), the first analysis presented here is based on interviews of guitarists. This analysis helps to highlight terms and notions describing different points of interest of such a setup for the musicians.

While section 2 covers works related to our topic, sections 3, 4 and 5 develop different points of the experiment. Section 6 describes further the built dataset and section 7 brings an analysis based on the interviews of the guitarists. Finally, section 8 highlights future work before the conclusion (section 9).

## 2. RELATED WORKS

Music datasets are mostly built and used in MIR-related contexts. Some contains multiple types of musical instru-

<footnote n="1">¹ As a comparison, on regular electric guitars, monophonic pickups mix the sound of all resonating strings down to one audio signal.</footnote>

<footnote>Copyright: © 2022 Loïc Reboursière et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.</footnote>

<footnote n="2">² https://www.cycfi.com/ (08/02/2022).</footnote>

<footnote n="3">³ https://www.synquanon.com/ (08/02/2022).</footnote>

<footnote n="4">⁴ https://www.cycfi.com/projects/nu-series/ (08/02/2022).</footnote>

<footnote n="5">⁵ https://github.com/numediart/GIHME (08/02/2022).</footnote>
ments [1], and some are oriented towards specific instru-
ments such as the piano [2, 3] or the guitar [4, 5]. Most
of these datasets are built for pitch and onset detection but
some of them are made for tasks such as guitar playing
techniques [6], chords [7], effects [8] and playing modes
[9] detection. Other approaches such as non-negative ma-
trix factorization has been used with the hexaphonic gui-
tar [10] to provide automatic transcription of the instru-
ment.

Aside from these datasets which were built to improve
existing results on the MIR-related tasks, some are built
to study musical and instrumental practices such as the
Weimar Jazz Database from the Jazzomat project [11]. This
project uses, as well as other tools, different MIR tech-
niques to extract relevant information to describe and clas-
sify jazz improvisations. As mentioned above, the dataset
presented in this paper goes a bit further by making avail-
able to the community recordings of guitarists' improvisa-
tions made using an hexaphonic multieffect and different
types of annotations.

First hexaphonic effects appear in the late 1970s and the
beginning of the 1980s. The ARP Avatar <fnref n="6" /> guitar synthe-
sizer and the Roland GR-100 <fnref n="7" /> both include an hexaphonic
distortion effect while the GR-300 <fnref n="8" /> integrates an hexa-
phonic harmonizer. Apart from these already-established
companies, engineer Keith McMillen developed the Poly-
Fuzz <fnref n="9" /> guitar pedal (used by guitarist John Abercrombie)
which integrates distortion, fuzz and suboctave effects. The
swiss luthier Matthias Grob built, under the brand Paradis
and around the same period, its PolyDistortion <fnref n="10" /> followed
by a multieffect. He then went on developing the Poly-
subbass <fnref n="11" /> (which octave down each string) and the Math-
ons VST plugins series <fnref n="12" />. More recently, several scien-
tific works dealt with the development of hexaphonic au-
dio processing tools [12] and their use in performance con-
text [13–15] <fnref n="13" />.

3. EXPERIMENT STRUCTURE

The experiment presented here was built up to investigate
specific uses of hexaphonic pickup and multieffect, name-
ly the distribution of audio effects on specific groups of
strings and the control of individual bypass of the audio ef-
fects. For this experiment, five guitarists have been recorded
playing the hexaphonic multieffect on specific predefined
scenarios. Four of them are professional guitarists and/or
composers and/or improvisers and the last one is an ama-

teur guitarist/composer <fnref n="14" />. The four professional guitarists
are part of a collective of musicians <fnref n="15" /> whose musical pro-
jects range from jazz and improvisation, to contemporary
and prepared instruments while the amateur guitarist most-
ly evolves in rock style-related music. While the practice
of the professional guitarists can vary greatly from one gui-
tarist to another, one could say that they mainly share the
same modes of playing as a common ground. As one may
have noticed, this dataset is clearly unbalanced in terms of
professionals vs. amateurs. The point of this first analysis
was not to compare guitarists in their use of the hexaphonic
setup, but to try to understand how one guitarist practice is
impacted (or not) by the use of this setup. In the rest of
the paper, we’ll be referring to the guitarists using number
from 1 to 5, 5 being the amateur guitarist.

3.1 Pre-Defined Scenarios

The whole process of this experiment go through the three
following scenarios:

* The first scenario is a “discovery scenario” where the guitarist, with the aid of the researcher, tests each hexaphonic effect, builds presets and create a “global sound” (i.e. chain of chosen effects and presets) to start working with on the next scenario. No improvisation was recorded during this scenario.

* The second scenario investigates the distribution of effects on specific guitar strings by applying the chosen “global sound” to different groups of strings while the remaining strings are left “dry” (i.e. without any effects).

* The third scenario studies how this setup can be con-
trolled in a performance context. A generic MIDI
foot controller (Behringer FCB1010 <fnref n="16" />) was used to
control different granularities of individual audio ef-
fects bypass controls (e.g. global effect bypass vs.
string-independent bypass).

As a matter of clarity, the “discovery scenario” was named
scenario 0 and the two remaining, respectively, scenario 1
and scenario 2.

3.2 Scenarios 1 and 2 Protocol

Scenarios 1 and 2 follow the same protocol:

* They are made out of five sub-scenarios;

* Each sub-scenario is made of three steps:

- Test: the guitarist plays with the proposed sub-scenario's configuration and tries to develop musical ideas that can be used during the improvisation;

<footnote n="6">⁶ http://www.vintagesynth.com/arp/ avatar.php
(08/02/2022).</footnote>

<footnote n="7">⁷ https://www.joness.com/gr300/GR-100.htm
(08/02/2022).</footnote>

<footnote n="8">⁸ https://www.joness.com/gr300/GR-300.htm
(08/02/2022).</footnote>

<footnote n="9">⁹ https://dokumen.tips/amp/documents/
keith-mcmillen-timeline.html (08/02/2022).</footnote>

<footnote n="10">¹⁰ http://www.matthiasgrob.org/pEE/sndhist.htm
(08/02/2022).</footnote>

<footnote n="11">¹¹ http://www.polybass.com (08/02/2022).</footnote>

<footnote n="12">¹² https://www.mathons.com/ (08/02/2022).</footnote>

<footnote n="13">¹³ Sound recordings of hexaphonic effects can be found online, https://soundcloud.com/medicationtime/sets/hexaphonic-effects (08/02/2022).</footnote>

<footnote n="14">¹⁴ https://www.youtube.com/channel/
UCLHyrUsYR-gE5r_4Vs45xkQ (08/02/2022).</footnote>

<footnote n="15">¹⁵ https://muzzix.info/ (08/02/2022).</footnote>

<footnote n="16">¹⁶ This controller provides ten buttons (configured with on/off behaviours) which can be linked to different mapping configurations (also called pages or banks in this type of devices). The browsing (going up or down) of this list of configurations can be done by two other buttons. Two continuous foot controller are also present on the device but were not used in the context of these experiments.</footnote>
<table><caption>Table 1. Scenario 1 sub-scenarios<fnref n="20" />.</caption><thead><tr><td>Name</td><td>Strings with effects</td><td>Strings with no effect</td></tr></thead><tbody><tr><td>1_1</td><td>6<sup>th</sup>, 5<sup>th</sup>, 4<sup>th</sup></td><td>3<sup>rd</sup>, 2<sup>nd</sup>, 1<sup>st</sup></td></tr><tr><td>1_2</td><td>3<sup>rd</sup>, 2<sup>nd</sup>, 1<sup>st</sup></td><td>6<sup>th</sup>, 5<sup>th</sup>, 4<sup>th</sup></td></tr><tr><td>1_3</td><td>6<sup>th</sup>, 4<sup>th</sup>, 2<sup>nd</sup></td><td>5<sup>th</sup>, 3<sup>rd</sup>, 1<sup>st</sup></td></tr><tr><td>1_4</td><td>5<sup>th</sup>, 3<sup>rd</sup>, 1<sup>st</sup></td><td>6<sup>th</sup>, 4<sup>th</sup>, 2<sup>nd</sup></td></tr><tr><td>1_5</td><td colspan="2">Distribution chosen by the guitarist</td></tr></tbody></table>

- Record: once the guitarist is satisfied with its findings, he records an improvisation of 3 to 4 minutes minimum (the longest being 18 minutes);

- Interview: the guitarist and the researcher talk about specific elements of the improvisation that was just recorded.

* The last two steps can be repeated any number of times the guitarist feels like trying out different presets (i.e. changing the “global sound” to fit one specific sub-scenario) and/or different modes of playing;

* While the first four sub-scenarios are predefined, the fifth one is created by the guitarist. In scenario 1, the guitarist can choose the group of strings on which the “global sound” can be applied. In scenario 2, the guitarist can decide which bypass control configuration he wants and create presets.

Table 1 and 2 summarize the different configurations of the sub-scenarios contained in each scenario. While sub-scenarios 1_1 and 1_2 uses separation between “low” and “high” strings which is already used by guitarists in different playing styles<fnref n="17" />, scenarios 1_3 and 1_4 are relatively unnatural for guitarists as effects are being applied on non-adjacent strings. It also has to be noted that sub-scenarios 2_3 and 2_4 come out of the foot controller structure. Indeed, as only ten buttons<fnref n="18" /> are available at once on the controller, mapping strategies needed to be defined in order to access the 36 individual bypasses of the multieffect (6 effects available on 6 strings). Another setup, e.g. with 2 Voes MX-18 (matrix of 6x3 foot switches) controllers<fnref n="19" /> (which we didn’t know of at the time of the experiment), may have only needed one sub-scenario to access the 36 individual bypass controls.

## 4. HEXAPHONIC MULTIEFFECT

The hexaphonic multieffect used in these experiments is depicted in figure 1. It has been developed using Cycling' 74 Max software. This patch is an adapted version of the one presented in [13]. The multieffect gathers four main

<table><caption>Table 2. Scenario 2 sub-scenarios.</caption><thead><tr><td>Name</td><td>Bypass controls mapping</td></tr></thead><tbody><tr><td>2_1</td><td>1 button controls the bypass of 1 hexaphonic effect on all strings</td></tr><tr><td>2_2</td><td>1 button controls the bypass of the effects applied on 1 string</td></tr><tr><td>2_3</td><td>1 bank per effect and 1 button per string</td></tr><tr><td>2_4</td><td>1 bank per string and 1 button per effect</td></tr><tr><td>2_5</td><td>Distribution (2_3 or 2_4) chosen by the guitarist and definition of recallable presets</td></tr></tbody></table>

elements: six hexaphonic audio effects, a bypass matrix grouping individual effects bypass, a routing system to define effects order, an output mixer to adjust strings output levels individually. In order to help guitarists focus on the hexaphony practice as much as possible, the six audio effects (overdrive, delay, ring modulation, flanger, tremolo, reverb) were chosen, arguably, among the most common for electric guitarists. Each of the effects integrates six instances (depicted by graphical colourful sliders) of each of its parameters, a preset system (integrating an interpolation option not used in these experiments) and individual bypasses. Those individual bypasses are gathered and developed graphically (bottom right part of the patch), forming a 6x6 graphical matrix that gives a more convenient visual feedback to the guitarist when using the foot controller in scenario 2. The default order of the effects routing system follows the graphical display of the patch (same as listed above). Only one of the guitarists changed this default setting by moving the delay just before the reverb effect. Lastly, an output mixer is used to balance the sound of each string. This mixer is particularly useful in the first scenario, where the difference in sound amplitude between the strings with effects and the strings without effects can be pretty significant.

## 5. AUDIO SETUP AND RECORDING PROCESS

The audio processing setup is made out of several components:

* Two Godin guitars equipped with RMC hexaphonic piezoelectric pickups were at the disposal of the guitarists. The first one, a Godin Multiac, is a nylon-string guitar whereas the second one, a Godin LGXT, is a steel-string guitar. Only one of the guitarists tried the first one, but eventually felt more at ease with the second one;

* A homemade breakout box is used to power pickup electronics and to convert the standard 13-pin din connector used for the hexaphonic pickup<fnref n="21" /> to six standard monophonic 6.35 mm female jack cable;

* The breakout box is connected to an RME Fireface UCX soundcard through six mono jack cables. The

<footnote n="17">¹⁷ We can think, e.g., of acoustic blues style where guitarists often plays the accompaniment part on the low strings (often with alternating bass technique) and the melody on the high strings.</footnote>

<footnote n="18">¹⁸ See footnote 16.</footnote>

<footnote n="19">¹⁹ https://www.voes.be/mx18.html (08/02/2022).</footnote>

<footnote n="20">²⁰ The standard tuning of a six-string guitar is E-A-D-G-B-E, the E string being the 6th string and the E string, the 1st string.</footnote>

<footnote n="21">²¹ This “standard” connector is mainly used by Roland which has been the main company developing guitar synthesizers.</footnote>
<figure><img src="image_4.png" /><figcaption>Figure 1. Hexaphonic multieffect used for the experiments.</figcaption></figure>

sampling rate was set to 44100 Hz and 16 bits of
precision were used;

* The computer is running Max software and the hexa-
phonic multieffect patch described in section 4;

* Hexaphonic outputs are then connected to a mixing desk and to a quadriphonic Genelec speaker system (complemented by a subwoofer). This diffusion system was the one present in the recording room and couldn't be moved. No specific spatialization was used as it was not the subject of this experiment. The resulting sounds of the six strings were all equally broadcast on each speaker.

Multiple audio and video files were recorded during the experiment. On the audio side, the hexaphonic audio signals were recorded before (“dry” or “clean”) and after (“wet”<sup>22</sup>) the chosen audio effects. As the result, the clean hexaphonic signals permit to easily use any type of algorithm detection to ease the annotation part and the wet hexaphonic signals give a resulting sound with great details. A monophonic reduction of the hexaphonic wet signals was also recorded in order to have a low quality format easier to work with (only the first two channels of hexaphonic audio files can be heard when listened to with classic audio player software). Moreover, the guitarist was asked to start (when ready) a mono mix recording of its improvisation. On the video side, a Nikon 5D mark IV camera was used to capture the guitarists while they were playing (during test, improvisation or interview parts). Video files were recorded with a resolution of 1920x1080 at 25 fps. The screen of the computer, on which the Max patch ran, was also recorded in order to keep track of the user changes to the patch’s GUI. The video conference software Zoom was used for that purpose so that recordings could be launched remotely without having the guitarist to do it.

All recorded signals were synced by sound. The guitarist
was asked to pluck the lowest string with a palm-muting
technique in order to record on each media a sharp event
that can easily be detected and referred to.

6. DATASET

Once all those raw signals are recorded, several steps are
necessary in order to obtain a useable dataset.

The first step was to extract from recorded audio and
video files, the media corresponding to each specific sub-
scenario recordings. Syncing points (made by plucking
the palm-muted lowest string), as well as timings of sub-
scenario parts (test, recording or interview) were annotated
by hand in order to automatically trim and export files cor-
responding to each sub-scenario. The trim and export steps
were done using ffmpeg<fnref n="23" /> and sox<fnref n="24" /> software, respec-
tively, for dealing with video and audio files. Each part of
each sub-scenario is then made out of five files:

* Two video files: one from the camera, one from the computer screen;

* Three audio files: clean and wet hexaphonic signals as well as monophonic mix down of wet hexaphonic signals.

The second step was to annotate each sub-scenario. Dif-
ferent types of annotations were used for different types of
collected data:

* Global information, such as scenario and sub-scenario number, instrument tuning (and its evolution if necessary), duration of the specific part, used audio effects and presets, were collected;

* Played notes have been retrieved by using a semi-automatic method similar to the one used in [5]. Six automatic pitch extractions were performed in parallel, on the hexaphonic clean signals. As tuning (i.e. pitches of the strings when no notes are fretted) of the guitar was known, fret number was inferred from pitch extraction result and string number. Aubio library's <fnref n="25" /> implementation of yin-fft algorithm [16] has been used to perform this task <fnref n="26" />. The data generated by the algorithm have then been manually verified in order to consolidate notes and frets information;

* The activation periods of each individual effects from scenario 2 have been automatically extracted from computer screen videos. A computer vision algorithm has been built to track the changes in the individual bypasses of hexaphonic effects of the graphical user interface (those changes have then been computed as time segments). Moreover, in order to be able to easily visualize those changes with the improvisation, the global bypass matrix from the computer screen videos has been cropped and overlaid on top of the camera video;

* Common guitar playing techniques, such as bend, harmonics, hammer-on, pull-off (see, e.g. [6], for

<footnote n="22">²² The “wet” term is often present in effects like reverb for example where levels of modified signals and non modified signals can be modified. In this context, this term refers to the level of the modified signal.</footnote>

<footnote n="23">²³ https://www.ffffff.org/ (08/02/2022).</footnote>

<footnote n="24">²⁴ http://sox.sourceforge.net/ (08/02/2022).</footnote>

<footnote n="25">²⁵ https://aubio.org/ (08/02/2022).</footnote>

<footnote n="26">²⁶ It has to be noticed that different pitch detection algorithms were tested inside of Sonic Visualizer software and that the yin-fft algorithm appeared to give the best results.</footnote>
<figure><img src="image_5.png" /><figcaption>Figure 2. GIHME annotations integration example in Ad-vene software.</figcaption></figure>

details on such techniques), have been manually an-
notated. Extended playing techniques like the use
of a bow, objects or preparations as well as more ad-
vanced techniques like *scordatura*<sup>27</sup> are also includ-
ing in the annotations. Definitions and explanations
of those kinds of extended or advanced techniques
can be found in [17], [18] and [19] or [20]. To the
best of our knowledge, these kinds of techniques are
most often not present in literature's datasets.

All the data detailed above are stored in various file for-
mats (json, txt or csv) in order to be compatible with
different visualization software like Sonic Visualizer<sup>28</sup> or
Advene<sup>29</sup>. An example of the collected data integrated
in Advene software can be seen on Figure 2. It has to be
noticed that at the time of the submission, the process of
acquiring all the objective annotations is not finished.

To complete the objective collected data, interviews made
after each improvisation have been transcribed. The me-
thod used for the transcription is the one of the verbatim.
The use of the verbatim here is made so that researchers
from other fields willing to analyse those texts can access
the data in minimally modified form. Only some com-
mon French oral language words or expressions have been
rewritten with more readable terms. The analysis that we
are making in the next section is specifically based on those
transcriptions.

7. INTERVIEWS ANALYSIS

This analysis of the words of the guitarists is a first step
towards understanding and characterizing how their prac-
tices have been altered by the use of a hexaphonic mul-
tieffect. The different points presented below give good
hints about the impact of hexaphonic multieffect on prac-
tices of guitarists, but, for now, it cannot be easily gener-
alizable as the number of guitarists who participate in that
experiment is pretty small.

7.1 Constraints and Limitations

Before talking about the impact of the hexaphonic multi-
effect itself, one common element to all guitarists is that,
at some point, they felt constrained. The limitations they
endured were most of the time either due to the configura-
tions of the scenarios or to the technical elements used for
the experiment.

It has to be noticed that the scenarios and sub-scenario
configurations do act as constraints already, as they put all
the guitarists in unusual situations, but most of the time the
guitarists have managed to work with them and felt stim-
ulated by them. The constraints listed below are the ones
where guitarists had a hard time integrating those configu-
ration in their improvisations.

Regarding to the hexaphonic multieffect, several con-
straints were mentioned : the balance between the volume
of dry strings and strings on which effects were applied in
scenario 1 was, for example, mentioned by guitarists 1 and
3 as being problematic (the issue was resolved by adding
the output mixer mentioned in Section 4). Some effects
configurations were felt uneasy : guitarist 1 had trouble
playing with different delay times when those were not
rhythmically related and guitarist 5 felt that applying dis-
tortion only on specific strings was not a natural fit for him.

On the scenario level, all the manipulations needed to ac-
cess the different individual bypasses in scenarios 2.3, 2.4
and 2.5 (due to the use of a foot controller FCB1010 as
mentioned above) was mentioned by guitarists 1, 4 and 5 as
being not intuitive and adding a strong inertia to the whole
process. Guitarist 2 felt a strong constraint with effect dis-
tribution in scenario1_3 and 1_4:

I was a bit helpless in fact with a setup like this
one. [...] I have my guitarist's reflexes that
try to find something but which is not there
any more. So, yeah, it tries [...], but it doesn't
work. (Guitarist 2, extract of post improvisa-
tion interview, scenario1_3-02, pers. comm.)

One last type of constraint is due to the acoustic and elec-
tric behaviour of the piezoelectric hexaphonic pickup. In-
deed crosstalk<sup>30</sup> and the acoustic transfer of the strings
vibrations through the bridge create resonances on non-
played strings. These two phenomena are particularly sig-
nificant in scenario 1, where notes played on strings with-
out effects would trigger low volume modified sounds on
strings with effects. This is especially true when the dis-
tortion is used on the string with effects as it significantly
increases the volume of the strings. However, these con-
straints were used by guitarist 2, 3 and 4 as a mode of
playing in itself.

7.2 Appropriation of the System

In spite of the constraints brought by the scenarios and by
the technical system, the guitarists have, for the great ma-
jority, succeeded in appropriating and integrating the com-
plexity of the system. They have, for the most part, put in
place various strategies to reduce this complexity.

<footnote n="27"><sup>27</sup> The *scordatura* is the action of strongly detuning the strings in order to be able to play with the timbres generated in part by the softness of the relaxed strings.</footnote>

<footnote n="28">²⁸ https://www.sonicvisualiser.org/ (08/02/2022).</footnote>

<footnote n="29">²⁹ https://www.advene.org/ (08/02/2022).</footnote>

<footnote n="30">³⁰ A small amount of the sound of a vibrating string is picked up by
adjacent pickups.</footnote>
Guitarist 3, for example, played several improvisations using only a limited number of prepared strings<sup>31</sup> (scenario 1.2) or a limited way to attack the strings (scenarios 2.1, 2.2, 2.3). In scenario 1.2, guitarist 4 used preparation (a small bar of metal inserted in between the strings close to the bridge)<sup>32</sup> on the three dry strings in order to bring their timbre closer to the ones of the strings with effects. The same guitarist used *scordatura* right from the beginning (scenario 1.1) to limit the harmonic range of the guitar<sup>33</sup>:

> It appeared to me as a way, in this chaos of the low strings [strings with effects, editor's note], to try to find an organization clearer for me. (Guitarist 4, extract of post improvisation interview, scenario1.1, pers. comm.)

In this improvisation, the standard tuning of the guitar, E-A-D-G-B-e, evolved to E-A-D-A-A-e (the D string was left unused during all the improvisation).

During scenario's test step, guitarist 5 played pieces of his repertoire while applying effects in a random manner. Both of these actions (known pieces playing and randomness) helped him to dig directly into hexaphonic effects timbre without overthinking too much. From there he narrowed down to specific choice of effects and presets as well as specific directions for his improvisations (most of the time inspired by the pieces he just played). It has to be noticed that this guitarist, despite having found ways to work with the setup, didn't finish the experiment as he felt stuck. For this specific guitarist, more time would have been needed to interact and play with the system.

## 7.3 Guitarist Practice

The playing of an electric guitar is necessarily modified by the addition of an hexaphonic pickup and of a string-individualized sound processing system. The gestures made by the guitarist no longer have the same scope, the same impact, which leads him to reexamine his relationship with his instrument and therefore its practice. However hexaphonic multieffect was not considered by the guitarist as something radically new. As guitarist 2 stated: "It's another instrument, but at the same time, it's a familiar one"<sup>34</sup>.

Guitarist 1 develops the same idea:

> [...] because there's also the habit of linking instrumental gesture with foot gestures on the pedals. So there you have it, it seems to me to fit more into the same logic, but at a higher level we'll say. (Guitarist 1, additional interview, pers.comm.)

With this quote, guitarist 1 emphasizes that working with the foot controller in scenario 2 is pretty close to its practice of monophonic guitar effect pedals but it adds more

relief or details to it. This idea of being part of a known and familiar practice which goes further and needs to adapt is developed by guitarist 2:

> It doesn't question the practice. The practice stays, it exists. However, it reconsiders it, in the sense where, as new things happen, you have to adapt. [...] It's a system that forces you to look deeply at the instrument for ways to adapt, even if it is a material that I know. I mean, all these effects, I've already used them in my life. They are part of the guitarist's landscape. And despite all that, the fact to use them in hexaphony, you rethink the effects differently too. You rethink, you adapt your playing, an interaction takes place. (Guitarist 2, extract of post improvisation interview, scenario2.5, pers. comm.)

The same guitarist sums up this idea in a pretty concise and concrete way when he mentions, "With this system, you have to redraw the geographical relationship with your instrument"<sup>35</sup>. This "geographical relationship" is close to the notion of "mapping" often used in NIME (New Interface for Musical Expression) context. Using an hexaphonic multieffect pushes the guitarist to modify the relationship, one could say the "inner mapping", he has built between its gesture and the resulting sound.

From these few excerpts, it appears that this hexaphonic setup can easily integrate into practices of guitarists as it uses the same elements as their regular practices, but guitarists also need to adapt their practices in terms of the relationship between the gesture and the resulting sound.

## 7.4 Hexaphonic Specificities

Regarding the produced sounds, several comments develop the idea of this setup having a rich palette of sounds. Guitarists 1 and 4 both make the parallel with the organ. This metaphor derives directly from the use of the foot controller in the second scenario but also from the vast number of sounds accessible through this controller and through the playing. Let's remember that scenario 2.5 gives access to individual bypass controls of individual effects as well as to presets of the whole bypass matrix. These options give the musicians the ability to change configurations completely or just to make tiny adjustments, these abilities to be modulated by guitarist's playing which itself acts as a "selection gesture" [21] inside the defined timbre palette. Guitarist 2 develops this idea of rich sound possibilities by comparing those to the ones accessible through Digital Audio Workstations (DAWs) software:

> Precisely, it helps going into fields that could be covered by the digital world and all that. I find that we come close to things like that, while respecting the instrumental practice. [...] I'm not saying that it's like MIDI [...], it's a kind of in between. (Guitarist 2, extract of

<footnote n="31"><sup>31</sup> Instrument's preparation is the process by which object are added or fixed on the strings in order to modify its original timbre.</footnote>

<footnote n="32"><sup>32</sup> See video recording at https://vimeo.com/639069813 (08/02/2022).</footnote>

<footnote n="33"><sup>33</sup> See video recordings at https://vimeo.com/639069333 (08/02/2022).</footnote>

<footnote n="34"><sup>34</sup> Guitarist 2, extract of post improvisation interview, scenario 2.5, pers.comm</footnote>

<footnote n="35"><sup>35</sup> Additional interview, Guitarist 1, pers. comm.</footnote>
post improvisation interview, scenario2_5, pers. comm.)

"Respect of instrumental practice" here, comes in oppo-
sition to MIDI control of instruments. With the hexaphonic
system, the guitarist uses the professional practice he spent
years developing (and not a reduction of it) accessing all
these sound possibilities.

The specificities of the practices used during this experi-
ment are also the centre of the last hexaphonic specificity
developed here. Indeed, it appears that practices that al-
ready made use of techniques to develop polyphonic or
multitimbral approaches of the guitar were enhanced by
this system. Guitarist 3 who extensively used preparations
during its improvisation points out at several moments the
gain of clarity due to the hexaphony :

[...] one can add a tremolo or a delay on just
one string, it's really nice. Or a reverb on one
string brings a bit of depth and it won't impact
everything. On a classic analogue effect, it
necessarily takes a huge proportion. Here, one
can add a huge reverb, but just on one string,
it's very convenient yeah. (Guitarist 4, extract
of post improvisation interview, scenario2_4-
01, pers. comm.)

Despite being obvious when said aloud, this remark brings
to the fore the idea that monophonic effect pedals are not
that well suited for prepared guitar practice. Indeed, added
preparations can be string-specific (as well as applied to
multiple strings) whereas monophonic effects do apply to
all the strings at the same time. In such a context, string-
specific effect system (aka hexaphonic multieffect) brings
a natural continuity to preparations. Guitarist 4 whose prac-
tice falls into classical, contemporary and flamenco styles
tends to formulate the same kind of idea:

With a classical guitar, we work with the aim
of being able to have an action as indepen-
dent as possible from each finger and there-
fore potentially also differentiated regarding
the strings. (Guitarist 4, complementary in-
terview, pers. comm.)

This quote comes as a justification of the idea brings by
guitarist 4 that the hexaphonic multieffect setup could be
of interest of classical guitarists who would want to move
on to electric guitar. As he mentions, the independence
of the finger from each other seems to find a continuity in
a string-specific audio effects system. This point (which
wasn't expected in the first place) and the example of the
prepared guitar practice seems to highlight that this kind of
system is a good fit for practices that seeks independence
in terms of gestures, of strings or of preparations.

8. FUTURE WORK

While bringing some important notions, the analysis pre-
sented above remains a first step. As mentioned above
the small number of guitarists limits the generalization of

the findings. More guitarists and a more balanced dataset
would definitely help improve the results. More amateur
guitarists evolving in rock-related style of music may have
helped understand better why guitarist five felt stuck. A
wider variety in the music styles would also help as styles
often come with specific modes of playing. Those specific
modes of playing can be not represented in our dataset,and
could, more than probably, adapt differently to hexaphonic
effects.

Another area that should profit from future work is the visualization part. Indeed, we showed a data visualization test in Advene software but, while being able to show all the data (except audio signals) at once, this software is not optimum to run a study with such a number of different types of data. Being able to switch between audio signals (clean or wet hexaphonic, monophonic or sound from video) while looking at the same video or to summarize relevant data for specific parts of an improvisation are some of the features which would be really useful to music analysis study.

Lastly, the first insights gained by the analysis of the in-
terviews could be broadened by the addition of the objec-
tive data and their analysis. The use of a fretboard pat-
terns theoretical analysis tool such as the model developed
in [22] could help in finding a common ground to the dif-
ferent practices analysis. This common ground appears to
be vital to conduct a more efficient comparison between
guitarists’ appropriation of the system. Further, in order
to really understand the impact of the hexaphonic setup on
the practice of a specific guitarist, an analysis of its regular
practice (i.e. without hexaphonic setup) should be done so
as to enable a "before/after" comparison.

9. CONCLUSION

This paper presents a new dataset including improvisations
of five different guitarists using hexaphonic guitar and mul-
tieffect. Such a system enables them to apply string-inde-
pendent effects. This dataset gathers 10 hours of improvi-
sations and transcription of 4.5 hours of interviews. Sub-
scenario's improvisation is annotated with pitch, string,
fret, playing techniques (including extended techniques),
effects configuration and effects activation timings. As
these annotations represent a large amount of data and as
the annotation process is not finished at the time of the
writing, the first analysis given in this paper is based on the
different interviews. According to the guitarists, this sys-
tem, while new, appears to be familiar: effects are a well-
known paradigm for electric guitarists and there's no need
to learn new gestures; the guitarists' practices can be used
as it is. What requires a new approach from the guitarists,
though, is the impact the instrumental gestures have on the
generated sound, what one could call "inner mapping". In-
deed by being string-specific, hexaphonic effects add an-
other level to the impact the gestures can have. As a wrap-
up, guitarists make several remarks that tends to emphasize
that this system is particularly relevant with practices that
already use the independence of fingers and strings exten-
sively, such as prepared or classical guitar practices.
## Acknowledgments

I'd like to thank Livio Ratti, student at the University of Lille (France) for his precious help during the recording sessions and the video editing steps. This work was partially funded by the Walloon Region and the EU, in the framework of ERDF project DigiSTORM.

## 10. REFERENCES

[1] J. Thickstun, Z. Harchaoui, and S. M. Kakade, "Learning features of music from scratch," in *5th International Conference on Learning Representations ICLR*, 2017.

[2] V. Emiya, R. Badeau, and B. David, "Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle," *IEEE Transactions on Audio, Speech and Language Processing*, vol. 18, no. 6, pp. 1643–1654, 2010.

[3] C. Hawthorne, A. Roberts, I. Simon, C. Raffel, J. Engel, S. Oore, and D. Eck, "Onsets and frames: dual-objective piano transcription," in *Proceedings of the International Symposium on Multimedia Information Retrieval*, 2018, 2018, pp. 50–57.

[4] C. Kehling, J. Abeßer, C. Dittmar, and G. Schuller, "Automatic Tablature Transcription of Electric Guitar Recordings by Estimation of Score- and Instrument-Related Parameters," *Proc. of the 17th Int. Conference on Digital Audio Effects (DAFx-14)*, pp. 1–8, 2014.

[5] Q. Xi, R. M. Bittner, J. Pauwels, X. Ye, and J. P. Bello, "Guitarset: A dataset for guitar transcription," in *Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR 2018)*, E. Gómez, X. Hu, E. Humphrey, and E. Benetos, Eds., 2018, pp. 453–460.

[6] L. Su, L.-F. Yu, and Y.-H. Yang, "Sparse cepstral and phase codes for guitar playing technique classification," in *Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014)*, 2014, pp. 9–14.

[7] C.-R. Nadar, J. Abeßer, and S. Grollmisch, "Towards cnn-based acoustic modeling of seventh chords for automatic chord recognition," in *Proceedings of the 16th International Conference on Sound And Music Computing (SMC)*, 2019.

[8] M. Stein, J. Abeßer, C. Dittmar, and G. Schuller, "Automatic Detection of Audio Effects in Guitar and Bass Recordings," in *Proceedings of the 128th Audio Engineering Society (AES) Convention*, 2010.

[9] R. Foulon, P. Roy, and F. Pachet, "Automatic Classification of Guitar Playing Modes," in *Sound, Music, and Motion: 10th International Symposium, CMMR 2013*, M. Aramaki, O. Derrien, R. Kronland-Martinet, and S. Ystad, Eds., vol. 8905. Marseille, France: Springer, 2014, pp. 58–71.

[10] P. D. O'Grady and S. T. Rickard, "Automatic Hexaphonic Guitar Transcription Using Non-Negative Constraints," in *Proceedings of the Irish Signal and Systems Conference*, 2009.

[11] M. Pfleiderer, K. Frieler, J. Abeßer, W.-G. Zaddach, and B. Burkhart, Eds., *Inside the Jazzomat - New Perspectives for Jazz Research*. Schott Campus, 2017.

[12] M. Puckette, "Patch for guitar," in *Proceedings, PD Convention*, 2007, 2007.

[13] L. Reboursière, "Traitement sonore polyphonique et contrôle gestuel instrumental : retour sur une mise en oeuvre pratique de la guitare hexaphonique (to be published)," in *Proceeding of the symposium "When the guitar electrifies !"*, ser. Collection MusiqueS, B. Navarret, M. Battier, P. Bruguiere, and P. Gonin, Eds., vol. MusiqueS & Sciences. Sorbonne Université Presses, 2020.

[14] R. Graham, "Expansion of electric guitar performance: Practice through the application and development of interactive digital music system," Ph.D. dissertation, Faculty of Creative Arts, University of Ulster, 2012.

[15] E. Bates, "The Composition and Performance of Spatial Music," Ph.D. dissertation, University of Dublin, 2009.

[16] P. Brossier, "Automatic annotation of musical audio for interactive systems," Ph.D. dissertation, Centre for Digital music, Queen Mary University of London, 2006.

[17] S. F. Josel and M. Tsao, *The Techniques of Guitar Playing*, C. Nobach, Ed. Barentreiter, 2014.

[18] J. Schneider, *The Contemporary Guitar*. Roman and Littlefield, 2015, ch. 7 - Microtones: The Well-Tuned Guitar, pp. 141–214.

[19] B. Hopkin and Y. Landman, *Nice Noise: Modifications and Preparations for Guitar*. Experimental Musical Instruments, 2012.

[20] M. Elgart and P. Yates, *Prepared Guitar Techniques*. California Guitar Archives, 1990.

[21] C. Cadoz, "Le geste canal de communication homme/machine - la communication "instrumentale";" in *Revue des Sciences et Technologies de l'Information - Série TSI : Technique et Science Informatiques*. Lavoisier, 1994, vol. 13, no. 1, pp. 31–61.

[22] J. De Souza, "Fretboard Transformations," *Journal of Music theory*, vol. 62, no. 1, pp. 1–39, 2018.