Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Three-Level Model for Fingering Decisionof String Instruments
Gen Hori1
Asia University
hori@asia-u.ac.jp
Abstract. Fingering decisions of string instruments and other instruments differ in that the former involves string assignments as well as finger assignments
while the latter is simply a matter of assigning fingers to notes. The present study
introduces a three-level model for fingering decision of string instruments to describe the structure of the problem and present problem settings of fingering decision based on the model. Our proposed three-level model provides clear perspective for some problem settings of fingering decision. We perform a simulation todemonstrate the flexibility of the three-level model.
Keywords: fingering decision, string instruments, hidden Markov model(HMM)

1Introduction

String instruments have overlaps in pitch ranges of their strings. As a consequence,
they have more than one way to play even a single note and thus numerous ways toplay a whole song. That is why the fingering decision for a given song is not always
an easy task for string players and therefore automatic fingering decision has been attempted by many researchers. As for applications of HMM to fingering decision, Horiet al.[1] applied input-output HMM to guitar fingering decision and arrangement, Nagata et al.[2] applied HMM to violin fingering decision, and Nakamura et al.[3] appliedmerged-output HMM to piano fingering decision. Hori and Sagayama.[4] and Hori[5]proposed extensions of the Viterbi algorithm for fingering decision.
The purpose of the present study is to point out that fingering decisions of string
instruments and other instruments differ in that the former involves string assignments
as well as finger assignments while the latter is simply a matter of assigning fingers tonotes. To describe the structure of fingering decision of string instruments, we introduce a three-level model for string instruments and provide a unified way of looking
at variations of problem settings of fingering decision. Our proposed three-level model
provides clear perspective for some problem settings of fingering decision. We performa simulation to demonstrate the flexibility of our three-level model with fingering decision from score with finger numbers.
The rest of the paper is organized as follows. Section 2 reproduces the guitar fingering decision model based on HMM[1]. Section 3 points out the difference in fingering
decision between string instruments and other instruments and introduces the threelevel model. Section 4 presents problem settings of fingering decision based on thethree-level model and Section 5 performs a simulation for one of the problem settings.
Section 6 concludes the paper.

93Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

2

Fingering Decision Based on HMM

This section reproduces the guitar fingering decision model based on HMM[1] whose
output symbols are musical notes and hidden states are left hand forms, which corresponds to the problem setting of Section 4.1 in this paper. Although we use the monophonic case as an example to simplify the explanation in the following sections, theresults apply to the polyphonic case as well. See [1] for details of the polyphonic case.
2.1

HMM for fingering decision

To play a single note with a guitar, a guitarist depresses a string-fret pair pi on fretboard,pi = (si , fi ),with a finger hi of the left hand and picks the string with the right hand. Therefore, aleft hand form qi for playing a single note can be expressed in a triplet qi ,qi = (si , fi , hi ),where si = 1, . . . , 6 is a string number (from the highest to the lowest), fi = 0, 1, . . .
is a fret number, and hi = 1, 2, 3, 4 is a finger number of the player’s left hand (1,2,3
and 4 means the index, middle, ring and pinky fingers). The fret number fi = 0 meansan open string. The MIDI note number of the note played by the form qi is calculated
as follows where osi denotes the MIDI note number of the open string si ,n(qi ) = osi + fi .
In this formulation, fingering decision is cast as a decoding problem of HMM where afingering is obtained as a sequence of hidden states qi given a score as a sequence of
output symbols nk .
2.2

Transition and output probabilities

The difficulty levels of the moves from forms to forms are implemented in the probabilities of the transitions from hidden states to hidden states; a small value of the transition
probability means the corresponding move is difficult and a large value means easy. We
assume that the four fingers of the left hand are always put on consecutive frets in thispaper for simplicity. This lets us calculate the index finger position (the fret number theindex finger is put on) of form qi as g(qi ) = fi −hi +1. Using the index finger position,
we set the transition probability from hidden state qi to hidden state qj as
1|g(qi ) − g(qj )|× PH (hj )
exp −
(1)
aij (dt ) ∝2dt
dt
where ∝ means proportional and the left hand side is normalized so that the summation
with respect to j equals 1 for all i. The first term of the right hand side is taken from theprobability density function of the Laplace distribution that concentrates on the center

94

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

and its variance dt is set to the time interval between the onsets of the (t− 1)-th noteand the t-th note. The second term PH (hj ) corresponds to the difficulty level of thedestination form qj defined by the finger number hj .
As for the output probability, because all the hidden states have unique output symbols in our HMM for fingering decision, it is 1 if the given output symbol nk is the onethat the hidden state qi outputs and 0 if the given output symbol is not,

1 (if nk = n(qi ))
bik =
.
(2)
0 (if nk = n(qi ))3

Three-Level Model for Fingering Decision of String Instruments

This section identifies the fundamental difference in fingering decision between string
instruments and other instruments, and then introduces a three-level model for fingering
decision of string instruments.
3.1

Note-tablature-form tree

For example, on the piano, there is only one key on the keyboard to press for each note,and therefore fingering decision for a given sequence of notes is a matter of deciding
which finger to press on the key for each note (Fig.1, right). On the other hand, withthe guitar, each note corresponds to several string-fret pairs that play it, and in addition, we have a matter of which finger to press for each string-fret pair (Fig.1, left). In
other words, fingering decision for the piano is simply a matter of finger assignments,
while fingering decision for the guitar consists of string assignments followed by finger assignments. This situation with the guitar is illustrated in a tree diagram (Fig.1,
left) which we call “note-tablature-form tree.” While the tree diagram in Fig.1 is for amonophonic note, we can draw the same diagrams for a polyphonic chord as well.

Fig. 1. Note-tablature-form tree for guitar (left) and corresponding diagram for piano (right) illustrating difference between string instruments and other instruments

3.2

Three-level model

To describe the above-explained situation with fingering decision of string instruments,
we introduce a three-level model for string instruments that consists of (1) note level,(2) tablature level, and (3) form level (Fig.2). In relation to the notation introducedin Section 2.1, the note level contains the information of n(qi ), the tablature level95

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021
pi = (si , fi ), and the form level qi = (si , fi , hi ), respectively. In guitar scores, thep(si ,thefi ),tablatureand the contains
form levelhi the), respectively.
In theguitarscores,thei = andi = (si , fi ,ofscore
theqinformationnote level andtablaturelevel,
score
and
the
tablaturecontains
the
information
of
the
note
level
and
the
tablaturelevel,p= (s , f ),The
andfinger
the formlevel attachedq = (s to, fthe
, h notes), respectively.
In guitarscores,respectively.
numbers
in the score,
togetherwith thepii = (sii , fii ), and the form level qii = (sii , fii , hii ), respectively. In guitar scores, therespectively.
The
numbers
to theinSection
thetogetherwiththescore and thetablaturetheattachedinformation
ofnotesthe(seenotelevelscore,andthetablaturelevel,
tablature,makeupfingerthe contains
informationof the form
level
4.3).
From
the
viewscore and the tablature contains the information of the note level and the tablature level,tablature,
make
up
the
information
of
the
form
level
(see
Section
4.3).
From
the
viewrespectively.
The finger
numbers
attached
thehidden
notes inthe corresponds
score, together
withformthepointof fingering
decisionbased on
HMM,tothestatesto therespectively. The finger numbers attached to the notes in the score, together with thepointof fingering
based symbols
onofHMM,
thelevel
hiddentosettings
theviewform
tablature,makeupdecisionthethe form(seestates
Section
From
the
level
and
the
setting
ofinformation
observedvariesdependingoncorresponds
the4.3).
problemastablature, make up the information of the form level (see Section 4.3). From the viewlevel
andtheinsettingof observed
symbolsvariesdependingoncorresponds
the problemtosettingsaspointof seefingeringdecision
based
on HMM,the hidden
statesthe form
wewill
the following
sections.
point of fingering decision based on HMM, the hidden states corresponds to the form
wethe following
sections.
levelwillandseetheinsettingof observed
symbols varies depending on the problem settings aslevel and the setting of observed symbols varies depending on the problem settings as
we will see in the following sections.
Note Level
we will see in the following sections.
Note Level
Note Level
Note Level
Tablature
Level
Tablature Level
Tablature Level
Tablature
Level
Form Level
Form Level
Fig. 2. Three-level modelfor Level
fingering decision of string instruments
Form
Form
Fig. 2. Three-level modelfor Level
fingering decision of string instruments

Fig. 2. Three-level model for fingering decision of string instruments

Fig. Settings
2. Three-level
model onfor fingering
decision ofstring instruments
4 Problem
Based
Three-Level
Model
4 Problem Settings Based on Three-Level Model
4This
Settings
Based
Three-Level
Modelsection provides
a unifiedwayon
of looking
at variationsin problem settings of fin4 Problem
Problem
Settings
Basedon
Three-Level
Model

This
sectionprovidesway of looking
at variationsin problemtaking
settings
fingeringdecisionbased aonunified
the three-level
model forstring instruments,
the ofguitargeringdecisionbased aonunified
thethree-level
model
string instruments,
takingthe ofguitarThis
provideswayisofcast
lookingatvariationsin problem
settings
finas
ansection
example.
Fingeringdecisionas a fordecoding
problemof HMM
where
theThis section provides a unified way of looking at variations in problem settings of finas
an example.
Fingeringdecisioniscast
as aondecoding
problemof HMM
where
thegeringdecisionbasedon thethree-level
model
for
string
instruments,
takingthe guitar
settingof observed
symbolsvariesdependingtheproblemsettings.
The first
probgering decision based on the three-level model for string instruments, taking the guitar
setting
observed
symbolsvariesdependingthe problem
settings.
The first
probas anisexample.
Fingering
cast asandaon
decodingof HMM
where
the
lem
aofconventional
one decision
while
theis
secondthird
onesproblem
obtainclear
perspectives
inas an example. Fingering decision is cast as a decoding problem of HMM where thelemisofaofconventional
one whilethesecond andonthird
ones obtainclear perspectives
insetting
observedvariesdependingthe problem
settings.
The first problightour
proposedsymbols
three-level
model.
setting of observed symbols varies depending on the problem settings. The first problight
proposed three-level
model.
lem isofaour
conventional
one while thesecond and third ones obtain clear perspectives in
lem is a conventional one while the second and third ones obtain clear perspectives inlight of our proposed three-level model.
light
our proposed
three-level
model.
4.1 ofFingeringdecisionfrom score
4.1 Fingering decision from scoreIn
problem setting,generate
4.1thisFingering
decisionwefrom
scorea sequence of forms from a score, taking the note
4.1thisFingering
decisionwefrom
score sequence of forms from a score, taking the noteInsetting,
generatelevel asproblem
the observed
symbols
and thea form
level as the hidden states (Fig.3, left). Thislevel
asproblemthe observed
andthea form
levelofas
thehiddenstates(Fig.3,In athissetting,
we generatesequencefroma score,takingleft).
the
note
is
conventional
andsymbols
commonproblemsetting
informs
guitarfingeringdecisionandThishas
In this problem setting, we generate a sequence of forms from a score, taking the noteisa conventional
andcommonproblemin theguitarfingeringdecisionandThis
has
level
as
thestudied
observed
symbols
andthe formsetting
level ashiddenstatesthat(Fig.3,been
wellincludingourprevious
study[1].
Here
we
note
the left).
transitionlevel as the observed symbols and the form level as the hidden states (Fig.3, left). This
been
well studied
includingourproblem
previous
study[1].
Herenote thattheintransition
is a conventional
andcommonsettingin guitar
fingeringdecision
andformhasprobability
reflectingthedifficultyof the form
transition
canwe
be definedonly
theis a conventional and common problem setting in guitar fingering decision and has
probability
reflecting
the difficultyofwhichthe form
canwe
bethe
definedonly
the formbeen andwell notstudiedourprevious
study[1].
Here
note
that
the
level
in theincluding
tablaturelevel,
wetransition
can seefromformulaofintransition
been well studied including our previous study[1]. Here we note that the transitionlevel
and not
in Eventhe tablature
wetransition
can see fromformulareflectingthedifficultyofwhichtheform
can bethedefinedonlyofto
intransition
theform
probability
(1).
when
welevel,
only
need
to
generatetablature,
we haveperformprobability reflecting the difficulty of the form transition can be defined only in the form
probability
(1).
when
welevel,onlywhichneed totablature,
we haveoftotransition
perform
level andnotin Eventhethetablature
wegenerate
can see fromthe formulaHMM
decoding
inform level.
level and not in the tablature level, which we can see from the formula of transition
HMM
decoding
in the whenform level.
probability(1). Even
we only need to generate tablature, we have to perform
probability (1). Even when we only need to generate tablature, we have to perform
HMM decoding in the form level.
HMM
decoding in
the formlevel.
4.2 Fingering
decisionfromtablature
4.2 Fingering decision from tablatureIn
problem setting,generatea sequence of forms from a tablature, taking the
4.2thisFingering
decisionwefromtablature
4.2thisFingering
decisionwefromtablatureIn
problemgeneratea sequenceof forms
tablature,takingthetablaturelevel assetting,the observed
symbols
and the form
levelfromas thea hidden
states(Fig.3,tablaturelevel
the thatobserved
symbols
andonly
the formlevel
as thea hidden
states(Fig.3,In this Hereproblemsetting,we
generatea sequenceof forms
fromtablature,takingtheright).
weasnotea tablatureshows
stringassignments
for notes
anddoes
In this problem setting, we generate a sequence of forms from a tablature, taking theright).
Here
weasnotetablatureshows
stringlevelassignments
notes
and
does
tablaturelevel
the thatobserved
symbols
andonly
thealthough
form
the hidden
states(Fig.3,
notcontaininformationofa finger
assignments,
it isaseasy
forforskilledguitarists
tablature level as the observed symbols and the form level as the hidden states (Fig.3,
notcontaininformation
fingerassignments,
it is foreasya forguitarists
right).
Here
we notefinger
thatofaassignments
tablatureshows
stringassignments
forskillednotes
and doestofindappropriate
and only
thusalthougha fingering
given
tablature.
Anright). Here we note that a tablature shows only string assignments for notes and doestoappropriate
fingerand isthus
a fingering
given
tablature.
An
notfindcontaininformation
ofassignments
fingerassignments,
although
it is foreasya forguitarists
application
exampleof thisproblemsettingdifficulty
assessmentofaskilledtablaturewhere
not contain information of finger assignments, although it is easy for skilled guitarists
application
exampleof thisasproblemsettingisthusdifficulty
assessmenta tablature
where
to findappropriate
fingerassignments
andof
fingering
fortransition
a of
given
tablature.
Anthedifficultyis calculated
the reciprocal
the aproductof theprobabilities
to find appropriate finger assignments and thus a fingering for a given tablature. An
thedifficultyis calculated
thereciprocal
the productof the transition
probabilities
application
exampleof thisasproblem
setting ofis difficulty
assessmentof a tablature
where
along
the generated
sequenceof forms.
application example of this problem setting is difficulty assessment of a tablature wherealong
the generated
sequenceof forms.
the difficultyis calculated
as thereciprocal of the product of the transition probabilities
the difficulty is calculated as the reciprocal of the product of the transition probabilities
along the generated sequence of forms.
along the generated sequence of forms. 96

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Note Level

Observed symbols

Note Level

Tablature Level

Form Level

Tablature Level

Form Level

Hidden states

Fingering Decision from Score

Observed symbols

Hidden states

Fingering Decision from Tablature

Fig. 3. Two problem settings based on three-level model

4.3

Fingering decision from score with finger numbers

There are guitar scores without tablatures with finger numbers attached to some key
notes (Fig.4, left), which is enough for skilled guitarists to find a fingering for wholephrase. From the viewpoint of our proposed three-level model, this is a case where
the whole information of the note level and the partial information of the form levelare given to generate a sequence of forms. The fingering decision in this case is implemented as a decoding problem of HMM whose observed symbols are the notes andhidden states are forms limited to ones with indicated finger numbers. We will see somesimulation results of this problem setting in the following section.
Note Level

information+ Partialfrom Form Level

Tablature Level

Form Level

Fig. 4. Score with finger numbers (left) and corresponding problem setting (right)

5

Simulation

From the problem settings described in the previous section, we perform a simulationof one presented in Section 4.3 to demonstrate the flexibility of our proposed threelevel model. The results for four scores are given in Fig.5 where the sequence of notes(C major scale) is common to all and the finger numbers with red circles are given
while other finger numbers and the tablatures are generated by HMM. In the transition
probability (1), we set PH (1) = 0.4, PH (2) = 0.3, PH (3) = 0.2 and PH (4) = 0.1
which means forms using the index finger are the easiest and the pinky finger the mostdifficult. From the results, we see that HMM generates appropriate fingerings for all
the scores minimizing change in the index finger position and that specifying a finger
number to one note can change fingerings for the rest seven notes.

97

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Fig. 5. Simulation results of fingering decision from score with finger numbers

6

Conclusion

We have pointed out the difference in fingering decision between string instruments andother instruments and introduced a three-level model for fingering decision of string
instruments. Based on the model, we have provided a unified way of looking at threevariations in problem settings of fingering decision and demonstrated the flexibility ofour proposed three-level model using a simulation for fingering decision from scorewith finger numbers. There are other instruments than string instruments for which wehave more than one way to play a single note. For such instruments, we can consider afingering model with a middle level corresponding to the tablature level of our model for
string instruments. We leave the extension of our three-level model to such instruments
to our future study.
Acknowledgments. This work was supported by JSPS KAKENHI Grant Number
21H03462.

References
1. Hori, G., Kameoka, H., Sagayama, S.: Input-Output HMM Applied to Automatic Arrangement for Guitars. Journal of Information Processing, 21, 2, pp. 264–271. (2013)
2. Nagata, W., Sako, S., Kitamura, T.: Violin Fingering Estimation According to Skill Level
Based on Hidden Markov Model. In: Proceedings of International Computer Music Conference and Sound and Music Computing Conference (ICMC/SMC2014), pp.1233–1238,
Athens, Greece. (2014)
3. Nakamura, E., Ono, N., Sagayama, S.: Merged-Output HMM for Piano Fingering of Both
Hands. In: Proceedings of International Society for Music Information Retrieval Conference(ISMIR2014), pp.531–536, Taipei, Taiwan. (2014)
4. Hori, G., Sagayama, S.: Minimax Viterbi Algorithm for HMM-Based Guitar Fingering Decision. In: Proceedings of International Society for Music Information Retrieval (ISMIR2016),
pp. 448–453, New York City, U.S.A. (2016)
5. Hori, G.: Extension of Decoding Problem of HMM Based on Lp -Norm. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3989–3993,
IEEE. (2018)98