A CHORUS-SECTION DETECTION METHOD FOR LYRICS TEXT
Kento Watanabe
Masataka Goto
National Institute of Advanced Industrial Science and Technology (AIST), Japan{kento.watanabe, m.goto}@aist.go.jp

ABSTRACTThis paper addresses the novel task of detecting chorus
sections in English and Japanese lyrics text. Althoughchorus-section detection using audio signals has been studied, whether chorus sections can be detected from text-onlylyrics is an open issue. Another open issue is whether patterns of repeating lyric lines such as those appearing in chorus sections depend on language. To investigate these issues,
we propose a neural network-based model for sequencelabeling. It can learn phrase repetition and linguistic features to detect chorus sections in lyrics text. It is, however,difficult to train this model since there was no dataset of
lyrics with chorus-section annotations as there was no priorwork on this task. We therefore generate a large amountof training data with such annotations by leveraging pairsof musical audio signals and their corresponding manually
time-aligned lyrics; we first automatically detect chorus
sections from the audio signals and then use their temporal
positions to transfer them to the line-level chorus-section
annotations for the lyrics. Experimental results show thatthe proposed model with the generated data contributes todetecting the chorus sections, that the model trained onJapanese lyrics can detect chorus sections surprisingly wellin English lyrics and that patterns of repeating lyric lines
are language-independent.
1. INTRODUCTION
The digitization of lyrics collections has opened various
areas of lyrics-based research in the Music Information
Retrieval (MIR) community, such as research on lyrics
browsing [1–3], lyrics genre classification [4–6] and lyricsto-audio synchronization [7–17]. Lyrics are usually plain
text without any annotations, and some researchers haveanalyzed their structure, such as paragraph structure andtopic transitions between paragraphs [18–22]. For example,
Fell et al. [18] and Watanabe et al. [19] estimated section
boundaries in lyrics text without empty lines but were notable to assign a section label such as verse or chorus to eachestimated section. Chorus sections were not detected in
lyrics text.
The goal of this paper is to achieve automatic chorusc Kento Watanabe, Masataka Goto. Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Attribution: Kento Watanabe, Masataka Goto, “A Chorus-Section Detection
Method for Lyrics Text”, in Proc. of the 21st Int. Society for Music
Information Retrieval Conf., Montréal, Canada, 2020.

section detection for lyrics text. This task has not beenstudied, though chorus-section detection, as well as music
structure analysis, for audio signals has been a popular topic
of research in the MIR community [23–41]. Since whether
chorus sections can be detected from text-only lyrics is an
open issue, it is worth investigating this issue from academic
viewpoints. Moreover, a chorus-section detection method
for lyrics text has potential applications. For example, when
listeners want to find lyrics with a chorus section having
a particular phrase such as “I love you” for the purpose of
singing, reusing its chorus section in a short video clip, etc.,
it is necessary for a lyrics search system to automatically detect which lines of the lyrics are included in chorus sections.
The detected lyric lines of chorus sections could be used
in a lyrics viewing function of music services displaying
lyrics with those lines highlighted by a different color ortypeface. Automatic lyric video generation technologies
could give those lines more vivid animations.
Chorus sections are the most repeated and memorable
portions of a song [39]. Since it is not easy to explore
heuristic rules to find such sections, most existing chorussection detection methods for audio signals have leveraged
repetitive patterns of those sections within a song. In thispaper, we propose a supervised model that can detect chorus sections in English and Japanese lyrics. Our model uses
both structural features that represent patterns of repeating
lyric lines and linguistic features that are calculated fromword2vec [42] and context2vec [43]. To detect chorus sections using only plain text without any labels or even empty
lines (i.e., section boundaries), we investigate a model andfeatures effective for chorus-section detection. Experimental results show that our proposed model outperforms alternative baseline models and that combining structural andlinguistic features contributes to better performance.
Although such a supervised model needs a large datasetof lyrics with line-level chorus-section annotations for its
training, there was no such dataset as there was no priorwork on chorus sections in lyrics text. To address thisissue of lacking training data, we generated a dataset consisting of 9,313 English and 91,459 Japanese lyrics withchorus-section annotations by utilizing pairs of musical audio signals and their corresponding manually time-aligned
lyrics. We first automatically detected chorus sections inaudio signals of a song [39]. Then, since each lyric line
had the corresponding start time within the song, we could
find lyric lines that temporally correspond to the durationof each detected chorus section. We thus obtained the annotated dataset by assigning a chorus label to those lyric351

Proceedings of the 21st ISMIR Conference, Montréal, Canada, October 11-16, 2020
Text (a sequence of lines 𝑥 " in lyrics)

Label 𝑦"1: ooo I wanna kiss you
2: loving you is my dream tonight
3: ooo hold me tenderly
4: loving me with all your heart
5: boy you never tell me that you love me
6: I'm going crazy wondering about you baby
7: do you really know boy how much I care
8: could you really show me how deep is your love ?
9: just close you eyes and hear my heart
10: the sweet sweet beat of my love
11: can't you tell I'm hungry baby
12: for only you can make me smile
13: ooo I wanna kiss you
14: loving you is my dream tonight
15: ooo hold me tenderly
16: loving me with all your heart
17: can't you understand me my point of view
18: do you really love me beyond all words
19: I just need to hear now from your sweet lips
20: I'm the only girl you ever want to kiss
21: just close your eyes and hear my heart
22: the sweet sweet beat of my love
23: can't you tell I'm hungry baby
24: for only you can make me smile
25: just close your eyes and hear my heart
26: the sweet sweet beat of my love
27: can't you tell I'm hungry baby
28: for only you can make me smile
29: ooo I wanna kiss you
30: loving you is my dream tonight
31: ooo hold me tenderly
32: loving me with all your heartchorus (1)chorus (1)chorus (1)chorus (1)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)chorus (1)chorus (1)chorus (1)chorus (1)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)not-chorus (0)chorus (1)chorus (1)chorus (1)chorus (1)Self-similarity Matrix
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Figure 1. Example of lyrics with chorus-section annotations and corresponding self-similarity matrix in which each cell
represents the similarity between two lyric lines. These lyrics are from “How Deep Is Your Love?” (RWC-MDB-P-2001 No.
81 in the RWC Music Database [44]).
lines and a not-chorus label to the other lines. Experimental results show that the model trained with this large
automatically generated dataset performs better than themodel trained with a smaller manually annotated dataset
and that the model trained on Japanese lyrics can detectchorus sections surprisingly well in English lyrics.
2. LYRICS CHORUS-SECTION DETECTIONTASKThe left side of Figure 1 shows an example of lyrics withchorus-section annotations (labels). The lyrics of a song
are a sequence of lyric lines, each line having a sentence orphrase. In this example there are three highlighted chorus
sections that have exactly the same four lines, though inother songs, lyrics of chorus sections are repeated with somemodifications. To maximize the applicability, as shown inthis example, we assume that the input text of lyrics does
not have any section boundaries. Even though some lyrics
contain empty lines at those boundaries, those lines aredeleted in advance. We also assume that the input text does
not have explicit chorus labels such as “(chorus)” at the
beginnings of chorus sections. Even though some lyrics
contain those labels, they are deleted as well. When lyrics
contain a repetition label such as “(* repeat)”, it is manuallyreplaced with the corresponding lyric lines.
We formulate this chorus-section detection task as asequence labeling problem: predicting the chorus or
not-chorus status for each lyric line. Let Xs be thelyrics of a song s composed of T lines of text: Xs ={x1 , ..., xt , ..., xT }. Each lyric line xt has a binary label yt .
If yt = 1, xt is in a chorus section. If yt = 0, xt is not in achorus section. Ys denotes a sequence of labels correspond352

ing to Xs : Ys = {y1 , ..., yt , ..., yT }. In the training step,
the model learns the conditional probability P (Ys |Xs ). In
the validation/testing step, the trained model has to predictlabels Ys for given lyric lines Xs .
Chorus sections cannot be detected by simply extracting
repeated lines since those lines often correspond to nonchorus sections. For example, lyric lines 9–12 and 21–24
in Figure 1 are exactly repeated, but those lines are not inchorus sections. It is also difficult to manually define a
set of rules to find various chorus sections. We therefore
prepare various features that could be useful for machinelearning to deal with various types of chorus sections.
3. COMPUTATIONAL MODELING OF CHORUS
SECTIONS IN LYRICS
We propose a neural network-based model for sequencelabeling by using structural features that are self-similarity
matrix (SSM) representations. SSM representations arewidely used in computational music structure analysis, but
we use different representations for lyrics. In addition to
structural features, our model utilizes linguistic features
such as word vectors and sentence vectors calculated fromword2vec [42] and context2vec [43], which are widely usedin natural language processing.
In the following sections, we first describe nine SSMs
for capturing patterns of repeating lyric lines and explain
how to encode the SSMs for neural networks (Section 3.1).
We then describe the linguistic features obtained by vectorizing the semantic/syntactic information of lines using
word2vec and context2vec (Section 3.2). Finally, we describe a neural-network-based sequence labeling modelwith these structural and linguistic features (Section 3.3).

…

…

Part-of-speech
similarity(sim
capture similaribetween the phonetictranscriptionof twolines.
thispos ):InTostudy, we used thepronunciation
dictionary
extract a normalized
tiesCMU
in grammaticalstructure,
we tocalculate(CNN) architecture, as the same as Fell et al, which allows
the phonetic transcriptionof editlines.distanceFor example
the thephonetic
Levenshtein
betweensequence of part-oftranscription of the line “I love you” is [AY1, L, AH1, V, Y,
the neural network
to Network
extract translation,scaling, and rotaFigure 3. Convolutional
Neuralto vectorize
speech of two lines.
UW1].
SMMs.
tion invariant features anywhere on the input image. Our
Word
vectorsimilarity
(sim
):
To
capturethesemantic
w2v similariPart-of-speech similarity (simpos ): To capture
study shares the same motivation to capture chorus sections
similarity
betweentwoOctoberlines,awesimplyaveragewords’
ties in grammaticalstructure,
we calculate
normalized
Proceedings of the 21st ISMIR Conference,Montréal,Canada,11-16,
2020
from various
SSM aspatterns,of SSM location and(CNN) architecture,as the same
Fell et regardlessal, which allows
vectorsof eachline using
pre-trained
Levenshtein edit
distance
betweenthe sequenceof word2vec
part-of- embeddings
relativesize. translation, scaling, and rotathe neural network
to extractspeech of two lines.
and compute its cosine similarity.
tion invariant features
anywhereon theinput image.
Our 3. After theThis CNN
structureis illustrated
in FigureWord vector similarity(simw2v): To capture
semantic
Context vector
similarity(simtheof wordc2v ): The average
3.1 Structural Features
Self-similarity
matricesstudy shares thesamearemotivation
to wecapturechorussections
SSMs
calculated,
extract
fixedwindow
sub-matrices
similarity between
twolines,
we
simplyaveragewords’
vector by word2vec assumes a “bag of words” from(i.e., variousthe
SSM patterns, regardless of SSM location andvectors of each line using pre-trained word2vec embeddingson the target line:difference between “dog bites person” and “person
bitessize. centeredrelativet
2w⇥T
compute itsfor
cosinesimilarity.
Most previous work on music structureandanalysis
audio
Sm= isSSM
1, ..., t3.+ Afterw] 2 R, where w
m [t win+Figure
cannot(simbe captured
in this assumption). To consider
This CNN structureillustrated
theContext vectordog”
similarityc2v ): The average of word
is a fixedwindow
size. window
The inputof the CNN is nine subsignals [23–41] identifies repeated musicalsectionsby
usSSMsarecalculated,
we
extract
fixed
sub-matrices
wordorder,
we
vectorize
the
lines
using
pre-trained
convector by word2vec assumes a “bag of words” (i.e., thett
2w⇥T ⇥9{Sstr , ..., SsyL } 2 R, where the number
centeredon thematrices
target line:
text2vec,
extension
ofand
word2vec,
which
lines
extract
between“doganbites
person”“personfixed
bites encodes
ing a SSM like that shown in Figure 1.difference
Repeated
sectionstwindowTosub-matrix
Sm= SSM[t windicate+ 1, ..., tthe
+ w]
2 R2w⇥T, where w
ofmSSMs
numberof channels.
The kernel sizedog” cannot beusing
capturedin thisconsider
LSTM.
Weassumption).
compute similarity
simc2v using cosine$ The$ } of-.CNN
×0×1is nine suba fixed window
input
lead to high values in diagonals of theword
matrix,andthosethe lines using pre-trained con- is Sub-matrices{𝐚first…,𝐚ℝlayerof thesize.
is (w + 1) ⇥ (w + 1) so that#$% ,convolutional#() ∈ theorder, wevectorize
similarity.
tt⇥9
matrices {Sstr, ..., Sfeature
R2w⇥T
the number
syL } 2cantext2vec,
extension ofreword2vec, which encodes lines
each
capture,awhere
prospective
chorus section. In
patterns are used to identify the structure.
Toan capture
Word syllable
count similarity (simsyW ): As aofclue
for indicate
SSMs
the number of channels. The kernel size
using LSTM. We compute similarity simc2v using cosine
thisnetwork,
allconvolutional
layersemploy batch normaldetecting
chorussections,
we
use
the
sequence
of
word
of
the
first
convolutional
layer
is
(w
+
1)
⇥
(w
+
1)
so
that
peated lyric lines that often appear in chorus
sections,
wesimilarity.
izationand athe
ReLU activation
function.InEach resulting
feature cancaptureprospective
chorus section.
syllablecounts on(simeachsyWline.
For aexample,
syllableWord
count similarity): As
clue for wordeachalso compute the SSM from lyrics text,
butsyllable
the designoftensoris downsampled
by max-poolingwith w ⇥ w kernel
thisaway”
network, allconvolutional
layers employ
batch normalcountsof twolost yourself
detecting chorus
sections,
welinesuse “Sometimes
the sequenceyouof wordAfter
max pooling,
we usea dropoutlayer (p = 0.1)
ReLU
activation
function.
Eachresultingthe similarity measure to compute eachcellcountsof theSSMsyllableon eachline. Forsyllableand“Everytime
youexample,
just closeword
your eyes”
are {2, 1,ization
1, 2, 1}and thesize.
tensor is downsampled
by max-poolingw ⇥ wthekernelfor regularization.
Then,with
we apply
1D convolutional
1
counts of two lines“Sometimes
lost yourself
away”
and {2,
1, 1, 1, 1, 1}yourespectively; linesthat have a similar
2D-convolution
is important. We propose to use the following
ninevariamax
pooling,
use asize
dropoutlayer(p = 0.1) by the vector
ReLU size. After max
layerwith awekernelof
w pooling
anddownsample
and “Everytimesyllable
you justcountclose sequence
your eyes”arearelikely{2, 1,to1,be
2,+1}
the
samefor
sectionregularization. Then, we apply the 1D convolutionaltions of similarity measures simm , where
the 1 lines thatand {2,m
1, 1,denotes
1, 1, 1} respectively
have a similarof each channel in the last max-pooling layer. We performin the song. In this;study,
we useDynamic Time layer
Warpingwith a kernel
sizesize:of w and downsample by the vector
Kernelare likely to be the same section
the above procedure independently for each line xt andvariation. Some of the similarities aresyllable
basedcounton sequenceprevious
to calculate
the similarity
between sequencesof eachofchannel in the last max-pooling layer. We performin the song. In (DTW)
this study,
we use Dynamic
Time Warping
obtainthe
CNN-based feature
vectorthe above procedure independently
for eachline vxtt. anddifferentsuch
as syllable
count sequences.
(DTW) to calculate
the lengths,
similarity
betweensequencesof
studies [18, 19].
Kernel size
the CNN-based feature vector v t .
Linesyllablecountcountsimilarity(sim ): We canobtain
also use
different lengths,
such
as syllable
sequences.

syL
String similarity (simstr ): a normalized
Levenshtein
edit
3.2 Linguistic Features
Line syllable countsimilarity(simsyLcounts): We canalsowords
use in each line.
the sumof the syllableof all
1D-convolution
3.2 Linguistic Features
the
sum
of
the
syllablecounts
of
all
words
in
each
max
pooling
distance [45] between the characters of two lyric lines.
For example, in the lyrics of Figure 1,line.
the
total
syllable+ ReLU expressions tend to appear in chorus relaSome linguistic
For example, inthe lyrics
ofFigure1, thethechorus
total syllable
Somelinguistic
expressions
to appear in chorus relacount
of
the
first
line
in
section
is
all
six,
but
Head similarity (simhead ): a normalized
edit
tionships. For tend
example,analyzing lyric data with chorus
countLevenshteinof the first line in
the chorus section is all six, buttionships.
example,network
analyzing lyric
datawith chorus
thecount
total syllable
count intheis second
line
is eight.
For- Forneural
Figure2.
Convolutional
forphrases
SSMs.
markup,
we
found
that
about
theasfuture such as
the
total
syllableinthesecondline
eight.
Fordistance between the characters of the first two wordsof
markup,
we
found
that
phrasesabout
the futuresuchmally, wesimilarityof the total syllable
mally, we calculated
the calculated
similarity theof thetotal syllable“I’ll”,“Let’s”
and“I
wanna”
tend
to chorus
appear in the chorus“I’ll”, “Let’s” and “I wanna” tend to appear in the
two lyric lines.
countof eachline inprocedure.
the following
We shiftcount of each linein thefollowing(1) procedure.
We shift (1) section,
the“have
past suchas “have been”
while section,phrases while
about phrases
the pastaboutsuch asbeen”
a windowof fourlines
overlyricsandover
extractfour
a windowof fourlines
lyricsandlines
extract four
lines
Lyric
Line
syllable(sim):lessWecanLevenshtein
edit
Tail similarity (simtail ): a normalized
andcount“didn’t”similarity
tend
to appeartend
less toin appear
thesection.
To section. To
syLchorusand “didn’t”
in thechorus
Lt = {xt , xt+1L, txt+2, xtt+3}. (2)
The
similarity
between= {x, xt+1, xt+2, xt+3}. (2) The
similarity exploit
betweensuch a exploit
linguistic
expression
ofin
theeachchoruslyric
section,awordslinguistic
expressionofthe chorus section,
also
total
count
ofsuchall
distance between the characters of thethelastline xtwo
xcalculated
DTWuseofby
Lthe
andDTW
Lt . of syllable
t and words
t is
tthe
the
line
xtof
and xtbyisthecalculated
Ltwe
andpropose
Lt . twofeatures.
welinguistic
propose two
linguistic features.
We
calculatednine
self-similarity
matrices
SSM
2m
For example,
inall
the chorus
sectionsshown
Fig-ofAverage
vectors(wordave ):We use inaverage
two lyric lines.
We calculated nineline.
self-similarity
matrices SSM
m 2 of word
Average of word vectors(word this): study,
We use average ofRT ⇥T , where eachcell is a similarity measure as described⇥T
the word vectors of a given line as features. Inave
RTcalculate
each cellis a1,
similarity
measureas described
ure
thethetotal
count
of
the
first
lyric
line
is
6
and In this study,
rhymesin, where
thefeature
Phonetic similarity (simphone ): To capture
above. Then,tovectors
fromabovesyllable
theword
vectors
of
a
given
line
as
features.
theaverageofword
vectorsusing
pre-trained
word2vec,to calculate featurefrom the abovenine SSMs, weabove.
exploitThen,the Convolutional
Network
the averageofwords.
word
using pre-trained
word2vec,that Neuralof thevectors
second
line
is 8.
We
calculatethevectors
similarity
ofskipping
out-of-vocabulary
lyrics, we calculate a normalized Levenshtein
editnine
distance
SSMs, we exploit the Convolutional Neural Network
skipping out-of-vocabulary
words.(wordseq ):Vector representations
for word sequences
The song is from the RWC Music Database(RWC-MDB-P-2001
such total syllable countsof each pairof lyric lines
by using
between the phonetic transcriptions of No.92)
two [3].
lyric lines.
We
Word order cannot
modeled by theforaverageof word (wordseq ):Vectorberepresentations
word sequences
1 The song is from the RWC Music Database (RWC-MDB-P-2001
the following procedure. (1) Word
We extract
a window
use the CMU pronunciation dictionary 1 to extractNo.92)
the[3].
phoorder cannot
be modeledof
byfourthe average of word
lyric lines Lt = {xt , xt+1 , xt+2 , xt+3 } and shift it over the
netic transcription. For example, the phonetic transcription
entire lyrics of a song. (2) The similarity between the lyricof “I love you” is [AY1, L, AH1, V, Y, UW1].
lines xt and xt0 is calculated by DTW of Lt and Lt0 .
Part-of-speech similarity (simpos ): To capture similariWe thus calculated nine SSMs Am ∈ RT ×T , where eachties in grammatical structure, we calculate a normalized
cell is a simm explained above. Then, to calculate feature
Levenshtein edit distance between the part-of-speech (POS)
vectors from the above nine SSMs, we exploit a convolusequences of two lines. We use the default POS tagger intional neural network (CNN) architecture to detect textual
the NLTK package [46].
macro structures from various patterns in SSMs regardlessWord vector similarity (simw2v ): To capture the semanof their locations and relative sizes in SSMs. Except for
tic similarity between two lyric lines, we simply average
network parameters, this CNN architecture is the same as
vectors of the words of each lyric line by using pre-trainedthat of Fell et al. [18], as we share the same motivation: toword2vec [42] and compute their cosine similarity. This
extract translation, scaling and rotation invariant features“bag of words” representation does not differentiate “dogfrom the input image (in our case, nine SSMs).
bites person” from “person bites dog”.
Figure 2 illustrates the CNN structure. After calcuContext vector similarity (simc2v ): To consider the word
latingthe nine SSMs, we extract fixed-size elongatedorder, we vectorize the lyric lines using pre-trained conrectangle
sub-matrices centered on the target lyric line:
text2vec [43], an extension of word2vec, which encodes
t
2w×T
a
=
A
, wherea sequence of words by using Long Short-Term Memorym [t − w + 1, ..., t + w; 1, ..., T ] ∈ R
m
w
is
a
fixed
window
size.
The
input
of
the
CNN
is
nine
sub(LSTM) networks [47]. We then compute their cosine simitt
2w×T ×9
matrices
{a
,
...,
a
}
∈
R
,
where
the
numberlarity to obtain simc2v .
strsyLof channels corresponds to the number of SSMs. The kernel
Word syllable count similarity (simsyW ): Since repeatedsize of the first 2D-convolutional layer is (w + 1) × (w + 1)
phrases sometimes have the same number of syllables evenso that each feature can capture a prospective chorus section.
if their words are different, we use a sequence of word
Each resulting tensor is downsampled by max-pooling withsyllable counts on each lyric line. For example, the word
w × w kernel size. We then apply the 1D-convolutional
syllable counts of the two lyric lines “Sometimes you lostlayer with a kernel size of w and the last max-pooling layer
yourself away” and “Everytime you just close your eyes” 2
downsamples the resulting vector to a scalar. In this netare {2, 1, 1, 2, 1} and {2, 1, 1, 1, 1, 1}, respectively. Whenwork, all convolutional layers employ the ReLU function.
successive lyric lines have similar syllable count sequences,
We can perform the above procedure independently for eachthey are likely to correspond to the repetition of sections.
lyric line xt and obtain the CNN-based feature vector v t .
We use dynamic time warping (DTW) [48] to calculate thesimilarity between syllable count sequences.
3.2 Linguistic Features
0

0

0

0

1

1 http://www.speech.cs.cmu.edu/cgi-bin/cmudict
2 This lyrics are taken from the RWC Music Database (RWC-MDB-P2001 No.92) [44].

Some expressions tend to appear in chorus sections. To
quantify this tendency, we calculate the difference between
353

Proceedings of the 21st ISMIR Conference, Montréal, Canada, October 11-16, 2020

word tri-gram probabilities in the chorus and non-chorus
sections. Table 1 shows the word tri-grams that frequently
appear in both of the sections. Here, Pc and Pn denoteword tri-gram probabilities in the chorus and non-chorus
sections, respectively. As shown in this table, we found thatphrases about the future (e.g., “I’ll” and “Let’s”) tend to
appear in chorus sections more often than do phrases about
the past (e.g., “have been” and “didn’t”). To exploit suchtendencies, we propose two linguistic features.
Average of word vectors (lingave ): We use the average ofword vectors of a given lyric line as a feature. The word
vectors are obtained using pre-trained word2vec [42], skipping out-of-vocabulary words.
Vector representations for word sequences (lingseq ): To
consider the word order that cannot be modeled by lingave ,we use pre-trained context2vec [43] that enables vectorization of a lyric line by putting a sequence of word vectors
into the LSTM.
We calculate the above linguistic feature vectors for eachlyric line xt and obtain their concatenated vector ut .
3.3 Neural Network-based Sequence Labeling Model
To solve the sequence labeling problem, we use the standard Bidirectional Long Short-Term Memory (Bi-LSTM)
networks [49] to compute the conditional probability
P (Ys |Xs ). The neural network structure is illustrated in
Figure 3.
The input to the Bi-LSTM layer at each time step t (lyricline) is a concatenation of two different types of feature
vectors: (1) structural feature vectors v t encoded fromnine variations of SSMs in Section 3.1 and (2) linguisticfeature vectors v t encoded in Section 3.2. Formally, theconditional probability P (Ys |Xs ) is calculated by using asoftmax function:s ,Ys ))P (Ys |Xs ) = P exp(Score(X.
00 exp(Score(Xs ,Y ))
s

Ys

(1)

The Score() is defined as
Score(Xs , Ys ) =X

BN(ht [yt ]),(2)twhere ht [yt ] is the output of the Bi-LSTM for each time
step t and BN() denotes batch normalization [50]. In themodel training step, we use a binary cross-entropy loss.
354

yt+1
1:chorus

ht-1[yt-1]

ht[yt]

ht+1[yt+1]

LSTM
LSTM

LSTM
LSTM

…

…

ut

Averaging

CNN

loving youisSub-matrices

LSTM

…

LSTM
LSTM

Concatenation

vt

…

LSTM

yt
1:chorus

LSTM

…

yt-1
0:not-chorus

LSTM

Table 1. Frequent word tri-grams in chorus and non-chorus
sections. An apostrophe is regarded as a word.

…

LSTM

Pn − Pc
0.04%
0.03%
0.03%
0.02%
0.02%
0.01%
0.01%
0.01%
0.01%
0.01%
0.01%
0.01%

LSTM

Tri-gram
there’s
I’ve
’s a
I’d
but I’’s notwhat’sna na nayeah yeah yeah’ve been’t takedidn’t

Bi-LSTM Layer LabelPc − Pn
0.12%
0.11%
0.05%
0.05%
0.04%
0.04%
0.04%
0.04%
0.03%
0.03%
0.03%
0.03%

Feature Vector

Tri-gramI’m
don’t
oh oh oh
I’ll
we’re
you’re’ll beI don’
Let’s
you got ta
I can’
can’t

my dream tonightLine xt

Figure 3. Neural-network-based sequence labeling model
for chorus-section detection.
4. EXPERIMENT
Inspired by audio-based chorus-section detection [39], we
evaluated the proposed method by using the F-measure (F )that is a harmonic mean of precision (P ) and recall (R),F = (2 · R · P )/(R + P ), wherelines in correctly detected chorus sections
P = # of lyric.
# of lyric lines in detected chorus sections

lyric lines in correctly detected chorus sections
R = ##ofoflyric
lines in correct (annotated) chorus sections .

We also used the pair-wise F-measure (p-F ), normalized
conditional entropy F-measure (n-F ) and V-measure (V )that are provided by the Python module mir_eval andcommonly used to evaluate computational music structure
analysis [51].
4.1 Methods Compared
To confirm the effectiveness of our Bi-LSTM method basedon the Bi-LSTM model that can learn dependencies between adjacent lyric lines, we compared its performancewith that of with two baseline methods:(1) Heuristic: We implemented the heuristic that “if lines
at the end of the lyrics are repeated with small modifications, all those repeated lines are chorus sections” by thefollowing procedure: (i) From the SSM that is the averageof the nine SSMs, we extracted diagonals whose cells hadvalues higher than a threshold λ, which was tuned on adevelopment set to be λ = 0.62. (ii) From the extracted diagonals, we selected the shortest diagonal among diagonals
placed at the bottom of the SSM (e.g., the diagonal startingat the cell SSM[29; 1] in Figure 1). (iii) Successive lines
corresponding to the rows where the selected diagonal was
located (e.g., lyric lines 29–32 in Figure 1) were assigned
the label chorus. (iv) Other successive lines that weresimilar to the chorus lines (e.g., lyric lines 1–4 and 13–16
in Figure 1) were also assigned chorus labels.
(2) Multi-Layer Perceptron (MLP): Like the Bi-LSTM
method, but with the Bi-LSTM model replaced by a standard MLP model. This method ignores transitions between
adjacent lyric lines and predicts yt from xt only.

Proceedings of the 21st ISMIR Conference, Montréal, Canada, October 11-16, 2020

We chose the number of kernels for the first and second
CNNs to be 200 and 400, respectively. We used w = 3
for the window size. In the MLP and Bi-LSTM methods,
we chose the dimension of the hidden state to be 600. The
word2vec [42] and context2vec [43] were pre-trained on
lyrics and were not updated in the model training step of our
method. The dimension of their output vectors was 300. We
used AdamW for parameter optimization [52]. The initial
learning rate was 0.001 with an exponential decay. We useda mini-batch size of 64. Training was run for 100 epochs,
and the model used for testing was the one that achieved
the best F-measure on the development set.
4.2 Dataset
To train our computational model that predicts whether thelabel of each lyric line is chorus or not-chorus, we
needed a large amount of lyrics data with line-level chorussection annotations like those illustrated in Figure 1. Since
there was no dataset for this, we generated a large amountof such lyrics data by the following procedure:(1) We prepared 100,772 pairs of musical audio signals
and their corresponding manually time-aligned (temporally
synchronized) lyrics 3 . To avoid unreliable lyrics, we confirmed that all lyrics had more than eight lines and less than
120 lines.
(2) We detected chorus sections of every song automaticallyby using its audio signals. In our experiments, we used theRefraiD method [39] to obtain the start and end times ofeach chorus section, but other methods could also be used.
(3) If the start time of a lyric line was within any chorus section detected in audio signals, that line was labeledchorus; otherwise, it was labeled not-chorus.
Of course, not all generated annotations were correct, but
by using over 100,000 training data, the model could be robustly trained without being influenced by errors or outliers.
The generated training data consisted of 9,313 English and91,459 Japanese songs, and we called them EN_auto and
JA_auto, respectively. 4
Furthermore, we manually annotated three sets of lyrics
data with more reliable line-level chorus-section annotations for three different purposes:(a) For training comparison: We annotated 1,103
Japanese lyrics and called them JA_man 5 . By comparing
the performance of the model trained on JA_auto with thatof the model trained on JA_man, we could confirm that ourgenerated data is reliable enough for training purposes.
(b) For tuning model parameters: We annotated the
lyrics of 21 English and 79 Japanese songs from RWCMDB-P-2001 and called them EN_RWC and JA_RWC,
3 In our experiments, English and Japanese lyrics text as well as the start
time of every lyric line were provided by a lyrics distribution company.
Automatic lyrics-to-audio synchronization [7–17] could also be used toestimate such start times.
4 The main genres are Rock (33%), Pop (25%) and Alternative (12%)
for EN_auto, and J-Pop (53%), Rock (20%) and Anime (9%) for JA_auto.
5 To investigate the accuracy of the automatic annotation method weused for generating EN_auto and JA_auto, we applied the same method tothe songs (audio signals and corresponding manually time-aligned lyrics)
in JA_man. The accuracy of the generated annotations was F = 68.0%,
thus the automatic annotation method seems to work decently well.

MethodHeuristic
MLP
Bi-LSTM

Training data / Testing data
EN_auto / EN_test
JA_auto / JA_test
F
p-F n-FV
F
p-F n-FV
57.8 73.8 43.0 35.8 57.1 73.2 43.6 36.3
74.2 76.8 47.7 43.0 80.6 82.8 62.6 59.1
78.1 77.7 50.8 47.3 83.4 83.5 64.9 61.4

Table 2. Experimental result: Comparison of different
methods (the unit is %).

FeaturesimalllingallbothTraining data / Testing data
EN_auto / EN_test
JA_auto / JA_test
F
p-F n-FV
F
p-F n-FV
77.9 76.1 48.6 45.5 81.2 82.7 63.6 59.6
57.4 59.9 16.5
6.9 55.2 61.8 22.1 16.7
78.1 77.7 50.8 47.3 83.4 83.5 64.9 61.4

Table 3. Experimental result: Importance of using both
structural and linguistic features.
respectively. These were used to tune model parameters.
(c) For testing: We annotated the lyrics of 118 other English songs and 128 other Japanese songs and called them
EN_test and JA_test, respectively 6 . These were used totest the chorus-section detection methods.
4.3 Comparison of Different Methods
Table 2 summarizes the evaluated performances of Heuristic, MLP and the proposed Bi-LSTM. We found that MLP
and Bi-LSTM performed better than Heuristic. This indicates that methods based on supervised learning are better
than a rule-based method. We also found that Bi-LSTM
was better than MLP and thus confirmed the importanceof learning dependencies between adjacent lines. Since weconcluded from these results that the proposed Bi-LSTM isthe best for the chorus-section detection task, in the subsequent experiments reported here we used only Bi-LSTM.
4.4 Importance of Using Both Structural and
Linguistic Features
To investigate the effectiveness of structural and linguisticfeatures, we compared their use individually and in combination. Table 3 summarizes the results. We found thatthe model with only the structural features simall greatly
outperformed the model with only the linguistic features
lingall . Using both kinds of features further improved theperformance. This not only confirms the importance of using SSMs, as had been shown for the audio-based detectionof chorus sections, but also confirms that the additional use
of linguistic features is helpful for detecting chorus sections,which has not been shown before.
4.5 Reliability of Generated Annotations
As stated in Section 4.2, we used JA_man for the proposedtraining comparison. Table 4 clearly shows that the modeltrained using JA_auto, automatically generated data theamount of which can be large, outperformed the modeltrained using JA_man, manually annotated data, the amount
6 The chorus and not-chorus labels were annotated only on the
lyrics. No audio signal is available for these test data.

355

Proceedings of the 21st ISMIR Conference, Montréal, Canada, October 11-16, 2020

Training data
JA_auto (91,459 songs)
JA_man (1,103 songs)

F
83.4
80.3

p-F
83.5
77.3

n-F
64.9
53.3

perform better. This could have an impact on low-resource
languages because large-scale training data can be createdby mixing other available language resources.

V
61.4
50.4

Table 4. Experimental result: Reliability of automaticallygenerated annotations.
Training data
EN_auto (9,313 songs)
JA_auto (91,459 songs)
EJ_auto (100,772 songs)

Testing data
EN_testEN_testEN_test

F
77.9
80.3
81.0

p-F
76.1
80.6
82.3

n-F
48.6
58.1
60.7

V
45.5
54.4
57.4

Table 5. Experimental result: Can the Japanese model
detect English chorus sections?
of which is usually very limited because of the laborious
manual effort its creation requires. The results also confirm
that even if annotations generated automatically are not
perfect they are reliable enough for training the model.
4.6 Training Data Size and Language Dependency
Tables 2 and 3 also show that the performances for English
lyrics were worse than those for Japanese lyrics. Since theamount of Japanese training data was about 10 times thanthat of English training data, we think that the amount oftraining data greatly affects the performance of the proposedmodel. We are thus interested in answering the question“Can a model trained on a large amount of Japanese datadetect English chorus sections?” In fact, although linguisticfeatures are language dependent and the process of computing SSMs is also language dependent, structural features
based on the resulting SSMs can be language independent
because our SSMs simply represent patterns of repeating
lyric lines, which could be universal in music.
As shown in the upper half of Table 5, which shows
results obtained without using linguistic features, we foundthat the structural-feature-based model trained on Japanesedata JA_auto succeeded in detecting English chorus sections in EN_test and its performance was better than thatof the model trained on the smaller dataset EN_auto. This
result indicates that the SSM-based model trained on a large
amount of data can detect chorus sections regardless of thelanguage of the test set. Moreover, this result is further
evidence that Japanese and English SSMs (i.e., patterns ofrepeating lyric lines) have similar structures.
Obviously, the above result raises another question:“Can a model trained on both EN_auto and JA_auto perform better than one trained on only EN_auto or JA_auto?”
To answer this question, we created training data EJ_auto
by including both EN_auto and JA_auto and constructed
yet another structural-feature-based model with EJ_auto.
As shown in the lower half of Table 5, we found that themodel trained on both languages performed better than themodel trained on only one.
These results confirm that chorus sections can be detected by a model trained on data in another language, thatpatterns of repeating lyric lines are language-independentand that mixing different language data allows the model tolearn the general structure of chorus sections and thereby
356

5. RELATED WORK
Previous work in the MIR community has addressed musical structure analysis and chorus-section detection based on
repeated patterns in musical audio signals [23–41]. Studies
in the chorus-section detection for audio signals typicallyused SSMs to capture repeated structures, and we share thismotivation. Our approach differs from those audio-based
approaches in that it exploits multiple lyrics-based SSMs
and linguistic features within chorus sections.
On the other hand, recent work in the NLP community
has tackled lyrics segmentation and summarization tasks byexploiting SSMs. Fell et al. and Watanabe et al. proposeda neural network model and logistic regression model forsegmenting paragraphs (sections) without labeling themby using SSMs as features [18, 19]. Those tasks, however,are essentially different from detecting all chorus sections
that are the most representative sections in lyrics text. Addressing a task similar to chorus-section detection, Fellet al. [53] proposed a method of summarizing lyrics bycombining general document summarization methods withaudio thumbnailing methods. They focus on extracting individual informative lines as a summary from lyrics text,
not redundant repeated lines. On the other hand, the focus
of our paper is to detect chorus sections whose successivelines are often repeated in lyrics text.

6. CONCLUSION
This paper has addressed the novel task of detecting chorus
sections in English and Japanese lyrics. We proposed aneural-network-based sequence labeling model that learns
structural (i.e., phrase-repetition) and linguistic features todetect lyric lines of chorus sections. We also generated over
100,000 training data with chorus-section annotations. No
previous work has ever conducted chorus-section detection
for text-only lyrics with this much data.
The contributions of this study are summarized as follows: (1) We designed a variety of features to capture structural and linguistic properties of chorus sections. (2) We
proposed a sequence labeling model that can detect chorus
sections in lyrics. (3) We showed how to generate a large
training dataset of lyrics with chorus-section annotations.
(4) We demonstrated that our Bi-LSTM-based method outperforms alternative baseline methods. (5) We thoroughly
investigated this detection task and the nature of chorus
sections of lyrics from different perspectives such as theimportance of features, the amount of training data, andlanguage dependency.
We plan to extend our method to detect other sections,such as verse and bridge sections. Future work will also
develop MIR applications using our method, such as those
discussed in Section 1.

Proceedings of the 21st ISMIR Conference, Montréal, Canada, October 11-16, 2020

7. ACKNOWLEDGMENTS
The authors appreciate SyncPower Corporation for providing lyrics data. This work was supported in part by JST
ACCEL Grant Number JPMJAC1602, Japan.
8. REFERENCES[1] K. Watanabe and M. Goto, “Query-by-Blending: A
music exploration system blending latent vector representations of lyric word, song audio, and artist,” inProceedings of the 20th International Society for Music
Information Retrieval Conference (ISMIR 2019), 2019,
pp. 144–151.
[2] K. Tsukuda, K. Ishida, and M. Goto, “Lyric Jumper: A
lyrics-based music exploratory web service by modelinglyrics generative process,” in Proceedings of the 18th
International Society for Music Information Retrieval
Conference (ISMIR 2017), 2017, pp. 544–551.
[3] S. Sasaki, K. Yoshii, T. Nakano, M. Goto, and S. Morishima, “LyricsRadar: A lyrics retrieval system basedon latent topics of lyrics,” in Proceedings of the 15th
International Society for Music Information Retrieval
Conference (ISMIR 2014), 2014, pp. 585–590.
[4] A. Tsaptsinos, “Lyrics-based music genre classification
using a hierarchical attention network,” in Proceedingsof the 18th International Society for Music Information
Retrieval Conference (ISMIR 2017), 2017, pp. 694–701.
[5] R. Mayer and A. Rauber, “Music genre classification byensembles of audio and lyrics features,” in Proceedingsof the 12th International Society for Music Information
Retrieval Conference (ISMIR 2011), 2011, pp. 675–680.
[6] R. Mayer, R. Neumayer, and A. Rauber, “Rhyme andstyle features for musical genre classification by song
lyrics,” in Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR 2008),
2008, pp. 337–342.
[7] B. Sharma, C. Gupta, H. Li, and Y. Wang, “Automatic
lyrics-to-audio alignment on polyphonic music using
singing-adapted acoustic models,” in Proceedings ofthe 2019 IEEE International Conference on Acoustics,
Speech and Signal Processing (IEEE ICASSP 2019),
2019, pp. 396–400.
[8] D. Stoller, S. Durand, and S. Ewert, “End-to-end lyricsalignment for polyphonic music using an audio-tocharacter recognition model,” in Proceedings of the2019 IEEE International Conference on Acoustics,
Speech and Signal Processing (IEEE ICASSP 2019),
2019, pp. 181–185.
[9] C. Gupta, E. Yilmaz, and H. Li, “Acoustic modeling forautomatic lyrics-to-audio alignment,” in Proceedings ofthe 20th Annual Conference of the International SpeechCommunication Association (Interspeech 2019), 2019,
pp. 2040–2044.

[10] C. Gupta, R. Tong, H. Li, and Y. Wang, “Semisupervised lyrics and solo-singing alignment,” in Proceedings of the 19th International Society for Music
Information Retrieval Conference (ISMIR 2018), 2018,
pp. 600–607.
[11] S. Chang and K. Lee, “Lyrics-to-audio alignment by
unsupervised discovery of repetitive patterns in vowel
acoustics,” IEEE Access, vol. 5, pp. 16 635–16 648,
2017.
[12] S. W. Lee and J. Scott, “Word level lyrics-audio synchronization using separated vocals,” in Proceedings ofthe 2017 IEEE International Conference on Acoustics,
Speech and Signal Processing (IEEE ICASSP 2017),
2017, pp. 646–650.
[13] Y. Chien, H. Wang, and S. Jeng, “Alignment of lyrics
with accompanied singing audio based on acousticphonetic vowel likelihood modeling,” IEEE/ACM Transactions on Audio, Speech, and Language Processing,vol. 24, no. 11, pp. 1998–2008, 2016.
[14] G. B. Dzhambazov and X. Serra, “Modeling ofphoneme durations for alignment between polyphonicaudio and lyrics,” in Proceedings of the 12th Sound andMusic Computing Conference (SMC 2015), 2015.
[15] M. Mauch, H. Fujihara, and M. Goto, “Integrating additional chord information into HMM-based lyrics-toaudio alignment,” IEEE Transactions on Audio, Speech,
and Language Processing, vol. 20, no. 1, pp. 200–210,
2012.
[16] H. Fujihara, M. Goto, J. Ogata, and H. G. Okuno,“LyricSynchronizer: Automatic synchronization system
between musical audio signals and lyrics,” IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 6,
pp. 1252–1261, 2011.
[17] M. Kan, Y. Wang, D. Iskandar, T. L. Nwe, and
A. Shenoy, “LyricAlly: Automatic synchronization of
textual lyrics to acoustic music signals,” IEEE Trans.
Speech Audio Process., vol. 16, no. 2, pp. 338–349,
2008.
[18] M. Fell, Y. Nechaev, E. Cabrio, and F. Gandon, “Lyrics
segmentation: Textual macrostructure detection using
convolutions,” in Proceedings of the 27th International
Conference on Computational Linguistics (COLING
2018), 2018, pp. 2044–2054.
[19] K. Watanabe, Y. Matsubayashi, N. Orita, N. Okazaki,
K. Inui, S. Fukayama, T. Nakano, J. B. L. Smith, and
M. Goto, “Modeling discourse segments in lyrics using repeated patterns,” in Proceedings of the 26th International Conference on Computational Linguistics(COLING 2016), 2016, pp. 1959–1969.
[20] A. Baratè, L. A. Ludovico, and E. Santucci, “Asemantics-driven approach to lyrics segmentation,” in357

Proceedings of the 21st ISMIR Conference, Montréal, Canada, October 11-16, 2020

Proceedings of the 8th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP 2013), 2013, pp. 73–79.
[21] J. P. G. Mahedero, A. Martinez, P. Cano, M. Koppenberger, and F. Gouyon, “Natural language processing oflyrics,” in Proceedings of the 13th ACM International
Conference on Multimedia (ACM Multimedia), 2005,
pp. 475–478.
[22] K. Watanabe, Y. Matsubayashi, K. Inui, S. Fukayama,
T. Nakano, and M. Goto, “Modeling storylines in lyrics,”
IEICE Transactions on Information and Systems, vol.
101-D, no. 4, pp. 1167–1179, 2018.
[23] G. Shibata, R. Nishikimi, E. Nakamura, and K. Yoshii,“Statistical music structure analysis based on ahomogeneity-, repetitiveness-, and regularity-aware hierarchical hidden semi-markov model,” in Proceedingsof the 20th International Society for Music Information
Retrieval Conference (ISMIR 2019), 2019, pp. 268–275.
[24] A. Maezawa, “Music boundary detection based on a
hybrid deep model of novelty, homogeneity, repetitionand duration,” in Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal
Processing (IEEE ICASSP 2019), 2019, pp. 206–210.
[25] G. Sargent, F. Bimbot, and E. Vincent, “Estimating
the structural segmentation of popular music pieces
under regularity constraints,” IEEE/ACM Transactions
on Audio, Speech, and Language Processing, vol. 25,
no. 2, pp. 344–358, 2017.
[26] T. Cheng, J. B. L. Smith, and M. Goto, “Music structure
boundary detection and labelling by a deconvolution ofpath-enhanced self-similarity matrix,” in Proceedings ofthe 2018 IEEE International Conference on Acoustics,
Speech and Signal Processing (IEEE ICASSP 2018),
2018, pp. 106–110.

[31] H. Grohganz, M. Clausen, N. Jiang, and M. Müller,“Converting path structures into block structures using
eigenvalue decompositions of self-similarity matrices,”in Proceedings of the 14th International Society forMusic Information Retrieval Conference (ISMIR 2013),
2013, pp. 209–214.
[32] O. Nieto and T. Jehan, “Convex non-negative matrix
factorization for automatic music structure identification,” in Proceedings of the 2013 IEEE International
Conference on Acoustics, Speech and Signal Processing(IEEE ICASSP 2013), 2013, pp. 236–240.
[33] F. Kaiser and G. Peeters, “A simple fusion method ofstate and sequence segmentation for music structure
discovery,” in Proceedings of the 14th International
Society for Music Information Retrieval Conference(ISMIR 2013), 2013, pp. 257–262.
[34] J. Serrà, M. Müller, P. Grosche, and J. L. Arcos, “Unsupervised detection of music boundaries by time series
structure features,” in Proceedings of the 26th AAAI
Conference on Artificial Intelligence, 2012.
[35] M. Müller, P. Grosche, and N. Jiang, “A segment-based
fitness measure for capturing repetitive structures of music recordings,” in Proceedings of the 12th International
Society for Music Information Retrieval Conference (ISMIR 2011), 2011, pp. 615–620.
[36] J. Paulus, M. Müller, and A. Klapuri, “State of theart report: Audio-based music structure analysis,” inProceedings of the 11th International Society for Music
Information Retrieval Conference (ISMIR 2010), 2010,
pp. 625–636.
[37] J. Paulus and A. Klapuri, “Music structure analysis
using a probabilistic fitness measure and a greedy searchalgorithm,” IEEE Transactions on Audio, Speech, andLanguage Processing, vol. 17, no. 6, pp. 1159–1170,
2009.

[27] J. B. L. Smith and M. Goto, “Using priors to improve
estimates of music structure,” in Proceedings of the 17th
International Society for Music Information Retrieval
Conference (ISMIR 2016), 2016, pp. 554–560.

[38] M. Müller and S. Ewert, “Joint structure analysis withapplications to music annotation and synchronization,”in Proceedings of the 9th International Conference onMusic Information Retrieval (ISMIR 2008), 2008, pp.
389–394.

[28] T. Grill and J. Schlüter, “Music boundary detection using neural networks on combined features and two-levelannotations,” in Proceedings of the 16th International
Society for Music Information Retrieval Conference (ISMIR 2015), 2015, pp. 531–537.

[39] M. Goto, “A chorus section detection method for musical audio signals and its application to a music listening
station,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 14, no. 5, pp. 1783–1794, 2006.

[29] B. McFee and D. Ellis, “Analyzing song structure with
spectral clustering,” in Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014), 2014, pp. 405–410.

[40] M. Cooper and J. Foote, “Summarizing popular music
via structural similarity analysis,” in Proceedings ofthe 2003 IEEE Workshop on Applications of Signal
Processing to Audio and Acoustics (WASPAA 2003),
2003, pp. 127–130.

[30] G. Peeters and V. Bisot, “Improving music structure segmentation using lag-priors,” in Proceedings of the 15th
International Society for Music Information Retrieval
Conference (ISMIR 2014), 2014, pp. 337–342.

[41] J. Foote, “Automatic audio segmentation using a measure of audio novelty,” in Proceedings of the 2000
IEEE International Conference on Multimedia andExpo (IEEE ICME 2000), 2000, p. 452.

358

Proceedings of the 21st ISMIR Conference, Montréal, Canada, October 11-16, 2020[42] T. Mikolov, I. Sutskever, K. Chen, G. Corrado, andJ. Dean, “Distributed representations of words andphrases and their compositionality,” in Proceedings ofthe 26th Annual Conference on Neural Information Processing Systems (NeurIPS 2013), 2013, pp. 3111–3119.

[53] M. Fell, E. Cabrio, F. Gandon, and A. Giboin, “Songlyrics summarization inspired by audio thumbnailing,” in Proceedings of the International Conference
on Recent Advances in Natural Language Processing(RANLP 2019), 2019, pp. 328–337.

[43] O. Melamud, J. Goldberger, and I. Dagan, “context2vec:
Learning generic context embedding with bidirectional
LSTM,” in Proceedings of the 20th SIGNLL Conference
on Computational Natural Language Learning, 2016,
pp. 51–61.
[44] M. Goto, H. Hashiguchi, T. Nishimura, and R. Oka,“RWC Music Database: Popular, classical and jazz music databases,” in Proceedings of the 3rd International
Conference on Music Information Retrieval (ISMIR
2002), vol. 2, 2002, pp. 287–288.
[45] Y. Li and B. Liu, “A normalized Levenshtein distancemetric,” IEEE Transactions on Pattern Analysis andMachine Intelligence, vol. 29, no. 6, pp. 1091–1095,
2007.
[46] S. Bird, “NLTK: the natural language toolkit,” in Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the
Association for Computational Linguistics (ACL 2006),
2006.
[47] S. Hochreiter and J. Schmidhuber, “Long short-term
memory,” Neural computation, vol. 9, no. 8, pp. 1735–
1780, 1997.
[48] D. J. Berndt and J. Clifford, “Using dynamic time warping to find patterns in time series,” in Proceedings ofWorkshop on Knowledge Discovery in Databases, 1994,
pp. 359–370.
[49] A. Graves, A. Mohamed, and G. E. Hinton, “Speech
recognition with deep recurrent neural networks,” inProceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (IEEEICASSP 2013), 2013, pp. 6645–6649.
[50] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal
covariate shift,” in Proceedings of the 32nd International Conference on Machine Learning (ICML 2015),
vol. 37, 2015, pp. 448–456.
[51] C. Raffel, B. McFee, E. J. Humphrey, J. Salamon, O. Nieto, D. Liang, and D. P. W. Ellis, “mir_eval: A transparent implementation of common MIR metrics,” inProceedings of the 15th International Society for Music
Information Retrieval Conference (ISMIR 2014), 2014,
pp. 367–372.
[52] I. Loshchilov and F. Hutter, “Decoupled weight decay
regularization,” in Proceedings of the 7th International
Conference on Learning Representations (ICLR 2019),
2019.
359