A Chorus-Section Detecting Methodfor Musical Audio Signals
Introduction

timeChorus Section Detectiono Chorus (refrain) sections• Most representative thematic sectionsin the music structure of popular songs

Previous Work

Most repeated and memorable portions

Automatic detection is essential for building acomputational model of music understandingo Previous chorus-detection methods[Bartsch and Wakefield, 2001][Cooper and Foote, 2002][Logan and Chu, 2000]• Only extract a single segment fromseveral chorus sections• Useful in various practical applications

Detect a repeated section of a given length
as the most representative of a song

Audio thumbnail, music retrieval, etc.

Did not address the problem ofdetecting all the chorus sections in a songidentifying both ends of the chorus sections
Not able to deal with modulated repetition

RefraiD (Refrain Detecting Method)o Exhaustively detect all the chorus sections

appearing in a song
• Obtain a list of the start and end points ofevery chorus section in CD recordings

o Previous music-summarization methods[Peeters, Burthe, and Rodet, 2002][Dannenberg and Hu, 2002]

Did not identify both ends of chorus sections
Not able to deal with modulated repetition• Detect modulated chorus sections (with key change)
time

audio signals of
popular musico It is rare for repeated sections

to be exactly the samechorus sections
modulationo Basic idea

Problems(key change)• Detect without using any prior information about
spectral characteristics of chorus sections

change of change ofchange oflyrics
melody lines accompaniments modulationo Problem 1: acoustical features and similarity• Typical power spectrum and MFCC features
are liable to change considerablyo Problem 2: repetition-judgment criterion• Chorus sections are usually
the most repeated sections in popular music• Appropriate criterion of the similarity for
judging the repetition depends on the song

1. Find various groups of repeated sectionstime

2. Output the chorus-like group appearing frequently

o Problem 3: integrating repeated sections• Identify both ends byo Real-time output• List of chorus sections• Intermediate-result list of repeated sections
Usually reflect the music structure of the songex) repetition of "verse A

verse Bchorus"

examining mutual
relationships among various repeated sectionso Problem 4: detecting modulated repetition• Acoustic features generally undergo asignificant change after modulation (key change)Masataka Goto
12

1,2

"Information and Human Activity," PRESTO, JST
National Institute of Advanced Industrial Sci. and Tech. (AIST)

Overview of RefraiD
Audio signals

Solution to Problem 1

Extracting 12-dimensional chroma vectors

o Extracting 12-dimensional chroma vectors• Sum of power at frequencies of each pitch class
Capture the overall harmony (pitch-class dist.)

Calculating similarity between chroma vectors
Listing repeated sectionsSTFT power spectrum
log-scale frequency

Integrating repeated sections
130 Hz

8 kHz

Integrating repeated sections with modulation

Summation over
six octaves

Chroma vector

Selecting chorus sections
List of chorus sections

chromaHelix representationof pitch perception

List of repeated sections[Shepard, 1964]

C C# D D# E F F# G G# A A# B

o Calculating the similarity between vectors
time

Solution to Problem 2t-l
tsimilarity r(t,l)

o Listing repeated sections

time: tlag: l• Find line segments in the similarity r(t,l)

Solution to Problem 3Line segments corresponding
to repetition of the half of Co Integrating repeated sections

Line segments corresponding
to repetition of C

• Group line segments having common sections

Line segment corresponding
to repetition of ABCC

- Revise both ends of sections in each group- Redetect missing line segments

Line segment
corresponding torepetition of C

A: verse A
B: verse B
C: chorus

redetect a line segmentusing information onother line segmentsr (t,l)r (t,l)0

A B C CA B C C C C t (time) Rall (t,l)

0r (t,l)t (time)0r (t,l)t (time)t (time)0line segment

thresholdthresholdthresholdadaptive repetition
judgment criterion

Solution to Problem 4o Integrating repeated sections w/ modulation

similaritysmoothed similarity

time• Automatic threshold selection method• Shift 12 elements of 12-D chroma vectors
before modulation

based on a discriminant criterion [Otsu, 1979]
Dichotomize peak heights into two classes
Discriminant criterion measures:maximizing class separability (between-class variance)

ωi ( µi - µT )µi : class mean of heights
ωi : probability of class occurrence µT : total mean of heights
iC C# D D# E F F# G G# A A# B

badafter modulation

C C# D D# E F F# G G# A A# B

good!

C C# D D# E F F# G G# A A# B

C# D D# E F F# G G# A A# B C

modulated by 1 semitone upward

modulation of 1 semitone: 1 shift

URL of the Masataka Goto's Home Page:
http://staff.aist.go.jp/m.goto/

Selecting Chorus Sections
o Evaluate chorus possibility for each group- # of sections is many
- reliability of section is high- length of section is long(length of section)(constant)(reliability of section) log• Three assumptions
1. Chorus has an appropriate length(7.7- 40 sec)
2. When there is a long repeated section,
the end of it is likely to be the chorus section
3. When a section has half-length repeated sub-sections,
it is likely to be the chorus section

Experimental Resultso Conditions• Tested on 100 songs
from "RWC Music Database: Popular Music"
RWC-MDB-P-2001 No. 1 - 100[Goto et al., 2002]• Correct chorus sections were labeled manually
Develop a music-structure labeling editor• F-measure: harmonic mean of recall & precision rates
Judged to be correct if F-measure > 0.75
F-measure =o Results

2RPR+P

Recall (R) =total length of correctly detected chorus sections
total length of correct chorus sections

Precision (P) =total length of correctly detected chorus sections
total length of detected chorus sections• 80 songs out of 100 were correct (F-measure mean: 0.938)
Modulation detection
Use of assumptions 2 & 3# of songs (out of 100)

80747268

Summaryo Propose the RefraiD method• Regard the most repeated sections

as the chorus sections in popular music• Detect all the chorus sections
with their start and end points• Detect modulated chorus sections,which previous methods could not detect• Robust enough to deal withreal-world audio signals in real time
2003/10/20 WASPAA 2003 poster