Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland

AN ADAPTIVE KARAOKE SYSTEM
THAT PLAYS ACCOMPANIMENT PARTS OF MUSIC AUDIO SIGNALSSYNCHRONOUSLY WITH USERS’ SINGING VOICESYusuke Wada

Yoshiaki Bando
Eita Nakamura
Katsutoshi Itoyama
Kazuyoshi Yoshii
Department of Intelligence Science and Technology
Graduate School of Informatics, Kyoto University, Japan{wada,yoshiaki,enakamura,itoyama,yoshii}@ sap.ist.i.kyoto-u.ac.jp
ABSTRACTThis paper presents an adaptive karaoke system that can
extract accompaniment sounds from music audio signals
in an online manner and play those sounds synchronouslywith users’ singing voices. This system enables a user toexpressively sing an arbitrary song by dynamically changing the tempo of the user’s singing voices. A key advantage of this systems is that users can immediately enjoy
karaoke without preparing musical scores (MIDI files). To
achieve this, we use online methods of singing voice separation and audio-to-audio alignment that can be executedin parallel. More specifically, music audio signals are separated into singing voices and accompaniment sounds fromthe beginning using an online extension of robust nonnegative matrix factorization. The separated singing voices
are then aligned with a user’s singing voices using online dynamic time warping. The separated accompanimentsounds are played back according to the estimated warping
path. The quantitative and subjective experimental results
showed that although there is room for improving the computational efficiency and alignment accuracy, the system
has a great potential for offering a new singing experience
1. INTRODUCTION
Karaoke is one of the most popular ways of enjoying musicin which people can sing their favorite songs synchronouslywith musical accompaniment sounds prepared in advance.
In the current karaoke industry, musical scores (MIDI files)
are assumed to be available for generating accompanimentsounds. Professional music transcribers are therefore asked
to manually transcribe music every time new commercial
CD recordings are released. The critical issues of this approach are that music transcription is very time-consuming
and technically demanding and that the quality of accompaniment sounds generated from MIDI files is inferior tothat of original musical audio signals.
It is impractical for the conventional approach to manually transcribe a huge number of songs on the Web. Consumer generated media (CGM) has recently been become
more and more popular and many non-professional peoplec 2017 Yusuke Wada et al. This is an open-access article distributed
Copyright: ⃝under the terms of the Creative Commons Attribution 3.0 Unported License, which
permits unrestricted use, distribution, and reproduction in any medium, provided
the original author and source are credited.

Feedbacks are shown here andadjusted accompanimentsound is played

User‘s singing voice
input from a microphone

Figure 1. An example of how to use the proposed system.
A user is allowed to expressively sing a song while accompaniment sounds are played back synchronously with theuser’s singing voices. The spectrograms and F0 trajectoriesof the user’s singing voices and those of the time-stretched
original singing voices can be compared in real time. The
progress of singing voice separation is also displayed.
have composed and distributed their own original songs on
the Web. In Japan, for example, over 120 thousand songs
have been uploaded on a media sharing Web service from
July 2007 [1]. It is thus necessary to generate high-quality
accompaniment sounds from arbitrary music audio signals
without using musical scores or lyrics.
Another limitation of the current karaoke systems is thatusers need to manually set the tempo of accompanimentsounds in advance. Although this limitation can be acceptable for standard popular music with steady tempo, somekinds of music (e.g., opera, gospel, and folk songs) areusually sung in an expressive way by dynamically changing the tempo of the music.
To solve these problems, we propose an adaptive karaokesystem that can extract accompaniment sounds from music audio signals in an online manner and play those sounds
synchronously with users’ singing voices.1 Figure 1 shows
how to use the proposed system. Once a song is selected, auser is allowed to immediately start to sing the song while
listening to adaptively played-back accompaniment sounds
separated from music audio signals. If the user gradually
1 A demo video of the proposed system is available online:

http://sap.ist.i.kyoto-u.ac.jp/members/wada/smc2017/index.html

SMC2017-110

Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finlandaccelerates (decelerates) the singing, the tempo of accompaniment sounds is accelerated (decelerated) accordingly
such that those sounds are synchronized with the user’s
singing. The pitches (fundamental frequencies, F0s) of
the singing voices can be compared with those of original singing voices in real time. To use this system, all theuser has to do is to prepare only music audio signals.
This system mainly consists of three components: karaoke
generation based on singing voice separation, audio-to-audio
singing-voice alignment, and time-stretching of accompaniment sounds (Figure 2). More specifically, accompaniment sounds are separated from music audio signals using an online extension of robust nonnegative matrix factorization (RNMF) [2]. The stretch rate of the separated
accompaniment sounds is estimated using online dynamic
time warping (DTW) between a user’s and original singing
voices. Finally, the stretched version of the accompaniment sounds is played back. Since these steps are in parallel, it can therefore conceal the processing time for thesinging voice separation from the user.
The main technical contribution of this study is to tackle
real-time audio-to-audio alignment between singing voices
whose pitches, timbres, and tempos many significantly varyover time. Note that conventional studies on singing-voicealignment focus on alignment between singing voices andsymbolic information such as musical scores or lyrics. Another contribution is to apply this fundamental technique toa practical application of music performance assistance.
2. RELATED WORKThis section reviews related work on singing information
processing and automatic accompaniment.
2.1 Karaoke Systems
Tachibana et al. [3] proposed a karaoke system that generates accompaniment sounds from input music audio signals without preparing musical scores or lyrics. This system uses a voice suppression technique to generate the accompaniment sounds, whose pitches can be changed manually. Inoue et al. [4] proposed another karaoke system that
automatically adjusts the tempo of accompaniment sounds
to a user’s singing voices, assuming that musical scoresand lyrics are prepared in advance.
2.2 Automatic Music Accompaniment
There have been many studies on automatic music accompaniment [5–11]. Dannenberg [5] proposed an online algorithm based on dynamic programming for automatic accompaniment. Vercoe [6] proposed an accompaniment system that supports live performances using traditional musical instruments. Raphael [7] used a hidden Markov model(HMM) to find optimal segmentation of the musical scoreof a target musical piece. Cont [8] designed an architecturethat features two coupled audio and tempo agents based ona hybrid hidden Markov/semi-Markov framework. Nakamura et al. [9] reduced the computational complexity ofpolyphonic MIDI score following using an outer-product
HMM. Nakamura et al. [10] also proposed an efficient scorefollowing algorithm under an assumption that the prior distributions of score positions before and after repeats or

Input
User’s singing voice

Musical audio signal
Singing voice separation
AccompanimentSinging voice
Audio-to-audio alignment
Stretch rate

Accompaniment stretching
OutputAdjusted accompaniment

Figure 2. An overview of the system implementation.
skips are independent of each other. Montecchio and Cont[11] proposed a particle-filter-based method of real-time
audio-to-audio alignment between polyphonic audio signals without using musical scores.
2.3 Singing Voice Alignment
Many studies have addressed audio-to-score or audio-tolyric alignment, where singing voices are aligned with symbolic data such as musical scores or lyrics [12–15]. Gong
et al. [12] attempted audio-to-score alignment based on ahidden semi-Markov model (HSMM) using melody andlyric information. A lot of effort has been devoted to audioto-lyric alignment. Fujihara et al. [13], for example, usedsinging voice separation and phoneme alignment for synchronizing musical audio signals with their corresponding
lyrics. Iskandar et al. [14] attempted syllabic-level alignment based on dynamic programming. Wang et al. [15]combined feature extraction from singing voices rhythmic
structure analysis of musical audio signals. Dzhambazov
et al. [16] modeled a duration of each phoneme based ona duration-explicit HMM using mel-frequency cepstral coefficients (MFCCs).
2.4 Singing Voice Separation
A typical approach to singing voice separation is to estimate a time-frequency mask that separates the spectrogramof a target music audio signal into a vocal spectrogramand an accompaniment spectrogram [17–20]. Huang et al.
[17] used robust principal component analysis (RPCA) to
extract accompaniment spectrograms with low-rank structures. Deep recurrent neural networks were also used [21].
Ikemiya et al. [18] improved the separation quality by combining RPCA with F0 estimation. Rafii and Pardo [19]proposed a similarity-based method to find repetitive patterns (accompaniment sounds) in polyphonic audio signals. As another approach, Yang et al. [20] used Bayesiannon-negative matrix factorization (NMF). Very few studies
have been conducted on online singing voice separation.
3. PROPOSED SYSTEM
This section describes the graphical user interface (GUI)of the proposed system and the implementation of the system based on singing voice separation and audio-to-audioalignment between singing voices.

SMC2017-111

Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland

1

2
3

4

Figure 4. Singing voice separation based on VB-RNMF.
The matrix corresponding to an input audio spectrogram isseparated into a sparse matrix corresponding to the magnitude spectrogram of singing voices and a low-rank matrix
corresponding to the magnitude spectrogram of accompaniment sounds.

5

6

3.3 Singing Voice Separation for Music Audio Signals

7

Figure 3. A screenshot of the user interface.
3.1 User Interface
Figure 3 shows a screenshot of the user interface that provides easy-to-use functions through seven GUI elements:(1) a selector for music audio signals, (2) a display of thecurrent stretch rate, (3) a display of the progress of singing
voice separation, (4) a display of the spectrograms of theuser’s and the original singing voices, (5) a display of theF0 trajectories of the user’s and the original singing voices,(6) play and stop buttons for controlling the playback, and(7) a volume control for the accompaniment sound.
The GUI elements numbered 2, 4, and 5 provides visual feedback of the user’s singing voice and the original
singing voice. The red area (number 2 in Figure 3) indicates whether the stretch rate is matching the user’s intention. The user can refer how the original singer sings withthe spectrograms displayed in the sky blue area (number 4
in Figure 3). For example, the user can know sections inwhich the original singer users a vibrato technique. In addition, the F0 trajectories displayed in the pink area (number 5 in Figure 3) helps the user to correct the pitch of theuser’s singing voice.
3.2 Implementation Policies
To reduce the user’s wait time, we specify three requirements for the system implementation. First, users shouldbe able to enjoy karaoke immediately after starting the system. Second, singing voice separation should to be processed in real time without prior learning. Third, automatic
accompaniment should also be processed in real time.
We chose and implemented a method for each component of the system so as to satisfy these three requirements.
More specifically, singing voice separation, recording of auser’s singing voices, singing-voice alignment, and playback of time-stretched accompaniment sounds are processed
in independent threads (Figure 2).

To separate a musical audio signal specified by the user
into singing voices and accompaniment sounds, we propose an online version of variational Bayesian robust NMF(VB-RNMF) [2]. Although there are many offline methodsof singing voice separation [17–20], our system requires
real-time separation in order to conceal the processing timeof the singing voice separation from the user. Figure 4
shows how online VB-RNMF separates a mini-batch spectrogram into a sparse singing-voice spectrogram and a lowrank accompaniment spectrogram.
More specifically, an input spectrogram Y = [y1 , . . . , yT ]is approximated as the sum of a low-rank spectrogram L =[l1 , . . . , lT ] and a sparse spectrogram S = [s1 , . . . , sT ]:yt ≈ lt + st ,(1)

where L is represented by the product of K spectral basis
vectors W = [w1 , . . . , wK ] and their temporal activation
vectors H = [h1 , . . . , hT ] as follows:yt ≈ Wht + st .

(2)

The tread-off between low-rankness and sparseness is controlled in a Bayesian manner stated below.
The Kullback-Leibler (KL) divergence is used for measuring the approximation error. Since the maximization of
the Poisson likelihood (denoted by P) corresponds to theminimization of the KL divergence, the likelihood function
is given by
" #
%
#$
!
#
p(Y|W, H, S) =P yf t #wf k hkt + sf t . (3)
#
f,t

k

Since the gamma distribution (denoted by G) is a conjugate prior for the Poisson distribution, gamma priors are
put on the basis and activation matrices of the low-rankcomponents as follows:!
p(W|αwh , β wh ) =G(wf k |αwh , β wh ),(4)

SMC2017-112f,kp(H|αwh , β wh ) =!
k,tG(hkt |αwh , β wh ),(5)Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
!
"

c

#
Warping pathcalculated byonline DTW

W

%

$
&

'

(

MaxRunCount)* + , - .
W

Figure 6. Online DTW with input length W = 8, search
width c = 4, and path constraint M axRunCount = 4.
All calculated cells are framed in bold and colored sky
blue, and the optimal path is colored orange.

Figure 5. A warping path obtained by online DTW.

where αwh and β wh are represent the shape and rate parameters of the gamma distribution.
To force the sparse components to take nonnegative values, gamma priors with rate parameters given the Jeffreys
hyperpriors are put on those components as the following:p(S|αs , β s ) =!
f,tG(sf t |αs , βfs t ),p(βfs t ) ∝ (βfs t )−1 .

is calculated as follows:⎧⎪⎨di,j−1
di,j = ||x′i − yj′ || + min di−1,j⎪⎩di−1,j−1 .

(6)
(7)

where αs represents the hyperparameter of the gamma distribution that controls the sparseness. Using Eqs. (3)–(7),the expected values of W, H, and S are estimated in amini-batch style using a VB technique.
3.4 Audio-to-Audio Alignment between Singing Voices
We use an online version of dynamic time warping (DTW)[22] that estimates an optimal warping path between a user’s
singing voices and original singing voices separated frommusic audio signals (Figure 5). Since the timbres and F0sof the user’s singing voices can significantly differ fromthose of the original singing voices according to the singing
skills of the user, we focus on both MFCCs and F0s of
singing voices for calculating a cost matrix in DTW. To
estimate the F0s, saliency-based F0 estimation called subharmonic summation [23] is performed.
First MFCCs and F0s are extracted from the mini-batch
spectrogram of the separated voice X = {x1 , · · · , xW }and that of the user’s singing voice Y = {y1 , · · · , yW }.
Suppose we represent the concatenated vector of MFCCs(x)(x)
and F0 extracted from the xi and yi as x′i = {mi , fi }
(y)
(y)
′
′
′
′
and yi = {mi , fi }, X = {x1 , · · · xW } and Y ′ =′{y1′ , · · · yW
} are input to the online DTW. In this concatenation, the weight of F0 is smaller than MFCCs. This isbecause F0 would be much less stable than MFCCs when
the user has poor skills. If those mini-bathes are not silent,
the MFCCs and F0s are extracted and the cost matrix D ={di,j }(i = 1, · · · , W ; j = 1, · · · , W ) is updated according to Algorithms 1 and 2 with constraint parameters W ,c, and M axRunCount, i.e., a partial row or column of D

(8)

The variable s and t in Algorithms 1 and 2 represent thecurrent position in the feature sequences X ′ and Y ′ respectively. The online DTW calculates the optimal warping path L = {o1 , · · · , ol }, oi = (ik , jk ) (0 ≤ ik ≤
ik+1 ≤ n; 0 ≤ jk ≤ jk+1 ≤ n), using the root meansquare for ||xi − yj || incrementally, without backtraking.
(ik , jk ) means that the frame xik corresponds to the frame
yjk . Figure 6 shows an example how the cost matrix andthe warping path is calculated. Each number in Figure 6
represents the order to calculate the cost matrix. The parameter W is the length of the input mini-batch. If thewarping path reaches the W -th row or column, then thecalculation stops. If the warping path ends at (W, k)(k <W ), the next warping path starts from that point. c restricts
the calculation of the cost matrix. At most c successive elements are calculated for each calculation of the cost matrix.
M axRunCount restricts the shape of the warping path.
The warping path is incremented at most M axRunCount.
The function GetInc decides which to increment a row,column, or both of the warping path. If the row is incremented from the position (s, t) in the cost matrix, then at
most c successive elements from (s − c, t) to (s, t) are calculated. Otherwise, successive elements from (s, t − c) to(s, t) are calculated.
For the system reported here, we set the parameters as
W = 300, c = 4, M axRunCount = 3.
3.5 Time Stretching of Accompaniment Sounds
Given the warping path L = {o1 , · · · , ol }, the system calculates a series of stretch rates R = {r1 , · · · , rW } for eachframe of a mini-batch. The stretch rate of the i-th frame,ri , is given by

SMC2017-113

ri =the amount of i in {i1 , · · · , il }.
the amount of i in {j1 , · · · , jl }(9)Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland
Algorithm 1 The online DTW algorithms ← 1, t ← 1, path ← (s, t), previous ← None
Calculate ds,t following the eq. 8
while s < W, t < W do
if GetInc(s, t) ̸= Column thens←s+1
for k = t − c + 1, · · · , t do
if k > 0 thenCalculate ds,t following the eq. 8
end ifend forend ifif GetInc(s, t) ̸= Row thent←t+1
for k = s − c + 1, · · · , s do
if k > 0 thenCalculate ds,t following the eq. 8
end ifend forend ifif GetInc(s, t) == previous thenrunCount ← runCount +1
else
runCount ← 1
end ifif GetInc(s, t) ̸= Both thenprevious ← GetInc(s, t)
end ifpath.append((s, t))
end while

Algorithm 2 The function GetInc(s, t)
if s < c thenreturn Bothend if
if runCount < MaxRunCount thenif previous == Row thenreturn Column
else
return Row
end ifend if(x, y) = arg min(D(k, l)), where k == s or l == t
if x < s thenreturn Rowelse if y < t thenreturn Column
else
return Bothend ifThen the stretch rate r for the current*W mini-batch is calcu1
lated as the mean of R as r = W
i=1 ri . r is updated oneach iteration of the online DTW. The system finally uses
a standard method of time-scale modification called phase
vocoder [24] to stretch the mini-batch of the separated accompaniment sounds by a factor of r. The phase vocoder
stretches the input sound globally by a factor of r.

Figure 7. Stretch rate RMSEs measured in the accuracy evaluation. The RMSEs represent how the estimatedstretch rates differ from the original rates.
separation actually does not work in real time, but it is sufficient to wait a short while before using the system. For
greater convenience, the performance of singing voice separation could be improved. One way to achieve this is toprocess singing voice separation on a graphics processing
unit (GPU).

4. EVALUATION

4.2 Accuracy Evaluation of Singing Voice Alignment

We conducted three experiments to evaluate the effectiveness of this proposed system. Quantitatively, we evaluatedthe efficiency of the system performance and the accuracyof real-time audio-to-audio alignment. We also conducteda subjective experiment.

To evaluate the accuracy of audio-to-audio alignment, we
randomly selected 10 pieces from the database. The singing
voices were separated from the 30-second spectrogram ofeach piece. The phase vocoder [24] was then used to stretch
the separated singing voices according to eleven kinds ofstretch rates, r = 0.5, 0.6, · · · , 1.4, 1.5. The separated
voice and the stretched version of it were input to online
DTW, and the stretch rate r′ was calculated from the estimated warping path. Then the r′ and r were compared.
The system uses the separated singing voice and the user’s
clean voice, but this evaluation uses the separated singing
voice and a stretched version of the voice to determine acorrect stretch rate.
Figure 7 shows the stretch rate root mean squared errors(RMSEs) between r and r′ . The average RMSE over 10
pieces was 0.92 and the standard deviation was 0.068. This
indicates that the performance difference of audio-to-audioalignment varied little over different songs, but the align-

4.1 Efficiency Evaluation of Singing Voice Separation
To evaluate the efficiency of singing voice separation, weused 100 pieces sampled at 44.1 kHz from the RWC popular music database [25]. Each piece was truncated to
30 seconds from the beginning. Spectrograms were thencalculated using short-term Fourier transform (STFT) withthe window size of 4096 samples and the hop size of 10 ms.
Each spectrogram was then split into 30 300-millisecond
mini-batches, which were input to online VB-RNMF.
The average processing time for a 300-millisecond minibatch was 538.93 ms, and no mini-batches were processedin less than 300 ms. This means that the singing voice

SMC2017-114

Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finlandsubject 1
subject 2
subject 3
subject 4question (1)partially yesyespartially yes
yesquestion (2)partially yes
partially yesnopartially yes

Table 1. The result of the subjective evaluation.
ment accuracy was not very high. This was because theseparated singing voices contained musical noise, and thetime stretch led to further noise, giving inaccurate MFCCs.
The number of songs used in this evaluation is rather
small. We plan further evaluation using more songs.
There are many possibilities for improving the accuracy.
First, HMM-based methods would be superior to the DTWbased method. HMM-based methods learn previous inputs
unlike DTW-based methods, and this would be useful toimprove the accuracy. Second, simultaneous estimation of
the alignment and the tempo estimation would improve theaccuracy. The result of tempo estimation could help to predict the alignment. An approach for this way of alignmentis a method using particle filters [11].
4.3 Subjective Evaluation
Each of four subjects was asked to sing a Japanese popular song after listening to the songs in advance. The songs
used for evaluation were an advertising jingle “Hitachi noKi,” a rock song “Rewrite” by Asian Kung-fu Generation,
a popular song “Shonen Jidai” by Inoue Yosui, and a popular song “Kimagure Romantic” by Ikimono Gakari. The
subjects were then asked two questions; (1) whether the
automatic accompaniment was accurate, and (2) whether
the user interface was appropriate.
The responses by the subjects are shown in Table 1. The
responses indicate, respectively, that the automatic accompaniment was partially accurate and practical, and the userinterface was useful.
The subjects also gave several opinions for the system.
First, the accompaniment sounds were low quality and it
was not obvious whether the automatic accompaniment was
accurate. We first need to evaluate quality of the singing
voice separation. An approach for this problem could beto add a mode of playing a click sound according to thecurrent tempo. Second, some of the subjects did not understand what the displayed spectrograms represented. Some
explanation should be added for further user-friendliness,or only the stretch rate and F0 trajectories should be displayed.
The number of the test sample used in this subjective
evaluation is rather small. We plan further evaluation bymore subjects.
5. CONCLUSION
This paper presented a novel adaptive karaoke system thatplays back accompaniment sounds separated from music
audio signals while adjusting the tempo of those sounds
to that of the user’s singing voices. The main componentsof the system are singing voice separation based on online
VB-RNMF and audio-to-audio alignment between singing

voices based on online DTW. This system enables a user toexpressively sing an arbitrary song by dynamically changing the tempo of the user’s singing voices. The quantitativeand subjective experimental results showed the effectiveness of the system.
We plan to improve separation and alignment of singing
voices. Using the tempo estimation result would help improvement of the audio-to-audio alignment. Automatic harmonization for users’ singing voices would be an interesting function as a smart karaoke system. Another importantresearch direction is to help users improve their singing
skills by analyzing the weak points from the history of thematching results between the user’s and original singing
voices.
Acknowledgments
This study was supported by JST OngaCREST and OngaACCEL Projects and JSPS KAKENHI Nos. 26700020,
24220006, 26280089, 16H01744, and 15K16654.
6. REFERENCES[1] M. Hamasaki et al., “Songrium: Browsing and listening environment for music content creation community,” in Proc. SMC, 2015, pp. 23–30.
[2] Y. Bando et al., “Variational bayesian multi-channel
robust nmf for human-voice enhancement with a deformable and partially-occluded microphone array,” inProc. EUSIPCO, 2016, pp. 1018–1022.
[3] H. Tachibana et al., “A real-time audio-to-audio
karaoke generation system for monaural recordings
based on singing voice suppression and key conversion
techniques,” J. IPSJ, vol. 24, no. 3, pp. 470–482, 2016.
[4] W. Inoue et al., “Adaptive karaoke system: Humansinging accompaniment based on speech recognition,”in Proc. ICMC, 1994, pp. 70–77.
[5] R. B. Dannenberg, “An on-line algorithm for real-timeaccompaniment,” in Proc. ICMC, 1984, pp. 193–198.
[6] B. Vercoe, “The synthetic performer in the context oflive performance,” in Proc. ICMC, 1984, pp. 199–200.
[7] C. Raphael, “Automatic segmentation of acoustic musical signals using hidden markov models,” J. IEEE
Trans. on PAMI, vol. 21, no. 4, pp. 360–370, 1999.
[8] A. Cont, “A coupled duration-focused architecture forrealtime music to score alignment,” J. IEEE Trans. on
PAMI, vol. 32, no. 6, pp. 974–987, 2010.
[9] E. Nakamura et al., “Outer-product hidden markov
model and polyphonic midi score following,” J. New
Music Res., vol. 43, no. 2, pp. 183–201, 2014.
[10] T. Nakamura et al., “Real-time audio-to-score alignment of music performances containing errors andarbitrary repeats and skips,” J. IEEE/ACM TASLP,
vol. 24, no. 2, pp. 329–339, 2016.
[11] N. Montecchio et al., “A unified approach to real time
audio-to-score and audio-to-audio alignment using sequential montecarlo inference techniques,” in Proc.
ICASSP, 2011.

SMC2017-115

Proceedings of the 14th Sound and Music Computing Conference, July 5-8, Espoo, Finland[12] R. Gong et al., “Real-time audio-to-score alignment ofsinging voice based on melody and lyric information,”in Proc. Interspeech, 2015.
[13] H. Fujihara et al., “LyricSynchronizer: Automatic synchronization system between musical audio signals andlyrics,” in Proc. IEEE Journal of Selected Topics in Signal Processing Conference, 2011, pp. 1252–1261.
[14] D. Iskandar et al., “Syllabic level automatic synchronization of music signals and text lyrics,” in Proc.
ACMMM, 2006, pp. 659–662.
[15] Y. Wang et al., “LyricAlly: Automatic synchronization of textual lyrics to acoustic music signals,” J. IEEETASLP, vol. 16, no. 2, pp. 338–349, 2008.
[16] G. Dzhambazov et al., “Modeling of phoneme durations for alignment between polyphonic audio andlyrics,” in Proc. SMC, 2015, pp. 281–286.
[17] P.-S. Huang et al., “Singing-voice separation frommonaural recordings using robust principal component
analysis,” in Proc. IEEE ICASSP, 2012, pp. 57–60.
[18] Y. Ikemiya et al., “Singing voice separation and vocal
F0 estimation based on mutual combination of robust
principal component analysis and subharmonic summation,” J. IEEE/ACM TASLP, vol. 24, no. 11, pp.
2084–2095, 2016.
[19] Z. Rafii et al., “Music/voice separation using the similarity matrix,” in Proc. ISMIR, 2012, pp. 583–588.
[20] P.-K. Yang et al., “Bayesian singing-voice separation,”in Proc. ISMIR, 2014, pp. 507–512.
[21] P.-S. Huang et al., “Singing-voice separation frommonaural recordings using deep recurrent neural networks,” in Proc. ISMIR, 2014, pp. 477–482.
[22] S. Dixon, “An on-line time warping algorithm for
tracking musical performances,” in Proc. the 19th IJCAI, 2005, pp. 1727–1728.
[23] D. J. Hermes, “Measurement of pitch by subharmonicsummation,” J. ASA, vol. 83, no. 1, pp. 257–264, 1988.
[24] J. Flanagan et al., “Phase vocoder,” Bell System Technical Journal, vol. 45, pp. 1493–1509, 1966.
[25] M. Goto et al., “Rwc music database: Popular, classical, and jazz music databases,” in Proc. ISMIR, 2002,
pp. 287–288.

SMC2017-116