INTERSPEECH 2016
September 8–12, 2016, San Francisco, USA

The NU-NAIST voice conversion system for
the Voice Conversion Challenge 2016
Kazuhiro Kobayashi1 , Shinnosuke Takamichi1 , Satoshi Nakamura1 , Tomoki Toda2
1
2
1

Nara Institute of Science and Technology (NAIST), Japan
Information Technology Center, Nagoya University, Japan{kazuhiro-k, shinnosuke-t, s-nakamura}@is.naist.jp, 2 tomoki@icts.nagoya-u.ac.jp

Abstract

VC is a statistical approach to developing a conversion function
using a parallel data set consisting of utterances of the source
and target speakers. As one of the most popular statistical VC
methods, a regression method using a Gaussian mixture model(GMM) was proposed [3]. To improve performance of the
GMM-based VC method, various VC methods have been proposed by implementing more sophisticated techniques, such as
Gaussian process regression [4, 5] deep neural networks [6, 7],
non-negative matrix factorization [8, 9], and so on. We havealso signiﬁcantly improved performance of the standard GMMbased VC method by incorporating a trajectory-based conversion algorithm to make it possible to consider temporal correlation in conversion [10], modeling additional features to alleviate
an over-smoothing effect of the converted speech parameters,such as global variance (GV) [10] and modulation spectrum(MS) [11], and implementing STRAIGHT [12] with mixed excitation [13]. Furthermore, a real-time conversion process has
also been successfully implemented for state-of-the-art GMMbased VC [14]. However, the speech quality of the converted
voices is still obviously degraded compared to that of the natural
voices. One of the biggest factors causing this quality degradation is the waveform generation process using a vocoder [15],which is still observed even when using high-quality vocoder
systems [12, 16, 17].

This paper presents the NU-NAIST voice conversion (VC) system for the Voice Conversion Challenge 2016 (VCC 2016) developed by a joint team of Nagoya University and Nara Instituteof Science and Technology. Statistical VC based on a Gaussian
mixture model makes it possible to convert speaker identity ofa source speaker’ voice into that of a target speaker by converting several speech parameters. However, various factors suchas parameterization errors and over-smoothing effects usuallycause speech quality degradation of the converted voice. To
address this issue, we have proposed a direct waveform modiﬁcation technique based on spectral differential ﬁltering and havesuccessfully applied it to singing voice conversion where excitation features are not necessary converted. In this paper, we
propose a method to apply this technique to a standard voice
conversion task where excitation feature conversion is needed.
The result of VCC 2016 demonstrates that the NU-NAIST VC
system developed by the proposed method yields the best conversion accuracy for speaker identity (more than 70% of thecorrect rate) and quite high naturalness score (more than 3 ofthe mean opinion score). This paper presents detail descriptions of the NU-NAIST VC system and additional results of its
performance evaluation.
Index Terms: voice conversion challenge 2016, speaker identity, segmental feature, Gaussian mixture model, STRAIGHT
analysis.

In singing VC (SVC), to avoid the quality degradation
caused by the vocoding process [15], we have proposed an intragender SVC method with direct waveform modiﬁcation basedon spectrum differential (DIFFSVC) [18] considering global
variance (GV) [19], focusing on F0 transformation is not necessary in the intra-gender SVC. The DIFFSVC framework can
avoid using the vocoder by directly ﬁltering an input singing
voice waveform with a time sequence of spectral parameter differentials estimated by a differential GMM (DIFFGMM) analytically derived from the conventional GMM used in the standard method. Moreover, to apply this DIFFSVC framework
to cross-gender DIFFSVC as well, we have proposed an F0transformation technique with direct residual signal modiﬁcation [20] based on time-scaling with waveform similarity-based
overlap-add [21] and resampling.

1. Introduction
Varieties of voice characteristics, such as voice timbre andfundamental frequency (F0 ) patterns, produced by individual
speakers are always restricted by their own physical constraint
due to the speech production mechanism. This constraint is
helpful for making it possible to produce a speech signal capable of simultaneously conveying not only linguistic information but also non-linguistic information such as speaker identity. However, it also causes various barriers in speech communication; e.g., severe vocal disorders are easily caused evenif speech organs are partially damaged; and we hesitate to talkabout something private using a cell phone if we are surroundedby others. If the individual speakers freely produced various voice characteristics over their own physical constraints, itwould break down these barriers and open up an entirely new
speech communication style.
Voice conversion (VC) is a potential technique to makeit possible for us to produce speech sounds beyond our own
physical constraints [1]. VC research was originally started toachieve speaker conversion to make it possible to transform thevoice identity of a source speaker into that of a target speaker
while preserving the linguistic content [2]. A mainstream of

Copyright © 2016 ISCA

In this paper, we develop a new VC system for speaker conversion based on the direct waveform modiﬁcation technique,which was submitted to the Voice Conversion Challenge 2016
(VCC 2016) [22] from our joint team of Nagoya University andNara Institute of Science and Technology (NAIST) as the NUNAIST VC system (called “new NAIST VC system”). The following techniques are newly implemented for our GMM-based
VC system: 1) voice conversion with direct waveform modiﬁcation with spectral differential (DIFFVC), 2) speech parameter trajectory smoothing in the GMM training, 3) post-ﬁltering
process based on MS for DIFFVC, and 4) excitation conver-1667

http://dx.doi.org/10.21437/Interspeech.2016-970

3. The NU-NAIST VC system for VCC 2016sion (EC) using STRAIGHT as preprocessing of spectral conversion. The results of the VCC 2016 have demonstrated that
the NU-NAIST VC system (system “J”) achieved the best conversion accuracy on speaker identity and high naturalness (morethan 3 on the mean opinion score scale). In this paper, we
also conduct subjective evaluations, demonstrating that the NUNAIST VC system achieves high speech quality and conversion
accuracy comparable to our conventional GMM-based VC system.

In the conventional VC, acoustic features such as spectral features and aperiodic components of a source speaker are converted into those of a target speaker based on previously trained
GMMs. F0 is transformed to compensate for the difference inpitch between the source and target speakers based on frameby-frame linear conversion. Finally, the converted voice is generated by synthesizing these converted acoustic features using a
vocoder.

In this paper, we proposed the following techniques: 1) DIFFVC, 2) GMM training with smoothed speech parameter trajectory, 3) post-ﬁltering process based on modulation spectrum(MS) for DIFFVC, and 4) excitation conversion with F0 andaperiodic components transformations using a vocoder. Figure
1 indicates the conversion ﬂow of the NU-NAIST VC system
for the VCC 2016. The NU-NAIST VC system performs excitation and spectral conversion. During excitation conversion,
F0 values and aperiodic components extracted from a source
voice are transformed within an analysis/synthesis frameworkusing a vocoder. During spectral conversion, spectral featuresof the source voice are converted into spectral feature differentials based on the DIFFGMM. Next, MS-based post-ﬁltering
is applied to the spectral feature differential. Finally, the converted speech waveform is generated by directly ﬁltering theanalysis-synthesized speech waveform generated during the excitation conversion step using the post-ﬁltered spectral feature
differentials.

2.1. Acoustic feature mapping based on GMM

3.1. DIFFVC based on DIFFGMM

Acoustic feature mapping based on GMM consists of a training
process and a conversion process.
In the training process, a joint probability density functionof acoustic features of the source and target speaker’ voices
are modeled with a GMM using a parallel data set. As theacoustic features of the source and target speakers, we employ 2D-dimensional joint static and dynamic feature vectors  of the source and Y t = [y ofX t = [xt , Δxt ]t , Δy t ]the target consisting of D-dimensional static feature vectors xt
and y t and their dynamic feature vectors Δxt and Δy t at framet, respectively, where  denotes the transposition of the vector.
Their joint probability density modeled by the GMM is givenbyAs part of the modelling step, the DIFFGMM is analyticallyderived from the traditional GMM (in Eq. (3)). Let D t = ddenote the static and dynamic differential featuret , Δdt
vector, where dt = y t −xt , the DIFFGMM is derived by transforming model parameters in the same manner as DIFFSVC[18] as follows:

2. VC based on GMM

P (X t , D t |λ) 
 (XX) (XD) MΣmΣ
X t μ(X)
m
. (3)
=
αm N
;
, m
(DX)
(DD)
D t μ(D)
Σm Σm
m
m=1

During the conversion step, a time sequence of the Ddimensional converted spectral feature differentials, d̂, is determined using MLE of the speech parameter trajectory using
the DIFFGMM [18]. Then, the converted speech waveform isgenerated by directly ﬁltering an input speech waveform with atime-variant synthesis ﬁlter designed from the spectral feature
differential sequence. This ﬁltering process modiﬁes a spectralenvelope sequence while basically preserving the natural excitation signals of the input speech waveform.

P (X t , Y t |λ) 
 (XX) (XY ) 
M
(X)

Σm
X t μm
Σ
, (1)
=
αm N
; (Y ) , m(Y X)(Y Y )
Y t μmΣm Σ mm=1where N (·; μ, Σ) denotes the normal distribution with a mean
vector μ and a covariance matrix Σ. The mixture component
index is m. The total number of mixture components is M . λis a GMM parameter set consisting of the mixture-component
weight αm , the mean vector μm , and the covariance matrix Σmof the m-th mixture component. The GMM is trained using
joint vectors of X t and Y t in the parallel data set, which areautomatically aligned to each other by dynamic time warping.
In the conversion process, the acoustic features of thesource speaker are converted into that of the target speaker using maximum likelihood estimation (MLE) of speech parameter
trajectories using the GMM and GV [10].

3.2. Speech parameter trajectory smoothing
Modulation Spectrum (MS) [11] is deﬁned as the log-scaled
power spectrum of the parameter sequence; i.e., temporal ﬂuctuation of the parameter sequence is decomposed into individual modulation frequency components and their power values
are represented as the MS. The MS, s (y), of the parameter sequence y is deﬁned as:s (y) =

2.2. F0 transformations1 (y) , · · · , sd (y) , · · · , sD (y) , (4)sd (y) = [sd,0 (y) , · · · , sd,f (y) , · · · , sd,Ds −1 (y)], (5)In both intra- and cross-gender conversions, F0 is transformed
frame-by-frame in order to line up pitch differences between
source and target speakers.

where 2Ds is the length of the discrete Fourier transform, andsd,f (y) is the f -th MS of the d-th dimension of the parameter sequence [y 1 (d) , · · · , y T (d)] . f is the modulation frequency index. As reported in [23, 24], the higher modulation
frequency components (more ﬂuctuating components of a temporal sequence) of spectral parameter sequences are negligible
for speech quality. By applying a low-pass ﬁlter (LPF) that removes the higher modulation frequency components (e.g., morethan 50 Hz (f > Ds /2)), we can improve training accuracyσ (y)(xt − μ(x) ) + μ(y) ,
(2)
σ (x)
where xt and ŷt are a log-scaled F0 of the source speaker andthe converted one at frame t. μ(x) and σ (x) are the mean andstandard deviation of log-scaled F0 of the source speaker andμ(y) and σ (y) are those of the target speaker.
ŷt =

1668

F0

Source voice

STRAIGHT
Analysis

Lineartransformation

Transformed F0

Aperiodicity

Aperiodicity
modification

Excitation
generation

Band aperiodicity

Convertedband aperiodicity

Synthesis filter

Converted
mel-cepstrum
differential

MSPF

GMM foraperiodic components

Converted voice(DIFFVC (EC))
F0 transformed
source voice
Synthesis filter

Spectrum envelope

Mel-cepstrum

DIFFGMM formel-cepstrum

Figure 1: Conversion process of the NU-NAIST VC system for VCC 2016.
of acoustic models as done for hidden Markov model-based
original aperiodic components at all frequency bins are shifted
speech synthesis [25]. Here, source and target speakers’ speechusing aperiodic differentials between the extracted and conparameter sequences, x and y, are LPFed, then the LPFed severted band-averaged aperiodic components. Finally, analysissynthesized speech is generated from these transformed excitaquences, x(LPF) and y (LPF) , are used to train the GMM. In
tion parameters using STRAIGHT. Note that full STRAIGHT
conversion, x(LPF) is used to generate the spectral differentials.
spectral representation is directly used in synthesis.
This excitation conversion method actually causes signiﬁ3.3. MS-based post-ﬁlter for VC with spectral differentials
cant quality degradation because original phase information is
Statistical modeling tends to deteriorate MSs of the converted
discarded. Nevertheless, we have found that this method yields
speech parameters, and keeping natural MSs is strongly effecbetter speech quality as well as better conversion accuracy thantive for improving the quality of the converted speech. An MSthe direct residual signal modiﬁcation [20].
based post-ﬁlter (MSPF) [11], which is applied after speechparameter conversion in conventional GMM-based VC, mod4. Experimental evaluation
iﬁes a converted speech parameter sequence so that the sequence has the target speaker’s natural MS. Here, we propose
In this section, we show results of the VCC 2016 to demonan MS-based post-ﬁltering process that modiﬁes spectral differstrate performance of the NU-NAIST VC system. Moreover,ˆ such that the ﬁnally synthesized speech has the target
we compare the following three systems:
entials, d,speaker’s natural MS.
• DIFFVC (EC): The NU-NAIST VC system submitted to
In training, we calculate MS statistics for target speaker’s
the VCC 2016,
natural and converted speech parameters from the training data,•
VC:
Our conventional VC system [13],ˆ (LPF) ]. Here, let μ(y) and μ(ỹ) be the mean ofy and ỹ = [d+xd,fd,f• DIFFVC: The NU-NAIST VC system w/o excitation(y)(ỹ)
sd,f (y) and sd,f (ỹ), and let σd,f and σd,f be their variance.
conversion.
The dˆ is generated by converting x(LPF) .
ˆ
4.1. Experimental conditions
In conversion, x(LPF) is ﬁrst added to the generated d.
Then, the MS, sd,f (ỹ) is converted as follows:
We evaluated speech quality and speaker identity of the consd,f (ỹ) =

(y)

σd,f

(ỹ)

σd,f


(ỹ)
(y)
sd,f (ỹ) − μd,f + μd,f .

verted voices to compare performance of the different VC systems in both intra-gender and cross-gender conversion tasks.
We used the English speech database used in the VCC 2016.
The number of source speakers was 5 including 3 females and2 males, and that of the target speakers was 5 including 2 females and 3 males who were different from the source female
and male speakers. The number of sentences uttered by eachspeaker was 216. The sampling frequency was set to 16 kHz.
STRAIGHT [12] was used to extract spectral envelopes,
which was parameterized into the 1-24th mel-cepstral coefﬁcients as the spectral feature. The frame shift was 5 ms. The
mel log spectrum approximation (MLSA) ﬁlter [27] was usedas the synthesis ﬁlter. As the source excitation features, we used
F0 and aperiodic components extracted with STRAIGHT [26].
The aperiodic components were averaged over ﬁve frequency
bands, i.e., 0-1, 1-2, 2-4, 4-6, and 6-8 kHz, to be modeled with
the GMM.
We used 162 sentences for training and the remaining 54
sentences were used for evaluation. The speaker-dependent
GMMs were separately trained for all combinations of source
and target speaker pairs. The number of mixture components
for the mel-cepstral coefﬁcients was 128 and for the aperiodic
components was 64.
Two preference tests were conducted. In the ﬁrst test,
speech quality of the converted voices was evaluated. The converted voice samples generated by two different VC systems forthe same sentences were presented to subjects in random order.
The subjects selected which sample had better speech quality.

(6)

The converted ỹ is determined using the converted MS and theoriginal phase components. The MSPFed spectral differentials,(MSPF)
can be determined by subtracting x(LPF) from thedˆconverted ỹ 1 . Note that, in this paper, we use mean-normalized
MSs and adopt a segment-level post-ﬁltering process [11].
3.4. Excitation conversion based on F0 and aperiodicity
transformations using a vocoder
Although we initially tried implementing the F0 transformation technique with direct residual signal modiﬁcation [20] forsinger conversion, we found that this technique was not effective for speaker conversion. In speaker conversion, we needto handle larger acoustic differences in excitation signals between the source and target speakers compared to singing voice
conversion. To address this issue, we implemented excitation
conversion using STRAIGHT [26] as high-quality vocoder. For
the F0 transformation, we perform the global linear transformation as described in Sect 2.2. For the aperiodic components,
band-averaged aperiodic components are extracted and converted with the GMM as in the conventional method [13]. Then,
1 Note that, because the MSPF process is non-linear to the speechparameter sequence, the sequence that x(LPF) is subtracted from theˆconverted ỹ is not equal to d.

1669

100

voices, i.e., from 4.6 to 3.0 in MOS. This quality degradation
is mainly caused by using a vocoder to perform the excitation
conversion as shown in the next section. Therefore, it is expected that the converted speech quality will be signiﬁcantly
improved by developing a better analysis/synthesis techniquethan STRAIGHT.

Preference score forconversion accuracy on
speaker identity [%]The NU-NAIST VC system

80

Target60
40
Source4.3. Results of subjective evaluation

20
Submitted systems

0

1

1.5

2

2.5

3

3.5

4

4.5

Figures 3 (a) and (b) indicate the results of the preference
test for speech quality. DIFFVC (EC) achieves equivalent
speech quality compared to VC in both intra/cross-gender conversions. On the other hand, DIFFVC achieves signiﬁcantly
higher speech quality compared to the other two methods in theintra-gender conversion. This is because DIFFVC can avoid using vocoding to generate converted speech waveforms, making
the conversion process free from various errors, such as F0 extraction errors and unvoiced/voiced decision errors. Note that
DIFFVC in cross-gender conversion condition does not resultin any signiﬁcant quality improvements as it suffers from mismatches between spectral envelope and F0 in the cross-genderconversion.
Figures 4 (a) and (b) indicate the results of the preference
test for speaker identity. Although DIFFVC (EC) has equivalent
conversion accuracy compared to VC in the intra-gender conversion, it tends to be degraded in the cross-gender conversion.
It is expected that the residual spectral envelope preserved in
the direct waveform modiﬁcation process still includes speakerdependent or gender-dependent features, and that this causes
adverse effects on conversion accuracy.
These results suggest that 1) the NU-NAIST VC systemdemonstrating the best conversion accuracy and high speech
quality in the VCC 2016 has an almost equivalent performance compared to the conventional VC system in both intragender and cross-gender conversions, and 2) the direct waveform modiﬁcation technique achieves signiﬁcantly higher converted speech quality compared to the conventional VC system
if the excitation conversion is not necessary as in the intragender conversion, and therefore, there is still large room toimprove the converted speech quality of the NU-NAIST VC
system.

5

MOS score for speech quality

Figure 2: Sound quality and conversion accuracy on speaker
identity in VCC 2016.

Preference score [%]100

DIFFVC(EC) w/ MSPF
VC w/ GVDIFFVC w/ MSPF
95% confidence interval

80
60
40
20
0

(a) Intra-genderconversion(b) Cross-genderconversion

Figure 3: AB preference test for speech quality.

Preference score [%]10080

DIFFVC(EC) w/ MSPF
VC w/ GVDIFFVC w/ MSPF
95% confidence interval

60
40
20
0

(a) Intra-genderconversion(b) Cross-genderconversion

Figure 4: XAB test for conversion accuracy on speaker identity.
In the second test, conversion accuracy in speaker identity wasevaluated. A natural voice sample of the target speaker was presented to the subjects ﬁrst as a reference. Then, the converted
voice samples generated by two different VC systems for thesame sentences were presented in random order. The subjects
selected which sample was more similar to the reference natural
voice in terms of speaker identity. The number of subjects was
10 and each listener evaluated 54 sample pairs in each evaluation. They were allowed to replay each sample pair as manytimes as necessary.

5. Conclusions
This paper describes the details of the NU-NAIST voice conversion (VC) system for the Voice Conversion Challenge 2016
(VCC 2016) developed by a joint team of Nagoya University
and Nara Institute of Science and Technology. In order to improve the quality of statistical VC based on Gaussian Mixture
Model (GMM), we applied the following techniques: 1) voice
conversion with direct waveform modiﬁcation with spectral differential (DIFFVC), 2) speech parameter trajectory smoothing,3) post-ﬁltering based on modulation spectrum for DIFFVC,
and 4) preprocessing for excitation conversion with F0 and aperiodic component transformations using high-quality vocoding.
The experimental results demonstrated that the NU-NAIST VC
system was highly ranked in the VCC 2016, its performancewas comparable to our conventional VC system, and the DIFFVC technique showed large potential to signiﬁcantly improvethe converted speech quality of the NU-NAIST VC system. In
future work, we plan to implement high quality F0 and aperiodicity transformation for the DIFFVC technique.
Acknowledgements This work was supported in part by JSPS
KAKENHI Grant Number 26280060 and Grant-in-Aid for
JSPS Research Fellow Number 16J10726.

4.2. Results of the VCC 2016
Figure 2 indicates an overall result of the VCC 2016. The NUNAIST VC system achieved quite high speech quality over 3.0
of MOS and the best conversion accuracy (about 74%) among
all submitted VC systems. In terms of the conversion accuracy,
our system achieved successful performance even though very
simple prosodic conversion was performed. However, it is observed that there is still a large gap between the converted voices
and the natural target voices. It is expected that further improvements will be yielded by implementing a conversion method of
prosodic patterns or asking the source speakers to mimic target
prosodic patterns, which would be possible in several practicalapplications. In terms of speech quality, the NU-NAIST VC
system causes serious quality degradation compared to natural1670

6. References[20] K. Kobayashi, T. Toda, and S. Nakamura, “Implementation of
f0 transformation for statistical singing voice conversion basedon direct waveform modiﬁcation,” Proc. ICASSP, pp. 5670–5674,
Mar. 2016.

[1] T. Toda, “Augmented speech production based on real-time statistical voice conversion,” Proc. GlobalSIP, pp. 755–759, Dec. 2014.
[2] M. Abe, S. Nakamura, K. Shikano, and H. Kuwabara, “Voice
conversion through vector quantization,” J. Acoust. Soc. Jpn (E),vol. 11, no. 2, pp. 71–76, 1990.

[21] W. Verhelst and M. Roelands, “An overlap-add technique based on
waveform similarity (WSOLA) for high quality time-scale modiﬁcation of speech,” Proc. ICASSP, pp. 554–557 vol.2, Apr. 1993.

[3] Y. Stylianou, O. Cappé, and E. Moulines, “Continuous probabilistic transform for voice conversion,” IEEE Trans. SAP, vol. 6,
no. 2, pp. 131–142, Mar. 1998.

[22] T. Toda, L.-H. Chen, D. Saito, F. Villavicencio, M. Wester, Z. Wu,
and J. Yamagishi, “The Voice Conversion Challenge 2016,” Proc.
INTERSPEECH, Sept. 2016.

[4] N. Pilkington, H. Zen, and M. Gales, “Gaussian process experts
for voice conversion,” Proc. INTERSPEECH, pp. 2761–2764,
Aug. 2011.

[23] S. Takamichi, T. Toda, A. W. Black, and S. Nakamura, “Parameter
generation algorithm considering modulation spectrum for HMMbased speech synthesis,” Proc. ICASSP, Apr. 2015.

[5] N. Xu, Y. Tang, J. Bao, A. Jiang, X. Liu, and Z. Yang, “Voice
conversion based on Gaussian processes by coherent and asymmetric training with limited training data,” Speech Communication, vol. 58, pp. 124–138, Mar. 2014.

[24] ——, “Modulation spectrum-constrained trajectory training algorithm for GMM-based voice conversion,” Proc. ICASSP, Apr.
2015.
[25] S. Takamichi, K. Kobayashi, K. Tanaka, T. Toda, and S. Nakamura, “The naist text-to-speech system for the blizzard challenge
2015,” Proc. Blizzard Challenge workshop, Sept. 2015.

[6] L.-H. Chen, Z.-H. Ling, L.-J. Liu, and L.-R. Dai, “Voice conversion using deep neural networks with layer-wise generative training,” IEEE/ACM Trans. ASLP, vol. 22, no. 12, pp. 1859–1872,
Dec. 2014.

[26] H. Kawahara, J. Estill, and O. Fujimura, “Aperiodicity extraction
and control using mixed mode excitation and group delay manipulation for a high quality speech analysis, modiﬁcation and systemstraight,” Proc. MAVEBA, Sept. 2001.

[7] L. Sun, S. Kang, K. Li, and H. Meng, “Voice conversion using
deep bidirectional long short-term memory based recurrent neuralnetworks,” Proc. ICASSP, pp. 4869–4873, Apr. 2015.

[27] K. Tokuda, T. Kobayashi, T. Masuko, and S. Imai, “Melgeneralized cepstral analysis – a uniﬁed approach to speech spectral estimation,” Proc. ICSLP, pp. 1043–1045, 1994.

[8] R. Takashima, T. Takiguchi, and Y. Ariki, “Exemplar-based voice
conversion using sparse representation in noisy environments,”
IEICE Trans. on Inf. and Syst., vol. E96-A, no. 10, pp. 1946–1953,
Oct. 2013.
[9] Z. Wu, T. Virtanen, E. Chng, and H. Li, “Exemplar-based sparse
representation with residual compensation for voice conversion,”
IEEE/ACM Trans. ASLP, vol. 22, no. 10, pp. 1506–1521, June
2014.
[10] T. Toda, A. W. Black, and K. Tokuda, “Voice conversion based on
maximum likelihood estimation of spectral parameter trajectory,”
IEEE Trans. ASLP, vol. 15, no. 8, pp. 2222–2235, Nov. 2007.
[11] S. Takamichi, T. Toda, A. W. Black, G. Neubig, S. Sakti, and
S. Nakamura, “Postﬁlters to modify the modulation spectrum
for statistical parametric speech synthesis,” IEEE Trans. ASLP,
vol. 24, no. 4, pp. 755–767, Jan. 2016.
[12] H. Kawahara, I. Masuda-Katsuse, and A. Cheveigné, “Restructuring speech representations using a pitch-adaptive time-frequency
smoothing and an instantaneous-frequency-based f0 extraction:
Possible role of a repetitive structure in sounds,” Speech Communication, vol. 27, no. 3-4, pp. 187–207, Apr. 1999.
[13] Y. Ohtani, T. Toda, H. Saruwatari, and K. Shikano, “Maximum
likelihood voice conversion based on GMM with STRAIGHT
mixed excitation,” Proc. INTERSPEECH, pp. 2266–2269, Sept.
2006.
[14] T. Toda, T. Muramatsu, and H. Banno, “Implementation of computationally efﬁcient real-time voice conversion,” Proc. INTERSPEECH, Sept. 2012.
[15] H. Dudley, “Remaking speech,” JASA, vol. 11, no. 2, pp. 169–177,
1939.
[16] Y. Stylianou, “Applying the harmonic plus noise model in concatenative speech synthesis,” IEEE Trans. SAP, vol. 9, no. 1, pp.
21–29, 2001.
[17] D. Erro, I. Sainz, E. Navas, and I. Hernaez, “Harmonics plus noisemodel based vocoder for statistical parametric speech synthesis,”IEEE J-STSP, vol. 8, no. 2, pp. 184–194, 2014.
[18] K. Kobayashi, T. Toda, G. Neubig, S. Sakti, and S. Nakamura,“Statistical singing voice conversion with direct waveform modiﬁcation based on the spectrum differential,” Proc. INTERSPEECH,
pp. 2514–2418, Sept. 2014.
[19] ——, “Statistical singing voice conversion based on direct waveform modiﬁcation with global variance,” Proc. INTERSPEECH,
pp. 2754–2758, Sept. 2015.

1671