Proceedings, APSIPA Annual Summit and Conference 2021

14-17 December 2021, Tokyo, Japan

StarGAN-based Emotional Voice Conversion forJapanese Phrases
Asuka Moritani∗ and Shoki Sakamoto∗ and Ryo Ozaki† and Hirokazu Kameoka‡ and Tadahiro Taniguchi∗∗ College of Information Science and Engineering, Ritsumeikan University, Japan

E-mail: {moritani.asuka, taniguchi, sakamoto.shoki}@em.ci.ritsumei.ac.jp/† Graduate School of Information Science and Engineering, Ritsumeikan University, Japan

E-mail: ryo.ozaki@em.ci.ritsumei.ac.jp‡ NTT Communication Science Laboratories, NTT Corporation, Japan

E-mail: hirokazu.kameoka.uh@hco.ntt.co.jp

Abstract—This paper shows that StarGAN-VC, a spectralenvelope transformation method for non-parallel many-to-many
voice conversion (VC), is capable of emotional VC (EVC). Although StarGAN-VC has been shown to enable speaker identity
conversion, its capability for EVC for Japanese phrases has notbeen clarified. In this paper, we describe the direct applicationof StarGAN-VC to an EVC task with minimal fundamental frequency and aperiodicity processing. Through subjective
evaluation experiments, we evaluated the performance of our
StarGAN-EVC system in terms of its ability to achieve EVC for
Japanese phrases. The subjective evaluation is conducted in termsof subjective classification and mean opinion score of neutrality
and similarity. In addition, the interdependence between thesource and target emotional domains was investigated from theperspective of the quality of EVC.

Index Terms: voice conversion, emotion, deep generativemodel, generative adversarial networks.
I. I NTRODUCTION
Emotional expression in our daily conversations is essential for conveying our intentions, emotions, and social attitudes. Therefore, generating speech signals with designatedemotional attributes is an important objective in the speechsynthesis field. Producing emotional utterances artificially andconverting a source utterance with a particular emotional
attribute to one that is different is an important task. This task
is termed emotional voice conversion (EVC). In this study,
we explore the StarGAN-VC’s ability of EVC for Japanesephrases.
EVC has long been studied [1]–[7]. Recently, Ming et al. developed a method using a deep bidirectional LSTM model [6],and Luo et al. proposed a method using dual supervised adversarial networks [7]. Prosody is widely considered an importantfeature for expressing emotion. Therefore, most EVC studies
and techniques focus mainly on the conversion of fundamental
frequency (F0 ) contours. In this paper, instead of focusing
on F0 and aperiodicity transformation, we investigate whether
EVC can be achieved by spectral envelope transformation
alone.
In addition, many existing EVC methods require a parallel
corpus of utterances with different emotional attributes. However, preparing such a corpus can be costly and not always
feasible. In this respect, a method that does not require a

978-988-14768-9-0/21/$31.00 ©2021 APSIPA

parallel corpus and can be trained even with a non-parallel
corpus would be very useful.
For speaker identity conversion, a method based on a variantof generative adversarial networks (GANs) called StarGAN[8] was proposed [9]. This method, called StarGAN-VC, hasbeen shown to be capable of non-parallel many-to-many VC(i.e., learning mappings between multiple speakers’ voices
using a non-parallel corpus). StarGAN-VC was developed as
an extension of CycleGAN-VC [10], which was designed tohandle non-parallel one-to-one VC. In this paper, we study
StarGAN-VC applied to the EVC task and call it StarGANbased EVC (StarGAN-EVC). In StarGAN-EVC, we focus onspectral envelope transformation only. For F0 and aperiodicity,
we use simple methods, i.e., logarithm Gaussian normalization
for F0 [11] and identity mapping for the aperiodicity. Namely,
the aim of this study is to investigate how well StarGAN-VC
can achieve EVC when it is applied only to spectral envelope
transformation. Note that a similar study on English speech
had already been done [12]. However, to the best of ourknowledge, this is the first study on StarGAN-VC-based EVC
evaluated on Japanese speech. Thus, the main contribution ofthis paper is the subjective evaluation experiment conducted
to show that StarGAN-VC is capable of EVC for Japanesephrases.
The remainder of this paper is organized as follows. Section II describes the original formulation of StarGAN-VC, andSection III describes the detail of StarGAN-EVC. Section IV
describes the experiment. Finally, Section V concludes thispaper with a discussion.
II. P RELIMINARIES : S TAR GAN-VC
StarGAN-VC is a non-parallel and many-to-many voice
conversion method based on a variant of GANs called StarGAN. The overview of the method is shown in Figure 1.
StarGAN-VC consists of three networks, a generator, real/fakediscriminator, and domain classifier. The generator takes aspeech feature sequence in the source domain and the index c of the target domain as the inputs and generates aconverted version of the feature sequence. At training time,
the feature sequence generated by the generator is evaluated by the real/fake discriminator and domain classifier.

836

APSIPA-ASC 2021

Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:22:20 UTC from IEEE Xplore. Restrictions apply.

14-17 December 2021, Tokyo, JapanRealdomain 𝑘′

DomainclassifierFig. 1. Concept figure of StarGAN-VC [9]Input waveform

Sourceemotionalstate𝒄’Analysis
Fundamental
frequency (F0)Aperiodicity

Mel- cepstralcoefficients (MCCs)

Logarithm Gaussian
normalization(𝜇 , 𝜎 )Fundamental
frequency (F0)StarGAN-VC

Aperiodicity

Mel- cepstralcoefficients (MCCs)𝒄𝒄

Analysis
Synthesized waveform− Ex∼p(x),c∼p(c) [log(1 − D(G(x, c), c))],Targetemotional
state

Fig. 2. Overview of StarGAN-Emotional-VC

G
Ladv(G) = −Ex∼p(x),c∼p(c) [log D(G(x, c), c)],where y ∼ p(y|c) denotes the feature sequence of a “real”
training utterance belonging to domain c, and x ∼ p(x)denotes the feature sequence of a training utterance belonging
D (D) becomes small
to an arbitrary domain. The value of Ladv
when D correctly distinguishes “fake” samples G(x, c) from
G (G) becomes“real” samples y. Conversely, the value of Ladv
small when the generated samples G(x, c) are misclassified
as “real” by D. Therefore, during training, we would like to
D (D) with respect to D and L G (G) with respect
minimize Ladv
adv
to G.
Domain classification loss: The above losses are only responsible for encouraging the generated samples G(x, c) look
“real”. To further encourage the generated samples G(x, c)
to belong to domain c, the following training losses for thedomain classifier C and G can be used.
C(C) = −Ec∼p(c),y∼p(y|c) [log pC (c|y)],Lcls
G
Lcls (G) = −Ex∼p(x),c∼p(c) [log pC (c|G(x, c))].
G (G)
Lcls

domain 𝑘

Fake

D
Ladv
(D) = −Ec∼p(c),y∼p(y|c) [log D(y, c)]

C (C)
Lcls

Generator

Real

Training example
in domain 𝑘′

Real/Fakediscriminator
Real/Fake

Training example
in domain 𝑘

1, … , 𝐾

The generator and discriminator are trained in an adversarial
manner. Namely, while the discriminator is trained to correctly
distinguish the feature sequences generated by the generator
from those of real speech, the generator is trained to deceive
the discriminator by making the generated feature sequences
as indistinguishable as possible from the feature sequences ofreal speech. In addition, the generator and domain classifier
are trained to cooperatively maximize the classification performance of the classifier. Namely, the generator is trained togenerate feature sequences so that they are correctly classified
by the classifier as belonging to the target domain.
Let us use ŷ = G(x, c) to express the output of the generator
G, where x ∈ RQ×N denotes an input feature sequence ofdimension Q and length N, and c denotes the target domain
index (the index of the domain in which G attempts to convertx). Here, c is assumed to be represented as a one-hot vector.
The discriminator D outputs the probability D(y, c) ∈ [0, 1]of an input feature sequence y being “real”, and the domainclassifier C outputs the probability pC (c|y) ∈ [0, 1] of an input
feature sequence y belonging to domain c. By using these
expressions, the training losses are defined as follows.
Adversarial loss: The losses for the generator G anddiscriminator D can be defined as follows:

domain 𝑘

Proceedings, APSIPA Annual Summit and Conference 2021

used to encourage G to learn linguistic-preserving conversions
Lcyc (G) = Ec0 ∼p(c),x∼p(x|c0 ),c∼p(c) ||G(G(x, c), c0 ) − x||ρ .
(1)
Here, x ∼ p(x|c0 ) denotes the feature sequence of a training
utterance belonging to domain c0 . | · |ρ denotes Lρ -norm whereρ is a positive number, typically set to ρ = 1 or 2.
Identity mapping loss: To let G not make any changes toan input when c = c0 , the identity mapping loss is introduced
as follows:
Lid (G) = Ec0 ∼p(c),x∼p(x|c0 ) [||G(x, c0 ) − x||ρ ].
Overall loss function: By using the above losses, the overallloss functions for the generator, discriminator, and domainclassifier can be defined as follows:GG
IG (G) = Ladv
(G) + λcls Lcls
(G) + λcyc Lcyc (G) + λid Lid (G),D
(D),
ID (D) = Ladv

The values ofandbecome small when
C correctly classifies y ∼ p(y|c) and G(x, c) as belonging
to domain c. Therefore, during training, we would like to
C (C) with respect to C and L G (G) with respect
minimize Lcls
cls
to G.
Cycle-consistency loss: Training G using only the adversarial and domain classification losses does not ensure thatthe linguistic content of an input of G is preserved. Like
CycleGAN [10], the following cycle-consistency loss can beC
IC (C) = Lcls
(C),

where the hyperparameters λcls ≥ 0 and λcyc ≥ 0, λid ≥ 0
represent the importance of the domain classification, cycleconsistency, and identity mapping losses, respectively. Thesehyperparameters can be set manually or optimized by hyperparameter search.
For more details, please refer to the original paper [9].
III. S TAR GAN-EVC
In StarGAN-EVC, an emotional state is used as the conditioning input c. Figure 2 illustrates the overall StarGANEVC system. First, an input waveform is transformed into the837
Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:22:20 UTC from IEEE Xplore. Restrictions apply.

Proceedings, APSIPA Annual Summit and Conference 2021

14-17 December 2021, Tokyo, Japan

sequences of the F0 s, mel-cepstral coefficients (MCCs), andaperiodicities, each extracted within a short-term frame. The
sequence of the MCC vectors is then fed into the StarGANVC module. In StarGAN-EVC, the domain c corresponds toan emotional state (e.g., joyful, sad, or surprised). At training
time, training utterances labeled with emotional states areused. At test time, the MCC sequence of input speech isconverted via the generator using the target emotional statelabel c.
The F0 contour of input speech is converted using logarithm
Gaussian normalization [11]. Logarithm Gaussian normalization is a method that simply adjusts the entire F0 contour using
a linear transformation in the logarithmic domainlog( fconverted ) =σc (log( finput ) − µc0 )
+ µc
σc0

where µc0 and σc0 are the mean and standard deviation of
the logarithmic F0 s of the source emotional state, and µc
and σc are those of the target emotional state. The statisticalparameters µc , σc , µc0 , and σc0 are calculated using the training
utterances.
The aperiodicity sequence extracted from input speech is
directly used without modification.

the domain classifier. For training StarGAN-VC, 17 of the 20
selected phrases were used as a non-parallel corpus for the
training dataset. The remaining three phrases, “amamizuwa,”“midori,” and “nami”2 , were used as test data. In the StarGANVC training phase, the batch size and the number of epochs
were set to 2 and 2000, respectively. We used Adam optimizer with (α.β1 ) = (1.0 × 10−3 , 0.9), (5.0 × 10−5 , 0.5), and(1.0 × 10−3 , 0.5) for the generator, domain classifier, andadversarial real/fake discriminator, respectively. Other parameters for Adam were set to the default values [17].
We performed emotional voice conversions using StarGANEmotional-VC. Test data with the “neutral” label were converted into “joyful,” “angry,” and “sad” speech. Those labeled“joyful” were converted into “neutral,” “angry,” and “sad”
speech. Those labeled “angry” were converted into “neutral,”“joyful,” and “sad” speech. Those labeled “sad” were converted into “neutral,” “joyful,” and “angry” speech. In total,
we converted 36 synthesized speech signals.
The 48 data units included 12 original test data units, i.e., 3
phrases with 4 emotional expressions and 36 synthesized data
units, i.e., 3 phrases with 4 emotional expressions converted
from 4 source input data corresponding to 4 emotional states.
B. Subjective evaluation

IV. E XPERIMENT
A. Experimental setup
We used the Keio University Japanese Emotional Speech
Database (Keio-ESD)1 developed by Moriyama [13], and evaluated the StarGAN-Emotional-VC by subjective evaluation.
The acoustic signals recorded in Keio-ESD were directly
recorded to the hard disk drive by a microphone connectedto the PC in a soundproof room. A 32-year old male speaker
who had acting experience spoke 20 short Japanese phrases
expressing 47 emotional states. The spoken phrases recorded
as Waveforms were digitized by 16-kHz sampling and 16-bit
quantization. The data were stored in the WAV format.
For analyzing the input waveforms and synthesizing the
waveforms of the converted acoustic features, we usedWORLD [14] (D4C edition [15]), which is a high-quality
speech analysis and synthesis system akin to a vocoder.
Logarithmic fundamental frequency (log F0 ), spectral envelope, and aperiodicity were extracted every 5 ms using
WORLD [14] (D4C edition [15]. For training StarGAN-VC,
36-dimensional MCCs calculated from the spectral envelope
were used.
In this experiment, we adopted four representative emotional
states. They were neutrality labeled (neutral) and three othertypical emotional states, i.e., joy (joyful), anger (angry), andsorrow (sad), which are distant from each other in the circumplex model of the affect grid proposed by Russell [16].
We employed the same network architecture as that of
the original paper [9]. Namely, gated convolutional neuralnetworks were used for the generator, the discriminator and1) Subjective classification: Ten participants were asked tolabel a converted voice as “neutral,” ‘joyful,” “angry,” or “sad”
3 . The 48 audio data units were played randomly to eachparticipant.
Table I shows the subjective classification results for theoriginal input data, i.e., test data. The highest values are bold
and underlined, and the second-highest values are underlined,
except for 0.0%, in the following tables. Each rate means
the ratio of the participants who did or did not identify theemotional category when they listened to the phrase includedin the set of test data for each emotional state label.
Table I shows that the participants almost all correctly
identified the emotion categories by listening to the data,
although “joyful” included over 40% misrecognitions. Thismeans that the four emotional states were evoked correctly inthe test data and the participants were capable of evaluating
the synthesized data.
Table II shows the subjective classification results for theconverted data with respect to the source and target attribute
domains. The results averaged over every source attribute
domain are shown in Table III.
This demonstrates that, generally, StarGAN-Emotional-VC
could, to a certain extent, convert neutral utterances intoutterances from each emotional category. With every result, theclassification rate of the source attribute domain, i.e., emotion
category, decreased, and that of the target attribute domainincreased. In particular, in the conversions from “neutral” to“angry” and “sad,” from “joyful” to another emotional state,
2 They mean “rainwater is,” “green,” and “wave” in English, respectively

1 Keio University Japanese Emotional Speech Database: http://research.nii.

ac.jp/src/en/Keio-ESD.html

3 The participants were 21–24 year old undergraduate and graduate students
and native Japanese speakers.

838
Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:22:20 UTC from IEEE Xplore. Restrictions apply.

Proceedings, APSIPA Annual Summit and Conference 2021

14-17 December 2021, Tokyo, Japannaturality

TABLE IS UBJECTIVE CLASSIFICATION RESULTS FOR THE ORIGINAL INPUT DATA

NeutralJoyfulAngry
Sad

Neutral100.0%
26.7%
10.0%
0.0%

Subjective classification
Joyful
Angry
Sad
0.0%
0.0%
0.0%
56.7%
16.7%
0.0%
0.0%
90.0%
0.0%
0.0%
0.0%
100.0%MOS

Category

4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0

similarity

SourceProposed

TABLE IIS UBJECTIVE CLASSIFICATION RESULTS FOR THE CONVERTED DATA
DomainSourceTargetJoyfulNeutralAngry
Sad
NeutralJoyfulAngry
Sad
NeutralAngry
JoyfulSad
NeutralSad
JoyfulAngry

Fig. 3. Result of average on all MOS and ESS

Subjective classification
NeutralJoyful
Angry
Sad
46.7%
30.0%
13.3%
10.0%
43.3%
10.0%
43.3%
3.3%
13.3%
23.3%
10.0%
53.3%
63.3%
6.7%
20.0%
10.0%
26.7%
16.7%
50.0%
6.7%
3.3%
20.0%
16.7%
60.0%
83.3%
6.7%
3.3%
6.7%
13.3%
50.0%
36.7%
0.0%
20.0%
26.7%
36.7%
16.7%
20.0%
0.0%
6.7%
73.3%
16.7%
16.7%
13.3%
53.3%
16.7%
10.0%
13.3%
60.0%

TABLE IVR ESULT OF MOS ( FOR NATURALNESS )DomainTargetNeutralJoyfulAngry
Sad

DomainTargetNeutralJoyfulAngry
Sad

AVERAGED RESPONSE RATE OF EMOTIONAL ATTRIBUTES OF CONVERTEDSPEECH REGARDLESS OF ATTRIBUTE OF INPUT DATA

Neutral
55.6%
25.6%
28.9%
12.2%

Domain Source
Joyful
Angry
2.50
2.50
–
2.50
2.00
–
2.71
1.96

Original
Sad
1.92
1.50
1.58
–

4.79
4.83
4.46
4.79

TABLE VR ESULT OF ESS ( FOR SIMILARITY )TABLE III

DomainTargetNeutralJoyfulAngry
Sad

Neutral
–
2.13
2.13
1.88

Subjective classification
Joyful
Angry
Sad
4.4%
10.0%
30.0%
32.2%
21.1%
21.1%
12.2%
35.6%
23.3%
23.3%
21.1%
43.3%

and from “angry” to “neutral” and “joyful,” the target attribute
domain successfully obtained the highest correctness score.
In contrast, every synthesized utterance converted from“sad” remained in the “sad” emotional state, i.e., the subjectiveclassification rate of “sad” received the highest score. We
consider that this is because aperiodicity is known to contributea “sad” overtone expression compared to other emotional
expressions [18]. Considering that StarGAN-Emotional-VC
converted only MCCs, this result is understandable.
However, collectively, Table III shows averaged subjectiveclassification rate of an emotional attribute of converted speech
regardless of the attribute of input data. This shows that theemotional voice conversion successfully generated utteranceswith target emotional expressions.
The result shows that StarGAN-VC could convert an emotional expression to a certain extent even though it does not
apply any modern techniques to the conversion of F0 andaperiodicity features.
2) Mean opinion and emotional similarity scores: Mean
opinion score (MOS) and emotional similarity score (ESS)
tests were also performed to determine whether the converted
speech signals were natural and similar to the target speech
signals corresponding to each emotional state, respectively.
Eight participants were asked to evaluate the converted speech

Neutral
–
2.54
2.63
3.58

Domain Source
Joyful
Angry
3.17
3.25
–
2.42
3.08
–
3.50
2.92

Original
Sad
3.04
1.92
2.04
–

4.67
3.33
4.54
4.92

signals. Each participant was requested to listen to and evaluateeach converted speech signal on a 5-point scale. The converted
speech signal was evaluated by participants based on how
natural and similar it was to the target voice. As a result,
we obtained MOS for “naturalness” and ESS for “similarity”.
Figure 3 shows the averaged MOS and ESS scores forthe original and converted voices. It shows that MOS for
naturalness was lower than ESS for similarity. This means theoverall quality of converted voice has room for improvement
though the emotional conversion was achieved to a certain
extent4 .
Tables IV and V show the MOS for “naturalness” and ESS
for “similarity” for each pair of emotional states. The final
columns of the tables show MOS or ESS of the original voice,i.e., non-converted voice, as references. The value of each cellis the average of 24 scores, i.e., eight participants for eachutterance and three utterances for each emotional state. The
tendencies we found in the tables, in general, share the samecharacteristics as the classification-based evaluation. However,some interesting features were observed.
When the source domain was “sad”, the MOS for eachtarget domain was less than 2. This means that the system,
i.e., GAN-based spectral envelope transformations, could not
produce a natural voice. This suggests that “sad” has a very
different spectral envelope from the viewpoint of GAN-based
4 It should be noted that the dataset size was small because of the scarcity
of available datasets for emotional voice conversion. Increasing the size ofthe dataset may improve naturalness to a certain extent.

839
Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:22:20 UTC from IEEE Xplore. Restrictions apply.

Proceedings, APSIPA Annual Summit and Conference 2021

14-17 December 2021, Tokyo, Japantransformations and typical characteristics of other features,e.g., aperiodicity and speech speed.
Furthermore, it is worth noting that Tables IV and V havedifferent tendencies and asymmetricity. Regarding a pair ofemotional states, high ESS does not mean high MOS, and thepossibility of voice conversion from emotional state A to B
does not guarantee that from B to A despite the contribution ofcycle-consistency loss (see (1)) in StarGAN-VC. This suggeststhat transformation of other features, e.g., speech speed, iscrucial for bi-directional emotional voice conversion.
V. C ONCLUSION
This study conducted emotional voice conversion using
StarGAN-VC, i.e., StarGAN-Emotional-VC, and evaluated thecapability of emotional voice conversion for Japanese phrases
through a subjective listening test. The result shows thatStarGAN-VC can perform emotional voice conversion mainly
through spectral envelope transformations. However, different
tendency was observed in subjective classification, MOS and
ESS depending on source and target emotional states.
Compared to modern state-of-the-art emotional voice conversion methods, the performance shown in the experiment
may be limited. However, StarGAN-VC promises further
extension and integration considering its non-parallel many-tomany conversion characteristics. We believe that this promising non-parallel many-to-many voice conversion method canbe used for EVC with the simultaneous use of other EVC
methods to process F0 and aperiodicity. Integrating StarGANVC with more sophisticated EVC methods for F0 and aperiodicity and applying the proposed method to sentence-level
EVC will be pursued in the future.

[6] H. Ming, D. Huang, L. Xie, J. Wu, M. Dong, and H. Li, “Deepbidirectional LSTM modeling of timbre and prosody for emotional voice
conversion,” INTERSPEECH, pp. 2453–2457, 2016.
[7] L. Zhaojie, C. Jinhui, T. Tetsuya, and A. Yasuo, “Emotional voice
conversion using dual supervised adversarial networks with continuous
wavelet transform f0 features,” IEEE/ACM Transactions on Audio,
Speech, and Language Processing, vol. 27, no. 10, pp. 1535–1548, 2019.
[8] Y. Choi, M. Choi, M. Kim, J.-W. Ha, S. Kim, and J. Choo, “StarGAN:
Unified generative adversarial networks for multi-domain image-toimage translation,” in Proceedings of the IEEE conference on computer
vision and pattern recognition, 2018, pp. 8789–8797.
[9] H. Kameoka, T. Kaneko, K. Tanaka, and N. Hojo, “StarGAN-VC: Nonparallel many-to-many voice conversion using star generative adversarial
networks,” in 2018 IEEE Spoken Language Technology Workshop (SLT),2018, pp. 266–273.
[10] T. Kaneko and H. Kameoka, “CycleGAN-VC: Non-parallel voice
conversion using cycle-consistent adversarial networks,” in 2018 26th
European Signal Processing Conference (EUSIPCO). IEEE, 2018, pp.
2100–2104.
[11] K. Liu, J. Zhang, and Y. Yan, “High quality voice conversion throughphoneme-based linear mapping functions with straight for mandarin,”in Fourth International Conference on Fuzzy Systems and Knowledge
Discovery (FSKD 2007), vol. 4, Aug 2007, pp. 410–414.
[12] J. Gao, D. Chakraborty, H. Tembine, and O. Olaleye, “Nonparallel emotional speech conversion,” in Proceedings of the Annual Conference of
the International Speech Communication Association, INTERSPEECH,
2019, pp. 2858–2862.
[13] T. Moriyama, S. Mori, and S. Ozawa, “A synthesis method of emotional
speech using subspace constraints in prosody,” Journal of Information
Processing, vol. 50, no. 3, pp. 1181–1191, 2009.
[14] M. Masanori, Y. Fumiya, and O. Kenji, “World: a vocoder-based
high-quality speech synthesis system for real-time applications,” IEICE
TRANSACTIONS on Information and Systems, vol. 99, no. 7, pp. 1877–
1884, 2016.
[15] M. Masanori, “D4c, a band-aperiodicity estimator for high-quality
speech synthesis,” Speech Communication, vol. 84, pp. 57–65, 2016.
[16] J. A. Russell, “A circumplex model of affect,” Journal of personality
and social psychology, vol. 39, no. 6, p. 1161, 1980.
[17] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”in International Conference on Learning Representations (ICLR), 2015.
[18] K. R. Scherer, “Vocal affect expression: A review and a model for future
research,” Psychological bulletin, vol. 99, no. 2, p. 143, 1986.

VI. ACKNOWLEDGEMENTS
This study is partially supported by the Japan Society for
the Promotion of Science (JSPS) KAKENHI Grant-in-Aid for
Scientific Research (B), grant number 18H03308, and Grantin-Aid for Scientific Research on Innovative Areas, grant
number 16H06569.
A PPENDIXR EFERENCES[1] S. Mori, T. Moriyama, and S. Ozawa, “Emotional speech synthesis using
subspace constraints in prosody,” in IEEE International Conference on
Multimedia and Expo, 2006, pp. 1093–1096.
[2] J. Tao, Y. Kang, and A. Li, “Prosody conversion from neutral speech toemotional speech,” IEEE transactions on Audio, Speech, and Language
processing, vol. 14, no. 4, pp. 1145–1154, 2006.
[3] Z. Inanoglu and S. Young, “A system for transforming the emotion in
speech: Combining data-driven conversion techniques for prosody andvoice quality,” in Eighth Annual Conference of the International SpeechCommunication Association, 2007.
[4] R. Aihara, R. Takashima, T. Takiguchi, and Y. Ariki, “GMM-based
emotional voice conversion using spectrum and prosody features,”
American Journal of Signal Processing, vol. 2, no. 5, pp. 134–138,
2012.
[5] H. Ming, D. Huang, M. Dong, H. Li, L. Xie, and S. Zhang, “Fundamental frequency modeling using wavelets for emotional voice conversion,”in International Conference on Affective Computing and Intelligent
Interaction (ACII), 2015, pp. 804–809.

840
Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:22:20 UTC from IEEE Xplore. Restrictions apply.