୍⯡♫ᅋἲே 㟁Ꮚ᝟ሗ㏻ಙᏛ఍                      ಙᏛᢏሗ‫ٻ‬‫ٻڇڮڞڤکڪڭگڞڠڧڠٻڡڪٻڠگڰگڤگڮکڤٻڠڣگ‬                     ৴ֶٕใ‫ٻۏۍۊۋۀڭٻۇڼھۄۉۃھۀگٻڠڞڤڠڤ‬ࣾஂ๏ਓ ిࢠ৘ใ௨৴ֶձ
‫ٻڮڭڠڠکڤڢکڠٻکڪڤگڜڞڤکڰڨڨڪڞٻڟکڜٻکڪڤگڜڨڭڪڡکڤ‬                TECHNICAL‫ٻڄڍڌڈڐڌڋڍڃڎڒڈڐڌڋڍګڮ‬THE INSTITUTE OF ELECTRONICS,REPORT OF IEICE.
‫ٻ‬INFORMATION AND COMMUNICATION ENGINEERSԻ੠೾‫ܗ‬Ճ޻ʹ‫ͮ͘ج‬ඇ฼‫ޠ‬Ի੠ͷ‫ܧ‬ଓ௕ิਖ਼๏ͷධՁͱ෼ੳ
۞ཏਅ໵†ߴಓ৻೭հ†‫ ాށ‬ஐ‫†ج‬,†† χϡʔϏοάɹάϥϜ†தଜαΫςΟɹαΫϦΞχ†఩†† ಸྑઌ୺Պֶٕज़େֶӃେֶ৘ใՊֶ‫ڀݚ‬Պ ˟ 630–0192 ಸྑ‫ݝ‬ੜۨࢢߴࢁொ 8916–5†† ໊‫ݹ‬԰େֶ৘ใ‫ج‬൫ηϯλʔ‫ڭ‬ҭ৘ใϝσΟΞ‫ڀݚ‬෦໳ ˟ 565–0456 Ѫ஌‫ݹ໊ݝ‬԰ࢢઍछ۠ෆ࿝ொ
E-mail: †{kura.shinya.ke9,shinnosuke-t,neubig,ssakti,s-nakamura}@is.naist.jp, ††tomoki@icts.nagoya-u.ac.jp͋Β·͠‫ֶޠݴ‬श౳΁ͷԠ༻Λ‫ݟ‬ਾ͑ɼඇ฼‫ޠ‬࿩ऀʹΑΓൃ࿩͞ΕͨԻ੠ʹରͯ͠ɼൃ࿩ϦζϜΛิਖ਼͢Δٕज़͕

‫͞ڀݚ‬Ε͍ͯΔɽ୅දతͳख๏ͱͯ͠ɼ฼‫ޠ‬࿩ऀͷԻ੠ΛࢀরԻ੠ͱͯ͠ɼಈత࣌ؒ৳ॖΛ༻͍ͯԻ੠ಛ௃ྔ‫ྻܥ‬Λม‫͢ܗ‬Δ͜ͱͰɼิਖ਼Ի੠Λ߹੒͢Δ࿮૊Έ͕ఏҊ͞Ε͍ͯΔɽॊೈͳൃ࿩ϦζϜิਖ਼͕ՄೳͱͳΔҰํͰɼԻ੠෼
ੳ߹੒ॲཧʹΑΔԻ࣭ྼԽ͸ආ͚ΒΕͳ͍ɽຊߘͰ͸ɼԻ੠෼ੳ߹੒ॲཧʹΑΔԻ࣭ྼԽΛճආ͢Δิਖ਼๏ͱͯ͠ɼ
ಈత࣌ؒ৳ॖͱԻ੠೾‫ܗ‬Ճ޻ॲཧʹΑΔൃ࿩ϦζϜิਖ਼๏ΛఏҊ͢Δɽ࣌ؒ৳ॖ‫ن‬ଇΛ֫ಘ͢Δࡍʹ͸ɼ౷‫ܭ‬త੠࣭ม‫׵‬Λద༻͢Δ͜ͱͰɼ࿩ऀੑͷࠩҟ͕Ի੠εϖΫτϧύϥϝʔλʹ༩͑ΔӨ‫ڹ‬Λ؇࿨͢Δͱͱ΋ʹɼඇ฼‫ޠ‬࿩ऀԻ੠ʹ‫ݟ‬ΒΕΔԻૉૠೖ‫ޡ‬Γʹ΋ରॲ͢Δɽ೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀʹΑΔӳ‫ޠ‬Ի੠ʢ೔ຊਓӳ‫ޠ‬ʣΛ༻͍࣮ͨ‫ݧ‬తධՁ͔ΒɼఏҊ๏ʹΑΓɼԻ࣭ྼԽΛ؇࿨ͭͭ͠ɼΑΓࣗવͳ‫ܧ‬ଓ௕Λ࣋ͭӳ‫ޠ‬Ի੠Λ߹੒Ͱ͖Δ͜ͱΛࣔ͢ɽ·ͨɼ࣌ؒ৳ॖ
ʹΑΓੜ͡Δ‫ॴہ‬తͳԻ࣭ྼԽͷӨ‫͍ͯͭʹڹ‬ɼԻ੠εϖΫτϧύϥϝʔλͷมௐεϖΫτϧΛ༻͍ͯ෼ੳ͢ΔɽΩʔϫʔυඇ฼‫ޠ‬Ի੠ɼ‫ܧ‬ଓ௕ิਖ਼ɼಈత࣌ؒ৳ॖɼ೾‫ܗ‬Ճ޻ɼมௐεϖΫτϧ

Evaluation and Analysis of Duration Correction for Non-Native Speech
Based on Waveform Modiﬁcation
Shinya KURA† , Shinnosuke TAKAMICHI† , Tomoki TODA†,†† , Graham NEUBIG† , Sakriani
SAKTI† , and Satoshi NAKAMURA†† Nara Institute of Science and Technology Takayama-cho 8916–5, Ikoma, Nara, 630–0192 Japan†† Information Media Division, Information Technology Center, Nagoya University Furo-cho, Chikusa-ku
Nagoya, Aichi, 464–8601 Japan
E-mail: †{kura.shinya.ke9,shinnosuke-t,neubig,ssakti,s-nakamura}@is.naist.jp, ††tomoki@icts.nagoya-u.ac.jp
Abstract There are several attempts at correcting durational patterns of non-native speech towards language
learning. One of the typical approaches modiﬁes a speech parameter sequence with Dynamic Time Warping (DTW)
using native speech as the reference, generating corrected speech from the modiﬁed speech parameter sequence.
Although this approach makes it possible to ﬂexibly modify durational patterns of non-native speech, quality of thecorrected speech signiﬁcantly degrades due to the use of analysis-synthesis process to generate the corrected speech.
In this report, we propose a method for correcting durational patterns using direct waveform modiﬁcation for performing DTW. In calculating a temporal warping function, statistical voice conversion is eﬀectively used to reducean adverse eﬀect caused by speaker diﬀerences. Moreover, phoneme insertion often observed in non-native speech
is also handled. We conducted an experimental evaluation using English speech read by Japanese, demonstratingthat the proposed method was capable of ﬂexibly modifying durational patterns while avoiding quality degradation caused by the analysis-synthesis process. Furthermore, waveform segments suﬀering from quality degradation
caused by temporal warping was analyzed using the modulation spectrum of spectral parameters.
Key words non-native speech, correction of durational patterns, dynamic time warping, waveform modiﬁcation,
modulation spectrum—1—This article is a technical report without peer review, and its polished and/or extended version may be published elsewhere.

Copyright ©2015 by IEICE

1. · ͑ ͕ ͖Εʹ൐͏඼࣭ྼԽ͸ආ͚ΒΕͳ͍ɽຊߘͰ͸ɼඇ฼‫ޠ‬Ի੠ʹର͢ΔߴԻ࣭ͳൃ࿩ϦζϜิਖ਼ॲཧ৘ใ௨৴ٕज़΍ަ௨ͷൃୡʹ൐͏‫ٸ‬଎ͳࠃࡍԽ͕ਐΈɼ‫ޠݴ‬ͱͯ͠ɼඇ฼‫ޠ‬Ի੠೾‫ʹܗ‬ର͢Δ௚઀తͳม‫ॲܗ‬ཧख๏ʹ͍ͭ‫ڭ‬ҭࢧԉͷधཁ͕ߴ·͍ͬͯΔɽಛʹɼ೔ຊʹ͓͍ͯ͸ɼ೔ຊ‫ޠ‬ͯ‫ݕ‬౼͢Δɽඇ฼‫ޠ‬Ի੠ͱಉҰจΛൃ࿩ͨ͠฼‫ޠ‬Ի੠Λ༻͍ͯɼ฼‫ޠ‬࿩ऀʹର͢Δӳ‫ֶޠ‬शࢧԉͷॏཁੑ͕஫໨͞Ε͍ͯΔ [1]ɽඇ฼‫ޠ‬Ի੠ʹରͯ͠ಈత࣌ؒ৳ॖ (Dynamic Time Warping:೔ຊ‫ͱޠ‬ӳ‫Ͱޠ‬͸ɼ‫ޠݴ‬ମ‫͕ܥ‬େ͖͘ҟͳΔͨΊɼ೔ຊ‫ޠ‬฼‫ޠ‬DTW) [14] Λࢪ͢͜ͱͰɼ࣌ؒ৳ॖ‫ن‬ଇΛਪఆ͢Δɽͦͷࡍʹ࿩ऀʹΑΔӳ‫ޠ‬ʢ೔ຊਓӳ‫ޠ‬ʣԻ੠͸ɼಠࣗͷಛ௃Λ࣋ͪ΍͢͸ɼ౷‫ܭ‬త੠࣭ม‫ٕ׵‬ज़ [15] Λ༻͍ͯɼ࿩ऀͷҧ͍ʹ‫ى‬Ҽ͢Δ͍ɽྫ͑͹ɼ೔ຊ‫ޠ‬͸ɼϞʔϥ୯Ґͷൃ࿩౳࣌ੑΛ࣋ͭϞʔϥԻ‫ڹ‬ಛ௃ྔͷࠩҟΛ؇࿨͢Δɽ࣍ʹɼ࣌ؒ৳ॖ‫ن‬ଇʹ‫͍ͯͮج‬ഥ‫( ޠݴ‬mora-timed languageʣͰ͋ΔҰํɼӳ‫ޠ‬͸ɼετϨεඇ฼‫ޠ‬Ի੠೾‫ʹܗ‬ର͠ WSOLA (Waveform Similarity based୯Ґͷ౳࣌ੑΛ࣋ͭ‫ڧ‬੎ഥ‫ޠݴ‬ʢstress-timed languageʣͰ͋

OverLap-Add) [16] Λࢪ͢͜ͱͰɼඇ฼‫ޠ‬Ի੠ͷൃ࿩ϦζϜΛΔ [2]ɽͦΕ‫ʹނ‬ɼ੠ͷ‫ڧ‬ऑɼൃ࿩ϦζϜɼΠϯτωʔγϣϯ౳

ิਖ਼͢Δɽ೔ຊਓӳ‫ޠ‬Ի੠Λ༻͍࣮ͨ‫݁ݧ‬Ռ͔Βɼ෼ੳ࠶߹੒

ͷӆ཯ಛ௃͕྆‫Ͱؒޠݴ‬େ͖͘ҟͳΔͨΊ [3]ɼ೔ຊਓӳ‫ޠ‬Ի੠Λ༻͍Δख๏ͱൺֱͯ͠ɼఏҊ๏ʹΑΓߴԻ࣭ͳิਖ਼ॲཧ͕Մ

ͷӆ཯ಛ௃͸ɼӳ‫ޠ‬ಠࣗͷ΋ͷͱ͸େ͖͘ҟͳΔ৔߹͕ଟʑ‫ݟ‬

ೳͰ͋Δ͜ͱΛࣔ͢ɽ͞Βʹɼ࣌ؒ৳ॖ‫ن‬ଇʹΑΓ‫ॴہ‬తʹੜΒΕΔɽ·ͨɼ෼અతಛ௃ʹ͓͍ͯ΋ɼԻૉηοτͷҧ͍ͷΈ͡ΔࣗવੑྼԽͷӨ‫ڹ‬ΛɼมௐεϖΫτϧ [17] Λ༻͍ͯ෼ੳ͢

ͳΒͣɼ೔ຊ‫ޠ‬ͷԻઅ͸ɼ฼Ի΋͘͠͸ࢠԻͱ฼ԻͷରʹΑΓΔͱͱ΋ʹɼࣗવੑྼԽՕॴͷࣗಈ‫ݕ‬ग़ΛࢼΈΔɽͳ͓ɼ[18] ͓ߏ੒͞ΕΔͷʹର͠ɼӳ‫ޠ‬ͷԻઅ͸ɼ฼ԻʹࢠԻ͕‫ޙ‬ଓͨ͠ΓɼΑͼ [19] ʹ͓͍ͯ΋ɼඇ฼‫ޠ‬Ի੠೾‫ʹܗ‬ର͢Δม‫ॲܗ‬ཧʹ‫ͮج‬ෳ਺ͷࢠԻ͕࿈ଓͨ͠Γ͢Δͱ͍ͬͨҧ͍͕͋ΔɽͦͷͨΊɼ͘ൃ࿩ϦζϜิਖ਼๏͕ఏҊ͞Ε͍ͯΔ͕ɼ[18] ͸ɼԻૉૠೖ‫ޡ‬೔ຊਓӳ‫ޠ‬Ի੠ʹ͸ɼࢠԻͷ௚‫ʹޙ‬฼ԻΛ෇͚Ճ͑ͯൃԻ͢Δ

Γ΁ͷରॲʹ͍ͭͯߟྀ͞Ε͓ͯΒͣɼ[19] ͸ɼ౷‫ܭ‬త੠࣭มʮૠೖʯɼ೔ຊ‫ʹޠ‬ଘࡏ͠ͳ͍ԻૉΛ೔ຊ‫ޠ‬ͷԻૉʹஔ͖‫ͯ͑׵‬‫͍ͯͭʹ׵‬͸ߟྀ͍ͯ͠ͳ͍ɽҰํͰɼຊߘͰ͸ɼ͜ΕΒΛߟൃԻ͢Δʮஔ‫׵‬ʯɼඞཁͳԻૉ͕୤མ͢Δʮ࡟আʯͱ͍ͬͨԻૉ

ྀͨ͠࿮૊ΈΛఏҊ͢Δͱͱ΋ʹɼ࣌ؒ৳ॖʹ‫ൃͮ͘ج‬࿩Ϧζ‫ޡ‬Γ͕ੜ͡΍͍͢ [4]ɽ͜ΕΒͷཁҼʹΑΓɼ೔ຊਓӳ‫ޠ‬Ի੠ͷϜิਖ਼ॲཧͷ͞ΒͳΔߴ඼࣭ԽΛ໨ࢦ͠ɼࣗવੑྼԽΛ΋ͨΒ໌ྎੑ͸େ͖͘௿Լ͠ɼ·ͨɼ͜ͷ໌ྎੑ௿Լͷ໰୊ʹ͍ͭͯɼ͢ཁҼΛௐࠪ͢Δɽ೔ຊਓӳ‫ޠ‬ͷൃ࿩ϦζϜ͕େ͖ͳཁҼͰ͋Δ͜ͱ΋஌ΒΕ͍ͯ
Δ [5]ɽͳ͓ɼӳ‫ݶʹޠ‬Βͣɼͦͷଞͷ‫͍͓ͯʹޠݴ‬΋ɼൃ࿩Ϧ

2. DTW Λ༻͍ͨඇ฼‫ޠ‬Ի੠ͷ‫ܧ‬ଓ௕ิਖ਼ζϜ͕໌ྎੑ௿ԼΛ΋ͨΒ͢ͱ͍͏஌‫͕ݟ‬ಘΒΕ͍ͯΔ [6]ɽఏҊ๏ͷॲཧखॱΛਤ 1 ʹࣔ͢ɽඇ฼‫ޠ‬Ի੠ͱ฼‫ޠ‬Ի੠ͷಉ೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀʹର͢Δӳ‫ֶޠ‬शࢧԉΛߦ͏্Ͱɼ྆‫ؒޠݴ‬Ұൃ࿩จΛ༻͍ͯɼ฼‫ޠ‬Ի੠ͷ࣌ؒ௕ʹҰக͢ΔΑ͏ʹඇ฼‫ޠ‬ͷ‫ޠݴ‬ମ‫ܥ‬ͷҧ͍ʹண໨͢Δ͜ͱ͸༗ޮͰ͋ΓɼԻ੠߹੒ٕज़
Λ‫͢༻׆‬Δख๏͕‫͞ڀݚ‬Ε͍ͯΔ [7]ɽྫ͑͹ɼ[8] Ͱ͸ɼ೔ຊԻ੠Λ࣌ؒ৳ॖ͢Δɽ

2. 1 ࣌ؒ৳ॖ‫ن‬ଇਪఆ෦‫ޠ‬ͷΞΫηϯτ‫ൃܕ‬Ի‫ڭ‬ҭͷ CALL γεςϜʹ͓͍ͯɼӆ཯ϝϧέϓετϥϜͻͣΈΛ‫཭ڑ‬ई౓ͱͨ͠ DTW ʹΑΓ࣌ؒͷ‫ޡ‬ΓΛमਖ਼ֶͨ͠शऀຊਓͷԻ੠ͷϑΟʔυόοΫΛఏࣔ͢৳ॖ‫ن‬ଇΛਪఆ͢Δɽͦͷࡍʹɼඇ฼‫ޠ‬࿩ऀͱ฼‫ޠ‬࿩ऀͷ࿩ऀ

Δٕज़͕ఏҊ͞Ε͍ͯΔɽ·ͨɼ[9] Ͱ͸ɼӅΕϚϧίϑϞσϧ

ੑͷࠩҟ͕ϝϧέϓετϥϜʹ༩͑ΔӨ‫ڹ‬Λ؇࿨͢ΔͨΊɼࠞ(Hidden Markov Model: HMM) ʹ‫ͮ͘ج‬ςΩετԻ੠߹੒ٕ

߹ਖ਼‫ن‬෼෍Ϟσϧ ʢGaussian Mixture Model: GMMʣ ʹ‫ج‬ज़ [10] Λ༻͍ͯɼࣄલऩ࿥ͨ͠೔ຊਓӳ‫ޠ‬Ի੠͔ΒɼΑΓࣗવͮ͘੠࣭ม‫׵‬๏Λ༻͍Δɽ

ͳൃ࿩ϦζϜΛ࣋ͭӳ‫ޠ‬Ի੠Λ߹੒͢Δख๏͕ఏҊ͞Ε͍ͯΔɽ

HMM Ի੠߹੒ͷదԠٕज़ [11] Λ༻͍ͯɼӳ‫ޠ‬฼‫ޠ‬࿩ऀͷ‫ܧ‬ଓ௕͓ΑͼύϫʔมԽͷಛ௃Λ೔ຊਓӳ‫ޠ‬Ի੠΁ͱ൓өͤ͞Δ͜
ͱͰɼֶशऀͷ‫ݸ‬ਓੑΛͰ͖Δ‫ݶ‬Γอ࣋ͭͭ͠ɼΑΓࣗવͳӳ‫ޠ‬Ի੠Λ߹੒ՄೳͱͳΔɽ͜ͷख๏ͷޮՌ͸ɼӳ‫ऀֶॳޠ‬ͷԻ੠Λ༻͍ͨ৔߹ʹ͓͍ͯ΋֬ೝ͞Ε͍ͯΔ [12]ɽ
ҰํͰɼ࣮ࡍʹൃ࿩͞Εͨ೔ຊਓӳ‫ޠ‬Ի੠ʹରͯ͠ม‫ॲܗ‬ཧΛࢪ͢͜ͱͰɼΑΓࣗવͳӳ‫ޠ‬Ի੠Λੜ੒͢Δख๏΋‫͞ڀݚ‬Ε
͍ͯΔɽςΩετԻ੠߹੒ͷΑ͏ʹෳࡶͳॲཧΛඞཁͱ͠ͳ͍
ͨΊɼ඼࣭ͷߴ͍ӳ‫ޠ‬Ի੠ͷੜ੒͕ՄೳͱͳΔɽ୅දతͳख๏
ͱͯ͠ɼඇෛ஋ۭ࣌ؒ෼ղΛ༻͍ͯɼൃ࿩͞Εͨඇ฼‫ޠ‬Ի੠ͷൃ࿩ϦζϜΛิਖ਼͢Δख๏͕ఏҊ͞Ε͍ͯΔ [13]ɽඇ฼‫ޠ‬Ի੠ͷεϖΫτϧύϥϝʔλ‫ྻܥ‬Λɼ֤ԻૉΛද͢‫ج‬ఈεϖΫτϧ
ͱ࣌ؒมԽ৘ใΛද͢ิؒॏΈ‫ʹྻܥ‬෼ղ͠ɼಉҰจΛൃ࿩ͨ͠฼‫ޠ‬Ի੠ͷ࣌ؒมԽ৘ใΛసࣸ͢Δ͜ͱͰɼิਖ਼Ի੠͕߹੒

2. 1. 1 GMM ੠࣭ม‫׵‬Λ༻͍ͨ࿩ऀੑͷӨ‫ڹ‬؇࿨ඇ฼‫ޠ‬࿩ऀͱ฼‫ޠ‬࿩ऀʹΑΔࣄલऩ࿥Ի੠͔Β੒ΔύϥϨϧ
σʔλ͔ΒɼGMM ͷύϥϝʔληοτ λ Λਪఆ͢Δɽλ̂ = argmaxλP (X t , Y t |λ)(1)

t͜͜ͰɼX t ɼY t ͸ͦΕͧΕɼϑϨʔϜ t ʹ͓͚Δඇ฼‫ޠ‬Ի੠͓Αͼ฼‫ޠ‬Ի੠ͷԻ੠ಛ௃ྔͰ͋ΓɼຊߘͰ͸ɼϝϧέϓετ
ϥϜ܎਺‫ͦͼٴ‬ͷ 1 ࣍ͷಈతಛ௃ྔͷ݁߹ϕΫτϧͱ͢ΔɽX t
ͱ Y t ͷରԠ෇͚͸ɼDTW ʹΑΔ࣌ؒϑϨʔϜΞϥΠϝϯτ
ʹΑΓߦ͏ɽ݁߹֬཰ີ౓ؔ਺ P (X t , Y t |λ) ͸ɼ࣍ࣜͰද͞ΕΔɽ

P (X t , Y t |λ) =

M




(ZZ)
αm N Z t ; μ(Z)m , Σm

(2)

m=1

ͨͩ͠ɼ

͞ΕΔɽԻ੠ಛ௃ྔͷม‫ʹܗ‬ΑΔॊೈͳൃ࿩ϦζϜิਖ਼͕Մೳ
ͱͳΔ͕ɼҰํͰԻ੠ͷ෼ੳ࠶߹੒ॲཧΛඞཁͱ͢ΔͨΊɼͦ

—2—

1RQQDWLYH
VSHHFK

1DWLYHVSHHFK)HDWXUHH[WUDFWLRQ
7UDLQLQJ‫ܨ‬଴ SDUDPHWHUV6SHFWUDOSDUDPHWHUV'7:ਤ 2 DTW ʹ͓͚ΔભҠίετɽx ͸ඇ฼‫ޠ‬࿩ऀͷ࣌ؒϑϨʔϜΛද*00

͠ɼy ͸฼‫ޠ‬࿩ऀͷ࣌ؒϑϨʔϜΛද͢ɽ
Fig. 2 Transtion cost in DTW, where x and y show time-frame of*00

non-native speech and that of native speech, respectively.

&RQYHUVLRQ&RQYHUWHGIHDWXUHV:DUSLQJIXQFWLRQ$¶ܵ௡ିଵܵ௡ିଵ ൅ ‫ܮ‬
E¶

D


%¶

2XWSXW
VLJQDO&RUUHFWLRQRIGXUDWLRQDOSDWWHUQV
3URSRVHGPHWKRG)HDWXUHEDVHG$%,QSXW
VLJQDO

'7:ܵ௡݀;݊ െ ͳሻ‫ܮ݊ ܮ‬:DYHIRUPEDVHG

ਤ3:DUSLQJIXQFWLRQ:DUSLQJIXQFWLRQŷ = argmax P (Y |X, λ) subject to Y = W yy   :DYHIRUPV


‫ܨ‬଴
0/6$
ILOWHU SDUDPHWHUVWSOLA ͷखॱ

Fig. 3 Procedure of WSOLA.

&RUUHFWLRQ
E\:62/$&RUUHFWLRQ
E\'7:

6SHFWUDOSDUDPHWHUV(4)͜͜Ͱ W ͸ɼ੩తಛ௃ྔ‫ྻܥ‬Λͦͷ 1 ࣍ͷಈతಛ௃ྔͷ݁߹ϕΫτϧ‫֦ʹྻܥ‬ு͢ΔߦྻͰ͋Δɽ৚݅෇֬཰ີ౓ؔ਺P (Y |X, λ) ͸ɼ݁߹֬཰ີ౓ؔ਺ͷࣜ (2) ͔Βղੳతʹಋग़    ͞ΕΔɽͳ͓ɼա৒ͳฏ‫׈‬ԽʹΑΔม‫׵‬εϖΫτϧύϥϝʔλͷ඼࣭ྼԽΛ؇࿨͢ΔͨΊɼ‫಺ྻܥ‬มಈΛߟྀ͢Δɽ

&RUUHFWHGVSHHFKਤ 1 ൃ࿩ϦζϜͷิਖ਼ॲཧखॱɽԼ෦ࠨ (Feature-based) ͸ɼैདྷ๏Ͱ͋ΔԻ੠ಛ௃ྔ‫ʹྻܥ‬ର͢Δิਖ਼ॲཧΛࣔ͠ɼԼ෦ӈ(Waveform-based) ͸ఏҊ๏Ͱ͋Δ೾‫ܗ‬Ճ޻ʹ‫ิͮ͘ج‬ਖ਼ॲཧΛࣔ͢ɽ
Fig. 1 Procedure for correcting durational patterns.

2. 1. 2 DTW Λ༻͍ͨ‫ن‬ଇਪఆ
GMM ੠࣭ม‫ʹ׵‬ΑΓม‫͞׵‬ΕͨϝϧέϓετϥϜ‫ͱྻܥ‬ɼ฼‫ޠ‬Ի੠ͷϝϧέϓετϥϜ‫ʹྻܥ‬ରͯ͠ɼDTW Λߦ͏͜ͱͰɼඇ฼‫ޠ‬Ի੠ʹద༻͢Δ࣌ؒ৳ॖ‫ن‬ଇΛਪఆ͢Δɽ฼‫ޠ‬Ի੠

A lowerͷϝϧέϓετϥϜ‫ྻܥ‬௕͸‫ݻ‬ఆͱ͠ɼม‫ޙ׵‬ͷ‫ͱྻܥ‬ͷϝϧ

left box (“feature-based”) shows a conventional methodέϓετϥϜ‫཭ڑ‬Λ࠷খͱ͢Δ࣌ؒ৳ॖ‫ن‬ଇΛ‫ٻ‬ΊΔɽຊߘͰusing analysis-synthesis process, and a lower right box

༻͍ΔભҠίετΛਤ 2 ʹࣔ͢ɽ(“waveform-based”) shows the proposed method using direct waveform modiﬁcation process.


Zt =

Xt





Yt

Σ(ZZ)
=
m

ɼμ(Z)
m =



Σm

(Y X)

Σm

Σm

WSOLA Λ༻͍ͨ‫ܧ‬ଓ௕ม‫׵‬Λࢪ͢͜ͱͰɼൃ࿩ϦζϜิਖ਼

ɼ


(XY )

(XX)

Σm

ਪఆ͞Εͨ࣌ؒ৳ॖ‫ن‬ଇʹ‫͖ͮج‬ɼඇ฼‫ޠ‬Ի੠ʹରͯ͠ɼ



μ(X)
m
)
μ(Y
m

2. 2 ‫ܧ‬ଓ௕ิਖ਼෦

ॲཧΛߦ͏ɽWSOLA ͷखॱΛਤ 3 ʹࣔ͢ɽ͜͜Ͱ͸ɼ‫ॴہ‬త

(3)(Y Y )

ʹ‫ܧ‬ଓ௕Λ୹͘͢Δʢ͢ͳΘͪɼൃ࿩ϦζϜΛ଎͘͢ΔʣࡍͷॲཧΛද͓ͯ͠Γɼ೾‫ܗ‬ηάϝϯτ (A) ͱ೾‫ܗ‬ηάϝϯτ (B)ͷ઀ଓॲཧΛߦ͏ɽ೾‫ܗ‬ηάϝϯτ (A) ͷத৺࣌ؒΛ Sn−1 ͱͰ͋Δɽ͜͜ͰɼGMM ύϥϝʔληοτ͸ M ‫ݸ‬ͷࠞ߹ཁૉ͠ɼ೾‫ܗ‬ηάϝϯτ (B) ͷத৺࣌ؒΛ Sn ͱ͢Δɽ͜͜ͰɼSnΛ༗͠ɼΠϯσοΫε m ͷࠞ߹ཁૉ͸ɼࠞ߹ॏΈ αm ɼฏ‫ۉ‬͸ɼ࣌ؒ Sn−1 ʹ͓͚Δ࣌ؒ৳ॖ‫ن‬ଇʹ‫ܾ͖ͮج‬ఆ͞ΕΔɽ·(ZZ)

ϕΫτϧ μ(Z)m ɼ‫ڞ‬෼ࢄߦྻ Σm͔Β੒Δਖ਼‫ن‬෼෍Ͱද͞Εͨɼ֤ηάϝϯτ௕͸ 2L ͱ͠ɼ೾‫ܗ‬ηάϝϯτ (A) Λ L ͚ͩ

(X)
Δɽμ(Z)m ͸ɼೖग़ྗԻ੠ͷಛ௃ྔͷฏ‫ۉ‬ϕΫτϧͰ͋Δ μmγϑτͤͨ͞೾‫ܗ‬ηάϝϯτΛ (A’) ͱ͢Δɽ઀ଓʹΑΔࣗવ

(ZZ)
)
ͱ μ(Y
͸ɼೖग़ྗԻ੠ͷಛ௃ྔͷ‫ڞ‬m ͔Β੒Δɽ·ͨɼΣm(XX)(Y Y )‫ ͼٴ‬Σmͱɼ૬‫ڞޓ‬෼ࢄߦྻͰ͋Δ
෼ࢄߦྻͰ͋Δ Σm
(Y X)
(XY )
Σm ɼΣm͔Β੒Δɽτͤ͞ɼ೾‫ܗ‬ηάϝϯτ (A’) ͱ࠷΋ྨࣅ౓͕ߴ͘ͳΔγϑτ࣍ʹɼඇ฼‫ޠ‬Ի੠ͷϝϧέϓετϥϜ‫͔ྻܥ‬Β฼‫ޠ‬࿩ऀͷ͢Δ೾‫ܗ‬ηάϝϯτ (B’) ͕ܾఆ͢Δɽ೾‫ܗ‬ηάϝϯτ (A) ͷੑྼԽΛ཈͑ΔͨΊʹɼ೾‫ܗ‬ηάϝϯτ (B) ͷத৺࣌ؒΛγϑ௕ d Λ୳ࡧ͢ΔɽಘΒΕͨγϑτ௕ʹ‫͖ͮج‬ɼSn − d Λத৺ͱϝϧέϓετϥϜ‫΁ྻܥ‬ม‫͢׵‬Δɽඇ฼‫ޠ‬࿩ऀͱ฼‫ޠ‬࿩ऀͷ‫ޙ‬൒෦ʢSn−1 ʙ Sn−1 + Lʣͱ೾‫ܗ‬ηάϝϯτ (B’) ͷલ൒෦ ͱಛ௃ྔ‫ྻܥ‬ϕΫτϧΛɼͦΕͧΕ X = [X 
1 , · · · , XT ]

ʢSn − d − L ʙ Sn − dʣʹରͯ͠ɼ૭ֻ͚Λࢪͯ͠ॏ৞ՃࢉΛ 
Y = [Y 
1 , · · · , Y T ] ͱ͢Δɽ͜͜Ͱ T ͸ϑϨʔϜ਺Ͱ͋Δɽ

ߦ͏͜ͱͰɼ೾‫ܗ‬ηάϝϯτ (A) ͱ೾‫ܗ‬ηάϝϯτ (B’) Λ઀ ͸ɼ࣍ࣜͰද
ม‫͞׵‬Εͨ੩తಛ௃ྔ‫ ྻܥ‬ŷ = [ŷ 
1 , · · · , ŷ T ]ଓ͢Δɽਤʹ͓͍ͯɼ(a) ͱ (b’) ͸૭ֻ͚͞Εͨ೾‫ܗ‬ηάϝϯ͞ΕΔɽτ (A) ͱ (B’) ʹରԠ͢Δɽ࣍ʹɼ೾‫ܗ‬ηάϝϯτ (B’) Λ೾‫ܗ‬—3—

Λ༻͍ΔɽຊߘͰ͸ɼϝϧέϓετϥϜ܎਺ͷ࣌‫͔ྻܥ‬Β‫ࢉܭ‬&RUUHFW%HJLQ

V,QVHUWLRQ
HUURU

U

W
X

D\

(QG

W

R

͞ΕΔର਺ύϫʔεϖΫτϧΛɼมௐεϖΫτϧͱ͢Δɽมௐ
εϖΫτϧͱ඼࣭ͷؔ܎Λௐࠪ͢Δͱͱ΋ʹɼ‫ॴہ‬తͳ඼࣭ྼ

R

ਤ 4 Իૉૠೖ‫ޡ‬ΓΛߟྀͨ͠ԻૉωοτϫʔΫͷྫɽൃ࿩୯‫ޠ‬͸“straight”ɽ
Fig. 4 An example of the phoneme network considering thephoneme insertion errors for a word fragment “straight.”ԽՕॴͷࣗಈ‫ݕ‬ग़ʹ͍ͭͯ΋ࢼΈΔɽ

3. ࣮‫ݧ‬తධՁ
3. 1 ࣮ ‫ ݧ‬৚ ݅ηάϝϯτ (A) ͱͯ͠ɼ͜ͷखॱΛԻ੠೾‫ܗ‬ͷऴ୺·Ͱ‫܁‬Γฦ೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀͷӳ‫ޠ‬Ի੠ʹର͢Δൃ࿩ϦζϜิਖ਼ॲཧʹ͢ɽຊߘͰ͸ɼγϑτ௕ d Λܾఆ͢Δࡍͷྨࣅ౓ͱͯ͠ɼ࣍ࣜ

͓͍ͯɼఏҊ๏ΛධՁ͢Δɽӳ‫ޠ‬฼‫ޠ‬࿩ऀͷԻ੠ʹ͸ɼCMU

Ͱ‫͞ࢉܭ‬ΕΔਖ਼‫ن‬Խ૬‫ޓ‬૬ؔ܎਺ r(d) Λ༻͍Δɽ

r(d) = Σt [(x(t) − μx ) · (y(t − d) − μy )]Σt [x(t) − μx ]2 · Σt [y(t − d) − μy ]2

ARCTIC Ի੠σʔλϕʔε [21] தͷஉੑ࿩ऀ 1 ໊ʹΑΔ 100จΛ༻͍Δɽ·ͨɼ೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀͷӳ‫ޠ‬Ի੠ͱͯ͠ɼཹֶ‫ܦ‬(5)‫ݧ‬ͷͳ͍೔ຊͷඪ४తͳӳ‫ڭޠ‬ҭΛड͚͖ͯͨஉࢠେֶӃੜ 2
໊ʹΑΔಉҰൃ࿩಺༰ͷ 100 จΛ༻͍Δɽ֤೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀʹ͜͜Ͱɼx(t)ɼy(t − d) ͸ɼྨࣅ౓‫ࢉܭ‬Λߦ͏֤೾‫ܗ‬ηάϝϯ͓͍ͯɼ͜ΕΒͷӳ‫ޠ‬฼‫ޠ‬࿩ऀԻ੠ 100 จ͓Αͼ೔ຊ‫ޠ‬฼‫ޠ‬࿩τͷৼ෯஋Λද͢ɽμx ͓Αͼ μy ͸ɼ֤೾‫ܗ‬ηάϝϯτʹ͓͚

ऀԻ੠ 100 จΛ༻͍ͯɼGMM Λֶश͢Δɽ͜ͷ಺ɼ֤೔ຊ‫ޠ‬Δৼ෯஋ͷฏ‫ۉ‬Λද͢ɽ·ͨɼγϑτ௕ͷ୳ࡧ෯͸ 10 msɼη฼‫ޠ‬࿩ऀʹରͯ͠ 25 จΛબ୒͠ɼ‫ ܭ‬50 จΛධՁσʔλͱͯ͠άϝϯτ௕͸ 2L =20 ms ͱ͠ɼ૭ֻ͚ʹ͸૭௕ 20 ms ͷࡾ֯࢖༻͢ΔɽԻ੠ͷαϯϓϦϯάप೾਺͸ 16 kHzɼԻ੠෼ੳ࣌ͷϑϨʔ૭ͷยଆ 10 ms Λ༻͍Δɽ

2. 3 HMM ֶशΛ༻͍ͨԻૉ‫ޡ‬Γͷ‫ݕ‬ग़Ϝγϑτ͸ 5 ms ͱ͢Δɽߴ඼࣭ͳԻ੠෼ੳ߹੒‫ͯ͠ͱܥ‬ɼ

DTW ʹΑΔ࣌ؒ৳ॖʹ‫ൃͮ͘ج‬࿩ϦζϜิਖ਼ॲཧͰ͸ɼରSTRAIGHT [22] Λ༻͍ΔɽεϖΫτϧύϥϝʔλͱͯ͠ɼ0৅ͱ͢Δ 2 ͭͷԻ੠ͷൃ࿩಺༰͕ಉҰͰ͋Δͱ͍͏ԾఆΛஔ͔࣍Β 24 ࣍ͷϝϧέϓετϥϜ܎਺Λ༻͍ɼԻ‫ݯ‬ύϥϝʔλ͘ɽͦͷͨΊɼ೔ຊਓӳ‫ޠ‬ͷ༷ʹɼԻૉૠೖ͕ੜ͡Δඇ฼‫ޠ‬Ի

ͱͯ͠ɼF0 ͓Αͼ 5 प೾਺ଳҬʹ͓͚Δฏ‫ۉ‬ඇप‫ظ‬੒෼ [23]੠Λର৅ͱ͢Δࡍʹ͸ɼԿΒ͔ͷରԠ͕ඞཁͱͳΔɽDTW ͷΛ࢖༻͢ΔɽGMM ͷֶश࣌ʹ͸ɼ୹࣌ؒ೾‫ܗ‬ύϫʔʹ‫͖ͮج‬ࡍʹ‫ڐ‬༰͢ΔεΩοϓϑϨʔϜ਺Λେ͖͘ઃఆ͢Δํ๏΋ߟ͑நग़͞Εͨ༗ԻϑϨʔϜͷΈΛର৅ͱ͠ɼ1 ͔࣍Β 24 ࣍ͷϝΒΕΔ͕ɼ೔ຊਓӳ‫ͯͬޡ͍͓ͯʹޠ‬ૠೖ͞ΕΔ฼Ի͸‫ܧ‬ଓ௕ϧέϓετϥϜ܎਺͓Αͼͦͷಈతಛ௃ྔΛ༻͍Δɽϝϧέϓ͕௕͍ͨΊɼे෼ͳޮՌ͕ಘΒΕΔͱ͸‫ݶ‬Βͣɼ·ͨɼଞͷՕετϥϜ܎਺ʹର͢Δ GMM ͷࠞ߹਺͸ 64 ͱ͠ɼඇप‫ظ‬੒෼ॴʹ͓͍ͯ΋ෆద੾ͳ࣌ؒ৳ॖΛ‫ڐ‬༰͢ΔՄೳੑ͕ߴ·Γɼѱʹର͢Δ GMM ͷࠞ߹਺͸ 16 ͱ͢Δɽֶश͞Εͨ GMM ʹ‫ج‬Ө‫͕ڹ‬ੜ͡Δ৔߹͕͋Δɽͦ͜ͰɼຊߘͰ͸ɼԻૉૠೖՕॴΛ͖ͮɼશϑϨʔϜʹର͢Δม‫ॲ׵‬ཧΛߦ͏ɽͦͷࡍʹɼ୹࣌ؒ‫ݕ‬ग़͠ɼલ΋ͬͯআ֎͢ΔॲཧΛࢪ͢ɽ೔ຊਓӳ‫ޠ‬ͷԻૉ‫ޡ‬Γ೾‫ܗ‬ύϫʔ͸อଘ͢Δɽ0 ͔࣍Β 24 ࣍·Ͱͷϝϧέϓετϥʢ஫ 1ʣͷ‫ن‬ଇੑ͸‫͔͘ݹ‬Β‫͞ڀݚ‬Ε͓ͯΓ [4]ɼਤ 4ͷ༷ʹɼൃ࿩

ςΩετͷԻૉྻ͔ΒԻૉૠೖ‫ޡ‬ΓΛ‫ؚ‬ΉԻૉωοτϫʔΫΛϜ܎਺͓Αͼಈతಛ௃ྔʹର͢Δ‫཭ڑ‬ई౓ʹ‫͖ͮج‬ɼDTW Λߦ͏͜ͱͰɼ‫ܧ‬ଓ௕ิਖ਼ॲཧͷͨΊͷ৳ॖ‫ن‬ଇΛܾఆ͢Δɽ‫ن‬ଇతʹߏஙՄೳͰ͋Δɽ͜ͷԻૉωοτϫʔΫΛ༻͍ͯɼඇ

Իૉૠೖ‫ݕ‬ग़ʹ༻͍ΔԻ‫ڹ‬Ϟσϧ͸ɼ5 ঢ়ଶ left-to-right ‫ܕ‬฼‫ޠ‬Ի੠ʹରͯ͠ HMM ʹΑΔঢ়ଶΞϥΠϝϯτΛߦ͏͜ͱͷ࿩ऀґଘϞϊϑΥϯ HMM ͱ͢Δɽ‫؍‬ଌಛ௃ྔ͸ඇ฼‫ޠ‬Ի੠Ͱɼ‫ͯͬޡ‬ૠೖ͞ΕͨԻૉΛࣗಈ‫ݕ‬ग़͢Δ [20]ɽ‫ݕ‬ग़͞ΕͨԻ

ͷ 0 ͔࣍Β 24 ࣍·ͰͷϝϧέϓετϥϜ܎਺ɼ‫ͼٴ‬ɼͦͷ 1ૉʹରͯ͠͸ɼ‫ܧ‬ଓ௕Λ 0 ʹ͢ΔΑ͏ʹ࣌ؒ৳ॖ‫ن‬ଇΛઃఆ͢

࣍ɼ2 ࣍ͷಈతಛ௃ྔͱ͢ΔɽHMM ͷग़ྗ֬཰ີ౓ؔ਺͸ 16

Δ͜ͱͰɼૠೖԻૉΛ࡟আ͢Δɽ

ࠞ߹ GMM ͱ͢Δɽൃ࿩ςΩετʹରͯ͠ɼFlite [24] ʹΑΔ

2. 4 ߟ

࡯ςΩετղੳΛߦ͍ɼԻૉ‫ྻܥ‬Λܾఆͨ͠‫ޙ‬ɼԻૉૠೖ‫ن‬ଇ [4]೔ຊਓӳ‫͍͓ͯʹޠ‬͸ɼԻૉૠೖͷଞʹ΋ɼԻૉஔ‫׵‬ɼԻૉʹ‫͖ͮج‬ɼԻૉωοτϫʔΫΛߏங͢ΔɽԻૉૠೖΛ‫͍ͳ·ؚ‬࡟আ͕ੜ͡΍͍͢ɽ͜ΕΒʹ͍ͭͯ͸ɼఏҊ๏ʹΑΔൃ࿩ϦζԻૉ‫ྻܥ‬Λ༻͍ͯɼϞϊϑΥϯ HMM Λֶशͨ͠‫ʹޙ‬ɼԻૉϜิਖ਼ͷΈͰ͸ରॲͰ͖ͳ͍ɽ·ͨɼૠೖԻૉͷ࡟আॲཧʹ͓ωοτϫʔΫʹ‫ ͮ͘ج‬Viterbi ΞϥΠϝϯτΛ࣮ࢪ͠ɼԻૉ‫ܥ‬͍ͯ΋ɼૠೖԻૉͷઌߦԻૉͱ‫ޙ‬ଓԻૉΛ௚઀઀ଓ͢ΔͨΊɼྻΛߋ৽͢Δɽ͜ΕΒΛෳ਺ճ࣮ࢪ͠ɼ࠷ऴతͳ HMM ΛಘΔɽௐԻ݁߹ͷӨ‫ྀ͞ߟ͕ڹ‬Ε͓ͯΒͣɼෆࣗવͳิਖ਼Ի੠͕ੜ੒

ఏҊ๏ͷ༗ޮੑΛௐࠪ͢ΔͨΊʹɼҎԼͷԻ੠Λ༻͍ͯԻ࣭

͞ΕΔ৔߹͕͋Δɽͳ͓ɼ‫ܧ‬ଓ௕Λେ෯ʹ୹͘͢ΔΑ͏ͳ࣌ؒ৳ॖΛ༻͍ͨࡍʹ΋ɼಉ༷ͷѱӨ‫͕ڹ‬ੜ͡΍͍͢ɽຊߘͰ͸ɼ඼࣭ྼԽΛ൐Θͳ͍ิਖ਼ॲཧͷ࣮‫͚ͯ޲ʹݱ‬ɼ࣌ؒ৳ॖʹ൐͏‫ॴہ‬తͳ඼࣭ྼԽΛ΋ͨΒ͢ཁҼʹؔ͢Δ෼ੳΛ͓Αͼ‫ܧ‬ଓ௕ͷࣗવੑʹؔ͢Δओ‫؍‬ධՁΛ࣮ࢪ͢Δɽ

Nativeɿ ӳ‫ޠ‬฼‫ޠ‬࿩ऀͷࣗવԻ੠
ERJɿ ೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀͷࣗવԻ੠Featɿ Ի੠෼ੳ࠶߹੒ʹΑΔิਖ਼Ի੠ʢैདྷ๏ʣ
Waveɿ ೾‫ܗ‬Ճ޻ʹΑΔิਖ਼Ի੠ʢఏҊ๏ʣߦ͏ɽ࣌ؒ৳ॖʹΑΓੜ͡ΔෆࣗવͳௐԻ݁߹Λଊ͑Δಛ௃ྔ

“Feat” ʹؔͯ͠͸ɼਪఆ͞Εͨ৳ॖ‫ن‬ଇʹ‫͖ͮج‬ɼඇ฼‫ޠ‬࿩ͱͯ͠ɼԻ੠εϖΫτϧύϥϝʔλ‫ྻܥ‬ͷมௐεϖΫτϧ [17]ऀͷεϖΫτϧύϥϝʔλ‫͓ྻܥ‬ΑͼԻ‫ݯ‬ύϥϝʔλ‫ྻܥ‬Λ৳ॖͤͨ͞‫ʹޙ‬ɼԻ੠߹੒ॲཧΛߦ͏ɽԻ࣭ධՁͱͯ͠ɼ5 ஈ

ʢ஫ 1ʣɿਤ 4 ͸ɼԻૉૠೖͷΈΛߟྀͨ͠ԻૉωοτϫʔΫͰ͋Δɽ֊ΦϐχΦϯείΞΛ༻͍ͨΦϐχΦϯධՁΛ࣮ࢪ͠ɼฏ‫ۉ‬Φ

—4—

,QGH[RIPHOFHSVWUDOFRHIILFLHQWV

95% confidence
interval

95% confidence
interval

%$'


















*22'







  

0RGXODWLRQIUHTXHQF\>+]@ਤ5Ի࣭ʹؔ͢Δओ‫؍‬ධՁ݁Ռਤ 6 ‫ܧ‬ଓ௕ͷࣗવੑʹؔ͢Δ

Fig. 5 5-point MOS scores on
Fig. 6speech quality.

ओ‫؍‬ධՁ݁Ռ
5-point DMOS scores on

duration naturalness.

ਤ 8 ิਖ਼લͱิਖ਼‫͚͓ʹޙ‬ΔϝϧέϓετϥϜ܎਺ͷมௐεϖΫτϧͷ෼෍ؒ‫( ཭ڑ‬KL μΠόʔδΣϯε)Fig. 8 KL divergence of Gaussian distribution of the modulation

spectrum of corrected mel-cepstral coeﬃcients from thatof original mel-cepstral coeﬃcients.

%HIRUH&RUUHFWLRQ
0HOFHSVWUDOFRHIILFLHQW
VHTXHQFH$IWHU&RUUHFWLRQ+DQQLQJ ZLQGRZ+DQQLQJ ZLQGRZ

ԻૉૠೖΛߟྀͨ͠ԻૉωοτϫʔΫΛ༻͍ͯ HMM Λֶश͠ɼૠೖԻૉͷ‫ݕ‬ग़Λߦͬͨ݁Ռɼ‫ݕ‬ग़͞ΕͨૠೖԻૉՕॴ͸

$

$¶

%

$

%

108 ՕॴͰ͋ͬͨɽ·ͨɼஉੑӳ‫ޠ‬฼‫ޠ‬࿩ऀ 1 ໊ͷಉҰൃ࿩ 50จʹରͯ͠ɼԻૉૠೖΛߟྀ͠ͳ͍ HMM ֶशΛߦͬͨɽ֤࿩

ਤ 7 มௐεϖΫτϧ෼ੳʹ༻͍ΔϝϧέϓετϥϜηάϝϯτͷྫऀʹର͢Δ HMM ঢ়ଶΞϥΠϝϯτͷ݁Ռ͔Β֤Իૉͷ‫ܧ‬ଓ

Fig. 7 An example of mel-cepstral coeﬃcient segments used for௕Λ‫ٻ‬Ίͨ‫ʹޙ‬ɼରԠ͢ΔԻૉʹ͓͚Δ‫ܧ‬ଓ௕ൺʹԠͨ͡ઢ‫ܗ‬

modulation spectrum analysis.

৳ॖͱͯ͠ɼ࣌ؒ৳ॖ‫ن‬ଇΛܾఆͨ͠ɽಘΒΕͨ࣌ؒ৳ॖ‫ن‬ଇϐχΦϯείΞʢMean Opinion Score: MOSʣΛ‫ٻ‬ΊΔɽҰʹ‫͖ͮج‬ɼWSOLA ʹΑΔൃ࿩ϦζϜิਖ਼ॲཧΛߦͬͨɽ͜͜

ํͰɼ‫ܧ‬ଓ௕ͷࣗવੑʹؔͯ͠͸ɼ“Native”ΛϦϑΝϨϯεͱͰɼ೔ຊਓӳ‫ޠ‬Ի੠ʹ͓͚ΔૠೖԻૉʹରͯ͠͸ɼ࣌ؒ৳ॖഒ

ͨ͠ 5 ஈ֊ DMOSʢDegradation MOSʣධՁΛߦ͏ɽͭ·Γɼ཰Λ 0 ͱ͢Δ͜ͱͰɼআ‫ॲڈ‬ཧΛߦͬͨɽ“Native” ʹର͢Δ֤Ի੠ͷൃ࿩ϦζϜͷྨࣅ౓ΛଌΔ͜ͱʹ࣍ʹɼิਖ਼Ի੠ʹ͓͍ͯɼ࣌ؒ৳ॖʹ൐͏‫ॴہ‬తͳ඼࣭ྼԽ

૬౰͢Δɽඃ‫ऀݧ‬͸ɼ೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀ 6 ໊Ͱ͋Δɽͨͩ͠ɼ͜

͕ੜ͍ͯ͡ΔՕॴͷநग़Λߦͬͨɽ·ͣɼૠೖԻૉ͕࡟আ͞Εͷ࣮‫Ͱݧ‬͸ HMM ΞϥΠϝϯτʹΑΔԻૉૠೖ‫ݕ‬ग़͸࣮ࢪ͠ͳͨ 108 Օॴʹରͯ͠ɼஉੑ೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀ 1 ໊ʹΑΔओ‫؍‬ධՁ͍ɽ·ͨɼ೾‫ܗ‬Ճ޻ʹΑΔൃ࿩ϦζϜิਖ਼ͷޮՌʹண໨͢Δͨ

Λߦ͍ɼ඼࣭ྼԽͷ༗ແΛ൑ఆͨ͠ɽ݁Ռɼ඼࣭ྼԽ͕ੜͯ͡

Ίɼ࣌ؒ৳ॖؔ਺͸ “Feat” ͓Αͼ “Wave” ͰಉҰͷ΋ͷΛ༻

͍Δͱ൑ఆ͞ΕͨՕॴ͸ɼ38 ՕॴͰ͋ͬͨɽ͞ΒʹɼૠೖԻૉ

͍Δɽ

࡟আՕॴҎ֎ʹରͯ͠΋ɼ‫ॴہ‬తͳ඼࣭ྼԽ͕ੜ͍ͯ͡ΔՕॴ

3. 2 Ի࣭ͱ‫ܧ‬ଓ௕ͷࣗવੑʹؔ͢Δ࣮‫݁ݧ‬Ռʹ͍ͭͯͷ൑ఆΛߦͬͨɽ݁Ռɼ50 จதͰ৽ͨʹ‫ݕ‬ग़͞ΕͨՕ

ਤ 5 ʹɼԻ࣭ʹؔ͢ΔධՁ݁ՌΛࣔ͠ɼਤ 6 ʹ‫ܧ‬ଓ௕ͷࣗॴ͸ 22 ՕॴͰ͋ͬͨɽ·ͨɼ‫ॴہ‬తͳ඼࣭ྼԽ͕ੜ͍ͯ͡ͳ

વੑʹؔ͢ΔධՁ݁ՌΛࣔ͢ɽਤ 5 ͔Βɼ“ERJ” ͱൺֱͯ͠

͍Օॴͱͯ͠ɼઌͷૠೖԻૉ͕࡟আ͞Εͨ 108 Օॴͷ಺ɼ্ड़“Feat” ͷԻ࣭͕‫ݦ‬ஶʹྼԽ͍ͯ͠Δ͜ͱ͕෼͔Δɽ͜Εʹରͷ 38 ՕॴΛআ͘ 70 Օॴͷத͔ΒɼϥϯμϜʹ 38 ՕॴΛநग़ͯ͠ɼԻ੠෼ੳ࠶߹੒ΛߦΘͳ͍ “Wave” Ͱ͸ɼԻ࣭ྼԽΛେͨ͠ɽૠೖԻૉͷ࡟আ͕ߦΘΕ͍ͯͳ͍Օॴ͔Β΋ɼ্ड़ͷ 22෯ʹ௿‫͖Ͱݮ‬Δ͜ͱ͕෼͔Δɽ͜ͷ͜ͱ͔ΒɼԻ੠෼ੳ࠶߹੒

ՕॴΛআ֎ͯ͠ɼϥϯμϜʹ඼࣭ྼԽ͕ੜ͍ͯ͡ͳ͍Օॴ 22

ʹΑΔԻ࣭ྼԽ͸େ͖͘ɼఏҊ๏ʹΑΔ඼࣭վળޮՌ͸େ͖͍ՕॴΛநग़ͨ͠ɽ

͜ͱ͕෼͔ΔɽҰํɼਤ 6 ͔Βɼ“Feat” ͓Αͼ “Wave” ͱ΋্‫ه‬ͷॲཧʹΑΓɼ‫ॴہ‬తͳ඼࣭ྼԽ͕ੜ͍ͯ͡ΔՕॴͱ͠ʹɼ“ERJ” ͱൺֱ͠ɼࣗવੑ͕‫ݦ‬ஶʹվળ͍ͯ͠Δ͜ͱ͔Βɼͯ‫ ܭ‬60 Օॴɼ‫ॴہ‬తͳ඼࣭ྼԽ͕ੜ͍ͯ͡ͳ͍Օॴͱͯ͠‫ܭ‬ൃ࿩ϦζϜิਖ਼ͷޮՌ͕ೝΊΒΕΔɽ·ͨɼ“Wave” ͸ “Feat”

60 ՕॴΛநग़ͨ͠ɽ

ͱಉ౳ͷࣗવੑվળޮՌ͕ಘΒΕ͍ͯΔɽҎ্ͷ͜ͱ͔Βɼఏ

4. 2 มௐεϖΫτϧʹΑΔൺֱ

Ҋ๏ʢ“Wave”ʣʹΑΔൃ࿩ϦζϜิਖ਼ʹΑΓɼߴ͍Ի࣭Λอ

4. 1 અͰநग़ͨ͠‫ ܭ‬120 Օॴʹରͯ͠ɼมௐεϖΫτϧʹ‫ج‬

࣋ͨ͠··ɼඇ฼‫ޠ‬࿩ऀԻ੠ͷࣗવੑΛେ͖͘վળͰ͖Δ͜ͱ

ͮ͘෼ੳΛߦ͏ɽ‫ܧ‬ଓ௕ิਖ਼લ‫ޙ‬ͷඇ฼‫ޠ‬Ի੠ʹରͯ͠ɼϝϧ͕෼͔Δɽ

4. ࣌ؒ৳ॖʹ൐͏‫ॴہ‬తͳ඼࣭ྼԽͷ෼ੳέϓετϥϜͷมௐεϖΫτϧΛ‫͢ࢉܭ‬Δɽਤ 7 ʹมௐεϖΫτϧͷ෼ੳ۠ؒΛࣔ͢ɽิਖ਼લͷԻ੠ʹରͯ͠͸ɼૠೖԻૉͷதԝ࣌ࠁʹରԠ͢Δ࣌ؒϑϨʔϜΛத৺ͱͯ͠૭ֻ͚Λߦ͍ɼ

4. 1 ෼ੳσʔληοτͷ࡞੒มௐεϖΫτϧΛ‫͢ࢉܭ‬Δɽิਖ਼‫ޙ‬ͷԻ੠ʹ͍ͭͯ͸ɼૠೖԻධՁʹ࢖༻͢ΔԻ੠ͱͯ͠ɼஉੑ೔ຊ‫ޠ‬฼‫ޠ‬࿩ऀ 1 ໊ʹΑΓɼૉ͕࡟আ͞Εͨ࣌ࠁʹରԠ͢Δ࣌ؒϑϨʔϜΛத৺ͱͯ͠૭ֻ

‫ޡ‬ΓԻૉΛҙਤతʹଟ͘‫ؚ‬ΉΑ͏ʹൃ࿩͞Εͨ CMU ARCTIC͚Λߦ͍ɼมௐεϖΫτϧΛ‫͢ࢉܭ‬Δɽͨͩ͠ɼิਖ਼‫ޙ‬ͷԻ੠Ի੠σʔλϕʔεதͷ 50 จΛ༻͍ͨɽαϯϓϦϯάप೾਺΍ͷϝϧέϓετϥϜ͸ɼWSOLA Λࢪͨ͠Ի੠೾‫ʹܗ‬ର͢ΔԻ੠ಛ௃ྔͳͲ͸ɼ3. 1 અͱಉ༷ͱͨ͠ɽ͜ͷ 50 จʹରͯ͠ɼSTRAIGHT ෼ੳʹΑΓநग़͢Δɽ૭ؔ਺͸ 128 ఺ͷϋχϯά

—5—૭Λ࢖༻͠ɼ཭ࢄϑʔϦΤม‫׵‬ͷλοϓ௕͸ 128 ఺Ͱ͋Δɽ‫ॴہ‬తͳ඼࣭ྼԽ͕ੜ͍ͯ͡Δ 60 Օॴ (“BAD”) ʹ͓͍ͯɼิਖ਼લ‫ޙ‬ͷԻ੠ͷมௐεϖΫτϧΛɼ֤ʑଟ࣍‫ݩ‬ਖ਼‫ن‬෼෍ͰϞσϧԽ͢Δɽ‫ڞ‬෼ࢄߦྻ͸ର֯੒෼ͷΈΛ༻͍Δɽิਖ਼લ (ࣗવԻ੠) ͷมௐεϖΫτϧʹର͢Δଟ࣍‫ݩ‬ਖ਼‫ن‬෼෍͔Βɼิਖ਼‫ܧ( ޙ‬ଓ௕ิਖ਼Ի੠) ͷมௐεϖΫτϧʹର͢Δଟ࣍‫ݩ‬ਖ਼‫ن‬෼෍΁ͷ KL μΠόʔδΣϯεΛ‫ٻ‬ΊΔɽಉ༷ʹɼ‫ॴہ‬తͳ඼࣭ྼԽ͕ੜ͍ͯ͡ͳ͍ 60 Օॴ (“GOOD”) ʹରͯ͠΋ɼKL μΠόʔδΣϯεΛ‫ٻ‬ΊΔɽਤ 8 ʹࣔͨ݁͠Ռ͔Βɼ“BAD” ͷํ͕ KL μΠόʔδΣϯε͕େ͖͘ͳΔ܏޲͕‫ݟ‬ΒΕɼಛʹ௿Ҭมௐप೾਺ଳҬʹ͓͍ͯɼ“BAD” ͱ “GOOD” ͷؒͰ KL μΠόʔδΣϯεͷҧ͍͕‫ݦ‬ஶͰ͋Δɽ

4. 3 มௐεϖΫτϧʹΑΔࣝผ
4. 1 અͰநग़ͨ͠‫ ܭ‬120 ՕॴΛ༻͍ͯɼ‫ॴہ‬తͳ඼࣭ྼԽ͕ੜ͍ͯ͡ΔՕॴͷࣝผ࣮‫ݧ‬Λߦ͏ɽࣝผ‫ͯ͠ͱث‬ɼαϙʔτϕΫλʔϚγϯΛ༻͍Δɽૉੑͱͯ͠ɼϝϧέϓετϥϜ‫ྻܥ‬ͷมௐεϖΫτϧɼF0 ‫ྻܥ‬ͷมௐεϖΫτϧɼඇप‫ظ‬੒෼‫ྻܥ‬ͷมௐεϖΫτϧΛ༻͍Δɽ‫ ܭ‬120 Օॴͷσʔληοτʹର͠
ͯɼ6 ෼ׂަࠩ‫ݕ‬ఆΛ 10 ճ࣮ࢪ͢Δɽަࠩ‫ݕ‬ఆͷ݁ՌಘΒΕͨฏ‫ࣝۉ‬ผ཰͸ 76.83 %Ͱ͋ΔɽνϟϯεϨϕϧ 50 %ͱൺ΂ͯ༗ҙʹߴ͍ࣝผ཰͕ಘΒΕ͓ͯΓɼมௐεϖΫτϧ͕‫ॴہ‬తͳ඼࣭ྼԽΛଊ͑Δಛ௃ྔͱͯ͠༗ޮͰ͋Δ͜ͱ͕෼͔Δɽ

5. · ͱ ΊຊใࠂͰ͸ɼඇ฼‫ޠ‬Ի੠ͱͯ͠೔ຊਓӳ‫ޠ‬Ի੠ʹண໨͠ɼ

DTW ͓Αͼ WSOLA ʹ‫ൃͮ͘ج‬࿩ϦζϜิਖ਼๏ΛఏҊͨ͠ɽ࣮‫ݧ‬తධՁ݁Ռ͔ΒɼఏҊ๏ʹΑΓɼԻ੠෼ੳ࠶߹੒ॲཧͷ࢖༻Λճආ͢Δ͜ͱͰɼԻ࣭ྼԽΛ࠷খ‫ʹݶ‬཈͑ͭͭॊೈͳ‫ܧ‬ଓ௕ม‫׵‬Λ࣮‫͢ݱ‬Δ͜ͱ͕Ͱ͖ɼࣗવੑΛେ෯ʹվળՄೳͳൃ࿩
ϦζϜิਖ਼ॲཧ͕࣮‫͖Ͱݱ‬Δ͜ͱ͕෼͔ͬͨɽ·ͨɼ࣌ؒ৳ॖ
ʹ൐͍ੜ͡Δ‫ॴہ‬తͳࣗવੑྼԽʹண໨͠ɼԻ੠εϖΫτϧύ
ϥϝʔλ‫ྻܥ‬ͷมௐεϖΫτϧʹ‫ͮ͘ج‬෼ੳΛߦͬͨɽͦͷ݁ՌɼมௐεϖΫτϧ͸ɼ‫ॴہ‬తͳࣗવੑྼԽΛଊ͑Δ͜ͱ͕෼͔ͬͨɽࠓ‫ޙ‬͸ɼ‫ॴہ‬తͳࣗવੑྼԽΛ؇࿨͢Δ‫ॲޙ‬ཧख๏ͷ‫ݕ‬౼΍ɼൃ࿩ϦζϜิਖ਼ॲཧͷվળʹऔΓ૊Ήɽँࣙຊ‫ڀݚ‬ͷҰ෦͸ɼJSPS Պ‫ݚ‬අ 26280060 ͷॿ੒Λड͚࣮ࢪͨ͠ɽ
จ‫ݙ‬[1] N. Minematsu, Y. Tomiyama, K. Yoshimoto, K. Shimizu,
S. Nakagawa, M. Dantsuji, and S. Makino, “Development ofEnglish speech database read by Japanese to support call research.,” Proc. ICA, vol.1, pp.557–560, Kyoto, Japan, Apr.
2004.
[2] P. Ladefoged, A Course in Phonetics, Third Edition, Harcourt Brace Jovanovich College Publishers, 1993.
[3] N. Minematsu, C. Guo, and K. Hirose, “CART-based factor
analysis of intelligibility reduction in Japanese English.,”Proc. INTERSPEECH, pp.2069–2072, Geneva, Switzerland, Sep. 2003.
[4] S. Kohmoto, Applied English phonology: teaching of English pronunciation to the native Japanese speaker, Tanaka
Press, Tokyo, Japan, 1965.
[5] R.L. Allington, Fluency: The Neglected Reading Goal,
vol.36, The Reading Teacher, 1983.
[6] H. Quené and L.E.V. Delft., “Non-native durational patterns decrease speech intelligibility,” Speech Communica-

tion, vol.52, no.11-12, pp.911–918, Mar. 2010.
[7] A.W. Black, “Speech synthesis for educational technology.,”Proc. SLaTE, pp.104–107, Farmington, U. S. A, Oct. 2007.
[8] δϟϯυϥϯϑϨσϦοΫɼኍ੉‫ܒ‬٢ɼ็দ৴໌ɼ“ΞΫηϯτ‫ܕ‬
CALL γεςϜͷͨΊͷԻ੠߹੒ʹΑΔ‫ڲ‬ਖ਼ϑΟʔυόοΫੜ੒ɼ” ిࢠ৘ใ௨৴ֶձٕज़‫ڀݚ‬ใࠂɼvol.102ɼno.618ɼpp.1–6ɼ
Jan. 2003ɽ
[9] Y. Oshima, S. Takamichi, T. Toda, G. Neubig, S. Sakti,and S. Nakamura, “Non-native speech synthesis preserving speaker individuality based on partial correction of
prosodic and phonetic characteristics,” Proc. INTERSPEECH, pp.299–303, Dresden, Germany, Sep. 2015.
[10] K. Tokuda, Y. Nankaku, T. Toda, H. Zen, J. Yamagishi, andK. Oura, “Speech synthesis based on hidden Markov models,” Proc. IEEE, vol.101, no.5, pp.1234–1252, Apr. 2013.
[11] J. Yamagishi and T. Kobayashi, “Average-voice-based
speech synthesis using HSMM-based speaker adaptation
and adaptive training.,” IEICE Trans. Inf. and Syst.,
vol.E90-D, no.2, pp.533–543, Feb. 2007.
[12] ߴಓ৻೭հɼ‫ాށ‬ஐ‫ج‬ɼG. NeubigɼS. Saktiɼதଜ ఩ɼ“HMMʹ‫ͮ͘ج‬೔ຊਓӳ‫ޠ‬Ի੠߹੒ʹ͓͚Δதֶੜెͷӳ‫ޠ‬Ի੠Λ༻͍” ೔ຊԻ‫ֶڹ‬ձ 2015 ೥ळ‫ߨق‬ԋ࿦จूɼpp.1231–1232ɼ
ͨධՁɼ
Sep. 2015ɽ
[13] S. Hiroya, “Non-negative temporal decomposition of speechparameters by multiplicative update rules.,” IEEE Trans.
ASLP, vol.21, no.10, pp.2108–2117, Oct. 2013.
[14] L. Juang and B.-H. Rabiner, Fundamentals of Speech
Recognition, Prentice Hall Inc., 1993.
[15] T. Toda, A.W. Black, and K. Tokuda, “Voice conversion
based on maximum-likelihood estimation of spectral parameter trajectory,” IEEE Trans. ASLP, vol.15, no.8, pp.2222–
2235, Nov. 2007.
[16] W. Verhelst and M. Roelands, “An overlap-add techniquebased on waveform similarity (WSOLA) for high quality
time-scale modiﬁcation of speech.,” Proc. ICASSP, vol.2,
pp.554–557, Minneapolis, U.S.A, Apr. 1993.
[17] S. Takamichi, T. Toda, A.W. Black, and N. Satoshi, “A
postﬁlter to modify the modulation spectrum in HMMbased speech synthesis.,” Proc. ICASSP, vol.2, pp.290–294,
Florence, Italy, May 2014.
[18] K. Cho and J.G. Harris, “Towards an automatic foreign accent reduction tool,” Proc. Speech Prosody, vol.2, pp.569–
572, Dresden, Germany, May 2006.
[19] K. Hirose, F. Gendrin, and N. Minematsu, “A pronunciation training system for Japanese lexical accents with corrective feedback in learner’s voice,” Proc. EUROSPEECH,
pp.3149–3152, Geneva, Switzerland, Sep. 2003.
[20] Y. Tsubota, T. Kawahara, and M. Dantsuji, “Recognition and veriﬁcation of English by Japanese students forcomputer-assisted language learning system.,” Proc. ICSLP,
pp.1205–1208, Denver, U. S. A, Sep. 2002.
[21] J. Kominek and A.W. Black, “CMU ARCTIC databases
for speech synthesis,” Technical report, Tech Report CMULTI-03-17, 2003.
[22] H. Kawahara, I. Masuda-Katsuse, and A.D. Cheveigné,“Restructuring speech representations using a pitchadaptive time-frequency smoothing and an instantaneousfrequency-based F0 extraction: Possible role of a repetitive structure in sounds,” Speech Commun., vol.27, no.3–4,
pp.187–207, Apr. 1999.
[23] Y. Ohtani, T. Toda, H. Saruwatari, and K. Shikano, “Maximum likelihood voice conversion based on GMM withSTRAIGHT mixed excitation,” Proc. INTERSPEECH,
pp.2266–2269, Pittsburgh, U. S. A, Sep. 2006.
[24] A.W. Black and K.A. Lenzo, “Flite: a small fast run-time
synthesis engine,” 4th ISCA Workshop on Speech Synthesis,
p.204, Perthshire, Scotland, Aug. 2001.

—6—