List of modifications made in HTS (for version 2.3)
HTS Working Group
December 25, 2015

1 Modifications in Model Definition
In HTS, the HTK HMM definition (please see HTKBook [1] Chapter 7) has been modified to support MSD [2],stream-level tying, and adaptation of multi-stream HMMs. This section gives its brief description.
First, <MSDInfo> is added to global options of the HTK HMM definition language The arguments to the<MSDInfo> option are the number of streams (default 1) and then for each stream, 0 (non-MSD stream) or 1
(MSD stream) of that stream. The full set of global options in HTS is given below.
globalOpts = option { option }option= <HmmSetId> string |<StreamInfo> short { short } |<MSDInfo> short { short } |<VecSize> short |<ProjSize> short |<InputXform> inputXform |<ParentXform> ∼a macro |covkind |durkind |parmkind

Second, the number of mixture specification is modified to support stream-level tying structure as follows:
HTK<State> 2<NumMixes> 1 2<SWeights> 2 0.9 1.1<Stream> 1<Mixture> 1 1.0<Mean> 4
0.3 0.2 0.1 0.0<Variance> 4
0.5 0.4 0.3 0.2<Stream> 2<Mixture> 1 0.4<Mean> 2
1.0 2.0<Variance> 2
4.0 8.0<Mixture> 2 0.6<Mean> 2
2.0 9.0<Variance> 2
3.0 6.0

HTS<State> 2<SWeights> 2 0.9 1.1<Stream> 1<NumMixes> 1<Mixture> 1 1.0<Mean> 4
0.3 0.2 0.1 0.0<Variance> 4
0.5 0.4 0.3 0.2<Stream> 2<NumMixes> 2<Mixture> 1 0.4<Mean> 2
1.0 2.0<Variance> 2
4.0 8.0<Mixture> 2 0.6<Mean> 2
2.0 9.0<Variance> 2
3.0 6.01

As you can see, <NumMixes> is moved from state-level to stream-level. This modification enables us to include
the number of mixture component in the stream-level macro. Based on this implementation, stream-level macro
was added. The various distinct points in the hierarchy of HMM parameters which can be tied in HTS is asfollows:
∼s
∼p
∼m
∼u
∼v
∼i
∼c
∼x
∼t
∼d
∼w

shared state distribution
shared stream
shared Gaussian mixture component
shared mean vector
shared diagonal variance vector
shared inverse full covariance matrix
shared Cholesky U matrix
shared arbitrary transform matrix
shared transition matrix
shared duration parameters
shared stream weight vector

Note that the ∼p macro is used by the HMM editor HHE D for building tied mixture systems in the original HTK
macro definition.
The resultant state definition of in the modified HTK HMM definition language is as follows:state
stateinfo= <State> short stateinfo= ∼s macro |[ weights ] stream { stream } [ duration ]macro= string
weights= ∼w macro | <SWeights> short vector
vector= float { float }stream= [ <Stream> short ] streaminfo
streaminfo = ∼p macro | [ <Stream> short ] [mixes] (mixture { mixture } | tmixpdf | discpdf)
mixes= <NumMixes> short {short}
tmixpdf= <TMix> macro weightList
weightList = repShort { repShort }repShort= short [ ∗ char ]discpdf= <DProb> weightList
mixture= [ <Mixture> short float ] mixpdf
mixpdf= ∼m macro | mean cov [ <GConst> float ]mean= ∼u macro | <Mean> short vectorcov= var | inv | xformvar= ∼v macro | <Variance> short vectorinv= ∼i macro |(<InvCovar> | <LLTCovar>) short tmatrix
xform= ∼x macro | <Xform> short short matrix
matrix= float {float}
tmatrix= matrix

It should be noted that <Stream> can doubly be specified in both stream and streaminfo. This is because<Stream> in ∼p macro is essential to specify stream index of this macro. This stream index information is usedin various HTS functions to check stream consistency.
Third, to support multi-stream HMM adaptation, the HTK HMM definition language for baseclasses is modified. A baseclass is defined as
baseClass =baseopts =StreamInfo =baseKind =classes=∼b macro baseopts classes<MMFIdMask> string <Parameters> baseKind [<StreamInfo>] <NumClasses> int
short { short } |MIXBASE | MEANBASE | COVBASE<Class> int itemlist { classes }where <StreamInfo> is optionally added to specify the stream structure.

22 Added Configuration Variables
A number of configuration variables have been added to HTK to control new functions implemented in HTS. Theirnames, default values, and brief descriptions are as follows:
ModuleHA DAPT

Name
SAVEFULLC

DefaultF

USEMAPLRUSEVBLR
USESTRUCTURALPRIORFFF

PRIORSCALE

1.0

SAVEALLNODEXFORMSTBANDWIDTH
DURUSEBIASFDURSPLITTHRESH

1000.0DURTRANSKIND
DURBLOCKSIZE

MLLRMEAN
full

DURBANDWIDTHDURBASECLASS

globalDURREGTREE
DURADAPTKIND

BASEMAXSTDDEVCOEF

10MINDUR5HM AP

APPLYVFLOOR

T

HG EN

MAXEMITER

20

EMEPSILON

1.0E-4HFB

3Description
Save transformed model set infull covariance form
Use MAP criterion [3]Use VB criterion [4]Perform structural approach(e.g., SMAPLR, CSMAPLR,
and SVBLR)
Prior parameter for MAPLR and
VBLR
Save all (unnecessary) linear transforms estimated in
SMAPLR/CSMAPLR/SVBLR
Bandwidth of transformation
matrices [5]Specify a bias with linear transforms
Minimum occupancy to generate a transform for state duration
model set
Transformation kind
Block structure of transform forstate duration model set
Bandwidth of transformation
matrices for state duration model
set
Macroname of baseclass forstate duration model set
Macroname of regression tree
for state duration model set
Use regression tree or baseclasses to adapt state duration
model set
Maximum duration to be evaluatedMinimum duration to be evaluated
Apply variance floor to modelset
Maximum number of EM iterations
Convergence factor for EM iteration

ModuleName
RNDPARMEAN

Default
0.0

RNDPARVAR

1.0

USEGV

F

USEGVPST
CDGV

F

LOGGV

F

MAXGVITERFGVEPSILON

1.0E-4MINEUCNORM

1.0E-2STEPINITSTEPDECSTEPINC
HMMWEIGHT
GVWEIGHT
GVINITWEIGHTOPTKINDRNDFLAGSGVMODELMMF
GVHMMLISTGVMODELDIRGVMODELEXTGVOFFMODEL

1.0
0.5
1.2
1.0
1.0
1.0
NEWTONUSEDAEMFDAEMITER20DAEMTEMPSCHEDULE

1.0

HM ODELIGNOREVALUE-1.0E+10

HC OMP V

NSHOWELEM

12VFLOORSCALEVFLOORSCALESTR

0.0

4

Description
Mean of Gaussian noise for random generation [6]Variance of Gaussian noise for
random generation
Use speech parameter generation
algorithm considering GV [7]
GV calculation flag for eachstreamUse context-dependent GV
model set
Use logarithmic GV instead oflinear GV
Max iterations in the speech parameter generation considering
GV
Convergence factor for GV iteration
Minimum Euclid norm of a gradient vector
Initial step size
Step size deceleration factor
Step size acceleration factor
Weight for HMM output prob
Weight for GV output prob
Initial weight of GV
Optimization method
Random generation flag
GV MMF file
GV model list
Dir containing GV models
Ext to be used with above Dir
Model names to be excluded
from GV calculation
Use the DAEM-based parameter
generation algorithm
Number of iterations for the
DAEM-based parameter generation algorithm
Temperature schedule parameterfor DAEM
Ignore value to indicate zerodimensional space in multispace probability distribution
Number of vector elements to beshowsvariance flooring scalevariance flooring scale vector for
streams

ModuleHER ESTName
APPLYVFLOOR

Default
T

DURMINVAR

0.0

DURVARFLOORPERCENTILE0APPLYDURVARFLOORTDURMAPTAU

0.0

ALIGNDURMMF
ALIGNDURLISTALIGNDURDIR

ALIGNDUREXT
ALIGNDURXFORMEXTALIGNDURXFORMDIR

DURINXFORMMASKDURPAXFORMMASKHHE DUSEPATTERNF

SINGLETREEF

APPLYMDLF

IGNORESTRWFREDUCEMEMFMINVAR

1.0E-6

5

Description
Apply variance floor to model
set
Minimum variance floor for state
duration model set
Maximum number of Gaussiancomponents (as the percentageof the total Gaussian components in the system) to undergo
variance floor for state duration
model set
Apply variance floor to state duration model set
MAP tau for state duration
model set [8]State duration MMF file foralignment (2-model reest)
State duration model list foralignment (2-model reest)
Dir containing state duration
models for alignment (2-model
reest)
Ext to be used with above Dir (2model reest)
Input transform ext for state duration model set to be used with
2-model reest
Input transform dir for state duration model set to be used with
2-model reest
Input transform mask for state
duration model set (default output transform mask)
Parent transform mask for state
duration model set (default output parent mask)
Use pattern instead of basephone for tree-based clustering
Construct single tree for eachstate position
Use the MDL criterion for treebased clustering [9]Ignore stream weight in treebased clustering
Use reduced memory implementation of tree-based clustering
Minimum variance floor formodel setModuleName
MDLFACTOR

Default
1.0

MINLEAFOCC

0.0

MINMIXOCC

0.0

SHRINKOCCTHRESH
HMG EN SSAVEBINARYF

OUTPDFPARMGENTYPE

F
0

MODELALIGNFSTATEALIGNFUSEALIGNF

USEHMMFBFINXFORMMASKPAXFORMMASKPDFSTRSIZEPDFSTRORDERPDFSTREXTWINEXT
WINDIRHSMMA LIGNWINFNINXFORMMASKPAXFORMMASKDURINXFORMMASKDURPAXFORMMASK

6

Description
Factor to control the model complexity term in the MDL criterion
Minimum occupancy count ineach leaf node
Minimum occupancy count ineach mixture component
Minimum occupancy count indecision trees shrinking
Save generated parameters in binary
Output pdf sequences
Type of parameter generation algorithm [10]Use model-level alignments
given from label files to determine model-level durations
Use state-level alignments given
from label files to determine
state-level durations
Use model-level alignments toprune EM-based parameter generation algorithm
Do not use state duration models
in the EM-based parameter generation algorithm
Input transform mask
Parent transform mask
Number of PdfStreams
Size of static feature in each PdfStream
Ext to be used for generated parameters from each PdfStream
Ext to be used for window coefficients file
Dir containing window coefficient files
Name of window coefficient files
Input transform mask
Parent transform mask
Input transform mask for state
duration model set (default output transform mask)
Parent transform mask for state
duration model set (default output parent mask)

ModuleHM GE T OOLNameSAVEBINARYPDFSTRSIZEPDFSTRORDER

DefaultF

MGETRNFLAGGVTRNFLAGACCERRFLAG
INVQUASIZE
VARWINSIZEGVDISTWGHTGAINWTFLAG
WINEXT
WINDIR
WINFN

Description
Save estimated models in binary
Number of PdfStreams
Size of static feature in each PdfStream
Whether perform MGE train forthe stream
Whether incorporate GV component into MGE training
Accumulate generation error
Bandwith of quasi-diagonal inversion matrix
Window for local variance calculation
GV weights
gain weight for generation error
Ext to be used for window coefficients file
Dir containing window coefficient files
Name of window coefficient files

Other configuration variables in HTK can also be used with HTS. Please refer to HTKBook [1] Chapter 18 forothers.

7

3 Added Command-Line Options
Various new command-line options have also been added to HTK tools. They are listed as follows:

HI NIT
Option-g

Ignore outlier vector in MSD

Defaultonoutput duration model to file sStore new hmm def in fn (name only)

Defaultnone
outDir/srcfn

HR EST
Option-g s-o fn

HER EST
OptionDefault-b
use an input linear transform for dur models
off-f s
extension for new duration model files
as src-g s
output duration model to file s
none
-k f
set temperature parameter for DAEM training
1.0-n s
dir to find duration model definitions
current-q s
save all xforms for duration to TMF file s
TMF-u tmvwapd update t)rans m)eans v)ars w)ghts
tmvwa)daptation xform p)rior useds)semi-tied xformd) switch to duration model update flag
-y s
extension for duration model filesnone-N mmf load duration macro file mmf-R dir dir to write duration macro files
current-W s [s] set dir for duration parent xform to soff
and optional extension-Y s [s] set dir for duration input xform to snone
and optional extension-Z s [s] set dir for duration output xform to s
none

HHE D
Option
-a f
-i
-m
-p
-q n

Default
factor to control the second term in the MDL
1.0
ignore stream weightoff
apply MDL principle for clusteringoff
use pattern instead of base phone
off
use reference tree for clusteringoff
0: clustering is stopped by threshold
1: clustering is stopped when leaf don’t have occ
2: clustering is stopped by threshold
After that, standard clustering is not performed
3: clustering is stopped when leaf don’t have occ

8

-r n

-s
-v f

After that, standard clustering is not performed
reduce memory usage on clustering
00: no memory reduction
1: mid reduction but fast
2: large reduction but slow
construct single tree
off
Set minimum variance to f
1.0E-6HMG EN S
Option-a-b-c n

Use an input linear transform for HMMs
Use an input linear transform for dur models
type of parameter generation algorithm
0: both mix and state sequences are given
1: state sequence is given,
but mix sequence is hidden
2: both state and mix sequences are hidden-d s
dir to find hmm definitions-euse model alignment from label for pruning-f fframe shift in 100 ns-g f
Mixture pruning threshold-h s [s] set speaker name pattern to s,
optionally set parent patterns-m
use model alignment for duration-n s
dir to find duration model definitions-poutput pdf sequences-r f
speaking rate factor (f<1: fast f>1: slow)-s
use state alignment for duration-t f [i l] set pruning to f [inc limit]-v fthreshold for switching spaces for MSD-x s
extension for hmm files-y sextension for duration model files-A
Print command line arguments-B
Save HMMs/transforms as binary-C cf
Set config file to cf-D
Display configuration variables-E s [s] set dir for parent xform to s
and optional extension-G fmt Set source label format to fmt-H mmf Load HMM macro file mmf-I mlf Load master label file mlf-J s [s] set dir for input xform to s
and optional extension-L dir Set input label (or net) dir-M dir Dir to write HMM macro files-N mmf Load duration macro file mmf-S f
Set script file to f
-T N
Set trace flags to N
-V
Print version information-W s [s] set dir for duration parent xform to s
and optional extension

9

Default
off
off
0

current
off
50000
10.0
*.%%%
off
current
off
1.0
off
inf
0.5
none
none
off
off
default
off
off
as config

nonecurrent
current
none
0
off
off

-X ext Set input label (or net) file ext-Y s [s] set dir for duration input xform to s
and optional extension

lab
none

HSMMA LIGN
Option-a
Use an input linear transform for HMMs-b
Use an input linear transform for dur models-cPrune by time information of label-d s
Dir to find hmm definitions-f
Output full state alignment-h s [s] Set speaker name pattern to s,
optionally set parent patterns-n s
Dir to find duration model definitions-m dir Set output label dir-r ext Output label file extension-s sprint statistics to file s-t iSet pruning threshold-w f
Duration weight-x s
Extension for hmm files-y sExtension for duration model files-A
Print command line arguments-C cf
Set config file to cf-D
Display configuration variables-E s [s] set dir for parent xform to s
and optional extension-F fmt Set source data format to fmt-G fmt Set source label format to fmt-H mmf Load HMM macro file mmf-I mlf Load master label file mlf-J s [s] set dir for input xform to s
and optional extension-L dir Set input label (or net) dir-N mmf Load duration macro file mmf-S f
Set script file to f
-T N
Set trace flags to N
-V
Print version information-W s [s] set dir for duration parent xform to s
and optional extension-X ext Set input label (or net) file ext-Y s [s] set dir for duration input xform to s
and optional extension

Default
off
off
off
current
off*.%%%current
current
lab
off
off
1.0
none
none
off
default
off
off
as config
as config

nonecurrent
none
0
off
off
lab
none

HM GE TOOL
Option-a i j-b i j-c

i: max times to shift the boundary
j: max length for each boundary shiftingi: end iteration for boundary adjustment
j: window size for boundary adjustment
output the process data
10

Default
none
1
none
5
off

-d dir HMM definition directory-eenable limit the updating rate for each step-f r
frame rate-genable multiply variance ratio for mean updating-i i j start/end iteration index of MGE training-j flg 0: eval 1: train 2: adapt-l dir output label directory-o ext HMM def file extension-p a b parameter for step size: 1/(a + b*n)-r file load HMM for reference-s file updating scale file-u mvwa update t)rans m)eans v)ars w)ghts
a)daptation xform-v fthreshold for switching spaces for MSD-w f
distance weight for gv component-x ext label file extension-A
Print command line arguments-B
Save HMMs/transforms as binary-C cf
Set config file to cf-D
Display configuration variables-G fmt Set source label format to fmt-H mmf Load HMM macro file mmf-I mlf Load master label file mlf-J s [s] set dir for input xform to s
and optional extension-K s [s] set dir for output xform to s
and optional extension-L dir Set input label (or net) dir-M dir Dir to write HMM macro files-S f
Set script file to f
-T N
Set trace flags to N
-V
Print version information-X ext Set input label (or net) file ext
Please also refer to HTKBook [1] Chapter 17 for other command-line options.

11

none
off
50000
off
0 0
1
none
none
1000.0 1.0
none
none
none
0.5
1.0
lab
off
off
default
off
as config

none
none
current
current
none
0
off
lab

4 Added Commands and Modifications in HHE D
Some HHE D commands have been added in HTS. They are as follows:AX filename- Set the Adapt XForm to filename
AX filename state_mapping_table mmf list output_filename- Set the Adapt XForm to another model and save one
CM directory- Convert models to pdf for speech synthesizer
CT directory- Convert trees/questions for speech synthesizer
DM type macroname- Delete macro from model-set
DR id- Convert decision trees to a regression tree
DV
- Convert full covariance to diagonal variances
IT filename- Clustering while imposing loaded tree structure
If any empty leaf nodes exist, loaded trees
are pruned
and then saved to filename
IX filename- Set the Input Xform to filename
JM hmmFile itemlist - Join Models on stream or state level
PX filename- Set the Parent Xform to filename
SM smtable mmf hlist - Output KLD-based state mapping table// comment- Comment line (ignored)
In many HHE D commands, we are required to specify item lists to specify a set of items to be processed. In
HTS, item list specification has been modified to specify stream-level items.
itemList=
itemSet=hmmName=identList =ident=
metachar =
state
=
index
=
intRange =stateComp =stream=mix=“{” itemSet { “,” itemSet } “}”hmmName . [“transP” | “state” state ]ident | identList“(” ident { “,” ident } “)”< char | metachar >“?” | “⋆”index [“.” stateComp ]“[” intRange { “,” intRange } “]”
integer [ “-” integer ]“dur” | “weights” | stream[ “ stream” index ] [ “.mix” mix ]index [ “.” ( “mean” | “cov” ) ]For example,
TI str1 {*.state[2].stream[1]}denotes tying streams in state 2 of all phonemes.

12Appendix A

History of HTS• Version 1.0 (December 2002)– Based on HTK-3.2.
– HHE D supports tree-based clustering based on the MDL criterion [9].
– HHE D supports stream-dependent tree-based clustering [11].
– HM ODEL supports multi-space probability distributions (MSD) [2].
– HER EST can generate state duration modeling [12].
– Speech parameter generation algorithm [10] is implemented in HG EN and HMG EN S.
– Demo using the CMU Communicator database.
• Version 1.1 (May 2003)– Based on HTK-3.2.
– Small run-time synthesis engine (hts engine).
– Demo using the CSTR TIMIT database.
– HTS voices for the Festival speech synthesis system [13].
• Version 1.1.1 (December 2003)– Based on HTK-3.2.1.
– HC OMP V supports variance flooring for MSD-HMMs.
– Demo using the CMU ARCTIC database [14].
– Demo using the Nitech Japanese database.
– Demo supports post-filtering [15].
– HTS voice for the Galatea toolkit [16].
• Version 2.0 (December 2006) [17]– Based on HTK-3.4.
– Support generating state duration PDFs in HR EST.
– Phoneme boundaries can be given to HER EST using the -e option [18].
– Reduced-memory implementation of tree-based clustering in HHE D with the -r option.
– Each decision tree can have a name with regular expressions in HHE D with the -p option.
– Flexible model structures in HMG EN S.
– Speech parameter generation algorithm based on the EM algorithm [10] in HMGenS.
– Random generation algorithm [6] in HMG EN S [6].
– State or phoneme-level alignments can be given to HMG EN S.
– The interface of HMG EN S has been switched to HER EST-style.
– Various kinds of linear transformations for MSD-HMMs are supported in HA DAPT.
∗ Constrained MLLR based adaptation [19].
∗ Adaptive training based on constrained MLLR [19].
∗ Precision matrix modeling based on semi-tied covariance matrices [20].
∗ Heteroscedastic linear discriminant analysis (HLDA) based feature transform [21].
∗ Phonetic decision trees can be used to define regression classes for adaptation [22]∗ Adapted HMMs can be converted to the run-time synthesis engine format.
– Maximum a posteriori (MAP) adaptation [8] for MSD-HMMs in HM AP.
• Version 2.0.1 (May 2007)– Based on HTK-3.4.
– HA DAPT supports band structure for linear transforms [5].
– HC OMP V supports stream-dependent variance flooring scales.
– Demo support LSP-type spectral parameters.
– β version of the runtime synthesis engine API (hts engine API).
– hts engine API supports speaker interpolation [23].
• Version 2.1 (June 2008)– Based on HTK-3.4.
– Released under the Modified BSD license [24].

13

– Simple documentation.
– 64-bit compile support.
– MAXSTRLEN (maximum length of strings), MAXFNAMELEN (maximum length of filenames),
PAT LEN (maximum length of pattern strings), and SMAX (maximum number of streams) definedin HShell.h can be set through configure script.
– HFB supports the forward/backward algorithm for hidden semi-Markov models (HSMMs) [25, 26].
– HA DAPT suports SMAPLR/CSMAPLR adaptation [27, 3].
– HG EN supports speech parameter generation algorithm considering global variance (GV) [7].
– HG EN supports random generation of transitions, durations, and mixture components.
– HER EST supports HSMM training and adaptation.
– HMG EN S supports speech parameter generation from HSMMs.
– Add DM command to HHE D to delete an existing macro from MMF.
– Add IT command to HHE D to impose pre-constructed trees in clustering.
– Add JM command to HHE D to join models on state or stream level.
– HHE D MU command supports ’*2’ style mixing up.
– HHE D MU command supports mixture-level occcupancy threshhold in mixing up.
– First stable version of the runtime synthesis engine API (hts engine API).
• Version 2.1.1 (May 2010)– Based on HTK-3.4.1– WFST converter for forced-alignment of HSMM is implemented in HF ST [28].
– Demo support context-dependent GV without silent and pause phoneme.
– Add initial GV weight for parameter generation in HMG EN S.
– Add memory reduction options for context-clustering in HHE D.
– Add model-level alignments given from label of singing voice to determine note-level durations.
– Demo using the Nitech Japanese database for singing voice synthesis [29].
– The API of runtime synthesis engine, hts engine API, is splitted from HTS itself and moved to SourceForge.
• Version 2.2 (July 2011)– Support DAEM algorithm in parameter estimation step [30].
– Support KLD-based state-mapping and cross-lingual speaker adaptation in HHE D [31].
– Add stand-alone HSMM-based forced-alignment command, HSMMA LIGN, instead of HF ST [28].
– Add HM GE T OOL for MGE training [32].
– Context-clustering can be started in the middle of the tree building.
– Change sampling frequency of demo from 16kHz to 48kHz.
– Support bark critical-band based aperiodic measure.
– Change speakers of Brazilian Portuguese and Japanese song demo.
– Release slides as a tutorial of HMM-based speech synthesis.
• Version 2.3 (December 2015)– Add VBLR adaptation in HER EST [4].
– Add DAEM-based parameter generation algorithm in HMG EN S.
– Add parallel mode to HC OMP V, HI NIT, and HR EST.
– Support DP search to determine state duration when the model alignments are given in HMG EN S.
– Speed up context-clustering by calculating differences between answers to current and previous questions.
– Add LSP postfilter in demo [33].
– Support mel-cepstrum based aperiodic measure in STRAIGHT demo.
– Support new HTS voice format for hts engine API.
– Turn off spectrum normalization in STRAIGHT demo.
– Add modulation spectrum-based postfilter in demo [34].
– Add untying weights function in HHEd.
– Support text files instead of utt files for general English database.
– Integrate normal demo and STRAIGHT demo.

14

References[1] S. Young, G. Evermann, M. Gales, T. Hain, D. Kershaw, X.-Y. Liu, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. Woodland. The Hidden Markov Model Toolkit (HTK) version 3.4, 2006. http://htk.eng.cam.ac.uk/.
[2] K. Tokuda, T. Masuko, N. Miyazaki, and T. Kobayashi. Multi-space probability distribution HMM. IEICE Trans. Inf. & Syst., E85D(3):455–464, Mar. 2002.
[3] Y. Nakano, M. Tachibana, J. Yamagishi, and T. Kobayashi. Constrained structural maximum a posteriori linear regression for averagevoice-based speech synthesis. In Proc. Interspeech, pages 2286–2289, 2006.
[4] K. Yu and M. Gales. Bayesian adaptation inference and adaptive training. IEEE Trans. Audio, Speech and Language Process., 15:1932–
1943, 2007.
[5] L. Qin, Y.-J. Wu, Z.-H. Ling, and R.-H. Wang. Improving the performance of HMM-based voice conversion using context clustering
decision tree and appropriate regression matrix. In Proc. of Interspeech (ICSLP), pages 2250–2253, 2006.
[6] K. Tokuda, H. Zen, and T. Kitamura. Reformulating the HMM as a trajectory model. In Proc. Beyond HMM – Workshop on statisticalmodeling approach for speech recognition, 2004.
[7] T. Toda and K. Tokuda. A speech parameter generation algorithm considering global variance for HMM-based speech synthesis. IEICE
Trans. Inf. & Syst., E90-D(5):816–824, 2007.
[8] J.L. Gauvain and C.-H. Lee. Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains. IEEE
Trans. on Speech & Audio Process., 2(2):291–298, 1994.
[9] K. Shinoda and T. Watanabe. MDL-based context-dependent subword modeling for speech recognition. J. Acoust. Soc. Jpn.(E), 21(2):79–
86, 2000.
[10] K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura. Speech parameter generation algorithms for HMM-based speechsynthesis. In Proc. ICASSP, pages 1315–1318, 2000.
[11] T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura. Simultaneous modeling of spectrum, pitch and duration in
HMM-based speech synthesis. In Proc. Eurospeech, pages 2347–2350, 1999.
[12] T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura. Duration modeling for HMM-based speech synthesis. In Proc.
ICSLP, pages 29–32, 1998.
[13] A.W. Black, P. Taylor, and R. Caley. The festival speech synthesis system. http://www.festvox.org/festival/.
[14] J. Kominek and A.W. Black. CMU ARCTIC databases for speech synthesis. Technical Report CMU-LTI-03-177, Carnegie Mellon
University, 2003.
[15] T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura. Incorporation of mixed excitation model and postfilter into
HMM-based text-to-speech synthesis. IEICE Trans. Inf. & Syst. (Japanese Edition), J87-D-II(8):1563–1571, Aug. 2004.
[16] Galatea – An open-source toolkit for anthropomorphic spoken dialogue agent. http://hil.t.u-tokyo.ac.jp/galatea/.
[17] H. Zen, T. Nose, J. Yamagishi, S. Sako, T. Masuko, A. W. Black, and K. Tokuda. The HMM-based speech synthesis system version 2.0.
In Proc. ISCA SSW6, pages 294–299, 2007.
[18] D. Huggins-Daines and A. Rudnicky. A constrained Baum-Welch algorithm for improved phoneme segmentation and efficient training.
In Proc. of Interspeech, pages 1205–1208, 2006.
[19] M.J.F. Gales. Maximum likelihood linear transformations for HMM-based speech recognition. Computer Speech & Language, 12(2):75–
98, 1998.
[20] M.J.F. Gales. Semi-tied covariance matrices for hidden Markov models. IEEE Transactions on Speech and Audio Processing, 7(3):272–
281, 1999.
[21] M.J.F. Gales. Maximum likelihood multiple projection schemes for hidden Markov models. IEEE Trans. Speech & Audio Process.,
10(2):37–47, 2002.
[22] J. Yamagishi, M. Tachibana, T. Masuko, and T. Kobayashi. Speaking style adaptation using context clustering decision tree for HMMbased speech synthesis. In Proc. ICASSP, pages 5–8, 2004.
[23] T. Yoshimura, T. Masuko, K. Tokuda, T. Kobayashi, and T. Kitamura. Speaker interpolation for HMM-based speech synthesis system.
J. Acoust. Soc. Jpn. (E), 21(4):199–206, 2000.
[24] http://www.opensource.org/licenses/category.
[25] H. Zen, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura. A hidden semi-Markov model-based speech synthesis system. IEICE
Trans. Inf. & Syst., E90-D(5):825–834, 2007.
[26] J. Yamagishi and T. Kobayashi. Average-voice-based speech synthesis using HSMM-based speaker adaptation and adaptive training.
IEICE Trans. Inf. & Syst., E90-D(2):533–543, 2007.
[27] O. Shiohan, Y. Myrvoll, and C.-H. Lee. Structural maximum a posteriori linear regression for fast HMM adaptation. Computer Speech& Language, 16(3):5–24, 2002.
[28] K. Oura, H. Zen, Y. Nankaku, A. Lee, and K. Tokuda. A fully consistent hidden semi-Markov model-based speech recognition system.
IEICE Trans. Inf. & Syst., E91-D(11):2693–2700, 2008.
[29] K. Oura, A. Mase, T. Yamada, S. Muto, Y. Nankaku, and K. Tokuda. Recent development of the HMM-based singing voice synthesis
system — Sinsy. In Proc. ISCA SSW7, pages 211–216, 2010.
[30] Y. Itaya, H. Zen, Y. Nankaku, C. Miyajima, K. Tokuda, and T. Kitamura. Deterministic annealing EM algorithm in acoustic modelingfor speaker and speech recognition. IEICE Trans. Inf. & Syst., E88-D(3):425–431, 2005.
[31] Y.-J. Wu and K. Tokuda. State mapping based method method for cross-lingual speaker adaptation in HMM-based speech synthesis. In
Proc. of Interspeech (ICSLP), pages 420–423, 2009.
[32] Y.-J. Wu and R.-H. Wang. Minimum generation error training for HMM-based speech synthesis. In Proc. ICASSP, pages 89–92, 2006.
[33] Z.-H. Ling, Y.-J. Wu, Y.-P. Wang, L. Qin, and R.-H. Wang. USTC system for Blizzard Challenge 2006 an improved HMM-based speechsynthesis method. In Blizzard Challenge Workshop, 2006.

15

[34] S. Takamichi, T. Toda, A.W. Black, and S. Nakamura. In Proc. of GlobalSIP, pages 710–714, 2014.

16