Abstracts of Manuscripts in Review Automatic Generation of Synthesis Unit and Prosodic Information for Chinese Concatenative Synthesis

In this paper, some approaches to the generation of synthesis units and prosodic information are proposed for Chinese text-to-speech (TTS) conversion. The monosyllables are adopted as the basic synthesis units. A set of synthesis units is selected from a large continuous speech database based on two cost functions that minimize the inter- and intrasyllable distortion. The speech database is also employed to establish a work-prosody-based template tree according to the linguistic features: tone combination, word length, part-of-speech (POS) of the word, and word position in a sentence. This template tree stores the prosodic features including pitch contour, average energy, and syllable duration of a word for possible combinations of linguistic features. Two modules for sentence intonation and template selection are proposed to generate the target prosodic templates. The experimental results for the TTS conversion system showed that synthesized prosodic features quite resembled their original counterparts for most syllables in the inside test. Evaluation by subjective experiments also confirmed the satisfactory performance of these approaches.  We explore the possibility of recognizing speech signals using a large collection of coarse {\it acoustic events} , which describe temporal relations between a small number of local features of the spectrogram. The major issue of invariance to changes in duration of speech signal events is addressed by defining temporal relations in a rather coarse manner, allowing for a large degree of slack. The approach is greedy in that it does not offer an "explanation" of the entire signal as the hidden Markov models (HMM's) approach does; rather, it accesses small amounts of relational information to determine a speech unit or class. This implies that we recognize words as units, without recognizing their subcomponents. Multiple randomized decision trees are used to access the large pool of acoustic events in a systematic manner and are aggregated to produce the classifier.  In this paper, methods of improving the robustness and accuracy of acoustic modeling using decision tree based state tying are described. First, a new two-level segmental clustering approach is devised, which combines the decision tree based state tying with agglomerative clustering of rare acoustic phonetic events. Next, a unified maximum likelihood framework for incorporating both phonetic and nonphonetic Publisher Item Identifier S 1063-6676(99)08074-8. features in decision tree based state tying is presented. In contrast to other heuristic data separation methods, which often lead to training data de- pletion, a tagging scheme is used to attach various features of interest and the selection of these features in the decision tree is data driven. Finally, two methods of using multiple-mixture parameterization to improve the quality of the evaluation function in decision tree state tying are described. One method is based on the approach of -means fitting and the other method is based on a novel use of a local multilevel optimal subtree. Both methods provide a more accurate likelihood evaluation in decision tree clustering and are consistent with the structure of the decision tree. Experimental results on Wall Street Journal corpora demonstrate that the proposed approaches lead to a significant improvement in model quality and recognition performance.  A statistical coarticulatory model is presented for sponta- neous speech recognition, where knowledge about the dynamic, target- directed behavior in the vocal tract resonance is incorporated into the model design, training, and in likelihood computation. The principal advantage of the new model over the conventional hidden Markov model (HMM) is the use of a compact, internal structure that parsimoniously represents long-span context dependence in the observable domain of speech acoustics without using additional, context-dependent model pa- rameters. The new model is formulated mathematically as a constrained, nonstationary, and nonlinear dynamic system, for which a version of the generalized expectation maximization (EM) algorithm is developed and implemented for automatically learning the compact set of model parameters. A series of experiments for speech recognition and model synthesis using spontaneous speech data from the Switchboard corpus are reported. The promise of the new model is demonstrated by showing its consistently superior performance, over a state-of-the-art benchmark HMM system under controlled experimental conditions, when exposed to the reference transcription. Experiments on model synthesis and analysis shed insight into the mechanism underlying such superiority in terms of the target-directed behavior and of the long-span context-dependence property, both inherent in the designed structure of the new dynamic model of speech.