Formant tracking using segmental phonemic information

A new formant tracking algorithm using phoneme dependent nominal formant values is tested. The algorithm consists of three phases: (1) analysis, (2) segmentation, and (3) formant tracking. In the analysis phase, formant candidates are obtained by solving for the roots of the linear prediction polynomial. In the segmentation phase, the input text is converted into a sequence of phonemic symbols. Then the sequence is time aligned with the speech utterance. Finally, a set of formant candidates that are close to the nominal formant estimates while satisfying the continuity constraints are chosen. The new algorithm significantly reduces the formant tracking error rate (3.62%) over a formant tracking algorithm using only continuity constraints (13.04%). We will also discuss how to further reduce the tracking error rate. INTRODUCTION In the Bell Labs' Text-To-Speech (TTS) system [1], a limited number of acoustic units is stored in the inventory table. Therefore, it is important to be able to choose the best candidate for each synthesis unit (diphone, triphone, etc). Formants values can be used for selecting the best units as well as for testing unit compatibility to determine whether any two synthesis units are connectable in term of spectral discrepancy [1]. Thus, reliable formant tracking can be one of the crucial components in TTS system construction, where a huge amount of speech data has to be processed. Due to the size of the speech corpus, it would be prohibitive to rely on human intervention for formant tracking error correction. For decades, researchers have put efforts into improving the performance of speech formant tracking algorithms. Nevertheless, state-of-the-art formant tracking algorithms are not reliable enough for unsupervised, automatic usage. Even though the errors are obvious to the human eye when displayed in a longer time frame, a human might not do much better than the automatic formant trackers given only local information. This observation has led to methods that impose continuity constraints on the formant selection process [2],[3]. However, they still tend to generate errors by enforcing the continuity constraints too strongly or too weakly. Especially in highly transient phone boundaries such as consonantvowel transitions, continuityconstraints often cause tracking errors [4],[5],[6] . Fortunately, in the TTS system construction process, transcriptions of the speech utterances are available. During speech corpus recording, a speaker is asked to read a set of texts that are carefully selected. From the text, the phonemic transcription can be generated automatically. Then, the transcription can be time aligned with the acoustic speech signal using signal processing techniques. Using this forced time alignment, the exact time stamp for each phonemic event can be obtained. In this paper, we test a new algorithm for tracking speech formant trajectories using segmental phonemic information. Given a speech interval, it is assumed that the phonemic identity and nominal formant values for the phoneme are available. This assumption holds always in TTS applications. The implementation is based on previous work [7] in which only continuity constraints were used. We will show how much improvement can be achieved by using phonemic information for formant tracking. ALGORITHM The formant tracking algorithm consists of three phases: (1) analysis, (2) segmentation/alignment, and (3) formant track selection. In the analysis phase, formant candidates are obtained by LPC analysis on pre-emphasized speech. Formant candidates are obtained by solving for the roots of the linear prediction polynomial. In the segmentation phase, the input text is converted into a sequence of phonemic symbols, and the phonemic symbols are time aligned with the speech utterance. Finally, in the formant tracking phase, the best combination of formant frequencies is selected from the candidates based on minimum cost criteria. For each analysis frame, we choose a set of formant candidates that are closest to the nominal formant estimates while satisfying the continuity constraints. Speech Analysis Autocorrelation LPC analysis is performed on the preemphasized speech. An LPC order of 12 is used for speech data collected at a sampling rate of 11.025 kHz. Thus, ten complex poles (five conjugate pairs) will be used to model five formants and the extra two poles for the spectral tilt that might have not been compensated for by the pre-emphasis process. Pitch-asynchronous LPC coefficients are calculated every 5 ms. A Hamming window of 25ms is applied to each analysis frame. Formant frequency candidates are calculated by solving the prediction polynomial using Bairstow's method [8]. Only complex poles are considered as formant candidates. Text to Phonetic Transcript Given the input text, a sequence of graphemes is converted into a sequence of phonemic symbols. We have used the text analysis front-end of the Bell Labs TTS system [1]. The front-end includes components such as sentence-boundary detection, abbreviation expansion, number expansion, etc. Then, morphological analysis is performed for lemmatization of inflected words using a finite state machine. Finally, the words are converted into phoneme sequences using dictionary lookup and letter-tosound rules. A probabilistic system that is not part of the TTS system is used to generate alternative pronunciations for a given phoneme sequence produced by TTS's frontend. This is required because of possible mismatches between the TTS phoneme sequence and actual speech. Automatic Speech Segmentation The next step is to align the phoneme sequence with the acoustic signal. Reliable automatic alignment/segmentation is also very critical for TTS design, i.e., manual segmentation is too labor-intensive to perform for hours of recording. We have used an automatic segmentation algorithm that adopts filter bank approach combined with wavelet convolution [9]. Preliminary evaluations indicate accuracy levels that, for most types of boundaries, are close to those of human segmentors. We also observe that even if the segmentor makes segmentation errors, most of the errors do not critically affect the performance of the proposed formant tracking algorithm. Nominal (target) formant values [10] and voicing probability (1:voiced, 0:unvoiced and 0.3:mixed) are assigned to each temporal center of a phoneme segment. Formants and voicing probabilities for the frames between these center points are linearly interpolated. Formant Tracking The next step is to choose the best set of formant trajectories for N formants over K analysis frames. At each frame, k, there are Lk ways to map (assign) the candidate frequencies to formants. The Lk mappings can be identified as