A Study on Articulatory Feature-based Phoneme Recognition and Voice Conversion

In this thesis, the behavior of articulatory feature (AF) as linguistic feature representation of the speech waveform in the task of both phoneme recognition (PR) and voice conversion (VC) is studied. Over the past few years, several studies have been conducted on the design of an optimal hidden Markov model (HMM) configuration for automatic speech recognition (ASR). Most of these studies are based on spectral-representation feature vectors. On the other hand, phonetic features, such as articulatory features (AF), have proved their robustness across speakers, against co-articulatory effects, and against noise. Despite these advantages, the literature on the design of an optimal parameter set for AF-based HMM speech recognition is still limited. Subsequent to our previous works of an AF extractor, the first part of this thesis will describe further our experimental studies on the design of an optimal AF-HMM-based classifier. In the beginning of the thesis, while we also intend to improve the phoneme recognizer performance, the main goal is rather to observe the behavior of AF as the speech representation for PR task. Several strategies for designing the optimal parameter set in AF-HMM-based PR are investigated. These strategies will consider extending sub-word unit from monophone to triphone, adding number of HMM states, conducting vowel group separation, tuning insertion penalty (IP), and applying Bakis HMM topology. Mel-frequency cepstral coefficient (MFCC)HMM-based PR experiments were also conducted for comparison purpose. Both of the PR systems experienced accuracy degradation during the extension from monophone-based PR to triphone-based PR. A large number of insertion errors were occurred, mostly during the recognition of fricative and vowel sound. Adding number of HMM states and conducting vowel group separation reduce the insertion errors on both of the AF-HMM and MFCC-HMM-based PR. The analysis showed different behavior between AF-HMM-based PR and MFCC-HMM-based PR in terms of their reaction to IP value. IP was imposed to reduce the insertion error, by balancing insertion error and deletion error. Compared to MFCC-HMMbased PR, AF needs larger insertion penalty value to be imposed. Morever, we found that compared with the linear topology, the Bakis topology worked well for improving both the correct rate and the accuracy of the AF-HMM and MFCC-HMM-based PR. AF-based PR with 5-state HMMs, separated vowel, triphone subword, Bakis topology, and optimal insertion penalty provides the highest accuracy among the experiments, i.e., 81.38% for the JNAS speech database. Furthermore in this thesis, the behavior of AF is also used to realize AF-based VC system. We focus our goal of this section to implement AF-based VC for a small number of target-speaker training data. VC transforms the voice from the source-speaker onto the target-speaker. When a source-speaker utters a certain sentence, the converted speech will sound as if a target-speaker is speaking the same sentence. The trend of VC has moved from text-dependent VC, in which it needs parallel utterances between source and target-speakers, into text-independent VC. However, this newer system still needs source speaker utterances as the training data. The flexibility of AF as speaker independent representation, as showed in PR task, can be used to extend the capability of an AF-based VC application. AF can be used in speaker adaptation technique to develop a VC application which maps features from arbitrary speakers into those of the expected target speakers. During the training process, our approach does not require sourcespeaker data to build the VC model. We propose VC based on AF to vocal-tract parameters (VTP) mapping. An artificial neural network (ANN) is applied to map AF to VTP and to convert a speaker’s voice to a targetspeaker’s voice. In order to investigate the effect of ANN architecture and different VTP orders on the performance of AF-ANN-based VC, six ANN architectures correspond to different VTP orders were compared. The architecture that provided the best result compared with other architectures was chosen for the remaining experiments. In addition to the feature vector mapping process, two types of F0 conversions were also conducted. The first F0 conversion was done using time stretching subsequent to sample rate transposing technique. Moreover, the second F0 conversion was done using F0 extraction and re-synthesis technique using MLSA filter. For comparison, a baseline VC system based on Gaussian mixture model (GMM) approach was conducted. Two types of evaluations were performed, i.e., objective evaluations and subjective evaluations. For objective evaluation, spectrum distortion (SD) is calculated to measure the distance between target-speaker spectrum and converted spectrum. Furthermore, for subjective evaluations, three listening tests were performed, i.e. the similarity test, XAB test, and mean opinion score (MOS) test. For the overall performance, AF-ANN-based VC outperforms MCEP-GMM-based VC for a small number of target-speaker training data. The proposed VC application was also realized for arbitrary source-speakers.