Integrating different acoustic and syntactic language models in a continuous speech recognition system

1 Continuous Speech Recognition (CSR) systems require acoustic models to represent the characteristics of the acoustic signal and Language Models (LM) to represent the syntactic constraints of the language. Both acoustic and LM probability distributions are usually independently obtained and evaluated. Then, the respective “best” models are selected to be integrated in the CSR systems. But, in this paper it was proved that the use of more accurate acoustic models (for example the use of semicontinuous models instead of discrete ones or the use of a bigger number of then representing a more complete set of sublexical units) didn’t always mean a better performance of the integrated system because the acoustic improvements were softened when the LM probabilities were applied. This experimental evaluation was carried out over a Spanish speech application task. 1 . Introduction The final goal of a Continuous Speech Recognition (CSR) system is to obtain a sequence of linguistic units W, given a sequence of acoustic observations A. Thus, acoustic probabilities P(A/W) and Language Model (LM) probabilities P(W) must be integrated using the wellknown Bayes s formula. Acoustic and LM probability distributions are usually independently obtained and evaluated. Then, the respective “best” models (the set of acoustic models which give a better performance in a phonetic-acoustic decoder and the LM with a lower perplexity value) are chosen to be integrated in the CSR system. In this work, the classical Hidden Markov Models (HMM) were evaluated. First, a basic set of context independent sub-lexical units was considered and a Discrete HMM was obtained for each unit. Then more accurate acoustic models were evaluated: a) semicontinuous instead of discrete ones 1 Work partially supported by the Spanish CICYT under grant TIC98-0423-C06-03 b) a bigger number of then representing a more complete set of context dependent sub-lexical units. The use of more accurate acoustic models usually gives a better performance when a phonetic-acoustic decodification is carried out but it doesn’t always mean a better performance of the integrated system when the LM probabilities are also applied. In fact, in this work we showed that discrete HMM can outperform semicontinuous HMM with significant faster decoding speeds, which is also reported in other recent works [1]. Acoustic models were first evaluated over an acoustic phonetic decoding task showing better performances when more accurate models were considered. Then, the three sets of acoustic HMM were integrated and evaluated in the CSR system [2] over a task oriented speech corpus representing a set of queries to a Spanish geography database (1208 words). The LM generation is usually based on statistical methods (N-grams). In this work, the k-Testable in the Strict Sense (k-TSS) LMs were used and evaluated with k=2.5. They are a sub-class of the regular grammars and a syntactic approach of the well-known N-grams [3]. It is well known that, to obtain the best performance of a CSR system is needed the modification of one of both P(W) or P(A/W) by introducing a balance parameter (α) [4]. This parameter seems to lessen the effects derived from the fact that, both P(A/W) and P(W) are independently obtained from speech and text training sets respectively. So that, several values of the balance parameter applied to the smoothed LM probabilities were also tested. In Section 2, the acoustic-phonetic decoding evaluation of the three proposed sets of acoustic models was presented. Section 3, deals with the experimental evaluation of the integrated CSR system taking into account the effect of scaling the LM probabilities. Finally, some concluding remarks are presented in Section 4. 2.Acoustic-Phonetic decoding. The basic sub-lexical unit set consisted of 24 phone-like units. A Discrete left to right HMM with 3 looped states and 4 discrete observations per state was obtained for each unit. Emission and transition probabilities were estimated using both the Baum-Welch and Viterbi procedures, applying the Maximum Likelihood criterion. But, to increase the accuracy of the acoustic models in the CSR system, two different ways were considered: a) The use of a semicontinuous HMM for each phone-like unit instead of the discrete one. The joint optimization of the codebook together with the parameters of the model is one of the main advantages of this kind of models [5]. b) To increase the number of sub-lexical units keeping the use of discrete HMM. The well known technique of decision tree clustering [6] was applied to obtain an optimal set of 101 trainable, discriminative and generalized context dependent units. An additional set of border units was specifically trained to generate lexical baseforms, covering all possible intraword context and being context independent to the outside [7]. Acoustic models were first evaluated over an acoustic phonetic decoding task with a balanced phonetic corpus both for training and testing purposes. The training corpus was composed of 1529 sentences, involving around 60000 phones. The -speaker independenttest corpus was composed of 700 sentences and around 33500 phones. In this experiments, the phone recognition rates (%REC) was measured as %REC=c/(i+s+d+c)*100, where c accounts for the number of correct recognitions, and i, s and d were the number of insertions, substitutions and deletions, respectively. Table 1 shows the obtained results when discrete and semicontinuous HMM were evaluated over the set of 24 phone-like units. Table 2 shows the obtained results when the 24 context independent and the 101 context dependent sets of units were evaluated using discrete HMM. Table 1. Phone recognition rates for a Spanish acoustic-phonetic decoding task, using discrete and semicontinuos HMM (24 phone-like units were represented) Kind of model % REC discrete 63.82 semicontinuous 65.07 As it was expected, the use of more accurate models gave higher phone recognition rates. In fact, the use of a bigger number of discrete HMM representing context dependent sub-lexical units seems to be even more efficient than the use of semicontinuous HMM over the 24 context independent units. Table 2. Phone recognition rates for a Spanish acoustic-phonetic decoding task, using phonelike units and context dependent units (discrete HMM were used). Type of unit # units % REC phone-like units 24 63.82 context dependent 101 66.44 In phonetic acoustic decodification , it was obvious that the use of more accurate models must be taking into account. In next section we could see if this behavior is kept when the evaluation is over the complete CSR system.